Commit Graph

7991 Commits

Author SHA1 Message Date
Yann Collet
610c41f90f improved 2-args macro to support both syntaxes
both `--long-commmand=arg` and `--long-command arg`
are supported by macro `NEXT_FIELD()`
so that the long-command only needs to be listed once.

This extends the syntax to support new syntaxes like
`-o=FILE` or `-D=dict`,
though there is no need to advertise this capability for the time being.

Also : added `NEXT_UINT32()`,
which is wrapper around `NEXT_FIELD()`
to read integer parameters.
Use the wrapper to new field, such as `--memlimit`
which can now support both syntaxes too.
2020-07-20 17:41:32 -07:00
Yann Collet
23941eec04 added tests for newly enabled syntax
for --patch-from origin
and --filelist list

Also : removed some constrained syntax tests,
as the new argument parsing syntax is more permissive.

For example :
    zstd file -of dest
used to be disallowed.

It's now allowed, and understood as:
    zstd file -o dest -f
2020-07-17 13:31:15 -07:00
Yann Collet
7280c1dafa extended 2-fields macro usage to number arguments
leading to further simplications
2020-07-17 13:09:23 -07:00
Yann Collet
a7e5652672 CLI: simplified handling of separated-fields arguments
this patch reduces complexity associated with
commands requiring a separated arguments
such as :
-o filename
-D dictionary
--output-dir-flat dir
--output-dir-mirror dir

It used to be a multi-stage process with explicit context,
it's now simplified as a single macro.

Thanks to this simplification,
separated arguments logic has also been extended to
--patch-from XXX
--filelist XXX
which extends existing capability using =XXX
--patch-from=XXX
--filelist=XXX

Separated argument is useful for filenames and directories,
as it benefits from shell expansion
such as ~/dir/file
where the ~ is automatically translated by the shell.
In contrast --long-command=FILE does not interpret FILE,
so ~/ is transmitted as is to the main() function,
generally resulting in incorrect file name.
2020-07-17 12:46:36 -07:00
Yann Collet
40f20b6453
Merge pull request #2197 from neheb/hj
meson: remove build requirement for distutils
2020-07-17 09:16:11 -07:00
Yann Collet
62df5bca4c
Merge pull request #2243 from facebook/uclibc
fix uclibc's st_mtim
2020-07-17 09:14:20 -07:00
Yann Collet
c224367ede ensure workspace is large enough
even when MAX_TABLELOG is reduced
2020-07-16 20:33:50 -07:00
Yann Collet
21c273da84 import some minor fixes from FSE project 2020-07-16 20:25:15 -07:00
Felix Handte
a35f135617
Merge pull request #2246 from ffontaine/dev
programs/util.c: fix build without st_mtime
2020-07-16 12:40:43 -04:00
Yann Collet
0d0fc4544c
Merge pull request #2247 from facebook/revert-2244-dev
Revert "Fix -Wunused-variable under FUZZING_BUILD_MODE..."
2020-07-15 12:42:31 -07:00
Yann Collet
a44671b281
Revert "Fix -Wunused-variable under FUZZING_BUILD_MODE..." 2020-07-15 12:42:18 -07:00
Yann Collet
7d3d5f2a20
Merge pull request #2244 from hctim/dev
Fix -Wunused-variable under FUZZING_BUILD_MODE...
2020-07-15 12:40:17 -07:00
Fabrice Fontaine
26d01bdb26 programs/util.c: fix build without st_mtime
Since version 1.4.5 and commit
5af8cb7aea, if st_mtime is not defined,
programs/util.c uses utime without including utime.h which will raise
the following build failure on some of the buildroot autobuilders:

util.c: In function 'UTIL_setFileStat':
util.c:161:24: error: storage size of 'timebuf' isn't known
         struct utimbuf timebuf;
                        ^~~~~~~

Fixes:
 - http://autobuild.buildroot.org/results/be902c5d110f37bce622a2215191f155b7d3e7e0

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
2020-07-15 21:24:13 +02:00
Mitch Phillips
23b55d6b3e Fix -Wunused-variable under FUZZING_BUILD_MODE...
Fuzzing build modes (FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) doesn't
necessarily imply that assert() is enabled, according to the manual.

When the current do-nothing is expanded under -Wunused-variable (-Wall),
it results in unused variables in some of the FUZZING_BUILD_MODE...
blocks.

This patch extends the do-nothing to avoid the unused variable.
2020-07-14 09:03:02 -07:00
Yann Collet
1dcc478796 fix uclibc's st_mtim
reported and suggested by @ewildgoose (#1872)
2020-07-13 14:16:33 -07:00
Nick Terrell
bd21e4b264
Merge pull request #2242 from evverx/cifuzz-ubsan-msan
cifuzz: turn on UBsan and MSan
2020-07-13 14:26:13 -04:00
Evgeny Vereshchagin
8e20f133e7 cifuzz: turn on UBsan and MSan 2020-07-13 01:20:11 +00:00
Yann Collet
16b353b207 minor doc clarification regarding MT parameters 2020-07-11 02:16:52 -07:00
Nick Terrell
bdf716bc64
Merge pull request #2239 from evverx/evverx-cifuzz
cifuzz: don't upload artifacts when CIFuzz fails to build fuzz targets
2020-07-10 11:26:15 -04:00
Yann Collet
a3296dae38
Merge pull request #2219 from xxie24/output-dir-mirror
Add output-dir-mirror option
2020-07-10 01:21:47 -07:00
Yann Collet
865ecf67ae
Merge pull request #2183 from tonytheodore/patch-1
Install pkg-config file on MINGW also
2020-07-10 01:20:59 -07:00
Yann Collet
78ebe4b5de
Merge pull request #2237 from marxin/gcda-to-gitignore
Add GCC PGO data files to .gitignore.
2020-07-09 08:48:36 -07:00
Yann Collet
c0ee26ec15
Merge pull request #2236 from marxin/pgo-fix-comment
Update comment about PGO build.
2020-07-09 08:48:04 -07:00
Yann Collet
2fbc438a5d
Merge pull request #2231 from marxin/use-threads-in-example
Use threads in streaming_compression.c example.
2020-07-09 08:41:44 -07:00
Martin Liska
7276ab15bc
Add GCC PGO data files to .gitignore. 2020-07-09 09:32:54 +02:00
Martin Liska
5b9d6b992a
Update comment about PGO build. 2020-07-09 09:31:51 +02:00
Martin Liska
461348c09a
Use threads in streaming_compression.c example.
Fixes #1584.
2020-07-09 09:20:06 +02:00
Evgeny Vereshchagin
3b9df5896f cifuzz: don't upload artifacts when CIFuzz fails to build fuzz targets
It should address https://github.com/google/oss-fuzz/issues/3982
2020-07-09 06:02:48 +00:00
Yann Collet
978cea5d56
Merge pull request #2230 from cwoffenden/minor-tidy
Remove superfluous build-time macros (single-file libs)
2020-07-08 11:49:40 -07:00
Yann Collet
fedecb29d8
Merge pull request #2235 from bimbashrestha/decom-macros
[actions] removing flakey min-decomp
2020-07-08 11:49:04 -07:00
Bimba Shrestha
2598af451f removing min decomp 2020-07-08 11:11:45 -07:00
Carl Woffenden
9bfd371d9f Remove superfluous build-time options 2020-07-08 11:39:17 +02:00
Yann Collet
2cdd33ae16
Merge pull request #2227 from yoshihitoh/single-file-dict-emscripten
[contrib] Fix single-file compilation error on Emscripten build.
2020-07-07 08:51:20 -07:00
yoshihitoh
bc5fe33243 Change Emscripten docker image to the official one which is used for testing the single-file feature. 2020-07-07 19:10:12 +09:00
Yann Collet
f014c3cd40
Merge pull request #2228 from marxin/rename-ZSTDMT_NBTHREADS_MAX
Rename ZSTDMT_NBTHREADS_MAX to ZSTDMT_NBWORKERS_MAX.
2020-07-06 16:06:49 -07:00
Martin Liska
4307e870a2
Rename ZSTDMT_NBTHREADS_MAX to ZSTDMT_NBWORKERS_MAX.
Rename it in zstd man page.
2020-07-07 00:18:20 +02:00
yoshihitoh
a035654ab8 Enable docker based Emscripten build on single file library test if emcc is not available. 2020-07-05 22:25:20 +09:00
Nick Terrell
a2b6a8fbc6
Merge pull request #2220 from dkcasset/fix-lib-Makefile-older-sed-version
Replace -E option with equivalent -r for older versions of sed
2020-06-29 17:19:24 -07:00
Nick Terrell
3690fe30f3
Merge pull request #2221 from BioDataAnalysis/bda_install_dll
Install MSVC runtime shared libraries (DLL) with cmake
2020-06-29 17:14:50 -07:00
dkcasset
82e7e2b47e Add variable for sed extended RE option (defaults to -E) 2020-06-29 13:44:23 -07:00
Mario Emmenlauer
37f36b9670 build/cmake/lib/CMakeLists.txt: Install shared library (dll) for Windows 2020-06-29 14:15:12 +02:00
yoshihitoh
c6548eac8e Rename static vars to avoid redefinition error. 2020-06-29 10:51:50 +09:00
dkcasset
b0ed66ef92 Replace -E option with equivalent -r for older versions of sed 2020-06-26 10:43:28 -07:00
Xin Xie
9a8ccd4ba3 Add output-dir-mirror option 2020-06-24 22:12:11 -07:00
Nick Terrell
7afd5d85d3
Merge pull request #2218 from terrelln/assert-seq
Fix unused variable warnings in fuzzing build mode without asserts
2020-06-22 17:41:18 -07:00
Nick Terrell
081691a3aa
Merge pull request #2217 from terrelln/cover-redundant
[cover] Remove unnecessary mask and dedup hash functions
2020-06-22 17:41:13 -07:00
Nick Terrell
370933fa20
Merge pull request #2209 from Niadb/dev
Explicitly use __cdecl for qsort, to avoid warning when default calling convention is not __cdecl
2020-06-22 17:41:03 -07:00
Nick Terrell
cce0edfdbe Fix unused variable warnings in fuzzing build mode without asserts
Fix unused vairable warnings when `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is defined but asserts are disabled.

Fixes #2210.
2020-06-22 12:56:57 -07:00
Nick Terrell
2312b819af [cover] Remove unnecessary mask and dedup hash functions
* Remove the unnecessary mask, since `ZSTD_hash*()` already ensures
  the output is mod 2^h.
* Dedup the hash functions and use `ZSTD_hash*()` directly.
2020-06-22 12:52:13 -07:00
Nick Terrell
78601d0806
Merge pull request #2212 from cwoffenden/single-file-dict
Single-file libs now include dictBuilder
2020-06-22 12:46:54 -07:00