Instead of calling `stat()`, these functions accept the result of a previous
`stat()` call on the file in question, which will allow us to make multiple
decisions around a file without redundant `stat()` calls.
I want to introduce versions of many of these functions that take pre-
populated `stat_t` objects and use those rather than doing their own redundant
`stat()` internally. These functions will have `...Stat()` suffixes. So this
commit renames these existing functions into the active voice, to avoid
confusion.
both `--long-commmand=arg` and `--long-command arg`
are supported by macro `NEXT_FIELD()`
so that the long-command only needs to be listed once.
This extends the syntax to support new syntaxes like
`-o=FILE` or `-D=dict`,
though there is no need to advertise this capability for the time being.
Also : added `NEXT_UINT32()`,
which is wrapper around `NEXT_FIELD()`
to read integer parameters.
Use the wrapper to new field, such as `--memlimit`
which can now support both syntaxes too.
for --patch-from origin
and --filelist list
Also : removed some constrained syntax tests,
as the new argument parsing syntax is more permissive.
For example :
zstd file -of dest
used to be disallowed.
It's now allowed, and understood as:
zstd file -o dest -f
this patch reduces complexity associated with
commands requiring a separated arguments
such as :
-o filename
-D dictionary
--output-dir-flat dir
--output-dir-mirror dir
It used to be a multi-stage process with explicit context,
it's now simplified as a single macro.
Thanks to this simplification,
separated arguments logic has also been extended to
--patch-from XXX
--filelist XXX
which extends existing capability using =XXX
--patch-from=XXX
--filelist=XXX
Separated argument is useful for filenames and directories,
as it benefits from shell expansion
such as ~/dir/file
where the ~ is automatically translated by the shell.
In contrast --long-command=FILE does not interpret FILE,
so ~/ is transmitted as is to the main() function,
generally resulting in incorrect file name.
Since version 1.4.5 and commit
5af8cb7aea, if st_mtime is not defined,
programs/util.c uses utime without including utime.h which will raise
the following build failure on some of the buildroot autobuilders:
util.c: In function 'UTIL_setFileStat':
util.c:161:24: error: storage size of 'timebuf' isn't known
struct utimbuf timebuf;
^~~~~~~
Fixes:
- http://autobuild.buildroot.org/results/be902c5d110f37bce622a2215191f155b7d3e7e0
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
This adds support for just showing the version string
without the full welcome message if the log level is
less than the default. That means that you pass -q once
and you will just see "1.4.5".
This makes it easier to parse in scripts.
* adding long support for patch-from
* adding refPrefix to dictionary_decompress
* adding refPrefix to dictionary_loader
* conversion nit
* triggering log mode on chainLog < fileLog and removing old threshold
* adding refPrefix to dictionary_round_trip
* adding docs
* adding enableldm + forceWindow test for dict
* separate patch-from logic into FIO_adjustParamsForPatchFromMode
* moving memLimit adjustment to outside ifdefs (need for decomp)
* removing refPrefix gate on dictionary_round_trip
* rebase on top of dev refPrefix change
* making sure refPrefx + ldm is < 1% of srcSize
* combining notes for patch-from
* moving memlimit logic inside fileio.c
* adding display for optimal parser and long mode trigger
* conversion nit
* fuzzer found heap-overflow fix
* another conversion nit
* moving FIO_adjustMemLimitForPatchFromMode outside ifndef
* making params immutable
* moving memLimit update before createDictBuffer call
* making maxSrcSize unsigned long long
* making dictSize and maxSrcSize params unsigned long long
* error on files larger than 4gb
* extend refPrefix test to include round trip
* conversion to size_t
* making sure ldm is at least 10x better
* removing break
* including zstd_compress_internal and removing redundant macros
* exposing ZSTD_cycleLog()
* using cycleLog instead of chainLog
* add some more docs about user optimizations
* formatting
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized
The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.
The copyright and license of `divsufsort.{h,c}` is not changed.
When building zst under cygwin or msys2 with std=c99 the build would fail because
of an undefined fileno()/_fileno(), which is used by the IS_CONSOLE() macro.
When building with -std=c99 (gcc otherwise defaults to gnu, which implies POSIX),
which is the default of the cmake build, then including unistd.h wont define
_POSIX_VERSION and all other headers also wont expose POSIX API.
To fix this make sure to define _POSIX_C_SOURCE with the version we want before including
unistd.h and so that _POSIX_VERSION is set to the version provided by the system.
Since Cygwin/MSYS2 just follow POSIX we can also remove their special cases for
defining IS_CONSOLE().
And, for completeness, also explicitly include stdio.h which is what actually declares fileno().
Tested with the normal make file and cmake under MSYS2 and Cygwin.
* Adding new cli endpoint --diff-from=
* Appveyor conversion nit
* Using bool set trick instead of direct set
* Removing --diff-from and only leaving --diff-from=#
* Throwing error when both dictFileName vars are set
* Clean up syntax
* Renaming diff-from to patch-from
* Revering comma separated syntax clean up
* Updating playtests with patch-from
* Uncommenting accidentally commented
* Updating remaining docs and var names to be patch-from instead of diff-from
* Constifying
* Using existing log2 function and removing newly created one
* Argument order (moving prefs to end)
* Using comma separated syntax
* Moving to outside #ifndef
Cygwin currently uses the fallback implementation which just returns 1 every time,
which leads to bad performance when zstd is called with -T0 for example.
Instead use the POSIX implementation used for the BSDs which works just fine under Cygwin.
Tested under Cygwin and MSYS2.
/dev/null permissions were modified when using sudo rights.
This fixes this bug during decompression.
More importantly, this patch adds a test, triggered in TravisCI,
ensuring unaltered /dev/null permissions.
