Commit Graph

405 Commits

Author SHA1 Message Date
Yann Collet
a8c66881e5
Merge pull request #2283 from senhuang42/progress_bars_for_multiple_files
Refreshing progress bar for processing multiple files
2020-08-26 11:54:50 -07:00
senhuang42
ead2387a7e Remove extraneous FIO_setNbFiles() 2020-08-26 12:05:04 -04:00
senhuang42
b6abbc3ce5 Fix single file decompression summary, remove escape code dependency, add currFileIdx field to FIO_prefs, general cleanups/bugfixes 2020-08-26 11:35:07 -04:00
senhuang42
da38891a87 Add initial live update with displayLevel = 2, add new field to FIO_prefs_t 2020-08-25 16:46:47 -04:00
senhuang42
e1e41addb3 Adjust #ifndef logic 2020-08-24 17:36:36 -04:00
senhuang42
44c54a3e31 Addressing comments: more comments, cleanup, remove extra function, checksum logic 2020-08-24 16:14:19 -04:00
senhuang42
ffaa0df76d Document change in CLI for --no-check during decompression in --help menu 2020-08-24 09:49:12 -04:00
Yann Collet
15c5e20023 updated documentation 2020-07-23 12:10:57 -07:00
Yann Collet
610c41f90f improved 2-args macro to support both syntaxes
both `--long-commmand=arg` and `--long-command arg`
are supported by macro `NEXT_FIELD()`
so that the long-command only needs to be listed once.

This extends the syntax to support new syntaxes like
`-o=FILE` or `-D=dict`,
though there is no need to advertise this capability for the time being.

Also : added `NEXT_UINT32()`,
which is wrapper around `NEXT_FIELD()`
to read integer parameters.
Use the wrapper to new field, such as `--memlimit`
which can now support both syntaxes too.
2020-07-20 17:41:32 -07:00
Yann Collet
23941eec04 added tests for newly enabled syntax
for --patch-from origin
and --filelist list

Also : removed some constrained syntax tests,
as the new argument parsing syntax is more permissive.

For example :
    zstd file -of dest
used to be disallowed.

It's now allowed, and understood as:
    zstd file -o dest -f
2020-07-17 13:31:15 -07:00
Yann Collet
7280c1dafa extended 2-fields macro usage to number arguments
leading to further simplications
2020-07-17 13:09:23 -07:00
Yann Collet
a7e5652672 CLI: simplified handling of separated-fields arguments
this patch reduces complexity associated with
commands requiring a separated arguments
such as :
-o filename
-D dictionary
--output-dir-flat dir
--output-dir-mirror dir

It used to be a multi-stage process with explicit context,
it's now simplified as a single macro.

Thanks to this simplification,
separated arguments logic has also been extended to
--patch-from XXX
--filelist XXX
which extends existing capability using =XXX
--patch-from=XXX
--filelist=XXX

Separated argument is useful for filenames and directories,
as it benefits from shell expansion
such as ~/dir/file
where the ~ is automatically translated by the shell.
In contrast --long-command=FILE does not interpret FILE,
so ~/ is transmitted as is to the main() function,
generally resulting in incorrect file name.
2020-07-17 12:46:36 -07:00
Xin Xie
9a8ccd4ba3 Add output-dir-mirror option 2020-06-24 22:12:11 -07:00
Tobias Hieta
b1cce9a6ad Short version string output
This adds support for just showing the version string
without the full welcome message if the log level is
less than the default. That means that you pass -q once
and you will just see "1.4.5".

This makes it easier to parse in scripts.
2020-06-09 08:40:41 +02:00
Yann Collet
f13eb2ec01
Merge pull request #2123 from facebook/zstd_help
updated `zstd -h`
2020-05-11 18:54:07 -07:00
Yann Collet
37b2dafbed minor clarification on -o file 2020-05-08 14:20:47 -07:00
Yann Collet
e5366bee16 updated zstd -h
to help readability.
Group arguments into categories,
try to give better visibility to more common arguments.
2020-05-08 14:14:46 -07:00
W. Felix Handte
376c26bf56 Use Unused Variable 2020-05-08 16:43:39 -04:00
W. Felix Handte
7dcca6bc64 Also Move programs/ Directory to Relative Includes 2020-05-04 15:20:26 -04:00
Bimba Shrestha
5b0a452cac
Adding --long support for --patch-from (#1959)
* adding long support for patch-from

* adding refPrefix to dictionary_decompress

* adding refPrefix to dictionary_loader

* conversion nit

* triggering log mode on chainLog < fileLog and removing old threshold

* adding refPrefix to dictionary_round_trip

* adding docs

* adding enableldm + forceWindow test for dict

* separate patch-from logic into FIO_adjustParamsForPatchFromMode

* moving memLimit adjustment to outside ifdefs (need for decomp)

* removing refPrefix gate on dictionary_round_trip

* rebase on top of dev refPrefix change

* making sure refPrefx + ldm is < 1% of srcSize

* combining notes for patch-from

* moving memlimit logic inside fileio.c

* adding display for optimal parser and long mode trigger

* conversion nit

* fuzzer found heap-overflow fix

* another conversion nit

* moving FIO_adjustMemLimitForPatchFromMode outside ifndef

* making params immutable

* moving memLimit update before createDictBuffer call

* making maxSrcSize unsigned long long

* making dictSize and maxSrcSize params unsigned long long

* error on files larger than 4gb

* extend refPrefix test to include round trip

* conversion to size_t

* making sure ldm is at least 10x better

* removing break

* including zstd_compress_internal and removing redundant macros

* exposing ZSTD_cycleLog()

* using cycleLog instead of chainLog

* add some more docs about user optimizations

* formatting
2020-04-17 15:58:53 -05:00
Nick Terrell
ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
Bimba Shrestha
43874730f4 Adding --content-size 2020-03-09 14:19:05 -05:00
Bimba Shrestha
6d8e761ba3 Inverting content size prefs bool 2020-03-09 14:16:10 -05:00
Bimba Shrestha
03ee18d9c5 Adding (void)noContentSize 2020-03-09 14:00:52 -05:00
Bimba Shrestha
167244a64f --no-content-size cli 2020-03-09 13:07:29 -05:00
Bimba Shrestha
23165b387e Adding hashLog to --show-default-cparams 2020-02-27 13:56:56 -08:00
Lehman Garrison
834d06b655 Allow some cli options to accept numeric values up to size_t instead of unsigned int 2020-02-20 14:29:58 -05:00
Philip Jones
4e728e26ca
Fix integer parsing in cli (#2003) 2020-02-18 15:30:59 -08:00
Bimba Shrestha
f627e1a1ae
Displaying level in --show-default-cparams (#1991)
* Displaying level in --show-default-cparams

* Displaying actual enum value instead of level
2020-02-06 13:51:43 -08:00
Bimba Shrestha
d1389ac3ff Adding assert(NB_STRATEGIES == upperLimit) 2020-02-03 09:52:39 -08:00
Bimba Shrestha
19473390d1 Adding (void)ZSTD_strategyMap 2020-01-31 10:54:02 -08:00
Bimba Shrestha
38d010eea0 Adding static assert and using it in cli 2020-01-31 10:47:17 -08:00
Bimba Shrestha
9d9cfa9ef9 Adding assert and using ZSTD_STRATEGY_MAX as ZSTD_NB_STRATEGIES 2020-01-30 17:54:01 -08:00
Bimba Shrestha
6ad2f0d7d6 Adding back one cast and using UTIL_FILESIZE_UNKNOWN to check unknown file size 2020-01-30 17:16:02 -08:00
Bimba Shrestha
c203341292 Removing more casts 2020-01-30 17:07:58 -08:00
Bimba Shrestha
2a0771cfdb Removing unnecessary cast 2020-01-30 17:05:33 -08:00
Bimba Shrestha
b0ceab4dc0 Moving default cparams back, checking for unknown file size, using getCParams() 2020-01-30 16:52:07 -08:00
Bimba Shrestha
b660ef87f8 Convertion nit U32 -> size_t 2020-01-30 14:29:04 -08:00
Bimba Shrestha
71754a26f8 Update comment 2020-01-30 14:14:27 -08:00
Bimba Shrestha
2f10019b92 Adding --show-default-cparams (show cparams before compressing 2020-01-30 14:12:03 -08:00
Nick Terrell
fa48af545f Add --single-thread to zstd --help 2020-01-17 13:54:49 -08:00
Bimba Shrestha
f25a6e9f8f Adding new cli endpoint --patch-from= (#1940)
* Adding new cli endpoint --diff-from=

* Appveyor conversion nit

* Using bool set trick instead of direct set

* Removing --diff-from and only leaving --diff-from=#

* Throwing error when both dictFileName vars are set

* Clean up syntax

* Renaming diff-from to patch-from

* Revering comma separated syntax clean up

* Updating playtests with patch-from

* Uncommenting accidentally commented

* Updating remaining docs and var names to be patch-from instead of diff-from

* Constifying

* Using existing log2 function and removing newly created one

* Argument order (moving prefs to end)

* Using comma separated syntax

* Moving to outside #ifndef
2020-01-10 14:25:24 -08:00
Yann Collet
4b4de7493f refactoring : remove global variable g_displayOut
displaying into stderr or stdout is now an explicit operation,
no longer depending on some global state set elsewhere in the code.
2019-12-02 16:19:03 -08:00
Yann Collet
a49417b5af fix recent issue combining -r with empty list of input files
This would resize the table of input filenames to zero,
delivering an empty table,
to which it was no longer possible to add stdin.
2019-12-02 14:28:18 -08:00
Yann Collet
c71bd45a3b Merge branch 'dev' into ahmed_file 2019-11-26 11:20:26 -08:00
Yann Collet
9a22140ef4 created UTIL_chmod()
protecting "/dev/null" from having its permissions changed.

also : minor : improved consistency of util.h API
2019-11-25 13:45:22 -08:00
Yann Collet
31a0abbfda updated pzstd and largeNbDicts to use the new FileNamesTable* abstraction 2019-11-06 09:10:05 -08:00
Yann Collet
b09f59390b refactor code to only use FileNamesTable* 2019-11-05 17:02:43 -08:00
Yann Collet
81c51483e9 Merge branch 'dev' into ahmed_file 2019-11-05 14:44:12 -08:00
Yann Collet
47034cd6c3
Merge pull request #1850 from bimbashrestha/named_pipes
Gating named pipe support on windows
2019-10-30 12:18:58 -07:00