AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7,964 Commits 1 Branch 0 Tags 27 MiB
f49c1563ff
Commit Graph

1826 Commits

Author SHA1 Message Date
Bimba Shrestha
23165b387e Adding hashLog to --show-default-cparams 2020-02-27 13:56:56 -08:00
Yann Collet
d2464776d1 man page : added --verbose long command 
also updated `zstd.1`
stacking prior updates completed on `zstd.1.md`.
 2020-02-26 14:29:30 -08:00
Felix Handte
61eeca4bd6
Merge pull request #2007 from lgarrison/cli-allow-sizet 
Allow some CLI options to accept numeric values up to size_t
 2020-02-21 08:51:37 -08:00
Lehman Garrison
834d06b655 Allow some cli options to accept numeric values up to size_t instead of unsigned int 2020-02-20 14:29:58 -05:00
Bimba Shrestha
c9ce6db069
Merge pull request #1999 from bket/fix_have_lz4 
Fix forgotten portable header prefix
 2020-02-20 10:55:28 -08:00
Philip Jones
4e728e26ca
Fix integer parsing in cli (#2003) 2020-02-18 15:30:59 -08:00
Björn Ketelaars
cdd59c50e7 Fix forgotten portable header prefix 
PR #1987 forgot to prefix another include causing lz4 detection to fail.
 2020-02-15 12:42:00 +01:00
Bimba Shrestha
06a57cf57e
[build-issue] More portable header prefix usage (#) (#1987) 
* make 4.3 build issue fix

* Changing header name and adding comment
 2020-02-06 14:10:51 -08:00
Bimba Shrestha
f627e1a1ae
Displaying level in --show-default-cparams (#1991) 
* Displaying level in --show-default-cparams

* Displaying actual enum value instead of level
 2020-02-06 13:51:43 -08:00
Bimba Shrestha
075783c1f0
Merge pull request #1980 from bimbashrestha/params 
Adding --show-default-cparams to cli
 2020-02-03 14:43:55 -08:00
Bimba Shrestha
d1389ac3ff Adding assert(NB_STRATEGIES == upperLimit) 2020-02-03 09:52:39 -08:00
Bimba Shrestha
f33baa21c6 Removing assert and changing ratio cSize 2020-01-31 11:54:14 -08:00
Bimba Shrestha
19473390d1 Adding (void)ZSTD_strategyMap 2020-01-31 10:54:02 -08:00
Bimba Shrestha
38d010eea0 Adding static assert and using it in cli 2020-01-31 10:47:17 -08:00
Bimba Shrestha
0fa6c90e89 Moving to same category as -v 2020-01-31 10:38:19 -08:00
Bimba Shrestha
f4ae6c0b94 Moving documentation and elaborating 2020-01-30 17:54:42 -08:00
Bimba Shrestha
9d9cfa9ef9 Adding assert and using ZSTD_STRATEGY_MAX as ZSTD_NB_STRATEGIES 2020-01-30 17:54:01 -08:00
Bimba Shrestha
6ad2f0d7d6 Adding back one cast and using UTIL_FILESIZE_UNKNOWN to check unknown file size 2020-01-30 17:16:02 -08:00
Bimba Shrestha
6bf05a53ab Documenting in the man file 2020-01-30 17:11:38 -08:00
Bimba Shrestha
c203341292 Removing more casts 2020-01-30 17:07:58 -08:00
Bimba Shrestha
2a0771cfdb Removing unnecessary cast 2020-01-30 17:05:33 -08:00
Bimba Shrestha
b0ceab4dc0 Moving default cparams back, checking for unknown file size, using getCParams() 2020-01-30 16:52:07 -08:00
Bimba Shrestha
b660ef87f8 Convertion nit U32 -> size_t 2020-01-30 14:29:04 -08:00
Bimba Shrestha
71754a26f8 Update comment 2020-01-30 14:14:27 -08:00
Bimba Shrestha
2f10019b92 Adding --show-default-cparams (show cparams before compressing 2020-01-30 14:12:03 -08:00
Nick Terrell
7ec87cfb28 Update comment in timefn 2020-01-22 11:17:30 -08:00
Nick Terrell
768adc0774 Fix timefn on android 2020-01-21 18:57:16 -08:00
Nick Terrell
fa48af545f Add --single-thread to zstd --help 2020-01-17 13:54:49 -08:00
Nick Terrell
f6d00c059f [util] Fix readLineFromFile on Cygwin 2020-01-13 14:37:22 -08:00
Nick Terrell
f27b4b4ec5
Merge pull request #1948 from lazka/cygwin-fix-c99-build 
Fix the c99/cmake build under Cygwin/MSYS2
 2020-01-13 11:35:28 -08:00
Christoph Reiter
ddd4c39736 Fix the c99/cmake build under Cygwin/MSYS2 
When building zst under cygwin or msys2 with std=c99 the build would fail because
of an undefined fileno()/_fileno(), which is used by the IS_CONSOLE() macro.

When building with -std=c99 (gcc otherwise defaults to gnu, which implies POSIX),
which is the default of the cmake build, then including unistd.h wont define
_POSIX_VERSION and all other headers also wont expose POSIX API.

To fix this make sure to define _POSIX_C_SOURCE with the version we want before including
unistd.h and so that _POSIX_VERSION is set to the version provided by the system.

Since Cygwin/MSYS2 just follow POSIX we can also remove their special cases for
defining IS_CONSOLE().

And, for completeness, also explicitly include stdio.h which is what actually declares fileno().

Tested with the normal make file and cmake under MSYS2 and Cygwin.
 2020-01-11 01:39:10 +01:00
Bimba Shrestha
f25a6e9f8f Adding new cli endpoint --patch-from= (#1940) 
* Adding new cli endpoint --diff-from=

* Appveyor conversion nit

* Using bool set trick instead of direct set

* Removing --diff-from and only leaving --diff-from=#

* Throwing error when both dictFileName vars are set

* Clean up syntax

* Renaming diff-from to patch-from

* Revering comma separated syntax clean up

* Updating playtests with patch-from

* Uncommenting accidentally commented

* Updating remaining docs and var names to be patch-from instead of diff-from

* Constifying

* Using existing log2 function and removing newly created one

* Argument order (moving prefs to end)

* Using comma separated syntax

* Moving to outside #ifndef
 2020-01-10 14:25:24 -08:00
Christoph Reiter
d0dcaf56c2 Make UTIL_countPhysicalCores() work under Cygwin (#1941) 
Cygwin currently uses the fallback implementation which just returns 1 every time,
which leads to bad performance when zstd is called with -T0 for example.

Instead use the POSIX implementation used for the BSDs which works just fine under Cygwin.

Tested under Cygwin and MSYS2.
 2020-01-07 15:48:26 -08:00
Bimba Shrestha
ef1684f29a [doc] Adding -M# decompression operation modifier documentation (#1938) 
* Adding -M# operation modifier documentation

* Adding long name, typo and update desc
 2020-01-03 16:36:47 -08:00
W. Felix Handte
e2a99db2dd Also Define _ATFILE_SOURCE 2019-12-23 13:24:28 -05:00
W. Felix Handte
5666835ea7 Add Comment 2019-12-23 13:24:28 -05:00
W. Felix Handte
5af8cb7aea Use statbuf->st_mtim Again 2019-12-23 13:24:28 -05:00
Yann Collet
901e545173
Merge pull request #1907 from selavy/check-fread-r1895 
Check for fread failure
 2019-12-03 15:31:11 -08:00
Yann Collet
4b4de7493f refactoring : remove global variable g_displayOut 
displaying into stderr or stdout is now an explicit operation,
no longer depending on some global state set elsewhere in the code.
 2019-12-02 16:19:03 -08:00
Yann Collet
a49417b5af fix recent issue combining -r with empty list of input files 
This would resize the table of input filenames to zero,
delivering an empty table,
to which it was no longer possible to add stdin.
 2019-12-02 14:28:18 -08:00
Yann Collet
d3ec368e13
Merge pull request #1910 from facebook/travisTest 
mingw cross compilation test
 2019-11-27 18:45:16 -08:00
Yann Collet
93ec5cfd05
Merge pull request #1883 from Ericson2314/fix-mingw-cross 
Fix MinGW cross
 2019-11-27 14:48:26 -08:00
Peter Lesslie
8eb499d354 Check for fread failure 
On failure fread may return either a short read or 0. Need to use
ferror to detect error versus eof.
 2019-11-26 20:47:58 -06:00
Yann Collet
d5b4a7ea58 removed scanbuild workaround 2019-11-26 17:46:57 -08:00
Yann Collet
96ee20758c assembleFNT() can no longer fail 2019-11-26 15:44:33 -08:00
Yann Collet
aaab618ae9 pushed aside stdio.h too 
since only UTIL_DISPLAY() depended on it.
 2019-11-26 15:25:32 -08:00
Yann Collet
7543cd055c moved UTIL_DISPLAY() inside util.c 2019-11-26 15:21:58 -08:00
Yann Collet
a684b82774 util: isolated some dependencies 
from *.h to *.c
so that they don't get transitively included
into users of util.h.
 2019-11-26 15:16:53 -08:00
Yann Collet
2d9fad4f52 fixed minor VS warning, on parameter difference 
complaining about a `const` property on one side but not the other.
 2019-11-26 14:53:37 -08:00
Yann Collet
f622c0adf3 switched UTIL_refFilename() to an assert() 2019-11-26 14:48:23 -08:00
Yann Collet
c71bd45a3b Merge branch 'dev' into ahmed_file 2019-11-26 11:20:26 -08:00
Yann Collet
5e657aca90 silence scan-build false positive 
blind attempt
 2019-11-25 15:50:58 -08:00
Yann Collet
9a3de0a535 changed name from createX to assembleX 
shows that the resulting object just takes ownership of provided buffer.
 2019-11-25 15:34:55 -08:00
Yann Collet
b1de3ecdd5 fix mode_t on Windows 2019-11-25 13:59:35 -08:00
Yann Collet
9a22140ef4 created UTIL_chmod() 
protecting "/dev/null" from having its permissions changed.

also : minor : improved consistency of util.h API
 2019-11-25 13:45:22 -08:00
Yann Collet
7aaac3f69c fix #1904 
/dev/null permissions were modified when using sudo rights.
This fixes this bug during decompression.

More importantly, this patch  adds a test, triggered in TravisCI,
ensuring unaltered /dev/null permissions.
 2019-11-25 10:35:36 -08:00
Yann Collet
d4ce04c3e7
Merge pull request #1897 from sergey-dryabzhinsky/patch-1 
WIP: Fix typo in util.c
 2019-11-22 17:25:55 -08:00
Yann Collet
c8fd2cf4c9
Merge pull request #1887 from bimbashrestha/doc-fix 
Fixing hashLog error 2^26=64Mib
 2019-11-19 16:14:20 -08:00
Sergey Dryabzhinsky
612a06eb3e
Update util.c 2019-11-19 23:24:00 +03:00
Sergey Dryabzhinsky
f62cf1fff5
Fix typo in util.c 
There must be mtim*e*
 2019-11-19 23:15:28 +03:00
Bimba Shrestha
b032a6e1d9 Updated some more outdated bounds 2019-11-19 09:55:13 -08:00
John Ericson
10552eaffe Fix MinGW cross install from Unix 
Needed a bunch of `$(EXT)`
 2019-11-14 14:19:07 -05:00
John Ericson
0ede342acc Fix MinGW cross build from Unix 
generate_res.bat seems rather pointless, am I missing something? I just
inlined it into the Makefile.
 2019-11-14 14:19:06 -05:00
Bimba Shrestha
982830b081 Fixing hashLog error 2^26=64Mib 2019-11-14 10:48:55 -08:00
John Ericson
106278e7e5 Use lowercase windows.h 
Helps for cross builds, doesn't matter on Windows itself.
 2019-11-12 19:03:01 -05:00
Yann Collet
2ed5344e84 added target zstd-dictBuilder 
check that the dictBuilder can be compiled
with only lib/common and lib/compress as dependencies.
 2019-11-06 15:46:58 -08:00
Yann Collet
9df49dc50a Visual compiler bug work-around 2019-11-06 15:23:44 -08:00
Yann Collet
be34969e5c minor man page fix 2019-11-06 15:07:35 -08:00
Yann Collet
a7e33e3e10 updated fuzz tests to use FileNamesTable* abstraction 2019-11-06 14:42:13 -08:00
Yann Collet
31a0abbfda updated pzstd and largeNbDicts to use the new FileNamesTable* abstraction 2019-11-06 09:10:05 -08:00
Yann Collet
b09f59390b refactor code to only use FileNamesTable* 2019-11-05 17:02:43 -08:00
Yann Collet
76b9e42b0b refactoring (simplification) of util.h public API 2019-11-05 14:59:45 -08:00
Yann Collet
81c51483e9 Merge branch 'dev' into ahmed_file 2019-11-05 14:44:12 -08:00
Yann Collet
47034cd6c3
Merge pull request #1850 from bimbashrestha/named_pipes 
Gating named pipe support on windows
 2019-10-30 12:18:58 -07:00
Shashank Tavildar
3c1649f139 Removed the optimization check 2019-10-29 15:59:20 -07:00
Shashank Tavildar
6d0b7bd6ce Changed extension comparision logic, added new test cases 2019-10-29 13:36:09 -07:00
Shashank Tavildar
c5060997e9 Added check to perform comparison only if extension is present 2019-10-29 12:56:04 -07:00
Shashank Tavildar
9ab6a747d4 Created utility function to extract extension from filename, fixed tests 2019-10-29 12:27:54 -07:00
Bimba Shrestha
da73b11b14
Merge branch 'dev' into named_pipes 2019-10-29 01:17:13 -04:00
Shashank Tavildar
0f2bff2faf Addressing comments, removing cyclic dependency with header file, updating tests 2019-10-28 18:21:47 -07:00
Shashank Tavildar
0e9a37daeb Fixing tests and moving flag to advanced usage 2019-10-28 15:22:26 -07:00
Yann Collet
65f2d97f33 updated UTIL_createFileNamesTable() inline documentation 2019-10-28 15:20:40 -07:00
Shashank Tavildar
f422914619 Resolving local merge conflicts 2019-10-28 15:15:13 -07:00
Yann Collet
d9c634e13b return final \0 directly from readLine() 2019-10-28 15:03:32 -07:00
Shashank Tavildar
02433e0b15 Addressing comments: -Created a list of extensions defined in fileio.h, -Updated test 2019-10-28 14:54:54 -07:00
Shashank Tavildar
48f856640e Added --exclude-compressed flag feature that skips compression of precompressed files 2019-10-28 13:57:40 -07:00
Yann Collet
d7f258d845 change command name to --filelist= 
updated documentation and man page
 2019-10-28 13:54:36 -07:00
Yann Collet
a78a8759a2 Merge branch 'dev' into ahmed_file 2019-10-28 13:43:28 -07:00
Yann Collet
12efa1ed89 yet another minor visual conversion warning 
this time for 32-bit systems
 2019-10-26 00:27:32 -07:00
Yann Collet
3e5c81ea36 fixed another minor conversion warning on Visual 
and made CONTROL() use exit() instead of abort().
 2019-10-26 00:01:11 -07:00
Yann Collet
74d872e987 fix minor conversion warning on 32-bit 2019-10-25 18:26:30 -07:00
Yann Collet
b40eaced94 minor cosmetic refactoring 2019-10-25 18:16:45 -07:00
Yann Collet
5fb84ca2cf no need to track tableBuf 
free() is compatible with NULL,
let's free() unconditionnally
 2019-10-25 17:34:29 -07:00
Yann Collet
8e414b586d test resilience on garbage file 
and clean traces
 2019-10-25 17:01:26 -07:00
Yann Collet
1ead0c5d5a improved --file=FILE implementation 
pass basic tests
 2019-10-25 16:36:59 -07:00
Shashank Tavildar
55ee7d56e4 Added --exclude-compressed flag feature that skips compression of precompressed files 2019-10-25 15:49:11 -07:00
Bimba Shrestha
5e59d51991 Merge branch 'master' of https://github.com/bimbashrestha/zstd into named_pipes 2019-10-25 15:43:41 -07:00
Bimba Shrestha
857268b32c Gating named pipe support in hedaer file 2019-10-25 15:15:28 -07:00
Bimba Shrestha
0b52d878b2 Cleaning up gate and adding comment to flag 2019-10-25 14:06:50 -07:00
Bimba Shrestha
5b45ff4f1c Gating named file support on windows 2019-10-25 11:32:38 -07:00
W. Felix Handte
4eccc82875 Minor Fixes 2019-10-25 13:59:19 -04:00
Yann Collet
14e9010bb5
Merge pull request #1844 from AhmedAbdellah19/adding_read_files_from_file_feature 
Adding --file=FILE feature
 2019-10-25 10:11:47 -07:00
W. Felix Handte
91c3f545cc Fix Build; Refactor 2019-10-24 20:18:57 -04:00
Felix Handte
506e1a1176
Merge branch 'tzst-ext-tmp' into tzst-ext 2019-10-24 17:49:34 -04:00
Ahmed Abdellah
5249085e11 fixing free const char** filenamesTable 2019-10-24 20:54:40 +01:00
Yann Collet
37f90930da
Merge pull request #1839 from bimbashrestha/named_pipes 
Allowing named pipes to go through in zstdcli
 2019-10-24 10:53:12 -07:00
Ahmed Abdellah
5f9e868ee8 fixing type conversion error 2019-10-24 16:20:58 +01:00
Ahmed Abdellah
0b3096596a fixing AppVeyor errors 2019-10-24 15:25:48 +01:00
Ahmed Abdellah
cddb05ef8c fixing some warning 2019-10-24 14:42:37 +01:00
Ahmed Abdellah
5e206fdd53 fixing some warning 2019-10-24 14:21:01 +01:00
Ahmed Abdellah
0e6a73b148 fixing newTable issues and some warnings 2019-10-24 13:51:12 +01:00
Ahmed Abdellah
639bb46954 removing extra logs 2019-10-24 11:35:26 +01:00
Ahmed Abdellah
849b8c6de8 fixing continuous integeration errors and removing a lot of logs 2019-10-24 11:10:13 +01:00
Ahmed Abdellah
47712c9b15 fixing c90 issue in util.c cont. 2019-10-24 10:30:05 +01:00
Ahmed Abdellah
c799f33899 fixing c90 issue in util.c cont. again 2019-10-24 10:23:36 +01:00
Ahmed Abdellah
8a9741b3ee fixing c90 issue in util.c cont. 2019-10-24 10:17:31 +01:00
Ahmed Abdellah
aefa18ee38 fixing c90 issue in util.c 2019-10-24 10:12:51 +01:00
Ahmed Abdellah
f43e45954f fixing memory leak issue and removing c90 issue 2019-10-24 09:39:16 +01:00
Ahmed Abdellah
8cbe42fcb0 solving the rest of C90 issues in defining local variables in middle of code and comparing uncompatible types 2019-10-23 20:22:07 +01:00
Ahmed Abdellah
9a454e9724 solving C90 issues in defining local variables in middle of code and comparing uncompatible types 2019-10-23 20:15:18 +01:00
AhmedAbdellah19
7a732e2bb2
Merge branch 'dev' into adding_read_files_from_file_feature 2019-10-23 20:29:01 +02:00
Bimba Shrestha
8a3974807e Allowing named pipes to go through zstdcli 2019-10-22 15:23:22 -07:00
Yann Collet
514f941a70
Merge pull request #1836 from facebook/dirFlat2 
Added --output-dir-flat=
 2019-10-22 15:19:04 -07:00
Nick Terrell
b1ec94e63c Fix ZSTD_f_zstd1_magicless for small data 
* Fix `ZSTD_FRAMEHEADERSIZE_PREFIX` and `ZSTD_FRAMEHEADERSIZE_MIN` to
  take a `format` parameter, so it is impossible to get the wrong size.
* Fix the places that called `ZSTD_FRAMEHEADERSIZE_PREFIX` without
  taking the format into account, which is now impossible by design.
* Call `ZSTD_frameHeaderSize_internal()` with `dctx->format`.
* The added tests catch both bugs in `ZSTD_decompressFrame()`.

Fixes #1813.
 2019-10-21 21:16:17 -07:00
Yann Collet
989e0f0cee Added --output-dir-flat= 
which is a preferred construction when applicable.

Also, added --output-dir-flat into man page.
 2019-10-21 14:39:18 -07:00
Yann Collet
6cf04c0344
Merge pull request #1834 from facebook/winFix 
Windows fixes
 2019-10-21 13:45:17 -07:00
Yann Collet
632e07747c minor refactor of FIO_compressGzFrame() 
mostly narrowing variables lifetime and mutability.
 2019-10-21 12:14:59 -07:00
Yann Collet
ca73c218be added mode documentation on ZSTD_CLEVEL 
providing range restriction [1-19] explicitly
partially answering #1829
 2019-10-18 17:22:45 -07:00
Yann Collet
0492c57013 fixed visual defined test 2019-10-18 17:08:52 -07:00
Yann Collet
f379637025 removed Visual warning 
for pointer casts
 2019-10-18 17:05:42 -07:00
Yann Collet
1bd6c15b6b fix function cast warning on Windows with gcc9 2019-10-18 15:45:57 -07:00
Yann Collet
58c59341d3 updated windows binary resources 2019-10-18 15:41:12 -07:00
Yann Collet
29e46ed0bd fix test on windows 
isDirectory() doesn't work on Windows
if directory name is followed by '/'
 2019-10-18 14:28:34 -07:00
Yann Collet
ad86a5d0bc rewrite FIO_createFilename_fromOutDir() 2019-10-18 11:15:10 -07:00
Yann Collet
caf40d0ae4 fix : no output file opened in test mode 
also : redistributed error code within fileio.c
for more precise diagnosis.
 2019-10-17 16:58:49 -07:00
Yann Collet
0a24d4ef18 removed regular file test on Windows 
since it does not work well on this platform
(tested with MinGW).

Note : could be an issue within UTIL_isRegularFile()
 2019-10-17 16:39:47 -07:00
Yann Collet
0ee360982d improved test mode -t 
The test mode do no longer open a file `/dev/null` nor write anything to output.

This is supposed to be more efficient than writing to `/dev/null`,
and more universal too : the previous method was failing on Windows.
 2019-10-17 16:09:53 -07:00
Yann Collet
1795133c45 refactored FIO_compressMultipleFilenames() prototype 
for consistency
 2019-10-17 15:32:03 -07:00
Yann Collet
7f86ae2867 fixed multiple implicit casts 2019-10-17 15:27:25 -07:00
Yann Collet
157479af0c fixed isSameFile() 2019-10-17 14:31:42 -07:00
Yann Collet
a71256a2ee fix several cast 2019-10-17 11:01:20 -07:00
Yann Collet
000404311f fix incorrect dictName/FileName comparison on Windows 
inode identification does not seem to work on Windows,
even with on a msys2 posix layer.
 2019-10-17 10:56:14 -07:00
Yann Collet
19741c7d99
Merge pull request #1815 from facebook/zlibwrap 
make zlibWrapper strict ISO-C90 compatible
 2019-10-16 16:45:15 -07:00
Yann Collet
6309be677c minor comments & refactoring 2019-10-15 16:09:18 -07:00
Ahmed Abdellah
779ea72953 Adding --file=FILE feature 2019-10-15 07:49:13 +01:00
Yann Collet
8b6d96827c
Merge pull request #1731 from chungy/stdin_nochmod 
[programs] don’t do chmod when coming from stdin or multiple files
 2019-10-14 09:34:56 -07:00
Yann Collet
b3e92ad346
Merge pull request #1812 from senhuang42/output-dir-flat 
Support for --output-dir-flat: output into a directory
 2019-10-14 09:25:05 -07:00
Eric van Gyzen
46ee10dfb5 Include errno in all fwrite error messages in fileio.c 2019-10-10 09:21:29 -05:00
Sen Huang
c5ebb37051 Only check for filename collisions when using --output-dir-flat 2019-10-09 09:39:52 -04:00
Sen Huang
6e406b55ee Casting to void* to avoid C4090 warnings 2019-10-08 09:54:59 -04:00