AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7,164 Commits 1 Branch 0 Tags 27 MiB
76b9e42b0b
Commit Graph

1655 Commits

Author SHA1 Message Date
Bimba Shrestha
8a3974807e Allowing named pipes to go through zstdcli 2019-10-22 15:23:22 -07:00
Yann Collet
514f941a70
Merge pull request #1836 from facebook/dirFlat2 
Added --output-dir-flat=
 2019-10-22 15:19:04 -07:00
Nick Terrell
b1ec94e63c Fix ZSTD_f_zstd1_magicless for small data 
* Fix `ZSTD_FRAMEHEADERSIZE_PREFIX` and `ZSTD_FRAMEHEADERSIZE_MIN` to
  take a `format` parameter, so it is impossible to get the wrong size.
* Fix the places that called `ZSTD_FRAMEHEADERSIZE_PREFIX` without
  taking the format into account, which is now impossible by design.
* Call `ZSTD_frameHeaderSize_internal()` with `dctx->format`.
* The added tests catch both bugs in `ZSTD_decompressFrame()`.

Fixes #1813.
 2019-10-21 21:16:17 -07:00
Yann Collet
989e0f0cee Added --output-dir-flat= 
which is a preferred construction when applicable.

Also, added --output-dir-flat into man page.
 2019-10-21 14:39:18 -07:00
Yann Collet
6cf04c0344
Merge pull request #1834 from facebook/winFix 
Windows fixes
 2019-10-21 13:45:17 -07:00
Yann Collet
632e07747c minor refactor of FIO_compressGzFrame() 
mostly narrowing variables lifetime and mutability.
 2019-10-21 12:14:59 -07:00
Yann Collet
ca73c218be added mode documentation on ZSTD_CLEVEL 
providing range restriction [1-19] explicitly
partially answering #1829
 2019-10-18 17:22:45 -07:00
Yann Collet
0492c57013 fixed visual defined test 2019-10-18 17:08:52 -07:00
Yann Collet
f379637025 removed Visual warning 
for pointer casts
 2019-10-18 17:05:42 -07:00
Yann Collet
1bd6c15b6b fix function cast warning on Windows with gcc9 2019-10-18 15:45:57 -07:00
Yann Collet
58c59341d3 updated windows binary resources 2019-10-18 15:41:12 -07:00
Yann Collet
29e46ed0bd fix test on windows 
isDirectory() doesn't work on Windows
if directory name is followed by '/'
 2019-10-18 14:28:34 -07:00
Yann Collet
ad86a5d0bc rewrite FIO_createFilename_fromOutDir() 2019-10-18 11:15:10 -07:00
Yann Collet
caf40d0ae4 fix : no output file opened in test mode 
also : redistributed error code within fileio.c
for more precise diagnosis.
 2019-10-17 16:58:49 -07:00
Yann Collet
0a24d4ef18 removed regular file test on Windows 
since it does not work well on this platform
(tested with MinGW).

Note : could be an issue within UTIL_isRegularFile()
 2019-10-17 16:39:47 -07:00
Yann Collet
0ee360982d improved test mode -t 
The test mode do no longer open a file `/dev/null` nor write anything to output.

This is supposed to be more efficient than writing to `/dev/null`,
and more universal too : the previous method was failing on Windows.
 2019-10-17 16:09:53 -07:00
Yann Collet
1795133c45 refactored FIO_compressMultipleFilenames() prototype 
for consistency
 2019-10-17 15:32:03 -07:00
Yann Collet
7f86ae2867 fixed multiple implicit casts 2019-10-17 15:27:25 -07:00
Yann Collet
157479af0c fixed isSameFile() 2019-10-17 14:31:42 -07:00
Yann Collet
a71256a2ee fix several cast 2019-10-17 11:01:20 -07:00
Yann Collet
000404311f fix incorrect dictName/FileName comparison on Windows 
inode identification does not seem to work on Windows,
even with on a msys2 posix layer.
 2019-10-17 10:56:14 -07:00
Yann Collet
19741c7d99
Merge pull request #1815 from facebook/zlibwrap 
make zlibWrapper strict ISO-C90 compatible
 2019-10-16 16:45:15 -07:00
Yann Collet
6309be677c minor comments & refactoring 2019-10-15 16:09:18 -07:00
Ahmed Abdellah
779ea72953 Adding --file=FILE feature 2019-10-15 07:49:13 +01:00
Yann Collet
8b6d96827c
Merge pull request #1731 from chungy/stdin_nochmod 
[programs] don’t do chmod when coming from stdin or multiple files
 2019-10-14 09:34:56 -07:00
Yann Collet
b3e92ad346
Merge pull request #1812 from senhuang42/output-dir-flat 
Support for --output-dir-flat: output into a directory
 2019-10-14 09:25:05 -07:00
Eric van Gyzen
46ee10dfb5 Include errno in all fwrite error messages in fileio.c 2019-10-10 09:21:29 -05:00
Sen Huang
c5ebb37051 Only check for filename collisions when using --output-dir-flat 2019-10-09 09:39:52 -04:00
Sen Huang
6e406b55ee Casting to void* to avoid C4090 warnings 2019-10-08 09:54:59 -04:00
Sen Huang
332b5988d8 Suppress C4090 warning 2019-10-08 09:44:24 -04:00
Sen Huang
f9b1da6f62 Hopefully fix VS compiler warning 2019-10-08 00:15:28 -04:00
Yann Collet
e0d413d648 fixed init warning 2019-10-04 15:09:52 -07:00
Sen Huang
6b81bfb257 Changed function name, fix test 2019-10-03 15:23:49 -04:00
Sen Huang
64bc441d7d Now constructs final destination path without allocating new table 2019-10-03 13:53:04 -04:00
Sen Huang
4dc604cab8 Addressed comments on malloc 2019-10-02 18:34:42 -04:00
Sen Huang
c763457e0a Static analyze fix 2019-10-02 15:30:24 -04:00
Sen Huang
b93f1b2a30 CI Tests fix 2019-10-02 11:29:34 -04:00
Sen Huang
147b761454 Removed strdup() dependency 2019-10-02 11:18:14 -04:00
Sen Huang
1e4d602412 Modified message 2019-10-02 11:11:35 -04:00
Sen Huang
f80437c586 Add support for --output-dir-flat 
New flag to specify output directory destination for multiple files.
 2019-10-02 11:08:20 -04:00
Yann Collet
ad2a2785f7 bump version number to v1.4.4 
so that future reports on `dev` branch use this number instead
 2019-09-24 15:15:33 -07:00
Sergey Ponomarev
dafe796e39 #1790 short tar's extensions tgz, txz, tlz4m .tzst should be decompressed with .tar suffix 2019-09-18 09:23:10 +03:00
Sergey Ponomarev
bfb4d830b2 FIO_determineDstName: extract dstFileNameEndPos variable 2019-09-18 09:21:00 +03:00
Sergey Ponomarev
59f369a6da Add short tar's extensions .tgz (.tar.gz), .txz (.tar.xz), .tzst (.tar.zst) 2019-09-14 21:30:15 +03:00
Sergey Ponomarev
a101721f4e Use one strstr() call instead of chain of strcmp() 2019-09-14 21:26:27 +03:00
Sergey Ponomarev
7d9cd22e21 #754 Add a hint about -o option 2019-09-14 21:23:47 +03:00
Sergey Ponomarev
8cc815a941 #754 sufixlist->SUFFIX_LIST 2019-09-14 21:15:24 +03:00
Sergey Ponomarev
b804dd3e5b #754 move sufixlist upper and improve error message on missing suffix 2019-09-14 21:14:43 +03:00
W. Felix Handte
e1ec8004cc Formatting and Clean Up 2019-09-12 16:27:05 -04:00
W. Felix Handte
5a9baae9cf Set M-Time Nanoseconds 2019-09-12 11:50:33 -04:00
Felix Handte
6ae1ec96bc
Merge pull request #1708 from neheb/dev 
zstd: Don't use utime on Linux
 2019-09-12 11:44:31 -04:00
Yann Collet
41416f0927
Merge pull request #1773 from bimbashrestha/rle_first_block_decompression_fix 
Removing redundant condition in decompression, making first block rle…
 2019-09-10 11:17:29 -07:00
Sen Huang
62616c4d90 fixes for windows compilation 2019-09-06 13:20:50 -07:00
Sen Huang
30bff50e06 fixes for tests and segfault 2019-09-06 11:10:53 -07:00
Bimba Shrestha
d687d603e4 Removing redundant condition in decompression, making first block rles valid to deocmpress 2019-09-06 10:46:19 -07:00
Sen Huang
a9c807a948 kill memory leaks, cleanup, fix some dumb bugs 2019-09-06 10:17:04 -07:00
Sen Huang
6beb3c0159 added decompression support 2019-09-05 17:56:24 -07:00
Sen Huang
7f98b46876 adding support for -O flag: multiple files into one directory for compressions (decompression to come) 2019-09-05 16:03:35 -07:00
Mike Swanson
25878b2de9 [programs] don’t do chmod when coming from stdin or multiple files 
Fixes #1729
 2019-09-02 18:02:32 -07:00
Nick Magerko
493f95c7df Fix merge conflicts 2019-08-22 11:51:41 -07:00
Nick Terrell
a505463710
Merge pull request #1726 from nmagerko/stream-size 
Add --stream-size=# option
 2019-08-22 11:31:15 -07:00
Nick Magerko
fd486a846a Differentiate --stream-size from --size-hint 2019-08-22 09:37:47 -07:00
Nick Magerko
2cdda8b3c4 Minor documentation update 2019-08-22 09:13:28 -07:00
Nick Magerko
05d7479a50 Document --size-hint 2019-08-20 14:08:26 -07:00
Nick Magerko
2d39b43906 Use int for srcSizeHint when sensible 2019-08-19 16:49:25 -07:00
Nick Magerko
dffbac5f89 Add --size-hint=# option 2019-08-19 11:38:49 -07:00
Nick Magerko
30bfa228e8 Keep content size flag set in stream size mode 2019-08-19 11:20:28 -07:00
Nick Magerko
a24dc3a935 Remove extraneous variables 2019-08-19 11:14:56 -07:00
Nick Magerko
f781cf672b Remove extraneous parameter 2019-08-19 11:07:43 -07:00
Nick Magerko
bbd83c2ab3 Update man page 2019-08-19 09:11:22 -07:00
Nick Magerko
c403b12f9d Set pledged size just before compression 2019-08-19 09:01:31 -07:00
Nick Magerko
af0c9501d1 Add --stream-size=# command 2019-08-16 11:34:39 -07:00
Ed Maste
b81d7cc6a0 remove extraneous doubled ;s 2019-08-15 21:17:06 -04:00
Yann Collet
01b2331ad1 bumped version number 
to v1.4.3
 2019-08-05 17:17:16 +02:00
Yann Collet
26351071bc
Merge pull request #1706 from LeeYoung624/dev 
add NULL pointer check in util.c
 2019-08-05 15:56:50 +02:00
Yann Collet
387e20d4f0 fixed minor conversion warning in datagen 2019-08-02 18:02:54 +02:00
Yann Collet
37f47e51a8 fixed datagen 
to produce same content on both 32 and 64-bit platforms
by removing floating from literal table determination.

also : added checksum trace in compression control test,
so that it's easier to determine if test fails
as a consequence of compressing a different sample.
 2019-08-02 17:34:53 +02:00
Rosen Penev
41e90653fe
zstd: Don't use utime on Linux 
utime is deprecated by POSIX 2008 and optionally not available with
uClibc-ng.

Got rid of a few useless headers in timefn.h.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
 2019-07-31 12:29:13 -07:00
LeeYoung624
c5caaf52a4 bug fix : NULL pointer 2019-07-29 17:05:50 +08:00
LeeYoung624
793b94b354 memory leak fix 2019-07-25 21:07:57 +08:00
Yann Collet
1f16245e9e updated man page 2019-07-24 16:04:37 -07:00
Yann Collet
0d38ee3c30
Merge pull request #1690 from piguin/dev 
fix compiling errors with clang-8
 2019-07-24 15:37:05 -07:00
Tyler Tran
5a61e66f7b previous commit did not undo all changes 2019-07-24 13:53:50 -07:00
Tyler Tran
12d60a9bd9 removing changes to zstd.1 2019-07-24 13:52:34 -07:00
Tyler Tran
f8c1d7979c modifying minor nit 2019-07-22 16:36:44 -07:00
Tyler Tran
02da4497f0 Adding documentation for shrink flag PR #1656 2019-07-22 16:33:22 -07:00
Qin Li
04a9d6b828 fix compiling errors with clang-8 
Compiling with clang-8 fails with the following errors:

largeNbDicts.c:562:37: error: implicit conversion turns floating-point
number into integer: 'const double' to 'U64' (aka 'unsigned long')
[-Werror,-Wfloat-conversion]
        U64 const dTime_ns = result.nanoSecPerRun;
                  ~~~~~~~~   ~~~~~~~^~~~~~~~~~~~~

zstdcli.c:300:5: error: '@return' command used in a comment that is
not attached to a function or method declaration
[-Werror,-Wdocumentation]
 * @return 1 means that cover parameters were correct
   ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

zstdcli.c:301:5: error: '@return' command used in a comment that is
not attached to a function or method declaration
[-Werror,-Wdocumentation]
 * @return 0 in case of malformed parameters
   ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 2019-07-18 19:41:00 -07:00
Yann Collet
8eda16c9f6
Merge pull request #1677 from LeeYoung624/gitignore_fix 
fix gitignore errors
 2019-07-09 14:36:38 -07:00
Yann Collet
b8ec4b0fd6 updated version number (to v1.4.1) 
also : added doc on context re-use, as suggested by @scherepanov at #1676
 2019-07-09 11:43:59 -07:00
LeeYoung624
654cb9d439 fix gitignore errors 2019-07-09 21:08:13 +08:00
Yann Collet
096714d1b8
Merge pull request #1671 from ephiepark/dev 
Adding targetCBlockSize param
 2019-07-03 17:47:44 -07:00
Ephraim Park
9007701670 Adding targetCBlockSize param 2019-07-03 15:41:52 -07:00
Yann Collet
857e608b51
Merge pull request #1658 from facebook/memset 
memset() rather than reduceIndex()
 2019-07-01 15:01:43 -07:00
Tyler-Tran
c55d2e7ba3 Adding shrinking flag for cover and fastcover (#1656) 
* Changed ERROR(GENERIC) excluding inits

* editing git ignore

* Edited init functions to size_t returns

* moved declarations earlier

* resolved issues with changes to init functions

* fixed style and an error check

* attempting to add tests that might trigger changes

* added && die to cases expecting to fail

* resolved no die on expected failed command

* fixed accel to be incorrect value

* Adding an automated shrinking option

* Fixing build

* finalizing fixes

* fix?

* Removing added comment in cover.h

* Styling fixes

* Merging with fb dev

* removing megic number for default regression

* Requested revisions

* fixing support for fast cover

* fixing casting errors

* parenthesis fix

* fixing some build nits

* resolving travis ci syntax

* might resolve all compilation issues

* removed unused variable

* remodeling the selectDict function

* fixing bad memory access

* fixing error checks

* fixed erroring check in selectDict

* fixing mixed declarations

* modify mixed declaration

* fixing nits and adding test cases

* Adding requested changes + fixed bug for error checking

* switched double comparison from != to <

* fixed declaration typing

* refactoring COVER_best_finish() and changing shrinkDict

* removing the const's

* modifying ZDICT_optimizeTrainFromBuffer_cover functions

* fixing potential bad memcpy

* fixing the error function for dict size
 2019-06-27 16:26:57 -07:00
Yann Collet
944e2e9e12 benchfn : added macro macro CONTROL() 
like assert() but cannot be disabled.
proper separation of user contract errors (CONTROL())
and invariant verification (assert()).
 2019-06-21 15:58:55 -07:00
Yann Collet
1166782eb7
Merge pull request #1640 from felixhandte/lstat-macro-guard 
Protect lstat() With Better Macro Guard
 2019-06-14 17:30:03 -07:00
Mike Swanson
af80f6dfac [programs] Don’t try to chmod a dst file if it can’t be opened 
Repairs an oversight in my last commit, thanks @Cyan4973
 2019-06-09 01:52:45 -07:00
Mike Swanson
3968160a91 [programs] set chmod 600 after opening destination file 
This resolves a race condition where zstd or unzstd may expose read
permissions beyond the original file allowed.  Mode 600 is used
temporarily during the compression and decompression write stage
and the new file inherits the original file’s mode at the end.

Fixes #1630
 2019-06-08 21:54:02 -07:00
W. Felix Handte
d2c48042e6 Switch Macro Guarding lstat() 2019-06-07 15:32:28 -04:00
Yosuke Tanigawa
f46a3603c9
Update README.md 
Fix minor typo in the documentation.
 2019-06-05 09:11:21 -07:00