AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7,184 Commits 1 Branch 0 Tags 27 MiB
9cf8e67aaf
Commit Graph

1625 Commits

Author SHA1 Message Date
Sen Huang
f9b1da6f62 Hopefully fix VS compiler warning 2019-10-08 00:15:28 -04:00
Yann Collet
e0d413d648 fixed init warning 2019-10-04 15:09:52 -07:00
Sen Huang
6b81bfb257 Changed function name, fix test 2019-10-03 15:23:49 -04:00
Sen Huang
64bc441d7d Now constructs final destination path without allocating new table 2019-10-03 13:53:04 -04:00
Sen Huang
4dc604cab8 Addressed comments on malloc 2019-10-02 18:34:42 -04:00
Sen Huang
c763457e0a Static analyze fix 2019-10-02 15:30:24 -04:00
Sen Huang
b93f1b2a30 CI Tests fix 2019-10-02 11:29:34 -04:00
Sen Huang
147b761454 Removed strdup() dependency 2019-10-02 11:18:14 -04:00
Sen Huang
1e4d602412 Modified message 2019-10-02 11:11:35 -04:00
Sen Huang
f80437c586 Add support for --output-dir-flat 
New flag to specify output directory destination for multiple files.
 2019-10-02 11:08:20 -04:00
Yann Collet
ad2a2785f7 bump version number to v1.4.4 
so that future reports on `dev` branch use this number instead
 2019-09-24 15:15:33 -07:00
Sergey Ponomarev
dafe796e39 #1790 short tar's extensions tgz, txz, tlz4m .tzst should be decompressed with .tar suffix 2019-09-18 09:23:10 +03:00
Sergey Ponomarev
bfb4d830b2 FIO_determineDstName: extract dstFileNameEndPos variable 2019-09-18 09:21:00 +03:00
Sergey Ponomarev
59f369a6da Add short tar's extensions .tgz (.tar.gz), .txz (.tar.xz), .tzst (.tar.zst) 2019-09-14 21:30:15 +03:00
Sergey Ponomarev
a101721f4e Use one strstr() call instead of chain of strcmp() 2019-09-14 21:26:27 +03:00
Sergey Ponomarev
7d9cd22e21 #754 Add a hint about -o option 2019-09-14 21:23:47 +03:00
Sergey Ponomarev
8cc815a941 #754 sufixlist->SUFFIX_LIST 2019-09-14 21:15:24 +03:00
Sergey Ponomarev
b804dd3e5b #754 move sufixlist upper and improve error message on missing suffix 2019-09-14 21:14:43 +03:00
W. Felix Handte
e1ec8004cc Formatting and Clean Up 2019-09-12 16:27:05 -04:00
W. Felix Handte
5a9baae9cf Set M-Time Nanoseconds 2019-09-12 11:50:33 -04:00
Felix Handte
6ae1ec96bc
Merge pull request #1708 from neheb/dev 
zstd: Don't use utime on Linux
 2019-09-12 11:44:31 -04:00
Yann Collet
41416f0927
Merge pull request #1773 from bimbashrestha/rle_first_block_decompression_fix 
Removing redundant condition in decompression, making first block rle…
 2019-09-10 11:17:29 -07:00
Sen Huang
62616c4d90 fixes for windows compilation 2019-09-06 13:20:50 -07:00
Sen Huang
30bff50e06 fixes for tests and segfault 2019-09-06 11:10:53 -07:00
Bimba Shrestha
d687d603e4 Removing redundant condition in decompression, making first block rles valid to deocmpress 2019-09-06 10:46:19 -07:00
Sen Huang
a9c807a948 kill memory leaks, cleanup, fix some dumb bugs 2019-09-06 10:17:04 -07:00
Sen Huang
6beb3c0159 added decompression support 2019-09-05 17:56:24 -07:00
Sen Huang
7f98b46876 adding support for -O flag: multiple files into one directory for compressions (decompression to come) 2019-09-05 16:03:35 -07:00
Mike Swanson
25878b2de9 [programs] don’t do chmod when coming from stdin or multiple files 
Fixes #1729
 2019-09-02 18:02:32 -07:00
Nick Magerko
493f95c7df Fix merge conflicts 2019-08-22 11:51:41 -07:00
Nick Terrell
a505463710
Merge pull request #1726 from nmagerko/stream-size 
Add --stream-size=# option
 2019-08-22 11:31:15 -07:00
Nick Magerko
fd486a846a Differentiate --stream-size from --size-hint 2019-08-22 09:37:47 -07:00
Nick Magerko
2cdda8b3c4 Minor documentation update 2019-08-22 09:13:28 -07:00
Nick Magerko
05d7479a50 Document --size-hint 2019-08-20 14:08:26 -07:00
Nick Magerko
2d39b43906 Use int for srcSizeHint when sensible 2019-08-19 16:49:25 -07:00
Nick Magerko
dffbac5f89 Add --size-hint=# option 2019-08-19 11:38:49 -07:00
Nick Magerko
30bfa228e8 Keep content size flag set in stream size mode 2019-08-19 11:20:28 -07:00
Nick Magerko
a24dc3a935 Remove extraneous variables 2019-08-19 11:14:56 -07:00
Nick Magerko
f781cf672b Remove extraneous parameter 2019-08-19 11:07:43 -07:00
Nick Magerko
bbd83c2ab3 Update man page 2019-08-19 09:11:22 -07:00
Nick Magerko
c403b12f9d Set pledged size just before compression 2019-08-19 09:01:31 -07:00
Nick Magerko
af0c9501d1 Add --stream-size=# command 2019-08-16 11:34:39 -07:00
Ed Maste
b81d7cc6a0 remove extraneous doubled ;s 2019-08-15 21:17:06 -04:00
Yann Collet
01b2331ad1 bumped version number 
to v1.4.3
 2019-08-05 17:17:16 +02:00
Yann Collet
26351071bc
Merge pull request #1706 from LeeYoung624/dev 
add NULL pointer check in util.c
 2019-08-05 15:56:50 +02:00
Yann Collet
387e20d4f0 fixed minor conversion warning in datagen 2019-08-02 18:02:54 +02:00
Yann Collet
37f47e51a8 fixed datagen 
to produce same content on both 32 and 64-bit platforms
by removing floating from literal table determination.

also : added checksum trace in compression control test,
so that it's easier to determine if test fails
as a consequence of compressing a different sample.
 2019-08-02 17:34:53 +02:00
Rosen Penev
41e90653fe
zstd: Don't use utime on Linux 
utime is deprecated by POSIX 2008 and optionally not available with
uClibc-ng.

Got rid of a few useless headers in timefn.h.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
 2019-07-31 12:29:13 -07:00
LeeYoung624
c5caaf52a4 bug fix : NULL pointer 2019-07-29 17:05:50 +08:00
LeeYoung624
793b94b354 memory leak fix 2019-07-25 21:07:57 +08:00
Yann Collet
1f16245e9e updated man page 2019-07-24 16:04:37 -07:00
Yann Collet
0d38ee3c30
Merge pull request #1690 from piguin/dev 
fix compiling errors with clang-8
 2019-07-24 15:37:05 -07:00
Tyler Tran
5a61e66f7b previous commit did not undo all changes 2019-07-24 13:53:50 -07:00
Tyler Tran
12d60a9bd9 removing changes to zstd.1 2019-07-24 13:52:34 -07:00
Tyler Tran
f8c1d7979c modifying minor nit 2019-07-22 16:36:44 -07:00
Tyler Tran
02da4497f0 Adding documentation for shrink flag PR #1656 2019-07-22 16:33:22 -07:00
Qin Li
04a9d6b828 fix compiling errors with clang-8 
Compiling with clang-8 fails with the following errors:

largeNbDicts.c:562:37: error: implicit conversion turns floating-point
number into integer: 'const double' to 'U64' (aka 'unsigned long')
[-Werror,-Wfloat-conversion]
        U64 const dTime_ns = result.nanoSecPerRun;
                  ~~~~~~~~   ~~~~~~~^~~~~~~~~~~~~

zstdcli.c:300:5: error: '@return' command used in a comment that is
not attached to a function or method declaration
[-Werror,-Wdocumentation]
 * @return 1 means that cover parameters were correct
   ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

zstdcli.c:301:5: error: '@return' command used in a comment that is
not attached to a function or method declaration
[-Werror,-Wdocumentation]
 * @return 0 in case of malformed parameters
   ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 2019-07-18 19:41:00 -07:00
Yann Collet
8eda16c9f6
Merge pull request #1677 from LeeYoung624/gitignore_fix 
fix gitignore errors
 2019-07-09 14:36:38 -07:00
Yann Collet
b8ec4b0fd6 updated version number (to v1.4.1) 
also : added doc on context re-use, as suggested by @scherepanov at #1676
 2019-07-09 11:43:59 -07:00
LeeYoung624
654cb9d439 fix gitignore errors 2019-07-09 21:08:13 +08:00
Yann Collet
096714d1b8
Merge pull request #1671 from ephiepark/dev 
Adding targetCBlockSize param
 2019-07-03 17:47:44 -07:00
Ephraim Park
9007701670 Adding targetCBlockSize param 2019-07-03 15:41:52 -07:00
Yann Collet
857e608b51
Merge pull request #1658 from facebook/memset 
memset() rather than reduceIndex()
 2019-07-01 15:01:43 -07:00
Tyler-Tran
c55d2e7ba3 Adding shrinking flag for cover and fastcover (#1656) 
* Changed ERROR(GENERIC) excluding inits

* editing git ignore

* Edited init functions to size_t returns

* moved declarations earlier

* resolved issues with changes to init functions

* fixed style and an error check

* attempting to add tests that might trigger changes

* added && die to cases expecting to fail

* resolved no die on expected failed command

* fixed accel to be incorrect value

* Adding an automated shrinking option

* Fixing build

* finalizing fixes

* fix?

* Removing added comment in cover.h

* Styling fixes

* Merging with fb dev

* removing megic number for default regression

* Requested revisions

* fixing support for fast cover

* fixing casting errors

* parenthesis fix

* fixing some build nits

* resolving travis ci syntax

* might resolve all compilation issues

* removed unused variable

* remodeling the selectDict function

* fixing bad memory access

* fixing error checks

* fixed erroring check in selectDict

* fixing mixed declarations

* modify mixed declaration

* fixing nits and adding test cases

* Adding requested changes + fixed bug for error checking

* switched double comparison from != to <

* fixed declaration typing

* refactoring COVER_best_finish() and changing shrinkDict

* removing the const's

* modifying ZDICT_optimizeTrainFromBuffer_cover functions

* fixing potential bad memcpy

* fixing the error function for dict size
 2019-06-27 16:26:57 -07:00
Yann Collet
944e2e9e12 benchfn : added macro macro CONTROL() 
like assert() but cannot be disabled.
proper separation of user contract errors (CONTROL())
and invariant verification (assert()).
 2019-06-21 15:58:55 -07:00
Yann Collet
1166782eb7
Merge pull request #1640 from felixhandte/lstat-macro-guard 
Protect lstat() With Better Macro Guard
 2019-06-14 17:30:03 -07:00
Mike Swanson
af80f6dfac [programs] Don’t try to chmod a dst file if it can’t be opened 
Repairs an oversight in my last commit, thanks @Cyan4973
 2019-06-09 01:52:45 -07:00
Mike Swanson
3968160a91 [programs] set chmod 600 after opening destination file 
This resolves a race condition where zstd or unzstd may expose read
permissions beyond the original file allowed.  Mode 600 is used
temporarily during the compression and decompression write stage
and the new file inherits the original file’s mode at the end.

Fixes #1630
 2019-06-08 21:54:02 -07:00
W. Felix Handte
d2c48042e6 Switch Macro Guarding lstat() 2019-06-07 15:32:28 -04:00
Yosuke Tanigawa
f46a3603c9
Update README.md 
Fix minor typo in the documentation.
 2019-06-05 09:11:21 -07:00
Ephraim Park
e498bb64b6 zstdcli : align output message with previous message 2019-06-04 09:42:18 -07:00
Ephraim Park
a38601f05a zstdcli : remove extra semicolon 2019-06-04 09:25:16 -07:00
Ephraim Park
ef2c836955 zstdcli : trying to fix declaration after statement 2019-06-04 09:23:49 -07:00
Ephraim Park
5fe974261b zstdcli : moving cpu load calculation from FIO_compressFilename_dstFile to FIO_compressFilename_internal 2019-06-04 09:04:35 -07:00
Ephraim Park
2841c79e87 zstdcli : fixing mixed declarations and code error 2019-06-03 17:36:09 -07:00
Ephraim Park
43d463eccb zstdcli : moving start time and cpu initialization after potential prompt 2019-06-03 17:31:14 -07:00
Ephraim Park
5029ebce2f zstdcli : exposing cpu load indicator for each file on -vv mode 2019-06-03 14:38:40 -07:00
W. Felix Handte
61025d5b7d zstdgrep: Handle -f Flag 2019-05-24 16:55:59 -04:00
Rosen Penev
23b5ee1236
fileio: Disable BACKTRACE under uClibc 
uClibc does not support BACKTRACE. It also defines __GLIBC__ and minor as both 2.
This fixes compilation under such a setup.
 2019-05-06 15:02:47 -07:00
cev
c4a40dbf65 set followLinks option true to cat, zcat and gzcat programs 2019-04-27 19:06:29 -04:00
myzhang1029
f837326f11
Fix #1591 - Not building on MinGW-W64 
Add a static function LONG_TELL for the forth #if branch
 2019-04-24 20:06:05 +08:00
Josh Soref
a880ca239b Spelling (#1582) 
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
 2019-04-12 11:18:11 -07:00
Yann Collet
1e01560b83 fixed timespec_get() initialization bug on some targets 
not sure why, but msan fires an "unitialized variable" error
when time gets properly initialized by timespec_get().
Maybe in some cases, not all bytes of the structure are initialized ?
Or maybe msan fails to detect the initialization ?

Anyway, pre-initializing the variable before passing it to timespec_get() works.
 2019-04-11 13:46:30 -07:00
Yann Collet
30c26ab726 fixed minor warning 
unused variable when assert() turned off in fileio.c
 2019-04-10 17:48:56 -07:00
Yann Collet
fbdd30d68e fixed cmake build script for test programs 2019-04-10 17:47:01 -07:00
Yann Collet
526ec646b7 alternate static assert 
to circumvent Visual's C4804 warning
 2019-04-10 16:05:02 -07:00
Yann Collet
9703a59121 fixed minor conversion warning 2019-04-10 15:54:55 -07:00
Yann Collet
885476fb5b FreeBSD_11 specific fix 
C11 mandates the definition of timespec_get() and TIME_UTC.
However, FreeBSD11 announce C11 compliance, but does not provifr timespec_get(),
breaking link stage for benchfn.
Since it does not provide TIME_UTC either, which is also required by C11,
test this macro: this will automatically rule out FreeBSD 11 for this code path
(it will use the backup C90 path instead, based on clock_t).

The issue seeems fixed in FreeBSD 12.
 2019-04-10 15:22:18 -07:00
Yann Collet
2c6b14ed22 fixed Windows header 
cmake build script: added timefn
 2019-04-10 14:54:13 -07:00
Yann Collet
3d346579d8 no more need for CLOCK_MONOTONIC 2019-04-10 14:16:39 -07:00
Yann Collet
36d2dfd846 moved C11 code path to timespec_get 2019-04-10 14:15:11 -07:00
Yann Collet
4765929271 fixed perror include 2019-04-10 14:04:11 -07:00
Yann Collet
70802cde6d fixed error message 
using stdlib's perror()
 2019-04-10 14:01:18 -07:00
Yann Collet
4b8185c7fc tried a blindfix for unix + c11 2019-04-10 13:26:27 -07:00
Yann Collet
59a7116cc2 benchfn dependencies reduced to only timefn 
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.

Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.

Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
 2019-04-10 12:37:03 -07:00
Yann Collet
094c000904 Merge branch 'dev' into benchfn 2019-04-10 11:57:05 -07:00
Yann Collet
90c0462d63 minor presentation refactoring 
and removed some // comment style
 2019-04-10 10:03:06 -07:00
Ori Peleg
bdeb4786b5 Fixed -r typo 2019-04-10 13:37:41 +03:00
shakeelrao
dca73db30c fix srcSize typo and add new UTIL func to comment 2019-03-28 17:50:34 -07:00
shakeelrao
5333e41ab3 Add NULL check for dict 2019-03-24 00:23:50 -07:00