AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
5,015 Commits 1 Branch 0 Tags 27 MiB
9a0643b633
Commit Graph

2203 Commits

Author SHA1 Message Date
Baruch Siach
9a0643b633 lib/Makefile: create include directory before headers installation 
Make sure that $(INCLUDEDIR) exists before copying the headers there.
Otherwise, the contest of header files is copied over
$(DESTDIR)$(INCLUDEDIR), making it a regular file.

While at it, remove $(DESTDIR)$(INCLUDEDIR) from the list of directories
to create in the install-pc target. The install-pc target does not need
this directory.
 2018-05-08 20:59:44 +03:00
Yann Collet
ad4524d605 fix ZSTD_compressBlock() associated with CDict 
reported by @let-def.

It's actually a bug in ZSTD_compressBegin_usingCDict()
which would pass a wrong pledgedSrcSize value (0 instead of ZSTD_CONTENTSIZE_UNKNOWN)
resulting in wrong window size, resulting in downsized seqStore,
resulting in segfault when writing into the seqStore later in the process.

Added a test in fuzzer to cover this use case (fails before the patch).
 2018-05-07 12:54:13 -07:00
Peter Seiderer
64bfdca5b9 Split library install target into pc, static, shared and include only target 
Signed-off-by: Peter Seiderer <ps.report@gmx.net>
 2018-04-30 20:32:32 +02:00
Nick Terrell
ca77822ddf Fix parameter adjustment with dictionary 
The new advanced API basically set `requestedParams = appliedParams` when
using a dictionary. This halted all parameter adjustment, which can hurt
compression ratio if, for example, the window log is small for the first
call, but the rest of the files are large.

This patch fixes the bug, and checks that the `requestedParams` don't change
in the new advanced API when using a dictionary, and generally in the fuzzer.
 2018-04-25 16:32:29 -07:00
Yann Collet
12f60b8c98 clarified documentation related to refPrefix() 2018-04-25 10:17:06 -07:00
Yann Collet
ace856a835 updated documentation of streaming compression api 2018-04-24 14:44:27 -07:00
taigacon
2c3ad05812 Fix the problem that enables DYNAMIC_BMI2 macro by mistake on ARM architecture with Clang (#1110) 2018-04-23 15:41:50 -07:00
Nick Terrell
e8c9dc5cea Fix documentation 2018-04-13 12:43:38 -07:00
Nick Terrell
c0987986e5 Only reset CDict in ZSTD_CCtx_resetParameters() 2018-04-13 11:26:40 -07:00
Nick Terrell
9f76eebd17 Add ZSTD_CCtx_resetParameters() function 
* Fix docs for `ZSTD_CCtx_reset()`.
* Add `ZSTD_CCtx_resetParameters()`.

Fixes #1094.
 2018-04-12 16:54:07 -07:00
Nick Terrell
3c3f59e68f
Enforce pledgeSrcSize whenever known (#1106) 
The test fails before the patch and passes after.

Fixes #1095.
 2018-04-12 16:02:03 -07:00
Nick Terrell
280a236e9e Add ZSTD_CCtx(Param)?_getParameter() function 
Closes #1096.
 2018-04-12 11:50:12 -07:00
Yann Collet
04212178b5 doc : clarified advanced API usage 
sticky parameters only work with `ZSTD_compress_generic()`
 2018-04-10 11:40:36 -07:00
Yann Collet
ad5ba6cdcf updated comment on parameters that can be changed during compression 2018-04-09 17:39:07 -07:00
Yann Collet
1da629f2ad
Merge pull request #1104 from terrelln/fast-train 
Allow negative compression levels in training
 2018-04-09 14:16:20 -07:00
Nick Terrell
569e2abccd Allow negative compression levels in training 
* Set `dictCLevel` in `zstdcli.c`.
* Only set to default level if the compression level `== 0`, not `<= 0`.
 2018-04-09 12:12:03 -07:00
Yann Collet
4195b36dd7
Merge pull request #1100 from bket/stable_sort 
zstd requires a stable sort.
 2018-04-05 11:39:27 -07:00
Yann Collet
f35b8ba9da updated ZSTD_p_chainLog description 2018-04-05 11:05:11 -07:00
Björn Ketelaars
462aed6811 zstd requires a stable sort. 
On OpenBSD qsort() is not guaranteed to be stable, their mergesort() is.
This fixes issue #1088. All the hard work has been done by @terrelln.
 2018-04-05 07:59:16 +02:00
Yann Collet
55f67502f4
Merge pull request #1098 from terrelln/nd-mt 
Only load extra table positions for CDicts
 2018-04-02 15:38:20 -07:00
Nick Terrell
295ab0dbfa Only load extra table positions for CDicts 
Zstdmt uses prefixes to load the overlap between segments. Loading extra
positions makes compression non-deterministic, depending on the previous
job the context was used for. Since loading extra position takes extra
time as well, only do it when creating a `ZSTD_CDict`.

Fixes #1077.
 2018-04-02 14:41:30 -07:00
Yann Collet
5b616fa269
Merge pull request #1090 from bket/openbsd 
Fix building zstd on OpenBSD.
 2018-04-02 14:15:26 -07:00
Björn Ketelaars
9d3048346d Fix building zstd on OpenBSD. 2018-03-31 10:46:20 +02:00
Yann Collet
8be984ec45 fixed comments as suggested by @terrelln 2018-03-30 20:09:27 -07:00
Yann Collet
e6e848bfe9 added ZSTD_getFrameHeader_advanced() 
makes it possible to request frame header from a magicless frame
 2018-03-29 17:51:08 -06:00
Yann Collet
a6694838e1 added more code documentation for ZSTD_getFrameHeader() 2018-03-29 15:24:17 -06:00
René Rebe
21eb26d664 fixed legacy/zstd_v* with older gcc version, by guarding builtin_* 
like in other files
 2018-03-25 20:35:15 +02:00
Yann Collet
ad15c1b724 added __has_attribute() define for non-clang compilers 2018-03-23 19:04:48 -07:00
Yann Collet
52ca7c6c56 make DYNAMIC_BMI2 support of clang conditional to __has_attribute() 
to support older clang versions such as 3.4
 2018-03-23 18:45:42 -07:00
Yann Collet
29b021f9a0
Merge pull request #1067 from facebook/targetLength 
removed limit ZSTD_TARGETLENGTH_MAX
 2018-03-22 10:38:33 -07:00
Nick Terrell
ad344033df Fix broken assertion 
The `avgJobSize` must not be lower than 256 KB for single-pass mode.
In `zstd.h` we say the minimum value for `ZSTD_p_jobSize` is 1 MB,
so ensure that we always pick a size >= 1 MB.

Found by libFuzzer fuzzer tests with large input limits.
 2018-03-21 16:20:30 -07:00
Yann Collet
153bc1c004 removed limit ZSTD_TARGETLENGTH_MAX 
this makes it possible to specify extremely large negative compression levels,
achieving the side effect as "no compression".

It will also be possible to define larger targetlength for ultra compression mode.

There is no adverse side effect due to removing this limit.
 2018-03-21 15:50:05 -07:00
Yann Collet
a99c4a3621 Merge branch 'dev' into advancedDecompress 2018-03-21 06:08:28 -07:00
Yann Collet
87b0cf05bd
Merge pull request #1057 from facebook/lrmSettings 
LRM parameters
 2018-03-21 05:59:39 -07:00
Yann Collet
d1bf609abf
Merge pull request #1059 from terrelln/mt-ldm 
Integrate ldm with zstdmt
 2018-03-20 17:50:20 -07:00
Yann Collet
e0cb8d19c6 fixed legacy test case 2018-03-20 17:48:22 -07:00
Yann Collet
878728dc26 fixed several comments by @terrelln 2018-03-20 16:35:14 -07:00
Yann Collet
e1c52faace
Merge pull request #1060 from facebook/compressImpl 
merge bmi2 implementation of encodeSequence into zstd_compress.c
 2018-03-20 16:19:42 -07:00
Yann Collet
6cda8c932c added test with ZSTD_decompress_generic() + ZSTD_DCtx_refPrefix() 
also :
clarified stage condition to accept new parameters,
fixed initializers correspondingly.
 2018-03-20 16:16:13 -07:00
Yann Collet
0dadb6b70d implemented ZSTD_DCtx_refPrefix*() 2018-03-20 15:45:56 -07:00
Yann Collet
569b8ba4d9 implemented ZSTD_DCtx_refDDict() 2018-03-20 15:43:49 -07:00
Nick Terrell
a3b76a77ef Quiet appveyor warnings 2018-03-20 15:34:40 -07:00
Yann Collet
6873fec658 changed dictMore for dictContentType 
which seems clearer to describe what the variable/argument is about.
 2018-03-20 15:13:14 -07:00
Yann Collet
31b54b6eea updated ZSTD_initStaticDDict() prototype 
can also specify dictContentType.
 2018-03-20 14:52:02 -07:00
Nick Terrell
136b9e2392 Fix external sequence corner cases 
* Clear external sequences when we reset the `ZSTD_CCtx`.
* Skip external sequences when a block is too small to compress.
 2018-03-20 14:50:28 -07:00
Yann Collet
353117c5d7 implemented ZSTD_DCtx_loadDictionary*() 
this required updating ZSTD_createDDict_advanced()
to accept a dictContentType parameter (raw, full, auto).
 2018-03-20 13:40:29 -07:00
Yann Collet
451357f37f
Merge pull request #1058 from facebook/cctxParams 
updated CCtxParams API
 2018-03-20 12:36:12 -07:00
Yann Collet
2ed5af0766 merge bmi2 implementation of encodeSequence into zstd_compress.c 2018-03-19 19:10:31 -07:00
Nick Terrell
d19f803a3b Fix window size for 1 worker + flushing 2018-03-19 18:56:39 -07:00
Nick Terrell
24d9edbdd8 Set ldmParams to 0 when disabled 2018-03-19 18:23:54 -07:00