AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
5,037 Commits 1 Branch 0 Tags 27 MiB
63eeeaa1dd
Commit Graph

5037 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet
63eeeaa1dd update table levels for blocks <= 16K 
also : allow hlog to be slighly larger than windowlog,
as it's apparently good for both speed and compression ratio.
 2018-05-16 16:13:37 -07:00
Yann Collet
c8c67f7c84 Merge branch 'dev' into tableLevels 2018-05-14 11:55:52 -07:00
Yann Collet
174bd3d4a7
Merge pull request #1131 from facebook/zstdcli 
minor: control numeric argument overflow
 2018-05-14 11:53:58 -07:00
Yann Collet
5d76201fee
Merge pull request #1130 from facebook/man 
fix #1115
 2018-05-14 11:52:53 -07:00
Yann Collet
902db38798
Merge pull request #1129 from facebook/paramgrill 
Paramgrill refactoring
 2018-05-14 11:52:41 -07:00
Yann Collet
3870db1ba5 Merge branch 'dev' into tableLevels 2018-05-14 11:52:05 -07:00
Yann Collet
4da0216db0
Merge pull request #1133 from felixhandte/travis-fix 
Make Travis CI Run `apt-get update`
 2018-05-14 09:59:43 -07:00
W. Felix Handte
e26be5a7b3 Travis CI Runs apt-get Update 2018-05-14 11:55:21 -04:00
Yann Collet
2c392952f9 paramgrill: use NB_LEVELS_TRACKED in loop 
make it easier to generate/track more levels
than ZSTD_maxClevel()
 2018-05-13 17:25:53 -07:00
Yann Collet
c9227ee16b update table for 128 KB blocks 2018-05-13 17:15:07 -07:00
Yann Collet
b4250489cf update compression levels for large inputs 2018-05-13 01:53:38 -07:00
Yann Collet
9cd5c63771 cli: control numeric argument overflow 
exit on overflow
backported from paramgrill
added associated test case
 2018-05-12 14:29:33 -07:00
Yann Collet
3f89cd1081 minor : factor out errorOut() 2018-05-12 14:09:32 -07:00
Yann Collet
b824d213cb fix #1115 2018-05-12 10:21:30 -07:00
Yann Collet
50993901b2 paramgrill: subtle change in level spacing 
distance between levels is slightly increased
to compensate for level 1 speed improvements
and the will to have stronger level 19
extending the range of speed to cover.
 2018-05-12 09:40:04 -07:00
Yann Collet
a3f2e84a37 added programmable constraints 2018-05-11 19:43:08 -07:00
Yann Collet
17c19fbbb5 generalized use of readU32FromChar() 
and check input overflow
 2018-05-11 17:32:26 -07:00
Yann Collet
66b81817b5
Merge pull request #1128 from facebook/libdir 
minor Makefile patch
 2018-05-11 11:47:59 -07:00
Yann Collet
3193d692c2 minor patch, ensuring LIBDIR is created before installation 
follow-up from #1123
 2018-05-11 11:31:48 -07:00
Yann Collet
498ab7bb12
Merge pull request #1123 from baruchsiach/fix-install-pc 
lib/Makefile: create include directory before headers installation
 2018-05-10 10:38:19 -07:00
Yann Collet
33dfc54069
Merge pull request #1124 from terrelln/playtests-gnu 
Write to /dev/random for test
 2018-05-09 09:39:02 -07:00
Nick Terrell
2dde9d5aba Write to /dev/random for test 2018-05-08 11:06:01 -07:00
Baruch Siach
9a0643b633 lib/Makefile: create include directory before headers installation 
Make sure that $(INCLUDEDIR) exists before copying the headers there.
Otherwise, the contest of header files is copied over
$(DESTDIR)$(INCLUDEDIR), making it a regular file.

While at it, remove $(DESTDIR)$(INCLUDEDIR) from the list of directories
to create in the install-pc target. The install-pc target does not need
this directory.
 2018-05-08 20:59:44 +03:00
Yann Collet
87125c2c01
Merge pull request #1122 from facebook/fixCDictBlock 
fix ZSTD_compressBlock() associated with ZSTD_CDict
 2018-05-07 17:13:59 -07:00
Yann Collet
ad4524d605 fix ZSTD_compressBlock() associated with CDict 
reported by @let-def.

It's actually a bug in ZSTD_compressBegin_usingCDict()
which would pass a wrong pledgedSrcSize value (0 instead of ZSTD_CONTENTSIZE_UNKNOWN)
resulting in wrong window size, resulting in downsized seqStore,
resulting in segfault when writing into the seqStore later in the process.

Added a test in fuzzer to cover this use case (fails before the patch).
 2018-05-07 12:54:13 -07:00
Yann Collet
ef1abd3c07
Merge pull request #1120 from lamby/897904-libzstd-please-make-the-build-reproducible 
Make the build reproducible
 2018-05-04 12:39:40 -07:00
Chris Lamb
2dbe408a49 Make the build reproducible 
Whilst working on the Reproducible Builds effort [0], we noticed
that zstd could not be built reproducibly.

This is due to the manual page encoding the number of CPUs from the
build machine and thus varies across builds.

This was originally filed in Debian as #897904 [1].

 [0] https://reproducible-builds.org/
 [1] https://bugs.debian.org/897904

Signed-off-by: Chris Lamb <lamby@debian.org>
 2018-05-04 08:39:51 -07:00
Yann Collet
5ec92c4d96
Merge pull request #1118 from terrelln/playtests-typo 
Fix playTests.sh typo
 2018-05-03 14:26:26 -07:00
Nick Terrell
263134ce66 Fix playTests.sh typo 
Thanks to @mestia for reporting and finding the bug!
Fixes #1116.
 2018-05-03 13:03:08 -07:00
cyan4973
b095bffac6 ignore Windows build/test artefacts 2018-05-01 05:45:46 -07:00
Yann Collet
1fc15dfd9d
Merge pull request #1114 from pseiderer/ps-devel-001 
Split library install target into pc, static, shared and include only target
 2018-04-30 14:59:59 -07:00
Yann Collet
82ad249645 Clarifications of Zstandard format specification 
from IETF RFC review
 2018-04-30 12:36:55 -07:00
Peter Seiderer
64bfdca5b9 Split library install target into pc, static, shared and include only target 
Signed-off-by: Peter Seiderer <ps.report@gmx.net>
 2018-04-30 20:32:32 +02:00
Yann Collet
a11f0c67bb
Merge pull request #1113 from terrelln/dict-fix 
Fix parameter adjustment with dictionary
 2018-04-26 10:31:25 -07:00
Nick Terrell
ca77822ddf Fix parameter adjustment with dictionary 
The new advanced API basically set `requestedParams = appliedParams` when
using a dictionary. This halted all parameter adjustment, which can hurt
compression ratio if, for example, the window log is small for the first
call, but the rest of the files are large.

This patch fixes the bug, and checks that the `requestedParams` don't change
in the new advanced API when using a dictionary, and generally in the fuzzer.
 2018-04-25 16:32:29 -07:00
Yann Collet
12f60b8c98 clarified documentation related to refPrefix() 2018-04-25 10:17:06 -07:00
Yann Collet
449cccf6c9
Merge pull request #1112 from facebook/lz4dec 
fixed decoding bogus lz4 frame
 2018-04-24 16:50:54 -07:00
Yann Collet
ace856a835 updated documentation of streaming compression api 2018-04-24 14:44:27 -07:00
cyan4973
62487b5e76 fixed decoding bogus lz4 frame 
FIO would keep presenting data after an LZ4F decoding error
resulting in a NULL pointer dereference
when associated with older liblz4 version (< v1.8.1.2)
 2018-04-23 18:50:16 -07:00
taigacon
2c3ad05812 Fix the problem that enables DYNAMIC_BMI2 macro by mistake on ARM architecture with Clang (#1110) 2018-04-23 15:41:50 -07:00
Yann Collet
9de03cd748
Merge pull request #1107 from terrelln/reset-params 
Add ZSTD_CCtx_resetParameters() function
 2018-04-13 13:23:47 -07:00
Nick Terrell
e8c9dc5cea Fix documentation 2018-04-13 12:43:38 -07:00
Nick Terrell
c0987986e5 Only reset CDict in ZSTD_CCtx_resetParameters() 2018-04-13 11:26:40 -07:00
Nick Terrell
9f76eebd17 Add ZSTD_CCtx_resetParameters() function 
* Fix docs for `ZSTD_CCtx_reset()`.
* Add `ZSTD_CCtx_resetParameters()`.

Fixes #1094.
 2018-04-12 16:54:07 -07:00
Nick Terrell
3c3f59e68f
Enforce pledgeSrcSize whenever known (#1106) 
The test fails before the patch and passes after.

Fixes #1095.
 2018-04-12 16:02:03 -07:00
Yann Collet
1f25b17c7d
Merge pull request #1105 from terrelln/get-param 
Add ZSTD_CCtx(Param)?_getParameter() function
 2018-04-12 13:25:57 -07:00
Nick Terrell
280a236e9e Add ZSTD_CCtx(Param)?_getParameter() function 
Closes #1096.
 2018-04-12 11:50:12 -07:00
Yann Collet
04212178b5 doc : clarified advanced API usage 
sticky parameters only work with `ZSTD_compress_generic()`
 2018-04-10 11:40:36 -07:00
Yann Collet
ad5ba6cdcf updated comment on parameters that can be changed during compression 2018-04-09 17:39:07 -07:00
Yann Collet
1da629f2ad
Merge pull request #1104 from terrelln/fast-train 
Allow negative compression levels in training
 2018-04-09 14:16:20 -07:00