AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
2,103 Commits 1 Branch 0 Tags 27 MiB
d82efd8a70
Commit Graph

2103 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Terrell
d82efd8a70 ZSTD_compress_usingDict() when dict gets loaded 
Specify that when `dict == NULL || dictSize < 8` no dictionary
gets loaded.
Also add some periods.
 2016-11-02 18:07:16 -07:00
Yann Collet
0a5a5fb7fd Fix #418 : printing selected segments in zdict debug mode can segfault with certain pathological patterns 2016-11-02 13:57:55 -07:00
Yann Collet
4c0b44f8ce minor display improvement in fuzzer when nb of tests is very large 2016-11-01 11:13:22 -07:00
Yann Collet
3d197d4c8c updated man page 2016-10-31 18:10:32 -07:00
Yann Collet
fe92398918 updated README, mentioning .zst suffix 2016-10-31 15:48:58 -07:00
Yann Collet
5a62e0d015 Merge pull request #436 from inikep/dev11 
gen-zstd-manual.sh
 2016-10-31 10:41:13 -07:00
Przemyslaw Skibinski
f882e132cf added gen-zstd-manual.sh 2016-10-31 10:51:58 +01:00
Przemyslaw Skibinski
1fd5b45c6d updated doc/zstd_manual.html 2016-10-31 10:44:44 +01:00
Przemyslaw Skibinski
33d751999b Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-10-31 10:43:07 +01:00
Yann Collet
31e660e7aa more accurate default maximum window size 2016-10-29 03:56:45 -07:00
Yann Collet
03982b5ddb Merge pull request #434 from terrelln/dev 
Pzstd Improvements
 2016-10-28 20:27:59 -07:00
Yann Collet
589f011761 changed ZBUFF_* by ZSTD_*Stream() within fullbench, 
in anticipation of future deprecation of ZBUFF_*.
 2016-10-28 15:17:38 -07:00
Yann Collet
b9550d6d43 fixed benchmark mode typo . 
added benchmark mode tests
 2016-10-28 14:43:24 -07:00
Nick Terrell
25086d9bc6 [pzstd] Move -I flags to PZSTD_CPPFLAGS 2016-10-28 14:24:15 -07:00
Yann Collet
22de81e87e updated man page 2016-10-28 13:58:31 -07:00
Yann Collet
6887b4b1c1 Merge pull request #433 from inikep/dev11 
Dev11
 2016-10-28 11:47:35 -07:00
Przemyslaw Skibinski
81c334b024 test-zstd-speed.py: default timeout changed to 1800 seconds 2016-10-28 20:40:21 +02:00
Yann Collet
7fb30f706e Merge pull request #432 from inikep/dev11 
windres
 2016-10-28 11:30:44 -07:00
Yann Collet
2115724c22 Merge pull request #430 from terrelln/exec-sequences 
ZSTD_execSequence() accepts match in last 7 bytes
 2016-10-28 10:45:05 -07:00
Przemyslaw Skibinski
53e7f5c2e4 test-zstd-speed.py: converted to python3 and added "--timeout" 2016-10-28 19:24:16 +02:00
Przemyslaw Skibinski
bd0bd0fb65 generate RES files during each compilation 2016-10-28 18:15:34 +02:00
Przemyslaw Skibinski
589a8f27b7 generate_res.bat moved to programs/windres 2016-10-28 13:17:35 +02:00
Przemyslaw Skibinski
76d66dffd0 generate_res moved to programs/windres 2016-10-28 13:16:06 +02:00
Przemyslaw Skibinski
86dc7223e9 generate_res.bat: automatic detection if windres.exe is available 2016-10-28 12:20:02 +02:00
Przemyslaw Skibinski
3575fcf28e updated build\VS2010\zstd\generate_res\verrsrc.h 2016-10-28 11:54:47 +02:00
Yann Collet
5f1cb45cff Merge pull request #431 from terrelln/perf-regr 
Fix ZSTD_execSequence() performance regression
 2016-10-27 20:45:52 -07:00
Nick Terrell
10bfd0c0d5 Fix ZSTD_execSequence() performance regression 
Commit ae1cb3b3d0 caused the regression.
It is an instruction alignment issue, because if it is `U64 i` instead
of `U32 i`, the regression returns.  This patch fixes the regression
in gcc, but only gets some of the clang performance back.

Benchmarks:
Run on `silesia.tar`.  I only show levels 1-5 because the performance
regression was uniform across all levels.  I did one run on levels
1-19 and it looked good.

| Build | Level | Before | While | After |
|-------|-------|-------:|------:|------:|
| gcc   |     1 |  931.4 | 904.4 | 932.8 |
| gcc   |     2 |  849.1 | 822.6 | 851.2 |
| gcc   |     3 |  815.6 | 790.6 | 818.9 |
| gcc   |     4 |  794.1 | 770.7 | 798.0 |
| gcc   |     5 |  785.7 | 760.7 | 788.8 |
| clang |     1 |  705.5 | 683.2 | 693.8 |
| clang |     2 |  670.0 | 649.2 | 660.7 |
| clang |     3 |  659.6 | 639.8 | 651.4 |
| clang |     4 |  652.5 | 634.7 | 645.9 |
| clang |     5 |  646.9 | 625.5 | 637.7 |
 2016-10-27 16:19:57 -07:00
Yann Collet
ee5b725823 ZSTD_initCStream() optimization : do not allocate a CDict when no dictionary used 2016-10-27 14:20:55 -07:00
Yann Collet
bfae317ae3 minor const added 2016-10-26 18:44:59 -07:00
Yann Collet
cf08958d9b print one global summary (instead of one per file) 2016-10-26 18:37:29 -07:00
Yann Collet
82613d72e4 added example multiple_streaming_compression 2016-10-26 18:10:43 -07:00
Przemyslaw Skibinski
55b50b0a68 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-10-26 09:39:35 +02:00
Nick Terrell
ac4310d303 [pzstd] Fix test mode for streaming input 2016-10-26 00:09:39 -07:00
Nick Terrell
eb7873a048 ZSTD_execSequence() accepts match in last 7 bytes 
The zstd reference compressor will not emit a match in the last 7
bytes of a block.  The decompressor will also not accept a match
in the last 7 bytes.  This patch makes the decompressor accept a
match in the last 7 bytes.
 2016-10-25 21:24:15 -07:00
Yann Collet
335ad5d4d4 added ZSTD_initDStream_usingDDict() . 
slightly optimized ZSTD_initDStream() when no dictionary .
fixed ZSTD_sizeof_CStream() .
 2016-10-25 17:47:02 -07:00
Yann Collet
9516234e67 first sketch for ZSTD_initCStream_usingCDict() 2016-10-25 16:19:52 -07:00
Yann Collet
62d9a7ddfd Merge pull request #429 from inikep/btopt2 
Btopt2
 2016-10-25 14:48:43 -07:00
Yann Collet
57cca28026 Merge pull request #428 from inikep/dev11 
windres
 2016-10-25 11:34:09 -07:00
Przemyslaw Skibinski
5c5f01f3da added ZSTD_btopt2 strategy 2016-10-25 12:25:07 +02:00
Przemyslaw Skibinski
347903cfb5 Windows resources moved to programs/windres/ 2016-10-25 10:34:41 +02:00
Przemyslaw Skibinski
5fc901d1ce images/ moved to doc/images/ 2016-10-25 10:05:20 +02:00
Yann Collet
7b5948cca7 Merge pull request #426 from terrelln/fixes 
Fix various {A, M}SAN bugs
 2016-10-24 23:42:26 -07:00
Yann Collet
37d130031d updated comments on context re-use 2016-10-24 17:22:12 -07:00
Nick Terrell
b2c39a22b0 Fix compiler narrowing warning 2016-10-24 14:50:13 -07:00
Nick Terrell
f698ad6deb Merge remote-tracking branch 'upstream/dev' into fixes 
* upstream/dev:
  added doc\zstd_manual.html
  added contrib\gen_html
  zstd_compression_format.md moved to doc/
  Fix small bug in ZSTD_execSequence()
  improved ZSTD_compressBlock_opt_extDict_generic
  protect ZSTD_decodeFrameHeader() from invalid usage, as suggested by @spaskob
  zstd_opt.h: small improvement in compression ratio
  improved dicitonary segment merge
  use implicit rules to compile zstd_decompress.c
  detect early impossible decompression scenario in legacy decoder v0.5
  no repeat mode in legacy v0.5
  fixed invalid invocation of dictionary in legacy decoder v0.5
  fix edge case
  fix command line interpretation
  fixed minor corner case
  zstd.h: added the Introduction section
  fixed clang 3.5 warnings
  zstd.h: updated comments
 2016-10-24 13:10:13 -07:00
Yann Collet
4239a207dd Merge pull request #425 from inikep/dev11 
Doc
 2016-10-24 11:11:40 -07:00
Nick Terrell
f9c9af3c2e Reject dictionaries with incomplete entropy tables 
If a dictionary specifies that a symbol has probability zero in its
`matchLength`, `literalLength`, or `offset` FSE table, but the symbol
appears when compressing input, the compressor fails.

Ensure that dictionaries support all `matchLength`, and `literalLength`
codes.  They must also support all of the `offset` codes required to
represent every possible offset that can appear in the first block.
 2016-10-24 10:42:44 -07:00
Przemyslaw Skibinski
86d9424c81 added doc\zstd_manual.html 2016-10-24 16:07:53 +02:00
Przemyslaw Skibinski
984b66cd72 added contrib\gen_html 2016-10-24 15:59:51 +02:00
Przemyslaw Skibinski
3ee94a7600 zstd_compression_format.md moved to doc/ 2016-10-24 15:58:07 +02:00