Commit Graph

2354 Commits

Author SHA1 Message Date
W. Felix Handte
ace9cfa950 Attach Dicts when Using ZSTD_btlazy2 2018-06-13 16:06:28 -04:00
W. Felix Handte
d53200a846 Fix Cast Warning 2018-06-13 14:58:36 -04:00
W. Felix Handte
b82063b266 Extend Dictionary Matches Backwards 2018-06-13 14:58:36 -04:00
W. Felix Handte
d53a04211c Update Dictionary Attachment Cutoff Values Again 2018-06-13 14:58:36 -04:00
W. Felix Handte
2162aa9f18 Do Not Inline DMS Search Function 2018-06-13 14:58:36 -04:00
W. Felix Handte
338bede9b5 Also Implement Depth Repcode Checks 2018-06-13 14:58:36 -04:00
W. Felix Handte
555ab9f8cf Apply Match Continuation Bug Fix 2018-06-13 14:58:36 -04:00
W. Felix Handte
c87dd2121d Update Dictionary Attachment Cutoff Values 2018-06-13 14:58:36 -04:00
W. Felix Handte
6204b6d592 Check Dict Match State in ZSTD_HcFindBestMatch_generic 2018-06-13 14:58:36 -04:00
W. Felix Handte
211a61b69b Focus on Non-BT Impls for the Moment 2018-06-13 14:58:36 -04:00
W. Felix Handte
2e93736a77 Remove Pre-Existing Repcode Check 2018-06-13 14:58:36 -04:00
W. Felix Handte
3b82a23a35 Second Repcode Check 2018-06-13 14:58:36 -04:00
W. Felix Handte
a2a24bebec First Repcode Check 2018-06-13 14:58:36 -04:00
W. Felix Handte
f74c2cd673 Disallow Too-Long Repcodes When Using an Attached Dict 2018-06-13 14:58:36 -04:00
W. Felix Handte
c14db94450 Rename base -> prefixLowest 2018-06-13 14:58:36 -04:00
W. Felix Handte
5d90708a0a Go Back to Separate Intermediate Functions for Different Dict Modes 2018-06-13 14:58:36 -04:00
W. Felix Handte
f84fc63a43 Further Templatize Intermediate Functions on dictMode 2018-06-13 14:58:36 -04:00
W. Felix Handte
529d3a5acd Convert Existing U32 extDict Vars to ZSTD_dictMode Enums 2018-06-13 14:58:36 -04:00
W. Felix Handte
33e2240fac Attach Dict When Using ZSTD_lazy Strategies 2018-06-13 14:58:36 -04:00
W. Felix Handte
90cfc799e5 Add _dictMatchState Stubs for ZSTD_lazy Functions 2018-06-13 14:58:36 -04:00
W. Felix Handte
a85ecb32bd Add dictMode Param to ZSTD_compressBlock_lazy_generic 2018-06-13 14:58:36 -04:00
Yann Collet
750ee87a92
Merge pull request #1175 from ryandesign/macos
Fix name of macOS
2018-06-13 11:32:06 -04:00
Yann Collet
b2632bcf6c
Merge pull request #1174 from duc0/document_default_level
Expose ZSTD_CLEVEL_DEFAULT and update documentation
2018-06-12 12:09:01 -07:00
Duc Ngo
869e2718f6 Line break 2018-06-11 10:02:15 -07:00
Duc Ngo
e8ef725e13 Address comments 2018-06-11 10:01:35 -07:00
Ryan Schmidt
b567ce9d68 Fix name of macOS 2018-06-09 14:31:17 -05:00
Duc Ngo
e34c000e44 Expose ZSTD_CLEVEL_DEFAULT and update documentation 2018-06-08 11:33:44 -07:00
Yann Collet
3050733042 Merge branch 'dev' into negLevels 2018-06-07 15:51:35 -07:00
Yann Collet
c2c47e24e0 support targetlen==0 with strategy==ZSTD_fast
to mean "normal compression",
targetlen >= 1 now means "disable huffman compression of literals"
2018-06-07 15:49:01 -07:00
Yann Collet
a57b4df85f removed literalCompression directive
in this version, literal compression is always disabled for ZSTD_fast strategy.

Performance parity between ZSTD_compress_advanced() and ZSTD_compress_generic()
2018-06-07 15:24:12 -07:00
Yann Collet
8537bfd85c fuzzer: make negative compression level fail
result of ZSTD_compress_advanced()
is different from ZSTD_compress_generic()
when using negative compression levels
because the disabling of huffman compression is not passed in parameters.
2018-06-07 15:12:13 -07:00
Yann Collet
8ef75547ef
Merge pull request #1165 from facebook/ctxSizeDown
Dynamic context downsize
2018-06-07 14:44:32 -07:00
Yann Collet
e3c42c739b clean ZSTD_compress() initialization
The (pretty old) code inside ZSTD_compress()
was making some pretty bold assumptions
on what's inside a CCtx and how to init it.

This is pretty fragile by design.
CCtx content evolve.
Knowledge of how to handle that should be concentrate in one place.

A side effect of this strategy
is that ZSTD_compress() wouldn't check for BMI2 capability,
and is therefore missing out some potential speed opportunity.

This patch makes ZSTD_compress() use
the same initialization and release functions
as the normal creator / destructor ones.

Measured on my laptop, with a custom version of bench
manually modified to use ZSTD_compress() (instead of the advanced API) :
This patch :
 1#silesia.tar       : 211984896 ->  73651053 (2.878), 312.2 MB/s , 723.8 MB/s
 2#silesia.tar       : 211984896 ->  70163650 (3.021), 226.2 MB/s , 649.8 MB/s
 3#silesia.tar       : 211984896 ->  66996749 (3.164), 169.4 MB/s , 636.7 MB/s
 4#silesia.tar       : 211984896 ->  65998319 (3.212), 136.7 MB/s , 619.2 MB/s
dev branch :
 1#silesia.tar       : 211984896 ->  73651053 (2.878), 291.7 MB/s , 727.5 MB/s
 2#silesia.tar       : 211984896 ->  70163650 (3.021), 216.2 MB/s , 655.7 MB/s
 3#silesia.tar       : 211984896 ->  66996749 (3.164), 162.2 MB/s , 633.1 MB/s
 4#silesia.tar       : 211984896 ->  65998319 (3.212), 130.6 MB/s , 618.6 MB/s
2018-06-07 14:05:25 -07:00
Yann Collet
b27c7389e3
Merge pull request #1164 from GeorgeLu97/CustomMacros
Partial Compilation Macros
2018-06-06 16:47:42 -07:00
Yann Collet
24319975b6 bumped version number to v1.3.5 2018-06-06 15:51:55 -07:00
Yann Collet
f1ea383f45 context can be sized down even with constant parameters
when parameters are "equivalent",
the context is re-used in continue mode,
hence needed workspace size is not recalculated.
This incidentally also evades the size-down check and action.

This patch intercepts the "continue mode"
so that the size-down check and action is actually triggered.
2018-06-06 15:04:12 -07:00
Yann Collet
e5e17d009f changed member name to workSpaceOversizedDuration 2018-06-06 15:00:27 -07:00
Yann Collet
f7392f3dc9 added test case 2018-06-05 14:53:28 -07:00
George Lu
11d5bfdaa9 Revert "Partial compilation test?"
This reverts commit b2496ab606.
2018-06-05 13:55:36 -07:00
George Lu
b2496ab606 Partial compilation test? 2018-06-05 13:24:00 -07:00
Yann Collet
3d523c741b added workSpaceTooLarge and workSpaceWasteful
also :
slightly increased speed of test fuzzer.16
2018-06-05 11:42:48 -07:00
George Lu
b3ef314830 Fix Typos 2018-06-04 17:19:06 -07:00
Yann Collet
357c648c3f changed a few variable names
to unify naming convention
2018-06-04 17:10:50 -07:00
George Lu
609d72b0ca Added Deprecated Dependencies 2018-06-04 14:33:21 -07:00
George Lu
9437021d2f Remove old file declaration 2018-06-04 13:32:41 -07:00
George Lu
6a617d70ed Documentation 2018-06-04 09:56:37 -07:00
George Lu
65de25a463 Created Macros 2018-06-04 09:56:29 -07:00
Yann Collet
2108decb41 Fixed a nasty corruption bug
recently introduce into the new dictionary mode.
The bug could be reproduced with this command :
./zstreamtest -v --opaqueapi --no-big-tests -s4092 -t639

error was in function ZSTD_count_2segments() :
the beginning of the 2nd segment corresponds to prefixStart
and not the beginning of the current block (istart == src).
This would result in comparing the wrong byte.
2018-06-01 18:54:34 -07:00
Yann Collet
143fc9ff6c
Merge pull request #1157 from facebook/decompressedSize
minor : improved zstd.h API code comment
2018-06-01 10:28:17 -07:00
Yann Collet
7c33b48221
Merge pull request #1151 from felixhandte/zstd-dfast-in-place-dict-goto
ZSTD_dfast: Support Searching the Dictionary Context In-Place (Alternate `goto` Implementation)
2018-05-31 17:37:09 -07:00