W. Felix Handte
bad74c4781
Use Working Ctx Logs when not in DMS Mode
...
We pre-hash the ptr for the dict match state sometimes. When that actually
happens, a hashlog of 0 can produce undefined behavior (right shift a long
long by 64). Only applies to unoptimized compilations, since when
optimizations are applied, those hash operations are dropped when we're not
actually in dms mode.
2018-09-28 17:12:54 -07:00
W. Felix Handte
c38acff94f
When Attaching Dictionary, Size Working Tables Based on Input Size Only
2018-09-28 17:12:54 -07:00
W. Felix Handte
9d87d50878
Remove Log Overriding for the Time Being
2018-09-28 17:12:54 -07:00
W. Felix Handte
77fd17d93f
Remove Strategy-Dependency in Making Attachment Decision
2018-09-28 17:12:54 -07:00
W. Felix Handte
00c088b32d
Support Split Logs in ZSTD_btopt..ZSTD_btultra
2018-09-28 17:12:54 -07:00
W. Felix Handte
0783492178
Bump Split Log Support to ZSTD_btultra
2018-09-28 17:12:54 -07:00
W. Felix Handte
e4ac4a0f16
Support Split Logs in ZSTD_greedy..ZSTD_btlazy2
2018-09-28 17:12:54 -07:00
W. Felix Handte
e710dc3369
Bump Split Log Support to ZSTD_btlazy2
2018-09-28 17:12:54 -07:00
W. Felix Handte
22fcb8d4c7
Support Split Logs in ZSTD_dfast
2018-09-28 17:12:54 -07:00
W. Felix Handte
a232b3bb7c
Bump Split Log Support to ZSTD_dfast
2018-09-28 17:12:54 -07:00
W. Felix Handte
fe96e98f81
Support a Separate Hash Log in ZSTD_fast
2018-09-28 17:12:54 -07:00
W. Felix Handte
bc880ebe8f
Stop Passing in hashLog and stepSize to ZSTD_compressBlock_fast_generic
2018-09-28 17:12:54 -07:00
W. Felix Handte
b3107c7799
Temporary Commit to Retain Requested Hash and Chain Logs During Dict Attach
2018-09-28 17:12:54 -07:00
W. Felix Handte
34e0193129
Allow Setting Hash and Chain Logs on Contexts with Attached CDict
2018-09-28 17:12:54 -07:00
W. Felix Handte
eae8232f50
For Supported Strategies, Attach Dict Even When Params Don't Match
2018-09-28 17:12:54 -07:00
W. Felix Handte
01ff945eae
Split Attach and Copy Reset Strategies into Separate Implementation Functions
2018-09-28 17:12:54 -07:00
W. Felix Handte
a6d6bbeae1
Pull Attachment Decision into Separate Function
2018-09-28 17:12:54 -07:00
W. Felix Handte
b7fba599ae
And Then Avoid the Unused Parameter Warning
2018-09-28 17:12:54 -07:00
W. Felix Handte
1f188ae655
Move Asserts into Function to Avoid Unused Function Warning
2018-09-28 17:12:54 -07:00
W. Felix Handte
7212b5e5c2
Move Match State CParams Setting into resetCCtx and continueCCtx
2018-09-28 17:12:54 -07:00
W. Felix Handte
01e34d365b
Strengthen Assertion to Assert Equality
2018-09-28 17:12:53 -07:00
W. Felix Handte
50cc1cf4d5
Remove CParams Arg from ZSTD_ldm_blockCompress
2018-09-28 17:12:53 -07:00
W. Felix Handte
14764de49f
Stop Separately Passing CParams in ZSTD_lazy Internal Functions
2018-09-28 17:12:53 -07:00
W. Felix Handte
97149f22c3
Stop Separately Passing CParams in ZSTD_opt Internal Functions
2018-09-28 17:10:42 -07:00
W. Felix Handte
dcdf437fed
Also Remove CParams from Table Filling Functions' Args
2018-09-28 17:10:42 -07:00
W. Felix Handte
3483f89101
Also Assert Equivalency When Filling MatchState with Prefix
2018-09-28 17:10:42 -07:00
W. Felix Handte
6cb2454646
Remove CParams from Block Compressor Functions' Args
2018-09-28 17:10:42 -07:00
W. Felix Handte
03103269de
Assert ctx and ms cparams Equivalency
2018-09-28 17:10:42 -07:00
W. Felix Handte
4e3ecee9ed
Remove cParams from CDict
2018-09-28 17:10:42 -07:00
W. Felix Handte
76ef87ed9d
Add ZSTD_compressionParameters to ZSTD_matchState_t
2018-09-28 17:10:42 -07:00
Yann Collet
73773c6b6a
fixed legacy compilation tests
...
for some reason, these tests started failing recently on CircleCI
2018-09-27 18:15:14 -07:00
Nick Terrell
a180ea07c4
Restore ZSTD_noCompressBlock() for clarity
2018-09-27 16:06:02 -07:00
Nick Terrell
aec1a3ec58
Change byte to value to avoid a GRUB typedef
2018-09-27 15:24:48 -07:00
Nick Terrell
109bd37474
Include stddef.h for size_t
2018-09-27 15:24:48 -07:00
Nick Terrell
f2d6db45cd
[zstd] Add -Wmissing-prototypes
2018-09-27 15:24:48 -07:00
Yann Collet
2a5cd8535a
Merge pull request #1342 from facebook/fixcatyd
...
fix : huge (>4GB) chain of blocks
2018-09-27 10:20:14 -07:00
Yann Collet
404a7bfed0
moved again overflow correction
...
cannot work from within ZSTD_compressBlock()
2018-09-26 18:06:53 -07:00
Yann Collet
0e2dbac18a
changed overflow correction place
...
keep one in compress_frameChunk(),
so that it's tested at every loop
in case some user simply some large mulit-GB input in a single invocation.
Add one in ZSTD_compressBlock(),
since compressBlock() explicitly skips frameChunk().
2018-09-26 15:35:38 -07:00
Yann Collet
e74eade251
Merge pull request #1339 from facebook/grep_colors
...
fixed usage of grep in Makefile
2018-09-26 14:39:20 -07:00
Yann Collet
8883af6a1e
Merge pull request #1327 from facebook/adapt
...
Adaptive compression
2018-09-26 14:39:08 -07:00
Yann Collet
f98c69d77c
fix : huge (>4GB) stream of blocks
...
experimental function ZSTD_compressBlock() is designed for very small data in mind,
for situation where saving the ~12 bytes of frame header can actually make a difference.
Some systems though may have to deal with small and large data entangled.
If it's larger than a block (> 128KB), compressBlock() cannot compress them in one round.
That's why it's possible to compress in multiple rounds.
This is a chain of compressed blocks.
Some users push this capability to the limit, encoding gigantic chain of blocks.
On crossing the 4GB limit, some internal overflow occurs.
This fix moves the overflow correction mechanism higher in the call chain,
so that it's applied also to gigantic chains of blocks.
Added a test case in fuzzer.c, which crashes before the fix, and pass now.
2018-09-26 14:24:28 -07:00
Yann Collet
8ff17a6a09
Merge pull request #1329 from facebook/v04isout
...
Changed default legacy support to v0.5+
2018-09-26 13:39:05 -07:00
Yann Collet
08f68d83c5
fixed usage of grep in Makefile
...
when terminal uses colors
as suggested by @danielshir (#1294 )
2018-09-25 16:56:53 -07:00
Yann Collet
04f47bbdd2
Merge branch 'dev' into adapt
2018-09-24 16:56:45 -07:00
Yann Collet
9bb6c15f79
Merge pull request #1332 from facebook/minclevel
...
defined a minimum negative level
2018-09-24 16:01:13 -07:00
Yann Collet
292d8e4a83
added some tests based on limits.h
...
in order to ensure proper type mapping
when not using stdint.h
2018-09-23 23:57:30 -07:00
Yann Collet
71a5210617
avoid recompiling dll every time under mingw
2018-09-21 17:40:30 -07:00
Yann Collet
c484345a82
Merge branch 'mingw' into adapt
2018-09-21 16:00:46 -07:00
Yann Collet
bfff4f4809
ensure all writes to job->cSize are mutex protected
...
even when reporting errors,
using a macro for code brevity, as suggested by @terrelln,
2018-09-21 16:00:39 -07:00
Yann Collet
32b7cf1bcf
fixed tautological tests
...
involving ZSTD_TARGETLENGTH_MIN (== 0)
2018-09-21 15:04:43 -07:00
