AuroraMiddleware/lz4
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
1,842 Commits 13 Branches 31 Tags 6.8 MiB
674eac3325
Commit Graph

124 Commits

Author SHA1 Message Date
Jennifer Liu
e778db373b Fixed bugs about incorrect acceleration calculation and benchmarking negative compresion level 2018-06-27 13:36:38 -07:00
Yann Collet
d7b6c726ed small extDict : fixed side-effect 
don't fix dictionaries of size 0.
setting dictEnd == source triggers prefix mode,
thus removing possibility to use CDict.
 2018-05-05 19:59:00 -07:00
Yann Collet
af12733467 fixed frametest error 
The error can be reproduced using following command :
./frametest -v -i100000000 -s1659 -t31096808

It's actually a bug in the stream LZ4 API,
when starting a new stream
and providing a first chunk to complete with size < MINMATCH.
In which case, the chunk becomes a dictionary.
No hash was generated and stored,
but the chunk is accessible as default position 0 points to dictStart,
and position 0 is still within MAX_DISTANCE.
Then, next attempt to read 32-bits from position 0 fails.

The issue would have been mitigated by starting from index 64 KB,
effectively eliminating position 0 as too far away.

The proper fix is to eliminate such "dictionary" as too small.
Which is what this patch does.
 2018-05-05 18:24:11 -07:00
Yann Collet
f3e84ffd41
Merge pull request #529 from felixhandte/lz4f-fast-reset-for-streaming-only 
LZ4F: Only Reset the LZ4_stream_t when Init'ing a Streaming Block
 2018-05-03 15:37:51 -07:00
W. Felix Handte
5406c2e479 Only Reset the LZ4 Stream when Init'ing a Streaming Block 2018-05-03 00:03:20 -04:00
Yann Collet
c25eb16666 random lz4f clarifications 
the initial intention was to update lz4f ring buffer strategy,
but lz4f doesn't use ring buffer.
Instead, it uses the destination buffer as much as possible,
and merely copies just what's required to preserve history
into its own buffer, at the end.
Pretty efficient.

This patch just clarifies a few comments and add some assert().
It's built on top of #528.

It also updates doc.
 2018-05-02 16:05:42 -07:00
Yann Collet
1e6ca25af3
Merge pull request #520 from felixhandte/frame-dict-nits 
Minor Fixes to Dictionary Preparation in LZ4 Frame
 2018-04-27 13:52:30 -07:00
Yann Collet
47d70e755e
Merge pull request #519 from lz4/fdParser 
Faster decoding speed
 2018-04-27 11:46:29 -07:00
W. Felix Handte
fefc40fc0a Avoid Possibly Redundant Table Clears When Loading HC Dict 2018-04-27 14:10:27 -04:00
W. Felix Handte
5076aa3e35 Remove Redundant LZ4_resetStream() Call 2018-04-27 13:59:02 -04:00
W. Felix Handte
7d11e34413 Rename LZ4F_applyCDict() -> LZ4F_initStream() 2018-04-27 13:57:10 -04:00
Yann Collet
0fb3a3b199 fixed a number of minor cast warnings 2018-04-26 18:08:28 -07:00
Yann Collet
5c7d3812d9 fasterDecSpeed can be triggered from cli with --favor-decSpeed 2018-04-26 15:49:32 -07:00
Yann Collet
3792d00168 favorDecSpeed feature can be triggered from lz4frame 
and lz4hc.
 2018-04-26 15:18:44 -07:00
W. Felix Handte
a2edeac201 Limit Dictionary Size During LZ4F Decompression 
Fixes lz4/lz4#517.
 2018-04-26 17:18:40 -04:00
W. Felix Handte
2dfc7cbe82 Change Over Includes in the Project 2018-04-24 16:22:28 -04:00
W. Felix Handte
5ed1463bf4 Remove Debug Log Statements 2018-04-24 11:58:51 -04:00
W. Felix Handte
d7347f9eea Add API for Attaching Dictionaries 2018-04-20 14:59:34 -04:00
W. Felix Handte
0a2abacd90 Use Fast Reset in LZ4F Again 2018-04-19 20:54:35 -04:00
W. Felix Handte
61c7ceffed Use Fast Reset API in LZ4F 2018-04-19 20:54:35 -04:00
W. Felix Handte
6289ff4fb1 Call LZ4F_applyCDict Even on NULL CDict 2018-04-19 20:54:35 -04:00
W. Felix Handte
fdeead0b09 Set dictCtx Rather than memcpy'ing Ctx 2018-04-19 20:54:35 -04:00
W. Felix Handte
e75153f508 Add Debug Log Statements to HC 2018-04-19 20:54:35 -04:00
W. Felix Handte
51a56c47c0 Minor Fixes 2018-04-11 18:06:48 -04:00
W. Felix Handte
3a0c571272 Add a LZ4_STATIC_LINKING_ONLY Macro to Guard Experimental APIs 2018-04-11 18:06:10 -04:00
W. Felix Handte
afa52c9b95 Expose dictCtx Functionality in LZ4 2018-04-11 16:28:56 -04:00
W. Felix Handte
21f0c9700b Rename _extState_noReset -> _extState_fastReset and Edit Comments 2018-04-11 15:13:01 -04:00
W. Felix Handte
59c7d95121 Expose a Faster Stream Reset Function 2018-04-10 13:26:17 -04:00
Yann Collet
863e24892d fix comment style 2018-03-21 07:07:24 -07:00
W. Felix Handte
c852f20c39 Switch ALLOC() to ALLOC_AND_ZERO() to Paper Over Existing Uninitialized Read 2018-03-13 17:47:34 -04:00
W. Felix Handte
995756f218 Split lz4CtxLevel into Two Fields 2018-03-13 17:45:09 -04:00
W. Felix Handte
640db34e43 Another Allocation Fail Check 2018-03-13 17:35:44 -04:00
W. Felix Handte
f9fef255a1 Renames and Comment Fixes 2018-03-12 18:34:41 -04:00
W. Felix Handte
1df5d911aa Hoist LZ4F Dictionary Setup into Helper LZ4F_applyCDict() 2018-03-12 16:11:55 -04:00
W. Felix Handte
3ecc1d7a5b Minor Style Fixes 2018-03-12 16:11:44 -04:00
W. Felix Handte
efc419a6d4 Replace calloc() Calls With malloc() Where Possible 2018-03-12 14:58:43 -04:00
W. Felix Handte
9dcd9abc14 Make LZ4F_compressFrame_usingCDict Take a Compression Context 2018-03-12 14:58:43 -04:00
W. Felix Handte
68c6bd17b8 Set Dictionary Context Pointer Rather than Copying the Context In 2018-03-12 14:58:43 -04:00
W. Felix Handte
7060bcabf0 Only Re-Alloc / Reset When Needed When Switching Between Regular and High Compression Modes 2018-03-12 14:58:43 -04:00
W. Felix Handte
b3628cb0c5 Avoid Resetting the Context When Possible 2018-03-12 14:58:43 -04:00
Yann Collet
87fb7a1d03 refactored frameCompress example 
to better reflect LZ4F API usage.
 2018-01-31 14:33:16 -08:00
Yann Collet
5e7780d2d8 lz4frame : removed some intermediate stage from LZ4F_decompress() 
ensure some strange jump cases are not possible
(they were already not possible, but static analyzer couldn't understand it).
 2018-01-14 00:15:07 -08:00
Yann Collet
18b4c66d25 ensure a ptr is non-null 
with an assert()
to help static analyzer understanding this condition.
 2018-01-13 22:47:46 -08:00
Yann Collet
4d61ebc9c8 modified formulation for LZ4F_compressBound() 
previous version used an intentional overflow,
which is defined since it uses unsigned type,
but static analyzer complain about it.
 2018-01-13 22:39:39 -08:00
Nick Terrell
c2dd686e96 [lz4f] Skip memcpy() on empty dictionary 2018-01-05 14:30:49 -08:00
Yann Collet
89821ac7a1 minor comment edit 2017-11-03 11:49:56 -07:00
Yann Collet
87968517f9 fixed decoding block checksum in lz4frame 2017-10-04 15:24:08 -07:00
Yann Collet
ceb868f442 minor lz4frame code refactor 
try to improve code readability.
minor optimization on condition to preserve history.
 2017-09-23 15:06:24 -07:00
W. Felix Handte
77f7245ea7 Fix Segfault When Copying Dict 
dctx must have been initialized before we can copy the dictionary in.
 2017-09-22 13:20:49 -07:00
Yann Collet
c10863b98e fix : asan error in dctx, due to increased maximum frame header size, reported by Craig Young 2017-08-17 10:55:30 -07:00