AuroraMiddleware/lz4
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
1,977 Commits 13 Branches 31 Tags 6.8 MiB
e953474464
Commit Graph

171 Commits

Author SHA1 Message Date
qiuyangs
660d21272e
lz4hc.c: change (length >> 8) to (length / 255) 
Every 0xff byte in the compressed block corresponds to a length of 255 (not 256) in the input data. For long repeating sequences, using (length >> 8) may generate bad compressed blocks.
 2019-01-06 16:29:30 +08:00
Bing Xu
17f5071e72 Enable amalgamation of lz4hc.c and lz4.c 2018-11-15 22:24:25 -08:00
Oleg Khabinov
f27ea0774e Adding information about dirty context for _HC_ family of functions 2018-10-10 10:33:04 -07:00
Yann Collet
8bea19d57c fixed minor cppcheck warnings in lib 2018-09-18 15:51:26 -07:00
Yann Collet
86023f01f2 avoid final trailing comma for enum lists 
as detected in #485 by @JoachimSchneider.

Refactored the c_standards tests
so that these issues get automatically detected in CI tests.
 2018-09-13 14:29:41 -07:00
Yann Collet
30f6f34328 removed one assert() condition 
which is not correct when using LZ4_HC with dictionary and starting from a low address (<0x10000).
 2018-09-05 11:25:10 -07:00
Yann Collet
2e4847c2d5 fixed #560 
it was a fairly complex scenario,
involving source files > 64K
and some extraordinary conditions related to specific layout of ranges of zeroes.
and only on level 9.
 2018-09-04 18:21:40 -07:00
Yann Collet
ba1c7148a5 renamed variable for clarity 2018-05-07 12:14:26 -07:00
Yann Collet
200b2960d5 fixed minor conversion warning 2018-05-06 18:26:14 -07:00
Yann Collet
24b9c485db small PA optimization 
which measurably improves speed
on levels 9+
 2018-05-06 16:53:33 -07:00
Yann Collet
cdb0275b7f lz4hc: fixed PA / SC parameter order 
also :
reserved PA for levels 9+ (instead of 8+).
In most cases, speed is lower, and compression benefit is not worth.
 2018-05-05 14:32:57 -07:00
Yann Collet
a4e918d7a6 lz4hc: SC only enabled for opt parser 
the trade off is not good for regular HC parser :
compression is a little bit better, but speed cost is too large in comparison.
 2018-05-05 14:25:37 -07:00
Yann Collet
d097bf93f8 fixed SC.opt integration with regular HC parser 
Only enabled when searching forward.

note : it slighly improves compression ratio,
but measurably decreases speed.
Trade-off to analyse.
 2018-05-05 13:46:45 -07:00
Yann Collet
fa89a9e18b lz4hc: fixed performance issue 
when combining both PA and CS optimizations
 2018-05-05 13:31:03 -07:00
Yann Collet
9699ba5ddf integrated chain swapper into HC match finder 
slower than expected
Pattern analyzer and Chain Swapper
work slower when both activated.
Reasons unclear.
 2018-05-04 19:13:33 -07:00
Yann Collet
434ace7244 implemented search accelerator 
greatly improves speed compared to non-accelerated,
especially for slower files.

On my laptop, -b12 :
```
calgary.tar :  4.3 MB/s =>  9.0 MB/s
enwik7      : 10.2 MB/s => 13.3 MB/s
silesia.tar :  4.0 MB/s =>  8.7 MB/s
```

Note : this is the simplified version,
without handling dictionaries, external buffer, nor pattern analyzer.
Current `dev` branch on these samples gives :
```
calgary.tar :  4.2 MB/s
enwik7      :  9.7 MB/s
silesia.tar :  3.5 MB/s
```

interestingly, it's slower,
presumably due to handling of dictionaries.
 2018-05-03 16:31:41 -07:00
Yann Collet
dc42707107 created LZ4HC_FindLongestMatch() 
simplified match finder
only searching forward and within current buffer,
for easier testing of optimizations.
 2018-05-03 15:38:32 -07:00
Yann Collet
85be6b8f6d increased nbAttempts for lz4 -12 
shaves one more kilobyte from silesia.tar
 2018-05-02 14:22:35 -07:00
Yann Collet
bd470ccd38
Merge pull request #521 from lz4/BD_deterministic 
fix lz4hc -BD non-determinism
 2018-04-30 20:40:34 -07:00
Cyan4973
6a7d501fed renamed variable for clarity 
lowLimit -> lowestMatchIndex
 2018-04-30 18:56:16 -07:00
Yann Collet
8c574990a9 lz4hc changed variable 
to reduce confusion
dictLowLimit => dictStart
 2018-04-30 16:08:16 -07:00
Yann Collet
1e6ca25af3
Merge pull request #520 from felixhandte/frame-dict-nits 
Minor Fixes to Dictionary Preparation in LZ4 Frame
 2018-04-27 13:52:30 -07:00
Yann Collet
de7b274d99 Merge branch 'dev' into BD_deterministic 2018-04-27 12:59:20 -07:00
Yann Collet
19b1267d44 fix lz4hc -BD non-determinism 
related to chain table update
 2018-04-27 12:46:49 -07:00
Yann Collet
72e99c8939 lz4hc : minor editions for clarity 2018-04-27 12:28:58 -07:00
W. Felix Handte
fefc40fc0a Avoid Possibly Redundant Table Clears When Loading HC Dict 2018-04-27 14:10:27 -04:00
Yann Collet
d294dd7fc6 ensure favorDecSpeed is properly initialized 
also :
- fix a potential malloc error
- proper use of ALLOC macro inside lz4hc
- update html API doc
 2018-04-27 09:04:09 -07:00
Yann Collet
0fb3a3b199 fixed a number of minor cast warnings 2018-04-26 18:08:28 -07:00
Yann Collet
5c7d3812d9 fasterDecSpeed can be triggered from cli with --favor-decSpeed 2018-04-26 15:49:32 -07:00
Yann Collet
3792d00168 favorDecSpeed feature can be triggered from lz4frame 
and lz4hc.
 2018-04-26 15:18:44 -07:00
Yann Collet
1148173c5d introduced ability to parse for decompression speed 
triggered through an enum.

Now, it's still necessary to properly expose this capability
all the way up to the cli.
 2018-04-26 13:01:59 -07:00
W. Felix Handte
5ed1463bf4 Remove Debug Log Statements 2018-04-24 11:58:51 -04:00
W. Felix Handte
ee67f25576 Change vLimit Calculation 2018-04-20 20:18:30 -04:00
W. Felix Handte
1895fa19a4 Remove Redundant Static Assert 2018-04-20 20:14:12 -04:00
W. Felix Handte
fcc99d1f31 Simpler loadDict() Reset 2018-04-20 19:37:28 -04:00
W. Felix Handte
a8cb2feffd Tolerate Base Pointer Underflow 2018-04-20 19:37:07 -04:00
W. Felix Handte
85cac61dd8 Don't Segfault on Malloc Failure 2018-04-20 19:35:51 -04:00
W. Felix Handte
756ed402da Sign-Extend -1 to Pointer Width 2018-04-20 17:56:26 -04:00
W. Felix Handte
86b381e40b Fix Constant Value 2018-04-20 17:13:40 -04:00
W. Felix Handte
1d2500d44e Handle Index Underflows Safely 2018-04-20 17:13:03 -04:00
W. Felix Handte
7874cf06b3 Consts and Asserts and Other Minor Nits 2018-04-20 15:30:08 -04:00
W. Felix Handte
d7347f9eea Add API for Attaching Dictionaries 2018-04-20 14:59:34 -04:00
W. Felix Handte
ca833f928f Also Reset the Chain Table 2018-04-20 14:16:27 -04:00
W. Felix Handte
8f118cf6e9 Remove inputBuffer from Context, Work Around its Absence 2018-04-20 14:08:06 -04:00
W. Felix Handte
0064e8ebc7 Remove Commented Out Support for Match Continuation over Segment Boundary 2018-04-20 13:14:37 -04:00
W. Felix Handte
14c577d4c9 Fix Signedness of Comparison 2018-04-19 20:54:35 -04:00
W. Felix Handte
f4b13e17ea Don't Clear the Dictionary Context Until No Longer Useful 2018-04-19 20:54:35 -04:00
W. Felix Handte
0abc23f72e Copy DictCtx into Working Context on Inputs Larger than 4 KB 2018-04-19 20:54:35 -04:00
W. Felix Handte
b67de2a327 Force Inline on HashChain 2018-04-19 20:54:35 -04:00
W. Felix Handte
22e16d5b50 Split DictCtx-using Code Into Separate Inlining Chain 2018-04-19 20:54:35 -04:00