Commit Graph

510 Commits

Author SHA1 Message Date
W. Felix Handte
5622c276e1 Return to Allowing Early Returns in LZ4_compress_generic()
Or: `goto` Considered Harmful

Or: https://xkcd.com/292/
2018-04-06 19:28:08 -04:00
Yann Collet
8c763aa900
Merge pull request #487 from felixhandte/better-obsoletion-comment
Better Describe Functionality of Obsolete Streaming Functions
2018-03-21 14:52:53 -07:00
W. Felix Handte
126f18d3e0 Also Fix a Comment 2018-03-21 11:48:35 -04:00
W. Felix Handte
a3a9b80dff Better Describe Functionality of Obsolete Streaming Functions 2018-03-21 11:39:41 -04:00
Yann Collet
863e24892d fix comment style 2018-03-21 07:07:24 -07:00
W. Felix Handte
b0a18896fe Move LZ4_compress_fast_extState_noReset Declaration to Unstable Section 2018-03-14 15:59:33 -04:00
W. Felix Handte
66b6fbfe6f Restore the Other Old Streaming Functions in a Degraded Fashion 2018-03-14 15:53:10 -04:00
W. Felix Handte
c852f20c39 Switch ALLOC() to ALLOC_AND_ZERO() to Paper Over Existing Uninitialized Read 2018-03-13 17:47:34 -04:00
W. Felix Handte
995756f218 Split lz4CtxLevel into Two Fields 2018-03-13 17:45:09 -04:00
W. Felix Handte
640db34e43 Another Allocation Fail Check 2018-03-13 17:35:44 -04:00
W. Felix Handte
146e676531 Restore LZ4_sizeofStreamState, We Didn't Actually Need to Delete It 2018-03-13 15:42:03 -04:00
W. Felix Handte
2be38a7429 Rename Enums and Add Comment 2018-03-13 15:16:52 -04:00
W. Felix Handte
b8e9c77855 Whitespace Fixes 2018-03-13 13:20:37 -04:00
W. Felix Handte
5149767a1b Add NULL Checks 2018-03-12 18:35:09 -04:00
W. Felix Handte
299f34909a Simpler Ternary Statements 2018-03-12 18:35:09 -04:00
W. Felix Handte
f9fef255a1 Renames and Comment Fixes 2018-03-12 18:34:41 -04:00
W. Felix Handte
1df5d911aa Hoist LZ4F Dictionary Setup into Helper LZ4F_applyCDict() 2018-03-12 16:11:55 -04:00
W. Felix Handte
3ecc1d7a5b Minor Style Fixes 2018-03-12 16:11:44 -04:00
W. Felix Handte
e34716cc3f Preserve currentOffset==0 When Possible 2018-03-12 14:59:28 -04:00
W. Felix Handte
5f8967b983 Specialize _extState() for Clean Ctx Rather Than Calling _safeExtState() 2018-03-12 14:58:43 -04:00
W. Felix Handte
6716325ae8 Remove Switch In Favor of Ternary Statement 2018-03-12 14:58:43 -04:00
W. Felix Handte
b4335a6585 Further Avoid a dictionary==NULL Check 2018-03-12 14:58:43 -04:00
W. Felix Handte
64bcbf400e Optimize Dict Check Condition 2018-03-12 14:58:43 -04:00
W. Felix Handte
b78cf67c96 Move to 4KB Cut-Off 2018-03-12 14:58:43 -04:00
W. Felix Handte
00eadadbfc Reset Table on Inputs Larger than 2KB 2018-03-12 14:58:43 -04:00
W. Felix Handte
d571d0cdba Avoid DictSmall Checks By Strategically Bumping CurrentOffset 2018-03-12 14:58:43 -04:00
W. Felix Handte
1c4601d643 Restore DictIssue Check 2018-03-12 14:58:43 -04:00
W. Felix Handte
d6ed9a7799 Avoid dictionary == NULL Check 2018-03-12 14:58:43 -04:00
W. Felix Handte
efc419a6d4 Replace calloc() Calls With malloc() Where Possible 2018-03-12 14:58:43 -04:00
W. Felix Handte
80790c587b Copy the Dict Table Into the Context for Large Compressions 2018-03-12 14:58:43 -04:00
W. Felix Handte
9dcd9abc14 Make LZ4F_compressFrame_usingCDict Take a Compression Context 2018-03-12 14:58:43 -04:00
W. Felix Handte
14ce912b70 Switch Current Offset to 1 Only When in External Dictionary Context Mode 2018-03-12 14:58:43 -04:00
W. Felix Handte
cea09d67a9 Hoist Table Reset One Level Up 2018-03-12 14:58:43 -04:00
W. Felix Handte
68c6bd17b8 Set Dictionary Context Pointer Rather than Copying the Context In 2018-03-12 14:58:43 -04:00
W. Felix Handte
73cc39327e Lookup Matches in Separate Dictionary Context 2018-03-12 14:58:43 -04:00
W. Felix Handte
62cb52b341 Initialize Current Offset to 1 2018-03-12 14:58:43 -04:00
W. Felix Handte
7060bcabf0 Only Re-Alloc / Reset When Needed When Switching Between Regular and High Compression Modes 2018-03-12 14:58:43 -04:00
W. Felix Handte
b3628cb0c5 Avoid Resetting the Context When Possible 2018-03-12 14:58:43 -04:00
W. Felix Handte
aa36e118f1 Const-ify Table Arg to LZ4_getPosition(OnHash) 2018-03-12 14:58:43 -04:00
W. Felix Handte
d6a3024dbb Add LZ4_compress_fast_safeExtState Function 2018-03-12 14:58:43 -04:00
W. Felix Handte
5709891de6 Add a Table Type Field to LZ4_stream_t 2018-03-12 14:58:43 -04:00
W. Felix Handte
6933f5ad9c Remove Obsolete Stream Functions to Free Space in LZ4_stream_t 2018-03-12 14:58:43 -04:00
W. Felix Handte
6d156fea56 Allow Empty Dictionaries 2018-03-12 14:58:43 -04:00
Yann Collet
b5233d3726 updated LZ4F_compressBound() documentation
to clarify it includes potentially buffered data.
2018-02-27 23:23:27 -08:00
Yann Collet
860ff77909
Merge pull request #478 from lz4/mergeOpt
merge lz4opt.h into lz4hc.c
2018-02-26 14:06:31 -08:00
Yann Collet
39fda9a447 bumped version number to v1.8.2
updated NEWS was current progresses
2018-02-26 13:50:04 -08:00
Yann Collet
ba115386fa update code comment on LZ4 streaming interface
notably regarding LZ4_saveDict() speed advantage,
answering #477.
2018-02-26 13:31:18 -08:00
Yann Collet
550b40849f merge lz4opt.h into lz4hc.c
Having a dedicated file for optimal parser
made sense during its creation,
it allowed Przemyslaw to work more freely on lz4opt, with less dependency on lz4hc,
moreover, the optimal parser was more complex, with its own search functions.

Since the optimal was rewritten last year, it's now a lot lighter.
It makes more sense now to integrate it directly inside lz4hc.c,
making it easier to edit (editors are a bit "lost" inside a `*.h` dependent on its #include position),
it also reduces the number of files in the project,
which fits pretty well with lz4 objectives.
(adding lz4hc requires "just" lz4hc.h and lz4hc.c).
2018-02-25 00:32:09 -08:00
Yann Collet
7173a631db edge case : compress up to end-mflimit (12 bytes)
The LZ4 block format specification
states that the last match must start
at a minimum distance of 12 bytes from the end of the block.

However, out of an abundance of caution,
the reference implementation would actually stop searching matches
at 13 bytes from the end of the block.

This patch fixes this small detail.
The new version is now able to properly compress a limit case
such as `aaaaaaaabaaa\n`
as reported by Gao Xiang (@hsiangkao).

Obviously, it doesn't change a lot of things.
This is just one additional match candidate per block, with a maximum match length of 7 (since last 5 bytes must remain literals).

With default policy, blocks are 4 MB long, so it doesn't happen too often
Compressing silesia.tar at default level 1 saves 5 bytes (100930101 -> 100930096).
At max level 12, it saves a grand 16 bytes (77389871 -> 77389855).

The impact is a bit more visible when blocks are smaller, hence more numerous.
For example, compressing silesia with blocks of 64 KB (using -12 -B4D) saves 543 bytes (77304583 -> 77304040).
So the smaller the packet size, the more visible the impact.

And it happens we have a ton of scenarios with little blocks using LZ4 compression ...

And a useless "hooray" sidenote :
the patch improves the LZ4 compression record of silesia (using -12 -B7D --no-frame-crc) by 16 bytes (77270672 -> 77270656)
and the record on enwik9 by 44 bytes (371680396 -> 371680352) (previously claimed by [smallz4](http://create.stephan-brumme.com/smallz4/) ).
2018-02-24 11:47:53 -08:00
Yann Collet
71e16fa11a
Merge pull request #471 from lz4/fasterHC
Faster HC
2018-02-20 21:04:07 -08:00