b4eda8d08f
I noticed that LZ4_decompress_generic is sometimes instantiated with
identical set of parameters, or (what's worse) with a subtly different
sets of parameters. For example, LZ4_decompress_fast_withPrefix64k is
instantiated as follows:
return LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize,
full, 0, withPrefix64k, (BYTE*)dest - 64 KB, NULL, 64 KB);
while the equivalent withPrefix64k call in LZ4_decompress_usingDict_generic
passes 0 for the last argument instead of 64 KB. It turns out that there
is no difference in this case: if you change 64 KB to 0 KB in
LZ4_decompress_fast_withPrefix64k, you get the same binary code.
Moreover, because it's been clarified that LZ4_decompress_fast doesn't
check match offsets, it is now obvious that both of these fast/withPrefix64k
instantiations are simply redundant. Exactly because LZ4_decompress_fast
doesn't check offsets, it serves well with any prefixed dictionary.
There's a difference, though, with LZ4_decompress_safe_withPrefix64k.
It also passes 64 KB as the last argument, and if you change that to 0,
as in LZ4_decompress_usingDict_generic, you get a completely different
binary code. It seems that passing 0 enables offset checking:
const int checkOffset = ((safeDecode) && (dictSize < (int)(64 KB)));
However, the resulting code seems to run a bit faster. How come
enabling extra checks can make the code run faster? Curiouser and
curiouser! This needs extra study. Currently I take the view that
the dictSize should be set to non-zero when nothing else will do,
i.e. when passing the external dictionary via dictStart. Otherwise,
lowPrefix betrays just enough information about the dictionary.
* * *
Anyway, with this change, I instantiate all the necessary cases as
functions with distinctive names, which also take fewer arguments and
are therefore less error-prone. I also make the functions non-inline.
(The compiler won't inline the functions because they are used more than
once. Hence I attach LZ4_FORCE_O2_GCC_PPC64LE to the instances while
removing from the callers.) The number of instances is now is reduced
from 18 (safe+fast+partial+4*continue+4*prefix+4*dict+2*prefix64+forceExtDict)
down to 7 (safe+fast+partial+2*prefix+2*dict). The size of the code is
not the only issue here. Separate helper function are much more
amenable to profile-guided optimization: it is enough to profile only
a few basic functions, while the other less-often used functions, such
as LZ4_decompress_*_continue, will benefit automatically.
This is the list of LZ4_decompress* functions in liblz4.so, sorted by size.
Exported functions are marked with a capital T.
$ nm -S lib/liblz4.so |grep -wi T |grep LZ4_decompress |sort -k2
0000000000016260 0000000000000005 T LZ4_decompress_fast_withPrefix64k
0000000000016dc0 0000000000000025 T LZ4_decompress_fast_usingDict
0000000000016d80 0000000000000040 T LZ4_decompress_safe_usingDict
0000000000016d10 000000000000006b T LZ4_decompress_fast_continue
0000000000016c70 000000000000009f T LZ4_decompress_safe_continue
00000000000156c0 000000000000059c T LZ4_decompress_fast
0000000000014a90 00000000000005fa T LZ4_decompress_safe
0000000000015c60 00000000000005fa T LZ4_decompress_safe_withPrefix64k
0000000000002280 00000000000005fa t LZ4_decompress_safe_withSmallPrefix
0000000000015090 000000000000062f T LZ4_decompress_safe_partial
0000000000002880 00000000000008ea t LZ4_decompress_fast_extDict
0000000000016270 0000000000000993 t LZ4_decompress_safe_forceExtDict
|
||
|---|---|---|
| .. | ||
| dll | ||
| .gitignore | ||
| liblz4.pc.in | ||
| LICENSE | ||
| lz4.c | ||
| lz4.h | ||
| lz4frame_static.h | ||
| lz4frame.c | ||
| lz4frame.h | ||
| lz4hc.c | ||
| lz4hc.h | ||
| Makefile | ||
| README.md | ||
| xxhash.c | ||
| xxhash.h | ||
LZ4 - Library Files
The /lib directory contains many files, but depending on project's objectives,
not all of them are necessary.
Minimal LZ4 build
The minimum required is lz4.c and lz4.h,
which provides the fast compression and decompression algorithm.
They generate and decode data using LZ4 block format.
High Compression variant
For more compression ratio at the cost of compression speed,
the High Compression variant called lz4hc is available.
Add files lz4hc.c and lz4hc.h.
The variant still depends on regular lib/lz4.* source files.
Frame variant, for interoperability
In order to produce compressed data compatible with lz4 command line utility,
it's necessary to encode lz4-compressed blocks using the official interoperable frame format.
This format is generated and decoded automatically by the lz4frame library.
Its public API is described in lib/lz4frame.h.
In order to work properly, lz4frame needs all other modules present in /lib,
including, lz4 and lz4hc, and also xxhash.
So it's necessary to include all *.c and *.h files present in /lib.
Advanced / Experimental API
A complex API defined in lz4frame_static.h contains definitions
which are not guaranteed to remain stable in future versions.
As a consequence, it must be used with static linking only.
Windows : using MinGW+MSYS to create DLL
DLL can be created using MinGW+MSYS with the make liblz4 command.
This command creates dll\liblz4.dll and the import library dll\liblz4.lib.
The import library is only required with Visual C++.
The header files lz4.h, lz4hc.h, lz4frame.h and the dynamic library
dll\liblz4.dll are required to compile a project using gcc/MinGW.
The dynamic library has to be added to linking options.
It means that if a project that uses LZ4 consists of a single test-dll.c
file it should be linked with dll\liblz4.dll. For example:
gcc $(CFLAGS) -Iinclude/ test-dll.c -o test-dll dll\liblz4.dll
The compiled executable will require LZ4 DLL which is available at dll\liblz4.dll.
Miscellaneous
Other files present in the directory are not source code. There are :
LICENSE: contains the BSD license textMakefile:makescript to compile and install lz4 library (static and dynamic)liblz4.pc.in: forpkg-config(used inmake install)README.md: this file
License
All source material within lib directory are BSD 2-Clause licensed. See LICENSE for details. The license is also reminded at the top of each source file.