2014-04-15 13:03:17 +00:00
|
|
|
LZ4 - Extremely fast compression
|
|
|
|
|
================================
|
|
|
|
|
|
2016-02-12 22:34:07 +00:00
|
|
|
LZ4 is lossless compression algorithm,
|
|
|
|
|
providing compression speed at 400 MB/s per core,
|
|
|
|
|
scalable with multi-cores CPU.
|
|
|
|
|
It features an extremely fast decoder,
|
|
|
|
|
with speed in multiple GB/s per core,
|
2015-03-31 08:44:56 +00:00
|
|
|
typically reaching RAM speed limits on multi-core systems.
|
|
|
|
|
|
2015-05-04 13:43:37 +00:00
|
|
|
Speed can be tuned dynamically, selecting an "acceleration" factor
|
|
|
|
|
which trades compression ratio for more speed up.
|
|
|
|
|
On the other end, a high compression derivative, LZ4_HC, is also provided,
|
|
|
|
|
trading CPU time for improved compression ratio.
|
2015-10-20 08:46:18 +00:00
|
|
|
All versions feature the same decompression speed.
|
|
|
|
|
|
2016-11-02 02:14:04 +00:00
|
|
|
LZ4 library is provided as open-source software using BSD 2-Clause license.
|
2015-05-04 13:43:37 +00:00
|
|
|
|
2014-04-15 13:03:17 +00:00
|
|
|
|
2014-05-03 01:49:14 +00:00
|
|
|
|Branch |Status |
|
|
|
|
|
|------------|---------|
|
2015-05-29 17:50:12 +00:00
|
|
|
|master | [![Build Status][travisMasterBadge]][travisLink] [![Build status][AppveyorMasterBadge]][AppveyorLink] [![coverity][coverBadge]][coverlink] |
|
|
|
|
|
|dev | [![Build Status][travisDevBadge]][travisLink] [![Build status][AppveyorDevBadge]][AppveyorLink] |
|
|
|
|
|
|
2016-11-03 17:52:03 +00:00
|
|
|
[travisMasterBadge]: https://travis-ci.org/lz4/lz4.svg?branch=master "Continuous Integration test suite"
|
|
|
|
|
[travisDevBadge]: https://travis-ci.org/lz4/lz4.svg?branch=dev "Continuous Integration test suite"
|
|
|
|
|
[travisLink]: https://travis-ci.org/lz4/lz4
|
2016-11-15 20:58:53 +00:00
|
|
|
[AppveyorMasterBadge]: https://ci.appveyor.com/api/projects/status/jc2yhgwyc7qqtsko/branch/master?svg=true "Windows test suite"
|
|
|
|
|
[AppveyorDevBadge]: https://ci.appveyor.com/api/projects/status/jc2yhgwyc7qqtsko/branch/dev?svg=true "Windows test suite"
|
|
|
|
|
[AppveyorLink]: https://ci.appveyor.com/project/YannCollet/lz4-1lndh
|
2015-05-29 17:50:12 +00:00
|
|
|
[coverBadge]: https://scan.coverity.com/projects/4735/badge.svg "Static code analysis of Master branch"
|
|
|
|
|
[coverlink]: https://scan.coverity.com/projects/4735
|
2014-04-15 13:03:17 +00:00
|
|
|
|
2015-03-16 10:26:49 +00:00
|
|
|
> **Branch Policy:**
|
2014-04-15 13:03:17 +00:00
|
|
|
|
2015-03-16 10:26:49 +00:00
|
|
|
> - The "master" branch is considered stable, at all times.
|
2015-04-13 19:43:54 +00:00
|
|
|
> - The "dev" branch is the one where all contributions must be merged
|
2015-03-31 08:44:56 +00:00
|
|
|
before being promoted to master.
|
2015-04-13 19:43:54 +00:00
|
|
|
> + If you plan to propose a patch, please commit into the "dev" branch,
|
|
|
|
|
or its own feature branch.
|
2015-03-31 08:44:56 +00:00
|
|
|
Direct commit to "master" are not permitted.
|
2014-04-15 13:03:17 +00:00
|
|
|
|
|
|
|
|
Benchmarks
|
|
|
|
|
-------------------------
|
|
|
|
|
|
2016-11-16 19:36:45 +00:00
|
|
|
The benchmark uses [lzbench], from @inikep
|
|
|
|
|
compiled with GCC v6.2.0 on Linux 64-bits.
|
|
|
|
|
The reference system uses a Core i7-3930K CPU @ 4.5GHz.
|
2015-03-31 08:44:56 +00:00
|
|
|
Benchmark evaluates the compression of reference [Silesia Corpus]
|
|
|
|
|
in single-thread mode.
|
2014-04-15 13:03:17 +00:00
|
|
|
|
2016-11-16 19:36:45 +00:00
|
|
|
[lzbench]: https://github.com/inikep/lzbench
|
|
|
|
|
[Silesia Corpus]: http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia
|
|
|
|
|
|
|
|
|
|
| Compressor | Ratio | Compression | Decompression |
|
|
|
|
|
| ---------- | ----- | ----------- | ------------- |
|
|
|
|
|
| memcpy | 1.000 | 7300 MB/s | 7300 MB/s |
|
|
|
|
|
|**LZ4 fast 8 (v1.7.3)**| 1.799 |**911 MB/s** | **3360 MB/s** |
|
|
|
|
|
|**LZ4 default (v1.7.3)**|**2.101**|**625 MB/s** | **3220 MB/s** |
|
|
|
|
|
| LZO 2.09 | 2.108 | 620 MB/s | 845 MB/s |
|
|
|
|
|
| QuickLZ 1.5.0 | 2.238 | 510 MB/s | 600 MB/s |
|
|
|
|
|
| Snappy 1.1.3 | 2.091 | 450 MB/s | 1550 MB/s |
|
|
|
|
|
| LZF v3.6 | 2.073 | 365 MB/s | 820 MB/s |
|
|
|
|
|
| [Zstandard] 1.1.1 -1 | 2.876 | 330 MB/s | 930 MB/s |
|
|
|
|
|
| [Zstandard] 1.1.1 -3 | 3.164 | 200 MB/s | 810 MB/s |
|
|
|
|
|
| [zlib] deflate 1.2.8 -1| 2.730 | 100 MB/s | 370 MB/s |
|
|
|
|
|
|**LZ4 HC -9 (v1.7.3)** |**2.720**| 34 MB/s | **3240 MB/s** |
|
|
|
|
|
| [zlib] deflate 1.2.8 -6| 3.099 | 33 MB/s | 390 MB/s |
|
2016-02-25 13:49:40 +00:00
|
|
|
|
|
|
|
|
[zlib]: http://www.zlib.net/
|
|
|
|
|
[Zstandard]: http://www.zstd.net/
|
2015-05-04 13:43:37 +00:00
|
|
|
|
2016-11-19 01:54:26 +00:00
|
|
|
LZ4 is also compatible and well optimized for x32 mode, for which it provides +10% speed performance.
|
|
|
|
|
|
2015-05-04 13:43:37 +00:00
|
|
|
|
|
|
|
|
Documentation
|
|
|
|
|
-------------------------
|
2015-03-30 17:32:21 +00:00
|
|
|
|
2015-04-06 00:03:49 +00:00
|
|
|
The raw LZ4 block compression format is detailed within [lz4_Block_format].
|
2015-03-26 18:49:15 +00:00
|
|
|
|
2015-05-04 13:43:37 +00:00
|
|
|
To compress an arbitrarily long file or data stream, multiple blocks are required.
|
2015-04-06 00:03:49 +00:00
|
|
|
Organizing these blocks and providing a common header format to handle their content
|
|
|
|
|
is the purpose of the Frame format, defined into [lz4_Frame_format].
|
2015-05-04 13:43:37 +00:00
|
|
|
Interoperable versions of LZ4 must respect this frame format.
|
|
|
|
|
|
2016-11-16 19:36:45 +00:00
|
|
|
[lz4_Block_format]: doc/lz4_Block_format.md
|
|
|
|
|
[lz4_Frame_format]: doc/lz4_Frame_format.md
|
|
|
|
|
|
2015-05-04 13:43:37 +00:00
|
|
|
|
|
|
|
|
Other source versions
|
|
|
|
|
-------------------------
|
|
|
|
|
|
2016-02-12 22:34:07 +00:00
|
|
|
Beyond the C reference source,
|
2015-05-29 19:36:59 +00:00
|
|
|
many contributors have created versions of lz4 in multiple languages
|
|
|
|
|
(Java, C#, Python, Perl, Ruby, etc.).
|
|
|
|
|
A list of known source ports is maintained on the [LZ4 Homepage].
|
2015-05-04 13:43:37 +00:00
|
|
|
|
2015-10-20 08:46:18 +00:00
|
|
|
[LZ4 Homepage]: http://www.lz4.org
|
