AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

value time

Browse Source
This commit is contained in:
Yann Collet 2015-10-28 14:05:37 +01:00
parent 1077bb8141
commit 10ba1834cc
2 changed files with 18 additions and 13 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
README.md
View File

@ -1,4 +1,4 @@
**Zstd**, short for Zstandard, is a new lossless compression algorithm, which provides both good compression ratio _and_ speed for your standard compression needs. "Standard" translates into everyday situations which neither look for highest possible ratio nor extreme speed.
**Zstd**, short for Zstandard, is a fast lossless compression algorithm, targeting real-time compression scenarios, and featuring zlib-level compression ratio.
It is provided as a BSD-license package, hosted on Github.
@ -7,15 +7,13 @@ It is provided as a BSD-license package, hosted on Github.
|master | [![Build Status](https://travis-ci.org/Cyan4973/zstd.svg?branch=master)](https://travis-ci.org/Cyan4973/zstd) |
|dev | [![Build Status](https://travis-ci.org/Cyan4973/zstd.svg?branch=dev)](https://travis-ci.org/Cyan4973/zstd) |
For a taste of its performance, here are a few benchmark numbers, completed on a Core i7-5600U @ 2.6 GHz, using [fsbench 0.14.3](http://encode.ru/threads/1371-Filesystem-benchmark?p=34029&viewfull=1#post34029), an open-source benchmark program by m^2.
For a taste of its performance, here are a few benchmark numbers from a number of compression codecs suitable for real-time. The test was completed on a Core i7-5600U @ 2.6 GHz, using [fsbench 0.14.3](http://encode.ru/threads/1371-Filesystem-benchmark?p=34029&viewfull=1#post34029), an open-source benchmark program by m^2.
|Name | Ratio | C.speed | D.speed |
|----------------|-------|--------:|--------:|
| | | MB/s | MB/s |
| [zlib 1.2.8] -6| 3.099 | 21 | 320 |
| **zstd 0.2** |**2.871**|**255**| **670** |
| [zlib 1.2.8] -1| 2.730 | 70 | 300 |
| [LZ4] HC r131 | 2.720 | 25 | 2100 |
| QuickLZ 1.5.1b6| 2.237 | 370 | 415 |
| LZO 2.06 | 2.106 | 400 | 580 |
| [LZ4] r131 | 2.101 | 450 | 2100 |
@ -25,10 +23,8 @@ For a taste of its performance, here are a few benchmark numbers, completed on a
[zlib 1.2.8]:http://www.zlib.net/
[LZ4]:http://www.lz4.org/
An interesting feature of zstd is that it can qualify as both a reasonably strong compressor and a fast one.
Zstd delivers very high decompression speed, at more than >600 MB/s per core.
Obviously, your exact mileage will vary depending on your target system.
Zstd strong feature is its very high decompression speed, at more than >600 MB/s per core.
Obviously, your exact mileage will vary depending on target system.
Zstd compression speed will be configurable to fit different situations.
The first available version is the fast one, at ~250 MB/s per core, which is suitable for a few real-time scenarios.
@ -42,6 +38,7 @@ Another property zstd is developed for is configurable memory requirement, with
Zstd entropy stage is provided by [Huff0 and FSE, from Finite State Entrop library](https://github.com/Cyan4973/FiniteStateEntropy).
Zstd has not yet reached "stable" status. Specifically, it doesn't guarantee yet that its current compressed format will remain stable and supported in future versions. It may still change to adapt further optimizations still being investigated. However, the library starts to be pretty robust, able to withstand hazards situations, including invalid input. The library reliability has been tested using [Fuzz Testing](https://en.wikipedia.org/wiki/Fuzz_testing), using both [internal tools](programs/fuzzer.c) and [external ones](http://lcamtuf.coredump.cx/afl). Therefore, you can now safely test zstd, even within production environments.
"Stable Format" is projected sometimes early 2016.

18
programs/paramgrill.c
View File

@ -452,8 +452,6 @@ static void BMK_printWinners2(FILE* f, const winnerInfo_t* winners, size_t srcSi
for (cLevel=0; cLevel <= ZSTD_HC_MAX_CLEVEL; cLevel++)
BMK_printWinner(f, cLevel, winners[cLevel].result, winners[cLevel].params, srcSize);
fflush(f);
}
@ -461,6 +459,7 @@ static void BMK_printWinners(FILE* f, const winnerInfo_t* winners, size_t srcSiz
{
fseek(f, 0, SEEK_SET);
BMK_printWinners2(f, winners, srcSize);
fflush(f);
BMK_printWinners2(stdout, winners, srcSize);
}
@ -480,7 +479,7 @@ static int BMK_seed(winnerInfo_t* winners, ZSTD_HC_parameters params,
if ( (testResult.cSpeed > g_cSpeedTarget[cLevel]) && (winners[cLevel].result.cSize==0) )
better = 1; /* first solution for this cLevel */
if ( (testResult.cSpeed > g_cSpeedTarget[cLevel]) && (testResult.cSize < winners[cLevel].result.cSize ))
if ( (testResult.cSpeed > g_cSpeedTarget[cLevel]) && ((double)testResult.cSize <= (double)winners[cLevel].result.cSize * 1.001 ))
{
/* Validate solution is "good enough" */
double W_ratioNote = log ( (double)srcSize / testResult.cSize);
@ -495,8 +494,17 @@ static int BMK_seed(winnerInfo_t* winners, ZSTD_HC_parameters params,
double W_CMemUsed_note = W_ratioNote * ( 35 + 5*cLevel) - log((double)W_CMemUsed);
double O_CMemUsed_note = O_ratioNote * ( 35 + 5*cLevel) - log((double)O_CMemUsed);
double W_CSpeed_note = W_ratioNote * ( 20 + 3*cLevel) + log((double)testResult.cSpeed);
double O_CSpeed_note = O_ratioNote * ( 20 + 3*cLevel) + log((double)winners[cLevel].result.cSpeed);
double W_DSpeed_note = W_ratioNote * ( 10 + cLevel) + log((double)testResult.dSpeed);
double O_DSpeed_note = O_ratioNote * ( 10 + cLevel) + log((double)winners[cLevel].result.dSpeed);
if (W_DMemUsed_note < O_DMemUsed_note) continue; /* uses too much Decompression memory for too little benefit */
if (W_CMemUsed_note < O_CMemUsed_note) continue; /* uses too much memory for compression for too little benefit */
if (W_CSpeed_note < O_CSpeed_note ) continue; /* too large compression speed difference for the compression benefit */
if (W_DSpeed_note < O_DSpeed_note ) continue; /* too large decompression speed difference for the compression benefit */
better = 1;
}
@ -652,7 +660,7 @@ static void BMK_benchMem(void* srcBuffer, size_t srcSize)
}
/* establish speed objectives (relative to level 1) */
for (i=2; i<ZSTD_HC_MAX_CLEVEL; i++) /* note : last level no speed limit */
for (i=2; i<=ZSTD_HC_MAX_CLEVEL; i++)
g_cSpeedTarget[i] = (g_cSpeedTarget[i-1] * 13) >> 4;
/* populate initial solution */
@ -886,7 +894,7 @@ int main(int argc, char** argv)
break;
}
/* target level9 objective, in MB/s */
/* target level1 speed objective, in MB/s */
case 'T':
argument++;
g_target = 0;