AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
3,722 Commits 1 Branch 0 Tags 27 MiB
7758ed8458
Commit Graph

167 Commits

Author SHA1 Message Date
Yann Collet
7bd1a2900e added ZSTD_dictMode_e to control dictionary loading mode 2017-06-21 11:50:33 -07:00
Yann Collet
c08e649e95 first implementation of bench.c with new API ZSTD_compress_generic() 
Doesn't speed optimize this buffer-to-buffer scenario yet.
Still internally defers to streaming implementation.

Also : fixed a long standing bug in ZSTDMT streaming API.
 2017-06-19 18:25:35 -07:00
Yann Collet
fe234bf48b fix attempts : fullbench for VS2008 2017-06-19 15:23:19 -07:00
Yann Collet
31533bacce Changed ZSTD_createCDict_advanced() 
It now only uses compressionParameters as argument.
It produces many changes throughout user code,
though hopefully they tend to be simple :
just provide the cParams part from existing ZSTD_parameters.

Some programs might depend on ZSTD_createCDict_advanced() to pass frame parameters.
This change will force them to revisit this strategy and fix it,
since frame parameters are effectively silently ignored in current version.
 2017-04-27 00:29:04 -07:00
Sean Purcell
42bac7fa84 Change ifndef's to undef's 2017-04-13 15:35:05 -07:00
Sean Purcell
f876f1200c Fix compilation on macOS 2017-04-13 12:33:45 -07:00
Yann Collet
c2007388a5 fixed bench.c : optional advanced parameters applied 
before creating cdict
 2017-04-04 15:35:06 -07:00
Yann Collet
81d6380139 minor bench.c adjustments 
shorter debug messages
no need to check decompressedLength==0 twice
 2017-04-04 15:21:09 -07:00
Yann Collet
5bde4be5a1 fix : bench automatically adapts parameters to srcSize 2017-03-29 12:10:38 -07:00
Sean Purcell
042ba122ae Change g_displayLevel to int and fix DISPLAYUPDATE flush 2017-03-23 11:21:59 -07:00
Yann Collet
c1c040eae1 added gzip tests 
also : made sure zstd --format=gzip -V
would fail if gzip compatibility is not supported
 2017-03-01 16:49:20 -08:00
Przemyslaw Skibinski
74dcd8d15f bench.c: use a single ticksPerSecond 2017-02-21 12:22:05 +01:00
Przemyslaw Skibinski
e052c60540 introduce UTIL_freq_t 2017-02-20 11:27:11 +01:00
Sean Purcell
5069b6c2c3 Merge branch 'dev' into multiframe 2017-02-10 10:08:55 -08:00
Sean Purcell
4e709712e1 Decompressed size functions now handle multiframes and distinguish cases 
- Add ZSTD_findDecompressedSize
    - Traverses multiple frames to find total output size
- Add ZSTD_getFrameContentSize
    - Gets the decompressed size of a single frame by reading header
- Deprecate ZSTD_getDecompressedSize
 2017-02-08 14:50:10 -08:00
Przemyslaw Skibinski
d05014c739 added the "--rt-prio" option 2017-02-07 16:48:01 +01:00
Przemyslaw Skibinski
94abd6a26c SET_REALTIME_PRIORITY 2017-02-07 16:36:19 +01:00
Nick Terrell
83c387eb8e Fix zstdmt_compress.h include 2017-01-26 15:25:32 -08:00
Yann Collet
f8804d1014 convert tabs to space 
joys of using multiple editors from multiple environments ...
 2017-01-20 17:23:19 -08:00
cyan4973
5fba09fa41 updated util's time for Windows compatibility 
Correctly measures time on Posix systems when running with
Multi-threading

Todo : check Windows measurement under multi-threading
 2017-01-20 12:57:31 -08:00
Yann Collet
458c8a94b4 minor refactoring : cleaner MT integration within bench 2017-01-19 17:44:15 -08:00
Yann Collet
500014af49 zstd cli can now compress using multi-threading 
added : command -T#
added : ZSTD_resetCStream() (zstdmt_compress)
added : FIO_setNbThreads()  (fileio)
 2017-01-19 17:04:28 -08:00
Yann Collet
736788f8e8 added streaming fuzzer tests for MT API 
Also : fixed corner case, where nb of jobs completed becomes > jobQueueSize
which is possible when many flushes are issued
while there is not enough dst buffer to flush completed ones.
 2017-01-19 12:15:29 -08:00
Yann Collet
5eb749e734 ZSTDMT_compress() creates a single frame 
The new strategy involves cutting frame at block level.
The result is a single frame, preserving ZSTD_getDecompressedSize()

As a consequence, bench can now make a full round-trip,
since the result is compatible with ZSTD_decompress().

This strategy will not make it possible to decode the frame with multiple threads
since the exact cut between independent blocks is not known.
MT decoding needs further discussions.
 2017-01-11 18:21:25 +01:00
Yann Collet
f1cb55192c fixed linux warnings 2017-01-02 01:11:55 +01:00
Yann Collet
0ec6a95ba1 minor fixes 2017-01-02 00:49:42 +01:00
Yann Collet
c6a6417458 bench correctly measures time for multi-threaded compression (posix only) 2016-12-31 03:31:26 +01:00
Yann Collet
e70912c72b Changed : input divided into roughly equal parts. 
Debug : can measure time waiting for mutexes to unlock.
 2016-12-29 01:24:01 +01:00
Yann Collet
ab7a579180 added -T command , to set nb of threads 2016-12-28 16:11:09 +01:00
Yann Collet
3d93f2fce7 first zstdmt sketch 2016-12-27 07:19:36 +01:00
Yann Collet
8333106b8a Merge branch 'dev' of github.com:facebook/zstd into dev 2016-12-21 16:44:24 +01:00
Yann Collet
1f57c2ed32 added : ZSTD_createCDict_byReference() 2016-12-21 16:20:11 +01:00
Przemyslaw Skibinski
7a8a03c20d util.h: restore BSD license for Facebook Open-Source 2016-12-21 15:08:44 +01:00
Przemyslaw Skibinski
e679741b18 _CRT_SECURE_NO_WARNINGS moved to util.h 2016-12-21 13:47:11 +01:00
Przemyslaw Skibinski
2f6ccee6af platform.h: removed Compiler Options 2016-12-21 13:23:34 +01:00
Przemyslaw Skibinski
16ae6563a2 executables use new util.h and platform.h 2016-12-21 09:06:14 +01:00
Przemyslaw Skibinski
f8046b8e72 Merge remote-tracking branch 'refs/remotes/facebook/dev' into v112 
# Conflicts:
#	appveyor.yml
 2016-12-19 08:20:26 +01:00
Przemyslaw Skibinski
b866e72826 tools use platform.h 2016-12-16 14:24:01 +01:00
Przemyslaw Skibinski
c71e552b2e fixed "strategy" in advanced compression parameters 2016-12-13 20:04:32 +01:00
Przemyslaw Skibinski
897b8bb5eb bench.c: support advanced compression parameters 2016-12-13 13:03:41 +01:00
Yann Collet
e63c631aaf decode benchmark, multi-files 2016-12-06 17:46:49 -08:00
Yann Collet
d946501d2c decode benchmark - single file (hidden option) 2016-12-06 16:49:23 -08:00
Yann Collet
167c494748 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-11-29 14:05:15 -08:00
Yann Collet
4f5350f610 long matches support overflow 2016-11-29 13:12:24 -08:00
Przemyslaw Skibinski
fd0ac93024 bench.c: use ZSTD_maxCLevel() 2016-11-23 21:45:29 +01:00
Przemyslaw Skibinski
5ddcd9d9ae bench.c: fixed MAX_CLEVEL 2016-11-21 16:37:56 +01:00
Przemyslaw Skibinski
2558b4cdbc bench.c without dict uses ZSTD_compressCCtx 2016-11-18 11:46:30 +01:00
Przemyslaw Skibinski
26306fcacf BMK_SetNbIterations renamed to BMK_SetNbSeconds 2016-11-03 11:38:01 +01:00
Yann Collet
7ae67bb18a small compression speed gains with using_CDict 2016-09-06 06:28:05 +02:00
Yann Collet
4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00