AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
5,253 Commits 1 Branch 0 Tags 27 MiB
3841dbac84
Commit Graph

208 Commits

Author SHA1 Message Date
Sean Purcell
042ba122ae Change g_displayLevel to int and fix DISPLAYUPDATE flush 2017-03-23 11:21:59 -07:00
Yann Collet
c1c040eae1 added gzip tests 
also : made sure zstd --format=gzip -V
would fail if gzip compatibility is not supported
 2017-03-01 16:49:20 -08:00
Przemyslaw Skibinski
74dcd8d15f bench.c: use a single ticksPerSecond 2017-02-21 12:22:05 +01:00
Przemyslaw Skibinski
e052c60540 introduce UTIL_freq_t 2017-02-20 11:27:11 +01:00
Sean Purcell
5069b6c2c3 Merge branch 'dev' into multiframe 2017-02-10 10:08:55 -08:00
Sean Purcell
4e709712e1 Decompressed size functions now handle multiframes and distinguish cases 
- Add ZSTD_findDecompressedSize
    - Traverses multiple frames to find total output size
- Add ZSTD_getFrameContentSize
    - Gets the decompressed size of a single frame by reading header
- Deprecate ZSTD_getDecompressedSize
 2017-02-08 14:50:10 -08:00
Przemyslaw Skibinski
d05014c739 added the "--rt-prio" option 2017-02-07 16:48:01 +01:00
Przemyslaw Skibinski
94abd6a26c SET_REALTIME_PRIORITY 2017-02-07 16:36:19 +01:00
Nick Terrell
83c387eb8e Fix zstdmt_compress.h include 2017-01-26 15:25:32 -08:00
Yann Collet
f8804d1014 convert tabs to space 
joys of using multiple editors from multiple environments ...
 2017-01-20 17:23:19 -08:00
cyan4973
5fba09fa41 updated util's time for Windows compatibility 
Correctly measures time on Posix systems when running with
Multi-threading

Todo : check Windows measurement under multi-threading
 2017-01-20 12:57:31 -08:00
Yann Collet
458c8a94b4 minor refactoring : cleaner MT integration within bench 2017-01-19 17:44:15 -08:00
Yann Collet
500014af49 zstd cli can now compress using multi-threading 
added : command -T#
added : ZSTD_resetCStream() (zstdmt_compress)
added : FIO_setNbThreads()  (fileio)
 2017-01-19 17:04:28 -08:00
Yann Collet
736788f8e8 added streaming fuzzer tests for MT API 
Also : fixed corner case, where nb of jobs completed becomes > jobQueueSize
which is possible when many flushes are issued
while there is not enough dst buffer to flush completed ones.
 2017-01-19 12:15:29 -08:00
Yann Collet
5eb749e734 ZSTDMT_compress() creates a single frame 
The new strategy involves cutting frame at block level.
The result is a single frame, preserving ZSTD_getDecompressedSize()

As a consequence, bench can now make a full round-trip,
since the result is compatible with ZSTD_decompress().

This strategy will not make it possible to decode the frame with multiple threads
since the exact cut between independent blocks is not known.
MT decoding needs further discussions.
 2017-01-11 18:21:25 +01:00
Yann Collet
f1cb55192c fixed linux warnings 2017-01-02 01:11:55 +01:00
Yann Collet
0ec6a95ba1 minor fixes 2017-01-02 00:49:42 +01:00
Yann Collet
c6a6417458 bench correctly measures time for multi-threaded compression (posix only) 2016-12-31 03:31:26 +01:00
Yann Collet
e70912c72b Changed : input divided into roughly equal parts. 
Debug : can measure time waiting for mutexes to unlock.
 2016-12-29 01:24:01 +01:00
Yann Collet
ab7a579180 added -T command , to set nb of threads 2016-12-28 16:11:09 +01:00
Yann Collet
3d93f2fce7 first zstdmt sketch 2016-12-27 07:19:36 +01:00
Yann Collet
8333106b8a Merge branch 'dev' of github.com:facebook/zstd into dev 2016-12-21 16:44:24 +01:00
Yann Collet
1f57c2ed32 added : ZSTD_createCDict_byReference() 2016-12-21 16:20:11 +01:00
Przemyslaw Skibinski
7a8a03c20d util.h: restore BSD license for Facebook Open-Source 2016-12-21 15:08:44 +01:00
Przemyslaw Skibinski
e679741b18 _CRT_SECURE_NO_WARNINGS moved to util.h 2016-12-21 13:47:11 +01:00
Przemyslaw Skibinski
2f6ccee6af platform.h: removed Compiler Options 2016-12-21 13:23:34 +01:00
Przemyslaw Skibinski
16ae6563a2 executables use new util.h and platform.h 2016-12-21 09:06:14 +01:00
Przemyslaw Skibinski
f8046b8e72 Merge remote-tracking branch 'refs/remotes/facebook/dev' into v112 
# Conflicts:
#	appveyor.yml
 2016-12-19 08:20:26 +01:00
Przemyslaw Skibinski
b866e72826 tools use platform.h 2016-12-16 14:24:01 +01:00
Przemyslaw Skibinski
c71e552b2e fixed "strategy" in advanced compression parameters 2016-12-13 20:04:32 +01:00
Przemyslaw Skibinski
897b8bb5eb bench.c: support advanced compression parameters 2016-12-13 13:03:41 +01:00
Yann Collet
e63c631aaf decode benchmark, multi-files 2016-12-06 17:46:49 -08:00
Yann Collet
d946501d2c decode benchmark - single file (hidden option) 2016-12-06 16:49:23 -08:00
Yann Collet
167c494748 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-11-29 14:05:15 -08:00
Yann Collet
4f5350f610 long matches support overflow 2016-11-29 13:12:24 -08:00
Przemyslaw Skibinski
fd0ac93024 bench.c: use ZSTD_maxCLevel() 2016-11-23 21:45:29 +01:00
Przemyslaw Skibinski
5ddcd9d9ae bench.c: fixed MAX_CLEVEL 2016-11-21 16:37:56 +01:00
Przemyslaw Skibinski
2558b4cdbc bench.c without dict uses ZSTD_compressCCtx 2016-11-18 11:46:30 +01:00
Przemyslaw Skibinski
26306fcacf BMK_SetNbIterations renamed to BMK_SetNbSeconds 2016-11-03 11:38:01 +01:00
Yann Collet
7ae67bb18a small compression speed gains with using_CDict 2016-09-06 06:28:05 +02:00
Yann Collet
4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
Yann Collet
87c18b2ebd fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
Yann Collet
0baa64a763 increased maximum memory size for 64-bits bench to 16 GB 2016-08-25 22:54:13 +02:00
Yann Collet
d1733f7417 fixed crc bug in rare timing conditions within bench.c 2016-08-21 01:04:46 +02:00
inikep
5a54870047 fixed Intel Compiler warnings with Visual Studio 
http://encode.ru/threads/2119-Zstandard?p=49504&viewfull=1#post49504
 2016-08-18 09:00:25 +02:00
inikep
7132fb15ba bench.c: removed benchResult_t 2016-08-10 14:59:18 +02:00
Yann Collet
6a21971f4a bench : implemented avgSize 2016-08-03 00:06:24 +02:00
Yann Collet
bf2bc112bb bench : controlled display update when loading lot of files 2016-08-02 23:48:13 +02:00
Yann Collet
de4c04f6c2 Fixed : ZSTD_compress* can compress > 4 GB in a single pass, reported by Nick Terrell 2016-08-02 11:27:05 +02:00
Yann Collet
a9febe81ae changed bench behavior for slow compression levels 2016-08-01 13:40:52 +02:00
Yann Collet
235911e13f removed "avg" evaluation from bench -q 
removed "sleeping" notification from bench -q
 2016-07-31 01:32:48 +02:00
Yann Collet
3c242e79d3 updated compression levels table 2016-07-13 14:56:24 +02:00
Yann Collet
3d2cd7f816 Introduced ZSTD_getParams() 
bench now uses ZSTD_createCDict_advanced()
 2016-06-27 15:12:26 +02:00
Yann Collet
ec224d256d removed useless context 2016-06-27 13:39:30 +02:00
Yann Collet
4c56f4a3cf fixed error messages 2016-06-27 13:36:54 +02:00
inikep
d7d251ccb5 bench.c: added support for ZSTD_GIT_COMMIT 2016-06-22 16:13:25 +02:00
inikep
f2f59d758e test-zstd-speed.py: added ZSTD_GIT_COMMIT 2016-06-22 15:42:26 +02:00
inikep
9bf5357101 bench.c: use ZSTD_VERSION_STRING 2016-06-21 11:01:29 +02:00
Yann Collet
ee1a084852 Integrated new dictionary API into bench module 2016-06-07 01:40:49 +02:00
Yann Collet
2cc72f1fd3 fixed initialization issue in bench 2016-06-06 17:50:07 +02:00
Yann Collet
d3b7f8d21f Merged zstd_static.h into zstd.h . Now requires ZSTD_STATIC_LINKING_ONLY macro 2016-06-04 19:47:02 +02:00
Yann Collet
70d1301d6e Changed ZSTD_adjustCParams() prototype 
`ZSTD_adjustCParams()` is now automatically invoked at the end of `ZSTD_getCParams()`
 2016-06-01 18:45:34 +02:00
inikep
957823f56f zstdcli: -r (operate recursively on directories) works with dictBuilder and compression 2016-05-25 15:30:55 +02:00
Yann Collet
e162aceeb6 minor simplification 2016-05-20 11:35:00 +02:00
inikep
0bdb6a8118 changed definition of UTIL_createFileList 2016-05-13 10:52:02 +02:00
inikep
4dbf7f4a3b dynamic memory allocation in UTIL_createFileList 2016-05-11 14:11:00 +02:00
inikep
95459458ed disable Visual Studio code analysis 2016-05-10 14:42:54 +02:00
inikep
3733797fcd bench.c: experimental -r (operate recursively on directories) for Windows and _POSIX_C_SOURCE >= 200112L 2016-05-10 14:22:55 +02:00
inikep
ed9a08538c Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 
# Conflicts:
#	lib/common/util.h
#	programs/paramgrill.c
#	visual/2013/fullbench/fullbench.vcxproj.filters
#	visual/2013/fuzzer/fuzzer.vcxproj.filters
 2016-05-10 13:20:01 +02:00
inikep
aaaf923a17 common/util.h moved to programs/util.h 
decompress/fse_decompress.c moved to common/
 2016-05-09 16:19:25 +02:00
inikep
13c8424ea0 code cleaning 2016-05-05 13:58:56 +02:00
inikep
0bd0faec32 fixed compatibility issues 2016-05-05 13:10:57 +02:00
inikep
9c22e57bfb Compiler Options moved to util.h 2016-05-05 11:53:42 +02:00
Yann Collet
6f9c056662 Fixed BMI behavior on x86 and x32 targets, reported by @tycho (#178) 2016-05-01 10:26:30 +02:00
inikep
bab4317961 util.h must the the first include to #define _POSIX_C_SOURCE 2016-04-29 15:19:40 +02:00
inikep
55d047aa92 getTotalFileSize moved to common/util.h 2016-04-28 16:50:13 +02:00
inikep
d5ff2c3d9a ordering of #include 2016-04-28 14:40:45 +02:00
inikep
83c76b4594 bench.c: time functions moved to common/util.h 2016-04-28 13:16:01 +02:00
inikep
69fcd7c0ae getFileSize moved to common/util.h 2016-04-28 12:23:33 +02:00
inikep
ea4ee3eee0 added compatibility with Visual C++ 2012 2016-04-25 13:09:06 +02:00
inikep
38654988f3 minor speed improvements 2 
bench.c: block size has to be bigger than 32 bytes
zstdcli.c: support for e.g. -B16k -B16m
 2016-04-21 12:18:47 +02:00
inikep
c5e1d295ee bench.c: force at least one compression and decompression loop 
fix for -i0 with small files
 2016-04-19 09:37:59 +02:00
inikep
6d157f1fbe bench.c: fixed rare compression and decompression speed bug 
concerns only big files with compression or decompression time longer
than 100 seconds
 2016-04-15 16:54:11 +02:00
inikep
4b3c5eed99 removed ZSTD_compressBegin_targetSrcSize 2016-04-14 13:43:51 +02:00
inikep
c0d5f4eb2e bench.c: ignore directories from a file list for benchmark 2016-04-13 10:48:04 +02:00
inikep
8844e21dff Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-04-11 16:49:27 +02:00
Yann Collet
9990986bdc fixed minor warnings when compiling with gcc -g (aka -std=gnu90) 2016-04-09 16:17:18 +02:00
inikep
1eeddde625 clock() is default timer for all platforms except Windows 2016-04-08 16:55:17 +02:00
inikep
19bd48f516 Merge from Cyan4973/dev060 2016-04-04 12:10:00 +02:00
inikep
1c556a3838 ticksPerSecond=0 2016-03-30 10:59:48 +02:00
inikep
33ad251236 removed clock_gettime (not portable) 2016-03-30 09:55:37 +02:00
inikep
4c12f232ec support for QueryPerformanceCounter and clock_gettime 2016-03-29 14:52:13 +02:00
inikep
06f793a3ab fix for BMK_clockSpan on Windows 2016-03-29 11:17:58 +02:00
Yann Collet
b9151406de fixed 32-bits compatibility 2016-03-26 17:18:11 +01:00
Yann Collet
a5b66e34c7 minor variable isolation 
and remove a goto
 2016-03-26 01:48:27 +01:00
inikep
eaba91a161 added ZSTD_compressBegin_targetSrcSize 2016-03-23 20:30:26 +01:00
inikep
7adceef974 hashLog3 added to ZSTD_CCtx 2016-03-23 15:53:38 +01:00
inikep
472638c861 added mili_sleep and setHighPriority 2016-03-23 12:28:28 +01:00
Yann Collet
fd69225086 bench : added HIGH_PRIORTY 2016-03-23 02:47:33 +01:00
Yann Collet
be391438ff first working version with both encoder and decode alternate LL + ML coding scheme. 
decompression speed highly impacted
 2016-03-22 23:19:28 +01:00