AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
3,815 Commits 1 Branch 0 Tags 27 MiB
11303778d0
Commit Graph

3815 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Stella Lau
11303778d0 Add function to make cctxParams from ZSTD_parameters 2017-08-22 14:53:13 -07:00
Stella Lau
23fc0e41fa Remove 'opaque' naming from internal functions 2017-08-22 14:24:47 -07:00
Stella Lau
8fd1636776 Remove unused functions 2017-08-22 13:33:58 -07:00
Stella Lau
60e1bc617c Explicitly create a job cctxParam for multithreading 2017-08-21 15:39:37 -07:00
Stella Lau
5b956f4753 Comment out CCtx_param versions of CDict functions 2017-08-21 14:49:16 -07:00
Stella Lau
fd8a25786e Check parameters are valid in initCCtxParams 2017-08-21 13:23:35 -07:00
Stella Lau
1c0dbe81b1 Add documentation for CCtx_params 2017-08-21 13:18:00 -07:00
Stella Lau
939f954285 Pass ZSTD_CCtx_params as const ptr when possible 2017-08-21 12:57:18 -07:00
Stella Lau
73c73bf16a Reduce code duplication in zstreamtest 2017-08-21 12:41:19 -07:00
Stella Lau
560b34f6d2 Return error code when initializing NULL cctxParams 2017-08-21 11:52:26 -07:00
Stella Lau
25be09c6b4 Set some parameters to zero before initializing cdict 2017-08-21 11:35:46 -07:00
Stella Lau
502031ca10 Use cctxParam version of createCDict internally 2017-08-21 11:00:44 -07:00
Stella Lau
91b30dbe84 Remove test parameter 2017-08-21 10:09:06 -07:00
Stella Lau
f181f33bdf Disable tests and refactor 2017-08-21 01:59:08 -07:00
Stella Lau
023b24e6d4 Add cctx param tests 2017-08-20 22:55:07 -07:00
Stella Lau
6cee6e07e5 Add internal createCDict function 2017-08-18 22:48:31 -07:00
Stella Lau
d775519296 Add cctxParam versions of internal functions 2017-08-18 17:37:58 -07:00
Stella Lau
63b8c98531 Pass cctx parameters to MTCtx 2017-08-18 16:17:24 -07:00
Stella Lau
399ae013d4 Add function to apply cctx params 2017-08-18 13:01:55 -07:00
Stella Lau
81d89d82a6 Move nbThreads to cctx params 2017-08-18 12:08:57 -07:00
Stella Lau
2300c58a6f Move dictContentByRef to cctx params 2017-08-18 12:03:16 -07:00
Stella Lau
b6cb2ed8cb Move dictMode to cctxParams 2017-08-18 11:43:31 -07:00
Stella Lau
97e27affcb Move compression level to cctx params 2017-08-18 11:20:08 -07:00
Stella Lau
c0221124d5 Add function to set opaque parameters 2017-08-17 19:30:22 -07:00
Stella Lau
4169f49171 Add initialization/allocation functions for opaque params 2017-08-17 18:45:04 -07:00
Stella Lau
ade95b8bed Add opaque interfaces for static initialization 2017-08-17 18:13:08 -07:00
Stella Lau
699f11b4f7 Create opaque parameter structure 2017-08-17 17:33:46 -07:00
Yann Collet
6466fd3400 Merge pull request #776 from terrelln/dev 
[libzstd] Fix CHECK_V_F() macro
 2017-07-25 14:39:32 -07:00
Nick Terrell
ae20d413da [libzstd] Fix CHECK_V_F macros 2017-07-25 12:52:01 -07:00
Yann Collet
38ba7002f2 fixed minor warning on unused variable in shell function 2017-07-20 18:39:04 -07:00
Yann Collet
a90b16e150 Visual blind fix 2 2017-07-20 15:57:55 -07:00
Yann Collet
4542d7118c Merge branch 'dev' of github.com:facebook/zstd into dev 2017-07-20 15:12:21 -07:00
Yann Collet
5e6c5203f3 fixed fuzzer test for non OS-X platforms 2017-07-20 15:11:56 -07:00
Yann Collet
57251f52cf Merge pull request #773 from terrelln/lk-v3 
[linux] Update patches for v3
 2017-07-20 14:47:26 -07:00
Nick Terrell
7d3ac0710d [linux] Update patches for v3 2017-07-20 13:33:55 -07:00
Yann Collet
b4d460f32c pool.c : blindfix for Visual warnings 2017-07-20 01:13:14 -07:00
Yann Collet
1ca1288689 added --memtest=# command to fuzzer 
to jump directly to relevant test section
 2017-07-19 16:01:16 -07:00
Yann Collet
3974d2b38a blind fix for Windows Multithreading module 
adds a fake 0 return value for mutex/cond init
 2017-07-19 13:33:21 -07:00
Yann Collet
b71363b967 check pthread_*_init() success condition 2017-07-19 01:05:40 -07:00
Yann Collet
44b0838253 Merge pull request #770 from terrelln/test-mode 
[zstdcli] Fix -t in streaming mode
 2017-07-18 15:40:59 -07:00
Yann Collet
2aba13b770 Merge pull request #769 from terrelln/real-block-split 
[libzstd] Fix bug in Huffman encoding
 2017-07-18 14:58:26 -07:00
Nick Terrell
d0b27483ae [zstdcli] Fix -t in streaming mode 2017-07-18 14:45:49 -07:00
Nick Terrell
cc1522351f [libzstd] Fix bug in Huffman encoding 
Summary:
Huffman encoding with a bad dictionary can encode worse than the
HUF_BLOCKBOUND(srcSize), since we don't filter out incompressible
input, and even if we did, the dictionaries Huffman table could be
ill suited to compressing actual data.

The fast optimization doesn't seem to improve compression speed,
even when I hard coded fast = 1, the speed didn't improve over hard coding
it to 0.

Benchmarks:
$ ./zstd.dev -b1e5
Benchmarking levels from 1 to 5
 1#Synthetic 50%     :  10000000 ->   3139163 (3.186), 524.8 MB/s ,1890.0 MB/s
 2#Synthetic 50%     :  10000000 ->   3115138 (3.210), 372.6 MB/s ,1830.2 MB/s
 3#Synthetic 50%     :  10000000 ->   3222672 (3.103), 223.3 MB/s ,1400.2 MB/s
 4#Synthetic 50%     :  10000000 ->   3276678 (3.052), 198.0 MB/s ,1280.1 MB/s
 5#Synthetic 50%     :  10000000 ->   3271570 (3.057), 107.8 MB/s ,1200.0 MB/s
$ ./zstd -b1e5
Benchmarking levels from 1 to 5
 1#Synthetic 50%     :  10000000 ->   3139163 (3.186), 524.8 MB/s ,1870.2 MB/s
 2#Synthetic 50%     :  10000000 ->   3115138 (3.210), 370.0 MB/s ,1810.3 MB/s
 3#Synthetic 50%     :  10000000 ->   3222672 (3.103), 223.3 MB/s ,1380.1 MB/s
 4#Synthetic 50%     :  10000000 ->   3276678 (3.052), 196.1 MB/s ,1270.0 MB/s
 5#Synthetic 50%     :  10000000 ->   3271570 (3.057), 106.8 MB/s ,1180.1 MB/s
$ ./zstd.dev -b1e5 ../silesia.tar
Benchmarking levels from 1 to 5
 1#silesia.tar       : 211988480 ->  73651685 (2.878), 429.7 MB/s ,1096.5 MB/s
 2#silesia.tar       : 211988480 ->  70158785 (3.022), 321.2 MB/s ,1029.1 MB/s
 3#silesia.tar       : 211988480 ->  66993813 (3.164), 243.7 MB/s , 981.4 MB/s
 4#silesia.tar       : 211988480 ->  66306481 (3.197), 226.7 MB/s , 972.4 MB/s
 5#silesia.tar       : 211988480 ->  64757852 (3.274), 150.3 MB/s , 963.6 MB/s
$ ./zstd -b1e5 ../silesia.tar
Benchmarking levels from 1 to 5
 1#silesia.tar       : 211988480 ->  73651685 (2.878), 429.7 MB/s ,1087.1 MB/s
 2#silesia.tar       : 211988480 ->  70158785 (3.022), 318.8 MB/s ,1029.1 MB/s
 3#silesia.tar       : 211988480 ->  66993813 (3.164), 246.5 MB/s , 981.4 MB/s
 4#silesia.tar       : 211988480 ->  66306481 (3.197), 229.2 MB/s , 972.4 MB/s
 5#silesia.tar       : 211988480 ->  64757852 (3.274), 149.3 MB/s , 963.6 MB/s

Test Plan:
I added a test case to the fuzzer which crashed with ASAN before the patch
and succeeded after.
 2017-07-18 13:20:40 -07:00
Yann Collet
77d67fb167 Merge pull request #766 from terrelln/real-block-split 
[libzstd] Pull optimal parser state out of seqStore_t
 2017-07-18 08:26:24 -07:00
Yann Collet
14c83b05c7 Merge pull request #765 from terrelln/real-block-split 
[libzstd] Remove ZSTD_CCtx* argument of ZSTD_compressSequences()
 2017-07-17 19:25:55 -07:00
Nick Terrell
7a28b9e4a3 [libzstd] Pull optimal parser state out of seqStore_t 2017-07-17 15:29:11 -07:00
Yann Collet
3381bf4b84 Merge pull request #764 from terrelln/real-block-split 
[libzstd] Refactor ZSTD_compressSequences()
 2017-07-17 14:46:01 -07:00
Nick Terrell
e198230645 [libzstd] Remove ZSTD_CCtx* argument of ZSTD_compressSequences() 2017-07-17 12:27:24 -07:00
Nick Terrell
634f012420 [libzstd] Refactor ZSTD_compressSequences() 2017-07-17 11:36:11 -07:00
Yann Collet
fa3aa04ccd Merge pull request #761 from paulcruz74/file-rename 
renamed pool.c to poolTests.c
 2017-07-14 09:09:45 -07:00