Stella Lau
|
024098a47d
|
Fix parameter retrieval from cdict
|
2017-08-25 17:58:28 -07:00 |
|
Stella Lau
|
2adde898c8
|
Fix typo with ZSTDMT_parameter
|
2017-08-25 16:13:40 -07:00 |
|
Stella Lau
|
18224608ff
|
Remove ZSTD_setCCtxParameter()
|
2017-08-25 13:58:41 -07:00 |
|
Stella Lau
|
0744592d38
|
Add function initializing cctxParams from clevel
|
2017-08-25 13:36:47 -07:00 |
|
Stella Lau
|
9911153723
|
Move jobSize and overlapLog in zstdmt to cctxParams
|
2017-08-25 13:14:51 -07:00 |
|
Stella Lau
|
de5193422d
|
Distinguish between jobParams and cctxParams in zstdmt
|
2017-08-25 11:36:17 -07:00 |
|
Stella Lau
|
eb7bbab36a
|
Remove ZSTD_p_refDictContent and dictContentByRef
|
2017-08-25 11:11:45 -07:00 |
|
Stella Lau
|
15fdeb9e41
|
Enforce nbThreads<=1 for estimateCCtxSize
|
2017-08-24 16:28:49 -07:00 |
|
Stella Lau
|
2fbf0285b2
|
Fix interaction with ZSTD_setCCtxParameter() and cleanup
|
2017-08-24 11:25:41 -07:00 |
|
Stella Lau
|
fd9bf42516
|
Fix forceWindow and dictMode setting for zstdmt jobs
|
2017-08-23 19:16:57 -07:00 |
|
Stella Lau
|
bf3108fb50
|
Ensure zstdmt uses 'job version' of cctx parameters
|
2017-08-23 17:03:31 -07:00 |
|
Stella Lau
|
1c81f725ff
|
Remove duplicated testing code
|
2017-08-23 15:47:15 -07:00 |
|
Stella Lau
|
64ce49426b
|
Fix cstream compression level
|
2017-08-23 12:30:47 -07:00 |
|
Stella Lau
|
5bc2c1e982
|
Add prototype support for customMem with cctxParams
|
2017-08-23 12:03:30 -07:00 |
|
Stella Lau
|
6f1a21c7e9
|
Remove formatting-only changes
|
2017-08-23 10:24:19 -07:00 |
|
Stella Lau
|
11303778d0
|
Add function to make cctxParams from ZSTD_parameters
|
2017-08-22 14:53:13 -07:00 |
|
Stella Lau
|
23fc0e41fa
|
Remove 'opaque' naming from internal functions
|
2017-08-22 14:24:47 -07:00 |
|
Stella Lau
|
8fd1636776
|
Remove unused functions
|
2017-08-22 13:33:58 -07:00 |
|
Stella Lau
|
60e1bc617c
|
Explicitly create a job cctxParam for multithreading
|
2017-08-21 15:39:37 -07:00 |
|
Stella Lau
|
5b956f4753
|
Comment out CCtx_param versions of CDict functions
|
2017-08-21 14:49:16 -07:00 |
|
Stella Lau
|
fd8a25786e
|
Check parameters are valid in initCCtxParams
|
2017-08-21 13:23:35 -07:00 |
|
Stella Lau
|
1c0dbe81b1
|
Add documentation for CCtx_params
|
2017-08-21 13:18:00 -07:00 |
|
Stella Lau
|
939f954285
|
Pass ZSTD_CCtx_params as const ptr when possible
|
2017-08-21 12:57:18 -07:00 |
|
Stella Lau
|
73c73bf16a
|
Reduce code duplication in zstreamtest
|
2017-08-21 12:41:19 -07:00 |
|
Stella Lau
|
560b34f6d2
|
Return error code when initializing NULL cctxParams
|
2017-08-21 11:52:26 -07:00 |
|
Stella Lau
|
25be09c6b4
|
Set some parameters to zero before initializing cdict
|
2017-08-21 11:35:46 -07:00 |
|
Stella Lau
|
502031ca10
|
Use cctxParam version of createCDict internally
|
2017-08-21 11:00:44 -07:00 |
|
Stella Lau
|
91b30dbe84
|
Remove test parameter
|
2017-08-21 10:09:06 -07:00 |
|
Stella Lau
|
f181f33bdf
|
Disable tests and refactor
|
2017-08-21 01:59:08 -07:00 |
|
Stella Lau
|
023b24e6d4
|
Add cctx param tests
|
2017-08-20 22:55:07 -07:00 |
|
Stella Lau
|
6cee6e07e5
|
Add internal createCDict function
|
2017-08-18 22:48:31 -07:00 |
|
Stella Lau
|
d775519296
|
Add cctxParam versions of internal functions
|
2017-08-18 17:37:58 -07:00 |
|
Stella Lau
|
63b8c98531
|
Pass cctx parameters to MTCtx
|
2017-08-18 16:17:24 -07:00 |
|
Stella Lau
|
399ae013d4
|
Add function to apply cctx params
|
2017-08-18 13:01:55 -07:00 |
|
Stella Lau
|
81d89d82a6
|
Move nbThreads to cctx params
|
2017-08-18 12:08:57 -07:00 |
|
Stella Lau
|
2300c58a6f
|
Move dictContentByRef to cctx params
|
2017-08-18 12:03:16 -07:00 |
|
Stella Lau
|
b6cb2ed8cb
|
Move dictMode to cctxParams
|
2017-08-18 11:43:31 -07:00 |
|
Stella Lau
|
97e27affcb
|
Move compression level to cctx params
|
2017-08-18 11:20:08 -07:00 |
|
Stella Lau
|
c0221124d5
|
Add function to set opaque parameters
|
2017-08-17 19:30:22 -07:00 |
|
Stella Lau
|
4169f49171
|
Add initialization/allocation functions for opaque params
|
2017-08-17 18:45:04 -07:00 |
|
Stella Lau
|
ade95b8bed
|
Add opaque interfaces for static initialization
|
2017-08-17 18:13:08 -07:00 |
|
Stella Lau
|
699f11b4f7
|
Create opaque parameter structure
|
2017-08-17 17:33:46 -07:00 |
|
Nick Terrell
|
ae20d413da
|
[libzstd] Fix CHECK_V_F macros
|
2017-07-25 12:52:01 -07:00 |
|
Yann Collet
|
a90b16e150
|
Visual blind fix 2
|
2017-07-20 15:57:55 -07:00 |
|
Yann Collet
|
b4d460f32c
|
pool.c : blindfix for Visual warnings
|
2017-07-20 01:13:14 -07:00 |
|
Yann Collet
|
3974d2b38a
|
blind fix for Windows Multithreading module
adds a fake 0 return value for mutex/cond init
|
2017-07-19 13:33:21 -07:00 |
|
Yann Collet
|
b71363b967
|
check pthread_*_init() success condition
|
2017-07-19 01:05:40 -07:00 |
|
Nick Terrell
|
cc1522351f
|
[libzstd] Fix bug in Huffman encoding
Summary:
Huffman encoding with a bad dictionary can encode worse than the
HUF_BLOCKBOUND(srcSize), since we don't filter out incompressible
input, and even if we did, the dictionaries Huffman table could be
ill suited to compressing actual data.
The fast optimization doesn't seem to improve compression speed,
even when I hard coded fast = 1, the speed didn't improve over hard coding
it to 0.
Benchmarks:
$ ./zstd.dev -b1e5
Benchmarking levels from 1 to 5
1#Synthetic 50% : 10000000 -> 3139163 (3.186), 524.8 MB/s ,1890.0 MB/s
2#Synthetic 50% : 10000000 -> 3115138 (3.210), 372.6 MB/s ,1830.2 MB/s
3#Synthetic 50% : 10000000 -> 3222672 (3.103), 223.3 MB/s ,1400.2 MB/s
4#Synthetic 50% : 10000000 -> 3276678 (3.052), 198.0 MB/s ,1280.1 MB/s
5#Synthetic 50% : 10000000 -> 3271570 (3.057), 107.8 MB/s ,1200.0 MB/s
$ ./zstd -b1e5
Benchmarking levels from 1 to 5
1#Synthetic 50% : 10000000 -> 3139163 (3.186), 524.8 MB/s ,1870.2 MB/s
2#Synthetic 50% : 10000000 -> 3115138 (3.210), 370.0 MB/s ,1810.3 MB/s
3#Synthetic 50% : 10000000 -> 3222672 (3.103), 223.3 MB/s ,1380.1 MB/s
4#Synthetic 50% : 10000000 -> 3276678 (3.052), 196.1 MB/s ,1270.0 MB/s
5#Synthetic 50% : 10000000 -> 3271570 (3.057), 106.8 MB/s ,1180.1 MB/s
$ ./zstd.dev -b1e5 ../silesia.tar
Benchmarking levels from 1 to 5
1#silesia.tar : 211988480 -> 73651685 (2.878), 429.7 MB/s ,1096.5 MB/s
2#silesia.tar : 211988480 -> 70158785 (3.022), 321.2 MB/s ,1029.1 MB/s
3#silesia.tar : 211988480 -> 66993813 (3.164), 243.7 MB/s , 981.4 MB/s
4#silesia.tar : 211988480 -> 66306481 (3.197), 226.7 MB/s , 972.4 MB/s
5#silesia.tar : 211988480 -> 64757852 (3.274), 150.3 MB/s , 963.6 MB/s
$ ./zstd -b1e5 ../silesia.tar
Benchmarking levels from 1 to 5
1#silesia.tar : 211988480 -> 73651685 (2.878), 429.7 MB/s ,1087.1 MB/s
2#silesia.tar : 211988480 -> 70158785 (3.022), 318.8 MB/s ,1029.1 MB/s
3#silesia.tar : 211988480 -> 66993813 (3.164), 246.5 MB/s , 981.4 MB/s
4#silesia.tar : 211988480 -> 66306481 (3.197), 229.2 MB/s , 972.4 MB/s
5#silesia.tar : 211988480 -> 64757852 (3.274), 149.3 MB/s , 963.6 MB/s
Test Plan:
I added a test case to the fuzzer which crashed with ASAN before the patch
and succeeded after.
|
2017-07-18 13:20:40 -07:00 |
|
Yann Collet
|
77d67fb167
|
Merge pull request #766 from terrelln/real-block-split
[libzstd] Pull optimal parser state out of seqStore_t
|
2017-07-18 08:26:24 -07:00 |
|
Yann Collet
|
14c83b05c7
|
Merge pull request #765 from terrelln/real-block-split
[libzstd] Remove ZSTD_CCtx* argument of ZSTD_compressSequences()
|
2017-07-17 19:25:55 -07:00 |
|