Stella Lau
502031ca10
Use cctxParam version of createCDict internally
2017-08-21 11:00:44 -07:00
Stella Lau
91b30dbe84
Remove test parameter
2017-08-21 10:09:06 -07:00
Stella Lau
f181f33bdf
Disable tests and refactor
2017-08-21 01:59:08 -07:00
Stella Lau
023b24e6d4
Add cctx param tests
2017-08-20 22:55:07 -07:00
Stella Lau
6cee6e07e5
Add internal createCDict function
2017-08-18 22:48:31 -07:00
Stella Lau
d775519296
Add cctxParam versions of internal functions
2017-08-18 17:37:58 -07:00
Stella Lau
63b8c98531
Pass cctx parameters to MTCtx
2017-08-18 16:17:24 -07:00
Stella Lau
399ae013d4
Add function to apply cctx params
2017-08-18 13:01:55 -07:00
Stella Lau
81d89d82a6
Move nbThreads to cctx params
2017-08-18 12:08:57 -07:00
Stella Lau
2300c58a6f
Move dictContentByRef to cctx params
2017-08-18 12:03:16 -07:00
Stella Lau
b6cb2ed8cb
Move dictMode to cctxParams
2017-08-18 11:43:31 -07:00
Stella Lau
97e27affcb
Move compression level to cctx params
2017-08-18 11:20:08 -07:00
Stella Lau
c0221124d5
Add function to set opaque parameters
2017-08-17 19:30:22 -07:00
Stella Lau
4169f49171
Add initialization/allocation functions for opaque params
2017-08-17 18:45:04 -07:00
Stella Lau
ade95b8bed
Add opaque interfaces for static initialization
2017-08-17 18:13:08 -07:00
Stella Lau
699f11b4f7
Create opaque parameter structure
2017-08-17 17:33:46 -07:00
Yann Collet
6466fd3400
Merge pull request #776 from terrelln/dev
...
[libzstd] Fix CHECK_V_F() macro
2017-07-25 14:39:32 -07:00
Nick Terrell
ae20d413da
[libzstd] Fix CHECK_V_F macros
2017-07-25 12:52:01 -07:00
Yann Collet
38ba7002f2
fixed minor warning on unused variable in shell function
2017-07-20 18:39:04 -07:00
Yann Collet
a90b16e150
Visual blind fix 2
2017-07-20 15:57:55 -07:00
Yann Collet
4542d7118c
Merge branch 'dev' of github.com:facebook/zstd into dev
2017-07-20 15:12:21 -07:00
Yann Collet
5e6c5203f3
fixed fuzzer test for non OS-X platforms
2017-07-20 15:11:56 -07:00
Yann Collet
57251f52cf
Merge pull request #773 from terrelln/lk-v3
...
[linux] Update patches for v3
2017-07-20 14:47:26 -07:00
Nick Terrell
7d3ac0710d
[linux] Update patches for v3
2017-07-20 13:33:55 -07:00
Yann Collet
b4d460f32c
pool.c : blindfix for Visual warnings
2017-07-20 01:13:14 -07:00
Yann Collet
1ca1288689
added --memtest=# command to fuzzer
...
to jump directly to relevant test section
2017-07-19 16:01:16 -07:00
Yann Collet
3974d2b38a
blind fix for Windows Multithreading module
...
adds a fake 0 return value for mutex/cond init
2017-07-19 13:33:21 -07:00
Yann Collet
b71363b967
check pthread_*_init() success condition
2017-07-19 01:05:40 -07:00
Yann Collet
44b0838253
Merge pull request #770 from terrelln/test-mode
...
[zstdcli] Fix -t in streaming mode
2017-07-18 15:40:59 -07:00
Yann Collet
2aba13b770
Merge pull request #769 from terrelln/real-block-split
...
[libzstd] Fix bug in Huffman encoding
2017-07-18 14:58:26 -07:00
Nick Terrell
d0b27483ae
[zstdcli] Fix -t in streaming mode
2017-07-18 14:45:49 -07:00
Nick Terrell
cc1522351f
[libzstd] Fix bug in Huffman encoding
...
Summary:
Huffman encoding with a bad dictionary can encode worse than the
HUF_BLOCKBOUND(srcSize), since we don't filter out incompressible
input, and even if we did, the dictionaries Huffman table could be
ill suited to compressing actual data.
The fast optimization doesn't seem to improve compression speed,
even when I hard coded fast = 1, the speed didn't improve over hard coding
it to 0.
Benchmarks:
$ ./zstd.dev -b1e5
Benchmarking levels from 1 to 5
1#Synthetic 50% : 10000000 -> 3139163 (3.186), 524.8 MB/s ,1890.0 MB/s
2#Synthetic 50% : 10000000 -> 3115138 (3.210), 372.6 MB/s ,1830.2 MB/s
3#Synthetic 50% : 10000000 -> 3222672 (3.103), 223.3 MB/s ,1400.2 MB/s
4#Synthetic 50% : 10000000 -> 3276678 (3.052), 198.0 MB/s ,1280.1 MB/s
5#Synthetic 50% : 10000000 -> 3271570 (3.057), 107.8 MB/s ,1200.0 MB/s
$ ./zstd -b1e5
Benchmarking levels from 1 to 5
1#Synthetic 50% : 10000000 -> 3139163 (3.186), 524.8 MB/s ,1870.2 MB/s
2#Synthetic 50% : 10000000 -> 3115138 (3.210), 370.0 MB/s ,1810.3 MB/s
3#Synthetic 50% : 10000000 -> 3222672 (3.103), 223.3 MB/s ,1380.1 MB/s
4#Synthetic 50% : 10000000 -> 3276678 (3.052), 196.1 MB/s ,1270.0 MB/s
5#Synthetic 50% : 10000000 -> 3271570 (3.057), 106.8 MB/s ,1180.1 MB/s
$ ./zstd.dev -b1e5 ../silesia.tar
Benchmarking levels from 1 to 5
1#silesia.tar : 211988480 -> 73651685 (2.878), 429.7 MB/s ,1096.5 MB/s
2#silesia.tar : 211988480 -> 70158785 (3.022), 321.2 MB/s ,1029.1 MB/s
3#silesia.tar : 211988480 -> 66993813 (3.164), 243.7 MB/s , 981.4 MB/s
4#silesia.tar : 211988480 -> 66306481 (3.197), 226.7 MB/s , 972.4 MB/s
5#silesia.tar : 211988480 -> 64757852 (3.274), 150.3 MB/s , 963.6 MB/s
$ ./zstd -b1e5 ../silesia.tar
Benchmarking levels from 1 to 5
1#silesia.tar : 211988480 -> 73651685 (2.878), 429.7 MB/s ,1087.1 MB/s
2#silesia.tar : 211988480 -> 70158785 (3.022), 318.8 MB/s ,1029.1 MB/s
3#silesia.tar : 211988480 -> 66993813 (3.164), 246.5 MB/s , 981.4 MB/s
4#silesia.tar : 211988480 -> 66306481 (3.197), 229.2 MB/s , 972.4 MB/s
5#silesia.tar : 211988480 -> 64757852 (3.274), 149.3 MB/s , 963.6 MB/s
Test Plan:
I added a test case to the fuzzer which crashed with ASAN before the patch
and succeeded after.
2017-07-18 13:20:40 -07:00
Yann Collet
77d67fb167
Merge pull request #766 from terrelln/real-block-split
...
[libzstd] Pull optimal parser state out of seqStore_t
2017-07-18 08:26:24 -07:00
Yann Collet
14c83b05c7
Merge pull request #765 from terrelln/real-block-split
...
[libzstd] Remove ZSTD_CCtx* argument of ZSTD_compressSequences()
2017-07-17 19:25:55 -07:00
Nick Terrell
7a28b9e4a3
[libzstd] Pull optimal parser state out of seqStore_t
2017-07-17 15:29:11 -07:00
Yann Collet
3381bf4b84
Merge pull request #764 from terrelln/real-block-split
...
[libzstd] Refactor ZSTD_compressSequences()
2017-07-17 14:46:01 -07:00
Nick Terrell
e198230645
[libzstd] Remove ZSTD_CCtx* argument of ZSTD_compressSequences()
2017-07-17 12:27:24 -07:00
Nick Terrell
634f012420
[libzstd] Refactor ZSTD_compressSequences()
2017-07-17 11:36:11 -07:00
Yann Collet
fa3aa04ccd
Merge pull request #761 from paulcruz74/file-rename
...
renamed pool.c to poolTests.c
2017-07-14 09:09:45 -07:00
Yann Collet
3841ee6fb3
Merge pull request #762 from facebook/errorCodes
...
pinned down error code enum values
2017-07-14 09:09:22 -07:00
Yann Collet
3b0cff3c33
fixed clang's -Wdocumentation
2017-07-13 18:58:30 -07:00
Yann Collet
2bd6440be0
pinned down error code enum values
...
Note : all error codes are changed by this new version,
but it's expected to be the last change for existing codes.
Codes are now grouped by category, and receive a manually attributed value.
The objective is to guarantee that
error code values will not change in the future
when introducing new codes.
Intentionnal empty spaces and ranges are defined
in order to keep room for potential new codes.
2017-07-13 17:12:16 -07:00
Yann Collet
3502426fd4
Merge branch 'dev' of github.com:facebook/zstd into dev
2017-07-13 15:49:19 -07:00
Yann Collet
6733c0777c
updated NEWS regarding #760
2017-07-13 15:34:44 -07:00
Nick Terrell
830ef4152a
[libzstd] Increase granularity of FSECTable repeat mode
2017-07-13 12:45:39 -07:00
Yann Collet
d985319337
Merge pull request #759 from terrelln/real-block-split
...
[libzstd] Pull CTables into sub-structure
2017-07-13 10:24:19 -07:00
Yann Collet
3a60efd3a9
policy change : ZSTDMT automatically caps nbThreads to ZSTDMT_NBTHREADS_MAX ( #760 )
...
Previously, ZSTDMT would refuse to create the compressor.
Also : increased ZSTDMT_NBTHREADS_MAX to 256,
updated doc,
and added relevant test
2017-07-13 10:17:23 -07:00
Yann Collet
132e6efd76
switched ZSTDMT_compress_advanced() last argument to overlapLog
...
overlapRLog (== 9 - overlapLog) was a bit "strange"
as all other public entry points use overlapLog
2017-07-13 02:22:58 -07:00
Yann Collet
4e77f7761d
clarified comment on ZSTD_p_contentSizeFlag
2017-07-13 02:09:07 -07:00
Nick Terrell
de0414b736
[libzstd] Pull CTables into sub-structure
2017-07-12 19:49:19 -07:00
