Yann Collet
33a6639039
fixed ZSTD_refPrefix with Multithread-enabled CCtx
2017-06-28 11:09:43 -07:00
Yann Collet
1fd2df9c2c
fixed zstreamtest
...
modifying compression parameters is not allowed after ZSTD_CCtx_loadDictionary()
but it is still allowed after ZSTD_refPrefix()
2017-06-28 09:57:59 -07:00
Yann Collet
2e4274262d
controlled dictMode
2017-06-27 17:09:12 -07:00
Yann Collet
b7372933b8
implemented ZSTD_refPrefix()
2017-06-27 15:49:12 -07:00
Yann Collet
fecc721fd9
added parameter ZSTD_p_refDictContent
2017-06-27 11:46:39 -07:00
Yann Collet
09a5bbe22e
Merge pull request #735 from terrelln/xz-default
...
[zstdcli] Support xz by default when liblzma is available
2017-06-27 09:42:02 -07:00
Nick Terrell
5b7fd7c422
[zdict] Make COVER the default algorithm
2017-06-26 21:09:22 -07:00
Yann Collet
20eeb243d1
Merge pull request #729 from paulcruz74/corpus
...
Corpus
2017-06-26 17:47:28 -07:00
Yann Collet
dde10b23fe
refactored ZSTD_estimateDStreamSize()
...
now uses windowSize as argument.
Also : created ZSTD_estimateDStreamSize_fromFrame()
2017-06-26 17:44:26 -07:00
Yann Collet
09ae03a570
ZSTD_estimateCDictSize_advanced()
...
ZSTD_estimateCDictSize() now uses same arguments as ZSTD_createCDict()
ZSTD_estimateCDictSize_advanced() uses same arguments as ZSTD_createCDict_advanced()
2017-06-26 16:47:32 -07:00
Yann Collet
0c9a915a28
ZSTD_estimateCStreamSize_advanced()
2017-06-26 16:02:25 -07:00
Yann Collet
31af8290d1
ZSTD_estimateCCtx_advanced()
...
ZSTD_estimateCCtx() is now a "simple" function,
taking int compressionLevel as single argument.
ZSTD_estimateCCtx_advanced() takes a CParams argument,
which is both more complete and more complex to generate.
2017-06-26 15:52:39 -07:00
Nick Terrell
6aeb50ea0d
[zstdcli] Add unlzma and unxz symlinks
2017-06-26 11:24:36 -07:00
Paul Cruz
298fe9f23b
added cli tests to test-all
2017-06-23 17:10:41 -07:00
Paul Cruz
3a295a91f8
added additional condition so large offsets into the dictionary are not generated past windowSize
2017-06-23 15:54:51 -07:00
Paul Cruz
2085375816
fixed bug detected by the API test
2017-06-23 13:44:24 -07:00
Paul Cruz
8cd134559d
type warnings
2017-06-23 12:00:48 -07:00
Paul Cruz
4219acc60a
fixed bus error bug
2017-06-23 11:22:29 -07:00
Yann Collet
ef269c1b68
Merge pull request #725 from facebook/advancedAPI2
...
New Advanced API
2017-06-23 09:50:47 -07:00
Yann Collet
cca1ed14db
Merge branch 'advancedAPI2' of github.com:facebook/zstd into advancedAPI2
2017-06-23 00:13:03 -07:00
Yann Collet
32f546c763
zstreamtest : disabled multi-threading tests with --newapi --no-big-tests
...
--no-big-tests is typically used in combination with qemu-user-static
qemu-user-static allocated 4 GB of RAM upfront.
On 2 GB VM, this can degenerate into a crash.
It's not a problem as long as memory is not used.
But with multi-threading enabled, memory fragmentation kicks in,
so the amoung of RAM effectively touched increases,
and can pass beyond the 2 GB limit of the VM.
In single-threaded mode, there is no such issue :
memory requirement is smaller, and remains well-located,
so very little fragmentation is expected.
This modification should make `qemu-arm-static zstreamtests --newapi --no-big-tests`
work fine on Travis CI.
2017-06-23 00:09:02 -07:00
Paul Cruz
2e8cc6f12a
added sizeof for clarity
2017-06-22 15:52:33 -07:00
Paul Cruz
6d7f442120
changed makefile test
2017-06-22 15:44:01 -07:00
Paul Cruz
b325a2e4db
changed assignment
2017-06-22 15:36:28 -07:00
Paul Cruz
2935239544
added description for dictionary cli tests
2017-06-22 15:23:59 -07:00
Paul Cruz
97a8f89bec
made changes to cli test so it runs properly
2017-06-22 15:13:41 -07:00
Paul Cruz
829eb29033
added cli test for decodecorpus inside tests/Makefile. Also changed calculation of offset
2017-06-22 14:43:44 -07:00
Paul Cruz
028fb9d01b
added cli test case in Makefile
2017-06-22 13:53:39 -07:00
Paul Cruz
98751f69e7
should be updating seed whenever multiple files are generated
2017-06-22 10:23:36 -07:00
Paul Cruz
84cfa07d2d
changed format of command to --use-dict=#
2017-06-22 10:04:14 -07:00
Yann Collet
f99c2c1a22
reduced --no-big-tests even more
...
to pass tests on qemu-aarch64
2017-06-21 23:35:58 -07:00
Paul Cruz
04094f37e9
fixed offset in this case os that it always goes past src start
2017-06-21 18:47:40 -07:00
Yann Collet
49f8459d11
fixed minor cast warning
2017-06-21 18:43:39 -07:00
Paul Cruz
0950b3159a
more meaningful names for count variables
2017-06-21 18:30:27 -07:00
Paul Cruz
0b6eedeace
malloc samples instead of static allocation
2017-06-21 18:24:19 -07:00
Yann Collet
4d3bdcf130
reduced CLevelMax for --no-big-tests
2017-06-21 18:20:02 -07:00
Paul Cruz
52a13abf3f
removed comment regarding windowSize requirement
2017-06-21 18:13:52 -07:00
Paul Cruz
32e36c2d9a
removed requirement that offset be less than windowSize
2017-06-21 18:00:47 -07:00
Yann Collet
bfc2f00080
--no-big-tests for zstreamtest
...
Apply --no-big-tests for tsan tests
2017-06-21 17:57:14 -07:00
Paul Cruz
eeff2d0e3c
added const x2
2017-06-21 17:48:37 -07:00
Paul Cruz
04253e21bb
added const
2017-06-21 17:48:01 -07:00
Paul Cruz
2ceaec1aad
got rid of unnecessary casting to void*
2017-06-21 17:46:37 -07:00
Paul Cruz
ccae9ec807
added comment addressing use of bits in frame header descriptor
2017-06-21 17:43:21 -07:00
Yann Collet
dce789281b
fixed : decompression of skippable frames in streaming mode
2017-06-21 15:53:42 -07:00
Yann Collet
c16748b137
added test cases for new command --list
2017-06-21 12:09:53 -07:00
Yann Collet
7bd1a2900e
added ZSTD_dictMode_e to control dictionary loading mode
2017-06-21 11:50:33 -07:00
Paul Cruz
db3606e249
added test for files with no frame content size
2017-06-20 17:43:36 -07:00
Paul Cruz
acf0df678c
fixed strange spacing
2017-06-20 15:41:43 -07:00
Paul Cruz
69bc4fab25
free the context object once done with decompression
2017-06-20 15:37:14 -07:00
Paul Cruz
ceb4f65620
allocate statically so we don't have to deal with freeing
2017-06-20 15:31:54 -07:00
Paul Cruz
a73c2a444a
added tests with null files, changed condition to check that the file is more than 0 bytes
2017-06-20 14:33:08 -07:00
Paul Cruz
58c19b4202
spacing matters for test scripts
2017-06-20 14:14:53 -07:00
Yann Collet
ff8f83bd47
fixed fuzzer test
2017-06-20 12:17:32 -07:00
Paul Cruz
b07d0af2ba
added additional tests for error detection
2017-06-20 11:54:44 -07:00
Paul Cruz
43ffcdde40
const dictOffset
2017-06-20 11:27:39 -07:00
Paul Cruz
1e0d125e76
const value
2017-06-20 11:15:26 -07:00
Paul Cruz
0034129270
made changes to accommodate single file use case
2017-06-20 11:07:38 -07:00
Paul Cruz
6a99de2dd8
change test to list info on .zst files
2017-06-20 10:45:06 -07:00
Paul Cruz
36e14b1963
added description to advanced usage, changed to use-dict
2017-06-20 10:34:34 -07:00
Paul Cruz
15102069a1
removed testing code from generation code
2017-06-20 10:25:16 -07:00
Paul Cruz
ff5589c091
added comments for clarity
2017-06-20 10:17:38 -07:00
Paul Cruz
551fb7ca05
changed error condition
2017-06-20 10:14:03 -07:00
Paul Cruz
b8237c6c66
refactoring for readability
2017-06-20 10:12:13 -07:00
Paul Cruz
6f17de2132
changed computation for readability
2017-06-20 09:50:28 -07:00
Paul Cruz
4a4f9ba262
const value that doesn't change
2017-06-19 17:46:21 -07:00
Paul Cruz
3a6856edca
changed scope to avoid errors
2017-06-19 17:43:16 -07:00
Paul Cruz
7d2b5613b4
adhere to min dict size
2017-06-19 17:37:14 -07:00
Paul Cruz
ea2af1ffc4
changed dictionary to stay reasonable size during test
2017-06-19 17:34:50 -07:00
Paul Cruz
fdbb07283a
added test to runTestMode
2017-06-19 17:29:15 -07:00
Paul Cruz
bdec7b8ce7
fixed for loop declaration, fixed test case
2017-06-19 17:18:43 -07:00
Paul Cruz
23098b53e8
wrote a test for dictionary corpus
2017-06-19 17:13:00 -07:00
Paul Cruz
7a26390989
changing for loop declaration
2017-06-19 16:39:57 -07:00
Paul Cruz
147ef05f12
for loop declaration not allowed
2017-06-19 16:38:51 -07:00
Paul Cruz
791352ad0e
scope change
2017-06-19 16:32:38 -07:00
Paul Cruz
365e265a8e
changed scope so that warning disappears
2017-06-19 16:28:40 -07:00
Paul Cruz
f4f928595b
changed scope to get rid of mixed code/variables warning
2017-06-19 16:27:05 -07:00
Paul Cruz
7ea025cc7d
added in struct to keep functions more orderly
2017-06-19 16:23:00 -07:00
Paul Cruz
b91b810c64
made some small changes to dictInfo struct and initDictInfo
2017-06-19 16:05:05 -07:00
Paul Cruz
1ee4f6b36c
cleaning up code
2017-06-19 15:55:01 -07:00
Paul Cruz
2a52d72712
added comment to anotate decompressed error code
2017-06-19 14:21:35 -07:00
Paul Cruz
73ebb646d8
small stylistic changes
2017-06-19 14:20:35 -07:00
Paul Cruz
34f4e5cab2
changed if statement styling
2017-06-19 14:07:56 -07:00
Paul Cruz
1fc1a35dc4
added calls to free to match calls to malloc, made some stylistic changes, added init function for new struct
2017-06-19 14:02:27 -07:00
Paul Cruz
8f4fa43fe8
got files to decompress
2017-06-19 13:21:50 -07:00
Yann Collet
688952062b
minor declaration statement warning fix
2017-06-19 11:53:42 -07:00
Yann Collet
d7a3bffba9
new api : setting compression parameters is refused if a dictionary is already loaded
2017-06-19 11:53:01 -07:00
Paul Cruz
da4ddc1f76
fixed decompression so that it uses compressed version
2017-06-19 11:40:45 -07:00
Paul Cruz
33467add60
made changes so that HUF_writeCTable no longer fails
2017-06-19 11:27:50 -07:00
Yann Collet
ed1d039127
newapi fuzzer tests : random flush orders in main loop
2017-06-19 11:07:33 -07:00
Paul Cruz
09ac51b56b
small style changes
2017-06-19 09:24:38 -07:00
Yann Collet
f5deae8a67
new api fuzzer tests : frame parameters are randonly set
2017-06-18 23:41:38 -07:00
Yann Collet
ea3630a889
new api fuzzer set : dictionary is randomly set
2017-06-18 23:31:55 -07:00
Yann Collet
8dee0ec99e
new api fuzzer tests : compression parameters are randomly set
2017-06-18 23:25:15 -07:00
Yann Collet
bd05b19713
removed macro CLAMP from decodecorpus
...
never used,
and would duplicate with CLAMP from zstd_compress.c
2017-06-16 18:16:27 -07:00
Yann Collet
01743a36e7
fuzzer tests for new API
2017-06-16 17:56:41 -07:00
Paul Cruz
2f30433cd7
fixing general style
2017-06-15 18:06:32 -07:00
Paul Cruz
ddd1ab710c
added tests in playTest.sh
2017-06-15 16:53:03 -07:00
Paul Cruz
623b8fce85
add print statements for debugging and change malloc with updated size
2017-06-15 11:37:24 -07:00
Paul Cruz
52934fbfc8
added print statements for debugging, fixed first segfault
2017-06-15 09:47:29 -07:00
Paul Cruz
03a208e27c
changed samples to geometric distribution so that it is huffman compressible
2017-06-14 18:06:16 -07:00
