Yann Collet
fc5145955a
updated ZSTD_estimateCCtxSize()
...
added a parameter streaming,
to estimate memory allocation size
when the CCtx is used for streaming (CStream).
Note : this function is not able to estimate
memory cost of a potential internal CDict
which can only happen when starting with ZSTD_initCStream_usingDict()
2017-05-08 17:07:59 -07:00
Yann Collet
0be6fd3429
merged CCtx and CStream as a single same object
...
To be changed : ZSTD_sizeof_CCtx(), ZSTD_estimateCCtxSize()
2017-05-08 16:08:01 -07:00
Yann Collet
01a1abfdb5
cli : -d and -t do not stop after a failed decompression
...
The problematic srcfile will be named on console/log,
but decompression/test will continue onto next file in the list.
2017-05-05 19:15:24 -07:00
Nick Terrell
f376d47c11
[CLI] Switch dictionary builder on CLI to cover
2017-05-02 11:18:27 -07:00
Yann Collet
a51cab6e68
Merge pull request #678 from facebook/apiChange
...
Breaking API Change around CDict
2017-04-28 10:02:45 -07:00
Yann Collet
36e79a1a1f
tests : -g is part of DEBUGFLAGS
2017-04-27 17:29:40 -07:00
Yann Collet
c6915429f2
shortened Appveyor release tests
...
fuzzer supports time suffix `s` for "seconds"
2017-04-27 16:24:53 -07:00
Yann Collet
a92cbb7004
Added a secondary test, checking dictID presence after setting noDictIdFLag=1
2017-04-27 15:08:56 -07:00
Yann Collet
7d283cdfa2
added test for ZSTD_initCStream_usingCDict_advanced()
2017-04-27 14:48:34 -07:00
Yann Collet
2f73427d35
added test for ZSTD_compress_usingCDict_advanced()
2017-04-27 14:39:39 -07:00
Yann Collet
7321345fd2
fixed another VS2008 init error
2017-04-27 14:19:34 -07:00
Yann Collet
f4bd857d81
created ZSTD_compress_usingCDict_advanced()
2017-04-27 11:31:55 -07:00
Yann Collet
31533bacce
Changed ZSTD_createCDict_advanced()
...
It now only uses compressionParameters as argument.
It produces many changes throughout user code,
though hopefully they tend to be simple :
just provide the cParams part from existing ZSTD_parameters.
Some programs might depend on ZSTD_createCDict_advanced() to pass frame parameters.
This change will force them to revisit this strategy and fix it,
since frame parameters are effectively silently ignored in current version.
2017-04-27 00:29:04 -07:00
Sean Purcell
4de8632957
Add LZ4 compress/decompress support to CLI
2017-04-24 16:48:25 -07:00
Yann Collet
230d7acc7d
cli : add support for --threads=# command
...
updated documentation
add relevant test case
2017-04-21 11:38:13 -07:00
Yann Collet
c17e020c9a
disable assert when compiling paramgrill
...
paramgrill is a benchmark calibration function.
Speed accuracy is critical, it cannot be altered by assert.
2017-04-20 12:50:02 -07:00
Yann Collet
2c5514c759
fixed ZSTDMT_initCStream_advanced()
...
Must use the new ZSTD_compressBegin_usingCDict_advanced()
to enforce correct frame parameters
2017-04-18 22:52:41 -07:00
Yann Collet
0bb381dad8
added test for ZSTD_initCStream_advanced()
...
when params.fParams.contentSizeFlags = 1
and pledgedSrcSize = 0
then srcSize should be considered 0 (empty), and not "unknown"
2017-04-18 15:08:52 -07:00
Yann Collet
4f818182b8
clarified frame parameters for ZSTD_compress*_usingCDict()
...
created ZSTD_compressBegin_usingCDict_internal(),
which gives direct control to frame Parameters.
ZSTD_resetCStream_internal() now points into it.
2017-04-17 18:29:06 -07:00
Yann Collet
9c4c970399
Merge branch 'dev' into compressionFlow
2017-04-17 16:33:09 -07:00
Sean Purcell
d845dab69c
Fix input size too small to trigger zstdmt
2017-04-17 12:10:58 -07:00
Sean Purcell
c424ec2eae
Add multithreading tests to playTests.sh
2017-04-17 11:41:46 -07:00
Yann Collet
2afb075382
Merge branch 'dev' into compressionFlow
2017-04-14 15:08:31 -07:00
Yann Collet
f913cbed33
fixed : memory leak in fuzzer test
2017-04-13 22:46:51 -07:00
Yann Collet
20d5e03893
content size is controlled at bufferless level
...
so it's active for all entry points
Also : added relevant test (wrong content size) in fuzzer
2017-04-11 18:34:02 -07:00
Sean Purcell
d37e1df2ab
Fix message
2017-04-11 17:33:26 -07:00
Sean Purcell
eb70d219fd
Add test of file > 4GB to playTests
2017-04-11 17:15:13 -07:00
Yann Collet
4ee6b15dac
force contentSizeFlag=0 when using ZSTD_initCStream_usingCDict()
...
because by definition srcSize is not known when using this prototype.
added relevant test
Note : this use was already working, because at a later stage
(both ZSTD_compressBegin_usingCDict() and ZSTD_copyCCtx())
pledgedSrcSize=0 is translated into "unknown", no matter the frame parameter.
This is not correct, but of little importance,
as the medium term plan is to no longer set fParams within CDict
2017-04-11 11:59:44 -07:00
Yann Collet
4b987ad8ce
Introduce ZSTD_initCStream_internal()
...
This is now the regroup point for ZSTD_initCStream*() functions
ZSTD_initCStream_advanced() now properly checks for parameters validity.
Also : added <assert.h> usage inside zstd_compress.c
Needs ZSTD_DEBUG=1 macro to be triggered.
Will be triggered by default from `tests` directory
2017-04-10 17:50:44 -07:00
Yann Collet
1fa3b75369
Merge pull request #648 from facebook/zstdmt
...
improved zstreamtest --mt to trap bug #644
2017-04-07 06:09:52 -07:00
Sean Purcell
1652172b2d
Add refuse to overwrite test and fix format
2017-04-06 17:06:30 -07:00
Sean Purcell
9da11c6aae
Add test for overwriting read-only
2017-04-06 12:58:49 -07:00
Yann Collet
ce80098f14
improved zstreamtest --mt to trap bug #644
2017-04-05 16:34:09 -07:00
Yann Collet
9631e603ae
paramgrill : results from optimizer use cli format
...
Best found configuration is displayed
using documented cli advanced parameter syntax.
2017-04-04 16:54:33 -07:00
Yann Collet
89b32f3db0
fix paramgrill -O#
...
ensure proposed config does not require more memory than necessary
2017-04-04 16:41:11 -07:00
Yann Collet
2724f25567
fixed paramgrill -O#
...
find optimal settings for a minimum speed
2017-04-04 16:31:17 -07:00
Yann Collet
c2007388a5
fixed bench.c : optional advanced parameters applied
...
before creating cdict
2017-04-04 15:35:06 -07:00
Nick Terrell
39a6cc5172
Make ZSTD_compress_usingCDict() respect contentSizeFlag
2017-04-03 21:09:55 -07:00
Nick Terrell
62ecad3819
Fix ZSTD_initCStream_usingCDict() to use dictionary
2017-04-03 21:05:59 -07:00
Yann Collet
30c7698970
optimize ZSTDMT_compress() memory usage
...
does no longer allocate temporary buffers
when there is enough room in dstBuffer to decompress directly there.
(previous method would skip that for 1st chunk only).
Also : fix ZSTD_compressBound() for small srcSize
2017-03-31 18:27:03 -07:00
Yann Collet
6476c51b86
Merge pull request #637 from facebook/zstdmt
...
Zstdmt
2017-03-30 14:18:37 -07:00
Nick Terrell
5152fb2cb2
Convert all tabs to spaces
2017-03-29 18:51:58 -07:00
Yann Collet
1e1e26f2ac
fixed #634 : ZSTDMT_compressCCtx() doesn't provide frame content size in header
2017-03-29 17:09:59 -07:00
Yann Collet
0ef6803707
added ZSTD_getFrameContentSize() to ZSTDMT unit test in fuzzer
...
Now tests fail, because ZSTDMT_compress() doesn't fill frame content size correctly.
2017-03-29 16:58:57 -07:00
Yann Collet
41fefd573a
Improved speed tests
...
Better compatibility with Mac OS-X
Force attribution to a selected core # (Linux)
2017-03-26 23:52:19 -07:00
Yann Collet
4c41d37fcc
changed test for new syntax
...
--dictID= and --maxdict=
2017-03-24 18:36:56 -07:00
Yann Collet
f332ece468
dictBuilder fails to create dictionary on certain input
...
Properly expressed with an error code (see zstd_errors.h)
and a cli return code != 0
2017-03-23 16:24:02 -07:00
Sean Purcell
680e4e0953
Ignore symbolic links unless --force specified
2017-03-23 11:52:09 -07:00
Sean Purcell
042ba122ae
Change g_displayLevel to int and fix DISPLAYUPDATE flush
2017-03-23 11:21:59 -07:00
Sean Purcell
1b418bd3da
Merge remote-tracking branch 'origin/dev' into appveyor
2017-03-21 10:59:54 -07:00