AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
4,185 Commits 1 Branch 0 Tags 27 MiB
767a0b3be1
Commit Graph

1774 Commits

Author SHA1 Message Date
Yann Collet
a90b16e150 Visual blind fix 2 2017-07-20 15:57:55 -07:00
Yann Collet
b4d460f32c pool.c : blindfix for Visual warnings 2017-07-20 01:13:14 -07:00
Yann Collet
3974d2b38a blind fix for Windows Multithreading module 
adds a fake 0 return value for mutex/cond init
 2017-07-19 13:33:21 -07:00
Paul Cruz
6945b3c43d removed previous version of completion for compression 2017-07-19 11:51:50 -07:00
Yann Collet
b71363b967 check pthread_*_init() success condition 2017-07-19 01:05:40 -07:00
Nick Terrell
cc1522351f [libzstd] Fix bug in Huffman encoding 
Summary:
Huffman encoding with a bad dictionary can encode worse than the
HUF_BLOCKBOUND(srcSize), since we don't filter out incompressible
input, and even if we did, the dictionaries Huffman table could be
ill suited to compressing actual data.

The fast optimization doesn't seem to improve compression speed,
even when I hard coded fast = 1, the speed didn't improve over hard coding
it to 0.

Benchmarks:
$ ./zstd.dev -b1e5
Benchmarking levels from 1 to 5
 1#Synthetic 50%     :  10000000 ->   3139163 (3.186), 524.8 MB/s ,1890.0 MB/s
 2#Synthetic 50%     :  10000000 ->   3115138 (3.210), 372.6 MB/s ,1830.2 MB/s
 3#Synthetic 50%     :  10000000 ->   3222672 (3.103), 223.3 MB/s ,1400.2 MB/s
 4#Synthetic 50%     :  10000000 ->   3276678 (3.052), 198.0 MB/s ,1280.1 MB/s
 5#Synthetic 50%     :  10000000 ->   3271570 (3.057), 107.8 MB/s ,1200.0 MB/s
$ ./zstd -b1e5
Benchmarking levels from 1 to 5
 1#Synthetic 50%     :  10000000 ->   3139163 (3.186), 524.8 MB/s ,1870.2 MB/s
 2#Synthetic 50%     :  10000000 ->   3115138 (3.210), 370.0 MB/s ,1810.3 MB/s
 3#Synthetic 50%     :  10000000 ->   3222672 (3.103), 223.3 MB/s ,1380.1 MB/s
 4#Synthetic 50%     :  10000000 ->   3276678 (3.052), 196.1 MB/s ,1270.0 MB/s
 5#Synthetic 50%     :  10000000 ->   3271570 (3.057), 106.8 MB/s ,1180.1 MB/s
$ ./zstd.dev -b1e5 ../silesia.tar
Benchmarking levels from 1 to 5
 1#silesia.tar       : 211988480 ->  73651685 (2.878), 429.7 MB/s ,1096.5 MB/s
 2#silesia.tar       : 211988480 ->  70158785 (3.022), 321.2 MB/s ,1029.1 MB/s
 3#silesia.tar       : 211988480 ->  66993813 (3.164), 243.7 MB/s , 981.4 MB/s
 4#silesia.tar       : 211988480 ->  66306481 (3.197), 226.7 MB/s , 972.4 MB/s
 5#silesia.tar       : 211988480 ->  64757852 (3.274), 150.3 MB/s , 963.6 MB/s
$ ./zstd -b1e5 ../silesia.tar
Benchmarking levels from 1 to 5
 1#silesia.tar       : 211988480 ->  73651685 (2.878), 429.7 MB/s ,1087.1 MB/s
 2#silesia.tar       : 211988480 ->  70158785 (3.022), 318.8 MB/s ,1029.1 MB/s
 3#silesia.tar       : 211988480 ->  66993813 (3.164), 246.5 MB/s , 981.4 MB/s
 4#silesia.tar       : 211988480 ->  66306481 (3.197), 229.2 MB/s , 972.4 MB/s
 5#silesia.tar       : 211988480 ->  64757852 (3.274), 149.3 MB/s , 963.6 MB/s

Test Plan:
I added a test case to the fuzzer which crashed with ASAN before the patch
and succeeded after.
 2017-07-18 13:20:40 -07:00
Yann Collet
77d67fb167 Merge pull request #766 from terrelln/real-block-split 
[libzstd] Pull optimal parser state out of seqStore_t
 2017-07-18 08:26:24 -07:00
Yann Collet
14c83b05c7 Merge pull request #765 from terrelln/real-block-split 
[libzstd] Remove ZSTD_CCtx* argument of ZSTD_compressSequences()
 2017-07-17 19:25:55 -07:00
Nick Terrell
7a28b9e4a3 [libzstd] Pull optimal parser state out of seqStore_t 2017-07-17 15:29:11 -07:00
Yann Collet
3381bf4b84 Merge pull request #764 from terrelln/real-block-split 
[libzstd] Refactor ZSTD_compressSequences()
 2017-07-17 14:46:01 -07:00
Nick Terrell
e198230645 [libzstd] Remove ZSTD_CCtx* argument of ZSTD_compressSequences() 2017-07-17 12:27:24 -07:00
Nick Terrell
634f012420 [libzstd] Refactor ZSTD_compressSequences() 2017-07-17 11:36:11 -07:00
Paul Cruz
50ce4eaeb6 added error detection for pthread initialization, added compression completion measurement, fixed const values 2017-07-17 10:12:44 -07:00
Yann Collet
3b0cff3c33 fixed clang's -Wdocumentation 2017-07-13 18:58:30 -07:00
Yann Collet
2bd6440be0 pinned down error code enum values 
Note : all error codes are changed by this new version,
but it's expected to be the last change for existing codes.

Codes are now grouped by category, and receive a manually attributed value.
The objective is to guarantee that
error code values will not change in the future
when introducing new codes.
Intentionnal empty spaces and ranges are defined
in order to keep room for potential new codes.
 2017-07-13 17:12:16 -07:00
Nick Terrell
830ef4152a [libzstd] Increase granularity of FSECTable repeat mode 2017-07-13 12:45:39 -07:00
Yann Collet
d985319337 Merge pull request #759 from terrelln/real-block-split 
[libzstd] Pull CTables into sub-structure
 2017-07-13 10:24:19 -07:00
Yann Collet
3a60efd3a9 policy change : ZSTDMT automatically caps nbThreads to ZSTDMT_NBTHREADS_MAX (#760) 
Previously, ZSTDMT would refuse to create the compressor.
Also : increased ZSTDMT_NBTHREADS_MAX to 256,
updated doc,
and added relevant test
 2017-07-13 10:17:23 -07:00
Yann Collet
132e6efd76 switched ZSTDMT_compress_advanced() last argument to overlapLog 
overlapRLog (== 9 - overlapLog) was a bit "strange"
as all other public entry points use overlapLog
 2017-07-13 02:22:58 -07:00
Yann Collet
4e77f7761d clarified comment on ZSTD_p_contentSizeFlag 2017-07-13 02:09:07 -07:00
Nick Terrell
de0414b736 [libzstd] Pull CTables into sub-structure 2017-07-12 19:49:19 -07:00
Yann Collet
8ef666c325 slightly increased buffer pool, to cover normal "full load" scenarios 
2 buffers per active worker
+ 1 buffer for input loading
+ 1 buffer for "next input" when submitting current one
+ 1 buffer stuck in queue
 2017-07-12 14:23:34 -07:00
Yann Collet
052a95f77c fix : ZSTDMT_compress_advanced() correctly generates checksum 
when params.fParams.checksumFlag==1.
This use case used to be impossible when only ZSTD_compress() was available
 2017-07-11 17:18:26 -07:00
Yann Collet
2a62f48bf4 release input buffers from inside worker thread 
buffers are released sooner, which makes them available faster for next job.
=> decreases total nb of buffers necessary
 2017-07-11 15:56:40 -07:00
Yann Collet
57236184af buffer pool : all buffers have same size 
to reduce memory fragmentation.
They can be used for in or out, interchangeably.
 2017-07-11 15:17:25 -07:00
Yann Collet
34b2b95631 zstdmt : intermediate outBuffer allocated from within worker 
reduces total amount of memory needed,
since jobs in queue do not have an outBuffer pre-reserved now
 2017-07-11 14:59:10 -07:00
Yann Collet
16261e6951 buffer pool can be invoked from multiple threads 2017-07-11 14:14:07 -07:00
Yann Collet
ef0ff7fe7f zstdmt: removed margin for improved memory usage 2017-07-11 08:54:29 -07:00
Yann Collet
4616fad18b improved ZSTDMT_compress() memory usage 
does not need the input buffer for streaming operations

also : reduced a few tests time length
 2017-07-10 17:16:41 -07:00
Yann Collet
670b1fc547 optimized memory usage for ZSTDMT_compress() 
Previously, each job would reserve a CCtx right before being posted.
The CCtx would be "part of the job description",
and only released when the job is completed (aka flushed).
For ZSTDMT_compress(), which creates all jobs first and only join at the end,
that meant one CCtx per job.
The nb of jobs used to be == nb of threads,
but since latest modification,
which reduces the size of jobs in order to spread the load of difficult areas,
it also increases the nb of jobs for large sources / small compression level.
This resulted in many more CCtx being created.

In this new version, CCtx are reserved within the worker thread.
It guaranteea there cannot be more CCtx reserved than workers (<= nb threads).

To do that, it required to make the CCtx Pool multi-threading-safe :
it can now be called from multiple threads in parallel.
 2017-07-10 16:30:55 -07:00
Yann Collet
3510efb02d fix : custom allocator correctly propagated to child contexts 2017-07-10 14:21:40 -07:00
Yann Collet
88da8f1816 fix : propagate custom allocator to ZSTDMT though ZSTD_CCtx_setParameter() 
also : compile fuzzer with MT enabled
 2017-07-10 14:02:33 -07:00
Yann Collet
e32fb0c1fe added ZSTD_sizeof_CCtx() test 2017-07-10 12:29:57 -07:00
Yann Collet
40156a4967 bumped version nb to v1.3.1 2017-07-08 04:55:09 -07:00
Yann Collet
0f4fc6c20a fixed several conversion warnings 2017-07-07 17:13:12 -07:00
Yann Collet
9bde061a0b fixed minor Visual compilation limitation 2017-07-07 16:14:17 -07:00
Yann Collet
593d517ebf fixed minor cast warning 2017-07-07 16:09:47 -07:00
Yann Collet
ead4dd48f6 new field frameHeader.headerSize 2017-07-07 15:51:24 -07:00
Yann Collet
46396523c0 ZSTD_getFrameHeader : control of windowSize limits is delegated to caller 
Extracting frame header is a separate operation.
It's now possible to get frame header, whatever the window size set in it.
 2017-07-07 15:32:12 -07:00
Yann Collet
990449b89d new field : ZSTD_frameHeader.frameType 
Makes frame type (zstd,skippable) detection more straighforward.
ZSTD_getFrameHeader set frameContentSize=ZSTD_CONTENTSIZE_UNKNOWN to mean "field not present"
 2017-07-07 15:21:35 -07:00
Yann Collet
e622330a3b extended frameHeader.windowSize to unsigned long long 2017-07-07 14:19:01 -07:00
Yann Collet
f04deff4fc fixed #718, reported by @GregSlazinski, solution suggested by @mcmilk 2017-07-06 01:42:46 -07:00
Yann Collet
d75c0e71c4 minor code refactoring 2017-07-05 18:10:07 -07:00
Yann Collet
49af41820d clarified status of zstdmt_compress.h API 2017-07-05 17:21:13 -07:00
Yann Collet
27e883371d fixed wrong assert() condition 
A single job created by ZSTDMT_compress() can be < 256KB
if data to compress is < 256 KB
(in which case it is delegated to single thread mode)
 2017-07-04 19:33:16 -07:00
Yann Collet
2cb9774f5e more precise estimation of amount to flush at end of stream (single thread mode) 
also : can use DEBUGLEVEL variable in /tests
 2017-07-04 12:39:26 -07:00
Yann Collet
6383372dec fixed : 0-copy in NULL is UB 2017-07-04 10:36:41 -07:00
Yann Collet
5051dd39ca Merge pull request #743 from facebook/fullbench 
compress_generic() automatic optimization opportunities
 2017-07-03 21:26:38 -07:00
Yann Collet
2de2396a36 refactor ZSTDMT_compress() 2017-07-03 16:23:36 -07:00
Yann Collet
2084b041f4 fixed comments 2017-07-03 15:52:19 -07:00
Yann Collet
5a77361595 fixed wrong function name in comment 2017-07-03 15:21:24 -07:00
Nick Terrell
c80fc50a8d [libzstd] Fix memcpy() on potential NULL source 
* `ZSTD_decompressStream_generic()` `ip` may be `NULL` for one of the calls
  to `memcpy()`
* Assert the source is not `NULL` for calls to `memcpy()` where I believe
  the source should not be `NULL`.
 2017-07-03 12:31:55 -07:00
Yann Collet
2485f88bf8 fixed legacy version init bug 2017-07-01 09:09:34 -07:00
cyan4973
21fdf97e00 Merge branch 'dev' into fullbench 2017-07-01 07:01:08 -07:00
cyan4973
1bafe393e4 fix : ZSTDMT_compressStream_generic() can accept NULL input 
also : converge implementations towards new version of ZSTDMT_compressStream_generic()
 2017-07-01 06:59:24 -07:00
Yann Collet
58bd0e70fc fixed : dictionary compression with new advanced API in Multi-threading mode 2017-06-30 16:01:02 -07:00
Yann Collet
d8b33a598d Optimized ZSTDMT single-pass mode speed on large sources 
by ensuring job sizes remain "not too large"
 2017-06-30 15:44:57 -07:00
Yann Collet
d5c046c609 implemented shortcut for zstd_compress_generic() in MT mode 
added ZSTDMT_compress_advanced() API
 2017-06-30 14:51:01 -07:00
Yann Collet
7f40bb1c39 Merge pull request #742 from stellamplau/stack-space 
Reduce stack usage of HUF_readDTableX4 and HUF_readDTableX2
 2017-06-30 14:50:23 -07:00
Stella Lau
32df49e9f8 Fix typo 2017-06-30 12:56:24 -07:00
Stella Lau
b0513b519c Add comment to HUF_DECOMPRESS_WORKSPACE_SIZE 2017-06-30 12:53:56 -07:00
Stella Lau
4c71f59c77 Clarify typedef of rankVal_t and rankValCol_t 2017-06-30 09:52:20 -07:00
Stella Lau
28f711ef95 Rename ALIGN and ALIGN_MASK to HUF_ALIGN and HUF_ALIGN_MASK 2017-06-30 09:38:11 -07:00
Stella Lau
70ad6829e7 Delegate HUF_decompress4X_hufOnly to workspace version 2017-06-29 16:22:32 -07:00
Stella Lau
104c4d57c1 Fix bitshift error 2017-06-29 15:40:49 -07:00
Yann Collet
a3d9926c40 compression optimization opportunity 
switch to single-pass mode directly into output buffer
when outputSize >= ZSTD_compressBound(inputSize).
Speed gains observed with fullbench (~+15% on level 1)
 2017-06-29 14:44:49 -07:00
Stella Lau
fedc94de8c Fix pointer casting warning 2017-06-29 13:04:15 -07:00
Stella Lau
c6a5275a28 Fix alignment warnings with pointer casting 2017-06-29 12:39:34 -07:00
Stella Lau
99e315999c Reduce stack usage of HUF_readDTableX4 and HUF_readDTableX2 2017-06-29 11:49:59 -07:00
Yann Collet
97f2bf66da minor : fix typo 2017-06-29 11:31:40 -07:00
Yann Collet
acbef3decd ZSTD_getFrameContentSize() is promoted to "stable" status 2017-06-29 05:19:51 -07:00
Yann Collet
590937df20 Merge pull request #739 from facebook/refPrefix 
ZSTD_refPrefix
 2017-06-29 04:36:03 -07:00
Yann Collet
811deaea6f Merge pull request #736 from terrelln/cover-default-api 
[zdict] Make COVER the default algorithm
 2017-06-28 20:25:36 -07:00
Yann Collet
037466245f refactor ZSTD_check_compressionLevel_monotonicIncrease_memoryBudget() 
use less macro statements
the initial version was meant to work with STATIC_ASSERT
but since it doesn't work and needs assert()
it's possible to rewrite it using normally compiled code
which is better for compiler.

Downside : the error message is less precise.
There is a DEBUGLOG(3,) to compensate.
 2017-06-28 20:24:08 -07:00
Yann Collet
2bf428df45 Merge branch 'advancedAPI2' into refPrefix 2017-06-28 16:35:49 -07:00
Yann Collet
1ca76039af fixed -Wdeclaration-after-statement 2017-06-28 15:40:21 -07:00
Yann Collet
813535105b added function to control monotonic memory budget increase of ZSTD_defaultCParameters[0] 
It's a runtime test, based on assert(),
played once, on first ZSTD_getCParams() usage,
when ZSTD_DEBUG is enabled.
 2017-06-28 15:34:56 -07:00
Yann Collet
adbe74a8ac adjusted compression levels to guarantee a monotonically increasing memory budget 2017-06-28 13:22:37 -07:00
Yann Collet
33a6639039 fixed ZSTD_refPrefix with Multithread-enabled CCtx 2017-06-28 11:09:43 -07:00
Yann Collet
2e4274262d controlled dictMode 2017-06-27 17:09:12 -07:00
Yann Collet
b7372933b8 implemented ZSTD_refPrefix() 2017-06-27 15:49:12 -07:00
Yann Collet
7d3816183f exposed ZSTD_MAGIC_DICTIONARY in zstd.h 
makes it easier to explain ZSTD_dictMode
 2017-06-27 13:50:34 -07:00
Yann Collet
fecc721fd9 added parameter ZSTD_p_refDictContent 2017-06-27 11:46:39 -07:00
Nick Terrell
5b7fd7c422 [zdict] Make COVER the default algorithm 2017-06-26 21:09:22 -07:00
Yann Collet
c7fb884eea fixed minor conversion warning 2017-06-26 18:02:23 -07:00
Yann Collet
dde10b23fe refactored ZSTD_estimateDStreamSize() 
now uses windowSize as argument.
Also : created ZSTD_estimateDStreamSize_fromFrame()
 2017-06-26 17:44:26 -07:00
Yann Collet
09ae03a570 ZSTD_estimateCDictSize_advanced() 
ZSTD_estimateCDictSize() now uses same arguments as ZSTD_createCDict()
ZSTD_estimateCDictSize_advanced() uses same arguments as ZSTD_createCDict_advanced()
 2017-06-26 16:47:32 -07:00
Yann Collet
0c9a915a28 ZSTD_estimateCStreamSize_advanced() 2017-06-26 16:02:25 -07:00
Yann Collet
31af8290d1 ZSTD_estimateCCtx_advanced() 
ZSTD_estimateCCtx() is now a "simple" function,
taking int compressionLevel as single argument.

ZSTD_estimateCCtx_advanced() takes a CParams argument,
which is both more complete and more complex to generate.
 2017-06-26 15:52:39 -07:00
Yann Collet
ef269c1b68 Merge pull request #725 from facebook/advancedAPI2 
New Advanced API
 2017-06-23 09:50:47 -07:00
Yann Collet
ecb0f46866 add controls over streaming buffers 2017-06-21 17:25:01 -07:00
Yann Collet
dce789281b fixed : decompression of skippable frames in streaming mode 2017-06-21 15:53:42 -07:00
Yann Collet
204b6b7ef6 fixed streaming buffered allocation with CDict compression 2017-06-21 15:13:00 -07:00
Yann Collet
1e4129b27b fixed dangling pointer risk, detected by @terrelln 2017-06-21 13:26:10 -07:00
Yann Collet
83095970e6 free cdictLocal faster, suggested by @terrelln 2017-06-21 12:26:40 -07:00
Yann Collet
7bd1a2900e added ZSTD_dictMode_e to control dictionary loading mode 2017-06-21 11:50:33 -07:00
Yann Collet
9c56b12938 Merge pull request #723 from paulcruz74/dev 
Adding zstd -l
 2017-06-21 09:41:55 -07:00
Yann Collet
e51d51bdf7 fixed memcpy() overlap 2017-06-20 17:44:55 -07:00
Yann Collet
466f92eaa6 removed one useless streaming compression stage, detected by @terrelln 2017-06-20 16:25:29 -07:00
Yann Collet
c3bce24ef4 fixed potential dangling pointer, detected by @terrelln 2017-06-20 16:09:11 -07:00
Yann Collet
78b8234554 fixed comments, following suggestion by @terrelln 2017-06-20 14:26:48 -07:00
Yann Collet
b44ab82f7a ensure new ZSTD_strategy starts at value 1 2017-06-20 14:11:49 -07:00
Yann Collet
c08e649e95 first implementation of bench.c with new API ZSTD_compress_generic() 
Doesn't speed optimize this buffer-to-buffer scenario yet.
Still internally defers to streaming implementation.

Also : fixed a long standing bug in ZSTDMT streaming API.
 2017-06-19 18:25:35 -07:00
Yann Collet
695a0a3449 fixed IA64 compilation error, by @mcmilk 2017-06-19 15:27:30 -07:00
Yann Collet
fe234bf48b fix attempts : fullbench for VS2008 2017-06-19 15:23:19 -07:00
Nick Terrell
55f9cd4942 [libzstd] Fix UBSAN failure 2017-06-19 15:12:28 -07:00
Yann Collet
bf99150be3 update new api presentation in zstd.h and manual 2017-06-19 12:56:25 -07:00
Yann Collet
d7a3bffba9 new api : setting compression parameters is refused if a dictionary is already loaded 2017-06-19 11:53:01 -07:00
Yuri
92bafda406 INSTALL_DATA instead of INSTALL_LIB for libzstd.a 
INSTALL_LIB can be passed -s flag to strip symbols. Static libraries should not be stripped, only dynamic ones should be stripped.
 2017-06-17 00:23:41 -07:00
Yann Collet
381e66cfbd added ZSTD_clampCParams() 
now ZSTD_adjustCParams() is always successful,
it always produces a valid CParams
 2017-06-16 17:34:54 -07:00
Yann Collet
aee916e37c fixed +/-1 error for pledgedSrcSizePlusOne 2017-06-16 17:02:35 -07:00
Yann Collet
d3de3d51a3 fix attempt 2 : Visual sign conversion warning 2017-06-16 16:51:33 -07:00
Yann Collet
944be54774 fixed attempt : minor Visual sign conversion warning 2017-06-16 14:05:01 -07:00
Yann Collet
b26728c9c8 added ZSTD_startNewCompression() 2017-06-16 14:00:46 -07:00
Yann Collet
a0ba849fe6 changed frameContentSize field to pledgedSrcSizePlusOne 
pledgedSrcSize is proper : it's a promise, not yet fulfilled.
It will be controlled at the end.

PlusOne is meant to have 0 (default) == unknown
 2017-06-16 13:29:17 -07:00
Yann Collet
2cf7755da7 fix : pledgedSrcSize correctly reset to unknown in "continue" mode 2017-06-16 12:34:41 -07:00
Yann Collet
9e73f2f320 fix : correctly reset pledgedSrcSize to unknown status 
when starting a new compression with an existing context
 2017-06-16 12:24:01 -07:00
Yann Collet
33873f0e74 fixed : new advanced AIP : setting nbThreads to the same value > 1 2017-06-16 12:04:21 -07:00
Yann Collet
559ee82e90 fixed : calling ZSTD_compress_generic() to end-flush a stream in multiple steps 2017-06-16 11:58:21 -07:00
Yann Collet
bd18c885a3 added ZSTD_CCtx_reset 2017-06-16 10:17:50 -07:00
Yann Collet
cc9f9b7f4c protection : ZSTD_CONTENTSIZE_UNKNOWN automatically disables contentSizeFlag 2017-06-15 18:17:34 -07:00
Yann Collet
05ae4b2190 added protection : MT incompatible with Static allocation 2017-06-15 18:03:34 -07:00
Paul Cruz
a9b77c83e5 cleaning up code for analyzing frames 2017-06-15 14:13:28 -07:00
Yann Collet
f129fd3970 disabled MT code path when ZSTD_MULTITHREAD is not defined 2017-06-11 18:46:09 -07:00
Yann Collet
23aace9778 added control stage to MT mode 2017-06-11 18:32:36 -07:00
Yann Collet
f35e2de61c linked newAPI to ZSTDMT 2017-06-05 18:32:48 -07:00
cyan4973
c59162e053 minor fix for -Wdocumentation 2017-06-05 00:12:13 -07:00
cyan4973
8bcbf42617 fixed g++ prototype mismatch 2017-06-04 23:52:00 -07:00
Yann Collet
8c910d2097 updated ZSTDMT streaming API 
ZSTDMT streaming API is now similar
and has same capabilites as single-thread streaming API.
It makes it easier to blend them together.
 2017-06-03 01:15:02 -07:00
Yann Collet
58e8d793e1 made debug definitions common within zstd_internal.h 2017-06-02 18:20:48 -07:00
Yann Collet
8ddf4c22d5 fixed missing initialization 2017-06-02 17:16:49 -07:00
Yann Collet
33a7e679e5 significant zlib wrapper code refactoring 
code indentation
variable scope and names
constify

Only coding style changes.
The logic should remain the same.
 2017-06-02 17:10:49 -07:00
Yann Collet
4effccbf56 zlib_wrapper's uncompress() uses ZSTD_isFrame() for routing 
more generic and safer than using own routing for magic number comparison
 2017-06-02 14:27:11 -07:00
Yann Collet
dcb7535352 ensure zlibwrapper uses ZSTD_malloc() and ZSTD_free() 
which is compatible with { NULL, NULL, NULL }
 2017-06-02 14:01:21 -07:00
Yann Collet
b877e834b1 minor indent 2017-06-02 13:47:11 -07:00
Yann Collet
6056e4c3eb added POOL_sizeof() for single-thread 2017-06-02 11:36:47 -07:00
Yann Collet
c35e535002 added support for multithreading parameters 2017-06-01 18:44:06 -07:00
Yann Collet
c4a5a21c5c created ZSTDMT_sizeof_CCtx() and POOL_sizeof() 
required by ZSTD_sizeofCCtx() while adding a ZSTDMT_CCtx*
 2017-06-01 17:56:14 -07:00
Yann Collet
cd2892fd1e protected impossible switch(){default:} with assert(0) 
can be converted into assume(0) in some future
 2017-06-01 09:44:54 -07:00
Yann Collet
06589fe516 Merge branch 'advancedAPI2' of github.com:facebook/zstd into advancedAPI2 2017-05-31 10:03:20 -07:00
Yann Collet
18ab5affa5 fixed visual warning 2017-05-31 09:59:22 -07:00
Yann Collet
9a691e0f55 fixed visual warnings 2017-05-31 01:17:44 -07:00
Yann Collet
01b1549f83 finally converted ZSTD_compressStream_generic() to use {in,ou}Buffer 
replacing the older read/write variables from ZBUFF_* era.
Mostly to help code readability.

Fixed relevant callers.
 2017-05-30 18:10:26 -07:00
Yann Collet
c4f46b94ce ZSTD_createCCtx_advanced() now uses ZSTD_calloc() 
initially uses calloc() instead of memset().

Performance improvement is unlikely measurable,
since ZSTD_CCtx is now very small,
with all tables transferred into workSpace.
 2017-05-30 17:45:37 -07:00
Yann Collet
deee6e523f expose ZSTD_compress_generic_simpleArgs() 
which is a binding towards ZSTD_compress_generic()
using only integral types for arguments.
 2017-05-30 17:42:00 -07:00
Yann Collet
ae728a43b8 removed defaultCustomMem 
now ZSTD_customCMem is promoted as new default.

Advantages : ZSTD_customCMem = { NULL, NULL, NULL},
so it's natural default after a memset.

ZSTD_customCMem is public constant
(defaultCustomMem was private only).

Also : makes it possible to introduce ZSTD_calloc(),
which can now default to stdlib's calloc()
when it detects system default.

Fixed zlibwrapper which depended on defaultCustomMem.
 2017-05-30 17:11:39 -07:00
Yann Collet
5bcef1ada2 removed mtctx->cstream 
use the first cctx in pool when ZSTDMT is used in single-thread mode
now that cctx and cstream are the same object.
 2017-05-30 16:37:19 -07:00
Yann Collet
beb62b15a8 Merge branch 'dev' into advancedAPI2 
Fixed conflic in zstd_decompress.c
 2017-05-30 16:18:57 -07:00
Yann Collet
44e45e8423 added ZSTDMT_createCCtx_advanced() 
make it possible to use custom allocators
 2017-05-30 16:12:06 -07:00
Yann Collet
f45ca527a1 Merge branch 'advancedAPI2' of github.com:facebook/zstd into advancedAPI2 2017-05-30 10:02:03 -07:00