AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
3,201 Commits 1 Branch 0 Tags 27 MiB
89df466b2f
Commit Graph

1426 Commits

Author SHA1 Message Date
Yann Collet
e42afbc6fa Comply with suggested comments by @terrelln 
created FSE_CTABLE_SIZE() and FSE_DTABLE_SIZE()
 2017-04-26 11:39:35 -07:00
Yann Collet
7271203bdb transferred entropy scratch space from CCtx into workSpace 
Saved 6 KB
 2017-04-20 23:21:19 -07:00
Yann Collet
a408645f50 made some room for entropy scratch space 2017-04-20 23:09:39 -07:00
Yann Collet
71aaa32c3c transferred FSE tables from CCtx into workspace 
Saved 5 KB from CCtx
 2017-04-20 23:03:38 -07:00
Yann Collet
71ddeb67b1 made room in workspace for FSE tables 
still need to be transferred from CCtx into workspace
 2017-04-20 22:54:54 -07:00
Yann Collet
a34a39c183 changed size evaluation of entropy tables 
so that memcpy() does no longer depends on fse pointer being a static table
 2017-04-20 18:26:25 -07:00
Yann Collet
7bb60b17d8 init entropy table pointers only once 
per workSpace resize
 2017-04-20 17:38:56 -07:00
Yann Collet
e6fa70a0a1 reorganized ZSTD_resetCCtx_internal() 
clearer separation between variables and buffers
clearer buffers category
kept static buffers at the beginning, favoring cache locality
(it will be easier to add FSE tables there later)

This break a few assumptions that hashTable was always at the beginning.
This is fixed.
And remaining assumptions (namely that tables stand next to each other in memory)
are now tested with assert.
 2017-04-20 17:28:31 -07:00
Yann Collet
c17e020c9a disable assert when compiling paramgrill 
paramgrill is a benchmark calibration function.
Speed accuracy is critical, it cannot be altered by assert.
 2017-04-20 12:50:02 -07:00
Yann Collet
16f9c572fc Merge branch 'dev' into compressionFlow 2017-04-20 11:16:40 -07:00
Yann Collet
e348dad305 minor long line reformatting 2017-04-20 11:14:13 -07:00
Yann Collet
e847730452 slightly refined README comments on lib-mt 2017-04-18 23:15:28 -07:00
Yann Collet
2c5514c759 fixed ZSTDMT_initCStream_advanced() 
Must use the new ZSTD_compressBegin_usingCDict_advanced()
to enforce correct frame parameters
 2017-04-18 22:52:41 -07:00
Sean Purcell
98cf7fcb2a Update README 2017-04-18 17:03:37 -07:00
Yann Collet
a4cab80183 added ZSTD_copyCCtx_internal() 
which respects provided fParams.
 2017-04-18 14:54:54 -07:00
Sean Purcell
ca6fae7808 Add MT enabled targets for libzstd 2017-04-18 14:13:01 -07:00
Yann Collet
30fb499208 Changed ZSTD_resetCCtx_advanced() into ZSTD_resetCCtx_internal() 
for naming consistency :
_advanced() can be invoked
while _internal() are strictly static
 2017-04-18 14:08:50 -07:00
Yann Collet
715b9aa113 created ZSTD_compressBegin_usingCDict_advanced() 2017-04-18 13:55:53 -07:00
Yann Collet
af4f45b682 Improved code comments for block functions 2017-04-18 03:17:44 -07:00
Yann Collet
4f818182b8 clarified frame parameters for ZSTD_compress*_usingCDict() 
created ZSTD_compressBegin_usingCDict_internal(),
which gives direct control to frame Parameters.
ZSTD_resetCStream_internal() now points into it.
 2017-04-17 18:29:06 -07:00
Yann Collet
c47c68f6ca proper evaluation of Huffman CTable size 2017-04-17 16:14:21 -07:00
Yann Collet
88009a8ba2 removed srcSize control from CStream 
since it's already done from lower bufferless API level
 2017-04-12 00:51:24 -07:00
Yann Collet
20d5e03893 content size is controlled at bufferless level 
so it's active for all entry points

Also : added relevant test (wrong content size) in fuzzer
 2017-04-11 18:34:02 -07:00
Yann Collet
4ee6b15dac force contentSizeFlag=0 when using ZSTD_initCStream_usingCDict() 
because by definition srcSize is not known when using this prototype.
added relevant test

Note : this use was already working, because at a later stage
(both ZSTD_compressBegin_usingCDict() and ZSTD_copyCCtx())
pledgedSrcSize=0 is translated into "unknown", no matter the frame parameter.
This is not correct, but of little importance,
as the medium term plan is to no longer set fParams within CDict
 2017-04-11 11:59:44 -07:00
Yann Collet
ab9162ebb4 simplified call graph 
by calling ZSTD_compressBegin_internal() instead of ZSTD_compressBegin_advanced()
 2017-04-11 10:46:20 -07:00
Yann Collet
e88034fe26 simplified ZSTD_initCStream*() flow 
all variants converge towards ZSTD_initCStream_stage2()
 2017-04-10 22:24:02 -07:00
Yann Collet
4b987ad8ce Introduce ZSTD_initCStream_internal() 
This is now the regroup point for ZSTD_initCStream*() functions

ZSTD_initCStream_advanced() now properly checks for parameters validity.

Also : added <assert.h> usage inside zstd_compress.c
Needs ZSTD_DEBUG=1 macro to be triggered.
Will be triggered by default from `tests` directory
 2017-04-10 17:50:44 -07:00
Yann Collet
0181fef545 ensure cctx internal buffer is correctly sized in case of memory error 2017-04-06 01:25:26 -07:00
Yann Collet
36c2a03757 updated comments for ZSTD_resetCStream() 2017-04-05 22:06:21 -07:00
Yann Collet
003a244324 DStream : ensure correct size of internal buffers in case of error 2017-04-05 15:28:56 -07:00
Yann Collet
02d37aa1c1 ensure correct size of internal buffers in case of error 2017-04-05 14:53:51 -07:00
Nick Terrell
405d2a1027 Explicitly convert scratchBuffer to unsigned* 2017-04-04 16:35:31 -07:00
Nick Terrell
16a739cab0 Switch call of FSE_count() to FSE_count_wksp() 2017-04-04 16:17:21 -07:00
Yann Collet
7cf78f1be7 Protects ZSTD_compressBegin_usingCDict() vs NULL cdict dereference 
Will issue an error (GENERIC) is cdict==NULL
 2017-04-04 12:38:14 -07:00
Nick Terrell
26b046a7c4 Remove unnecessary dictID store 2017-04-03 21:46:28 -07:00
Nick Terrell
39a6cc5172 Make ZSTD_compress_usingCDict() respect contentSizeFlag 2017-04-03 21:09:55 -07:00
Nick Terrell
62ecad3819 Fix ZSTD_initCStream_usingCDict() to use dictionary 2017-04-03 21:05:59 -07:00
Yann Collet
30c7698970 optimize ZSTDMT_compress() memory usage 
does no longer allocate temporary buffers
when there is enough room in dstBuffer to decompress directly there.
(previous method would skip that for 1st chunk only).

Also : fix ZSTD_compressBound() for small srcSize
 2017-03-31 18:27:03 -07:00
Yann Collet
3f75d52527 Changed ZSTD_compressBound() 
required so that if Total = A+B
compressBound(Total) <= compressBound(A) + compressBound(B)
under condition of a minimum size for A and B

Will help for ZSTDMT_compress() memory allocation
 2017-03-31 17:11:38 -07:00
Yann Collet
7b70a1969e Merge branch 'dev' into zstdmt 2017-03-31 16:22:33 -07:00
Yann Collet
53203e7c38 Merge pull request #640 from facebook/memAccess 
Changed memory strategy to __packed for gcc
 2017-03-31 15:49:12 -07:00
Yann Collet
eea7858e2b fixed minor warnings in debug code 2017-03-30 16:47:19 -07:00
Yann Collet
34cc487d05 overlap at full windowSize for max compression level 
as it provides max compression ratio
 2017-03-30 16:23:22 -07:00
Yann Collet
458e955c23 improved ZSTDMT_compress() 
Use a bit more threads by default.
Uses overlap segments to boost compression ratio (like the streaming variant)
 2017-03-30 15:51:58 -07:00
Yann Collet
6476c51b86 Merge pull request #637 from facebook/zstdmt 
Zstdmt
 2017-03-30 14:18:37 -07:00
Yann Collet
274f59919d Changed memory strategy to __packed for gcc 
Method 1 __packed is always as good or better than memcpy().
But it's not portable, as it depends on compiler extension.

For gcc, __pakced directive works fine.
Furthermore, gcc has serious performance issues with memcpy() on ARM 32 bits.
See #620
 2017-03-30 12:52:14 -07:00
Nick Terrell
5152fb2cb2 Convert all tabs to spaces 2017-03-29 18:51:58 -07:00
Yann Collet
ca5a8bbe36 re-added patch ... 2017-03-29 17:15:27 -07:00
Yann Collet
2e2e78de47 removed unnecessary restriction on minmatchLength 
it's now transparently translated to nearest value when unsupported
(7->6) (3->4)
 2017-03-29 16:02:47 -07:00
Yann Collet
26769d88bc Merge branch 'dev' of github.com:facebook/zstd into dev 2017-03-29 15:21:30 -07:00