AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
8,454 Commits 1 Branch 0 Tags 27 MiB
master
Commit Graph

8454 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Terrell
6d5dc93d4e [lib] Compress directly into output when ZSTD_c_stableOutBuffer is set 
When we have a stable output buffer always compress directly into the
`ZSTD_outBuffer`. We are allowed to return `dstSizeTooSmall`.
 2020-10-30 10:55:34 -07:00
Nick Terrell
987cb4ca6a [lib] Take the shortcut when ZSTD_c_stableOutBuffer is set 
When we have a stable output buffer take the single-pass shortcut.
It is okay to return `dstSizeTooSmall` if the output buffer isn't
big enough, because we know it will never grow.
 2020-10-30 10:55:34 -07:00
Nick Terrell
809b2f2071 [lib] Set ZSTD_c_stable{In,Out}Buffer in ZSTD_compress2() 
Sets these parameters in ZSTD_compress2() then resets them to their
orignal values after the compression call.

An alternative design could be to add a flush mode `ZSTD_e_singlePass`
which implies `ZSTD_c_stable{In,Out}Buffer` but only for a single
compression call, by directly setting the applied parameters. I've opted
for the smaller change, but this is open for discussion.
 2020-10-30 10:55:34 -07:00
Nick Terrell
c74be3f6de [lib] Validate buffers when ZSTD_c_stable{In,Out}Buffer is set 
Adds the validation of the input/output buffers only. They are still
unused.
 2020-10-30 10:55:34 -07:00
Nick Terrell
e3e0775cc8 [API] Add ZSTD_c_stable{In,Out}Buffer parameters 
This commit adds the parameters and sets the value in the CCtxParams
but it does not do anything with the value.
 2020-10-30 10:54:39 -07:00
Nick Terrell
e2581d9572 [lib] Set appliedParams in zstdmt mode 
Previously only `nbWorkers` was set. Set all parameters, because that is
what is expected. This is needed for the `ZSTD_c_stable{In,Out}Buffer`
parameters.
 2020-10-30 10:54:38 -07:00
senhuang42
f0da97642a Specify that getSequences() will always emit block boundary sequences 2020-10-30 12:31:17 -04:00
senhuang42
536e89c723 Sequence extractor should update CBlockState 2020-10-30 12:13:19 -04:00
senhuang42
32cac2627a Emit last literals of 0 size as well, to indicate block boundary 2020-10-29 16:41:17 -04:00
senhuang42
69bd5f0654 Correct literalsRead calculation to include longLength 2020-10-29 14:49:37 -04:00
sen
ff93440fc6
Merge pull request #2375 from senhuang42/ldm_oss_fuzz_testcase 
Add a test case for LDM + opt parser with small uncompressible block
 2020-10-29 09:32:05 -04:00
senhuang42
7198ebb213 Un-mix declarations and code 2020-10-28 18:51:03 -04:00
senhuang42
59624f3163 Remove implicit typecast to appease appVeyor windows build 2020-10-28 16:25:09 -04:00
senhuang42
60a52c29e6 Add check for allocation 2020-10-28 16:22:22 -04:00
Yann Collet
bb4ea40e4d
Merge pull request #2373 from facebook/libzstd_autoconf_full 
Automatic detection of configuration changes
 2020-10-28 11:57:08 -07:00
Yann Collet
09e3bb95d2 Merge branch 'dev' into libzstd_autoconf_full 2020-10-28 10:53:08 -07:00
Yann Collet
0adce4631d Merge branch 'libzstd_autoconf_full' of github.com:facebook/zstd into libzstd_autoconf_full 2020-10-28 10:25:55 -07:00
Yann Collet
f6ecf1568f minor Makefile refactor 
hopefully improving readability
 2020-10-28 09:39:15 -07:00
senhuang42
3ed5d053d8 Clarify comments in zstd.h some more 2020-10-28 09:53:09 -04:00
Nick Terrell
599ff58e08
Merge pull request #2339 from terrelln/zstdmt-stability 
Fix zstdmt stability issues and clean up the zstdmt code
 2020-10-27 19:43:13 -07:00
Yann Collet
ceccd7ae2d Merge branch 'dev' into libzstd_autoconf_full 2020-10-27 15:45:30 -07:00
Yann Collet
2d2507b9db
Merge pull request #2374 from bket/portability 
'head -c BYTES' is non-portable
 2020-10-27 14:15:35 -07:00
senhuang42
169fc07aa1 Move test to appropriate location 2020-10-27 16:59:43 -04:00
senhuang42
db0b5d7d1e Add test to fuzzer.c 2020-10-27 16:57:24 -04:00
sen
17b700d78a
Merge pull request #2366 from senhuang42/enable_ldm_by_default 
Enable LDM by default if window size >= 128MB and strategy uses opt parser
 2020-10-27 14:59:28 -04:00
Nick Terrell
0953645837
Merge pull request #2362 from senhuang42/fix_ldm_fuzz_issue 
Fix long distance matcher OSS-fuzz issue
 2020-10-27 11:13:03 -07:00
senhuang42
3163909d14 Remove unused variable position 2020-10-27 12:58:12 -04:00
senhuang42
dc448563e9 Add test compatibility with last literals in sequences 2020-10-27 12:35:28 -04:00
Björn Ketelaars
1f661b5f6b 'head -c BYTES' is non-portable 2020-10-27 16:55:23 +01:00
senhuang42
1d221ecc03 Add support for representing last literals in the extracted seqs 2020-10-27 11:19:48 -04:00
senhuang42
9171f920cd Improve documentation of seqStore_t 2020-10-27 10:50:22 -04:00
senhuang42
96b0ff7886 Improve documentation regarding various operations in copyBlockSequences 2020-10-27 10:36:06 -04:00
senhuang42
3a11c7eb03 Modify ZSTD_copyBlockSequences to agree with new API 2020-10-27 10:31:40 -04:00
senhuang42
761f40d1c6 Clarify and modify ZSTD_Sequence definition 2020-10-27 09:41:32 -04:00
Yann Collet
59ab4dcd32 Merge branch 'dev' into libzstd_autoconf_full 2020-10-26 15:17:39 -07:00
Yann Collet
284d84ecf7
Merge pull request #2370 from facebook/libzstd_autoconf 
Automatic configuration detection
 2020-10-26 15:16:04 -07:00
Yann Collet
456db0c377 make install only rebuild binaries if they don't exist 
Now `make` followed by `make install` doesn't rebuild binaries

also : only generated target directories if they don't already exist
 2020-10-23 16:46:49 -07:00
Yann Collet
a6ee614a44 make zstd is now differentiated from zstd-nomt 
avoids mixing object files using different flags
 2020-10-23 16:08:21 -07:00
Yann Collet
89b961ea46 simplified silent mode maintenance 2020-10-23 10:41:17 -07:00
Yann Collet
ffe8d9e428 fix partial lib test 2020-10-23 10:27:12 -07:00
Yann Collet
b5d4728713 simplified silent mode 2020-10-23 10:22:52 -07:00
Yann Collet
a7ad05bf57 fixed building libzstd with manual BUILD_DIR 
and when HASH is not found
 2020-10-23 10:14:04 -07:00
Yann Collet
d3f1a9b5bd fix partial-build test 
sometimes, the scope difference is solely determined by the list of source files,
not by the flags.
 2020-10-22 21:36:09 -07:00
Yann Collet
98787a3018 Merge branch 'dev' into libzstd_autoconf 2020-10-22 18:55:30 -07:00
Yann Collet
a912ef0952 can integrate later dynamic flags changes 
for example `libzstd-mt` is `differentiated from `libzstd`
 2020-10-22 18:48:06 -07:00
Yann Collet
f90424da2d
Merge pull request #2368 from facebook/progressive_libzstd 
faster rebuild of libzstd
 2020-10-22 17:36:56 -07:00
Yann Collet
ce6cd07c33 updated build documentation 2020-10-22 12:31:23 -07:00
Yann Collet
e3867fb735 fixed libzstd.dll compilation on mingw 
and zstd linking
 2020-10-22 11:52:19 -07:00
Yann Collet
91a8cb9559 fix DEBUGLEVEL redefinition from tests/ 2020-10-22 00:20:40 -07:00
Yann Collet
494f7169ed fix directory creation for Windows' libzstd 2020-10-22 00:15:31 -07:00