AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
5,937 Commits 1 Branch 0 Tags 27 MiB
596f7d1256
Commit Graph

5937 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
W. Felix Handte
596f7d1256 Fix #1412: Perform Signed Comparison When Setting Attach Dict Param 2018-11-12 12:07:57 -08:00
Yann Collet
7b0c551bff
Merge pull request #1411 from facebook/prefetch_dict 
Improves decompression speed when using cold dictionary
 2018-11-09 11:31:35 -08:00
Yann Collet
1b4a9c518b
Merge pull request #1410 from facebook/prefetch_dec 
improve long-range decoder speed
 2018-11-08 18:41:58 -08:00
Yann Collet
483759a3de Improves decompression speed when using cold dictionary 
by triggering the prefetching decoder path
(which used to be dedicated to long-range offsets only).

Figures on my laptop :
no content prefetch : ~300 MB/s (for reference)
full content prefetch : ~325 MB/s (before this patch)
new prefetch path : ~375 MB/s (after this patch)

The benchmark speed is already significant,
but another side-effect is that this version
prefetch less data into memory,
since it only prefetches what's needed, instead of the full dictionary.

This is supposed to help highly active environments
such as active databases,
that can't be properly measured in benchmark environment (too clean).

Also :
fixed the largeNbDict test program
which was working improperly when setting nbBlocks > nbFiles.
 2018-11-08 17:00:23 -08:00
Yann Collet
a801ba643c
Merge pull request #1409 from terrelln/tsan-dict 
Add TSAN dictionary tests
 2018-11-08 13:43:01 -08:00
Yann Collet
20fb9e7f36 reduced assertion strength 
one limit case can apparently be generated during fuzzer tests
 2018-11-08 12:57:34 -08:00
Yann Collet
f40c6b4395 added a simple test case for decode-only benchmark mode 
as it was broken and no one noticed
 2018-11-08 12:51:42 -08:00
Yann Collet
9126da5b5c improve long-range decoder speed 
on enwik9 at level 22 (which is almost a worst case scenario),
speed improves by +7% on my laptop (415 -> 445 MB/s)
 2018-11-08 12:47:46 -08:00
Yann Collet
8bed4012bd fixed decompression-only benchmark 2018-11-08 12:36:39 -08:00
Nick Terrell
a40b6958c9 Run TSAN fuzztest 2018-11-08 10:59:01 -08:00
Nick Terrell
103d1ee7a4 Add multithreaded dictbuilder tests to fuzzer.c 2018-11-08 10:58:51 -08:00
Nick Terrell
a8daa2d683 Signal before unlocking in pool.c 2018-11-08 10:45:53 -08:00
Yann Collet
5b598bb203
Merge pull request #1408 from szreder/dev 
Prevent deadlock on malloc() failure.
 2018-11-08 10:43:40 -08:00
Bartosz Szreder
5c5c476338 Prevent deadlock on malloc() failure. 2018-11-08 10:29:31 +01:00
Yann Collet
f6eb12084d
Merge pull request #1405 from pitrou/allow_cmake_debug_builds 
Allow creating debug builds with CMake
 2018-11-06 13:58:50 -08:00
Antoine Pitrou
2186fbcf67 Allow creating debug builds with CMake 
Only set the build type to Release if the user didn't select any.
See discussion in https://github.com/facebook/zstd/pull/1278#issuecomment-436327923
and below.
 2018-11-06 21:24:29 +01:00
Yann Collet
e0701d3c5d
Merge pull request #1404 from facebook/T36302429 
fixed T36302471
 2018-11-06 11:53:20 -08:00
Yann Collet
af892e5f1c
Merge pull request #1403 from facebook/assert_opt 
just add an assert() in ZSTD_insertBtAndGetAllMatches()
 2018-11-05 17:57:30 -08:00
Yann Collet
3e5cdf1b6a fixed T36302429 2018-11-05 17:50:30 -08:00
Yann Collet
2caa995558 just add an assert() in ZSTD_insertBtAndGetAllMatches() 
to express a condition on ll0 .
May help static analyzer as in #1397
 2018-11-05 17:13:32 -08:00
Yann Collet
3a90229616
Merge pull request #1395 from facebook/decompressblock 
created zstd_decompress_block module
 2018-10-29 16:28:09 -07:00
Yann Collet
2a0c9e0806
Merge pull request #1398 from bmwiedemann/pgo 
Fix building zstd-pgo
 2018-10-29 16:00:26 -07:00
Yann Collet
acd75a1448 fixed a second memset() on NULL 
not sure why it only triggers now,
this code has been around for a while.

Introduced a new error code : dstBuffer_null,
I couldn't express anything even remotely similar with existing error codes set.
 2018-10-29 15:03:57 -07:00
Yann Collet
9c58098200 fixed memcpy() on NULL warning 
memcpy(NULL, src, 0) is undefined behavior.
 2018-10-29 13:57:37 -07:00
Yann Collet
ea966c8fb1
Merge pull request #1396 from facebook/huf_refactor 
refactor HUF_compress_internal for clarity
 2018-10-29 13:06:45 -07:00
Bernhard M. Wiedemann
572279e73e pgo: also optimize compression 
and clean *.o so that linking does not fail from code
that is partially augmented with coverage generation instructions
 2018-10-29 17:20:03 +01:00
Bernhard M. Wiedemann
a38ad53ace zstd-pgo: do not make clean and zstd in parallel 
otherwise, there might not be a ./zstd to run
 2018-10-29 15:11:28 +01:00
Yann Collet
fc20b3c441 added flag -Wc++-compat 
for library and cli
 2018-10-26 16:38:23 -07:00
Yann Collet
4860311d99 renamed NEWS as CHANGELOG 
to conform fb publication rules.
 2018-10-26 15:57:13 -07:00
Yann Collet
9bf00707c7 minor clarifications of history update rules 2018-10-26 15:51:51 -07:00
Yann Collet
c3c7deb1e1
Merge pull request #1392 from coetry/dev 
provide consistent spacing to enum field
 2018-10-26 15:29:07 -07:00
Yann Collet
0db109c041
Merge pull request #1394 from terrelln/circleci-packaging 
Publish uncompressed, zstd, and gzipped packages
 2018-10-26 15:28:44 -07:00
Yann Collet
1866bd374a Merge branch 'dev' into huf_Refactor 2018-10-26 15:25:01 -07:00
Yann Collet
8d56f4baee added a few comments for clarifications 2018-10-26 15:21:52 -07:00
Yann Collet
450356b5af Merge branch 'dev' into decompressblock 2018-10-26 15:03:43 -07:00
Yann Collet
7d4960a5e8
Merge pull request #1390 from facebook/nullAsOutput 
support decompressing an empty frame into NULL
 2018-10-26 14:43:16 -07:00
Allen Hai
3783720f70 vertically align code comment 2018-10-26 16:16:06 -05:00
Yann Collet
7b74405150 refactor HUF_compress_internal for clarity 
changed workspace parameter convention
to always provide workspaceSize,
so that size can be explicitly checked.

Also, use more enum to make the meaning of some parameters more explicit.
 2018-10-26 13:21:37 -07:00
Nick Terrell
191e19aff8 Publish uncompressed, zstd, and gzipped packages 2018-10-26 13:06:53 -07:00
Yann Collet
4ed9233db6 fixed VS2010 project 2018-10-26 11:25:40 -07:00
Yann Collet
b832498aee fixed : extraneous function prototype in fullbench.c 
now provided through zstd_internal.h
 2018-10-26 10:39:49 -07:00
Allen Hai
26e34d8a73 provide consistent spacing to enum field 2018-10-25 18:45:20 -05:00
Yann Collet
2b4914082e created zstd_decompress_block module 
isolate all logic associated with block decompression
into its own module.

zstd_decompress is still in charge
of context creation/destruction,
frames, headers, streaming, special blocks, etc.

Compressed blocks themselves are now handled within zstd_decompress_block .
 2018-10-25 16:28:41 -07:00
Yann Collet
e26126caca
Merge pull request #1388 from facebook/ddict 
separate DDict logic into its own module
 2018-10-25 10:20:40 -07:00
Yann Collet
cb320a9fc0 added comment on public ddict functions 2018-10-24 16:50:03 -07:00
Yann Collet
806a5c84e4 support decompressing an empty frame into NULL 
fix #1385
decompressing into NULL was an automatic error.
It is now allowed, as long as the content of the frame is empty.

Seems to simplify things for `arrow`.
Maybe some other projects rely on this behavior ?
 2018-10-24 16:34:35 -07:00
Yann Collet
b7421f8e69 fix and refactored libzstd_partial_build.sh 
make the test intention clearer
(easier to maintain)

also resist the existence of zstd_ddict.o within libzstd.a
 2018-10-24 11:32:09 -07:00
Yann Collet
debff3929b fixed warnings in testpools 2018-10-24 10:36:06 -07:00
Yann Collet
cc3612e1c5 added simple guard macros 
in case of accidental multi-includes
 2018-10-23 17:55:23 -07:00
Yann Collet
2f7e1c8737 fixed VS2010 build script 2018-10-23 17:44:43 -07:00