AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
4,677 Commits 1 Branch 0 Tags 27 MiB
6025465e42
Commit Graph

4677 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet
6025465e42 ZSTDMT : minor CCtx memory optimization 
can be useful when a compression job only has small amount of data to compress.
 2018-01-16 15:34:41 -08:00
Yann Collet
2e23333094 ZSTDMT can now work in non-blocking mode with 1 thread 
it still fallbacks to single-thread blocking invocation
when input is small (<1job)
or when invoking ZSTDMT_compress(), which is blocking.

Also : fixed a bug in new block-granular compression routine.
 2018-01-16 15:28:43 -08:00
Yann Collet
8e83c5c910 Merge branch 'dev' into progressiveMT 2018-01-16 12:54:33 -08:00
Yann Collet
f0d703243f
Merge pull request #956 from terrelln/mf-struct 
Split ZSTD_CCtx into smaller sub-structures
 2018-01-16 12:37:44 -08:00
Yann Collet
0bc7a75468
Merge pull request #987 from facebook/checkTag 
Check tag
 2018-01-16 11:52:32 -08:00
Yann Collet
f4e58455f6 ensure MOREFLAGS are not lost in root->tests Makefile invocation 2018-01-16 11:50:16 -08:00
Yann Collet
19cb37f837 travis ci : added gcc-7 test 
also added `-Werror` to sanitizer tests
 2018-01-16 11:40:42 -08:00
Nick Terrell
aae267a2e1 Reorganize block state 2018-01-16 11:17:50 -08:00
Nick Terrell
887cd4e35e Split ZSTD_CCtx into smaller sub-structures 2018-01-16 11:17:50 -08:00
Yann Collet
a187486e3c
Merge pull request #990 from krab/dev-cmake-libdir 
CMake: use GNUInstallDirs for library install dir
 2018-01-15 18:55:21 -08:00
Alexey Ivanov
403e2db139 CMake: use GNUInstallDirs for library install dir 
Libraries now will be installed in the correct directory on x86_64 linux systems,
and can be changed with `-DCMAKE_INSTALL_LIBDIR=<dirname>` option.
 2018-01-15 22:48:46 +03:00
Yann Collet
815eddeda4 added tag-triggered test 
ensure tag and libzstd version are compatible
 2018-01-14 17:06:21 -08:00
Yann Collet
f6e17b8c7a added tests/checkTag 
compared provided tag with current libzstd version
 2018-01-14 17:03:45 -08:00
Yann Collet
4792ac6689 make -C tests legacy : minor flag alteration 
ZSTD_LEGACY_SUPPORT is a macro constant,
so it should be part of CPPFLAGS, instead of CFLAGS
 2018-01-14 14:09:17 -08:00
Yann Collet
99a0a8bbd2 tests/Makefile : fixed target allnothread 
ensures MT is disabled
 2018-01-13 22:00:05 -08:00
Yann Collet
f8a05932b9 added make list capability to tests/Makefile 2018-01-13 21:54:21 -08:00
Yann Collet
9477f6529d
Merge pull request #984 from terrelln/dict-load 
Load more dictionary positions into table if empty
 2018-01-13 13:20:42 -08:00
Yann Collet
58ecf13e02 zstdmt : can compress at block granularity 
offering perspective of more accurate progression report.
 2018-01-13 13:18:57 -08:00
Nick Terrell
9a211d1f05 Load more dictionary positions into table if empty 
If the hash table is empty load positions into the hash table
that we would otherwise skip.

| Level | Data Set     | Improvement |
|-------|--------------|-------------|
| 1     | github       | 0.44%       |
| 1     | hg-changelog | 0.13%       |
| 1     | hg-commands  | 1.28%       |
| 1     | hg-manifest  | 0.70%       |
| 3     | github       | 0.74%       |
| 3     | hg-changelog | 0.87%       |
| 3     | hg-commands  | 1.74%       |
| 3     | hg-manifest  | 0.23%       |
 2018-01-12 16:17:22 -08:00
Yann Collet
863b2f8db4
Merge pull request #983 from terrelln/dict-wlog 
Increase windowLog from CDict based on the srcSize when known
 2018-01-12 07:47:43 -08:00
Yann Collet
9d65a5c316
Merge pull request #970 from facebook/dubtlazy 
Delayed update Binary Tree for Lazy strategy
 2018-01-12 07:28:59 -08:00
Nick Terrell
4b7c4e5f41 Add test for cdict window log adjustment 2018-01-11 16:45:16 -08:00
Nick Terrell
b610b777d3 Increase windowLog from CDict based on the srcSize when known 2018-01-11 16:23:21 -08:00
Yann Collet
cacf47cbee Merge branch 'dev' into dubtlazy 
and fixed conflicts
 2018-01-11 13:25:08 -08:00
Yann Collet
04c00f9388
Merge pull request #982 from facebook/fix304 
Fix for #304 and #977 : error during dictionary creation
 2018-01-11 13:20:59 -08:00
Yann Collet
b9a14900ff changed function name to ZSTD_DUBT_findBestMatch() 2018-01-11 12:38:31 -08:00
Yann Collet
1d623e60a1
Merge pull request #981 from facebook/fix976 
fixed bug #976, reported by @indygreg
 2018-01-11 11:40:07 -08:00
Yann Collet
752bae4a48 added warning message 
when pathological dataset is detected
(note : cover_optimize needs -v to display the warning)
 2018-01-11 11:29:28 -08:00
Yann Collet
e8093dde09 fixed #304 
Pathological samples may result in literal section being incompressible.
This case is now detected,
and literal distribution is replaced by one that can be written into the dictionary.
 2018-01-11 11:16:32 -08:00
Yann Collet
218e9fe0fc added a test case for dictBuilder failure 
cyclic data set makes the entropy stage fails
now, onto a fix for #304 ...
 2018-01-11 09:42:38 -08:00
Yann Collet
55cdbdb465
Merge pull request #980 from conor42/dev 
Fix break condition in decompression noise test
 2018-01-11 08:33:35 -08:00
Yann Collet
2103a62b3d fixed minor warning on prototype definition 2018-01-11 04:49:19 -08:00
Yann Collet
ff795580f2 fixed bug #976, reported by @indygreg 
constants in zstd.h should not depend on MIN() macro which existence is not guaranteed.

Added a test to check the specific constants.
The test is a bit too specific.
But I have found no way to control a more generic "are all macro already defined" condition,
especially as this is a valid construction (the missing macro might be defined later, intentionnally).
 2018-01-10 20:33:45 -08:00
conor42
0e88f6e97b
Fix break condition in decompression noise test 
The bug prevents noise being added
 2018-01-11 11:42:58 +10:00
Yann Collet
06995775b0
Merge pull request #979 from facebook/memdoc 
Update API documentation related to memory management capabilities
 2018-01-10 17:39:37 -08:00
Yann Collet
1edf33764e
Merge pull request #974 from terrelln/dstfile 
[fileio] Improve safety of output file modifications
 2018-01-10 19:02:48 +01:00
Yann Collet
292eeb672f api doc : grouped all ZSTD_create*_advanced() functions together 
in a new "custom memory allocator" paragraph
which is itself part of "memory management" category.

This makes it simpler to see the relation between the type and its usages.
 2018-01-10 09:07:47 -08:00
Yann Collet
3ea156368c API doc : grouped ZSTD_initStatic*() together 
within "memory management" category.
 2018-01-10 08:49:50 -08:00
Yann Collet
b17fb488b0 fixed msan test 
a pointer calculation was wrong in a corner case
 2018-01-06 20:50:36 +01:00
Yann Collet
76321d6bbc
Merge pull request #965 from facebook/tests_obj 
%.o objects files in /tests
 2018-01-06 13:15:53 +01:00
Yann Collet
658d6b8588 Merge branch 'dev' into dubtlazy 2018-01-06 12:40:58 +01:00
Yann Collet
a927fae2a1 fixed ZSTD_reduceIndex() 
following suggestions from @terrelln.
Also added some comments to present logic behind ZSTD_preserveUnsortedMark().
 2018-01-06 12:31:26 +01:00
Yann Collet
3d578eef0c minor tests/Makefile change, as suggested by @terrelln 
https://github.com/facebook/zstd/pull/965#discussion_r160008006
 2018-01-06 07:13:09 +01:00
Yann Collet
752880ffed
Merge pull request #963 from facebook/benchfix 
fix: bench can accept hlog custom parameter
 2018-01-06 06:57:02 +01:00
Nick Terrell
ed9611dc62 [fileio] Don't call FIO_remove() on stdout or /dev/null 2018-01-05 11:50:24 -08:00
Nick Terrell
282ad05e0a [fileio] Use FIO_remove() everywhere for safety 2018-01-05 11:44:45 -08:00
Nick Terrell
fd63140e1c [util] Refuse to set file stat on non-regular file 2018-01-05 11:44:25 -08:00
Yann Collet
dce386f658
Merge pull request #972 from pixelb/bz1530049 
zstd: fix crash when not overwriting existing files
 2018-01-04 12:14:50 +01:00
Yann Collet
f51e861665
Merge pull request #973 from terrelln/test-case 
Add test case for PR #972
 2018-01-04 12:14:36 +01:00
Nick Terrell
8adebbd0f8 Add test case for PR #972 2018-01-03 15:52:18 -08:00