AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
3,927 Commits 1 Branch 0 Tags 27 MiB
5adceeed01
Commit Graph

1677 Commits

Author SHA1 Message Date
Yann Collet
dab5ea93f2 Merge pull request #515 from iburinoc/emptydict 
Don't create dict in streaming apis if dictSize == 0
 2017-01-19 09:02:42 -08:00
Yann Collet
6073b3e6b8 ZSTDMT_endStream : nullify input buffer after flush 
There will be no more input after ZSTDMT_endStream invocation :
only flush/end is allowed (to fully collect compressed result).
 2017-01-18 15:32:38 -08:00
Yann Collet
3a01c46b26 ZSTDMT_initCStream() supports restart from invalid state 
ZSTDMT_initCStream() will correcly scrub for resources
when it detects that previous compression was not properly finished.
 2017-01-18 15:18:17 -08:00
Yann Collet
4885f591b3 trap compression errors, collect back resources from workers 2017-01-18 14:11:37 -08:00
Sean Purcell
0b5370ae38 Prefix notes with /**< 2017-01-18 13:45:02 -08:00
Yann Collet
563ef8acf4 CCtxPool starts empty, as suggested by @terrelln 
Also : make zstdmt now a target from root
 2017-01-18 12:12:10 -08:00
Yann Collet
a6db7a7b9b fixed cmaketest 
(buffer_t){NULL,0} is not considered a constant.
{NULL,0} is.
 2017-01-18 11:57:34 -08:00
Yann Collet
0d6b8f65a9 ZSTDMT_free() scrubs potentially unfinished jobs to release their resources 
In some complex scenarios (free() without finishing compression),
it is possible that some resources are still into jobs
and not collected back into pools.
In which case, previous version of free() would miss them.
This would be equivalent to a leak.

New version ensures that it even foes after such resource.
It requires job consumers to properly mark resources as released,
by replacing entries by NULL after releasing back to the pool.

Obviously, it's not recommended to free() zstdmt context mid-term,
still that's now a supported scenario.

The same methodology is also used to ensure proper resource collection
after an error is detected.

Still to do :
- detect compression errors (not just allocation ones)
- properly manage resource when init() is called without finishing previous compression.
 2017-01-17 17:46:33 -08:00
Yann Collet
d0a1d45582 ZSTDMT_{flush,end}Stream() now block on next job completion when nothing to flush 
The main issue was to avoid a caller to continually loop on {flush,end}Stream()
when there was nothing ready to be flushed but still some compression work ongoing in a worker thread.
The continuous loop would have resulted in wasted energy.
The new version makes call to {flush,end}Stream blocking when there is nothing ready to be flushed.
Of course, if all worker threads have exhausted job, it will return zero (all flush completed).

Note : There are still some remaining issues to report error codes
and properly collect back resources into pools when an error is triggered.
 2017-01-17 16:15:18 -08:00
Yann Collet
a73c412932 completed ZSTDMT streaming compression 
Provides the baseline compression API :
size_t ZSTDMT_initCStream(ZSTDMT_CCtx* zcs, int compressionLevel);
size_t ZSTDMT_compressStream(ZSTDMT_CCtx* zcs, ZSTD_outBuffer* output, ZSTD_inBuffer* input);
size_t ZSTDMT_flushStream(ZSTDMT_CCtx* zcs, ZSTD_outBuffer* output);
size_t ZSTDMT_endStream(ZSTDMT_CCtx* zcs, ZSTD_outBuffer* output);

Not tested yet
 2017-01-17 15:31:16 -08:00
Sean Purcell
57d423c5df Don't create dict in streaming apis if dictSize == 0 2017-01-17 14:31:35 -08:00
Przemyslaw Skibinski
8a0bc30a2d Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2017-01-17 13:02:29 +01:00
Przemyslaw Skibinski
d72f4b6b7a added "Makefile is validated" 2017-01-17 12:40:06 +01:00
Gregory Szorc
7d6f478d15 Set dictionary ID in ZSTD_initCStream_usingCDict() 
When porting python-zstandard to use ZSTD_initCStream_usingCDict()
so compression dictionaries could be reused, an automated test
failed due to compressed content changing.

I tracked this down to ZSTD_initCStream_usingCDict() not
setting the dictID field of the ZSTD_CCtx attached to the
ZSTD_CStream instance.

I'm not 100% convinced this is the correct or full solution,
as I'm still seeing one automated test failing with this change.
 2017-01-14 17:44:54 -08:00
Yann Collet
5b726dbe4d fix gcc-arm warning "suggest braces around empty body" 2017-01-12 17:46:46 +01:00
Yann Collet
ad9f6bd123 zstdmt : fix : resources properly collected even when early fail 
In previous version, main function would return early when detecting a job error.
Late threads resources were therefore not collected back into pools.
New version just register the error, but continue the collecting process.
All buffers and context should be released back to pool before leaving main function.
 2017-01-12 03:06:35 +01:00
Sean Purcell
834ab50fa3 Fixed decompress_usingDict not propagating corrupted dictionary error 2017-01-11 17:31:34 -08:00
Yann Collet
b05c4828ea zstdmt : correctly check for cctx and buffer allocation 
Result from getBuffer and getCCtx could be NULL when allocation fails.
Now correctly checks : job creation stop and last job reports an allocation error.
releaseBuffer and releaseCCtx are now also compatible with NULL input.

Identified a new potential issue :
when early job fails, later jobs are not collected for resource retrieval.
 2017-01-12 02:01:28 +01:00
Yann Collet
107bcbbbc2 zstdmt : changed internal naming from frame to chunk 
Since the result of mt compression is a single frame,
changed naming, which implied the concatenation of multiple frames.

minor : ensures that content size is written in header
 2017-01-12 01:25:46 +01:00
Yann Collet
5eb749e734 ZSTDMT_compress() creates a single frame 
The new strategy involves cutting frame at block level.
The result is a single frame, preserving ZSTD_getDecompressedSize()

As a consequence, bench can now make a full round-trip,
since the result is compatible with ZSTD_decompress().

This strategy will not make it possible to decode the frame with multiple threads
since the exact cut between independent blocks is not known.
MT decoding needs further discussions.
 2017-01-11 18:21:25 +01:00
Yann Collet
04cbc36499 minor refactor (release CCtx 1st) and comment clarification 2017-01-11 16:08:08 +01:00
Yann Collet
085179bb78 fixed ZSTDMT_createCCtx() : checked inner objects are properly created 2017-01-11 15:58:05 +01:00
Yann Collet
8ce1cc2bec improved ZSTD_createCCtxPool() cancellation 
use ZSTD_freeCCtxPool() to release the partially created pool.
avoids to duplicate logic.

Also : identified a new difficult corner case :
when freeing the Pool, all CCtx should be previously released back to the pool.
Otherwise, it means some CCtx are still in use.
There is currently no clear policy on what to do in such a case.
Note : it's supposed to never happen.
Since pool creation/usage is static, it has no external user,
which limits risks.
 2017-01-11 15:44:26 +01:00
Yann Collet
47557ba2b2 fixed ZSTDMT_createCCtxPool() when inner CCtx creation fails 2017-01-11 15:35:56 +01:00
Nick Terrell
8d984699db Document memory requirements for COVER algorithm 2017-01-09 18:20:10 -08:00
Nick Terrell
555e281637 Handle large input size in 32-bit mode correctly 2017-01-09 18:20:06 -08:00
Nick Terrell
3a1fefcf00 Simplify COVER parameters 2017-01-02 17:51:38 -08:00
Nick Terrell
96b39f65fa Add COVER dictionary builder 2017-01-02 13:22:51 -08:00
Yann Collet
6334b04d61 compile object files, for faster recompilation 2017-01-02 03:22:18 +01:00
Yann Collet
f1cb55192c fixed linux warnings 2017-01-02 01:11:55 +01:00
Yann Collet
0ec6a95ba1 minor fixes 2017-01-02 00:49:42 +01:00
Yann Collet
2ec635a162 use pthread_cond to send signals between threads 2017-01-01 17:31:33 +01:00
Nick Terrell
bb13387d7d Fix pool for threading.h 2016-12-31 19:10:47 -05:00
Nick Terrell
4204e03e77 Add threading.h condition variables 2016-12-31 19:10:29 -05:00
Yann Collet
3b9d434356 extended ZSTDMT code support for non-MT systems and WIN32 (preliminary) 2016-12-31 16:32:19 +01:00
Yann Collet
c8efc1c874 simplified Buffer Pool 2016-12-31 14:45:33 +01:00
Yann Collet
3b29dbd9e8 new zstdmt version using generic treadpool 2016-12-31 06:04:25 +01:00
Yann Collet
c6a6417458 bench correctly measures time for multi-threaded compression (posix only) 2016-12-31 03:31:26 +01:00
Yann Collet
f765a375a5 Merge pull request #504 from terrelln/thread-pool 
[zstdmt] Add thread pool
 2016-12-30 15:31:49 +01:00
Nick Terrell
e777a5be6b Add a thread pool for ZSTDMT and COVER 2016-12-29 23:39:44 -08:00
Yann Collet
e70912c72b Changed : input divided into roughly equal parts. 
Debug : can measure time waiting for mutexes to unlock.
 2016-12-29 01:24:01 +01:00
Yann Collet
6c0ed9483a compression threads use ZSTD_compressCCtx() 2016-12-28 17:08:28 +01:00
Yann Collet
8d7432914f Merge pull request #503 from inikep/dev11 
Dev11
 2016-12-28 16:50:39 +01:00
Yann Collet
ce9e1452fd protect buffer pool with a mutex 2016-12-28 15:31:19 +01:00
Przemyslaw Skibinski
75f3a3a335 changed default PREFIX and MANDIR 2016-12-28 12:32:41 +01:00
Yann Collet
3d93f2fce7 first zstdmt sketch 2016-12-27 07:19:36 +01:00
Yann Collet
39c105c605 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-12-23 22:25:31 +01:00
Yann Collet
aca113f4f5 fixed ZSTD_sizeof_?Dict() 2016-12-23 22:25:03 +01:00
Yann Collet
c07d2e3a31 Merge pull request #499 from inikep/dev11 
improved *BSD and Solaris compatibility
 2016-12-23 21:32:03 +01:00
Przemyslaw Skibinski
63b0014b96 BSD: improved "make install" 2016-12-23 10:05:49 +01:00
Nick Terrell
1b5d4a7d53 ZDICT_finalizeDictionary() flipped comparison 2016-12-22 18:14:57 -08:00
Nick Terrell
bcbe77e994 ZDICT_finalizeDictionary() flipped comparison 
`ZDICT_finalizeDictionary()` had a flipped comparison.
I also allowed `dictBufferCapacity == dictContentSize`.
It might be the case that the user wants to fill the dictionary
completely up, and then let zstd take exactly the space it needs
for the entropy tables.
 2016-12-22 18:01:14 -08:00
Nick Terrell
78a0072d5a Fix failing test due to deprecation warning 2016-12-22 17:36:16 -08:00
Yann Collet
d76d1a9ef0 added ZDICT_finalizeDictionary() 2016-12-22 20:18:43 +01:00
Przemyslaw Skibinski
b999170311 Solaris: working "make -C lib install" 2016-12-22 20:14:37 +01:00
Yann Collet
ba75e9d8c3 fix : zlib wrapper compile in gnu90 mode 2016-12-21 19:57:18 +01:00
Yann Collet
0819abe3c1 added ZSTD_createDDict_byReference() body 2016-12-21 19:25:15 +01:00
Yann Collet
4e5eea61a8 added ZSTD_createDDict_byReference() 2016-12-21 16:44:35 +01:00
Yann Collet
8333106b8a Merge branch 'dev' of github.com:facebook/zstd into dev 2016-12-21 16:44:24 +01:00
Yann Collet
0d7e84899f Merge pull request #489 from inikep/v112 
improved detection of POSIX
 2016-12-21 16:42:46 +01:00
Yann Collet
1f57c2ed32 added : ZSTD_createCDict_byReference() 2016-12-21 16:20:11 +01:00
Przemyslaw Skibinski
2f6ccee6af platform.h: removed Compiler Options 2016-12-21 13:23:34 +01:00
Przemyslaw Skibinski
5736db219e fix basic types redefinition 2016-12-21 09:26:00 +01:00
Nick Terrell
8157a4c3cc Fix dictionary loading bug causing an MSAN failure 
Offset rep codes must be in the range `[1, dictSize)`.
Fix dictionary loading to reject `0` as a offset rep code.
 2016-12-20 10:47:52 -08:00
Przemyslaw Skibinski
f8046b8e72 Merge remote-tracking branch 'refs/remotes/facebook/dev' into v112 
# Conflicts:
#	appveyor.yml
 2016-12-19 08:20:26 +01:00
Yann Collet
d564faa3c6 fix : ZSTD_initCStream_srcSize() correctly set srcSize in frame header 2016-12-18 21:39:15 +01:00
Yann Collet
1496c3dc47 Fix : size estimation when some samples are very large 2016-12-18 11:58:23 +01:00
Yann Collet
d46ecb58a5 added dll compilation tests 2016-12-17 16:28:12 +01:00
Nick Terrell
8de46ab51a Export all API functions 2016-12-16 13:27:30 -08:00
Przemyslaw Skibinski
0a1caeef6d VS: fixed 32-bit DLL compilation 2016-12-15 12:12:46 +01:00
Przemyslaw Skibinski
4e10bd339d appveyor.yml: added tests of fullbench-dll fullbench-lib 2016-12-15 12:09:23 +01:00
Przemyslaw Skibinski
60f10aab6c introduced ZSTDLIB_VISIBILITY 2016-12-15 11:32:31 +01:00
Yann Collet
2b36b238d3 changed variable name to estimatedSrcSize, to emphasize it does not need to be exact 2016-12-13 17:59:55 +01:00
Yann Collet
e795c8a5f6 Added ZSTD_initCStream_srcSize(). 
Added relevant test cases in zstreamtest
 2016-12-13 17:00:14 +01:00
Yann Collet
5397a66b19 minor BMI version check 2016-12-13 15:21:06 +01:00
Yann Collet
35168679bd Merge pull request #478 from terrelln/wildcopy-ub 
Fix execSequence wildcopy undefined behavior
 2016-12-13 11:33:00 +01:00
Nick Terrell
064a143520 Fix execSequence wildcopy undefined behavior 
execSequence relied on pointer overflow to handle cases where
`sequence.matchLength < 8`.  Instead of passing an `size_t` to
wildcopy, pass a `ptrdiff_t`.
 2016-12-12 19:01:23 -08:00
Nick Terrell
e474aa55b4 Fix decompression buffer overrun 
Allows an adversary to write up to 3 bytes beyond the end of the buffer.
Occurs if the match overlaps the `extDict` and `currentPrefix`, and the
match length in the `currentPrefix` is less than `MINMATCH`, and
`op-(16-MINMATCH) >= oMatchEnd > op-16`.
 2016-12-12 18:05:30 -08:00
Yann Collet
c3a5c4bef8 introduced cycleLog 2016-12-12 00:47:30 +01:00
Yann Collet
c261f71f6a minor variation of rescale fix 2016-12-12 00:25:07 +01:00
Nick Terrell
3826207a70 Simplify segfault fix 
Take advantage of the fact that `chainLog <= windowLog`.
 2016-12-10 18:46:55 -08:00
Nick Terrell
0012332ce0 Fix compression segfault 
When the overflow protection kicks in, it makes sure that ip - ctx->base
isn't too large.  However, it didn't ensure that saved offsets are
still valid.  This change ensures that any valid offsets (<= windowLog)
are still representable after the update.

The bug would shop up on line 1056, when `offset_1 > current + 1`, which
causes an underflow.  This in turn, would cause a segfault on line 1063.

The input must necessarily be longer than 1 GB for this issue to occur.
Even then, it only occurs if one of the last 3 matches is larger than
the chain size and block size.
 2016-12-09 17:15:33 -08:00
Yann Collet
383b8088a3 minor lib build refactoring 2016-12-08 18:42:27 -08:00
Yann Collet
6e754fe76a fixed lib soname. 
example : simple_compression : size overflow check
 2016-12-08 18:26:56 -08:00
Przemyslaw Skibinski
7687913178 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-12-08 10:42:42 +01:00
Yann Collet
426a9d4b71 changed : dll : only approved ZSTD symbols are now exposed. All other symbols remain internal. 2016-12-07 16:39:34 -08:00
Przemyslaw Skibinski
4da53219a0 zstd Manual updated to 1.1.2 2016-12-07 11:18:40 +01:00
Yann Collet
379908be3d fixed zstd.h for manual 2016-12-06 10:36:15 -08:00
Yann Collet
9dfd0af164 ZBUFF_ as a wrapper to ZSTD streaming API. 2016-12-06 17:16:41 +01:00
Yann Collet
825dffbc43 moved zbuff source files into lib/deprecated 2016-12-05 19:28:19 -08:00
Yann Collet
8f8e2b0b4a fixed initialization warning 2016-12-05 18:00:50 -08:00
Yann Collet
e7a41a5955 added : dictID retrieval functions. 
added : unit tests for dictID retrieval functions
 2016-12-05 16:21:06 -08:00
Yann Collet
9ffbeea875 API : changed : streaming decompression : implicit reset on starting new frames 2016-12-02 18:37:38 -08:00
Yann Collet
2238312c2f fix dict loading 2016-12-02 11:36:11 -08:00
Przemyslaw Skibinski
821bf1febc fixed Doxygen trailing comment 2016-12-02 16:13:41 +01:00
Yann Collet
b89af20353 reduced table sizes for HUF_readDTableX4 2016-12-01 18:24:59 -08:00
Yann Collet
a0d742b1e4 introduced HUF_buildCTable_wksp(), to reduce stack memory usage 2016-12-01 17:47:30 -08:00
Yann Collet
643d9a234b replaced usage of FSE_buildCTable by FSE_buildCTable_wksp, using less stack space in the process 2016-12-01 16:24:04 -08:00
Yann Collet
e928f7e16d introduced ext_wksp variants of count to reduce stack memory usage 2016-12-01 16:13:35 -08:00
Yann Collet
979cab412b fixed some minor visual silent cast warnings. 
introduced FSE_count_parallel_wksp().
 2016-11-30 18:10:38 -08:00
Yann Collet
5e00b848a8 FSE_compress_wksp() uses less stack space 2016-11-30 16:46:13 -08:00
Yann Collet
d79a9a00d9 Introduced FSE_compress_wksp() and FSE_buildCTable_wksp() to reduce stack memory usage 2016-11-30 15:52:20 -08:00
Yann Collet
766431909f introduced FSE_decompress_wksp(), to use less stack space 2016-11-30 12:36:45 -08:00
Yann Collet
95eb43be09 updated pkg config file 2016-11-30 11:06:58 -08:00
Yann Collet
42247705a3 Merge pull request #461 from obache/neatsrc/pkgconfig 
libzstd.pc.in: Change to use variables for libdir and includedir
 2016-11-30 20:03:42 +01:00
Yann Collet
ff504de391 minor decompression speed improvement 2016-11-29 17:42:46 -08:00
Yann Collet
25f46dcc0f minor const 2016-11-29 16:59:27 -08:00
Yann Collet
a56ac2815c restored normal decoder speed 2016-11-29 15:30:23 -08:00
Yann Collet
37870d7a66 fixed minor visual warning 2016-11-29 14:31:57 -08:00
Yann Collet
013a2b58ae Merge pull request #464 from terrelln/guards 
Fix ZSTD_STATIC_LINKING_ONLY with double include
 2016-11-29 23:06:10 +01:00
Yann Collet
167c494748 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-11-29 14:05:15 -08:00
Yann Collet
4f5350f610 long matches support overflow 2016-11-29 13:12:24 -08:00
Nick Terrell
05c00f2ff7 Fix ZSTD_STATIC_LINKING_ONLY with double include 2016-11-29 11:54:39 -08:00
OBATA Akio
c53eea7c1a libzstd.pc.in: Change to use variables for libdir and includedir 2016-11-29 16:47:53 +09:00
Yann Collet
52e136ed3d long decoder compatible with round and separate buffers 2016-11-28 19:59:11 -08:00
Yann Collet
ce3527ca0c combined normal and long decoder 2016-11-28 18:38:52 -08:00
Yann Collet
8993bee997 restored normal mode 2016-11-28 16:11:30 -08:00
Yann Collet
764e70a4f3 added decodeSequencesLong 2016-11-28 15:50:16 -08:00
Yann Collet
73f88a66f1 added prefetch 2016-11-23 15:43:30 -08:00
Yann Collet
50524bf0da delayed decompression 2016-11-23 15:11:07 -08:00
Przemyslaw Skibinski
fc4193bda5 fixed g++ warnings 2016-11-23 18:17:18 +01:00
Przemyslaw Skibinski
9ca65af810 zstd_opt.h: improved price function 2016-11-23 17:22:54 +01:00
Przemyslaw Skibinski
ad3e94512c fixed warnings from static analyzer in zstd_opt.h 2016-11-21 20:22:12 +01:00
Przemyslaw Skibinski
eec700a3b7 exclude zbuff_compress.c and zbuff_decompress.c 2016-11-21 14:34:03 +01:00
Przemyslaw Skibinski
62d19a6f39 lib\README.md: added Using MinGW+MSYS to create DLL 2016-11-21 14:22:08 +01:00
Przemyslaw Skibinski
b85f767743 files to generate ZSTD Windows binary package 2016-11-21 14:10:55 +01:00
Przemyslaw Skibinski
8bb86e330b create DLL with Windows 2016-11-21 12:51:01 +01:00
Przemyslaw Skibinski
93a09eedf1 added libzstd.def 2016-11-21 12:33:27 +01:00
Przemyslaw Skibinski
5a17223691 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-11-18 11:47:01 +01:00
Przemyslaw Skibinski
3d18088b38 updated windres 2016-11-17 18:04:41 +01:00
Yann Collet
0d761dbe95 Merge pull request #453 from inikep/dev11 
fullbench-dll
 2016-11-16 15:45:30 -08:00
Yann Collet
52afb3993e zbuff API now generates deprecation warnings 2016-11-16 08:50:54 -08:00
Przemyslaw Skibinski
179555c1d1 working fullbench-dll 2016-11-15 18:05:46 +01:00
Nick Terrell
4359d21ad7 Merge two memset() calls into one 2016-11-14 17:52:51 -08:00
Nick Terrell
24701de877 Fix uninitialized memory read 2016-11-14 13:57:05 -08:00
Yann Collet
8e4901eccd removed zbuff.h from include installation 2016-11-08 15:45:39 -08:00
Yann Collet
fd3be6bc97 bump version number to 1.1.2 2016-11-07 14:35:41 -08:00
Nick Terrell
dc904ad17b Fix bug in zstd v0.{5, 6} dictionary decompression 
Introduced by bb68062c59.
 2016-11-04 16:18:59 -07:00
Przemyslaw Skibinski
38b590ad69 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 
# Conflicts:
#	lib/Makefile
 2016-11-04 10:10:54 +01:00
Yann Collet
407a11f63e fixed Visual compatibility 2016-11-03 15:52:01 -07:00
Yann Collet
11812260d1 Merge pull request #439 from terrelln/dev 
ZSTD_compress_usingDict() specify dict == NULL
 2016-11-03 14:15:36 -07:00
Nick Terrell
c8a9ac312b Fix dynamic libzstd symlinks 2016-11-03 12:32:48 -07:00
Przemyslaw Skibinski
3a415594b1 fixed MinGW compilation 2016-11-03 12:59:20 +01:00
Yann Collet
7347869fb6 fixed make install 2016-11-02 22:28:37 -07:00
Nick Terrell
d82efd8a70 ZSTD_compress_usingDict() when dict gets loaded 
Specify that when `dict == NULL || dictSize < 8` no dictionary
gets loaded.
Also add some periods.
 2016-11-02 18:07:16 -07:00
Yann Collet
179b19776f fileio.c does no longer need ZSTD_LEGACY_SUPPORT, and does no longer depend on zstd_legacy.h 
Added : ZSTD_isFrame() in experimental section
 2016-11-02 17:30:49 -07:00
Yann Collet
f3f13211ae Fix #419 : no warning when setting custom LDFLAGS 2016-11-02 17:02:45 -07:00
Yann Collet
0a5a5fb7fd Fix #418 : printing selected segments in zdict debug mode can segfault with certain pathological patterns 2016-11-02 13:57:55 -07:00
Yann Collet
31e660e7aa more accurate default maximum window size 2016-10-29 03:56:45 -07:00
Yann Collet
2115724c22 Merge pull request #430 from terrelln/exec-sequences 
ZSTD_execSequence() accepts match in last 7 bytes
 2016-10-28 10:45:05 -07:00
Nick Terrell
10bfd0c0d5 Fix ZSTD_execSequence() performance regression 
Commit ae1cb3b3d0 caused the regression.
It is an instruction alignment issue, because if it is `U64 i` instead
of `U32 i`, the regression returns.  This patch fixes the regression
in gcc, but only gets some of the clang performance back.

Benchmarks:
Run on `silesia.tar`.  I only show levels 1-5 because the performance
regression was uniform across all levels.  I did one run on levels
1-19 and it looked good.

| Build | Level | Before | While | After |
|-------|-------|-------:|------:|------:|
| gcc   |     1 |  931.4 | 904.4 | 932.8 |
| gcc   |     2 |  849.1 | 822.6 | 851.2 |
| gcc   |     3 |  815.6 | 790.6 | 818.9 |
| gcc   |     4 |  794.1 | 770.7 | 798.0 |
| gcc   |     5 |  785.7 | 760.7 | 788.8 |
| clang |     1 |  705.5 | 683.2 | 693.8 |
| clang |     2 |  670.0 | 649.2 | 660.7 |
| clang |     3 |  659.6 | 639.8 | 651.4 |
| clang |     4 |  652.5 | 634.7 | 645.9 |
| clang |     5 |  646.9 | 625.5 | 637.7 |
 2016-10-27 16:19:57 -07:00
Yann Collet
ee5b725823 ZSTD_initCStream() optimization : do not allocate a CDict when no dictionary used 2016-10-27 14:20:55 -07:00
Nick Terrell
eb7873a048 ZSTD_execSequence() accepts match in last 7 bytes 
The zstd reference compressor will not emit a match in the last 7
bytes of a block.  The decompressor will also not accept a match
in the last 7 bytes.  This patch makes the decompressor accept a
match in the last 7 bytes.
 2016-10-25 21:24:15 -07:00
Yann Collet
335ad5d4d4 added ZSTD_initDStream_usingDDict() . 
slightly optimized ZSTD_initDStream() when no dictionary .
fixed ZSTD_sizeof_CStream() .
 2016-10-25 17:47:02 -07:00
Yann Collet
9516234e67 first sketch for ZSTD_initCStream_usingCDict() 2016-10-25 16:19:52 -07:00
Yann Collet
62d9a7ddfd Merge pull request #429 from inikep/btopt2 
Btopt2
 2016-10-25 14:48:43 -07:00
Przemyslaw Skibinski
5c5f01f3da added ZSTD_btopt2 strategy 2016-10-25 12:25:07 +02:00
Yann Collet
7b5948cca7 Merge pull request #426 from terrelln/fixes 
Fix various {A, M}SAN bugs
 2016-10-24 23:42:26 -07:00
Yann Collet
37d130031d updated comments on context re-use 2016-10-24 17:22:12 -07:00
Nick Terrell
b2c39a22b0 Fix compiler narrowing warning 2016-10-24 14:50:13 -07:00
Nick Terrell
f698ad6deb Merge remote-tracking branch 'upstream/dev' into fixes 
* upstream/dev:
  added doc\zstd_manual.html
  added contrib\gen_html
  zstd_compression_format.md moved to doc/
  Fix small bug in ZSTD_execSequence()
  improved ZSTD_compressBlock_opt_extDict_generic
  protect ZSTD_decodeFrameHeader() from invalid usage, as suggested by @spaskob
  zstd_opt.h: small improvement in compression ratio
  improved dicitonary segment merge
  use implicit rules to compile zstd_decompress.c
  detect early impossible decompression scenario in legacy decoder v0.5
  no repeat mode in legacy v0.5
  fixed invalid invocation of dictionary in legacy decoder v0.5
  fix edge case
  fix command line interpretation
  fixed minor corner case
  zstd.h: added the Introduction section
  fixed clang 3.5 warnings
  zstd.h: updated comments
 2016-10-24 13:10:13 -07:00
Yann Collet
4239a207dd Merge pull request #425 from inikep/dev11 
Doc
 2016-10-24 11:11:40 -07:00
Nick Terrell
f9c9af3c2e Reject dictionaries with incomplete entropy tables 
If a dictionary specifies that a symbol has probability zero in its
`matchLength`, `literalLength`, or `offset` FSE table, but the symbol
appears when compressing input, the compressor fails.

Ensure that dictionaries support all `matchLength`, and `literalLength`
codes.  They must also support all of the `offset` codes required to
represent every possible offset that can appear in the first block.
 2016-10-24 10:42:44 -07:00
Przemyslaw Skibinski
984b66cd72 added contrib\gen_html 2016-10-24 15:59:51 +02:00
Przemyslaw Skibinski
3ee94a7600 zstd_compression_format.md moved to doc/ 2016-10-24 15:58:07 +02:00
Yann Collet
97611611a3 Merge pull request #423 from terrelln/exec-seq-patch 
Fix small bug in ZSTD_execSequence()
 2016-10-21 17:02:06 -07:00
Nick Terrell
ae1cb3b3d0 Fix small bug in ZSTD_execSequence() 
`memmove(op, match, sequence.matchLength)` is not the desired behavior.
Overlap is allowed, and handled as if we did `*op++ = *match++`, which
is not how `memmove()` handles overlap.

Only triggered if both of the following conditions are met:
* The match spans extDict & currentPrefixSegment
* `oLitEnd <= oend_w < oLitEnd + length1 < oMatchEnd <= oend`.

These two conditions imply that the block is less than 15 bytes long.
This bug isn't triggered by the streaming API, because it allocates
enough space for the window size + the block size, so there cannot be
a match that is within 8 bytes of the end and overlaps with itself.
It cannot be triggered by the block decompression API because all of
the decompressed data is in the currentPrefixSegment.

Introduced by commit 7158584399
 2016-10-21 12:13:44 -07:00
Przemyslaw Skibinski
4732074a71 improved ZSTD_compressBlock_opt_extDict_generic 2016-10-21 11:19:00 +02:00
Yann Collet
da3bd8b6de protect ZSTD_decodeFrameHeader() from invalid usage, as suggested by @spaskob 2016-10-20 20:11:00 -07:00
Przemyslaw Skibinski
d365ae3497 zstd_opt.h: small improvement in compression ratio 2016-10-20 11:49:02 +02:00
Przemyslaw Skibinski
575ab00db7 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-10-20 11:01:52 +02:00
Nick Terrell
d760529a05 Fix stack buffer overrun when weightTotal == 0 
If `weightTotal == 0`, then `BIT_highbit32(weightTotal)` is
undefined behavior in the case that it calls `__builtin_clz()`.
If `tableLog == HUF_TABLELOG_ABSOLUTEMAX` then we will access one
byte beyond the end of the buffer.
 2016-10-19 11:39:11 -07:00
Nick Terrell
bb68062c59 Unitialized memory read in ZSTD_decodeSeqHeaders() 
Caused by two things:
1. Not checking that `ip` is in range except for the first byte.
2. `ZSTDv0{5,6}_decodeLiteralsBlock()` could return a value larger than `srcSize`.
 2016-10-18 16:41:33 -07:00
Yann Collet
52c1bf93fe improved dicitonary segment merge 2016-10-18 16:34:58 -07:00
Nick Terrell
7b06ad7a05 Backport fix from commit 125d817 
This fixes a read of unitialized memory.
Full commit hash: 125d81774f.
 2016-10-18 14:52:34 -07:00
Nick Terrell
f45b157d95 Backport fix from commit 9e8b09a 
Fixes uninitialized memory reads.
Full commit hash: 9e8b09a7bd
 2016-10-18 14:22:49 -07:00
Yann Collet
f7906d5955 detect early impossible decompression scenario in legacy decoder v0.5 2016-10-18 13:48:32 -07:00
Yann Collet
9313c8d953 no repeat mode in legacy v0.5 2016-10-18 13:36:15 -07:00
Yann Collet
83d7bdee4b fixed invalid invocation of dictionary in legacy decoder v0.5 2016-10-18 12:25:43 -07:00
Yann Collet
197a55ee7b fix edge case 2016-10-18 11:27:52 -07:00
Nick Terrell
fd98087047 Fix stack buffer overflow in HUF_readCTable() 
If `w ==0` on line 153, then `CTable[n].nbBits == tableLog + 1`.
Then `nbPerRank[CTable[n].nbBits]` and `valPerRank[CTable[n].nbBits]`
are stack buffer overflows.
 2016-10-17 18:16:59 -07:00
Yann Collet
06573e17be fixed minor corner case 2016-10-17 17:28:28 -07:00
Nick Terrell
bfd943ace5 Fix buffer overrun in ZSTD_loadDictEntropyStats() 
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadDictEntropyStats()`.  This caused `FSE_buildCTable()`
to stack/heap overflow in a few places.

The benchmarks look good, there is no obvious compression performance regression:

  > ./zstds/zstd.opt.0 -i10 -b1 -e10 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 271.6 MB/s , 716.8 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 204.8 MB/s , 671.1 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 156.8 MB/s , 658.6 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 136.4 MB/s , 665.3 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  98.9 MB/s , 647.0 MB/s
   6#silesia.tar       : 211988480 ->  62979643 (3.366),  65.2 MB/s , 670.4 MB/s
   7#silesia.tar       : 211988480 ->  61974560 (3.421),  44.9 MB/s , 688.2 MB/s
   8#silesia.tar       : 211988480 ->  61028308 (3.474),  32.4 MB/s , 711.9 MB/s
   9#silesia.tar       : 211988480 ->  60416751 (3.509),  21.1 MB/s , 718.1 MB/s
  10#silesia.tar       : 211988480 ->  60174239 (3.523),  22.2 MB/s , 721.8 MB/s

  > ./compress_zstds/zstd.opt.1 -i10 -b1 -e10 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 273.8 MB/s , 722.0 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 203.2 MB/s , 666.6 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 157.4 MB/s , 666.5 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 132.1 MB/s , 661.9 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  96.8 MB/s , 641.6 MB/s
   6#silesia.tar       : 211988480 ->  62979643 (3.366),  63.1 MB/s , 677.0 MB/s
   7#silesia.tar       : 211988480 ->  61974560 (3.421),  44.3 MB/s , 678.2 MB/s
   8#silesia.tar       : 211988480 ->  61028308 (3.474),  33.1 MB/s , 708.9 MB/s
   9#silesia.tar       : 211988480 ->  60416751 (3.509),  21.5 MB/s , 710.1 MB/s
  10#silesia.tar       : 211988480 ->  60174239 (3.523),  21.9 MB/s , 723.9 MB/s
 2016-10-17 16:55:52 -07:00
Nick Terrell
4db751668f Fix buffer overrun in ZSTD_loadEntropy() 
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadEntropy()`.  This caused `FSE_buildDTable(dctx->MLTable, ...)`
to overwrite the beginning of `dctx->hufTable`.

The benchmarks look good, there is no obvious performance regression:

  > ./zstds/zstd.opt.0 -i10 -b1 -e5 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 268.2 MB/s , 701.0 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 199.5 MB/s , 666.9 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 154.9 MB/s , 655.6 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 128.9 MB/s , 648.4 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  98.4 MB/s , 633.4 MB/s

  > ./zstds/zstd.opt.2 -i10 -b1 -e5 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 266.1 MB/s , 703.7 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 199.0 MB/s , 666.6 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 156.2 MB/s , 656.2 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 133.2 MB/s , 647.4 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  96.3 MB/s , 633.3 MB/s
 2016-10-17 15:51:15 -07:00
Nick Terrell
ccfcc643da Check if dict is empty before reading first byte 2016-10-17 11:46:03 -07:00
Yann Collet
2b361cf2f1 minor opt 2016-10-14 16:09:07 -07:00
Yann Collet
7933434fdf Merge branch 'dev' of github.com:facebook/zstd into dev 2016-10-14 13:32:35 -07:00
Yann Collet
d4cda27b63 new command -M#, to limit memory usage during decompression (#403) 2016-10-14 13:32:20 -07:00
Nick Terrell
3b9cdf9220 Fix ubsan failures (pass NULL to memcpy) 2016-10-12 20:54:42 -07:00
Yann Collet
5d919e7ac3 added ZSTD_error_frameParameter_windowTooLarge (#403) 2016-10-12 17:29:24 -07:00
Yann Collet
e19111c42f make creates libzstd binaries (#415) 2016-10-12 11:09:36 -07:00
Yann Collet
8b70d012f0 fix cmake 2016-10-12 10:23:53 -07:00
Yann Collet
38fb0dc4cf Merge pull request #416 from terrelln/exec-sequence 
Fix ZSTD_execSequence() edge case
 2016-10-12 10:17:53 -07:00
Nick Terrell
7158584399 Fix ZSTD_execSequence() edge case 2016-10-12 10:05:26 -07:00
Yann Collet
f52cd03e73 bumped version number 2016-10-11 17:29:27 -07:00
Yann Collet
ef2357d0d3 created error_private.c, so that a single list of error strings get included 2016-10-11 17:24:50 -07:00
Yann Collet
14efab827b added zstd_errors.h to include installation 2016-10-11 16:51:29 -07:00
Yann Collet
a17fd7312a changed error_public.h into zstd_errors.h 2016-10-11 16:41:09 -07:00
Yann Collet
18b51b99c0 sync fse 2016-10-11 08:21:09 -07:00
inikep
2d2613399a zstd.h: added the Introduction section 2016-10-06 16:28:21 +02:00
inikep
ba1db376ac fixed clang 3.5 warnings 2016-10-06 14:22:48 +02:00
inikep
82057aa7ec zstd.h: updated comments 2016-10-06 13:23:52 +02:00
Yann Collet
df6797447f update dictionary builder warning comments 2016-09-27 15:14:32 +02:00
Yann Collet
47094ea66b added comment on filePos 2016-09-26 18:03:33 +02:00
Yann Collet
cf409a7e2a fixed : init*_advanced() followed by reset() with different pledgedSrcSiz 2016-09-26 16:41:05 +02:00
Yann Collet
2f2639438a zstreamtest can fuzztest pledgedSrcSize 2016-09-26 14:06:08 +02:00
Christophe Chevalier
dc245e91cb Changed to use ZSTDLIBv06_API and ZSTDLIBv07_API for DLL exports to fix warning 
- changed name to prevent collision with ZSTDLIB_API used by non-legacy dll exports
 2016-09-23 17:09:36 +02:00
Yann Collet
21412bb3f6 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-09-22 15:57:56 +02:00
Yann Collet
51f4d566c2 small decompression speed boost for very small data 2016-09-22 15:57:28 +02:00
Yann Collet
97b378a6f8 Streaming : dictionary compression on multiple files / segments can correctly provide srcSize into header (when provided) using pledgedSrcSize. 2016-09-21 17:20:19 +02:00
Yann Collet
993060e0f2 cli : better adaptation to small files 2016-09-21 16:46:08 +02:00
Yann Collet
1eb2fdc74f bumped version number 2016-09-18 12:21:47 +02:00
Yann Collet
a6bdf55759 fixed memory leak 2016-09-15 17:02:06 +02:00
Yann Collet
644a8da88a fixed minor conversion warning 2016-09-15 16:16:21 +02:00
Yann Collet
4cb212938c introduced ZSTD_resetCStream() 2016-09-15 14:54:07 +02:00
Yann Collet
fa0c09760c variable renaming 2016-09-15 14:11:01 +02:00
Yann Collet
d7c6589df8 support ZSTD_sizeof_*() on NULL 
added ZSTD_sizeof_CDict()
 2016-09-15 02:57:27 +02:00
Yann Collet
e91c4b4cef introduced ZSTD_resetDStream() . 
added : ZSTD_sizeof_DDict()
 2016-09-14 16:55:44 +02:00
Yann Collet
d092d77cfc minor variable renaming 2016-09-14 16:14:57 +02:00
Yann Collet
64deef3bee Fixed srcSize=1 2016-09-14 00:16:07 +02:00
Yann Collet
26ec254066 new strategy for faster DDict decompression 2016-09-13 16:52:16 +02:00
Yann Collet
ac175d46d4 updated comments 2016-09-13 00:51:47 +02:00
Yann Collet
a3481d6de0 make uninstall 2016-09-12 05:04:26 +02:00
Yann Collet
b3060f7a9e changed streaming decoder behavior : now, when all compressed frame is consumed, it means decompression is completed, with regenerated data fully flushed. 2016-09-09 16:44:16 +02:00
Yann Collet
01c199226a updated decompression streaming example 2016-09-08 19:29:04 +02:00
Yann Collet
5c6d244973 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-09-07 14:54:54 +02:00
Yann Collet
ac8bace6b1 support large skippable frames 2016-09-07 14:54:23 +02:00
Yann Collet
0e07bf3f60 added comments on searchLength min / max (#337) 2016-09-07 06:33:02 +02:00
Yann Collet
95d07d7447 introduced CHECK_E 2016-09-06 16:38:51 +02:00
Yann Collet
3e21ec5b01 introduced CHECK_F 2016-09-06 15:36:19 +02:00
Yann Collet
5c956d593c FORCE_INLINE common definition 2016-09-06 15:05:19 +02:00
Yann Collet
edbcd9f5b2 fixed zbufftest 2016-09-06 14:30:57 +02:00
Yann Collet
b624922b14 fixed checksum 2016-09-06 11:16:57 +02:00
Yann Collet
a7737f6a60 improved compression on small files when using same parameters 2016-09-06 09:44:59 +02:00
Yann Collet
7ae67bb18a small compression speed gains with using_CDict 2016-09-06 06:28:05 +02:00
Yann Collet
1d4208c029 clarified streaming decompression inlined doc 2016-09-06 05:16:40 +02:00
Yann Collet
7c83dfd5c2 ZSTD_frameHeaderSize_prefix (#340), as result of ZSTD_initStream 2016-09-05 19:47:43 +02:00
Yann Collet
fa72f6bdce clarified inline doc for streaming 2016-09-05 17:39:56 +02:00
Yann Collet
c73a8109bb Merge pull request #344 from inikep/dev10 
unified error codes for legacy decoders
 2016-09-05 07:46:33 -07:00
inikep
45db83f98d ZSTD_decodeLiteralsBlock renamed to ZSTDv01_decodeLiteralsBlock 2016-09-05 14:46:24 +02:00
inikep
476964f6a1 ZSTD_decodeSeqHeaders renamed to ZSTDv01_decodeSeqHeaders 2016-09-05 13:34:57 +02:00
inikep
c13faa1b0f legacy decoders: restored #include <intrin.h> for VC++ 2016-09-05 13:25:07 +02:00
inikep
8161e7321a unified error codes for legacy decoders 2016-09-05 12:29:51 +02:00
Thomas Klausner
b85cdabd50 Enable install targets for NetBSD. 2016-09-04 14:37:57 +02:00
Yann Collet
33a0465a51 fixed a few links 2016-09-02 22:11:49 -07:00
Yann Collet
d56dbc02d3 removed g_displayLevel 2016-09-02 17:28:41 -07:00
Yann Collet
855766d73d clarified dictionary in format description 2016-09-02 17:04:49 -07:00
Yann Collet
d725427a3c g_time => local displayTime 2016-09-02 15:32:39 -07:00
Yann Collet
1563bfeabc fixing FORCE_INLINE for older compilers (#330) 2016-09-02 11:44:21 -07:00
Yann Collet
7304eb7c09 bumped version number 2016-09-01 15:49:26 -07:00
Yann Collet
901e85fe26 version bump 2016-08-31 07:51:25 -07:00
Yann Collet
1c59c20903 removed redundant files 2016-08-31 07:15:44 -07:00
Yann Collet
599c69d917 minor Makefile updates 2016-08-30 13:33:20 -07:00
David Lam
e10f7f3dcb merge 2016-08-30 12:03:36 -07:00
Yann Collet
4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
Yann Collet
3b15f1f10f minor refactor 2016-08-30 09:58:50 -07:00
Yann Collet
240795bef7 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-08-30 06:51:55 -07:00
Yann Collet
14200a20f0 Fixed issue #304, reported by @borzunov 2016-08-30 06:51:00 -07:00
David Lam
da9d3b7057 Cleanup some errors in typedef comments and remove duplicated HOWTO from zbuff_decompress.c 2016-08-29 17:31:51 -07:00
Yann Collet
09c3c8e885 Merge pull request #307 from inikep/dev08 
updated README.md
 2016-08-29 16:32:33 -07:00
inikep
6416b0d705 updated README.md 2016-08-29 13:04:26 +02:00
Yann Collet
23b6e05d8e ZSTD_malloc() and ZSTD_free(), to simplify customMem 2016-08-28 21:05:43 -07:00
Yann Collet
5f53b0335e fixed continuation context 2016-08-28 10:00:49 -07:00
Yann Collet
767d8f66fa legacy contexts can be re-used 2016-08-28 08:19:47 -07:00
Yann Collet
4bf317dd00 first version supporting legacy streams (transparent decoding) 2016-08-28 07:43:34 -07:00
Yann Collet
e19a9ef05d update compression level table 2016-08-26 20:02:49 +02:00
Yann Collet
9a021c1aae fixed some minor clang warnings 2016-08-26 09:05:06 +02:00
Yann Collet
cb5a320705 made -Wdocumentation a clang only flag 2016-08-26 08:06:36 +02:00
Yann Collet
87c18b2ebd fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
Yann Collet
0d59a6f73a removed debug strings 2016-08-25 22:42:46 +02:00
Yann Collet
5a02b69215 reinforced fix for huge files 2016-08-25 22:24:59 +02:00
Yann Collet
96bdd87de4 fixed : compression bug on very large files 2016-08-25 14:34:42 +02:00
inikep
a3a47ec4d0 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into Other 2016-08-24 21:25:49 +02:00
Yann Collet
a2cdffe556 fixed wrong parameter issue 2016-08-24 19:42:15 +02:00
inikep
e416e30019 remove unnecessary comments 2016-08-24 17:32:09 +02:00
inikep
4e90f6c1e0 removed ZSTD_LOG_ENCODE and ZSTD_LOG_BLOCK 2016-08-24 17:24:11 +02:00
inikep
83388e109f removed ZSTD_LOG_PARSER 2016-08-24 17:22:20 +02:00
inikep
8a36f8527c removed stats in debug mode 2016-08-24 17:19:12 +02:00
inikep
57ef4b1a0d zstd_v07.c: removed unused macros 2016-08-24 17:16:56 +02:00
Yann Collet
24b68a5b23 update cLevel table for 256KB 2016-08-24 14:22:26 +02:00
Yann Collet
c54692faeb improved level 3 2016-08-24 01:45:46 +02:00
Yann Collet
17e482efdd added ZSTD_setDStreamParameter() 2016-08-23 16:58:10 +02:00
Yann Collet
3071c3e303 STREAM_WINDOW_MAX : protect streaming from unreasonable memory requirements 2016-08-23 01:34:34 +02:00
Yann Collet
70e3b31306 fixed playtests on os-x 2016-08-23 01:18:06 +02:00
Yann Collet
cb3276329a added sizeof CStream and DStream 2016-08-23 00:31:59 +02:00
Yann Collet
d1733f7417 fixed crc bug in rare timing conditions within bench.c 2016-08-21 01:04:46 +02:00
Yann Collet
8baf78a291 minor coding style 2016-08-20 13:04:20 +02:00
Yann Collet
1bee2d5e08 slight decompression speed improvement 2016-08-20 02:59:04 +02:00
Yann Collet
0cfe2ec2fd sync fse version 2016-08-20 00:26:26 +02:00
Yann Collet
af1960396b sync huff0 2016-08-19 19:38:19 +02:00
Yann Collet
7be46bf8f2 promoted streaming API to stable 
(except _advanced() variants)
 2016-08-19 18:39:36 +02:00
Yann Collet
fdba57d513 update version number 2016-08-19 18:32:30 +02:00
Yann Collet
da3fbcb302 Added ZDICT_getDictID() 2016-08-19 14:23:58 +02:00
Yann Collet
a5dbf9f629 Merge pull request #297 from borzunov/dev 
Export functions related to dictionary compression from DLL
 2016-08-18 15:05:01 +02:00
Yann Collet
49d105cfcf better warning and error messages in case of dictionary training failure (#292) 2016-08-18 15:02:11 +02:00
Alexander Borzunov
0f6f17a14f Rename ZSTDLIB_API to ZDICTLIB_API in zdict.h 2016-08-18 16:47:06 +05:00
Alexander Borzunov
1f48382b1a Export functions related to dictionary compression from DLL 2016-08-18 16:12:49 +05:00
Yann Collet
e80d15304a Merge pull request #296 from inikep/Other 
Other
 2016-08-18 11:48:48 +02:00
inikep
a7bb322a93 removed never referenced functions 2016-08-18 10:30:21 +02:00
Yann Collet
18442c1482 minor refactoring 2016-08-18 01:40:32 +02:00
Yann Collet
c411902230 fixed g++ conversion warning 2016-08-17 01:50:54 +02:00
Yann Collet
53e17fbd5e updated streaming API 2016-08-17 01:39:22 +02:00
Yann Collet
655393cc72 updated doc for streaming API 2016-08-16 15:11:28 +02:00
Yann Collet
104e5b072d added : streaming decompression API 2016-08-16 15:11:28 +02:00
Yann Collet
5a0c8e2439 new streaming API (compression) 2016-08-16 15:11:27 +02:00
Yann Collet
ba92046031 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-08-11 22:10:04 +02:00
Yann Collet
e9b414d825 fixed msan warning (#281) 2016-08-11 22:09:09 +02:00
inikep
5f49eba512 added usage of rep[0]-1 for the optimal parser 2016-08-10 15:01:53 +02:00
inikep
98e08cbe34 fixed: tree not updated after finding very long rep matches 2016-08-10 15:00:30 +02:00
inikep
038d1497c9 fixed compilation with Visual Studio 2005 2016-08-10 14:30:10 +02:00
inikep
48849f86f0 fixed compilation with Intel Compiler with Windows 2016-08-10 14:26:35 +02:00
Yann Collet
1ea5622a32 updated xxhash 2016-08-10 09:40:08 +02:00
Yann Collet
666398e7ed added : xxhash namespace enforced from xxhash.h. 
added : xxhash namespace test.
removed : -DXXH_NAMESPACE
 2016-08-10 08:16:51 +02:00
Yann Collet
8ded0b84aa update xxhash to v0.6.2 2016-08-10 07:40:40 +02:00
Yann Collet
280f9a8754 minor comment 2016-08-08 00:44:00 +02:00
Yann Collet
e0b4a2d40f fixed dictionary generation, reported by Bartosz Taudul 2016-08-03 03:36:03 +02:00
Yann Collet
ae40b18d55 bumped library number 2016-08-03 01:59:21 +02:00
Yann Collet
0763905f44 ZSTD_compress_usingCDict() correctly provides original size by default in frame header 
Fixed dictionary examples
 2016-08-03 01:57:57 +02:00
Yann Collet
bf2bc112bb bench : controlled display update when loading lot of files 2016-08-02 23:48:13 +02:00
Yann Collet
346efccc35 fixed doc typo 2016-08-02 14:26:00 +02:00
Yann Collet
f116e87f59 fixed analyzer warning 2016-08-01 19:15:18 +02:00
Yann Collet
9ba929f1d4 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-08-01 02:26:59 +02:00
Yann Collet
3ca750372d updated doc (#269) 2016-08-01 02:26:20 +02:00
Yann Collet
c55eb18c11 Merge pull request #267 from inikep/dev08 
fixed ZSTD_compressBlock_opt_extDict_generic
 2016-07-31 22:00:16 +02:00
inikep
056df510aa fixed ZSTD_compressBlock_opt_extDict_generic 2016-07-31 20:08:53 +02:00
Yann Collet
917fe188f1 Implemented repOffset "minus 1" on ll==0 2016-07-31 04:01:57 +02:00
Yann Collet
2a2ba3691c Merge pull request #266 from jrmarino/master 
Enable build on FreeBSD ports (includes DragonFly BSD) [dev branch]
 2016-07-31 02:21:26 +02:00
jrmarino
0d07ec0c0c Enable build on FreeBSD ports (includes DragonFly BSD) 
Zstd has been introduced to FreeBSD ports
(http://www.freshports.org/archivers/zstd/) which DragonFly BSD also
uses.  FreeBSD and DragonFly use the install targets (albeit modified in
some cases) so they must be added to the associated Makefile filters.
 2016-07-30 19:11:15 -05:00
Yann Collet
8cebfd1d26 fix attempt on test-zstd-speed 2016-07-31 01:59:23 +02:00
Yann Collet
66f69e58d2 restore decompression speed on fizzle 2016-07-30 15:32:47 +02:00
Yann Collet
3b2bd1d11c zstd_opt uses same tables as zstd_compress 2016-07-30 13:21:41 +02:00
Yann Collet
f714f59c16 fixed visual warning 2016-07-30 12:05:28 +02:00
Yann Collet
761f8dbbd2 back to normal table cell copy 2016-07-30 11:43:53 +02:00
Yann Collet
3c6b808870 minor decompression speed gains 2016-07-30 03:20:47 +02:00
Yann Collet
70a9ff4af3 fixed too large selectivity level, reported by Ilona Papava 2016-07-30 01:09:14 +02:00
Yann Collet
c0ce4f1211 slightly improved compression speed 2016-07-30 00:55:13 +02:00
Yann Collet
ed57d8530a new seqStore 2016-07-29 21:22:17 +02:00
Yann Collet
6b615d32cd Updated API comments, following suggestions by Bryan O'Sullivan 2016-07-29 19:40:37 +02:00
Yann Collet
c00d30fbe4 Merge pull request #264 from inikep/dev08 
Dev08
 2016-07-29 17:42:30 +02:00
inikep
6b68ba2079 zstd_opt.h: fixed checking of rep codes (2) 2016-07-29 16:45:39 +02:00
inikep
59b86fc141 zstd_opt.h: fixed checking of rep codes 2016-07-29 11:00:33 +02:00
Yann Collet
6a82f0f8bf minor comments 2016-07-29 00:55:45 +02:00
Yann Collet
ffa7d0ac1e clarified comment 2016-07-28 21:01:17 +02:00
Yann Collet
4c5bbf64f9 fixed : frame concatenation without checksum 2016-07-28 20:30:25 +02:00
Yann Collet
60ba31c570 zbuff uses ZSTD_compressEnd() 2016-07-28 19:55:09 +02:00
Yann Collet
16e73033ad introduced stage zbf_end 2016-07-28 16:32:34 +02:00
Yann Collet
62470b4bab Changed ZSTD_compressEnd() 2016-07-28 15:29:08 +02:00
Yann Collet
e7bf9156d1 Clarified API comments, from suggestions by ‎Bryan O'Sullivan‎ 2016-07-28 05:00:57 +02:00
Yann Collet
d469a98c01 Clarified API comments, from suggestions by ‎Bryan O'Sullivan‎ 2016-07-28 04:55:03 +02:00
Yann Collet
19c1002e46 applied ZSTD_compressContinueThenEnd() 2016-07-28 01:25:46 +02:00
Yann Collet
5b56739b63 created ZSTD_compressContinueThenEnd() 2016-07-28 01:17:22 +02:00
Yann Collet
c991cc1828 new frame end, 32-bits checksums 2016-07-28 00:55:43 +02:00
Yann Collet
d4180cad9c minor code refactoring 2016-07-27 21:21:36 +02:00
Yann Collet
731ef16fc1 minor code style refactoring 2016-07-27 21:05:12 +02:00
Yann Collet
4b9ca0a6b5 minor example variation 2016-07-27 19:53:19 +02:00
Yann Collet
4110534886 ZSTD_maxCLevel() is promoted to "stable" API (#254, by @FrancescAlted) 2016-07-27 15:09:11 +02:00
Yann Collet
55a8bea0b5 fixed dictionary generation 2016-07-27 14:48:47 +02:00
Yann Collet
c154d9d6a2 better support for large dictionaries (> 128 KB) 2016-07-27 14:37:00 +02:00
Yann Collet
07626dfa51 improved dictbuilder notifications on selectivity 2016-07-27 13:28:46 +02:00
Yann Collet
f796f7ab45 removed fastscan mode 2016-07-27 12:53:54 +02:00
Yann Collet
dd25a27702 added tutorial warning messages for dictBuilder 2016-07-27 12:43:09 +02:00
inikep
003c7a8568 optimal parser: removed ZSTD_REP_INIT 2016-07-27 11:07:13 +02:00
Yann Collet
04cdd8660d Merge pull request #262 from ebiggers/misc_updates 
Miscellaneous updates
 2016-07-27 01:25:45 +02:00
Eric Biggers
0a55e7a0bb ZSTD_decompressFrame(): use remainingSize instead of iend - ip 
Same behavior, but no need to have redundant variables.
 2016-07-26 13:22:27 -07:00
Eric Biggers
aa6c70bf60 ZSTD_decompressFrame(): pass up error code from ZSTD_decodeFrameHeader() 2016-07-26 13:22:27 -07:00
Eric Biggers
e4d0265ea9 Replace remaining references to "direct mode" with "single segment mode" 2016-07-26 13:22:27 -07:00
Yann Collet
d50f9db3ea Improved speed on clang and gcc -O2, thanks to @ebiggers ! (#263) 2016-07-26 21:30:35 +02:00
Yann Collet
7adc2328a3 fixed --test on zero-length files, reported by @amnilsson 2016-07-26 15:49:24 +02:00
inikep
4178f5c289 fixed gcc warning: always_inline function might not be inlinable 2016-07-25 21:17:45 +02:00
inikep
fca90f8f60 legacy decoder for v0.7 format 2016-07-25 17:49:08 +02:00
Yann Collet
cbc5e9dc19 fixes oob read 2016-07-24 18:02:04 +02:00
Yann Collet
38b75ddeb2 removed special case all-1 huffman distribution 2016-07-24 15:35:59 +02:00
Yann Collet
7ed5e33b89 minor comment changes 2016-07-24 14:26:11 +02:00
Yann Collet
10b9c13d07 fixed doc on cLevel default, reported by Oliver Lange 2016-07-24 01:21:53 +02:00
Yann Collet
f8e7b5363f unified encoding types 2016-07-23 16:31:49 +02:00
Yann Collet
571a59034a changed enccoding type order : raw, rle, compressed, repeat-stats 2016-07-23 15:52:05 +02:00
Yann Collet
c2e1a68d81 changed streamNb order to 1-4-4-4 2016-07-22 17:30:52 +02:00
Yann Collet
772d912c2f more complete support for literals repeat mode 2016-07-22 15:04:25 +02:00
Yann Collet
9f2d82d4a4 fixed : big-endian decoding 2016-07-22 14:37:10 +02:00
Yann Collet
32faf6c8e7 fixed conversion warnings 2016-07-22 14:37:09 +02:00
Yann Collet
5e45a5fbb3 force loop-align to 32 for zstd_decompress 2016-07-22 14:37:09 +02:00
Yann Collet
5288ac0cb7 changed filed order 2016-07-22 14:37:09 +02:00
Yann Collet
198e6aac44 Literals header fields use little endian convention 2016-07-22 14:37:09 +02:00
Yann Collet
6fa05a2371 cBlockSize uses little-endian convention 2016-07-22 14:37:09 +02:00
Yann Collet
7bf72bbf5e update header to v0.8 2016-07-22 14:37:09 +02:00
Yann Collet
5894ea8d01 updated cLevels 2016-07-22 14:36:46 +02:00
Yann Collet
d5c5a77990 minor comments clarifications 2016-07-20 13:35:14 +02:00
Yann Collet
572b817be3 Merge pull request #253 from gymdis/heapmode_off_legacy_fix 
Fix compile issue with ZSTD_LEGACY_SUPPORT=1 and ZSTD_HEAPMODE=0
 2016-07-19 13:52:03 +02:00
Christopher Bergqvist
780a9fa857 Fix compile issue with ZSTD_LEGACY_SUPPORT=1 and ZSTD_HEAPMODE=0 2016-07-19 13:25:38 +02:00
Yann Collet
cf05b9d477 ZSTD_getBlockSizeMax() 2016-07-18 16:52:10 +02:00
Yann Collet
16aa38b0e0 minor doc clarifications 2016-07-18 03:52:47 +02:00
Yann Collet
85f3919960 moved zstd.h to /lib 2016-07-17 20:42:21 +02:00
Yann Collet
9375590462 update version to v0.7.5 2016-07-17 16:44:18 +02:00
Yann Collet
e557fd5e92 minor compression level corrections 2016-07-17 16:21:37 +02:00
Yann Collet
d54b2d23b4 minor static assert for 32/64 bits system. Suggested by @ebiggers 2016-07-17 15:53:18 +02:00
Yann Collet
972e5806ee fixed : premature frame end on zero-sized raw block - reported by @ebiggers 2016-07-17 15:39:24 +02:00
luben karavelov
10f999f856 Add legacy support for the low-level streaming API 2016-07-17 01:03:26 +02:00
Yann Collet
6cacd34d44 minor formatting changes 2016-07-15 17:58:13 +02:00
Yann Collet
f6ff53cd4e implemented dictID reserved ranges 2016-07-15 17:03:38 +02:00
Yann Collet
98c8884999 added target zstd in root Makefile 2016-07-15 16:12:38 +02:00
Yann Collet
961b6a0e34 ZSTD_compressBlock() limits block size depending on windowLog parameter 2016-07-15 11:58:49 +02:00
Yann Collet
227cc39e15 improved efficiency for large messages with small dictionaries 2016-07-15 11:27:09 +02:00
Yann Collet
ea2ecdc315 fixed issue with small dictionary 2016-07-14 23:27:31 +02:00
Yann Collet
e9ed5cdc94 fixed minor coverity warning 2016-07-14 21:02:57 +02:00
Yann Collet
b23e1ce319 removed debugging traces 2016-07-14 17:46:38 +02:00
Yann Collet
17508f1a16 fixed a few minor coverity warnings 2016-07-14 17:18:20 +02:00
Yann Collet
8847238cac simplified ZSTD_estimateCCtxSize() 2016-07-14 17:05:38 +02:00
Yann Collet
69c2cdb45c fixed conversion warning 2016-07-14 16:52:45 +02:00
Yann Collet
5e80dd3261 fixed minor coverity warnings 2016-07-13 19:21:57 +02:00
Yann Collet
3c174f4da9 fixed minor coverity warning 2016-07-13 17:25:53 +02:00
Yann Collet
2b1a3638e6 changed macro name to ZSTDCLI_CLEVEL_DEFAULT 2016-07-13 15:16:00 +02:00
Yann Collet
3c242e79d3 updated compression levels table 2016-07-13 14:56:24 +02:00
Yann Collet
fbc69f8649 changed for #245 2016-07-13 13:52:58 +02:00
Yann Collet
eed2081e55 fixed conversion warning 2016-07-12 15:11:40 +02:00
Yann Collet
a43a854cdb updated paramgrill 2016-07-12 13:42:10 +02:00
Yann Collet
73d74a05b9 fixed dfast strategy 2016-07-12 13:03:48 +02:00
Yann Collet
45dc35628c first version of doubleFast 2016-07-12 09:47:31 +02:00
Yann Collet
d158c35e9f added ZSTD_estimateDCtxSize() 2016-07-11 13:46:25 +02:00
Yann Collet
8e0ee681b8 added ZSTD_sizeofDCtx() 2016-07-11 13:09:52 +02:00
Yann Collet
3ae543ce75 added ZSTD_estimateCCtxSize() 2016-07-11 03:12:17 +02:00
Yann Collet
25c506601c promote ZSTD_getDecompressedSize() to stable API 2016-07-10 01:46:18 +02:00
Yann Collet
3b6ae77e15 comment clarification 2016-07-08 23:42:22 +02:00
Yann Collet
722e14bb65 fixed compilation error in decompression module 2016-07-08 19:22:16 +02:00
Yann Collet
bd10607063 updated spec 2016-07-08 19:16:57 +02:00
Yann Collet
c5fb5b7fcd support offset > 128 MB 2016-07-08 13:13:37 +02:00
Yann Collet
ed3845d3fa introduced ZSTD_WINDOWLOG_MAX_32 (#239), suggested by @GregSlazinski 2016-07-08 12:57:10 +02:00
Yann Collet
26f681451f updated doc 2016-07-08 11:45:08 +02:00
Yann Collet
19c27d27f1 simplified legacy functions, no longer need magic number 2016-07-07 14:40:13 +02:00
Yann Collet
e72efeb0a1 removed "error_public.h" dependency from "zstd.h" 2016-07-07 14:17:40 +02:00
Yann Collet
974f52fc5d Added "dictionary decompression" example 2016-07-07 14:08:00 +02:00
Yann Collet
e09d38e921 removed mem.h dependency from zbuff.h (experimental section) 2016-07-07 13:17:37 +02:00
Yann Collet
f323bf7d32 added : ZSTD_getDecompressedSize() 2016-07-07 13:14:21 +02:00
Yann Collet
52c04fe58f removed mem.h dependency from zstd.h (experimental section) 2016-07-07 11:53:18 +02:00
Yann Collet
f246cf5423 ZSTD_decompress_usingDDict() compatible with Legacy mode 2016-07-06 20:32:27 +02:00
Yann Collet
29652e2618 sample set limitation closer to 2 GB 2016-07-06 16:25:46 +02:00
Yann Collet
99b045b70a dictBuilder protection vs huge sample sets (>2 GB) 2016-07-06 16:12:38 +02:00
Yann Collet
445d49d898 fixed conversion warning 2016-07-06 13:27:22 +02:00
Yann Collet
a295b3170f fixed conversion warning 2016-07-06 13:13:12 +02:00
Yann Collet
517e1ba623 fixed dictBuilder issue with HC levels. Reported by Bartosz Taudul. 2016-07-06 12:35:09 +02:00
Yann Collet
fe07eaa972 simplified ZSTD_decodeSequence() 2016-07-06 02:25:44 +02:00
Yann Collet
9ca73364e6 updated spec 2016-07-05 10:53:38 +02:00
Yann Collet
f9cac7a734 Added GNU separator --, to specifies that all following arguments are necessary file names (and not commands). Suggested by @chipturner (#230) 2016-07-04 18:18:24 +02:00
Yann Collet
23f05ccc6b updated specifications 2016-07-04 16:13:11 +02:00
Yann Collet
d916c908e0 updated doc 2016-07-04 00:42:58 +02:00
Yann Collet
698cb63305 Updated specifications 2016-07-03 18:49:35 +02:00
Yann Collet
d57dffbe76 ZSTD_storeSeq takes an U32 as offset type 2016-07-03 01:48:26 +02:00
Yann Collet
302ff036f6 simplified repcodes for lazy_extDict 2016-07-03 01:28:16 +02:00
Yann Collet
9634f67107 fix lazy parser 2016-07-03 01:23:58 +02:00
Yann Collet
92d75667e4 fix for fast mode 2016-07-03 01:10:53 +02:00
Yann Collet
5e734ad09b revert fix 2016-07-02 23:55:34 +02:00
Yann Collet
0d5bf8f06f fixed risk of segfault on very large files (multiple GB) 2016-07-02 21:39:47 +02:00
Yann Collet
2fa9904844 update specification and comments 2016-07-01 20:55:28 +02:00
Yann Collet
c093208ab8 fix : potential leak (#229) 2016-06-30 14:07:30 +02:00
Yann Collet
6c6e1751f6 use ZSTD_getParams() to simplify code 2016-06-27 15:28:45 +02:00
Yann Collet
3d2cd7f816 Introduced ZSTD_getParams() 
bench now uses ZSTD_createCDict_advanced()
 2016-06-27 15:12:26 +02:00
Yann Collet
529d9c7dee updated version to v0.7.2 2016-06-27 10:03:10 +02:00
Yann Collet
d4f4e58ee1 fixed ZSTD_decompressBlock() using multiple blocks 2016-06-27 01:31:35 +02:00
Yann Collet
63b5e7a2ea Improved comments 2016-06-26 17:42:15 +02:00
Yann Collet
3755eb8fea fixed strict-aliasing warning on gcc6 2016-06-22 13:15:53 +02:00
Yann Collet
23042929da Fixed : dictBuilder fails if first sample is too small 2016-06-22 11:05:34 +02:00
Yann Collet
391a128794 fix : segfault in command line during automatic overwrite protection mode 2016-06-21 17:06:25 +02:00
Yann Collet
bda68c253b refactored ZBUFF_compressEnd() for better maintainability 2016-06-21 15:18:11 +02:00
Yann Collet
aa29226b7c fix : ZBUFF_compressEnd() gives right amount remaining to flush, including future epilogue 2016-06-21 14:04:57 +02:00
Yann Collet
f15c1cb00c Fixed : ZBUFF_compressEnd() called multiple times with too small dst buffer (#206) 2016-06-21 13:11:48 +02:00
Yann Collet
a49e066b26 clarified comments on ZSTD_compressContinue() 2016-06-21 11:54:03 +02:00
Yann Collet
d4f38d0dcd updated library to v0.7.1 2016-06-21 10:15:43 +02:00
Yann Collet
22d76322ce minor refactor 2016-06-21 08:01:51 +02:00
Yann Collet
a436a529bc minor : fast_extDict does no longer skip first byte 2016-06-20 23:34:04 +02:00
Yann Collet
4623d11571 new correction, less extreme replacement value 2016-06-20 19:15:37 +02:00
Yann Collet
5477cc25f7 fixed corruption error related to inter-blocks rep-offset 2016-06-20 18:31:25 +02:00
Yann Collet
e4811ba761 Modified : ZSTD_createDDict() accepts dictionary < 8 bytes in pure content mode (reported by @chipturner) 2016-06-19 23:06:54 +02:00
Yann Collet
06d9a73b48 minor refactor, using WILDCOPY_OVERLENGTH macro instead of hard-coded 8 2016-06-19 14:27:21 +02:00
Yann Collet
19cab46f2f Joined seqStore initialization at dispatch point 2016-06-17 12:54:52 +02:00
Yann Collet
510cff3570 minor comment change 2016-06-16 16:39:55 +02:00
Yann Collet
4948f270b3 make room for reserved "information bit" in frame header 2016-06-16 15:38:51 +02:00
Yann Collet
23ba41533a Fixed zstd_opt encoding error with repeat-offsets 2016-06-16 13:20:46 +02:00
Yann Collet
80d033fb43 fixed ptr arithmetic warning 2016-06-16 01:41:50 +02:00
Yann Collet
ad39b7a718 zdict stores standard rep-offset. 
It can use custom ones, but the proper formula and impact on statistics is not done yet.
 2016-06-16 01:14:41 +02:00
Yann Collet
736d419289 strengthened dict loading on decompresson side 2016-06-16 01:05:04 +02:00
Yann Collet
8e36a9c169 decoder restores repOffsets from dictionary 2016-06-16 01:05:04 +02:00
Yann Collet
52a0622beb RepsCodes are saved into Dict 
(uncomplete : need decompression to regenerate them)
 2016-06-16 01:05:04 +02:00
Yann Collet
efd0b4993a fixed fuzzer error (inter-block repeated offsets) 2016-06-16 00:53:56 +02:00
Yann Collet
9b998e4d08 Fixed decompression of literals in dictionary mode 2016-06-15 23:11:20 +02:00
Yann Collet
d059092897 fixed conversion warnings 2016-06-14 15:34:24 +02:00
Yann Collet
45c03c564f fixed corruption with inter-blocks repeated offsets 2016-06-14 13:46:11 +02:00
Yann Collet
4266c0a2fd adding inter-blocks rep-offsets 2016-06-14 01:49:25 +02:00
Yann Collet
43dfe01919 Check repIndex for validity 2016-06-13 21:43:06 +02:00
Yann Collet
18c8f79f3e fixed gcc warning on uninitialized structure variable 2016-06-12 22:51:52 +02:00
Yann Collet
cd98f93cff Fixed decompression issue with invalid data 2016-06-11 23:26:22 +02:00
Yann Collet
37fece22e8 enable repeat-entropic-stats mode 2016-06-11 02:52:42 +02:00
Yann Collet
d60a5bf900 Literal decompression builds Huffman tables within shared space (for later re-use) 2016-06-11 02:35:31 +02:00
Yann Collet
237ad4beb3 Added single-stream decompression variant using external DTable 2016-06-11 01:46:03 +02:00
Yann Collet
289bbd52e5 Updated huff0 2016-06-11 01:31:54 +02:00
Yann Collet
1869f7966e Merge pull request #205 from inikep/dev 
legacy decoder for v0.6
 2016-06-10 17:13:07 +02:00
Yann Collet
0974f681a4 completed .gitignore 2016-06-10 14:44:16 +02:00
Yann Collet
9dd12742f3 litBlockType_t is an enum 2016-06-10 00:12:26 +02:00
inikep
4923222412 fixed warnings from Travis 2016-06-09 20:03:30 +02:00
inikep
4000945a1d project updated for legacy decoder zstd_v06.c 2016-06-09 18:12:06 +02:00
inikep
bf853d5510 added legacy decoder for v0.6 format 2016-06-09 17:59:18 +02:00
Yann Collet
662a541431 updated huff0 - now generates a common HUF_DTable type for all decoding tables 2016-06-08 11:11:02 +02:00