AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
2,154 Commits 1 Branch 0 Tags 27 MiB
3d18088b38
Commit Graph

1094 Commits

Author SHA1 Message Date
Yann Collet
f246cf5423 ZSTD_decompress_usingDDict() compatible with Legacy mode 2016-07-06 20:32:27 +02:00
Yann Collet
29652e2618 sample set limitation closer to 2 GB 2016-07-06 16:25:46 +02:00
Yann Collet
99b045b70a dictBuilder protection vs huge sample sets (>2 GB) 2016-07-06 16:12:38 +02:00
Yann Collet
445d49d898 fixed conversion warning 2016-07-06 13:27:22 +02:00
Yann Collet
a295b3170f fixed conversion warning 2016-07-06 13:13:12 +02:00
Yann Collet
517e1ba623 fixed dictBuilder issue with HC levels. Reported by Bartosz Taudul. 2016-07-06 12:35:09 +02:00
Yann Collet
fe07eaa972 simplified ZSTD_decodeSequence() 2016-07-06 02:25:44 +02:00
Yann Collet
9ca73364e6 updated spec 2016-07-05 10:53:38 +02:00
Yann Collet
f9cac7a734 Added GNU separator --, to specifies that all following arguments are necessary file names (and not commands). Suggested by @chipturner (#230) 2016-07-04 18:18:24 +02:00
Yann Collet
23f05ccc6b updated specifications 2016-07-04 16:13:11 +02:00
Yann Collet
d916c908e0 updated doc 2016-07-04 00:42:58 +02:00
Yann Collet
698cb63305 Updated specifications 2016-07-03 18:49:35 +02:00
Yann Collet
d57dffbe76 ZSTD_storeSeq takes an U32 as offset type 2016-07-03 01:48:26 +02:00
Yann Collet
302ff036f6 simplified repcodes for lazy_extDict 2016-07-03 01:28:16 +02:00
Yann Collet
9634f67107 fix lazy parser 2016-07-03 01:23:58 +02:00
Yann Collet
92d75667e4 fix for fast mode 2016-07-03 01:10:53 +02:00
Yann Collet
5e734ad09b revert fix 2016-07-02 23:55:34 +02:00
Yann Collet
0d5bf8f06f fixed risk of segfault on very large files (multiple GB) 2016-07-02 21:39:47 +02:00
Yann Collet
2fa9904844 update specification and comments 2016-07-01 20:55:28 +02:00
Yann Collet
c093208ab8 fix : potential leak (#229) 2016-06-30 14:07:30 +02:00
Yann Collet
6c6e1751f6 use ZSTD_getParams() to simplify code 2016-06-27 15:28:45 +02:00
Yann Collet
3d2cd7f816 Introduced ZSTD_getParams() 
bench now uses ZSTD_createCDict_advanced()
 2016-06-27 15:12:26 +02:00
Yann Collet
529d9c7dee updated version to v0.7.2 2016-06-27 10:03:10 +02:00
Yann Collet
d4f4e58ee1 fixed ZSTD_decompressBlock() using multiple blocks 2016-06-27 01:31:35 +02:00
Yann Collet
63b5e7a2ea Improved comments 2016-06-26 17:42:15 +02:00
Yann Collet
3755eb8fea fixed strict-aliasing warning on gcc6 2016-06-22 13:15:53 +02:00
Yann Collet
23042929da Fixed : dictBuilder fails if first sample is too small 2016-06-22 11:05:34 +02:00
Yann Collet
391a128794 fix : segfault in command line during automatic overwrite protection mode 2016-06-21 17:06:25 +02:00
Yann Collet
bda68c253b refactored ZBUFF_compressEnd() for better maintainability 2016-06-21 15:18:11 +02:00
Yann Collet
aa29226b7c fix : ZBUFF_compressEnd() gives right amount remaining to flush, including future epilogue 2016-06-21 14:04:57 +02:00
Yann Collet
f15c1cb00c Fixed : ZBUFF_compressEnd() called multiple times with too small dst buffer (#206) 2016-06-21 13:11:48 +02:00
Yann Collet
a49e066b26 clarified comments on ZSTD_compressContinue() 2016-06-21 11:54:03 +02:00
Yann Collet
d4f38d0dcd updated library to v0.7.1 2016-06-21 10:15:43 +02:00
Yann Collet
22d76322ce minor refactor 2016-06-21 08:01:51 +02:00
Yann Collet
a436a529bc minor : fast_extDict does no longer skip first byte 2016-06-20 23:34:04 +02:00
Yann Collet
4623d11571 new correction, less extreme replacement value 2016-06-20 19:15:37 +02:00
Yann Collet
5477cc25f7 fixed corruption error related to inter-blocks rep-offset 2016-06-20 18:31:25 +02:00
Yann Collet
e4811ba761 Modified : ZSTD_createDDict() accepts dictionary < 8 bytes in pure content mode (reported by @chipturner) 2016-06-19 23:06:54 +02:00
Yann Collet
06d9a73b48 minor refactor, using WILDCOPY_OVERLENGTH macro instead of hard-coded 8 2016-06-19 14:27:21 +02:00
Yann Collet
19cab46f2f Joined seqStore initialization at dispatch point 2016-06-17 12:54:52 +02:00
Yann Collet
510cff3570 minor comment change 2016-06-16 16:39:55 +02:00
Yann Collet
4948f270b3 make room for reserved "information bit" in frame header 2016-06-16 15:38:51 +02:00
Yann Collet
23ba41533a Fixed zstd_opt encoding error with repeat-offsets 2016-06-16 13:20:46 +02:00
Yann Collet
80d033fb43 fixed ptr arithmetic warning 2016-06-16 01:41:50 +02:00
Yann Collet
ad39b7a718 zdict stores standard rep-offset. 
It can use custom ones, but the proper formula and impact on statistics is not done yet.
 2016-06-16 01:14:41 +02:00
Yann Collet
736d419289 strengthened dict loading on decompresson side 2016-06-16 01:05:04 +02:00
Yann Collet
8e36a9c169 decoder restores repOffsets from dictionary 2016-06-16 01:05:04 +02:00
Yann Collet
52a0622beb RepsCodes are saved into Dict 
(uncomplete : need decompression to regenerate them)
 2016-06-16 01:05:04 +02:00
Yann Collet
efd0b4993a fixed fuzzer error (inter-block repeated offsets) 2016-06-16 00:53:56 +02:00
Yann Collet
9b998e4d08 Fixed decompression of literals in dictionary mode 2016-06-15 23:11:20 +02:00
Yann Collet
d059092897 fixed conversion warnings 2016-06-14 15:34:24 +02:00
Yann Collet
45c03c564f fixed corruption with inter-blocks repeated offsets 2016-06-14 13:46:11 +02:00
Yann Collet
4266c0a2fd adding inter-blocks rep-offsets 2016-06-14 01:49:25 +02:00
Yann Collet
43dfe01919 Check repIndex for validity 2016-06-13 21:43:06 +02:00
Yann Collet
18c8f79f3e fixed gcc warning on uninitialized structure variable 2016-06-12 22:51:52 +02:00
Yann Collet
cd98f93cff Fixed decompression issue with invalid data 2016-06-11 23:26:22 +02:00
Yann Collet
37fece22e8 enable repeat-entropic-stats mode 2016-06-11 02:52:42 +02:00
Yann Collet
d60a5bf900 Literal decompression builds Huffman tables within shared space (for later re-use) 2016-06-11 02:35:31 +02:00
Yann Collet
237ad4beb3 Added single-stream decompression variant using external DTable 2016-06-11 01:46:03 +02:00
Yann Collet
289bbd52e5 Updated huff0 2016-06-11 01:31:54 +02:00
Yann Collet
1869f7966e Merge pull request #205 from inikep/dev 
legacy decoder for v0.6
 2016-06-10 17:13:07 +02:00
Yann Collet
0974f681a4 completed .gitignore 2016-06-10 14:44:16 +02:00
Yann Collet
9dd12742f3 litBlockType_t is an enum 2016-06-10 00:12:26 +02:00
inikep
4923222412 fixed warnings from Travis 2016-06-09 20:03:30 +02:00
inikep
4000945a1d project updated for legacy decoder zstd_v06.c 2016-06-09 18:12:06 +02:00
inikep
bf853d5510 added legacy decoder for v0.6 format 2016-06-09 17:59:18 +02:00
Yann Collet
662a541431 updated huff0 - now generates a common HUF_DTable type for all decoding tables 2016-06-08 11:11:02 +02:00
Yann Collet
302fb53a76 Removed ZSTD_*_usingPrepared?Ctx() declaration from public space 2016-06-07 12:16:49 +02:00
Yann Collet
81e13ef7cf first implementation of the new dictionary API (untested) 2016-06-07 00:51:51 +02:00
Yann Collet
9d504ae85b Added decoding of RLE blocks 2016-06-06 19:52:35 +02:00
Yann Collet
2cc72f1fd3 fixed initialization issue in bench 2016-06-06 17:50:07 +02:00
Yann Collet
e3d529403d fixed initialization mismatch in ZSTD_copyCCtx() 2016-06-06 11:07:33 +02:00
Yann Collet
142acbdea7 fixed minor visual conversion warning 2016-06-06 00:46:56 +02:00
Yann Collet
673f0d7cdc new frame format, allowing custom window size 2016-06-06 00:26:38 +02:00
Yann Collet
89703d20fb reduced dependencies 2016-06-05 01:50:33 +02:00
Yann Collet
51778b7cca updated README following merging of *_static.h 2016-06-05 01:38:10 +02:00
Yann Collet
a91ca620cf removed HUF_readStats() from public space 2016-06-05 01:33:55 +02:00
Yann Collet
d0e2cd15cb Merged fse_static into fse.h . Now requires FSE_STATIC_LINKING_ONLY macro. 2016-06-05 00:58:01 +02:00
Yann Collet
130fe11394 merged huf_static.h into huf.h . Requires HUF_STATIC_LINKING_ONLY macro. 2016-06-05 00:42:28 +02:00
Yann Collet
dc048d18d3 minor comment (detailing an #include motivation) 2016-06-05 00:32:23 +02:00
Yann Collet
49bb0041af removed ZSTD_highbit() from zstd_internal.h, as it is only used by zstd_compress.c 2016-06-04 20:17:38 +02:00
Yann Collet
d3b7f8d21f Merged zstd_static.h into zstd.h . Now requires ZSTD_STATIC_LINKING_ONLY macro 2016-06-04 19:47:02 +02:00
Yann Collet
ac110a1f21 Removed ZBUFF internal util function from public area 2016-06-04 19:16:49 +02:00
Yann Collet
5347aee8f7 merged zbuff_static.h into zbuff.h . Now requires ZBUFF_STATIC_LINKING_ONLY macro 2016-06-04 19:12:48 +02:00
Yann Collet
e69b8ccceb merged zdict_static.h into zdict.h. Now requires ZDICT_STATIC_LINKING_ONLY macro. 2016-06-04 18:56:23 +02:00
Yann Collet
198d127b35 minor comment change (unfinished description of new header format) 2016-06-04 18:40:55 +02:00
Yann Collet
f4f5affdf7 restore ZBUFF full-block-size, for better performance on small input 2016-06-03 23:09:28 +02:00
Yann Collet
ab7b6f1ece Merge pull request #198 from inikep/dev070 
Dev070
 2016-06-03 21:37:49 +02:00
inikep
3640396b1a fixed: deallocation of structures in case of error in ZBUFF_createCCtx and ZBUFF_createDCtx 2016-06-03 16:36:50 +02:00
Yann Collet
fe48775868 minor decoder code refactoring 2016-06-03 15:41:51 +02:00
inikep
2a74609b90 zlibWrapper: ZWRAP_createCCtx and ZWRAP_freeCCtx use custom memory allocation functions 2016-06-03 14:53:51 +02:00
inikep
3763c77f6b defaultCustomNULL replaced with defaultCustomMem 2016-06-03 13:28:20 +02:00
inikep
36fac00149 removed calloc calls from lib/ 2016-06-03 13:23:04 +02:00
inikep
db2f540414 added defaultCustomNULL 2016-06-03 12:56:56 +02:00
inikep
b74a468fad Merge remote-tracking branch 'refs/remotes/Cyan4973/dev070' into dev070 2016-06-02 22:09:09 +02:00
Yann Collet
923938edde Added -Wdeclaration-after-statement compilation flag 2016-06-02 17:56:00 +02:00
inikep
ff9114aee3 zlibWrapper: added support for custom memory allocation functions 2016-06-02 16:52:36 +02:00
inikep
c4807f4d2f default custom allocation functions moved to zstd_internal.h 2016-06-02 15:11:39 +02:00
inikep
2866951558 opaque parameter for custom memory allocation functions 2016-06-02 13:04:18 +02:00
inikep
9242816b56 fparamsPtr->windowLog==0 means that a frame is skippable 2016-06-01 18:47:04 +02:00
Yann Collet
70d1301d6e Changed ZSTD_adjustCParams() prototype 
`ZSTD_adjustCParams()` is now automatically invoked at the end of `ZSTD_getCParams()`
 2016-06-01 18:45:34 +02:00
Yann Collet
83c3f4427c upgraded zbufftest to also test advanced frame parameters no/checksum no/dictID 2016-06-01 17:44:53 +02:00
inikep
13f42d9085 VS2010 project: reverted zstdlib.rc 2016-06-01 14:44:31 +02:00
inikep
5c2771710d Merge remote-tracking branch 'refs/remotes/Cyan4973/dev070' into dev070 
# Conflicts:
#	.gitignore
#	lib/decompress/zstd_decompress.c
#	programs/zbufftest.c
 2016-06-01 09:16:11 +02:00
Yann Collet
202844ebd0 fixed zbufftest : 2016-06-01 00:44:36 +02:00
Yann Collet
8e3a36a6db decompression validates frame content checksum 2016-06-01 00:18:28 +02:00
inikep
a6b942018d Merge remote-tracking branch 'refs/remotes/origin/dev' into dev070 
# Conflicts:
#	.travis.yml
#	Makefile
#	lib/common/zstd_static.h
#	programs/Makefile
#	projects/VS2008/zstd/zstd.vcproj
#	projects/VS2008/zstdlib/zstdlib.vcproj
#	projects/cmake/lib/CMakeLists.txt
#	projects/cmake/programs/CMakeLists.txt
 2016-06-01 00:07:09 +02:00
Yann Collet
f2a3b6e7b4 added : frame content checksum 2016-05-31 22:23:45 +02:00
inikep
43aa9fe8b3 fixed skippable frame 2016-05-31 19:36:51 +02:00
inikep
f772bf54a5 support for skippable frames 2016-05-31 12:43:46 +02:00
Giuseppe Ottaviano
370b751e24 Expose function to add entropy tables to pre-built dictionary. 
In some cases a custom dictionary building algorithm tailored for a specific
input can be more effective than the one produced by `ZDICT_trainFromBuffer`,
but with the current API it's not possible encode the entropy tables into the
custom-built dictionary.

This commit extracts the logic to add entropy tables to a dictionary from
`ZDICT_trainFromBuffer` and exposes it as a function
`ZDICT_addEntropyTablesFromBuffer`.
 2016-05-30 19:50:09 -07:00
Yann Collet
290aaa7521 Added : ability to manually select the dictionary ID of a newly created dictionary 2016-05-30 21:18:52 +02:00
Yann Collet
30009521d7 fuzzer tests dictBuilder. 
Added : ability to not store dictID during compression; decompression doesn't check dictID then
 2016-05-30 16:17:33 +02:00
Yann Collet
c0a9bf3c2e minor code refactoring 2016-05-30 04:48:32 +02:00
Yann Collet
c46fb924df added dictionary ID (incomplete) 2016-05-29 05:01:04 +02:00
Yann Collet
f51e0660f4 Simplified list of *.c files 2016-05-29 01:39:19 +02:00
Yann Collet
0c5e8b17ad moved xxhash to lib/common 2016-05-29 01:06:30 +02:00
inikep
957823f56f zstdcli: -r (operate recursively on directories) works with dictBuilder and compression 2016-05-25 15:30:55 +02:00
inikep
e75909ef93 ZBUFF_createCCtx_advanced moved to zbuff_static.h 2016-05-25 11:31:16 +02:00
inikep
02c244bf78 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-05-24 17:15:04 +02:00
inikep
fb5df613cf zstd_stats.h included only in debug mode 2016-05-24 15:36:37 +02:00
inikep
71dd67cf57 fullbench: tests for ZBUFF_createCCtx_advanced and ZBUFF_createDCtx_advanced 2016-05-24 10:57:14 +02:00
inikep
13ba880b49 added ZBUFF_createCCtx_advanced and ZBUFF_createDCtx_advanced 2016-05-23 17:04:23 +02:00
inikep
107e243195 added ZSTD_createDCtx_advanced 2016-05-23 16:24:52 +02:00
inikep
50e82c015d added ZSTD_createCCtx_advanced 2016-05-23 15:49:09 +02:00
inikep
35b891c5b0 zstd_stats.h updated to v0.6 2016-05-20 19:42:20 +02:00
Yann Collet
f22a0d653d huff0 dynamic reduction 2016-05-20 14:36:36 +02:00
Yann Collet
45f84ab7cc all programs use ZSTD_VERSION_STRING from zstd.h to display their version 2016-05-20 12:34:40 +02:00
Yann Collet
de4d4da52d changed version to 0.6.2 
removed make-controlled version (now used from zstd.h)
removed std=c99 by default
added : c90/gno90/c99/gnu99/c11 tests
 2016-05-20 12:20:12 +02:00
Yann Collet
7591a7f224 minor refactoring 2016-05-20 11:44:43 +02:00
Yann Collet
236d94fa9a reverted default compression level to 1 2016-05-18 12:06:33 +02:00
Yann Collet
78aa05228a fixed Visual warning in debug mode 2016-05-17 14:30:19 +02:00
Yann Collet
c8e9c4046f Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-05-17 12:02:35 +02:00
Yann Collet
ea4daa308d added ZSTDv05_getFrameParams() to "zstd_v05.h" 2016-05-17 12:01:55 +02:00
klemens
268775dae8 spelling-fixes, as of lintian.debian.org 2016-05-16 22:23:24 +02:00
Yann Collet
d9b3cca4d3 restore openVMS support (detected by @thatsafunnyname) 2016-05-13 12:43:36 +02:00
inikep
63ecd747de added common/entropy_common.c 2016-05-13 11:27:56 +02:00
Yann Collet
ccd6e86d45 improved decoding speed by inlining x_isError() functions 2016-05-12 15:55:26 +02:00
Yann Collet
16871680e3 minor refactor : more accurate variable scope 2016-05-12 14:27:00 +02:00
Yann Collet
b19d5d76ec restored inline for better decompression speed 2016-05-12 14:06:31 +02:00
Yann Collet
1ceb5a9295 restored DStream init 
due to strange speed regressions and warnins on gcc 5.3
 2016-05-12 13:50:13 +02:00
Yann Collet
ce27e85209 fixed clang conversion warning 2016-05-11 18:55:27 +02:00
Yann Collet
1032fbe714 update FSE library 2016-05-11 18:30:24 +02:00
Yann Collet
249ae0ca99 better big-endian / little-endian r/w support 2016-05-11 16:38:18 +02:00
Yann Collet
3329d98df9 fixed minor asan warning in legacy decoder 2016-05-11 11:07:54 +02:00
Yann Collet
c75e4c25e8 Added : ZSTD_getErrorString(), to get error string from error enum (#168) 2016-05-10 17:47:11 +02:00
Yann Collet
b3a9938d0c Merge pull request #182 from inikep/dev 
support for Visual Studio 2012, 2013, 2015
 2016-05-10 17:18:12 +02:00
Yann Collet
bd39d54576 zbuff supports byte-by-byte decompression scenarios 2016-05-10 14:14:19 +02:00
inikep
ed9a08538c Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 
# Conflicts:
#	lib/common/util.h
#	programs/paramgrill.c
#	visual/2013/fullbench/fullbench.vcxproj.filters
#	visual/2013/fuzzer/fuzzer.vcxproj.filters
 2016-05-10 13:20:01 +02:00
Yann Collet
918b934a26 minor comment update 2016-05-10 07:00:07 +02:00
inikep
349fcf7ea2 fixed Linux compilation 2016-05-09 18:04:07 +02:00
inikep
aaaf923a17 common/util.h moved to programs/util.h 
decompress/fse_decompress.c moved to common/
 2016-05-09 16:19:25 +02:00
inikep
d44ec48b1c detection of snprintf for Visual <= 2012 2016-05-09 14:37:41 +02:00
Yann Collet
59b6ba7677 Merge pull request #180 from thatsafunnyname/patch-1 
No stdint.h on OpenVMS.
 2016-05-09 11:56:42 +02:00
Yann Collet
4ff73a2eff added : midipix support (#181) 2016-05-09 11:44:00 +02:00
Yann Collet
d275236e6a Fixed Windows API, as detected by @KrzysFR (#171) 2016-05-09 08:19:27 +02:00
Yann Collet
f6ca09b5ff Reduced console display on loading lots of files with zstd --train. Reported by @KrzysFR, see #177 2016-05-09 04:44:45 +02:00
Yann Collet
e1a69a88c0 Expose Advanced Streaming API for v0.5.x legacy 2016-05-08 09:13:40 +02:00
Yann Collet
a5dacdf551 updated fse (minor refactor) 2016-05-06 16:55:27 +02:00
Yann Collet
18dedece91 Support legacy ZSTD_decompress_usingDict() (starting v0.5+) 2016-05-06 16:43:23 +02:00
Yann Collet
8283a2f0aa Fixed decoding corruption error with quad-symbols huffman on legacy decoders 2016-05-06 01:51:31 +02:00
inikep
13c8424ea0 code cleaning 2016-05-05 13:58:56 +02:00
inikep
0bd0faec32 fixed compatibility issues 2016-05-05 13:10:57 +02:00
Yann Collet
6d1d25299a fixed Huff0 quad-symbols decoder (#173) 2016-05-05 12:41:36 +02:00
inikep
9c22e57bfb Compiler Options moved to util.h 2016-05-05 11:53:42 +02:00
Yann Collet
ddb8ebd5b3 Stricter gnu90 tests 2016-05-05 04:59:53 +02:00
Yann Collet
3f8ed50bf1 Added gnu90 test 
work-around : disabled automatic selection of huff0 quad-decoder (see #173)
 2016-05-05 03:01:13 +02:00
inikep
3163403855 UTIL_prepareFileList 2016-05-05 00:25:38 +02:00
thatsafunnyname
b0e5427d1a No stdint.h on OpenVMS. 
No stdint.h on OpenVMS, it is inttypes.h instead, some projects use HAVE_STDINT_H.
 2016-05-04 15:43:42 +01:00
Yann Collet
6f9c056662 Fixed BMI behavior on x86 and x32 targets, reported by @tycho (#178) 2016-05-01 10:26:30 +02:00
inikep
55d047aa92 getTotalFileSize moved to common/util.h 2016-04-28 16:50:13 +02:00
inikep
d5ff2c3d9a ordering of #include 2016-04-28 14:40:45 +02:00
inikep
83c76b4594 bench.c: time functions moved to common/util.h 2016-04-28 13:16:01 +02:00
inikep
69fcd7c0ae getFileSize moved to common/util.h 2016-04-28 12:23:33 +02:00
inikep
1007a1fe3d get rid of some Visual C++ warnings 2016-04-25 15:23:09 +02:00
inikep
6cb083fe3d fixed 64-bit compilation with Visual C++ 2016-04-25 14:42:15 +02:00
inikep
ea4ee3eee0 added compatibility with Visual C++ 2012 2016-04-25 13:09:06 +02:00
inikep
a8138fd767 updated lib/README.md part 2 2016-04-25 11:36:44 +02:00
inikep
49794316c2 updated lib/README.md 2016-04-25 11:31:28 +02:00
inikep
f066816998 fix for g++ compilation 2016-04-22 18:54:05 +02:00
inikep
db3964382c introduced ZSTD_NODECOMPRESS to link only compressor 2016-04-22 18:22:30 +02:00
inikep
a1febea01d Error functions moved to common/zstd_common.c 2016-04-22 17:14:25 +02:00
inikep
3c7c3527d0 introduced ZSTD_NOCOMPRESS to generate decompressor only 2016-04-22 13:59:05 +02:00
inikep
23a0889301 separation of lib/ into common/, compress/, decompress/, dictBuilder/, legacy/ 2016-04-22 12:43:18 +02:00
inikep
4699b3be03 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-04-21 12:30:58 +02:00
inikep
38654988f3 minor speed improvements 2 
bench.c: block size has to be bigger than 32 bytes
zstdcli.c: support for e.g. -B16k -B16m
 2016-04-21 12:18:47 +02:00
inikep
ef51941822 zst_opt.h: minor compression speed improvement 2016-04-21 11:08:43 +02:00
Yann Collet
e02808ffc5 bumped lib version to 0.6.1 
added ZSTD_VERSION_STRING
 2016-04-20 22:46:16 +02:00
inikep
c5e1d295ee bench.c: force at least one compression and decompression loop 
fix for -i0 with small files
 2016-04-19 09:37:59 +02:00
inikep
2c5eeea635 introduced ZSTD_DEAFULT_CLEVEL for (compressionLevel<=0) 2016-04-15 13:44:46 +02:00
inikep
4b3c5eed99 removed ZSTD_compressBegin_targetSrcSize 2016-04-14 13:43:51 +02:00
inikep
c70c90e8a3 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-04-13 10:48:26 +02:00
Yann Collet
16748729c9 Updated DICT_MAGIC, since internal dictionary format has changed (length encoding) 2016-04-12 23:26:23 +02:00
Yann Collet
c5b6d9741a Added : zbuff.c to library; reported by Yipei Zhang 2016-04-12 21:46:18 +02:00
Yann Collet
389648cc6d Fixed : contentSize + copyCCtx() 2016-04-12 19:13:08 +02:00
Yann Collet
541dc7c7d4 Added : fuzzer test, checking contentLength value after copyCCtx() usage 2016-04-12 18:00:20 +02:00
Yann Collet
6236eba991 removed headerBuffer 2016-04-12 15:52:33 +02:00
Yann Collet
0085cd3138 created ZSTD_writeFrameHeader() 2016-04-12 14:14:10 +02:00
Yann Collet
abb5c653f1 minor speed improvement for ZSTD_copyCCtx() 2016-04-11 20:42:31 +02:00
Yann Collet
887e7da785 Fixed copyCCtx() on uninitialized cctx (reported by Oliver Lange) 2016-04-11 20:12:27 +02:00
inikep
8844e21dff Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-04-11 16:49:27 +02:00
inikep
f7d210b2e9 cache literal prices for ZSTD_btopt 2016-04-11 16:35:13 +02:00
Yann Collet
582933fde5 fixed round-trip corruption, reported by Oliver Lange 2016-04-11 16:25:56 +02:00
Yann Collet
125d81774f fixed uninitialization error, reported by Maciej Adamczyk 2016-04-11 14:22:56 +02:00
Yann Collet
d673d4cf66 fixed uninitialization error, reported by Maciej Adamczyk 2016-04-11 14:10:23 +02:00
Yann Collet
ccbba296e1 Fix infinite loop with invalid data (fuzzer test) 2016-04-10 14:42:26 +02:00
Yann Collet
793c649ead updated cLevels 2016-04-09 20:32:00 +02:00
Yann Collet
b144bc3e1c Fixed uninitialized warning, reported by Maciej Adamczyk 2016-04-09 17:26:22 +02:00
Yann Collet
9990986bdc fixed minor warnings when compiling with gcc -g (aka -std=gnu90) 2016-04-09 16:17:18 +02:00
Yann Collet
029267ab3f Integrated Legacy v0.5.x decoder (provided by @inikep) 2016-04-09 09:42:27 +02:00
Yann Collet
7eff39f7ea fixed decoding error 2016-04-09 01:51:36 +02:00
Yann Collet
ea63bb7b5e converted fuzzer timer to clock_t 2016-04-08 15:25:32 +02:00
Yann Collet
78267d1abe updated cLevel for block <= 256K 2016-04-08 12:36:19 +02:00
Yann Collet
72cc704514 Merge branch 'dev060' of github.com:Cyan4973/zstd into dev060 2016-04-08 02:02:31 +02:00
Yann Collet
0dbf2874ee faster level 1 at 256 KB 2016-04-08 02:02:12 +02:00
Yann Collet
04a4fd638b Merge pull request #164 from inikep/dev 
only repcode 0 for lazy, lazy2 and btlazy2 strategies
 2016-04-07 20:46:13 +02:00
inikep
afe1f7994b Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into dev 
# Conflicts:
#	lib/zstd_compress.c
 2016-04-07 19:50:03 +02:00
Yann Collet
9e8b09a7bd Fixed memory initialization issue, reported by Maciej Adamczyk 2016-04-07 19:35:23 +02:00
inikep
2bf79f9bff use extDict only when needed 2016-04-07 19:24:33 +02:00
inikep
64d7bcb0dc only repcode 0 for lazy, lazy2 and btlazy2 strategies 2016-04-07 19:14:09 +02:00
inikep
a69640cca1 Revert "removed void ZSTD_compressBlock_greedy" 
This reverts commit 9296400a69.
 2016-04-07 18:22:29 +02:00
Yann Collet
d7bee33caf Fixed make install (reported by Oliver Lange) 2016-04-07 18:16:42 +02:00
Yann Collet
5d39357bc4 minor compression speed optimization 2016-04-07 17:19:00 +02:00
inikep
9296400a69 removed void ZSTD_compressBlock_greedy 2016-04-07 16:58:04 +02:00
Yann Collet
485371600a Implemented default coding table for lengths and offset 2016-04-07 15:24:29 +02:00
Yann Collet
79459da14b Merge pull request #163 from inikep/dev 
3 repcodes and MINMATCH=3
 2016-04-07 14:37:30 +02:00
inikep
81a345b0ce minor speed improvements 2016-04-07 11:35:17 +02:00
inikep
e47f33f980 reordered code 2016-04-06 20:58:00 +02:00
inikep
2db1eb78e9 reordered functions 2016-04-06 17:14:19 +02:00
inikep
e5b6e9e551 fixed support for searchLength=4 2016-04-06 13:15:38 +02:00
inikep
75716851d4 MINMATCH=3 for lazy strategy 2016-04-06 12:34:42 +02:00
inikep
7bc19b6b3e MINMATCH=3 2016-04-06 09:46:01 +02:00
inikep
f8a339d274 working ZSTD_compressBlock_opt_extDict_generic 2016-04-05 23:58:51 +02:00
inikep
83417bbbd2 code formatting 2016-04-05 21:30:03 +02:00
inikep
5ce00aee37 ZSTD_compressBlock_opt_extDict_generic in progress 2016-04-05 21:03:43 +02:00
inikep
faa8d8aae1 introduced ZSTD_compressBlock_greedy 2016-04-05 19:01:10 +02:00
inikep
908fcb384a 3 repcodes with ZSTD_compressBlock_lazy_extDict_generic 2016-04-05 18:16:38 +02:00
inikep
a4385aa23b ZSTD_REP_NUM=3 only 2016-04-05 17:11:56 +02:00
inikep
d5359b2e55 Merge remote-tracking branch 'refs/remotes/origin/repcodes' into dev 2016-04-05 17:07:36 +02:00
inikep
ba73d1bb6b support for ZSTD_REP_NUM=3 2016-04-05 10:30:05 +02:00
inikep
74e3a7ba4c removed kSlotNew 2016-04-05 10:18:37 +02:00
inikep
b3532793a9 experiments with ZSTD_decodeSequence 2016-04-05 10:08:25 +02:00
Yann Collet
46bdb6267e Merge pull request #161 from inikep/dev 
Dev
 2016-04-05 09:59:59 +02:00
inikep
2ec2698ef9 working 4 reps 2016-04-05 09:34:13 +02:00
inikep
19140cd80e zstd_stats.h moved to .debug/ 2016-04-05 08:52:57 +02:00
inikep
d6f208badf improved logging 2016-04-04 21:15:23 +02:00
inikep
78e52044de Merge remote-tracking branch 'refs/remotes/origin/dev' into repcodes 
# Conflicts:
#	lib/zstd_compress.c
#	lib/zstd_decompress.c
#	lib/zstd_internal.h
#	lib/zstd_opt.h
#	programs/bench.c
 2016-04-04 16:28:40 +02:00
inikep
cb70c8ab90 fixed -Werror,-Wconversion warning 2016-04-04 15:43:45 +02:00
inikep
0c7456c5b7 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into dev 
# Conflicts:
#	lib/zstd_compress.c
#	lib/zstd_static.h
 2016-04-04 14:54:53 +02:00
Yann Collet
b59bf963a8 fixed Visual conversion warnings 2016-04-04 14:53:16 +02:00
inikep
227dd4bc74 changed price estimation for optimal parser 2016-04-04 14:34:02 +02:00
Yann Collet
8a57b928ba changed contentLog -> chainLog 2016-04-04 13:49:18 +02:00
Yann Collet
1005fc1276 reduced memory usage of very hich compression levels in 32-bits mode 2016-04-04 13:28:28 +02:00
inikep
19bd48f516 Merge from Cyan4973/dev060 2016-04-04 12:10:00 +02:00
Yann Collet
1535414927 Fixed 32-bits windowLog limitation 
Fixed some macro errors in zbufftest
 2016-04-04 04:22:53 +02:00
Yann Collet
5a88a70f97 fixed buffer sync error in zbuff 2016-04-03 20:24:25 +02:00
Yann Collet
ef36390e4a refined exception for hashlog < min 2016-04-02 00:46:40 +02:00
Yann Collet
40358d01aa Fix : rare bugs when hashLog < min in bt mode 2016-04-02 00:40:09 +02:00
Yann Collet
1d803d8deb fixed zbuff error 775-3 (internal buffer management) 2016-04-01 18:57:41 +02:00
Yann Collet
efb18309ef fixed minor visual 32-64bits conversion warning 2016-04-01 18:54:13 +02:00
Yann Collet
afa60e0f14 fixed decoding error, reported by Maciej Adamczyk 2016-04-01 15:49:41 +02:00
Yann Collet
27caf2afca minor variable isolation 2016-04-01 15:48:48 +02:00
Yann Collet
9bb87e5995 fixed harmless comparison warning 2016-03-30 21:28:15 +02:00
Yann Collet
51d500425f fixed paramgrill 2016-03-30 20:42:19 +02:00
Yann Collet
dd6466a211 fixed clang conversion warning 2016-03-30 20:06:26 +02:00
Yann Collet
3b71925c2d separate params into compressionParams and frameParams 2016-03-30 19:48:05 +02:00
Yann Collet
21588e370b changed validateParams() into checkParams() + adjustParams() 2016-03-30 16:50:44 +02:00
Yann Collet
b44be74244 Fixed zdict 
more dictionary compression tests
 2016-03-26 20:52:14 +01:00
Yann Collet
74bd11954b fixed ARM compatibility 2016-03-26 17:50:26 +01:00
Yann Collet
b9151406de fixed 32-bits compatibility 2016-03-26 17:18:11 +01:00
inikep
97c88e716c fix for -Wshorten-64-to-32 warning in ZSTD_statsUpdatePrices 2016-03-25 13:39:14 +01:00
Yann Collet
433a5cce7e improved decompression speed (680) 2016-03-25 11:43:48 +01:00
inikep
5cc4efdaf8 created zstd_stats.h 2016-03-25 10:52:25 +01:00
Yann Collet
646693e3be support for alternate offset (fusion) 2016-03-24 02:42:15 +01:00
Yann Collet
b21ce15efe minor variables isolation 2016-03-24 02:02:32 +01:00
Yann Collet
7cbe79ab25 fixed write overflow found by fuzzer 2016-03-23 22:31:57 +01:00
Yann Collet
72d706a020 fixed crash at -O3 with customized block size (wrong alignment) 2016-03-23 20:44:12 +01:00
inikep
eaba91a161 added ZSTD_compressBegin_targetSrcSize 2016-03-23 20:30:26 +01:00
Yann Collet
862a85976f Added BMI instructions 2016-03-23 18:45:23 +01:00
inikep
7adceef974 hashLog3 added to ZSTD_CCtx 2016-03-23 15:53:38 +01:00
Yann Collet
6cf45dac81 shared const BIT_mask 2016-03-23 14:41:54 +01:00
Yann Collet
3c017867de minor optimization (DSpeed 665) 2016-03-23 14:09:51 +01:00
Yann Collet
2512597576 minor reordering (DSpeed 665) 2016-03-23 14:00:09 +01:00
Yann Collet
afab02098a improved decoding speed (660) 2016-03-23 13:57:49 +01:00
inikep
472638c861 added mili_sleep and setHighPriority 2016-03-23 12:28:28 +01:00
Yann Collet
9c34df93b6 new decodeSequence, merging parts 2016-03-23 01:54:25 +01:00
Yann Collet
add08d6f61 minor variation - DSpeed at 640 2016-03-23 01:32:41 +01:00
Yann Collet
be391438ff first working version with both encoder and decode alternate LL + ML coding scheme. 
decompression speed highly impacted
 2016-03-22 23:19:28 +01:00
inikep
2872b6f01f Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into dev 
# Conflicts:
#	lib/zstd_compress.c
#	programs/bench.c
 2016-03-22 14:38:34 +01:00
Yann Collet
fadda6c875 first prototype with ML encoding scheme (but not yet decoding scheme) 2016-03-22 12:14:26 +01:00
inikep
2b942881ac improved 4reps in ZSTD_compressBlock_lazy_generic 
kSlotNew = 0
 2016-03-22 11:56:22 +01:00
Yann Collet
1dcf930f72 Merge branch 'dev060' into alternateLLcode 2016-03-22 00:29:57 +01:00
inikep
f2fa0e1198 final repcodes for ZSTD_btopt 2016-03-21 20:14:03 +01:00
inikep
378be62a9e better repcodes 2016-03-21 19:48:54 +01:00
inikep
fc1d7bd93b changed update logic of repcodes 2016-03-21 19:02:16 +01:00
inikep
e7ec8f2b9b improved ratio 2016-03-21 17:23:02 +01:00
Yann Collet
be4605d851 improved decompression speed 2016-03-21 14:29:26 +01:00
Yann Collet
e91477c171 minor variable isolation 2016-03-21 14:22:38 +01:00
Yann Collet
b0aec17a90 code simplification (but reduce decompression speed ...) 2016-03-21 13:24:16 +01:00
Yann Collet
6c62b7dfc8 support default LL distribution 2016-03-21 00:39:19 +01:00
Yann Collet
d64f435f63 handles litLength >= 65535 2016-03-21 00:07:42 +01:00
Yann Collet
597847a2ae first emulation 2016-03-20 19:14:22 +01:00
Yann Collet
ecabfe3777 Improved index overflow protection 2016-03-20 16:20:06 +01:00
Yann Collet
5244738046 minor code refactoring 2016-03-20 16:00:00 +01:00
Yann Collet
de406eebcd minor code refactor 2016-03-20 15:46:10 +01:00
Yann Collet
516ba88022 updated huff0 2016-03-20 05:40:39 +01:00
Yann Collet
a5c2c08c68 code refactoring 2016-03-20 01:09:18 +01:00
Yann Collet
70e4577179 minor code refactoring 2016-03-19 18:08:32 +01:00
Yann Collet
c6eea2b2f2 Improved speed for small files 2016-03-19 17:18:00 +01:00
Yann Collet
e74215e3c7 performance fix for small files 2016-03-19 16:09:09 +01:00
inikep
1269e2f7db working kSlotNew for ZSTD_btopt 2016-03-19 15:32:21 +01:00
Yann Collet
bb604486f1 fixed clang warning 2016-03-19 15:18:42 +01:00
Yann Collet
37f3d1b2ab minor refactor 2016-03-19 15:11:42 +01:00
Yann Collet
90a03a95a1 minor performance fix (decompression) 2016-03-19 14:47:31 +01:00
Yann Collet
01e5b95199 code refactor 2016-03-19 14:14:31 +01:00
Yann Collet
d1d210f3fb minor code refactor 2016-03-19 12:12:07 +01:00
Yann Collet
4db09efde4 simplifications 2016-03-18 22:23:49 +01:00
Yann Collet
39c596ca58 minor simplifications 2016-03-18 21:40:56 +01:00
inikep
fade9b7ae3 ZSTD_btopt with 4 repcodes 2016-03-18 16:10:35 +01:00
inikep
1c1e6916e4 improved timer accuracy 2016-03-18 12:49:44 +01:00
Yann Collet
a910dc8238 minor code refactoring 2016-03-18 12:37:45 +01:00
inikep
5b15865413 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into repcodes 
# Conflicts:
#	lib/zstd_decompress.c
#	programs/bench.c
 2016-03-18 11:14:58 +01:00
inikep
eca2ec0b3b introduced ZSTD_REP_MOVE and ZSTD_REP_INIT 2016-03-18 11:03:43 +01:00
inikep
c56e32e782 4 repcodes when litLen==0 2016-03-17 19:53:38 +01:00
inikep
3e743052cd kSlotNew 2016-03-17 17:24:17 +01:00
Yann Collet
3fd164e656 minor refactoring 2016-03-17 13:43:32 +01:00
Yann Collet
b40287aaef Improved decompression speed (by @inikep) 2016-03-17 12:11:16 +01:00
inikep
7d1742825b support for ZSTD_REP_NUM == 1 and ZSTD_REP_NUM == 4 2016-03-17 11:21:00 +01:00
inikep
5945308983 working 4 reps 2016-03-16 15:35:14 +01:00
Yann Collet
569b81adb1 changed BLOCKSIZE into ZSTD_BLOCKSIZE_MAX 2016-03-16 15:26:51 +01:00
inikep
62d694795d ZSTD_compressBlock_lazy supports more reps 2016-03-16 12:57:07 +01:00
inikep
0ac4a551b9 ZSTD_REP_NUM removed from ZSTD_storeSeq 2016-03-16 12:03:07 +01:00
inikep
df164e6d35 introduced U32 rep[ZSTD_REP_NUM] 2016-03-16 11:56:59 +01:00
inikep
4569667c85 ZSTD_REP_NUM 2016-03-16 11:40:09 +01:00
Yann Collet
6cb41356e8 update doc / comments 2016-03-15 15:47:38 +01:00
Yann Collet
346bffbc68 Added frame content size test within fuzzer 2016-03-15 15:24:52 +01:00
Yann Collet
09b21ee2f6 fix fileio bug with new zbuff 
simplified zbuff decoding
 2016-03-15 12:56:03 +01:00
inikep
d700a1a8b9 ZSTD_setAdditionalParam 2016-03-15 12:18:44 +01:00
Yann Collet
1c2c2bcaff minor refactoring 2016-03-15 01:33:36 +01:00
Yann Collet
d1b26849e5 Frame content size supported 2016-03-15 01:24:33 +01:00
inikep
5fdd0b46d3 added additionalParam 2016-03-14 19:51:11 +01:00
Yann Collet
fb7973580f minor comments update 2016-03-13 11:08:40 +01:00
Yann Collet
03ea59b17b Decompression can decode frame content size 2016-03-12 01:25:40 +01:00
Yann Collet
0e491c01fe created ZSTD_frameParams 2016-03-11 21:58:04 +01:00
inikep
cc6c283633 removed hashLog3 from ZSTD_parameters 2016-03-10 15:17:24 +01:00
inikep
260a9e3886 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into dev 2016-03-10 15:04:12 +01:00
inikep
b0c640d934 small ratio improvement with MINMATCH=3 2016-03-10 13:18:36 +01:00
Yann Collet
da5be72fda update compression level parameters 2016-03-10 11:01:44 +01:00
Yann Collet
dcc000f0eb minor doc clarifications 2016-03-10 01:09:41 +01:00
Yann Collet
e3193c4081 update compresson levels 2016-03-09 16:57:09 +01:00