AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
4,766 Commits 1 Branch 0 Tags 27 MiB
5ec97fff17
Commit Graph

361 Commits

Author SHA1 Message Date
Nick Terrell
d051cd5b43 Use workspace for count and CTable 2017-03-02 16:38:07 -08:00
Sean Purcell
3d95925a59 Merge remote-tracking branch 'origin/dev' into m32 2017-03-02 15:17:56 -08:00
Nick Terrell
a419777eb1 Allow compressor to repeat Huffman tables 
* Compressor saves most recently used Huffman table and reuses it
  if it produces better results.
* I attempted to preserve CPU usage profile.
  I intentionally left all of the existing heuristics in place.
  There is only a speed difference on the second block and later.
  When compressing large enough blocks (say >= 4 KiB) there is
  no significant difference in compression speed.
  Dictionary compression of one block is the same speed for blocks
  with literals <= 1 KiB, and after that the difference is not
  very significant.
* In the synthetic data, with blocks 10 KB or smaller, most blocks
  can't use repeated tables because the previous block did not
  contain a symbol that the current block contains.
  Once blocks are about 12 KB or more, most previous blocks have
  valid Huffman tables for the current block, and the compression
  ratio and decompression speed jumped.
* In silesia blocks as small as 4KB can frequently reuse the
  previous Huffman table (85%), but it isn't as profitable, and
  the previous Huffman table only gets used about 3% of the time.
* Microbenchmarks show that `HUF_validateCTable()` takes ~55 ns
  and `HUF_estimateCompressedSize()` takes ~35 ns.
  They are decently well optimized, the first versions took 90 ns
  and 120 ns respectively. `HUF_validateCTable()` could be twice as
  fast, if we cast the `HUF_CElt*` to a `U32*` and compare to 0.
  However, `U32` has an alignment of 4 instead of 2, so I think that
  might be undefined behavior.
* I've ran `zstreamtest` compiled normally, with UASAN and with MSAN
  for 4 hours each.

The worst case for the speed difference is a bunch of small blocks
in the same frame. I modified `bench.c` to compress the input in a
single frame but with blocks of the given block size, set by `-B`.
Benchmarks on level 1:

|  Program  | Block size |   Corpus  | Ratio | Compression MB/s | Decompression MB/s |
|-----------|------------|-----------|-------|------------------|--------------------|
| zstd.base |        256 | synthetic | 2.364 |            110.0 |              297.0 |
|      zstd |        256 | synthetic | 2.367 |            108.9 |              297.0 |
| zstd.base |        256 | silesia   | 2.204 |             93.8 |              415.7 |
|      zstd |        256 | silesia   | 2.204 |             93.4 |              415.7 |
| zstd.base |        512 | synthetic | 2.594 |            144.2 |              420.0 |
|      zstd |        512 | synthetic | 2.599 |            141.5 |              425.7 |
| zstd.base |        512 | silesia   | 2.358 |            118.4 |              432.6 |
|      zstd |        512 | silesia   | 2.358 |            119.8 |              432.6 |
| zstd.base |       1024 | synthetic | 2.790 |            192.3 |              594.1 |
|      zstd |       1024 | synthetic | 2.794 |            192.3 |              600.0 |
| zstd.base |       1024 | silesia   | 2.524 |            148.2 |              464.2 |
|      zstd |       1024 | silesia   | 2.525 |            148.2 |              467.6 |
| zstd.base |       4096 | synthetic | 3.023 |            300.0 |             1000.0 |
|      zstd |       4096 | synthetic | 3.024 |            300.0 |             1010.1 |
| zstd.base |       4096 | silesia   | 2.779 |            223.1 |              623.5 |
|      zstd |       4096 | silesia   | 2.779 |            223.1 |              636.0 |
| zstd.base |      16384 | synthetic | 3.131 |            350.0 |             1150.1 |
|      zstd |      16384 | synthetic | 3.152 |            350.0 |             1630.3 |
| zstd.base |      16384 | silesia   | 2.871 |            296.5 |              883.3 |
|      zstd |      16384 | silesia   | 2.872 |            294.4 |              898.3 |
 2017-03-02 13:27:52 -08:00
Sean Purcell
d44703d145 Offsets >= 32MB in 32-bits mode 2017-03-01 16:27:56 -08:00
Yann Collet
76f0494089 xxhash can be included twice in any order 
Previously,

followed by :

would fail to include the static definitions,
because the second include was simply skipped by guard macro.

Now it works as intended :
the missing static part is included during the second include.
 2017-03-01 13:29:29 -08:00
Yann Collet
4bcc69b761 solves warnings when compiling with global XXH_STATIC_LINKING_ONLY 
XXH_STATIC_LINKING_ONLY protection macro is intended to be triggered just before the include.
The main idea is to keep this setting local :
user module shall explicitly understand and accept the static linking restriction
which becomes transparent when triggering the macro at project level.
Global definition also triggers redefinition warnings for user modules which do locally define the macro.

This new version compiles lib and cli without warning when the macro is set globally.
That's not a scenario to be recommended, since it trades a local effect for a global one,
but it was easy enough to provide from zstd side.
 2017-03-01 11:33:25 -08:00
Yann Collet
0b9b894b2d reduced ZSTD_DDict memory usage 
saved 128 KB
 2017-02-27 00:27:30 -08:00
Anders Oleson
517577bf53 spelling fixes in comments 
i.e. occurred labeled Huffman
 2017-02-20 12:08:59 -08:00
Yann Collet
2252d29a5a Merge branch 'dev' of github.com:facebook/zstd into dev 2017-02-15 12:00:50 -08:00
Yann Collet
4596037042 updated fse version 
feature minor refactoring (removing FSE_abs())
also : fix a few minor issues recently introduced in examples
 2017-02-15 12:00:03 -08:00
Yann Collet
f0b9a8dddb Merge pull request #547 from inikep/dev11 
Avoid fseek()'s 2GiB barrier with MacOS and *BSD
 2017-02-14 12:29:00 -08:00
ds77
08e6a88a97 avoid empty translation unit warning without #pragma 2017-02-14 00:46:47 +01:00
Przemyslaw Skibinski
09c8e5390d __builtin_bswap requires gcc 4.3+ 2017-02-13 12:45:53 +01:00
Sean Purcell
e0b3265e87 Fix ZSTD_getErrorString and add tests 2017-02-08 17:28:49 -08:00
Yann Collet
cc3d1bc262 Merge pull request #525 from terrelln/covermt 
Multithreaded COVER dictionary training
 2017-01-30 10:15:33 -08:00
Nick Terrell
b42dd27ef5 Add include guards and extern C 2017-01-27 16:00:19 -08:00
Nick Terrell
e628eaf87a Fix pool.c threading.h import 2017-01-26 15:29:10 -08:00
cyan4973
2e3b659ae1 fixed minor warnings (Visual, conversion, doxygen) 2017-01-20 14:43:09 -08:00
cyan4973
5fba09fa41 updated util's time for Windows compatibility 
Correctly measures time on Posix systems when running with
Multi-threading

Todo : check Windows measurement under multi-threading
 2017-01-20 12:57:31 -08:00
Yann Collet
0f984d94c4 changed MT enabling macro to ZSTD_MULTITHREAD 2017-01-19 14:05:07 -08:00
Yann Collet
32dfae6f98 fixed Multi-threaded compression 
MT compression generates a single frame.
Multi-threading operates by breaking the frames into independent sections.
But from a decoder perspective, there is no difference :
it's just a suite of blocks.

Problem is, decoder preserves repCodes from previous block to start decoding next block.
This is also valid between sections, since they are no different than changing block.

Previous version would incorrectly initialize repcodes to their default value at the beginning of each section.
When using them, there was a mismatch between encoder (default values) and decoder (values from previous block).

This change ensures that repcodes won't be used at the beginning of a new section.
It works by setting them to 0.
This only works with regular (single segment) variants : extDict variants will fail !
Fortunately, sections beyond the 1st one belong to this category.

To be checked : btopt strategy.
This change was only validated from fast to btlazy2 strategies.
 2017-01-19 10:32:55 -08:00
Yann Collet
f1cb55192c fixed linux warnings 2017-01-02 01:11:55 +01:00
Nick Terrell
bb13387d7d Fix pool for threading.h 2016-12-31 19:10:47 -05:00
Nick Terrell
4204e03e77 Add threading.h condition variables 2016-12-31 19:10:29 -05:00
Yann Collet
3b9d434356 extended ZSTDMT code support for non-MT systems and WIN32 (preliminary) 2016-12-31 16:32:19 +01:00
Yann Collet
3b29dbd9e8 new zstdmt version using generic treadpool 2016-12-31 06:04:25 +01:00
Yann Collet
c6a6417458 bench correctly measures time for multi-threaded compression (posix only) 2016-12-31 03:31:26 +01:00
Nick Terrell
e777a5be6b Add a thread pool for ZSTDMT and COVER 2016-12-29 23:39:44 -08:00
Yann Collet
0819abe3c1 added ZSTD_createDDict_byReference() body 2016-12-21 19:25:15 +01:00
Nick Terrell
8de46ab51a Export all API functions 2016-12-16 13:27:30 -08:00
Yann Collet
5397a66b19 minor BMI version check 2016-12-13 15:21:06 +01:00
Nick Terrell
064a143520 Fix execSequence wildcopy undefined behavior 
execSequence relied on pointer overflow to handle cases where
`sequence.matchLength < 8`.  Instead of passing an `size_t` to
wildcopy, pass a `ptrdiff_t`.
 2016-12-12 19:01:23 -08:00
Yann Collet
825dffbc43 moved zbuff source files into lib/deprecated 2016-12-05 19:28:19 -08:00
Przemyslaw Skibinski
821bf1febc fixed Doxygen trailing comment 2016-12-02 16:13:41 +01:00
Yann Collet
b89af20353 reduced table sizes for HUF_readDTableX4 2016-12-01 18:24:59 -08:00
Yann Collet
a0d742b1e4 introduced HUF_buildCTable_wksp(), to reduce stack memory usage 2016-12-01 17:47:30 -08:00
Yann Collet
e928f7e16d introduced ext_wksp variants of count to reduce stack memory usage 2016-12-01 16:13:35 -08:00
Yann Collet
5e00b848a8 FSE_compress_wksp() uses less stack space 2016-11-30 16:46:13 -08:00
Yann Collet
d79a9a00d9 Introduced FSE_compress_wksp() and FSE_buildCTable_wksp() to reduce stack memory usage 2016-11-30 15:52:20 -08:00
Yann Collet
766431909f introduced FSE_decompress_wksp(), to use less stack space 2016-11-30 12:36:45 -08:00
Yann Collet
167c494748 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-11-29 14:05:15 -08:00
Yann Collet
52e136ed3d long decoder compatible with round and separate buffers 2016-11-28 19:59:11 -08:00
Przemyslaw Skibinski
9ca65af810 zstd_opt.h: improved price function 2016-11-23 17:22:54 +01:00
Yann Collet
0d761dbe95 Merge pull request #453 from inikep/dev11 
fullbench-dll
 2016-11-16 15:45:30 -08:00
Yann Collet
52afb3993e zbuff API now generates deprecation warnings 2016-11-16 08:50:54 -08:00
Przemyslaw Skibinski
179555c1d1 working fullbench-dll 2016-11-15 18:05:46 +01:00
Yann Collet
2115724c22 Merge pull request #430 from terrelln/exec-sequences 
ZSTD_execSequence() accepts match in last 7 bytes
 2016-10-28 10:45:05 -07:00
Nick Terrell
eb7873a048 ZSTD_execSequence() accepts match in last 7 bytes 
The zstd reference compressor will not emit a match in the last 7
bytes of a block.  The decompressor will also not accept a match
in the last 7 bytes.  This patch makes the decompressor accept a
match in the last 7 bytes.
 2016-10-25 21:24:15 -07:00
Nick Terrell
d760529a05 Fix stack buffer overrun when weightTotal == 0 
If `weightTotal == 0`, then `BIT_highbit32(weightTotal)` is
undefined behavior in the case that it calls `__builtin_clz()`.
If `tableLog == HUF_TABLELOG_ABSOLUTEMAX` then we will access one
byte beyond the end of the buffer.
 2016-10-19 11:39:11 -07:00
Nick Terrell
ccfcc643da Check if dict is empty before reading first byte 2016-10-17 11:46:03 -07:00
Yann Collet
5d919e7ac3 added ZSTD_error_frameParameter_windowTooLarge (#403) 2016-10-12 17:29:24 -07:00
Yann Collet
ef2357d0d3 created error_private.c, so that a single list of error strings get included 2016-10-11 17:24:50 -07:00
Yann Collet
a17fd7312a changed error_public.h into zstd_errors.h 2016-10-11 16:41:09 -07:00
Yann Collet
18b51b99c0 sync fse 2016-10-11 08:21:09 -07:00
Yann Collet
51f4d566c2 small decompression speed boost for very small data 2016-09-22 15:57:28 +02:00
Yann Collet
95d07d7447 introduced CHECK_E 2016-09-06 16:38:51 +02:00
Yann Collet
3e21ec5b01 introduced CHECK_F 2016-09-06 15:36:19 +02:00
Yann Collet
5c956d593c FORCE_INLINE common definition 2016-09-06 15:05:19 +02:00
Yann Collet
1563bfeabc fixing FORCE_INLINE for older compilers (#330) 2016-09-02 11:44:21 -07:00
David Lam
e10f7f3dcb merge 2016-08-30 12:03:36 -07:00
Yann Collet
4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
David Lam
da9d3b7057 Cleanup some errors in typedef comments and remove duplicated HOWTO from zbuff_decompress.c 2016-08-29 17:31:51 -07:00
Yann Collet
23b6e05d8e ZSTD_malloc() and ZSTD_free(), to simplify customMem 2016-08-28 21:05:43 -07:00
Yann Collet
4bf317dd00 first version supporting legacy streams (transparent decoding) 2016-08-28 07:43:34 -07:00
Yann Collet
87c18b2ebd fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
inikep
a3a47ec4d0 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into Other 2016-08-24 21:25:49 +02:00
inikep
e416e30019 remove unnecessary comments 2016-08-24 17:32:09 +02:00
inikep
4e90f6c1e0 removed ZSTD_LOG_ENCODE and ZSTD_LOG_BLOCK 2016-08-24 17:24:11 +02:00
inikep
83388e109f removed ZSTD_LOG_PARSER 2016-08-24 17:22:20 +02:00
inikep
8a36f8527c removed stats in debug mode 2016-08-24 17:19:12 +02:00
Yann Collet
17e482efdd added ZSTD_setDStreamParameter() 2016-08-23 16:58:10 +02:00
Yann Collet
0cfe2ec2fd sync fse version 2016-08-20 00:26:26 +02:00
inikep
5f49eba512 added usage of rep[0]-1 for the optimal parser 2016-08-10 15:01:53 +02:00
inikep
48849f86f0 fixed compilation with Intel Compiler with Windows 2016-08-10 14:26:35 +02:00
Yann Collet
1ea5622a32 updated xxhash 2016-08-10 09:40:08 +02:00
Yann Collet
666398e7ed added : xxhash namespace enforced from xxhash.h. 
added : xxhash namespace test.
removed : -DXXH_NAMESPACE
 2016-08-10 08:16:51 +02:00
Yann Collet
8ded0b84aa update xxhash to v0.6.2 2016-08-10 07:40:40 +02:00
Yann Collet
280f9a8754 minor comment 2016-08-08 00:44:00 +02:00
Yann Collet
8cebfd1d26 fix attempt on test-zstd-speed 2016-07-31 01:59:23 +02:00
Yann Collet
c0ce4f1211 slightly improved compression speed 2016-07-30 00:55:13 +02:00
Yann Collet
ed57d8530a new seqStore 2016-07-29 21:22:17 +02:00
Yann Collet
c00d30fbe4 Merge pull request #264 from inikep/dev08 
Dev08
 2016-07-29 17:42:30 +02:00
Yann Collet
6a82f0f8bf minor comments 2016-07-29 00:55:45 +02:00
Yann Collet
ffa7d0ac1e clarified comment 2016-07-28 21:01:17 +02:00
Yann Collet
60ba31c570 zbuff uses ZSTD_compressEnd() 2016-07-28 19:55:09 +02:00
Yann Collet
e7bf9156d1 Clarified API comments, from suggestions by ‎Bryan O'Sullivan‎ 2016-07-28 05:00:57 +02:00
Yann Collet
c991cc1828 new frame end, 32-bits checksums 2016-07-28 00:55:43 +02:00
Yann Collet
c154d9d6a2 better support for large dictionaries (> 128 KB) 2016-07-27 14:37:00 +02:00
inikep
003c7a8568 optimal parser: removed ZSTD_REP_INIT 2016-07-27 11:07:13 +02:00
Yann Collet
d50f9db3ea Improved speed on clang and gcc -O2, thanks to @ebiggers ! (#263) 2016-07-26 21:30:35 +02:00
Yann Collet
cbc5e9dc19 fixes oob read 2016-07-24 18:02:04 +02:00
Yann Collet
38b75ddeb2 removed special case all-1 huffman distribution 2016-07-24 15:35:59 +02:00
Yann Collet
7ed5e33b89 minor comment changes 2016-07-24 14:26:11 +02:00
Yann Collet
f8e7b5363f unified encoding types 2016-07-23 16:31:49 +02:00
Yann Collet
571a59034a changed enccoding type order : raw, rle, compressed, repeat-stats 2016-07-23 15:52:05 +02:00
Yann Collet
6fa05a2371 cBlockSize uses little-endian convention 2016-07-22 14:37:09 +02:00
Yann Collet
d5c5a77990 minor comments clarifications 2016-07-20 13:35:14 +02:00
Yann Collet
85f3919960 moved zstd.h to /lib 2016-07-17 20:42:21 +02:00
Yann Collet
9375590462 update version to v0.7.5 2016-07-17 16:44:18 +02:00
Yann Collet
e557fd5e92 minor compression level corrections 2016-07-17 16:21:37 +02:00
Yann Collet
d54b2d23b4 minor static assert for 32/64 bits system. Suggested by @ebiggers 2016-07-17 15:53:18 +02:00
Yann Collet
961b6a0e34 ZSTD_compressBlock() limits block size depending on windowLog parameter 2016-07-15 11:58:49 +02:00
Yann Collet
8847238cac simplified ZSTD_estimateCCtxSize() 2016-07-14 17:05:38 +02:00
Yann Collet
45dc35628c first version of doubleFast 2016-07-12 09:47:31 +02:00
Yann Collet
d158c35e9f added ZSTD_estimateDCtxSize() 2016-07-11 13:46:25 +02:00
Yann Collet
8e0ee681b8 added ZSTD_sizeofDCtx() 2016-07-11 13:09:52 +02:00
Yann Collet
3ae543ce75 added ZSTD_estimateCCtxSize() 2016-07-11 03:12:17 +02:00
Yann Collet
25c506601c promote ZSTD_getDecompressedSize() to stable API 2016-07-10 01:46:18 +02:00
Yann Collet
3b6ae77e15 comment clarification 2016-07-08 23:42:22 +02:00
Yann Collet
ed3845d3fa introduced ZSTD_WINDOWLOG_MAX_32 (#239), suggested by @GregSlazinski 2016-07-08 12:57:10 +02:00
Yann Collet
e72efeb0a1 removed "error_public.h" dependency from "zstd.h" 2016-07-07 14:17:40 +02:00
Yann Collet
e09d38e921 removed mem.h dependency from zbuff.h (experimental section) 2016-07-07 13:17:37 +02:00
Yann Collet
f323bf7d32 added : ZSTD_getDecompressedSize() 2016-07-07 13:14:21 +02:00
Yann Collet
52c04fe58f removed mem.h dependency from zstd.h (experimental section) 2016-07-07 11:53:18 +02:00
Yann Collet
f246cf5423 ZSTD_decompress_usingDDict() compatible with Legacy mode 2016-07-06 20:32:27 +02:00
Yann Collet
d916c908e0 updated doc 2016-07-04 00:42:58 +02:00
Yann Collet
698cb63305 Updated specifications 2016-07-03 18:49:35 +02:00
Yann Collet
2fa9904844 update specification and comments 2016-07-01 20:55:28 +02:00
Yann Collet
6c6e1751f6 use ZSTD_getParams() to simplify code 2016-06-27 15:28:45 +02:00
Yann Collet
3d2cd7f816 Introduced ZSTD_getParams() 
bench now uses ZSTD_createCDict_advanced()
 2016-06-27 15:12:26 +02:00
Yann Collet
529d9c7dee updated version to v0.7.2 2016-06-27 10:03:10 +02:00
Yann Collet
63b5e7a2ea Improved comments 2016-06-26 17:42:15 +02:00
Yann Collet
3755eb8fea fixed strict-aliasing warning on gcc6 2016-06-22 13:15:53 +02:00
Yann Collet
a49e066b26 clarified comments on ZSTD_compressContinue() 2016-06-21 11:54:03 +02:00
Yann Collet
d4f38d0dcd updated library to v0.7.1 2016-06-21 10:15:43 +02:00
Yann Collet
510cff3570 minor comment change 2016-06-16 16:39:55 +02:00
Yann Collet
52a0622beb RepsCodes are saved into Dict 
(uncomplete : need decompression to regenerate them)
 2016-06-16 01:05:04 +02:00
Yann Collet
45c03c564f fixed corruption with inter-blocks repeated offsets 2016-06-14 13:46:11 +02:00
Yann Collet
4266c0a2fd adding inter-blocks rep-offsets 2016-06-14 01:49:25 +02:00
Yann Collet
18c8f79f3e fixed gcc warning on uninitialized structure variable 2016-06-12 22:51:52 +02:00
Yann Collet
cd98f93cff Fixed decompression issue with invalid data 2016-06-11 23:26:22 +02:00
Yann Collet
237ad4beb3 Added single-stream decompression variant using external DTable 2016-06-11 01:46:03 +02:00
Yann Collet
289bbd52e5 Updated huff0 2016-06-11 01:31:54 +02:00
Yann Collet
0974f681a4 completed .gitignore 2016-06-10 14:44:16 +02:00
Yann Collet
9dd12742f3 litBlockType_t is an enum 2016-06-10 00:12:26 +02:00
Yann Collet
662a541431 updated huff0 - now generates a common HUF_DTable type for all decoding tables 2016-06-08 11:11:02 +02:00
Yann Collet
302fb53a76 Removed ZSTD_*_usingPrepared?Ctx() declaration from public space 2016-06-07 12:16:49 +02:00
Yann Collet
81e13ef7cf first implementation of the new dictionary API (untested) 2016-06-07 00:51:51 +02:00
Yann Collet
673f0d7cdc new frame format, allowing custom window size 2016-06-06 00:26:38 +02:00
Yann Collet
89703d20fb reduced dependencies 2016-06-05 01:50:33 +02:00
Yann Collet
a91ca620cf removed HUF_readStats() from public space 2016-06-05 01:33:55 +02:00
Yann Collet
d0e2cd15cb Merged fse_static into fse.h . Now requires FSE_STATIC_LINKING_ONLY macro. 2016-06-05 00:58:01 +02:00
Yann Collet
130fe11394 merged huf_static.h into huf.h . Requires HUF_STATIC_LINKING_ONLY macro. 2016-06-05 00:42:28 +02:00
Yann Collet
49bb0041af removed ZSTD_highbit() from zstd_internal.h, as it is only used by zstd_compress.c 2016-06-04 20:17:38 +02:00
Yann Collet
d3b7f8d21f Merged zstd_static.h into zstd.h . Now requires ZSTD_STATIC_LINKING_ONLY macro 2016-06-04 19:47:02 +02:00
Yann Collet
ac110a1f21 Removed ZBUFF internal util function from public area 2016-06-04 19:16:49 +02:00
Yann Collet
5347aee8f7 merged zbuff_static.h into zbuff.h . Now requires ZBUFF_STATIC_LINKING_ONLY macro 2016-06-04 19:12:48 +02:00
inikep
2a74609b90 zlibWrapper: ZWRAP_createCCtx and ZWRAP_freeCCtx use custom memory allocation functions 2016-06-03 14:53:51 +02:00
inikep
3763c77f6b defaultCustomNULL replaced with defaultCustomMem 2016-06-03 13:28:20 +02:00
inikep
db2f540414 added defaultCustomNULL 2016-06-03 12:56:56 +02:00
inikep
b74a468fad Merge remote-tracking branch 'refs/remotes/Cyan4973/dev070' into dev070 2016-06-02 22:09:09 +02:00
inikep
ff9114aee3 zlibWrapper: added support for custom memory allocation functions 2016-06-02 16:52:36 +02:00
inikep
c4807f4d2f default custom allocation functions moved to zstd_internal.h 2016-06-02 15:11:39 +02:00
inikep
2866951558 opaque parameter for custom memory allocation functions 2016-06-02 13:04:18 +02:00
inikep
9242816b56 fparamsPtr->windowLog==0 means that a frame is skippable 2016-06-01 18:47:04 +02:00
Yann Collet
70d1301d6e Changed ZSTD_adjustCParams() prototype 
`ZSTD_adjustCParams()` is now automatically invoked at the end of `ZSTD_getCParams()`
 2016-06-01 18:45:34 +02:00
Yann Collet
83c3f4427c upgraded zbufftest to also test advanced frame parameters no/checksum no/dictID 2016-06-01 17:44:53 +02:00
inikep
5c2771710d Merge remote-tracking branch 'refs/remotes/Cyan4973/dev070' into dev070 
# Conflicts:
#	.gitignore
#	lib/decompress/zstd_decompress.c
#	programs/zbufftest.c
 2016-06-01 09:16:11 +02:00
Yann Collet
8e3a36a6db decompression validates frame content checksum 2016-06-01 00:18:28 +02:00
inikep
a6b942018d Merge remote-tracking branch 'refs/remotes/origin/dev' into dev070 
# Conflicts:
#	.travis.yml
#	Makefile
#	lib/common/zstd_static.h
#	programs/Makefile
#	projects/VS2008/zstd/zstd.vcproj
#	projects/VS2008/zstdlib/zstdlib.vcproj
#	projects/cmake/lib/CMakeLists.txt
#	projects/cmake/programs/CMakeLists.txt
 2016-06-01 00:07:09 +02:00
Yann Collet
f2a3b6e7b4 added : frame content checksum 2016-05-31 22:23:45 +02:00
inikep
f772bf54a5 support for skippable frames 2016-05-31 12:43:46 +02:00
Yann Collet
c0a9bf3c2e minor code refactoring 2016-05-30 04:48:32 +02:00
Yann Collet
c46fb924df added dictionary ID (incomplete) 2016-05-29 05:01:04 +02:00
Yann Collet
0c5e8b17ad moved xxhash to lib/common 2016-05-29 01:06:30 +02:00
inikep
957823f56f zstdcli: -r (operate recursively on directories) works with dictBuilder and compression 2016-05-25 15:30:55 +02:00
inikep
e75909ef93 ZBUFF_createCCtx_advanced moved to zbuff_static.h 2016-05-25 11:31:16 +02:00
inikep
02c244bf78 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-05-24 17:15:04 +02:00
inikep
fb5df613cf zstd_stats.h included only in debug mode 2016-05-24 15:36:37 +02:00
inikep
71dd67cf57 fullbench: tests for ZBUFF_createCCtx_advanced and ZBUFF_createDCtx_advanced 2016-05-24 10:57:14 +02:00
inikep
13ba880b49 added ZBUFF_createCCtx_advanced and ZBUFF_createDCtx_advanced 2016-05-23 17:04:23 +02:00
inikep
107e243195 added ZSTD_createDCtx_advanced 2016-05-23 16:24:52 +02:00
inikep
50e82c015d added ZSTD_createCCtx_advanced 2016-05-23 15:49:09 +02:00
inikep
35b891c5b0 zstd_stats.h updated to v0.6 2016-05-20 19:42:20 +02:00
Yann Collet
f22a0d653d huff0 dynamic reduction 2016-05-20 14:36:36 +02:00
Yann Collet
de4d4da52d changed version to 0.6.2 
removed make-controlled version (now used from zstd.h)
removed std=c99 by default
added : c90/gno90/c99/gnu99/c11 tests
 2016-05-20 12:20:12 +02:00
Yann Collet
78aa05228a fixed Visual warning in debug mode 2016-05-17 14:30:19 +02:00
Yann Collet
d9b3cca4d3 restore openVMS support (detected by @thatsafunnyname) 2016-05-13 12:43:36 +02:00
inikep
63ecd747de added common/entropy_common.c 2016-05-13 11:27:56 +02:00
Yann Collet
ccd6e86d45 improved decoding speed by inlining x_isError() functions 2016-05-12 15:55:26 +02:00
Yann Collet
16871680e3 minor refactor : more accurate variable scope 2016-05-12 14:27:00 +02:00
Yann Collet
1ceb5a9295 restored DStream init 
due to strange speed regressions and warnins on gcc 5.3
 2016-05-12 13:50:13 +02:00
Yann Collet
ce27e85209 fixed clang conversion warning 2016-05-11 18:55:27 +02:00
Yann Collet
1032fbe714 update FSE library 2016-05-11 18:30:24 +02:00
Yann Collet
249ae0ca99 better big-endian / little-endian r/w support 2016-05-11 16:38:18 +02:00
Yann Collet
c75e4c25e8 Added : ZSTD_getErrorString(), to get error string from error enum (#168) 2016-05-10 17:47:11 +02:00
Yann Collet
b3a9938d0c Merge pull request #182 from inikep/dev 
support for Visual Studio 2012, 2013, 2015
 2016-05-10 17:18:12 +02:00
inikep
ed9a08538c Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 
# Conflicts:
#	lib/common/util.h
#	programs/paramgrill.c
#	visual/2013/fullbench/fullbench.vcxproj.filters
#	visual/2013/fuzzer/fuzzer.vcxproj.filters
 2016-05-10 13:20:01 +02:00
Yann Collet
918b934a26 minor comment update 2016-05-10 07:00:07 +02:00
inikep
aaaf923a17 common/util.h moved to programs/util.h 
decompress/fse_decompress.c moved to common/
 2016-05-09 16:19:25 +02:00
inikep
d44ec48b1c detection of snprintf for Visual <= 2012 2016-05-09 14:37:41 +02:00
Yann Collet
59b6ba7677 Merge pull request #180 from thatsafunnyname/patch-1 
No stdint.h on OpenVMS.
 2016-05-09 11:56:42 +02:00
Yann Collet
4ff73a2eff added : midipix support (#181) 2016-05-09 11:44:00 +02:00
Yann Collet
d275236e6a Fixed Windows API, as detected by @KrzysFR (#171) 2016-05-09 08:19:27 +02:00
Yann Collet
f6ca09b5ff Reduced console display on loading lots of files with zstd --train. Reported by @KrzysFR, see #177 2016-05-09 04:44:45 +02:00
inikep
13c8424ea0 code cleaning 2016-05-05 13:58:56 +02:00
inikep
0bd0faec32 fixed compatibility issues 2016-05-05 13:10:57 +02:00
Yann Collet
6d1d25299a fixed Huff0 quad-symbols decoder (#173) 2016-05-05 12:41:36 +02:00
inikep
9c22e57bfb Compiler Options moved to util.h 2016-05-05 11:53:42 +02:00
Yann Collet
ddb8ebd5b3 Stricter gnu90 tests 2016-05-05 04:59:53 +02:00