AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
4,444 Commits 1 Branch 0 Tags 27 MiB
86b8134cad
Commit Graph

4444 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Terrell
1c23b64049 [fuzz] fuzz.py can minimize and zip corpora 
* "minimize" minimizes the corpora into an output directory.
* "zip" zips up the minimized corpora, which are ready to deploy.
 2017-09-25 12:04:12 -07:00
Yann Collet
96f0cde31a minor function rename 
ZSTD_estimateCStreamSize_advanced_usingCParams -> ZSTD_estimateCStreamSize_usingCParams
_usingX is clear.
_advanced feels redundant
 2017-09-24 16:47:02 -07:00
Yann Collet
7c3dea42ce added prototypes for advanced parameters for decompression API 
required to decode custom formats
 2017-09-24 15:57:29 -07:00
Yann Collet
e60f48c549 Merge branch 'dev' into newFormats 2017-09-24 14:33:37 -07:00
W. Felix Handte
360238733a Adds LZ4 support by default if LZ4 is available 
Simple makefile change + quick typename change

Test:

  make clean
  make
  # successfully produces binary without lz4 support

  make clean
  # with flags to pick up my lz4 build
  make MOREFLAGS="-L/home/felixh/prog/lz4/lib -I/home/felixh/prog/lz4/lib"
  # successfully produces binary with lz4 support

  echo "TEST TEST TEST THIS IS A TEST STRING PLEASE TEST THIS PLEASE OK THANK YOU" | \
    ./lz4/lz4 | \
    LD_LIBRARY_PATH=/home/felixh/prog/lz4/lib ./zstd/zstd -d
  # successfully prints TEST TEST TEST THIS IS A TEST STRING PLEASE TEST THIS PLEASE OK THANK YOU
 2017-09-22 13:28:56 -07:00
Yann Collet
8977224b9b Merge pull request #859 from terrelln/31 
Prepare for ZSTD_WINDOWLOG_MAX == 31
 2017-09-22 09:01:39 -07:00
Yann Collet
14946af10c Merge pull request #858 from facebook/controlSize 
added control of frame content size at end of decompression
 2017-09-21 17:24:43 -07:00
Nick Terrell
d6abb28951 Prepare for ZSTD_WINDOWLOG_MAX == 31 2017-09-21 17:18:41 -07:00
Yann Collet
cd3115b284 added control from frame content size at end of decompression 
adding check at end of single-pass ZSTD_decompressFrame().
Check within ZSTD_decompressContinue() was already added in a previous patch : b3f33ccfb3
 2017-09-21 16:21:10 -07:00
Yann Collet
645563583e Merge branch 'dev' into newFormats 2017-09-21 16:08:06 -07:00
Yann Collet
f97c2dbd39 created ZSTD_format declaration 2017-09-21 16:07:29 -07:00
Yann Collet
da74aabc00 Merge pull request #850 from terrelln/fse-optimal 
[fse] Fix FSE_optimalTableLog() for srcSize==1
 2017-09-19 14:59:21 -07:00
Yann Collet
c399ab4804 Merge pull request #849 from terrelln/30 
[bitstream] Allow adding 31 bits at a time
 2017-09-19 14:25:10 -07:00
Yann Collet
a361e9af08 Merge pull request #857 from terrelln/bad-assert 
Fix minor fuzz test failures
 2017-09-19 14:22:08 -07:00
Nick Terrell
74718d7e43 [bitstream] Allow adding 31 bits at a time 2017-09-19 13:57:33 -07:00
Nick Terrell
6c9ed76676 [ldm] Fix corner case where minMatch < 8 
There is a potential read buffer overflow when minMatch < 8.

fix-fuzz-failure
 2017-09-19 13:49:37 -07:00
Nick Terrell
18442a31ff [libzstd] Fix bad window size assert 
The window size is not validated or used in the one-pass API, so there
shouldn't be an assert based on it.

fix-fuzz-failure
 2017-09-19 13:47:59 -07:00
Yann Collet
bdc83f523b Merge branch 'dev' of github.com:facebook/zstd into dev 2017-09-18 15:50:14 -07:00
Yann Collet
9c1908a3cd added streaming_memory_usage example 2017-09-18 15:49:59 -07:00
Yann Collet
b0c0e3a3fb Merge pull request #853 from terrelln/blog 
[zstdcli] Fix LDM advanced options parsing
 2017-09-18 15:21:23 -07:00
Yann Collet
79e8ae61a7 Merge pull request #852 from facebook/fix851 
fix #851 : sudo zstd -t file.zst changes /dev/null permissions
 2017-09-18 15:10:02 -07:00
Nick Terrell
1fe762e236 [zstdcli] Fix LDM advanced options parsing 2017-09-18 14:49:35 -07:00
Yann Collet
cb8b471e8b Merge branch 'dev' of github.com:facebook/zstd into dev 2017-09-18 14:48:23 -07:00
Yann Collet
7d1ff3817b fix ZSTD_sizeof_CCtx() / ZSTD_sizeof_CStream() 
previous result was over-estimated
by counting streaming buffers twice
 2017-09-18 14:47:34 -07:00
Nick Terrell
cae3e3c652 [fse] Fix FSE_optimalTableLog() for srcSize==1 2017-09-18 14:11:18 -07:00
Yann Collet
72a80515ec Merge pull request #848 from terrelln/fparams 
[block] Don't use fParams in ZSTD_decompressBlock()
 2017-09-18 13:48:31 -07:00
Yann Collet
92889709f9 fix #851 : sudo zstd -t file.zst changes /dev/null permissions 
reported by @mike155
 2017-09-18 13:41:54 -07:00
Yann Collet
539b91ee9b minor : added assert in bt 2017-09-16 23:41:58 -07:00
Yann Collet
4a52a89026 Merge pull request #846 from facebook/splitDict 
Split dict
 2017-09-15 23:26:23 -07:00
Nick Terrell
5f22479517 [block] Don't use fParams in ZSTD_decompressBlock() 2017-09-15 17:37:20 -07:00
Yann Collet
1722055799 add comment on using -B# to split input file for dictionary training 2017-09-15 16:23:50 -07:00
Yann Collet
c68d17f2da ensures that sampleSizes table is large enough 
as recommended by @terrelln
 2017-09-15 15:31:31 -07:00
Yann Collet
31829cb057 Merge pull request #847 from terrelln/fuzzer 
[fuzzer] Fuzz long range matching & new API
 2017-09-15 12:09:00 -07:00
Yann Collet
25a60488dd fixed 64-to-32 conversion warnings 2017-09-15 11:55:13 -07:00
Yann Collet
a9694231ca fixed minor conversion warning 2017-09-15 10:16:26 -07:00
Yann Collet
086b9597d9 added ability to split input files for dictionary training 
using command -B#
This is the same behavior as benchmark module,
which can also split input into arbitrary size blocks, using -B#.
 2017-09-14 16:45:10 -07:00
Yann Collet
77c137b3ae minor comment refactor 2017-09-14 15:12:57 -07:00
Nick Terrell
39357c41cb [fuzzer] Fuzz long range matching & new API 2017-09-14 14:48:08 -07:00
Yann Collet
218c09e5b3 Merge pull request #844 from terrelln/fuzzer 
Fuzzer
 2017-09-14 11:40:25 -07:00
Nick Terrell
9712d5ebe6 [fuzzer] Fix bugs in fuzz.py 2017-09-13 19:08:35 -07:00
Nick Terrell
a6f08b4783 [fuzzer] Fix FUZZ_seed() 2017-09-13 18:41:32 -07:00
Nick Terrell
6c6412cef9 [fuzzer] Update README.md 2017-09-13 18:23:52 -07:00
Nick Terrell
6b8236cf7e [fuzz] Add fuzzing helper script 2017-09-13 17:45:21 -07:00
Nick Terrell
b7e1522330 Add block fuzzers 2017-09-13 17:44:41 -07:00
Nick Terrell
def3214d74 [fuzzer] Handle single empty directory 2017-09-13 17:44:30 -07:00
Yann Collet
739b620814 Merge pull request #842 from stellamplau/decodeCorpus-maxSize 
Add flag to limit max decompressed size in decodeCorpus
 2017-09-13 17:26:55 -07:00
Yann Collet
335780c427 fixed too strong alignment assert in ZSTD_initStaticCCtx() 
64-bits fields are only 32-bits aligned on 32-bits CPU
 2017-09-13 16:35:29 -07:00
Nick Terrell
8b6c80ada8 Update fuzzer Makefile 2017-09-13 16:16:57 -07:00
Nick Terrell
677c2cbf89 Update fuzzer sources 2017-09-13 16:16:57 -07:00
Stella Lau
963558a072 Fix implicit conversion error 2017-09-13 16:01:16 -07:00