AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7,207 Commits 1 Branch 0 Tags 27 MiB
826b555463
Commit Graph

1266 Commits

Author SHA1 Message Date
Yann Collet
989e0f0cee Added --output-dir-flat= 
which is a preferred construction when applicable.

Also, added --output-dir-flat into man page.
 2019-10-21 14:39:18 -07:00
Yann Collet
6cf04c0344
Merge pull request #1834 from facebook/winFix 
Windows fixes
 2019-10-21 13:45:17 -07:00
Yann Collet
2ad75bb574 validated 'make test' for mingw environment 2019-10-18 15:37:07 -07:00
Yann Collet
85a016ed92 made make check faster 
for shorter feedback loop
 2019-10-18 15:21:50 -07:00
Nick Terrell
243824551f [threading] Add debug utilities 2019-10-18 15:05:34 -07:00
Yann Collet
29e46ed0bd fix test on windows 
isDirectory() doesn't work on Windows
if directory name is followed by '/'
 2019-10-18 14:28:34 -07:00
Nick Terrell
8c11f089a1 [fuzz] Increase output buffer size of stream_round_trip 
Fixes OSS-Fuzz crash.
Credit to OSS-Fuzz
 2019-10-18 13:39:08 -07:00
Yann Collet
ad86a5d0bc rewrite FIO_createFilename_fromOutDir() 2019-10-18 11:15:10 -07:00
Yann Collet
ba7e2b6da7 tests: can override isTerminal with environment variable 2019-10-17 15:07:47 -07:00
Yann Collet
6323966e53 updated erroneous comments using ZSTD_dm_* 
instead of the current ZSTD_dct_*,
reported by @nigeltao (#1822)
 2019-10-16 16:14:04 -07:00
Yann Collet
b3e92ad346
Merge pull request #1812 from senhuang42/output-dir-flat 
Support for --output-dir-flat: output into a directory
 2019-10-14 09:25:05 -07:00
Yann Collet
fb77afc626
Merge pull request #1760 from bimbashrestha/extract_sequences_api 
Adding api for extracting sequences from seqstore
 2019-10-10 13:11:18 -07:00
Bimba Shrestha
36528b96c4 Manually moving instead of memcpy on decoder and using genBuffer() 2019-10-03 09:26:51 -07:00
Sen Huang
f80437c586 Add support for --output-dir-flat 
New flag to specify output directory destination for multiple files.
 2019-10-02 11:08:20 -04:00
Bimba Shrestha
b63a1e7ae5 Typo fix 2019-09-27 07:20:20 -07:00
Bimba Shrestha
91daee5c06 Fixing appveyor test 2019-09-26 16:21:57 -07:00
Bimba Shrestha
75b1286354 Fixing shortest failure 2019-09-26 16:07:34 -07:00
Bimba Shrestha
bb27472afc Adding more realistic test for get sequences 2019-09-26 15:38:31 -07:00
Bimba Shrestha
be0bebd24e Adding test and null check for malloc 2019-09-23 15:08:18 -07:00
Nick Terrell
7451c6578c
Merge pull request #1804 from terrelln/wild-and-fast 
Optimize (de)compression and fix wildcopy overread
 2019-09-21 17:04:36 -07:00
Felix Handte
c047fcf7bf
Merge pull request #1806 from felixhandte/estimate-cctx-doc 
Update Comment on `ZSTD_estimateCCtxSize()`
 2019-09-20 15:36:00 -04:00
W. Felix Handte
f7d9b36835 Update Comment on ZSTD_estimateCCtxSize() 2019-09-20 14:11:29 -04:00
Nick Terrell
e068bd01df [tests] Fix decodecorpus 2019-09-20 01:09:47 -07:00
Nick Terrell
51990246c3 [test] Test the bounds of ZSTD_c_srcSizeHint 2019-09-18 11:05:08 -07:00
Bimba Shrestha
3cacc0a30b Casting void pointer to ZSTD_Sequence pointer 2019-09-17 17:44:08 -07:00
Bimba Shrestha
5b038f128f Merge branch 'extract_sequences_api' of https://github.com/bimbashrestha/zstd into extract_sequences_api 2019-09-16 13:35:49 -07:00
Bimba Shrestha
1f93be0f6d Handling memory leak and potential side effect 2019-09-16 13:35:45 -07:00
Bimba Shrestha
a874435478
Merge branch 'dev' into extract_sequences_api 2019-09-16 13:29:59 -07:00
Felix Handte
2164a130f3
Merge pull request #1780 from felixhandte/workspace-efficiency-3 
Avoid Clearing Tables Even When Changing CParams
 2019-09-16 14:37:05 -04:00
Nick Terrell
d721fcf3ee [fuzz] Fix leak in block_round_trip 2019-09-13 10:32:38 -07:00
Nick Terrell
7c4578160e [fuzz] Generate seed data up to 256KB 2019-09-12 15:02:01 -07:00
Dario Pavlovic
51e9d29a51 Merge branch 'improvDataGen' of github.com:darxsys/zstd into improvDataGen 2019-09-12 13:11:02 -07:00
Dario Pavlovic
cd8588077e It's time for all of rng seed code to go. Goodbye 2019-09-12 13:10:34 -07:00
Dario Pavlovic
47bb4c6a23
Update tests/fuzz/fuzz_data_producer.h 2019-09-12 12:45:28 -07:00
Dario Pavlovic
92c58c4d5d Use range instead of the generic uint32 method to use less bytes when generating necessary numbers. 2019-09-12 12:40:12 -07:00
W. Felix Handte
194c542598 Fix Memory Leak in Test 2019-09-11 14:25:30 -04:00
W. Felix Handte
ff67c62458 Fix Compilation Error (uint32_t -> size_t) 2019-09-11 13:59:09 -04:00
W. Felix Handte
5707c8a9d5 Speed Up Test a Little 2019-09-11 13:23:59 -04:00
W. Felix Handte
ed4c2c60c3 Add Fuzzer Test Case for Index Reduction 2019-09-11 13:17:19 -04:00
Dario Pavlovic
b5b24c2a0d Combining fuzz_data_producer restrict calls into a single function 2019-09-11 10:09:29 -07:00
Dario Pavlovic
23cc2d8510 All tests should give some portion of data to the producer and use the rest. 2019-09-10 16:52:38 -07:00
Dario Pavlovic
4dfc1bbf48 Merge branch 'dev' into improvDataGen 2019-09-10 16:20:27 -07:00
Dario Pavlovic
0630d084cb [Fuzz] Improve data generation #1723 
Converting the rest of the tests to use the new data producer.
 2019-09-10 16:14:43 -07:00
Nick Terrell
458b602bb9
Merge pull request #1777 from darxsys/improvDataGen 
[Fuzz] Improve data generation #1723
 2019-09-10 11:51:23 -07:00
Yann Collet
41416f0927
Merge pull request #1773 from bimbashrestha/rle_first_block_decompression_fix 
Removing redundant condition in decompression, making first block rle…
 2019-09-10 11:17:29 -07:00
Bimba Shrestha
caaf43b258 Using a number instead of M prefix in head call 2019-09-10 09:30:37 -07:00
Bimba Shrestha
9e7bb55e14 Addressing comments 2019-09-09 20:04:46 -07:00
Dario Pavlovic
ea1ad123da Addressing nits 2019-09-09 16:13:24 -07:00
Dario Pavlovic
3932fcfebc Fixing issues with double usage of data. 2019-09-09 15:39:04 -07:00
Bimba Shrestha
e6be4cf4eb Changing test file directory names to be more descriptive 2019-09-09 12:08:33 -07:00
Bimba Shrestha
0b25ab2202 Testing first block decompression cli 2019-09-09 11:54:43 -07:00
Dario Pavlovic
a71bbba7be [Fuzz] Improve data generation #1723 2019-09-09 08:43:22 -07:00
Bimba Shrestha
5f8b0f6890 Changing api to get sequences across all blocks 2019-08-30 09:18:44 -07:00
Nick Terrell
8ce9c1917e [regression] small improvement from RLE blocks 2019-08-29 18:05:58 -07:00
Yann Collet
5198347382
Merge pull request #1744 from bimbashrestha/dev 
Generate RLE blocks in the encoder
 2019-08-29 15:19:10 -07:00
bimbashrestha
e5704bbfdf Added test for multiple blocks of zeros and fixed nit about comments 2019-08-28 08:32:34 -07:00
Nick Terrell
793c71d6ae [test][dictBuilder] Add multithreaded tests 2019-08-26 18:32:08 -07:00
Nick Terrell
d0750a1c9c
Merge pull request #1733 from nmagerko/size-hint 
Add --size-hint=# option
 2019-08-23 10:16:10 -07:00
Nick Terrell
e2030a2c40 [fuzz] Add a DEBUGLOG(3) statement to print file 
Enable it by building with this command:

```
./fuzz.py build all --debug 3
```
 2019-08-22 17:27:15 -07:00
Nick Magerko
493f95c7df Fix merge conflicts 2019-08-22 11:51:41 -07:00
Nick Terrell
a505463710
Merge pull request #1726 from nmagerko/stream-size 
Add --stream-size=# option
 2019-08-22 11:31:15 -07:00
Nick Magerko
b3540507f5 Remove bc from play tests 2019-08-21 10:27:54 -07:00
Nick Terrell
188311dd4a
Merge pull request #1736 from terrelln/fuzz-fix 
[fuzz] Improve fuzzer build script and docs
 2019-08-21 10:09:38 -07:00
Nick Terrell
3982935aef [fuzz] Improve fuzzer build script and docs 
* Remove the `make libFuzzer` target since it is broken and obsoleted
  by `CC=clang CXX=clang++ ./fuzz.py build all --enable-fuzzer`. The
  new `-fsanitize=fuzzer` is much better because it works with MSAN
  by default.
* Improve the `./fuzz.py gen` command by making the input type explicit
  when creating a new target.
* Update the `README` for `--enable-fuzzer`.

Fixes #1727.
 2019-08-20 16:44:50 -07:00
Nick Magerko
c7a24d7a14 Define ZSTD_SRCSIZEHINT_MIN as 0 2019-08-20 13:06:15 -07:00
Nick Magerko
f23402f1f5 Remove unnecessary test case 2019-08-19 17:20:46 -07:00
Nick Magerko
294f1e5cfe Fix typo in test 2019-08-19 16:53:02 -07:00
Nick Magerko
f9af70ca8a Fix playTests and add additional cases 2019-08-19 16:48:35 -07:00
Nick Magerko
ea9d35922c Add size-hint to fuzz tests 2019-08-19 15:12:29 -07:00
Nick Magerko
dffbac5f89 Add --size-hint=# option 2019-08-19 11:38:49 -07:00
Nick Magerko
85d07c6c47 Tweak stdout, stderr redirection in new playTests 2019-08-16 12:49:21 -07:00
Nick Magerko
af0c9501d1 Add --stream-size=# command 2019-08-16 11:34:39 -07:00
Ed Maste
b81d7cc6a0 remove extraneous doubled ;s 2019-08-15 21:17:06 -04:00
Yann Collet
0b0b83e8f3 fix test 122 
it's an unsupported scenario.
 2019-08-03 16:51:26 +02:00
Yann Collet
efe8496755 minor test refactoring 
just for clarity, for the currently failing unit test
 2019-08-02 19:31:19 +02:00
Yann Collet
387e20d4f0 fixed minor conversion warning in datagen 2019-08-02 18:02:54 +02:00
Yann Collet
37f47e51a8 fixed datagen 
to produce same content on both 32 and 64-bit platforms
by removing floating from literal table determination.

also : added checksum trace in compression control test,
so that it's easier to determine if test fails
as a consequence of compressing a different sample.
 2019-08-02 17:34:53 +02:00
Yann Collet
d1927f0b39 regenerate sample to compress 
to reduce chances of differences between 32 and 64-bit fuzzer tests
 2019-08-02 15:31:00 +02:00
Yann Collet
5cf1b24aca fixed strategies greedy, lazy & lazy2 
restore dictionary compression ratio
 2019-08-02 14:21:39 +02:00
Yann Collet
2115292616 minor : fixed ptr arithmetic 
invalid on void ptr
 2019-08-01 17:12:26 +02:00
Yann Collet
810a9cac08 added efficiency test 
to detect gross CR variations after a patch.

Tests normal and dictionary compression.
 2019-08-01 16:59:22 +02:00
Yann Collet
98692c2838 fixed compression ratio regression when dictionary-compressing medium-size inputs at levels 1-3 2019-08-01 15:58:17 +02:00
Yann Collet
f2620697c7
Merge pull request #1685 from vivekmig/dev 
Add Check if Block Size Exceeds Maximum
 2019-07-19 15:22:29 -07:00
Vivek Miglani
a3ce0c9d04 Fixing decodecorpus test issue 2019-07-18 14:32:09 -07:00
Nick Terrell
4c2943df23 [regression] Update results for ZSTD_double_fast update 2019-07-15 11:25:22 -07:00
Nick Terrell
75cfe1dc69
[ldm] Fix bug in overflow correction with large job size (#1678) 
* [ldm] Fix bug in overflow correction with large job size

* [zstdmt] Respect ZSTDMT_JOBSIZE_MAX (1G in 64-bit mode)

* [test] Add test that exposes the bug

Sadly the test fails on our CI because it uses too much memory, so
I had to comment it out.
 2019-07-12 18:45:18 -04:00
LeeYoung624
654cb9d439 fix gitignore errors 2019-07-09 21:08:13 +08:00
Nick Terrell
e962f07d19
[fuzz] Add a compression fuzzer with randomly sized output buffer (#1670) 2019-07-02 22:05:07 -07:00
Yann Collet
4d611ca405
Merge pull request #1664 from ephiepark/dev 
decodecorpus
 2019-07-01 14:13:49 -07:00
Ephraim Park
28309520c0 reflect code review comments 2019-07-01 10:17:30 -07:00
Ephraim Park
01e8384000 Add test case for short bistream 2019-06-27 17:37:39 -07:00
Tyler-Tran
c55d2e7ba3 Adding shrinking flag for cover and fastcover (#1656) 
* Changed ERROR(GENERIC) excluding inits

* editing git ignore

* Edited init functions to size_t returns

* moved declarations earlier

* resolved issues with changes to init functions

* fixed style and an error check

* attempting to add tests that might trigger changes

* added && die to cases expecting to fail

* resolved no die on expected failed command

* fixed accel to be incorrect value

* Adding an automated shrinking option

* Fixing build

* finalizing fixes

* fix?

* Removing added comment in cover.h

* Styling fixes

* Merging with fb dev

* removing megic number for default regression

* Requested revisions

* fixing support for fast cover

* fixing casting errors

* parenthesis fix

* fixing some build nits

* resolving travis ci syntax

* might resolve all compilation issues

* removed unused variable

* remodeling the selectDict function

* fixing bad memory access

* fixing error checks

* fixed erroring check in selectDict

* fixing mixed declarations

* modify mixed declaration

* fixing nits and adding test cases

* Adding requested changes + fixed bug for error checking

* switched double comparison from != to <

* fixed declaration typing

* refactoring COVER_best_finish() and changing shrinkDict

* removing the const's

* modifying ZDICT_optimizeTrainFromBuffer_cover functions

* fixing potential bad memcpy

* fixing the error function for dict size
 2019-06-27 16:26:57 -07:00
Ephraim Park
734eff70b8 enable repeat mode on rle 2019-06-26 16:39:00 -07:00
Nick Terrell
8e16b872e5 [tests] Add tests for big dictionaries 2019-06-21 17:58:24 -07:00
Nick Terrell
06d5e5ff5a [regression] Update results for small wlog patch PR#1624 2019-06-17 10:40:33 -07:00
Yann Collet
1166782eb7
Merge pull request #1640 from felixhandte/lstat-macro-guard 
Protect lstat() With Better Macro Guard
 2019-06-14 17:30:03 -07:00
Felix Handte
344f942573
Merge pull request #1620 from michaelforney/test-no-threads 
Skip --adapt and --rsyncable tests when built without thread support
 2019-06-11 15:05:25 -04:00
Nick Terrell
81276c4ee4
Merge pull request #1646 from terrelln/oss-fuzz 
[fuzz] Remove max_len from the options
 2019-06-10 22:13:16 -07:00
Nick Terrell
6810dd6191 [fuzz] Remove max_len from the options 2019-06-10 11:05:45 -07:00
W. Felix Handte
b878bd08dd Clean Up Temp Files Produced By playTests.sh 2019-06-07 15:32:28 -04:00