AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7,148 Commits 1 Branch 0 Tags 27 MiB
4b141b63e0
Commit Graph

1252 Commits

Author SHA1 Message Date
Bimba Shrestha
b63a1e7ae5 Typo fix 2019-09-27 07:20:20 -07:00
Bimba Shrestha
91daee5c06 Fixing appveyor test 2019-09-26 16:21:57 -07:00
Bimba Shrestha
75b1286354 Fixing shortest failure 2019-09-26 16:07:34 -07:00
Bimba Shrestha
bb27472afc Adding more realistic test for get sequences 2019-09-26 15:38:31 -07:00
Bimba Shrestha
be0bebd24e Adding test and null check for malloc 2019-09-23 15:08:18 -07:00
Nick Terrell
7451c6578c
Merge pull request #1804 from terrelln/wild-and-fast 
Optimize (de)compression and fix wildcopy overread
 2019-09-21 17:04:36 -07:00
Felix Handte
c047fcf7bf
Merge pull request #1806 from felixhandte/estimate-cctx-doc 
Update Comment on `ZSTD_estimateCCtxSize()`
 2019-09-20 15:36:00 -04:00
W. Felix Handte
f7d9b36835 Update Comment on ZSTD_estimateCCtxSize() 2019-09-20 14:11:29 -04:00
Nick Terrell
e068bd01df [tests] Fix decodecorpus 2019-09-20 01:09:47 -07:00
Nick Terrell
51990246c3 [test] Test the bounds of ZSTD_c_srcSizeHint 2019-09-18 11:05:08 -07:00
Bimba Shrestha
3cacc0a30b Casting void pointer to ZSTD_Sequence pointer 2019-09-17 17:44:08 -07:00
Bimba Shrestha
5b038f128f Merge branch 'extract_sequences_api' of https://github.com/bimbashrestha/zstd into extract_sequences_api 2019-09-16 13:35:49 -07:00
Bimba Shrestha
1f93be0f6d Handling memory leak and potential side effect 2019-09-16 13:35:45 -07:00
Bimba Shrestha
a874435478
Merge branch 'dev' into extract_sequences_api 2019-09-16 13:29:59 -07:00
Felix Handte
2164a130f3
Merge pull request #1780 from felixhandte/workspace-efficiency-3 
Avoid Clearing Tables Even When Changing CParams
 2019-09-16 14:37:05 -04:00
Nick Terrell
d721fcf3ee [fuzz] Fix leak in block_round_trip 2019-09-13 10:32:38 -07:00
Nick Terrell
7c4578160e [fuzz] Generate seed data up to 256KB 2019-09-12 15:02:01 -07:00
Dario Pavlovic
51e9d29a51 Merge branch 'improvDataGen' of github.com:darxsys/zstd into improvDataGen 2019-09-12 13:11:02 -07:00
Dario Pavlovic
cd8588077e It's time for all of rng seed code to go. Goodbye 2019-09-12 13:10:34 -07:00
Dario Pavlovic
47bb4c6a23
Update tests/fuzz/fuzz_data_producer.h 2019-09-12 12:45:28 -07:00
Dario Pavlovic
92c58c4d5d Use range instead of the generic uint32 method to use less bytes when generating necessary numbers. 2019-09-12 12:40:12 -07:00
W. Felix Handte
194c542598 Fix Memory Leak in Test 2019-09-11 14:25:30 -04:00
W. Felix Handte
ff67c62458 Fix Compilation Error (uint32_t -> size_t) 2019-09-11 13:59:09 -04:00
W. Felix Handte
5707c8a9d5 Speed Up Test a Little 2019-09-11 13:23:59 -04:00
W. Felix Handte
ed4c2c60c3 Add Fuzzer Test Case for Index Reduction 2019-09-11 13:17:19 -04:00
Dario Pavlovic
b5b24c2a0d Combining fuzz_data_producer restrict calls into a single function 2019-09-11 10:09:29 -07:00
Dario Pavlovic
23cc2d8510 All tests should give some portion of data to the producer and use the rest. 2019-09-10 16:52:38 -07:00
Dario Pavlovic
4dfc1bbf48 Merge branch 'dev' into improvDataGen 2019-09-10 16:20:27 -07:00
Dario Pavlovic
0630d084cb [Fuzz] Improve data generation #1723 
Converting the rest of the tests to use the new data producer.
 2019-09-10 16:14:43 -07:00
Nick Terrell
458b602bb9
Merge pull request #1777 from darxsys/improvDataGen 
[Fuzz] Improve data generation #1723
 2019-09-10 11:51:23 -07:00
Yann Collet
41416f0927
Merge pull request #1773 from bimbashrestha/rle_first_block_decompression_fix 
Removing redundant condition in decompression, making first block rle…
 2019-09-10 11:17:29 -07:00
Bimba Shrestha
caaf43b258 Using a number instead of M prefix in head call 2019-09-10 09:30:37 -07:00
Bimba Shrestha
9e7bb55e14 Addressing comments 2019-09-09 20:04:46 -07:00
Dario Pavlovic
ea1ad123da Addressing nits 2019-09-09 16:13:24 -07:00
Dario Pavlovic
3932fcfebc Fixing issues with double usage of data. 2019-09-09 15:39:04 -07:00
Bimba Shrestha
e6be4cf4eb Changing test file directory names to be more descriptive 2019-09-09 12:08:33 -07:00
Bimba Shrestha
0b25ab2202 Testing first block decompression cli 2019-09-09 11:54:43 -07:00
Dario Pavlovic
a71bbba7be [Fuzz] Improve data generation #1723 2019-09-09 08:43:22 -07:00
Bimba Shrestha
5f8b0f6890 Changing api to get sequences across all blocks 2019-08-30 09:18:44 -07:00
Nick Terrell
8ce9c1917e [regression] small improvement from RLE blocks 2019-08-29 18:05:58 -07:00
Yann Collet
5198347382
Merge pull request #1744 from bimbashrestha/dev 
Generate RLE blocks in the encoder
 2019-08-29 15:19:10 -07:00
bimbashrestha
e5704bbfdf Added test for multiple blocks of zeros and fixed nit about comments 2019-08-28 08:32:34 -07:00
Nick Terrell
793c71d6ae [test][dictBuilder] Add multithreaded tests 2019-08-26 18:32:08 -07:00
Nick Terrell
d0750a1c9c
Merge pull request #1733 from nmagerko/size-hint 
Add --size-hint=# option
 2019-08-23 10:16:10 -07:00
Nick Terrell
e2030a2c40 [fuzz] Add a DEBUGLOG(3) statement to print file 
Enable it by building with this command:

```
./fuzz.py build all --debug 3
```
 2019-08-22 17:27:15 -07:00
Nick Magerko
493f95c7df Fix merge conflicts 2019-08-22 11:51:41 -07:00
Nick Terrell
a505463710
Merge pull request #1726 from nmagerko/stream-size 
Add --stream-size=# option
 2019-08-22 11:31:15 -07:00
Nick Magerko
b3540507f5 Remove bc from play tests 2019-08-21 10:27:54 -07:00
Nick Terrell
188311dd4a
Merge pull request #1736 from terrelln/fuzz-fix 
[fuzz] Improve fuzzer build script and docs
 2019-08-21 10:09:38 -07:00
Nick Terrell
3982935aef [fuzz] Improve fuzzer build script and docs 
* Remove the `make libFuzzer` target since it is broken and obsoleted
  by `CC=clang CXX=clang++ ./fuzz.py build all --enable-fuzzer`. The
  new `-fsanitize=fuzzer` is much better because it works with MSAN
  by default.
* Improve the `./fuzz.py gen` command by making the input type explicit
  when creating a new target.
* Update the `README` for `--enable-fuzzer`.

Fixes #1727.
 2019-08-20 16:44:50 -07:00
Nick Magerko
c7a24d7a14 Define ZSTD_SRCSIZEHINT_MIN as 0 2019-08-20 13:06:15 -07:00
Nick Magerko
f23402f1f5 Remove unnecessary test case 2019-08-19 17:20:46 -07:00
Nick Magerko
294f1e5cfe Fix typo in test 2019-08-19 16:53:02 -07:00
Nick Magerko
f9af70ca8a Fix playTests and add additional cases 2019-08-19 16:48:35 -07:00
Nick Magerko
ea9d35922c Add size-hint to fuzz tests 2019-08-19 15:12:29 -07:00
Nick Magerko
dffbac5f89 Add --size-hint=# option 2019-08-19 11:38:49 -07:00
Nick Magerko
85d07c6c47 Tweak stdout, stderr redirection in new playTests 2019-08-16 12:49:21 -07:00
Nick Magerko
af0c9501d1 Add --stream-size=# command 2019-08-16 11:34:39 -07:00
Ed Maste
b81d7cc6a0 remove extraneous doubled ;s 2019-08-15 21:17:06 -04:00
Yann Collet
0b0b83e8f3 fix test 122 
it's an unsupported scenario.
 2019-08-03 16:51:26 +02:00
Yann Collet
efe8496755 minor test refactoring 
just for clarity, for the currently failing unit test
 2019-08-02 19:31:19 +02:00
Yann Collet
387e20d4f0 fixed minor conversion warning in datagen 2019-08-02 18:02:54 +02:00
Yann Collet
37f47e51a8 fixed datagen 
to produce same content on both 32 and 64-bit platforms
by removing floating from literal table determination.

also : added checksum trace in compression control test,
so that it's easier to determine if test fails
as a consequence of compressing a different sample.
 2019-08-02 17:34:53 +02:00
Yann Collet
d1927f0b39 regenerate sample to compress 
to reduce chances of differences between 32 and 64-bit fuzzer tests
 2019-08-02 15:31:00 +02:00
Yann Collet
5cf1b24aca fixed strategies greedy, lazy & lazy2 
restore dictionary compression ratio
 2019-08-02 14:21:39 +02:00
Yann Collet
2115292616 minor : fixed ptr arithmetic 
invalid on void ptr
 2019-08-01 17:12:26 +02:00
Yann Collet
810a9cac08 added efficiency test 
to detect gross CR variations after a patch.

Tests normal and dictionary compression.
 2019-08-01 16:59:22 +02:00
Yann Collet
98692c2838 fixed compression ratio regression when dictionary-compressing medium-size inputs at levels 1-3 2019-08-01 15:58:17 +02:00
Yann Collet
f2620697c7
Merge pull request #1685 from vivekmig/dev 
Add Check if Block Size Exceeds Maximum
 2019-07-19 15:22:29 -07:00
Vivek Miglani
a3ce0c9d04 Fixing decodecorpus test issue 2019-07-18 14:32:09 -07:00
Nick Terrell
4c2943df23 [regression] Update results for ZSTD_double_fast update 2019-07-15 11:25:22 -07:00
Nick Terrell
75cfe1dc69
[ldm] Fix bug in overflow correction with large job size (#1678) 
* [ldm] Fix bug in overflow correction with large job size

* [zstdmt] Respect ZSTDMT_JOBSIZE_MAX (1G in 64-bit mode)

* [test] Add test that exposes the bug

Sadly the test fails on our CI because it uses too much memory, so
I had to comment it out.
 2019-07-12 18:45:18 -04:00
LeeYoung624
654cb9d439 fix gitignore errors 2019-07-09 21:08:13 +08:00
Nick Terrell
e962f07d19
[fuzz] Add a compression fuzzer with randomly sized output buffer (#1670) 2019-07-02 22:05:07 -07:00
Yann Collet
4d611ca405
Merge pull request #1664 from ephiepark/dev 
decodecorpus
 2019-07-01 14:13:49 -07:00
Ephraim Park
28309520c0 reflect code review comments 2019-07-01 10:17:30 -07:00
Ephraim Park
01e8384000 Add test case for short bistream 2019-06-27 17:37:39 -07:00
Tyler-Tran
c55d2e7ba3 Adding shrinking flag for cover and fastcover (#1656) 
* Changed ERROR(GENERIC) excluding inits

* editing git ignore

* Edited init functions to size_t returns

* moved declarations earlier

* resolved issues with changes to init functions

* fixed style and an error check

* attempting to add tests that might trigger changes

* added && die to cases expecting to fail

* resolved no die on expected failed command

* fixed accel to be incorrect value

* Adding an automated shrinking option

* Fixing build

* finalizing fixes

* fix?

* Removing added comment in cover.h

* Styling fixes

* Merging with fb dev

* removing megic number for default regression

* Requested revisions

* fixing support for fast cover

* fixing casting errors

* parenthesis fix

* fixing some build nits

* resolving travis ci syntax

* might resolve all compilation issues

* removed unused variable

* remodeling the selectDict function

* fixing bad memory access

* fixing error checks

* fixed erroring check in selectDict

* fixing mixed declarations

* modify mixed declaration

* fixing nits and adding test cases

* Adding requested changes + fixed bug for error checking

* switched double comparison from != to <

* fixed declaration typing

* refactoring COVER_best_finish() and changing shrinkDict

* removing the const's

* modifying ZDICT_optimizeTrainFromBuffer_cover functions

* fixing potential bad memcpy

* fixing the error function for dict size
 2019-06-27 16:26:57 -07:00
Ephraim Park
734eff70b8 enable repeat mode on rle 2019-06-26 16:39:00 -07:00
Nick Terrell
8e16b872e5 [tests] Add tests for big dictionaries 2019-06-21 17:58:24 -07:00
Nick Terrell
06d5e5ff5a [regression] Update results for small wlog patch PR#1624 2019-06-17 10:40:33 -07:00
Yann Collet
1166782eb7
Merge pull request #1640 from felixhandte/lstat-macro-guard 
Protect lstat() With Better Macro Guard
 2019-06-14 17:30:03 -07:00
Felix Handte
344f942573
Merge pull request #1620 from michaelforney/test-no-threads 
Skip --adapt and --rsyncable tests when built without thread support
 2019-06-11 15:05:25 -04:00
Nick Terrell
81276c4ee4
Merge pull request #1646 from terrelln/oss-fuzz 
[fuzz] Remove max_len from the options
 2019-06-10 22:13:16 -07:00
Nick Terrell
6810dd6191 [fuzz] Remove max_len from the options 2019-06-10 11:05:45 -07:00
W. Felix Handte
b878bd08dd Clean Up Temp Files Produced By playTests.sh 2019-06-07 15:32:28 -04:00
Nick Terrell
f3800ba073 [regression] Update results.csv 2019-06-05 20:40:52 -07:00
Yann Collet
78056a3e9c consistency : changed constant name to kSampleSizeDefault 
as suggested by @terrelln.

added <assert.h> include
 2019-05-30 10:13:56 -07:00
Yann Collet
ab3625cffd define CONTROL, to ensure check cannot be disabled 
assert() can be disabled with NDEBUG
 2019-05-29 16:18:22 -07:00
Yann Collet
904d4da239 fullbench : minor refactoring, for readability 2019-05-29 16:08:49 -07:00
Yann Collet
ed38b645db fullbench: pass proper parameters in scenario 43 2019-05-29 15:26:06 -07:00
Yann Collet
c63081623f fullbench 43: ensure context is freed after each usage 2019-05-28 17:27:52 -07:00
Yann Collet
eb6b199038 fullbench: added streaming with fresh CCtx scenario 
worst case situation, where context must be recreated every time
but without knowledge of the input size
thus sizing the context for some large input.
 2019-05-28 16:50:49 -07:00
Yann Collet
4c4149452c fullbench can now select sample size with command -B# 2019-05-28 16:37:03 -07:00
Yann Collet
4baecdf72a added comments to better understand enforceMaxDist() 2019-05-28 13:15:48 -07:00
Michael Forney
1bc77ee174 Skip --adapt and --rsyncable tests when built without thread support 
These options require ZSTD_MULTITHREAD.
 2019-05-26 03:20:37 -07:00
W. Felix Handte
6a0638048a Add Test 2019-05-24 17:21:44 -04:00
Tyler-Tran
cb47871a0a [dictBuilder] Be more specific than ERROR(generic) (#1616) 
* Specify errors at a finer granularity than `ERROR(generic)`.
* Add tests for bad parameters in the dictionary builder.
 2019-05-22 18:57:50 -07:00
Nick Terrell
bfbef51afb
Merge pull request #1610 from terrelln/pool-test 
[test] Fix poolTests deadlock and flakiness
 2019-05-02 15:13:50 -07:00
Nick Terrell
81f1f76479 [tests] Only sleep 10ms in poolTests 2019-05-02 14:35:25 -07:00