AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
6,523 Commits 1 Branch 0 Tags 27 MiB
a371df534d
Commit Graph

6523 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Terrell
a8db4bd3fe
Merge pull request #1593 from terrelln/legacy-fix 
[fuzzer] Run fuzzers in legacy mode and fix legacy code
 2019-04-18 16:07:42 -07:00
Nick Terrell
785331acbd [paramgrill] Fix mingw build errors 2019-04-18 15:06:56 -07:00
Nick Terrell
579f3d7794 [legacy] Fix bug in ZSTD_decodeSeqHeaders() 2019-04-18 13:41:10 -07:00
Nick Terrell
ac098c7f5f [legacy] Fix a bug in ZSTDv06_findFrameSizeInfoLegacy() 2019-04-18 13:33:26 -07:00
Nick Terrell
610a81ecf9 [fuzzer] Compile with legacy support 2019-04-18 12:44:55 -07:00
Nick Terrell
cc669006dc [fuzzer] Size the decompression output buffer randomly 2019-04-18 12:44:21 -07:00
Nick Terrell
af3531ee35
Merge pull request #1590 from terrelln/frame-info-fuzz 
[fuzzer] Fuzz frame info functions
 2019-04-17 12:13:49 -07:00
Nick Terrell
58bcc328a4 [fuzz] Add a seedcorpora target for oss-fuzz 2019-04-17 12:13:06 -07:00
Nick Terrell
ee130a9889 [libzstd] Check the size in readSkippableFrameSize() 2019-04-17 11:41:55 -07:00
Nick Terrell
5922f4e2ae [legacy] Return the right error code 2019-04-17 11:34:52 -07:00
Nick Terrell
09caa4d800 [fuzzer] Add a fuzzer for frame info functions 
Add a fuzzer that fuzzes all helper functions that take compressed
input. This fuzzer caught one out of bounds read in
`ZSTD_decompressBound()`.
 2019-04-17 11:29:42 -07:00
Nick Terrell
450feb0f95 [libzstd] Fix ZSTD_decompressBound() on bad skippable frames 
The function didn't verify that the skippable frame size is correct.
 2019-04-17 11:29:42 -07:00
Nick Terrell
b85f7d7790
Merge pull request #1589 from terrelln/appveyorTest 
[libzstd] Require ZSTD_MULTITHREAD to create a ZSTDMT_CCtx
 2019-04-16 12:32:19 -07:00
Nick Terrell
a17fe4c9e5 [visual] Fix unreachable code warning 2019-04-16 11:32:35 -07:00
Yann Collet
41ad46f1a9
Merge pull request #1588 from terrelln/release-prep 
[fullbench] Fix speed measurements
 2019-04-16 10:11:28 -07:00
Nick Terrell
82d5a9870e [visual] Compile with ZSTD_MULTITHREAD=1 
This branch `appveyorTests` runs the tests that `master` runs, so
passing Appveyor means that the tests are fixed.
 2019-04-15 23:04:46 -07:00
Nick Terrell
de0499f7fa [libzstd] Require ZSTD_MULTITHREAD to create a ZSTDMT_CCtx 
ZSTDMT was broken when compiled without ZSTD_MULTITHREAD defined,
because `ZSTD_CCtx_setParameter(cctx, ZSTD_c_nbWorkers, nbWorkerss)`
failed. It was detected by the MSVC test which runs the fuzzer with
multithreading disabled.

This is a very niche use case of a deprecated API, because the API is
inefficient and synchronous, since `threading.h` will be synchronous.
Users almost certainly don't want this, and anyone who tested their code
should realize that it is broken. Therefore, I think it is safe to
require `ZSTD_MULTITHREAD` to be defined to use ZSTDMT.
 2019-04-15 23:04:46 -07:00
Nick Terrell
a48e32ca50 [fullbench] Fix measurements 2019-04-15 21:01:48 -07:00
Nick Terrell
a821aee3b1
Merge pull request #1587 from terrelln/release-prep 
[playTets] Don't pass status=none to dd
 2019-04-15 16:42:26 -07:00
Nick Terrell
6996804d05 [playTets] Don't pass status=none to dd 
The OS X test machine on TravisCI doesn't recognize the parameter
https://travis-ci.org/facebook/zstd/jobs/520517933.
 2019-04-15 16:21:46 -07:00
Nick Terrell
d86c7ad458
Merge pull request #1585 from terrelln/release-prep 
Prepare for the zstd-1.4.0 release
 2019-04-15 14:41:50 -07:00
Nick Terrell
bef61221a3 Update the changelog for zstd-1.4.0 2019-04-15 14:33:46 -07:00
Nick Terrell
203aac5bc6 Update benchmarks in README 2019-04-15 13:57:14 -07:00
Josh Soref
a880ca239b Spelling (#1582) 
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
 2019-04-12 11:18:11 -07:00
Nick Terrell
13fddac46d
Merge pull request #1583 from lzutao/meson 
Fix tests and meson build
 2019-04-12 11:13:20 -07:00
Lzu Tao
e31e7ca9ed travis: Make Mesonbuild non-optional 2019-04-12 23:43:34 +07:00
Lzu Tao
6c9b023f3e meson: Fix build 2019-04-12 21:28:36 +07:00
Lzu Tao
fb6901b2c0 tests: Add missing header timefn.h 2019-04-12 21:28:17 +07:00
Yann Collet
8ac2831f3d
Merge pull request #1581 from facebook/benchfn 
benchfn's reduced dependencies
 2019-04-11 14:23:04 -07:00
Yann Collet
1e01560b83 fixed timespec_get() initialization bug on some targets 
not sure why, but msan fires an "unitialized variable" error
when time gets properly initialized by timespec_get().
Maybe in some cases, not all bytes of the structure are initialized ?
Or maybe msan fails to detect the initialization ?

Anyway, pre-initializing the variable before passing it to timespec_get() works.
 2019-04-11 13:46:30 -07:00
Yann Collet
058da605cb fixed minor conversion warning 2019-04-11 12:25:27 -07:00
Yann Collet
f8e9bec73a fixed poolTests on Windows 
must use ZSTD_ prefix in front of pthread types
so that they get properly translated for Windows.
 2019-04-11 12:03:42 -07:00
Yann Collet
8f56fa2f58 fixed poolTests 
added poolTests to all
 2019-04-11 09:50:39 -07:00
Nick Terrell
9563fb4bfc
Merge pull request #1580 from terrelln/dict-fuzz 
[libzstd] Fix decompression dictionary bugs and clean up initialization
 2019-04-10 19:29:09 -07:00
Yann Collet
30c26ab726 fixed minor warning 
unused variable when assert() turned off in fileio.c
 2019-04-10 17:48:56 -07:00
Yann Collet
fbdd30d68e fixed cmake build script for test programs 2019-04-10 17:47:01 -07:00
Nick Terrell
aafe97b67d [libzstd] Switch dictUses to an enum 2019-04-10 16:50:35 -07:00
Yann Collet
526ec646b7 alternate static assert 
to circumvent Visual's C4804 warning
 2019-04-10 16:05:02 -07:00
Yann Collet
9703a59121 fixed minor conversion warning 2019-04-10 15:54:55 -07:00
Yann Collet
885476fb5b FreeBSD_11 specific fix 
C11 mandates the definition of timespec_get() and TIME_UTC.
However, FreeBSD11 announce C11 compliance, but does not provifr timespec_get(),
breaking link stage for benchfn.
Since it does not provide TIME_UTC either, which is also required by C11,
test this macro: this will automatically rule out FreeBSD 11 for this code path
(it will use the backup C90 path instead, based on clock_t).

The issue seeems fixed in FreeBSD 12.
 2019-04-10 15:22:18 -07:00
Yann Collet
2fa4f2e246 updated Visual projects 
added timefn
 2019-04-10 15:07:36 -07:00
Yann Collet
2c6b14ed22 fixed Windows header 
cmake build script: added timefn
 2019-04-10 14:54:13 -07:00
Yann Collet
3d346579d8 no more need for CLOCK_MONOTONIC 2019-04-10 14:16:39 -07:00
Yann Collet
36d2dfd846 moved C11 code path to timespec_get 2019-04-10 14:15:11 -07:00
Yann Collet
4765929271 fixed perror include 2019-04-10 14:04:11 -07:00
Yann Collet
70802cde6d fixed error message 
using stdlib's perror()
 2019-04-10 14:01:18 -07:00
Yann Collet
4b8185c7fc tried a blindfix for unix + c11 2019-04-10 13:26:27 -07:00
Nick Terrell
50b9c41196 [libzstd] Fix decompression dictionary bugs and clean up initialization 
Bugs:

* `ZSTD_DCtx_refPrefix()` didn't clear the dictionary after the first
  use. Fix and add a test case.
* `ZSTD_DCtx_reset()` always cleared the dictionary. Fix and add a test
  case.
* After calling `ZSTD_resetDStream()` you could no longer load a
  dictionary, since the stage was set to `zdss_loadHeader`. Fix and add
  a test case.

Cleanup:

* Make `ZSTD_initDStream*()` and `ZSTD_resetDStream()` wrap the new
 advanced API, and add test cases.
* Document the equivalent of these functions in the advanced API and
  document the unstable functions as deprecated.
 2019-04-10 12:59:02 -07:00
Yann Collet
59a7116cc2 benchfn dependencies reduced to only timefn 
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.

Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.

Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
 2019-04-10 12:37:03 -07:00
Yann Collet
094c000904 Merge branch 'dev' into benchfn 2019-04-10 11:57:05 -07:00