Commit Graph

2738 Commits

Author SHA1 Message Date
Soojin Nam
6e18d33122 original size unknown 2017-02-21 09:51:40 +09:00
Yann Collet
3b89eb0c20 updated NEWS with meson build by Dima Krasner 2017-02-20 01:26:05 -08:00
Yann Collet
4dca56ed83 Merge pull request #551 from dimkr/meson
Added a Meson-based build system
2017-02-20 01:23:51 -08:00
Dima Krasner
107c9a4e42 Moved to contrib 2017-02-18 23:30:57 +02:00
Dima Krasner
4c05b09f27 Added a message when multhread=true 2017-02-17 12:32:16 +02:00
Dima Krasner
da145123c5 Updated the README 2017-02-17 12:29:23 +02:00
Dima Krasner
faaded1976 Added multi-threaded library support 2017-02-17 12:27:13 +02:00
Yann Collet
f2da1b1f22 Merge pull request #550 from iburinoc/matchoverlenwrite
execSequence copies up to 2*WILDCOPY_OVERLENGTH extra
2017-02-16 18:14:47 -08:00
Yann Collet
ff176d3208 Merge pull request #548 from inikep/gz_compress
Circle CI tests
2017-02-16 14:57:59 -08:00
Sean Purcell
0ed3901b05 Update overlength match test case 2017-02-16 13:36:57 -08:00
Sean Purcell
6b010dec80 execSequence copies up to 2*WILDCOPY_OVERLENGTH extra 2017-02-16 12:05:40 -08:00
Przemyslaw Skibinski
7a8811f3f5 circle.yml: make travis-install 2017-02-16 19:04:22 +01:00
Przemyslaw Skibinski
6f508421eb faster start of containers 2017-02-16 18:45:17 +01:00
Dima Krasner
c1f25eaec0 Added a Meson project 2017-02-16 19:37:08 +02:00
Przemyslaw Skibinski
40dadd65cc join tests into pairs 2017-02-16 18:19:36 +01:00
Przemyslaw Skibinski
6babbff58d move MOREFLAGS to circle.yml 2017-02-16 17:52:49 +01:00
Yann Collet
3ebec8383b Merge pull request #549 from iburinoc/matchoverlenwrite
Fix wildcopy overwriting data still in window
2017-02-16 08:39:52 -08:00
Przemyslaw Skibinski
84452ca29f more balanced tests 2017-02-16 16:27:40 +01:00
Przemyslaw Skibinski
21d9022b88 two groups of tests 2017-02-16 15:59:00 +01:00
Przemyslaw Skibinski
6b64abb287 improved clangtest 2017-02-16 15:28:08 +01:00
Przemyslaw Skibinski
9a0161d376 imporved test-zstd 2017-02-16 15:13:33 +01:00
Przemyslaw Skibinski
fa492a3eca Tests for thread 1 (when CIRCLE_NODE_TOTAL=1) or thread 2 2017-02-16 14:39:21 +01:00
Przemyslaw Skibinski
cb76944861 final colon 2017-02-16 13:51:21 +01:00
Przemyslaw Skibinski
d3ff834562 check CIRCLE_NODE_TOTAL 2017-02-16 13:45:40 +01:00
Przemyslaw Skibinski
9e97a8a45a check $CIRCLE_NODE_INDEX 2017-02-16 13:36:12 +01:00
Przemyslaw Skibinski
f8a5749c22 circle.yml: run only short tests 2017-02-16 13:08:30 +01:00
Przemyslaw Skibinski
b0511aeca5 fix travis.yml 2017-02-16 12:33:25 +01:00
Przemyslaw Skibinski
e0d2a146d1 .travis.yml: detect "$TRAVIS_EVENT_TYPE" = "cron" 2017-02-16 12:29:08 +01:00
Sean Purcell
887eaa9e21 Fix wildcopy overwriting data still in window 2017-02-15 16:43:45 -08:00
Yann Collet
2252d29a5a Merge branch 'dev' of github.com:facebook/zstd into dev 2017-02-15 12:00:50 -08:00
Yann Collet
4596037042 updated fse version
feature minor refactoring (removing FSE_abs())
also : fix a few minor issues recently introduced in examples
2017-02-15 12:00:03 -08:00
Yann Collet
44f82d781f Merge pull request #545 from terrelln/force-window
[zstdmt] Fix MSAN failure with ZSTD_p_forceWindow
2017-02-15 10:20:15 -08:00
Przemyslaw Skibinski
90e5412a4e added -I/usr/include/x86_64-linux-gnu for asan32 2017-02-14 23:30:23 +01:00
Yann Collet
f0b9a8dddb Merge pull request #547 from inikep/dev11
Avoid fseek()'s 2GiB barrier with MacOS and *BSD
2017-02-14 12:29:00 -08:00
Przemyslaw Skibinski
9b5a1e9d97 added circle.yml 2017-02-14 20:06:41 +01:00
Przemyslaw Skibinski
226a6387a9 Merge remote-tracking branch 'refs/remotes/facebook/dev' into gz_compress 2017-02-14 20:04:56 +01:00
Yann Collet
c09d16ba8c preset behavior for gzip, gunzip and gzcat
when zstd is called through a link named gzip, gunzip or gzcat,
provides the same behavior as the related program.
gzip compresses using --format=gz
both gzip and gunzip enable --rm by default
2017-02-14 10:45:19 -08:00
Yann Collet
aca067f19d Merge pull request #543 from inikep/gz_compress
Gz compress
2017-02-14 10:30:22 -08:00
Nick Terrell
74b81ada25 Don't run test-pool with QEMU
> make test -n
    ...
    ./pool
    > make test -n QEMU_SYS=valgrind
    ...
    ./legacy
    # ./pool not run
2017-02-14 10:08:18 -08:00
Yann Collet
9696bfc2ad Merge pull request #544 from ds77/avoid-empty
Portable way to avoid empty unit warning in threading.c
2017-02-14 00:54:55 -08:00
Przemyslaw Skibinski
ce13d087d9 fix LONG_SEEK 2017-02-14 09:52:52 +01:00
Yann Collet
1decff24b8 Merge pull request #546 from zefanxu/dev
fix memory leak
2017-02-14 00:49:06 -08:00
Przemyslaw Skibinski
970419535f fixed function name (2) 2017-02-14 09:47:29 +01:00
Przemyslaw Skibinski
bf336572bf Avoid fseek()'s 2GiB barrier with MacOS and *BSD 2017-02-14 09:45:33 +01:00
Przemyslaw Skibinski
abd6302423 Windows resources updated to v1.1.4 2017-02-14 09:39:09 +01:00
Przemyslaw Skibinski
442c75f132 removed UTIL_doesFileExists (replaced with UTIL_isRegFile) 2017-02-14 09:38:51 +01:00
Przemyslaw Skibinski
b876b96ce1 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2017-02-14 09:26:03 +01:00
Przemyslaw Skibinski
98509a70ac fixed function name 2017-02-14 09:23:32 +01:00
zefanxu2
2bb6fc2a94 fix memory leak 2017-02-13 21:12:59 -06:00
Nick Terrell
ecf90ca24b [zstdmt] Fix MSAN failure with ZSTD_p_forceWindow
Reproduction steps:

```
make zstreamtest CC=clang CFLAGS="-O3 -g -fsanitize=memory -fsanitize-memory-track-origins"
./zstreamtest -vv -t4178 -i4178 -s4531
```

How to get to the error in gdb (may be a more efficient way):

* 2 breaks at zstd_compress.c:2418  -- in ZSTD_compressContinue_internal()
* 2 breaks at zstd_compress.c:2276  -- in ZSTD_compressBlock_internal()
* 1 break at zstd_compress.c:1547

Why the error occurred:

When `zc->forceWindow == 1`, after calling `ZSTD_loadDictionaryContent()` we
have `zc->loadedDictEnd == zc->nextToUpdate == 0`. But, we've really loaded up
to `iend` into the dictionary. Then in `ZSTD_compressBlock_internal()` we see
that `current > zc->nextToUpdate + 384`, so we load the last 192 bytes a second
time. In this case the bytes we are loading are a block of all 0s, starting in
the previous block. So when we are loading the last 192 bytes, we find a `match`
in the future, 183 bytes beyond `ip`. Since the block is all 0s, the match
extends to the end of the block. But in `ZSTD_count()` we only check that
`pIn < pInLoopLimit`, but since `pMatch > pIn`, `pMatch` eventually points past
the end of the buffer, causing the MSAN failure.

The fix:

The line changed sets sets `zc->nextToUpdate` to the end of the dictionary.
This is the behavior that existed before `ZSTD_p_forceWindow` was introduced.
This fixes the exposing test case. Since the code doesn't fail without
`zc->forceWindow`, it makes sense that this works. I've run the command
`./zstreamtest -T2mn` 64 times without failures. CI should also verify nothing
obvious broke.
2017-02-13 19:11:22 -08:00