Commit Graph

7095 Commits

Author SHA1 Message Date
0x123456789A
26ffc18634
Introduce ZSTD_PROGRAMS_LINK_SHARED
The CMake variable ZSTD_PROGRAMS_LINK_SHARED indicactes wether or not to link the zstd programs dynamically or statically.
2019-10-29 14:31:02 +01:00
Nick Terrell
5c0bdde922
Merge pull request #1856 from terrelln/assert-fix
Fix assert in ZSTD_safecopy
2019-10-28 21:44:22 -07:00
Nick Terrell
9c1860861e Fix assert in ZSTD_safecopy
In the case that `op >= oend_w` it is possible that `diff < 8` because
the two buffers could be adjacent.

Credit to OSS-Fuzz, which found the bug. It isn't reproducible because
it depends on the memory layout.
2019-10-28 17:51:17 -07:00
Yann Collet
6d5e0f1c9f Merge branch 'dev' of github.com:facebook/zstd into dev 2019-10-28 12:57:23 -07:00
Yann Collet
faddd2aa1d updated CHANGELOG for v1.4.4 2019-10-28 12:57:05 -07:00
Felix Handte
01ec595b85
Merge pull request #1851 from felixhandte/pkg-config-prefix-fix
In pkg-config File, Derive Lib and Include Dir from Prefix at Use-Time
2019-10-28 14:24:56 -04:00
Yann Collet
74065da4c5 updated API inline doc and manual
regarding ZSTD_CDict created without a dictBuffer.
2019-10-28 11:15:41 -07:00
Yann Collet
b4037b18ef
Merge pull request #1848 from felixhandte/tzst-ext-tmp
Support Decompressing Short Tar Extensions
2019-10-28 10:08:20 -07:00
Yann Collet
66a96caf22
Merge pull request #1847 from facebook/zlibwrap_visual
fix zlibWrapper for Visual Studio
2019-10-25 17:42:42 -07:00
W. Felix Handte
74bd76c3ff In pkg-config File, Derive Lib and Include Dir from Prefix at Use-Time
Addresses #1794. Instead of deriving the lib dir and include dir at
build-time, let's do it like everyone else does at pkg-config run-time.

This has the disadvantage that we can no longer override LIBDIR and
INCLUDEDIR in the Makefile and have that reflected in the .pc file.
2019-10-25 15:07:31 -04:00
W. Felix Handte
4eccc82875 Minor Fixes 2019-10-25 13:59:19 -04:00
Yann Collet
c2140e9db0
Merge pull request #1845 from facebook/zbuff
improve deprecation warning macro
2019-10-25 09:59:00 -07:00
W. Felix Handte
24499036ba Add Tests 2019-10-24 20:19:10 -04:00
W. Felix Handte
91c3f545cc Fix Build; Refactor 2019-10-24 20:18:57 -04:00
Yann Collet
42a22af78b fix zlibWrapper for Visual Studio
As per https://github.com/facebook/zstd/issues/1800#issuecomment-545945050,
Visual does not support `ssize_t` type,
which is an issue for `gzread.c`.

Added a work around, suggested by @bluenlive

Note : I have not been able to confirm the problem,
so this is a blind fix.
This seems safe outside of Visual, since it is gated by _MSC_VER macro.
2019-10-24 15:19:05 -07:00
Felix Handte
30b27ee0c9
Merge pull request #1790 from stokito/tzst-ext
Add short tar's extensions .tgz (.tar.gz), .txz (.tar.xz), .tzst (.tar.zst)
2019-10-24 17:50:22 -04:00
Felix Handte
506e1a1176
Merge branch 'tzst-ext-tmp' into tzst-ext 2019-10-24 17:49:34 -04:00
Yann Collet
f93668c904
Merge pull request #1841 from bimbashrestha/travis
Adding arm64 make check to travis.ci
2019-10-24 12:58:50 -07:00
Yann Collet
37f90930da
Merge pull request #1839 from bimbashrestha/named_pipes
Allowing named pipes to go through in zstdcli
2019-10-24 10:53:12 -07:00
Yann Collet
a9a216a846
Merge pull request #1824 from senhuang42/new_path_for_cdict
Avoid using CDict params when input is large.
2019-10-23 12:04:40 -07:00
Yann Collet
63e435dda1 improve deprecation warning macro
fix #1488

although, curiously enough, I was never able to reproduce the issue
(according to the bug report, it should be present while using gcc 4.8).
2019-10-23 11:59:32 -07:00
Bimba Shrestha
4899b6fdbd Moving arm64 test outside the meson scope 2019-10-23 10:43:20 -07:00
Nick Terrell
3c8ff35013
Merge pull request #1840 from terrelln/regression-fix
[regression] Fix the old streaming regression test
2019-10-22 17:58:31 -07:00
Bimba Shrestha
5228c0718d Adding arm64 make check 2019-10-22 17:43:29 -07:00
Yann Collet
f966cd080a added documentation on DYNAMIC_BMI2 build macro 2019-10-22 17:43:09 -07:00
Bimba Shrestha
95f7880ddb
Merge pull request #4 from facebook/dev
Merging dev
2019-10-22 17:33:12 -07:00
Bimba Shrestha
10f7147013 Adding shell test and removing bash test file 2019-10-22 17:09:44 -07:00
Yann Collet
09d0b52998
Merge pull request #1838 from facebook/fixAdvanced
fix initCStream_advanced() for fast strategies
2019-10-22 16:22:55 -07:00
Bimba Shrestha
edae3c8f3c Adding bash test for named pipes 2019-10-22 16:15:59 -07:00
Nick Terrell
cf21003995 [regression] Fix the old streaming regression test
* A copy-paste error made it so we weren't running the advanced/cdict
  streaming tests with the old API.
* Clean up the old streaming tests to skip incompatible configs.
* Update `results.csv`.

The tests now catch the bug in #1787.
2019-10-22 15:43:17 -07:00
Bimba Shrestha
8a3974807e Allowing named pipes to go through zstdcli 2019-10-22 15:23:22 -07:00
Yann Collet
514f941a70
Merge pull request #1836 from facebook/dirFlat2
Added --output-dir-flat=
2019-10-22 15:19:04 -07:00
Yann Collet
5d5c895b18 fix initCStream_advanced() for fast strategies
Compression ratio of fast strategies (levels 1 & 2)
was seriously reduced, due to accidental disabling of Literals compression.

Credit to @QrczakMK, which perfectly described the issue, and implementation details,
making the fix straightforward.

Example : initCStream with level 1 on synthetic sample P50 :
Before : 5,273,976 bytes
After  : 3,154,678 bytes
ZSTD_compress (for comparison) : 3,154,550

Fix #1787.

To follow : refactor the test which was supposed to catch this issue (and failed)
2019-10-22 15:01:38 -07:00
Yann Collet
111b0c53b0 update documentation on deprecated functions
mostly : note that these functions will soon generate deprecation warnings
2019-10-22 13:51:18 -07:00
Nick Terrell
222ec7e6ec
Merge pull request #1837 from terrelln/magic-isnt-real
Fix ZSTD_f_zstd1_magicless for small data
2019-10-22 10:50:14 -07:00
Nick Terrell
b1ec94e63c Fix ZSTD_f_zstd1_magicless for small data
* Fix `ZSTD_FRAMEHEADERSIZE_PREFIX` and `ZSTD_FRAMEHEADERSIZE_MIN` to
  take a `format` parameter, so it is impossible to get the wrong size.
* Fix the places that called `ZSTD_FRAMEHEADERSIZE_PREFIX` without
  taking the format into account, which is now impossible by design.
* Call `ZSTD_frameHeaderSize_internal()` with `dctx->format`.
* The added tests catch both bugs in `ZSTD_decompressFrame()`.

Fixes #1813.
2019-10-21 21:16:17 -07:00
Sen Huang
c2e1e54f24 ((x or y) or z) == (x or y or z), remove brackets 2019-10-21 19:16:50 -04:00
Sen Huang
59c81aa31b Line up comments :) 2019-10-21 19:12:15 -04:00
Sen Huang
dbda8c318a Trailing comma 2019-10-21 19:10:13 -04:00
Sen Huang
0c00455ea6 Merge branch 'dev' of github.com:senhuang42/zstd into new_path_for_cdict 2019-10-21 19:06:51 -04:00
Sen Huang
5b2f4ac1a8 merge 2019-10-21 19:02:52 -04:00
Sen Huang
2ab484a5f9 Fix bad merge 2019-10-21 18:55:17 -04:00
Nick Terrell
919d1d8e93
Merge pull request #1831 from terrelln/zstdmt-bad-memset
[zstdmt] Don't memset the jobDescription
2019-10-21 15:53:57 -07:00
Sen Huang
b6c3459d50 merge 2019-10-21 18:46:17 -04:00
Yann Collet
989e0f0cee Added --output-dir-flat=
which is a preferred construction when applicable.

Also, added --output-dir-flat into man page.
2019-10-21 14:39:18 -07:00
Yann Collet
6cf04c0344
Merge pull request #1834 from facebook/winFix
Windows fixes
2019-10-21 13:45:17 -07:00
Yann Collet
bad35bd307 turned off zlib tests during msan
since the local zlib library is not msan-instrumented
2019-10-21 12:55:39 -07:00
Sen Huang
676f89902a Added multiplier, renamed new enum to something more useful 2019-10-21 15:36:12 -04:00
Sen Huang
1f3a51fb52 Updated forceAttachDict param bounds 2019-10-21 15:36:12 -04:00
Sen Huang
8f69c47643 Add enum to decision process 2019-10-21 15:36:12 -04:00