Commit Graph

7675 Commits

Author SHA1 Message Date
Yann Collet
7ea2ae6649 added test linking user program to multi-threaded libzstd 2020-04-28 21:18:29 -07:00
Yann Collet
f77fd5ced0 generalized pattern rules 2020-04-28 18:43:55 -07:00
Yann Collet
c6ae2e83bc fix libzstd-mt underlinking issue
fix #2045
When compiling `libzstd` in multithreading mode,
the `libzstd-mt` recipe would not include `-pthread`,
resulting in an underlinked dynamic library.

Added a test on Travis to check that the library is fully linked.

This makes it possible, in some future release,
to build a multi-threaded `libzstd` dynamic library by default
as it would no longer impact the build script of user programs.
2020-04-28 18:29:20 -07:00
Nick Terrell
0ed07f6dfe
Merge pull request #2094 from terrelln/stable-dst
[lib] Add ZSTD_d_stableOutBuffer + fix single-pass mode for empty frames
2020-04-28 17:53:24 -07:00
Yann Collet
f17ac423b2 new tests created new artifacts
they were not properly ignored
2020-04-28 15:58:22 -07:00
Nick Terrell
55a57d46be Add extra warnings about not modifying the ZSTD_outBuffer 2020-04-28 12:07:42 -07:00
Nick Terrell
108a5572a5
Merge pull request #2048 from nocnokneo/ctest-support
Add CTest support
2020-04-28 11:01:13 -07:00
Nick Terrell
1343b815f8 [fuzz] Fuzz test ZSTD_d_stableOutBuffer 2020-04-27 20:04:04 -07:00
Nick Terrell
77a2945c43 Add some comments 2020-04-27 20:04:04 -07:00
Nick Terrell
f33de06c3e [lib] Fix single-pass mode for empty frames 2020-04-27 20:04:01 -07:00
Nick Terrell
a4ff217baf [lib] Add ZSTD_d_stableOutBuffer 2020-04-27 18:09:44 -07:00
Nick Terrell
b104f8e3eb [zstd] Fix typo in ZSTD_dParameter 2020-04-27 12:12:28 -07:00
Bimba Shrestha
6b4a3e019f
Merge pull request #2088 from bimbashrestha/bug
[bug] Making compressStream2 fail when passing rawContent but claiming fullDict
2020-04-23 14:16:56 -05:00
Bimba Shrestha
861a73744d
Merge pull request #2086 from bimbashrestha/quick3
[bug] passing cParams.hashLog instead of comprParams->hashLog for ldm trigger
2020-04-23 13:02:57 -05:00
Bimba Shrestha
1875f616ce passing dictContentType instead of rawContent every time 2020-04-21 22:29:35 -07:00
Bimba Shrestha
f7a7409a49 adding fail test when passing wrong fullDict using refPrefix 2020-04-21 22:26:48 -07:00
Bimba Shrestha
dba02245bf bash to shell conversion 2020-04-21 20:31:11 -07:00
Bimba Shrestha
659ff85cf5 changing to cParams.hashLog 2020-04-21 21:12:50 -05:00
Bimba Shrestha
0b107188b8 adding test for long mode trigger 2020-04-21 21:09:49 -05:00
Bimba Shrestha
38a6d2a8f2
Merge pull request #2085 from bimbashrestha/quick2
[doc-bug] Updating the recommended max chainlog for patch-from
2020-04-20 14:09:23 -05:00
Bimba Shrestha
b640802ff8 updating chainlog bound 2020-04-20 11:34:24 -07:00
Bimba Shrestha
c6311584aa
Merge pull request #2083 from bimbashrestha/quick1
[nit] Adding missing newline at end of patch-from recommendations
2020-04-20 13:24:34 -05:00
Bimba Shrestha
587a20a12a adding newline 2020-04-20 10:25:58 -07:00
Bimba Shrestha
5b0a452cac
Adding --long support for --patch-from (#1959)
* adding long support for patch-from

* adding refPrefix to dictionary_decompress

* adding refPrefix to dictionary_loader

* conversion nit

* triggering log mode on chainLog < fileLog and removing old threshold

* adding refPrefix to dictionary_round_trip

* adding docs

* adding enableldm + forceWindow test for dict

* separate patch-from logic into FIO_adjustParamsForPatchFromMode

* moving memLimit adjustment to outside ifdefs (need for decomp)

* removing refPrefix gate on dictionary_round_trip

* rebase on top of dev refPrefix change

* making sure refPrefx + ldm is < 1% of srcSize

* combining notes for patch-from

* moving memlimit logic inside fileio.c

* adding display for optimal parser and long mode trigger

* conversion nit

* fuzzer found heap-overflow fix

* another conversion nit

* moving FIO_adjustMemLimitForPatchFromMode outside ifndef

* making params immutable

* moving memLimit update before createDictBuffer call

* making maxSrcSize unsigned long long

* making dictSize and maxSrcSize params unsigned long long

* error on files larger than 4gb

* extend refPrefix test to include round trip

* conversion to size_t

* making sure ldm is at least 10x better

* removing break

* including zstd_compress_internal and removing redundant macros

* exposing ZSTD_cycleLog()

* using cycleLog instead of chainLog

* add some more docs about user optimizations

* formatting
2020-04-17 15:58:53 -05:00
Yann Collet
88ecdc939b
Merge pull request #2080 from facebook/minor_fileio_refactor
minor refactor of `fileio.c`
2020-04-13 20:47:34 -07:00
Yann Collet
458a1a1723 minor refactor
- fix a few comments
- reorder some parameters, to enforce "mutable references first"
- simplified fwriteSparse()
2020-04-13 14:09:57 -07:00
Taylor Braun-Jones
105d9b62ae Minor CMake refactoring/simplification 2020-04-09 17:16:12 -04:00
Taylor Braun-Jones
b674a1eec3 Fix that installation of the libzstd.pc does not respect DESTDIR installations 2020-04-09 17:16:12 -04:00
Taylor Braun-Jones
c22551b903 Generate a better CMake Package Config file 2020-04-09 17:16:12 -04:00
Nick Terrell
5fcbc484c8
Merge pull request #2040 from caoyzh/dev-2
Optimize by prefetching on aarch64
2020-04-08 13:14:47 -07:00
Bimba Shrestha
c0d4b2b5a3
Merge pull request #2075 from bimbashrestha/dict_fuzzer_ref
[bug] handling case where prefix is NULL or 0 sized in refPrefix_advanced
2020-04-07 17:37:19 -05:00
Bimba Shrestha
1658ae75cd handling nil case for refprefix 2020-04-07 14:41:53 -07:00
Yann Collet
dde98d833d
Merge pull request #2065 from cwoffenden/single-file-lib
Single-file decoder script can now (optionally) create an encoder
2020-04-07 10:17:05 -07:00
Carl Woffenden
3eaa525138 Fixed clash when projects are already using xxHash
Undefing XXH_* macros allows the `.c` to build standalone without clashes. Removing `xxhash.c` and only including the header is the correct usage (according to `XXH_PRIVATE_API`).
2020-04-07 18:17:59 +02:00
Carl Woffenden
4e6027f8ca
Fixed-up single-file libs directory 2020-04-07 13:35:19 +02:00
Carl Woffenden
d01e7c2fad Renamed directory 2020-04-07 13:34:19 +02:00
Carl Woffenden
59812afac7 Improved documentation 2020-04-07 13:19:52 +02:00
Carl Woffenden
a93fadfcd9 Further replication removed
`CHECK_F` is now in `error_private.h`. Minor tidy.
2020-04-07 11:25:16 +02:00
Carl Woffenden
7af7735fa3 Merge remote-tracking branch 'upstream/dev' into single-file-lib 2020-04-07 11:13:02 +02:00
Carl Woffenden
edd9a07322 Code replicated in compression and decompression moved to shared headers
`CHECK_F` macro moved to `error_private.h` (shared between `fse_compress.c` and `fse_decompress.c`). `ZSTD_limitCopy()` moved to `zstd_internal.h` (shared between `zstd_compress.c` and `zstd_decompress.c`). Erroneous build artefact `zstd.h` removed from repo.
2020-04-07 11:02:06 +02:00
Bimba Shrestha
794f03459e adding refPrefix 2020-04-06 22:57:49 -07:00
Bimba Shrestha
2847d1a606
Merge pull request #2069 from bimbashrestha/doversize
[memory opt] Downsize decompression buffers when large for too long
2020-04-06 16:06:37 -05:00
Bimba Shrestha
31e76f1ed4 adding test for dctx size reduction 2020-04-04 08:49:24 -07:00
Bimba Shrestha
ebc422b0b7
Merge pull request #2068 from bimbashrestha/quick
[bug] Fixing Incorrect/inconsistent reported compression percentage for empty files
2020-04-03 19:28:42 -05:00
Bimba Shrestha
0154866749 moving consts to zstd_internal and reusing them 2020-04-03 14:26:15 -07:00
Bimba Shrestha
0a172c5e43 converting to if 2020-04-03 14:21:24 -07:00
Bimba Shrestha
3a4c8cc9b3 adding dctx to function name 2020-04-03 14:14:46 -07:00
Bimba Shrestha
ae47d50355 only computing sizes once 2020-04-03 14:12:23 -07:00
Bimba Shrestha
a4cbe79ccb Using in and out size together 2020-04-03 14:09:21 -07:00
Bimba Shrestha
009a92f749
Merge pull request #2067 from bimbashrestha/res-leak
[bug] adding fclose before return
2020-04-03 15:51:59 -05:00