Commit Graph

7494 Commits

Author SHA1 Message Date
Yann Collet
88ecdc939b
Merge pull request #2080 from facebook/minor_fileio_refactor
minor refactor of `fileio.c`
2020-04-13 20:47:34 -07:00
Yann Collet
458a1a1723 minor refactor
- fix a few comments
- reorder some parameters, to enforce "mutable references first"
- simplified fwriteSparse()
2020-04-13 14:09:57 -07:00
Nick Terrell
5fcbc484c8
Merge pull request #2040 from caoyzh/dev-2
Optimize by prefetching on aarch64
2020-04-08 13:14:47 -07:00
Bimba Shrestha
c0d4b2b5a3
Merge pull request #2075 from bimbashrestha/dict_fuzzer_ref
[bug] handling case where prefix is NULL or 0 sized in refPrefix_advanced
2020-04-07 17:37:19 -05:00
Bimba Shrestha
1658ae75cd handling nil case for refprefix 2020-04-07 14:41:53 -07:00
Yann Collet
dde98d833d
Merge pull request #2065 from cwoffenden/single-file-lib
Single-file decoder script can now (optionally) create an encoder
2020-04-07 10:17:05 -07:00
Carl Woffenden
3eaa525138 Fixed clash when projects are already using xxHash
Undefing XXH_* macros allows the `.c` to build standalone without clashes. Removing `xxhash.c` and only including the header is the correct usage (according to `XXH_PRIVATE_API`).
2020-04-07 18:17:59 +02:00
Carl Woffenden
4e6027f8ca
Fixed-up single-file libs directory 2020-04-07 13:35:19 +02:00
Carl Woffenden
d01e7c2fad Renamed directory 2020-04-07 13:34:19 +02:00
Carl Woffenden
59812afac7 Improved documentation 2020-04-07 13:19:52 +02:00
Carl Woffenden
a93fadfcd9 Further replication removed
`CHECK_F` is now in `error_private.h`. Minor tidy.
2020-04-07 11:25:16 +02:00
Carl Woffenden
7af7735fa3 Merge remote-tracking branch 'upstream/dev' into single-file-lib 2020-04-07 11:13:02 +02:00
Carl Woffenden
edd9a07322 Code replicated in compression and decompression moved to shared headers
`CHECK_F` macro moved to `error_private.h` (shared between `fse_compress.c` and `fse_decompress.c`). `ZSTD_limitCopy()` moved to `zstd_internal.h` (shared between `zstd_compress.c` and `zstd_decompress.c`). Erroneous build artefact `zstd.h` removed from repo.
2020-04-07 11:02:06 +02:00
Bimba Shrestha
794f03459e adding refPrefix 2020-04-06 22:57:49 -07:00
Bimba Shrestha
2847d1a606
Merge pull request #2069 from bimbashrestha/doversize
[memory opt] Downsize decompression buffers when large for too long
2020-04-06 16:06:37 -05:00
Bimba Shrestha
31e76f1ed4 adding test for dctx size reduction 2020-04-04 08:49:24 -07:00
Bimba Shrestha
ebc422b0b7
Merge pull request #2068 from bimbashrestha/quick
[bug] Fixing Incorrect/inconsistent reported compression percentage for empty files
2020-04-03 19:28:42 -05:00
Bimba Shrestha
0154866749 moving consts to zstd_internal and reusing them 2020-04-03 14:26:15 -07:00
Bimba Shrestha
0a172c5e43 converting to if 2020-04-03 14:21:24 -07:00
Bimba Shrestha
3a4c8cc9b3 adding dctx to function name 2020-04-03 14:14:46 -07:00
Bimba Shrestha
ae47d50355 only computing sizes once 2020-04-03 14:12:23 -07:00
Bimba Shrestha
a4cbe79ccb Using in and out size together 2020-04-03 14:09:21 -07:00
Bimba Shrestha
009a92f749
Merge pull request #2067 from bimbashrestha/res-leak
[bug] adding fclose before return
2020-04-03 15:51:59 -05:00
Bimba Shrestha
1d267dc5d6 returning on null check 2020-04-03 13:39:02 -07:00
Bimba Shrestha
d598c88fb3 adding fclose before return 2020-04-03 13:31:47 -07:00
Bimba Shrestha
936aa63ff1 adding oversized check on decompression 2020-04-03 13:25:32 -07:00
Bimba Shrestha
05574ec141 adding oversizeDuration to dctx and macros 2020-04-03 13:08:29 -07:00
Bimba Shrestha
d0412f3aba no percentage on readsize == 0 2020-04-03 12:10:02 -07:00
Bimba Shrestha
bf1856c26f removing max(1, ..) 2020-04-03 12:07:17 -07:00
Carl Woffenden
88da79b42e Minor typo 2020-04-03 20:50:54 +02:00
Carl Woffenden
0442be5aa4 Added missing -pthread flag (was breaking on BSD) 2020-04-03 20:34:47 +02:00
Carl Woffenden
7c420344d2 Single-file decoder script can now (optionally) create an encoder
To complement the single-file decoder a new script was added to create an amalgamated single-file of all of the Zstd source, along with examples and (simple) tests.
2020-04-03 19:07:46 +02:00
Carl Woffenden
7202184ee0
Fixes decompressor when using -Wshorten-64-to-32 (#2062)
Spotted on iOS when building with `-Wshorten-64-to-32` (since `__builtin_expect` returns a `long`).
2020-04-03 02:55:29 -07:00
Nick Terrell
1665462573
Merge pull request #2054 from terrelln/license-fix
Standardize and fix copyright and licenses
2020-03-27 11:00:01 -07:00
Nick Terrell
8fda220d70
Merge pull request #2053 from terrelln/fix-play-test
[test] Fix playTests.sh with space in binary path
2020-03-26 21:08:10 -07:00
Nick Terrell
c7ae28a67d [circleci] Run test-license.py
Run it first because it is very fast (~1 second)
2020-03-26 20:13:16 -07:00
Nick Terrell
ef9e6fe227 [test] Fix playTests.sh with space in binary path
playTests.sh didn't work when `ZSTD_BIN` or `DATAGEN_BIN` had a space in
the path name. This happens for me because I split the cmake build
directory by compiler name, like "Clang 9.0.0".

The fix is to replace all instances of `$ZSTD` with the `zstd()`
function, and the replace `$DATAGEN` with `datagen()`. This will allow
us to change how we call zstd/datagen in the future without having to
change every callsite.
2020-03-26 19:52:19 -07:00
Nick Terrell
1f144351b7 [test] Add a test that checks for valid copyright and licenses
Tests all `.h`, `.c`, `.py`, and `Makefile` files for valid copyright
and license lines. Excludes a small number of exceptions (threading, and
divsufsort).

* Copyright does not contains `present`
* Copyright contains `Facebook, Inc`
* Copyright contains the current year
* License contains exactly the lines we expect
2020-03-26 17:02:09 -07:00
Nick Terrell
ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
Nick Terrell
f5029e285f
Merge pull request #2050 from terrelln/align
Align decompress sequences loop to 32+16 bytes
2020-03-24 11:42:59 -07:00
Nick Terrell
8d0ee37ac0 Align decompress sequences loop to 32+16 bytes
The alignment is added before the loop, so this shouldn't hurt
performance in any case. The only way it hurts is if there is already
performance instability, and we force it to be stable but in the bad
case.

This consistently gets us into the good case with gcc-{7,8,9} on an
Intel i9-9900K and clang-9. gcc-5 is 5% worse than its best case but has
stable performance. We get consistently good behavior on my Macbook Pro
compiled with both clang and gcc-8. It ends up in the 50% from DSB and
50% from MITE case, but the performance is the same as the 85% DSB case,
so thats fine.
2020-03-23 19:40:31 -07:00
Nick Terrell
d34204a7b7
Merge pull request #2029 from terrelln/minor-opt
[opt] Update repcodes less often
2020-03-23 18:12:32 -07:00
Bimba Shrestha
40414f984a
Merge pull request #2036 from bimbashrestha/edist
[contrib] Edit distance match finder
2020-03-23 12:53:44 -05:00
Bimba Shrestha
91a62bad47
Merge pull request #2047 from bimbashrestha/doc
[doc] targetLen -> targetLength
2020-03-23 08:27:54 -05:00
Bimba Shrestha
22e7123a9a targetLen -> targetLength 2020-03-23 06:25:08 -07:00
caoyzh
7201980650 Optimize by prefetching on aarch64 2020-03-14 15:25:59 +08:00
Bimba Shrestha
dbdeca1412 Edit distance match finder 2020-03-13 09:39:33 -05:00
Bimba Shrestha
66607d0eac
Merge pull request #2033 from bimbashrestha/icc
[opt] Small icc level 1 compression speed gain using #pragma vector
2020-03-10 20:42:19 -05:00
Bimba Shrestha
5c05d419e8
Merge pull request #2031 from bimbashrestha/bitscan
_BitScan{Reverse,Forward} add check for undefined
2020-03-10 16:22:25 -05:00
Bimba Shrestha
a89c45bdbd Typo 2020-03-10 15:19:48 -05:00