Commit Graph

7482 Commits

Author SHA1 Message Date
Carl Woffenden
a93fadfcd9 Further replication removed
`CHECK_F` is now in `error_private.h`. Minor tidy.
2020-04-07 11:25:16 +02:00
Carl Woffenden
7af7735fa3 Merge remote-tracking branch 'upstream/dev' into single-file-lib 2020-04-07 11:13:02 +02:00
Carl Woffenden
edd9a07322 Code replicated in compression and decompression moved to shared headers
`CHECK_F` macro moved to `error_private.h` (shared between `fse_compress.c` and `fse_decompress.c`). `ZSTD_limitCopy()` moved to `zstd_internal.h` (shared between `zstd_compress.c` and `zstd_decompress.c`). Erroneous build artefact `zstd.h` removed from repo.
2020-04-07 11:02:06 +02:00
Bimba Shrestha
2847d1a606
Merge pull request #2069 from bimbashrestha/doversize
[memory opt] Downsize decompression buffers when large for too long
2020-04-06 16:06:37 -05:00
Bimba Shrestha
31e76f1ed4 adding test for dctx size reduction 2020-04-04 08:49:24 -07:00
Bimba Shrestha
ebc422b0b7
Merge pull request #2068 from bimbashrestha/quick
[bug] Fixing Incorrect/inconsistent reported compression percentage for empty files
2020-04-03 19:28:42 -05:00
Bimba Shrestha
0154866749 moving consts to zstd_internal and reusing them 2020-04-03 14:26:15 -07:00
Bimba Shrestha
0a172c5e43 converting to if 2020-04-03 14:21:24 -07:00
Bimba Shrestha
3a4c8cc9b3 adding dctx to function name 2020-04-03 14:14:46 -07:00
Bimba Shrestha
ae47d50355 only computing sizes once 2020-04-03 14:12:23 -07:00
Bimba Shrestha
a4cbe79ccb Using in and out size together 2020-04-03 14:09:21 -07:00
Bimba Shrestha
009a92f749
Merge pull request #2067 from bimbashrestha/res-leak
[bug] adding fclose before return
2020-04-03 15:51:59 -05:00
Bimba Shrestha
1d267dc5d6 returning on null check 2020-04-03 13:39:02 -07:00
Bimba Shrestha
d598c88fb3 adding fclose before return 2020-04-03 13:31:47 -07:00
Bimba Shrestha
936aa63ff1 adding oversized check on decompression 2020-04-03 13:25:32 -07:00
Bimba Shrestha
05574ec141 adding oversizeDuration to dctx and macros 2020-04-03 13:08:29 -07:00
Bimba Shrestha
d0412f3aba no percentage on readsize == 0 2020-04-03 12:10:02 -07:00
Bimba Shrestha
bf1856c26f removing max(1, ..) 2020-04-03 12:07:17 -07:00
Carl Woffenden
88da79b42e Minor typo 2020-04-03 20:50:54 +02:00
Carl Woffenden
0442be5aa4 Added missing -pthread flag (was breaking on BSD) 2020-04-03 20:34:47 +02:00
Carl Woffenden
7c420344d2 Single-file decoder script can now (optionally) create an encoder
To complement the single-file decoder a new script was added to create an amalgamated single-file of all of the Zstd source, along with examples and (simple) tests.
2020-04-03 19:07:46 +02:00
Carl Woffenden
7202184ee0
Fixes decompressor when using -Wshorten-64-to-32 (#2062)
Spotted on iOS when building with `-Wshorten-64-to-32` (since `__builtin_expect` returns a `long`).
2020-04-03 02:55:29 -07:00
Nick Terrell
1665462573
Merge pull request #2054 from terrelln/license-fix
Standardize and fix copyright and licenses
2020-03-27 11:00:01 -07:00
Nick Terrell
8fda220d70
Merge pull request #2053 from terrelln/fix-play-test
[test] Fix playTests.sh with space in binary path
2020-03-26 21:08:10 -07:00
Nick Terrell
c7ae28a67d [circleci] Run test-license.py
Run it first because it is very fast (~1 second)
2020-03-26 20:13:16 -07:00
Nick Terrell
ef9e6fe227 [test] Fix playTests.sh with space in binary path
playTests.sh didn't work when `ZSTD_BIN` or `DATAGEN_BIN` had a space in
the path name. This happens for me because I split the cmake build
directory by compiler name, like "Clang 9.0.0".

The fix is to replace all instances of `$ZSTD` with the `zstd()`
function, and the replace `$DATAGEN` with `datagen()`. This will allow
us to change how we call zstd/datagen in the future without having to
change every callsite.
2020-03-26 19:52:19 -07:00
Nick Terrell
1f144351b7 [test] Add a test that checks for valid copyright and licenses
Tests all `.h`, `.c`, `.py`, and `Makefile` files for valid copyright
and license lines. Excludes a small number of exceptions (threading, and
divsufsort).

* Copyright does not contains `present`
* Copyright contains `Facebook, Inc`
* Copyright contains the current year
* License contains exactly the lines we expect
2020-03-26 17:02:09 -07:00
Nick Terrell
ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
Nick Terrell
f5029e285f
Merge pull request #2050 from terrelln/align
Align decompress sequences loop to 32+16 bytes
2020-03-24 11:42:59 -07:00
Nick Terrell
8d0ee37ac0 Align decompress sequences loop to 32+16 bytes
The alignment is added before the loop, so this shouldn't hurt
performance in any case. The only way it hurts is if there is already
performance instability, and we force it to be stable but in the bad
case.

This consistently gets us into the good case with gcc-{7,8,9} on an
Intel i9-9900K and clang-9. gcc-5 is 5% worse than its best case but has
stable performance. We get consistently good behavior on my Macbook Pro
compiled with both clang and gcc-8. It ends up in the 50% from DSB and
50% from MITE case, but the performance is the same as the 85% DSB case,
so thats fine.
2020-03-23 19:40:31 -07:00
Nick Terrell
d34204a7b7
Merge pull request #2029 from terrelln/minor-opt
[opt] Update repcodes less often
2020-03-23 18:12:32 -07:00
Bimba Shrestha
40414f984a
Merge pull request #2036 from bimbashrestha/edist
[contrib] Edit distance match finder
2020-03-23 12:53:44 -05:00
Bimba Shrestha
91a62bad47
Merge pull request #2047 from bimbashrestha/doc
[doc] targetLen -> targetLength
2020-03-23 08:27:54 -05:00
Bimba Shrestha
22e7123a9a targetLen -> targetLength 2020-03-23 06:25:08 -07:00
Bimba Shrestha
dbdeca1412 Edit distance match finder 2020-03-13 09:39:33 -05:00
Bimba Shrestha
66607d0eac
Merge pull request #2033 from bimbashrestha/icc
[opt] Small icc level 1 compression speed gain using #pragma vector
2020-03-10 20:42:19 -05:00
Bimba Shrestha
5c05d419e8
Merge pull request #2031 from bimbashrestha/bitscan
_BitScan{Reverse,Forward} add check for undefined
2020-03-10 16:22:25 -05:00
Bimba Shrestha
a89c45bdbd Typo 2020-03-10 15:19:48 -05:00
Bimba Shrestha
43fc88f443 Adding comment and remvoing ivdep 2020-03-10 14:57:27 -05:00
Bimba Shrestha
10f915f041
Merge pull request #2032 from bimbashrestha/no-size
Adding --[no-]content-size to the cli
2020-03-09 18:14:05 -05:00
Bimba Shrestha
9388dac2c0 Setting ZSTD_c_contentSizeFlag instead of UNKNOWN filesize 2020-03-09 15:40:18 -05:00
Bimba Shrestha
84d6bd4c7f Adding documentation 2020-03-09 14:50:49 -05:00
Bimba Shrestha
fae64b3390 Adding test for --[no-]content-size 2020-03-09 14:44:38 -05:00
Bimba Shrestha
43874730f4 Adding --content-size 2020-03-09 14:19:05 -05:00
Bimba Shrestha
6d8e761ba3 Inverting content size prefs bool 2020-03-09 14:16:10 -05:00
Bimba Shrestha
03ee18d9c5 Adding (void)noContentSize 2020-03-09 14:00:52 -05:00
Bimba Shrestha
167244a64f --no-content-size cli 2020-03-09 13:07:29 -05:00
Bimba Shrestha
dba3abc95a Missed returns 2020-03-05 12:20:59 -08:00
Bimba Shrestha
a75e5f2ffc bitscan add undef check 2020-03-05 11:52:15 -08:00
Bimba Shrestha
bc0e10ae78
Merge pull request #2030 from bimbashrestha/quick
Removing no-tree-vectorize for intel
2020-03-05 10:41:06 -08:00