mgrice
5d89771529
fix warning: always_inline function might not be inlinable
2019-08-29 12:32:15 -07:00
mgrice
b830599582
Improvements in zstd decode performance
...
Summary: The idea behind wildcopy is that it can be cheaper to copy more bytes (say 8) than it is to copy less (say, 3). This change takes that further by exploiting some properties:
1. it's almost always OK to copy 16 bytes instead of 8, which means fewer copy instructions, and fewer branches
2. A 16 byte chunk size means that ~90% of wildcopy invocations will have a trip count of 1, so branch prediction will be improved.
Speedup on Xeon E5-2680v4 is in the range of 3-5%.
Measured wildcopy length distributions on silesia.tar:
level <=8 <=16 <=24 >24
1 78.05% 11.49% 3.52% 6.94%
3 82.14% 8.99% 2.44% 6.43%
6 85.81% 6.51% 2.92% 4.76%
8 83.02% 7.31% 3.64% 6.03%
10 84.13% 6.67% 3.29% 5.91%
15 77.58% 7.55% 5.21% 9.66%
16 80.07% 7.20% 3.98% 8.75%
Test Plan: benchmark silesia, make check
2019-08-29 12:25:56 -07:00
Yann Collet
d944197e79
Merge pull request #1754 from facebook/decTest
...
Tests for single-file decompressor
2019-08-29 12:09:09 -07:00
Yann Collet
64102f08da
Merge branch 'dev' into decTest
2019-08-29 09:48:12 -07:00
Yann Collet
ccb7c8e903
Merge pull request #1755 from cwoffenden/amalgamate-c99-fixes
...
C99 and older GCC fixes
2019-08-29 09:47:28 -07:00
Carl Woffenden
72e51ac246
C99 and older GCC fixes
2019-08-29 11:16:57 +02:00
Yann Collet
e7ad20c9dc
update test script
2019-08-28 13:46:18 -07:00
Yann Collet
4b3a8fe1c4
fix create_ script for sh
2019-08-28 13:23:48 -07:00
Yann Collet
9589e8e4bb
Merge pull request #1749 from facebook/rmadapt
...
removed adaptive-compression
2019-08-28 12:26:29 -07:00
Yann Collet
8af941d2d7
Merge branch 'dev' into decTest
2019-08-28 12:17:29 -07:00
Yann Collet
979ddc1271
Merge pull request #1750 from cwoffenden/amalgamate-bash2sh
...
Rewrote the scripts to use sh instead of bash
2019-08-28 12:14:29 -07:00
Carl Woffenden
cdf73e915e
Rewrote the scripts to sh instead of bash
2019-08-28 19:20:42 +02:00
Yann Collet
416157df80
try to debug issues on CirrusCI
2019-08-27 16:38:21 -07:00
Yann Collet
cd43f7d965
added single_file_decoder to contrib/ tests
2019-08-27 16:04:12 -07:00
Yann Collet
f61e8a231f
minor script renaming, for clarity
2019-08-27 16:01:39 -07:00
Yann Collet
517aeb89dc
changed contrib project name for clarity
2019-08-27 15:50:47 -07:00
Yann Collet
b12a5b9117
removed associated test
2019-08-27 14:52:16 -07:00
Yann Collet
5ed1b1e11d
removed adaptive-compression
...
the functionality is already integrated into `zstd` through `--adapt` command
2019-08-27 14:47:40 -07:00
Nick Terrell
e9c0fc12d2
Merge pull request #1748 from terrelln/cover-deadlock
...
[dictBuilder] Fix deadlock in *COVER error case
2019-08-27 10:17:28 -07:00
Yann Collet
a8391cc5a8
Merge pull request #1739 from cwoffenden/amalgamate
...
Tweaks to create a single-file decompressor
2019-08-27 10:05:26 -07:00
Carl Woffenden
51868964ef
Fixed test failure when Emscripten not present
2019-08-27 17:12:57 +02:00
Carl Woffenden
6213b7b3b4
Minor repetition
2019-08-27 16:57:23 +02:00
Carl Woffenden
59052d5fd8
Typo
2019-08-27 16:55:03 +02:00
Carl Woffenden
ec12721538
Added clarification
2019-08-27 15:53:26 +02:00
Carl Woffenden
6712a644fa
Added reasoning
2019-08-27 15:51:14 +02:00
Carl Woffenden
4f2a8b752a
Typo
2019-08-27 15:38:34 +02:00
Carl Woffenden
a57de4ac89
Added test script; tidied and documented
...
The test script combines the sources then builds and runs an example. A futher example is built if the Emscripten compiler is available on the system. Documentation covers building.
2019-08-27 15:36:06 +02:00
Nick Terrell
793c71d6ae
[test][dictBuilder] Add multithreaded tests
2019-08-26 18:32:08 -07:00
Nick Terrell
0932de54bc
[dictBuilder] Fix deadlock in *COVER error case
...
The COVER and FASTCOVER dictionary builders can deadlock when
dictionary construction errors, likely because there are too few
samples, or too few distinct dmers. The deadlock only occurs when
there are errors.
Fixes #1746 .
2019-08-26 18:19:29 -07:00
Carl Woffenden
7c6fa81579
Added Emscripten example, removed Buck, minor tidy
...
Work-in-progress. Added simple Emscripten WebGL example that adds 25kB when build with Zstd. Removed Buck (will replace). Minor correctness.
2019-08-26 21:28:19 +02:00
Carl Woffenden
ea8f6d2a07
Able to test combine script; minor tidy
2019-08-26 07:48:57 +02:00
Carl Woffenden
d760e35ebc
Preparing to run tests
...
Combine script more robust and can output to a specified file. Initial buck files added (work in progress).
2019-08-25 22:49:01 +02:00
Carl Woffenden
36a59336da
Minor fix for files with spaces. Typo.
2019-08-23 23:09:13 +02:00
Carl Woffenden
c690f22e96
Merge branch 'dev' into amalgamate
2019-08-23 23:05:02 +02:00
Carl Woffenden
5144e66095
Revert "Merge remote-tracking branch 'origin/master' into dev"
...
This reverts commit 0df29a4e5f69c875a0cc
2019-08-23 23:04:21 +02:00
Carl Woffenden
0fcaa675e0
Merge remote-tracking branch 'upstream/dev' into dev
2019-08-23 23:03:52 +02:00
Carl Woffenden
0df29a4e5f
Merge remote-tracking branch 'origin/master' into dev
2019-08-23 22:57:06 +02:00
Nick Terrell
d0750a1c9c
Merge pull request #1733 from nmagerko/size-hint
...
Add --size-hint=# option
2019-08-23 10:16:10 -07:00
Nick Terrell
5b811cb41d
Merge pull request #1745 from terrelln/regression-driver
...
[fuzz] Add a DEBUGLOG(3) statement to print file
2019-08-23 10:15:52 -07:00
Carl Woffenden
0a49353a46
Added generator script and simple test
...
The script will combine decompressor sources into a single file. The example shows this in use.
2019-08-23 18:43:29 +02:00
Nick Terrell
e2030a2c40
[fuzz] Add a DEBUGLOG(3) statement to print file
...
Enable it by building with this command:
```
./fuzz.py build all --debug 3
```
2019-08-22 17:27:15 -07:00
Nick Magerko
493f95c7df
Fix merge conflicts
2019-08-22 11:51:41 -07:00
Nick Terrell
a505463710
Merge pull request #1726 from nmagerko/stream-size
...
Add --stream-size=# option
2019-08-22 11:31:15 -07:00
Nick Magerko
fd486a846a
Differentiate --stream-size from --size-hint
2019-08-22 09:37:47 -07:00
Nick Magerko
2cdda8b3c4
Minor documentation update
2019-08-22 09:13:28 -07:00
Nick Magerko
b3540507f5
Remove bc from play tests
2019-08-21 10:27:54 -07:00
Nick Terrell
54ad33448c
Merge pull request #1737 from terrelln/legacy-fix
...
[legacy] Fix buffer overflow in v0.2 and v0.4 raw literals decompression
2019-08-21 10:10:24 -07:00
Nick Terrell
188311dd4a
Merge pull request #1736 from terrelln/fuzz-fix
...
[fuzz] Improve fuzzer build script and docs
2019-08-21 10:09:38 -07:00
Carl Woffenden
901ea61f83
Tweaks to create a single-file decoder
...
The CHECK_F macros differ slightly (but eventually do the same thing). Older GCC needs to fallback on the old-style pragma optimisation flags.
2019-08-21 17:49:17 +02:00
Yann Collet
69c875a0cc
Merge pull request #1724 from facebook/blockSize
...
clarifications on field `Block_Size`
2019-08-21 05:19:43 -07:00
