Commit Graph

5053 Commits

Author SHA1 Message Date
Baldur Karlsson
430a2fec19 Remove non-ASCII characters in header file comments
* Replaced a non-breaking space and an en dash with a plain space and
  a hyphen.
* This means the files are simple ASCII and less likely to run into
  codepage issues.
2018-03-13 20:05:53 +00:00
Yann Collet
6e388eeac2
Merge pull request #1042 from JesseTG/jtg/t0-printout
Made -H's printout specify the semantics of -T0
2018-03-12 18:28:34 -07:00
Yann Collet
530eeb41a7
Merge pull request #1039 from facebook/zstd_decompress
Removed zstd_decompress_impl.h
2018-03-12 18:21:46 -07:00
Jesse Talavera-Greenberg
2f70fbf2a3
Made -H's printout specify the semantics of -T0 2018-03-12 20:43:32 -04:00
Yann Collet
2291b85a1e changed ZSTD_p_literalCompression into ZSTD_p_compressLiterals
prefer verb+object construction
2018-03-12 11:44:10 -07:00
Yann Collet
a57d43d4d4 updated documentation of targetLength 2018-03-12 11:35:01 -07:00
Yann Collet
f24566b597 minor bench improvements
- do not test level 0, as it is converted into level 3,
  which feels strange when compressing multiple levels
- Use direct synchronous mode when a single worker is requested.
2018-03-12 04:02:57 -07:00
Yann Collet
6a9b41b731 create command --fast[=#]
access negative compression levels from command line
for both compression and benchmark modes.

also : ensure proper propagation of parameters
through ZSTD_compress_generic() interface.

added relevant cli tests.
2018-03-11 20:01:23 -07:00
Yann Collet
a146ee04ae added negative compression levels
negative compression level trade compression ratio for more compression speed.
They turn off huffman compression of literals,
and use row 0 as baseline with a stepSize = -cLevel.

added associated test in fuzzer

also added : new advanced parameter ZSTD_p_literalCompression
2018-03-11 05:21:53 -07:00
Yann Collet
facc09aa03 minor compression level adaptation
level 12 compresses slightly more and faster
due to better btlazy2 mode
2018-03-11 03:06:52 -07:00
Yann Collet
e0ba7ccdec
Merge pull request #1038 from HaydnTrigg/dev
Visual Studio 2017 build scripts
2018-03-10 10:38:30 -08:00
Haydn Trigg
c6351021e4 Visual Studio 2017 build scripts 2018-03-11 00:15:31 +10:30
Yann Collet
fe321f9e2a re-integrate ZSTD_decompressSequencesLong() into zstd_decompress.c
removed zstd_decompress_impl.h
2018-03-09 19:48:06 -08:00
Yann Collet
89a2ebb971 incorporated ZSTD_decompressSequences() into zstd_decompress() 2018-03-09 19:35:57 -08:00
Yann Collet
cdb1f1433e incorporated ZSTD_initFseState() inside zstd_decompress.c 2018-03-09 18:16:10 -08:00
Yann Collet
a166eae1ba incorporate ZSTD_decodeSequenceLong() within zstd_decompress.c 2018-03-09 18:11:14 -08:00
Yann Collet
17626ba56e restored ZSTD_decodeSequence() into zstd_decompress.c 2018-03-09 18:03:25 -08:00
Yann Collet
8c6dbf490b updated NEWS in preparation for v1.3.4 2018-03-09 16:22:34 -08:00
Yann Collet
5414b8ea01 Merge branch 'dev' of github.com:facebook/zstd into dev 2018-03-09 11:53:24 -08:00
Yann Collet
e916b9090e gen_html: changed CFLAGS for CXXFLAGS
since it's associated with $(CXX)
2018-03-09 11:52:14 -08:00
Yann Collet
51169575a8
Merge pull request #1036 from terrelln/thread-void
[threading] Cast unused arguments to void
2018-03-07 12:14:05 -08:00
Yann Collet
0379d83951
Merge pull request #1034 from facebook/longOffsetMode
Dynamic selection of long offset mode
2018-03-07 10:26:35 -08:00
Nick Terrell
7e103cdaf5 [threading] Cast unused arguments to void 2018-03-06 18:36:40 -08:00
Yann Collet
db147ea620 improved comments
following @terrelln suggestions
2018-03-06 18:15:26 -08:00
Yann Collet
51262bd832
Merge pull request #1033 from facebook/benchDecode
fix benchmark issue when measuring decoding speed only
2018-03-06 17:55:23 -08:00
Yann Collet
06ca9c7d7c fixed 0-seq blocks in block-decompression mode 2018-03-06 01:50:19 -08:00
Yann Collet
9a91afe6ef long offset mode : new default threshold for 32-bit 2018-03-05 16:41:08 -08:00
Yann Collet
7bd7a3ad43 long offset mode : new default threshold for 64-bits mode 2018-03-05 16:16:49 -08:00
Yann Collet
c0393a538f fixed counting long distance weights 2018-03-05 15:12:10 -08:00
Yann Collet
a70f7e10fa Merge branch 'benchDecode' into longOffsetMode 2018-03-05 14:09:00 -08:00
Yann Collet
03e7e14192 fix benchmark issue when measuring only decoding speed
zstd bench module can focus on decompression speed _only_.
This is useful when trying to measure performance
on large input data compressed using a high level
as compression time becomes problematic (too long).

This mode is triggered by command : zstd -b -d

Problem was : in such a mode,
measured decoding speed was > 10% slower
than in nominal mode (compression + decompression),
making decompression benchmark mode much less useful.

This patch fixes the issue.
It's not completely clear why, but
moving the `memcpy()` operation sooner in the pipeline fixed it.

I can still measure some difference, but it is in the < 2% range,
so it's much more tolerable.

also : it doesn't matter anymore in which order are selected
commands `-b` and `-d`.
The combination always triggers bench_decodeOnly mode.
2018-03-05 13:57:41 -08:00
Yann Collet
41bd10446e Merge branch 'dev' into longOffsetMode 2018-03-05 13:10:10 -08:00
Yann Collet
cb789d2df8 re-inserted offset evaluation 2018-03-05 13:08:59 -08:00
Yann Collet
99afe72576
Merge pull request #1032 from facebook/bmi2
Enable DYNAMIC_BMI2 for clang
2018-03-05 13:03:24 -08:00
Yann Collet
b91ddf0ae6 Merge branch 'dev' into longOffsetMode 2018-03-05 11:59:54 -08:00
Yann Collet
403741130d
Merge pull request #1029 from cemeyer/dev
FIO_addFInfo: Fully initialize output 'total' struct
2018-03-05 11:49:48 -08:00
Yann Collet
d02b44cf55 DYNAMIC_BMI2 enabled for clang
clang only claims compatibility with gcc 4.2.
Consequently, recent patch which reserved DYNAMIC_BMI2 for gcc >= 4.8
also disabled it for clang.

fix : __clang__ is now enough to enable DYNAMIC_BMI2
(associated with other existing conditions : x64/x64, !bmi2)
2018-03-04 16:05:59 -08:00
Yann Collet
3ba307b240
Merge pull request #1031 from facebook/inline48
force_inline HUF_decodeSymbol*()
2018-03-01 17:52:15 -08:00
Yann Collet
45b09e7625 limit DYNAMIC_BMI2 to gcc >= 4.8
attribute bmi2 not supported by gcc 4.4
2018-03-01 15:02:18 -08:00
Yann Collet
b01552a07a force inlining of HUF_decodeSymbol*() functions
which was not done properly by gcc 4.8
resulting in major performance difference.

ex :
zstd -b1 silesia.tar
before : dec 680 MB/s
after  : dec 710 MB/s  (without bmi2)
after  : dec 770 MB/s  (with DYNAMIC_BMI2)
2018-03-01 11:31:45 -08:00
Conrad Meyer
606374269c FIO_addFInfo: Fully initialize output 'total' struct
Silence a Coverity warning about 'windowSize' being uninitialized.
(Yes, nothing that calls this routine actually uses the windowSize
value.  Still, appeasing Coverity is pretty harmless in this case.)
2018-02-28 15:23:05 -08:00
Yann Collet
564cb1b640 update doc/README.md on provided tools to test 3rd party implementations 2018-02-27 17:37:05 -08:00
Yann Collet
ccb7184a76
Merge pull request #1026 from terrelln/lrm-window
LDM manages its own window round buffer
2018-02-27 17:09:10 -08:00
Nick Terrell
0a0e64c641 LDM manages its own window round buffer 2018-02-27 12:13:23 -08:00
Yann Collet
2c4d3f339a
Merge pull request #1025 from facebook/huf
Huf
2018-02-27 09:57:01 -08:00
Yann Collet
33a3f18848 fixed wrong size test 2018-02-26 18:27:51 -08:00
Yann Collet
d18d43aaf9
Merge pull request #1024 from terrelln/window-split
Split the window state into substructure
2018-02-26 17:18:33 -08:00
Yann Collet
89741653ab added error code workSpace_tooSmall 2018-02-26 15:11:50 -08:00
Yann Collet
6cdf690441 minor cleaning of huff0
Update code documentation, and properly names a few "magic constants".
Also, HUF_compress_internal() gets a cleaner way
to determine size of tables inside workspace.
2018-02-26 14:52:23 -08:00
Nick Terrell
6b88d592fd Reduce ZSTD_CHAINLOG_MAX to 29 in 32-bit mode 2018-02-26 13:30:24 -08:00