AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
5,141 Commits 1 Branch 0 Tags 27 MiB
b2496ab606
Commit Graph

5141 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
George Lu
b2496ab606 Partial compilation test? 2018-06-05 13:24:00 -07:00
George Lu
b3ef314830 Fix Typos 2018-06-04 17:19:06 -07:00
George Lu
609d72b0ca Added Deprecated Dependencies 2018-06-04 14:33:21 -07:00
George Lu
9437021d2f Remove old file declaration 2018-06-04 13:32:41 -07:00
George Lu
6a617d70ed Documentation 2018-06-04 09:56:37 -07:00
George Lu
65de25a463 Created Macros 2018-06-04 09:56:29 -07:00
Yann Collet
ac4f7ead3b
Merge pull request #1154 from facebook/altOpt 
minor optimal parser optimization
 2018-05-30 13:05:51 -07:00
Yann Collet
809f2f9322 minor update of literal cost function 
just assert() there is no negative cost evaluation for literals
 2018-05-29 15:34:50 -07:00
Yann Collet
463a0fe38b simplified optimal parser 
removed "cached" structure.
prices are now saved in the optimal table.

Primarily done for simplification.
Might improve speed by a little.
But actually, and surprisingly, also improves ratio in some circumstances.
 2018-05-29 14:07:25 -07:00
Yann Collet
bb6eaf6495
Merge pull request #1153 from facebook/dynThreshold 
changed dynamic fse threshold for offset
 2018-05-26 08:43:45 -07:00
Yann Collet
e916c365a1 fixed minor visual warning 2018-05-25 20:43:09 -07:00
Yann Collet
c6ccf435ae
Merge pull request #1152 from facebook/lowCompression 
btultra accepts blocks with poorer compression ratio
 2018-05-25 17:46:11 -07:00
Yann Collet
a7fdceeccd changed dynamic fse threshold for offset 
recent experienced showed that
default distribution table for offset
can get it wrong pretty quickly with the nb of symbols,
while it remains a reasonable choice much longer for lengths symbols.

Changed the formula,
so that dynamic threshold is now 32 symbols for offsets.
It remains at 64 symbols for lengths.

Detection based on defaultNormLog
 2018-05-25 17:41:16 -07:00
Yann Collet
4b3a36d5d8 Merge branch 'dev' into lowCompression 2018-05-25 15:45:03 -07:00
Yann Collet
744f36aac4
Merge pull request #1141 from facebook/staticDictCost 
Random stuff on High Compression mode
 2018-05-25 15:43:32 -07:00
Yann Collet
5f177f1c53 btultra accepts blocks with poorer compression ratio 
zstd rejects blocks which do not compress by at least a certain amount.
In which case, such block is simply emitted uncompressed (even if a little bit of compression could be achieved).
This is better for decompression speed, hence for energy.

The logic is controlled by ZSTD_minGain().
The rule is applied uniformly, at all compression levels.

This change makes btultra accepts blocks with poor compression ratios.
We presume that users of btultra mode prefers compression ratio over some decompress speed gains.

The threshold for minimum gain is lowered for btultra
from s>>6 (~1.5% minimum gain)
to s>>7 (~0.8% minimum gain).

This is a prudent change.
Not sure if it's large enough.
 2018-05-25 15:19:52 -07:00
Yann Collet
e2c0e3d437 slightly nudge choices towards less sequences 
also slightly improve some strange detrimental corner cases.
 2018-05-25 14:52:21 -07:00
Yann Collet
f6ad59ab5c Merge branch 'dev' into staticDictCost 2018-05-24 16:21:02 -07:00
Yann Collet
a7c75740b3
Merge pull request #1149 from terrelln/fuzz-py 
Small fixes to fuzz.py
 2018-05-24 15:56:09 -07:00
Nick Terrell
fdd4d8510f Improve compiler detection to work on Mac 2018-05-24 14:21:12 -07:00
Nick Terrell
ac852abb8b Define BIT_DEBUG for --debug 2018-05-24 14:21:12 -07:00
Nick Terrell
2a9975f77b Increase the maximum file size 2018-05-24 14:21:12 -07:00
Nick Terrell
e712a3a0a3 Small fixes to fuzz.py 2018-05-24 14:21:12 -07:00
Yann Collet
47d2331ed8
Merge pull request #1150 from facebook/fracFse 
fix corner case when requiring cost of an FSE symbol
 2018-05-24 14:19:30 -07:00
Yann Collet
b5ef32fea7 Merge branch 'dev' into fracFse 2018-05-24 14:09:49 -07:00
Yann Collet
776128d16f fix corner case when requiring cost of an FSE symbol 
ensure that, when frequency[symbol]==0,
result is (tableLog + 1) bits
with both upper-bit and fractional-bit estimates.

Also : enable BIT_DEBUG in /tests
 2018-05-24 13:59:11 -07:00
Yann Collet
08c5be5db3
Merge pull request #1117 from felixhandte/zstd-fast-in-place-dict 
ZSTD_fast: Support Searching the Dictionary Context In-Place
 2018-05-23 19:32:25 -07:00
Nick Terrell
06b70179da
Work around bug in zstd decoder (#1147) 
Work around bug in zstd decoder

Pull request #1144 exercised a new path in the zstd decoder that proved to
be buggy. Avoid the extremely rare bug by emitting an uncompressed block.
 2018-05-23 18:02:30 -07:00
Yann Collet
712318a244
Merge pull request #1146 from terrelln/fse-fix 
[zstd] Fix decompression edge case
 2018-05-23 16:41:42 -07:00
Nick Terrell
f2d0924b87 Variable declarations 2018-05-23 14:58:58 -07:00
W. Felix Handte
d9c7e67125 Assert that Dict and Current Window are Adjacent in Index Space 2018-05-23 17:53:03 -04:00
W. Felix Handte
298d24fa57 Make loadedDictEnd an Index, not the Dict Len 2018-05-23 17:53:03 -04:00
W. Felix Handte
7ef85e0618 Fixes in re Comments 2018-05-23 17:53:03 -04:00
W. Felix Handte
582b7f85ed Don't Attach Empty Dict Contents 
In weird corner cases, they produce unexpected results...
 2018-05-23 17:53:03 -04:00
W. Felix Handte
9c92223468 Avoid Undefined Behavior in Match Ptr Calculation 2018-05-23 17:53:03 -04:00
W. Felix Handte
a44ab3b475 Remove Out-of-Date Comment 2018-05-23 17:53:03 -04:00
W. Felix Handte
95bdf20a87 Moar Renames 2018-05-23 17:53:03 -04:00
W. Felix Handte
7e0402e738 Also Attach Dict When Source Size is Unknown 2018-05-23 17:53:03 -04:00
W. Felix Handte
3ba70cc759 Clear the Dictionary When Sliding the Window 2018-05-23 17:53:03 -04:00
W. Felix Handte
b05ae9b608 Refine ip Initialization to Avoid ARM Weirdness 2018-05-23 17:53:03 -04:00
W. Felix Handte
1a7b34ef28 Use New Index Invariant to Simplify Conditionals 2018-05-23 17:53:03 -04:00
W. Felix Handte
2d598e6fed Force Working Context Indices Greater than Dict Indices 2018-05-23 17:53:03 -04:00
W. Felix Handte
d005e5daf4 Whitespace Fix 2018-05-23 17:53:03 -04:00
W. Felix Handte
154eb09419 Switch to Original Match Calc for noDict Repcode Check 2018-05-23 17:53:03 -04:00
W. Felix Handte
191fc74a51 Rename 'hasDict' to 'dictMode' 2018-05-23 17:53:03 -04:00
W. Felix Handte
ae4fcf7816 Respond to PR Comments; Formatting/Style/Lint Fixes 2018-05-23 17:53:03 -04:00
W. Felix Handte
ca26cecc7a Rename and Reformat 2018-05-23 17:53:03 -04:00
W. Felix Handte
66bc1ca641 Change Cut-Off to 8 KB 2018-05-23 17:53:03 -04:00
W. Felix Handte
c31ee3c7f8 Fix Rep Code Initialization 2018-05-23 17:53:03 -04:00
W. Felix Handte
b67196f30d Coalesce hasDictMatchState and extDict Checks into One Enum and Rename Stuff 2018-05-23 17:53:03 -04:00