AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
5,446 Commits 1 Branch 0 Tags 27 MiB
3abfbee215
Commit Graph

1020 Commits

Author SHA1 Message Date
Yann Collet
6e490a2f09
Merge pull request #1237 from terrelln/init-cstream-adv 
Set requestedParams in ZSTD_initCStream*()
 2018-07-18 16:33:30 +02:00
cyan4973
53e1f0504e zstdmt debug traces compatibles with mingw 
since mingw does not have `sys/times.h`,
remove this path when detecting mingw compilation.
 2018-07-17 14:39:44 +02:00
Nick Terrell
6d222c437c Set requestedParams in ZSTD_initCStream*() 
The correct parameters are used once, but once `ZSTD_resetCStream()` is
called the default parameters (level 3) are used. Fix this by setting
`requestedParams` in the `ZSTD_initCStream*()` functions.

The added tests both fail before this patch and pass after.
 2018-07-12 18:35:55 -07:00
Yann Collet
4489daec09 slightly adjusted default-distribution threshold 
depending on strategy.
fast favors faster compression and decompression speeds.
 2018-06-26 20:10:45 -07:00
Nick Terrell
b426bcc097
[zstdmt] Fix jobsize bugs (#1205) 
[zstdmt] Fix jobsize bugs

* `ZSTDMT_serialState_reset()` should use `targetSectionSize`, not `jobSize` when sizing the seqstore.
  Add an assert that checks that we sized the seqstore using the right job size.
* `ZSTDMT_compressionJob()` should check if `rawSeqStore.seq == NULL`.
* `ZSTDMT_initCStream_internal()` should not adjust `mtctx->params.jobSize` (clamping to MIN/MAX is okay).
 2018-06-25 15:21:08 -07:00
Yann Collet
3b53bfe4f3
Merge pull request #1200 from felixhandte/zstd-attach-dict-pref 
Add CCtx Param Controlling Dict Attachment Behavior
 2018-06-25 12:42:31 -07:00
Yann Collet
3934e010a2
Merge pull request #1197 from facebook/poolResize 
Thread Pool resize
 2018-06-22 14:20:07 -07:00
Yann Collet
fbd5dfc1b1 changed POOL_resize() return type to int 
return is now just en error code.
This guarantee that `ctx` remains valid after POOL_resize().
Gets rid of internal POOL_free() operation.
 2018-06-22 12:14:59 -07:00
Yann Collet
1d5648ca10
Merge pull request #1196 from felixhandte/zstd-btopt-in-place-dict 
ZSTD_btopt: Support Searching the Dictionary Context In-Place
 2018-06-22 11:53:23 -07:00
Yann Collet
f6242d30b7
Merge pull request #1202 from facebook/barelyCompressible 
Increase threshold detection of poorly compressible data
 2018-06-22 11:52:52 -07:00
Yann Collet
698fd00afb huf: increase threshold detection of poorly compressible data 2018-06-21 18:32:38 -07:00
W. Felix Handte
01bb1c1016 Add CCtx Param Controlling Dict Attachment Behavior 2018-06-21 17:29:25 -04:00
W. Felix Handte
3e91dc4d6a Add Repcode Bounds Check 2018-06-21 15:54:41 -04:00
W. Felix Handte
5bd3d4b7d2 Add Debug Log Statement 2018-06-21 15:54:07 -04:00
W. Felix Handte
3caba150c6 Fix dmsBtLow Test 2018-06-21 15:53:40 -04:00
W. Felix Handte
5da9bbc38e Conceivably Dedup ZSTD_noDict and ZSTD_dictMatchState _insertBt1 Impls 
By reverting to the bool extDict flag, we call ZSTD_insertBt1 with the same
const args in both non-extDict dictModes.
 2018-06-21 11:20:01 -04:00
W. Felix Handte
5d81f71e83 Consistency in Guarding DMS-Only Variable Initializations 2018-06-20 16:54:53 -04:00
W. Felix Handte
9c14eafe3d Also Use matchLow for HC3 Match 2018-06-20 15:51:14 -04:00
W. Felix Handte
0a6cf7cd1d Minor Changes 2018-06-20 15:27:23 -04:00
W. Felix Handte
ae1f3898a2 Remove Dead(!) HC3 DMS Lookup 2018-06-20 15:27:12 -04:00
Yann Collet
93702a7a62
Merge pull request #1198 from facebook/msdebug 
made Visual Studio compatible with DEBUGLEVEL >= 2
 2018-06-20 12:26:31 -07:00
cyan4973
ae0b7ffa0a made Visual Studio compatible with DEBUGLEVEL >= 2 2018-06-20 09:45:02 -07:00
Yann Collet
166901dc72 reduced POOL_resize() restriction 
It's not necessary to ensure that no job is ongoing.
The pool is only expanded, existing threads are preserved.
In case of error, the only option is to return NULL and terminate the thread pool anyway.
 2018-06-19 18:07:18 -07:00
Yann Collet
066fbbfe1c make zstdmt resize its context 
when nbThreads change.

Technically, it only expands.
But when instructed to use less threads,
the thread pool will limit nb of concurrent threads.
 2018-06-19 17:28:56 -07:00
Yann Collet
6768cf53fd
Merge pull request #1190 from terrelln/ldm-adjust 
Adjust advanced parameters to source size
 2018-06-19 14:40:56 -07:00
W. Felix Handte
03c39c540b Fix Incorrect Param 2018-06-19 15:36:33 -04:00
W. Felix Handte
de639502aa Update Dict Attachment Cut-Offs 2018-06-19 15:36:13 -04:00
W. Felix Handte
f0a13bcd68 Make Sure Position 0 Gets Into the Tree 2018-06-19 15:10:06 -04:00
W. Felix Handte
87fe4788a3 Fix Compression Ratio Regression #1 2018-06-19 13:01:21 -04:00
W. Felix Handte
4bb79f9c55 Misc Changes 2018-06-19 13:01:21 -04:00
W. Felix Handte
2091f34e9e Find Proper Matches 2018-06-19 13:01:21 -04:00
W. Felix Handte
64348a15f1 Misc Fixes 2018-06-19 13:01:21 -04:00
W. Felix Handte
ade8586ce6 Find mls == 3 Matches 2018-06-19 13:01:21 -04:00
W. Felix Handte
ce743312e2 Fix Typo 2018-06-19 13:01:21 -04:00
W. Felix Handte
a075864756 Switch != ZSTD_extDict to == ZSTD_noDict 2018-06-19 13:01:21 -04:00
W. Felix Handte
1e03377bde Implement RepCode Check 2018-06-19 13:01:21 -04:00
W. Felix Handte
ccbf067973 Add _dictMatchState Functions 2018-06-19 13:01:21 -04:00
W. Felix Handte
d5d8240967 Convert extDict Flag to dictMode Enum 2018-06-19 13:01:21 -04:00
W. Felix Handte
93c3184d44 Attach Dicts when Using ZSTD_btopt and ZSTD_btultra 2018-06-19 13:01:21 -04:00
Nick Terrell
3841dbac84 Adjust advanced parameters to source size 
In the new advanced API, adjust the parameters even if they are explicitly
set. This mainly applies to the `windowLog`, and accordingly the `hashLog`
and `chainLog`, when the source size is known.
 2018-06-18 15:49:31 -07:00
Yann Collet
e30f13bde0
Merge pull request #1185 from felixhandte/zstd-btlazy-in-place-dict 
ZSTD_btlazy2: Support Searching the Dictionary Context In-Place
 2018-06-18 13:29:44 -07:00
Yann Collet
9698d2fb72
Merge pull request #1189 from facebook/hist 
histogram module
 2018-06-14 20:39:52 -04:00
Yann Collet
6901c94cd6 avoid duplicate code comments 
when a function is decribed in hist.h,
do not describe it again in hist.c
to avoid future doc synchronization issues.
 2018-06-14 19:47:05 -04:00
Yann Collet
a71513bec6
Merge pull request #1184 from facebook/debug 
Grouped debug functions into debug.h
 2018-06-14 16:21:53 -04:00
W. Felix Handte
0c654d22c8 Force Inline BtFindBestMatch 2018-06-14 14:54:39 -04:00
Yann Collet
2d76defbfe grouped all histogram functions into hist.c 
renamed functions with HIST_* prefix
 2018-06-13 19:49:31 -04:00
W. Felix Handte
0551de4b5a Search Dict for Matches 2018-06-13 16:06:28 -04:00
W. Felix Handte
ace9cfa950 Attach Dicts when Using ZSTD_btlazy2 2018-06-13 16:06:28 -04:00
Yann Collet
fa41bcc2c2 grouped debug functions into debug.h 
There were 2 competing set of debug functions
within zstd_internal.h and bitstream.h.
They were mostly duplicate, and required care to avoid messing with each other.

There is now a single implementation, shared by both.

Significant change :
The macro variable ZSTD_DEBUG does no longer exist,
it has been replaced by DEBUGLEVEL,
which required modifying several source files.
 2018-06-13 15:43:09 -04:00
W. Felix Handte
d53200a846 Fix Cast Warning 2018-06-13 14:58:36 -04:00