AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
5,525 Commits 1 Branch 0 Tags 27 MiB
b3544217b7
Commit Graph

5525 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet
7639db939f updated Zstandard frame format 
adding clarifications from IETF RFC DISCUSS.
 2018-06-21 17:55:55 -07:00
George Lu
ab26f24c9c benchFunction Timed Wrappers 
Add BMK_benchFunctionTimed
Add BMK_init_customResultCont..
Change benchMem to use benchFunctionTimed
Minor Fixes/Adjustments
 2018-06-21 16:23:55 -07:00
George Lu
a8eea99ebe Incremental Display + Fn Separations 
Seperate syntheticTest and fileTableTest (now renamed as benchFiles)
Add incremental display to benchMem
Change to only iterMode for benchFunction
Make Synthetic test's compressibility configurable from cli (using -P#)
 2018-06-21 16:23:18 -07:00
Yann Collet
818e72b4d5 added extended POOL test 
abrupt end + downsizing with running jobs remaining in queue.

also : POOL_resize() requires numThreads >= 1
 2018-06-21 14:58:59 -07:00
W. Felix Handte
01bb1c1016 Add CCtx Param Controlling Dict Attachment Behavior 2018-06-21 17:29:25 -04:00
W. Felix Handte
3e91dc4d6a Add Repcode Bounds Check 2018-06-21 15:54:41 -04:00
W. Felix Handte
5bd3d4b7d2 Add Debug Log Statement 2018-06-21 15:54:07 -04:00
W. Felix Handte
3caba150c6 Fix dmsBtLow Test 2018-06-21 15:53:40 -04:00
Yann Collet
7d80ada5ca added a test for POOL (multithreading) 
ensuring all jobs in queue are nonetheless completed
when POOL is instructed to end abruptly (POOL_free())
 2018-06-21 12:24:36 -07:00
W. Felix Handte
5da9bbc38e Conceivably Dedup ZSTD_noDict and ZSTD_dictMatchState _insertBt1 Impls 
By reverting to the bool extDict flag, we call ZSTD_insertBt1 with the same
const args in both non-extDict dictModes.
 2018-06-21 11:20:01 -04:00
Yann Collet
6de249c1c6 fixed: bug when counting nb of active threads 
when queueSize > 1

also : added a test in testpool.c
       verifying resizing is effective.
 2018-06-20 18:28:49 -07:00
Yann Collet
6b48eb12c0 change control of threadLimit 
now limits maximum nb of active threads
even when queueSize > 1.
 2018-06-20 14:35:39 -07:00
Jennifer Liu
a3cb97b038 set up sample config.yml 2018-06-20 14:23:14 -07:00
W. Felix Handte
5d81f71e83 Consistency in Guarding DMS-Only Variable Initializations 2018-06-20 16:54:53 -04:00
W. Felix Handte
9c14eafe3d Also Use matchLow for HC3 Match 2018-06-20 15:51:14 -04:00
W. Felix Handte
0a6cf7cd1d Minor Changes 2018-06-20 15:27:23 -04:00
W. Felix Handte
ae1f3898a2 Remove Dead(!) HC3 DMS Lookup 2018-06-20 15:27:12 -04:00
Yann Collet
93702a7a62
Merge pull request #1198 from facebook/msdebug 
made Visual Studio compatible with DEBUGLEVEL >= 2
 2018-06-20 12:26:31 -07:00
cyan4973
ae0b7ffa0a made Visual Studio compatible with DEBUGLEVEL >= 2 2018-06-20 09:45:02 -07:00
Yann Collet
62469c9f41 fixed wrong size in pthread struct transfer 2018-06-19 20:14:03 -07:00
Yann Collet
166901dc72 reduced POOL_resize() restriction 
It's not necessary to ensure that no job is ongoing.
The pool is only expanded, existing threads are preserved.
In case of error, the only option is to return NULL and terminate the thread pool anyway.
 2018-06-19 18:07:18 -07:00
Yann Collet
066fbbfe1c make zstdmt resize its context 
when nbThreads change.

Technically, it only expands.
But when instructed to use less threads,
the thread pool will limit nb of concurrent threads.
 2018-06-19 17:28:56 -07:00
Yann Collet
4567c57199 finalized POOL_resize() 
POOL_ctx* POOL_resize(POOL_ctx* ctx, size_t numThreads)

The function may fail, and returns a NULL pointer in this case.
 2018-06-19 16:03:12 -07:00
Yann Collet
6768cf53fd
Merge pull request #1190 from terrelln/ldm-adjust 
Adjust advanced parameters to source size
 2018-06-19 14:40:56 -07:00
Yann Collet
c0b6ce95b1
Merge pull request #1179 from supertopher/dev 
Improves UX for --list command's lack of support for pipes
 2018-06-19 14:36:30 -07:00
W. Felix Handte
03c39c540b Fix Incorrect Param 2018-06-19 15:36:33 -04:00
W. Felix Handte
de639502aa Update Dict Attachment Cut-Offs 2018-06-19 15:36:13 -04:00
W. Felix Handte
f0a13bcd68 Make Sure Position 0 Gets Into the Tree 2018-06-19 15:10:06 -04:00
Yann Collet
cb45b41f16
Merge pull request #1194 from facebook/no-opaqueapi 
removed specific --opaqueapi test
 2018-06-19 10:24:26 -07:00
W. Felix Handte
87fe4788a3 Fix Compression Ratio Regression #1 2018-06-19 13:01:21 -04:00
W. Felix Handte
4bb79f9c55 Misc Changes 2018-06-19 13:01:21 -04:00
W. Felix Handte
2091f34e9e Find Proper Matches 2018-06-19 13:01:21 -04:00
W. Felix Handte
64348a15f1 Misc Fixes 2018-06-19 13:01:21 -04:00
W. Felix Handte
ade8586ce6 Find mls == 3 Matches 2018-06-19 13:01:21 -04:00
W. Felix Handte
ce743312e2 Fix Typo 2018-06-19 13:01:21 -04:00
W. Felix Handte
a075864756 Switch != ZSTD_extDict to == ZSTD_noDict 2018-06-19 13:01:21 -04:00
W. Felix Handte
1e03377bde Implement RepCode Check 2018-06-19 13:01:21 -04:00
W. Felix Handte
ccbf067973 Add _dictMatchState Functions 2018-06-19 13:01:21 -04:00
W. Felix Handte
d5d8240967 Convert extDict Flag to dictMode Enum 2018-06-19 13:01:21 -04:00
W. Felix Handte
93c3184d44 Attach Dicts when Using ZSTD_btopt and ZSTD_btultra 2018-06-19 13:01:21 -04:00
Topher Lubaway
5bac1db28f Tests to verify piped input to --list exits 1 
I'm following the pattern that i saw in the rest of the test file
please tell me if i am using the wrong conventions
 2018-06-19 09:56:37 -07:00
Yann Collet
1c714fda3f introduced POOL_resize() 
not complete yet :
finalize behavior in case of unfinished expansion
 2018-06-18 20:46:39 -07:00
Yann Collet
c9e8ee93a7 removed specific --opaqueapi test 
from zstreamtest.

This test is now integrated within --newapi,
which dynamically switches between the 2 modes randomly.

The main outcome is reduced testing time.
 2018-06-18 19:20:37 -07:00
Nick Terrell
1d0fcde45d Use debug.h in fileio.c 2018-06-18 15:51:21 -07:00
Nick Terrell
3841dbac84 Adjust advanced parameters to source size 
In the new advanced API, adjust the parameters even if they are explicitly
set. This mainly applies to the `windowLog`, and accordingly the `hashLog`
and `chainLog`, when the source size is known.
 2018-06-18 15:49:31 -07:00
George Lu
a3c8b59990 Fix cli no print 
Change looping behavior to match old
 2018-06-18 15:38:14 -07:00
Yann Collet
e30f13bde0
Merge pull request #1185 from felixhandte/zstd-btlazy-in-place-dict 
ZSTD_btlazy2: Support Searching the Dictionary Context In-Place
 2018-06-18 13:29:44 -07:00
George Lu
e482e328cd Reorder Arguments 
make initFn nullable
 2018-06-18 13:21:42 -07:00
George Lu
0d1ee22990 Requested Changes 
Add Comment
Simplify Interface (Remove resultSet)
Reorder Arguments
Remove customBench displayLevel
Reorder bench.h
Change benchFiles return type to match advanced
Rename stuff
 2018-06-18 12:01:12 -07:00
George Lu
8522346322 Make Fullbench use new function 
Rearrange Args
Add nothing function
Use new function, change locals to match
New Display
Comment cleanup
Change builds
 2018-06-15 11:37:49 -04:00