Commit Graph

2552 Commits

Author SHA1 Message Date
Yann Collet
085179bb78 fixed ZSTDMT_createCCtx() : checked inner objects are properly created 2017-01-11 15:58:05 +01:00
Yann Collet
8ce1cc2bec improved ZSTD_createCCtxPool() cancellation
use ZSTD_freeCCtxPool() to release the partially created pool.
avoids to duplicate logic.

Also : identified a new difficult corner case :
when freeing the Pool, all CCtx should be previously released back to the pool.
Otherwise, it means some CCtx are still in use.
There is currently no clear policy on what to do in such a case.
Note : it's supposed to never happen.
Since pool creation/usage is static, it has no external user,
which limits risks.
2017-01-11 15:44:26 +01:00
Yann Collet
47557ba2b2 fixed ZSTDMT_createCCtxPool() when inner CCtx creation fails 2017-01-11 15:35:56 +01:00
Yann Collet
b8cdc16969 Merge pull request #509 from terrelln/dict-builder-32
Handle cover dictionary builder maximum input size for 32-bit mode
2017-01-10 06:30:28 +01:00
Nick Terrell
8d984699db Document memory requirements for COVER algorithm 2017-01-09 18:20:10 -08:00
Nick Terrell
555e281637 Handle large input size in 32-bit mode correctly 2017-01-09 18:20:06 -08:00
Nick Terrell
c220d4c74d Use COVER_MEMMULT when training with COVER. 2017-01-09 16:49:04 -08:00
Yann Collet
56958500fc minor man page update 2017-01-09 19:47:09 +01:00
Yann Collet
4736dffd41 Merge pull request #507 from terrelln/dict-builder-optimize
Add new dictionary builder
2017-01-03 20:33:33 +01:00
Nick Terrell
a8b4fe0481 Add COVER dictionary builder to fuzzer unit tests 2017-01-02 18:45:19 -08:00
Nick Terrell
3a1fefcf00 Simplify COVER parameters 2017-01-02 17:51:38 -08:00
Yann Collet
8566799781 separated ppc and ppc64 tests, for more regular timing 2017-01-03 00:25:01 +01:00
Nick Terrell
cbb3ce376b Add cover cli to playtests 2017-01-02 14:43:11 -08:00
Nick Terrell
df8415c502 Add COVER to the zstd cli 2017-01-02 14:43:08 -08:00
Nick Terrell
9103ed6c8b Add cover.c to build files 2017-01-02 13:22:53 -08:00
Nick Terrell
96b39f65fa Add COVER dictionary builder 2017-01-02 13:22:51 -08:00
Yann Collet
6334b04d61 compile object files, for faster recompilation 2017-01-02 03:22:18 +01:00
Yann Collet
747452677d fixed cmake tests 2017-01-02 02:05:45 +01:00
Yann Collet
cdb2763f4a new Makefile target zstdmt 2017-01-02 01:43:56 +01:00
Yann Collet
f1cb55192c fixed linux warnings 2017-01-02 01:11:55 +01:00
Yann Collet
0ec6a95ba1 minor fixes 2017-01-02 00:49:42 +01:00
Yann Collet
2ec635a162 use pthread_cond to send signals between threads 2017-01-01 17:31:33 +01:00
Yann Collet
e4f70cdbbc Merge pull request #505 from terrelln/pthread_cond_t
pthread_cond_t
2017-01-01 15:53:51 +01:00
Nick Terrell
5ca0fd2045 Shorten thread pool tests 2016-12-31 22:39:32 -05:00
Nick Terrell
bb13387d7d Fix pool for threading.h 2016-12-31 19:10:47 -05:00
Nick Terrell
4204e03e77 Add threading.h condition variables 2016-12-31 19:10:29 -05:00
Nick Terrell
d132433534 Switch thread pool test to threading.h 2016-12-31 19:10:13 -05:00
Yann Collet
3b9d434356 extended ZSTDMT code support for non-MT systems and WIN32 (preliminary) 2016-12-31 16:32:19 +01:00
Yann Collet
c8efc1c874 simplified Buffer Pool 2016-12-31 14:45:33 +01:00
Yann Collet
3b29dbd9e8 new zstdmt version using generic treadpool 2016-12-31 06:04:25 +01:00
Yann Collet
c6a6417458 bench correctly measures time for multi-threaded compression (posix only) 2016-12-31 03:31:26 +01:00
Yann Collet
f765a375a5 Merge pull request #504 from terrelln/thread-pool
[zstdmt] Add thread pool
2016-12-30 15:31:49 +01:00
Nick Terrell
9c499648e3 Add thread pool tests 2016-12-29 23:41:03 -08:00
Nick Terrell
e777a5be6b Add a thread pool for ZSTDMT and COVER 2016-12-29 23:39:44 -08:00
Przemyslaw Skibinski
59292ef6e1 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-12-29 15:24:18 +01:00
Yann Collet
e70912c72b Changed : input divided into roughly equal parts.
Debug : can measure time waiting for mutexes to unlock.
2016-12-29 01:24:01 +01:00
Yann Collet
6c0ed9483a compression threads use ZSTD_compressCCtx() 2016-12-28 17:08:28 +01:00
Yann Collet
8d7432914f Merge pull request #503 from inikep/dev11
Dev11
2016-12-28 16:50:39 +01:00
Yann Collet
ab7a579180 added -T command , to set nb of threads 2016-12-28 16:11:09 +01:00
Yann Collet
ce9e1452fd protect buffer pool with a mutex 2016-12-28 15:31:19 +01:00
Yann Collet
9b587e2135 Merge pull request #502 from Chocobo1/cpp11
Move -std=c++11 cxxflag to PZSTD_CXXFLAGS
2016-12-28 14:38:48 +01:00
Przemyslaw Skibinski
75f3a3a335 changed default PREFIX and MANDIR 2016-12-28 12:32:41 +01:00
Yann Collet
3d93f2fce7 first zstdmt sketch 2016-12-27 07:19:36 +01:00
Chocobo1
37a2fb4ce1 Move -std=c++11 cxxflag to PZSTD_CXXFLAGS
Fixes the problem that the compiler doesn't enable c++11 mode by default
and the package build system has its own CXXFLAGS
2016-12-26 23:08:23 +08:00
Yann Collet
39c105c605 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-12-23 22:25:31 +01:00
Yann Collet
aca113f4f5 fixed ZSTD_sizeof_?Dict() 2016-12-23 22:25:03 +01:00
Yann Collet
c07d2e3a31 Merge pull request #499 from inikep/dev11
improved *BSD and Solaris compatibility
2016-12-23 21:32:03 +01:00
Yann Collet
63fcb87b07 Merge pull request #497 from apjanke/zstdless-with-shebang
zstdless: add shebang and quote $@
2016-12-23 11:18:10 +01:00
Yann Collet
d9da24042a Merge pull request #498 from terrelln/zdict-deprecated
Fix failing test due to deprecation warning
2016-12-23 11:16:23 +01:00
Przemyslaw Skibinski
63b0014b96 BSD: improved "make install" 2016-12-23 10:05:49 +01:00