AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
6,880 Commits 1 Branch 0 Tags 27 MiB
0e76000dee
Commit Graph

1551 Commits

Author SHA1 Message Date
Yann Collet
18434d76b8 added strerror in comment 
as suggested by @felixhandte
 2018-12-20 17:27:08 -08:00
Yann Collet
41b45b84a1
Merge pull request #1465 from facebook/noFilePresent 
fixed : detection of non-existing file
 2018-12-20 17:21:04 -08:00
Yann Collet
ed2fb6bd57 fixed : better error message when dictionary missing 
during benchmark.
Also : refactored ZSTD_fillHashTable(),
just for readability (it does the same thing)
 2018-12-20 17:20:07 -08:00
Yann Collet
6e9512a70c
Merge pull request #1463 from yijinfb/getenv 
Add support for environment variable ZSTD_CLEVEL in CLI
 2018-12-20 15:17:00 -08:00
Yann Collet
0ed8ee4a37 fixed wrong assert condition 2018-12-20 14:46:23 -08:00
Yann Collet
ffba142406 fixed file identity detection in 32-bit mode 
also :
some library decided to use `index` as a global variable declared in standard header
shadowing the ones used in fastcover.c  :(
 2018-12-20 14:30:30 -08:00
Yann Collet
65a441a8f0 fixed stdlib implementation of time functions 
generated redefinitions
 2018-12-20 14:02:50 -08:00
Yann Collet
72dbf1bcd0 removed strncpy() from util.c 
as Visual surprisingly complains about their usage.
Replaced by memcpy()
 2018-12-20 12:27:12 -08:00
Yann Collet
105fa953cb use strerror() to generate error message 
as suggested by @terrelln .

also:
- hopefully fixed Windows version
- changed the test, so that it passes on non-english OS stdlib errors.
 2018-12-20 09:16:40 -08:00
Yi Jin
9b2d708850 change function name: _readU32FromChar() -> readU32FromCharChecked() 2018-12-19 23:41:18 -08:00
Yann Collet
173ef9dea2 fixed : detection of non-existing file 
better error message
with test
 2018-12-19 18:30:57 -08:00
Yi Jin
f7b1841b6f make init_cLevel(...) static 2018-12-19 17:56:45 -08:00
Yi Jin
30ffc24ad7 fix the code ahead of declaration issue 2018-12-19 17:49:04 -08:00
Yi Jin
0700335f57 change int init_cLevel() to int init_cLevel(void) 2018-12-19 17:38:28 -08:00
Yann Collet
0f2d443e10 added system error strings 
to more failure cases in fileio.c
as requested by fbpkg
 2018-12-19 17:25:58 -08:00
Yi Jin
cdc7bbf8b2 edit README.md 2018-12-19 17:09:54 -08:00
Yi Jin
29c7d82390 add a section on restricted support of environment variables to README.md 2018-12-19 17:06:56 -08:00
Yi Jin
26a9ae3f5f refactor readU32FromChar(...), improve init_cLevel(...), and add env var ZSTD_CLEVEL tests 2018-12-19 16:45:42 -08:00
Yi Jin
5324b1e386 add support for setting compression level through environment variable ZSTD_CLEVEL 2018-12-19 13:26:27 -08:00
Nick Terrell
7abd6acebf Also compare device thanks to @felixhandte 2018-12-18 15:40:27 -08:00
Nick Terrell
cd2c8defad [zstdcli] Refuse to overwrite input file 
Compare the input and output files by their inode number and
refuse to open the output file if the input file is the same.

This doesn't work when (de)compressing multiple files to a single
file, but that is a very uncommon use case, mostly used for
benchmarking by me.

Fixes #1422.
 2018-12-18 15:29:54 -08:00
Yann Collet
517d8c984c
Merge pull request #1449 from facebook/ovlog_def 
overlapLog default values
 2018-12-18 09:45:53 -08:00
Nick Terrell
bdfcaecc0a [zstdcli] Add --no-progress flag 
The `--no-progress` flag disables zstd's progress bars, but leaves
the summary.

I've added simple tests to `playTests.sh` to make sure the parsing
works.
 2018-12-14 11:50:25 -08:00
Yann Collet
1993f5d412 fixed ovlog tests 
and updated man page
 2018-12-12 21:09:14 -08:00
Yann Collet
f2f86d369b Merge branch 'btultra2' into ovlog_def 2018-12-12 20:58:14 -08:00
Yann Collet
9792acda3b Merge branch 'dev' into btultra2 2018-12-12 20:18:27 -08:00
Yann Collet
c313a85ee2 removed exception code for overlapLog level 22 
CLI used to set overlapLog at value 9 when level == 22.
This is no longer necessary (handled internally within library)
 2018-12-11 18:13:06 -08:00
Yann Collet
9b784dec7f changed parameter name to ZSTD_c_overlapLog 
from overlapSizeLog.

Reasoning :
`overlapLog` is already used everwhere, in the code, command line and documentation.
`ZSTD_c_overlapSizeLog` feels unnecessarily different.
 2018-12-11 16:55:33 -08:00
Yann Collet
9c3265a53f
Merge pull request #1417 from facebook/advancedAPI 
Advanced API
 2018-12-10 18:48:15 -08:00
Yann Collet
d613fd9afe linked btultra2 as strategy9 
and ensure zstdbench detects out-of-bound parameters
 2018-12-06 19:27:37 -08:00
Yann Collet
34aa401afd updated documentation 
introducing ZSTD_btultra2
 2018-12-06 17:22:19 -08:00
Yann Collet
be9e561da4 changed ZSTD_c_compressionStrategy into ZSTD_c_strategy 
also : fixed paramgrill, and limit conditions
 2018-12-06 15:00:52 -08:00
Yann Collet
3583d19c4e changed parameter names from ZSTD_p_* to ZSTD_c_* 
for naming consistency
 2018-12-05 17:26:02 -08:00
Yann Collet
34e146f548 advanced decompression function replaces by normal streaming one 
advanced parameters compatible with ZSTD_decompressStream().
 2018-12-04 10:28:36 -08:00
Yann Collet
7ef7dc561a check availability of --color=never command on grep and egrep 
before applying them.
Fixes #1436
 2018-12-03 15:46:55 -08:00
Yann Collet
d8e215cbee created ZSTD_compress2() and ZSTD_compressStream2() 
ZSTD_compress_generic() is renamed ZSTD_compressStream2().

Note that, for the time being,
the "stable" API and advanced one use different parameter planes :
setting parameters using the advanced API does not influence ZSTD_compressStream()
and using ZSTD_initCStream() does not influence parameters for ZSTD_compressStream2().
 2018-11-30 11:25:56 -08:00
Yann Collet
f6591e4a04
Merge pull request #1430 from lzutao/fix-zstdgrep 
Fix  #1428 - zstdgrep now returns 1 on unmatch
 2018-11-28 14:07:40 -08:00
Lzu Tao
3d18b4764d Prevent globbing on non-quoting variable 2018-11-29 03:04:40 +07:00
Lzu Tao
c046e0b626 Fix #1428 - zstdgrep now returns 1 on unmatch 2018-11-29 02:45:01 +07:00
Denis Ahrens
71b8ee1bf1 fixed a typo 2018-11-28 04:51:16 +01:00
Yann Collet
41c7d0b1e1 changed hashEveryLog into hashRateLog 2018-11-21 14:36:57 -08:00
Yann Collet
e874dacc08 changed searchLength into minMatch 
refactored all relevant API and calls
for consistency.
 2018-11-20 14:56:07 -08:00
Yann Collet
3b838abf97 ZSTD_CCtx_setParameter : value argument is now int 
for compatibility with compression level
 2018-11-20 11:53:01 -08:00
Ryan Schmidt
46d470c20c Fix feature detection with multiple -arch flags 
When multiple -arch flags are used, the compiler invokes itself once for
each architecture. Apparently, input on stdin is consumed by the
compilation of the first arch and is no longer available to the
compilation of the second arch, which results in a build failure and the
potentially incorrect determination that a feature is not available. So
write the feature detection source to a file instead of using stdin.
 2018-11-16 03:49:15 -06:00
Yann Collet
5c68639186 updated ZSTD_DCtx_reset() 
signature and behavior is now the same as ZSTD_CCtx_reset()
 2018-11-15 16:12:39 -08:00
Yann Collet
06c8d5a4f4 Merge branch 'dev' into advancedAPI 
fixed rsyncable
 2018-11-15 10:51:24 -08:00
Nick Terrell
4316409427 [cli] Set the job size 2018-11-14 17:00:29 -08:00
Nick Terrell
f9a671ac37 [cli] Add rsyncable 2018-11-14 17:00:16 -08:00
Yann Collet
7b0391e37e finalized retrofit of ZSTD_CCtx_reset() 
updated all depending sources
 2018-11-14 13:05:35 -08:00
Yann Collet
5adbad4059 Merge branch 'dev' into advancedAPI 2018-11-14 13:00:37 -08:00
Yann Collet
c584e84e68
Merge pull request #1415 from facebook/benchfn 
Separating benchfn and benchzstd
 2018-11-14 13:00:08 -08:00
Yann Collet
d7e10a774a added constant ZSTD_WINDOWLOG_LIMIT_DEFAULT 
answering #1407.

Also : removed obsolete function ZSTD_setDStreamParameter()
which could only be used with one parameter (DStream_p_maxWindowSize).
Now replaced by ZSTD_DCtx_setWindowSize() (which exists since a few revisions)
 2018-11-13 18:12:34 -08:00
Yann Collet
2c8fde538f added constant ZSTD_MAGIC_SKIPPABLE_MASK 
and updated several API comments
 2018-11-13 17:36:35 -08:00
Yann Collet
b83d1e7714 removed some static const variables 
and replaced by traditional macro constants.

Unfortunately, C doesn't consider `static const` to mean "constant"
 2018-11-13 16:56:32 -08:00
Yann Collet
092c4abd4c bumped version number to v1.3.8 2018-11-13 15:53:38 -08:00
Yann Collet
3ba0d6dd27 fixed decode-only test condition 2018-11-13 14:15:12 -08:00
Yann Collet
a29e3d1d01 fix debug trace 2018-11-13 13:27:22 -08:00
Yann Collet
b830ccca5c changed benchfn api 
to use structure for function parameters
as it expresses much clearer than a long list of parameters,
since each parameter can now be named.
 2018-11-13 13:12:50 -08:00
Yann Collet
9867cdb847 benchfn can provided faulty return value 
with BMK_extract_returnValue()
 2018-11-13 12:01:17 -08:00
Yann Collet
d38063f8ae separated bench module into benchfn and benchzstd 
it shall be possible to use benchfn
without any dependency on zstd.
 2018-11-13 11:01:59 -08:00
Yann Collet
9126da5b5c improve long-range decoder speed 
on enwik9 at level 22 (which is almost a worst case scenario),
speed improves by +7% on my laptop (415 -> 445 MB/s)
 2018-11-08 12:47:46 -08:00
Yann Collet
8bed4012bd fixed decompression-only benchmark 2018-11-08 12:36:39 -08:00
Bernhard M. Wiedemann
572279e73e pgo: also optimize compression 
and clean *.o so that linking does not fail from code
that is partially augmented with coverage generation instructions
 2018-10-29 17:20:03 +01:00
Bernhard M. Wiedemann
a38ad53ace zstd-pgo: do not make clean and zstd in parallel 
otherwise, there might not be a ./zstd to run
 2018-10-29 15:11:28 +01:00
Yann Collet
fc20b3c441 added flag -Wc++-compat 
for library and cli
 2018-10-26 16:38:23 -07:00
Yann Collet
fab150ce9f fix : removed zstdmt symlink on uninstall 2018-10-22 12:51:56 -07:00
Yann Collet
f181799082 fix decodecorpus incorrect frame generation 
fix #1379
decodecorpus was generating one extraneous byte when `nbSeq==0`.
This is disallowed by the specification.

The reference decoder was just skipping the extraneous byte.
It is now stricter, and flag such situation as an error.
 2018-10-20 18:56:21 -07:00
Yann Collet
f2cff22804
Merge pull request #1368 from rkjain89/test-branch 
Moving Code To util.c
 2018-10-19 16:31:10 -07:00
Sam Russell
622699706c fix up zgrep/zless references 2018-10-16 08:42:02 +13:00
Sam Russell
490f652974 Add man pages for zstdgrep and zstdless 2018-10-15 22:25:03 +13:00
Rohit Jain
c7251e5151 Address most comments 2018-10-11 18:05:15 -07:00
Rohit Jain
91b2fed8f8 Moving more function to the new C file 2018-10-11 17:34:47 -07:00
Rohit Jain
a47f6e6f2d Removing static and making extern 2018-10-11 16:51:29 -07:00
Yann Collet
5d419071be
Merge pull request #1369 from facebook/v137 
bumped version number to v1.3.7
 2018-10-11 15:59:57 -07:00
Rohit Jain
705e0b18ab Making changes to make it compile on my laptop 2018-10-11 15:51:57 -07:00
Rohit Jain
b19140bc13 Fixed makefile and removed multiple definitions from util.h (UTIL_getTime) 2018-10-11 15:17:40 -07:00
Rohit Jain
d6d240f016 Removed UTIL_setFileStat from util.h and move it to util.c 2018-10-11 15:07:12 -07:00
Yann Collet
1e6208e75e bumped version number to v1.3.7 
updated documentation
 2018-10-11 14:40:12 -07:00
Rohit Jain
f881ee8c48 Moving code to util.c 2018-10-11 14:38:41 -07:00
Yann Collet
433059bbb2 fixed minor gcc warning 
gcc-8 on Linux doesn't like usage of strncat :
`warning: ‘strncat’ output truncated before terminating nul copying as many bytes from a string as its length`.

Not sure what was wrong, it might be a false positive,
but the logic is simple enough to replaced by a simple `memcpy()`,
thus avoiding the shenanigans of null-terminated strings.
 2018-10-10 17:06:25 -07:00
Yann Collet
b304b679e5 use #if BACKTRACE_ENABLE directly 
as suggested by @terrelln
 2018-10-09 17:56:59 -07:00
Julian Fessard
0ea286f740 Merge branch 'backtrace' of github.com:facebook/zstd into backtrace 2018-10-09 17:24:48 -07:00
Julian Fessard
70d8c2a031 fileio.c: Disable backtrace when built with address sanitizer 
Covers clang and gcc's sanitizer flags.
Can still be overridden through CFLAGS on commandline.
 2018-10-09 17:14:57 -07:00
Yann Collet
e0ab6b61b7 fixed explicit BACKTRACE order 
and automatic linux backtrace detection :
__GLIBC__ must be tested after #include <features.h>
 2018-10-09 17:12:21 -07:00
Yann Collet
1e0c5466c5 fixed BACKTRACE_ENABLE macro test 2018-10-09 16:36:57 -07:00
Yann Collet
f17c1df1ac backtrace support compiled with more conditions 
following #1356,
only enable backtrace compilation on linux+glibc.

Also, disable backtrace by default from "release" compilation,
so that less platforms get impacted by the new requirements.
Can be manually enabled/disabled using BACKTRACE=1/0.
 2018-10-08 17:03:06 -07:00
maxice8
1d75697d9b
don't assume __linux__ means __GLIBC__ on BACKTRACES_ENABLE 
check for __GLIBC__ instead of __linux__ as musl libc doesn't provide
execinfo.h

fixes compilation on Alpine Linux and Void Linux musl arches.
 2018-10-05 21:23:45 -03:00
Yann Collet
0854b06d1b
Merge pull request #1351 from facebook/haikubuild 
portability macro flags updates, for Haiku
 2018-10-04 14:59:35 -07:00
Yann Collet
efbc3e823d fixed paramgrill wrong assert() conditions 
and slightly refactored affected function.

Honestly, the formula calculating variance should get a second reviewing round,
it's not clear if it's correct.
 2018-10-04 14:27:13 -07:00
Yann Collet
4a85b126d9 changed ZSTD_NANOSLEEP_SUPPORT definition 
to please `-Wexpansion-to-defined`
 2018-10-03 15:34:41 -07:00
Yann Collet
549c19b42e portability macro flags updates, for Haiku 
some non-trivial changes to platform.h and util.h,
initially related to compilation for Haiku,
but I used this opportunity to make them cleaner
and add some documentation.

Noticed several tests that could be improved
(too harsh conditions, useless exception, etc.)
but I did not dare modifying too many tests just before release.
 2018-10-03 14:56:46 -07:00
Yann Collet
bc93b801f0
Merge pull request #1330 from korli/haiku 
Enable building zstd on Haiku.
 2018-10-03 13:36:00 -07:00
Yann Collet
b1407f9acd fixed wrong assert() position 
could fire on invalid input.
blocking for afl tests.
 2018-10-03 12:43:59 -07:00
Jerome Duval
87c10e2f58 Enable building zstd on Haiku. 2018-10-03 09:51:56 +02:00
Yann Collet
c9843ec232
Merge pull request #1348 from facebook/donotdelete 
Fix #1082
 2018-10-02 16:37:58 -07:00
Yann Collet
3ca6261223 fixed static analyzer warnings 
note : for some reason,
scan-build version on my laptop found problems within fastcover.c
that scan-build on travisCI does not flag.

They are, as usual, false positive :
the analyzer does not understand that a table (`offset`) is correctly filled before usage.
 2018-10-02 15:59:11 -07:00
Yann Collet
d98733b37e restored backtrace on failure 
for Linux and Mac OS-X.

Note : the backtraces fires up through a trap
before the sanitizer get a chance to report.
There are situations where the sanitizer report is actually preferable.

It might be good to consider a kind of build macro
which can disable backtrace
when sanitizer is enabled.
 2018-10-01 17:50:16 -07:00
Yann Collet
9012b6cba0 ./zstd -f do no longer overwrite destination file 
if source file does not exist (#1082)
 2018-10-01 17:16:34 -07:00
Yann Collet
c7bd6a41ab zstd -d -f do no longer erase destination file 
when source file does not exist (#1082)
 2018-10-01 14:04:00 -07:00
Yann Collet
1ab71a8e72 regroup name creation logic into its own function 
for a cleaner main file decompression loop
 2018-09-28 18:19:23 -07:00
Yann Collet
ec1cb8e996 changed macro name 
from EXIT_IF() to RETURN_IF()

EXIT could be misunderstood as exit(), which terminates program execution.
But the macro only leaves the function, not the program.
 2018-09-28 16:04:00 -07:00
Yann Collet
05c0a072b7 minor improvement in the multi-format suffix selection 2018-09-28 15:57:35 -07:00
Nick Terrell
09231dad4d [util] Fix lstat feature test macro 2018-09-28 15:23:09 -07:00
Yann Collet
d987ab5983 fixed unreachable section warning on Visual 2018-09-28 09:34:16 -07:00
Yann Collet
913a0365b6 Merge branch 'dev' into donotdelete 2018-09-27 19:23:55 -07:00
Yann Collet
ef1272737b fixed minor Visual conversion warnings 2018-09-27 18:29:15 -07:00
Yann Collet
9b45db7fa6 minor refactoring of --list 
trying to reduce recurrent patterns.
 2018-09-27 16:49:08 -07:00
Nick Terrell
f2d6db45cd [zstd] Add -Wmissing-prototypes 2018-09-27 15:24:48 -07:00
Yann Collet
e74eade251
Merge pull request #1339 from facebook/grep_colors 
fixed usage of grep in Makefile
 2018-09-26 14:39:20 -07:00
Yann Collet
8883af6a1e
Merge pull request #1327 from facebook/adapt 
Adaptive compression
 2018-09-26 14:39:08 -07:00
Yann Collet
8ff17a6a09
Merge pull request #1329 from facebook/v04isout 
Changed default legacy support to v0.5+
 2018-09-26 13:39:05 -07:00
Yann Collet
08f68d83c5 fixed usage of grep in Makefile 
when terminal uses colors
as suggested by @danielshir (#1294)
 2018-09-25 16:56:53 -07:00
Yann Collet
6c51bf420c bounds for --adapt mode 
can supply min and max compression level through advanced command :
--adapt=min=#,max=#
 2018-09-25 16:03:28 -07:00
Yann Collet
04f47bbdd2 Merge branch 'dev' into adapt 2018-09-24 16:56:45 -07:00
Yann Collet
9bb6c15f79
Merge pull request #1332 from facebook/minclevel 
defined a minimum negative level
 2018-09-24 16:01:13 -07:00
Yann Collet
0250ac74ce fixed minor scan-build warnings 2018-09-24 00:52:19 -07:00
Yann Collet
0fc07eb1fd fixed zstd-decompress 
which cannot support ZSTD_minCLevel()
 2018-09-22 17:21:39 -07:00
Yann Collet
364041c6dd enforce minimum compression level limit 
using ZSTD_minCLevel()
 2018-09-22 16:10:10 -07:00
Yann Collet
00c18c0c88 simplified "slows down when compression blocked" 2018-09-21 16:35:43 -07:00
Yann Collet
c484345a82 Merge branch 'mingw' into adapt 2018-09-21 16:00:46 -07:00
Yann Collet
bfff4f4809 ensure all writes to job->cSize are mutex protected 
even when reporting errors,
using a macro for code brevity, as suggested by @terrelln,
 2018-09-21 16:00:39 -07:00
Yann Collet
32b7cf1bcf fixed tautological tests 
involving ZSTD_TARGETLENGTH_MIN (== 0)
 2018-09-21 15:04:43 -07:00
Yann Collet
54001f3dbc fix mingw compatibility 
only enable backtraces for platforms we know support it
aka mac OS-X and Linux.
can be extended later.
 2018-09-21 14:46:09 -07:00
Yann Collet
b2939163e1 Changed default legacy support to v0.5+ 
thus dropping read support for v0.4.

It's always possible to re-enable it, by changing build macro ZSTD_LEGACY_SUPPORT to 4.
 2018-09-20 14:30:20 -07:00
Yann Collet
15519479ba fixed minor gcc warning on a unused variable 2018-09-20 13:00:11 -07:00
Yann Collet
45010da074 updated man page 
and added `--adapt` test in `playTests.sh`
 2018-09-19 17:37:22 -07:00
Yann Collet
ca02ebee07 removed static variables 
so that --adapt can work on multiple input files too
 2018-09-19 15:25:50 -07:00
Yann Collet
89bc309d90 error out when --adapt is associated with --single-thread 
since they are not compatible
 2018-09-19 14:49:13 -07:00
Yann Collet
2f78228f65 Merge branch 'dev' into adapt 2018-09-19 12:43:42 -07:00
Casey McGinty
d019d15f2b
Add build define to manually enable -rdynamic LD flag 2018-09-12 14:00:08 -07:00
Casey McGinty
b703181f93
Disable -rdynamic LD option on Windows 2018-09-11 16:19:34 -07:00
Casey McGinty
b9118ecdab
Update comments, and LD flag usage in Make 2018-09-11 14:49:47 -07:00
Casey McGinty
2224ecd718
Remove dead code and method name typo 2018-09-11 11:56:50 -07:00
Casey McGinty
d4337b6f1d
Move ABRThandler func out of internal lib 2018-09-11 11:39:49 -07:00
Casey McGinty
2a3967b7c4
Refactor abort signal handling 
- Print signal name to term
- Add -rdynamic option to generate Linux symbol names in backtrace
- Raise default signal after handler to ensure program termination
 2018-09-11 10:40:45 -07:00
Casey McGinty
a06574fc97
Print a stack trace on unexpected term signal (e.g. SIGABRT) 
For OSX and Linux, add a signal handler to SIGABRT, SGIFPE, SIGILL,
SIGSEGV, and SIGBUS. When the program terminates unexpectedly the
handler will print the current stack to the terminal to help determine
the location of the failure.

On OSX the output will look like:

```
Stack trace:
4   zstd                                0x000000010927ed96 main + 16886
5   libdyld.dylib                       0x00007fff767d1015 start + 1
6   ???                                 0x0000000000000001 0x0 + 1
```

On Linux the output will look like:

```
Stack trace:
./zstd() [0x4b8e1b]
./zstd() [0x4b928a]
./zstd() [0x403dc2]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f5e0fbb0445]
./zstd() [0x405754]
```

As is, the code does not function on WIN32.

See also: https://oroboro.com/stack-trace-on-crash/
 2018-09-06 18:46:52 -07:00
Yann Collet
deb03d8a26 fixed UTIL_isLink() for Mac OS-X 2018-09-06 17:32:16 -07:00
Yann Collet
50b216146f
Merge pull request #1304 from facebook/largeNbDicts 
contrib/largeNbDicts
 2018-09-06 09:50:56 -07:00
Jennifer Liu
21721b75a3 Change default f to 20 2018-09-04 17:15:14 -07:00
Yann Collet
39c55a118f fixed minor compatibility issues with older compilers 2018-08-30 16:00:57 -07:00
Jennifer Liu
f87383507d Update comment about default dictionary builder 2018-08-30 15:46:39 -07:00
Yann Collet
4086b2871b largeNbDicts compatible with multiple source files 
splitting is disabled by default, but can be re-enabled using usual command -B#
update commands to look like zstd ones
 2018-08-30 14:38:49 -07:00
Yann Collet
0c66a44d1b first working test program 
measures :
- compression ratio with / without dictionary
- create one dictionary per block
- memory budget for dictionaries
- decompression speed, using one different dictionary per block

current limitations :
- only one file
- 4K blocks only
- automatic dictionary built with 4K size

dictionary can be selected on command line, with -D
 2018-08-28 15:47:07 -07:00
Yann Collet
b37a0a6bde
Merge pull request #1298 from facebook/bench 
Refactored bench.c
 2018-08-28 12:25:02 -07:00
Yann Collet
0491037db9 Merge branch 'bench' into largeNbDicts 2018-08-28 11:26:46 -07:00
Yann Collet
55affc09de timedFn : measurement delay is programmable 
instead of hard-coded 1 second per measurement
 2018-08-28 11:26:27 -07:00
Yann Collet
d97e92dfad Merge branch 'bench' into largeNbDicts 2018-08-27 12:12:51 -07:00
Yann Collet
01dcd0fd17 bench: minor api update, for consistency 
BMK_benchTimedFn()
BMK_isCompleted_TimedFn() uses TimedFnState
 2018-08-26 21:30:18 -07:00
Yann Collet
6782725155 first sketch for largeNbDicts test program 2018-08-26 19:29:12 -07:00
Yann Collet
c3a4baaf6e fixed minor warnings 
valgrind: memory leak of a few bytes in fullbench
static analyzer: uninitialized data passed as result
 2018-08-24 23:25:35 -07:00