AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
5,982 Commits 1 Branch 0 Tags 27 MiB
d3a0c71259
Commit Graph

134 Commits

Author SHA1 Message Date
Yann Collet
c005df136f
Merge pull request #947 from facebook/fix944 
Fix #944
 2017-12-14 10:01:52 -08:00
Nick Terrell
4680e85bdf Allow -o with multiple files 2017-12-13 17:44:34 -08:00
Nick Terrell
82bc8fe0cc [fileio] Refuse to remove non-regular file 2017-12-13 13:38:26 -08:00
Yann Collet
311878dec3 Improved tests 
- building cli from /tests preserves potential flags in MOREFLAGS (such as asan/usan)
- MT dictionary tests check for MT capability (MT is not enabled by default for zstd32)
 2017-12-13 11:48:30 -08:00
Yann Collet
98ccc115ed added cli test veryfying bug #944 
this test fails on v1.3.2
 2017-12-12 19:47:38 -08:00
Yann Collet
5891f1d952 added a check to ensure 0-size is present in frame header 2017-10-17 16:23:20 -07:00
Yann Collet
241c57adff playtest : minor cosmetic changes 2017-10-16 14:01:42 -07:00
Yann Collet
43914f6a20 add test for presence of decompressed size field in header 
previous test was "display" only, it wouldn't trigger an error in case of missing header.
 2017-10-13 23:47:01 -07:00
Yann Collet
4252621e26 playtests: do not use cat on large files 
some target have limitation making cat incompatible with large files
(namely debian hurd-i386)
 2017-10-05 20:21:59 -07:00
Yann Collet
e4ec427720 Merge branch 'dev' into shorterTests 
fixed conflicts
 2017-09-28 12:19:28 -07:00
Yann Collet
54a827fff0 Merge branch 'dev' into newFormats 
Fixed conflicts in zstdmt_compress.c
 2017-09-27 16:39:40 -07:00
Yann Collet
02502191e5 separated --long tests between short and long tests 
A fast subset of these tests is now played in short test mode
 2017-09-27 15:48:06 -07:00
Yann Collet
f9de54acfb reduced memory requirements for --long tests 
in new --long test section
 2017-09-27 15:38:27 -07:00
Yann Collet
60059df051 shorter make test to avoid time out on travis CI 
Timed tests (fuzzer) are reduced
Long tests are shortened (less data generated)
 2017-09-27 15:16:27 -07:00
Nick Terrell
c233bdbaee Increase maximum window size 
* Maximum window size in 32-bit mode is 1GB, since allocations for 2GB fail
  on my Mac.
* Maximum window size in 64-bit mode is 2GB, since that is the largest
  power of 2 that works with the overflow prevention.
* Allow `--long=windowLog` to set the window log, along with
  `--zstd=wlog=#`. These options also set the window size during
  decompression, but don't override `--memory=#` if it is set.
* Present a helpful error message when the window size is too large during
  decompression.
* The long range matcher defaults to a hash log 7 less than the window log,
  which keeps it at 20 for window log 27.
* Keep the default long range matcher window size and the default maximum
  window size at 27 for the API and CLI.
* Add tests that use the maximum window size and hash size for compression
  and decompression.
 2017-09-26 14:00:01 -07:00
Yann Collet
62568c9a42 added capability to generate magic-less frames 
decoder not implemented yet
 2017-09-25 14:26:26 -07:00
Nick Terrell
1fe762e236 [zstdcli] Fix LDM advanced options parsing 2017-09-18 14:49:35 -07:00
Stella Lau
17d8e0bdcc Merge remote-tracking branch 'upstream/longRangeMatcher' into ldm-integrate 2017-09-01 10:19:38 -07:00
Stella Lau
6a546efb8c Add long distance matcher 
Move last literals section to ZSTD_block_internal
 2017-08-31 12:53:19 -07:00
Nick Terrell
3587556873 [cover] Test small maxdict 2017-08-21 11:16:47 -07:00
Nick Terrell
abe12b3399 [libzstd] Fix bug in Huffman decompresser 
The zstd format specification doesn't enforce that Huffman compressed
literals (including the table) have to be smaller than the uncompressed
literals. The compressor will never Huffman compress literals if the
compressed size is larger than the uncompressed size. The decompresser
doesn't accept Huffman compressed literals with 4 streams whose compressed
size is at least as large as the uncompressed size.

* Make the decompresser accept Huffman compressed literals whose size
  increases.
* Add a test case that exposes the bug. The compressed file has to be
  statically generated, since the compressor won't normally produce files
  that expose the bug.
 2017-08-07 12:37:48 -07:00
Nick Terrell
d0b27483ae [zstdcli] Fix -t in streaming mode 2017-07-18 14:45:49 -07:00
Yann Collet
3a60efd3a9 policy change : ZSTDMT automatically caps nbThreads to ZSTDMT_NBTHREADS_MAX (#760) 
Previously, ZSTDMT would refuse to create the compressor.
Also : increased ZSTDMT_NBTHREADS_MAX to 256,
updated doc,
and added relevant test
 2017-07-13 10:17:23 -07:00
Yann Collet
4616fad18b improved ZSTDMT_compress() memory usage 
does not need the input buffer for streaming operations

also : reduced a few tests time length
 2017-07-10 17:16:41 -07:00
Nick Terrell
6aeb50ea0d [zstdcli] Add unlzma and unxz symlinks 2017-06-26 11:24:36 -07:00
Yann Collet
c16748b137 added test cases for new command --list 2017-06-21 12:09:53 -07:00
Paul Cruz
db3606e249 added test for files with no frame content size 2017-06-20 17:43:36 -07:00
Paul Cruz
a73c2a444a added tests with null files, changed condition to check that the file is more than 0 bytes 2017-06-20 14:33:08 -07:00
Paul Cruz
58c19b4202 spacing matters for test scripts 2017-06-20 14:14:53 -07:00
Paul Cruz
b07d0af2ba added additional tests for error detection 2017-06-20 11:54:44 -07:00
Paul Cruz
6a99de2dd8 change test to list info on .zst files 2017-06-20 10:45:06 -07:00
Paul Cruz
ddd1ab710c added tests in playTest.sh 2017-06-15 16:53:03 -07:00
Yann Collet
01a1abfdb5 cli : -d and -t do not stop after a failed decompression 
The problematic srcfile will be named on console/log,
but decompression/test will continue onto next file in the list.
 2017-05-05 19:15:24 -07:00
Nick Terrell
f376d47c11 [CLI] Switch dictionary builder on CLI to cover 2017-05-02 11:18:27 -07:00
Sean Purcell
4de8632957 Add LZ4 compress/decompress support to CLI 2017-04-24 16:48:25 -07:00
Yann Collet
230d7acc7d cli : add support for --threads=# command 
updated documentation
add relevant test case
 2017-04-21 11:38:13 -07:00
Sean Purcell
d845dab69c Fix input size too small to trigger zstdmt 2017-04-17 12:10:58 -07:00
Sean Purcell
c424ec2eae Add multithreading tests to playTests.sh 2017-04-17 11:41:46 -07:00
Sean Purcell
d37e1df2ab Fix message 2017-04-11 17:33:26 -07:00
Sean Purcell
eb70d219fd Add test of file > 4GB to playTests 2017-04-11 17:15:13 -07:00
Sean Purcell
1652172b2d Add refuse to overwrite test and fix format 2017-04-06 17:06:30 -07:00
Sean Purcell
9da11c6aae Add test for overwriting read-only 2017-04-06 12:58:49 -07:00
Yann Collet
4c41d37fcc changed test for new syntax 
--dictID= and --maxdict=
 2017-03-24 18:36:56 -07:00
Yann Collet
f332ece468 dictBuilder fails to create dictionary on certain input 
Properly expressed with an error code (see zstd_errors.h)
and a cli return code != 0
 2017-03-23 16:24:02 -07:00
Sean Purcell
680e4e0953 Ignore symbolic links unless --force specified 2017-03-23 11:52:09 -07:00
Sean Purcell
9a38dfa794 Only run IS_CONSOLE tests with a TTY 2017-03-17 12:32:18 -07:00
Sean Purcell
d973071e90 Add tests for compressed data on console 2017-03-16 16:25:19 -07:00
Nick Terrell
aa8bcf360f Add xz and lzma support. 
Finish feature started by @inikep.

* Add xz and lzma compression and decompression support to target `xzstd`.
* Fix bug in gzip decompression that silently accepted truncated files.
* Add gzip frame composition tests.
* Add xz/lzma compatibility tests.
* Add xz/lzma frame composition tests.
 2017-03-13 18:45:41 -07:00
Yann Collet
27526c7201 make : added target shortest 
shortest only run fast part of playTests.sh .
cc @iburinoc
 2017-03-01 17:02:49 -08:00
Yann Collet
c1c040eae1 added gzip tests 
also : made sure zstd --format=gzip -V
would fail if gzip compatibility is not supported
 2017-03-01 16:49:20 -08:00
Yann Collet
a81c2e7e44 Merge pull request #573 from facebook/ddict 
Improved DDict memory usage
 2017-02-27 20:54:42 -08:00
Yann Collet
67d86a74a5 added test case : --rm on stdin 
must remain silent (instead of failing)
 2017-02-27 16:09:20 -08:00
Yann Collet
bd7fa21deb added ZSTD_refDDict() 
Now DDict does no longer depends on DCtx duplication
 2017-02-26 14:43:07 -08:00
Yann Collet
bd6bc22612 playtest.sh : changed sdiff into $DIFF 2017-01-22 15:54:14 -08:00
Nick Terrell
3a1fefcf00 Simplify COVER parameters 2017-01-02 17:51:38 -08:00
Nick Terrell
cbb3ce376b Add cover cli to playtests 2017-01-02 14:43:11 -08:00
Przemyslaw Skibinski
5f5a902453 "make test" is now compatible with Solaris 2016-12-22 18:05:07 +01:00
Przemyslaw Skibinski
f9a56668a6 improved playTests.sh (3 2016-12-14 18:43:06 +01:00
Przemyslaw Skibinski
24a4236111 improved playTests.sh (2) 2016-12-14 18:07:31 +01:00
Przemyslaw Skibinski
ab5ed6fa7f improved playTests.sh 2016-12-14 17:10:38 +01:00
Przemyslaw Skibinski
9b4fa0ddf7 playTests.sh: added Advanced compression parameters 2016-12-14 16:50:00 +01:00
Dimitry Andric
12df6da83b Fix running test suite on FreeBSD 
* Remove last bashism from tests/playTests.sh
* Use gmd5sum from the sysutils/coreutils port
 2016-12-12 19:22:47 +01:00
Yann Collet
94d1a93d28 changed environment variable comparison to sh compatible 2016-12-06 12:02:56 -08:00
Yann Collet
743b33f57e fix zstdcat 2016-12-02 15:18:57 -08:00
Yann Collet
4f5350f610 long matches support overflow 2016-11-29 13:12:24 -08:00
Przemyslaw Skibinski
eb977a42ef "file" moved to tests/Makefile 2016-11-14 17:26:58 +01:00
Przemyslaw Skibinski
e579ab5faa introduced QEMU_SYS 2016-11-14 12:57:05 +01:00
Yann Collet
b9550d6d43 fixed benchmark mode typo . 
added benchmark mode tests
 2016-10-28 14:43:24 -07:00
Yann Collet
d7b120ab5c added long commands --memory= and --memlimit-decompress= 2016-10-14 14:22:32 -07:00
Yann Collet
1122349ac2 added long comment --memlimit= 2016-10-14 14:07:11 -07:00
Yann Collet
d4cda27b63 new command -M#, to limit memory usage during decompression (#403) 2016-10-14 13:32:20 -07:00
Yann Collet
714464f05d fixed : cli : forgotten mandatory argument 2016-09-21 16:05:03 +02:00
Yann Collet
27b5ac666e Implemented "command must be followed by argument" protection 
suggested by @terrelln (#375)
 2016-09-21 14:31:22 +02:00
Yann Collet
3ecbe6a37c fileio uses ZSTD_resetDStream() 2016-09-14 17:26:59 +02:00
Yann Collet
b3060f7a9e changed streaming decoder behavior : now, when all compressed frame is consumed, it means decompression is completed, with regenerated data fully flushed. 2016-09-09 16:44:16 +02:00
Yann Collet
c843142ad0 zstd -d writes to stdout when input is stdin 2016-09-01 15:05:57 -07:00
Yann Collet
bb93d77ab8 reinforced fix for huge files 2016-08-25 22:22:50 +02:00
Yann Collet
94376acfb0 fixed test on CentOS7 2016-08-25 19:09:21 +02:00
Yann Collet
24a738c668 removed TRAVIS_OS_NAME 2016-08-25 15:15:06 +02:00
Yann Collet
70e3b31306 fixed playtests on os-x 2016-08-23 01:18:06 +02:00
inikep
b7d3449d5d playTests.sh working with tests/ 2016-08-18 15:13:41 +02:00
inikep
6ca9b853a1 cmake project: fixed include directories 2016-08-18 14:42:28 +02:00
inikep
63e335704e updated scripts +x permissions 2016-08-18 14:04:57 +02:00
inikep
2868d565e4 test-related files moved to tests/ 2016-08-18 13:18:11 +02:00