AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7,330 Commits 1 Branch 0 Tags 27 MiB
d1cc9d2797
Commit Graph

73 Commits

Author SHA1 Message Date
Shashank Tavildar
0f2bff2faf Addressing comments, removing cyclic dependency with header file, updating tests 2019-10-28 18:21:47 -07:00
W. Felix Handte
4eccc82875 Minor Fixes 2019-10-25 13:59:19 -04:00
W. Felix Handte
91c3f545cc Fix Build; Refactor 2019-10-24 20:18:57 -04:00
Felix Handte
506e1a1176
Merge branch 'tzst-ext-tmp' into tzst-ext 2019-10-24 17:49:34 -04:00
Yann Collet
0a24d4ef18 removed regular file test on Windows 
since it does not work well on this platform
(tested with MinGW).

Note : could be an issue within UTIL_isRegularFile()
 2019-10-17 16:39:47 -07:00
Yann Collet
0ee360982d improved test mode -t 
The test mode do no longer open a file `/dev/null` nor write anything to output.

This is supposed to be more efficient than writing to `/dev/null`,
and more universal too : the previous method was failing on Windows.
 2019-10-17 16:09:53 -07:00
Yann Collet
1795133c45 refactored FIO_compressMultipleFilenames() prototype 
for consistency
 2019-10-17 15:32:03 -07:00
Sen Huang
64bc441d7d Now constructs final destination path without allocating new table 2019-10-03 13:53:04 -04:00
Sergey Ponomarev
59f369a6da Add short tar's extensions .tgz (.tar.gz), .txz (.tar.xz), .tzst (.tar.zst) 2019-09-14 21:30:15 +03:00
Sen Huang
30bff50e06 fixes for tests and segfault 2019-09-06 11:10:53 -07:00
Sen Huang
6beb3c0159 added decompression support 2019-09-05 17:56:24 -07:00
Sen Huang
7f98b46876 adding support for -O flag: multiple files into one directory for compressions (decompression to come) 2019-09-05 16:03:35 -07:00
Nick Magerko
493f95c7df Fix merge conflicts 2019-08-22 11:51:41 -07:00
Nick Magerko
dffbac5f89 Add --size-hint=# option 2019-08-19 11:38:49 -07:00
Nick Magerko
af0c9501d1 Add --stream-size=# command 2019-08-16 11:34:39 -07:00
Ephraim Park
9007701670 Adding targetCBlockSize param 2019-07-03 15:41:52 -07:00
Nick Terrell
0c53c5ad4a [zstdcli] Add a flag to control literals compression 2019-02-15 15:00:45 -08:00
Yann Collet
07e047873e fixed fileio.c compilation with LZ4 enabled 
was broken by #1505.

I'm surprised it passed CI tests.
LZ4 tests are part of the "Extended" tests on Travis CI,
which are run on "master" and in "cron" jobs.

Since latest cron job did not failed,
especially this one : https://travis-ci.org/facebook/zstd/jobs/484365040
it suggests cron jobs are no longer using `dev` branch.

To be investigated
 2019-01-25 14:42:44 -08:00
Karl Ostmo
5e220bf4b5 Remove global parameters, pass into public functions instead 2019-01-23 16:06:27 -08:00
Nick Terrell
bdfcaecc0a [zstdcli] Add --no-progress flag 
The `--no-progress` flag disables zstd's progress bars, but leaves
the summary.

I've added simple tests to `playTests.sh` to make sure the parsing
works.
 2018-12-14 11:50:25 -08:00
Yann Collet
41c7d0b1e1 changed hashEveryLog into hashRateLog 2018-11-21 14:36:57 -08:00
Nick Terrell
f9a671ac37 [cli] Add rsyncable 2018-11-14 17:00:16 -08:00
Yann Collet
6c51bf420c bounds for --adapt mode 
can supply min and max compression level through advanced command :
--adapt=min=#,max=#
 2018-09-25 16:03:28 -07:00
Yann Collet
2f78228f65 Merge branch 'dev' into adapt 2018-09-19 12:43:42 -07:00
Casey McGinty
d4337b6f1d
Move ABRThandler func out of internal lib 2018-09-11 11:39:49 -07:00
Yann Collet
33f7709c71 fileio: changed parameter type from ptr to plain structure 
safer : this parameter is read-only,
we don't want original structure to be modified
 2018-08-13 13:02:03 -07:00
Yann Collet
e7a49c6683 introduced command --adapt 2018-08-11 20:48:06 -07:00
Yann Collet
209df52ba2 Changed nbThreads for nbWorkers 
This makes it easier to explain that nbWorkers=0 --> single-threaded mode,
while nbWorkers=1 --> asynchronous mode (one mode thread on top of the "main" caller thread).
No need for an additional asynchronous mode flag.
nbWorkers>=2 works the same as nbThreads>=2 previously.
 2018-02-01 19:29:30 -08:00
Nick Terrell
4680e85bdf Allow -o with multiple files 2017-12-13 17:44:34 -08:00
Stella Lau
eb3327c10a Merge branch 'dev' of https://github.com/facebook/zstd into ldm-mergeDev 2017-09-11 15:00:01 -07:00
Yann Collet
3128e03be6 updated license header 
to clarify dual-license meaning as "or"
 2017-09-08 00:09:23 -07:00
Stella Lau
67d4a6161c Add ldmBucketSizeLog param 2017-09-02 21:55:29 -07:00
Stella Lau
a1f04d518d Move hashEveryLog to cctxParams and update cli 2017-09-01 15:05:47 -07:00
Stella Lau
17d8e0bdcc Merge remote-tracking branch 'upstream/longRangeMatcher' into ldm-integrate 2017-09-01 10:19:38 -07:00
Stella Lau
6a546efb8c Add long distance matcher 
Move last literals section to ZSTD_block_internal
 2017-08-31 12:53:19 -07:00
Yann Collet
32fb407c9d updated a bunch of headers 
for the new license
 2017-08-18 16:52:05 -07:00
Paul Cruz
bda57754a2 moved signifcant operations to fileio.c 2017-06-20 12:43:10 -07:00
Paul Cruz
0f06f4f266 added display for compressed size 2017-06-06 09:21:42 -07:00
Paul Cruz
901435e9ef setup basic functions for adding --list functionality 2017-06-05 14:45:31 -07:00
Sean Purcell
4de8632957 Add LZ4 compress/decompress support to CLI 2017-04-24 16:48:25 -07:00
Nick Terrell
aa8bcf360f Add xz and lzma support. 
Finish feature started by @inikep.

* Add xz and lzma compression and decompression support to target `xzstd`.
* Fix bug in gzip decompression that silently accepted truncated files.
* Add gzip frame composition tests.
* Add xz/lzma compatibility tests.
* Add xz/lzma frame composition tests.
 2017-03-13 18:45:41 -07:00
Przemyslaw Skibinski
98509a70ac fixed function name 2017-02-14 09:23:32 +01:00
Przemyslaw Skibinski
cfd4dc299a add "--format=gzip" option 2017-02-08 15:17:55 +01:00
Yann Collet
6be2337c26 added command --block-size= 
for Multi-threading only.
alias : -B#
 2017-01-30 11:17:26 -08:00
Yann Collet
8dafb1acf5 CLI : automatically set overlap size to max (windowSize) for max compression level 2017-01-25 17:01:13 -08:00
Yann Collet
512cbe8c10 zstdmt cli and API allow selection of section sizes 
By default, section sizes are 4x window size.
This new setting allow manual selection of section sizes.
The larger they are, the (slightly) better the compression ratio,
but also the higher the memory allocation cost,
and eventually the lesser the nb of possible threads,
since each section is compressed by a single thread.

It also introduces a prototype to set generic parameters,
ZSTDMT_setMTCtxParameter()

The idea is that it's possible to add enums
to extend the list of parameters that can be set this way.
This is more long-term oriented than a fixed-size struct.
Consider it as a test.
 2017-01-24 17:08:53 -08:00
Yann Collet
500014af49 zstd cli can now compress using multi-threading 
added : command -T#
added : ZSTD_resetCStream() (zstdmt_compress)
added : FIO_setNbThreads()  (fileio)
 2017-01-19 17:04:28 -08:00
Przemyslaw Skibinski
8349d675e0 fileio.c: support advanced compression parameters 2016-12-13 13:24:59 +01:00
Yann Collet
743b33f57e fix zstdcat 2016-12-02 15:18:57 -08:00
Przemyslaw Skibinski
d0815583d9 Changed stdinmark and stdoutmark 2016-11-04 11:37:27 +01:00