Yann Collet
717c65d690
Merge pull request #519 from inikep/dev11
...
Dev11
2017-01-26 14:23:44 -08:00
Yann Collet
8dafb1acf5
CLI : automatically set overlap size to max (windowSize) for max compression level
2017-01-25 17:01:13 -08:00
Yann Collet
bb0027405a
fixed zstdmt corruption issue when enabling overlapped sections
...
see Asana board for detailed explanation on why and how to fix it
2017-01-25 16:25:38 -08:00
Yann Collet
943cff9c37
fixed zstdmt cli freeze issue with large nb of threads
...
fileio.c was continually pushing more content without giving a chance to flush compressed one.
It would block the job queue when input data was accumulated too fast (requiring to define many threads).
Fixed : fileio flushes whatever it can after each input attempt.
2017-01-25 12:35:19 -08:00
Przemyslaw Skibinski
5022a18d51
improved #232 fix
2017-01-25 13:11:26 +01:00
Przemyslaw Skibinski
64fa2dbc5e
Fixed https://github.com/facebook/zstd/issues/232
2017-01-25 13:02:33 +01:00
Yann Collet
512cbe8c10
zstdmt cli and API allow selection of section sizes
...
By default, section sizes are 4x window size.
This new setting allow manual selection of section sizes.
The larger they are, the (slightly) better the compression ratio,
but also the higher the memory allocation cost,
and eventually the lesser the nb of possible threads,
since each section is compressed by a single thread.
It also introduces a prototype to set generic parameters,
ZSTDMT_setMTCtxParameter()
The idea is that it's possible to add enums
to extend the list of parameters that can be set this way.
This is more long-term oriented than a fixed-size struct.
Consider it as a test.
2017-01-24 17:08:53 -08:00
Yann Collet
500014af49
zstd cli can now compress using multi-threading
...
added : command -T#
added : ZSTD_resetCStream() (zstdmt_compress)
added : FIO_setNbThreads() (fileio)
2017-01-19 17:04:28 -08:00
Przemyslaw Skibinski
7a8a03c20d
util.h: restore BSD license for Facebook Open-Source
2016-12-21 15:08:44 +01:00
Przemyslaw Skibinski
97a258d71d
updated comments
2016-12-21 14:00:41 +01:00
Przemyslaw Skibinski
e679741b18
_CRT_SECURE_NO_WARNINGS moved to util.h
2016-12-21 13:47:11 +01:00
Przemyslaw Skibinski
2f6ccee6af
platform.h: removed Compiler Options
2016-12-21 13:23:34 +01:00
Przemyslaw Skibinski
a35b9448ac
improved MinGW support
2016-12-21 11:18:45 +01:00
Przemyslaw Skibinski
20b089e53d
simplified zstdcli.c
2016-12-21 09:19:15 +01:00
Przemyslaw Skibinski
16ae6563a2
executables use new util.h and platform.h
2016-12-21 09:06:14 +01:00
Przemyslaw Skibinski
f8046b8e72
Merge remote-tracking branch 'refs/remotes/facebook/dev' into v112
...
# Conflicts:
# appveyor.yml
2016-12-19 08:20:26 +01:00
Przemyslaw Skibinski
b866e72826
tools use platform.h
2016-12-16 14:24:01 +01:00
Przemyslaw Skibinski
c71e552b2e
fixed "strategy" in advanced compression parameters
2016-12-13 20:04:32 +01:00
Przemyslaw Skibinski
8349d675e0
fileio.c: support advanced compression parameters
2016-12-13 13:24:59 +01:00
Przemyslaw Skibinski
7c6bbc3298
updated FIO_passThrough
2016-12-05 18:31:14 +01:00
Przemyslaw Skibinski
6b508b1770
updated test-gzstd
2016-12-05 18:02:40 +01:00
Przemyslaw Skibinski
b493e3b3d3
introduced srcBufferLoaded
2016-12-05 17:39:38 +01:00
Przemyslaw Skibinski
3c69760275
improved FIO_decompressGzFrame
2016-12-05 15:58:23 +01:00
Przemyslaw Skibinski
8489f184f6
improved detection of -lz
2016-12-05 13:47:00 +01:00
Yann Collet
743b33f57e
fix zstdcat
2016-12-02 15:18:57 -08:00
Yann Collet
5bd4237beb
minor refactor
2016-12-02 12:40:57 -08:00
Przemyslaw Skibinski
690753ea1a
improved formatting in FIO_decompressSrcFile
2016-12-02 16:20:16 +01:00
Przemyslaw Skibinski
4e49580407
removed testing artifacts
2016-12-02 15:19:00 +01:00
Przemyslaw Skibinski
c5eebca128
rewritten FIO_decompressGzFile
2016-12-02 15:01:31 +01:00
Przemyslaw Skibinski
b0f2ef2119
improved gzip_* functions
2016-12-02 13:50:29 +01:00
Przemyslaw Skibinski
4b504f131a
added gzip_open, gzip_close, gzip_read
2016-12-02 13:11:39 +01:00
Przemyslaw Skibinski
daaf754539
detect stream with ungetc
2016-12-01 13:29:19 +01:00
Przemyslaw Skibinski
19aad42ee1
added FIO_decompressGzFile
2016-12-01 11:56:31 +01:00
Przemyslaw Skibinski
abfb51f5f2
gzstd: decompresses .gz files
2016-11-30 15:05:54 +01:00
Przemyslaw Skibinski
0e14675df2
fileio.c: detect .gz files
2016-11-30 13:34:21 +01:00
Yann Collet
cdff19c4b3
minor comment change
2016-11-11 17:26:54 -08:00
Yann Collet
0018ca28dc
zstd cli : displays total decoded size, even when a stream consists of multiple frames
2016-11-07 14:42:37 -08:00
Przemyslaw Skibinski
1f60eb6b17
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
2016-11-03 11:38:04 +01:00
Przemyslaw Skibinski
26306fcacf
BMK_SetNbIterations renamed to BMK_SetNbSeconds
2016-11-03 11:38:01 +01:00
Yann Collet
179b19776f
fileio.c does no longer need ZSTD_LEGACY_SUPPORT, and does no longer depend on zstd_legacy.h
...
Added : ZSTD_isFrame() in experimental section
2016-11-02 17:30:49 -07:00
Przemyslaw Skibinski
fcf22e3473
set permissions, access and modification times
2016-11-02 14:08:07 +01:00
Przemyslaw Skibinski
a42794df61
preserve file modification time
2016-11-02 13:08:39 +01:00
Yann Collet
d4cda27b63
new command -M#, to limit memory usage during decompression ( #403 )
2016-10-14 13:32:20 -07:00
Nick Terrell
3e82f2b61a
Remove references to GPLv2 license
...
License headers added to Makefiles were taken from `zstd/Makefile`.
2016-10-11 15:28:40 -07:00
Yann Collet
993060e0f2
cli : better adaptation to small files
2016-09-21 16:46:08 +02:00
Yann Collet
43eeea4725
fileio uses ZSTD_resetCStream()
2016-09-15 15:38:44 +02:00
Yann Collet
3ecbe6a37c
fileio uses ZSTD_resetDStream()
2016-09-14 17:26:59 +02:00
Yann Collet
d77f2ceee6
Merge branch 'dev' of github.com:Cyan4973/zstd into dev
2016-09-12 03:14:03 +02:00
Yann Collet
5c6d244973
Merge branch 'dev' of github.com:facebook/zstd into dev
2016-09-07 14:54:54 +02:00
Yann Collet
ac8bace6b1
support large skippable frames
2016-09-07 14:54:23 +02:00
