Yann Collet
2ecd34ee5e
fixed unused variables warnings
2017-08-19 01:23:49 -07:00
Yann Collet
23706fb743
updated doc on compilation variables
2017-08-19 01:14:36 -07:00
Yann Collet
9203003d5f
fixed zstd-nolegacy
...
and added it to allVariants for CI testings
2017-08-19 01:01:53 -07:00
Yann Collet
4b387729b6
fixed zstd-small
...
and added it to shortest for CI tests
2017-08-19 00:48:29 -07:00
Yann Collet
7729ab83bb
Merge branch 'dev' into variants
2017-08-19 00:37:06 -07:00
Yann Collet
32e943b3ef
Merge branch 'dev' of github.com:facebook/zstd into dev
2017-08-19 00:36:37 -07:00
Yann Collet
582e19b01c
Merge pull request #801 from facebook/newLicense
...
New license
2017-08-19 00:34:34 -07:00
Yann Collet
288e263ec4
added zstd-decompress to the list of variants tested
2017-08-18 18:39:39 -07:00
Yann Collet
166645e7b3
fixed zstd-compress
...
file-information is dependent on decompression functions.
it should only be enabled when ZSTD_NODECOMPRESS is not set.
also : added zstd-compress compilation test into `make shortest`
2017-08-18 18:30:41 -07:00
Yann Collet
f207b39f55
blindfix for Windows conversion warning
...
long type is 32-bits on Windows 64,
while it's 64-bits on Unix.
64-to-32 shortening conversion for long is a specific Windows issue.
2017-08-18 17:06:12 -07:00
Yann Collet
32fb407c9d
updated a bunch of headers
...
for the new license
2017-08-18 16:52:05 -07:00
Yann Collet
88d2f72df9
fixed --list command in presence of special blocks
...
block type RLE is special, compressed size is always 1.
block type 3 is "reserved", aka not supported.
2017-08-18 16:18:20 -07:00
Yann Collet
c523c93b26
improved and fixed --list command, original patches by @ib ( #772 )
...
accepts all skippable frame identifiers.
display in MB or KB, depending on frame size.
fixed combination of skippable and zstd frames.
2017-08-18 15:57:53 -07:00
Yann Collet
3f54d788e9
removed --list from cli help (-h), reported by Agostino Sarubbo (@asarubbo) ( #800 )
...
redundant with shorter -l.
both -l and --list do the same thing, and are documented in man page.
2017-08-18 15:15:31 -07:00
codicodi
0fb7b23fcc
fix typo in lz4 support code
2017-08-14 14:04:24 +02:00
Nick Terrell
d0b27483ae
[zstdcli] Fix -t in streaming mode
2017-07-18 14:45:49 -07:00
Yann Collet
3a60efd3a9
policy change : ZSTDMT automatically caps nbThreads to ZSTDMT_NBTHREADS_MAX ( #760 )
...
Previously, ZSTDMT would refuse to create the compressor.
Also : increased ZSTDMT_NBTHREADS_MAX to 256,
updated doc,
and added relevant test
2017-07-13 10:17:23 -07:00
Yann Collet
3345a91964
cli : use new advanced API by default
2017-07-05 17:34:15 -07:00
Yann Collet
b8280fe929
minor code refactoring
2017-07-03 15:14:55 -07:00
Yann Collet
368b974387
fixed gz error reporting
2017-07-03 13:47:46 -07:00
Yann Collet
c9f21c8a71
gzip : only display inflateEnd error when there is no error before
2017-07-03 13:45:09 -07:00
Yann Collet
6c35112d08
gzip decoding does no longer exit() on invalid input
2017-07-03 13:24:50 -07:00
Yann Collet
e97ff3b54a
decoding malformed lz4 frames does no longer exit()
2017-07-03 11:27:29 -07:00
Yann Collet
c34185df13
fileio : decoding malformed lzma frame does no longer exit()
...
makes it possible to continue decoding file list
2017-07-03 10:27:16 -07:00
Yann Collet
1d39550471
Merge pull request #744 from terrelln/dev
...
[pzstd] Remove appveyor tests
2017-07-01 05:58:33 -07:00
Nick Terrell
56e3964d85
[man] Specify that strategies start at 1
2017-06-30 16:29:37 -07:00
Yann Collet
811deaea6f
Merge pull request #736 from terrelln/cover-default-api
...
[zdict] Make COVER the default algorithm
2017-06-28 20:25:36 -07:00
Nick Terrell
5b7fd7c422
[zdict] Make COVER the default algorithm
2017-06-26 21:09:22 -07:00
Nick Terrell
d88a441406
[zstdcli] Document HAVE_LZMA and zstd-noxz
2017-06-26 11:25:00 -07:00
Nick Terrell
6aeb50ea0d
[zstdcli] Add unlzma and unxz symlinks
2017-06-26 11:24:36 -07:00
Nick Terrell
849ecf3510
[zstdcli] Support xz by default when liblzma is available
2017-06-23 17:11:38 -07:00
Yann Collet
ef269c1b68
Merge pull request #725 from facebook/advancedAPI2
...
New Advanced API
2017-06-23 09:50:47 -07:00
Yann Collet
991f9dfcde
switched fileio.c to ZSTD_DEBUG trigger macro
2017-06-21 15:16:13 -07:00
Paul Cruz
5be0f5544e
added newline to align output
2017-06-21 12:41:10 -07:00
Paul Cruz
6f5fe71041
added error check for when file could not be opened
2017-06-21 12:37:23 -07:00
Paul Cruz
74a725da69
reversed calculation of ratio
2017-06-21 12:27:20 -07:00
Yann Collet
7bd1a2900e
added ZSTD_dictMode_e to control dictionary loading mode
2017-06-21 11:50:33 -07:00
Paul Cruz
a73c2a444a
added tests with null files, changed condition to check that the file is more than 0 bytes
2017-06-20 14:33:08 -07:00
Yann Collet
b44ab82f7a
ensure new ZSTD_strategy starts at value 1
2017-06-20 14:11:49 -07:00
Paul Cruz
0765602903
fixed error where extremely small files were not being detected as not compressed with zstd
2017-06-20 14:04:46 -07:00
Paul Cruz
3701dde3ba
changed to use LONG_SEEK
2017-06-20 13:44:05 -07:00
Paul Cruz
93a5510962
added progress metric to display
2017-06-20 13:26:25 -07:00
Paul Cruz
bda57754a2
moved signifcant operations to fileio.c
2017-06-20 12:43:10 -07:00
Paul Cruz
c594507871
removed double if statements for same condition
2017-06-20 11:59:45 -07:00
Paul Cruz
72e0ac9929
minor update to logic
2017-06-20 11:58:27 -07:00
Paul Cruz
cea55a9217
size_t const value
2017-06-20 11:56:11 -07:00
Paul Cruz
bed0dc95bc
changed displayInfo alignment and const
2017-06-20 11:47:24 -07:00
Paul Cruz
cb2dbe6434
changed fileInfo initialization, updated error code
2017-06-20 09:35:21 -07:00
Yann Collet
c08e649e95
first implementation of bench.c with new API ZSTD_compress_generic()
...
Doesn't speed optimize this buffer-to-buffer scenario yet.
Still internally defers to streaming implementation.
Also : fixed a long standing bug in ZSTDMT streaming API.
2017-06-19 18:25:35 -07:00
Yann Collet
fe234bf48b
fix attempts : fullbench for VS2008
2017-06-19 15:23:19 -07:00
Paul Cruz
55b9b28dad
made style changes, added description in man file
2017-06-19 09:55:55 -07:00
Paul Cruz
4a0e4c856c
fixing more spacing on loops
2017-06-15 18:08:22 -07:00
Paul Cruz
b3e62446c6
added in correct error return from main in zstdcli.c
2017-06-15 17:46:49 -07:00
Paul Cruz
0d7c4d766a
initialized info manually to avoid error
2017-06-15 17:30:18 -07:00
Paul Cruz
d3b34e4a25
changed spacing on if statements
2017-06-15 17:12:41 -07:00
Paul Cruz
b316691f91
added const
2017-06-15 17:03:59 -07:00
Paul Cruz
60a2e55e2e
added case for when file is not compressed with zstd (incorrect magic number)
2017-06-15 17:00:59 -07:00
Paul Cruz
acaefb531b
refactoring brackets to allow for const values
2017-06-15 16:27:38 -07:00
Paul Cruz
0757eae6ff
rearranging code to allow for consts
2017-06-15 16:12:04 -07:00
Paul Cruz
e49afae2ab
changed implicit precision loss again
2017-06-15 16:03:40 -07:00
Paul Cruz
54e0b6c66b
got rid of imlpicit precision loss
2017-06-15 16:01:52 -07:00
Paul Cruz
b766211e10
made another function static
2017-06-15 15:55:49 -07:00
Paul Cruz
e7f02fc58a
closed file at end of function, created another variable to keep exit points simpler
2017-06-15 15:53:02 -07:00
Paul Cruz
8b3ff7c9bc
changed info to static allocation
2017-06-15 15:35:37 -07:00
Paul Cruz
96cabf7233
cleaned up if statements
2017-06-15 15:25:20 -07:00
Paul Cruz
9437cc7498
added --list command
2017-06-15 15:07:54 -07:00
Paul Cruz
4495e9a826
allow analysis even if file does not have zst extension
2017-06-15 15:02:54 -07:00
Paul Cruz
a9b77c83e5
cleaning up code for analyzing frames
2017-06-15 14:13:28 -07:00
Paul Cruz
e208992529
cleaning up code
2017-06-15 12:27:32 -07:00
Paul Cruz
fc428ab350
changing formatting again
2017-06-14 13:26:19 -07:00
Paul Cruz
607ee9f4bc
print message when no files are given, rearrange code for readability
2017-06-14 13:23:36 -07:00
Paul Cruz
ebd60349f5
changed U64 to unsigned long to get rid of build test errors
2017-06-14 13:11:18 -07:00
Paul Cruz
81fa33b55a
cleaning up code
2017-06-13 10:06:49 -07:00
Paul Cruz
a56dcbfcf8
changed print format for non-verbose version
2017-06-12 17:47:33 -07:00
Paul Cruz
c828b52111
yet more refactoring
2017-06-12 17:19:53 -07:00
Paul Cruz
28dbf8ee6c
more refactoring for C90
2017-06-12 17:09:08 -07:00
Paul Cruz
618a7b66e2
refactoring for C90 syntax
2017-06-12 16:53:50 -07:00
Paul Cruz
6e33c74133
formatting issues resolved
2017-06-12 16:29:00 -07:00
Paul Cruz
173a739717
code refactorings with explicit conversion
2017-06-12 16:14:04 -07:00
Paul Cruz
f3d6ab28a2
formated information for verbose output
2017-06-12 15:59:28 -07:00
Paul Cruz
6e0204470a
displayed decompressed size
2017-06-12 15:51:59 -07:00
Paul Cruz
dcd6ba6dc6
incremented decompressedSize instead of setting value
2017-06-12 15:40:47 -07:00
Paul Cruz
6996bd2598
removed useless lines
2017-06-12 15:24:50 -07:00
Paul Cruz
9cb602ee25
added in logic for parsing through blocks/frames
2017-06-12 15:22:48 -07:00
Paul Cruz
786b7cac27
added code to analyze the first frame header
2017-06-12 13:46:39 -07:00
Paul Cruz
a3d54cf73d
added line spacing for clarity
2017-06-12 10:58:34 -07:00
Yann Collet
bb0aaf9579
minor man update on -B# option in benchmark mode
2017-06-12 05:19:15 -07:00
Yann Collet
9e6a2eaab6
added MT support to NEWAPI
2017-06-11 18:39:46 -07:00
Paul Cruz
0f06f4f266
added display for compressed size
2017-06-06 09:21:42 -07:00
Paul Cruz
4128f67a4a
added in check suffix check to ensure file was compressed with zstd
2017-06-05 15:00:06 -07:00
Paul Cruz
901435e9ef
setup basic functions for adding --list functionality
2017-06-05 14:45:31 -07:00
cyan4973
8bcbf42617
fixed g++ prototype mismatch
2017-06-04 23:52:00 -07:00
Yann Collet
6d4fef36de
Added ZSTD_compress_generic()
...
Used in fileio.c (zstd cli).
Need to set macro ZSTD_NEWAPI to trigger it.
2017-05-17 18:36:15 -07:00
Yann Collet
23c256e44b
removed useless variable from CCtx
...
CStream's pledgedSrcSize is no longer necessary
srcSize control is realized within bufferless interface.
2017-05-16 18:10:11 -07:00
Yann Collet
e12ae02357
minor coding style update
...
reduced some long lines
2017-05-16 17:32:33 -07:00
Yann Collet
2d4d31c18a
removed gcc compilation flag -Wbad-function-cast
...
It makes it more difficult to directly cast the result of a function,
requiring to store the result in an intermediate variable.
It does not necessarily help readability,
and this restriction can be difficult to overcome in some constructions,
like some macros.
also : fixed minor Visual conversion warnings in datagencli.c
2017-05-16 11:34:38 -07:00
Yann Collet
46ac9ade9b
fixed vla warning on linux
2017-05-15 18:15:08 -07:00
Yann Collet
83d0c764dc
added several compilation flags
2017-05-15 17:15:46 -07:00
Nick Terrell
a99ece9e79
Update zstd.1.md to use btultra
2017-05-10 16:51:18 -07:00
Nick Terrell
5f2c7213c7
Merge remote-tracking branch 'upstream/dev' into btopt
...
* upstream/dev: (305 commits)
added test for ZSTD_estimateCStreamSize()
changed variable name, for clarity
fixed ZSTD_estimateCStreamSize()
shortened ZSTD_createCStream_Advanced()
fixed symbols test
added ZSTD_estimateDStreamSize()
changed name frameParams into frameHeader
regroup memory usage function declarations
separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
bumped version number
added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
Updated ZSTD_freeCCtx()
updated ZSTD_estimateCCtxSize()
Updated ZSTD_sizeof_CCtx()
merged CCtx and CStream as a single same object
cli : -d and -t do not stop after a failed decompression
added dev branch CircleCI badge
added dev branch Appveyor badge
keep dev branch status only
creates a binary archive without the `programs` directory
...
2017-05-10 16:49:58 -07:00
Yann Collet
01a1abfdb5
cli : -d and -t do not stop after a failed decompression
...
The problematic srcfile will be named on console/log,
but decompression/test will continue onto next file in the list.
2017-05-05 19:15:24 -07:00
Yann Collet
a00e9599f1
removed -g from DEBUGFLAGS
...
It inflates binary sizes, which is negative for the Windows build.
It also makes it impossible to check if 2 different source codes
get nonetheless compiled to the same binary,
since checksum will be different, due to integrated source code.
2017-05-04 17:24:29 -07:00
Yann Collet
c08e56861e
updated dict graphs to 2D mode
2017-05-03 14:33:28 -07:00
Yann Collet
710497d8ea
updated programs/README.md, to introduce compilation variables
...
make it possible to enable/disable features individually
2017-05-02 17:18:24 -07:00
Yann Collet
f47284fec0
reorganized Makefile for multiple targets
2017-05-02 16:55:57 -07:00
Yann Collet
3791d2105d
added xzstd4 target
...
support for all formats, xz/lzma/lz4 included
2017-05-02 16:38:37 -07:00
Yann Collet
2e63a877f3
fixed xzstd
...
--format=xz was missing a break, making the execution continue into lz4 error message
2017-05-02 15:40:42 -07:00
Nick Terrell
f376d47c11
[CLI] Switch dictionary builder on CLI to cover
2017-05-02 11:18:27 -07:00
Nick Terrell
020b960e13
[cover] Make optimization faster
2017-05-02 11:02:48 -07:00
Nick Terrell
f2d9ef1dc0
[cover] Optimize case where d <= 8
2017-05-02 11:02:43 -07:00
Nick Terrell
feba969a69
Fix LZ4_MSG in xzstd
2017-05-02 11:02:16 -07:00
Yann Collet
31533bacce
Changed ZSTD_createCDict_advanced()
...
It now only uses compressionParameters as argument.
It produces many changes throughout user code,
though hopefully they tend to be simple :
just provide the cParams part from existing ZSTD_parameters.
Some programs might depend on ZSTD_createCDict_advanced() to pass frame parameters.
This change will force them to revisit this strategy and fix it,
since frame parameters are effectively silently ignored in current version.
2017-04-27 00:29:04 -07:00
Sean Purcell
eab41c1872
Fix LZ4 wrapper deprecation warnings
2017-04-26 10:17:38 -07:00
Sean Purcell
2c4b6fe6b3
Make lz4 compression/decompression compatible with library r123
2017-04-25 11:00:54 -07:00
Sean Purcell
4de8632957
Add LZ4 compress/decompress support to CLI
2017-04-24 16:48:25 -07:00
Yann Collet
230d7acc7d
cli : add support for --threads=# command
...
updated documentation
add relevant test case
2017-04-21 11:38:13 -07:00
Sean Purcell
9606256a8d
Fix no thread message
2017-04-18 13:52:00 -07:00
Yann Collet
1c286950af
Merge pull request #663 from iburinoc/zstdmt-default
...
Compile CLI using files instead of objs
2017-04-18 02:59:54 -07:00
Sean Purcell
c8b2df7d62
Compile CLI using files instead of objs
...
This avoids conflicts between how the library was
configured and how the CLI was configured.
2017-04-17 17:13:47 -07:00
Sean Purcell
e6c504dbe6
Update -T0 comment in man page
2017-04-17 17:12:02 -07:00
Sean Purcell
5935c990a0
Add zstdmt and -T0 to man page
2017-04-17 16:05:20 -07:00
Sean Purcell
f6ef4db20e
Install zstdmt even without threading support
2017-04-17 12:21:11 -07:00
Sean Purcell
5a61f36474
Make zstd compile with mt by default
2017-04-17 12:06:02 -07:00
Baptiste Daroussin
7dd14d03b0
Enable multithreading on BSD
2017-04-15 16:25:08 +02:00
Sean Purcell
e4f3235c85
Add 0 initializers to static variables
2017-04-13 16:34:28 -07:00
Sean Purcell
42bac7fa84
Change ifndef's to undef's
2017-04-13 15:35:05 -07:00
Sean Purcell
9227aae001
Fix clang linux compilation
2017-04-13 14:06:40 -07:00
Sean Purcell
3b6207d4bd
Fix compilation on windows
2017-04-13 14:04:10 -07:00
Sean Purcell
f876f1200c
Fix compilation on macOS
2017-04-13 12:33:45 -07:00
Sean Purcell
afa48518e2
-T0 detects number of physical cores
2017-04-13 12:28:28 -07:00
Yann Collet
5c42d0edc8
cli : better status display for zstdmt in 1-thread mode
2017-04-11 16:57:32 -07:00
Yann Collet
0e30059ba1
cli : FIO_createDictBuffer() replaces FIO_loadFile()
...
makes it more explicit that it allocates a buffer
and that it's meant to be used for dictionary.
Also : simplified function a bit,
now only works for dictionaries up to DICTSIZE_MAX
2017-04-11 14:41:02 -07:00
Sean Purcell
279be2015b
Let zstd overwrite read-only files
2017-04-06 12:56:40 -07:00
Yann Collet
c2007388a5
fixed bench.c : optional advanced parameters applied
...
before creating cdict
2017-04-04 15:35:06 -07:00
Yann Collet
81d6380139
minor bench.c adjustments
...
shorter debug messages
no need to check decompressedLength==0 twice
2017-04-04 15:21:09 -07:00
Yann Collet
a36330a27d
minor man update, for overlapSize
2017-04-03 14:57:24 -07:00
Yann Collet
805c5a3efb
updated documentation on multithreading modes
2017-04-01 00:36:31 -07:00
Nick Terrell
eb38617175
Clean up default sparse logic
2017-03-31 15:20:50 -07:00
Nick Terrell
96fe545a18
Turn off sparse mode for OS X by default
2017-03-31 15:16:43 -07:00
Nick Terrell
5152fb2cb2
Convert all tabs to spaces
2017-03-29 18:51:58 -07:00
Yann Collet
5bde4be5a1
fix : bench automatically adapts parameters to srcSize
2017-03-29 12:10:38 -07:00
Sean Purcell
efdaf8bb7c
Fix inline compile errors
2017-03-27 12:26:40 -07:00
Sean Purcell
894bf49713
Fix IS_CONSOLE returning 1 for NUL on windows
2017-03-27 12:19:30 -07:00
Yann Collet
0246d5c531
Merge pull request #630 from facebook/advancedCliCommands
...
changed advanced commands --maxdict= and --dictID=
2017-03-26 00:13:35 -07:00
Sean Purcell
ef30af843f
Ignore extension in command name matching
2017-03-24 17:06:09 -07:00
Yann Collet
96aa3019b2
changed advanced commands --maxdict= and --dictID=
...
now works with the `=` variant, which is the recommended one.
Old variant `--dictID #` still works, for compatibility with existing scripts.
Long term objective is to remove the old variant..
2017-03-24 16:04:29 -07:00
Yann Collet
f332ece468
dictBuilder fails to create dictionary on certain input
...
Properly expressed with an error code (see zstd_errors.h)
and a cli return code != 0
2017-03-23 16:24:02 -07:00
Sean Purcell
dee08caabb
Fix displayLevel redefinition issues
2017-03-23 12:09:35 -07:00
Sean Purcell
680e4e0953
Ignore symbolic links unless --force specified
2017-03-23 11:52:09 -07:00
Sean Purcell
042ba122ae
Change g_displayLevel to int and fix DISPLAYUPDATE flush
2017-03-23 11:21:59 -07:00