Commit Graph

135 Commits

Author SHA1 Message Date
Shawn Landden
914d983879
fix unbounded range
I think you meant 8 MiB or smaller, instead of an unbounded (and illogical) range
2017-12-21 16:15:12 -08:00
Yann Collet
78de28239f minor readme formatting update 2017-12-15 13:26:39 -08:00
Yann Collet
2cff66b62f version bump to v1.3.3 2017-12-14 16:11:20 -08:00
Yann Collet
bd88f633ac zstreamtest : in -T#s, s considered a suffix meaning "seconds"
avoid unintentionnally triggering `seedset`,
so that seed gets automatically determined when not set.
2017-11-27 12:15:23 -08:00
Yann Collet
4154aec679 fixed comment, as suggested by @terrelln 2017-11-21 10:26:17 -08:00
Yann Collet
fccb46fbe0 minor spelling fixes 2017-11-18 11:28:00 -08:00
Yann Collet
ca1a9ebac5 fixed zlib wrapper
it was invoking ZSTD_initCStream_advanced() with pledgedSrcSize==0 and contentSizeFlag=1
which means "empty"
while the intention was to mean "unknown".

The contentSizeFlag==1 is new, it is a consequence of setting this value to 1 by default.

The solution selected here is to pass ZSTD_CONTENTSIZE_UNKNOWN to mean "unknown".
So contentSizeFlag remains set (it wasn't in previous versions).
2017-10-18 11:22:23 -07:00
Yann Collet
5eed8e7a55 changed API comments to invite using macro ZSTD_CONTENTSIZE_UNKNOWN
to mean "pledgedSrcSize is not known at init time" instead of `0`.
Note that, a few prototypes created and documented with `0` to mean "unknown" still interpret "0" as unknown,
to avoid breaking 3rd party applications which depend on this behavior.
But this value is no longer recommended to mean "unknown".

In some future version, it might be possible to switch "0" to mean "empty",
as is already the case for several prototypes.
The advantage is that pledgedSrcSize field would have same behavior accross entire API,
making it easier to reason about.

Note that all concerned prototypes belong to the "experimental" API section.

srcSize is controlled at end of compression,
so if someone uses "0" to mean "unknown" while it effectively means "empty",
this is immediately caught by the compression function, which generates an error code : ZSTD_ERROR_srcSize_wrong
2017-10-14 00:32:06 -07:00
Nick Terrell
6d8778d8e7 [doc] Add images for release notes 2017-10-09 16:44:31 -07:00
Yann Collet
7f6a783862 fixed a small error in decodeCorpus
a compressed block must be strictly smaller than its decompressed size.
2017-10-07 15:19:52 -07:00
Yann Collet
ecf1778e23 updated ZSTD_format_e value validation
also updated manual
2017-09-27 11:19:21 -07:00
Yann Collet
52a1d1c6dc added ZSTD_DCtx_reset() 2017-09-25 16:56:48 -07:00
Yann Collet
f2a913862c added ZSTD_decompress_generic_simpleArgs() 2017-09-25 15:46:34 -07:00
Yann Collet
6ee05a02b8 added ZSTD_decompress_generic()
same as ZSTD_decompressStream(),
just for a similar feeling as the compression side, which uses ZSTD_compress_generic()
2017-09-25 15:41:48 -07:00
Yann Collet
96f0cde31a minor function rename
ZSTD_estimateCStreamSize_advanced_usingCParams -> ZSTD_estimateCStreamSize_usingCParams
_usingX is clear.
_advanced feels redundant
2017-09-24 16:47:02 -07:00
Yann Collet
7c3dea42ce added prototypes for advanced parameters for decompression API
required to decode custom formats
2017-09-24 15:57:29 -07:00
Yann Collet
086b9597d9 added ability to split input files for dictionary training
using command -B#
This is the same behavior as benchmark module,
which can also split input into arbitrary size blocks, using -B#.
2017-09-14 16:45:10 -07:00
Yann Collet
f1571dad8f Merge pull request #838 from stellamplau/ldm-mergeDev
Add long distance matcher
2017-09-13 13:24:08 -07:00
Yann Collet
ce31004f20 fix following suggestions by @terrelln 2017-09-11 13:12:52 -07:00
Yann Collet
058ed2ad33 ZSTD_decodingBufferSize_min()
supporting function for bufferless streaming API (ZSTD_decompressContinue())
makes it possible to correctly size a round buffer for decoding using this API.

also : added field blockSizeMax within ZSTD_frameHeader,
as it's a necessary information to know when to restart at beginning of decoding buffer.
2017-09-09 01:03:29 -07:00
Stella Lau
eeff55dfa8 Merge remote-tracking branch 'upstream/dev' into ldm-mergeDev 2017-09-06 15:56:32 -07:00
Stella Lau
8c33cfe0bc Add ldm documentation in README 2017-09-06 15:21:01 -07:00
Yann Collet
0c314cde4b updated zstd API manual
for new CCtxParams object
2017-08-31 18:28:19 -07:00
Yann Collet
b0cb081dc8 last batch of header files changed to reflect new license (#825)
only remains to update contrib/linux-kernel (@terrelln)
2017-08-31 12:20:50 -07:00
Yann Collet
e21384fffb fixed more file headers after license change (#825) 2017-08-31 12:11:57 -07:00
Yann Collet
e8d35cc5e9 minor formulation change, recommended by @ulikunitz 2017-08-20 10:39:20 -07:00
Yann Collet
d0d06e421f added alternative representation for huffman bistream 2017-08-19 12:26:09 -07:00
Yann Collet
8b12812147 fix #803 : wrong example in huffman bitstream section, reported by @ulikunitz 2017-08-19 12:17:57 -07:00
Paul Cruz
bef5eda8d9 const vars, change copy_literals() to only take size_t literal_length 2017-08-16 11:11:52 -07:00
Paul Cruz
8d3f18af2c renamed IO functions for clarity 2017-08-14 17:51:51 -07:00
Paul Cruz
6aebcfa0bc added comments for new functions 2017-08-14 15:11:01 -07:00
Paul Cruz
9d56c21279 added separate function for executing match copy command 2017-08-14 15:06:03 -07:00
Paul Cruz
d3e57db0bd created separate function for offset computation 2017-08-14 14:20:12 -07:00
Paul Cruz
b6d6be58c9 created separate function for copying literals during sequence execution 2017-08-14 14:05:16 -07:00
Paul Cruz
93c1309fd4 added free to free_dictionary() 2017-08-14 13:08:30 -07:00
Paul Cruz
38f4e43381 added error checking for dictionary initialized with null src 2017-08-14 09:41:04 -07:00
Paul Cruz
7ef9c6f4b2 made separate API for dictionary management 2017-08-11 18:40:19 -07:00
Paul Cruz
bfc6db8d6a exposed dictionary functions/types 2017-08-11 17:53:37 -07:00
Paul Cruz
bd308d806b remove debug symbols when cleaning, added a simple test 2017-08-11 14:42:15 -07:00
Paul Cruz
9f67e8652e fixed warnings shown by compiler 2017-08-11 14:41:44 -07:00
Paul Cruz
d0dc675596 add makefile 2017-08-11 14:35:13 -07:00
Paul Cruz
736a28d835 reduce educational decoder to single frame decompression 2017-08-11 14:34:49 -07:00
Yann Collet
132e6efd76 switched ZSTDMT_compress_advanced() last argument to overlapLog
overlapRLog (== 9 - overlapLog) was a bit "strange"
as all other public entry points use overlapLog
2017-07-13 02:22:58 -07:00
Yann Collet
40156a4967 bumped version nb to v1.3.1 2017-07-08 04:55:09 -07:00
Yann Collet
ead4dd48f6 new field frameHeader.headerSize 2017-07-07 15:51:24 -07:00
Yann Collet
990449b89d new field : ZSTD_frameHeader.frameType
Makes frame type (zstd,skippable) detection more straighforward.
ZSTD_getFrameHeader set frameContentSize=ZSTD_CONTENTSIZE_UNKNOWN to mean "field not present"
2017-07-07 15:21:35 -07:00
Yann Collet
e622330a3b extended frameHeader.windowSize to unsigned long long 2017-07-07 14:19:01 -07:00
Yann Collet
7758ed8458 fixed fullbench, part 2 2017-07-06 02:48:00 -07:00
Yann Collet
037466245f refactor ZSTD_check_compressionLevel_monotonicIncrease_memoryBudget()
use less macro statements
the initial version was meant to work with STATIC_ASSERT
but since it doesn't work and needs assert()
it's possible to rewrite it using normally compiled code
which is better for compiler.

Downside : the error message is less precise.
There is a DEBUGLOG(3,) to compensate.
2017-06-28 20:24:08 -07:00
Yann Collet
2bf428df45 Merge branch 'advancedAPI2' into refPrefix 2017-06-28 16:35:49 -07:00