Commit Graph

21 Commits

Author SHA1 Message Date
Nick Terrell
4359d21ad7 Merge two memset() calls into one 2016-11-14 17:52:51 -08:00
Nick Terrell
24701de877 Fix uninitialized memory read 2016-11-14 13:57:05 -08:00
Nick Terrell
f698ad6deb Merge remote-tracking branch 'upstream/dev' into fixes
* upstream/dev:
  added doc\zstd_manual.html
  added contrib\gen_html
  zstd_compression_format.md moved to doc/
  Fix small bug in ZSTD_execSequence()
  improved ZSTD_compressBlock_opt_extDict_generic
  protect ZSTD_decodeFrameHeader() from invalid usage, as suggested by @spaskob
  zstd_opt.h: small improvement in compression ratio
  improved dicitonary segment merge
  use implicit rules to compile zstd_decompress.c
  detect early impossible decompression scenario in legacy decoder v0.5
  no repeat mode in legacy v0.5
  fixed invalid invocation of dictionary in legacy decoder v0.5
  fix edge case
  fix command line interpretation
  fixed minor corner case
  zstd.h: added the Introduction section
  fixed clang 3.5 warnings
  zstd.h: updated comments
2016-10-24 13:10:13 -07:00
Nick Terrell
ae1cb3b3d0 Fix small bug in ZSTD_execSequence()
`memmove(op, match, sequence.matchLength)` is not the desired behavior.
Overlap is allowed, and handled as if we did `*op++ = *match++`, which
is not how `memmove()` handles overlap.

Only triggered if both of the following conditions are met:
* The match spans extDict & currentPrefixSegment
* `oLitEnd <= oend_w < oLitEnd + length1 < oMatchEnd <= oend`.

These two conditions imply that the block is less than 15 bytes long.
This bug isn't triggered by the streaming API, because it allocates
enough space for the window size + the block size, so there cannot be
a match that is within 8 bytes of the end and overlaps with itself.
It cannot be triggered by the block decompression API because all of
the decompressed data is in the currentPrefixSegment.

Introduced by commit 7158584399
2016-10-21 12:13:44 -07:00
Nick Terrell
d760529a05 Fix stack buffer overrun when weightTotal == 0
If `weightTotal == 0`, then `BIT_highbit32(weightTotal)` is
undefined behavior in the case that it calls `__builtin_clz()`.
If `tableLog == HUF_TABLELOG_ABSOLUTEMAX` then we will access one
byte beyond the end of the buffer.
2016-10-19 11:39:11 -07:00
Nick Terrell
bb68062c59 Unitialized memory read in ZSTD_decodeSeqHeaders()
Caused by two things:
1. Not checking that `ip` is in range except for the first byte.
2. `ZSTDv0{5,6}_decodeLiteralsBlock()` could return a value larger than `srcSize`.
2016-10-18 16:41:33 -07:00
Nick Terrell
4db751668f Fix buffer overrun in ZSTD_loadEntropy()
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadEntropy()`.  This caused `FSE_buildDTable(dctx->MLTable, ...)`
to overwrite the beginning of `dctx->hufTable`.

The benchmarks look good, there is no obvious performance regression:

  > ./zstds/zstd.opt.0 -i10 -b1 -e5 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 268.2 MB/s , 701.0 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 199.5 MB/s , 666.9 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 154.9 MB/s , 655.6 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 128.9 MB/s , 648.4 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  98.4 MB/s , 633.4 MB/s

  > ./zstds/zstd.opt.2 -i10 -b1 -e5 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 266.1 MB/s , 703.7 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 199.0 MB/s , 666.6 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 156.2 MB/s , 656.2 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 133.2 MB/s , 647.4 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  96.3 MB/s , 633.3 MB/s
2016-10-17 15:51:15 -07:00
Nick Terrell
ccfcc643da Check if dict is empty before reading first byte 2016-10-17 11:46:03 -07:00
Nick Terrell
7158584399 Fix ZSTD_execSequence() edge case 2016-10-12 10:05:26 -07:00
Christophe Chevalier
dc245e91cb Changed to use ZSTDLIBv06_API and ZSTDLIBv07_API for DLL exports to fix warning
- changed name to prevent collision with ZSTDLIB_API used by non-legacy dll exports
2016-09-23 17:09:36 +02:00
inikep
8161e7321a unified error codes for legacy decoders 2016-09-05 12:29:51 +02:00
Yann Collet
1563bfeabc fixing FORCE_INLINE for older compilers (#330) 2016-09-02 11:44:21 -07:00
Yann Collet
4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
Yann Collet
5f53b0335e fixed continuation context 2016-08-28 10:00:49 -07:00
Yann Collet
87c18b2ebd fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
inikep
57ef4b1a0d zstd_v07.c: removed unused macros 2016-08-24 17:16:56 +02:00
inikep
a7bb322a93 removed never referenced functions 2016-08-18 10:30:21 +02:00
inikep
48849f86f0 fixed compilation with Intel Compiler with Windows 2016-08-10 14:26:35 +02:00
Yann Collet
7adc2328a3 fixed --test on zero-length files, reported by @amnilsson 2016-07-26 15:49:24 +02:00
inikep
4178f5c289 fixed gcc warning: always_inline function might not be inlinable 2016-07-25 21:17:45 +02:00
inikep
fca90f8f60 legacy decoder for v0.7 format 2016-07-25 17:49:08 +02:00