Commit Graph

262 Commits

Author SHA1 Message Date
Yann Collet
7f40bb1c39 Merge pull request #742 from stellamplau/stack-space
Reduce stack usage of HUF_readDTableX4 and HUF_readDTableX2
2017-06-30 14:50:23 -07:00
Stella Lau
32df49e9f8 Fix typo 2017-06-30 12:56:24 -07:00
Stella Lau
b0513b519c Add comment to HUF_DECOMPRESS_WORKSPACE_SIZE 2017-06-30 12:53:56 -07:00
Stella Lau
99e315999c Reduce stack usage of HUF_readDTableX4 and HUF_readDTableX2 2017-06-29 11:49:59 -07:00
Yann Collet
33a6639039 fixed ZSTD_refPrefix with Multithread-enabled CCtx 2017-06-28 11:09:43 -07:00
Yann Collet
7d3816183f exposed ZSTD_MAGIC_DICTIONARY in zstd.h
makes it easier to explain ZSTD_dictMode
2017-06-27 13:50:34 -07:00
Yann Collet
7bd1a2900e added ZSTD_dictMode_e to control dictionary loading mode 2017-06-21 11:50:33 -07:00
Yann Collet
c3bce24ef4 fixed potential dangling pointer, detected by @terrelln 2017-06-20 16:09:11 -07:00
Yann Collet
559ee82e90 fixed : calling ZSTD_compress_generic() to end-flush a stream in multiple steps 2017-06-16 11:58:21 -07:00
cyan4973
c59162e053 minor fix for -Wdocumentation 2017-06-05 00:12:13 -07:00
cyan4973
8bcbf42617 fixed g++ prototype mismatch 2017-06-04 23:52:00 -07:00
Yann Collet
8c910d2097 updated ZSTDMT streaming API
ZSTDMT streaming API is now similar
and has same capabilites as single-thread streaming API.
It makes it easier to blend them together.
2017-06-03 01:15:02 -07:00
Yann Collet
58e8d793e1 made debug definitions common within zstd_internal.h 2017-06-02 18:20:48 -07:00
Yann Collet
4effccbf56 zlib_wrapper's uncompress() uses ZSTD_isFrame() for routing
more generic and safer than using own routing for magic number comparison
2017-06-02 14:27:11 -07:00
Yann Collet
dcb7535352 ensure zlibwrapper uses ZSTD_malloc() and ZSTD_free()
which is compatible with { NULL, NULL, NULL }
2017-06-02 14:01:21 -07:00
Yann Collet
6056e4c3eb added POOL_sizeof() for single-thread 2017-06-02 11:36:47 -07:00
Yann Collet
c4a5a21c5c created ZSTDMT_sizeof_CCtx() and POOL_sizeof()
required by ZSTD_sizeofCCtx() while adding a ZSTDMT_CCtx*
2017-06-01 17:56:14 -07:00
Yann Collet
9a691e0f55 fixed visual warnings 2017-05-31 01:17:44 -07:00
Yann Collet
ae728a43b8 removed defaultCustomMem
now ZSTD_customCMem is promoted as new default.

Advantages : ZSTD_customCMem = { NULL, NULL, NULL},
so it's natural default after a memset.

ZSTD_customCMem is public constant
(defaultCustomMem was private only).

Also : makes it possible to introduce ZSTD_calloc(),
which can now default to stdlib's calloc()
when it detects system default.

Fixed zlibwrapper which depended on defaultCustomMem.
2017-05-30 17:11:39 -07:00
Yann Collet
44e45e8423 added ZSTDMT_createCCtx_advanced()
make it possible to use custom allocators
2017-05-30 16:12:06 -07:00
Yann Collet
b81f19ffce move MEM_readMINMATCH() into zstd_opt.h
which is its only user.
Use case too narrow to belong to mem.h.
renamed to ZSTD_readMINMATCH()
2017-05-23 15:41:55 -07:00
Nick Terrell
a1280406b0 [libzstd] Allow users to define custom visibility 2017-05-19 18:01:59 -07:00
Yann Collet
bfff8999c5 added prototype ZSTD_versionString() 2017-05-16 16:12:23 -07:00
Jos Collin
05286fdd5a lib/common: warning: this statement may fall through
The following warning appears during the build. Fixed the review comments too.

zstd/lib/common/bitstream.h: In function ‘BIT_initDStream’:
zstd/lib/common/bitstream.h:277:33: warning: this statement may fall through [-Wimplicit-fallthrough=]
      case 7: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[6]) <<
      (sizeof(bitD->bitContainer)*8 - 16);

Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-05-11 09:10:02 +05:30
Nick Terrell
5f2c7213c7 Merge remote-tracking branch 'upstream/dev' into btopt
* upstream/dev: (305 commits)
  added test for ZSTD_estimateCStreamSize()
  changed variable name, for clarity
  fixed ZSTD_estimateCStreamSize()
  shortened ZSTD_createCStream_Advanced()
  fixed symbols test
  added ZSTD_estimateDStreamSize()
  changed name frameParams into frameHeader
  regroup memory usage function declarations
  separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
  bumped version number
  added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
  Updated ZSTD_freeCCtx()
  updated ZSTD_estimateCCtxSize()
  Updated ZSTD_sizeof_CCtx()
  merged CCtx and CStream as a single same object
  cli : -d and -t do not stop after a failed decompression
  added dev branch CircleCI badge
  added dev branch Appveyor badge
  keep dev branch status only
  creates a binary archive without the `programs` directory
  ...
2017-05-10 16:49:58 -07:00
Yann Collet
d47709b6ea Merge pull request #654 from iburinoc/splittable
[RFC] Splittable Format and API
2017-05-08 13:41:56 -07:00
Yann Collet
33c38b0925 fixed const in prototype, that Visual doesn't accept 2017-05-01 11:12:30 -07:00
Yann Collet
f39a6731ec sync bitstream.h from fse library 2017-05-01 09:56:03 -07:00
Yann Collet
202082f285 sync bitstream from FSE project
add assert into unsafe *_fast() variants
2017-04-28 17:00:31 -07:00
Yann Collet
68a7d3d49a added HUF_PUBLIC_API macro to huf.h
to make it possible to control symbol visibility.
Also : better separation and comments between "public" and "static" sections
2017-04-28 12:46:48 -07:00
Yann Collet
d3694e6c70 removed C4204 2017-04-27 14:29:35 -07:00
Yann Collet
69a54d138a fixed compilation warning : declaration-after-statement 2017-04-27 01:11:26 -07:00
Yann Collet
e42afbc6fa Comply with suggested comments by @terrelln
created FSE_CTABLE_SIZE() and FSE_DTABLE_SIZE()
2017-04-26 11:39:35 -07:00
Sean Purcell
7d37ca1d5b Merge remote-tracking branch 'origin/dev' into splittable 2017-04-21 14:18:39 -07:00
Yann Collet
e6fa70a0a1 reorganized ZSTD_resetCCtx_internal()
clearer separation between variables and buffers
clearer buffers category
kept static buffers at the beginning, favoring cache locality
(it will be easier to add FSE tables there later)

This break a few assumptions that hashTable was always at the beginning.
This is fixed.
And remaining assumptions (namely that tables stand next to each other in memory)
are now tested with assert.
2017-04-20 17:28:31 -07:00
Sean Purcell
0f7bd772e6 Update seekable API to simplify IO 2017-04-18 16:48:30 -07:00
Yann Collet
4f818182b8 clarified frame parameters for ZSTD_compress*_usingCDict()
created ZSTD_compressBegin_usingCDict_internal(),
which gives direct control to frame Parameters.
ZSTD_resetCStream_internal() now points into it.
2017-04-17 18:29:06 -07:00
Yann Collet
c47c68f6ca proper evaluation of Huffman CTable size 2017-04-17 16:14:21 -07:00
Sean Purcell
5ee1135f30 s/chunk/frame/ 2017-04-12 11:15:50 -07:00
Yann Collet
20d5e03893 content size is controlled at bufferless level
so it's active for all entry points

Also : added relevant test (wrong content size) in fuzzer
2017-04-11 18:34:02 -07:00
Sean Purcell
d048fefef7 Move seekable format content to /contrib 2017-04-11 14:38:56 -07:00
Sean Purcell
c3ba15e48f Seekable compression demo 2017-04-11 13:53:09 -07:00
Nick Terrell
16a739cab0 Switch call of FSE_count() to FSE_count_wksp() 2017-04-04 16:17:21 -07:00
Yann Collet
274f59919d Changed memory strategy to __packed for gcc
Method 1 __packed is always as good or better than memcpy().
But it's not portable, as it depends on compiler extension.

For gcc, __pakced directive works fine.
Furthermore, gcc has serious performance issues with memcpy() on ARM 32 bits.
See #620
2017-03-30 12:52:14 -07:00
Nick Terrell
5152fb2cb2 Convert all tabs to spaces 2017-03-29 18:51:58 -07:00
Yann Collet
f332ece468 dictBuilder fails to create dictionary on certain input
Properly expressed with an error code (see zstd_errors.h)
and a cli return code != 0
2017-03-23 16:24:02 -07:00
Sean Purcell
8fe5c6862c Fix undefined behaviour in decompressor 2017-03-10 10:17:42 -08:00
Nick Terrell
f35ef5c8e9 Whitespace only: tabs to spaces 2017-03-09 12:51:33 -08:00
Yann Collet
1f2c95c5f3 minor code refactor in HUF module 2017-03-05 21:07:20 -08:00
Nick Terrell
54c4babd8f Always check Huffman tables for ZSTD_lazy+
The compressor always reuses the existing Huffman table if the literals
size is at most 1 KiB. If the compression strategy is `ZSTD_lazy` or
stronger always check to see if reusing the previous table or creating
a new table is better.

This doesn't yet weigh in decompression speed. I don't want to add any
heuristics there until I have real data to work with to ensure that the
heuristic works for at least one use case, preferably more.
2017-03-03 16:49:38 -08:00