Commit Graph

2738 Commits

Author SHA1 Message Date
W. Felix Handte
501eb25102 Rename FORWARD_ERROR -> FORWARD_IF_ERROR 2019-01-29 12:56:07 -05:00
W. Felix Handte
429987c9a6 Add Comment 2019-01-28 17:35:31 -05:00
W. Felix Handte
2179ce00e1 Remove CHECK_E Macro 2019-01-28 17:33:13 -05:00
W. Felix Handte
03e040a966 Replace Uses of CHECK_E with RETURN_ERROR_IF(*_isError(... 2019-01-28 17:33:01 -05:00
W. Felix Handte
7ebd897157 Remove CHECK_F Macro 2019-01-28 17:16:32 -05:00
W. Felix Handte
64bb6640f2 Replace CHECK_F Uses in zstdmt_compress.c and zstd_ddict.c 2019-01-28 17:15:57 -05:00
W. Felix Handte
cafc3b1bcb Also Convert zstd_compress.c 2019-01-28 17:05:18 -05:00
W. Felix Handte
324e9654d3 Add grep-able String to Error Macros 2019-01-28 12:50:36 -05:00
W. Felix Handte
32fed9c7be Switch CHECK_F Calls to FORWARD_ERROR 2019-01-28 12:45:34 -05:00
W. Felix Handte
800c87fed0 Switch Unconditional RETURN_ERROR_IF Calls to RETURN_ERROR 2019-01-28 12:45:34 -05:00
W. Felix Handte
a3538bbc6f Add RETURN_ERROR and FORWARD_ERROR Macros 2019-01-28 12:45:26 -05:00
W. Felix Handte
c823237d7b Convert Checks in zstd_decompress.c to RETURN_ERROR_IF 2019-01-28 12:23:14 -05:00
W. Felix Handte
ea031f4ea2 Convert Checks in zstd_decompress_block.c to RETURN_ERROR_IF 2019-01-28 11:56:39 -05:00
W. Felix Handte
54fa31f03b Add RETURN_ERROR_IF Macro That Logs Debug Information When Check Fails 2019-01-28 11:43:33 -05:00
Yann Collet
f9e4f89252 improved comments for adjustCParams() and getCParams() 2019-01-02 12:18:40 -08:00
Yann Collet
0fb4b21d1a updated libzstd documentation 2018-12-25 03:10:07 -08:00
Yann Collet
e980ba212f
Merge pull request #1471 from facebook/nofloat
guard functions using floating point for debug mode only
2018-12-23 12:35:51 -08:00
Yann Collet
aae5bc538a
Merge pull request #1470 from facebook/U32
fix confusion between unsigned <-> U32
2018-12-23 12:35:39 -08:00
Yann Collet
c9dfb7e445 guard functions using floating point for debug mode only
they are only used to print debug messages.
Requested in #1386,
2018-12-22 09:09:40 -08:00
Yann Collet
ededcfca57 fix confusion between unsigned <-> U32
as suggested in #1441.

generally U32 and unsigned are the same thing,
except when they are not ...

case : 32-bit compilation for MIPS (uint32_t == unsigned long)

A vast majority of transformation consists in transforming U32 into unsigned.
In rare cases, it's the other way around (typically for internal code, such as seeds).

Among a few issues this patches solves :
- some parameters were declared with type `unsigned` in *.h,
  but with type `U32` in their implementation *.c .
- some parameters have type unsigned*,
  but the caller user a pointer to U32 instead.

These fixes are useful.

However, the bulk of changes is about %u formating,
which requires unsigned type,
but generally receives U32 values instead,
often just for brevity (U32 is shorter than unsigned).
These changes are generally minor, or even annoying.

As a consequence, the amount of code changed is larger than I would expect for such a patch.

Testing is also a pain :
it requires manually modifying `mem.h`,
in order to lie about `U32`
and force it to be an `unsigned long` typically.
On a 64-bit system, this will break the equivalence unsigned == U32.
Unfortunately, it will also break a few static_assert(), controlling structure sizes.
So it also requires modifying `debug.h` to make `static_assert()` a noop.
And then reverting these changes.

So it's inconvenient, and as a consequence,
this property is currently not checked during CI tests.
Therefore, these problems can emerge again in the future.

I wonder if it is worth ensuring proper distinction of U32 != unsigned in CI tests.
It's another restriction for coding, adding more frustration during merge tests,
since most platforms don't need this distinction (hence contributor will not see it),
and while this can matter in theory, the number of platforms impacted seems minimal.

Thoughts ?
2018-12-21 18:09:41 -08:00
Yann Collet
c8d1fda982 update aarch64 test to xenial
in an attempt to circumvent the `ld` bug
2018-12-21 15:08:48 -08:00
Yann Collet
8f35c7f94c
Merge pull request #1466 from facebook/noDictPresent
fixed : better error message
2018-12-20 19:01:27 -08:00
Yann Collet
41b45b84a1
Merge pull request #1465 from facebook/noFilePresent
fixed : detection of non-existing file
2018-12-20 17:21:04 -08:00
Yann Collet
ed2fb6bd57 fixed : better error message when dictionary missing
during benchmark.
Also : refactored ZSTD_fillHashTable(),
just for readability (it does the same thing)
2018-12-20 17:20:07 -08:00
Yann Collet
e4ae24c229
Merge pull request #1420 from felixhandte/zstd-decompress-minimal
Various Macros to Allow Building Extremely Minimal Decoder Library
2018-12-20 15:17:37 -08:00
Yann Collet
95784c654c fixed shadowing of stat variable
some standard lib declares a `stat` variable at global scope
shadowing local declarations ....
2018-12-20 14:56:44 -08:00
Yann Collet
ffba142406 fixed file identity detection in 32-bit mode
also :
some library decided to use `index` as a global variable declared in standard header
shadowing the ones used in fastcover.c  :(
2018-12-20 14:30:30 -08:00
W. Felix Handte
91b7309115 Mask Off Unused Functions When ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG 2018-12-20 12:20:34 -08:00
W. Felix Handte
038aabde28 Mask Off Unused Functions When ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT 2018-12-20 12:15:07 -08:00
Yann Collet
2898afab52 fixed OSSfuzz 11849
The problem was already masked,
due to no longer accepting tiny blocks for statistics.

But in case it could still happen with not-so-tiny blocks,
there is a stricter control which ensures that
nothing was already loaded prior to statistics collection.
2018-12-19 16:54:15 -08:00
W. Felix Handte
8e61ac8161 Use Unused Variable in ERR_getErrorString() 2018-12-19 12:36:10 -08:00
Yann Collet
8e0e495ce8 fixed: compression ratio discrepancy
depending on initialization,
the first byte of a new frame was invalidated or not.

As a consequence, one match opportunity was available or not,
resulting in slightly different compressed sizes
(on average, 1 or 2 bytes once every 20 frames).

It impacted ratio comparison between one-shot and streaming modes.

This fix makes the first byte of a new frame always a valid match.
Now compressed size is always the same.
It also improves compressed size by a negligible amount.
2018-12-19 10:11:06 -08:00
Yann Collet
d0e15f8d32
Merge pull request #1458 from terrelln/estimate
[libzstd] Fix estimate with negative levels
2018-12-18 15:12:21 -08:00
Yann Collet
04baecaeed
Merge pull request #1457 from facebook/btultra2.1
btultra2 and very small input
2018-12-18 14:46:55 -08:00
Nick Terrell
d7def456d8 [libzstd] Fix estimate with negative levels
* Fix `ZSTD_estimateCCtxSize()` with negative levels.
* Fix `ZSTD_estimateCStreamSize()` with negative levels.
* Add a unit test to test for this error.
2018-12-18 14:24:49 -08:00
Yann Collet
ef984e7307 fix debug levels
as reported by @terrelln.
2 is reserved for temporary usage only.
2018-12-18 13:40:07 -08:00
W. Felix Handte
0d606ee3db Fix Incorrect assert() 2018-12-18 13:36:39 -08:00
W. Felix Handte
bd4afc389f Add Logic to Makefile to Convert Make Vars to Defines 2018-12-18 13:36:39 -08:00
W. Felix Handte
ece2c18372 Document Macros in README 2018-12-18 13:36:39 -08:00
W. Felix Handte
c2d51637d9 Add Mutual-Exclusion Error 2018-12-18 13:36:39 -08:00
W. Felix Handte
c560e34c86 Add HUF_FORCE_DECOMPRESS_X2 2018-12-18 13:36:39 -08:00
W. Felix Handte
abd1567d3c Move HUF_DGEN Up Out of X1 Definitions 2018-12-18 13:36:39 -08:00
W. Felix Handte
4a0572b215 Refactor Huffman Decompression Away From Ternary Tree in ZSTD_decodeLiteralsBlock 2018-12-18 13:36:39 -08:00
W. Felix Handte
432314b58a Rename HUF_DECOMPRESS_MINIMAL -> HUF_FORCE_DECOMPRESS_X1 2018-12-18 13:36:39 -08:00
W. Felix Handte
4bbb8a48ad Add ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG
This macro forces behavior in the opposite direction.
2018-12-18 13:36:39 -08:00
W. Felix Handte
64553a0e35 Rename ZSTD_DECOMPRESS_MINIMAL -> ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT 2018-12-18 13:36:39 -08:00
W. Felix Handte
605dd576ee Remove Error Strings with ZSTD_STRIP_ERROR_STRINGS 2018-12-18 13:36:39 -08:00
W. Felix Handte
9d5f3963ff Add Option to Not Request Inlining with ZSTD_NO_INLINE 2018-12-18 13:36:39 -08:00
W. Felix Handte
df28e5babd Add ZSTD_DECOMPRESS_MINIMAL Macro, Which Reduces Branching of Decompress Variants 2018-12-18 13:36:39 -08:00
W. Felix Handte
f45c9df42e Totally Hide/Disable X2 Variants when HUF_DECOMPRESS_MINIMAL is Defined 2018-12-18 13:36:39 -08:00