Commit Graph

159 Commits

Author SHA1 Message Date
Josh Soref
a880ca239b Spelling (#1582)
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
2019-04-12 11:18:11 -07:00
Yann Collet
59a7116cc2 benchfn dependencies reduced to only timefn
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.

Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.

Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
2019-04-10 12:37:03 -07:00
Li-Wen Hsu
6ff9d5e881 Include unistd.h on unix platforms for explicit function declaration 2019-01-16 04:55:43 +08:00
Yann Collet
ededcfca57 fix confusion between unsigned <-> U32
as suggested in #1441.

generally U32 and unsigned are the same thing,
except when they are not ...

case : 32-bit compilation for MIPS (uint32_t == unsigned long)

A vast majority of transformation consists in transforming U32 into unsigned.
In rare cases, it's the other way around (typically for internal code, such as seeds).

Among a few issues this patches solves :
- some parameters were declared with type `unsigned` in *.h,
  but with type `U32` in their implementation *.c .
- some parameters have type unsigned*,
  but the caller user a pointer to U32 instead.

These fixes are useful.

However, the bulk of changes is about %u formating,
which requires unsigned type,
but generally receives U32 values instead,
often just for brevity (U32 is shorter than unsigned).
These changes are generally minor, or even annoying.

As a consequence, the amount of code changed is larger than I would expect for such a patch.

Testing is also a pain :
it requires manually modifying `mem.h`,
in order to lie about `U32`
and force it to be an `unsigned long` typically.
On a 64-bit system, this will break the equivalence unsigned == U32.
Unfortunately, it will also break a few static_assert(), controlling structure sizes.
So it also requires modifying `debug.h` to make `static_assert()` a noop.
And then reverting these changes.

So it's inconvenient, and as a consequence,
this property is currently not checked during CI tests.
Therefore, these problems can emerge again in the future.

I wonder if it is worth ensuring proper distinction of U32 != unsigned in CI tests.
It's another restriction for coding, adding more frustration during merge tests,
since most platforms don't need this distinction (hence contributor will not see it),
and while this can matter in theory, the number of platforms impacted seems minimal.

Thoughts ?
2018-12-21 18:09:41 -08:00
Yann Collet
068c9b89be fixed zlibwrapper examples build 2018-12-20 15:28:03 -08:00
Yann Collet
de4eb06a77 fixed 1 transitive include 2018-12-19 18:38:37 -08:00
Yann Collet
2e7fd6a2cb fixed remaining searchLength invocations 2018-11-20 15:13:27 -08:00
Yann Collet
b83d1e7714 removed some static const variables
and replaced by traditional macro constants.

Unfortunately, C doesn't consider `static const` to mean "constant"
2018-11-13 16:56:32 -08:00
Rohit Jain
705e0b18ab Making changes to make it compile on my laptop 2018-10-11 15:51:57 -07:00
Nick Terrell
5aa9a1dd2d Fix minigzip in std=c99 mode 2018-09-28 15:52:25 -07:00
Yann Collet
123fac6b6d fix pzstd compatibility with mingw
some details changed with introduction of gcc7
2018-09-21 17:36:00 -07:00
Yann Collet
da55865e47 ensure dependency for zlib wrapper 2018-08-15 16:43:13 -07:00
Yann Collet
6873fec658 changed dictMore for dictContentType
which seems clearer to describe what the variable/argument is about.
2018-03-20 15:13:14 -07:00
Nick Terrell
b495140f67 Update BUCK files
* Correct XXH namespace (Fixes #901)
* Multithreading always enabled
* GZIP/LZ4/LZMA always enabled
* Legacy support always fully enabled
2017-10-25 12:47:57 -07:00
Yann Collet
9d377c6e88 Merge pull request #895 from facebook/fileSize_unknown
Distinguish 0-size from size-unavailable
2017-10-18 16:50:17 -07:00
Yann Collet
ca1a9ebac5 fixed zlib wrapper
it was invoking ZSTD_initCStream_advanced() with pledgedSrcSize==0 and contentSizeFlag=1
which means "empty"
while the intention was to mean "unknown".

The contentSizeFlag==1 is new, it is a consequence of setting this value to 1 by default.

The solution selected here is to pass ZSTD_CONTENTSIZE_UNKNOWN to mean "unknown".
So contentSizeFlag remains set (it wasn't in previous versions).
2017-10-18 11:22:23 -07:00
Yann Collet
18b795374a UTIL_getFileSize() returns UTIL_FILESIZE_UNKNOWN on failure
UTIL_getFileSize() used to return zero on failure.
This made it impossible to distinguish a failure from a genuine empty file.
Both cases where coalesced.

Adding UTIL_FILESIZE_UNKNOWN constant has many consequences on user code,
since in many places, the `0` was assumed to mean "error".
This is no longer the case, and the error code must be actively checked.
2017-10-17 16:14:25 -07:00
Yann Collet
c95c0c9725 modified util::time API
for easier invocation.
- no longer expose frequency timer :
it's either useless, or stored internally in a static variable (init is only necessary once).
- UTIL_getTime() provides result by function return.
2017-09-12 18:12:46 -07:00
Yann Collet
058ed2ad33 ZSTD_decodingBufferSize_min()
supporting function for bufferless streaming API (ZSTD_decompressContinue())
makes it possible to correctly size a round buffer for decoding using this API.

also : added field blockSizeMax within ZSTD_frameHeader,
as it's a necessary information to know when to restart at beginning of decoding buffer.
2017-09-09 01:03:29 -07:00
Yann Collet
3128e03be6 updated license header
to clarify dual-license meaning as "or"
2017-09-08 00:09:23 -07:00
Yann Collet
d7ad99b2ab Merge branch 'longRangeMatcher' into dev 2017-08-31 18:08:37 -07:00
Stella Lau
c88fb9267f Replace 'byReference' with enum 2017-08-29 11:55:02 -07:00
Yann Collet
232d62b637 fixed a few headers that were too hastily copy/pasted during last license change 2017-08-21 11:24:32 -07:00
Yann Collet
32fb407c9d updated a bunch of headers
for the new license
2017-08-18 16:52:05 -07:00
Yann Collet
7bd1a2900e added ZSTD_dictMode_e to control dictionary loading mode 2017-06-21 11:50:33 -07:00
Yann Collet
33a7e679e5 significant zlib wrapper code refactoring
code indentation
variable scope and names
constify

Only coding style changes.
The logic should remain the same.
2017-06-02 17:10:49 -07:00
Yann Collet
4effccbf56 zlib_wrapper's uncompress() uses ZSTD_isFrame() for routing
more generic and safer than using own routing for magic number comparison
2017-06-02 14:27:11 -07:00
Yann Collet
dcb7535352 ensure zlibwrapper uses ZSTD_malloc() and ZSTD_free()
which is compatible with { NULL, NULL, NULL }
2017-06-02 14:01:21 -07:00
Yann Collet
ae728a43b8 removed defaultCustomMem
now ZSTD_customCMem is promoted as new default.

Advantages : ZSTD_customCMem = { NULL, NULL, NULL},
so it's natural default after a memset.

ZSTD_customCMem is public constant
(defaultCustomMem was private only).

Also : makes it possible to introduce ZSTD_calloc(),
which can now default to stdlib's calloc()
when it detects system default.

Fixed zlibwrapper which depended on defaultCustomMem.
2017-05-30 17:11:39 -07:00
Yann Collet
31533bacce Changed ZSTD_createCDict_advanced()
It now only uses compressionParameters as argument.
It produces many changes throughout user code,
though hopefully they tend to be simple :
just provide the cParams part from existing ZSTD_parameters.

Some programs might depend on ZSTD_createCDict_advanced() to pass frame parameters.
This change will force them to revisit this strategy and fix it,
since frame parameters are effectively silently ignored in current version.
2017-04-27 00:29:04 -07:00
Yann Collet
4f818182b8 clarified frame parameters for ZSTD_compress*_usingCDict()
created ZSTD_compressBegin_usingCDict_internal(),
which gives direct control to frame Parameters.
ZSTD_resetCStream_internal() now points into it.
2017-04-17 18:29:06 -07:00
Sean Purcell
680e4e0953 Ignore symbolic links unless --force specified 2017-03-23 11:52:09 -07:00
Sean Purcell
042ba122ae Change g_displayLevel to int and fix DISPLAYUPDATE flush 2017-03-23 11:21:59 -07:00
Przemyslaw Skibinski
5c42aad169 zlibWrapper: better description of ZWRAP_useZSTDcompression 2017-02-23 18:28:48 +01:00
Przemyslaw Skibinski
e052c60540 introduce UTIL_freq_t 2017-02-20 11:27:11 +01:00
Przemyslaw Skibinski
94abd6a26c SET_REALTIME_PRIORITY 2017-02-07 16:36:19 +01:00
Nick Terrell
9c018cc140 Add BUCK files for Nuclide support 2017-01-27 10:43:12 -08:00
Przemyslaw Skibinski
29157320fb improved ZSTD_compressBlock_opt_extDict_generic 2017-01-27 10:43:02 -08:00
Przemyslaw Skibinski
502966ab9c zlibWrapper: added the totalInBytes flag - we need it as strm->total_in can be reset by user 2017-01-19 12:10:52 +01:00
Przemyslaw Skibinski
957a6d596b updated link to copyright notice 2017-01-18 19:04:00 +01:00
Przemyslaw Skibinski
c3a04deda2 fixed clang warnings in gzread.c and gzwrite.c 2017-01-18 14:36:10 +01:00
Przemyslaw Skibinski
c9512db301 gzcompatibility.h updated to zlib 1.2.11 2017-01-18 12:51:44 +01:00
Przemyslaw Skibinski
3805a00904 gzwrite.c updated to zlib 1.2.11 2017-01-18 12:47:32 +01:00
Przemyslaw Skibinski
5735fd74ee gzread.c updated to zlib 1.2.11 2017-01-18 12:14:01 +01:00
Przemyslaw Skibinski
7f82aad187 gzlib.c updated to zlib 1.2.11 2017-01-18 12:08:08 +01:00
Przemyslaw Skibinski
69f7990fc5 gzguts.h updated to zlib 1.2.11 2017-01-18 12:01:50 +01:00
Przemyslaw Skibinski
5edab91bbb get_crc_table only with ZLIB_VERNUM >= 0x1270 2017-01-18 10:39:39 +01:00
Przemyslaw Skibinski
5b114d3c01 zlibWrapper: added get_crc_table 2017-01-17 13:02:06 +01:00
Yann Collet
9ceb49e097 fixed zlib_wrapper conversion warnings 2016-12-22 15:26:33 +01:00
Yann Collet
ba75e9d8c3 fix : zlib wrapper compile in gnu90 mode 2016-12-21 19:57:18 +01:00