Commit Graph

127 Commits

Author SHA1 Message Date
senhuang42
b9ab6bc061 Fix various conversion warnings 2020-12-08 10:07:28 -05:00
senhuang42
260b85acf5 Fix MSVC 2019 warnings 2020-12-03 10:36:45 -05:00
Nick Terrell
ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
Nick Terrell
785331acbd [paramgrill] Fix mingw build errors 2019-04-18 15:06:56 -07:00
Josh Soref
a880ca239b Spelling (#1582)
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
2019-04-12 11:18:11 -07:00
Yann Collet
59a7116cc2 benchfn dependencies reduced to only timefn
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.

Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.

Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
2019-04-10 12:37:03 -07:00
Yann Collet
cee31bbd25 updated fullbench and paramgrill
to use new benchfn
returning a double.
2019-01-25 15:38:08 -08:00
Yann Collet
ededcfca57 fix confusion between unsigned <-> U32
as suggested in #1441.

generally U32 and unsigned are the same thing,
except when they are not ...

case : 32-bit compilation for MIPS (uint32_t == unsigned long)

A vast majority of transformation consists in transforming U32 into unsigned.
In rare cases, it's the other way around (typically for internal code, such as seeds).

Among a few issues this patches solves :
- some parameters were declared with type `unsigned` in *.h,
  but with type `U32` in their implementation *.c .
- some parameters have type unsigned*,
  but the caller user a pointer to U32 instead.

These fixes are useful.

However, the bulk of changes is about %u formating,
which requires unsigned type,
but generally receives U32 values instead,
often just for brevity (U32 is shorter than unsigned).
These changes are generally minor, or even annoying.

As a consequence, the amount of code changed is larger than I would expect for such a patch.

Testing is also a pain :
it requires manually modifying `mem.h`,
in order to lie about `U32`
and force it to be an `unsigned long` typically.
On a 64-bit system, this will break the equivalence unsigned == U32.
Unfortunately, it will also break a few static_assert(), controlling structure sizes.
So it also requires modifying `debug.h` to make `static_assert()` a noop.
And then reverting these changes.

So it's inconvenient, and as a consequence,
this property is currently not checked during CI tests.
Therefore, these problems can emerge again in the future.

I wonder if it is worth ensuring proper distinction of U32 != unsigned in CI tests.
It's another restriction for coding, adding more frustration during merge tests,
since most platforms don't need this distinction (hence contributor will not see it),
and while this can matter in theory, the number of platforms impacted seems minimal.

Thoughts ?
2018-12-21 18:09:41 -08:00
Yann Collet
e129174d1d fixed shadowing of variable time
some standard lib do define `time` as a global variable
shadowing local declarations ...
2018-12-20 14:54:05 -08:00
Yann Collet
b71bfb6cf2 paramgrill: add status line
get information on which config is currently tested
so that console get animated during long tests.
2018-12-07 16:02:24 -08:00
Yann Collet
e68c2d86e7 refactor paramgrill for clarity
restored ability to copy/paste the resulting compression level table into zstd_compress.c .
2018-12-07 14:07:54 -08:00
Yann Collet
39e28982cf introduced constants ZSTD_STRATEGY_MIN and ZSTD_STRATEGY_MAX 2018-12-06 16:16:16 -08:00
Yann Collet
be9e561da4 changed ZSTD_c_compressionStrategy into ZSTD_c_strategy
also : fixed paramgrill, and limit conditions
2018-12-06 15:00:52 -08:00
Yann Collet
3583d19c4e changed parameter names from ZSTD_p_* to ZSTD_c_*
for naming consistency
2018-12-05 17:26:02 -08:00
Yann Collet
34e146f548 advanced decompression function replaces by normal streaming one
advanced parameters compatible with ZSTD_decompressStream().
2018-12-04 10:28:36 -08:00
Yann Collet
d8e215cbee created ZSTD_compress2() and ZSTD_compressStream2()
ZSTD_compress_generic() is renamed ZSTD_compressStream2().

Note that, for the time being,
the "stable" API and advanced one use different parameter planes :
setting parameters using the advanced API does not influence ZSTD_compressStream()
and using ZSTD_initCStream() does not influence parameters for ZSTD_compressStream2().
2018-11-30 11:25:56 -08:00
Yann Collet
e874dacc08 changed searchLength into minMatch
refactored all relevant API and calls
for consistency.
2018-11-20 14:56:07 -08:00
Yann Collet
5c68639186 updated ZSTD_DCtx_reset()
signature and behavior is now the same as ZSTD_CCtx_reset()
2018-11-15 16:12:39 -08:00
Yann Collet
7b0391e37e finalized retrofit of ZSTD_CCtx_reset()
updated all depending sources
2018-11-14 13:05:35 -08:00
Yann Collet
b830ccca5c changed benchfn api
to use structure for function parameters
as it expresses much clearer than a long list of parameters,
since each parameter can now be named.
2018-11-13 13:12:50 -08:00
Yann Collet
d38063f8ae separated bench module into benchfn and benchzstd
it shall be possible to use benchfn
without any dependency on zstd.
2018-11-13 11:01:59 -08:00
Yann Collet
efbc3e823d fixed paramgrill wrong assert() conditions
and slightly refactored affected function.

Honestly, the formula calculating variance should get a second reviewing round,
it's not clear if it's correct.
2018-10-04 14:27:13 -07:00
Nick Terrell
f2d6db45cd [zstd] Add -Wmissing-prototypes 2018-09-27 15:24:48 -07:00
Yann Collet
0e211bdd18 fixed constant comparison on 32-bits systems 2018-09-21 18:23:32 -07:00
Yann Collet
484f40697b fix constant redeclaration in paramgrill 2018-09-21 17:28:37 -07:00
Yann Collet
55affc09de timedFn : measurement delay is programmable
instead of hard-coded 1 second per measurement
2018-08-28 11:26:27 -07:00
Yann Collet
0ff9b67552 paramgrill: removed useless tests
designed to compensate iter_mode,
but since only time_mode is available now,
all tests are guaranteed to last a minimum amount of time.
2018-08-27 19:07:17 -07:00
Yann Collet
9e26893e07 paramgrill: fixed a bunch of div-by-zero
they were pretty easy to trigger by the way,
just start an extended paramgrill session
to find a compression table based on any sample,
it would necessarily happen at some point.
2018-08-27 18:47:09 -07:00
Yann Collet
0071e8348f restored assert() in paramgrill
assert() in paramgrill are not in the benchmark path.
They should remain active, as they don't impact measurements, and their runtime is insignificant.
2018-08-27 17:52:04 -07:00
Yann Collet
01dcd0fd17 bench: minor api update, for consistency
BMK_benchTimedFn()
BMK_isCompleted_TimedFn() uses TimedFnState
2018-08-26 21:30:18 -07:00
Yann Collet
c3a4baaf6e fixed minor warnings
valgrind: memory leak of a few bytes in fullbench
static analyzer: uninitialized data passed as result
2018-08-24 23:25:35 -07:00
Yann Collet
2279f3d127 bench: reduce nb of return type
runOutcome is enough
removed timedFnOutcome
2018-08-24 17:28:38 -07:00
Yann Collet
4da5bdf482 fixed zstd -b speed result
the benchmark was displaying the speed of last run
instead of the best of all previous runs.
2018-08-23 18:13:49 -07:00
Yann Collet
b0e1f3982d fixed paramgrill
to work with new bench.c
2018-08-23 17:21:38 -07:00
Yann Collet
77e805e3db bench: changed creation/reset function to timedFnState
for consistency
2018-08-21 18:19:27 -07:00
George Lu
8175b28f03 Fix negative lvl display value
Also fix synthetic benchmark parameter setting
2018-08-16 16:46:37 -07:00
George Lu
239e114d62 prune comments 2018-08-15 16:04:34 -07:00
George Lu
8a296d3e1f Move Stuff around
Group similar functions together, remove outdated comments
2018-08-15 16:04:34 -07:00
George Lu
3f8b10baa1 consts 2018-08-15 16:04:34 -07:00
George Lu
46be2ef5d8 Remove unused stuff 2018-08-15 16:04:34 -07:00
George Lu
ee77ddc28d Fix wraparound 2018-08-15 14:01:32 -07:00
George Lu
1e8d352930 silencing params 2018-08-15 14:01:32 -07:00
George Lu
2c5fdae0ae Clean up repetitive display
Add documentation
2018-08-15 14:01:32 -07:00
George Lu
4d9c6f51b8 -q -v options 2018-08-15 14:01:32 -07:00
George Lu
3dcfe5cc2c begin display changes 2018-08-15 14:01:32 -07:00
George Lu
b1d9ca737a Add memoTable options
-hashing memotable
-no memotable
2018-08-15 10:19:38 -07:00
George Lu
8c918edd3a MAke it easier to add params
Make memoTable size limited
2018-08-14 16:15:46 -07:00
George Lu
96725989ef Temp fix perf regression 2018-08-14 16:14:37 -07:00
George Lu
3f2d024dca forceAttachDict 2018-08-14 14:24:41 -07:00
George Lu
e3c679484a Add Time Checks
Fix double -> U64 display
2018-08-14 14:24:41 -07:00