Commit Graph

578 Commits

Author SHA1 Message Date
Yann Collet
9589e8e4bb
Merge pull request #1749 from facebook/rmadapt
removed adaptive-compression
2019-08-28 12:26:29 -07:00
Carl Woffenden
cdf73e915e Rewrote the scripts to sh instead of bash 2019-08-28 19:20:42 +02:00
Yann Collet
5ed1b1e11d removed adaptive-compression
the functionality is already integrated into `zstd` through `--adapt` command
2019-08-27 14:47:40 -07:00
Carl Woffenden
51868964ef Fixed test failure when Emscripten not present 2019-08-27 17:12:57 +02:00
Carl Woffenden
6213b7b3b4 Minor repetition 2019-08-27 16:57:23 +02:00
Carl Woffenden
59052d5fd8 Typo 2019-08-27 16:55:03 +02:00
Carl Woffenden
ec12721538 Added clarification 2019-08-27 15:53:26 +02:00
Carl Woffenden
6712a644fa Added reasoning 2019-08-27 15:51:14 +02:00
Carl Woffenden
4f2a8b752a Typo 2019-08-27 15:38:34 +02:00
Carl Woffenden
a57de4ac89 Added test script; tidied and documented
The test script combines the sources then builds and runs an example. A futher example is built if the Emscripten compiler is available on the system. Documentation covers building.
2019-08-27 15:36:06 +02:00
Carl Woffenden
7c6fa81579 Added Emscripten example, removed Buck, minor tidy
Work-in-progress. Added simple Emscripten WebGL example that adds 25kB when build with Zstd. Removed Buck (will replace). Minor correctness.
2019-08-26 21:28:19 +02:00
Carl Woffenden
ea8f6d2a07 Able to test combine script; minor tidy 2019-08-26 07:48:57 +02:00
Carl Woffenden
d760e35ebc Preparing to run tests
Combine script more robust and can output to a specified file. Initial buck files added (work in progress).
2019-08-25 22:49:01 +02:00
Carl Woffenden
36a59336da Minor fix for files with spaces. Typo. 2019-08-23 23:09:13 +02:00
Carl Woffenden
0a49353a46 Added generator script and simple test
The script will combine decompressor sources into a single file. The example shows this in use.
2019-08-23 18:43:29 +02:00
Felix Handte
2314906b68
Merge pull request #1699 from felixhandte/seekable-gitignore
Add New Seekable Compression Example to .gitignore
2019-07-24 19:07:55 -04:00
Yann Collet
0d38ee3c30
Merge pull request #1690 from piguin/dev
fix compiling errors with clang-8
2019-07-24 15:37:05 -07:00
W. Felix Handte
15da57820d Add New Seekable Compression Example to .gitignore 2019-07-24 18:22:20 -04:00
Sean Purcell
671d533ea7 Fix seekable decompression in-memory api 2019-07-21 23:22:25 -04:00
Qin Li
04a9d6b828 fix compiling errors with clang-8
Compiling with clang-8 fails with the following errors:

largeNbDicts.c:562:37: error: implicit conversion turns floating-point
number into integer: 'const double' to 'U64' (aka 'unsigned long')
[-Werror,-Wfloat-conversion]
        U64 const dTime_ns = result.nanoSecPerRun;
                  ~~~~~~~~   ~~~~~~~^~~~~~~~~~~~~

zstdcli.c:300:5: error: '@return' command used in a comment that is
not attached to a function or method declaration
[-Werror,-Wdocumentation]
 * @return 1 means that cover parameters were correct
   ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

zstdcli.c:301:5: error: '@return' command used in a comment that is
not attached to a function or method declaration
[-Werror,-Wdocumentation]
 * @return 0 in case of malformed parameters
   ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2019-07-18 19:41:00 -07:00
Josh Soref
a880ca239b Spelling (#1582)
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
2019-04-12 11:18:11 -07:00
Yann Collet
59a7116cc2 benchfn dependencies reduced to only timefn
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.

Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.

Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
2019-04-10 12:37:03 -07:00
Peter (Stig) Edwards
4a9e0502e6
-Wformat-security not needed with -Wformat=2 2019-02-01 09:29:08 +00:00
Peter (Stig) Edwards
2b7120ec71
-Wformat-security not needed with -Wformat=2 2019-02-01 09:28:41 +00:00
Dmitry V. Levin
8b2210411a contrib/pzstd/Makefile: fix build of tests
Apparently, Options.o cannot be linked in without $(PROGDIR)/util.o
2018-12-28 19:02:22 +00:00
Yann Collet
ededcfca57 fix confusion between unsigned <-> U32
as suggested in #1441.

generally U32 and unsigned are the same thing,
except when they are not ...

case : 32-bit compilation for MIPS (uint32_t == unsigned long)

A vast majority of transformation consists in transforming U32 into unsigned.
In rare cases, it's the other way around (typically for internal code, such as seeds).

Among a few issues this patches solves :
- some parameters were declared with type `unsigned` in *.h,
  but with type `U32` in their implementation *.c .
- some parameters have type unsigned*,
  but the caller user a pointer to U32 instead.

These fixes are useful.

However, the bulk of changes is about %u formating,
which requires unsigned type,
but generally receives U32 values instead,
often just for brevity (U32 is shorter than unsigned).
These changes are generally minor, or even annoying.

As a consequence, the amount of code changed is larger than I would expect for such a patch.

Testing is also a pain :
it requires manually modifying `mem.h`,
in order to lie about `U32`
and force it to be an `unsigned long` typically.
On a 64-bit system, this will break the equivalence unsigned == U32.
Unfortunately, it will also break a few static_assert(), controlling structure sizes.
So it also requires modifying `debug.h` to make `static_assert()` a noop.
And then reverting these changes.

So it's inconvenient, and as a consequence,
this property is currently not checked during CI tests.
Therefore, these problems can emerge again in the future.

I wonder if it is worth ensuring proper distinction of U32 != unsigned in CI tests.
It's another restriction for coding, adding more frustration during merge tests,
since most platforms don't need this distinction (hence contributor will not see it),
and while this can matter in theory, the number of platforms impacted seems minimal.

Thoughts ?
2018-12-21 18:09:41 -08:00
Yann Collet
34f01e600f fixed multiple conversions
from 64-bit to 32-bit
2018-12-13 14:02:22 -08:00
Yann Collet
9c3265a53f
Merge pull request #1417 from facebook/advancedAPI
Advanced API
2018-12-10 18:48:15 -08:00
Yann Collet
3583d19c4e changed parameter names from ZSTD_p_* to ZSTD_c_*
for naming consistency
2018-12-05 17:26:02 -08:00
Lzu Tao
beb13bd87e Move contrib/meson to build/meson 2018-12-01 23:18:59 +07:00
Lzu Tao
c0e71cae55 Add enable_lz4 build option and fix lzma dependency 2018-12-01 23:18:59 +07:00
Lzu Tao
5c4965c351 Add pedantic flag 2018-12-01 23:18:59 +07:00
Lzu Tao
6f3f1a8d3a No install zstd_manual.html 2018-12-01 23:18:59 +07:00
Lzu Tao
f660825d9f Install missed zstdgrep and zstdless 2018-12-01 23:18:59 +07:00
Lzu Tao
3f27e2a072 Install zstdmt.1 manpage [skip ci] 2018-12-01 23:18:59 +07:00
Lzu Tao
d3134a3ed3 Rename meson variables 2018-12-01 23:18:59 +07:00
Lzu Tao
1985e427c7 Add manpage install warning [skip ci]
We link new manpages with gz compressed format of the target manpage.
I have not tested it on Windows. So just place a warning here.
2018-12-01 23:18:59 +07:00
Lzu Tao
9c862c6a53 Fix manpage symlinks [skip ci] 2018-12-01 23:18:59 +07:00
Lzu Tao
d79df2a370 Apply new InstallSymlink script 2018-12-01 23:18:59 +07:00
Lzu Tao
ef2e761937 Helper script to install symlink in meson 2018-12-01 23:18:59 +07:00
Lzu Tao
3175188407 No need these helpers 2018-12-01 23:18:59 +07:00
Lzu Tao
337f914dc8 Fix lib soversion and no install cover.h header 2018-12-01 23:18:59 +07:00
Lzu Tao
c9f0144302 Fix meson tests build 2018-12-01 23:18:59 +07:00
Lzu Tao
5a36a57cf5 Bump to 1.3.8 and fix run_command function
The run_command is run from an unspecified directory. Therefore we cannot assume
which directory it is running our command.
2018-12-01 23:18:59 +07:00
Lzu Tao
8a160680d1 Update legacy support to 5 2018-12-01 23:18:59 +07:00
Lzu Tao
f727808731 Minor fix for meson build
Use files function instead of constructing path with meson.current_source_dir()
2018-12-01 23:18:59 +07:00
Lzu Tao
9a721e5216 Update meson build system
NOTE: This commit only tested on Linux (Ubuntu 18.04). Windows
build may not work as expected.

* Use meson >= 0.47.0 cause we use install_man function
* Add three helper Python script:
  * CopyFile.py: To copy file
  * CreateSymlink.py: To make symlink (both Windows and Unix)
  * GetZstdLibraryVersion.py: Parse lib/zstd.h to get zstd version
  These help emulating equivalent functions in CMake and Makefile.
* Use subdir from meson to split meson.build
  * Add contrib build
  * Fix other build
* Add new build options
  * build_programs: Enable programs build
  * build_contrib: Enable contrib build
  * build_tests: Enable tests build
  * use_static_runtime: Link to static run-time libraries on MSVC
  * zlib_support: Enable zlib support
  * lzma_support: Enable lzma support
2018-11-28 01:08:34 +07:00
Lzu Tao
9bd8f6a00c Rename and update build instruction in README file to README.md 2018-11-28 01:08:34 +07:00
Lzu Tao
2abd5139a5 Add meson build guide 2018-11-28 01:08:34 +07:00
Yann Collet
5adbad4059 Merge branch 'dev' into advancedAPI 2018-11-14 13:00:37 -08:00