Commit Graph

248 Commits

Author SHA1 Message Date
Yann Collet
c601622aa7
Merge pull request #1619 from j301scott/dev
CMake: Check for existing custom target 'uninstall'
2019-06-10 11:56:49 -07:00
Jen Pollock
176a543dde In VS2010+, turn off assembler output for libzstd & libzstd-dll, and don't export functions from libzstd 2019-06-07 17:21:51 -06:00
l00292966
b31df043a0 add cmake lz4 support
add some instructions for build/cmake/README.md
2019-06-03 18:22:50 +08:00
Jonathan Scott
3f5737afd7 CMake: Check for existing custom target 'uninstall' 2019-05-25 17:30:04 -07:00
lzutao
1d70bc3ba9 meson: Enable testing in CI (#1609)
* tests: Fix shellcheck warnings in playTests.sh

* tests: Do not use ../programs which is relative to tests dirs

This commit fixes error when running playTests.sh in Meson.
Mesonbuild runs out of tree, so ./datagen not in `zstd/tests` dir,
it lies in <mesonbuilddir>/tests. This leads to ../programs invalid.

* tests: Replace relative paths for zstd/tests dir

* playTests: Set shell options explicitly, not in shebang

* playTests: Replace echo -e with printf

* meson: Fix test-zstd

Use std=gnu99 to build and test just like `make test`.

* meson: Fix legacy test

* meson: Enable testing in CI

Run build under release mode for faster test time.

* meson: Increase timeout time for test-zstream
2019-05-02 12:35:37 -07:00
Lzu Tao
5d900ff747 meson: Error out if fail to extracted version number 2019-04-30 13:12:00 +07:00
Lzu Tao
4107b734bd meson: Update default project version
* Update to use ninja v1.9.0 on CI
2019-04-27 01:42:22 +07:00
Nick Terrell
82d5a9870e [visual] Compile with ZSTD_MULTITHREAD=1
This branch `appveyorTests` runs the tests that `master` runs, so
passing Appveyor means that the tests are fixed.
2019-04-15 23:04:46 -07:00
Josh Soref
a880ca239b Spelling (#1582)
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
2019-04-12 11:18:11 -07:00
Lzu Tao
6c9b023f3e meson: Fix build 2019-04-12 21:28:36 +07:00
Yann Collet
fbdd30d68e fixed cmake build script for test programs 2019-04-10 17:47:01 -07:00
Yann Collet
2fa4f2e246 updated Visual projects
added timefn
2019-04-10 15:07:36 -07:00
Yann Collet
2c6b14ed22 fixed Windows header
cmake build script: added timefn
2019-04-10 14:54:13 -07:00
Nehal J Wani
7ac2052dbc
[libzstd] Specify soversion and version correctly for CMake build
Fixes #1512
2019-03-23 17:37:37 +05:30
Peter (Stig) Edwards
188c748208
-Wformat-security not needed with -Wformat=2 2019-02-01 09:30:31 +00:00
Hans Johnson
62c0dd7aff ENH: Simplify conditional logic
Based on excellent comment in #1489, the logic
to set cmake policies was simplified.  This will
provide an easier mechanism for maintaining the
conditional logic across many versions of cmake.
2019-01-08 14:25:56 -06:00
Erik Webb
9448a3790d Check CMake minor version support for VERSION_LESS_EQUAL
VERSION_LESS_EQUAL is only available to CMake 3.7+. This adds additional
logic to check that CMAKE_MINOR_VERSION is at least 7.

Fixes #1489
2019-01-03 15:26:20 -05:00
Hans Johnson
97d1de3d22 Provide forward compatible cmake paradigms
Automatically extract version information
from the zstd.h file.  Use naming of variables
consisent with modern cmake and https://semver.org/
(Semantic Versioning 2.0.0, MAJOR, MINOR, PATCH)

Modern versions of cmake provide consistent
paradigms for configuring project external
interface values.

This set of changes provide a back port of
some of cmake 3+ paradigms back to cmake 2.8.9.
Set and allow use of the current cmake policies
for newer versions of cmake when available to
allow for modern compiler features to be
utilized when available.

NOTE: The intent is that future modifications to
cmake will enable (conditional on cmake version support)
the ability to support modern linkage, and target
export mechanisms.  Those future changes will
make incorporating zstd into other packages
much easier.

This patch also allows the more rigourous error
checking of commmon cmake errors to be preformed
by cmake (i.e. more stringent syntax checking and
create errors for common hard to find misuses of
cmake variables).

This patch also provides support for modern
compiler support options by cmake (like
enabling interprocedural optimization
if link time optimizations are known to be supported
by the compiler envirionment.  IPO can be supported
by setting the CMAKE_INTERPROCEDURAL_OPTIMIZATION variable
for newer versions of cmake.
2018-12-28 13:47:35 -06:00
Yann Collet
23c73fe3e2
Merge pull request #1474 from hjmjohnson/cmake-set-default-build-type
Cmake set default build type
2018-12-28 11:02:22 -08:00
Yann Collet
6746de8dd6
Merge pull request #1478 from hjmjohnson/remove-duplicates-validation
BUG: list sub-command REMOVE_DUPLICATES requires list to be present
2018-12-28 10:50:03 -08:00
Yann Collet
bad7ed161a
Merge pull request #1483 from lzutao/meson-readme
meson: Update build guide
2018-12-28 09:46:13 -08:00
Yann Collet
663c55eba0
Merge pull request #1484 from lzutao/meson-pkgconf
meson: Correct generating pkgconf after Meson v0.49.0
2018-12-27 22:39:07 -08:00
Lzu Tao
787a72cdfe meson: Correct generating pkgconf after Meson v0.49.0 2018-12-28 11:20:33 +07:00
Lzu Tao
226cdffd69 meson: Update build guide [skip ci] 2018-12-28 11:17:11 +07:00
Yann Collet
470344d33e
Merge pull request #1479 from facebook/visualTest
Fix remaining Windows errors
2018-12-27 04:42:44 -08:00
Hans Johnson
1a279ae85a BUG: list sub-command REMOVE_DUPLICATES requires list to be present
When compiling without c++ enabled, some variables are not present.
This is a check enforced in recent Cmake versions.

CMake Error at CMakeModules/AddZstdCompilationFlags.cmake:54 (list):
  list sub-command REMOVE_DUPLICATES requires list to be present.
Call Stack (most recent call first):
  CMakeLists.txt:53 (ADD_ZSTD_COMPILATION_FLAGS)
2018-12-26 13:26:59 -06:00
Hans Johnson
77ef7847f7 Improve setting the default build type
The CMAKE_BUILD_TYPE variable is a CACHE variable
and should be set in a way that persists and is
documented in the CACHE.  Also set the default
values for the gui to ease selection of types.

These changes provide better support for GUI
configurators that support cmake.
2018-12-26 13:23:21 -06:00
Hans Johnson
304df8b883 ENH: Update to -DZSTD_LEGACY_SUPPORT=5 2018-12-26 13:18:39 -06:00
Yann Collet
1fdba696ca fixed VS2008 project
bench.c => benchfn.c + benchzstd.c
wrong util.c from prior patch

and warnings :
C4127
2018-12-25 16:14:14 -08:00
Hans Johnson
443b5c40e4 DOC: Provide documentation for cmake best practices. 2018-12-24 07:06:31 -06:00
Hans Johnson
d3fc848ddd STYLE: Remove CMake-language block-end command arguments
Ancient versions of CMake required else(), endif(), and similar block
termination commands to have arguments matching the command starting the block.
This is no longer the preferred style.
2018-12-22 19:32:39 -06:00
Hans Johnson
2f1ff84119 STYLE: Convert CMake-language commands to lower case
Ancient CMake versions required upper-case commands.  Later command names
became case-insensitive.  Now the preferred style is lower-case.
2018-12-22 19:32:39 -06:00
Yann Collet
5bdbd997ae
Merge pull request #1452 from lzutao/meson_getversion
meson: Remove unused sys import
2018-12-14 16:50:41 -08:00
Lzu Tao
066cfc069b meson: Remove unused sys import 2018-12-14 11:03:04 +07:00
Lzu Tao
3ee5504fb2 Simplify logic by setting default value for MESON_INSTALL_DESTDIR_PREFIX 2018-12-13 18:07:01 +07:00
Lzu Tao
ce22f76668 meson: Update man1 extension on meson 0.49.0 2018-12-13 14:58:17 +07:00
Lzu Tao
abfde03cb5 meson: Update meson symlink script usage 2018-12-13 14:58:17 +07:00
Lzu Tao
fa2fc274fd meson: Correct support for building on Windows
Let soversion base on version if not set. For example, if version is 3.6.0
and soversion is not defined, it is set to 3.
2018-12-13 14:58:17 +07:00
Lzu Tao
67babb6d23 Replace many os.path methods with pathlib one's
Use MESON_INSTALL_DESTDIR_PREFIX variable instead of DESTDIR.
2018-12-13 14:58:17 +07:00
Lzu Tao
f897523655 meson: Update usage of InstallSymlink helper 2018-12-03 11:02:42 +07:00
Lzu Tao
9d6cf606f8 meson: Update tests timeout to run properly 2018-12-03 03:23:09 +07:00
Lzu Tao
65507666bb Use -Dlegacy_level build option to control ZSTD_LEGACY_SUPPORT macro in test 2018-12-03 00:36:40 +07:00
Lzu Tao
437ec5f47f meson helper: Use Python conventional name for naming function [skip ci] 2018-12-02 22:48:11 +07:00
Lzu Tao
838de08439 meson: Fix soversion 2018-12-02 22:45:46 +07:00
Lzu Tao
23d751507e meson: Use -werror build option instead of adding -Werror [skip ci] 2018-12-02 22:33:43 +07:00
Lzu Tao
38728b4518 Use argparse instead of manually parsing [skip ci] 2018-12-02 22:33:43 +07:00
Lzu Tao
7da18bc85a Add missed .1 extension for manpage [skip ci] 2018-12-02 00:18:00 +07:00
Lzu Tao
cbf2a924dd meson: Cleanup installing symlinks 2018-12-01 23:18:59 +07:00
Lzu Tao
519b2de6f7 Update README 2018-12-01 23:18:59 +07:00
Lzu Tao
8d5252b68d meson: Remove redundant join_paths function 2018-12-01 23:18:59 +07:00
Lzu Tao
24bc513ea1 meson: Change build options' name 2018-12-01 23:18:59 +07:00
Lzu Tao
39f49ac39f Add almost all test cases in tests/Makefile 2018-12-01 23:18:59 +07:00
Lzu Tao
c55d65b35d Add clang cflags 2018-12-01 23:18:59 +07:00
Lzu Tao
0a0683f5b2 Initial support for Windows build 2018-12-01 23:18:59 +07:00
Lzu Tao
2337429e8d Change tabsize 2018-12-01 23:18:59 +07:00
Lzu Tao
c4fb45ffe8 Add comment about looking for dependencies 2018-12-01 23:18:59 +07:00
Lzu Tao
918e0d5c36 Rename some meson build options to match autoconf convention 2018-12-01 23:18:59 +07:00
Lzu Tao
9ac9cbc9de Enable debugging flag 2018-12-01 23:18:59 +07:00
Lzu Tao
0669392937 Update relevant files after moving meson to build/
Update meson build instructions
2018-12-01 23:18:59 +07:00
Lzu Tao
beb13bd87e Move contrib/meson to build/meson 2018-12-01 23:18:59 +07:00
Yann Collet
e4abd28769 fixed VS2010+ build script 2018-11-13 13:22:16 -08:00
Yann Collet
f15312e884 fixed cmake build script 2018-11-13 13:16:41 -08:00
Antoine Pitrou
2186fbcf67 Allow creating debug builds with CMake
Only set the build type to Release if the user didn't select any.
See discussion in https://github.com/facebook/zstd/pull/1278#issuecomment-436327923
and below.
2018-11-06 21:24:29 +01:00
Yann Collet
4ed9233db6 fixed VS2010 project 2018-10-26 11:25:40 -07:00
Yann Collet
2b4914082e created zstd_decompress_block module
isolate all logic associated with block decompression
into its own module.

zstd_decompress is still in charge
of context creation/destruction,
frames, headers, streaming, special blocks, etc.

Compressed blocks themselves are now handled within zstd_decompress_block .
2018-10-25 16:28:41 -07:00
Yann Collet
2f7e1c8737 fixed VS2010 build script 2018-10-23 17:44:43 -07:00
Yann Collet
ccd2d426fc separate DDict logic into its own module
created zstd_ddict.c within lib/decompress
2018-10-23 17:25:49 -07:00
Yann Collet
f2cff22804
Merge pull request #1368 from rkjain89/test-branch
Moving Code To util.c
2018-10-19 16:31:10 -07:00
Rohit Jain
c430c1068b Made the changes in zstd vcproj file for VS2008 as requested 2018-10-16 23:33:17 -07:00
Sam Russell
490f652974 Add man pages for zstdgrep and zstdless 2018-10-15 22:25:03 +13:00
Rohit Jain
9056fe2290 Adding util.c to zstd VS file 2018-10-12 14:22:05 -07:00
Rohit Jain
b91f982e1e Adding chagnes for cmake and VStudio 2018-10-12 14:08:13 -07:00
Yann Collet
33d643124d fixed fullbench-lib target 2018-10-04 18:12:36 -07:00
Yann Collet
9ac8f2d7b9 fixed VS2017Community build script
reported by @epicabsol
2018-10-03 18:42:44 -07:00
Yann Collet
b2939163e1 Changed default legacy support to v0.5+
thus dropping read support for v0.4.

It's always possible to re-enable it, by changing build macro ZSTD_LEGACY_SUPPORT to 4.
2018-09-20 14:30:20 -07:00
Jennifer Liu
9d6ed9def3 Merge fastCover into DictBuilder (#1274)
* Minor fix

* Run non-optimize FASTCOVER 5 times in benchmark

* Merge fastCover into dictBuilder

* Fix mixed declaration issue

* Add fastcover to symbol.c

* Add fastCover.c and cover.h to build

* Change fastCover.c to fastcover.c

* Update benchmark to run FASTCOVER in dictBuilder

* Undo spliting fastcover_param into cover_param and f

* Remove convert param functions

* Assign f to parameter

* Add zdict.h to Makefile in lib

* Add cover.h to BUCK

* Cast 1 to U64 before shifting

* Remove trimming of zero freq head and tail in selectSegment and rebenchmark

* Remove f as a separate parameter of tryParam

* Read 8 bytes when d is 6

* Add trimming off zero frequency head and tail

* Use best functions from COVER and remove trimming part(which leads to worse compression ratio after previous bugs were fixed)

* Add finalize= argument to FASTCOVER to specify percentage of training samples passed to ZDICT_finalizeDictionary

* Change nbDmer to always read 8 bytes even when d=6

* Add skip=# argument to allow skipping dmers in computeFrequency in FASTCOVER

* Update comments and benchmarking result

* Change default method of ZDICT_trainFromBuffer to ZDICT_optimizeTrainFromBuffer_fastCover

* Add dictType enum and fix bug about passing zParam when converting to coverParam

* Combine finalize and skip into a single parameter

* Update acceleration parameters and benchmark on 3 sample sets

* Change default splitPoint of FASTCOVER to 0.75 and benchmark first 3 sample sets

* Initialize variables outside of for loop in benchmark.c

* Update benchmark result for hg-manifest

* Remove cover.h from install-includes

* Add explanation of f

* Set default compression level for trainFromBuffer to 3

* Add assertion of fastCoverParams in DiB_trainFromFiles

* Add checkTotalCompressedSize function + some minor fixes

* Add test for multithreading fastCovr

* Initialize segmentFreqs in every FASTCOVER_selectSegment and move mutex_unnlock to end of COVER_best_finish

* Free segmentFreqs

* Initialize segmentFreqs before calling FASTCOVER_buildDictionary instead of in FASTCOVER_selectSegment

* Add FASTCOVER_MEMMULT

* Minor fix

* Update benchmarking result
2018-08-23 12:06:20 -07:00
Timo Gurr
e400a86f17 Use GNUInstallDirs DOCDIR for installing the manual 2018-08-17 13:50:00 +02:00
Eden Zik
a6df961497 Cmake now builds with CMAKE_BUILD_TYPE=Release by default, both while
being invoked from the main Makefile (via cmakebuild) or directly from
the build/cmake directory. Suggested by @pdknsk (#1081).
2018-08-13 20:28:52 -04:00
cyan4973
e85b4c56b2 speed up appveyor tests 2018-07-30 17:13:13 +02:00
Yann Collet
689bfecd48
Merge pull request #1188 from GeorgeLu97/BenchModule
Bench module
2018-07-02 13:33:27 -07:00
oleid
4e196b2ac3
Correct multithread logic, fixing 'unsupported parameter' error
The original conditions only worked, when both, static and shared variants where built, resulting in an inconsistency between programs and library. The program was built with MT support enabled, the library not. That lead to error 11 'unsupported parameter' when compressing anything with the command line tool.

When changing the AND condition to `ZSTD_MULTITHREAD_SUPPORT AND (ZSTD_BUILD_SHARED OR ZSTD_BUILD_SHARED)`, cmake stopps complaining one of the targets wasn't built. This commit works for any case.
2018-06-26 08:36:41 +02:00
cyan4973
ae0b7ffa0a made Visual Studio compatible with DEBUGLEVEL >= 2 2018-06-20 09:45:02 -07:00
George Lu
8522346322 Make Fullbench use new function
Rearrange Args
Add nothing function
Use new function, change locals to match
New Display
Comment cleanup
Change builds
2018-06-15 11:37:49 -04:00
Yann Collet
507bef196d added debug.h for cmake 2018-06-14 19:48:40 -04:00
Yann Collet
78e5e2887e Visual Studio project blind fix 2018-06-14 16:30:52 -04:00
Yann Collet
2d76defbfe grouped all histogram functions into hist.c
renamed functions with HIST_* prefix
2018-06-13 19:49:31 -04:00
cyan4973
b095bffac6 ignore Windows build/test artefacts 2018-05-01 05:45:46 -07:00
Nick Terrell
268cc2a516 Fix CMake build
`MAN_INSTALL_DIR` was undefined.

Tested with `make cmakebuild`, and saw that the man pages got installed to the right place:

```
-- Installing: /Users/terrelln/install_test_dir/lib/pkgconfig/libzstd.pc
-- Installing: /Users/terrelln/install_test_dir/include/zstd.h
-- Installing: /Users/terrelln/install_test_dir/include/zbuff.h
-- Installing: /Users/terrelln/install_test_dir/include/zdict.h
-- Installing: /Users/terrelln/install_test_dir/include/zstd_errors.h
-- Installing: /Users/terrelln/install_test_dir/lib/libzstd.1.3.4.dylib
-- Installing: /Users/terrelln/install_test_dir/lib/libzstd.dylib
-- Installing: /Users/terrelln/install_test_dir/lib/libzstd.a
-- Installing: /Users/terrelln/install_test_dir/bin/zstd
-- Up-to-date: /Users/terrelln/install_test_dir/bin/zstd
-- Installing: /Users/terrelln/install_test_dir/bin/zstdcat
-- Installing: /Users/terrelln/install_test_dir/bin/unzstd
-- Installing: /Users/terrelln/install_test_dir/share/man/man1/zstd.1
-- Installing: /Users/terrelln/install_test_dir/share/man/man1/zstdcat.1
-- Installing: /Users/terrelln/install_test_dir/share/man/man1/unzstd.1
-- Installing: /Users/terrelln/install_test_dir/bin/zstdmt
-- Installing: /Users/terrelln/install_test_dir/bin/pzstd
-- Installing: /Users/terrelln/install_test_dir/zstd_manual.html
```
2018-03-26 14:59:26 -07:00
Yann Collet
bffb4b46a1 updated VS studio readme 2018-03-26 09:55:10 -07:00
Yann Collet
2dc31dc842
Merge pull request #1065 from HaydnTrigg/dev
Generic Visual Studio 2017 build script
2018-03-26 09:21:12 -07:00
Haydn Trigg
2960f74c29 Swapped VS2017 preference order 2018-03-26 02:03:53 +10:30
Haydn Trigg
57e6b98eb1 Refined build script 2018-03-22 09:10:39 +10:30
Haydn Trigg
321de5682c Generic Visual Studio 2017 build script 2018-03-21 22:57:42 +10:30
Varunram
0034c759b0 Update man page variables in CMakeLists 2018-03-21 14:10:42 +05:30
Varunram
90c598f089 Add missing checks to CMakeLists;closes #1023 2018-03-18 15:48:58 +05:30
Haydn Trigg
c6351021e4 Visual Studio 2017 build scripts 2018-03-11 00:15:31 +10:30
Alexey Ivanov
22303da601 CMake: fixed multithreading build on Windows
`ZSTD_MULTITHREAD_SUPPORT` option fixed for Windows.

Signed-off-by: Alexey Ivanov <alexey.ivanes@gmail.com>
2018-01-17 10:27:52 +03:00
Alexey Ivanov
403e2db139 CMake: use GNUInstallDirs for library install dir
Libraries now will be installed in the correct directory on x86_64 linux systems,
and can be changed with `-DCMAKE_INSTALL_LIBDIR=<dirname>` option.
2018-01-15 22:48:46 +03:00
Yann Collet
72c0f4574c cmake: fixed lib/CMakeLists.txt
changed file name from zstd_compress.h to zstd_compress_internal.h
2017-11-10 09:39:11 -08:00
Stella Lau
360428c5d9 Move ldm functions to their own file 2017-09-06 18:09:26 -07:00