AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
8,454 Commits 1 Branch 0 Tags 27 MiB
master
Commit Graph

198 Commits

Author SHA1 Message Date
Yann Collet
5373e44ba7 fixed contrib/adaptive-compression 2018-03-15 17:10:15 -07:00
Yann Collet
97816400ca added /contrib projects to make all 2018-03-15 16:40:14 -07:00
Yann Collet
cb5eba8e20 add zcat symlink support, suggested by @wtarreau 
added some test
also updated relevant doc

+ fixed a mistake in `lz4` symlink support :
  lz4 utility doesn't remove source files by default (like zstd, but unlike gzip).
  The symlink must behave the same.
 2018-01-19 11:26:35 -08:00
Yann Collet
f4e58455f6 ensure MOREFLAGS are not lost in root->tests Makefile invocation 2018-01-16 11:50:16 -08:00
Yann Collet
19cb37f837 travis ci : added gcc-7 test 
also added `-Werror` to sanitizer tests
 2018-01-16 11:40:42 -08:00
Yann Collet
c6f5bf071c fixed powerpc test 
misspelled CFLAGS
 2017-12-01 17:35:29 -08:00
Yann Collet
5b957ba899 minor interface adjustments 2017-11-17 01:21:40 -08:00
Yann Collet
05dffe43a7 Fixed Btree update 
ZSTD_updateTree() expected to be followed by a Bt match finder, which would update zc->nextToUpdate.
With the new optimal match finder, it's not necessarily the case : a match might be found during repcode or hash3, and stops there because it reaches sufficient_len, without even entering the binary tree.
Previous policy was to nonetheless update zc->nextToUpdate, but the current position would not be inserted, creating "holes" in the btree, aka positions that will no longer be searched.
Now, when current position is not inserted, zc->nextToUpdate is not update, expecting ZSTD_updateTree() to fill the tree later on.

Solution selected is that ZSTD_updateTree() takes care of properly setting zc->nextToUpdate,
so that it no longer depends on a future function to do this job.

It took time to get there, as the issue started with a memory sanitizer error.
The pb would have been easier to spot with a proper `assert()`.
So this patch add a few of them.

Additionnally, I discovered that `make test` does not enable `assert()` during CLI tests.
This patch enables them.

Unfortunately, these `assert()` triggered other (unrelated) bugs during CLI tests, mostly within zstdmt.
So this patch also fixes them.

- Changed packed structure for gcc memory access : memory sanitizer would complain that a read "might" reach out-of-bound position on the ground that the `union` is larger than the type accessed.
  Now, to avoid this issue, each type is independent.
- ZSTD_CCtxParams_setParameter() : @return provides the value of parameter, clamped/fixed appropriately.
- ZSTDMT : changed constant name to ZSTDMT_JOBSIZE_MIN
- ZSTDMT : multithreading is automatically disabled when srcSize <= ZSTDMT_JOBSIZE_MIN, since only one thread will be used in this case (saves memory and runtime).
- ZSTDMT : nbThreads is automatically clamped on setting the value.
 2017-11-16 12:18:56 -08:00
W. Felix Handte
e2bf70ece1 Add check Target to Makefile 2017-10-17 16:02:39 -04:00
Yann Collet
e580dc6a4a Merge pull request #860 from felixhandte/zstd-lz4-support-tests 
Add Default LZ4 Support When Available
 2017-09-29 22:32:54 -07:00
W. Felix Handte
5705d9f25a Add basic tests for the lz4 integration 2017-09-28 19:16:43 -04:00
Yann Collet
bc32b40b98 reduced zstreamtest --mt memory load 
adjust compression level, hence memory usage, depending on nb threads
in order to run correctly on memory-starved VM.
 2017-09-27 17:27:38 -07:00
Yann Collet
60ca44b545 switched name to cxxtest 2017-09-27 12:24:13 -07:00
Yann Collet
763f8b5e45 Change c++ test to use CXX and CXXFLAGS environment variables 
Fix OS-X warning on compiling .c files with clang++
Also changed test name from gpptest to cpptest, since it's no longer g++ specific
 2017-09-27 12:09:52 -07:00
Nick Terrell
6bb781e0f1 [fuzz] Add regressiontest targets 2017-09-25 15:31:33 -07:00
Yann Collet
8e298382a8 changed target allarch into allzstd 
allzstd contains only zstd-related tests.
allmost = allzstd + zwrapper tests (which require zlib)
 2017-08-31 14:30:52 -07:00
Yann Collet
e9dc204f42 fixed a bunch of headers after license change (#825) 2017-08-31 11:24:54 -07:00
Yann Collet
166645e7b3 fixed zstd-compress 
file-information is dependent on decompression functions.
it should only be enabled when ZSTD_NODECOMPRESS is not set.

also : added zstd-compress compilation test into `make shortest`
 2017-08-18 18:30:41 -07:00
Yann Collet
0ab9d55e7a Merge branch 'dev' of github.com:facebook/zstd into dev 2017-08-11 10:17:49 -07:00
Nick Terrell
9ba97182d1 [CI] Add gcc7build test 2017-08-08 13:28:56 -07:00
Yann Collet
ceda7a9a58 minor Makefile refactor 2017-07-21 11:44:39 -07:00
Yann Collet
842644e42e target gpptest uses CXX environment variable 2017-07-07 15:55:41 -07:00
Yann Collet
afb0aca739 zstreamtest : big tests are only enabled in 64-bits mode 
to avoid requesting too much memory in 32-bits mode during MT tests
 2017-06-29 18:19:09 -07:00
Nick Terrell
c2edb78cbe [tests] Fix HAVE_LZMA flag 2017-06-26 22:58:31 -07:00
Nick Terrell
eb7c2074e8 [zstdcli] Disable lzma support for MSAN testing 2017-06-26 11:23:56 -07:00
Yann Collet
6122af9cf5 --no-big-tests for msan 
msan tests require too much RAM for 2 GB VM
 2017-06-23 00:37:15 -07:00
Yann Collet
bfc2f00080 --no-big-tests for zstreamtest 
Apply --no-big-tests for tsan tests
 2017-06-21 17:57:14 -07:00
cyan4973
8bcbf42617 fixed g++ prototype mismatch 2017-06-04 23:52:00 -07:00
Yann Collet
16f9c572fc Merge branch 'dev' into compressionFlow 2017-04-20 11:16:40 -07:00
Milan Ševčík
f49f760b41 Test new cmake branches with Circle CI 2017-04-19 21:34:17 +02:00
Yann Collet
b402f1ef37 added make list 2017-04-18 14:34:24 -07:00
Yann Collet
6851db48e0 created contrib/cleanTabs 2017-03-30 12:42:44 -07:00
Yann Collet
e498ed5f16 Merge pull request #611 from Majlen/cmake-improvement 
Further cmake improvement
 2017-03-21 11:23:27 -07:00
Milan Ševčík
0d09c778a5 Add build of new cmake branches to Makefile 2017-03-21 11:11:53 +01:00
Milan Ševčík
8d562311c1 Test new cmake stuff 2017-03-21 11:11:53 +01:00
Milan Ševčík
a8c0c2af10 Fix cmake install prefix with cmake-3.x 2017-03-21 10:43:11 +01:00
Sean Purcell
0b0b10ec83 Add --no-big-tests flag for qemu fuzz-tests 2017-03-20 11:32:24 -07:00
Sean Purcell
caf0ee8d20 Make signed integer overflow recoverable in UBsan 2017-03-09 17:28:08 -08:00
Sean Purcell
daec40db24 Update .travis.yml and Makefile for medium tests 2017-03-09 16:05:22 -08:00
Sean Purcell
3437bf2feb Add build targets to the Makefile, and update CircleCI tests 2017-03-06 15:05:02 -08:00
Yann Collet
8b1d004031 added -Wformat-security flag, as recommended by @pixelb 2017-03-05 21:17:32 -08:00
Yann Collet
27526c7201 make : added target shortest 
shortest only run fast part of playTests.sh .
cc @iburinoc
 2017-03-01 17:02:49 -08:00
Yann Collet
db478ad27e Merge pull request #558 from facebook/manual 
Manual
 2017-02-23 13:39:36 -08:00
Yann Collet
88ba64702e fixed c90/gnu90/gnu99 tests 2017-02-22 10:52:36 -08:00
Przemyslaw Skibinski
d41c048394 added arm-ppc-compilation Makefile target 2017-02-22 11:07:28 +01:00
Yann Collet
7757577341 added manual target in root Makefile 
`manual` target is added to `all` target
 2017-02-22 01:10:43 -08:00
Przemyslaw Skibinski
21911ad6cb move Ubuntu packages install to Makefile 2017-02-22 08:54:56 +01:00
Przemyslaw Skibinski
5dd18b314b travis.yml: reduce number of jobs to 7 2017-02-22 08:15:17 +01:00
Przemyslaw Skibinski
f58ac79f51 fix uasan-test32 2017-02-21 23:40:21 +01:00
Przemyslaw Skibinski
8a51c69218 travis.yml: added uasan-test and uasan-test32 2017-02-21 22:48:04 +01:00
Yann Collet
c2a4632789 release builds use less debug symbols and warnings 
release build are triggered through either `make`,
or their specific target `make zstd-release` and `make lib-release`.
 2017-02-02 20:54:41 -08:00
Yann Collet
d7e3cb58c5 Resolved merge conflict dev+zstdmt 2017-01-20 16:44:50 -08:00
Yann Collet
900f39e709 skip zstdmt at root directory 2017-01-20 16:36:29 -08:00
Yann Collet
563ef8acf4 CCtxPool starts empty, as suggested by @terrelln 
Also : make zstdmt now a target from root
 2017-01-18 12:12:10 -08:00
Yann Collet
33fce03045 added test checking dictID when using ZSTD_initCStream_usingCDict() 
It shows that dictID is not properly added into frame header
 2017-01-16 19:46:22 -08:00
Yann Collet
5eb749e734 ZSTDMT_compress() creates a single frame 
The new strategy involves cutting frame at block level.
The result is a single frame, preserving ZSTD_getDecompressedSize()

As a consequence, bench can now make a full round-trip,
since the result is compatible with ZSTD_decompress().

This strategy will not make it possible to decode the frame with multiple threads
since the exact cut between independent blocks is not known.
MT decoding needs further discussions.
 2017-01-11 18:21:25 +01:00
Yann Collet
7cedbd1936 fixed allmost target 2016-12-22 12:43:00 +01:00
Yann Collet
a86a09ea0d removed examples from standard C tests, since they contain some POSIX elements 2016-12-22 11:31:39 +01:00
Yann Collet
1f57c2ed32 added : ZSTD_createCDict_byReference() 2016-12-21 16:20:11 +01:00
Yann Collet
c0a1d6deb0 better cleaning 2016-12-07 15:58:32 -08:00
Przemyslaw Skibinski
e579ab5faa introduced QEMU_SYS 2016-11-14 12:57:05 +01:00
Yann Collet
d41380ea5d make zstd a phony target at root 2016-11-07 14:55:12 -08:00
Przemyslaw Skibinski
3a415594b1 fixed MinGW compilation 2016-11-03 12:59:20 +01:00
Yann Collet
f3f13211ae Fix #419 : no warning when setting custom LDFLAGS 2016-11-02 17:02:45 -07:00
Yann Collet
e405c7a126 updated visual projects 2016-10-12 15:29:22 -07:00
Yann Collet
e19111c42f make creates libzstd binaries (#415) 2016-10-12 11:09:36 -07:00
inikep
68cd4766c9 initialization of strm->adler 2016-09-23 12:42:21 +02:00
inikep
4602e53021 added valgrindTest for zlibWrapper 2016-09-23 10:43:37 +02:00
inikep
bd76017329 Merge remote-tracking branch 'refs/remotes/facebook/dev' into zlibWrapper 2016-09-22 10:28:13 +02:00
inikep
3be5c6e6db improved zlibWrapper\Makefile 2016-09-22 10:23:58 +02:00
Yann Collet
0704df3259 fixed cmake test 2016-09-19 16:55:35 +02:00
Yann Collet
4c9a4c18a9 changed projects to build 2016-09-19 14:58:14 +02:00
Yann Collet
a2664649df better error handling 2016-09-09 19:33:56 +02:00
Thomas Klausner
b85cdabd50 Enable install targets for NetBSD. 2016-09-04 14:37:57 +02:00
Yann Collet
599c69d917 minor Makefile updates 2016-08-30 13:33:20 -07:00
Yann Collet
cb5a320705 made -Wdocumentation a clang only flag 2016-08-26 08:06:36 +02:00
inikep
2d5eb2beb9 updated .travis.yml and appveyor.yml for tests/ 2016-08-18 13:53:26 +02:00
inikep
a1c8115df5 improved tests\Makefile 2016-08-18 13:35:36 +02:00
inikep
156ab8f4a1 APT dependencies moved from Makefile to .travis.yml 2016-08-17 14:17:54 +02:00
Yann Collet
6c36449ef3 minor comments 2016-08-12 00:05:59 +02:00
Yann Collet
2a2ba3691c Merge pull request #266 from jrmarino/master 
Enable build on FreeBSD ports (includes DragonFly BSD) [dev branch]
 2016-07-31 02:21:26 +02:00
jrmarino
0d07ec0c0c Enable build on FreeBSD ports (includes DragonFly BSD) 
Zstd has been introduced to FreeBSD ports
(http://www.freshports.org/archivers/zstd/) which DragonFly BSD also
uses.  FreeBSD and DragonFly use the install targets (albeit modified in
some cases) so they must be added to the associated Makefile filters.
 2016-07-30 19:11:15 -05:00
inikep
003c7a8568 optimal parser: removed ZSTD_REP_INIT 2016-07-27 11:07:13 +02:00
inikep
5e0ed484f5 Travis CI: removed -Werror from powerpc compilation 2016-07-26 08:37:53 +02:00
Yann Collet
32faf6c8e7 fixed conversion warnings 2016-07-22 14:37:09 +02:00
Yann Collet
98c8884999 added target zstd in root Makefile 2016-07-15 16:12:38 +02:00
Yann Collet
677ed26aa7 Added examples/Makefile 2016-07-10 14:25:38 +02:00
Yann Collet
c35cd5b1db New : Zstandard Frame format specification 2016-06-30 15:40:28 +02:00
Yann Collet
8d30cd562a Merge pull request #219 from inikep/dev 
added gcc5test and gcc6test for Travis
 2016-06-21 21:51:52 +02:00
inikep
c9bb102401 Makefile: added gcc5test and gcc6test 
improved Travis tests
 2016-06-21 18:04:13 +02:00
Yann Collet
06d9a73b48 minor refactor, using WILDCOPY_OVERLENGTH macro instead of hard-coded 8 2016-06-19 14:27:21 +02:00
Yann Collet
19cab46f2f Joined seqStore initialization at dispatch point 2016-06-17 12:54:52 +02:00
Yann Collet
c28f9e2713 attempt to re-enable msan tests 2016-06-06 18:00:00 +02:00
inikep
c7f008b069 Travis cmaketest will also run "make install" and "make uninstall" 2016-06-03 11:40:15 +02:00
inikep
7cab86f647 ZWRAP_allocFunction and ZWRAP_freeFunction are static 2016-06-02 18:24:07 +02:00
inikep
13f42d9085 VS2010 project: reverted zstdlib.rc 2016-06-01 14:44:31 +02:00
inikep
a6b942018d Merge remote-tracking branch 'refs/remotes/origin/dev' into dev070 
# Conflicts:
#	.travis.yml
#	Makefile
#	lib/common/zstd_static.h
#	programs/Makefile
#	projects/VS2008/zstd/zstd.vcproj
#	projects/VS2008/zstdlib/zstdlib.vcproj
#	projects/cmake/lib/CMakeLists.txt
#	projects/cmake/programs/CMakeLists.txt
 2016-06-01 00:07:09 +02:00
Yann Collet
815580a538 added cmake --version to check cmake version on Travis 2016-05-30 18:57:39 +02:00
Yann Collet
118bfc4ccb fixed cmaketest 2016-05-30 18:50:28 +02:00
inikep
1981d5a730 Travis tests for PowerPC and OS X 2016-05-30 10:18:56 +02:00