AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7,585 Commits 1 Branch 0 Tags 27 MiB
efc656c9a6
Commit Graph

7585 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet
efc656c9a6
Merge pull request #2114 from facebook/verbose 
support for verbose make
 2020-05-07 13:16:33 -07:00
Yann Collet
1afe57cff7
Merge pull request #2112 from facebook/cfast 
small speed improvement for strategy fast
 2020-05-07 13:13:34 -07:00
caoyzh
969ba4f2b9 Change the modification of ZSTD_wildcopy() 2020-05-07 13:10:46 -07:00
caoyzh
a7e34ff693 revert ZSTD_reduceTable_internal()'s modificatiion 2020-05-07 13:10:46 -07:00
caoyzh
9e802ede9c Modify indent of comments 2020-05-07 13:10:46 -07:00
caoyzh
7f75f05e84 Change "arm_neon.h" to system include <arm_neon.h> 2020-05-07 13:10:46 -07:00
caoyzh
b2e56f7f7f Optimize compression by using neon function. 2020-05-07 13:10:46 -07:00
Nick Terrell
45c66dd298 [zdict] Stabilize ZDICT_finalizeDictionary() 2020-05-07 10:37:01 -07:00
Bimba Shrestha
625924774e
Merge pull request #2115 from bimbashrestha/docs 
[doc] Adding static analysis section and note
 2020-05-07 12:03:33 -05:00
Bimba Shrestha
1b1cc3d1b7 adding some static-anslysis docs 2020-05-07 10:38:06 -05:00
Yann Collet
cf854f4660 support for verbose make 
A commonly accepted makefile idiom is V=1 or VERBOSE=1
to request the printing of all commands.

This is not "default" though, and must be manually added.

Example :
Before :
```
make libzstd
compiling dynamic library 1.4.5
creating versioned links

make libzstd V=1
compiling dynamic library 1.4.5
creating versioned links
```

After :
```
make libzstd
compiling dynamic library 1.4.5
creating versioned links

make libzstd V=1
compiling dynamic library 1.4.5
cc -DXXH_NAMESPACE=ZSTD_ -DZSTD_LEGACY_SUPPORT=5 -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wmissing-prototypes -Wc++-compat  -O3 common/debug.c common/entropy_common.c common/error_private.c common/fse_decompress.c common/pool.c common/threading.c common/xxhash.c common/zstd_common.c compress/fse_compress.c compress/hist.c compress/huf_compress.c compress/zstd_compress.c compress/zstd_compress_literals.c compress/zstd_compress_sequences.c compress/zstd_compress_superblock.c compress/zstd_double_fast.c compress/zstd_fast.c compress/zstd_lazy.c compress/zstd_ldm.c compress/zstd_opt.c compress/zstdmt_compress.c decompress/huf_decompress.c decompress/zstd_ddict.c decompress/zstd_decompress.c decompress/zstd_decompress_block.c deprecated/zbuff_common.c deprecated/zbuff_compress.c deprecated/zbuff_decompress.c dictBuilder/cover.c dictBuilder/divsufsort.c dictBuilder/fastcover.c dictBuilder/zdict.c legacy/zstd_v05.c legacy/zstd_v06.c legacy/zstd_v07.c -shared -fPIC -fvisibility=hidden -Wl,-soname=libzstd.so.1 -o libzstd.so.1.4.5
creating versioned links
ln -sf libzstd.so.1.4.5 libzstd.so.1
ln -sf libzstd.so.1.4.5 libzstd.so
```
 2020-05-07 08:04:10 -07:00
Yann Collet
54144285fd small speed improvement for strategy fast 
gcc 9.3.0 :
kennedy : 459 -> 466
silesia : 360 -> 365
enwik8  : 267 -> 269

clang 10.0.0 :
kennedy : 436 -> 441
silesia : 364 -> 366
enwik8  : 271 -> 272
 2020-05-07 06:15:58 -07:00
Nick Terrell
5717bd39ee [lib] Fix NULL pointer dereference 
When the output buffer is `NULL` with size 0, but the frame content size
is non-zero, we will write to the NULL pointer because our bounds check
underflowed.

This was exposed by a recent PR that allowed an empty frame into the
single-pass shortcut in streaming mode.

* Fix the bug.
* Fix another NULL dereference in zstd-v1.
* Overflow checks in 32-bit mode.
* Add a dedicated test.
* Expose the bug in the dedicated simple_decompress fuzzer.
* Switch all mallocs in fuzzers to return NULL for size=0.
* Fix a new timeout in a fuzzer.

Neither clang nor gcc show a decompression speed regression on x86-64.
On x86-32 clang is slightly positive and gcc loses 2.5% of speed.

Credit to OSS-Fuzz.
 2020-05-06 12:09:02 -07:00
Felix Handte
ad8dbae1b7
Merge pull request #2103 from felixhandte/relative-includes 
Migrate Includes to Relative Paths
 2020-05-06 09:42:23 -07:00
Bimba Shrestha
e7df0d41bb
Merge pull request #2095 from bimbashrestha/grep 
[bugs] zstdgrep/grep inconsistencies
 2020-05-06 11:18:15 -05:00
Yann Collet
d116f24070
Merge pull request #2109 from facebook/fasterCygwin 
faster cygwin test
 2020-05-05 20:50:40 -07:00
Bimba Shrestha
250184adf6 adding tests back 2020-05-05 16:51:06 -07:00
Yann Collet
e09cdf9080
Merge pull request #2108 from facebook/hufc_wconv 
some more conversion fixes …
 2020-05-05 16:38:17 -07:00
Yann Collet
4f755bdafc faster cygwin test 
The cygwin test on Appveyor is way too long (>30 mn).
The main issue is the very long zstreamtest, because no time out has been programmed.
Ensures that both fuzzer and zstreamtest receive a 30sec slot.
Should reduce cygwin tests duration by -30mn.
 2020-05-05 14:08:35 -07:00
W. Felix Handte
5163778c02 Yay It Works, Clean Up Logging A Bit 2020-05-05 15:27:27 -04:00
W. Felix Handte
005ffd3bdb Fix Sed Regex Escaping for Extended Regex Rules...... 2020-05-05 14:54:04 -04:00
W. Felix Handte
bdfb50f7c4 Uhhh It's Still Failing, Let's Try Printing Debug Info? 2020-05-05 13:58:22 -04:00
Yann Collet
ef7d98e55a Merge branch 'dev' into hufc_wconv 2020-05-05 10:23:00 -07:00
W. Felix Handte
229cbd056e Fix Fallbacks to Work on FreeBSD 2020-05-05 13:18:45 -04:00
Yann Collet
c29fd7cd8b some more conversion warnings 
hunting down some static analyzer warnings
 2020-05-05 10:16:59 -07:00
Bimba Shrestha
6ff16b4246 adding help text 2020-05-05 10:03:57 -07:00
Bimba Shrestha
a06a508bf3 fixing inconsistencies 2020-05-05 09:56:15 -07:00
Yann Collet
7120d97cb3
fix minor conversion warnings (#2106) 2020-05-05 09:08:05 -07:00
W. Felix Handte
6381b7ee3f Add Fallback When realpath Doesn't Support --relative-to 2020-05-04 18:59:47 -04:00
W. Felix Handte
c7cba8e87c Mark zstddeclib-in.c Not Executable 2020-05-04 18:59:26 -04:00
Yann Collet
c1b836f4c3 fix minor conversion warnings 2020-05-04 14:43:09 -07:00
Felix Handte
8b327149a8
Merge pull request #1976 from felixhandte/minimal-lib-target 
Add Minification Variable to `lib/Makefile`
 2020-05-04 12:42:56 -07:00
W. Felix Handte
b48f6c7d26 Try to Fix Single File Library Combiner Script to Handle Relative Includes 2020-05-04 15:20:26 -04:00
W. Felix Handte
7b75d772b1 Remove Useless Assignment in Makefile 2020-05-04 15:20:26 -04:00
W. Felix Handte
7dcca6bc64 Also Move programs/ Directory to Relative Includes 2020-05-04 15:20:26 -04:00
W. Felix Handte
6028827fee Rewrite Include Paths to be Relative 
Addresses #1998.
 2020-05-04 15:20:26 -04:00
Bimba Shrestha
0301ef5d04
[bench] Extending largeNbDicts to compression (#2089) 
* adding cdict_collection_t

* adding shuffleCDictionaries()

* adding compressInstructions

* adding compress()

* integrating compression into bench()

* copy paste error fix

* static analyzer uninit value complaint fix

* changing to control

* removing assert

* changing to control

* moving memcpy to seperate function

* fixing static analyzer complaint

* another hacky solution attempt

* Copying createbuffer logic
 2020-05-04 10:42:22 -07:00
Felix Handte
7e9aabd652
Merge pull request #2099 from felixhandte/compile-under-pedantic 
Compile Under `-pedantic -Werror` and `-std=c90`
 2020-05-04 10:07:13 -07:00
W. Felix Handte
0b81035101 Test Flag in Travis CI 2020-05-04 11:22:07 -04:00
W. Felix Handte
fa5e01c467 Add Space-Optimized Helper Variable to Lib Makefile 
This diff reorganizes the `lib/Makefile` to extract various settings that a
user would normally invoke together (supposing that they were aware of them)
if they were trying to build the smallest `libzstd` possible. It collects
these settings under a master setting `ZSTD_LIB_MIN_SIZE`.

Also document this new option.
 2020-05-04 11:19:25 -04:00
Felix Handte
816ed80774
Merge pull request #1984 from MeghnaM/1636-Reduce-stack-usage-of-HUF_sort 
Reduce stack usage of HUF_sort()
 2020-05-04 08:15:31 -07:00
W. Felix Handte
2cf72d56a6 Try to Fix MSVC Error 
It's complaining about the `memcpy`s, saying:

"warning C4090: 'function': different 'const' qualifiers"

Let's try explicitly casting to the argument types...
 2020-05-04 10:59:15 -04:00
W. Felix Handte
6528fa088e Do a -pedantic Build in CircleCI 2020-05-04 10:59:15 -04:00
W. Felix Handte
dacbcd2cc1 Fix Up Some Pointer Handling in Tests 2020-05-04 10:59:15 -04:00
W. Felix Handte
3764859060 Switch Helper Declaration to Not Force Inline 
It was causing build issues in ANSI mode.
 2020-05-04 10:59:15 -04:00
W. Felix Handte
c7da66c9cf Purge C++-Style Comments (// ...), Make Compilation Succeed Under C90 2020-05-04 10:59:15 -04:00
W. Felix Handte
952427aebf Avoid inline Keyword in C90 
Previously we would use it for all gcc-like compilations, even when a
restrictive mode that disallowed it had been selected.
 2020-05-04 10:59:15 -04:00
W. Felix Handte
baa4e2e36c Don't Evaluate Arguments to Dummy Function 2020-05-04 10:59:15 -04:00
W. Felix Handte
450542d3a7 Allow Empty Format Strings in Error Macro Invocations 
`-Wall` implies `-Wformat-zero-length`, which will cause compilation to fail
under `-Werror` when an empty string is passed as the format string to a
`printf`-family function. This commit moves us back to prefixing the provided
format string, which successfully avoids that warning.

However, this removes the failure mode where that `RAWLOG` invocation would
fail to compile when no format string was provided at all (which was desirable
to avoid having code that would successfully compile normally but fail under
`-pedantic`, which *does* require that a non-zero number of args are provided).

So this commit also introduces a function which does nothing at all, but will
fail to compile if not provided with at least one argument, which is a string.
This successfully links the compilability of pedantic and non-pedantic builds.
 2020-05-04 10:59:15 -04:00
W. Felix Handte
6696933b32 Make All Invocations Start With Literal Format String 2020-05-04 10:59:15 -04:00