Commit Graph

5654 Commits

Author SHA1 Message Date
Nick Terrell
f6daddf2db Also allow x86 2018-09-12 12:05:32 -07:00
Nick Terrell
1e0bac6a9c [libzstd] Fix cpu for MSFT ARM
The `__cpuid()` and `__cpuidex()` intrinsics are only available
on x86 and x86_64.
2018-09-12 10:35:16 -07:00
Yann Collet
cf5db388f3
Merge pull request #1311 from cmcginty/dev
Print a stack trace on unexpected term signal (e.g. SIGABRT)
2018-09-12 10:13:23 -07:00
Casey McGinty
b703181f93
Disable -rdynamic LD option on Windows 2018-09-11 16:19:34 -07:00
Casey McGinty
b9118ecdab
Update comments, and LD flag usage in Make 2018-09-11 14:49:47 -07:00
Casey McGinty
2224ecd718
Remove dead code and method name typo 2018-09-11 11:56:50 -07:00
Casey McGinty
d4337b6f1d
Move ABRThandler func out of internal lib 2018-09-11 11:39:49 -07:00
Casey McGinty
2a3967b7c4
Refactor abort signal handling
- Print signal name to term
- Add -rdynamic option to generate Linux symbol names in backtrace
- Raise default signal after handler to ensure program termination
2018-09-11 10:40:45 -07:00
Yann Collet
51a246da82
Merge pull request #1310 from facebook/Dworkspace
reduce DDict size by 2KB
2018-09-10 13:11:11 -07:00
Yann Collet
3675ef4762 added comment about minimum size of FSE tables
required for DDict creation,
which use this space as workspace during Hufman table building stage.
2018-09-10 11:24:17 -07:00
Yann Collet
7ff74c89fa
Merge pull request #1307 from facebook/spec
spec clarification
2018-09-07 15:42:04 -07:00
Casey McGinty
a06574fc97
Print a stack trace on unexpected term signal (e.g. SIGABRT)
For OSX and Linux, add a signal handler to SIGABRT, SGIFPE, SIGILL,
SIGSEGV, and SIGBUS. When the program terminates unexpectedly the
handler will print the current stack to the terminal to help determine
the location of the failure.

On OSX the output will look like:

```
Stack trace:
4   zstd                                0x000000010927ed96 main + 16886
5   libdyld.dylib                       0x00007fff767d1015 start + 1
6   ???                                 0x0000000000000001 0x0 + 1
```

On Linux the output will look like:

```
Stack trace:
./zstd() [0x4b8e1b]
./zstd() [0x4b928a]
./zstd() [0x403dc2]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f5e0fbb0445]
./zstd() [0x405754]
```

As is, the code does not function on WIN32.

See also: https://oroboro.com/stack-trace-on-crash/
2018-09-06 18:46:52 -07:00
Yann Collet
112c18d6d5
Merge pull request #1309 from facebook/islink
fixed UTIL_isLink() for Mac OS-X
2018-09-06 18:01:10 -07:00
Yann Collet
f97ca36eab strengthened conditions for using workplace into fse table space
ensure that the structure layout is as expected.
will trigger an error if it changes in the future.

Another solution would be to use a union,
this would be cleaner and get rid of these static asserts.

However, in order to keep the current code unmodified,
it would be necessary to use an un-named unions.
And apparently, un-named unions are only possible on "recent" compilers (C99+).
2018-09-06 17:54:13 -07:00
Yann Collet
b036b9a177 Merge branch 'islink' into Dworkspace 2018-09-06 17:33:27 -07:00
Yann Collet
deb03d8a26 fixed UTIL_isLink() for Mac OS-X 2018-09-06 17:32:16 -07:00
Yann Collet
87406548f0 reduced DDict size, by -2KB
corresponding to the removal of workspace
which is needed while building huffman table
and is now either present in DCtx,
or temporarily borrowed from available FSE table space.
2018-09-06 17:07:53 -07:00
Yann Collet
79106d8de9
Merge pull request #1308 from facebook/codeofconduct
added code of conduct
2018-09-06 14:38:32 -07:00
Yann Collet
161ecddf3c added code of conduct
as requested by Facebook OSS program.
2018-09-06 11:20:39 -07:00
Yann Collet
50b216146f
Merge pull request #1304 from facebook/largeNbDicts
contrib/largeNbDicts
2018-09-06 09:50:56 -07:00
Yann Collet
c57a856d64 fixed minor static analyzer warning 2018-09-05 14:33:51 -07:00
Yann Collet
55a8f84a2c spec clarification
following #1305 comments from @ulikunitz
2018-09-05 12:31:33 -07:00
Nick Terrell
d492ef4e07
Merge pull request #1306 from jennifermliu/dev
Change default f of FASTCOVER to 20
2018-09-04 18:03:36 -07:00
Jennifer Liu
21721b75a3 Change default f to 20 2018-09-04 17:15:14 -07:00
Nick Terrell
7a02df8dbe
Merge pull request #1302 from jennifermliu/splitpoint
Always use splitPoint=1.0 for non-optimize cover and fastcover and some minor changes on comments
2018-09-04 16:59:41 -07:00
Yann Collet
1d487d587f updated documentation 2018-09-04 14:57:45 -07:00
Yann Collet
11b8b8c100 silenced false-positive scan-build warning 2018-08-31 10:01:06 -07:00
Yann Collet
cc1f14a16b
Merge pull request #1303 from cmcginty/dev
Zstandard - better `make list`
2018-08-30 18:09:33 -07:00
Casey McGinty
986ead646b
Add optional description field support to "list" target 2018-08-30 17:00:17 -07:00
Yann Collet
0ff67511e6 fixed link order for old compilers 2018-08-30 16:43:28 -07:00
Yann Collet
f76253bb70 minor : createDictionaryBuffer() can create dictionaries of different sizes 2018-08-30 16:24:44 -07:00
Yann Collet
39c55a118f fixed minor compatibility issues with older compilers 2018-08-30 16:00:57 -07:00
Jennifer Liu
f87383507d Update comment about default dictionary builder 2018-08-30 15:46:39 -07:00
Jennifer Liu
944c9986e0 Update comment on default steps of cover and fastcover 2018-08-30 15:37:29 -07:00
Yann Collet
39ef91a599 -std=c99 for largeNbDicts 2018-08-30 14:59:23 -07:00
Jennifer Liu
16db0337b1 Always use splitPoint=1.0 for non-optimize cover and fastcover 2018-08-30 14:59:22 -07:00
Jennifer Liu
e45d82ab0b Merge branch 'dev' of github.com:facebook/zstd into dev 2018-08-30 14:57:16 -07:00
Yann Collet
4086b2871b largeNbDicts compatible with multiple source files
splitting is disabled by default, but can be re-enabled using usual command -B#
update commands to look like zstd ones
2018-08-30 14:38:49 -07:00
Yann Collet
a5a77965d3 make all includes contrib/largeNbDicts 2018-08-29 16:17:22 -07:00
Yann Collet
d89fa814c1 added a README
for documentation
2018-08-28 18:19:19 -07:00
Yann Collet
6444c50035 increases randomness of ddict ptrs 2018-08-28 18:13:46 -07:00
Yann Collet
6c398df241 level, block size and nb dicts can be set on command line 2018-08-28 18:05:31 -07:00
Yann Collet
31ebb26945
Merge pull request #1301 from terrelln/lit-size
[zstd] Fix seqStore growth
2018-08-28 17:10:25 -07:00
Yann Collet
0c66a44d1b first working test program
measures :
- compression ratio with / without dictionary
- create one dictionary per block
- memory budget for dictionaries
- decompression speed, using one different dictionary per block

current limitations :
- only one file
- 4K blocks only
- automatic dictionary built with 4K size

dictionary can be selected on command line, with -D
2018-08-28 15:47:07 -07:00
Nick Terrell
e3b5286197 Fix decodecorpus 2018-08-28 13:56:47 -07:00
Nick Terrell
e984d01912 Small test fixes 2018-08-28 13:42:01 -07:00
Nick Terrell
5e580de6da [zstd] Fix seqStore growth
We could undersize the literals buffer by up to 11 bytes,
due to a combination of 2 bugs:
* The literals buffer didn't have `WILDCOPY_OVERLENGTH` extra
  space, like it is supposed to.
* We didn't check the literals buffer size in `ZSTD_sufficientBuff()`.
2018-08-28 13:24:44 -07:00
Nick Terrell
5a4e6c9f3d [fuzzer] Test growing the seqStore_t 2018-08-28 13:20:37 -07:00
Yann Collet
b37a0a6bde
Merge pull request #1298 from facebook/bench
Refactored bench.c
2018-08-28 12:25:02 -07:00
Yann Collet
0491037db9 Merge branch 'bench' into largeNbDicts 2018-08-28 11:26:46 -07:00