Commit Graph

2054 Commits

Author SHA1 Message Date
Yann Collet
352bbdb98e updated man page
with --favor-decSpeed
2019-04-12 13:15:05 -07:00
Yann Collet
21ff1a839a updated doc to underline difference between block and frame
as this is a very frequent source of confusion for new users.
2019-04-12 11:27:44 -07:00
Yann Collet
780aac520b
Merge pull request #665 from lz4/dict
Dictionary compression
2019-04-11 15:43:40 -07:00
Yann Collet
723ba904e2
Merge pull request #664 from lz4/maxdist
introduce LZ4_DISTANCE_MAX build macro
2019-04-11 15:42:54 -07:00
Yann Collet
bcb26565a7 improved documentation for LZ4 dictionary compression 2019-04-11 15:20:46 -07:00
Yann Collet
8d76c8a44a introduce LZ4_DISTANCE_MAX build macro
make it possible to generate LZ4-compressed block
with a controlled maximum offset (necessarily <= 65535).

This could be useful for compatibility with decoders
using a very limited memory budget (<64 KB).

Answer #154
2019-04-11 14:15:33 -07:00
Yann Collet
013fee5665 updated block format with more details on MF_LIMIT
aka distance from end of block for the last match.
2019-04-11 13:55:35 -07:00
Yann Collet
5580f4333a
Merge pull request #663 from lz4/headerSize
made LZ4F_getHeaderSize() public
2019-04-09 21:30:14 -07:00
Yann Collet
db60ed19b1 added versions in comments 2019-04-09 18:23:32 -07:00
Yann Collet
9e501702bd made LZ4F_getHeaderSize() public 2019-04-09 18:10:58 -07:00
Yann Collet
9fdc1a0118
Merge pull request #659 from lz4/resetFast
LZ4_resetStream*_fast()
2019-04-09 16:26:49 -07:00
Yann Collet
d8d5f14138 fixed loadDictHC
by making a full initialization
instead of a fast reset.
2019-04-09 15:37:59 -07:00
Yann Collet
887e8a4d01 re-enable LZ4_resetStreamHC()
towards deprecation, but still available and fully supported
2019-04-09 14:00:30 -07:00
Yann Collet
14c71dfa9c modified LZ4_initStreamHC() to look like LZ4_initStream()
it is now a pure initializer, for statically allocated states.
It can initialize any memory area, and because of this, requires size.
2019-04-09 13:55:42 -07:00
Yann Collet
5ef4f3ce91 check some more initialization result
ensure it's not NULL.
2019-04-08 16:51:22 -07:00
Yann Collet
111df0fa45 removed LZ4_stream_t alignment test on Visual
it fails on x86 32-bit mode :
Visual reports an alignment of 8-bytes (even with alignof())
but actually only align LZ4_stream_t on 4 bytes.
The alignment check then fails, resulting in missed initialization.
2019-04-08 16:47:21 -07:00
Yann Collet
da19cc79da checkFrame: removed %zu
not liked by mingw
2019-04-08 14:21:09 -07:00
Yann Collet
b92681f3c0 fullbench: assert lz4_stream_t initialization 2019-04-08 14:10:59 -07:00
Yann Collet
3f5f10c63e fixed cleaning tmp directory 2019-04-08 14:01:32 -07:00
Yann Collet
249703ae2e slightly faster huge file test
level down 5->4
size down 6G->5G
2019-04-08 13:49:26 -07:00
Yann Collet
5db9a2f8b6 added make list target to tests 2019-04-08 13:45:01 -07:00
Yann Collet
c198a39a66 LZ4_initStream() checks alignment restriction
updated associated documentation
2019-04-08 12:49:54 -07:00
Yann Collet
34f0004a5e added comment on initStream + _extState_
as suggested by @felixhandte
2019-04-05 15:35:19 -07:00
Yann Collet
2ece0d8380 created LZ4_initStream()
- promoted LZ4_resetStream_fast() to stable
- moved LZ4_resetStream() into deprecate, but without triggering a compiler warning
- update all sources to no longer rely on LZ4_resetStream()

note : LZ4_initStream() proposal is slightly different :
it's able to initialize any buffer, provided that it's large enough.
To this end, it accepts a void*, and returns an LZ4_stream_t*.
2019-04-05 12:56:26 -07:00
Yann Collet
2a94faf462 fixed _GNU_SOURCE duplicate 2019-04-05 11:55:34 -07:00
Yann Collet
a4f1635b0a fuzzer: fixed strict c99 with mmap+MAP_ANONYMOUS 2019-04-05 11:47:06 -07:00
Yann Collet
c3f8928f87 fixed strict iso C90 2019-04-05 10:41:26 -07:00
Yann Collet
c491df54ec created LZ4_initStreamHC()
- promoted LZ4_resetStreamHC_fast() to stable
- moved LZ4_resetStreamHC() to deprecated (but do not generate a warning yet)
- Updated doc, to highlight difference between init and reset
- switched all invocations of LZ4_resetStreamHC() onto LZ4_initStreamHC()
- misc: ensure `make all` also builds /tests
2019-04-04 17:05:11 -07:00
Yann Collet
f66abc45d9
Merge pull request #658 from lz4/_fast
Deprecated LZ4_decompres_fast*() functions
2019-04-04 13:42:58 -07:00
Yann Collet
7a39fb8fb6 make _fast*() decoder generate a deprecation warning
updated modification
2019-04-04 12:47:36 -07:00
Yann Collet
ab91300509 moved LZ4_decompress_fast*() into deprecated section 2019-04-04 12:24:46 -07:00
Yann Collet
1a41b2b25c
Merge pull request #657 from lz4/destSize
moved _destSize() into "stable API" status
2019-04-03 17:15:06 -07:00
Yann Collet
b5bedefabe fixed an old bug in LZ4F_flush()
which remained undetected so far,
as it requires a fairly large number of conditions to be triggered,
starting with enabling Block checksum, which is disabled by default,
and which usage is known to be extremely rare.
2019-04-03 16:28:42 -07:00
Yann Collet
50f01d881d fixed doc
and bumped version number fo v1.9.0
2019-04-03 14:27:21 -07:00
Yann Collet
25da6cc967 moved _destSize() into "stable API" status
as requested in #642
2019-04-03 14:18:00 -07:00
Yann Collet
27a4c3c308
Merge pull request #656 from lz4/armtest
LZ4_FAST_DEC_LOOP macros
2019-04-03 10:03:26 -07:00
Yann Collet
f2755c9887 minor comments and reformatting 2019-04-03 08:59:29 -07:00
Yann Collet
753076bfa4 fixed minor conversion warnings 2019-04-02 17:16:43 -07:00
Yann Collet
2589c4424f created LZ4_FAST_DEC_LOOP build macro 2019-04-02 16:22:11 -07:00
Yann Collet
7d9d00f4df fixed a few minor conversion warnings 2019-04-02 16:06:37 -07:00
Yann Collet
c438548312
Merge pull request #652 from vtorri/dev
Allow installation of lz4 for Windows 10 with MSYS2
2019-03-03 13:40:45 -08:00
Vincent Torri
0360981f68 Allow installation of lz4 for Windows 10 with MSYS2 2019-03-03 22:06:38 +01:00
Yann Collet
d85bdb4ff2
Merge pull request #645 from djwatson/optimize_decompress_generic
Optimize decompress generic
2019-02-11 16:58:53 -08:00
Dave Watson
5d7d1166cb decompress_generic: Limit fastpath to x86
New fastpath currently shows a regression on qualcomm
arm chips.  Restrict it to x86 for now
2019-02-11 11:44:51 -08:00
Dave Watson
75fb878a90 decompress_generic: Add fastpath for small offsets
For small offsets of size 1, 2, 4 and 8, we can set a single uint64_t,
and then use it to do a memset() variation.  In particular, this makes
the somewhat-common RLE (offset 1) about 2-4x faster than the previous
implementation - we avoid not only the load blocked by store, but also
avoid the loads entirely.
2019-02-08 13:57:23 -08:00
Dave Watson
faac110e20 decompress_generic: Unroll loops a bit more
Generally we want our wildcopy loops to look like the
memcpy loops from our libc, but without the final byte copy checks.
We can unroll a bit to make long copies even faster.

The only catch is that this affects the value of FASTLOOP_SAFE_DISTANCE.
2019-02-08 13:57:23 -08:00
Dave Watson
1fbaf84306 decompress_generic: remove msan write
This store is also causing load-blocked-by-store issues, remove it.
The msan warning will have to be fixed another way if it is still an issue.
2019-02-08 13:57:23 -08:00
Dave Watson
28b824921d decompress_generic: re-add fastpath
This is the remaineder of the original 'shortcut'.  If true, we can avoid
the loop in LZ4_wildCopy, and directly copy instead.
2019-02-08 13:57:23 -08:00
Dave Watson
232f1e261f decompress_generic: drop partial copy check in fast loop
We've already checked that we are more than FASTLOOP_SAFE_DISTANCE
away from the end, so this branch can never be true, we will have
already jumped to the second decode loop.
2019-02-08 13:57:23 -08:00
Dave Watson
59332a3026 decompress_generic: Optimize literal copies
Use LZ4_wildCopy16 for variable-length literals.  For literal counts that
fit in the flag byte, copy directly.  We can also omit oend checks for
roughly the same reason as the previous shortcut:  We check once that both
match length and literal length fit in FASTLOOP_SAFE_DISTANCE, including
wildcopy distance.
2019-02-08 13:57:23 -08:00