AuroraMiddleware/lz4
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
2,417 Commits 13 Branches 31 Tags 6.8 MiB
ab89dda91d
Commit Graph

2417 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet
ab89dda91d improved last literals run on LZ4_compress_destSize 
applying new more accurate formula from LZ4_compress_HC_destSize()

also : fix some minor display issue in tests/frametest
 2020-09-28 11:39:00 -07:00
Yann Collet
89736e4e27 ensure last match not too close to end 
must respect MFLIMIT distance from oend
 2020-09-27 23:59:56 -07:00
Yann Collet
a13c79d56d fix incorrect counting 
after truncation of last sequence
 2020-09-27 22:47:05 -07:00
Yann Collet
e7fe105ac6 fix efficiency of LZ4_compress_HC_destSize() 
LZ4_compress_HC_destSize() had a tendency
to discard its last match when this match overflowed specified dstBuffer limit.
The impact is generally moderate,
but occasionally huge,
typically when this last match is very large
(such as compressing a bunch of zeroes).

Issue #784 fixed for both Chain and Opt implementations.

Added a unit test suggested by @remittor checking this topic.
 2020-09-27 21:04:40 -07:00
Yann Collet
20856da7c5
Merge pull request #919 from attilaolah/cli 
Add CMake option LZ4_BUILD_CLI
 2020-09-23 08:23:59 -07:00
Attila Oláh
63942cdeba
Add CMake option LZ4_BUILD_CLI. 
Defaults to ON so nothing changes, but it can be set to OFF so that the
CLI is not built at all, only the library.
 2020-09-23 12:56:01 +00:00
Yann Collet
dc27b7ae41 comment bug on older versions of ZSTD_compress_destSize() 
following investigation in #859
 2020-09-17 20:59:01 -07:00
Yann Collet
d568a6d15c
Merge pull request #918 from lz4/usan_null 
Properly fix #847
 2020-09-17 20:23:06 -07:00
Yann Collet
4f57219a22 fixed ubsan tests with dynamic library 2020-09-17 16:49:21 -07:00
Yann Collet
b223f11b21 ubsan: make pointer overflow recoverable 
the way `base` is used must allow 2-complement address overflow.
`base` is effectively a virtual pointer, never dereferenced directly.
What matters is `base + index`.
This is the address that will be dereference and must be valid (it is properly validated).
 2020-09-17 15:34:03 -07:00
Yann Collet
10d2e1c694 fixed lz4frame with blocks of size 1 
properly track history
 2020-09-17 14:43:02 -07:00
Yann Collet
43bbb7decc Merge branch 'usan_null' of github.com:Cyan4973/lz4 into usan_null 2020-09-17 10:38:59 -07:00
Yann Collet
cfd1d8f9d0 Merge branch 'usan_null' of github.com:lz4/lz4 into usan_null 2020-09-17 10:32:05 -07:00
Yann Collet
b4a64e0eac fix ubsan test 
which now fails with a clear error as intended
(not just print a status and move on).
should be reproduced on travisCI
 2020-09-17 10:29:31 -07:00
Yann Collet
ee01df1271 added the actual code change 2020-09-16 23:46:39 -07:00
Yann Collet
1328c38a15
Merge pull request #913 from lz4/usan_null 
fix #847
 2020-09-16 23:24:41 -07:00
Yann Collet
da1272979f fix #847 
support NULL input
without triggering undefined sanitizer
 2020-09-15 14:54:53 -07:00
Yann Collet
4f33a5935b added test triggering NULL arithmetic with usan 
described in #847
 2020-09-15 13:45:06 -07:00
Yann Collet
440c8461d7
Merge pull request #910 from lz4/extraInput 
Fix issue #783
 2020-08-27 11:00:28 -07:00
Yann Collet
8b75d403d8 fixed minor pre-existing printf formatting 
different signedness
 2020-08-27 09:20:01 -07:00
Yann Collet
875308f31f fixed strict c90 support 2020-08-27 00:38:58 -07:00
Yann Collet
c5d6f8a8be fix #783 
LZ4_decompress_safe_partial()
now also supports a scenario where
nb_bytes_to_generate is <= block_decompressed_size
And
nb_bytes_to_read is >= block_compressed_size.

Previously, the only supported scenario was
nb_bytes_to_read == block_compress_size.

Pay attention that,
if nb_bytes_to_read is > block_compressed_size,
then, necessarily, it requires that
nb_bytes_to_generate is <= block_decompress_size.
If both are larger, it will generate corrupted data.
 2020-08-27 00:17:57 -07:00
Yann Collet
3e3a006c6f Merge branch 'dev' into extraInput 2020-08-26 23:20:28 -07:00
Yann Collet
b73cd37bab
Merge pull request #909 from lz4/fasterCount 
Improved software byte count
 2020-08-26 08:13:52 -07:00
Yann Collet
a7ba363514 Merge branch 'dev' into fasterCount 2020-08-25 22:18:54 -07:00
Yann Collet
5243173b23 added documentation about LZ4_FORCE_SW_BITCOUNT 
Also : added memory-frugal software byte count for big endian 64-bit cpus.
Disabled by default.
 2020-08-25 22:17:29 -07:00
Yann Collet
fe0bfb0c77
Merge pull request #908 from lz4/stdin_size 
Added ability to determine content-size
 2020-08-25 21:15:31 -07:00
Yann Collet
5b3f945310 fix test under non-interactive environment 2020-08-25 14:53:08 -07:00
Yann Collet
224735ea1b added more tests around --content-size 
notably in association with `stdin`
 2020-08-25 14:42:15 -07:00
Yann Collet
65f081ccaf use variables for program invocation 2020-08-25 14:25:51 -07:00
Yann Collet
442005ff8e
Merge pull request #905 from nh2/use-fstat 
Use fstat() to determine file size
 2020-08-25 14:19:40 -07:00
Niklas Hambüchen
a58b7ac6ce fixup: Portable fileno() 2020-08-25 03:37:41 +02:00
Niklas Hambüchen
86edeb6784 fixup: C90 compliance, clean up stat() types/calls 2020-08-25 03:36:57 +02:00
Yann Collet
ee0a3cfa0c
Merge pull request #898 from aqrit/aqrit-prefixlen 
rejigger bit counting intrinsics
 2020-08-24 15:13:18 -07:00
Yann Collet
d64962a44b
Merge pull request #902 from lz4/realhw 
added target aarch64, ppc64lw and s390x to TravisCI
 2020-08-24 15:07:18 -07:00
Yann Collet
e8bd270d35
Merge pull request #907 from terrelln/travis 
[CI][ossfuzz] Fix string equal
 2020-08-24 12:20:10 -07:00
Yann Collet
e46fa0ff71
Merge pull request #903 from lz4/nofastforce 
removed LZ4_compress_fast_force()
 2020-08-24 11:28:47 -07:00
Nick Terrell
f803995b77 [CI][ossfuzz] Fix string equal 2020-08-24 11:23:04 -07:00
Niklas Hambüchen
9a7658070a Use fstat() to determine file size. 
This allows us to get the file size even when the input file is passed
via stdin. This fixes `--content-size` not working in situations like

    $ lz4 -v --content-size < /tmp/test > /tmp/test.lz4
    Warning : cannot determine input content size

With this change, it works.

Also helps with #904.
 2020-08-24 06:27:28 +02:00
Yann Collet
0002563be8 removed LZ4_compress_fast_force() 
which serves no more purpose.

The comment implies that the simple presence of this unused function was affecting performance,
and that's the reason why it was not removed earlier.
This is likely another side effect of instruction alignment.

It's obviously unreliable to rely on it in this way,
meaning that the impact will be different, positive of negative,
with any minor code change, and any minor compiler version change, even parameter change.
 2020-08-21 19:23:49 -07:00
Yann Collet
184a487d0d added target aarch64, ppc64lw and s390x to TravisCI 
more portability tests
 2020-08-21 18:12:13 -07:00
Yann Collet
34fe7c9d7f
Merge pull request #886 from servusDei2018/dev 
Optimized by replacing `endl` with `'\n'`
 2020-08-21 14:17:43 -07:00
Yann Collet
04a425d047
Merge pull request #900 from terrelln/cmake-build 
[build] Move CMake and Visual build systems to build/
 2020-08-19 20:08:01 -07:00
Yann Collet
02ee3559f1
Merge pull request #901 from terrelln/travis 
[CI][Fuzz] Fix Travis-CI fuzzer tests
 2020-08-19 20:07:50 -07:00
Nick Terrell
e97ff29627 [CI][Fuzz] Fix Travis-CI fuzzer tests 
Fixes #781
 2020-08-19 18:48:10 -07:00
Nick Terrell
31d477db07 [build] Move CMake and Visual build systems to build/ 
Fixes #852.
 2020-08-19 18:08:43 -07:00
Yann Collet
f18dbeb8b5 Merge branch 'dev' into extraInput 2020-08-18 15:34:51 -07:00
Yann Collet
fa5435bca9
Merge pull request #897 from lz4/lz4wlib 
added target lz4-wlib
 2020-08-18 10:24:36 -07:00
aqrit
e45defa8bd
silence warning 
MSVC debug mode complains
 2020-08-17 17:53:07 -04:00
Yann Collet
9a6e93859d
Merge pull request #899 from lz4/endMark 
Clarifies and fix EndMark
 2020-08-14 15:48:21 -07:00