AuroraMiddleware/lz4
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
497 Commits 13 Branches 31 Tags 6.8 MiB
8b8e5efefe
Commit Graph

204 Commits

Author SHA1 Message Date
Yann Collet
046bd3abee Merge pull request #52 from KyleJHarper/r128/multiple_inputs 
Adding support for multiple file processing.  Fixes Google code issue 151.
 2015-03-03 01:04:40 +01:00
Takayuki MATSUOKA
dcdd628d5e Fix sentinel bit pattern 
Set all bits of U64.
 2015-03-02 13:38:30 +09:00
Takayuki MATSUOKA
de5c930c90 Fix sentinel size miscalculation 
Since is_nul() style wild-comparing stride is U64, we must add sizeof(U64) to the sentinel.
 2015-03-02 13:38:29 +09:00
Takayuki MATSUOKA
eed7952101 Add GNU coreutil's is_nul() method to isSparse() 
See original source
http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/system.h
 2015-03-02 13:38:28 +09:00
Takayuki MATSUOKA
b372f45093 Add Neil's method to isSparse() 2015-03-02 13:38:27 +09:00
Takayuki MATSUOKA
01a24afdcf Improve isSparse() 2015-03-02 13:38:25 +09:00
Takayuki MATSUOKA
4a5d92b1d9 Adjust coding style 2015-03-02 13:38:24 +09:00
Takayuki MATSUOKA
97679fa5a1 Google Code Issue 155: lz4 cli should support sparse file 
https://code.google.com/p/lz4/issues/detail?id=155

This is experimental implementation. Just a proof of concept.
It works Linux and Windows.

# Build

To build experimental version, define 'LZ4IO_ENABLE_SPARSE_FILE' like the following command :

    make lz4programs 'CFLAGS=-O3 -DLZ4IO_ENABLE_SPARSE_FILE=1'
    ./programs/lz4 -h

You will see "EXPERIMENTAL_SPARSE_FILE" as lz4 revision :

    "*** LZ4 command line interface 64-bits EXPERIMENTAL_SPARSE_FILE, by Yann Collet (...) ***"

# Experiment

This experimental version adds option "-x" for sparse file for decompression.
You can use this option like this :

    ./programs/lz4 -9 -f my-file
    ./programs/lz4 -d -f -x my-file.lz4 my-file.lz4.out
    cmp my-file my-file.lz4.out
 2015-03-02 13:38:23 +09:00
kyle
fa27d236b4 Added support for multiple input files to act more like other compressors. For example: gzip file1 file2 file3. You can now do: lz4 [args] -m file1 file2 file3. Fixes 151. 2015-03-01 19:00:40 -06:00
Yann Collet
67f3b4141a Merge pull request #49 from t-mat/msvc-fseeki64 
Avoid MSVC fseek()'s 2GiB barrier
 2015-03-02 00:41:38 +01:00
Yann Collet
e68d1c98c1 restored lz4 hc compression ratio 2015-03-02 00:11:34 +01:00
Takayuki MATSUOKA
41b6ed3c5b Replace fseek with _fseeki64 to avoid MSVC's 2GiB barrier 
Since MSVC fseek()'s SEEK_CUR mode has 2GiB barrier, our fseek() calling in program/lz4io.c/selectDecoder() will fail for large (>2GiB) .lz4 file.
This commit just replace fseek with _fseeki64 by macro. Second argument is automatically casted to __int64 (signed 64bit integer). Other things are completely same.
 2015-02-14 12:48:11 +09:00
Yann Collet
f68eead36c Added : -pedantic compilation option 2014-12-17 12:32:49 +01:00
Yann Collet
0569a68edd C90 compatibility 2014-12-17 12:20:17 +01:00
Yann Collet
8a9fb8cf32 Fixed : older compiler don't like nameless unions, reported by Cheyi Lin 2014-12-16 22:03:16 +01:00
Yann Collet
95cc6cef64 Fixed : bug within LZ4 HC streaming mode, reported by James Boyle 2014-12-16 02:13:19 +01:00
Yann Collet
daa320f3f7 New : lz4frame integrated into liblz4 (v1.5.0) 2014-12-13 15:05:46 +01:00
Yann Collet
7dd50ed2f6 minor memtest correction 2014-12-10 23:00:50 +01:00
Yann Collet
e68d7dcf22 Fixed : LZ4_compress_limitedOutput() bug, as reported by Christopher Speller 2014-12-10 17:58:15 +01:00
Yann Collet
3477cbac73 stronger tests for LZ4_compress_limitedOutput() 2014-12-10 09:05:44 +01:00
Yann Collet
c5decf7562 minor refactoring 2014-12-06 17:10:54 +01:00
Yann Collet
9f6826fdc9 lz4cli : legacy arguments are now disabled by default 2014-12-01 01:25:18 +01:00
Yann Collet
d008c87151 New directory structure : library source files into /lib directory 2014-11-30 23:32:12 +01:00
Yann Collet
6de52c2a8c LZ4IO : modified to use lz4frame instead 2014-11-30 17:59:31 +01:00
Yann Collet
d0a6cccf75 Fixed : some minor Visual warnings 2014-11-30 12:58:00 +01:00
Yann Collet
326cada9fa Updated lz4hc : re-use most shared elements from lz4 (endianess / align / bus detection routines) 2014-11-29 20:19:39 +01:00
Yann Collet
dc43a1f6b5 Fixed : decompression issue on 32-bits CPU without unaligned memory access 2014-11-29 16:41:28 +01:00
Yann Collet
2d98faf238 Improved decoding speed 2014-11-27 22:44:36 +01:00
Yann Collet
86fa21b301 Added : $(EXT) within install for cross-compilation support 2014-11-23 18:36:04 +01:00
Yann Collet
2ad37dbe58 Corrected lz4io 2014-11-23 01:14:04 +01:00
Yann Collet
73554386ed Makefile : refactored Travis CI tests 2014-11-04 20:56:38 +01:00
Yann Collet
89d8b98d0d Fixed issue 44 : stopped concatenated stream on null size frame (thanks to Brendan O'Connor) 2014-11-04 12:11:14 +01:00
Yann Collet
d01bb4a193 Makefile : added -Wcast-align compilation warning 2014-11-04 10:32:50 +01:00
Yann Collet
7050d4b767 Updated unsynchronized buffers tests 2014-11-04 10:04:37 +01:00
Yann Collet
b005e1b42e fullbench : display Algorithm Nb 2014-11-03 09:18:15 +01:00
Yann Collet
61289dea1d Optimized LZ4_saveDictHC() 2014-11-02 22:32:12 +01:00
Yann Collet
55e8a47474 Added : valgrind test for fuzzer 2014-10-29 01:47:43 +01:00
Yann Collet
5a5a15969e minor fuzzer test correction 2014-10-29 01:39:47 +01:00
Yann Collet
d4f2d8061e Fixed : frametest for Travis CI build matrix 2014-10-28 15:58:25 +01:00
Yann Collet
9bf331afc6 new Travis CI build matrix 2014-10-28 15:35:43 +01:00
Yann Collet
6436eca4fa Added : frametest32 2014-10-28 11:32:42 +01:00
Yann Collet
7a677a77a6 LZ4 HC streaming integrated within lz4frame 2014-10-28 00:39:06 +01:00
Yann Collet
508855c488 HC streaming : support for very long streaming (> 4 GB) scenarios 2014-10-26 11:22:15 +01:00
Yann Collet
2b421e97d4 HC streaming : support small ringbuffer scenarios 2014-10-25 20:52:10 +01:00
Yann Collet
e468a4eb96 fixed LZ4F_compressFrameBound (dynamic block resize) 2014-10-22 00:36:32 +01:00
Yann Collet
e450018588 LZ4 HC : fixed small dictionary streaming compression 2014-10-21 00:12:55 +01:00
Yann Collet
3dab5f476a LZ4 HC : External Dictionary compression : First implementation 2014-10-20 01:08:21 +01:00
Yann Collet
d618673057 fuzzer : easier random replay 2014-10-19 16:06:33 +01:00
Yann Collet
f91d5aae79 fuzzer : fixed test 21 2014-10-19 15:25:35 +01:00
Yann Collet
6b89dca1a9 fuzzer : faster jump to testNb 2014-10-19 13:47:22 +01:00
Yann Collet
68ea218a72 fuzzer : fixed minor command lines issues 2014-10-18 13:27:32 +01:00
Yann Collet
d239a23337 updated LZ4HC API 2014-10-18 11:18:14 +01:00
Yann Collet
4da47a2c67 fuzzer : pause at the end is disabled by default 2014-10-10 20:58:42 +01:00
Yann Collet
92574da1a0 modified : fuzzer automatically selects seed 2014-10-09 22:15:57 +01:00
Yann Collet
2ab6f9a387 Updated : xxHash to r37 2014-10-06 11:13:56 +01:00
Yann Collet
c14a342cc2 Fixed : LZ4F_compressBound() using null preferencesPtr 
Updated : frametest, to check LZ4F_compressBound() using null preferencesPtr
 2014-10-06 00:48:45 +01:00
Yann Collet
8c1ae99837 fixed : clang warnings 2014-09-22 18:42:00 +01:00
Yann Collet
abb6f7806a Added : suooirt for s390x, thanks to Nobuhiro Iwamatsu 
Multiple warning fixes for Visual Studio 2012
 2014-09-22 17:38:17 +01:00
Yann Collet
6e1049872a LZ4F_compressFrame : fix potential crash on selecting custom preferences 
frame fuzzer tests : new random tests using LZ4F_compressFrame
 2014-09-22 02:59:42 +01:00
Yann Collet
eeb9011467 lz4frame : support stableDst option 
Multiple bugfixes within lz4frame decompression
Small decompression speed improvements
Improved fuzzer test, with more thorough and complex tests
 2014-09-21 09:56:21 +01:00
Yann Collet
d71b9e25b7 small improvements to lz4frame compression 2014-09-15 00:59:30 +01:00
Yann Collet
562b34f660 changed : lz4 test mode (-t) to no longer ask for confirmation, as suggested by Nguyen Thary 2014-09-13 23:49:45 +01:00
Yann Collet
7716c03dbf changed xxhash directory 
changed makefile
 2014-09-13 23:44:07 +01:00
Yann Collet
3f80a22819 Merge pull request #26 from Cyan4973/frame 
Frame
 2014-09-14 00:30:01 +02:00
Yann Collet
457dc35e6a Restored variable output size fuzzer test 
Quickfix frame decompression
Small speed optimization frame decompression
 2014-09-13 21:21:41 +01:00
Yann Collet
56c2b79ed0 Frame decompression speed optimization 2014-09-13 19:49:01 +01:00
Yann Collet
38912f55e3 fullbench : added benchmark for LZ4F_decompress() 2014-09-13 15:24:16 +01:00
Yann Collet
658ab6cfca LZ4F_decompressFrame : implemented srcSizeHint as function result 2014-09-13 12:15:54 +01:00
Yann Collet
bd704cf70a lz4frame : implemented option stableSrc 
Improved LZ4_compressFrame() speed
 2014-09-13 10:08:55 +01:00
Yann Collet
a586208597 lz4frame : autoflush mode : saves memory, and improves compression speed 
LZ4F_getFrameInfo() fix, thanks to Christopher Jackson
 2014-09-12 19:28:44 +01:00
Yann Collet
b1d022fa72 slightly improved frame compression speed 2014-09-11 22:27:14 +01:00
Yann Collet
c71de79688 Improved lz4frame compression speed 
Added : fullbench test -c14 (LZ4F_compressFrame)
 2014-09-10 22:17:03 +01:00
Yann Collet
ed4a6bf2cb More tests : random flushes 2014-09-10 13:53:42 +01:00
Yann Collet
0400451ac2 Fix : streaming mode bug (re-using context & buffers) 2014-09-10 13:00:39 +01:00
Yann Collet
eac83cd850 Added : streaming mode 2014-09-09 23:54:22 +01:00
Yann Collet
6b407c65a4 Added : valgrind verification for frametest 2014-09-07 14:15:33 +01:00
Yann Collet
d1d1f8835d API change : compressionContext more easily reusable 2014-09-07 12:57:09 +01:00
Yann Collet
cf28c27809 Added : fuzzer : ability to jump to specified testNb 2014-09-07 12:12:46 +01:00
Yann Collet
cdececa38d Added : variable dstBuffer size decompression test 2014-09-07 11:04:29 +01:00
Yann Collet
535120bbe3 More complex compression tests 2014-09-07 09:13:02 +01:00
Yann Collet
04f5b17b72 Improved frame decompression test 
Fixed : frame decompression bug
 2014-09-06 22:48:03 +01:00
Yann Collet
03bc760af3 minor frametest display improvement 2014-09-06 13:06:28 +01:00
Yann Collet
e6d9a17188 minor test code refactoring 2014-09-06 12:19:26 +01:00
Yann Collet
5c32b50b94 More tests (variation of blocksize & checksum) 
fixed : checksum error on dealing with uncompressed blocks
 2014-09-06 09:47:28 +01:00
Yann Collet
fd8665320f Added some more tests 2014-09-05 16:32:04 +01:00
Yann Collet
b03f8f0e6f fixed : LZ4F_decompress() 
more decompress tests
 2014-09-04 22:56:51 +01:00
Yann Collet
e619cfe852 Completed first version of lz4frame decompress 
Added a first decompression test
 2014-09-03 19:49:59 +01:00
Yann Collet
f7f67e778c Added : preliminary frame decompression function 2014-09-01 22:44:02 +01:00
Yann Collet
2f33c77d53 upgraded lz4frame compression tests 2014-08-31 23:14:20 +01:00
Yann Collet
f66721d303 lz4frame compression API v1 completed 
Added : test program frametest
 2014-08-30 18:14:44 +01:00
Yann Collet
3b4e3f2b01 minor fuzzer code cleanup 2014-08-30 18:13:45 +01:00
Yann Collet
14a28c0633 minor fuzzer cleaning 2014-08-30 12:32:09 +01:00
Yann Collet
557ff22334 Fixed crlf issue 2014-08-26 15:53:39 +02:00
Yann Collet
da07465b64 Simplified travis test flow 2014-08-26 14:39:09 +01:00
Yann Collet
ff94eec374 Added macro, for large file support on 32-bits AIX 2014-08-20 16:37:34 +01:00
Yann Collet
a74f7ee811 Update xxHash to r36 2014-08-15 10:24:24 +01:00
Yann Collet
4dc1eca5a9 updated release number 2014-08-13 16:44:44 +01:00
Yann Collet
205735a9f2 Fixed crlf issue 2014-08-06 13:50:38 +02:00
Yann Collet
4ea832b8ff Updated Makefile : make dist 
Minor formatting changes for xxhash
 2014-08-06 08:55:08 +01:00
Yann Collet
a0755949b6 fix : minor compilation warning under mingw 2014-08-05 06:42:40 +01:00