AuroraMiddleware/zstd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
2,682 Commits 1 Branch 0 Tags 27 MiB
abd6302423
Commit Graph

841 Commits

Author SHA1 Message Date
Przemyslaw Skibinski
abd6302423 Windows resources updated to v1.1.4 2017-02-14 09:39:09 +01:00
Przemyslaw Skibinski
442c75f132 removed UTIL_doesFileExists (replaced with UTIL_isRegFile) 2017-02-14 09:38:51 +01:00
Przemyslaw Skibinski
b876b96ce1 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2017-02-14 09:26:03 +01:00
Yann Collet
db2666c10c Merge pull request #536 from iburinoc/multiframe 
Simple API multiframe decoding
 2017-02-13 12:18:16 -08:00
Przemyslaw Skibinski
35bf23c086 MinGW-w64 requires _FILE_OFFSET_BITS 64 2017-02-13 13:57:29 +01:00
ds77
6220bfc924 fix indentation in previous commit 2017-02-13 12:00:59 +01:00
ds77
168d9b8006 fix seeking 2GB+ files under Windows 
Replace fseek() in FIO_fwriteSparse() and FIO_fwriteSparseEnd() with macro expanding to 64-bit fseek version provided by the platform (includes  fallback workaround using Win32 API).
 2017-02-12 16:05:19 +01:00
Yann Collet
acfa151622 Merge pull request #540 from ds77/dev-stat64-fix 
zstdcli: Fix reporting incorrect sizes of large flies on MinGW
 2017-02-11 21:08:08 -08:00
Przemyslaw Skibinski
eb132530cd revert last commit 2017-02-10 21:15:49 +01:00
Przemyslaw Skibinski
645f5b9856 fix for original MinGW 2017-02-10 20:09:28 +01:00
Sean Purcell
5069b6c2c3 Merge branch 'dev' into multiframe 2017-02-10 10:08:55 -08:00
ds77
45f0c207ab use _stati64() in UTIL_getFileSize() when compiling with mingw, get rid of introduces previously preprocessor checks. 2017-02-10 18:37:57 +01:00
Przemyslaw Skibinski
456330948a Merge remote-tracking branch 'refs/remotes/facebook/dev' into mingw 
# Conflicts:
#	programs/platform.h
 2017-02-10 13:38:54 +01:00
Yann Collet
bbba42acd1 Merge pull request #537 from terrelln/small-bugs 
Fix small bugs
 2017-02-10 04:35:43 -08:00
-
7ec315df0d fix previous commit 
* struct _stat64 is not defined by (non-w64) MinGW releases, __stat64 should be everywhere
* proper detection of _stat64() availability (as in MinGW sys/stat.h)
 2017-02-10 13:27:43 +01:00
-
19f61b534e use _stat64 only when targetting Win2k or later 2017-02-10 10:56:45 +01:00
Przemyslaw Skibinski
429e13099a fix 64-bit file support for MinGW 2017-02-10 10:36:44 +01:00
Nick Terrell
d08019813b Improvement from @inikep 2017-02-09 14:20:52 -08:00
ds77
9cde3f8b2e use _stat64() on MinGW 
On MinGW, use _stat64() and struct _stat64 instead of stat() and struct stat_t. This fixes reporting incorrect sizes for large files.
 2017-02-09 22:49:31 +01:00
Przemyslaw Skibinski
896638a8a2 echo replaced with printf 2017-02-09 17:01:17 +01:00
Przemyslaw Skibinski
13127fd05b don't use "echo -e" 2017-02-09 11:32:21 +01:00
Nick Terrell
a0f9006e5a #undef _POSIX_C_SOURCE if already defined 2017-02-08 17:25:01 -08:00
Sean Purcell
0f5c95af44 Disambiguate pledgedSrcSize == 0 
- Modify ZSTD CLI to only set contentSizeFlag if it _knows_ the size
- Change pzstd to stop setting contentSizeFlag without accurate pledgedSrcSize
 2017-02-08 15:12:46 -08:00
Sean Purcell
4e709712e1 Decompressed size functions now handle multiframes and distinguish cases 
- Add ZSTD_findDecompressedSize
    - Traverses multiple frames to find total output size
- Add ZSTD_getFrameContentSize
    - Gets the decompressed size of a single frame by reading header
- Deprecate ZSTD_getDecompressedSize
 2017-02-08 14:50:10 -08:00
Przemyslaw Skibinski
93901fe85c remove redundant "ifeq ($(HAVE_ZLIB), 1)" 2017-02-08 21:11:18 +01:00
Przemyslaw Skibinski
ca20edd960 fixed zlib detection with MinGW 2017-02-08 14:32:49 +01:00
Przemyslaw Skibinski
cdf5a7bd9f Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2017-02-08 13:49:35 +01:00
Przemyslaw Skibinski
40580ff669 added description of "--priority=rt" 2017-02-08 13:49:06 +01:00
Przemyslaw Skibinski
0665a359aa "--rt-prio" renamed to "--priority=rt" 2017-02-07 20:12:59 +01:00
Przemyslaw Skibinski
d05014c739 added the "--rt-prio" option 2017-02-07 16:48:01 +01:00
Przemyslaw Skibinski
94abd6a26c SET_REALTIME_PRIORITY 2017-02-07 16:36:19 +01:00
Nick Terrell
7e3fc73795 Ensure <zlib.h> can be included in HAVE_ZLIB test 2017-02-06 11:54:31 -08:00
Nick Terrell
2cb8ee8784 Change zlib include to be a system include 2017-02-06 11:32:13 -08:00
Yann Collet
f6e9c4f1e1 Merge pull request #529 from facebook/gzstd_default 
Gzstd default
 2017-02-05 18:03:36 -08:00
cyan4973
613087c02b Silence zlib detection routine 
When it fails, $(CC) sends error message into stderr
redirected to /dev/null
 2017-02-04 23:36:12 -08:00
Nick Terrell
b4016ff02f Add cover dictionary training to zstd.1 
Tested with `make install && man zstd` and visual inspection.
 2017-02-03 16:42:07 -08:00
Yann Collet
762ddeeb9e fixed zstdmt compilation under Windows minGW/MSYS2, by @inikep 2017-02-03 14:35:42 -08:00
Yann Collet
21eb80d485 remove zlib detection artefact 
result of compilation test is sent to /dev/null
 2017-02-03 14:34:41 -08:00
Yann Collet
b02ac8d613 fixed pointer conversion warnings (C++) 
in gz module
 2017-02-03 08:43:06 -08:00
Yann Collet
c2a4632789 release builds use less debug symbols and warnings 
release build are triggered through either `make`,
or their specific target `make zstd-release` and `make lib-release`.
 2017-02-02 20:54:41 -08:00
Yann Collet
c3cba9d858 fixed silent conversion warnings in GZDECOMPRESS path 2017-02-02 17:12:50 -08:00
Yann Collet
030ac243a0 Changed Makefile to generate zstd with .gz support by default 
.gz support is detected by a runtime test.
 2017-02-02 16:49:34 -08:00
Yann Collet
3dc85bae66 minor : fixed zstd-frugal 
fixed a minor unused variable warning when compiling zstd-frugal target
 2017-02-02 16:19:09 -08:00
Yann Collet
48bed91606 Merge pull request #527 from facebook/zstdmt 
zstdmt refinements
 2017-01-31 16:36:46 -08:00
Yann Collet
8d8513fb64 fixed C constant restrictions 2017-01-30 14:37:08 -08:00
Yann Collet
3672d06d06 zstdmt : section size is set to be a minimum of overlapSize 
the minimum size condition size is applied transparently (no warning, no error)
like previous minimum section size condition (1 KB) which still applies.
 2017-01-30 13:35:45 -08:00
Yann Collet
6ccd37c8d4 cli : added advanced parameter overlapLog 
as a hidden (undocumented) parameter for now
 2017-01-30 13:07:24 -08:00
Yann Collet
6be2337c26 added command --block-size= 
for Multi-threading only.
alias : -B#
 2017-01-30 11:17:26 -08:00
Yann Collet
cc3d1bc262 Merge pull request #525 from terrelln/covermt 
Multithreaded COVER dictionary training
 2017-01-30 10:15:33 -08:00
Yann Collet
5d9b894e46 Fixed status display for zstdmt 
There is a large buffering effect when using zstdmt in MT mode.
Consequently, data is read first, pushed to workers,
and only later will the compressed result come out.
That means there is no longer immediate correlation
between amount of data read, and amount of data written.

This patch disables the displaying of % compression
when multi-threading is enabled.

It adds the displaying of total size when it can be determined
(it usually can be determined for files, but not for stdin)
so the user has a sense of "how far from the end" the compression compressed is.

There is no modification to decompression side,
since decompression is only single-threaded for now.
 2017-01-27 13:30:18 -08:00