updated programs/README.md, to introduce compilation variables

make it possible to enable/disable features individually
This commit is contained in:
Yann Collet 2017-05-02 17:18:24 -07:00
parent f47284fec0
commit 710497d8ea
2 changed files with 50 additions and 22 deletions

View File

@ -138,19 +138,16 @@ all: zstd
$(ZSTDDECOMP_O): CFLAGS += $(ALIGN_LOOP)
zstd-nogz zstd xzstd zstd4 xzstd4 : CPPFLAGS += $(THREAD_CPP)
zstd-nogz zstd xzstd zstd4 xzstd4 : LDFLAGS += $(THREAD_LD)
zstd xzstd zstd4 xzstd4 : CPPFLAGS += $(ZLIBCPP)
zstd xzstd zstd4 xzstd4 : LDFLAGS += $(ZLIBLD)
zstd xzstd zstd4 xzstd4 : CPPFLAGS += $(THREAD_CPP) $(ZLIBCPP)
zstd xzstd zstd4 xzstd4 : LDFLAGS += $(THREAD_LD) $(ZLIBLD)
xzstd xzstd4 : CPPFLAGS += $(LZMACPP)
xzstd xzstd4 : LDFLAGS += $(LZMALD)
zstd4 xzstd4 : CPPFLAGS += $(LZ4CPP)
zstd4 xzstd4 : LDFLAGS += $(LZ4LD)
zstd-nogz : ZLIB_MSG := - gzip support is disabled
zstd zstd-nogz zstd4 : LZMA_MSG := - xz/lzma support is disabled
zstd zstd-nogz xzstd : LZ4_MSG := - lz4 support is disabled
zstd zstd-nogz xzstd zstd4 xzstd4 : CPPFLAGS += -DZSTD_LEGACY_SUPPORT=$(ZSTD_LEGACY_SUPPORT)
zstd zstd-nogz xzstd zstd4 xzstd4 : $(ZSTDLIB_FILES) zstdcli.o fileio.o bench.o datagen.o dibio.o
zstd zstd4 : LZMA_MSG := - xz/lzma support is disabled
zstd xzstd : LZ4_MSG := - lz4 support is disabled
zstd xzstd zstd4 xzstd4 : CPPFLAGS += -DZSTD_LEGACY_SUPPORT=$(ZSTD_LEGACY_SUPPORT)
zstd xzstd zstd4 xzstd4 : $(ZSTDLIB_FILES) zstdcli.o fileio.o bench.o datagen.o dibio.o
@echo "$(THREAD_MSG)"
@echo "$(ZLIB_MSG)"
@echo "$(LZMA_MSG)"
@ -170,7 +167,6 @@ ifneq (,$(filter Windows%,$(OS)))
endif
$(CC) -m32 $(FLAGS) $^ $(RES32_FILE) -o $@$(EXT)
zstd-nolegacy : clean_decomp_o
$(MAKE) zstd ZSTD_LEGACY_SUPPORT=0
@ -179,6 +175,12 @@ zstd-nomt : THREAD_LD :=
zstd-nomt : THREAD_MSG := - multi-threading disabled
zstd-nomt : zstd
zstd-nogz : ZLIBCPP :=
zstd-nogz : ZLIBLD :=
zstd-nogz : ZLIB_MSG := - gzip support is disabled
zstd-nogz : zstd
zstd-pgo : MOREFLAGS = -fprofile-generate
zstd-pgo : clean zstd
./zstd -b19i1 $(PROFILE_WITH)

View File

@ -11,8 +11,29 @@ There are however other Makefile targets that create different variations of CLI
- `zstd-decompress` : decompressor-only version of CLI; without dictionary builder, benchmark, and support for decompression of legacy zstd versions
#### Compilation variables
`zstd` tries to detect and use the following features automatically :
- __HAVE_THREAD__ : multithreading is automatically enabled when `pthread` is detected.
It's possible to disable multithread support, by either compiling `zstd-nomt` target or using HAVE_THREAD=0 variable.
Example : make zstd HAVE_THREAD=0
It's also possible to force compilation with multithread support, using HAVE_THREAD=1.
In which case, linking stage will fail if `pthread` library cannot be found.
This might be useful to prevent silent feature disabling.
- __HAVE_ZLIB__ : `zstd` can compress and decompress files in `.gz` format.
This is done through command `--format=gzip`.
Alternatively, symlinks named `gzip` or `gunzip` will mimic intended behavior.
.gz support is automatically enabled when `zlib` library is detected at build time.
It's possible to disable .gz support, by either compiling `zstd-nogz` target or using HAVE_ZLIB=0 variable.
Example : make zstd HAVE_ZLIB=0
It's also possible to force compilation with zlib support, using HAVE_ZLIB=1.
In which case, linking stage will fail if `zlib` library cannot be found.
This might be useful to prevent silent feature disabling.
#### Aggregation of parameters
CLI supports aggregation of parameters i.e. `-b1`, `-e18`, and `-i1` can be joined into `-b1e18i1`.
CLI supports aggregation of parameters i.e. `-b1`, `-e18`, and `-i1` can be joined into `-b1e18i1`.
#### Dictionary builder in Command Line Interface
@ -23,7 +44,7 @@ which can be loaded before compression and decompression.
Using a dictionary, the compression ratio achievable on small data improves dramatically.
These compression gains are achieved while simultaneously providing faster compression and decompression speeds.
Dictionary work if there is some correlation in a family of small data (there is no universal dictionary).
Dictionary work if there is some correlation in a family of small data (there is no universal dictionary).
Hence, deploying one dictionary per type of data will provide the greater benefits.
Dictionary gains are mostly effective in the first few KB. Then, the compression algorithm
will rely more and more on previously decoded content to compress the rest of the file.
@ -35,7 +56,6 @@ Usage of the dictionary builder and created dictionaries with CLI:
3. Decompress with the dictionary: `zstd --decompress FILE.zst -D dictionaryName`
#### Benchmark in Command Line Interface
CLI includes in-memory compression benchmark module for zstd.
The benchmark is conducted using given filenames. The files are read into memory and joined together.
@ -48,7 +68,6 @@ One can select compression levels starting from `-b` and ending with `-e`.
The `-i` parameter selects minimal time used for each of tested levels.
#### Usage of Command Line Interface
The full list of options can be obtained with `-h` or `-H` parameter:
```
@ -62,33 +81,40 @@ Arguments :
-d : decompression
-D file: use `file` as Dictionary
-o file: result stored into `file` (only if 1 input file)
-f : overwrite output without prompting
-f : overwrite output without prompting and (de)compress links
--rm : remove source file(s) after successful de/compression
-k : preserve source file(s) (default)
-h/-H : display help/long help and exit
Advanced arguments :
-V : display Version number and exit
-v : verbose mode; specify multiple times to increase log level (default:2)
-v : verbose mode; specify multiple times to increase verbosity
-q : suppress warnings; specify twice to suppress errors too
-c : force write to standard output, even if it is the console
-r : operate recursively on directories
--ultra : enable levels beyond 19, up to 22 (requires more memory)
-T# : use # threads for compression (default:1)
-B# : select size of each job (default:0==automatic)
--no-dictID : don't write dictID into header (dictionary compression)
--[no-]check : integrity check (default:enabled)
-r : operate recursively on directories
--format=gzip : compress files to the .gz format
--test : test compressed file integrity
--[no-]sparse : sparse mode (default:enabled on file, disabled on stdout)
--[no-]sparse : sparse mode (default:disabled)
-M# : Set a memory usage limit for decompression
-- : All arguments after "--" are treated as files
Dictionary builder :
--train ## : create a dictionary from a training set of files
--train-cover[=k=#,d=#,steps=#] : use the cover algorithm with optional args
--train-legacy[=s=#] : use the legacy algorithm with selectivity (default: 9)
-o file : `file` is dictionary name (default: dictionary)
--maxdict ## : limit dictionary to specified size (default : 112640)
-s# : dictionary selectivity level (default: 9)
--dictID ## : force dictionary ID to specified value (default: random)
--maxdict=# : limit dictionary to specified size (default : 112640)
--dictID=# : force dictionary ID to specified value (default: random)
Benchmark arguments :
-b# : benchmark file(s), using # compression level (default : 1)
-e# : test all compression levels from -bX to # (default: 1)
-i# : minimum evaluation time in seconds (default : 3s)
-B# : cut file into independent blocks of size # (default: no block)
```
--priority=rt : set process priority to real-time
```