updated build documentation

lib/README.md

@@ -168,6 +168,26 @@ file it should be linked with `dll\libzstd.dll`. For example:
 The compiled executable will require ZSTD DLL which is available at `dll\libzstd.dll`.
 
 
+#### Advanced Build options
+
+The build system requires a hash function in order to
+separate object files created with different compilation flags.
+By default, it tries to use `md5sum` or equivalent.
+The hash function can be manually switched by setting the `HASH` variable.
+For example : `make HASH=xxhsum`
+The hash function needs to generate at least 64-bit using hexadecimal format.
+When no hash function is found,
+the Makefile just generates all object files into the same default directory,
+irrespective of compilation flags.
+This functionality only matters if `libzstd` is compiled multiple times
+with different build flags.
+
+The build directory, where object files are stored
+can also be manually controlled using variable `BUILD_DIR`,
+for example `make BUILD_DIR=objectDir/v1`.
+In which case, the hash function doesn't matter.
+
+
 #### Deprecated API
 
 Obsolete API on their way out are stored in directory `lib/deprecated`.

programs/README.md

@@ -3,7 +3,7 @@ Command Line Interface for Zstandard library
 
 Command Line Interface (CLI) can be created using the `make` command without any additional parameters.
 There are however other Makefile targets that create different variations of CLI:
-- `zstd` : default CLI supporting gzip-like arguments; includes dictionary builder, benchmark, and support for decompression of legacy zstd formats
+- `zstd` : default CLI supporting gzip-like arguments; includes dictionary builder, benchmark, and supports decompression of legacy zstd formats
 - `zstd_nolegacy` : Same as `zstd` but without support for legacy zstd formats
 - `zstd-small` : CLI optimized for minimal size; no dictionary builder, no benchmark, and no support for legacy zstd formats
 - `zstd-compress` : version of CLI which can only compress into zstd format
@@ -147,76 +147,91 @@ FILE    : a filename
 Arguments :
  -#     : # compression level (1-19, default: 3)
  -d     : decompression
- -D file: use `file` as Dictionary
- -o file: result stored into `file` (only if 1 input file)
- -f     : overwrite output without prompting and (de)compress links
+ -D DICT: use DICT as Dictionary for compression or decompression
+ -o file: result stored into `file` (only 1 output file)
+ -f     : overwrite output without prompting, also (de)compress links
 --rm    : remove source file(s) after successful de/compression
  -k     : preserve source file(s) (default)
  -h/-H  : display help/long help and exit
 
 Advanced arguments :
  -V     : display Version number and exit
+ -c     : force write to standard output, even if it is the console
  -v     : verbose mode; specify multiple times to increase verbosity
  -q     : suppress warnings; specify twice to suppress errors too
- -c     : force write to standard output, even if it is the console
- -l     : print information about zstd compressed files
---exclude-compressed:  only compress files that are not previously compressed
+--no-progress : do not display the progress counter
+ -r     : operate recursively on directories
+--filelist FILE : read list of files to operate upon from FILE
+--output-dir-flat DIR : processed files are stored into DIR
+--output-dir-mirror DIR : processed files are stored into DIR respecting original directory structure
+--[no-]check : during compression, add XXH64 integrity checksum to frame (default: enabled). If specified with -d, decompressor will ignore/validate checksums in compressed frame (default: validate).
+--      : All arguments after "--" are treated as files
+
+Advanced compression arguments :
 --ultra : enable levels beyond 19, up to 22 (requires more memory)
 --long[=#]: enable long distance matching with given window log (default: 27)
 --fast[=#]: switch to very fast compression levels (default: 1)
 --adapt : dynamically adapt compression level to I/O conditions
---stream-size=# : optimize compression parameters for streaming input of given number of bytes
---size-hint=# optimize compression parameters for streaming input of approximately this size
---target-compressed-block-size=# : make compressed block near targeted size
  -T#    : spawns # compression threads (default: 1, 0==# cores)
  -B#    : select size of each job (default: 0==automatic)
+--single-thread : use a single thread for both I/O and compression (result slightly different than -T1)
 --rsyncable : compress using a rsync-friendly method (-B sets block size)
---no-dictID : don't write dictID into header (dictionary compression)
---[no-]check : integrity check (default: enabled)
+--exclude-compressed: only compress files that are not already compressed
+--stream-size=# : specify size of streaming input from `stdin`
+--size-hint=# optimize compression parameters for streaming input of approximately this size
+--target-compressed-block-size=# : generate compressed block of approximately targeted size
+--no-dictID : don't write dictID into header (dictionary compression only)
 --[no-]compress-literals : force (un)compressed literals
- -r     : operate recursively on directories
---output-dir-flat[=directory]: all resulting files stored into `directory`.
 --format=zstd : compress files to the .zst format (default)
 --format=gzip : compress files to the .gz format
+--format=xz : compress files to the .xz format
+--format=lzma : compress files to the .lzma format
+--format=lz4 : compress files to the .lz4 format
+
+Advanced decompression arguments :
+ -l     : print information about zstd compressed files
 --test  : test compressed file integrity
---[no-]sparse : sparse mode (default: disabled)
  -M#    : Set a memory usage limit for decompression
---no-progress : do not display the progress bar
---      : All arguments after "--" are treated as files
+--[no-]sparse : sparse mode (default: disabled)
 
 Dictionary builder :
 --train ## : create a dictionary from a training set of files
 --train-cover[=k=#,d=#,steps=#,split=#,shrink[=#]] : use the cover algorithm with optional args
 --train-fastcover[=k=#,d=#,f=#,steps=#,split=#,accel=#,shrink[=#]] : use the fast cover algorithm with optional args
 --train-legacy[=s=#] : use the legacy algorithm with selectivity (default: 9)
- -o file : `file` is dictionary name (default: dictionary)
+ -o DICT : DICT is dictionary name (default: dictionary)
 --maxdict=# : limit dictionary to specified size (default: 112640)
 --dictID=# : force dictionary ID to specified value (default: random)
 
 Benchmark arguments :
  -b#    : benchmark file(s), using # compression level (default: 3)
- -e#    : test all compression levels from -bX to # (default: 1)
+ -e#    : test all compression levels successively from -b# to -e# (default: 1)
  -i#    : minimum evaluation time in seconds (default: 3s)
  -B#    : cut file into independent blocks of size # (default: no block)
+ -S     : output one benchmark result per input file (default: consolidated result)
 --priority=rt : set process priority to real-time
 ```
 
 ### Passing parameters through Environment Variables
+There is no "generic" way to pass "any kind of parameter" to `zstd` in a pass-through manner.
+Using environment variables for this purpose has security implications.
+Therefore, this avenue is intentionally restricted and only supports `ZSTD_CLEVEL` and `ZSTD_NBTHREADS`.
+
 `ZSTD_CLEVEL` can be used to modify the default compression level of `zstd`
 (usually set to `3`) to another value between 1 and 19 (the "normal" range).
-`ZSTD_NBTHREADS` can be used to specify number of threads that `zstd` will use during compression, which by default is `1`.
+
+`ZSTD_NBTHREADS` can be used to specify a number of threads
+that `zstd` will use for compression, which by default is `1`.
 This functionality only exists when `zstd` is compiled with multithread support.
+`0` means "use as many threads as detected cpu cores on local system".
 The max # of threads is capped at: `ZSTDMT_NBWORKERS_MAX==200`.
 
 This functionality can be useful when `zstd` CLI is invoked in a way that doesn't allow passing arguments.
 One such scenario is `tar --zstd`.
 As `ZSTD_CLEVEL` and `ZSTD_NBTHREADS` only replace the default compression level
-and number of threads, respectively, they can both be overridden by corresponding command line arguments:
+and number of threads respectively, they can both be overridden by corresponding command line arguments:
 `-#` for compression level and `-T#` for number of threads.
 
-There is no "generic" way to pass "any kind of parameter" to `zstd` in a pass-through manner.
-Using environment variables for this purpose has security implications.
-Therefore, this avenue is intentionally restricted and only supports `ZSTD_CLEVEL` and `ZSTD_NBTHREADS`.
 
 ### Long distance matching mode
 The long distance matching mode, enabled with `--long`, is designed to improve