zstd/tests

Programs and scripts for automated testing of Zstandard

This directory contains the following programs and scripts:

• datagen : Synthetic and parametrable data generator, for tests
• fullbench : Precisely measure speed for each zstd inner functions
• fuzzer : Test tool, to check zstd integrity on target platform
• paramgrill : parameter tester for zstd
• test-zstd-speed.py : script for testing zstd speed difference between commits
• test-zstd-versions.py : compatibility test between zstd versions stored on Github (v0.1+)
• zbufftest : Test tool to check ZBUFF (a buffered streaming API) integrity
• zstreamtest : Fuzzer test tool for zstd streaming API
• legacy : Test tool to test decoding of legacy zstd frames
• decodecorpus : Tool to generate valid Zstandard frames, for verifying decoder implementations

test-zstd-versions.py - script for testing zstd interoperability between versions

This script creates versionsTest directory to which zstd repository is cloned. Then all tagged (released) versions of zstd are compiled. In the following step interoperability between zstd versions is checked.

test-zstd-speed.py - script for testing zstd speed difference between commits

This script creates speedTest directory to which zstd repository is cloned. Then it compiles all branches of zstd and performs a speed benchmark for a given list of files (the testFileNames parameter). After sleepTime (an optional parameter, default 300 seconds) seconds the script checks repository for new commits. If a new commit is found it is compiled and a speed benchmark for this commit is performed. The results of the speed benchmark are compared to the previous results. If compression or decompression speed for one of zstd levels is lower than lowerLimit (an optional parameter, default 0.98) the speed benchmark is restarted. If second results are also lower than lowerLimit the warning e-mail is send to recipients from the list (the emails parameter).

Additional remarks:

• To be sure that speed results are accurate the script should be run on a "stable" target system with no other jobs running in parallel
• Using the script with virtual machines can lead to large variations of speed results
• The speed benchmark is not performed until computers' load average is lower than maxLoadAvg (an optional parameter, default 0.75)
• The script sends e-mails using mutt; if mutt is not available it sends e-mails without attachments using mail; if both are not available it only prints a warning

The example usage with two test files, one e-mail address, and with an additional message:

./test-zstd-speed.py "silesia.tar calgary.tar" "email@gmail.com" --message "tested on my laptop" --sleepTime 60

To run the script in background please use:

nohup ./test-zstd-speed.py testFileNames emails &

The full list of parameters:

positional arguments:
  testFileNames         file names list for speed benchmark
  emails                list of e-mail addresses to send warnings

optional arguments:
  -h, --help            show this help message and exit
  --message MESSAGE     attach an additional message to e-mail
  --lowerLimit LOWERLIMIT
                        send email if speed is lower than given limit
  --maxLoadAvg MAXLOADAVG
                        maximum load average to start testing
  --lastCLevel LASTCLEVEL
                        last compression level for testing
  --sleepTime SLEEPTIME
                        frequency of repository checking in seconds

decodecorpus - tool to generate Zstandard frames for decoder testing

Command line tool to generate test .zst files.

This tool will generate .zst files with checksums, as well as optionally output the corresponding correct uncompressed data for extra verfication.

Example:

./decodecorpus -ptestfiles -otestfiles -n10000 -s5

will generate 10,000 sample .zst files using a seed of 5 in the testfiles directory, with the zstd checksum field set, as well as the 10,000 original files for more detailed comparison of decompression results.

./decodecorpus -t -T1mn

will choose a random seed, and for 1 minute, generate random test frames and ensure that the zstd library correctly decompresses them in both simple and streaming modes.

paramgrill - tool for generating compression table parameters and optimizing parameters on file given constraints

Full list of arguments

 -T#          : set level 1 speed objective
 -B#          : cut input into blocks of size # (default : single block)
 -i#          : iteration loops
 -S           : benchmarks a single run (example command: -Sl3w10h12)
    w# - windowLog
    h# - hashLog
    c# - chainLog
    s# - searchLog
    l# - searchLength
    t# - targetLength
    S# - strategy
    L# - level
 --zstd=      : Single run, parameter selection syntax same as zstdcli
 --optimize=  : find parameters to maximize compression ratio given parameters
    Can use all --zstd= commands to constrain the type of solution found in addition to the following constraints
    cSpeed= - Minimum compression speed
    dSpeed= - Minimum decompression speed
    cMem= - compression memory
    lvl= - Automatically sets compression speed constraint to the speed of that level
 -P#          : generated sample compressibility 
 -t#          : Caps runtime of operation in seconds (default : 99999 seconds (about 27 hours )) 
 -v           : Prints Benchmarking output
 -D           : Next argument dictionary file

Any inputs afterwards are treated as files to benchmark.