zstd/lib/dictBuilder
Nick Terrell 08981d2638 [lib] Allow compression dictionaries with missing symbols
Allow compression to use dictionaries with missing symbols in their
entropy tables. We set the FSE repeat mode to check when there are
missing symbols, and set the FSE repeat mode to valid when all symbols
are present.

Note that when not all symbols are present, the heuristics which favor
dictionary tables for lower compression levels won't activate.

Tested by manually creating a dictionary with missing symbols of every
type, and validing that the compressor rejects it before this change,
and accepts it after this change. Also, I ran the `dictionary_loader`
fuzzer for >1 hour of CPU time without running into cases where
compression succeeds, but decompression fails.

Fixes #2174.
2020-06-12 17:57:19 -07:00
..
cover.c Rewrite Include Paths to be Relative 2020-05-04 15:20:26 -04:00
cover.h Rewrite Include Paths to be Relative 2020-05-04 15:20:26 -04:00
divsufsort.c fixed more issues detected by recent version of scan-build 2018-08-16 15:20:25 -07:00
divsufsort.h fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
fastcover.c Rewrite Include Paths to be Relative 2020-05-04 15:20:26 -04:00
zdict.c [lib] Allow compression dictionaries with missing symbols 2020-06-12 17:57:19 -07:00
zdict.h [zdict] Stabilize ZDICT_finalizeDictionary() 2020-05-07 10:37:01 -07:00