brotli/c/enc
Eugene Kliuchnikov 35e69fc7cf
New feature: "Large Window Brotli" (#640)
* New feature: "Large Window Brotli"

By setting special encoder/decoder flag it is now possible to extend
LZ-window up to 30 bits; though produced stream will not be RFC7932
compliant.

Added new dictionary generator - "DSH". It combines speed of "Sieve"
and quality of "DM". Plus utilities to prepare train corpora
(remove unique strings).

Improved compression ratio: now two sub-blocks could be stitched:
the last copy command could be extended to span the next sub-block.

Fixed compression ineffectiveness caused by floating numbers rounding and
wrong cost heuristic.

Other C changes:
 - combined / moved `context.h` to `common`
 - moved transforms to `common`
 - unified some aspects of code formatting
 - added an abstraction for encoder (static) dictionary
 - moved default allocator/deallocator functions to `common`

brotli CLI:
 - window size is auto-adjusted if not specified explicitly

Java:
 - added "eager" decoding both to JNI wrapper and pure decoder
 - huge speed-up of `DictionaryData` initialization

* Add dictionaryless compressed dictionary

* Fix `sources.lst`

* Fix `sources.lst` and add a note that `libtool` is also required.

* Update setup.py

* Fix `EagerStreamTest`

* Fix BUILD file

* Add missing `libdivsufsort` dependency

* Fix "unused parameter" warning.
2018-02-26 09:04:36 -05:00
..
backward_references_hq.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
backward_references_hq.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
backward_references_inc.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
backward_references.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
backward_references.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
bit_cost_inc.h Move files & update paths (#541) 2017-04-23 14:07:08 +02:00
bit_cost.c Update (#630) 2017-12-12 14:33:12 +01:00
bit_cost.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
block_encoder_inc.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
block_splitter_inc.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
block_splitter.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
block_splitter.h Update (#630) 2017-12-12 14:33:12 +01:00
brotli_bit_stream.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
brotli_bit_stream.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
cluster_inc.h Update (#560) 2017-06-13 12:52:56 +02:00
cluster.c Update (#630) 2017-12-12 14:33:12 +01:00
cluster.h Update (#630) 2017-12-12 14:33:12 +01:00
command.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
compress_fragment_two_pass.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
compress_fragment_two_pass.h Update (#630) 2017-12-12 14:33:12 +01:00
compress_fragment.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
compress_fragment.h Update (#630) 2017-12-12 14:33:12 +01:00
dictionary_hash.c Update (#630) 2017-12-12 14:33:12 +01:00
dictionary_hash.h Move files & update paths (#541) 2017-04-23 14:07:08 +02:00
encode.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
encoder_dict.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
encoder_dict.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
entropy_encode_static.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
entropy_encode.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
entropy_encode.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
fast_log.h Update (#630) 2017-12-12 14:33:12 +01:00
find_match_length.h Update (#630) 2017-12-12 14:33:12 +01:00
hash_forgetful_chain_inc.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
hash_longest_match64_inc.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
hash_longest_match_inc.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
hash_longest_match_quickly_inc.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
hash_to_binary_tree_inc.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
hash.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
histogram_inc.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
histogram.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
histogram.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
literal_cost.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
literal_cost.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
memory.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
memory.h Update (#630) 2017-12-12 14:33:12 +01:00
metablock_inc.h Update (#630) 2017-12-12 14:33:12 +01:00
metablock.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
metablock.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
params.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
prefix.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
quality.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
ringbuffer.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
static_dict_lut.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
static_dict.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
static_dict.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
utf8_util.c New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00
utf8_util.h Update (#630) 2017-12-12 14:33:12 +01:00
write_bits.h New feature: "Large Window Brotli" (#640) 2018-02-26 09:04:36 -05:00