Add a BrotliCompress() method to the public encoder API
that uses the BrotliIn and BrotliOut classes and use
that in the 'bro' command-line tool.
Use the streaming api in BrotliCompressBuffer() and
BrotliCompressor::WriteMetaBlock().
Use the appropiate hashers for quality <= 9.
Disable all slow features for quality <= 9 (literal cost modeling,
dictionary, context modeling, advanced block splitting).
Change vector<Command> arguments of internal functions
to Command* and size_t.
Add a version of the brotli encoder that compresses
each meta-block independently, only using the
original input data from previous meta-blocks
and nothing from the compressor state.
This is a proof-of-concept to show that the
current format is flexible enough to support
parallel multi-threaded compression.
This will enable processing the input in smaller
chunks than the currently default 2MB for the
slow brotli, while still benefiting from the
larger sliding window.
Remove the hard-coded constants for window size
and meta-block size.
Initialize internal storage for each metablock
separately and reserve only as much as needed
for the actual input.
The new mode can be used by setting the greedy_block_split
field of BrotliParams to true.
This commit moves all the meta-block processing code
into its own library and moves the meta-block encoding
code to brotli_bit_stream.cc from encode.cc
As reported by @anthrotype, log2() is missing from MSVS 2010.
This patch uses log() and a multiplication in FastLog2()
for _MSV_VER <= 1600 and uses FastLog2() in literal_cost.cc
instead of log2().
Changes suggested by @r-lyeh and @anthrotype.
- Use a portable simple PRNG instead of rand_r()
- add missing <assert.h> include
- disambiguate log2() argument type
- remove endian.h include from write_bits.h
The new interface of the backward reference search
function makes it possible to use it in a streaming
manner.
Using the advanced cost model and static dictionary
can be turned on/off by template parameters.
The distance short codes are now computed as part of
the backward reference search.
Added a faster version of the Hasher.
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last month. Most important changes:
* Format change: don't push distances representing static dictionary words to the distance cache.
* Fix decoder invalid memory access bug caused by building a non-complete Huffman tree.
* Add a mode parameter to the encoder interface.
* Use different hashers for text and font mode.
* Add a heuristics to the hasher for skipping non-compressible data.
* Exhaustive search of static dictionary during backward reference search.
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last month. Most important changes:
* Fixes to the spec.
* Change of code length code order.
* Use a 2-level Huffman lookup table in the decoder.
* Faster uncompressed meta-block decoding.
* Optimized encoding of the Huffman code.
* Detection of UTF-8 input encoding.
* UTF-8 based literal cost modeling for improved
backward reference selection.
This change removes the redundant HCLEN, HLENINC and HLEN
fields from the encoding of the complex Huffman codes and
derives these from an invariant of the code length sequence.
Based on a patch by Robert Obryk.
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last month. Most important changes:
* Updated spec
* Changed Huffman code length alphabet to use run length codes more
efficiently, based on a suggestion by Robert Obryk
* Changed encoding of the number of Huffman code lengths (HLEN)
* Changed encoding of the number of Huffman trees (NTREES)
* Added support for uncompressed meta-blocks
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last three weeks. Most important changes:
* Added UTF8 context model for good text compression.
* Simplified context modeling by having only 4 context modes.
* Per-block context mode selection.
* Faster backward copying and bit reading functions.
* More efficient histogram coding.
* Streaming support for the decoder and encoder.
