Commit Graph

13 Commits

Author SHA1 Message Date
Zoltan Szabadka
65f3fc55f5 Bug fixes for the brotli encoder.
* Fix an out-of-bounds access to depth_histo in the
    bit cost calculation function.

  * Change type of distance symbol to uint16_t in block
    splitter, because if all postfix bits are used, there
    can be 520 distance symbols.

  * Save the distance cache between meta-blocks at the
    correct place. This fixes a roundtrip failure that
    can occur when there is an uncompressed metablock
    between two compressed metablocks.

  * Fix a bug when setting lgwin to 24 in the encoder parameters
    It ended up making metablocks larger than 24 bits in size.

  * Fix out-of-bounds memory accesses in parallel encoder.
    CreateBackwardReferences can read up to 4 bytes past end of
    input if the end of input is before mask.

  * Add missing header for memcpy() in port.h
2015-06-12 16:11:50 +02:00
Zoltan Szabadka
667f70adcb Speedups to brotli quality 11.
* Cluster at most 64 histograms at a time in the first
    round of clustering.

  * Use a faster histogram cost estimation function.

  * Don't compute the log2(total) multiple times in the
    block splitter.
2015-06-12 15:29:06 +02:00
Zoltan Szabadka
98539223f5 Remove quality parameter from bitstream writing functions.
Fix a few crashes related to some quality and param combinations.
2015-04-23 16:20:29 +02:00
Zoltan Szabadka
6d80610f03 Fix entropy calculation. 2015-04-23 15:35:16 +02:00
Zoltan Szabadka
ca3a7a98f2 Use FastLog2() instead of log() in BitsEntropy(). 2015-03-30 13:41:52 +02:00
Zoltan Szabadka
534654def1 Add a faster but less dense compression mode.
The new mode can be used by setting the greedy_block_split
field of BrotliParams to true.

This commit moves all the meta-block processing code
into its own library and moves the meta-block encoding
code to brotli_bit_stream.cc from encode.cc
2015-03-27 14:20:35 +01:00
Zoltan Szabadka
d6d9fc60e1 Factor out serialization functions into their own file.
Create a brotli_bit_stream library that is responsible for writing
various structures (headers, Huffman codes, etc.) directly into the
bit-stream.
2014-10-15 14:01:36 +02:00
Zoltan Szabadka
e7650080a8 Updates to Brotli compression format, decoder and encoder
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last month. Most important changes:

   * Format change: don't push distances representing static dictionary words to the distance cache.
   * Fix decoder invalid memory access bug caused by building a non-complete Huffman tree.
   * Add a mode parameter to the encoder interface.
   * Use different hashers for text and font mode.
   * Add a heuristics to the hasher for skipping non-compressible data.
   * Exhaustive search of static dictionary during backward reference search.
2014-03-20 14:32:35 +01:00
Zoltan Szabadka
0454ab4ec0 Updates to Brotli compression format, decoder and encoder
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last month. Most important changes:

   * Fixes to the spec.
   * Change of code length code order.
   * Use a 2-level Huffman lookup table in the decoder.
   * Faster uncompressed meta-block decoding.
   * Optimized encoding of the Huffman code.
   * Detection of UTF-8 input encoding.
   * UTF-8 based literal cost modeling for improved
     backward reference selection.
2014-02-14 15:04:23 +01:00
Zoltan Szabadka
1447345cbb Brotli format change: improved encoding of Huffman codes
This change removes the redundant HCLEN, HLENINC and HLEN
fields from the encoding of the complex Huffman codes and
derives these from an invariant of the code length sequence.
Based on a patch by Robert Obryk.
2013-12-17 17:17:57 +01:00
Zoltan Szabadka
60c24c0c2d Updates to Brotli compression format, decoder and encoder
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last month. Most important changes:

   * Updated spec
   * Changed Huffman code length alphabet to use run length codes more
     efficiently, based on a suggestion by Robert Obryk
   * Changed encoding of the number of Huffman code lengths (HLEN)
   * Changed encoding of the number of Huffman trees (NTREES)
   * Added support for uncompressed meta-blocks
2013-12-12 13:18:04 +01:00
Zoltan Szabadka
c6b9c7c5c8 Updates to Brotli compression format, decoder and encoder
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last three weeks. Most important changes:

  * Added UTF8 context model for good text compression.
  * Simplified context modeling by having only 4 context modes.
  * Per-block context mode selection.
  * Faster backward copying and bit reading functions.
  * More efficient histogram coding.
  * Streaming support for the decoder and encoder.
2013-11-15 19:02:17 +01:00
Zoltan Szabadka
c66e4e3e4f Add brotli compressor
This commit is for the encoder for brotli compression format.
Brotli is a generic byte-level compression algorithm.
2013-10-23 13:06:13 +02:00