Commit Graph

52 Commits

Author SHA1 Message Date
Zoltan Szabadka
1216684d0b Merge remote-tracking branch 'upstream/master' 2015-09-23 12:26:07 +02:00
Zoltan Szabadka
a7cedfc26d Update brotli decoder with latest improvements. 2015-09-23 12:24:06 +02:00
Marcin Karpinski
21ac39f7c8 Fix typos. 2015-09-21 21:04:07 +02:00
Eugene Klyuchnikov
127aa48a96 Fix bug in decoder.
Minor cleanup.
2015-09-01 12:18:22 +02:00
Lode Vandevenne
db71549ac5 update brotli decoder with latest improvements 2015-08-28 15:20:24 +02:00
Zoltan Szabadka
944c3b8290 Remove non-standard <malloc.h> from decode.c
The already included <stdlib.h> is enough for the malloc/free calls.
2015-08-11 11:07:26 +02:00
Zoltan Szabadka
4b2fd00e24 Use a single lookup table for insert/copy offsets and extra bits.
Remove safe_malloc.c since all the bounds checking is done inside
decode.c now.
2015-08-10 16:39:50 +02:00
Lode Vandevenne
94cd7085f7 brotli decoder performance improvements 2015-08-10 13:35:23 +02:00
Lode Vandevenne
bad0f4edf1 Brotli Bug Fixes 2015-06-26 17:37:00 +02:00
Zoltan Szabadka
b43df8f699 Brotli custom LZ77 dictionary support.
Adds functions to prepend such dictionary to the
encoder and decoder, and twiddles their internal
parameters to do as if that was a previous part of
the input. This dictionary is just a prefilled LZ77
window, it is not related to the built in transformable
brotli dictionary.
2015-06-12 15:43:54 +02:00
Zoltan Szabadka
641bc15882 Speedups and fixes to the decoder.
* Read data by 4-byte runs.
    This resolves unaligned read (Bus error) on arm-android.

  * Get rid of malloc/free in BrotliBuildHuffmanTable.

  * Tweak order of instructions when reading Huffman codes.
2015-06-12 15:12:23 +02:00
Zoltan Szabadka
54f69c9ef7 Support window bits 10 - 15 in the decoder.
The previous window bit value 17 is used to
extend the range, since it has not been used
in any previous encoders.
2015-05-07 17:44:33 +02:00
Zoltan Szabadka
5f39d607ef Decoder optimizations for ARM architecture. 2015-05-07 17:10:27 +02:00
Zoltan Szabadka
83aa24dc86 Speed and memory usage improvements for the decoder.
* Change order of members of bit reader state structure.

* Remove unused includes for assert. Add BROTLI_DCHECK
  macros and use it instead of assert.

* Do not calculate nbits in common case of ReadSymbol.

* Introduce and use PREDICT_TRUE / PREDICT_FALSE macros.

* Allocate less memory in the brotli decoder if it knows
  the result size beforehand. Before this, the decoder
  would always allocate 16MB if the encoder annotated the
  window size as 22 bit (which is the default), even if the
  file is only a few KB uncompressed. Now, it'll only
  allocate a ringbuffer as large as needed for the result file.
  But only if it can know the filesize, it's not possible
  to know that if there are multiple metablocks or too large
  uncompressed metablock.
2015-05-07 16:53:43 +02:00
Zoltan Szabadka
fe14d7b32a Fix an error propagation bug in the decoder. 2015-04-28 10:07:12 +02:00
Zoltan Szabadka
64c261113f Add partial output support to the decoder. 2015-04-22 17:33:21 +02:00
Zoltan Szabadka
28b1f7a6b1 Implement some stricter format checks in the decoder.
- Reject brotli streams where the number of
    nibbles is too large for the size of the
    meta-block
  - Reject brotli streams where the padding
    bits after a meta-block are not all zero
  - Reject brotli streams where the symbol
    in the simple prefix code is not in
    the symbol alphabet
2015-04-22 14:35:21 +02:00
Zoltan Szabadka
a81f2ef433 Decoder support for new empty meta-block format.
This change enforces the new 16MB limit on the
size of the meta-blocks and adds support for
empty meta-blocks with optional ignored metadata.
2015-04-22 14:25:08 +02:00
Zoltan Szabadka
2a7bbfc553 Update the previous two bytes when decoding an
uncompressed metablock.
2015-04-01 16:23:18 +02:00
Zoltan Szabadka
fac8993bef Add a missing state transition to CopyUncompressedBlockToOutput(). 2015-04-01 16:20:16 +02:00
Zoltan Szabadka
fe6e5d1cd9 Faster decoder for data with trivial context maps.
This makes it 20% faster when decoding data with
trivial context map (where the histogram type
depends only on the block type and not the
context).
2015-04-01 16:15:39 +02:00
Lode Vandevenne
f9e5a2d9f2 bugfixes affecting streaming decoding 2015-03-30 18:00:40 +02:00
Zoltan Szabadka
8f3092a0b1 Add some missing state transitions to DecodeContextMap().
Update the states in DecodeContextMap to the next one,
else it would continue at the wrong spot if exactly there
the partial input is split.
2015-03-30 13:39:20 +02:00
Zoltan Szabadka
c5ccd0dff9 Change the return value of decoder functions from integer codes to an enum. 2015-03-27 13:54:43 +01:00
Lode Vandevenne
85abce6c6e Propagate error condition from ReadHuffmanCodeLengths() 2015-03-26 17:40:07 +01:00
Lode Vandevenne
b142207917 a few more tweaks to the streaming support 2015-03-20 17:03:11 +01:00
Lode Vandevenne
8270250b40 support for partial input (streaming) 2015-03-20 16:13:15 +01:00
Zoltan Szabadka
6da0c5c992 Implement a 32-bit bitstream decoder.
The 64-bit bitstream decoder seems to have portability problems
with emscripten/asm.js as it does not compile into working code.
2015-02-25 13:32:17 +01:00
Zoltan Szabadka
68d63f5966 Fix a sign-conversion warning in the decoder. 2015-02-24 11:22:29 +01:00
Kunihiko Sakamoto
65d655dad5 Compile decoder with -Wall flag
This also fixes two "comparison between signed and unsigned" warnings.
2015-02-24 17:48:26 +09:00
Zoltan Szabadka
4278934e82 Fix undefined behavior in decoder.
Use memmove() for copying overlapping buffers.
2014-11-26 10:37:33 +01:00
Zoltan Szabadka
460dda1b50 Decoder code cleanup.
1) Use a tighter upper bound on the Huffman table size.
2) Remove unused argument len in ToUpperCase.
3) Remove unused no-op assignment in BrotliDecompress.
4) Fix include guard name.
2014-10-28 13:57:46 +01:00
Zoltan Szabadka
79d2b89d2d Fix BrotliDecompressedSize() to work for an uncompressed plus an empty meta-block. 2014-10-15 13:41:00 +02:00
Zoltan Szabadka
347781947a Update the dictionary and the transforms. 2014-03-25 16:48:25 +01:00
Zoltan Szabadka
e7650080a8 Updates to Brotli compression format, decoder and encoder
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last month. Most important changes:

   * Format change: don't push distances representing static dictionary words to the distance cache.
   * Fix decoder invalid memory access bug caused by building a non-complete Huffman tree.
   * Add a mode parameter to the encoder interface.
   * Use different hashers for text and font mode.
   * Add a heuristics to the hasher for skipping non-compressible data.
   * Exhaustive search of static dictionary during backward reference search.
2014-03-20 14:32:35 +01:00
Zoltan Szabadka
cddab4adef Enable the static dictionary in the Brotli decoder. 2014-03-06 17:25:43 +01:00
Zoltan Szabadka
2f268ad158 Add the initial version of the static dictionary and transforms to Brotli. 2014-02-17 14:25:36 +01:00
Zoltan Szabadka
0454ab4ec0 Updates to Brotli compression format, decoder and encoder
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last month. Most important changes:

   * Fixes to the spec.
   * Change of code length code order.
   * Use a 2-level Huffman lookup table in the decoder.
   * Faster uncompressed meta-block decoding.
   * Optimized encoding of the Huffman code.
   * Detection of UTF-8 input encoding.
   * UTF-8 based literal cost modeling for improved
     backward reference selection.
2014-02-14 15:04:23 +01:00
Zoltan Szabadka
d01c71c4ad Fix -Wconversion compiler warnings in the brotli decoder. 2014-01-08 12:34:35 +01:00
Zoltan Szabadka
2bcd58bb5a Brotli format change: small improvement to the encoding of Huffman codes
Combine the HSKIP and the simple/complex Huffman code type bits.
2014-01-08 12:28:28 +01:00
Zoltan Szabadka
d762bc6845 Bug fixes for the brotli encoder and decoder. 2014-01-06 16:01:57 +01:00
Zoltan Szabadka
1447345cbb Brotli format change: improved encoding of Huffman codes
This change removes the redundant HCLEN, HLENINC and HLEN
fields from the encoding of the complex Huffman codes and
derives these from an invariant of the code length sequence.
Based on a patch by Robert Obryk.
2013-12-17 17:17:57 +01:00
Zoltan Szabadka
b8a1008569 Use C-style comments in the brotli decoder. 2013-12-16 14:45:57 +01:00
Zoltan Szabadka
29bb7cb1af Fix Microsoft VisualStudio 64-bit build of brotli 2013-12-13 15:30:20 +01:00
Zoltan Szabadka
354349d7ae Fix Microsoft VisualStudio build of brotli
- Move all variable declarations to the beginning of the block
- #ifdef-out read/write calls
2013-12-13 10:39:46 +01:00
Zoltan Szabadka
60c24c0c2d Updates to Brotli compression format, decoder and encoder
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last month. Most important changes:

   * Updated spec
   * Changed Huffman code length alphabet to use run length codes more
     efficiently, based on a suggestion by Robert Obryk
   * Changed encoding of the number of Huffman code lengths (HLEN)
   * Changed encoding of the number of Huffman trees (NTREES)
   * Added support for uncompressed meta-blocks
2013-12-12 13:18:04 +01:00
Zoltan Szabadka
8d7081f2d0 Add draft specification of the brotli format 2013-11-28 17:37:13 +01:00
Roderick Sheeter
1cdcbd851f Added Brotli compress/decompress utilities and makefiles 2013-11-19 14:32:56 -08:00
Zoltan Szabadka
c6b9c7c5c8 Updates to Brotli compression format, decoder and encoder
This commit contains a batch of changes that were made to the Brotli
compression algorithm in the last three weeks. Most important changes:

  * Added UTF8 context model for good text compression.
  * Simplified context modeling by having only 4 context modes.
  * Per-block context mode selection.
  * Faster backward copying and bit reading functions.
  * More efficient histogram coding.
  * Streaming support for the decoder and encoder.
2013-11-15 19:02:17 +01:00
Zoltan Szabadka
6455522172 Make the brotli decoder more C90-compatible.
(1) Move all variable declarations to the beginning of the block.
(2) Remove 'z' printf modifiers.
(3) Fix 'comma at the end of enumeration list' warning.
2013-10-22 15:02:54 +02:00