Documentation:
- add note that brotli is a "stream" format, not an archive-like
- regenerate .1 with Pandoc
Build:
- drop legacy "BROTLI_BUILD_PORTABLE" option
- drop "BROTLI_SANITIZED" definition
Code:
- c: comb includes
- c/enc: extract encoder state into separate header
- c/enc: drop designated q10 codepath
- c/enc: dealing better with flushing of empty stream
- fix MSVC compilation
API:
- py: use library version instead of one in version.h
- c: add plugable API to report consumed input / produced output
- c/java: support "lean" prepared dictionaries (without copy of source)
* IMPORTANT: decoder: fix potential overflow when input chunk is >2GiB
* simplify max Huffman table size calculation
* eliminate symbol duplicates (static arrays in .h files)
* minor combing in research/ code
- fix formatting
- fix type conversion
- fix no-op arithmetic with null-pointer
- improve performance of hash_longest_match64
- go: detect read after close
- java decoder: support compound dictionary
- remove executable flag on non-scripts
The macro that checks for NEON support should be __ARM_NEON, not
__ARM_NEON__. [1]
AArch64 compilers define __ARM_NEON but not __ARM_NEON__.
AArch32 compilers currently seem to define both, but could be within their
rights to drop __ARM_NEON__ in future versions.
This change moves the check into the common/platform.h file, checks for
both forms, and sets BROTLI_TARGET_NEON if NEON support is available.
[1] Section 6.5.4 of the ARM C Language Extensions.
(At the time of writing, the latest version was Release 2.1.)
* platform: add macro for using the 'aligned' attribute
* decode: add accessor macros for HuffmanCode fields
Adds a constructor function for building HuffmanCode values
so they can be accessed quickly on different architectures.
Also adds macros for marking a HuffmanCode table pointer
that can be accessed quickly (BROTLI_HC_MARK_TABLE_FOR_FAST_LOAD),
adjusting the index into that table (BROTLI_HC_ADJUST_TABLE_INDEX),
and getting the .bits or .value fields out of the table at the
current index (BROTLI_HC_GET_BITS/VALUE).
For example, assuming |table| contains a HuffmanCode pointer:
BROTLI_HC_MARK_TABLE_FOR_FAST_LOAD(table);
BROTLI_HC_ADJUST_TABLE_INDEX(table, index_into_table);
*bits = BROTLI_HC_GET_BITS(table);
*value = BROTLI_HC_GET_VALUE(table);
BROTLI_HC_ADJUST_TABLE_INDEX(table, offset);
*bits2 = BROTLI_HC_GET_BITS(table);
*value2 = BROTLI_HC_GET_VALUE(table);
All uses of the HuffmanCode have been updated appropriately.
* decode: add alternative accessors for HuffmanCode on Arm AArch32
Ensures that Aarch32 Arm builds with an Armv8 compiler do not set
BROTLI_64_BITS.
This scenario is possible with ChromeOS builds, as they may use a
toolchain with the target armv7-cros-gnueabi, but with -march=armv8.
This will set __ARM_ARCH to 8 (defining BROTLI_TARGET_ARMV8), but will
also set __ARM_32BIT_STATE and not __ARM_64BIT_STATE. Without this,
illegal 64-bit non-word-aligned reads (LDRD) may be emitted.
Also fix unaligned 64-bit reads on AArch32 - STRD was still possible to
emit.
Ensures that Aarch32 Arm builds with an Armv8 compiler do not set
BROTLI_64_BITS.
This scenario is possible with ChromeOS builds, as they may use a
toolchain with the target armv7-cros-gnueabi, but with -march=armv8.
This will set __ARM_ARCH to 8 (defining BROTLI_TARGET_ARMV8), but will
also set __ARM_32BIT_STATE and not __ARM_64BIT_STATE. Without this,
illegal 64-bit non-word-aligned reads (LDRD) may be emitted.
Also fix unaligned 64-bit reads on AArch32 - STRD was still possible to
emit.
* Update
* fix ifdef style
* get back to fine-compiler-version-based-macros (use Hedley)
* fix q=0 histogram collection for very long copy/insert commands
* New feature: "Large Window Brotli"
By setting special encoder/decoder flag it is now possible to extend
LZ-window up to 30 bits; though produced stream will not be RFC7932
compliant.
Added new dictionary generator - "DSH". It combines speed of "Sieve"
and quality of "DM". Plus utilities to prepare train corpora
(remove unique strings).
Improved compression ratio: now two sub-blocks could be stitched:
the last copy command could be extended to span the next sub-block.
Fixed compression ineffectiveness caused by floating numbers rounding and
wrong cost heuristic.
Other C changes:
- combined / moved `context.h` to `common`
- moved transforms to `common`
- unified some aspects of code formatting
- added an abstraction for encoder (static) dictionary
- moved default allocator/deallocator functions to `common`
brotli CLI:
- window size is auto-adjusted if not specified explicitly
Java:
- added "eager" decoding both to JNI wrapper and pure decoder
- huge speed-up of `DictionaryData` initialization
* Add dictionaryless compressed dictionary
* Fix `sources.lst`
* Fix `sources.lst` and add a note that `libtool` is also required.
* Update setup.py
* Fix `EagerStreamTest`
* Fix BUILD file
* Add missing `libdivsufsort` dependency
* Fix "unused parameter" warning.
