Common:
* wrap dictionary data into `BrotliDictionary` structure
* replace public constant with getter `BrotliGetDictionary`
* reformat dictionary data
Decoder:
* adopt common changes
* clarify acceptable instance usage patterns
* hold reference to dictionary in state
Encoder:
* adopt common changes
* eliminate PIC spots in `CreateBackwardReferences`
* add per-chunk ratio guards for q0 and q1
* precompute relative distances to avoid repeated calculations
* prostpone hasher allocation/initialization
* refactor Hashers to be class-like structure
* further improvements for 1MiB+ inputs
* added new hasher type; made hashers more configurable
Java:
* Pull byte->int magic to `IntReader` from `BitReader`
* Blacklist PGI from using conformant array parameters.
There is a bug in pgcc with conformant array parameters where the
length argument is a pointer which triggers a compiler error
(PGC-S-0094, to be specific). The issue has been reported to PGI and
is being tracked internally as TPR 23778. For more information, see
https://www.pgroup.com/userforum/viewtopic.php?t=5501
* travis: Add PGI Community Edition build.
For details on the installation script, see
https://github.com/nemequ/pgi-travis
Decoder may have produced invalid output if:
* at offset 0..3 dictionary word with index 3..0 for some length N is used
and distance is encoded with direct distance code 0, and
* at least one of next 4 commands use value from distance ringbuffer
* pull `BROTLI_MAX_BACKWARD_LIMIT` to constants
* split generic and Zopfli backward references code
* pull hashers init and stitch invocation to encoder
* make `dictionary_hash` a compilation unit
* add `size hint` parameter
* add new hasher
* use `size hint` to pick new hasher for q4
* modernize clz guard (fix#495)
* move `hash to binary tree` to separate file
* add `Initialize` and `Cleanup` to all hashers
* do not raise OOM if malloc(0) == NULL (fix#500)
* pad dictionary LUTs to length 32, etc. (#493)
* avoid using INFINITY constant (#496)
* make dictionary_hash.h more compact
* add "disable literal context modelling" parameter
* fix comment position in `context.h`
* fix typo in internal quality constant name
* deduplicate `BuildMetaBlockGreedy` code
* simplify aggregation in `ChooseContextMap`
* Create brotlidump.py
Sorry, I am a newbie. I couldn't find my file anymore when I wanted to edit it. Hope I don't waste your time.
* Fixed a bug where it couldn't read its own compression.
The problem was that a prefix code ending with a 16 "repeat" didn't realize the table was full already.
Also minor bug fixes, comments and stuff.
* Major refactoring
Rewrote almost everything.
Now can dump its own compression.
* Now more or less complete
Appears to handle all files completely (including metablock data).
Used as inspiration for the the hex example (see makehexexample.py)
* Update python brotli wrapper
* release GIL on CPU intensive blocks, fixes#476
* use BrotliDecoderTakeOutput (less memory, less memcpy)
* Python: Convert bro.py tests to unittest style (#478)
* Create unittest-style tests for `bro.py` decompression and compression
* Delete old tests for `bro.py`
* Update test method generation to properly create a Cartesian product
of iterables using `itertools.product`
* Update python brotli wrapper
* release GIL on CPU intensive blocks, fixes#476
* use BrotliDecoderTakeOutput (less memory, less memcpy)
* Create unittest-style tests for `bro.py` decompression and compression
* Delete old tests for `bro.py`
* Update test method generation to properly create a Cartesian product
of iterables using `itertools.product`
* doxygenize and update API documentation
* fix spelling
* add "fuzz" corpus for java decoder to improve coverage
* use upper-case-snake names for dictionary constant definitions
* use `LDFLAGS` in conventional `Makefile`
Previously, the Python package consisted of a single extension
module, so `build_ext` was sufficient. Now, the package
contains a native module and an extension module, so both
`build_py` and `build_ext` are required. Instead, run `build`,
which calls both `build_py` and `build_ext` automatically.
This adds `flush` and `finish` methods to the `Compressor`
object in the extension module, renames the `compress` method to
`process`, and updates that method to only process data. Now,
one or more `process` calls followed by a `finish` call will be
equivalent to a module-level `compress` call.
Note: To maximize the compression efficiency (and match
underlying Brotli behavior, the `Compressor` object `process`
method does not guarantee all input is immediately written to
output. To ensure immediate output, call `flush` to manually
flush the compression buffer. Extraneous flushing can increase
the size, but may be required when processing streaming data.
Progress on #191
