Commit Graph

31 Commits

Author SHA1 Message Date
Evgenii Kliuchnikov
e7313b0c4e tune memory manager for BROTLI_EXPERIMENTAL
PiperOrigin-RevId: 560703386
2023-08-28 07:20:49 -07:00
Evgenii Kliuchnikov
2a5a088b03 more tuning for BROTLI_EXPERIMENTAL + clean-on-oom
PiperOrigin-RevId: 558771745
2023-08-21 06:36:24 -07:00
Evgenii Kliuchnikov
3ebb2d30ab Move serialized dictionary feature behind the flag.
BROTLI_SHARED_DICTIONARY_SERIALIZED enum value is a part of API,
but it should not be used (will cause failures).
Changing how serialized dictionaries work won't be considered as an API change, until this feature is enabled.
Enabling this feature in the future will be considered as a "compatible" change.

PiperOrigin-RevId: 558091676
2023-08-18 02:55:33 -07:00
Evgenii Kliuchnikov
0f2157cc5e Update comment; fixes #1061
PiperOrigin-RevId: 557501089
2023-08-16 08:55:14 -07:00
Evgenii Kliuchnikov
04f294b18a Fix emitting 1-byte long metadata block
PiperOrigin-RevId: 505484299
2023-01-30 09:10:28 +00:00
Aron Parker
71fe6cac06
Fix BrotliEncoderEstimatePeakMemoryUsage (#1002)
Fixes https://github.com/google/brotli/issues/1001
2023-01-07 22:01:47 +01:00
Evgenii Kliuchnikov
a8f5813b84 Update
Documentation:
  - add note that brotli is a "stream" format, not an archive-like
  - regenerate .1 with Pandoc
Build:
  - drop legacy "BROTLI_BUILD_PORTABLE" option
  - drop "BROTLI_SANITIZED" definition
Code:
  - c: comb includes
  - c/enc: extract encoder state into separate header
  - c/enc: drop designated q10 codepath
  - c/enc: dealing better with flushing of empty stream
  - fix MSVC compilation
API:
  - py: use library version instead of one in version.h
  - c: add plugable API to report consumed input / produced output
  - c/java: support "lean" prepared dictionaries (without copy of source)
2022-11-17 13:03:09 +00:00
Jyrki Alakuijala
4ec67035c0
Merge pull request #929 from jbms/fix-vla-parameter
Fix -Werror=vla-parameter errors with GCC 11.2.0
2021-12-07 01:47:07 +01:00
Eugene Kliuchnikov
8376f72ed6
Prepare for copybara (#939)
Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
2021-11-10 10:34:39 +01:00
Jeremy Maitin-Shepard
27dd726540 Fix -Werror=vla-parameter errors with GCC 11.2.0 2021-09-14 12:27:45 -07:00
Eugene Kliuchnikov
62662f87cd
Strip "./" in includes (#925)
Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
2021-09-08 09:18:45 +02:00
Eugene Kliuchnikov
68f1b90ad0
Update (#918)
Prepare to use copybara worklow.
2021-08-18 19:15:07 +02:00
Eugene Kliuchnikov
19d86fb9a6
Merge-in SharedDictionary feature (#916)
Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
2021-08-04 14:42:02 +02:00
Eugene Kliuchnikov
630b5084ee
Update (#914)
* slimmer stack frames in encoder
 * fix MSAN problem in hasher_composite
   (not dangerous, only in large_window mode)
 * fix JNI decoder wrapper - power-of-two payloads fail to decode sometimes
 * reformat polyfil.js and decode_test.js
2021-07-29 22:29:43 +02:00
Adrián Herrera Arcila
0a3944c8c9
Fix VLA parameter warning (#893)
Make VLA buffer types consistent in declarations and definitions.
Resolves build crash when using -Werror due to "vla-parameter" warning.

Signed-off-by: Adrian Herrera <adr.her.arc.95@gmail.com>
2021-06-23 09:53:59 +02:00
Eugene Kliuchnikov
7f740f1308
Update (#807)
- fix formatting
 - fix type conversion
 - fix no-op arithmetic with null-pointer
 - improve performance of hash_longest_match64
 - go: detect read after close
 - java decoder: support compound dictionary
 - remove executable flag on non-scripts
2020-05-15 11:06:21 +02:00
Eugene Kliuchnikov
78e7bbc3c3
Update (#753)
* fix executable mode of decode.js
 * explain clang-analyser about non-nullability
 * fix "dead assignment"
 * rename proguard.cfg -> proguard.pgcfg
2019-05-03 11:51:11 +02:00
Eugene Kliuchnikov
4b2b2d4f83
Update (#749)
Update:

 * Bazel: fix MSVC configuration
 * C: common: extended documentation and helpers around distance codes
 * C: common: enable BROTLI_DCHECK in "debug" builds
 * C: common: fix implicit trailing zero in `kPrefixSuffix`
 * C: dec: fix possible bit reader discharge for "large-window" mode
 * C: dec: simplify distance decoding via lookup table
 * C: dec: reuse decoder state members memory via union with lookup table
 * C: dec: add decoder state diagram
 * C: enc: clarify access to static dictionary
 * C: enc: improve static dictionary hash
 * C: enc: add "stream offset" parameter for parallel encoding
 * C: enc: reorganize hasher; now Q2-Q3 require exactly 256KiB
           to avoid global TCMalloc lock
 * C: enc: fix rare access to uninitialized data in ring-buffer
 * C: enc: reorganize logging / checks in `write_bits.h`
 * Java: dec: add "large-window" support
 * Java: dec: improve speed
 * Java: dec: debug and 32-bit mode are now activated via system properties
 * Java: dec: demystify some state variables (use better names)
 * Dictionary generator: add single input mode
 * Java: dec: modernize tests
 * Bazel: js: pick working commit for closure rules
2019-04-12 13:57:42 +02:00
Eugene Kliuchnikov
d0ffe60b87
Verbose CLI + start pulling "Shared-Brotli" (#722)
* Verbose CLI + start pulling "Shared-Brotli"

 * vesbose CLI output; fix #666
 * pull `SHIFT` transforms; currently this is semantically dead code;
   later it will be used by "Shared-Brotli"
2018-10-24 16:06:09 +02:00
Eugene Kliuchnikov
f7cbc97c96
Fix typo / minor formatting (#716)
* Fix typo / minor formatting / pull computable constant to the place of use.
2018-10-16 16:46:54 +02:00
Eugene Kliuchnikov
8544ae858d
Update (#680)
* fix MSVC warnings
 * cleanups
2018-06-09 11:17:13 +02:00
Eugene Kliuchnikov
68db5c0272
Update (#660)
* Update
 * improve q=1 compression on small files
 * fix "left shift before promotion"
 * fix osx Travis builds
2018-04-13 11:44:34 +02:00
Eugene Kliuchnikov
0f3c84e745
Update (#656)
* proper fix for the "fall through" warning"
 * automatic NDIRECT/NPOSTFIX tuning (better compression)
 * fix unaligned access for `aarch64`-cross-`armhf` build
 * fix `aarch64` detection (10% decoder speedup)
 * expose `large_window` CLI option
 * make default window size 16MiB
 * ramp up version to 1.0.4
2018-03-27 22:29:22 +02:00
Eugene Kliuchnikov
631fe194a1
Update (#651)
* fix `bazel` build (ignore switch case fall-through)
* add `NPOSTFIX` / `NDIRECT` encoder parameters
* fix source file lists (add `params.h`)
* fix bug in `durchschlag`
* print clarifying messages wheb CLI argument parsing fails
2018-03-20 17:37:41 +06:00
Eugene Kliuchnikov
533843e354
Update (#643)
Update
 * make the zopflification aware of `NDIRECT`, `NPOSTFIX`
   (better compression in `font` mode)
 * add small and simple decoder tool
 * fix typo
 * Java: wrapper: make decoder channel more async-friendly

Ramp up version to 1.0.3 / 1.0.3
2018-03-02 15:49:58 +01:00
Eugene Kliuchnikov
35e69fc7cf
New feature: "Large Window Brotli" (#640)
* New feature: "Large Window Brotli"

By setting special encoder/decoder flag it is now possible to extend
LZ-window up to 30 bits; though produced stream will not be RFC7932
compliant.

Added new dictionary generator - "DSH". It combines speed of "Sieve"
and quality of "DM". Plus utilities to prepare train corpora
(remove unique strings).

Improved compression ratio: now two sub-blocks could be stitched:
the last copy command could be extended to span the next sub-block.

Fixed compression ineffectiveness caused by floating numbers rounding and
wrong cost heuristic.

Other C changes:
 - combined / moved `context.h` to `common`
 - moved transforms to `common`
 - unified some aspects of code formatting
 - added an abstraction for encoder (static) dictionary
 - moved default allocator/deallocator functions to `common`

brotli CLI:
 - window size is auto-adjusted if not specified explicitly

Java:
 - added "eager" decoding both to JNI wrapper and pure decoder
 - huge speed-up of `DictionaryData` initialization

* Add dictionaryless compressed dictionary

* Fix `sources.lst`

* Fix `sources.lst` and add a note that `libtool` is also required.

* Update setup.py

* Fix `EagerStreamTest`

* Fix BUILD file

* Add missing `libdivsufsort` dependency

* Fix "unused parameter" warning.
2018-02-26 09:04:36 -05:00
Eugene Kliuchnikov
da254cffdb
Update (#630)
* merge {dec|enc}/port.h into common/platform.h
 * fix one-shot q=10 1-byte input compression
 * fix some unprefixed definitions
 * make hashers host-endianness-independent
 * extract enc/params.h from enc/quality.h
 * fix API documentation / typos
 * improve `BrotliEncoderMaxCompressedSize`
2017-12-12 14:33:12 +01:00
Eugene Kliuchnikov
37fb83ec0d Update: (#600)
* encoder: relax backward references candidates asserts
 * encoder: make RNG more platform-independent
 * encoder: remove "unused" param (context mode)
 * CLI: improve first-encounter experience
 * Java: update SynthTest
 * Java: refine proguard config
 * Java/JNI: fix one-shot compression workflow
2017-09-19 15:57:15 +02:00
Eugene Kliuchnikov
d7bce1e092 Update (#593)
* Update:
 * fix CLI error messages
 * fix CLI console IO on Windows
2017-09-07 20:27:49 +02:00
Eugene Kliuchnikov
d63e8f75f5 Update API, and more (#581)
Update API, and more:
 * remove "custom dictionary" support
 * c/encoder: fix #580: big-endian build
 * Java: reduce jar size
 * Java: speedup decoding
 * Java: add 32-bit CPU support
 * Java: make source code JS transpiler-ready
2017-08-04 10:02:56 +02:00
Eugene Kliuchnikov
6ece1d8791 Move files & update paths (#541)
* Move files & update paths

* Rename build to scripts.

* Fix paths

* Fix script.
2017-04-23 14:07:08 +02:00