Brotli compression format
Go to file
Evgenii Kliuchnikov dbfebd13dc Workaround for GitHub / CodeQL bug
Sometimes GitHub Actions uses bare branch name whereas CodeQL always expects ref.
See https://github.com/github/codeql-action/issues/796

PiperOrigin-RevId: 550504283
2023-07-24 03:30:15 -07:00
.github/workflows Workaround for GitHub / CodeQL bug 2023-07-24 03:30:15 -07:00
c bake in runtime constant 2023-07-20 04:18:46 -07:00
csharp Transpile Java speedup (#548) 2017-05-07 17:40:12 +02:00
docs Decoder API: added API to attach metadata blocks callbacks 2023-01-31 16:03:16 +00:00
fetch-spec Update shared-brotli-fetch-spec.txt 2017-11-17 01:38:05 +01:00
go simplify building of fuzzer 2023-07-10 11:43:27 +00:00
java add synth test for metadata block 2023-07-14 07:26:14 -07:00
js add synth test for metadata block 2023-07-14 07:26:14 -07:00
python Small Python modernization of Brotli code. 2023-07-19 05:44:36 -07:00
research simplify building of fuzzer 2023-07-10 11:43:27 +00:00
scripts Update 2022-11-17 13:03:09 +00:00
tests add tests with UTF8/UTF16 non-ASCII text 2023-07-05 19:15:11 +00:00
.editorconfig Add an EditorConfig file to provide consistent style across editors. (#579) 2017-08-02 16:58:43 +02:00
.gitattributes Re-add python bindings to sources tarball 2020-08-27 16:01:44 +02:00
.gitignore Add *.d to gitignore (#975) 2022-12-21 09:52:31 +01:00
BUILD.bazel simplify building of fuzzer 2023-07-10 11:43:27 +00:00
CHANGELOG.md add CHANGELOG.md 2023-07-18 05:24:13 -07:00
CMakeLists.txt Internal changes 2023-07-04 07:55:25 +00:00
compiler_config_setting.bzl Internal changes 2023-07-04 07:55:25 +00:00
CONTRIBUTING.md Apply 80-chars limit to CONTRIBUTING.md 2016-02-18 15:20:59 +01:00
LICENSE Prepare to release 0.4.0 2016-06-14 15:07:13 +02:00
MANIFEST.in Change MANIFEST.in to include python regression tests in tarball (#841) 2020-09-21 13:24:13 +02:00
pyproject.toml add pyproject.toml (#987) 2022-11-15 18:39:13 +01:00
README Fix #741 2019-02-18 11:31:48 +01:00
README.md drop make / automake files 2023-07-10 07:31:18 -07:00
SECURITY.md Add security policy (#996) 2023-01-03 18:24:47 +01:00
setup.cfg Fix for future versions of python (#911) 2022-01-10 13:08:10 +03:00
setup.py Python: use a new output buffer code (#902) 2022-12-29 14:07:16 +01:00
WORKSPACE.bazel simplify building of fuzzer 2023-07-10 11:43:27 +00:00

Brotli

SECURITY NOTE

Please consider updating brotli to version 1.0.9 (latest).

Version 1.0.9 contains a fix to "integer overflow" problem. This happens when "one-shot" decoding API is used (or input chunk for streaming API is not limited), input size (chunk size) is larger than 2GiB, and input contains uncompressed blocks. After the overflow happens, memcpy is invoked with a gigantic num value, that will likely cause the crash.

Introduction

Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression.

The specification of the Brotli Compressed Data Format is defined in RFC 7932.

Brotli is open-sourced under the MIT License, see the LICENSE file.

Please note: brotli is a "stream" format; it does not contain meta-information, like checksums or uncompresssed data length. It is possible to modify "raw" ranges of the compressed stream and the decoder will not notice that.

Brotli mailing list: https://groups.google.com/forum/#!forum/brotli

GitHub Actions Build Status Fuzzing Status

Build instructions

Vcpkg

You can download and install brotli using the vcpkg dependency manager:

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install brotli

The brotli port in vcpkg is kept up to date by Microsoft team members and community contributors. If the version is out of date, please create an issue or pull request on the vcpkg repository.

Bazel

See Bazel

CMake

The basic commands to build and install brotli are:

$ mkdir out && cd out
$ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=./installed ..
$ cmake --build . --config Release --target install

You can use other CMake configuration.

Python

To install the latest release of the Python module, run the following:

$ pip install brotli

To install the tip-of-the-tree version, run:

$ pip install --upgrade git+https://github.com/google/brotli

See the Python readme for more details on installing from source, development, and testing.

Benchmarks

Disclaimer: Brotli authors take no responsibility for the third party projects mentioned in this section.

Independent decoder implementation by Mark Adler, based entirely on format specification.

JavaScript port of brotli decoder. Could be used directly via npm install brotli

Hand ported decoder / encoder in haxe by Dominik Homberger. Output source code: JavaScript, PHP, Python, Java and C#

7Zip plugin

Dart native bindings

Dart compression framework with fast FFI-based Brotli implementation with ready-to-use prebuilt binaries for Win/Linux/Mac