Brotli compression format
Go to file
Alex Nicksay 5632315d35 Python: Support streamed compression with the Compressor object (#448)
This adds `flush` and `finish` methods to the `Compressor`
object in the extension module, renames the `compress` method to
`process`, and updates that method to only process data.  Now,
one or more `process` calls followed by a `finish` call will be
equivalent to a module-level `compress` call.

Note: To maximize the compression efficiency (and match
underlying Brotli behavior, the `Compressor` object `process`
method does not guarantee all input is immediately written to
output. To ensure immediate output, call `flush` to manually
flush the compression buffer.  Extraneous flushing can increase
the size, but may be required when processing streaming data.

Progress on #191
2016-10-24 13:28:56 +02:00
appveyor [appveyor/install.ps1] remove unused functions for miniconda 2015-10-07 12:46:16 +01:00
common Build shared libraries by default 2016-10-18 16:45:32 +02:00
dec Fix -Wcast-align warnings 2016-10-20 10:28:44 +02:00
docs Update the spec reference to RFC 7932, remove the old internet draft. 2016-08-02 13:27:29 +02:00
enc Fix "take output" flush workflow. 2016-10-19 16:19:26 +02:00
include/brotli * leave static compilation declaration intouch (e.g. Python build) 2016-10-18 17:14:49 +02:00
java Fix POM files sources paths 2016-10-18 15:28:43 +02:00
python Python: Support streamed compression with the Compressor object (#448) 2016-10-24 13:28:56 +02:00
research Update research 2016-09-22 11:32:23 +02:00
terryfy@8bb673f441 add support for Travis CI (OS X only for now) 2015-08-14 11:28:26 +01:00
tests Add mingw support. 2016-07-28 19:32:33 -07:00
tools Update research 2016-09-22 11:32:23 +02:00
.gitignore Update build systems 2016-06-24 15:32:51 +02:00
.gitmodules Replace sais.hxx by submodule hillbig/esaxx. 2016-09-19 19:12:30 +02:00
.travis.sh .travis.sh: skip source venv on linux 2016-08-10 19:27:17 +01:00
.travis.yml Fix OSX gcc-4.x compilation (#455) 2016-10-20 14:16:00 +02:00
appveyor.yml appveyor: add Visual Studio builds (based on CMake) 2016-07-29 13:24:17 -07:00
BUILD Remove the underscore in the name of brotli libraries. #326 2016-10-12 18:24:04 +02:00
CMakeLists.txt * leave static compilation declaration intouch (e.g. Python build) 2016-10-18 17:14:49 +02:00
configure Update build systems 2016-06-24 15:32:51 +02:00
CONTRIBUTING.md Apply 80-chars limit to CONTRIBUTING.md 2016-02-18 15:20:59 +01:00
LICENSE Prepare to release 0.4.0 2016-06-14 15:07:13 +02:00
Makefile Update research 2016-09-22 11:32:23 +02:00
MANIFEST.in Move "public" to "include/brotli" 2016-08-23 14:40:33 +02:00
premake5.lua Remove the underscore in the name of brotli libraries. #326 2016-10-12 18:24:04 +02:00
README.md Fix build instructions for cmake 2016-10-19 21:44:42 +02:00
setup.cfg Python: Update README with information about code formatting 2016-10-17 13:57:56 -04:00
setup.py Python: Create native brotli module and move extension to _brotli 2016-10-17 09:35:27 -04:00
WORKSPACE Add Java port of Brotli decoder. 2016-10-17 14:04:59 +02:00

Introduction

Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression.

The specification of the Brotli Compressed Data Format is defined in RFC 7932.

Brotli is open-sourced under the MIT License, see the LICENSE file.

Brotli mailing list: https://groups.google.com/forum/#!forum/brotli

Build Status

Build instructions

Make

To build and run tests, simply do:

$ ./configure && make

If you want to install brotli, use one of the more advanced build systems below.

Bazel

See Bazel

CMake

The basic commands to build, test and install brotli are:

$ mkdir out && cd out && cmake .. && make
$ make test
$ make install

You can use other CMake configuration. For example, to build static libraries and use a custom installation directory:

$ mkdir out-static && \
  cd out-static && \
  cmake .. -DBUILD_SHARED_LIBS=0 -DCMAKE_INSTALL_PREFIX='/my/install/dir/'
$ make install

Premake5

See Premake5

Python

The basic commands to build, test, and install the Python module are:

$ python setup.py build_ext test
$ python setup.py install

See the Python readme for more details.

Benchmarks

Independent decoder implementation by Mark Adler, based entirely on format specification.

JavaScript port of brotli decoder. Could be used directly via npm install brotli