Commit Graph

5 Commits

Author SHA1 Message Date
Eugene Kliuchnikov
35e69fc7cf
New feature: "Large Window Brotli" (#640)
* New feature: "Large Window Brotli"

By setting special encoder/decoder flag it is now possible to extend
LZ-window up to 30 bits; though produced stream will not be RFC7932
compliant.

Added new dictionary generator - "DSH". It combines speed of "Sieve"
and quality of "DM". Plus utilities to prepare train corpora
(remove unique strings).

Improved compression ratio: now two sub-blocks could be stitched:
the last copy command could be extended to span the next sub-block.

Fixed compression ineffectiveness caused by floating numbers rounding and
wrong cost heuristic.

Other C changes:
 - combined / moved `context.h` to `common`
 - moved transforms to `common`
 - unified some aspects of code formatting
 - added an abstraction for encoder (static) dictionary
 - moved default allocator/deallocator functions to `common`

brotli CLI:
 - window size is auto-adjusted if not specified explicitly

Java:
 - added "eager" decoding both to JNI wrapper and pure decoder
 - huge speed-up of `DictionaryData` initialization

* Add dictionaryless compressed dictionary

* Fix `sources.lst`

* Fix `sources.lst` and add a note that `libtool` is also required.

* Update setup.py

* Fix `EagerStreamTest`

* Fix BUILD file

* Add missing `libdivsufsort` dependency

* Fix "unused parameter" warning.
2018-02-26 09:04:36 -05:00
Cosimo Lupo
4f455cac32 disable buidling/deployment of python wheels (#583)
* [appveyor] remove 'deploy' stage; only test python 2.7 and 3.6

all the other python versions are being built and tested on
https://github.com/google/brotli-wheels/blob/d571d63/appveyor.yml

* remove terrify submodule as not needed any more

* [travis] just test py2.7 and 3.6 on linux; remove extra osx python builds

All the other python versions for OSX are being built/tested on:
https://github.com/google/brotli-wheels/blob/d571d63/.travis.yml

Also, there's no need to build and deploy wheels here, as that's done
in the separate repository.

* [setup.py] only rebuild if dependency are newer; fix typo in list of 'depends'

https://github.com/python/cpython/blob/v3.6.2/Lib/distutils/command/build_ext.py#L485-L500

* [ci] only run 'python setup.py test'

if we run 'python setup.py built test', the setuptools 'test' command will
forcibly re-run the build_ext subcommand because it wants to pass the --inplace
option (it ignores whether it's up to date, just re-runs it all the time).

with this we go from running built_ext twice, to running it only once per build

* [Makefile] run 'build_ext --inplace' instead of 'develop' as default target

The 'develop' command is like 'install' in the sense that it
modifies the user's python environment.
The default make target should be less intrusive, i.e. just building
the extension module in-place without modify anything in the user's
environment.

We don't need to tell make about the dependency between 'test' and
'build' target as that is baked in the `python setup.py test` command.

* [Makefile] add 'develop' target; remove unnecessary 'tests' target

`make test` is good enough

* [Makefile] `setup.py test` requires setuptools; run `python -m unittest`

This will work even if setuptools is not installed, which is unlikely
nowadays but still our `setup.py` works with plain distutils, so
we may well have our tests work without setuptools.

* [python/README.md] add ref to 'develop' target; remove 'tests', just 'make test'

* [setup.py] import modules as per nicksay's comment

https://github.com/google/brotli/pull/583#discussion_r131981049

* [Makefile] add 'develop' to .PHONY targets

remove 'tests' from .PHONY

* [appveyor] remove unused setup scripts

We don't need to install custom python versions, we are
using the pre-installed ones on Appveyor.

* [appveyor] remove unneeded setup code
2017-08-23 20:45:13 +02:00
Eugene Kliuchnikov
6ece1d8791 Move files & update paths (#541)
* Move files & update paths

* Rename build to scripts.

* Fix paths

* Fix script.
2017-04-23 14:07:08 +02:00
Ivan Nikulin
9294022929 Replace sais.hxx by submodule hillbig/esaxx. 2016-09-19 19:12:30 +02:00
Cosimo Lupo
bb3d9248c0 add support for Travis CI (OS X only for now)
Use 'terrify' utilities from MacPython to build precompiled wheel packages
2015-08-14 11:28:26 +01:00