Commit Graph

948 Commits

Author SHA1 Message Date
Eugene Kliuchnikov
8544ae858d
Update (#680)
* fix MSVC warnings
 * cleanups
2018-06-09 11:17:13 +02:00
Eugene Kliuchnikov
1e7ea1d8e6
Inverse bazel project/workspace tree (#677)
* Inverse bazel workspace tree.

Now each subproject directly depends on root (c) project.

This helps to mitigate Bazel bug bazelbuild/bazel#2391; short summary:
Bazel does not work if referenced subproject `WORKSPACE` uses any
repositories that embedding project does not.

Bright side: building C project is much faster;
no need to download closure, go and JDK...
2018-06-04 17:53:16 +02:00
Eugene Kliuchnikov
29dc2cce90
Update golang and JS Bazel plugins to latest stable versions. (#676) 2018-05-31 13:21:04 +02:00
davidlt
f9b8c02673 Add RISC-V 64-bit (riscv64) platform configuration (#669)
Signed-off-by: David Abdurachmanov <david.abdurachmanov@gmail.com>
2018-05-22 14:35:04 +02:00
Eugene Kliuchnikov
48a25b3fa4
Fix #671 (#672) 2018-05-18 22:07:52 +02:00
Eugene Kliuchnikov
a4581c158e
Add tools to download and transform static dictionary data. (#670) 2018-05-16 12:59:09 +02:00
Eugene Kliuchnikov
f5ed35d065
Update (#664)
* Update
 * fix ifdef style
 * get back to fine-compiler-version-based-macros (use Hedley)
 * fix q=0 histogram collection for very long copy/insert commands
2018-05-03 11:16:21 +02:00
Cosimo Lupo
f94cd51b5c appveyor: fix issue self-upgradig pip to v10 (#663)
Installing with --user will leave the old pip.exe script in the $PATH,
but running this will fail because pip 10 moved 'main' to internal
modules.

https://github.com/pypa/pip/issues/5240#issuecomment-382989420
2018-04-20 19:31:51 +02:00
Eugene Kliuchnikov
6000396155
Remove unprefixed macros from public headers (#662) 2018-04-20 14:10:55 +02:00
Eugene Kliuchnikov
68db5c0272
Update (#660)
* Update
 * improve q=1 compression on small files
 * fix "left shift before promotion"
 * fix osx Travis builds
2018-04-13 11:44:34 +02:00
Eugene Kliuchnikov
c6333e1e79
Fix MSVC compilation (#657)
* tell bazel not to pass strict options to a fancy compiler
 * fix signed-unsigned comparison warning found by MSVC
2018-03-29 10:37:07 +02:00
Eugene Kliuchnikov
0f3c84e745
Update (#656)
* proper fix for the "fall through" warning"
 * automatic NDIRECT/NPOSTFIX tuning (better compression)
 * fix unaligned access for `aarch64`-cross-`armhf` build
 * fix `aarch64` detection (10% decoder speedup)
 * expose `large_window` CLI option
 * make default window size 16MiB
 * ramp up version to 1.0.4
2018-03-27 22:29:22 +02:00
Adrian Perez
515fc62313 Tell CMake to not check for a C++ compiler (#653)
By default CMake checks both for C and C++ compilers, while the latter
is not needed. Setting the list of languages to just "C" in the call to
project() removes the unneeded check.
2018-03-26 21:41:18 +06:00
Eugene Kliuchnikov
2c03482569
Fix "memory leak" in python tests (#652)
OOMs on RPi (1GB)
2018-03-23 02:09:00 +06:00
Tobe O
a238f5bac9 Update README.md (#646)
Add mention of Dart native bindings
2018-03-20 17:53:32 +06:00
Eugene Kliuchnikov
631fe194a1
Update (#651)
* fix `bazel` build (ignore switch case fall-through)
* add `NPOSTFIX` / `NDIRECT` encoder parameters
* fix source file lists (add `params.h`)
* fix bug in `durchschlag`
* print clarifying messages wheb CLI argument parsing fails
2018-03-20 17:37:41 +06:00
Eugene Kliuchnikov
533843e354
Update (#643)
Update
 * make the zopflification aware of `NDIRECT`, `NPOSTFIX`
   (better compression in `font` mode)
 * add small and simple decoder tool
 * fix typo
 * Java: wrapper: make decoder channel more async-friendly

Ramp up version to 1.0.3 / 1.0.3
2018-03-02 15:49:58 +01:00
Eugene Kliuchnikov
35e69fc7cf
New feature: "Large Window Brotli" (#640)
* New feature: "Large Window Brotli"

By setting special encoder/decoder flag it is now possible to extend
LZ-window up to 30 bits; though produced stream will not be RFC7932
compliant.

Added new dictionary generator - "DSH". It combines speed of "Sieve"
and quality of "DM". Plus utilities to prepare train corpora
(remove unique strings).

Improved compression ratio: now two sub-blocks could be stitched:
the last copy command could be extended to span the next sub-block.

Fixed compression ineffectiveness caused by floating numbers rounding and
wrong cost heuristic.

Other C changes:
 - combined / moved `context.h` to `common`
 - moved transforms to `common`
 - unified some aspects of code formatting
 - added an abstraction for encoder (static) dictionary
 - moved default allocator/deallocator functions to `common`

brotli CLI:
 - window size is auto-adjusted if not specified explicitly

Java:
 - added "eager" decoding both to JNI wrapper and pure decoder
 - huge speed-up of `DictionaryData` initialization

* Add dictionaryless compressed dictionary

* Fix `sources.lst`

* Fix `sources.lst` and add a note that `libtool` is also required.

* Update setup.py

* Fix `EagerStreamTest`

* Fix BUILD file

* Add missing `libdivsufsort` dependency

* Fix "unused parameter" warning.
2018-02-26 09:04:36 -05:00
Eugene Kliuchnikov
3af18990f5
Update go and closure bazel rules (#637)
* Update go and closure bazel rules
* Follow the new bazel go rules guide
* Swap go & closure rules initialization
* Update bazel to 0.10.0 in appveyor build
2018-02-08 14:38:10 +01:00
Daniel Chýlek
b5033d0e1e Fix brotlidump.py crashing when complex prefix code has exactly 1 non-zero code length (#635)
According to the format specification regarding complex prefix codes:

> If there are at least two non-zero code lengths, any trailing zero
> code lengths are omitted, i.e., the last code length in the
> sequence must be non-zero.  In this case, the sum of (32 >> code
> length) over all the non-zero code lengths must equal to 32.

> If the lengths have been read for the entire code length alphabet
> and there was only one non-zero code length, then the prefix code
> has one symbol whose code has zero length.

The script does not handle a case where there is just 1 non-zero code
length where the sum rule doesn't apply, which causes a StopIteration
exception when it attempts to read past the list boundaries.

An example of such file is tests/testdata/mapsdatazrh.compressed. I made
sure this change doesn't break anything by processing all *.compressed
files from the testdata folder with no thrown exceptions.
2018-02-08 12:48:24 +01:00
Eugene Kliuchnikov
da254cffdb
Update (#630)
* merge {dec|enc}/port.h into common/platform.h
 * fix one-shot q=10 1-byte input compression
 * fix some unprefixed definitions
 * make hashers host-endianness-independent
 * extract enc/params.h from enc/quality.h
 * fix API documentation / typos
 * improve `BrotliEncoderMaxCompressedSize`
2017-12-12 14:33:12 +01:00
Jeremy Bicha
63e15bb3a6 Don't set rpath (#629) 2017-12-07 20:39:07 +01:00
Bernard Spil
62194f204d Work around Linuxisms (#627)
Missed this in my previous tests. Sorry for that.

On BSDs, both bc and sed are part of the base operating system. For sed this results in an error as the check construct (--version) is a GNU-ism and only works for GNU sed, not for bsd sed.
Similarly, BSD sed does not take parameters after the filename(s) operated on. Moving `-i` to the front fixes that. `-r` is provided for GNU compat in BSD sed as an alias of `-E`. The `-i` option in BSD sed requires an extension to work in-place.

(thank you for picking up the nginx module too!)
2017-12-04 15:17:49 +01:00
Eugene Kliuchnikov
2d6b298e11
Update Travis matrix (#626)
* Use Clang-5.0
* Disable unholy ASAN leak detector (to unbreak build)
* Reduce build matrix and use faster env, where compiler version is not important
* Add autotools build to Travis matrix
2017-11-30 20:54:04 +01:00
Eugene Kliuchnikov
c8c8389ed3
Do not rely on bash arithmetic in bootstrap (#625) 2017-11-30 11:02:54 +01:00
Bernard Spil
1ca15159d6 Fix missing symbols errors in libbrotlienc and dec (#623)
When using autotools to build the binary and libraries, the resulting libraries don't link `brotlicommon` or `m`.
This was detected when building cURL 7.57.0 which has now has brotli support. During configure it was failing
```
checking run-time libs availability... failed
configure: error: one or more libs available at link-time are not available run-time. Libs used at link-time: -lbrotlidec -lz -L/usr/local/lib
```
inspection of config.log showed missing symbols from libbrotlicommon as the cause.

This patch results in the encryption and decryption libs to be properly linked against libbrotlicommon and libm.
See also https://bugs.freebsd.org/223966
2017-11-29 22:38:16 +01:00
Eugene Kliuchnikov
0ad94eed00
Update (#620)
* add autotools build
* separate semantic and ABI version
* extract sources.lst (used by CMake and Automake)
* share pkgconfig templates (used by CMake and Automake)
* decoder: always set `total_out`
* encoder: fix `BROTLI_ENSURE_CAPACITY` macro (no-op after preprocessor)
* decoder/encoder: refine `free_func` contract
2017-11-28 15:37:28 +01:00
Lode Vandevenne
273de5a22f
Update shared-brotli-fetch-spec.txt 2017-11-17 01:38:05 +01:00
Lode Vandevenne
a755ba3bd0
Update shared-brotli-fetch-spec.txt 2017-11-16 19:15:13 +01:00
Lode Vandevenne
bdda95ee55
Create shared-brotli-fetch-spec.txt 2017-11-16 19:13:39 +01:00
Eugene Kliuchnikov
3e58ea5f90 Update (#617)
* remove `const` on `BrotliDictionary` members
 * extend `ZofliNode` distance range to 128MiB
 * add missing `port.h` include to `quality.h`
 * fix typo in encoder API-doc
 * regenerate `decode.min.js`
2017-10-13 14:50:51 +02:00
Eugene Kliuchnikov
39ef4bbdcf Add new (fast) dictionary generator engine. (#616)
Add CLI for dictionary generation.
Add BUILD file for research folder
2017-10-13 11:25:03 +02:00
Eugene Kliuchnikov
9c75a2a26a Use bazel in appveyor (#612)
+publish jni dll
2017-10-11 22:26:37 +02:00
Tomáš Popela
a0c7dafe28 Fix permissions of various files in project (#613)
Move from 755 to 644.
2017-10-10 11:24:13 +02:00
Eugene Kliuchnikov
42d78807bb Improve Bazel/JNI portability (#611)
* Improve Bazel/JNI portability
* Update go and closure bazel addons
2017-10-09 17:07:34 +02:00
Eugene Kliuchnikov
4f8cd4c0f4 Fix fuzzer test script and add it to travis matrix (#606) 2017-09-26 13:49:30 +02:00
Eugene Kliuchnikov
5b4769990d Ramp up to version to 1.0.1 2017-09-22 14:05:06 +02:00
Eugene Kliuchnikov
bf6a6cda56 Fix parallel test execution 2017-09-22 13:13:22 +02:00
Eugene Kliuchnikov
7748a1dc69 Update README.md 2017-09-22 10:28:15 +02:00
Eugene Kliuchnikov
c60563591a Fix API documentation + theoretical NPEs (#602) 2017-09-20 15:02:01 +02:00
Eugene Kliuchnikov
b6a017492e Install static libraries as well (#601) 2017-09-20 10:04:06 +02:00
Eugene Kliuchnikov
37fb83ec0d Update: (#600)
* encoder: relax backward references candidates asserts
 * encoder: make RNG more platform-independent
 * encoder: remove "unused" param (context mode)
 * CLI: improve first-encounter experience
 * Java: update SynthTest
 * Java: refine proguard config
 * Java/JNI: fix one-shot compression workflow
2017-09-19 15:57:15 +02:00
Eugene Kliuchnikov
61a5015938 Update README.md 2017-09-19 11:29:55 +02:00
Eugene Kliuchnikov
4760f7db47 Update MANIFEST.in 2017-09-19 11:25:55 +02:00
Eugene Kliuchnikov
248bddd0d7 Update README.md 2017-09-19 10:46:10 +02:00
Eugene Kliuchnikov
52f0483332 Build both static and shared libs with CMake (#599) 2017-09-19 09:40:48 +02:00
Eugene Kliuchnikov
87b43eb61b Reduce / update travis build matrix. (#598) 2017-09-18 13:52:53 +02:00
Eugene Kliuchnikov
6b1d0ab53d CI config
* Appveyor: publish artifacts on bintray
* Appveyor & Travis: build only master branch
2017-09-18 13:05:47 +02:00
Eugene Kliuchnikov
26a34a435c Employ make/gcc on Appveyor + push artifacts (#596) 2017-09-14 16:14:05 +02:00
Eugene Kliuchnikov
d7bce1e092 Update (#593)
* Update:
 * fix CLI error messages
 * fix CLI console IO on Windows
2017-09-07 20:27:49 +02:00