Commit Graph

872 Commits

Author SHA1 Message Date
Eugene Kliuchnikov
4363f2d74b Speedup Java decoder. (#547)
* geo corpus decodes ~5% faster
 * fetchlogs corpus decodes ~25% faster
2017-05-07 17:13:03 +02:00
Stefan Bodewig
a015b42683 turn java library into an OSGi bundle (#545) 2017-05-04 20:27:42 +02:00
Eugene Kliuchnikov
d00ccae57f Split auto-CMake and plain CMake build manual 2017-04-28 13:16:59 +02:00
Eugene Kliuchnikov
6ece1d8791 Move files & update paths (#541)
* Move files & update paths

* Rename build to scripts.

* Fix paths

* Fix script.
2017-04-23 14:07:08 +02:00
Eugene Kliuchnikov
04de756ad2 Simplify go brotli wrapper. (#540)
Based on PR #533.

Kudos to Bryan (bcmillis@).
2017-04-13 20:05:36 +02:00
Eugene Kliuchnikov
f2aa4d1e8c Add C# transpilation script. (#538) 2017-04-10 21:16:08 +02:00
Eugene Kliuchnikov
66e798d46a Update API to v1.0.0 (#537)
Make Java decoder fully transpilable to C#.
2017-04-10 15:39:00 +02:00
Eugene Kliuchnikov
46c1a881b4 Pull down version for v0.6.0 release 2017-04-10 10:42:24 +02:00
Eugene Kliuchnikov
21c118ba77 Update c- and java-decoder: (#536)
* speedup java decoder
   * avoid masking
   * avoid excessive fillBits
   * streamline uncompressed block processing
 * make java decoder more transpilation-friendly
 * avoid non-essential goto in c-decoder
2017-04-05 18:50:01 +02:00
Eugene Kliuchnikov
e12a7a2d28 Add Brotli logo to README head (#535) 2017-04-04 13:21:36 +02:00
Eugene Kliuchnikov
7a09531f69 Cleanup 2017-03-30 16:50:06 +02:00
Eugene Kliuchnikov
e5b7c16b98 Same file name is not permitted overall! (#532) 2017-03-24 14:49:02 +01:00
Eugene Kliuchnikov
e77799b084 Fix bintray release structure (#531)
Same file name is not allowed across packages in one version.
2017-03-24 14:31:30 +01:00
Eugene Kliuchnikov
51d6780b04 Actually publich artifacts to Bintray (#530) 2017-03-24 13:40:30 +01:00
Eugene Kliuchnikov
6715130c24 Fix bintray json (#529) 2017-03-24 13:03:16 +01:00
Eugene Kliuchnikov
187904a4b8 Upload binaries to bintray (#528) 2017-03-24 12:54:20 +01:00
Eugene Kliuchnikov
29ad4db47d Break build on sha256sum mismatch (#527) 2017-03-23 16:23:57 +01:00
Eugene Kliuchnikov
22421ebe30 Bazel build on linux/osx (#526) 2017-03-23 13:35:53 +01:00
Eugene Kliuchnikov
ee5c719057 Build and test java decoder with Maven 2017-03-22 19:13:59 +01:00
Eugene Kliuchnikov
a657d9969d Add go wrapper, streamline java decoder: (#524)
* add (c)brotli golang wrapper
 * remove (language-specific) enums in java decoder
2017-03-22 12:41:19 +01:00
Eugene Kliuchnikov
8a06e02935 Better compression (#523)
Better compression:
 * use more complex content modeling on 1MiB+ files
2017-03-21 16:08:23 +01:00
Eugene Kliuchnikov
1ff78b877f Prevent fuzzer timeouts on compression-bomb samples (#522)
* Prevent fuzzer timeouts on compression-bomb samples.

* Fix fuzzer lanucher
2017-03-10 16:01:49 +01:00
Eugene Kliuchnikov
52ce8670eb Fix typos (#521) 2017-03-09 17:34:16 +01:00
Eugene Kliuchnikov
cdca91b6f5 Update common, decoder, encoder, java (#520)
Common:
 * wrap dictionary data into `BrotliDictionary` structure
 * replace public constant with getter `BrotliGetDictionary`
 * reformat dictionary data

Decoder:
 * adopt common changes
 * clarify acceptable instance usage patterns
 * hold reference to dictionary in state

Encoder:
 * adopt common changes
 * eliminate PIC spots in `CreateBackwardReferences`
 * add per-chunk ratio guards for q0 and q1
 * precompute relative distances to avoid repeated calculations
 * prostpone hasher allocation/initialization
 * refactor Hashers to be class-like structure
 * further improvements for 1MiB+ inputs
 * added new hasher type; made hashers more configurable

Java:
 * Pull byte->int magic to `IntReader` from `BitReader`
2017-03-06 14:22:45 +01:00
Ian Duncan
aaa4424d9b Fix CMakeLists.txt specifying a nonexistent pkgconfig package (#518) 2017-03-01 16:19:57 +01:00
Eugene Kliuchnikov
c931e576d2 Move java/ to java/org/brotli/ to fix sources.jar structure (#517)
Also added man pages to `docs/`
2017-02-28 16:59:52 +01:00
Eugene Kliuchnikov
aaac88a1e0 Switch to 0.2.0-SNAPSHOT (#515) 2017-02-20 16:16:45 +01:00
Eugene Kliuchnikov
527db7af8c Release org.brotli.* 0.1.0 (#514) 2017-02-20 15:51:48 +01:00
Eugene Kliuchnikov
56a7fda830 Java: fix typos and adjust visibility. (#513) 2017-02-20 14:04:55 +01:00
Evan Nemerson
d03c38da7f Blacklist PGI from using conformant array parameters. (#511)
* Blacklist PGI from using conformant array parameters.

There is a bug in pgcc with conformant array parameters where the
length argument is a pointer which triggers a compiler error
(PGC-S-0094, to be specific).  The issue has been reported to PGI and
is being tracked internally as TPR 23778.  For more information, see

  https://www.pgroup.com/userforum/viewtopic.php?t=5501

* travis: Add PGI Community Edition build.

For details on the installation script, see
https://github.com/nemequ/pgi-travis
2017-02-19 10:06:13 +01:00
Eugene Kliuchnikov
53366083e0 Prepare org.brotli.dec for deployment. (#512) 2017-02-17 15:39:34 +01:00
Eugene Kliuchnikov
9fa1ad5a91 Fix "zero-distance-code", take 2 (#506) 2017-02-08 21:14:01 +01:00
Eugene Kliuchnikov
0749d9ca8b Fix #502 decoder bug (#505)
Decoder may have produced invalid output if:
 * at offset 0..3 dictionary word with index 3..0 for some length N is used
   and distance is encoded with direct distance code 0, and
 * at least one of next 4 commands use value from distance ringbuffer
2017-02-07 15:35:03 +01:00
Eugene Kliuchnikov
11df843cf0 Update encoder (#504)
* pull `BROTLI_MAX_BACKWARD_LIMIT` to constants
 * split generic and Zopfli backward references code
 * pull hashers init and stitch invocation to encoder
 * make `dictionary_hash` a compilation unit
 * add `size hint` parameter
 * add new hasher
 * use `size hint` to pick new hasher for q4
 * modernize clz guard (fix #495)
 * move `hash to binary tree` to separate file
 * add `Initialize` and `Cleanup` to all hashers
 * do not raise OOM if malloc(0) == NULL (fix #500)
2017-02-06 14:20:43 +01:00
Eugene Kliuchnikov
8d3fdc1dfe Update encoder (#497)
* pad dictionary LUTs to length 32, etc. (#493)
 * avoid using INFINITY constant (#496)
 * make dictionary_hash.h more compact
 * add "disable literal context modelling" parameter
2017-01-26 11:32:18 +01:00
Eugene Kliuchnikov
7e347a7c84 Update encoder (#492)
* fix comment position in `context.h`
 * fix typo in internal quality constant name
 * deduplicate `BuildMetaBlockGreedy` code
 * simplify aggregation in `ChooseContextMap`
2016-12-22 15:55:05 +01:00
Eugene Kliuchnikov
27d94590a2 Research (#491)
* add advanced mode for optimal references generator
 * fix #489

Thanks to Ivan Nikulin for working on it.
2016-12-22 13:03:28 +01:00
Eugene Kliuchnikov
fe9f9a9182 Split brotli common/dec/enc .pc files (#490)
Add URL, and use DEPENS_PRIVATE generator params
2016-12-22 08:57:44 +01:00
Alex Nicksay
6ab0a5cee7 Python: Create Makefile for development shortcuts (#488) 2016-12-21 10:17:11 +01:00
Eugene Kliuchnikov
fd96151b2a Move brotlidump.py to research/ (#487) 2016-12-20 18:00:51 +01:00
Eugene Kliuchnikov
5814438791 Add configure-cmake (#474) (#486)
* Add configure-cmake

 * `curl https://raw.githubusercontent.com/nemequ/configure-cmake/7b0464af79bbaca535f0279316558e1d84e5c124/configure > configure-cmake`

* Add `--disable-shared-libs` parameter.

* Unix-friendly script prologue.

* Update README.md
2016-12-20 17:45:40 +01:00
jneb
f62cd2bcd2 brotlidump.py: disassemble brotli file (revisited) (#314)
* Create brotlidump.py

Sorry, I am a newbie. I couldn't find my file anymore when I wanted to edit it. Hope I don't waste your time.

* Fixed a bug where it couldn't read its own compression.

The problem was that a prefix code ending with a 16 "repeat" didn't realize the table was full already.
Also minor bug fixes, comments and stuff.

* Major refactoring

Rewrote almost everything.
Now can dump its own compression.

* Now more or less complete

Appears to handle all files completely (including metablock data).
Used as inspiration for the the hex example (see makehexexample.py)
2016-12-20 14:41:47 +01:00
Alex Nicksay
89a5b6e625 Python: Simplify test suite generation by using unittest discovery (#485) 2016-12-20 14:40:47 +01:00
Alex Nicksay
6f227228ce Python: Use a temporary directory for generated files in tests (#481) 2016-12-12 10:28:44 +01:00
Alex Nicksay
4651f7c000 Python: Format bro.py with yapf (#480) 2016-12-12 10:28:15 +01:00
Eugene Kliuchnikov
0ee416139f Update python brotli wrapper (#479)
* Update python brotli wrapper
 * release GIL on CPU intensive blocks, fixes #476
 * use BrotliDecoderTakeOutput (less memory, less memcpy)

* Python: Convert bro.py tests to unittest style (#478)

* Create unittest-style tests for `bro.py` decompression and compression
* Delete old tests for `bro.py`
* Update test method generation to properly create a Cartesian product
  of iterables using `itertools.product`

* Update python brotli wrapper
 * release GIL on CPU intensive blocks, fixes #476
 * use BrotliDecoderTakeOutput (less memory, less memcpy)
2016-12-12 10:27:13 +01:00
Alex Nicksay
4a60128c13 Python: Convert bro.py tests to unittest style (#478)
* Create unittest-style tests for `bro.py` decompression and compression
* Delete old tests for `bro.py`
* Update test method generation to properly create a Cartesian product
  of iterables using `itertools.product`
2016-12-09 13:44:05 +01:00
Frank Denis
50bc3a7145 Do not assume that bash is installed in /bin (#477)
This is required in order to run the tests on *BSD.
2016-12-09 08:58:13 +01:00
Eugene Kliuchnikov
ccabf811ff Added fuzzer and updated decoder (#475)
* log dictionary usage
 * remove dead assignment
 * added fuzzer for https://github.com/google/oss-fuzz
 * added standalone test for fuzzer
2016-12-08 12:55:18 +01:00
Eugene Kliuchnikov
222564a95d Fix encoder (#472)
* fix undefined behavior introduced with PR #468
2016-12-02 13:32:20 +01:00