Commit Graph

1085 Commits

Author SHA1 Message Date
Eugene Kliuchnikov
fe9f9a9182 Split brotli common/dec/enc .pc files (#490)
Add URL, and use DEPENS_PRIVATE generator params
2016-12-22 08:57:44 +01:00
Alex Nicksay
6ab0a5cee7 Python: Create Makefile for development shortcuts (#488) 2016-12-21 10:17:11 +01:00
Eugene Kliuchnikov
fd96151b2a Move brotlidump.py to research/ (#487) 2016-12-20 18:00:51 +01:00
Eugene Kliuchnikov
5814438791 Add configure-cmake (#474) (#486)
* Add configure-cmake

 * `curl https://raw.githubusercontent.com/nemequ/configure-cmake/7b0464af79bbaca535f0279316558e1d84e5c124/configure > configure-cmake`

* Add `--disable-shared-libs` parameter.

* Unix-friendly script prologue.

* Update README.md
2016-12-20 17:45:40 +01:00
jneb
f62cd2bcd2 brotlidump.py: disassemble brotli file (revisited) (#314)
* Create brotlidump.py

Sorry, I am a newbie. I couldn't find my file anymore when I wanted to edit it. Hope I don't waste your time.

* Fixed a bug where it couldn't read its own compression.

The problem was that a prefix code ending with a 16 "repeat" didn't realize the table was full already.
Also minor bug fixes, comments and stuff.

* Major refactoring

Rewrote almost everything.
Now can dump its own compression.

* Now more or less complete

Appears to handle all files completely (including metablock data).
Used as inspiration for the the hex example (see makehexexample.py)
2016-12-20 14:41:47 +01:00
Alex Nicksay
89a5b6e625 Python: Simplify test suite generation by using unittest discovery (#485) 2016-12-20 14:40:47 +01:00
Alex Nicksay
6f227228ce Python: Use a temporary directory for generated files in tests (#481) 2016-12-12 10:28:44 +01:00
Alex Nicksay
4651f7c000 Python: Format bro.py with yapf (#480) 2016-12-12 10:28:15 +01:00
Eugene Kliuchnikov
0ee416139f Update python brotli wrapper (#479)
* Update python brotli wrapper
 * release GIL on CPU intensive blocks, fixes #476
 * use BrotliDecoderTakeOutput (less memory, less memcpy)

* Python: Convert bro.py tests to unittest style (#478)

* Create unittest-style tests for `bro.py` decompression and compression
* Delete old tests for `bro.py`
* Update test method generation to properly create a Cartesian product
  of iterables using `itertools.product`

* Update python brotli wrapper
 * release GIL on CPU intensive blocks, fixes #476
 * use BrotliDecoderTakeOutput (less memory, less memcpy)
2016-12-12 10:27:13 +01:00
Alex Nicksay
4a60128c13 Python: Convert bro.py tests to unittest style (#478)
* Create unittest-style tests for `bro.py` decompression and compression
* Delete old tests for `bro.py`
* Update test method generation to properly create a Cartesian product
  of iterables using `itertools.product`
2016-12-09 13:44:05 +01:00
Frank Denis
50bc3a7145 Do not assume that bash is installed in /bin (#477)
This is required in order to run the tests on *BSD.
2016-12-09 08:58:13 +01:00
Eugene Kliuchnikov
ccabf811ff Added fuzzer and updated decoder (#475)
* log dictionary usage
 * remove dead assignment
 * added fuzzer for https://github.com/google/oss-fuzz
 * added standalone test for fuzzer
2016-12-08 12:55:18 +01:00
Eugene Kliuchnikov
222564a95d Fix encoder (#472)
* fix undefined behavior introduced with PR #468
2016-12-02 13:32:20 +01:00
Piotr Sikora
6a4bf43968 Fix build with -Wconditional-uninitialized. (#471)
Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2016-12-02 09:52:54 +01:00
Eugene Kliuchnikov
396309a529 Update (#470)
* condense generated `static_dict_lut.h`
 * implement BrotliInputStream.close()
2016-11-30 13:36:20 +01:00
Eugene Kliuchnikov
5db62dcc9d Fixes: (#468)
* fix slow-down after a long copy (q10-11)
 * more thorough hashing for long ranges (q10-11)
 * minor documentation fixes
 * bazel.io -> bazel.build
2016-11-09 14:04:09 +01:00
Alex Nicksay
1e5ea6aedd Python: Add unit tests for brotli.compress and brotli.decompress (#467)
Also
  - rename `test_utils` to `_test_utils`
  - refactor shared code into `_test_utils`
2016-11-09 12:21:13 +01:00
Evan Nemerson
12750768c2 bro: check return values of chown and chmod (#465)
Apparently some libc versions declare chown with the warn_unused_result
attribute, which is enabled by default.
2016-11-02 14:03:06 +01:00
Evan Nemerson
6c47009892 FInishing touches for installing libbrotli with CMake (#464)
* build: fix bundled mode + BUILD_SHARED_LIBS

* cmake: add soversion information

* cmake: generate pkg-config file
2016-11-01 10:03:29 +01:00
Eugene Kliuchnikov
e9b278ac6e Update docs and add more java tests (#463)
* doxygenize and update API documentation
 * fix spelling
 * add "fuzz" corpus for java decoder to improve coverage
 * use upper-case-snake names for dictionary constant definitions
 * use `LDFLAGS` in conventional `Makefile`
2016-10-31 14:33:59 +01:00
Alex Nicksay
a260b6ba73 Python: Add tests for streamed compression (#458)
Progress on #191
2016-10-31 13:24:01 +01:00
Alex Nicksay
9203765492 Python: Use "build" instead of "build_ext" in scripts (#460)
Previously, the Python package consisted of a single extension
module, so `build_ext` was sufficient.  Now, the package
contains a native module and an extension module, so both
`build_py` and `build_ext` are required.  Instead, run `build`,
which calls both `build_py` and `build_ext` automatically.
2016-10-31 12:58:45 +01:00
Alex Nicksay
1a8ee40de9 Python: Run Appveyor tests in CMD mode (#461)
Any command executed in PowerShell mode that writes to `stderr`
is treated as failing.  To avoid this problem, run tests in CMD
mode instead.
2016-10-27 17:25:05 +02:00
Mo DeJong
3b9d4a227d enable rbit instruction for arm64 (#459) 2016-10-27 17:21:12 +02:00
Eugene Kliuchnikov
4e157c409a Update API (#457)
* explicitly define `BROTLI_BOOL` to be `int`
 * add `BROTLI_` prefix to `MAKE_UINT64_T` macros
 * replace `true`/`false`/`1`/`0` mentions with `BROTLI_TRUE`/`FALSE`
 * add `BrotliEncoderSetParameter` documentation
 * add explicit caution to `BrotliEncoderMaxCompressedSize`
 * fix formatting in `port.h`
2016-10-25 16:02:05 +02:00
Alex Nicksay
afb1272792 Python: Publicly expose the Compressor object in the Python API (#456)
Progress on #191
2016-10-25 10:19:29 +02:00
Alex Nicksay
5632315d35 Python: Support streamed compression with the Compressor object (#448)
This adds `flush` and `finish` methods to the `Compressor`
object in the extension module, renames the `compress` method to
`process`, and updates that method to only process data.  Now,
one or more `process` calls followed by a `finish` call will be
equivalent to a module-level `compress` call.

Note: To maximize the compression efficiency (and match
underlying Brotli behavior, the `Compressor` object `process`
method does not guarantee all input is immediately written to
output. To ensure immediate output, call `flush` to manually
flush the compression buffer.  Extraneous flushing can increase
the size, but may be required when processing streaming data.

Progress on #191
2016-10-24 13:28:56 +02:00
Eugene Kliuchnikov
678f8627d3 Fix OSX gcc-4.x compilation (#455)
Fix OSX gcc-4.x compilation
2016-10-20 14:16:00 +02:00
Eugene Kliuchnikov
b1db6f149a Fix -Wcast-align warnings 2016-10-20 10:28:44 +02:00
Eugene Kliuchnikov
74147a1a41 Merge pull request #454 from fred-wang/brotli-readme
Fix build instructions for cmake
2016-10-19 22:27:45 +02:00
Frédéric Wang
82c297f356 Fix build instructions for cmake 2016-10-19 21:44:42 +02:00
Eugene Kliuchnikov
058a113dd9 Merge pull request #451 from eustas/flush
Fix "take output" flush workflow.
2016-10-19 20:03:43 +02:00
Eugene Kliuchnikov
8bcaabb0d1 Fix "take output" flush workflow. 2016-10-19 16:19:26 +02:00
Eugene Kliuchnikov
1b364aeb42 Merge pull request #450 from eustas/master
Build shared libraries by default
2016-10-18 20:52:04 +02:00
Eugene Kliuchnikov
b93cb69831 * leave static compilation declaration intouch (e.g. Python build) 2016-10-18 17:14:49 +02:00
Eugene Kliuchnikov
f5ba0b6c17 (compress_fragment_two_pass) 2016-10-18 16:56:39 +02:00
Eugene Kliuchnikov
69982c25f1 Build shared libraries by default
* Declare `BUILD_SHARED_LIBS` option for CMake
* Define `${LIB}_SHARED_COMPILATION` when compiling shared library
* Define and use BROTLI_xxx_API
* Fix remaining unprefixed defines in port.h
2016-10-18 16:45:32 +02:00
Eugene Kliuchnikov
0781cb10ab Merge pull request #449 from eustas/master
Fix POM files sources paths
2016-10-18 15:29:39 +02:00
Eugene Kliuchnikov
d18c7369d9 Fix POM files sources paths
* also add javadocs and sources generation
2016-10-18 15:28:43 +02:00
Eugene Kliuchnikov
2d441179bb Merge pull request #446 from nicksay/py-3-compressor-object
Python: Create an extension Compressor object
2016-10-18 10:31:05 +02:00
Eugene Kliuchnikov
606a70b779 Merge pull request #447 from nicksay/py-yapf
Python: Update README with information about code formatting
2016-10-18 10:28:28 +02:00
Eugene Kliuchnikov
81962c3892 Merge pull request #444 from eustas/master
Eliminate more magic constants.
2016-10-18 10:27:22 +02:00
Alex Nicksay
b04f4ea185 Python: Update README with information about code formatting
Also, add a `yapf` section to `setup.cfg` to ensure YAPF runs
format code with the Google style.
2016-10-17 13:57:56 -04:00
Alex Nicksay
595a5246b4 Python: Create an extension Compressor object
- Create a `Compressor` object in the extension module
- Move the `compress` method into the native module and use
  the new `Compressor` object to do the compression

Note: This does not change the module-level Python API.  The
`Compressor` object will not be publicly exposed until its
methods have stabilized.
2016-10-17 13:03:58 -04:00
Eugene Kliuchnikov
d60aa23116 Merge pull request #443 from nicksay/py-2-package-structure
Python: Create native brotli module and move extension to _brotli
2016-10-17 17:40:32 +02:00
Eugene Kliuchnikov
9521d968f3 Eliminate more magic constants.
Author: Ivan Nikulin
2016-10-17 17:33:12 +02:00
Eugene Kliuchnikov
4219fece59 Merge pull request #424 from mdejong/master
check for __ARM64_ARCH_8__ in dec/port.h so that arm64 arch under cla…
2016-10-17 15:44:39 +02:00
Alex Nicksay
f7b5b3dc2c Python: Create native brotli module and move extension to _brotli 2016-10-17 09:35:27 -04:00
Eugene Kliuchnikov
541dd651e0 Merge pull request #435 from nicksay/py-1-cleanup-setup
Python: Clean up setup.py file
2016-10-17 14:31:42 +02:00
Eugene Kliuchnikov
d767ab9eb0 Merge pull request #439 from fred-wang/remove-underscore
Remove the underscore in the name of brotli libraries. #326
2016-10-17 14:29:38 +02:00