Commit Graph

1010 Commits

Author SHA1 Message Date
Dmitry Rozhkov
7e8e207ce2
Fix clang-10 compilation issue (#839)
clang-10 throws the following error:
In file included from external/org_brotli/c/enc/bit_cost.c:9:
external/org_brotli/c/enc/./bit_cost.h:48:16: error: implicit conversion
from 'size_t' (aka 'unsigned long') to 'double' may lose precision
[-Werror,-Wimplicit-int-float-conversion]
  if (retval < sum) {
             ~ ^~~
1 error generated.

Make the conversion explicit.
2020-09-07 09:40:03 +02:00
Eugene Kliuchnikov
09b0992b6a
Revert "Add runtime linker path to pkg-config files (#740)" (#838)
This reverts commit 31754d4ffc.
2020-09-02 11:38:26 +02:00
Evgenii Kliuchnikov
0545759b2e Address issues noted in #833 2020-08-28 10:14:08 +02:00
Evgenii Kliuchnikov
e61745a6b7 Re-release 2020-08-27 16:12:55 +02:00
Evgenii Kliuchnikov
db361a0bb9 Re-add python bindings to sources tarball 2020-08-27 16:01:44 +02:00
Eugene Kliuchnikov
d518e55ba7
Update README.md 2020-08-26 18:46:19 +02:00
Eugene Kliuchnikov
d052918255
Fix build files (#829) 2020-08-26 17:13:31 +02:00
Eugene Kliuchnikov
665e81dc9b
New version: 1.0.8 (#827) 2020-08-26 14:36:02 +02:00
Eugene Kliuchnikov
223d80cfbe
Update (#826)
* IMPORTANT: decoder: fix potential overflow when input chunk is >2GiB
 * simplify max Huffman table size calculation
 * eliminate symbol duplicates (static arrays in .h files)
 * minor combing in research/ code
2020-08-26 12:32:27 +02:00
Eugene Kliuchnikov
0c5603e07b
Fix output parameter type for _BitScanReverse (#819)
Fix #811
2020-07-09 16:40:01 +02:00
Eugene Kliuchnikov
e8155d67b0
CMake: change default ("") build type to Release (#818)
Fix #817
2020-07-09 15:35:57 +02:00
Eugene Kliuchnikov
fc823290a7
Mute strerror/strcpy warnings is MSVC build. (#815) 2020-07-02 19:45:57 +02:00
Eugene Kliuchnikov
5519352661
Add workaround for lying feof. (#814)
Should fix #812
2020-07-02 17:57:40 +02:00
fisherwky
d2ea198232
Update platform.h (#813)
fix compile error (platform.h:362: error: cast discards qualifiers from pointer target type)
2020-06-30 11:23:07 +02:00
Nils Goroll
31754d4ffc
Add runtime linker path to pkg-config files (#740)
Otherwise libraries will not be found at runtime when installing to a
path not included in the default runtime linker's path with programs
linking brotli configured via pkg-config.
2020-05-15 13:11:01 +02:00
OZone
8f093f5e84
.gitignore: Ignore .obj files (#805)
EDK II windows build produces .obj files in source tree
2020-05-15 13:05:03 +02:00
Eugene Kliuchnikov
f0db711f46
Filter sources for the tarball. (#808) 2020-05-15 13:04:17 +02:00
Eugene Kliuchnikov
7f740f1308
Update (#807)
- fix formatting
 - fix type conversion
 - fix no-op arithmetic with null-pointer
 - improve performance of hash_longest_match64
 - go: detect read after close
 - java decoder: support compound dictionary
 - remove executable flag on non-scripts
2020-05-15 11:06:21 +02:00
Eugene Kliuchnikov
f83aa5169e
Update bazel to 2.2 + update config (#798)
Newer bazel does not support `maven_jar` rule anymore...
2020-03-31 14:38:01 +02:00
Clinton Ingram
924b2b2b9d
Move TZCNT and BSR intrinsics to platform.h, add MSVC versions (#636) 2020-03-19 11:57:56 +01:00
Paul Vollmer
0503d8b766
Added go.mod file to go/cbrotli directory (#754)
* Added go.mod file

* go.mod removed go version
2020-03-19 11:54:51 +01:00
Cristi Vîjdea
f503cb709c
Add HAVE_LOG2 build macro (#783)
* Add HAVE_LOG2 build macro

Fixes #781

* Rename macro to BROTLI_HAVE_LOG2 and move comment for visibility
2020-03-19 10:46:52 +01:00
Leo Neat
36ac0feaf9
Adding CIFuzz (#797) 2020-03-19 09:52:07 +01:00
shenglei10
666c3280cc
Make types of variable match (#796) 2020-02-14 10:40:02 +01:00
agrieve
4b5771bee7 Add missing "const" to a couple of kConstants (#780)
These showed up in a Chromium audit:
https://bugs.chromium.org/p/chromium/issues/detail?id=747064#c8

Although already effectively const, adding "const" causes the symbols to
be moved into the read-only section of the binary.
2019-12-20 00:15:58 +01:00
Griffin Downs
c435f06675 Add vcpkg installation instructions (#776) 2019-10-01 22:53:10 +02:00
James Hilliard
5c3a9a937b Fix license in setup.py (#769) 2019-08-16 16:32:14 +02:00
Ammar Askar
afc4a74273 Add oss-fuzz fuzzing status badge to README (#767) 2019-08-13 15:49:30 +02:00
Eugene Kliuchnikov
35ef5c554d
Disable PIC in EMCC mode. (#768) 2019-08-13 15:23:04 +02:00
Eugene Kliuchnikov
ca21dac8e5
Add an option to avoid building shared libraries. (#766)
Add an option to avoid building shared libraries (for building with EMCC)

Drive-by:
* maven: ramp up java level to minimal required
* travis: replace deprecated clang-5.0 with clang-7
* maven: fallback to jdk10 to void javadoc bug
2019-08-07 10:51:55 +02:00
Eugene Kliuchnikov
3d1767186d
Fix include for EMCC build (#765) 2019-07-30 10:01:21 +02:00
Eugene Kliuchnikov
f1124c8524
More careful sanitizer detection (#764) 2019-07-22 14:29:51 +02:00
Eugene Kliuchnikov
c8b37e8fd1
Update (#762)
* put LICENSE file into .jar
 * fix typo
 * add clarification comment in PY wrapper
2019-07-17 14:39:56 +02:00
Eugene Kliuchnikov
40f0fdcdc1
Explicitly mark tests/testdata/* as binary. (#761)
Fixes #760

Drive-by:
 * update go_rules
 * modernize brotli_inc
 * fix wrapper build
 * update PY to 3 in Travis / OSX / Bazel build
 * upgrade JS Bazel rules.
2019-07-16 17:49:14 +02:00
Eugene Kliuchnikov
78e7bbc3c3
Update (#753)
* fix executable mode of decode.js
 * explain clang-analyser about non-nullability
 * fix "dead assignment"
 * rename proguard.cfg -> proguard.pgcfg
2019-05-03 11:51:11 +02:00
Eugene Kliuchnikov
4b2b2d4f83
Update (#749)
Update:

 * Bazel: fix MSVC configuration
 * C: common: extended documentation and helpers around distance codes
 * C: common: enable BROTLI_DCHECK in "debug" builds
 * C: common: fix implicit trailing zero in `kPrefixSuffix`
 * C: dec: fix possible bit reader discharge for "large-window" mode
 * C: dec: simplify distance decoding via lookup table
 * C: dec: reuse decoder state members memory via union with lookup table
 * C: dec: add decoder state diagram
 * C: enc: clarify access to static dictionary
 * C: enc: improve static dictionary hash
 * C: enc: add "stream offset" parameter for parallel encoding
 * C: enc: reorganize hasher; now Q2-Q3 require exactly 256KiB
           to avoid global TCMalloc lock
 * C: enc: fix rare access to uninitialized data in ring-buffer
 * C: enc: reorganize logging / checks in `write_bits.h`
 * Java: dec: add "large-window" support
 * Java: dec: improve speed
 * Java: dec: debug and 32-bit mode are now activated via system properties
 * Java: dec: demystify some state variables (use better names)
 * Dictionary generator: add single input mode
 * Java: dec: modernize tests
 * Bazel: js: pick working commit for closure rules
2019-04-12 13:57:42 +02:00
Eugene Kliuchnikov
9cd01c0437
Update WORKSPACE files. (#742) 2019-02-19 11:14:20 +01:00
Eugene Kliuchnikov
8109882ecf
Fix #741 2019-02-18 11:31:48 +01:00
Justin Ridgewell
5805f99a53 Ensure decompression consumes all input (#730)
* Ensure decompression consumes all input

If not, it's a corrupt stream.

* Use byte strings
2018-11-12 10:36:00 +01:00
Eugene Kliuchnikov
d0ffe60b87
Verbose CLI + start pulling "Shared-Brotli" (#722)
* Verbose CLI + start pulling "Shared-Brotli"

 * vesbose CLI output; fix #666
 * pull `SHIFT` transforms; currently this is semantically dead code;
   later it will be used by "Shared-Brotli"
2018-10-24 16:06:09 +02:00
Eugene Kliuchnikov
d6d98957ca
Ramp up version to 1.0.7 2018-10-23 12:24:40 +02:00
Eugene Kliuchnikov
a1e44975a7
Fix #698 2018-10-19 17:01:54 +02:00
Eugene Kliuchnikov
a799e34c7f
Remove dependency to full JDK. This should speedup clean builds. (#719)
* Remove dependency to full JDK. This should speedup clean builds.

* Upgrade appveyor bazel
2018-10-18 17:25:05 +02:00
Stephen Kyle
7a153ebb09 make/build: ensure NEON is enabled and tested (#718)
Make sure the travis CI aarch32 bot tests NEON, and also that running
CROSS_COMPILE=arm-linux-gnueabihf make enables the use of NEON to
accelerate the back-reference copying.
2018-10-17 17:29:32 +02:00
Eugene Kliuchnikov
ce8951c3e9
Fix <arm_neon.h> inclusion guard. (#717) 2018-10-16 17:19:37 +02:00
Eugene Kliuchnikov
f7cbc97c96
Fix typo / minor formatting (#716)
* Fix typo / minor formatting / pull computable constant to the place of use.
2018-10-16 16:46:54 +02:00
Stephen Kyle
cc7a74f15f decode: fix NEON inclusion (#714)
The macro that checks for NEON support should be __ARM_NEON, not
__ARM_NEON__. [1]

AArch64 compilers define __ARM_NEON but not __ARM_NEON__.
AArch32 compilers currently seem to define both, but could be within their
rights to drop __ARM_NEON__ in future versions.

This change moves the check into the common/platform.h file, checks for
both forms, and sets BROTLI_TARGET_NEON if NEON support is available.

[1] Section 6.5.4 of the ARM C Language Extensions.
    (At the time of writing, the latest version was Release 2.1.)
2018-10-08 15:40:11 +02:00
Alexey Ivanov
c94c6f805c tools/brotli: improve window size autodetect (#710)
Window size is defined as:
    `(1 << BROTLI_PARAM_LGWIN) - 16`
in `c/include/brotli/encode.h`

Therefore we should probably take these 16 bytes into account.

Done basic manual testing:
$ python3 -c 'print ("A"*2046)' > t
$ bazel run -- //:brotli -w 0 -f -o $(realpath t).br $(realpath ./t)
$ python3 research/brotlidump.py t.br |& fgrep WSIZE
0000  c1                1000001 WSIZE   windowsize=(1<<12)-16=4080

New version properly detects window size of `4080`, while previous one used `2032`:
$ python3 research/brotlidump.py t.br |& fgrep WSIZE
0000  b1                0110001 WSIZE   windowsize=(1<<11)-16=2032
2018-10-02 16:28:37 +02:00
Stephen Kyle
9402ac5c08 decode: faster huffman code loading on 32-bit Arm (#703)
* platform: add macro for using the 'aligned' attribute

* decode: add accessor macros for HuffmanCode fields

Adds a constructor function for building HuffmanCode values
so they can be accessed quickly on different architectures.

Also adds macros for marking a HuffmanCode table pointer
that can be accessed quickly (BROTLI_HC_MARK_TABLE_FOR_FAST_LOAD),
adjusting the index into that table (BROTLI_HC_ADJUST_TABLE_INDEX),
and getting the .bits or .value fields out of the table at the
current index (BROTLI_HC_GET_BITS/VALUE).

For example, assuming |table| contains a HuffmanCode pointer:

  BROTLI_HC_MARK_TABLE_FOR_FAST_LOAD(table);
  BROTLI_HC_ADJUST_TABLE_INDEX(table, index_into_table);
  *bits = BROTLI_HC_GET_BITS(table);
  *value = BROTLI_HC_GET_VALUE(table);
  BROTLI_HC_ADJUST_TABLE_INDEX(table, offset);
  *bits2 = BROTLI_HC_GET_BITS(table);
  *value2 = BROTLI_HC_GET_VALUE(table);

All uses of the HuffmanCode have been updated appropriately.

* decode: add alternative accessors for HuffmanCode on Arm AArch32
2018-09-27 13:15:46 +02:00
Stephen Kyle
67f059eaf5 Cross compilation support (#709)
* build: add cross-compilation support to make

Set CROSS_COMPILE when running make to use the selected cross
compilation toolchain, such as arm-linux-gnueabihf, or
aarch64-linux-gnu.

Testing requires the presence of qemu - 'qemu-$(ARCH)' will be executed,
where ARCH is the first part of the toolchain triplet.

* build: add cross-compilation support to cmake

If C_COMPILER/CXX_COMPILER/CC/CXX are found to have cross-compilation
triplets in front of the compiler, then qemu will be used to execute the
tests.

* CI: add arm-linux-gnueabihf-gcc builder to Travis

The version of qemu available in Ubuntu trusty (as provided by Travis)
appears to have a bug in qemu-aarch64, which leads to the compatibility
tests failing on some inputs, erroneously rejecting the input as
corrupt.

Once Travis supports xenial, we could add an aarch64-gnu-linux-gcc
builder as well.

* CI: propagate cmake errors out of .travis.sh

Seems like even if cmake fails, the error isn't picked up by Travis.
2018-09-27 11:00:33 +02:00