Commit Graph

1044 Commits

Author SHA1 Message Date
Eugene Kliuchnikov
f0db711f46
Filter sources for the tarball. (#808) 2020-05-15 13:04:17 +02:00
Eugene Kliuchnikov
7f740f1308
Update (#807)
- fix formatting
 - fix type conversion
 - fix no-op arithmetic with null-pointer
 - improve performance of hash_longest_match64
 - go: detect read after close
 - java decoder: support compound dictionary
 - remove executable flag on non-scripts
2020-05-15 11:06:21 +02:00
Eugene Kliuchnikov
f83aa5169e
Update bazel to 2.2 + update config (#798)
Newer bazel does not support `maven_jar` rule anymore...
2020-03-31 14:38:01 +02:00
Clinton Ingram
924b2b2b9d
Move TZCNT and BSR intrinsics to platform.h, add MSVC versions (#636) 2020-03-19 11:57:56 +01:00
Paul Vollmer
0503d8b766
Added go.mod file to go/cbrotli directory (#754)
* Added go.mod file

* go.mod removed go version
2020-03-19 11:54:51 +01:00
Cristi Vîjdea
f503cb709c
Add HAVE_LOG2 build macro (#783)
* Add HAVE_LOG2 build macro

Fixes #781

* Rename macro to BROTLI_HAVE_LOG2 and move comment for visibility
2020-03-19 10:46:52 +01:00
Leo Neat
36ac0feaf9
Adding CIFuzz (#797) 2020-03-19 09:52:07 +01:00
shenglei10
666c3280cc
Make types of variable match (#796) 2020-02-14 10:40:02 +01:00
agrieve
4b5771bee7 Add missing "const" to a couple of kConstants (#780)
These showed up in a Chromium audit:
https://bugs.chromium.org/p/chromium/issues/detail?id=747064#c8

Although already effectively const, adding "const" causes the symbols to
be moved into the read-only section of the binary.
2019-12-20 00:15:58 +01:00
Griffin Downs
c435f06675 Add vcpkg installation instructions (#776) 2019-10-01 22:53:10 +02:00
James Hilliard
5c3a9a937b Fix license in setup.py (#769) 2019-08-16 16:32:14 +02:00
Ammar Askar
afc4a74273 Add oss-fuzz fuzzing status badge to README (#767) 2019-08-13 15:49:30 +02:00
Eugene Kliuchnikov
35ef5c554d
Disable PIC in EMCC mode. (#768) 2019-08-13 15:23:04 +02:00
Eugene Kliuchnikov
ca21dac8e5
Add an option to avoid building shared libraries. (#766)
Add an option to avoid building shared libraries (for building with EMCC)

Drive-by:
* maven: ramp up java level to minimal required
* travis: replace deprecated clang-5.0 with clang-7
* maven: fallback to jdk10 to void javadoc bug
2019-08-07 10:51:55 +02:00
Eugene Kliuchnikov
3d1767186d
Fix include for EMCC build (#765) 2019-07-30 10:01:21 +02:00
Eugene Kliuchnikov
f1124c8524
More careful sanitizer detection (#764) 2019-07-22 14:29:51 +02:00
Eugene Kliuchnikov
c8b37e8fd1
Update (#762)
* put LICENSE file into .jar
 * fix typo
 * add clarification comment in PY wrapper
2019-07-17 14:39:56 +02:00
Eugene Kliuchnikov
40f0fdcdc1
Explicitly mark tests/testdata/* as binary. (#761)
Fixes #760

Drive-by:
 * update go_rules
 * modernize brotli_inc
 * fix wrapper build
 * update PY to 3 in Travis / OSX / Bazel build
 * upgrade JS Bazel rules.
2019-07-16 17:49:14 +02:00
Eugene Kliuchnikov
78e7bbc3c3
Update (#753)
* fix executable mode of decode.js
 * explain clang-analyser about non-nullability
 * fix "dead assignment"
 * rename proguard.cfg -> proguard.pgcfg
2019-05-03 11:51:11 +02:00
Eugene Kliuchnikov
4b2b2d4f83
Update (#749)
Update:

 * Bazel: fix MSVC configuration
 * C: common: extended documentation and helpers around distance codes
 * C: common: enable BROTLI_DCHECK in "debug" builds
 * C: common: fix implicit trailing zero in `kPrefixSuffix`
 * C: dec: fix possible bit reader discharge for "large-window" mode
 * C: dec: simplify distance decoding via lookup table
 * C: dec: reuse decoder state members memory via union with lookup table
 * C: dec: add decoder state diagram
 * C: enc: clarify access to static dictionary
 * C: enc: improve static dictionary hash
 * C: enc: add "stream offset" parameter for parallel encoding
 * C: enc: reorganize hasher; now Q2-Q3 require exactly 256KiB
           to avoid global TCMalloc lock
 * C: enc: fix rare access to uninitialized data in ring-buffer
 * C: enc: reorganize logging / checks in `write_bits.h`
 * Java: dec: add "large-window" support
 * Java: dec: improve speed
 * Java: dec: debug and 32-bit mode are now activated via system properties
 * Java: dec: demystify some state variables (use better names)
 * Dictionary generator: add single input mode
 * Java: dec: modernize tests
 * Bazel: js: pick working commit for closure rules
2019-04-12 13:57:42 +02:00
Eugene Kliuchnikov
9cd01c0437
Update WORKSPACE files. (#742) 2019-02-19 11:14:20 +01:00
Eugene Kliuchnikov
8109882ecf
Fix #741 2019-02-18 11:31:48 +01:00
Justin Ridgewell
5805f99a53 Ensure decompression consumes all input (#730)
* Ensure decompression consumes all input

If not, it's a corrupt stream.

* Use byte strings
2018-11-12 10:36:00 +01:00
Eugene Kliuchnikov
d0ffe60b87
Verbose CLI + start pulling "Shared-Brotli" (#722)
* Verbose CLI + start pulling "Shared-Brotli"

 * vesbose CLI output; fix #666
 * pull `SHIFT` transforms; currently this is semantically dead code;
   later it will be used by "Shared-Brotli"
2018-10-24 16:06:09 +02:00
Eugene Kliuchnikov
d6d98957ca
Ramp up version to 1.0.7 2018-10-23 12:24:40 +02:00
Eugene Kliuchnikov
a1e44975a7
Fix #698 2018-10-19 17:01:54 +02:00
Eugene Kliuchnikov
a799e34c7f
Remove dependency to full JDK. This should speedup clean builds. (#719)
* Remove dependency to full JDK. This should speedup clean builds.

* Upgrade appveyor bazel
2018-10-18 17:25:05 +02:00
Stephen Kyle
7a153ebb09 make/build: ensure NEON is enabled and tested (#718)
Make sure the travis CI aarch32 bot tests NEON, and also that running
CROSS_COMPILE=arm-linux-gnueabihf make enables the use of NEON to
accelerate the back-reference copying.
2018-10-17 17:29:32 +02:00
Eugene Kliuchnikov
ce8951c3e9
Fix <arm_neon.h> inclusion guard. (#717) 2018-10-16 17:19:37 +02:00
Eugene Kliuchnikov
f7cbc97c96
Fix typo / minor formatting (#716)
* Fix typo / minor formatting / pull computable constant to the place of use.
2018-10-16 16:46:54 +02:00
Stephen Kyle
cc7a74f15f decode: fix NEON inclusion (#714)
The macro that checks for NEON support should be __ARM_NEON, not
__ARM_NEON__. [1]

AArch64 compilers define __ARM_NEON but not __ARM_NEON__.
AArch32 compilers currently seem to define both, but could be within their
rights to drop __ARM_NEON__ in future versions.

This change moves the check into the common/platform.h file, checks for
both forms, and sets BROTLI_TARGET_NEON if NEON support is available.

[1] Section 6.5.4 of the ARM C Language Extensions.
    (At the time of writing, the latest version was Release 2.1.)
2018-10-08 15:40:11 +02:00
Alexey Ivanov
c94c6f805c tools/brotli: improve window size autodetect (#710)
Window size is defined as:
    `(1 << BROTLI_PARAM_LGWIN) - 16`
in `c/include/brotli/encode.h`

Therefore we should probably take these 16 bytes into account.

Done basic manual testing:
$ python3 -c 'print ("A"*2046)' > t
$ bazel run -- //:brotli -w 0 -f -o $(realpath t).br $(realpath ./t)
$ python3 research/brotlidump.py t.br |& fgrep WSIZE
0000  c1                1000001 WSIZE   windowsize=(1<<12)-16=4080

New version properly detects window size of `4080`, while previous one used `2032`:
$ python3 research/brotlidump.py t.br |& fgrep WSIZE
0000  b1                0110001 WSIZE   windowsize=(1<<11)-16=2032
2018-10-02 16:28:37 +02:00
Stephen Kyle
9402ac5c08 decode: faster huffman code loading on 32-bit Arm (#703)
* platform: add macro for using the 'aligned' attribute

* decode: add accessor macros for HuffmanCode fields

Adds a constructor function for building HuffmanCode values
so they can be accessed quickly on different architectures.

Also adds macros for marking a HuffmanCode table pointer
that can be accessed quickly (BROTLI_HC_MARK_TABLE_FOR_FAST_LOAD),
adjusting the index into that table (BROTLI_HC_ADJUST_TABLE_INDEX),
and getting the .bits or .value fields out of the table at the
current index (BROTLI_HC_GET_BITS/VALUE).

For example, assuming |table| contains a HuffmanCode pointer:

  BROTLI_HC_MARK_TABLE_FOR_FAST_LOAD(table);
  BROTLI_HC_ADJUST_TABLE_INDEX(table, index_into_table);
  *bits = BROTLI_HC_GET_BITS(table);
  *value = BROTLI_HC_GET_VALUE(table);
  BROTLI_HC_ADJUST_TABLE_INDEX(table, offset);
  *bits2 = BROTLI_HC_GET_BITS(table);
  *value2 = BROTLI_HC_GET_VALUE(table);

All uses of the HuffmanCode have been updated appropriately.

* decode: add alternative accessors for HuffmanCode on Arm AArch32
2018-09-27 13:15:46 +02:00
Stephen Kyle
67f059eaf5 Cross compilation support (#709)
* build: add cross-compilation support to make

Set CROSS_COMPILE when running make to use the selected cross
compilation toolchain, such as arm-linux-gnueabihf, or
aarch64-linux-gnu.

Testing requires the presence of qemu - 'qemu-$(ARCH)' will be executed,
where ARCH is the first part of the toolchain triplet.

* build: add cross-compilation support to cmake

If C_COMPILER/CXX_COMPILER/CC/CXX are found to have cross-compilation
triplets in front of the compiler, then qemu will be used to execute the
tests.

* CI: add arm-linux-gnueabihf-gcc builder to Travis

The version of qemu available in Ubuntu trusty (as provided by Travis)
appears to have a bug in qemu-aarch64, which leads to the compatibility
tests failing on some inputs, erroneously rejecting the input as
corrupt.

Once Travis supports xenial, we could add an aarch64-gnu-linux-gcc
builder as well.

* CI: propagate cmake errors out of .travis.sh

Seems like even if cmake fails, the error isn't picked up by Travis.
2018-09-27 11:00:33 +02:00
Jørgen Ibsen
6eba239a5b Fix auto detect of bundled mode (#704)
Set bundled mode to ON when parent directory is not empty. Due to the
peculiarities of CMake if, comparing an undefined variable to the empty
string is false, so this likely never triggered.
2018-09-13 13:31:23 -04:00
Eugene Kliuchnikov
2216a0dd63
Update (#706)
Update
 * add ASAN/MSAN unaligned read specializations
 * add "brotli" prefix to u_uint64 type
 * increment version to 1.0.06
 * fix CoverityScan "unused assignment" warning
 * fix JDK 8<->9 incompatibility
 * add encoder optimization for empty input
 * regenerate JS decoder
 * unbreak Travis builds
2018-09-13 08:09:32 -04:00
Stephen Kyle
d4cd6cdf1c platform: fix unaligned 64-bit accesses on AArch32 (#702)
Ensures that Aarch32 Arm builds with an Armv8 compiler do not set
BROTLI_64_BITS.

This scenario is possible with ChromeOS builds, as they may use a
toolchain with the target armv7-cros-gnueabi, but with -march=armv8.
This will set __ARM_ARCH to 8 (defining BROTLI_TARGET_ARMV8), but will
also set __ARM_32BIT_STATE and not __ARM_64BIT_STATE. Without this,
illegal 64-bit non-word-aligned reads (LDRD) may be emitted.

Also fix unaligned 64-bit reads on AArch32 - STRD was still possible to
emit.
2018-07-25 11:43:06 +02:00
Eugene Kliuchnikov
8a073bd9e2
Revert "platform: fix unaligned 64-bit accesses on AArch32 (#699)" (#701)
This reverts commit 6d027d1648.
2018-07-24 17:32:13 +02:00
Stephen Kyle
6d027d1648 platform: fix unaligned 64-bit accesses on AArch32 (#699)
Ensures that Aarch32 Arm builds with an Armv8 compiler do not set
BROTLI_64_BITS.

This scenario is possible with ChromeOS builds, as they may use a
toolchain with the target armv7-cros-gnueabi, but with -march=armv8.
This will set __ARM_ARCH to 8 (defining BROTLI_TARGET_ARMV8), but will
also set __ARM_32BIT_STATE and not __ARM_64BIT_STATE. Without this,
illegal 64-bit non-word-aligned reads (LDRD) may be emitted.

Also fix unaligned 64-bit reads on AArch32 - STRD was still possible to
emit.
2018-07-24 17:29:50 +02:00
William A. Kennington III
fc4d345968 Fix missing header files (#695)
Our dist tarball is missing hash_rolling_inc.h and
hash_composite_inc.h, which causes subsequent autotools
builds to fail. Fix this by adding it to the sources list.

Signed-off-by: William A. Kennington III <william@wkennington.com>
2018-07-09 10:40:08 +02:00
Eugene Kliuchnikov
b601fe817b Ramp up version to 1.0.5 2018-06-27 17:03:45 +02:00
Cody Schroeder
ee2a5e1540 Update go_library to use standard importpath (#690)
* Update go_library to use standard importpath

Instead of using go_prefix, which is deprecated, the importpath attribute is made explicit.

* Add description to go/BUILD
2018-06-26 18:08:07 +02:00
Eugene Kliuchnikov
eb12ec04eb
Update (#688)
* add rolling-composite-hasher for large-window mode
* make API methods explicitly public
2018-06-20 15:14:10 +02:00
Eugene Kliuchnikov
7505290ef9
Convert fuzzer to C99. (#686) 2018-06-18 14:39:38 +02:00
Eugene Kliuchnikov
ff05c35166
Add VS2017 release Appveyor build (#685) 2018-06-18 13:13:23 +02:00
Eugene Kliuchnikov
09cd3e877f
Update 2018-06-11 15:17:26 +02:00
Eugene Kliuchnikov
8544ae858d
Update (#680)
* fix MSVC warnings
 * cleanups
2018-06-09 11:17:13 +02:00
Eugene Kliuchnikov
1e7ea1d8e6
Inverse bazel project/workspace tree (#677)
* Inverse bazel workspace tree.

Now each subproject directly depends on root (c) project.

This helps to mitigate Bazel bug bazelbuild/bazel#2391; short summary:
Bazel does not work if referenced subproject `WORKSPACE` uses any
repositories that embedding project does not.

Bright side: building C project is much faster;
no need to download closure, go and JDK...
2018-06-04 17:53:16 +02:00
Eugene Kliuchnikov
29dc2cce90
Update golang and JS Bazel plugins to latest stable versions. (#676) 2018-05-31 13:21:04 +02:00
davidlt
f9b8c02673 Add RISC-V 64-bit (riscv64) platform configuration (#669)
Signed-off-by: David Abdurachmanov <david.abdurachmanov@gmail.com>
2018-05-22 14:35:04 +02:00