AuroraMiddleware/brotli
1
0
Fork 0
mirror of https://github.com/google/brotli.git synced 2024-11-22 11:40:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,224 Commits 5 Branches 19 Tags 46 MiB
cec846f88e
Commit Graph

31 Commits

Author SHA1 Message Date
Evgenii Kliuchnikov
3396c67fea add brcat alias + flag to decompress concatenated streams 
PiperOrigin-RevId: 598652401
 2024-01-15 12:49:56 -08:00
Evgenii Kliuchnikov
033940f97c add comment (fingerprint) CLI feature 
PiperOrigin-RevId: 597489910
 2024-01-11 02:04:37 -08:00
Evgenii Kliuchnikov
c1bd196833 comb HAVE_UTIMENSAT definition 
PiperOrigin-RevId: 560011681
 2023-08-25 01:07:12 -07:00
Evgenii Kliuchnikov
d639a81d35 add option to delete files that are not "compressed" 
PiperOrigin-RevId: 552472135
 2023-07-31 07:19:14 -07:00
Evgenii Kliuchnikov
27a9a80992 simplify CMake build 
PiperOrigin-RevId: 552238545
 2023-07-30 03:45:11 -07:00
Evgenii Kliuchnikov
2e6164d7b0 verbose error report in CLI 
PiperOrigin-RevId: 546833411
 2023-07-10 11:43:42 +00:00
Eugene Kliuchnikov
509d4419bd
Copy ns time stat (#992) 2022-12-22 16:05:25 +01:00
Evgenii Kliuchnikov
a8f5813b84 Update 
Documentation:
  - add note that brotli is a "stream" format, not an archive-like
  - regenerate .1 with Pandoc
Build:
  - drop legacy "BROTLI_BUILD_PORTABLE" option
  - drop "BROTLI_SANITIZED" definition
Code:
  - c: comb includes
  - c/enc: extract encoder state into separate header
  - c/enc: drop designated q10 codepath
  - c/enc: dealing better with flushing of empty stream
  - fix MSVC compilation
API:
  - py: use library version instead of one in version.h
  - c: add plugable API to report consumed input / produced output
  - c/java: support "lean" prepared dictionaries (without copy of source)
 2022-11-17 13:03:09 +00:00
Eugene Kliuchnikov
8376f72ed6
Prepare for copybara (#939) 
Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
 2021-11-10 10:34:39 +01:00
Eugene Kliuchnikov
19d86fb9a6
Merge-in SharedDictionary feature (#916) 
Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
 2021-08-04 14:42:02 +02:00
Pavel Rosický
90fd2b60cc
add execution time (#834) 2020-09-07 10:53:03 +02:00
Eugene Kliuchnikov
fc823290a7
Mute strerror/strcpy warnings is MSVC build. (#815) 2020-07-02 19:45:57 +02:00
Eugene Kliuchnikov
5519352661
Add workaround for lying feof. (#814) 
Should fix #812
 2020-07-02 17:57:40 +02:00
Eugene Kliuchnikov
c8b37e8fd1
Update (#762) 
* put LICENSE file into .jar
 * fix typo
 * add clarification comment in PY wrapper
 2019-07-17 14:39:56 +02:00
Eugene Kliuchnikov
4b2b2d4f83
Update (#749) 
Update:

 * Bazel: fix MSVC configuration
 * C: common: extended documentation and helpers around distance codes
 * C: common: enable BROTLI_DCHECK in "debug" builds
 * C: common: fix implicit trailing zero in `kPrefixSuffix`
 * C: dec: fix possible bit reader discharge for "large-window" mode
 * C: dec: simplify distance decoding via lookup table
 * C: dec: reuse decoder state members memory via union with lookup table
 * C: dec: add decoder state diagram
 * C: enc: clarify access to static dictionary
 * C: enc: improve static dictionary hash
 * C: enc: add "stream offset" parameter for parallel encoding
 * C: enc: reorganize hasher; now Q2-Q3 require exactly 256KiB
           to avoid global TCMalloc lock
 * C: enc: fix rare access to uninitialized data in ring-buffer
 * C: enc: reorganize logging / checks in `write_bits.h`
 * Java: dec: add "large-window" support
 * Java: dec: improve speed
 * Java: dec: debug and 32-bit mode are now activated via system properties
 * Java: dec: demystify some state variables (use better names)
 * Dictionary generator: add single input mode
 * Java: dec: modernize tests
 * Bazel: js: pick working commit for closure rules
 2019-04-12 13:57:42 +02:00
Eugene Kliuchnikov
d0ffe60b87
Verbose CLI + start pulling "Shared-Brotli" (#722) 
* Verbose CLI + start pulling "Shared-Brotli"

 * vesbose CLI output; fix #666
 * pull `SHIFT` transforms; currently this is semantically dead code;
   later it will be used by "Shared-Brotli"
 2018-10-24 16:06:09 +02:00
Alexey Ivanov
c94c6f805c tools/brotli: improve window size autodetect (#710) 
Window size is defined as:
    `(1 << BROTLI_PARAM_LGWIN) - 16`
in `c/include/brotli/encode.h`

Therefore we should probably take these 16 bytes into account.

Done basic manual testing:
$ python3 -c 'print ("A"*2046)' > t
$ bazel run -- //:brotli -w 0 -f -o $(realpath t).br $(realpath ./t)
$ python3 research/brotlidump.py t.br |& fgrep WSIZE
0000  c1                1000001 WSIZE   windowsize=(1<<12)-16=4080

New version properly detects window size of `4080`, while previous one used `2032`:
$ python3 research/brotlidump.py t.br |& fgrep WSIZE
0000  b1                0110001 WSIZE   windowsize=(1<<11)-16=2032
 2018-10-02 16:28:37 +02:00
Eugene Kliuchnikov
68db5c0272
Update (#660) 
* Update
 * improve q=1 compression on small files
 * fix "left shift before promotion"
 * fix osx Travis builds
 2018-04-13 11:44:34 +02:00
Eugene Kliuchnikov
0f3c84e745
Update (#656) 
* proper fix for the "fall through" warning"
 * automatic NDIRECT/NPOSTFIX tuning (better compression)
 * fix unaligned access for `aarch64`-cross-`armhf` build
 * fix `aarch64` detection (10% decoder speedup)
 * expose `large_window` CLI option
 * make default window size 16MiB
 * ramp up version to 1.0.4
 2018-03-27 22:29:22 +02:00
Eugene Kliuchnikov
631fe194a1
Update (#651) 
* fix `bazel` build (ignore switch case fall-through)
* add `NPOSTFIX` / `NDIRECT` encoder parameters
* fix source file lists (add `params.h`)
* fix bug in `durchschlag`
* print clarifying messages wheb CLI argument parsing fails
 2018-03-20 17:37:41 +06:00
Eugene Kliuchnikov
35e69fc7cf
New feature: "Large Window Brotli" (#640) 
* New feature: "Large Window Brotli"

By setting special encoder/decoder flag it is now possible to extend
LZ-window up to 30 bits; though produced stream will not be RFC7932
compliant.

Added new dictionary generator - "DSH". It combines speed of "Sieve"
and quality of "DM". Plus utilities to prepare train corpora
(remove unique strings).

Improved compression ratio: now two sub-blocks could be stitched:
the last copy command could be extended to span the next sub-block.

Fixed compression ineffectiveness caused by floating numbers rounding and
wrong cost heuristic.

Other C changes:
 - combined / moved `context.h` to `common`
 - moved transforms to `common`
 - unified some aspects of code formatting
 - added an abstraction for encoder (static) dictionary
 - moved default allocator/deallocator functions to `common`

brotli CLI:
 - window size is auto-adjusted if not specified explicitly

Java:
 - added "eager" decoding both to JNI wrapper and pure decoder
 - huge speed-up of `DictionaryData` initialization

* Add dictionaryless compressed dictionary

* Fix `sources.lst`

* Fix `sources.lst` and add a note that `libtool` is also required.

* Update setup.py

* Fix `EagerStreamTest`

* Fix BUILD file

* Add missing `libdivsufsort` dependency

* Fix "unused parameter" warning.
 2018-02-26 09:04:36 -05:00
Tomáš Popela
a0c7dafe28 Fix permissions of various files in project (#613) 
Move from 755 to 644.
 2017-10-10 11:24:13 +02:00
Eugene Kliuchnikov
c60563591a Fix API documentation + theoretical NPEs (#602) 2017-09-20 15:02:01 +02:00
Eugene Kliuchnikov
37fb83ec0d Update: (#600) 
* encoder: relax backward references candidates asserts
 * encoder: make RNG more platform-independent
 * encoder: remove "unused" param (context mode)
 * CLI: improve first-encounter experience
 * Java: update SynthTest
 * Java: refine proguard config
 * Java/JNI: fix one-shot compression workflow
 2017-09-19 15:57:15 +02:00
Eugene Kliuchnikov
d7bce1e092 Update (#593) 
* Update:
 * fix CLI error messages
 * fix CLI console IO on Windows
 2017-09-07 20:27:49 +02:00
Eugene Kliuchnikov
6535435413 Update (#589) 
* cleanup
 * fix `unbrotli` CLI
 * Java retouch for faster JS decoder
 2017-08-24 13:29:48 +02:00
Eugene Kliuchnikov
d63e8f75f5 Update API, and more (#581) 
Update API, and more:
 * remove "custom dictionary" support
 * c/encoder: fix #580: big-endian build
 * Java: reduce jar size
 * Java: speedup decoding
 * Java: add 32-bit CPU support
 * Java: make source code JS transpiler-ready
 2017-08-04 10:02:56 +02:00
Eugene Kliuchnikov
52441069ef Update (#574) 
* Update
 * decoder: better behavior after failure
 * encoder: replace "len_x_code" with delta
 * research: add experimental dictionary generator
 * python: test combing
 2017-07-21 10:07:24 +02:00
Eugene Kliuchnikov
1becbbf231 Update (#569) 
* add misssing fclose in `brotli.c`
 * add basic tests for python `Decompressor` type
 * minor lint fixes in `_brotli.cc`
 2017-06-30 13:09:50 +02:00
Eugene Kliuchnikov
05d5f3d77a Update (#560) 
Update:
 * add decoder API to avoid ringbuffer reallocation
 * fix MSVC warnings
 * remove dead code
 2017-06-13 12:52:56 +02:00
Eugene Kliuchnikov
03739d2b11 Update (#555) 
Update:
 * new CLI; bro -> brotli; + man page
 * JNI wrappers preparation (for bazel build)
 * add raw binary dictionary representation `dictionary.bin`
 * add ability to side-load brotli RFC dictionary
 * decoder persists last error now
 * fix `BrotliDecoderDecompress` documentation
 * go reader don't block until necessary
 * more consistent bazel target names
 * Java dictionary data compiled footprint reduced
 * Java tests refactoring
 2017-05-29 17:55:14 +02:00