Commit Graph

113 Commits

Author SHA1 Message Date
eustas
226d21c403 Make encoder API more safe
Add flag to check if "last" block has been already processed
2016-05-18 11:01:16 +02:00
Frank Aurich
d9a268c60c Fix compilation with VS2012 2016-04-22 10:55:31 +02:00
Eugene Kliuchnikov
e7e3849835 Update decoder. Add encoder interface wrapper.
* condense printf in port.h; use BROTLI_LOG everywhere
 * mark non-exported functions with BROTLI_INTERNAL
 * use BROTLI_DUMP instead of (void)(BROTLI_FAILURE())
 * fix problems with CustomDictionary
 * make decode.h independent of state.h
 * fix "double-new-lines"
 * fix some strict compilation warnings
 * fix bro.cc compilation for MSVS
 * added compressor.h as a replacement for encode.h + streams.h
2016-04-19 16:29:10 +02:00
Piotr Sikora
501cb86172 Fix build with -Wmissing-declarations.
While there, add -Wmissing-prototypes and -Wmissing-declarations
to shared.mk in order to catch similar errors in the future.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2016-03-18 19:18:59 -07:00
Zoltan Szabadka
846575ff6c Fix TODO formatting. 2016-03-15 11:13:07 +01:00
Zoltan Szabadka
b820c39bd9 Reduce memory usage of brotli encoder at quality 10 and 11. 2016-03-15 10:50:16 +01:00
Eugene Kliuchnikov
25e3796f83 Fix most of ()->(void) and some missing includes. 2016-02-29 14:41:24 +01:00
Zoltan Szabadka
dbb53e6434 Use a hash-to-binary-tree data structure for quality 11 as suggested by Issue #180. 2016-01-27 09:50:39 +01:00
Zoltan Szabadka
14d6ae74a9 Reorder members of the Command struct plus fix some warnings.
This may save 8 bytes of padding per Command (32 -> 24 bytes).
2016-01-26 11:25:53 +01:00
Zoltan Szabadka
82c9e1972d Relax the prerequisites of WriteBrotliData().
Instead of returning false, if it is called with no
new input for a non-last block, just check if it has
any already processed data to flush, and if not,
return true with empty output.
2016-01-12 14:45:35 +01:00
Zoltan Szabadka
b4c223cf27 Add missing headers. 2016-01-11 12:17:44 +01:00
Zoltan Szabadka
417107b3dd Add two more fast modes to the brotli compressor.
The new modes process the input data in independent blocks,
using backward references only from within an input block.

The new modes can be used by specifying quality 0 or quality 1,
the old quality 1 and quality 2 modes are renamed quality 2 and
quality 3, respectively, and the old quality 3 mode is removed.
2016-01-11 11:21:42 +01:00
Zoltan Szabadka
1bf1b0a598 Faster entropy coding phase for quality 1.
In quality 1, use static Huffman codes for distance
and command histograms with <= 128 symbols and dynamic
Huffman codes with static code length codes for the other
histograms.
2016-01-08 10:10:22 +01:00
Zoltan Szabadka
4dd9114c97 Partial Hasher initialization for small input data.
This increases compression speed of very small files (< 1KB) for quality <= 3.
2016-01-07 17:10:34 +01:00
Zoltan Szabadka
8844b7f0d7 Fix more conversion warnings. 2016-01-07 16:27:49 +01:00
Eugene Klyuchnikov
24ffa78414 Fix headers 2015-12-11 11:11:51 +01:00
eustas
bc5da25a43 Merge pull request #272 from eustas/master
Upgrade license to MIT.
2015-12-09 16:25:06 +01:00
Eugene Klyuchnikov
901cd82f4f Fix WriteMetadata (unaligned and out-of-bounds write). 2015-12-04 16:09:40 +01:00
Eugene Klyuchnikov
771eb10798 Update license statement in source files. 2015-11-27 11:27:11 +01:00
Eugene Klyuchnikov
bb26d1919f Fix sign-comparison warnings
+ add more debug runtime checks
+ minor cleanup
2015-11-23 11:05:12 +01:00
Eugene Klyuchnikov
152e33c3a0 Add more explicit type conversions.
Remove dead code.
Fix includes.
2015-11-17 13:45:41 +01:00
Zoltan Szabadka
8d061836ac Fix assertion in 32-bit build. 2015-11-12 20:13:58 +01:00
Zoltan Szabadka
ea48ce5a6f Fix --Wconversion and --pedantic-erros for the encoder. 2015-10-28 17:44:47 +01:00
Zoltan Szabadka
a89b57b90c Use uint32_t positions in the hasher and compute distances modulo 2^32. 2015-10-26 17:08:57 +01:00
Zoltan Szabadka
d2e857d83e Fix integer overflow and slowness in entropy estimation. 2015-10-23 11:19:04 +02:00
Mayhem
2a1a1f7282 Remove useless BrotliCompressor instantiation in BrotliCompressBuffer 2015-10-21 19:54:35 +02:00
Zoltan Szabadka
512b9b8a3c Remove 'static' from kBitCostThreshold declaration. 2015-10-11 13:03:51 +02:00
Zoltan Szabadka
2726b8a4f6 Encoder fixes.
* Remove default constructors.
* Initialize bit_cost in histogram.Clear().
* Check fseek result in FileSize.
* Replace malloc in BrotliFileIn constructor with "new".
* Catch bad_alloc in bro tool.
2015-10-06 11:23:44 +02:00
Cosimo Lupo
66fa4ff403 [types.h] make std ints types for _MSC_VER compatible with CFFI
As defined in _cffi_include.h: 21fef94ca0/cffi/_cffi_include.h (_cffi_include.h-15)
2015-10-05 11:32:12 +01:00
Zoltan Szabadka
b39eec8810 Remove C++11 vector::data() calls from encoder. 2015-10-05 11:43:49 +02:00
Zoltan Szabadka
99aae450b8 Initialize min_cost_cmd_ in constructor. 2015-10-05 11:43:31 +02:00
Zoltan Szabadka
b6689b1504 Remove unnecessary branch from literal cost calculation. 2015-10-05 11:42:45 +02:00
szabadka
0477473ba2 Merge pull request #184 from IIoTeP9HuY/master
Support large inputs/outputs in memory adaptors
2015-10-02 13:22:15 +02:00
Zoltan Szabadka
754deaed2f Reduce command buffer memory usage. 2015-10-01 17:08:59 +02:00
Zoltan Szabadka
4c37566f4b Move literal cost computation to where it's used.
Move utf8 heuristics functions to their own file.
2015-10-01 15:10:42 +02:00
Zoltan Szabadka
3b8bef70a5 Add extern "C" linkage to the encoder and decoder dictionary definitions. 2015-10-01 14:30:22 +02:00
Zoltan Szabadka
d4cc4f8f6f Define the encoder dictionary in the .cc file and link only once. 2015-10-01 13:08:43 +02:00
Zoltan Szabadka
4a7024dcde Make the brotli encoder C++98 compatible. 2015-10-01 12:08:14 +02:00
Zoltan Szabadka
dfdf2dd4c4 Encoder bug fixes.
* Fix forward declaration mismatch.
* Fix division by zero in 64X test.
* Avoid shadowing of variables in encoder.
2015-10-01 11:40:05 +02:00
acid
08e98d8d89 Support large inputs/outputs in memory adaptors 2015-09-28 16:03:51 +03:00
Marcin Karpinski
21ac39f7c8 Fix typos. 2015-09-21 21:04:07 +02:00
Eugene Klyuchnikov
b58317a652 Fix bug in encoder. 2015-09-01 12:18:41 +02:00
Lode Vandevenne
6511d6b016 update brotli encoder with latest improvements 2015-08-28 16:09:23 +02:00
Zoltan Szabadka
7de70dbcc7 Add missing <stdlib.h> to streams.cc 2015-08-11 11:09:04 +02:00
Lode Vandevenne
17ed258993 msan bugfixes to the brotli encoder 2015-08-10 13:25:45 +02:00
Zoltan Szabadka
29c2679500 Fix encoder bug.
Under some circumstances CopyLiteralsToByteArray tried
to read begind ringbuffer.

In this patch we force it to read completely from range [0..mask]
2015-07-30 17:42:02 +02:00
Zoltan Szabadka
95ddb48a11 Fix some VS compilation errors in the encoder.
- Use std::numeric_limits<double>::infinity() instead of 1.0 / 0.0
  - Use FastLog2() instead of log2() in cost model
2015-06-29 14:20:25 +02:00
Lode Vandevenne
bad0f4edf1 Brotli Bug Fixes 2015-06-26 17:37:00 +02:00
Zoltan Szabadka
618287b373 Deprecate greedy_block_split and enable_context_modeling brotli params.
These affected only quality 11, and now it does not make sense
to disable block splitting or context modeling because most of
the time is spent in zopfli anyway.

Now all speed vs size compromises are controlled by the quality param.
2015-06-12 16:50:49 +02:00
Zoltan Szabadka
66098830a2 Use a static hash table to look up dictionary words and transforms.
This is used for quality 11, for qualities <= 9 we already
have a simpler hash table.

The static data size is 252 kB, and this removes the
need to initialize a huge hash map at startup, which was
the reason why transforms had to be disabled by default.
In comparison, the static dictionary itself is 120 kB.
This supports every transform, except the kOmitFirstN.
2015-06-12 16:45:17 +02:00