c05b270edc
This changes the size limit on compressed blocks to match those of the other block types: they may not be larger than the `Block_Maximum_Decompressed_Size`, which is the smaller of the `Window_Size` and 128 KB, removing the additional restriction that had been placed on `Compressed_Block`s, that they be smaller than the decompressed content they represent. Several things motivate removing this restriction. On the one hand, this restriction is not useful for decoders: the decoder must nonetheless be prepared to accept compressed blocks that are the full `Block_Maximum_Decompressed_Size`. And on the other, this bound is actually artificially limiting. If block representations were entirely independent, a compressed representation of a block that is larger than the contents of the block would be ipso facto useless, and it would be strictly better to send it as an `Raw_Block`. However, blocks are not entirely independent, and it can make sense to pay the cost of encoding custom entropy tables in a block, even if that pushes that block size over the size of the data it represents, because those tables can be re-used by subsequent blocks. Finally, as far as I can tell, this restriction in the spec is not currently enforced in any Zstandard implementation, nor has it ever been. This change should therefore be safe to make. |
||
|---|---|---|
| .. | ||
| educational_decoder | ||
| images | ||
| README.md | ||
| zstd_compression_format.md | ||
| zstd_manual.html | ||
Zstandard Documentation
This directory contains material defining the Zstandard format,
as well as detailed instructions to use zstd library.
zstd_manual.html : Documentation of zstd.h API, in html format.
Click on this link: http://zstd.net/zstd_manual.html
to display documentation of latest release in readable format within a browser.
zstd_compression_format.md : This document defines the Zstandard compression format.
Compliant decoders must adhere to this document,
and compliant encoders must generate data that follows it.
Should you look for resources to develop your own port of Zstandard algorithm, you may find the following resources useful :
educational_decoder : This directory contains an implementation of a Zstandard decoder,
compliant with the Zstandard compression format.
It can be used, for example, to better understand the format,
or as the basis for a separate implementation of Zstandard decoder.
decode_corpus :
This tool, stored in /tests directory, is able to generate random valid frames,
which is useful if you wish to test your decoder and verify it fully supports the specification.