3e321a3acd
Spot candidates by running these commands:
git ls-files |xargs perl -0777 -n \
-e 'while (/\b(then?|[iao]n|i[fst]|but|f?or|at|and|[dt]o)\s+\1\b/gims)' \
-e '{$n=($` =~ tr/\n/\n/ + 1); ($v=$&)=~s/\n/\\n/g; print "$ARGV:$n:$v\n"}'
Thanks to Jim Meyering for the original patch.
167 lines
5.6 KiB
Plaintext
167 lines
5.6 KiB
Plaintext
|
|
The .lzma File Format
|
|
=====================
|
|
|
|
0. Preface
|
|
0.1. Notices and Acknowledgements
|
|
0.2. Changes
|
|
1. File Format
|
|
1.1. Header
|
|
1.1.1. Properties
|
|
1.1.2. Dictionary Size
|
|
1.1.3. Uncompressed Size
|
|
1.2. LZMA Compressed Data
|
|
2. References
|
|
|
|
|
|
0. Preface
|
|
|
|
This document describes the .lzma file format, which is
|
|
sometimes also called LZMA_Alone format. It is a legacy file
|
|
format, which is being or has been replaced by the .xz format.
|
|
The MIME type of the .lzma format is `application/x-lzma'.
|
|
|
|
The most commonly used software to handle .lzma files are
|
|
LZMA SDK, LZMA Utils, 7-Zip, and XZ Utils. This document
|
|
describes some of the differences between these implementations
|
|
and gives hints what subset of the .lzma format is the most
|
|
portable.
|
|
|
|
|
|
0.1. Notices and Acknowledgements
|
|
|
|
This file format was designed by Igor Pavlov for use in
|
|
LZMA SDK. This document was written by Lasse Collin
|
|
<lasse.collin@tukaani.org> using the documentation found
|
|
from the LZMA SDK.
|
|
|
|
This document has been put into the public domain.
|
|
|
|
|
|
0.2. Changes
|
|
|
|
Last modified: 2011-04-12 11:55+0300
|
|
|
|
|
|
1. File Format
|
|
|
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+==========================+
|
|
| Header | LZMA Compressed Data |
|
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+==========================+
|
|
|
|
The .lzma format file consist of 13-byte Header followed by
|
|
the LZMA Compressed Data.
|
|
|
|
Unlike the .gz, .bz2, and .xz formats, it is not possible to
|
|
concatenate multiple .lzma files as is and expect the
|
|
decompression tool to decode the resulting file as if it were
|
|
a single .lzma file.
|
|
|
|
For example, the command line tools from LZMA Utils and
|
|
LZMA SDK silently ignore all the data after the first .lzma
|
|
stream. In contrast, the command line tool from XZ Utils
|
|
considers the .lzma file to be corrupt if there is data after
|
|
the first .lzma stream.
|
|
|
|
|
|
1.1. Header
|
|
|
|
+------------+----+----+----+----+--+--+--+--+--+--+--+--+
|
|
| Properties | Dictionary Size | Uncompressed Size |
|
|
+------------+----+----+----+----+--+--+--+--+--+--+--+--+
|
|
|
|
|
|
1.1.1. Properties
|
|
|
|
The Properties field contains three properties. An abbreviation
|
|
is given in parentheses, followed by the value range of the
|
|
property. The field consists of
|
|
|
|
1) the number of literal context bits (lc, [0, 8]);
|
|
2) the number of literal position bits (lp, [0, 4]); and
|
|
3) the number of position bits (pb, [0, 4]).
|
|
|
|
The properties are encoded using the following formula:
|
|
|
|
Properties = (pb * 5 + lp) * 9 + lc
|
|
|
|
The following C code illustrates a straightforward way to
|
|
decode the Properties field:
|
|
|
|
uint8_t lc, lp, pb;
|
|
uint8_t prop = get_lzma_properties();
|
|
if (prop > (4 * 5 + 4) * 9 + 8)
|
|
return LZMA_PROPERTIES_ERROR;
|
|
|
|
pb = prop / (9 * 5);
|
|
prop -= pb * 9 * 5;
|
|
lp = prop / 9;
|
|
lc = prop - lp * 9;
|
|
|
|
XZ Utils has an additional requirement: lc + lp <= 4. Files
|
|
which don't follow this requirement cannot be decompressed
|
|
with XZ Utils. Usually this isn't a problem since the most
|
|
common lc/lp/pb values are 3/0/2. It is the only lc/lp/pb
|
|
combination that the files created by LZMA Utils can have,
|
|
but LZMA Utils can decompress files with any lc/lp/pb.
|
|
|
|
|
|
1.1.2. Dictionary Size
|
|
|
|
Dictionary Size is stored as an unsigned 32-bit little endian
|
|
integer. Any 32-bit value is possible, but for maximum
|
|
portability, only sizes of 2^n and 2^n + 2^(n-1) should be
|
|
used.
|
|
|
|
LZMA Utils creates only files with dictionary size 2^n,
|
|
16 <= n <= 25. LZMA Utils can decompress files with any
|
|
dictionary size.
|
|
|
|
XZ Utils creates and decompresses .lzma files only with
|
|
dictionary sizes 2^n and 2^n + 2^(n-1). If some other
|
|
dictionary size is specified when compressing, the value
|
|
stored in the Dictionary Size field is a rounded up, but the
|
|
specified value is still used in the actual compression code.
|
|
|
|
|
|
1.1.3. Uncompressed Size
|
|
|
|
Uncompressed Size is stored as unsigned 64-bit little endian
|
|
integer. A special value of 0xFFFF_FFFF_FFFF_FFFF indicates
|
|
that Uncompressed Size is unknown. End of Payload Marker (*)
|
|
is used if and only if Uncompressed Size is unknown.
|
|
|
|
XZ Utils rejects files whose Uncompressed Size field specifies
|
|
a known size that is 256 GiB or more. This is to reject false
|
|
positives when trying to guess if the input file is in the
|
|
.lzma format. When Uncompressed Size is unknown, there is no
|
|
limit for the uncompressed size of the file.
|
|
|
|
(*) Some tools use the term End of Stream (EOS) marker
|
|
instead of End of Payload Marker.
|
|
|
|
|
|
1.2. LZMA Compressed Data
|
|
|
|
Detailed description of the format of this field is out of
|
|
scope of this document.
|
|
|
|
|
|
2. References
|
|
|
|
LZMA SDK - The original LZMA implementation
|
|
http://7-zip.org/sdk.html
|
|
|
|
7-Zip
|
|
http://7-zip.org/
|
|
|
|
LZMA Utils - LZMA adapted to POSIX-like systems
|
|
http://tukaani.org/lzma/
|
|
|
|
XZ Utils - The next generation of LZMA Utils
|
|
http://tukaani.org/xz/
|
|
|
|
The .xz file format - The successor of the .lzma format
|
|
http://tukaani.org/xz/xz-file-format.txt
|
|
|