In coding theory, a variable-length code is a code which maps source symbols to a variable number of bits. The equivalent concept in computer science is bit string.
Variable-length codes can allow sources to be compressed and decompressed with zero error (lossless data compression) and still be read back symbol by symbol. With the right coding strategy an independent and identically-distributed source may be compressed almost arbitrarily close to its entropy. This is in contrast to fixed-length coding methods, for which data compression is only possible for large blocks of data, and any compression beyond the logarithm of the total number of possibilities comes with a finite (though perhaps arbitrarily small) probability of failure.
Some examples of well-known variable-length coding strategies are Huffman coding, Lempel–Ziv coding, arithmetic coding, and context-adaptive variable-length coding.
The extension of a code is the mapping of finite length source sequences to finite length bit strings, that is obtained by concatenating for each symbol of the source sequence the corresponding codeword produced by the original code.
Using terms from formal language theory, the precise mathematical definition is as follows: Let
S
T
C:S\toT*
S
T
C
S*
T*
Variable-length codes can be strictly nested in order of decreasing generality as non-singular codes, uniquely decodable codes and prefix codes. Prefix codes are always uniquely decodable, and these in turn are always non-singular:
A code is non-singular if each source symbol is mapped to a different non-empty bit string, i.e. the mapping from source symbols to bit strings is injective.
M1=\{a\mapsto0,b\mapsto0,c\mapsto1\}
M2=\{a\mapsto1,b\mapsto011,c\mapsto01110,d\mapsto1110,e\mapsto10011,f\mapsto0\}
A code is uniquely decodable if its extension is § non-singular. Whether a given code is uniquely decodable can be decided with the Sardinas–Patterson algorithm.
M3=\{a\mapsto0,b\mapsto01,c\mapsto011\}
M2
See main article: Prefix code.
A code is a prefix code if no target bit string in the mapping is a prefix of the target bit string of a different source symbol in the same mapping. This means that symbols can be decoded instantaneously after their entire codeword is received. Other commonly used names for this concept are prefix-free code, instantaneous code, or context-free code.
M3
Symbol | Codeword | |
---|---|---|
a | 0 | |
b | 10 | |
c | 110 | |
d | 111 |
Example of encoding and decoding:
→ 00100110111010 → |0|0|10|0|110|111|0|10| →
A special case of prefix codes are block codes. Here all codewords must have the same length. The latter are not very useful in the context of source coding, but often serve as forward error correction in the context of channel coding.
Another special case of prefix codes are LEB128 and variable-length quantity (VLQ) codes, which encode arbitrarily large integers as a sequence of octets—i.e., every codeword is a multiple of 8 bits.
The advantage of a variable-length code is that unlikely source symbols can be assigned longer codewords and likely source symbols can be assigned shorter codewords, thus giving a low expected codeword length. For the above example, if the probabilities of (a, b, c, d) were
|
1 | |
4 |
,
1 | |
8 |
,
1 | |
8 |
\right)
1 x | 1 | +2 x |
2 |
1 | +3 x | |
4 |
1 | +3 x | |
8 |
1 | = | |
8 |
7 | |
4 |