Information source (mathematics) explained

In mathematics, an information source is a sequence of random variables ranging over a finite alphabet Γ, having a stationary distribution.

The uncertainty, or entropy rate, of an information source is defined as

H\{ X \} = \lim _{n\toinfty} H(X _{n} | X _{0,} X _{1,} ... , X _{n-1} )

where

X _{0,} X _{1,} ... , X _{n}

is the sequence of random variables defining the information source, and

H(X _{n} | X _{0,} X _{1,} ... , X _{n-1} )

is the conditional information entropy of the sequence of random variables. Equivalently, one has

H\{ X \} = \lim _{n\toinfty} (H(X _{0,} X _{1,} ... , X _{n-1} , X _{n)})/(n +1) .

See also

• Markov information source
• Asymptotic equipartition property

References

• Robert B. Ash, Information Theory, (1965) Dover Publications.