Bussgang theorem explained

In mathematics, the Bussgang theorem is a theorem of stochastic analysis. The theorem states that the cross-correlation between a Gaussian signal before and after it has passed through a nonlinear operation are equal to the signals auto-correlation up to a constant. It was first published by Julian J. Bussgang in 1952 while he was at the Massachusetts Institute of Technology.^[1]

Statement

Let

\left\{X(t)\right\} be a zero-mean stationary Gaussian random process and \left \{ Y(t) \right\} = g(X(t)) where g( ⋅ ) is a nonlinear amplitude distortion.

If

R _{X(\tau)} is the autocorrelation function of \left\{ X(t) \right\} , then the cross-correlation function of \left\{ X(t) \right\} and \left\{ Y(t) \right\} isR _{XY} (\tau) = CR _{X(\tau),} where C is a constant that depends only on g( ⋅ ) .

It can be further shown that

C = (1)/(\sigma ^{3\sqrt{2\pi}) }\int_^\infty ug(u)e^ \, du.

Derivation for One-bit Quantization

It is a property of the two-dimensional normal distribution that the joint density of

y _{1} and y _{2} depends only on their covariance and is given explicitly by the expressionp(y _{1,y} _{2)} = (1)/(2 \pi \sqrt{ 1- \rho ^{2}) } e^

where

y _{1} and y _{2} are standard Gaussian random variables with correlation \phi _{y _{1y} _{2}} = \rho .

Assume that

r _{2} = Q(y _{2)} , the correlation between y _{1} and r _{2} is,\phi _{y _{1r} _{2}} = (1)/(2 \pi \sqrt{ 1- \rho ^{2}) } \int_^ \int_^ y_1 Q(y_2) e^ \, dy_1 dy_2 .

Since

\int ^{inf ty} _{-inf ty} y _{1} e ^{- (1)/(y ^{2} _{1} + (\rho y _{2})/(1- \rho ^{2}) y _{1} 2 ( 1- \rho ^{2)})} dy _{1} = \rho \sqrt{ 2 \pi ( 1- \rho ^{2)}} y _{2} e ^{(\rho ^{2} y ^{2} _{2})/(2 ( 1- \rho ^{2)})} ,the correlation \phi _{y _{1} r _{2}} may be simplified as\phi _{y _{1} r _{2}} = (\rho)/(\sqrt{ 2 \pi) } \int_^ y_2 Q(y_2) e^ \, dy_2 .The integral above is seen to depend only on the distortion characteristic Q and is independent of \rho .

Remembering that

\rho = \phi _{y _{1} y _{2}} , we observe that for a given distortion characteristic Q , the ratio (\phi _{y _{1} r _{2}})/(\phi _{y _{1} y _{2}}) is K _{Q = (1)/(\sqrt{ 2 \pi)} } \int_^ y_2 Q(y_2) e^ \, dy_2.

Therefore, the correlation can be rewritten in the form

\phi _{y _{1} r _{2}} = K _{Q} \phi _{y _{1} y _{2}} .

The above equation is the mathematical expression of the stated "Bussgang‘s theorem".

If

Q(x) = sign (x) , or called one-bit quantization, then K _{Q=} (2)/(\sqrt{ 2 \pi) } \int_^ y_2 e^ \, dy_2 = \sqrt.

^{[2] [3] [4]}

Arcsine law

If the two random variables are both distorted, i.e.,

r _{1} = Q(y _{1),} r _{2} = Q(y _{2)} , the correlation of r _{1} and r _{2} is

\phi _{r _{1} r _{2}} = \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} Q(y _{1)} Q(y _{2)} p(y _{1,} y _{2)} dy _{1} dy _{2} .

When Q(x) = sign (x) , the expression becomes,

\phi _{r _{1} r _{2}} = (1)/(2 \pi \sqrt{ 1- \rho ^{2}) } \left[\int_{0}^{\infty} \int_{0}^{\infty} e^{-\alpha} \, dy_1 dy_2 + \int_{-\infty}^{0} \int_{-\infty}^{0} e^{-\alpha} \, dy_1 dy_2 - \int_{0}^{\infty} \int_{-\infty}^{0} e^{-\alpha} \, dy_1 dy_2 - \int_{-\infty}^{0} \int_{0}^{\infty} e^{-\alpha} \, dy_1 dy_2 \right]

where \alpha = (y ^{2} _{1} + y ^{2} _{2} - 2 \rho y _{1} y _{2})/(2 ( 1- \rho ^{2)}) .

Noticing that

$\backslash int\_^\; \backslash int\_^\; p(y\_1,y\_2)\; \backslash ,\; dy\_1\; dy\_2\; =\; \backslash frac\; \backslash left[\backslash int\_\{0\}^\{\backslash infty\}\; \backslash int\_\{0\}^\{\backslash infty\}\; e^\{-\backslash alpha\}\; \backslash ,\; dy\_1\; dy\_2\; +\; \backslash int\_\{-\backslash infty\}^\{0\}\; \backslash int\_\{-\backslash infty\}^\{0\}\; e^\{-\backslash alpha\}\; \backslash ,\; dy\_1\; dy\_2\; +\; \backslash int\_\{0\}^\{\backslash infty\}\; \backslash int\_\{-\backslash infty\}^\{0\}\; e^\{-\backslash alpha\}\; \backslash ,\; dy\_1\; dy\_2\; +\; \backslash int\_\{-\backslash infty\}^\{0\}\; \backslash int\_\{0\}^\{\backslash infty\}\; e^\{-\backslash alpha\}\; \backslash ,\; dy\_1\; dy\_2\; \backslash right]=1$,

and

\int ^{inf ty} _{0} \int ^{inf ty} _{0} e ^{-\alpha} dy _{1} dy _{2} = \int ^{0} _{-inf ty} \int ^{0} _{-inf ty} e ^{-\alpha} dy _{1} dy _{2} , \int ^{inf ty} _{0} \int ^{0} _{-inf ty} e ^{-\alpha} dy _{1} dy _{2} = \int ^{0} _{-inf ty} \int ^{inf ty} _{0} e ^{-\alpha} dy _{1} dy _{2} ,

we can simplify the expression of

\phi _{r _{1r} _{2}} as

\phi _{r _{1} r _{2}} = (4)/(2 \pi \sqrt{ 1- \rho ^{2}) } \int_^ \int_^ e^ \, dy_1 dy_2-1

Also, it is convenient to introduce the polar coordinate $y\_1\; =\; R\; \backslash cos\; \backslash theta,\; y\_2\; =\; R\; \backslash sin\; \backslash theta$. It is thus found that

$\backslash phi\_\; =\backslash frac\; \backslash int\_^\; \backslash int\_^\; e^\; R\; \backslash ,\; dR\; d\backslash theta-1=\backslash frac\; \backslash int\_^\; \backslash int\_^\; e^\; R\; \backslash ,\; dR\; d\backslash theta\; -1$.

Integration gives

\phi _{r _{1} r _{2}} = (2 \sqrt{ 1- \rho ^{2})/() } \int_^ \frac - 1= - \frac \arctan \left(\frac \right) \Bigg|_^ -1 =\frac \arcsin(\rho) ，

This is called "Arcsine law", which was first found by J. H. Van Vleck in 1943 and republished in 1966. The "Arcsine law" can also be proved in a simpler way by applying Price's Theorem.^[5]

The function

f(x) = (2)/(\pi) \arcsin x can be approximated as f(x) ≈ (2)/(\pi) x when x is small.

Price's Theorem

Given two jointly normal random variables

y _{1} and y _{2} with joint probability function

{\displaystyle p(y _{1} ,y _{2} ) = { (1)/(2 \pi {\sqrt { 1- \rho ^{2}) }}}e^},

we form the mean

I(\rho) = E(g(y _{1,y} _{2))=\int} ^{+inf ty} ^{-inf ty} \int ^{+inf ty} _{-inf ty} g(y _{1,} y _{2)} p(y _{1,} y _{2)} dy _{1} dy _{2}

of some function g(y _{1,y} _{2)} of (y _{1,} y _{2)} . If g(y _{1,} y _{2)} p(y _{1,} y _{2)} → 0 as (y _{1,} y _{2)} → 0 , then

(\partial ^{n} I(\rho))/(\partial \rho ^{n}) = \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} (\partial ^{2n} g(y _{1,} y _{2)})/(\partial y ^{n} _{1} \partial y ^{n} _{2}) p(y _{1,} y _{2)} dy _{1} dy _{2 =E} \left( (\partial ^{2n} g(y _{1,} y _{2)})/(\partial y ^{n} _{1} \partial y ^{n} _{2}) \right) .

Proof. The joint characteristic function of the random variables y _{1} and y _{2} is by definition the integral

\Phi(\omega _{1,} \omega _{2)=\int} ^{inf ty} ^{-inf ty} \int ^{inf ty} _{-inf ty} p(y _{1,} y _{2)} e ^{j (\omega _{1} y _{1} + \omega _{2} y _{2} )} dy _{1} dy _{2} = \exp \left\{ - (\omega ^{2} _{1} + \omega ^{2} _{2} + 2 \rho \omega _{1} \omega _{2})/(2) \right\} .

From the two-dimensional inversion formula of Fourier transform, it follows that

p(y _{1,} y _{2)} = (1)/(4 \pi ^{2}) \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} \Phi(\omega _{1,} \omega ^{- j (\omega _{1} y _{1} + \omega _{2} y _{2)}} _{2 ) e} d\omega _{1} d\omega _{2 = (1)/(4 \pi ^{2})} \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} \exp \left\{ - (\omega ^{2} _{1} + \omega ^{2} _{2} + 2 \rho \omega _{1} \omega _{2})/(2) \right\} e ^{- j (\omega _{1} y _{1} + \omega _{2} y _{2)}} d\omega _{1} d\omega _{2} .

Therefore, plugging the expression of p(y _{1,} y _{2)} into I(\rho) , and differentiating with respect to \rho , we obtain

\begin{align} (\partial ^{n} I(\rho))/(\partial \rho ^{n}) & = \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} g(y _{1,} y _{2)} p(y _{1,} y _{2)} dy _{1} dy _{2} \\ & = \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} g(y _{1,} y _{2)} \left( (1)/(4 \pi ^{2}) \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} (\partial ^{n} \Phi(\omega _{1,} \omega _{2)})/(\partial \rho ^{n}) e ^{- j(\omega _{1} y _{1} + \omega _{2} y _{2)}} d\omega _{1} d\omega _{2} \right) dy _{1} dy _{2} \\ & = \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} g(y _{1,} y _{2)} \left( (( -1 ) ^{n})/(4 \pi ^{2}) \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} \omega ^{n} _{1} \omega ^{n} _{2} \Phi(\omega _{1,} \omega ^{- j(\omega _{1} y _{1} + \omega _{2} y _{2)}} _{2 ) e} d\omega _{1} d\omega _{2} \right) dy _{1} dy _{2} \\ & = \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} g(y _{1,} y _{2)} \left( (1)/(4 \pi ^{2}) \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} \Phi(\omega _{1,} \omega _{2 ) (\partial ^{2 n} e ^{- j(\omega _{1} y _{1} + \omega _{2} y _{2)}})/(\partial y ^{n} _{1} \partial y ^{n} _{2})} d\omega _{1} d\omega _{2} \right) dy _{1} dy _{2} \\ & = \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} g(y _{1,} y _{2)} (\partial ^{2n} p(y _{1,} y _{2)})/(\partial y ^{n} _{1} \partial y ^{n} _{2}) dy _{1} dy _{2} \\ \end{align}

After repeated integration by parts and using the condition at inf ty , we obtain the Price's theorem.

\begin{align} (\partial ^{n} I(\rho))/(\partial \rho ^{n}) & = \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} g(y _{1,} y _{2)} (\partial ^{2n} p(y _{1,} y _{2)})/(\partial y ^{n} _{1} \partial y ^{n} _{2}) dy _{1} dy _{2} \\ & = \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} (\partial ^{2} g(y _{1,} y _{2)})/(\partial y _{1} \partial y _{2}) (\partial ^{2n-2} p(y _{1,} y _{2)})/(\partial y ^{n -1} _{1} \partial y ^{n -1} _{2}) dy _{1} dy _{2 \\ &= …} \\ & = \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} (\partial ^{2n} g(y _{1,} y _{2)})/(\partial y ^{n} _{1} \partial y ^{n} _{2}) p(y _{1,} y _{2)} dy _{1} dy _{2 \end{align}}

Proof of Arcsine law by Price's Theorem

If

g(y _{1,} y _{2)} = sign (y _{1)} sign (y _{2)} , then (\partial ^{2} g(y _{1,} y _{2)})/(\partial y _{1} \partial y _{2}) = 4 \delta(y _{1)} \delta(y _{2)} where \delta is the Dirac delta function.

Substituting into Price's Theorem, we obtain,

(\partial E( sign (y _{1)} sign (y _{2))})/(\partial \rho) = (\partial I(\rho))/(\partial \rho) = \int ^{inf ty} _{-inf ty} \int ^{inf ty} _{-inf ty} 4 \delta(y _{1)} \delta(y _{2)} p(y _{1,} y _{2)} dy _{1} dy _{2= (2)/(\pi \sqrt{ 1- \rho ^{2})} }.

When \rho =0 , I(\rho) =0 . Thus

E \left( sign (y _{1)} sign (y _{2)} \right) = I(\rho) = (2)/(\pi) \int ^{\rho} _{0} (1)/(\sqrt{ 1- \rho ^{2}) } \, d\rho=\frac \arcsin(\rho),

which is Van Vleck's well-known result of "Arcsine law".

Application

This theorem implies that a simplified correlator can be designed. Instead of having to multiply two signals, the cross-correlation problem reduces to the gating of one signal with another.

Further reading

• E.W. Bai; V. Cerone; D. Regruto (2007) "Separable inputs for the identification of block-oriented nonlinear systems", Proceedings of the 2007 American Control Conference (New York City, July 11–13, 2007) 1548 - 1553

Notes and References

1. J.J. Bussgang,"Cross-correlation function of amplitude-distorted Gaussian signals", Res. Lab. Elec., Mas. Inst. Technol., Cambridge MA, Tech. Rep. 216, March 1952.
2. Vleck. J. H. Van. The Spectrum of Clipped Noise. Radio Research Laboratory Report of Harvard University. 51.
3. Vleck. J. H. Van. Middleton. D.. January 1966. The spectrum of clipped noise. Proceedings of the IEEE. 54. 1. 2–19. 10.1109/PROC.1966.4567. 1558-2256.
4. Price. R.. June 1958. A useful theorem for nonlinear devices having Gaussian inputs. IRE Transactions on Information Theory. 4. 2. 69–72. 10.1109/TIT.1958.1057444. 2168-2712.
5. Book: Papoulis, Athanasios. Probability, Random Variables, and Stochastic Processes. McGraw-Hill. 2002. 0-07-366011-6. 396.