Bussgang theorem

In mathematics, the Bussgang theorem is a theorem of stochastic analysis. The theorem states that the cross-correlation of a Gaussian signal before and after it has passed through a nonlinear operation are equal up to a constant. It was first published by Julian J. Bussgang in 1952 while he was at the Massachusetts Institute of Technology.

Statement

Let $\left\{X(t)\right\}$ be a zero-mean stationary Gaussian random process and $\left\{Y(t)\right\}=g(X(t))$ where $g(\cdot )$ is a nonlinear amplitude distortion.

If $R_{X}(\tau )$ is the autocorrelation function of $\left\{X(t)\right\}$, then the cross-correlation function of $\left\{X(t)\right\}$ and $\left\{Y(t)\right\}$ is

$R_{XY}(\tau )=CR_{X}(\tau ),$

where $C$ is a constant that depends only on $g(\cdot )$.

It can be further shown that

$C={\frac {1}{\sigma ^{3}{\sqrt {2\pi }}}}\int _{-\infty }^{\infty }ug(u)e^{-{\frac {u^{2}}{2\sigma ^{2}}}}\,du.$

Derivation for One-bit Quantization

It is a property of the two-dimensional normal distribution that the joint density of $y_{1}$ and $y_{2}$ depends only on their covariance and is given explicitly by the expression

$p(y_{1},y_{2})={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}$

where $y_{1}$ and $y_{2}$ are standard Gaussian random variables with correlation $\phi _{y_{1}y_{2}}=\rho$.

Assume that $r_{2}=Q(y_{2})$, the correlation between $y_{1}$ and $r_{2}$ is,

$\phi _{y_{1}r_{2}}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }y_{1}Q(y_{2})e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}\,dy_{1}dy_{2}$.

Since

$\int _{-\infty }^{\infty }y_{1}e^{-{\frac {1}{2(1-\rho ^{2})}}y_{1}^{2}+{\frac {\rho y_{2}}{1-\rho ^{2}}}y_{1}}\,dy_{1}=\rho {\sqrt {2\pi (1-\rho ^{2})}}y_{2}e^{\frac {\rho ^{2}y_{2}^{2}}{2(1-\rho ^{2})}}$,

the correlation $\phi _{y_{1}r_{2}}$ may be simplified as

$\phi _{y_{1}r_{2}}={\frac {\rho }{\sqrt {2\pi }}}\int _{-\infty }^{\infty }y_{2}Q(y_{2})e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}$.

The integral above is seen to depend only on the distortion characteristic $Q()$ and is independent of $\rho$.

Remembering that $\rho =\phi _{y_{1}y_{2}}$, we observe that for a given distortion characteristic $Q()$, the ratio ${\frac {\phi _{y_{1}r_{2}}}{\phi _{y_{1}y_{2}}}}$ is $K_{Q}={\frac {1}{\sqrt {2\pi }}}\int _{-\infty }^{\infty }y_{2}Q(y_{2})e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}$.

Therefore, the correlation can be rewritten in the form

$\phi _{y_{1}r_{2}}=K_{Q}\phi _{y_{1}y_{2}}$.

The above equation is the mathematical expression of the stated "Bussgang‘s theorem".

If $Q(x)={\text{sign}}(x)$, or called one-bit quantization, then $K_{Q}={\frac {2}{\sqrt {2\pi }}}\int _{0}^{\infty }y_{2}e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}={\sqrt {\frac {2}{\pi }}}$.

Arcsine law

If the two random variables are both distorted, i.e., $r_{1}=Q(y_{1}),r_{2}=Q(y_{2})$, the correlation of $r_{1}$ and $r_{2}$ is

$\phi _{r_{1}r_{2}}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }Q(y_{1})Q(y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}$.

When $Q(x)={\text{sign}}(x)$, the expression becomes,

$\phi _{r_{1}r_{2}}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\left[\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}-\int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}-\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}\right]$

where $\alpha ={\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}$.

Noticing that

$\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }p(y_{1},y_{2})\,dy_{1}dy_{2}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\left[\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}+\int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}\right]=1$,

and $\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}=\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}$, $\int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}=\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}$,

we can simplify the expression of $\phi _{r_{1}r_{2}}$ as

$\phi _{r_{1}r_{2}}={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}-1$

Also, it is convenient to introduce the polar coordinate $y_{1}=R\cos \theta ,y_{2}=R\sin \theta$. It is thus found that

$\phi _{r_{1}r_{2}}={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\pi /2}\int _{0}^{\infty }e^{-{\frac {R^{2}-2R^{2}\rho \cos \theta \sin \theta \ }{2(1-\rho ^{2})}}}R\,dRd\theta -1={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\pi /2}\int _{0}^{\infty }e^{-{\frac {R^{2}(1-\rho \sin 2\theta )}{2(1-\rho ^{2})}}}R\,dRd\theta -1$.

Integration gives

$\phi _{r_{1}r_{2}}={\frac {2{\sqrt {1-\rho ^{2}}}}{\pi }}\int _{0}^{\pi /2}{\frac {d\theta }{1-\rho \sin 2\theta }}-1=-{\frac {2}{\pi }}\arctan \left({\frac {\rho -\tan \theta }{\sqrt {1-\rho ^{2}}}}\right){\Bigg |}_{0}^{\pi /2}-1={\frac {2}{\pi }}\arcsin(\rho )$，

This is called "Arcsine law", which was first found by J. H. Van Vleck in 1943 and republished in 1966. The "Arcsine law" can also be proved in a simpler way by applying Price's Theorem.

The function $f(x)={\frac {2}{\pi }}\arcsin x$ can be approximated as $f(x)\approx {\frac {2}{\pi }}x$ when $x$ is small.

Price's Theorem

Given two jointly normal random variables $y_{1}$ and $y_{2}$ with joint probability function

${\displaystyle p(y_{1},y_{2})={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}}$,

we form the mean

$I(\rho )=E(g(y_{1},y_{2}))=\int _{-\infty }^{+\infty }\int _{-\infty }^{+\infty }g(y_{1},y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}$

of some function $g(y_{1},y_{2})$ of $(y_{1},y_{2})$. If $g(y_{1},y_{2})p(y_{1},y_{2})\rightarrow 0$ as $(y_{1},y_{2})\rightarrow 0$, then

${\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}p(y_{1},y_{2})\,dy_{1}dy_{2}=E\left({\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\right)$.

Proof. The joint characteristic function of the random variables $y_{1}$ and $y_{2}$ is by definition the integral

$\Phi (\omega _{1},\omega _{2})=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }p(y_{1},y_{2})e^{j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,dy_{1}dy_{2}=\exp \left\{-{\frac {\omega _{1}^{2}+\omega _{2}^{2}+2\rho \omega _{1}\omega _{2}}{2}}\right\}$.

From the two-dimensional inversion formula of Fourier transform, it follows that

$p(y_{1},y_{2})={\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\Phi (\omega _{1},\omega _{2})e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}={\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\exp \left\{-{\frac {\omega _{1}^{2}+\omega _{2}^{2}+2\rho \omega _{1}\omega _{2}}{2}}\right\}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}$.

Therefore, plugging the expression of $p(y_{1},y_{2})$ into $I(\rho )$, and differentiating with respect to $\rho$, we obtain

${\begin{aligned}{\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{n}\Phi (\omega _{1},\omega _{2})}{\partial \rho ^{n}}}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {(-1)^{n}}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\omega _{1}^{n}\omega _{2}^{n}\Phi (\omega _{1},\omega _{2})e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\Phi (\omega _{1},\omega _{2}){\frac {\partial ^{2n}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2}){\frac {\partial ^{2n}p(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,dy_{1}dy_{2}\\\end{aligned}}$

After repeated integration by parts and using the condition at $\infty$, we obtain the Price's theorem.

${\begin{aligned}{\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2}){\frac {\partial ^{2n}p(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2}g(y_{1},y_{2})}{\partial y_{1}\partial y_{2}}}{\frac {\partial ^{2n-2}p(y_{1},y_{2})}{\partial y_{1}^{n-1}\partial y_{2}^{n-1}}}\,dy_{1}dy_{2}\\&=\cdots \\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}p(y_{1},y_{2})\,dy_{1}dy_{2}\end{aligned}}$

Proof of Arcsine law by Price's Theorem

If $g(y_{1},y_{2})={\text{sign}}(y_{1}){\text{sign}}(y_{2})$, then ${\frac {\partial ^{2}g(y_{1},y_{2})}{\partial y_{1}\partial y_{2}}}=4\delta (y_{1})\delta (y_{2})$ where $\delta ()$ is the Dirac delta function.

Substituting into Price's Theorem, we obtain,

${\frac {\partial E({\text{sign}}(y_{1}){\text{sign}}(y_{2}))}{\partial \rho }}={\frac {\partial I(\rho )}{\partial \rho }}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }4\delta (y_{1})\delta (y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}={\frac {2}{\pi {\sqrt {1-\rho ^{2}}}}}$.

When $\rho =0$, $I(\rho )=0$. Thus

$E\left({\text{sign}}(y_{1}){\text{sign}}(y_{2})\right)=I(\rho )={\frac {2}{\pi }}\int _{0}^{\rho }{\frac {1}{\sqrt {1-\rho ^{2}}}}\,d\rho ={\frac {2}{\pi }}\arcsin(\rho )$,

which is Van Vleck's well-known result of "Arcsine law".

Application

This theorem implies that a simplified correlator can be designed. Instead of having to multiply two signals, the cross-correlation problem reduces to the gating of one signal with another.

Further reading

• E.W. Bai; V. Cerone; D. Regruto (2007) "Separable inputs for the identification of block-oriented nonlinear systems", Proceedings of the 2007 American Control Conference (New York City, July 11–13, 2007) 1548–1553