# Schur product theorem

In mathematics, particularly in linear algebra, the Schur product theorem states that the Hadamard product of two positive definite matrices is also a positive definite matrix. The result is named after Issai Schur[1] (Schur 1911, p. 14, Theorem VII) (note that Schur signed as J. Schur in Journal für die reine und angewandte Mathematik.[2][3])

## Proof

### Proof using the trace formula

For any matrices ${\displaystyle M}$ and ${\displaystyle N}$, the Hadamard product ${\displaystyle M\circ N}$ considered as a bilinear form acts on vectors ${\displaystyle a,b}$ as

${\displaystyle a^{*}(M\circ N)b=\operatorname {tr} (M^{T}\operatorname {diag} (a^{*})N\operatorname {diag} (b))}$

where ${\displaystyle \operatorname {tr} }$ is the matrix trace and ${\displaystyle \operatorname {diag} (a)}$ is the diagonal matrix having as diagonal entries the elements of ${\displaystyle a}$.

Suppose ${\displaystyle M}$ and ${\displaystyle N}$ are positive definite, and so Hermitian. We can consider their square-roots ${\displaystyle M^{\frac {1}{2}}}$ and ${\displaystyle N^{\frac {1}{2}}}$, which are also Hermitian, and write

${\displaystyle \operatorname {tr} (M^{T}\operatorname {diag} (a^{*})N\operatorname {diag} (b))=\operatorname {tr} ({\overline {M}}^{\frac {1}{2}}{\overline {M}}^{\frac {1}{2}}\operatorname {diag} (a^{*})N^{\frac {1}{2}}N^{\frac {1}{2}}\operatorname {diag} (b))=\operatorname {tr} ({\overline {M}}^{\frac {1}{2}}\operatorname {diag} (a^{*})N^{\frac {1}{2}}N^{\frac {1}{2}}\operatorname {diag} (b){\overline {M}}^{\frac {1}{2}})}$

Then, for ${\displaystyle a=b}$, this is written as ${\displaystyle \operatorname {tr} (A^{*}A)}$ for ${\displaystyle A=N^{\frac {1}{2}}\operatorname {diag} (a){\overline {M}}^{\frac {1}{2}}}$ and thus is strictly positive for ${\displaystyle A\neq 0}$, which occurs if and only if ${\displaystyle a\neq 0}$. This shows that ${\displaystyle (M\circ N)}$ is a positive definite matrix.

### Proof using Gaussian integration

#### Case of M = N

Let ${\displaystyle X}$ be an ${\displaystyle n}$-dimensional centered Gaussian random variable with covariance ${\displaystyle \langle X_{i}X_{j}\rangle =M_{ij}}$. Then the covariance matrix of ${\displaystyle X_{i}^{2}}$ and ${\displaystyle X_{j}^{2}}$ is

${\displaystyle \operatorname {Cov} (X_{i}^{2},X_{j}^{2})=\langle X_{i}^{2}X_{j}^{2}\rangle -\langle X_{i}^{2}\rangle \langle X_{j}^{2}\rangle }$

Using Wick's theorem to develop ${\displaystyle \langle X_{i}^{2}X_{j}^{2}\rangle =2\langle X_{i}X_{j}\rangle ^{2}+\langle X_{i}^{2}\rangle \langle X_{j}^{2}\rangle }$ we have

${\displaystyle \operatorname {Cov} (X_{i}^{2},X_{j}^{2})=2\langle X_{i}X_{j}\rangle ^{2}=2M_{ij}^{2}}$

Since a covariance matrix is positive definite, this proves that the matrix with elements ${\displaystyle M_{ij}^{2}}$ is a positive definite matrix.

#### General case

Let ${\displaystyle X}$ and ${\displaystyle Y}$ be ${\displaystyle n}$-dimensional centered Gaussian random variables with covariances ${\displaystyle \langle X_{i}X_{j}\rangle =M_{ij}}$, ${\displaystyle \langle Y_{i}Y_{j}\rangle =N_{ij}}$ and independent from each other so that we have

${\displaystyle \langle X_{i}Y_{j}\rangle =0}$ for any ${\displaystyle i,j}$

Then the covariance matrix of ${\displaystyle X_{i}Y_{i}}$ and ${\displaystyle X_{j}Y_{j}}$ is

${\displaystyle \operatorname {Cov} (X_{i}Y_{i},X_{j}Y_{j})=\langle X_{i}Y_{i}X_{j}Y_{j}\rangle -\langle X_{i}Y_{i}\rangle \langle X_{j}Y_{j}\rangle }$

Using Wick's theorem to develop

${\displaystyle \langle X_{i}Y_{i}X_{j}Y_{j}\rangle =\langle X_{i}X_{j}\rangle \langle Y_{i}Y_{j}\rangle +\langle X_{i}Y_{i}\rangle \langle X_{j}Y_{j}\rangle +\langle X_{i}Y_{j}\rangle \langle X_{j}Y_{i}\rangle }$

and also using the independence of ${\displaystyle X}$ and ${\displaystyle Y}$, we have

${\displaystyle \operatorname {Cov} (X_{i}Y_{i},X_{j}Y_{j})=\langle X_{i}X_{j}\rangle \langle Y_{i}Y_{j}\rangle =M_{ij}N_{ij}}$

Since a covariance matrix is positive definite, this proves that the matrix with elements ${\displaystyle M_{ij}N_{ij}}$ is a positive definite matrix.

### Proof using eigendecomposition

#### Proof of positive semidefiniteness

Let ${\displaystyle M=\sum \mu _{i}m_{i}m_{i}^{T}}$ and ${\displaystyle N=\sum \nu _{i}n_{i}n_{i}^{T}}$. Then

${\displaystyle M\circ N=\sum _{ij}\mu _{i}\nu _{j}(m_{i}m_{i}^{T})\circ (n_{j}n_{j}^{T})=\sum _{ij}\mu _{i}\nu _{j}(m_{i}\circ n_{j})(m_{i}\circ n_{j})^{T}}$

Each ${\displaystyle (m_{i}\circ n_{j})(m_{i}\circ n_{j})^{T}}$ is positive semidefinite (but, except in the 1-dimensional case, not positive definite, since they are rank 1 matrices). Also, ${\displaystyle \mu _{i}\nu _{j}>0}$ thus the sum ${\displaystyle M\circ N}$ is also positive semidefinite.

#### Proof of definiteness

To show that the result is positive definite requires further proof. We shall show that for any vector ${\displaystyle a\neq 0}$, we have ${\displaystyle a^{T}(M\circ N)a>0}$. Continuing as above, each ${\displaystyle a^{T}(m_{i}\circ n_{j})(m_{i}\circ n_{j})^{T}a\geq 0}$, so it remains to show that there exist ${\displaystyle i}$ and ${\displaystyle j}$ for which the inequality is strict. For this we observe that

${\displaystyle a^{T}(m_{i}\circ n_{j})(m_{i}\circ n_{j})^{T}a=\left(\sum _{k}m_{i,k}n_{j,k}a_{k}\right)^{2}}$

Since ${\displaystyle N}$ is positive definite, there is a ${\displaystyle j}$ for which ${\displaystyle n_{j,k}a_{k}}$ is not 0 for all ${\displaystyle k}$, and then, since ${\displaystyle M}$ is positive definite, there is an ${\displaystyle i}$ for which ${\displaystyle m_{i,k}n_{j,k}a_{k}}$ is not 0 for all ${\displaystyle k}$. Then for this ${\displaystyle i}$ and ${\displaystyle j}$ we have ${\displaystyle \left(\sum _{k}m_{i,k}n_{j,k}a_{k}\right)^{2}>0}$. This completes the proof.

## References

1. ^ "Bemerkungen zur Theorie der beschränkten Bilinearformen mit unendlich vielen Veränderlichen". Journal für die reine und angewandte Mathematik (Crelle's Journal). 1911 (140): 1–28. 1911. doi:10.1515/crll.1911.140.1.
2. ^ Zhang, Fuzhen, ed. (2005). "The Schur Complement and Its Applications". Numerical Methods and Algorithms. 4. doi:10.1007/b105056. ISBN 0-387-24271-6., page 9, Ch. 0.6 Publication under J. Schur
3. ^ Ledermann, W. (1983). "Issai Schur and His School in Berlin". Bulletin of the London Mathematical Society. 15 (2): 97–106. doi:10.1112/blms/15.2.97.