Quadratic form (statistics)

From Wikipedia, the free encyclopedia
Jump to: navigation, search

In multivariate statistics, if \epsilon is a vector of n random variables, and \Lambda is an n-dimensional symmetric matrix, then the scalar quantity \epsilon^T\Lambda\epsilon is known as a quadratic form in \epsilon.


It can be shown that[1]

\operatorname{E}\left[\epsilon^T\Lambda\epsilon\right]=\operatorname{tr}\left[\Lambda \Sigma\right] + \mu^T\Lambda\mu

where \mu and \Sigma are the expected value and variance-covariance matrix of \epsilon, respectively, and tr denotes the trace of a matrix. This result only depends on the existence of \mu and \Sigma; in particular, normality of \epsilon is not required.

A book treatment of the topic of quadratic forms in random variables is [2]


Since the quadratic form is a scalar quantity  \operatorname{E}\left[\epsilon^T\Lambda\epsilon\right] = \operatorname{tr}(\operatorname{E}[\epsilon^T\Lambda\epsilon]). Note that both \operatorname{E} and \operatorname{tr} are linear operators, so  \operatorname{E} \circ \operatorname{tr} = \operatorname{tr} \circ \operatorname{E} . It follows that

 \operatorname{tr}(\operatorname{E}\left[\epsilon^T\Lambda\epsilon\right]) = \operatorname{E}[\operatorname{tr}(\epsilon^T\Lambda\epsilon)],

and that, by the cyclic property of the trace operator,

 \operatorname{E}[\operatorname{tr}(\epsilon^T\Lambda\epsilon)] = \operatorname{E}[\operatorname{tr}(\Lambda\epsilon\epsilon^T)] 
= \operatorname{tr} (\Lambda(\Sigma + \mu\mu^T)) = \operatorname{tr}(\Lambda\Sigma) + \mu^T\Lambda\mu.


In general, the variance of a quadratic form depends greatly on the distribution of \epsilon. However, if \epsilon does follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable. Assume for the moment that \Lambda is a symmetric matrix. Then,

\operatorname{var}\left[\epsilon^T\Lambda\epsilon\right]=2\operatorname{tr}\left[\Lambda \Sigma\Lambda \Sigma\right] + 4\mu^T\Lambda\Sigma\Lambda\mu

In fact, this can be generalized to find the covariance between two quadratic forms on the same \epsilon (once again, \Lambda_1 and \Lambda_2 must both be symmetric):

\operatorname{cov}\left[\epsilon^T\Lambda_1\epsilon,\epsilon^T\Lambda_2\epsilon\right]=2\operatorname{tr}\left[\Lambda _1\Sigma\Lambda_2 \Sigma\right] + 4\mu^T\Lambda_1\Sigma\Lambda_2\mu

Computing the variance in the non-symmetric case[edit]

Some texts incorrectly state that the above variance or covariance results hold without requiring \Lambda to be symmetric. The case for general \Lambda can be derived by noting that




But this is a quadratic form in the symmetric matrix \tilde{\Lambda}=\left(\Lambda+\Lambda^T\right)/2, so the mean and variance expressions are the same, provided \Lambda is replaced by \tilde{\Lambda} therein.

Examples of quadratic forms[edit]

In the setting where one has a set of observations y and an operator matrix H, then the residual sum of squares can be written as a quadratic form in y:


For procedures where the matrix H is symmetric and idempotent, and the errors are Gaussian with covariance matrix \sigma^2I, \textrm{RSS}/\sigma^2 has a chi-squared distribution with k degrees of freedom and noncentrality parameter \lambda, where


may be found by matching the first two central moments of a noncentral chi-squared random variable to the expressions given in the first two sections. If Hy estimates \mu with no bias, then the noncentrality \lambda is zero and \textrm{RSS}/\sigma^2 follows a central chi-squared distribution.


  1. ^ Douglas, Bates. "Quadratic Forms of Random Variables" (PDF). STAT 849 lectures. Retrieved August 21, 2011. 
  2. ^ Mathai, A. M. and Provost, Serge B. (1992). Quadratic Forms in Random Variables. CRC Press. p. 424. ISBN 978-0824786915. 

See also[edit]