Definite matrix
In linear algebra, a positive-definite matrix is a (Hermitian) matrix which in many ways is analogous to a positive real number. The notion is closely related to a positive-definite symmetric bilinear form (or a sesquilinear form in the complex case).
Definition
An n × n real symmetric matrix M is positive definite if zTMz > 0 for all non-zero vectors z with real entries (), where zT denotes the transpose of z.
For complex matrices, this definition becomes: a Hermitian matrix M is positive definite if z*Mz > 0 for all non-zero complex vectors z, where z* denotes the conjugate transpose of z. The quantity z*Mz is always real because M is a Hermitian matrix. For this reason, positive-definite matrices are often defined to be Hermitian matrices satisfying z*Mz > 0 for non-zero z. The section Non-Hermitian matrices discusses the consequences of dropping the requirement that M be Hermitian.
Examples
The matrix is positive definite. For a vector with entries the quadratic form is when the entries z0, z1 are real and at least one of them nonzero, this is positive.
The matrix is not positive definite. When the quadratic form at z is then
Another example of positive definite matrix is given by
It is positive definite since for any non-zero vector , we have
For a huge class of examples, consider that in Statistics positive definite matrices appears as covariance matrices. In fact all positive definite matrices are a covariance matrix for some probability distribution.
Characterizations
Let M be an n × n Hermitian matrix. The following properties are equivalent to M being positive definite:
1. | All eigenvalues of are positive. Recall that any Hermitian M, by the spectral theorem, may be regarded as a real diagonal matrix D that has been re-expressed in some new coordinate system (i.e., for some unitary matrix P whose rows are orthonormal eigenvectors of M, forming a basis). So this characterization means that M is positive definite if and only if the diagonal elements of D (the eigenvalues) are all positive. In other words, in the basis consisting of the eigenvectors of M, the action of M is component-wise multiplication with a (fixed) element in Cn with positive entries. |
2. | The sesquilinear form
defines an inner product on Cn. (In fact, every inner product on Cn arises in this fashion from a Hermitian positive definite matrix.) |
3. | M is the Gram matrix of some collection of linearly independent vectors
for some k. That is, M satisfies: The vectors xi may optionally be restricted to fall in Cn. In other words, M is of the form A*A where A is not necessarily square but must be injective in general. |
4. | All the following matrices have a positive determinant (Sylvester's criterion):
In other words, all of the leading principal minors are positive. For positive semidefinite matrices, all principal minors have to be non-negative. The leading principal minors alone do not imply positive semidefiniteness, as can be seen from the example |
5. | There exists a unique lower triangular matrix , with strictly positive diagonal elements, that allows the factorization of into
where is the conjugate transpose of . This factorization is called Cholesky decomposition. |
For real symmetric matrices, these properties can be simplified by replacing with , and "conjugate transpose" with "transpose."
Quadratic forms
Echoing condition 2 above, one can also formulate positive-definiteness in terms of quadratic forms. Let K be the field R or C, and V be a vector space over K. A Hermitian form
is a bilinear map such that B(x, y) is always the complex conjugate of B(y, x). Such a function B is called positive definite if B(x, x) > 0 for every nonzero x in V.
Negative-definite, semidefinite and indefinite matrices
The n × n Hermitian matrix M is said to be negative-definite if
for all non-zero (or, all non-zero for the real matrix).
It is called positive-semidefinite (or sometimes nonnegative-definite) if
for all (or, all for the real matrix).
It is called negative-semidefinite if
for all (or, all for the real matrix).
A matrix M is positive-semidefinite if and only if it arises as the Gram matrix of some set of vectors. In contrast to the positive-definite case, these vectors need not be linearly independent.
For any matrix A, the matrix A*A is positive semidefinite, and rank(A) = rank(A*A). Conversely, any positive semidefinite matrix M can be written as M = A*A; this is the Cholesky decomposition.
A Hermitian matrix which is neither positive- nor negative-semidefinite is called indefinite.
A matrix is negative definite if all kth order leading principal minors are negative if k is odd and positive if k is even.
A matrix is negative-definite, negative-semidefinite, or positive-semidefinite if and only if all of its eigenvalues are negative, non-positive, or non-negative, respectively.
Further properties
If is positive-semidefinite, one sometimes writes and if is positive-definite one writes .[1]The notion comes from functional analysis where positive definite matrices define positive operators.
For arbitrary square matrices we write if , i.e. is positive semi-definite. This defines a partial ordering on the set of all square matrices. One can similarly define a strict partial ordering .
1. |
Every positive definite matrix is invertible and its inverse is also positive definite.[2] If then .[3] |
2. | If is positive definite and is a real number, then is positive definite.[4]
If and are positive definite, then the sum [4] and the products and are also positive definite. If , then is also positive definite. |
3. | If then the diagonal entries are real and positive. As a consequence tr. Furthermore
|
4. | A matrix is positive semi-definite if and only if there is a positive semi-definite matrix with . This matrix B is unique[6], is called the square root of M, and is denoted with (the square root B is not to be confused with the matrix L in the Cholesky factorization M = LL*, which is also sometimes called the square root of M). If then . |
5. | If then (Here denotes Kronecker product.) |
6. | For matrices and , write M ˆ N for the entry-wise product of and , i.e. the matrix whose entry is . Then M ˆ N is the Hadamard product of and . The Hadamard product of two positive-definite matrices is again positive-definite and the Hadamard product of two positive-semidefinite matrices is again positive-semidefinite (this result is often called the Schur product theorem).[7] Furthermore, if M and N are positive-semidefinite, then the following inequality, due to Oppenheim, holds: |
7. | Let and Hermitian. If () then ( ) |
8. | If are real matrices then |
9. | If is real, then there is a such that where is the identity matrix. |
Non-Hermitian matrices
A real matrix M may have the property that xTMx > 0 for all nonzero real vectors x without being symmetric. The matrix
satisfies this property, because for all real vectors such that ,
In general, we have xTMx > 0 for all real nonzero vectors x if and only if the symmetric part, (M + MT) / 2, is positive definite.
The situation for complex matrices may be different, depending on how one generalizes the inequality z*Mz > 0. If z*Mz is real for all complex vectors z, then the matrix M is necessarily Hermitian. So, if we require that z*Mz be real and positive, then M is automatically Hermitian. On the other hand, we have that Re(z*Mz) > 0 for all complex nonzero vectors z if and only if the Hermitian part, (M + M*) / 2, is positive definite.
In summary, the distinguishing feature between the real and complex case is that, a bounded positive operator on a complex Hilbert space is necessarily Hermitian, or self adjoint. The general claim can be argued using the polarization identity. That is no longer true in the real case.
There is no agreement in the literature on the proper definition of positive-definite for non-Hermitian matrices.
See also
- Cholesky decomposition
- Positive-definite function
- Positive definite kernel
- Schur complement
- Square root of a matrix
- Sylvester's criterion
- Covariance matrix
Notes
- ^ This may be confusing, as sometimes nonnegative matrices are also denoted in this way. A common alternative notation is and for positive semidefinite and positive definite matrices, respectively.
- ^ Horn & Johnson (1985), p. 397
- ^ Horn & Johnson (1985), Corollary 7.7.4(a)
- ^ a b Horn & Johnson (1985), Observation 7.1.3
- ^ Horn & Johnson (1985), p. 398
- ^ Horn & Johnson (1985), Theorem 7.2.6 with k = 2
- ^ Horn & Johnson (1985), Theorem 7.5.3
- ^ Horn & Johnson (1985), Theorem 7.8.6
References
- Horn, Roger A.; Johnson, Charles R. (1985), Matrix Analysis, Cambridge University Press, ISBN 978-0-521-38632-6.
- Rajendra Bhatia. Positive definite matrices. Princeton Series in Applied Mathematics, 2007. ISBN 978-0691129181.