For real matrices and vectors, the condition of being Hermitian reduces to that of being symmetric, and the conjugate transpose to the usual transpose. Note that for any non-zero real scalar c. Recall that a Hermitian (or real symmetric) matrix has real eigenvalues. It can be shown that, for a given matrix, the Rayleigh quotient reaches its minimum value (the smallest eigenvalue of M) when x is (the corresponding eigenvector). Similarly, and .
The range of the Rayleigh quotient (for matrix that is not necessarily Hermitian) is called a numerical range, (or spectrum in functional analysis). When the matrix is Hermitian, the numerical range is equal to the spectral norm. Still in functional analysis, is known as the spectral radius. In the context of C*-algebras or algebraic quantum mechanics, the function that to M associates the Rayleigh-Ritz quotient R(M,x) for a fixed x and M varying through the algebra would be referred to as "vector state" of the algebra.
As stated in the introduction, it is . This is immediate after observing that the Rayleigh quotient is a weighted average of eigenvalues of M:
where is the th eigenpair after orthonormalization and is the th coordinate of x in the eigenbasis. It is then easy to verify that the bounds are attained at the corresponding eigenvectors .
The fact that the quotient is a weighted average of the eigenvalues can be used to identify the second, the third, ... largest eigenvalues. Let be the eigenvalues in decreasing order. If is constrained to be orthogonal to , in which case , then has the maximum , which is achieved when .
An empirical covariance matrixM can be represented as the product A' A of the data matrixA pre-multiplied by its transpose A'. Being a positive semi-definite matrix, M has non-negative eigenvalues, and orthogonal (or othogonalisable) eigenvectors, which can be demonstrated as follows.
Firstly, that the eigenvalues are non-negative:
Secondly, that the eigenvectors vi are orthogonal to one another:
If the eigenvalues are different – in the case of multiplicity, the basis can be orthogonalized.
To now establish that the Rayleigh quotient is maximised by the eigenvector with the largest eigenvalue, consider decomposing an arbitrary vector x on the basis of the eigenvectors vi:
is the coordinate of x orthogonally projected onto vi. Therefore we have:
which, by orthogonality of the eigenvectors, becomes:
The last representation establishes that the Rayleigh quotient is the sum of the squared cosines of the angles formed by the vector x and each eigenvector vi, weighted by corresponding eigenvalues.
If a vector x maximizes , then any non-zero scalar multiple kx also maximizes R, so the problem can be reduced to the Lagrange problem of maximizing under the constraint that .
Define: βi = α2 i. This then becomes a linear program, which always attains its maximum at one of the corners of the domain. A maximum point will have and for all i > 1 (when the eigenvalues are ordered by decreasing magnitude).
Thus, as advertised, the Rayleigh quotient is maximised by the eigenvector with the largest eigenvalue.