Vectorization (mathematics)

From Wikipedia, the free encyclopedia
Jump to: navigation, search

In mathematics, especially in linear algebra and matrix theory, the vectorization of a matrix is a linear transformation which converts the matrix into a column vector. Specifically, the vectorization of an m×n matrix A, denoted by vec(A), is the mn × 1 column vector obtained by stacking the columns of the matrix A on top of one another:

\mathrm{vec}(A) = [a_{1,1}, ..., a_{m,1}, a_{1,2}, ..., a_{m,2}, ..., a_{1,n}, ..., a_{m,n}]^T

Here a_{i,j} represents the (i,j)-th element of matrix A and the superscript ^T denotes the transpose. Vectorization expresses the isomorphism \mathbf{R}^{m \times n} := \mathbf{R}^m \otimes \mathbf{R}^n \cong \mathbf{R}^{mn} between these vector spaces (of matrices and vectors) in coordinates.

For example, for the 2×2 matrix A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}, the vectorization is \mathrm{vec}(A) = \begin{bmatrix} a \\ c \\ b \\ d \end{bmatrix}.

Compatibility with Kronecker products[edit]

The vectorization is frequently used together with the Kronecker product to express matrix multiplication as a linear transformation on matrices. In particular,

 \mbox{vec}(ABC)=(C^{T}\otimes A)\mbox{vec}(B)

for matrices A, B, and C of dimensions k×l, l×m, and m×n. For example, if  \mbox{ad}_A(X) = AX-XA (the adjoint endomorphism of the Lie algebra gl(n,C) of all n×n matrices with complex entries), then \mbox{vec}(\mbox{ad}_A(X)) = (I_n\otimes A - A^T \otimes I_n ) \mbox{vec}(X), where I_n is the n×n identity matrix.

There are two other useful formulations:

 \mbox{vec}(ABC)=(I_n\otimes AB)\mbox{vec}(C) =(C^{T}B^{T}\otimes I_k)\mbox{vec}(A)
 \mbox{vec}(AB)=(I_m\otimes A)\mbox{vec}(B) =(B^{T}\otimes I_k)\mbox{vec}(A)

Compatibility with Hadamard products[edit]

Vectorization is an algebra homomorphism from the space of n×n matrices with the Hadamard (entrywise) product to Cn with its Hadamard product[disambiguation needed]:

vec(A \circ B) = vec(A) \circ vec(B).

Compatibility with inner products[edit]

Vectorization is a unitary transformation from the space of n×n matrices with the Frobenius (or Hilbert-Schmidt) inner product to Cn :

tr(A* B) = vec(A)* vec(B)

where the superscript * denotes the conjugate transpose.

Half-vectorization[edit]

For a symmetric matrix A, the vector vec(A) contains more information than is strictly necessary, since the matrix is completely determined by the symmetry together with the lower triangular portion, that is, the n(n+1)/2 entries on and below the main diagonal. For such matrices, the half-vectorization is sometimes more useful than the vectorization. The half-vectorization, vech(A), of a symmetric n×n matrix A is the n(n+1)/2 × 1 column vector obtained by vectorizing only the lower triangular part of A:

vech(A) = [ A1,1, ..., An,1, A2,2, ..., An,2, ..., An-1,n-1,An-1,n, An,n ]T.

For example, for the 2×2 matrix A = \begin{bmatrix} a & b \\ b & d \end{bmatrix}, the half-vectorization is vech(A) = \begin{bmatrix} a \\ b \\ d \end{bmatrix}.

There exist unique matrices transforming the half-vectorization of a matrix to its vectorization and vice-versa called, respectively, the duplication matrix and the elimination matrix.

Programming language[edit]

Programming languages that implement matrices may have easy means for vectorization. In Matlab/GNU Octave a matrix A can be vectorized by A(:). In Python NumPy arrays implement the 'flatten' method (although this stacks the rows of the matrix, not the columns), while in R the desired effect can be achieved via the 'c()' or 'as.vector()' functions.

See also[edit]

References[edit]

  • Jan R. Magnus and Heinz Neudecker (1999), Matrix Differential Calculus with Applications in Statistics and Econometrics, 2nd Ed., Wiley. ISBN 0-471-98633-X.
  • Jan R. Magnus (1988), Linear Structures, Oxford University Press. ISBN 0-85264-299-7.