In mathematics, especially in linear algebra and matrix theory, the vectorization of a matrix is a linear transformation which converts the matrix into a column vector. Specifically, the vectorization of an m × n matrix A, denoted vec(A), is the mn × 1 column vector obtained by stacking the columns of the matrix A on top of one another:
For example, for the 2×2 matrix = , the vectorization is .
Compatibility with Kronecker products
for matrices A, B, and C of dimensions k×l, l×m, and m×n. For example, if (the adjoint endomorphism of the Lie algebra gl(n, C) of all n×n matrices with complex entries), then , where is the n×n identity matrix.
There are two other useful formulations:
Compatibility with Hadamard products
Compatibility with inner products
- tr(AH B) = vec(A)H vec(B)
where the superscript H denotes the conjugate transpose.
Vectorization as a linear sum
The matrix vectorization operation can be written in terms of a linear sum. Let X be an m × n that we want to vectorize, and let ei be the i-th canonical basis vector for the n-dimensional space, that is . Let Bi be a (mn) × m block matrix defined as follows:
Bi consists of n block matrices of size m × m, stacked column-wise, and all these matrices are all-zero except for the i-th one, which is a m × m identity matrix Im.
Then the vectorized version of X can be expressed as follows:
Multiplication of X by ei extracts the i-th column, while multiplication by Bi puts it into the desired position in the final vector.
Alternatively, the linear sum can be expressed using the Kronecker product:
For a symmetric matrix A, the vector vec(A) contains more information than is strictly necessary, since the matrix is completely determined by the symmetry together with the lower triangular portion, that is, the n(n + 1)/2 entries on and below the main diagonal. For such matrices, the half-vectorization is sometimes more useful than the vectorization. The half-vectorization, vech(A), of a symmetric n × n matrix A is the n(n + 1)/2 × 1 column vector obtained by vectorizing only the lower triangular part of A:
- vech(A) = [ A1,1, ..., An,1, A2,2, ..., An,2, ..., An−1,n−1,An,n−1, An,n ]T.
For example, for the 2×2 matrix A = , the half-vectorization is vech(A) = .
Programming languages that implement matrices may have easy means for vectorization.
In Matlab/GNU Octave a matrix
A can be vectorized by
GNU Octave also allows vectorization and half-vectorization with
vech(A) respectively. Julia has the
vec(A) function as well.
In Python NumPy arrays implement the 'flatten' method, while in R the desired effect can be achieved via the
as.vector() functions. In R, function
vec() of package 'ks' allows vectorization and function
vech() implemented in both packages 'ks' and 'sn' allows half-vectorization.
- 1.^ ^ The identity for row-major vectorization is .
- H.D. Macedo, J.N. Oliveira, Typing linear algebra: A biproduct-oriented approach, Science of Computer Programming, Volume 78, Issue 11, 2013, Pages 2160-2191.
- Adelchi Azzalini (2017), The R package 'sn': The Skew-Normal and Related Distributions such as the Skew-t, R package version 1.5.1. http://azzalini.stat.unipd.it/SN
- Jan R. Magnus and Heinz Neudecker (1999), Matrix Differential Calculus with Applications in Statistics and Econometrics, 2nd Ed., Wiley. ISBN 0-471-98633-X.
- Jan R. Magnus (1988), Linear Structures, Oxford University Press. ISBN 0-85264-299-7.
- Tarn Duong (2018), ks: Kernel Smoothing, R package version 1.11.0. https://CRAN.R-project.org/package=ks