User:SirMeowMeow/sandbox/Matrices

Definition

For some natural choice of $r$ rows and $c$ columns, a matrix $\Phi$ of size $r\times c$ over a field ${\mathsf {F}}$ is a collection of elements $\Phi _{ij}\in {\mathsf {F}}$ indexed by $(i,j)\in [r]\times [c]$. Unless specified, the elements of a matrix are assumed to be scalars, but may also be elements from a ring, or something more general.

$\Phi ={\begin{bmatrix}\Phi _{11}&\Phi _{12}&\cdots &\Phi _{1c}\\\Phi _{21}&\Phi _{22}&\cdots &\Phi _{2c}\\\vdots &\vdots &\ddots &\vdots \\\Phi _{r1}&\Phi _{r2}&\cdots &\Phi _{rc}\\\end{bmatrix}}$

The set of all matrices with elements from ${\mathsf {F}}$ and with indices from $(i,j)\in [r]\times [c]$ is denoted ${\mathcal {M}}(r,c:{\mathsf {F}})$ or ${\mathsf {F}}^{r,c}$.

Notation

$\mathbf {r\times c}$	Example
$1\times 3$	${\begin{bmatrix}1&2&3\\\end{bmatrix}}$
$3\times 1$	${\begin{bmatrix}1\\2\\3\\\end{bmatrix}}$
$2\times 3$	${\begin{bmatrix}1&2&3\\4&5&6\\\end{bmatrix}}$
$3\times 2$	${\begin{bmatrix}1&4\\2&5\\3&6\\\end{bmatrix}}$
$3\times 3$	${\begin{bmatrix}-4&-3&-2\\-1&0&1\\2&3&4\\\end{bmatrix}}$

Let $\Phi$ be an $r\times c$ matrix whose elements are from ${\mathsf {F}}$. Any individual entry may be referenced as $\Phi _{ij}$ for the $i$-th row and the $j$-th column.

Rows

The $i$-th row vector in a matrix $\Phi$ is defined as the subset of elements which shares some $i$-th index, and ordered by the $j$-th index.

The column index can be omitted for brevity by simply noting $\Phi _{i}$ for the $i$-th row.$$\operatorname {row} (\Phi )=[\Phi _{1*}\cdots \Phi _{r*}]$$

Columns

Let $\operatorname {col}$ be a function which maps a matrix to a partition of its elements with an equivalence relation on the column row index, ordered by its column index.$$\operatorname {col} (\Phi )=[\Phi _{*1}\cdots \Phi _{*c}]$$

Addition of Matrices

Let $\Phi ,\Psi$ be matrices from ${\mathsf {F}}^{r\times c}$. Then the sum of matrices is defined as entry-wise field addition.

${\begin{bmatrix}\Phi _{11}&\cdots &\Phi _{1c}\\\vdots &\ddots &\vdots \\\Phi _{r1}&\cdots &\Phi _{rc}\\\end{bmatrix}}+{\begin{bmatrix}\Psi _{11}&\cdots &\Psi _{1c}\\\vdots &\ddots &\vdots \\\Psi _{r1}&\cdots &\Psi _{rc}\\\end{bmatrix}}:={\begin{bmatrix}(\Phi _{11}+\Psi _{11})&\cdots &(\Phi _{1c}+\Psi _{1c})\\\vdots &\ddots &\vdots \\(\Phi _{r1}+\Psi _{r1})&\cdots &(\Phi _{rc}+\Psi _{rc})\\\end{bmatrix}}$

Scaling of Matrices

Let $\Phi$ be a matrix from ${\mathsf {F}}^{r\times c}$, and let $\lambda \in {\mathsf {F}}$. The scalar multiplication of matrices is defined:

$\lambda {\begin{bmatrix}\Phi _{11}&\cdots &\Phi _{1c}\\\vdots &\ddots &\vdots \\\Phi _{r1}&\cdots &\Phi _{rc}\\\end{bmatrix}}:={\begin{bmatrix}\lambda \Phi _{11}&\cdots &\lambda \Phi _{1c}\\\vdots &\ddots &\vdots \\\lambda \Phi _{r1}&\cdots &\lambda \Phi _{rc}\\\end{bmatrix}}$

Transposition

As a matrix is a collection of double-indexed scalars $\Phi _{ij}\in {\mathsf {F}}$, the transposition is a function ${\mathsf {F}}^{r,c}\to {\mathsf {F}}^{c,r}$ of the form $(\Phi )\mapsto \Phi ^{\intercal }$, defined as a mapping which swaps the positions of indices.

$\Phi _{ij}^{\intercal }:=\Phi _{ji}$

$transpose{\begin{bmatrix}\Phi _{11}&\cdots &\Phi _{1c}\\\vdots &\ddots &\vdots \\\Phi _{r1}&\cdots &\Phi _{rc}\\\end{bmatrix}}:={\begin{bmatrix}\Phi _{11}&\cdots &\Phi _{1r}\\\vdots &\ddots &\vdots \\\Phi _{c1}&\cdots &\Phi _{cr}\\\end{bmatrix}}$

Observations

The transposition of a product $\Psi \Phi$ is equal to the product of their transpositions, but in reverse order.

$(\Psi \Phi )^{\intercal }=\Phi ^{\intercal }\Psi ^{\intercal }$

Matrix-Vector Product

Let $\Phi :{\mathsf {F}}^{c}\to {\mathsf {F}}^{r}$.

Let $\Phi ^{r\times c}$ be a matrix, let $[\alpha _{1}\cdots \alpha _{c}]\in {\mathsf {F}}^{c}$, and let $[\beta _{1}\cdots \beta _{r}]\in {\mathsf {F}}^{r}$.

A matrix-vector product is is a mapping ${\mathsf {F}}^{r\times c}\times {\mathsf {F}}^{c}\to {\mathsf {F}}^{r}$, such that:

${\begin{bmatrix}\Phi _{11}&\cdots &\Phi _{1c}\\\vdots &\ddots &\vdots \\\Phi _{r1}&\cdots &\Phi _{rc}\\\end{bmatrix}}{\begin{bmatrix}\alpha _{1}\\\vdots \\\alpha _{c}\\\end{bmatrix}}={\begin{bmatrix}\beta _{1}\\\vdots \\\beta _{r}\\\end{bmatrix}}$

Column Perspective

Let $\Phi :{\mathsf {F}}^{c}\to {\mathsf {F}}^{r}$, and let ${\vec {\alpha }}=[\alpha _{1}\cdots \alpha _{c}]\in {\mathsf {F}}^{c}$ . Then the matrix-vector product can be defined as the linear combination of pairing scalar coefficients in $[\alpha _{1}\cdots \alpha _{c}]$ to vectors in $\operatorname {col} (\Phi )$.

$\Phi {\vec {\alpha }}=\sum _{i=1}^{c}\alpha _{i}\operatorname {col} (\Phi )_{i}$

$\alpha _{1}{\begin{bmatrix}\Phi _{1,1}\\\vdots \\\Phi _{r,1}\\\end{bmatrix}}+\alpha _{2}{\begin{bmatrix}\Phi _{1,2}\\\vdots \\\Phi _{r,2}\\\end{bmatrix}}+\cdots +\alpha _{c}{\begin{bmatrix}\Phi _{1,c}\\\vdots \\\Phi _{r,c}\\\end{bmatrix}}$

Row Perspective

$\Phi {\vec {\alpha }}={\begin{bmatrix}{\vec {\Phi }}_{1}\cdot {\vec {\alpha }}\\\vdots \\{\vec {\Phi }}_{r}\cdot {\vec {\alpha }}\\\end{bmatrix}}$

$(\Phi {\vec {\alpha }})_{i}={\vec {\Phi }}_{i}\cdot {\vec {\alpha }}$

Product of Matrices

Let $\Phi :{\mathsf {F}}^{m}\to {\mathsf {F}}^{n}$ and $\Psi :{\mathsf {F}}^{n}\to {\mathsf {F}}^{p}$ be matrices. Then the product $\Psi \Phi$ is defined:

$(\Psi \Phi )_{ij}=\sum _{k=1}^{n}\Psi _{jk}\Phi _{ki}$

For all natural pairs $(i,j)\in [m]\times [p]$.

Column Perspective

For the product $\Psi \Phi$, the $i$-th column of the matrix is defined by the application of $\Psi$ on the $i$-th column of $\Phi$.

$\operatorname {col} (\Psi \Phi )_{i}=\Psi \operatorname {col} (\Phi )_{i}$

Rank and Image

The rank of a matrix $\Phi ^{r\times c}$ is the number of independent column vectors. The image of a matrix is the span of its columns.

An injective matrix is any full-rank matrix.

A surjective matrix is any full row-rank matrix.

Kernel and Nullity

The kernel of a matrix $\Phi :{\mathsf {F}}^{r}\to {\mathsf {F}}^{c}$ is the set of vectors which map to ${\vec {0}}$.

${\begin{bmatrix}\Phi _{11}&\cdots &\Phi _{1c}\\\vdots &\ddots &\vdots \\\Phi _{r1}&\cdots &\Phi _{rc}\\\end{bmatrix}}{\begin{bmatrix}x_{1}\\\vdots \\x_{c}\\\end{bmatrix}}={\begin{bmatrix}0_{1}\\\vdots \\0_{r}\\\end{bmatrix}}$

The nullity is the dimension of the kernel.

$\operatorname {null} \Phi =\{{\vec {v}}\in {\mathsf {F}}^{c}\mid \Phi {\vec {v}}={\vec {0}}\}$

Identity Matrix

Main articles: Identity matrix and Kronecker delta

For any matrix $\Phi :{\mathsf {F}}^{c}\to {\mathsf {F}}^{r}$ there also exists matrices $\mathrm {I} _{c},\mathrm {I} _{r}$ which act as the unique right and left identity element under the product of maps.

$\Phi \mathrm {I} _{c}=\mathrm {I} _{r}\Phi =\Phi$

Any matrix which fulfills this condition is known as the identity matrix, denoted $\mathrm {I}$ or with a subscript $\mathrm {I} _{n}$ for some dimension $n$. All identity matrices are square matrices whose values are defined for any index $(i,j)\in [n]\times [n]$:

$(\mathrm {I} _{n})_{ij}:={\begin{cases}0\quad (i\neq j)\\1\quad (i=j)\\\end{cases}}$

An example of an identity matrix of $n$ dimensions.

${\begin{bmatrix}1_{11}&0_{12}&\cdots &0_{1n}\\0_{21}&1_{22}&\cdots &0_{2n}\\\vdots &\vdots &\ddots &\vdots \\0_{n1}&0_{n2}&\cdots &1_{nn}\\\end{bmatrix}}$

Inverse Matrix

Main articles: Generalized inverse, Bijection, and Isomorphism

A matrix $\Phi :{\mathsf {F}}^{n}\to {\mathsf {F}}^{n}$ is invertible if there exists a matrix $\Phi ^{-1}:{\mathsf {F}}^{n}\to {\mathsf {F}}^{n}$ such that:

$\Phi ^{-1}\Phi =\Phi \Phi ^{-1}=\mathrm {I} _{n}$

• An invertible matrix may also be known as a non-singular matrix, a linear isomorphism or bijection.
• The set of all invertible matrices of $n$ size is known as the $\operatorname {GL} (n,{\mathsf {F}})$.
• All invertible matrices are full-rank square matrices, and thus the kernel is trivial.

• For endomorphisms over finite-dimensional modules, surjection, injection, and bijection are all equivalent conditions.
• The determinant of an invertible matrix is non-zero.

Left Inverse

Although only square matrices are strictly invertible, an injective matrix will have a left-inverse by definition.

$\Phi _{\textrm {L}}^{-1}=(\Phi ^{\intercal }\Phi )^{-1}\Phi ^{\intercal }$

Right Inverse

$\Phi _{\textrm {R}}^{-1}=\Phi ^{\intercal }(\Phi \Phi ^{\intercal })^{-1}$

Orthonormal Matrix

Main articles: Orthonormal matrix and Unitary matrix

An orthonormal matrix $\Phi :{\mathsf {F}}^{n}\to {\mathsf {F}}^{n}$ is an invertible matrix which preserves the norms between vector spaces. It is also the matrix where the transpose is the multiplicative inverse $\Phi ^{-1}$.

$\Phi \Phi ^{\intercal }=\Phi ^{\intercal }\Phi =\mathrm {I} _{n}$

The set of all $n\times n$ orthogonal matrices over ${\mathsf {F}}$ forms the orthogonal group $\mathrm {O} (n,{\mathsf {F}})$. The subset of $\mathrm {O} (n,{\mathsf {F}})$ which has only a determinant of $+1$ is known as the special orthogonal group $\mathrm {S} \mathrm {O} (n,{\mathsf {F}})$, and all matrices from this group are rotational matrices.

Observations

• Let ${\vec {v}}_{1},{\vec {v}}_{2}\in {\mathsf {F}}^{n}$. If $\Phi :{\mathsf {F}}^{n}\to {\mathsf {F}}^{n}$ is orthonormal then $\langle {\vec {v}}_{1},{\vec {v}}_{2}\rangle =\langle \Phi {\vec {v}}_{1},\Phi {\vec {v}}_{2}\rangle$.
• The determinant of $\Phi$ is either $+1$ or $-1$.
• If $\Phi$ is orthonormal then so is its transpose.

Example

${\frac {1}{\sqrt {2}}}{\begin{bmatrix}1&1\\1&-1\\\end{bmatrix}}$

${\frac {1}{3}}{\begin{bmatrix}2&-2&1\\1&2&2\\2&1&-2\\\end{bmatrix}}$

Gram-Schmidt Process

Main article: Gram–Schmidt process

Given the columns of a full-rank matrix, the Gram-Schmidt process can generate a similar orthonormal basis.

Trace of a Square Matrix

Let $\Phi :{\mathsf {F}}^{n}\to {\mathsf {F}}^{n}$.

$\operatorname {trace} (\Phi ):=\sum _{i=1}^{n}\Phi _{ii}$

The trace of a product of matrices is the sum of their individual traces.

$\operatorname {trace} (\Psi \Phi ):=\operatorname {trace} (\Psi )+\operatorname {trace} (\Phi )$

Matrix Decompositions

Rank Factorization (CR)

Main article: Rank factorization

Rank factorization, or the column-row (CR) form of a matrix $\Phi :{\mathsf {F}}^{c}\to {\mathsf {F}}^{r}$ means to decompose $\Phi =\mathrm {C} \mathrm {R}$, where $\mathrm {C}$ represents independent columns from $\Phi$, and $\mathrm {R}$ represents independent rows from $\Phi$.

${\begin{bmatrix}2&0&4\\0&3&6\\0&0&0\\\end{bmatrix}}={\begin{bmatrix}2&0\\0&3\\0&0\\\end{bmatrix}}{\begin{bmatrix}1&0&2\\0&1&2\\\end{bmatrix}}$

This factorization is motivated mostly by pedagogy and demonstrates basic properties of matrix multiplication.

Orthonormal (QR)

Main article: QR decomposition

Singular Value Decomposition (SVD)

Main article: Singular value decomposition

Any real or complex matrix of size $c\times r$ may be decomposed into the triple product $\mathrm {U} \mathrm {\Sigma } \mathrm {V} ^{*}$, where $\mathrm {U}$ is $c\times c$ and orthonormal, $\mathrm {\Sigma }$ is $r\times r$ is a positive-definite real diagonal matrix, and $\mathrm {V} ^{*}$ is $r\times r$ and orthonormal.

$$\Phi ={\begin{bmatrix}10&8&0&0&0&0\\7&6&8&0&0&0\\9&8&7&4&3&0\\7&6&8&0&0&0\\0&0&0&8&9&8\\\end{bmatrix}}$$For $\mathrm {U}$ we have a relationship between rows and "concepts." For $\mathrm {V} ^{*}$ we have a relationship between columns and concepts. For $\mathrm {\Sigma }$ we have a matrix which represents the eigenvalues of each concept.

Sources

Textbook

• Katznelson, Yitzhak; Katznelson, Yonatan R. (2008). A (Terse) Introduction to Linear Algebra. American Mathematical Society. ISBN 978-0-8218-4419-9.
• Roman, Steven (2005). Advanced Linear Algebra (2nd ed.). Springer. ISBN 0-387-24766-1.
• Süli, Endre; Mayers, David (2011) [2003]. An Introduction to Numerical Analysis. Cambridge University Press. ISBN 978-0-521-00794-8.
• Trefethen, Lloyd Nicholas; Bau III, David (1997). Numerical Linear Algebra. SIAM. ISBN 978-0-898713-61-9.

Web

• "Transpose of a Matrix Product". ProofWiki. Retrieved 11 Mar 2021.{{cite web}}: CS1 maint: url-status (link)
• "matrix". nLab. 11 Mar 2021. Retrieved 11 Mar 2021.{{cite web}}: CS1 maint: url-status (link)