Tensor product

In mathematics, the tensor product, denoted by ⊗, may be applied in different contexts to vectors, matrices, tensors, vector spaces, algebras, topological vector spaces, and modules, among many other structures or objects. In each case the significance of the symbol is the same: the most general bilinear operation. In some contexts, this product is also referred to as outer product. The term "tensor product" is also used in relation to monoidal categories. The $\boxtimes$ variant of ⊗ is used in control theory that expresses that the elements in the tensor product are vectors, matrices or tensors which define the vertexes of a given polytopic model, see wikipedia page TP model transformation.

Tensor products of modules over a ring

Main article: Tensor product of modules

The tensor product of two modules over a ring $R$ is defined as follows: For $A_{R}$ and $_{R}B$ right and left modules over $R$ define $F$ to be the free Abelian group over $A\times B$ . Let $K$ be the subgroup of $F$ generated by all elements of the forms below for all $a,a'\in A;b,b'\in B;r\in R$ .

${\begin{aligned}&(i)&&(a+a',b)-(a,b)-(a',b)\\&(ii)&&(a,b+b')-(a,b)-(a,b')\\&(iii)&&(ar,b)-(a,rb)\end{aligned}}$

Then the quotient group ${\frac {F}{K}}$ is the tensor product of $A$ and $B$ and is denoted by $A\otimes _{R}B$. By construction this is an abelian group. The coset $(a,b)+K$ is denoted by $a\otimes b$. Also, given $A_{R}$ and $_{R}B$ right and left modules over $R$ , the map $\phi :A\times B\to A\otimes _{R}B$ defined by $(a,b)\mapsto a\otimes b$ is a middle linear map; that is, it satisfies:

${\begin{aligned}&(iv)&&\phi (a+a',b)=\phi (a,b)+\phi (a',b)\\&(v)&&\phi (a,b+b')=\phi (a,b)+\phi (a,b')\\&(vi)&&\phi (ar,b)=\phi (a,rb)\end{aligned}}$

The properties $(iv)$ and $(v)$ make $\phi$ a homomorphism of the abelian group $A\times B$. For any middle linear map $\psi$ of $A\times B$ , a unique group homomorphism $f$ of $A\otimes _{R}B$ satisfies $\psi =f\circ \phi$, and this property determines $\phi$ within group isomorphism. See the main article for details.

The map $\phi$ is referred to as the "Canonical Middle Linear Map".

Tensor product of vector spaces

The tensor product V ⊗_K W of two vector spaces V and W over a field K can be defined by the method of generators and relations. (The tensor product is often denoted V ⊗ W when the underlying field K is understood.)

To construct V ⊗ W, one begins with the set of ordered pairs in the Cartesian product V × W. For the purposes of this construction, regard this Cartesian product as a set rather than a vector space. The free vector space F on V × W is defined by taking the vector space in which the elements of V × W are a basis. In set-builder notation,

$F(V\times W)=\left\{\sum _{i=1}^{n}\alpha _{i}e_{(v_{i},w_{i})}\ {\Bigg |}\ n\in \mathbb {N} ,\alpha _{i}\in K,(v_{i},w_{i})\in V\times W\right\},$

where we have used the symbol e_(v,w) to emphasize that these are taken to be linearly independent by definition for distinct (v, w) ∈ V × W.

The tensor product arises by defining the following four equivalence relations in F(V × W):

${\begin{aligned}e_{(v_{1}+v_{2},w)}&\sim e_{(v_{1},w)}+e_{(v_{2},w)}\\e_{(v,w_{1}+w_{2})}&\sim e_{(v,w_{1})}+e_{(v,w_{2})}\\ce_{(v,w)}&\sim e_{(cv,w)}\sim e_{(v,cw)}\end{aligned}}$

where v, v₁ and v₂ are vectors from V, while w, w₁, and w₂ are vectors from W, and c is from the underlying field K. We denote the equivalence class of the zero vector generated by these four equivalence relations by R. The tensor product of the two vector spaces V and W is then the quotient space

$V\otimes W=F(V\times W)/R.$

It is also called the tensor product space of V and W and is a vector space (which can be verified by directly checking the vector space axioms). The tensor product of two elements v and w is the equivalence class (e_(v,w) + R) of e_(v,w) in V ⊗ W, denoted v ⊗ w. This notation can somewhat obscure the fact that tensors are always cosets: manipulations performed via the representatives (v,w) must always be checked that they do not depend on the particular choice of representative.

The space R is mapped to zero in V ⊗ W, so that the above four equivalence relations become equalities in the tensor product space:

${\begin{aligned}(v_{1}+v_{2})\otimes w&=v_{1}\otimes w+v_{2}\otimes w;\\v\otimes (w_{1}+w_{2})&=v\otimes w_{1}+v\otimes w_{2};\\cv\otimes w&=v\otimes cw=c(v\otimes w).\end{aligned}}$

Given bases {v_i} and {w_i} for V and W respectively, the tensors {v_i ⊗ w_j} form a basis for V ⊗ W. The dimension of the tensor product therefore is the product of dimensions of the original spaces; for instance R^m ⊗ Rⁿ will have dimension mn.

Elements of V ⊗ W are sometimes referred to as tensors, although this term refers to many other related concepts as well. An element of V ⊗ W of the form v ⊗ w is called a pure or simple tensor. In general, an element of the tensor product space is not a pure tensor, but rather a finite linear combination of pure tensors. That is to say, if v₁ and v₂ are linearly independent, and w₁ and w₂ are also linearly independent, then v₁ ⊗ w₁ + v₂ ⊗ w₂ cannot be written as a pure tensor. The number of simple tensors required to express an element of a tensor product is called the tensor rank (not to be confused with tensor order, which is the number of spaces one has taken the product of, in this case 2; in notation, the number of indices), and for linear operators or matrices, thought of as (1,1) tensors (elements of the space V ⊗ V*), it agrees with matrix rank.

Characterization by a universal property

The tensor product of V and W can be defined (up to isomorphism) by any pair (L, $\varphi$), with L a vector space on K and $\varphi :V\times W\to L$ a bilinear map such that for any K-vector space Z and any bilinear map $h:V\times W\to Z$, there exists a unique linear map ${\tilde {h}}:L\to Z$ verifying $h={\tilde {h}}\circ \varphi$

In this sense, φ is the most general bilinear map that can be built from VxW.

It is easy to check that (V ⊗ W, ⊗) satisfies this universal property.

As a functor

The tensor product also operates on linear maps between vector spaces. Specifically, given two linear maps S : V → X and T : W → Y between vector spaces, the tensor product of the two linear maps S and T is a linear map

$S\otimes T:V\otimes W\rightarrow X\otimes Y$

defined by

$(S\otimes T)(v\otimes w)=S(v)\otimes T(w).$

In this way, the tensor product becomes a bifunctor from the category of vector spaces to itself, covariant in both arguments.

The Kronecker product of two matrices is the matrix of the tensor product of the two corresponding linear maps under a standard choice of bases of the tensor products (see the article on Kronecker products).

More than two vector spaces

The construction and the universal property of the tensor product can be extended to allow for more than two vector spaces. For example, suppose that V₁, V₂, and V₃ are three vector spaces. The tensor product V₁ ⊗ V₂ ⊗ V₃ is defined along with a trilinear mapping from the direct product

$\varphi :V_{1}\times V_{2}\times V_{3}\to V_{1}\otimes V_{2}\otimes V_{3}$

so that, any trilinear map F from the direct product to a vector space W

$F:V_{1}\times V_{2}\times V_{3}\to W$

factors uniquely as

$F=L\circ \varphi$

where L is a linear map. The tensor product is uniquely characterized by this property, up to a unique isomorphism.

This construction is related to repeated tensor products of two spaces. For example, if V₁, V₂, and V₃ are three vector spaces, then there are (natural) isomorphisms

$V_{1}\otimes V_{2}\otimes V_{3}\cong V_{1}\otimes (V_{2}\otimes V_{3})\cong (V_{1}\otimes V_{2})\otimes V_{3}.$

More generally, the tensor product of an arbitrary indexed family V_i, i ∈ I, is defined to be universal with respect to multilinear mappings of the direct product $\scriptstyle \prod _{i\in I}V_{i}.$

Tensor powers and braiding

Let n be a non-negative integer. The nth tensor power of the vector space V is the n-fold tensor product of V with itself. That is

$V^{\otimes n}\;{\overset {\mathrm {def} }{=}}\;\underbrace {V\otimes \cdots \otimes V} _{n}.$

A permutation σ of the set {1, 2, ..., n} determines a mapping of the nth Cartesian power of V

$\sigma :V^{n}\to V^{n}$

defined by

$\sigma (v_{1},v_{2},\dots ,v_{n})=(v_{\sigma 1},v_{\sigma 2},\dots ,v_{\sigma n}).$

Let

$\varphi :V^{n}\to V^{\otimes n}$

be the natural multilinear embedding of the Cartesian power of V into the tensor power of V. Then, by the universal property, there is a unique isomorphism

$\tau _{\sigma }:V^{\otimes n}\to V^{\otimes n}$

such that

$\varphi \circ \sigma =\tau _{\sigma }\circ \varphi .$

The isomorphism τ_σ is called the braiding map associated to the permutation σ.

Tensor product of Hilbert spaces

Main article: Tensor product of Hilbert spaces

Topological tensor product

Main article: Topological tensor product

Tensor product of graded vector spaces

Main article: Graded vector space § Operations on graded vector spaces

Tensor product of algebras

Main article: Tensor product of algebras

Tensor product of fields

Main article: Tensor product of fields

Tensor product of two tensors

A tensor on V is an element of a vector space of the form

$T_{s}^{r}(V)=\underbrace {V\otimes \dots \otimes V} _{r}\otimes \underbrace {V^{*}\otimes \dots \otimes V^{*}} _{s}=V^{\otimes r}\otimes V^{*\otimes s}$

for non-negative integers r and s. There is a general formula for the components of a (tensor) product of two (or more) tensors. For example, if F and G are two covariant tensors of rank m and n (respectively) (i.e. F ∈ T_m⁰, and G ∈ T_n⁰), then the components of their tensor product are given by

$(F\otimes G)_{i_{1}i_{2}...i_{m+n}}=F_{i_{1}i_{2}...i_{m}}G_{i_{m+1}i_{m+2}i_{m+3}...i_{m+n}}.$

In this example, it is assumed that there is a chosen basis B of the vector space V, and the basis on any tensor space T_s^r is tacitly assumed to be the standard one (this basis is described in the article on Kronecker products). Thus, the components of the tensor product of two tensors are the ordinary product of the components of each tensor.

Note that in the tensor product, the factor F consumes the first rank(F) indices, and the factor G consumes the next rank(G) indices, so

$\mathrm {rank} (F\otimes G)=\mathrm {rank} (F)+\mathrm {rank} (G).$

The tensor $\scriptstyle T_{s}^{r}(V)$ may be naturally viewed as a module for the Lie algebra End(V) by means of the diagonal action: for simplicity let us assume r = s = 1, then, for each $\scriptstyle u\in \mathrm {End} (V)$,

$u(a\otimes b)=u(a)\otimes b-a\otimes u^{*}(b),$

where u* in End(V*) is the transpose of u, that is, in terms of the obvious pairing on V ⊗ V*,

$\langle u(a),b\rangle =\langle a,u^{*}(b)\rangle$.

There is a canonical isomorphism $\scriptstyle T_{1}^{1}(V)\rightarrow \mathrm {End} (V)$ given by

$(a\otimes b)(x)=\langle x,b\rangle a.$

Under this isomorphism, every u in End(V) may be first viewed as an endomorphism of $\scriptstyle T_{1}^{1}(V)$ and then viewed as an endomorphism of End(V). In fact it is the adjoint representation ad(u) of End(V) .

Example

See also: Classical treatment of tensors

Let U be a tensor of type (1,1) with components U^α_β, and let V be a tensor of type (1,0) with components V^γ. Then

$U^{\alpha }{}_{\beta }V^{\gamma }=(U\otimes V)^{\alpha }{}_{\beta }{}^{\gamma }$

and

$V^{\mu }U^{\nu }{}_{\sigma }=(V\otimes U)^{\mu \nu }{}_{\sigma }.$

The tensor product inherits all the indices of its factors.

Kronecker product of two matrices

Main article: Kronecker product

With matrices this operation is usually called the Kronecker product, a term used to make clear that the result has a particular block structure imposed upon it, in which each element of the first matrix is replaced by the second matrix, scaled by that element. For matrices U and V this is:

$U\otimes V={\begin{bmatrix}u_{1,1}V&u_{1,2}V&\cdots \\u_{2,1}V&u_{2,2}V\\\vdots &&\ddots \end{bmatrix}}={\begin{bmatrix}u_{1,1}v_{1,1}&u_{1,1}v_{1,2}&\cdots &u_{1,2}v_{1,1}&u_{1,2}v_{1,2}&\cdots \\u_{1,1}v_{2,1}&u_{1,1}v_{2,2}&&u_{1,2}v_{2,1}&u_{1,2}v_{2,2}\\\vdots &&\ddots \\u_{2,1}v_{1,1}&u_{2,1}v_{1,2}\\u_{2,1}v_{2,1}&u_{2,1}v_{2,2}\\\vdots \end{bmatrix}}.$

For example, the tensor product of two two-dimensional square matrices:

${\begin{bmatrix}a_{1,1}&a_{1,2}\\a_{2,1}&a_{2,2}\\\end{bmatrix}}\otimes {\begin{bmatrix}b_{1,1}&b_{1,2}\\b_{2,1}&b_{2,2}\\\end{bmatrix}}={\begin{bmatrix}a_{1,1}{\begin{bmatrix}b_{1,1}&b_{1,2}\\b_{2,1}&b_{2,2}\\\end{bmatrix}}&a_{1,2}{\begin{bmatrix}b_{1,1}&b_{1,2}\\b_{2,1}&b_{2,2}\\\end{bmatrix}}\\&\\a_{2,1}{\begin{bmatrix}b_{1,1}&b_{1,2}\\b_{2,1}&b_{2,2}\\\end{bmatrix}}&a_{2,2}{\begin{bmatrix}b_{1,1}&b_{1,2}\\b_{2,1}&b_{2,2}\\\end{bmatrix}}\\\end{bmatrix}}={\begin{bmatrix}a_{1,1}b_{1,1}&a_{1,1}b_{1,2}&a_{1,2}b_{1,1}&a_{1,2}b_{1,2}\\a_{1,1}b_{2,1}&a_{1,1}b_{2,2}&a_{1,2}b_{2,1}&a_{1,2}b_{2,2}\\a_{2,1}b_{1,1}&a_{2,1}b_{1,2}&a_{2,2}b_{1,1}&a_{2,2}b_{1,2}\\a_{2,1}b_{2,1}&a_{2,1}b_{2,2}&a_{2,2}b_{2,1}&a_{2,2}b_{2,2}\\\end{bmatrix}}.$

The resultant rank is at most 4, and the resultant dimension 16. Here rank denotes the tensor rank (number of requisite indices), while the matrix rank counts the number of degrees of freedom in the resulting array.

A representative case is the Kronecker product of any two rectangular arrays, considered as matrices. A dyadic product is the special case of the tensor product between two vectors of the same dimension.

Tensor product of quadratic forms

Main article: Tensor product of quadratic forms

Tensor product of multilinear maps

Given multilinear maps $\scriptstyle f(x_{1},\dots ,x_{k})$ and $\scriptstyle g(x_{1},\dots ,x_{m})$ their tensor product is the multilinear function

$(f\otimes g)(x_{1},\dots ,x_{k+m})=f(x_{1},\dots ,x_{k})g(x_{k+1},\dots ,x_{k+m}).$

Relation with the dual space

In the discussion on the universal property, replacing Z by the underlying scalar field of V and W yields that the space (V ⊗ W)* (the dual space of V ⊗ W, containing all linear functionals on that space) is naturally identified with the space of all bilinear functionals on V × W In other words, every bilinear functional is a functional on the tensor product, and vice versa.

Whenever V and W are finite dimensional, there is a natural isomorphism between V* ⊗ W* and (V ⊗ W)*, whereas for vector spaces of arbitrary dimension we only have an inclusion V* ⊗ W* ⊂ (V ⊗ W)*. So, the tensors of the linear functionals are bilinear functionals. This gives us a new way to look at the space of bilinear functionals, as a tensor product itself.

Types of tensors

Linear subspaces of the bilinear operators (or in general, multilinear operators) determine natural quotient spaces of the tensor space, which are frequently useful. See wedge product for the first major example. Another would be the treatment of algebraic forms as symmetric tensors.

Tensor product of graphs

Main article: Tensor product of graphs

Tensor product of line bundles

Main article: Vector bundle § Operations on vector bundles

Tensor product for computer programmers

Array programming languages

Array programming languages may have this pattern built in. For example, in APL the tensor product is expressed as $\scriptstyle \circ .\times$ (for example $\scriptstyle A\circ .\times B$ or $\scriptstyle A\circ .\times B\circ .\times C$). In J the tensor product is the dyadic form of */ (for example a */ b or a */ b */ c).

Note that J's treatment also allows the representation of some tensor fields, as a and b may be functions instead of constants. This product of two functions is a derived function, and if a and b are differentiable, then a*/b is differentiable.

However, these kinds of notation are not universally present in array languages. Other array languages may require explicit treatment of indices (for example, MATLAB), and/or may not support higher-order functions such as the Jacobian derivative (for example, Fortran/APL).

See also

• Extension of scalars
• Tensor algebra
• Topological tensor product
• Dyadic product
• Tensor contraction

References

• Bourbaki, Nicolas (1989), Elements of mathematics, Algebra I, Springer-Verlag, ISBN 3-540-64243-9.
• Halmos, Paul (1974), Finite dimensional vector spaces, Springer, ISBN 0-387-90093-4.
• Lang, Serge (2002), Algebra, Graduate Texts in Mathematics, vol. 211 (Revised third ed.), New York: Springer-Verlag, ISBN 978-0-387-95385-4, MR 1878556, Zbl 0984.00001
• Mac Lane, S.; Birkhoff, G. (1999), Algebra, AMS Chelsea, ISBN 0-8218-1646-2.