Linear algebra

Linear algebra is a branch of mathematics concerned with the study of vectors, with families of vectors called vector spaces or linear spaces, and with functions which input one vector and output another, according to certain rules. These functions are called linear maps or linear transformations and are often represented by matrices. Linear algebra is central to modern mathematics and its applications. An elementary application of linear algebra is to the solution of a systems of linear equations in several unknowns. More advanced applications are ubiquitous, in areas as diverse as abstract algebra and functional analysis. Linear algebra has a concrete representation in analytic geometry and is generalized in operator theory. It has extensive applications in the natural sciences and the social sciences. Nonlinear mathematical models can often be approximated by linear ones.

History

Many of the basic tools of linear algebra, particularly those concerned with the solution of systems of linear equations, date to antiquity. See, for example, the history of Gaussian elimination. But the abstract study of vectors and vector spaces does not begin until the 1600s. The origin of many of these ideas is discussed in the article on determinants. The method of least squares, first used by Gauss in the 1790s, is an early and significant application of the ideas of linear algebra.

The subject began to take its modern form in the mid-19th century, which saw many ideas and methods of previous centuries generalized as abstract algebra. Matrices and tensors were introduced and well understood by the turn of the 20th century. The use of these objects in special relativity, statistics, and quantum mechanics did much to spread the subject of linear algebra beyond pure mathematics.

Main structures

The main structures of linear algebra are vector spaces and linear maps between them. A vector space is a set whose elements can be added together and multiplied by the scalars, or numbers. In many physical applications, the scalars are real numbers, R. More generally, the scalars may form any field F — thus one can consider vector spaces over the field Q of rational numbers, the field C of complex numbers, or a finite field F_q. These two operations must behave similarly to the usual addition and multiplication of numbers: addition is commutative and associative, multiplication distributes over addition, and so on. More precisely, the two operations must satisfy a list of axioms chosen to emulate the properties of addition and scalar multiplication of Euclidean vectors in the coordinate n-space Rⁿ. One of the axioms stipulates the existence of zero vector, which behaves analogously to the number zero with respect to addition. Elements of a general vector space V may be objects of any nature, for example, functions or polynomials, but when viewed as elements of V, they are frequently called vectors.

Given two vector spaces V and W over a field F, a linear transformation is a map

$T:V\to W$

which is compatible with addition and scalar multiplication:

$T(u+v)=T(u)+T(v),\quad T(rv)=rT(v)$

for any vectors u,v ∈ V and a scalar r ∈ F.

A fundamental role in linear algebra is played by the notions of linear combination, span, and linear independence of vectors and basis and the dimension of a vector space. Given a vector space V over a field F, an expression of the form

$r_{1}v_{1}+r_{2}v_{2}+\ldots +r_{k}v_{k},$

where v₁, v₂, …, v_k are vectors and r₁, r₂, …, r_k are scalars, is called the linear combination of the vectors v₁, v₂, …, v_k with coefficients r₁, r₂, …, r_k. The set of all linear combinations of vectors v₁, v₂, …, v_k is called their span. A linear combination of any system of vectors with all zero coefficients is zero vector of V. If this is the only way to express zero vector as a linear combination of v₁, v₂, …, v_k then these vectors are linearly independent. A linearly independent set of vectors that spans a vector space V is a basis of V. If a vector space admits a finite basis then any two bases have the same number of elements (called the dimension of V) and V is a finite-dimensional vector space. This theory can be extended to infinite-dimensional spaces.

There is an important distinction between the coordinate n-space Rⁿ and a general finite-dimensional vector space V. While Rⁿ has a standard basis {e₁, e₂, …, e_n}, a vector space V typically does not come equipped with a basis and many different bases exist (although they all consist of the same number of elements equal to the dimension of V). Having a particular basis {v₁, v₂, …, v_n} of V allows one to construct a coordinate system in V: the vector with coordinates (r₁, r₂, …, r_n) is the linear combination

$r_{1}v_{1}+r_{2}v_{2}+\ldots +r_{n}v_{n}.$

The condition that v₁, v₂, …, v_n span V guarantees that each vector v can be assigned coordinates, whereas the linear independence of v₁, v₂, …, v_n further assures that these coordinates are determined in a unique way (i.e. there is only one linear combination of the basis vectors that is equal to v). In this way, once a basis of a vector space V over F has been chosen, V may be identified with the coordinate n-space Fⁿ. Under this identification, addition and scalar multiplication of vectors in V correspond to addition and scalar multiplication of their coordinate vectors in Fⁿ. Furthermore, if V and W are an n-dimensional and m-dimensional vector space over F, and a basis of V and a basis of W have been fixed, then any linear transformation T: V → W may be encoded by an m × n matrix A with entries in the field F, called the matrix of T with respect to these bases. Therefore, by and large, the study of linear transformations, which were defined axiomatically, may be replaced by the study of matrices, which are concrete objects. This is a major technique in linear algebra.

Vector spaces over the complex numbers

Remarkably, the 2 × 2 complex matrices were studied before 2 × 2 real matrices. Early topics of interest included biquaternions and Pauli algebra. Investigation of 2 × 2 real matrices revealed the less common split-complex numbers and dual numbers, which are at variance with the Euclidean nature of the ordinary complex number plane.

Some useful theorems

• Every vector space has a basis.
• Any two bases of the same vector space have the same cardinality; equivalently, the dimension of a vector space is well-defined.
• A square matrix is invertible if and only if its determinant is nonzero.
• A matrix is invertible if and only if the linear map represented by the matrix is an isomorphism.
• If a square matrix has a left inverse or a right inverse then it is invertible (see invertible matrix for other equivalent statements).
• A matrix is positive semidefinite if and only if each of its eigenvalues is greater than or equal to zero.
• A matrix is positive definite if and only if each of its eigenvalues is greater than zero.
• An n×n matrix is diagonalizable (i.e. there exists an invertible matrix P and a diagonal matrix D such that A = PDP⁻¹) if and only if it has n linearly independent eigenvectors.
• The spectral theorem states that a matrix is orthogonally diagonalizable if and only if it is symmetric.

For more information regarding the invertibility of a matrix, consult the invertible matrix article.

Generalizations and related topics

Since linear algebra is a successful theory, its methods have been developed in other parts of mathematics. In module theory one replaces the field of scalars by a ring. In multilinear algebra one considers multivariable linear transformations, that is, mappings which are linear in each of a number of different variables. This line of inquiry naturally leads to the idea of the tensor product. Functional analysis mixes the methods of linear algebra with those of mathematical analysis.

See also

• List of linear algebra topics
• Numerical linear algebra

References

See also: Linear algebra § Further reading

Textbooks

• Axler, Sheldon Jay (1997), Linear Algebra Done Right (2nd ed.), Springer-Verlag, ISBN 0387982590
• Lay, David C. (August 22, 2005), Linear Algebra and Its Applications (3rd ed.), Addison Wesley, ISBN 978-0321287137
• Meyer, Carl D. (February 15, 2001), Matrix Analysis and Applied Linear Algebra, Society for Industrial and Applied Mathematics (SIAM), ISBN 978-0898714548
• Poole, David (2006), Linear Algebra: A Modern Introduction (2nd ed.), Brooks/Cole, ISBN 0-534-99845-3
• Anton, Howard (2005), Elementary Linear Algebra (Applications Version) (9th ed.), Wiley International
• Leon, Steven J. (2006), Linear Algebra With Applications (7th ed.), Pearson Prentice Hall

History

• Fearnley-Sander, Desmond, "Hermann Grassmann and the Creation of Linear Algebra" (via JSTOR), American Mathematical Monthly 86 (1979), pp. 809–817.
• Grassmann, Hermann, Die lineale Ausdehnungslehre ein neuer Zweig der Mathematik: dargestellt und durch Anwendungen auf die übrigen Zweige der Mathematik, wie auch auf die Statik, Mechanik, die Lehre vom Magnetismus und die Krystallonomie erläutert, O. Wigand, Leipzig, 1844.

Further reading

Introductory textbooks

• Axler, Sheldon (February 26, 2004), Linear Algebra Done Right (2nd ed.), Springer, ISBN 978-0387982588
• Bretscher, Otto (June 28, 2004), Linear Algebra with Applications (3rd ed.), Prentice Hall, ISBN 978-0131453340
• Farin, Gerald; Hansford, Dianne (December 15, 2004), Practical Linear Algebra: A Geometry Toolbox, AK Peters, ISBN 978-1568812342
• Friedberg, Stephen H.; Insel, Arnold J.; Spence, Lawrence E. (November 11, 2002), Linear Algebra (4th ed.), Prentice Hall, ISBN 978-0130084514
• Kolman, Bernard; Hill, David R. (May 3, 2007), Elementary Linear Algebra with Applications (9th ed.), Prentice Hall, ISBN 978-0132296540
• Strang, Gilbert (July 19, 2005), Linear Algebra and Its Applications (4th ed.), Brooks Cole, ISBN 978-0030105678

Advanced textbooks

• Bhatia, Rajendra (November 15, 1996), Matrix Analysis, Graduate Texts in Mathematics, Springer, ISBN 978-0387948461
• Demmel, James W. (August 1, 1997), Applied Numerical Linear Algebra, SIAM, ISBN 978-0898713893
• Golan, Johnathan S. (January 2007), The Linear Algebra a Beginning Graduate Student Ought to Know (2nd ed.), Springer, ISBN 978-1402054945
• Golub, Gene H.; Van Loan, Charles F. (October 15, 1996), Matrix Computations, Johns Hopkins Studies in Mathematical Sciences (3rd ed.), The Johns Hopkins University Press, ISBN 978-0801854149
• Greub, Werner H. (October 16, 1981), Linear Algebra, Graduate Texts in Mathematics (4th ed.), Springer, ISBN 978-0801854149
• Hoffman, Kenneth; Kunze, Ray (April 25, 1971), Linear Algebra (2nd ed.), Prentice Hall, ISBN 978-0135367971
• Halmos, Paul R. (August 20, 1993), Finite-Dimensional Vector Spaces, Undergraduate Texts in Mathematics, Springer, ISBN 978-0387900933
• Horn, Roger A.; Johnson, Charles R. (February 23, 1990), Matrix Analysis, Cambridge University Press, ISBN 978-0521386326
• Horn, Roger A.; Johnson, Charles R. (June 24, 1994), Topics in Matrix Analysis, Cambridge University Press, ISBN 978-0521467131
• Lang, Serge (March 9, 2004), Linear Algebra, Undergraduate Texts in Mathematics (3rd ed.), Springer, ISBN 978-0387964126
• Roman, Steven (March 22, 2005), Advanced Linear Algebra, Graduate Texts in Mathematics (2nd ed.), Springer, ISBN 978-0387247663
• Shilov, Georgi E. (June 1, 1977), Linear algebra, Dover Publications, ISBN 978-0486635187
• Shores, Thomas S. (December 6, 2006), Applied Linear Algebra and Matrix Analysis, Undergraduate Texts in Mathematics, Springer, ISBN 978-0387331942
• Smith, Larry (May 28, 1998), Linear Algebra, Undergraduate Texts in Mathematics, Springer, ISBN 978-0387984551

Study guides and outlines

• Leduc, Steven A. (May 1, 1996), Linear Algebra (Cliffs Quick Review), Cliffs Notes, ISBN 978-0822053316
• Lipschutz, Seymour; Lipson, Marc (December 6, 2000), Schaum's Outline of Linear Algebra (3rd ed.), McGraw-Hill, ISBN 978-0071362009
• Lipschutz, Seymour (January 1, 1989), 3,000 Solved Problems in Linear Algebra, McGraw-Hill, ISBN 978-0070380233
• McMahon, David (October 28, 2005), Linear Algebra Demystified, McGraw-Hill Professional, ISBN 978-0071465793
• Zhang, Fuzhen (April 7, 2009), Linear Algebra: Challenging Problems for Students, The Johns Hopkins University Press, ISBN 978-0801891250

External links

• International Linear Algebra Society
• MIT Professor Gilbert Strang's Linear Algebra Course Homepage : MIT Course Website
• MIT Linear Algebra Lectures: free videos from MIT OpenCourseWare
• Linear Algebra Toolkit.
• Linear Algebra on MathWorld.
• Linear Algebra overview and notation summary on PlanetMath.
• Matrix and Linear Algebra Terms on Earliest Known Uses of Some of the Words of Mathematics
• Earliest Uses of Symbols for Matrices and Vectors on Earliest Uses of Various Mathematical Symbols
• Linear Algebra by Elmer G. Wiens. Interactive web pages for vectors, matrices, linear equations, etc.
• Linear Algebra Solved Problems: Interactive forums for discussion of linear algebra problems, from the lowest up to the hardest level (Putnam).
• Linear Algebra for Informatics. José Figueroa-O'Farrill, University of Edinburgh
• Online Notes / Linear Algebra Paul Dawkins, Lamar University
• Elementary Linear Algebra textbook with solutions
• Linear Algebra Wiki
• Linear algebra (math 21b) homework and exercises

Online books

• Beezer, Rob, A First Course in Linear Algebra
• Connell, Edwin H., Elements of Abstract and Linear Algebra
• Hefferon, Jim, Linear Algebra
• Matthews, Keith, Elementary Linear Algebra
• Sharipov, Ruslan, Course of linear algebra and multidimensional geometry