Trace (linear algebra) explained

In linear algebra, the trace of a square matrix, denoted,^[1] is defined to be the sum of elements on the main diagonal (from the upper left to the lower right) of . The trace is only defined for a square matrix .

In mathematical physics texts, if = 0 then the matrix is said to be traceless. This is a misnomer, but widely used, such as in the Pauli Matrices.

It can be proven that the trace of a matrix is the sum of its eigenvalues (counted with multiplicities). It can also be proven that for any two matrices and of appropriate sizes. This implies that similar matrices have the same trace. As a consequence one can define the trace of a linear operator mapping a finite-dimensional vector space into itself, since all matrices describing such an operator with respect to a basis are similar.

The trace is related to the derivative of the determinant (see Jacobi's formula).

Definition

The trace of an square matrix is defined as^{[1] [2] [3]} $$\backslash operatorname(\backslash mathbf)\; =\; \backslash sum\_^n\; a\_\; =\; a\_\; +\; a\_\; +\; \backslash dots\; +\; a\_$$where denotes the entry on the row and column of . The entries of can be real numbers, complex numbers, or more generally elements of a field . The trace is not defined for non-square matrices.

Example

Let be a matrix, with

Then$$\backslash operatorname(\backslash mathbf)\; =\; \backslash sum\_^\; a\_\; =\; a\_\; +\; a\_\; +\; a\_\; =\; 1\; +\; 5\; +\; (-5)\; =\; 1$$

Properties

Basic properties

The trace is a linear mapping. That is,for all square matrices and, and all scalars .^[3]

A matrix and its transpose have the same trace:^[3] $$\backslash operatorname(\backslash mathbf)\; =\; \backslash operatorname\backslash left(\backslash mathbf^\backslash mathsf\backslash right).$$

This follows immediately from the fact that transposing a square matrix does not affect elements along the main diagonal.

Trace of a product

The trace of a square matrix which is the product of two matrices can be rewritten as the sum of entry-wise products of their elements, i.e. as the sum of all elements of their Hadamard product. Phrased directly, if and are two matrices, then:$$\backslash operatorname\backslash left(\backslash mathbf^\backslash mathsf\backslash mathbf\backslash right)\; =\backslash operatorname\backslash left(\backslash mathbf\backslash mathbf^\backslash mathsf\backslash right)\; =\backslash operatorname\backslash left(\backslash mathbf^\backslash mathsf\backslash mathbf\backslash right)\; =\backslash operatorname\backslash left(\backslash mathbf\backslash mathbf^\backslash mathsf\backslash right)\; =\backslash sum\_^m\; \backslash sum\_^n\; a\_b\_\; \backslash ;\; .$$

If one views any real matrix as a vector of length (an operation called vectorization) then the above operation on and coincides with the standard dot product. According to the above expression, is a sum of squares and hence is nonnegative, equal to zero if and only if is zero.^[4] Furthermore, as noted in the above formula, . These demonstrate the positive-definiteness and symmetry required of an inner product; it is common to call the Frobenius inner product of and . This is a natural inner product on the vector space of all real matrices of fixed dimensions. The norm derived from this inner product is called the Frobenius norm, and it satisfies a submultiplicative property, as can be proven with the Cauchy–Schwarz inequality:$$0\; \backslash leq\backslash left[\backslash operatorname\{tr\}(\backslash mathbf\{A\}\; \backslash mathbf\{B\})\backslash right]^2\; \backslash leq\backslash operatorname\backslash left(\backslash mathbf^2\backslash right)\; \backslash operatorname\backslash left(\backslash mathbf^2\backslash right)\; \backslash leq\backslash left[\backslash operatorname\{tr\}(\backslash mathbf\{A\})\backslash right]^2\; \backslash left[\backslash operatorname\{tr\}(\backslash mathbf\{B\})\backslash right]^2\; \backslash ,$$if and are real positive semi-definite matrices of the same size. The Frobenius inner product and norm arise frequently in matrix calculus and statistics.

The Frobenius inner product may be extended to a hermitian inner product on the complex vector space of all complex matrices of a fixed size, by replacing by its complex conjugate.

The symmetry of the Frobenius inner product may be phrased more directly as follows: the matrices in the trace of a product can be switched without changing the result. If and are and real or complex matrices, respectively, then^{[3] [5]}

This is notable both for the fact that does not usually equal, and also since the trace of either does not usually equal .^[6] The similarity-invariance of the trace, meaning that for any square matrix and any invertible matrix of the same dimensions, is a fundamental consequence. This is proved by$$\backslash operatorname\backslash left(\backslash mathbf^(\backslash mathbf\backslash mathbf)\backslash right)\; =\backslash operatorname\backslash left((\backslash mathbf\; \backslash mathbf)\backslash mathbf^\backslash right)\; =\backslash operatorname(\backslash mathbf).$$Similarity invariance is the crucial property of the trace in order to discuss traces of linear transformations as below.

Additionally, for real column vectors

a \in R ^{n} and b \in R ^{n} , the trace of the outer product is equivalent to the inner product:

Cyclic property

More generally, the trace is invariant under circular shifts, that is,

This is known as the cyclic property.

Arbitrary permutations are not allowed: in general,$$\backslash operatorname(\backslash mathbf\backslash mathbf\backslash mathbf)\; \backslash ne\; \backslash operatorname(\backslash mathbf\backslash mathbf\backslash mathbf).$$

However, if products of three symmetric matrices are considered, any permutation is allowed, since:$$\backslash operatorname(\backslash mathbf\backslash mathbf\backslash mathbf)\; =\; \backslash operatorname\backslash left(\backslash left(\backslash mathbf\backslash mathbf\backslash mathbf\backslash right)^\backslash right)\; =\; \backslash operatorname(\backslash mathbf\backslash mathbf\backslash mathbf)\; =\; \backslash operatorname(\backslash mathbf\backslash mathbf\backslash mathbf),$$where the first equality is because the traces of a matrix and its transpose are equal. Note that this is not true in general for more than three factors.

Trace of a Kronecker product

The trace of the Kronecker product of two matrices is the product of their traces:$$\backslash operatorname(\backslash mathbf\; \backslash otimes\; \backslash mathbf)\; =\; \backslash operatorname(\backslash mathbf)\backslash operatorname(\backslash mathbf).$$

Characterization of the trace

The following three properties:characterize the trace up to a scalar multiple in the following sense: If

f is a linear functional on the space of square matrices that satisfies f(xy) = f(yx), then f and \operatorname{tr} are proportional.^[7]

For

n x n matrices, imposing the normalization f( I ) = n makes f equal to the trace.

Trace as the sum of eigenvalues

Given any matrix, there is

where are the eigenvalues of counted with multiplicity. This holds true even if is a real matrix and some (or all) of the eigenvalues are complex numbers. This may be regarded as a consequence of the existence of the Jordan canonical form, together with the similarity-invariance of the trace discussed above.

Trace of commutator

When both and are matrices, the trace of the (ring-theoretic) commutator of and vanishes:, because and is linear. One can state this as "the trace is a map of Lie algebras from operators to scalars", as the commutator of scalars is trivial (it is an Abelian Lie algebra). In particular, using similarity invariance, it follows that the identity matrix is never similar to the commutator of any pair of matrices.

Conversely, any square matrix with zero trace is a linear combination of the commutators of pairs of matrices.^[8] Moreover, any square matrix with zero trace is unitarily equivalent to a square matrix with diagonal consisting of all zeros.

Relationship to the characteristic polynomial

The trace of an

n x n matrix A is the coefficient of t ^{n-1} in the characteristic polynomial, possibly changed of sign, according to the convention in the definition of the characteristic polynomial.

Relationship to eigenvalues

If is a linear operator represented by a square matrix with real or complex entries and if are the eigenvalues of (listed according to their algebraic multiplicities), then

This follows from the fact that is always similar to its Jordan form, an upper triangular matrix having on the main diagonal. In contrast, the determinant of is the product of its eigenvalues; that is,$$\backslash det(\backslash mathbf)\; =\; \backslash prod\_i\; \backslash lambda\_i.$$

Everything in the present section applies as well to any square matrix with coefficients in an algebraically closed field.

Derivative relationships

If is a square matrix with small entries and denotes the identity matrix, then we have approximately

$$\backslash det(\backslash mathbf+\backslash mathbf)\backslash approx\; 1\; +\; \backslash operatorname(\backslash mathbf).$$

$$d\backslash det(\backslash mathbf)\; =\; \backslash operatorname\; \backslash big(\backslash operatorname(\backslash mathbf)\backslash cdot\; d\backslash mathbf\backslash big)$$

is more general and describes the differential of the determinant at an arbitrary square matrix, in terms of the trace and the adjugate of the matrix.

From this (or from the connection between the trace and the eigenvalues), one can derive a relation between the trace function, the matrix exponential function, and the determinant:$$\backslash det(\backslash exp(\backslash mathbf))\; =\; \backslash exp(\backslash operatorname(\backslash mathbf)).$$

A related characterization of the trace applies to linear vector fields. Given a matrix, define a vector field on by . The components of this vector field are linear functions (given by the rows of). Its divergence is a constant function, whose value is equal to .

By the divergence theorem, one can interpret this in terms of flows: if represents the velocity of a fluid at location and is a region in, the net flow of the fluid out of is given by, where is the volume of .

The trace is a linear operator, hence it commutes with the derivative:$$d\; \backslash operatorname\; (\backslash mathbf)\; =\; \backslash operatorname(d\backslash mathbf)\; .$$

Trace of a linear operator

In general, given some linear map (where is a finite-dimensional vector space), we can define the trace of this map by considering the trace of a matrix representation of, that is, choosing a basis for and describing as a matrix relative to this basis, and taking the trace of this square matrix. The result will not depend on the basis chosen, since different bases will give rise to similar matrices, allowing for the possibility of a basis-independent definition for the trace of a linear map.

Such a definition can be given using the canonical isomorphism between the space of linear maps on and, where is the dual space of . Let be in and let be in . Then the trace of the indecomposable element is defined to be ; the trace of a general element is defined by linearity. The trace of a linear map can then be defined as the trace, in the above sense, of the element of corresponding to f under the above mentioned canonical isomorphism. Using an explicit basis for and the corresponding dual basis for, one can show that this gives the same definition of the trace as given above.

Numerical algorithms

Stochastic estimator

The trace can be estimated unbiasedly by "Hutchinson's trick":^[9]

Given any matrix

W\in \R ^{n x} , and any random u\in \R ^{n} with E[uu ^{T]} = I , we have E[u ^{T} W u] = tr(W) . (Proof: expand the expectation directly.)

Usually, the random vector is sampled from N( 0 , I) (normal distribution) or \{\pm n ^{-1/2} \} ^{n} (Rademacher distribution).

More sophisticated stochastic estimators of trace have been developed.^[10]

Applications

If a 2 x 2 real matrix has zero trace, its square is a diagonal matrix.

The trace of a 2 × 2 complex matrix is used to classify Möbius transformations. First, the matrix is normalized to make its determinant equal to one. Then, if the square of the trace is 4, the corresponding transformation is parabolic. If the square is in the interval, it is elliptic. Finally, if the square is greater than 4, the transformation is loxodromic. See classification of Möbius transformations.

The trace is used to define characters of group representations. Two representations of a group are equivalent (up to change of basis on) if for all .

The trace also plays a central role in the distribution of quadratic forms.

Lie algebra

The trace is a map of Lie algebras

\operatorname{tr}: ak{gl} _{n\to} K from the Lie algebra ak{gl} _{n} of linear operators on an -dimensional space (matrices with entries in K ) to the Lie algebra of scalars; as is Abelian (the Lie bracket vanishes), the fact that this is a map of Lie algebras is exactly the statement that the trace of a bracket vanishes:$$\backslash operatorname([\backslash mathbf\{A\},\; \backslash mathbf\{B\}])\; =\; 0\; \backslash text\backslash mathbf\; A,\backslash mathbf\; B\backslash in\backslash mathfrak\_n.$$ak{sl} _{n} , which is the Lie algebra of the special linear group of matrices with determinant 1. The special linear group consists of the matrices which do not change volume, while the special linear Lie algebra is the matrices which do not alter volume of infinitesimal sets.

In fact, there is an internal direct sum decomposition

ak{gl} _{n} = ak{sl} _{n} ⊕ K of operators/matrices into traceless operators/matrices and scalars operators/matrices. The projection map onto scalar operators can be expressed in terms of the trace, concretely as:$$\backslash mathbf\; \backslash mapsto\; \backslash frac\backslash operatorname(\backslash mathbf)\backslash mathbf.$$

Formally, one can compose the trace (the counit map) with the unit map

K\to ak{gl} _{n} of "inclusion of scalars" to obtain a map ak{gl} _{n\toak{gl}} _{n} mapping onto scalars, and multiplying by . Dividing by makes this a projection, yielding the formula above.

In terms of short exact sequences, one has$$0\; \backslash to\; \backslash mathfrak\_n\; \backslash to\; \backslash mathfrak\_n\; \backslash overset\; K\; \backslash to\; 0$$which is analogous to$$1\; \backslash to\; \backslash operatorname\_n\; \backslash to\; \backslash operatorname\_n\; \backslash overset\; K^*\; \backslash to\; 1$$(where

K ^{*=K\setminus\{0\}} ) for Lie groups. However, the trace splits naturally (via 1 /n times scalars) so ak{gl} _{n=ak{sl}} _{n ⊕} K , but the splitting of the determinant would be as the th root times scalars, and this does not in general define a function, so the determinant does not split and the general linear group does not decompose:$$\backslash operatorname\_n\; \backslash neq\; \backslash operatorname\_n\; \backslash times\; K^*.$$

Bilinear forms

The bilinear form (where, are square matrices)$$B(\backslash mathbf,\; \backslash mathbf)\; =\; \backslash operatorname(\backslash operatorname(\backslash mathbf)\backslash operatorname(\backslash mathbf))\backslash quad\; \backslash text\; \backslash operatorname(\backslash mathbf)\backslash mathbf\; =\; [\backslash mathbf\{X\},\; \backslash mathbf\{Y\}]\; =\; \backslash mathbf\backslash mathbf\; -\; \backslash mathbf\backslash mathbf$$is called the Killing form, which is used for the classification of Lie algebras.

The trace defines a bilinear form:$$(\backslash mathbf,\; \backslash mathbf)\; \backslash mapsto\; \backslash operatorname(\backslash mathbf\backslash mathbf).$$

The form is symmetric, non-degenerate^[11] and associative in the sense that:$$\backslash operatorname(\backslash mathbf[\backslash mathbf\{Y\},\; \backslash mathbf\{Z\}])\; =\; \backslash operatorname([\backslash mathbf\{X\},\; \backslash mathbf\{Y\}]\backslash mathbf).$$

For a complex simple Lie algebra (such as), every such bilinear form is proportional to each other; in particular, to the Killing form.

Two matrices and are said to be trace orthogonal if$$\backslash operatorname(\backslash mathbf\backslash mathbf)\; =\; 0.$$

There is a generalization to a general representation

(\rho, ak{g},V) of a Lie algebra ak{g} , such that \rho is a homomorphism of Lie algebras \rho: ak{g} → End (V). The trace form tr _{V} on End (V) is defined as above. The bilinear form$$\backslash phi(\backslash mathbf,\backslash mathbf)\; =\; \backslash text\_V(\backslash rho(\backslash mathbf)\backslash rho(\backslash mathbf))$$is symmetric and invariant due to cyclicity.

Generalizations

The concept of trace of a matrix is generalized to the trace class of compact operators on Hilbert spaces, and the analog of the Frobenius norm is called the Hilbert–Schmidt norm.

(e _{n)} _{n} , the trace is given by$$\backslash operatorname(K)\; =\; \backslash sum\_n\; \backslash left\backslash langle\; e\_n,\; Ke\_n\; \backslash right\backslash rangle,$$and is finite and independent of the orthonormal basis.^[12]

The partial trace is another generalization of the trace that is operator-valued. The trace of a linear operator which lives on a product space is equal to the partial traces over and :$$\backslash operatorname(Z)\; =\; \backslash operatorname\_A\; \backslash left(\backslash operatorname\_B(Z)\backslash right)\; =\; \backslash operatorname\_B\; \backslash left(\backslash operatorname\_A(Z)\backslash right).$$

For more properties and a generalization of the partial trace, see traced monoidal categories.

If is a general associative algebra over a field, then a trace on is often defined to be any map which vanishes on commutators; for all . Such a trace is not uniquely defined; it can always at least be modified by multiplication by a nonzero scalar.

A supertrace is the generalization of a trace to the setting of superalgebras.

The operation of tensor contraction generalizes the trace to arbitrary tensors.

Traces in the language of tensor products

Given a vector space, there is a natural bilinear map given by sending to the scalar . The universal property of the tensor product automatically implies that this bilinear map is induced by a linear functional on .^[13]

Similarly, there is a natural bilinear map given by sending to the linear map . The universal property of the tensor product, just as used previously, says that this bilinear map is induced by a linear map . If is finite-dimensional, then this linear map is a linear isomorphism. This fundamental fact is a straightforward consequence of the existence of a (finite) basis of, and can also be phrased as saying that any linear map can be written as the sum of (finitely many) rank-one linear maps. Composing the inverse of the isomorphism with the linear functional obtained above results in a linear functional on . This linear functional is exactly the same as the trace.

Using the definition of trace as the sum of diagonal elements, the matrix formula is straightforward to prove, and was given above. In the present perspective, one is considering linear maps and, and viewing them as sums of rank-one maps, so that there are linear functionals and and nonzero vectors and such that and for any in . Then

(S\circ T)(u) = \sum _{i\varphi} _{i\left(\sum} _{j\psi} _{j(u)w} _{j\right)v} _{i=\sum} _{i\sum} _{j\psi} _{j(u)\varphi} _{i(w} _{j)v} _{i} for any in . The rank-one linear map has trace and so\operatorname{tr}(S\circ T) = \sum _{i\sum} _{j\psi} _{j(v} _{i)\varphi} _{i(w} _{j)=\sum} _{j\sum} _{i\varphi} _{i(w} _{j)\psi} _{j(v} _{i).} Following the same procedure with and reversed, one finds exactly the same formula, proving that equals .

The above proof can be regarded as being based upon tensor products, given that the fundamental identity of with is equivalent to the expressibility of any linear map as the sum of rank-one linear maps. As such, the proof may be written in the notation of tensor products. Then one may consider the multilinear map given by sending to . Further composition with the trace map then results in, and this is unchanged if one were to have started with instead. One may also consider the bilinear map given by sending to the composition, which is then induced by a linear map . It can be seen that this coincides with the linear map . The established symmetry upon composition with the trace map then establishes the equality of the two traces.

For any finite dimensional vector space, there is a natural linear map ; in the language of linear maps, it assigns to a scalar the linear map . Sometimes this is called coevaluation map, and the trace is called evaluation map. These structures can be axiomatized to define categorical traces in the abstract setting of category theory.

See also

• Trace of a tensor with respect to a metric tensor
• Characteristic function
• Field trace
• Golden–Thompson inequality
• Singular trace
• Specht's theorem
• Trace class
• Trace identity
• Trace inequalities
• von Neumann's trace inequality

References

• Book: Gantmacher, F.R. . Felix Gantmacher . 1959 . The Theory of Matrices . K.A. . Hirsch . Kurt Hirsch . New York, NY . . 0107649.
• Book: 2978290 . Horn . R.A. . Roger Horn . Johnson . C.R. . Charles Royal Johnson . 2013 . 1985 . Matrix Analysis . 2nd . . Cambridge, UK . 978-0-521-54823-6.
• Book: Strang, G. . Gilbert Strang . 2004 . 1976 . 4th . Linear Algebra and its Applications . . 978-003010567-8.

Notes and References

1. Web site: Rank, trace, determinant, transpose, and inverse of matrices. 2020-09-09. fourier.eng.hmc.edu.
2. Encyclopedia: Trace (matrix) . Weisstein . Eric W. . Eric W. Weisstein . Eric W. . Weisstein . . 2nd . 1999 . 2003 . . Boca Raton, FL . 1-58488-347-2. 1944431 . 2020-09-09. 1079.00009. 10.1201/9781420035223.
3. Book: Seymour . Lipschutz . Marc . Lipson . September 2005 . Theory and Problems of Linear Algebra . Schaum's Outline . McGraw-Hill . 9780070605022 .
4. Book: Matrix Analysis . 2nd . Roger A. . Horn . Charles R. . Johnson . 9780521839402 . Cambridge University Press. 2013.
5. This is immediate from the definition of the matrix product:$$\backslash operatorname(\backslash mathbf\backslash mathbf)\; =\; \backslash sum\_^m\; \backslash left(\backslash mathbf\backslash mathbf\backslash right)\_\; =\; \backslash sum\_^m\; \backslash sum\_^n\; a\_\; b\_\; =\; \backslash sum\_^n\; \backslash sum\_^m\; b\_\; a\_\; =\; \backslash sum\_^n\; \backslash left(\backslash mathbf\backslash mathbf\backslash right)\_\; =\; \backslash operatorname(\backslash mathbf\backslash mathbf).$$
6. For example, if

   then the product isand the traces are .

7. Proof: Let e _{ij} the standard basis and note that f\left(e _{ij} \right) = 0 if and only if i ≠ j and f\left(e _{jj} \right) = f\left(e _{11} \right) $$f(\backslash mathbf)\; =\; \backslash sum\_\; [\backslash mathbf\{A\}]\_\; f\backslash left(e\_\backslash right)\; =\; \backslash sum\_i\; [\backslash mathbf\{A\}]\_\; f\backslash left(e\_\backslash right)\; =\; f\backslash left(e\_\backslash right)\; \backslash operatorname(\backslash mathbf).$$

   More abstractly, this corresponds to the decomposition$$\backslash mathfrak\_n\; =\; \backslash mathfrak\_n\; \backslash oplus\; k,$$as

   \operatorname{tr}(AB) = \operatorname{tr}(BA) (equivalently, \operatorname{tr}([A, B]) = 0 ) defines the trace on ak{sl} _{n,} which has complement the scalar matrices, and leaves one degree of freedom: any such map is determined by its value on scalars, which is one scalar parameter and hence all are multiple of the trace, a nonzero such map.
8. Proof: ak{sl} _{n} is a semisimple Lie algebra and thus every element in it is a linear combination of commutators of some pairs of elements, otherwise the derived algebra would be a proper ideal.
9. Hutchinson . M.F. . January 1989 . A Stochastic Estimator of the Trace of the Influence Matrix for Laplacian Smoothing Splines . Communications in Statistics - Simulation and Computation . en . 18 . 3 . 1059–1076 . 10.1080/03610918908812806 . 0361-0918.
10. Avron . Haim . Toledo . Sivan . 2011-04-11 . Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix . Journal of the ACM . 58 . 2 . 8:1–8:34 . 10.1145/1944345.1944349 . 5827717 . 0004-5411.
11. This follows from the fact that if and only if .
12. Book: Teschl, G. . Mathematical Methods in Quantum Mechanics . Graduate Studies in Mathematics . 157 . 30 October 2014 . American Mathematical Society . 978-1470417048 . 2nd.
13. Book: Kassel. Christian. Quantum groups. Graduate Texts in Mathematics. 155. Springer-Verlag. New York. 1995. 0-387-94370-6. 1321145. 10.1007/978-1-4612-0783-2. 0808.17003.