Centering matrix explained

In mathematics and multivariate statistics, the centering matrix^[1] is a symmetric and idempotent matrix, which when multiplied with a vector has the same effect as subtracting the mean of the components of the vector from every component of that vector.

Definition

The centering matrix of size n is defined as the n-by-n matrix

C _{n} = I _{n} - \tfrac{ 1 }{n}J _{n} where I _{n} is the identity matrix of size n and J _{n} is an n-by-n matrix of all 1's.

For example

C _{1} = \begin{bmatrix} 0 \end{bmatrix} ,C _{2=} \left[ \begin{array}{rrr} 1 & 0 \\ 0 & 1 \end{array} \right] - (1)/(2) \left[ \begin{array}{rrr} 1 & 1 \\ 1 & 1 \end{array} \right] = \left[ \begin{array}{rrr} (1)/(2) & - (1)/(\\ - 2) (1)/(2) & (1)/(2) \end{array} \right] , C _{3} = \left[ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right] - (1)/(3) \left[ \begin{array}{rrr} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \end{array} \right] = \left[ \begin{array}{rrr} (2)/(3) & - (1)/(3) & - (1)/(\\ - 3) (1)/(3) & (2)/(3) & - (1)/(\\ - 3) (1)/(3) & - (1)/(3) & (2)/(3) \end{array} \right]

Properties

Given a column-vector,

v of size n, the centering property of C _{n} can be expressed asC _{nv} = v - (\tfrac{ 1 }{n}J ^{rm{T} v )J} _{n, 1} where J _{n,1} is a column vector of ones and \tfrac{ 1 }{n}J ^{rm{T} v} _{n, 1} is the mean of the components of v .C _{n} is symmetric positive semi-definite.C _{n} is idempotent, so that C ^{k = C} _{n} , for k =1 , 2 ,\ldots . Once the mean has been removed, it is zero and removing it again has no effect.C _{n} is singular. The effects of applying the transformation C _{nv} cannot be reversed.C _{n} has the eigenvalue 1 of multiplicity n - 1 and eigenvalue 0 of multiplicity 1.C _{n} has a nullspace of dimension 1, along the vector J _{n,1} .C _{n} is an orthogonal projection matrix. That is, C _{nv} is a projection of v onto the (n - 1)-dimensional subspace that is orthogonal to the nullspace J _{n,1} . (This is the subspace of all n-vectors whose components sum to zero.)

The trace of

C _{n} is n(n -1 )/n = n -1 .

Application

Although multiplication by the centering matrix is not a computationally efficient way of removing the mean from a vector, it is a convenient analytical tool. It can be used not only to remove the mean of a single vector, but also of multiple vectors stored in the rows or columns of an m-by-n matrix

X .

The left multiplication by

C _{m} subtracts a corresponding mean value from each of the n columns, so that each column of the product C _{mX} has a zero mean. Similarly, the multiplication by C _{n} on the right subtracts a corresponding mean value from each of the m rows, and each row of the product XC _{n} has a zero mean.The multiplication on both sides creates a doubly centred matrix C _{mXC} _{n} , whose row and column means are equal to zero.

The centering matrix provides in particular a succinct way to express the scatter matrix,

S = (X - \mu J ^{T} _{n, 1} )(X - \mu J ^{T} _{n, 1} ) ^{T} of a data sample X , where \mu = \tfrac{ 1 }{n}X J _{n,1} is the sample mean. The centering matrix allows us to express the scatter matrix more compactly asS = XC _{n(XC} ^{T} ^{n)} = XC _{nC} ^{T} ^{nX} = XC ^{T} _{nX} . C _{n} is the covariance matrix of the multinomial distribution, in the special case where the parameters of that distribution are k = n , and p _{1=p} _{2= … =p} ^{n = (1)/(n)} .

Notes and References

1. John I. Marden, Analyzing and Modeling Rank Data, Chapman & Hall, 1995,, page 59.