Cholesky decomposition explained

In linear algebra, the Cholesky decomposition or Cholesky factorization (pronounced) is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for efficient numerical solutions, e.g., Monte Carlo simulations. It was discovered by André-Louis Cholesky for real matrices, and posthumously published in 1924.^[1] When it is applicable, the Cholesky decomposition is roughly twice as efficient as the LU decomposition for solving systems of linear equations.^[2]

Statement

The Cholesky decomposition of a Hermitian positive-definite matrix, is a decomposition of the form

$$\backslash mathbf\; =\; \backslash mathbf^,$$

where is a lower triangular matrix with real and positive diagonal entries, and * denotes the conjugate transpose of . Every Hermitian positive-definite matrix (and thus also every real-valued symmetric positive-definite matrix) has a unique Cholesky decomposition.^[3]

The converse holds trivially: if can be written as for some invertible, lower triangular or otherwise, then is Hermitian and positive definite.

When is a real matrix (hence symmetric positive-definite), the factorization may be written$$\backslash mathbf\; =\; \backslash mathbf^\backslash mathsf,$$where is a real lower triangular matrix with positive diagonal entries.^{[4] [5] [6]}

Positive semidefinite matrices

If a Hermitian matrix is only positive semidefinite, instead of positive definite, then it still has a decomposition of the form where the diagonal entries of are allowed to be zero.^[7] The decomposition need not be unique, for example:for any . However, if the rank of is, then there is a unique lower triangular with exactly positive diagonal elements and columns containing all zeroes.^[8]

Alternatively, the decomposition can be made unique when a pivoting choice is fixed. Formally, if is an positive semidefinite matrix of rank, then there is at least one permutation matrix such that has a unique decomposition of the form with,where is an lower triangular matrix with positive diagonal.^[9]

LDL decomposition

A closely related variant of the classical Cholesky decomposition is the LDL decomposition,

$$\backslash mathbf\; =\; \backslash mathbf^*,$$

where is a lower unit triangular (unitriangular) matrix, and is a diagonal matrix. That is, the diagonal elements of are required to be 1 at the cost of introducing an additional diagonal matrix in the decomposition. The main advantage is that the LDL decomposition can be computed and used with essentially the same algorithms, but avoids extracting square roots.^[10]

For this reason, the LDL decomposition is often called the square-root-free Cholesky decomposition. For real matrices, the factorization has the form and is often referred to as decomposition (or decomposition, or LDL′). It is reminiscent of the eigendecomposition of real symmetric matrices,, but is quite different in practice because and are not similar matrices.

The LDL decomposition is related to the classical Cholesky decomposition of the form as follows:

$$\backslash mathbf\; =\; \backslash mathbf^*\; =\; \backslash mathbf\; L\; \backslash mathbf\; D^\; \backslash left(\backslash mathbf\; D^\; \backslash right)^*\; \backslash mathbf\; L^*\; =\backslash mathbf\; L\; \backslash mathbf\; D^\; \backslash left(\backslash mathbf\; L\; \backslash mathbf\; D^\backslash right)^*.$$

Conversely, given the classical Cholesky decomposition $\backslash mathbf\; A\; =\; \backslash mathbf\; C\; \backslash mathbf\; C^*$ of a positive definite matrix, if is a diagonal matrix that contains the main diagonal of $\backslash mathbf\; C$, then can be decomposed as $\backslash mathbf\; L\; \backslash mathbf\; D\; \backslash mathbf\; L^*$ where$$\backslash mathbf\; L\; =\; \backslash mathbf\; C\; \backslash mathbf\; S^$$ (this rescales each column to make diagonal elements 1),$$\backslash mathbf\; D\; =\; \backslash mathbf\; S^2.$$

If is positive definite then the diagonal elements of are all positive.For positive semidefinite, an $\backslash mathbf\; L\; \backslash mathbf\; D\; \backslash mathbf\; L^*$ decomposition exists where the number of non-zero elements on the diagonal is exactly the rank of .^[11] Some indefinite matrices for which no Cholesky decomposition exists have an LDL decomposition with negative entries in : it suffices that the first leading principal minors of are non-singular.

Example

Here is the Cholesky decomposition of a symmetric real matrix:

And here is its LDL^T decomposition:

Geometric interpretation

See also: Whitening transformation. The Cholesky decomposition is equivalent to a particular choice of conjugate axes of an ellipsoid.^[12] In detail, let the ellipsoid be defined as $y^TAy\; =\; 1$, then by definition, a set of vectors $v\_1,\; ...,\; v\_n$ are conjugate axes of the ellipsoid iff $v\_i^T\; A\; v\_j\; =\; \backslash delta\_$. Then, the ellipsoid is precisely$$\backslash left\backslash \; =\; f(\backslash mathbb\; S^n)$$where $f$ maps the basis vector $e\_i\; \backslash mapsto\; v\_i$, and $\backslash mathbb\; S^n$ is the unit sphere in n dimensions. That is, the ellipsoid is a linear image of the unit sphere.

Define the matrix $V\; :=\; [v\_1\; |\; v\_2\; |\; \backslash cdots\; |\; v\_n]$, then $v\_i^T\; A\; v\_j\; =\; \backslash delta\_$ is equivalent to $V^TAV\; =\; I$. Different choices of the conjugate axes correspond to different decompositions.

The Cholesky decomposition corresponds to choosing $v\_1$ to be parallel to the first axis, $v\_2$ to be within the plane spanned by the first two axes, and so on. This makes $V$ an upper-triangular matrix. Then, there is $A\; =\; LL^T$, where $L\; =\; (V^)^T$ is lower-triangular.

Similarly, principal component analysis corresponds to choosing $v\_1,\; ...,\; v\_n$ to be perpendicular. Then, let $\backslash lambda\; =\; 1/\backslash |v\_i\backslash |^2$ and $\backslash Sigma\; =\; \backslash mathrm(\backslash lambda\_1,\; ...,\; \backslash lambda\_n)$, and there is $V\; =\; U\backslash Sigma^$ where $U$ is an orthogonal matrix. This then yields $A\; =\; U\backslash Sigma\; U^T$.

Applications

Numerical solution of system of linear equations

The Cholesky decomposition is mainly used for the numerical solution of linear equations $\backslash mathbf\; =\; \backslash mathbf$. If is symmetric and positive definite, then $\backslash mathbf\; =\; \backslash mathbf$ can be solved by first computing the Cholesky decomposition $\backslash mathbf\; =\; \backslash mathbf^\backslash mathrm$, then solving $\backslash mathbf\; =\; \backslash mathbf$ for by forward substitution, and finally solving $\backslash mathbf\; =\; \backslash mathbf$ for by back substitution.

An alternative way to eliminate taking square roots in the $\backslash mathbf^\backslash mathrm$ decomposition is to compute the LDL decomposition $\backslash mathbf\; =\; \backslash mathbf^\backslash mathrm$, then solving $\backslash mathbf\; =\; \backslash mathbf$ for, and finally solving $\backslash mathbf^\backslash mathrm\backslash mathbf\; =\; \backslash mathbf$.

For linear systems that can be put into symmetric form, the Cholesky decomposition (or its LDL variant) is the method of choice, for superior efficiency and numerical stability. Compared to the LU decomposition, it is roughly twice as efficient.^[2]

Linear least squares

Systems of the form with symmetric and positive definite arise quite often in applications. For instance, the normal equations in linear least squares problems are of this form. It may also happen that matrix comes from an energy functional, which must be positive from physical considerations; this happens frequently in the numerical solution of partial differential equations.

Non-linear optimization

Non-linear multi-variate functions may be minimized over their parameters using variants of Newton's method called quasi-Newton methods. At iteration k, the search steps in a direction $p\_k$ defined by solving $B\_k\; p\_k\; =\; -g\_k$ for $p\_k$, where $p\_k$ is the step direction, $g\_k$ is the gradient, and $B\_k$ is an approximation to the Hessian matrix formed by repeating rank-1 updates at each iteration. Two well-known update formulas are called Davidon–Fletcher–Powell (DFP) and Broyden–Fletcher–Goldfarb–Shanno (BFGS). Loss of the positive-definite condition through round-off error is avoided if rather than updating an approximation to the inverse of the Hessian, one updates the Cholesky decomposition of an approximation of the Hessian matrix itself.^[13]

Monte Carlo simulation

The Cholesky decomposition is commonly used in the Monte Carlo method for simulating systems with multiple correlated variables. The covariance matrix is decomposed to give the lower-triangular . Applying this to a vector of uncorrelated observations in a sample produces a sample vector Lu with the covariance properties of the system being modeled.^[14]

The following simplified example shows the economy one gets from the Cholesky decomposition: suppose the goal is to generate two correlated normal variables $x\_1$ and $x\_2$ with given correlation coefficient $\backslash rho$. To accomplish that, it is necessary to first generate two uncorrelated Gaussian random variables $z\_1$ and $z\_2$ (for example, via a Box–Muller transform). Given the required correlation coefficient $\backslash rho$, the correlated normal variables can be obtained via the transformations $x\_1\; =\; z\_1$ and $x\_2\; =\; \backslash rho\; z\_1\; +\; \backslash sqrt\; z\_2$.

Kalman filters

Unscented Kalman filters commonly use the Cholesky decomposition to choose a set of so-called sigma points. The Kalman filter tracks the average state of a system as a vector of length and covariance as an matrix . The matrix is always positive semi-definite and can be decomposed into LL^T. The columns of can be added and subtracted from the mean to form a set of vectors called sigma points. These sigma points completely capture the mean and covariance of the system state.

Matrix inversion

The explicit inverse of a Hermitian matrix can be computed by Cholesky decomposition, in a manner similar to solving linear systems, using $n^3$ operations ($\backslash tfrac\; n^3$ multiplications).^[10] The entire inversion can even be efficiently performed in-place.

A non-Hermitian matrix can also be inverted using the following identity, where BB* will always be Hermitian:

$$\backslash mathbf^\; =\; \backslash mathbf^*\; (\backslash mathbf^*)^.$$

Computation

There are various methods for calculating the Cholesky decomposition. The computational complexity of commonly used algorithms is in general. The algorithms described below all involve about FLOPs (multiplications and the same number of additions) for real flavors and FLOPs for complex flavors,^[15] where is the size of the matrix . Hence, they have half the cost of the LU decomposition, which uses FLOPs (see Trefethen and Bau 1997).

Which of the algorithms below is faster depends on the details of the implementation. Generally, the first algorithm will be slightly slower because it accesses the data in a less regular manner.

The Cholesky algorithm

The Cholesky algorithm, used to calculate the decomposition matrix, is a modified version of Gaussian elimination.

The recursive algorithm starts with and

.

At step, the matrix has the following form:where denotes the identity matrix of dimension .

If the matrix is defined by(note that since is positive definite),then can be written as$$\backslash mathbf^\; =\; \backslash mathbf\_\; \backslash mathbf^\; \backslash mathbf\_^$$whereNote that is an outer product, therefore this algorithm is called the outer-product version in (Golub & Van Loan).

This is repeated for from 1 to . After steps, is obtained, and hence, the lower triangular matrix sought for is calculated as

$$\backslash mathbf\; :=\; \backslash mathbf\_\; \backslash mathbf\_\; \backslash dots\; \backslash mathbf\_.$$

The Cholesky–Banachiewicz and Cholesky–Crout algorithms

If the equation

is written out, the following is obtained:

and therefore the following formulas for the entries of :

$$L\_\; =\; (\backslash pm)\backslash sqrt,$$$$L\_\; =\; \backslash frac\; \backslash left(A\_\; -\; \backslash sum\_^\; L\_\; L\_\; \backslash right)\; \backslash quad\; \backslash text\; i>j.$$

For complex and real matrices, inconsequential arbitrary sign changes of diagonal and associated off-diagonal elements are allowed. The expression under the square root is always positive if is real and positive-definite.

For complex Hermitian matrix, the following formula applies:

$$L\_\; =\; \backslash sqrt,$$$$L\_\; =\; \backslash frac\; \backslash left(A\_\; -\; \backslash sum\_^\; L\_^*\; L\_\; \backslash right)\; \backslash quad\; \backslash text\; i>j.$$

So it now is possible to compute the entry if the entries to the left and above are known. The computation is usually arranged in either of the following orders:

• The Cholesky–Banachiewicz algorithm starts from the upper left corner of the matrix and proceeds to calculate the matrix row by row.

for (i = 0; i < dimensionSize; i++) The above algorithm can be succinctly expressed as combining a dot product and matrix multiplication in vectorized programming languages such as Fortran as the following,do i = 1, size(A,1) L(i,i) = sqrt(A(i,i) - dot_product(L(i,1:i-1), L(i,1:i-1))) L(i+1:,i) = (A(i+1:,i) - matmul(conjg(L(i,1:i-1)), L(i+1:,1:i-1))) / L(i,i)end dowhere conjg refers to complex conjugate of the elements.

• The Cholesky–Crout algorithm starts from the upper left corner of the matrix and proceeds to calculate the matrix column by column.

for (j = 0; j < dimensionSize; j++) The above algorithm can be succinctly expressed as combining a dot product and matrix multiplication in vectorized programming languages such as Fortran as the following,do i = 1, size(A,1) L(i,i) = sqrt(A(i,i) - dot_product(L(1:i-1,i), L(1:i-1,i))) L(i,i+1:) = (A(i,i+1:) - matmul(conjg(L(1:i-1,i)), L(1:i-1,i+1:))) / L(i,i)end dowhere conjg refers to complex conjugate of the elements.

Either pattern of access allows the entire computation to be performed in-place if desired.

Stability of the computation

Suppose that there is a desire to solve a well-conditioned system of linear equations. If the LU decomposition is used, then the algorithm is unstable unless some sort of pivoting strategy is used. In the latter case, the error depends on the so-called growth factor of the matrix, which is usually (but not always) small.

Now, suppose that the Cholesky decomposition is applicable. As mentioned above, the algorithm will be twice as fast. Furthermore, no pivoting is necessary, and the error will always be small. Specifically, if, and denotes the computed solution, then solves the perturbed system (where$$\backslash |\backslash mathbf\backslash |\_2\; \backslash le\; c\_n\; \backslash varepsilon\; \backslash |\backslash mathbf\backslash |\_2.$$Here ||·||₂ is the matrix 2-norm, c_n is a small constant depending on, and denotes the unit round-off.

One concern with the Cholesky decomposition to be aware of is the use of square roots. If the matrix being factorized is positive definite as required, the numbers under the square roots are always positive in exact arithmetic. Unfortunately, the numbers can become negative because of round-off errors, in which case the algorithm cannot continue. However, this can only happen if the matrix is very ill-conditioned. One way to address this is to add a diagonal correction matrix to the matrix being decomposed in an attempt to promote the positive-definiteness.^[16] While this might lessen the accuracy of the decomposition, it can be very favorable for other reasons; for example, when performing Newton's method in optimization, adding a diagonal matrix can improve stability when far from the optimum.

LDL decomposition

An alternative form, eliminating the need to take square roots when is symmetric, is the symmetric indefinite factorization^[17]

The following recursive relations apply for the entries of and :$$D\_j\; =\; A\_\; -\; \backslash sum\_^\; L\_^2\; D\_k,$$$$L\_\; =\; \backslash frac\; \backslash left(A\_\; -\; \backslash sum\_^\; L\_\; L\_\; D\_k\; \backslash right)\; \backslash quad\; \backslash text\; i>j.$$

This works as long as the generated diagonal elements in stay non-zero. The decomposition is then unique. and are real if is real.

For complex Hermitian matrix, the following formula applies:

$$D\_\; =\; A\_\; -\; \backslash sum\_^\; L\_L\_^*\; D\_k,$$$$L\_\; =\; \backslash frac\; \backslash left(A\_\; -\; \backslash sum\_^\; L\_\; L\_^*\; D\_k\; \backslash right)\; \backslash quad\; \backslash text\; i>j.$$

Again, the pattern of access allows the entire computation to be performed in-place if desired.

Block variant

When used on indefinite matrices, the LDL* factorization is known to be unstable without careful pivoting;^[18] specifically, the elements of the factorization can grow arbitrarily. A possible improvement is to perform the factorization on block sub-matrices, commonly 2 × 2:^[19]

where every element in the matrices above is a square submatrix. From this, these analogous recursive relations follow:

$$\backslash mathbf\; D\_j\; =\; \backslash mathbf\; A\_\; -\; \backslash sum\_^\; \backslash mathbf\; L\_\; \backslash mathbf\; D\_k\; \backslash mathbf\; L\_^\backslash mathrm\; T,$$$$\backslash mathbf\; L\_\; =\; \backslash left(\backslash mathbf\; A\_\; -\; \backslash sum\_^\; \backslash mathbf\; L\_\; \backslash mathbf\; D\_k\; \backslash mathbf\; L\_^\backslash mathrm\; T\backslash right)\; \backslash mathbf\; D\_j^.$$

This involves matrix products and explicit inversion, thus limiting the practical block size.

Updating the decomposition

A task that often arises in practice is that one needs to update a Cholesky decomposition. In more details, one has already computed the Cholesky decomposition $\backslash mathbf\; =\; \backslash mathbf\backslash mathbf^*$ of some matrix $\backslash mathbf$, then one changes the matrix $\backslash mathbf$ in some way into another matrix, say $\backslash tilde$, and one wants to compute the Cholesky decomposition of the updated matrix: $\backslash tilde\; =\; \backslash tilde\; \backslash tilde^*$. The question is now whether one can use the Cholesky decomposition of $\backslash mathbf$ that was computed before to compute the Cholesky decomposition of $\backslash tilde$.

Rank-one update

The specific case, where the updated matrix $\backslash tilde$ is related to the matrix $\backslash mathbf$ by $\backslash tilde\; =\; \backslash mathbf\; +\; \backslash mathbf\; \backslash mathbf^*$, is known as a rank-one update.

Here is a function^[20] written in Matlab syntax that realizes a rank-one update:function [L] = cholupdate(L, x) n = length(x); for k = 1:n r = sqrt(L(k, k)^2 + x(k)^2); c = r / L(k, k); s = x(k) / L(k, k); L(k, k) = r; if k < n L((k+1):n, k) = (L((k+1):n, k) + s * x((k+1):n)) / c; x((k+1):n) = c * x((k+1):n) - s * L((k+1):n, k); end endendA rank-n update is one where for a matrix $\backslash mathbf$ one updates the decomposition such that $\backslash tilde\; =\; \backslash mathbf\; +\; \backslash mathbf\; \backslash mathbf^*$. This can be achieved by successively performing rank-one updates for each of the columns of $\backslash mathbf$.

Rank-one downdate

A rank-one downdate is similar to a rank-one update, except that the addition is replaced by subtraction: $\backslash tilde\; =\; \backslash mathbf\; -\; \backslash mathbf\; \backslash mathbf^*$. This only works if the new matrix $\backslash tilde$ is still positive definite.

The code for the rank-one update shown above can easily be adapted to do a rank-one downdate: one merely needs to replace the two additions in the assignment to r and L((k+1):n, k) by subtractions.

Adding and removing rows and columns

If a symmetric and positive definite matrix $\backslash mathbf\; A$ is represented in block form as

and its upper Cholesky factor

then for a new matrix $\backslash tilde$, which is the same as $\backslash mathbf\; A$ but with the insertion of new rows and columns,

Now there is an interest in finding the Cholesky factorization of $\backslash tilde$, which can be called $\backslash tilde$, without directly computing the entire decomposition.

Writing $\backslash mathbf\; A\; \backslash setminus\; \backslash mathbf$ for the solution of $\backslash mathbf\; A\; \backslash mathbf\; x\; =\; \backslash mathbf\; b$, which can be found easily for triangular matrices, and $\backslash text\; (\backslash mathbf\; M)$ for the Cholesky decomposition of $\backslash mathbf\; M$, the following relations can be found:

These formulas may be used to determine the Cholesky factor after the insertion of rows or columns in any position, if the row and column dimensions are appropriately set (including to zero). The inverse problem,

with known Cholesky decomposition

and the desire to determine the Cholesky factor

of the matrix $\backslash mathbf\; A$ with rows and columns removed,

yields the following rules:

Notice that the equations above that involve finding the Cholesky decomposition of a new matrix are all of the form $\backslash tilde\; =\; \backslash mathbf\; \backslash pm\; \backslash mathbf\; \backslash mathbf^*$, which allows them to be efficiently calculated using the update and downdate procedures detailed in the previous section.^[21]

Proof for positive semi-definite matrices

Proof by limiting argument

The above algorithms show that every positive definite matrix $\backslash mathbf$ has a Cholesky decomposition. This result can be extended to the positive semi-definite case by a limiting argument. The argument is not fully constructive, i.e., it gives no explicit numerical algorithms for computing Cholesky factors.

If $\backslash mathbf$ is an $n\; \backslash times\; n$ positive semi-definite matrix, then the sequence $\backslash left(\backslash mathbf\_k\backslash right)\_k\; :=\; \backslash left(\backslash mathbf\; +\; \backslash frac\; \backslash mathbf\_n\backslash right)\_k$ consists of positive definite matrices. (This is an immediate consequence of, for example, the spectral mapping theorem for the polynomial functional calculus.) Also,$$\backslash mathbf\_k\; \backslash rightarrow\; \backslash mathbf\backslash quad\; \backslash text\; \backslash quadk\; \backslash rightarrow\; \backslash infty$$ in operator norm. From the positive definite case, each $\backslash mathbf\_k$ has Cholesky decomposition $\backslash mathbf\_k\; =\; \backslash mathbf\_k\backslash mathbf\_k^*$. By property of the operator norm,

$$\backslash |\; \backslash mathbf\_k\; \backslash |^2\; \backslash leq\; \backslash |\; \backslash mathbf\_k\; \backslash mathbf\_k^*\; \backslash |\; =\; \backslash |\; \backslash mathbf\_k\; \backslash |\; \backslash ,.$$

The $\backslash leq$ holds because $M\_n(\backslash mathbb)$ equipped with the operator norm is a C* algebra. So $\backslash left(\backslash mathbf\_k\; \backslash right)\_k$ is a bounded set in the Banach space of operators, therefore relatively compact (because the underlying vector space is finite-dimensional). Consequently, it has a convergent subsequence, also denoted by $\backslash left(\backslash mathbf\_k\; \backslash right)\_k$, with limit $\backslash mathbf$. It can be easily checked that this $\backslash mathbf$ has the desired properties, i.e. $\backslash mathbf\; =\; \backslash mathbf\backslash mathbf^*$, and $\backslash mathbf$ is lower triangular with non-negative diagonal entries: for all $x$ and $y$,

$$\backslash langle\; \backslash mathbf\; x,\; y\; \backslash rangle\; =\; \backslash left\backslash langle\; \backslash lim\; \backslash mathbf\_k\; x,\; y\; \backslash right\backslash rangle\; =\; \backslash langle\; \backslash lim\; \backslash mathbf\_k\; \backslash mathbf\_k^*\; x,\; y\; \backslash rangle\; =\; \backslash langle\; \backslash mathbf\; \backslash mathbf^*x,\; y\; \backslash rangle\; \backslash ,.$$

Therefore, $\backslash mathbf\; =\; \backslash mathbf\backslash mathbf^*$. Because the underlying vector space is finite-dimensional, all topologies on the space of operators are equivalent. So $\backslash left(\backslash mathbf\_k\; \backslash right)\_k$ tends to $\backslash mathbf$ in norm means $\backslash left(\backslash mathbf\_k\; \backslash right)\_k$ tends to $\backslash mathbf$ entrywise. This in turn implies that, since each $\backslash mathbf\_k$ is lower triangular with non-negative diagonal entries, $\backslash mathbf$ is also.

Proof by QR decomposition

Let $\backslash mathbf$ be a positive semi-definite Hermitian matrix. Then it can be written as a product of its square root matrix, $\backslash mathbf\; =\; \backslash mathbf\; \backslash mathbf^*$. Now QR decomposition can be applied to $\backslash mathbf^*$, resulting in $\backslash mathbf^*\; =\; \backslash mathbf\backslash mathbf$, where $\backslash mathbf$ is unitary and $\backslash mathbf$ is upper triangular. Inserting the decomposition into the original equality yields $A\; =\; \backslash mathbf\; \backslash mathbf^*\; =\; (\backslash mathbf)^*\backslash mathbf\; =\; \backslash mathbf^*\backslash mathbf^*\backslash mathbf\; =\; \backslash mathbf^*\backslash mathbf$. Setting $\backslash mathbf\; =\; \backslash mathbf^*$ completes the proof.

Generalization

The Cholesky factorization can be generalized to (not necessarily finite) matrices with operator entries. Let $\backslash$ be a sequence of Hilbert spaces. Consider the operator matrix

acting on the direct sum

$$\backslash mathcal\; =\; \backslash bigoplus\_n\; \backslash mathcal\_n,$$

where each

$$\backslash mathbf\_\; :\; \backslash mathcal\_j\; \backslash rightarrow\; \backslash mathcal\; \_i$$

is a bounded operator. If is positive (semidefinite) in the sense that for all finite and for any

$$h\; \backslash in\; \backslash bigoplus\_^k\; \backslash mathcal\_k,$$

there is $\backslash langle\; h,\; \backslash mathbf\; h\backslash rangle\; \backslash ge\; 0$, then there exists a lower triangular operator matrix such that . One can also take the diagonal entries of to be positive.

Implementations in programming libraries

• C programming language: the GNU Scientific Library provides several implementations of Cholesky decomposition.
• Maxima computer algebra system: function cholesky computes Cholesky decomposition.
• GNU Octave numerical computations system provides several functions to calculate, update, and apply a Cholesky decomposition.
• The LAPACK library provides a high performance implementation of the Cholesky decomposition that can be accessed from Fortran, C and most languages.
• In Python, the function cholesky from the numpy.linalg module performs Cholesky decomposition.
• In Matlab, the chol function gives the Cholesky decomposition. Note that chol uses the upper triangular factor of the input matrix by default, i.e. it computes $A\; =\; R^*\; R$ where $R$ is upper triangular. A flag can be passed to use the lower triangular factor instead.
• In R, the chol function gives the Cholesky decomposition.
• In Julia, the cholesky function from the LinearAlgebra standard library gives the Cholesky decomposition.
• In Mathematica, the function "CholeskyDecomposition" can be applied to a matrix.
• In C++, multiple linear algebra libraries support this decomposition:
  • The Armadillo (C++ library) supplies the command chol to perform Cholesky decomposition.
  • The Eigen library supplies Cholesky factorizations for both sparse and dense matrices.
  • In the ROOT package, the TDecompChol class is available.
• In Analytica, the function Decompose gives the Cholesky decomposition.
• The Apache Commons Math library has an implementation which can be used in Java, Scala and any other JVM language.

See also

• Cycle rank
• Incomplete Cholesky factorization
• Matrix decomposition
• Minimum degree algorithm
• Square root of a matrix
• Sylvester's law of inertia
• Symbolic Cholesky decomposition

References

• Dereniowski . Dariusz . Kubale . Marek . Cholesky Factorization of Matrices in Parallel and Ranking of Graphs . 10.1007/978-3-540-24669-5_127 . 985–992 . Springer-Verlag . Lecture Notes on Computer Science . 5th International Conference on Parallel Processing and Applied Mathematics . 3019 . 2004 . 978-3-540-21946-0 . dead . https://web.archive.org/web/20110716060800/http://www.eti.pg.gda.pl/katedry/kams/wwwkams/pdf/Cholesky_fmprg.pdf . 2011-07-16 .
• Book: Gene H. . Golub . Gene H. Golub . Charles F. . Van Loan . Charles F. Van Loan . 1996 . Matrix Computations . 3rd . Johns Hopkins . Baltimore . 978-0-8018-5414-9.
• Book: Roger A. . Horn . Charles R. . Johnson . 1985 . Matrix Analysis . Cambridge University Press . 0-521-38632-2 .
• S. J. Julier and J. K. Uhlmann. "A General Method for Approximating Nonlinear Transformations of ProbabilityDistributions".
• S. J. Julier and J. K. Uhlmann, "A new extension of the Kalman filter to nonlinear systems", in Proc. AeroSense: 11th Int. Symp. Aerospace/Defence Sensing, Simulation and Controls, 1997, pp. 182–193.
• Book: Trefethen . Lloyd N. . Lloyd N. Trefethen . Bau . David . Numerical linear algebra . Society for Industrial and Applied Mathematics . Philadelphia . 978-0-89871-361-9 . 1997.
• Osborne. Michael. 2010. Bayesian Gaussian Processes for Sequential Prediction, Optimisation and Quadrature. thesis. University of Oxford.
• Ruschel, João Paulo Tarasconi, Bachelor degree "Parallel Implementations of the Cholesky Decomposition on CPUs and GPUs" Universidade Federal Do Rio Grande Do Sul, Instituto De Informatica, 2016, pp. 29-30.

External links

History of science

• Sur la résolution numérique des systèmes d'équations linéaires, Cholesky's 1910 manuscript, online and analyzed on BibNum [for English, click 'A télécharger']

Information

• • Cholesky Decomposition, The Data Analysis BriefBook
• Cholesky Decomposition on www.math-linux.com
• Cholesky Decomposition Made Simple on Science Meanderthal

Computer code

• LAPACK is a collection of FORTRAN subroutines for solving dense linear algebra problems (DPOTRF, DPOTRF2, details performance)
• ALGLIB includes a partial port of the LAPACK to C++, C#, Delphi, Visual Basic, etc. (spdmatrixcholesky, hpdmatrixcholesky)
• libflame is a C library with LAPACK functionality.
• Notes and video on high-performance implementation of Cholesky factorization at The University of Texas at Austin.
• Cholesky : TBB + Threads + SSE is a book explaining the implementation of the CF with TBB, threads and SSE (in Spanish).
• library "Ceres Solver" by Google.
• LDL decomposition routines in Matlab.
• Armadillo is a C++ linear algebra package
• Rosetta Code is a programming chrestomathy site. on page topic.
• AlgoWiki is an open encyclopedia of algorithms’ properties and features of their implementations on page topic
• Intel® oneAPI Math Kernel Library Intel-Optimized Math Library for Numerical Computing ?potrf, ?potrs

Use of the matrix in simulation

• Generating Correlated Random Variables and Stochastic Processes, Martin Haugh, Columbia University

Online calculators

• Online Matrix Calculator Performs Cholesky decomposition of matrices online.

Notes and References

1. Benoit. 1924. Note sur une méthode de résolution des équations normales provenant de l'application de la méthode des moindres carrés à un système d'équations linéaires en nombre inférieur à celui des inconnues (Procédé du Commandant Cholesky). Bulletin Géodésique. fr. 2. 66–67. 10.1007/BF03031308.
2. Book: Press, William H.. Saul A. Teukolsky. William T. Vetterling. Brian P. Flannery. Numerical Recipes in C: The Art of Scientific Computing. second. Cambridge University England EPress. 1992. 994. 0-521-43108-5. 2009-01-28.
3. ,, .
4. .
5. Web site: matrices - Diagonalizing a Complex Symmetric Matrix. MathOverflow. 2020-01-25.
6. Schabauer. Hannes. Pacher. Christoph. Sunderland. Andrew G.. Gansterer. Wilfried N.. 2010-05-01. Toward a parallel solver for generalized complex symmetric eigenvalue problems. Procedia Computer Science. ICCS 2010. en. 1. 1. 437–445. 10.1016/j.procs.2010.04.047. 1877-0509. free.
7. .
8. Book: Gentle, James E. . 1998 . Numerical Linear Algebra for Applications in Statistics . 978-1-4612-0623-1 . Springer . en . 94.
9. Book: Higham, Nicholas J. . http://eprints.maths.manchester.ac.uk/1193/ . Reliable Numerical Computation . Oxford University Press . 1990 . 978-0-19-853564-5 . Cox . M. G. . Oxford, UK . 161–185 . en . Hammarling . S. J. . Analysis of the Cholesky Decomposition of a Semi-definite Matrix.
10. Krishnamoorthy. Aravindh. Menon, Deepak. Matrix Inversion Using Cholesky Decomposition. 1111. 4144. 2011. 2011arXiv1111.4144K. 1111.4144.
11. So . Anthony Man-Cho . A Semidefinite Programming Approach to the Graph Realization Problem: Theory, Applications and Extensions . 2007 . en . PhD. Theorem 2.2.6.
12. Pope, Stephen B. "Algorithms for ellipsoids." Cornell University Report No. FDA (2008): 08-01.
13. Book: Arora, Jasbir Singh . Introduction to Optimum Design . 2004-06-02 . Elsevier . 978-0-08-047025-2 . en.
14. http://www.mathworks.com/help/techdoc/ref/randn.html Matlab randn documentation
15. ?potrf Intel® Math Kernel Library https://software.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-c/top/lapack-routines/lapack-linear-equation-routines/lapack-linear-equation-computational-routines/matrix-factorization-lapack-computational-routines/potrf.html#potrf
16. Fang. Haw-ren. O'Leary, Dianne P.. Dianne P. O'Leary . Modified Cholesky Algorithms: A Catalog with New Approaches. 8 August 2006.
17. Book: Watkins, D. . 1991 . Fundamentals of Matrix Computations . registration . New York . Wiley . 84 . 0-471-61414-9 .
18. Book: Nocedal, Jorge. Numerical Optimization. 2000. Springer.
19. Fang. Haw-ren. Analysis of Block LDLT Factorizations for Symmetric Indefinite Matrices. 24 August 2007.
20. Based on: Book: Stewart, G. W.. Basic decompositions. 1998. Soc. for Industrial and Applied Mathematics. Philadelphia. 0-89871-414-1.
21. Osborne, M. (2010), Appendix B.