Cholesky Decomposition

Cholesky Decomposition is one of the types of many decompositions in linear algebra, which is a branch of mathematics that deals with linear equations and vectors. Decomposition is the term related to the factorization of matrices in linear algebra, and Cholesky is one of the ways to factorize or decompose the matrix into two matrices.

Cholesky Decomposition is the decomposition of a Hermitian, positive-definite matrix into the multiplication of two matrices, where one is a positive diagonal lower triangular matrix and the other is its conjugate transpose matrix, i.e., an upper triangular matrix. As Cholesky decomposition can represent matrices as a product of two matrices, it is also called Cholesky Factorization.

The Cholesky decomposition is defined specially for symmetric matrices, and Cholesky Decomposition is used widely as it is faster than the LU decomposition. Let A be any Hermitian, positive definite matrix; then the Cholesky decomposition can be given by:

A = LL*

Where,

• L is the lower triangular matrix of A with a positive diagonal
• L* is the conjugate transpose matrix of A

Note: Every Hermitian positive definite matrix has a unique Cholesky decomposition.

How to Evaluate Cholesky Decomposition?

To decompose a matrix X using Cholesky decomposition, we first decompose the matrix A in the form A = LL*, where L is assumed to be a positive diagonal lower triangular matrix and L* is the conjugate transpose of L. Then, we have to find the elements of L. To find the value of the diagonal elements, we use the formula

L_vv = √(A_vv - ∑_u<v L_vu (L_vu)*)

And to find the value of non-diagonal elements, we use the formula

L_tv = (1 / L_vv)(A_tv - ∑_u<v L_tu (L_vu)*)

After finding all the elements, arrange them in the lower triangular matrix L and then find L*, which is the conjugate transpose of L. Finally, evaluate the product of LL* to get the simplified matrix of A.

Steps for Cholesky Decomposition

To decompose or factorize any Hermitian symmetric matrix, we can use the following steps:

Step 1: First write the given matrix in the decomposed form.

Let A be the positive definite symmetric matrix which can be decomposed as A = LL^*.

Step 2: Now, we have to evaluate matrix L where L is defined as:

\bold{L = \begin {bmatrix}L_{11}&0\\L_{21}& L_{22}\end{bmatrix}}

Step 3: To evaluate the diagonal element, use the formula:

L_vv = √(A_vv - ∑_u<v L_vu (L_vu)*)

Where L* is the conjugate of L.

Step 4: To evaluate the non-diagonal element, use the formula:

L_tv = (1 / L_vv)(A_tv - ∑_u<v L_tu (L_vu)*)

Where L* is the conjugate of L.

Step 5: Put all the elements in L and evaluate A = LL^*.

Example: Find the Cholesky decomposition for a matrix A = \bold{\begin {bmatrix}81 & -9i\\9i & 45\end{bmatrix}}

Solution:

A = \begin {bmatrix}81& -9i\\9i & 45\end{bmatrix}

According to Cholesky decomposition the matrix can be written as:

A = LL*

where L is the lower triangular matrix of A and L* is the conjugate transpose matrix of L.

First, we have to find the lower triangular matrix of A.

Let the lower triangular matrix for P be L = \begin {bmatrix}L_{11}&0\\L_{21}& L_{22}\end{bmatrix}

We calculate the elements of lower triangular matrix using the following formulas:

The diagonal element of L is calculated using:

L_vv = √(A_vv - ∑_u<v L_vu (L_vu)*)

The non-diagonal element is calculated using formula:

L_tv = (1 / L_vv)(A_tv - ∑_u<v L_tu (L_vu)*)

L₁₁ = √(A₁₁ - ∑_u<1 L_1u (L_1u)*) [Since, there is no value u which is lesser than 1(here, v=1) so, ∑_u<v L_vu (L_vu)*= 0 ]

⇒ L11 = √(81 - 0) = √81 = 9

L₂₁ = (1 / L₁₁)(A₂₁ )

⇒ L₂₁ = (1 / 9)(9i) = i

(L₂₁)* = -i (conjugate of i)

L₂₂ = L_tv = (1 / L_vv)(A_tv - ∑u<v L_tu (L_vu)*)

(∑_u<v L_tu (L_vu)* is the summation of the values of L where u<v)

⇒ L₂₂ = √(A₂₂ - ∑_u<2 L₂₁ (L₂₁)*) Here, v = 2 so u should be the values less than 2. By putting values in formula)

⇒ L₂₂ = √(A₂₂ - L₂₁ (L₂₁)*)

⇒ L₂₂ = √(45 - (-i) (i))

⇒ L₂₂ = √(45 + i²)

⇒ L₂₂ = √(45 - 1) [i² = -1]

⇒ L₂₂ = √44

The lower triangular matrix of P according to Cholesky decomposition is:

L = \begin {bmatrix}9&0\\i& \sqrt{44}\end{bmatrix}

Now, we will find L* i.e., conjugate transpose of L

L* = \begin {matrix}9&-i\\0& \sqrt{44}\end{matrix}

Finally, we apply

A = LL*

⇒ A = \begin {bmatrix}9&0\\i& \sqrt{44}\end{bmatrix}\begin {bmatrix}9&-i\\0& \sqrt{44}\end{bmatrix}

The final matrix A after applying Cholesky decomposition is:

A = \begin {bmatrix}81&-9i\\9i& 45\end{bmatrix} = \begin {bmatrix}9&0\\i& \sqrt{44}\end{bmatrix}\begin {bmatrix}9&-i\\0& \sqrt{44}\end{bmatrix}

Application of Cholesky Decomposition

Some applications of the Cholesky decomposition of the positive definite symmetric matrix are the following:

• It is used to solve the systems of linear equations.
• It can be used to compute the inverse of the matrix.
• Cholesky decomposition is widely used in Monte Carlo simulation.

Related Articles

• Matrices
• Types of Matrices
• Row Echelon Form
• Row Matrix
• Column Matrix

Solved Examples

Example 1: Find the Cholesky decomposition for a matrix X whose lower triangular matrix is given by L = \bigg[\begin {matrix}2&0\\2-5i& 1\end{matrix}\bigg]

Solution:

The lower triangular matrix is given:

L = \bigg[\begin {matrix}2&0\\2-5i& 1\end{matrix}\bigg]

The conjugate transpose of the above lower triangular matrix is:

L* = \bigg[\begin {matrix}2&2+5i\\0& 1\end{matrix}\bigg]

From Cholesky decomposition X can be written as:

X = L L*

⇒ X = \bigg[\begin {matrix}2&0\\2-5i& 1\end{matrix}\bigg] \bigg[\begin {matrix}2&2+5i\\0& 1\end{matrix}\bigg]

⇒ X = \bigg[\begin {matrix}4&4+10i\\4-10i& 29\end{matrix}\bigg]

Example 2: Find the lower triangular matrix using Cholesky decomposition for a matrix A = \bold{\bigg[\begin {matrix}36&12i\\-12i& 42\end{matrix}\bigg]}

Solution:

A = \bigg[\begin {matrix}36&12i\\-12i& 42\end{matrix}\bigg]

Let the lower triangular matrix for A be L = \bigg[\begin {matrix}L_{11}&0\\L_{21}& L_{22}\end{matrix}\bigg]

The diagonal element of L is calculated using:

L_vv = √(A_vv - ∑_u<v L_vu (L_vu)*)

The non-diagonal element is calculated using formula:

L_tv = (1 / L_vv)(A_tv - ∑_u<v L_tu (L_vu)*)

L₁₁ = √(A₁₁ - ∑_u<1 L_1u (L_1u)*) [∑_u<1 L_1u (L_1u)* = 0 as v = 1 is the least value and there is no u lesser than v]

⇒ L₁₁ = √(36 - 0)

⇒ L₁₁ = 6

L₂₁ = (1 / L₁₁)(A₂₁ - ∑_u<1 L_2u (L_1u)*)

[Here, there is no u which is lesser than 1 so, ∑_u<1 L_2u (L_1u)* = 0]

⇒ L₂₁ = (1 / L₁₁)(A₂₁ - 0)

⇒ L₂₁ = (1 / 6)(-12i) = -2i

Thus, (L₂₁)* = 2i

L₂₂ = √(A₂₂ - ∑_u<2 L₂₁ (L₂₁)*) [Here, only u = 1 is the value satisfying the condition u < 2. Putting the values in the formula]

⇒ L₂₂ = √(A₂₂ - L₂₁ (L₂₁)*)

⇒ L₂₂ = √(42 - (-2i) (2i))

⇒ L₂₂ = √(42 + 4i²)

⇒ L₂₂ = √(42 - 4) [i² = -1]

⇒ L₂₂ = √38

The lower triangular matrix of A according to Cholesky decomposition is:

L = \bigg[\begin {matrix}6&0\\-2i& \sqrt{38}\end{matrix}\bigg]

Example 3: Find the Cholesky decomposition for a matrix X whose lower triangular matrix is given by L = \begin {bmatrix}1&0&0\\2& 3i&0\\5& i &2i\end{bmatrix}

Solution:

The lower triangular matrix is given:

L =\begin {bmatrix}1&0&0\\2& 3i&0\\5& i &2i\end{bmatrix}

The conjugate transpose of the above lower triangular matrix is:

L* = \begin {bmatrix}1&2&5\\0& -3i&-i\\0& 0 &-2i\end{bmatrix}

From Cholesky decomposition X can be written as:

X = L L*

⇒ X =\begin {bmatrix}1&0&0\\2& 3i&0\\5& i &2i\end{bmatrix} \begin {bmatrix}1&2&5\\0& -3i&-i\\0& 0 &-2i\end{bmatrix}

⇒ X = \begin {bmatrix}1&2&5\\1& 13&13\\5& 13 &30\end{bmatrix}

Practice Problems

• Problem 1: Compute the Cholesky decomposition of the following matrix:

A =\begin{bmatrix} 4 &-2 & 4 \\ -2 & 6 &-2 \\ 4 &-2 & 9 \end{bmatrix}

• Problem 2: Perform Cholesky decomposition on the matrix A given below

A = \begin{bmatrix} 9 & -3& 3 \\ -3& 6 &-2 \\ 3& -2& 7 \end{bmatrix}

• Problem 3: Calculate the Cholesky Decomposition for the given positive definite matrix:

A = \begin{bmatrix} 4& 2& 2 \\ 2 & 5& 1 \\ 2& 1& 6 \end{bmatrix}