Singular Value Decomposition (SVD)

Background

  • For a matrix , we have that and are Symmetric Matrices.
  • If is a non-zero eigenvalue of then it is also a non-zero eigenvalue of .
  • Eigenvalues of and are non-negative.
  • , and .
  • , and

Definition

If then Orthogonal Matrices and such that:

The singular values of are where and are the corresponding non-zero eigenvalues of and .

The columns of matrix are composed of vectors these are called the left singular vectors. The columns of matrix are composed of vectors these are called the right singular vectors. The following relations are then used:

There are several other properties related to SVD:

Let be the largest singular value of then and where is the smallest singular value of . So the Condition Number of is .

We can write the multiplication for two matrices and as . This means we can express the SVD of from earlier as:

Note

The thin SVD of is where , , and .

Example

Go to Find SVD to see a worked example of finding the SVD of a matrix.