Ch05. Eigenvalues and Eigenvectors

5.3 Diagonalization

• In many cases, the eigenvalue–eigenvector information contained within a matrix $A$ can be displayed in a useful factorization of the form $A = PDP^{-1}$ where $D$ is a diagonal matrix.
• In this section, the factorization enables us to compute $A^k$ quickly for large values of $k$, a fundamental idea in several applications of linear algebra.
• Later, in Sections 5.6 and 5.7, the factorization will be used to analyze (and decouple) dynamical systems.

Example 2:

Let $A=\begin{bmatrix} 7 & 2 \\ -4 & 1 \end{bmatrix}$. Find a formula for $A^k$, given that $A=PDP^{-1}$, where $$P =\begin{bmatrix} 1 & 1 \\ -1 & -2 \end{bmatrix} \text{ and } D=\begin{bmatrix} 5 & 0 \\ 0 & 3 \end{bmatrix}$$

Solution:

The standard formula for the inverse of a $2 \times 2$ matrix yields $$P^{-1} = \begin{bmatrix} 2 & 1 \\ -1 & -1 \end{bmatrix}$$

• Then, by associativity of matrix multiplication, $$\begin{align} A^2 &= (PDP^{-1})(PDP^{-1}) = PD(P^{-1}P)DP = PDIDP^{-1} = PDDP^{-1} \\ &=PD^{2}P^{-1}= \begin{bmatrix} 1 & 1\\-1 & -2 \end{bmatrix} \begin{bmatrix} 5^2& 0\\ 0 & 3^2 \end{bmatrix} \begin{bmatrix} 2 & 1\\-1 & -1 \end{bmatrix} \end{align}$$
• Again, $$A^3 = (PDP^{-1})A^2 = (PDP^{-1})PD^2P^{-1}=PDD^2P^{-1}=PD^3P^{-1}$$
• In general, for $k \ge 1$, $$\begin{align} A^k &= PD^kP^{-1} = \begin{bmatrix} 1& 1 \\ -1 & -2 \end{bmatrix} \begin{bmatrix} 5^k& 0\\0 & 3^k \end{bmatrix} \begin{bmatrix} 2& 1\\-1 & -1 \end{bmatrix} \\ &= \begin{bmatrix} 2\cdot5^k-3^k & 5^k-3^k \\2 \cdot 3^k - 2 \cdot 5^k & 2 \cdot 3^k - 5^k \end{bmatrix} \\ & & \blacksquare \end{align}$$

A square matrix $A$ is said to be diagonalizable

• if $A$ is similar to a diagonal matrix, that is,
• if $A=PDP^{-1}$ for some invertible matrix $P$ and some diagonal matrix $D$.

• Example 2는 diagnoalization(혹은 factorization)의 효용성을 보여줌.
• Theorem 5는 diagonalizable matrix의 properties와 어떻게 factorization하는지를 알려줌.

Theorem 5: The Diagonalization Theorem

An $n \times n$ matrix $A$ is diagonalizable if and only if $A$ has $n$ linearly independent eigenvectors.

• In fact, $A = PDP^{-1}$ with $D$ a diagonal matrix, if and only if
  • the columns of $P$ are $n$ linearly independent eigenvectors of $A$.
  • In this case, the diagonal entries of $D$ are eigenvalues of $A$ that correspond, respectively, to the eigenvectors in $P$.
• In other words, $A$ is diagonalizable if and only if
  • there are enough eigenvectors to form a basis of $\mathbb{R}^n$.
  • We call such a basis an eigenvector basis of $\mathbb{R}^n$.

Proof:

First, observe that if $P$ is any $n \times n$ matrix with columns $\textbf{v}_1 \cdots \textbf{v}_n$ and if $D$ is any diagonal matrix with diagonal entries $\lambda_1, \cdots \lambda_n$, then $$AP = A\begin{bmatrix} \textbf{v}_1 & \textbf{v}_2 & \cdots & \textbf{v}_n \end{bmatrix} = \begin{bmatrix} A\textbf{v}_1 & A\textbf{v}_2 & \cdots & A\textbf{v}_n \end{bmatrix} \tag{1}$$ while $$PD=P \begin{bmatrix} \lambda_1 & 0 \cdots & 0 \\ 0 & \lambda_2 &\cdots & 0 \\ \vdots & \vdots & & \vdots \\ 0 & 0 & \cdots & \lambda_n \end{bmatrix} = \begin{bmatrix} \lambda_1\textbf{v}_1 & \lambda_2 \textbf{v}_2 \cdots \lambda_n \textbf{v}_n \end{bmatrix} \tag{2}$$

Now suppose $A$ is diagonalizable and $A=PDP^{-1}$. Then right-multiplying this relation by $P$, we have $AP = PD$

• In this case, equation (1) and (2) imply that $$\begin{bmatrix} A\textbf{v}_1 & A\textbf{v}_2 & \cdots & A \textbf{v}_n \end{bmatrix} = \begin{bmatrix} \lambda_1 \textbf{v}_1 & \lambda_2 \textbf{v}_2 \cdots \lambda_n \textbf{v}_n \end{bmatrix} \tag{3}$$
• Equating columns, we find that $$A\textbf{v}_1 = \lambda_1 \textbf{v}_1, A\textbf{v}_2 = \lambda_2 \textbf{v}_2, \cdots, A\textbf{v}_n = \lambda_n \textbf{v}_n \tag{4}$$
• Since $P$ is invertible, its columns $\textbf{v}_1, \cdots \textbf{v}_n$ must be linear independnet.
• Also, since these columns are nonzero, the equations in (4) show that $\lambda_1, \cdots \lambda_n$ are eigenvalues and $\textbf{v}_1, \cdots \textbf{v}_n$ are corresponding eigenvectors.
• This argument proves the “only if ” parts of the first and second statements, along with the third statement, of the theorem.
• Finally, given any $n$ eigenvectors $\textbf{v}_1, \cdots \textbf{v}_n$, use them to construct the columns of $P$ and use corresponding eigenvalues $\lambda_1, \cdots \lambda_n$ to construct $D$.
• By equation (1)-(3), $AP=PD$.
• This is true without any condition on the eigenvectors.
• If, in fact, the eigenvectors are linearly independent, then $P$ is invertible (by the Invertible Matrix Theorem), and $AP=PD$ implies that $A=PDP^{-1}$. $\blacksquare$

Diagonalizaing Matrices

Example 3:

Diagonlize the following matrix, if possible.

$$A = \begin{bmatrix} 1 & 3 & 3 \\ -3 & -5 & -3 \\ 3 & 3 & 1 \end{bmatrix}$$

That is, find an invetible matrix $P$ and a diagonal matrix $D$ such that $A=PDP^{-1}$.

Solution:

There are four steps to implement the description in Theorem 5.

• Step 1. Find the eigenvalues of $A$.
  • Here, the characteristic equation turns out to involve a cubic polynomial that can be factored: $$\begin{align} 0 = \text{det}(A-\lambda I) &= -\lambda^3 -3 \lambda^2 + 4 \\ &= - (\lambda-1)(\lambda+2)^2 \end{align}$$
  • The eigenvalues are $\lambda=1 \text { and } \lambda=-2$.
• Find three linearly independent eigenvectors of $A$.
  • Three vectors are needed because $A$ is a $3 \times 3$ matrix.
  • This is a critical step.
  • If it fails, then Theorem 5 says that $A$ cannot be diagonalized.
  • Basis for $\lambda=1: \textbf{v}_1=\begin{bmatrix}1 \\ -1 \\ 1 \end{bmatrix}$
  • Basis for $\lambda=-2: \textbf{v}_2=\begin{bmatrix}-1 \\ 1 \\ 0 \end{bmatrix} \text{ and } \textbf{v}_3=\begin{bmatrix}-1 \\ 0 \\ 1 \end{bmatrix}$
  • You can check that $\left\{\textbf{v}_1, \textbf{v}_2, \textbf{v}_3 \right\}$ is a linearly independent set.
• Step 3. Construct $P$ from the vectors in step 2.
  • The order of the vectors is unimportant.
  • Using the order chosen in step 2, form $$P = \begin{bmatrix} \textbf{v}_1 & \textbf{v}_2 & \textbf{v}_3 \end{bmatrix} = \begin{bmatrix} 1 & -1 & -1 \\ -1 & 1 & 0 \\ 1 & 0 & 1 \end{bmatrix}$$
• Step 4. Construct D from the corresponding eigenvalues.
  • In this step, it is essential that the order of the eigenvalues matches the order chosen for the columns of $P$.
  • Use the eigenvalue $\lambda=-2$ twice, once for each of the eigenvectors corresponding to $\lambda=-2$: $$D=\begin{bmatrix} 1 & 0 & 0 \\ 0 &-2 & 0 \\ 0 & 0 & -2 \end{bmatrix}$$
  • To avoid computing $P^{-1}$, simply verify that $AP=PD$.
  • Compute $$AP = \begin{bmatrix} 1 & 3 & 3 \\ -3 &-5 & -3 \\ 3 & 3 & 1 \end{bmatrix} \begin{bmatrix} 1 & -1 & -1 \\ -1 & 1 & 0 \\ 1 & 0 & 1 \end{bmatrix} = \begin{bmatrix} 1 & 2 & 2 \\ -1 & -2 & 0 \\ 1 & 0 & -2 \end{bmatrix} \\ PD = \begin{bmatrix} 1 & -1 & -1 \\ -1 & 1 & 0 \\ 1 & 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ 0 & -2 & 0 \\ 0 & 0 & -2 \end{bmatrix} = \begin{bmatrix} 1 & 2 & 2 \\ -1 & -2 & 0 \\ 1 & 0 & -2 \end{bmatrix}$$

Theorem 6:

An $n \times n$ matrix to have $n$ distinct eigenvalues is diagonalizable.

Proof:

Let $\textbf{v}_1, \cdots ,\textbf{v}_n$ be eigenvectors corresponding to the $n$ distinct eigenvalues of a matrix $A$.

• Then ${\textbf{v}_1, \cdots ,\textbf{v}_n}$ is linearly independent, by Theorem2 in Section 5.1.
• Hence $A$ is diagonalizable, by Theorem 5.

Matrices Whose Eigenvalues Are Not Distinct

• It is not necessary for an $n \times n$ matrix to have $n$ distinct eigenvalues in order to be diagonalizable.
• The $3\times 3$ matrix in Example 3 is diagonalizable even though it has only two distinct eigenvalues.
• If an $n \times n$ matrix $A$ has $n$ distinct eigenvalues, with corresponding eigenvectors $\textbf{v}_1, \cdots \textbf{v}_n$ and if $P = \begin{bmatrix} \textbf{v}_1& \cdots &\textbf{v}_n \end{bmatrix}$, then $P$ is automatically invertible because its columns are linearly independent, by Theorem 2.
• When $A$ is diagonalizable but has fewer than $n$ distinct eigenvalues, it is still possible to build $P$ in a way that makes $P$ automatically invertible, as the next theorem shows.

Theorem 7:

Let $A$ be an $n \times n$ matrix whose distinct eigenvalues are $\lambda_1, \cdots, \lambda_p$.

1. For $1 \le k \le p$, the dimension of the eigenspace for $\lambda_k$ is less than or equal to the multiplicity(중복도) of the eigenvalue $\lambda_k$.
2. The matrix $A$ is diagonalizable if and only if the sum of the dimensions of the eigenspaces equals $n$, and this happens if and only if (i) the characteristic polynomial factors completely into linear factors and (ii) the dimension of the eigenspace for each $\lambda_k$ equals the multiplicity of $\lambda_k$.
3. If $A$ is diagonalizable and $B_k$ is a basis for the eigenspace corresponding to $\lambda_k$ for each $k$, then the total collection of vectors in the sets $B_1, \cdots B_p$ forms an eigenvector basis for $$\mathbb{R}^n.