Ch02. Matrix Algebra

The Inverse of a Matrix

• An $n \times n$ matrix $A$ is said to be invertible if there is an $n \times n$ matrix $C$ such that $$CA=I \text{ and } AC=I$$ where $I=I_n$, the $n \times n$ identity matrix.
• In this case, $C$ is an inverse of $A$.
• In fact, $C$ is uniquely determined by $A$, because if $B$ were another inverse of $A$, then $$B=BI=B(AC)=(BA)C=IC=C$$
• This unique inverse is denoted by $A^{-1}$, so that $$A^{-1}A=I \text{ and } AA^{-1}=I$$

Theorem 4

Let $A=\begin{bmatrix}a & b \\ c & d \end{bmatrix}$. If $ad-bc \ne 0$, then $A$ is invertible and

$$A^{-1}=\frac{1}{ad-bc}\begin{bmatrix}d &-b \\ -c & a\end{bmatrix}$$

If $ad-bc=0$, then $A$ is not invertible.

• The quantity $ad-bc$ is called the determinant of $A$, and we write $\textbf{det } A = ad-bc$.
• This theorem says that a $2\times 2$ matrix $A$ is invertible if and only if $\textbf{det } A \ne 0$.

Theorem 5

If $A$ is an invertible $n \times n$ matrix, then for each $b$ in $\mathbb{R}^n$, the equation $A\textbf{x}=\textbf{b}$ has the unique solution $\textbf{x}=A^{-1}\textbf{b}$.

Proof:

• Take any $\textbf{b} \in \mathbb{R}^n$.
• A solution exists because if $A^{-1}\textbf{b}$ is substituted for $\textbf{x}$, then $A\textbf{x}=A(A^{-1}\textbf{b})=(AA^{-1})\textbf{b}=I\textbf{b}=\textbf{b}$.
• So $A^{-1}\textbf{b}$ is a solution.
• To prove that the solution is unique, show that if $\textbf{u}$ is any solution, then $\textbf{u}$ must be $A^{-1}\textbf{b}$.
• If $A\textbf{u}=\textbf{b}$ we can multiply both sides by $\textbf{A}^{-1}$ and obtain $$\begin{aligned}A^{-1}A\textbf{u}&=A^{-1}\textbf{b}, \\ I\textbf{u}&=A^{-1}\textbf{b} \text{, and} \\ \textbf{u}&=A^{-1}\textbf{b} \end{aligned}$$ $\blacksquare$.

Theorem 6

1. If $A$ is an invertible matrix, then $A^{-1}$ is invertible and $$(A^{-1})^{-1}=A$$
2. If $A$ and $B$ are $n \times n$ invertible matrices, then so is $AB$ and the inverse of $AB$ is the product of the inverses of $A$ and $B$ in the reverse order. That is, $$(AB)^{-1}=B^{-1}A^{-1}$$
3. If $A$ is an invertible matrix, then so is $A^{T}$, and the inverse of $A^T$ is the transpose of $A^{-1}$. That is, $$(A^T)^{-1}=(A^{-1})^T$$

Proof:

To verify statement 1., find a matrix $C$ such that $$A^{-1}C=I \text{ and } CA^{-1}=I$$

• These equations are satisfied with $A$ in place of $C$. Hence $A^{-1}$ is invertible, and $A$ is its inverse.

Next, to prove statement 2., compute: $$(AB)(B^{-1}A^{-1})=A(BB^{-1})A^{-1}=AIA^{-1}=AA^{-1}=I$$

• A similar calculation shows that $(B^{-1}A^{-1})(AB)=I$.

For statement 3., use Theorem 3(4), read from right to left, $$(A^{-1})^TA^T = (AA^{-1})^T=I^T=I.$$

• Similarly, $A^T(A^{-1})^T=I^T=I$.
• Hence $A^{T}$ is invertible, and its inverse is $(A^{-1})^T$.

$\blacksquare$

The generalization of Theorem 6(2) is as follows:

The product of $n \times n$ invertible matrices is invertible, and the inverse is the product of their inverses in the reverse order.

• There is an important connection between invertible matrices and row operations that leads to a method for computing inverses.
• An invertible matrix $A$ is row equivalent to an identity matrix, and we can find $A^{-1}$ by watching the row reduction of $A$ to $I$.

Elementary Matrices

• An elementary matrix is one that is obtained by performing a single elementary row operation on an identity matrix.

Elementary Row Operations 에 대한 복습 반드시 필요함. elementary row operation

Example 5

Let $E_1 = \begin{bmatrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ -4 & 0 & 1 \end{bmatrix}$ , $E_2 = \begin{bmatrix}0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix}$, $E_3 = \begin{bmatrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 5 \end{bmatrix}$, $A = \begin{bmatrix}a & b & c \\ d & e & f \\ g & h & i \end{bmatrix}$

Compute $E_1A, E_2A$, and $E_3 A$, and describe how these products can be obtained by elementary row operations on $A$.

Solution:

Verify that

$$E_1A = \begin{bmatrix} a & b & c \\ d & e & f \\ g-4a & h-4b & i-4c \end{bmatrix}, E_2A = \begin{bmatrix} d & e & f \\ a & b & c \\ g & h & i \end{bmatrix}, E_3 A = \begin{bmatrix} a & b & c \\ d & e & f \\ 5g & 5h & 5i \end{bmatrix}$$

• Addition of -4 time row 1 of $A$ to row 3 produces $E_1A$ : Relplacement.
• An interchange of rows 1 and 2 of $A$ produces $E_2 A$, and
• multiplication of row 3 of $A$ by 5 produces $E_3A$ : Scaling.
• Left-multiplication (that is, multiplication on the left) by $E_1$ in Example 1 has the same effect on any $3 \times n$ matrix.

  매우 중요함. Elementary Matrix를 우리가 배우는 이유 중 하나임.

• Since $E_1 I = E_1$, we see that $E_1$ itself is produced by this same row operation on the identity.

  매우 쉽게 Elementary Matrix를 구할 수 있음. row operation을 product of matrix로 구현할 수 있음.

Example 5 illustrates the following general fact about elementary matrices.

• If an elementary row operation is performed on an $m \times n$ matrix $A$, the resulting matrix can be written as $EA$
  where the $m \times m$ matrix $E$ is created by performing the same row operation on $I_m$.
• Each elementary matrix $E$ is invertible. The inverse of $E$ is the elementary matrix of the same type that transforms $E$ back into $I$.

Theorem 7

An $n \times n$ matrix $A$ is invertible if and only if $A$ is row equivalent to $I_n$, and in this case, any sequence of elementary row operations that reduces $A$ to $I_n$ also transforms $I_n$ into $A^{-1}$.

Proof:

• Suppose that $A$ is invertible.
• Then, since the equation $A\textbf{x}=\textbf{b}$ has a solution for each $\textbf{b}$(Theorem 5), $A$ has a pivot position in every row.

• Because $A$ is square, the $n$ pivot positions must be on the diagonal, which implies that the reduced echelon form of $A$ is $I_n$. That is, $A \sim I_n$.

• Now suppose, conversely, that $A \sim I_n$.

• Then, since each step of the row reduction of $A$ corresponds to left-multiplication by an elementary matrix, there exist elementary matrices $E_1, \cdots, E_p$ such that $$A \sim E_1A \sim E_2(E_1A) \sim \cdots E_p(E_{p-1}\cdots E_1 A)=I_n$$ .

• That is, $$E_p \cdots E_1 A = I_n \tag{1}$$

• Since the product $E_p \cdots E_1$ of invertible matrices is invertible, (1) leads to $$\begin {aligned} (E_p \cdots E_1)^{-1}(E_p \cdots E_1)A &= (E_p \cdots E_1)^{-1} I_n \\ A&= (E_p \cdots E_1)^{-1} \end {aligned}$$

• Thus $A$ is invertible, as it is the inverse of an invertible matrix (Theorem 6). Also, $$A^{-1}= [ (E_p \cdots E_1)^{-1}]^{-1} = E_p \cdots E_1$$ .

• Then $A^{-1} = E_p \cdots E_1 I_n$, which says that $A^{-1}$ results from applying $E_p \cdots E_1$ successively to $I_n$.
• This is the same sequence in (1) that reduced $A$ to $I_n$. $\blacksquare$

Algorithm for Finding A to the Negative First Power

• Row reduce the augmented matrix $\begin{bmatrix}A & I\end{bmatrix}$.
• If $A$ is row equivalent to $I$, then $\begin{bmatrix}A & I\end{bmatrix}$ is row equivalent to $\begin{bmatrix}I & A^{-1}\end{bmatrix}$.
• Otherwise, $A$ does not have an inverse.

Example 2

Find the inverse of the matrix $A= \begin{bmatrix} 0 & 1 & 2 \\ 1 & 0 & 3 \\ 4 & -3 & 8 \end{bmatrix}$, if it exists.

Solution:

$$\begin{aligned} \begin{bmatrix}A & I\end{bmatrix} &= \begin{bmatrix} 0 & 1 & 2 & 1 & 0 & 0 \\ 1 & 0 & 3 & 0 & 1 & 0 \\ 4 & -3 & 8 & 0 & 0 & 1 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & 3 & 0 & 1 & 0 \\ 0 & 1 & 2 & 1 & 0 & 0 \\ 4 & -3 & 8 & 0 & 0 & 1 \end{bmatrix} \\ & \sim \begin{bmatrix} 1 & 0 & 3 & 0 & 1 & 0 \\ 0 & 1 & 2 & 1 & 0 & 0 \\ 0 & -3 & -4 & 0 & -4 & 1 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & 3 & 0 & 1 & 0 \\ 0 & 1 & 2 & 1 & 0 & 0 \\ 0 & 0 & 2 & 3 & -4 & 1 \end{bmatrix} \\ & \sim \begin{bmatrix} 1 & 0 & 3 & 0 & 1 & 0 \\ 0 & 1 & 2 & 1 & 0 & 0 \\ 0 & 0 & 1 & \frac{3}{2} & -2 & \frac{1}{2} \end{bmatrix} \\ & \sim \begin{bmatrix} 1 & 0 & 0 & -\frac{9}{2} & 7 & -\frac{3}{2} \\ 0 & 1 & 0 & -2 & 4 & -1 \\ 0 & 0 & 1 & \frac{3}{2} & -2 & \frac{1}{2} \end{bmatrix} \end{aligned}$$

• Theorem 7 shows, since $A \sim I$, that $A$ is invertible, and

  $\sim$ means that $A$ is row equivalent to $I_n$.

$$A^{-1} = \begin{bmatrix} -\frac{9}{2} & 7 & -\frac{3}{2} \\ -2 & 4 & -1 \\ \frac{3}{2} & -2 & \frac{1}{2} \end{bmatrix}$$

• Now, check the final answer.

$$AA^{-1}=\begin{bmatrix} 0 & 1 & 2 \\ 1 & 0 & 3 \\ 4 & -3 & 8 \end{bmatrix} \begin{bmatrix} -\frac{9}{2} & 7 & -\frac{3}{2} \\ -2 & 4 & -1 \\ \frac{3}{2} & -2 & \frac{1}{2} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}$$

• It is not necessary to check that $A^{-1}A=I$ since $A$ is invertible.

Another View of Matrix Inversion

• Denote the columns of $I_n$ by $\textbf{e}_1,\cdots,\textbf{e}_n$.
• Then row reduction of $\begin{bmatrix}A &I \end{bmatrix}$ to $\begin{bmatrix}I & A^{-1}\end{bmatrix}$ can be viewed as the simultaneous solution of the $n$ systems $A\textbf{x}=\textbf{e}_1,A\textbf{x}=\textbf{e}_2,\cdots,A\textbf{x}=\textbf{e}_n \tag{2}$ where the “augmented columns” of these systems have all been placed next to $A$ to form $\begin{bmatrix}A & \textbf{e}_1 &\textbf{e}_1 & \cdots &\textbf{e}_n \end{bmatrix}=\begin{bmatrix}A & I\end{bmatrix}$.
• The equation $AA^{-1}=I$ and the definition of matrix multiplication show that the columns of $A^{-1}$ are precisely the solutions of the systems in (2).