B.3 Matrix Identities

For square, invertible matrices $\mathbf{A}$ and $\mathbf{B}$ , the following are equivalent:

$\mathbf{B}{(\mathbf{A}+\mathbf{B})}^{-1}$			$\mathbf{I}-\mathbf{A}{(\mathbf{A}+\mathbf{B})}^{-1}$
${(\mathbf{I}+\mathbf{A}{\mathbf{B}}^{-1})}^{-1}$			$\mathbf{I}-{(\mathbf{I}+\mathbf{B}{\mathbf{A}}^{-1})}^{-1}$
${({\mathbf{A}}^{-1}+{\mathbf{B}}^{-1})}^{-1}{\mathbf{A}}^{-1}$			$\mathbf{I}-{({\mathbf{A}}^{-1}+{\mathbf{B}}^{-1})}^{-1}{\mathbf{B}}^{-1}$

The proofs are by construction:

\begin{split}\mathbf{B}=\mathopen{}\mathclose{{}\left(\mathbf{A}+\mathbf{B}}% \right)-\mathbf{A}&{}\implies\mathbf{B}\mathopen{}\mathclose{{}\left(\mathbf{A% }+\mathbf{B}}\right)^{-1}=\mathbf{I}-\mathbf{A}\mathopen{}\mathclose{{}\left(% \mathbf{A}+\mathbf{B}}\right)^{-1}\\ \mathbf{B}=\mathopen{}\mathclose{{}\left(\mathbf{A}+\mathbf{B}}\right)-\mathbf% {A}&{}\implies\mathbf{I}=\mathopen{}\mathclose{{}\left(\mathbf{A}+\mathbf{B}}% \right)\mathbf{B}^{-1}-\mathbf{A}\mathbf{B}^{-1}\\ &{}\implies\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{A}\mathbf{B}^{-1}}% \right)^{-1}=\mathbf{B}\mathopen{}\mathclose{{}\left(\mathbf{A}+\mathbf{B}}% \right)^{-1}\\ \mathbf{A}^{-1}=\mathopen{}\mathclose{{}\left(\mathbf{A}^{-1}+\mathbf{B}^{-1}}% \right)-\mathbf{B}^{-1}&{}\implies\mathopen{}\mathclose{{}\left(\mathbf{A}^{-1% }+\mathbf{B}^{-1}}\right)^{-1}\mathbf{A}^{-1}=\mathbf{I}-\mathopen{}\mathclose% {{}\left(\mathbf{A}^{-1}+\mathbf{B}^{-1}}\right)^{-1}\mathbf{B}^{-1}\\ \mathbf{A}^{-1}=\mathopen{}\mathclose{{}\left(\mathbf{A}^{-1}+\mathbf{B}^{-1}}% \right)-\mathbf{B}^{-1}&{}\implies\mathbf{B}\mathbf{A}^{-1}=\mathbf{B}% \mathopen{}\mathclose{{}\left(\mathbf{A}^{-1}+\mathbf{B}^{-1}}\right)-\mathbf{% I}\\ &{}\implies\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{B}\mathbf{A}^{-1}}% \right)^{-1}=\mathopen{}\mathclose{{}\left(\mathbf{A}^{-1}+\mathbf{B}^{-1}}% \right)^{-1}\mathbf{B}^{-1}\\ &{}\implies\mathbf{I}-\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{B}% \mathbf{A}^{-1}}\right)^{-1}=\mathbf{I}-\mathopen{}\mathclose{{}\left(\mathbf{% A}^{-1}+\mathbf{B}^{-1}}\right)^{-1}\mathbf{B}^{-1}\\ \mathbf{B}^{-1}=\mathopen{}\mathclose{{}\left(\mathbf{A}^{-1}+\mathbf{B}^{-1}}% \right)-\mathbf{A}^{-1}&{}\implies\mathbf{A}\mathbf{B}^{-1}=\mathbf{A}% \mathopen{}\mathclose{{}\left(\mathbf{A}^{-1}+\mathbf{B}^{-1}}\right)-\mathbf{% I}\\ &{}\implies\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{A}\mathbf{B}^{-1}}% \right)^{-1}=\mathopen{}\mathclose{{}\left(\mathbf{A}^{-1}+\mathbf{B}^{-1}}% \right)^{-1}\mathbf{A}^{-1}\\ \end{split}

The Woodbury inversion lemma.

For any “conformable” matrices $\mathbf{M}$ and $\mathbf{N}$ , it is clearly the case that

\begin{split}\mathbf{M}\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{N}% \mathbf{M}}\right)=\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{M}\mathbf{% N}}\right)\mathbf{M}\implies\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{M% }\mathbf{N}}\right)^{-1}\mathbf{M}=\mathbf{M}\mathopen{}\mathclose{{}\left(% \mathbf{I}+\mathbf{N}\mathbf{M}}\right)^{-1}.\end{split}

We now find an alternative expression for $\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{M}\mathbf{N}}\right)^{-1}$ using the above equation and the fact that $\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{M}\mathbf{N}}\right)$ is its inverse:

\begin{split}\mathbf{I}&{}=\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{M}% \mathbf{N}}\right)^{-1}\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{M}% \mathbf{N}}\right)=\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{M}\mathbf{% N}}\right)^{-1}+\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{M}\mathbf{N}}% \right)^{-1}\mathbf{M}\mathbf{N}\\ \implies\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{M}\mathbf{N}}\right)^% {-1}&{}=\mathbf{I}-\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{M}\mathbf{% N}}\right)^{-1}\mathbf{M}\mathbf{N}\\ &{}=\mathbf{I}-\mathbf{M}\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{N}% \mathbf{M}}\right)^{-1}\mathbf{N}.\end{split}

Finally, let $\mathbf{M}\stackrel{{\scriptstyle\text{set}}}{{=}}\mathbf{A}^{-1}\mathbf{U}$ and $\mathbf{N}\stackrel{{\scriptstyle\text{set}}}{{=}}\mathbf{C}\mathbf{V}$ in the above equation. Then

equation (B.17) (B.17)

\begin{split}\mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{A}^{-1}\mathbf{U% }\mathbf{C}\mathbf{V}}\right)^{-1}&{}=\mathbf{I}-\mathbf{A}^{-1}\mathbf{U}% \mathopen{}\mathclose{{}\left(\mathbf{I}+\mathbf{C}\mathbf{V}\mathbf{A}^{-1}% \mathbf{U}}\right)^{-1}\mathbf{C}\mathbf{V}\\ \implies\mathopen{}\mathclose{{}\left(\mathbf{A}+\mathbf{U}\mathbf{C}\mathbf{V% }}\right)^{-1}&{}=\mathbf{A}^{-1}-\mathbf{A}^{-1}\mathbf{U}\mathopen{}% \mathclose{{}\left(\mathbf{C}^{-1}+\mathbf{V}\mathbf{A}^{-1}\mathbf{U}}\right)% ^{-1}\mathbf{V}\mathbf{A}^{-1}.\end{split}

This final form is known as the Woodbury matrix-inversion lemma. Notice that we have assumed squareness and invertibility for $\mathbf{A}$ and $\mathbf{C}$ , but not $\mathbf{U}$ or $\mathbf{V}$ .

The special case in which $\mathbf{U}$ and $\mathbf{V}$ are vectors, $\bm{u}$ and $\bm{v}^{\text{T}}$ , and (without further loss of generality) $\mathbf{C}=1$ , is known as the Sherman-Morrison formula:

equation (B.18) (B.18)

\mathopen{}\mathclose{{}\left(\mathbf{A}+\bm{u}\bm{v}^{\text{T}}}\right)^{-1}=% \mathbf{A}^{-1}-\frac{\mathbf{A}^{-1}\bm{u}\bm{v}^{\text{T}}\mathbf{A}^{-1}}{1% +\bm{v}^{\text{T}}\mathbf{A}^{-1}\bm{u}}

Figure B.1: A taxonomy of matrices.