\section{The Inverse Function Theorem}
Let ${\cal S}$ be an open set in $\mb{\Bb R}^n$ and let $\b f:{\cal S} \to \mb{\Bb R}^n$ be a map, that is,
\[
\b f(\b x)=\left[\begin{array}{cccc}
f_1(\b x) & f_2(\b x) & \cdots & f_n(\b x)
\end{array}\right].
\]
If all partial derivatives of all functions $f_i$ exist and are continuous, then we say that $\b f$ is a ${\cal C}^1$ map. We define a ${\cal C}^p$ map in an analogous way.

Let $\b f:\mb{\Bb R}^n \to \mb{\Bb R}^n$ be a ${\cal C}^p$-map, $p>0$, on an open subset ${\cal S}$ of $\mb{\Bb R}^n$, and let $\b x^{(0)}$ be a point of ${\cal S}$. We say that $\b f$ is locally ${\cal C}^p$-invertible at $\b x^{(0)}$ if there is an open set ${\cal U} \subset {\cal S}$ containing $\b x^{(0)}$ and an open set ${\cal V}$ containing $\b f(\b x^{(0)})$ so that $\b f:{\cal U} \to {\cal V}$ has an inverse $\b h:{\cal V} \to {\cal U}$ of class ${\cal C}^p$ such that for all $\b y \in {\cal V}$
\[
\b f \circ \b h=\b h \circ \b f=\b I=\mb{identity}.
\]
We illustrate the above concepts in Figure~\ref{fig:inverse}.
\begin{figure}[h]
\vspace{3in}
\hskip1in\special{bmp:c:/mybook/inverse.bmp x=4in y=3in}
\caption{An illustartion of the inverse function theorem.}
\label{fig:inverse}
\end{figure}
\begin{thm}
Let $\b f:\mb{\Bb R}^n \to \mb{\Bb R}^n$ be a ${\cal C}^p$-map, $p>0$, on an open subset ${\cal S}$ of $\mb{\Bb R}^n$, and let $\b x^{(0)}$ be a point of ${\cal S}$ such that the Jacobian matrix $D\b f(\b x^{(0)}$ is invertible. Then,  $\b f$ is locally ${\cal C}^p$-invertible at $\b x^{(0)}$. 
\end{thm}
{\bf Proof} \quad To get a feeling about the ideas of the proof, we first consder the case of one variable, $y=f(x)$, where $f:\mb{\Bb R} \to \mb{\Bb R}$, of class ${\cal C}^1$ is defined on an open set ${\cal S}$ containing $x^{(0)}$. By assumption $f'(x^{(0)}) \ne 0$, and therefore $f'(x) \ne 0$ in some neighborhood ${\cal U}$ of $x^{(0)}$, where ${\cal U} \subset {\cal S}$. Let
\[
{\cal V}=f({\cal U}),
\]
and $x^{(1)}$ and $x^{(1)}$be any two points in ${\cal U}$. Then, by the mean value theorem
\[
f(x^{(2)})-f(x^{(2)})=f'(\xi)(x^{(2)}-x^{(1)}),
\]
for some $\xi \in {\cal U}$. Hence, $x^{(2)}-x^{(1)} \ne 0$ implies $f(x^{(2)})-f(x^{(2)}) \ne 0$ because $f'(\xi)\ne 0$. Thus, $f:{\cal U} \to {\cal V}$ is one-to-one, and an inverse function $h:{\cal V} \to {\cal U}$ exists near $x^{(0)}$. We represent $h$ as $x=h(y)$. We next rewrite the above equation as
\[
y^{(2)}-y^{(1)}=f'(\xi)(h(y^{(2)})-h(y^{(1)})),
\]
or equivalently as
\[
h(y^{(2)})-h(y^{(1)})=\frac{y^{(2)}-y^{(1)}}{f'(\xi)}.
\]
The above implies that $h$ is continuous in ${\cal V}$ because
\[
|h(y^{(2)})-h(y^{(1)})|\le L|y^{(2)}-y^{(1)}|,
\]
for some $L\ge 0$. The existence of a finite constant $L$ follows from the continuity of $f'(x)$ in ${\cal U}$. To show that $h$ is differentiable in ${\cal V}$, consider the expression
\begin{eqnarray*}
\lim_{y^{(2)} \to y^{(1)}}\frac{h(y^{(2)})-h(y^{(1)})}{y^{(2)}-y^{(1)}}&=& 
\lim_{y^{(2)} \to y^{(1)}}\frac{x^{(2)} - x^{(1)}}{f'(\xi)(x^{(2)}- x^{(1)})}\\
&=& \lim_{x^{(2)} \to x^{(1)}}\frac{1}{f'(\xi)}\\
&=& \frac{1}{f'(x^{(1)})}.
\end{eqnarray*}
Hence,
\[
\frac{dh}{dy}=\frac{1}{f'(h(y))}
\]
which is continuous in ${\cal V}$, and $h$ is of class ${\cal C}^1$.

Observe that $h$ is a solution of the differential equation
\[
\frac{dx}{dy}=\frac{1}{f'(x)},\quad h(y^{(0)})=x^{(0)}.
\]
Therefore if $f(x)$ is ${\cal C}^p$ so is $h(y)$.

We now extend the above proof to the case when $n>1$, that is, when $f:\mb{\Bb R}^n \to \mb{\Bb R}^n$. By the mean value theorem applied to the $i$th component $f_i:\mb{\Bb R}^n \to \mb{\Bb R}$ of $\b f$, if $f_i$ is differentiable on an open set containing the line segment joining two points $\b x^{(1)}$ and $\b x^{(2)}$ in $\mb{\Bb R}^n$, then there is a point $\b \xi^{(i)}$ on this line segment such that
\[
f_i(\b x^{(2)})-f_i(\b x^{(1)})=Df_i(\b \xi^{(i)})\left( \b x^{(2)}-\b x^{(1)}\right),
\]
where $Df_i=\left[\begin{array}{cccc}
\frac{\partial f_i}{\partial x_1}&\frac{\partial f_i}{\partial x_2}&\cdots&\frac{\partial f_i}{\partial x_n}
\end{array}\right]$. Assume that the subset ${\cal S}$ of $\mb{\Bb R}^n$ is convex. So for every two points $\b x^{(1)}$ and $\b x^{(2)}$ in ${\cal S}$ we have
\[
f_i(\b x^{(2)})-f_i(\b x^{(1)})=Df_i(\b \xi^{(i)})\left( \b x^{(2)}-\b x^{(1)}\right),\quad i=1,2,\ldots,n,
\]
where $\b \xi^{(i)} \in {\cal S}$. By assumption the Jacobian matrix of $\b f$ is invertible at $\b x^{(0)}$, that is,
\[
\det D\b f(\b x^{(0)})=\det \left[\begin{array}{c}
Df_1(\b x^{(0)})\\
Df_2(\b x^{(0)})\\
\vdots\\
Df_n(\b x^{(0)})
\end{array}\right] \ne 0.
\]
Since $D\b f$ is continuous, there is a subset ${\cal U}$ of ${\cal S}$ within which $\det D\b f(\b x) \ne 0$. Hence, for $\b x^{(1)}$ and $\b x^{(2)}$ in ${\cal U}$ it follows from the equations
\[
f_i(\b x^{(2)})-f_i(\b x^{(1)})=Df_i(\b \xi^{(i)})\left( \b x^{(2)}-\b x^{(1)}\right),\quad i=1,2,\ldots,n,
\]
that $\b f:{\cal U}^n \to {\cal V}^n$ is one-to-one. Indeed, the above equations can be represented in matrix form as
\[
\b f\left( \b x^{(2)}\right)-\b f\left( \b x^{(1)}\right)=\left[\begin{array}{c}
Df_1(\b \xi^{(1)})\\
Df_2(\b \xi^{(2)})\\
\vdots\\
Df_n(\b \xi^{(n)})
\end{array}\right]\left( \b x^{(2)}-\b x^{(1)}\right),
\]
where
\[
\det \left[\begin{array}{c}
Df_1(\b \xi^{(1)})\\
Df_2(\b \xi^{(2)})\\
\vdots\\
Df_n(\b \xi^{(n)})
\end{array}\right] \ne 0.
\]
Hence, given $\b x^{(1)}$ and $\b x^{(2)}$ in ${\cal U}$ such that $\b x^{(1)} \ne \b x^{(2)}$ then, $\b f\left(\b x^{(1)}\right) \ne \b f\left(\b x^{(2)}\right)$. Therefore, an inverse function $\b h:{\cal V} \to {\cal U}$ exists, and $x_i=h_i(\b y)$ for $\b y \in {\cal V}$. We also have
\[
\b h\left( \b y^{(2)}\right)-\b h\left( \b y^{(1)}\right)=\left[\begin{array}{c}
Df_1(\b \xi^{(1)})\\
Df_2(\b \xi^{(2)})\\
\vdots\\
Df_n(\b \xi^{(n)})
\end{array}\right]^{-1}\left( \b y^{(2)}-\b y^{(1)}\right).
\]
Therefore,
\[
\left\Arrowvert \b h\left( \b y^{(2)}\right)-\b h\left( \b y^{(1)}\right)\right\Arrowvert \le L \left\Arrowvert \b y^{(2)}-\b y^{(1)}\right\Arrowvert
\]
for some $L\ge 0$, which means that $\b h$ is continuous.

To prove that $\b h$ is ${\cal C}^1$ consider the following equations:
\[
y_i\left( \b x^{(2)}\right)-y_i\left( \b x^{(1)}\right)=Df_i(\b \xi^{(i)})\left( \b h\left(\b y^{(2)}\right)-\b h\left(\b y^{(1)}\right)\right),\quad i=1,2,\ldots,n.
\]
Let
\[
\left[\begin{array}{c}
Df_1(\b \xi^{(1)})\\
Df_2(\b \xi^{(2)})\\
\vdots\\
Df_n(\b \xi^{(n)})
\end{array}\right]^{-1}=\left[\begin{array}{c}
D^{-1}f_1(\b \xi^{(1)})\\
D^{-1}f_2(\b \xi^{(2)})\\
\vdots\\
D^{-1}f_n(\b \xi^{(n)})
\end{array}\right],
\]
then we can write
\[
h_i\left( \b y^{(2)}\right)-h_i\left( \b y^{(1)}\right)=D^{-1}f_i(\b \xi^{(i)})\left(\b y^{(2)} - \b y^{(1)} \right),\quad i=1,2,\ldots,n.
\]
Consider now the limit
\[
\lim_{\b y^{(2)} \to \b y^{(1)}}\frac{\left\arrowvert h_i\left( \b y^{(2)}\right)-h_i\left( \b y^{(1)}\right)-D^{-1}f_i(\b x^{(1)})\left(\b y^{(2)} - \b y^{(1)} \right)\right\arrowvert }{\left\Arrowvert \b y^{(2)} - \b y^{(1)} \right\Arrowvert },
\]
where $D^{-1}f_i(\b x^{(1)})$ is the inverse, that is, the $i$th row of $D^{-1}\b f\left(\b x^{(1)}\right)$. Then, we have
\[
\lim_{\b y^{(2)} \to \b y^{(1)}}\frac{\left\arrowvert \left(D^{-1}f_i(\b \xi^{(1)})-D^{-1}f_i(\b x^{(1)})\right)\left(\b y^{(2)} - \b y^{(1)} \right)\right\arrowvert }{\left\Arrowvert \b y^{(2)} - \b y^{(1)} \right\Arrowvert }=0
\]
because
\[
\lim_{\b x^{(2)} \to \b x^{(1)}}D^{-1}f_i(\b \xi^{(1)})=D^{-1}f_i(\b x^{(1)}).
\]
Thus,
\[
Dh_i(\b y)=D^{-1}f_i(\b h(\b y))
\]
which is continuous, and so $\b h$ is of class ${\cal C}^1$. If $\b f$ is of class ${\cal C}^p$, so is $\b h$.\\
\hspace*{5.5in} $\Box$

\section{The Implicit Function Theorem}
The implicit function theorem is closely related to the inverse function theorem. We illustrate the concept behind the implicit function theorem on a function on $\mb{\Bb R}^2$. Let ${\cal U}$ be an open set in $\mb{\Bb R}^2$, and let
\[
f:{\cal U} \to \mb{\Bb R}
\]
be a ${\cal C}^1$-function. Let $\left[\begin{array}{cc}
x_0 & y_0
\end{array}\right]^T$ be a point in ${\cal U}$, and let
\[
f(x_0,y_0)=0.
\]
Note that the right hand side of the above equation can be an arbitray constant. Without loss of generality we can assume that this constant is $0$, as in the above equation. Our goal will be to determine whether there is a differentiable function $y=\phi(x)$ defined near $x=x_0$ such that $\phi(x_0)=y_0$ and
\[
f(x,\phi(x))=0
\]
for all $x$ near $x_0$. If such a function $\phi$ exixts, then we say that $y=\phi(x)$ is the function determined implicitly by $f$.

As another illustrating example, consider a system of linear equations written as
\[
\left[\begin{array}{cc}
\b D & \b B
\end{array}\right]\left[\begin{array}{c}
\b x\\
\b y
\end{array}\right]-\b b=\b 0,
\]
where $\b D\in \mb{\Bb R}^{m \times n}$, $\b B\in \mb{\Bb R}^{m \times m}$, $\b x\in \mb{\Bb R}^n$, and $\b y \in \mb{\Bb R}^m$. A unique solution for $\b y$ exists if $\det \b B \ne 0$. We now generalize this result to a nonlinear case.
\begin{thm}
Let
\[
\b f:\mb{\Bb R}^{n+m} \to \mb{\Bb R}^m
\]
be a ${\cal C}^p$, $p \ge 1$, function, where the $i$th component of $\b f$ is 
\[
f_i(x_1,x_2,\ldots,x_n,y_1,y_2,\ldots,y_m),\quad i=1,2,\ldots,m.
\]
Let $\b x_0 \in \mb{\Bb R}^n$ and $\b y_0 \in \mb{\Bb R}^m$, and let $\b f\left( \b x_0,\b y_0\right) =\b 0$. Assume that at $\left( \b x_0,\b y_0\right)$
\[
\det \frac{\partial \b f}{\partial \b y} \ne 0.
\]
Then, there exists an open set ${\cal U} \subset \mb{\Bb R}^n$ containing $\b x_0$ and open set ${\cal V} \subset \mb{\Bb R}^m$ containing $\b y_0$ such that there exists a ${\cal C}^p$ function
\[
\b \phi:{\cal U} \to {\cal V}, \quad \b y=\b \phi(\b x),
\]
with $\b \phi(\b x_0)=\b y_0$ and $\b f(\b x,\b \phi (\b x))=\b 0$ for all $\b x$ in ${\cal U}$.
\end{thm} 
{\bf Proof} \quad Consider the following system of equations:
\begin{eqnarray*}
z_1 &=& x_1\\
z_2 &=& x_2\\
{}&\vdots&{}\\
z_n &=& x_n\\
z_{n+1} &=& f_1(\b x,\b y)\\
z_{n+2} &=& f_2(\b x,\b y)\\
{}&\vdots&{}\\
z_{n+m} &=& f_m(\b x,\b y),
\end{eqnarray*}
where
\begin{eqnarray*}
z_{10} &=& x_{10}\\
z_{20} &=& x_{20}\\
{}&\vdots&{}\\
z_{n0} &=& x_{n0}\\
z_{n+1,0} &=& f_1(\b x_0,\b y_0)=0\\
z_{n+2,0} &=& f_2(\b x_0,\b y_0)=0\\
{}&\vdots&{}\\
z_{n+m,0} &=& f_m(\b x_0,\b y_0)=0.
\end{eqnarray*}
The above system defines a mapping $\b F:\mb{\Bb R}^{n+m} \to \mb{\Bb R}^{n+m}$. We claim that $\b F$ is locally ${\cal C}^p$-invertible at $\left[\begin{array}{cc}
\b x^T_0,&\b y^T_0
\end{array}\right]^T \in \mb{\Bb R}^{n+m}$. We compute the Jacobian matrix of $\b F$. We have
\[
D\b F=\left[\begin{array}{cc}
\b I_n & \b 0\\
\frac{\partial \b f}{\partial \b x} & \frac{\partial \b f}{\partial \b y}
\end{array}\right].
\]
The determinant of the above Jacobian matrix evaluated at $\left(\b x_0,\b y_0\right)$ is nonzero because
\[
\det D\b F=\det \frac{\partial \b f}{\partial \b y}.
\]
Hence, the inverse mapping theorem applies, and we have the inverse system
\begin{eqnarray*}
x_1 &=& z_1\\
x_2 &=& z_2\\
{}&\vdots&{}\\
x_n &=& z_n\\
y_1 &=& h_1(z_1,z_2,\ldots,z_{n+m})\\
y_2 &=& h_2(z_1,z_2,\ldots,z_{n+m})\\
{}&\vdots&{}\\
y_m &=& h_m(z_1,z_2,\ldots,z_{n+m}).
\end{eqnarray*}
such that
\begin{eqnarray*}
z_{n+1} &=&  f_1(\b x,h_1(\b x,z_{n+1},\ldots,z_{n+m}),\ldots,h_m(\b x,z_{n+1},\ldots,z_{n+m}))\\
z_{n+2} &=& f_2(\b x,h_1(\b x,z_{n+1},\ldots,z_{n+m}),\ldots,h_m(\b x,z_{n+1},\ldots,z_{n+m}))\\
{}&\vdots&{}\\
z_{n+m} &=& f_m(\b x,h_1(\b x,z_{n+1},\ldots,z_{n+m}),\ldots,h_m(\b x,z_{n+1},\ldots,z_{n+m}))
\end{eqnarray*}
for all $z_{n+1},z_{n+2},\ldots,z_{n+m}$. In particular, for
\[
z_{n+1}=z_{n+2}=\ldots=z_{n+m}=0
\]
we have
\[
\b f\left( \b x,\b h(\b x,\b 0)\right)=\b 0,
\]
and
\[
\b h\left(\b x_0,\b 0\right)=\b y_0.
\]
So we write
\[
\b \phi(\b x)=\b h(\b x,\b 0).
\]
Note that so defined function $\b \phi$ is indeed the required function because
\[
\b y_0=\b h(\b x_0,\b 0)=\b \phi(\b x_0).
\]
This proves the implicit function theorem.\\
\hspace*{5.5in}$\Box$