%\setcounter{chapter}{4}
\chapter{Stability}
\label{chapter:instab}
\section{Introduction}
In general, it may be difficult to obtain an explicit solution of nonlinear differential equations modeling a dynamical system. However, often we do not need to know the explicit solution of the modeling equations. Instead, we are interested in the behavior of the solution as time tends to infinity. Furthermore, in many models of physical systems a ``small'' change in the initial conditions results in a ``small'' change in the solution. Specifically, if a system is perturbed from its rest, or equilibrium position, then it starts to move. We can roughly say that the equilibrium position is stable if the system does not go far from this position for small initial perturbations. This concept may be illustrated, as in Willems~\cite{willems70}, Brogan~\cite{brogan91}, or in Barnett~\cite{barnett75}, by means of a ball resting in equilibrium on a sheet of metal bent into various shapes with cross-sections depicted in Figure~\ref{fig:eq-pts}. This figure illustrates four possible scenarios. If we neglect frictional forces, then a small perturbation from the equilibrium leads to:
\begin{itemize}
\item[(1)] oscillatory motion about equilibrium irrespective of the magnitude of the initial perturbation. In this case the equilibrium position is globally stable,
\item[(2)] the new position is also an equilibrium and the ball will remain in it. Here the equilibrium position is stable,
\item[(3)] the ball moving away without returning to the equilibrium. This equilibrium position is unstable,
\item[(4)] oscillatory motion about the equilibrium, unless the initial perturbation is so large that the ball is forced to oscillate about the new equilibrium position. This equilibrium position is locally stable.
\end{itemize}
If friction is taken into account, then the oscillatory motion steadily decreases until the ball returns to the equilibrium positions as in items~(1) and (4) above. We refer to such equilibrium positions as being asymptotically stable.
\begin{figure}
\vspace{3in}
\hskip1.4in\special{bmp:c:/mybook/eq-pts.bmp x=4in y=3in}
\caption{Illustration of stable and unstable equilibrium positions.}
\label{fig:eq-pts}
\end{figure}
In our discussion of stability concepts, we implicitly assume, as in item~(4), that there is some ``small'' region for allowed initial perturbations for which subsequent motions converge to the equilibrium position. If however, as in item~(1), any initial perturbation yields a motion that converges to the equilibrium position, then we have global asymptotic stability, which is also referred to as asymptotic stability in the large. 

The above informal discussion should serve as a preparation for an in-depth analysis of stability of dynamical systems. We begin our analysis by formally defining the equilibrium state. We then define stability, asymptotic stability, and other relevant notions.
\section{Basic Definitions of Stability}
We consider a class of dynamical systems modeled by the equation:
\[
\dot{\b x}(t)=\b f(t,\b x(t)), \;{\b x}(t_0)={\b x}_0,
\]
where ${\b x}(t)\in \mb{\Bb R}^n$ is the state vector and ${\b f}:\mb{\Bb R}\times \mb{\Bb R}^n \to \mb{\Bb R}$ is a vector-valued function with the components
\[
f_i(t,x_1,x_2,\ldots ,x_n):\mb{\Bb R}\times \mb{\Bb R}^n \to \mb{\Bb R},\:i=1,2,\ldots ,n.
\]
We assume that ${f_i}$'s are continuous and have continuous first partial derivatives. Under these assumptions, the solution to $\dot{\b x}(t)={\b f}(t,{\b x}(t)), \;{\b x}(t_0)={\b x}_0$, exists and is unique. We denote this solution by ${\b x}(t)={\b x}(t;t_0,{\b x}_0)$. If the  ${f_i}$'s  do not depend explicitly on $t$, then the system represented by  $\dot{\b x}(t)={\b f}(t,{\b x}(t))={\b f}({\b x}(t))$ is called {\em autonomous}, otherwise it is called {\em non-autonomous}. An {\em equilibrium point} (at time $t^*\in \mb{\Bb R}$) is a constant vector, say ${\b x}_e$, such that 
\[
{\b f}(t,{\b x}_e)={\b 0}\quad \mbox{for all}\quad t\ge t^*.
\]
Other terms for equilibrium point are {\em stationary point}, {\em critical point\/}, {\em rest position\/}, and {\em equilibrium state\/}. Observe that an equilibrium point ${\b x}_e$ is a constant solution to
\[
\dot{\b x}(t)={\b f}(t,{\b x}(t)), \quad{\b x}(t_0)={\b x}_e.
\]
If ${\b x}_e$ is an equilibrium point, we can arrange for this equilibrium point to be transferred to the origin of $\mb{\Bb R}^n$ by introducing the new variable
\[
\tilde{\b x}={\b x-\b x_e}.
\]
We assume that this has been done for an equilibrium point under consideration. Therefore, we have
\[
{\b f}(t,{\b 0})={\b 0} \quad \mbox{for all}\quad t.
\]
There may be several equilibrium points for a given system. We assume that there is no other equilibrium point in a neighborhood of the origin of $\mb{\Bb R}^n$. Thus, the origin is an isolated equilibrium point.

We now introduce basic definitions of stability in the sense of A. M. Lyapunov\footnote{Sometimes transliterated in the literature as Liapunov or Liapunoff.} (1857-1918). In the following considerations we use the standard Euclidean norm of a vector. Thus, if ${\b x} \in \mb{\Bb R}^n$ then
\[
\|{\b x}\|=\|{\b x}\|_2=({\b x}^T{\b x})^{\frac{1}{2}}.
\]
\begin{defn}
\label{def:stab}
An equilibrium state ${\b x}_e$ is said to be stable if for any given $t_0$ and any positive scalar $\varepsilon$, there exists a positive scalar $\delta = \delta(t_0,\varepsilon)$ such that if $\|{\b x}(t_0)-{\b x}_e\|<\delta$, then $\|{\b x}(t)-{\b x}_e\|<\varepsilon$  for all $t\ge t_0$.
\end{defn}
\begin{defn}
\label{def:astab}
An equilibrium state ${\b x}_e$ is said to be {\em attractive\/} if for any given $t_0$ there is a positive scalar ${\delta}_1={\delta}_1(t_0)$ such that if $\|{\b x}(t_0)-{\b x}_e\|<{\delta}_1$ then,
\[
\lim_{t \to \infty}{\b x}(t;t_0,{\b x}_0)={\b x}_e.
\]
\end{defn}
We can restate Definition~\ref{def:astab} as follows. An equilibrium state ${\b x}_e$ is attractive if for any $t_0$ there is a positive scalar ${\delta}_1={\delta}_1(t_0)$ such that if $\|{\b x}(t_0)-{\b x}_e\|<{\delta}_1$, then for any given ${\varepsilon}_1>0$ there exists a $T=T({\varepsilon}_1,t_0,{\b x}_0)$ such that $\|{\b x}(t;t_0,{\b x}_0)-{\b x}_e\|<{\varepsilon}_1$ for all $t>t_0+T$.
\begin{defn}
An equilibrium state ${\b x}_e$ is asymptotically stable if it is stable and attractive.
\end{defn}
\begin{defn}
An equilibrium state ${\b x}_e$ is said to be uniformly stable if the positive scalar $\delta={\delta}(t_0,\varepsilon)$ introduced in Definition~\ref{def:stab} can be taken independent of the initial time $t_0$.
\end{defn}
\begin{defn}
An equilibrium point ${\b x}_e$ is said to be uniformly attractive, if the positive scalars $\delta _1={\delta}_1(t_0)$ and $T=T({\varepsilon}_1,t_0,{\b x}_0)$  introduced in Definition~\ref{def:astab} are independent of $t_0$.
\end{defn}

The last two definitions allow us to introduce the notion of uniform asymptotic stability.
\begin{defn}
An equilibrium point ${\b x}_e$ is said to be uniformly asymptotically stable if it is uniformly stable and uniformly attractive.
\end{defn}
Notice that the solution ${\b x}(t)={\b x}_e$ must first be stable in order to qualify to be asymptotically stable. This is to prevent a system trajectory from straying arbitrarily far from the equilibrium state before converging toward it. Note also that the concepts of Lyapunov stability and asymptotic stability are local concepts. They apply to the system's behavior in the small or in a small neighborhood about the equilibrium point (see Hsu and Meyer~\cite{hsu68}).
\begin{defn}
An equilibrium point ${\b x}_e$ is said to be unstable if there is an $\varepsilon>0$ so that for any $\delta>0$ there exists ${\b x}(t_0)$ such that if $\|{\b x}(t_0)-{\b x}_e\|<\delta$, then $\|{\b x}(t_1)-{\b x}_e\| \ge \varepsilon$ for some $t_1>t_0$.
\end{defn}
A two dimensional geometric interpretation of the above definitions is shown in Figure~\ref{fig:2d-stab}.
\begin{figure}
\vspace{4.5in}
\special{bmp:c:/mybook/twostab.bmp x=6in y=4.5in}
\caption{Illustration of stability concepts in two dimensions; (a) stable equilibrium state in the sense of Lyapunov, (b) asymptotically stable equilibrium state, (c) unstable equilibrium state.}
\label{fig:2d-stab}
\end{figure}
We assume that $\b x_e=\b 0$ is the equilibrium state. If the equilibrium state is stable, as in Figure~\ref{fig:2d-stab}(a), then given an outer circle of radius $\varepsilon$, there exists an inner circle of radius $\delta$ such that trajectories starting within the $\delta$-circle never leave the $\varepsilon$-circle. If the equilibrium state is asymptotically stable, then trajectories in addition tend to the equilibrium state as $t$ tends to $\infty$. This situation is depicted in Figure~\ref{fig:2d-stab}(b). Finally, the equilibrium state is unstable, as in Figure~\ref{fig:2d-stab}(c), then there exists an $\varepsilon$-circle such that for every $\delta$-circle there exists a trajectory starting within it that leaves the $\varepsilon$-circle at some later time. The corresponding geometrical interpretation of the above described scenarios in one dimension is given in Figure~\ref{fig:1d_stab}.
\begin{figure}
\vspace{3in}
\hskip1in\special{bmp:c:/mybook/staba.bmp x=4in y=3in}
\begin{center}
(a) stable equilibrium state in the sense of Lyapunov
\end{center}
%\end{figure}
%\begin{figure}
\vspace{3in}
\hskip1in\special{bmp:c:/mybook/stabb.bmp x=4in y=3in}
\begin{center}
(b) asymptotically stable equilibrium state
\end{center}
\end{figure}
\begin{figure}
\vspace{3in}
\hskip1in\special{bmp:c:/mybook/stabc.bmp x=4in y=3in}
\begin{center}
(c) unstable equilibrium state
\end{center}
\caption{Illustration of stability concepts in one dimension.}
\label{fig:1d_stab}
\end{figure}
 
\section{Lyapunov Stability of Linear Systems}
We know that a linear time-invariant dynamical system modeled by the equation
\begin{equation}
\dot{\b x}={\b A\b x}, \quad {\b x}(t_0)={\b x}_0
\label{eq:simlinsysfls}
\end{equation}
is asymptotically stable if and only if, the solution to equation~(\ref{eq:simlinsysfls}) decays to zero as $t \to \infty$ for any initial state ${\b x}_0$. We can view the vector $\b x(t)\in \mb{\Bb R}^n$ as defining the coordinates of a point moving in an $n$-dimensional state space $\mb{\Bb R}^n$. In an asymptotically stable system modeled by equation~(\ref{eq:simlinsysfls}), this point converges to the origin of $\mb{\Bb R}^n$. We can argue that if a trajectory is converging to the origin of the state space, then it should be possible to find a family of nested surfaces described by $V(x_1,x_2, \ldots,x_n)=c$, $c\ge0$, such that monotonically decreasing values of $c$ give surfaces progressively shrinking in on the origin with the limiting surface $V(x_1,x_2, \ldots,x_n)=0$ being the origin $\b x=\b 0$. Furthermore, along every trajectory of~(\ref{eq:simlinsysfls}), the parameter $c$ should steadily decrease. We now introduce some notation and definitions that we use in the ensuing discussion.

Let $V({\b x})=V(x_1,x_2, \ldots,x_n):\mb{\Bb R}^n \to \mb{\Bb R}$ be a real-valued function and let ${\cal S} \subseteq \mb{\Bb R}^n$ be a compact region containing the origin ${\b x=\b 0}$ in its interior.
\begin{defn}
We say that the function $V=V({\b x})$ is {\em positive semi-definite} (p.s.d.) in ${\cal S}$ if
\begin{enumerate}
\item  $V$ is continuously differentiable, that is, $V \in {\cal C}^1$,
\item $V({\b 0})=0$,
\item $V({\b x}) \ge 0$ for all ${\b x} \in {\cal S}$.
\end{enumerate}
\end{defn}
\begin{defn}
We say that the function $V=V({\b x})$ is {\em positive definite} (p.d.) in ${\cal S}$ if
\begin{enumerate}
\item $V \in {\cal C}^1$,
\item $V({\b 0})=0$,
\item \label{it:stmin} $V({\b x}) > 0$ for all ${\b x} \in {\cal S} \setminus \{ {\b 0} \}$.
\end{enumerate}
\end{defn}

The symbol ${\cal S} \setminus \{ {\b 0} \}$ in item~\ref{it:stmin} above denotes the set $\{ {\b x} : {\b x} \in {\cal S}, {\b x \ne \b 0} \}$.

{\em Negative semi-definite} (n.s.d.) and {\em negative definite} (n.d.) functions are defined in a similar fashion by reversing inequality signs in item~\ref{it:stmin} in both definitions above.

There are many functions that describe a nested family of surfaces. In the stability analysis of linear systems one class is best suited. This is the positive definite quadratic form, a label attached to a nested family of ellipses and their extensions to higher dimensional spaces---ellipsoids for $n = 3$, and hyper-ellipsoids for $n \ge 4$. 

Let
\[
V({\b x})={\b x}^T{\b P\b x},
\]
where ${\b P}$ is a real symmetric $n \times n$ matrix.
The time derivative of $V({\b x}(t))$ evaluated on a solution of $\dot{\b x}={\b A\b x}$ is
\begin{eqnarray*}
\frac{dV({\b x}(t))}{dt} &=& \dot{V} \\
&=& \dot{\b x}^T \b P \b x + \b x^T{\b P} \dot{\b x}.
\end{eqnarray*}

Substituting $\dot{\b x}={\b A\b x}$ into the above equation yields
\begin{eqnarray*}
\dot{V} &=& {\b x}^T{\b A}^T{\b P\b x}+{\b x}^T{\b P\b A\b x} \\
&=&{\b x}^T({\b A}^T{\b P}+{\b P\b A}){\b x}.
\end{eqnarray*}
Let 
\[
{\b Q}=-({\b A}^T{\b P+\b P\b A}).
\]
Then,
\[
\dot{V}=-{\b x}^T{\b Q\b x}.
\]

We are now ready to state and prove the following theorem concerning the stability of the system $\dot{\b x}={\b A\b x}$, or equivalently, the matrix ${\b A} \in \mb{\Bb R}^n$.
\begin{thm}[A. M. Lyapunov, 1892]
\label{Lyapunov}
The system $\dot{\b x}={\b A\b x}$, ${\b x}(t_0)={\b x}_0$,  is asymptotically stable if and only if for any given real symmetric positive definite (r.s.p.d.) matrix ${\b Q}$ the solution ${\b P}$ to the continuous Lyapunov matrix equation,
\[
{\b A}^T{\b P+\b P\b A=-\b Q}
\]
is also (real symmetric) positive definite.
\end{thm}
{\bf Proof} \quad($\Rightarrow$)\quad We prove necessity, for asymptotic stability, by contradiction. We have $-{\b x}^T{\b Q\b x}<0$ for all ${\b x \neq \b 0}$. We assume that ${\b A}$ is an asymptotically stable matrix and that for some nonzero ${\b x}_0$ we have ${\b x}_0^T{\b P \b x}_0 \le 0$. From the discussion before the Theorem, it follows that
\[
\frac{d}{dt}({\b x}^T(t){\b P\b x}(t))=-{\b x}^T(t){\b Q\b x}(t) < 0.
\]
The above means that $V({\b x}(t))={\b x}^T(t){\b P\b x}(t)$ decreases on the trajectories of the system $\dot{\b x}={\b A\b x}$ as $t \to \infty$. On the other hand, since ${\b A}$ is asymptotically stable, we have
\[
\lim_{t \rightarrow \infty}{\b x}(t)={\b 0}.
\]
Hence
\begin{eqnarray*}
\lim_{t \to \infty}{\b x}^T(t){\b P\b x}(t)&=&\lim_{{\b x}(t) \to {\b 0}}{\b x}^T(t){\b P\b x}(t) \\
&=&0.
\end{eqnarray*}
Thus, we arrived at a contradiction. This completes the proof of the first part of the Theorem that states that asymptotic stability of ${\b A}$ implies that for any r.s.p.d matrix ${\b Q}$ the solution ${\b P}$ to the equation ${\b A}^T{\b P+\b P\b A=-\b Q}$ is r.s.p.d..

($\Leftarrow$)\quad We prove sufficiency, also by contradiction. We assume that
\[
\b x^T\b P\b x>0\quad\mb{and}\quad -\b x^T\b Q\b x<0\quad \mb{for all}\quad \b x \ne \b 0,
\]
and that the matrix ${\b A}$ is not asymptotically stable. This means that the solution to $\dot{\b x}(t)={\b A\b x}(t)$, ${\b x}_0 \ne {\b 0}$, which has the form ${\b x}(t)=e^{{\b A}(t-t_0)}{\b x}_0$, is bounded away from the origin ${\b x=\b 0}$. Thus, there is a constant $b > 0$ such that
\[
{\b x}^T(t){\b Q\b x}(t)>b>0 \quad \mbox{for all }t>t_0.
\] 
Integrating $\frac{d}{dt}({\b x}^T(t){\b P\b x}(t))=-{\b x}^T(t){\b Q\b x}(t)$, we obtain
\[
{\b x}^T(t){\b P\b x}(t)={\b x}^T(t_0){\b P\b x}(t_0)+\int_{t_0}^{t}(-{\b x}^T(s){\b Q\b x}(s))ds.
\]
Taking into account that $-{\b x}^T(t){\b Q\b x}(t)<-b<0 \quad \mbox{for all }t>t_0$, we get for $t>t_0$,
\begin{eqnarray*}
{\b x}^T(t){\b P\b x}(t)&=&{\b x}^T(t_0){\b P\b x}(t_0)+\int_{t_0}^{t}(-{\b x}^T(s){\b Q\b x}(s))ds \\
&<&{\b x}^T(t_0){\b P\b x}(t_0)-b(t-t_0).
\end{eqnarray*}
The above implies that for a sufficiently large $t$ we have ${\b x}^T(t){\b P\b x}(t)<0$, which contradicts the assumption that ${\b x}^T(t){\b P\b x}(t)>0$. This means that ${\b A}$ must be asymptotically stable, and the proof is complete.\\
\hspace*{5.5in} $\Box$
\begin{example} \rm
Let
\[
{\b A}=\left[\begin{array}{cc}
0 & 1\\
-1 & -2
\end{array}\right],
\]
and $\b Q=\b I_2$. We solve the Lyapunov equation $\b A^T\b P+\b P\b A=-\b Q$ for $\b P$. Assume $\b P$ to have the form,
\[
{\b P}=\left[ \begin{array}{cc}
p_1 & p_2\\
p_2 & p_3
\end{array}
\right]= \b P^T.
\]
Then, we have
\begin{eqnarray*}
\b A^T\b P+\b P\b A &=&\left[ \begin{array}{cc}
0 & -1\\
1 & -2
\end{array}
\right]\left[ \begin{array}{cc}
p_1 & p_2\\
p_2 & p_3
\end{array}
\right]+\left[ \begin{array}{cc}
p_1 & p_2\\
p_2 & p_3
\end{array}
\right]\left[ \begin{array}{cc}
0 & 1\\
-1 & -2
\end{array}
\right]\\
&=& \left[ \begin{array}{cc}
-2p_2 & -p_3+p_1-2p_2\\
-p_3+p_1-2p_2 & 2p_2-4p_3
\end{array}
\right]\\
&=&\left[ \begin{array}{cc}
-1 & 0\\
0 & -1
\end{array}
\right]. 
\end{eqnarray*}
Solving the corresponding linear equations yields
\[
\b P=\left[ \begin{array}{cc}
3/2 & 1/2\\
1/2 & 1/2
\end{array}
\right].
\]
The solution matrix $\b P$ is positive definite. Hence, the matrix $\b A$ must have its eigenvalues in the open left-half plane. Indeed, the eigenvalues of $\b A$ are located at $-1$.
\end{example}\markex

\begin{defn}
A positive definite function $V({\b x})$ whose time derivative evaluated on the solutions of the system $\dot{\b x}={\b A\b x}$ is negative definite (or negative semi-definite) is called a Lyapunov function for this system.
\end{defn}

Lyapunov's theorem gives us a necessary and sufficient condition for asymptotic stability of linear time-invariant dynamical systems. To check if a given matrix ${\b A}$ is asymptotically stable or not, it is enough to take an arbitrary r.s.p.d. matrix ${\b Q}$, solve the equation ${\b A}^T{\b P+\b P\b A=-\b Q}$ for real symmetric ${\b P}$ and check if the obtained ${\b P}$ is positive definite, or not. If ${\b P} > 0$, then ${\b A}$ is asymptotically stable. If ${\b P}$ is not positive definite, then ${\b A}$ cannot be asymptotically stable. Since ${\b Q}$ can be an arbitrary r.s.p.d. matrix, ${\b Q = \b I}_n$, the $n \times n$ identity matrix, is a good choice. It is important to note that there would be no use, as the following example shows, in first choosing a matrix ${\b P}$ and then calculating ${\b Q}$ from the equation ${\b A}^T{\b P+\b P\b A=-\b Q}$.
\begin{example} \rm
Let
\[
{\b A}=\left[ \begin{array}{cc}
-1 & 3\\
0 & -1
\end{array}
\right],
\]
which is asymptotically stable.
Let
\[
{\b P}=\left[ \begin{array}{cc}
1 & 0 \\
0 & 1
\end{array}
\right]
={\b I}_2 > 0.
\]
Then,
\[
{\b Q}=-({\b A}^T{\b P+\b P\b A})=\left[ \begin{array}{cc}
2 & -3 \\
-3 & 2
\end{array}
\right]
\]
which is indefinite. Thus, unless ${\b Q}$ turns out to be definite, nothing can be inferred about asymptotic stability from the Lyapunov theorem if we start with ${\b P}$ and then calculate ${\b Q}$.
\end{example}\markex

We now give a geometric interpretation of the Lyapunov theorem. First, note that the function
\[
\dot{V}=\frac{dV({\b x}(t))}{dt}
\]
can be expressed, using the chain rule, as
\[
\dot{V}=\nabla^T V \dot{\b x}=\| \nabla V \|\, \| \dot{\b x} \| \cos \theta,
\]
where $\theta$ is the angle between the vectors $\nabla V$ and $\dot{\b x}$. If $\| \dot{\b x} \|=1$, then $\nabla^T V \dot{\b x}$ can be viewed as the rate of increase of $V$ at $\b x$ in the direction $\dot{\b x}$. Thus, $\dot{V}<0$ if and only if, $90 \deg < \theta < 270 \deg$. In such a case the representative point moves in the direction of decreasing $V$. We illustrate the above discussion in Figure~\ref{fig:geolyap}.
\begin{figure}
\vspace{3.5in}
\hskip.8in\special{bmp:c:/mybook/geolyap.bmp x=4.66in y=3.5in}
\caption{Geometric interpretation of the Lyapunov theorem.}
\label{fig:geolyap}
\end{figure}

We now prove a variant of Theorem~\ref{Lyapunov}, which is very useful in numerical verification of stability.
\begin{thm}
\label{thm:ver_lyap}
A matrix $\b A$ is asymptotically stable if and only if for any ${\b Q=\b C^T\b C}$, $\b C\in \mb{\Bb R}^{p\times n}$, such that the pair $(\b A, \b C)$ is observable, the solution ${\b P}$ to the Lyapunov matrix equation ${\b A}^T\b P+\b P\b A=-\b C^T\b C$ is also positive definite.
\end{thm} 
{\bf Proof}\quad We first prove the necessity condition $(\Rightarrow )$ for asymptotic stability. If $\b A$ is asymptotically stable, that is, $\Re\lambda_i(\b A) <0$, then we will show that
\begin{equation}
\b P=\int_0^{\infty}e^{\b A^Tt}\b C^T\b Ce^{\b At}dt
\label{eq:integsotl}
\end{equation}
is the symmetric positive definite solution to the Lyapunov equation. Indeed, the integrand of~(\ref{eq:integsotl}) is a sum of terms of the form
\[
t^ke^{\lambda_it},
\]
where $\lambda_i$ is the $i$-th eigenvalue of $\b A$. Since $\Re\lambda_i(\b A)<0$, the integral in~(\ref{eq:integsotl}) exists because the terms in $\b P$ do not diverge when $t=\infty$. Indeed, let $-\alpha_i=\Re\lambda_i$, where $\alpha_i>0$, then
\[
\lim_{t\to \infty}t^ke^{-\alpha_i t}=\lim_{t \to \infty}\frac{t^k}{e^{\alpha_it}},
\]
We differentiate the numerator and denominator of the above $k$ times and use L'Hospital's rule to get
\begin{eqnarray}
\lim_{t \to \infty}\frac{t^k}{e^{\alpha_it}} &=& \lim_{t\to \infty}\frac{kt^{k-1}}{\alpha_ie^{\alpha_i t}}\nonumber\\
&=& \lim_{t\to \infty}\frac{k(k-1)t^{k-1}}{\alpha^2_ie^{\alpha_i t}}\nonumber\\
&\vdots& {}\nonumber\\
&=& \lim_{t\to \infty}\frac{k!}{\alpha^k_i e^t}\nonumber\\
&=&0.
\label{eq:lhospitwne}
\end{eqnarray}
Thus, by~(\ref{eq:lhospitwne}), the formula for $\b P$ given by~(\ref{eq:integsotl}) is well defined. Next, note that $\b P=\b P^T$. The matrix $\b P$ is also positive definite since the pair $(\b A,\b C)$ is observable. Substituting the expression for $\b P$ into the Lyapunov equation and performing manipulations yields
\begin{eqnarray*}
\b A^T\b P+\b P\b A &=&\int_0^{\infty}\b A^Te^{\b A^Tt}\b C^T\b Ce^{\b At}dt +\int_0^{\infty}e^{\b A^Tt}\b C^T\b Ce^{\b At}\b A dt\\
&=& \int_0^{\infty}\frac{d}{dt}\left(e^{\b A^Tt}\b C^T\b Ce^{\b At}\right)dt\\
&=& \left.e^{\b A^Tt}\b C^T\b Ce^{\b At}\right\arrowvert^{\infty}_0.\\
&=& -\b C^T\b C.
\end{eqnarray*}

The sufficiency part $(\Leftarrow )$ can be proven using the arguments we used while proving the sufficiency part of the Lyapunov Theorem and noting that the observability of the pair $(\b A,\b C)$ implies that $\b x^T(t)\b C^T\b C\b x(t)$ is not identically zero along any nonzero solution of $\dot{\b x}(t)=\b A\b x(t)$.\\
\hspace*{5.5in}$\Box$

We now show that if $\b A$ is asymptotically stable, then the Lyapunov equation
\[
\b A^T\b P+\b P\b A=-\b Q
\]
has a unique solution for every $\b Q$. Uniqueness can be proven in many ways. We use the Kronecker product in our proof. Using the results of Section~\ref{sec:kroneckerpro}, we represent the Lyapunov equation, $\b A^T\b P+\b P\b A=-\b Q$, as
\[
\left(\b I_n\otimes \b A^T+\b A^T\otimes \b I_n \right)s(\b P)=-s(\b Q).
\]
The condition for a solution $s(\b P)$ of the above equation to be unique is
\[
\lambda_i(\b A^T)+\lambda_j(\b A) \ne 0,\quad i,j=1,2,\ldots,n.
\]
Since $\lambda_i(\b A^T)=\lambda_i(\b A)$, and $\b A$ is asymptotically stable by assumption, the condition $\lambda_i(\b A^T)+\lambda_j(\b A) \ne 0$ is met, and hence the Lyapunov equation has a unique solution.


\section{The Hurwitz and Routh Stability Criteria}
\label{sec:routhhurwitz}
In this section, we use the Lyapunov theorem to derive two tests for determining if the zeros of an $n$-th degree polynomial are located in the open left-half complex plane. Consider the $n$-th degree polynomial
\begin{equation}
p(s)=a_ns^n+a_{n-1}s^{n-1}+\cdots+a_1s+a_0
\label{eq:polyhurrou}
\end{equation}
with real coefficients. Associated with the polynomial $p(s)$ is the $n\times n$ real Hurwitz matrix
\begin{equation}
\b H=\left[\begin{array}{ccccc}
a_{n-1} & a_{n-3} & a_{n-5} & a_{n-7} & \cdots\\
a_n & a_{n-2} & a_{n-4} & a_{n-6} & \cdots\\
0 & a_{n-1}& a_{n-3} & a_{n-5} & \cdots\\
0 & a_{n} & a_{n-2} & a_{n-4} & \cdots\\
\vdots&{}&{}&\vdots&\ddots
\end{array}\right].
\label{eq:hurmatnbn}
\end{equation}
In our further discussion we assume, without loss of generality, that $a_n=1$. The leading principal minors of the Hurwitz matrix are
\begin{equation}
H_1 = a_{n-1},\; H_2=\det \left[\begin{array}{cc}
a_{n-1} & a_{n-3}\\
1 & a_{n-2}
\end{array}\right],\; H_3=\det \left[\begin{array}{ccc}
a_{n-1} & a_{n-3}& a_{n-5}\\
1 & a_{n-2}& a_{n-4}\\
0 & a_{n-1} & a_{n-3}
\end{array}\right],\ldots ,H_n=\det \b H.
\label{eq:hurleaprmi}
\end{equation}
We next define
\[
b_1=H_1,\quad b_2=\frac{H_2}{H_1},\quad b_3=\frac{H_3}{H_2H_1},
\]
and
\[
b_k=\frac{H_{k-3}H_k}{H_{k-2}H_{k-1}}\quad \mb{for}\quad k=4,5,\ldots,n.
\]
Thus, for example, $b_4=\frac{H_1H_4}{H_2H_3}$. Let
\[
\b B=\left[\begin{array}{cccccccc}
0 & 1 & 0 &0&\cdots& 0 & 0&0\\
-b_n & 0 & 1 &0& \cdots& 0 & 0&0\\
0 & -b_{n-1}& 0 & 1 & \cdots & 0 & 0&0\\
\vdots& {} & {}& {}&{}&{}&{}&\vdots\\
0&0&0&0&\cdots&-b_3&0&1\\
0&0&0&0&\cdots&0&-b_2&-b_1
\end{array}\right]\in \mb{\Bb R}^{n \times n}.
\]
For example, for $n=4$, we have
\[
\b B=\left[\begin{array}{cccc}
0 & 1 & 0 & 0\\
-b_4 & 0 & 1 & 0\\
0 & -b_3 & 0 & 1\\
0 & 0 & -b_2 & -b_1
\end{array}\right].
\]
Let
\[
\b A=\left[\begin{array}{cccccc}
0 & 1 & 0 &\cdots & 0 & 0\\
0 & 0 & 1& \cdots & 0 & 0\\
\vdots& {}& {}& {}& {}& \vdots\\
0 & 0 & 0 & \cdots & 0 & 1\\
-a_0& -a_1 & -a_2&\cdots&-a_{n-2}&-a_{n-1}
\end{array}\right]\in \mb{\Bb R}^{n \times n}.
\]
\begin{lemma}
The matrices $\b B$ and $\b A$ are similar, that is, there is a nonsingular matrix $\b T\in \mb{\Bb R}^{n \times n}$ such that
\begin{equation}
\b B=\b T^{-1}\b A\b T.
\label{eq:simtrhur}
\end{equation}
\end{lemma}
{\bf Proof}\quad The similarity matrix is lower triangular and has the form
\begin{equation}
\b T=\left[\begin{array}{cccccccc}
1 & 0 & 0 & 0&0&0&\cdots & 0\\
0 & 1 & 0 & 0 & 0&0&\cdots & 0\\
t_{31}& 0 & 1&0&0&0&\cdots&0\\
0&t_{42}&0&1&0&0&\cdots&0\\
t_{51}&0&t_{53}&0&1&0&\cdots&0\\
0&t_{62}&0&t_{64}&0&1&\cdots&0\\
\vdots&{}&{}&{}&{}&{}&{}&\vdots
\end{array}\right].
\label{eq:tramaelo}
\end{equation}
To determine the nonfixed coefficients $t_{ij}$ of the matrix $\b T$, we represent~(\ref{eq:simtrhur}) as
\begin{equation}
\b T\b B=\b A\b T,
\label{eq:tbeatmeq}
\end{equation}
and compare both sides of~(\ref{eq:tbeatmeq}) to determine the coefficients $t_{ij}$. We obtain, for example, $t_{31}=-b_n$, $t_{42}=-b_n-b_{n-1}$, etc.\\
\hspace*{5.5in} $\Box$

Let $\b P$ be an $n\times n$ diagonal matrix defined as
\begin{equation}
\b P=\mb{diag}\left[\begin{array}{ccccc}
b_1b_2\cdots b_n, & b_1b_2\cdots b_{n-1}, & \cdots,  & b_1b_2, & b_1
\end{array}\right],
\label{eq:mapfohuthm}
\end{equation}
and $\b Q$ be an $n\times n$ diagonal matrix defined as
\[
\b Q=\mb{diag}\left[\begin{array}{ccccc}
0 & 0 & \cdots & 0 & 2 b_1^2
\end{array}\right].
\]
We represent the matrix $\b Q$  as $\b Q=\b c^T\b c$, where
\[
\b c=\left[\begin{array}{ccccc}
0 & 0 & \cdots & 0 & \sqrt{2} b_1
\end{array}\right].
\]
Note that the pair $\left( \b B,\b c\right)$ is observable. The matrices $\b B$, $\b P$, and $\b Q$ satisfy the Lyapunov matrix equation,
\begin{equation}
\b B^T\b P+\b P\b B=-\b Q.
\label{eq:lyapwbpqhu}
\end{equation}
By Theorem~\ref{thm:ver_lyap}, on page~\pageref{thm:ver_lyap}, applied to~(\ref{eq:lyapwbpqhu}), the matrix $\b B$ is asymptotically stable if and only if the symmetric matrix $\b P$ is positive definite. By Theorem~\ref{thm:psosdeftests}, the matrix $\b P$ is positive definite if and only if its leading principal minors are positive. Using the above results we can prove the following theorem:
\begin{thm}[Hurwitz]
The real polynomial $p(s)$ of degree $n$ with positive coefficients $a_i$, has its zeros in the open left-half plane if and only if the leading principal minors of the Hurwitz matrix are positive.
\end{thm}
{\bf Proof}\quad The polynomial $p(s)$ has its zeros in the open left plane if and only if the matrix $\b B$ is asymptotically stable. This in turn is true if and only if the diagonal elements of $\b P$, given by~(\ref{eq:mapfohuthm}), are positive. We have
\[
b_1=H_1,\quad b_1b_2=H_2,
\]
and for $k=3,\ldots,n$,
\[
b_1b_2\cdots b_k=\frac{H_k}{H_{k-2}}.
\]
Thus, the diagonal elements of $\b P$ are all positive, if and only if the leading principal minors of the Hurwitz matrix are positive, which completes the proof.\\
\hspace*{5.5in} $\Box$

We will now prove the Routh necessary and sufficient condition for the real polynomial~(\ref{eq:polyhurrou}) to have its zeros in the open left-half plane. We first use the polynomial coefficients to form the following $(n+1)$-row Routh array:
\begin{equation}
\begin{array}{cc|ccccc}
s^n & \mb{row} & a_n&a_{n-2}&a_{n-4}& a_{n-6}& \cdots\\
s^{n-1} & \mb{row} & a_{n-1}&a_{n-3}&a_{n-5}& a_{n-7}& \cdots\\
s^{n-2} & \mb{row} & r_{21}&r_{22}&r_{23}& r_{24}& \cdots\\
s^{n-3} & \mb{row} & r_{31}&r_{32}&r_{33}& r_{34}& \cdots\\
{}&\vdots&{}&\vdots&{}&\vdots&{}\\
s^0&\mb{row}& r_{n1}&{}&{}&{}&{}
\end{array}
\label{eq:rouhurarr}
\end{equation}
The elements that comprise the $s^n$ and $s^{n-1}$ rows are appropriately ordered coefficients of $p(s)$. The elements of the $s^{n-2}$ row are formed from its preceding two rows. They are:
\begin{eqnarray*}
r_{21} &=& -\frac{1}{a_{n-1}}\det\left[\begin{array}{cc}
a_n & a_{n-2}\\
a_{n-1} & a_{n-3}
\end{array}\right]=\frac{a_{n-1}a_{n-2}-a_na_{n-3}}{a_{n-1}}\\
r_{22} &=& -\frac{1}{a_{n-1}}\det\left[\begin{array}{cc}
a_n & a_{n-4}\\
a_{n-1} & a_{n-5}
\end{array}\right]=\frac{a_{n-1}a_{n-4}-a_na_{n-5}}{a_{n-1}}\\
r_{23} &=& -\frac{1}{a_{n-1}}\det\left[\begin{array}{cc}
a_n & a_{n-6}\\
a_{n-1} & a_{n-7}
\end{array}\right]=\frac{a_{n-1}a_{n-6}-a_na_{n-7}}{a_{n-1}}\\
\vdots&{}&\qquad\vdots
\end{eqnarray*}
The elements of the $s^{n-3}$ row of the Routh array are generated using its preceding two rows. They are:
\begin{eqnarray*}
r_{31} &=& -\frac{1}{r_{21}}\det\left[\begin{array}{cc}
a_{n-1} & a_{n-3}\\
r_{21} & r_{22}
\end{array}\right]=\frac{r_{21}a_{n-3}-r_{22}a_{n-1}}{r_{21}}\\
r_{32} &=& -\frac{1}{r_{21}}\det\left[\begin{array}{cc}
a_{n-1} & a_{n-5}\\
r_{21} & r_{23}
\end{array}\right]=\frac{r_{21}a_{n-5}-r_{23}a_{n-1}}{r_{21}}\\
r_{33} &=& -\frac{1}{r_{21}}\det\left[\begin{array}{cc}
a_{n-1} & a_{n-7}\\
r_{21} & r_{24}
\end{array}\right]=\frac{r_{21}a_{n-7}-r_{24}a_{n-1}}{r_{21}}\\
\vdots&{}&\qquad\vdots
\end{eqnarray*}
Subsequent rows of the array are formed from their preceding rows in an analogous fashion. Undefined positions can be filled with zeros. The last two rows of the array will have one term each. We will now formulate the stability test that in some texts (for example \cite{rohrs93} or \cite{wolovich94}) is also called the Routh-Hurwitz criterion. 
\begin{thm}[Routh]
The zeros of the polynomial $p(s)$ are all in the open left-half plane if and only if the elements of the first column of the Routh-Hurwitz array are all positive.
\end{thm}
{\bf Proof}\quad We perform elementary row operations on the Hurwitz matrix that are equivalent to premultiplying it by lower triangular matrices with ones on the diagonal. For example,
\[
\left[\begin{array}{ccccc}
1& 0 & 0&0&\cdots\\
-a_n/a_{n-1}& 1& 0&0&\cdots\\
0&0&1&0&\cdots\\
\vdots&{}&{}&\ddots&\vdots\\
0&0&0&0&1
\end{array}\right]\left[\begin{array}{cccc}
a_{n-1} & a_{n-3} & a_{n-5} & a_{n-7}\\
a_n & a_{n-2} & a_{n-4} & a_{n-6} \\
0 & a_{n-1}& a_{n-3} & a_{n-5} \\
0 & a_{n} & a_{n-2} & a_{n-4} \\
\vdots&{}&\vdots&\ddots
\end{array}\right]=\left[\begin{array}{cccc}
a_{n-1} & a_{n-3} & a_{n-5} & a_{n-7}\\
0 & r_{21} & r_{22} & r_{23} \\
0 & a_{n-1}& a_{n-3} & a_{n-5} \\
0 & a_{n} & a_{n-2} & a_{n-4} \\
\vdots&{}&\vdots&\ddots
\end{array}\right].
\]
Let $\b U$ be the product of the matrices corresponding to the above described elementary operations on $\b H$. Then,
\[
\b U=\left[\begin{array}{cccc}
1&0 &\cdots &0\\
u_{12}& 1& \cdots & 0\\
\vdots& {}&\ddots& \vdots\\
u_{n1}&u_{n2}&\cdots&1
\end{array}\right],
\]
where, for example, $u_{12}=-a_n/a_{n-1}$. Premultiplying the Hurwitz matrix~(\ref{eq:hurmatnbn}) by the matrix $\b U$ given above yields
\begin{equation}
\b U\b H=\left[\begin{array}{ccccc}
a_{n-1} & a_{n-3} & a_{n-5} & \cdots & \cdots\\
0& r_{21} & r_{22} & \cdots & r_{2,n-1}\\
0 & 0 & r_{31} & \cdots & r_{3,n-2}\\
\vdots&{}&\vdots&\ddots&\vdots\\
0&0&0&\cdots&r_{n1}
\end{array}\right].
\label{eq:multbyhum}
\end{equation}
Since the matrix $\b U$ is lower triangular with the ones on the diagonal, the leading principal minors of $\b H$ and $\b U\b H$ are the same, where
\[
H_1=a_{n-1},\quad H_2=a_{n-1}r_{21},\quad\ldots\quad ,H_i=a_{n-1}r_{21}\cdots r_{i1},\quad i=1,2,\ldots,n.
\]
Hence,
\[
a_{n-1}=H_1,\quad\mb{and}\quad r_{i1}=\frac{H_i}{H_{i-1}}\quad\mb{for}\quad i=2,\dots,n.
\]
By the Hurwitz Theorem, the zeros of the polynomial $p(s)$ are in the open left-half plane if the coefficients $r_{i1}$, $i=1,2\ldots,n$, which are the elements of the first column of the Routh array, are all positive. The proof is complete.\\
\hspace*{5.5in} $\Box$

Note that the Routh theorem could be formulated as follows: The zeros of the polynomial $p(s)$ are all in the open left-half plane if and only if the elements of the first column of the Routh array all have the same sign and are nonzero.

If the polynomial coefficients are negative, we can multiply both sides of the equation $p(s)=0$ by $-1$ thus making the coefficients positive and then we can proceed with the Hurwitz or Routh test to determine the stability of instability of a given polynomial.

A necessary, but not sufficient, condition for the polynomial $p(s)$ to have all its zeros in the open left-half plane is that all its coefficients are nonzero and have the same sign.

It is possible to extend the above Routh stability criterion so that it can be used to find the number of polynomial zeros with positive real parts. The number of zeros of $p(s)$ with positive real parts is equal to the number of sign changes in the first column of the Routh array. For a proof of this statement using the law of inertia of quadratic forms, see, for example, Kaczorek~\cite[Section~8.4.3]{kaczorek93}.


\section{The Discrete-Time Lyapunov Equation}
In the following, we present a background material that we will use in constructing a controller for a discrete time chaotic system. We begin with the Lyapunov approach to the stability analysis of of the discrete-time linear time-invariant systems modeled by
\begin{equation}
\b x(k+1)=\b A\b x(k),\quad \b x(k_0)=\b x_0.
\label{eq:disclitisy}
\end{equation}
By analogy with the continuous-time case, consider a quadratic positive definite form,
\[
V(\b x(k))=\b x^T(k)\b P\b x(k).
\]
We evaluate the first difference of $V$, defined $\Delta V=V(\b x(k+1))-V(\b x(k))$, on the trajectories of~(\ref{eq:disclitisy}) to obtain
\begin{eqnarray}
\Delta V(\b x(k)) &=& V(\b x(k+1))-V(\b x(k))\nonumber\\
&=&\b x^T(k+1)\b P\b x(k+1)-\b x^T(k)\b P\b x(k)\nonumber\\
&=& \b x^T(k)\b A^T\b P\b A\b x(k)-\b x^T(k)\b P\b x(k)\nonumber\\
&=& \b x^T(k)\left(\b A^T\b P\b A-\b P\right)\b x(k), 
\label{eq:frstdiofv}
\end{eqnarray}
where we replaced $\b x(k+1)$ in the above with the right-hand side of~(\ref{eq:disclitisy}), that is, with $\b A\b x(k)$. For the solutions of~(\ref{eq:disclitisy}) to decrease the ``energy function'' $V$ it is necessary and sufficient that $\Delta V(\b x(k))<0$, or equivalently, that for some positive definite $\b Q$, 
\begin{equation}
\Delta V(\b x(k))=-\b x^T(k)\b Q\b x(k).
\label{eq:delvwqpsd}
\end{equation}
Combining~(\ref{eq:disclitisy}) and~(\ref{eq:delvwqpsd}), we obtain the so called {\em discrete-time Lyapunov equation},
\begin{equation}
\fbox{$\displaystyle \b A^T\b P\b A-\b P=-\b Q $}
\label{eq:dtlyaeqfltis}
\end{equation}
We will now show that the matrix $\b A$ has its eigenvalues in the open unit disk if and only if for any positive definite $\b Q$ the solution $\b P$ to~(\ref{eq:dtlyaeqfltis}) is positive definite. We can use the Kronecker product, and in particular relation~(\ref{eq:stack_three}) to represent the discrete-time Lyapunov equation as
\begin{equation}
\left(\b I_{n^2}-\b A^T\otimes \b A^T \right)\b s(\b P)=s(\b Q),
\label{eq:dtleuskpro}
\end{equation} 
where $\otimes$ denotes the Kronecker product and $s(\cdot)$ is the stacking operator. It is easy to see that the above equation has a unique solution if and only if
\[
1-\lambda_i \lambda_j \ne 0,\quad\mb{for all}\quad i,j=1,2,\ldots,n,
\]
where $\{\lambda_i\}$ is the set of eigenvalues of $\b A$. Note that $|\lambda_i|<1$ is a sufficient condition for~(\ref{eq:dtleuskpro}) and hence~(\ref{eq:dtlyaeqfltis}) to have a unique solution.


\section{Stability of Nonlinear Systems}
\label{ch:stability}
In this section, we will be concerned with stability analysis of nonlinear time-varying dynamical system models described by
\[
\dot{\mb{\bm$x$}}(t)=\mb{\bm$f$}(t,\mb{\bm$x$}(t)), \qquad \mb{\bm$x$}(t_0)=\mb{\bm$x$}_0,
\]
where $\mb{\bm$x$}(t) \in \mb{\Bb R}^n$ is the state vector, and \mb{\bm$f$} is a vector-valued function.

To proceed, we need the following lemma. Denote by $B_q(\b 0)$ an open ball with the center at $\b 0$ and radius $q$. Let ${\cal S}\subset \mb{\Bb R}^n$ be a compact set that contains the origin $\b x=\b 0$ in its interior.
\begin{lemma}
\label{lemma:con_kap}
For a continuously differentiable function $V : \mb{\Bb R}^n \to \mb{\Bb R}$, where $V(\mb{\bm$0$}) = 0$, we have $V( \mb{\bm$x$} ) > 0$ for all $\mb{\bm$x$} \in {\cal S} \setminus \{ \mb{\bm$0$} \}$, if and only if there is a continuous, strictly increasing, function $\psi : \mb{\Bb R} \to \mb{\Bb R}$ such that $\psi (0) = 0$, $\psi (r) > 0$ for $r>0$, and $V(\mb{\bm$x$}) \ge \psi (r)$, where $r=\| \mb{\bm$x$} \|$, and $r \le R = \max \{ q : B_q(\mb{\bm$0$})  \subseteq {\cal S}\}$.
\end{lemma}
{\bf Proof}\quad Sufficiency $(\,\Leftarrow\,)$ is clear.

To prove necessity $(\, \Rightarrow \,)$ define the function $\kappa : \mb{\Bb R} \to \mb{\Bb R}$,
\begin{equation}
\kappa (y) = \min_{\stackrel{\mb{\bm$x$} \in {\cal S}}{\| \mb{\bm$x$}\| = y}} V(\mb{\bm$x$} )
\label{eq:kappdefn}
\end{equation}
Note that the function $\kappa$ is well defined since the set
\begin{equation}
\{ \mb{\bm$x$} : \|\mb{\bm$x$} \| = y \} \cap {\cal S}
\label{eq:comsetforkapp}
\end{equation}
is compact and $V$ is continuous. Therefore, by the Weierstrass theorem on page~\pageref{thm:weierstrassgen}, the function $V$ achieves its minimum over the set~(\ref{eq:comsetforkapp}).

We will now show that $\kappa$ is continuous on $B_R(\mb{\bm$0$})$. Then, we will construct an appropriate function $\psi$ satisfying the hypotheses of the lemma. To show that $\kappa$ is continuous on $B_R(\mb{\bm$0$})$, we will use the fact that $V$ is uniformly continuous in ${\cal S}$. To this end let $\mb{\bm$x$}^{(1)} \in {\cal S}$ be such that $\|\mb{\bm$x$}^{(1)}\| = y$ and $\kappa (y) = V \left(\mb{\bm$x$}^{(1)} \right)$. Next, let $\Delta y$ be an increment of $y$ and let $\mb{\bm$x$}^{(2)} \in {\cal S}$ be a vector such that $y+\Delta y = \|\mb{\bm$x$}^{(2)}\|$. For example, we can take
\[
\b x^{(2)}=\b x^{(1)}+\frac{\b x^{(1)}}{\|\b x^{(1)}\|}\Delta y.
\]
Observe that
\[
V \left(\mb{\bm$x$}^{(2)} \right) \ge \kappa ( y + \Delta y).
\]
Choose now a vector $\mb{\bm$x$}^{(3)}$ such that
\[
\|\mb{\bm$x$}^{(3)}\| = y + \Delta y,\qquad \mb{and} \qquad V\left(\mb{\bm$x$}^{(3)} \right)= \kappa (y + \Delta y).
\]
Note that for any vector, say $\mb{\bm$x$}^{(4)}$, such that $\|\mb{\bm$x$}^{(4)}\|=y$ we have $V\left( \mb{\bm$x$}^{(4)}\right) \ge \kappa(y)$---see Figure~\ref{fig:con_kap}. We can take
\[
\b x^{(4)}=\b x^{(3)}-\frac{\b x^{(3)}}{\|\b x^{(3)}\|}\Delta y.
\]
\begin{figure}
\vspace{3in}
\hskip1.25in\special{bmp:c:/mybook/con_kap.bmp x=4in y=3in}
\caption{A two-dimensional illustration of the proof of the continuity of the function $\kappa$ given by~(\ref{eq:kappdefn}).}
\label{fig:con_kap}
\end{figure}
By the uniform continuity of $V$, for any $\epsilon > 0$ there exists a $\delta > 0$ such that if
\[
\|\mb{\bm$x$}^{(1)}-\mb{\bm$x$}^{(2)}\| < \delta \quad \mb{and} \quad \|\mb{\bm$x$}^{(3)}-\mb{\bm$x$}^{(4)}\| < \delta,
\]
then
\[
\left\arrowvert V\left(\mb{\bm$x$}^{(1)} \right)-V\left(\mb{\bm$x$}^{(2)} \right)\right\arrowvert < \epsilon \quad \mb{and} \quad \left\arrowvert V\left(\mb{\bm$x$}^{(3)} \right)-V\left(\mb{\bm$x$}^{(4)} \right)\right\arrowvert < \epsilon.
\]
From $\left\arrowvert V\left(\mb{\bm$x$}^{(1)} \right)-V\left(\mb{\bm$x$}^{(2)} \right)\right\arrowvert < \epsilon$, we obtain
\[
-\epsilon < V\left(\mb{\bm$x$}^{(1)} \right)-V\left(\mb{\bm$x$}^{(2)} \right) < \epsilon.
\]
Thus
\[
V\left(\mb{\bm$x$}^{(1)} \right) > V\left(\mb{\bm$x$}^{(2)} \right) - \epsilon.
\]
From $\left\arrowvert V\left(\mb{\bm$x$}^{(3)} \right)-V\left(\mb{\bm$x$}^{(4)} \right)\right\arrowvert < \epsilon$ we get
\[
V\left(\mb{\bm$x$}^{(3)} \right)>V\left(\mb{\bm$x$}^{(4)} \right)- \epsilon.
\]
From the definition of the function $\kappa$ it follows that
\[
V \left( \mb{\bm$x$}^{(2)} \right) \ge V \left( \mb{\bm$x$}^{(3)} \right)
\]
and
\[
V \left( \mb{\bm$x$}^{(4)} \right) \ge V \left( \mb{\bm$x$}^{(1)} \right).
\]
Since $V\left(\mb{\bm$x$}^{(1)} \right) > V\left(\mb{\bm$x$}^{(2)} \right) - \epsilon$, we obtain
\[
V\left(\mb{\bm$x$}^{(1)} \right) > V\left(\mb{\bm$x$}^{(3)} \right) - \epsilon
\]
and therefore
\begin{equation}
- \epsilon < V\left(\mb{\bm$x$}^{(1)} \right) - V\left(\mb{\bm$x$}^{(3)} \right).
\label{eq:miepsxoxt}
\end{equation}
Since $V\left(\mb{\bm$x$}^{(3)} \right) > V\left(\mb{\bm$x$}^{(4)} \right) - \epsilon$,
\[
V\left(\mb{\bm$x$}^{(3)} \right) > V\left(\mb{\bm$x$}^{(1)} \right) - \epsilon,
\]
and thus
\begin{equation}
V\left(\mb{\bm$x$}^{(1)} \right) - V\left(\mb{\bm$x$}^{(3)} \right) < \epsilon.
\label{eq:pluepsxost}
\end{equation}
Combining~(\ref{eq:miepsxoxt}) and~(\ref{eq:pluepsxost}), we get
\[
\left\arrowvert V\left(\mb{\bm$x$}^{(1)} \right) - V\left(\mb{\bm$x$}^{(3)} \right)\right\arrowvert < \epsilon,
\]
or equivalently,
\[
\left|\kappa(y)- \kappa(y + \Delta y)\right| < \epsilon
\]
if
\[
\Delta y < \delta,
\]
which proves the continuity of $\kappa$.


We are now ready to construct a function $\psi$ that satisfies the hypotheses of the lemma. It has the form
\begin{equation}
\fbox{$ \displaystyle \psi (r) = r \min _{r \le y \le R} \frac{\kappa(y)}{y} $}
\label{eq:funpsidefn}
\end{equation}
Note that $\psi$ is continuous, $\psi (0) = 0$, and it is positive and strictly increasing---see Figure~\ref{fig:fun_psi} for a graphical illustration of the way $\psi$ is constructed from $\kappa$.
\begin{figure}
\vspace{3in}
\hskip1.5in\special{bmp:c:/mybook/fun_psi.bmp x=4in y=3in}
\caption{Constructing the function $\psi$ given by~(\ref{eq:funpsidefn}).}
\label{fig:fun_psi}
\end{figure}
The proof of the lemma is now complete.\\
\hspace*{5.5in}$\Box$


We can characterize a positive definite function $V$ in another way as in the following lemma.
\begin{lemma}
A function $V : \mb{\Bb R}^n \to \mb{\Bb R}$ is positive definite, with respect to $\b x=\b 0$, if and only if there exists a continuous nondecreasing function $\alpha : \mb{\Bb R} \to \mb{\Bb R}$, such that $\alpha (0) = 0$, $\alpha (r) > 0$ for $r>0$, and $V(\mb{\bm$x$}) \ge \alpha(r)$, where $r=\|\mb{\bm$x$}\|$, and $r \le R = \max \{q : B_q(\mb{\bm$0$}) \subseteq {\cal S}\}$.
\end{lemma}
A possible choice for $\alpha$ is
\[
\alpha(r)=\min_{r \le \|\mb{\bm$x$}\| \le R} V(\mb{\bm$x$}).
\]
Note that the function $\alpha$ is nondecreasing because as $r$ increases, the minimum is taken over a smaller region. Furthermore, $\alpha(r) > 0$ whenever $r>0$ since $V(\mb{\bm$x$}) > 0$ for $\mb{\bm$x$} \in {\cal S} \setminus \{\mb{\bm$0$}\}$. The function $\alpha$ is continuous since $V$ is continuous.

So far we only considered positive definite, or negative definite functions, that are time-invariant. We now extend our definition of positive definiteness to time-varying functions of the form $W(t,\mb{\bm$x$})$. We first define a time-varying positive semi-definite function.
\begin{defn}
A real-valued function $W : \mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ is positive semi-definite, with respect to $\b x=\b 0$, in a region ${\cal S} \subseteq \mb{\Bb R}^n$ if for all $t$ and all $\mb{\bm$x$}$ in $\cal S$
\begin{itemize}
\item[(i)] $W$ is continuous and has continuous partial derivatives with respect to all of its arguments, that is, $W \in {\cal C}^1$,
\item[(ii)] $W(t,\mb{\bm$0$})=0$,
\item[(iii)] $W(t,\mb{\bm$x$}) \ge 0$.
\end{itemize}
\end{defn}
\begin{defn}
A real-valued function $W : \mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ is positive definite, with respect to $\b x=\b 0$, in a region ${\cal S} \subseteq \mb{\Bb R}^n$ if for all $t$ and all $\mb{\bm$x$}$ in $\cal S$
\begin{itemize}
\item[(i)] $W \in {\cal C}^1$,
\item[(ii)] $W(t,\mb{\bm$0$})=0$,
\item[(iii)] there exists a positive definite time-invariant function $V$ such that $W(t,\mb{\bm$x$}) \ge V(\mb{\bm$x$})$.
\end{itemize}
\end{defn}
We next introduce a notion of a {\em decrescent\/} function.
\begin{defn}
A time-varying function $W : \mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ is decrescent in a region $\cal S$ if a positive definite function $\tilde{V}$ exists such that for all $\mb{\bm$x$}$ in $\cal S$ and all $t$
\[
|W(t,\mb{\bm$x$})| \le \tilde{V}(\mb{\bm$x$}),
\]
that is, $W$ tends to zero uniformly with respect to $t$ as $\| \mb{\bm$x$} \| \to 0$.
\end{defn}

Consider now a dynamical system modeled by $\dot{\mb{\bm$x$}}(t)=\mb{\bm$f$}(t,\mb{\bm$x$}(t))$ subject to the initial condition $\mb{\bm$x$}(t_0)=\mb{\bm$x$}_0$. Suppose that the system has an isolated equilibrium point at $\mb{\bm$x$}_{e}$, that is, $\mb{\bm$f$}(t,\mb{\bm$x$}_{e})=\mb{\bm$0$}$ for all $t$. Let $W(t,\mb{\bm$x$})$ be a time-dependent real-valued function whose time derivative along the solution $\mb{\bm$x$}(t)=\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)$ of the given differential equation, is
\[
\frac{dW(t,\mb{\bm$x$})}{dt}=\dot{W}(t,\mb{\bm$x$})=\frac{\partial W}{\partial t}+\left(\nabla W \right)^T \dot{\mb{\bm$x$}}.
\]
Then, we have the following theorem concerning the stability of the equilibrium point in the sense of Lyapunov.
\begin{thm}
\label{thm:loc_stab}
If in some neighborhood $\cal S$ of the equilibrium state $\mb{\bm$x$}_{e}$, there is a positive definite function $W$, with respect to $\b x_{e}$, such that its time derivative along any solution of $\dot{\mb{\bm$x$}}(t)=\mb{\bm$f$}(t,\mb{\bm$x$}(t))$ is negative semi-definite in $\cal S$, then the equilibrium state $\mb{\bm$x$}_{e}$ is stable. If $W$ is also decrescent, then the equilibrium state is uniformly stable.
\end{thm}
{\bf Proof}\quad Without loss of generality, we assume that $\mb{\bm$x$}_{e}=\mb{\bm$0$}$. Since $W$ is positive definite, by Lemma~\ref{lemma:con_kap} there is a strictly increasing function $\psi _1 (r)$, $\psi (0)=0$, such that
\[
W(t,\mb{\bm$x$}) \ge \psi _1 (\|\mb{\bm$x$}\|),
\]
where $\|\mb{\bm$x$}\| \le R$, and $R$ is the radius of the largest ball contained in $\cal S$.\\
Let
\[
\psi _2 (t_0,r)= \max _{\|\mb{\bm$x$}\| \le r} W(t_0,\mb{\bm$x$}).
\]
Then
\[
\psi _1 (r) \le \psi _2 (t_0,r).
\]
Thus, for any $\epsilon > 0$ there is a positive $\delta$ that depends on $\epsilon$ and $t_0$, such that
\[
\psi _2 (t_0, \delta ) < \min \{ \psi _1 ( \epsilon ), \psi _1 (R) \}.
\]
Figure~\ref{fig:fin_del} illustrates how to determine a $\delta$ given an $\epsilon$.
\begin{figure}
\vspace{3in}
\hskip1.5in\special{bmp:c:/mybook/fin_del.bmp x=4in y=3in}
\caption{Determining a $\delta$ given an $\epsilon$.}
\label{fig:fin_del}
\end{figure}
By assumption $\frac{dW(t,\mb{\bm$x$})}{dt} \le 0$ along any solution of the system. Hence $W(t,\mb{\bm$x$})$ is a non-increasing function and therefore
\[
W(t,\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)) \le W(t_0,\mb{\bm$x$}_0) \qquad \mb{for all} \qquad t \ge t_0.
\]
If $\|\mb{\bm$x$}_0\| < \delta$, then for $t \ge t_0$ we have
\[
\psi _1(\epsilon)>\psi_2(t_0, \delta) \ge W(t_0,\mb{\bm$x$}_0) \ge W(t,\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)) \ge \psi_1(\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|).
\]
Therefore,
\[
\psi_1 (\epsilon) > \psi_1(\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|).
\]
Since $\psi_1$ is strictly increasing, we have to have
\[
\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\| < \epsilon
\]
for all $t \ge t_0$. This means that the null solution is (locally) stable, which completes the proof of the first part of the theorem.


If $W$ is also decrescent, then there is a nondecreasing function $\tilde{\psi}_2:\mb{\Bb R} \to \mb{\Bb R}$, $\tilde{\psi}_2(0)=0$, and
\[
W(t,\mb{\bm$x$}) \le \tilde{\psi}_2 (\|\mb{\bm$x$}\|)
\]
for all $\mb{\bm$x$}$ such that $\|\mb{\bm$x$}\| \le R$ and all $t$. Indeed, since $W(t,\b x)$ is positive definite and decrescent, there exists a positive definite $V(\b x)$ such that $W(t,\b x)\le V(\b x)$. Then, we can take $\tilde{\psi}_2$ defined as
\[
\tilde{\psi}_2(r)=\max_{\|\b x\|\le r}V(\b x).
\]
Let
\[
\tilde{\psi}_2 (\delta ) < \min \{\psi_1 (\epsilon ), \psi_1 ( R )\}.
\]
Note that $\delta$ depends now only on $\epsilon$, and not on $t_0$ as before. To show the uniform stability of the null solution, we repeat the argument above with $\tilde{\psi}_2$ replacing $\psi_2$. This completes the proof of the theorem.\\
\hspace*{5.5in} $\Box$


The next theorem gives us a sufficiency condition for an equilibrium point to be uniformly asymptotically stable.
\begin{thm}
\label{thm:uniform_as}
If in some neighborhood $S$ of the given equilibrium point $\mb{\bm$x$}_{e}$ there is a positive definite, with respect to $\b x_{e}$, and decrescent function $W$ such that its time derivative along any solution of the system is negative definite in $S$, then the equilibrium state is uniformly asymptotically stable. 
\end{thm}
{\bf Proof}\quad As before, without loss of generality, we assume that $\mb{\bm$x$}_{e}= \mb{\bm$0$}$. The previous theorem ensures uniform stability of the equilibrium state $\mb{\bm$x$}_{e}$. Thus, only the uniform convergence remains to be shown. From the assumptions of the theorem it follows that there exist strictly increasing functions $\psi_1$ and $\psi_3$, and a non-decreasing function $\psi_2$ such that
\begin{eqnarray*}
\psi_1 (0) &=& \psi_2 (0) = \psi_3 (0),\\
\psi_1(\|\mb{\bm$x$}\|) &\le& W(t,\mb{\bm$x$}) \le \psi_2(\|\mb{\bm$x$}\|),\\
\psi_3(\|\mb{\bm$x$}\|) &\le& - \dot{W}(t,\mb{\bm$x$}),\end{eqnarray*}
for all $t$ and all $\mb{\bm$x$}$ such that $\|\mb{\bm$x$}\| \le R$. Given an $\epsilon_1$. We define positive constants $\delta_1$, $\delta_2$, and $T$ as follows:
\begin{eqnarray}
\psi_2(\delta_1) &<& \psi_1(R),\label{eq:p2p1r}\\
\psi_2(\delta_2) &<& \min \{\psi_1 (\epsilon_1 ), \psi_2 (\delta _1) \},\label{eq:p2de2mip1p2}\\
T&=&\psi_1 (R)/\psi_3(\delta _2).\label{eq:catp1pt}
\end{eqnarray}
Note that $\delta_1$ is independent of $t_0$, and $T$ depends on $\epsilon_1$ and $\delta_1$.\\
We shall prove the theorem in two stages. First, we shall show that for some $t_2 \in \left[ t_0,t_0+T\right]$,
\[
\|\mb{\bm$x$}(t_2;t_0,\mb{\bm$x$}_0)\| < \delta_2
\]
whenever $\|\mb{\bm$x$}_0 \|< \delta_1$. Then, we shall show that once $\|\mb{\bm$x$}(t_2;t_0,\mb{\bm$x$}_0) \| < \delta_2$, we have 
\[
\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0) \| < \epsilon_1 \quad \mb{for all} \quad t \ge t_2.
\]
We begin with proving the claim that for some $t_2 \in \left[ t_0,t_0+T\right]$,
\[
\|\mb{\bm$x$}(t_2;t_0,\mb{\bm$x$}_0)\| < \delta_2
\]
whenever $\|\mb{\bm$x$}_0 \|< \delta_1$. We prove this claim by contradiction. We thus assume that for all $t\in \left[t_0,t_0+T\right]$,
\begin{equation}
\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\| \ge \delta_2
\label{eq:noxgedt}
\end{equation}
if $\|\mb{\bm$x$}_0 \|< \delta_1$. Using~(\ref{eq:noxgedt}), (\ref{eq:catp1pt}), and~(\ref{eq:p2p1r}), we obtain
\begin{eqnarray*}
0 < \psi_1 (\delta _2 ) &\le& W[t_0+T,\mb{\bm$x$}(t_0+T;t_0,\mb{\bm$x$}_0)]\\
&=& W(t_0,\mb{\bm$x$}_0) + \int_{t_0}^{t_0+T} \dot{W}[t,\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)]dt\\
&\le& W(t_0,\mb{\bm$x$}_0) - \int_{t_0}^{t_0+T} \psi_3(\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|)dt\\
&\le& W(t_0,\mb{\bm$x$}_0) - T\psi_3(\delta_2)\\
&\le& \psi_2(\|\mb{\bm$x$}_0\|)-T\psi_3(\delta_2)\\
&<& \psi_2(\delta_1)-\psi_1(R)\\
&<&0,
\end{eqnarray*}
which is a contradiction, and thus we proved the first part of the theorem. We now proceed with the second part. From the first part we have that if $\|\mb{\bm$x$}_0\| < \delta_1$, then for some $t_2 \in \left[t_0,t_0+T\right]$ we have $\|\mb{\bm$x$}(t_2;t_0,\mb{\bm$x$}_0)\| < \delta_2$. For $t \ge t_2$ we have
\[
\psi_1(\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|) \le W\left(t,\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\right) \le W\left(t_2,\mb{\bm$x$}(t_2;t_0,\mb{\bm$x$}_0)\right) < \psi_2(\delta_2)<\psi_1(\epsilon_1).
\]
Hence
\[
\psi_1(\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|)<\psi_1(\epsilon_1),
\]
and since $\psi_1$ is strictly increasing,
\[
\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|< \epsilon_1
\]
for all $t \ge t_2$ which proves the uniform convergence. The proof of the theorem is thus complete.\\
\hspace*{5.5in} $\Box$
\begin{example}\rm
Given a dynamical system model
\[
\dot{\b x}(t)=\left[\begin{array}{cc}
-0.75 & 0.3536\\
0.3536 & -0.5
\end{array}\right]\b x(t), \quad \b x(0)=\b x_0.
\]
The solution $\b P=\b P^T$ to the Lyapunov matrix equation $\b A^T\b P+\b P\b A=-\b Q$, where $\b Q=2\b I_2$, is
\[
\b P=\left[\begin{array}{cc}
2 & \sqrt{2}\\
\sqrt{2} & 3
\end{array}\right].
\]
The eigenvalues of $\b P$ are
\[
\lambda_1=\lambda_{\min}(\b P)=1,\quad \lambda_2=\lambda_{\max}(\b P)=4.
\]
In general, for a symmetric matrix $\b P$,
\[
\lambda_{\min}(\b P)\|\b x\|^2 \le \b x^T\b P\b x\le \lambda_{\max}(\b P)\|\b x\|^2.
\]
Therefore, we can take
\[
\psi_1(\|\b x\|)=\lambda_{\min}(\b P)\|\b x\|^2,\quad \psi_2(\|\b x\|)=\lambda_{\max}(\b P)\|\b x\|^2,\quad \mb{and}\quad \psi_3(\|\b x\|)=\lambda_{\min}(\b Q)\|\b x\|^2.
\]
In our example, $\lambda_{\min}(\b Q)=2$.
\begin{itemize}
\item Given $\varepsilon=4$. We will find ${\delta}>0$ such that if $\|\b x_0\|\le \delta$, then $\|\b x(t)\|\le\varepsilon$ for $t\ge 0$.\\
We obtain $\delta=\delta(\varepsilon)$ by solving the equation,
\[
\psi_1(\varepsilon)=\psi_2(\delta),
\]
to get
\[
\delta = \sqrt{\frac{\lambda_{\min}(\b P)}{\lambda_{\max}(\b P)}}\varepsilon= 2.
\]
\item Given $\b x_0=\left[\begin{array}{cc}
8 &6
\end{array}\right]^T$ and $\varepsilon_1=2$. We will find $T$ such that $\|\b x(t)\|\le\varepsilon_1$ for all $t\ge T$.\\
We have $\|\b x_0\|_2=10$. Let $\delta_1=10$. We then compute $R$, $\delta_2$, and $T$ from the equations:
\begin{eqnarray*}
\psi_2(\delta_1) &=& \psi_1(R)\\
\psi_2(\delta_2) &=& \min \left( \psi_1(\varepsilon_1),\psi_2(\delta_1)\right)\\
T &=& \frac{\psi_1(R)}{\psi_3(\delta_2)}.
\end{eqnarray*}
Using the first equation, we find
\[
R = \sqrt{\frac{\lambda_{\max}(\b P)}{\lambda_{\min}(\b P)}}\delta_1= 20.
\]
We then solve for $\delta_2 \le \delta_1$ to get
\[
\delta_2 = \sqrt{\frac{\lambda_{\min}(\b P)}{\lambda_{\max}(\b P)}}\varepsilon_1= 1.
\]
Hence,
\[
T = \frac{\lambda_{\min}(\b P)R^2}{\lambda_{\min}(\b Q)\delta_2^2}= \frac{\lambda_{\min}(\b P)\frac{\lambda_{\max}(\b P)}{\lambda_{\min}(\b P)}\delta_1^2}{\lambda_{\min}(\b Q)\frac{\lambda_{\min}(\b P)}{\lambda_{\max}(\b P)}\varepsilon_1^2}= \frac{\lambda_{\max}^2(\b P)\delta_1^2}{\lambda_{\min}(\b Q)\lambda_{\min}(\b P)\varepsilon_1^2}= 200.
\]
\end{itemize}
\end{example}\markex
The above theorems were concerned with local stability. We now proceed to prove sufficiency conditions for global stability. For this we need the following definitions. We first define a {\em radially unbounded\/} or {\em coercive\/} function.
\begin{defn}
A time-varying function $W : \mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ is radially unbounded, or coercive, if as $\|\mb{\bm$x$}\| \to \infty$, the function $W$ tends to infinity uniformly with respect to $t$, that is, for any given positive real number $M$, there is a positive real number $N$ so that $W(t,\mb{\bm$x$}) > M$ for all $t$ and all $\mb{\bm$x$}$ whenever $\|\mb{\bm$x$}\| > N$.
\end{defn}
The next two definitions are concerned with {\em Lagrange stable equilibrium states}.
\begin{defn}
\label{defn:lagr_stab}
The equilibrium state $\mb{\bm$x$}_{e}$ is called Lagrange stable, or bounded, if for every real number $d>0$ there exists a positive real number $b=b(t_0,d)$ such that if
\[
\| \mb{\bm$x$}_0 - \mb{\bm$x$}_{e} \| < d ,
\]
then
\[
\| \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)-\mb{\bm$x$}_{e} \| < b
\]
for all $t \ge t_0$.
\end{defn}
\begin{defn}
The equilibrium state $\mb{\bm$x$}_{e}$ is uniformly bounded if the number $b$ in Definition~\ref{defn:lagr_stab} can be taken independent of the initial time $t_0$.
\end{defn}
We now define {\em uniformly asymptotically stable in the large\/} equilibrium state.
\begin{defn}
If the equilibrium state $\mb{\bm$x$}_{e}$ is uniformly stable, uniformly bounded, and all trajectories are uniformly convergent to $\mb{\bm$x$}_{e}$, then this equilibrium state is called uniformly asymptotically stable in the large, or globally uniformly asymptotically stable.
\end{defn}
The following theorem gives a sufficiency condition for an equilibrium state to be globally uniformly asymptotically stable.
\begin{thm}
\label{thm:guas}
If there is a function $W$ that is positive definite for all $\mb{\bm$x$} \ne \mb{\bm$x$}_{e}$, $W(t,\mb{\bm$x$}_{e})=0$, decrescent, radially unbounded, and if its time derivative along any solution of the system is negative definite for all $\mb{\bm$x$}$, then the equilibrium state $\mb{\bm$x$}_{e}$ is globally uniformly asymptotically stable. 
\end{thm}
{\bf Proof}\quad Assume, for simplicity, that $\b x_{e}=\b 0$. By Theorem~\ref{thm:loc_stab}, the equilibrium state $\mb{\bm$x$}_{e}$ is uniformly stable. Similarly as in Theorem~\ref{thm:uniform_as}, we construct the functions $\psi_1$, $\psi_2$, and $\psi_3$. Since, by assumption, $W$ is radially unbounded, $\psi_1(\|\mb{\bm$x$}\|) \to \infty$ as $\|\mb{\bm$x$}\| \to \infty$. Therefore, for any constant $d>0$, there is a constant $b > 0$ such that $\psi_1(b) > \psi_2(d)$. See Figure~\ref{fig:fin_bpr} for an illustration of the procedure that can be used to find the constant $b$.
\begin{figure}
\vspace{3in}
\hskip1in\special{bmp:c:/mybook/fin_bpr.bmp x=4in y=3in}
\caption{Determining a constant $b>0$ given $d>0$.}
\label{fig:fin_bpr}
\end{figure}
If $\| \mb{\bm$x$}_0 \|<d$, then $\| \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0) \|<b$ for all $t \ge t_0$. Indeed, since
\[
\psi_1(b)> \psi_2(d) \ge W(t_0,\mb{\bm$x$}_0) \ge W(t,\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)) \ge \psi_1(\| \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0) \|),
\]
then
\[
\psi_1(b)>\psi_1(\| \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0) \|).
\]
The above implies that
\[
\| \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0) \|<b
\]
since $\psi_1$ is strictly increasing. Hence all trajectories of the system are uniformly bounded.\\
To prove their convergence to the equilibrium state, we define $\delta_2$, $T$, and $b$, given $d$ and $\epsilon_1$, as
\begin{eqnarray*}
\psi_2(d) &<& \psi(b),\\
\psi_2(\delta_2) &<& \psi_1(\epsilon_1), \\
T &=& \psi_1(b)/ \psi_3(\delta_2).
\end{eqnarray*}
If $\|\mb{\bm$x$}_0\|<d$, then for all $t>t_0+T$, we have
\[
\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|<\epsilon_1,
\]
which can be shown using the same arguments as in the proof of Theorem~\ref{thm:uniform_as} with $b$ playing the role of $R$, and $d$ playing the role of $\delta_1$. This proves that all trajectories of the system converge uniformly to the equilibrium state. Hence, the equilibrium state is globally uniformly asymptotically stable.\\
\hspace*{5.5in} $\Box$


We now give a theorem that can be used as a test for checking the instability of nonlinear dynamical systems. This Instability Test is illustrated in Figure~\ref{fig:instabt}.
\begin{thm}
Let $\mb{\bm$x$}_{e}$ be an isolated equilibrium state of the system $\dot{\mb{\bm$x$}}=\mb{\bm$f$}(t,\mb{\bm$x$})$, that is, $\mb{\bm$f$}(t,\mb{\bm$x$}_{e})=\mb{\bm$0$}$. If there exists a function $W:\mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ such that in the region ${\cal S}=\{ \mb{\bm$x$}:\|\mb{\bm$x$}-\mb{\bm$x$}_{e}\| \le R \}$
\begin{itemize}
\item[(i)]
$W \in {\cal C}^1$,
\item[(ii)]
$W(t,\mb{\bm$x$}_{e})=0$,
\item[(iii)]
$\dot{W}(t,\mb{\bm$x$}) > 0$ for all $\mb{\bm$x$} \in {\cal S} \setminus \{\mb{\bm$x$}_{e} \}$,
\item[(iv)]
in every neighborhood of $\mb{\bm$x$}_{e}$ in ${\cal S}$, for any $t_0$ there is at least one point $\mb{\bm$x$}$ at which $W(t_0,\mb{\bm$x$})>0$,
\end{itemize}
then the equilibrium state $\mb{\bm$x$}_{e}$ is unstable.
\end{thm}
{\bf Proof}\quad Without loss of generality, assume that $\mb{\bm$x$}_{e}=\mb{\bm$0$}$. By assumption~{\it (iv)}, for any $t_0$ and any $\delta$, $0<\delta<R$ there exists $\mb{\bm$x$}_{\delta}$, $0<\|\mb{\bm$x$}_{\delta}\| < \delta$ such that $W(t_0,\mb{\bm$x$}_{\delta}) > 0$. The system trajectory $\mb{\bm$x$}(t)=\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_{\delta})$ cannot approach the equilibrium state $\mb{\bm$x$}_{e}=\mb{\bm$0$}$ as $t \to \infty$ because $W(t,\mb{\bm$0$})=0$ but $W(t_0,\mb{\bm$x$}_{\delta}) > 0$ and $\dot{W}(t,\mb{\bm$x$})>0$ for $\mb{\bm$x$} \ne \mb{\bm$x$}_{e}$.
\begin{figure}
\vspace{3in}
\hskip1in\special{bmp:c:/mybook/instabt.bmp x=4in y=3in}
\caption{An illustration of the Instability Test.}
\label{fig:instabt}
\end{figure}
Since $\dot{W}$ is continuous, because $W \in {\cal C}^1$, and the trajectory is bounded away from the equilibrium state, there is a constant $m>0$ such that for all $t \ge t_0$,
\[
\dot{W}(t,\mb{\bm$x$}(t)) \ge m >0,
\]
as long as
\[
\|\mb{\bm$x$}(t)\| \le R, \qquad \mb{\bm$x$} \ne \mb{\bm$0$}.
\]
Hence,
\begin{equation}
W(t,\mb{\bm$x$})=W(t_0,\mb{\bm$x$}_{\delta})+\int_{t_0}^{t} \dot{W}(s,\mb{\bm$x$}(s))ds \ge W(t_0,\mb{\bm$x$}_{\delta})+m(t-t_0).
\label{eq:instexwgt}
\end{equation}
The function $W$ is continuous and therefore bounded on the compact set ${\cal S}$. However, the right hand side of~(\ref{eq:instexwgt}) is unbounded. Therefore the trajectory $\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_{\delta})$ cannot remain infinitely long in $\cal S$. Thus, for some $t_1>t_0$ the trajectory reaches the boundary of the region $\|\mb{\bm$x$}\| \le R$. Hence, given any $\epsilon$, $0<\epsilon<R$, then for any $\delta>0$, no matter how small, there is at least one trajectory such that $\|\mb{\bm$x$}_{\delta}\| < \delta$ but $\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_{\delta})\| \ge \epsilon$ for some $t_1 > t_0$. Therefore, the equilibrium state is unstable.\\
\hspace*{5.5in} $\Box$


\begin{corollary}
If there exists a function $W:\mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ such that in the region ${\cal S}=\{ \mb{\bm$x$}:\|\mb{\bm$x$}-\mb{\bm$x$}_{e}\| \le R \}$
\begin{itemize}
\item[(i)]
$W \in {\cal C}^1$,
\item[(ii)]
$W(t,\mb{\bm$x$}_{e})=0$,
\item[(iii)]
$\dot{W}(t,\mb{\bm$x$}) < 0$ for all $\mb{\bm$x$} \in {\cal S} \setminus \{\mb{\bm$x$}_{e} \}$,
\item[(iv)]
in every neighborhood of $\mb{\bm$x$}_{e}$ in ${\cal S}$, for any $t_0$ there is at least one point $\mb{\bm$x$}$ at which $W(t_0,\mb{\bm$x$})<0$,
\end{itemize}
then the equilibrium state $\mb{\bm$x$}_{e}$ is unstable.
\end{corollary}


\section{Lyapunov Like Analysis}
\begin{lemma}[Barbalat]
\label{lemma:barbalats}
Given a differentiable function $f=f(t):\mb{\Bb R} \to \mb{\Bb R}$ such that $\dot{f}$ is uniformly continuous. If $\lim_{t \rightarrow \infty}{{f}(t)}$ exists, then
\[
\lim_{t \rightarrow \infty}{\dot{f}(t)}=0.
\] 
\end{lemma}
{\bf Proof}\quad We prove the lemma by contraposition. We assume that $\dot{f}(t)$ is bounded away from $0$ as $t\to \infty$. Therefore, we can construct an infinite sequence of $t_i$'s such that $t_i \to \infty$ as $i\to \infty$, and for each $t_i$ we have $|\dot{f}(t_i)|\ge \varepsilon$ for some $\varepsilon >0$. By assumption $\dot{f}$ is uniformly continuous. Hence, given $\varepsilon/2>0$ there is $\delta_{\varepsilon/2}>0$ so that for any $t$ and $t_i$ such that $\left|t - t_i\right| < \delta_{\varepsilon/2}$, we have
\begin{equation}
\left|\: \left|\dot{f}(t)\right|-\left|\dot{f}(t_i )\right|\:\right|\le\left| \dot{f}(t )-\dot{f}(t_i )\right|<\frac{\varepsilon}{2}.
\label{eq:ttileeph}
\end{equation}
Suppose that we have
\[
\dot{f}(t_i)\ge \varepsilon.
\]
Then, it follows from~(\ref{eq:ttileeph}) that for any $t$ such that $|t-t_i|<\delta_{\varepsilon/2}$,
\[
\frac{\varepsilon}{2}>\left|\: \left|\dot{f}(t_i )\right|-\left|\dot{f}(t )\right|\:\right|\ge\left|\varepsilon- \left|\dot{f}(t)\right|\right|>\varepsilon-\dot{f}(t).
\]
Hence,
\[
\dot{f}(t) > \frac{\varepsilon}{2}.
\]
Using the above, we obtain
\begin{eqnarray}
\int_{t_i}^{t_i+\delta_{\varepsilon/2}}\dot{f}(t)dt&>& \int_{t_i}^{t_i+\delta_{\varepsilon/2}}\frac{\varepsilon}{2}dt\nonumber\\
&=& \frac{\varepsilon}{2}\delta_{\varepsilon/2}.
\label{eq:ittitipde}
\end{eqnarray}
However,
\begin{equation}
\int_{t_i}^{t_i+\delta_{\varepsilon/2}}\dot{f}(t)dt=f(t_i+\delta_{\varepsilon/2})-f(t_i).
\label{eq:ftipdmfti}
\end{equation}
Combining~(\ref{eq:ittitipde}) and(\ref{eq:ftipdmfti}) yields
\[
f(t_i+\delta_{\varepsilon/2})-f(t_i)>\frac{\varepsilon}{2}\delta_{\varepsilon/2}\quad\mb{for every}\quad i,
\]
which contradicts the assumption that $f(t)$ has a finite limit as $t \to \infty$. The proof of the lemma is complete.\\
\hspace*{5.5in} $\Box$

A simple sufficient condition for a differentiable function to be uniformly continuous is boundedness of its derivative. Thus if a function $f$ is twice differentiable, then its derivative, $\dot{f}$, is uniformly continuous if its second derivative, $\ddot{f}$, is bounded.

The following lemma follows immediately from Barbalat's lemma.
\begin{lemma}[``Lyapunov-Like Lemma'']
\label{lemma:lyapunov_like}
Given a real-valued function $W(t,\b x)$ such that
\begin{enumerate}
\item $W(t,\b x)$ is bounded below;
\item $\dot{W}(t,\b x)$ is negative semi-definite;
\item $\dot{W}(t,\b x)$ is uniformly continuous in time,
\end{enumerate}
then
\[
\dot{W}(t,\b x) \to 0\quad\mb{as}\quad t\to \infty.
\]
\end{lemma}
\begin{example}\rm
Investigate stability of the following dynamical system model:
\begin{equation}
\left[\begin{array}{c}
\dot{x}_1\\
\dot{x}_2
\end{array}\right]=\left[\begin{array}{c}
-2x_1+x_2u\\
-x_1u
\end{array}\right],
\label{eq:exforbarb}
\end{equation}
where the input $u$ is bounded.

Consider the Lyapunov function candidate
\[
V=\|\b x\|^2=x_1^2+x_2^2.
\]
Note that $V$ is bounded below. The time derivative of $V$ evaluated on the trajectories of the system model~(\ref{eq:exforbarb}) is
\[
\dot{V}=2x_1\dot{x}_1+2x_2\dot{x}_2=2x_1\left( -2x_1+x_2u\right)-2x_2x_1u=-2x_1^2 \le 0.
\]
Thus, the system modeled by~(\ref{eq:exforbarb}) is uniformly stable. Furthermore, using arguments similar to the ones in the proof of Theorem~\ref{thm:guas}, we can show the system's trajectories are uniformly bounded, and hence $x_1$ and $x_2$ are bounded. We now use Lemma~\ref{lemma:lyapunov_like} to show that in addition, $x_1 \to 0$ as $t\to \infty$. To use this Lemma, we have to show that $\dot{V}$ is uniformly continuous in $t$. For this it is enough to show that the derivative of $\dot{V}$ is bounded. The derivative of $\dot{V}$ is
\[
\ddot{V}=-4x_1\dot{x}_1=-4x_1\left(-2x_1+x_2u \right),
\]
which is bounded because $u$ is bounded by assumption and we showed that $x_1$ and $x_2$ are also bounded. Hence, $\dot{V}$ is uniformly continuous. The function $V$ satisfies all the conditions of Lemma~\ref{lemma:lyapunov_like} and therefore $x_1 \to 0$ as $t\to \infty$.
\end{example}\markex

\section{Nonlinear Discrete-Time System Stability}
\subsection{Notation and Definitions}
In this section, we analyze stability of dynamical systems modeled by the difference equation
\begin{equation}
\b x(k+1)=\b F(\b x(k)),
\label{eq:discsyswbf}
\end{equation}
where $k\in {\cal Z}_{+}$, ${\cal Z}_{+}$ denotes the set of nonnegative integers, $\b x:{\cal Z}_{+}\to \mb{\Bb R}^n$, that is, $\b x(k)$ is the state of the system model~(\ref{eq:discsyswbf}) at time $k$, and $\b F:\mb{\Bb R}^n \to \mb{\Bb R}^n$. We assume that the vector-valued function $\b F$ is continuous on $\mb{\Bb R}^n$, that is, if $\b x^j \to \b y$ then $\b F(\b x^j) \to \b F(\b y)$. If $\b x$ is a vector, then $\b F(\b x)$ is a vector and is the value of the function $\b F$ at $\b x$.

If $\b x$ is a function, the symbol $\b F \b x$ denotes the composition of functions
\[
\left(\b F \b x\right)(k)=\b F(\b x(k)).
\]
Thus the equation $\b x(k+1)=\b F(\b x(k))$ can also be written as $\b x(k+1)=\b F \b x(k)$.

The solution to the initial value problem
\[
\b x(k+1)=\b F \b x(k), \quad \b x(0)=\b x^0
\]
is
\[
\b x(k)= \b x^k=\b F^k\left(\b x^0\right),
\]
where $\b F^k$ is the $k$th iterate of $\b F$, that is, $\b F^0=\b I_n$, and $\b F^k=\b F \b F^{k-1}$.

A state ${\b x}_e \in \mb{\Bb R}^n$ is a {\em fixed point\/} or an {\em equilibrium state\/} of the system $\b x(k+1)=\b F \b x(k)$ if $\b F\left({\b x}_e\right)={\b x}_e$. Thus the solution of the system that starts at ${\b x}_e$ stays at ${\b x}_e$ for all $k\ge 0$.

A subset ${\cal A} \subset \mb{\Bb R}^n$ is called {\em invariant\/} if $\b F({\cal A}) \subset {\cal A}$, that is, if $\b x \in {\cal A}$ then $\b F(\b x) \in {\cal A}$. It is clear that an equilibrium solution is invariant.

The solution through $\b x^0$ is bounded if there is a constant $M$ such that $\|\b F^k\left(\b x^0\right)\| \le M$ for all $k \ge 0$.

Consider again a particular solution $\b F^k\left(\b x^0\right)$. A point $\b y \in \mb{\Bb R}^n$ is a {\em limit point\/} or a {\em point of accumulation\/} of $\b F^k\left(\b x^0\right)$ if there is a subsequence $k_i$ such that $\b F^{k_i}\left(\b x^0\right) \to \b y$. The set of all limit points of $\b F^k\left(\b x^0\right)$ is called the {\em limit set\/} and denoted $\Omega\left(\b x^0\right)$.

A sequence $\{\b x^k\}$ is said to approach a set ${\cal A}$ if
\[
\rho\left(\b x^k,{\cal A}\right)=\inf \left\{\|\b x^k-\b y\|:\b y \in{\cal A} \right\} \to 0.
\]
\begin{lemma}
\label{lemma_lim_set}
Let $\b F$ be continuous. If $\b F^k\left(\b x^0\right)$ is bounded for all $k \ge 0$, then $\Omega\left(\b x^0\right)$ is non-empty, compact invariant set. Furthermore, $\b F^k\left(\b x^0\right) \to \Omega\left(\b x^0\right)$ as $k \to \infty$.
\end{lemma}
{\bf Proof}\quad Since $\left\{\b x^k\right\}$ is bounded, by the Bolzano-Weierstrass theorem, stated on page~\pageref{thm:bolzweierstr}, there is at least one limit point of $\left\{\b x^k\right\}$ and so $\Omega\left(\b x^0\right)$ is non-empty and bounded. The set $\Omega\left(\b x^0\right)$ contains only its limit points and hence it is closed. Since it is also bounded, it is compact.

We now show that $\Omega\left(\b x^0\right)$ is invariant.  For this let $\b y \in \Omega\left(\b x^0\right)$. Hence, there is a subsequence $k_i$ such that $\b x^{k_i} \to \b y$. Since $\b F$ is continuous, $\b x^{k_i + 1} = \b F\left(\b x^{k_i}\right) \to \b F(\b y)$ as $i \to \infty$. Therefore $\b F(\b y) \in \Omega\left(\b x^0\right)$ which implies that $\Omega\left(\b x^0\right)$ is invariant. Note that we do not have to have $\b y=\b F(\b y)$.

Finally, we show that $\b F^k\left(\b x^0\right) \to \Omega\left(\b x^0\right)$ as $k \to \infty$. We prove the statement by contradiction. First observe that $\rho\left(\b x^k,\Omega\left(\b x^0\right)\right)$ is bounded since both $\left\{\b x^k\right\}$ and $\Omega\left(\b x^0\right)$ are. Assume that $\rho\left(\b x^k,\Omega\left(\b x^0\right)\right)$ does not converge to zero. Then, there is a subsequence $k_i$ such that $\b x^{k_i} \to \b y$ and $\rho\left(\b x^{k_i},\Omega\left(\b x^0\right)\right) \to a>0$. But since $\b y \in \Omega\left(\b x^0\right)$,
\[
\rho\left(\b x^{k_i},\Omega\left(\b x^0\right)\right)\le \|\b x^{k_i}-\b y\| \to 0,
\]
and thus $\rho\left(\b x^{k_i},\Omega\left(\b x^0\right)\right) \to 0$ which is a contradiction. The proof of the lemma is now complete.\\
\hspace*{5.5in} $\Box$


\subsection{LaSalle's Invariance Principle}
Let $V:\mb{\Bb R}^n \to \mb{\Bb R}$. Define the {\em Lyapunov difference\/} of $V$ relative to $\b x(k+1)=\b F \b x(k)$, or $\b F$, as
\[
\Delta V(\b x)=V\left(\b F(\b x)\right)-V(\b x).
\]
If $\b x(k)$ is a solution of $\b x(k+1)=\b F \b x(k)$, then the Lyapunov difference of $V$ relative to $\b F$ is
\[
\Delta V(\b x(k))=V\left(\b x(k+1)\right)-V(\b x(k)).
\]
If $\Delta V(\b x(k))\le 0$, then $V$ is non-increasing along solutions of the system.

We say that $V$ is a {\em Lyapunov\/} function for $\b x(k+1)=\b F \b x(k)$ on a set ${\cal G} \subseteq \mb{\Bb R}^n$ if
\begin{enumerate}
\item $V$ is continuous on $\mb{\Bb R}^n$, and
\item $\Delta V(\b x(k))\le 0$ for all $\b x(k) \in {\cal G}$.
\end{enumerate}
\begin{thm}[LaSalle's Invariance Principle]
If
\begin{enumerate}
\item $V$ is a Lyapunov function for $\b x(k+1)=\b F \b x(k)$ in ${\cal G}$, and
\item the solution $\b F^k\left(\b x^0\right)$ is bounded and in ${\cal G}$,
\end{enumerate}
then, there is a number $c$ such that $\b F^k\left(\b x^0\right) \to {\cal M} \cap V^{-1}(c)$, where ${\cal M}$ is the largest invariant set contained in the set ${\cal E}=\left\{\b x \in \mb{\Bb R}^n:\Delta V = 0\right\}\cap \bar{\cal G}$, where $\bar{\cal G}$ denotes the closure of ${\cal G}$.
\end{thm}
{\bf Proof} \quad Since $\b x^k=\b F^k\left(\b x^0\right)$ is bounded and in ${\cal G}$, by Lemma~\ref{lemma_lim_set} the limit set $\Omega\left(\b x^0\right)$ is non-empty and in ${\cal G}$. Furthermore, $\b x^k$ tends to $\Omega\left(\b x^0\right)$. Now $\left\{V\left(\b x^k\right)\right\}$ is non-increasing and bounded below, so $V\left(\b x^k\right) \to c$ for some $c$. If $\b y \in \Omega\left(\b x^0\right)$, then there is a subsequence $k_i$ such that $\b x^{k_i} \to \b y$, and hence $V\left(\b x^{k_i}\right) \to V(\b y)$, that is, $V(\b y)=c$. Thus $V\left(\Omega\left(\b x^0\right) \right)=c$, and hence $\Omega\left(\b x^0\right) \subset V^{-1}(c)$. Since $V\left(\Omega\left(\b x^0\right) \right)=c$ and $\Omega\left(\b x^0\right)$ is invariant, we have $\Delta V\left(\Omega\left(\b x^0\right)\right)=0$. In summary
\[
\b x^k \to \Omega\left(\b x^0\right) \subset \left\{\b x \in \mb{\Bb R}^n: \Delta V=0 \right\}\cap \bar{\cal G} \cap V^{-1}(c).
\]
Since $\Omega\left(\b x^0\right)$ is invariant we have $\Omega\left(\b x^0\right) \subset {\cal M}$, and the proof of the theorem is complete.\\
\hspace*{5.5in} $\Box$\\
Using the above theorem we can study stability properties of difference equations.

The general approach of the LaSalle Invariance Principle can be used to prove a result concerning the instability of isolated equilibrium states. An equilibrium point ${\b x}_e$ is isolated if there is an $\varepsilon >0$ such that there is no other equilibrium point, other than ${\b x}_e$, in the ball $B_{\varepsilon}\left({\b x}_e\right)=\left\{\b x:\|\b x-\b x_e\|<\varepsilon \right\}$.
\begin{thm}
Let ${\b x}_e$ be an isolated equilibrium state of $\b x(k+1)=\b F \b x(k)$. Let $\Delta V$ be positive definite with respect to ${\b x}_e$, and let $V$ take positive values arbitrarily close to ${\b x}_e$, and $V\left({\b x}_e\right)=0$, then ${\b x}_e$ is unstable.
\end{thm}
{\bf Proof} \quad We prove the theorem by contradiction. We thus assume that ${\b x}_e$ is stable. Let $\varepsilon > 0$ be so small that $\Delta V(\b x) >0$ for all $\b x \in B_{\varepsilon}\left({\b x}_e\right)\setminus \left\{{\b x}_e\right\}$. Since by assumption ${\b x}_e$ is stable, there is $\delta > 0$ so that if $\b x^0 \in B_{\delta}\left({\b x}_e\right)$ then $\b x^k = \b F^k\left(\b x^0\right) \in B_{\varepsilon}\left({\b x}_e\right)$ for all $k \ge 0$. By assumption there is a point $\b x^0$ such that $V\left( \b x^0 \right) >0$. Since $\b x^k$ is bounded and remains in $B_{\varepsilon}\left({\b x}_e\right)$, by Lemma~\ref{lemma_lim_set} the sequence $\left\{\b x^k\right\}$ tends to $\left\{\b x:\Delta V(\b x) = 0 \right\} \cap B_{\varepsilon}\left({\b x}_e\right)={\b x}_e$. Another way to arrive at the above observation is as follows. Note that $-V$ is a Lyapunov function because $\Delta (-V)<0$. Since $\b x^k$ is bounded we can apply the Invariance Principle to conclude that $\b x^k$ tends to $\left\{ \b x:\Delta (-V(\b x))=0 \right\}$. But $\left\{ \b x:\Delta (-V(\b x))=0 \right\}=\left\{{\b x}_e\right\}$. Thus, indeed $\b x^k \to {\b x}_e$. Using the above, we conclude that $V\left(\b x^k\right) \to V\left({\b x}_e \right) = 0$. However, $\Delta V\left(\b x^k\right) > 0$ and so
\[
V\left(\b x^k\right) \ge V\left(\b x^{k-1}\right)\ge \cdots V\left(\b x^0\right) > 0.
\]
Hence, $V\left(\b x^k\right)>0$. The above contradiction completes the proof.\\
\hspace*{5.5in} $\Box$
\section{Notes}
A classical text on stability of dynamical systems is by Hahn~\cite{hahn63}. The accounts of stability analysis using the direct method of Lyapunov are by Yoshizawa~\cite{yoshizawa75} and Rouche et al.~\cite{rouche77}.  Stability of discrete dynamical systems is presented by Hahn in~\cite{hahn58} and LaSalle~\cite{lasalle86}. Extensive coverage of stability issues can also be found in Willems~\cite{willems70}, Slotine and Li~\cite{slotine91}, Vidyasagar~\cite{vidyasagar93}, and Khalil~\cite{khalil96}.

There is an interesting story behind the origins of the Routh test. The problem of investigating the stability of a liner dynamical system in terms of the location of the roots of its characteristic equation was posed by Maxwell~\cite{maxwell68}. On page~271 of his 1868 paper, Maxwell writes, ``This condition is mathematically equivalent to the condition that all the possible roots, and all the possible parts of the impossible roots, of a certain equation shall be negative.'' By impossible roots, Maxwell means complex roots and possible parts mean real parts in today's terminology. He continues, ``I have not been able completely to determine these conditions for equations of a higher degree than the third; but I hope that the subject will obtain the attention of mathematicians.'' A complete general solution to the problem was published by Routh in 1877. Maxwell and Routh were classmates at Cambridge University. In 1854, Routh was placed first in their final math examination ahead of Maxwell. A story says, Smith~\cite[page~55]{smith96}, that apparently Maxwell was so sure of coming  first on this exam that he did not even bother to rise early to hear the exam results read out in the Senate House. Maxwell sent his servant instead. When the servant came back, Maxwell asked, ``Well, tell me who's second!''. Obviously, he did not expect to receive the reply, ``You are, sir''. Years later Routh received the prestigous Adams Prize, in 1877, for solving the problem posed by Maxwell in 1868. According to an account by Truxal~\cite[page~527]{truxal72}, Routh presented his solution with the opening remark, ``It has recently come to my attention that my good friend James Clerk Maxwell has had difficulty with a rather trivial problem....''
