%\setcounter{chapter}{4}
\chapter{Stability}\label{chapter:instab}
\begin{flushright}
\parbox{5.5in}{And consider the patience of our Lord as salvation, as our beloved brother Paul, according to the wisdom given to him, also wrote to you, speaking of these things as he does in all his letters. In them there are some things hard to understand that the ignorant and unstable distort to their own destruction, just as they do the other scriptures.~(2 Peter 3:15--16)}
\end{flushright}

\bigskip
\section{An Informal Introduction to Stability}
In general, it can be difficult to obtain an explicit solution\index{solution!explicit}\index{explicit solution} of nonlinear differential equations\index{nonlinear!differential equations} modeling a dynamical system. However, often we do not need to know the explicit solution of the modeling equations. Instead, we are interested in the behavior of the solution as time tends to infinity\index{solution!as time tends to infinity}. Furthermore, in many models of physical systems a ``small'' change in the initial conditions results in a ``small'' change in the solution\index{small change!in initial conditions}\index{small change!in solution}. Specifically, if a system is perturbed from its rest, or equilibrium position, then it starts to move. We can roughly say that the equilibrium position is stable\index{equilibrium!stable} if the system does not go far from this position for small initial perturbations\index{initial!perturbations}. This concept may be illustrated, as in Willems~\cite{willems70}, Brogan~\cite{brogan91}, or in Barnett~\cite{barnett75}, by means of a ball resting in equilibrium on a sheet of metal bent into various shapes with cross-sections depicted in Figure~\ref{fig:eq-pts}. This figure illustrates four possible scenarios. If we neglect frictional forces, a small perturbation from the equilibrium leads to:
\begin{itemize}
\item[(1)] oscillatory motion about equilibrium\index{oscillatory!motion about equilibrium} irrespective of the magnitude of the initial perturbation. In this case the equilibrium position is globally stable,
\item[(2)] the new position is also an equilibrium and the ball will remain in it. Here the equilibrium position is stable,
\item[(3)] the ball moving away without returning to the equilibrium\index{moving away!without returning to equilibrium}. This equilibrium position is unstable,
\item[(4)] oscillatory motion about the equilibrium, unless the initial perturbation is so large that the ball is forced to oscillate about the new equilibrium position. This equilibrium position is locally stable\index{equilibrium!locally stable}.
\end{itemize}
If friction is taken into account, then the oscillatory motion steadily decreases\index{oscillatory!motion steadily decreases} until the ball returns to the equilibrium positions as in items~(1) and (4) above. We refer to such equilibrium positions as being asymptotically stable\index{equilibrium!asymptotically stable}.
\begin{figure}
\centering
\includegraphics[scale=0.55]{eq-pts.eps}
%\vspace{3in}
%\centerbmp{4in}{3in}{c:/mybook/eq-pts.bmp}
%\hskip1.4in\special{bmp:c:/mybook/eq-pts.bmp x=4in y=3in}
\caption{Illustration of stable and unstable equilibrium positions.}
\label{fig:eq-pts}
\end{figure}
In our discussion of stability concepts, we implicitly assume, as in item~(4), that there is some ``small'' region for allowed initial perturbations for which subsequent motions converge to the equilibrium position. If however, as in item~(1), any initial perturbation yields a motion that converges to the equilibrium position, then we have global asymptotic stability\index{asymptotic stability!global}, which is also referred to as asymptotic stability in the large\index{asymptotic stability!in the large}.

The above informal discussion should serve as a preparation for an in-depth analysis of stability of dynamic systems. We begin our analysis by formally defining the equilibrium state. We then define stability, asymptotic stability, and other relevant notions.

\section{Basic Definitions of Stability}
We consider a class of dynamic systems modeled by the equation:
\begin{equation}
\dot{\b x}(t)=\b f(t,\b x(t)), \;{\b x}(t_0)={\b x}_0,
\label{eq:gensysmo}
\end{equation}
where ${\b x}(t)\in \mb{\Bb R}^n$ is the state vector and ${\b f}:\mb{\Bb R}\times \mb{\Bb R}^n \to \mb{\Bb R}^n$ is a vector-valued function with the components
\[
f_i(t,x_1,x_2,\ldots ,x_n):\mb{\Bb R}\times \mb{\Bb R}^n \to \mb{\Bb R},\:i=1,2,\ldots ,n.
\]
We assume that ${f_i}$'s are continuous and have continuous first partial derivatives. Under these assumptions, the solution to $\dot{\b x}(t)={\b f}(t,{\b x}(t)), \;{\b x}(t_0)={\b x}_0$, exists and is unique\index{solution!exists and unique}. We denote this solution as ${\b x}(t)={\b x}(t;t_0,{\b x}_0)$. If the  ${f_i}$'s  do not depend explicitly on $t$, then the system represented by  $\dot{\b x}(t)={\b f}(t,{\b x}(t))={\b f}({\b x}(t))$ is called {\em autonomous}\index{system!autonomous} or \emph{time-invariant}\index{system!time-invariant}, otherwise it is called {\em non-autonomous}\index{system!non-autonomous} or \emph{time-varying}\index{system!time-varying}.

A point ${\b x}_e$ is an {\em equilibrium point}, or state, at time $0$ if for all $t\ge 0$,
\[
{\b f}(t,{\b x}_e)={\b 0}.
\]
We note that if $\b x_e$ is an equilibrium state at time $0$, then it is also an equilibrium state for all initial times greater than or equal $0$.

We also note that an equilibrium state ${\b x}_e$ is a constant solution to~(\ref{eq:gensysmo})\index{equilibrium!constant solution}\index{solution!equilibrium} for $t\ge 0$.

Observe that for the time-invariant system
\[
\dot{\b x}(t)={\b f}(\b x)
\]
a point is an equilibrium at some time if and only if is is an equilibrium at all times.


If ${\b x}_e$ is an equilibrium point, we can arrange for this equilibrium point to be transferred to the origin of $\mb{\Bb R}^n$ by introducing the new variable
\[
\tilde{\b x}={\b x-\b x_e}.
\]
We assume that this has been done for an equilibrium point under consideration. Therefore, we have
\[
{\b f}(t,{\b 0})={\b 0} \quad \mbox{for all}\quad t\ge t_0.
\]
There may be several equilibrium points for a given system. We assume that there is no other equilibrium point in a neighborhood of the origin of $\mb{\Bb R}^n$. Thus, the origin is an isolated equilibrium point\index{equilibrium!isolated}. We formally define isolated equilibrium in Definition~\ref{defn:iso_equilb}.

We now introduce basic definitions of stability in the sense of A. M. Lyapunov\footnote{Sometimes transliterated in the literature as Liapunov or Liapunoff.} (1857-1918)\index{stability!in the sense of Lyapunov}. In the following considerations we use the standard Euclidean norm of a vector. Thus, if ${\b x} \in \mb{\Bb R}^n$ then
\[
\|{\b x}\|=\|{\b x}\|_2=({\b x}^{\top}{\b x})^{\frac{1}{2}}.
\]
\begin{defn}
\label{def:stab}
An equilibrium state ${\b x}_e$ is said to be stable if for any given $t_0\ge 0$ and any positive scalar $\varepsilon$, there exists a positive scalar $\delta = \delta(t_0,\varepsilon)$ such that if $\|{\b x}(t_0)-{\b x}_e\|<\delta$, then $\|{\b x}(t;t_0,{\b x}_0)-{\b x}_e\|<\varepsilon$  for all $t\ge t_0$.
\end{defn}\markdefn

Friedland~\cite[page~43]{friedland96} makes the following comment regarding the above definition of stability: ``If this ``epsilonics'' definition\index{epsilonics definition} leaves you cold, think of a contest between you, the system designer, and an adversary (nature?). The adversary picks a region in state space of radius $\varepsilon$ and challenges you to find another region, of radius $\delta$, such that if the initial state starts out inside your region, it remains inside his region. If you can do this, your system design is stable.'' We add that you have to find an appropriate $\delta$ for any choice of $\varepsilon$ for the equilibrium state to be stable.

In summary, an equilibrium state, $\b x_e$, is stable if whenever the initial state is near $\b x_e$, the state trajectory, $\b x(t)$, remains near the equilibrium, perhaps even tending toward this equilibrium state as time increases. Note that Lyapunov's stability does not imply that $\b x(t)$ converges to $\b x_e$ as $t\to \infty$. The states are only required to ``hover'' around the equilibrium state $\b x_e$. However, the stability condition bounds the amount of wiggling room for $\b x(t)$ in the sense that small perturbations in the initial conditions produce small perturbations in the solution $\b x(t)$.

\begin{defn}
\label{def:astab}
An equilibrium state ${\b x}_e$ is said to be {\em convergent\/}\index{equilibrium!convergent}, or {\em attractive\/}\index{equilibrium!attractive}, if for any given $t_0\ge 0$ there is a positive scalar ${\delta}_1={\delta}_1(t_0)$ such that if $\|{\b x}(t_0)-{\b x}_e\|<{\delta}_1$, then
\[
\lim_{t \to \infty}{\b x}(t;t_0,{\b x}_0)={\b x}_e.
\]
\end{defn}\markdefn

We can restate Definition~\ref{def:astab} as follows. An equilibrium state ${\b x}_e$ is convergent if for any $t_0\ge 0$ there is a positive scalar ${\delta}_1={\delta}_1(t_0)$ such that if $\|{\b x}(t_0)-{\b x}_e\|<{\delta}_1$, then for any given ${\varepsilon}_1>0$ there exists a $T=T({\varepsilon}_1,t_0,{\b x}_0)$ such that $\|{\b x}(t;t_0,{\b x}_0)-{\b x}_e\|<{\varepsilon}_1$ for all $t>t_0+T$.
\begin{defn}\label{defn:stab_plus_conver}
An equilibrium state ${\b x}_e$ is asymptotically stable if it is stable and convergent.
\end{defn}\markdefn

We note that for linear time-invariant\index{linear time-invariant (LTI)} (LTI)\index{LTI system!see{linear time-invariant}} systems asymptotic stability is equivalent with convergence. That is, in a convergent LTI system, stability condition is automatically satisfied. In general, however, it is not sufficient that just
\[
\b x(t;t_0,\b x_0) \to \b x_e\quad\mb{as}\quad t\to \infty
\]
for asymptotic stability\index{asymptotic stability!stability and convergence}. We need stability too! Why? Hahn~\cite{hahn67}, see also~\cite[p.~451]{antsaklis97},  constructed a nonlinear system example~\ref{eq:hahns_examp} where convergence does not mean asymptotic stability as in the case of linear time-invariant systems. In this example the origin is attractive but not asymptotically stable\index{Hahn's example}.
\begin{example}\label{ex:hahns_example}\rm
This is a second-order nonlinear system described by
\begin{equation}
\left[\begin{array}{c}
\dot{x}_1\\
\dot{x}_2
\end{array}\right]=\left[\begin{array}{c}
\frac{x_1^2(x_2-x_1)+x_2^5}{(x_1^2+x_2^2)(1+(x_1^2+x_2^2)^2)}\\
\frac{x_2^2(x_2-2x_1)}{(x_1^2+x_2^2)(1+(x_1^2+x_2^2)^2)}
\end{array}\right].
\label{eq:hahns_examp}
\end{equation}
A phase portrait\index{Hahn's example!phase portrait} of the above system is shown in Figure~\ref{fig:hahn_example}. All trajectories, solutions, are approaching the equilibrium state $\b x_e=\b 0$, without this equilibrium being asymptotically stable.
\begin{figure}
\centering
\includegraphics[scale=0.575]{hahn_example.eps}
\caption{A phase portrait of system~\eqref{eq:hahns_examp} where the origin is attractive but not asymptotically stable.}
\label{fig:hahn_example}
\end{figure}
\end{example}\markex


\begin{defn}
An equilibrium state ${\b x}_e$ is said to be uniformly stable\index{equilibrium!uniformly stable} if the positive scalar $\delta={\delta}(t_0,\varepsilon)$ introduced in Definition~\ref{def:stab} can be taken independent of the initial time $t_0$.
\end{defn}\markdefn

\begin{example}\label{ex:notuniform}\rm
The following example that comes from Rugh~\cite[page~100]{rugh96}\index{Rugh's example} nicely illustrates the notion of uniform stability. The solution to the differential equation,
\begin{equation}
\dot{x}(t)=(4 t \sin t -2t ) x(t),\quad x\left(t_0\right)=x_0
\label{eqn:nonuode}
\end{equation}
is
\begin{equation}
x(t)=x_0 \exp \left(  4 \sin t - 4 t \cos t - t^2 - 4 \sin t_0 + 4 t_0 \cos t_0 + t_0^2\right).
\label{eqn:solutononu}
\end{equation}
One can verify this using MATLAB's Symbolic Toolbox\index{MATLAB!Symbolic Toolbox} by typing the command,
\begin{verbatim}
x=dsolve('Dx=(4*t*sin(t)-2*t)*x, x(t_0)=x_0','t')
\end{verbatim}
It is easy to show that for fixed $t_0$ there is a $\gamma$ such that~(\ref{eqn:solutononu}) is bounded by $\gamma |x_0|$ for all $t \ge t_0$. This is because the term $(-t^2)$ dominates the exponent as $t$ increases. Therefore, for a given $\varepsilon$, it is enough to choose $x_0$ so that $\gamma |x_0|<\varepsilon$, or equivalently $\delta < \varepsilon/\gamma$, to show that the equilibrium solution $x_e=0$ is stable. However, the equilibrium solution $x_e=0$ is not uniformly stable. To show this, fix initial state $x_0$ and consider a sequence of initial times $t_0 = 2 k \pi$, where $k=0, 1,\ldots$, and the values of the respective solutions at times $\pi$ units later:
\[
x(2k \pi+\pi)=x_0 \exp \left((4k+1)\pi(4-\pi) \right).
\]
One can see that there is no bound on the exponential factor\index{exponential factor} and hence a candidate bound $\gamma$ increases as $k$ increases, where increasing $k$ corresponds to the increasing initial time.
\end{example}\markex
\begin{defn}
An equilibrium state ${\b x}_e$ is said to be uniformly convergent\index{equilibrium!uniformly convergent}, or uniformly attractive\index{equilibrium!uniformly attractive}, if the positive scalars $\delta _1={\delta}_1(t_0)$ and $T=T({\varepsilon}_1,t_0,{\b x}_0)$  introduced in Definition~\ref{def:astab} are independent of $t_0$.
\end{defn}\markdefn

The last two definitions allow us to introduce the notion of uniform asymptotic stability.
\begin{defn}
An equilibrium state ${\b x}_e$ is said to be uniformly asymptotically stable\index{uniformly asymptotically stable} if it is uniformly stable and uniformly convergent\index{asymptotically stable!uniformly}.
\end{defn}\markdefn

Notice that the solution ${\b x}(t)={\b x}_e$ must first be stable in order to qualify to be asymptotically stable. This is to prevent a system trajectory from straying arbitrarily far from the equilibrium state before converging toward it. Note also that the concepts of Lyapunov stability and asymptotic stability are local concepts\index{stability!local concept}. They apply to the system's behavior in the small or in a ``small'' neighborhood about the equilibrium point (Hsu and Meyer~\cite{hsu68}).

A special case of uniform asymptotic stability is {\em uniform exponential stability\/}\index{uniform exponential stability}\index{exponential stability!uniform } that implies uniform stability, and imposes an additional requirement that all solutions originating ``close'' to the given equilibrium, $\b x_e=\b 0$, approach this equilibrium exponentially as $t\to \infty$. More rigorously, we say that $\b{x}_e=\b 0$ is uniformly exponentially stable if it is stable and there exist finite positive constants $\gamma$ and $\lambda$ such that for any $t_0$ and $\b x_0$, any solution starting in a neighborhood of $\b 0$ satisfies
\[
\left\Arrowvert \b x(t)\right\Arrowvert \le \gamma e^{-\lambda \left(t-t_0\right)}\left\Arrowvert \b x_0\right\Arrowvert.
\]
Note that $\gamma$ is no less than unity, and uniform implies that $\gamma$ and $\lambda$ are independent of $t_0$---see Figure~\ref{fig:ues}.
%\begin{figure}
%\par
%\centerline{\psfig{figure=ues.eps,height=3cm}}
%\par
%\caption{Illustration of uniform exponential stability: a decaying-%exponential bound is independent of the initial time $t_0$.}
%\label{fig:ues}
%\end{figure}
\begin{figure}
\centering
\includegraphics[scale=0.6]{ues.eps}
%\vspace{3.5in}
%\hskip1in\special{bmp:c:/mybook/ues.bmp x=4.67in y=3.5in}
\caption{Illustration of uniform exponential stability: a decaying-exponential bound is independent of the initial time $t_0$.}
\label{fig:ues}
\end{figure}


An equilibrium state is unstable\index{equilibrium!unstable} if it is not stable. An $\varepsilon$-$\delta$ definition of instability is given below.
\begin{defn}
An equilibrium state ${\b x}_e$ is said to be unstable if there is an $\varepsilon>0$ so that for any $\delta>0$ there exists ${\b x}(t_0)$ such that if $\|{\b x}(t_0)-{\b x}_e\|<\delta$, then $\|{\b x}(t_1)-{\b x}_e\| \ge \varepsilon$ for some $t_1>t_0$.
\end{defn}\markdefn

A two dimensional geometric interpretation\index{stability!geometric interpretation} of the above definitions is shown in Figure~\ref{fig:2d-stab}.
\begin{figure}[h]
\centering
\includegraphics[scale=0.6]{twostab.eps}
%\vspace{4.5in}
%\centerbmp{5.33in}{4in}{c:/mybook/twostab.bmp}
%\special{bmp:c:/mybook/twostab.bmp x=6in y=4.5in}
\caption{Illustration of stability concepts in two dimensions: (a)~stable equilibrium state in the sense of Lyapunov, (b)~asymptotically stable equilibrium state, (c)~unstable equilibrium state.}
\label{fig:2d-stab}
\end{figure}
We assume that $\b x_e=\b 0$ is the equilibrium state. If the equilibrium state is stable, as in Figure~\ref{fig:2d-stab}(a), then given an outer circle of radius $\varepsilon$, there exists an inner circle of radius $\delta$ such that trajectories starting within the $\delta$-circle never leave the $\varepsilon$-circle. If the equilibrium state is asymptotically stable, then trajectories tend to the equilibrium state as $t$ tends to $\infty$. This situation is depicted in Figure~\ref{fig:2d-stab}(b). Finally, the equilibrium state is unstable, as in Figure~\ref{fig:2d-stab}(c), then there exists an $\varepsilon$-circle such that for every $\delta$-circle there exists a trajectory starting within it that leaves the $\varepsilon$-circle at some later time. A geometric interpretation of the above described scenarios in one dimension is given in Figure~\ref{fig:1d_stab}.
\begin{figure}
\centering
\includegraphics[scale=0.55]{staba.eps}
%\vspace{3in}
%\centerbmp{2.67in}{2in}{c:/mybook/staba.bmp}
%\hskip1in\special{bmp:c:/mybook/staba.bmp x=4in y=3in}
%\begin{center}
%(a) stable equilibrium state in the sense of Lyapunov
%\end{center}
%\end{figure}
%\begin{figure}
%\vspace{3in}
\centering
\includegraphics[scale=0.55]{stabb.eps}
%\centerbmp{2.67in}{2in}{c:/mybook/stabb.bmp}
%\hskip1in\special{bmp:c:/mybook/stabb.bmp x=4in y=3in}
%\begin{center}
%(b) asymptotically stable equilibrium state
%\end{center}
\end{figure}
\begin{figure}
%\vspace{3in}
\centering
\includegraphics[scale=0.55]{stabc.eps}
%\centerbmp{2.67in}{2in}{c:/mybook/stabc.bmp}
%\hskip1in\special{bmp:c:/mybook/stabc.bmp x=4in y=3in}
%\begin{center}
%(c) unstable equilibrium state
%\end{center}
\caption{Illustration of stability concepts in one dimension: (a)~stable equilibrium state in the sense of Lyapunov, (b)~asymptotically stable equilibrium state, (c)~unstable equilibrium state.}
\label{fig:1d_stab}
\end{figure}

\section{Stability of Continuous-Time LTI Systems}
It follows from the discussion in the previous sections that a linear time-invariant~(LTI) dynamical system modeled by the equation
\begin{equation}
\dot{\b x}={\b A\b x}, \quad {\b x}(t_0)={\b x}_0
\label{eq:simlinsysfls}
\end{equation}
is asymptotically stable if and only if, the solution to equation~(\ref{eq:simlinsysfls}) decays to zero as $t \to \infty$ for any initial state ${\b x}_0$. We can view the vector $\b x(t)\in \mb{\Bb R}^n$ as defining the coordinates of a point moving in an $n$-dimensional state space $\mb{\Bb R}^n$. In an asymptotically stable system modeled by equation~(\ref{eq:simlinsysfls}), this point converges to the origin of $\mb{\Bb R}^n$. We can argue that if a trajectory is converging to the origin of the state space, then it should be possible to find a family of nested surfaces described by $V(x_1,x_2, \ldots,x_n)=c$, $c\ge0$, such that monotonically decreasing\index{decreasing!monotonically} values of $c$ give surfaces progressively shrinking in on the origin with the limiting surface $V(x_1,x_2, \ldots,x_n)=0$ being the origin $\b x=\b 0$. Furthermore, along every trajectory of~(\ref{eq:simlinsysfls}), the parameter $c$ should steadily decrease. We now introduce some notation and definitions that we use in the ensuing discussion.

Let $V({\b x})=V(x_1,x_2, \ldots,x_n):\mb{\Bb R}^n \to \mb{\Bb R}$ be a real-valued function and let ${\cal S} \subseteq \mb{\Bb R}^n$ be a compact region containing the origin ${\b x=\b 0}$ in its interior.
\begin{defn}
We say that the function $V=V({\b x})$ is {\em positive semi-definite}\index{function!positive semi-definite} (p.s.d.) in ${\cal S}$,\index{positive semi-definite (p.s.d.)}\index{p.s.d.|see{positive semi-definite}} with respect to $\b x=\b 0$, if
\begin{enumerate}
\item  $V$ is continuously differentiable, that is, $V \in {\cal C}^1$,
\item $V({\b 0})=0$,
\item $V({\b x}) \ge 0$ for all ${\b x} \in {\cal S}$.
\end{enumerate}
\end{defn}\markdefn

\begin{defn}
We say that the function $V=V({\b x})$ is {\em positive definite} (p.d.)\index{positive definite (p.d.)}\index{p.d.|see{positive definite}} in ${\cal S}$, with respect to $\b x=\b 0$, if
\begin{enumerate}
\item $V \in {\cal C}^1$,
\item $V({\b 0})=0$,
\item \label{it:stmin} $V({\b x}) > 0$ for all ${\b x} \in {\cal S} \setminus \{ {\b 0} \}$.
\end{enumerate}
\end{defn}\markdefn

The symbol ${\cal S} \setminus \{ {\b 0} \}$ in item~\ref{it:stmin} above denotes the set $\{ {\b x} : {\b x} \in {\cal S}, {\b x \ne \b 0} \}$.

{\em Negative semi-definite} (n.s.d.)\index{negative semi-definite (n.s.d.)}\index{n.s.d.|see{negative semi-definite}} and {\em negative definite} (n.d.)\index{negative definite (n.d.)}\index{n.s.d.|see{negative definite}} functions are defined in a similar fashion by reversing inequality signs in item~\ref{it:stmin} in both definitions above.

There are many functions that satisfy the above definitions. In the stability analysis of dynamic systems one class is particularly useful. This is the positive definite quadratic form, a label attached to a nested family of ellipses and their extensions to higher dimensional spaces---ellipsoids for $n = 3$, and hyper-ellipsoids for $n \ge 4$. Quadratic forms are reviewed in Section~\ref{sec:quadratform}. We consider the following quadratic form\index{quadratic form},
\[
V({\b x})={\b x}^{\top}{\b P\b x},
\]
where ${\b P}$ is a real symmetric $n \times n$ matrix.
The time derivative of $V({\b x}(t))$ evaluated on a solution of $\dot{\b x}={\b A\b x}$ is
\begin{eqnarray*}
\frac{dV({\b x}(t))}{dt} &=& \dot{V}(\b x(t)) \\
&=& \dot{\b x}(t)^{\top} \b P \b x(t) + \b x(t)^{\top}{\b P} \dot{\b x}(t).
\end{eqnarray*}
Substituting $\dot{\b x}={\b A\b x}$ and $\dot{\b x}^{\top}=\b x^{\top} \b A^{\top}$ into the above equation yields
\begin{eqnarray*}
\dot{V} &=& {\b x}^{\top}{\b A}^{\top}{\b P\b x}+{\b x}^{\top}{\b P\b A\b x} \\
&=&{\b x}^{\top}({\b A}^{\top}{\b P}+{\b P\b A}){\b x}.
\end{eqnarray*}
Let
\[
{\b Q}=-({\b A}^{\top}{\b P+\b P\b A}).
\]
Then,
\begin{eqnarray}
\dot{V}&=& \frac{dV({\b x})}{dt}\nonumber\\
&=&-{\b x}^{\top}{\b Q\b x}.
\label{eq:derofq}
\end{eqnarray}

We are ready to state and prove the theorem\index{Lyapunov's theorem} concerning the stability of the system $\dot{\b x}={\b A\b x}$, or equivalently, the matrix ${\b A} \in \mb{\Bb R}^{n\times n}$.
\begin{thm}[A. M. Lyapunov, 1892]
\label{Lyapunov}
The system $\dot{\b x}={\b A\b x}$, ${\b x}(t_0)={\b x}_0$,  is asymptotically stable if and only if for any given real symmetric positive definite (r.s.p.d.) matrix ${\b Q}$ the solution ${\b P}$ to the continuous Lyapunov matrix equation\index{Lyapunov matrix equation!continuous}\index{Lyapunov equation}
\begin{equation}
{\b A}^{\top}\b P+\b P\b A=-\b Q
\label{eq:layamateqn}
\end{equation}
is also (real symmetric) positive definite.
\end{thm}
{\bf Proof} \quad($\Rightarrow$)\quad We prove necessity, for asymptotic stability\index{stability!asymptotic}, by contradiction. We have $-{\b x}^{\top}{\b Q\b x}<0$ for all ${\b x \neq \b 0}$. We assume that ${\b A}$ is an asymptotically stable matrix and that for some nonzero ${\b x}_0$ we have ${\b x}_0^{\top}{\b P \b x}_0 \le 0$. From the discussion before the theorem, and in particular from~(\ref{eq:derofq}), it follows that
\[
\frac{d}{dt}\left({\b x}(t)^{\top}{\b P\b x}(t)\right)=-{\b x}(t)^{\top}{\b Q\b x}(t) < 0.
\]
The above means that $V({\b x}(t))={\b x}(t)^{\top}{\b P\b x}(t)$ decreases on the trajectories of the system $\dot{\b x}(t)=\b A\b x(t)$ as $t \to \infty$. On the other hand, because ${\b A}$ is asymptotically stable, we have
\[
\lim_{t \rightarrow \infty}{\b x}(t)={\b 0}.
\]
Hence,
\[
\lim_{t \to \infty}{\b x}(t)^{\top}{\b P\b x}(t)=\lim_{{\b x}(t) \to {\b 0}}{\b x}(t)^{\top}{\b P\b x}(t) = 0.
\]
Thus, we arrived at a contradiction. This completes the proof of the first part of the theorem that states that asymptotic stability of ${\b A}$ implies that for any r.s.p.d matrix ${\b Q}$ the solution ${\b P}$ to the equation ${\b A}^{\top}{\b P+\b P\b A=-\b Q}$ is r.s.p.d..

($\Leftarrow$)\quad We prove sufficiency, also by contradiction. We assume that
\[
\b x^{\top}\b P\b x>0\quad\mb{and}\quad -\b x^{\top}\b Q\b x<0\quad \mb{for all}\quad \b x \ne \b 0,
\]
and that the matrix ${\b A}$ is not asymptotically stable\index{stable!asymptotically}. This means that the solution to $\dot{\b x}(t)={\b A\b x}(t)$, ${\b x}_0 \ne {\b 0}$, which has the form ${\b x}(t)=e^{{\b A}(t-t_0)}{\b x}_0$, is bounded away from the origin ${\b x=\b 0}$. Thus, there is a constant $b > 0$ such that
\[
{\b x}(t)^{\top}{\b Q\b x}(t)>b>0 \quad \mbox{for all }t>t_0.
\]
Integrating $\frac{d}{dt}\left({\b x}(t)^{\top}{\b P\b x}(t)\right)=-{\b x}(t)^{\top}{\b Q\b x}(t)$ yields
\[
{\b x}(t)^{\top}{\b P\b x}(t)={\b x}(t_0)^{\top}{\b P\b x}(t_0)+\int_{t_0}^{t}(-{\b x}(s)^{\top}{\b Q\b x}(s))ds.
\]
Taking into account that $-{\b x}(t)^{\top}{\b Q\b x}(t)<-b<0 \quad \mbox{for all }t>t_0$, we obtain for $t>t_0$,
\begin{eqnarray*}
{\b x}(t)^{\top}{\b P\b x}(t)&=&{\b x}(t_0)^{\top}{\b P\b x}(t_0)+\int_{t_0}^{t}(-{\b x}(s)^{\top}{\b Q\b x}(s))ds \\
&<&{\b x}(t_0)^{\top}{\b P\b x}(t_0)-b(t-t_0).
\end{eqnarray*}
The above implies that for a sufficiently large $t$ we would have ${\b x}(t)^{\top}{\b P\b x}(t)<0$, which contradicts the assumption that ${\b x}(t)^{\top}{\b P\b x}(t)>0$. This means that ${\b A}$ must be asymptotically stable, and the proof is complete.\\
\hspace*{5.5in} $\Box$
\begin{example} \rm
Let
\[
{\b A}=\left[\begin{array}{cc}
0 & 1\\
-1 & -2
\end{array}\right],
\]
and $\b Q=\b I_2$. We solve the Lyapunov equation $\b A^{\top}\b P+\b P\b A=-\b Q$ for $\b P$. Assume $\b P$ to have the form,
\[
{\b P}=\left[ \begin{array}{cc}
p_1 & p_2\\
p_2 & p_3
\end{array}
\right]= \b P^{\top}.
\]
Then, we have
\begin{eqnarray*}
\b A^{\top}\b P+\b P\b A &=&\left[ \begin{array}{cc}
0 & -1\\
1 & -2
\end{array}
\right]\left[ \begin{array}{cc}
p_1 & p_2\\
p_2 & p_3
\end{array}
\right]+\left[ \begin{array}{cc}
p_1 & p_2\\
p_2 & p_3
\end{array}
\right]\left[ \begin{array}{cc}
0 & 1\\
-1 & -2
\end{array}
\right]\\
&=& \left[ \begin{array}{cc}
-2p_2 & -p_3+p_1-2p_2\\
-p_3+p_1-2p_2 & 2p_2-4p_3
\end{array}
\right]\\
&=&-\b Q=\left[ \begin{array}{cc}
-1 & 0\\
0 & -1
\end{array}
\right].
\end{eqnarray*}
Solving the corresponding linear equations yields
\[
\b P=\left[ \begin{array}{cc}
3/2 & 1/2\\
1/2 & 1/2
\end{array}
\right].
\]
The solution matrix $\b P$ is positive definite\index{positive definite}. Hence, the matrix $\b A$ must have its eigenvalues in the open left-half plane. Indeed, the eigenvalues of $\b A$ are both located at $-1$.
\end{example}\markex
\begin{defn}
A positive definite function $V({\b x})$ whose time derivative evaluated on the solutions of the system $\dot{\b x}={\b A\b x}$ is negative definite (or negative semi-definite) is called a Lyapunov function\index{Lyapunov function} for this system.
\end{defn}\markdefn

Lyapunov's theorem gives us a necessary and sufficient condition for asymptotic stability of linear time-invariant dynamical systems. To check if a given matrix ${\b A}$ is asymptotically stable\index{asymptotically stable}\index{stable!asymptotically} or not, it is enough to take an arbitrary r.s.p.d. matrix ${\b Q}$, solve the equation ${\b A}^{\top}{\b P+\b P\b A=-\b Q}$ for real symmetric ${\b P}$ and check if the obtained ${\b P}$ is positive definite, or not. If ${\b P} > 0$, then ${\b A}$ is asymptotically stable. If ${\b P}$ is not positive definite, then ${\b A}$ cannot be asymptotically stable. Because ${\b Q}$ can be an arbitrary r.s.p.d. matrix, ${\b Q = \b I}_n$, the $n \times n$ identity matrix, is a good choice. It is important to note that there would be no use, as the following example shows, in first choosing a matrix ${\b P}$ and then calculating ${\b Q}$ from the equation ${\b A}^{\top}{\b P+\b P\b A=-\b Q}$.
\begin{example} \rm
Let
\[
{\b A}=\left[ \begin{array}{cc}
-1 & 3\\
0 & -1
\end{array}
\right],
\]
which is asymptotically stable.
Let
\[
{\b P}=\left[ \begin{array}{cc}
1 & 0 \\
0 & 1
\end{array}
\right]
={\b I}_2 > 0.
\]
Then,
\[
{\b Q}=-({\b A}^{\top}{\b P+\b P\b A})=\left[ \begin{array}{cc}
2 & -3 \\
-3 & 2
\end{array}
\right]
\]
which is indefinite\index{indefinite}. Thus, unless ${\b Q}$ turns out to be definite, nothing can be inferred about asymptotic stability from the Lyapunov theorem if we start with ${\b P}$ and then calculate ${\b Q}$.
\end{example}\markex

We now give a geometric interpretation of the Lyapunov's theorem\index{Lyapunov's theorem!geometric interpretation}. First, note that the function
\[
\dot{V}=\frac{dV({\b x}(t))}{dt}
\]
can be expressed, using the chain rule, as
\[
\dot{V}=\nabla V^{\top} \dot{\b x}=\| \nabla V \|\, \| \dot{\b x} \| \cos \theta,
\]
where $\theta$ is the angle between the vectors $\nabla V$ and $\dot{\b x}$. If $\| \dot{\b x} \|=1$, then $\nabla V^{\top} \dot{\b x}$ can be viewed as the rate of increase of $V$ at $\b x$ in the direction $\dot{\b x}$. Thus, $\dot{V}<0$ if and only if, $90 \deg < \theta < 270 \deg$. In such a case the velocity vector $\dot{\b x}(t)$ points in the direction of decreasing $V$. We illustrate the above discussion in Figure~\ref{fig:geolyap}.
\begin{figure}
\centering
\includegraphics[scale=0.55]{geolyap.eps}
%\vspace{3.5in}
%\centerbmp{4in}{3in}{c:/mybook/geolyap.bmp}
%\hskip.8in\special{bmp:c:/mybook/geolyap.bmp x=4.66in y=3.5in}
\caption{Geometric interpretation of the Lyapunov theorem.}
\label{fig:geolyap}
\end{figure}

Our next theorem is very useful in numerical verification of stability; it is a variant of Theorem~\ref{Lyapunov}.
\begin{thm}
\label{thm:ver_lyap}
A matrix $\b A$ is asymptotically stable\index{asymptotic stability!Lyapunov equation} if and only if for any ${\b Q=\b C^{\top}\b C}$, $\b C\in \mb{\Bb R}^{p\times n}$, such that the pair $(\b A, \b C)$ is observable\index{Lyapunov equation!observability}, the solution ${\b P}$ to the Lyapunov matrix equation ${\b A}^{\top}\b P+\b P\b A=-\b C^{\top}\b C$ is positive definite.
\end{thm}
{\bf Proof}\quad We first prove the necessity condition $(\Rightarrow )$ for asymptotic stability\index{asymptotic stability!necessity condition}. If $\b A$ is asymptotically stable, that is, $\Re\lambda_i(\b A) <0$, then we will show that
\begin{equation}
\b P=\int_0^{\infty}e^{\b A^{\top}t}\b C^{\top}\b Ce^{\b At}dt
\label{eq:integsotl}
\end{equation}
is the symmetric positive definite solution to the Lyapunov equation. Indeed, the integrand of~(\ref{eq:integsotl}) is a sum of terms of the form
\[
t^ke^{\lambda_it},
\]
where $\lambda_i$ is the $i$-th eigenvalue of $\b A$. Since $\Re\lambda_i(\b A)<0$, the integral in~(\ref{eq:integsotl}) exists because the terms in $\b P$ do not diverge when $t \to \infty$. Indeed, let $-\alpha_i=\Re\lambda_i$, where $\alpha_i>0$, then
\[
\lim_{t\to \infty}t^ke^{-\alpha_i t}=\lim_{t \to \infty}\frac{t^k}{e^{\alpha_it}},
\]
Differentiating the numerator and denominator of the above $k$ times and using L'Hospital's rule gives
\begin{eqnarray}
\lim_{t \to \infty}\frac{t^k}{e^{\alpha_it}} &=& \lim_{t\to \infty}\frac{kt^{k-1}}{\alpha_ie^{\alpha_i t}}\nonumber\\
&=& \lim_{t\to \infty}\frac{k(k-1)t^{k-2}}{\alpha^2_ie^{\alpha_i t}}\nonumber\\
&\vdots& {}\nonumber\\
&=& \lim_{t\to \infty}\frac{k!}{\alpha^k_i e^{\alpha_i t}}\nonumber\\
&=&0.
\label{eq:lhospitwne}
\end{eqnarray}
Thus, by~(\ref{eq:lhospitwne}), the formula for $\b P$ given by~(\ref{eq:integsotl}) is well defined. Next, note that $\b P=\b P^{\top}$. The matrix $\b P$ is also positive definite because the pair $(\b A,\b C)$ is observable. Substituting the expression for $\b P$ into the Lyapunov equation and performing manipulations yields
\begin{eqnarray*}
\b A^{\top}\b P+\b P\b A &=&\int_0^{\infty}\b A^{\top}e^{\b A^{\top}t}\b C^{\top}\b Ce^{\b At}dt +\int_0^{\infty}e^{\b A^{\top}t}\b C^{\top}\b Ce^{\b At}\b A dt\\
&=& \int_0^{\infty}\frac{d}{dt}\left(e^{\b A^{\top}t}\b C^{\top}\b Ce^{\b At}\right)dt\\
&=& \left.e^{\b A^{\top}t}\b C^{\top}\b Ce^{\b At}\right\arrowvert^{\infty}_0\\
&=& -\b C^{\top}\b C.
\end{eqnarray*}

The sufficiency\index{asymptotic stability!sufficiency condition} part $(\Leftarrow )$ can be proven using the arguments of the proof of the sufficiency part of the Lyapunov theorem and noting that the observability\index{observability!not zero along nonzero solution} of the pair $(\b A,\b C)$ implies that $\b x(t)^{\top}\b C^{\top}\b C\b x(t)$ is not identically zero along any nonzero solution of $\dot{\b x}(t)=\b A\b x(t)$.\\
\hspace*{5.5in}$\Box$

We now show that if $\b A$ is asymptotically stable, then the Lyapunov equation~(\ref{eq:layamateqn}) has a unique solution for every $\b Q$. Uniqueness can be proven in many ways. We use the Kronecker product\index{Kronecker product} in our proof. Using the results of Section~\ref{sec:kroneckerpro}, we represent the Lyapunov equation~(\ref{eq:layamateqn})\index{Lyapunov equation!Kronecker product} as
\begin{equation}
\left(\b I_n\otimes \b A^{\top}+\b A^{\top}\otimes \b I_n \right)\mb{vec}(\b P)=-\mb{vec}(\b Q).
\label{eq:lyapinkron}
\end{equation}
The condition for a solution $s(\b P)$ of the above equation to be unique is
\[
\lambda_i(\b A^{\top})+\lambda_j(\b A) \ne 0,\quad i,j=1,2,\ldots,n.
\]
Because $\lambda_i(\b A^{\top})=\lambda_i(\b A)$, and $\b A$ is asymptotically stable by assumption, the condition $\lambda_i(\b A^{\top})+\lambda_j(\b A) \ne 0$ is met, and hence the Lyapunov equation has a unique solution\index{Lyapunov equation!uniqueness of solution}.

\section{Evaluating Quadratic Indices}
In this Section, we apply the Lyapunov theory to evaluate the quadratic indices\index{quadratic indices}
\[
J_r=\int_0^{\infty}t^r\b x(t)^{\top}\b Q\b x(t)dt,\quad r=0,1,\ldots
\]
subject to
\[
\dot{\b x}(t)=\b A\b x(t),\quad \b x(0)=\b x_0,
\]
where $\b Q=\b C^{\top}\b C$ such that the pair $(\b A,\b C)$ is observable, and the matrix $\b A$ is asymptotically stable. We reformulate the above problem as follows. Evaluate the quadratic performance indices\index{quadratic indices!performance indices}
\[
J_r=\int_0^{\infty}t^r\b y(t)^{\top}\b y(t)dt, \quad r=0,1,\ldots
\]
subject to
\begin{eqnarray*}
\dot{\b x}(t) &=& \b A\b x(t),\quad \b x(0)=\b x_0,\\
\b y(t) &=& \b C\b x(t),
\end{eqnarray*}
where the pair $(\b A,\b C)$ is observable, and the matrix $\b A$ is asymptotically stable.

To evaluate
\[
J_0=\int_0^{\infty}\b y(t)^{\top}\b y(t) dt=\int_0^{\infty}\b x(t)^{\top}\b C^{\top}\b C\b x(t) dt=\int_0^{\infty}\b x(t)^{\top}\b Q\b x(t) dt,
\]
we note that
\begin{equation}
\frac{d}{dt}\left(\b x(t)^{\top}\b P\b x(t)\right)=-\b x(t)^{\top}\b Q\b x(t),
\label{eq:deopwtedq}
\end{equation}
where $\b P$ and $\b Q=\b C^{\top}\b C$ satisfy the Lyapunov matrix equation, $\b A^{\top}\b P+\b P\b A=-\b Q$. Hence,
\begin{equation}
J_0=-\int_0^{\infty}\frac{d}{dt}\left(\b x(t)^{\top}\b P\b x(t)\right)dt.
\label{eq:jopeinwde}
\end{equation}
Integrating both sides of~(\ref{eq:deopwtedq}) with respect to $t$ yields
\begin{eqnarray*}
J_0 &=&-\int_0^{\infty}\frac{d}{dt}\left(\b x(t)^{\top}\b P\b x(t)\right)dt\\
&=&-\left.\left(\b x(t)^{\top}\b P\b x(t)\right)\right\arrowvert_0^{\infty}\\
&=&\lim_{t \to \infty}\left(-\b x(t)^{\top}\b P\b x(t)\right)+\b x(0)^{\top}\b P\b x(0)\\
&=&\b x(0)^{\top}\b P\b x(0)
\end{eqnarray*}
because by assumption $\b A$ is asymptotically stable and therefore, $\lim_{t \to \infty}\b x(t)=\b 0$ for all $\b x(0)$. Furthermore, $\b P=\b P^{\top}>0$, and thus $J_0>0$ for all $\b x(0)\ne \b 0$.

Repeating the above arguments leads to a similar expression for $J_1$. First observe that
\begin{eqnarray*}
\frac{d}{dt}\left(t\b x(t)^{\top}\b P\b x(t)\right)&=&\b x(t)^{\top}\b P\b x(t)+t\frac{d}{dt}\left(\b x(t)^{\top}\b P\b x(t)\right)\\
&=&\b x(t)^{\top}\b P\b x(t)-t\b x(t)^{\top}\b Q\b x(t),
\end{eqnarray*}
where $\b P$ satisfies the Lyapunov matrix equation, $\b A^{\top}\b P+\b P\b A=-\b Q$. Hence,
\begin{eqnarray}
J_1 &=& \int_0^{\infty}t\b x(t)^{\top}\b Q\b x(t)dt\nonumber\\
&=& \int_0^{\infty}\b x(t)^{\top}\b P\b x(t)dt - \int_0^{\infty}\frac{d}{dt}\left(t\b x(t)^{\top}\b P\b x(t)\right) dt.
\label{eq:joafopat}
\end{eqnarray}
Because $\b P=\b P^{\top}>0$, we can solve
\[
\b A^{\top}\b P _1+\b P _1\b A=-\b P
\]
for $\b P _1=\b P^{\top}_1>0$. The above equation can also be represented as
\begin{equation}
\frac{d}{dt}\left(\b x(t)^{\top}\b P _1\b x(t)\right)=-\b x(t)^{\top}\b P\b x(t).
\label{eq:analtrepop}
\end{equation}
Substituting~(\ref{eq:analtrepop}) into~(\ref{eq:joafopat}) yields
\begin{eqnarray*}
J_1 &=& - \int_0^{\infty}\frac{d}{dt}\left(\b x(t)^{\top}\b P _1\b x(t)\right) dt-\left.\left( t\b x(t)^{\top}\b P\b x(t)\right)\right\arrowvert_0^{\infty}\\
&=& \lim_{t\to \infty}\left(-\b x(t)^{\top}\b P_1\b x(t)\right)+\b x(0)^{\top}\b P_1\b x(0)-0 +0\\
&=&\b x(0)^{\top}\b P_1\b x(0).
\end{eqnarray*}

Using similar arguments, we can evaluate $J_r$, $r>1$, which involves successive solutions\index{successive solutions!of Lyapunov equations} of the Lyapunov matrix equations.

In many control problems a dynamical system model is given in terms of a transfer function rather than in terms of a system of differential equations. We discuss a method of converting a given rational function into a system of first order differential equations. After this process is completed, we use the Lyapunov theory to evaluate a performance index\index{performance index} of interest. Suppose we are given a rational function\index{rational function} of the form
\[
E(s)=\frac{b_{n-1}s^{n-1}+\cdots+b_1s+b_0}{s^n+a_{n-1}s^{n-1}+\cdots+a_1s+a_0},
\]
where $E(s)={\cal L}(e(t))$ is the Laplace transform\index{Laplace transform} of $e(t)$. We now show that $e(t)$ can be generated as the solution of the differential equation
\begin{equation}
\frac{d^ne(t)}{dt^n}+a_{n-1}\frac{d^{n-1}e(t)}{dt^{n-1}}+\cdots+a_1\frac{de(t)}{dt}+a_0e(t)=0,
\label{eq:noodewrho}
\end{equation}
subject to a set of initial conditions that we shall derive. We follow the method of Power and Simpson~\cite[pages~220--221]{power78}. We first take the Laplace transform of the differential equation and then rearrange the resulting terms in an appropriate matrix equation. Recall that
\begin{equation}
{\cal L}\left(\frac{d^ie(t)}{dt^i}\right) = s^iE(s)-s^{i-1}e(0)-s^{i-2}\frac{de(0)}{dt}-\cdots-\frac{d^{i-1}e(0)}{dt^{i-1}}.
\label{eq:laptroid}
\end{equation}
Applying the above to~(\ref{eq:noodewrho}) and comparing corresponding terms with $E(s)$ leads to the equation for the initial conditions,
\[
\left[\begin{array}{ccccc}
a_1 & a_2 & \cdots & a_{n-1}& 1\\
a_2 & a_3 & \cdots & 1 & 0\\
\vdots & \vdots&{}& \vdots & \vdots\\
a_{n-1}& 1& \cdots& 0 &0\\
1& 0 & \cdots & 0 & 0
\end{array}\right]\left[\begin{array}{c}
e(0)\\
\frac{de(0)}{dt}\\
\vdots\\
\frac{d^{n-2}e(0)}{dt^{n-2}}\\
\frac{d^{n-1}e(0)}{dt^{n-1}}
\end{array}\right]=\left[\begin{array}{c}
b_0\\
b_1\\
\vdots\\
b_{n-2}\\
b_{n-1}
\end{array}\right].
\]
The coefficient matrix in the above equation is nonsingular. Thus, there is a unique solution to this equation, that is, we can uniquely determine the initial conditions. We next define the state vector
\[
\b x=\left[\begin{array}{cccc}
e&\frac{de}{dt}&\cdots&\frac{d^{n-1}e}{dt^{n-1}}
\end{array}\right]^{\top},
\]
and represent the differential equation~(\ref{eq:noodewrho}) as
\[
\dot{\b x}=\b A\b x,
\]
where
\[
\b A=\left[\begin{array}{cccccc}
0 & 1 & 0 & \cdots & 0 & 0\\
0 & 0 & 1 & \cdots & 0 & 0\\
\vdots& \vdots& \vdots &{}& \vdots & \vdots\\
0 & 0 & 0 & \cdots & 0 & 1\\
-a_0 & -a_1 & -a_2 & \cdots & -a_{n-2} & -a_{n-1}
\end{array}\right].
\]
The vector of initial conditions is
\[
\b x(0) = \left[\begin{array}{c}
e(0)\\
\frac{de(0)}{dt}\\
\vdots\\
\frac{d^{n-2}e(0)}{dt^{n-2}}\\
\frac{d^{n-1}e(0)}{dt^{n-1}}
\end{array}\right]
= \left[\begin{array}{ccccc}
a_1 & a_2 & \cdots & a_{n-1}& 1\\
a_2 & a_3 & \cdots & 1 & 0\\
\vdots & \vdots&{}& \vdots & \vdots\\
a_{n-1}& 1& \cdots& 0 &0\\
1& 0 & \cdots & 0 & 0
\end{array}\right]^{-1}\left[\begin{array}{c}
b_0\\
b_1\\
\vdots\\
b_{n-2}\\
b_{n-1}
\end{array}\right].
\]
Note that $e=\left[\begin{array}{ccccc}
1 & 0 & \cdots & 0 & 0
\end{array}\right]\b x$.

The above procedure for converting a rational function description of a process into its equivalent state-space representation shows a relation between the zeros of the given rational function and the initial conditions of the associated state-space realization.

We will now use the above method of constructing a state-space realization of a given rational function in the following example that involves evaluation of a quadratic index.
\begin{example}\label{exam:erropt}\rm
Consider a block diagram of a dynamic control system model shown in Figure~\ref{fig:itse}. This system comes from Dorf and Bishop~\cite[page~257]{dorf98}. The values of the parameters are $K_1=0.5$, and $K_1K_2K_p=2.5$.
\begin{figure}
\centering
\includegraphics[scale=0.55]{itse.eps}
%\vspace{3.5in}
%\hskip.9in\special{bmp:c:/mybook/itse.bmp x=4.67in y=3.5in}
\caption{Block diagram of the optimized control system model of Example~\ref{exam:erropt}.}
\label{fig:itse}
\end{figure}
We shall use the Lyapunov theory to select the gain $K_3$ that minimizes the effect of the disturbance $u(t)={\cal L}^{-1}(U(s))$ in the sense of the integral of the square error (ISE) criterion. In other words, we wish to find $K_3$ that leads to minimization of the performance index
\[
J_0=\int_0^{\infty}c(t)^2dt,
\]
where $c(t)={\cal L}^{-1}(C(s))$. We assume a unit step disturbance and $R(s)=0$.

We first find a rational function that models $C(s)$. From the block diagram shown in Figure~\ref{fig:itse}, we obtain
\[
C(s)=U(s)-\frac{K_2}{s}\frac{K_1}{s+K_1K_3}K_pC(s).
\]
Hence
\[
C(s)=\frac{s+0.5K_3}{s^2+0.5K_3s+2.5}.
\]
The next step is to determine the state-space representation corresponding to $C(s)$. We use the method given before this example. We first form the differential equation for $c(t)$,
\[
\frac{d^2c(t)}{dt^2}+0.5K_3\frac{dc(t)}{dt}+2.5c(t)=0
\]
with initial conditions
\[
\left[\begin{array}{cc}
0.5 K_3 & 1\\
1 & 0
\end{array}\right]\left[\begin{array}{c}
c(0)\\
\frac{dc(0)}{dt}
\end{array}\right]=\left[\begin{array}{c}
0.5K_3\\
1
\end{array}\right].
\]
Solving the above equation yields
\[
\left[\begin{array}{c}
c(0)\\
\frac{dc(0)}{dt}
\end{array}\right]=\left[\begin{array}{c}
1\\
0
\end{array}\right].
\]
Let $x_1=c$, $x_2=\frac{dc}{dt}$. The differential equation in vector-matrix form corresponding to $C(s)$ is
\[
\left.\begin{array}{lll}
\dot{\b x}(t)&=&\left[\begin{array}{cc}
0 & 1\\
-2.5 & -0.5K_3
\end{array}\right]\b x(t),\quad \b x(0)=\left[\begin{array}{c}
1\\
0
\end{array}\right]\\
c(t)&=&\b c\b x(t),
\end{array}\right\}
\]
where $\b c=\left[\begin{array}{cc}
1& 0
\end{array}\right]$. We are now ready to evaluate the performance index $J_0$,
\[
J_0 = \int_0^{\infty}c(t)^2dt= \int_0^{\infty}\b x(t)^{\top}\left[\begin{array}{cc}
1 & 0\\
0 & 0
\end{array}\right]\b x(t)dt= \int_0^{\infty}\b x(t)^{\top}\b c^{\top}\b c\,\b x(t)dt,
\]
The pair $(\b A,\b c)$ is observable. Therefore, we can solve the Lyapunov equation
\[
\b A^{\top}\b P+\b P\b A=-\b c^{\top}\b c
\]
for $\b P$ to obtain
\[
\b P=\left[\begin{array}{cc}
\frac{K_3}{10}+\frac{1}{K_3} & \frac{1}{5}\\
\frac{1}{5} & \frac{2}{5K_3}
\end{array}\right].
\]
Hence
\[
J_0 = \b x(0)^{\top}\b P\b x(0)= \frac{K_3}{10} +\frac{1}{K_3}.
\]
Differentiating $J_0$ and equating the result to zero gives
\[
\frac{dJ_0}{dK_3} = \frac{1}{10}-\frac{1}{K^2_3}= 0.
\]
Selecting
\[
K_3=\sqrt{10},
\]
yields a positive definite $\b P$. Note that because
\[
\left.\frac{d^2J_0}{dK_3^2}\right\arrowvert_{K_3 =\sqrt{10}}=\left.\frac{2}{K^3_3}\right\arrowvert_{K_3=\sqrt{10}}>0,
\]
that is, the second order sufficiency condition for the minimum is satisfied, we conclude that $K_3=\sqrt{10}\approx 3.16$ minimizes $J_0$.

We now wish to determine the gain $K_3$ that leads to minimization of the effect of the disturbance $u(t)$ in the sense of the integral of time multiplied by the squared error (ITSE) criterion. That is, we wish to select $K_3$ to minimize the performance index
\[
J_1=\int_0^{\infty}tc(t)^2dt.
\]
We first solve the Lyapunov equation,
\[
\b A^{\top}\b P_1+\b P_1\b A=-\b P,
\]
to get
\[
\b P_1=\left[\begin{array}{cc}
\frac{2}{K^2_3}+\frac{K_3^2}{100} & \frac{1}{5}\left( \frac{1}{K_3}+\frac{K_3}{10}\right)\\
\frac{1}{5}\left( \frac{1}{K_3}+\frac{K_3}{10}\right) & \frac{4}{5K_3^2}+\frac{1}{25}
\end{array}\right].
\]
Hence
\[
J_1 = \b x(0)^{\top}\b P_1\b x(0)= \frac{2}{K^2_3}+\frac{K_3^2}{100}.
\]
Differentiating $J_1$ and equating the result to zero gives
\[
\frac{dJ_1}{dK_3} = -\frac{4}{K_3^3} + \frac{K_3}{50}= 0.
\]
We select $K_3=\sqrt[4]{200}$ to keep $\b P_1$ positive definite. For this choice of $K_3$ the second order sufficiency condition for the minimum of $J_1$ is satisfied. Thus, $K_3 \approx 3.76$ minimizes $J_1$.
\end{example}\markex

\section{Discrete-Time Lyapunov Equation}
We will now use the Lyapunov approach to the stability analysis of the discrete-time linear time-invariant systems modeled by
\begin{equation}
\b x(k+1)=\b A\b x(k),\quad \b x(k_0)=\b x_0.
\label{eq:disclitisy}
\end{equation}
By analogy with the continuous-time case, consider a quadratic positive definite form,
\[
V(\b x(k))=\b x(k)^{\top}\b P\b x(k).
\]
We evaluate the first difference of $V$, defined $\Delta V=V(\b x(k+1))-V(\b x(k))$, on the trajectories of~(\ref{eq:disclitisy}) to obtain
\begin{eqnarray}
\Delta V(\b x(k)) &=& V(\b x(k+1))-V(\b x(k))\nonumber\\
&=&\b x(k+1)^{\top}\b P\b x(k+1)-\b x(k)^{\top}\b P\b x(k)\nonumber\\
&=& \b x(k)^{\top}\b A^{\top}\b P\b A\b x(k)-\b x(k)^{\top}\b P\b x(k)\nonumber\\
&=& \b x(k)^{\top}\left(\b A^{\top}\b P\b A-\b P\right)\b x(k),
\label{eq:frstdiofv}
\end{eqnarray}
where we replaced $\b x(k+1)$ in the above with $\b A\b x(k)$. For the solutions of~(\ref{eq:disclitisy}) to decrease the ``energy function'' $V$, at each $k>k_0$, it is necessary and sufficient that $\Delta V(\b x(k))<0$, or equivalently, that for some positive definite $\b Q$,
\begin{equation}
\Delta V(\b x(k))=-\b x(k)^{\top}\b Q\b x(k).
\label{eq:delvwqpsd}
\end{equation}
Combining~(\ref{eq:disclitisy}) and~(\ref{eq:delvwqpsd}), we obtain the so called {\em discrete-time Lyapunov equation},
\begin{equation}
\fbox{$\displaystyle \b A^{\top}\b P\b A-\b P=-\b Q $}
\label{eq:dtlyaeqfltis}
\end{equation}
\begin{thm}\label{thm:dlme}
The matrix $\b A$ has its eigenvalues in the open unit disk if and only if for any positive definite $\b Q$ the solution $\b P$ to~(\ref{eq:dtlyaeqfltis}) is positive definite.
\end{thm}
{\bf Proof}\quad $(\Rightarrow )$ We can use the Kronecker product, and in particular relation~(\ref{eq:stack_three}) to represent the discrete-time Lyapunov equation as
\begin{equation}
\left(\b I_{n^2}-\b A^{\top}\otimes \b A^{\top} \right)\mb{vec}(\b P) = \mb{vec}(\b Q),
\label{eq:dtleuskpro}
\end{equation}
where $\otimes$ denotes the Kronecker product and $\mb{vec}(\cdot)$ is the stacking operator. It is easy to see that the above equation has a unique solution if and only if
\[
1-\lambda_i \lambda_j \ne 0,\quad\mb{for all}\quad i,j=1,2,\ldots,n,
\]
where $\{\lambda_i\}$ is the set of eigenvalues of $\b A$. By assumption, $|\lambda_i|<1$, which is a sufficient condition for~(\ref{eq:dtleuskpro}) and hence~(\ref{eq:dtlyaeqfltis}) to have a unique solution. The solution $\b P$ can be expressed as
\begin{equation}
\b P=\sum_{k=0}^{\infty}\left(\b A^{\top} \right)^k\b Q\b A^k.
\label{eq:solslme}
\end{equation}
The above expression is well defined because all eigenvalues of $\b A$ have magnitude (strictly) less than unity. Substituting~(\ref{eq:solslme}) into~(\ref{eq:dtlyaeqfltis}) gives
\begin{eqnarray*}
\b A^{\top}\b P\b A-\b P&=&\b A^{\top}\left(\sum_{k=0}^{\infty}\left(\b A^{\top} \right)^{k}\b Q\b A^{k}\right)\b A-\sum_{k=0}^{\infty}\left(\b A^{\top} \right)^k\b Q\b A^k\\
&=& \sum_{k=1}^{\infty}\left(\b A^{\top} \right)^k\b Q\b A^k-\sum_{k=1}^{\infty}\left(\b A^{\top} \right)^k\b Q\b A^k-\b Q\\
&=&-\b Q.
\end{eqnarray*}
Thus, indeed~(\ref{eq:solslme}) satisfies the discrete-time Lyapunov equation. Note that if $\b Q$ is symmetric so is $\b P$. If $\b Q$ is positive definite, so is $\b P$.

$(\Leftarrow )$ Let $\lambda_i$ be an eigenvalue of $\b A$ and $\b v_i$ be a corresponding eigenvector, that is, $\b A\b v_i=\lambda_i \b v_i$. Observe that $\lambda_i$ and $\b v_i$ may be complex. We denote by $\b v^*_i$ the complex conjugate transpose of $\b v_i$. Note that $\b v_i^*\b P\b v_i$ is a scalar. We will show that its complex conjugate equals itself, which means that $\b v_i^*\b P\b v_i$ is real. Indeed,
\[
\overline{\b v_i^*\b P\b v_i}= \b v_i^*\b P^*\b v_i=\b v_i^*\b P^{\top}\b v_i=\b v_i^*\b P\b v_i,
\]
where we used the fact that the complex conjugate transpose of a real matrix reduces to its transpose. Thus, $\b v_i^*\b P\b v_i$ is real for any complex $\b v_i$.

Premultiplying now~(\ref{eq:dtlyaeqfltis}) by $\b v^*_i$ and postmultiplying by $\b v_i$ gives
\begin{equation}
\b v_i^*\left(\b A^{\top}\b P\b A-\b P\right)\b v_i = -\b v_i^*\b Q\b v_i.
\label{eq:aftermuldle}
\end{equation}
The left-hand side of the above equality evaluates to
\begin{eqnarray*}
\b v_i^*\b A^{\top}\b P\b A\b v_i - \b v_i^*\b P\b v_i&=& \bar{\lambda}_i\b v_i^* \b P\b v_i\lambda_i-\b v_i^*\b P\b v_i\\
&=& \left(|\lambda_i|^2-1 \right)\b v_i^*\b P\b v_i.
\end{eqnarray*}
Comparing the above with the right-hand side of~(\ref{eq:aftermuldle}) yields
\[
\left(1-|\lambda_i|^2\right)\b v_i^*\b P\b v_i=\b v_i^*\b Q\b v_i>0
\]
because $\b Q$ is positive definite. Since $\b P$ is positive definite, we have to have $|\lambda_i|<1$. The proof is complete.\\
\hspace*{5.5in}$\Box$


We now concern ourselves with evaluating performance indices on the trajectories of~(\ref{eq:disclitisy}). Specifically, we wish to evaluate
\begin{equation}
K_r=\sum_{k=0}^{\infty}k^r\b x(k)^{\top}\b Q\b x(k),\quad r=0,1,
\label{eq:distiperind}
\end{equation}
where $\b Q=\b Q^{\top}>0$ subject to~(\ref{eq:disclitisy}). We assume that the matrix $\b A$ has its eigenvalues in the open unit disk, that is, the matrix $\b A$ is {\em convergent\/}. We first consider the case when $r=0$. For this case, $K_r$ becomes
\begin{equation}
K_0=\sum_{k=0}^{\infty}\b x(k)^{\top}\b Q\b x(k).
\label{eq:diperindkeo}
\end{equation}
Because the matrix $\b A$ is convergent, the solution $\b P$ to~(\ref{eq:dtlyaeqfltis}) is positive definite. Using~(\ref{eq:frstdiofv}), (\ref{eq:delvwqpsd}) and~(\ref{eq:disclitisy}), we write
\begin{eqnarray}
\b x(k)^{\top}\b Q\b x(k) &=& -\b x(k)^{\top}\left( \b A^{\top}\b P\b A-\b P\right)\b x(k)\nonumber\\
&=&\b x(k)^{\top}\b P\b x(k)-\b x(k+1)^{\top}\b P\b x(k+1)
\label{eq:rewridle}
\end{eqnarray}
Summing both sides of the above form $k=0$ to $k=\infty$ and using the fact that $\b x(\infty)=\b 0$ because the matrix $\b A$ is convergent, we obtain
\begin{equation}
K_0=\b x(0)^{\top}\b P\b x(0).
\label{eq:afsubsidls}
\end{equation}

We will now evaluate
\[
K_1=\sum_{k=0}^{\infty}k\b x(k)^{\top}\b Q\b x(k),
\]
subject to~(\ref{eq:disclitisy}). To proceed, we multiply both sides of~(\ref{eq:rewridle}) by $k$ to get
\[
k\b x(k)^{\top}\b Q\b x(k) =k\b x(k)^{\top}\b P\b x(k)-k\b x(k+1)^{\top}\b P\b x(k+1).
\]
Summing the above from $k=0$ to $k=\infty$ gives
\begin{eqnarray}
K_1 &=& \b x(1)^{\top}\b P\b x(1)-\b x(2)^{\top}\b P\b x(2)+2\b x(2)^{\top}\b P\b x(2)-2\b x(3)^{\top}\b P\b x(3)+\cdots\nonumber\\
&=& \sum_{k=0}^{\infty}\b x(k)^{\top}\b P\b x(k)-\b x(0)^{\top}\b P\b x(0).
\label{eq:subositocko}
\end{eqnarray}
Using the method we employed to find the value of $K_0$, we now evaluate the first term on the right-hand side of~(\ref{eq:subositocko}) to get
\[
\sum_{k=0}^{\infty}\b x(k)^{\top}\b P\b x(k)=\b x(0)^{\top}\b P_1\b x(0),
\]
where $\b P_1$ satisfies
\[
\b A^{\top}\b P_1\b A-\b P_1=-\b P.
\]
Taking the above into account, we represent~(\ref{eq:subositocko}) as
\[
K_1=\b x(0)^{\top}\left(\b P_1-\b P\right)\b x(0).
\]
Proceeding in a similar manner, we can find expressions for $K_r$, where $r>1$.


\section{Constructing Robust Linear Controllers}
In this section, we use the Lyapunov theorem to construct robust state-feedback controllers. One of the basic issues in the control of dynamic systems is the effect of uncertainties, or neglected nonlinearities, on the stability of the closed-loop system. A controller is viewed robust if it maintains stability for all uncertainties in an expected range. We consider a class of dynamic systems modeled by
\begin{equation}
\dot{\b x}(t) = \b A\b x(t)+\b B\b u(t) +\b f(t, \b x(t))+\b B\b h(t,\b x(t),\b u(t)),
\label{eq:uncersysmolic}
\end{equation}
where $\b x(t) \in \mb{\Bb R}^n$, $\b u(t) \in \mb{\Bb R}^m$, and the functions $\b h$ and $\b f$ model uncertainties, or nonlinearities, in the system. We refer to $\b h$ as the {\em matched uncertainty\/} because it affects the system dynamics via the input matrix $\b B$ in the same fashion as the input $\b u$ does. In other words, the uncertainty $\b h$ matches the system input $\b u$. The vector $\b f$ models the {\em unmatched uncertainty\/}. We assume that the uncertain elements $\b h$ and $\b f$ satisfy the following norm bounds:
\begin{enumerate}
\item $\|\b h(t, \b x, \b u)\| \le \gamma_h\|\b u\|+\alpha_h\|\b x\|$,
\item $\|\b f(t, \b x)\| \le \alpha_f\|\b x\|$,
\end{enumerate}
where $\gamma_h$, $\alpha_h$, and $\alpha_f$ are known nonnegative constants. We further assume that the matrix $\b A$ is asymptotically stable. If this is not the case, we apply a preliminary state-feedback controller $\b u=-\b K\b x$ such that $\b A-\b B\b K$ is asymptotically stable. Such a feedback exists provided that the pair $(\b A,\b B)$ is stabilizable. Our goal is to construct a linear state-feedback controller that would make the closed-loop system asymptotically stable for arbitrary $\b f$ and $\b h$ that satisfy the above norm bounds.
\begin{thm}\label{thm:layap_stabcon}
Suppose that $\b A$ is asymptotically stable, and $\b P=\b P^{\top} >0$ is the solution to the Lyapunov matrix equation $\b A^{\top}\b P+\b P\b A=-2\b Q$ for some $\b Q=\b Q^{\top} >0$. Suppose also that
\[
\alpha_f < \frac{\lambda_{\min}(\b Q)}{\lambda_{\max}(\b P)}\quad\mb{and}\quad \gamma_h < 1.
\]
Then, the state-feedback controller $\b u=-\gamma\b B^{\top}\b P\b x$, where
\[
\gamma > \frac{\alpha_h^2}{4\left(\lambda_{\min}(\b Q)-\alpha_f \lambda_{\max}(\b P) \right)(1-\gamma_h)}
\]
stabilizes the uncertain system model~(\ref{eq:uncersysmolic}) for arbitrary $\b f$ and $\b h$ that satisfy the norm bounds.
\end{thm}
{\bf Proof}\quad The time derivative of the positive definite function $V=\b x^{\top}\b P\b x$ evaluated on any trajectory of the closed-loop system is
\begin{eqnarray*}
\dot{V} &=& 2\b x^{\top}\b P\dot{\b x}\\
&=& -2\b x^{\top}\b Q\b x -2\gamma\b x^{\top}\b P\b B\b B^{\top}\b P\b x +2\b x^{\top} \b P\b f+2\b x^{\top} \b P\b B\b h.
\end{eqnarray*}
Our goal is to determine $\tilde{\gamma} >0$ such that if $\gamma > \tilde{\gamma}$, then $\dot{V}<0$, which in turn implies the asymptotic stability of the closed-loop system. To proceed further, recall that if $\b Q$ is a symmetric matrix, then
\[
\lambda_{\min}(\b Q)\|\b x\|^2 \le \b x^{\top}\b Q\b x \le \lambda_{\max}(\b Q)\|\b x\|^2,
\]
and therefore
\[
-\b x^{\top}\b Q \b x \le -\lambda_{\min}(\b Q)\|\b x\|^2.
\]
Furthermore,
\[
\b x^{\top}\b P\b B\b B^{\top}\b P\b x =\b x^{\top}\b P\b B\left( \b x^{\top}\b P\b B\right)^{\top}=\|\b x^{\top}\b P\b B\|^2.
\]
For a symmetric positive definite matrix $\b P$, its induced 2-norm is
\[
\|\b P\|=\lambda_{\max}(\b P).
\]
Taking the above relations into account, using the norm bounds on the uncertain elements, and performing some manipulations yields
\begin{eqnarray}
\dot{V} &\le& -2\lambda_{\min}(\b Q)\|\b x\|^2-2\gamma\|\b x^{\top}\b P\b B\|^2+2\alpha_f\lambda_{\max}(\b P)\|\b x\|^2\nonumber\\
&{}&\quad+2\|\b x^{\top}\b P\b B\|\left( \alpha_h\|\b x\|+\gamma_h \gamma\|\b x^{\top}\b P\b B\|\right).
\label{eq:vdotless}
\end{eqnarray}
Let
\begin{equation}
\tilde{\b Q}=\left[\begin{array}{cc}
\lambda_{\min}(\b Q)-\alpha_f\lambda_{\max}(\b P) & -\frac{\alpha_h}{2}\\
-\frac{\alpha_h}{2} & \gamma(1-\gamma_h)
\end{array}\right]\in \mb{\Bb R}^{2 \times 2}.
\label{eq:big_matrix}
\end{equation}
Then, we can represent~(\ref{eq:vdotless}) as
\begin{equation}
\dot{V} \le -2\left[\begin{array}{cc}
\|\b x\| & \|\b x^{\top}\b P\b B\|
\end{array}\right]\tilde{\b Q}\left[\begin{array}{c}
\|\b x\|\\
\|\b x^{\top}\b P\b B\|
\end{array}\right].
\label{eq:vdotq}
\end{equation}
By Theorem~\ref{thm:psosdeftests} on page~\pageref{thm:psosdeftests}, for $\dot{V}$ to be negative it is enough that the leading principal minors of the matrix $\tilde{\b Q}$ given by~(\ref{eq:big_matrix}) be positive. The first-order leading principal minor is positive by assumption. For the second-order leading principal minor to be positive it is sufficient that $\gamma_h < 1$ and
\[
\gamma > \frac{\alpha_h^2}{4\left(\lambda_{\min}(\b Q)-\alpha_f \lambda_{\max}(\b P) \right)(1-\gamma_h)}.
\]
The proof is now complete.\\
\hspace*{5.5in} $\Box$

Note that if $\gamma_h=0$, then the closed-loop system is asymptotically stable if
\[
\alpha_f < \frac{\lambda_{\min}(\b Q)}{\lambda_{\max}(\b P)}\quad\mb{and}\quad \gamma > \frac{\alpha_h^2}{4\left(\lambda_{\min}(\b Q)-\alpha_f \lambda_{\max}(\b P) \right)}.
\]
\begin{example}\rm
Consider a dynamic system model
\begin{eqnarray*}
\dot{\b x} &=& \b A\b x+\b b(u+h(\b x,u))\\ &=&\left[\begin{array}{cc}
-2 & 0\\
0 & -1
\end{array}\right]\b x+\left[\begin{array}{c}
1\\
1
\end{array}\right](u+h(\b x,u)).
\end{eqnarray*}
Let $\b P$ be the solution of the matrix Lyapunov equation $\b A^{\top}\b P+\b P\b A=-2\b I_2$. We will design a linear state-feedback controller $u=-\gamma \b b^{\top}\b P\b x$ so that the origin of the closed-loop system is uniformly asymptotically stable for any $h(\b x,u)$ such that
\begin{equation}
|h(\b x,u)|\le 4\|\b x\|+\frac{1}{2}|u|.
\label{eq:nobonhinex}
\end{equation}
In particular, we will find a bound $\tilde{\gamma}>0$ so that for any $\gamma>\tilde{\gamma}$ the closed-loop system is uniformly asymptotically stable in the face of uncertainties satisfying the norm bound~(\ref{eq:nobonhinex}).

Solving the Lyapunov equation $\b A^{\top}\b P+\b P\b A=-2\b I_2$ yields
\[
\b P=\left[\begin{array}{cc}
1/2 & 0\\
0 & 1
\end{array}\right].
\]
Hence,
\[
u = -\gamma\b b^{\top}\b P\b x= -\gamma\left(\frac{1}{2}x_1+x_2\right).
\]
We obtain a bound on $\gamma$ by evaluating $\frac{d}{dt}\left( \b x^{\top}\b P\b x\right)$ on the trajectories of the closed-loop system. First, we evaluate
\begin{eqnarray*}
\frac{d}{dt}\left( \b x^{\top}\b P\b x\right)&=& 2\b x^{\top}\b P\dot{\b x}\\
&=& 2\b x^{\top}\b P\left( \b A\b x+\b b(u+h(\b x,u))\right)\\
&=& -2\b x^{\top}\b x-2\gamma\b x^{\top}\b P\b b\b b^{\top}\b P\b x+2\b x^{\top}\b P\b b h.
\end{eqnarray*}
Taking the norms and performing manipulations yields
\begin{eqnarray*}
\frac{d}{dt}\left( \b x^{\top}\b P\b x\right)&\le& -2\b x^{\top}\b x-2\gamma\left|\b x^{\top}\b P\b b\right|^2+2\left| \b x^{\top}\b P\b b\right|\left( 4\|\b x\|+\frac{1}{2}\gamma\left| \b x^{\top}\b P\b b\right|\right)\\
&=&-2\|\b x\|^2-\gamma\left|\b x^{\top}\b P\b b\right|^2+8\left|\b x^{\top}\b P\b b\right|\|\b x\|\\
&=&-2\left[\begin{array}{cc}
\|\b x\| & \left|\b x^{\top}\b P\b b\right|
\end{array}\right]\left[\begin{array}{cc}
1 & -2\\
-2 & \gamma/2
\end{array}\right]\left[\begin{array}{c}
\|\b x\|\\
\left|\b x^{\top}\b P\b b\right|
\end{array}\right].
\end{eqnarray*}
For $\frac{d}{dt}\left( \b x^{\top}\b P\b x\right)$ to be negative definite it is enough that the matrix
\[
\left[\begin{array}{cc}
1 & -2\\
-2 & \gamma/2
\end{array}\right]
\]
be positive definite. This is the case whenever
\[
\gamma > \tilde{\gamma} = 8.
\]
\end{example}\markex
\section{Hurwitz and Routh Stability Criteria}
\label{sec:routhhurwitz}
In this section, we present two tests for determining if the zeros of an $n$-th degree polynomial are located in the open left-half complex plane. In deriving these tests, we will use the Lyapunov matrix equation given by~(\ref{eq:layamateqn}). We consider the $n$-th degree polynomial
\begin{equation}
p(s)=a_n s^n+a_{n-1} s^{n-1}+\cdots+a_1 s+a_0
\label{eq:polyhurrou}
\end{equation}
with real coefficients. Associated with the polynomial $p(s)$ is the $n\times n$ real Hurwitz matrix,
\begin{equation}
\b H=\left[\begin{array}{ccccc}
a_{n-1} & a_{n-3} & a_{n-5} & a_{n-7} & \cdots\\
a_n & a_{n-2} & a_{n-4} & a_{n-6} & \cdots\\
0 & a_{n-1}& a_{n-3} & a_{n-5} & \cdots\\
0 & a_{n} & a_{n-2} & a_{n-4} & \cdots\\
\vdots&{}&{}&\vdots&\ddots
\end{array}\right].
\label{eq:hurmatnbn}
\end{equation}
In our further discussion we assume, without loss of generality, that $a_n=1$. The leading principal minors of the Hurwitz matrix are
\begin{equation}
H_1 = a_{n-1},\; H_2=\det \left[\begin{array}{cc}
a_{n-1} & a_{n-3}\\
1 & a_{n-2}
\end{array}\right],\; H_3=\det \left[\begin{array}{ccc}
a_{n-1} & a_{n-3}& a_{n-5}\\
1 & a_{n-2}& a_{n-4}\\
0 & a_{n-1} & a_{n-3}
\end{array}\right],\ldots ,H_n=\det \b H.
\label{eq:hurleaprmi}
\end{equation}
We next define
\[
b_1=H_1,\quad b_2=\frac{H_2}{H_1},\quad b_3=\frac{H_3}{H_2H_1},
\]
and
\[
b_k=\frac{H_{k-3}H_k}{H_{k-2}H_{k-1}}\quad \mb{for}\quad k=4,5,\ldots,n.
\]
Thus, for example, $b_4=\frac{H_1H_4}{H_2H_3}$. Let
\[
\b B=\left[\begin{array}{cccccccc}
0 & 1 & 0 &0&\cdots& 0 & 0&0\\
-b_n & 0 & 1 &0& \cdots& 0 & 0&0\\
0 & -b_{n-1}& 0 & 1 & \cdots & 0 & 0&0\\
\vdots& {} & {}& {}&{}&{}&{}&\vdots\\
0&0&0&0&\cdots&-b_3&0&1\\
0&0&0&0&\cdots&0&-b_2&-b_1
\end{array}\right]\in \mb{\Bb R}^{n \times n}.
\]
For example, for $n=4$, we have
\[
\b B=\left[\begin{array}{cccc}
0 & 1 & 0 & 0\\
-b_4 & 0 & 1 & 0\\
0 & -b_3 & 0 & 1\\
0 & 0 & -b_2 & -b_1
\end{array}\right].
\]
Let
\[
\b A=\left[\begin{array}{cccccc}
0 & 1 & 0 &\cdots & 0 & 0\\
0 & 0 & 1& \cdots & 0 & 0\\
\vdots& {}& {}& {}& {}& \vdots\\
0 & 0 & 0 & \cdots & 0 & 1\\
-a_0& -a_1 & -a_2&\cdots&-a_{n-2}&-a_{n-1}
\end{array}\right]\in \mb{\Bb R}^{n \times n}.
\]
\begin{lemma}
The matrices $\b B$ and $\b A$ are similar, that is, there is a nonsingular matrix $\b T\in \mb{\Bb R}^{n \times n}$ such that
\begin{equation}
\b B=\b T^{-1}\b A\b T.
\label{eq:simtrhur}
\end{equation}
\end{lemma}
{\bf Proof}\quad The similarity matrix is lower triangular and has the form
\begin{equation}
\b T=\left[\begin{array}{cccccccc}
1 & 0 & 0 & 0&0&0&\cdots & 0\\
0 & 1 & 0 & 0 & 0&0&\cdots & 0\\
t_{31}& 0 & 1&0&0&0&\cdots&0\\
0&t_{42}&0&1&0&0&\cdots&0\\
t_{51}&0&t_{53}&0&1&0&\cdots&0\\
0&t_{62}&0&t_{64}&0&1&\cdots&0\\
\vdots&{}&{}&{}&{}&{}&{}&\vdots
\end{array}\right].
\label{eq:tramaelo}
\end{equation}
To determine the nonfixed coefficients $t_{ij}$ of the matrix $\b T$, we represent~(\ref{eq:simtrhur}) as
\begin{equation}
\b T\b B=\b A\b T,
\label{eq:tbeatmeq}
\end{equation}
and compare both sides of~(\ref{eq:tbeatmeq}) to determine the coefficients $t_{ij}$. We obtain, for example, $t_{31}=-b_n$, $t_{42}=-b_n-b_{n-1}$, etc.\\
\hspace*{5.5in} $\Box$

Let $\b P$ be an $n\times n$ diagonal matrix defined as
\begin{equation}
\b P=\mb{diag}\left[\begin{array}{ccccc}
b_1b_2\times\cdots\times b_n & b_1b_2\times\cdots\times b_{n-1} & \cdots  & b_1b_2 & b_1
\end{array}\right],
\label{eq:mapfohuthm}
\end{equation}
and $\b Q$ be an $n\times n$ diagonal matrix defined as
\[
\b Q=\mb{diag}\left[\begin{array}{ccccc}
0 & 0 & \cdots & 0 & 2 b_1^2
\end{array}\right].
\]
We represent the matrix $\b Q$  as $\b Q=\b c^{\top}\b c$, where
\[
\b c=\left[\begin{array}{ccccc}
0 & 0 & \cdots & 0 & \sqrt{2} b_1
\end{array}\right].
\]
Note that the pair $\left( \b B,\b c\right)$ is observable. The matrices $\b B$, $\b P$, and $\b Q$ satisfy the Lyapunov matrix equation,
\begin{equation}
\b B^{\top}\b P+\b P\b B=-\b Q.
\label{eq:lyapwbpqhu}
\end{equation}
By Theorem~\ref{thm:ver_lyap}, on page~\pageref{thm:ver_lyap}, applied to~(\ref{eq:lyapwbpqhu}), the matrix $\b B$ is asymptotically stable if and only if the symmetric matrix $\b P$ is positive definite. By Theorem~\ref{thm:psosdeftests}, the matrix $\b P$ is positive definite if and only if its leading principal minors are positive. Using the above results we can prove the following theorem:
\begin{thm}[Hurwitz]
The real polynomial $p(s)$ of degree $n$ with positive coefficients $a_i$, has its zeros in the open left-half plane if and only if the leading principal minors of the Hurwitz matrix are positive.
\end{thm}
{\bf Proof}\quad The polynomial $p(s)$ has its zeros in the open left plane if and only if the matrix $\b B$ is asymptotically stable. This in turn is true if and only if the diagonal elements of $\b P$, given by~(\ref{eq:mapfohuthm}), are positive. We have
\[
b_1=H_1,\quad b_1b_2=H_2,
\]
and for $k=3,\ldots,n$,
\[
b_1b_2\times\cdots\times b_k=\frac{H_k}{H_{k-2}}.
\]
Thus, the diagonal elements of $\b P$ are all positive, if and only if the leading principal minors of the Hurwitz matrix are positive, which completes the proof.\\
\hspace*{5.5in} $\Box$

We will now prove the Routh necessary and sufficient condition for the real polynomial~(\ref{eq:polyhurrou}) to have its zeros in the open left-half plane. We first use the polynomial coefficients to form the following $(n+1)$-row Routh array:
\begin{equation}
\begin{array}{cc|ccccc}
s^n & \mb{row} & a_n&a_{n-2}&a_{n-4}& a_{n-6}& \cdots\\
s^{n-1} & \mb{row} & a_{n-1}&a_{n-3}&a_{n-5}& a_{n-7}& \cdots\\
s^{n-2} & \mb{row} & r_{21}&r_{22}&r_{23}& r_{24}& \cdots\\
s^{n-3} & \mb{row} & r_{31}&r_{32}&r_{33}& r_{34}& \cdots\\
\vdots&{}&\vdots&{}&{}&\vdots&{}\\
s^0&\mb{row}& r_{n1}&{}&{}&{}&{}
\end{array}
\label{eq:rouhurarr}
\end{equation}
The elements that comprise the $s^n$ and $s^{n-1}$ rows are appropriately ordered coefficients of $p(s)$. The elements of the $s^{n-2}$ row are formed from its preceding two rows as follows,
\begin{eqnarray*}
r_{21} &=& -\frac{1}{a_{n-1}}\det\left[\begin{array}{cc}
a_n & a_{n-2}\\
a_{n-1} & a_{n-3}
\end{array}\right]=\frac{a_{n-1}a_{n-2}-a_na_{n-3}}{a_{n-1}}\\
r_{22} &=& -\frac{1}{a_{n-1}}\det\left[\begin{array}{cc}
a_n & a_{n-4}\\
a_{n-1} & a_{n-5}
\end{array}\right]=\frac{a_{n-1}a_{n-4}-a_na_{n-5}}{a_{n-1}}\\
r_{23} &=& -\frac{1}{a_{n-1}}\det\left[\begin{array}{cc}
a_n & a_{n-6}\\
a_{n-1} & a_{n-7}
\end{array}\right]=\frac{a_{n-1}a_{n-6}-a_na_{n-7}}{a_{n-1}}\\
\vdots&{}&\qquad\vdots
\end{eqnarray*}
The elements of the $s^{n-3}$ row of the Routh array are generated using its preceding two rows,
\begin{eqnarray*}
r_{31} &=& -\frac{1}{r_{21}}\det\left[\begin{array}{cc}
a_{n-1} & a_{n-3}\\
r_{21} & r_{22}
\end{array}\right]=\frac{r_{21}a_{n-3}-r_{22}a_{n-1}}{r_{21}}\\
r_{32} &=& -\frac{1}{r_{21}}\det\left[\begin{array}{cc}
a_{n-1} & a_{n-5}\\
r_{21} & r_{23}
\end{array}\right]=\frac{r_{21}a_{n-5}-r_{23}a_{n-1}}{r_{21}}\\
r_{33} &=& -\frac{1}{r_{21}}\det\left[\begin{array}{cc}
a_{n-1} & a_{n-7}\\
r_{21} & r_{24}
\end{array}\right]=\frac{r_{21}a_{n-7}-r_{24}a_{n-1}}{r_{21}}\\
\vdots&{}&\qquad\vdots
\end{eqnarray*}
Subsequent rows of the array are formed from their preceding rows in an analogous fashion. Undefined positions can be filled with zeros. The last two rows of the array will have one term each. We will now formulate the stability test that in some texts (for example, in \cite{rohrs93} or \cite{wolovich94}) is also called the Routh-Hurwitz criterion.
\begin{thm}[Routh]
The zeros of the polynomial $p(s)=a_n s^n+a_{n-1} s^{n-1}+\cdots+a_1 s+a_0$, whose coefficients are all non-zero and have the same sign, are all in the open left-half plane if and only if the elements of the first column of the associated Routh array all have the same sign and are nonzero.
\end{thm}
{\bf Proof}\quad Without loss of generality, we assume that the polynomial coefficients are all positive. If they are negative, we can multiply both sides of the equation $p(s)=0$ by $-1$ thus making the coefficients positive. We then perform elementary row operations on the associated Hurwitz matrix, $\b H$. These operations are equivalent to premultiplying the Hurwitz matrix by lower triangular matrices with ones on the diagonal. For example,
\[
\left[\begin{array}{ccccc}
1& 0 & 0&0&\cdots\\
-a_n/a_{n-1}& 1& 0&0&\cdots\\
0&0&1&0&\cdots\\
\vdots&{}&{}&\ddots&\vdots\\
0&0&0&0&1
\end{array}\right]\left[\begin{array}{cccc}
a_{n-1} & a_{n-3} & a_{n-5} & a_{n-7}\\
a_n & a_{n-2} & a_{n-4} & a_{n-6} \\
0 & a_{n-1}& a_{n-3} & a_{n-5} \\
0 & a_{n} & a_{n-2} & a_{n-4} \\
\vdots&{}&\vdots&\ddots
\end{array}\right]=\left[\begin{array}{cccc}
a_{n-1} & a_{n-3} & a_{n-5} & a_{n-7}\\
0 & r_{21} & r_{22} & r_{23} \\
0 & a_{n-1}& a_{n-3} & a_{n-5} \\
0 & a_{n} & a_{n-2} & a_{n-4} \\
\vdots&{}&\vdots&\ddots
\end{array}\right].
\]
Let $\b U$ be the product of the matrices corresponding to the above described elementary operations on $\b H$. Then,
\[
\b U=\left[\begin{array}{cccc}
1&0 &\cdots &0\\
u_{12}& 1& \cdots & 0\\
\vdots& {}&\ddots& \vdots\\
u_{n1}&u_{n2}&\cdots&1
\end{array}\right],
\]
where, for example, $u_{12}=-a_n/a_{n-1}$. Premultiplying the Hurwitz matrix~(\ref{eq:hurmatnbn}) by the matrix $\b U$ given above yields
\begin{equation}
\b U\b H=\left[\begin{array}{ccccc}
a_{n-1} & a_{n-3} & a_{n-5} & \cdots & \cdots\\
0& r_{21} & r_{22} & \cdots & r_{2,n-1}\\
0 & 0 & r_{31} & \cdots & r_{3,n-2}\\
\vdots&{}&\vdots&\ddots&\vdots\\
0&0&0&\cdots&r_{n1}
\end{array}\right].
\label{eq:multbyhum}
\end{equation}
Because the matrix $\b U$ is lower triangular with the ones on the diagonal, the leading principal minors of $\b H$ and $\b U\b H$ are the same, where
\[
H_1=a_{n-1},\quad H_2=a_{n-1}r_{21},\quad\ldots\quad ,H_i=a_{n-1}r_{21}\times\cdots\times r_{i1},\quad i=1,2,\ldots,n.
\]
Hence,
\[
a_{n-1}=H_1,\quad\mb{and}\quad r_{i1}=\frac{H_i}{H_{i-1}}\quad\mb{for}\quad i=2,\dots,n.
\]
By the Hurwitz theorem, the zeros of the polynomial $p(s)$ are in the open left-half plane if and only if the coefficients $r_{i1}$, $i=1,2\ldots,n$, which are the elements of the first column of the Routh array, are all positive. The proof is complete.\\
\hspace*{5.5in} $\Box$

It is easy to verify that a necessary and sufficient condition for the first- and second-order polynomials to have their zeros in the open left-half plane is that all their coefficients be nonzero and have the same sign. However, this condition is only necessary and not sufficient for third and higher order polynomials to have all their zeros in the open left-half plane.

It is possible to extend the above Routh stability criterion so that it can be used to find the number of polynomial zeros with positive real parts. The number of zeros of $p(s)$ with positive real parts is equal to the number of sign changes in the first column of the Routh array. For a proof of this statement using the law of inertia of quadratic forms see, for example, Kaczorek~\cite[Section~8.4.3]{kaczorek93}.

\begin{example}\rm
We will use the Routh stability criterion to analyze stability of the linearized model of the flyball governor. The governor's linearized model is given by~(\ref{eq:linmoofwgo}) on page~\pageref{eq:linmoofwgo}. The model's characteristic equation is
\begin{eqnarray}
\det\left[\begin{array}{ccc}
s & -1 & 0\\
\frac{g \sin^2 x_{1e}}{\cos x_{1e}} & s+\frac{b}{m} & -\frac{2g\sin\,x_{1e}}{x_{3e}}\\
\frac{\kappa}{I}\sin x_{1e} & 0 & s
\end{array}\right] &=& s^3+\frac{b}{m}s^2+\frac{g\sin^2 x_{1e}}{\cos x_{1e}}s+\frac{2g\kappa \sin^2 x_{1e}}{I x_{3e}}\nonumber\\
&=&s^3+a_2s^2 + a_1 s+a_0.
\label{eq:chareqofwg}
\end{eqnarray}
The coefficients of the above third order polynomial are all positive. Therefore, the necessary condition for stability is satisfied. We next construct the Routh array for~(\ref{eq:chareqofwg}) to get,
\begin{equation}
\begin{array}{c|cc}
s^3& 1& a_1\\
s^2 &a_2 & a_0\\
s^1&\frac{1}{a_2}\left( a_2 a_1-a_0\right)\\
s^0& a_0
\end{array}
\label{eq:rharfwg}
\end{equation}
Because the characteristic polynomial coefficients are all positive, the linearized model is asymptotically stable if and only if
\begin{equation}
a_2 a_1-a_0=\frac{b}{m}\frac{g\sin^2 x_{1e}}{\cos x_{1e}}-\frac{2g\kappa \sin^2 x_{1e}}{I x_{3e}}>0.
\label{eq:stacofowg}
\end{equation}
Let
\[
\nu=\frac{x_{3e}}{2\kappa \cos x_{1e}}.
\]
Using the above notation, we express~(\ref{eq:stacofowg}) as
\begin{equation}
\frac{b I}{m}\nu>1.
\label{eq:stcoforwnf}
\end{equation}
It follows from condition~(\ref{eq:stcoforwnf}) that
\begin{enumerate}
\item an increase of the mass $m$ of the weights of the governor has a deteriorating effect on the governor's stability;
\item a decrease of the friction coefficient $b$ has also a harmful effect on stability;
\item a decrease of the moment of inertia $I$ of the flywheel harms stability;
\item a decrease of the coefficient $\nu$ harms stability.
\end{enumerate}
The above deductions are attributed by Pontryagin~\cite[page~219]{pontryagin62} to the Russian engineer Vyshnegradskiy. Vyshnegradskiy used these observations to propose, in~1876, design rules for governors to regulate engines without engine speeds fluctuating.
\end{example}\markex


\section{Stability of Nonlinear Systems}
\label{ch:stability}
In this section, we will be concerned with stability analysis of nonlinear time-varying dynamic system models described by~(\ref{eq:gensysmo}), that is, models of the form
\[
\dot{\b x}(t)=\b f(t,\b x(t)), \qquad \b x(t_0)=\mb{\bm$x$}_0,
%\label{eq:nondysys}
\]
where $\b x(t) \in \mb{\Bb R}^n$ is the state vector, and $\b f$ is a vector-valued function.

To proceed, we need the following lemma. Let $B_q(\b 0)$ denote an open ball with the center at $\b 0$ and radius $q$, and let ${\cal S}=\bar{B}_R(\b 0)\subset \mb{\Bb R}^n$ be a closed ball, a compact set, that contains the origin $\b x=\b 0$ in its interior.
\begin{lemma}
\label{lemma:con_kap}
For a continuously differentiable function $V : \mb{\Bb R}^n \to \mb{\Bb R}$, where $V(\mb{\bm$0$}) = 0$, we have $V( \mb{\bm$x$} ) > 0$ for all $\mb{\bm$x$} \in {\cal S} \setminus \{ \mb{\bm$0$} \}$, if and only if there is a continuous, strictly increasing, function $\psi : \mb{\Bb R} \to \mb{\Bb R}$ such that $\psi (0) = 0$, $\psi (r) > 0$ for $r>0$, and $V(\mb{\bm$x$}) \ge \psi (r)$, where $r=\| \mb{\bm$x$} \|$ and $r \le R$.
\end{lemma}
{\bf Proof}\quad Sufficiency $(\,\Leftarrow\,)$ is clear.

To prove necessity $(\, \Rightarrow \,)$, we define, following Willems~\cite[page~21]{willems70}, the function $\kappa : \mb{\Bb R} \to \mb{\Bb R}$,
\begin{equation}
\kappa (y) = \min_{\stackrel{\mb{\bm$x$} \in {\cal S}}{\| \mb{\bm$x$}\| = y}} V(\mb{\bm$x$} )
\label{eq:kappdefn}
\end{equation}
Note that the function $\kappa$ is well defined because the set
\begin{equation}
\{ \mb{\bm$x$} : \|\mb{\bm$x$} \| = y \} \cap {\cal S}
\label{eq:comsetforkapp}
\end{equation}
is compact and $V$ is continuous. Therefore, by the Weierstrass theorem on page~\pageref{thm:weierstrassgen}, the function $V$ achieves its infimum over the set~(\ref{eq:comsetforkapp}).

We next show that $\kappa$ is continuous on ${\cal S}=\bar{B}_R(\mb{\bm$0$})$. Then, we construct an appropriate function $\psi$ satisfying the hypotheses of the lemma. To show that $\kappa$ is continuous on ${\cal S}$, we use the fact that $V$ is uniformly continuous in ${\cal S}$. To this end let $\mb{\bm$x$}^{(1)} \in {\cal S}$ be such that $\|\mb{\bm$x$}^{(1)}\| = y$ and $\kappa (y) = V \left(\mb{\bm$x$}^{(1)} \right)$. Next, let $\Delta y$ be an increment of $y$ and let $\mb{\bm$x$}^{(2)} \in {\cal S}$ be a vector such that $y+\Delta y = \|\mb{\bm$x$}^{(2)}\|$. For example, we can take
\[
\b x^{(2)}=\b x^{(1)}+\frac{\b x^{(1)}}{\|\b x^{(1)}\|}\Delta y.
\]
Observe that
\[
V \left(\mb{\bm$x$}^{(2)} \right) \ge \kappa ( y + \Delta y).
\]
Choose now a vector $\mb{\bm$x$}^{(3)}$ such that
\[
\|\mb{\bm$x$}^{(3)}\| = y + \Delta y\quad \mb{and} \quad V\left(\mb{\bm$x$}^{(3)} \right)= \kappa (y + \Delta y).
\]
Note that for any vector, say $\mb{\bm$x$}^{(4)}$, such that $\|\mb{\bm$x$}^{(4)}\|=y$ we have $V\left( \mb{\bm$x$}^{(4)}\right) \ge \kappa(y)$---see Figure~\ref{fig:con_kap}. We can take
\[
\b x^{(4)}=\b x^{(3)}-\frac{\b x^{(3)}}{\|\b x^{(3)}\|}\Delta y.
\]
\begin{figure}
\centering
\includegraphics[scale=0.55]{con_kap.eps}
%\vspace{3in}
%\hskip1.25in\special{bmp:c:/mybook/con_kap.bmp x=4in y=3in}
\caption{A two-dimensional illustration of the proof of the continuity of the function $\kappa$ given by~(\ref{eq:kappdefn}).}
\label{fig:con_kap}
\end{figure}
By the uniform continuity of $V$, for any $\epsilon > 0$ there exists a $\delta > 0$ such that if
\[
\|\mb{\bm$x$}^{(1)}-\mb{\bm$x$}^{(2)}\| < \delta \quad \mb{and} \quad \|\mb{\bm$x$}^{(3)}-\mb{\bm$x$}^{(4)}\| < \delta,
\]
then
\[
\left\arrowvert V\left(\mb{\bm$x$}^{(1)} \right)-V\left(\mb{\bm$x$}^{(2)} \right)\right\arrowvert < \epsilon \quad \mb{and} \quad \left\arrowvert V\left(\mb{\bm$x$}^{(3)} \right)-V\left(\mb{\bm$x$}^{(4)} \right)\right\arrowvert < \epsilon.
\]
From $\left\arrowvert V\left(\mb{\bm$x$}^{(1)} \right)-V\left(\mb{\bm$x$}^{(2)} \right)\right\arrowvert < \epsilon$, we obtain
\[
-\epsilon < V\left(\mb{\bm$x$}^{(1)} \right)-V\left(\mb{\bm$x$}^{(2)} \right) < \epsilon.
\]
Thus
\[
V\left(\mb{\bm$x$}^{(1)} \right) > V\left(\mb{\bm$x$}^{(2)} \right) - \epsilon.
\]
From $\left\arrowvert V\left(\mb{\bm$x$}^{(3)} \right)-V\left(\mb{\bm$x$}^{(4)} \right)\right\arrowvert < \epsilon$, we get
\[
V\left(\mb{\bm$x$}^{(3)} \right)>V\left(\mb{\bm$x$}^{(4)} \right)- \epsilon.
\]
From the definition of the function $\kappa$ it follows that
\[
V \left( \mb{\bm$x$}^{(2)} \right) \ge V \left( \mb{\bm$x$}^{(3)} \right)\quad\mb{and}\quad V \left( \mb{\bm$x$}^{(4)} \right) \ge V \left( \mb{\bm$x$}^{(1)} \right).
\]
Because $V\left(\mb{\bm$x$}^{(1)} \right) > V\left(\mb{\bm$x$}^{(2)} \right) - \epsilon$, we obtain
\[
V\left(\mb{\bm$x$}^{(1)} \right) > V\left(\mb{\bm$x$}^{(3)} \right) - \epsilon
\]
and therefore
\begin{equation}
- \epsilon < V\left(\mb{\bm$x$}^{(1)} \right) - V\left(\mb{\bm$x$}^{(3)} \right).
\label{eq:miepsxoxt}
\end{equation}
Because $V\left(\mb{\bm$x$}^{(3)} \right) > V\left(\mb{\bm$x$}^{(4)} \right) - \epsilon$, we have
\[
V\left(\mb{\bm$x$}^{(3)} \right) > V\left(\mb{\bm$x$}^{(1)} \right) - \epsilon,
\]
and thus
\begin{equation}
V\left(\mb{\bm$x$}^{(1)} \right) - V\left(\mb{\bm$x$}^{(3)} \right) < \epsilon.
\label{eq:pluepsxost}
\end{equation}
Combining~(\ref{eq:miepsxoxt}) and~(\ref{eq:pluepsxost}), we get
\[
\left\arrowvert V\left(\mb{\bm$x$}^{(1)} \right) - V\left(\mb{\bm$x$}^{(3)} \right)\right\arrowvert < \epsilon,
\]
or equivalently,
\[
\left|\kappa(y)- \kappa(y + \Delta y)\right| < \epsilon
\]
if
\[
\Delta y < \delta,
\]
which proves the continuity of $\kappa$.


We are now ready to construct a function $\psi$ that satisfies the hypotheses of the lemma. This function comes from Willems~\cite[page~22]{willems70} and has the form
\begin{equation}
\fbox{$ \displaystyle \psi (r) = r \min _{r \le y \le R} \frac{\kappa(y)}{y} $}
\label{eq:funpsidefn}
\end{equation}
Note that $\psi$ is continuous, $\psi (0) = 0$, and it is positive and strictly increasing---see Figure~\ref{fig:fun_psi} for a graphical illustration of the way $\psi$ is constructed from $\kappa$.
\begin{figure}
\centering
\includegraphics[scale=0.55]{fun_psi.eps}
%\vspace{3in}
%\centerbmp{4in}{3in}{c:/mybook/fun_psi.bmp}
%\hskip1.5in\special{bmp:c:/mybook/fun_psi.bmp x=4in y=3in}
\caption{Constructing the function $\psi$ given by~(\ref{eq:funpsidefn}).}
\label{fig:fun_psi}
\end{figure}
The proof of the lemma is now complete.\\
\hspace*{5.5in}$\Box$


We can characterize a positive definite function $V$ in another way as in the following lemma.
\begin{lemma}
A continuously differentiable function $V : \mb{\Bb R}^n \to \mb{\Bb R}$ is positive definite in ${\cal S}=\bar{B}_R(\b 0)\subset \mb{\Bb R}^n$, with respect to $\b x=\b 0$, if and only if there exists a continuous nondecreasing function $\alpha : \mb{\Bb R} \to \mb{\Bb R}$, such that $\alpha (0) = 0$, $\alpha (r) > 0$ for $r>0$, and $V(\mb{\bm$x$}) \ge \alpha(r)$, where $r=\|\mb{\bm$x$}\|$ and $r \le R$.
\end{lemma}
A possible choice for $\alpha$ is
\[
\alpha(r)=\min_{r \le \|\mb{\bm$x$}\| \le R} V(\mb{\bm$x$}).
\]
Note that the function $\alpha$ is nondecreasing because as $r$ increases, the minimum is taken over a smaller region. Furthermore, $\alpha(r) > 0$ whenever $r>0$ because $V(\mb{\bm$x$}) > 0$ for $\mb{\bm$x$} \in {\cal S} \setminus \{\mb{\bm$0$}\}$. The function $\alpha$ is continuous because $V$ is continuous.

The main difference between the above two characterizations of positive definite functions is that in Lemma~\ref{lemma:con_kap} the function $\psi$ is strictly increasing rather than just nondecreasing. In our further discussion, we will exploit  extensively the existence of a such a function $\psi$ associated with a positive definite function $V$.


So far we only considered positive definite, or negative definite functions, that are time-invariant. We now extend our definition of positive definiteness to time-varying functions of the form $W(t,\mb{\bm$x$})$. We first define a time-varying positive semi-definite function. As before, we let ${\cal S}$ to denote a compact subset of $\mb{\Bb R}^n$ containing the origin $\b x=\b 0$ in its interior.
\begin{defn}
A real-valued function $W : \mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ is positive semi-definite, with respect to $\b x=\b 0$, in a region ${\cal S} \subseteq \mb{\Bb R}^n$ if for all $t$ and all $\mb{\bm$x$}$ in $\cal S$
\begin{itemize}
\item[(i)] $W$ is continuous and has continuous partial derivatives with respect to all of its arguments, that is, $W \in {\cal C}^1$,
\item[(ii)] $W(t,\mb{\bm$0$})=0$,
\item[(iii)] $W(t,\mb{\bm$x$}) \ge 0$.
\end{itemize}
\end{defn}\markdefn

\begin{defn}\label{defn:posdeffot}
A real-valued function $W : \mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ is positive definite, with respect to $\b x=\b 0$, in a region ${\cal S} \subseteq \mb{\Bb R}^n$ if for all $t$ and all $\mb{\bm$x$}$ in $\cal S$
\begin{itemize}
\item[(i)] $W \in {\cal C}^1$,
\item[(ii)] $W(t,\mb{\bm$0$})=0$,
\item[(iii)] there exists a positive definite time-invariant function $V$ such that $W(t,\mb{\bm$x$}) \ge V(\mb{\bm$x$})$.
\end{itemize}
\end{defn}\markdefn

We next introduce a notion of a {\em decrescent\/} function.
\begin{defn}
A time-varying function $W : \mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ is decrescent in a region $\cal S$ if a positive definite function $\tilde{V}$ exists such that for all $\mb{\bm$x$}$ in $\cal S$ and all $t$
\[
|W(t,\mb{\bm$x$})| \le \tilde{V}(\mb{\bm$x$}),
\]
that is, $W$ tends to zero uniformly with respect to $t$ as $\| \mb{\bm$x$} \| \to 0$.
\end{defn}\markdefn


Consider now an equilibrium state of the dynamic system modeled by~(\ref{eq:gensysmo}), that is, a constant state $\b x_e$ such that $\mb{\bm$f$}(t,\mb{\bm$x$}_{e})=\mb{\bm$0$}$ for all $t$. Let $W(t,\mb{\bm$x$})$ be a time-dependent real-valued function whose time derivative along the solution $\mb{\bm$x$}(t)=\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)$ of the given differential equation, is
\[
\frac{dW(t,\mb{\bm$x$})}{dt}=\dot{W}(t,\mb{\bm$x$})=\frac{\partial W}{\partial t}+\nabla W^{\top} \dot{\mb{\bm$x$}}.
\]
Then, we have the following theorem concerning the stability of the equilibrium point in the sense of Lyapunov.
\begin{thm}
\label{thm:loc_stab}
If in some neighborhood $\cal S$ of the equilibrium state $\mb{\bm$x$}_{e}$, there is a positive definite function $W$, with respect to $\b x_{e}$, such that its time derivative along any solution of $\dot{\mb{\bm$x$}}(t)=\mb{\bm$f$}(t,\mb{\bm$x$}(t))$ is negative semi-definite in $\cal S$, then the equilibrium state $\mb{\bm$x$}_{e}$ is stable. If $W$ is also decrescent, then the equilibrium state is uniformly stable.
\end{thm}
{\bf Proof}\quad Without loss of generality, we assume that $\mb{\bm$x$}_{e}=\mb{\bm$0$}$ and follow the arguments of Willems~\cite[pages~29,~30]{willems70}. Because $W$ is positive definite, by Definition~\ref{defn:posdeffot} and Lemma~\ref{lemma:con_kap} there is a strictly increasing function $\psi _1 (r)$, $\psi (0)=0$, such that
\[
W(t,\mb{\bm$x$}) \ge \psi _1 (\|\mb{\bm$x$}\|),
\]
where $\|\mb{\bm$x$}\| \le R$, and $R$ is the radius of the largest ball contained in $\cal S$. Let
\[
\psi _2 (t_0,r)= \max _{\|\mb{\bm$x$}\| \le r} W(t_0,\mb{\bm$x$}).
\]
Then
\[
\psi _1 (r) \le \psi _2 (t_0,r).
\]
Thus, for any $\epsilon > 0$ there is a positive $\delta$ that depends on $\epsilon$ and $t_0$, such that
\[
\psi _2 (t_0, \delta ) < \min \{ \psi _1 ( \epsilon ), \psi _1 (R) \}.
\]
Figure~\ref{fig:fin_del} illustrates how to determine a $\delta$ given an $\epsilon$.
\begin{figure}
\centering
\includegraphics[scale=0.55]{fin_del.eps}
%\vspace{3in}
%\centerbmp{3.33in}{2.5in}{c:/mybook/fin_del.bmp}
%\hskip1.5in\special{bmp:c:/mybook/fin_del.bmp x=4in y=3in}
\caption{Determining a $\delta$ given an $\epsilon$.}
\label{fig:fin_del}
\end{figure}
By assumption $dW(t,\b x)/dt \le 0$ along any solution of the system. Hence $W(t,\mb{\bm$x$})$ is a non-increasing function and therefore
\[
W(t,\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)) \le W(t_0,\mb{\bm$x$}_0) \qquad \mb{for all} \qquad t \ge t_0.
\]
If $\|\mb{\bm$x$}_0\| < \delta$, then for $t \ge t_0$ we have
\[
\psi _1(\epsilon)>\psi_2(t_0, \delta) \ge W(t_0,\mb{\bm$x$}_0) \ge W(t,\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)) \ge \psi_1(\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|).
\]
Therefore,
\[
\psi_1 (\epsilon) > \psi_1(\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|).
\]
Because $\psi_1$ is strictly increasing, we have to have
\[
\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\| < \epsilon
\]
for all $t \ge t_0$. This means that the null solution is (locally) stable, which completes the proof of the first part of the theorem.


If $W$ is also decrescent, then there is a nondecreasing function $\tilde{\psi}_2:\mb{\Bb R} \to \mb{\Bb R}$, $\tilde{\psi}_2(0)=0$, and
\[
W(t,\mb{\bm$x$}) \le \tilde{\psi}_2 (\|\mb{\bm$x$}\|)
\]
for all $\mb{\bm$x$}$ such that $\|\mb{\bm$x$}\| \le R$ and all $t$. Indeed, because $W(t,\b x)$ is positive definite and decrescent, there exists a positive definite $V(\b x)$ such that $W(t,\b x)\le V(\b x)$. Then, we can take $\tilde{\psi}_2$ defined as
\[
\tilde{\psi}_2(r)=\max_{\|\b x\|\le r}V(\b x).
\]
Let
\[
\tilde{\psi}_2 (\delta ) < \min \{\psi_1 (\epsilon ), \psi_1 ( R )\}.
\]
Note that $\delta$ depends now only on $\epsilon$, and not on $t_0$ as before. To show the uniform stability of the null solution, we repeat the argument above with $\tilde{\psi}_2$ replacing $\psi_2$. This completes the proof of the theorem.\\
\hspace*{5.5in} $\Box$


The next theorem gives us a sufficiency condition for an equilibrium point to be uniformly asymptotically stable.
\begin{thm}
\label{thm:uniform_as}
If in some neighborhood $\cal S$ of the given equilibrium point $\mb{\bm$x$}_{e}$ there is a positive definite, with respect to $\b x_{e}$, and decrescent function $W$ such that its time derivative along any solution of the system model~(\ref{eq:gensysmo}) is negative definite in $\cal S$, then the equilibrium state is uniformly asymptotically stable.
\end{thm}
{\bf Proof}\quad As before, without loss of generality, we assume that $\mb{\bm$x$}_{e}= \mb{\bm$0$}$ and then follow the arguments of Willems~\cite[pages~30,~31]{willems70}. The previous theorem ensures uniform stability of the equilibrium state $\mb{\bm$x$}_{e}$. Thus, only the uniform convergence remains to be shown. From the assumptions of the theorem it follows that there exist strictly increasing functions $\psi_1$ and $\psi_3$, and a non-decreasing function $\psi_2$ such that
\begin{eqnarray*}
\psi_1 (0) &=& \psi_2 (0) = \psi_3 (0) = 0,\\
\psi_1(\|\mb{\bm$x$}\|) &\le& W(t,\mb{\bm$x$}) \le \psi_2(\|\mb{\bm$x$}\|),\\
\psi_3(\|\mb{\bm$x$}\|) &\le& - \dot{W}(t,\mb{\bm$x$}),\end{eqnarray*}
for all $t$ and all $\mb{\bm$x$}$ such that $\|\mb{\bm$x$}\| \le R$, where $R$ is the radius of the largest ball contained in $\cal S$. First, select $\delta_1$ so that
\begin{equation}
\psi_2(\delta_1) < \psi_1(R).
\label{eq:p2p1r}
\end{equation}
Next, given an $\epsilon_1>0$, define positive constants $\delta_2$ and $T$ as follows:
\begin{eqnarray}
\psi_2(\delta_2) &<& \min \{\psi_1 (\epsilon_1 ), \psi_2 (\delta _1) \},\label{eq:p2de2mip1p2}\\
T&=&\psi_1 (R)/\psi_3(\delta _2).\label{eq:catp1pt}
\end{eqnarray}
Note that $\delta_1$ and $T$ are independent of $t_0$.

We prove the theorem in two stages. First, we show that for some $t_2 \in \left[ t_0,t_0+T\right]$,
\[
\|\mb{\bm$x$}(t_2;t_0,\mb{\bm$x$}_0)\| < \delta_2
\]
whenever $\|\mb{\bm$x$}_0 \|< \delta_1$. Then, we show that once $\|\mb{\bm$x$}(t_2;t_0,\mb{\bm$x$}_0) \| < \delta_2$, then
\[
\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0) \| < \epsilon_1 \quad \mb{for all} \quad t \ge t_2.
\]
We begin with proving the claim that for some $t_2 \in \left[ t_0,t_0+T\right]$,
\[
\|\mb{\bm$x$}(t_2;t_0,\mb{\bm$x$}_0)\| < \delta_2
\]
whenever $\|\mb{\bm$x$}_0 \|< \delta_1$. We prove this claim by contradiction. We assume that for all $t\in \left[t_0,t_0+T\right]$,
\begin{equation}
\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\| \ge \delta_2\quad\mb{if}\quad \|\mb{\bm$x$}_0 \|< \delta_1.
\label{eq:noxgedt}
\end{equation}
Using~(\ref{eq:noxgedt}), (\ref{eq:catp1pt}), and~(\ref{eq:p2p1r}), we obtain
\begin{eqnarray*}
0 < \psi_1 (\delta _2 ) &\le& \psi_1(\|\b x(t_0+T;t_0,\b x_0)\|)\\
&\le& W[t_0+T,\mb{\bm$x$}(t_0+T;t_0,\mb{\bm$x$}_0)]\\
&=& W(t_0,\mb{\bm$x$}_0) + \int_{t_0}^{t_0+T} \dot{W}[t,\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)]dt\\
&\le& W(t_0,\mb{\bm$x$}_0) - \int_{t_0}^{t_0+T} \psi_3(\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|)dt\\
&\le& W(t_0,\mb{\bm$x$}_0) - T\psi_3(\delta_2)\\
&\le& \psi_2(\|\mb{\bm$x$}_0\|)-T\psi_3(\delta_2)\\
&<& \psi_2(\delta_1)-\psi_1(R)\\
&<&0,
\end{eqnarray*}
which is a contradiction, and thus we proved the first part of the theorem. We now proceed with the second part. It follows from the first part that if $\|\mb{\bm$x$}_0\| < \delta_1$, then for some $t_2 \in \left[t_0,t_0+T\right]$, we have $\|\mb{\bm$x$}(t_2;t_0,\mb{\bm$x$}_0)\| < \delta_2$. Hence, for $t \ge t_2$,
\[
\psi_1(\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|) \le W\left(t,\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\right) \le W\left(t_2,\mb{\bm$x$}(t_2;t_0,\mb{\bm$x$}_0)\right) < \psi_2(\delta_2)<\psi_1(\epsilon_1).
\]
Therefore,
\[
\psi_1(\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|)<\psi_1(\epsilon_1),
\]
and because $\psi_1$ is strictly increasing,
\[
\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|< \epsilon_1
\]
for all $t \ge t_2$ which proves the uniform convergence. The proof of the theorem is thus complete.\\
\hspace*{5.5in} $\Box$
\begin{example}\rm
Given a dynamic system model
\[
\dot{\b x}(t)=\left[\begin{array}{cc}
-0.75 & 0.3536\\
0.3536 & -0.5
\end{array}\right]\b x(t), \quad \b x(0)=\b x_0.
\]
The solution $\b P=\b P^{\top}$ to the Lyapunov matrix equation $\b A^{\top}\b P+\b P\b A=-\b Q$, where $\b Q=2\b I_2$, is
\[
\b P=\left[\begin{array}{cc}
2 & \sqrt{2}\\
\sqrt{2} & 3
\end{array}\right].
\]
The eigenvalues of $\b P$ are
\[
\lambda_1=\lambda_{\min}(\b P)=1,\quad \lambda_2=\lambda_{\max}(\b P)=4.
\]
In general, for a symmetric matrix $\b P$,
\[
\lambda_{\min}(\b P)\|\b x\|^2 \le \b x^{\top}\b P\b x\le \lambda_{\max}(\b P)\|\b x\|^2.
\]
Therefore, we can take
\[
\psi_1(\|\b x\|)=\lambda_{\min}(\b P)\|\b x\|^2,\quad \psi_2(\|\b x\|)=\lambda_{\max}(\b P)\|\b x\|^2,\quad \mb{and}\quad \psi_3(\|\b x\|)=\lambda_{\min}(\b Q)\|\b x\|^2.
\]
In our example, $\lambda_{\min}(\b Q)=2$.
\begin{itemize}
\item Given $\varepsilon=4$. We will find ${\delta}>0$ such that if $\|\b x_0\|\le \delta$, then $\|\b x(t)\|\le\varepsilon$ for $t\ge 0$.\\
We obtain $\delta=\delta(\varepsilon)$ by solving the equation,
\[
\psi_1(\varepsilon)=\psi_2(\delta),
\]
to get
\[
\delta = \sqrt{\frac{\lambda_{\min}(\b P)}{\lambda_{\max}(\b P)}}\varepsilon= 2.
\]
\item Given $\b x_0=\left[\begin{array}{cc}
8 &6
\end{array}\right]^{\top}$ and $\varepsilon_1=2$. We will find $T$ such that $\|\b x(t)\|\le\varepsilon_1$ for all $t\ge T$.\\
We have $\|\b x_0\|_2=10$. Let $\delta_1=10$. We then compute $R$, $\delta_2$, and $T$ from the equations:
\begin{eqnarray*}
\psi_2(\delta_1) &=& \psi_1(R)\\
\psi_2(\delta_2) &=& \min \left( \psi_1(\varepsilon_1),\psi_2(\delta_1)\right)\\
T &=& \frac{\psi_1(R)}{\psi_3(\delta_2)}.
\end{eqnarray*}
Using the first equation, we find
\[
R = \sqrt{\frac{\lambda_{\max}(\b P)}{\lambda_{\min}(\b P)}}\delta_1= 20.
\]
We then solve for $\delta_2 \le \delta_1$ to get
\[
\delta_2 = \sqrt{\frac{\lambda_{\min}(\b P)}{\lambda_{\max}(\b P)}}\varepsilon_1= 1.
\]
Hence,
\[
T = \frac{\lambda_{\min}(\b P)R^2}{\lambda_{\min}(\b Q)\delta_2^2}= \frac{\lambda_{\min}(\b P)\frac{\lambda_{\max}(\b P)}{\lambda_{\min}(\b P)}\delta_1^2}{\lambda_{\min}(\b Q)\frac{\lambda_{\min}(\b P)}{\lambda_{\max}(\b P)}\varepsilon_1^2}= \frac{\lambda_{\max}^2(\b P)\delta_1^2}{\lambda_{\min}(\b Q)\lambda_{\min}(\b P)\varepsilon_1^2}= 200.
\]
\end{itemize}
\end{example}\markex
\begin{proposition}
\label{pro:exponsaofz}
If in some neighborhood $\cal S$ of $\b x_e=\b 0$ the positive definite and decrescent function $W$ satisfies
\[
c_1\|\b x\|^2 \le W(t,\b x) \le c_2 \|\b x\|^2
\]
for some positive constants $c_1$ and $c_2$ and its time derivative along any solution of the system~(\ref{eq:gensysmo}) is negative definite in $\cal S$ and such that for some positive constant $c_3$,
\[
\dot{W}(t,\b x)\le -c_3\|\b x\|^2,
\]
then the equilibrium state $\b x_e=\b 0$ is uniformly exponentially stable.
\end{proposition}
{\bf Proof}\quad We have
\begin{eqnarray*}
\dot{W}(t,\b x) &\le & -c_3\|\b x\|^2\\
&=& -\frac{c_3}{c_2}\left( c_2\|\b x\|^2\right)\\
&\le & -\frac{c_3}{c_2}{W}(t,\b x).
\end{eqnarray*}
Applying to the above Theorem~\ref{thm:comparison}, which is called the comparison theorem, we obtain
\[
{W}(t,\b x)\le {W}(t_0,\b x_0)\exp\left( -\frac{c_3}{c_2}\left(t-t_0\right)\right).
\]
Hence,
\begin{eqnarray*}
\|\b x\| &\le & \sqrt{\frac{{W}(t,\b x)}{c_1}}\\
&\le & \sqrt{\frac{{W}(t_0,\b x_0)\exp\left( -\frac{c_3}{c_2}\left(t-t_0\right)\right)}{c_1}}\\
&\le & \left( \frac{c_2}{c_1}\right)^{\frac{1}{2}}\|\b x(t_0)\| \exp\left( -\frac{c_3}{2c_2}(t-t_0)\right),
\end{eqnarray*}
which means that the equilibrium state $\b x_e=\b 0$ is uniformly exponentially stable.\\
\hspace*{5.5in} $\Box$

The above theorems were concerned with local stability. We now proceed to prove sufficiency conditions for global stability. For this we need the following definitions. We first define a {\em radially unbounded\/} or {\em coercive\/} function.
\begin{defn}
A time-varying function $W : \mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ is radially unbounded, or coercive, if as $\|\mb{\bm$x$}\| \to \infty$, the function $W$ tends to infinity uniformly with respect to $t$, that is, for any given positive real number $M$, there is a positive real number $N$ so that $W(t,\mb{\bm$x$}) > M$ for all $t$ and all $\mb{\bm$x$}$ whenever $\|\mb{\bm$x$}\| > N$.
\end{defn}\markdefn

The next two definitions are concerned with {\em Lagrange stable equilibrium states}.
\begin{defn}
\label{defn:lagr_stab}
The equilibrium state $\mb{\bm$x$}_{e}$ is called Lagrange stable, or bounded, if for every real number $d>0$ there exists a positive real number $b=b(t_0,d)$ such that if
\[
\| \mb{\bm$x$}_0 - \mb{\bm$x$}_{e} \| < d ,
\]
then
\[
\| \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)-\mb{\bm$x$}_{e} \| < b
\]
for all $t \ge t_0$.
\end{defn}\markdefn

\begin{defn}\label{defn:unifo_bounded}
The equilibrium state $\mb{\bm$x$}_{e}$ is uniformly bounded if the number $b$ in Definition~\ref{defn:lagr_stab} can be taken independent of the initial time $t_0$.
\end{defn}\markdefn

We now define {\em uniformly asymptotically stable in the large\/} equilibrium state.
\begin{defn}
If the equilibrium state $\mb{\bm$x$}_{e}$ is uniformly stable, uniformly bounded, and all trajectories are uniformly convergent to $\mb{\bm$x$}_{e}$, then this equilibrium state is called uniformly asymptotically stable in the large, or globally uniformly asymptotically stable.
\end{defn}\markdefn

The following theorem gives a sufficiency condition for an equilibrium state to be globally uniformly asymptotically stable.
\begin{thm}
\label{thm:guas}
If there is a function $W$ that is positive definite for all $\mb{\bm$x$} \ne \mb{\bm$x$}_{e}$, $W(t,\mb{\bm$x$}_{e})=0$, decrescent, radially unbounded, and if its time derivative along any solution of the system is negative definite for all $\mb{\bm$x$}$, then the equilibrium state $\mb{\bm$x$}_{e}$ is globally uniformly asymptotically stable.
\end{thm}
{\bf Proof}\quad We assume, for simplicity, that $\b x_{e}=\b 0$ and follow the arguments of Willems~\cite[pages~31,~32]{willems70}. By Theorem~\ref{thm:loc_stab}, the equilibrium state $\mb{\bm$x$}_{e}$ is uniformly stable. Similarly as in Theorem~\ref{thm:uniform_as}, we construct the functions $\psi_1$, $\psi_2$, and $\psi_3$. Since, by assumption, $W$ is radially unbounded, $\psi_1(\|\mb{\bm$x$}\|) \to \infty$ as $\|\mb{\bm$x$}\| \to \infty$. Therefore, for any constant $d>0$, there is a constant $b > 0$ such that $\psi_1(b) > \psi_2(d)$. See Figure~\ref{fig:fin_bpr} for an illustration of the procedure that can be used to find the constant $b$.
\begin{figure}
\centering
\includegraphics[scale=0.55]{fin_bpr.eps}
%\vspace{3in}
%\centerbmp{3.33in}{2.5in}{c:/mybook/fin_bpr.bmp}
%\hskip1in\special{bmp:c:/mybook/fin_bpr.bmp x=4in y=3in}
\caption{Determining a constant $b>0$ given $d>0$ in the proof of Theorem~\ref{thm:guas}.}
\label{fig:fin_bpr}
\end{figure}
If $\| \mb{\bm$x$}_0 \|<d$, then $\| \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0) \|<b$ for all $t \ge t_0$. Indeed, because
\[
\psi_1(b)> \psi_2(d) \ge W(t_0,\mb{\bm$x$}_0) \ge W(t,\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)) \ge \psi_1(\| \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0) \|),
\]
then
\[
\psi_1(b)>\psi_1(\| \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0) \|).
\]
The above implies that
\[
\| \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0) \|<b
\]
because $\psi_1$ is strictly increasing. Hence all trajectories of the system are uniformly bounded.\\
To prove their convergence to the equilibrium state, we define $\delta_2$, $T$, and $b$, given $d$ and $\epsilon_1$, as
\begin{eqnarray*}
\psi_2(d) &<& \psi_1(b),\\
\psi_2(\delta_2) &<& \psi_1(\epsilon_1), \\
T &=& \psi_1(b)/ \psi_3(\delta_2).
\end{eqnarray*}
If $\|\mb{\bm$x$}_0\|<d$, then for all $t>t_0+T$, we have
\[
\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|<\epsilon_1,
\]
which can be shown using the same arguments as in the proof of Theorem~\ref{thm:uniform_as} with $b$ playing the role of $R$, and $d$ playing the role of $\delta_1$. This proves that all trajectories of the system converge uniformly to the equilibrium state. Hence, the equilibrium state is globally uniformly asymptotically stable.\\
\hspace*{5.5in} $\Box$


We now give a theorem that can be used as a test for checking the instability of nonlinear dynamic systems. This instability test is illustrated in Figure~\ref{fig:instabt}.
\begin{thm}
\label{thm:instability}
Let $\mb{\bm$x$}_{e}$ be an equilibrium state of the system $\dot{\mb{\bm$x$}}=\mb{\bm$f$}(t,\mb{\bm$x$})$, that is, $\mb{\bm$f$}(t,\mb{\bm$x$}_{e})=\mb{\bm$0$}$. If there exists a function $W:\mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ such that in the region ${\cal S}=\{ \mb{\bm$x$}:\|\mb{\bm$x$}-\mb{\bm$x$}_{e}\| \le R \}$
\begin{itemize}
\item[(i)]
$W \in {\cal C}^1$,
\item[(ii)]
$W(t,\mb{\bm$x$}_{e})=0$,
\item[(iii)]
$\dot{W}(t,\mb{\bm$x$}) > 0$ for all $\mb{\bm$x$} \in {\cal S} \setminus \{\mb{\bm$x$}_{e} \}$,
\item[(iv)]
in every neighborhood of $\mb{\bm$x$}_{e}$ in ${\cal S}$, for any $t_0$ there is at least one point $\mb{\bm$x$}$ at which $W(t_0,\mb{\bm$x$})>0$, and
\item[(v)] $W(t,\b x)$ is bounded in ${\cal S}$,
\end{itemize}
then the equilibrium state $\mb{\bm$x$}_{e}$ is unstable.
\end{thm}
{\bf Proof}\quad Without loss of generality, assume that $\mb{\bm$x$}_{e}=\mb{\bm$0$}$. By assumption~{\it (iv)}, for any $t_0$ and any $\delta$, $0<\delta<R$, there exists $\mb{\bm$x$}_{\delta}$, $0<\|\mb{\bm$x$}_{\delta}\| < \delta$, such that $W(t_0,\mb{\bm$x$}_{\delta}) > 0$. The system trajectory $\mb{\bm$x$}(t)=\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_{\delta})$ cannot approach the equilibrium state $\mb{\bm$x$}_{e}=\mb{\bm$0$}$ as $t \to \infty$ because $W(t,\mb{\bm$0$})=0$ but $W(t_0,\mb{\bm$x$}_{\delta}) > 0$ and $\dot{W}(t,\mb{\bm$x$})>0$ for $\mb{\bm$x$} \ne \mb{\bm$x$}_{e}$.
\begin{figure}
\centering
\includegraphics[scale=0.55]{instabt.eps}
%\vspace{3in}
%\centerbmp{3.33in}{2.5in}{c:/mybook/instabt.bmp}
%\hskip1in\special{bmp:c:/mybook/instabt.bmp x=4in y=3in}
\caption{An illustration of Theorem~\ref{thm:instability}.}
\label{fig:instabt}
\end{figure}
Since $\dot{W}$ is continuous, because $W \in {\cal C}^1$, and the trajectory is bounded away from the equilibrium state, there is a constant $m>0$ such that for all $t \ge t_0$,
\[
\dot{W}(t,\mb{\bm$x$}(t)) \ge m >0,
\]
as long as
\[
\|\mb{\bm$x$}(t)\| \le R, \qquad \mb{\bm$x$} \ne \mb{\bm$0$}.
\]
Hence,
\begin{equation}
W(t,\mb{\bm$x$})=W(t_0,\mb{\bm$x$}_{\delta})+\int_{t_0}^{t} \dot{W}(s,\mb{\bm$x$}(s))ds \ge W(t_0,\mb{\bm$x$}_{\delta})+m(t-t_0).
\label{eq:instexwgt}
\end{equation}
By condition (v), the function $W$ is bounded in ${\cal S}$.. However, the right hand side of~(\ref{eq:instexwgt}) is unbounded. Therefore the trajectory $\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_{\delta})$ cannot remain infinitely long in $\cal S$. Thus, for some $t_1>t_0$ the trajectory reaches the boundary of the region $\|\mb{\bm$x$}\| \le R$. Hence, given any $\epsilon$, $0<\epsilon<R$, then for any $\delta>0$, no matter how small, there is at least one trajectory such that $\|\mb{\bm$x$}_{\delta}\| < \delta$ but $\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_{\delta})\| \ge \epsilon$ for some $t_1 > t_0$. Therefore, the equilibrium state is unstable.\\
\hspace*{5.5in} $\Box$


\begin{corollary}\label{cor:instability}
If there exists a function $W:\mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ such that in the region ${\cal S}=\{ \mb{\bm$x$}:\|\mb{\bm$x$}-\mb{\bm$x$}_{e}\| \le R \}$
\begin{itemize}
\item[(i)]
$W \in {\cal C}^1$,
\item[(ii)]
$W(t,\mb{\bm$x$}_{e})=0$,
\item[(iii)]
$\dot{W}(t,\mb{\bm$x$}) < 0$ for all $\mb{\bm$x$} \in {\cal S} \setminus \{\mb{\bm$x$}_{e} \}$,
\item[(iv)]
in every neighborhood of $\mb{\bm$x$}_{e}$ in ${\cal S}$, for any $t_0$ there is at least one point $\mb{\bm$x$}$ at which $W(t_0,\mb{\bm$x$})<0$, and
\item[(v)] $W(t,\b x)$ is bounded in ${\cal S}$,
\end{itemize}
then the equilibrium state $\mb{\bm$x$}_{e}$ is unstable.
\end{corollary}

Note that if $W(t, \b x)=W(\b x)$, that is, if $W$ is time-invariant, then Condition (v) of Theorem~\ref{thm:instability} and Corollary~\ref{cor:instability} is not needed because $W=W(\b x)$ being continuous on the compact set ${\cal S}$ must be bounded in ${\cal S}$.
\section{Lyapunov's Indirect Method}
\label{sec:lyap_indirec}
In this section, we are concerned with the problem of investigating stability properties of an equilibrium state of a nonlinear system based on its linearization about the given equilibrium. We devise a method that allows one to determine whether the equilibrium of the nonlinear system is asymptotically stable or unstable based on the dynamics of the linearized system about this equilibrium. The method is sometimes referred to as the {\em Lyapunov's first method\/} or {\em Lyapunov's indirect method\/}. For simplicity of notation, we assume that the equilibrium point to be tested for stability is the origin. We consider a nonlinear system model,
\begin{equation}
\dot{\b x}=\b f(\b x),
\label{eq:lin_stabi}
\end{equation}
where $\b f:{\cal D}\to \mb{\Bb R}^n$ is a continuously differentiable function from a domain ${\cal D}\subseteq \mb{\Bb R}^n$ into $\mb{\Bb R}^n$. We assume that the origin $\b x=\b 0$ is in the interior of ${\cal D}$ and is an equilibrium state of~(\ref{eq:lin_stabi}), that is, $\b f(\b 0)=\b 0$. Let
\begin{equation}
\b A=\left.\frac{\partial \b f}{\partial \b x}\right\arrowvert_{\b x=\b 0}=\left.\left[\begin{array}{cccc}
\frac{\partial f_1}{\partial x_1} & \frac{\partial f_1}{\partial x_2} & \cdots & \frac{\partial f_1}{\partial x_n}\\
\vdots & {} & \ddots & \vdots\\
\frac{\partial f_n}{\partial x_1} & \frac{\partial f_n}{\partial x_2} & \cdots & \frac{\partial f_n}{\partial x_n}
\end{array}\right]\right\arrowvert_{\b x=\b 0}
\label{eq:jacobatzero}
\end{equation}
be the Jacobian matrix of $\b f$ evaluated at $\b x=\b 0$. Then, applying Taylor's theorem to the right hand side of~(\ref{eq:lin_stabi}) yields
\begin{equation}
\dot{\b x}=\b A \b x+\b g(\b x),
\label{eq:aftertaylor}
\end{equation}
where the function $\b g$ represents higher-order terms and has the property,
\begin{equation}
\lim_{\Arrowvert\b x\Arrowvert\to 0} \frac{\Arrowvert\b g(\b x)\Arrowvert}{\Arrowvert\b x\Arrowvert}=0.
\label{eq:hotg}
\end{equation}
The above suggests that in a ``small'' neighborhood of the origin, we can approximate the nonlinear system~(\ref{eq:lin_stabi}) with its linearized system about the origin, $\dot{\b x}=\b A\b x$. The following theorems give conditions under which we can infer about stability properties of the origin of the nonlinear system based on stability properties of the linearized system.
\begin{thm}\label{thm:firstlayap}
Let the origin, $\b x=0$, be an equilibrium state of the nonlinear system, $\dot{\b x}=\b f(\b x)$. Then, the origin is an asymptotically stable equilibrium of the nonlinear system if $\b A$, the Jacobian matrix of $\b f$ evaluated at the origin, has all its eigenvalues in the open left-half plane.
\end{thm}
{\bf Proof}\quad Our proof is based on that of Khalil~\cite[pages~127--128]{khalil96}. To begin with, we suppose that all the eigenvalues of $\b A$ are in the open left-half plane. Then, by Theorem~\ref{Lyapunov}, for any positive definite symmetric matrix $\b Q$, the solution $\b P$ to the Lyapunov equation~(\ref{eq:layamateqn}) is also positive definite. We employ the positive definite quadratic form $V(\b x)=\b x^T\b P\b x$ as a Lyapunov function candidate for the nonlinear system $\dot{\b x}=\b f(\b x)$. We use~(\ref{eq:aftertaylor}) and the Lyapunov matrix equation~(\ref{eq:layamateqn}) when evaluating the time derivative of $V(\b x)$ on the trajectories of the nonlinear system. We obtain
\begin{eqnarray}
\dot{V}(\b x) &=& 2\b x^{\top}\b P\b f(\b x)\nonumber\\
&=& 2\b x^{\top}\b P\left(\b A\b x+\b g(\b x) \right)\nonumber\\
&=&\b x^{\top}\left(\b A^{\top}\b P+\b P\b A\right) \b x+2\b x^{\top}\b P\b g(\b x)\nonumber\\
&=& -\b x^{\top}\b Q\b x +2\b x^{\top}\b P\b g(\b x).
\label{eq:lyaderlyafirst}
\end{eqnarray}
The first term on the right hand side of the above equation is negative definite, while the second term is, in general, indefinite. The function $\b g$ satisfies
\[
\frac{\left\Arrowvert \b g(\b x) \right\Arrowvert_2}{\left\Arrowvert \b x \right\Arrowvert_2}\to 0\quad\mb{as}\quad \left\Arrowvert \b x \right\Arrowvert_2\to 0.
\]
Therefore, for any $\gamma>0$ there exists $r>0$ such that
\begin{equation}
\left\Arrowvert \b g(\b x) \right\Arrowvert_2 < \gamma \left\Arrowvert \b x \right\Arrowvert_2 \quad\mb{for all}\quad \left\Arrowvert \b x \right\Arrowvert_2 < r.
\label{eq:gamrg}
\end{equation}
Substituting the above into~(\ref{eq:lyaderlyafirst}) gives
\begin{eqnarray}
\dot{V}(\b x) &<& -\b x^{\top}\b Q\b x+2\gamma\left\Arrowvert \b P \right\Arrowvert_2\left\Arrowvert \b x \right\Arrowvert_2^2\nonumber\\
&\le & -\left(\lambda_{\min}(\b Q)- 2\gamma\lambda_{\max}(\b P)\right)\left\Arrowvert \b x \right\Arrowvert_2^2
\quad\mb{for all}\quad \left\Arrowvert \b x \right\Arrowvert_2<r
\label{eq:vdotlforr}
\end{eqnarray}
Thus, selecting
\[
\gamma<\frac{\lambda_{\min}(\b Q)}{2\lambda_{\max}(\b P)}
\]
ensures that $\dot{V}(\b x)$ is negative definite. By Theorem~\ref{thm:uniform_as}, we conclude that the origin, $\b x=\b 0$, is an asymptotically stable equilibrium state of the nonlinear system, which completes the proof.\\
\hspace*{5.5in} $\Box$\\
\begin{thm}\label{thm:firstlayapuns}
Let the origin, $\b x=\b 0$, be an equilibrium state of the nonlinear system, $\dot{\b x}=\b f(\b x)$. Then, the origin is unstable if $\b A$ has one or more eigenvalues in the right-half plane.
\end{thm}
{\bf Proof}\quad Our proof follows that of Khalil~\cite[pages~128--130]{khalil96}. We first consider the special case when $\b A$ has no eigenvalues on the imaginary axis. Suppose that the eigenvalues cluster into a group of eigenvalues in the open right-plane and a group of eigenvalues in the open left-half plane. Then, we can find a nonsingular matrix $\b T$ such that
\[
\b T\b A\b T^{-1}=\left[\begin{array}{cc}
-\b A_1 & \b O\\
\b O & \b A_2
\end{array}\right],
\]
where both $\b A_1$ and $\b A_2$ have their eigenvalues in the open left-half plane. A possible candidate for $\b T$ would be the matrix that transforms $\b A$ into its real Jordan form. Now, let
\[
\b z=\b T\b x=\left[\begin{array}{c}
\b z_1\\
\b z_2
\end{array}\right],
\]
where the partition of $\b z$ is compatible with the dimensions of $\b A_1$ and $\b A_2$. The change of variables, $\b z=\b T\b x$, transforms the nonlinear system~(\ref{eq:aftertaylor}) into the form
\begin{equation}
\left.\begin{array}{llr}
\dot{\b z}_1 &=& -\b A_1\b z_1 + \b g_1(\b z)\\
\dot{\b z}_2 &=& \b A_2\b z_2 + \b g_2(\b z)
\end{array}\right\}
\label{eq:systinnewcoo}
\end{equation}
where the functions $\b g_i$, $i=1,2$, have the property that  for any $\gamma>0$, there exists $r>0$ such that for $i=1,2$, we have
\begin{equation}
\left\Arrowvert \b g_i(\b z)\right\Arrowvert_2 \le \gamma\left\Arrowvert\b z \right\Arrowvert_2\quad\mb{for all}\quad \left\Arrowvert\b z \right\Arrowvert_2 \le r.
\label{eq:twoestfogi}
\end{equation}
The origin $\b z=\b 0$ is an equilibrium state for the nonlinear system~(\ref{eq:systinnewcoo}). Since the matrix $\b T$ is nonsingular, stability properties of $\b z=\b 0$ in the $\b z$-coordinates are the same as the stability properties of $\b x=\b 0$ in the $\b x$-coordinates. We will now show that $\b z=\b 0$ is an unstable equilibrium of the system~(\ref{eq:systinnewcoo}), which implies that $\b x=\b 0$ is an unstable equilibrium of the system~(\ref{eq:lin_stabi}). We use Theorem~\ref{thm:instability}, which entails constructing a function $V$ satisfying the hypotheses of that theorem. Let $\b Q_1$ and $\b Q_2$ be positive definite symmetric matrices of the dimensions of $\b A_1$ and $\b A_2$, respectively. Because $\b A_1$ and $\b A_2$ have their eigenvalues in the open left-half plane, the Lyapunov matrix equations,
\[
\b A_i^{\top}\b P_i+\b P_i\b A_i=-\b Q_i, \quad i=1,2,
\]
have unique positive definite solutions $\b P_1$ and $\b P_2$. Let
\begin{equation}
V(\b z)=\b z_1^{\top}\b P_1\b z_1-\b z_2^{\top}\b P_2\b z_2=\b z^{\top}\left[\begin{array}{cc}
\b P_1 & \b O\\
\b O & -\b P_2
\end{array}\right]\b z.
\label{eq:bigvwitm}
\end{equation}
In the subspace, $\left\{\b z=\left[\begin{array}{cc}
\b z_1^{\top} & \b z_2^{\top}
\end{array}\right]^{\top}:\b z_2=\b 0\right\}$, $V(\b z)>0$ in every neighborhood of the origin. Let
\[
{\cal U}=\left\{\b z\in \mb{\Bb R}^n:\left\Arrowvert \b z\right\Arrowvert_2 \le r\right\}.
\]
The time derivative of $V$ evaluated on the trajectories of the system~(\ref{eq:systinnewcoo}) is
\begin{eqnarray}
\dot{V}(\b z) &=& 2\b z_1^{\top}\b P_1\dot{\b z}_1-2\b z_2^{\top}\b P_2\dot{\b z}_2\nonumber\\
&=& 2\b z_1^{\top}\b P_1\left( -\b A_1\b z_1 + \b g_1(\b z)\right)-2\b z_2^{\top}\b P_2\left( \b A_2\b z_2 + \b g_2(\b z)\right)\nonumber\\
&=& \b z_1^{\top}\b Q_1\b z_1+2\b z_1^{\top}\b P_1\b g_1(\b z)+\b z_2^{\top}\b Q_2\b z_2-2\b z_2^{\top}\b P_2\b g_2(\b z)\nonumber\\
& \ge & \lambda_{\min}(\b Q_1)\left\Arrowvert \b z_1\right\Arrowvert_2^2 + \lambda_{\min}(\b Q_2)\left\Arrowvert \b z_2\right\Arrowvert_2^2+2\b z^{\top}\left[\begin{array}{c}
\b P_1\b g_1(\b z)\\
-\b P_2\b g_2(\b z)
\end{array}\right]\nonumber\\
&\ge& \lambda_{\min}(\b Q_1)\left\Arrowvert \b z_1\right\Arrowvert_2^2 + \lambda_{\min}(\b Q_2)\left\Arrowvert \b z_2\right\Arrowvert_2^2\nonumber\\
&{}&\quad - 2 \left\Arrowvert \b z\right\Arrowvert_2\sqrt{\left\Arrowvert \b P_1\right\Arrowvert_2^2\left\Arrowvert \b g_1(\b z)\right\Arrowvert_2^2 + \left\Arrowvert \b P_2\right\Arrowvert_2^2\left\Arrowvert \b g_2(\b z)\right\Arrowvert_2^2}.
\label{lyadeofsysnewcoo}
\end{eqnarray}
Let
\[
\alpha=\min\left(\lambda_{\min}(\b Q_1), \lambda_{\min}(\b Q_2) \right)\quad\mb{and}\quad \beta=\max\left( \left\Arrowvert \b P_1\right\Arrowvert_2, \left\Arrowvert \b P_2\right\Arrowvert_2\right).
\]
With~(\ref{eq:twoestfogi}) and the above taken into account, (\ref{lyadeofsysnewcoo}) evaluates to
\begin{equation}
\dot{V}(\b z)\ge \left(\alpha- 2\sqrt{2}\beta\gamma\right)\left\Arrowvert \b z\right\Arrowvert_2^2\quad\mb{for all}\quad \b z \in {\cal U}.
\label{eq:vdotgret}
\end{equation}
If we select
\[
\gamma<\frac{\alpha}{2\sqrt{2}\beta},
\]
then $\dot{V}(\b z)>0$ in ${\cal U}\setminus \{\b 0\}$. Thus, all hypotheses of Theorem~\ref{thm:instability} are satisfied and hence the origin is an unstable equilibrium of the system~(\ref{eq:lin_stabi}).

In the general case when $\b A$ has eigenvalues on the imaginary axis, in addition to eigenvalues in the open right-half plane, we shift the imaginary axis in order to reduce this case to the previous one. Specifically, suppose that $\b A$ has $m$ eigenvalues in the open right-half plane with $\Re \lambda_i > \delta >0$. Then, the matrix
\[
\b A-\frac{\delta}{2}\b I
\]
still has $m$ eigenvalues in open right-half plane, but no eigenvalues on the imaginary axis. We can proceed similarly as in the previous case to conclude that the origin is an unstable equilibrium of the corresponding nonlinear system~(\ref{eq:lin_stabi}). For details of this part of the proof, we refer to Khalil~\cite[page~130]{khalil96}.\\
\hspace*{5.5in} $\Box$

In summary, the above two theorems provide us with a simple method for determining asymptotic stability or instability of an equilibrium state of the nonlinear system~(\ref{eq:lin_stabi}). We compute the Jacobian matrix~(\ref{eq:jacobatzero}) of the nonlinear system, evaluate it at the equilibrium state of interest, and test its eigenvalues. If $\Re \lambda_i<0$ for all $i$, then the equilibrium state is asymptotically stable. If for some $i$, $\Re \lambda_i>0$, then the equilibrium is unstable. Finally, if $\Re \lambda_i \le 0$ for all $i$ with $\Re \lambda_i = 0$ for some $i$, linearization fails to determine asymptotic stability of the equilibrium of the nonlinear system. In this case, the stability of the nonlinear system depends on the nature of the higher-order terms.
\begin{example}\label{ex:hopf_lyap}\rm
For the nonlinear system model,
\begin{equation}
\left[\begin{array}{c}
\dot{x}_1\\
\dot{x}_2
\end{array}\right]=\left[\begin{array}{c}
-1.1 x_1+g_2(x_2)\\
g_1(x_1)-1.1 x_2
\end{array}\right],
\label{eq:hopfienonlisys}
\end{equation}
where
\[
g_i(x_i)=\frac{2}{\pi}\arctan\left( 0.7\pi x_i\right),\quad i=1,2,
\]
we will find its equilibrium states and determine their stability properties using the linearization method. We mention here that the above system was analyzed in~\cite{hopfield84, michel89}.

To find the system equilibrium states, we solve the algebraic equations $\dot{x}_1=0$ and $\dot{x}_2=0$, that is,
\[
\left.\begin{array}{ccc}
-1.1 x_1+\frac{2}{\pi}\arctan\left( 0.7\pi x_2\right)&=&0\\
\frac{2}{\pi}\arctan\left( 0.7\pi x_1\right)-1.1 x_2&=&0
\end{array}\right\}
\]
We obtain three equilibrium states:
\[
\b x^{(1)}=\left[\begin{array}{c}
0.454\\
0.454
\end{array}\right], \quad \b x^{(2)}=\left[\begin{array}{c}
-0.454\\
-0.454
\end{array}\right],\quad\mb{and}\quad \b x^{(3)}=\left[\begin{array}{c}
0\\
0
\end{array}\right].
\]
Let
\[
\Delta \b x=\b x-\b x^{(i)},\quad i=1,2,3.
\]
Then, a linearized model about an equilibrium state has the form
\[
\frac{d}{dt}\Delta \b x= \b A (\Delta \b x),
\]
where
\begin{equation}
\b A=\left.\left[\begin{array}{cc}
\frac{\partial f_1(\b x)}{\partial x_1} & \frac{\partial f_1(\b x)}{\partial x_2}\\
\frac{\partial f_2(\b x)}{\partial x_1}& \frac{\partial f_2(\b x)}{\partial x_2}
\end{array}\right]\right\arrowvert_{\b x=\b x^{(i)}}=\left.\left[\begin{array}{cc}
-1.1 & \frac{1.4}{1+\left(0.7 \pi x_2\right)^2}\\
\frac{1.4}{1+\left(0.7 \pi x_1\right)^2} & -1.1\end{array}\right]\right\arrowvert_{\b x=\b x^{(i)}}.
\label{eq:hopfiellin}
\end{equation}
We have three linearized models:\begin{itemize}
\item[(i)] about $\b x^{(1)}$,
\[
\frac{d}{dt}\Delta \b x=\left[\begin{array}{cc}
-1.1 & 0.7073\\
0.7073 & -1.1
\end{array}\right]\Delta \b x,
\]
\item[(ii)] about $\b x^{(2)}$,
\[
\frac{d}{dt}\Delta \b x=\left[\begin{array}{cc}
-1.1 & 0.7073\\
0.7073 & -1.1
\end{array}\right]\Delta \b x,
\]
\item[(iii)] about $\b x^{(3)}$,
\[
\frac{d}{dt}\Delta \b x=\left[\begin{array}{cc}
-1.1 & 1.4\\
1.4 & -1.1
\end{array}\right]\Delta \b x.
\]
\end{itemize}
By Theorem~\ref{thm:firstlayap}, the states $\b x^{(1)}$ and $\b x^{(2)}$ are asymptotically stable because the corresponding linearized models have their eigenvalues in the open left-half plane: -1.8073 and -0.3927. By Theorem~\ref{thm:firstlayapuns}, the equilibrium state $\b x^{(3)}$ is unstable because the linearized model has an eigenvalue in the open right-half plane, at 0.3 (the other eigenvalue is at -2.5). In Figure~\ref{fig:hop_phs}, we show a phase portrait of the above nonlinear system, where one can see clearly the stability properties of the equilibrium states.
\begin{figure}
\centering
\includegraphics[scale=0.55]{hop_phs.eps}
%\vspace{3.5in}
%\centerline{\psfig{figure=hop_phs.eps,height=3.5in}}
%\hskip1.5in{\special{eps:c:/mybook/hop_phs.eps y=3.5in}}
\caption{Phase portrait of the nonlinear system of Example~(\ref{ex:hopf_lyap}).}
\label{fig:hop_phs}
\end{figure}
\end{example}\markex


\begin{example}\rm
In this example, we employ the Lyapunov's indirect method in the analysis of the hypothalamic-pituitary-adrenal~(HPA)\index{hypothalamic-pituitary-adrenal (HPA) axis}\index{HPA axis} axis model presented in Section~\ref{subsec:HPA_axis_model}. Recall that the HPA axis is the body's primary stress management system. In our analysis of the HPA axis, we use plots of equilibrium values of the state variables as a function of the external stressor, $d$. These plots are shown in Figures~\ref{fig:x1_and_x3_vs_d} and~\ref{fig:x2_and_x4_vs_d}.
\begin{figure}
  \centering
  \includegraphics[width=3.25in]{x1_and_x3_vs_d}
  \caption{Plots of the equilibrium values of $x_1$ and $x_3$ versus $d$ for $u=0$.}
  \label{fig:x1_and_x3_vs_d}
\end{figure}

\begin{figure}
  \centering
  \includegraphics[width=3.25in]{x2_and_x4_vs_d}
  \caption{Plots of the equilbrium values of $x_2$ and $x_4$ versus $d$ for $u=0$.}
  \label{fig:x2_and_x4_vs_d}
\end{figure}

In a chronically stressed individual, cortisol concentration, $x_4$, is very low. Thus a healthy individual subjected to a prolonged extreme stress, $d>0.168$, would settle down in a stable equilibrium state corresponding to depressed cortisol concentration, $x_4$, corresponding to the lower branch of the curve in Figure~\ref{fig:x2_and_x4_vs_d}. When the stress subsides, that is, $d=0$, the individual will stay in the new equilibrium state corresponding to a depressed cortisol concentration. This is because the equilibrium corresponding to $d=0$ is asymptotically stable and so states ``close'' to it will be attracted by this asymptotically stable low cortisol equilibrium.  We note that the middle branch in Figure~\ref{fig:x2_and_x4_vs_d}, for the values of the stressor $d$ where there are three equilibria, corresponds to unstable equilibria. As an illustration of the above statement, we show the equilibrium states corresponding to $d=0.1$; we have three equilibrium states:
\[
\b x_e^{(1)} = \left[\begin{array}{c}
0.6727\\
0.0635\\
0.0931\\
0.0635
\end{array}\right], \quad \b x_e^{(2)} = \left[\begin{array}{c}
0.6840\\
0.0608\\
0.2048\\
0.0608
\end{array}\right],\quad\mb{and}\quad \b x_e^{(3)}=\left[\begin{array}{c}
0.7198\\
0.0528\\
0.6871\\
 0.0528
\end{array}\right].
\]
The corresponding linear models are;
\[
\frac{d}{dt} \Delta \b x = \b A^{(i)}\Delta \b x + \b b_2^{(i)}\Delta d,\quad i=1,2,3,
\]
where
\[
\b A^{(1)}=\left[\begin{array}{cccc}
-1.0000 &   0  &  0  & -4.1141\\
0.9442 & -10.0000 &  -0.3809 &  -0.5582\\
0   &   0  & -0.1988  &  1.0276\\
0  &  1.0000  &   0 &  -1.0000
\end{array}\right],\quad \b b_2^{(1)}=\left[\begin{array}{c}
0.6116\\
0\\
0\\
0
\end{array}\right],
\]
\[
\b A^{(2)}=\left[\begin{array}{cccc}
-1.0000 &  0 & 0 & -4.2530\\
0.8892 & -10.0000 &  -0.3290 &  -1.1077\\
0 &  0 & 0.2355 & 3.8235\\
0 & 1.0000 & 0 & -1.0000
\end{array}\right],\quad \b b_2^{(2)}=\left[\begin{array}{c}
0.6218\\
0\\
0\\
0
\end{array}\right],
\]
and
\[
\b A^{(3)}=\left[\begin{array}{cccc}
-1.0000 & 0 & 0 & -4.7104\\
0.7337 & -10.0000 &  -0.2047 &  -2.6627\\
0 &  0 &  -0.1859  &  9.2897\\
0 &   1.0000 &  0 &  -1.0000
\end{array}\right],\quad \b b_2^{(3)}=\left[\begin{array}{c}
0.6544\\
0\\
0\\
0
\end{array}\right].
\]
The eigenvalues of $\b A^{(i)}$, $i=1,2,3$, are:
\begin{eqnarray*}
\mb{eig}(\b A^{(1)}) &=& \{-9.990413, -0.989890, -0.989890, -0.228634 \},\\
\mb{eig}(\b A^{(2)}) &=& \{-9.937246, -0.992395, -0.992395,\,  0.157563 \},\\
\mb{eig}(\b A^{(3)}) &=& \{-9.763819, -1.051757, -1.051757, -0.318609 \}.
\end{eqnarray*}
The second linear model is unstable because one of the eigenvalues of $\b A^{(2)}$ is positive.

Ben-Zvi et al.~\cite{benzvi09} proposed a treatment strategy whereby the chronically stressed individual is moved to the healthy state corresponding to the normal cortisol concentration, by moving the individual into a negative stress region, after which internal regulatory processes translates the individual into the healthy equilibrium which is asymptotically stable and maintains it there. Effective treatment can be accomplished when the states as well as stress levels are available. Thus a method to estimate state variables as well as the stress level are highly desirable. This is discussed in Chapter~\ref{chap:UIO_chapter}.
\end{example}\markex


\section{Discontinuous Robust Controllers}
\label{sec:discconuncs}
In this section, we consider the problem of constructing stabilizing state-feedback controllers for nonlinear dynamic systems whose mathematical models contain uncertainties. The design of the controllers is based on the Lyapunov theory presented in the previous sections. The class of dynamic systems considered here is modeled by
\begin{equation}
\dot{\b x}(t)=\b f(t,\b x(t))+\b G(t,\b x(t))\left(\b u(t) + \mb{\bm$\xi$}(t,\b x(t))\right),\qquad\b x(t_0)=\b x_0,
\label{eq:uncertmode}
\end{equation}
where $\mb{\bm$x$}(t)\in \mb{\Bb R}^n$, $\mb{\bm$u$}(t)\in \mb{\Bb R}^m$, and $\mb{\bm$\xi$}(t,\mb{\bm$x$}(t))\in \mb{\Bb R}^m$ represents the lumped uncertainties. We assume that the uncertain element $\mb{\bm$\xi$}$ is bounded by a known continuous non-negative function $\rho$, that is,
\[
\|\mb{\bm$\xi$}(t,\mb{\bm$x$}(t))\| \le \rho(t,\mb{\bm$x$}(t)).
\]
We also assume that the equilibrium state $\mb{\bm$x$}=\mb{\bm$0$}$ of the uncontrolled nominal system
\begin{equation}
\dot{\mb{\bm$x$}}(t)=\mb{\bm$f$}(t,\mb{\bm$x$}(t))
\label{eq:nonunconguas}
\end{equation}
is globally uniformly asymptotically stable, and that we were able to construct a Lyapunov function $W:\mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ corresponding to this equilibrium state. The Lyapunov function $W$ is assumed to satisfy the following conditions: there exist three continuous and strictly increasing functions $\psi_i$, $i=1,2,3$, such that
\begin{equation}
\left.\begin{array}{rcl}
\psi_i(0) &=& 0,\quad i=1,2,3,\\
\lim_{r \to \infty} \psi_1(r)&=& \infty,\\
\psi_1(\|\mb{\bm$x$}\|) &\le& W(t,\mb{\bm$x$}) \le \psi_2(\|\mb{\bm$x$}\|),\\
\psi_3\left(\| \mb{\bm$x$}\| \right) &\le& - \dot{W}(t,\mb{\bm$x$})
\end{array}\right\}
\label{eq:thconspsis}
\end{equation}
for all $t$ and all $\mb{\bm$x$}$. We will show that if we apply the control strategy
\begin{equation}
\mb{\bm$u$}(t,\mb{\bm$x$})=\left\{\begin{array}{ccc}
-\frac{\mb{\bm$\mu$}(t,\mb{\bm$x$})}{\|\mb{\bm$\mu$}(t,\mb{\bm$x$})\|} \rho(t,\mb{\bm$x$})&\mb{if}& (t,\mb{\bm$x$}) \notin \Omega\\
\mb{\bm$0$} &\mb{if}& (t,\mb{\bm$x$})\in \Omega,
\end{array} \right.
\label{eq:minmaxcs}
\end{equation}
where $\mb{\bm$\mu$}(t,\mb{\bm$x$})=\mb{\bm$G$}^{\top}(t,\mb{\bm$x$}) \nabla_{\mb{\bm$x$}}W(t,\mb{\bm$x$})$, and $\Omega=\left\{(t,\mb{\bm$x$}):\mb{\bm$\mu$}=\mb{\bm$0$}\right\}$, then the equilibrium state $\mb{\bm$x$}=\mb{\bm$0$}$ is a globally uniformly asymptotically stable solution of the closed-loop system. For this, consider the Lyapunov function $W$ for the uncontrolled nominal system. The time derivative of $W$ evaluated on the solutions of the closed-loop system is
\begin{eqnarray}
\dot{W}(t,\mb{\bm$x$}) &=& \frac{\partial W}{\partial t}+\nabla_{\mb{\bm$x$}} W^{\top} \left( \mb{\bm$f$}(t,\mb{\bm$x$})+\mb{\bm$G$}(t,\mb{\bm$x$})\left(\mb{\bm$u$}+\mb{\bm$\xi$}(t,\mb{\bm$x$})\right)\right)\nonumber\\
&\le& -\psi_3(\| \mb{\bm$x$} \|)+ \nabla_{\mb{\bm$x$}} W^{\top} \mb{\bm$G$}(t,\mb{\bm$x$}) \mb{\bm$u$} + \nabla_{\mb{\bm$x$}} W^{\top} \mb{\bm$G$}(t,\mb{\bm$x$})\mb{\bm$\xi$}(t,\mb{\bm$x$})\nonumber\\
&\le& -\psi_3(\| \mb{\bm$x$} \|)+ \b \mu^{\top}(t,\mb{\bm$x$}) \mb{\bm$u$} + \| \mb{\bm$\mu$}(t,\mb{\bm$x$})\|\rho(t,\mb{\bm$x$}).
\label{eq:lyapderfmimx}
\end{eqnarray}
For $(t,\b x)\notin \Omega$,
\begin{eqnarray}
\dot{W}(t,\mb{\bm$x$}) &\le& -\psi_3(\| \mb{\bm$x$} \|)- \b \mu^{\top}(t,\b x) \frac{\mb{\bm$\mu$}(t,\mb{\bm$x$})}{\|\mb{\bm$\mu$}(t,\mb{\bm$x$})\|} \rho(t,\mb{\bm$x$}) + \| \mb{\bm$\mu$}(t,\mb{\bm$x$})\|\rho(t,\mb{\bm$x$})\nonumber\\
&=& -\psi_3(\| \mb{\bm$x$} \|)-\frac{\|\mb{\bm$\mu$}(t,\mb{\bm$x$})\|^2}{\|\mb{\bm$\mu$}(t,\mb{\bm$x$})\|} \rho(t,\mb{\bm$x$}) + \| \mb{\bm$\mu$}(t,\mb{\bm$x$})\|\rho(t,\mb{\bm$x$})\nonumber\\
&=& -\psi_3(\| \mb{\bm$x$} \|)- \| \mb{\bm$\mu$}(t,\mb{\bm$x$})\|\rho(t,\mb{\bm$x$})+\| \mb{\bm$\mu$}(t,\mb{\bm$x$})\|\rho(t,\mb{\bm$x$})\nonumber\\
&=& -\psi_3(\| \mb{\bm$x$} \|)\nonumber\\
&<&0.
\label{eq:noinommimx}
\end{eqnarray}
For $(t,\b x)\in \Omega$, we have $\b \mu(t,\b x)=\b 0$, and using~(\ref{eq:lyapderfmimx}), we obtain
\begin{equation}
\dot{W}(t,\mb{\bm$x$}) \le -\psi_3(\| \mb{\bm$x$} \|)< 0.
\label{eq:inmumimxc}
\end{equation}
Thus, $W$ is also a Lyapunov function for the closed-loop system, which means that the uncertain system~(\ref{eq:uncertmode}) driven by the control law~(\ref{eq:minmaxcs}) is indeed globally uniformly asymptotically stable.
\begin{example}\rm
For the uncertain system model
\[
\dot{\b x}=\b A\b x+\b \xi_1(\b x)+\b b\left(u+\xi_2(\b x, u)\right),
\]
where
\[
\b A=\left[\begin{array}{cc}
-2 & 1\\
0 & -3
\end{array}\right],\quad \b b=\left[\begin{array}{c}
0\\
1
\end{array}\right],\quad \left\Arrowvert \b \xi_1(\b x) \right\Arrowvert \le 1.75 \|\b x\|,\quad\mb{and}\quad |\xi_2(\b x, u)|\le 10\|\b x\|+0.5|u|,
\]
design stabilizing control strategy of the form~(\ref{eq:minmaxcs}).

The uncontrolled nominal system $\dot{\b x}=\b A\b x$ is asymptotically stable. Let
\[
\b Q=\left[\begin{array}{cc}
4 & -1\\
-1 & 6
\end{array}\right].
\]
The eigenvalues of $\b Q$ are $\lambda_{\min}(\b Q)=3.5858$ and $\lambda_{\max}(\b Q)=6.4142$. Solving the Lyapunov matrix equation $\b A^{\top}\b P+\b P\b A=-\b Q$ yields
\[
\b P=\b I_2.
\]
The proposed controller has the form
\begin{equation}
u=-\eta(\b x)\frac{\mu(\b x)}{|\mu(\b x)|}=-\eta(\b x)\frac{\b b^{\top} \b P\b x}{|\b b^{\top}\b P\b x|}=-\eta(\b x)\frac{\b b^{\top} \b x}{| \b b^{\top} \b x|}=-\eta(\b x)\mb{sign}\left(\b b^{\top} \b x\right).
\label{eq:conmimxex}
\end{equation}
Our goal is to determine a bound $\eta^*(\b x)>0$ such that for any $\eta(\b x)\ge\eta^*(\b x)$ the uncertain closed-loop system is globally uniformly asymptotically stable. To begin, consider
\[
V(\b x)=\b x^{\top}\b P\b x=\b x^{\top}\b x
\]
as the Lyapunov function candidate for the closed-loop system. Evaluating the time derivative of $V$ on the trajectories of the closed-loop system yields
\begin{eqnarray}
\frac{d}{dt}V(\b x) &=& 2\b x^{\top}\dot{\b x}\nonumber\\
&=&2\b x^{\top}\left( \b A\b x+\b \xi_1(\b x)+\b b\left(u+\xi_2(\b x, u)\right)\right)\nonumber\\
&=&-\b x^{\top}\b Q\b x+2\b x^{\top}\b \xi_1(\b x)+2\b x^{\top}\b b u(\b x)+2\b x^{\top}\b b\xi_2(\b x, u).
\label{eq:lyapderforsime}
\end{eqnarray}
Substituting into the above the expression for the controller given by~(\ref{eq:conmimxex}), taking the norms, and using bounds on the uncertain elements, we obtain
\begin{eqnarray}
\frac{d}{dt}V(\b x)&\le&-\lambda_{\min}(\b Q)\|\b x\|^2+2\|\b x\|\|\b \xi_1(\b x)\|-2\eta(\b x)\frac{\left\arrowvert\b x^{\top}\b b\right\arrowvert^2}{\left\arrowvert\b x^{\top}\b b\right\arrowvert} +2\left\arrowvert\b x^{\top}\b b\right\arrowvert \left\arrowvert\xi_2(\b x, u)\right\arrowvert\nonumber\\
&\le&(-3.5858+3.5)\|\b x\|^2-2\eta(\b x)\left\arrowvert\b x^{\top}\b b\right\arrowvert+2\left( 10\|\b x\|+0.5\eta(\b x)\right)\left\arrowvert\b x^{\top}\b b\right\arrowvert .
\label{eq:afsubuantr}
\end{eqnarray}
In the above, we used the fact that
\[
|u|=\eta(\b x).
\]
Collecting the terms in~(\ref{eq:afsubuantr}) yields
\begin{equation}
\frac{d}{dt}V(\b x)\le(-3.5858+3.5)\|\b x\|^2+\left( 20\|\b x\|-\eta(\b x) \right)\left\arrowvert\b x^{\top}\b b\right\arrowvert
\label{eq:aftcolltr}.
\end{equation}
Therefore, $\frac{d V}{dt}<0$ for any
\begin{equation}
\eta(\b x)\ge\eta^*(\b x)=20\|\b x\|.
\label{eq:rhgtrs}
\end{equation}
In summary, the uncertain closed-loop system will be globally uniformly asymptotically stable if the controller gain $\eta(\b x)$ is chosen to satisfy~(\ref{eq:rhgtrs}).
\end{example}\markex
We now use the Lyapunov approach to design a model-reference tracking controller. A block diagram of the model-reference control system is shown in Figure~\ref{fig:refm}.
\begin{figure}
\centering
\includegraphics[scale=0.55]{refm.eps}
%\vspace{4in}
%\centerbmp{4.67in}{3.5in}{c:/mybook/refm.bmp}
%\hskip.5in\special{bmp:c:/mybook/refm.bmp x=5.33in y=4in}
\caption{Model-reference control system block diagram with input scaling.}
\label{fig:refm}
\end{figure}
Model-reference control can be used for plants with nonlinearities and time-varying parameters. One of the assumptions that is made while designing a model-reference controller is that a plant model is in a controller companion form. Here, we propose a method for designing model-reference controllers for a class of multi-input systems that does not require a plant model to be in a specific form. The crux of our method is in an appropriate selection of the model-reference system. Consider a plant model of the form
\begin{equation}
\dot{\b x}=\b A\b x+\b B\b u,
\label{eq:liplamotc}
\end{equation}
where $\b A\in \mb{\Bb R}^{n\times n}$ and $\b B\in \mb{\Bb R}^{n\times m}$. The model-reference signal generator is described by
\begin{equation}
\dot{\b x}_d = \b A_d \b x_d + \b B_d \b r
\end{equation}
where $\b A_d \in \mb{\Bb R}^{n\times n}$, $\b B_d \in\mb{\Bb R}^{n\times m}$, and $\b r\in \mb{\Bb R}^m$ is a command signal that is chosen so that $\b x_d$ represents a desired state trajectory of the plant. The matrix $\b A_d$ of the reference system is chosen from the set
\begin{equation}
\left\{\b A_d:\b A_d=\b A-\b B\b K_d, \b K_d\in \mb{\Bb R}^{m\times n}\right\},
\end{equation}
and $\b B_d=\b B$. Thus, the reference system is state-feedback equivalent with the plant model~(\ref{eq:liplamotc}). We assume that the pair $(\b A, \b B)$ is controllable. Therefore, we can select $\b A_d$ to have its eigenvalues in prescribed locations in the open-left hand plane. To proceed further, we define the tracking error $\b e$ as
\begin{equation}
\b e(t) = \b x(t) - \b x_d(t).
\end{equation}
Our goal is to construct a state-feedback control law so that
\begin{equation}
\lim_{t \to \infty}\|\b e(t)\|=0.
\end{equation}
We now form the differential equation that describes the error dynamics. For this we evaluate the time derivative of the error vector to get
\begin{eqnarray}
\dot{\b e}(t) &=& \dot{\b x}(t) - \dot{\b x}_d(t)\nonumber\\
&=& \b A \b x(t) + \b B \b u - \b A_d \b x_d(t) - \b B \b r(t).
\label{eq:ermorefve}
\end{eqnarray}
Recall that $\b A_d = \b A - \b B \b K_d$, hence $\b A = \b A_d + \b B\b K_d$. Substituting this expression for $\b A$ into the error equation~(\ref{eq:ermorefve}), we obtain
\begin{eqnarray}
\dot{\b e}(t) &=& \b A_d(\b x(t) - \b x_d(t)) + \b B(\b u +\b K_d \b x(t) - \b r(t))\nonumber\\
&=& \b A_d \b e(t) + \b B(\b u+\b K_d\b x(t) - \b r(t)).
\label{eq:erreqftrc}
\end{eqnarray}
We will use a Lyapunov function approach in our further analysis that will result in a control law forcing the error $\b e(t)$ to go asymptotically to zero. As a Lyapunov function candidate for~(\ref{eq:erreqftrc}), we take
\begin{equation}
V(\b e)=\b e^{\top}\b P\b e,
\end{equation}
where $\b P$ solves the Lyapunov matrix equation, $\b A_d^{\top}\b P+\b P\b A_d=-\b Q$, for some $\b Q=\b Q^{\top}>0$. Because by construction $\b A_d$ is asymptotically stable, the solution to the Lyapunov matrix equation is positive definite, that is, $\b P=\b P^{\top}>0$. We next evaluate the time derivative of $V(\b e(t))$ on the trajectories of~(\ref{eq:erreqftrc}),
\begin{eqnarray}
\dot{V}(\b e) &=& 2 \b e^{\top} \b P \dot{\b e}\nonumber\\
&=& 2 \b e^{\top}\b P (\b A_d \b e + \b B(\b u + \b K_d \b x - \b r))\nonumber\\
&=& \b e^{\top}(\b A_d^{\top}\b P + \b P\b A_d) \b e +  2 \b e^{\top} \b P \b B (\b u + \b K_d \b x - \b r).
\label{eq:evldtcomi}
\end{eqnarray}
Using the Lyapunov matrix equation and taking the norms of the above yields
\begin{equation}
\dot{V}(\b e) \leq -\b e^{\top}\b Q\b e + 2\b e^{\top} \b P \b B\b u + 2\left\Arrowvert\b e^{\top} \b P \b B\right\Arrowvert \left\Arrowvert\b K_d\b x\right\Arrowvert + 2\left\Arrowvert\b e^{\top}\b P \b B\right\Arrowvert \|\b r\|.
\label{eq:doftrcmimx}
\end{equation}
If $\b e$ is such that $\b B^{\top} \b P\b e=\b 0$, then
\begin{equation}
\dot{V}(\b e) \leq -\b e^{\top}\b Q\b e < 0.
\label{eq:fctcmimx}
\end{equation}
We now analyze the case when $\b B^{\top} \b P\b e\ne\b 0$. Let the proposed control law be
\begin{equation}
\b u=-\rho(\b x,\b r)\frac{\b B^{\top}\b P\b e}{\left\Arrowvert\b B^{\top}\b P\b e\right\Arrowvert},
\label{eq:trcomimx}
\end{equation}
where $\rho(\b x,\b r)=\|\b K_d\b x\|+\|\b r\|$. Substituting~(\ref{eq:trcomimx}) into~(\ref{eq:doftrcmimx}), we obtain
\begin{eqnarray*}
\dot{V}(\b e) &\le& -\b e^{\top}\b Q\b e -2\left\Arrowvert\b e^{\top}\b P\b B\right\Arrowvert\left(\rho(\b x,r)-\left\Arrowvert\b K_d\b x\right\Arrowvert-\|\b r\|\right)\\
&=& -\b e^{\top}\b Q\b e\\
&<& 0.
\end{eqnarray*}
Since for all $\b e \in \mb{\Bb R}^n$, we have $V(\b e)>0$ and $\dot{V}(\b e)<0$, the equilibrium point $\b e=\b 0$ of the error equation~(\ref{eq:erreqftrc}) is globally uniformly asymptotically stable, which means that $\b x(t)\to \b x_d(t)$.
\begin{example}\rm
\label{ex:ummxveh}
We now apply the above tracking controller to the vehicle model derived in the Subsection~\ref{subs:vehmod}, where $\delta_r=0$. That is, we wish to construct a tracking controller for a front wheel steered vehicle. The first step is to select the poles of the model reference system. We choose the model reference system to have the following set of asymptotically stable poles:
\begin{equation}
\{-4-2j,-4+2j,-4.5,-5.0\}.
\end{equation}
The pair $(\b A,\b b)$ is controllable, hence we can find $\b k_d$ such that the eigenvalues of the matrix $\b A_d = \b A - \b b \b k_d$ are at the above selected locations. The feedback gain is
\begin{equation}
\b k_d =
\left[\begin{array}{cccc}
0.0329 &2.2854& 0.2429& -0.1653
\end{array} \right],
\end{equation}
and the resulting model reference plant matrix is
\begin{equation}
\b A_d =
\left[\begin{array}{cccc}
-6.6638 & -107.1281 & -29.6350 & 7.7479\\
0 & 0 & 1 & 0\\
-0.9205 & -65.8195 & -10.8362&  4.7603\\
-1 & -18.3 & 0 & 0
\end{array}
\right].
\end{equation}
The next step is to find $\b P$ that solves the Lyapunov equation, $\b A_d^{\top}\b P + \b P\b A_d = -\b Q$. We take $\b Q=\b I_4$. Solving the Lyapunov equation, we get
\begin{equation}
\b P =
\left[\begin{array}{cccc}
0.1145 & 0.7530 & -0.1809& -0.0966\\
0.7530 & 34.6174 & -0.2883 & -3.3437\\
-0.1809 & -0.2883 & 0.5144&  0.0522\\
-0.0966 & -3.3437& 0.0522 & 0.6216
\end{array}\right].
\end{equation}
Now that we have $\b P$, we are ready to construct the controller. We have
\begin{eqnarray}
u &=& -\rho(\b x,r)\frac{\b b^{\top}\b P\b e}{\left\arrowvert\b b^{\top}\b P\b e\right\arrowvert}\nonumber\\
&=&-(|\b k_d\b x|+|r|)\mb{sign}(\b b^{\top}\b P(\b x-\b x_d))\nonumber\\
&=&-\left(\left|\left[\begin{array}{c}
0.0329\\
2.2854\\
0.2429\\
-0.1653
\end{array}\right]^{\top}\b x \right| + |r| \right)\mbox{sign}\left(\left[\begin{array}{c}
0.1571\\
26.9931\\
6.3324\\
-3.0250\end{array}\right]^{\top}(\b x-\b x_d)\right).
\label{eq:ummxvehtr}
\end{eqnarray}
Our goal is to have the reference input signal $u_r$ to correspond to a desired vehicle path. This requires scaling the reference input. The scaling factor is
\begin{equation}
\eta=\left( \lim_{s\to 0}\b c(s\b I_4-\b A_d)^{-1}\b b\right)^{-1},
\end{equation}
where $\b c=\left[\begin{array}{cccc}
0 & 0 & 0 & 1
\end{array}\right]$. Note that $\eta$ is well defined because the matrix $\b A_d$ is asymptotically stable. In our example, $\eta=-0.1653$. Thus, we have $r=\eta u_r$. The reference input is $u_r=4h(t-1\,\mb{sec})$, where $h(t)$ is the unit step function.

The performance of the controller, for zero initial conditions, is shown in Figures~\ref{fig:ummxlad} and~\ref{fig:ummxveh}. In Figure~\ref{fig:ummxlad}, the closed-loop system output is shown in the upper subplot, while the desired output is shown in the bottom subplot. The vehicle tracks the desired path with no noticeable steady-state error. The control action is shown in Figure~\ref{fig:ummxveh}.
\begin{figure}
\centering
\includegraphics[scale=0.55]{ummxlad.eps}
%\vspace{3.5in}
%\centerbmp{4in}{3in}{c:/mybook/ummxlad.bmp}
%\hskip1in\special{bmp:c:/mybook/ummxlad.bmp x=4.67in y=3.5in}
\caption{Plots of the actual output of the closed-loop system and the desired output in Example~\ref{ex:ummxveh}.}
\label{fig:ummxlad}
\end{figure}
\begin{figure}
\centering
\includegraphics[scale=0.55]{ummxveh.eps}
%\vspace{3.5in}
%\centerbmp{4in}{3in}{c:/mybook/ummxveh.bmp}
%\hskip1.1in\special{bmp:c:/mybook/ummxveh.bmp x=4.67in y=3.5in}
\caption{Plot of the controller action versus time in Example~\ref{ex:ummxveh}.}
\label{fig:ummxveh}
\end{figure}
\end{example}\markex
The proposed tracking controller is discontinuous in the state, which results in its high activity referred to as chattering. Chattering can be reduced by using a continuous version of the above controller. The continuous controller is obtained by introducing a so-called boundary layer that smooths out the discontinuous controller. We discuss the problem  of constructing boundary layer controllers in the following section.

\section{Uniform Ultimate Boundedness}
A drawback to the discontinuous controllers, discussed in the previous section, is that rapid switching tends to excite high frequency modes of the plant and this may degrade the performance of the controller. These problems can be alleviated by adding a so called {\em boundary layer\/} to the discontinuous controller resulting in a continuous control strategy. However, there is a price for making a discontinuous controller a continuous one. The closed-loop system driven by a boundary layer controller may only be {\em uniformly ultimately bounded\/} rather than asymptotically stable. The notions we are concerned with in this section are described in the following three definitions.
\begin{defn}
The closed ball $\bar{B}_q(\b 0)=\left\{ \mb{\bm$x$}\,:\, \|\mb{\bm$x$}\| \le q \right\}$ is uniformly stable if for any $\varepsilon > q$, there is a $\delta=\delta(\varepsilon) > 0$ so that if $\|\mb{\bm$x$}(t_0) \| \le \delta$, then $\|\mb{\bm$x$}(t)\| \le \varepsilon$ for $t \ge t_0$.
\end{defn}\markdefn

For the benefit of the reader, we now re-phrase Definition~\ref{defn:unifo_bounded} of uniform boundedness from Section~\ref{ch:stability}.
\begin{defn}
The solutions of $\dot{\b x}=\b f(t, \b x)$ are uniformly bounded if for any given $d>0$, there exists $b=b(d)< \infty$ so that for each solution $\mb{\bm$x$}(t) = \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)$ with $\|\mb{\bm$x$}_0\| \le d$, we have $\|\mb{\bm$x$}(t)\| \le b$ for all $t\ge t_0$.
\end{defn}\markdefn

We next define the notion of uniform ultimate boundedness~(UUB) of a system.
\begin{defn}
The solutions of $\dot{\b x}=\b f(t, \b x)$ are uniformly ultimately bounded~(UUB) with respect to the ball $\bar{B}_q(\b 0)$ if for every $d>0$, there exists $T(q,d)$ such that for each solution $\mb{\bm$x$}(t)=\b x(t;t_0,\b x_0)$ with $\|\mb{\bm$x$}_0\| \le d$, we have $\mb{\bm$x$}(t) \in \bar{B}_q(\b 0)$ for $t \ge t_0+T(q,d)$.
\end{defn}\markdefn

We now consider a dynamic system model~(\ref{eq:uncertmode}),
\[
\dot{\mb{\bm$x$}}(t)=\mb{\bm$f$}(t,\mb{\bm$x$}(t))+\mb{\bm$G$}(t,\mb{\bm$x$}(t))\left(\mb{\bm$u$}(t)+\mb{\bm$\xi$}(t,\mb{\bm$x$}(t))\right),\qquad\mb{\bm$x$}(t_0)=\mb{\bm$x$}_0,
\]
where $\mb{\bm$x$}(t)\in \mb{\Bb R}^n$, $\mb{\bm$u$}(t)\in \mb{\Bb R}^m$, and $\mb{\bm$\xi$}(t,\mb{\bm$x$}(t))\in \mb{\Bb R}^m$ represents the lumped uncertainties. We assume that the uncertain element $\mb{\bm$\xi$}$ is bounded by a known continuous non-negative function $\rho$, that is, $\|\mb{\bm$\xi$}(t,\mb{\bm$x$}(t))\| \le \rho(t,\mb{\bm$x$}(t))$. As in Section~\ref{sec:discconuncs}, we assume that the equilibrium state $\mb{\bm$x$}=\mb{\bm$0$}$ of the uncontrolled nominal system, $\dot{\b x}(t)=\b f(t,\b x(t))$, is globally uniformly asymptotically stable, and that we were able to construct a Lyapunov function $W:\mb{\Bb R} \times \mb{\Bb R}^n \to \mb{\Bb R}$ satisfying the conditions~(\ref{eq:thconspsis}) for all $t$ and all $\mb{\bm$x$}$.

We showed in the previous section that if we apply the discontinuous control strategy
\[
\b u(t,\b x)=\left\{\begin{array}{ccc}
-\frac{\mb{\bm$\mu$}(t,\mb{\bm$x$})}{\|\mb{\bm$\mu$}(t,\mb{\bm$x$})\|} \rho(t,\mb{\bm$x$})&\mb{if}& \b{\mu}(t,\mb{\bm$x$}) \ne \b 0\\
\mb{\bm$0$} &\mb{if}& \b{\mu}(t,\mb{\bm$x$})=\b 0,
\end{array} \right.
\]
where $\mb{\bm$\mu$}(t,\mb{\bm$x$})=\mb{\bm$G$}(t,\mb{\bm$x$})^{\top} \nabla_{\mb{\bm$x$}}W(t,\mb{\bm$x$})$, then the equilibrium state $\mb{\bm$x$}=\mb{\bm$0$}$ is a globally uniformly asymptotically stable solution of the closed-loop system. However, the above controller is discontinuous in $\mb{\bm$x$}$. Following Corless and Leitmann~\cite{corless81}, we modify it to make it continuous by introducing a boundary layer. The proposed boundary layer controller is
\begin{equation}
\mb{\bm$u$}(t,\mb{\bm$x$})=\left\{\begin{array}{lcc}
-\frac{\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})}{\| \mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|} \rho(t,\mb{\bm$x$}) &\mb{if}& \| \mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$}) \|> \nu\\
\b p(t,\b x),\; \|\mb{\bm$p$}(t,\mb{\bm$x$})\| \le \rho(t,\mb{\bm$x$}) &\mb{if}& \|\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\| \le \nu,
\end{array}\right.
\label{eq:boundlayerge}
\end{equation}
where $\nu$ is the thickness of the boundary layer about the naturally defined switching surface $\left\{(t,\mb{\bm$x$}): \| \mb{\bm$\mu$}(t,\mb{\bm$x$}) \rho(t,\mb{\bm$x$}) \|=0 \right\}$, and $\b p$ is a continuous function  such that
\[
\b p=-\frac{\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})}{\| \mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|} \rho(t,\mb{\bm$x$})\quad\mb{if}\quad \|\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|=\nu.
\]
A particular example of the above type of a controller is
\begin{equation}
\mb{\bm$u$}(t,\mb{\bm$x$})=\left\{\begin{array}{ccc}
-\frac{\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})}{\| \mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|} \rho(t,\mb{\bm$x$}) &\mb{if}& \| \mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$}) \|> \nu\\
-\frac{\mb{\bm$\mu$}(t,\mb{\bm$x$})}{\nu}\rho^2(t,\mb{\bm$x$} ) &\mb{if}& \| \mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$}) \| \le \nu.
\end{array}\right.
\label{eq:bounlaspcs}
\end{equation}
We will now show that the closed-loop system driven by the boundary layer controller~(\ref{eq:boundlayerge}) is UUB with respect to a neighborhood of the zero state. This neighborhood can be made arbitrarily small by letting $\nu \to 0$. Before we state the results, we first compute the time derivative of a Lyapunov function candidate on the trajectories of the closed-loop system. Consider the Lyapunov function $W$ for the uncontrolled nominal system. The time derivative of $W$ evaluated on the solutions of the closed-loop system is
\begin{eqnarray*}
\dot{W}(t,\mb{\bm$x$}) &=& \frac{\partial W}{\partial t}+\nabla_{\mb{\bm$x$}} W^{\top} \left( \mb{\bm$f$}(t,\mb{\bm$x$})+\mb{\bm$G$}(t,\mb{\bm$x$})\left(\mb{\bm$u$}+\mb{\bm$\xi$}(t,\mb{\bm$x$})\right)\right)\\
&\le& -\psi_3(\| \mb{\bm$x$} \|)+ \nabla_{\mb{\bm$x$}} W^{\top} \mb{\bm$G$}(t,\mb{\bm$x$}) \mb{\bm$u$} + \| \mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|.
\end{eqnarray*}
We now consider two cases. The first case is when $\|\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|>\nu$. Taking into account the controller structure for this case yields
\begin{eqnarray*}
\dot{W}(t,\mb{\bm$x$}) &\le& -\psi_3(\|\mb{\bm$x$}\|) - \nabla_{\mb{\bm$x$}} W^{\top} \mb{\bm$G$}(t,\mb{\bm$x$})\frac{\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})}{\| \mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|} \rho(t,\mb{\bm$x$}) + \| \mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|\\
&=& -\psi_3(\|\mb{\bm$x$}\|) - \frac{\|\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|^2}{\|\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|} + \|\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|\\
&=& -\psi_3(\|\mb{\bm$x$}\|).
\end{eqnarray*}
If, on the other hand, $\|\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\| \le \nu$, then using~(\ref{eq:boundlayerge}), we obtain
\begin{eqnarray*}
\dot{W}(t,\mb{\bm$x$}) &\le& -\psi_3(\|\mb{\bm$x$}\|) + \nabla_{\mb{\bm$x$}} W^{\top} \mb{\bm$G$}(t,\mb{\bm$x$})\mb{\bm$u$} + \|\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|\\
&\le& -\psi_3(\|\mb{\bm$x$}\|) + \|\nabla_{\mb{\bm$x$}} W^{\top} \mb{\bm$G$}(t,\mb{\bm$x$})\|\,\|\mb{\bm$u$}\| + \|\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|\\
&\le& -\psi_3(\|\mb{\bm$x$}\|) + 2\|\mb{\bm$\mu$}(t,\mb{\bm$x$})\rho(t,\mb{\bm$x$})\|\\
&\le& -\psi_3(\|\mb{\bm$x$}\|) + 2\nu.
\end{eqnarray*}
We conclude that for all $t$ and all $\mb{\bm$x$}$, we have
\begin{equation}
\fbox{$ \displaystyle \dot{W}(t,\mb{\bm$x$}) \le -\psi_3(\|\mb{\bm$x$}\|)+ 2\nu $}
\label{eq:derinwholesp}
\end{equation}
To proceed further, we need an assumption on the function $\psi_3$; if $\psi_3$ is bounded and
\begin{equation}
\lim_{r \to \infty}\psi_3(r)=l<\infty,
\label{eq:psithrllti}
\end{equation}
then we assume
\begin{equation}
2\nu < l.
\label{eq:twonulel}
\end{equation}
\begin{proposition}
\label{prop:prac_ubd}
The solutions of the system~(\ref{eq:uncertmode}) driven by the boundary layer controller~(\ref{eq:boundlayerge}) are uniformly bounded.
\end{proposition}
{\bf Proof}\quad Let $R$ be such that
\[
\psi_3(R)=2\nu.
\]
Note that $R$ is well defined by virtue of~(\ref{eq:twonulel}) and the fact that $\psi_3$ is strictly increasing. Because $\psi_3$ is strictly increasing, we can write
\[
R=\psi^{-1}_3(2\nu).
\]
Figure~\ref{fig:psi_the} illustrates the above construction.
\begin{figure}
\centering
\includegraphics[scale=0.55]{psi_the.eps}
%\vspace{3in}
%\centerbmp{3.33in}{2.5in}{c:/mybook/psi_the.bmp}
%\hskip1.5in\special{bmp:c:/mybook/psi_the.bmp x=4in y=3in}
\caption{Finding $R$ for a given $\nu$ in the proof of Proposition~\ref{prop:prac_ubd}.}
\label{fig:psi_the}
\end{figure}

For a given $d>0$, where $\| \mb{\bm$x$}_0\| \le d$, let
\[
\tilde{d}=\max \{d,R\}.
\]
We claim that
\[
b=b(d)=\psi^{-1}_1 (\psi_2(\tilde{d})),
\]
that is, the solutions of (\ref{eq:uncertmode}) driven by the boundary layer controller~(\ref{eq:boundlayerge}) are uniformly bounded. In other words, if $\| \mb{\bm$x$}_0 \| \le d$ then $\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\|\le b$ for $t \ge t_0$. First note that $\|\mb{\bm$x$}_0 \| \le \tilde{d}$ and that $\psi_1(b)=\psi_2(\tilde{d})$. Hence, $\tilde{d} \le b$---see Figure~\ref{fig:uni_bdd} for an illustration of this construction, and thus
\begin{eqnarray}
\dot{W}(t,\mb{\bm$x$}(t)) &\le& -\psi_3 ( \| \mb{\bm$x$}(t) \| )+2\nu\nonumber\\
&=& -\psi_3 ( \| \mb{\bm$x$}(t) \| )+\psi_3(R).
\label{eq:wdolemptptr}
\end{eqnarray}
\begin{figure}
\centering
\includegraphics[scale=0.55]{uni_bdd.eps}
%\vspace{3in}
%\centerbmp{3.33in}{2.5in}{c:/mybook/uni_bdd.bmp}
%\hskip1.5in\special{bmp:c:/mybook/uni_bdd.bmp x=4in y=3in}
\caption{Finding $b$ for a given $d$ in the proof of Proposition~\ref{prop:prac_ubd}.}
\label{fig:uni_bdd}
\end{figure}
Suppose first that $d>R$. Then for $\| \mb{\bm$x$}_0 \| > R$ and $\|\b x(t;t_0,\b x_0)\|>R$, $\dot{W}<0$ and therefore
\begin{eqnarray*}
\psi_1(b) &=& \psi_2(\tilde{d})\\
&\ge& W(t_0,\b x(t_0;t_0,\mb{\bm$x$}_0))\\
&\ge& W(t,\b x(t;t_0,\mb{\bm$x$}_0))\\
&\ge& \psi_1(\| \mb{\bm$x$}(t;t_0,\b x_0)\|),
\end{eqnarray*}
which implies that $\| \mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\| \le b$ for $t \ge t_0$.

If, on the other hand, $\| \mb{\bm$x$}_0 \| \le R$, then because $\dot{W}(t,\mb{\bm$x$}(t)) \le -\psi_3 ( \| \mb{\bm$x$}(t) \| )+2\nu$, the solution may reach the sphere $\| \mb{\bm$x$} \| = R$ at some time $t_2 \ge t_0$. However, by~(\ref{eq:wdolemptptr}), $\dot{W} < 0$ outside $\| \mb{\bm$x$} \| = R$. Therefore, $\| \mb{\bm$x$}(t;t_0,\b x_0) \| \le b$. Thus the solution is uniformly bounded.\\
\hspace*{5.5in} $\Box$\\
We illustrate Proposition~\ref{prop:prac_ubd} with Figure~\ref{fig:unib_2d}.
\begin{figure}
\centering
\includegraphics[scale=0.55]{unib_2d.eps}
%\vspace{3in}
%\centerbmp{3.33in}{2.5in}{c:/mybook/unib_2d.bmp}
%\hskip1.5in\special{bmp:c:/mybook/unib_2d.bmp x=4in y=3in}
\caption{A two-dimensional illustration of an uniformly ultimately bounded~(UUB) solution.}
\label{fig:unib_2d}
\end{figure}
The initial condition in Figure~\ref{fig:unib_2d} is such that $\| \mb{\bm$x$}_0 \|< d$ and $d<R$. The trajectory is confined to stay within the ball of radius $b=\psi_1^{-1}(\psi_2(R))$.

We now show, using arguments similar to those of Corless and Leitmann~\cite{corless81}, that the solutions of the closed-loop system are UUB with respect to any ball of radius $\tilde{b}>\psi^{-1}_1(\psi_2(R))$.
\begin{thm}
Let $\mb{\bm$x$}(t)=\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)$ be a solution of the closed-loop system for $[t_0,\infty)$ such that~(\ref{eq:derinwholesp}) is satisfied. If $\| \mb{\bm$x$}_0 \| \le d$, then for a given
\begin{equation}
\tilde{b} > \psi_1^{-1}(\psi_2(R)),
\label{eq:btilpsopst}
\end{equation}
$\|\mb{\bm$x$}(t;t_0,\mb{\bm$x$}_0)\| \le \tilde{b}$ for all $t \ge t_0 + T(\tilde{b},d)$, where
\[
T(\tilde{b},d)=\left\{\begin{array}{ccc}
0 & \mb{if} & d \le \tilde{R}\\
\frac{\psi_2(d)-\psi_1(\tilde{R})}{\psi_3(\tilde{R})-2\nu}& \mb{if}& d > \tilde{R},
\end{array}\right.
\]
and
\begin{equation}
\psi_2\left(\tilde{R}\right) = \psi_1(\tilde{b}).
\label{eq:pstrtipso}
\end{equation}
\end{thm}
{\bf Proof}\quad From assumption~(\ref{eq:btilpsopst}),
\[
\psi_1(\tilde{b}) > \psi_2(R),
\]
and by~(\ref{eq:pstrtipso}),
\[
\psi_1(\tilde{b}) = \psi_2(\tilde{R}).
\]
Hence, by~(\ref{eq:thconspsis}), where we assumed that $\psi_2$ is strictly increasing,
\begin{equation}
\tilde{R} > R.
\label{eq:rtistrbir}
\end{equation}
We consider two cases. The first case is when $d \le \tilde{R}$. By uniform boundedness, for all $t \ge t_0$,
\[
\| \mb{\bm$x$}(t) \| \le \tilde{b},
\]
for any $\tilde{b}$ such that $\psi_1(\tilde{b}) > \psi_2(\tilde{R})$. Hence in this case $T(\tilde{b},d)=0$.

We now consider the case when $d>\tilde{R}$. We show, by contradiction, that for some $t_1 \in [t_0,t_0+T(\tilde{b},d)]$,
\[
\| \mb{\bm$x$}(t_1) \| \le \tilde{R}.
\]
From this we shall conclude the result by referring to the previous case. So suppose that $\| \mb{\bm$x$}(t) \| > \tilde{R}$ for all $t \in \left[t_0, t_0 + T(\tilde{b},d) \right]$. Then,
\begin{eqnarray*}
\psi_1\left( \left\Arrowvert \mb{\bm$x$}\left(t_0+T(\tilde{b},d);t_0,\mb{\bm$x$}_0\right)\right\Arrowvert \right) &\le& W \left( t_0+T(\tilde{b},d),\mb{\bm$x$}\left(t_0+T(\tilde{b},d);t_0,\mb{\bm$x$}_0\right)\right)\\
&=& W(t_0,\mb{\bm$x$}_0)+\int_{t_0}^{t_0+T(\tilde{b},d)}\dot{W}dt\\
&\le& W(t_0,\mb{\bm$x$}_0)+\int_{t_0}^{t_0+T(\tilde{b},d)}\left( - \psi_3(\tilde{R}) + 2\nu\right)dt\\
&\le& \psi_2(d) - \int_{t_0}^{t_0+T(\tilde{b},d)}\left(\psi_3(\tilde{R})-2\nu\right)dt.
\end{eqnarray*}
Using the fact that $\psi_3(\tilde{R})-2\nu > 0$ because by~(\ref{eq:rtistrbir}), $\tilde{R} > R$, we obtain
\begin{eqnarray*}
\psi_1\left(\left\Arrowvert \mb{\bm$x$}\left(t_0+T(\tilde{b},d);t_0,\mb{\bm$x$}_0\right)\right\Arrowvert \right) &\le& \psi_2(d)-\left(\psi_3(\tilde{R})-2\nu\right)T(\tilde{b},d)\\
&=& \psi_2(d)-\left(\psi_3(\tilde{R})-2\nu\right)\frac{\psi_2(d)-\psi_1(\tilde{R})}{\psi_3(\tilde{R})-2\nu}\\
&=&\psi_1\left(\tilde{R}\right).
\end{eqnarray*}
Thus, because $\psi_1$ is strictly increasing,
\[
\left\Arrowvert \mb{\bm$x$}\left(t_0+T(\tilde{b},d);t_0,\mb{\bm$x$}_0\right)\right\Arrowvert \le \tilde{R},
\]
which contradicts our assumption that for all $t \in \left[t_0, t_0 + T(\tilde{b},d) \right]$ it should have been $\| \mb{\bm$x$}(t)\| > \tilde{R}$. Hence, there exists $t_1 \in \left[t_0, t_0 + T(\tilde{b},d) \right]$ such that $\| \mb{\bm$x$}(t_1)\| \le \tilde{R}$. Then using the same argument as in the previous case we can show that
\[
\| \mb{\bm$x$}(t) \| \le \tilde{b}
\]
for all $t \ge t_1$ and the proof is complete.\\
\hspace*{5.5in} $\Box$\\
We have the following consequence of the above theorem.
\begin{corollary}\label{cor:uniball}
The ball
\[
\left\{ \mb{\bm$x$}:\|\mb{\bm$x$}\| \le \psi_1^{-1}(\psi_2(R)) \right\}
\]
is uniformly stable.
\end{corollary}
{\bf Proof}\quad Let
\[
\varepsilon > \psi_1^{-1}(\psi_2(R)),
\]
and consider $\delta=\delta(\varepsilon)$ obtained from
\[
\psi_1(\varepsilon)=\psi_2(\delta).
\]
Obviously $\delta \ge R$. Figure~\ref{fig:ub_ball} illustrates the above relations.
\begin{figure}
\centering
\includegraphics[scale=0.55]{ub_ball.eps}
%\vspace{3in}
%\centerbmp{3.33in}{2.5in}{c:/mybook/ub_ball.bmp}
%\hskip1.5in\special{bmp:c:/mybook/ub_ball.bmp x=4in y=3in}
\caption{Finding $\delta$ given $\varepsilon$ for an uniformly stable ball in the proof of Corollary~\ref{cor:uniball}.}
\label{fig:ub_ball}
\end{figure}
Hence, by uniform boundedness if $\| \mb{\bm$x$}_0 \| \le \delta$, then $\| \mb{\bm$x$}(t) \| \le \varepsilon$ for all $t \ge t_0$ which means that the ball is uniformly stable.\\
\hspace*{5.5in} $\Box$
\begin{example}\rm
Consider a dynamic system model
\[
\dot{\b x} = \b A\b x+\b b(u+h(t,\b x)),
\]
where
\[
\b A=\left[\begin{array}{cc}
0 & 1\\
-1 & -1
\end{array}\right],\quad \b b=\left[\begin{array}{c}
0\\
1
\end{array}\right],\quad\mb{and}\quad |h(t,\b x)|\le 2.
\]
Let $\b P$ be the solution of the matrix Lyapunov equation $\b A^{\top}\b P+\b P\b A=-2\b I_2$, and let
\[
u=\left\{\begin{array}{ccc}
-2\,\mb{sign}(\b b^{\top}\b P\b x)& \mb{if}& |\b b^{\top}\b P\b x| \ge \nu\\
-2\frac{\b b^{\top}\b P\b x}{\nu} & \mb{if} & |\b b^{\top}\b P\b x| < \nu,
\end{array}\right.
\]
where $\nu=1$. Suppose that the initial condition, $\b x(0)$, is such that $\|\b x(0)\|_2\le 10$. Find the radius, $b$, of a ball such that $\|\b x(t)\|_2\le b$ for all $t\ge0$.

Solving the Lyapunov equation $\b A^{\top}\b P+\b P\b A=-2\b I_2$ yields
\[
\b P=\left[\begin{array}{cc}
3 & 1\\
1 & 2
\end{array}\right].
\]
We next evaluate $\frac{d}{dt}\left( \b x^{\top}\b P\b x\right)$ on the trajectories of the closed-loop system. For the case when $\left| \b b^{\top}\b P\b x\right|\ge \nu$, we get
\[
\frac{d}{dt}\left( \b x^{\top}\b P\b x\right)\le -2\|\b x\|^2.
\]
For the case when $\left| \b b^{\top}\b P\b x\right| < \nu$, we obtain
\begin{eqnarray*}
\frac{d}{dt}\left( \b x^{\top}\b P\b x\right)&=&-2\|\b x\|^2+2\b x^{\top}\b P\b b(u+h)\\
&\le& -2\|\b x\|^2-4\frac{\left| \b x^{\top}\b P\b b\right|^2}{\nu}+4\left| \b x^{\top}\b P\b b\right|\\
&\le& -2\|\b x\|^2 + 4\nu\\
&=& -2\|\b x\|^2+4.
\end{eqnarray*}
Hence, $\frac{d}{dt}\left( \b x^{\top}\b P\b x\right)<0$ outside of the ball
\[
\bar{B}_R(\b 0)=\{\b x:\|\b x\| \le R=\sqrt{2} \}.
\]
To find the radius, $b$, of a ball, such that $\|\b x(t)\|_2\le b$ for all $t\ge 0$ and $\|\b x(0)\|_2\le 10=d$, we first observe that
\[
\lambda_{\min}(\b P)\|\b x\|^2 =\psi_1(\|\b x\|)\le \b x^{\top}\b P\b x \le \psi_2(\|\b x\|)=\lambda_{\max}(\b P)\|\b x\|^2.
\]
Let $\tilde{d}=\max\{d,R\}=10$. Then, the radius $b$ is computed by solving the equation
\[
\psi_1(b)=\psi_2(\tilde{d}),
\]
that is,
\[
\lambda_{\min}(\b P)b^2=\lambda_{\max}(\b P)\tilde{d}^2.
\]
We obtain
\[
b=\tilde{d}\sqrt{\frac{\lambda_{\max}(\b P)}{\lambda_{\min}(\b P)}}=10\sqrt{\frac{(5+\sqrt{5})/2}{(5-\sqrt{5})/2}}=16.18.
\]
\end{example}\markex
\begin{example}\rm
\label{ex:ublveh}
In this example, we investigate the performance of a boundary layer tracking controller on the tracking problem considered in Example~\ref{ex:ummxveh}. Specifically, we replace the discontinuous controller given by~(\ref{eq:ummxvehtr}) with a boundary layer continuous controller obtained by replacing the sign nonlinearity with a linear saturation nonlinearity. A linear saturation nonlinearity, denoted $\mb{sat}_{\nu}$, can be described by
\begin{equation}
\mb{sat}_{\nu}(x)=\frac{1}{2\nu}\left(|x+\nu|-|x-\nu| \right),
\label{eq:linsatu}
\end{equation}
where the parameter $\nu$ controls the width of the boundary layer. In other words,
\begin{equation}
\mb{sat}_{\nu}(x)=\left\{\begin{array}{ccl}
1&\mb{if}& x\ge \nu\\
\frac{1}{\nu}x&\mb{if}& -\nu<x<\nu\\
-1&\mb{if}& x\le -\nu.
\end{array}\right.
\label{eq:worstdesu}
\end{equation}
A plot of~(\ref{eq:linsatu}) for $\nu=0.5$ is shown in Figure~\ref{fig:saturau}.
\begin{figure}
\centering
\includegraphics[scale=0.55]{saturau.eps}
%\vspace{3.5in}
%\centerbmp{4.67in}{3.5in}{c:/mybook/saturau.bmp}
%\hskip1in\special{bmp:c:/mybook/saturau.bmp x=4.67in y=3.5in}
\caption{Plot of the linear saturation function described by~(\ref{eq:worstdesu}).}
\label{fig:saturau}
\end{figure}
A boundary layer version of the discontinuous tracking controller~(\ref{eq:ummxvehtr}) is
\begin{eqnarray}
u &=& -\rho(\b x,r)\mb{sat}_{\nu}\left(\b b^{\top}\b P\b e\right)\nonumber\\
&=&-(|\b k_d\b x|+|r|)\mb{sat}_{\nu}(\b b^{\top}\b P(\b x-\b x_d))\nonumber\\
&=&-\left(\left|\left[\begin{array}{c}
0.0329\\
2.2854\\
0.2429\\
-0.1653
\end{array}\right]^{\top}\b x \right| + |r| \right)\mbox{sat}_{\nu}\left(\left[\begin{array}{c}
0.1571\\
26.9931\\
6.3324\\
-3.0250\end{array}\right]^{\top}(\b x-\b x_d)\right).
\label{eq:blvodcmmx}
\end{eqnarray}
The plot of the actual output, $Y$, of the closed-loop system driven by~(\ref{eq:blvodcmmx}) for $\nu=0.5$ and zero initial conditions was indistinguishable from that shown in Figure~\ref{fig:ummxlad} when the discontinuous controller was used. The boundary layer controller action versus time is shown in Figure~\ref{fig:uveh_bla}. Chattering is basically eliminated in this case.
\begin{figure}
\centering
\includegraphics[scale=0.55]{uveh_bla.eps}
%\vspace{3.5in}
%\centerbmp{4.67in}{3.5in}{c:/mybook/uveh_bla.bmp}
%\hskip1in\special{bmp:c:/mybook/uveh_bla.bmp x=4.67in y=3.5in}
\caption{Plot of the boundary layer controller action versus time in Example~\ref{ex:ublveh}.}
\label{fig:uveh_bla}
\end{figure}
\end{example}\markex

\section{Lyapunov Like Analysis}
\label{sec:barbalat}
Usually, it is challenging to analyze the asymptotic stability of time-varying systems because it is very difficult to find Lyapunov functions with negative definite Lyapunov derivatives. One may ask the following question: when can we conclude asymptotic stability when the Lyapunov derivative is only negative semi-definite?

In this section, we present a number of results that attempt to answer our question. In addition these results are very useful in the analysis and design of adaptive controllers. To proceed, we review some properties of time-varying functions. In particular, we need a condition that guarantees that $\dot{f}(t) \to 0$  as $t\to \infty$.

We present some basic properties of a function $f:\mb{\Bb R} \to \mb{\Bb R}$ in the following list,
\begin{itemize}
\item Recall that $\dot{f}(t) \to 0$ does not imply that $f(t)$ has a limit as $t\to \infty$. On the other hand, $f(t)$ has a limit as $t\to \infty$ does not imply that $\dot{f}(t) \to 0$.

\item A well-known result from calculus states that if $f(t)$ is lower bounded and decreasing, that is, $\dot{f}(t) \le 0$, then it converges to a limit. But we do not know whether $\dot{f}(t) \to 0$ or not as $t\to \infty$.

\item Recall also that a function $f(t)$ is continuous if small changes in $t$ result in small changes in its value $f(t)$. Intuitively, a continuous function is a function whose graph can be drawn without lifting the pencil from the paper.

\item Continuity is a local property of a function, that is, a function $f$ is continuous, or not, at a particular point $t$.

\item A function being continuous on an interval means only that it is continuous at each point of the interval.

\item A function $f$ is uniformly continuous if it is continuous and, in addition, the size of the changes in $f(t)$ depends only on the size of the changes in $t$ but not on $t$ itself.

\item The slope of an uniformly continuous function is bounded, that is, $\dot{f}$ is bounded.

\item Uniform continuity is a global property of a function.

\item Every uniformly continuous function is continuous, but the converse is not true.

\item A function is uniformly continuous, or not, on an entire interval.

\item A function may be continuous at each point of an interval without being uniformly continuous on the entire interval.
\end{itemize}

\begin{example}\rm
The function $f(t)=\sin (t)$ is uniformly continuous. Note that the slope of the above function is bounded.
\end{example}\markex

\begin{example}\rm
The function $f(t)=1/t$ is continuous but not uniformly continuous on positive real numbers. Note that as $t$ approaches $0$, the changes in $f(t)$ become unbounded.
\end{example}\markex

In the next example from Slotine and Li~\cite[p.~124]{slotine91}, we show that in an asymptotically stable LTI system with bounded input, the output is uniformly continuous.
\begin{example}\rm
We consider an asymptotically stable LTI system
\begin{eqnarray*}
\dot{\b x} &=& \b A\b x + \b u\\
\b y &=& \b C \b x.
\end{eqnarray*}
The state $\b x(t)$ is bounded because $\b u$ is bounded and $\b A$ is asymptotically stable. Because $\b x$ is bounded and $\b u$ is bounded, we have that $\dot{\b x}$ is bounded as well. We now consider the time derivative of the output,
\[
\dot{\b y}=\b C\dot{\b x}.
\]
The time derivative of $\b y$ is bounded because $\dot{\b x}$ is bounded. Hence, $\b y$ is uniformly continuous.
\end{example}\markex

With the above background, we are now ready to state and prove the following lemma.
\begin{lemma}[Barbalat]
\label{lemma:barbalats}
Given a differentiable function $f=f(t):\mb{\Bb R} \to \mb{\Bb R}$ such that $\dot{f}$ is uniformly continuous. If $\lim_{t \rightarrow \infty}{{f}(t)}$ exists, then
\[
\lim_{t \rightarrow \infty}{\dot{f}(t)}=0.
\]
\end{lemma}
{\bf Proof}\quad We prove the lemma by contraposition. We assume that $\dot{f}(t)$ does not converge to $0$ as $t\to \infty$. Therefore, there exists an $\varepsilon > 0$ such that for every $T>0$ we can find $t_i\ge T$ so that $|\dot{f}(t_i)|\ge \varepsilon$. By assumption $\dot{f}$ is uniformly continuous. Hence, given $\varepsilon/2>0$ there is $\delta_{\varepsilon/2}>0$ so that for every $t_i$ and every $\tau\in\left[0 ,  \delta_{\varepsilon/2}\right]$, we have
\begin{equation}
\left| \dot{f}(t_i+\tau )-\dot{f}(t_i )\right|<\frac{\varepsilon}{2}.
\label{eq:ttileeph}
\end{equation}
Hence, for all $t\in \left[t_i, t_i+ \delta_{\varepsilon/2} \right]$, we have
\begin{eqnarray*}
\left|\dot{f}(t)\right| &=& \left|\dot{f}(t)-\dot{f}(t_i)+\dot{f}(t_i)\right|  \\
&\ge & \left|\dot{f}(t_i)\right|-\left|\dot{f}(t)-\dot{f}(t_i)\right|\\
&>& \varepsilon-\frac{\varepsilon}{2}\\
&=&\frac{\varepsilon}{2}.
\end{eqnarray*}
Suppose that $\dot{f}(t_i) > 0$. (The other case is when $\dot{f}(t_i)< 0$ and its analysis is similar to the case we analyze now.) It follows from the above that for any $t\in \left[t_i, t_i+ \delta_{\varepsilon/2} \right]$,
\[
\dot{f}(t) > \frac{\varepsilon}{2}.
\]
Using the above, we obtain
\begin{equation}
\int_{t_i}^{t_i+\delta_{\varepsilon/2}}\dot{f}(t)dt> \int_{t_i}^{t_i+\delta_{\varepsilon/2}}\frac{\varepsilon}{2}dt
= \frac{\varepsilon}{2}\delta_{\varepsilon/2}.
\label{eq:ittitipde}
\end{equation}
However,
\begin{equation}
\int_{t_i}^{t_i+\delta_{\varepsilon/2}}\dot{f}(t)dt=f(t_i+\delta_{\varepsilon/2})-f(t_i).
\label{eq:ftipdmfti}
\end{equation}
Combining~(\ref{eq:ittitipde}) and(\ref{eq:ftipdmfti}) yields
\[
f(t_i+\delta_{\varepsilon/2})-f(t_i)>\frac{\varepsilon}{2}\delta_{\varepsilon/2}\quad\mb{for every}\quad i,
\]
which contradicts the assumption that $f(t)$ has a finite limit as $t \to \infty$. The proof of the lemma is complete.\\
\hspace*{5.5in} $\Box$

A simple sufficient condition for a differentiable function to be uniformly continuous is boundedness of its derivative. Thus if a function $f$ is twice differentiable, then its derivative $\dot{f}$ is uniformly continuous if its second derivative $\ddot{f}$ is bounded.

The following lemma, called by Slotine and Li~\cite[page~125]{slotine91}  the ``Lyapunov-like lemma'', follows immediately from Barbalat's lemma.
\begin{lemma}[``Lyapunov-Like Lemma'']
\label{lemma:lyapunov_like}
Given a real-valued function $W(t,\b x)$ such that
\begin{enumerate}
\item $W(t,\b x)$ is bounded below;
\item $\dot{W}(t,\b x)$ is negative semi-definite;
\item $\dot{W}(t,\b x)$ is uniformly continuous in time,
\end{enumerate}
then
\[
\dot{W}(t,\b x) \to 0\quad\mb{as}\quad t\to \infty.
\]
\end{lemma}
\begin{example}\rm
Investigate stability of the following dynamical system model:
\begin{equation}
\left[\begin{array}{c}
\dot{x}_1\\
\dot{x}_2
\end{array}\right]=\left[\begin{array}{c}
-2x_1+x_2u\\
-x_1u
\end{array}\right],
\label{eq:exforbarb}
\end{equation}
where the input $u$ is bounded.

Consider the Lyapunov function candidate
\[
V=\|\b x\|^2=x_1^2+x_2^2.
\]
Note that $V$ is bounded below. The time derivative of $V$ evaluated on the trajectories of the system model~(\ref{eq:exforbarb}) is
\[
\dot{V}=2x_1\dot{x}_1+2x_2\dot{x}_2=2x_1\left( -2x_1+x_2u\right)-2x_2x_1u=-4x_1^2 \le 0.
\]
Thus, the system modeled by~(\ref{eq:exforbarb}) is uniformly stable, and hence $x_1$ and $x_2$ are bounded. We now use Lemma~\ref{lemma:lyapunov_like} to show that in addition, $x_1(t) \to 0$ as $t\to \infty$. To use this lemma, we have to show that $\dot{V}$ is uniformly continuous in $t$. For this it is enough to show that the derivative of $\dot{V}$ is bounded. The derivative of $\dot{V}$ is
\[
\ddot{V}=-8x_1\dot{x}_1=-8x_1\left(-2x_1+x_2u \right),
\]
which is bounded because $u$ is bounded by assumption and we showed that $x_1$ and $x_2$ are also bounded. Hence, $\dot{V}$ is uniformly continuous. The function $V$ satisfies all the conditions of Lemma~\ref{lemma:lyapunov_like} and therefore $x_1(t) \to 0$ as $t\to \infty$.
\end{example}\markex
\begin{example}\label{ex:lyap_like_l}\rm
Consider a dynamical system model,
\begin{equation}
\left.
\begin{array}{lll}
\dot{\b x} & = & \b A\b x +\b b u\\
e &=& \b c\b x
\end{array}
\right\}
\label{eq:moforadap}
\end{equation}
where
\begin{equation}
u=\b{\phi}^{\top} \b{\xi}
\label{eq:daptcontrol}
\end{equation}
The components $\phi_i=\phi_i(t)$ of the vector-valued function $\b{\phi}=\b{\phi}(t)\in \mb{\Bb R}^K$ are adjustable parameters according to the adaptation law
\begin{equation}
\dot{\b{\phi}}=-\b{\Gamma}e\b{\xi},
\label{eq:adaplawex}
\end{equation}
where $\b{\Gamma}=\b{\Gamma}^{\top}>0$ is a design parameter matrix. The vector-valued function $\b{\xi}=\b{\xi}(t)\in \mb{\Bb R}^K$ is bounded for all $t$. The matrix $\b A \in \mb{\Bb R}^{n\times n}$ is asymptotically stable, and $\b b\in \mb{\Bb R}^n$. The row vector $\b c\in \mb{\Bb R}^{1\times n}$ satisfies the equation
\begin{equation}
\b c=\b b^{\top}\b P,
\label{eq:conforc}
\end{equation}
where $\b P=\b P^{\top}>0$ is the solution of the matrix Lyapunov equation
\[
\b A^{\top}\b P+\b P\b A=-2\b Q
%\label{eq:layafordaptlaw}
\]
for some $\b Q=\b Q^{\top}>0$. We will use the Lyapunov function candidate,
\begin{equation}
V(\b x, \b{\phi})=\frac{1}{2}\b x^{\top}\b P\b x+\frac{1}{2}\b{\phi}^{\top}\b{\Gamma}^{-1}\b{\phi}
\label{eq:layfuncanforada}
\end{equation}
and the Lyapunov-like lemma to show that $\b x$ and $\b{\phi}$ are both bounded and that
\[
\lim_{t\to \infty}\b x(t)=\b 0.
\]
We first evaluate the time derivative of the Lyapunov function candidate $V=V(\b x, \b{\phi})$ on the trajectories of the closed-loop system,
\begin{equation}
\left.
\begin{array}{lll}
\dot{\b x} & = & \b A\b x +\b b(\b{\phi}^{\top}\b{\xi})\\
\dot{\b{\phi}} &=& -\b{\Gamma} e \b{\xi}
\end{array}
\right\}
\label{eq:sclos_loo_adap}
\end{equation}
We have
\begin{eqnarray*}
\dot{V} &=& \b x^{\top}\b P\dot{\b x}+\b{\phi}^{\top}\b{\Gamma}^{-1}\dot{\b{\phi}}\\
&=& \b x^{\top}\b P\left(\b A\b x + \b b\left(\b{\phi}^{\top}\b{\xi}\right)\right)-\b{\phi}^{\top} e \b{\Gamma}^{-1}\b{\Gamma}\b{\xi}\\
&=& -\b x^{\top}\b Q\b x + \left(e\b{\phi}^{\top}-e\b{\phi}^{\top} \right)\b{\xi}\\
&=& -\b x^{\top}\b Q\b x,
\end{eqnarray*}
where we used the fact that $\b c=\b b^{\top}\b P$ and hence $e=\b c\b x=\b x^{\top}\b c^{\top}=\b x^{\top} \b P\b b$. Thus, $\dot{V}$ is negative semi-definite in the $\left[\begin{array}{c}
\b x\\
\b{\phi}
\end{array}\right]$-space, which implies that $\b x$ and $\b{\phi}$ are bounded.

To show the convergence of $\b x$, we use the Lyapunov-like lemma. For this, we first show that $\dot{V}$ is uniformly continuous. Indeed, because
\[
\ddot{V}=-2\b x^{\top}\b Q\dot{\b x}
\]
is bounded, since $\b x$ and $\dot{\b x}$ are bounded, $\dot{V}$ is uniformly continuous. Hence, $\dot{V}\to 0$ as $t\to \infty$. Since $\dot{V}=-\b x^{\top}\b Q\b x$ and $\b Q=\b Q^{\top}>0$, we have to have $\b x(t)\to \b 0$ as $t\to \infty$.
\end{example}\markex
We can re-state Barbalat's lemma in alternative form as follows (see, for example,~\cite[page~192]{khalil96}).
\begin{lemma}
\label{lemma:barb_alter}
Let $\phi:\mb{\Bb R} \to \mb{\Bb R}$ be a uniformly continuous function on $[0,\infty )$. Suppose that $\lim_{t\to \infty} \int_0^{t} \phi(\tau)d \tau$ exists and is finite. Then,
\[
\phi (t) \to 0\quad\mb{as}\quad t\to \infty .
\]
\end{lemma}

%\section{LaSalle's Invariance Principle}
%In this section, we generalize the Lyapunov function concept using LaSalle's invariant set. We first present this generalization, following the exposition in~\cite{lasalle86}, for nonlinear dynamic systems modeled by the difference equation
%\begin{equation}
%\b x(k+1)=\b f(\b x(k)),
%\label{eq:discsyswbf}
%\end{equation}
%where $k\in {\cal Z}_{+}$ with ${\cal Z}_{+}$ being the set of nonnegative integers, $\b x:{\cal Z}_{+}\to \mb{\Bb R}^n$, that is, $\b x(k)$ is the state of the system model~(\ref{eq:discsyswbf}) at time $k$, and $\b f:\mb{\Bb R}^n \to \mb{\Bb R}^n$. We assume that the vector-valued function $\b f$ is continuous on $\mb{\Bb R}^n$, that is, if $\b x^j \to \b y$ then $\b f(\b x^j) \to \b f(\b y)$. If $\b x$ is a vector, then $\b f(\b x)$ is a vector and is the value of the function $\b f$ at $\b x$.
%
%%If $\b x$ is a function, the symbol $\b f \b x$ denotes the composition of %functions
%%\[
%%\left(\b f \b x\right)(k)=\b f(\b x(k)).
%%\]
%%Thus the equation $\b x(k+1)=\b f(\b x(k))$ can also be written as $\b %x(k+1)=\b f \b x(k)$.
%
%The solution to the initial value problem
%\[
%\b x(k+1)=\b f (\b x(k)), \quad \b x(0)=\b x^0
%\]
%is
%\[
%\b x(k)= \b x^k=\b f^k\left(\b x^0\right),
%\]
%where $\b f^k$ is the $k$-th iterate of $\b f$, that is, $\b f^0=\b I_n$, and $\b f^k=\b f \b f^{k-1}$.
%
%A state ${\b x}_e \in \mb{\Bb R}^n$ is a {\em fixed point\/} or an {\em equilibrium state\/} of the system $\b x(k+1)=\b f (\b x(k))$ if ${\b x}_e=\b f\left({\b x}_e\right)$. Thus the solution of the system that starts at ${\b x}_e$ stays at ${\b x}_e$ for all $k\ge 0$.
%
%A subset ${\cal A} \subset \mb{\Bb R}^n$ is called {\em positively invariant\/} if $\b f({\cal A}) \subset {\cal A}$, that is, if $\b x \in {\cal A}$ then $\b f(\b x) \in {\cal A}$. It is clear that an equilibrium solution is invariant. The adjective ``positive'' is added to emphasize the fact that we will be interested in the behavior of the system for $k\ge 0$.
%
%The solution through $\b x^0$ is bounded if there is a constant $M$ such that $\|\b f^k\left(\b x^0\right)\| \le M$ for all $k \ge 0$.
%
%Consider again a particular solution $\b f^k\left(\b x^0\right)$. A point $\b y \in \mb{\Bb R}^n$ is a {\em positive limit point\/} or a {\em positive point of accumulation\/} of $\b f^k\left(\b x^0\right)$ if there is a subsequence $k_i$ such that $\b f^{k_i}\left(\b x^0\right) \to \b y$. The set of all positive limit points of $\b f^k\left(\b x^0\right)$ is called the {\em positive limit set\/} and denoted $\Omega\left(\b x^0\right)$.
%
%A sequence $\{\b x^k\}$ is said to approach a set ${\cal A}$ if
%\[
%\mb{dist}\left(\b x^k,{\cal A}\right)=\inf \left\{\|\b x^k-\b y\|:\b y \in{\cal A} \right\} \to 0,
%\]
%where $\mb{dist}(\b p,{\cal S})$ denotes the distance from a point $\b p$ to a set $\cal S$.
%\begin{lemma}
%\label{lemma_lim_set}
%If $\b f$ is continuous and $\b f^k\left(\b x^0\right)$ is bounded for all $k \ge 0$, then $\Omega\left(\b x^0\right)$ is non-empty, compact, positively invariant set. Furthermore, $\b f^k\left(\b x^0\right) \to \Omega\left(\b x^0\right)$ as $k \to \infty$.
%\end{lemma}
%{\bf Proof}\quad Because the sequence $\left\{\b x^k\right\}$ is bounded, by the Bolzano-Weierstrass theorem, stated on page~\pageref{thm:bolzweierstr}, there is at least one positive limit point of $\left\{\b x^k\right\}$ and so $\Omega\left(\b x^0\right)$ is non-empty and bounded.
%
%We now show that the set $\Omega\left(\b x^0\right)$ is closed, that is, if $\b y^k \to \b y$ with $\b y^k \in \Omega\left(\b x^0\right)$, then $\b y\in \Omega\left(\b x^0\right)$. Because $\Omega\left(\b x^0\right)$ is non-empty and bounded, there is a sequence, say $\{\b y^l\}$, $\b y^l\in \Omega\left(\b x^0\right)$, such that $\b y^l\to \b y$. Because each $\b y^l$ is in $\Omega\left(\b x^0\right)$, for each $\b y^l$ there is a subsequence, $\{\b x_l^{k_i}\}$, of $\{\b x^k\}$  such that $\b x_l^{k_i} \to \b y^l$. The sequences $\{\b x_l^{k_i} \}$ are convergent. Therefore, we can construct a sequence $\{\b x_l\}$, a subsequence of $\{\b x^k\}$, such that for any $\varepsilon >0$ there exist $N_1$ so that
%\[
%\| \b x_l-\b y^l \| < \varepsilon/2\quad\mb{for}\quad l\ge N_1.
%\]
%The construction of such a sequence can be accomplished as follows. For $l=1$ select a term $\b x_1$ from $\{\b x_1^{k_i}\}$ such that
%\[
%\| \b x_1-\b y^1 \| < 1.
%\]
%Next, for $l=2$ select a term $\b x_2$ form $\{\b x_2^{k_j}\}$ such that
%\[
%\| \b x_2-\b y^2 \| < 1/2,
%\]
%and so on. The sequence $\{ \b y^l\}$ is also convergent. Hence, for any $\varepsilon >0$ there is $N_2$ such that
%\[
%\|\b y^l-\b y\|< \varepsilon/2\quad\mb{for}\quad l\ge N_2.
%\]
%Combining the above gives
%\[
%\|\b x_l-\b y\|\le \|\b x_l-\b y^l\|+\|\b y^l-\b y\|< \varepsilon \quad\mb{for}\quad l\ge \max\{N_1,N_2\},
%\]
%which implies that $\b y\in \Omega\left(\b x^0\right)$. That is, the limit of any convergent sequence with elements in $\Omega\left(\b x^0\right)$ is also in this set, which means that the set $\Omega\left(\b x^0\right)$ is closed. Because $\Omega\left(\b x^0\right)$ is also bounded, it is compact.
%
%We now show that $\Omega\left(\b x^0\right)$ is positively invariant.  For this let $\b y \in \Omega\left(\b x^0\right)$. Hence, there is a subsequence $k_i$ such that $\b x^{k_i} \to \b y$. Because $\b f$ is continuous, $\b x^{k_i + 1} = \b f\left(\b x^{k_i}\right) \to \b f(\b y)$ as $i \to \infty$. Therefore $\b f(\b y) \in \Omega\left(\b x^0\right)$ which implies that $\Omega\left(\b x^0\right)$ is invariant. Note that we do not have to have $\b y=\b f(\b y)$.
%
%Finally, we will show that $\b f^k\left(\b x^0\right) \to \Omega\left(\b x^0\right)$ as $k \to \infty$. We shall prove the statement by contradiction. First observe that $\mb{dist}\left(\b x^k,\Omega\left(\b x^0\right)\right)$ is bounded because both $\left\{\b x^k\right\}$ and $\Omega\left(\b x^0\right)$ are. Assume that $\mb{dist}\left(\b x^k,\Omega\left(\b x^0\right)\right)$ does not converge to zero. Then, there is a subsequence $k_i$ such that $\b x^{k_i} \to \b y$ and $\mb{dist}\left(\b x^{k_i},\Omega\left(\b x^0\right)\right) \to a>0$. But since $\b y \in \Omega\left(\b x^0\right)$,
%\[
%\mb{dist}\left(\b x^{k_i},\Omega\left(\b x^0\right)\right)\le \|\b x^{k_i}-\b y\| \to 0,
%\]
%and thus $\mb{dist}\left(\b x^{k_i},\Omega\left(\b x^0\right)\right) \to 0$ which is a contradiction. The proof of the lemma is now complete.\\
%\hspace*{5.5in} $\Box$
%
%
%Let $V:\mb{\Bb R}^n \to \mb{\Bb R}$. Define the {\em Lyapunov difference\/} of $V$ relative to $\b x(k+1)=\b f (\b x(k))$, where $\b x(k)\in{\cal G} \subseteq \mb{\Bb R}^n$, as
%\[
%\Delta V(\b x)=V\left(\b f(\b x)\right)-V(\b x),
%\]
%that is,
%\[
%\Delta V(\b x(k))=V\left(\b x(k+1)\right)-V(\b x(k)).
%\]
%If $\Delta V(\b x(k))\le 0$, then $V$ is non-increasing along solutions of the system.
%
%We say that $V$ is a {\em Lyapunov\/} function for $\b x(k+1)=\b f (\b x(k))$ on a set ${\cal G}$ if
%\begin{enumerate}
%\item $V$ is continuous on $\mb{\Bb R}^n$, and
%\item $\Delta V(\b x(k))\le 0$ for all $\b x(k) \in {\cal G}$.
%\end{enumerate}
%A $c$-level surface of $V$ is the set
%\[
%V^{-1}(c)=\{\b x:V(\b x)=c,\; \b x\in \mb{\Bb R}^n\}.
%\]
%\begin{thm}[LaSalle's invariance principle]
%If
%\begin{enumerate}
%\item $V$ is a Lyapunov function for $\b x(k+1)=\b f (\b x(k))$ in a compact set ${\cal G}$, and
%\item the solution $\b f^k\left(\b x^0\right)$ is bounded and in ${\cal G}$,
%\end{enumerate}
%then, there is a number $c$ such that $\b f^k\left(\b x^0\right) \to {\cal M} \cap V^{-1}(c)$, where ${\cal M}$ is the largest positively invariant set contained in the set ${\cal E}=\left\{\b x \in \mb{\Bb R}^n:\Delta V = 0\right\}$.
%\end{thm}
%{\bf Proof} \quad Because $\b x^k=\b f^k\left(\b x^0\right)$ is bounded and in ${\cal G}$, by Lemma~\ref{lemma_lim_set} the positive limit set $\Omega\left(\b x^0\right)$ is non-empty and in ${\cal G}$. Furthermore, $\b x^k$ tends to $\Omega\left(\b x^0\right)$. Now $\left\{V\left(\b x^k\right)\right\}$ is non-increasing and bounded below, so $V\left(\b x^k\right) \to c$ for some $c$. If $\b y \in \Omega\left(\b x^0\right)$, then there is a subsequence $k_i$ such that $\b x^{k_i} \to \b y$, and hence $V\left(\b x^{k_i}\right) \to V(\b y)$, that is, $V(\b y)=c$. Thus $V\left(\Omega\left(\b x^0\right) \right)=c$, and hence $\Omega\left(\b x^0\right) \subset V^{-1}(c)$. Because $V\left(\Omega\left(\b x^0\right) \right)=c$ and $\Omega\left(\b x^0\right)$ is positively invariant, we have for arbitrary $\b y\in \Omega\left(\b x^0\right)$,
%\[
%\Delta V(\b y)=V(F(\b y))-V(\b y)=c-c=0,
%\]
%that is, $\Delta V\left(\Omega\left(\b x^0\right)\right)=0$. In summary,
%\[
%\b x^k \to \Omega\left(\b x^0\right) \subset \left\{\b x \in \mb{\Bb R}^n: \Delta V=0 \right\}\cap V^{-1}(c).
%\]
%Because $\Omega\left(\b x^0\right)$ is positively invariant we have $\Omega\left(\b x^0\right) \subset {\cal M}$, and the proof of the theorem is complete.\\
%\hspace*{5.5in} $\Box$\\
%
%In Figure~\ref{fig:lasalle_2009}, we illustrate the above theorem.
%\begin{figure}
%\centering
%\includegraphics[scale=0.4]{lasalle_2009.eps}
%\caption{Illustration of LaSalle's invariance principle.}
%\label{fig:lasalle_2009}
%\end{figure}
%%Using the preceding theorem we can study stability properties of difference equations.
%
%LaSalle's invariance principle can be used to prove a result concerning the instability of equilibrium states.
%%An equilibrium point ${\b x}_e$ is isolated if there is an $\varepsilon >0$ %such that there is no other equilibrium point, other than ${\b x}_e$, in %the ball $B_{\varepsilon}\left({\b x}_e\right)=\left\{\b x:\|\b x-\b %x_e\|<\varepsilon \right\}$.
%\begin{thm}
%Let ${\b x}_e$ be an equilibrium state of $\b x(k+1)=\b f (\b x(k))$, where $\{\b x(k)\}$ is bounded. Let $\Delta V$ be positive definite with respect to ${\b x}_e$, and let $V$ take positive values arbitrarily close to ${\b x}_e$, and $V\left({\b x}_e\right)=0$, then ${\b x}_e$ is unstable.
%\end{thm}
%{\bf Proof} \quad We prove the theorem by contradiction.
%%We thus assume that ${\b x}_e$ is stable. Let $\varepsilon > 0$ be %sufficiently small so that $\Delta V(\b x) >0$ for all $\b x \in %B_{\varepsilon}\left({\b x}_e\right)\setminus \left\{{\b x}_e\right\}$. %Because by assumption ${\b x}_e$ is stable, there is $\delta > 0$ so that %if $\b x^0 \in B_{\delta}\left({\b x}_e\right)$ then $\b x^k = \b %F^k\left(\b x^0\right) \in B_{\varepsilon}\left({\b x}_e\right)$ for all $k %\ge 0$. By assumption there is a point $\b x^0$ such that $V\left( \b x^0 %\right) >0$. Since $\b x^k$ is bounded and remains in %$B_{\varepsilon}\left({\b x}_e\right)$, by Lemma~\ref{lemma_lim_set} the %sequence $\left\{\b x^k\right\}$ tends to $\left\{\b x:\Delta V(\b x) = 0 %\right\} \cap B_{\varepsilon}\left({\b x}_e\right)={\b x}_e$.
%%Another way to arrive at the above observation is as follows.
%First observe that $-V$ is a Lyapunov function because $\Delta (-V)<0$. Because $\b x^k=\b f^k\left(\b x^0\right)$ is bounded, we can apply the invariance principle to conclude that $\b x^k$ tends to $\left\{ \b x:\Delta (-V(\b x))=0 \right\}=\left\{{\b x}_e\right\}$, that is, $\b x^k \to {\b x}_e$. Using the above, we conclude that $V\left(\b x^k\right) \to V\left({\b x}_e \right) = 0$. However, $\Delta V\left(\b x^k\right) > 0$ and so
%\[
%V\left(\b x^k\right) > V\left(\b x^{k-1}\right)> \cdots > V\left(\b x^0\right) > 0.
%\]
%Hence, for some positive $\varepsilon$, we have $V\left(\b x^k\right)>\varepsilon >0$ and thus we would have to have $V(\b x_e)>0$. The above contradiction completes the proof.\\
%\hspace*{5.5in} $\Box$
%
%We now present LaSalle's invariance principle for continuous-time dynamic systems models of the form,
%\begin{equation}
%\dot{\b x}=\b f(\b x),
%\label{eq:lasalcont}
%\end{equation}
%where $\b f:{\cal D} \to \mb{\Bb R}^n$ with ${\cal D}\subseteq \mb{\Bb R}^n$ being open and connected. The function $\b f$ is assumed to be locally Lipschitz on ${\cal D}$, that is, each point of ${\cal D}$ has a neighborhood such that $\b f$ satisfies the Lipschitz condition in this neighborhood with some Lipschitz constant. Each neighborhood can have a different Lipschitz constant. The above assumptions are made in order to ensure the existence and uniqueness of a solution to~(\ref{eq:lasalcont}) for a given initial condition $\b x(0)$.
%
%To proceed, we need a few definitions. Suppose that $\b x(t)$ is a solution of~(\ref{eq:lasalcont}). Then, a point $\b p$ is a {\em positive limit point\/} of $\b x(t)$ if there is a sequence $\{t_n\}$ with $t_n\to \infty$ as $n\to \infty$ such that $\b x(t_n)\to \b p$ as $n\to \infty$. The set of all positive limit points of $\b x(t)$ is called the positive limit set of $\b x(t)$. A set $\cal A$ is a positively invariant set with respect to~(\ref{eq:lasalcont}) if
%\[
%\b x(0)\in {\cal A}\quad\mb{implies}\quad \b x(t) \in {\cal A}\quad\mb{for all}\quad t\ge 0.
%\]
%We say that $\b x(t)$ approaches a set $\cal A$, as $t\to \infty$, if for each $\varepsilon > 0$ there is $T > 0$ such that
%\[
%\mb{dist}(\b x(t),{\cal A}) < \varepsilon \quad\mb{for all}\quad t > T.
%\]
%Employing arguments similar to those used in the proof of Lemma~\ref{lemma_lim_set}, we can prove the following lemma.
%\begin{lemma}
%If a solution $\b x(t)$ to~(\ref{eq:lasalcont}) is bounded and belongs to $\cal D$ for $t\ge 0$, then its positive limit set $\Omega$ is non-empty, compact, and invariant. Furthermore,
%\[
%\b x(t) \to \Omega\quad\mb{as}\quad t \to \infty.
%\]
%\end{lemma}
%Using the above lemma, we can prove the following version of the LaSalle's invariance principle for the continuous-time case:
%\begin{thm}
%If $V:\mb{\Bb R}^n\to \mb{\Bb R}$ is a continuously differentiable function such that $\dot{V}(x)\le 0$ on the solutions of~(\ref{eq:lasalcont}) that reside in a compact set ${\cal G}\subset {\cal D}$, then there is a number $c$ such that $\b x(t) \to {\cal M} \cap V^{-1}(c)$, where ${\cal M}$ is the largest positively invariant set contained in the set ${\cal E}=\left\{\b x \in \mb{\Bb R}^n:\dot{V} = 0\right\}$.
%\end{thm}
%For further discussion of the continuous-time LaSalle's invariance principle, we refer to Khalil~\cite[pages~114--115]{khalil96}. Applications of LaSalle's invariance principle to the stability analysis of various neural networks can be found in~\cite{varga96,chan_info99}, while in~\cite{khalil_p96,seshagiri00} positively invariant sets were used to analyze stability of a class of nonlinear systems driven by adaptive feedback controllers.

\section{Notes}
A classical text on stability of dynamical systems is by Hahn~\cite{hahn63}. Stability of discrete dynamical systems is presented by Hahn in~\cite{hahn58} and LaSalle~\cite{lasalle86}. Extensive coverage of stability issues can also be found in Willems~\cite{willems70}, Slotine and Li~\cite{slotine91}, Vidyasagar~\cite{vidyasagar93}, Krsti\'{c}, Kanellakopoulos, and Kokotovi\'{c}~\cite{krstic95}, and Khalil~\cite{khalil96}.

There is an interesting story behind the origins of the Routh test. The problem of investigating the stability of a linear dynamical system in terms of the location of the roots of its characteristic equation was posed by Maxwell~\cite{maxwell68}. On page~271 of his 1868 paper, Maxwell writes, ``This condition is mathematically equivalent to the condition that all the possible roots, and all the possible parts of the impossible roots, of a certain equation shall be negative.'' By impossible roots, Maxwell means complex roots, and, by possible parts, means real parts in today's terminology. He continues, ``I have not been able completely to determine these conditions for equations of a higher degree than the third; but I hope that the subject will obtain the attention of mathematicians.'' A complete general solution to the problem was published by Routh in 1877. When Maxwell and Routh were classmates at Cambridge University in 1854, Routh was placed first in their final math examination ahead of Maxwell. A story says (Smith~\cite[page~55]{smith96}) that apparently Maxwell was so sure of coming in first on this exam that he did not even bother to rise early to hear the exam results read out in the Senate House. Maxwell sent his servant instead. When the servant came back, Maxwell asked, ``Well, tell me who's second!''. Obviously, he did not expect to receive the reply, ``You are, sir''. In 1877, Routh received the prestigious Adams Prize for solving the problem posed by Maxwell in 1868. According to an account by Truxal~\cite[page~527]{truxal72}, Routh presented his solution with the opening remark, ``It has recently come to my attention that my good friend James Clerk Maxwell has had difficulty with a rather trivial problem... .''

\section{Exercises}
\begin{exercise}\rm\begin{itemize}\item[\bf (a)] Consider a negative unity feedback closed-loop system whose open-loop transfer function is
\[
G(s)=\frac{6s^3+11s^2}{s^4+6s+K}.
\]
For what values of the parameter $K$ the closed-loop system is asymptotically stable?
\item[\bf (b)]\label{exerc:akplane}  Figure~\ref{fig:how5fg2} shows a closed-loop system in which the error, measured by a transducer that has a time constant of 1~sec, is applied to a proportional-plus-derivative~(PD) controller. The output of the controller drives a third-order plant. Use the Routh-Hurwitz criterion to find the region of asymptotic stability, of the closed-loop feedback system, in the $(a,K)$-plane.
\begin{figure}
\centering
\includegraphics[scale=0.55]{how5fg2.eps}
%\vspace{3.5in}
%\hskip.75in\special{bmp:c:/ee382/how5fg2.bmp x=4.67in y=3.5in}
%\centerbmp{4in}{3in}{c:/mybook/how5fg2.bmp}
\caption{A closed-loop feedback system of Exercise~\ref{exerc:akplane}.}
\label{fig:how5fg2}
\end{figure}
\end{itemize}
\end{exercise}
\begin{exercise}\rm\begin{itemize}\item[\bf (a)] Determine if the polynomial
\[
P(s)=s^7+2s^6+5s^5+7s^4+2s^3-5s^2-8s-4
\]
is Hurwitz. Use the Routh-Hurwitz criterion to find the number of the polynomial RHP zeros and determine the $j\omega$-axis zeros.
\item[\bf (b)] Consider a negative unity feedback system with the forward loop transfer function
\[
G(s)=\frac{K}{s(s+3)(s+10)}.
\]
Use the Routh-Hurwitz criterion to find the interval of $K$ for which the closed-loop system is asymptotically stable.
\item[\bf (c)] Consider a negative unity feedback system with the forward loop transfer function
\[
G(s)=\frac{K}{s(Ts+1)(0.5\,s+1)}.
\]
Use the Routh-Hurwitz criterion to derive the domain of asymptotic stability of the closed-loop system in the $(T,K)$-plane, and plot it using MATLAB.
\end{itemize}
\end{exercise}

\begin{exercise}\rm
For a dynamical system model
\[
\left[ \begin{array}{c}
\dot{x}_1\\
\dot{x}_2
\end{array}\right]=\left[ \begin{array}{c}
x_2\\
x_1-x_2 \sin x_1
\end{array} \right],
\]
\begin{itemize}
\item[\bf (a)] find an equilibrium point for this system;
\item[\bf (b)] linearize the system about the found equilibrium point;
\item[\bf (c)] determine if the linearized system is stable, asymptotically stable, or unstable.
\end{itemize}
\label{exercise:stalinear}
\end{exercise}
\begin{exercise}\rm
Consider the following model of a dynamical system,
\begin{equation}
\left.\begin{array}{lll}
\dot{x}_1 &=& x_2\\
\dot{x}_2 &=& -x_1^5-x_2^3
\end{array}\right\}
\label{eq:non_systforqe}
\end{equation}
\begin{itemize}
\item[\bf (a)] Find the equilibrium state of the system. Then, consider the following Lyapunov function candidate,
\[
V(x_1,x_2)=\frac{1}{2}x_2^2+\int_0^{x_1}s^5 ds.
\]
Evaluate $\frac{d}{dt}V(x_1,x_2)$ on the trajectories of the system~(\ref{eq:non_systforqe}).
\item[\bf (b)] Use the results obtained in~(a) to decide whether the equilibrium state is stable, asymptotically stable, or unstable.
\end{itemize}
\end{exercise}
\begin{exercise}\rm
For a dynamical system modeled by $\dot{\b x}=\b A\b x+\mbox{\boldmath$\zeta $}(t,\b x)$, where
\[
\b A=\left[ \begin{array}{cc}
0 & 1\\
-1/2 & -3/2
\end{array}\right],\; \mbox{and}\;\|\mbox{\boldmath$\zeta $}(t, \b x)\| \le k\|\b x\|,
\]
find $k^* > 0$ such that for any $k<k^*$ the system is globally uniformly asymptotically stable.
\end{exercise}

\begin{exercise}\rm %Funwrk 3 fall 2010 P3
Is the system,
\begin{eqnarray*}
\dot{\b x} &=& \left[\begin{array}{ccc}
-2 & 0 & 0\\
1 & 0 & 1 \\
0 & -2 & -2
\end{array}\right]\b x + \left[\begin{array}{c}
1\\
0\\
1
\end{array}\right]u\\
y &=& \left[\begin{array}{ccc}
1 & -1 & 0
\end{array}\right]\b x - 2u,
\end{eqnarray*}
BIBO stable?
\end{exercise}

\begin{exercise}\rm %Funwrk 3 fall 2010 P4
Determine the stability of the system
\[
\dot{\b x} = \left[\begin{array}{ccc}
-2 & 0 & 0\\
1 & 0 & 1 \\
0 & -2 & -2
\end{array}\right]\b x
\]
in the sense of Lyapunov by solving the Lyapunov matrix equation using the Kronecker product. Take $\b Q=\b I$. Verify your solution using the MATLAB function \verb!lyap!.
\end{exercise}

\begin{exercise}\rm %Funwrk 3 fall 2010 P5
Is the system,
\begin{eqnarray*}
\b x[k+1] &=& \left[\begin{array}{ccc}
-0.8 & 0 & 0\\
0.4 & 0 & 0.4 \\
0 & -0.8 & -0.8
\end{array}\right]\b x[k] + \left[\begin{array}{c}
1\\
0\\
1
\end{array}\right]u[k]\\
y &=& \left[\begin{array}{ccc}
1 & -1 & 0
\end{array}\right]\b x[k] - 2u[k],
\end{eqnarray*}
BIBO stable?
\end{exercise}

\begin{exercise}\rm %Funwrk 3 fall 2010 P6
Determine the stability of the system
\[
\b x[k+1] = \left[\begin{array}{ccc}
-0.8 & 0 & 0\\
0.4 & 0 & 0.4 \\
0 & -0.8 & -0.8
\end{array}\right]\b x[k]
\]
in the sense of Lyapunov by solving the Lyapunov matrix equation using the Kronecker product. Take $\b Q=\b I$. Verify your solution using the MATLAB function \verb!dlyap!.
\end{exercise}

\begin{exercise}[J. Hu]\label{fw4_ltv_F2010}\rm For the linear time-invariant (LTV) system model,
\[
\dot{\b x}(t) = \left[\begin{array}{cc}
-e^{-t} & \frac{1}{t+1}\\
0 & -e^{-t}
\end{array}\right]\b x(t),
\]
\begin{itemize}
\item Find the state transition matrix $\b{\Phi}(t,\tau)$,
    \item Is the system asymptotically stable?
    \item Let $\b x(0)=\left[\begin{array}{cc}
    1 & 0
    \end{array}\right]^{\top}$. Will the system trajectory $\b x(t)$ stay bounded as $t\to \infty$?
\end{itemize}
\end{exercise}

\begin{exercise}\rm
\begin{itemize}\item[\bf (a)] Find the value of
\[
J_0=2\int_0^{\infty}\|\b x(t)\|^2 dt
\]
subject to
\[
\dot{\b x}=\left[\begin{array}{cc}
0 & 1\\
-1 & -1
\end{array}\right]\b x, \quad \b x(0)=\left[\begin{array}{c}
0\\
-1
\end{array}\right].
\]
\item[\bf (b)] Find the value of
\[
J_0=\sum_{k=0}^{\infty}\|\b x(k)\|^2
\]
subject to
\[
\b x(k+1)=\left[\begin{array}{cc}
1/2 & 0\\
0 & 1/4
\end{array}\right]\b x(k), \quad \b x(0)=\left[\begin{array}{c}
3\\
0
\end{array}\right].
\]
\end{itemize}
\end{exercise}
\begin{exercise}\rm
Determine the stability or instability of the zero solution of the system,
\[
\left.\begin{array}{lll}
\dot{x}_1 &=& x_2\\
\dot{x}_2 &=& x_1-x_2 \sin x_1
\end{array}\right\}
\]
using the second method of Lyapunov and compare the result of your analysis with the solution to Exercise~\ref{exercise:stalinear}.
\end{exercise}
\begin{exercise}\rm {\bf (based on~\cite[page~329]{toptscheev77})} Construct a Lyapunov function for the system modeled by
\[
\left.\begin{array}{lll}
\dot{x}_1 &=& x_2\\
\dot{x}_2 &=& -x_2 - x_1^3.
\end{array}\right\}
\]
Show that the origin is a globally stable equilibrium state of the system.
\end{exercise}
\begin{exercise}\rm
Investigate the stability of the following dynamical system model:
\[
\left.\begin{array}{lll}
\dot{x}_1 &=& x_2-2x_1x_2^2\\
\dot{x}_2 &=& -x_1-x_2-2x_2^3
\end{array}\right\}
\]
Is the null solution, $x_1(t)=x_2(t)=0$, $t\ge 0$, locally stable, globally stable, locally asymptotically stable, globally asymptotically stable, or unstable? Justify your answer.
\end{exercise}
\begin{exercise}\rm
Use the second method of Lyapunov to estimate the region of asymptotic stability of the equilibrium state $\left[\begin{array}{cc}
2/3 & 2/3
\end{array}\right]^{\top}$ of the dynamic system model
\[
\left.\begin{array}{ccl}
\dot{x}_1 &=&\left( 1-x_1-0.5x_2\right)x_1\\
\dot{x}_2 &=& \left( 1-0.5x_1-x_2\right)x_2
\end{array}\right\}
\]
\end{exercise}
\begin{exercise}\rm
For the nonlinear system model, $\dot{\b x}=\b f(\b x)$, where
\begin{equation}
\b f(\b x)=\left[\begin{array}{c}
f_1(\b x)\\
f_2(\b x)
\end{array}\right]=\left[\begin{array}{l}
4x_1-2x_1^2x_2\\
-16x_2 + 4x_1^2x_2
\end{array}\right],
\label{eq:ex:nonlisysthre}
\end{equation}
find its equilibrium points and determine their stability properties using the linearization method.
\end{exercise}
\begin{exercise}\rm
Consider a dynamical system modeled by
\[
\dot{\b x}=\b A\b x,
\]
where $\b A \in \mb{\Bb R}^{n \times n}$. The eigenvalues $\lambda_i$, $i=1,2, \ldots, n$, of the matrix $\b A$ are all negative, and the matrix $\b A$ is diagonalizable, that is, there exists a nonsingular matrix $\b T \in \mb{\Bb R}^{n \times n}$ such that
\[
\b T\b A\b T^{-1}={\b D}=\mb{diag} \{\lambda_1,\lambda_2, \ldots \lambda_n \}.
\]
Let
\[
V({\b x})=\frac{1}{2}{\b x}^{\top}{\b T}^{\top}{\b T\b x}={\b x}^{\top}{\b P\b x}.
\]
\begin{itemize}
\item[\bf (a)] Find $\dot{V}$, that is, find the derivative of $V({\b x}(t))$ with respect to time along the solutions of the system $\dot{\b x}={\b A\b x}$.
\item[\bf (b)] Find the upper bound on $\dot{V}$ such that
\[
\dot{V}({\b x}) \leq c \| {\b x} \|,
\]
where the constant $c<0$ depends on an eigenvalue of ${\b A}$ and an eigenvalue of ${\b P}$.
\end{itemize}
\end{exercise}
\begin{exercise}\rm
Consider a system of linear equations of the form,
\[
\b A\b x=\b b,
\]
where $\b A\in \mb{\Bb R}^{m\times n}$, $m\ge n$, $\mb{rank}(\b A)=n$, and $\b x$ and $\b b$ are of appropriate dimensions. Consider also an associated iterative algorithm,
\[
\b x(k+1)=\b x(k)+\alpha \b A^{\top}\left(\b b-\b A\b x(k)\right),\quad \b x(0)=\b x_0,
\]
where $\alpha >0$. Treat the above algorithm as a discrete-time dynamic system.
\begin{itemize}
\item[\bf (a)] Find the equilibrium state of the algorithm;
\item[\bf (b)] Use the conditions for asymptotic stability of discrete-time systems to find the largest range of values of $\alpha$ for which the algorithm converges to its equilibrium state;
\item[\bf (c)] Show that the algorithm, for the range of $\alpha$ found in {\bf (b)}, minimizes the objective function
\[
J(\b x)=\left(\b b-\b A\b x\right)^{\top}\left(\b b-\b A\b x\right).
\]
\end{itemize}
\end{exercise}
\begin{exercise}\rm Consider a system of linear equations of the form,
\[
\b A\b x=\b b,
\]
where $\b A\in \mb{\Bb R}^{m\times n}$, $m\ge n$, $\mb{rank}(\b A)=n$, and $\b x$ and $\b b$ are of appropriate dimensions. Consider also associated objective function
\[
J(\b x)=\left(\b b-\b A\b x\right)^{\top}\left(\b b-\b A\b x\right).
\]
Write down an iterative steepest descent algorithm, in terms of $\b A$ and $\b b$, for minimizing the above objective function. Carefully derive an expression for the step size $\alpha_k$ in terms of $\nabla J\left(\b x^{(k)} \right)$ and $\b A$ only.
\end{exercise}
\begin{exercise}\rm\begin{itemize}\item[{\bf (a)}] Given two symmetric positive definite matrices ${\b P}$ and $\tilde{\b P}$. Show, by contradiction, that if for all $\b x$,
\[
\b x^{\top} {\b P}\b x \ge \b x^{\top} \tilde{\b P}\b x,
\]
then
\[
\lambda_{\max}\left({\b P} \right) \ge \lambda_{\max}\left( \tilde{\b P}\right).
\]
\item[{\bf (b)}] Let $\b P=\b P^{\top} >0$ be the unique solution to
\begin{equation}
\b A^{\top}\b P+\b P\b A=-2\b Q,
\label{eq:lyaforexinra}
\end{equation}
where $\b Q=\b Q^{\top}>0$, and let $\hat{\b P}=\hat{\b P}^{\top} >0$ be the unique solution to
\begin{equation}
\b A^{\top}\hat{\b P}+\hat{\b P}\b A=-2\hat{\b Q},
\label{eq:lyawhate}
\end{equation}
where $\hat{\b Q}=q\b Q$, $q>0$. Show that
\[
\frac{\lambda_{\min}\left(\hat{\b Q} \right)}{\lambda_{\max}\left(\hat{\b P} \right)}=\frac{\lambda_{\min}(\b Q)}{\lambda_{\max}(\b P)}.
\]
\item[{\bf (c)}] Prove the following lemma of Patel and Toda~\cite{patel80}.
\begin{lemma} Let $\b P=\b P^{\top}>0$ be the unique solution to
\[
\b A^{\top}\b P+\b P\b A=-2\b Q,
\]
where $\b Q=\b Q^{\top}>0$. Then, the ratio
\[
\frac{\lambda_{\min}(\b Q)}{\lambda_{\max}(\b P)}
\]
is maximized if $\b Q=\b I$.
\end{lemma}
\end{itemize}
\end{exercise}
\begin{exercise}\rm
Consider the Lyapunov matrix equation,
\[
\b A^{\top}\b P+\b P\b A=-2\b I_n,
\]
where $\b A\in \mb{\Bb R}^{n \times n}$ is asymptotically stable.
\begin{itemize}
\item[\bf (a)] Let $\sigma_{\min}$ be the smallest singular value of $\b A$, that is,
\[
\sigma_{\min}=\sqrt{\lambda_{\min}\left(\b A\b A^{\top} \right)},
\]
where $\lambda_{\min}\left(\b A\b A^{\top} \right)$ denotes the smallest eigenvalue of $\b A\b A^{\top}$. Show that $\sigma_{\min}>0$.
\item[\bf (b)] Let $\lambda_{\max}(\b P)$ be the largest eigenvalue of $\b P$, the solution of the above Lyapunov equation. Show that
\[
\lambda_{\max}(\b P)\ge \frac{1}{\sigma_{\min}}.
\]
\end{itemize}
\end{exercise}
\begin{exercise}[based on~Theorem~1 in~\cite{brockett70}]\rm
Consider an asymptotically stable strictly proper transfer function
\[
T(s)=\frac{n(s)}{d(s)}.
\]
Suppose that the triple $(\b c, \b A, \b b)$ is a controllable and observable realization of the transfer function $T(s)$. Thus $\b A$ must be asymptotically stable. This and the observability assumption guarantees that we can solve the Lyapunov equation
\[
\b A^{\top}\b P+\b P\b A=-\b c^{\top}\b c
%\label{eq:lyaeqwcctex}
\]
for $\b P=\b P^{\top}>0$. Let
\[
q(s)=d(s)\b b^{\top}\b P\left(s\b I-\b A \right)^{-1}\b b.
%\label{eq:qpopoatfu}
\]
Show that
\[
d(s)q(-s)+q(s)d(-s)=n(s)n(-s).
\]
\end{exercise}
\begin{exercise}\rm
Consider a dynamic system model,
\[
\dot{\b x}=\b A\b x, \quad \b x(0)=\b x_0,
\]
where $\b A=\left[\begin{array}{cc}
0 & 1\\
-1 &-1
\end{array}\right]$ and $\|\b x_0\| \le 10$. Use the Lyapunov approach to  find $T\ge 0$, given $\varepsilon_1=20$, such that
\[
\|\b x(t;0,\b x_0)\|\le \varepsilon_1\quad \mb{for all}\quad t\ge T.
\]
You can use, for example,
\[
\b Q=\left[\begin{array}{cc}
2 & 1\\
1 & 2
\end{array}\right]
\]
when solving the matrix Lyapunov equation $\b A^{\top}\b P+\b P\b A=-\b Q$.
\end{exercise}
\begin{exercise}\rm
The role of an automotive suspension is to support the vehicle body on the axles. A simple linear model of the suspension assembly associated with a single wheel, called the quarter-car model, is shown in Figure~\ref{fig:actsuspen}. We represent the vehicle body by the ``sprung'' mass, while the tire and axle are represented by the ``unsprung'' mass.
\begin{figure}
\centering
\includegraphics[scale=0.55]{actsus.eps}
%\vspace{4in}
%\hskip1in\special{bmp:c:/mybook/actsus.bmp x=5.33in y=4in}
%\centerbmp{4in}{3in}{c:/mybook/actsus.bmp}
\caption{Linear quarter-car model}
\label{fig:actsuspen}
\end{figure}
We model the tire as a spring with spring coefficient $k_t$, and assume its damping to be negligible. The suspension consists of the spring, the shock absorber, and the variable force element. The role of the variable force element is to generate a force that compensates for the uneven road which is the source of the system disturbances. We assume that the variable force element $F_a$ can instantaneously provide any desired force; thus, we have a linear active suspension. The equations modeling the dynamics of the two degree of freedom system depicted in Figure~\ref{fig:actsuspen} are derived by applying Newton's law to each mass of the system, and by taking into account the positive direction of the $z$ coordinates. We obtain
\begin{eqnarray*}
m_s\ddot{z}_s&=& -k_s(z_s-z_u)-b_s(\dot{z}_s-\dot{z}_u)+F_a,\\
m_u\ddot{z}_u &=& k_s(z_s-z_u)+b_s(\dot{z}_s-\dot{z}_u)-F_a-k_t(z_u-z_r).
\end{eqnarray*}
Define the following state variables:
\begin{eqnarray*}
x_1 &=& z_s-z_u=\mb{suspension stroke},\\
x_2 &=& \dot{z}_s =\mb{sprung mass velocity},\\
x_3 &=& z_u-z_r =\mb{tire deflection},\\
x_4 &=& \dot{z}_u=\mb{unsprung mass velocity}.
\end{eqnarray*}
Let $u=F_a$ be the system input, and let $\delta(t)=\dot{z}_r(t)$ be the disturbance input due to the road roughness.
\begin{itemize}
\item[\bf (a)] Represent the active suspension model in state-space format.
\item[\bf (b)] Substitute the vehicle data, given in Table~\ref{tab:actsuspen}, into the modeling equations.
\begin{table}
\caption{Data for a vehicle suspension taken from~\cite{sunwoo90}.}
\begin{center}
\begin{tabular}{cc}\hline
$m_s$ & 355\,kg\\
$m_u$ & 46\,kg\\
$k_s$ & 14384\,N/m\\
$b_s$ & 1860\,N/m/sec\\
$k_t$ & 300000\,N/m\\ \hline
\end{tabular}
\end{center}
\label{tab:actsuspen}
\end{table}
\item[\bf (c)] Our goal is to isolate the sprung mass from road disturbances. Specifically, we wish to design a control law such that the sprung mass velocity $\dot{z}_s(t)={x}_2$ is as close to $0$ as possible in spite of road disturbances. Thus, we assume that we can measure the sprung mass velocity, which means that the output of the plant is
\[
y(t) = \b c\b x(t)
= \left[\begin{array}{cccc}
0 & 1 & 0 & 0
\end{array}\right]\b x(t).
\]
To investigate the effect of road disturbances on the vertical velocity of the sprung mass, set $u=0$ and $\delta(t)=d\sin (6t)$. Generate a plot of $y$ versus $t$ for $d=0.5$.
\item[\bf (d)] Construct a state-feedback Lyapunov-based controller of  Theorem~\ref{thm:layap_stabcon}. Generate a plot of $y$ of the closed-loop system versus $t$ for $d=0.5$. Discuss your strategy of the above state-feedback controller design that would minimize the influence of the disturbance $\delta(t)$ on the sprung mass velocity $y$.
\end{itemize}

\end{exercise}
\begin{exercise}\rm
Consider a dynamical system model
\[
\dot{\b x} = \left[\begin{array}{cc}
-2 & 0\\
0 & -1
\end{array}\right]\b x+\left[\begin{array}{c}
1\\
1
\end{array}\right](u+h(t,\b x)),
\]
where
\[
|h(t,\b x)|\le 3.
\]
Let $\b P$ be the solution of the matrix Lyapunov equation $\b A^{\top}\b P+\b P\b A=-2\b I_2$, and let
\[
u=\left\{\begin{array}{ccc}
-3\,\mb{sign}(\b b^{\top}\b P\b x)& \mb{if}& |\b b^{\top}\b P\b x| \ge \nu\\
-3\frac{\b b^{\top}\b P\b x}{\nu} & \mb{if} & |\b b^{\top}\b P\b x| < \nu,
\end{array}\right.
\]
where $\nu=0.5$. Find the radius of the ball, in $\mb{\Bb R}^2$, that is uniformly stable for the closed-loop system.
\end{exercise} 