\documentstyle[12pt]{article}
\renewcommand{\baselinestretch}{1.5}
\setlength{\oddsidemargin}{0.25in}
%\setlength{\evensidemargin}{0in}
\setlength{\textwidth}{6in}
\setlength{\topmargin}{-0.25in}
\setlength{\textheight}{8.5in}
\newtheorem{thm}{Theorem}
\newtheorem{lemma}{Lemma}
\newcommand{\mb}{\mbox}
\newcommand{\bm}{\boldmath}
\def\b#1{\mb{\bm$#1$}}
\newfont{\Bb}{msbm10 scaled\magstep1}
\begin{document}
\title{Neurofuzzy Tracking Controllers for Uncertain Systems}
\author{Marcelo C. M. Teixeira\thanks{Department of Electrical Engineering, FEIS/UNESP, Av. Brasil 56, 15385-000-Ilha Solteira-SP, Brazil} \and  Stanislaw H. \.{Z}ak \thanks{School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907} \and  E. Bruce Lee\thanks{Department of Electrical Engineering, University of Minnesota, 4-174 EE/CSci Building, 200 Union Street S.E., Minneapolis, MN 55455}}
\maketitle

\begin{abstract}
We propose a tracking controller for uncertain dynamic system. The controller is composed of a fuzzy stabilizer and a neural network based inverse dynamics identifier. Most of fuzzy logic controllers, and in particular stabilizers, use control error and the change in the control error as their inputs. With these inputs, the resulting controller is not able to differentiate in which region the process operates. This may result in an inferior performance in some of the operating regions. To alleviate the problem we use a multiregion fuzzy logic stabilizer that takes into account the regions in which the process operates. We next integrate the multiregion fuzzy stabilizer with a neural inverse dynamics identifier into a tracking controller architecture. We test the proposed controller on a nonlinear one link robot manipulator whose mathematical model is assumed to be unknown to us. 
\end{abstract}

\section{Introduction}
The essential elements of a control problem, as described, for example, by Owens~\cite{owens81}, are:
\begin{itemize}
\item a mathematical model of the controlled system;
\item a specified objective for the system;
\item a set of admissible controllers;
\item a means of measuring the performance to evaluate the effectiveness of any given control strategy.
\end{itemize}
What can a control designer do when a mathematical model of the system is not available or is incomplete? If the parameters of the system step response can be obtained, the Ziegler and Nichols~\cite{ziegler42} rules can then be used to tune one of the family of the P, PI, or PID controllers to get an acceptable step response of the closed-loop system. To improve limitations of, for example, a conventional PI controller, one can modify its structure by adding a fuzzy logic component, as suggested by Lee~\cite{lee93}. However, this approach may not be sufficient to design a quality tracking controller. In some applications one can use the adaptive neural network based controller proposed by Etxebarria~\cite{etxebarria94}. Etxebarria's controller works well for uncertain single-input single-output discrete stable plants. An alternative approach is to use fuzzy logic if a linguistic description in terms of If-Then rules of the system to be controlled is available to us. The availability of such rules is a basic requirement for implementing a fuzzy logic controller. A fuzzy logic controller based on the If-Then rules provides a means of converting a linguistic description of the system operation into an automatic control strategy---see for example Lee~\cite{lee90}, \cite{lee90b}, or Pedrycz~\cite{pedrycz93} for an overview of fuzzy logic control design methods. But if the rule base is not complete, this may not be enough to design a high performance tracking controller, though the available rule base may be sufficient to synthesize a fuzzy stabilizer. A practical design procedure of fuzzy logic stabilizers with guidelines for parameter tuning was developed by Ying~\cite{ying94}. To enhance the performance of a fuzzy logic controller, Lu and Chen~\cite{lu94} proposed a fuzzy sliding mode controller. Lin and Lee~\cite{lin91}, on the other hand, presented a neural network based fuzzy logic controller. Lin and Lee's fuzzy logic controller has the form of a feedforward neural network with learning capabilities. It is synthesized during the learning phase from training examples. When obtaining the representative training examples is difficult, Lin and Lee~\cite{lin94} proposed to use a reinforcement neural network based fuzzy logic controller. Here again, the controller is synthesized during the training phase. Wang~\cite{wang94} proposed a fuzzy logic controller that was trained using the back-propagation algorithm. In the case when the rule base is not available to the designer, Nie and Linkens~\cite{nie94} presented a hybrid neural network based self-organizing controller. Their controller consists of a variable structure competitive network and a standard feedforward neural network that uses the back-propagation algorithm to extract If-Then rules from the process being controlled.   

In this paper, we use a neural network inverse dynamics identifier in conjunction with a fuzzy logic stabilizer to the design problem of a tracking controller, when instead of a mathematical model of the system we have If-Then rules describing the system operation available to us. The feedforward neural network identifier operates on-line. Our scheme is based on the architecture of Kuschewski, Hui, and \.{Z}ak~\cite{kuschewski93}. In Kuschewski et al.~\cite{kuschewski93} a neural network inverse dynamics identifier was employed to enhance the performance of a nonlinear controller. The nonlinear controller, though was designed using the mathematical model of the plant. The algorithm presented in this paper needs no mathematical model of the system and is applicable to multi-input multi-output systems.

The paper is organized as follows. In the next section we describe a general structure of the tracking controller we study in this paper. In Section~3, we present the design of a fuzzy logic stabilizer and illustrate its operation with a numerical example. In Section~4, we analyze neural network inverse dynamics identifier. We use this identifier as one of the components in the proposed tracking controller. We discuss the way we integrate the above components into the tracking controller in Section~5. Then, we test the proposed neurofuzzy tracking controller on a nonlinear one link robot manipulator and include the obtained computer simulation results. Conclusions are given in Section~6.

\section{Tracking Controller Architecture}
We now discuss an architecture that we use as a stepping stone in the synthesis of the proposed neurofuzzy tracking controller. Consider a block diagram of a closed-loop system shown in Figure~\ref{fig:cfcmcon}. A plant to be controlled is designated by an operator $P$. The operator $P$ represents an input-output model of the plant. We assume that the operator $P$ is invertible, that is, $P^{-1}$ exists. The feedforward controllers $A=P^{-1}B$ and $B$ coordinate the request command $r$ producing the desired responses. In particular, the dynamic element $B$ secures the causality of $A$, and is used to generate the appropriate desired output $y_r$. The role of the component $G$ is to trim any errors that are generated, and to force the plant to track the command signal $y_r$. In other words, the component $G$ is used to stabilize the feedback loop consisting of the plant $P$ and the component $G$ itself. The above architecture is derived from the coordination of the feedforward control method (CFCM) studied by Sain and Peczkowski~\cite{sain85b}. In Figure~\ref{fig:cfcmcon}, we present a block diagram of the design elements of this method.
\begin{figure}
\vspace{3.5in}
\hskip.6in\special{bmp:c:/mybook/cfc.bmp x=4.67in y=3.5in}
\caption{Design elements of the coordination of the feedforward control method.}
\label{fig:cfcmcon}
\end{figure}

To proceed with an analysis of the above configuration, we observe that if the tracking error
\[
e=y_r-y=0,\quad \mb{and}\quad Ge=0,
\]
then,
\begin{eqnarray*}
y&=&P(P^{-1}B)r\\
&=&Br\\
&=&y_r,
\end{eqnarray*}
which is the goal of the tracking controller.


To further familiarize the reader with the architecture shown in Figure~\ref{fig:cfcmcon}, we perform its basic analysis assuming that the plant is a single-input single-output linear dynamic system modeled by a transfer function $P=P(s)$. Let the desired transfer function of the feedback consisting of the plant and the component $G$ be given by $T_l=T_l(s)$. Then, the dynamic elements $A=A(s)$, $B=B(s)$, $G=G(s)$ must be designed to produce the desired responses. The two key equations that govern the design are:
\begin{eqnarray*}
A &=& P^{-1}B\\
PG &=& T_l(I-T_l)^{-1}.
\end{eqnarray*}
The second design equation can be written as
\[
G=P^{-1}T_l(I-T_l)^{-1}.
\]
Sain and his associates~\cite{sain85}, and in particular Sain and Peczkowski~\cite{sain85b}, generalized the above analysis to a class of nonlinear system.

Note that the above tracking controller does not require the availability of the plant state. However, to implement the controller we need to be able to stabilize the loop consisting from the plant $P$ and the element $G$. Furthermore, we need the plant inverse dynamics. In the case when a mathematical model of the plant is available, we can use the above equations to synthesize the tracking controller. However, we are faced with the problem of designing a tracking controller for a plant whose mathematical model is not available. Instead, we have a linguistic description of a plant's operation. In the next section, we present a method for constructing a stabilizer for uncertain plants using fuzzy logic control. After that, we present a method for on-line inverse dynamics identification using an adaptive neural network.
 
\section{Fuzzy Logic Stabilizer}
In this section, we discuss a method for designing a fuzzy logic stabilizing controller, or fuzzy logic stabilizer (FLS) for short, for a dynamic system whose mathematical model is not available to us. The configuration of the closed-loop system with a FLS is shown in Figure~\ref{fig:typ_fuz}.
\begin{figure}
\vspace{3in}
\hskip1.5in\special{bmp:c:/mybook/typ_fuz.bmp x=4in y=3in}
\caption{A diagram of the closed-loop system with a fuzzy logic stabilizer.}
\label{fig:typ_fuz}
\end{figure}
The stabilizer uses the error, $e$, and the change in the error, $\Delta e = e(t) - e(t-T_d)$, as its inputs, where $T_d$ is the delay duration which is a design parameter. A typical control architecture is depicted in Figure~\ref{fig:typflc1}. A stabilizing controller FLS is composed of three basic components. They are: a fuzzifier, a decision making logic module also referred to as an inference engine, and a defuzzifier. Crisp inputs $e$ and $\Delta e$ are converted into corresponding fuzzy sets by a fuzzifier. Then, the inference engine processes the fuzzified inputs using a knowledge base. The outputs of the inference engine are fuzzy sets that are converted into crisp values, defuzzified, by a defuzzifier.
\begin{figure}
\vspace{3in}
\hskip1.5in\special{bmp:c:/mybook/typflc.bmp x=4in y=3in}
\caption{A diagram of a typical fuzzy logic controller.}
\label{fig:typflc1}
\end{figure}
Designing a FLS can be summarized in the following six-step algorithm:
\begin{enumerate}
\item Identify ranges of the controller inputs;
\item Identify ranges of the controller outputs;
\item Create fuzzy sets for each input and output;
\item Translate the interaction of the inputs and outputs into If-Then rules, and then form the rules matrix;
\item Decide on the fuzzifier, the inference engine, and the defuzzifier, and then apply them to crisp inputs to produce crisp outputs;
\item Implement the controller, test it, and modify it if necessary.
\end{enumerate}

We now apply the above algorithm to the control of a nonlinear one-link robot manipulator. The example comes from \.{Z}ak and MacCarley~\cite{zak86}. This is a simple example of a nonlinear system whose operating point is unstable. Despite its simplicity it does capture essential properties of nonlinear dynamic systems---see \.{Z}ak and MacCarley~\cite{zak86}, and Lin~\cite[pages 298--300]{cflin94}, where this example is used to illustrate the method of state-input feedback linearization. This example was also used by Kawamoto~\cite{kawamoto96} to illustrate stability analysis of fuzzy controllers. We show on the above example the limitation of the simple fuzzy logic controller that uses only as its inputs control error and the change in the control error. We then show how to alleviate the problem. The example is used as a testbed for the proposed controllers throughout the paper. In Figure~\ref{fig:onelink}, we present a diagram of the above mentioned one-link manipulator. The motion of the link is controlled by a DC motor via a gear train.
\begin{figure}
\vspace{3.5in}
\hskip1in\special{bmp:c:/mybook/onelink.bmp x=4.67in y=3.5in}
\caption{A diagram of a one-link manipulator controlled by a DC motor via a gear train.}
\label{fig:onelink}
\end{figure}
The DC motor is armature-controlled and its diagram is shown in Figure~\ref{fig:dcmotor}.
\begin{figure}
\vspace{3in}
\hskip1.25in\special{bmp:c:/mybook/dcmotor.bmp x=4in y=3in}
\caption{A diagram of an armature-controlled DC motor.}
\label{fig:dcmotor}
\end{figure}
For the simulation purposes only, we construct a mathematical model of the system. However, while designing the controllers, we assume that the mathematical model of the system is not available to us. We assume that the motor moment of inertia is negligible compared with the robot arm. We model the arm as a point mass, $m$, attached to the end of a massless rod of length $l$. Hence, the arm inertia is $J_a=ml^2$. We assume that the gear train has no backlash and all connecting shafts are rigid. We now construct a state-space model of the one-link robot. We choose the following state variables:
\[
x_1=\theta_p, \qquad x_2=\frac{d\theta_p}{dt}=\omega_p,\qquad x_3=I_a.
\]
The plant output is $y=x_1$. Then, the model in state-space format is:
\begin{eqnarray*}
\left[\begin{array}{c}
\dot{x}_1\\
\dot{x}_2\\
\dot{x}_3
\end{array} \right] &=& \left[\begin{array}{c}
x_2\\
\frac{g}{l}\sin x_1+\frac{NK_m}{ml^2}x_3\\
-\frac{K_bN}{L_a}x_2-\frac{R_a}{L_a}x_3
\end{array}\right]+\left[\begin{array}{c}
0\\
0\\
\frac{1}{L_a}
\end{array}\right]u\\
y&=&x_1,
\end{eqnarray*}
where $K_m$ is the motor torque constant, $K_b$ is the back emf constant, $N$ is the gear ratio, and $g=9.8$ m/s$^2$ is the gravitational constant. Reasonable parameters for the robot are: $l=1$ m, $m=1$ kg, $N=10$, $K_m=0.1$ Nm/A, $K_b=0.1$ Vs/rad, $R_a=1\,\Omega$, $L_a=100$ mH. With the above parameters the robot model takes the form:
\begin{eqnarray*}
\left[\begin{array}{c}
\dot{x}_1\\
\dot{x}_2\\
\dot{x}_3
\end{array} \right] &=& \left[\begin{array}{c}
x_2\\
9.8\,\sin x_1+x_3\\
-10\,x_2-10\,x_3
\end{array}\right]+\left[\begin{array}{c}
0\\
0\\
10
\end{array}\right]u\\
y&=&x_1.
\end{eqnarray*}
As can be seen from Figure~\ref{fig:onelink}, counterclockwise rotation of the arm is defined as positive and clockwise rotation as negative. The response of the manipulator subject to the initial conditions $x_1=0.01$ rad, $x_2=0$ rad/s, $x_3=0$ A, and $u=0$ V is shown in Figure~\ref{fig:ol_resp}.
\begin{figure}
\vspace{3.5in}
\hskip.25in\special{bmp:c:/mybook/ol_resp.bmp x=4.67in y=3.5in}
\caption{Response of the manipulator subject to $x_1=0.01$ rad, $x_2=0$ rad/s, $x_3=0$~A, and $u=0$ V.}
\label{fig:ol_resp}
\end{figure}
We can show, using for example the first method of Lyapunov, that the equilibrium point $\mb{\bm$x$} = \mb{\bm$0$}$, $u=0$ is unstable. Our goal is to design a fuzzy logic stabilizer (FLS) that stabilizes this equilibrium point. We first attempt to design an FLS that can return the arm to the equilibrium point when its initial displacement is in the interval $[-0.1\,\mb{rad},\;0.1\,\mb{rad}]$. Following the design algorithm, we first identify the stabilizer inputs and the output ranges. The error is assumed to be in the interval $[-0.1\,\mb{rad},\,0.1\,\mb{rad}]$, while the change-in-error to be in the interval $[-0.005\,\mb{rad},\,0.005\,\mb{rad}]$. The controller output range, which is the plant input, is assumed to be in the interval $[-1.5\,\mb{V},\,1.5\,\mb{V}]$. We use the bell-shaped membership functions of the form
\[
\mu(x)=\exp\left(-\left(\frac{x-m}{\sigma}\right)^2\right),
\]
where $m$ and $\sigma$ are adjustable parameters. For simplicity of the exposition, we use only five membership functions for the inputs and the output. The $m$ and $\sigma$-parameters of the membership functions are shown in Table~\ref{tab:small}. In selecting the membership functions, we followed a rule of thumb according to which there should be an odd number, between five and nine, of membership functions associated with each linguistic variable. Furthermore, it is recommended that the membership functions overlap with their neighbors. The recommended overlap should be between 10 and 50 percent. It is further recommended that around the desired response, the membership functions should be ``thin'', and should get ``fat'' as the distance from the desired response increases.   
\begin{table}
\begin{center}
{\small\begin{tabular}{|c c|c c|c c|}\hline
\multicolumn{2}{|c|}{\bf error} &
\multicolumn{2}{c|}{\bf change-in-error} &
\multicolumn{2}{c|}{\bf output}\\ \hline
$m$ & $\sigma$ & $m$ & $\sigma$ & $m$ & $\sigma$ \\ \hline \hline
$m_{LN}=-0.1$ & $\sigma _{LN}=0.04$ & $m_{LN}=-0.005$ & $\sigma _{LN}=0.002$ & $m_{LN}=-1.5$ & $\sigma_{LN}=0.6$ \\  
$m_{N}=-0.04$ & $\sigma _{N}=0.03$ & $m_{N}=-0.002$ & $\sigma _{N}=0.0015$ & $m_{N}=-0.6$ & $\sigma_{N}=0.5$ \\ 
$m_{ZE}=0$ & $\sigma _{ZE}=0.02$ & $m_{ZE}=0$ & $\sigma _{ZE}=0.0015$ & $m_{ZE}=0$ & $\sigma_{ZE}=0.3$ \\ 
$m_{P}=0.04$ & $\sigma _{P}=0.03$ & $m_{P}=0.002$ & $\sigma _{P}=0.0015$ & $m_{P}=0.6$ & $\sigma_{P}=0.5$ \\ 
$m_{LP}=0.1$ & $\sigma _{LP}=0.04$ & $m_{LP}=0.005$ & $\sigma _{LP}=0.002$ & $m_{LP}=1.5$ & $\sigma_{LP}=0.6$\\\hline 
\end{tabular}}
\end{center}
\caption{The $m$ and $\sigma$-parameters of the membership functions for the inputs and the output.}
\label{tab:small}
\end{table}
After deciding upon the membership functions, we generate the rules. They are shown in Table~\ref{tab:rule_s}, where LN denotes large negative, N means negative, ZE---zero, P---positive, and LP refers to large positive. The rules were chosen based on our knowledge of the plant.
\begin{table}
\begin{center}
\begin{tabular}{|c||c|c|c|c|c|} \hline
{\bf change-in-error} & LN & N & ZE & P & LP \\ \cline{1-1}
{\bf error}& {}&{}&{}&{}&{}\\ \hline \hline
LN & LN & LN & LN & LN & LN \\ \hline
N & LN & LN & N & N & N \\ \hline
ZE & N & N & ZE & P & P \\ \hline
P & P & P & P & LP & LP \\ \hline
LP & LP & LP & LP & LP & LP \\ \hline
\end{tabular}
\end{center}
\caption{The rules matrix.}
\label{tab:rule_s}
\end{table}


The next step involves choosing an inference engine and a defuzzifier. Let $\mu_k^o$, $k=1,2,\ldots,5$, be the output membership functions, and let $\mu_{i}^{e}$, $\mu_{j}^{ec}$ be the membership functions corresponding to the linguistic variables {\em ERROR} and {\em CHANGE-IN-ERROR}, respectively. Following the rules shown in Table~\ref{tab:rule_s}, we form
\[
u_k=\mu_{i}^{e}\mu_j^{ce},\quad k=1,2,\ldots,25,\quad i,j=1,2,\ldots,5.
\] 
Then, using the product inference rule and the modified center average defuzzifier to obtain
\[
u_f=\frac{\sum_{k=1}^{25}u_k m_k^o/ \sigma_k^o}{\sum_{k=1}^{25}u_k /\sigma_k^o},
\]
The modified center average defuzzifier takes into account the shape of the membership functions, and in particular their ``spread''. As it is pointed out in Wang~\cite{wang94}, the sharper the shape of $\mu$, the stronger is our belief that the controller output $u_f$ should be closer to its $m$. This observation is reflected in the formula for this defuzzifier where we divide the weights $u_k$ by $\sigma_k^o$.

We next create a nonlinear mapping from the input space into the output space and form a table lookup to implement the obtained nonlinear mapping representing our fuzzy logic stabilizer. The closed-loop system with the fuzzy stabilizer implemented as a table lookup has the form as in Figure~\ref{fig:typ_fuz}, where now $y_r=0$. The size of the table lookup is $21 \times 21$ with $-0.1 \le e \le 0.1$ and $-0.005 \le \Delta e \le 0.005$ being uniformly spaced out. We modeled and simulated the closed-loop system using SIMULINK. We used the 2-D Lookup Table available in SIMULINK's Block Libraries to implement the stabilizer table lookup. The change-in-error, $\Delta e = e(t)-e(t-T_d)$, is implemented using the Variable Transport Delay block, where the delay $T_d$ used in our implementation is $T_d=0.01$~sec. Simulation results for two different initial conditions are shown in Figure~\ref{fig:sim_sc}.
\begin{figure}
\vspace{4.5in}
\special{bmp:c:/mybook/sim_sc.bmp x=6in y=4.5in}
\caption{Stabilized system response subject to different initial conditions on $x_1$.}
\label{fig:sim_sc}
\end{figure}
As we can see from Figure~\ref{fig:sim_sc}, for small deviations of the arm from the operating point at $x_1=0$, the fuzzy stabilizer forces it to return back to the vertical position quite fast. However, if the initial displacements are greater than $|0.1\,\mb{rad}|$, the stabilizer has difficulty to force the arm to return to the vertical position as can be seen in Figure~\ref{fig:sim_sbc}.
\begin{figure}
\vspace{4.5in}
\special{bmp:c:/mybook/sim_sbc.bmp x=6in y=4.5in}
\caption{Stabilized system response subject to $x_1=0.13$ rad.}
\label{fig:sim_sbc}
\end{figure} 
For larger deviations from the operating point, the actuator needs a larger input signal. We could try to design a fuzzy logic stabilizer, using five membership functions for each input and the output, for a large range of displacements from the operating point. The closed-loop system with such a controller would be able to operate over a wider range of initial arm displacements while sacrificing performance in the small displacement region. We could use more membership functions for $e$ and $\Delta e$ over a larger operating region. This however would result in explosion of the number of fuzzy rules required. To alleviate the problem, we designed a fuzzy logic stabilizer that uses different control strategies in different operating regions. Based on our knowledge of the plant, we divided the error range into three regions: small, medium, and large. The error, $e$, is considered small if $|e|\le\,0.08$ rad, it is medium if $0.08\,\mb{rad}<|e|\le\,0.8\,\mb{rad}$, and it is large if $0.8\,\mb{rad}<|e|\le\,3.2\,\mb{rad}$. We then designed a fuzzy logic stabilizer for each of the regions using regional information. The design process of the stabilizers was similar to the design of the stabilizer for small error, except of course we used different ranges for the error, the change-in-error, and the controller input. We used the rules matrix shown in Table~\ref{tab:rule_s} for each of the stabilizer. We implemented the stabilizers using table lookups. Finally, all the stabilizers were combined to give a multiregion stabilizer that has a satisfactory performance in all three regions. In combining the individual stabilizers we used the Switch block. The response of the closed-loop system, driven by the multiregion fuzzy logic stabilizer, subject to two different initial displacements from the operating point is shown in Figure~\ref{fig:clr_mrc}.
\begin{figure}
\vspace{4.5in}
\special{bmp:c:/mybook/clr_mrc.bmp x=6in y=4.5in}
\caption{Closed-loop system response, with multiregion fuzzy logic stabilizer, subject to two different initial displacements $x_1=3.0$ rad and $x_1=-1.5$ rad.}
\label{fig:clr_mrc}
\end{figure}  
The idea of using a multiregion fuzzy logic control was already used by Li and Lau in \cite{li89}. Li and Lau applied a two-region fuzzy logic controller to control a servomotor. When the magnitude of the control error was large, the fuzzy logic coarse controller was in action. For the error magnitude small, the fuzzy logic fine controller was taking over. An alternative approach to designing multiregion fuzzy logic controllers is proposed by Qin and Borders in \cite{qin94}. Qin and Borders divide the operating region of the plant gain into fuzzy regions such as low gain, medium gain, and large gain. Then, they introduce an auxiliary variable that detects the plant operating region and activates control strategy corresponding to the operating region.

\section{Inverse Dynamics Identification}
To synthesize the controller that is shown in Figure~\ref{fig:cfcmcon}, we need to know the inverse dynamics of the plant. We now present a feedforward neural network based inverse dynamics identifier. In the next section, we describe the way we integrate the fuzzy stabilizer and neural identifier into the tracking controller. A basic scheme for identifying the inverse dynamics of a dynamic system that we use is depicted in Figure~\ref{fig:invdyn}.
\begin{figure}
\vspace{3in}
\hskip1.5in\special{bmp:c:/mybook/invdyn.bmp x=4in y=3in}
\caption{Block diagram of an inverse dynamics identifier.}
\label{fig:invdyn}
\end{figure}
The neural network we use to implement the inverse dynamics identifier is shown in Figure~\ref{fig:twofnn}. It is a two layer feedforward neural network.
\begin{figure}
\vspace{3in}
\hskip1.5in\special{bmp:c:/mybook/twofnn.bmp x=4in y=3in}
\caption{A two layer feedforward neural network used to implement the inverse dynamics identifier.}
\label{fig:twofnn}
\end{figure}
We now analyze the feedforward neural network shown in Figure~\ref{fig:twofnn}. The input signal to the network at time instant $kT$, consists of the samples of the plant output $y(kT)$, where $T$ is the sampling period. For simplicity of notation we write $y_k=y(kT)$. In our simulations, we used $T=0.01$ sec. We obtain the samples by passing $y$ through a series of Unit Delay blocks connected in a transversal filter. The number $n_i$ of the Unit Delay blockss in the transversal filter, is a design parameter that determines the number of neurons in the input layer. The matrix $\mb{\bm$W$}_{1,k} \in \mb{\Bb R}^{n_i \times n_i}$ is the weight matrix of the input layer at time $k$. The nonlinear activation operator $\mb{\bm$\Gamma$}:\mb{\Bb R}^{n_i} \to \mb{\Bb R}^{n_i}$ is an odd operator, that is, $\mb{\bm$\Gamma(-z)=-\Gamma(z)$}$. In our implementation we use a diagonal $\mb{\bm$\Gamma$}$ of the form
\[
\mb{\bm$\Gamma(z)$}=\left[\begin{array}{c}
\mb{sign}(z_1)\\
\mb{sign}(z_2)\\
\vdots\\
\mb{sign}(z_{n_i})
\end{array}\right].
\]
The matrix $\mb{\bm$W$}_{2,k} \in \mb{\Bb R}^{m \times n_i}$ is the weight matrix of the output layer. The desired output signal from the net is designated by $u$. The weight matrix update algorithm, we use, was derived by \.{Z}ak and Sira-Ramirez~\cite{zak92}. The algorithm has the form:
\[
\mb{\bm$W$}_{i,k+1}=\mb{\bm$W$}_{i,k}+\mb{\bm$\Delta$}\mb{\bm$W$}_{i,k} \qquad i=1,2,
\]
where
\[
\mb{\bm$\Delta$}\mb{\bm$W$}_{1,k}=\left\{ \begin{array}{ccc}
\frac{-2 \mb{\bm$z$}_k\mb{\bm$\Theta$}_1^T \mb{\bm$(y$}_k)}{\mb{\bm$\Theta$}_1^T \mb{\bm$(y$}_k)\mb{\bm$y$}_k} &\mb{if}& \mb{\bm$\Theta$}_1^T \mb{\bm$(y$}_k)\mb{\bm$y$}_k \ne 0\\
0 &\mb{if}& \mb{\bm$\Theta$}_1^T \mb{\bm$(y$}_k)\mb{\bm$y$}_k=0,
\end{array}\right.
\]
where $\mb{\bm$\Theta$}_1 : \mb{\Bb R}^{n_i} \to \mb{\Bb R}^{n_i}$ is an operator, that is a design parameter. We use
\[
\mb{\bm$\Theta$}_1 (\b y_k)=\left[\begin{array}{c}
\mb{sign}(y(kT))\\
\mb{sign}(y((k-1)T))\\
\vdots\\
\mb{sign}(y((k-n_i)T))
\end{array}\right].
\]
The correction term $\mb{\bm$\Delta$}\mb{\bm$W$}_{2,k}$ of the output layer weight matrix update equation is
\[
\mb{\bm$\Delta$}\mb{\bm$W$}_{2,k}=\left\{ \begin{array}{ccc}
-2\mb{\bm$W$}_{2,k}-\frac{\mb{\bm$Ae$}_k \mb{\bm$\Theta$}_2^T( \mb{\bm$\Gamma(z$}_k))}{\mb{\bm$\Theta$}_2^T( \mb{\bm$\Gamma(z$}_k))\mb{\bm$\Gamma (z$}_k)} &\mb{if}& \mb{\bm$\Theta$}_2^T( \mb{\bm$\Gamma(z$}_k))\mb{\bm$\Gamma (z$}_k) \ne 0\\
0 &\mb{if}& \mb{\bm$\Theta$}_2^T \mb{\bm$(\Gamma(z$}_k))\mb{\bm$\Gamma (z$}_k)=0,
\end{array}\right.
\]
where $\mb{\bm$\Theta$}_2 : \mb{\Bb R}^{n_i} \to \mb{\Bb R}^{n_i}$ is an operator, that is a design parameter. We use the same operators for $\mb{\bm$\Theta$}_1$ and $\mb{\bm$\Theta$}_2$. The matrix $\mb{\bm$A$} \in \mb{\Bb R}^{m \times m}$ is also a design parameter. It has to be chosen in such a way so that the eigenvalues of the matrix $\mb{\bm$I$}_m - \mb{\bm$A$}$ are contained in the open unit disk of the complex plane. For applications of the above algorithm to the forward as well as inverse dynamics identification problems, we refer the reader to Sira-Ram\'{i}rez and \.{Z}ak~\cite{sira91}, and to Sira-Ram\'{i}rez and Colina-Morles~\cite{sira95} for further analysis of a related algorithm.

We now construct the inverse dynamics neural identifier for the one link robot manipulator described in a previous section. The plant is a single-input single-output nonlinear dynamic system. We use a simple neural network based identifier consisting of two neurons in the input layer and just one neuron in the output layer. This means that the input to the network has the form $\b y_k=\left[\begin{array}{cc}
y(kT) & y((k-1)T)
\end{array}\right]^T=\left[\begin{array}{cc}
y_k & y_{k-1}
\end{array}\right]^T$, where $y(kT)$ is the plant output value at $t=kT$. Thus, the input weight matrix 
\[
\mb{\bm$W$}_{1,k}=\left[\begin{array}{cc}
w_{1,k}^{(1,1)} & w_{1,k}^{(1,2)}\\
w_{1,k}^{(2,1)} & w_{1,k}^{(2,2)}
\end{array}\right]\in \mb{\Bb R}^{2 \times 2}
\]
and the output weight matrix
\[
\mb{\bm$W$}_{2,k} =\left[\begin{array}{cc}
w_{2,k}^{(1)} & w_{2,k}^{(2)}
\end{array}\right] \in \mb{\Bb R}^{1 \times 2}.
\]
While deriving the weight update equations we used the following useful relation:
\[
\mb{sign}(y)=\frac{y}{|y|}=\frac{|y|}{y}.
\]
The above leads to the following simplification
\[
\left[\begin{array}{cc}
\mb{sign}(y_1) & \mb{sign}(y_2)
\end{array}\right] \left[\begin{array}{c}
y_1\\
y_2
\end{array}\right]=|y_1|+|y_2|.
\]
To simplify the implementation of the adaptive algorithm, we modify the original expression for the correction term $\Delta \b W_{1,k}$. We add $\epsilon = 10^{-6}$ to the sum $|y_1|+|y_2|$ in the denominators of the update equations to bound the denominators away from zero. Let $\mb{\bm$z$}_k=\left[\begin{array}{cc}
z_k^1 & z_k^2
\end{array}\right]^T$. Taking into account the above relations, we obtain the input weight matrix update equations
\begin{eqnarray*}
w_{1,k+1}^{(1,1)} &=& w_{1,k}^{(1,1)}-\frac{2z_k^1 \, \mb{sign}(y_1)}{|y_1|+|y_2|+\epsilon}\\
w_{1,k+1}^{(1,2)} &=& w_{1,k}^{(1,2)}-\frac{2z_k^1 \, \mb{sign}(y_2)}{|y_1|+|y_2|+\epsilon}\\
w_{1,k+1}^{(2,1)} &=& w_{1,k}^{(2,1)}-\frac{2z_k^2 \, \mb{sign}(y_1)}{|y_1|+|y_2|+\epsilon}\\
w_{1,k+1}^{(2,2)} &=& w_{1,k}^{(2,2)}-\frac{2z_k^2 \, \mb{sign}(y_2)}{|y_1|+|y_2|+\epsilon}.
\end{eqnarray*}
The output layer weight matrix update equations are
\begin{eqnarray*}
w_{2,k+1}^{(1)} &=& -2w_{2,k}^{(1)}-\frac{a\varepsilon_k\, \mb{sign}(z_k^1)}{2}\\
w_{2,k+1}^{(2)} &=& -2w_{2,k}^{(2)}-\frac{a\varepsilon_k\, \mb{sign}(z_k^2)}{2},
\end{eqnarray*}
where
\[
\varepsilon_k=u_k-\tilde{u}_k=u_k-w_{2,k}^{(1)}\, \mb{sign}(z_k^1) - w_{2,k}^{(2)}\, \mb{sign}(z_k^2),
\]
and the parameter $a$ should satisfy $|1-a|<1$ as it follows from our discussion of the general update algorithm.

\section{Controller Integration}
In this section, we integrate the individual components that we analyzed in the previous sections into a neurofuzzy tracking controller whose block diagram is shown in Figure~\ref{fig:cfcmnfc}.
\begin{figure}
\vspace{4in}
\hskip.25in\special{bmp:c:/mybook/cfcn.bmp x=5.33in y=4in}
\caption{Block diagram of a neurofuzzy tracking controller for uncertain dynamic systems.}
\label{fig:cfcmnfc}
\end{figure}
The controller consists of a multiregion fuzzy logic stabilizer, whose feasibility was demonstrated in the previous section, and the on-line neural inverse dynamics identifier. We point out that the neural identifier operates in real time, no off-line training is needed. The master neural network receives the actual plant output $y$ and it is supposed to generate the actual plant input $u$, thus encoding in its weights the inverse dynamics of the plant. The weights of the master neural network are copied into the slave neural net allowing us to obtain the feedforward element $A=P^{-1}B$. An important feature of the scheme is that the master and slave neural networks receive similar signals as their inputs. Indeed, the plant output $y$ is required to track the $y_r$ command signal. Hence, we can expect the slave net to give us a good approximation to the $P^{-1}$ corresponding to the command signal $y_r$.

We test the proposed tracking controller on the single link robot manipulator described in a previous section. We choose $B$ of the form
\[
B=\frac{\omega _n^2}{s^2 + 2 \zeta \omega _n s + \omega _n^2}.
\]
The above choice of $B$ means that we want the closed-loop system to behave like a linear second order system. Suppose we desire a settling time $t_s=4$ sec and a damping ratio $\zeta = .707$. Applying the $2\,\%$ settling time criterion,
\[
t_s=\frac{4}{\xi \,  \omega _n}
\]
yields $\omega _n = 1.414$. Taking the above into account we obtain
\[
B=\frac{2}{s^2 + 2s +2}.
\]
We note that when applying the CFCM to a linear single-input single-output plant modeled by a transfer function $P(s)$, we should choose $B=B(s)$ so that $A(s)=P^{-1}(s)B(s)$ is causal. In our example the transfer function of the linearized plant about the equilibrium point $\mb{\bm$x=0$}$, $u=0$ is
\[
P(s) = \frac{10}{s^3 + 10s^2 +0.2s-98}.
\]
Thus, to ensure the causality of $A(s)$, one would have to select $B(s)$ with at least three more finite poles than zeros. Of course, had we known the plant model, we would have do so. Here, however we assume that we do not have any information about the plant model and so we try $B=\frac{2}{s^2 + 2s +2}$. This $B$ would be unacceptable if we were to implement the CFCM linear tracking controller using the transfer function $P(s)$ of the linearized plant. In the simulations, using VisSim again, we used the 2nd order Runge-Kutta integration algorithm with the step size $h=0.001$ sec. We chose $a=0.008$. Recall that the parameter $a$ controls the neural network inverse dynamics identifier stability. We set zero initial conditions for the plant model. The initial weights of the identifier were chosen randomly. The plots of the output of the closed-loop system $y$ and the reference signal $y_r$ that $y$ is supposed to track are shown in Figure~\ref{fig:final_4}. The reference signal $y_r$ is the output of $B$ whose input is
\[
r=\left\{\begin{array}{ccc}
0.4 \, t & \mb{for}& 0 \le t < 5\\
2 & \mb{for} & 5 \le t < 20\\
0 & \mb{for} & 20 \le t < 25\\
-1 & \mb{for} & 25 \le t \le 40.
\end{array} \right.
\] 
\begin{figure}
\vspace{4.5in}
\hskip.2in\special{bmp:c:/mybook/final_4.bmp x=6in y=4.5in}
\caption{Tracking performance of the closed-loop system driven by the neurofuzzy controller.}
\label{fig:final_4}
\end{figure}
We can see from Figure~\ref{fig:final_4} that the closed-loop system driven by the neurofuzzy controller performs very well. We point out that one can improve the performance of the proposed controller by designing the fuzzy stabilizer using more than just five membership functions for the input and output signals. Another way to further improve the tracking performance is to reduce time delay $T_d$ in $\Delta e$. A more complex neural identifier may also help to increase tracking performance. In our simulations we limited ourselves to very simple structures so that the reader can easily verify our results.

\section{Conclusions}
In many control applications, a mathematical model of a process is incomplete or available in linguistic terms in the form of If-Then rules. For such control applications we proposed a neurofuzzy tracking controller and illustrated its viability with computer simulations. We point out that the proposed controller is based on a well developed control scheme known as the coordination of the feedforward control method (CFCM) of Sain and Peczkowski~\cite{sain85b}. The robustness properties of the controllers designed with the CFCM makes it possible to fine tune our neurofuzzy controller to give a satisfactory performance. We believe that the stability analysis of the CFCM controllers is a good starting point for a qualitative stability analysis of the closed-loop systems driven by the proposed neurofuzzy controller.

\subsection*{Acknowledgement}
M. C. M. Teixeira gratefully acknowledges financial support of Funda\c{c}\~{a}o de Amparo \`{a} Pesquisa do Estado de S\~{a}o Paulo (FAPESP) of Brazil.

\bibliography{neur_fuz}
\bibliographystyle{ieeetr}
\end{document}