Systems and Control: A Computational Approach to Modeling, Analysis, and Design

Second Edition

Stanislaw H. Żak

School of Electrical and Computer Engineering

Purdue University West Lafayette, IN 47907

August 26, 2011

To J.M.J. ii

Contents

Preface xiii

1 Vocabulary and Description of Dynamical Systems 1

1.1 What Is a System? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Memoryless, Causal, and Lumped Systems . . . . . . . . . . . . . . . 4 1.1.2 Formulation of the Control Problem . . . . . . . . . . . . . . . . . . . 6 1.1.3 Open-Loop Versus Closed-Loop . . . . . . . . . . . . . . . . . . . . . 8

1.2 Linearity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.3 Input-Output Description of Linear Systems . . . . . . . . . . . . . . . . . . 14

1.3.1 The Impulse Function and Its Properties . . . . . . . . . . . . . . . . 14 1.3.2 Zero-State Response Description of Continuous-Time, Relaxed, Causal

Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.3.3 Input-Output Description of Continuous Time-Invariant Systems . . 20 1.3.4 State-Space Description of Continuous Time-Invariant Systems …. 23 1.3.5 Input-Output Description of Discrete Time-Invariant Systems …. 25 1.3.6 State-Space Description of Discrete Time-Invariant Systems . . . . . 27

1.4 From Transfer Function to State-Space Description . . . . . . . . . . . . . . 30 1.5 System Zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 1.6 Summary and Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 1.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2 Dynamical System Modeling 55 2.1 Mathematical Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 2.2 Review of Work and Energy Concepts . . . . . . . . . . . . . . . . . . . . . 57 2.3 The Lagrange Equations of Motion in Generalized Coordinates . . . . . . . . 60 2.4 Model Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

2.4.1 Centrifugal Governor . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 2.4.2 Ground Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 2.4.3 Permanent Magnet Stepper Motor . . . . . . . . . . . . . . . . . . . 81 2.4.4 Stick Balancer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 2.4.5 One-Link Robot Manipulator . . . . . . . . . . . . . . . . . . . . . . 95 2.4.6 Double Pendulum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 2.4.7 Population Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 2.4.8 The Hypothalamic-Pituitary-Adrenal Axis Model . . . . . . . . . . . 105

2.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 2.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

3 Analysis of Modeling Equations 121 3.1 State Equation Solution of Uncontrolled System . . . . . . . . . . . . . . . . 122 3.2 Computing State Transition Matrix of LTI Systems . . . . . . . . . . . . . . 122

3.2.1 The Inverse Laplace Transform Method . . . . . . . . . . . . . . . . . 123 3.2.2 Diagonalization Method . . . . . . . . . . . . . . . . . . . . . . . . . 124

3.3 Computing Functions of a Square Matrix . . . . . . . . . . . . . . . . . . . . 127 3.3.1 The Jordan Canonical Form . . . . . . . . . . . . . . . . . . . . . . . 127 3.3.2 Computing Polynomial Functions of a Square Matrix . . . . . . . . . 137

3.4 State Equation Solution of Controlled System . . . . . . . . . . . . . . . . . 146 3.5 From Continuous to Discrete Models . . . . . . . . . . . . . . . . . . . . . . 147 3.6 State Plane Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

3.6.1 Examples of Phase Portraits . . . . . . . . . . . . . . . . . . . . . . . 151 3.6.2 The Method of Isoclines . . . . . . . . . . . . . . . . . . . . . . . . . 155

3.7 Numerical Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 3.7.1 The Method of Taylor Series . . . . . . . . . . . . . . . . . . . . . . . 163 3.7.2 Forward Euler Method . . . . . . . . . . . . . . . . . . . . . . . . . . 166 3.7.3 Backward Euler Method . . . . . . . . . . . . . . . . . . . . . . . . . 170 3.7.4 Predictor-Corrector Method . . . . . . . . . . . . . . . . . . . . . . . 171 3.7.5 Runge’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 3.7.6 Runge-Kutta Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

3.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 3.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

4 Linearization 183 4.1 Taylor Linearization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 4.2 Linearizing Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . 187 4.2.1 Time-Invariant Unforced System . . . . . . . . . . . . . . . . . . . . 187 4.2.2 Time-Varying Forced System . . . . . . . . . . . . . . . . . . . . . . 190

4.3 Describing Function Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 4.3.1 Scalar Product of Functions . . . . . . . . . . . . . . . . . . . . . . . 194 4.3.2 Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 4.3.3 Describing Function in the Analysis of Nonlinear Systems . . . . . . . 199

4.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 4.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

5 Linear Time-Invariant (LTI) Systems 215 5.1 Reachability and Controllability . . . . . . . . . . . . . . . . . . . . . . . . . 216 5.2 Observability and Constructability . . . . . . . . . . . . . . . . . . . . . . . 229 5.3 Minimal Realizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 5.4 Companion Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

5.4.1 Controller Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 5.4.2 Observer Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

5.5 Linear State-Feedback Control . . . . . . . . . . . . . . . . . . . . . . . . . . 250 5.6 Observers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

5.6.1 Full-Order Observer . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 5.6.2 Reduced-Order Observer . . . . . . . . . . . . . . . . . . . . . . . . . 264

5.7 Combined Controller-Observer Compensator . . . . . . . . . . . . . . . . . . 268 5.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 5.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

6 A Glimpse at Linear Time-Varying (LTV) Systems 293 6.1 Motivating Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 6.2 Solving the Uncontrolled LTV System . . . . . . . . . . . . . . . . . . . . . . 298 6.3 Solving the LTV Controlled System . . . . . . . . . . . . . . . . . . . . . . . 305 6.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 6.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

7 Stability 311 7.1 An Informal Introduction to Stability . . . . . . . . . . . . . . . . . . . . . . 311 7.2 Basic Definitions of Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 7.3 Stability of Continuous-Time LTI Systems . . . . . . . . . . . . . . . . . . . 320 7.4 Evaluating Quadratic Indices . . . . . . . . . . . . . . . . . . . . . . . . . . 327 7.5 Discrete-Time Lyapunov Equation . . . . . . . . . . . . . . . . . . . . . . . . 333 7.6 Constructing Robust Linear Controllers . . . . . . . . . . . . . . . . . . . . . 336 7.7 Hurwitz and Routh Stability Criteria . . . . . . . . . . . . . . . . . . . . . . 340 7.8 Stability of Nonlinear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 347 7.9 Lyapunov’s Indirect Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 7.10 Discontinuous Robust Controllers . . . . . . . . . . . . . . . . . . . . . . . . 370 7.11 Uniform Ultimate Boundedness . . . . . . . . . . . . . . . . . . . . . . . . . 377 7.12 Lyapunov Like Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 7.13 Input-Output Stability of LTI Systems . . . . . . . . . . . . . . . . . . . . . 395 7.14 Input-Output Stability of LTV Systems . . . . . . . . . . . . . . . . . . . . . 400 7.15 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401 7.16 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402

8 A Computational Tool for Control Design 413 8.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 8.2 Linear Matrix Inequality and Its Properties . . . . . . . . . . . . . . . . . . 414 8.3 LMI Solvers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419

8.3.1 Solving the Feasibility Problem . . . . . . . . . . . . . . . . . . . . . 420 8.3.2 Minimization of a Linear Objective Under LMI Constraints . . . . . . 423 8.3.3 Generalized Eigenvalue Minimization Problem . . . . . . . . . . . . . 425

8.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 8.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426

9 Optimal Control 429 9.1 Performance Indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431 9.2 A Glimpse at the Calculus of Variations . . . . . . . . . . . . . . . . . . . . 433

9.2.1 Variation and Its Properties . . . . . . . . . . . . . . . . . . . . . . . 436 9.2.2 Euler-Lagrange Equation . . . . . . . . . . . . . . . . . . . . . . . . . 441

9.3 Linear Quadratic Regulator . . . . . . . . . . . . . . . . . . . . . . . . . . . 452 9.3.1 Algebraic Riccati Equation (ARE) . . . . . . . . . . . . . . . . . . . 452 9.3.2 Solving the ARE Using the Eigenvector Method . . . . . . . . . . . . 457 9.3.3 Optimal Systems With Prescribed Poles . . . . . . . . . . . . . . . . 460 9.3.4 Optimal Saturating Controllers . . . . . . . . . . . . . . . . . . . . . 476 9.3.5 Linear Quadratic Regulator for Discrete Systems on an Infinite Time

Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481

9.4 Dynamic Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485 9.4.1 Discrete-Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 485 9.4.2 Discrete Linear Quadratic Regulator Problem . . . . . . . . . . . . . 490 9.4.3 Continuous Minimum Time Regulator Problem . . . . . . . . . . . . 493 9.4.4 The Hamilton-Jacobi-Bellman (HJB) Equation . . . . . . . . . . . . 496

9.5 Pontryagin’s Minimum Principle . . . . . . . . . . . . . . . . . . . . . . . . . 506 9.5.1 Optimal Control With Constraints on Inputs . . . . . . . . . . . . . . 506 9.5.2 A Two-Point Boundary-Value Problem . . . . . . . . . . . . . . . . . 520

9.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523 9.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524

10 Sliding Modes 535 10.1 Simple Variable Structure Systems . . . . . . . . . . . . . . . . . . . . . . . 535 10.2 Sliding Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538 10.3 A Simple Sliding Mode Controller . . . . . . . . . . . . . . . . . . . . . . . . 545 10.4 Sliding in Multi-Input Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 549 10.5 Sliding Surface Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551 10.6 Sliding Mode and System Zeros . . . . . . . . . . . . . . . . . . . . . . . . . 556 10.7 Regulator Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558

10.7.1 Outline of the Approach . . . . . . . . . . . . . . . . . . . . . . . . . 559 10.7.2 System Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . 560 10.7.3 Regulator Construction . . . . . . . . . . . . . . . . . . . . . . . . . . 561 10.7.4 Analysis of the Closed-Loop System Behavior . . . . . . . . . . . . . 563 10.7.5 Example: Two-link robotic manipulator . . . . . . . . . . . . . . . . 568

10.8 State Estimation of Uncertain Systems . . . . . . . . . . . . . . . . . . . . . 569 10.9 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573 10.10Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574

11 Vector Field Methods 579 11.1 Background Results From Differential Geometry . . . . . . . . . . . . . . . . 579 11.2 Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582 11.3 A Nonlinear Plant Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585 11.4 Controller Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586 11.5 Linearizing State-Feedback Control . . . . . . . . . . . . . . . . . . . . . . . 598 11.6 Observer Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 599 11.7 Asymptotic Observer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604 11.8 Combined Controller-Observer Compensator . . . . . . . . . . . . . . . . . . 605 11.9 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607 11.10Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607

12 Unknown Input Observer (UIO) Design 613 12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614 12.2 Uncertain System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614 12.3 A Projection Operator Approach . . . . . . . . . . . . . . . . . . . . . . . . 615

12.3.1 Background Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615 12.3.2 Constructing Full-Order UIO . . . . . . . . . . . . . . . . . . . . . . 618 12.3.3 Reduced-Order UIO . . . . . . . . . . . . . . . . . . . . . . . . . . . 623 12.3.4 Relations With Other UIO Architectures . . . . . . . . . . . . . . . . 629

12.4 Sliding Mode UIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631 12.4.1 Sliding Mode UIO Architecture . . . . . . . . . . . . . . . . . . . . . 631 12.4.2 Reconstructing the Unknown Input . . . . . . . . . . . . . . . . . . . 633 12.4.3 Sliding Mode UIO Analysis and Construction . . . . . . . . . . . . . 634 12.4.4 Sliding Mode UIO Design Using LMIs . . . . . . . . . . . . . . . . . 648

12.5 The Edwards and Spurgeon Sliding Mode UIO . . . . . . . . . . . . . . . . . 649 12.5.1 The Canonical Form—The Design Platform . . . . . . . . . . . . . . 649 12.5.2 Observer Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 652 12.5.3 Reconstructing the Unknown Input . . . . . . . . . . . . . . . . . . . 653

12.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655 12.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 657

13 Model-based Predictive Control (MPC) 659 13.1 Simple Discrete-Time MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . 660 13.2 MPC With Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 665

13.2.1 Constraints on the Rate of Change of the Control Action . . . . . . . 666 13.2.2 Constraints on the Control Action Magnitude . . . . . . . . . . . . . 667 13.2.3 Constraints on the Plant Output . . . . . . . . . . . . . . . . . . . . 669

13.3 Solving Constrained Optimization Problems . . . . . . . . . . . . . . . . . . 670 13.3.1 Gradient Descent Method . . . . . . . . . . . . . . . . . . . . . . . . 671 13.3.2 Second-Order Necessary Conditions for a Minimum . . . . . . . . . . 671 13.3.3 Newton’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 673

13.4 Minimization Subject to Equality Constraints . . . . . . . . . . . . . . . . . 674 13.5 Lagrangian Algorithm for Equality Constraints . . . . . . . . . . . . . . . . 676 13.6 Minimization Subject to Inequality Constraints . . . . . . . . . . . . . . . . 677 13.7 Lagrangian Algorithm for Inequality Constraints . . . . . . . . . . . . . . . . 679 13.8 Observer-Based MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 682 13.9 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 682 13.10Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684

14 Proportional-Integral-Derivative (PID) Control 687 14.1 The Internal Model Principle (IMP) . . . . . . . . . . . . . . . . . . . . . . . 688 14.2 PID Controller Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 693 14.3 PID Controller Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694

14.3.1 The Open-Loop Method—Tuning Based on Open-Loop Tests on the

Plant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695 14.3.2 The Closed-Loop Method—Tuning Based on Closed-Loop Tests on the

Plant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 698

14.4 Implementing PID Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . 700 14.4.1 PIDF Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 700 14.4.2 Integrator Windup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703

14.5 Modified PID Control Schemes . . . . . . . . . . . . . . . . . . . . . . . . . 706 14.6 Digital Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 708 14.7 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 709 14.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 710

15 Fuzzy Systems 713 15.1 Motivation and Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . . 713 15.2 Standard Additive Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 718 15.3 Fuzzy Logic Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 722 15.4 Stabilization Using Fuzzy Models . . . . . . . . . . . . . . . . . . . . . . . . 736

15.4.1 Fuzzy Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 736 15.4.2 Constructing a Fuzzy Design Model Using a Nonlinear Model …. 738 15.4.3 Stabilizability of Fuzzy Models . . . . . . . . . . . . . . . . . . . . . 744 15.4.4 A Lyapunov Based Stabilizer . . . . . . . . . . . . . . . . . . . . . . 749

15.5 Stability of Discrete Fuzzy Models . . . . . . . . . . . . . . . . . . . . . . . . 756 15.6 Fuzzy Observer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 760

15.6.1 The Comparison Method for Linear Systems . . . . . . . . . . . . . . 762 15.6.2 Stability Analysis of the Closed-Loop System . . . . . . . . . . . . . 764

15.7 Adaptive Fuzzy Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 766 15.7.1 Plant Model and Control Objective . . . . . . . . . . . . . . . . . . . 767 15.7.2 Background Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767 15.7.3 Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772 15.7.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 780

15.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787 15.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 791

16 Neural Networks 799 16.1 Threshold Logic Unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 801 16.2 Backpropagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 811 16.3 Neural Fuzzy Identifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 818 16.4 Radial-Basis Function (RBF) Networks . . . . . . . . . . . . . . . . . . . . . 822

16.4.1 Interpolation Using RBF Networks . . . . . . . . . . . . . . . . . . . 824 16.4.2 Identification of a Single-Input Single-State System . . . . . . . . . . 827 16.4.3 Learning Algorithm for the RBF Identifier . . . . . . . . . . . . . . . 828 16.4.4 Growing RBF Network . . . . . . . . . . . . . . . . . . . . . . . . . . 831 16.4.5 Identification of Multivariable Systems . . . . . . . . . . . . . . . . . 835

16.5 A Self-Organizing Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 840 16.6 Hopfield Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 843

16.6.1 Hopfield Neural Network Modeling and Analysis . . . . . . . . . . . . 843 16.6.2 Analog-to-Digital Converter . . . . . . . . . . . . . . . . . . . . . . . 850

16.7 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 853 16.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855

17 Genetic and Evolutionary Algorithms 857 17.1 Genetics as an Inspiration for an Optimization Approach . . . . . . . . . . . 858 17.2 Implementing a Canonical Genetic Algorithm . . . . . . . . . . . . . . . . . 860 17.3 Analysis of the Canonical Genetic Algorithm . . . . . . . . . . . . . . . . . . 865 17.4 Simple Evolutionary Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 868 17.5 Evolutionary Fuzzy Logic Controllers . . . . . . . . . . . . . . . . . . . . . . 870

17.5.1 Vehicle Model and Control Objective . . . . . . . . . . . . . . . . . . 870 17.5.2 Case 1: EA Tunes Fuzzy Rules . . . . . . . . . . . . . . . . . . . . . 872 17.5.3 Case 2: EA Tunes Fuzzy Membership Functions . . . . . . . . . . . . 875 17.5.4 Case 3: EA Tunes Fuzzy Rules and Membership Functions . . . . . . 879

17.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 883 17.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 886 18 Chaotic Systems and Fractals 889

18.1 Chaotic Systems Are Dynamical Systems With Wild Behavior . . . . . . . . 889 18.2 Chaotic Behavior of the Logistic Equation . . . . . . . . . . . . . . . . . . . 890

18.2.1 The Logistic Equation—An Example From Ecology . . . . . . . . . . 890 18.2.2 Stability Analysis of the Logistic Map . . . . . . . . . . . . . . . . . . 893 18.2.3 Period Doubling to Chaos . . . . . . . . . . . . . . . . . . . . . . . . 902 18.2.4 The Feigenbaum Numbers . . . . . . . . . . . . . . . . . . . . . . . . 907

18.3 Fractals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 908 18.3.1 The Mandelbrot Set . . . . . . . . . . . . . . . . . . . . . . . . . . . 911 18.3.2 Julia Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 911 18.3.3 Iterated Function Systems . . . . . . . . . . . . . . . . . . . . . . . . 912

18.4 Lyapunov Exponents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 915 18.5 Discretization Chaos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 924 18.6 Controlling Chaotic Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 927

18.6.1 Ingredients of Chaotic Control Algorithm . . . . . . . . . . . . . . . . 927 18.6.2 Chaotic Control Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 929

18.7 An Application to Secure Communication . . . . . . . . . . . . . . . . . . . 941 18.7.1 Fuzzy Representation of Chaotic Systems . . . . . . . . . . . . . . . . 944 18.7.2 Constructing Secure Communication System . . . . . . . . . . . . . . 945

18.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 949 18.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 950

A Math Review 955 A.1 Notation and Methods of Proof . . . . . . . . . . . . . . . . . . . . . . . . . 956 A.2 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 958 A.3 Matrices and Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . 963 A.4 Quadratic Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 968 A.5 The Kronecker Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 971 A.6 Upper and Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . 973 A.7 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 975 A.8 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 983 A.9 Linear Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 986 A.10 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 990 A.11 Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994 A.12 Contraction Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 997 A.13 First-Order Differential Equation . . . . . . . . . . . . . . . . . . . . . . . . 1000 A.14 Integral and Differential Inequalities . . . . . . . . . . . . . . . . . . . . . . . 1002

A.14.1 The Bellman-Gronwall Lemma . . . . . . . . . . . . . . . . . . . . . 1002 A.14.2 A Comparison Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 1003

A.15 The Laplace Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1005 A.15.1 Linearity Property of the Laplace Transform . . . . . . . . . . . . . . 1007 A.15.2 Laplace Transforms of Derivatives . . . . . . . . . . . . . . . . . . . . 1009 A.15.3 Solving Differential Equations Using Laplace Transform . . . . . . . . 1011 A.15.4 Laplace Transforms of Integrals . . . . . . . . . . . . . . . . . . . . . 1013

A.16 The Z-Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1016 A.16.1 Definition and Simple Examples . . . . . . . . . . . . . . . . . . . . . 1016 A.16.2 Some Properties of the Z-Transform . . . . . . . . . . . . . . . . . . 1018

A.17 Curves and Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1021 A.18 Matrix Calculus Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1030 A.19 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1031 A.20 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1031

Bibliography 1067

Index 1067

Preface

Objectives

This book presents an introduction to the modeling, analysis, and control of linear and nonlinear dynamical systems. Its objective is to familiarize the reader with the basics of dynamical system theory while, at the same time, equipping him with the tools necessary for control system design. The emphasis is on design in order to show how dynamical system theory fits into practical applications. In particular there is a chapter on the design of proportional-integral-derivative (PID) controllers as well as a chapter devoted to the modelbased predictive control (MPC) design. In addition there is a chapter presenting a powerful control design tool—the linear matrix inequalities (LMIs). The above approaches to the controller synthesis are the “bread and butter” of control design in industry. Even in chapters that are concerned with analysis, design examples and problems are included wherever possible.

The scope of the book is broad in order to show the multidisciplinary role of dynamics and control. In particular, neural networks, fuzzy systems, and genetic algorithms are presented and an introduction to chaotic systems is provided. Classical methods and techniques of post-modern control engineering, like fuzzy logic, neural networks, and genetic algorithms are presented in a unified fashion. This book aims to demonstrate how these tools supplement the classical ones.

Coverage

Whenever one wishes to analyze a real world system, the first order of business is the construction of a mathematical model of the system. In this book, we discuss mathematical modeling of systems from mechanical and electrical engineering, as well as from physics and biology. We also construct models of dynamical systems using linguistic descriptions that employ IF-THEN rules.

Two main types of dynamical systems are common in applications: those for which the time variable is discrete and those for which the time variable is continuous. When the time variable is discrete, the dynamics of such systems are usually modeled using difference equations. In the case when the time is continuous, ordinary differential equations are frequently chosen for modeling purposes. Both types of models are considered in the book.

Nonlinear systems are emphasized to acknowledge the critical role that nonlinear phenomena play in science and technology. The models presented are the ones that can be used to design controllers. They are constructed from the control engineering point of view. This book is an invitation to the control engineering of complex, nonlinear, dynamical systems using multidisciplinary tools.

Lyapunov’s stability theory is used in the analysis of dynamical system models. The objective of the stability analysis is to determine the system behavior without solving the differential, or difference, equations modeling the system. In fact, the Lyapunov theory is used as a unifying medium for the different types of dynamical systems analyzed. In particular, optimal, sliding mode, fuzzy, and chaotic controllers, as well as unknown input observers are all constructed with the aid of the Lyapunov method.

Changes in the Second Edition

Six new chapters were added to this edition. Three chapters are concerned with controller design methods. These chapters make this text more attractive for the reader interested in applications. A chapter on unknown input observers was added to account for recent interest in applying those types of observers in fault detection. A separate chapter collects models of the different dynamical systems that are being referred to throughout the text. Finally, a chapter on linear time-varying (LTV) system was added to make this book also suitable for a first graduate course on dynamical systems. On the other hand, a number of sections in different chapters containing esoteric material at the advanced graduate level were deleted.

Intended Audience

This text can be used in a variety of ways. It is suitable for an introductory graduate-level course on systems and control. The text can also be used in an introductory course on nonlinear systems. Finally, it can be used in an introductory course to modern automatic control. No special background is necessary to follow this text beyond basic differential equations and elements of linear algebra. A one semester undergraduate junior level course on feedback systems is all that is needed to enjoy this text. The book is also suitable for self-study. To accomplish this goal, the required mathematical techniques and terminology are reviewed in the Appendix. Depending on the background, the reader may refer to the material in the Appendix as needed.

Resources

In addition to worked examples, the book also contains exercises at the end of each chapter. A number of solved problems, as well as the exercises, require the use of a computer. Although there are now many quality software packages available that could be used with this book, MATLAB^{1} was chosen because it appears to be the most widely used math-tools program in engineering circles, and it is also popular among science students. The solutions manual, with complete solutions to all end-of-chapter exercises, is available from the publisher to the instructors who adopt this book in their courses.

There is a companion web-site for this book, where the current errata are available as well as details of solutions of examples solved in the text along with MATLAB programs used to obtain the solutions. The URL for the web-site is

http://cobweb.ecn.purdue.edu/~zak/systems/Index.html

1MATLAB is a registered trademark of the MathWorks, Inc., 24 Prime Park Way, Natick, MA 01760-1500.

Phone: 508-647-7000, fax: 508-647-7001, e-mail: info@mathworks.com, http://www.mathworks.com

Chapter 1

Vocabulary and Description of Dynamical Systems

It’s in the preface [of In Retrospect]. I know the lines. I know what they say: “We shall not cease from exploration And the end of all our exploring Will be to arrive where we started And know the place for the first time.”

Now, I haven’t ceased from exploring, but I’m a little farther along than I was fifteen or thirty years ago, and I think I see a little more clearly— not as clearly as I will a few years hence or before I die—but I see events more clearly today than I saw five, ten, thirty years ago. Robert McNamara in [202, pages 344, 345]

1.1 What Is a System?

A system is a collection of interacting components. An electric motor, an airplane, as well as a biological unit such as the human arm are examples of systems. A system is characterized by two properties. They are:

1. the interrelations between the components that are contained within the system,

2. the system boundaries that separate the components within the system from the components outside.

2 CHAPTER 1. VOCABULARY AND DESCRIPTION OF DYNAMICAL SYSTEMS

The system boundaries can be real or imagined. They are elastic in the sense that we may choose, at any stage of the system analysis, to consider only a part of the original system as a system on its own. We call it a subsystem of the original system. On the other hand, we may decide to expand the boundaries of the original system to include new components. In Figure 1.1, we represent system’s boundaries using a box.

[figure: Input; System; Output]

Figure 1.1: Representation of a system.

The interactions between the system components may be governed, for example, by physical, biological, or economical laws. In dealing with systems, we are interested in the effects of external quantities upon the behavior of the system quantities. We refer to the external quantities acting on the system as the inputs to the system. The output of the system is its response to the input. The outputs are internal quantities available for measurement.

A system is a continuous-time system if it accepts continuous-time signals as its inputs and produces continuous-time signals as its outputs. We use the lowercase italic u(t) to denote the input of the continuous-time single-input system, where the time t is assumed to range from −∞ to ∞. For the multi-input continuous-time system, we use the boldface italic u(t) to denote its input, where u(t) is an m × 1 column vector, that is,

u(t) = [ u_{1}(t) ; u_{2}(t) ; ⋮ ; u_{m}(t) ] .

Similarly, the output of the single-output continuous-time system is denoted as y(t), while the output of the multi-output continuous-time system is denoted as

y(t) = [ y_{1}(t) ; y_{2}(t) ; ⋮ ; y_{p}(t) ] ,

that is, the output y(t) is a column vector composed of p components.

Figure 1.2: An example of a discrete-time signal with the sampling period h = 0.2 sec.

A system is a discrete-time system if it accepts discrete-time signals as its inputs and produces discrete-time signals as its outputs. An example of a discrete-time signal is shown in Figure 1.2.

A discrete-time signal can be viewed as a sequence of points obtained from measurements at successive times. Thus a discrete-time sinal has a natural temporal ordering. We consider only the case where all discrete-time signals have the same sampling period denoted h.

We use the lowercase italic u[k] to denote the input of the discrete-time single-input system, where k denotes discrete time instant and is assumed to range from −∞ to ∞. Thus,

u[k] := u(kh),

where the symbol “:=” denotes arithmetic assignment. Thus the statement “u[k] := u(kh)” should be interpreted to mean “u[k] becomes u(kh).” For the multi-input discrete-time system, we use the boldface italic u[k] to denote its input, where u[k] := u(kh) is an m × 1 vector, that is,

u[k] = [ u_{1}[k] ; u_{2}[k] ; ⋮ ; u_{m}[k] ] .

Similarly, the output of the single-output discrete-time system will be denoted as y[k], where y[k] := y(kh). The output of the multi-output discrete-time system will be denoted as

y[k] = [ y_{1}[k] ; y_{2}[k] ; ⋮ ; y_{p}[k] ] ,

that is, the output y[k] is a column vector composed of p components.

1.1.1 Memoryless, Causal, and Lumped Systems

A system is a memoryless system if its output at time t_{0} depends only on the input applied at t_{0} and is independent of the input applied before t_{0}, that is, current output of a memoryless system depends only on its current input; it is independent of past inputs. An example of memoryless system is a circuit consisting of only resistors.

In a causal or non-anticipatory system its current output may depend on current and past inputs but not on future inputs. In a non-causal system, its current output depends on future input. Such a system can predict or anticipate what inputs will be applied in the future. Physical systems have no such capability, that is, physical systems are causal.

To proceed, we consider, following Kailath [168, p. 63], a simple system consisting of a capacitor and a voltage source as shown in Figure 1.3. The input u is a current through the

Figure 1.3: A circuit where the the past input from −∞ up to t affects the current output at time t.

capacitor and the output y is a voltage x across the capacitor. The relation between voltage and current for a capacitor has the form,

y(t) = \frac{1}{C} ∫_{−∞}^{t} u(τ)dτ.

It follows from the above that the system output is affected by its past input from −∞ to time t. Tracking the past inputs may be impractical if not impossible. To alleviate the problem, the concept of state is introduced that we discuss next.

The condition or the state of the system are described by the state variables denoted x_{i}. The state variables provide the information that together with the knowledge of the system inputs enable us to determine the future state of the system. In other words, “A dynamical system consists of a set of possible states, together with a rule that determines the present state in terms of past states” [7, page 1].

Definition 1.1 [55, p. 6] The state x(t_{0}) of a system at time t_{0} is the information at t_{0} that, together with the input u(t), for t ≥ t_{0}, determines uniquely the output y(t) for all t ≥ t_{0}.

◇

Returning to the example above, we select the voltage across the capacitor as the system state, which is also its output. Then, we write

y(t) = x(t)
= \frac{1}{C} ∫_{−∞}^{t} u(τ)dτ
= \frac{1}{C} ∫_{−∞}^{t_{0}} u(τ)dτ + \frac{1}{C} ∫_{t_{0}}^{t} u(τ)dτ
= x(t_{0}) + \frac{1}{C} ∫_{t_{0}}^{t} u(τ)dτ.

Thus, if we know the initial state x(t_{0}) and the current input u(t), then the past input is irrelevant in the process of calculating the output y(t) for t ≥ t_{0} because the initial state “summarizes” the past input. Kailath [168, p. 63] compares this with a situation when one is faced with a new expenditure; then it is the current state of one’s bank balance and not the many ways it could have reached its present value that is relevant to the issue at hand.

In summary, the voltage across the capacitor is

x(t) = x(t_{0}) + \frac{1}{C} ∫_{t_{0}}^{t} u(τ)dτ.

Differentiating the above, we obtain equivalent representation in the form of a differential equation,

ẋ(t) = \frac{1}{C} u(t), x(t_{0}) = x_{0}.

A system is a lumped system if the number of state variables is finite, that is, the state vector x is composed of finite number of components. A simple example of lumped system is the circuit shown in Figure 1.3. More examples of lumped systems are given in Section 2.4.

A system is distributed if its state consists of infinitely many components. We now give an example of a distributed system.

Example 1.1 [55, p. 7] Consider the unit-time delay system model of the form,

y(t) = u(t − 1).

In this example, the output is the input delayed by one time unit. To determine y(t) for t > t_{0}, we need the information about u(t) on the time interval [t_{0} − 1, t_{0}]. Thus the initial state of this time-delay system is the set of points, {u(t) : t ∈ [t_{0} − 1, t_{0}]}. However, there are infinitely many points u(t) on this interval. Thus, the above time-delay system is a distributed system because its state is infinitely dimensional.

In practice it is often not possible or too expensive to measure or determine the values of all of the state variables. Instead, only their subset or combination can be measured. The system quantities whose behavior can be measured or observed constitute the system’s outputs.

1.1.2 Formulation of the Control Problem

In engineering applications, when dealing with dynamical systems, we are interested in specifying the system inputs that force the system states or outputs to behave with time in some pre-specified manner. That is, we are interested in controlling the system states or outputs. This is accomplished by means of a controller whose task is to produce the required system’s inputs that in turn result in the desired system’s outputs. An interconnection of the system and a controller is called a control system. In Figure 1.4 and 1.5, we show two different types of control systems that we discuss in the next section. Constructing a controller is a part of the control problem. The essential elements of the control problem, as described by Owens [260, page 181], are:

1. a dynamical system to be controlled,

2. a specified objective for the system,

3. a set of admissible controllers, and

4. a means of measuring the performance of any given control strategy to evaluate its effectiveness.

We now examine these elements one by one. The first step in the controller design procedure is the construction of a truth model of the dynamics of the process to be controlled. The truth model is a simulation model that includes all the relevant characteristics of the process. The truth model is too complicated for use in the controller design. Thus, we need to develop a simplified model that can be used to design a controller. Such a simplified model is labeled by Friedland [108] as the design model. The design model should capture the essential feature of the process. During the modeling process, properties of the system, physical constraints, and technical requirements should be taken into account. We discuss in more detail the process of constructing mathematical models of physical systems in Chapter 2.

The objective of a control system is to complete some specified task. This can be expressed as a combination of constraints on the output or state variables and limits on the time available to complete the control objective. For example, the objective of a controller might be to force the output, of a single output system, to settle within a certain percentage of the given value after pre-specified time.

Control input signals are obtained from physical devices capable of providing only a limited amount of energy. We term the class of controllers that can be considered for the given control design problem as the set of admissible controllers.

We can evaluate the performance of any given control law by visual inspection of the transient characteristics of the controlled system after the design is complete. But this method of assessment of the effectiveness of the given control strategy is inaccurate. We therefore try to quantify the means of evaluating performance in the form of a performance index or cost functional at the beginning of the design process. We construct a numerical index of performance whose value reflects the quality of any admissible controller in accomplishing the system objective. This performance index is assumed to have the property that its numerical value decreases as the quality of the controller increases. This is the reason why we sometimes refer to the performance index as the penalty functional. The “best” controller then is the one that produces the smallest value of the performance index. We call an admissible controller that simultaneously ensures the completion of the system objective and the minimization of the performance index an optimal controller for the system.

[figure: Controller; Reference input; Desired output; Controller output; Plant input; Plant; Actual output]

Figure 1.4: Open-loop control system.

1.1.3 Open-Loop Versus Closed-Loop

We distinguish between two types of control systems. They are:

• open-loop control systems, and

• closed-loop control systems.

An open-loop control system usually contains:

1. A process to be controlled, labeled plant,

2. The controlling variable of the plant, called the plant input, or just input for short,

3. The controlled variable of the plant, called the plant output, or just output for short,

4. A reference input, which dictates the desired value of the output,

5. A controller that acts upon the reference input in order to form the system input which would force the behavior of the output in accordance with the reference signal.

Note that the plant and controller themselves can be considered as systems on their own. A schematic representation of an open-loop system is depicted in Figure 1.4. In an open-loop control system the output has no influence on the input or reference signal. The controller operates without taking into account the output. Thus, the plant input is formed with no influence of the output. A household appliance such as an iron is a simple example of an open-loop control system. In this example, we consider the iron itself as the plant or process to be controlled. Here we set the temperature control mechanism for the desired temperature. This setting, for example very hot, is the reference input. We can view the potentiometer, inside the iron, as a controller, or a control actuator. It regulates the temperature of the iron by allowing the necessary amount of resistance to produce the desired temperature. The plant’s output is the temperature of the heated iron.

[figure: Desired output; Reference input; +; −; Actuating signal; Controller; Controller output; Plant input; Plant; Actual output; Sensor; Measured output]

Figure 1.5: Closed-loop control system.

Not every control system is an open-loop system. Another type of a control system is a closed-loop system. We can convert an open-loop system into a closed-loop system by adding, to an open-loop system, the following components:

6. The feedback loop where the output signal is measured with a sensor and then the measured signal is fed back to the summing junction,

7. The summing junction, where the measured output signal is subtracted from the reference, command, input signal in order to generate an error signal, also labeled as an actuating signal.

A schematic of a closed-loop control system, or feedback system, is shown in Figure 1.5. In a closed-loop system the error signal causes an appropriate action of the controller, which in turn “instructs” the plant to behave in a certain way in order to approach the desired output, as specified by the reference input signal. Thus, in the closed-loop system, output information is fed back to the controller, and the controller then appropriately modifies the plant output behavior. A controller, also called a compensator, can be placed either in the forward loop, or in the feedback loop. Thus, a central component of the closed-loop system is feedback. Norbert Wiener defined feedback in 1954 as follows: “Feedback is a method of controlling a system by reinserting into it the results of its past performance.” [175, p. 439].

If the goal of the controller is to maintain the output at a constant value, then we have a regulator control system. A “centrifugal speed regulator”, or the “flyball governor,” commonly known as Watt’s governor, is an example of a regulator in a closed-loop system. A schematic of Watt’s governor is shown in Figure 1.6.

10 CHAPTER 1. VOCABULARY AND DESCRIPTION OF DYNAMICAL SYSTEMS Figure 1.6: A simplified schematic of the flyball governor.

The task of the centrifugal speed regulator is to control setting of a throttle valve automatically to maintain the desired engine speed. The nominal speed is set by adjusting the valve in the throttle. As the engine speeds up the weights are thrown outwards, the throttle partially closes and the engine slows down. As the engine slows down below its nominal speed, the throttle is opened up and the engine gains its speed. The operation of an engine equipped with a centrifugal speed regulator can be represented using the block diagram of Figure 1.5. The reference input to the closed-loop system is the desired engine speed. The desired engine speed is obtained by appropriately setting the valve. The system output is the actual engine speed. The flyball governor is our controller, while the engine represents the plant. The control signal generated by the controller is the setting of the throttle valve.

We mention at this juncture that Watt’s application of the centrifugal speed governor in 1788, to regulate the speed of the steam engine, marks the starting point for the development of automatic control, and in particular feedback control, as a science [226]. Before 1788, steam engines were controlled manually using the throttle valve. The flyball governor spread widely and quickly. It has been estimated that by 1868 there were about 75,000 flyball governors working in England alone [226]. The flyball governor is credited for the hatching of the industrial revolution. We devise a mathematical model of this revolutionary invention in Subsection 2.4.1, and in Section 7.7 we perform a quantitative analysis of its dynamical behavior.

If the goal of the controller is to force the output to follow a desired trajectory, then we have a servomechanism or tracking control system. As we mentioned before, to design a controller, we need first to analyze the plant quantitatively. The analysis requires a mathematical or linguistic description of the interrelations between the system quantities themselves as well as the interrelations between system quantities and system inputs.

1.2 Linearity

It follows from Definition 1.1 that a system accepts the input, u(t), and the initial state, x(t_{0}), to produce its output, y(t). We view a system as an operator that acts on two inputs, u(t) and x(t_{0}) to produce an output, y(t). We use the symbol L to represent a system. Then its operation can be described as

L(u(t), x(t_{0})) = y(t).

A system L is said to possess an additivity property if for any t_{0} and any x_{i}(t_{0}) and u_{i}(t), 12 CHAPTER 1. VOCABULARY AND DESCRIPTION OF DYNAMICAL SYSTEMS t ≥ t_{0}, if

L(u_{i}(t), x_{i}(t_{0})) = y_{i}(t), i = 1, 2,

then

L(u_{1}(t) + u_{2}(t), x_{1}(t_{0}) + x_{2}(t_{0})) = y_{1}(t) + y_{2}(t).

We illustrate the above property in Figure 1.7.

[figure: u_{1}(t) + u_{2}(t); L(·, ·); System; x_{1}(t_{0}) + x_{2}(t_{0}); y_{1}(t) + y_{2}(t)]

Figure 1.7: Illustration of the additivity property.

A system L is said to possess a homogeneity property if for any t_{0} and any x(t_{0}) and u(t), t ≥ t_{0}, if

L(u(t), x(t_{0})) = y(t),

then

L(cu(t), cx(t_{0})) = cy(t),

where c is real or complex constant. We illustrate the above property in Figure 1.8.

[figure: c_{i}u_{i}(t); L(·, ·); System; c_{i}x_{i}(t_{0}); c_{i}y_{i}(t)]

Figure 1.8: Illustration of the homogeneity property.

A system L is said to possess a superposition property if for any t_{0} and any x_{i}(t_{0}) and u_{i}(t), t ≥ t_{0}, if

L(u_{i}(t), x_{i}(t_{0})) = y_{i}(t), i = 1, 2,

[figure: c_{1}u_{1}(t) + c_{2}u_{2}(t); L(·, ·); System; c_{1}x_{1}(t_{0}) + c_{2}x_{2}(t_{0}); c_{1}y_{1}(t) + c_{2}y_{2}(t)]

Figure 1.9: Illustration of the superposition property.

then

L(c_{1}u_{1}(t) + c_{2}u_{2}(t), c_{1}x_{1}(t_{0}) + c_{2}x_{2}(t_{0})) = c_{1}y_{1}(t) + c_{2}y_{2}(t),

where c_{1} and c_{2} are real or complex constants. We illustrate the above property in Figure 1.9.

Note that the superposition property is a combination of the additivity and homogeneity properties.

Definition 1.2 A system is said to be linear if it possesses the superposition property. A system is said to be nonlinear if it does not have the superposition property.

◇

For an informative discussion on some subtleties in the definition of linearity, we recommend Kailath [168, Section 1.1].

If u(t) = 0 for t ≥ 0, then the system’s output will be produced exclusively by the initial condition x(t_{0}). We call such an output the zero-input response and denote it as y_{zi}.

If, on the other hand, the initial state x(t_{0}) = 0, then the output will be produced exclusively by the input. This output is called the zero-state response and is denoted y_{zs}.

If the system is linear, then the additivity property implies that the total response, y(t) is the sum of the zero-input and zero-state responses, that is,

y(t) = y_{zi}(t) + y_{zs}(t), t ≥ t_{0}.

For linear systems the superposition property holds for zero-state and zero-input responses separately as well. In our discussion here, for the sake of simplicity, we do not display the second argument in the operator L when we set zero initial state or zero input.

Now, if we set the initial state to zero, x(t_{0}) = 0 and if

L(u_{j}(t)) = y_{zs,j}(t), j = 1, 2,

then

L(c_{1}u_{1}(t) + c_{2}u_{2}(t)) = c_{1}y_{zs,1}(t) + c_{2}y_{zs,2}(t).

Similarly, if we set u(t) = 0 for t ≥ t_{0} and if

L(x_{j}(t_{0})) = y_{zi,j}(t), j = 1, 2,

then

L(c_{1}x_{1}(t_{0}) + c_{2}x_{2}(t_{0})) = c_{1}y_{zi,1}(t) + c_{2}y_{zi,2}(t).

1.3 Input-Output Description of Linear Systems

In this Section, we develop a mathematical description of the zero-state response of linear systems. We assume that the initial state is zero and so the system’s output is exclusively due to the system input.

Definition 1.3 We say that the system is relaxed at t_{0} if its initial condition at t_{0} is zero, that is, x(t_{0}) = 0.

◇

To proceed, we need certain facts about the impulse function, δ(t).

1.3.1 The Impulse Function and Its Properties

The impulse function, denoted δ(t), also called the Dirac function, is a signal of infinite amplitude, zero duration, and unity area. We can construct an impulse function as the limit of pulse functions

p_{ε_{i}}(t) = \frac{1}{ε_{i}} (1(t) − 1(t − ε_{i}))

as ε_{i} → 0, where 1(t) is the unit step function defined as

1(t) = \{ \begin{matrix}1 & if & t ≥ 0 \\ 0 & if & t < 0. \end{matrix}

The pulse p_{ε_{i}} is illustrated in Figure 1.10. Note that the pulse functions have the following features as ε_{i} → 0:

1. the amplitude approaches infinity,

2. the duration of the pulses approaches zero, Figure 1.10: Generating the unit impulse function as the limit of the pulse functions.

3. the area under each pulse is constant; in our example the area equals unity.

The unit impulse function is defined as

∫_{−∞}^{∞} δ(t)dt = 1 and δ(t) = 0 for t ≠ 0.

The above definition states that the area under the impulse function is constant. The area represents the strength of the impulse function. The impulse function of strength K is denoted Kδ(t). Its graphical representation is depicted in Figure 1.11. The strength of the impulse is shown next to the arrow’s head. The shifted impulse function of strength K is also shown in Figure 1.11. The unit impulse function can be thought of as a derivative of the unit step function, that is,

δ(t) = \frac{d1(t)}{dt}.

The function shown in Figure 1.12 approaches the unit step function as ε → 0. The function shown in Figure 1.13, which is the derivative of the function from Figure 1.12, approaches the unit impulse function as ε → 0.

The impulse function has the sifting property,

∫_{−∞}^{∞} f(t)δ(t − a)dt = f(a)  (1.1)

where f(t) is a continuous function of time. It follows from the above that the impulse function sifts out everything except the value of f at t = a—hence the name of the property.

Figure 1.11: Graphical representation of the impulse Kδ(t) and Kδ(t − a).

Figure 1.12: The function 1_{ϵ} approaches the unit step as ε → 0.

To verify the validity of the sifting property, we note that δ(t − a) is zero everywhere except at t = a. Hence, we can write,

∫_{−∞}^{∞} f(t)δ(t − a)dt = ∫_{a−ε}^{a+ε} f(t)δ(t − a)dt.

By assumption, f is continuous at a. Therefore, it must take the value of f(a) as t → a. Thus,

∫_{a−ε}^{a+ε} f(t)δ(t − a)dt = f(a) ∫_{a−ε}^{a+ε} δ(t − a)dt = f(a),

which was to be demonstrated.

Figure 1.13: The derivative of 1_{ε}(t) shown in Figure 1.12. As ε → 0, δ_{ε}(t) approches δ(t).

1.3.2 Zero-State Response Description of Continuous-Time, Relaxed, Causal Systems

Suppose that the system under consideration is a single-input single-output (SISO) linear system. Let δ_{Δ}(t − t_{1}) be the pulse depicted in Figure 1.14. The pulse has width Δ and height 1/Δ.

We can use the pulse δ_{Δ}(t − t_{1}) to approximate an input signal u(t) as illustrated in Figure 1.15. That is, we can approximate the control signal u(t) as

u(t) ≈ ∑_{i} u(t_{i})δ_{Δ}(t − t_{i})Δ.  (1.2)

Let y(t) = g_{Δ}(t, t_{i}) be the system output at time t produced by the pulse u(t) = δ_{Δ}(t−t_{i}) applied at time t_{i}—see Figure 1.16. Then by the homogeneity property, the input u(t) = δ_{Δ}(t − t_{i})u(t_{i})Δ would produce the output, y(t) = g_{Δ}(t, t_{i})u(t_{i})Δ. Applying the additivity property, we conclude that the input given by (1.2) will produce the output

y(t) ≈ ∑_{i} g_{Δ}(t, t_{i})u(t_{i})Δ.  (1.3)

Now if Δ → 0, then

δ_{Δ}(t − t_{i}) → δ(t − t_{i})

and the corresponding output of the system is denoted g(t, t_{i}). Next, as Δ → 0 the approximation in (1.3) becomes an equality, the summation an integration, the discrete time t_{i} becomes a continuum that we denote using the symbol τ, while Δ will be represented as dτ. Taking the above into account, (1.3) takes the form,

y(t) = ∫_{−∞}^{∞} g(t, τ)u(τ)dτ.  (1.4)

Figure 1.14: The pulse δ_{Δ}(t − t_{1}) at t = t_{1}.

Figure 1.15: Approximating the control signal u(t).

[figure: δ_{Δ}(t − t_{i}); System; g_{Δ}(t, t_{i})]

Figure 1.16: The system output g_{Δ}(t, t_{i}) is produced by the input δ_{Δ}(t − t_{i}).

We note that g(t, τ) is a function of two variables. The first variable denotes the time t at which the output is observed, while the second variable, τ, denotes the time at which the impulse input is applied.

We call g(t, τ) the impulse response because it is the system’s output when excited with the impulse input.

For the causal system, the output cannot appear before the input is applied. Therefore, for the causal systems,

g(t, τ) = 0 for t < τ

and we can replace the upper integration limit in (1.4) with t to obtain

y(t) = ∫_{−∞}^{t} g(t, τ)u(τ)dτ.  (1.5)

For the relaxed system at t_{0}, its initial state, x(t_{0}) = 0. Hence, the system output is produced exclusively by the input u(t) for t ≥ t_{0}. Thus we can replace the lower integration limit in (1.5) with t_{0}, and we obtain for the linear, relaxed at t_{0}, causal system the following expression for its response in terms of its impulse response;

y(t) = ∫_{t_{0}}^{t} g(t, τ)u(τ)dτ.  (1.6)

For the multi-input multi-output (MIMO) linear, relaxed at t_{0}, casual system with p output components and m inputs, the above expression generalizes to

y(t) = ∫_{t_{0}}^{t} G(t, τ)u(τ)dτ,  (1.7)

where

G(t, τ) = [ \begin{matrix}g_{11}(t, τ) & g_{12}(t, τ) & ⋯ & g_{1m}(t, τ) \\ g_{21}(t, τ) & g_{22}(t, τ) & ⋯ & g_{2m}(t, τ) \\ ⋮ & ⋮ & ... & ⋮ \\ g_{p1}(t, τ) & g_{p2}(t, τ) & ⋯ & g_{pm}(t, τ) \end{matrix} ] .

In the above, g_{ij}(t, τ) is the impulse response at time t at the i-th output due to an impulse applied at time τ at the j-th input, with other inputs set to zero.

We refer to the matrix G as the impulse response matrix.

1.3.3 Input-Output Description of Continuous Time-Invariant Systems

Suppose that we have a system with the initial state x(t_{0}), the input u(t) applied from t_{0} that produced the output y(t) for t ≥ t_{0}. We compactly describe the above mental experiment as

L(u(t), x(t_{0})) = y(t), for t ≥ t_{0}.

The above system is said to be time-invariant if for any T, if the initial state is shifted to time t_{0} + T, the same input is applied from t_{0} + T rather than at t_{0}, then the output is the same except that now it starts to appear at t_{0} + T, that is,

L(u(t − T), x(t_{0} + T)) = y(t − T), for t ≥ t_{0} + T.

Therefore, the impulse response of a linear, relaxed at t_{0}, causal, time invariant system can be described as

g(t, τ) = g(t + T, τ + T) = g(t − τ, 0),  (1.8)

which we compactly denote as g(t − τ). Note that the impulse response of a linear timeinvariant (LTI) system, g(t − τ), is a function of one variable, where g(t) = g(t − 0) is the system output at time t to an impulse applied at τ = 0. The LTI system is causal if g(t) = 0 for t < 0.

Taking into account (1.8), we represent the expression for the output (1.6) of a relaxed at t_{0}, causal, LTI system as

y(t) = ∫_{0}^{t} g(t − τ)u(τ)dτ,  (1.9)

where, without loss of generality, we replaced t_{0} with 0. The integration in (1.9) is known as the convolution integral. The above integral relation is also written in a shorthand notation as

y(t) = g(t) ∗ u(t),

where the asterisk signifies the integral relation between g(t) and u(t). The integral relation g(t) ∗ u(t) is read as “g(t) is convolved with u(t).”

Note that the convolution is a commutative operation, that is,

y(t) = ∫_{0}^{t} g(t − τ)u(τ)dτ = ∫_{0}^{t} g(τ)u(t − τ)dτ,  (1.10)

which can be represented as

y(t) = g(t) ∗ u(t) = u(t) ∗ g(t).

A very useful tool in the analysis and design of LTI dynamic systems is the Laplace transform that we review in Section A.15. Applying the Laplace transform to (1.9) gives

Y (s) = L(y(t))
= ∫_{0^{−}}^{∞} (∫_{τ=0}^{t} g(t − τ)u(τ)dτ) e^{−st}dt.

For causal systems, g(t) = 0 for t < 0^{−}; equivalently, g(t − τ) = 0 for τ > t^{−}. This means that we can replace the upper integration limit t in the above with ∞, and performing some simple manipulations, we obtain

Y (s) = ∫_{0^{−}}^{∞} (∫_{τ=0^{−}}^{∞} g(t − τ)u(τ)dτ) e^{−st}dt
= ∫_{0^{−}}^{∞} (∫_{τ=0^{−}}^{∞} g(t − τ)u(τ)dτ) e^{−s(t−τ)}e^{−sτ}dt
= ∫_{0^{−}}^{∞} (∫_{τ=0^{−}}^{∞} g(t − τ)e^{−s(t−τ)}dt) u(τ)e^{−sτ}dτ
= ∫_{0^{−}}^{∞} g(t − τ)e^{−s(t−τ)}dt ∫_{τ=0^{−}}^{∞} u(τ)e^{−sτ}dτ
= G(s)U(s),  (1.11)

where G(s) is called the transfer function of the system and U(s) is the Laplace transform of the input signal.

The Laplace transform of the convolution of two functions of time is equal to the product of their Laplace transforms, that is,

L(g(t) ∗ u(t)) = L(g(t))L(u(t)) = G(s)U(s).

Thus, to the product in the s-domain there corresponds the convolution in the time domain.

If a linear time-invariant system is lumped, its transfer function, G(s), is a rational function of s, that is,

G(s) = \frac{N(s)}{D(s)},

where N(s) and D(s) are polynomials in s.

For the MIMO LTI dynamical system, (1.11) takes the form,

[ Y_{1}(s) ; Y_{2}(s) ; ⋮ ; Y_{p}(s) ] = [ \begin{matrix}G_{11}(s) & G_{12}(s) & ⋯ & G_{1m}(s) \\ G_{21}(s) & G_{22}(s) & ⋯ & G_{2m}(s) \\ ⋮ & ⋮ & ... & ⋮ \\ G_{p1}(s) & G_{p2}(s) & ⋯ & G_{pm}(s) \end{matrix} ] [ U_{1}(s) ; U_{2}(s) ; ⋮ ; U_{m}(s) ] .

We represent the above in a vector-matrix notation as

Y (s) = G(s)U(s),

where G(s) is called the transfer function matrix.

Example 1.2 If we apply the impulse input in the unit-time delay system of Example 1.1, then the output is δ(t − 1). Hence the impulse response of this system is

g(t) = δ(t − 1).  (1.12)

The transfer function of the unit-time delay system of Example 1.1 is

G(s) = e^{−s},  (1.13)

which is not a rational function because this system is distributed.

Example 1.3 The integrator can be described as

y(t) = k_{I} ∫_{−∞}^{t} u(τ)dτ,  (1.14)

where k_{I} is the gain. We can equivalently describe the integrator as

\frac{dy(t)}{dt} = k_{I}u(t).  (1.15)

The impulse response of the integrator is the unit step function also known as the Heaviside step function.

The transfer function of the integrator described by (1.14) or by (1.15) is

G(s) = \frac{k_{I}}{s} .

Example 1.4 The ideal differentiator is described as

y(t) = k_{D} \frac{du(t)}{dt},

where k_{D} is the differentiator’s gain. Its impulse response is

g(t) = k_{D} \frac{dδ(t)}{dt}
= \{ \begin{matrix}0 & for & t ≠ 0 \\ +∞ & for & t = 0^{−} \\ −∞ & for & t = 0^{+}, \end{matrix}  (1.16)

The transfer function of the ideal differentiator is

G(s) = k_{D}s.

1.3.4 State-Space Description of Continuous Time-Invariant Systems

Every linear time-invariant (LTI) lumped causal system can be described by a set of equations of the form,

ẋ(t) = Ax(t) + Bu(t), x(0) = x_{0}  (1.17)
y(t) = Cx(t) + Du(t),  (1.18)

where

x(t) = [ x_{1}(t) ; x_{2}(t) ; ⋮ ; x_{n}(t) ]

is the state vector. The matrices A, B, C, and D are constant real matrices. A block diagram representation of the state-space description given by (1.17) and (1.18) is depicted in Figure 1.17.

Figure 1.17: Block diagram representation of the state-space description given by (1.17) and (1.18).

Taking the Laplace transform of (1.17) and (1.18) gives

sX(s) − x(0^{−}) = AX(s) + BU(s)  (1.19)
Y (s) = CX(s) + DU(s).  (1.20)

We compute X(s) from (1.19),

X(s) = (sI_{n} − A)^{−1} x(0^{−}) + (sI_{n} − A)^{−1} BU(s)

Substituting the above into (1.20) yields

Y (s) = C (sI_{n} − A)^{−1} x(0^{−}) + C (sI_{n} − A)^{−1} BU(s) + DU(s).

If the initial condition is zero, then

Y (s) = (C (sI_{n} − A)^{−1} B + D) U(s),

where

G(s) = C (sI_{n} − A)^{−1} B + D  (1.21)

is the transfer function matrix as a function of the parameters of a state-space description of a given causal, lumped LTI system.

1.3.5 Input-Output Description of Discrete Time-Invariant Systems

Recall that a discrete-time system accepts discrete-time signals and produces discrete-time signals as its output.

As in with a continuous-time system, a discrete-time system is linear if it possesses the superposition property.

The response of a discrete-time linear system is the sum of the zero-state and zero-input responses. Both the zero-state and zero-input responses possess the superposition property.

A discrete-time system is causal if its current output depends on the current and past inputs.

The discrete system state, x[k_{0}], is the information at time k_{0} that together with u[k] for k ≥ k_{0} uniquely determines the system output y[k] for k ≥ k_{0}.

If the number of the components in the state vector x is finite, then this discrete system is lumped; otherwise it is distributed. If there is a time delay in a continuous-time system, then this system is distributed. On the other hand, if in a discrete-time system the time delay is an integer multiple of the sampling period h, then such a system is a lumped system.

To proceed, we define the impulse sequence denoted δ[k − m] and defined as

δ[k − m] = \{ \begin{matrix}1 & if & k = m \\ 0 & if & k ≠ m, \end{matrix}

where both k and m are integers that denote sampling instances.

Suppose now that we have an input sequence denoted u[k]. We express this input sequence as

u[k] = ∑_{m=−∞}^{∞} u[k]δ[k − m].  (1.22)

Let g[k, m] denote the discrete-time system output at time k when excited with the impulse sequence, δ[k−m], at the time instant m. Because by assumption the discrete system under consideration is linear, hence by the homogeneity property the input δ[k − m]u[m] will yield the output, g[k, m]u[m]. Combining the above with the additivity property, we conclude that the input, ∑_{m=−∞}^{∞} δ[k − m]u[m] will yield the output, ∑_{m=−∞}^{∞} g[k, m]u[m]. We illustrate the above in Figure 1.18.

In a causal discrete-time linear system, no output can appear before an input is applied. Hence,

g[k, m] = 0 for k < m.

∑_{m=−∞}^{∞} δ[k − m]u[m]

Linear Discrete System

∑_{m=−∞}^{∞} g[k, m]u[m]

Figure 1.18: Output of linear discrete-time system in response to the input sequence (1.22).

If, in addition, the system is relaxed at k_{0}, then the output will take the form,

y[k] = ∑_{m=k_{0}}^{k} g[k, m]u[m].

Note that for time-invariant linear causal system we can always set k_{0} = 0 to obtain

y[k] = ∑_{m=0}^{k} g[k − m]u[m].  (1.23)

The above is referred to as a discrete convolution. It is easy to check that the discrete convolution possesses the commutativity property, that is,

y[k] = ∑_{m=0}^{k} g[k − m]u[m] = ∑_{m=0}^{k} g[m]u[k − m].  (1.24)

Applying the z-transform to (1.23) gives

Y (z) = Z(y[k])
= ∑_{k=0}^{∞} (∑_{m=0}^{k} g[k − m]u[m]) z^{−k}.

Taking into account the causality assumption, we can replace the upper integration limit k with ∞. Then performing simple manipulations, we obtain

Y (z) = ∑_{k=0}^{∞} (∑_{m=0}^{∞} g[k − m]u[m]) z^{−(k−m)}z^{−m}
= ∑_{m=0}^{∞} (∑_{m=0}^{∞} g[k − m]z^{−(k−m)}) u[m]z^{−m}
= (∑_{l=0}^{∞} g[l]z^{−l}) (∑_{m=0}^{∞} u[m]z^{−m})
= G(z)U(z),

where we interchanged the order of summations, introduced the new variable l = k − m and took into account the fact that g[l] = 0 for l < 0.

The z-transform of the impulse response, denoted G(z), is called the discrete transfer function. Note that transfer functions describe only the zero-state responses.

Example 1.5 [55, p. 33] Consider the unit-sampling-time delay system model of the form,

y[k] = u[k − 1].

The impulse response of the above system is

g[k] = δ[k − 1].

The discrete transfer function is obtained by taking the z-transform of the impulse response. We obtain

G(z) = Z(δ[k − 1])
= \frac{1}{z}.

The above transfer function is a rational function of z. Thus the above system is a lumped system. Note that in the continuous-time case, systems involving time delays are distributed system. As the above example shows, this is not necessarily the case in discrete-time systems.

1.3.6 State-Space Description of Discrete Time-Invariant Systems

Every discrete linear time-invariant (LTI) lumped causal system can be described by a set of equations of the form,

x[k + 1] = Ax[k] + Bu[k], x[0] = x_{0}  (1.25)
y[k] = Cx[k] + Du[k],  (1.26)

where

x[k] = [ x_{1}[k] ; ⋮ ; x_{n}[k] ]

Figure 1.19: Block diagram representation of the state-space description given by (1.25) and (1.26).

is the state vector. The matrices A, B, C, and D are constant real matrices. A block diagram representation of the state-space description given by (1.25) and (1.26) is depicted in Figure 1.19.

Let X(z) denote the z-transform of x[k], that is,

X(z) = Z(x[k]).

Then, the z-transform of x[k + 1] is

Z(x[k + 1]) = ∑_{k=0}^{∞} x[k + 1]z^{−k}
= z ∑_{k=0}^{∞} x[k + 1]z^{−(k+1)}
= z (∑_{j=1}^{∞} x[j]z^{−j} + x[0] − x[0])
= z (X(z) − x[0]) .  (1.27)

Applying the z-transform to (1.25) and (1.26) and taking into account (1.27) gives

zX(z) − zx[0] = AX(z) + BU(z), x[0] = x_{0}  (1.28)
Y (z) = CX(z) + DU(z).  (1.29)

We calculate X(z) from (1.28) to obtain

X(z) = (zI_{n} − A)^{−1} zx[0] + (zI_{n} − A)^{−1} BU(z).

Substituting the above into (1.29) yields

Y (z) = C (zI_{n} − A)^{−1} zx[0] + C (zI_{n} − A)^{−1} BU(z) + DU(z).

If set the initial condition to zero, that is, x[0] = 0, then we have

Y (z) = (C (zI_{n} − A)^{−1} B + D) U(z).

Thus the transfer function of the discrete time-invariant system described by (1.25) and (1.26) is

G(z) = C (zI_{n} − A)^{−1} B + D.  (1.30)

Note that the above is the discrete equivalent of (1.21).

We add that to a model represented in a state-space format, there corresponds a unique transfer function matrix. On the hand, there are infinitely many state-space representations that have the same transfer function matrix.

A classification of linear systems discussed by us in this chapter is shown in Table 1.1.

Table 1.1: Table summarizing linear system classification discussed in Chapter 1; adapted from Chen [55, p. 37].

System  Internal Description External Description

Distributed, linear y(t) = ∫_{t_{0}}^{t} G(t, τ)u(τ)dτ

Lumped, linear \begin{aligned}ẋ & = & A(t)x + B(t)u \\ y & = & C(t)x + D(t)u \end{aligned} y(t) = ∫_{t_{0}}^{t} G(t, τ)u(τ)dτ

Distributed, linear, time-invariant

y(t) = ∫_{t_{0}}^{t} G(t − τ)u(τ)dτ
Y (s) = G(s)U(s),

G(s) irrational

Lumped, linear, time-invariant

\begin{aligned}ẋ & = & Ax + Bu \\ y & = & Cx + Du \end{aligned} y(t) = ∫_{t_{0}}^{t} G(t − τ)u(τ)dτ Y (s) = G(s)U(s),

G(s) rational

1.4 From Transfer Function to State-Space Description

If the Laplace variable s is replaced in (1.21) with the z-transform variable z then we obtain (1.30), and vice versa. Thus a method of converting a transfer function into a state-space representation for continuous linear time-invariant lumped systems is applicable also for discrete linear time-invariant lumped systems.

We say that a transfer function G(s) is realizable if there exists a quadruple of constant matrices (A, B, C, D) such that G(s) = C (sI_{n} − A)^{−1} B + D. We call such a quadruple (A, B, C, D) a realization of G(s).

Definition 1.4 The dimension of a realization is the size of the matrix A, that is, if A is an n-by-n matrix then we say that the dimension of the corresponding realization is n.

◇

We begin our discussion with a single-input single-output (SISO) system modeled by a transfer function,

\frac{Y (s)}{U(s)} = G(s)
= \frac{N(s)}{D(s)}
= \frac{b_{m}s^{m} + b_{m−1}s^{m−1} + ⋯ + b_{1}s + b_{0}}{s^{n} + a_{n−1}s^{n−1} + ⋯ + a_{1}s + a_{0}}  (1.31)

that is a proper rational function, which means that

deg_{s} N(s) ≤ deg_{s} D(s).

Note that in (1.31) the highest coefficient of the denominator polynomial is unity. If this was not the case, we would divide the numerator and the denominator by the highest coefficient of the denominator thus forcing a_{n} = 1.

If G(s) is proper but not strictly, that is, deg_{s} N(s) = deg_{s} D(s), equivalently, m = n, then we divide the numerator N(s) by the denominator D(s) to obtain

G(s) = G(s)_{sp} + G(∞),  (1.32)

where G(s)_{sp} denotes the strictly proper part of G(s) and G(∞) = b_{m}.

Our goal is to find a realization of G(s), that is, a quadruple, (A, b, c, d) such that

G(s) = c(sI − A)^{−1}b + d.  (1.33)

[figure: \frac{1}{s^{n}+a_{n−1}s^{n−1}+···+a_{0}}; U(s); C(s); b_{m}s^{m} + ⋯ + b_{0}; Y (s)]

Figure 1.20: Decomposition of G(s) to construct its state-space realization.

Note that c(sI − A)^{−1}b is a strictly proper rational function and d is a scalar. Therefore, in our construction of a realization of G(s), we first extract the strictly proper part as in (1.32). We then find a triple (A, b, c) such that

G(s)_{sp} = c(sI − A)^{−1}b.  (1.34)

A realization of G(s) will have the form (A, b, c, G(∞)), that is, d = G(∞).

To proceed, we assume for simplicity that G(s) = G(s)_{sp}. We split our procedure of finding a triple (A, b, c) such that (1.34) holds into two steps. In the first step, we introduce an intermediate Laplace variable, C(s), such that

\frac{Y (s)}{U(s)} = \frac{C(s)}{U(s)} \frac{Y (s)}{C(s)}.

Thus

\frac{Y (s)}{U(s)} = \frac{1}{D(s)}N(s)
= \frac{1}{s^{n} + a_{n−1}s^{n−1} + ⋯ + a_{1}s + a_{0}} (b_{m}s^{m} + b_{m−1}s^{m−1} + ⋯ + b_{1}s + b_{0})
= \frac{C(s)}{U(s)} \frac{Y (s)}{C(s)}.  (1.35)

We illustrate the operation given by (1.35) in Figure 1.20. We first concern ourselves with the transfer function

\frac{C(s)}{U(s)} = \frac{1}{s^{n} + a_{n−1}s^{n−1} + ⋯ + a_{0}} .

Performing cross-multiplication gives

(s^{n} + a_{n−1}s^{n−1} + ⋯ + a_{0}) C(s) = U(s).

Taking the inverse Laplace transform and assuming zero initial conditions yields

c^{(n)} + a_{n−1}c^{(n−1)} + ⋯ + a_{1}ċ + a_{0}c = u.

We define the state variables:

x_{1} = c ; x_{2} = ċ = ẋ_{1} ; ⋮ ; x_{n−1} = c^{(n−2)} = ẋ_{n−2} ; x_{n} = c^{(n−1)} = ẋ_{n−1} }  (1.36)

Note that

ẋ_{n} = c^{(n)}
= −a_{0}c − a_{1}ċ − ⋯ − a_{n−1}c^{(n−1)} + u
= −a_{0}x_{1} − a_{1}x_{2} − ⋯ − a_{n−1}x_{n} + u.  (1.37)

We represent (1.36) and (1.37) in the matrix-vector format as

[ ẋ_{1} ; ẋ_{2} ; ⋮ ; ẋ_{n−1} ; ẋ_{n} ] = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & 1 & ⋯ & 0 & 0 \\ ⋮ & ⋮ & & ... & ⋮ & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & 1 \\ −a_{0} & −a_{1} & −a_{2} & ⋯ & −a_{n−2} & −a_{n−1} \end{matrix} ] [ x_{1} ; x_{2} ; ⋮ ; x_{n−1} ; x_{n} ] + [ 0 0 ⋮ 0 1 ] u.  (1.38)

The above set of n first-order ordinary differential equations can be simulated using a circuit shown in Figure 1.21.

We next concern ourselves with the transfer function

\frac{Y (s)}{C(s)} = b_{m}s^{m} + b_{m−1}s^{m−1} + ⋯ + b_{1}s + b_{0},

which we represent as

Y (s) = b_{m}s^{m}C(s) + b_{m−1}s^{m−1}C(s) + ⋯ + b_{1}sC(s) + b_{0}C(s).

Taking the inverse Laplace transform, while assuming zero initial conditions, we obtain

y = b_{m}c^{(m)} + b_{m−1}c^{(m−1)} + ⋯ + b_{1}ċ + b_{0}c  (1.39)

Note that

c = x_{1}
ċ = ẋ_{1} = x_{2}
⋮
c^{(m−1)} = ẋ_{m−1} = x_{m}
c^{(m)} = ẋ_{m} = x_{m+1}.

1.4. FROM TRANSFER FUNCTION TO STATE-SPACE DESCRIPTION 33

Figure 1.21: Implementation of (1.38).

Hence, we can represent y in terms of the state variables,

y = b_{m}x_{m+1} + b_{m−1}x_{m} + ⋯ + b_{1}x_{2} + b_{0}x_{1},

or, equivalently, as

y = [ b_{0} b_{1} ⋯ b_{m−1} b_{m} 0 ⋯ 0 ] [ x_{1} ; x_{2} ; ⋮ ; x_{n−1} ; x_{n} ] .  (1.40)

Combining (1.38) and (1.40), we obtain a realization of a strictly proper transfer function (1.31). This state-space realization has the form,

ẋ = [ \begin{matrix}0 & 1 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & ... & ⋮ & ⋮ \\ 0 & 0 & ⋯ & 0 & 1 \\ −a_{0} & −a_{1} & ⋯ & −a_{n−2} & −a_{n−1} \end{matrix} ] x + [ 0 0 ⋮ 0 1 ] u  (1.41)
y = [ b_{0} ⋯ b_{m} ⋯ 0 ] x.  (1.42)

We depict the above realization in Figure 1.22.

Figure 1.22: Implementation of (1.22).

Example 1.6 To find a realization of the transfer function

G(s) = \frac{4s^{3} − 2s^{2} + 3s + 1}{s^{3} + 3s^{2} − 5s + 7}  (1.43)

we represent it first as

G(s) = G(s)_{sp} + G(∞)
= \frac{−14s^{2} + 23s − 27}{s^{3} + 3s^{2} − 5s + 7} + 4.

Then, using the results of the above discussion, we obtain the following realization of the transfer function (1.43),

ẋ = [ \begin{matrix}0 & 1 & 0 \\ 0 & 0 & 1 \\ −7 & 5 & −3 \end{matrix} ] x + [ \begin{matrix}0 \\ 0 \\ 1 \end{matrix} ] u
y = [ −27 23 −14 ] x + 4u.

The above realization is only one of the infinitely many possible realizations of a given rational function G(s). We can easily obtain another realization of G(s). First note that G(s) can be viewed as an 1 × 1 matrix; therefore, its transpose equals itself, that is,

G(s)^{⊤} = G(s).

Applying the transposition operation to both sides of (1.33) and using the property of the transpose of the product of matrices, we obtain

G(s) = G(s)^{⊤}
= (c(sI − A)^{−1}b + d)^{⊤}
= b^{⊤} ((sI − A)^{−1})^{⊤} c^{⊤} + d^{⊤}
= b^{⊤} (sI − A^{⊤})^{−1} c^{⊤} + d
= c̃ (sI − Ã)^{−1} b̃ + d.

Thus, we have another realization of G(s) of the form

˙ ; x̃ = [ \begin{matrix}0 & 0 & ⋯ & 0 & −a_{0} \\ 1 & 0 & ⋯ & 0 & −a_{1} \\ ⋮ & & ... & ⋮ & ⋮ \\ 0 & 0 & ⋯ & 0 & −a_{n−2} \\ 0 & 0 & ⋯ & 1 & −a_{n−1} \end{matrix} ] x̃ + [ b_{0} ; b_{1} ; ⋮ ; b_{n−2} ; b_{n−1} ] u  (1.44)

y = [ 0 0 ⋯ 0 0 ⋯ 1 ] x̃ + G(∞).  (1.45)

We generalize the above discussion for multi-input multi-output transfer functions in the form of the following theorem.

Theorem 1.1 A transfer function G(s) is realizable if and only if G(s) is a proper rational matrix.

Proof (⟸) We first prove that for a given a rational proper transfer function G(s) ∈ ℝ^{p×m}(s) there is a realization. The proof is constructive. We begin by decomposing G(s) as

G(s) = G(s)_{sp} + G(∞),

where G(s)_{sp} denotes the strictly proper part of G(s). Let

d(s) = s^{r} + α_{r−1}s^{r−1} + ⋯ + α_{1}s + α_{0}

be the least common denominator of all rational function components of G(s)_{sp}. Note that the highest coefficient of d(s) is unity. We can always make α_{r} = 1 by dividing, if necessary, the corresponding numerator polynomials and d(s) by α_{r}. We then express G(s)_{sp} as

G(s)_{sp} = \frac{N(s)}{d(s)}
= \frac{N_{r−1}s^{r−1} + ⋯ + N_{1}s + N_{0}}{d(s)} ,

where each N_{i} is an p × m constant matrix.

We next show that the following state-space representation yields a realization of G(s):

\begin{matrix}ẋ & = & [ \begin{matrix}O & I_{m} & O & ⋯ & O & O \\ O & O & I_{m} & ⋯ & O & O \\ ⋮ & ⋮ & ⋮ & ... & ⋮ & ⋮ \\ O & O & O & ⋯ & O & I_{m} \\ −α_{0}I_{m} & −α_{1}I_{m} & −α_{2}I_{m} & ⋯ & −α_{r−2}I_{m} & −α_{r−1}I_{m} \end{matrix} ] x + [ O ; O ; ⋮ ; O ; I_{m} ] u \\ y & = & [ N_{0} N_{1} N_{2} ⋯ N_{r−2} N_{r−1} ] x + G(∞)u, \end{matrix} \}

(1.46)

where D = G(∞). Note that the matrix A ∈ ℝ^{rm×rm} in the above realization is a block matrix; its blocks are m × m matrices.

To show that the above state-space representation is indeed a realization of G(s), we define

Z = [ Z_{0} ; Z_{1} ; ⋮ ; Z_{r−1} ] = (sI − A)^{−1}B,  (1.47)

where Z_{i} is an m × m sub-matrix of the rm × m block matrix Z. Then the transfer function of (1.46) is

C(sI − A)^{−1}B + G(∞) = N_{0}Z_{0} + N_{1}Z_{1} + ⋯ + N_{r−1}Z_{r−1} + G(∞).  (1.48)

We pre-multiply both sides of (1.47) by (sI − A) and represent the result as

sZ = AZ + B.  (1.49)

Applying the above operation to the matrices A and B given in (1.46), we obtain

sZ_{0} = Z_{1}, sZ_{1} = Z_{2}, sZ_{r−2} = Z_{r−1},  (1.50)

and

sZ_{r−1} = −α_{0}Z_{0} − α_{1}Z_{1} − ⋯ − α_{r−1}Z_{r−1} + I_{m}.  (1.51)

Note that Z_{2} = sZ_{1} = s^{2}Z_{0} and in general,

Z_{i} = s^{i}Z_{0}, i = 1, …, r − 1  (1.52)

Substituting (1.52) into (1.51) and performing simple manipulations gives

(s^{r} + α_{r−1}s^{r−1} + ⋯ + α_{1}s + α_{0}) Z_{0} = I_{m},

that is,

Z_{0} = \frac{1}{d(s)}I_{m}.  (1.53)

Using (1.50) or (1.52) and the above yields

Z_{1} = \frac{s}{d(s)}I_{m} … Z_{r−1} = \frac{s^{r−1}}{d(s)}I_{m}.  (1.54)

Substituting (1.53) and (1.54) into (1.48), we obtain

C(sI − A)^{−1}B + G(∞) = \frac{1}{d(s)} (N_{0} + N_{1}s + ⋯ + N_{r−1}s^{r−1}) + G(∞)
= G(s),  (1.55)

which shows that (1.46) is a realization of G(s). The proof of this part of the theorem is complete. (⟹) If the transfer function G(s) is realizable then there exists a quadruple (A, B, C, D) such that

G(s) = C(sI − A)^{−1}B + D.

Note that C(sI − A)^{−1}B is a strictly proper rational matrix and because D is a constant matrix, C(sI − A)^{−1}B + D is a proper rational matrix, which completes the proof of this part of the theorem.

□

Example 1.7 We find a state-space realization of the transfer function matrix,

G(s) = [ −\frac{1}{s+2} \frac{1−s}{s^{2}+3s+2} \frac{1}{s^{2}+3s+2}^{\frac{1}{s(s+2)}  \frac{2s−1}{s(s+2)}  \frac{s−1}{s(s+2)}} ] .  (1.56)

The above transfer function is strictly proper; therefore

D = G(∞) = [ \begin{matrix}0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} ] .

We proceed to represent G(s) as

G(s) = \frac{1}{s^{3} + 3s^{2} + 2s} [ \begin{matrix}s + 1 & 2s^{2} + s − 1 & s^{2} − 1 \\ −s^{2} − s & −s^{2} + s & s \end{matrix} ]
= \frac{1}{s^{3} + 3s^{2} + 2s} ([ \begin{matrix}0 & 2 & 1 \\ −1 & −1 & 0 \end{matrix} ] s^{2} + [ \begin{matrix}1 & 1 & 0 \\ −1 & 1 & 1 \end{matrix} ] s + [ \begin{matrix}1 & −1 & −1 \\ 0 & 0 & 0 \end{matrix} ])
= \frac{1}{s^{3} + 3s^{2} + 2s} (N_{2}s^{2} + N_{1}s + N_{0}) .

38 CHAPTER 1. VOCABULARY AND DESCRIPTION OF DYNAMICAL SYSTEMS Using (1.46), we obtain

A = [ \begin{matrix}O & I_{3} & O \\ O & O & I_{3} \\ O & −2I_{3} & −3I_{3} \end{matrix} ]

= [ \begin{matrix}0 & 0 & 0 & ⋮ & 1 & 0 & 0 & ⋮ & 0 & 0 & 0 \\ 0 & 0 & 0 & ⋮ & 0 & 1 & 0 & ⋮ & 0 & 0 & 0 \\ 0 & 0 & 0 & ⋮ & 0 & 0 & 1 & ⋮ & 0 & 0 & 0 \\ ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & 0 & ⋮ & 0 & 0 & 0 & ⋮ & 1 & 0 & 0 \\ 0 & 0 & 0 & ⋮ & 0 & 0 & 0 & ⋮ & 0 & 1 & 0 \\ 0 & 0 & 0 & ⋮ & 0 & 0 & 0 & ⋮ & 0 & 0 & 1 \\ ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & 0 & ⋮ & −2 & 0 & 0 & ⋮ & −3 & 0 & 0 \\ 0 & 0 & 0 & ⋮ & 0 & −2 & 0 & ⋮ & 0 & −3 & 0 \\ 0 & 0 & 0 & ⋮ & 0 & 0 & −2 & ⋮ & 0 & 0 & −3 \end{matrix} ] .  (1.57)

The input matrix B is

B = [ \begin{matrix}O \\ O \\ I & 3 \end{matrix} ] = [ \begin{matrix}0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ ⋯ & ⋯ & ⋯ \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ ⋯ & ⋯ & ⋯ \\ 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} ] ,  (1.58)

and the matrix C has the form

C = [ N_{0} N_{1} N_{2} ] = [ \begin{matrix}1 & −1 & −1 & ⋮ & 1 & 1 & 0 & ⋮ & 0 & 2 & 1 \\ 0 & 0 & 0 & ⋮ & −1 & 1 & 1 & ⋮ & −1 & −1 & 0 \end{matrix} ] .  (1.59)

The dimension of this realization is 9.

We can obtain another realization of G(s) from (1.46). We begin by constructing a realization for G(s)^{⊤} using (1.46) to obtain

G(s)^{⊤} = C̃(sI − Ã)^{−1}B̃ + D̃.  (1.60)

We next transpose both sides of (1.60) to obtain

(G(s)^{⊤})^{⊤} = G(s)
= (C̃(sI − Ã)^{−1}B̃ + D̃)^{⊤}
= B̃^{⊤} (sI − Ã^{⊤})^{−1} C̃^{⊤} + D̃^{⊤}.

The resulting realization of G(s) has the form

\begin{matrix}˙ ; x̃ & = & [ \begin{matrix}O & O & O & ⋯ & O & −α_{0}I_{p} \\ I_{p} & O & I_{m} & ⋯ & O & −α_{1}I_{p} \\ ⋮ & ⋮ & ⋮ & ... & ⋮ & ⋮ \\ O & O & O & ⋯ & O & −α_{r−2}I_{p} \\ O & O & O & ⋯ & I_{p} & −α_{r−1}I_{p} \end{matrix} ] x̃ + [ N_{0} ; N_{1} ; ⋮ ; N_{r−2} ; N_{r−1} ] u \\ y & = & [ O O O ⋯ O I_{p} ] x̃ + G(∞)u. \end{matrix} \}  (1.61)

Note that although the realization (1.61) is based on the realization (1.46), their dimensions will be different if p ≠ m.

Example 1.8 We use the above method to construct a state-space realization of the transfer function matrix given by (1.56). We apply (1.46) to G(s)^{⊤}, where

G(s)^{⊤} = [ −\frac{1}{s+2} \frac{1−s}{s^{2}+3s+2} \frac{1}{s^{2}+3s+2}^{\frac{1}{s(s+2)}  \frac{2s−1}{s(s+2)}  \frac{s−1}{s(s+2)}} ]^{⊤}

= \frac{1}{s^{3} + 3s^{2} + 2s} [ \begin{matrix}s + 1 & −s^{2} − s \\ 2s^{2} + s − 1 & −s^{2} + s \\ s^{2} − 1 & s \end{matrix} ]

= \frac{1}{s^{3} + 3s^{2} + 2s} ([ \begin{matrix}0 & −1 \\ 2 & −1 \\ 1 & 0 \end{matrix} ] s^{2} + [ \begin{matrix}1 & −1 \\ 1 & 1 \\ 0 & 1 \end{matrix} ] s + [ \begin{matrix}1 & 0 \\ −1 & 0 \\ −1 & 0 \end{matrix} ])
= \frac{1}{s^{3} + 3s^{2} + 2s} (N_{2}^{⊤}s^{2} + N_{1}^{⊤}s + N_{0}^{⊤}) .

Applying to the above (1.61), we obtain another state-space realization of the transfer function G(s) given by (1.56), where

Â = [ \begin{matrix}O & O & O \\ I_{2} & O & −2I_{2} \\ O & I_{2} & −3I_{2} \end{matrix} ]

= [ \begin{matrix}0 & 0 & ⋮ & 0 & 0 & ⋮ & 0 & 0 \\ 0 & 0 & ⋮ & 0 & 0 & ⋮ & 0 & 0 \\ ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 1 & 0 & ⋮ & 0 & 0 & ⋮ & −2 & 0 \\ 0 & 1 & ⋮ & 0 & 0 & ⋮ & 0 & −2 \\ ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & ⋮ & 1 & 0 & ⋮ & −3 & 0 \\ 0 & 0 & ⋮ & 0 & 1 & ⋮ & 0 & −3 \end{matrix} ] .  (1.62)

The input matrix is

B̂ = [ \begin{matrix}N & 0 \\ N & 1 \\ N & 2 \end{matrix} ] = [ \begin{matrix}1 & −1 & −1 \\ 0 & 0 & 0 \\ ⋯ & ⋯ & ⋯ \\ 1 & 1 & 0 \\ −1 & 1 & 1 \\ ⋯ & ⋯ & ⋯ \\ 0 & 2 & 1 \\ −1 & −1 & 0 \end{matrix} ] ,  (1.63)

and the output matrix has the form

Ĉ = [ O O I_{2} ] = [ \begin{matrix}0 & 0 & ⋮ & 0 & 0 & ⋮ & 1 & 0 \\ 0 & 0 & ⋮ & 0 & 0 & ⋮ & 0 & 1 \end{matrix} ] .  (1.64)

The obtained realization is of lower dimension than the previous one. This does not mean that the alternative method always yields lower dimension realizations than the first method.

We present yet another method of constructing a state-space realization of a given transfer function matrix. This method is being used by the MATLAB’s function tf2ss. We begin by denoting the i-th column of a proper transfer function matrix G(s) as g_{i}(s), and observe that the input-output relation of an MIMO system can be expressed as

Y (s) = G(s)U(s)
= g_{1}(s)U_{1}(s) + g_{2}(s)U_{2}(s) + ⋯ + g_{m}(s)U_{m}(s),

where U_{i}(s) is the i-th component of the input vector U(s). We construct a state-space realization of each column g_{i}(s) and then combine these realizations to obtain the statespace realization of G(s) as follows. If the quadruple (A_{i}, b_{i}, C_{i}, d_{i}) is the realization of g_{i}(s), then a realization (A, B, C, D) of G(s) has the form,

\begin{matrix}A & = & [ \begin{matrix}A_{1} & O & ⋯ & O \\ O & A_{2} & ⋯ & O \\ ⋮ & ⋮ & ... & ⋮ \\ O & O & ⋯ & A_{m} \end{matrix} ] , B = [ \begin{matrix}b_{1} & 0 & ⋯ & 0 \\ 0 & b_{2} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & b_{m} \end{matrix} ] , \\ C & = & [ C_{1} C_{2} ⋯ C_{m} ] , and D = [ d_{1} d_{2} ⋯ d_{m} ] . \end{matrix} \}  (1.65)

We can also use the row-by-row approach to construct a state-space-realization of a given transfer function matrix—see Exercise 1.14.

Example 1.9 We use the column-by-column method to construct a realization of the transfer function matrix,

G(s) = [ \begin{matrix}\frac{1}{s+1} & \frac{2}{s+1} \\ \frac{−1}{s^{2}+3s+2} & \frac{1}{s+2} \end{matrix} ] = [ g_{1}(s) g_{2}(s) ] .  (1.66)

This transfer function matrix is strictly proper, therefore

D = [ \begin{matrix}0 & 0 \\ 0 & 0 \end{matrix} ] .

We apply (1.46) to

g_{1}(s) = [ \frac{1}{s+1} ; \frac{−1}{s^{2}+3s+2} ]
= \frac{1}{s^{2} + 3s + 2} [ \begin{matrix}s + 2 \\ −1 \end{matrix} ]
= \frac{1}{s^{2} + 3s + 2} ([ \begin{matrix}1 \\ 0 \end{matrix} ] s + [ \begin{matrix}2 \\ −1 \end{matrix} ])
= \frac{1}{s^{2} + 3s + 2} (N_{1}s + N_{0}) .

We obtain

A_{1} = [ \begin{matrix}0 & 1 \\ −2 & −3 \end{matrix} ] , b_{1} = [ \begin{matrix}0 \\ 1 \end{matrix} ] , C_{1} = [ \begin{matrix}2 & 1 \\ −1 & 0 \end{matrix} ] , d_{1} = [ \begin{matrix}0 \\ 0 \end{matrix} ] .

We next concern ourselves with the second column of G(s),

g_{2}(s) = [ \frac{2}{s+1} ; \frac{1}{s+2} ]
= \frac{1}{s^{2} + 3s + 2} [ \begin{matrix}2(s + 2) \\ s + 1 \end{matrix} ]
= \frac{1}{s^{2} + 3s + 2} ([ \begin{matrix}2 \\ 1 \end{matrix} ] s + [ \begin{matrix}4 \\ 1 \end{matrix} ])
= \frac{1}{s^{2} + 3s + 2} (N_{1}s + N_{0}) .

We apply (1.46) to obtain

A_{2} = [ \begin{matrix}0 & 1 \\ −2 & −3 \end{matrix} ] , b_{2} = [ \begin{matrix}0 \\ 1 \end{matrix} ] , C_{2} = [ \begin{matrix}4 & 2 \\ 1 & 1 \end{matrix} ] , d_{2} = [ \begin{matrix}0 \\ 0 \end{matrix} ] .

Combining the above realizations gives an overall realization of G(s) of the form,

A = [ \begin{matrix}A_{1} & O \\ O & A_{2} \end{matrix} ] = [ \begin{matrix}0 & 1 & ⋮ & 0 & 0 \\ −2 & −3 & ⋮ & 0 & 0 \\ ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & ⋮ & 0 & 1 \\ 0 & 0 & ⋮ & −2 & −3 \end{matrix} ] , B = [ \begin{matrix}b_{1} & 0 \\ 0 & b_{2} \end{matrix} ] = [ \begin{matrix}0 & ⋮ & 0 \\ 1 & ⋮ & 0 \\ ⋯ & ⋯ & ⋯ \\ 0 & ⋮ & 0 \\ 0 & ⋮ & 1 \end{matrix} ] ,

C = [ C_{1} C_{2} ] = [ \begin{matrix}2 & 1 & ⋮ & 4 & 2 \\ −1 & 0 & ⋮ & 1 & 1 \end{matrix} ] , and D = [ d_{1} d_{2} ] = [ \begin{matrix}0 & ⋮ & 0 \\ 0 & ⋮ & 0 \end{matrix} ] .

Example 1.10 We combine the method that generates the state-space realization given by (1.61) with the column-by-column method and apply it to the transfer function (1.66) from the previous example. We begin by transposing G(s) to obtain

G(s)^{⊤} = [ \begin{matrix}\frac{1}{s+1} & \frac{−1}{s^{2}+3s+2} \\ \frac{2}{s+1} & \frac{1}{s+2} \end{matrix} ] = [ g̃_{1}(s) g̃_{2}(s) ] .

The column g̃_{1}(s) can be represented as

g̃_{1}(s) = \frac{1}{s + 1} [ \begin{matrix}1 \\ 2 \end{matrix} ] .

Hence, we have one-dimensional realization of g̃_{1}(s),

Ã_{1} = [ −1 ] , b̃_{1} = [ 1 ] , C̃_{1} = [ \begin{matrix}1 \\ 2 \end{matrix} ] , nad d̃_{1} = [ \begin{matrix}0 \\ 0 \end{matrix} ] .  (1.67)

We then find a realization of g̃_{2}(s), which we first represent as

g̃_{2}(s) = [ \frac{−1}{s^{2}+3s+2} ; \frac{1}{s+2} ]
= \frac{1}{s^{2} + 3s + 2} [ \begin{matrix}−1 \\ s + 1 \end{matrix} ]
= \frac{1}{s^{2} + 3s + 2} ([ \begin{matrix}0 \\ 1 \end{matrix} ] s + [ \begin{matrix}−1 \\ 1 \end{matrix} ])
= \frac{1}{s^{2} + 3s + 2} (Ñ_{1}s + Ñ_{0}) .

We obtain a second-order realization of g̃_{2}(s), where

Ã_{2} = [ \begin{matrix}0 & 1 \\ −2 & −3 \end{matrix} ] , b̃_{2} = [ \begin{matrix}0 \\ 1 \end{matrix} ] , C̃_{2} = [ \begin{matrix}−1 & 0 \\ 1 & 1 \end{matrix} ] , and d̃_{2} = [ \begin{matrix}0 \\ 0 \end{matrix} ] .  (1.68)

Combining (1.67) and (1.68) and applying the transposition operation we obtain a thirdorder realization of (1.66),

Â = [ \begin{matrix}Ã_{1} & 0 \\ 0 & Ã_{2} \end{matrix} ]^{⊤} = [ \begin{matrix}−1 & 0 & 0 \\ 0 & 0 & −2 \\ 0 & 1 & −3 \end{matrix} ] , B̂ = [ C̃_{1} C̃_{2} ]^{⊤} = [ \begin{matrix}1 & 2 \\ −1 & 1 \\ 0 & 1 \end{matrix} ] ,

Ĉ = [ \begin{matrix}b̃_{1} & 0 \\ 0 & b̃_{2} \end{matrix} ]^{⊤} = [ \begin{matrix}1 & 0 & 0 \\ 0 & 0 & 1 \end{matrix} ] , D̂ = [ D̃_{1} D̃_{2} ]^{⊤} = [ \begin{matrix}0 & 0 \\ 0 & 0 \end{matrix} ] .

In the above examples, we obtained realizations of different dimensions for the same transfer function. Recall that the dimension of a realization is the size n of the matrix A. Natural question arises of how to find a minimal realization for a given transfer function matrix. To answer this question we need notions of controllability and observability. It turns out that a realization is minimal if and only if it is both reachable and observable. We present methods for constructing minimal realizations in Section 5.3.

1.5 System Zeros

Recall that the zeros of a transfer function of a single-input single-output (SISO) system are defined as the zeros of the numerator polynomial of this transfer function. Our objective is to extend this definition to multi-input multi-output (MIMO) systems. To proceed, suppose that we have a SISO dynamical system modeled as

ẋ = Ax + bu
y = cx.

Then, we have

G(s) = \frac{N(s)}{D(s)}
= c(sI_{n} − A)^{−1}b
= \frac{c adj(sI_{n} − A)b}{det(sI_{n} − A)}

= \frac{det [ \begin{matrix}sI_{n} − A & −b \\ c & 0 \end{matrix} ]}{det(sI_{n} − A)}.  (1.69)

To verify the above, we note that

[ \begin{matrix}I_{n} & 0 \\ −c(sI_{n} − A)^{−1} & 1 \end{matrix} ] [ \begin{matrix}sI_{n} − A & −b \\ c & 0 \end{matrix} ] = [ \begin{matrix}sI_{n} − A & −b \\ 0 & c(sI_{n} − A)^{−1}b \end{matrix} ] .

Then observe that

det ([ \begin{matrix}I_{n} & 0 \\ −c(sI_{n} − A)^{−1} & 1 \end{matrix} ] [ \begin{matrix}sI_{n} − A & −b \\ c & 0 \end{matrix} ]) = 1 × det [ \begin{matrix}sI_{n} − A & −b \\ c & 0 \end{matrix} ]
= det [ \begin{matrix}sI_{n} − A & −b \\ 0 & c(sI_{n} − A)^{−1}b \end{matrix} ] .

Hence

det [ \begin{matrix}sI_{n} − A & −b \\ c & 0 \end{matrix} ] = det [ \begin{matrix}sI_{n} − A & −b \\ 0 & c(sI_{n} − A)^{−1}b \end{matrix} ]
= det(sI_{n} − A) × c(sI_{n} − A)^{−1}b
= det(sI_{n} − A)\frac{c adj(sI_{n} − A)b}{det(sI_{n} − A)}
= c adj(sI_{n} − A)b,

which gives us (1.69).

We define the system zeros of a SISO dynamical system in terms of its state-space description as those roots of the polynomial equation

det [ \begin{matrix}sI_{n} − A & −b \\ c & 0 \end{matrix} ] = 0.

If

deg D(s) = deg (det[sI_{n} − A]) = n,

then the transfer function zeros and the system zeros are the same. When, on the other hand,

deg D(s) < deg (det[sI_{n} − A]) = n

then the transfer function zeros are a subset of the system zeros. We add that the transfer function zeros are also referred to as the transmission zeros.

We now analyze a MIMO dynamical system model

ẋ = Ax + Bu, x(0^{−}) = x_{0}
y = Cx.

Taking the Laplace transforms of the above equations gives

sX(s) − x(0^{−}) = AX(s) + BU(s)
Y (s) = CX(s).

We represent the above system of equations in the form

[ \begin{matrix}sI_{n} − A & −B \\ C & O \end{matrix} ] [ \begin{matrix}X(s) \\ Y (s) \end{matrix} ] = [ \begin{matrix}x(0^{−}) \\ Y (s) \end{matrix} ] .

We refer to the matrix

P (s) = [ \begin{matrix}sI_{n} − A & −B \\ C & O \end{matrix} ]

as the Rosenbrock’s system matrix. Note that P (s) is a polynomial matrix.

We wish to generalize the definition of system zeros to the MIMO case. For this we need the following definition.

Definition 1.5 The normal rank, denoted normal-rank, of a polynomial matrix is the maximally possible rank of this matrix for at least one s. When we say that a polynomial matrix is of rank r, we refer to its normal rank.

◇

Example 1.11 The rank of the polynomial matrix

M_{1}(s) = [ \begin{matrix}s + 1 & s^{2} − 1 \\ s & s(s − 1) \end{matrix} ]

is 1 because det M_{1}(s) = 0 for all s.

On the other hand the rank of the matrix

M_{2}(s) = [ \begin{matrix}s + 1 & s + 1 \\ s & 1 \end{matrix} ]

is 2 because for any s ≠ ±1 the rank of M_{2} is 2.

We now formally define system zeros of an MIMO LTI system.

Definition 1.6 A complex number z_{0} is a system zero, or an invariant zero, of the system (A, B, C, D) if

rank [ \begin{matrix}z_{0}I_{n} − A & −B \\ C & D \end{matrix} ] < normal-rank [ \begin{matrix}sI_{n} − A & −B \\ C & D \end{matrix} ] .

◇

Example 1.12 The system matrix of the system defined by the quadruple

A = [ \begin{matrix}0 & 1 \\ −3 & −2 \end{matrix} ] , b = [ \begin{matrix}0 \\ 1 \end{matrix} ] , c = [ 1 0 ] , and d = 0,

is

P (s) = [ \begin{matrix}sI_{n} − A & −b \\ c & d \end{matrix} ] = [ \begin{matrix}s & −1 & 0 \\ 3 & s + 2 & −1 \\ 1 & 0 & 0 \end{matrix} ]

Note that det P (s) = 1, which means that the above system has no system zeros. (Strictly speaking this system has no finite system zeros.)

1.6 Summary and Notes

A landmark paper on mathematical description of linear dynamical systems from control point of view is by Kalman [169]. The first chapter of Sontag’s book [310] is a comprehensive introduction to the subject of mathematical control theory.

Mayr writes on page 109 in [235], “It is still widely believed that the steam-engine governor is the oldest feedback device, and that James Watt had not only invented but also patented it. While both errors are easily refuted, we are still not able to reconstruct the history of this invention in all desired completeness.” Watt did not patent the governor. He did not invent it either. On page 112 of his book [235], Mayr adds the following: “But the application of the centrifugal pendulum in a system of speed regulation of steam engines was a new breakthrough for which the firm of Boulton & Watt, if not James Watt himself, clearly deserves the credit.”

1.7 Exercises

Exercise 1.1 Draw a block diagram illustrating the operation of the Watt’s governor shown in Figure 1.6.

Exercise 1.2 An example of a closed-loop feedback system is a toilet flushing device connected to a water tank. A toilet system is shown in Figure 1.23. The control objective is to maintain the level of water in the tank at a constant level. Draw a block diagram of the system.

Exercise 1.3 Consider a system in which its input u and output y are related as,

y(t) = L_{α,β}(u(t))
= \{ \begin{matrix}u(t) & for & t ≤ α \\ β & for & t > α, \end{matrix}  (1.70)

where α and β are design parameters. For what values of the design parameters the above system is linear?

Exercise 1.4 (based on Chen [55, p. 33]) Compute the transfer function of a discrete-time system whose impulse response is

g[k] = \{ \begin{matrix}0 & for & k ≤ 0 \\ \frac{1}{k} & for & k ≥ 1. \end{matrix}

Is this system lumped or distributed?

Figure 1.23: Toilet-flushing system for Exercise 1.2.

Exercise 1.5 Compute the impulse response of a dynamical system whose transfer function is given by

G(s) = [ \begin{matrix}\frac{s+1}{(s+2)^{2}} & \frac{1}{(2s+1)(s+2)} & \frac{3s−1}{s} \\ \frac{3}{s+2} & \frac{4s−10}{2s+1} & \frac{2s+1}{s^{2}+1} \end{matrix} ] .

Exercise 1.6 Can you construct a state-space model of a linear system whose output is zero in response to any input?

Exercise 1.7 For the circuit shown in Figure 1.24 draw a block diagram with the current i(t) as the system’s input and the voltage v(t) as the system’s output. Then, find the transfer function, V (s)/I(s).

Exercise 1.8 In the circuit shown in Figure 1.25, the input is the source current and the output is the voltage across the source terminals.

• Draw a block diagram of the circuit;

• Find the circuit transfer function;

• Construct a state-space realization;

Figure 1.24: Circuit for Exercise 1.7.

• Find the impulse response and determine if the circuit is bounded-input bounded output (BIBO) stable. If not, then find a bounded input, I, that produces an unbounded output, V.

Exercise 1.9 In the circuit shown in Figure 1.26, the input is the source voltage, v_{i} and the output is the voltage across the capacitor, v_{c}.

• Draw a block diagram of the circuit;

• Find the circuit transfer function;

• Construct a state-space realization;

• Find the impulse response and determine if the circuit is BIBO stable. If not, then find a bounded input that produces an unbounded output.

Exercise 1.10 Find the transfer function V_{out}(s)/V_{in}(s) of the circuit shown in Figure 1.27 that contains an ideal operational amplifier.

Exercise 1.11 The purpose of this Exercise is to show some benefits that the state-space description can offer in the analysis of control systems. Begin by verifying that the transfer

Figure 1.25: Circuit for Exercise 1.8.

function Y (s)/R(s) for both systems shown in Figure 1.28 and 1.29 is the same. Obtain the step response for this transfer function.

Next obtain state-space realizations of the systems shown in Figure 1.28 and 1.29. That is, construct state-space realizations of G_{c}(s) = \frac{s−1}{s+2} and G_{p}(s) = \frac{3}{s−1} and then construct the overall state-space model of the cascade systems. The state-space models should of of second-order. Verify that the transfer functions of the models are the same as in the part above. Implement the obtained state-space models in SIMULINK. Obtain plots of the states and the outputs in response to a unit step input. Perform your simulations for different initial conditions that you can impose on the integrators. Compare the external, that is, input-output system behavior against the internal behavior. Can you give an explanation for an unexpected behavior of the state variables even when the external behavior seems to be acceptable.

Exercise 1.12 Suppose that G(s) ∈ ℝ^{p×m} is a proper rational matrix such that

G(s) = G(s)_{sp} + G(∞),

where

G(s)_{sp} = \frac{N_{r−1}s^{r−1} + ⋯ + N_{1}s + N_{0}}{d(s)}

1.7. EXERCISES 51

Figure 1.26: Circuit for Exercise 1.9.

and d(s) = s^{r} +α_{r−1}s^{r−1} +⋯+α_{1}s+α_{0}. Show that the following state-space representation is a realization of G(s):

\begin{matrix}˙ ; x̂ & = & [ \begin{matrix}−α_{r−1}I_{m} & −α_{r−2}I_{m} & −α_{r−3}I_{m} & ⋯ & −α_{1}I_{m} & −α_{0}I_{m} \\ I_{m} & O & O & ⋯ & O & O \\ ⋮ & ⋮ & ⋮ & ... & ⋮ & ⋮ \\ O & O & O & ⋯ & O & O \\ O & O & O & ⋯ & I_{m} & O \end{matrix} ] x̂ + [ I_{m} ; O ; ⋮ ; O ; O ] u \\ y & = & [ N_{r−1} N_{r−2} N_{r−3} ⋯ N_{1} N_{0} ] x̂ + G(∞)u, \end{matrix} \}

(1.71)

Exercise 1.13 For a proper rational matrix

G(s) = [ \begin{matrix}\frac{s+1}{(s+2)^{2}} & \frac{1}{(2s+1)(s+2)} & \frac{3s−1}{s} \\ \frac{3}{s+2} & \frac{4s−10}{2s+1} & \frac{2s+1}{s^{2}+1} \end{matrix} ] ,  (1.72)

(i) construct its two realizations; (ii) use MATLAB’s tf2ss command to find a realization of G(s).

Figure 1.27: Circuit for Exercise 1.10.

[figure: \frac{s−1}{s+2}; r; u; \frac{3}{s−1}; y]

Figure 1.28: Cascade compensation for Exercise 1.11: compensator preceding the plant.

Exercise 1.14 Let g_{i}(s)^{⊤} denote an i-th row of a given proper transfer function matrix G(s). Thus, we have

Y_{i}(s) = g_{i}(s)^{⊤}U(s), i = 1, 2 …, p

where Y_{i}(s) is the i-th component of the output vector Y (s). Suppose that a realization of g_{i}(s)^{⊤} has the form (A_{i}, B_{i}, c_{i}, d_{i}), where i = 1, 2, …, p. What are the matrices A, B, C, and D of a realization of G(s) in terms of (A_{i}, B_{i}, c_{i}, d_{i})? Test this realization algorithm on the transfer function given by (1.56).

[figure: \frac{3}{s−1}; r; \frac{s−1}{s+2}; y]

Figure 1.29: Compensator following the plant.

54 CHAPTER 1. VOCABULARY AND DESCRIPTION OF DYNAMICAL SYSTEMS

Dynamical System Modeling

To catch a mouse alive, try this trick: When you see the beast running around a room, open a black umbrella and hold it on the floor at an angle that allows the mouse to run in (it helps if someone else can chase the mouse). Your prey, drawn to dark places, will probably head straight for the umbrella. Then quickly close it, go outside, and set your prisoner free. [281, page 135]

In this chapter, we discuss the Lagrange equations of motion that are useful in mathematical modeling of dynamic systems. We first review relevant concepts that are needed to derive the Lagrange equations. We then construct mathematical models of selected dynamical systems.

2.1 Mathematical Modeling

Mathematical models of systems that we wish to control are key elements in their analysis and controller design. We devote this section 2.1 to mathematical modeling. Constructing models using linguistic description of the process to be controlled will be discussed in Subsection 15.4.1.

The goal of mathematical modeling is to provide a mathematical description of the interrelations between the system quantities as well as the relations between these quantities and external inputs. A mathematical model can be viewed as “a mathematical representation of the significant, relevant aspects of an existing or proposed physical system. (Significance and relevance being in relation to an application where the model is to be used).” [208, page xi]. Another definition of mathematical model that we found in [97, page 1] is: “A mathematical model is a system of equations that purports to represent some phenomenon in a way that gives insight into the origins and consequences of its behavior.”

At this point, we wish to point out that it is important to distinguish between a physical system and its model. A physical system may be modeled using different models. In particular, it may be advantageous to model the physical system using different models for its different operating regions. For example, it is common to model an electric circuit using different models for different ranges of frequencies in which the circuit operates. In what follows, we refer to the physical system’s model as the system.

In general, the more accurate the mathematical model is, the more complex are the modeling equations. This means that during the mathematical modeling process we may be forced to find a middle ground between accuracy of the model and its complexity. Our goal is to generate the most accurate possible description of the given system while still being able to carry out the model analysis and computations in a reasonable amount of time. At first, we tend to make oversimplifying assumptions in order to obtain a very simple model. However, we should follow the principle of maximum simplicity, which states that ”everything should be as simple as possible—but no simpler.” After extracting some information about the system behavior based on this first, possibly crude model, it may be desirable to construct a more accurate model. We try to create a mathematical description of a given physical process that incorporates realistic assumptions and physical constraints. We then extract information from this model and compare it with the experimental data. We perform this comparative study to determine the value of the obtained model. If the model appears to be inadequate for our needs, then it must be adjusted. The process of mathematical modeling is summarized in the diagram in Figure 2.1. In the process of mathematical modeling, one can use different types of equations and their combinations like algebraic, ordinary and partial differential equations, difference, integral, or functional equations. In this book, only difference and ordinary differential equations are used to model dynamical systems.

For the controller design purposes, we first construct a truth model of the given plant and then its design model. The truth model is the simulation model that should include all the relevant characteristics of the physical system to be controlled. The control designs are being simulated using the truth model. A design model is developed by eliminating “all unpleasant nonlinear effects while capturing all the essential features of the process, is much more amenable to use for the design of a control system” [108, p. 26].

Figure 2.1: Diagram of mathematical modeling process and its use in the controller design process.

2.2 Review of Work and Energy Concepts

The purpose of this section is to review and reacquaint ourselves with some basic notions from physics like work, energy, etc. After a brief review of work and energy concepts, we derive the Lagrange equations of motion and use them to devise mathematical models of dynamic systems.

Suppose we are given a particle of a constant mass m subjected to a force F. Then, by Newton’s second law we have

F = m\frac{dv}{dt} = ma.  (2.1)

Here the force F and the velocity v are vectors. Therefore, they can be represented as

F = [ F_{x_{1}} F_{x_{2}} F_{x_{3}} ]^{⊤} and v = [ v_{x_{1}} v_{x_{2}} v_{x_{3}} ]^{⊤} = [ ẋ_{1} ẋ_{2} ẋ_{3} ]^{⊤} .

Using the above notation, we represent equation (2.1) as

[ \begin{matrix}F_{x_{1}} \\ F_{x_{2}} \\ F_{x_{3}} \end{matrix} ] = m [ \frac{dv_{x_{1}}}{dt} ; \frac{dv_{x_{2}}}{dt} ; \frac{dv_{x_{3}}}{dt} ] .  (2.2)

Suppose now that a force F is acting on a particle located at a point A and the particle moves to a point B. The work δW done by F along infinitesimally small distance δs is

δW = F^{⊤}δs
= [ F_{x_{1}} F_{x_{2}} F_{x_{3}} ] [ \begin{matrix}δx_{1} \\ δx_{2} \\ δx_{3} \end{matrix} ]
= F_{x_{1}}δx_{1} + F_{x_{2}}δx_{2} + F_{x_{3}}δx_{3}.

where δs = [ δx_{1} δx_{2} δx_{3} ]^{⊤}. The work W_{AB} done on the path from A to B is obtained by integrating the above equation,

W_{AB} = ∫_{A}^{B} (F_{x_{1}}dx_{1} + F_{x_{2}}dx_{2} + F_{x_{3}}dx_{3}) .

We now would like to establish a relation between work and kinetic energy. For this observe that

ẍ dx = \frac{dẋ}{dt}dx = ẋdẋ.

Using the above relation and Newton’s second law, we express W_{AB} in a different way as

W_{AB} = ∫_{A}^{B} (mẍ_{1}dx_{1} + mẍ_{2}dx_{2} + mẍ_{3}dx_{3})
= ∫_{A}^{B} m (ẋ_{1}dẋ_{1} + ẋ_{2}dẋ_{2} + ẋ_{3}dẋ_{3})
= m (\frac{ẋ_{1}^{2}}{2} + \frac{ẋ_{2}^{2}}{2} + \frac{ẋ_{3}^{2}}{2} )|_{A}^{B}
= m (\frac{∥v_{B}∥^{2}}{2} − \frac{∥v_{A}∥^{2}}{2} )
= \frac{m∥v_{B}∥^{2}}{2} − \frac{m∥v_{A}∥^{2}}{2} .  (2.3)

where v_{A} = [ ẋ_{1A} ẋ_{2A} ẋ_{3A} ]^{⊤}, and v_{B} is defined in an analogous manner, and ∥v∥^{2} = v^{⊤}v. Observing that \frac{m∥v_{B}∥^{2}}{2} is the kinetic energy of the particle at the point B and \frac{m∥v_{A}∥^{2}}{2} is its kinetic energy at the point A, we obtain

W_{AB} = K_{B} − K_{A}.

The above relation can be used to define the kinetic energy of a particle as the work required to change its velocity from some value v_{A} to a final value v_{B}. The relation, W_{AB} = K_{B} − K_{A} = ΔK is also known as the work-energy theorem for a particle. A force is conservative if the work done by the force on a particle that moves through any round trip is zero. In other words, a force is conservative if the work done by it on a particle that moves between two points depends only on these points and not on the path followed.

We next review the notion of potential energy or what is the same, the energy of configuration. Recall that if the kinetic energy K of a particle changes by ΔK, then the potential energy U must change by an equal but opposite amount so that the sum of the two changes is zero, that is,

ΔK + ΔU = 0.

This is equivalent to saying that any change in the kinetic energy of the particle is compensated for by an equal but opposite change in the potential energy U of the particle so that their sum remains constant, that is,

K + U = constant.

See Exercise 2.1 for a rigorous proof of the above relation. The potential energy of a particle represents a form of stored energy that can be recovered and converted into the kinetic energy. If we now use the work-energy theorem and the equation ΔK + ΔU = 0, then we obtain

W = ΔK = −ΔU.

The work done by a conservative force depends only on the starting and the end points of motion and not on the path followed between them. Therefore, for motion in one dimension,

ΔU = −W = − ∫_{x_{0}}^{x} F (s)ds.  (2.4)

We thus can write

F (x) = −\frac{dU(x)}{dx}.  (2.5)

A generalization of equation (2.4) to motion in three dimensions yields

ΔU = − ∫_{x_{10}}^{x_{1}} F_{x_{1}}ds − ∫_{x_{20}}^{x_{2}} F_{x_{2}}ds − ∫_{x_{30}}^{x_{3}} F_{x_{3}}ds,

and a generalization of equation (2.5) to motion in three dimensions is

F (x) = − [ \frac{∂U}{∂x_{1}}  \frac{∂U}{∂x_{2}}  \frac{∂U}{∂x_{3}} ]^{⊤} = −∇U (x) ,  (2.6)

where x = [ x_{1} x_{2} x_{3} ]^{⊤}.

For a formal proof of the property of a conservative vector field that the work done by it on a particle that moves between two points depends only on these points and not on the path followed, see Exercise 2.2.

We next represent Newton’s equation in an equivalent format that establishes a connection with the Lagrange equations of motion to be discussed in the following section. To proceed, note that

K = \frac{m \left∥{ \atop ẋ}\right∥^{2}}{2} = \frac{mẋ^{⊤}ẋ}{2}.  (2.7)

Hence,

\frac{∂K}{∂ẋ_{i}} = mẋ_{i}, i = 1, 2, 3.  (2.8)

Combining (2.6) and (2.8), we represent Newton’s equations as

\frac{d}{dt} (\frac{∂K}{∂ẋ_{i}} ) + (∇U)_{i} = 0, i = 1, 2, 3.  (2.9)

Let

L = K − U.  (2.10)

The function L defined above is called the Lagrangian function or just the Lagrangian. Note that

\frac{∂L}{∂ẋ_{i}} = \frac{∂K}{∂ẋ_{i}} ,  (2.11)

and

\frac{∂L}{∂x_{i}} = −\frac{∂U}{∂x_{i}} ,  (2.12)

Substituting (2.11) and (2.12) into (2.9), we obtain

\frac{d}{dt} (\frac{∂L}{∂ẋ_{i}} ) − \frac{∂L}{∂x_{i}} = 0, i = 1, 2, 3.  (2.13)

Equations (2.13) are called the Lagrange equations of motion, in Cartesian coordinates, for a single particle. They are just an equivalent representation of Newton’s equations given by (2.2).

2.3 The Lagrange Equations of Motion in Generalized Coordinates

2.3. THE LAGRANGE EQUATIONS OF MOTION IN GENERALIZED COORDINATES61

A single particle with no constraints is free to take up any position in the three dimensional space. This is the reason we need three coordinates to describe its position. In the case of a system of p particles there are 3p degrees of freedom. In general, if there are constraints imposed on the particles, then the number of the degrees of freedom is

n = 3p − number of constraints.

For example, if the system consists of only one particle in space and if the particle is restricted to move along a line, then n = 3 − 2 = 1. We can use many different coordinate systems to describe a location of a particle in space. We use the letter q as a symbol for a coordinate regardless of its nature. A generalized coordinate may represent, for example, angular displacement or electric charge. This indicates that the results we are going to obtain are of general nature and are not be restricted to mechanical systems. We refer to q as a generalized coordinate. Thus, if

\begin{matrix}x & = & x(q_{1}, q_{2}, q_{3}) \\ y & = & y(q_{1}, q_{2}, q_{3}) \\ z & = & z(q_{1}, q_{2}, q_{3}) \end{matrix} \}

then

ẋ = \frac{∂x}{∂q_{1}} q̇_{1} + \frac{∂x}{∂q_{2}} q̇_{2} + \frac{∂x}{∂q_{3}} q̇_{3},

and so on. We assume that the functions x, y, z are twice continuously differentiable, that is, x, y, z ∈ C^{2}, which implies that their corresponding mixed second partial derivatives are equal.

Suppose now that a particle is confined to move on a surface. Therefore, to describe the location of the particle, we need only two generalized coordinates. The position of the particle on a surface in a three-dimensional space is then described as

x = x(q_{1}, q_{2}),
y = y(q_{1}, q_{2}),
z = z(q_{1}, q_{2}),

and hence

\begin{matrix}δx & = & \frac{∂x}{∂q_{1}} δq_{1} + \frac{∂x}{∂q_{2}} δq_{2} \\ δy & = & \frac{∂y}{∂q_{1}} δq_{1} + \frac{∂y}{∂q_{2}} δq_{2} \\ δz & = & \frac{∂z}{∂q_{1}} δq_{1} + \frac{∂z}{∂q_{2}} δq_{2}. \end{matrix} \}  (2.14)

We know that the work δW done by the force F is δW = F_{x}δx + F_{y}δy + F_{z}δz. On the other hand, this work is equal to the change in the kinetic energy. Therefore,

m (ẍδx + ÿδy + z̈δz) = F_{x}δx + F_{y}δy + F_{z}δz  (2.15)

The above equation is known as D’Alembert’s equation.

If we have a system of particles acted upon by forces F_{1}, …, F_{p} and if each of the particles undergoes displacements δs_{1}, …, δs_{p}, then the total work done by the forces is

δW = ∑_{i=1}^{p} (F_{x_{i}}δx_{i} + F_{y_{i}}δy_{i} + F_{z_{i}}δz_{i}) .  (2.16)

Substituting (2.14) into D’Alembert’s equation and rearranging, we obtain

δW = m (ẍ\frac{∂x}{∂q_{1}} + ÿ\frac{∂y}{∂q_{1}} + z̈\frac{∂z}{∂q_{1}} ) δq_{1} + m (ẍ\frac{∂x}{∂q_{2}} + ÿ\frac{∂y}{∂q_{2}} + z̈\frac{∂z}{∂q_{2}} ) δq_{2}
= (F_{x} \frac{∂x}{∂q_{1}} + F_{y} \frac{∂y}{∂q_{1}} + F_{z} \frac{∂z}{∂q_{1}} ) δq_{1} + (F_{x} \frac{∂x}{∂q_{2}} + F_{y} \frac{∂y}{∂q_{2}} + F_{z} \frac{∂z}{∂q_{2}} ) δq_{2}. (2.17)

Let

F_{q_{1}} = F_{x} \frac{∂x}{∂q_{1}} + F_{y} \frac{∂y}{∂q_{1}} + F_{z} \frac{∂z}{∂q_{1}} ,

and

F_{q_{2}} = F_{x} \frac{∂x}{∂q_{2}} + F_{y} \frac{∂y}{∂q_{2}} + F_{z} \frac{∂z}{∂q_{2}} .

Then, (2.17) can be represented as,

δW = F_{q_{1}}δq_{1} + F_{q_{2}}δq_{2}.

To proceed further, note that

\frac{d}{dt} (ẋ\frac{∂x}{∂q_{1}} ) = ẍ\frac{∂x}{∂q_{1}} + ẋ\frac{d}{dt} (\frac{∂x}{∂q_{1}} ) .

Hence

ẍ\frac{∂x}{∂q_{1}} = \frac{d}{dt} (ẋ\frac{∂x}{∂q_{1}} ) − ẋ\frac{d}{dt} (\frac{∂x}{∂q_{1}} )  (2.18)

The time derivative of x = x(q_{1}, q_{2}) is

ẋ = \frac{∂x}{∂q_{1}} q̇_{1} + \frac{∂x}{∂q_{2}} q̇_{2}.  (2.19)

Differentiating the above partially with respect to q̇_{1} yields

\frac{∂ẋ}{∂q̇_{1}} = \frac{∂x}{∂q_{1}}  (2.20)

Finally, we show that

\frac{d}{dt} (\frac{∂x}{∂q_{1}} ) = \frac{∂ẋ}{∂q_{1}}  (2.21)

Indeed, because x = x(q_{1}, q_{2}) and the partial derivative ∂x/∂q_{1} is in general a function of q_{1} and q_{2}, we have

\frac{d}{dt} (\frac{∂x}{∂q_{1}} ) = \frac{∂}{∂q_{1}} (\frac{∂x}{∂q_{1}} ) q̇_{1} + \frac{∂}{∂q_{2}} (\frac{∂x}{∂q_{1}} ) q̇_{2}.  (2.22)

Taking the partial derivative of (2.19) with respect to q_{1}, we get

\frac{∂ẋ}{∂q_{1}} = \frac{∂}{∂q_{1}} (\frac{∂x}{∂q_{1}} ) q̇_{1} + \frac{∂}{∂q_{1}} (\frac{∂x}{∂q_{2}} ) q̇_{2}
= \frac{∂}{∂q_{1}} (\frac{∂x}{∂q_{1}} ) q̇_{1} + \frac{∂}{∂q_{2}} (\frac{∂x}{∂q_{1}} ) q̇_{2},  (2.23)

because by assumption x, y, z ∈ C^{2} and hence

\frac{∂}{∂q_{1}} (\frac{∂x}{∂q_{2}} ) = \frac{∂}{∂q_{2}} (\frac{∂x}{∂q_{1}} ) .

With the above in mind, comparing the right hand sides of (2.22) and (2.23) yields (2.21). We now use (2.18), (2.20), and (2.21) to obtain the Lagrange equation of motion for a single particle. We first substitute (2.20) and (2.21) into (2.18) to get

ẍ\frac{∂x}{∂q_{1}} = \frac{d}{dt} (ẋ\frac{∂ẋ}{∂q̇_{1}} ) − ẋ\frac{∂ẋ}{∂q_{1}} .  (2.24)

Note that

\frac{∂}{∂q̇_{1}} (\frac{ẋ^{2}}{2} ) = ẋ\frac{∂ẋ}{∂q̇_{1}} ,  (2.25)

and

\frac{∂}{∂q_{1}} (\frac{ẋ^{2}}{2} ) = ẋ\frac{∂ẋ}{∂q_{1}} .  (2.26)

Substituting (2.25) and (2.26) into (2.24) gives

ẍ\frac{∂x}{∂q_{1}} = \frac{d}{dt} (\frac{∂ (\frac{ẋ^{2}}{2} )}{∂q̇_{1}} ) − \frac{∂ (\frac{ẋ^{2}}{2} )}{∂q_{1}}.

We can obtain similar expressions for y and z. Taking this into account, the expression (2.17) when δq_{2} = 0 becomes,

δW = (\frac{d}{dt} (\frac{∂}{∂q̇_{1}} \frac{m (ẋ^{2} + ẏ^{2} + ż^{2})}{2}) − \frac{∂}{∂q_{1}} \frac{m (ẋ^{2} + ẏ^{2} + ż^{2})}{2}) δq_{1}
= (F_{x} \frac{∂x}{∂q_{1}} + F_{y} \frac{∂y}{∂q_{1}} + F_{z} \frac{∂z}{∂q_{1}} ) δq_{1}
= F_{q_{1}}δq_{1}.  (2.27)

Let K = \frac{1}{2}m (ẋ^{2} + ẏ^{2} + ż^{2}) denote the kinetic energy of the particle. Then, we represent (2.27) as

\frac{d}{dt} (\frac{∂K}{∂q̇_{1}} ) − \frac{∂K}{∂q_{1}} = F_{q_{1}}  (2.28)

This equation is called the Lagrange equation of motion for the q_{1} coordinate. Using the same arguments as above, we can derive the Lagrange equation of motion for the q_{2} coordinate. In general there are as many the Lagrange equations of motion as there are degrees of freedom of the particle.

Recall that the expression δW = F_{x}δx + F_{y}δy + F_{z}δz is a work done by a force F = [ F_{x} F_{y} F_{z} ]^{⊤} for a general displacement δs = [ δx δy δz ]^{⊤} . In terms of generalized coordinates and when, say, δq_{2} = 0, we have

δW = (F_{x} \frac{∂x}{∂q_{1}} + F_{y} \frac{∂y}{∂q_{1}} + F_{z} \frac{∂z}{∂q_{1}} ) δq_{1}
= F_{q_{1}}δq_{1}
= δW_{q_{1}}.

A generalized force F_{q_{r}} is of such nature that the product F_{q_{r}}δq_{r} is the work done by driving forces when q_{r} alone is changed by δq_{r}. We mention that a generalized force does not have to be a force in the usual sense. For example, if q_{r} is an angle, then F_{q_{r}} must be a torque in order that F_{q_{r}}δq_{r} be work.

We now derive a more general version of the Lagrange equations of motion for a single particle. In our derivation we need the notion of nonconservative forces that we discuss next. First recall that we interpret the kinetic energy of a particle as its ability to do work by virtue of its motion. This motion is the result of conservative and nonconservative forces acting upon the particle. A force is nonconservative if the work done by the force on a particle that moves through any round trip is not zero. Thus a force is nonconservative if the work done by that force on a particle that moves between two points depends on the path taken between these points.

Let us now suppose that in addition to the conservative forces, a single nonconservative force due to friction acts on the particle. Denote by ∑ W_{c} the sum of the work done by the conservative forces, and by W_{f} the work done by friction. We then have

W_{f} + ∑ W_{c} = ΔK.  (2.29)

Thus, if a nonconservative force, like friction, acts on a particle, then the total mechanical energy is not constant, but changes by the amount of work done by a nonconservative force according to the relation (2.29). Let us denote by E the final mechanical energy of the particle and by E_{0} its initial mechanical energy. Then, we can write

E − E_{0} = ΔE = W_{f}.

The work done by friction, the nonconservative force, is always negative. Hence, the final mechanical energy E is less than the initial mechanical energy. The “lost” mechanical energy is transformed into internal energy U_{int}. In other words, the loss of mechanical energy equals the gain in internal energy and we write

ΔE + U_{int} = 0.

In general, we have

ΔK + ∑ ΔU + U_{int} + (change in other forms of energy) = 0.

This means that the total energy, kinetic plus potential plus internal plus all other forms of energy, does not change. This is the principle of the conservation of energy. To proceed further, recall that any generalized force F_{q_{i}} acting on our particle can be expressed as

F_{q_{i}} = F_{x} \frac{∂x}{∂q_{i}} + F_{y} \frac{∂y}{∂q_{i}} + F_{z} \frac{∂z}{∂q_{i}} .

Assuming that the forces are conservative, we get

F_{q_{i}} = − (\frac{∂U}{∂x} \frac{∂x}{∂q_{i}} + \frac{∂U}{∂y} \frac{∂y}{∂q_{i}} + \frac{∂U}{∂z} \frac{∂z}{∂q_{i}} ) .

We thus have

F_{q_{i}} = −\frac{∂U}{∂q_{i}} .

Therefore, the Lagrange equations of motion for a single particle for conservative forces can be represented as

\frac{d}{dt} (\frac{∂K}{∂q̇_{i}} ) − \frac{∂K}{∂q_{i}} = −\frac{∂U}{∂q_{i}} .

We can rearrange the above equations as

\frac{d}{dt} (\frac{∂K}{∂q̇_{i}} ) − \frac{∂}{∂q_{i}} (K − U) = 0.

We now introduce the Lagrangian function L, defined as

L = K − U.

Figure 2.2: A simple pendulum.

Note that because U is not a function of q̇_{i}, we have

\frac{∂K}{∂q̇_{i}} = \frac{∂L}{∂q̇_{i}}.

Taking the above into account, we can represent the Lagrange equations of motion for conservative forces as

\frac{d}{dt} (\frac{∂L}{∂q̇_{i}}) − \frac{∂L}{∂q_{i}} = 0.

If some of the forces acting on the particle are nonconservative, the Lagrange equations of motion for our particle will take the form

\frac{d}{dt} (\frac{∂L}{∂q̇_{i}}) − \frac{∂L}{∂q_{i}} = F̃_{q_{i}}  (2.30)

where F̃_{q_{i}} are nonconservative forces acting on the particle.

Example 2.1 Consider the simple pendulum shown in Figure 2.2. The simple pendulum is an idealized body consisting of a point mass M, suspended by a weightless inextensible cord of length l. The simple pendulum is an example of a one degree of freedom system with a generalized coordinate being the angular displacement θ. The pendulum kinetic energy is

K = \frac{1}{2}Ml^{2}θ̇^{2}.

We assume that the potential energy of the pendulum is zero when the pendulum is at rest. Hence, its potential energy is

U = Mgl(1 − cos θ),

where g = 10 m/sec^{2} is the acceleration due to gravity. The Lagrangian function is

L = K − U
= \frac{1}{2}Ml^{2}θ̇^{2} − Mgl(1 − cos θ).

We compute

\frac{∂L}{∂θ} = −Mgl sin θ

and

\frac{∂L}{∂θ̇} = Ml^{2}θ̇.

Hence

\frac{d}{dt} (\frac{∂L}{∂θ̇}) = Ml^{2}θ̈.

Combining the above terms together we obtain the Lagrange equation describing the pendulum motion,

\frac{d}{dt} (\frac{∂L}{∂θ̇}) − \frac{∂L}{∂θ} = Ml^{2}θ̈ + Mgl sin θ
= 0.

Solving for θ̈, we obtain

θ̈ = −\frac{g}{l} sin θ.  (2.31)

In our example the above equation evaluates to

θ̈ = −5 sin θ.  (2.32)

Example 2.2 In Figure 2.3, the θ-r robot manipulator is shown. This example was adapted from Snyder [306, Chapter 10]. A lumped mass representation of the θ-r robot manipulator is shown in Figure 2.4. The mass m_{1} = 10 kg represents the mass of the outer cylinder positioned at its center of mass. The constant distance r_{1} = 1m designates the fixed distance between the center of the mass of the outer cylinder and the center of rotation. The mass of the load is represented by m_{2} = 3kg and is assumed to be located at the end of a piston of a telescoping arm that is a variable radial distance r from the hub or center of rotation. The angle of rotation of the manipulator arm is denoted by θ. The inputs to the system are assumed to be a torque T_{θ} applied at the hub in the direction of θ, and a translational force F_{r} applied in the direction r. Finally, g = 10 m/s^{2} is the gravitational constant. We 68 CHAPTER 2. DYNAMICAL SYSTEM MODELING

Figure 2.3: The θ-r robot manipulator.

Figure 2.4: A lumped mass representation of the θ-r robot manipulator. construct the Lagrange equations of motion for the system, and then represent the obtained model in state space format. We first find the total kinetic energy of the system. The position of the mass m_{1} is

\begin{matrix}x_{1} & = & r_{1} cos θ \\ y_{1} & = & r_{1} sin θ. \end{matrix} \}

Note that r_{1} is a constant. Differentiating x_{1} and y_{1} with respect to time yields

\begin{matrix}ẋ_{1} & = & −r_{1}θ̇ sin θ \\ ẏ_{1} & = & r_{1}θ̇ cos θ. \end{matrix} \}

The magnitude squared of the velocity vector of the mass m_{1} is

v_{1}^{2} = ẋ_{1}^{2} + ẏ_{1}^{2}
= r_{1}^{2}θ̇^{2} sin^{2} θ + r_{1}^{2}θ̇^{2} cos^{2} θ
= r_{1}^{2}θ̇^{2}.

Therefore, the kinetic energy of the mass m_{1} is

K_{1} = \frac{1}{2}m_{1}v_{1}^{2} = \frac{1}{2}m_{1}r_{1}^{2}θ̇^{2}.

We now derive an expression for the kinetic energy of the second mass. The position of the mass m_{2} is

\begin{matrix}x_{2} & = & r cos θ \\ y_{2} & = & r sin θ. \end{matrix} \}

Note that the distance r is not a constant. Differentiating x_{2} and y_{2} with respect to time yields

\begin{matrix}ẋ_{2} & = & ṙ cos θ − rθ̇ sin θ \\ ẏ_{2} & = & ṙ sin θ + rθ̇ cos θ. \end{matrix} \}

The magnitude squared of the velocity vector of the mass m_{2} is

v_{2}^{2} = \binom{}{ẋ_{2}}^{2} + \binom{}{ẏ_{2}}^{2}
= (ṙ cos θ − rθ̇ sin θ)^{2} + (ṙ sin θ + rθ̇ cos θ)^{2}
= ṙ^{2} + r^{2}θ̇^{2}.

Hence, the kinetic energy of the mass m_{2} is

K_{2} = \frac{1}{2}m_{2}v_{2}^{2} = \frac{1}{2}m_{2} (ṙ^{2} + r^{2}θ̇^{2}) .

The total kinetic energy of the system is

K = K_{1} + K_{2} = \frac{1}{2}m_{1}r_{1}^{2}θ̇^{2} + \frac{1}{2}m_{2} (ṙ^{2} + r^{2}θ̇^{2}) .

We now find the potential energy of the system. The potential energy of the mass m_{1} is

U_{1} = m_{1}gr_{1} sin θ.

The potential energy of the mass m_{2} is

U_{2} = m_{2}gr sin θ.

The total potential energy of the system is

U = U_{1} + U_{2} = m_{1}gr_{1} sin θ + m_{2}gr sin θ.

The Lagrangian function for the system is

L = K − U = \frac{1}{2}m_{1}r_{1}^{2}θ̇^{2} + \frac{1}{2}m_{2}ṙ^{2} + \frac{1}{2}m_{2}r^{2}θ̇^{2} − m_{1}gr_{1} sin θ − m_{2}gr sin θ.

The manipulator has two degrees of freedom. Hence, we have two Lagrange’s equations of motion for this system,

\begin{matrix}\frac{d}{dt} (\frac{∂L}{∂θ̇}) − \frac{∂L}{∂θ} & = & T_{θ} \\ \frac{d}{dt} (\frac{∂L}{∂ṙ}) − \frac{∂L}{∂r} & = & F_{r}. \end{matrix} \}

The first Lagrange equation yields

m_{1}r_{1}^{2}θ̈ + m_{2}r^{2}θ̈ + 2m_{2}rṙθ̇ + g cos θ(m_{1}r_{1} + m_{2}r) = T_{θ}.

The second Lagrange equation gives

m_{2}r̈ − m_{2}rθ̇^{2} + m_{2}g sin θ = F_{r}.

We next represent the above modeling equations in state space format. (One can use, for example, MATLAB’s Symbolic Math Toolbox to perform needed calculations.) Let the state variables be defined as:

z_{1} = θ, z_{2} = θ̇, z_{3} = r, z_{4} = ṙ.

Then, we can write

[ ż_{1} ; ż_{2} ; ż_{3} ; ż_{4} ] = [ z_{2} ; \frac{−2m_{2}z_{2}z_{3}z_{4}−g cos z_{1}(m_{1}r_{1}+m_{2}z_{3})+u_{1}}{m_{1}r_{1}^{2}+m_{2}z_{3}^{2}} ; z_{4} ; z_{2}^{2}z_{3} − g sin z_{1} + u_{2}/m_{2} ] .  (2.33)

In the following section, we develop models of a few more dynamical systems using basic laws of physics.

2.4 Model Catalog

2.4.1 Centrifugal Governor

In this subsection, we model the centrifugal governor. In our modeling, we follow Pontryagin [276, pages 213–220]. For an alternative approach to modeling the governor see Maxwell [231] and Fuller [109]. Some, like for example MacFarlane [226, page 250], claim that Maxwell founded the theory of automatic control systems with his 1868 paper “On governors”. However, this landmark paper is written in a terse and cryptic style. Fuller [109] does a wonderful job elucidating, correcting, and expanding Maxwell’s paper. Mayr [235, Chapter X] gives a fascinating historical account of the application of the centrifugal governor to the speed regulation of the steam engine and tracks the origins of the flyball governor.

A schematic of the governor is depicted in Figure 1.6, while forces acting on the governor are shown in Figure 2.5. If the weights rotate with angular velocity ω_{gov}, then the arms form

Figure 2.5: Forces acting on the weights in the governor. an angle φ with the vertical position. Let v_{weight} denote the linear velocity of the weights, and r be the radius of the circle along which the weights rotate. Then, v_{weight} = ω_{gov}r. Note that r = L sin φ. The resulting centrifugal force acting on each weight is

m\frac{v_{weight}^{2}}{r} = mω_{gov}^{2}r
= mω_{gov}^{2}L sin φ.

Simultaneously, a gravitational force mg acts on each weight. In addition, a frictional force in the hinge joints acts on each weight. We assume that the frictional force is proportional to φ̇ with the friction coefficient b. Summing forces along the axis perpendicular to an arm yields an equation of motion of the weight

mφ̈L = mω_{gov}^{2}L sin φ cos φ − mg sin φ − bφ̇
= \frac{1}{2}mω_{gov}^{2}L sin 2φ − mg sin φ − bφ̇.  (2.34)

The governor is connected to the engine via a gear with the gear ratio N such that

ω_{gov} = Nω,  (2.35)

where ω is the angular velocity of the flywheel. Substituting (2.35) into (2.34) gives

mφ̈L = \frac{1}{2}mN^{2}ω^{2}L sin 2φ − mg sin φ − bφ̇.

Let I be the moment of inertia of the flywheel, τ_{1} the torque delivered by the engine, and τ_{l} the load torque. Then, the equation modeling the engine is

Iω̇ = τ_{1} − τ_{l}.  (2.36)

The collar of the governor is connected with the valve so that at the nominal value of φ = φ_{0} the engine delivers the nominal torque τ_{d} that results in the desired angular velocity ω = ω_{d} of the flywheel. We model the interaction of the collar and the valve using the equation

τ_{1} = τ_{d} + κ(cos φ − cos φ_{0}),  (2.37)

where κ > 0 is a proportionality constant that depends on the governor. Substituting (2.37) into (2.36) yields

Iω̇ = τ_{1} − τ_{l}
= τ_{d} + κ(cos φ − cos φ_{0}) − τ_{l}
= κ cos φ − τ,

where τ = τ_{l} − τ_{d} + κ cos φ_{0}. We thus have two equations that model the governor,

mφ̈L = \frac{1}{2}mN^{2}ω^{2}L sin 2φ − mg sin φ − bφ̇,
Iω̇ = κ cos φ − τ.

We use the state variables

x_{1} = φ, x_{2} = φ̇, and x_{3} = ω

to represent the above model in state space format,

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & \frac{1}{2}N^{2}x_{3}^{2} sin 2x_{1} − \frac{g}{L} sin x_{1} − \frac{b}{mL}x_{2} \\ ẋ_{3} & = & \frac{κ}{I} cos x_{1} − \frac{τ}{I} . \end{matrix} \}  (2.38)

2.4.2 Ground Vehicle

In this subsection, we model a ground vehicle in a turning maneuver, such as, a step lane change maneuver. During such a maneuver, the vehicle undergoes translational as well as rotational motion. When modeling a moving vehicle, it is convenient to use reference coordinate frame attached to and moving with the vehicle. This is because with respect to the vehicle coordinate frame moments of inertias of the vehicle are constant. We use the Society of Automotive Engineers (SAE) standard coordinate system as shown in Figure 2.6. Our modeling follows that of Wong [354] and Will and Żak [350]. We use lower case letters to label axes of the vehicle coordinate system. Following the SAE convention, we define the coordinates as follows:

• the x-coordinate is directed forward and on the longitudinal plane of symmetry of the vehicle,

• the y-coordinate has a lateral direction out the right side of the vehicle,

• the z-coordinate is directed downward with respect to the vehicle,

• the yaw velocity about the z-axis is denoted θ̇.

In developing our simple model, we neglect the effects of rollindexroll—seeground vehicle and pitchindexpitch—seeground vehicle. The origin of the vehicle coordinate system is located at the vehicle center of gravity (CG). We also use a global, or fixed coordinate system. Its axes are labeled using upper case letters as shown in Figure 2.7. To form the equations of motion Figure 2.6: The Society of Automotive Engineers (SAE) coordinate system attached to the vehicle.

Figure 2.7: Analyzing vehicle motion using coordinate frame attached to the vehicle; snapshots of the vehicle velocity at times t and t + Δt.

Figure 2.8: Computing the change in the longitudinal and lateral components of the vehicle velocity.

of the vehicle, we need an expression for the vehicle CG acceleration. We first compute the change in the longitudinal component of the velocity. Using Figure 2.7 and Figure 2.8, and assuming small angles, we obtain

(v_{x} + Δv_{x}) cos Δθ − (v_{y} + Δv_{y}) sin Δθ − v_{x} ≈ Δv_{x} − v_{y}Δθ
= Δẋ − ẏΔθ.

We form the Newton’s quotient using the above. Then, taking the limit, we obtain the total longitudinal acceleration component a_{x},

a_{x} = lim_{Δt→0} \frac{Δv_{x} − v_{y}Δθ}{Δt}
= v̇_{x} − v_{y}θ̇
= ẍ − ẏω_{z},  (2.39)

Figure 2.9: A simplified, bicycle model of a vehicle.

where ω_{z} = θ̇. Using similar argument to the one above, we obtain the lateral acceleration component a_{y} to be

a_{y} = v̇_{y} + v_{x}θ̇
= ÿ + ẋω_{z}.  (2.40)

We assume that the vehicle is symmetric with respect to the x-z plane. We use the so called bicycle model that is shown in Figure 2.9, where δ_{f} denotes the front wheels’ steering angle and δ_{r} is the rear wheels’ steering angle. We denote by 2F_{xf} lumped longitudinal force at the front axle and by 2F_{xr} lumped longitudinal force at the rear axle. The lumped lateral force at the front axle is denoted 2F_{yf}, while lumped lateral force at the rear axle is denoted 2F_{yr}. Summing the forces along the x axis and assuming small angles, we obtain

m (ẍ − ẏω_{z}) = 2F_{xf} cos δ_{f} − 2F_{yf} sin δ_{f} + 2F_{xr} cos δ_{r} + 2F_{yr} sin δ_{r}
≈ 2F_{xf} + 2F_{xr}.  (2.41)

Next, summing the forces along the y axis and assuming small angles yields

m (ÿ + ẋω_{z}) = 2F_{xf} sin δ_{f} + 2F_{yf} cos δ_{f} + 2F_{yr} cos δ_{r} − 2F_{xr} sin δ_{r}
≈ 2F_{yf} + 2F_{yr}.  (2.42)

The third equation of motion is obtained by summing the torques with respect to the z-axis. Let I_{z} denote the moment of inertia of the vehicle about the z axis. Then, referring again to Figure 2.9 and assuming small angles, we obtain:

I_{z}ω̇_{z} = 2l_{1}F_{yf} cos δ_{f} + 2l_{1}F_{xf} sin δ_{f} − 2l_{2}F_{yr} cos δ_{r} + 2l_{2}F_{xr} sin δ_{r}
≈ 2l_{1}F_{yf} − 2l_{2}F_{yr}.  (2.43)

In our further analysis, we assume v_{x} = ẋ = constant, that is, the driver neither is accelerating nor braking the vehicle. This means that we need to use only the last two equations of motion in our model. To proceed, we need to define tire slip angles and cornering stiffness coefficients. The tire slip angle is the difference between the tire’s actual travel direction and the tire’s heading direction, which is the steer angle. In other words, the tire slip angle is the angle between the desired direction of motion of the tire specified by the steer angle and the actual direction of travel of the center of the tire contact. We denote by C_{αf} the cornering stiffness coefficient of the front tires and by C_{αr} the cornering stiffness coefficient of the rear tires. The coefficients C_{αf} and C_{αr} are measured experimentally. A typical relation between a tire slip angle and a cornering force is shown in Figure 2.10. It is a nonlinear relation, where the cornering coefficient is

C_{αw} = \frac{dF_{yw}}{dα_{w}} ,

where w = f or w = r. The lateral forces F_{yf} and F_{yr} that appear in (2.41)–(2.43) are functions of the corresponding tire slip angles α_{f} and α_{r} and cornering stiffness coefficients C_{αf} and C_{αr}. In our simple linearized model, we assume that C_{αf} = C_{αr} = constant, and model dependencies between the lateral forces and tire slip angles as

F_{yf} = C_{αf}α_{f},  (2.44)

and

F_{yr} = C_{αr}α_{r}.  (2.45)

We formulate expressions for the tire slip angles using Figure 2.11. Assuming small angles, we form an expression for the front tire slip angle

α_{f} = δ_{f} − tan^{−1} (\frac{l_{1}ω_{z} + v_{y}}{v_{x}} )
≈ δ_{f} − \frac{l_{1}ω_{z} + v_{y}}{v_{x}}
= δ_{f} − \frac{l_{1}ω_{z} + ẏ}{ẋ} .  (2.46)

78 CHAPTER 2. DYNAMICAL SYSTEM MODELING Figure 2.10: Typical cornering characteristic.

2.4. MODEL CATALOG 79

Figure 2.11: Simplified vehicle model for computing tire slip angles.

The rear tire slip angle is

α_{r} = δ_{r} − tan^{−1} (−\frac{l_{2}ω_{z} − v_{y}}{v_{x}} )
= δ_{r} + tan^{−1} (\frac{l_{2}ω_{z} − v_{y}}{v_{x}} )
≈ δ_{r} + \frac{l_{2}ω_{z} − v_{y}}{v_{x}}
= δ_{r} + \frac{l_{2}ω_{z} − ẏ}{ẋ} .  (2.47)

Hence

F_{yf} = C_{αf} (δ_{f} − \frac{l_{1}ω_{z} + ẏ}{ẋ} ) ,  (2.48)

and

F_{yr} = C_{αr} (δ_{r} + \frac{l_{2}ω_{z} − ẏ}{ẋ} ) .  (2.49)

Substituting the above two equations into (2.42), and assuming small angles yields

mÿ + mẋω_{z} = 2C_{αf}δ_{f} − 2C_{αf} \frac{l_{1}ω_{z} + ẏ}{v_{x}} + 2C_{αr}δ_{r} + 2C_{αr} \frac{l_{2}ω_{z} − ẏ}{v_{x}} .

Rearranging terms and performing some manipulations, we get

ÿ = − (\frac{2C_{αf} + 2C_{αr}}{mv_{x}} ) ẏ + (−v_{x} − \frac{2C_{αf}l_{1} − 2C_{αr}l_{2}}{mv_{x}} ) ω_{z} + \frac{2C_{αf}}{m} δ_{f} + \frac{2C_{αr}}{m} δ_{r}. (2.50)

Substituting the expressions for F_{yf} and F_{yr} into (2.43) and assuming small angles, we obtain

I_{z}ω̇_{z} = 2l_{1}C_{αf} (δ_{f} − \frac{l_{1}ω_{z} + ẏ}{v_{x}} ) − 2l_{2}C_{αr} (δ_{r} + \frac{l_{2}ω_{z} − ẏ}{v_{x}} ) .

Rearranging and performing manipulations yields

ω̇_{z} = − (\frac{2l_{1}C_{αf} − 2l_{2}C_{αr}}{I_{z}v_{x}} ) ẏ − (\frac{2l_{1}^{2}C_{αf} + 2l_{2}^{2}C_{αr}}{I_{z}v_{x}} ) ω_{z} + \frac{2l_{1}C_{αf}}{I_{z}} δ_{f} − \frac{2l_{2}C_{αr}}{I_{z}} δ_{r}. (2.51)

Later, we will analyze a vehicle performance in the fixed coordinates. In particular, we will be interested in an emergency lane change maneuver. This requires that we transform the vehicle lateral variable into the fixed coordinates. Assuming small angles and referring to Figure 2.12, we obtain

Ẏ = −ẋ sin ψ − ẏ cos ψ
≈ −v_{x}ψ − ẏ.  (2.52)

Note that

Ẋ = ẋ cos ψ − ẏ sin ψ.

For the step lane change, the yaw rate of the vehicle is equal to the yaw rate error, that is,

ω_{z} = ψ̇.

We further assume that

θ = ψ.  (2.53)

We can now represent the vehicle modeling equations in a state space format. Let

x_{1} = ẏ, x_{2} = θ, x_{3} = ω_{z}, x_{4} = Y.

Then, using (2.50), (2.51), (2.53) and (2.52), we obtain

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ; ẋ_{4} ] = [ \begin{matrix}−\frac{2C_{αf} +2C_{αr}}{mv_{x}} & 0 & −v_{x} − \frac{2C_{αf} l_{1}−2C_{αr}l_{2}}{mv_{x}} & 0 \\ 0 & 0 & 1 & 0 \\ −\frac{2l_{1}C_{αf} −2l_{2}C_{αr}}{I_{z}v_{x}} & 0 & −\frac{2l_{1}^{2}C_{αf} +2l_{2}^{2}C_{αr}}{I_{z}v_{x}} & 0 \\ −1 & −v_{x} & 0 & 0 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \end{matrix} ]

+ [ \begin{matrix}\frac{2C_{αf}}{m} & \frac{2C_{αr}}{m} \\ 0 & 0 \\ \frac{2l_{1}C_{αf}}{I_{z}} & −\frac{2l_{2}C_{αr}}{I_{z}} \\ 0 & 0 \end{matrix} ] [ \begin{matrix}δ_{f} \\ δ_{r} \end{matrix} ] .  (2.54)

Figure 2.12: Computing the velocity of a vehicle with respect to the fixed coordinates.

A possible vehicle parameters are given in Table 2.1.

2.4.3 Permanent Magnet Stepper Motor

In this subsection, we develop a model of a permanent magnet (PM) stepper motor. Stepper motors can be viewed as digital positioning devices. Specifically, stepper motors are electromechanical motion devices that convert input information in digital form into output that has mechanical form. The stepper motor can be driven by digital pulses, for example of the form: move clockwise or move counterclockwise. We now describe an idea of an operation

Table 2.1: Typical vehicle data.

C_{αf} = C_{αr} 30, 000.00 N/rad
l_{1}  1.20 m
l_{2}  1.22 m
m  1280 kg
I_{z}  2500 kg·m^{2}
v_{x}  18.3 m/sec

Figure 2.13: A simplified schematic of an operation of a stepper motor.

of a PM stepper motor and then derive its modeling equations.

A simplified organization of a stepper motor is shown in Figure 2.13. The armature, or rotor, is a magnet. At any instant of the operation, exactly one pair of windings of the stator is energized. A pair of windings represents a phase. Each pair of the stator windings is wound in such a way that their magnetic fields are collinear, for example, windings A and A’. The rotor then rotates until its magnetic field is aligned with the fields of the energized pair of the stator windings. Figure 2.13 shows the situation where the pair of windings A and A’ is energized. If the rotor is at rest, as shown in Figure 2.13, and the pair A, A’ is de-energized, while the pair B, B’ is energized, the rotor rotates to the new position so that its magnetic field is aligned with the magnetic field of the pair B, B’. In this example of the three-phase motor, a single step results in the rotor motion of 360^{∘}/6 = 60^{∘}. We next give a more detailed description of the electrical organization of the stepper motor and construct its model.

The basic PM stepper motor consists of a slotted stator and a rotor. There can be two sets of teeth on the rotor. These two sets of teeth are out of alignment with each other by

Figure 2.14: A possible rotor structure of a PM stepper motor.

a tooth width, as shown in Figure 2.14. Specifically, the rotor teeth of the left end-cap are displaced one half pitch from the teeth on the right end-cap. A magnet, or an electromagnet, that is mounted on the rotor magnetizes the iron end-caps. The left end-cap is magnetized as a north pole, while the right end-cap is magnetized as a south pole. The slotted stator is equipped with two or more individual coils. Figure 2.15 shows a four-pole stator and a five-pole rotor structure.

We now discuss the way in which the PM stepper motor can be used to perform precise positioning. Assume that the starting point is as represented in Figure 2.16. Here, coil A-A’ is energized so that the upper stator pole is given an S-pole polarity and the corresponding lower stator is given an N-pole polarity. The rotor remains in this position as long as stator coil voltage, and hence stator polarities, remain unchanged. This is because the net torque acting on the rotor is zero. If we now de-energize coil A-A’ and energize coil B-B’ so that the left pole becomes an S-pole and the right pole assumes an N-polarity, then this results in the new steady-state position shown in Figure 2.17. We can proceed by alternatively energizing the two stator coils. The PM stepper motor can thus be controlled by digital pulses applied 84 CHAPTER 2. DYNAMICAL SYSTEM MODELING Figure 2.15: Construction features of a four-pole stator and a five-pole rotor PM stepper motor.

2.4. MODEL CATALOG 85 Figure 2.16: Coil A-A’ energized for an S-N orientation, while coil B-B’ de-energized.

86 CHAPTER 2. DYNAMICAL SYSTEM MODELING Figure 2.17: Coil A-A’ de-energized, while coil B-B’ energized for an S-N orientation. to its two inputs: move clockwise or move counterclockwise.

Our next objective is to devise a mathematical model of the PM stepper motor. In our discussion we use the following notation. We denote by R_{t} the number of rotor teeth. The angular displacement between rotor teeth is called the tooth pitch and denoted T_{p}. The angular displacement of the rotor as we change the input voltage from one pair of windings to the other is called the step length and denoted S_{l}. The number of phases, that is, the number of pairs of stator windings, is denoted N_{p}. In our example, we have N_{p} = 2, R_{t} = 5, and T_{p} = 2π/R_{t} = 72^{∘}. In this example, it takes four input voltage switchings, and hence steps, to advance the rotor one tooth pitch. Indeed, because

T_{p} = 2N_{p}S_{l} = \frac{2π}{R_{t}} ,  (2.55)

hence

S_{l} = \frac{π}{R_{t}N_{p}} = 18^{∘}.  (2.56)

We denote the rotor angular displacement by θ.

Following Krause and Wasynczuk [192, pages 350–351], we trace the main path of flux linking the A-A’ windings for the rotor position shown in Figure 2.18. Flux leaves the left end-cap through the rotor tooth at the top that is aligned with the stator tooth having the A part of the A-A’ winding. The flux then travels up through the stator tooth in the stator iron. Next, the flux splits and travels around the circumference of the stator, as shown in Figure 2.18(a). The flux returns to the south pole of the rotor through the stator tooth positioned at the bottom of the A’ part of the A-A’ winding, as shown in Figure 2.18(b). We denote the peak flux linkage produced by the rotor permanent magnet, in the A-A’ stator winding by nϕ_{m}, where n is the number of turns of the winding. For θ = 0^{∘}, corresponding to the rotor position in Figure 2.18, the flux linkage in the A-A’ winding of the stator is maximal and is approximately equal to nϕ_{m}. For θ = 90^{∘}/R_{t}, corresponding to the rotor position in Figure 2.17, the flux linkage in the A-A’ winding of the stator is minimal, approximately equal to zero. We model the flux linkage, ψ_{a}, produced by the permanent magnet, in the A-A’ stator winding as

ψ_{a} = nϕ_{m} cos(R_{t}θ).  (2.57)

The (main) flux, linking the B-B’ winding for the rotor position shown in Figure 2.17, would enter the stator tooth on which the B part of the B-B’ winding is located from the rotor tooth on the left. The flux then would travel around the circumference of the stator and return to the rotor through the stator pole on which the B’ part of the B-B’ winding is wound. For θ = 90^{∘}/R_{t}, corresponding to the rotor position in Figure 2.17, the flux linkage 88 CHAPTER 2. DYNAMICAL SYSTEM MODELING

Figure 2.18: The main path of flux linking the A-A’ winding.

Table 2.2: Symbols used in modeling equations of the PM stepper motor.

T_{p} = tooth pitch S_{l} = step length R_{t} = number of teeth on rotor N_{p} = number of phases, that is, stator winding pairs θ = angular displacement of the rotor ω = angular velocity of the rotor v_{a}(t) = voltage applied to phase a, that is, A-A’ stator winding v_{b}(t) = voltage applied to phase b, that is, B-B’ stator winding i_{a}(t) = current in phase a i_{b}(t) = current in phase b R = stator winding resistance per phase L = stator winding inductance per phase B = viscous friction coefficient I = rotor inertia K_{b} = back emf constant

in the B-B’ winding of the stator is maximal and is approximately equal to nϕ_{m}. For θ = 0^{∘}, corresponding to the rotor position in Figure 2.18, the flux linkage in the B-B’ winding is minimal, approximately equal to zero. We model the flux linkage in the B-B’ phase of the stator by

ψ_{b} = nϕ_{m} sin(R_{t}θ).  (2.58)

In the following development, we use symbols that are described in Table 2.2. Applying Kirchhoff’s voltage law to the circuit of phase a yields

v_{a}(t) = Ri_{a}(t) + L\frac{di_{a}(t)}{dt} + e_{a}(t),

where e_{a} is the back emf induced in phase a, that is,

e_{a} = \frac{dψ_{a}}{dt}
= \frac{dψ_{a}}{dθ} \frac{dθ}{dt}
= −nR_{t}ϕ_{m} sin(R_{t}θ)\frac{dθ}{dt}
= −K_{b} ω sin(R_{t}θ),

where K_{b} = nR_{t}ϕ_{m}. Hence, the voltage equation for phase a can be represented as

v_{a} = Ri_{a} + L\frac{di_{a}}{dt} − K_{b} ω sin(R_{t}θ).

Applying now Kirchhoff’s voltage law to the circuit of phase b yields

v_{b}(t) = Ri_{b}(t) + L\frac{di_{b}(t)}{dt} + e_{b}(t),

where e_{b} is the back emf induced in phase b, that is,

e_{b} = \frac{dψ_{b}}{dt}
= \frac{dψ_{b}}{dθ} \frac{dθ}{dt}
= nR_{t}ϕ_{m} cos(R_{t}θ)\frac{dθ}{dt}
= K_{b} ω cos(R_{t}θ),

where as before K_{b} = nR_{t}ϕ_{m}. Hence, the voltage equation for phase b can be represented as

v_{b} = Ri_{b} + L\frac{di_{b}}{dt} + K_{b} ω cos(R_{t}θ).

Let T_{a} and T_{b} denote torques produced by the currents i_{a} and i_{b}, respectively. Then, applying Newton’s second law to the rotor yields

I \frac{d^{2}θ}{dt^{2}} + B \frac{dθ}{dt} = T_{a} + T_{b}.

The PM stepper motor is an electromechanical device in which electric power is transformed into mechanical power. The mechanical power, P_{a}, due to torque T_{a} is

P_{a} = T_{a} \frac{dθ}{dt}.  (2.59)

Assuming that there are no losses, this mechanical power is equal to the power developed in phase a,

P_{a} = i_{a} \frac{dψ_{a}}{dt} = i_{a} \frac{dψ_{a}}{dθ} \frac{dθ}{dt}.  (2.60)

Equating the right hand sides of (2.59) and (2.60) and performing simple manipulations, we obtain

T_{a} = i_{a} \frac{dψ_{a}}{dθ} = −i_{a}K_{b} sin(R_{t}θ).

Applying similar arguments to phase b, we get

T_{b} = i_{b} \frac{dψ_{b}}{dθ} = i_{b}K_{b} cos(R_{t}θ).

Taking the above equations into account, we represent the equation of motion of the rotor as

I \frac{d^{2}θ}{dt^{2}} + B \frac{dθ}{dt} = T_{a} + T_{b} = −i_{a}K_{b} sin(R_{t}θ) + i_{b}K_{b} cos(R_{t}θ).

Combining the above equations, we obtain a model of a two-phase, that is, a two input PM stepper motor. We have

\frac{dθ}{dt} = ω
\frac{dω}{dt} = −\frac{K_{b}}{I} i_{a} sin(R_{t}θ) + \frac{K_{b}}{I} i_{b} cos(R_{t}θ) − \frac{B}{I} ω
\frac{di_{a}}{dt} = −\frac{R}{L}i_{a} + \frac{K_{b}}{L} ω sin(R_{t}θ) + \frac{1}{L}v_{a}
\frac{di_{b}}{dt} = −\frac{R}{L}i_{b} − \frac{K_{b}}{L} ω cos(R_{t}θ) + \frac{1}{L}v_{b}.

We now define the state variables: x_{1} = θ, x_{2} = ω, x_{3} = i_{a}, and x_{4} = i_{b}. We let K_{1} = B/I, K_{2} = K_{b}/I, K_{3} = R_{t}, K_{4} = K_{b}/L, K_{5} = R/L, u_{1} = \frac{1}{L}v_{a}, and u_{2} = \frac{1}{L}v_{b}. Using the above notation, we represent the PM stepper motor model in state space format:

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ; ẋ_{4} ] = [ \begin{matrix}x_{2} \\ −K_{1}x_{2} − K_{2}x_{3} sin(K_{3}x_{1}) + K_{2}x_{4} cos(K_{3}x_{1}) \\ K_{4}x_{2} sin(K_{3}x_{1}) − K_{5}x_{3} \\ −K_{4}x_{2} cos(K_{3}x_{1}) − K_{5}x_{4} \end{matrix} ] + [ \begin{matrix}0 & 0 \\ 0 & 0 \\ 1 & 0 \\ 0 & 1 \end{matrix} ] [ \begin{matrix}u_{1} \\ u_{2} \end{matrix} ] .(2.61)

2.4.4 Stick Balancer

In this subsection, we develop a nonlinear model of the stick balancer, also known as the cart with an inverted pendulum. In our derivation of the model, we use Newton’s laws, in a similar fashion as Kwakernaak and Sivan [199], and Ogata [255] did, where linearized models of the stick balancer were developed. An alternative derivation, using the method of D’Alembert, can be found in Cannon [50, Section 22.4].

A free body diagram of a cart on which a stick, that is, an inverted pendulum, is mounted is shown in Figure 2.19. Let H = H(t) and V = V (t) be horizontal and vertical reaction forces, respectively. The x and y are the coordinates of the fixed, nonrotating coordinate frame x − y. The angular displacement of the stick from the vertical position is θ = θ(t). The mass of the cart is M, while the mass of the stick is m. The length of the stick is 2l, and its center of gravity is at its geometric center. The control force applied to the cart is labeled as u. We assume that the wheels of the cart do not slip. We model the frictional force of the cart wheels on the track by

f_{c} = μ_{c}sign(ẋ),  (2.62)

92 CHAPTER 2. DYNAMICAL SYSTEM MODELING Figure 2.19: Free body diagram of a cart with an inverted pendulum. where μ_{c} is the cart friction coefficient. Friedland [108, page 201] refers to this model of friction as the classical Coulomb friction model. Let (x_{G}, y_{G}) be the coordinates of the center of gravity of the stick. Then,

\{ \begin{matrix}x_{G} & = & x + l sin(θ) \\ y_{G} & = & l cos(θ). \end{matrix}  (2.63)

We are now ready to write down equations modeling the system. The equation that describes the rotational motion of the stick about its center of gravity is obtained by applying the rotational version of Newton’s second law. Summing the moments about the center of gravity of the stick, we obtain

I \frac{d^{2}θ}{dt^{2}} = V l sin(θ) − Hl cos(θ),  (2.64)

where

I = ∫_{−l}^{l} r^{2}dm = ml^{2}/3,  (2.65)

is the moment of inertia of the stick with respect to its center of gravity.

We next write the equation that describes the horizontal motion of the center of gravity of the stick. Applying Newton’s second law along the x axis yields

m\frac{d^{2}}{dt^{2}} (x + l sin(θ)) = H.  (2.66)

Performing the required differentiation, we get

m (ẍ + l (−θ̇^{2} sin(θ) + θ̈cos(θ))) = H.  (2.67)

The equation that describes the vertical motion of the center of gravity of the stick is obtained by applying Newton’s second law along the y axis. We have

m\frac{d^{2}}{dt^{2}} (l cos(θ)) = V − mg.  (2.68)

Performing the needed differentiation yields

ml (−θ̇^{2} cos(θ) − θ̈sin(θ)) = V − mg.  (2.69)

Finally, we apply Newton’s second law to the cart to get

M \frac{d^{2}x}{dt^{2}} = u − H − f_{c}.  (2.70)

Substituting (2.70) into (2.67) gives

mẍ + mlθ̈cos(θ) − mlθ̇^{2} sin(θ) + f_{c} = u − Mẍ.  (2.71)

Substituting now equations (2.69) and (2.70) into (2.64) yields

Iθ̈ = (mg − mlθ̇^{2} cos(θ) − mlθ̈sin(θ)) l sin(θ) + f_{c}l cos(θ) − (u − Mẍ) l cos(θ). (2.72)

We next substitute u − Mẍ from (2.71) into (2.72) and perform manipulations to get

Iθ̈ = mgl sin(θ) − ml^{2}θ̈ − mẍl cos(θ).  (2.73)

Let

a = \frac{1}{m + M} .

Then, we can represent (2.71) as

ẍ = −malθ̈cos(θ) + malθ̇^{2} sin(θ) − af_{c} + au.  (2.74)

We substitute (2.74) into (2.73) to obtain

θ̈ = \frac{mgl sin(θ) − m^{2}l^{2}aθ̇^{2} sin(2θ)/2 + mal cos(θ)f_{c} − mal cos(θ)u}{I − m^{2}l^{2}a cos^{2}(θ) + ml^{2}}.  (2.75)

Let x_{1} = θ and x_{2} = θ̇. Using the expression for I, given by (2.65), we represent (2.75) in state space format as follows

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & \frac{g sin(x_{1})−mlax_{2}^{2} sin(2x_{1})/2+a cos(x_{1})f_{c}}{4l/3−mla cos^{2}(x_{1})} − \frac{mal cos(x_{1})u}{4ml^{2}/3−m^{2}l^{2}a cos^{2}(x_{1})}. \end{matrix} \}  (2.76)

We next substitute θ̈, obtained from (2.73), into (2.74) to get

ẍ = \frac{−mag sin(2x_{1})/2 + ax_{2}^{2} sin(x_{1})4ml/3 + (u − f_{c})4a/3}{4/3 − ma cos^{2}(x_{1})}.  (2.77)

Let x_{3} = x and x_{4} = ẋ. Then, combining (2.76) and (2.77) we obtain a state space model of the inverted pendulum on a cart of the form,

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ; ẋ_{4} ] = [ x_{2} \frac{g sin(x_{1})−mlax_{2}^{2} sin(2x_{1})/2}{4l/3−mla cos^{2}(x_{1})} x_{4} \frac{−mag sin(2x_{1})/2+alx_{2}^{2} sin(x_{1})4m/3}{4/3−ma cos^{2}(x_{1})} ] + [ 0 \frac{−a cos(x_{1})}{4l/3−mla cos^{2}(x_{1})} 0 \frac{4a/3}{4/3−ma cos^{2}(x_{1})} ] (u − f_{c}).  (2.78)

While designing a controller for the stick balancer system, we will use an equivalent representation of (2.78),

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ; ẋ_{4} ] = [ x_{2} \frac{g sin(x_{1})}{4l/3−mla cos^{2}(x_{1})} x_{4} \frac{−mag sin(2x_{1})/2}{4/3−ma cos^{2}(x_{1})} ] + [ 0 \frac{−a cos(x_{1})}{4l/3−mla cos^{2}(x_{1})} 0 \frac{4a/3}{4/3−ma cos^{2}(x_{1})} ] (u − f_{c} + mlx_{2}^{2} sin(x_{1})).  (2.79)

2.4.5 One-Link Robot Manipulator

The example comes from [368]. It is a simple model of a one-link robot manipulator shown in Figure 2.20. The motion of the robot arm is controlled by a DC motor via a gear. The

Figure 2.20: One-link manipulator controlled by a DC motor via a gear.

DC motor is armature-controlled and its schematic is shown in Figure 2.21. We assume that the motor moment of inertia is negligible compared with that of the robot arm. We model the arm as a point mass m attached to the end of a massless rod of length l. Hence the arm inertia is I_{a} = ml^{2}. We assume that the gear train has no backlash, and all connecting shafts are rigid. As can be seen from Figure 2.20, counterclockwise rotation of the arm is defined as positive and clockwise rotation of the arm as negative, while counter-clockwise rotation of the motor shaft is defined to be negative, and clockwise rotation of the shaft as positive.

Figure 2.21: Schematic of an armature-controlled DC motor of Example 9.13.

The torque delivered by the motor is

T_{m} = K_{m}i_{a},

where K_{m} is the motor-torque constant, and i_{a} is the armature current. Let N denote the gear ratio. Then we have

\frac{θ_{p}}{θ_{m}} = \frac{radius of motor gear}{radius of arm gear} = \frac{number of teeth of motor gear}{number of teeth of arm gear} = \frac{1}{N} .

This is because the gears are in contact and thus

θ_{p} × radius of arm gear = θ_{m} × radius of motor gear,

and the radii of the gears are proportional to their number of teeth. The work done by the gears must be equal. Let T_{p} denote the torque applied to the robot arm. Then,

T_{p}θ_{p} = T_{m}θ_{m}.

Thus, the torque applied to the pendulum is

T_{p} = NT_{m} = NK_{m}i_{a}.

We use Newton’s second law to write the equation modeling the arm dynamics,

I_{a} \frac{d^{2}θ_{p}}{dt^{2}} = mgl sin θ_{p} + T_{p}.  (2.80)

Substituting into (2.80) the expressions for I_{a} and T_{p} and then rearranging yields

ml^{2} \frac{d^{2}θ_{p}}{dt^{2}} = mgl sin θ_{p} + NK_{m}i_{a},  (2.81)

where g = 9.8 m/s^{2} is the gravitational constant. We next apply Kirchhoff’s voltage law to the armature circuit to obtain

L_{a} \frac{di_{a}}{dt} + Ri_{a} + e_{b} = L_{a} \frac{di_{a}}{dt} + Ri_{a} + K_{b}N \frac{dθ_{p}}{dt} = u,

where K_{b} is the back emf constant. We can now construct a third-order state-space model of the one-link robot. For this we choose the following state variables:

x_{1} = θ_{p}, x_{2} = \frac{dθ_{p}}{dt} = ω_{p}, x_{3} = i_{a}.

Then, the model in state-space format is:

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ] = [ \begin{matrix}x_{2} \\ \frac{g}{l} sin x_{1} + \frac{NK_{m}}{ml^{2}} x_{3} \\ −\frac{K_{b}N}{L_{a}} x_{2} − \frac{R_{a}}{L_{a}} x_{3} \end{matrix} ] + [ 0 ; 0 ; \frac{1}{L_{a}} ] u,  (2.82)

Reasonable parameters of the robot are: l = 1 m, m = 1 kg, N = 10, K_{m} = 0.1 Nm/A, K_{b} = 0.1 Vs/rad, R_{a} = 1 Ω, L_{a} = 100 mH. With the above parameters the robot model takes the form:

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ] = [ \begin{matrix}x_{2} \\ 9.8 sin x_{1} + x_{3} \\ −10 x_{2} − 10 x_{3} \end{matrix} ] + [ \begin{matrix}0 \\ 0 \\ 10 \end{matrix} ] u.  (2.83)

We assume that the output, y, is

y = x_{1}.  (2.84)

2.4.6 Double Pendulum

We construct a model of the double pendulum shown in Figure 2.22. This double pendulum can be viewed as a model of a double-link robot manipulator. We assume that the input torques, T_{1} and T_{2} are applied at each joint, ω_{i} = θ̇_{i}, i = 1, 2, are the angular velocities of the pendulum arms, Δθ = θ_{2} − θ_{1}, and m_{i} and l_{i} are the masses and arm lengths, respectively.

Figure 2.22: Double pendulum.

Table 2.3: Parameter numerical values of double pendulum system shown in Figure 2.22.

m_{1} = m_{2} 1 kg
l_{1} = l_{2}  0.5 m
g  9.8 m/sec^{2}

The parameter values are given in Table 2.3. We write the Lagrange equations of motion for the system and represent them in state-space format, where the state variables are defined as, x_{1} = θ_{1}, x_{2} = θ̇_{1}, x_{3} = θ_{2}, and x_{4} = θ̇_{2}. Lagrange Equations of Motion of the Double Pendulum System

Recall that the Lagrangian L is defined as L = K − U, where K is the kinetic energy of the double pendulum system shown in Figure 2.22 and U is the potential energy of the system. The kinetic energy is

K = K_{m_{1}} + K_{m_{2}}
= \frac{1}{2}m_{1}∥v_{m_{1}}∥^{2} + \frac{1}{2}m_{2}∥v_{m_{2}}∥^{2},

where v_{m_{i}}, i = 1, 2, are the velocities of the point masses. The velocity vectors can be computed from the position of the point masses. The position of the first mass is described by

x_{1} = l_{1} sin(θ_{1}), y_{1} = l_{1} cos(θ_{1}).  (2.85)

The position of m_{2} is determined by

x_{2} = l_{1} sin(θ_{1}) + l_{2} sin(θ_{2}), y_{2} = l_{1} cos(θ_{1}) + l_{2} cos(θ_{2}).  (2.86)

We next find time derivatives of the expressions in (2.85) and (2.86). The time derivatives of x_{1} and y_{1} are,

ẋ_{1} = l_{1}θ̇_{1} cos(θ_{1}), ẏ_{1} = −l_{1}θ̇_{1} sin(θ_{1}).

The time derivatives of x_{2} and y_{2} are,

ẋ_{2} = l_{1}θ̇_{1} cos(θ_{1}) + l_{2}θ̇_{2} cos(θ_{2}), ẏ_{2} = −l_{1}θ̇_{1} sin(θ_{1}) − l_{2}θ̇_{2} sin(θ_{2}).

Using the above expressions, we compute

∥v_{m_{1}}∥^{2} = ẋ_{1}^{2} + ẏ_{1}^{2} = (l_{1}θ̇_{1})^{2}

and

∥v_{m_{2}}∥^{2} = ẋ_{2}^{2} + ẏ_{2}^{2}
= (l_{1}θ̇_{1})^{2} cos^{2}(θ_{1}) + 2l_{1}l_{2}θ̇_{1}θ̇_{2} cos(θ_{1}) cos(θ_{1}) + (l_{2}θ̇_{2})^{2} cos^{2}(θ_{2})
+ (l_{1}θ̇_{1})^{2} sin^{2}(θ_{1}) + 2l_{1}l_{2}θ̇_{1}θ̇_{2} sin(θ_{1}) sin(θ_{1}) + (l_{2}θ̇_{2})^{2} sin^{2}(θ_{2})
= (l_{1}θ̇_{1})^{2} + (l_{2}θ̇_{2})^{2} + 2l_{1}l_{2}θ̇_{1}θ̇_{2} cos(θ_{2} − θ_{1}),

where we used the well known trigonometric identity,

cos(θ_{2} − θ_{1}) = cos(θ_{2}) cos(θ_{1}) + sin(θ_{2}) sin(θ_{1}).

In view of the above, the kinetic energy of the system is

K = K_{m_{1}} + K_{m_{2}}
= \frac{1}{2}m_{1}∥v_{m_{1}}∥^{2} + \frac{1}{2}m_{2}∥v_{m_{2}}∥^{2}
= \frac{1}{2}m_{1} (l_{1}θ̇_{1})^{2} + \frac{1}{2}m_{2} ((l_{1}θ̇_{1})^{2} + (l_{2}θ̇_{2})^{2} + 2l_{1}l_{2}θ̇_{1}θ̇_{2} cos(θ_{2} − θ_{1})) .

To compute the potential energy, we observe the positive direction of the y-coordinate and the direction of the gravity force are the same. Therefore, the potential energy is

U = −m_{1}gy_{1} − m_{2}gy_{2}
= −m_{1}gl_{1} cos(θ_{1}) − m_{2}g (l_{1} cos(θ_{1}) + l_{2} cos(θ))
= −(m_{1} + m_{2})gl_{1} cos(θ_{1}) − m_{2}gl_{2} cos(θ_{2}).

Figure 2.23: Angles in the double pendulum system of Figure 2.22.

Hence, the Lagrangian function of the double pendulum system is

L = K − U
= \frac{1}{2}m_{1} (l_{1}θ̇_{1})^{2} + \frac{1}{2}m_{2} ((l_{1}θ̇_{1})^{2} + (l_{2}θ̇_{2})^{2} + 2l_{1}l_{2}θ̇_{1}θ̇_{2} cos(θ_{2} − θ_{1}))
+(m_{1} + m_{2})gl_{1} cos(θ_{1}) + m_{2}gl_{2} cos(θ_{2}).

We next derive the Lagrange equations of motion of the system. For this, we need to determine the generalized forces that constitute the right hand sides of the Lagrange equations of motion. Referring to Figure 2.23, we write,

δW = T_{1}δθ_{1} + T_{2}δ(θ_{2} − θ_{1})
= (T_{1} − T_{2})δθ_{1} + T_{2}δθ_{2},  (2.87)

where T_{1} and T_{2} are the torques applied at each joint. Hence the generalized forces are

F_{θ_{1}} = T_{1} − T_{2} and F_{θ_{2}} = T_{2}.  (2.88)

To proceed, we need to find partial derivatives of the Lagrangian function. We first evaluate ∂L/∂θ̇_{1} to get

\frac{∂L}{∂θ̇_{1}} = m_{1}l_{1}^{2}θ̇_{1} + m_{2}l_{1}^{2}θ̇_{1} + m_{2}l_{1}l_{2}θ̇_{2} cos(θ_{2} − θ_{1}).  (2.89)

Next, we find

\frac{d}{dt} \frac{∂L}{∂θ̇_{1}} = m_{1}l_{1}^{2}θ̈_{1} + m_{2}l_{1}^{2}θ̈_{1} + m_{2}l_{1}l_{2}θ̈_{2} cos(θ_{2} − θ_{1}) − m_{2}l_{1}l_{2}θ̇_{2}(θ̇_{2} − θ̇_{1}) sin(θ_{2} − θ_{1}). (2.90)

The partial derivative ∂L/∂θ_{1} is

\frac{∂L}{∂θ_{1}} = m_{2}l_{1}l_{2}θ̇_{1}θ̇_{2} sin(θ_{2} − θ_{1}) − (m_{1} + m_{2})gl_{1} sin(θ_{1}).  (2.91)

Combining (2.88), (2.90), and (2.91), we obtain the first Lagrange equation of motion,

(m_{1} + m_{2})l_{1}^{2}θ̈_{1} + m_{2}l_{1}l_{2}θ̈cos(θ_{2} − θ_{1}) − m_{2}l_{1}l_{2}θ̇_{2}^{2} sin(θ_{2} − θ_{1}) + (m_{1} + m_{2})gl_{1} sin(θ_{1}) = T_{1} − T_{2}.

(2.92)

We next compute partial derivatives of L that we use to write the second Lagrange equation of motion. We first find ∂L/∂θ̇_{2},

\frac{∂L}{∂θ̇_{2}} = m_{2}l_{2}^{2}θ̇_{2} + m_{2}l_{2}l_{2}θ̇_{1} cos(θ_{2} − θ_{1}).  (2.93)

The time derivative of the above expression is

\frac{d}{dt} \frac{∂L}{∂θ̇_{2}} = m_{2}l_{2}^{2}θ̈_{2} + m_{2}l_{1}l_{2}θ̈_{1} cos(θ_{2} − θ_{1}) − m_{2}l_{1}l_{2}(θ̇_{2} − θ̇_{1})θ̇_{1} sin(θ_{2} − θ_{1}).  (2.94)

The partial derivative ∂L/∂θ_{2} is

\frac{∂L}{∂θ_{2}} = −m_{2}l_{1}l_{2}θ̇_{1}θ̇_{2} sin(θ_{2} − θ_{1}) − m_{2}gl_{2} sin(θ_{2}).  (2.95)

Combining (2.88), (2.94), and (2.95), we obtain the second Lagrange equation of motion,

m_{2}l_{2}^{2}θ̈_{2} + m_{2}l_{1}l_{2}θ̈_{1} cos(θ_{2} − θ_{1}) + m_{2}l_{1}l_{2}θ̇_{1}^{2} sin(θ_{2} − θ_{1}) + m_{2}gl_{2} sin(θ_{2}) = T_{2}.  (2.96)

State-Space Model of the Double Pendulum System Taking into account the definition of the state variables, we obtain

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ; ẋ_{4} ] = [ x_{2} ; θ̈_{1} ; x_{4} ; θ̈_{2} ] .  (2.97)

Thus, to obtain a state-space model of the double pendulum system, we need to find the acceleration of each link. For this, we represent Lagrange equations (2.92) and (2.96) in matrix form,

[ \begin{matrix}(m_{1} + m_{2})l_{1}^{2} & m_{2}l_{1}l_{2} cos(θ_{2} − θ_{1}) \\ m_{2}l_{1}l_{2} cos(θ_{2} − θ_{1}) & m_{2}l_{2}^{2} \end{matrix} ] [ θ̈_{1} ; θ̈_{2} ]
= [ \begin{matrix}T_{1} − T_{2} + m_{2}l_{1}l_{2}θ̇_{2}^{2} sin(θ_{2} − θ_{1}) − (m_{1} + m_{2})gl_{1} sin(θ_{1}) \\ T_{2} − m_{2}l_{1}l_{2}θ̇_{1}^{2} sin(θ_{2} − θ_{1}) − m_{2}gl_{2} sin(θ_{2}) \end{matrix} ] . (2.98)

Note that the square matrix, call it M, on the left-hand side of the above equation is invertible because its determinant is

det M = (m_{1} + m_{2})m_{2}l_{1}^{2}l_{2}^{2} − m_{2}^{2}l_{1}^{2}l_{2}^{2} cos^{2}(θ_{2} − θ_{1}) > 0.

Solving (2.98) for θ̈_{1} and θ̈_{2}, we obtain

θ̈_{1} = \frac{m_{2}l_{2}^{2} (T_{1} − T_{2} + m_{2}l_{1}l_{2}θ̇_{2}^{2} sin(θ_{2} − θ_{1}) − (m_{1} + m_{2})gl_{1} sin(θ_{1})) −m_{2}l_{1}l_{2} cos(θ_{2} − θ_{1}) (T_{2} − m_{2}l_{1}l_{2}θ̇_{1}^{2} sin(θ_{2} − θ_{1}) − m_{2}gl_{2} sin(θ_{2}))}{(m_{1} + m_{2})m_{2}l_{1}^{2}l_{2}^{2} − m_{2}^{2}l_{1}^{2}l_{2}^{2} cos^{2}(θ_{2} − θ_{1})}  (2.99)

and

θ̈_{2} = \frac{−m_{2}l_{1}l_{2} cos(θ_{2} − θ_{1}) (T_{1} − T_{2} + m_{2}l_{1}l_{2}θ̇_{2}^{2} sin(θ_{2} − θ_{1}) − (m_{1} + m_{2})gl_{1} sin(θ_{1})) +(m_{1} + m_{2})l_{1}^{2} (T_{2} − m_{2}l_{1}l_{2}θ̇_{1}^{2} sin(θ_{2} − θ_{1}) − m_{2}gl_{2} sin(θ_{2}))}{(m_{1} + m_{2})m_{2}l_{1}^{2}l_{2}^{2} − m_{2}^{2}l_{1}^{2}l_{2}^{2} cos^{2}(θ_{2} − θ_{1})} .

(2.100)

The input vector is defined as

u = [ u_{1} u_{2} ]^{⊤} = [ T_{1} T_{2} ]^{⊤} .

Taking into account the above definition of the input, the definition of the state variables given by (2.97) and relations (2.99) and (2.100), we obtain the following state-space model of the double pendulum system

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ; ẋ_{4} ] = f(x, u)

= [ x_{2} \frac{m_{2}l_{2}^{2} (u_{1} − u_{2} + m_{2}l_{1}l_{2}x_{4}^{2} sin(x_{3} − x_{1}) − (m_{1} + m_{2})gl_{1} sin(x_{1})) −m_{2}l_{1}l_{2} cos(x_{3} − x_{1}) (u_{2} − m_{2}l_{1}l_{2}x_{2}^{2} sin(x_{3} − x_{1}) − m_{2}gl_{2} sin(x_{3}))}{(m_{1}+m_{2})m_{2}l_{1}^{2}l_{2}^{2}−m_{2}^{2}l_{1}^{2}l_{2}^{2} cos^{2}(x_{3}−x_{1})} x_{4} \frac{−m_{2}l_{1}l_{2} cos(x_{3} − x_{1}) (u_{1} − u_{2} + m_{2}l_{1}l_{2}x_{4}^{2} sin(x_{3} − x_{1}) − (m_{1} + m_{2})gl_{1} sin(x_{1})) +(m_{1} + m_{2})l_{1}^{2} (u_{2} − m_{2}l_{1}l_{2}x_{2}^{2} sin(x_{3} − x_{1}) − m_{2}gl_{2} sin(x_{3}))}{(m_{1}+m_{2})m_{2}l_{1}^{2}l_{2}^{2}−m_{2}^{2}l_{1}^{2}l_{2}^{2} cos^{2}(x_{3}−x_{1})} ] .

2.4.7 Population Dynamics

We first model dynamics of a single population. Then, we use the obtained model to investigate two interacting populations.

Let x = ϕ(t) be the population of the given species at a time t. The simplest model of the population dynamics of the given species is the one resulting from the hypothesis that the net rate of change of x is proportional to the current value of x. Let r be the rate of growth of the population. Then, the simple model of the population dynamics is

ẋ = rx.  (2.101)

Solving (2.101) subject to the initial condition, x(0) = x_{0}, yields

x = e^{rt}x_{0}.  (2.102)

In the above linear model, the growth rate, r, is constant. It does not account for the limitation of space, food supply, etc. Therefore, we modify the unrestricted growth rate r to take into account the fact that the environment of the population can only support a certain number, say K, of the species. The constant K is called the carrying capacity of the environment or the saturation level. Specifically, if x > K, then because there is not enough food or space to support x, more species will die than are born, which results in a negative growth rate. On the other hand, if x < K, then there is extra food and space and the population growth is positive. One function that satisfies the above requirements has the form

r (1 − \frac{x}{K} ) .

Using the above model of the growth rate, we obtain the following model of the population growth,

ẋ = r (1 − \frac{x}{K} ) x.  (2.103)

Let c = r/K. Then, we represent (2.103) as

ẋ = (r − cx) x  (2.104)

Equation (2.104) is known as the Verhulst equation or the logistic equation. According to Boyce and DiPrima [40, page 60], Verhulst, a Belgian mathematician, introduced equation (2.104) in 1838 to model human population growth. For reasons somewhat unclear, Verhulst referred to it as logistic growth, hence (2.104) is often called the logistic equation.

We use the logistic equation to analyze interacting species. We first model two interacting species that do not prey upon each other, but compete for the same resources, for example for a common food supply. Let x_{1} and x_{2} be the populations, at time t, of the two competing species. We assume that the population of each of the species in the absence of the other is governed by a logistic equation. Thus, a model of two non-interacting species has the form

\begin{matrix}ẋ_{1} & = & (r_{1} − c_{1}x_{1}) x_{1} \\ ẋ_{2} & = & (r_{2} − c_{2}x_{2}) x_{2}, \end{matrix} \}  (2.105)

where r_{1} and r_{2} are the unrestricted growth rates of the two populations, while r_{1}/c_{1} and r_{2}/c_{2} are their saturation levels. When both species are present, each will affect the growth rate of the other. Since x_{1} and x_{2} use the same resources, the growth rate of x_{1} will be lowered by an amount proportional to the size of x_{2}. We modify the growth equation for species x_{1} as

ẋ_{1} = (r_{1} − c_{1}x_{1} − a_{1}x_{2}) x_{1},  (2.106)

where a_{1} is a measure by which species x_{2} lower the growth rate of x_{1}. Similarly, the growth rate of x_{2} is reduced by a_{2}x_{1}, where a_{2} is a measure of the degree by which species x_{1} lower the growth rate of x_{2}. Therefore, the growth equation for species x_{2} becomes

ẋ_{2} = (r_{2} − c_{2}x_{2} − a_{2}x_{1}) x_{2}.  (2.107)

In summary, equations (2.106) and (2.107) model two species that interact by competing for a common resource like food or other supply.

We now analyze the situation in which one species, the predator, preys upon the other species, the prey, and there is no competition between the predator and prey for a common resource. Let x_{1} and x_{2} denote the populations of the prey and predator, respectively. We make the following modeling assumptions:

1. When there is no predator, that is, x_{2} = 0, the prey enjoys an exponential growth

governed by

ẋ_{1} = r_{1}x_{1}, r_{1} > 0.  (2.108)

2. When there is no prey, that is, x_{1} = 0, the predator perishes. We model this scenario

using the equation

ẋ_{2} = −r_{2}x_{2}, r_{2} > 0.  (2.109)

3. The larger the population of the predator, the more prey is eaten. The larger the population of the prey, the easier hunting is for the predator. We model the growth rate decrease of the prey by including in (2.108) the term −dx_{1}x_{2}, d > 0. The growth rate increase of the predator is modeled by adding the term ex_{1}x_{2} in (2.109), where e > 0.

The resulting model of the predator-prey relationship is

\begin{aligned}ẋ_{1} & = & (r_{1} − dx_{2}) x_{1} \\ ẋ_{2} & = & (−r_{2} + ex_{1}) x_{2} \end{aligned}  (2.110)

Equations (2.110) are known as the Lotka-Volterra equations. They were published by Lotka in 1925 and by Volterra in 1926.

2.4.8 The Hypothalamic-Pituitary-Adrenal Axis Model

The hypothalamic-pituitary-adrenal (HPA) axis is the body’s primary stress management system. The role of the HPA axis is to maintain bodily functions in the presence of physical and mental stressors. The (HPA) axis is a part of the endocrine system. The endocrine system as well as its subsystem, the HPA axis, uses hormones to communicate between the regions of the body. The regulation of hormones maintains homeostasis—the process by which bodily functions are maintained at a constant level. This leads to a definition of stress as a state of disharmony in which the homeostasis of the organism is threatened.

Another approach to define stress was proposed by McEwen [237] in 2002. To define stress McEwen introduced a notion of allostasis—the process by which the body functions change in response to surrounding stimuli. The term allostasis is the opposite to the notion of homeostasis. An example of allostais is the fight-or-flight response in which the sympathetic nervous system as well as the HPA axis are involved. Dysregulation of the HPA axis is associated with a number of neuroimmune disorders such as chronic fatigue syndrome (CFS), depression, Gulf War illness (GWI), or posttraumatic stress disorder (PTSD), among other stress related diseases [29]. At present, it is not clear what causes dysregulation of the HPA axis.

The HPA model presented here was developed by Gupta et al. [122] and modified by Ben-Zvi et al. [29]. A simplified schematic diagram of the HPA is shown in Figure 2.24.

The HPA axis is responsible for a rapid response to stress stimuli. An activation of the hypothalamus by a stressor causes the release of the corticotropin releasing hormone (CRH). The hypothalamus is the control center of most of the body’s hormonal systems. Upon reaching the pituitary gland, the CRH hormone induces the release of the adrenocorticotropic hormone (ACTH) by the pituitary into the circulation that reaches the adrenal glands that are located on top of the kidneys. The ACTH stimulates the secretion of cortisol by the adrenal glands. The release of cortisol initiates metabolic effects to fight the harmful effects of stress through negative feedback to the hypothalamus and pituitary—see Figure 2.24.

Figure 2.24: A simplified schematic diagram of the HPA axis.

Table 2.4: Description of variables in the HPA model.

Variable Description
x_{1}  CRH concentration
x_{2}  ACTH concentration
x_{3}  Free GR concentration
x_{4}  Cortisol concentration
d  Unknown input modeling

stress action

u  Control variable modeling

treatment action

Once the state of stress subsides, the concentration of ACTH and cortisol decreases. In their model, Gupta et al. [122] also include the dynamics of the glucocorticoid receptor (GR) to model more accurately the HPA axis dynamics.

We use the variables described in Table 2.4 in the HPA axis modeling equations.

We present the HPA model following the development of Gupta et al. [122]. In this model, stress, modeled by the signal d, stimulates the hypothalamus to secrete the CRH denoted x_{1}. The equation modeling the hypothalamus response to stress is

ẋ_{1} = (1 − \frac{x_{4}}{k_{i1}} ) (k_{c} + d) − k_{cd}x_{1},  (2.111)

where the terem −k_{cd}x_{1} models a constant degradation rate of the CRH in the blood. The term (1 − \frac{x_{4}}{k_{i1}} ) (k_{c} + d) models a circadian production and stress being reduced by the inhibition term (1 − \frac{x_{4}}{k_{i1}} ). For small values of x_{4}/k_{i1}, we may write

(1 − \frac{x_{4}}{k_{i1}} ) (k_{c} + d) ≈ \frac{k_{c} + d}{1 + \frac{x_{4}}{k_{i1}}} .

Using the above approximation, we obtain the following equation modeling the hypothalamus dynamics,

ẋ_{1} = \frac{k_{c} + d}{1 + \frac{x_{4}}{k_{i1}}} − k_{cd}x_{1}.  (2.112)

The adrenocorticotropic hormone (ACTH) produced by the pituitary gland is modeled as

ẋ_{2} = \frac{x_{1}}{1 + \frac{x_{1}x_{4}}{k_{i2}}} − k_{ad}x_{2}.  (2.113)

The dynamics of the glucocorticoid receptor (GR) is modeled by Gupta et al. [122] as

ẋ_{3} = \frac{(x_{1}x_{4})^{2}}{k + (x_{3}x_{4})^{2}} − k_{rd}x_{3}.  (2.114)

Finally, the production of the cortisol by the adrenal glands is modeled as

ẋ_{4} = x_{2} − x_{4}.

Ben-Zvi et al. [29] propose to augment the above equation with the control term u modeling the control action,

ẋ_{4} = x_{2} − x_{4} + u.  (2.115)

Combining equations (2.112), (2.113), (2.114), and (2.115), we obtain a fourth-order model of the HPA axis,

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ; ẋ_{4} ] = [ \begin{matrix}_{\frac{1}{1+ \frac{x_{4}}{k_{i1}}}} − k_{cd}x_{1} \\ _{\frac{x_{1}}{1+ \frac{x_{1}x_{4}}{k_{i2}}}} − k_{ad}x_{2} \\ \frac{(x_{1}x_{4})^{2}}{k+(x_{3}x_{4})^{2}} − k_{rd}x_{3} \\ x_{2} − x_{4} \end{matrix} ] + [ \begin{matrix}0 \\ 0 \\ 0 \\ 1 \end{matrix} ] u + [ \frac{1}{1+ \frac{x_{4}}{k_{i1}}} ; 0 ; 0 ; 0 ] d.  (2.116)

The normalized parameter values listed in Ben-Zvi et al. [29] are given in Table 2.5.

Table 2.5: Parameter values in the HPA model

Parameter Description  Value
k_{i1}  Inhibition constant for CRH synthesis 0.100
k_{cd} CRH degradation constant 1.000
k_{i2}  Inhibition constant for ACTH synthesis 0.100
k_{ad}  ACTH degradation constant  10.000
k_{c}  circadian production  1
k_{cr}  GR synthesis constant  0.050
k_{rd}  GR degradation constant  0.900
k  Inhibition constant for GR synthesis  0.001

2.5 Notes

For more examples of dynamical system models from mechanical and electrical engineering, the reader is referred to Truxal [329], Banks [21], Raven [280], Mohler [244], and D’Azzo and Houpis [73]. Wertz [349] has many excellent real-life models from space science. A wellwritten paper on vehicle modeling and control for broad audience is by Ackermann [2]. In addition to references [354] and [350], vehicle and tire dynamics modeling are investigated in detail by Nalecz [248], Nalecz and Bindemann [250, 249], and Smith and Starkey [302, 303]. For more on stepper motors, the reader is referred to Acarnley [1], Kenjo [176], Kenjo and Nagamori [177], Del Toro [78, Chapter 9], Krause and Wasynczuk [192, Chapter 8], and Zribi and Chiasson [376]. For further analysis of the equations modeling competing species and the predator-prey equations, we recommend Boyce and DiPrima [40, Sections 9.4 and 9.5]. Our derivation of the logistic equation follows that of Sandefur [293]. The logistic equation as well as equations modeling interacting species can be used to mathematical modeling of tumor dynamics and interactions between tumor and immune system. A survey of models for tumor-immune system dynamics can be found in the well-edited book by Adam and Bellomo [4].

A number of mathematical models of the hypothalamic-pituitary-adrenal (HPA) axis were proposed in the last six decades. Building on the previous models, Bingzhen, Zhenye, and Lainsong [34] proposed a third-order dynamical model and tested it on clinical data. This model was later improved by Liu et al. [219]. Kyrylov, Severyanova and Vieira [201] modified the model of Liu et al. [219] to include additional properties of the HPA axis. Conrad et al. [64] proposed and analyzed a second-order non-linear mathematical model of the HPA axis. Lenbury and Pornsawad [209] used delay-differential equations in their HPA axis model to account for the delays associated with the action of the glands in response to the stimulating hormones. Gupta et al. [122], on the other hand, proposed a fourth-order non-linear state-space model of the HPA axis in which the effect of stress on the HPA axis dynamical behavior is modeled. Ben-Zvi, Vernon, and Broderick [29] modified the model of Gupta et al. [122] by adding the control input representing the treatment. The HPA model proposed by Ben-Zvi et al. [29] can be viewed as a dynamical system with unknown input. Such systems are a subject of Chapter 12.

2.6 Exercises

Exercise 2.1 Prove the law of conservation of energy as stated in the following theorem:

Theorem 2.1 Given conservative vector field F, that is, F = −∇U, where U is the potential energy. Assume that a particle of mass m, satisfying Newton’s law, moves on a differentiable curve C. Then, the sum of the potential energy and kinetic energy is constant.

Hint Denote by C(t) a differentiable curve along which a particle of mass m moves. Hence

F (C(t)) = mC̈(t).

Then, show that

U(C(t)) + \frac{1}{2}m ∥Ċ(t)∥^{2} = constant.

It is enough to differentiate U(C(t)) + \frac{1}{2}m ∥Ċ(t)∥^{2} with respect to time and show that the time derivative is zero, hence the sum of the energies is constant.

Exercise 2.2 Show that in a conservative vector field the work done by the field on a particle that moves between two points depends only on these points and not on the path followed. Hint Note the work done against the conservative force field F along a curve C between points C(t_{1}) and C(t_{2}) is

∫_{t_{1}}^{t_{2}} F (C(t))^{⊤}Ċ(t)dt.

By assumption F is a conservative vector field. Hence,

F = −∇U.  (2.117)

Combining the above yields

∫_{t_{1}}^{t_{2}} F (C(t))^{⊤}Ċ(t)dt = − ∫_{t_{1}}^{t_{2}} ∇U(C(t))^{⊤}Ċ(t)dt.

Figure 2.25: A two-tank pumped storage system for Exercise 2.3.

Use the chain rule to represent the expression inside the integral on the right hand side as

∇U(C(t))^{⊤}Ċ(t) = \frac{dU(C(t))}{dt}.

Exercise 2.3 The goal of this exercise is to develop a model of a two-tank pumped storage system. Figure 2.25 illustrates such a system. A pumped storage system can be used to supply water to a turbine producing electric power. The two storage tanks are connected together through a valve that may be modeled as a linear resistance R_{1}. The resistance of the turbine supply pump is modeled by a linear resistance R_{2}. Suppose that we wish to design a controller that would control the inflow rate q_{i} in order to maintain the output flow rate q_{o} at some desired value q_{d}. The first stage of our project is to develop a design model of the pumped storage system. We assume that linearized flow rates q(t) and q_{o}(t) are proportional to the corresponding water heights, that is,

q(t) = \frac{h_{1}(t) − h_{2}(t)}{R_{1}},
q_{o}(t) = \frac{h_{2}(t)}{R_{2}}.

Let A_{1} and A_{2} represent the surface areas in the two tanks. Then, the principle of conser-Table 2.6: Numerical data for the two-tank pumped storage system model for Exercise 2.3.

Tank 1 diameter 2 m
Tank 2 diameter 3 m
R_{1}  10 sec/m^{2}
R_{2}  20 sec/m^{2}

vation of mass for an incompressible fluid yields

q_{i}(t) − q(t) = A_{1}ḣ_{1}(t),
q(t) − q_{o}(t) = A_{2}ḣ_{2}(t).

Numerical data of a two-tank pumped storage system is given in Table 2.6. Find the transfer function L(q_{o}(t))/L(q_{i}(t)).

Exercise 2.4 The objective of this exercise is to derive a model of an automotive suspension. The role of an automotive suspension is to support the vehicle body on the axles. A simple linear model of the suspension assembly associated with a single wheel, called quarter-car model, is shown in Figure 2.26. The vehicle body is represented by the “sprung” mass, m_{s}, while the tire and axle by the “unsprung” mass, m_{u}. Model the tire as a spring with spring coefficient k_{t}, and assume its damping to be negligible. The suspension consists of the spring, shock absorber, and the variable force element. The role of the variable force element is to generate a force that compensates for the uneven road which is the source of the system disturbances. Assume that the variable force element F_{a} can instantaneously provide any desired force. The above described model of an automotive suspension is referred to as a linear active suspension. Write down equations modeling the dynamics of the two degree of freedom system depicted in Figure 2.26 by applying Newton’s law to each mass of the system, and taking into account the positive direction of the z coordinates. Define the state variables to be:

x_{1} = z_{s} − z_{u} = suspension stroke,
x_{2} = ż_{s} = sprung mass velocity,
x_{3} = z_{u} − z_{r} = tire deflection,
x_{4} = ż_{u} = unsprung mass velocity.

Let u = F_{a} be the system input, and let d(t) = ż_{r}(t) be the disturbance input due to the road roughness. Represent the obtained linear model of active suspension in state-space

Figure 2.26: Linear quarter-car model for Exercise 2.4.

format. Use data of a commercial vehicle given in Table 2.7. The data comes from Sunwoo and Cheok [314]. Substitute the vehicle data into the modeling equations. A design goal of an automotive engineer may be to design a control law to isolate the sprung mass from road disturbances. In other words, he may wish to design a control law such that the vertical acceleration of the sprung mass z̈_{s}(t) = ẋ_{2} is as close to 0 as possible in spite of road disturbances. Assume that we can measure the vertical acceleration of the sprung mass. Write the output equation.

Exercise 2.5 The objective of this exercise is to develop a model of a single axis machine

Table 2.7: Parameters of the vehicle suspension of Exercise 2.4.

m_{s}  355 kg
m_{u}  46 kg
k_{s} 14384 N/m
b_{s} 1860 N/m/sec
k_{t} 300000 N/m

Figure 2.27: A schematic of a machine tool drive system for Exercise 2.5.

tool drive in which a motor drives a lead screw through a gear. Figure 2.27 illustrates such a system. This model is adapted from Driels [87, pages 613–619]. The system operates as follows. The lead screw moves the table with a workpiece on it under the cutter to perform the desired operation. The table is positioned by a control system. We wish to place the table in a desirable position as rapidly and accurately as possible. This means, that while modeling the machine tool drive system we should, in particular, take into account the flexibility of the lead screw. The length of the lead screw implies that its rotation at the gear box will not be the same as its rotation at the table. Thus, the angular twist along the length of the lead screw cannot be neglected. Model the lead screw flexibility using a torsional spring as illustrated in Figure 2.28. The system input is the motor torque τ_{m}, and the system output is the rotation of the end of the lead screw labeled by θ_{l}. Assume that the damping torque, due to viscous friction, acting on the motor is proportional to its angular velocity θ̇_{m} = ω_{m} with the coefficient of viscous friction denoted c_{m}. Denote the motor moment of inertia by I_{m}, and by τ_{1} the load torque on the motor gear due to the rest of the gear train. Write the equation modeling the dynamics of the motor shaft.

114 CHAPTER 2. DYNAMICAL SYSTEM MODELING Figure 2.28: A graphical model of the machine tool drive system for Exercise 2.5.

Table 2.8: Data for the machine tool drive system model for Exercise 2.5.

I_{m}  0.0001 Nm·s^{2}/rad
I_{l}  0.001 Nm·s^{2}/rad
c_{m} = c_{l} 0.01 Nm·s/rad
k  1 Nm/rad
N  1/5

Table 2.9: Vehicle data for Exercise 2.6.

l_{1} = l_{2}  1 m
v_{x}  10 m/sec
m  1,000 kg
I_{z}  1,000 kg·m^{2}
C_{f} = C_{r} 10,000 N/rad

Let τ_{2} be the torque transmitted to the load shaft, and let N = N_{1}/N_{2} be the gear ratio, where N_{1} denotes the number of teeth of the first gear, and respectively N_{2} is the number of teeth of the second gear. The work done by the first gear is equal to that of the second. Model the load shaft flexibility with a torsional spring. The torsional spring coefficient is k. Assume that the moment of inertia of the lead screw is negligible compared to that of the load. Denote the moment of inertia of the load by I_{l}. The torque due to viscous friction of the load is c_{l}ω_{l}, where c_{l} is the coefficient of the viscous friction of the load. The angular displacement of the end of the lead screw is θ_{l}. Write the equation modeling the dynamics of the load. Define the state variables to be

x_{1} = θ_{m}, x_{2} = θ̇, x_{3} = θ_{l}, x_{4} = θ̇_{l}.

Represent the modeling equations in state space format. Typical numerical data for a machine tool drive system are shown in Table 2.8.

Exercise 2.6 Consider a bicycle model of a front wheel steering vehicle shown in Figure 2.29. Construct a two-dimensional state-space model of the vehicle, where x_{1} = ẏ and x_{2} = θ̇. The input u = δ_{f} and the output y = x_{2}. In your derivation assume small angles. Vehicle data are given in Table 2.9.

116 CHAPTER 2. DYNAMICAL SYSTEM MODELING Figure 2.29: Bicycle model of a vehicle for Exercise 2.6.

2.6. EXERCISES 117 Figure 2.30: The two-mass-spring system for Exercise 2.7.

Figure 2.31: Mechanical system for Exercise 2.8.

Exercise 2.7 Consider the two-mass-spring system depicted in Figure 2.30.

(i) Form the Lagrangian of the system. (ii) Write the Lagrange equations of motion for the system. (iii) Represent the obtained Lagrange equations of motion in state-space format. Exercise 2.8 Consider the mechanical system shown in Figure 2.31.

(i) Form the Lagrangian for the system. (ii) Write the Lagrange equation of motion for the system. (iii) Represent the system model in state space format. Exercise 2.9 Consider a dynamical system that consists of a cart with an inverted pendulum (point mass on a massless shaft) attached to it as depicted in Figure 2.32.

(i) Write the Lagrange equations of motion for the system,

Figure 2.32: An inverted pendulum moving on a cart for Exercise 2.9.

(ii) Represent the obtained model in state-space format using the state variables

x_{1} = x, x_{2} = ẋ, x_{3} = θ, x_{4} = θ̇.

Exercise 2.10 Consider a schematic of a field-controlled DC motor shown in Figure 2.33. Let x_{1} = θ, x_{2} = θ̇, x_{3} = i_{f}, and y = x_{1}. Treat the field inductance L_{f} as a parasitic element and denote it μ.

(i) Form the actual plant model P_{μ} in state space format;

(ii) Find the reduced-order design model P_{0}.

120 CHAPTER 2. DYNAMICAL SYSTEM MODELING Figure 2.33: A diagram of a field-controlled DC motor for Exercise 2.10.

Analysis of Modeling Equations

Folk wisdom says that if you leave a cold alone, it lasts seven days; if you treat it, it will go away in a week. Nonetheless, people have sworn by such “cures” as this: hang your hat on the bedpost, drink gin until you see two hats, then pull the covers up until the siege is over. [281, page 52]

Most of dynamical systems analyzed analyzed by us in this book are modeled by ordinary differential equations. A main use of a mathematical model is to predict the system transient behavior. Unlike linear systems, where closed-form solutions can be written in terms of the system’s eigenvalues and eigenvectors, finding analytical, exact, solutions to nonlinear differential equations can be very difficult or impossible. However, we can approximate the solutions of differential equations with difference equations whose solutions can be obtained easily using a computer. In this chapter, after discussing methods for solving state equations modeling linear systems, we present a number of methods for solving a more general class of differential equations. The first class of methods allows us to graphically determine solutions to second-order differential equations. Then, we discuss numerical techniques for solving differential equations.

Linear systems can be modeled using convolutions and in the case of lumped systems, we can use state-space equations. Analytical computation of convolutions can be quite involved. An efficient method of obtaining a solution to a model of a lumped system is to convert the transfer function into a state-space model and then solve the state equation. We begin with a time-invariant case and then proceed with the time-varying case.

3.1 State Equation Solution of Uncontrolled System

Given a dynamical system’s time-invariant model of the form

ẋ(t) = Ax(t)  (3.1)

subject to an initial condition

x(0) = x_{0},  (3.2)

where A ∈ ℝ^{n×n}. The solution to (3.1) subject to (3.2) is

x(t) = e^{At}x_{0}  (3.3)

The above is a generalization of the situation when A is a scalar. The exponential matrix e^{At} is defined as

e^{At} = I_{n} + tA + \frac{t^{2}}{2!}A^{2} + \frac{t^{3}}{3!}A^{3} + ⋯

To verify that (3.3) solves (3.1) subject to (3.2), we use the fact that

\frac{d}{dt}e^{At} = Ae^{At} = e^{At}A.

Suppose that the initial condition x(0) = x_{0} is replaced with a more general condition

x(t_{0}) = x_{0},

where t_{0} is a given initial time instant. Then, the solution to ẋ(t) = Ax(t) subject to x(t_{0}) = x_{0} is

x(t) = e^{A^{(t−t_{0})}}x_{0}

The matrix e^{A^{(t−t_{0})}} is often written as

e^{A^{(t−t_{0})}} = Φ (t, t_{0})

and is called the state transition matrix, because it relates the state at any instant of time t_{0} to the state at any other time t.

3.2 Computing State Transition Matrix of LTI Systems

For an overview of methods to calculate the state transition matrix for a linear, timeinvariant (LTI) system, we refer the reader to the classic paper on the subject by Moler and Van Loan [245], where the numerical issues related to the computation of the matrix exponential are discussed. In what follows, we present a few simple methods to compute the matrix exponential.

3.2.1 The Inverse Laplace Transform Method

In the case when the matrix A is of low dimensions, we can use the formula that results from applying the Laplace transform to the time-invariant system ẋ = Ax, x(0) = x_{0}. The Laplace transform of the above matrix differential equation is

sX(s) − x_{0} = AX(s).

Hence

X(s) = (sI_{n} − A)^{−1} x_{0},

and the inverse transform yields

e^{At} = L^{−1} {(sI_{n} − A)^{−1}}  (3.4)

where L^{−1} denotes the inverse Laplace transform operator.

Example 3.1 We use the inverse Laplace method to compute e^{At} for

A = [ \begin{matrix}0 & 3 \\ 2 & 1 \end{matrix} ] .

We have

e^{At} = L^{−1} ([sI_{2} − A]^{−1})
= L^{−1} ([ \begin{matrix}s & −3 \\ −2 & s − 1 \end{matrix} ]^{−1})
= L^{−1} (\frac{1}{(s − 3)(s + 2)} [ \begin{matrix}s − 1 & 3 \\ 2 & s \end{matrix} ])
= [ \begin{matrix}0.4e^{3t} + 0.6e^{−2t} & 0.6 (e^{3t} − e^{−2t}) \\ 0.4 (e^{3t} − e^{−2t}) & 0.6e^{3t} + 0.4e^{−2t} \end{matrix} ] .  (3.5)

3.2.2 Diagonalization Method

A convenient way to calculate e^{At} is when the matrix A has distinct eigenvalues. In such a case the n-by-n matrix A has n linearly independent eigenvectors as per the following theorem.

Theorem 3.1 Suppose the characteristic equation,

Δ(λ) = det[λI_{n} − A]
= λ^{n} + a_{n−1}λ^{n−1} + ⋯ + a_{1}λ + a_{0}
= 0,

has n distinct roots, λ_{1}, λ_{2}, …, λ_{n}. Then, there exist n linearly independent vectors v_{1}, v_{2}, …, v_{n} such that

Av_{i} = λ_{i}v_{i}, i = 1, 2, …, n.

Proof See Exercise 3.2.

□

Let

T = [ v_{1} v_{2} ⋯ v_{n} ] .

The matrix T is invertible because the eigenvectors v_{i}, i = 1, 2, …, n are linearly independent. Then, because T^{−1}T = I_{n}, we have

T^{−1}AT = T^{−1}A [ v_{1} v_{2} ⋯ v_{n} ]
= T^{−1} [ Av_{1} Av_{2} ⋯ Av_{n} ]
= T^{−1} [ λ_{1}v_{1} λ_{2}v_{2} ⋯ λ_{n}v_{n} ]

= T^{−1}T [ \begin{matrix}λ_{1} & 0 & 0 & ⋯ & 0 \\ 0 & λ_{2} & 0 & ⋯ & 0 \\ ⋮ & ⋮ & & ... & ⋮ \\ 0 & 0 & 0 & ⋯ & λ_{n} \end{matrix} ]

= Λ.  (3.6)

We use the above result to compute the transition matrix e^{At}. Recall that

e^{At} = I_{n} + At + \frac{A^{2}t^{2}}{2!} + \frac{A^{3}t^{3}}{3!} + ⋯

Note that

Λ^{k} = [ \begin{matrix}λ_{1}^{k} & 0 & 0 & ⋯ & 0 \\ 0 & λ_{2}^{k} & 0 & ⋯ & 0 \\ ⋮ & ⋮ & & ... & ⋮ \\ 0 & 0 & 0 & ⋯ & λ_{n}^{k} \end{matrix} ] , k ≥ 1

Hence

e^{Λt} = [ \begin{matrix}e^{λ_{1}t} & 0 & 0 & ⋯ & 0 \\ 0 & e^{λ_{2}t} & 0 & ⋯ & 0 \\ ⋮ & ⋮ & & ... & ⋮ \\ 0 & 0 & 0 & ⋯ & e^{λ_{n}t} \end{matrix} ] .

From (3.6), we obtain

A = T ΛT^{−1}.

Therefore,

A^{k} = (T ΛT^{−1})^{k}
= (T ΛT^{−1}) (T ΛT^{−1}) × ⋯ × (T ΛT^{−1})
= T Λ^{k}T^{−1},

and hence

e^{At} = e^{T ΛT^{−1}}
= I_{n} + T ΛT^{−1}t + \frac{(T ΛT^{−1})^{2} t^{2}}{2!} + \frac{(T ΛT^{−1})^{3} t^{3}}{3!} + ⋯
= T (I_{n} + Λt + \frac{Λ^{2}t^{2}}{2!} + \frac{Λ^{3}t^{3}}{3!} ) T^{−1}
= T e^{Λt}T^{−1}.  (3.7)

Example 3.2 We use the diagonalization method to compute e^{At} for

A = [ \begin{matrix}0 & 3 \\ 2 & 1 \end{matrix} ] .

Note that this is the same matrix A as in Example 3.1. The eigenvalues of A are distinct so the diagonalization method is applicable here. We first compute an eigenvector, v_{1}, corresponding to the eigenvalue λ_{1} = 3 by solving the eigenvalue-eigenvector equation,

(A − λ_{1}I_{2}) v_{1} = 0.

That is,

(A − λ_{1}I_{2}) v_{1} = [ \begin{matrix}−3 & 3 \\ 2 & −2 \end{matrix} ] [ \begin{matrix}v_{11} \\ v_{21} \end{matrix} ] = [ \begin{matrix}0 \\ 0 \end{matrix} ] .

We take

v_{1} = [ \begin{matrix}1 \\ 1 \end{matrix} ] .

We next compute an eigenvector corresponding to the eigenvalue λ_{2} = −2 by solving the equation

(A − λ_{2}I_{2}) v_{2} = [ \begin{matrix}2 & 3 \\ 2 & 3 \end{matrix} ] [ \begin{matrix}v_{12} \\ v_{22} \end{matrix} ] = [ \begin{matrix}0 \\ 0 \end{matrix} ] .

We take

v_{2} = [ \begin{matrix}−3 \\ 2 \end{matrix} ] .

We use the above eigenvectors to form the transformation matrix

T = [ v_{1} v_{2} ] = [ \begin{matrix}1 & −3 \\ 1 & 2 \end{matrix} ] .

Its inverse is

T^{−1} = \frac{1}{5} [ \begin{matrix}2 & 3 \\ −1 & 1 \end{matrix} ] .

We are now ready to compute e^{At} using (3.7),

e^{At} = T e^{Λt}T^{−1}
= \frac{1}{5} [ \begin{matrix}1 & −3 \\ 1 & 2 \end{matrix} ] [ \begin{matrix}e^{3t} & 0 \\ 0 & e^{−2t} \end{matrix} ] [ \begin{matrix}2 & 3 \\ −1 & 1 \end{matrix} ]
= \frac{1}{5} [ \begin{matrix}2e^{3t} + 3e^{−2t} & 3 (e^{3t} − e^{−2t}) \\ 2 (e^{3t} − e^{−2t}) & 3e^{3t} + 2e^{−2t} \end{matrix} ] .

The result is the same as when we used the inverse Laplace transform method.

The above example illustrates that the transformation T producing the diagonal form of A is a powerful computational tool. The existence of a diagonalizing T is not always guaranteed. The inability to transform A to diagonal form is connected with the existence of repeated eigenvalues that we discuss next.

3.3 Computing Functions of a Square Matrix

In this section, we discuss methods of computing functions of a square matrix that is not necessarily diagonalizable. The methods discussed allow us to easily compute the matrix exponential and hence the state transition matrix of LTI systems. A major tool used by the methods discussed is the so called Jordan canonical form that we discuss next.

3.3.1 The Jordan Canonical Form

If a square n-by-n matrix A has repeated eigenvalues, then it may not be diagonalizable. Although some matrices possessing repeated eigenvalues can be transformed into diagonal form, in many cases it is only possible to obtain the so-called “Jordan canonical form” or just the “Jordan from,” for short. To illustrate the point, we consider the case when the characteristic equation, Δ(λ) = (λ−λ_{1})^{n}, that is, the matrix A has just one eigenvalue with multiplicity n. For simplicity, we analyze the case when n = 4 and suppose that

rank [ A − λ_{1}I_{4} ] = 3.

That is, the nullity of the matrix [ A − λ_{1}I_{4} ] is one, which means that there is only one eigenvector corresponding to the eigenvalue λ_{1}. Thus such a matrix A cannot be diagonalized. We need three more linearly independent vectors to represent A in a new basis, where it has a particularly simple form that is as close as possible to being diagonal.

To proceed, we need a notion of a generalized eigenvector that we define next.

Definition 3.1 A vector v is called a generalized eigenvector of grade k if

(A − λI_{n})^{k} v = 0

and

(A − λI_{n})^{k−1} v ≠ 0.
◇

For k = 1, we have

(A − λI_{n}) v = 0,

which means that v is an ordinary eigenvector corresponding to the eigenvalue λ.

Suppose now that A ∈ ℝ^{4×4} and v is a generalized eigenvector of grade n = 4 corresponding to the repeated eigenvalue λ_{1} whose multiplicity is also 4. Then by Definition 3.1,

(A − λ_{1}I_{4})^{4} v = 0 but (A − λ_{1}I_{4})^{3} v ≠ 0.

Let

v_{1} = (A − λ_{1}I_{4})^{3} v.

Then,

(A − λ_{1}I_{4}) v_{1} = (A − λ_{1}I_{4})^{4} v
= 0.

Hence

Av_{1} = λ_{1}v_{1}.  (3.8)

We also have to have

(A − λ_{1}I_{4})^{2} v ≠ 0,

otherwise we would have (A − λ_{1}I_{4})^{3} v = 0 as well, which would mean that v is not a generalized eigenvector of grade 4.

Let

v_{2} = (A − λ_{1}I_{4})^{2} v.

Note that

(A − λ_{1}I_{4}) v_{2} = (A − λ_{1}I_{4})^{3} v
= v_{1}.

Therefore,

Av_{2} = v_{1} + λ_{1}v_{2}.  (3.9)

We proceed in a similar way by letting

v_{3} = (A − λ_{1}I_{4}) v.

We observer that

(A − λ_{1}I_{4}) v_{3} = (A − λ_{1}I_{4})^{2} v
= v_{2}.

Hence,

Av_{3} = v_{2} + λ_{1}v_{3}.  (3.10)

Finally, we let

v_{4} = v

and we note that

(A − λ_{1}I_{4}) v_{4} = (A − λ_{1}I_{4}) v
= v_{3}.

Therefore,

Av_{4} = v_{3} + λ_{1}v_{4}.  (3.11)

Combining (3.8)–(3.11), we obtain

A [ v_{1} v_{2} v_{3} v_{4} ] = [ λ_{1}v_{1} v_{1} + λ_{1}v_{2} v_{2} + λ_{1}v_{3} v_{3} + λ_{1}v_{4} ]

= [ v_{1} v_{2} v_{3} v_{4} ] [ \begin{matrix}λ_{1} & 1 & 0 & 0 \\ 0 & λ_{1} & 1 & 0 \\ 0 & 0 & λ_{1} & 1 \\ 0 & 0 & 0 & λ_{1} \end{matrix} ] .  (3.12)

Let

T = [ v_{1} v_{2} v_{3} v_{4} ] .

The matrix T is invertible by construction, that is, the set of vectors {v_{1}, v_{2}, v_{3}, v_{4}} forms a basis for ℝ^{4}. Therefore we can represent (3.12) as

T^{−1}AT = [ \begin{matrix}λ_{1} & 1 & 0 & 0 \\ 0 & λ_{1} & 1 & 0 \\ 0 & 0 & λ_{1} & 1 \\ 0 & 0 & 0 & λ_{1} \end{matrix} ]

= J.  (3.13)

The representation of A in the basis {v_{1}, v_{2}, v_{3}, v_{4}} has the form of a Jordan block of order 4.

Example 3.3 Consider the matrix

A = [ \begin{matrix}5 & 2 & 4 \\ 0 & 5 & 3 \\ 0 & 0 & 5 \end{matrix} ] .

The characteristic polynomial is Δ(λ) = (λ − 5)^{3} and

rank [ A − λI_{3} ]|_{λ=5} = 2,

130 CHAPTER 3. ANALYSIS OF MODELING EQUATIONS that is, the nullity of the characteristic matrix, [ A − λI_{3} ]|_{λ=5}, is 1. We proceed to find v_{1} such that

[A − λI_{3}]|_{λ=5} v_{1} = [ \begin{matrix}0 & 2 & 4 \\ 0 & 0 & 3 \\ 0 & 0 & 0 \end{matrix} ] [ \begin{matrix}v_{11} \\ v_{21} \\ v_{31} \end{matrix} ]
= 0.

We take

v_{1} = [ \begin{matrix}1 \\ 0 \\ 0 \end{matrix} ] .

Then we proceed to find v_{2} such that

[A − λI_{3}]|_{λ=5} v_{2} = v_{1},

that is, we solve for v_{2} satisfying

[ \begin{matrix}0 & 2 & 4 \\ 0 & 0 & 3 \\ 0 & 0 & 0 \end{matrix} ] [ \begin{matrix}v_{12} \\ v_{22} \\ v_{32} \end{matrix} ] = [ \begin{matrix}1 \\ 0 \\ 0 \end{matrix} ] .

We can take

v_{2} = [ \begin{matrix}0 \\ 1/2 \\ 0 \end{matrix} ] .

Finally we find v_{3} such that

[A − λI_{3}]|_{λ=5} v_{3} = v_{2},

that is, we solve for v_{3} satisfying

[ \begin{matrix}0 & 2 & 4 \\ 0 & 0 & 3 \\ 0 & 0 & 0 \end{matrix} ] [ \begin{matrix}v_{13} \\ v_{23} \\ v_{33} \end{matrix} ] = [ \begin{matrix}0 \\ 1/2 \\ 0 \end{matrix} ] .

We take

v_{3} = [ \begin{matrix}0 \\ −2/6 \\ 1/6 \end{matrix} ] .

The transformation matrix T has the form

T = [ v_{1} v_{2} v_{3} ] = [ \begin{matrix}1 & 0 & 0 \\ 0 & 1/2 & −2/6 \\ 0 & 0 & 1/6 \end{matrix} ] ,

and

T^{−1} = [ \begin{matrix}1 & 0 & 0 \\ 0 & 2 & 4 \\ 0 & 0 & 6 \end{matrix} ] .

The matrix A in the new basis has the form

T^{−1}AT = [ \begin{matrix}1 & 0 & 0 \\ 0 & 2 & 4 \\ 0 & 0 & 6 \end{matrix} ] [ \begin{matrix}5 & 2 & 4 \\ 0 & 5 & 3 \\ 0 & 0 & 5 \end{matrix} ] [ \begin{matrix}1 & 0 & 0 \\ 0 & 1/2 & −2/6 \\ 0 & 0 & 1/6 \end{matrix} ]

= [ \begin{matrix}5 & 1 & 0 \\ 0 & 5 & 1 \\ 0 & 0 & 5 \end{matrix} ] ,

which is a third order Jordan block.

If the rank of the characteristic matrix evaluated at λ_{1} is n − 2, then the equation

[A − λ_{1}I_{n}] v = 0,

has two linearly independent solutions. Therefore, A has two linearly independent eigenvectors. Let u_{1} and v_{1} be linearly independent eigenvectors associated with the eigenvalue λ_{i}. In order to transform A into its Jordan form, we need to find n − 2 generalized eigenvectors. In this case we have two chains of generalized eigenvectors, {u_{1}, u_{2}, …, u_{k}} and {v_{1}, u_{2}, …, v_{l}}, where k + l = n. In the basis

{u_{1}, u_{2}, …, u_{k}, v_{1}, u_{2}, …, v_{l}}

the matrix A will have a block diagonal form

T^{−1}AT = [ \begin{matrix}J_{1} & O \\ O & J_{2} \end{matrix} ] ,

where J_{1} is a Jordan block of order k and J_{2} is a Jordan block of order l.

Example 3.4 Let

A = [ \begin{matrix}3 & 1 & −1 \\ 0 & 6 & −3 \\ 0 & 3 & 0 \end{matrix} ] .

The characteristic polynomial

Δ(λ) = (λ − 3)^{3}

and

rank [A − λI_{3}]|_{λ=3} = [ \begin{matrix}0 & 1 & −1 \\ 0 & 3 & −3 \\ 0 & 3 & −3 \end{matrix} ]
= 1

As two linearly independent solutions to

[A − λI_{3}]|_{λ=3} v = 0,

we can take

u_{1} = [ \begin{matrix}1 \\ 3 \\ 3 \end{matrix} ] and v_{1} = [ \begin{matrix}0 \\ 1 \\ 1 \end{matrix} ] .

Now we have to find one generalized eigenvector that together with u_{1} and v_{1} would form a basis for ℝ^{3} and enable us to construct a transformation that brings A into its Jordan form. We try first u_{1} to construct u_{2} such that

[A − λI_{3}]|_{λ=3} u_{2} = u_{1},

that is, we solve for u_{2} satisfying

[ \begin{matrix}0 & 1 & −1 \\ 0 & 3 & −3 \\ 0 & 3 & −3 \end{matrix} ] [ \begin{matrix}u_{12} \\ u_{22} \\ u_{32} \end{matrix} ] = [ \begin{matrix}1 \\ 3 \\ 3 \end{matrix} ] .

We can take

u_{2} = [ \begin{matrix}1 \\ 1 \\ 0 \end{matrix} ] .

3.3. COMPUTING FUNCTIONS OF A SQUARE MATRIX 133 We form the transformation matrix

T = [ u_{1} u_{2} v_{1} ] = [ \begin{matrix}1 & 1 & 0 \\ 3 & 1 & 1 \\ 3 & 0 & 1 \end{matrix} ] .

Its inverse is

T^{−1} = [ \begin{matrix}1 & −1 & 1 \\ 0 & 1 & −1 \\ −3 & 3 & −2 \end{matrix} ] .

It is then easy to check that

T^{−1}AT = [ \begin{matrix}J_{1} & O \\ O & J_{2} \end{matrix} ] = [ \begin{matrix}3 & 1 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 3 \end{matrix} ] ,

where

J_{1} = [ \begin{matrix}3 & 1 \\ 0 & 3 \end{matrix} ] and J_{2} = 3.

In general, the Jordan canonical form is a block diagonal matrix, where each diagonal block is a Jordan block corresponding to a possibly different eigenvalue, that is,

T^{−1}AT = [ \begin{matrix}λ_{1} & 1 & 0 & ⋯ & 0 \\ 0 & λ_{1} & 1 & ⋯ & 0 \\ 0 & 0 & λ_{1} & ⋯ & 0 \\ ⋮ & ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & 0 & ⋯ & λ_{1} \\ & & & & & . & . & . \\ & & & & & & & & λ_{r} & 1 & 0 & ⋯ & 0 \\ & & & & & & & & 0 & λ_{r} & 1 & ⋯ & 0 \\ & & & & & & & & 0 & 0 & λ_{r} & ⋯ & 0 \\ & & & & & & & & ⋮ & ⋮ & ⋮ & ... & ⋮ \\ & & & & & & & & 0 & 0 & 0 & ⋯ & λ_{r} \end{matrix} ]

= [ \begin{matrix}J_{1} & O & ⋯ & O \\ O & J_{2} & & O \\ ⋮ & & ... & ⋮ \\ O & O & ⋯ & J_{q} \end{matrix} ] ,  (3.14)

where each r_{i}×r_{i} Jordan block has the form J_{i} = λ_{i} if r_{i} = 1, or, for r_{i} > 1 the corresponding Jordan block has a non-diagonal structure. Note that

n = r_{1} + r_{2} + ⋯ + r_{q}

and

q ≥ r,

which means that there may be more than one Jordan blocks associated with one eigenvalue as was the case in Example 3.4. Example 3.5 To illustrate the general case depicted in (3.14), we consider, as in Chen [55, p. 60], a 5-by-5 matrix A with repeated eigenvalue λ_{1} of multiplicity 4 and a simple eigenvalue λ_{2}, that is, the characteristic polynomial of A is

Δ(λ) = (λ − λ_{1})^{4}(λ − λ_{2}).

Depending on the nullity of the matrix [A − λ_{1}I_{5}], the Jordan form of the matrix A make take one of the following forms:

• If the nullity of [A − λ_{1}I_{5}] is 1, then there is only one eigenvector, say u_{1} associated with λ_{1} and we need to compute three generalized eigenvectors to form a basis {u_{1}, u_{2}, u_{3}, u_{4}, v_{1}}, where v_{1} is an eigenvector associated with λ_{2}. The matrix A in this basis will have the form:

Â_{1} = T _{1}^{−1}AT_{1} = [ \begin{matrix}λ_{1} & 1 & 0 & 0 & ⋮ & 0 \\ 0 & λ_{1} & 1 & 0 & ⋮ & 0 \\ 0 & 0 & λ_{1} & 1 & ⋮ & 0 \\ 0 & 0 & 0 & λ_{1} & ⋮ & 0 \\ ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & 0 & 0 & ⋮ & λ_{2} \end{matrix} ] .

Note that we can re-order the basis vectors as {v_{1}, u_{1}, u_{1}, u_{3}, u_{4}}. In this basis the matrix A representation has the form

Â_{2} = T _{2}^{−1}AT_{2} = [ \begin{matrix}λ_{2} & ⋮ & 0 & 0 & 0 & 0 \\ ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & ⋮ & λ_{1} & 1 & 0 & 0 \\ 0 & ⋮ & 0 & λ_{1} & 1 & 0 \\ 0 & ⋮ & 0 & 0 & λ_{1} & 1 \\ 0 & ⋮ & 0 & 0 & 0 & λ_{1} \end{matrix} ] .

• If the nullity of [A − λ_{1}I_{5}] is 2, then there are two eigenvectors associated with λ_{1}, that is, there are two Jordan blocks associated with λ_{1}. The matrix A may take one of the following forms:

Â_{3} = T _{3}^{−1}AT_{3} = [ \begin{matrix}λ_{1} & 1 & 0 & ⋮ & 0 & & 0 \\ 0 & λ_{1} & 1 & ⋮ & 0 & & 0 \\ 0 & 0 & λ_{1} & ⋮ & 0 & & 0 \\ ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & 0 & ⋮ & λ_{1} & ⋮ & 0 \\ & & & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & 0 & & 0 & ⋮ & λ_{2} \end{matrix} ]

or

Â_{4} = T _{4}^{−1}AT_{4} = [ \begin{matrix}λ_{1} & 1 & ⋮ & 0 & 0 & & 0 \\ 0 & λ_{1} & ⋮ & 0 & 0 & & 0 \\ ⋯ & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & ⋮ & λ_{1} & 1 & ⋮ & 0 \\ 0 & 0 & ⋮ & 0 & λ_{1} & ⋮ & 0 \\ & & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & & 0 & 0 & ⋮ & λ_{2} \end{matrix} ] .

• If the nullity of [A − λ_{1}I_{5}] is 3, then there are three eigenvectors associated with λ_{1}, that is, there are three Jordan blocks associated with λ_{1}. The matrix A may take the following form:

Â_{5} = T _{5}^{−1}AT_{5} = [ \begin{matrix}λ_{1} & 1 & ⋮ & 0 & & 0 & & 0 \\ 0 & λ_{1} & ⋮ & 0 & & 0 & & 0 \\ ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & ⋮ & λ_{1} & ⋮ & 0 & & 0 \\ & & ⋯ & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & & 0 & ⋮ & λ_{1} & ⋮ & 0 \\ & & & & ⋯ & ⋯ & ⋯ & ⋯ \\ 0 & 0 & & 0 & & 0 & ⋮ & λ_{2} \end{matrix} ] .

• Finally, if the nullity of [A − λ_{1}I_{5}] is 4, then there are four eigenvectors associated with λ_{1}, that is, there are four Jordan blocks associated with λ_{1}. The Jordan form of the matrix matrix A is a diagonal matrix,

Â_{6} = T _{6}^{−1}AT_{6} = [ \begin{matrix}λ_{1} & 0 & 0 & 0 & 0 \\ 0 & λ_{1} & 0 & 0 & 0 \\ 0 & 0 & λ_{1} & 0 & 0 \\ 0 & 0 & 0 & λ_{1} & 0 \\ 0 & 0 & 0 & 0 & λ_{2} \end{matrix} ] .

To summarize our discussion, we need the following definitions.

Definition 3.2 For a given eigenvalue λ_{i} of an n-by-n matrix A, its algebraic multiplicity is the multiplicity of it in the characteristics polynomial Δ(λ), that is, the algebraic multiplicity of λ_{i} is the sum of orders of all Jordan blocks associated with λ_{i}.

◇

Definition 3.3 For a given eigenvalue λ_{i} of an n-by-n matrix A, its geometric multiplicity is the nullity of the associated characteristic matrix [ A − λ_{i}I_{n} ], that is, the geometric multiplicity of λ_{i} equals the number of Jordan blocks associated with it.

◇

Then we have the following theorem:

Theorem 3.2 A given n-by-n matrix A is diagonalizable if and only if geometric multiplicities of its eigenvalues are the same as their algebraic multiplicities, that is, if and only if A has n linearly independent eigenvectors.

We close our discussion on the Jordan canonical form with an observation by Owens [260, p. 34] that in almost all situations of practical relevance, the matrix A has distinct eigenvalues. So in a sense the cases described in this section can be considered “pathological” from the applications point of view. In the literature, see for example [55, p. 61], if A cannot be diagonalized then it is called to be defective. Calculation of the Jordan canonical is numerically ill conditioned. One can try MATLAB’s function jordan. However, the matrix A must be known exactly, which in MATLAB’s language means that its elements must be integers or ratios of small integers. Finally, to diagonalize a square matrix using MATLAB, one can call the function eig. If, however, A is not diagonalizable, then the function eig will give incorrect solution.

3.3. COMPUTING FUNCTIONS OF A SQUARE MATRIX 137

3.3.2 Computing Polynomial Functions of a Square Matrix

If A is an n-by-n square matrix then its k-th power is

A^{k} = \underbrace{AA × ⋯ × A}_{k times},

where

A^{0} = I_{n}.

Suppose now that we have a polynomial

f(λ) = λ^{n} + α_{n−1}λ^{n−1} + ⋯ + α_{1}λ + α_{0}.

Then we define f(A) as

f(A) = A^{n} + α_{n−1}A^{n−1} + ⋯ + α_{1}A + α_{0}I_{n}.

For example, if

f(λ) = λ^{2} + 2λ − 3 = (λ + 3)(λ − 1),

then

f(A) = A^{2} + 2A − 3I_{n} = (A + 3I_{n})(A − I_{n}).

Now if the matrix A is block diagonal, for example, such as

A = [ \begin{matrix}A_{1} & O & O \\ O & A_{2} & O \\ O & O & A_{3} \end{matrix} ] ,

where A_{i} are square matrices and O denotes a matrix of zeros of appropriate dimensions, then we have

A^{k} = [ \begin{matrix}A_{1}^{k} & O & O \\ O & A_{2}^{k} & O \\ O & O & A_{3}^{k} \end{matrix} ]

and

f(A) = [ \begin{matrix}f(A_{1}) & O & O \\ O & f(A_{2}) & O \\ O & O & f(A_{3}) \end{matrix} ] .

Consider a similar matrix Â of the given matrix A, that is, Â = T^{−1}AT , where T ∈ ℝ^{n×n} is an invertible matrix. The above can be re-written as

A = T ÂT^{−1},

and we have

A^{k} = (T ÂT^{−1}) (T ÂT^{−1}) × ⋯ × (T ÂT^{−1})
= T Â^{k}T^{−1},

or

Â^{k} = (T^{−1}AT ) (T^{−1}AT ) × ⋯ × (T^{−1}AT )
= T^{−1}A^{k}T .

Thus, we have

f (A) = T f(Â)T^{−1}  (3.15)

or

f(Â) = T^{−1}f (A) T .  (3.16)

Now let λ_{i} be the i-th eigenvalue of A and suppose that the algebraic multiplicity of λ_{i} is n_{i}. Then the characteristic polynomial of A can be written as,

Δ(λ) = det(λI_{n} − A) = ∏_{i=1}^{r} (λ − λ_{i})^{n_{i}} .  (3.17)

We now discuss a very useful property of Jordan blocks that we utilize in the proof of the very important theorem of linear algebra, the theorem of Cayley and Hamilton. Suppose that we have a fourth order Jordan block,

J = [ \begin{matrix}λ_{i} & 1 & 0 & 0 \\ 0 & λ_{i} & 1 & 0 \\ 0 & 0 & λ_{i} & 1 \\ 0 & 0 & 0 & λ_{i} \end{matrix} ] .

Then we have

(J − λ_{i}I) = [ \begin{matrix}0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{matrix} ] .

Let us now compute successive powers of (J − λI). We obtain

(J − λ_{i}I)^{2} = [ \begin{matrix}0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{matrix} ] , (J − λ_{i}I)^{3} = [ \begin{matrix}0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{matrix} ] ,

and

(J − λ_{i}I)^{k} = O for k ≥ 4.

A matrix with above property is called a nilpotent matrix.

In general, if we have a Jordan block of order r_{i} then

(J − λ_{i}I)^{k} = O for k ≥ r_{i}.  (3.18)

We have all we need to state and prove the theorem of Cayley and Hamilton.

Theorem 3.3 (Cayley-Hamilton theorem) Let

Δ(λ) = det[λI_{n} − A] = ∏_{i=1}^{r}(λ − λ_{i})^{n_{i}},

be the characteristic polynomial of a matrix A. A square matrix satisfies its characteristic polynomial equation,

Δ(A) = ∏_{i=1}^{r}(A − λ_{i}I_{n})^{n_{i}} = O.

Proof Suppose that Â is the Jordan canonical form of A. Thus there exists a non-singular matrix T such that

Â = T^{−1}AT = [ \begin{matrix}J_{1} & O & ⋯ & O \\ O & J_{2} & & O \\ ⋮ & & ... & ⋮ \\ O & O & ⋯ & J_{q} \end{matrix} ] .

140 CHAPTER 3. ANALYSIS OF MODELING EQUATIONS Let us now consider the matrix polynomial Δ(A). We use (3.15) to obtain

Δ(A) = T Δ(Â)T^{−1}
= T ∏_{i=1}^{r} (Â − λ_{i}I_{n})^{n_{i}} T^{−1}
= T ∏_{i=1}^{r} [ \begin{matrix}J_{1} − λ_{i}I_{r_{1}} & O & ⋯ & O \\ O & J_{2} − λ_{i}I_{r_{2}} & & O \\ ⋮ & & ... & ⋮ \\ O & O & ⋯ & J_{q} − λ_{i}I_{r_{q}} \end{matrix} ]^{n_{i}} T^{−1}
= T ∏_{i=1}^{r} [ \begin{matrix}(J_{1} − λ_{i}I_{r_{1}})^{n_{i}} & O & ⋯ & O \\ O & (J_{2} − λ_{i}I_{r_{2}})^{n_{i}} & & O \\ ⋮ & & ... & ⋮ \\ O & O & ⋯ & (J_{q} − λ_{i}I_{r_{q}})^{n_{i}} \end{matrix} ] T^{−1}.

The order of any Jordan block associated with λ_{i} cannot exceed the corresponding algebraic multiplicity. Taking into account the nilpotent property given by (3.18), we obtain

Δ(A) = T [ \begin{matrix}O & O & ⋯ & O \\ O & (J_{2} − λ_{1}I_{r_{2}})^{n_{1}} & & O \\ ⋮ & & ... & ⋮ \\ O & O & ⋯ & (J_{q} − λ_{1}I_{r_{q}})^{n_{1}} \end{matrix} ] ×

⋯ × [ \begin{matrix}(J_{1} − λ_{r}I_{r_{1}})^{n_{r}} & O & ⋯ & O \\ O & (J_{2} − λ_{r}I_{r_{2}})^{n_{r}} & & O \\ ⋮ & & ... & ⋮ \\ O & O & ⋯ & O \end{matrix} ] T^{−1}

= T [ \begin{matrix}O & O & ⋯ & O \\ O & O & & O \\ ⋮ & & ... & ⋮ \\ O & O & ⋯ & O \end{matrix} ] T^{−1}
= O,  (3.19)

which completes the proof of the theorem.

□

It follows from the Cayley-Hamilton theorem that Δ(A) = A^{n} + α_{n−1}A^{n−1} + ⋯ + α_{1}A + α_{0}I_{n} = O, that is,

A^{n} = −α_{n−1}A^{n−1} − ⋯ − α_{1}A − α_{0}I_{n}.  (3.20)

Thus A^{n} can be expressed as a linear combination of the matrices

{I_{n}, A, …, A^{n−1}}.

If we multiply both sides of (3.20) by A, then we obtain

A^{n+1} = −α_{n−1}A^{n} − ⋯ − α_{1}A^{2} − α_{0}A.  (3.21)

Thus, A^{n+1} can be expressed as a linear combination of

{A, A^{2}, …, A^{n}}.

Using (3.20), we conclude that A^{n+1} can be expressed as a linear combination of {I_{n}, A, …, A^{n−1}}. Using the same argument, we conclude that any A^{k}, k ≥ 0, can be expressed as a linear combination of {I_{n}, A, …, A^{n−1}}. Therefore, any matrix polynomial f(A) can also be expressed as

f(A) = β_{0}I_{n} + β_{1}A + ⋯ + β_{n−1}A^{n−1}

for some β_{i}, i = 0, 1, …, n − 1. The above property is used to efficiently compute matrix polynomials of A, which can then be used to compute the matrix exponential e^{At} as well as the state transition matrix for discrete-time systems. In what follows, we present such a method and illustrate it with examples. We follow the presentation of Chen [55, pp. 63–67].

Suppose we are given a polynomial f(λ) and we are interested in calculating f(A). We begin with dividing f(λ) by Δ(λ) to obtain

f(λ) = q(λ)Δ(λ) + h(λ),  (3.22)

where q(λ) is the quotient, h(λ) is the remainder, and

deg_{λ} h(λ) < n.

We evaluate f(A) using (3.22) and the Cayley-Hamilton theorem to obtain

f(A) = q(A)Δ(A) + h(A)
= q(A)O + h(A)
= h(A).

Thus instead of evaluating the matrix polynomial f(A) that may contain very high powers of A, we can find f(A) by computing h(A), where the highest power of A is less than n.

In addition, rather than performing the division (3.22), we can find h(λ) by assuming that it has the form

h(λ) = β_{0} + β_{1}λ + ⋯ + β_{n−1}λ^{n−1}

and then solve n equations for β_{i}, i = 0, 1, …, n − 1. Consider first the case when A has distinct eigenvalues. To obtain equations involving the coefficients β_{i}, we note that

f(λ_{i}) = q(λ_{i})Δ(λ_{i}) + h(λ_{i}) = h(λ_{i}), i = 1, 2, …, n.

Example 3.6 For a given

f(λ) = λ^{7},

we compute

f(A) = A^{7},

where

A = [ \begin{matrix}0 & 2 \\ 1 & 1 \end{matrix} ] .  (3.23)

The characteristic polynomial of A is

Δ(λ) = λ^{2} − λ − 2 = (λ + 1)(λ − 2).

Let

h(λ) = β_{0} + β_{1}λ.

Then we have two equations:

f(λ_{i}) = h(λ_{i}), i = 1, 2,

that is,

f(−1) = (−1)^{7} = −1 = β_{0} − β_{1}

and

f(2) = 2^{7} = 128 = β_{0} + 2β_{1}.

We represent the above linear equations for β_{0} and β_{1} in the matrix format as

[ \begin{matrix}1 & −1 \\ 1 & 2 \end{matrix} ] [ \begin{matrix}β_{0} \\ β_{1} \end{matrix} ] = [ \begin{matrix}−1 \\ 128 \end{matrix} ] .

Solving, we obtain β_{0} = 42 and β_{1} = 43. Hence

f(A) = A^{7} = 42I_{2} + 43A = [ \begin{matrix}42 & 86 \\ 43 & 85 \end{matrix} ] .

Example 3.7 We can use the above method to evaluate e^{At}, where A is given by (3.23). In other words, let f(λ) = e^{λt}; we wish to find

f(A) = e^{At}.

Let h(λ) = β_{0} + β_{1}λ. Then

f(−1) = h(−1)

yields

e^{−t} = β_{0} − β_{1},

and

f(2) = h(2)

yields

e^{2t} = β_{0} + 2β_{1}.

We thus have two equations for β_{0} and β_{1},

[ \begin{matrix}1 & −1 \\ 1 & 2 \end{matrix} ] [ \begin{matrix}β_{0} \\ β_{1} \end{matrix} ] = [ \begin{matrix}e^{−t} \\ e^{2t} \end{matrix} ] .

Solving the above gives

[ \begin{matrix}β_{0} \\ β_{1} \end{matrix} ] = \frac{1}{3} [ \begin{matrix}2 & 1 \\ −1 & 1 \end{matrix} ] [ \begin{matrix}e^{−t} \\ e^{2t} \end{matrix} ] = \frac{1}{3} [ \begin{matrix}2e^{−t} + e^{2t} \\ −e^{−t} + e^{2t} \end{matrix} ] .

Hence

e^{At} = β_{0}I_{2} + β_{1}A = \frac{1}{3} [ \begin{matrix}2e^{−t} + e^{2t} & −2e^{−t} + 2e^{2t} \\ −e^{−t} + e^{2t} & e^{−t} + 2e^{2t} \end{matrix} ] .

When the characteristic polynomial has repeated zeros, we use the derivatives of f(λ) and h(λ) to obtain additional equations to determine the coefficients of h(λ) as per the following theorem.

Theorem 3.4 Chen [55, p. 64] We are given f(λ) and an n×n matrix A with characteristic polynomial

Δ(λ) = ∏_{i=1}^{m}(λ − λ_{i})^{n_{i}},

where n = ∑_{i=1}^{m} n_{i}. Let

h(λ) = β_{0} + β_{1}λ + ⋯ + β_{n−1}λ^{n−1}.

It is a polynomial of degree n − 1 with n unknown coefficients. These n unknowns are to be solved from the following set of n equations:

f^{(l)}(λ_{i}) = h^{(l)}(λ_{i}) for l = 0, 1, …, n_{i} − 1 and i = 1, 2, …, m

where

f^{(l)}(λ_{i}) = \frac{d^{l}f(λ)}{dλ^{l}} |_{λ=λ_{i}}

and h^{(l)}(λ_{i}) is similarly defined. Then we have

f(A) = h(A)

and h(λ) is said to be equal f(λ) on the spectrum of A.

Example 3.8 We compute e^{At} for

A = [ \begin{matrix}3 & 0 & 1 \\ −1 & 1 & 2 \\ −2 & 0 & 0 \end{matrix} ] .

The characteristic polynomial is

Δ(λ) = (λ − 1)^{2}(λ − 2).

Let f(λ) = e^{λt} and let h(λ) = β_{0} + β_{1}λ + β_{2}λ^{2}. Then using Theorem 3.4, we obtain

f(1) = h(1), e^{t} = β_{0} + β_{1} + β_{2},
f′(1) = h′(1), te^{t} = β_{1} + 2β_{2},
f(2) = h(2), e^{2t} = β_{0} + 2β_{1} + 4β_{2}.

Thus we have a set of equations

[ \begin{matrix}1 & 1 & 1 \\ 0 & 1 & 2 \\ 1 & 2 & 4 \end{matrix} ] [ \begin{matrix}β_{0} \\ β_{1} \\ β_{2} \end{matrix} ] = [ \begin{matrix}e^{t} \\ te^{t} \\ e^{2t} \end{matrix} ] .

3.3. COMPUTING FUNCTIONS OF A SQUARE MATRIX 145 Solving the above gives

[ \begin{matrix}β_{0} \\ β_{1} \\ β_{2} \end{matrix} ] = [ \begin{matrix}0 & −2 & 1 \\ 2 & 3 & −2 \\ −1 & −1 & 1 \end{matrix} ] [ \begin{matrix}e^{t} \\ te^{t} \\ e^{2t} \end{matrix} ] = [ \begin{matrix}−2te^{t} + e^{2t} \\ 2e^{t} + 3te^{t} − 2e^{2t} \\ −e^{t} − te^{t} + e^{2t} \end{matrix} ] .

Hence

e^{At} = h(A)
= β_{0}I_{3} + β_{1}A + β_{2}A^{2}
= (−2te^{t} + e^{2t}) [ \begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} ] + (2e^{t} + 3te^{t} − 2e^{2t}) [ \begin{matrix}3 & 0 & 1 \\ −1 & 1 & 2 \\ −2 & 0 & 0 \end{matrix} ]

+ (−e^{t} − te^{t} + e^{2t}) [ \begin{matrix}7 & 0 & 3 \\ −8 & 1 & 1 \\ −6 & 0 & −2 \end{matrix} ]

= [ \begin{matrix}−e^{t} + 2e^{2t} & 0 & −e^{t} + e^{2t} \\ 6e^{t} + 5te^{t} − 6e^{2t} & e^{t} & 3e^{t} + 5te^{t} − 3e^{2t} \\ 2e^{t} − 2e^{2t} & 0 & 2e^{t} − e^{2t} \end{matrix} ] .

Example 3.9 Suppose we are given the Jordan block of order n_{i},

J = [ \begin{matrix}λ_{i} & 1 & 0 & ⋯ & 0 & 0 \\ 0 & λ_{i} & 1 & ⋯ & 0 & 0 \\ 0 & 0 & λ_{i} & ⋯ & 0 & 0 \\ ⋮ & ⋮ & & ... & & ⋮ \\ 0 & 0 & 0 & ⋯ & λ_{i} & 1 \\ 0 & 0 & 0 & ⋯ & 0 & λ_{i} \end{matrix} ] .

The characteristic polynomial of J is Δ(λ) = (λ − λ_{i})^{n_{i}}. For computational simplicity we select the (n_{i} − 1)-st order polynomial h(λ) of the form

h(λ) = β_{0} + β_{1}(λ − λ_{i}) + β_{2}(λ − λ_{i})^{2} + ⋯ + β_{n_{i}−1}(λ − λ_{i})^{n_{i}−1}.

The condition f(λ) = h(λ) on the spectrum of J yields

β_{0} = f(λ_{i}), β_{1} = f′(λ_{i}), β_{2} = \frac{f′′(λ_{i})}{2!}, …, β_{n_{i}−1} = \frac{f^{(n_{i}−1)}(λ_{i})}{(n_{i} − 1)!}.

Hence

f(J) = f(λ_{i})I_{n_{i}} + \frac{f′(λ_{i})}{1!}(J − λ_{i}I_{n_{i}}) + \frac{f′′(λ_{i})}{2!}(J − λ_{i}I_{n_{i}})^{2} + ⋯

⋯ + \frac{f^{(n_{i}−1)}(λ_{i})}{(n_{i} − 1)!}(J − λ_{i}I_{n_{i}})^{n_{i}−1}.

Using the nilpotent property of the matrix (J − λ_{i}I_{n_{i}}), we obtain

f(J) = [ \begin{matrix}f(λ_{i}) & f′(λ_{i})/1! & f′′(λ_{i})/2! & ⋯ & f^{(n_{i}−1)}(λ_{i})/(n_{i} − 1)! \\ 0 & f(λ_{i}) & f′(λ_{i})/1! & ⋯ & f^{(n_{i}−2)}(λ_{i})/(n_{i} − 2)! \\ 0 & 0 & f(λ_{i}) & ⋯ & f^{(n_{i}−3)}(λ_{i})/(n_{i} − 3)! \\ ⋮ & ⋮ & ⋮ & & ⋮ \\ 0 & 0 & 0 & ... & f′(λ_{i})/1! \\ 0 & 0 & 0 & ⋯ & f(λ_{i}) \end{matrix} ] .

In particular, if f(λ) = e^{λt}, then we have

e^{J t} = [ \begin{matrix}e^{λ_{i}t} & te^{λ_{i}t} & t^{2}e^{λ_{i}t}/2! & ⋯ & t^{n_{i}−1}e^{λ_{i}t}/(n_{i} − 1)! \\ 0 & e^{λ_{i}t} & te^{λ_{i}t} & ⋯ & t^{n_{i}−2}e^{λ_{i}t}/(n_{i} − 2)! \\ 0 & 0 & e^{λ_{i}t} & ⋯ & t^{n_{i}−3}e^{λ_{i}t}/(n_{i} − 3)! \\ ⋮ & ⋮ & ⋮ & & ⋮ \\ 0 & 0 & 0 & ... & te^{λ_{i}t} \\ 0 & 0 & 0 & ⋯ & e^{λ_{i}t} \end{matrix} ] .

3.4 State Equation Solution of Controlled System

Consider a model of a controlled dynamical system,

ẋ(t) = Ax(t) + Bu(t)  (3.24)

subject to a given initial condition x(0) = x_{0}, where B is an n × m matrix, and m < n as is usually in practice. Premultiplying both sides of (3.24) by the integrating factor, e^{−At}, yields

e^{−At}ẋ(t) = e^{−At}Ax(t) + e^{−At}Bu(t).

We then rearrange the above equation as

e^{−At}ẋ(t) − e^{−At}Ax(t) = e^{−At}Bu(t).  (3.25)

Noting that

\frac{d}{dt} (e^{−At}x(t)) = −Ae^{−At}x(t) + e^{−At}ẋ(t),

we can represent (3.25) in the form

\frac{d}{dt} (e^{−At}x(t)) = e^{−At}Bu(t).

Integrating both sides of the resulting equation gives

e^{−At}x(t) − x(0) = ∫_{0}^{t} e^{−Aτ}Bu(τ)dτ.

Hence

x(t) = e^{At}x(0) + ∫_{0}^{t} e^{A(t−τ)}Bu(τ)dτ.

If the initial condition is x(t_{0}) = x_{0}, then integration from t_{0} to t yields

x(t) = e^{A(t−t_{0})}x(t_{0}) + ∫_{t_{0}}^{t} e^{A(t−τ)}Bu(τ)dτ  (3.26)

As in the case of the uncontrolled system, we can also use the Laplace transform technique to obtain the solution to (3.24) subject to the initial condition x(0) = x_{0}. Indeed, taking the Laplace transform of (3.24) and performing some manipulations yields

X(s) = (sI_{n} − A)^{−1} x(0) + (sI_{n} − A)^{−1} BU(s).

Hence

x(t) = L^{−1} {(sI_{n} − A)^{−1} x(0)} + L^{−1} {(sI_{n} − A)^{−1} BU(s)} .

If x_{0} = x(t_{0}), then

x(t) = L^{−1} {e^{−t_{0}s} (sI_{n} − A)^{−1} x(t_{0})} + L^{−1} {e^{−t_{0}s} (sI_{n} − A)^{−1} BU(s)} .

3.5 From Continuous to Discrete Models

Our objective here is to present a method for constructing linear discrete-time models from given linear continuous-time models.

We use a sample-and-hold device that transforms a continuous signal, f(t), into the staircase signal,

f(kh), kh ≤ t < (k + 1)h,

Figure 3.1: Sample and zero-order hold (ZOH) element operating on a continuous function.

where h is the sampling period. The sample and zero-order hold (ZOH) operation is illustrated in Figure 3.1.

Suppose that we are given a continuous-time model,

ẋ(t) = Ax(t) + Bu(t), x_{0} = x(t_{0})  (3.27)
y(t) = Cx(t).  (3.28)

The solution to the state equation is

x(t) = e^{A(t−t_{0})}x(t_{0}) + ∫_{t_{0}}^{t} e^{A(t−τ)}Bu(τ)dτ.  (3.29)

We assume that the input to the system is generated by a sample and zero-order device and has the form,

u(t) = u(k), kh ≤ t < (k + 1)h.

Let t_{0} = kh and t = (k + 1)h and let us use shorthand notation,

x[k] = x(kh)

Then taking into account that u(t) is constant on the interval [kh, (k + 1)h) we represent (3.29) as

x[k + 1] = e^{Ah}x[k] + ∫_{kh}^{(k+1)h} e^{A(kh+h−τ)}Bu[k]dτ
= e^{Ah}x[k] + ∫_{kh}^{(k+1)h} e^{A(kh+h−τ)}Bdτ u[k].  (3.30)

Consider now the second term on the right-hand side of the above equation. Let

η = kh + h − τ.

Then we can represent (3.30) as

x[k + 1] = e^{Ah}x[k] + ∫_{kh}^{(k+1)h} e^{A(kh+h−τ)}Bdτ u[k]
= e^{Ah}x[k] + ∫_{0}^{h} e^{Aη}Bdη u[k]
= A_{d}x[k] + B_{d}u[k],  (3.31)

where

A_{d} = e^{Ah} and B_{d} = ∫_{0}^{h} e^{Aη}Bdη.

The discrete output equation has the form

y[k] = Cx[k].

We point out that the above discrete model, employing the zero-order hold, is exact in the sense that the state, input, and output of this model are exactly the same at the sampling times as the state, input, and output of the underlying continuous-time model.

Example 3.10 We construct an exact linear discrete state-space model for the continuous system model,

ẋ(t) = [ \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ] x(k) + [ \begin{matrix}2 \\ 1 \end{matrix} ] u(t)
y(t) = [ 1 0 ] x(t).

The sampling period is h = 1 sec.

We first evaluate A_{d} = e^{Ah}. Note that A^{2} = O. Hence

A_{d} = I_{2} + Ah
= [ \begin{matrix}1 & 1 \\ 0 & 1 \end{matrix} ] .

Next, we evaluate

B_{d} = ∫_{0}^{h} e^{Aτ}Bdτ.

We obtain

B_{d} = ∫_{0}^{h} e^{Aτ}Bdτ
= ∫_{0}^{h} [ \begin{matrix}1 & τ \\ 0 & 1 \end{matrix} ] [ \begin{matrix}2 \\ 1 \end{matrix} ] dτ
= ∫_{0}^{1} [ \begin{matrix}2 + τ \\ 1 \end{matrix} ] dτ
= [ \begin{matrix}2τ + \frac{1}{2}τ^{2} \\ τ \end{matrix} ]|_{0}^{1}
= [ \begin{matrix}2.5 \\ 1 \end{matrix} ] .

The discrete model then has the form

x[k + 1] = [ \begin{matrix}1 & 1 \\ 0 & 1 \end{matrix} ] x[k] + [ \begin{matrix}2.5 \\ 1 \end{matrix} ]
y[k] = [ 1 0 ] x[k].

3.6 State Plane Analysis

The state plane, which is a state space of two-dimensional systems, is also called the phase plane. Analysis in the state plane is applicable to linear and nonlinear systems modeled by second-order ordinary differential equations. The state-plane methods are graphical procedures for solving such equations. Using state-plane methods one can graphically determine the transient response of a second-order dynamical system. We now introduce relevant terms. Consider a class of second-order systems whose dynamics are modeled by the differential equation,

ẍ = f(x, ẋ).  (3.32)

Define the state variables,

x_{1} = x and x_{2} = ẋ.

Then, equation (3.32) can be represented as

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & f(x_{1}, x_{2}) \end{matrix} \}  (3.33)

Figure 3.2: A spring-mass system of Example 3.11.

The plane with coordinates x_{1} and x_{2} is labeled as the state plane or phase plane. A solution of (3.32) can be illustrated by plotting x versus t, or by plotting x_{2} versus x_{1} using t as a parameter. To each value of the state x(t) = [ x_{1}(t) x_{2}(t) ]^{T} there corresponds a point in the (x_{1}, x_{2})-plane called the representative point (RP). As t varies, the RP describes a curve in the state plane called a trajectory. A family of trajectories is called a phase portrait.

3.6.1 Examples of Phase Portraits

Example 3.11 Consider the spring-mass system shown in Figure 3.2, where x is the displacement of the mass M from equilibrium, and k is the linear spring constant. The spring is assumed to be linear, that is, it obeys Hooke’s law. Applying Newton’s law, we obtain the equation of motion of the mass,

Mẍ + kx = 0.

Let ω^{2} = k/M. Then, the above equation can be represented in standard form,

ẍ + ω^{2}x = 0.  (3.34)

Observe that

ẍ = ẋ\frac{ẍ}{ẋ} = ẋ \frac{\frac{dẋ}{dt}}{\frac{dx}{dt}} = ẋ\frac{dẋ}{dx}.

Figure 3.3: A phase portrait for the spring-mass system of Example 3.11.

Using the above, we represent equation (3.34) as

ẋ\frac{dẋ}{dx} + ω^{2}x = 0.

Separating the variables yields

\frac{ẋ}{ω^{2}} dẋ + xdx = 0.

Integrating the above gives

(\frac{ẋ}{ω} )^{2} + x^{2} = A^{2},

where A is the integration constant determined by the initial conditions. We see that a phase portrait of equation (3.34) consists of a family of circles centered at the origin of the (x, \frac{ẋ}{ω})- plane as illustrated in Figure 3.3. Different circular trajectories, in this plane, correspond to different values of the constant A representing initial conditions. Note that

x = ω ∫ \frac{ẋ}{ω} dt,

where ω > 0. Hence, for positive values of ẋ/ω the displacement x increases. This means that for the values ẋ/ω > 0 the arrows on the trajectories indicate x moving to the right. For negative values of ẋ/ω, that is, for the values of ẋ/ω below the horizontal axis, the arrows on the trajectories indicate x decreasing by going to the left. Therefore, as time increases the RP moves in a clockwise direction along the trajectory.

Figure 3.4: A model of a rigid satellite of Example 3.12.

Example 3.12 In this example, we consider a model of a satellite shown in Figure 3.4. We assume that the satellite is rigid, and in a frictionless environment. It can rotate about an axis perpendicular to the page as a result of torque τ applied to the satellite by firing the thrusters. The system input is the applied torque, and the system output is the attitude angle θ. The satellite’s moment of inertia is I. Applying the rotational version of Newton’s second law, we obtain the satellite’s equation of motion,

I \frac{d^{2}θ}{dt^{2}} = τ,

which can be represented in the following equivalent form:

\frac{d^{2}θ}{dt^{2}} = \frac{1}{I}τ.

Let

\frac{1}{I}τ = u,

then the equation describing the satellite’s motion becomes

\frac{d^{2}θ}{dt^{2}} = u.  (3.35)

We assume that when the thrusters fire, the thrust is constant, that is, u = ±U. For simplicity, let U = 1. Using the state variables x_{1} = θ and x_{2} = θ̇, we obtain the satellite’s state-space model,

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ 1 \end{matrix} ] u,  (3.36)

where u = ±1. We first consider the case when u = 1. With this control, the system equations are:

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & 1. \end{matrix} \}

Solving the second equation yields

x_{2} = t + c_{1},

where c_{1} is an integration constant. We substitute x_{2} = t + c_{1} into ẋ_{1} = x_{2} to get

ẋ_{1} = t + c_{1}.

Solving the above equation, we obtain

x_{1} = ∫ (t + c_{1})dt
= \frac{1}{2}(t + c_{1})^{2} + c_{2}
= \frac{1}{2}x_{2}^{2} + c_{2},

where c_{2} is an integration constant. In the state plane the above equation represents a family of parabolas open toward the positive x_{1} axis. The representative point moves upward along these parabolas because ẋ_{2} = 1, that is, ẋ_{2} > 0. Typical system trajectories for this case are shown in Figure 3.5.

Similarly, when u = −1, we have

ẋ_{2} = −1.

Solving the above differential equation, yields

x_{2} = −t + c̃_{1},

where c̃_{1} is an integration constant. Substituting x_{2} = −t+c̃_{1} into ẋ_{1} = x_{2}, and then solving the resulting equation yields

x_{1} = −\frac{1}{2}x_{2}^{2} + c̃_{2},

Figure 3.5: Trajectories of the system (3.36) for u = 1.

where c̃_{2} is an integration constant. In the state plane, this equation represents a family of parabolas open toward the negative x_{1} axis. The representative point moves downward along these parabolas because ẋ_{2} = −1, that is, ẋ < 0. Typical system trajectories for this case are shown in Figure 3.6.

In the above examples, systems’ trajectories were constructed analytically. Analytical methods are useful for systems modeled by differential equations that can be easily solved. If the system of differential equations cannot be easily solved analytically, we can use graphical or numerical methods. We now describe a graphical method for solving second-order, or a system of two first-order, differential equations.

3.6.2 The Method of Isoclines

An isocline is a curve along which the trajectories’ slope is constant. In other words, an isocline is a locus, a set of points, of the trajectories’ constant slope. To obtain equations that describe isoclines, we consider a system of two first-order differential equations of the

Figure 3.6: Trajectories of the system (3.36) for u = −1.

form

\begin{matrix}ẋ_{1} & = & \frac{dx_{1}}{dt} = f_{1}(x_{1}, x_{2}) \\ ẋ_{2} & = & \frac{dx_{2}}{dt} = f_{2}(x_{1}, x_{2}). \end{matrix} \}  (3.37)

Dividing the second of the above equations by the first one, we obtain

\frac{dx_{2}}{dx_{1}} = \frac{f_{2}(x_{1}, x_{2})}{f_{1}(x_{1}, x_{2})}.  (3.38)

Thus, we eliminated the independent variable t from the set of the first-order differential equations given by (3.37). In equation (3.38), we consider x_{1} as the independent variable, and x_{2} as dependent. Let

m = m(x_{1}, x_{2}) = \frac{dx_{2}}{dx_{1}} .

Note that m is just the slope of the tangent to the trajectory passing through the point [ x_{1} x_{2} ]^{T}. The locus of the trajectory constant slope

\frac{dx_{2}}{dx_{1}} = m(x_{1}, x_{2}) = constant,

can be obtained from the equation

f_{2}(x_{1}, x_{2}) = mf_{1}(x_{1}, x_{2}).

Figure 3.7: Isoclines and a typical trajectory for the spring-mass system of Example 3.13.

The curve that satisfies the above equation is an isocline corresponding to the trajectories’ slope m because a trajectory crossing the isocline will have its slope equal to m. The idea of the method of isoclines is to construct several isoclines in the state plane. This then allows one to construct a field of local tangents m. Then, the trajectory passing through any given point in the state plane is obtained by drawing a continuous curve following the directions of the field.

Example 3.13 Consider the spring-mass system from Example 3.11. Defining the state variables x_{1} = x and x_{2} = ẋ, we obtain the state equations

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & −ω^{2}x_{1}. \end{matrix} \}

Let, for simplicity, ω = 1. Then,

\frac{dx_{2}}{dx_{1}} = m = −\frac{x_{1}}{x_{2}} .

From the above we obtain the equation of isoclines

x_{2} = −\frac{1}{m}x_{1}.

This equation describes a family of straight lines through the origin. In Figure 3.7, we show some of the isoclines and a typical trajectory for the above spring-mass system. Note that in this example the trajectory slope is m, while the isocline slope is −1/m. This means that trajectories of the spring-mass system are perpendicular to the isoclines, that is, the trajectories are circles centered at the origin of the state plane.

Example 3.14 Given a second-order differential equation

ẍ = −4ηẋ − 4x,  (3.39)

where η is a parameter. We will find values of η for which there are isoclines along which the trajectory slope and the isocline slope are equal. Such isoclines, if they exist, are called the asymptotes.

Let x_{1} = x and x_{2} = ẋ. Then, the second-order differential equation given by (3.39) can be represented as a system of first-order differential equations,

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & −4ηx_{2} − 4x_{1} \end{matrix} \}

The isocline equation is

\frac{dx_{2}}{dx_{1}} = \frac{−4ηx_{2} − 4x_{1}}{x_{2}} = m,

or equivalently

x_{2} = − (\frac{4}{m + 4η} ) x_{1}.  (3.40)

A trajectory slope is equal to an isocline slope if and only if

−\frac{4}{m + 4η} = m,

that is,

m^{2} + 4ηm + 4 = 0.  (3.41)

The discriminant of (3.41) is

Δ = 4\sqrt{η^{2} − 1}.

Hence, the condition for the existence of asymptotes is

|η| ≥ 1.  (3.42)

Example 3.15 Consider the system of differential equations,

\begin{matrix}ẋ_{1} & = & x_{1} + x_{2} + 1 \\ ẋ_{2} & = & −x_{2} + 2. \end{matrix} \}

(i) Find the equation of the isocline corresponding to the trajectory slope m = 1;

(ii) Does the given system have asymptotes? If yes, then write the equations of the asymptotes.

The trajectory slope is

\frac{dx_{2}}{dx_{1}} = m = \frac{−x_{2} + 2}{x_{1} + x_{2} + 1}.

Hence, the equation of the isocline corresponding to the trajectory slope m = 1 is obtained by solving

\frac{−x_{2} + 2}{x_{1} + x_{2} + 1} = 1.

We obtain

x_{2} = −\frac{1}{2}x_{1} + \frac{1}{2}.

The trajectory slope is

m = \frac{−x_{2} + 2}{x_{1} + x_{2} + 1}.  (3.43)

From (3.43), we obtain an equation of isoclines of the form

x_{2} = −\frac{m}{m + 1}x_{1} − \frac{m − 2}{m + 1}.

An asymptote is the isocline for which its slope and the trajectory slope are equal. Therefore, we can obtain the equations of asymptotes by solving

m = −\frac{m}{m + 1}.

We obtain

m^{2} + 2m = m(m + 2) = 0.

Hence, the equations of asymptotes are

x_{2} = 2 and x_{2} = −2x_{1} − 4.

We can use a computer to draw phase portraits interactively. Below is an example of MATLAB’s script that was used to sketch a phase portrait of the van der Pol equation shown in Figure 3.8.

t0=0;                      % initial time
tf=20;                     % final time
tspan=tf-t0;
x0=[-4 -4]';               % first initial condition
button=1;
p=4*[-1 0;1 0];            % preparation to draw internal axes
clf;plot(p(:,1),p(:,2))
hold on
plot(p(:,2),p(:,1))
axis(4*[-1 1 -1 1])
    while(button==1)
% Select a new initial condition and press the left button
% Press the right button if you wish to stop
      [t,x]=ode45(@my_xdot,tspan,x0); % invoke your own m-file
      plot(x(:,1),x(:,2))
      [x1,x2,button]=ginput(1);
      x0=[x1 x2]';
    end

The van der Pol equation has the form

ẍ − μ(1 − x^{2})ẋ + x = 0,  (3.44)

where μ is a parameter. We can equivalently represent the van der Pol equation as

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & μ(1 − x_{1}^{2})x_{2} − x_{1} \end{matrix} \}  (3.45)

An inspection of the phase portrait of the van der Pol equation reveals the existence of a closed trajectory in the state plane. This solution is an example of a limit cycle.

Figure 3.8: A phase-plane portrait of the van der Pol equation for μ = 1.

Definition 3.4 A limit cycle is a closed trajectory in the state plane, such that no other closed trajectory can be found arbitrarily close to it.

◇

Note that the differential equation modeling the spring-mass system does not have a limit cycle because the solutions are not isolated. Limit cycles can only appear in nonlinear systems; they do not exist in linear systems.

Theorem 3.5 (Bendixson) If f_{1} and f_{2} in (3.38) have continuous partial derivatives in a domain A bounded by a closed path C and if ∂f_{1}/∂x_{1} + ∂f_{2}/∂x_{2} does not vanish and is of constant sign in that domain, then (3.38) can have no limit cycle in that domain.

Proof We prove the theorem by contradiction. Equation (3.38) is satisfied on any of its trajectory. Therefore, if (3.38) has a limit cycle then clearly

f_{1}dx_{2} − f_{2}dx_{1} = 0.  (3.46)

We now apply Green’s theorem, stated on page 1030, to the vector field

[ −f_{2} f_{1} ]^{T}

to obtain

∫C f_{1} dx_{2} − f_{2}dx_{1} = ∫∫_{A} (\frac{∂f_{1}}{∂x_{1}} + \frac{∂f_{2}}{∂x_{2}} ) dx_{1}dx_{2}.  (3.47)

By (3.46), the right-hand side of (3.47) must vanish, which contradicts the hypothesis. Thus, if ∂f_{1}/∂x_{1} + ∂f_{2}/∂x_{2} does not vanish and is of constant sign in a domain A bounded by a closed path C, then no limit cycle can exist in A. The proof of the theorem is complete.

□

Example 3.16 We apply Bendixson’s theorem to the van der Pol equation represented by (3.45). We have

f_{1} = x_{2}, and f_{2} = μ(1 − x_{1}^{2})x_{2} − x_{1}.

We calculate ∂f_{1}/∂x_{1} + ∂f_{2}/∂x_{2} to get

∂f_{1}/∂x_{1} + ∂f_{2}/∂x_{2} = μ(1 − x_{1}^{2}).

It follows from the Bendixson theorem that there are no limit cycles in the region |x_{1}| < 1 because ∂f_{1}/∂x_{1} + ∂f_{2}/∂x_{2} > 0 in that region.

Bendixson’s theorem gives us a necessary condition for a closed curve to be a limit cycle. It is useful for the purpose of establishing the nonexistence of a limit cycle. In general, it is very difficult to establish the existence of a limit cycle. Sufficient conditions for the existence of a limit cycle are discussed, for example, by Hochstadt [138, Chapter 7]. A method for predicting limit cycles, called the describing function method, is presented in Section 4.3.

3.7 Numerical Techniques

In this section, we discuss numerical techniques for solving differential equations. We begin our presentation of numerical techniques that allow us to determine approximate solutions to the given system of differential equations with a method based on a Taylor series expansion. Much of this section presentation is based on the Class Notes of the late Professor V. B. Haas [124].

3.7.1 The Method of Taylor Series

The basis for many numerical methods for solving differential equations is the Taylor formula. Suppose we are given a state-space model of the form

ẋ(t) = f(t, x(t), u(t)), x(t_{0}) = x_{0}.  (3.48)

The method, we discuss, depends on the fact that if f(t, x, u) is differentiable with respect to its n + m + 1 variables x_{1}, …, x_{n}, u_{1}, …, u_{m}, t in some (n + m + 1)-dimensional region D, and if x(t) is a solution of (3.48), where (t_{0}, x_{0}, u(t_{0})) lies in D, then we can expand x(t) into a Taylor series to obtain

x(t) = x_{0} + ẋ(t_{0})(t − t_{0}) + ẍ(t_{0})\frac{(t − t_{0})^{2}}{2} + ⋯
= x_{0} + f(t_{0}, x_{0}, u(t_{0}))(t − t_{0})

+ [\frac{∂f(t_{0}, x_{0}, u(t_{0}))}{∂t} + ∑_{i=1}^{n} \frac{∂f(t_{0}, x_{0}, u(t_{0}))}{∂x_{i}} ẋ_{i}

+ ∑_{j=1}^{m} \frac{∂f(t_{0}, x_{0}, u(t_{0}))}{∂u_{j}} u̇_{j}] \frac{(t − t_{0})^{2}}{2} + ⋯

If we stop after the q-th term of the Taylor series solution, then the remainder after q terms of a Taylor series gives us an estimate of the error. We can expect the Taylor series solution to give a fairly accurate approximation of the solution of the state-space equations only in a small interval about t_{0}. Thus, if Δt = t − t_{0} is some small time interval such that the Taylor series converges for t ∈ [ t_{0}, t_{0} + Δt ], we can use the expansion

x(t) = x_{0} + \frac{dx(t_{0})}{dt}(t − t_{0}) + \frac{d^{2}x(t_{0})}{dt^{2}} \frac{(t − t_{0})^{2}}{2} + ⋯

to evaluate x(t_{1}) = x(t_{0} + Δt). Then, we can start over again solving ẋ(t) = f(t, x(t), u(t)) with the new initial condition x(t_{1}). We illustrate the method with simple examples.

Example 3.17 Consider the first-order differential equation

\frac{dx}{dt} = t^{2} − x^{2}, x(1) = 1.

Expanding x = x(t) into a Taylor series about x(1) = 1 yields

x(t) = x(1) + \frac{dx(1)}{dt}(t − 1) + \frac{d^{2}x(1)}{dt^{2}} \frac{(t − 1)^{2}}{2} + \frac{d^{3}x(1)}{dt^{3}} \frac{(t − 1)^{3}}{3!} + \frac{d^{4}x(1)}{dt^{4}} \frac{(t − 1)^{4}}{4!} ⋯

164 CHAPTER 3. ANALYSIS OF MODELING EQUATIONS Because \frac{dx}{dt} = t^{2} − x^{2}, we obtain by successive differentiations

\frac{d^{2}x}{dt^{2}} = 2t − 2x\frac{dx}{dt},
\frac{d^{3}x}{dt^{3}} = 2 − 2 (\frac{dx}{dt})^{2} − 2x\frac{d^{2}x}{dt^{2}},
\frac{d^{4}x}{dt^{4}} = −6\frac{dx}{dt} \frac{d^{2}x}{dt^{2}} − 2x\frac{d^{3}x}{dt^{3}},
⋮

Evaluating the above at t = 1, yields

\frac{dx(1)}{dt} = 0,
\frac{d^{2}x(1)}{dt^{2}} = 2,
\frac{d^{3}x(1)}{dt^{3}} = −2,
\frac{d^{4}x(1)}{dt^{4}} = 4,
⋮

Hence,

x(t) = 1 + (t − 1)^{2} − \frac{(t − 1)^{3}}{3} + \frac{(t − 1)^{4}}{6} + · · · .

Example 3.18 In this example, we consider the second-order differential equation

\frac{d^{2}x}{dt^{2}} + 3\frac{dx}{dt} + 6tx = 0,

subject to the initial conditions x(0) = 1, and \frac{dx(0)}{dt} = 1. Using \frac{d^{2}x}{dt^{2}} = −3\frac{dx}{dt} − 6tx, we obtain by successive differentiation

\frac{d^{3}x}{dt^{3}} = −3\frac{d^{2}x}{dt^{2}} − 6t\frac{dx}{dt} − 6x,
\frac{d^{4}x}{dt^{4}} = −3\frac{d^{3}x}{dt^{3}} − 6t\frac{d^{2}x}{dt^{2}} − 12\frac{dx}{dt},
\frac{d^{5}x}{dt^{5}} = −3\frac{d^{4}x}{dt^{4}} − 6t\frac{d^{3}x}{dt^{3}} − 18\frac{d^{2}x}{dt^{2}},
⋮

Taking into account the initial conditions, we evaluate all coefficients at t = 0 to get

\frac{d^{2}x(0)}{dt^{2}} = −3,
\frac{d^{3}x(0)}{dt^{3}} = 3,
\frac{d^{4}x(0)}{dt^{4}} = −21,
\frac{d^{5}x(0)}{dt^{5}} = 117,
⋮

Hence,

x(t) = x(0) + \frac{dx(0)}{dt}t + \frac{d^{2}x(0)}{dt^{2}} \frac{t^{2}}{2} + \frac{d^{3}x(0)}{dt^{3}} \frac{t^{3}}{3!} + \frac{d^{4}x(0)}{dt^{4}} \frac{t^{4}}{4!} + \frac{d^{5}x(0)}{dt^{5}} \frac{t^{5}}{5!} + ⋯
= 1 + t − \frac{3}{2}t^{2} + \frac{1}{2}t^{3} − \frac{7}{8}t^{4} + \frac{117}{120}t^{5} + · · · .

We now illustrate the method of Taylor series when applied to a system of first-order differential equations.

Example 3.19 We consider the following system of two first-order equations

\begin{matrix}\frac{dx_{1}}{dt} & = & f_{1}(t, x_{1}, x_{2}) \\ \frac{dx_{2}}{dt} & = & f_{2}(t, x_{1}, x_{2}) \end{matrix} \}

The initial conditions at t = t_{0} are: x_{1}(t_{0}) = x_{10}, x_{2}(t_{0}) = x_{20}. Having the initial conditions, we can determine the values \frac{dx_{1}(t_{0})}{dt} and \frac{dx_{2}(t_{0})}{dt} from the original equations. We then differentiate \frac{dx_{1}}{dt} and \frac{dx_{2}}{dt} to obtain

\frac{d^{2}x_{1}}{dt^{2}} = \frac{d}{dt}f_{1} (t, x_{1}, x_{2}, \frac{dx_{1}}{dt} , \frac{dx_{2}}{dt} )
\frac{d^{2}x_{2}}{dt^{2}} = \frac{d}{dt}f_{2} (t, x_{1}, x_{2}, \frac{dx_{1}}{dt} , \frac{dx_{2}}{dt} ) .

Now, \frac{d^{2}x_{1}}{dt^{2}} and \frac{d^{2}x_{2}}{dt^{2}} can be calculated by substituting known quantities on the right-hand sides of the above equations. By successive differentiation and substitution, we can determine \frac{d^{3}x_{1}}{dt^{3}} and \frac{d^{3}x_{2}}{dt^{3}} as well as higher derivatives at t = t_{0}. The solution to the given system is then computed as

x_{1}(t) = x_{10} + \frac{dx_{1}(t_{0})}{dt}(t − t_{0}) + \frac{d^{2}x_{1}(t_{0})}{dt^{2}} \frac{(t − t_{0})^{2}}{2} + \frac{d^{3}x_{1}(t_{0})}{dt^{3}} \frac{(t − t_{0})^{3}}{3!} + ⋯
x_{2}(t) = x_{20} + \frac{dx_{2}(t_{0})}{dt}(t − t_{0}) + \frac{d^{2}x_{2}(t_{0})}{dt^{2}} \frac{(t − t_{0})^{2}}{2} + \frac{d^{3}x_{2}(t_{0})}{dt^{3}} \frac{(t − t_{0})^{3}}{3!} + ⋯

The Taylor series method can be programmed on a computer. However, this is not very efficient method from numerical point of view. In the following, we discuss other methods for numerical solution of the state equations that are related to the method of Taylor series. We first present two simple numerical techniques known as the forward and backward Euler methods.

3.7.2 Forward Euler Method

Suppose that we wish to obtain the solution of the modeling equation

ẋ(t) = f(t, x(t), u(t))

in the time interval [ t_{0}, t_{f} ], subject to the initial condition x(t_{0}) = x_{0}, and the input vector u(t). We divide the interval [ t_{0}, t_{f} ] into N equal subintervals of width

h = \frac{t_{f} − t_{0}}{N} .

We call h the step length. We set

t_{k} = t_{0} + kh.

To proceed further, we approximate the derivative \frac{dx(t)}{dt} at time t_{k} by

\frac{dx(t_{k})}{dt} ≈ \frac{x(t_{k+1}) − x(t_{k})}{h},

and write

\frac{x(t_{k+1}) − x(t_{k})}{h} = f(t_{k}, x(t_{k}), u(t_{k})).

Hence, for k = 1, 2, …, N, we have

x(t_{k+1}) = x(t_{k}) + hf(t_{k}, x(t_{k}), u(t_{k}))  (3.49)

Figure 3.9: Illustration of the rectangular rule of integration.

The above is known as the forward Euler algorithm. We see that if h is sufficiently small, we can approximately determine the state at time t_{1} = t_{0} + h from the initial condition x_{0} to get

x(t_{1}) = x_{0} + hf(t_{0}, x_{0}, u(t_{0})).

Once we have determined the approximate solution at time t_{1}, we can determine the approximate solution at time t_{2} = t_{1} + h, and so on. The forward Euler method can also be arrived at when instead of considering the differential equation ẋ(t) = f(t, x(t), u(t)), we rather start with its equivalent integral representation

x(t) = x_{0} + ∫_{t_{0}}^{t} f(τ, x(τ), u(τ))dτ.  (3.50)

We then use the rectangular rule for numerical integration that can be stated as follows. The area under the curve z = g(t) between t = t_{k} and t = t_{k} + h is approximately equal to the area of the rectangle ABCD as shown in Figure 3.9, that is,

∫_{t_{k}}^{t_{k}+h} g(t)dt ≈ hg(t_{k}).  (3.51)

Applying the rectangular rule of integration to (3.50), we obtain the forward Euler method.

Figure 3.10: Rotating a point mass from α to α + θ.

Example 3.20 We combine basic MATLAB’s handle graphics commands with the forward Euler method to write a simple program that animates the motion of a simple pendulum. First, we present a few basic definitions from handle graphics. We start with a description of a handle—it is a floating-point number that MATLAB assigns to every object in the figure window such as a line, tex, label, axes, figure, etc. Thus a handle can be viewed as an object’s ID.

Each object has different properties. If we wish to change a property, we need to get hold of the object in question. We use object’s handle to get hold of it and then change the object’s property using the following commands,

get(handle,'PropertyName')
set(handle,'PropertyName','PropertyValue')

In our animation of the motion of a pendulum, we use the rotation matrix that we discuss next. Suppose that we have a point mass on a circle of radius r as shown in Figure 3.10. The point mass is connected to the circle center; we call it hinge. The line segment, bar, connecting the hinge and the mass forms the angle α. Therefore, the point mass has the coordinates

x_{present} = [ \begin{matrix}r cos(α) \\ r sin(α) \end{matrix} ] .

Suppose that the bar was rotated by an angle θ from its initial displacement angle α. The new angle of the bar, with the positive x-axis, is then α+θ. We use the following well-known trigonometric identities:

cos(α + θ) = cos(α) cos(θ) − sin(α) sin(θ)
sin(α + θ) = sin(α) cos(θ) + cos(α) sin(θ),

to obtain the mass new coordinates,

x_{next} = [ \begin{matrix}cos(θ) & − sin(θ) \\ sin(θ) & cos(θ) \end{matrix} ] x_{present}
= [ \begin{matrix}cos(θ) & − sin(θ) \\ sin(θ) & cos(θ) \end{matrix} ] [ \begin{matrix}r cos(α) \\ r sin(α) \end{matrix} ] .

Equipped with the above background results, we are ready to animate the motion of the simple pendulum from Example 2.1. Let

x_{1} = θ and x_{2} = θ̇.

Then, we represent the second-order differential equation (2.32) as a system of first-order differential equations,

ẋ_{1} = x_{2}
ẋ_{2} = −5 sin(x_{1}).

We approximate the above system of differential equations using Euler’s forward method,

\frac{x_{1next} − x_{1present}}{h} = x_{2present}
\frac{x_{2next} − x_{2present}}{h} = −5 sin(x_{1present}),

where h is the step size. Combining the above with the handle graphics gives the following simple MATLAB program that animates the simple pendulum motion. This code is based on the code of Example 3 in Pratap [279, p. 207].

theta=input('Initial angular displacement of the pendulum in deg= ');
theta=theta*pi/180;           % convert from degrees to radians
tfinal=input('Final time tfinal = ');
data_init=[0 -2]';            % pendulum at rest
R=[cos(theta) -sin(theta);sin(theta) cos(theta)];    % rotation matrix
data=R*data_init;             % initial pendulum position
axis(2*[-1 1 -1 1])
axis('equal')
axis('off')
bar=line('xdata',[0 data(1)],'ydata',[0 data(2)]','linewidth',7,...
    'erase','xor');

mass=line('xdata',data(1),'ydata',data(2),'linestyle','.',...
    'markersize',50,'erase','xor');
hinge=line('xdata',0,'ydata',0,'linestyle','o','markersize',12,...
    'erase','xor');
h=0.0025;     % step-size for solving differential
t=0;           % initial time
thetadot=0;    % initial angular speed
% solve the differential eqns using the Euler's method
while(t<tfinal)
   t=t+h;
   theta = theta + thetadot*h;
   thetadot=thetadot - 5*sin(theta)*h;
   R=[cos(theta) -sin(theta);sin(theta) cos(theta)];
   datanew=R*data_init;
% change the property values of the bar and hinge objects
   set(bar,'xdata',[0 datanew(1)],'ydata',[0 datanew(2)]);
   set(mass,'xdata',datanew(1),'ydata',datanew(2));
   set(hinge,'xdata',0,'ydata',0)
   drawnow;
end

3.7.3 Backward Euler Method

The backward Euler method differs from the forward Euler method in the way we approximate the derivative,

\frac{dx(t_{k})}{dt} ≈ \frac{x(t_{k}) − x(t_{k−1})}{h},

This approximation gives the backward Euler formula,

x(t_{k+1}) = x(t_{k}) + hf(t_{k+1}, x(t_{k+1}), u(t_{k+1}))

In the above formula, x(t_{k+1}) is a function of itself. For this reason, we say that the backward Euler method is an implicit integration algorithm. Implicit integration algorithms require

Figure 3.11: The trapezoidal rule of integration.

additional computation to solve for x(t_{k+1}). For further discussion of this issue, we refer to Parker and Chua [264].

Euler’s methods are rather elementary, and thus, not as accurate as some of the more sophisticated techniques that we are going to discuss next.

3.7.4 Predictor-Corrector Method

The predictor-corrector method, also known as Heun’s method, is based on the trapezoidal rule for numerical integration. The trapezoidal rule states that the area under the curve z = g(t) between t = t_{k} and t = t_{k} + h is approximately equal to the area of the trapezoid ABCD as depicted in Figure 3.11, that is,

∫_{t_{k}}^{t_{k}+h} g(t)dt ≈ h\frac{g(t_{k}) + g(t_{k} + h)}{2}.  (3.52)

Suppose we wish to find x(t) at t = t_{1} > t_{0}. Applying the trapezoidal rule of integration to (3.48), we obtain

x(t_{1}) = x_{0} + \frac{h}{2} (f(t_{0}, x_{0}, u(t_{0})) + f(t_{1}, x(t_{1}), u(t_{1}))) .

The above is an implicit algorithm and it may be viewed as a merging of the forward and backward Euler algorithms. If the function f is nonlinear, then we will, in general, not be able to solve the above equation for x(t_{1}) exactly. We attempt to obtain x(t_{1}) iteratively. Let

h = t_{k+1} − t_{k},
x_{1}^{∗} = x_{0} + hf(t_{0}, x_{0}, u(t_{0})),
m_{0} = f(t_{0}, x_{0}, u(t_{0})),
m_{1} = f(t_{1}, x_{1}^{∗}, u(t_{1})).

A first approximation to x(t_{1}) is

x_{1}^{∗} = x_{0} + hf(t_{0}, x_{0}, u(t_{0}))

and is called a predictor. The next approximation is

x(t_{1}) = x_{0} + \frac{h}{2} (m_{0} + m_{1})

and is called a corrector.

Suppose now that we wish to find x(t) at t = t_{f} > t_{0}. We first divide the interval [ t_{0}, t_{f} ] into N equal subintervals using evenly spaced subdivision points to obtain

t_{0} < t_{1} < t_{2} < … < t_{N} = t_{f}.

Once we have found x(t_{1}), we can apply the same procedure to find x(t_{2}), and so on.

Example 3.21 We use the predictor-corrector method to find x(1), where the step length h = 0.5, for the following differential equation

\frac{dx}{dt} = tx^{2} − x, x(0) = 1.  (3.53)

The calculations are summarized in Table 3.1.

We could use x(t_{1}) obtained from the corrector formula to construct a next approximation to x(t_{1}). In general, we can use the formula

x^{(k)}(t_{1}) = x_{0} + \frac{h}{2} (f(t_{0}, x_{0}, u(t_{0})) + f(t_{1}, x^{(k−1)}(t_{1}), u(t_{1}))

to evaluate x(t_{1}) until two successive iterations agree to the desired accuracy. This general algorithm is referred to as a second-order predictor-corrector method.

Table 3.1: Solving the differential equation (3.53) using the predictor-corrector method.

t_{0} x_{0} t_{1}  m_{0}  x_{1}^{∗}  m_{1} m = \frac{m_{0}+m_{1}}{2} Δx
0  1 0.5 −1.0 0.5 −0.375 −0.688 −0.344
0.5 0.656 1.0 −0.441 0.435 −0.246 −0.344 −0.172
1.0 0.484

An n-dimensional system of first-order differential equations can be worked analogously. We illustrate the procedure for n = 2, where

\begin{matrix}\frac{dx}{dt} & = & f(t, x, y), x(t_{0}) = x_{0} \\ \frac{dy}{dt} & = & g(t, x, y), y(t_{0}) = y_{0} \end{matrix} \}

Let

m_{0} = f(t_{0}, x_{0}, y_{0}),
n_{0} = g(t_{0}, x_{0}, y_{0}).

Then, the predictor equations are

x_{1}^{∗} = x_{0} + m_{0} h,
y_{1}^{∗} = y_{0} + n_{0} h.

Let

m_{1} = f(t_{1}, x_{1}^{∗}, y_{1}^{∗}),
n_{1} = g(t_{1}, x_{1}^{∗}, y_{1}^{∗}).

Then, the corrector equations are

\begin{matrix}x(t_{1}) & = & x_{0} + \frac{m_{0}+m_{1}}{2} h \\ y(t_{1}) & = & y_{0} + \frac{n_{0}+n_{1}}{2} h \end{matrix} \}

Using the above, we can generate the corresponding formulas for t_{k}.

We now show that the predictor-corrector method yields a solution that agrees with the Taylor series solution through terms of degree two. We consider the scalar case, where f is assumed to be differentiable with respect to its arguments and ∂^{2}f/∂t∂x = ∂^{2}f/∂x∂t. First observe that m_{1} = f (t_{1}, x_{1}^{∗}) can be written as

f (t_{1}, x_{1}^{∗}) = f (t_{0} + h, x_{0} + m_{0}h)
≈ f (t_{0}, x_{0}) + [ f_{t} f_{x} ] [ \begin{matrix}h \\ m_{0}h \end{matrix} ] + \frac{1}{2} [ h m_{0}h ] [ \begin{matrix}f_{tt} & f_{tx} \\ f_{xt} & f_{xx} \end{matrix} ] [ \begin{matrix}h \\ m_{0}h \end{matrix} ] ,

where f_{t} = ∂f/∂t, f_{x} = ∂f/∂x, and so on. Substituting the above into the corrector equation yields

x(t_{1}) = x_{0} + \frac{h}{2} (m_{0} + m_{1})
= x_{0} + \frac{h}{2} (m_{0} + m_{0}) + \frac{h}{2} ([ f_{t} f_{x} ] [ \begin{matrix}h \\ m_{0}h \end{matrix} ]

+ \frac{1}{2} [ h m_{0}h ] [ \begin{matrix}f_{tt} & f_{tx} \\ f_{xt} & f_{xx} \end{matrix} ] [ \begin{matrix}h \\ m_{0}h \end{matrix} ])

= x_{0} + m_{0}h + (f_{t} + f_{x}m_{0}) \frac{h^{2}}{2} + (f_{tt} + 2f_{tx}m_{0} + f_{xx}m_{0}^{2}) \frac{h^{3}}{4} .

The Taylor series expansion, on the other hand, gives

x (t_{1}) = x_{0} + ẋ h + ẍ \frac{h^{2}}{2} + x^{(3)} \frac{h^{3}}{6} + ⋯
= x_{0} + f(t_{0}, x_{0}) h + (f_{t} + f_{x}ẋ) \frac{h^{2}}{2}

+ (f_{tt} + 2f_{tx}ẋ + f_{xx} \binom{}{ẋ}^{2} + f_{x}ẍ) \frac{h^{3}}{6} + ⋯,

where f_{t}, f_{x}, ẋ, etc. are all evaluated at t = t_{0}. Comparing the right-hand sides of the last two equations, we see that the solution obtained using the predictor-corrector method agrees with the Taylor series expansion of the true solution through terms of degree two.

3.7.5 Runge’s Method

Runge’s method is an improvement over the predictor-corrector method in the sense that it agrees with the Taylor series solution through terms of degree three. The method depends upon integration by means of Simpson’s rule, which says that the area under the curve x = g(t) between t_{0} and t_{0} + h is given approximately by

∫_{t_{0}}^{t_{0}+h} g(t)dt ≈ \frac{h}{6} (x_{0} + 4x_{1} + x_{2}) ,  (3.54)

Figure 3.12: Integration using Simpson’s rule.

where x_{0} = g(t_{0}), x_{1} = g (t_{0} + h/2), and x_{2} = g(t_{0} + h). The above formula is derived by passing a parabola through the three points (t_{0}, x_{0}), (t_{0} + h/2, x_{1}), and (t_{0} + h, x_{2}), and assuming that the area under the given curve is approximately equal to the area under the parabola as it is illustrated in Figure 3.12. To verify Simpson’s formula, we translate the axes so that in the new coordinates (t̃, x̃) we have t = t_{0} + h/2 + t̃, x = x̃. In the new coordinates the three points become

(−h/2, x_{0}) , (0, x_{1}) , (h/2, x_{2}) .

The parabola through the above points is described by the equation

x̃ = at̃^{2} + bt̃+ c.

Hence

x_{0} = a\frac{h^{2}}{4} − b\frac{h}{2} + c,
x_{1} = c,
x_{2} = a\frac{h^{2}}{4} + b\frac{h}{2} + c.

176 CHAPTER 3. ANALYSIS OF MODELING EQUATIONS Using the above three equations, we calculate

c = x_{1},
b = \frac{x_{2} − x_{0}}{h} ,
a = \frac{2}{h^{2}} (x_{0} − 2x_{1} + x_{2}) .

Then, the area under the parabola is

∫_{−h/2}^{h/2} (at̃^{2} + bt̃+ c) dt̃ = ∫_{−h/2}^{h/2} (\frac{2}{h^{2}} (x_{0} − 2x_{1} + x_{2}) t̃^{2} + \frac{x_{2} − x_{0}}{h} t̃+ x_{1}) dt̃
= \frac{h}{6} (x_{0} + 4x_{1} + x_{2}) .

If we are given the differential equation

\frac{dx}{dt} = f(t, x), x(t_{0}) = x_{0},

we integrate it to obtain

x(t_{1}) = x(t_{0}) + ∫_{t_{0}}^{t_{1}} f(s, x(s))ds.

Then, we approximate the integral

∫_{t_{0}}^{t_{1}} f(s, x(s))ds

by

\frac{h}{6} (m_{0} + 4m_{1} + m_{3}) ,

where

h = t_{1} − t_{0},
m_{0} = f(t_{0}, x_{0}),
m_{1} = f (t_{0} + \frac{h}{2}, x_{0} + m_{0} \frac{h}{2}) ,
m_{2} = f (t_{0} + h, x_{0} + m_{0}h) ,
m_{3} = f (t_{0} + h, x_{0} + m_{2}h) .

Thus, we may write

x(t_{1}) = x_{0} + Δx,

where Δx = mh, and m = \frac{1}{6} (m_{0} + 4m_{1} + m_{3}).

Table 3.2: Solving the differential equation (3.53) using Runge’s method.

t_{0} x_{0}  m_{0}  m_{1}  m_{2}  m_{3}  m  Δx
0  1  −1 −0.610 −0.375 −0.482 −0.654 −0.327
0.5 0.673 −0.446 −0.325 −0.248 −0.248 −0.332 −0.166
1.0 0.507

Example 3.22 We use Runge’s method to find x(1), where the step length h = 0.5, for the differential equation (3.53). We found x(1) using the predictor-corrector method in Example 3.21. The calculations for Runge’s method are summarized in Table 3.2.

Runge’s method can, of course, be used for systems of first-order differential equations. If, for example,

\frac{dx}{dt} = f(t, x, y), x(t_{0}) = x_{0},
\frac{dy}{dt} = g(t, x, y), y(t_{0}) = y_{0},

then

x(t_{1}) = x_{0} + Δx,
y(t_{1}) = y_{0} + Δy,

where

Δx = \frac{h}{6} (m_{0} + 4m_{1} + m_{3}) ,
Δy = \frac{h}{6} (n_{0} + 4n_{1} + n_{3}) .

In the above expressions,

m_{0} = f(t_{0}, x_{0}, y_{0}),
m_{1} = f (t_{0} + h/2, x_{0} + m_{0} h/2, y_{0} + n_{0} h/2) ,
m_{2} = f (t_{0} + h, x_{0} + m_{0}h, y_{0} + n_{0}h) ,
m_{3} = f (t_{0} + h, x_{0} + m_{2}h, y_{0} + n_{2}h) ,
n_{0} = g(t_{0}, x_{0}, y_{0}),
n_{1} = g (t_{0} + h/2, x_{0} + m_{0} h/2, y_{0} + n_{0} h/2) ,
n_{2} = g (t_{0} + h, x_{0} + m_{0}h, y_{0} + n_{0}h) ,
n_{3} = g (t_{0} + h, x_{0} + m_{2}h, y_{0} + n_{2}h) .

3.7.6 Runge-Kutta Method

This method is an improvement over the Runge method, and agrees with the Taylor series solution through terms of degree four—for this reason it is sometimes called the fourth-order Runge-Kutta algorithm. It is based on the use of the formula

x(t_{1}) = x(t_{0}) + \frac{h}{6} (m_{0} + 2m_{1} + 2m_{2} + m_{3}) ,  (3.55)

where

m_{0} = f(t_{0}, x_{0}),
m_{1} = f (t_{0} + h/2, x_{0} + m_{0} h/2) ,
m_{2} = f (t_{0} + h/2, x_{0} + m_{1} h/2) ,
m_{3} = f (t_{0} + h, x_{0} + h m_{2}) .

Thus, we may write

x(t_{1}) = x_{0} + Δx,

where Δx = mh, and m = \frac{1}{6} (m_{0} + 2m_{1} + 2m_{2} + m_{3}).

Example 3.23 We apply the Runge-Kutta method to find x(1), where the step length h = 0.5, for the differential equation (3.53) that we used Examples 3.21 and 3.22. Our calculations for the Runge-Kutta method are summarized in Table 3.3.

Table 3.3: Solving the differential equation (3.53) using the Runge-Kutta method.

t_{0} x_{0}  m_{0}  m_{1}  m_{2}  m_{3}  m  Δx
0  1  −1.0 −0.609 −0.668 −0.444 −0.666 −0.333
0.5 0.667 −0.444 −0.324 −0.328 −0.250 −0.333 −0.167
1.0 0.500

The above problem has been worked by the predictor-corrector, Runge’s, and the Runge-Kutta methods. This problem can be solved exactly by means of the Taylor series. The Taylor series solution is

x(t) = 1 − t + t^{2} − t^{3} + ⋯

The series converges to 1/(1 + t) for −1 < t < 1. A check on the differential equation shows that

x(t) = \frac{1}{1 + t}

is indeed a solution for all t > −1. Thus, x(1) = 0.500.

The algorithms presented above require only one input point, x(t_{k}), at each step to compute x(t_{k+1}). Such algorithms are referred to as single-step algorithms. In the multistep algorithms, a number of previous points are used, and the corresponding values of f, to evaluate x(t_{k+1}), that is, the multi-step algorithms reuse past information about the trajectory when evaluating its new point. In general, it may be difficult to decide which type of an algorithm is more efficient because the performance of a particular algorithm is problem dependent. Well known multi-step algorithms are: Adams-Bashforth, Adams-Moulton, and Gear. More information about the above mentioned algorithms can be found in [264, Section 4.1].

3.8 Notes

The system of differential equations (3.56) in Exercise 3.11 comes from Boyce and DiPrima [40, page 523]. Al-Khafaji and Tooley [6] provide an introductory treatment of numerical solutions of differential equations. For further discussion of solving differential equations using numerical methods and analyses of their computational efficiency, the reader is referred to Salvadori and Baron [292], Conte and de Boor [65, Chapter 8], or Parker and Chua [264].

3.9 Exercises

Exercise 3.1 Suppose that x = [ 1 2 3 ]^{⊤} is the representation of the given vector in the natural basis {e_{1}, e_{2}, e_{3}}. Find the representation of this vector in the basis {e′_{1}, e′_{2}, e′_{3}}, where

e′_{1} = e_{1} + e_{3}
e′_{2} = e_{2} − e_{3}
e′_{3} = e_{3}.

Exercise 3.2 Prove Theorem 3.1.

Exercise 3.3 Find a similarity transformation that transforms

A = [ \begin{matrix}λ & 0 & 0 & 0 \\ 0 & λ & 0 & 1 \\ 1 & 0 & λ & 0 \\ 0 & 0 & 1 & λ \end{matrix} ]

into its Jordan form. Do not fix λ; construct a transformation that works for any λ. Verify your calculations using MATLAB’s Symbolic Toolbox.

Exercise 3.4 Calculate the solution for the system,

ẋ = [ \begin{matrix}0 & −1 \\ 1 & −1 \end{matrix} ] x + [ \begin{matrix}1 \\ 0 \end{matrix} ] u, x(0) = [ \begin{matrix}x_{1}(0) \\ x_{2}(0) \end{matrix} ] ,

where u(t) is the unit step, that is, u(t) = 1(t). Plot the system trajectories on the time interval [0, 2] for x(0) = 0.

Exercise 3.5 Construct an exact linear discrete state-space model for the system,

ẋ = [ \begin{matrix}0 & −1 \\ 1 & −1 \end{matrix} ] x + [ \begin{matrix}1 \\ 0 \end{matrix} ] u
y = [ 0 1 ] x

The sampling period is h = 0.2 sec. Verify you calculations using MATLAB’s function c2d. Find the transfer function of the discrete systems and its step response. Plot the step response of the discrete system on the time interval [0, 2 sec].

Figure 3.13: A trajectory for Exercise 3.9.

Exercise 3.6 Sketch a phase-plane portrait of the differential equation

ẍ − ẋ + 2x + 2sign (2x + ẋ) = 0.

Exercise 3.7 For the system of differential equations

\begin{matrix}ẋ_{1} & = & −\frac{7}{4}x_{1} + \frac{1}{4}x_{2} \\ ẋ_{2} & = & \frac{3}{4}x_{1} − \frac{5}{4}x_{2} \end{matrix} \}

find the phase-plane asymptotes. Exercise 3.8 For a dynamical system modeled by

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & −x_{1} \end{matrix} \}

find the time T it takes the system trajectory to move from the state x(0) = [ 0 1 ]^{T} to the state x(T) = [ 1 0 ]^{T}. Exercise 3.9 Consider a trajectory in the state plane shown in Figure 3.9. Find the time it takes the representative point to move from A to B. Exercise 3.10 Given the following initial value problem

ẋ = f(t, x), x(t_{0}) = x_{0}.

(i) Derive the Euler formula for solving the above initial value problem using a Taylor series.

(ii) Make use of a Taylor series with a remainder to compute the local error formula assuming that the data at the n-th step are correct.

Exercise 3.11 Use Bendixson’s theorem to investigate the existence of limit cycles for the system

\begin{matrix}ẋ_{1} & = & x_{1} + x_{2} − x_{1} (x_{1}^{2} + x_{2}^{2}) \\ ẋ_{2} & = & −x_{1} + x_{2} − x_{2} (x_{1}^{2} + x_{2}^{2}) \end{matrix} \}  (3.56)

Sketch a phase-plane portrait for this system.

Exercise 3.12 Use Bendixson’s theorem to investigate the existence of limit cycles for the system

\begin{matrix}ẋ_{1} & = & x_{2} − x_{1} (x_{1}^{2} + x_{2}^{2} − c) \\ ẋ_{2} & = & −x_{1} − x_{2} (x_{1}^{2} + x_{2}^{2} − c) \end{matrix} \}  (3.57)

where c is a constant Sketch phase-plane portraits for this system for c = −0.2 and c = 0.2. The above nonlinear system was analyzed in [264, page 204].

Exercise 3.13 Verify that the Runge method agrees with the Taylor series solution through terms of degree three.

Exercise 3.14 Verify that the Runge-Kutta method agrees with the Taylor series solution through terms of degree four.

Linearization

Early amplifiers were made of crude materials that tended to disintegrate over use, causing the amp to “run away.” Not only would an aging relay amplify the phone signal, it would mistakenly compound any tiny deviation from the range it expected until the mushrooming error filled and killed the system. What was needed was Heron’s regula, a counter signal to rein in the chief signal, to dampen the effect of the perpetual recycling. Black came up with a negative feedback loop, which was designated negative in contrast to the snowballing positive loop of the amplifier. Conceptually, the electrical negative feedback loop is a toilet flusher or thermostat. This braking circuit keeps the amplifier honed in on a steady amplification in the same way a thermostat hones in on a steady temperature. But instead of metallic levers, a weak train of electrons talks to itself. [175, page 116]

The replacement of a nonlinear system model by its linear approximation is called linearization. The motivation for linearization is that the dynamical behavior of many nonlinear system models can be well approximated within some range of variables by linear system models. Then, we can use well developed techniques for analysis and synthesis of linear systems to analyze a nonlinear system at hand. However, the results of analysis of nonlinear systems using their linearized models should be carefully interpreted in order to avoid unacceptably large errors due to the approximation in the process of linearization. We present two different approaches to linear approximation of nonlinear systems.

4.1 Taylor Linearization

We begin our discussion of linear approximations of nonlinear systems by considering a nonlinear element with a state variable x and an output variable y that are related by the equation

y = h(x),

where the function h : ℝ → ℝ is continuously differentiable, that is, h ∈ C^{1}. Let x_{0} be a point of interest that we refer to as an operating point. We expand h into a Taylor series about the operating point x_{0} to obtain

y = h(x)
= h(x_{0}) + \frac{dh(x_{0})}{dx}(x − x_{0}) + higher order terms.

Linearization of h(x) about the point x_{0} consists of replacing h by the linear approximation

y = h(x_{0}) + \frac{dh(x_{0})}{dx}(x − x_{0})
= y_{0} + \frac{dh(x_{0})}{dx}(x − x_{0}),  (4.1)

where y_{0} = h(x_{0}). Let Δy = y − y_{0} and Δx = x − x_{0}. Then, we can represent (4.1) as

Δy = \frac{dh(x_{0})}{dx}Δx.

Over a small range of Δx, the line (4.1) is a good approximation to the curve y = h(x) in a neighborhood of the operating point x_{0}—see Figure 4.1 for an illustration of the above statement. We illustrate the process of linearization on the simple pendulum nonlinear model.

Example 4.1 Consider the simple pendulum shown in Figure 4.2. Let g be the gravity constant. The tangential force component, mg sin θ, acting on the mass m returns the pendulum to its rest, or equilibrium position. By Newton’s second law

τ = −mgl sin θ,  (4.2)

where τ = Iθ̈ is the torque acting on the mass m and I = ml^{2} is the moment of inertia of the mass m.

We are interested in linearizing the above pendulum model about its rest position, or equilibrium; call it θ_{e}. At such a point, τ(θ_{e}) = 0.

4.1. TAYLOR LINEARIZATION 185

Figure 4.1: Linear approximation to a function y = h(x). Figure 4.2: Forces acting on the simple pendulum.

Figure 4.3: Linearizing the equation modeling the simple pendulum.

There are many equilibrium points of the pendulum model (4.2). We select θ_{e} = 0^{∘}. Note that τ_{e} = τ (0^{∘}) = 0. Therefore,

Δτ = τ − τ_{e} = τ and Δθ = θ − θ_{e} = θ.

We also have

\frac{dτ}{dθ} (0^{∘}) = −mgl cos 0^{∘} = −mgl.

Hence, the linearized model, about the equilibrium position θ = 0^{∘} has the form

τ = −mglθ.

Thus, for small angular displacements θ, the torque τ is proportional to the angular displacements. This approximation is quite accurate for −π/4 ≤ θ ≤ π/4 as can be seen in Figure 4.3.

If h : ℝ^{n} → ℝ, that is, y = h(x_{1}, x_{2}, …, x_{n}) which means that the dependent variable depends upon several variables, the same principle applies. Let

x_{0} = [ x_{10} x_{20} ⋯ x_{n0} ]^{⊤}

be the operating point. The Taylor series expansion of h about the operating point x_{0} yields

y − h(x_{0}) = ∇h(x_{0})^{⊤}(x − x_{0}) + higher order terms,

Figure 4.4: Tangent plane as a linear approximation for a function of two variables.

where

∇h(x_{0})^{⊤} = [ \frac{∂h}{∂x_{1}} |_{x=x_{0}} \frac{∂h}{∂x_{2}} |_{x=x_{0}} ⋯ \frac{∂h}{∂x_{n}} |_{x=x_{0}} ] .

Geometrically, the linearization of h about x_{0} can be thought of as placing a tangent plane onto the nonlinear surface at the operating point x_{0} as illustrated in Figure 4.4.

4.2 Linearizing Differential Equations

4.2.1 Time-Invariant Unforced System

We consider a dynamical system modeled by a set of first-order differential equations,

\frac{dx_{1}}{dt} = f_{1}(x_{1}, x_{2}, …, x_{n}) ; \frac{dx_{2}}{dt} = f_{2}(x_{1}, x_{2}, …, x_{n}) ; ⋮ ; \frac{dx_{n}}{dt} = f_{n}(x_{1}, x_{2}, …, x_{n}) \}

We assume that the functions f_{i}, i = 1, 2, …, n are continuously differentiable. We represent the above system of differential equations in vector form as

ẋ = f(x).  (4.3)

Note that the right-hand side of the above modeling equations does not depend explicitly on time. Thus (4.3) is a time-invariant system model. We discuss linearizing time-varying systems in Subsection 4.2.2.

We are interested in constructing a linear model approximation of the above nonlinear model in a vicinity of a point of interest, which is an equilibrium point of the model.

Definition 4.1 A point x_{e} is an equilibrium state, or an equilibrium point, of the system modeled by (4.3) if it has the property that when the system starts at x_{e}, it will stay at x_{e} for all future time. The equilibrium points are obtained by solving the algebraic equation

f(x) = 0.
◇

A nonlinear system may have a number of equilibrium states.

Example 4.2 The nonlinear system model

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}x_{2} \\ x_{1} − x_{2} − x_{1}^{2} \end{matrix} ] ,

has two equilibrium states

x_{e}^{(1)} = [ \begin{matrix}0 \\ 0 \end{matrix} ] and x_{e}^{(2)} = [ \begin{matrix}1 \\ 0 \end{matrix} ] .

We are interested here in isolated equilibrium states; these are the equilibrium states that have no other equilibrium states in some neighborhood. In the above example, the equilibrium states are isolated. We can formally define an isolated equilibrium state using notion of the r-neighborhood.

Definition 4.2 The r-neighborhood is a set of points

{x : ∥x − x_{e}∥ < r},

where ∥ · ∥ can be any p-norm on ℝ^{n}—see Section A.2 in the Appendix.

◇

Having the above definition, we formally define an isolated equilibrium state.

Definition 4.3 An equilibrium state x_{e} is isolated if there is an r > 0 such that the rneighborhood of x_{e} contains no equilibrium state other than x_{e}.

◇

We note that a linear system, ẋ = Ax, can have only one isolated equilibrium state. Note also that if there two different points, x_{e}^{(1)} and x_{e}^{(2)} in ℝ^{n}, that are equilibrium states of the linear system, then so is any point on the line connecting these two points.

To proceed, we expand the right-hand side of (4.3) into a Taylor series about an isolated equilibrium state x_{e} to obtain

\frac{d}{dt}x = f(x_{e}) + \frac{∂f}{∂x}(x_{e}) Δx + higher-order terms,  (4.4)

where \frac{∂f}{∂x}(x_{e}) is the Jacobian matrix of f with respect x, evaluated at the equilibrium state x_{e} and Δx = x − x_{e}. Observe that

\frac{d}{dt}x = \frac{d}{dt}x_{e} + \frac{d}{dt}Δx = \frac{d}{dt}Δx

because x_{e} is constant. Taking the above into account and the fact that f(x_{e}) = 0, we obtain a linear approximation of (4.4),

\frac{d}{dt}Δx = \frac{∂f}{∂x}(x_{e}) Δx

Let

A = \frac{∂f}{∂x}(x_{e}).

Then, a linear model of (4.3) is written as

\frac{d}{dt}Δx = A Δx.  (4.5)

Example 4.3 For the nonlinear dynamical system model,

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}2 − x_{1}x_{2} \\ −4 + x_{1}^{2} \end{matrix} ] ,

• find the equilibrium states;

• construct the corresponding linear models about the equilibrium states found above.

190 CHAPTER 4. LINEARIZATION

• To find equilibrium states, we solve the algebraic equations, ẋ_{1} = ẋ_{2} = 0, that is,

2 − x_{1}x_{2} = 0 and − 4 + x_{1}^{2} = 0.

We have two equilibriums states,

x_{e}^{(1)} = [ \begin{matrix}2 \\ 1 \end{matrix} ] and x_{e}^{(2)} = [ \begin{matrix}−2 \\ −1 \end{matrix} ] .

• A linear model has the form,

\frac{d}{dt}Δx = A Δx,

where

A = [ \begin{matrix}\frac{∂f_{1}}{∂x_{1}} & \frac{∂f_{1}}{∂x_{2}} \\ \frac{∂f_{2}}{∂x_{1}} & \frac{∂f_{2}}{∂x_{2}} \end{matrix} ] = [ \begin{matrix}−x_{2} & −x_{1} \\ 2x_{1} & 0 \end{matrix} ]

evaluated at the equilibrium state about which we linearize the nonlinear system. We have two linear models:

1. about x_{e}^{(1)},

\frac{d}{dt}Δx = [ \begin{matrix}−1 & −2 \\ 4 & 0 \end{matrix} ] Δx,

2. about x_{e}^{(2)},

\frac{d}{dt}Δx = [ \begin{matrix}1 & 2 \\ −4 & 0 \end{matrix} ] Δx.

4.2.2 Time-Varying Forced System

We now consider a dynamical system modeled by a set of nonlinear differential equations

\frac{dx_{1}}{dt} = f_{1}(t, x_{1}, x_{2}, …, x_{n}, u_{1}, u_{2}, …, u_{m}) ; \frac{dx_{2}}{dt} = f_{2}(t, x_{1}, x_{2}, …, x_{n}, u_{1}, u_{2}, …, u_{m}) ; ⋮ ; \frac{dx_{n}}{dt} = f_{n}(t, x_{1}, x_{2}, …, x_{n}, u_{1}, u_{2}, …, u_{m}) \}

The above set of differential equations can be represented in vector form as

ẋ = f(t, x, u).  (4.6)

Let u_{e} = [ u_{1e} u_{2e} ⋯ u_{me} ]^{⊤} be a constant input that forces the system (4.6) to settle

into a constant, equilibrium, state x_{e} = [ x_{1e} x_{2e} ⋯ x_{ne} ]^{⊤} at time t_{0}, that is, u_{e} and x_{e} satisfy

f(t, x_{e}, u_{e}) = 0 for all t ≥ t_{0}.

We say that the pair (x_{e}, u_{e}) is the equilibrium pair at t_{0}. We note that if the pair (x_{e}, u_{e}) is an equilibrium pair at t_{0}, then it is also an equilibrium pair for all t̃_{0} ≥ t_{0}.

We perturb the equilibrium state by allowing

x = x_{e} + Δx, u = u_{e} + Δu.

Taylor’s expansion yields

\frac{d}{dt}x = f(t, x_{e} + Δx, u_{e} + Δu)
= f(t, x_{e}, u_{e}) + \frac{∂f}{∂x}(t, x_{e}, u_{e}) Δx + \frac{∂f}{∂u}(t, x_{e}, u_{e}) Δu + higher order terms,

where

\frac{∂f}{∂x}(t, x_{e}, u_{e}) = [ \begin{matrix}\frac{∂f_{1}}{∂x_{1}} & ⋯ & \frac{∂f_{1}}{∂x_{n}} \\ ⋮ & & ⋮ \\ \frac{∂f_{n}}{∂x_{1}} & ⋯ & \frac{∂f_{n}}{∂x_{n}} \end{matrix} ]|_{{x=xe \atop u=ue}}and \frac{∂f}{∂u}(t, x_{e}, u_{e}) = [ \begin{matrix}\frac{∂f_{1}}{∂u_{1}} & ⋯ & \frac{∂f_{1}}{∂u_{m}} \\ ⋮ & & ⋮ \\ \frac{∂f_{n}}{∂u_{1}} & ⋯ & \frac{∂f_{n}}{∂u_{m}} \end{matrix} ]|_{{x=xe \atop u=ue}}

are the Jacobian matrices of f with respect x and u respectively, evaluated at the equilibrium pair, [ x_{e}^{⊤} u_{e}^{⊤} ]^{⊤}. Note that

\frac{d}{dt}x = \frac{d}{dt}x_{e} + \frac{d}{dt}Δx = \frac{d}{dt}Δx

because x_{e} is constant. Furthermore f(t, x_{e}, u_{e}) = 0 for all t ≥ t_{0}. Let

A(t) = \frac{∂f}{∂x}(t, x_{e}, u_{e}) and B(t) = \frac{∂f}{∂u}(t, x_{e}, u_{e}).

Neglecting higher order terms, we arrive at the linear approximation

\frac{d}{dt}Δx = A(t)Δx + B(t)Δu.

Similarly, if the outputs of the nonlinear system model are of the form

y_{1} = h_{1}(t, x_{1}, x_{2}, …, x_{n}, u_{1}, u_{2}, …, u_{m}) ; y_{2} = h_{2}(t, x_{1}, x_{2}, …, x_{n}, u_{1}, u_{2}, …, u_{m}) ; ⋮ ; y_{p} = h_{p}(t, x_{1}, x_{2}, …, x_{n}, u_{1}, u_{2}, …, u_{m}) }

or in vector notation

y = h(t, x, u),

then Taylor’s series expansion can again be used to yield the linear approximation of the above output equations. Indeed, let

y = y_{e} + Δy,

where y_{e} = h(t, x_{e}, u_{e}). Then we obtain

Δy = C(t)Δx + D(t)Δu,

where

C(t) = \frac{∂h}{∂x}(t, x_{e}, u_{e}) = [ \begin{matrix}\frac{∂h_{1}}{∂x_{1}} & ⋯ & \frac{∂h_{1}}{∂x_{n}} \\ ⋮ & & ⋮ \\ \frac{∂h_{p}}{∂x_{1}} & ⋯ & \frac{∂h_{p}}{∂x_{n}} \end{matrix} ]|_{{x=xe \atop u=ue}} ∈ ℝ^{p×n},

and

D(t) = \frac{∂h}{∂u}(t, x_{e}, u_{e}) = [ \begin{matrix}\frac{∂h_{1}}{∂u_{1}} & ⋯ & \frac{∂h_{1}}{∂u_{m}} \\ ⋮ & & ⋮ \\ \frac{∂h_{p}}{∂u_{1}} & ⋯ & \frac{∂h_{p}}{∂u_{m}} \end{matrix} ]|_{{x=xe \atop u=ue}} ∈ ℝ^{p×m}

are the Jacobian matrices of h with respect x and u respectively, evaluated at the equilibrium pair [ x_{e}^{⊤} u_{e}^{⊤} ]^{⊤}.

Example 4.4 In this example, we linearize the model of Watt’s governor that we derived in Section 2.4.1. Recall that the equations modeling the governor have the form:

ẋ_{1} = x_{2}
ẋ_{2} = \frac{1}{2}N^{2}x_{3}^{2} sin 2x_{1} − \frac{g}{L} sin x_{1} − \frac{b}{mL}x_{2}
ẋ_{3} = \frac{κ}{I} cos x_{1} − \frac{τ}{I}

We linearize the above model about an equilibrium state of the form

x_{e} = [ x_{1e} 0 x_{3e} ]^{⊤}

corresponding to the equilibrium control u_{e} = τ.

Equating the right-hand sides of the above engine-governor equations to zero yields

\begin{matrix}x_{2e} & = & 0 \\ N^{2} x_{3e}^{2} & = & \frac{g}{L cos x_{1e}} \\ cos x_{1e} & = & \frac{τ}{κ} \end{matrix} \}

The Jacobian matrix of the engine-governor model evaluated at the equilibrium point is

\frac{∂f}{∂x}|_{{x=xe \atop u=ue}} = [ \begin{matrix}0 & 1 & 0 \\ N^{2}x_{3e}^{2} cos 2x_{1e} − \frac{g}{L} cos x_{1e} & −\frac{b}{mL} & N^{2}x_{3e} sin 2x_{1e} \\ −\frac{κ}{I} sin x_{1e} & 0 & 0 \end{matrix} ] .

We use the following trigonometric identities,

cos 2α = cos^{2} α − sin^{2} α, sin 2α = 2 sin α cos α,

and the previously obtained relation, N^{2} x_{3e}^{2} = \frac{g}{L cos x_{1e}}, to eliminate N^{2} x_{3e}^{2} from the Jacobian matrix. Performing needed manipulations, we obtain

\frac{∂f}{∂x}|_{{x=xe \atop u=ue}} = [ \begin{matrix}0 & 1 & 0 \\ −\frac{g sin^{2} x_{1e}}{L cos x_{1e}} & −\frac{b}{mL} & \frac{2g sin x_{1e}}{Lx_{3e}} \\ −\frac{κ}{I} sin x_{1e} & 0 & 0 \end{matrix} ] .

Hence, the linearized model of the engine-governor system can be represented as

\frac{d}{dt} [ \begin{matrix}Δx_{1} \\ Δx_{2} \\ Δx_{3} \end{matrix} ] = [ \begin{matrix}0 & 1 & 0 \\ −\frac{g sin^{2} x_{1e}}{L cos x_{1e}} & −\frac{b}{mL} & \frac{2g sin x_{1e}}{Lx_{3e}} \\ −\frac{κ}{I} sin x_{1e} & 0 & 0 \end{matrix} ] [ \begin{matrix}Δx_{1} \\ Δx_{2} \\ Δx_{3} \end{matrix} ] + [ \begin{matrix}0 \\ 0 \\ −\frac{1}{I} \end{matrix} ] Δu.  (4.7)

4.3 Describing Function Method

The describing function method allows us to apply familiar frequency domain techniques, used in the linear system analysis, to the analysis of a class of nonlinear dynamical systems. The method can be used to predict limit cycles in nonlinear systems. The describing function method can be viewed as a “harmonic linearization” of a nonlinear element. The method provides a “linear approximation” to the nonlinear element based on the assumption that the input to the nonlinear element is a sinusoid of known, constant amplitude. The fundamental harmonic of the element’s output is compared with the input sinusoid to determine the steady-state amplitude and phase relation. This relation is the describing function for the nonlinear element. The describing function method is based on the Fourier series. In our review of Fourier series, we use notion of a scalar product over a function space that we discuss next. Our presentation of the scalar product in the context of spaces of functions follows that of Lang [203]. The reader familiar with the Fourier series may go directly to Subsection 4.3.3 where the describing function method analysis is presented.

4.3.1 Scalar Product of Functions

Let C([a, b]) be the set of continuous functions on the interval [a, b] ⊂ ℝ.

Definition 4.4 The scalar product of two real-valued functions u, v ∈ C([a, b]) is

⟨u, v⟩ = ∫_{a}^{b} u(t)v(t)dt.
◇

Using simple properties of the integral, we can verify that the above scalar product satisfies the following conditions:

1. for all u, v ∈ C([a, b]), we have ⟨u, v⟩ = ⟨v, u⟩,

2. if u, v, w are elements of C([a, b]), then ⟨u, v + w⟩ = ⟨u, v⟩ + ⟨u, w⟩,

3. if x is a number, then ⟨xu, v⟩ = x⟨u, v⟩ = ⟨u, xv⟩,

4. for all v ∈ C([a, b]), we have ⟨v, v⟩ ≥ 0, and ⟨v, v⟩ > 0 if v ≠ 0.

The functions u, v ∈ C([a, b]) are said to be orthogonal if ⟨u, v⟩ = 0. A set of functions from C([a, b]) is said to be mutually orthogonal if each distinct pair of functions in the set is orthogonal.

Example 4.5 The functions ψ_{n}(t) = sin nωt, ϕ_{n}(t) = cos nωt, n = 1, 2, …, and ϕ_{0}(t) = 1 form a mutually orthogonal set of functions on the interval [t_{0}, t_{0} + 2π/ω]. This can be verified by direct integration. For example, for positive m and n such that m ≠ n, we have

⟨ψ_{m}, ψ_{n}⟩ = ∫_{t_{0}}^{t_{0}+2π/ω} sin mωt sin nωt dt
= \frac{1}{2} ∫_{t_{0}}^{t_{0}+2π/ω} (cos (m − n)ωt − cos (m + n)ωt) dt
= \frac{1}{2ω} (\frac{sin (m − n)ωt}{m − n} − \frac{sin (m + n)ωt}{m + n})|_{t_{0}}^{t_{0}+2πω}
= 0  (4.8)

Note that m + n ≠ 0 because m and n are positive. Using similar computations we can show that for m ≠ n

⟨ϕ_{m}, ϕ_{n}⟩ = 0,  (4.9)

and for all non-negative m and positive n,

⟨ϕ_{m}, ψ_{n}⟩ = 0.  (4.10)

We define the norm of v ∈ C([a, b]) to be

∥v∥ = \sqrt{⟨v, v⟩}.

Example 4.6 Consider ψ_{n}(t) = sin nωt on the interval [t_{0}, t_{0} + 2π/ω]. Then,

∥ψ_{n}∥^{2} = ∫_{t_{0}}^{t_{0}+2π/ω} sin nωt sin nωt dt
= ∫_{t_{0}}^{t_{0}+2π/ω} sin^{2} nωt dt
= \frac{1}{2} ∫_{t_{0}}^{t_{0}+2π/ω} (1 − cos 2nωt) dt
= \frac{1}{2} (t − \frac{sin 2nωt}{2nω})|_{t_{0}}^{t_{0}+2π/ω}
= \frac{π}{ω} .  (4.11)

Similarly,

∥ϕ_{n}∥^{2} = \frac{π}{ω} , n = 1, 2 …,  (4.12)

and

∥ϕ_{0}∥^{2} = \frac{2π}{ω} .  (4.13)

So far we discussed only continuous functions. In many applications we have to work with more general functions, specifically with piecewise continuous functions.

Definition 4.5 A function f is said to be piecewise continuous on an interval [a, b] if the interval can be partitioned by a finite number of points, say t_{i}, so that

1. a = t_{0} < t_{1} < … < t_{n} = b;

2. the function f is continuous on each open subinterval (t_{i−1}, t_{i}),

3. the function f approaches a finite limit as the end points of each subinterval are approached from within the subinterval, that is, the limits

lim_{{h→0 \atop h>0}} f(t_{i} − h) and lim_{{h→0 \atop h>0}} f(t_{i} + h)

both exist.

Figure 4.5: A graph of a piecewise continuous function.

◇

The graph of a piecewise continuous function is shown in Figure 4.5. In further discussion, we assume that two piecewise functions f and g are equal if they have the same values at the points where they are continuous. Thus, the functions shown in Figure 4.6 are considered to be equal, though they differ at the points of discontinuity.

Theorem 4.1 Let V be the space of functions that are piecewise continuous on the interval [a, b]. Let f ∈ V. Then, ∥f∥ = 0 if and only if f(x) = 0 for all but finite number of points x in the interval [a, b].

Proof (⇐) It is clear that if f(x) = 0 except for a finite number of x ∈ [a, b], then

∥f∥^{2} = ∫_{a}^{b} f^{2}(x)dx = 0.

(⇒) Suppose f is piecewise continuous on [a, b], and let a = t_{0} < t_{1} < … < t_{n} = b be a partition of the interval [a, b] such that f is continuous on each subinterval [t_{i−1}, t_{i}] except possibly at the end points. Suppose that ∥f∥ = 0. Then, also ∥f∥^{2} = ⟨f, f⟩ = 0. The above means that

∫_{a}^{b} f^{2}(x)dx = 0,

Figure 4.6: Two piecewise continuous functions that we consider to be equal.

where the integral is the sum of the integrals over the intervals [t_{i−1}, t_{i}], that is,

∫_{a}^{b} f^{2}(x)dx = ∑_{i=1}^{n} ∫_{t_{i−1}}^{t_{i}} f^{2}(x)dx = 0.

Each integral satisfies

∫_{t_{i−1}}^{t_{i}} f^{2}(x)dx ≥ 0.

Hence each such integral must be equal to 0. The function f is continuous on (t_{i−1}, t_{i}). Therefore,

f^{2}(x) = 0 for t_{i−1} < x < t_{i},

which means that

f(x) = 0 for t_{i−1} < x < t_{i}.

Hence, f(x) = 0 except at a finite number of points. The proof is complete.

□

The above theorem implies that the scalar product on the space V of piecewise continuous functions is positive definite. In other words, for a piecewise continuous function f ∈ V, we have ⟨f, f⟩ ≥ 0, and ⟨f, f⟩ = 0 if and only if f = 0 for all but a finite number of points in the interval [a, b].

4.3.2 Fourier Series

We begin with the definition of a periodic function.

Definition 4.6 A function f : ℝ → ℝ is said to be periodic with the period T if the domain of f contains t + T whenever it contains t, and if

f(t + T) = f(t)

for every value of t. The smallest positive value of T for which f(t + T) = f(t) is called the fundamental period of f.

◇

It follows from the above definition that if T is a period of f, then 2T is also a period, and so is any integral multiple of T.

A periodic function, satisfying certain assumptions spelled out later, may be represented by the series

f(t) = A_{0} + ∑_{k=1}^{∞} (A_{k} cos kωt + B_{k} sin kωt) ,  (4.14)

where ω = 2π/T, with T being the fundamental period of f. On the set of points where the series (4.14) converges, it defines f, whose value at each point is the sum of the series for that value of t, and we say that the series (4.14) is the Fourier series for f.

Suppose that the series (4.14) converges. We use the results of the previous subsection to find expressions for the coefficients A_{k} and B_{k}. We first compute A_{k} coefficients. Multiplying (4.14) by ϕ_{m} = cos mωt, where m > 0 is a fixed positive integer, and then integrating both sides with respect to t from t_{0} to t_{0} + 2π/ω yields

⟨ϕ_{m}, f⟩ = A_{0}⟨ϕ_{m}, ϕ_{0}⟩ + ∑_{k=1}^{∞} A_{k}⟨ϕ_{m}, ϕ_{k}⟩ + ∑_{k=1}^{∞} B_{k}⟨ϕ_{m}, ψ_{k}⟩,  (4.15)

where ϕ_{0} = 1. It follows from (4.8), (4.9), and (4.10) that the only nonvanishing term on the right of (4.15) is the one for which k = m in the first summation. Using (4.12), we obtain

A_{m} = \frac{⟨ϕ_{m}, f⟩}{⟨ϕ_{m}, ϕ_{m}⟩}
= \frac{ω}{π} ∫_{t_{0}}^{t_{0}+2π/ω} f(t) cos mωt dt, m = 1, 2, …  (4.16)

To obtain A_{0}, we integrate (4.14) and using (4.13), we get

A_{0} = \frac{⟨ϕ_{0}, f⟩}{⟨ϕ_{0}, ϕ_{0}⟩} = \frac{ω}{2π} ∫_{t_{0}}^{t_{0}+2π/ω} f(t)dt.  (4.17)

An expression for B_{m} may be obtained by multiplying (4.14) by sin mωt and then integrating term by term from t_{0} to t_{0} + 2π/ω. Using the orthogonality relations (4.8)–(4.10) yields

B_{m} = \frac{⟨ψ_{m}, f⟩}{⟨ψ_{m}, ψ_{m}⟩} = \frac{ω}{π} ∫_{t_{0}}^{t_{0}+2π/ω} f(t) sin mωt dt, m = 1, 2, …  (4.18)

Suppose now that a function f is given that is periodic with period 2π/ω and integrable on the interval [t_{0}, t_{0}+2π/ω]. Then, coefficients A_{k} and B_{k} can be computed using (4.16)–(4.18) and series of the form (4.14) can be formally constructed. However, we do not know whether this series converges for each value of t, and if so, whether its sum is f(t). The answer is given by the theorem that we state after explaining the notation used. We denote the limit of f(t) as t approaches c from the right by f(c+), that is,

f(c+) = lim_{t→c+} f(t).

Similarly, f(c−) = lim_{t→c−} f(t) denotes the limit of f(t) as t → c from the left. The mean value of the right and left hand limits at the point c is

(f(c+) + f(c−)) /2.

Note that at any point c where f is continuous, we have

f(c+) = f(c−) = f(c).

Theorem 4.2 Assume that f and its derivative are piecewise continuous on the interval [t_{0}, t_{0} + 2π/ω]. Furthermore, suppose that f is defined outside the interval [t_{0}, t_{0} + 2π/ω] so that it is periodic with period 2π/ω. Then, f has a Fourier series of the form (4.14) whose coefficients are given by (4.16)–(4.18). The Fourier series converges to (f(t+) + f(t−)) /2 at all points of the interval [t_{0}, t_{0} + 2π/ω].

With the above background of the Fourier series we are ready to discuss the describing function method for analyzing a class of nonlinear control systems.

4.3.3 Describing Function in the Analysis of Nonlinear Systems

The describing function method can be considered as an extension of the Nyquist stability criterion to nonlinear systems. In our further discussion, we sometimes use a double box to represent a nonlinear element as illustrated in Figure 4.7.

Consider a feedback system that contains a nonlinear element as shown in Figure 4.8. Assume that the input to the nonlinear element is a sinusoidal signal x(t) = X sin ωt, where 200 CHAPTER 4. LINEARIZATION

[figure]

Figure 4.7: A symbol for a nonlinear element.

Figure 4.8: A nonlinear feedback system with a nonlinear element in the forward path.

X is the amplitude of the input sinusoid. The output of the nonlinear element is, in general, not sinusoidal in response to a sinusoidal input. Suppose that the nonlinear element output is periodic with the same period as its input and that it may be expanded in the Fourier series,

y(t) = A_{0} + ∑_{k=1}^{∞} (A_{k} cos kωt + B_{k} sin kωt)
= A_{0} + ∑_{k=1}^{∞} Y_{k} sin (kωt + ϕ_{k}) .  (4.19)

Thus y(t) consists of a fundamental component

Y_{1} sin (ωt + ϕ_{1}) ,

together with a mean level A_{0} and harmonic components at frequencies 2ω, 3ω, . . .. The describing function method relies on the assumption that only the fundamental harmonic is significant. This assumption is called the filtering hypothesis. The filtering hypothesis can be expressed as

|G(jω)| ≫ |G(kjω)| for k = 2, 3, …,

which requires that the linear element, modeled by the transfer function G(s), be a low-pass filter. The filtering hypothesis is often valid because many, if not most, control systems are low-pass filters. This results in higher harmonics being more attenuated compared with the fundamental harmonic component. The nonlinear element is then approximately modeled by its describing function

N(X) = \frac{Y_{1}}{X}^{}ϕ̸_{1} = \frac{fundamental component of output}{input}.  (4.20)

The describing function is, in general, a complex-valued function of the amplitude X of the input sinusoid.

The stability of the nonlinear closed-loop system depicted in Figure 4.8 can be analyzed using Nyquist’s criterion applied to the associated linear system shown in Figure 4.9. The associated linear closed-loop system characteristic equation is

1 + N(X)G(s) = 0.  (4.21)

In the G(jω) complex plane, the locus traced out by −1/N as X is varied, is being considered as a generalization of the (−1, j0) point in the linear case. The stability of the nonlinear system is determined by computing the encirclements of the −1/N locus by the G(jω) plot.

202 CHAPTER 4. LINEARIZATION Figure 4.9: Associated linear system used in the describing function analysis of the nonlinear system shown in Figure 4.8.

Figure 4.10: Stability analysis using describing function.

Figure 4.11: Nonlinear feedback control system analyzed in Example 4.7.

We might have situations shown in Figure 4.10. In case (i), the −1/N locus is not encircled by the G(jω) plot. Assuming that the poles of G(s) are in the left-half complex plane, the closed-loop system is asymptotically stable. For a limit cycle to exist, we must have an intersection between the polar plot of G(jω) and the −1/N locus. We have two intersections between −1/N and G(jω) in case (ii). At intersection point 1, any increase in X will cause the operating point to be encircled and consequently X to grow. If there is a decrease in X, then the oscillations will decay. Thus, point 1 represents an unstable limit cycle. At intersection point 2, an increase in X would shift the operating point outside the G(jω) plot resulting in the decaying oscillations. Point 2 represents a stable limit cycle. In case (iii), any oscillation will grow without limit.

Example 4.7 Consider the nonlinear control system shown in Figure 4.11 that was analyzed in [278, pages 339–341]. We first derive the describing function of the ideal relay. When the input of the ideal relay is greater than zero, its output is M. When the relay’s input is less than zero, its output is −M. Thus, the ideal relay can be described as

y = \{ \begin{matrix}M & for & x > 0 \\ −M & for & x < 0. \end{matrix}  (4.22)

We assume that the input to the relay is a sinusoid, x(t) = X sin ωt, drawn vertically down from the relay characteristic in Figure 4.12. The output y(t) of the relay is a rectangular

Figure 4.12: Input and output waveforms of an ideal relay.

wave. It is drawn horizontally to the right of the relay characteristic in Figure 4.12. This rectangular wave is an odd function. For an odd function, we have A_{k} = 0, k = 0, 1, …, in its Fourier series. Thus, the Fourier series of the relay’s output has the form

y(t) = ∑_{k=1}^{∞} B_{k} sin kωt.

The fundamental harmonic of y(t) is

B_{1} sin ωt = Y_{1} sin ωt,

where

B_{1} = \frac{ω}{π} ∫_{0}^{2π/ω} y(t) sin ωt dt
= \frac{2ω}{π} ∫_{0}^{π/ω} M sin ωt dt
= \frac{2Mω}{π} ∫_{0}^{π/ω} sin ωt dt
= −\frac{2M}{π} cos (ωt)|_{0}^{π/ω}
= \frac{4M}{π} .  (4.23)

Figure 4.13: A plot of the describing function for the ideal relay.

Hence, the describing function of the ideal relay is

N(X) = \frac{Y_{1}}{X}^{}0̸^{∘} = \frac{4M}{πX} .

A normalized plot of the describing function of the ideal relay nonlinearity is depicted in Figure 4.13. In our example, M = 1 and thus the describing function of the relay is

N(X) = \frac{4}{πX} .

The sinusoidal transfer function of the linear plant model is

G(jω) = \frac{16}{jω(jω + 2)^{2}} = \frac{−64ω + j16(ω^{2} − 4)}{ω(4 + ω^{2})^{2}}.

At ω = 2 rad/sec the polar plot of G(jω) intersects the plot of −1/N, see Figure 4.14, to form a stable limit cycle. The amplitude of the limit cycle is obtained from the relation, G(j2) = −1/N, that is,

\frac{−64}{(4 + 4)^{2}} = −\frac{πX}{4} .

Hence, the amplitude of the limit cycle is

X = \frac{4}{π} ,

Figure 4.14: Plots of G(jω) and −1/N of Example 4.7.

while the amplitude of the fundamental harmonic of the plant output, c, is

\frac{4}{π} |G(j2)| = \frac{4}{π} .

Note that the filtering hypothesis is satisfied in this example. Indeed, the Fourier series of the output of the relay is

\frac{4}{π} (sin ωt + \frac{1}{3} sin 3ωt + \frac{1}{5} sin 5ωt) .

Thus, for example, the amplitude of the third harmonic of the plant output is

\frac{4}{3π} |G(j6)| = \frac{1}{45} \frac{4}{π} .

The higher harmonics are even more attenuated compared with the fundamental harmonic.

4.4 Notes

For an in-depth treatment of the describing function method, we refer to Graham and McRuer [118] and Chapters 6 and 7 of Hsu and Meyer [145]. Graham and McRuer [118] make an interesting remark about the origins of the describing function method. They note on page 92 of their 1961 book: “While the notion of representing a nonlinearity by an “equivalent” linear element is quite old and has been used by many writers, the first instance of a major systematic exploitation of the technique was probably by N. M. Kryloff and N. N. Bogoliuboff, Introduction to Non-linear Mechanics (a free translation of the Russian edition of 1937, by S. Lefschetz), Princeton University Press, Princeton, N. J., 1943. See also N. Minorsky, Introduction to Non-linear Mechanics, J. W. Edwards, Ann Arbor, Mich., 1947.” The so-called sinusoidal describing function that we analyzed in the previous section, was developed almost simultaneously in several different countries during and just after World War II. The first to introduce the method in the open literature appears to be A. Tustin of England who published his paper in the Journal of the IEE in 1947.

The behavior of numerous dynamical systems is often modeled by a set of linear, firstorder, differential equations. Frequently, however, a linear model is a result of linearization of a nonlinear model. Linear models seem to dominate the controls’ literature. Yet nature is nonlinear. “All physical systems are nonlinear and have time-varying parameters in some degree.” [118, page 1]. How can one reconcile this paradox? Graham and McRuer [118, page 1] have the following answer: “Where the effect of the nonlinearity is very small, or if the parameters vary only slowly with time, linear constant-parameter methods of analysis can be applied to give an approximate answer which is adequate for engineering purposes. The analysis and synthesis of physical systems, predicated on the study of linear constantparameter mathematical models, has been, in fact, an outstandingly successful enterprise. A system represented as linear, however, is to some extent a mathematical abstraction that can never be encountered in a real world. Either by design or because of nature’s ways it is often true that experimental facts do not, or would not, correspond with any prediction of linear constant-parameter theory. In this case nonlinear or time-varying-parameter theory is essential to the description and understanding of physical phenomena.”

4.5 Exercises

Exercise 4.1 Consider the nonlinear mass-spring-damper system depicted in Figure 4.15. The spring coefficient K = 0.5x^{2} N/m, the damping coefficient B = 0.1ẋ^{2} N·sec/m, and M = 1 kg.

(i) Write the second-order differential equation describing the motion of the system.

(ii) Choose x_{1} = x and x_{2} = ẋ. Represent the equation of motion derived in (i) in state-

Figure 4.15: The mass-spring-damper system of Exercise 4.1.

space format.

(iii) For the model from (ii), find the equilibrium state corresponding to u = 0. Then, linearize the model about the obtained equilibrium point.

Exercise 4.2 (based on [328, pages 29–31]) A cyclotron, whose operation is illustrated in Figure 4.16, accelerates charged particles into high energies so that they can be used in atom-smashing experiments. The operation of the cyclotron can be described as follows. Positive ions enter the cyclotron from the ion source, S, and are available to be accelerated. An electric oscillator establishes an accelerating potential difference across the gap of the two D-shaped objects, called dees. An emerging ion from the ion source, S, finds the dee that is negative. It will accelerate toward this dee and will enter it. Once inside the dee, the ion is screened from electric fields by the metal walls of the dees. The dees are immersed in a magnetic field so that the ion’s trajectory bends in a circular path. The radius of the path depends on the ion’s velocity. After some time, the ion emerges from the dee on the other side of the ion source. In the meantime, the accelerating potential changes its sign. Thus the ion again faces a negative dee. The ion further accelerates and describes a semicircle of larger radius. This process goes on until the ion reaches the outer edge of one dee, where it is pulled out of the cyclotron by a negatively charged deflector plate. The moving ion is acted upon by a deflecting force of magnitude

q(v + Δv)(B + ΔB),

Figure 4.16: Schematic of a cyclotron of Exercise 4.2.

where q is the charge of the ion, v is the ion’s speed, Δv is the increase in the ion’s speed, B is the magnetic field acting on the ion. The nominal radius of the ion’s orbit is r. The change in the magnetic field as the result of a change in the orbit of the ion is ΔB. As the ion speed increases, the relativistic mass m increases by Δm. The ion moving in a circular path undergoes centrifugal force of magnitude

\frac{(m + Δm)(v + Δv)^{2}}{r + Δr} .

By Newton’s second law, the equation describing the motion of the ion is

(m + Δm)\frac{d^{2}}{dt^{2}} (r + Δr) = \frac{(m + Δm)(v + Δv)^{2}}{r + Δr} − q(v + Δv)(B + ΔB).

Linearize the above equation. The motion of the ion along a nominal path is described by

m\frac{d^{2}r}{dt^{2}} = \frac{mv^{2}}{r} − qvB.

Find the transfer function relating the change in radius of motion, Δr, of the ion, and the change in the momentum of the ion

Δp = mΔv + Δmv.

In other words, find the transfer function L(Δr)/L(Δp), where L is the Laplace transform operator.

Exercise 4.3 For the nonlinear model of the θ-r robot manipulator in Example 2.2, on page 67, find the equilibrium point

(z_{e}, u_{e}) = ([ π/4 0 2 0 ]^{⊤} , u_{e})

and linearize the non-linear model about the obtained equilibrium. You may wish to use MATLAB’s jacobian and subs functions to perform linearization.

Exercise 4.4 Linearize the permanent-magnet stepper motor model, analyzed in Subsection 2.4.3, about x_{1eq} = θ_{eq} = constant.

Exercise 4.5 For the nonlinear model of a double-link robot manipulator in Subsection 2.4.6, find the equilibrium point

(x_{e}, u_{e}) = ([ 135^{∘} 0 180^{∘} 0 ]^{⊤} , u_{e})

and linearize the non-linear model about the obtained equilibrium.

Exercise 4.6 Let v_{1}, v_{2}, …, v_{n} be non-zero, mutually orthogonal elements of the space V and let v be an arbitrary element of V.

(i) Find the component of v along v_{j}.

(ii) Let c_{j} denote the component of v along v_{j}. Show that the element

v − c_{1}v_{1} − ⋯ c_{n}v_{n}

is orthogonal to v_{1}, v_{2}, …, v_{n}.

(iii) Let c_{j} be the component of v ∈ V along v_{j}. Show that

∑_{i=1}^{n} c_{i}^{2} ≤ ∥v∥^{2}.

The above relation is called the Bessel inequality.

Exercise 4.7 Consider characteristics of two nonlinear elements that are shown in Figure 4.17. Let

N_{S}(x) = \frac{2}{π} [arcsin \frac{1}{x} + \frac{1}{x} cos (arcsin \frac{1}{x})] .  (4.24)

Figure 4.17: Characteristics of a limiter and dead zone nonlinear elements of Exercise 4.7.

Then, the describing function of the limiter is

N_{l} = \{ \begin{matrix}K, & M ≤ S \\ KN_{S} (\frac{M}{S} ) , & M > S. \end{matrix}

The describing function of the dead zone is

N_{d−z} = \{ \begin{matrix}0, & M ≤ A \\ K [1 − N_{S} (\frac{M}{A} )] , & M > A. \end{matrix}

Derive the describing function of the nonlinear element whose characteristic is shown in Figure 4.18.

Exercise 4.8 The describing function of the nonlinear element shown in Figure 4.19, is

N_{1} = \{ \begin{matrix}0, & M ≤ A \\ K [1 − N_{S} (\frac{M}{A} )] + \frac{4B}{πM} \sqrt{1 − (\frac{A}{M} )^{2}}, & M > A, \end{matrix}

where N_{S}(x) is given by (4.24). The describing function of the dead-zone nonlinearity, is given in Exercise 4.7 and its characteristic is shown in Figure 4.17. Derive the describing function of the ideal relay with dead zone whose characteristic is shown in Figure 4.20.

212 CHAPTER 4. LINEARIZATION

Figure 4.18: Characteristic of a nonlinear element of Exercise 4.7. Figure 4.19: Nonlinear element with the describing function N_{1} of Exercise 4.8.

4.5. EXERCISES 213 Figure 4.20: Characteristic of the ideal relay with dead zone.

214 CHAPTER 4. LINEARIZATION

Linear Time-Invariant (LTI) Systems

In bear country, wear bells to avoid surprising bruins. Don’t approach a bear, but don’t run away from one either. Movement may incite a charge, and a bear can outrun you. If a bear approaches you, talk to it calmly in a low, firm voice. Slowly back away until the bear is out of sight. If you climb a tree, get at least 12 feet up. If a bear senses that its food may be stolen or that its cub is in danger, it may charge. If this happens, drop to the ground and assume a fetal position, protecting your head with your arms, and play dead. Keep your pack on; it may serve as armor. [281, pages 18–19]

We begin our discussion of linear system theory with a description of the concept of linearity by Mitchell J. Feigenbaum [270, page 3], who is a researcher of nonlinear systems. “Linearity means that the rule that determines what a piece of a system is going to do next is not influenced by what it is doing now. More precisely, this is intended in a differential or incremental sense: For a linear spring, the increase of its tension is proportional to the increment whereby it is stretched, with the ratio of these increments exactly independent of how much it has already been stretched. Such a spring can be stretched arbitrarily far, and in particular will never snap or break. Accordingly, no real spring is linear.” The real world is nonlinear. However, there is a number of reasons to investigate linear systems. Linear models are often used to approximate nonlinear systems. Many times, this approximation is sufficient for the controller design for the underlying nonlinear system. Knowledge of linear system theory helps to understand the intricate theory of nonlinear systems.

In this chapter, we acquaint ourselves with basic concepts from linear systems that are used in subsequent chapters.

5.1 Reachability and Controllability

We first consider a linear, time-invariant (LTI), discrete-time system modeled by

x[k + 1] = Ax[k] + Bu[k], x[0] = x_{0},  (5.1)

where A ∈ ℝ^{n×n}, B ∈ ℝ^{n×m}. The solution to (5.1) is

x[i] = A^{i}x[0] + ∑_{k=0}^{i−1} A^{i−k−1}Bu[k],

which can be represented as

x[i] = A^{i}x[0] + [ B AB ⋯ A^{i−1}B ] [ u[i − 1] ; u[i − 2] ; ⋮ ; u[0] ] .  (5.2)

Let

U_{i} = [ B AB ⋯ A^{i−1}B ] , i = 1, 2, ….

Note that U_{i} ∈ ℝ^{n×mi}. If for some positive l,

rank U_{l} = rank U_{l+1},

then all the columns of A^{l}B are linearly dependent on those of U_{l}. This, in turn, implies that all the columns of A^{l+1}B, A^{l+2}B, … must also be linearly dependent on those of U_{l}. Hence,

rank U_{l} = rank U_{l+1} = rank U_{l+2} = …  (5.3)

The above implies that the rank of U_{i} increases by at least one when i is increased by one, until the maximum value of rank U_{i} is attained. The maximum value of the rank U_{i} is guaranteed to be achieved for i = n, which means that, we always have

rank U_{n} = rank U_{n+1} = rank U_{n+2} = …  (5.4)

It follows from (5.3) that if rank B = r, then

rank [ B AB ⋯ A^{n−1}B ] = rank [ B AB ⋯ A^{n−r}B ] .  (5.5)

Definition 5.1 We say that the system (5.1) is reachable if for any x_{f} there exists an integer q > 0 and a control sequence, {u[i] : i = 0, 1, …, q −1}, that transfers x[0] = 0 to x[q] = x_{f}.

◇

Theorem 5.1 The system (5.1) is reachable if and only if

rank [ B AB ⋯ A^{n−1}B ] = n.

Proof Using (5.2) where x[0] = 0 and x[q] = x_{f}, we obtain

x_{f} = ∑_{k=0}^{q−1} A^{q−k−1}Bu[k]

= [ B AB ⋯ A^{q−1}B ] [ u[q − 1] ; u[q − 2] ; ⋮ ; u[0] ] .  (5.6)

It follows from the above that for any x_{f} there exists a control sequence {u[i] : i = 0, 1, …, q − 1} that transfers x[0] = 0 to x[q] = x_{f} if and only if

rank [ B AB ⋯ A^{q−1}B ] = rank U_{q} = n.

By (5.4) the maximum value of the rank of U_{q} is guaranteed to be attained for q = n, which proves the theorem.

□

A closely related notion to reachability is the notion of controllability. Definition 5.2 A system is controllable if it can be transferred from any given state to the origin 0 of the state space by means of an appropriately chosen control law.

◇

Theorem 5.2 The system (5.1) is controllable if and only if

range (A^{n}) ⊂ range [ B AB ⋯ A^{n−1}B ] .

Proof Using the solution formula for the system (5.1), for the case when x[0] = x_{0} and x_{f} = x[q] = 0, we obtain

A^{q}x_{0} = − ∑_{k=0}^{q−1} A^{q−k−1}Bu[k]

= − [ B AB ⋯ A^{q−1}B ] [ u[q − 1] ; u[q − 2] ; ⋮ ; u[0] ] .  (5.7)

The system (5.1) is controllable if and only if for some q > 0 and arbitrary initial condition x_{0}, the vector A^{q}x_{0} is in the range of [ B AB ⋯ A^{q−1}B ] = U_{q}. Taking into account that the maximal range of U_{q} is guaranteed to be attained for q = n, the system (5.1) is controllable if and only if the condition of the theorem is satisfied.

□

Note that the condition of the above theorem can be equivalently represented as

rank [ B AB ⋯ A^{n−1}B ] = rank [ B AB ⋯ A^{n−1}B A^{n} ] .  (5.8)

We illustrate the above discussion with a simple example that we found in [165, page 482].

Example 5.1 Consider a discrete-time system modeled by

x[k + 1] = Ax[k] + bu[k]
= [ \begin{matrix}0 & a \\ 0 & 0 \end{matrix} ] x[k] + [ \begin{matrix}1 \\ 0 \end{matrix} ] u[k], a ∈ ℝ.  (5.9)

The above system model is not reachable because

rank [ b Ab ] = rank [ \begin{matrix}1 & 0 \\ 0 & 0 \end{matrix} ] = 1 < 2.

It is easy to check using (5.6) that, for example, the state x_{f} = [ 0 1 ]^{⊤} is not reachable from x[0] = 0 because x_{f} does not belong to the range of [ b Ab ]. In fact, nonreachable states form the set,

\{c [ \begin{matrix}0 \\ 1 \end{matrix} ] , c ∈ ℝ\} .

However, the system (5.9) is controllable because for any a ∈ ℝ,

rank [ b Ab ] = rank [ \begin{matrix}1 & 0 \\ 0 & 0 \end{matrix} ] = rank [ b Ab A^{2} ] = rank [ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{matrix} ] .

That is,

range (A^{2}) = range (O) ⊂ range [ b Ab ] .

Using (5.7), we can easily see that arbitrary initial state x[0] ∈ ℝ^{2} of the system (5.9) can be transferred to the origin of ℝ^{2} using, for example, the zero control sequence, u[0] = u[1] = 0. This is because A^{2} = O and hence A^{2}x[0] = 0 for any x[0] ∈ ℝ^{2}.

In summary, for discrete-time linear systems, reachability implies controllability, and the two notions are equivalent if the matrix A of the given discrete system is nonsingular.

We next discuss the notions of reachability and controllability for continuous-time linear systems.

Definition 5.3 We say that the system ẋ(t) = Ax(t)+Bu(t), where A ∈ ℝ^{n×n}, B ∈ ℝ^{n×m}, or equivalently, the pair (A, B), is reachable if for any x_{f} ∈ ℝ^{n} there is t_{1} and a control law u(t) that transfers x(t_{0}) = 0 to x(t_{1}) = x_{f}.

◇

Definition 5.4 We say that the system ẋ(t) = Ax(t) + Bu(t), or equivalently, the pair (A, B), is controllable if there is a control law u(t) that transfers any initial state x_{0} = x(t_{0}) at any time t_{0} to the origin 0 of the state space at some time t_{1} > t_{0}.

◇

It turns out that in the case of continuous, time invariant, systems the two notions are equivalent. Indeed, from the solution formula for the controlled system (3.26), for x(0) = 0 and x(t_{1}) = x_{f}, we obtain

x_{f} = ∫_{0}^{t_{1}} e^{A(t_{1}−t)}Bu(t)dt.

Premultiplying the above equation by e^{−At_{1}} and rearranging, we get

v = ∫_{0}^{t_{1}} e^{−At}Bu(t)dt,  (5.10)

where v = e^{−At_{1}}x_{f}. On the other hand, from the solution formula for the controlled system (3.26), for x(0) = x_{0} and x(t_{1}) = 0, we obtain

0 = e^{At_{1}}x_{0} + ∫_{0}^{t_{1}} e^{A(t_{1}−t)}Bu(t)dt.

Premultiplying the above equation by e^{−At_{1}} and rearranging yields

−x_{0} = ∫_{0}^{t_{1}} e^{−At}Bu(t)dt.  (5.11)

Comparing (5.10) and (5.11), we conclude that reachability of the system ẋ(t) = Ax(t) + Bu(t) is equivalent to its controllability. This is not true in the case of discrete-time systems, where reachability implies controllability, and the two notions are equivalent if the matrix A of the given discrete system is nonsingular.

There are many criteria for checking if a given system is reachable/controllable or not. We present a few of them in this section.

Theorem 5.3 The following are equivalent:

(1) The system ẋ(t) = Ax(t) + Bu(t) is reachable;

(2) rank [ B AB ⋯ A^{n−1}B ] = n;

(3) The matrix

W (t_{0}, t_{1}) = ∫_{t_{0}}^{t_{1}} e^{−At}BB^{⊤}e^{−A^{⊤}t}dt

is nonsingular for all t_{1} > t_{0};

Proof We first prove the implication (1) ⇒ (2), that is, if the pair (A, B) is reachable, then the so called controllability matrix

[ B AB ⋯ A^{n−1}B ] ∈ ℝ^{n×mn}

is of full rank n. We prove this statement by contraposition. Thus, we assume that

rank [ B AB ⋯ A^{n−1}B ] < n,

then, there is a constant n-vector q ≠ 0 so that

q^{⊤}B = 0^{⊤}, q^{⊤}AB = 0^{⊤}, …, q^{⊤}A^{n−1}B = 0^{⊤}.

By the Cayley-Hamilton theorem the matrix A satisfies its own characteristic equation. Hence

A^{n} = −a_{n−1}A^{n−1} − ⋯ − a_{1}A − a_{0}I_{n}.

Therefore

q^{⊤}A^{n}B = q^{⊤} (−a_{n−1}A^{n−1}B − ⋯ − a_{1}AB − a_{0}B) = 0^{⊤}.

By induction

q^{⊤}A^{i}B = 0^{⊤} for i = n + 1, n + 2, …

Let now x(0) = 0 and x(t_{1}) = q. We will show that there is no control law that can transfer the system from x(0) = 0 to x(t_{1}) = q. From the solution formula for the controlled system, we obtain

q = ∫_{0}^{t_{1}} e^{A(t_{1}−t)}Bu(t)dt.

We premultiply the resulting equation by q^{⊤} to get

0 ≠ ∥q∥^{2} = ∫_{0}^{t_{1}} q^{⊤}e^{A(t_{1}−t)}Bu(t)dt = 0

because

q^{⊤}e^{A(t_{1}−t)}B = q^{⊤} (B + (t_{1} − t)AB + \frac{(t_{1} − t)^{2}}{2!} A^{2}B + ⋯) = 0^{⊤}.

Thus the system cannot be transferred from x(0) = 0 to x(t_{1}) = q, and hence the pair (A, B) is not reachable.

We next prove, by contraposition, the implication (2) ⇒ (3), that is, if the controllability matrix is of full rank, then det W (t_{0}, t_{1}) ≠ 0 for any t_{1} > t_{0}. So assume that det W (t_{0}, t_{1}) = 0. Then, there is a nonzero n-vector q ∈ ℝ^{n} such that

q^{⊤} ∫_{t_{0}}^{t_{1}} e^{−At}BB^{⊤}e^{−A^{⊤}t}dt = 0

for all t ∈ [t_{0}, t_{1}]. We postmultiply the above equation by q to get

∫_{t_{0}}^{t_{1}} (q^{⊤}e^{−At}B) (B^{⊤}e^{−A^{⊤}t}q) dt = 0.

Let v(t) = B^{⊤}e^{−A^{⊤}t}q, then the above equation can be written as

∫_{t_{0}}^{t_{1}} ∥v(t)∥^{2}dt = 0.

However, the above is true if and only if v(t) = 0, that is, if and only if the rows of e^{−At}B are linearly dependent over the set of real numbers for all t ∈ [t_{0}, t_{1}]. Now, evaluating

q^{⊤}e^{−At}B = 0^{⊤}

and its successive derivatives with respect to t at t = 0, we obtain

q^{⊤}B = 0^{⊤}, q^{⊤}AB = 0^{⊤}, …, q^{⊤}A^{n−1}B = 0^{⊤},

which means that rank [ B AB ⋯ A^{n−1}B ] < n.

We now prove, constructively, the implication (3) ⇒ (1). Let x(t_{0}) = x_{0} be an arbitrary initial state and x(t_{1}) = x_{f} an arbitrary final state. Then, we claim that the control law

u(t) = B^{⊤}e^{−A^{⊤}t}W^{−1}(t_{0}, t_{1}) (e^{−At_{1}}x(t_{1}) − e^{−At_{0}}x_{0})  (5.12)

Figure 5.1: Schematic of an armature-controlled DC motor system of Example 5.2.

will transfer the system from x_{0} to x_{f}. Note that to construct the above control law the matrix W (t_{0}, t_{1}) must be invertible. Substituting (5.12) into

x(t_{1}) = e^{A(t_{1}−t_{0})}x_{0} + ∫_{t_{0}}^{t_{1}} e^{A(t_{1}−τ)}Bu(τ)dτ

yields

x(t_{1}) = e^{A(t_{1}−t_{0})}x_{0}
+e^{At_{1}} ∫_{t_{0}}^{t_{1}} e^{−Aτ}BB^{⊤}e^{−A^{⊤}τ}dτW^{−1}(t_{0}, t_{1}) (e^{−At_{1}}x(t_{1}) − e^{−At_{0}}x_{0}) .

Performing easy manipulations gives x(t_{1}) = x(t_{1}), which means that indeed the proposed control law does the required job. Thus, we have established equivalence between the three statements of the theorem.

□

Example 5.2 Consider an armature-controlled DC motor system whose schematic is shown in Figure 5.1, where the system parameters are: R_{a} = 1 Ω, L_{a} ≈ 0 H, K_{b} = 5 V/rad/sec, K_{i} = 5 Nm/A, and the effective moment of inertia is I_{eq} = 0.1 kg·m^{2}. The gear inertia and friction are negligible.

Applying Kirchhoff’s voltage law to the armature circuit gives

R_{a}i_{a} + K_{b}θ̇ = e_{a}.  (5.13)

The armature-controlled DC motor uses a constant field current, and therefore the motor torque is

τ_{m} = K_{i}i_{a}.

The motor torque is equal to the torque delivered to the load. Hence,

I_{eq}θ̈ = K_{i}i_{a}.  (5.14)

Substituting i_{a} from (5.13) into the above equation and dividing both sides by I_{eq} yields

θ̈ = \frac{K_{i}}{I_{eq}} (\frac{e_{a} − K_{b}θ̇}{R_{a}})
= −\frac{K_{i}K_{b}}{I_{eq}R_{a}} θ̇ + \frac{K_{i}}{I_{eq}R_{a}} e_{a}.

Let x_{1} = θ, x_{2} = θ̇, and u = e_{a}. Then taking into account the system parameters, we represent the above equation in a state space format,

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}0 & 1 \\ 0 & −250 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ 50 \end{matrix} ] u.

We construct a control u that transfers the system from the initial state x(t_{0}) = x(0) = [ 10 1 ]^{⊤} to the final state x(2) = [ 0 0 ]^{⊤}. Recall that the solution of a controlled system has the form

x(t) = e^{A(t−t_{0})}x(t_{0}) + ∫_{t_{0}}^{t} e^{A(t−τ)}Bu(τ)dτ.

Our goal is to find u so that

0 = e^{2A}x(0) + ∫_{0}^{2} e^{A(2−τ)}bu(τ)dτ.

One can verify that one such u has the form

u(t) = −b^{⊤}e^{A^{⊤}(2−t)} (∫_{0}^{2} e^{A(2−τ)}bb^{⊤}e^{A^{⊤}(2−τ)}dτ)^{−1} e^{2A}x(0),

where

e^{At} = [ \begin{matrix}1 & \frac{1}{250} (1 − e^{−250t}) \\ 0 & e^{−250t} \end{matrix} ] .

Hence,

∫_{0}^{2} e^{A(2−τ)}bb^{⊤}e^{A^{⊤}(2−τ)}dτ = ∫_{0}^{2} [ \begin{matrix}1 & \frac{1}{250} (1 − e^{−250(2−τ)}) \\ 0 & e^{−250(2−τ)} \end{matrix} ] [ \begin{matrix}0 \\ 50 \end{matrix} ] [ 0 50 ]

× [ \begin{matrix}1 & 0 \\ \frac{1}{250} (1 − e^{−250(2−τ)}) & e^{−250(2−τ)} \end{matrix} ] dτ

= [ \begin{matrix}0.07976 & 0.02 \\ 0.02 & 5 \end{matrix} ] .

The inverse of the above matrix is

[ \begin{matrix}12.5502 & −0.0502 \\ −0.0502 & 0.2002 \end{matrix} ] .

Hence,

u(t) = − [ 0 50 ] [ \begin{matrix}1 & 0 \\ \frac{1}{250} (1 − e^{−250(2−t)}) & e^{−250(2−t)} \end{matrix} ]

× [ \begin{matrix}12.5502 & −0.0502 \\ −0.0502 & 0.2002 \end{matrix} ] e^{2A} [ \begin{matrix}10 \\ 1 \end{matrix} ]

= 50.22e^{(250t−500)} − 25.11.

The above control strategy transfers the system from x(0) = [ 10 1 ]^{⊤} to x(2) = [ 0 0 ]^{⊤}. Plots of x_{1}, x_{2} versus time are shown in Figure 5.2. A plot of u versus time is given in Figure 5.3.

We now give a necessary and sufficient condition for the pair (A, B) to be nonreachable.

Theorem 5.4 The pair (A, B) is nonreachable if and only if there is a similarity transformationindextransformation!similarity z = T x such that

Ã = T AT^{−1} = [ \begin{matrix}A_{1} & A_{2} \\ O & A_{4} \end{matrix} ] , B̃ = T B = [ \begin{matrix}B & 1 \\ O \end{matrix} ] ,

where the pair (A_{1}, B_{1}) is reachable, A_{1} ∈ ℝ^{r×r}, B_{1} ∈ ℝ^{r×m}, and the rank of the controllability matrix of the pair (A, B) equals r.

5.1. REACHABILITY AND CONTROLLABILITY 225

Figure 5.2: Plots of state variables versus time of Example 5.2. Figure 5.3: A plot of the control action versus time of Example 5.2.

Proof We first prove necessity (⇒). After forming the controllability matrix of the pair (A, B), we use elementary row operations to get

T [ B AB ⋯ A^{n−1}B ] = [ T B (T AT^{−1}) (T B) ⋯ (T AT^{−1})^{n−1} (T B) ]

= [ \begin{matrix}x & x & x & ⋯ & x & x & x \\ 0 & x & x & ⋯ & x & x & x \\ ⋮ & & & ⋮ & & & ⋮ \\ 0 & 0 & 0 & ⋯ & x & x & x \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ ⋮ & & & ⋮ & & & ⋮ \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{matrix} ]

= [ \begin{matrix}B_{1} & A_{1}B_{1} & A_{1}^{2}B_{1} & ⋯ & A_{1}^{n−1}B_{1} \\ O & O & O & ⋯ & O \end{matrix} ] ,  (5.15)

where the symbol x denotes a “don’t care”, that is, an unspecified scalar. We now show that indeed

Ã = T AT^{−1} = [ \begin{matrix}A_{1} & A_{2} \\ A_{3} & A_{4} \end{matrix} ] , B̃ = T B = [ \begin{matrix}B & 1 \\ O \end{matrix} ] ,

where A_{3} = O. It is clear that T B must have the above form. If A_{3} were not a zero matrix, then we would have

ÃB̃ = [ \begin{matrix}A & 1 & B & 1 \\ A & 3 & B & 1 \end{matrix} ] .

Comparing the above with (5.15), we conclude that A_{3}B_{1} = O. Thus,

ÃB̃ = [ \begin{matrix}A & 1 & B & 1 \\ O \end{matrix} ] .

Next we consider

Ã^{2}B̃ = [ \begin{matrix}A & {2 \atop 1} & B & 1 \\ A & 3 & A & 1 & B & 1 \end{matrix} ] .

Again, comparing the above with (5.15) gives

A_{3}A_{1}B_{1} = O,

and thus

Ã^{2}B̃ = [ \begin{matrix}A & {2 \atop 1} & B & 1 \\ O \end{matrix} ] .

Continuing in this manner, we conclude that A_{3}A_{1}^{n−1}B_{1} = O. Thus it follows that

A_{3} [ B_{1} A_{1}B_{1} ⋯ A_{1}^{n−1}B_{1} ] = O.

Because rank[ B_{1} A_{1}B_{1} ⋯ A_{1}^{n−1}B_{1} ] = r, that is, the controllability matrix of the pair (A_{1}, B_{1}) is of full rank, we have to have A_{3} = O, and the proof of necessity is complete.

To prove sufficiency (⇐), note that if there is a similarity transformation such that

Ã = T AT^{−1} = [ \begin{matrix}A_{1} & A_{2} \\ O & A_{4} \end{matrix} ] , B̃ = T B = [ \begin{matrix}B & 1 \\ O \end{matrix} ] ,

then the controllability matrix of the pair (Ã, B̃) has the form

[ \begin{matrix}B_{1} & A_{1}B_{1} & A_{1}^{2}B_{1} & ⋯ & A_{1}^{n−1}B_{1} \\ O & O & O & ⋯ & O \end{matrix} ] .

Hence, the pair (Ã, B̃) is nonreachable. Because the similarity transformation preserves the reachability property, the pair (A, B) is nonreachable.

□

Example 5.3 For the system model,

ẋ = [ \begin{matrix}−1 & 1 & 0 & 1 \\ 1 & 1 & 0 & 0 \\ 0 & 0 & 4 & 0 \\ 1 & 1 & −1 & 2 \end{matrix} ] x + [ \begin{matrix}1 & 1 \\ 0 & 0 \\ 0 & 0 \\ 0 & 1 \end{matrix} ] u,

construct a state variable transformation so that in the new coordinates the reachable part is separated and distinct from the nonreachable part.

We first form the controllability matrix of the pair (A, B) to get

U = [ B AB A^{2}B A^{3}B ] = [ \begin{matrix}1 & 1 & −1 & 0 & 3 & 4 & −1 & 4 \\ 0 & 0 & 1 & 1 & 0 & 1 & 3 & 5 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 1 & 1 & 3 & 2 & 7 & 7 & 19 \end{matrix} ] .

We then use the MATLAB command [Q,R]=qr(C) to obtain

Q = [ \begin{matrix}−1 & 0 & 0 & 0 \\ 0 & 0 & −1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & −1 & 0 & 0 \end{matrix} ] , R = [ \begin{matrix}−1 & −1 & 1 & 0 & −3 & −4 & 1 & −4 \\ 0 & −1 & −1 & −3 & −2 & −7 & −7 & −19 \\ 0 & 0 & −1 & −1 & 0 & −1 & −3 & −5 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{matrix} ] ,

where the matrices Q and R satisfy U = QR, and R is upper row triangular such that rank R = rank U. Thus, premultiplying the controllability matrix, U, by Q^{−1} reduces this matrix to an upper row triangular matrix R because Q^{−1}U = R. Let z = Q^{−1}x be the state variable transformation. Then in the new coordinates the matrices A and B take the form

Q^{−1}AQ = [ \begin{matrix}−1 & 1 & 1 & 0 \\ 1 & 2 & 1 & 1 \\ 1 & 0 & 1 & 0 \\ 0 & 0 & 0 & 4 \end{matrix} ] and Q^{−1}B = [ \begin{matrix}−1 & −1 \\ 0 & −1 \\ 0 & 0 \\ 0 & 0 \end{matrix} ] .

The dimension of the nonreachable part is 1.

Definition 5.5 If the eigenvalues of the noncontrollable part of a given pair (A, B) are all in the open left-hand complex plane, that is, the nonreachable part is asymptotically stable, then the pair (A, B) is called stabilizable.

◇

We now give yet another test for reachability. This test is also known as the Popov-Belevitch-Hautus (PBH) eigenvector test. We denote the set of eigenvalues of a matrix A as eig(A).

Theorem 5.5 The pair (A, B) is reachable if and only if

rank [ sI_{n} − A B ] = n for all s ∈ eig(A).

Proof We first prove the necessity part (⇒) by contraposition. If there is a vector q ≠ 0 such that for some s ∈ eig(A),

q^{⊤} [ sI_{n} − A B ] = 0^{⊤},

that is,

q^{⊤}A = sq^{⊤}, q^{⊤}B = 0^{⊤},

then

q^{⊤}AB = sq^{⊤}B = 0^{⊤},
q^{⊤}A^{2}B = sq^{⊤}AB = 0^{⊤},

and so on, until

q^{⊤}A^{n−1}B = sq^{⊤}A^{n−2}B = 0^{⊤}.

We write the above equations in the form

q^{⊤} [ B AB ⋯ A^{n−1}B ] = 0^{⊤},

which means that the pair (A, B) is nonreachable.

We now prove the sufficiency part (⇐) also by contraposition. So we assume that rank [ B AB ⋯ A^{n−1}B ] = r < n. By Theorem 5.4, there exists a state variable transformation such that in the new basis the pair (A, B) takes the form

Ã = T AT^{−1} = [ \begin{matrix}A_{1} & A_{2} \\ O & A_{4} \end{matrix} ] , B̃ = T B = [ \begin{matrix}B & 1 \\ O \end{matrix} ] .

We choose q of the form

q^{⊤} = [ 0 z ] ,

where z ≠ 0 is a left eigenvector of A_{4}, that is,

zA_{4} = sz,

where s ∈ eig(A_{4}). Thus

q^{⊤}Ã = [ 0 z ] Ã
= [ 0 sz ]
= sq^{⊤}.

Clearly

q^{⊤} [ sI_{n} − Ã B̃ ] = 0^{⊤},

which means that the rank of the matrix [ sI_{n} − Ã B̃ ] is less than n, and therefore the proof of the sufficiency part is complete.

□

5.2 Observability and Constructability

As we have seen in the previous section, system reachability implies the ability to completely control the entire state of the system. Suppose now that the system state is not directly accessible. Instead, we have the output of the system,

y = Cx + Du,

where C ∈ ℝ^{p×n}, p ≤ n, and D ∈ ℝ^{p×m}. However, we still want to know the dynamical behavior of the entire state.

Definition 5.6 We say that the system

\begin{matrix}ẋ & = & Ax + Bu \\ y & = & Cx + Du, \end{matrix} \}

or equivalently the pair (A, C), is observable if there is a finite t_{1} > t_{0} such that for arbitrary u(t) and resulting y(t) over [t_{0}, t_{1}], we can determine x(t_{0}) from complete knowledge of the system input u and output y.

◇

Note that once x(t_{0}) is known, we can determine x(t) from knowledge of u(t) and y(t) over any finite time interval [t_{0}, t_{1}]. We now show how to constructively determine x(t_{0}), given u(t) and y(t). While doing so we establish a sufficiency condition for the system to be observable. The condition is also necessary for the system observability as we show it next. To proceed with constructive determination of the state vector, we note that the solution y(t) is given by

y(t) = Ce^{A(t−t_{0})}x(t_{0}) + ∫_{t_{0}}^{t} Ce^{A(t−τ)}Bu(τ)dτ + Du(t).

We subtract ∫_{t_{0}}^{t} Ce^{A(t−τ)}Bu(τ)dτ + Du(t) from both sides of the above equation. Let

g(t) = y(t) − ∫_{t_{0}}^{t} Ce^{A(t−τ)}Bu(τ)dτ − Du(t),

then we have

g(t) = Ce^{A(t−t_{0})}x(t_{0}),

where g is known to us. Our goal now is to determine x(t_{0}). Having x(t_{0}), we can determine the entire x(t) for all t ∈ [t_{0}, t_{1}] from the formula

x(t) = e^{A(t−t_{0})}x(t_{0}) + ∫_{t_{0}}^{t} e^{A(t−τ)}Bu(τ)dτ.

Premultiplying both sides of g(t) by e^{A^{⊤}(t−t_{0})}C^{⊤} and integrating the resulting expression between the limits t_{0} and t_{1}, we get

∫_{t_{0}}^{t_{1}} e^{A^{⊤}(t−t_{0})}C^{⊤}Ce^{A(t−t_{0})}dt x(t_{0}) = ∫_{t_{0}}^{t_{1}} e^{A^{⊤}(t−t_{0})}C^{⊤}g(t)dt.

Let

V (t_{0}, t_{1}) = ∫_{t_{0}}^{t_{1}} e^{A^{⊤}t}C^{⊤}Ce^{At}dt.

Then, after performing some manipulations and assuming that the matrix V (t_{0}, t_{1}) is invertible, we obtain

x(t_{0}) = e^{At_{0}}V^{−1}(t_{0}, t_{1})e^{A^{⊤}t_{0}} ∫_{t_{0}}^{t_{1}} e^{A^{⊤}(t−t_{0})}C^{⊤}g(t)dt.

Knowledge of x(t_{0}) allows us to reconstruct the entire state x(t) over the interval [t_{0}, t_{1}]. We conclude that if the matrix V (t_{0}, t_{1}) is invertible, then the system is observable. Example 5.4 For a dynamical system modeled by

\begin{matrix}ẋ & = & [ \begin{matrix}−1/2 & 1/2 \\ −1/2 & 1/2 \end{matrix} ] x + [ \begin{matrix}−1 \\ 1 \end{matrix} ] u \\ y & = & [ 1 1 ] x, \end{matrix} \}

we determine x(t) over the time interval [0, 10] knowing that u(t) = 1 for t ≥ 0 and y(t) = t^{2} + 2t for t ≥ 0.

We first find x(0). Knowledge of x(0) will allow us to find the entire state vector for all t ∈ [0, 10]. We have t_{0} = 0 and t_{1} = 10. Hence,

x(t_{0}) = x(0) = e^{At_{0}}V^{−1}(t_{0}, t_{1})e^{A^{⊤}t_{0}} ∫_{t_{0}}^{t_{1}} e^{A^{⊤}(t−t_{0})}c^{⊤}g(t)dt
= V^{−1}(0, 10) ∫_{0}^{10} e^{A^{⊤}t}c^{⊤}g(t)dt,

where

g(t) = y(t) − c ∫_{0}^{t} e^{A(t−τ)}bu(τ)dτ.

To evaluate the above expression, we first use formula (3.4) to compute,

e^{At} = L^{−1} ([sI_{2} − A]^{−1}) = L^{−1} ([ \begin{matrix}\frac{1}{s} − \frac{1}{2} \frac{1}{s^{2}} & \frac{1}{2} \frac{1}{s^{2}} \\ −\frac{1}{2} \frac{1}{s^{2}} & \frac{1}{s} + \frac{1}{2} \frac{1}{s^{2}} \end{matrix} ]) = [ \begin{matrix}1 − \frac{1}{2}t & \frac{1}{2}t \\ −\frac{1}{2}t & 1 + \frac{1}{2}t \end{matrix} ] .

Next, we compute

V (0, 10) = ∫_{0}^{10} e^{A^{⊤}t}c^{⊤}ce^{At}dt
= ∫_{0}^{10} ([ \begin{matrix}1 − t \\ 1 + t \end{matrix} ] [ 1 − t 1 + t ]) dt = ∫_{0}^{10} [ \begin{matrix}1 − 2t + t^{2} & 1 − t^{2} \\ 1 − t^{2} & 1 + 2t + t^{2} \end{matrix} ] dt
= [ \begin{matrix}t − t^{2} + \frac{t^{3}}{3} & t − \frac{t^{3}}{3} \\ t − \frac{t^{3}}{3} & t + t^{2} + \frac{t^{3}}{3} \end{matrix} ]|_{0}^{10} = [ \begin{matrix}243.333 & −323.333 \\ −323.333 & 443.333 \end{matrix} ] .

The inverse of V (0, 10) is

V^{−1}(0, 10) = [ \begin{matrix}0.133 & 0.097 \\ 0.97 & 0.073 \end{matrix} ] .

We can now compute g(t) to get

g(t) = y(t) − c ∫_{0}^{t} e^{A(t−τ)}bu(τ)dτ
= t^{2} + 2t − [ 1 1 ] ∫_{0}^{t} [ \begin{matrix}−1 + t − τ \\ 1 + t + τ \end{matrix} ] dτ = t^{2} + 2t − ∫_{0}^{t}(2t − 2τ)dτ
= t^{2} + 2t − 2t(τ)|_{0}^{t} + τ^{2}|_{0}^{t} = t^{2} + 2t − t^{2}
= 2t.

Hence,

∫_{0}^{10} e^{A^{⊤}t}c^{⊤}g(t)dt = ∫_{0}^{10} [ \begin{matrix}(1 − t)2t \\ (1 + t)2t \end{matrix} ] dt = [ \begin{matrix}t^{2} − \frac{2}{3}t^{3} \\ t^{2} + \frac{2}{3}t^{3} \end{matrix} ]|_{0}^{10} = [ \begin{matrix}−566.6667 \\ 766.6667 \end{matrix} ] .

Finally, we compute

x(0) = V^{−1}(0, 10) ∫_{0}^{10} e^{A^{⊤}t}c^{⊤}g(t)dt = [ \begin{matrix}−1 \\ 1 \end{matrix} ] .

Alternatively, we could use the Laplace transform method to find x(0). Knowing x(0), we can find x(t) for t ≥ 0,

x(t) = e^{At}x(0) + ∫_{0}^{t} e^{A(t−τ)}bu(τ)dτ
= [ \begin{matrix}−1 + t \\ 1 + t \end{matrix} ] + [ \begin{matrix}(−1 + t)t − \frac{t^{2}}{2} \\ (1 + t)t − \frac{t^{2}}{2} \end{matrix} ] = [ \begin{matrix}\frac{t^{2}}{2} − 1 \\ \frac{t^{2}}{2} + 2t + 1 \end{matrix} ] .

We next show that the invertibility of V (t_{0}, t_{1}) is also necessary for observability. Hence, if V (t_{0}, t_{1}) is singular, there exists a nonzero constant vector, say x_{a}, such that V (t_{0}, t_{1})x_{a} = 0. This implies that

Ce^{At}x_{a} = 0,

and therefore we can write

g(t) = Ce^{At}x_{0} = Ce^{At} (x_{0} + x_{a}) .

Thus, x(0) = x_{0} + x_{a} yields the same response as x(0) = x_{0}, which means that we cannot determine the system state, or in other words, the state fails to be observable. In view of the above result, it follows that by repeating essentially the same arguments used in the previous section we can prove the following theorem:

Theorem 5.6 The following statements are equivalent:

(1) The pair (A, C) is observable;

(2) The matrix V (t_{0}, t_{1}) is nonsingular for all t_{1} > t_{0};

(3) The observability matrix

[ C ; CA ; ⋮ ; CA^{n−1} ] ∈ ℝ^{pn×n}

is of full rank n.

(4)

rank [ \begin{matrix}sI_{n} − A \\ C \end{matrix} ] = n for all s ∈ eig(A).

Example 5.5 For the dynamical system model,

\begin{matrix}ẋ & = & Ax = [ \begin{matrix}2 & 0 \\ 3 & −1 \end{matrix} ] x \\ y & = & cx = [ −1 1 ] x, \end{matrix} \}

we find a non-zero initial vector x(0) = x_{0} such that y(t) = 0 for all t ≥ 0.

We have, x(t) = e^{At}x_{0}, where

e^{At} = L^{−1} ((sI − A)^{−1}) = [ \begin{matrix}e^{2t} & 0 \\ e^{2t} − e^{−t} & e^{−t} \end{matrix} ] .

Our objective is to find x_{0} such that

[ −1 1 ] e^{At}x_{0} = 0 = y(t), for all t ≥ 0.

For example, if

x_{0} = [ 1 1 ]^{⊤} ,

then y(t) = 0 for all t ≥ 0. Note that we were able to find x_{0} ≠ 0 such that y(t) = 0 for all t ≥ 0, because for all t ≥ 0, the columns of ce^{At} are linearly dependent over the real numbers. This is because the pair (A, c) is nonobservable. The vector x_{0} is in the null-space of the observability matrix

[ \begin{matrix}c \\ c & A \end{matrix} ] .

We can also prove analogous result to Theorem 5.4 for the given pair (A, C), that is, using a similarity transformation we can separate the observable part from the nonobservable.

Definition 5.7 If the eigenvalues of the nonobservable part of a given pair (A, C) are all in the open left-hand complex plane, that is, if the nonobservable part is asymptotically stable, then we say that the pair (A, C) is detectable.

◇

A related notion to observability is the notion of constructability.

Definition 5.8 We say that the system

\begin{matrix}ẋ & = & Ax + Bu \\ y & = & Cx + Du, \end{matrix} \}

or equivalently the pair (A, C), is constructable if there is a finite t_{1} > t_{0} such that for arbitrary u(t) and resulting y(t) over [t_{0}, t_{1}] we can determine x(t_{1}) from complete knowledge of the system input u and output y.

◇

One can verify that in the case of continuous, time invariant systems, observability is equivalent to constructability. In the case of discrete-time systems observability implies constructability, and the two notions are equivalent if the matrix A of the given discrete system is nonsingular.

5.3 Minimal Realizations

In Section 1.4, we discussed several methods of converting a transfer function into a statespace representation for both continuous and discrete LTI lumped systems. Recall that if G(s) is a proper rational transfer function matrix then a quadruple of constant matrices (A, B, C, D) is a realization of G(s) if G(s) = C (sI_{n} − A)^{−1} B + D. Recall also that the dimension of a realization is the size of the matrix A, that is, if A is an n-by-n matrix then the dimension of the corresponding realization is n.

Different methods may yield realizations of different dimensions as is the case in Examples 1.7 and 1.8. It is easy to see that a proper rational transfer function matrix has infinitely many realizations of different dimensions. A realization with the smallest possible dimension is called a minimal realization. Necessary and sufficient conditions for a realization to be minimal was given by Kalman [169] in 1963.

Theorem 5.7 A realization (A, B, C, D) of a given transfer function matrix G(s) is minimal if and only if the pair (A, B) is reachable and the pair (A, C) is observable.

We can use Theorem 5.4 to construct a minimal realization using any realization, say (A, B, C, D), of a given transfer function matrix. Indeed, given a realization (A, B, C, D). We check if the pair (A, B) is reachable. If not, we apply Theorem 5.4 to represent (A, B, C, D) in the new coordinates where the reachable part is separated from non-reachable one. In other words, we construct a similarity transformation z = T x such that

Ã = T AT^{−1} = [ \begin{matrix}A_{1} & A_{2} \\ O & A_{4} \end{matrix} ] , B̃ = T B = [ \frac{B_{1}}{O} ] ,

where the pair (A_{1}, B_{1}) is reachable, A_{1} ∈ ℝ^{r×r}, B_{1} ∈ ℝ^{r×m}. The matrix C becomes

C̃ = CT^{−1} = [ C_{1} C_{2} ] ,

where C_{1} ∈ ℝ^{p×r}, and D̃ = D. The transfer functions of (A, B, C, D) and (Ã, B̃, C̃, D) are the same, that is, the transfer function is invariant under similarity transformations. We have

G(s) = C (sI_{n} − A)^{−1} B + D
= C̃ (sI_{n} − Ã)^{−1} B̃ + D
= [ C_{1} C_{2} ] [ \begin{matrix}sI_{r} − A_{1} & −A_{2} \\ O & sI_{n−r} − A_{4} \end{matrix} ]^{−1} [ \frac{B_{1}}{O} ] + D
= [ C_{1} C_{2} ] [ \begin{matrix}[sI_{r} − A_{1}]^{−1} & [sI_{r} − A_{1}]^{−1}A_{2}[sI_{n−r} − A_{4}]^{−1} \\ O & [sI_{n−r} − A_{4}]^{−1} \end{matrix} ] [ \frac{B_{1}}{O} ] + D
= C_{1}[sI_{r} − A_{1}]^{−1}B_{1} + D  (5.16)

Thus the quadruple (A_{1}, B_{1}, C_{1}, D) is a realization of G(s) of lower dimension then the realization (A, B, C, D).

We proceed to check if the pair (A_{1}, C_{1}) is observable or not. If it is, then (A_{1}, B_{1}, C_{1}, D) is a minimal realization of G(s). If, on the other hand, the pair (A_{1}, C_{1}) is not observable, we apply a dual version of Theorem 5.4 to the pair (A_{1}, C_{1}) to extract from it an observable part. This observable part along with corresponding input submatrix and the matrix D form a minimal realization of G(s).

We can obtain a minimal realization of a given transfer function matrix using MATLAB’s function ss. We illustrate the use of this function with a numerical example.

Example 5.6 We obtain a minimal realization of the transfer function matrix considered in Examples 1.7 and 1.8. Recall that

G(s) = [ −\frac{1}{s+2} \frac{1−s}{s^{2}+3s+2} \frac{1}{s^{2}+3s+2}^{\frac{1}{s(s+2)}  \frac{2s−1}{s(s+2)}  \frac{s−1}{s(s+2)}} ] .

We represent the above transfer function matrix as

G=[tf([1],[1 2 0]) tf([2 -1],[1 2 0]) tf([1 -1],[1 2 0]);...
   tf([-1],[1 2]) tf([-1 1],[1 3 2])  tf([1],[1 3 2])]

We then type in

sys = ss(G,'min')

to obtain a fourth dimensional state-space realization of G(s), where

A = [ \begin{matrix}−0.2586 & −0.4843 & 0.1328 & −0.3330 \\ 0.1033 & −1.8720 & 0.7555 & 0.3066 \\ −1.1740 & 0.5232 & −1.1520 & 0.6336 \\ 1.0110 & −0.0522 & 1.1670 & −1.7180 \end{matrix} ] , B = [ \begin{matrix}0.4118 & −0.3924 & −0.4021 \\ 0.6211 & 0.9522 & 0.1655 \\ −0.4053 & −0.9915 & −0.2931 \\ −0.5748 & 1.1860 & 0.8804 \end{matrix} ] ,

C = [ \begin{matrix}−0.4876 & −0.2338 & −1.5020 & 0.4572 \\ −0.8541 & −1.1970 & −0.0004 & −0.1651 \end{matrix} ] , D = O.

It is easy to check, using, for example, MATLAB’s functions ctrb and obsv that the pair (A, B) is reachable and the pair (A, C) is observable, thus confirming the fact that the above quadruple is indeed a minimal realization of the given transfer function matrix G(s).

We may have different minimal realizations of a given transfer function matrix. If this is the case, then they are related via a similarity transformation as shown by Kalman [169]. That is, minimal realizations of a given transfer function matrix are equivalent. Indeed, suppose we have two different minimal realizations (A_{1}, B_{1}, C_{1}, D_{1}) and (A_{2}, B_{2}, C_{2}, D_{2}) of a given proper transfer function matrix G(s). Then

G(s) = C_{1}(sI_{n} − A_{1})^{−1}B_{1} + D_{1} = C_{2}(sI_{n} − A_{2})^{−1}B_{2} + D_{2}

and hence, we have to have

D_{1} = D_{2}

and

C_{1}(sI_{n} − A_{1})^{−1}B_{1} = C_{2}(sI_{n} − A_{2})^{−1}B_{2}.

Taking the inverse Laplace transform of both sides yields

C_{1}e^{A_{1}t}B_{1} = C_{2}e^{A_{2}t}B_{2}.

Taking into account that

e^{At} = I + tA + \frac{t^{2}}{2!}A^{2} + ⋯ = ∑_{k=0}^{∞} \frac{t^{k}}{k!}A^{k}

we obtain

∑_{k=0}^{∞} \frac{t^{k}}{k!}C_{1}A_{1}^{k}B_{1} = ∑_{k=0}^{∞} \frac{t^{k}}{k!}C_{2}A_{2}^{k}B_{2},

that is,

C_{1}A_{1}^{k}B_{1} = C_{2}A_{2}^{k}B_{2}, k = 0, 1, 2, …  (5.17)

The matrices C_{i}A_{i}^{k}B_{i} are called Markov parameters. Thus Markov parameters of different realizations of a given transfer function matrix are equal.

Using (5.17), we obtain

[ C_{1} ; C_{1}A_{1} ; ⋮ ; C_{1}A_{1}^{n−1} ] [ B_{1} A_{1}B_{1} ⋯ A_{1}^{n−1}B_{1} ] = [ C_{2} ; C_{2}A_{2} ; ⋮ ; C_{2}A_{2}^{n−1} ] [ B_{2} A_{2}B_{2} ⋯ A_{2}^{n−1}B_{2} ] .

(5.18)

Let

U_{i} = [ B_{i} A_{i}B_{i} ⋯ A_{i}^{n−1}B_{i} ] , i = 1, 2

and

V_{i} = [ C_{i} ; C_{i}A_{i} ; ⋮ ; C_{i}A_{i}^{n−1} ] , i = 1, 2.

Then, we represent (5.18) in a compact format as

V_{1}U_{1} = V_{2}U_{2}.  (5.19)

Pre-multiplying both sides of (5.19) by V_{1}^{⊤} gives

V_{1}^{⊤}V_{1}U_{1} = V_{1}^{⊤}V_{2}U_{2}.  (5.20)

Both realizations are minimal, therefore their controllability and observability matrices have full ranks, that is,

rank U_{i} = rankV_{i} = n, i = 1, 2.

Hence, the n × n matrix

V_{1}^{⊤}V_{1}

is invertible. We pre-multiply both sides of (5.20) by

[V_{1}^{⊤}V_{1}]^{−1}

to obtain

U_{1} = [V_{1}^{⊤}V_{1}]^{−1} V_{1}^{⊤}V_{2}U_{2}.  (5.21)

Let

T_{1} = [V_{1}^{⊤}V_{1}]^{−1} V_{1}^{⊤}V_{2},  (5.22)

then we represent (5.21) as

U_{1} = T_{1}U_{2}.

We re-write the above as

[ B_{1} A_{1}B_{1} ⋯ A_{1}^{n−1}B_{1} ] = T_{1} [ B_{2} A_{2}B_{2} ⋯ A_{2}^{n−1}B_{2} ]
= [ T_{1}B_{2} T_{1}A_{2}T _{1}^{−1}T_{1}B_{2} ⋯ T_{1}A_{2}^{n−1}T _{1}^{−1}T_{1}B_{2} ] .

taking into account that

T_{1}A_{2}^{k}T _{1}^{−1} = (T_{1}A_{2}T _{1}^{−1})^{k} ,

we conclude that the pairs (A_{1}, B_{1}) and (A_{2}, B_{2}) are equivalent, that is, they are related via the similarity transformation. In other words,

A_{1} = T_{1}A_{2}T _{1}^{−1} and B_{1} = T_{1}B_{2}.

The transformation matrix (5.22) is expressed in terms of the observability matrices. We can express the transformation matrix in terms of the controllability matrices as well. Indeed, post-multiplying both sides of (5.22) by U_{1}^{⊤} gives

V_{1}U_{1}U_{1}^{⊤} = V_{2}U_{2}U_{1}^{⊤}.  (5.23)

The n × n matrix is invertible. We post-multiply both sides of (5.23) by the matrix

[U_{1}U_{1}^{⊤}]^{−1}

to obtain

V_{1} = V_{2}U_{2}U_{1}^{⊤}[U_{1}U_{1}^{⊤}]^{−1}.  (5.24)

Let

T _{2}^{−1} = U_{2}U_{1}^{⊤}[U_{1}U_{1}^{⊤}]^{−1},  (5.25)

then, we can represent (5.24) as

V_{1} = V_{2}T _{2}^{−1}.

We re-write the above as

[ C_{1} ; C_{1}A_{1} ; ⋮ ; C_{1}A_{1}^{n−1} ] = [ C_{2} ; C_{2}A_{2} ; ⋮ ; C_{2}A_{2}^{n−1} ] T _{2}^{−1}
= [ C_{2}T _{2}^{−1} ; C_{2}T _{2}^{−1}T_{2}A_{2}T _{2}^{−1} ; ⋮ ; C_{2}T _{2}^{−1}T_{2}A_{2}^{n−1}T _{2}^{−1} ] .

Taking into account that

T_{2}A_{2}^{k}T _{2}^{−1} = (T_{2}A_{2}T _{2}^{−1})^{k}

we conclude that the pairs (A_{1}, C_{1}) and (A_{2}, C_{2}) are equivalent, that is,

A_{1} = T_{2}A_{2}T _{2}^{−1} and C_{1} = C_{2}T _{2}^{−1}.

It is easy to verify that

T_{1} = T_{2}.

Indeed, it is enough to show that

T_{1}T _{2}^{−1} = I_{n}.

Using (5.22) and (5.25) we obtain,

T_{1}T _{2}^{−1} = [V_{1}^{⊤}V_{1}]^{−1} V_{1}^{⊤}V_{2}U_{2}U_{1}^{⊤}[U_{1}U_{1}^{⊤}]^{−1}.

Substituting into the above (5.19), that is, V_{2}U_{2} = V_{1}U_{1}, gives

T_{1}T _{2}^{−1} = [V_{1}^{⊤}V_{1}]^{−1} V_{1}^{⊤}V_{1}U_{1}U_{1}^{⊤}[U_{1}U_{1}^{⊤}]^{−1}
= I_{n}.

Hence,

T_{1} = T_{2} = T .

In view of the above, we conclude that different minimal realizations of the same transfer function matrix are equivalent, that is,

A_{1} = T A_{2}T^{−1}, B_{1} = T B_{2}, C_{1} = C_{2}T^{−1}, D_{1} = D_{2},  (5.26)

where the transformation matrix T can be computed using (5.22) or (5.25).

5.4 Companion Forms

In this section, we present two types of special forms of a given linear dynamical system model represented by a quadruple (A, B, C, D). These forms are obtained by a change of state variables using similarity transformations. They reveal structural properties of the model. We use the above mentioned forms to construct algorithms for pole placement. The pole placement algorithms are then used to design state-feedback controllers, as well as observers. We begin by introducing the controller companion form, or for short, controller form.

5.4.1 Controller Form

We begin by considering a single-input, continuous-time, LTI model,

ẋ(t) = Ax(t) + bu(t),

where the pair (A, b) is assumed to be reachable, or equivalently, controllable. This means that

rank [ b Ab ⋯ A^{n−1}b ] = n.

We select the last row of the inverse of the controllability matrix. Let q_{1} be that row. Next, we form the matrix

T = [ q_{1} ; q_{1}A ; ⋮ ; q_{1}A^{n−1} ] .  (5.27)

Note that T is invertible. Indeed, because

T [ b Ab ⋯ A^{n−1}b ] = [ \begin{matrix}0 & 0 & ⋯ & 0 & 1 \\ 0 & 0 & ⋯ & 1 & x \\ ⋮ & ⋮ & & ⋮ & ⋮ \\ 1 & x & ⋯ & x & x \end{matrix} ] ,

this implies that T must be nonsingular. In the above, we used the symbol x to denote “don’t care”, unspecified, scalars in our present discussion. Consider now the state variable transformation, x̃ = T x. Then, in the new coordinates, the system model is

˙x̃(t) = T AT^{−1}x̃(t) + T bu(t)
= Ãx̃(t) + b̃u(t).

The matrices Ã and b̃ have particular structures that we now discuss. We first analyze the structure of b̃. Because q_{1} is the last row of the inverse of the controllability matrix, we have

q_{1}b = q_{1}Ab = ⋯ = q_{1}A^{n−2}b = 0,

and

q_{1}A^{n−1}b = 1.

Hence,

b̃ = T b = [ q_{1}b ; q_{1}Ab ; ⋮ ; q_{1}A^{n−2}b ; q_{1}A^{n−1}b ] = [ 0 0 ⋮ 0 1 ] .

The structure of Ã is revealed by considering the relation T AT^{−1} = Ã represented as

T A = ÃT .

The left-hand side of the above matrix equation is

T A = [ q_{1}A ; q_{1}A^{2} ; ⋮ ; q_{1}A^{n−1} ; q_{1}A^{n} ] .

By the Cayley-Hamilton theorem,

A^{n} = −a_{0}I_{n} − a_{1}A − ⋯ − a_{n−1}A^{n−1},

and hence

q_{1}A^{n} = −a_{0}q_{1} − a_{1}q_{1}A − ⋯ − a_{n−1}q_{1}A^{n−1}.

Comparing both sides of the equation T A = ÃT, and taking into account the Cayley-Hamilton theorem, we get

Ã = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & 1 & ⋯ & 0 & 0 \\ ⋮ & ⋮ & & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & 1 \\ −a_{0} & −a_{1} & −a_{2} & ⋯ & −a_{n−2} & −a_{n−1} \end{matrix} ] .

Note that the coefficients of the characteristic polynomial of A are immediately apparent by inspecting the last row of Ã. There are other benefits of the pair (A, b) being given in its controller form that we discuss in Section 5.5. We illustrate the above algorithm for reducing a single-input system into the controller form with a numerical example.

Example 5.7 Consider the following dynamical system model:

ẋ = [ \begin{matrix}0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ −2 & 1 & 0 & 0 \\ 1 & −1 & 0 & 0 \end{matrix} ] x + [ \begin{matrix}0 \\ 0 \\ 1 \\ 0 \end{matrix} ] u.

The controllability matrix of the pair (A, b) is

[ b Ab A^{2}b A^{3}b ] = [ \begin{matrix}0 & 1 & 0 & −2 \\ 0 & 0 & 0 & 1 \\ 1 & 0 & −2 & 0 \\ 0 & 0 & 1 & 0 \end{matrix} ] .

The pair (A, b) is controllable because the controllability matrix is nonsingular. We find the last row of the inverse of the controllability matrix to be q_{1} = [ 0 1 0 0 ]. Hence the transformation matrix is

T = [ \begin{matrix}q & 1 \\ q & 1 & A \\ q & 1 & A & 2 \\ q & 1 & A & 3 \end{matrix} ] = [ \begin{matrix}0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ 1 & −1 & 0 & 0 \\ 0 & 0 & 1 & −1 \end{matrix} ] .

The matrices A and b in the new coordinates have the form,

T AT^{−1} = [ \begin{matrix}0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ 1 & −1 & 0 & 0 \\ 0 & 0 & 1 & −1 \end{matrix} ] [ \begin{matrix}0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ −2 & 1 & 0 & 0 \\ 1 & −1 & 0 & 0 \end{matrix} ] [ \begin{matrix}1 & 0 & 1 & 0 \\ 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 1 \\ 0 & 1 & 1 & 0 \end{matrix} ] = [ \begin{matrix}0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ −1 & 0 & −3 & 0 \end{matrix} ] ,

and

T b = [ \begin{matrix}0 \\ 0 \\ 0 \\ 1 \end{matrix} ] .

We generalize the above method of transforming the given pair (A, B) into the controller form for multi-input systems. We consider a controllable pair (A, B) where A ∈ ℝ^{n×n}, B ∈ ℝ^{n×m}, m ≤ n, and rank B = m. The last assumption we made is just for convenience to simplify further manipulations. This assumption implies that all inputs are mutually independent, which is usually the case in practical applications. We first form the controllability matrix of the pair (A, B) and represent it as

[ b_{1} b_{2} ⋯ b_{m} Ab_{1} Ab_{2} ⋯ Ab_{m} ⋯ A^{n−1}b_{1} ⋯ A^{n−1}b_{m} ] ,  (5.28)

where b_{i}, i = 1, 2, …, m, is the i-th column of B. The above controllability matrix is of full rank because we assumed that the pair (A, B) is controllable. Thus, we can select n linearly independent columns from (5.28). We select these columns proceeding from left to right. Then, we rearrange the selected columns to form a n × n nonsingular matrix L of the form

L = [ b_{1} Ab_{1} ⋯ A^{d_{1}−1}b_{1} b_{2} ⋯ A^{d_{2}−1}b_{2} ⋯ b_{m} ⋯ A^{d_{m}−1}b_{m} ] .

We call the m integers d_{i} the controllability indices of the pair (A, B). Note that

∑_{i=1}^{m} d_{i} = n.

We call the integer

d = max{d_{i} : i = 1, 2, …, m},

the controllability index of the pair (A, B). Observe that all m columns of B are present in L because we assumed that B is of full rank and the way we selected mutually independent columns from the controllability matrix (5.28) to construct the matrix L. Furthermore, if the column A^{k}b_{j} is present in L, then so is A^{k−1}b_{j}.

To proceed further, we let

σ_{k} = ∑_{i=1}^{k} d_{i}, for k = 1, 2, …, m.

Note that σ_{1} = d_{1}, σ_{2} = d_{1} + d_{2}, …, σ_{m} = ∑_{i=1}^{m} d_{i} = n. We select m rows from L^{−1}. We denote the selected rows by q_{k}, where q_{k} is the σ_{k}-th row of L^{−1}. Next, we form the n × n matrix

T = [ q_{1} ; q_{1}A ; ⋮ ; q_{1}A^{d_{1}−1} ; q_{2} ; ⋮ ; q_{2}A^{d_{2}−1} ; ⋮ ; q_{m} ; ⋮ ; q_{m}A^{d_{m}−1} ] .

We can show that T is nonsingular by computing the product T L and noticing that det(T L) = ±1. We define a state-variable transformation x̃ = T x, and then using similar arguments as in the single-input case, we arrive at the special structure of the pair

(Ã, B̃) = (T AT^{−1}, T B) ,

where

Ã = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 & 0 & 0 & ⋯ & 0 & ⋯ & 0 & 0 & ⋯ & 0 \\ 0 & 0 & 1 & ⋯ & 0 & 0 & 0 & ⋯ & 0 & & 0 & 0 & ⋯ & 0 \\ & & & ... & & ⋮ & & & ⋮ & & ⋮ & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 1 & 0 & 0 & ⋯ & 0 & & 0 & 0 & ⋯ & 0 \\ x & x & x & ⋯ & x & x & x & ⋯ & x & ⋯ & x & x & ⋯ & x \\ 0 & 0 & 0 & ⋯ & 0 & 0 & 1 & ⋯ & 0 & ⋯ & 0 & 0 & ⋯ & 0 \\ ⋮ & & & & ⋮ & ⋮ & & ... & ⋮ & & ⋮ & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & 0 & 0 & ⋯ & 1 & & 0 & 0 & ⋯ & 0 \\ x & x & x & ⋯ & x & x & x & ⋯ & x & ⋯ & x & x & ⋯ & x \\ ⋮ & & & & & & ⋮ & & & ... & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & 0 & 0 & ⋯ & 0 & ⋯ & 0 & 1 & ⋯ & 0 \\ ⋮ & & & & ⋮ & ⋮ & & & ⋮ & & ⋮ & & ... & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & 0 & 0 & ⋯ & 0 & & 0 & 0 & ⋯ & 1 \\ x & x & x & ⋯ & x & x & x & ⋯ & x & ⋯ & x & x & ⋯ & x \end{matrix} ] .

Note that the m diagonal blocks of Ã have the structure of the controller form as in the single-input case. The dimensions of the diagonal blocks are d_{i} × d_{i}, i = 1, 2, …, m. The off-diagonal blocks consist of zero elements except for their respective last rows. Thus, all information about Ã, as well as A, can be derived from knowledge of the controllability indices, and the m nonzero rows containing the elements marked using x, of the matrix Ã.

The matrix B̃ has the form,

B̃ = [ \begin{matrix}0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ 1 & x & x & x & x \\ 0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ 0 & 1 & x & x & x \\ ⋮ & & & & ⋮ \\ 0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ 0 & 0 & 0 & 0 & 1 \end{matrix} ] .

The nonzero rows of B̃ correspond to the rows of Ã containing the elements marked using x.

We illustrate the above algorithm for reducing a multi-input system model into the controller form with a numerical example.

Example 5.8 Consider the pair (A, B), where

A = [ \begin{matrix}−1 & 1 & 0 & 1 \\ 1 & 1 & 0 & 0 \\ 0 & 0 & 4 & 0 \\ 1 & 1 & −1 & 2 \end{matrix} ] , B = [ \begin{matrix}1 & 1 \\ 2 & 0 \\ 1 & 0 \\ 0 & 1 \end{matrix} ] .

We transform the above pair into its controller companion form. Following the algorithm, we find,

rank [ b_{1} b_{2} Ab_{1} Ab_{2} ] = 4.

Hence d_{1} = d_{2} = 2. We next form the matrix L,

L = [ b_{1} Ab_{1} b_{2} Ab_{2} ] = [ \begin{matrix}1 & 1 & 1 & 0 \\ 2 & 3 & 0 & 1 \\ 1 & 4 & 0 & 0 \\ 0 & 2 & 1 & 3 \end{matrix} ] ,

and

L^{−1} = [ \begin{matrix}0.20 & 0.60 & −0.40 & −0.20 \\ −0.05 & −0.15 & 0.35 & 0.05 \\ 0.85 & −0.45 & 0.05 & 0.15 \\ −0.25 & 0.25 & −0.25 & 0.25 \end{matrix} ] .

Hence

q_{1} = [ −0.05 −0.15 0.35 0.05 ] ,

and

q_{2} = [ −0.25 0.25 −0.25 0.25 ] .

The transformation matrix is

T = [ \begin{matrix}q & 1 \\ q & 1 & A \\ q & 2 \\ q & 2 & A \end{matrix} ] = [ \begin{matrix}−0.05 & −0.15 & 0.35 & 0.05 \\ −0.05 & −0.15 & 1.35 & 0.05 \\ −0.25 & 0.25 & −0.25 & 0.25 \\ 0.75 & 0.25 & −1.25 & 0.25 \end{matrix} ] .

The matrices A and B in the new coordinates have the form,

T AT^{−1} = [ \begin{matrix}0 & 1 & 0 & 0 \\ −4 & 5 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ 3 & −3 & 4 & 1 \end{matrix} ] and T B = [ \begin{matrix}0 & 0 \\ 1 & 0 \\ 0 & 0 \\ 0 & 1 \end{matrix} ] .

5.4.2 Observer Form

In this subsection, we use duality and the algorithms for reducing reachable systems into controller forms to derive analogous results for observable systems. For the sake of simplicity and without loss of generality, we consider an observable dynamical system model

\begin{matrix}ẋ(t) & = & Ax(t) \\ y(t) & = & Cx(t), \end{matrix} \}  (5.29)

or, equivalently, the observable pair (A, C), where A ∈ ℝ^{n×n}, C ∈ ℝ^{p×n}, p ≤ n, and rank C = p. We omitted the input variable u for convenience because it plays no role in the subsequent discussion. We next consider the dual system model associated with the system (5.29),

ż(t) = A^{⊤}z(t) + C^{⊤}v(t).

Because the pair (A, C) is observable, the dual pair (A^{⊤}, C^{⊤}) is reachable. Therefore, the pair (A^{⊤}, C^{⊤}) can be transformed into the controller form using the algorithm from the previous subsection. We then take the dual of the result to get

\begin{matrix}˙x̂(t) & = & Âx̂ \\ y(t) & = & Ĉx̂, \end{matrix} \}

where

Â = [ \begin{matrix}Â_{11} & Â_{12} & ⋯ & Â_{1p} \\ Â_{21} & Â_{22} & ⋯ & Â_{2p} \\ ⋮ & & & ⋮ \\ Â_{p1} & Â_{p2} & ⋯ & Â_{pp} \end{matrix} ] .

Each diagonal submatrix Â_{ii} ∈ ℝ^{d̂_{i}×d̂_{i}} of Â has the form

Â_{ii} = [ \begin{matrix}0 & 0 & ⋯ & 0 & x \\ 1 & 0 & ⋯ & 0 & x \\ ⋮ & & ... & & ⋮ \\ 0 & 0 & ⋯ & 1 & x \end{matrix} ] ,

which is just the transpose of the controller form for the single-input case. The integers d̂_{i}, i = 1, 2, …, p are the controllability indices of the controllable pair (A^{⊤}, C^{⊤}). The off-diagonal blocks, j ≠ i, have the form

Â_{ji} = [ \begin{matrix}0 & 0 & ⋯ & 0 & x \\ 0 & 0 & ⋯ & 0 & x \\ ⋮ & & & & ⋮ \\ 0 & 0 & ⋯ & 0 & x \end{matrix} ] ,

that is, they consist of zeros except for their last columns. The matrix Ĉ consists of p blocks and has the form

Ĉ = [ \begin{matrix}0 & 0 & ⋯ & 1 & 0 & 0 & ⋯ & 0 & ⋯ & 0 & 0 & ⋯ & 0 \\ 0 & 0 & ⋯ & x & 0 & 0 & ⋯ & 1 & ⋯ & 0 & 0 & ⋯ & 0 \\ ⋮ & & & ⋮ & ⋮ & & & ⋮ & & ⋮ & & & ⋮ \\ 0 & 0 & ⋯ & x & 0 & 0 & ⋯ & x & ⋯ & 0 & 0 & ⋯ & 1 \end{matrix} ] .

The p controllability indices of the pair (A^{⊤}, C^{⊤}) are called the observability indices of the pair (A, C). Note that

∑_{i=1}^{p} d̂_{i} = n.

Similarly as in the controller form, we define the observability index of the pair (A, C) to be the integer

d̂= max{d̂_{i} : i = 1, 2, …, p}.

We illustrate the above algorithm for transforming an observable pair into the observer form with a numerical example.

Example 5.9 Consider the observable pair (A, C),

A = [ \begin{matrix}0 & 0 & 1 & 0 \\ 1 & 0 & 2 & 0 \\ 0 & 1 & 3 & 0 \\ 0 & 0 & −21 & 5 \end{matrix} ] , C = [ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ] .

We transform the pair (A^{⊤}, C^{⊤}) into the controller form. Thus, we first form the controllability matrix for (A^{⊤}, C^{⊤}), then proceeding from left to right, we select the first n = 4 linearly independent columns. They are

c_{1}^{⊤}, c_{2}^{⊤}, A^{⊤}c_{1}^{⊤}, (A^{⊤})^{2} c_{1}^{⊤}.

Therefore, the observability indices are: d̂_{1} = 3 and d̂_{2} = 1. Hence,

L = [ c_{1}^{⊤} A^{⊤}c_{1}^{⊤} (A^{⊤})^{2} c_{1}^{⊤} c_{2}^{⊤} ] = [ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 1 & 3 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ] ,

and

L^{−1} = [ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & −3 & 1 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ] .

The two vectors that we need to form the desired transformation are

q_{1} = [ 0 1 0 0 ] and q_{2} = [ 0 0 0 1 ] .

The transformation matrix is

T̂ = [ \begin{matrix}0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 1 & 2 & 3 & −21 \\ 0 & 0 & 0 & 1 \end{matrix} ] .

The observer form of the pair (A, C) is given by the matrices

Â = T̂^{−T} AT̂^{⊤} = [ \begin{matrix}0 & 0 & 1 & 0 \\ 1 & 0 & 2 & 0 \\ 0 & 1 & 3 & 0 \\ 0 & 0 & −105 & 5 \end{matrix} ] and Ĉ = CT̂^{⊤} = [ \begin{matrix}0 & 0 & 1 & 0 \\ 0 & 0 & −21 & 1 \end{matrix} ] .

5.5 Linear State-Feedback Control

In the following sections, we discuss methods for a feedback controller design for linear systems. The design process involves three steps. The first step is the design of a statefeedback control law. In this step, we assume that we have all states available to us for feedback purposes. In general, this may not be the case. The assumption that all the states are available allows us to proceed with the state-feedback control law design. We then proceed with the second step that involves the design of an observer to estimate the state vector. The observer should use only the input and output of the plant to be controlled to generate an estimate of the plant state. The last step is to combine the first two steps in that the control law, designed in the first step, uses the state estimate instead of the true state vector. The result of this step is a combined controller-observer compensator. We first discuss the problem of designing a linear state-feedback control law. In the following sections, we present methods for constructing observers and combined controller-observer compensators.

The linear state-feedback control law, for a system modeled by ẋ = Ax + Bu, is the feedback of a linear combination of all the state variables, and has the form

u = −Kx + v,

where K ∈ ℝ^{m×n} is a constant matrix. Figure 5.4 illustrates the linear state-feedback applied to the linear system ẋ = Ax + Bu.

Figure 5.4: Linear state-feedback.

The closed-loop system is described as

ẋ = (A − BK) x + Bv.

The poles of the closed-loop system are the roots of the characteristic equation

det (sI_{n} − A + BK) = 0.

The linear state-feedback control law design consists of selecting the gains

k_{ij}, i = 1, 2, …, m, j = 1, 2, …, n,

so that the roots of the closed-loop characteristic equation

det (sI_{n} − A + BK) = 0

are in desirable locations in the complex plane. We assume that a designer has made the selection of the desired poles of the closed-loop system and they are:

s_{1}, s_{2}, …, s_{n}.

The desired closed-loop poles can be real or complex. If they are complex, then they must come in complex conjugate pairs. This is because we use only real gains k_{ij}. Having selected the desired closed-loop poles we can form the desired closed-loop characteristic polynomial,

α_{c}(s) = (s − s_{1})(s − s_{2}) ⋯ (s − s_{n})
= s^{n} + α_{n−1}s^{n−1} + ⋯ + α_{1}s + α_{0}.

Our goal is to select a feedback matrix K such that

det (sI_{n} − A + BK) = s^{n} + α_{n−1}s^{n−1} + ⋯ + α_{1}s + α_{0}.

The above problem is also referred to as the pole placement problem. We first discuss the pole placement problem for the single-input plants. In this case K = k ∈ ℝ^{1×n}. The solution to the problem is easily obtained if the pair (A, b) is already in the controller companion form. In such a case we have

A − bk = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & 1 & ⋯ & 0 & 0 \\ ⋮ & & & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & 1 \\ −a_{0} − k_{1} & −a_{1} − k_{2} & −a_{2} − k_{3} & ⋯ & −a_{n−2} − k_{n−1} & −a_{n−1} − k_{n} \end{matrix} ] .

Hence, the desired gains are

k_{1} = α_{0} − a_{0},
k_{2} = α_{1} − a_{1},
⋮
k_{n} = α_{n−1} − a_{n−1}.

If the pair (A, b) is not in the controller companion form, we first transform it into the companion form, and then compute the gain vector k̃ such that

det (sI_{n} − Ã + b̃k̃) = s^{n} + α_{n−1}s^{n−1} + ⋯ + α_{1}s + α_{0}.

Thus,

k̃ = [ α_{0} − a_{0} α_{1} − a_{1} ⋯ α_{n−1} − a_{n−1} ] .

Then,

k = k̃T ,

where T is the transformation that brings the pair (A, b) into the controller companion form.

We can represent the above formula for the gain matrix in an alternative way. For this note that

k̃T = [ α_{0} − a_{0} α_{1} − a_{1} ⋯ α_{n−1} − a_{n−1} ] [ q_{1} ; q_{1}A ; ⋮ ; q_{1}A^{n−1} ]

= q_{1} (α_{0}I_{n} + α_{1}A + ⋯ + α_{n−1}A^{n−1}) − q_{1} (a_{0}I_{n} + a_{1}A + ⋯ + a_{n−1}A^{n−1}) .

By the Cayley-Hamilton theorem,

A^{n} = − (a_{0}I_{n} + a_{1}A + ⋯ + a_{n−1}A^{n−1}) .

Hence,

k = q_{1}α_{c} (A) .

The above expression for the gain row vector was proposed by Ackermann in 1972, and is now referred to as the Ackermann’s formula for pole placement.

Example 5.10 For the linear dynamical system

ẋ = [ \begin{matrix}1 & −1 \\ 1 & −2 \end{matrix} ] x + [ \begin{matrix}2 \\ 1 \end{matrix} ] u,

we will use the Ackermann’s formula to design a state-feedback controller, u = −kx, such that the closed-loop poles are located at {−1, −2}.

To use the Ackermann’s formula, we first form the controllability matrix of the pair (A, b) and then find the last row of its inverse denoted q_{1}. The controllability matrix is

U = [ b Ab ] = [ \begin{matrix}2 & 1 \\ 1 & 0 \end{matrix} ] .

Its inverse is

U^{−1} = [ \begin{matrix}0 & 1 \\ 1 & −2 \end{matrix} ] .

Hence, q_{1} = [ 1 −2 ]. The desired closed-loop characteristic polynomial is

α_{c}(s) = (s + 1)(s + 2) = s^{2} + 3s + 2.

Therefore,

k = q_{1}α_{c}(A)
= q_{1} (A^{2} + 3A + 2I_{2})
= q_{1} ([ \begin{matrix}1 & −1 \\ 1 & −2 \end{matrix} ] [ \begin{matrix}1 & −1 \\ 1 & −2 \end{matrix} ] + 3 [ \begin{matrix}1 & −1 \\ 1 & −2 \end{matrix} ] + 2 [ \begin{matrix}1 & 0 \\ 0 & 1 \end{matrix} ])
= q_{1} ([ \begin{matrix}0 & 1 \\ −1 & 3 \end{matrix} ] + [ \begin{matrix}3 & −3 \\ 3 & −6 \end{matrix} ] + [ \begin{matrix}2 & 0 \\ 0 & 2 \end{matrix} ])
= [ 1 −2 ] [ \begin{matrix}5 & −2 \\ 2 & −1 \end{matrix} ]
= [ 1 0 ]

We now present a pole placement algorithm for multi-input systems. If the pair (A, B) is already in the controller companion form, then we may proceed as follows. First, represent the matrix B as

B = [ \begin{matrix}0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ 1 & x & x & x & x \\ 0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ 0 & 1 & x & x & x \\ ⋮ & & & & ⋮ \\ 0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ 0 & 0 & 0 & 0 & 1 \end{matrix} ] = [ \begin{matrix}0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ 1 & 0 & 0 & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ 0 & 1 & 0 & 0 & 0 \\ ⋮ & & & & ⋮ \\ 0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ 0 & 0 & 0 & 0 & 1 \end{matrix} ] [ \begin{matrix}1 & x & ⋯ & x & x \\ 0 & 1 & ⋯ & x & x \\ ⋮ & & ... & & ⋮ \\ 0 & 0 & ⋯ & 1 & x \\ 0 & 0 & ⋯ & 0 & 1 \end{matrix} ] = B̂Γ,

where the square and nonsingular matrix Γ consists of the nonzero rows of B. Next, let

K̂ = ΓK.

Notice that

B̂K̂ = [ \begin{matrix}0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ k̂_{11} & k̂_{12} & ⋯ & k̂_{1 n−1} & k̂_{1n} \\ 0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ k̂_{21} & k̂_{22} & ⋯ & k̂_{2 n−1} & k̂_{2n} \\ ⋮ & & & & ⋮ \\ 0 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & ⋯ & 0 & 0 \\ ⋮ & & & & ⋮ \\ k̂_{m1} & k̂_{m2} & ⋯ & k̂_{m n−1} & k̂_{mn} \end{matrix} ] ,

that is, the nonzero rows in the product B̂K̂ match the nonzero rows of the matrix A in the controller companion form. We can then, for example, select the gains k_{ij}, i = 1, 2, …, m, j = 1, 2, …, n, so that

A − BK = A − B̂K̂ = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & 1 & ⋯ & 0 & 0 \\ ⋮ & & & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & 1 \\ −α_{0} & −α_{1} & −α_{2} & ⋯ & −α_{n−2} & −α_{n−1} \end{matrix} ] ,

where

K = Γ^{−1}K̂.

If the pair (A, B) is not in the controller companion form, we first transform it into this form, then compute the gain matrix that allocates the closed-loop poles into the desired locations. The gain matrix K that allocates the closed-loop poles into the prespecified locations for the pair (A, B) in the original coordinates is then given by

K = Γ^{−1}K̂T ,

where T is the transformation matrix that brings the pair (A, B) into the controller companion form.

Example 5.11 For the pair

A = [ \begin{matrix}0 & 0 & 1 & 0 \\ 1 & 0 & 2 & 0 \\ 0 & 1 & 3 & 0 \\ 0 & 0 & −21 & 5 \end{matrix} ] , B = [ \begin{matrix}1 & 0 \\ 0 & 0 \\ 0 & 0 \\ 0 & 1 \end{matrix} ] ,

we use its controller companion form to find the matrix K ∈ ℝ^{2×4} so that closed-loop poles are located at

−2, −3 + j, −3 − j, −4.

We first transform the pair (A, B) into its controller companion form. We begin by forming the controllability matrix

[ b_{1} b_{2} Ab_{1} Ab_{2} A^{2}b_{1} ⋯] = [ \begin{matrix}1 & 0 & 0 & 0 & 0 & ⋯ \\ 0 & 0 & 1 & 0 & 0 & ⋯ \\ 0 & 0 & 0 & 0 & 1 & ⋯ \\ 0 & 1 & 0 & 5 & 0 & ⋯ \end{matrix} ] .

We then select, proceeding from left to right, the first four linearly independent columns from the above controllability matrix. We obtain

[ b_{1} b_{2} Ab_{1} A^{2}b_{1} ] .

Thus, the controllability indices are d_{1} = 3 and d_{2} = 1. We rearrange the above columns and form the matrix L of the form

L = [ b_{1} Ab_{1} A^{2}b_{1} b_{2} ] = I_{4} = L^{−1}.

The two row vectors that we need in constructing the transformation are

q_{1} = [ 0 0 1 0 ] and q_{2} = [ 0 0 0 1 ] .

The transformation matrix is

T = [ \begin{matrix}q & 1 \\ q & 1 & A \\ q & 1 & A & 2 \\ q & 2 \end{matrix} ] = [ \begin{matrix}0 & 0 & 1 & 0 \\ 0 & 1 & 3 & 0 \\ 1 & 3 & 11 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ] .

The pair (A, B) in the new coordinates has the form

Ã = T AT^{−1} = [ \begin{matrix}0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 1 & 2 & 3 & 0 \\ −21 & 0 & 0 & 5 \end{matrix} ] and B̃ = T B = [ \begin{matrix}0 & 0 \\ 0 & 0 \\ 1 & 0 \\ 0 & 1 \end{matrix} ] .

The desired closed-loop characteristic polynomial is

α_{c}(s) = (s + 2)(s + 3 − j)(s + 3 + j)(s + 4) = s^{4} + 12s^{3} + 54s^{2} + 108s + 80.

A possible choice of the gain matrix K̃, from among many, that does the job is K̃ such that

Ã − B̃K̃ = [ \begin{matrix}0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ −80 & −108 & −54 & −12 \end{matrix} ] .

Note that Γ = I_{2}. Therefore,

K̃ = [ \begin{matrix}1 & 2 & 3 & −1 \\ 59 & 108 & 54 & 17 \end{matrix} ] ,

and thus

K = K̃T = [ \begin{matrix}3 & 11 & 40 & −1 \\ 54 & 270 & 977 & 17 \end{matrix} ] .

The above algorithm for pole placement for multi-input systems is more of a theoretical value rather than practical. This algorithm is not well suited for numerical implementations because the transformation of a given pair (A, B) into the controller companion form suffers from poor numerical properties. For a discussion of pole placement numerically stable algorithms for multi-input systems, we refer the reader to Jamshidi, Tarokh, and Shafai [160].

Unlike in the single-input case, a solution to the pole placement problem for a multiinput system is not unique. Therefore, one can use the remaining degrees of freedom to achieve secondary goals. We discuss a method for constructing a linear state-feedback control law that allocates the closed-loop poles into prespecified locations, and at the same time minimizes a quadratic performance index, in Subsection 9.3.3.

Using the results preceding the above discussion, we state and prove a fundamental theorem of linear systems.

Theorem 5.8 The pole placement problem is solvable for all choices of n desired closed-loop poles, symmetric with respect to the real axis, if and only if the given pair (A, B) is reachable.

Proof The proof of necessity (⇐) follows immediately from the discussion preceding Example 5.11. Indeed, the pair (A, B) is reachable, if and only if it can be transformed into the controller canonical form. Once the transformation is performed, we can easily solve the pole placement problem for the given set of desired closed-loop poles, symmetric with respect to the real axis. We then transform the pair (A, B) and the gain matrix back into the original coordinates. Because the similarity transformation does not affect neither controllability nor the characteristic polynomial, the eigenvalues of A − BK are precisely the desired prespecified closed-loop poles. This completes the proof of the necessity part.

We use a proof by contraposition to prove the sufficiency part (⇒). Assume that the pair (A, B) is nonreachable. By Theorem 5.4, there is a similarity transformation z = T x such that the pair (A, B) in the new coordinates has the form

Ã = T AT^{−1} = [ \begin{matrix}A_{1} & A_{2} \\ O & A_{4} \end{matrix} ] , B̃ = T B = [ \begin{matrix}B & 1 \\ O \end{matrix} ] ,

where the pair (A_{1}, B_{1}) is reachable, A_{1} ∈ ℝ^{r×r}, and B_{1} ∈ ℝ^{r×m}. Let

u = −K̃z = − [ K_{1} K_{2} ] z,

where K_{1} ∈ ℝ^{m×r} and K_{2} ∈ ℝ^{m×(n−r)}. Then,

Ã − B̃K̃ = [ \begin{matrix}A_{1} − B_{1}K_{1} & A_{2} − B_{1}K_{2} \\ O & A_{4} \end{matrix} ] .  (5.30)

It follows from (5.30) that the nonreachable portion of the system is not affected by the state-feedback, that is, the eigenvalues of A_{4} cannot be allocated. Hence, the pole placement problem cannot be solved in general if the pair (Ã, B̃), or equivalently the pair (A, B), is nonreachable, which completes the proof of the sufficiency part.

□

For an alternative proof of this important result, the reader may consult Wonham [355].

5.6 Observers

In the previous section, we presented algorithms for solving the pole placement problem using linear state-feedback control law. To implement such a control law, one needs availability of all the state variables. Often, this requirement is not met, either because measuring of all the state variables would require excessive number of sensors, or because the state variables are not accessible for direct measurement. Instead, only a subset of state variables or their combination may be available. To implement the control law constructed in the previous section, we will use estimates of state variables rather than the true states.

We now discuss the problem of constructing observers. Much of the literature refers to observers as “state estimators.” For example, Franklin and Powell [106, page 139] argue that estimator is much more descriptive in its function because observer implies a direct measurement. On the other hand, estimator implies non-deterministic approach. Here we use only a deterministic approach. Therefore we follow the original terminology of the observer’s inventor.

The first observer was proposed by Luenberger in the early nineteen sixties [221, 222, 223] for the purpose of estimating the state of a dynamical system, referred to as a plant, based on limited measurements of that system. More specifically, an observer is a deterministic dynamical system that can generate an estimate of the plant’s state using that plant’s input and output signals. A block diagram of a general observer structure is depicted in Figure 5.5. The plant’s state estimate can then be employed, in place of the true state, to close the control loop. Observers can be used as “software” or “virtual” sensors as opposed to hardware sensing devices directly measuring physical variables, thus augmenting or replacing sensors in a control system [94]. Observers have been applied in a variety of different areas such

Figure 5.5: General observer architecture.

as secure communication using chaotic synchronization [120, 16, 214, 216, 163, 58], machine vision [59], wind energy systems [296], speed-sensorless control of induction motors [81, 80], or in model-based predictive control (MPC) [240], among many other applications.

We present two types of observers. First, full-order observers are discussed and then a reduced-order observers are presented.

5.6.1 Full-Order Observer

We discuss the problem of constructing a full-order observer for dynamical systems modeled by

\begin{matrix}ẋ(t) & = & Ax(t) + Bu(t) \\ y(t) & = & Cx(t), \end{matrix} \}

where C ∈ ℝ^{p×n}, p ≤ n. We assume that the pair (A, C) is observable. Our goal is to construct a dynamical system that will estimate the state vector x based on the plant input u and the plant output y. One could consider constructing a model of the plant dynamics and connecting the resulting dynamical system, referred to as an open-loop observer, as in Figure 5.6. An alternative representation of the open-loop observer is depicted in Figure 5.7. This open-loop observer is described by

˙x̃(t) = Ax̃(t) + Bu(t),

where x̃(t) is the estimate of x(t). Let

e(t) = x(t) − x̃(t)

be the estimation error. Then, the dynamics of the observation error are described by

ė(t) = ẋ(t) − ˙ x̃(t) = Ae(t),

Figure 5.6: Open-loop observer.

with the initial estimation error

e(0) = x(0) − x̃(0).

If the eigenvalues of the matrix A are in the open left-hand plane, then the error converges to zero. However, we have no control over the convergence rate. Furthermore, the matrix A does not have to have all its eigenvalues in the open left-hand plane. Thus, the openloop observer is impractical. We modify this estimator by adding a feedback term to it. The resulting structure, depicted in Figure 5.8, is called the closed-loop observer, or the Luenberger observer, or the asymptotic full-order observer. An alternative representation of the closed-loop observer is depicted in Figure 5.9. The dynamics of the closed-loop observer are described by

˙x̃(t) = Ax̃(t) + Bu(t) + L (y(t) − Cx̃(t)) ,

and the dynamics of the observation error are governed by

ė(t) = ẋ(t) − ˙ x̃(t)
= (A − LC) e(t), e(0) = x(0) − x̃(0).

The pair (A, C) is observable, if and only if the dual pair (A^{⊤}, C^{⊤}) is reachable. By assumption, the pair (A, C) is observable, and therefore the pair (A^{⊤}, C^{⊤}) is reachable. By Theorem 5.8, we can solve the pole placement problem for the dual pair (A^{⊤}, C^{⊤}), that is, 5.6. OBSERVERS 261

Figure 5.7: Alternative representation of the open-loop observer. Figure 5.8: Closed-loop observer.

Figure 5.9: Alternative representation of the closed-loop observer.

for any set of prespecified n complex numbers, symmetric with respect to the real axis, there is a matrix, call it L^{⊤}, such that the eigenvalues of A^{⊤} − C^{⊤}L^{⊤} and hence of A − LC are in the prespecified locations. It follows from the above that if the pair (A, C) is observable, then in addition to forcing the observation error to converge to zero, we can also control its rate of convergence by appropriately selecting the eigenvalues of the matrix A − LC. We also note that the selection of the closed-loop observer gain matrix L can be approached in exactly the same fashion as the construction of the gain matrix K in the linear state-feedback control law design. We illustrate the above point with a numerical example.

Example 5.12 For the given observable pair

A = [ \begin{matrix}0 & 0 & 1 & 0 \\ 1 & 0 & 2 & 0 \\ 0 & 1 & 3 & 0 \\ 0 & 0 & −21 & 5 \end{matrix} ] , C = [ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ] ,

construct the matrix L ∈ ℝ^{4×2} so that the eigenvalues of A − LC are located at

{−2, −3 + j, −3 − j, −4}.

In our construction of the gain matrix L we use the observer companion form of the pair (A, C) that we computed in Example 5.9. Our goal then is to construct a matrix L so that the characteristic polynomial of the matrix A − LC is

det(sI_{4} − A + LC) = s^{4} + 12s^{3} + 54s^{2} + 108s + 80.

Using the observer form of the pair (A, C), we select L̂ so that

Â − L̂Ĉ = [ \begin{matrix}0 & 0 & 0 & −80 \\ 1 & 0 & 0 & −108 \\ 0 & 1 & 0 & −54 \\ 0 & 0 & 1 & −12 \end{matrix} ] .

We have

L̂ = [ \begin{matrix}1681 & 80 \\ 2270 & 108 \\ 1137 & 54 \\ 251 & 17 \end{matrix} ] ,

and hence

L = T̂^{⊤}L̂ = [ \begin{matrix}1137 & 54 \\ 3955 & 188 \\ 5681 & 270 \\ −23626 & −1117 \end{matrix} ] .

We note that there are other possible gain matrices L that could be used to allocate the eigenvalues of A − LC into desired locations.

We now present an alternative approach to the closed-loop observer design. This approach will then be used to construct a reduced-order observer that we discuss in the subsequent section. We assume that the dynamics of an observer are given by

ż(t) = Dz(t) + Eu(t) + Gy(t),  (5.31)

where z(t) ∈ ℝ^{n}. To begin with the development, suppose that we would like to obtain

z = T x,

where T ∈ ℝ^{n×n} is invertible. Premultiplying ẋ(t) = Ax(t) + Bu(t) by T yields

T ẋ(t) = T Ax(t) + T Bu(t).  (5.32)

Substituting into (5.31) the relation y(t) = Cx(t) gives

ż(t) = Dz(t) + Eu(t) + GCx(t).  (5.33)

Subtracting (5.33) from (5.32) and taking into account that z = T x and ż = T ẋ yields

0 = (T A − DT − GC) x(t) + (T B − E) u(t)

for all t and for arbitrary input u. Hence, we must have

T A − DT = GC
E = T B.

If z ≠ T x, but the two above equations hold, then subtracting (5.32) from (5.33) and taking into account the above relations yields

\frac{d}{dt} (z − T x) = Dz − T Ax + GCx
= Dz − DT x
= D (z − T x) .

If the eigenvalues of D all have negative real parts, then

z − T x → 0 as t → ∞.

Note that if we try T = I_{n}, then we have the previously analyzed case, where G = L, and

E = B,
A − LC = D.

The resulting observer, when T = I_{n}, was called the identity observer by its developer Luenberger [221, 222]. Given the desired eigenvalues, symmetric with respect to the real axis, the equation A − LC = D can be solved for L so that the resulting D has its eigenvalues in the desired locations, if and only if the pair (A, C) is observable. However, it is not true that given an arbitrary D we can solve for L so that A − LC = D even if the pair (A, C) is observable. We use the results of the above discussion to construct an observer of reduced order in the following section.

5.6.2 Reduced-Order Observer

The full-order observer possesses a certain degree of redundancy. It estimates all n state variables even though a linear combination of the states in the form of the output, y = Cx, is already known to us. If the output y is a p-vector with p < n, we would like to use the p outputs to determine p of the x_{i}’s and design an observer of order n − p to estimate the rest of state variables. Specifically, if rank C = p, then we can use the equation y = Cx to solve for p of the x_{i}’s in terms of the y_{i}’s and the remaining n − p state variables x_{k}’s. Then, we construct an observer of order n − p. We present two methods for constructing reduced-order observers of order n − p.

Method 1 for Reduced-Order Observer Construction

To recover all states, we should like to solve, for x, the equations

z = T x
y = Cx,

where now T ∈ ℝ^{(n−p)×n}. This can be done if and only if

rank [ \begin{matrix}T \\ C \end{matrix} ] = n.  (5.34)

Given D ∈ ℝ^{(n−p)×(n−p)}, G ∈ ℝ^{(n−p)×p}, and the triple (A, B, C), we can use the Kronecker product to show that the equation

T A − DT = GC

can be uniquely solved for T if and only if no eigenvalue of D is also an eigenvalue of A. Indeed, the above equation is a special case of the matrix equation (A.54) that we analyzed in Section A.5. Note that when choosing the matrices D and G, we must make sure that the condition (5.34) is satisfied. The structure of the resulting reduced-order observer is shown in Figure 5.10.

Method 2 for Reduced-Order Observer Construction

This method is presented in Luenberger [223, pp. 304–307]. In this method, we first find a full-rank matrix N ∈ ℝ^{n×(n−p)} such that

CN = O.

Then, we construct a state-variable transformation

x̃ = [ \begin{matrix}N & ⊤ \\ C \end{matrix} ] = T̃ x.

266 CHAPTER 5. LINEAR TIME-INVARIANT (LTI) SYSTEMS

Figure 5.10: Architecture of the reduced-order observer designed using method 1. Note that by construction, det T̃ ≠ 0, and therefore T̃ is invertible. Let

x̃ = [ \begin{matrix}w \\ y \end{matrix} ] = [ \begin{matrix}N & ⊤ & x \\ C & x \end{matrix} ] .

The plant model in the new coordinates has the form

\frac{d}{dt} [ \begin{matrix}w \\ y \end{matrix} ] = T̃ AT̃^{−1} [ \begin{matrix}w \\ y \end{matrix} ] + T̃ Bu
= [ \begin{matrix}A_{11} & A_{12} \\ A_{21} & A_{22} \end{matrix} ] [ \begin{matrix}w \\ y \end{matrix} ] + [ \begin{matrix}B & 1 \\ B & 2 \end{matrix} ] u  (5.35)
y = CT̃^{−1}
= [ O I_{p} ] [ \begin{matrix}w \\ y \end{matrix} ] .  (5.36)

We represent (5.35) as

ẇ = A_{11}w + A_{12}y + B_{1}u  (5.37)
ẏ = A_{21}w + A_{22}y + B_{2}u.  (5.38)

We can use the PBH test for observability to show that if the pair (A, C) is observable, then so is the pair (A_{11}, A_{21}). Therefore, we can construct a matrix L_{r} ∈ ℝ^{(n−p)×p} such that the matrix (A_{11} − L_{r}A_{21}) has its eigenvalues in pre-specified locations in the open-left half complex plane. Our next step is to premultiply both sides of (5.38) by L_{r} and subtract the resulting equation from (5.37) to obtain

ẇ − L_{r}ẏ = (A_{11} − L_{r}A_{21})w + (A_{12} − L_{r}A_{22})y + (B_{1} − L_{r}B_{2})u

We need a term (w − L_{r}y) on the right hand-side of the above equation. We add and subtract the term (A_{11} − L_{r}A_{21})L_{r}y and perform simple manipulations to obtain

\frac{d}{dt}(w−L_{r}y) = (A_{11}−L_{r}A_{21})(w−L_{r}y)+(A_{11}L_{r}−L_{r}A_{21}L_{r}+A_{12}−L_{r}A_{22})y+(B_{1}−L_{r}B_{2})u.

(5.39)

Let

v = w − L_{r}y.  (5.40)

Then we represent (5.39) as

v̇ = (A_{11} − L_{r}A_{21})v + (B_{1} − L_{r}B_{2})u + (A_{11}L_{r} − L_{r}A_{21}L_{r} + A_{12} − L_{r}A_{22})y. (5.41)

Let

\begin{matrix}D & = & A_{11} − L_{r}A_{21} \\ E & = & B_{1} − L_{r}B_{2} \\ G & = & A_{11}L_{r} − L_{r}A_{21}L_{r} + A_{12} − L_{r}A_{22}. \end{matrix} \}  (5.42)

We construct a trivial observer of v,

ż = Dz + Eu + Gy.  (5.43)

Note that

\frac{d}{dt}(v − z) = (A_{11} − L_{r}A_{21})(v − z),

which means that we fully control the dynamics of the observation error (v − z) because we are free to select the gain matrix L_{r} to place the eigenvalues of (A_{11} − L_{r}A_{21}) at any pre-specified locations. Thus

z(t) → v(t) = w − L_{r}y.

We use z and y to obtain the estimate of x, denoted x̃, that is,

x̃ = T̃^{−1} [ ŵ ; y ] = T̃^{−1} [ \begin{matrix}z + L_{r}y \\ y \end{matrix} ] .  (5.44)

The structure of the above observer has the format somewhat similar to the one depicted in Figure 5.10, which is the structure of the reduced-order observer designed using Method 1. However, the observers’ matrices are computed quite differently in both methods.

The last step in the control design process involving an observer is to combine the control law, constructed in the previous section, with the observer.

Figure 5.11: Closed-loop system driven by the combined controller-observer compensator.

5.7 Combined Controller-Observer Compensator

We start by combining a control law with a full-order observer. We write the equations that govern the behavior of the dynamical system consisting of the plant model and the full-order observer. The resulting system’s order is 2n, and the equations that describe the system are

\begin{matrix}[ \begin{matrix}ẋ(t) \\ ˙x̃(t) \end{matrix} ] & = & [ \begin{matrix}A & O \\ LC & A − LC \end{matrix} ] [ \begin{matrix}x(t) \\ x̃(t) \end{matrix} ] + [ \begin{matrix}B \\ B \end{matrix} ] u(t) \\ y(t) & = & [ C O ] [ \begin{matrix}x(t) \\ x̃(t) \end{matrix} ] \end{matrix} \}  (5.45)

Suppose that we now apply the control law

u(t) = −Kx̃(t) + v(t)  (5.46)

instead of the actual state-feedback control law. In the above, the vector v denotes an external input signal. A schematic of a closed-loop system driven by the combined controllerobserver compensator is shown in Figure 5.11. A model of the closed-loop system is obtained by combining equations (5.45) and (5.46),

[ \begin{matrix}ẋ(t) \\ ˙x̃(t) \end{matrix} ] = [ \begin{matrix}A & −BK \\ LC & A − LC − BK \end{matrix} ] [ \begin{matrix}x(t) \\ x̃(t) \end{matrix} ] + [ \begin{matrix}B \\ B \end{matrix} ] v(t)
y(t) = [ C O ] [ \begin{matrix}x(t) \\ x̃(t) \end{matrix} ] .

To analyze the above closed-loop system, it is convenient to perform a change of state variables using the following transformation

[ x − x̃ ; x̃ ] = [ \begin{matrix}I_{n} & −I_{n} \\ O & I_{n} \end{matrix} ] [ x ; x̃ ] .

Note that

[ \begin{matrix}I_{n} & −I_{n} \\ O & I_{n} \end{matrix} ]^{−1} = [ \begin{matrix}I_{n} & I_{n} \\ O & I_{n} \end{matrix} ] .

In the new coordinates the equations describing the closed-loop system are

[ \begin{matrix}ẋ(t) − ˙ x̃(t) \\ ˙x̃(t) \end{matrix} ] = [ \begin{matrix}A − LC & O \\ LC & A − BK \end{matrix} ] [ \begin{matrix}x(t) − x̃(t) \\ x̃(t) \end{matrix} ] + [ \begin{matrix}O \\ B \end{matrix} ] v(t)
y(t) = [ C C ] [ \begin{matrix}x(t) − x̃(t) \\ x̃(t) \end{matrix} ] .

Note that in the above system, the subsystem corresponding to the error component e(t) = x(t) − x̃(t) is uncontrollable. Furthermore, the 2n poles of the closed-loop system are equal to the individual eigenvalues of both A − LC and A − BK, which means that the design of the observer is separated from the construction of the control law. This is what is known as the separation principle. The closed-loop transfer function relating Y (s) and V (s) is

Y (s) = [ C C ] [ \begin{matrix}sI_{n} − A + LC & O \\ −LC & sI_{n} − A + BK \end{matrix} ]^{−1} [ \begin{matrix}O \\ B \end{matrix} ] V (s)
= C (sI_{n} − A + BK)^{−1} BV (s).

The above expression is identical to that for the closed-loop system if we applied the statefeedback control law u(t) = −Kx(t) + v(t). Thus, the combined controller-observer compensator yields the same closed-loop transfer function as the actual state-feedback control law.

We now briefly discuss the issue of the observer pole selection. Franklin, Powell, and Emami-Naeini [107, page 552] recommend that the real parts of the observer poles, that is, the real parts of the eigenvalues of the matrix A − LC, be a factor of 2 to 6 times deeper in the open left-half plane than the real parts of the controller poles which are the eigenvalues of the matrix A − BK. Such a choice ensures a faster decay of the observer error e(t) = x̃(t) − x(t) compared with the desired controller dynamics. This in turn causes the controller poles to dominate the closed-loop system response. Because the observer poles represent a measure of the speed with which the error e(t) = x̃(t) − x(t) decays to zero, one would tend to assign observer poles deep in the left-hand plane. However, fast decay requires large gains which may lead to saturation of some signals and unpredictable nonlinear effects. If the observer poles were slower than the controller poles, the closed-loop system response would be dominated by the observer, which is undesirable. As it is usual in engineering practice, the term compromise could be used to describe the process of constructing the final compensator structure.

We next analyze the combined controller-observer compensator that uses the reducedorder observer. The closed-loop system driven by this type of compensator also enjoys the separation property as in the case of the full-order observer. To prove the above statement, we begin with writing the equations describing the closed-loop system,

\begin{matrix}ẋ(t) & = & Ax(t) + Bu(t) \\ ż(t) & = & Dz(t) + T Bu(t) + GCx(t) \\ y(t) & = & Cx(t) \\ u(t) & = & −Kx̃(t) + v(t), \end{matrix} \}

where

x̃ = [ \begin{matrix}T \\ C \end{matrix} ]^{−1} [ \begin{matrix}z \\ y \end{matrix} ] .

Let

[ \begin{matrix}T \\ C \end{matrix} ]^{−1} = [ M N ] ,

where M ∈ ℝ^{n×(n−p)} and N ∈ ℝ^{n×p}. Note that

MT + NC = I_{n}.

Hence, the control input can be represented as

u(t) = −K [ M N ] [ \begin{matrix}z(t) \\ y(t) \end{matrix} ] + v(t)
= −KMz(t) − KNy(t) + v(t).

Using the above equations, and noting that y(t) = Cx(t), we represent equations that 5.7. COMBINED CONTROLLER-OBSERVER COMPENSATOR 271 describe the closed-loop system as

[ \begin{matrix}ẋ(t) \\ ż(t) \end{matrix} ] = [ \begin{matrix}A & O \\ GC & D \end{matrix} ] [ \begin{matrix}x(t) \\ z(t) \end{matrix} ] + [ \begin{matrix}B \\ T & B \end{matrix} ] (−KMz(t) − KNy(t) + v(t))
= [ \begin{matrix}A − BKNC & −BKM \\ GC − T BKNC & D − T BKM \end{matrix} ] [ \begin{matrix}x(t) \\ z(t) \end{matrix} ] + [ \begin{matrix}B \\ T & B \end{matrix} ] v(t)
y(t) = [ C O ] [ \begin{matrix}x(t) \\ z(t) \end{matrix} ] .

To proceed further, we perform a transformation of the state variable of the above system,

[ \begin{matrix}x \\ z − T x \end{matrix} ] = [ \begin{matrix}I_{n} & O \\ −T & I_{n−p} \end{matrix} ] [ \begin{matrix}x \\ z \end{matrix} ] .

Note that

[ \begin{matrix}I_{n} & O \\ −T & I_{n−p} \end{matrix} ]^{−1} = [ \begin{matrix}I_{n} & O \\ T & I_{n−p} \end{matrix} ] .

In the new coordinates the equations describing the closed-loop system become

[ \begin{matrix}ẋ(t) \\ ż(t) − T ẋ(t) \end{matrix} ] = [ \begin{matrix}A − BKNC − BKMT & −BKM \\ −T A + GC + DT & D \end{matrix} ] [ \begin{matrix}x(t) \\ z(t) − T x(t) \end{matrix} ]

+ [ \begin{matrix}B \\ O \end{matrix} ] v(t)

y(t) = [ C O ] [ \begin{matrix}x(t) \\ z(t) − T x(t) \end{matrix} ] .

Taking into account the equations

T A − DT − GC = 0 and MT = I_{n} − NC,

we obtain

[ \begin{matrix}ẋ(t) \\ ż(t) − T ẋ(t) \end{matrix} ] = [ \begin{matrix}A − BK & −BKM \\ O & D \end{matrix} ] [ \begin{matrix}x(t) \\ z(t) − T x(t) \end{matrix} ] + [ \begin{matrix}B \\ O \end{matrix} ] v(t)
y(t) = [ C O ] [ \begin{matrix}x(t) \\ z(t) − T x(t) \end{matrix} ] .

It is now apparent that the separation principle also holds if we use the reduced-order observer in constructing the compensator, that is, the selection of the gain matrix K that assigns

Figure 5.12: Schematic of an armature-controlled DC motor system of Example 5.13.

closed-loop poles is independent, separate, from the selection of the observer gain matrix L that assigns the observer poles. Furthermore, the transfer function matrix relating Y (s) and V (s) in the closed-loop system with the reduced-order observer is exactly the same as the one that would be obtained using the actual state-feedback control law u(t) = −Kx(t) + v(t).

Example 5.13 Consider a schematic of an armature-controlled DC motor system, shown in Figure 5.12, where the system parameters are: R_{a} = 5 Ω, L_{a} = 200 mH, K_{b} = 0.1 V/rad/sec, K_{i} = 0.1 Nm/A, the gear ratio N_{1}/N_{2} = 1/50. The armature inertia is I_{armature} = 2 × 10^{−3} kg·m^{2}. The load of 10 kg is located at an effective radius of 0.2 m. The gear inertia and friction are negligible.

We first construct a state-space model of the DC motor system. We start with writing the equation relating torque to angular acceleration:

T_{m} = I_{eq}θ̈,  (5.47)

where

I_{eq} = I_{armature} + (N_{1}/N_{2})^{2} I_{l}
= 2 × 10^{−3} kg·m^{2} + (1/50)^{2} × 10 × (0.2)^{2} kg·m^{2}
= 2.16 × 10^{−3} kg·m^{2}.

Kirchhoff’s voltage law applied to the armature circuit gives

R_{a}i_{a} + L_{a} \frac{di_{a}}{dt} + e_{b} = e_{a},

where e_{b} = K_{b} \frac{dθ}{dt}. The equation for the developed torque is

T_{m} = K_{i}i_{a}.  (5.48)

Combining (5.47) and (5.48) gives

θ̈ = \frac{K_{i}}{I_{eq}} i_{a}.

Let x_{1} = i_{a}, x_{2} = θ, x_{3} = θ̇ = ω, u = e_{a}, and y = θ_{l}. Taking into account the definition of the state variables, we represent the above modeling equations in state-space format:

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ] = [ \begin{matrix}−\frac{R_{a}}{L_{a}} & 0 & −\frac{K_{b}}{L_{a}} \\ 0 & 0 & 1 \\ \frac{K_{i}}{I_{eq}} & 0 & 0 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \\ x_{3} \end{matrix} ] + [ \frac{1}{L_{a}} ; 0 ; 0 ] u

y = [ 0 \frac{N_{1}}{N_{2}} 0 ] [ \begin{matrix}x_{1} \\ x_{2} \\ x_{3} \end{matrix} ] .

Substituting the given parameter values, we get

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ] = [ \begin{matrix}−25 & 0 & −0.5 \\ 0 & 0 & 1 \\ 46.296 & 0 & 0 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \\ x_{3} \end{matrix} ] + [ \begin{matrix}5 \\ 0 \\ 0 \end{matrix} ] u = Ax + bu

y = [ 0 0.02 0 ] [ \begin{matrix}x_{1} \\ x_{2} \\ x_{3} \end{matrix} ] = cx.

Our next step is to design a state-feedback controller u = −kx+v, such that the eigenvalues of A − bk are

−1 + j, −1 − j, −10.

To allocate the poles into the desired locations, we first transform the system model into the controller companion form. We first form the controllability matrix

U = [ b Ab A^{2}b ] = [ \begin{matrix}5 & −125 & 3009.3 \\ 0 & 0 & 231.5 \\ 0 & 231.5 & −5787 \end{matrix} ] .

274 CHAPTER 5. LINEAR TIME-INVARIANT (LTI) SYSTEMS The system is controllable. The last row of the inverse of the controllability matrix is

q_{1} = [ 0 0.0043 0 ] ,

and hence the transformation matrix we are seeking has the form

T = [ \begin{matrix}q & 1 \\ q & 1 & A \\ q & 1 & A & 2 \end{matrix} ] = [ \begin{matrix}0 & 0.0043 & 0 \\ 0 & 0 & 0.0043 \\ 0.2 & 0 & 0 \end{matrix} ] .

The system’s matrices in the new coordinates are

Ã = T AT^{−1} = [ \begin{matrix}0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & −23.148 & −25 \end{matrix} ] ,

b̃ = T b = [ \begin{matrix}0 \\ 0 \\ 1 \end{matrix} ] ,

and

c̃ = cT^{−1} = [ 4.63 0 0 ] .

The desired closed-loop characteristic polynomial α_{c}(s) is

α_{s}(s) = (s + 1 − j)(s + 1 + j)(s + 10) = s^{3} + 12s^{2} + 22s + 20.

We now find k̃ so that

Ã − b̃k̃ = [ \begin{matrix}0 & 1 & 0 \\ 0 & 0 & 1 \\ −20 & −22 & −12 \end{matrix} ] .

We have

k̃ = [ 20 −1.148 −13 ] ,

and

k = k̃T = [ −2.6 0.0864 −0.005 ] .

Hence,

A − bk = [ \begin{matrix}−12 & −0.432 & −0.4752 \\ 0 & 0 & 1 \\ 46.294 & 0 & 0 \end{matrix} ] .

Figure 5.13: Step response of the closed-loop DC motor system with state feedback.

The transfer function of the closed-loop system is

\frac{L(y(t))}{L(v(t))} = \frac{Y (s)}{V (s)} = c (sI − A + bk)^{−1} b = \frac{4.6296}{s^{3} + 12s^{2} + 22s + 20}.

The unit-step response of the closed-loop system is shown in Figure 5.13.

Next, we design a full-order observer placing its poles at {−4, −5 + 2j, −5 − 2j}, and then synthesize the combined controller-observer compensator. To compute the observer gain vector l, we first transform the pair (A, c) into the observer companion form. This is equivalent to transforming the pair (A^{⊤}, c^{⊤}) into the controller companion form. The transformation matrix that brings the pair (A, c) into its observer companion form is

Q̂ = T̂^{⊤} = [ \begin{matrix}1.1 & −27 & 650 \\ 0 & 0 & 50 \\ 0 & 50 & −1250 \end{matrix} ] .

The matrices A and c in the new coordinates have the form:

Â = Q̂^{−1}AQ̂ = [ \begin{matrix}0 & 0 & 0 \\ 1 & 0 & −23.1481 \\ 0 & 1 & −25 \end{matrix} ] and ĉ = cQ̂ = [ 0 0 1 ] .

The desired characteristic polynomial of the observer is

det (I_{3} − A + lc) = (s + 4)(s + 5 − 2j)(s + 5 + 2j) = s^{3} + 14s^{2} + 69s + 116.

We now find l̂ so that

Â − l̂ĉ = [ \begin{matrix}0 & 0 & −116 \\ 1 & 0 & −69 \\ 0 & 1 & −14 \end{matrix} ] .

We have

l̂ = [ \begin{matrix}116 \\ 45.8519 \\ −11 \end{matrix} ] and therefore l = Q̂l̂ = [ \begin{matrix}−8263 \\ −550 \\ 16043 \end{matrix} ] .

The dynamics of the full-order observer are described by

˙x̃ = (A − lc)x̃ + bu + ly,

that is,

˙x̃ = [ \begin{matrix}−25 & 165.2544 & −0.5 \\ 0 & 11 & 1 \\ 46.2963 & −320.8519 & 0 \end{matrix} ] x̃ + [ \begin{matrix}5 \\ 0 \\ 0 \end{matrix} ] u + [ \begin{matrix}−8263 \\ −550 \\ 16043 \end{matrix} ] y.

We connect the full-order observer to the DC motor system, thus obtaining a closedloop system with the combined controller-full-order-observer compensator in the loop as in Figure 5.11. In Figure 5.14, we show a plot of x_{1} and its estimate for the closed-loop system with the observer in the loop, where v = 0, x(0) = [ 1 0.2 −0.1 ]^{⊤}, and x̃(0) = 0.

We point out that the step response of the closed-loop system with the combined controllerfull-order-observer compensator in the loop as in Figure 5.11, with zero initial conditions on the plant and the full-order observer, is identical to that shown in Figure 5.13.

Our next part of this example is to design the reduced-order observer using the design method 1. We choose the reduced-order observer’s poles at {−5+2j, −5−2j}. The reducedorder observer dynamics are given by

ż = Dz + Eu + Gy.

Let

G = [ \begin{matrix}100 \\ 100 \end{matrix} ] .

The observer’s poles are supposed to be −5+2j and −5−2j. Thus, the desired characteristic polynomial of D should be

det(sI_{2} − D) = s^{2} + 10s + 29.

Figure 5.14: A plot of x_{1} and its estimate versus time for the closed-loop system with the full-order observer in the loop.

Let

D = [ \begin{matrix}0 & 1 \\ −29 & −10 \end{matrix} ] .

We now solve the matrix equation,

−DT + T A = Gc,

for T. We first convert the above equation in the format similar to that given by (A.55) in Section A.5 and then solve the resulting equation. Employing the following MATLAB’s commands,

Gc=G*c;
t=(-kron(eye(3),D)+kron(A',eye(2)))\Gc(:);
T=[t(1:2) t(3:4) t(5:6)]

we obtain

T = [ \begin{matrix}−0.5493 & 0.7586 & −0.2507 \\ 2.1271 & −2.0000 & 1.0332 \end{matrix} ] .

Note that

det [ \begin{matrix}T \\ c \end{matrix} ] = 6.8691 × 10^{−4} ≠ 0,

and

[ \begin{matrix}T \\ c \end{matrix} ]^{−1} = [ \begin{matrix}−30.0840 & −7.2980 & 411.3200 \\ 0 & 0 & 50 \\ 61.9338 & 15.9921 & −750 \end{matrix} ] .

Figure 5.15: A plot of x_{1} and its estimate versus time for the closed-loop system with the reduced-order observer, designed using method 1, in the loop.

We also have

E = T b = [ \begin{matrix}−2.7463 \\ 10.6357 \end{matrix} ] .

Hence, the dynamics of the reduced-order observer are described by

ż = [ \begin{matrix}0 & 1 \\ −29 & −10 \end{matrix} ] z + [ \begin{matrix}−2.7463 \\ 10.6357 \end{matrix} ] u + [ \begin{matrix}100 \\ 100 \end{matrix} ] y.

An estimate of the entire state can be obtained from

x̂ = [ \begin{matrix}T \\ c \end{matrix} ]^{−1} [ \begin{matrix}z \\ y \end{matrix} ] = [ \begin{matrix}−30.0840 & −7.2980 & 411.3200 \\ 0 & 0 & 50 \\ 61.9338 & 15.9921 & −750 \end{matrix} ] [ \begin{matrix}z \\ y \end{matrix} ] .

A plot of x_{1} and its estimate versus time for the closed-loop system with the combined controller-reduced-order-observer compensator is depicted in Figure 5.15, where v = 0, x(0) = [ 1 0.2 −0.1 ]^{⊤}, and z(0) = 0.

Our last part of this example is to construct the reduced-order observer employing the design method 2. This reduced-order observer matrices are computed using (5.42). The state estimate is obtained from (5.44). A plot of x_{1} and its estimate versus time for the closedloop system with the combined controller-reduced-order-observer compensator is depicted in Figure 5.16, where, as before, v = 0, x(0) = [ 1 0.2 −0.1 ]^{⊤}, and z(0) = 0.

Of course, for zero initial conditions, the step responses of the closed-loop systems with the full-order, as well as with the reduced-order observers in the loop, are the same as the Figure 5.16: A plot of x_{1} and its estimate versus time for the closed-loop system with the reduced-order observer, designed using method 2, in the loop.

step response of the closed-loop system with the state-feedback controller u = −kx+v. This follows from our analyses of the combined controller-observer compensators.

5.8 Notes

For further reading on the subject of linear system theory, see Brockett [43], Wolovich [352], Barnett [23], Kailath [168], Skelton [300], DeCarlo [75], Rugh [288], Antsaklis and Michel [12], or Chen [55]. For more information on canonical forms, see the papers by Popov [277] and Denham [79].

One may argue that Maxwell’s [231] analysis of Watt’s governor constitutes the first work in linear system theory: the physical object, its linear model and a method of analysis. For example, MacFarlane [226, page 251] states that “Maxwell’s fundamentally important contribution lay in recognizing that the behavior of an automatic feedback control system in the vicinity of an equilibrium condition could be approximated by a linear differential equation, and hence the stability of the control system could be discussed in terms of the location of the roots of an associated algebraic equation.” On the other hand, L. Markus in [97, page xxvii] makes the following statement: “But certainly Watt’s application of the centrifugal governor for regulating steam engines (1775) can be taken as the start of modern control engineering.”

In Kalman’s [169] well-known 1963 paper, there is a theorem which states that the state space, X, of a linear system model may be decomposed as a direct sum

X = X_{A} ⊕ X_{B} ⊕ X_{C} ⊕ X_{D},

where

X_{A} = states that are reachable but not observable
X_{B} = states that are reachable and observable
X_{C} = states that are neither reachable nor observable
X_{D} = states that are not reachable but observable

This decomposition became, to a certain extent, an icon of linear systems. For example, the logo on the covers of the old issues of the International Journal of Systems Science was the above decomposition. Many authors included Kalman’s original decomposition algorithm in their textbooks on linear systems. Yet almost twenty years after Kalman’s publication of the decomposition theorem, Boley [37] generated an example which revealed some shortcomings of Kalman’s original algorithm. In his reply, Kalman [170] revised his decomposition algorithm and fixed his original proof of the decomposition theorem.

5.9 Exercises

Exercise 5.1 (adapted from [23, page 94]) Consider the system, shown in Figure 5.17, which is composed of two masses moving on a smooth, that is, frictionless, horizontal plane. The springs are linear, that is, they obey Hooke’s law,

force = k × extension,

where k is a constant called the force constant of the spring. In addition to the springs, the masses are joined by linear dampers. In a linear damper, the force is proportional to the velocity of one end of the damper relative to the other end. The proportionality constant is called the damping coefficient. In this exercise, damping coefficients are b_{1} and b_{2}, respectively. A force u is applied to the right end mass. Obtain the system’s state space model in the form ẋ = Ax+bu. Determine under what condition, in terms of k_{1}, the system model is controllable for m_{1} = m_{2} = 1 kg, b_{1} = b_{2} = 1 Ns/m, and k_{2} = 1/4 N/m.

Exercise 5.2 In the circuit shown in Figure 5.18, take the state variables to be the voltage across the capacitor and the current through the inductor. The source voltage v_{s} is the

Figure 5.17: A mechanical system for Exercise 5.1.

control variable and the current i is the system output. Obtain the state-space model of the this circuit. Obtain the condition, in terms of R_{1}, R_{2}, C, and L, under which the model is uncontrollable and unobservable. Find the transfer function of the circuit model.

Exercise 5.3 In the circuit shown in Figure 5.19, take the state variables to be the voltage across the capacitor C_{1}, the current through the inductor L_{1}, and the current through the inductor L_{2}. The source current i_{s} is the control variable and the current through the inductor L_{2} is the system output. Obtain the state space model of this circuit. Analyze the circuit model controllability and observability properties. Find the transfer function of the circuit model.

Exercise 5.4 For the system

ẋ = x + u,

find the set of points that can be steered to the origin x = 0 by an admissible control |u| ≤ 1.

Exercise 5.5 (a) Find e^{At} for

A = [ \begin{matrix}0 & 1 \\ −ab & −a − b \end{matrix} ] .

(b) Given a dynamical system modeled by the differential equation

ẍ + 2ẋ + x = ṙ + r,

282 CHAPTER 5. LINEAR TIME-INVARIANT (LTI) SYSTEMS Figure 5.18: Electric circuit for Exercise 5.2. Figure 5.19: Electric circuit for Exercise 5.3. where r = r(t) = sin t, and the initial conditions are x(0) = 1 and ẋ(0) = 0. Choose the state variables x_{1} = x and x_{2} = ẋ_{1}. Represent the differential equation in state-space format. Solve for x(t) for t ≥ 0.

In both cases, verify your calculations using MATLAB’s Symbolic Toolbox.

Exercise 5.6 Find sufficient conditions, in terms of a, b, c, and d, for reachability of the following pairs:

(a)

A = [ \begin{matrix}−1 & 1 & 0 & 0 & 0 \\ 0 & −1 & 1 & 0 & 0 \\ 0 & 0 & −1 & 0 & 0 \\ 0 & 0 & 0 & −2 & 1 \\ 0 & 0 & 0 & 0 & −2 \end{matrix} ] , B = [ \begin{matrix}0 & 0 \\ 0 & 0 \\ a & b \\ 0 & 0 \\ c & d \end{matrix} ];

(b)

A = [ \begin{matrix}−1 & 1 & 0 & 0 & 0 \\ 0 & −1 & 1 & 0 & 0 \\ 0 & 0 & −1 & 0 & 0 \\ 0 & 0 & 0 & −1 & 1 \\ 0 & 0 & 0 & 0 & −1 \end{matrix} ] , B = [ \begin{matrix}0 & 0 \\ 0 & 0 \\ a & b \\ 0 & 0 \\ c & d \end{matrix} ] .

Exercise 5.7 A system composed of two subsystems connected in cascade is depicted in the block diagram shown in Figure 5.20. Take the state variables of the system to be: x_{1} = y_{1} − u_{1} and x_{2} = y_{2}. Assume that the system input is u = u_{1} and the system output is y = y_{2}. Obtain the state space model of the interconnected system. When is the system uncontrollable? Find the transfer function of the uncontrollable system.

Exercise 5.8 Derive a state-space description of a system represented by the block diagram given in Figure 5.21. Construct a state-variable transformation z = T x such that the matrix T AT^{−1} has a diagonal form. Find the eigenvalue of T AT^{−1} corresponding to the noncontrollable state and the eigenvalue corresponding to the nonobservable state. Find the transfer function of the system.

284 CHAPTER 5. LINEAR TIME-INVARIANT (LTI) SYSTEMS Figure 5.20: A block diagram for Exercise 5.7. Figure 5.21: A block diagram for Exercise 5.8.

Exercise 5.9 Consider a linear time-invariant (LTI) system model,

ẋ(t) = [ \begin{matrix}−1 & 1 & 0 & 0 & 0 & 0 & 0 \\ 0 & −1 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & −1 & 0 & 0 & 0 & 0 \\ 0 & 0 & −1 & −1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 & 0 \end{matrix} ] x(t) + [ \begin{matrix}0 \\ 1 \\ −1 \\ 1 \\ 1 \\ 0 \\ 2 \end{matrix} ] u(t)

y(t) = [ \begin{matrix}1 & 0 & 2 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 2 & 3 & 0 & 0 \end{matrix} ] x(t).

(i) Use the PBH test to determine if the system is controllable;

(ii) Use the PBH test to determine if the system is observable.

Exercise 5.10 Examples 1.7 and 5.6 are concerned with constructing realizations of the same transfer function matrix. Apply Theorem 5.4 to extract a minimal realization from the realization given in Example 1.7. You can use MATLAB’s function obsvf to efficiently solve this part of the problem. Then compute an equivalent transformation relating the obtained minimal realization with the minimal realization given in Example 5.6 as shown in (5.26).

Exercise 5.11 Examples 1.8 and 5.6 are concerned with constructing realizations of the same transfer function matrix. Apply a dual version of Theorem 5.4 to extract a minimal realization from the realization given in Example 1.8. You can use MATLAB’s function ctrbf to efficiently solve this part of the problem. Then compute an equivalent transformation relating the obtained minimal realization with the minimal realization given in Example 5.6 as shown in (5.26). Finally, construct an equivalent transformation relating the obtained minimal realization in this Exercise with the minimal realization given in Exercise 5.10.

Exercise 5.12 (adapted from Barnett [23, page 95]) A simple representation of a vehicle suspension system is shown in Figure 5.22. Assume that the forces produced by the springs and dampers act at the end points P and Q and x_{1} and x_{2} are the displacements of these points from equilibrium. The force exerted by a damper is proportional to the velocity of one end of the damper relative to the other end and the springs obey Hooke’s law. Assume that the mass of the platform can be neglected so that the spring motions can be regarded as independent. The control force 4u is applied one quarter the way along from one end of

Figure 5.22: A simple representation of a vehicle suspension system of Exercise 5.12.

the platform. Derive the two-dimensional state space model of this simple representation of a vehicle suspension system. Next, suppose that the ends of the platform are each given an initial displacement of 10 units, find a control law that returns the platform to equilibrium at t = 1 sec.

Exercise 5.13 For the dynamical system model, ẋ = Ax + bu, where

A = [ \begin{matrix}0 & 1 \\ 1 & 0 \end{matrix} ] and b = [ \begin{matrix}0 \\ 1 \end{matrix} ] ,

design a state-feedback controller that stabilizes the system about the operating state corresponding to the constant input u = 2. The desired closed-loop poles are −2, −3.

Exercise 5.14 Consider the vehicle model given in Subsection 2.4.2. Let the input signal be u = δ_{f}—the front wheel steer angle, and δ_{r} = 0. Compute the transfer function C(s)/U(s). Plot the vehicle response Y versus X for X changing from 0 m to 100 m, when the input δ_{f}, in the vehicle coordinates, has the form δ_{f}(t) = 0.0872 h(t) rad, where h(t) is the unit step function.

Suppose now that the plot of δ_{f} versus time, in the car coordinates, has the form shown in Figure 5.23. Note that the values of δ_{f} are given in deg. The value of δ_{f} changes form −2.5 deg to 2.5 deg. In your simulations you will need to convert deg to rad. Plot the vehicle response Y versus X for X changing from 0 m to 100 m.

Figure 5.23: Time history of the front steer angle in the car coordinates of Exercise 5.14.

Construct the state-feedback controller u = −kx + r such that eig(A − bk) = {−2, −3 + j, −3 − j, −4}. Plot u and r versus time, and Y versus X, when r = 0.0872 h(t) rad.

Exercise 5.15 Show that the controllability property is invariant under the following set of transformations:

(i) nonsingular coordinate transformation in the state-space;

(ii) nonsingular transformation of the inputs;

(iii) state feedback;

(iv) output feedback.

Exercise 5.16 Given the following model of a dynamical system

ẋ(t) = [ \begin{matrix}0 & 2 \\ 1 & 0 \end{matrix} ] x(t) + [ \begin{matrix}1 \\ 0 \end{matrix} ] u(t)
y(t) = [ 0 1 ] x(t).

(a) Design a full-order asymptotic observer for the above system with the observer poles located at −2 and −3. Write the equations of the observer dynamics. Note that the pair (A, c) is already in the observer form, which should facilitate your observer design.

(b) Denote the observer state vector by z. Let u = − [ 2 3 ] z + v. Find the transfer function of the closed-loop system Y (s)/V (s).

Exercise 5.17 Let

ẋ = Ax + bu = [ \begin{matrix}−1 & 1 \\ 0 & 2 \end{matrix} ] x + [ \begin{matrix}1 \\ 1 \end{matrix} ] u,
y = cx = [ 1 0 ] x.

(a) Find the state transformation that transforms the pair (A, c) into the observer form.

(b) Find the representation of the dynamical system in the new coordinates.

Exercise 5.18 For the following model of a dynamical system,

ẋ = [ \begin{matrix}0 & 1 \\ 2 & 0 \end{matrix} ] x + [ \begin{matrix}1 \\ 0 \end{matrix} ] u
y = [ 1 0 ] x + 2u,

(a) design a full-order observer with its poles located at −3 and −4. Write the equations of the observer dynamics;

(b) denote the observer state by z. Let u = − [ 3 2 ] z + r. Find the transfer function, Y (s)/R(s), of the closed-loop system.

Exercise 5.19 Given a dynamical system modeled by:

ẋ = Ax + bu
y = cx,

where

A = [ \begin{matrix}1 & −3 \\ 4 & 2 \end{matrix} ] , b = [ \begin{matrix}1 \\ 1 \end{matrix} ] , c = [ 2 3 ] .

(a) Design a reduced-order observer

ż = dz + eu + gy,

where d = −5 and g = 1.

(b) Write an expression for the estimated states.

5.9. EXERCISES 289 Exercise 5.20 For the plant

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}1 & −1 \\ 1 & −2 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \end{matrix} ] + [ \begin{matrix}2 \\ 1 \end{matrix} ] u
y = [ 0 1 ] [ \begin{matrix}x_{1} \\ x_{2} \end{matrix} ] ,

(a) Design the reduced-order observer for G = [1] and D = [−2].

(b) Write the equations of the estimated state in terms of the plant’s output and the observer state.

Exercise 5.21 Consider a dynamical system that consists of a cart with an inverted pendulum (point mass on a massless shaft) attached to it as depicted in Figure 5.24.

Figure 5.24: An inverted pendulum moving on a cart of Exercise 5.21.

(i) Obtain a nonlinear state-space of the system using the state variables

x_{1} = x, x_{2} = ẋ, x_{3} = θ, x_{4} = θ̇.

Designate x_{1} as the system output. (ii) Linearize the model about the operating point x = 0, u = 0.

(iii) Design a state feedback control law u = −kx such that the closed-loop poles are

located at

−1, −2, −1 ± j

(iv) Write a simple MATLAB script that animates the motion of the linearized closed-loop model of the inverted pendulum on the cart along with animation of the displacement x and the angle θ data. The basic handle graphics of MATLAB can be used to write the script. A script contained in Example 4 in the book [279, page 208] can be helpful here. Alternatively, you can employ SIMULINK.

Exercise 5.22 For the nonlinear model of the one-link robot manipulator of Subsection 2.4.5:

(a) Find the constant value of u = u_{e} and x_{3e} so that the state x_{e} = [ 30π/180 0 x_{3e} ]^{⊤} becomes the equilibrium state, that is, the obtained x_{e} and u_{e} satisfy the equation

0 = f(x_{e}, u_{e}).

(b) Linearize the nonlinear model, ẋ = f(x, u), about the operating point [ x_{e}^{⊤} u_{e} ]^{⊤} to

obtain

\frac{d}{dt}Δx = AΔx + bΔu.

where,

Δx = x − x_{e}, Δu = u − u_{e},

and

A = \frac{∂f}{∂x}(x_{e}, u_{e}) and b = \frac{∂f}{∂u} (x_{e}, u_{e}).

(c) Design a state feedback control law Δu = −k Δx such that the closed-loop poles are

located at

−2, −2 ± j

(d) Implement the obtained control law on the nonlinear model, ẋ = f(x, u). Write a MATLAB script that animates the dynamical behavior of the nonlinear closed-loop system along with animation of the angle θ_{p} and the input u data.

(e) Design a full-order observer for the linearized model. The observer’s poles should be

located at

−5, −5 ± j

(f) Implement the combined controller-observer compensator on the nonlinear model and write a MATLAB script that animates dynamic behavior of the nonlinear closed-loop system along with animation of the angle θ_{p} and the input u data.

Exercise 5.23 For the nonlinear model of a double-link robot manipulator in Subsection 2.4.6, use the linearized model of this robot obtained in Exercise 4.5 to

(a) design a state-feedback controller using the linearized model. Write a MATLAB script that animates the behavior of the closed-loop system comprised of the state-feedback controller driving the nonlinear model of the robot for different initial conditions;

(b) use the linearized model to design a Luenberger observer. Implement and perform animation of the closed-loop system comprised of the combined controller-observer compensator driving the nonlinear model of the robot.

292 CHAPTER 5. LINEAR TIME-INVARIANT (LTI) SYSTEMS

A Glimpse at Linear Time-Varying (LTV) Systems

”As we shall see, the concept of time has no meaning before the beginning of the universe. This was first pointed out by St. Augustine. When asked: What did God do before he created the universe? Augustine didn’t reply: He was preparing Hell for people who asked such questions. Instead, he said that time was a property of the universe that God created, and that time did not exist before the beginning of the universe.” Stephen W. Hawking [133, page 8]

6.1 Motivating Example

Of considerable importance in many applications are linear systems in which the elements of A and B are continuous functions of time. We present a linear time-varying (LTV) model of a battery as a motivation for our discussion on linear time-varying systems. Our derivation follows that of Chiasson and Vairamohan [60, 61].

The simplest battery model is shown in Figure 6.1, which is called a practical voltage source. This model consists of an ideal voltage source that produces voltage V_{oc}. We refer to V_{oc} as the open-circuit voltage. Thus the open-circuit voltage is obtained by measuring the voltage across the open-circuit, that is, when I_{b} = 0. The internal resistance of the practical voltage source is denoted R_{int}. The battery terminal voltage is labeled as V_{b}. This model, however, does not take into account the fact that the internal resistance is different under discharge and charge conditions. To account for this, the circuit of Figure 6.1 is modified by introducing two different internal resistances; R_{c}—the charging resistance and R_{d}—the

Figure 6.1: Practical voltage source as a simple battery model.

Figure 6.2: Battery model accounting for the different charging, R_{c}, and discharging, R_{d}, resistances.

discharging resistance. These resistances are connected to the diodes to model charging and discharging modes of the battery operation. The modified circuit is shown in Figure 6.2. This model, on the other hand, does not take into account the diffusion of the electrolytic through the battery. To model this effect, a polarization capacitor is added to model electrochemical reaction taking place in the battery. The resulting model is depicted in Figure 6.3.

In electric vehicles, the state of charge (SOC) of the battery corresponds to a fuel gauge in a conventional, internal combustion, cars. To define the state of charge, consider a completely charged battery. Let I_{b}(t) be the discharging current. Then, the charge delivered by the Figure 6.3: Battery model accounting for the different charging, R_{c}, and discharging, R_{d}, resistances and polarization capacitance, C.

battery is

Q(t) = ∫_{t_{0}}^{t} I_{b}(τ)dτ.

Let Q_{0} = ∫_{t_{0}}^{∞} I_{b}(τ)dτ be the total charge that battery can hold. Then the state of charge (SOC) of the battery is

SOC(t) = \frac{Q_{0} − Q(t)}{Q_{0}} × 100
= \frac{Q_{0} − ∫_{t_{0}}^{t} I_{b}(τ)dτ}{Q_{0}} × 100.  (6.1)

Direct application of (6.1) to estimate the battery SOC is not accurate. An alternative approach is to use an estimate of the open circuit voltage, V_{oc} of the battery. Then, the SOC can be evaluated from the relation,

SOC(t) = \frac{V_{oc}(t) − a_{0}}{a_{1}} ,  (6.2)

where a_{0} is the battery terminal voltage when SOC(t) = 0% and a_{1} is obtained using a_{0} and V_{oc} at SOC(t) = 100%. Thus estimating the SOC is equivalent to estimating the battery open-circuit voltage V_{oc}(t). Now, in order to measure V_{oc}, the battery must be disconnected from its load, which is not possible during the electric vehicle operation that uses the battery whose SOC we wish to estimate. The approach proposed by Chiasson and Vairamohan [60, 61] is to construct the open-circuit voltage estimator. This estimator construction is based on the battery model that we develop next.

Figure 6.4: A battery model in the discharging mode of operation.

We analyze the condition when the battery is in the discharging mode, that is, when I_{b} ≥ 0. The charging mode is analyzed in a similar way. The battery model in the discharging mode is depicted in Figure 6.4. To proceed, we apply Kirchhoff’s current law (KCL) at the node between the resistances R_{d} and R_{b} to obtain

I_{c} + I_{b} − I_{d} = 0,

that is,

I_{c} = I_{d} − I_{b}.

Applying Ohm’s law gives

I_{d} = \frac{V_{oc} − V_{p}}{R_{d}} and I_{b} = \frac{V_{p} − V_{b}}{R_{b}} .  (6.3)

We also have

I_{c} = CV̇_{p}.

Combining the above equations yields

CV̇_{p} = I_{d} − I_{b}
= \frac{V_{oc} − V_{p}}{R_{d}} − I_{b}
= −\frac{V_{p}}{R_{d}} + \frac{V_{oc}}{R_{d}} − I_{b}.  (6.4)

Dividing both sides by C, we obtain

V̇_{p} = −\frac{V_{p}}{R_{d}C} + \frac{V_{oc}}{R_{d}C} − \frac{1}{C} I_{b}.  (6.5)

The output equation is obtained from the second of the equations in (6.3),

V_{b} = V_{p} − R_{b}I_{b}  (6.6)

We define the state variables,

x_{1} = V_{p}, x_{2} = \frac{1}{R_{d}C} , x_{3} = \frac{V_{oc}}{R_{d}C} , x_{4} = \frac{1}{C} , and x_{5} = R_{b}.

Applying the state variable definitions to (6.5) and (6.6), and taking into account that x_{2}, x_{3}, x_{4}, and x_{5} are constant, we obtain a nonlinear time-varying state-space model,

\begin{matrix}ẋ_{1} & = & −x_{1}x_{2} + x_{3} − I_{b}(t)x_{4} \\ ẋ_{2} & = & 0 \\ ẋ_{3} & = & 0 \\ ẋ_{4} & = & 0 \\ ẋ_{5} & = & 0 \\ y & = & V_{b} = x_{1} − I_{b}(t)x_{5} \end{matrix} \}  (6.7)

The initial state vector is unknown to us. The state variable x_{3} depends on V_{oc}. Recall that the state variables, x_{2}, x_{3}, x_{4} and x_{5} are constant. Our objective is to compute the open-circuit voltage V_{oc}, which is a constant. We note that

x_{3} = \frac{V_{oc}}{R_{d}C} = V_{oc}x_{2}.

Therefore

V_{oc} = \frac{x_{3}}{x_{2}} = \frac{x_{30}}{x_{20}}

because V_{oc} is constant. Hence, we can determine V_{oc} if we can estimate the initial values of the state variables x_{20} and x_{30}. We proceed by redefining the state variables as

z_{1} = x_{1}, z_{2} = x_{3}, z_{3} = x_{4}, z_{4} = x_{5}.

With the above in mind, and using the fact that x_{2} = constant = x_{20}, we re-write (6.7) as

[ ż_{1} ; ż_{2} ; ż_{3} ; ż_{4} ] = [ \begin{matrix}−x_{20} & 1 & −I_{b}(t) & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{matrix} ] [ \begin{matrix}z_{1} \\ z_{2} \\ z_{3} \\ z_{4} \end{matrix} ]  (6.8)

y = [ 1 0 0 −I_{b}(t) ] [ \begin{matrix}z_{1} \\ z_{2} \\ z_{3} \\ z_{4} \end{matrix} ] .  (6.9)

The above is a LTV system because the current I_{b}(t) is time-varying. In summary, we have a homogenous, or free, or unforced, LTV system model

ż = A(t, x_{20})z, z(t_{0})
y = c(t)z.

Our objective is to estimate z(t_{0}) from knowledge of the output y(t) and the current I_{b}(t) on some time interval [t_{0}, t_{f}].

We now outline a solution to the problem of estimating z(t_{0}). The first step is to compute the state-transition matrix, Φ(t, t_{0}), which relates the state at time t to the initial state,

z(t) = Φ(t, t_{0})z(t_{0}).

We then use the output equation to obtain

y(t) = cz(t)
= cΦ(t, t_{0})z(t_{0}).

Premultiply both sides of the above by Φ(t, t_{0})^{⊤}c^{⊤} and then integrate both sides to get

∫_{t_{0}}^{t_{f}} Φ(t, t_{0})^{⊤}c^{⊤}y(t)dt = ∫_{t_{0}}^{t_{f}} Φ(t, t_{0})^{⊤}c^{⊤}cΦ(t, t_{0})dt z(t_{0}).

Let

V (t_{0}, t_{f}) = ∫_{t_{0}}^{t_{f}} Φ(t, t_{0})^{⊤}c^{⊤}cΦ(t, t_{0})dt,

which, as in the LTI case, we refer to as the observability gramian. If V (t_{0}, t_{f}) is invertible then

z(t_{0}) = V^{−1}(t_{0}, t_{f}) ∫_{t_{0}}^{t_{f}} Φ(t, t_{0})^{⊤}c^{⊤}y(t)dt.

Having obtained z(t_{0}) we can compute V_{oc} and then the SOC. We refer to Chiasson and Vairamohan [60, 61] for a complete solution to this problem; see also Pang et al. [262], and McIntyre et al. [238] for alternative solutions.

To be able to solve this problem, we need to acquaint ourselves with the state transition matrix for LTV systems. We start with the LTV uncontrolled system.

6.2 Solving the Uncontrolled LTV System

In general, it is not possible to give explicit expressions for solutions of linear time-varying systems. We obtain some general properties that for some cases will be used to obtain explicit solutions. We first consider the uncontrolled system

ẋ(t) = A(t)x(t), x(t_{0}) = x_{0}.  (6.10)

Theorem 6.1 If A(t) is continuous for t ≥ t_{0} then the uncontrolled system modeled by (6.10) has a unique solution for t ≥ t_{0} given by

x(t) = X(t)x_{0},

where X(t) is the unique n × n matrix satisfying

\frac{dX}{dt} = A(t)X, X(t_{0}) = I_{n}.

Proof See Rugh [287, Chp. 3] or Antsaklis and Michel [13, Section 1.8]. See also Exercise 6.2.

□

Note that it follows from the above theorem that for every initial state x^{(i)}(t_{0}), i = 1, 2, …, n, there exists a unique solution x^{(i)}(t), i = 1, 2, …, n. We arrange these n solutions as

X(t) = [ x^{(1)}(t) x^{(2)}(t) ⋯ x^{(n)}(t) ] .

Because every x^{(i)}(t) satisfies (6.10), we have

Ẋ(t) = A(t)X(t).

If X(t_{0}) is nonsingular, then X(t) is called a fundamental matrix of ẋ = A(t)x. Because the initial states are linearly independent, the fundamental matrix is nonsingular—see Exercise 6.1. The fundamental matrix is not unique because we can choose initial states arbitrarily.

We now analyze a scalar LTV case,

ẋ = a(t)x, x(0) = x_{0}.  (6.11)

Recall that when a(t) = a =constant, then the solution has the form

x(t) = e^{at}x(0) = e^{∫_{0}^{t} adτ}x(0).

The above formula works for the scalar LTV case, that is,

x(t) = e^{∫_{0}^{t} a(τ)dτ}x(0).  (6.12)

It is easy to verify that the above is the solution of (6.11) because

\frac{d}{dt}e^{∫_{0}^{t} a(τ)dτ} = e^{∫_{0}^{t} a(τ)dτ}a(t)
= a(t)e^{∫_{0}^{t} a(τ)dτ}.

That is, a(t) and ∫_{0}^{t} a(τ)dτ commute.

Example 6.1 We use the solution formula (6.12) to solve the equation,

ẋ = (sin t)x, x(0) = x_{0}.

We have

x(t) = e^{∫_{0}^{t} a(τ)dτ}x_{0}
= x_{0}e^{∫_{0}^{t} sin(τ)dτ}
= x_{0} e^{− cos t}|_{0}^{t}
= e^{− cos t+1}x_{0}
= e × e^{− cos t}x_{0}.

The solution formula (6.12) does not generalize to the multi-dimensional case because, in general,

\frac{d}{dt}e^{∫_{0}^{t} A(τ)dτ} ≠ A(t)e^{∫_{0}^{t} A(τ)dτ}.

If, however, A(t) and ∫_{0}^{t} A(τ)dτ commute, then the solution to (6.10) can be written as

x(t) = e^{∫_{0}^{t} A(τ)dτ}x(0).

We note that A(t) and ∫_{0}^{t} A(τ)dτ commute if

A(t)A(s) = A(s)A(t).

We now analyze a general case when t_{0} is not necessarily equal to 0. If A(t) = A in (6.10), then the solution is

x(t) = e^{A(t−t_{0})}x(t_{0}).

We re-write the above as

x(t) = e^{At}e^{−At_{0}}x(t_{0}).

6.2. SOLVING THE UNCONTROLLED LTV SYSTEM 301 Let X(t) = e^{At}, then

X(t_{0})^{−1} = e^{−At_{0}}

and we represent the solution as

x(t) = X(t)X(t_{0})^{−1}x(t_{0})
= Φ(t, t_{0})x(t_{0}),  (6.13)

where Φ(t, t_{0}) is the state transition matrix. Definition 6.1 Let X(t) be a fundamental matrix of (6.10). Then

Φ(t, t_{0}) = X(t)X(t_{0})^{−1}

is the state transition matrix of (6.10). The state transition matrix is the unique solution of

\frac{∂}{∂t}Φ(t, t_{0}) = A(t)Φ(t, t_{0}), Φ(t_{0}, t_{0}) = I_{n}.

◇

As in the LTI case, the state transition matrix in the LTV case has the following properties (see Exercise 6.5):

Φ(τ, τ) = I_{n}  (6.14)
Φ(t, τ)^{−1} = Φ(τ, t)  (6.15)
Φ(t, t_{0}) = Φ(t, t_{1})Φ(t_{1}, t_{0})  (6.16)
\frac{∂}{∂τ} Φ(t, τ) = −Φ(t, τ)A(τ), for all t, τ  (6.17)

The third of the above properties is referred to as the semigroup property. Example 6.2 The state transition matrix of

ẋ = (sin t)x, x(t_{0}) = x_{0},

has the form

Φ(t, t_{0}) = X(t)X(t_{0})^{−1}
= \frac{e^{− cos t+1}}{e^{− cos t_{0}+1}} .

302 CHAPTER 6. A GLIMPSE AT LINEAR TIME-VARYING (LTV) SYSTEMS Example 6.3 We construct the state transition matrix for the LTV system,

ẋ = [ \begin{matrix}0 & −t \\ 0 & 0 \end{matrix} ] x, x(0) = x(0).

The above system consists of two first-order linear differential equations,

ẋ_{1} = −tx_{2}, and ẋ_{2} = 0.

We can easily obtain the solution to the second of the above equations,

x_{2}(t) = x_{2}(0).  (6.18)

We next obtain the solution to ẋ_{1} = −tx_{2} by integrating its both sides,

x_{1}(t) = x_{1}(0) − ∫_{0}^{t} τx_{2}(0)dτ
= x_{1}(0) − \frac{t^{2}x_{2}(0)}{2}.  (6.19)

We can now construct a fundamental matrix. Let

x^{(1)}(0) = [ \begin{matrix}1 \\ 1 \end{matrix} ] and x^{(2)}(0) = [ \begin{matrix}0 \\ 1 \end{matrix} ] .

Then using (6.19) and (6.18), we obtain the fundamental matrix corresponding to the above initial conditions is

X(t) = [ x^{(1)}(t) x^{(2)}(t) ]
= [ \begin{matrix}1 − \frac{t^{2}}{2} & −\frac{t^{2}}{2} \\ 1 & 1 \end{matrix} ] .

If we select the initial conditions

x^{(1)}(0) = [ \begin{matrix}1 \\ 0 \end{matrix} ] and x^{(2)}(0) = [ \begin{matrix}0 \\ 1 \end{matrix} ]

then we obtain a different fundamental matrix,

X(t) = [ \begin{matrix}1 & −\frac{t^{2}}{2} \\ 0 & 1 \end{matrix} ] .

The state transition matrix, however, is unique and has the form

Φ(t, t_{0}) = X(t)X(t_{0})^{−1}
= [ \begin{matrix}1 − \frac{t^{2}}{2} & −\frac{t^{2}}{2} \\ 1 & 1 \end{matrix} ] [ \begin{matrix}1 − \frac{t_{0}^{2}}{2} & −\frac{t_{0}^{2}}{2} \\ 1 & 1 \end{matrix} ]^{−1}
= [ \begin{matrix}1 − \frac{t^{2}}{2} & −\frac{t^{2}}{2} \\ 1 & 1 \end{matrix} ] [ \begin{matrix}1 & \frac{t_{0}^{2}}{2} \\ −1 & 1 − \frac{t_{0}^{2}}{2} \end{matrix} ]
= [ \begin{matrix}1 & \frac{1}{2}(t_{0}^{2} − t^{2}) \\ 0 & 1 \end{matrix} ] .

Example 6.4 We construct the state-transition matrix for the LTV system model found in Luenberger [223, p. 115],

ẋ = [ \begin{matrix}0 & 0 \\ t & \frac{1}{t} \end{matrix} ] x, t ≥ 1.

Then, for the given x(2) = [ 2 −1 ]^{⊤}, we compute x(1).

First, we solve for x_{1} to obtain,

x_{1}(t) = x_{1}(1), t ≥ 1

Substituting the above into the equation for ẋ_{2} gives

ẋ_{2} = tx_{1}(t) + \frac{x_{2}(t)}{t}.

Next, we make the substitution

z(t) = \frac{x_{2}(t)}{t}.

Computing the derivative of z(t) yields

ż = \frac{ẋ_{2}t − x_{2}}{t^{2}}
= \frac{ẋ_{2}}{t} − \frac{x_{2}}{t^{2}} .

Multiplying both sides of the above by t, we obtain

tż = ẋ_{2} − \frac{x_{2}}{t^{2}}
= ẋ_{2} − z.  (6.20)

304 CHAPTER 6. A GLIMPSE AT LINEAR TIME-VARYING (LTV) SYSTEMS We substitute ẋ_{2} = tx_{1}(1) + z into 6.20 to obtain

tż + z = tx_{1}(1) + z,

that is,

ż = x_{1}(1).

Integrating the above from 1 to t gives

z(t) = z(1) + x_{1}(1)(t − 1).

Therefore,

z = \frac{x_{2}}{t}
= z(1) + x_{1}(1)(t − 1),

and multiplying both sides by t, we obtain

x_{2}(t) = tx_{2}(1) + tx_{1}(1)(t − 1).

We are now ready to generate a fundamental matrix. Let

x^{(1)}(1) = [ \begin{matrix}1 \\ 0 \end{matrix} ] and x^{(2)}(1) = [ \begin{matrix}0 \\ 1 \end{matrix} ] ,

then we have

x^{(1)}(t) = [ \begin{matrix}1 \\ t^{2} − t \end{matrix} ] and x^{(2)}(t) = [ \begin{matrix}0 \\ t \end{matrix} ] .

The fundamental matrix corresponding to the above initial conditions is

X(t) = [ x^{(1)}(t) x^{(2)}(t) ] = [ \begin{matrix}1 & 0 \\ t^{2} − t & t \end{matrix} ] .

The state transition matrix is

Φ(t, τ) = X(t)X(τ)^{−1}
= [ \begin{matrix}1 & 0 \\ t^{2} − t & t \end{matrix} ] [ \begin{matrix}1 & 0 \\ τ^{2} − τ & τ \end{matrix} ]^{−1}
= [ \begin{matrix}1 & 0 \\ t^{2} − t & t \end{matrix} ] [ \begin{matrix}1 & 0 \\ −τ + 1 & \frac{1}{τ} \end{matrix} ]
= [ \begin{matrix}1 & 0 \\ t(t − τ) & \frac{t}{τ} \end{matrix} ] .

6.3. SOLVING THE LTV CONTROLLED SYSTEM 305 Note that

x(t) = Φ(t, τ)x(τ).

Thus, for example, if we are given x(2) = [ 2 −1 ]^{⊤} and we wish to find x(1), we have,

x(1) = Φ(1, 2)x(2)
= [ \begin{matrix}1 & 0 \\ −1 & \frac{1}{2} \end{matrix} ] [ \begin{matrix}2 \\ −1 \end{matrix} ]
= [ \begin{matrix}2 \\ −\frac{5}{2} \end{matrix} ] .

6.3 Solving the LTV Controlled System

We now discuss the problem of obtaining the solution to the LTV controlled system

ẋ = A(t)x + B(t)u, x(t_{0}) = x_{0}.  (6.21)

We claim that, by analogy with the LTI controlled system, the solution to (6.21) is

x(t) = Φ(t, t_{0})x_{0} + ∫_{t_{0}}^{t} Φ(t, τ)B(τ)u(τ)dτ.  (6.22)

We can prove our claim by verification utilizing Leibniz’s rule (see Exercise 6.7),

\frac{∂}{∂t} ∫_{t_{0}}^{t} f(t, τ)dτ = ∫_{t_{0}}^{t} (\frac{∂}{∂t}f(t, τ)) dτ + f(t, τ)|_{τ=t}  (6.23)

Example 6.5 (R. A. DeCarlo) The LTV state model,

ẋ(t) = A(t)x(t) + [ \begin{matrix}πe^{−α(t)^{2}} \\ 0 \end{matrix} ] u(t),

has associated state transition matrix

Φ(t, τ) = e^{α(τ)^{2}−α(t)^{2}} [ \begin{matrix}1 & 0 \\ cos(πt) − cos(πτ) & 1 \end{matrix} ] ,

where α(·) is a continuous function.

1. Compute A(t) using the given state transition matrix.

2. Find the zero-state response when the input u(t) is the unit step.

We first compute A(t) using the given state transition matrix. We use the state transition matrix properties to accomplish this. Recall that the state transition matrix satisfies the equation

\frac{∂}{∂t}Φ(t, τ) = A(t)Φ(t, τ), Φ(τ, τ) = I_{n}.

Note that, we can also write.

Φ(t, τ)|_{τ=t} = I_{n}

Taking this into account, we obtain

A(t) = \frac{∂}{∂t}Φ(t, τ)|_{τ=t} .

To proceed, we first evaluate

\frac{∂}{∂t}e^{α(τ)^{2}−α(t)^{2}} |_{τ=t} = e^{α(τ)^{2}−α(t)^{2}}(−2α(t)α̇(t))|_{τ=t}
= −2α(t)α̇(t).

We next evaluate the derivative

\frac{∂}{∂t} (e^{α(τ)^{2}−α(t)^{2}} (cos(πt) − cos(πτ)))|_{τ=t} = −2α(t)α̇(t)(cos(πt) − cos(πt)) − π sin(πt)
= −π sin(πt).

Hence

A(t) = \frac{∂}{∂t}Φ(t, τ)|_{τ=t}
= [ \begin{matrix}−2α(t)α̇(t) & 0 \\ −π sin(πt) & −2α(t)α̇(t) \end{matrix} ] .

We now compute the zero-state response. The formula for the zero-state response is obtained from (6.22) by setting x_{0} = 0,

x(t) = ∫_{t_{0}}^{t} Φ(t, τ)B(τ)u(τ)dτ.

We first compute the integrand

Φ(t, τ)B(τ) = e^{α(τ)^{2}−α(t)^{2}} [ \begin{matrix}1 & 0 \\ cos(πt) − cos(πτ) & 1 \end{matrix} ] [ \begin{matrix}πe^{−α(τ)^{2}} \\ 0 \end{matrix} ]
= πe^{−α(t)^{2}} [ \begin{matrix}1 \\ cos(πt) − cos(πτ) \end{matrix} ] .

Therefore, the zero-state response when u(t) = 1(t) is

x(t) = ∫_{t_{0}}^{t} Φ(t, τ)B(τ)u(τ)dτ
= πe^{−α(t)^{2}} ∫_{0}^{t} [ \begin{matrix}1 \\ cos(πt) − cos(πτ) \end{matrix} ] dτ
= πe^{−α(t)^{2}} [ \begin{matrix}t \\ t cos(πt) − \frac{sin(πτ)}{π} \end{matrix} ] .

6.4 Notes

For additional information on the fundamentals of the LTV systems, we recommend Kamen [173]. An in-depth treatment of the LTV systems can be found in Rugh [288], and in Antsaklis and Michel [13]. For results on observers for LTV systems, see O’Reilly [258]. A novel approach to the analysis and control of LTV systems is proposed by Bourlès and Marinescu [39].

6.5 Exercises

Exercise 6.1 Prove that a fundamental matrix is nonsingular for any t.

Exercise 6.2 Use the contraction map theorem A.19 to prove that (6.10) has a unique solution going through x_{0} at t = t_{0}, where t_{0} is an element of a finite time interval [a, b]. (Start with the Lipschitz condition.)

Exercise 6.3 (a) Find the state-transition matrix Φ(t, τ) associated with a homogenous LTI system with

A = T [ \begin{matrix}−2 & 0 \\ 0 & 0 \end{matrix} ] T^{−1},

where

T = [ \begin{matrix}1 & 0 \\ −3 & 1 \end{matrix} ] .

(b) Find the state-transition matrix Φ(t, τ) associated with a homogenous LTV system

with

A(t) = T [ \begin{matrix}−2 & 0 \\ 0 & cos t \end{matrix} ] T^{−1},

where

T = [ \begin{matrix}1 & 0 \\ −3 & 1 \end{matrix} ] .

Exercise 6.4 Compute the state transition matrix Φ(t, τ) for the linear time-varying (LTV) system model,

ẋ(t) = [ \begin{matrix}0 & 0 \\ 1 & \frac{1}{t} \end{matrix} ] x(t)

defined for t ≥ 1. Then, for the given x(2) = [ 0 4 ]^{⊤}, compute x(1).

Exercise 6.5 Use the definition of the state transition matrix to prove its properties (6.14), (6.15), (6.16), and (6.17).

Exercise 6.6 (a) The value of the state-transition matrix for a homogeneous linear system

is

Φ(1, 3) = [ \begin{matrix}1 & 1 \\ 1 & 2 \end{matrix} ] .

Find x(3) when

x(1) = [ \begin{matrix}1 \\ 3 \end{matrix} ] .

(b) The values of the state-transition matrix for a homogeneous linear system is

Φ(1, 3) = [ \begin{matrix}1 & 1 \\ 1 & 2 \end{matrix} ] and Φ(2, 3) = [ \begin{matrix}2 & 3 \\ 4 & 5 \end{matrix} ] .

If

x(1) = [ \begin{matrix}2 \\ −3 \end{matrix} ] ,

find x(2).

6.5. EXERCISES 309 Exercise 6.7 Derive Leibniz’s rule given by (6.23). Exercise 6.8 (a) The state-transition matrix of a homogenous LTV system is

Φ(t, τ) = [ \begin{matrix}e^{cos t−cos τ} & 0 \\ 0 & e^{− sin t+sin τ} \end{matrix} ] .

Find the associated A(t)-matrix.

(b) A fundamental matrix associated with a homogenous continuous-time LTV state model

has the form,

X(t) = [ \begin{matrix}2 & −1 \\ −1 & 1 \end{matrix} ] [ \begin{matrix}1 & 0 \\ t^{2} − t & t \end{matrix} ] .

Compute the state-transition matrix Φ(t, τ). Exercise 6.9 (R. Balakrishnan) For the linear time-varying (LTV) system model,

ẋ(t) = [ \begin{matrix}0 & −2t \\ 0 & −t \end{matrix} ] x(t) + [ \begin{matrix}0 \\ t \end{matrix} ] u(t),

• Find the state transition matrix Φ(t, τ);

• Let x(1) = [ 1 1 ]^{⊤} and u(t) = 0 for t ≥ 0. Find x(0);

• Let x(0) = 0 and u(t) = 1 for t ≥ 0. Find x(t) for t ≥ 0.

310 CHAPTER 6. A GLIMPSE AT LINEAR TIME-VARYING (LTV) SYSTEMS

Stability

With the spread of the centrifugal governor in the early 19th century a number of major problems became apparent. First, because of the absence of integral action, the governor could not remove offset: in the terminology of the time it could not regulate but only moderate. Second, its response to a change in load was slow. And thirdly, (nonlinear) frictional forces in the mechanism could lead to hunting (limit cycling). A number of attempts were made to overcome these problems: for example, the Siemens chronometric governor effectively introduced integral action through differential gearing, as well as mechanical amplification. Other approaches to the design of an isochronous governor (one with no offset) were based on ingenious mechanical constructions, but often encountered problems of stability.—Bissell [36, p. 56]

7.1 An Informal Introduction to Stability

In general, it can be difficult to obtain an explicit solution of nonlinear differential equations modeling a dynamical system. However, often we do not need to know the explicit solution of the modeling equations. Instead, we are interested in the behavior of the solution as time tends to infinity. Furthermore, in many models of physical systems a “small” change in the initial conditions results in a “small” change in the solution. Specifically, if a system is perturbed from its rest, or equilibrium position, then it starts to move. We can roughly say that the equilibrium position is stable if the system does not go far from this position for small initial perturbations. This concept may be illustrated, as in Willems [351], Brogan [44], or in Barnett [23], by means of a ball resting in equilibrium on a sheet of metal bent into

Figure 7.1: Illustration of stable and unstable equilibrium positions.

various shapes with cross-sections depicted in Figure 7.1. This figure illustrates four possible scenarios. If we neglect frictional forces, a small perturbation from the equilibrium leads to:

(1) oscillatory motion about equilibrium irrespective of the magnitude of the initial perturbation. In this case the equilibrium position is globally stable,

(2) the new position is also an equilibrium and the ball will remain in it. Here the equilibrium position is stable,

(3) the ball moving away without returning to the equilibrium. This equilibrium position is unstable,

(4) oscillatory motion about the equilibrium, unless the initial perturbation is so large that the ball is forced to oscillate about the new equilibrium position. This equilibrium position is locally stable.

If friction is taken into account, then the oscillatory motion steadily decreases until the ball returns to the equilibrium positions as in items (1) and (4) above. We refer to such equilibrium positions as being asymptotically stable. In our discussion of stability concepts, we implicitly assume, as in item (4), that there is some “small” region for allowed initial perturbations for which subsequent motions converge to the equilibrium position. If however, as in item (1), any initial perturbation yields a motion that converges to the equilibrium position, then we have global asymptotic stability, which is also referred to as asymptotic stability in the large.

The above informal discussion should serve as a preparation for an in-depth analysis of stability of dynamic systems. We begin our analysis by formally defining the equilibrium state. We then define stability, asymptotic stability, and other relevant notions.

7.2 Basic Definitions of Stability

We consider a class of dynamical systems modeled by the equation:

ẋ(t) = f(t, x(t)), x(t_{0}) = x_{0},  (7.1)

where x(t) ∈ ℝ^{n} is the state vector and f : ℝ × ℝ^{n} → ℝ^{n} is a vector-valued function with the components

f_{i}(t, x_{1}, x_{2}, …, x_{n}) : ℝ × ℝ^{n} → ℝ, i = 1, 2, …, n.

We assume that f_{i}’s are continuous and have continuous first partial derivatives. Under these assumptions, the solution to ẋ(t) = f(t, x(t)), x(t_{0}) = x_{0}, exists and is unique. We denote this solution as x(t) = x(t; t_{0}, x_{0}). If the f_{i}’s do not depend explicitly on t, then the system represented by ẋ(t) = f(t, x(t)) = f(x(t)) is called autonomous or time-invariant, otherwise it is called non-autonomous or time-varying.

A point x_{e} is an equilibrium point, or state, at time 0 if for all t ≥ 0,

f(t, x_{e}) = 0.

We note that if x_{e} is an equilibrium state at time 0, then it is also an equilibrium state for all initial times greater than or equal 0.

We also note that an equilibrium state x_{e} is a constant solution to (7.1) for t ≥ 0.

Observe that for the time-invariant system

ẋ(t) = f(x)

a point is an equilibrium at some time if and only if is is an equilibrium at all times.

If x_{e} is an equilibrium point, we can arrange for this equilibrium point to be transferred to the origin of ℝ^{n} by introducing the new variable

x̃ = x − x_{e}.

We assume that this has been done for an equilibrium point under consideration. Therefore, we have

f(t, 0) = 0 for all t ≥ t_{0}.

There may be several equilibrium points for a given system. We assume that there is no other equilibrium point in a neighborhood of the origin of ℝ^{n}. Thus, the origin is an isolated equilibrium point. We formally define isolated equilibrium in Definition 4.3.

We now introduce basic definitions of stability in the sense of A. M. Lyapunov^{1} (1857-1918). In the following considerations we use the standard Euclidean norm of a vector. Thus, if x ∈ ℝ^{n} then

∥x∥ = ∥x∥_{2} = (x^{⊤}x)^{\frac{1}{2}} .

Definition 7.1 An equilibrium state x_{e} is said to be stable if for any given t_{0} ≥ 0 and any positive scalar ε, there exists a positive scalar δ = δ(t_{0}, ε) such that if ∥x(t_{0}) − x_{e}∥ < δ, then ∥x(t; t_{0}, x_{0}) − x_{e}∥ < ε for all t ≥ t_{0}.

◇

Friedland [108, page 43] makes the following comment regarding the above definition of stability: “If this “epsilonics” definition leaves you cold, think of a contest between you, the system designer, and an adversary (nature?). The adversary picks a region in state space of radius ε and challenges you to find another region, of radius δ, such that if the initial state starts out inside your region, it remains inside his region. If you can do this, your system design is stable.” We add that you have to find an appropriate δ for any choice of ε for the equilibrium state to be stable.

In summary, an equilibrium state, x_{e}, is stable if whenever the initial state is near x_{e}, the state trajectory, x(t), remains near the equilibrium, perhaps even tending toward this equilibrium state as time increases. Note that Lyapunov’s stability does not imply that x(t) converges to x_{e} as t → ∞. The states are only required to “hover” around the equilibrium state x_{e}. However, the stability condition bounds the amount of wiggling room for x(t) in the sense that small perturbations in the initial conditions produce small perturbations in the solution x(t).

Definition 7.2 An equilibrium state x_{e} is said to be convergent, or attractive, if for any given t_{0} ≥ 0 there is a positive scalar δ_{1} = δ_{1}(t_{0}) such that if ∥x(t_{0}) − x_{e}∥ < δ_{1}, then

lim_{t→∞} x(t; t_{0}, x_{0}) = x_{e}.

1Sometimes transliterated in the literature as Liapunov or Liapunoff.

◇

We can restate Definition 7.2 as follows. An equilibrium state x_{e} is convergent if for any t_{0} ≥ 0 there is a positive scalar δ_{1} = δ_{1}(t_{0}) such that if ∥x(t_{0}) − x_{e}∥ < δ_{1}, then for any given ε_{1} > 0 there exists a T = T (ε_{1}, t_{0}, x_{0}) such that ∥x(t; t_{0}, x_{0}) − x_{e}∥ < ε_{1} for all t > t_{0} + T.

Definition 7.3 An equilibrium state x_{e} is asymptotically stable if it is stable and convergent.

◇

We note that for linear time-invariant (LTI) systems asymptotic stability is equivalent with convergence. That is, in a convergent LTI system, stability condition is automatically satisfied. In general, however, it is not sufficient that just

x(t; t_{0}, x_{0}) → x_{e} as t → ∞

for asymptotic stability. We need stability too! Why? Hahn [129], see also [12, p. 451], constructed a nonlinear system example 7.2 where convergence does not mean asymptotic stability as in the case of linear time-invariant systems. In this example the origin is attractive but not asymptotically stable.

Example 7.1 This is a second-order nonlinear system described by

[ ẋ_{1} ; ẋ_{2} ] = [ \frac{x_{1}^{2}(x_{2}−x_{1})+x_{2}^{5}}{(x_{1}^{2}+x_{2}^{2})(1+(x_{1}^{2}+x_{2}^{2})^{2})} ; \frac{x_{2}^{2}(x_{2}−2x_{1})}{(x_{1}^{2}+x_{2}^{2})(1+(x_{1}^{2}+x_{2}^{2})^{2})} ] .  (7.2)

A phase portrait of the above system is shown in Figure 7.2. All trajectories, solutions, are approaching the equilibrium state x_{e} = 0, without this equilibrium being asymptotically stable.

Definition 7.4 An equilibrium state x_{e} is said to be uniformly stable if the positive scalar δ = δ(t_{0}, ε) introduced in Definition 7.1 can be taken independent of the initial time t_{0}.

◇

Example 7.2 The following example that comes from Rugh [288, page 100] nicely illustrates the notion of uniform stability. The solution to the differential equation,

ẋ(t) = (4t sin t − 2t)x(t), x (t_{0}) = x_{0}  (7.3)

is

x(t) = x_{0} exp (4 sin t − 4t cos t − t^{2} − 4 sin t_{0} + 4t_{0} cos t_{0} + t_{0}^{2}) .  (7.4)

One can verify this using MATLAB’s Symbolic Toolbox by typing the command, Figure 7.2: A phase portrait of system (7.2) where the origin is attractive but not asymptotically stable.

x=dsolve('Dx=(4*t*sin(t)-2*t)*x, x(t_0)=x_0','t')

It is easy to show that for fixed t_{0} there is a γ such that (7.4) is bounded by γ|x_{0}| for all t ≥ t_{0}. This is because the term (−t^{2}) dominates the exponent as t increases. Therefore, for a given ε, it is enough to choose x_{0} so that γ|x_{0}| < ε, or equivalently δ < ε/γ, to show that the equilibrium solution x_{e} = 0 is stable. However, the equilibrium solution x_{e} = 0 is not uniformly stable. To show this, fix initial state x_{0} and consider a sequence of initial times t_{0} = 2kπ, where k = 0, 1, …, and the values of the respective solutions at times π units later:

x(2kπ + π) = x_{0} exp ((4k + 1)π(4 − π)) .

One can see that there is no bound on the exponential factor and hence a candidate bound γ increases as k increases, where increasing k corresponds to the increasing initial time.

Definition 7.5 An equilibrium state x_{e} is said to be uniformly convergent, or uniformly attractive, if the positive scalars δ_{1} = δ_{1}(t_{0}) and T = T (ε_{1}, t_{0}, x_{0}) introduced in Definition 7.2 are independent of t_{0}.

Figure 7.3: Illustration of uniform exponential stability: a decaying-exponential bound is independent of the initial time t_{0}.

◇

The last two definitions allow us to introduce the notion of uniform asymptotic stability.

Definition 7.6 An equilibrium state x_{e} is said to be uniformly asymptotically stable if it is uniformly stable and uniformly convergent.

◇

Notice that the solution x(t) = x_{e} must first be stable in order to qualify to be asymptotically stable. This is to prevent a system trajectory from straying arbitrarily far from the equilibrium state before converging toward it. Note also that the concepts of Lyapunov stability and asymptotic stability are local concepts. They apply to the system’s behavior in the small or in a “small” neighborhood about the equilibrium point (Hsu and Meyer [145]).

A special case of uniform asymptotic stability is uniform exponential stability that implies uniform stability, and imposes an additional requirement that all solutions originating “close” to the given equilibrium, x_{e} = 0, approach this equilibrium exponentially as t → ∞. More rigorously, we say that x_{e} = 0 is uniformly exponentially stable if it is stable and there exist finite positive constants γ and λ such that for any t_{0} and x_{0}, any solution starting in a neighborhood of 0 satisfies

∥x(t)∥ ≤ γe^{−λ(t−t_{0})} ∥x_{0}∥ .

Note that γ is no less than unity, and uniform implies that γ and λ are independent of t_{0}—see Figure 7.3.

An equilibrium state is unstable if it is not stable. An ε-δ definition of instability is given below.

Definition 7.7 An equilibrium state x_{e} is said to be unstable if there is an ε > 0 so that for any δ > 0 there exists x(t_{0}) such that if ∥x(t_{0}) − x_{e}∥ < δ, then ∥x(t_{1}) − x_{e}∥ ≥ ε for some t_{1} > t_{0}.

◇

A two dimensional geometric interpretation of the above definitions is shown in Figure 7.4. We assume that x_{e} = 0 is the equilibrium state. If the equilibrium state is stable, as in

Figure 7.4: Illustration of stability concepts in two dimensions: (a) stable equilibrium state in the sense of Lyapunov, (b) asymptotically stable equilibrium state, (c) unstable equilibrium state.

Figure 7.4(a), then given an outer circle of radius ε, there exists an inner circle of radius δ such that trajectories starting within the δ-circle never leave the ε-circle. If the equilibrium state is asymptotically stable, then trajectories tend to the equilibrium state as t tends to ∞. This situation is depicted in Figure 7.4(b). Finally, the equilibrium state is unstable, as in Figure 7.4(c), then there exists an ε-circle such that for every δ-circle there exists a trajectory starting within it that leaves the ε-circle at some later time. A geometric interpretation of the above described scenarios in one dimension is given in Figure 7.5.

7.2. BASIC DEFINITIONS OF STABILITY 319 Figure 7.5: Illustration of stability concepts in one dimension: (a) stable equilibrium state in the sense of Lyapunov, (b) asymptotically stable equilibrium state, (c) unstable equilibrium state.

7.3 Stability of Continuous-Time LTI Systems

It follows from the discussion in the previous sections that a linear time-invariant (LTI) dynamical system modeled by the equation

ẋ = Ax, x(t_{0}) = x_{0}  (7.5)

is asymptotically stable if and only if, the solution to equation (7.5) decays to zero as t → ∞ for any initial state x_{0}. We can view the vector x(t) ∈ ℝ^{n} as defining the coordinates of a point moving in an n-dimensional state space ℝ^{n}. In an asymptotically stable system modeled by equation (7.5), this point converges to the origin of ℝ^{n}. We can argue that if a trajectory is converging to the origin of the state space, then it should be possible to find a family of nested surfaces described by V (x_{1}, x_{2}, …, x_{n}) = c, c ≥ 0, such that monotonically decreasing values of c give surfaces progressively shrinking in on the origin with the limiting surface V (x_{1}, x_{2}, …, x_{n}) = 0 being the origin x = 0. Furthermore, along every trajectory of (7.5), the parameter c should steadily decrease. We now introduce some notation and definitions that we use in the ensuing discussion.

Let V (x) = V (x_{1}, x_{2}, …, x_{n}) : ℝ^{n} → ℝ be a real-valued function and let S ⊆ ℝ^{n} be a compact region containing the origin x = 0 in its interior.

Definition 7.8 We say that the function V = V (x) is positive semi-definite (p.s.d.) in S, with respect to x = 0, if

1. V is continuously differentiable, that is, V ∈ C^{1},

2. V (0) = 0,

3. V (x) ≥ 0 for all x ∈ S.

◇

Definition 7.9 We say that the function V = V (x) is positive definite (p.d.) in S, with respect to x = 0, if

1. V ∈ C^{1},

2. V (0) = 0,

3. V (x) > 0 for all x ∈ S \ {0}.

◇

The symbol S \ {0} in item 3 above denotes the set {x : x ∈ S, x ≠ 0}.

Negative semi-definite (n.s.d.) and negative definite (n.d.) functions are defined in a similar fashion by reversing inequality signs in item 3 in both definitions above.

There are many functions that satisfy the above definitions. In the stability analysis of dynamic systems one class is particularly useful. This is the positive definite quadratic form, a label attached to a nested family of ellipses and their extensions to higher dimensional spaces—ellipsoids for n = 3, and hyper-ellipsoids for n ≥ 4. Quadratic forms are reviewed in Section A.4. We consider the following quadratic form,

V (x) = x^{⊤}P x,

where P is a real symmetric n × n matrix. The time derivative of V (x(t)) evaluated on a solution of ẋ = Ax is

\frac{dV (x(t))}{dt} = V̇ (x(t))
= ẋ(t)^{⊤}P x(t) + x(t)^{⊤}P ẋ(t).

Substituting ẋ = Ax and ẋ^{⊤} = x^{⊤}A^{⊤} into the above equation yields

V̇ = x^{⊤}A^{⊤}P x + x^{⊤}P Ax
= x^{⊤}(A^{⊤}P + P A)x.

Let

Q = −(A^{⊤}P + P A).

Then,

V̇ = \frac{dV (x)}{dt}
= −x^{⊤}Qx.  (7.6)

We are ready to state and prove the theorem concerning the stability of the system ẋ = Ax, or equivalently, the matrix A ∈ ℝ^{n×n}.

Theorem 7.1 (A. M. Lyapunov, 1892) The system ẋ = Ax, x(t_{0}) = x_{0}, is asymptotically stable if and only if for any given real symmetric positive definite (r.s.p.d.) matrix Q the solution P to the continuous Lyapunov matrix equation

A^{⊤}P + P A = −Q  (7.7)

is also (real symmetric) positive definite.

Proof (⇒) We prove necessity, for asymptotic stability, by contradiction. We have −x^{⊤}Qx < 0 for all x ≠ 0. We assume that A is an asymptotically stable matrix and that for some nonzero x_{0} we have x_{0}^{⊤}P x_{0} ≤ 0. From the discussion before the theorem, and in particular from (7.6), it follows that

\frac{d}{dt} (x(t)^{⊤}P x(t)) = −x(t)^{⊤}Qx(t) < 0.

The above means that V (x(t)) = x(t)^{⊤}P x(t) decreases on the trajectories of the system ẋ(t) = Ax(t) as t → ∞. On the other hand, because A is asymptotically stable, we have

lim_{t→∞} x(t) = 0.

Hence,

lim_{t→∞} x(t)^{⊤}P x(t) = lim_{x(t)→0} x(t)^{⊤}P x(t) = 0.

Thus, we arrived at a contradiction. This completes the proof of the first part of the theorem that states that asymptotic stability of A implies that for any r.s.p.d matrix Q the solution P to the equation A^{⊤}P + P A = −Q is r.s.p.d..

(⇐) We prove sufficiency, also by contradiction. We assume that

x^{⊤}P x > 0 and − x^{⊤}Qx < 0 for all x ≠ 0,

and that the matrix A is not asymptotically stable. This means that the solution to ẋ(t) = Ax(t), x_{0} ≠ 0, which has the form x(t) = e^{A(t−t_{0})}x_{0}, is bounded away from the origin x = 0. Thus, there is a constant b > 0 such that

x(t)^{⊤}Qx(t) > b > 0 for all t > t_{0}.

Integrating \frac{d}{dt} (x(t)^{⊤}P x(t)) = −x(t)^{⊤}Qx(t) yields

x(t)^{⊤}P x(t) = x(t_{0})^{⊤}P x(t_{0}) + ∫_{t_{0}}^{t}(−x(s)^{⊤}Qx(s))ds.

Taking into account that −x(t)^{⊤}Qx(t) < −b < 0 for all t > t_{0}, we obtain for t > t_{0},

x(t)^{⊤}P x(t) = x(t_{0})^{⊤}P x(t_{0}) + ∫_{t_{0}}^{t}(−x(s)^{⊤}Qx(s))ds
< x(t_{0})^{⊤}P x(t_{0}) − b(t − t_{0}).

The above implies that for a sufficiently large t we would have x(t)^{⊤}P x(t) < 0, which contradicts the assumption that x(t)^{⊤}P x(t) > 0. This means that A must be asymptotically stable, and the proof is complete.

□

Example 7.3 Let

A = [ \begin{matrix}0 & 1 \\ −1 & −2 \end{matrix} ] ,

and Q = I_{2}. We solve the Lyapunov equation A^{⊤}P + P A = −Q for P. Assume P to have the form,

P = [ \begin{matrix}p_{1} & p_{2} \\ p_{2} & p_{3} \end{matrix} ] = P^{⊤}.

Then, we have

A^{⊤}P + P A = [ \begin{matrix}0 & −1 \\ 1 & −2 \end{matrix} ] [ \begin{matrix}p_{1} & p_{2} \\ p_{2} & p_{3} \end{matrix} ] + [ \begin{matrix}p_{1} & p_{2} \\ p_{2} & p_{3} \end{matrix} ] [ \begin{matrix}0 & 1 \\ −1 & −2 \end{matrix} ]
= [ \begin{matrix}−2p_{2} & −p_{3} + p_{1} − 2p_{2} \\ −p_{3} + p_{1} − 2p_{2} & 2p_{2} − 4p_{3} \end{matrix} ]
= −Q = [ \begin{matrix}−1 & 0 \\ 0 & −1 \end{matrix} ] .

Solving the corresponding linear equations yields

P = [ \begin{matrix}3/2 & 1/2 \\ 1/2 & 1/2 \end{matrix} ] .

The solution matrix P is positive definite. Hence, the matrix A must have its eigenvalues in the open left-half plane. Indeed, the eigenvalues of A are both located at −1.

Definition 7.10 A positive definite function V (x) whose time derivative evaluated on the solutions of the system ẋ = Ax is negative definite (or negative semi-definite) is called a Lyapunov function for this system.

◇

Lyapunov’s theorem gives us a necessary and sufficient condition for asymptotic stability of linear time-invariant dynamical systems. To check if a given matrix A is asymptotically stable or not, it is enough to take an arbitrary r.s.p.d. matrix Q, solve the equation A^{⊤}P + P A = −Q for real symmetric P and check if the obtained P is positive definite, or not. If P > 0, then A is asymptotically stable. If P is not positive definite, then A cannot be asymptotically stable. Because Q can be an arbitrary r.s.p.d. matrix, Q = I_{n}, the n × n identity matrix, is a good choice. It is important to note that there would be no use, as the following example shows, in first choosing a matrix P and then calculating Q from the equation A^{⊤}P + P A = −Q.

Example 7.4 Let

A = [ \begin{matrix}−1 & 3 \\ 0 & −1 \end{matrix} ] ,

which is asymptotically stable. Let

P = [ \begin{matrix}1 & 0 \\ 0 & 1 \end{matrix} ] = I_{2} > 0.

Then,

Q = −(A^{⊤}P + P A) = [ \begin{matrix}2 & −3 \\ −3 & 2 \end{matrix} ]

which is indefinite. Thus, unless Q turns out to be definite, nothing can be inferred about asymptotic stability from the Lyapunov theorem if we start with P and then calculate Q.

Figure 7.6: Geometric interpretation of the Lyapunov theorem.

We now give a geometric interpretation of the Lyapunov’s theorem. First, note that the function

V̇ = \frac{dV (x(t))}{dt}

can be expressed, using the chain rule, as

V̇ = ∇V^{⊤}ẋ = ∥∇V ∥ ∥ẋ∥ cos θ,

where θ is the angle between the vectors ∇V and ẋ. If ∥ẋ∥ = 1, then ∇V^{⊤}ẋ can be viewed as the rate of increase of V at x in the direction ẋ. Thus, V̇ < 0 if and only if, 90 deg < θ < 270 deg. In such a case the velocity vector ẋ(t) points in the direction of decreasing V. We illustrate the above discussion in Figure 7.6.

Our next theorem is very useful in numerical verification of stability; it is a variant of Theorem 7.1.

Theorem 7.2 A matrix A is asymptotically stable if and only if for any Q = C^{⊤}C, C ∈ ℝ^{p×n}, such that the pair (A, C) is observable, the solution P to the Lyapunov matrix equation A^{⊤}P + P A = −C^{⊤}C is positive definite.

Proof We first prove the necessity condition (⇒) for asymptotic stability. If A is asymptotically stable, that is, ℜλ_{i}(A) < 0, then we will show that

P = ∫_{0}^{∞} e^{A^{⊤}t}C^{⊤}Ce^{At}dt  (7.8)

is the symmetric positive definite solution to the Lyapunov equation. Indeed, the integrand of (7.8) is a sum of terms of the form

t^{k}e^{λ_{i}t},

where λ_{i} is the i-th eigenvalue of A. Since ℜλ_{i}(A) < 0, the integral in (7.8) exists because the terms in P do not diverge when t → ∞. Indeed, let −α_{i} = ℜλ_{i}, where α_{i} > 0, then

lim_{t→∞} t^{k}e^{−α_{i}t} = lim_{t→∞} \frac{t^{k}}{e^{α_{i}t}} ,

Differentiating the numerator and denominator of the above k times and using L’Hospital’s rule gives

lim_{t→∞} \frac{t^{k}}{e^{α_{i}t}} = lim_{t→∞} \frac{kt^{k−1}}{α_{i}e^{α_{i}t}}
= lim_{t→∞} \frac{k(k − 1)t^{k−2}}{α_{i}^{2}e^{α_{i}t}}
⋮
= lim_{t→∞} \frac{k!}{α_{i}^{k}e^{α_{i}t}}
= 0.  (7.9)

Thus, by (7.9), the formula for P given by (7.8) is well defined. Next, note that P = P^{⊤}. The matrix P is also positive definite because the pair (A, C) is observable. Substituting the expression for P into the Lyapunov equation and performing manipulations yields

A^{⊤}P + P A = ∫_{0}^{∞} A^{⊤}e^{A^{⊤}t}C^{⊤}Ce^{At}dt + ∫_{0}^{∞} e^{A^{⊤}t}C^{⊤}Ce^{At}Adt
= ∫_{0}^{∞} \frac{d}{dt} (e^{A^{⊤}t}C^{⊤}Ce^{At}) dt
= e^{A^{⊤}t}C^{⊤}Ce^{At}|_{0}^{∞}
= −C^{⊤}C.

The sufficiency part (⇐) can be proven using the arguments of the proof of the sufficiency part of the Lyapunov theorem and noting that the observability of the pair (A, C) implies that x(t)^{⊤}C^{⊤}Cx(t) is not identically zero along any nonzero solution of ẋ(t) = Ax(t).

□

We now show that if A is asymptotically stable, then the Lyapunov equation (7.7) has a unique solution for every Q. Uniqueness can be proven in many ways. We use the Kronecker product in our proof. Using the results of Section A.5, we represent the Lyapunov equation (7.7) as

(I_{n} ⊗ A^{⊤} + A^{⊤} ⊗ I_{n}) vec(P) = −vec(Q).  (7.10)

The condition for a solution s(P) of the above equation to be unique is

λ_{i}(A^{⊤}) + λ_{j}(A) ≠ 0, i, j = 1, 2, …, n.

Because λ_{i}(A^{⊤}) = λ_{i}(A), and A is asymptotically stable by assumption, the condition λ_{i}(A^{⊤}) + λ_{j}(A) ≠ 0 is met, and hence the Lyapunov equation has a unique solution.

7.4 Evaluating Quadratic Indices

In this Section, we apply the Lyapunov theory to evaluate the quadratic indices

J_{r} = ∫_{0}^{∞} t^{r}x(t)^{⊤}Qx(t)dt, r = 0, 1, …

subject to

ẋ(t) = Ax(t), x(0) = x_{0},

where Q = C^{⊤}C such that the pair (A, C) is observable, and the matrix A is asymptotically stable. We reformulate the above problem as follows. Evaluate the quadratic performance indices

J_{r} = ∫_{0}^{∞} t^{r}y(t)^{⊤}y(t)dt, r = 0, 1, …

subject to

ẋ(t) = Ax(t), x(0) = x_{0},
y(t) = Cx(t),

where the pair (A, C) is observable, and the matrix A is asymptotically stable. To evaluate

J_{0} = ∫_{0}^{∞} y(t)^{⊤}y(t)dt = ∫_{0}^{∞} x(t)^{⊤}C^{⊤}Cx(t)dt = ∫_{0}^{∞} x(t)^{⊤}Qx(t)dt,

we note that

\frac{d}{dt} (x(t)^{⊤}P x(t)) = −x(t)^{⊤}Qx(t),  (7.11)

where P and Q = C^{⊤}C satisfy the Lyapunov matrix equation, A^{⊤}P + P A = −Q. Hence,

J_{0} = − ∫_{0}^{∞} \frac{d}{dt} (x(t)^{⊤}P x(t)) dt.  (7.12)

Integrating both sides of (7.11) with respect to t yields

J_{0} = − ∫_{0}^{∞} \frac{d}{dt} (x(t)^{⊤}P x(t)) dt
= − (x(t)^{⊤}P x(t))|_{0}^{∞}
= lim_{t→∞} (−x(t)^{⊤}P x(t)) + x(0)^{⊤}P x(0)
= x(0)^{⊤}P x(0)

because by assumption A is asymptotically stable and therefore, lim_{t→∞} x(t) = 0 for all x(0). Furthermore, P = P^{⊤} > 0, and thus J_{0} > 0 for all x(0) ≠ 0.

Repeating the above arguments leads to a similar expression for J_{1}. First observe that

\frac{d}{dt} (tx(t)^{⊤}P x(t)) = x(t)^{⊤}P x(t) + t\frac{d}{dt} (x(t)^{⊤}P x(t))
= x(t)^{⊤}P x(t) − tx(t)^{⊤}Qx(t),

where P satisfies the Lyapunov matrix equation, A^{⊤}P + P A = −Q. Hence,

J_{1} = ∫_{0}^{∞} tx(t)^{⊤}Qx(t)dt
= ∫_{0}^{∞} x(t)^{⊤}P x(t)dt − ∫_{0}^{∞} \frac{d}{dt} (tx(t)^{⊤}P x(t)) dt.  (7.13)

Because P = P^{⊤} > 0, we can solve

A^{⊤}P_{1} + P_{1}A = −P

for P_{1} = P _{1}^{⊤} > 0. The above equation can also be represented as

\frac{d}{dt} (x(t)^{⊤}P_{1}x(t)) = −x(t)^{⊤}P x(t).  (7.14)

Substituting (7.14) into (7.13) yields

J_{1} = − ∫_{0}^{∞} \frac{d}{dt} (x(t)^{⊤}P_{1}x(t)) dt − (tx(t)^{⊤}P x(t))|_{0}^{∞}
= lim_{t→∞} (−x(t)^{⊤}P_{1}x(t)) + x(0)^{⊤}P_{1}x(0) − 0 + 0
= x(0)^{⊤}P_{1}x(0).

Using similar arguments, we can evaluate J_{r}, r > 1, which involves successive solutions of the Lyapunov matrix equations.

In many control problems a dynamical system model is given in terms of a transfer function rather than in terms of a system of differential equations. We discuss a method of converting a given rational function into a system of first order differential equations. After this process is completed, we use the Lyapunov theory to evaluate a performance index of interest. Suppose we are given a rational function of the form

E(s) = \frac{b_{n−1}s^{n−1} + ⋯ + b_{1}s + b_{0}}{s^{n} + a_{n−1}s^{n−1} + ⋯ + a_{1}s + a_{0}} ,  (7.15)

where E(s) = L(e(t)) is the Laplace transform of e(t). We show that e(t) can be generated as the solution of the differential equation

\frac{d^{n}e(t)}{dt^{n}} + a_{n−1} \frac{d^{n−1}e(t)}{dt^{n−1}} + ⋯ + a_{1} \frac{de(t)}{dt} + a_{0}e(t) = 0,  (7.16)

subject to a set of initial conditions that we derive next. We follow the method of Power and Simpson [278, pages 220–221]. We first take the Laplace transform of the differential equation and then rearrange the resulting terms in an appropriate matrix equation. Recall that

L (\frac{d^{i}e(t)}{dt^{i}}) = s^{i}E(s) − s^{i−1}e(0) − s^{i−2} \frac{de(0)}{dt} − ⋯ − \frac{d^{i−1}e(0)}{dt^{i−1}}.  (7.17)

Note that (7.16) in the s-domain for zero initial conditions takes the form

(s^{n} + a_{n−1}s^{n−1} + ⋯ + a_{1}s + a_{0}) E(s) = 0.  (7.18)

We apply (7.17) to (7.16), take into account the above relation, and compare coefficients of like powers to obtain

[ \begin{matrix}a_{1} & a_{2} & ⋯ & a_{n−1} & 1 \\ a_{2} & a_{3} & ⋯ & 1 & 0 \\ ⋮ & ⋮ & & ⋮ & ⋮ \\ a_{n−1} & 1 & ⋯ & 0 & 0 \\ 1 & 0 & ⋯ & 0 & 0 \end{matrix} ] [ e(0) ; \frac{de(0)}{dt} ; ⋮ ; \frac{d^{n−2}e(0)}{dt^{n−2}} ; \frac{d^{n−1}e(0)}{dt^{n−1}} ] = [ b_{0} ; b_{1} ; ⋮ ; b_{n−2} ; b_{n−1} ] .  (7.19)

The coefficient matrix in the above equation is nonsingular. Thus, there is a unique solution to this equation, that is, we can uniquely determine the initial conditions. We define the state vector

x = [ e \frac{de}{dt} ⋯ \frac{d^{n−1}e}{dt^{n−1}} ]^{⊤} ,

and represent the differential equation (7.16) as

ẋ = Ax,

where

A = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & 1 & ⋯ & 0 & 0 \\ ⋮ & ⋮ & ⋮ & & ⋮ & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & 1 \\ −a_{0} & −a_{1} & −a_{2} & ⋯ & −a_{n−2} & −a_{n−1} \end{matrix} ] .

The vector of initial conditions is

x(0) = [ e(0) ; \frac{de(0)}{dt} ; ⋮ ; \frac{d^{n−2}e(0)}{dt^{n−2}} ; \frac{d^{n−1}e(0)}{dt^{n−1}} ] = [ \begin{matrix}a_{1} & a_{2} & ⋯ & a_{n−1} & 1 \\ a_{2} & a_{3} & ⋯ & 1 & 0 \\ ⋮ & ⋮ & & ⋮ & ⋮ \\ a_{n−1} & 1 & ⋯ & 0 & 0 \\ 1 & 0 & ⋯ & 0 & 0 \end{matrix} ]^{−1} [ b_{0} ; b_{1} ; ⋮ ; b_{n−2} ; b_{n−1} ] .

Note that e = [ 1 0 ⋯ 0 0 ] x.

The above procedure for converting a rational function description of a process into its equivalent state-space representation shows a relation between the zeros of the given rational function and the initial conditions of the associated state-space realization.

We use the above method of constructing a state-space realization of a given rational function in the following example that involves evaluation of a quadratic index.

Example 7.5 Consider a block diagram of a space telescope control system model shown in Figure 7.7. This system comes from Dorf and Bishop [85, page 333]. The values of the parameters are K_{1} = 0.5, and K_{1}K_{2}K_{p} = 2.5. We use the Lyapunov theory to select the gain K_{3} that minimizes the effect of the disturbance u(t) = L^{−1}(U(s)) in the sense of the integral of the square error (ISE) criterion. In other words, we wish to find K_{3} that leads to minimization of the performance index

J_{0} = ∫_{0}^{∞} c(t)^{2}dt,

where c(t) = L^{−1}(C(s)). We assume a unit step disturbance and R(s) = 0.

We first find a rational function that models C(s). From the block diagram shown in Figure 7.7, we obtain

C(s) = U(s) − \frac{K_{2}}{s} \frac{K_{1}}{s + K_{1}K_{3}} K_{p}C(s).

Figure 7.7: Block diagram of the optimized control system model of Example 7.5.

Hence

C(s) = \frac{s + 0.5K_{3}}{s^{2} + 0.5K_{3}s + 2.5}.

The next step is to determine the state-space representation corresponding to C(s). We use the method given before this example. We first form the differential equation for c(t),

\frac{d^{2}c(t)}{dt^{2}} + 0.5K_{3} \frac{dc(t)}{dt} + 2.5c(t) = 0

with the equation for the initial conditions

[ \begin{matrix}0.5K_{3} & 1 \\ 1 & 0 \end{matrix} ] [ c(0)_{\frac{dc(0)}{dt}} ] = [ \begin{matrix}0.5K_{3} \\ 1 \end{matrix} ]

obtained by applying (7.19). Solving the above equation yields

[ c(0)_{\frac{dc(0)}{dt}} ] = [ \begin{matrix}1 \\ 0 \end{matrix} ] .

Let x_{1} = c, x_{2} = \frac{dc}{dt}. The differential equation in vector-matrix form corresponding to C(s) is

\begin{matrix}ẋ(t) & = & [ \begin{matrix}0 & 1 \\ −2.5 & −0.5K_{3} \end{matrix} ] x(t), x(0) = [ \begin{matrix}1 \\ 0 \end{matrix} ] \\ c(t) & = & cx(t), \end{matrix} \}

where c = [ 1 0 ]. We evaluate the performance index J_{0},

J_{0} = ∫_{0}^{∞} c(t)^{2}dt = ∫_{0}^{∞} x(t)^{⊤} [ \begin{matrix}1 & 0 \\ 0 & 0 \end{matrix} ] x(t)dt = ∫_{0}^{∞} x(t)^{⊤}c^{⊤}c x(t)dt,

The pair (A, c) is observable. Therefore, we can solve the Lyapunov equation

A^{⊤}P + P A = −c^{⊤}c

for P to obtain

P = [ \frac{K_{3}}{10} + \frac{1}{K_{3}}  \frac{1}{5}_{\frac{1}{5}  \frac{2}{5K_{3}}} ] .

Hence

J_{0} = x(0)^{⊤}P x(0) = \frac{K_{3}}{10} + \frac{1}{K_{3}} .

Differentiating J_{0} and equating the result to zero gives

\frac{dJ_{0}}{dK_{3}} = \frac{1}{10} − \frac{1}{K_{3}^{2}} = 0.

Selecting

K_{3} = \sqrt{10},

yields a positive definite P. Note that because

\frac{d^{2}J_{0}}{dK_{3}^{2}} |_{K_{3}=\sqrt{10}} = \frac{2}{K_{3}^{3}} |_{K_{3}=\sqrt{10}} > 0,

that is, the second order sufficiency condition for the minimum is satisfied, we conclude that K_{3} = \sqrt{10} ≈ 3.16 minimizes J_{0}.

Next, we wish to determine the gain K_{3} that leads to minimization of the effect of the disturbance u(t) in the sense of the integral of time multiplied by the squared error (ITSE) criterion. That is, we wish to select K_{3} to minimize the performance index

J_{1} = ∫_{0}^{∞} tc(t)^{2}dt.

We first solve the Lyapunov equation,

A^{⊤}P_{1} + P_{1}A = −P ,

to get

P_{1} = [ \begin{matrix}\frac{2}{K_{3}^{2}} + \frac{K_{3}^{2}}{100} & \frac{1}{5} (\frac{1}{K_{3}} + \frac{K_{3}}{10} ) \\ \frac{1}{5} (\frac{1}{K_{3}} + \frac{K_{3}}{10} ) & \frac{4}{5K_{3}^{2}} + \frac{1}{25} \end{matrix} ] .

Hence

J_{1} = x(0)^{⊤}P_{1}x(0) = \frac{2}{K_{3}^{2}} + \frac{K_{3}^{2}}{100}.

Differentiating J_{1} and equating the result to zero gives

\frac{dJ_{1}}{dK_{3}} = −\frac{4}{K_{3}^{3}} + \frac{K_{3}}{50} = 0.

We select K_{3} = \sqrt[4]{200} to keep P_{1} positive definite. For this choice of K_{3}, the second order sufficiency condition for the minimum of J_{1} is satisfied. Thus, K_{3} ≈ 3.76 minimizes J_{1}.

7.5 Discrete-Time Lyapunov Equation

In this Section, we use the Lyapunov approach to the stability analysis of the discrete linear time-invariant (LTI) systems modeled by

x[k + 1] = Ax[k], x[k_{0}] = x_{0}.  (7.20)

By analogy with the continuous-time case, consider a quadratic positive definite form,

V (x[k]) = x[k]^{⊤}P x[k].

We evaluate the first difference of V, defined ΔV = V (x[k+1])−V (x[k]), on the trajectories of (7.20) to obtain

ΔV (x[k]) = V (x[k + 1]) − V (x[k])
= x[k + 1]^{⊤}P x[k + 1] − x[k]^{⊤}P x[k]
= x[k]^{⊤}A^{⊤}P Ax[k] − x[k]^{⊤}P x[k]
= x[k]^{⊤} (A^{⊤}P A − P ) x[k],  (7.21)

where we replaced x[k + 1] in the above with Ax[k]. For the solutions of (7.20) to decrease the “energy function” V, at each k > k_{0}, it is necessary and sufficient that ΔV (x[k]) < 0, or equivalently, that for some positive definite Q,

ΔV (x[k]) = −x[k]^{⊤}Qx[k].  (7.22)

Combining (7.20) and (7.22), we obtain the so called discrete Lyapunov matrix equation,

A^{⊤}P A − P = −Q  (7.23)

Theorem 7.3 The matrix A has its eigenvalues in the open unit disk if and only if for any positive definite Q the solution P to (7.23) is positive definite.

Proof (⇒) We can use the Kronecker product, and in particular relation (A.62) to represent the discrete-time Lyapunov equation as

(I_{n^{2}} − A^{⊤} ⊗ A^{⊤}) vec(P) = vec(Q),  (7.24)

where ⊗ denotes the Kronecker product and vec(·) is the stacking operator. It is easy to see that the above equation has a unique solution if and only if

1 − λ_{i}λ_{j} ≠ 0, for all i, j = 1, 2, …, n,

where {λ_{i}} is the set of eigenvalues of A. By assumption, |λ_{i}| < 1, which is a sufficient condition for (7.24) and hence (7.23) to have a unique solution. The solution P can be expressed as

P = ∑_{k=0}^{∞} (A^{⊤})^{k} QA^{k}.  (7.25)

The above expression is well defined because all eigenvalues of A have magnitude (strictly) less than unity. Substituting (7.25) into (7.23) gives

A^{⊤}P A − P = A^{⊤} (∑_{k=0}^{∞} (A^{⊤})^{k} QA^{k}) A − ∑_{k=0}^{∞} (A^{⊤})^{k} QA^{k}
= ∑_{k=1}^{∞} (A^{⊤})^{k} QA^{k} − ∑_{k=1}^{∞} (A^{⊤})^{k} QA^{k} − Q
= −Q.

Thus, indeed (7.25) satisfies the discrete-time Lyapunov equation. Note that if Q is symmetric so is P. If Q is positive definite, so is P.

(⇐) Let λ_{i} be an eigenvalue of A and v_{i} be a corresponding eigenvector, that is, Av_{i} = λ_{i}v_{i}. Observe that λ_{i} and v_{i} may be complex. We denote by v_{i}^{∗} the complex conjugate transpose of v_{i}. Note that v_{i}^{∗}P v_{i} is a scalar. We show that its complex conjugate equals itself, which means that v_{i}^{∗}P v_{i} is real. Indeed,

\overline{v_{i}^{∗}P v_{i}} = v_{i}^{∗}P^{∗}v_{i} = v_{i}^{∗}P^{⊤}v_{i} = v_{i}^{∗}P v_{i},

where we used the fact that the complex conjugate transpose of a real matrix reduces to its transpose. Thus, v_{i}^{∗}P v_{i} is real for any complex v_{i}.

Premultiplying now (7.23) by v_{i}^{∗} and postmultiplying by v_{i} gives

v_{i}^{∗} (A^{⊤}P A − P ) v_{i} = −v_{i}^{∗}Qv_{i}.  (7.26)

The left-hand side of the above equality evaluates to

v_{i}^{∗}A^{⊤}P Av_{i} − v_{i}^{∗}P v_{i} = λ̄_{i}v_{i}^{∗}P v_{i}λ_{i} − v_{i}^{∗}P v_{i}
= (|λ_{i}|^{2} − 1) v_{i}^{∗}P v_{i}.

Comparing the above with the right-hand side of (7.26) yields

(1 − |λ_{i}|^{2}) v_{i}^{∗}P v_{i} = v_{i}^{∗}Qv_{i} > 0

because Q is positive definite. Since P is positive definite, we have to have |λ_{i}| < 1. The proof is complete.

□

We now concern ourselves with evaluating performance indices on the trajectories of (7.20). Specifically, we wish to evaluate

K_{r} = ∑_{k=0}^{∞} k^{r}x[k]^{⊤}Qx[k], r = 0, 1,  (7.27)

where Q = Q^{⊤} > 0 subject to (7.20). We assume that the matrix A has its eigenvalues in the open unit disk, that is, the matrix A is convergent. We first consider the case when r = 0. For this case, K_{r} becomes

K_{0} = ∑_{k=0}^{∞} x[k]^{⊤}Qx[k].  (7.28)

Because the matrix A is convergent, the solution P to (7.23) is positive definite. Using (7.21), (7.22) and (7.20), we write

x[k]^{⊤}Qx[k] = −x[k]^{⊤} (A^{⊤}P A − P ) x[k]
= x[k]^{⊤}P x[k] − x[k + 1]^{⊤}P x[k + 1]  (7.29)

Summing both sides of the above form k = 0 to k = ∞ and using the fact that x[∞] = 0 because the matrix A is convergent, we obtain

K_{0} = x[0]^{⊤}P x[0].  (7.30)

We now evaluate

K_{1} = ∑_{k=0}^{∞} kx[k]^{⊤}Qx[k],

subject to (7.20). To proceed, we multiply both sides of (7.29) by k to get

kx[k]^{⊤}Qx[k] = kx[k]^{⊤}P x[k] − kx[k + 1]^{⊤}P x[k + 1].

Summing the above from k = 0 to k = ∞ gives

K_{1} = x[1]^{⊤}P x[1] − x[2]^{⊤}P x[2] + 2x[2]^{⊤}P x[2] − 2x[3]^{⊤}P x[3] + ⋯
= ∑_{k=0}^{∞} x[k]^{⊤}P x[k] − x[0]^{⊤}P x[0].  (7.31)

Using the method we employed to find the value of K_{0}, we evaluate the first term on the right-hand side of (7.31) to get

∑_{k=0}^{∞} x[k]^{⊤}P x[k] = x[0]^{⊤}P_{1}x[0],

where P_{1} satisfies

A^{⊤}P_{1}A − P_{1} = −P .

Taking the above into account, we represent (7.31) as

K_{1} = x[0]^{⊤} (P_{1} − P) x[0].

Proceeding in a similar manner, we can find expressions for K_{r}, where r > 1.

7.6 Constructing Robust Linear Controllers

In this section, we use the Lyapunov theorem to construct robust state-feedback controllers. One of the basic issues in the control of dynamical systems is the effect of uncertainties, or neglected nonlinearities, on the stability of the closed-loop system. A controller is viewed robust if it maintains stability for all uncertainties in an expected range. We consider a class of dynamical systems modeled by

ẋ(t) = Ax(t) + Bu(t) + f(t, x(t)) + Bh(t, x(t), u(t)),  (7.32)

where x(t) ∈ ℝ^{n}, u(t) ∈ ℝ^{m}, and the functions h and f model uncertainties, or nonlinearities, in the system. We refer to h as the matched uncertainty because it affects the system dynamics via the input matrix B in the same fashion as the input u does. In other words, the uncertainty h matches the system input u. The vector f models the unmatched uncertainty. We assume that the uncertain elements h and f satisfy the following norm bounds:

1. ∥h(t, x, u)∥ ≤ γ_{h}∥u∥ + α_{h}∥x∥,

2. ∥f(t, x)∥ ≤ α_{f}∥x∥,

where γ_{h}, α_{h}, and α_{f} are known nonnegative constants. We further assume that the matrix A is asymptotically stable. If this is not the case, we apply a preliminary state-feedback controller u = −Kx such that A − BK is asymptotically stable. Such a feedback exists provided that the pair (A, B) is stabilizable. Our goal is to construct a linear state-feedback controller that would make the closed-loop system asymptotically stable for arbitrary f and h that satisfy the above norm bounds.

Theorem 7.4 Suppose that A is asymptotically stable, and P = P^{⊤} > 0 is the solution to the Lyapunov matrix equation A^{⊤}P + P A = −2Q for some Q = Q^{⊤} > 0. Suppose also that

α_{f} < \frac{λ_{min}(Q)}{λ_{max}(P)} and γ_{h} < 1.

Then, the state-feedback controller u = −γB^{⊤}P x, where

γ > \frac{α_{h}^{2}}{4 (λ_{min}(Q) − α_{f}λ_{max}(P)) (1 − γ_{h})}

stabilizes the uncertain system model (7.32) for arbitrary f and h that satisfy the norm bounds.

Proof The time derivative of the positive definite function V = x^{⊤}P x evaluated on a generic trajectory of the closed-loop system is

V̇ = 2x^{⊤}P ẋ
= −2x^{⊤}Qx − 2γx^{⊤}P BB^{⊤}P x + 2x^{⊤}P f + 2x^{⊤}P Bh.

Our goal is to determine γ̃ > 0 such that if γ > γ̃, then V̇ < 0, which in turn implies the asymptotic stability of the closed-loop system. To proceed further, recall that if Q is a symmetric matrix, then

λ_{min}(Q)∥x∥^{2} ≤ x^{⊤}Qx ≤ λ_{max}(Q)∥x∥^{2},

and therefore

−x^{⊤}Qx ≤ −λ_{min}(Q)∥x∥^{2}.

Furthermore,

x^{⊤}P BB^{⊤}P x = x^{⊤}P B (x^{⊤}P B)^{⊤} = ∥x^{⊤}P B∥^{2}.

For a symmetric positive definite matrix P, its induced 2-norm is

∥P ∥ = λ_{max}(P).

Taking the above relations into account, using the norm bounds on the uncertain elements, and performing some manipulations yields

V̇ ≤ −2λ_{min}(Q)∥x∥^{2} − 2γ∥x^{⊤}P B∥^{2} + 2α_{f}λ_{max}(P)∥x∥^{2}
+2∥x^{⊤}P B∥ (α_{h}∥x∥ + γ_{h}γ∥x^{⊤}P B∥) .  (7.33)

Let

Q̃ = [ \begin{matrix}λ_{min}(Q) − α_{f}λ_{max}(P) & −\frac{α_{h}}{2} \\ −\frac{α_{h}}{2} & γ(1 − γ_{h}) \end{matrix} ] ∈ ℝ^{2×2}.  (7.34)

Then, we can represent (7.33) as

V̇ ≤ −2 [ ∥x∥ ∥x^{⊤}P B∥ ] Q̃ [ \begin{matrix}∥x∥ \\ ∥x^{⊤}P B∥ \end{matrix} ] .  (7.35)

By Theorem A.6 on page 969, for V̇ to be negative it is enough that the leading principal minors of the matrix Q̃ given by (7.34) be positive. The first-order leading principal minor is positive by assumption. For the second-order leading principal minor to be positive it is sufficient that γ_{h} < 1 and

γ > \frac{α_{h}^{2}}{4 (λ_{min}(Q) − α_{f}λ_{max}(P)) (1 − γ_{h})}.

The proof is now complete.

□

Note that if γ_{h} = 0, then the closed-loop system is asymptotically stable if

α_{f} < \frac{λ_{min}(Q)}{λ_{max}(P)} and γ > \frac{α_{h}^{2}}{4 (λ_{min}(Q) − α_{f}λ_{max}(P))}.

Example 7.6 Consider a dynamic system model

ẋ = Ax + b(u + h(x, u))
= [ \begin{matrix}−2 & 0 \\ 0 & −1 \end{matrix} ] x + [ \begin{matrix}1 \\ 1 \end{matrix} ] (u + h(x, u)).

Let P be the solution of the matrix Lyapunov equation A^{⊤}P + P A = −2I_{2}. We design a linear state-feedback controller u = −γb^{⊤}P x so that the origin of the closed-loop system is uniformly asymptotically stable for any h(x, u) such that

|h(x, u)| ≤ 4∥x∥ + \frac{1}{2}|u|.  (7.36)

In particular, we will find a bound γ̃ > 0 so that for any γ > γ̃ the closed-loop system is uniformly asymptotically stable in the face of uncertainties satisfying the norm bound (7.36).

Solving the Lyapunov equation A^{⊤}P + P A = −2I_{2} yields

P = [ \begin{matrix}1/2 & 0 \\ 0 & 1 \end{matrix} ] .

Hence,

u = −γb^{⊤}P x = −γ (\frac{1}{2}x_{1} + x_{2}) .

We obtain a bound on γ by evaluating \frac{d}{dt} (x^{⊤}P x) on the trajectories of the closed-loop system. First, we evaluate

\frac{d}{dt} (x^{⊤}P x) = 2x^{⊤}P ẋ
= 2x^{⊤}P (Ax + b(u + h(x, u)))
= −2x^{⊤}x − 2γx^{⊤}P bb^{⊤}P x + 2x^{⊤}P bh.

Taking the norms and performing manipulations yields

\frac{d}{dt} (x^{⊤}P x) ≤ −2x^{⊤}x − 2γ |x^{⊤}P b|^{2} + 2 |x^{⊤}P b| (4∥x∥ + \frac{1}{2}γ |x^{⊤}P b|)
= −2∥x∥^{2} − γ |x^{⊤}P b|^{2} + 8 |x^{⊤}P b| ∥x∥
= −2 [ ∥x∥ |x^{⊤}P b| ] [ \begin{matrix}1 & −2 \\ −2 & γ/2 \end{matrix} ] [ \begin{matrix}∥x∥ \\ | & x & ⊤ & P & b & | \end{matrix} ] .

For \frac{d}{dt} (x^{⊤}P x) to be negative definite it is enough that the matrix

[ \begin{matrix}1 & −2 \\ −2 & γ/2 \end{matrix} ]

be positive definite. This is the case whenever

γ > γ̃ = 8.

7.7 Hurwitz and Routh Stability Criteria

In this section, we present two tests for determining if the zeros of an n-th degree polynomial are located in the open left-half complex plane. In deriving these tests, we use the Lyapunov matrix equation given by (7.7). We consider the n-th degree polynomial

p(s) = a_{n}s^{n} + a_{n−1}s^{n−1} + ⋯ + a_{1}s + a_{0}  (7.37)

with real coefficients. Associated with the polynomial p(s) is the n × n real Hurwitz matrix,

H = [ \begin{matrix}a_{n−1} & a_{n−3} & a_{n−5} & a_{n−7} & ⋯ \\ a_{n} & a_{n−2} & a_{n−4} & a_{n−6} & ⋯ \\ 0 & a_{n−1} & a_{n−3} & a_{n−5} & ⋯ \\ 0 & a_{n} & a_{n−2} & a_{n−4} & ⋯ \\ ⋮ & & & ⋮ & ... \end{matrix} ] .  (7.38)

In our further discussion we assume, without loss of generality, that a_{n} = 1. The leading principal minors of the Hurwitz matrix are

H_{1} = a_{n−1}, H_{2} = det [ \begin{matrix}a_{n−1} & a_{n−3} \\ 1 & a_{n−2} \end{matrix} ] , H_{3} = det [ \begin{matrix}a_{n−1} & a_{n−3} & a_{n−5} \\ 1 & a_{n−2} & a_{n−4} \\ 0 & a_{n−1} & a_{n−3} \end{matrix} ] , …, H_{n} = det H.

(7.39)

We next define

b_{1} = H_{1}, b_{2} = \frac{H_{2}}{H_{1}} , b_{3} = \frac{H_{3}}{H_{2}H_{1}} ,

and

b_{k} = \frac{H_{k−3}H_{k}}{H_{k−2}H_{k−1}} for k = 4, 5, …, n.

Thus, for example, b_{4} = \frac{H_{1}H_{4}}{H_{2}H_{3}} . Let

B = [ \begin{matrix}0 & 1 & 0 & 0 & ⋯ & 0 & 0 & 0 \\ −b_{n} & 0 & 1 & 0 & ⋯ & 0 & 0 & 0 \\ 0 & −b_{n−1} & 0 & 1 & ⋯ & 0 & 0 & 0 \\ ⋮ & & & & & & & ⋮ \\ 0 & 0 & 0 & 0 & ⋯ & −b_{3} & 0 & 1 \\ 0 & 0 & 0 & 0 & ⋯ & 0 & −b_{2} & −b_{1} \end{matrix} ] ∈ ℝ^{n×n}.

For example, for n = 4, we have

B = [ \begin{matrix}0 & 1 & 0 & 0 \\ −b_{4} & 0 & 1 & 0 \\ 0 & −b_{3} & 0 & 1 \\ 0 & 0 & −b_{2} & −b_{1} \end{matrix} ] .

Let

A = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & 1 & ⋯ & 0 & 0 \\ ⋮ & & & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & 1 \\ −a_{0} & −a_{1} & −a_{2} & ⋯ & −a_{n−2} & −a_{n−1} \end{matrix} ] ∈ ℝ^{n×n}.

Lemma 7.1 The matrices B and A are similar, that is, there is a nonsingular matrix T ∈ ℝ^{n×n} such that

B = T^{−1}AT .  (7.40)

Proof The similarity matrix is lower triangular and has the form

T = [ \begin{matrix}1 & 0 & 0 & 0 & 0 & 0 & ⋯ & 0 \\ 0 & 1 & 0 & 0 & 0 & 0 & ⋯ & 0 \\ t_{31} & 0 & 1 & 0 & 0 & 0 & ⋯ & 0 \\ 0 & t_{42} & 0 & 1 & 0 & 0 & ⋯ & 0 \\ t_{51} & 0 & t_{53} & 0 & 1 & 0 & ⋯ & 0 \\ 0 & t_{62} & 0 & t_{64} & 0 & 1 & ⋯ & 0 \\ ⋮ & & & & & & & ⋮ \end{matrix} ] .  (7.41)

To determine the nonfixed coefficients t_{ij} of the matrix T, we represent (7.40) as

T B = AT ,  (7.42)

and compare both sides of (7.42) to determine the coefficients t_{ij}. We obtain, for example, t_{31} = −b_{n}, t_{42} = −b_{n} − b_{n−1}, etc.

□

Let P be an n × n diagonal matrix defined as

P = diag [ b_{1}b_{2} × ⋯ × b_{n} b_{1}b_{2} × ⋯ × b_{n−1} ⋯ b_{1}b_{2} b_{1} ] ,  (7.43)

and Q be an n × n diagonal matrix defined as

Q = diag [ 0 0 ⋯ 0 2b_{1}^{2} ] .

We represent the matrix Q as Q = c^{⊤}c, where

c = [ 0 0 ⋯ 0 \sqrt{2}b_{1} ] .

Note that the pair (B, c) is observable. The matrices B, P, and Q satisfy the Lyapunov matrix equation,

B^{⊤}P + P B = −Q.  (7.44)

By Theorem 7.2, on page 325, applied to (7.44), the matrix B is asymptotically stable if and only if the symmetric matrix P is positive definite. By Theorem A.6, the matrix P is positive definite if and only if its leading principal minors are positive. Using the above results we can prove the following theorem of Hurwitz:

Theorem 7.5 (Hurwitz) The real polynomial p(s) of degree n with positive coefficients a_{i}, has its zeros in the open left-half plane if and only if the leading principal minors of the Hurwitz matrix are positive.

Proof The polynomial p(s) has its zeros in the open left plane if and only if the matrix B is asymptotically stable. This in turn is true if and only if the diagonal elements of P, given by (7.43), are positive. We have

b_{1} = H_{1}, b_{1}b_{2} = H_{2},

and for k = 3, …, n,

b_{1}b_{2} × ⋯ × b_{k} = \frac{H_{k}}{H_{k−2}} .

Thus, the diagonal elements of P are all positive, if and only if the leading principal minors of the Hurwitz matrix are positive, which completes the proof.

□

We now prove the Routh necessary and sufficient condition for the real polynomial (7.37) to have its zeros in the open left-half plane. We first use the polynomial coefficients to form the following (n + 1)-row Routh array:

s^{n} row a_{n} a_{n−2} a_{n−4} a_{n−6} ⋯
s^{n−1} row a_{n−1} a_{n−3} a_{n−5} a_{n−7} ⋯
s^{n−2} row r_{21} r_{22} r_{23} r_{24} ⋯
s^{n−3} row r_{31} r_{32} r_{33} r_{34} ⋯
⋮ ⋮ ⋮
s^{0}  row r_{n1}

(7.45)

The elements that comprise the s^{n} and s^{n−1} rows are appropriately ordered coefficients of p(s). The elements of the s^{n−2} row are formed from its preceding two rows as follows,

r_{21} = −\frac{1}{a_{n−1}} det [ \begin{matrix}a_{n} & a_{n−2} \\ a_{n−1} & a_{n−3} \end{matrix} ] = \frac{a_{n−1}a_{n−2} − a_{n}a_{n−3}}{a_{n−1}}
r_{22} = −\frac{1}{a_{n−1}} det [ \begin{matrix}a_{n} & a_{n−4} \\ a_{n−1} & a_{n−5} \end{matrix} ] = \frac{a_{n−1}a_{n−4} − a_{n}a_{n−5}}{a_{n−1}}
r_{23} = −\frac{1}{a_{n−1}} det [ \begin{matrix}a_{n} & a_{n−6} \\ a_{n−1} & a_{n−7} \end{matrix} ] = \frac{a_{n−1}a_{n−6} − a_{n}a_{n−7}}{a_{n−1}}
⋮  ⋮

The elements of the s^{n−3} row of the Routh array are generated using its preceding two rows,

r_{31} = −\frac{1}{r_{21}} det [ \begin{matrix}a_{n−1} & a_{n−3} \\ r_{21} & r_{22} \end{matrix} ] = \frac{r_{21}a_{n−3} − r_{22}a_{n−1}}{r_{21}}
r_{32} = −\frac{1}{r_{21}} det [ \begin{matrix}a_{n−1} & a_{n−5} \\ r_{21} & r_{23} \end{matrix} ] = \frac{r_{21}a_{n−5} − r_{23}a_{n−1}}{r_{21}}
r_{33} = −\frac{1}{r_{21}} det [ \begin{matrix}a_{n−1} & a_{n−7} \\ r_{21} & r_{24} \end{matrix} ] = \frac{r_{21}a_{n−7} − r_{24}a_{n−1}}{r_{21}}
⋮  ⋮

Subsequent rows of the array are formed from their preceding rows in an analogous fashion. Undefined positions can be filled with zeros. The last two rows of the array will have one term each. We now state the stability test that in some texts (for example, in [283] or [353]) is also called the Routh-Hurwitz criterion.

Theorem 7.6 (Routh) The zeros of the polynomial p(s) = a_{n}s^{n} +a_{n−1}s^{n−1} +⋯+a_{1}s+a_{0}, whose coefficients are all non-zero and have the same sign, are all in the open left-half plane if and only if the elements of the first column of the associated Routh array all have the same sign and are nonzero.

Proof Without loss of generality, we assume that the polynomial coefficients are all positive. If they are negative, we can multiply both sides of the equation p(s) = 0 by −1 thus making the coefficients positive. We then perform elementary row operations on the associated Hurwitz matrix, H. These operations are equivalent to premultiplying the Hurwitz matrix by lower triangular matrices with ones on the diagonal. For example,

[ \begin{matrix}1 & 0 & 0 & 0 & ⋯ \\ −a_{n}/a_{n−1} & 1 & 0 & 0 & ⋯ \\ 0 & 0 & 1 & 0 & ⋯ \\ ⋮ & & & ... & ⋮ \\ 0 & 0 & 0 & 0 & 1 \end{matrix} ] [ \begin{matrix}a_{n−1} & a_{n−3} & a_{n−5} & a_{n−7} \\ a_{n} & a_{n−2} & a_{n−4} & a_{n−6} \\ 0 & a_{n−1} & a_{n−3} & a_{n−5} \\ 0 & a_{n} & a_{n−2} & a_{n−4} \\ ⋮ & & ⋮ & ... \end{matrix} ] = [ \begin{matrix}a_{n−1} & a_{n−3} & a_{n−5} & a_{n−7} \\ 0 & r_{21} & r_{22} & r_{23} \\ 0 & a_{n−1} & a_{n−3} & a_{n−5} \\ 0 & a_{n} & a_{n−2} & a_{n−4} \\ ⋮ & & ⋮ & ... \end{matrix} ] .

Let U be the product of the matrices corresponding to the above described elementary operations on H. Then,

U = [ \begin{matrix}1 & 0 & ⋯ & 0 \\ u_{12} & 1 & ⋯ & 0 \\ ⋮ & & ... & ⋮ \\ u_{n1} & u_{n2} & ⋯ & 1 \end{matrix} ] ,

where, for example, u_{12} = −a_{n}/a_{n−1}. Premultiplying the Hurwitz matrix (7.38) by the matrix U given above yields

UH = [ \begin{matrix}a_{n−1} & a_{n−3} & a_{n−5} & ⋯ & ⋯ \\ 0 & r_{21} & r_{22} & ⋯ & r_{2,n−1} \\ 0 & 0 & r_{31} & ⋯ & r_{3,n−2} \\ ⋮ & & ⋮ & ... & ⋮ \\ 0 & 0 & 0 & ⋯ & r_{n1} \end{matrix} ] .  (7.46)

Because the matrix U is lower triangular with the ones on the diagonal, the leading principal minors of H and UH are the same, where

H_{1} = a_{n−1}, H_{2} = a_{n−1}r_{21}, …, H_{i} = a_{n−1}r_{21} × ⋯ × r_{i1}, i = 1, 2, …, n.

Hence,

a_{n−1} = H_{1}, and r_{i1} = \frac{H_{i}}{H_{i−1}} for i = 2, …, n.

By the Hurwitz theorem, the zeros of the polynomial p(s) are in the open left-half plane if and only if the coefficients r_{i1}, i = 1, 2 …, n, which are the elements of the first column of the Routh array, are all positive. The proof is complete.

□

It is easy to verify that a necessary and sufficient condition for the first- and secondorder polynomials to have their zeros in the open left-half plane is that all their coefficients be nonzero and have the same sign. However, this condition is only necessary and not sufficient for third and higher order polynomials to have all their zeros in the open left-half plane.

It is possible to extend the above Routh stability criterion so that it can be used to find the number of polynomial zeros with positive real parts. The number of zeros of p(s) with positive real parts is equal to the number of sign changes in the first column of the Routh array. For a proof of this statement using the law of inertia of quadratic forms see, for example, Kaczorek [164, Section 8.4.3].

Example 7.7 We use the Routh stability criterion to analyze stability of the linearized model of the flyball governor. The governor’s linearized model is given by (4.7) on page 193. The model’s characteristic equation is

det [ \begin{matrix}s & −1 & 0 \\ \frac{g sin^{2} x_{1e}}{cos x_{1e}} & s + \frac{b}{m} & −\frac{2g sin x_{1e}}{x_{3e}} \\ \frac{κ}{I} sin x_{1e} & 0 & s \end{matrix} ] = s^{3} + \frac{b}{m}s^{2} + \frac{g sin^{2} x_{1e}}{cos x_{1e}} s + \frac{2gκ sin^{2} x_{1e}}{Ix_{3e}}
= s^{3} + a_{2}s^{2} + a_{1}s + a_{0}.  (7.47)

The coefficients of the above third order polynomial are all positive. Therefore, the necessary condition for stability is satisfied. We next construct the Routh array for (7.47) to get,

\begin{matrix}s^{3} & 1 & a_{1} \\ s^{2} & a_{2} & a_{0} \\ s^{1} & \frac{1}{a_{2}} (a_{2}a_{1} − a_{0}) \\ s^{0} & a_{0} \end{matrix}  (7.48)

Because the characteristic polynomial coefficients are all positive, the linearized model is asymptotically stable if and only if

a_{2}a_{1} − a_{0} = \frac{b}{m} \frac{g sin^{2} x_{1e}}{cos x_{1e}} − \frac{2gκ sin^{2} x_{1e}}{Ix_{3e}} > 0.  (7.49)

Let

ν = \frac{x_{3e}}{2κ cos x_{1e}} .

Using the above notation, we express (7.49) as

\frac{bI}{m} ν > 1.  (7.50)

It follows from condition (7.50) that

1. an increase of the mass m of the weights of the governor has a deteriorating effect on the governor’s stability;

2. a decrease of the friction coefficient b has also a harmful effect on stability;

3. a decrease of the moment of inertia I of the flywheel harms stability;

4. a decrease of the coefficient ν harms stability.

The above deductions are attributed by Pontryagin [276, page 219] to the Russian engineer Vyshnegradskiy. Vyshnegradskiy used these observations to propose, in 1876, design rules for governors to regulate engines without engine speeds fluctuating.

Example 7.8 This example comes from Chen [55, p. 139]. Its purpose is to show that in an LTV system, ẋ = A(t)x, even if the eigenvalues of A(t) are defined at every t and in the open left-hand complex plane, the system does not have to be stable; in fact it may be unstable. That is, the concept of eigenvalues is not useful in the stability investigation in LTV systems. The LTV system we consider is modeled by

ẋ = A(t)x = [ \begin{matrix}−1 & e^{2t} \\ 0 & −1 \end{matrix} ] x.

It is easy to see that the matrix A(t) has a repeated eigenvalue at −1 for all t. In the LTI case this would imply asymptotic stability. One can verify that the state transition matrix, Φ(t, 0), for the above LTV system has the form

Φ(t, 0) = [ \begin{matrix}e^{−t} & 0.5 (e^{t} − e^{−t}) \\ 0 & e^{−t} \end{matrix} ] .

The (1, 2)-th element of the state transition matrix is unbounded, which means that the system is unstable!

7.8 Stability of Nonlinear Systems

In this section, we are concerned with stability analysis of nonlinear time-varying dynamical system models described by (7.1), that is, models of the form

ẋ(t) = f(t, x(t)), x(t_{0}) = x_{0},

where x(t) ∈ ℝ^{n} is the state vector, and f is a vector-valued function.

To proceed, we need the following lemma. Let B_{q}(0) denote an open ball with the center at 0 and radius q, and let S = B̄_{R}(0) ⊂ ℝ^{n} be a closed ball, a compact set, that contains the origin x = 0 in its interior.

Lemma 7.2 For a continuously differentiable function V : ℝ^{n} → ℝ, where V (0) = 0, we have V (x) > 0 for all x ∈ S \ {0}, if and only if there is a continuous, strictly increasing, function ψ : ℝ → ℝ such that ψ(0) = 0, ψ(r) > 0 for r > 0, and V (x) ≥ ψ(r), where r = ∥x∥ and r ≤ R.

Proof Sufficiency ( ⇐) is clear.

To prove necessity ( ⇒), we define, following Willems [351, page 21], the function κ : ℝ → ℝ,

κ(y) = min_{∥x∥=y^{x∈S}} V (x)  (7.51)

Note that the function κ is well defined because the set

{x : ∥x∥ = y} ∩ S  (7.52)

is compact and V is continuous. Therefore, by the Weierstrass theorem on page 985, the function V achieves its infimum over the set (7.52).

We next show that κ is continuous on S = B̄_{R}(0). Then, we construct an appropriate function ψ satisfying the hypotheses of the lemma. To show that κ is continuous on S, we use the fact that V is uniformly continuous in S. To this end let x^{(1)} ∈ S be such that ∥x^{(1)}∥ = y and κ(y) = V (x^{(1)}). Next, let Δy be an increment of y and let x^{(2)} ∈ S be a point, a vector, such that y + Δy = ∥x^{(2)}∥. For example, we can take

x^{(2)} = x^{(1)} + \frac{x^{(1)}}{∥x^{(1)}∥}Δy.

Observe that

V (x^{(2)}) ≥ κ(y + Δy).

Figure 7.8: A two-dimensional illustration of the proof of the continuity of the function κ given by (7.51).

Choose now a vector x^{(3)} such that

∥x^{(3)}∥ = y + Δy and V (x^{(3)}) = κ(y + Δy).

Note that for any vector, say x^{(4)}, such that ∥x^{(4)}∥ = y we have V (x^{(4)}) ≥ κ(y)—see Figure 7.8. We can take

x^{(4)} = x^{(3)} − \frac{x^{(3)}}{∥x^{(3)}∥}Δy.

By the uniform continuity of V, for any ϵ > 0 there exists a δ > 0 such that if

∥x^{(1)} − x^{(2)}∥ < δ and ∥x^{(3)} − x^{(4)}∥ < δ,

then

|V (x^{(1)}) − V (x^{(2)})| < ϵ and |V (x^{(3)}) − V (x^{(4)})| < ϵ.

From |V (x^{(1)}) − V (x^{(2)})| < ϵ, we obtain

−ϵ < V (x^{(1)}) − V (x^{(2)}) < ϵ.

Thus

V (x^{(1)}) > V (x^{(2)}) − ϵ.

From |V (x^{(3)}) − V (x^{(4)})| < ϵ, we get

V (x^{(3)}) > V (x^{(4)}) − ϵ.

From the definition of the function κ it follows that

V (x^{(2)}) ≥ V (x^{(3)}) and V (x^{(4)}) ≥ V (x^{(1)}) .

Because V (x^{(1)}) > V (x^{(2)}) − ϵ, we obtain

V (x^{(1)}) > V (x^{(3)}) − ϵ

and therefore

−ϵ < V (x^{(1)}) − V (x^{(3)}) .  (7.53)

Because V (x^{(3)}) > V (x^{(4)}) − ϵ, we have

V (x^{(3)}) > V (x^{(1)}) − ϵ,

and thus

V (x^{(1)}) − V (x^{(3)}) < ϵ.  (7.54)

Combining (7.53) and (7.54), we get

|V (x^{(1)}) − V (x^{(3)})| < ϵ,

or equivalently,

|κ(y) − κ(y + Δy)| < ϵ

if

Δy < δ,

which proves the continuity of κ.

We are now ready to construct a function ψ that satisfies the hypotheses of the lemma. This function comes from Willems [351, page 22] and has the form

ψ(r) = r min_{r≤y≤R} \frac{κ(y)}{y}  (7.55)

Note that ψ is continuous, ψ(0) = 0, and it is positive and strictly increasing—see Figure 7.9 for a graphical illustration of the way ψ is constructed from κ. The proof of the lemma is now complete.

□

We can characterize a positive definite function V in another way as in the following lemma.

Figure 7.9: Constructing the function ψ given by (7.55).

Lemma 7.3 A continuously differentiable function V : ℝ^{n} → ℝ is positive definite in S = B̄_{R}(0) ⊂ ℝ^{n}, with respect to x = 0, if and only if there exists a continuous nondecreasing function α : ℝ → ℝ, such that α(0) = 0, α(r) > 0 for r > 0, and V (x) ≥ α(r), where r = ∥x∥ and r ≤ R.

A possible choice for α(·) is

α(r) = min_{r≤∥x∥≤R} V (x).

Note that the function α(·) is nondecreasing because as r increases, the minimum is taken over a smaller region. Furthermore, α(r) > 0 whenever r > 0 because V (x) > 0 for x ∈ S \ {0}. The function α is continuous because V is continuous.

The main difference between the above two characterizations of positive definite functions is that in Lemma 7.2 the function ψ is strictly increasing rather than just nondecreasing. In our further discussion, we will exploit extensively the existence of a such a function ψ associated with a positive definite function V.

So far we only considered positive definite, or negative definite functions, that are timeinvariant. We now extend our definition of positive definiteness to time-varying functions of the form W (t, x). We first define a time-varying positive semi-definite function. As before, we let S to denote a compact subset of ℝ^{n} containing the origin x = 0 in its interior.

Definition 7.11 A real-valued function W : ℝ × ℝ^{n} → ℝ is positive semi-definite, with respect to x = 0, in a region S ⊆ ℝ^{n} if for all t and all x in S

(i) W is continuous and has continuous partial derivatives with respect to all of its arguments, that is, W ∈ C^{1},

(ii) W (t, 0) = 0,

(iii) W (t, x) ≥ 0.

◇

Definition 7.12 A real-valued function W : ℝ × ℝ^{n} → ℝ is positive definite, with respect to x = 0, in a region S ⊆ ℝ^{n} if for all t and all x in S

(i) W ∈ C^{1},

(ii) W (t, 0) = 0,

(iii) there exists a positive definite time-invariant function V such that W (t, x) ≥ V (x).

◇

We next introduce a notion of a decrescent function.

Definition 7.13 A time-varying function W : ℝ × ℝ^{n} → ℝ is decrescent in a region S if a positive definite function Ṽ exists such that for all x in S and all t

|W (t, x)| ≤ Ṽ (x),

that is, W tends to zero uniformly with respect to t as ∥x∥ → 0.

◇

Consider now an equilibrium state of the dynamic system modeled by (7.1), that is, a constant state x_{e} such that f(t, x_{e}) = 0 for all t. Let W (t, x) be a time-dependent realvalued function whose time derivative along the solution x(t) = x(t; t_{0}, x_{0}) of the given differential equation, is

\frac{dW (t, x)}{dt} = Ẇ (t, x) = \frac{∂W}{∂t} + ∇W^{⊤}ẋ.

Then, we have the following theorem concerning the stability of the equilibrium point in the sense of Lyapunov.

Theorem 7.7 If in some neighborhood S of the equilibrium state x_{e}, there is a positive definite function W, with respect to x_{e}, such that its time derivative along any solution of ẋ(t) = f(t, x(t)) is negative semi-definite in S, then the equilibrium state x_{e} is stable. If W is also decrescent, then the equilibrium state is uniformly stable.

Proof Without loss of generality, we assume that x_{e} = 0 and follow the arguments of Willems [351, pages 29, 30]. Because W is positive definite, by Definition 7.12 and Lemma 7.2 there is a strictly increasing function ψ_{1}(r), ψ(0) = 0, such that

W (t, x) ≥ ψ_{1}(∥x∥),

where ∥x∥ ≤ R, and R is the radius of the largest ball contained in S. Let

ψ_{2}(t_{0}, r) = max_{∥x∥≤r} W (t_{0}, x).

Then

ψ_{1}(r) ≤ ψ_{2}(t_{0}, r).

Thus, for any ϵ > 0 there is a positive δ that depends on ϵ and t_{0}, such that

ψ_{2}(t_{0}, δ) < min{ψ_{1}(ϵ), ψ_{1}(R)}.

Figure 7.10 illustrates how to determine a δ given an ϵ. By assumption dW (t, x)/dt ≤ 0 along any solution of the system. Hence W (t, x) is a non-increasing function and therefore

W (t, x(t; t_{0}, x_{0})) ≤ W (t_{0}, x_{0}) for all t ≥ t_{0}.

If ∥x_{0}∥ < δ, then for t ≥ t_{0} we have

ψ_{1}(ϵ) > ψ_{2}(t_{0}, δ) ≥ W (t_{0}, x_{0}) ≥ W (t, x(t; t_{0}, x_{0})) ≥ ψ_{1}(∥x(t; t_{0}, x_{0})∥).

Therefore,

ψ_{1}(ϵ) > ψ_{1}(∥x(t; t_{0}, x_{0})∥).

Because ψ_{1} is strictly increasing, we have to have

∥x(t; t_{0}, x_{0})∥ < ϵ

for all t ≥ t_{0}. This means that the null solution is (locally) stable, which completes the proof of the first part of the theorem.

Figure 7.10: Determining a δ given an ϵ.

If W is also decrescent, then there is a nondecreasing function ψ̃_{2} : ℝ → ℝ, ψ̃_{2}(0) = 0, and

W (t, x) ≤ ψ̃_{2}(∥x∥)

for all x such that ∥x∥ ≤ R and all t. Indeed, because W (t, x) is positive definite and decrescent, there exists a positive definite V (x) such that W (t, x) ≤ V (x). Then, we can take ψ̃_{2} defined as

ψ̃_{2}(r) = max_{∥x∥≤r} V (x).

Let

ψ̃_{2}(δ) < min{ψ_{1}(ϵ), ψ_{1}(R)}.

Note that δ depends now only on ϵ, and not on t_{0} as before. To show the uniform stability of the null solution, we repeat the argument above with ψ̃_{2} replacing ψ_{2}. This completes the proof of the theorem.

□

The next theorem gives us a sufficiency condition for an equilibrium point to be uniformly asymptotically stable.

Theorem 7.8 If in some neighborhood S of the given equilibrium point x_{e} there is a positive definite, with respect to x_{e}, and decrescent function W such that its time derivative along any solution of the system model (7.1) is negative definite in S, then the equilibrium state is uniformly asymptotically stable.

Proof As before, without loss of generality, we assume that x_{e} = 0 and then follow the arguments of Willems [351, pages 30, 31]. The previous theorem ensures uniform stability of the equilibrium state x_{e}. Thus, only the uniform convergence remains to be shown. From the assumptions of the theorem it follows that there exist strictly increasing functions ψ_{1} and ψ_{3}, and a non-decreasing function ψ_{2} such that

ψ_{1}(0) = ψ_{2}(0) = ψ_{3}(0) = 0,
ψ_{1}(∥x∥) ≤ W (t, x) ≤ ψ_{2}(∥x∥),
ψ_{3}(∥x∥) ≤ −Ẇ (t, x),

for all t and all x such that ∥x∥ ≤ R, where R is the radius of the largest ball contained in S. First, select δ_{1} so that

ψ_{2}(δ_{1}) < ψ_{1}(R).  (7.56)

Next, given an ϵ_{1} > 0, define positive constants δ_{2} and T as follows:

ψ_{2}(δ_{2}) < min{ψ_{1}(ϵ_{1}), ψ_{2}(δ_{1})},  (7.57)
T = ψ_{1}(R)/ψ_{3}(δ_{2}).  (7.58)

Note that δ_{1} and T are independent of t_{0}.

We prove the theorem in two stages. First, we show that for some t_{2} ∈ [t_{0}, t_{0} + T],

∥x(t_{2}; t_{0}, x_{0})∥ < δ_{2}

whenever ∥x_{0}∥ < δ_{1}. Then, we show that once ∥x(t_{2}; t_{0}, x_{0})∥ < δ_{2}, then

∥x(t; t_{0}, x_{0})∥ < ϵ_{1} for all t ≥ t_{2}.

We begin with proving the claim that for some t_{2} ∈ [t_{0}, t_{0} + T],

∥x(t_{2}; t_{0}, x_{0})∥ < δ_{2}

whenever ∥x_{0}∥ < δ_{1}. We prove this claim by contradiction. We assume that for all t ∈ [t_{0}, t_{0} + T],

∥x(t; t_{0}, x_{0})∥ ≥ δ_{2} if ∥x_{0}∥ < δ_{1}.  (7.59)

Using (7.59), (7.58), and (7.56), we obtain

0 < ψ_{1}(δ_{2}) ≤ ψ_{1}(∥x(t_{0} + T; t_{0}, x_{0})∥)
≤ W [t_{0} + T, x(t_{0} + T; t_{0}, x_{0})]
= W (t_{0}, x_{0}) + ∫_{t_{0}}^{t_{0}+T} Ẇ [t, x(t; t_{0}, x_{0})]dt
≤ W (t_{0}, x_{0}) − ∫_{t_{0}}^{t_{0}+T} ψ_{3}(∥x(t; t_{0}, x_{0})∥)dt
≤ W (t_{0}, x_{0}) − T ψ_{3}(δ_{2})
≤ ψ_{2}(∥x_{0}∥) − T ψ_{3}(δ_{2})
< ψ_{2}(δ_{1}) − ψ_{1}(R)
< 0,

which is a contradiction, and thus we proved the first part of the theorem. We now proceed with the second part. It follows from the first part that if ∥x_{0}∥ < δ_{1}, then for some t_{2} ∈ [t_{0}, t_{0} + T], we have ∥x(t_{2}; t_{0}, x_{0})∥ < δ_{2}. Hence, for t ≥ t_{2},

ψ_{1}(∥x(t; t_{0}, x_{0})∥) ≤ W (t, x(t; t_{0}, x_{0})) ≤ W (t_{2}, x(t_{2}; t_{0}, x_{0})) < ψ_{2}(δ_{2}) < ψ_{1}(ϵ_{1}).

Therefore,

ψ_{1}(∥x(t; t_{0}, x_{0})∥) < ψ_{1}(ϵ_{1}),

and because ψ_{1} is strictly increasing,

∥x(t; t_{0}, x_{0})∥ < ϵ_{1}

for all t ≥ t_{2} which proves the uniform convergence. The proof of the theorem is thus complete.

□

Example 7.9 Given a dynamical system model

ẋ(t) = [ \begin{matrix}−0.75 & 0.3536 \\ 0.3536 & −0.5 \end{matrix} ] x(t), x(0) = x_{0}.

The solution P = P^{⊤} to the Lyapunov matrix equation A^{⊤}P +P A = −Q, where Q = 2I_{2}, is

P = [ \begin{matrix}2 & \sqrt{2} \\ \sqrt{2} & 3 \end{matrix} ] .

356 CHAPTER 7. STABILITY

The eigenvalues of P are

λ_{1} = λ_{min}(P) = 1, λ_{2} = λ_{max}(P) = 4.

In general, for a symmetric matrix P,

λ_{min}(P)∥x∥^{2} ≤ x^{⊤}P x ≤ λ_{max}(P)∥x∥^{2}.

Therefore, we can take

ψ_{1}(∥x∥) = λ_{min}(P)∥x∥^{2}, ψ_{2}(∥x∥) = λ_{max}(P)∥x∥^{2}, and ψ_{3}(∥x∥) = λ_{min}(Q)∥x∥^{2}. In our example, λ_{min}(Q) = 2.

• Given ε = 4; we find δ > 0 such that if ∥x_{0}∥ ≤ δ, then ∥x(t)∥ ≤ ε for t ≥ 0.

We obtain δ = δ(ε) by solving the equation,

ψ_{1}(ε) = ψ_{2}(δ),

to get

δ = \sqrt{\frac{λ_{min}(P)}{λ_{max}(P)}}ε = 2.

• Given x_{0} = [ 8 6 ]^{⊤} and ε_{1} = 2; we find T such that ∥x(t)∥ ≤ ε_{1} for all t ≥ T.

We have ∥x_{0}∥_{2} = 10. Let δ_{1} = 10. We then compute R, δ_{2}, and T from the equations:

ψ_{2}(δ_{1}) = ψ_{1}(R)
ψ_{2}(δ_{2}) = min (ψ_{1}(ε_{1}), ψ_{2}(δ_{1}))
T = \frac{ψ_{1}(R)}{ψ_{3}(δ_{2})}.

Using the first equation, we find

R = \sqrt{\frac{λ_{max}(P)}{λ_{min}(P)}}δ_{1} = 20.

We then solve for δ_{2} ≤ δ_{1} to get

δ_{2} = \sqrt{\frac{λ_{min}(P)}{λ_{max}(P)}}ε_{1} = 1.

Hence,

T = \frac{λ_{min}(P)R^{2}}{λ_{min}(Q)δ_{2}^{2}} = \frac{λ_{min}(P)\frac{λ_{max}(P)}{λ_{min}(P)}δ_{1}^{2}}{λ_{min}(Q)\frac{λ_{min}(P)}{λ_{max}(P)}ε_{1}^{2}} = \frac{λ_{max}^{2}(P)δ_{1}^{2}}{λ_{min}(Q)λ_{min}(P)ε_{1}^{2}} = 200.

Proposition 7.1 If in some neighborhood S of x_{e} = 0 the positive definite and decrescent function W satisfies

c_{1}∥x∥^{2} ≤ W (t, x) ≤ c_{2}∥x∥^{2}

for some positive constants c_{1} and c_{2} and its time derivative along any solution of the system (7.1) is negative definite in S and such that for some positive constant c_{3},

Ẇ (t, x) ≤ −c_{3}∥x∥^{2},

then the equilibrium state x_{e} = 0 is uniformly exponentially stable.

Proof We have

Ẇ (t, x) ≤ −c_{3}∥x∥^{2}
= −\frac{c_{3}}{c_{2}} (c_{2}∥x∥^{2})
≤ −\frac{c_{3}}{c_{2}} W (t, x).

Applying to the above Theorem A.20, which is called the comparison theorem, we obtain

W (t, x) ≤ W (t_{0}, x_{0}) exp (−\frac{c_{3}}{c_{2}} (t − t_{0})) .

Hence,

∥x∥ ≤ \sqrt{\frac{W (t, x)}{c_{1}}}

≤ \sqrt{\frac{W (t_{0}, x_{0}) exp (−\frac{c_{3}}{c_{2}} (t − t_{0}))}{c_{1}}}
≤ (\frac{c_{2}}{c_{1}} )^{\frac{1}{2}} ∥x(t_{0})∥ exp (−\frac{c_{3}}{2c_{2}} (t − t_{0})) ,

which means that the equilibrium state x_{e} = 0 is uniformly exponentially stable.

□

The above theorems were concerned with local stability. We now proceed to prove sufficiency conditions for global stability. For this we need the following definitions. We first define a radially unbounded or coercive function.

Definition 7.14 A time-varying function W : ℝ × ℝ^{n} → ℝ is radially unbounded, or coercive, if as ∥x∥ → ∞, the function W tends to infinity uniformly with respect to t, that is, for any given positive real number M, there is a positive real number N so that W (t, x) > M for all t and all x whenever ∥x∥ > N.

◇

The next two definitions are concerned with Lagrange stable equilibrium states.

Definition 7.15 The equilibrium state x_{e} is called Lagrange stable, or bounded, if for every real number d > 0 there exists a positive real number b = b(t_{0}, d) such that if

∥x_{0} − x_{e}∥ < d,

then

∥x(t; t_{0}, x_{0}) − x_{e}∥ < b

for all t ≥ t_{0}.

◇

Definition 7.16 The equilibrium state x_{e} is uniformly bounded if the number b in Definition 7.15 can be taken independent of the initial time t_{0}.

◇

We now define uniformly asymptotically stable in the large equilibrium state.

Definition 7.17 If the equilibrium state x_{e} is uniformly stable, uniformly bounded, and all trajectories are uniformly convergent to x_{e}, then this equilibrium state is called uniformly asymptotically stable in the large, or globally uniformly asymptotically stable.

◇

The following theorem gives a sufficiency condition for an equilibrium state to be globally uniformly asymptotically stable.

Theorem 7.9 If there is a function W that is positive definite for all x ≠ x_{e}, W (t, x_{e}) = 0, decrescent, radially unbounded, and if its time derivative along any solution of the system is negative definite for all x, then the equilibrium state x_{e} is globally uniformly asymptotically stable.

Figure 7.11: Determining a constant b > 0 given d > 0 in the proof of Theorem 7.9.

Proof We assume, for simplicity, that x_{e} = 0 and follow the arguments of Willems [351, pages 31, 32]. By Theorem 7.7, the equilibrium state x_{e} is uniformly stable. Similarly as in Theorem 7.8, we construct the functions ψ_{1}, ψ_{2}, and ψ_{3}. Since, by assumption, W is radially unbounded, ψ_{1}(∥x∥) → ∞ as ∥x∥ → ∞. Therefore, for any constant d > 0, there is a constant b > 0 such that ψ_{1}(b) > ψ_{2}(d). See Figure 7.11 for an illustration of the procedure that can be used to find the constant b. If ∥x_{0}∥ < d, then ∥x(t; t_{0}, x_{0})∥ < b for all t ≥ t_{0}. Indeed, because

ψ_{1}(b) > ψ_{2}(d) ≥ W (t_{0}, x_{0}) ≥ W (t, x(t; t_{0}, x_{0})) ≥ ψ_{1}(∥x(t; t_{0}, x_{0})∥),

then

ψ_{1}(b) > ψ_{1}(∥x(t; t_{0}, x_{0})∥).

The above implies that

∥x(t; t_{0}, x_{0})∥ < b

because ψ_{1} is strictly increasing. Hence all trajectories of the system are uniformly bounded. To prove their convergence to the equilibrium state, we define δ_{2}, T, and b, given d and ϵ_{1}, as

ψ_{2}(d) < ψ_{1}(b),
ψ_{2}(δ_{2}) < ψ_{1}(ϵ_{1}),
T = ψ_{1}(b)/ψ_{3}(δ_{2}).

If ∥x_{0}∥ < d, then for all t > t_{0} + T, we have

∥x(t; t_{0}, x_{0})∥ < ϵ_{1},

which can be shown using the same arguments as in the proof of Theorem 7.8 with b playing the role of R, and d playing the role of δ_{1}. This proves that all trajectories of the system converge uniformly to the equilibrium state. Hence, the equilibrium state is globally uniformly asymptotically stable.

□

We now give a theorem that can be used as a test for checking the instability of nonlinear dynamic systems. This instability test is illustrated in Figure 7.12.

Theorem 7.10 Let x_{e} be an equilibrium state of the system ẋ = f(t, x), that is, f(t, x_{e}) = 0. If there exists a function W : ℝ×ℝ^{n} → ℝ such that in the region S = {x : ∥x−x_{e}∥ ≤ R}

(i) W ∈ C^{1},

(ii) W (t, x_{e}) = 0,

(iii) Ẇ (t, x) > 0 for all x ∈ S \ {x_{e}},

(iv) in every neighborhood of x_{e} in S, for any t_{0} there is at least one point x at which W (t_{0}, x) > 0, and

(v) W (t, x) is bounded in S,

then the equilibrium state x_{e} is unstable.

Proof Without loss of generality, assume that x_{e} = 0. By assumption (iv), for any t_{0} and any δ, 0 < δ < R, there exists x_{δ}, 0 < ∥x_{δ}∥ < δ, such that W (t_{0}, x_{δ}) > 0. The system trajectory x(t) = x(t; t_{0}, x_{δ}) cannot approach the equilibrium state x_{e} = 0 as t → ∞ because W (t, 0) = 0 but W (t_{0}, x_{δ}) > 0 and Ẇ (t, x) > 0 for x ≠ x_{e}. Since Ẇ is continuous, because W ∈ C^{1}, and the trajectory is bounded away from the equilibrium state, there is a constant m > 0 such that for all t ≥ t_{0},

Ẇ (t, x(t)) ≥ m > 0,

Figure 7.12: An illustration of Theorem 7.10.

as long as

∥x(t)∥ ≤ R, x ≠ 0.

Hence,

W (t, x) = W (t_{0}, x_{δ}) + ∫_{t_{0}}^{t} Ẇ (s, x(s))ds ≥ W (t_{0}, x_{δ}) + m(t − t_{0}).  (7.60)

By condition (v), the function W is bounded in S.. However, the right hand side of (7.60) is unbounded. Therefore the trajectory x(t; t_{0}, x_{δ}) cannot remain infinitely long in S. Thus, for some t_{1} > t_{0} the trajectory reaches the boundary of the region ∥x∥ ≤ R. Hence, given any ϵ, 0 < ϵ < R, then for any δ > 0, no matter how small, there is at least one trajectory such that ∥x_{δ}∥ < δ but ∥x(t; t_{0}, x_{δ})∥ ≥ ϵ for some t_{1} > t_{0}. Therefore, the equilibrium state is unstable.

□

Corollary 7.1 If there exists a function W : ℝ × ℝ^{n} → ℝ such that in the region S = {x : ∥x − x_{e}∥ ≤ R}

(i) W ∈ C^{1},

(ii) W (t, x_{e}) = 0,

(iii) Ẇ (t, x) < 0 for all x ∈ S \ {x_{e}}, (iv) in every neighborhood of x_{e} in S, for any t_{0} there is at least one point x at which

W (t_{0}, x) < 0, and

(v) W (t, x) is bounded in S,

then the equilibrium state x_{e} is unstable.

Note that if W (t, x) = W (x), that is, if W is time-invariant, then Condition (v) of Theorem 7.10 and Corollary 7.1 is not needed because W = W (x) being continuous on the compact set S must be bounded in S.

7.9 Lyapunov’s Indirect Method

In this section, we are concerned with the problem of investigating stability properties of an equilibrium state of a nonlinear system based on its linearization about the given equilibrium. We devise a method that allows one to determine whether the equilibrium of the nonlinear system is asymptotically stable or unstable based on the dynamics of the linearized system about this equilibrium. The method is sometimes referred to as the Lyapunov’s first method or Lyapunov’s indirect method. For simplicity of notation, we assume that the equilibrium point to be tested for stability is the origin. We consider a nonlinear system model,

ẋ = f(x),  (7.61)

where f : D → ℝ^{n} is a continuously differentiable function from a domain D ⊆ ℝ^{n} into ℝ^{n}. We assume that the origin x = 0 is in the interior of D and is an equilibrium state of (7.61), that is, f(0) = 0. Let

A = \frac{∂f}{∂x}|_{x=0} = [ \begin{matrix}\frac{∂f_{1}}{∂x_{1}} & \frac{∂f_{1}}{∂x_{2}} & ⋯ & \frac{∂f_{1}}{∂x_{n}} \\ ⋮ & & ... & ⋮ \\ \frac{∂f_{n}}{∂x_{1}} & \frac{∂f_{n}}{∂x_{2}} & ⋯ & \frac{∂f_{n}}{∂x_{n}} \end{matrix} ]|_{x=0}  (7.62)

be the Jacobian matrix of f evaluated at x = 0. Then, applying Taylor’s theorem to the right hand side of (7.61) yields

ẋ = Ax + g(x),  (7.63)

where the function g represents higher-order terms and has the property,

lim_{∥x∥→0} \frac{∥g(x)∥}{∥x∥} = 0.  (7.64)

The above suggests that in a “small” neighborhood of the origin, we can approximate the nonlinear system (7.61) with its linearized system about the origin, ẋ = Ax. The following theorems give conditions under which we can infer about stability properties of the origin of the nonlinear system based on stability properties of the linearized system.

Theorem 7.11 Let the origin, x = 0, be an equilibrium state of the nonlinear system, ẋ = f(x). Then, the origin is an asymptotically stable equilibrium of the nonlinear system if A, the Jacobian matrix of f evaluated at the origin, has all its eigenvalues in the open left-half plane.

Proof Our proof is based on that of Khalil [178, pages 127–128]. To begin with, we suppose that all the eigenvalues of A are in the open left-half plane. Then, by Theorem 7.1, for any positive definite symmetric matrix Q, the solution P to the Lyapunov equation (7.7) is also positive definite. We employ the positive definite quadratic form V (x) = x^{⊤}P x as a Lyapunov function candidate for the nonlinear system ẋ = f(x). We use (7.63) and the Lyapunov matrix equation (7.7) when evaluating the time derivative of V (x) on the trajectories of the nonlinear system. We obtain

V̇ (x) = 2x^{⊤}P f(x)
= 2x^{⊤}P (Ax + g(x))
= x^{⊤} (A^{⊤}P + P A) x + 2x^{⊤}P g(x)
= −x^{⊤}Qx + 2x^{⊤}P g(x).  (7.65)

The first term on the right hand side of the above equation is negative definite, while the second term is, in general, indefinite. The function g satisfies

\begin{matrix}∥g(x)∥ & 2 \\ ∥x∥ & 2 \end{matrix} → 0 as ∥x∥_{2} → 0.

Therefore, for any γ > 0 there exists r > 0 such that

∥g(x)∥_{2} < γ ∥x∥_{2} for all ∥x∥_{2} < r.  (7.66)

Substituting the above into (7.65) gives

V̇ (x) < −x^{⊤}Qx + 2γ ∥P ∥_{2} ∥x∥_{2}^{2}
≤ − (λ_{min}(Q) − 2γλ_{max}(P)) ∥x∥_{2}^{2} for all ∥x∥_{2} < r  (7.67)

Thus, selecting

γ < \frac{λ_{min}(Q)}{2λ_{max}(P)}

ensures that V̇ (x) is negative definite. By Theorem 7.8, we conclude that the origin, x = 0, is an asymptotically stable equilibrium state of the nonlinear system, which completes the proof.

□

Theorem 7.12 Let the origin, x = 0, be an equilibrium state of the nonlinear system, ẋ = f(x). Then, the origin is unstable if A has one or more eigenvalues in the right-half plane.

Proof Our proof follows that of Khalil [178, pages 128–130]. We first consider the special case when A has no eigenvalues on the imaginary axis. Suppose that the eigenvalues cluster into a group of eigenvalues in the open right-plane and a group of eigenvalues in the open left-half plane. Then, we can find a nonsingular matrix T such that

T AT^{−1} = [ \begin{matrix}−A_{1} & O \\ O & A_{2} \end{matrix} ] ,

where both A_{1} and A_{2} have their eigenvalues in the open left-half plane. A possible candidate for T would be the matrix that transforms A into its real Jordan form. Now, let

z = T x = [ \begin{matrix}z & 1 \\ z & 2 \end{matrix} ] ,

where the partition of z is compatible with the dimensions of A_{1} and A_{2}. The change of variables, z = T x, transforms the nonlinear system (7.63) into the form

\begin{matrix}ż_{1} & = & −A_{1}z_{1} + g_{1}(z) \\ ż_{2} & = & A_{2}z_{2} + g_{2}(z) \end{matrix} \}  (7.68)

where the functions g_{i}, i = 1, 2, have the property that for any γ > 0, there exists r > 0 such that for i = 1, 2, we have

∥g_{i}(z)∥_{2} ≤ γ ∥z∥_{2} for all ∥z∥_{2} ≤ r.  (7.69)

The origin z = 0 is an equilibrium state for the nonlinear system (7.68). Since the matrix T is nonsingular, stability properties of z = 0 in the z-coordinates are the same as the stability properties of x = 0 in the x-coordinates. We show that z = 0 is an unstable equilibrium of the system (7.68), which implies that x = 0 is an unstable equilibrium of the system (7.61). We use Theorem 7.10, which entails constructing a function V satisfying the hypotheses of that theorem. Let Q_{1} and Q_{2} be positive definite symmetric matrices of the dimensions of A_{1} and A_{2}, respectively. Because A_{1} and A_{2} have their eigenvalues in the open left-half plane, the Lyapunov matrix equations,

A_{i}^{⊤}P_{i} + P_{i}A_{i} = −Q_{i}, i = 1, 2,

have unique positive definite solutions P_{1} and P_{2}. Let

V (z) = z_{1}^{⊤}P_{1}z_{1} − z_{2}^{⊤}P_{2}z_{2} = z^{⊤} [ \begin{matrix}P_{1} & O \\ O & −P_{2} \end{matrix} ] z.  (7.70)

In the subspace, {z = [ z_{1}^{⊤} z_{2}^{⊤} ]^{⊤} : z_{2} = 0}, V (z) > 0 in every neighborhood of the origin. Let

U = {z ∈ ℝ^{n} : ∥z∥_{2} ≤ r} .

The time derivative of V evaluated on the trajectories of the system (7.68) is

V̇ (z) = 2z_{1}^{⊤}P_{1}ż_{1} − 2z_{2}^{⊤}P_{2}ż_{2}
= 2z_{1}^{⊤}P_{1} (−A_{1}z_{1} + g_{1}(z)) − 2z_{2}^{⊤}P_{2} (A_{2}z_{2} + g_{2}(z))
= z_{1}^{⊤}Q_{1}z_{1} + 2z_{1}^{⊤}P_{1}g_{1}(z) + z_{2}^{⊤}Q_{2}z_{2} − 2z_{2}^{⊤}P_{2}g_{2}(z)
≥ λ_{min}(Q_{1}) ∥z_{1}∥_{2}^{2} + λ_{min}(Q_{2}) ∥z_{2}∥_{2}^{2} + 2z^{⊤} [ \begin{matrix}P_{1}g_{1}(z) \\ −P_{2}g_{2}(z) \end{matrix} ]
≥ λ_{min}(Q_{1}) ∥z_{1}∥_{2}^{2} + λ_{min}(Q_{2}) ∥z_{2}∥_{2}^{2}
−2 ∥z∥_{2} \sqrt{∥P_{1}∥_{2}^{2} ∥g_{1}(z)∥_{2}^{2} + ∥P_{2}∥_{2}^{2} ∥g_{2}(z)∥_{2}^{2}}.  (7.71)

Let

α = min (λ_{min}(Q_{1}), λ_{min}(Q_{2})) and β = max (∥P_{1}∥_{2} , ∥P_{2}∥_{2}) .

With (7.69) and the above taken into account, (7.71) evaluates to

V̇ (z) ≥ (α − 2\sqrt{2}βγ) ∥z∥_{2}^{2} for all z ∈ U.  (7.72)

If we select

γ < \frac{α}{2\sqrt{2}β} ,

then V̇ (z) > 0 in U \ {0}. Thus, all hypotheses of Theorem 7.10 are satisfied and hence the origin is an unstable equilibrium of the system (7.61).

In the general case when A has eigenvalues on the imaginary axis, in addition to eigenvalues in the open right-half plane, we shift the imaginary axis in order to reduce this case to the previous one. Specifically, suppose that A has m eigenvalues in the open right-half plane with ℜλ_{i} > δ > 0. Then, the matrix

A − \frac{δ}{2}I

still has m eigenvalues in open right-half plane, but no eigenvalues on the imaginary axis. We can proceed similarly as in the previous case to conclude that the origin is an unstable equilibrium of the corresponding nonlinear system (7.61). For details of this part of the proof, we refer to Khalil [178, page 130].

□

In summary, the above two theorems provide us with a simple method for determining asymptotic stability or instability of an equilibrium state of the nonlinear system (7.61). We compute the Jacobian matrix (7.62) of the nonlinear system, evaluate it at the equilibrium state of interest, and test its eigenvalues. If ℜλ_{i} < 0 for all i, then the equilibrium state is asymptotically stable. If for some i, ℜλ_{i} > 0, then the equilibrium is unstable. Finally, if ℜλ_{i} ≤ 0 for all i with ℜλ_{i} = 0 for some i, linearization fails to determine asymptotic stability of the equilibrium of the nonlinear system. In this case, the stability of the nonlinear system depends on the nature of the higher-order terms.

Example 7.10 For the nonlinear system model,

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}−1.1x_{1} + g_{2}(x_{2}) \\ g_{1}(x_{1}) − 1.1x_{2} \end{matrix} ] ,  (7.73)

where

g_{i}(x_{i}) = \frac{2}{π} arctan (0.7πx_{i}) , i = 1, 2,

we find its equilibrium states and determine their stability properties using the linearization method. We mention here that the above system was analyzed in [140, 241].

To find the system equilibrium states, we solve the algebraic equations ẋ_{1} = 0 and ẋ_{2} = 0, that is,

\begin{matrix}−1.1x_{1} + \frac{2}{π} arctan (0.7πx_{2}) & = & 0 \\ \frac{2}{π} arctan (0.7πx_{1}) − 1.1x_{2} & = & 0 \end{matrix} \}

We obtain three equilibrium states:

x^{(1)} = [ \begin{matrix}0.454 \\ 0.454 \end{matrix} ] , x^{(2)} = [ \begin{matrix}−0.454 \\ −0.454 \end{matrix} ] , and x^{(3)} = [ \begin{matrix}0 \\ 0 \end{matrix} ] .

Let

Δx = x − x^{(i)}, i = 1, 2, 3.

Then, a linearized model about an equilibrium state has the form

\frac{d}{dt}Δx = AΔx,

where

A^{(i)} = [ \begin{matrix}\frac{∂f_{1}(x)}{∂x_{1}} & \frac{∂f_{1}(x)}{∂x_{2}} \\ \frac{∂f_{2}(x)}{∂x_{1}} & \frac{∂f_{2}(x)}{∂x_{2}} \end{matrix} ]|_{x=x^{(i)}} = [ \begin{matrix}−1.1 & \frac{1.4}{1+(0.7πx_{2})^{2}} \\ \frac{1.4}{1+(0.7πx_{1})^{2}} & −1.1 \end{matrix} ]|_{x=x^{(i)}} .  (7.74)

We have three linearized models:

(i) about x^{(1)},

\frac{d}{dt}Δx = [ \begin{matrix}−1.1 & 0.7073 \\ 0.7073 & −1.1 \end{matrix} ] Δx,

(ii) about x^{(2)},

\frac{d}{dt}Δx = [ \begin{matrix}−1.1 & 0.7073 \\ 0.7073 & −1.1 \end{matrix} ] Δx,

(iii) about x^{(3)},

\frac{d}{dt}Δx = [ \begin{matrix}−1.1 & 1.4 \\ 1.4 & −1.1 \end{matrix} ] Δx.

By Theorem 7.11, the states x^{(1)} and x^{(2)} are asymptotically stable because the corresponding linearized models have their eigenvalues in the open left-half plane: -1.8073 and -0.3927. By Theorem 7.12, the equilibrium state x^{(3)} is unstable because the linearized model has an eigenvalue in the open right-half plane, at 0.3 (the other eigenvalue is at -2.5). In Figure 7.13, we show a phase portrait of the above nonlinear system, where one can visually confirm the stability properties of the equilibrium states.

Example 7.11 In this example, we employ the Lyapunov’s indirect method in the analysis of the hypothalamic-pituitary-adrenal (HPA) axis model presented in Section 2.4.8. Recall that the HPA axis is the body’s primary stress management system. In our analysis of the HPA axis, we use plots of equilibrium values of the state variables as a function of the external stressor, d. These plots are shown in Figures 7.14 and 7.15.

In a chronically stressed individual, cortisol concentration, x_{4}, is very low. Thus a healthy individual subjected to a prolonged extreme stress, d > 0.168, would settle down in a stable 368 CHAPTER 7. STABILITY

Figure 7.13: Phase portrait of the nonlinear system of Example (7.10). Figure 7.14: Plots of the equilibrium values of x_{1} and x_{3} versus d for u = 0.

Figure 7.15: Plots of the equilbrium values of x_{2} and x_{4} versus d for u = 0.

equilibrium state corresponding to depressed cortisol concentration, x_{4}, corresponding to the lower branch of the curve in Figure 7.15. When the stress subsides, that is, d = 0, the individual will stay in the new equilibrium state corresponding to a depressed cortisol concentration. This is because the equilibrium corresponding to d = 0 is asymptotically stable and so states “close” to it are attracted by this asymptotically stable low cortisol equilibrium. We note that the middle branch in Figure 7.15, for the values of the stressor d where there are three equilibria, corresponds to unstable equilibria. As an illustration of the above statement, we show the equilibrium states corresponding to d = 0.1; we have three equilibrium states:

x_{e}^{(1)} = [ \begin{matrix}0.6727 \\ 0.0635 \\ 0.0931 \\ 0.0635 \end{matrix} ] , x_{e}^{(2)} = [ \begin{matrix}0.6840 \\ 0.0608 \\ 0.2048 \\ 0.0608 \end{matrix} ] , and x_{e}^{(3)} = [ \begin{matrix}0.7198 \\ 0.0528 \\ 0.6871 \\ 0.0528 \end{matrix} ] .

The corresponding linear models are;

\frac{d}{dt}Δx = A^{(i)}Δx + b_{2}^{(i)}Δd, i = 1, 2, 3,

where

A^{(1)} = [ \begin{matrix}−1.0000 & 0 & 0 & −4.1141 \\ 0.9442 & −10.0000 & −0.3809 & −0.5582 \\ 0 & 0 & −0.1988 & 1.0276 \\ 0 & 1.0000 & 0 & −1.0000 \end{matrix} ] , b_{2}^{(1)} = [ \begin{matrix}0.6116 \\ 0 \\ 0 \\ 0 \end{matrix} ] ,

A^{(2)} = [ \begin{matrix}−1.0000 & 0 & 0 & −4.2530 \\ 0.8892 & −10.0000 & −0.3290 & −1.1077 \\ 0 & 0 & 0.2355 & 3.8235 \\ 0 & 1.0000 & 0 & −1.0000 \end{matrix} ] , b_{2}^{(2)} = [ \begin{matrix}0.6218 \\ 0 \\ 0 \\ 0 \end{matrix} ] ,

and

A^{(3)} = [ \begin{matrix}−1.0000 & 0 & 0 & −4.7104 \\ 0.7337 & −10.0000 & −0.2047 & −2.6627 \\ 0 & 0 & −0.1859 & 9.2897 \\ 0 & 1.0000 & 0 & −1.0000 \end{matrix} ] , b_{2}^{(3)} = [ \begin{matrix}0.6544 \\ 0 \\ 0 \\ 0 \end{matrix} ] .

The eigenvalues of A^{(i)}, i = 1, 2, 3, are:

eig(A^{(1)}) = {−9.990413, −0.989890, −0.989890, −0.228634},
eig(A^{(2)}) = {−9.937246, −0.992395, −0.992395, 0.157563},
eig(A^{(3)}) = {−9.763819, −1.051757, −1.051757, −0.318609}.

The second linear model is unstable because one of the eigenvalues of A^{(2)} is positive.

Ben-Zvi et al. [29] proposed a treatment strategy whereby the chronically stressed individual is moved to the healthy state corresponding to the normal cortisol concentration, by moving the individual into a negative stress region, after which internal regulatory processes translates the individual into the healthy equilibrium which is asymptotically stable and maintains it there. Effective treatment can be accomplished when the states as well as stress levels are available. Thus a method to estimate state variables as well as the stress level are highly desirable. This subject is discussed in Chapter 12.

7.10 Discontinuous Robust Controllers

In this section, we consider the problem of constructing stabilizing state-feedback controllers for nonlinear dynamical systems whose mathematical models contain uncertainties. The design of the controllers is based on the Lyapunov theory presented in the previous sections. The class of dynamical systems considered here is modeled by

ẋ(t) = f(t, x(t)) + G(t, x(t)) (u(t) + ξ(t, x(t))) ,  x(t_{0}) = x_{0},  (7.75)

where x(t) ∈ ℝ^{n}, u(t) ∈ ℝ^{m}, and ξ(t, x(t)) ∈ ℝ^{m} represents the lumped uncertainties. We assume that the uncertain element ξ is bounded by a known continuous non-negative function ρ, that is,

∥ξ(t, x(t))∥ ≤ ρ(t, x(t)).

Stabilizing State-Feedback Controller

We assume that the equilibrium state x = 0 of the uncontrolled nominal system

ẋ(t) = f(t, x(t))  (7.76)

is globally uniformly asymptotically stable, and that we were able to construct a Lyapunov function W : ℝ × ℝ^{n} → ℝ corresponding to this equilibrium state. The Lyapunov function W is assumed to satisfy the following conditions: there exist three continuous and strictly increasing functions ψ_{i}, i = 1, 2, 3, such that

\begin{matrix}ψ_{i}(0) & = & 0, i = 1, 2, 3, \\ lim_{r→∞} ψ_{1}(r) & = & ∞, \\ ψ_{1}(∥x∥) & ≤ & W (t, x) ≤ ψ_{2}(∥x∥), \\ ψ_{3} (∥x∥) & ≤ & −Ẇ (t, x) \end{matrix} \}  (7.77)

for all t and all x. We show that if we apply the control strategy

u(t, x) = \{ \begin{matrix}−\frac{μ(t,x)}{∥μ(t,x)∥}ρ(t, x) & if & (t, x) ∉ Ω \\ 0 & if & (t, x) ∈ Ω, \end{matrix}  (7.78)

where μ(t, x) = G^{⊤}(t, x)∇xW (t, x), and Ω = {(t, x) : μ = 0}, then the equilibrium state x = 0 is a globally uniformly asymptotically stable solution of the closed-loop system. For this, consider the Lyapunov function W for the uncontrolled nominal system. The time derivative of W evaluated on the solutions of the closed-loop system is

Ẇ (t, x) = \frac{∂W}{∂t} + ∇xW^{⊤} (f(t, x) + G(t, x) (u + ξ(t, x)))
≤ −ψ_{3}(∥x∥) + ∇xW^{⊤}G(t, x)u + ∇xW^{⊤}G(t, x)ξ(t, x)
≤ −ψ_{3}(∥x∥) + μ^{⊤}(t, x)u + ∥μ(t, x)∥ρ(t, x).  (7.79)

For (t, x) ∉ Ω,

Ẇ (t, x) ≤ −ψ_{3}(∥x∥) − μ^{⊤}(t, x)\frac{μ(t, x)}{∥μ(t, x)∥}ρ(t, x) + ∥μ(t, x)∥ρ(t, x)
= −ψ_{3}(∥x∥) − \frac{∥μ(t, x)∥^{2}}{∥μ(t, x)∥} ρ(t, x) + ∥μ(t, x)∥ρ(t, x)
= −ψ_{3}(∥x∥) − ∥μ(t, x)∥ρ(t, x) + ∥μ(t, x)∥ρ(t, x)
= −ψ_{3}(∥x∥)
< 0.  (7.80)

For (t, x) ∈ Ω, we have μ(t, x) = 0, and using (7.79), we obtain

Ẇ (t, x) ≤ −ψ_{3}(∥x∥) < 0.  (7.81)

Thus, W is also a Lyapunov function for the closed-loop system, which means that the uncertain system (7.75) driven by the control law (7.78) is indeed globally uniformly asymptotically stable.

Example 7.12 For the uncertain system model

ẋ = Ax + ξ_{1}(x) + b (u + ξ_{2}(x, u)) ,

where

A = [ \begin{matrix}−2 & 1 \\ 0 & −3 \end{matrix} ] , b = [ \begin{matrix}0 \\ 1 \end{matrix} ] , ∥ξ_{1}(x)∥ ≤ 1.75∥x∥, and |ξ_{2}(x, u)| ≤ 10∥x∥ + 0.5|u|,

we design stabilizing control strategy of the form (7.78).

The uncontrolled nominal system ẋ = Ax is asymptotically stable. Let

Q = [ \begin{matrix}4 & −1 \\ −1 & 6 \end{matrix} ] .

The eigenvalues of Q are λ_{min}(Q) = 3.5858 and λ_{max}(Q) = 6.4142. Solving the Lyapunov matrix equation A^{⊤}P + P A = −Q yields

P = I_{2}.

The proposed controller has the form

u = −η(x)\frac{μ(x)}{|μ(x)|} = −η(x)\frac{b^{⊤}P x}{|b^{⊤}P x|} = −η(x)\frac{b^{⊤}x}{|b^{⊤}x|} = −η(x)sign (b^{⊤}x) .  (7.82)

Our goal is to determine a bound η^{∗}(x) > 0 such that for any η(x) ≥ η^{∗}(x) the uncertain closed-loop system is globally uniformly asymptotically stable. To begin, consider

V (x) = x^{⊤}P x = x^{⊤}x

as the Lyapunov function candidate for the closed-loop system. Evaluating the time derivative of V on the trajectories of the closed-loop system yields

\frac{d}{dt}V (x) = 2x^{⊤}ẋ
= 2x^{⊤} (Ax + ξ_{1}(x) + b (u + ξ_{2}(x, u)))
= −x^{⊤}Qx + 2x^{⊤}ξ_{1}(x) + 2x^{⊤}bu(x) + 2x^{⊤}bξ_{2}(x, u).  (7.83)

Substituting into the above the expression for the controller given by (7.82), taking the norms, and using bounds on the uncertain elements, we obtain

\frac{d}{dt}V (x) ≤ −λ_{min}(Q)∥x∥^{2} + 2∥x∥∥ξ_{1}(x)∥ − 2η(x)\begin{matrix}|x^{⊤}b| & 2 \\ | & x & ⊤ & b & | \end{matrix} + 2 |x^{⊤}b| |ξ_{2}(x, u)|
≤ (−3.5858 + 3.5)∥x∥^{2} − 2η(x) |x^{⊤}b| + 2 (10∥x∥ + 0.5η(x)) |x^{⊤}b| .(7.84)

In the above, we used the fact that

|u| = η(x).

Collecting the terms in (7.84) yields

\frac{d}{dt}V (x) ≤ (−3.5858 + 3.5)∥x∥^{2} + (20∥x∥ − η(x)) |x^{⊤}b| .  (7.85)

Therefore, \frac{dV}{dt} < 0 for any

η(x) ≥ η^{∗}(x) = 20∥x∥.  (7.86)

In summary, the uncertain closed-loop system will be globally uniformly asymptotically stable if the controller gain η(x) is chosen to satisfy (7.86).

Model-Reference Tracking Controller

We now use the Lyapunov approach to design a model-reference tracking controller. A block diagram of the model-reference control system is shown in Figure 7.16. Model-reference control can be used for plants with nonlinearities and time-varying parameters. One of the

Figure 7.16: Model-reference control system block diagram with input scaling.

assumptions that is made while designing a model-reference controller is that a plant model is in a controller companion form. Here, we propose a method for designing model-reference controllers for a class of multi-input systems that does not require a plant model to be in a specific form. The crux of our method is in an appropriate selection of the model-reference system. Consider a plant model of the form

ẋ = Ax + Bu,  (7.87)

where A ∈ ℝ^{n×n} and B ∈ ℝ^{n×m}. The model-reference signal generator is described by

ẋ_{d} = A_{d}x_{d} + B_{d}r  (7.88)

where A_{d} ∈ ℝ^{n×n}, B_{d} ∈ ℝ^{n×m}, and r ∈ ℝ^{m} is a command signal that is chosen so that x_{d} represents a desired state trajectory of the plant. The matrix A_{d} of the reference system is chosen from the set

{A_{d} : A_{d} = A − BK_{d}, K_{d} ∈ ℝ^{m×n}} ,  (7.89)

and B_{d} = B. Thus, the reference system is state-feedback equivalent with the plant model (7.87). We assume that the pair (A, B) is controllable. Therefore, we can select A_{d} to have its eigenvalues in prescribed locations in the open-left hand plane. To proceed further, we define the tracking error e as

e(t) = x(t) − x_{d}(t).  (7.90)

Our goal is to construct a state-feedback control law so that

lim_{t→∞} ∥e(t)∥ = 0.  (7.91)

We form the differential equation that describes the error dynamics. For this we evaluate the time derivative of the error vector to get

ė(t) = ẋ(t) − ẋ_{d}(t)
= Ax + Bu − A_{d}x_{d} − Br
= Ae + B(u − r).  (7.92)

Recall that A_{d} = A − BK_{d}, hence A = A_{d} + BK_{d}. Substituting this expression for A into the error equation (7.92), we obtain

ė(t) = (A_{d} + BK_{d})e + B(u + −r)
= A_{d}e(t) + B(u + K_{d}x − r).  (7.93)

We use a Lyapunov function approach in our further analysis that to obtain a control law forcing the error e(t) to go asymptotically to zero. As a Lyapunov function candidate for (7.93), we take

V (e) = e^{⊤}P e,  (7.94)

where P solves the Lyapunov matrix equation, A_{d}^{⊤}P + P A_{d} = −Q, for some Q = Q^{⊤} > 0. Because by construction A_{d} is asymptotically stable, the solution to the Lyapunov matrix equation is positive definite, that is, P = P^{⊤} > 0. We next evaluate the time derivative of V (e(t)) on the trajectories of (7.93),

V̇ (e) = 2e^{⊤}P ė
= 2e^{⊤}P (A_{d}e + B(u + K_{d}x − r))
= e^{⊤}(A_{d}^{⊤}P + P A_{d})e + 2e^{⊤}P B(u + K_{d}x − r).  (7.95)

Using the Lyapunov matrix equation, A_{d}^{⊤}P + P A_{d} = −Q, and taking the norms of the above yields

V̇ (e) ≤ −e^{⊤}Qe + 2e^{⊤}P Bu + 2 ‖e^{⊤}P B‖ ∥K_{d}x∥ + 2 ‖e^{⊤}P B‖ ∥r∥.  (7.96)

If e is such that B^{⊤}P e = 0, then

V̇ (e) ≤ −e^{⊤}Qe < 0.  (7.97)

We next analyze the case when B^{⊤}P e ≠ 0. Let the proposed control law be

u = −ρ(x, r)\begin{matrix}B & ⊤ & P & e \\ ‖ & B & ⊤ & P & e & ‖ \end{matrix},  (7.98)

where ρ(x, r) = ∥K_{d}x∥ + ∥r∥. Substituting (7.98) into (7.96), we obtain

V̇ (e) ≤ −e^{⊤}Qe − 2 ‖e^{⊤}P B‖ (ρ(x, r) − ∥K_{d}x∥ − ∥r∥)
= −e^{⊤}Qe
< 0.

Since for all e ∈ ℝ^{n}, we have V (e) > 0 and V̇ (e) < 0, the equilibrium point e = 0 of the error equation (7.93) is globally uniformly asymptotically stable, which means that x(t) → x_{d}(t).

Example 7.13 We apply the above tracking controller to the vehicle model derived in the Subsection 2.4.2, where δ_{r} = 0. That is, we wish to construct a tracking controller for a front wheel steered vehicle. The first step is to select the poles of the model reference system. We choose the model reference system to have the following set of asymptotically stable poles:

{−4 − 2j, −4 + 2j, −4.5, −5.0}.  (7.99)

The pair (A, b) is controllable, hence we can find k_{d} such that the eigenvalues of the matrix A_{d} = A − bk_{d} are at the above selected locations. The feedback gain is

k_{d} = [ 0.0329 2.2854 0.2429 −0.1653 ] ,  (7.100)

and the resulting model reference plant matrix is

A_{d} = [ \begin{matrix}−6.6638 & −107.1281 & −29.6350 & 7.7479 \\ 0 & 0 & 1 & 0 \\ −0.9205 & −65.8195 & −10.8362 & 4.7603 \\ −1 & −18.3 & 0 & 0 \end{matrix} ] .  (7.101)

The next step is to find P that solves the Lyapunov equation, A_{d}^{⊤}P + P A_{d} = −Q. We take Q = I_{4}. Solving the Lyapunov equation, we get

P = [ \begin{matrix}0.1145 & 0.7530 & −0.1809 & −0.0966 \\ 0.7530 & 34.6174 & −0.2883 & −3.3437 \\ −0.1809 & −0.2883 & 0.5144 & 0.0522 \\ −0.0966 & −3.3437 & 0.0522 & 0.6216 \end{matrix} ] .  (7.102)

Now that we have P, we are ready to construct the controller. We have

u = −ρ(x, r)\begin{matrix}b & ⊤ & P & e \\ | & b & ⊤ & P & e & | \end{matrix}
= −(|k_{d}x| + |r|)sign(b^{⊤}P (x − x_{d}))

= − (|[ \begin{matrix}0.0329 \\ 2.2854 \\ 0.2429 \\ −0.1653 \end{matrix} ]^{⊤} x| + |r|) sign ([ \begin{matrix}0.1571 \\ 26.9931 \\ 6.3324 \\ −3.0250 \end{matrix} ]^{⊤} (x − x_{d})) . (7.103)

Our goal is to have the reference input signal u_{r} to correspond to a desired vehicle path. This requires scaling the reference input. The scaling factor is

η = (lim_{s→0} c(sI_{4} − A_{d})^{−1}b)^{−1} ,  (7.104)

where c = [ 0 0 0 1 ]. Note that η is well defined because the matrix A_{d} is asymptotically stable. In our example, η = −0.1653. Thus, we have r = ηu_{r}. The reference input is u_{r} = 41(t − 1 sec), where 1(t) is the unit step function.

The performance of the controller, for zero initial conditions, is shown in Figures 7.17 and 7.18. In Figure 7.17, the closed-loop system output is shown in the upper subplot, while the desired output is shown in the bottom subplot. The vehicle tracks the desired path with no noticeable steady-state error. The control action is shown in Figure 7.18.

The proposed tracking controller is discontinuous in the state, which results in its high activity referred to as chattering. Chattering can be reduced by using a continuous version of the above controller. The continuous controller is obtained by introducing a so-called boundary layer that smooths out the discontinuous controller. We discuss the problem of constructing boundary layer controllers in the following section.

7.11 Uniform Ultimate Boundedness

A drawback to the discontinuous controllers, discussed in the previous section, is that rapid switching tends to excite high frequency modes of the plant and this may degrade the performance of the controller. These problems can be alleviated by adding a so called boundary 378 CHAPTER 7. STABILITY Figure 7.17: Plots of the actual output of the closed-loop system and the desired output in Example 7.13.

Figure 7.18: Plot of the controller action versus time in Example 7.13. layer to the discontinuous controller resulting in a continuous control strategy. However, there is a price for approximating a discontinuous controller with a continuous one. The closed-loop system driven by a boundary layer controller may only be uniformly ultimately bounded rather than asymptotically stable. The notions we are concerned with in this section are described in the following three definitions.

Definition 7.18 The closed ball B̄_{q}(0) = {x : ∥x∥ ≤ q} is uniformly stable if for any ε > q, there is a δ = δ(ε) > 0 so that if ∥x(t_{0})∥ ≤ δ, then ∥x(t)∥ ≤ ε for t ≥ t_{0}.

◇

For the benefit of the reader, we now re-phrase Definition 7.16 of uniform boundedness from Section 7.8.

Definition 7.19 The solutions of ẋ = f(t, x) are uniformly bounded if for any given d > 0, there exists b = b(d) < ∞ so that for each solution x(t) = x(t; t_{0}, x_{0}) with ∥x_{0}∥ ≤ d, we have ∥x(t)∥ ≤ b for all t ≥ t_{0}.

◇

We next define the notion of uniform ultimate boundedness (UUB) of a system.

Definition 7.20 The solutions of ẋ = f(t, x) are uniformly ultimately bounded (UUB) with respect to the ball B̄_{q}(0) if for every d > 0, there exists T (q, d) such that for each solution x(t) = x(t; t_{0}, x_{0}) with ∥x_{0}∥ ≤ d, we have x(t) ∈ B̄_{q}(0) for t ≥ t_{0} + T (q, d).

◇

Recall a dynamical system model (7.75),

ẋ(t) = f(t, x(t)) + G(t, x(t)) (u(t) + ξ(t, x(t))) , x(t_{0}) = x_{0},

where x(t) ∈ ℝ^{n}, u(t) ∈ ℝ^{m}, and ξ(t, x(t)) ∈ ℝ^{m} represents the lumped uncertainties. We assume that the uncertain element ξ is bounded by a known continuous non-negative function ρ, that is, ∥ξ(t, x(t))∥ ≤ ρ(t, x(t)). As in Section 7.10, we assume that the equilibrium state x = 0 of the uncontrolled nominal system, ẋ(t) = f(t, x(t)), is globally uniformly asymptotically stable, and that we were able to construct a Lyapunov function W : ℝ×ℝ^{n} → ℝ satisfying the conditions (7.77) for all t and all x.

We showed in the previous section that if we apply the discontinuous control strategy

u(t, x) = \{ \begin{matrix}−\frac{μ(t,x)}{∥μ(t,x)∥}ρ(t, x) & if & μ(t, x) ≠ 0 \\ 0 & if & μ(t, x) = 0, \end{matrix}

where μ(t, x) = G(t, x)^{⊤}∇xW (t, x), then the equilibrium state x = 0 is a globally uniformly asymptotically stable solution of the closed-loop system. However, the above controller is discontinuous in x. Following Corless and Leitmann [69], we approximate it with a continuous controller by introducing a boundary layer. The proposed boundary layer controller is

u(t, x) = \{ \begin{matrix}−\frac{μ(t,x)ρ(t,x)}{∥μ(t,x)ρ(t,x)∥}ρ(t, x) & if & ∥μ(t, x)ρ(t, x)∥ > ν \\ p(t, x), ∥p(t, x)∥ ≤ ρ(t, x) & if & ∥μ(t, x)ρ(t, x)∥ ≤ ν, \end{matrix}  (7.105)

where ν is the thickness of the boundary layer about the switching surface {(t, x) : ∥μ(t, x)ρ(t, x)∥ = 0}, and p is a continuous function such that

p = −\frac{μ(t, x)ρ(t, x)}{∥μ(t, x)ρ(t, x)∥}ρ(t, x) if ∥μ(t, x)ρ(t, x)∥ = ν.

A particular example of the above type of a controller is

u(t, x) = \{ \begin{matrix}−\frac{μ(t,x)ρ(t,x)}{∥μ(t,x)ρ(t,x)∥}ρ(t, x) & if & ∥μ(t, x)ρ(t, x)∥ > ν \\ −\frac{μ(t,x)}{ν}ρ^{2}(t, x) & if & ∥μ(t, x)ρ(t, x)∥ ≤ ν. \end{matrix}  (7.106)

We show that the closed-loop system driven by the boundary layer controller (7.105) is UUB with respect to a neighborhood of the zero state. This neighborhood can be made arbitrarily small by letting ν → 0. Before we state the results, we first compute the time derivative of a Lyapunov function candidate on the trajectories of the closed-loop system. Consider the Lyapunov function W for the uncontrolled nominal system. The time derivative of W evaluated on the solutions of the closed-loop system is

Ẇ (t, x) = \frac{∂W}{∂t} + ∇xW^{⊤} (f(t, x) + G(t, x) (u + ξ(t, x)))
≤ −ψ_{3}(∥x∥) + ∇xW^{⊤}G(t, x)u + ∥μ(t, x)ρ(t, x)∥.

We now consider two cases. The first case is when ∥μ(t, x)ρ(t, x)∥ > ν. Taking into account the controller structure for this case yields

Ẇ (t, x) ≤ −ψ_{3}(∥x∥) − ∇xW^{⊤}G(t, x)\frac{μ(t, x)ρ(t, x)}{∥μ(t, x)ρ(t, x)∥}ρ(t, x) + ∥μ(t, x)ρ(t, x)∥
= −ψ_{3}(∥x∥) − \frac{∥μ(t, x)ρ(t, x)∥^{2}}{∥μ(t, x)ρ(t, x)∥} + ∥μ(t, x)ρ(t, x)∥
= −ψ_{3}(∥x∥).

If, on the other hand, ∥μ(t, x)ρ(t, x)∥ ≤ ν, then using (7.105), we obtain

Ẇ (t, x) ≤ −ψ_{3}(∥x∥) + ∇xW^{⊤}G(t, x)u + ∥μ(t, x)ρ(t, x)∥
≤ −ψ_{3}(∥x∥) + ∥∇xW^{⊤}G(t, x)∥ ∥u∥ + ∥μ(t, x)ρ(t, x)∥
≤ −ψ_{3}(∥x∥) + 2∥μ(t, x)ρ(t, x)∥
≤ −ψ_{3}(∥x∥) + 2ν.

We conclude that for all t and all x, we have

Ẇ (t, x) ≤ −ψ_{3}(∥x∥) + 2ν  (7.107)

To proceed further, we need an assumption on the function ψ_{3}; namely if ψ_{3} is bounded and

lim_{r→∞} ψ_{3}(r) = l < ∞,  (7.108)

then we assume

2ν < l.  (7.109)

Proposition 7.2 The solutions of the system (7.75) driven by the boundary layer controller (7.105) are uniformly bounded.

Proof Let R be such that

ψ_{3}(R) = 2ν.

Note that R is well defined by virtue of (7.109) and the fact that ψ_{3} is strictly increasing. Because ψ_{3} is strictly increasing, we can write

R = ψ_{3}^{−1}(2ν).

Figure 7.19 illustrates the above construction.

For a given d > 0, where ∥x_{0}∥ ≤ d, let

d̃= max{d, R}.

We claim that

b = b(d) = ψ_{1}^{−1}(ψ_{2}(d̃)),

that is, the solutions of (7.75) driven by the boundary layer controller (7.105) are uniformly bounded. In other words, if ∥x_{0}∥ ≤ d then ∥x(t; t_{0}, x_{0})∥ ≤ b for t ≥ t_{0}. First note that 382 CHAPTER 7. STABILITY

Figure 7.19: Finding R for a given ν in the proof of Proposition 7.2. Figure 7.20: Finding b for a given d in the proof of Proposition 7.2.

∥x_{0}∥ ≤ d̃ and that ψ_{1}(b) = ψ_{2}(d̃). Hence, d̃ ≤ b—see Figure 7.20 for an illustration of this construction, and thus

Ẇ (t, x(t)) ≤ −ψ_{3}(∥x(t)∥) + 2ν
= −ψ_{3}(∥x(t)∥) + ψ_{3}(R).  (7.110)

Suppose first that d > R. Then for ∥x_{0}∥ > R and ∥x(t; t_{0}, x_{0})∥ > R, Ẇ < 0 and therefore

ψ_{1}(b) = ψ_{2}(d̃)
≥ W (t_{0}, x(t_{0}; t_{0}, x_{0}))
≥ W (t, x(t; t_{0}, x_{0}))
≥ ψ_{1}(∥x(t; t_{0}, x_{0})∥),

which implies that ∥x(t; t_{0}, x_{0})∥ ≤ b for t ≥ t_{0}.

If, on the other hand, ∥x_{0}∥ ≤ R, then because Ẇ (t, x(t)) ≤ −ψ_{3}(∥x(t)∥) + 2ν, the solution may reach the sphere ∥x∥ = R at some time t_{2} ≥ t_{0}. However, by (7.110), Ẇ < 0 outside ∥x∥ = R. Therefore, ∥x(t; t_{0}, x_{0})∥ ≤ b. Thus the solution is uniformly bounded.

□

We illustrate Proposition 7.2 with Figure 7.21. The initial condition in Figure 7.21 is such that ∥x_{0}∥ < d and d < R. The trajectory is confined to stay within the ball of radius b = ψ_{1}^{−1}(ψ_{2}(R)).

We now show, using arguments similar to those of Corless and Leitmann [69], that the solutions of the closed-loop system are UUB with respect to any ball of radius b̃ > ψ_{1}^{−1}(ψ_{2}(R)).

Theorem 7.13 Let x(t) = x(t; t_{0}, x_{0}) be a solution of the closed-loop system for [t_{0}, ∞) such that (7.107) is satisfied. If ∥x_{0}∥ ≤ d, then for a given

b̃ > ψ_{1}^{−1}(ψ_{2}(R)),  (7.111)

∥x(t; t_{0}, x_{0})∥ ≤ b̃ for all t ≥ t_{0} + T (b̃, d), where

T (b̃, d) = \{ \begin{matrix}0 & if & d ≤ R̃ \\ \frac{ψ_{2}(d)−ψ_{1}(R̃)}{ψ_{3}(R̃)−2ν} & if & d > R̃, \end{matrix}

and

ψ_{2} \binom{}{R̃} = ψ_{1}(b̃).  (7.112)

Figure 7.21: A two-dimensional illustration of an uniformly ultimately bounded (UUB) solution.

Proof From assumption (7.111),

ψ_{1}(b̃) > ψ_{2}(R),

and by (7.112),

ψ_{1}(b̃) = ψ_{2}(R̃).

Hence, by (7.77), where we assumed that ψ_{2} is strictly increasing,

R̃ > R.  (7.113)

We consider two cases. The first case is when d ≤ R̃. By uniform boundedness, for all t ≥ t_{0},

∥x(t)∥ ≤ b̃,

for any b̃ such that ψ_{1}(b̃) > ψ_{2}(R̃). Hence in this case T (b̃, d) = 0.

We now consider the case when d > R̃. We show, by contradiction, that for some t_{1} ∈ [t_{0}, t_{0} + T (b̃, d)],

∥x(t_{1})∥ ≤ R̃.

From this we shall conclude the result by referring to the previous case. So suppose that ∥x(t)∥ > R̃ for all t ∈ [t_{0}, t_{0} + T (b̃, d)]. Then,

ψ_{1} (‖x (t_{0} + T (b̃, d); t_{0}, x_{0})‖) ≤ W (t_{0} + T (b̃, d), x (t_{0} + T (b̃, d); t_{0}, x_{0}))
= W (t_{0}, x_{0}) + ∫_{t_{0}}^{t_{0}+T (b̃,d)} Ẇ dt
≤ W (t_{0}, x_{0}) + ∫_{t_{0}}^{t_{0}+T (b̃,d)} (−ψ_{3}(R̃) + 2ν) dt
≤ ψ_{2}(d) − ∫_{t_{0}}^{t_{0}+T (b̃,d)} (ψ_{3}(R̃) − 2ν) dt.

Using the fact that ψ_{3}(R̃) − 2ν > 0 because by (7.113), R̃ > R, we obtain

ψ_{1} (‖x (t_{0} + T (b̃, d); t_{0}, x_{0})‖) ≤ ψ_{2}(d) − (ψ_{3}(R̃) − 2ν) T (b̃, d)
= ψ_{2}(d) − (ψ_{3}(R̃) − 2ν) \frac{ψ_{2}(d) − ψ_{1}(R̃)}{ψ_{3}(R̃) − 2ν}
= ψ_{1} \binom{}{R̃} .

Thus, because ψ_{1} is strictly increasing,

‖x (t_{0} + T (b̃, d); t_{0}, x_{0})‖ ≤ R̃,

which contradicts our assumption that for all t ∈ [t_{0}, t_{0} + T (b̃, d)] it should have been ∥x(t)∥ > R̃. Hence, there exists t_{1} ∈ [t_{0}, t_{0} + T (b̃, d)] such that ∥x(t_{1})∥ ≤ R̃. Then using the same argument as in the previous case we can show that

∥x(t)∥ ≤ b̃

for all t ≥ t_{1} and the proof is complete.

□

We have the following consequence of the above theorem.

Corollary 7.2 The ball

{x : ∥x∥ ≤ ψ_{1}^{−1}(ψ_{2}(R))}

is uniformly stable.

386 CHAPTER 7. STABILITY Figure 7.22: Finding δ given ε for an uniformly stable ball in the proof of Corollary 7.2. Proof Let

ε > ψ_{1}^{−1}(ψ_{2}(R)),

and consider δ = δ(ε) obtained from

ψ_{1}(ε) = ψ_{2}(δ).

Obviously δ ≥ R. Figure 7.22 illustrates the above relations. Hence, by uniform boundedness if ∥x_{0}∥ ≤ δ, then ∥x(t)∥ ≤ ε for all t ≥ t_{0} which means that the ball is uniformly stable.

□

Example 7.14 Consider a dynamical system model

ẋ = Ax + b(u + h(t, x)),

where

A = [ \begin{matrix}0 & 1 \\ −1 & −1 \end{matrix} ] , b = [ \begin{matrix}0 \\ 1 \end{matrix} ] , and |h(t, x)| ≤ 2.

Let P be the solution of the matrix Lyapunov equation A^{⊤}P + P A = −2I_{2}, and let

u = \{ \begin{matrix}−2 sign(b^{⊤}P x) & if & |b^{⊤}P x| ≥ ν \\ −2\frac{b^{⊤}P x}{ν} & if & |b^{⊤}P x| < ν, \end{matrix}

where ν = 1. Suppose that the initial condition, x(0), is such that ∥x(0)∥_{2} ≤ 10. Find the radius, b, of a ball such that ∥x(t)∥_{2} ≤ b for all t ≥ 0.

Solving the Lyapunov equation A^{⊤}P + P A = −2I_{2} yields

P = [ \begin{matrix}3 & 1 \\ 1 & 2 \end{matrix} ] .

We next evaluate \frac{d}{dt} (x^{⊤}P x) on the trajectories of the closed-loop system. For the case when |b^{⊤}P x| ≥ ν, we get

\frac{d}{dt} (x^{⊤}P x) ≤ −2∥x∥^{2}.

For the case when |b^{⊤}P x| < ν, we obtain

\frac{d}{dt} (x^{⊤}P x) = −2∥x∥^{2} + 2x^{⊤}P b(u + h)
≤ −2∥x∥^{2} − 4\frac{|x^{⊤}P b|^{2}}{ν} + 4 |x^{⊤}P b|
≤ −2∥x∥^{2} + 4ν
= −2∥x∥^{2} + 4.

Hence, \frac{d}{dt} (x^{⊤}P x) < 0 outside of the ball

B̄_{R}(0) = \{x : ∥x∥ ≤ R = \sqrt{2}\}.

To find the radius, b, of a ball, such that ∥x(t)∥_{2} ≤ b for all t ≥ 0 and ∥x(0)∥_{2} ≤ 10 = d, we first observe that

λ_{min}(P)∥x∥^{2} = ψ_{1}(∥x∥) ≤ x^{⊤}P x ≤ ψ_{2}(∥x∥) = λ_{max}(P)∥x∥^{2}.

Let d̃= max{d, R} = 10. Then, the radius b is computed by solving the equation

ψ_{1}(b) = ψ_{2}(d̃),

that is,

λ_{min}(P)b^{2} = λ_{max}(P)d̃^{2}.

We obtain

b = d̃\sqrt{\frac{λ_{max}(P)}{λ_{min}(P)}} = 10\sqrt{\frac{(5 + \sqrt{5})/2}{(5 − \sqrt{5})/2}} = 16.18.

Figure 7.23: Plot of the linear saturation function described by (7.115).

Example 7.15 In this example, we investigate the performance of a boundary layer tracking controller on the tracking problem considered in Example 7.13. Specifically, we approximate the discontinuous controller given by (7.103) with a boundary layer continuous controller obtained by replacing the sign nonlinearity with a linear saturation nonlinearity. A linear saturation nonlinearity, denoted sat_{ν}, can be described as

sat_{ν}(x) = \frac{1}{2ν} (|x + ν| − |x − ν|) ,  (7.114)

where the parameter ν controls the width of the boundary layer. In other words,

sat_{ν}(x) = \{ \begin{matrix}1 & if & x ≥ ν \\ \frac{1}{ν}x & if & −ν < x < ν \\ −1 & if & x ≤ −ν. \end{matrix}  (7.115)

A plot of (7.114) for ν = 0.5 is shown in Figure 7.23. A boundary layer approximation of

Figure 7.24: Plot of the boundary layer controller action versus time in Example 7.15.

the discontinuous tracking controller (7.103) is

u = −ρ(x, r)sat_{ν} (b^{⊤}P e)
= −(|k_{d}x| + |r|)sat_{ν}(b^{⊤}P (x − x_{d}))

= − (|[ \begin{matrix}0.0329 \\ 2.2854 \\ 0.2429 \\ −0.1653 \end{matrix} ]^{⊤} x| + |r|) sat_{ν} ([ \begin{matrix}0.1571 \\ 26.9931 \\ 6.3324 \\ −3.0250 \end{matrix} ]^{⊤} (x − x_{d})) . (7.116)

The plot of the actual output, Y, of the closed-loop system driven by (7.116) for ν = 0.5 and zero initial conditions was indistinguishable from that shown in Figure 7.17 when the discontinuous controller was used. The boundary layer controller action versus time is shown in Figure 7.24. Chattering is basically eliminated in this case.

7.12 Lyapunov Like Analysis

Usually, it is challenging to analyze the asymptotic stability of time-varying systems because it is very difficult to find Lyapunov functions with negative definite Lyapunov derivatives.

One may ask the following question: when can we conclude asymptotic stability when the Lyapunov derivative is only negative semi-definite?

In this section, we present a number of results that attempt to answer our question. In addition these results are very useful in the analysis and design of adaptive controllers. To proceed, we review some properties of time-varying functions. In particular, we need a condition that guarantees that ḟ(t) → 0 as t → ∞.

We present some basic properties of a function f : ℝ → ℝ in the following list,

• Recall that ḟ(t) → 0 does not imply that f(t) has a limit as t → ∞. On the other hand, f(t) has a limit as t → ∞ does not imply that ḟ(t) → 0.

• A well-known result from calculus states that if f(t) is lower bounded and decreasing, that is, ḟ(t) ≤ 0, then it converges to a limit. But we do not know whether ḟ(t) → 0 or not as t → ∞.

• Recall also that a function f(t) is continuous if small changes in t result in small changes in its value f(t). Intuitively, a continuous function is a function whose graph can be drawn without lifting the pencil from the paper.

• Continuity is a local property of a function, that is, a function f is continuous, or not, at a particular point t.

• A function being continuous on an interval means only that it is continuous at each point of the interval.

• A function f is uniformly continuous if it is continuous and, in addition, the size of the changes in f(t) depends only on the size of the changes in t but not on t itself.

• The slope of an uniformly continuous function is bounded, that is, ḟ is bounded.

• Uniform continuity is a global property of a function.

• Every uniformly continuous function is continuous, but the converse is not true.

• A function is uniformly continuous, or not, on an entire interval.

• A function may be continuous at each point of an interval without being uniformly continuous on the entire interval.

Example 7.16 The function f(t) = sin(t) is uniformly continuous. Note that the slope of the above function is bounded.

Example 7.17 The function f(t) = 1/t is continuous but not uniformly continuous on positive real numbers. Note that as t approaches 0, the changes in f(t) become unbounded.

In the next example from Slotine and Li [301, p. 124], we show that in an asymptotically stable LTI system with bounded input, the output is uniformly continuous.

Example 7.18 We consider an asymptotically stable LTI system

ẋ = Ax + u
y = Cx.

The state x(t) is bounded because u is bounded and A is asymptotically stable. Because x is bounded and u is bounded, we have that ẋ is bounded as well. We now consider the time derivative of the output,

ẏ = Cẋ.

The time derivative of y is bounded because ẋ is bounded. Hence, y is uniformly continuous.

With the above background, we are now ready to state and prove the following lemma.

Lemma 7.4 (Barbalat) Given a differentiable function f = f(t) : ℝ → ℝ such that ḟ is uniformly continuous. If lim_{t→∞} f(t) exists, then

lim_{t→∞} ḟ(t) = 0.

Proof We prove the lemma by contraposition. We assume that ḟ(t) does not converge to 0 as t → ∞. Therefore, there exists an ε > 0 such that for every T > 0 we can find t_{i} ≥ T so that |ḟ(t_{i})| ≥ ε. By assumption ḟ is uniformly continuous. Hence, given ε/2 > 0 there is δ_{ε/2} > 0 so that for every t_{i} and every τ ∈ [0, δ_{ε/2}], we have

|ḟ(t_{i} + τ) − ḟ(t_{i})| < \frac{ε}{2}.  (7.117)

Hence, for all t ∈ [t_{i}, t_{i} + δ_{ε/2}], we have

|ḟ(t)| = |ḟ(t) − ḟ(t_{i}) + ḟ(t_{i})|
≥ |ḟ(t_{i})| − |ḟ(t) − ḟ(t_{i})|
> ε − \frac{ε}{2}
= \frac{ε}{2}.

Suppose that ḟ(t_{i}) > 0. (The other case is when ḟ(t_{i}) < 0 and its analysis is similar to the case we analyze now.) It follows from the above that for any t ∈ [t_{i}, t_{i} + δ_{ε/2}],

ḟ(t) > \frac{ε}{2}.

Using the above, we obtain

∫_{t_{i}}^{t_{i}+δ_{ε/2}} ḟ(t)dt > ∫_{t_{i}}^{t_{i}+δ_{ε/2}} \frac{ε}{2}dt = \frac{ε}{2}δ_{ε/2}.  (7.118)

However,

∫_{t_{i}}^{t_{i}+δ_{ε/2}} ḟ(t)dt = f(t_{i} + δ_{ε/2}) − f(t_{i}).  (7.119)

Combining (7.118) and(7.119) yields

f(t_{i} + δ_{ε/2}) − f(t_{i}) > \frac{ε}{2}δ_{ε/2} for every i,

which contradicts the assumption that f(t) has a finite limit as t → ∞. The proof of the lemma is complete.

□

A simple sufficient condition for a differentiable function to be uniformly continuous is boundedness of its derivative. Thus if a function f is twice differentiable, then its derivative ḟ is uniformly continuous if its second derivative f̈ is bounded.

The following lemma, called by Slotine and Li [301, page 125] the “Lyapunov-like lemma”, follows immediately from Barbalat’s lemma.

Lemma 7.5 (“Lyapunov-Like Lemma”) Given a real-valued function W (t, x) such that

1. W (t, x) is bounded below;

2. Ẇ (t, x) is negative semi-definite;

3. Ẇ (t, x) is uniformly continuous in time,

then

Ẇ (t, x) → 0 as t → ∞.

Example 7.19 We investigate stability of the following dynamical system model:

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}−2x_{1} + x_{2}u \\ −x_{1}u \end{matrix} ] ,  (7.120)

where the input u is bounded.

Consider the Lyapunov function candidate

V = ∥x∥^{2} = x_{1}^{2} + x_{2}^{2}.

Note that V is bounded below. The time derivative of V evaluated on the trajectories of the system model (7.120) is

V̇ = 2x_{1}ẋ_{1} + 2x_{2}ẋ_{2} = 2x_{1} (−2x_{1} + x_{2}u) − 2x_{2}x_{1}u = −4x_{1}^{2} ≤ 0.

Thus, the system modeled by (7.120) is uniformly stable, and hence x_{1} and x_{2} are bounded. We use Lemma 7.5 to show that in addition, x_{1}(t) → 0 as t → ∞. To use this lemma, we have to show that V̇ is uniformly continuous in t. For this it is enough to show that the derivative of V̇ is bounded. The derivative of V̇ is

V̈ = −8x_{1}ẋ_{1} = −8x_{1} (−2x_{1} + x_{2}u) ,

which is bounded because u is bounded by assumption and we showed that x_{1} and x_{2} are also bounded. Hence, V̇ is uniformly continuous. The function V satisfies all the conditions of Lemma 7.5 and therefore x_{1}(t) → 0 as t → ∞.

Example 7.20 Consider a dynamical system model,

\begin{matrix}ẋ & = & Ax + bu \\ e & = & cx \end{matrix} \}  (7.121)

where

u = ϕ^{⊤}ξ  (7.122)

The components ϕ_{i} = ϕ_{i}(t) of the vector-valued function ϕ = ϕ(t) ∈ ℝ^{K} are adjustable parameters according to the adaptation law

ϕ̇ = −Γeξ,  (7.123)

where Γ = Γ^{⊤} > 0 is a design parameter matrix and

e = cx.

The vector-valued function ξ = ξ(t) ∈ ℝ^{K} is bounded for all t. The matrix A ∈ ℝ^{n×n} is asymptotically stable, and b ∈ ℝ^{n}. The row vector c ∈ ℝ^{1×n} satisfies the equation

c = b^{⊤}P ,  (7.124)

where P = P^{⊤} > 0 is the solution of the matrix Lyapunov equation

A^{⊤}P + P A = −2Q

for some Q = Q^{⊤} > 0, where the scalar 2 that multiplies Q is just for convenience to simplify manipulations.

Using the Lyapunov function candidate,

V (x, ϕ) = \frac{1}{2}x^{⊤}P x + \frac{1}{2}ϕ^{⊤}Γ^{−1}ϕ  (7.125)

and the Lyapunov-like lemma, we show that x and ϕ are both bounded and that

lim_{t→∞} x(t) = 0.

We first evaluate the time derivative of the Lyapunov function candidate V = V (x, ϕ) on the trajectories of the closed-loop system,

\begin{matrix}ẋ & = & Ax + b(ϕ^{⊤}ξ) \\ ϕ̇ & = & −Γeξ \end{matrix} \}  (7.126)

Then, we have

V̇ = x^{⊤}P ẋ + ϕ^{⊤}Γ^{−1}ϕ̇
= x^{⊤}P (Ax + b (ϕ^{⊤}ξ)) − ϕ^{⊤}eΓ^{−1}Γξ
= −x^{⊤}Qx + (eϕ^{⊤} − eϕ^{⊤}) ξ
= −x^{⊤}Qx,

where we used the fact that c = b^{⊤}P and hence e = cx = x^{⊤}c^{⊤} = x^{⊤}P b. Thus, V̇ is

negative semi-definite in the [ \begin{matrix}x \\ ϕ \end{matrix} ]-space, which implies that x and ϕ are bounded.

To show the convergence of x, we use the Lyapunov-like lemma. For this, we first show that V̇ is uniformly continuous. Indeed, because

V̈ = −2x^{⊤}Qẋ

is bounded, since x and ẋ are bounded, V̇ is uniformly continuous. Hence, V̇ → 0 as t → ∞. Since V̇ = −x^{⊤}Qx and Q = Q^{⊤} > 0, we have to have x(t) → 0 as t → ∞.

We can re-state Barbalat’s lemma in alternative form as follows (see, for example, [178, page 192]).

Lemma 7.6 Let ϕ : ℝ → ℝ be a uniformly continuous function on [0, ∞). Suppose that lim_{t→∞} ∫_{0}^{t} ϕ(τ)dτ exists and is finite. Then,

ϕ(t) → 0 as t → ∞.

7.13 Input-Output Stability of LTI Systems

The response of a linear time-invariant (LTI) system can be decomposed as the zero-state response and the zero-input response. In the previous sections we studied methods that can be used to investigate the stability properties of the zero-input response of linear systems. Here, we devote ourselves to investigating the stability properties of the zero-state response. To proceed, we need a definition of a bounded signal.

Definition 7.21 A signal u(t) is bounded if there exists a constant B < ∞ such that

|u(t)| ≤ B for all t ≥ 0.
◇

Using the above definition, we define the bounded-input bounded-output (BIBO) stability of an LTI causal system. This type of stability is concerned with a particular property of the zero-state response of the system.

Definition 7.22 A system is BIBO stable if every bounded input produces a bounded output.

◇

Example 7.21 Consider an LTI system whose transfer function is

\frac{Y (s)}{U(s)} = G(s) = \frac{1}{s},

which is an integrator. A block diagram of this system is shown in Figure 7.25. The

[figure: U(s); \frac{1}{s}; Y (s)]

Figure 7.25: System of Example 7.21. This system is not BIBO stable.

integrator is not BIBO stable. Indeed, suppose that the system’s input is the unit step, that is, u(t) = 1(t). The Laplace transform of the unit step is U(s) = 1/s. Then the Laplace transform of the system’s output is

Y (s) = \frac{1}{s^{2}} ,

and therefore

y(t) = L^{−1} (\frac{1}{s^{2}} ) = t1(t),

which is an unbounded signal.

Theorem 7.14 A single-input single-output (SISO) continuous LTI causal system is BIBO stable if and only if its impulse response, g(t), is absolutely integrable on [0, ∞), that is,

∫_{0}^{∞} |g(t)|dt ≤ M < ∞

for some constant M.

Proof We prove that the absolute integrability of the impulse response is sufficient for BIBO stability. For a proof that the absolute integrability of the impulse response is also necessary for BIBO stability, we refer to Chen [55, p. 122].

So assume that g(t) is absolutely integrable and let the system’s input u(t) be bounded on [0, ∞). Then, using the formula for an LTI system response given by (1.10) on page 21, we obtain

|y(t)| = |∫_{0}^{t} g(τ)u(t − τ)dτ|
≤ ∫_{0}^{t} |g(τ)||u(t − τ)|dτ
≤ B ∫_{0}^{t} |g(τ)|dτ
≤ BM,

which means that the output y(t) is bounded for any bounded input u(t).

□

Example 7.22 The system whose block diagram is depicted in Figure 7.26 is BIBO stable. Indeed, the system’s impulse response is

[figure: U(s); \frac{2}{s+1}; Y (s)]

Figure 7.26: System of Example 7.22. This system is BIBO stable.

g(t) = L^{−1}(G(s))
= L^{−1} (\frac{2}{s + 1})
= 2e^{−t}1(t).

The above impulse response is absolutely integrable because

∫_{0}^{∞} |2e^{−t}| dt ≤ 2 < ∞.

Therefore, the system in this example is BIBO stable.

In the case of a discrete SISO LTI causal relaxed at k = 0 system, its output can be expressed as given by (1.24) on page 26, that is,

y[k] = ∑_{m=0}^{k} g[k − m]u[m]
= ∑_{m=0}^{k} g[m]u[k − m],

where g[k] is the impulse response sequence.

By analogy with the continuous case, an input sequence u[k] is bounded if for some constant B < ∞, we have

|u[k]| ≤ B for all k = 0, 1, 2, …

Theorem 7.15 A single-input single-output (SISO) discrete LTI causal system is BIBO stable if and only if its impulse response, g[k], is absolutely summable, that is,

∑_{k=0}^{∞} |g[k]| ≤ M < ∞

for some constant M.

Example 7.23 The system whose block diagram is depicted in Figure 7.26 is not BIBO stable. The system’s impulse response is

[figure: U(z); \frac{z}{z−1}; Y (z)]

Figure 7.27: System of Example 7.23. This system is not BIBO stable.

g[k] = 1 for k = 0, 1, 2, …

The above impulse response is not absolutely summable because

∑_{k=0}^{∞} |g[k]| = ∑_{k=0}^{∞} 1^{k} = ∞.

Therefore, the system in this example is not BIBO stable.

Example 7.24 The system with the impulse response,

g[k] = (−0.5)^{k}, k = 0, 1, 2, …

is BIBO stable because this impulse response is summable. Indeed, it is easy to verify that

∑_{k=0}^{∞} |g[k]| = ∑_{k=0}^{∞}(−0.5)^{k} = \frac{2}{3}.

The above tests for checking if a given system is BIBO stable or not are not particulary convenient. We give now tests for BIBO stability that are very easy to use. Theorem 7.16 A continuous SISO LTI system with a proper rational transfer function is BIBO stable if and only if every pole of the transfer function has a negative real part.

Using the above theorem, we can by inspection determine that the system of Example 7.21 is not BIBO stable while the system of Example 7.22 is BIBO stable. Theorem 7.17 A discrete SISO LTI system with a proper rational transfer function is BIBO stable if and only if every pole of the transfer function has a magnitude less than 1. Equivalently, all poles of the transfer function are located in the open unit disc in the z-plane.

Using the above theorem, we can by inspection determine that the system of Example 7.23 is not BIBO stable. Theorem 7.18 A continuous MIMO LTI system with a proper rational transfer function matrix, G(s), is BIBO stable if and only if every pole of G_{ij}(s) has a negative real part, where

G(s) = [ \begin{matrix}G_{11}(s) & G_{12}(s) & ⋯ & G_{1m}(s) \\ ⋮ & ⋮ & ... & ⋮ \\ G_{p1}(s) & G_{p2}(s) & ⋯ & G_{pm}(s) \end{matrix} ] .

Note that Theorem 7.16 is a special case of Theorem 7.18. Theorem 7.19 A discrete MIMO LTI system with a proper rational transfer function matrix, G(z), is BIBO stable if and only if every pole of G_{ij}(z) is located in the open unit disc in the z-plane, where

G(z) = [ \begin{matrix}G_{11}(z) & G_{12}(z) & ⋯ & G_{1m}(z) \\ ⋮ & ⋮ & ... & ⋮ \\ G_{p1}(z) & G_{p2}(z) & ⋯ & G_{pm}(z) \end{matrix} ] .

Note that Theorem 7.19 is a generalization of Theorem 7.17.

7.14 Input-Output Stability of LTV Systems

Consider a SISO LTV relaxed at t_{0}, causal system, whose response is expressed by (1.6) on page 19, that is,

y(t) = ∫_{t_{0}}^{t} g(t, τ)u(τ)dτ.

A block diagram of LTV system is shown in Figure 7.28. A necessary and sufficient condition

[figure: u(t); g(t, τ); y(t)]

Figure 7.28: Linear time-varying system with the impulse response g(t, τ).

for the above system to be BIBO stable is that

∫_{t_{0}}^{t} |g(t, τ)|dτ ≤ M < ∞  (7.127)

for all t and t_{0} with t ≥ t_{0}. The proof is similar to that for the LTI case.

For the MIMO case the output of an LTV relaxed at t_{0}, causal system is given by (1.7) on page 19, that is,

y(t) = ∫_{t_{0}}^{t} G(t, τ)u(τ)dτ,

where

G(t, τ) = [ \begin{matrix}g_{11}(t, τ) & g_{12}(t, τ) & ⋯ & g_{1m}(t, τ) \\ g_{21}(t, τ) & g_{22}(t, τ) & ⋯ & g_{2m}(t, τ) \\ ⋮ & ⋮ & ... & ⋮ \\ g_{p1}(t, τ) & g_{p2}(t, τ) & ⋯ & g_{pm}(t, τ) \end{matrix} ] .

The MIMO LTV system is BIBO stable if and only if every entry of the impulse response matrix G(t, τ) satisfies the condition given by (7.127). This condition can be expressed more compactly in an equivalent fashion using norms. Chen [55, p. 137] suggests using the infinite-norms because of their convenience in BIBO stability study,

∥u∥_{∞} = max_{i} |u_{i}| and ∥G∥_{∞} = largest row absolute sum.

Then, a necessary and sufficient condition for the MIMO LTV system to be BIBO stable is that

∫_{t_{0}}^{t} ∥G(t, τ)∥_{∞}dτ ≤ M < ∞  (7.128)

for all t and t_{0} with t ≥ t_{0}.

7.15 Notes

A classical text on stability of dynamical systems is by Hahn [128]. Stability of discrete dynamical systems is presented by Hahn in [127] and LaSalle [204]. Extensive coverage of stability issues can also be found in Willems [351], Slotine and Li [301], Vidyasagar [336], Krstić, Kanellakopoulos, and Kokotović [195], and Khalil [178]. Lyapunov-based controller design methods for a class of uncertain systems are presented in [213]. For more information on the subject of input-output stability, we recommend Antsaklis and Michel [13, Sections 4.7 and 4.85].

There is an interesting story behind the origins of the Routh test. The problem of investigating the stability of a linear dynamical system in terms of the location of the roots of its characteristic equation was posed by Maxwell [231]. On page 271 of his 1868 paper, Maxwell writes, “This condition is mathematically equivalent to the condition that all the possible roots, and all the possible parts of the impossible roots, of a certain equation shall be negative.” By impossible roots, Maxwell means complex roots, and, by possible parts, means real parts in today’s terminology. He continues, “I have not been able completely to determine these conditions for equations of a higher degree than the third; but I hope that the subject will obtain the attention of mathematicians.” A complete general solution to the problem was published by Routh in 1877. When Maxwell and Routh were classmates at Cambridge University in 1854, Routh was placed first in their final math examination ahead of Maxwell. A story says (Smith [305, page 55]) that apparently Maxwell was so sure of coming in first on this exam that he did not even bother to rise early to hear the exam results read out in the Senate House. Maxwell sent his servant instead. When the servant came back, Maxwell asked, “Well, tell me who’s second!”. Obviously, he did not expect to receive the reply, “You are, sir”. In 1877, Routh received the prestigious Adams Prize for solving the problem posed by Maxwell in 1868. According to an account by Truxal [329, page 527], Routh presented his solution with the opening remark, “It has recently come to my attention that my good friend James Clerk Maxwell has had difficulty with a rather trivial problem... .”

Figure 7.29: A closed-loop feedback system of Exercise 7.1.

7.16 Exercises

Exercise 7.1 (a) Consider a negative unity feedback closed-loop system whose open-loop

transfer function is

G(s) = \frac{6s^{3} + 11s^{2}}{s^{4} + 6s + K} .

For what values of the parameter K the closed-loop system is asymptotically stable?

(b) Figure 7.29 shows a closed-loop system in which the error, measured by a transducer that has a time constant of 1 sec, is applied to a proportional-plus-derivative (PD) controller. The output of the controller drives a third-order plant. Use the Routh-Hurwitz criterion to find the region of asymptotic stability, of the closed-loop feedback system, in the (a, K)-plane.

Exercise 7.2 (a) Determine if the polynomial

P (s) = s^{7} + 2s^{6} + 5s^{5} + 7s^{4} + 2s^{3} − 5s^{2} − 8s − 4

is Hurwitz. Use the Routh-Hurwitz criterion to find the number of the polynomial RHP zeros and determine the jω-axis zeros.

(b) Consider a negative unity feedback system with the forward loop transfer function

G(s) = \frac{K}{s(s + 3)(s + 10)}.

Use the Routh-Hurwitz criterion to find the interval of K for which the closed-loop system is asymptotically stable.

(c) Consider a negative unity feedback system with the forward loop transfer function

G(s) = \frac{K}{s(T s + 1)(0.5 s + 1)}.

Use the Routh-Hurwitz criterion to derive the domain of asymptotic stability of the closed-loop system in the (T, K)-plane, and plot it using MATLAB.

Exercise 7.3 For a dynamical system model

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}x_{2} \\ x_{1} − x_{2} sin x_{1} \end{matrix} ] ,

(a) find an equilibrium point for this system;

(b) linearize the system about the found equilibrium point;

(c) determine if the linearized system is stable, asymptotically stable, or unstable.

Exercise 7.4 Consider the following model of a dynamical system,

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & −x_{1}^{5} − x_{2}^{3} \end{matrix} \}  (7.129)

(a) Find the equilibrium state of the system. Then, consider the following Lyapunov

function candidate,

V (x_{1}, x_{2}) = \frac{1}{2}x_{2}^{2} + ∫_{0}^{x_{1}} s^{5}ds.

Evaluate \frac{d}{dt}V (x_{1}, x_{2}) on the trajectories of the system (7.129).

(b) Use the results obtained in (a) to decide whether the equilibrium state is stable, asymptotically stable, or unstable.

404 CHAPTER 7. STABILITY Exercise 7.5 For a dynamical system modeled by ẋ = Ax + ζ(t, x), where

A = [ \begin{matrix}0 & 1 \\ −1/2 & −3/2 \end{matrix} ] , and ∥ζ(t, x)∥ ≤ k∥x∥,

find k^{∗} > 0 such that for any k < k^{∗} the system is globally uniformly asymptotically stable. Exercise 7.6 Is the system,

ẋ = [ \begin{matrix}−2 & 0 & 0 \\ 1 & 0 & 1 \\ 0 & −2 & −2 \end{matrix} ] x + [ \begin{matrix}1 \\ 0 \\ 1 \end{matrix} ] u
y = [ 1 −1 0 ] x − 2u,

BIBO stable? Exercise 7.7 Determine the stability of the system

ẋ = [ \begin{matrix}−2 & 0 & 0 \\ 1 & 0 & 1 \\ 0 & −2 & −2 \end{matrix} ] x

in the sense of Lyapunov by solving the Lyapunov matrix equation using the Kronecker product. Take Q = I. Verify your solution using the MATLAB function lyap. Exercise 7.8 Is the system,

x[k + 1] = [ \begin{matrix}−0.8 & 0 & 0 \\ 0.4 & 0 & 0.4 \\ 0 & −0.8 & −0.8 \end{matrix} ] x[k] + [ \begin{matrix}1 \\ 0 \\ 1 \end{matrix} ] u[k]
y = [ 1 −1 0 ] x[k] − 2u[k],

BIBO stable? Exercise 7.9 Determine the stability of the system

x[k + 1] = [ \begin{matrix}−0.8 & 0 & 0 \\ 0.4 & 0 & 0.4 \\ 0 & −0.8 & −0.8 \end{matrix} ] x[k]

in the sense of Lyapunov by solving the Lyapunov matrix equation using the Kronecker product. Take Q = I. Verify your solution using the MATLAB function dlyap.

7.16. EXERCISES 405 Exercise 7.10 (J. Hu) For the linear time-invariant (LTV) system model,

ẋ(t) = [ \begin{matrix}−e^{−t} & \frac{1}{t+1} \\ 0 & −e^{−t} \end{matrix} ] x(t),

• Find the state transition matrix Φ(t, τ),

• Is the system asymptotically stable?

• Let x(0) = [ 1 0 ]^{⊤}. Will the system trajectory x(t) stay bounded as t → ∞? Exercise 7.11 (a) Find the value of

J_{0} = 2 ∫_{0}^{∞} ∥x(t)∥^{2}dt

subject to

ẋ = [ \begin{matrix}0 & 1 \\ −1 & −1 \end{matrix} ] x, x(0) = [ \begin{matrix}0 \\ −1 \end{matrix} ] .

(b) Find the value of

K_{0} = ∑_{k=0}^{∞} ∥x[k]∥^{2}

subject to

x[k + 1] = [ \begin{matrix}1/2 & 0 \\ 0 & 1/4 \end{matrix} ] x[k], x[0] = [ \begin{matrix}3 \\ 0 \end{matrix} ] .

Exercise 7.12 Determine the stability or instability of the zero solution of the system,

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & x_{1} − x_{2} sin x_{1} \end{matrix} \}

using the second method of Lyapunov and compare the result of your analysis with the solution to Exercise 7.3. Exercise 7.13 (based on [328, page 329]) Construct a Lyapunov function for the system modeled by

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & −x_{2} − x_{1}^{3}. \end{matrix} \}

Show that the origin is a globally stable equilibrium state of the system.

Exercise 7.14 Investigate the stability of the following dynamical system model:

\begin{matrix}ẋ_{1} & = & x_{2} − 2x_{1}x_{2}^{2} \\ ẋ_{2} & = & −x_{1} − x_{2} − 2x_{2}^{3} \end{matrix} \}

Is the null solution, x_{1}(t) = x_{2}(t) = 0, t ≥ 0, locally stable, globally stable, locally asymptotically stable, globally asymptotically stable, or unstable? Justify your answer.

Exercise 7.15 Use the second method of Lyapunov to estimate the region of asymptotic stability of the equilibrium state [ 2/3 2/3 ]^{⊤} of the dynamical system model

\begin{matrix}ẋ_{1} & = & (1 − x_{1} − 0.5x_{2}) x_{1} \\ ẋ_{2} & = & (1 − 0.5x_{1} − x_{2}) x_{2} \end{matrix} \}

Exercise 7.16 For the nonlinear system model, ẋ = f(x), where

f(x) = [ \begin{matrix}f_{1}(x) \\ f_{2}(x) \end{matrix} ] = [ \begin{matrix}4x_{1} − 2x_{1}^{2}x_{2} \\ −16x_{2} + 4x_{1}^{2}x_{2} \end{matrix} ] ,  (7.130)

find its equilibrium points and determine their stability properties using the linearization method.

Exercise 7.17 Consider a dynamical system modeled by

ẋ = Ax,

where A ∈ ℝ^{n×n}. The eigenvalues λ_{i}, i = 1, 2, …, n, of the matrix A are all negative, and the matrix A is diagonalizable, that is, there exists a nonsingular matrix T ∈ ℝ^{n×n} such that

T AT^{−1} = D = diag{λ_{1}, λ_{2}, … λ_{n}}.

Let

V (x) = \frac{1}{2}x^{⊤}T^{⊤}T x = x^{⊤}P x.

(a) Find V̇, that is, find the derivative of V (x(t)) with respect to time along the solutions of the system ẋ = Ax.

(b) Find the upper bound on V̇ such that

V̇ (x) ≤ c∥x∥,

where the constant c < 0 depends on an eigenvalue of A and an eigenvalue of P.

Exercise 7.18 Consider a system of linear equations of the form,

Ax = b,

where A ∈ ℝ^{m×n}, m ≥ n, rank(A) = n, and x and b are of appropriate dimensions. Consider also an associated iterative algorithm,

x[k + 1] = x[k] + αA^{⊤} (b − Ax[k]) , x[0] = x_{0},

where α > 0. Treat the above algorithm as a discrete-time dynamical system.

(a) Find the equilibrium state of the algorithm;

(b) Use the conditions for asymptotic stability of discrete-time systems to find the largest range of values of α for which the algorithm converges to its equilibrium state;

(c) Show that the algorithm, for the range of α found in (b), minimizes the objective

function

J(x) = (b − Ax)^{⊤} (b − Ax) .

Exercise 7.19 Consider a system of linear equations of the form,

Ax = b,

where A ∈ ℝ^{m×n}, m ≥ n, rank(A) = n, and x and b are of appropriate dimensions. Consider also associated objective function

J(x) = (b − Ax)^{⊤} (b − Ax) .

Write down an iterative steepest descent algorithm, in terms of A and b, for minimizing the above objective function. Carefully derive an expression for the step size α_{k} in terms of ∇J (x^{(k)}) and A only.

Exercise 7.20 (a) Given two symmetric positive definite matrices P and P̃. Show, by contradiction, that if for all x,

x^{⊤}P x ≥ x^{⊤}P̃ x,

then

λ_{max} (P) ≥ λ_{max} \binom{}{P̃} .

408 CHAPTER 7. STABILITY (b) Let P = P^{⊤} > 0 be the unique solution to

A^{⊤}P + P A = −2Q,  (7.131)

where Q = Q^{⊤} > 0, and let P̂ = P̂^{⊤} > 0 be the unique solution to

A^{⊤}P̂ + P̂ A = −2Q̂,  (7.132)

where Q̂ = qQ, q > 0. Show that

\frac{λ_{min} \binom{}{Q̂}}{λ_{max} \binom{}{P̂}} = \frac{λ_{min}(Q)}{λ_{max}(P)}.

(c) Prove the following lemma of Patel and Toda [265].

Lemma 7.7 Let P = P^{⊤} > 0 be the unique solution to

A^{⊤}P + P A = −2Q,

where Q = Q^{⊤} > 0. Then, the ratio

\frac{λ_{min}(Q)}{λ_{max}(P)}

is maximized if Q = I. Exercise 7.21 Consider the Lyapunov matrix equation,

A^{⊤}P + P A = −2I_{n},

where A ∈ ℝ^{n×n} is asymptotically stable.

(a) Let σ_{min} be the smallest singular value of A, that is,

σ_{min} = \sqrt{λ_{min} (AA^{⊤})},

where λ_{min} (AA^{⊤}) denotes the smallest eigenvalue of AA^{⊤}. Show that σ_{min} > 0.

(b) Let λ_{max}(P) be the largest eigenvalue of P, the solution of the above Lyapunov equa-

tion. Show that

λ_{max}(P) ≥ \frac{1}{σ_{min}} .

Exercise 7.22 (based on Theorem 1 in [43]) Consider an asymptotically stable strictly proper transfer function

T (s) = \frac{n(s)}{d(s)}.

Suppose that the triple (c, A, b) is a controllable and observable realization of the transfer function T (s). Thus A must be asymptotically stable. This and the observability assumption guarantees that we can solve the Lyapunov equation

A^{⊤}P + P A = −c^{⊤}c

for P = P^{⊤} > 0. Let

q(s) = d(s)b^{⊤}P (sI − A)^{−1} b.

Show that

d(s)q(−s) + q(s)d(−s) = n(s)n(−s).

Exercise 7.23 Consider a dynamical system model,

ẋ = Ax, x(0) = x_{0},

where A = [ \begin{matrix}0 & 1 \\ −1 & −1 \end{matrix} ] and ∥x_{0}∥ ≤ 10. Use the Lyapunov approach to find T ≥ 0, given

ε_{1} = 20, such that

∥x(t; 0, x_{0})∥ ≤ ε_{1} for all t ≥ T.

You can use, for example,

Q = [ \begin{matrix}2 & 1 \\ 1 & 2 \end{matrix} ]

when solving the matrix Lyapunov equation A^{⊤}P + P A = −Q.

Exercise 7.24 The role of an automotive suspension is to support the vehicle body on the axles. A simple linear model of the suspension assembly associated with a single wheel, called the quarter-car model, is shown in Figure 7.30. We represent the vehicle body by the “sprung” mass, while the tire and axle are represented by the “unsprung” mass. We model the tire as a spring with spring coefficient k_{t}, and assume its damping to be negligible. The suspension consists of the spring, the shock absorber, and the variable force element. The role of the variable force element is to generate a force that compensates for the uneven road which is the source of the system disturbances. We assume that the variable force element F_{a} can instantaneously provide any desired force; thus, we have a linear active suspension. The

Figure 7.30: Linear quarter-car model

equations modeling the dynamics of the two degree of freedom system depicted in Figure 7.30 are derived by applying Newton’s law to each mass of the system, and by taking into account the positive direction of the z coordinates. We obtain

m_{s}z̈_{s} = −k_{s}(z_{s} − z_{u}) − b_{s}(ż_{s} − ż_{u}) + F_{a},
m_{u}z̈_{u} = k_{s}(z_{s} − z_{u}) + b_{s}(ż_{s} − ż_{u}) − F_{a} − k_{t}(z_{u} − z_{r}).

Define the following state variables:

x_{1} = z_{s} − z_{u} = suspension stroke,
x_{2} = ż_{s} = sprung mass velocity,
x_{3} = z_{u} − z_{r} = tire deflection,
x_{4} = ż_{u} = unsprung mass velocity.

Let u = F_{a} be the system input, and let δ(t) = ż_{r}(t) be the disturbance input due to the road roughness.

(a) Represent the active suspension model in state-space format.

(b) Substitute the vehicle data, given in Table 7.1, into the modeling equations.

Table 7.1: Data for a vehicle suspension taken from [314].

m_{s}  355 kg
m_{u}  46 kg
k_{s} 14384 N/m
b_{s} 1860 N/m/sec
k_{t} 300000 N/m

(c) Our goal is to isolate the sprung mass from road disturbances. Specifically, we wish to design a control law such that the sprung mass velocity ż_{s}(t) = x_{2} is as close to 0 as possible in spite of road disturbances. Thus, we assume that we can measure the sprung mass velocity, which means that the output of the plant is

y(t) = cx(t) = [ 0 1 0 0 ] x(t).

To investigate the effect of road disturbances on the vertical velocity of the sprung mass, set u = 0 and δ(t) = d sin(6t). Generate a plot of y versus t for d = 0.5.

(d) Construct a state-feedback Lyapunov-based controller of Theorem 7.4. Generate a plot of y of the closed-loop system versus t for d = 0.5. Discuss your strategy of the above state-feedback controller design that would minimize the influence of the disturbance δ(t) on the sprung mass velocity y.

Exercise 7.25 Consider a dynamical system model

ẋ = [ \begin{matrix}−2 & 0 \\ 0 & −1 \end{matrix} ] x + [ \begin{matrix}1 \\ 1 \end{matrix} ] (u + h(t, x)),

where

|h(t, x)| ≤ 3.

Let P be the solution of the matrix Lyapunov equation A^{⊤}P + P A = −2I_{2}, and let

u = \{ \begin{matrix}−3 sign(b^{⊤}P x) & if & |b^{⊤}P x| ≥ ν \\ −3\frac{b^{⊤}P x}{ν} & if & |b^{⊤}P x| < ν, \end{matrix}

where ν = 0.5. Find the radius of the ball, in ℝ^{2}, that is uniformly stable for the closed-loop system.

412 CHAPTER 7. STABILITY

A Computational Tool for Control Design

Henry Ford offered customers the Model T Ford “in any color the customer wants, as long as it is black.” Henry Ford’s policy resulted from subordination to the assembly line constraint that could not keep up with market demand. This was a resource constraint. To maximize throughput it was important for Ford not to change colors during production. This policy assured maximum profit and value for Ford by subordinating the whole system (including customers) to production. Henry Ford’s failure was that he stuck to his policy even when new competitors entered the market; then the resource constraint became a market constraint resulting in a steep drop in sales and market share. The right step would have been to realize that there was now a market constraint and subordinate the system to the constraint. Competition and customers’ needs should have driven Ford to produce a variety of cars: different styles, colors, or amenities.— Ronen and Pass [284, p. 84]

8.1 Motivation

Many control design problems for linear and nonlinear systems can be formulated as convex optimization problems. To define a convex optimization problem, we need definitions of a convex set and a convex function.

Definition 8.1 A set Θ ⊆ ℝ^{n} is convex if for any x and y in Θ, the line segment between x and y lies in Θ, that is,

αx + (1 − α)y ∈ Θ for any α ∈ (0, 1).
◇

Examples of convex sets include the empty set, ℝ^{n}, a subspace, a line, a line segment.

Definition 8.2 A real-valued function

f : Ω → ℝ

defined on a convex set Θ ⊆ ℝ^{n} is convex if for all x, y ∈ Ω and all α ∈ (0, 1),

f(αx + (1 − α)y) ≤ αf(x) + (1 − α)f(y).
◇

Thus a real-valued function on n variables is convex over a convex set if for all x, y ∈ Ω the points on the line segment connecting (x, f(x)) and (y, f(y)) line on or above the graph of f. For more information on the subject of convex function, we recommend [62, Section 21.2].

A convex optimization problem is the one where the objective function to be minimized or maximized is convex and the constraint set, over which we optimize the objective function, is a convex set.

Semidefinite programming (SDP) is a subfield of convex optimization concerned with minimizing or maximizing a linear objective function subject to a matrix inequality. The matrix inequality constraint is convex and defines a convex feasible set over which the linear objective function is being optimized. Semidefinite programming can be viewed as an extension of linear programming, where the componentwise inequalities on vectors are replaced by matrix inequalities. For further reading on the subject of semidefinite programming, we recommend an excellent survey paper by Vandenberghe and Boyd [334].

8.2 Linear Matrix Inequality and Its Properties

Consider n + 1 real symmetric matrices

F_{i} = F _{i}^{⊤} ∈ ℝ^{m×m}, i = 0, 1, …, n

and a vector

x = [ x_{1} x_{2} ⋯ x_{n} ]^{⊤} ∈ ℝ^{n}.

Then,

F (x) = F_{0} + x_{1}F_{1} + … + x_{n}F_{n}
= F_{0} + ∑_{i=1}^{n} x_{i}F_{i}

is an affine function of x, rather than a linear function of x because F (x) is composed of a linear term, ∑_{i=1}^{n} x_{i}F_{i}, and a constant term, F_{0}.

Consider now an expression of the form

F (x) = F_{0} + x_{1}F_{1} + ⋯ + x_{n}F_{n} ≥ 0.

The above is to be interpreted as follows: find a set of vectors x such that

z^{⊤}F (x)z ≥ 0 for all z ∈ ℝ^{m},

that is, F (x) is positive definite. Recall that F_{i}s are constant matrices, x is unknown, and F (x) = F (x)^{⊤} is an affine function x.

The expression, F (x) = F_{0} + x_{1}F_{1} + ⋯ + x_{n}F_{n} ≥ 0, is referred to as in the literature as the linear matrix inequality (LMI) although the term the affine matrix inequality would be a correct term to use.

It is easy to verify, see Exercise 8.1, that the set, {x : F (x) ≥ 0}, is a convex set.

A system of LMIs,

F_{1}(x) ≥ 0, F_{2}(x) ≥ 0, …, F_{k}(x) ≥ 0,

can be represented as one single LMI,

F (x) = [ \begin{matrix}F_{1}(x) \\ & F_{2}(x) \\ & & . & . & . \\ & & & & & F_{k}(x) \end{matrix} ] ≥ 0.

A linear matrix inequality, involving an m-by-n constant matrix A, of the form,

Ax ≤ b

can be represented as m LMIs

b_{i} − a_{i}^{⊤}x ≥ 0, i = 1, 2, …, m,

where a_{i}^{⊤} is the i-th row of the matrix A. We can view each scalar inequality as an LMI. We then represent m LMIs as one LMI,

F (x) = [ \begin{matrix}b_{1} − a_{1}^{⊤}x \\ & b_{2} − a_{2}^{⊤}x \\ & & . & . & . \\ & & & & & b_{m} − a_{m}^{⊤}x \end{matrix} ] ≥ 0.

With the above facts as a background, we can now give an example of semidefinite programming,

minimize c^{⊤}x
subject to F (x) ≥ 0.

The matrix property that we discuss next is useful when converting some nonlinear matrix inequalities into linear inequalities. We start with a simple observation. Let P = P^{⊤} be a nonsingular n-by-n matrix and let x = Mz, where M ∈ ℝ^{n×n} such that det M ≠ 0. Then, we have

x^{⊤}P x ≥ 0 if and only if z^{⊤}M^{⊤}P Mz ≥ 0,

that is,

P ≥ 0 if and only if M^{⊤}P M ≥ 0.

Suppose that we have a square block matrix

[ \begin{matrix}A & B \\ B^{⊤} & D \end{matrix} ] .

Then

[ \begin{matrix}A & B \\ B^{⊤} & D \end{matrix} ] ≥ 0 if and only if [ \begin{matrix}O & I \\ I & O \end{matrix} ] [ \begin{matrix}A & B \\ B^{⊤} & D \end{matrix} ] [ \begin{matrix}O & I \\ I & O \end{matrix} ] ≥ 0,

where I is an identity matrix of appropriate dimension. In other words,

[ \begin{matrix}A & B \\ B^{⊤} & D \end{matrix} ] ≥ 0 if and only if [ \begin{matrix}D & B^{⊤} \\ B & A \end{matrix} ] ≥ 0.

Consider now a square block matrix of the form

[ \begin{matrix}A_{11} & A_{12} \\ A_{21} & A_{22} \end{matrix} ] ,

where A_{11} and A_{22} are square submatrices. Let us now suppose that the matrix A_{11} is invertible. Then we have

[ \begin{matrix}A_{11} & A_{12} \\ A_{21} & A_{22} \end{matrix} ] = [ \begin{matrix}I & O \\ A_{21}A_{11}^{−1} & I \end{matrix} ] [ \begin{matrix}A_{11} & O \\ O & Δ \end{matrix} ] [ \begin{matrix}I & A_{11}^{−1}A_{12} \\ O & I \end{matrix} ] ,

where

Δ = A_{22} − A_{21}A_{11}^{−1}A_{12}.

The matrix Δ is called the Schur complement of A_{11}. For the case when A_{12} = A_{21}^{⊤}, we have

[ \begin{matrix}A_{11} & A_{21}^{⊤} \\ A_{21} & A_{22} \end{matrix} ] = [ \begin{matrix}I & O \\ A_{21}A_{11}^{−1} & I \end{matrix} ] [ \begin{matrix}A_{11} & O \\ O & Δ \end{matrix} ] [ \begin{matrix}I & A_{11}^{−1}A_{21}^{⊤} \\ O & I \end{matrix} ] ,

where

Δ = A_{22} − A_{21}A_{11}^{−1}A_{21}^{⊤}.

Hence,

[ \begin{matrix}A_{11} & A_{21}^{⊤} \\ A_{21} & A_{22} \end{matrix} ] > 0 if and only if [ \begin{matrix}A_{11} & O \\ O & Δ \end{matrix} ] > 0,

that is,

[ \begin{matrix}A_{11} & A_{21}^{⊤} \\ A_{21} & A_{22} \end{matrix} ] > 0 if and only if A_{11} > 0 and Δ > 0.

Many problems of optimization, control design, and signal processing can be formulated in terms of LMIs. To test whether or not there exist solutions x to F (x) > 0 is called a feasibility problem. We say that the LMI is non-feasible if no solution exists.

Example 8.1 We present an example illustrating the LMI feasibility problem. It is well known that a constant square matrix A ∈ ℝ^{n×n} has its eigenvalues in the open left halfcomplex plane if and only if for any real, symmetric, positive definite Q ∈ ℝ^{n×n} the solution P = P^{⊤} to the Lyapunov matrix equation

A^{⊤}P + P A = −Q,

is positive definite. That is, real parts of the eigenvalues of A are all negative if and only if there exists a real symmetric positive definite matrix P such that

A^{⊤}P + P A < 0,

or equivalently,

−A^{⊤}P − P A > 0.

Thus, the location of all eigenvalues of A in the open left half-complex plane is equivalent to feasibility of the following LMI,

[ \begin{matrix}P & O \\ O & −A^{⊤}P − P A \end{matrix} ] ≥ 0,

that is, the existence of a symmetric positive definite P such that the above inequality holds.

Remark 8.1 Most of the optimization solvers do not handle strict inequalities. Therefore, the operator > is the same as ≥, and so > implements the non-strict inequality ≥.

◇

We now show that finding P = P^{⊤} > 0 such that A^{⊤}P + P A < 0 is indeed an LMI. For this let

P = [ \begin{matrix}x_{1} & x_{2} & ⋯ & x_{n} \\ x_{2} & x_{n+1} & ⋯ & x_{2n−1} \\ ⋮ & & & ⋮ \\ x_{n} & x_{2n−1} & ⋯ & x_{q} \end{matrix} ] ,

where

q = \frac{n(n + 1)}{2}.

We next define the following matrices,

P_{1} = [ \begin{matrix}1 & 0 & 0 & ⋯ & 0 \\ 0 & 0 & 0 & ⋯ & 0 \\ 0 & 0 & 0 & ⋯ & 0 \\ ⋮ & & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 \end{matrix} ] ,

P_{2} = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 \\ 1 & 0 & 0 & ⋯ & 0 \\ 0 & 0 & 0 & ⋯ & 0 \\ ⋮ & & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 \end{matrix} ] ,

⋮

P_{q} = [ \begin{matrix}0 & 0 & 0 & ⋯ & 0 \\ 0 & 0 & 0 & ⋯ & 0 \\ 0 & 0 & 0 & ⋯ & 0 \\ ⋮ & & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 1 \end{matrix} ] .

Note that P_{i} has only non-zero elements corresponding to x_{i} in P. Let

F_{i} = −A^{⊤}P_{i} − P_{i}A, i = 1, 2, …, q.

We then write

A^{⊤}P + P A = x_{1} (A^{⊤}P_{1} + P_{1}A) + x_{2} (A^{⊤}P_{2} + P_{2}A) + ⋯ + x_{q} (A^{⊤}P_{q} + P_{q}A)
= −x_{1}F_{1} − x_{2}F_{2} − ⋯ − x_{q}F_{q}
< 0.

Let

F (x) = x_{1}F_{1} + x_{2}F_{2} + ⋯ + x_{q}F_{q},

then

P = P^{⊤} > 0 and A^{⊤}P + P A < 0

if and only if

F (x) > 0.

More examples of linear matrix inequalities in system and control theory can be found in the book by Boyd et al. [41].

8.3 LMI Solvers

The LMI F (x) = F_{0} + x_{1}F_{1} + ⋯ + x_{n}F_{n} ≥ 0 is called the canonical representation of an LMI. The LMIs in the canonical form are very inefficient from a storage view-point as well as from the efficiency of the LMI solvers view-point. The LMI solvers use a structured representation of LMIs.

One can use MATLAB’s toolbox to efficiently solve LMIs. This toolbox has three types of LMI solvers that we discuss next.

8.3.1 Solving the Feasibility Problem

This solver computes a feasible solution, that is, is solves the feasible problem defined by a given system of LMI constraints. Using this solver, we can solve any system of LMIs of the form

N^{⊤}L(X_{1}, …, X_{k})N < M^{⊤}R(X_{1}, …, X_{k})M,

where X_{1}, …, X_{k} are matrix variables, N is the left outer factor, M is the right outer factor, L(X_{1}, …, X_{k}) is the left inner factor and R(X_{1}, …, X_{k}) is the right inner factor. The matrices L(·) and R(·) are, in general, symmetric block matrices. We note that the term “left-hand side” refers to what is on the “smaller” side of the inequality 0 < X. Thus in X > 0, the matrix X is still on the right-hand side because it is on the “larger ” side of the inequality.

We now provide a description of an approach that can be used to solve the given LMI system feasibility problem. To initialize the LMI system description, we type setlmis([]). Then we declare matrix variables using the command lmivar. The command lmiterm allows us to specify LMIs that constitute the LMI system under consideration. Next, we need to obtain an internal representation using the command getlmis. We next compute a feasible solution to the given LMI system using the command feasp. After that we extract matrix variable values with the command dec2mat. In summary, a general structure of a MATLAB program for finding a feasible solution to the set of LMIs could have the form,

setlmis([])
lmivar
lmiterm
.
.
.
lmiterm
getlmis
feasp
dec2mat

We now analyze the above commands in some detail so that the reader can write simple MATLAB programs for solving LMIs after finishing reading this section. First, to create a new matrix-valued variable, say, X, in the given LMI system, we use the command

X=lmivar(type,structure)

The input type specifies the structure of the variable X. There may be three structures of matrix variables. When type=1, we have a symmetric block diagonal matrix variable. The input type=1 refers to a full rectangular matrix variable. Finally, type=3 refers to other cases. The second input structure gives additional information on the structure of the matrix variable X. For example the matrix variable X could have the form

X = [ \begin{matrix}D_{1} & O & ⋯ & O \\ O & D_{2} & ⋯ & O \\ ⋮ & & ... & ⋮ \\ O & O & ⋯ & D_{r} \end{matrix} ] ,

where each D_{i} is a square symmetric matrix. For the above example we would use type=1. The above matrix variable has r blocks. The input structure is then an r × 2 matrix whose i-th row describes the i-th block, where the first component of each row gives the corresponding block size, while the second element of each row specifies the block type. For example,

X=lmivar(1,[3 1])

specifies a full symmetric 3 × 3 matrix variable. On the other hand,

X=lmivar(2,[2 3])

specifies a rectangular 2 × 3 matrix variable. Finally, the matrix variable S of the form

S = [ \begin{matrix}s_{1} & 0 & | & 0 & 0 \\ 0 & s_{1} & | & 0 & 0 \\ −− & −− & | & −− & −− \\ 0 & 0 & | & s_{2} & s_{3} \\ 0 & 0 & | & s_{3} & s_{4} \end{matrix} ] ,

can be declared as follows:

S=lmivar(1,[2 0;2 1])

Note that in the above the second component of the first row of the second input has the value of zero, that is, structure(1,2)=0. This describes a scalar block matrix of the form

D_{1} = s_{1}I_{2}.

Note that the second block is a 2 × 2 symmetric full block.

We next take a closer look at the next command whose purpose is to specify the terms of the LMI system of interest. This command has the form

lmiterm(termid,A,B,flag)

We briefly describe each of the four inputs of this command. The first input, termid, is a row with four elements that specify the terms of each LMI of the LMI system. We have termid(1)=n to specify the left-hand side of the n-th LMI. We use termid(1)=-n to specify the right-hand side of the n-th LMI. The middle two elements of the input termid specify the block location. Thus termid(2,3)=[i j] refers to the term that belongs to the (i, j) block of the LMI specified by the first component. Finally, termid(4)=0 for the constant term, termid(4)=X for the variable term in the form AXB, while termid(4)=-X for the variable term in the form AX^{⊤}B. The third and the fourth inputs of the command lmiterm give the values of the left and right outer factors, that is, A and B give the value of the constant outer factors in the variable terms, AXB or in AX^{⊤}B. Finally the fourth input to lmiterm serves as a compact way to specify the expression

AXB + (AXB)^{⊤}.

Thus flag='s' can be used to denote a symmetrized expression. We illustrate the above command on the following LMI,

P A + (P A)^{⊤} < 0.

We have one LMI with two terms. We could use the following description of this single LMI,

lmiterm([1 1 1 P],1,A)
lmiterm([1 1 1 -P],A',1)

On the other hand, we can compactly describe the above LMI with the use of the flag as follows,

lmiterm([1 1 1 P],1,A,'s')

Now to solve the feasibility problem we could have typed

[tmin,xfeas]=feas(lmis)

In general, for a given LMI feasibility problem of the form

find x
such that L(x) < R(x)

the command feasp solves the auxiliary convex problem

minimize t
subject to L(x) < R(x) + tI.

The system of LMIs is feasible if the minimal t is negative. We add that the current value of t is displayed by feasp at each iteration.

Finally we convert the output of the LMI solver into matrix variables using the command

P=dec2mat(lmis,xfeas,P).

8.3.2 Minimization of a Linear Objective Under LMI Constraints

This solver is invoked using the function mincx. It solves the convex problem

minimize c^{⊤}x
subject to A(x) < B(x).

The notation A(x) < B(x) is a shorthand notation for general structured LMI systems.

Thus to solve a mincx problem, in addition to specifying the LMI constraints as in the feasp problem, we also declare the linear objective function. Then we invoke the function mincx. We illustrate both the feasp and mincx solvers in the following example.

Example 8.2 We consider the optimization problem

minimize c^{⊤}x
subject to Ax ≤ b,

where

c^{⊤} = [ 4 5 ] ,
A = [ \begin{matrix}1 & 1 \\ 1 & 3 \\ 2 & 1 \end{matrix} ] , b = [ \begin{matrix}8 \\ 18 \\ 14 \end{matrix} ] .

We first solve the feasibility problem, that is, we find an x such that Ax ≤ b, using the feasp solver. After that we solve the above minimization problem using the mincx solver. A simple MATLAB code accomplishing the above tasks is shown below.

% Enter problem data
A =[1 1;1 3;2 1];
b=[8 18 14]';
c=[-4 -5]';
setlmis([]);
X=lmivar(2,[2 1]);
lmiterm([1 1 1 X],A(1,:),1);
lmiterm([1 1 1 0],-b(1));
lmiterm([1 2 2 X],A(2,:),1);
lmiterm([1 2 2 0],-b(2));
lmiterm([1 3 3 X],A(3,:),1);
lmiterm([1 3 3 0],-b(3));
lmis=getlmis;
%------------------------------------
disp('-------------feasp result---------------')
[tmin,xfeas]=feasp(lmis);
x_feasp=dec2mat(lmis,xfeas,X)
disp('-------------mincx result---------------')
[objective,x_mincx]=mincx(lmis,c,[0.0001 1000 0 0 1])

The feasp produces

x_{feasp} = [ \begin{matrix}−64.3996 \\ −25.1712 \end{matrix} ] .

The mincx produces

x_{mincx} = [ \begin{matrix}3.0000 \\ 5.0000 \end{matrix} ] .

In the next example we discuss the function defcx that we can use to construct the vector c used by the LMI solver mincx.

Example 8.3 Suppose that we wish to solve the optimization problem

minimize trace(P)
subject to A^{⊤}P + P A < 0,

where P = P^{⊤} > 0.

We can use the function mincx to solve the above problem. However, to use the function mincx, we need a vector c such that

c^{⊤}x = trace(P).

After specifying the LMIs and obtaining their internal representation using, for example, the command lmisys=getlmis, we can obtain the desired c with the following MATLAB code,

q=decnbr(lmisys);
c=zeros(q,1);
for j=1:q
    Pj=defcx(lmisys,j,P);
    c(j)=trace(Pj);
end

Having obtained the vector c, we can use the function mincx to solve the optimization problem.

8.3.3 Generalized Eigenvalue Minimization Problem

This problem can be stated as

minimize λ

subject to

C(x) < D(x)
0 < B(x)
A(x) < λB(x).

In the above, we need to distinguish between standard LMI constraints of the form C(x) < D(x) and the linear-fractional LMIs of the form A(x) < λB(x) that are concerned with the generalized eigenvalue λ. For more information on this type of the LMI solver, we refer to the LMI Lab in the MATLAB’s Robust Control Toolbox user’s guide.

8.4 Notes

A quick introduction to the LMI toolbox is the tutorial that can be accessed by typing in lmidem. It is very instructive and we recommend to try it.

We mention that in addition to the MATLAB’s LMI Toolbox there is another toolbox for solving LMIs called LMITOOL, a built in software package in Scilab toolbox, developed at INRIA in France. Scilab offers free software for numerical optimization. There is a version of LMITOOL for MATLAB that can be uploaded from the website of the Scilab Consortium.

Yet Another LMI Package (YALMIP) for solving LMIs was developed in Switzerland in the Automatic Control Laboratory at ETH. YALMIP is an “intuitive and flexible modelling language for solving optimization problems in MATLAB.”

8.5 Exercises

Exercise 8.1 Show that the set

{x : F_{0} + x_{1}F_{1} + ⋯ + x_{n}F_{n} ≥ 0}

is convex.

Exercise 8.2 Consider the following symmetric matrix,

Q = [ \begin{matrix}2 & 2 & 1 \\ 2 & 3 & 1 \\ 1 & 1 & 2 \end{matrix} ] = [ \begin{matrix}Q_{11} & Q_{12} \\ Q_{21} & Q_{22} \end{matrix} ] .

(i) Compute the Schur complement, Δ_{11}, of Q_{11};

(ii) Compute the Schur complement, Δ_{22}, of Q_{22}.

Exercise 8.3 Use the Schur complement to show that solving the discrete Lyapunov matrix equation

A^{⊤}P A − P < 0

for P = P^{⊤} > 0 is equivalent to solving the LMI,

[ \begin{matrix}P & A^{⊤}P \\ P A & P \end{matrix} ] > 0

Exercise 8.4 Write a MATLAB program that finds a positive definite P such that

A_{1}^{⊤}P + P A_{1} < 0
A_{2}^{⊤}P + P A_{2} < 0,

and

P < 5I_{2},

where

A_{1} = [ \begin{matrix}−1 & 0 \\ 0 & −1 \end{matrix} ] and A_{2} = [ \begin{matrix}−2 & 0 \\ 1 & −1 \end{matrix} ] .

Exercise 8.5 It is known, see Example 15.7, that for the matrices

A_{1} = [ \begin{matrix}−1 & 4 \\ 0 & −2 \end{matrix} ] , A_{2} = [ \begin{matrix}−1 & 0 \\ 4 & −2 \end{matrix} ]

there is no common positive definite P such that A_{i}^{⊤}P + P A_{i} < 0 for i = 1, 2, that is, there is no P = P^{⊤} > 0 such that

A_{1}^{⊤}P + P A_{1} < 0 and A_{2}^{⊤}P + P A_{2} < 0.

Verify this using LMIs.

Exercise 8.6 Formulate LMIs that find a state feedback controller

u = −Kx

that stabilizes each individual system for a family of linear systems,

ẋ = A_{i}x + B_{i}u, i = 1, …, N.

One way to solve this is to formulate LMIs that solve the problem of finding P = P^{⊤} > 0 such that

(A_{i} − B_{i}K)^{⊤}P + P (A − B_{i}K) < 0, i = 1, …, N.

428 CHAPTER 8. A COMPUTATIONAL TOOL FOR CONTROL DESIGN 429

Chapter 9

Optimal Control

Nobel laureate H. A. Simon revolutionized management by identifying a managerial phenomenon that causes decision-making failures. Simon (1957) claimed that executives, engineers, and decision makers are trying to become optimizers while making decisions.

————————

Optimizer: Executive, engineer, or decision maker who wants to make the best possible decision, without consideration of time constraints.

————————

Simon proposes an alternative approach and suggests that decision makers behave as “satisficres,” that is, aspire to reach a “satisfactory solution,” a “good enough solution,” and not necessarily a perfect one (the “optimal” one).

————————

Satisficer: An executive or decision maker who is satisfied with a reasonable solution that will significantly improve the system and does not strive for the optimal solution.

————————

The satisficer sets a level of aspiration, a threshold he aspires to achieve. The objective is no longer to maximize or minimize some performance measure, but to achieve a solution that will improve the measure beyond the predefined level of aspiration.—Ronen and Pass [284, pp.15–16]

One of the essential elements of the control problem, discussed in Section 1.1, is a means of testing the performance of any proposed control law. Whenever we use the term “best” or “optimal” to describe the effectiveness of a given control strategy, we do so with respect to some numerical index of performance called the performance index. We assume that the value of the performance index decreases as the quality of the given admissible control law increases. The admissible controller that ensures the completion of the system objective and at the same time minimizes the performance index is called an optimal controller for the system. In this chapter, we discuss various methods of constructing optimal controllers.

9.1 Performance Indices

Constructing a performance index, that is, choosing a means to measure the system performance, can be considered as a part of the system modeling. Here, we discuss some typical choices of performance indices.

First, suppose that the objective is to control a dynamical system modeled by the equations,

ẋ(t) = Ax(t) + Bu(t), x(t_{0}) = x_{0}  (9.1)
y(t) = Cx(t),  (9.2)

on a fixed interval [t_{0}, t_{f}] so that the components of the state vector are “small”. A suitable performance index to be minimized would be

J_{1} = ∫_{t_{0}}^{t_{f}} x(t)^{⊤}x(t)dt.  (9.3)

Obviously, if J_{1} is small, then the state vector norm, ∥x(t)∥, is small in the sense of the above performance index.

If the objective is to control the system so that the components of the output, y(t), are to be small, then we could use the performance index

J_{2} = ∫_{t_{0}}^{t_{f}} y(t)^{⊤}y(t)dt
= ∫_{t_{0}}^{t_{f}} x(t)^{⊤}C^{⊤}Cx(t)dt
= ∫_{t_{0}}^{t_{f}} x(t)^{⊤}Qx(t)dt,  (9.4)

where the weight matrix Q = C^{⊤}C is symmetric positive semi-definite.

In the case when we wish to control the system in such a manner that the components of the input, u(t), are “not too large”, a suitable performance index to be minimized is

J_{3} = ∫_{t_{0}}^{t_{f}} u(t)^{⊤}u(t)dt,  (9.5)

or

J_{4} = ∫_{t_{0}}^{t_{f}} u(t)^{⊤}Ru(t)dt,  (9.6)

where the weight matrix R is symmetric positive definite. There is no loss of generality in assuming the weight matrix R to be symmetric. If R was not symmetric, we could represent the quadratic term u^{⊤}Ru equivalently as

u^{⊤}Ru = u^{⊤} (\frac{R + R^{⊤}}{2} ) u,

where the matrix \frac{1}{2} (R + R^{⊤}) is symmetric.

As pointed out by Owens [260, page 186], we cannot simultaneously minimize the performance indices (9.3) and (9.5) because minimization of (9.3) requires large control signals, while minimization of (9.5) requires small control signals. To solve the dilemma, we could compromise between the two conflicting objectives by minimizing the performance index that is a convex combination of J_{1} and J_{3},

J = λJ_{1} + (1 − λ)J_{3}
= ∫_{t_{0}}^{t_{f}} (λx(t)^{⊤}x(t) + (1 − λ)u(t)^{⊤}u(t)) dt,  (9.7)

where λ is a parameter in the range [0, 1]. If λ = 1, then J = J_{1} and if λ = 0, then J = J_{3}. By trial and error, we could select λ from the interval [0, 1] to compromise between the two extremes. A generalization of the performance index (9.7) is

J = ∫_{t_{0}}^{t_{f}} (x(t)^{⊤}Qx(t) + u(t)^{⊤}Ru(t)) dt.

In certain applications, we may wish the final state x(t_{f}) to be as close as possible to 0. Then, a possible performance measure to be minimized is

x(t_{f})^{⊤}F x(t_{f}),  (9.8)

where F is a symmetric positive definite matrix.

We can combine the performance measures (9.4), (9.6), and (9.8) when our control aim is to keep the state “small”, the control “not too large”, and the final state as near to 0 as possible. The resulting performance index is

J = \frac{1}{2}x(t_{f})^{⊤}F x(t_{f}) + \frac{1}{2} ∫_{t_{0}}^{t_{f}} (x(t)^{⊤}Qx(t) + u(t)^{⊤}Ru(t)) dt,  (9.9)

where the factor 1/2 is placed to simplify subsequent algebraic manipulations. Minimizing (9.9) subject to (9.1) is called the linear quadratic regulator problem, or the LQR problem for short.

In some cases, the controller’s goal is to force the system state to track a desired state trajectory, x_{d}(t), throughout the interval [t_{0}, t_{f}] while maintaining the deviations of the actual state x(t) “small” from the desired trajectory with the control effort u(t) “not too large” and the final state x(t_{f}) being as near as possible to some desired state x_{d}(t_{f}). A suitable performance index to be minimized in this case might be

J = \frac{1}{2} (x(t_{f}) − x_{d}(t_{f}))^{⊤} F (x(t_{f}) − x_{d}(t_{f}))
+\frac{1}{2} ∫_{t_{0}}^{t_{f}} ((x(t) − x_{d}(t))^{⊤} Q (x(t) − x_{d}(t)) + u(t)^{⊤}Ru(t)) dt.  (9.10)

9.2 A Glimpse at the Calculus of Variations

The goal of this section is to provide a brief introduction to the subject of the calculus of variations. The calculus of variations deals with minimization or maximization of functionals. A functional is a mapping or a transformation that depends on one or more functions and the values of the functional are numbers. Examples of functionals are performance indices that we introduced in the previous section. We now give two more examples of functionals.

Example 9.1 The length l of the arc connecting two given points A(x_{0}, y_{0}) and B(x_{1}, y_{1}) in the x-y plane, as depicted in Figure 9.1, is an example of a functional. We know that

l = l(y(x)) = ∫_{x_{0}}^{x_{1}} \sqrt{1 + (dy(x)/dx)^{2}}dx,

subject to the constraints y(x_{0}) = y_{0} and y(x_{1}) = y_{1}.

Figure 9.1: The length of arcs joining two given points in the plane is an example of a functional.

Figure 9.2: The area of the surface of revolution connecting two circular disks is an example of a functional.

Example 9.2 Another example of a functional is the area of the surface of revolution formed by a surface connecting two circular disks as illustrated in Figure 9.2. In particular, the surface of revolution can be formed by a soap film clinging to two circular disks. When a wire circle is dipped into a soap solution and withdrawn, a circular disk of soap film bounded by the circle is formed. If a second smaller circle is made to touch the disk and then moved away, the two circles will be joined by a surface of film that is a surface of revolution in the particular case when the circles are parallel and have their centers on the same axis perpendicular to their planes. If we denote the arc of intersection with the x-y plane by y = y(x), then the area of the surface of revolution is

v = v (y(x)) = 2π ∫_{x_{0}}^{x_{1}} y(x)\sqrt{1 + (dy(x)/dx)^{2}}dx,

subject to the constraints y(x_{0}) = y_{0}, y(x_{1}) = y_{1}. One may then want to minimize v = v (y(x)) to find y = y(x) resulting in the surface of revolution of minimum area.

The problem of optimization of functionals dates back to Johann Bernoulli. In 1696 he published a letter in which he posed the problem, which is now known as the brachistochrone problem. Brachistochrone translates from Greek as the shortest time. The brachistochrone problem can be described as follows. A particle of mass m is to slide down along a frictionless track connecting two given points. The motion of the particle is to be constrained only by gravity and the shape of the curve of the track. Find the curve of the track that minimizes the time travel between the two given points. We illustrate the problem in Figure 9.3. The brachistochrone problem was solved by Johann Bernoulli himself, his brother Jacob, Isaac Newton, the Marquis de L’Hospital, Gottfried Leibniz and Tschirnhaus. A solution to the brachistochrone problem is presented in Example 9.23. In 1697 Johann Bernoulli posed and solved the geodesics problem which can be formulated as follows. Find a line of shortest length that connects two given points on a given surface—see Figure 9.4 for an illustration of the problem. The curves of the least length connecting pairs of points on a surface are called geodesics. A general procedure for solving the above type of problems, using the calculus of variations, was given by Leonhard Euler (who flourished between 1707 and 1783).

To acquaint ourselves with the methods of the calculus of variations, we need first to introduce concepts concerning functionals.

Figure 9.3: Illustration of the brachistochrone problem.

9.2.1 Variation and Its Properties

A function can be viewed as a rule, or a mapping, that assigns to each element of some set a unique element of a possibly different set. In particular, a function x : ℝ → ℝ of a real variable t, assigns to each real number a unique real number. An increment of the argument of a function of one variable t is Δt = t − t_{1}.

Similarly, a functional is a mapping that assigns to each function, from some class of functions, a unique number. Thus, we can say that a functional is a “function of a function.” Let x : ℝ → ℝ be an argument of a functional. By a variation δx of an argument x of a functional v we mean the difference of two functions

δx = x − x_{1}.

We assume that x can change in an arbitrary way in some class of functions.

Recall, that a function x : ℝ → ℝ is continuous if a small change of its argument t corresponds to a small change of the value x(t) of the function. Similarly, a functional v is said to be continuous if a “small” change of its argument x corresponds to a small change of the value of the functional.

A functional v is called linear if

v (ax_{1} + x_{2}) = av (x_{1}) + v (x_{2}) ,

Figure 9.4: The problem of finding geodesics.

where a is a constant.

We will now introduce the notion of the variation of a functional. This notion is analogous to the notion of a function differential. To connect the two, we first consider the case of a function of one variable. Let x be a differentiable function defined on an open interval U, and let t ∈ U. The derivative of x at t is defined as

x′(t) = lim_{Δt→0} \frac{x(t + Δt) − x(t)}{Δt}.  (9.11)

Let

φ(Δt) = \frac{x(t + Δt) − x(t)}{Δt} − x′(t).  (9.12)

The function φ is not defined at Δt = 0, however

lim_{Δt→0} φ(Δt) = 0.

We rewrite (9.12) as

x(t + Δt) − x(t) = x′(t)Δt + φ(Δt)Δt.  (9.13)

The above relation has meaning only when Δt ≠ 0. To make it hold at Δt = 0, we define φ(Δt)|_{Δt=0} = 0. To proceed, let

β(Δt) = φ(Δt) if Δt > 0
β(Δt) = −φ(Δt) if Δt < 0. (9.14)

Then, if x is differentiable, there exists a function β such that

x(t + Δt) − x(t) = Δx = x′(t)Δt + β(Δt)|Δt|
= L(t, Δt) + β(Δt)|Δt|,  (9.15)

where lim_{Δt→0} β(Δt) = 0, and L(t, Δt) = x′(t)Δt is a linear function in Δt.

For a real-valued function f = f(x) : ℝ^{n} → ℝ, relation (9.15) takes the form

f(x + Δx) − f(x) = Δf = Df(x)Δx + β(Δx)∥Δx∥
= L(x, Δx) + β(Δx)∥Δx∥,  (9.16)

where lim_{Δx→0} β(Δx) = 0, and

L(x, Δx) = Df(x)Δx = ∇f(x)^{⊤}Δx

is a linear function in Δx. Relation (9.16) holds for operators defined over Banach spaces with scalars being either real or complex numbers—see Definition A.4 of Banach space on page 994. The functionals we consider are operators over a Banach space of continuous functions, C([t_{0}, t_{1}]), with the norm given by (A.99) on page 993.

If an increment Δv = v (x + δx) − v (x) of a functional v can be represented as

Δv = L (x, δx) + β (δx) ∥δx∥,

where L (x, δx) is a linear functional with respect to δx, the term ∥δx∥ = max_{t∈[t_{0},t_{1}]} |δx| denotes the maximal value of |δx|, and β (δx) → 0 if ∥δx∥ → 0, then the linear part L of Δv is called the variation of the functional and is denoted δv, that is,

δv = L (x, δx) .

Example 9.3 We find the variation of the functional

v = ∫_{0}^{1} (2x^{2}(t) + x(t)) dt.

For this we first calculate its increment to get

Δv = v (x + δx) − v (x)
= ∫_{0}^{1} (2(x + δx)^{2} + (x + δx)) dt − ∫_{0}^{1} (2x^{2} + x) dt
= ∫_{0}^{1} (2x^{2} + 4xδx + 2(δx)^{2} + x + δx − 2x^{2} − x) dt
= ∫_{0}^{1}(4x + 1)δxdt + 2 ∫_{0}^{1}(δx)^{2}dt.

The linear part of Δv is

δv = ∫_{0}^{1}(4x + 1)δxdt,

which is the variation of the given functional.

To proceed, note that the linear part on the right-hand side of (9.16) can be computed as

Df(x)Δx = L(x, Δx) = \frac{d}{dα}f(x + αΔx)|_{α=0}  (9.17)

As it was mentioned before, the above also holds for operators over Banach spaces other than Euclidean spaces. Using this observation, we state and prove a lemma that allows us to calculate the variation of a given functional more efficiently than the method we used in Example 9.3.

Lemma 9.1

δv = \frac{d}{dα}v (x + αδx)|_{α=0} .

Proof Suppose that for a given functional v there exists its variation. This means that we can represent Δv as

Δv = v (x + αδx) − v (x) = L(x, αδx) + β(αδx)|α| ∥δx∥.

Then, the derivative of v (x + αδx) with respect to α evaluated at α = 0 is equal to

lim_{α→0} \frac{Δv}{α} = lim_{α→0} \frac{L(x, αδx) + β(αδx)|α| ∥δx∥}{α}
= lim_{α→0} \frac{L(x, αδx)}{α} + lim_{α→0} \frac{β(αδx)|α| ∥δx∥}{α}
= L(x, δx),

because L(·, ·) is linear with respect to the second argument and hence,

L(x, αδx) = αL(x, δx).

Furthermore,

lim_{α→0} \frac{β(αδx)|α| ∥δx∥}{α} = lim_{α→0} β(αδx)∥δx∥ = 0.

This completes the proof.

□

440 CHAPTER 9. OPTIMAL CONTROL Example 9.4 We use Lemma 9.1 to find the variation of the functional considered in Example 9.3. We have

δv = \frac{d}{dα}v (x + αδx) |_{α=0}
= \frac{d}{dα} (∫_{0}^{1} (2(x + αδx)^{2} + (x + αδx)) dt)|_{α=0}
= ∫_{0}^{1} (4(x + αδx)δx + δx) dt|_{α=0}
= ∫_{0}^{1}(4x + 1)δxdt,

which is the variation of the given functional in Example 9.3.

Example 9.5 We find the variation of the functional

v(x_{1}, x_{2}) = v(x_{1}(t), x_{2}(t)) = ∫_{0}^{1} x_{1}e^{−x_{2}}dt,

using the formula

δv = \frac{d}{dα}v(x + αδx)|_{α=0} .

We have

δv = \frac{d}{dα} (∫_{0}^{1} (x_{1} + αδx_{1}) e^{−(x_{2}+αδx_{2})}dt)|_{α=0}
= ∫_{0}^{1} \frac{d}{dα} ((x_{1} + αδx_{1}) e^{−(x_{2}+αδx_{2})}dt)|_{α=0}
= ∫_{0}^{1} (δx_{1}e^{−(x_{2}+αδx_{2})} − δx_{2}e^{−(x_{2}+αδx_{2})}(x_{1} + αδx_{1})) dt|_{α=0}
= ∫_{0}^{1} (e^{−x_{2}}δx_{1} − x_{1}e^{−x_{2}}δx_{2}) dt.

Thus,

δv = ∫_{0}^{1} (e^{−x_{2}}δx_{1} − x_{1}e^{−x_{2}}δx_{2}) dt.

Definition 9.1 A functional v is maximal on some function x_{0} if the value of the functional on any function x close to x_{0} is less than or equal to that on x_{0}, that is,

Δv = v (x) − v (x_{0}) ≤ 0.
◇

We now state a first order necessary condition for a given functional to achieve a maximum or minimum on a curve.

Theorem 9.1 If for a functional v there exists a variation and the functional reaches a maximum or a minimum on a curve x_{0}, then

δv (x_{0}) = 0.

Proof Suppose that x_{0} and δx are fixed. Then

v (x_{0} + αδx) = Φ(α),

that is, v (x_{0} + αδx) is a function of α that for α = 0 achieves its maximum or minimum. This implies that \frac{d}{dα}Φ = 0, or equivalently,

\frac{d}{dα}v (x_{0} + αδx)|_{α=0} = 0.

Hence, by Lemma 9.1, we have δv = 0. Therefore, a functional that achieves its maximum or a minimum on a function x_{0}, has its variation evaluated at x_{0} equal to zero.

□

The above results are used in the next section to determine conditions that a curve has to satisfy to be a maximizer or a minimizer, that is, to be an extremal of a given functional.

9.2.2 Euler-Lagrange Equation

In this section, we consider the following optimization problem.

Problem 1 Suppose that we are given two points (t_{0}, x_{0}) and (t_{1}, x_{1}) in the (t, x)-plane. We wish to find a curve, trajectory, joining the given points such that the functional

v (x) = ∫_{t_{0}}^{t_{1}} F (t, x, ẋ) dt  (9.18)

along this trajectory can achieve its extremal, that is, maximal or minimal, value. The problem is illustrated in Figure 9.5. In solving Problem 1, we use the following lemma.

Figure 9.5: Illustration of Problem 1.

Lemma 9.2 Suppose that a function ϕ : ℝ → ℝ is continuous on the interval [t_{0}, t_{1}]. Then,

∫_{t_{0}}^{t_{1}} ϕ(t)δx(t)dt = 0

if and only if ϕ(t) = 0 at every point of the interval [t_{0}, t_{1}].

Proof Sufficiency is obvious because we can satisfy ∫_{t_{0}}^{t_{1}} ϕ(t)δx(t)dt = 0 by choosing ϕ that vanishes at every point of the interval [t_{0}, t_{1}].

We prove necessity by contraposition. We thus prove that if ϕ is nonzero on some subinterval of [t_{0}, t_{1}], then ∫_{t_{0}}^{t_{1}} ϕ(t)δx(t)dt ≠ 0. Specifically, assume, without loss of generality, that ϕ is positive on the interval [a, b], where t_{0} ≤ a ≤ b ≤ t_{1}. Note that if ϕ were nonzero at only one point, then because it is continuous, it would have to be nonzero in some neighborhood of that point. Hence, we can assume that a < b. Consider a variation given by

δx = δx(t) = \{ \begin{matrix}k^{2}(t − a)^{2}(t − b)^{2} & for & a ≤ t ≤ b \\ 0 & & elsewhere. \end{matrix}  (9.19)

If we assumed ϕ to be negative on the interval [a, b], then we would use δx to be negative of (9.19). We illustrate δx in Figure 9.6. Because δx is positive in [a, b] and zero elsewhere

Figure 9.6: Illustration of the proof of Lemma 9.2.

in the interval [t_{0}, t_{1}], the integrand of ∫_{t_{0}}^{t_{1}} ϕ(t)δx(t)dt will have the same sign as ϕ in [a, b] and will be zero outside this subinterval. Thus

∫_{t_{0}}^{t_{1}} ϕ(t)δx(t)dt = ∫_{a}^{b} ϕ(t)δx(t)dt ≠ 0,

and the proof is complete.

□

Solution to Problem 1 We assume that there is an optimal trajectory x = x(t) joining the points (t_{0}, x_{0}) and (t_{1}, x_{1}) such that δv(x(t)) = 0. Let us then consider an arbitrary acceptable curve x^{∗} = x^{∗}(t) that is close to x and from a family of curves

x(t, α) = x(t) + α (x^{∗}(t) − x(t)) = x(t) + αδx(t).

Note that for α = 0 we get x(t), and for α = 1 we obtain x^{∗}(t). Furthermore,

\frac{d}{dt}(δx(t)) = \frac{d}{dt} (x^{∗}(t) − x(t)) = δx′,

and analogously

\frac{d^{2}}{dt^{2}} (δx(t)) = \frac{d^{2}}{dt^{2}} (x^{∗}(t) − x(t)) = δx′′.

We are now ready to investigate the behavior of the functional v given by (9.18) on the curves from the family x(t, α). Note that the functional v can be considered as a function of α, that is, v (x(t, α)) = Φ(α). It follows from the first order necessary condition for a point to be an extremizer of a function of one variable that α = 0 is a candidate to be an extremizer of Φ if

\frac{dΦ(α)}{dα}|_{α=0} = 0,

where

Φ(α) = ∫_{t_{0}}^{t_{1}} F (t, x(t, α), x′(t, α)) dt,

and x′(t, α) = \frac{d}{dt}x(t, α). We evaluate \frac{dΦ(α)}{dα} to obtain

\frac{dΦ(α)}{dα} = ∫_{t_{0}}^{t_{1}} (F_{x} \frac{d}{dα}x(t, α) + F_{x′} \frac{d}{dα}x′(t, α)) dt,

where F_{x} = \frac{∂}{∂x}F (t, x(t, α), x′(t, α)), and F_{x′} = \frac{∂}{∂x′} F (t, x(t, α), x′(t, α)). Because

\frac{d}{dα}x(t, α) = \frac{d}{dα} (x(t) + αδx(t)) = δx(t),

and

\frac{d}{dα}x′(t, α) = \frac{d}{dα} (x′(t) + αδx′(t)) = δx′(t),

we can write

\frac{d}{dα}Φ(α) = ∫_{t_{0}}^{t_{1}} (F_{x}(t, x(t, α), x′(t, α))δx(t) + F_{x′}(t, x(t, α), x′(t, α))δx′(t)) dt.

By Lemma 9.1,

\frac{d}{dα}Φ(0) = δv.

Therefore,

δv = ∫_{t_{0}}^{t_{1}} (F_{x}δx + F_{x′}δx′) dt,

where for convenience we dropped the argument t. Integrating by parts the last term, and taking into account that δx′ = (δx)′ yields

δv = (F_{x′}δx)|_{t_{0}}^{t_{1}} + ∫_{t_{0}}^{t_{1}} (F_{x} − \frac{d}{dt}F_{x′}) δxdt.

The end points (t_{0}, x_{0}) and (t_{1}, x_{1}) are fixed. This implies that

δx|_{t=t_{0}} = x^{∗}(t_{0}) − x(t_{0}) = 0,

and

δx|_{t=t_{1}} = x^{∗}(t_{1}) − x(t_{1}) = 0.

Hence

δv = ∫_{t_{0}}^{t_{1}} (F_{x} − \frac{d}{dt}F_{x′}) δxdt.

By Lemma 9.2,

δv = 0 if and only if F_{x} − \frac{d}{dt}F_{x′} = 0.

The equation,

F_{x} − \frac{d}{dt}F_{x′} = 0  (9.20)

is known as the Euler-Lagrange equation. The curves x = x(t, C_{1}, C_{2}) that are solutions to the Euler-Lagrange equations are called extremals. The constants C_{1} and C_{2} are the integration constants. Thus, a functional can only achieve its extremum on the extremals.

The Euler-Lagrange equation F_{x} − \frac{d}{dt}F_{x′} = 0 can also be represented as

F_{x} − F_{x′t} − F_{x′x}x′ − F_{x′x′}x′′ = 0.

Example 9.6 We find the extremal of the functional

v = ∫0^{\frac{π}{2}} ((x′)^{2} − x^{2}) dt,  (9.21)

where x(0) = 0, and x (\frac{π}{2}) = 1. The Euler-Lagrange equation for the problem takes the form

F_{x} − \frac{d}{dt}F_{x′} = \frac{∂}{∂x} ((x′)^{2} − x^{2}) − \frac{d}{dt} (\frac{∂}{∂x′} ((x′)^{2} − x^{2}))
= −2x − \frac{d}{dt} (2x′)
= −2x − 2x′′
= 0.

Equivalently, the Euler-Lagrange equation for the functional (9.21) can be represented as

x′′ + x = 0.

The characteristic equation of the above differential equation is s^{2} + 1 = 0. Therefore, the general solution of the Euler-Lagrange equation has the form

x(t) = C_{1} cos t + C_{2} sin t.

446 CHAPTER 9. OPTIMAL CONTROL Taking into account the end-point conditions gives

x(0) = C_{1} = 0,
x (\frac{π}{2} ) = C_{2} = 1.

Thus, we have one extremal

x(t) = sin t.

The functional (9.21) can be minimized or maximized only on the extremal x(t) = sin t.

Example 9.7 We find the extremal of the functional

v = ∫_{0}^{1} ((x′)^{2} + 12xt) dt,  (9.22)

where x(0) = 0, and x(1) = 1. The Euler-Lagrange equation for the this functional is

F_{x} − \frac{d}{dt}F_{x′} = \frac{∂}{∂x} ((x′)^{2} + 12xt) − \frac{d}{dt} (\frac{∂}{∂x′} ((x′)^{2} + 12xt))
= 12t − \frac{d}{dt} (2x′)
= 12t − 2x′′
= 0.

Equivalently, the Euler-Lagrange equation for the functional (9.22) can be represented as

x′′ = 6t.

The general solution to the above differential equation is

x(t) = C_{1}t + C_{2} + t^{3}.

From the end-point conditions, we get

x(0) = C_{2} = 0,
x(1) = C_{1} + 1 = 1.

Hence, C_{1} = 0, and the extremal has the form x(t) = t^{3}.

Suppose now that A = A(t_{0}, x_{10}, …, x_{n0}) and B = B(t_{1}, x_{11}, …, x_{n1}) are two given points in the (n + 1)-dimensional space of (n + 1)-tuples, (t, x_{1}, …, x_{n}), along with the functional

v = v(x_{1}, …, x_{n}) = ∫_{t_{0}}^{t_{1}} F (t, x_{1}, …, x_{n}, ẋ_{1}, …, ẋ_{n})dt.

To find a trajectory along which δv = 0, we consider only the variation in one coordinate, say x_{i}, while other coordinates are fixed. We perform this process for each coordinate. The functional v will depend only upon x_{i}, and we write

v(x_{1}, …, x_{i}, …, x_{n}) = ṽ(x_{i}).

Thus, we reduced the given n-dimensional problem to n one-dimensional problems. Proceeding as before, we conclude that the functional v = v(x_{1}, …, x_{n}) can only be extremized on the trajectories that satisfy n Euler-Lagrange equations,

F_{x_{i}} − \frac{d}{dt}Fẋ_{i} = 0, i = 1, …, n.  (9.23)

Observe that if in (9.23) we set F = L = K − U, where L is the Lagrangian, K = K(q̇_{1}, …, q̇_{n}) is the kinetic energy of the particle, U = U(q_{1}, …, q_{n}) is its potential energy, and x_{i} = q_{i}, then (9.23) becomes

\frac{∂L}{∂q_{i}} − \frac{d}{dt} (\frac{∂L}{∂q̇_{i}}) = 0, i = 1, 2, …, n,

which are the Lagrange equations of motion for a particle. Referring to the solution to Problem 1 on page 443, we conclude that a particle moves along a path on which the functional

J = ∫_{t_{1}}^{t_{2}} Ldt,

is extremized, which agrees with the Hamilton principle saying that a particle moves along the path that minimizes the integral J.

In our discussion up to now, both end-points were fixed. We would like to solve a more general optimization problem, where the end-points need not be fixed. For simplicity, we assume first that one of the end-points is fixed, while the other is free. After we solve this problem, we can easily generalize to the case when both end-points are free.

Problem 2 In the (t, x)-plane the left-end point A is fixed, while the right end-point B is free—see Figure 9.7. It is required to find a trajectory x = x(t) starting at A on which the functional v = ∫_{t_{0}}^{t_{1}} F (t, x, x′)dt can achieve its extremal value.

Figure 9.7: Illustration of Problem 2.

Solution to Problem 2 Following Elsgolc [96, pages 64–72], we begin by assuming that a solution to Problem 2 exists, that is, there is a trajectory x = x(t) joining (t_{0}, x_{0}) and (t_{1}, x_{1}) such that δv = 0. This trajectory must satisfy the Euler-Lagrange equation. Consider an adjoining trajectory specified by x + δx joining (t_{0}, x_{0}) and (t_{1} + δt_{1}, x_{1} + δx_{1}). Then,

Δv = v(x + δx) − v(x)
= ∫_{t_{0}}^{t_{1}+δt_{1}} F (t, x + δx, x′ + δx′)dt − ∫_{t_{0}}^{t_{1}} F (t, x, x′)dt
= ∫_{t_{1}}^{t_{1}+δt_{1}} F (t, x + δx, x′ + δx′)dt + ∫_{t_{0}}^{t_{1}} (F (t, x + δx, x′ + δx′) − F (t, x, x′)) dt.

Our goal is to extract from Δv its linear portion in δx, which we know is the variation of the given functional. We begin by approximating the first integral as

∫_{t_{1}}^{t_{1}+δt_{1}} F (t, x + δx, x′ + δx′)dt ≈ F (t, x, x′)|_{t=t_{1}} δt_{1}.

Expanding the integrand of the second integral into a Taylor series and retaining only firstorder terms yields

∫_{t_{0}}^{t_{1}} (F (t, x + δx, x′ + δx′) − F (t, x, x′)) dt = ∫_{t_{0}}^{t_{1}} (F_{x}(t, x, x′)δx + F_{x′}(t, x, x′)δx′) dt.

Integrating the second term by parts and combining with the first term gives

∫_{t_{0}}^{t_{1}} (F_{x}δx + F_{x′}δx′) dt = (F_{x′}δx) |_{t_{0}}^{t_{1}} + ∫_{t_{0}}^{t_{1}} (F_{x} − \frac{d}{dt}F_{x′}) δxdt,

where for simplicity of notation we suppressed the arguments of F_{x} and F_{x′}. We assumed that a solution to the problem exists. Hence, an extremal must satisfy the Euler-Lagrange equation F_{x} − \frac{d}{dt}F_{x′} = 0. Because the left end-point is fixed, we obtain

δx|_{t=t_{0}} = 0.

Taking into account the results of the above manipulations yields

Δv ≈ F (t, x, x′)|_{t=t_{1}} δt_{1} + (F_{x′}δx)|_{t=t_{1}} .  (9.24)

The final step in obtaining δv is to evaluate (F_{x′}δx) |_{t_{1}}. The problem is two dimensional so we can illustrate it graphically as shown in Figure 9.8. Referring to Figure 9.8, we obtain

DB = δx|_{t=t_{1}},
F C = δx_{1},
EC ≈ x′(t_{1})δt_{1},
DB = F C − EC.

Hence,

δx|_{t=t_{1}} ≈ δx_{1} − x′(t_{1})δt_{1}.

Substituting the above into (9.24) gives

δv = F (t, x, x′)|_{t=t_{1}}δt_{1} + F_{x′}|_{t=t_{1}} (δx_{1} − x′(t_{1})δt_{1})
= (F − x′F_{x′}) |_{t=t_{1}}δt_{1} + F_{x′}|_{t=t_{1}}δx_{1}.

If the variations δt_{1} and δx_{1} are arbitrary, we obtain δv = 0 if in addition to the Euler-Lagrange equation the following end-point conditions are also satisfied,

(F − x′F_{x′}) |_{t=t_{1}} = 0 and F_{x′}|_{t=t_{1}} = 0

The above end-point relations are called the transversality conditions. In conclusion, the optimal trajectory must satisfy both the Euler-Lagrange equation and the transversality conditions.

Figure 9.8: A geometric interpretation of calculating the variation of the functional for Problem 2.

Example 9.8 Consider the functional of Example 9.7, where the left end-point is fixed as before, that is, x(0) = 0, while the right end-point x(1) is free. Here, we have t_{1} = 1. Because δt_{1} = 0 and δx_{1} is arbitrary, the transversality conditions for the right end-point are reduced to one condition,

F_{x′}|_{t=t_{1}} = 0.

Taking into account the solution to the Euler-Lagrange equation obtained in Example 9.7, and then using the fact that the left end-point is fixed, we obtain

0 = F_{x′}|_{t=t_{1}}
= 2x′(t_{1})
= 2(C_{1} + 3t^{2})|_{t=1}
= 2(C_{1} + 3).

Hence, C_{1} = −3 and the extremal for this case is

x(t) = −3t + t^{3}.

In Figure 9.9, we show a plot of the above extremal.

Figure 9.9: A plot of the extremal, x(t) = −3t + t^{3}, of Example 9.8.

Example 9.9 We consider again the same functional as in Example 9.7, where, this time, the left end-point is fixed, as before, while at the right-end point t_{1} is free and x(t_{1}) = 1. Hence, in this case, δx_{1} = 0 while δt_{1} is arbitrary. The transversality conditions reduce to one condition,

(F − x′F_{x′}) |_{t=t_{1}} = 0.

We perform simple calculations to obtain,

F − x′F_{x′} = 12xt − (x′)^{2},

where

x′ = (C_{1}t + t^{3})′ = C_{1} + 3t^{2}.

Note that because x(t_{1}) = C_{1}t_{1} + t_{1}^{3} = 1, we have

C_{1} = \frac{1 − t_{1}^{3}}{t_{1}} .

Taking into account the above and the fact that x(t_{1}) = 1, we perform simple algebraic manipulations to arrive at the following equation for t_{1},

(F − x′F_{x′}) |_{t=t_{1}} = (12x(t_{1})t_{1} − (x′(t_{1}))^{2}) = −4t_{1}^{6} + 8t_{1}^{3} − 1 = 0.

Solving the above equation, we obtain the following two solutions,

t_{1} = (1 ± \sqrt{\frac{3}{4}})^{1/3} .

Figure 9.10: Plots of the extremals of Example 9.9.

Plots of the two extremals for this example are shown in Figure 9.10.

If the position of the end point B is restricted to a curve x = g_{1}(t), then

δx_{1} ≈ g′_{1}(t_{1})δt_{1},

and hence

δv = (F + (g′_{1} − x′) F_{x′}) δt_{1} = 0

if

(F + (g′_{1} − x′) F_{x′}) |_{t=t_{1}} = 0.  (9.25)

The above is the transversality condition for the case when the end-point B is restricted to the curve x = g_{1}(t).

In the next section, we discuss a specific problem of minimizing a functional subject to constraints in the form of differential equations.

9.3 Linear Quadratic Regulator

9.3.1 Algebraic Riccati Equation (ARE)

Consider a linear system modeled by

ẋ = Ax + Bu, x(0) = x_{0},

and the associated performance index

J = ∫_{0}^{∞} (x^{⊤}Qx + u^{⊤}Ru) dt,

where Q = Q^{⊤} ≥ 0 and R = R^{⊤} > 0. Our goal is to construct a stabilizing linear statefeedback controller of the form u = −Kx that minimizes the performance index J. We denote such a linear control law by u^{∗}.

First, we assume that a linear state-feedback optimal controller exists such that the optimal closed-loop system

ẋ = (A − BK)x

is asymptotically stable. The above assumption implies that there is a Lyapunov function V = x^{⊤}P x for the closed-loop system, that is, for some positive definite matrix P the time derivative dV/dt evaluated on the trajectories of the closed-loop system is negative definite.

Theorem 9.2 If the state-feedback controller u^{∗} = −Kx is such that

{min \atop u} (\frac{dV}{dt} + x^{⊤}Qx + u^{⊤}Ru) = 0,  (9.26)

for some V = x^{⊤}P x, then the controller is optimal.

Proof We can rewrite the condition of the theorem as

\frac{dV}{dt} |_{u=u^{∗}} + x^{⊤}Qx + u^{∗⊤}Ru^{∗} = 0.

Hence,

\frac{dV}{dt} |_{u=u^{∗}} = −x^{⊤}Qx − u^{∗⊤}Ru^{∗}.

Integrating both sides of the resulting equation with respect to time from 0 to ∞, we obtain

V (x(∞)) − V (x(0)) = − ∫_{0}^{∞} (x^{⊤}Qx + u^{∗⊤}Ru^{∗}) dt.

Because by assumption the closed-loop system is asymptotically stable, x(∞) = 0, and therefore

V (x(0)) = x_{0}^{⊤}P x_{0} = ∫_{0}^{∞} (x^{⊤}Qx + u^{∗⊤}Ru^{∗}) dt.

Thus, we showed that if a linear state-feedback controller satisfies the assumption of the theorem, then the value of the performance index for such a controller is

J (u^{∗}) = x_{0}^{⊤}P x_{0}.

To show that such a controller is indeed optimal, we use a proof by contradiction. We assume that (9.26) holds and that u^{∗} is not optimal. Suppose that ũ yields a smaller value of J, that is,

J \binom{}{ũ} < J (u^{∗}) .

It follows from (9.26) that

\frac{dV}{dt} |_{u=u^{}}˜ + x^{⊤}Qx + ũ^{⊤}Rũ ≥ 0,

that is,

\frac{dV}{dt} |_{u=u^{}}˜ ≥ −x^{⊤}Qx − ũ^{⊤}Rũ.

Integrating the above with respect to time from 0 to ∞ yields

V (x(0)) ≤ ∫_{0}^{∞} (x^{⊤}Qx + ũ^{⊤}Rũ) dt

implying that

J(u^{∗}) ≤ J(ũ),

which is a contradiction, and the proof is complete.

□

It follows from the above theorem that the synthesis of the optimal control law involves finding an appropriate Lyapunov function, or equivalently, the matrix P. The appropriate P is found by minimizing

f(u) = \frac{dV}{dt} + x^{⊤}Qx + u^{⊤}Ru,  (9.27)

We first apply to (9.27) the necessary condition for unconstrained minimization,

\frac{∂}{∂u} (\frac{dV}{dt} + x^{⊤}Qx + u^{⊤}Ru)|_{u=u^{∗}} = 0.

Differentiating the above yields

\frac{∂}{∂u} (\frac{dV}{dt} + x^{⊤}Qx + u^{⊤}Ru) = \frac{∂}{∂u} (2x^{⊤}P ẋ + x^{⊤}Qx + u^{⊤}Ru)
= \frac{∂}{∂u} (2x^{⊤}P Ax + 2x^{⊤}P Bu + x^{⊤}Qx + u^{⊤}Ru)
= 2x^{⊤}P B + 2u^{⊤}R
= 0.

Hence, the candidate for an optimal control law has the form

u^{∗} = −R^{−1}B^{⊤}P x = −Kx,  (9.28)

where K = R^{−1}B^{⊤}P. Note that

\frac{∂^{2}}{∂u^{2}} (\frac{dV}{dt} + x^{⊤}Qx + u^{⊤}Ru) = \frac{∂^{2}}{∂u^{2}} (2x^{⊤}P Ax + 2x^{⊤}P Bu + x^{⊤}Qx + u^{⊤}Ru)
= \frac{∂}{∂u} (2x^{⊤}P B + 2u^{⊤}R)
= 2R
> 0.

Thus the second order sufficiency condition for u^{∗} to minimize (9.27) is satisfied.

We next turn our attention to finding an appropriate matrix P. The optimal closed-loop system has the form

ẋ = (A − BR^{−1}B^{⊤}P ) x, x(0) = x_{0}.

Our optimal controller satisfies the equation

\frac{dV}{dt} |_{u=u^{∗}} + x^{⊤}Qx + u^{∗⊤}Ru^{∗} = 0,

that is,

2x^{⊤}P Ax + 2x^{⊤}P Bu^{∗} + x^{⊤}Qx + u^{∗⊤}Ru^{∗} = 0.

We substitute the expression for u^{∗}, given by (9.28), into the above equation and represent it as

x^{⊤} (A^{⊤}P + P A) x − 2x^{⊤}P BR^{−1}B^{⊤}P x + x^{⊤}Qx + x^{⊤}P BR^{−1}B^{⊤}P x = 0.

Factoring out x yields

x^{⊤} (A^{⊤}P + P A + Q − P BR^{−1}B^{⊤}P ) x = 0.

The above equation should hold for any x. For this to be true we have to have

A^{⊤}P + P A + Q − P BR^{−1}B^{⊤}P = O  (9.29)

The above equation is referred to as the algebraic Riccati equation (ARE). In conclusion, the synthesis of the optimal linear state feedback controller minimizing the performance index

J = ∫_{0}^{∞} (x^{⊤}Qx + u^{⊤}Ru) dt

456 CHAPTER 9. OPTIMAL CONTROL

subject to

ẋ = Ax + Bu, x(0) = x_{0}

requires solving the ARE given by (9.29). Example 9.10 Consider the following model of a dynamical system:

ẋ = 2u_{1} + 2u_{2}, x(0) = 3,

and the associated performance index

J = ∫_{0}^{∞} (x^{2} + ru_{1}^{2} + ru_{2}^{2}) dt,

where r > 0 is a parameter.

(i) We first find the solution to the ARE corresponding to the linear state-feedback optimal

controller. We have

A = 0, B = [ 2 2 ] , Q = 1, R = rI_{2}.

The ARE for this problem is

O = A^{⊤}P + P A + Q − P BR^{−1}B^{⊤}P = 1 − \frac{8}{r}p^{2},

whose solution is

p = \sqrt{\frac{r}{8}}.

(ii) We write the equation of the closed-loop system driven by the optimal controller,

u = −R^{−1}B^{⊤}P x = −\frac{1}{\sqrt{2r}} [ \begin{matrix}1 \\ 1 \end{matrix} ] x.

The closed-loop optimal system is described by

ẋ = [ 2 2 ] u = −\frac{4}{\sqrt{2r}}x.

(iii) Finally, we find the value of J for the optimal closed-loop system. We have

J = x(0)^{⊤}P x(0) = \frac{9}{2}\sqrt{\frac{r}{2}}.

We can directly verify that J = \frac{9}{2} \sqrt{\frac{r}{2}} as follows:

min J = min ∫_{0}^{∞} (x^{2} + ru_{1}^{2} + ru_{2}^{2}) dt
= ∫_{0}^{∞} 9 exp (−\frac{8t}{\sqrt{2r}}) (1 + 1) dt = 9\frac{\sqrt{2r}}{4} = \frac{9}{2}\sqrt{\frac{r}{2}}.

A major difficulty when solving the ARE is that it is a nonlinear equation. In the next section, we present an efficient method for solving the ARE.

9.3.2 Solving the ARE Using the Eigenvector Method

We now present a method for solving the ARE referred to as the MacFarlane and Potter method, or the eigenvector method. In our discussion of the method, we follow the development of Power and Simpson [278]. We begin by representing the ARE in the form

[ P −I_{n} ] [ \begin{matrix}A & −BR^{−1}B^{⊤} \\ −Q & −A^{⊤} \end{matrix} ] [ \begin{matrix}I & n \\ P \end{matrix} ] = O.  (9.30)

The 2n × 2n matrix in the middle is called the Hamiltonian matrix. We use the symbol H to denote the Hamiltonian matrix, that is,

H = [ \begin{matrix}A & −BR^{−1}B^{⊤} \\ −Q & −A^{⊤} \end{matrix} ] .

Thus, the ARE can be represented as

[ P −I_{n} ] H [ \begin{matrix}I & n \\ P \end{matrix} ] = O.

We premultiply the above equation by X^{−1}, and then postmultiply it by X, where X is a nonsingular n × n matrix,

[ X^{−1}P −X^{−1} ] H [ \begin{matrix}X \\ P & X \end{matrix} ] = O.  (9.31)

Observe that if we can find matrices X and P X such that

H [ \begin{matrix}X \\ P & X \end{matrix} ] = [ \begin{matrix}X \\ P & X \end{matrix} ] Λ,

then equation (9.31) becomes

[ X^{−1}P −X^{−1} ] [ \begin{matrix}X \\ P & X \end{matrix} ] Λ = O.

We have thus reduced the problem of solving the ARE to that of constructing appropriate matrices X and P X. To proceed further, let v_{i} be an eigenvector of H and s_{i} the corresponding eigenvalue, then

Hv_{i} = s_{i}v_{i}.

In our further discussion, we assume that H has at least n distinct real eigenvalues among its 2n eigenvalues. (The results obtained can be generalized for the case when the eigenvalues of H are complex or non-distinct.) Thus, we can write

H [ v_{1} v_{2} ⋯ v_{n} ] = [ v_{1} v_{2} ⋯ v_{n} ] [ \begin{matrix}s_{1} & 0 & ⋯ & 0 \\ 0 & s_{2} & ⋯ & 0 \\ ⋮ & ⋮ & & ⋮ \\ 0 & 0 & ⋯ & s_{n} \end{matrix} ] .

Let

[ \begin{matrix}X \\ P & X \end{matrix} ] = [ v_{1} v_{2} ⋯ v_{n} ] ,

and

Λ = [ \begin{matrix}s_{1} & 0 & ⋯ & 0 \\ 0 & s_{2} & ⋯ & 0 \\ ⋮ & ⋮ & & ⋮ \\ 0 & 0 & ⋯ & s_{n} \end{matrix} ] .

The above choice of X and P X constitutes a possible solution to the equation

[ X^{−1}P −X^{−1} ] [ \begin{matrix}X \\ P & X \end{matrix} ] Λ = O.

To construct P, we partition the 2n × n eigenvector matrix [ v_{1} v_{2} ⋯ v_{n} ] into two n × n submatrices as follows

[ v_{1} v_{2} ⋯ v_{n} ] = [ \begin{matrix}W \\ Z \end{matrix} ] .

Thus,

[ \begin{matrix}X \\ P & X \end{matrix} ] = [ \begin{matrix}W \\ Z \end{matrix} ] .

Taking X = W, P X = Z, and assuming that W is invertible we obtain

P = ZW^{−1}  (9.32)

We now have to decide what set of n eigenvalues should be chosen from those of H to construct the particular P. In the case when all 2n eigenvalues of H are distinct, the number of different matrices P generated by the above described method is

\frac{(2n)!}{(n!)^{2}}.

Let Q = C^{⊤}C be a full rank factorization of Q, as defined on page 970. It follows from Theorem 3 of Kučera [196, page 346] that the Hamiltonian matrix H has n eigenvalues in the open left-half complex plane and n in the open right-half plane if and only if the pair (A, B) is stabilizable and the pair (A, C) is detectable. The matrix P that we seek corresponds to the asymptotically stable eigenvalues of H. The eigenvalues of the Hamiltonian matrix come in pairs ±s_{i} (see Exercise 9.15). The characteristic polynomial of H contains only even powers of s. With such constructed matrix P, we have the following result:

Theorem 9.3 The poles of the closed-loop system

ẋ(t) = (A − BR^{−1}B^{⊤}P ) x(t)

are those eigenvalues of H having negative real parts.

Proof Because [ v_{1} v_{2} ⋯ v_{n} ] = [ \begin{matrix}W \\ Z \end{matrix} ], we can write

[ \begin{matrix}A & −BR^{−1}B^{⊤} \\ −Q & −A^{⊤} \end{matrix} ] [ \begin{matrix}W \\ Z \end{matrix} ] = [ \begin{matrix}W \\ Z \end{matrix} ] Λ.

Performing appropriate multiplication of n × n block matrices yields

AW − BR^{−1}B^{⊤}Z = W Λ,

or

A − BR^{−1}B^{⊤}ZW^{−1} = A − BR^{−1}B^{⊤}P = W ΛW^{−1},

because P = ZW^{−1}. Thus, the matrix A − BR^{−1}B^{⊤}P is similar to the matrix Λ whose eigenvalues are the asymptotically stable eigenvalues of H. The proof is complete.

□

Example 9.11 Consider the following model of a dynamical system

ẋ = 2x + u,

and the associated performance index

J = ∫_{0}^{∞}(x^{2} + ru^{2}) dt.

Find the value of r such that the optimal closed-loop system has its pole at −3. We form the associated Hamiltonian matrix

H = [ \begin{matrix}A & −BR^{−1}B^{⊤} \\ −Q & −A^{⊤} \end{matrix} ] = [ \begin{matrix}2 & −\frac{1}{r} \\ −1 & −2 \end{matrix} ] .

The characteristic equation of H is

det(sI_{2} − H) = s^{2} − 4 − \frac{1}{r} = 0.

Hence,

r = \frac{1}{5}

results in the optimal closed-loop system having its pole located at −3.

We mention again that the eigenvector method for solving the ARE can be generalized to the case when the eigenvalues of the Hamiltonian matrix H are not necessarily distinct or real.

9.3.3 Optimal Systems With Prescribed Poles

We can always find a state-feedback u = −Kx such that the closed-loop system ẋ = (A − BK)x will have its poles in prescribed locations (symmetric with respect to the real axis) if and only if the pair (A, B) is reachable. In the multi-input systems the gain matrix K that shifts the poles to the prescribed locations is not unique. We would like to use the remaining degrees of freedom to construct a gain matrix K that shifts the system poles to the desired locations and at the same time minimizes a performance index. Specifically, we are given a system model,

ẋ(t) = Ax(t) + Bu(t),

the desired set of closed-loop poles

{s_{1}, s_{2}, …, s_{n}},

and a m×m positive definite matrix R = R^{⊤}. We wish to construct a state-feedback control law u(t) = −Kx(t) and an n×n positive semi-definite matrix Q = Q^{⊤} such that the closedloop system has its poles in the desired locations, and at the same time the performance index

J = ∫_{0}^{∞} (x(t)^{⊤}Qx(t) + u(t)^{⊤}Ru(t)) dt

is minimized. In other words, we are interested in designing an optimal control system with prescribed poles. We illustrate the problem with a very simple example.

Example 9.12 For a dynamical system modeled by

ẋ = 2x + u
= Ax + Bu,

and the performance index

J = ∫_{0}^{∞}(qx^{2} + u^{2})dt
= ∫_{0}^{∞} (x^{⊤}Qx + u^{⊤}Ru) dt,

find q so that the eigenvalue of

A − BR^{−1}B^{⊤}P

is equal to −3.

The corresponding Hamiltonian matrix is

H = [ \begin{matrix}2 & −1 \\ −q & −2 \end{matrix} ] .

The characteristic polynomial of H is

det[sI_{2} − H] = det [ \begin{matrix}s − 2 & 1 \\ q & s + 2 \end{matrix} ] = s^{2} − 4 − q.

We want

det[sI_{2} − H] = (s + 3)(s − 3) = s^{2} − 9.

Hence, the desired value of q is q = 5.

We now present a method for designing optimal control systems with prescribed poles. In constructing the gain matrix K and the weight matrix Q, we use repeatedly a state variable transformation. To proceed, we analyze the system model, the performance index, and the Hamiltonian matrix in a new basis. Let

x = T x̃,

where det T ≠ 0. Then, the system model in the new coordinates has the form

˙x̃(t) = T^{−1}AT x̃(t) + T^{−1}Bu(t)
= Ãx̃(t) + B̃u(t).

The performance index J, after state variable transformation, takes the form

J̃ = ∫_{0}^{∞} (x̃(t)^{⊤}T^{⊤}QT x̃(t) + u(t)^{⊤}Ru(t)) dt.

The Hamiltonian matrix is transformed to

H̃ = [ \begin{matrix}T^{−1}AT & −T^{−1}BR^{−1}B^{⊤}T^{−⊤} \\ −T^{T} QT & −T^{⊤}A^{⊤}T^{−⊤} \end{matrix} ]
= [ \begin{matrix}Ã & −B̃R^{−1}B̃^{⊤} \\ −Q̃ & −Ã^{⊤} \end{matrix} ] ,

where T^{−⊤} = (T^{−1})^{⊤} = (T^{⊤})^{−1}. Observe that

H̃ = [ \begin{matrix}T^{−1} & 0 \\ 0 & T^{⊤} \end{matrix} ] H [ \begin{matrix}T & 0 \\ 0 & T^{−⊤} \end{matrix} ] ,

that is, H̃ and H are similarity matrices because

[ \begin{matrix}T^{−1} & 0 \\ 0 & T^{⊤} \end{matrix} ] [ \begin{matrix}T & 0 \\ 0 & T^{−⊤} \end{matrix} ] = I_{2n}.

Hence, the eigenvalues of H and H̃ are identical. Our next step is to compute the characteristic equation of H̃ as a function of Q̃. Computing det (sI_{2n} − H̃) is facilitated after we perform some manipulations on the characteristic matrix [sI_{2n} − H̃] that do not change its determinant. Specifically, we premultiply the characteristic matrix [sI_{2n} − H̃] by a 2n×2n matrix whose determinant equals 1. Then, the determinant of the product is the same as det (sI_{2n} − H̃), but it is much easier to evaluate. We have

[ \begin{matrix}I_{n} & 0 \\ −Q̃ (sI_{n} − Ã)^{−1} & I_{n} \end{matrix} ] [ \begin{matrix}sI_{n} − Ã & B̃R^{−1}B̃^{⊤} \\ Q̃ & sI_{n} + Ã^{⊤} \end{matrix} ]

= [ \begin{matrix}sI_{n} − Ã & B̃R^{−1}B̃^{⊤} \\ 0 & sI_{n} + Ã^{⊤} − Q̃ (sI_{n} − Ã)^{−1} B̃R^{−1}B̃^{⊤} \end{matrix} ] .

Let

F = B̃R^{−1}B̃^{⊤},

then

det (sI_{2n} − H̃) = det [ \begin{matrix}sI_{n} − Ã & F \\ 0 & sI_{n} + Ã^{⊤} − Q̃ (sI_{n} − Ã)^{−1} F \end{matrix} ]
= det (sI_{n} − Ã) det (sI_{n} + Ã^{⊤} − Q̃ (sI_{n} − Ã)^{−1} F ) .

Suppose that the matrix A has n linearly independent eigenvectors, hence it is diagonalizable. Let

T = [ t_{1} t_{2} ⋯ t_{n} ]

be a matrix composed of the eigenvectors of A. Therefore,

Ã = T^{−1}AT = [ \begin{matrix}λ_{1} & 0 & ⋯ & 0 \\ 0 & λ_{2} & ⋯ & 0 \\ ⋮ & ⋮ & & ⋮ \\ 0 & 0 & ⋯ & λ_{n} \end{matrix} ] .

Note that Ã^{⊤} = Ã. Assume that

Q̃ = Q̃_{j} = [ \begin{matrix}0 & 0 & 0 & ⋯ & 0 \\ ⋮ & ⋮ & ⋮ & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 \\ 0 & 0 & q̃_{jj} & ⋯ & 0 \\ ⋮ & ⋮ & ⋮ & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 \end{matrix} ] ,

that is, Q̃_{j} has only one non-zero element q̃_{jj}. Under the above assumptions the matrix [sI_{n} + Ã^{⊤} − Q̃_{j} (sI_{n} − Ã)^{−1} F ] takes the form

sI_{n} + Ã^{⊤} − Q̃_{j} (sI_{n} − Ã)^{−1} F = [ \begin{matrix}s + λ_{1} & 0 & ⋯ & 0 \\ 0 & s + λ_{2} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & s + λ_{n} \end{matrix} ]

− [ \begin{matrix}0 & 0 & 0 & ⋯ & 0 \\ ⋮ & ⋮ & ⋮ & & ⋮ \\ 0 & 0 & q̃_{jj} & 0 & 0 \\ 0 & 0 & 0 & ⋯ & 0 \end{matrix} ] [ \begin{matrix}\frac{1}{s−λ_{1}} & 0 & ⋯ & 0 \\ 0 & \frac{1}{s−λ_{2}} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & \frac{1}{s−λ_{n}} \end{matrix} ] [ \begin{matrix}f_{11} & f_{12} & ⋯ & f_{1n} \\ f_{21} & f_{22} & ⋯ & f_{2n} \\ ⋮ & ⋮ & ... & ⋮ \\ f_{n1} & f_{n2} & ⋯ & f_{nn} \end{matrix} ]

= [ \begin{matrix}s + λ_{1} & 0 & 0 & ⋯ & 0 \\ 0 & s + λ_{2} & 0 & ⋯ & 0 \\ ⋮ & ⋮ & ⋮ & & ⋮ \\ 0 & 0 & s + λ_{j} & & 0 \\ ⋮ & ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & 0 & ⋯ & s + λ_{n} \end{matrix} ] − [ \begin{matrix}0 & 0 & 0 & ⋯ & 0 \\ 0 & 0 & 0 & ⋯ & 0 \\ ⋮ & ⋮ & ⋮ & & ⋮ \\ \frac{q̃_{jj}f_{j1}}{s−λ_{j}} & \frac{q̃_{jj}f_{j2}}{s−λ_{j}} & \frac{q̃_{jj}f_{jj}}{s−λ_{j}} & ⋯ & \frac{q̃_{jj}f_{jn}}{s−λ_{j}} \\ ⋮ & ⋮ & ⋮ & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 \end{matrix} ]

= [ \begin{matrix}s + λ_{1} & 0 & 0 & ⋯ & 0 & 0 \\ 0 & s + λ_{2} & 0 & ⋯ & 0 & 0 \\ ⋮ & ⋮ & ⋮ & & ⋮ & ⋮ \\ −\frac{q̃_{jj}f_{j1}}{s−λ_{j}} & −\frac{q̃_{jj}f_{j2}}{s−λ_{j}} & ⋯ & ⋯ & s + λ_{j} − \frac{q̃_{jj}f_{jj}}{s−λ_{j}} & −\frac{q̃_{jj}f_{jn}}{s−λ_{j}} \\ ⋮ & ⋮ & ⋮ & ... & ⋮ & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & s + λ_{n} \end{matrix} ] .

Hence,

det (sI_{n} + Ã^{⊤} − Q̃_{j} (sI_{n} − Ã)^{−1} F ) = (∏_{i≠j^{i=1}}^{n}(s + λ_{i})) (s + λ_{j} − \frac{q̃_{jj}f_{jj}}{s − λ_{j}} ) .

We can now evaluate the determinant of the Hamiltonian matrix,

det(sI_{2n} − H) = det(sI_{2n} − H̃)
= (∏_{i=1}^{n}(s − λ_{i})) (∏_{i≠j^{i=1}}^{n}(s + λ_{i})) (s + λ_{j} − \frac{q̃_{jj}f_{jj}}{s − λ_{j}} )

= ∏_{i≠j^{i=1}}^{n}(s + λ_{i})(s − λ_{i}) ((s + λ_{j})(s − λ_{j}) − q̃_{jj}f_{jj}) .

From the form of the determinant of the Hamiltonian matrix, we conclude that selecting an appropriate value of q̃_{jj} allows us to shift the j-th pole of the closed-loop system into a prespecified location while leaving the other poles in their original positions. To see this we represent the term that involves q̃_{jj} as

(s + λ_{j})(s − λ_{j}) − q̃_{jj}f_{jj} = s^{2} − λ_{j}^{2} − q̃_{jj}f_{jj}
= s^{2} − (λ_{j}^{2} + q̃_{jj}f_{jj})
= (s − s_{j})(s + s_{j}),

where

s_{j} = ±\sqrt{λ_{j}^{2} + q̃_{jj}f_{jj}}.

From the above expression for s_{j}, we calculate q̃_{jj} that results in the j-th pole being shifted to −\sqrt{λ_{j}^{2} + q̃_{jj}f_{jj}}. We obtain

q̃_{jj} = \frac{s_{j}^{2} − λ_{j}^{2}}{f_{jj}}  (9.33)

Note that while selecting the desired location for s_{j}, we have to make sure that q̃_{jj} > 0 to ensure positive semidefiniteness of the weight matrix Q̃_{j}. Having Q̃_{j}, we compute Q_{j} = T^{−⊤}Q̃_{j}T^{−1} and solve the Riccati equation

A^{⊤}P_{j} + P_{j}A + Q_{j} − P_{j}BR^{−1}B^{⊤}P_{j} = O  (9.34)

for P_{j}. We compute u = −R^{−1}B^{⊤}P_{j}x + u_{j} = −K_{j}x + u_{j} that shifts only one pole of the closed-loop system

ẋ = (A − BK_{j})x + Bu_{j}
= A_{j}x + Bu_{j}

to its specified position. We then start with the new system

ẋ = A_{j}x + Bu_{j}

and shift the next pole to a specified location, and then combine the obtained feedback gains. The above method is based on the result, established by Solheim [307], that different feedback matrices K_{j} and the corresponding weight matrices Q_{j} can be combined to give the final optimal gain K and the corresponding weight matrix Q. Indeed, suppose we apply the above method to the system ẋ = (A − BK_{j})x + Bu_{j}. After finding an appropriate weight matrix Q_{j+1}, we solve the Riccati equation

(A − BK_{j})^{⊤}P_{j+1} + P_{j+1}(A − BK_{j}) + Q_{j+1} − P_{j+1}BR^{−1}B^{⊤}P_{j+1} = O  (9.35)

for P_{j+1} to compute the gain K_{j+1}. Adding the two Riccati equations (9.34) and (9.35), and performing simple manipulations yields

A^{⊤}(P_{j} + P_{j+1}) + (P_{j} + P_{j+1})A + Q_{j} + Q_{j+1} − (P_{j} + P_{j+1})BR^{−1}B^{⊤}(P_{j} + P_{j+1}) = O.

Let P = P_{j} + P_{j+1} and Q = Q_{j} + Q_{j+1}, then

K = K_{j} + K_{j+1}
= R^{−1}B^{⊤}P_{j} + R^{−1}B^{⊤}P_{j+1}.

In a similar manner we can show that the method works for more than just two iterations.

Example 9.13 We consider a model of one-link robot manipulator presented in Subsection 2.4.5. Applying Kirchhoff’s voltage law to the armature circuit yields

L_{a} \frac{di_{a}}{dt} + R_{a}i_{a} + K_{b}N \frac{dθ_{p}}{dt} = u,

where K_{b} is the back emf constant. We assume now that L_{a} ≈ 0. Hence,

u = R_{a}i_{a} + K_{b}N \frac{dθ_{p}}{dt} ,  (9.36)

We next compute i_{a} from (9.36) and substitute the result into (2.81) to get

ml^{2} \frac{d^{2}θ_{p}}{dt^{2}} = mgl sin θ_{p} + NK_{m} (\frac{u}{R_{a}} − \frac{K_{b}N \frac{dθ_{p}}{dt}}{R_{a}} ) .  (9.37)

We construct a reduced-order state-space model of the one-link robot manipulator. For this we choose the following state and output variables:

x_{1} = θ_{p}, x_{2} = \frac{dθ_{p}}{dt} = ω_{p}, and y = x_{1}.

Then, using (9.37), we obtain the following simple state-space model of the robot manipulator:

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}x_{2} \\ \frac{g}{l} sin x_{1} − \frac{K_{b}K_{m}N^{2}}{ml^{2}R_{a}} x_{2} + \frac{NK_{m}}{ml^{2}R_{a}} u \end{matrix} ]
y = x_{1}.

Reasonable parameters for the robot are: l = 1 m, m = 1 kg, N = 10, K_{m} = 0.1 Nm/A, K_{b} = 0.1 Vs/rad, R_{a} = 1 Ω. With the above parameters the robot model takes the form:

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}x_{2} \\ 9.8 sin x_{1} − x_{2} + u \end{matrix} ]
y = x_{1}.

Time histories of state trajectories of the uncontrolled nonlinear system, u = 0, are shown in Figure 9.11 for the initial conditions are x_{1}(0) = 1 and x_{2}(0) = 0. A phase portrait

Figure 9.11: Plots of y = x_{1} and x_{2} versus time of the uncontrolled nonlinear system of Example 9.13.

of the uncontrolled nonlinear system is shown in Figure 9.12. The linearized model about

Figure 9.12: A phase portrait of the uncontrolled nonlinear system of Example 9.13.

x = 0, u = 0 has the form

\begin{matrix}\frac{d}{dt}Δx & = & [ \begin{matrix}0 & 1 \\ 9.8 & −1 \end{matrix} ] Δx + [ \begin{matrix}0 \\ 1 \end{matrix} ] Δu \\ Δy & = & [ 1 0 ] Δx \end{matrix} \}  (9.38)

Plots of y = x_{1} and x_{2} versus time are shown in Figure 9.13. A phase portrait of the linearized system is shown in Figure 9.14. Let

J = ∫_{0}^{∞} (y^{2} + u^{2}) dt.

We find the linear state-feedback control law u = −kx that minimizes J subject to the equations given by (9.38). We have

Q = c^{⊤}c = [ \begin{matrix}1 & 0 \\ 0 & 0 \end{matrix} ] and R = [1].

Solving the associated Riccati equation yields

P = [ \begin{matrix}72.3371 & 19.6509 \\ 19.6509 & 5.3484 \end{matrix} ] .  (9.39)

9.3. LINEAR QUADRATIC REGULATOR 469

Figure 9.13: Plots of x_{1} and x_{2} versus time of the linearized system of Example 9.13. Figure 9.14: A phase portrait of the linearized system of Example 9.13.

Hence,

k = [ 19.6509 5.3484 ] ,  (9.40)

and

A_{c} = [ \begin{matrix}0 & 1 \\ −9.8509 & −6.3484 \end{matrix} ] .

Plots of x_{1} and x_{2} versus time of the closed-loop linear system

ẋ = (A − bk) x = A_{c}x
y = x_{1},

when the initial conditions are x_{1}(0) = 1 and x_{2}(0) = 0 are shown in Figure 9.15. A phase

Figure 9.15: Plots of x_{1} and x_{2} versus time of the linear closed-loop system of Example 9.13.

portrait of the closed-loop linear system is shown in Figure 9.16. We now apply the above optimal controller to the nonlinear model. Plots of x_{1} and x_{2} versus time of the closed-loop nonlinear systems are shown in Figure 9.17. A phase portrait of the nonlinear closed-loop system is shown in Figure 9.18. The closed-loop poles of the linearized system, that is, the eigenvalues of A_{c} are λ_{1} = −2.7003 and λ_{2} = −3.6481.

9.3. LINEAR QUADRATIC REGULATOR 471

Figure 9.16: A phase portrait of the linear closed-loop system of Example 9.13. Figure 9.17: Plots of x_{1} and x_{2} of the nonlinear closed-loop system of Example 9.13.

Figure 9.18: A phase portrait of the nonlinear closed-loop system of Example 9.13.

Example 9.14 We use the data from the previous example to find the weight matrix Q_{2} and the corresponding k_{2} so that λ_{2} is shifted to s_{2} = −5, that is, the new poles of the closed-loop system

ẋ = (A_{c} − bk_{2}) x

are s_{1} = λ_{1} and s_{2} = −5. For the control law u = −kx + r, the linear closed-loop system is

ẋ = [ \begin{matrix}0 & 1 \\ −9.8509 & −6.3484 \end{matrix} ] x + [ \begin{matrix}0 \\ 1 \end{matrix} ] u
= A_{c}x + br,

where for simplicity we used x rather than Δx, and u instead of Δu. The eigenvalues of A_{c} are λ_{1} = −2.7003 and λ_{2} = −3.6481. We have λ_{1} ≠ λ_{2} and thus the matrix A_{c} is diagonalizable. Note that different eigenvalues are only a sufficient condition for diagonalizability. We construct the similarity transformation using the eigenvectors of A_{c} The eigenvectors of A_{c} are

v_{1} = [ \begin{matrix}0.3473 \\ −0.9378 \end{matrix} ] and v_{2} = [ \begin{matrix}−0.2644 \\ −0.9644 \end{matrix} ] .

9.3. LINEAR QUADRATIC REGULATOR 473 Let x = T z, where T = [ v_{1} v_{2} ]. Then, A_{c} in the new coordinates has the form

Ã_{c} = T^{−1}A_{c}T = [ \begin{matrix}−2.7003 & 0 \\ 0 & −3.6481 \end{matrix} ] ,

and

b̃ = T^{−1}b = [ \begin{matrix}3.0383 \\ 3.9912 \end{matrix} ] .

Next, we compute

T^{−1}bR^{−1}b^{⊤}T^{−⊤} = [ \begin{matrix}9.2312 & 12.1264 \\ 12.1264 & 15.9296 \end{matrix} ] .

Hence,

q̃_{22} = \frac{s_{2}^{2} − λ_{2}^{2}}{15.9296} = 0.7339.

Therefore,

Q̃_{2} = [ \begin{matrix}0 & 0 \\ 0 & 0.7339 \end{matrix} ] = T^{⊤}Q_{2}T ,

and

Q_{2} = T^{−⊤}Q̃_{2}T^{−1} = [ \begin{matrix}85.2450 & 31.5686 \\ 31.5686 & 11.6907 \end{matrix} ] .

Solving the ARE with A_{c}, b, Q_{2}, and R yields

P_{2} = [ \begin{matrix}9.8572 & 3.6504 \\ 3.6504 & 1.3518 \end{matrix} ] ,

and

k_{2} = [ 3.6504 1.3518 ] .

As a check, if we take

Q_{new} = Q + Q_{2} = [ \begin{matrix}86.2450 & 31.5686 \\ 31.5686 & 11.6907 \end{matrix} ] ,

where Q = c^{⊤}c as in Example 9.13, and R_{new} = R, then solving the associated ARE using A, rather then A_{c}, we obtain

P_{new} = [ \begin{matrix}82.1943 & 23.3013 \\ 23.3013 & 6.7002 \end{matrix} ] = P + P_{2},

where P is given by (9.39), and

k_{new} = [ 23.3013 6.7002 ] = k + k_{2},

where k is the optimal gain given by (9.40).

Example 9.15 For the linearized model given by (9.38) and R = [2], we determine Q and the corresponding k that result in the closed-loop system ẋ = (A − bk) x with poles located at s_{1} = −4 and s_{2} = −5, where

A = [ \begin{matrix}0 & 1 \\ 9.8 & −1 \end{matrix} ] , b = [ \begin{matrix}0 \\ 1 \end{matrix} ] .

We shift poles into the desired locations one by one. We first diagonalize A using similarity transformation constructed from eigenvectors of A. The eigenvalues of A are λ_{1} = 2.6702 and λ_{2} = −3.6702. We denote the corresponding eigenvectors by v_{1} and v_{2}, respectively. We obtain

T^{−1}AT = [ \begin{matrix}2.6702 & 0 \\ 0 & −3.6702 \end{matrix} ] ,

where

T = [ v_{1} v_{2} ] = [ \begin{matrix}0.3507 & −0.2629 \\ 0.9365 & 0.9648 \end{matrix} ] .

We construct k_{1} so that λ_{1} is shifted to s_{1} = −4, while λ_{2} remains unchanged, that is, the eigenvalues of A − bk_{1} become s_{1} and λ_{2}. For this, we first compute

F = T^{−1}bR^{−1}b^{⊤}T^{−⊤} = [ \begin{matrix}0.1011 & 0.1349 \\ 0.1349 & 0.1800 \end{matrix} ] .

Hence, f_{11} = 0.1011,

q̃_{11} = \frac{4^{2} − (2.6702)^{2}}{0.1011} = 87.7352,

and

Q̃_{1} = [ \begin{matrix}87.7352 & 0 \\ 0 & 0 \end{matrix} ] = T^{⊤}Q_{1}T .

Hence,

Q_{1} = T^{−⊤}Q̃_{1}T^{−1} = [ \begin{matrix}239.0024 & 65.1202 \\ 65.1202 & 17.7431 \end{matrix} ] .

9.3. LINEAR QUADRATIC REGULATOR 475 Solving the associated ARE, using R = [2], yields

P_{1} = [ \begin{matrix}179.7014 & 48.9626 \\ 48.9626 & 13.3407 \end{matrix} ] .

Therefore,

k_{1} = [ 24.4813 6.6703 ] ,

and

A_{1} = A − bk_{1} = [ \begin{matrix}0 & 1 \\ −14.6813 & −7.6703 \end{matrix} ] .

We construct k_{2} that shifts λ_{2} to s_{2} leaving s_{1} unchanged. We first diagonalize A_{1} using its eigenvectors. Note that A_{1} and A share the common eigenvector v_{2}. The other eigenvector of A_{1} corresponding to its eigenvalue s_{1} is u_{1} = [ −0.2425 0.9701 ]^{⊤}. We have

T^{−1}A_{1}T = [ \begin{matrix}s_{1} & 0 \\ 0 & λ_{2} \end{matrix} ] ,

where

T = [ u_{1} v_{2} ] = [ \begin{matrix}−0.2425 & 0.2629 \\ 0.9701 & −0.9648 \end{matrix} ] .

We next compute

T^{−1}bR^{−1}b^{⊤}T^{−⊤} = [ \begin{matrix}78.0607 & 72.0157 \\ 72.0157 & 66.4389 \end{matrix} ] .

Thus, f̃_{22} = 66.4389,

q̃_{22} = \frac{5^{2} − (−3.6702)^{2}}{66.4389} = 0.1735,

and

Q̃_{2} = [ \begin{matrix}0 & 0 \\ 0 & 0.1735 \end{matrix} ] = T^{⊤}Q_{2}T .

From the above, we obtain

Q_{2} = T^{−⊤}Q̃_{2}T^{−1} = [ \begin{matrix}368.9003 & 92.2211 \\ 92.2211 & 23.0543 \end{matrix} ] .

Solving the associated ARE, we get

P_{2} = [ \begin{matrix}42.5495 & 10.6369 \\ 10.6369 & 2.6591 \end{matrix} ] ,

and

k_{2} = [ 5.3185 1.3296 ] .

The final weight matrix Q that results in shifting poles into the desired locations is

Q = Q_{1} + Q_{2} = [ \begin{matrix}607.9027 & 157.3413 \\ 157.3413 & 40.7974 \end{matrix} ] .

The corresponding P is

P = P_{1} + P_{2} = [ \begin{matrix}222.2509 & 59.5996 \\ 59.5996 & 15.9998 \end{matrix} ] .

The corresponding optimal gain vector k is

k = k_{1} + k_{2} = [ 29.7998 7.9999 ] .

9.3.4 Optimal Saturating Controllers

Asymptotic stability of an equilibrium state of a nonlinear system is a local property. This means that the system’s trajectories starting in a neighborhood of the equilibrium state will approach this equilibrium asymptotically. Often, we are interested in determining how far from the equilibrium state the system’s trajectories can start and still converge to the equilibrium asymptotically. The region around an equilibrium state within which all trajectories approach the equilibrium is called a region of attraction, or region of asymptotic stability abbreviated as RAS. Finding the exact RAS analytically can be a challenging task. A tool often used to estimate the exact RAS is a Lyapunov function. Specifically, if V = V (x) is a Lyapunov function of the equilibrium state of interest, that is, V is positive definite with respect to the equilibrium state in a domain S and the time derivative of V evaluated on the trajectories of the system is negative definite in S, then the set

Ω_{c} = {x : x ∈ S, V (x) ≤ c, V̇ (x) < 0}

is an estimate of the exact RAS. In other words, a region of attraction is a set of states x for which V̇ (x) < 0 and V (x) ≤ c for some positive constant c. The set Ω_{c} may be a conservative estimate, often a crude underestimate, of the true RAS. In order to obtain the least conservative estimate of the domain of attraction, corresponding to the given Lyapunov

Figure 9.19: Estimating the region of asymptotic stability.

function, we could try to maximize the size of Ω_{c} by finding the largest V contour that can just fit on the equilibrium side of the surface described by V̇ (x) = 0. This can be accomplished by solving the optimization problem for c such that

c = min_{V̇ (x)=0} V (x).  (9.41)

Solving the above optimization problem corresponds to maximizing the size of the region {x : V (x) ≤ c} in which V̇ (x) < 0 by making the boundary of {x : V (x) ≤ c} to be tangent to the boundary of {x : V̇ (x) < 0}. This is illustrated in Figure 9.19.

We next estimate the RAS for single-input linear dynamical systems with saturating linear optimal control. Our development follows that of Friedland [108, Section 5.1]. We will use such controllers in Chapter 18 to stabilize chaotic systems. We consider a single-input linear system model,

ẋ = Ax + b U_{max}sat (u/U_{max}) ,  (9.42)

where x ∈ ℝ^{n} and

sat(z) = \{ \begin{matrix}1 & if & z ≥ 1 \\ z & if & |z| < 1 \\ −1 & if & z ≤ −1. \end{matrix}

In the linear region, |u| ≤ U_{max}, (9.42) becomes ẋ = Ax+bu. We consider using the control law of the form

u = U_{max}sat (−kx/U_{max}) .  (9.43)

When |kx| ≤ U_{max}, the control law given by (9.43) becomes a linear state-feedback controller, u = −kx. For simplicity of notation, we can assume that U_{max} = 1. Indeed, we can always scale the input vector b so that U_{max} = 1. The gain vector k is obtained by solving the associated LQR problem, with the performance index,

J = ∫_{0}^{∞} (x^{⊤}Qx + ru^{2}) dt,

where we assume that the pair (A, b) is reachable, Q = Q^{⊤} > 0 and r > 0. Solving the algebraic Riccati equation,

A^{⊤}P + P A + Q − P bb^{⊤}P /r = O,  (9.44)

we use its solution, P = P^{⊤} > 0, to construct the gain vector,

k = b^{⊤}P /r.  (9.45)

We then proceed with evaluating of \frac{d}{dt}V = \frac{d}{dt}x^{⊤}P x on the trajectories of the closed-loop system,

V̇ = 2x^{⊤}P ẋ
= x^{⊤}(A^{⊤}P + P A)x − 2x^{⊤}P b sat(kx).  (9.46)

By (9.45),

b^{⊤}P = rk.  (9.47)

Using the above and (9.44) gives

A^{⊤}P + P A = rk^{⊤}k − Q.  (9.48)

Substituting (9.48) and (9.47) into (9.46) yields

V̇ = −x^{⊤}Qx − 2rkx sat(kx) + r(kx)^{2}  (9.49)

For x such that |kx| ≤ 1, the above becomes

V̇ = −x^{⊤}Qx − r(kx)^{2},

which is negative definite. Thus, the origin is (locally) asymptotically stable for the closedloop system.

Recall that an estimate of the RAS is a set of states bounded by the surface V = constant for which V̇ < 0. To find as large an estimate as possible, we attempt to find the surface V̇ = 0 and find c = min V (x) on that surface. Then, the region bounded by V (x) = c is an estimate of the RAS. Solving the above problem amounts to solving the optimization problem,

min_{x} x^{⊤}P x
subject to − x^{⊤}Qx − 2rkx sat(kx) + r(kx)^{2} = 0.  (9.50)

Solving the above problem is not a simple task. Instead, we solve an easier problem resulting in a more conservative estimate of the RAS. We first rewrite (9.50) as

y^{2} − 2y sat(y) − x^{⊤}Qx/r = 0,  (9.51)

where y = kx. We investigate the case when |y| ≥ 1 because for |y| < 1, we have V̇ < 0 and we are interested in the solution to the above when V̇ = 0. When y ≥ 1, (9.51) becomes

y^{2} − 2y − x^{⊤}Qx/r = 0.  (9.52)

because for y ≥ 1, sat(y) = 1. The solution to (9.52) for y ≥ 1 is

y = 1 + \sqrt{1 + x^{⊤}Qx/r} > 2,  (9.53)

while for y ≤ −1, the solution to (9.51) has the form

y = −1 − \sqrt{1 + x^{⊤}Qx/r} < −2.  (9.54)

The above solutions satisfy the inequality,

|y| ≥ 2.

Thus, we have V̇ < 0 for |y| = |kx| ≤ 2 and hence the RAS includes the region

{x : V (x) ≤ c̃},

where c̃ is obtained by solving the optimization problem

c̃ = max_{|kx|≤2} x^{⊤}P x

We use the Lagrange multipliers method to solve the above constrained optimization problem. We solve the equivalent problem,

max x^{⊤}P x
subject to (kx)^{2} = 4.

The Lagrangian for the above problem is

l(x, λ) = x^{⊤}P x + λ (4 − (kx)^{2}) .

Applying to the Lagrangian l the first-order necessary conditions for unconstrained minimization gives

D_{x}l(x, λ) = 2x^{⊤}P − 2λ(kx)k = 0,  (9.55)
D_{λ}l(x, λ) = 4 − (kx)^{2} = 0.  (9.56)

From (9.55),

x^{⊤}P x = λ(kx)^{2}.

Because (kx)^{2} = 4,

λ = \frac{1}{4}x^{⊤}P x,

and therefore,

c̃ = x^{⊤}P x = 4λ.

Again, using (9.55), we obtain

x^{⊤}P = λ(kx)k.

Postmultiplying both sides of the above by P^{−1}k^{⊤} gives

x^{⊤}k^{⊤} = kx = λ(kx)kP^{−1}k^{⊤}.  (9.57)

Note that kx ≠ 0 because (kx)^{2} = 4. Hence, from (9.57),

λ = \frac{1}{kP^{−1}k^{⊤}} .

Substituting (9.45) into the above gives

λ = \frac{1}{kP^{−1}k^{⊤}} = \frac{r^{2}}{b^{⊤}P b}.

Hence,

c̃ = 4λ = \frac{4r^{2}}{b^{⊤}P b}

9.3.5 Linear Quadratic Regulator for Discrete Systems on an Infinite Time Interval

Consider a discrete linear plant model,

x[k + 1] = Ax[k] + Bu[k], k = 0, 1, 2, …,  (9.58)

with a specified initial condition x(0) = x_{0}, where x[k] ∈ ℝ^{n} and u[k] ∈ ℝ^{m}. We assume that the pair (A, B) is reachable. Our objective is to construct a stabilizing linear state-feedback controller,

u[k] = −Kx[k]

that minimizes the quadratic performance index

J(u) = ∑_{k=0}^{∞} {x[k]^{⊤}Qx[k] + u[k]^{⊤}Ru[k]} ,  (9.59)

where Q = Q^{⊤} ≥ 0, and R = R^{⊤} > 0. We assume that the components of the control vector are unconstrained. An optimal control law will be denoted u^{∗}.

Because we seek the controller among the ones that stabilize the system (9.58), the closed-loop system,

x[k + 1] = (A − BK) x[k],  (9.60)

is asymptotically stable, that is, the eigenvalues of the matrix A − BK are in the open unit disk. This implies that there exists a Lyapunov function, V (x[k]) = x[k]^{⊤}P x[k], for the system (9.60). Therefore the first forward difference, ΔV (x[k]) = V (x[k + 1]) − V (x[k]) evaluated on the trajectories of (9.60) is negative definite. We now state and prove a sufficient condition for u[k] = −Kx[k] to be optimal.

Theorem 9.4 If the state feedback controller, u[k] = −Kx[k], is such that

{min \atop u} (ΔV (x[k]) + x[k]^{⊤}Qx[k] + u[k]^{⊤}Ru[k]) = 0,  (9.61)

then it is optimal.

Proof We can represent (9.61) as

ΔV (x[k])|_{u=u^{∗}} + x[k]^{⊤}Qx[k] + u^{∗}[k]^{⊤}Ru^{∗}[k] = 0.

Hence,

ΔV (x[k])|_{u=u^{∗}} = (V (x[k + 1]) − V (x[k])|_{u=u^{∗}} = −x[k]^{⊤}Qx[k] − u^{∗}[k]^{⊤}Ru^{∗}[k].

We sum both sides from k = 0 to k = ∞ to get

V (x[∞]) − V (x[0]) = − ∑_{k=0}^{∞} (x[k]^{⊤}Qx[k] + u^{∗}[k]^{⊤}Ru^{∗}[k]) .

Because by assumption the closed-loop system is asymptotically stable, we have x[∞] = 0. Hence,

V (x[0]) = x[0]^{⊤}P x[0] = ∑_{k=0}^{∞} (x[k]^{⊤}Qx[k] + u^{∗}[k]^{⊤}Ru^{∗}[k]) .

Thus, if a linear stabilizing controller satisfies the hypothesis of the theorem, then the value of the performance index (9.59) resulting from applying this controller is

J = x[0]^{⊤}P x[0].

To show that such a controller is an optimal one, we use a proof by contradiction. We assume that the hypothesis of the theorem holds but the controller u^{∗} that satisfies this hypothesis is not optimal, that is, there is a controller ũ such that

J \binom{}{ũ} < J (u^{∗}) .  (9.62)

The hypothesis of the theorem implies that

ΔV (x[k])|_{u=u^{}}˜ + x[k]^{⊤}Qx[k] + ũ[k]^{⊤}Rũ[k] ≥ 0.  (9.63)

We represent (9.63) as

ΔV (x[k])|_{u=u^{}}˜ ≥ −x[k]^{⊤}Qx[k] − ũ[k]^{⊤}Rũ[k].

Summing the above from k = 0 to k = ∞ yields

V (x[0]) ≤ ∑_{k=0}^{∞} (x[k]^{⊤}Qx[k] + ũ[k]^{⊤}Rũ[k]) ,

that is,

J \binom{}{ũ} ≥ J (u^{∗}) ,

which contradicts (9.62). The proof is complete.

□

Finding an optimal controller involves finding an appropriate quadratic Lyapunov function V (x) = x^{⊤}P x, which then is used to construct the optimal controller. We first find u^{∗} that minimizes the function

f = f(u[k]) = ΔV (x[k]) + x[k]^{⊤}Qx[k] + u[k]^{⊤}Ru[k].  (9.64)

To proceed, we perform preliminary manipulations on the function f. In particular, we use the definition of ΔV and substitute into (9.64) the system equation (9.58) to obtain

f(u[k]) = x[k + 1]^{⊤}P x[k + 1] − x[k]^{⊤}P x[k] + x[k]^{⊤}Qx[k] + u[k]^{⊤}Ru[k]
= (Ax[k] + Bu[k])^{⊤} P (Ax[k] + Bu[k]) − x[k]^{⊤}P x[k] + x[k]^{⊤}Qx[k]
+u[k]^{⊤}Ru[k].  (9.65)

Applying to the above function the first-order necessary condition for a relative minimum gives

\frac{∂f(u[k])}{∂u[k]} = 2 (Ax[k] + Bu[k])^{⊤} P B + 2u[k]^{⊤}R
= 2x[k]^{⊤}A^{⊤}P B + 2u[k]^{⊤} (R + B^{⊤}P B)
= 0.

The matrix R+B^{⊤}P B is positive definite because R is, and therefore the matrix R+B^{⊤}P B is invertible. Hence,

u^{∗}[k] = − (R + B^{⊤}P B)^{−1} B^{⊤}P Ax[k] = −Kx[k],  (9.66)

where

K = (R + B^{⊤}P B)^{−1} B^{⊤}P A.

Let

S = R + B^{⊤}P B.

Then, we represent (9.66) as

u^{∗}[k] = −S^{−1}B^{⊤}P Ax[k].  (9.67)

We next check if u^{∗} satisfies the second-order sufficient condition for a relative minimizer of the function (9.64). We have,

\frac{∂^{2}}{∂u^{2}(k)} (x[k + 1]^{⊤}P x[k + 1] − x[k]^{⊤}P x[k] + x[k]^{⊤}Qx[k] + u[k]^{⊤}Ru[k])

= \frac{∂}{∂u[k]} (2x[k]^{⊤}A^{⊤}P B + 2u[k]^{⊤} (R + B^{⊤}P B))

= 2 (R + B^{⊤}P B)

> 0,  (9.68)

that is, u^{∗} satisfies the second-order sufficient condition for a relative minimizer of the function (9.64).

The optimal controller (9.67) can be constructed if we have found an appropriate positive definite matrix P. Our next task is to devise a method that would allow us to compute the desired matrix P. For this, we first find the equation describing the closed-loop system driven by the optimal controller given by (9.67),

x[k + 1] = (A − BS^{−1}B^{⊤}P A) x[k].  (9.69)

Our controller satisfies the hypothesis of Theorem 9.4, that is,

x[k + 1]^{⊤}P x[k + 1] − x[k]^{⊤}P x[k] + x[k]^{⊤}Qx[k] + u^{∗}[k]^{⊤}Ru^{∗}[k] = 0.

Substituting into the above (9.69), the expression for u^{∗} given by (9.67), and performing simple manipulations gives

x[k]^{⊤} (A − BS^{−1}B^{⊤}P A)^{⊤} P (A − BS^{−1}B^{⊤}P A) x[k] − x[k]^{⊤}P x[k]
+x[k]^{⊤}Qx[k] + x[k]^{⊤}A^{⊤}P BS^{−1}RS^{−1}B^{⊤}P Ax[k]
= x[k]^{⊤}A^{⊤}P Ax[k] − x[k]^{⊤}A^{⊤}P BS^{−1}B^{⊤}P Ax[k]
−x[k]^{⊤}A^{⊤}P BS^{−1}B^{⊤}P Ax[k] + x[k]^{⊤}A^{⊤}P BS^{−1}B^{⊤}P BS^{−1}B^{⊤}P Ax[k]
−x[k]^{⊤}P x[k] + x[k]^{⊤}Qx[k] + x[k]^{⊤}A^{⊤}P BS^{−1}RS^{−1}B^{⊤}P Ax[k]
= x[k]^{⊤}A^{⊤}P Ax[k] − x[k]^{⊤}P x[k] + x[k]^{⊤}Qx[k]
−2x[k]^{⊤}A^{⊤}P BS^{−1}B^{⊤}P Ax[k]
+x[k]^{⊤}A^{⊤}P BS^{−1} (R + B^{⊤}P B) S^{−1}B^{⊤}P Ax[k]
= x[k]^{⊤}A^{⊤}P Ax[k] − x[k]^{⊤}P x[k] + x[k]^{⊤}Qx[k]
−2x[k]^{⊤}A^{⊤}P BS^{−1}B^{⊤}P Ax[k]
+x[k]^{⊤}A^{⊤}P BS^{−1}B^{⊤}P Ax[k]
= x[k]^{⊤} (A^{⊤}P A − P + Q − A^{⊤}P BS^{−1}B^{⊤}P A) x[k]
= 0.  (9.70)

The above equation should hold for any x. For this to be true we have to have

A^{⊤}P A − P + Q − A^{⊤}P BS^{−1}B^{⊤}P A = O  (9.71)

The above equation is called the discrete algebraic Riccati equation.

9.4 Dynamic Programming

The objective of this section is to introduce the reader to the method of dynamic programming invented by Richard E. Bellman (1920–1984) in 1957. Dynamic programming can be used to synthesize optimal controllers for nonlinear, time-varying dynamical systems. We first present dynamic programming for discrete-time systems and then discuss the continuous version of dynamic programming.

9.4.1 Discrete-Time Systems

Dynamic programming is based on the principle of optimality (PO), that was formulated by Bellman as follows:

An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision.

In other words, an optimal trajectory has the property that at an intermediate point, no matter how it was reached, the rest of the trajectory must coincide with an optimal trajectory as computed from this intermediate point as the initial point. We justify the PO by contradiction. Suppose that we are given an optimal trajectory starting from initial state x_{0} = x(t_{0}) at time t_{0} and terminating at the final state x(t_{f}) at time t_{f}. Let x(t) be an intermediate state on the optimal trajectory and let J^{∗}(t, x(t)) be the optimal cost of traveling from x(t) to the final state x(t_{f}). Thus, J^{∗}(t_{0}, x(t_{0})) is the optimal cost of traveling from x_{0} = x(t_{0}) to x(t_{f}). It then follows that the portion of the optimal trajectory from x(t) to x(t_{f}) is also optimal. For if some other path, indicated by dashed curve in Figure 9.20, connecting x(t) and x(t_{f}) resulted in a smaller cost than the solid path, then this would provide a less expensive route from x_{0} to x(t_{f}), thus contradicting the optimality of the original trajectory.

We note that the PO seems to be already known to Johann Bernoulli who stated in 1706 that “... all curves, which should give a maximum, preserve also in all their sections the laws of the same maximum”. Here, the reader should interpret maximum as optimum.

Bellman’s principle of optimality serves to “limit the number of potentially optimal control strategies that must be investigated.” [210, page 315]. The optimal control strategy is determined, using the PO, by working backward from the final stage.

We illustrate this “backwardness” nature of the PO on an example of a sequential decision making process. We define a decision as the choice of alternative paths leaving a given node.

Figure 9.20: Illustration of the Principle of Optimality.

Example 9.16 We use the PO to find the minimum cost path that starts at the node a and ends at any one of the nodes k, l, m in the routing network depicted in Figure 9.21. The travel costs are shown beside each path segment. The line segments can only be traversed from left to right. Applying the PO to this routing problem, we find the optimal path by performing a backward pass through the network. Let J_{q}^{∗} be the minimum cost from a generic node q to any one of the three possible final nodes. Then, we have

J_{g}^{∗} = 16, J_{h}^{∗} = 8, J_{i}^{∗} = 7, J_{j}^{∗} = 6.

The next stage, yields

J_{d}^{∗} = min{3 + J_{g}^{∗}, 11 + J_{h}^{∗}} = min{19, 19} = 19.

Thus, in this case, we have two optimal paths from node d. We can take either dgk or dhk paths. We then compute optimal paths from nodes e and f. We have

J_{e}^{∗} = min{6 + J_{h}^{∗}, 10 + J_{i}^{∗}} = min{6 + 8, 10 + 7} = 14,

and

J_{f}^{∗} = min{4 + J_{i}^{∗}, 12 + J_{j}^{∗}} = min{4 + 7, 12 + 6} = 11.

Figure 9.21: A routing network.

Thus, the optimal path emanating from node e is ehk, and from node f is fil. The next stage yields

J_{b}^{∗} = min{9 + J_{d}^{∗}, 2 + J_{e}^{∗}} = min{9 + 19, 2 + 14} = 16,

and

J_{c}^{∗} = min{5 + J_{e}^{∗}, 14 + J_{f}^{∗}} = min{5 + 14, 14 + 11} = 19.

Therefore, the optimal path starting from b is behk, and starting from c the optimal path is cehk. Finally, the initial, and hence final, stage yields

J_{a}^{∗} = min{4 + J_{b}^{∗}, 7 + J_{c}^{∗}} = min{4 + 16, 7 + 19} = 20.

Hence, the optimal, minimum total cost, route from a to any one of the three possible termination nodes k, l, or m is abehk. The total cost of this path is 20.

We next consider a class of discrete-time dynamical systems modeled by the difference equation

x[k + 1] = f (k, x[k], u[k]) ,  (9.72)

with a given initial condition x[k_{0}] = x_{0}, and the associated performance index

J = J_{0} = Φ(N, x[N]) + ∑_{k=0}^{N−1} F (k, x[k], u[k]).

Let J_{k}^{∗}(x[k]) denote the minimum cost of transferring the system from x[k] to the terminal point x[N]. Because x[N] is the terminal point, therefore J_{N}^{∗}(x[N]) = Φ(N, x[N]). Using the PO, we obtain the cost of transfer from x[N − 1] to the terminal point,

J_{N−1}^{∗}(x[N − 1]) = min_{u[N−1]} {F (N − 1, x[N − 1], u[N − 1]) + J_{N}^{∗}(x[N])}
= min_{u[N−1]} {F (N − 1, x[N − 1], u[N − 1]) + Φ(N, x[N])} .

When applying the PO to solving optimal control problems for discrete-time dynamical systems, we usually perform two stage-by-stage passes through the time stages. We begin with a backward pass. For this, we use (9.72) to eliminate x[N]. Next, we carry out the minimization to find u^{∗}[N − 1]. We then repeat the process. A typical step of the backward pass is described as,

J_{k}^{∗}(x[k]) = min_{u[k]} {F (k, x[k], u[k]) + J_{k+1}^{∗}(x[k])}
= min_{u[k]} {F (k, x[k], u[k]) + J_{k+1}^{∗} (f(k, x[k], u[k]))} .

The backward pass is completed when the initial time k_{0} is reached. Now, because x_{0} = x[k_{0}] is known, we find u_{0} = u[k_{0}] in terms of this state. We can then proceed with the forward pass. We use x_{0}, u_{0}, and (9.72) to compute x[k_{0} + 1]. The state x[k_{0} + 1] is then used to compute u[k_{0} + 1] from x[k_{0} + 2] = f(k_{0} + 1, x[k_{0} + 1], u[k_{0} + 1]), etc.

Example 9.17 Consider a scalar discrete-time dynamical system modeled by

x[k + 1] = 2x[k] − 3u[k], x[0] = 4,

and the performance index

J = J_{0} = (x[2] − 10)^{2} + \frac{1}{2} ∑_{k=0}^{1} (x[k]^{2} + u[k]^{2}) .

Note that the final state, in this example, is free. We use the PO to find optimal u[0] and u[1]. There are no constraints on u[k]. We begin with the backward pass. We have

J^{∗}(x[2]) = (x[2] − 10)^{2}.

Then,

J^{∗}(x[1]) = min_{u[1]} \{\frac{1}{2} (x[1]^{2} + u[1]^{2}) + J^{∗}(x[2])\}
= min_{u[1]} \{\frac{1}{2} (x[1]^{2} + u[1]^{2}) + (2x[1] − 3u[1] − 10)^{2}\} .

Because there are no constraints on u[1], we find optimal u[1] as a function of x[1] by applying the first-order necessary condition for unconstrained optimization to get

\frac{∂}{∂u[1]} \{\frac{1}{2} (x[1]^{2} + u[1]^{2}) + (2x[1] − 3u[1] − 10)^{2}\} = 0.

Hence,

u[1] = \frac{1}{19} (12x[1] − 60) ,

and therefore

J^{∗}(x[1]) = \frac{1}{2} (x[1]^{2} + (\frac{12x[1] − 60}{19})^{2}) + (2x[1] − 3\frac{12x[1] − 60}{19} − 10)^{2} .

Next, we compute u[0]. For this observe that

J^{∗}(x[0]) = min_{u[0]} \{\frac{1}{2} (x[0]^{2} + u[0]^{2}) + J^{∗}(x[1])\} .

Taking into account that

x[1] = 2x[0] − 3u[0], x[0] = 4,

and the fact that there are no constraints on u[0], we again use the first-order necessary condition for unconstrained optimization to find optimal u[0]. We compute

\frac{∂}{∂u[0]} \{\frac{1}{2} (x[0]^{2} + u[0]^{2}) + J^{∗}(x[1])\} = 0

to get

\frac{∂}{∂u[0]} \{8 + \frac{1}{2}u[0]^{2} + \frac{1}{2}(8 − 3u[0])^{2} + \frac{1}{2} (\frac{12(8 − 3u[0]) − 60}{19})^{2}\}

+\frac{∂}{∂u[0]} \{2(8 − 3u[0]) − 3\frac{12(8 − 3u[0]) − 60}{19} − 10\}^{2} = 0.

Performing some manipulations yields 13.7895u[0] = 27.7895. Hence, u[0] = 2.0153. This completes the backward pass. We are now ready for the forward pass. We use the system difference equation, x[0], and u[0] to compute x[1] = 1.9541. Using the above, we find

u[1] = \frac{12x[1] − 60}{19} = −1.9237.

Therefore,

x[2] = 2x[1] − 3u[1] = 9.6794,

which completes the forward pass.

9.4.2 Discrete Linear Quadratic Regulator Problem

In Example 9.17 above, we solved a simple discrete linear quadratic regulator problem where we found the open-loop optimal control strategy. We apply now the PO to find the optimal control of a linear state-feedback form for the discrete linear quadratic regulator problem. This problem is stated as follows. Given a discrete linear plant model,

x[k + 1] = Ax[k] + Bu[k], 0 ≤ k ≤ N − 1,

with a specified initial condition x[0] = x_{0}. We wish to calculate the optimal control sequence,

{u^{∗}[0], u^{∗}[1], …, u^{∗}[N − 1]} ,

that minimizes the quadratic performance index

J = J_{0} = \frac{1}{2}x[N]^{⊤}H_{N}x[N] + \frac{1}{2} ∑_{k=0}^{N−1} \{x[k]^{⊤}Qx[k] + u[k]^{⊤}Ru[k]\} ,

where H_{N} = H_{N}^{⊤} ≥ 0, Q = Q^{⊤} ≥ 0, and R = R^{⊤} > 0. We assume that the components of the control vector are unconstrained. To solve this problem we use the PO. To begin, we note that

J^{∗}(x[N]) = \frac{1}{2}x[N]^{⊤}H_{N}x[N].

The above is the penalty for being in a state x[N] at a time N.

We now decrement k to N − 1 to get

J_{N−1}(x[N − 1]) = \frac{1}{2}x[N]^{⊤}H_{N}x[N] + \frac{1}{2}x[N − 1]^{⊤}Qx[N − 1] + \frac{1}{2}u[N − 1]^{⊤}Ru[N − 1].

We use the state equation to eliminate x[N] from J_{N−1} to obtain

J_{N−1} = \frac{1}{2} (Ax[N − 1] + Bu[N − 1])^{⊤} H_{N} (Ax[N − 1] + Bu[N − 1])

+\frac{1}{2}x[N − 1]^{⊤}Qx[N − 1] + \frac{1}{2}u[N − 1]^{⊤}Ru[N − 1].

Because there are no constraints on control, we can apply the first-order necessary condition from static optimization to find u^{∗}[N − 1] as a function of x[N − 1],

\frac{∂J_{N−1}}{∂u[N − 1]} = (Ax[N − 1] + Bu[N − 1])^{⊤} H_{N}B + u[N − 1]^{⊤}R
= 0.

Solving for the control satisfying the first-order necessary condition, we obtain

u^{∗}[N − 1] = − (R + B^{⊤}H_{N}B)^{−1} B^{⊤}H_{N}Ax[N − 1].

Let

K_{N−1} = (R + B^{⊤}H_{N}B)^{−1} B^{⊤}H_{N}A,

then

u^{∗}[N − 1] = −K_{N−1}x[N − 1].

Computing the optimal cost of transferring the system from N − 1 to N, yields

J_{N−1}^{∗} = \frac{1}{2}x[N − 1]^{⊤}(A − BK_{N−1})^{⊤}H_{N}(A − BK_{N−1})x[N − 1]

+\frac{1}{2}x[N − 1]^{⊤} (K_{N−1}^{⊤}RK_{N−1}+Q) x[N − 1].

Let

H_{N−1} = (A − BK_{N−1})^{⊤}H_{N}(A − BK_{N−1}) + K_{N−1}^{⊤}RK_{N−1} + Q,

then

J_{N−1}^{∗} = \frac{1}{2}x[N − 1]^{⊤}H_{N−1}x[N − 1].

Decrementing k to N − 2 yields

J_{N−2} = \frac{1}{2}x[N − 1]^{⊤}H_{N−1}x[N − 1] + \frac{1}{2}x[N − 2]^{⊤}Qx[N − 2] + \frac{1}{2}u[N − 2]^{⊤}Ru[N − 2].

Note that J_{N−2} has the same form as J_{N−1}. Thus, we obtain optimal feedback gain where N is being replaced with N − 1. Continuing in this fashion, we get the following results for each k = N − 1, N − 2, …, 0:

K_{k} = (R + B^{⊤}H_{k+1}B)^{−1} B^{⊤}H_{k+1}A,
u^{∗}[k] = −K_{k}x[k],
H_{k} = (A − BK_{k})^{⊤}H_{k+1}(A − BK_{k}) + K_{k}^{⊤}RK_{k} + Q,

and

J_{k}^{∗} = \frac{1}{2}x[k]^{⊤}H_{k}x[k].

The above control scheme can be implemented by computing the sequence of gain matrices {K_{k}} off-line and stored. Then, we can implement the controller u^{∗}[k] = −K_{k}x[k]. Note that the time-varying nature of the feedback controller may make an implementation of the control strategy quite complicated, especially for large n and N. However, as we go to the limit as N → ∞, we can simplify the solution to the problem. Specifically, let us consider a (reachable) system model

x[k + 1] = Ax[k] + Bu[k], k ≥ 0, x(0) = x_{0},

and the performance index

J = \frac{1}{2} ∑_{k=0}^{∞} \{x[k]^{⊤}Qx[k] + u[k]^{⊤}Ru[k]\}

obtained from the previously considered performance index by letting N → ∞ and setting H_{N} = 0. Then, the optimal controller takes the form

u^{∗}[k] = −K_{∞}x[k], k ≥ 0,

where

K_{∞} = (R + B^{⊤}H_{∞}B)^{−1} B^{⊤}H_{∞}A  (9.73)

and H_{∞} is defined as the limit obtained by continuing the recursion

H_{k} = (A − BK_{k})^{⊤}H_{k+1}(A − BK_{k}) + K_{k}^{⊤}RK_{k} + Q  (9.74)

back to k = −∞, that is,

lim_{k→−∞} H_{k} = H_{∞}.

Alternatively, letting k → −∞ in (9.74), we obtain

H_{∞} = (A − BK_{∞})^{⊤}H_{∞}(A − BK_{∞}) + K_{∞}^{⊤}RK_{∞} + Q,  (9.75)

where K_{∞} is given by (9.73). Substituting (9.73) into (9.75) yields

H_{∞}
= (A − B (R + B^{⊤}H_{∞}B)^{−1} B^{⊤}H_{∞}A)^{⊤} H_{∞} (A − B (R + B^{⊤}H_{∞}B)^{−1} B^{⊤}H_{∞}A)
+ ((R + B^{⊤}H_{∞}B)^{−1} B^{⊤}H_{∞}A)^{⊤} R ((R + B^{⊤}H_{∞}B)^{−1} B^{⊤}H_{∞}A) + Q. (9.76)

Thus, we can compute H_{∞} by solving the above algebraic equation. We note that the above formidable looking equation is just the discrete algebraic Riccati equation. Indeed, setting P = H_{∞}, using the notation,

S = R + B^{⊤}H_{∞}B = R + B^{⊤}P B,

and performing simple manipulations, we obtain (9.71).

We next discuss a continuous form of dynamic programming. We begin our discussion by analyzing the minimum time regulator problem. Then, we focus our attention on a more general optimal control problem.

9.4.3 Continuous Minimum Time Regulator Problem

In this subsection, we consider a control system modeled by

ẋ = f(t, x, u)

with a given initial state x(t_{0}) = x_{0}. We restrict the controller u(t) to be restrained by the compact set U ⊂ ℝ^{m}. For example, we may restrict the components u_{i}, i = 1, 2, …, m of the control vector u to satisfy the amplitude constraints

|u_{i}| ≤ k_{i},

where k_{i} are fixed real numbers. A piecewise continuous control vector u satisfying the above constraints is called admissible. The set of admissible controllers is denoted U. Our goal is to determine an admissible control law u^{∗} that steers the system’s trajectory from a given initial state x(t_{0}) = x_{0} to a specified final state x_{f} in the shortest possible time. This is equivalent to minimizing the performance index

J = ∫_{t_{0}}^{t_{f}} dt = t_{f} − t_{0}.

We call such a controller time optimal. We assume that for any x ≠ x_{f} there exists admissible, time optimal control transferring the system from x to x_{f} in minimum time. We denote by J^{∗}(t, x) = J^{∗}(t, x(t)) the time of the optimal transfer from x(t) to the specified x_{f}. Observe that J^{∗}(t, x) = J^{∗}(t, x_{1}, x_{2} …, x_{n}), that is,

J^{∗} : ℝ^{n+1} → ℝ,

which means that J^{∗} is a real valued function of n+1 variables. We assume that the function J^{∗} is continuous and has continuous partial derivatives everywhere except at the point x_{f}.

Figure 9.22: Illustration of the derivation of the condition that a controller must satisfy when transferring a dynamical system from an initial state to the final state in an optimal manner.

Now, let x_{0} = x(t_{0}) ≠ x_{f} be an arbitrary initial state in ℝ^{n}. Assume that we applied some admissible u_{0} ∈ U on the time interval [t_{0}, t]. As a result of this control action, the system is transferred from x(t_{0}) to x(t). The time spent on the motion from x(t_{0}) to x(t) is t − t_{0}. We then apply an admissible optimal control that transfers the system in minimum time J^{∗}(t, x(t)) from x(t) to x_{f}. The above considerations are illustrated in Figure 9.22. Denoting the optimal time of transferring the system from x(t_{0}) to x_{f} by J^{∗}(t_{0}, x(t_{0})), we obtain

(t − t_{0}) + J^{∗}(t, x(t)) ≥ J^{∗}(t_{0}, x(t_{0})).

Rearranging the terms of the above equation and dividing by (t − t_{0}) yields

1 + \frac{J^{∗}(t, x(t)) − J^{∗}(t_{0}, x(t_{0}))}{t − t_{0}} ≥ 0.

Letting t → t_{0}, and taking into account that ẋ_{i} = f_{i}(t, x, u), i = 1, 2, …, n, we obtain

1 + \frac{∂J^{∗}(t, x)}{∂t} + ∑_{i=1}^{n} \frac{∂J^{∗}(t, x)}{∂x_{i}} ẋ_{i} = 1 + \frac{∂J^{∗}(t, x)}{∂t} + ∑_{i=1}^{n} \frac{∂J^{∗}(t, x)}{∂x_{i}}f_{i}
= 1 + \frac{∂J^{∗}}{∂t} + \frac{∂J^{∗}}{∂x} f
≥ 0.

We conclude that for any x_{0} ≠ x_{f} and for any u ∈ U,

1 + \frac{∂J^{∗}}{∂t} + \frac{∂J^{∗}}{∂x} f ≥ 0.

We next define the Hamiltonian function for the time optimal control problem to be

H(t, x, u, p) = 1 + p^{⊤}f(t, x, u),

where

p = [ \frac{∂J^{∗}}{∂x_{1}} \frac{∂J^{∗}}{∂x_{2}} … \frac{∂J^{∗}}{∂x_{n}} ]^{⊤} .

Thus, we can say that for any x_{0} ≠ x_{f} and for any u ∈ U,

\frac{∂J^{∗}(t, x)}{∂t} + H(t, x, u) ≥ 0.

We will use the PO to show that

\frac{∂J^{∗}(t, x)}{∂t} + H(t, x, u) = 0

for any optimal transfer, that is,

\frac{∂J^{∗}(t, x)}{∂t} + min_{u∈U} H(t, x, u) = 0 for any x ≠ x_{f}.

Indeed, suppose that u(t) ∈ U is an optimal controller that transfers the system from the initial state x(t_{0}) ≠ x_{f} to the given final state x_{f} in a time t_{f}. Thus, we have J^{∗}(t_{0}, x(t_{0})) = t_{f} − t_{0}. The transfer from x(t) to x_{f} requires (t_{f} − t) time units. See Figure 9.23 for an illustration of the arguments. Hence,

t_{f} − t = J^{∗}(t_{0}, x(t_{0})) − (t − t_{0}),

and it is not possible to go from x(t) to x_{f} in a time less than J^{∗}(t_{0}, x(t_{0})) − (t − t_{0}). For if such a more rapid motion were to exist, indicated by the dashed curve in Figure 9.23, then moving from x_{0} to x(t) during the time t − t_{0}, and then from x(t) to x_{f} during a time less than J^{∗}(t_{0}, x(t_{0})) − (t − t_{0}), we would accomplish the transfer from x_{0} to x_{f} during a time less than J^{∗}(t_{0}, x(t_{0})), which is impossible. Therefore, we can write

t_{f} − t = J^{∗}(t, x(t))
= J^{∗}(t_{0}, x(t_{0})) − (t − t_{0}).

Rearranging terms, dividing by t − t_{0}, and letting t → t_{0}, yields

\frac{∂J^{∗}(t, x)}{∂t} + min_{u∈U} H(t, x, u) = 0 for any x ≠ x_{f}.

We summarize our considerations in the following theorem.

Figure 9.23: Illustration of the derivation of the Hamilton-Jacobi-Bellman equation.

Theorem 9.5 Let ẋ = f(t, x, u) be a control system model, U a set of admissible controllers, and x_{f} a prescribed final state. Suppose that

(i) for any x ≠ x_{f} there exists an admissible optimal control transferring x to x_{f},

(ii) the function J^{∗}(t, x(t)), which is the time of optimal transfer of the system from x, at time t, to x_{f}, is continuously differentiable except at the point x_{f},

then

\frac{∂J^{∗}(t, x)}{∂t} + H(t, x, u) ≥ 0 for any u ∈ U and x ≠ x_{f},
\frac{∂J^{∗}(t, x)}{∂t} + min_{u∈U} H(t, x, u) = 0 for any optimal transfer, where x ≠ x_{f}.

The above theorem is a special case of a more general result known as the continuous form of the dynamic programming. We now use the PO to derive a continuous form of dynamic programming for a class of problems with more general performance indices.

9.4.4 The Hamilton-Jacobi-Bellman (HJB) Equation

We consider a control process model of the form

ẋ = f(t, x, u), x(t_{0}) = x_{0},  (9.77)

9.4. DYNAMIC PROGRAMMING 497 and we consider the performance index that we wish to minimize

J(t_{0}, x(t_{0}), u) = Φ(t_{f}, x(t_{f})) + ∫_{t_{0}}^{t_{f}} F (τ, x(τ), u(τ))dτ.  (9.78)

Define

J(t, x(t), u(τ)) = Φ(t_{f}, x(t_{f})) + ∫_{t}^{t_{f}} F (τ, x(τ), u(τ))dτ

with t ≤ τ ≤ t_{f}, and let

J^{∗}(t, x(t)) = {min \atop u} J(t, x(t), u(τ)).

We subdivide the interval [t, t_{f}] as

[t, t_{f}] = [t, t + Δt] ∪ [t + Δt, t_{f}].

Then, we write

J^{∗}(t, x(t), u(τ)) = {min \atop u} \{∫_{t}^{t+Δt} F dτ + ∫_{t+Δt}^{t_{f}} F dτ + Φ(t_{f}, x(t_{f}))\} .

By the PO the trajectory on the interval [t + Δt, t_{f}] must be optimal. Hence,

J^{∗}(t, x(t)) = {min \atop u} \{∫_{t}^{t+Δt} F dτ + J^{∗}(t + Δt, x(t + Δt))\} .

We next expand J^{∗}(t + Δt, x(t + Δt)) in a Taylor series about the point (t, x(t)) to get

J^{∗}(t, x(t)) = {min \atop u} \{∫_{t}^{t+Δt} F dτ + J^{∗} + \frac{∂J^{∗}}{∂t} Δt + \frac{∂J^{∗}}{∂x} (x(t + Δt) − x(t)) + H.O.T.\} ,

where H.O.T. stands for higher order terms. Because J^{∗}(t, x(t)) is independent of u, we cancel the terms J^{∗}(t, x(t)) out and use the fact that x(t + Δt) − x(t) ≈ ẋΔt to obtain

0 = {min \atop u} \{∫_{t}^{t+Δt} F dτ + \frac{∂J^{∗}}{∂t} Δt + \frac{∂J^{∗}}{∂x} ẋΔt + H.O.T.\} .

By assumption Δt is small, hence we can write

0 = {min \atop u} \{F Δt + \frac{∂J^{∗}}{∂t} Δt + \frac{∂J^{∗}}{∂x} fΔt + H.O.T.\} .

Dividing the above equation by Δt and letting Δt → 0 yields

0 = \frac{∂J^{∗}}{∂t} + {min \atop u} \{F + \frac{∂J^{∗}}{∂x} f\} .  (9.79)

Let H = F + \frac{∂J^{∗}}{∂x} f denote the Hamiltonian function. Then, we write the partial differential equation (9.79) as

0 = \frac{∂J^{∗}}{∂t} + {min \atop u} H  (9.80)

subject to the boundary condition

J^{∗}(t_{f}, x(t_{f})) = Φ(t_{f}, x(t_{f})).

The partial differential equation (9.80) for the optimal cost J^{∗}(t, x(t)) is called the Hamilton-Jacobi-Bellman (HJB) equation. It provides the solution to the optimal control problem for general nonlinear dynamical systems. However, analytical solution to the HJB equation is difficult to obtain in most cases. We now illustrate, with three examples, how the HJB equation can be used to construct optimal controllers.

Example 9.18 We consider a general class of dynamical systems modeled by the scalar differential equation

ẋ = ax + bu, x(t_{0}) = x_{0},

with the associated cost index

J(t_{0}) = \frac{1}{2}fx(t_{f})^{2} + \frac{1}{2} ∫_{t_{0}}^{t_{f}} (qx(t)^{2} + ru(t)^{2}) dt,

where the initial time t_{0} = 0, the final time t_{f} < ∞ is fixed, and the final state x(t_{f}) is free. Our goal is to construct the control u^{∗} that minimizes the cost index J.

We first form the Hamiltonian function for the problem,

H (t, x, u, \frac{∂J^{∗}}{∂x} ) = \frac{1}{2}qx^{2} + \frac{1}{2}ru^{2} + \frac{∂J^{∗}}{∂x} (ax + bu).

Because there are no constraints on the control u, we can determine the form of the optimal control by applying the first order necessary condition for static optimization. We compute

\frac{∂H}{∂u} = ru + b\frac{∂J^{∗}}{∂x} = 0.

The optimal control, therefore, has the form

u^{∗} = −\frac{1}{r}b\frac{∂J^{∗}}{∂x} .

This control indeed minimizes the Hamiltonian function H because

\frac{∂^{2}H}{∂u^{2}} = r > 0,

that is, the second order sufficient condition for static minimization is satisfied. We thus reduced the problem of constructing the optimal control to that of finding \frac{∂J^{∗}}{∂x}. Substituting u^{∗} into the HJB equation yields

\frac{∂J^{∗}}{∂t} + \frac{1}{2}qx^{2} + \frac{b^{2}}{2r} (\frac{∂J^{∗}}{∂x} )^{2} + ax\frac{∂J^{∗}}{∂x} − \frac{b^{2}}{r} (\frac{∂J^{∗}}{∂x} )^{2}

= \frac{∂J^{∗}}{∂t} + \frac{1}{2}qx^{2} − \frac{b^{2}}{2r} (\frac{∂J^{∗}}{∂x} )^{2} + ax\frac{∂J^{∗}}{∂x}

= 0.

Let us assume that J^{∗} is a quadratic function of the state for all t ≤ t_{f}, that is,

J^{∗} = \frac{1}{2}p(t)x^{2},

where p(t) is a scalar function of time to be determined. Partial derivatives of J^{∗} are

\frac{∂J^{∗}}{∂t} = \frac{1}{2}ṗ(t)x^{2} and \frac{∂J^{∗}}{∂x} = p(t)x.

Substituting the above into the HJB equation and performing some manipulations gives

(\frac{1}{2}ṗ(t) + \frac{1}{2}q + ap(t) − \frac{b^{2}p(t)^{2}}{2r} ) x^{2} = 0.

The above holds for all x = x(t) if and only if

\frac{1}{2}ṗ(t) + \frac{1}{2}q + ap(t) − \frac{b^{2}p(t)^{2}}{2r} = 0,

or equivalently

ṗ(t) + q + 2ap(t) − \frac{b^{2}p(t)^{2}}{r} = 0  (9.81)

subject to the boundary condition p(t_{f}) = f that we obtain by comparing J^{∗} = \frac{1}{2}p(t)x^{2} and J(t_{f}) = \frac{1}{2}fx(t_{f})^{2}. To solve the above equation we assume that

p(t) = ρ\frac{ẇ(t)}{w(t)},

where the parameter ρ is to be determined. Note that

ṗ(t) = ρ\frac{ẅ(t)w(t) − ẇ(t)^{2}}{w(t)^{2}} .

500 CHAPTER 9. OPTIMAL CONTROL Substituting the above into (9.81), we obtain

ṗ + q + 2ap − \frac{b^{2}p^{2}}{r}

= ρ\frac{ẅw − ẇ^{2}}{w^{2}} + q \frac{w^{2}}{w^{2}} + 2aρ\frac{ẇ}{w} − \frac{b^{2}ρ^{2}ẇ^{2}/r}{w^{2}}

= \frac{ρẅw − ρẇ^{2} + qw^{2} + 2aρẇw − b^{2}ρ^{2}ẇ^{2}/r}{w^{2}}

= 0.

We choose ρ so that the terms nonlinear in ẇ cancel out. For this it is enough that

−ρẇ^{2} − b^{2}ρ^{2}ẇ^{2}/r = 0.

Hence, if we set

ρ = −\frac{r}{b^{2}} ,

then the equation to be solved takes the form

\frac{(−\frac{r}{b^{2}} ẅ − \frac{2ar}{b^{2}} ẇ + qw) w}{w^{2}} = 0.

The above is satisfied if

−\frac{r}{b^{2}} ẅ − \frac{2ar}{b^{2}} ẇ + qw = 0,

or equivalently

ẅ + 2aẇ − \frac{qb^{2}}{r} w = 0,

which is just a second order linear differential equation. Its characteristic equation has two real roots:

s_{1,2} = −a ± \sqrt{a^{2} + \frac{qb^{2}}{r}} .

Hence

w(t) = C_{1}e^{s_{1}t} + C_{2}e^{s_{2}t},

and

p(t) = ρ\frac{ẇ(t)}{w(t)} = ρ\frac{s_{1}C_{1}e^{s_{1}t} + s_{2}C_{2}e^{s_{2}t}}{C_{1}e^{s_{1}t} + C_{2}e^{s_{2}t}} .

Using the boundary value p(t_{f}) = f, we express C_{1} in terms of C_{2} to obtain

C_{1} = \frac{ρs_{2}C_{2}e^{s_{2}t_{f}} − fC_{2}e^{s_{2}t_{f}}}{fe^{s_{1}t_{f}} − ρs_{1}e^{s_{1}t_{f}}} .

Figure 9.24: Schematic of an armature-controlled DC motor system of Example 9.19.

We substitute the above expression for C_{1} into p(t) and cancel out C_{2}. Then, we use the obtained result to construct the optimal state-feedback controller

u^{∗} = −\frac{1}{r}b\frac{∂J^{∗}}{∂x} = −\frac{1}{r}bp(t)x

Example 9.19 Consider the schematic of an armature controlled DC motor, given in Figure 9.24, where the system parameters are: R_{a} = 2 Ω, L_{a} ≈ 0 H, K_{b} = 2 V/rad/sec, K_{i} = 2 Nm/A, and the equivalent moment of inertia referred to the motor shaft is I_{eq} = 1 kg·m^{2}. The friction is assumed to be negligible.

(i) Construct the first-order differential equation modeling the DC motor;

(ii) Use the Hamilton-Jacobi-Bellman equation to find the optimal state feedback controller, e_{a} = k(t)ω, that minimizes the performance index

J = \frac{1}{2}ω(10)^{2} + ∫_{0}^{10} R_{a}i_{a}(t)^{2}dt,

where i_{a} is the armature current. The final state is free. There are no constraints on e_{a}. Assume J^{∗} = \frac{1}{2}p(t)ω^{2}.

Applying Kirchhoff’s voltage law to the armature circuit yields

R_{a}i_{a} + K_{b}ω = e_{a}.

The torque developed by the motor, T_{m}, is

I_{eq}ω̇ = T_{m} = K_{i}i_{a}.

Substituting i_{a} from the first equation into the second and dividing both sides by I_{eq} yields

ω̇ = −\frac{K_{i}K_{b}}{R_{a}I_{eq}} ω + \frac{K_{i}}{I_{eq}R_{a}} e_{a}.

Substituting into the above the parameter values, we obtain

ω̇ = −2ω + e_{a}.  (9.82)

We represent the performance index, J, in terms of ω and e_{a}. Applying Ohm’s law to the armature circuit, we get

e_{a} − K_{b}ω = R_{a}i_{a}.

Hence,

J = \frac{1}{2}ω(10)^{2} + ∫_{0}^{10} \frac{(e_{a} − K_{b}ω)^{2}}{R_{a}} dt.

The Hamiltonian function is

H = \frac{(e_{a} − K_{b}ω)^{2}}{R_{a}} + \frac{∂J^{∗}}{∂ω} (−\frac{K_{i}K_{b}}{R_{a}I_{eq}} ω + \frac{K_{i}}{I_{eq}R_{a}} e_{a}) = \frac{(e_{a} − 2ω)^{2}}{2} + \frac{∂J^{∗}}{∂ω} (−2ω + e_{a}) .

Because there are no constraints on e_{a}, we can find the optimal control by solving the equation

0 = \frac{∂H}{∂e_{a}} = \frac{2 (e_{a} − K_{b}ω)}{R_{a}} + \frac{K_{i}}{I_{eq}R_{a}} \frac{∂J^{∗}}{∂ω} .

Hence,

e_{a}^{∗} = 2ω − \frac{∂J^{∗}}{∂ω} .  (9.83)

Substituting (9.83) into the HJB equation yields

0 = \frac{∂J^{∗}}{∂t} + \frac{1}{2} (−\frac{∂J^{∗}}{∂ω} )^{2} + \frac{∂J^{∗}}{∂ω} (−\frac{∂J^{∗}}{∂ω} ) = \frac{∂J^{∗}}{∂t} − \frac{1}{2} (\frac{∂J^{∗}}{∂ω} )^{2} .

By assumption J^{∗} = \frac{1}{2}p(t)ω^{2}. Hence,

\frac{∂J^{∗}}{∂t} = \frac{1}{2}ṗω^{2} and \frac{∂J^{∗}}{∂ω} = pω.

Substituting the above into the HJB equation, we get

\frac{1}{2} (ṗ − p^{2}) ω^{2} = 0.

We next solve the nonlinear differential equation

ṗ − p^{2} = 0.  (9.84)

We assume that p = ρ\frac{ẇ}{w}. Therefore,

ṗ = ρ\frac{ẅw − ẇ^{2}}{w^{2}} .

We substitute the expressions for p and ṗ into (9.84) to get

\frac{ρẅw − ρẇ^{2} − ρ^{2}ẇ^{2}}{w^{2}} = 0.

Selecting ρ = −1 eliminates the nonlinear terms in the numerator and we are left with the equation

ẅ = 0,

whose solution is

w = w(t) = C_{1}t + C_{2},

where C_{1} and C_{2} are integration constants. Hence,

p = −\frac{ẇ}{w} = −\frac{C_{1}}{C_{1}t + C_{2}} .

Because

J^{∗}(10) = \frac{1}{2}ω(10)^{2} = \frac{1}{2}p(10)ω(10)^{2},

we conclude that p(10) = 1. We use this information to eliminate one of the integration constants. We obtain

p(10) = 1 = −\frac{C_{1}}{10C_{1} + C_{2}} .

Hence,

C_{2} = −11C_{1},

and

p = p(t) = −\frac{C_{1}}{C_{1}t + C_{2}} = −\frac{C_{1}}{C_{1}t − 11C_{1}} = −\frac{1}{t − 11}.

Therefore,

e_{a}^{∗} = 2ω − \frac{∂J^{∗}}{∂ω} = 2ω − pω = (2 + \frac{1}{t − 11}) ω

In the above examples, the final time t_{f} was fixed and t_{f} < ∞. In the following example, we consider the linear quadratic regulator design for t_{f} = ∞ using the HJB equation.

Example 9.20 For the dynamical system model,

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}0 & 1 \\ 0 & −1 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ 1 \end{matrix} ] u,

and the performance index

J = \frac{1}{2} ∫_{0}^{∞} (x_{1}^{2} + x_{2}^{2} + u^{2}) dt,

we will use the HJB equation to find the optimal state-feedback controller. We first form the Hamiltonian function

H = \frac{1}{2} (x_{1}^{2} + x_{2}^{2} + u^{2}) + \frac{∂J^{∗}}{∂x_{1}} x_{2} + \frac{∂J^{∗}}{∂x_{2}} (−x_{2} + u).

We assume that J^{∗} has the form

J^{∗} = \frac{1}{2} (αx_{1}^{2} + 2βx_{1}x_{2} + γx_{2}^{2}) .

Hence,

\frac{∂J^{∗}}{∂t} = 0,

and the HJB equation takes the form

min_{u} H = 0,

with the boundary condition

J^{∗}(x(t_{f})) = J^{∗}(x(∞)) = 0.

9.4. DYNAMIC PROGRAMMING 505 Because there are no constraints on u, we apply the first-order necessary condition for unconstrained optimization to find u minimizing H,

\frac{∂H}{∂u} = 0.

We denote by u^{∗} the solution to the above equation. Differentiating, we obtain

u^{∗} + \frac{∂J^{∗}}{∂x_{2}} = 0.

Hence,

u^{∗} = −\frac{∂J^{∗}}{∂x_{2}} .

We substitute u^{∗} into the HJB equation to get

\frac{1}{2} (x_{1}^{2} + x_{2}^{2} + (\frac{∂J^{∗}}{∂x_{2}} )^{2}) + \frac{∂J^{∗}}{∂x_{1}} x_{2} − \frac{∂J^{∗}}{∂x_{2}} x_{2} − (\frac{∂J^{∗}}{∂x_{2}} )^{2} = 0.

Note that

\frac{∂J^{∗}}{∂x_{1}} = αx_{1} + βx_{2} and \frac{∂J^{∗}}{∂x_{2}} = βx_{1} + γx_{2}.

Substituting the above into the HJB equation yields

\frac{1}{2} (x_{1}^{2} + x_{2}^{2} + (βx_{1} + γx_{2})^{2}) + (αx_{1} + βx_{2})x_{2} − (βx_{1} + γx_{2})x_{2} − (βx_{1} + γx_{2})^{2} = 0.

We rearrange the above equation to obtain

\frac{1}{2}x_{1}^{2}(1 − β^{2}) + x_{1}x_{2}(α − β − βγ) + \frac{1}{2}x_{2}^{2}(1 − 2γ − γ^{2} + 2β) = 0.

For this equation to hold, we have to have

\begin{matrix}1 − β^{2} & = & 0 \\ α − β − βγ & = & 0 \\ 1 − 2γ − γ^{2} + 2β & = & 0. \end{matrix} \}

There are three solutions to the above system of equations:

1. α = 2, β = γ = 1;

2. α = −2, β = 1, γ = −3;

3. α = 0, β = γ = −1.

Thus, we have three candidate solutions to the problem:

u^{(1)} = −x_{1} − x_{2},
u^{(2)} = −x_{1} + 3x_{2},
u^{(3)} = x_{1} + x_{2}.

However, only u^{(1)} stabilizes the plant. Hence, the optimal control law for the problem is

u^{∗} = −x_{1} − x_{2}.

Note, that only for this controller the boundary condition, J^{∗}(x(t_{f})) = J^{∗}(x(∞)) = 0, is satisfied.

9.5 Pontryagin’s Minimum Principle

9.5.1 Optimal Control With Constraints on Inputs

In this section, we provide a derivation of the minimum principle of Pontryagin, which is a generalization of the Euler-Lagrange equations that also includes problems with constraints on the control inputs. Only a special case of the minimum principle is stated. However, this special case covers a large class of control problems. We consider the problem of minimizing the performance index given by

J = Φ(x(t_{f})) + ∫_{t_{0}}^{t_{f}} F (x(t), u(t))dt  (9.85)

subject to

ẋ = f(x, u), x(t_{0}) = x_{0}, and x(t_{f}) = x_{f}.  (9.86)

We consider two cases: fixed final state and free final state.

To proceed we define the Hamiltonian function H(x, u, p) as

H(x, u, p) = H = F + p^{⊤}f,

where the costate vector p will be determined in the analysis to follow.

We discuss the case when the final time t_{f} is fixed. We follow the development of Luenberger [223, Section 11.1]. We “adjoin” to J additional terms that sum up to zero. We note that because the state trajectories must satisfy the equation, ẋ = f(x, u), we have

f(x, u) − ẋ = 0.  (9.87)

We introduce the modified objective performance index,

J̃ = J + ∫_{t_{0}}^{t_{f}} p(t)^{⊤}(f(x, u) − ẋ) dt.  (9.88)

Because for any state trajectory, (9.87) is satisfied, for any choice of p(t) the value of J̃ is the same as that of J. Therefore, we can express J̃ as

J̃ = Φ(x(t_{f})) + ∫_{t_{0}}^{t_{f}} F (x(t), u(t)) dt + ∫_{t_{0}}^{t_{f}} p(t)^{⊤}(f(x, u) − ẋ) dt
= Φ(x(t_{f})) + ∫_{t_{0}}^{t_{f}} (H(x, u, p) − p^{⊤}ẋ) dt.  (9.89)

Let u(t) be a nominal control strategy; it determines a corresponding state trajectory x(t). If we apply another control strategy, say v(t), that is “close” to u(t), then v(t) will produce a state trajectory close to the nominal trajectory. This new state trajectory is just a perturbed version of x(t) and it can be represented as

x(t) + δx(t).

The change in the state trajectory yields a corresponding change in the modified performance index. We represent this change as δJ̃; it has the form,

δJ̃ = Φ(x(t_{f}) + δx(t_{f})) − Φ(x(t_{f})) + ∫_{t_{0}}^{t_{f}} (H(x + δx, v, p) − H(x, u, p) − p^{⊤}δẋ) dt. (9.90)

We evaluate the above integral. We first perform an integration by parts to obtain

∫_{t_{0}}^{t_{f}} p^{⊤}δẋ dt = p(t_{f})^{⊤}δx(t_{f}) − p(t_{0})^{⊤}δx(t_{0}) − ∫_{t_{0}}^{t_{f}} ṗ^{⊤}δx dt.  (9.91)

Note that δx(t_{0}) = 0 because a change in the control strategy does not change the initial state. Taking into account the above, we represent (9.90) as

δJ̃ = Φ(x(t_{f}) + δx(t_{f})) − Φ(x(t_{f})) − p(t_{f})^{⊤}δx(t_{f})
+ ∫_{t_{0}}^{t_{f}} (H(x + δx, v, p) − H(x, u, p) + ṗ^{⊤}δx) dt.

We next replace (Φ(x(t_{f}) + δx(t_{f})) − Φ(x(t_{f}))) with its first-order approximation, and add and subtract the term H(x, v, p) under the integral to obtain

δJ̃ = (∇xΦ|_{t=t_{f}} − p(t_{f}))^{⊤} δx(t_{f})

+ ∫_{t_{0}}^{t_{f}} (H(x + δx, v, p) − H(x, v, p) + H(x, v, p) − H(x, u, p) + ṗ^{⊤}δx) dt.

Replacing (H(x + δx, v, p) − H(x, v, p)) with its first-order approximation,

H(x + δx, v, p) − H(x, v, p) = \frac{∂H}{∂x} δx,

gives

δJ̃ = (∇xΦ|_{t=t_{f}} − p(t_{f}))^{⊤} δx(t_{f})

+ ∫_{t_{0}}^{t_{f}} ((\frac{∂H}{∂x} + ṗ^{⊤}) δx + H(x, v, p) − H(x, u, p)) dt.

Selecting in the above p as the solution to the differential equation

\frac{∂H}{∂x} + ṗ^{⊤} = 0^{⊤}

with the final condition

p(t_{f}) = ∇xΦ|_{t=t_{f}} ,

reduces δJ̃ to

δJ̃ = ∫_{t_{0}}^{t_{f}} (H(x, v, p) − H(x, u, p)) dt.

If the original control u(t) is optimal, then we should have δJ̃ ≥ 0, that is, H(x, v, p) ≥ H(x, u, p). We summarize our development in the following theorem.

Theorem 9.6 Necessary conditions for u ∈ U to minimize (9.85) subject to (9.86) are:

ṗ = − (\frac{∂H}{∂x} )^{⊤} ,

where H = H(x, u, p) = F (x, u) + p^{⊤}f(x, u),

H(x^{∗}, u^{∗}, p^{∗}) = min_{u∈U} H(x, u, p).

If the final state, x(t_{f}), is free, then in addition to the above conditions it is required that the following end-point condition is satisfied,

p(t_{f}) = ∇xΦ|_{t=t_{f}} .

The equation,

ṗ = − (\frac{∂H}{∂x} )^{⊤}  (9.92)

is called in the literature the adjoint or costate equation.

We illustrate the above theorem with the following well-known example. Example 9.21 We consider a controlled object modeled by

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ 1 \end{matrix} ] u
= Ax + bu

with the performance index

J = ∫_{0}^{t_{f}} dt.

The control is required to satisfy

|u(t)| ≤ 1

for all t ∈ [0, t_{f}]. This constraint means that the control must have magnitude no greater than 1. Our objective is to find admissible control that minimizes J and transfers the system from a given initial state x_{0} to the origin.

We begin by finding the Hamiltonian function for the problem,

H = 1 + p^{⊤}(Ax + bu)
= 1 + [ p_{1} p_{2} ] ([ \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ 1 \end{matrix} ] u)
= 1 + p_{1}x_{2} + p_{2}u.

The costate equations are

[ ṗ_{1} ; ṗ_{2} ] = − [ \frac{∂H}{∂x_{1}} ; \frac{∂H}{∂x_{2}} ] = [ \begin{matrix}0 \\ −p_{1} \end{matrix} ] .

Solving the costate equations yields

p_{1} = d_{1} and p_{2} = −d_{1}t + d_{2},

where d_{1} and d_{2} are integration constants. We now find an admissible control minimizing the Hamiltonian,

arg_{u} min H = arg_{u} min(1 + p_{1}x_{2} + p_{2}u)
= arg_{u} min(p_{2}u)
= \{ \begin{matrix}u(t) = 1 & if & p_{2} < 0 \\ u(t) = −1 & if & p_{2} > 0 \end{matrix} .

Hence,

u^{∗}(t) = −sign(p_{2}^{∗}) = −sign(−d_{1}t + d_{2}),

where

sign(z) = \{ \begin{matrix}1 & if & z > 0 \\ −1 & if & z < 0. \end{matrix}

Thus, the optimal control law is piecewise constant taking the values 1 or −1. This control law has at most two intervals of constancy because the argument of the sign function is a linear function, −d_{1}t + d_{2}, that changes its sign at most once. This type of control is called a bang-bang control because it switches back and forth between its extreme values. System trajectories for u = 1 and u = −1 are families of parabolas and their equations were derived in Example 3.12. A phase portrait for u = 1 is shown in Figure 3.5, while for u = −1 in Figure 3.6. Note that only one parabola from each family passes through the specified terminal point x = [ 0 0 ]^{⊤} in the state plane. Segments of the two parabolas through the origin form the switching curve,

x_{1} = −\frac{1}{2}x_{2}^{2} sign(x_{2}).

This means that if an initial state is above the switching curve, then u = −1 is used until the switching curve is reached. Then, u = 1 is used to reach the origin. For an initial state below the switching curve, the control u = 1 is used first to reach the switching curve, and then the control is switched to u = −1. We implement the above control action as

u = −sign(v) = \{ \begin{matrix}1 & if & v < 0 \\ −1 & if & v > 0, \end{matrix}

where v = v(x_{1}, x_{2}) = x_{1} + \frac{1}{2}x_{2}^{2}sign(x_{2}) = 0 is the equation describing the switching curve. We can use the above equation to synthesize a closed-loop system such that starting at an arbitrary initial state in the state plane, the trajectory will always be moving in an optimal fashion towards the origin. Once the origin is reached, the trajectory will stay there. A phase portrait of the closed-loop time optimal system is given in Figure 9.25.

Example 9.22 Consider the schematic of an armature-controlled DC motor, given in Figure 9.24. Use the minimum principle to find the armature voltage e_{a}(t) such that the motor Figure 9.25: A phase-plane portrait of the time optimal closed-loop system of Example 9.21.

angular velocity ω changes from ω(0) = 0 rad/sec at t = 0 to ω(1) = 10 rad/sec while minimizing the energy dissipated in the armature resistor, R_{a}. There are no constraints on e_{a}. Note that the energy dissipated in the armature resistor is

J = ∫_{0}^{1} R_{a}i_{a}(t)^{2}dt,

where i_{a} is the armature current.

The modeling equation of the DC motor was derived in Example 9.19. It is given by (9.82) on page 502. We represent the expression for the energy dissipated in the armature resistor in terms of ω and e_{a}. Applying Ohm’s law to the armature circuit, we get

e_{a} − K_{b}ω = R_{a}i_{a}.

Hence,

J = ∫_{0}^{1} R_{a}i_{a}(t)^{2}dt = ∫_{0}^{1} \frac{(e_{a} − K_{b}ω)^{2}}{R_{a}} dt.

The Hamiltonian function is

H = \frac{(e_{a} − K_{b}ω)^{2}}{R_{a}} + p (−\frac{K_{i}K_{b}}{R_{a}I_{eq}} ω + \frac{K_{i}}{I_{eq}R_{a}} e_{a}) .

Because there are no constraints on e_{a}, we can find the optimal control by solving the equation

0 = \frac{∂H}{∂e_{a}} = \frac{2 (e_{a} − K_{b}ω)}{R_{a}} + \frac{K_{i}}{I_{eq}R_{a}} p.

Hence,

e_{a} = 2ω − p.  (9.93)

The costate equation is

ṗ = −\frac{∂H}{∂ω} = \frac{2K_{b} (e_{a} − K_{b}ω)}{R_{a}} + \frac{K_{i}K_{b}}{R_{a}I_{eq}} p = 2(e_{a} − 2ω) + 2p.  (9.94)

We substitute (9.93) into (9.94) to obtain

ṗ = 0.

Hence, p(t) = constant = c.

Therefore,

e_{a} = 2ω − c.  (9.95)

Substituting (9.95) into (9.82), we get

ω̇ = −c.

Solving the above yields

ω(t) = −ct + ω(0) = −ct,

because ω(0) = 0. We next use the boundary condition ω(1) = 10 to find c = −10. Hence, the optimal armature voltage is

e_{a}^{∗}(t) = 20t + 10

Example 9.23 In this example, we will solve the Johann Bernoulli’s brachistochrone problem, introduced in Section 9.2, using the minimum principle. We first recall the problem as re-stated by Johann Bernoulli in 1697.

“Among the infinitely many curves which join the two given points ... choose the one such that, if the curve is replaced by a thin tube or groove, and a small sphere placed in it and released, then this will pass from one point to the other in the shortest time.” quoted from W. Dunham [89, page 200].

It is difficult to resist not to mention about the reward that one to obtain for solving the problem:

“Let who can seize quickly the prize which we have promised to the solver. Admittedly this prize is neither of gold nor silver, for these appeal only to base and venal souls. ... Rather, since virtue itself is its own most desirable reward and a fame is a powerful incentive, we offer the prize fitting for the man of noble blood, compounded of honor, praise, and approbation...” quoted from W. Dunham [89, page 200]. We add that Johann Bernoulli wrote the above after he already knew the solution to the problem! Model and the Performance Index

The field is conservative, hence the sums of the kinetic and potential energy is constant,

\frac{1}{2}mv(t)^{2} − mgy(t) = \frac{1}{2}mv^{2}(t_{0}) − mgy(t_{0}) = 0,

where we assumed that the particle is at rest at t_{0}. Using the above, we can find the particle’s speed at any time t ≥ t_{0},

v(t) = \sqrt{2gy(t)}.

Then, the brachistochrone problem can be formulated as an optimal control problem of the form,

minimize J(t_{0}) = ∫_{t_{0}}^{T} dt
subject to \{ \begin{matrix}ẋ & = & v cos θ \\ ẏ & = & v sin θ, \end{matrix}

where θ is the angle that the particle velocity vector forms with the positive x-axis, while ẋ and ẏ are the horizontal and vertical components of the particle velocity vector. The path angle θ is considered as the control input to be determined. Particle Trajectory

To proceed, we form the Hamiltonian function,

H = 1 + p_{1}v cos θ + p_{2}v sin θ,  (9.96)

where p_{1} and p_{2} are the costate variables. The costate equations are:

[ ṗ_{1} ; ṗ_{2} ] = − [ \frac{∂H}{∂x} ; \frac{∂H}{∂y} ] = [ \begin{matrix}0 \\ −\frac{g}{v}(p_{1} cos θ + p_{2} sin θ) \end{matrix} ] .

It follows from the above that p_{1} = K = constant. Because there are no constraints on the control input θ, the necessary condition for the control optimality is

\frac{∂H}{∂θ} = 0,

where

\frac{∂H}{∂θ} = −p_{1}v sin θ + p_{2}v cos θ.

Hence,

p_{2} = p_{1} tan θ = K tan θ.

We next find an expression for θ̇. First note that

ṗ_{2} = \frac{∂p_{2}}{∂θ} θ̇ = \frac{∂(K tan θ)}{∂θ}θ̇ = \frac{K}{cos^{2} θ}θ̇.  (9.97)

On the other hand,

ṗ_{2} = −\frac{g}{v} (p_{1} cos θ + p_{2} sin θ)
= −\frac{g}{v} (K cos θ + K tan θ sin θ)
= −\frac{Kg}{v} (cos θ + \frac{sin^{2} θ}{cos θ})
= −\frac{Kg}{v} \frac{cos^{2} θ + sin^{2} θ}{cos θ})
= −\frac{Kg}{v cos θ}.  (9.98)

Combining (9.97) and (9.98) yields

θ̇ = −\frac{g}{v} cos θ.  (9.99)

We now use the boundary conditions to express x and y as functions of time and θ. First, using the above we can write

ẏ = \frac{dy}{dθ} θ̇ = \frac{dy}{dθ} (−\frac{g}{v} cos θ) .

On the other hand,

ẏ = v sin θ.

Equating the right-hand sides of the above two equations yields

\frac{dy}{v^{2}} = \frac{dy}{2gy} = −\frac{1}{g} tan θdθ.

9.5. PONTRYAGIN’S MINIMUM PRINCIPLE 515 Integrating both sides gives

\frac{1}{2g} ∫ \frac{dy}{y} = −\frac{1}{g} ∫ tan θdθ,

that is,

\frac{1}{2g} ln |y| = \frac{1}{g} ln | cos θ|.

Performing manipulations, we obtain

y = C cos^{2} θ,

where C is an integration constant. Evaluating the above at t = T gives

y(T) = y_{1} = C cos^{2} θ(T).

Hence,

C = \frac{y_{1}}{cos^{2} θ(T)}

and

y(t) = \frac{y_{1}}{cos^{2} θ(T)} cos^{2} θ(t)  (9.100)

In a similar way, we can obtain an expression for x as a function of time and θ. Indeed, on the one hand,

ẋ = \frac{dx}{dθ}θ̇ = −\frac{dx}{dθ} \frac{g}{v} cos θ,

on the other hand,

ẋ = v cos θ.

Comparing the right-hand sides of the above two equations gives

\frac{dx}{dθ} = −\frac{v^{2}}{g} = −2y = −2\frac{y_{1}}{cos^{2} θ(T)} cos θ.

We represent the above as

dx = −2y = −2\frac{y_{1}}{cos^{2} θ(T)} cos θdθ.

Integrating both sides, using the identity, cos^{2} θ = \frac{1}{2}(1 + cos 2θ), yields

x = −2\frac{y_{1}}{cos^{2} θ(T)} (\frac{θ}{2} + \frac{sin 2θ}{4}) + C̃,

where C̃ is the integration constant. Evaluating the above at t = T gives

x_{1} = −2\frac{y_{1}}{cos^{2} θ(T)} (\frac{θ(T)}{2} + \frac{sin 2θ(T)}{4}) + C̃.

Hence,

C̃ = x_{1} + \frac{y_{1}}{cos^{2} θ(T)} (θ(T) + \frac{sin 2θ(T)}{2}) .

Using the above, we obtain

x(t) = x_{1} + \frac{y_{1}}{2 cos^{2} θ(T)} (2(θ(T) − θ(t)) + sin 2θ(T) − sin 2θ(t))

(9.101)

We point out that the equation for x(t), y(t) and θ̇ specify an implicit feedback control law in the sense that the control θ can be calculated given the state (x(t), y(t)).

We use the boundary conditions to find the description of the time-optimal closed loop system. We assume, without loss of generality, that x(t_{0}) = y(t_{0}) = x(0) = y(0) = 0. Then, using the expression for y(t) we note that y(t_{0}) = 0 = \frac{y_{1}}{cos^{2} θ(T)} cos^{2} θ(t_{0}). Thus, the angle of departure is θ(t_{0}) = 90^{∘}. With the above initial path angle, y, and hence v, is increased at the outset as quickly as possible. Using this data, we next evaluate x at t = t_{0} to form an equation that can be used to find the angle of arrival θ(T). The equation is

2x_{1} cos^{2} θ(T) + y_{1} (2θ(T) − π + sin 2θ(T)) = 0.

The above equation can be solved easily using MATLAB’s Symbolic Toolbox or the MAT-LAB’s fsolve function. Once θ(T) is found, we use the expression for y to find cos θ, where

cos θ(t) = \sqrt{\frac{y(t)}{y_{1}}} cos θ(T).

Hence,

sin θ(t) = \sqrt{1 − cos^{2} θ(t)}.

We substitute the above expression into ẋ = v cos θ and ẏ = v sin θ to obtain the state-space model of the time optimal motion of the particle moving under the force of gravity from a given point to another specified point. The equations are,

\{ \begin{matrix}ẋ(t) & = & \sqrt{2gy(t)}\sqrt{\frac{y(t)}{y_{1}}} cos θ(T) \\ ẏ(t) & = & \sqrt{2gy(t)}\sqrt{1 − \frac{y(t)}{y_{1}} cos^{2} θ(T)} \end{matrix}

subject to the boundary conditions [ x(t_{0}) y(t_{0}) ] = [ x_{0} y_{0} ] and [ x(T) y(T) ] = [ x_{1} y_{1} ]. The Time of Quickest Descent

We derive an expression that can be used to compute the time of the particle travel. We first establish an auxiliary result that we will use in our derivation. Specifically, we will show that θ̇(t) is constant on the interval [t_{0}, T]. Differentiating both sides of equation (9.99) gives

\frac{dθ̇}{dt} = \frac{d}{dt} (−\frac{g}{\sqrt{2gy}} cos θ)
= −\frac{g}{\sqrt{2g}} (\frac{d}{dt} (\frac{1}{\sqrt{y}} cos θ))
= −\frac{g}{\sqrt{2g}} (−\frac{1}{2} \frac{ẏ cos θ}{y^{3/2}} − \frac{sin θ}{\sqrt{y}}θ̇)
= \frac{g}{\sqrt{2g}} (\frac{1}{2} \frac{ẏ cos θ}{y^{3/2}} + \frac{sin θ}{\sqrt{y}}θ̇) .

Substituting into the above the expressions for θ̇ and ẏ gives

\frac{dθ̇}{dt} = \frac{g}{\sqrt{2g}} (\frac{1}{2} \frac{\sqrt{2gy} sin θ cos θ}{y^{3/2}} − g \frac{sin θ cos θ}{\sqrt{y}\sqrt{2gy}})
= \frac{g}{\sqrt{2g}} (\frac{\sqrt{g}}{y\sqrt{2}} sin θ cos θ − \frac{\sqrt{g}}{y\sqrt{2}} sin θ cos θ)
= 0.

Because θ̇ is constant on the interval [t_{0}, T], we use (9.99) to write

\frac{dθ}{dt} = −\frac{g}{\sqrt{2gy(T)}} cos θ(T).

Separating the variables yields

dθ = −\frac{g}{\sqrt{2gy(T)}} cos θ(T)dt.

Integrating both sides of the above, we obtain

∫_{θ(t)}^{θ(T)} dθ = −\frac{g}{\sqrt{2gy(T)}} cos θ(T) ∫_{t_{0}}^{T} dt.

Hence,

θ(T) − θ(t) = −\frac{g}{\sqrt{2gy(T)}} cos θ(T)(T − t_{0}).

To obtain an expression for the particle travel time, we set in the above equation t = t_{0} = 0 and perform simple manipulations to obtain

T = \frac{π/2 − θ(T)}{cos θ(T)}\sqrt{2y(T)g}

Figure 9.26: Constructing an upside-down cycloid—the solution to the brachistochrone problem.

It’s an Upside-Down Cycloid!

The path of quickest descent is an upside-down cycloid. A cycloid is a curve described by a point on the circumference of a circle that rolls without slipping—see Figure 9.26. The equations describing a cycloid are

\begin{matrix}x(ϕ) & = & R(ϕ − sin ϕ) \\ y(ϕ) & = & R(1 − cos ϕ) \end{matrix} \}

We will now show that equations (9.101) and (9.100) describe a cycloid. To proceed, we define

R = \frac{y_{1}}{2 cos^{2} θ(T)} and ϕ(t) = π − 2θ(t).  (9.102)

Using the above notation, and taking into account that sin(π−ϕ) = sin ϕ, we re-write (9.101) as

x(t) − x_{1} + R(ϕ(T) − sin ϕ(T)) = R(ϕ(t) − sin ϕ(t)).

Using the notation given in (9.102) and the fact that cos 2ϕ = 2 cos^{2} ϕ−1, we represent (9.100) as

y(t) = \frac{y_{1}}{cos^{2} θ(T)} cos^{2} θ(t)
= \frac{y_{1}}{cos^{2} θ(T)} \frac{cos 2θ(t) + 1}{2}
= R(1 + cos(π − ϕ(t))
= R(1 − cos ϕ(t)),

because cos(π − ϕ(t)) = − cos ϕ.

In the following, we discuss a general minimum time regulator problem for a class of multi-input dynamical systems modeled by

ẋ = Ax + Bu.

The goal is to transfer the system from some initial state to the origin in minimum time subject to

|u_{i}| ≤ k_{i}, k_{i} > 0, i = 1, 2, …, m.

Thus, the performance index for the problem is J = ∫_{t_{0}}^{t_{f}} dt. We begin our analysis by forming the associated Hamiltonian function,

H = 1 + p^{⊤}(Ax + Bu)
= 1 + p^{⊤}Ax + ∑_{i=1}^{m} (p^{⊤}b_{i}) u_{i},

where b_{i} is the i-th column of B. Let

s_{i}(t) = p^{∗⊤}(t)b_{i}.

Then, applying Pontryagin’s minimum principle we obtain the following form of the optimal control,

u_{i}^{∗}(t) = −k_{i}sign(s_{i}(t)), i = 1, 2, …, m.

Thus, the optimal control for the i-th control channel has the bang-bang form, u_{i}^{∗} = ±k_{i}. Note that s_{i}(t) = p^{∗⊤}(t)b_{i} = 0 is the switching curve for the i-th control channel. We compute the costate vector from the costate equation,

ṗ^{∗⊤} = −\frac{∂H}{∂x}
= −\frac{∂}{∂x} (p^{∗⊤}Ax)
= −p^{∗⊤}A,

or

ṗ^{∗} = −A^{⊤}p^{∗}.

The solution to the costate equation is

p^{∗}(t) = e^{−A^{⊤}t}p^{∗}(0).

Thus, the i-th switching curve can be represented as

s_{i}(t) = p^{∗}(0)^{⊤}e^{−At}b_{i} = 0.

Observe that if s_{i}(t) = 0 on some time interval, then u_{i}^{∗}(t) is indefinite on this interval. We show that under some mild assumptions there is no time interval on which s_{i}(t) = 0, that is, there is no time interval on which u_{i}^{∗}(t) is indefinite. First, we assume that b_{i} ≠ 0. Note that in our problem here, min_{u∈U} H(x, u) = 0 because J^{∗}(t, x) = J^{∗}(x) and therefore \frac{∂J^{∗}}{∂t} = 0. Hence, 1 + p^{∗}(t)^{⊤}(Ax^{∗} + Bu^{∗}) = 0 for all t for the optimal process. Thus, the optimal costate vector p^{∗}(t) cannot be zero for any value of t. Finally, if s_{i} = p^{∗⊤}b_{i} is zero on some time interval, that is, s_{i} is constant on a time interval, then this implies that on the above time interval,

ṡ_{i} = ṗ^{∗⊤}b_{i} = −p^{∗⊤}Ab_{i} = 0.

Similarly, higher derivatives of s_{i}(t) = p^{∗⊤}b_{i} will also be zero on the time interval on which s_{i}(t) = p^{∗⊤}b_{i} is zero, that is, s_{i}^{(j)}(t) = (−1)^{j}p^{∗⊤}A^{j}b_{i} = 0, j = 1, 2, . . .. We represent the above relations as

p^{∗⊤} [ b_{i} Ab_{i} ⋯ A^{n−1}b_{i} ⋯] = 0.

If the system is controllable by the i-th input acting alone, then

rank [ b_{i} Ab_{i} ⋯ A^{n−1}b_{i} ⋯] = n,

and we would have to have p^{∗} = p^{∗}(t) = 0. However, we already ruled out this case and thus s_{i}(t) = p^{∗⊤}b_{i} cannot be zero on any time interval. In conclusion, if the system is controllable by each input acting alone, then there is no time interval on which optimal control vector is indefinite.

9.5.2 A Two-Point Boundary-Value Problem

We consider the problem of minimizing the quadratic performance index

J = \frac{1}{2}x(t_{f})^{⊤}F x(t_{f}) + \frac{1}{2} ∫_{t_{0}}^{t_{f}} (x(t)^{⊤}Qx(t) + u(t)^{⊤}Ru(t)) dt  (9.103)

subject to

ẋ(t) = Ax(t) + Bu(t), x(t_{0}) = x_{0}, x(t_{f}) is free.  (9.104)

We assume that there are no constraints on the control input u. The weight matrices F and Q are real symmetric positive semi-definite, and the matrix R is real symmetric positive definite. The Hamiltonian function for the above linear quadratic control problem is

H = \frac{1}{2}x^{⊤}Qx + \frac{1}{2}u^{⊤}Ru + p^{⊤} (Ax + Bu) ,  (9.105)

where p ∈ ℝ^{n} is the costate vector. It follows from Theorem 9.6 that the optimal controller must minimize the Hamiltonian function. Because the control vector is unconstrained, the necessary condition for optimality of the control u is

\frac{∂H}{∂u} = 0.  (9.106)

Evaluating (9.106) yields

\frac{∂H}{∂u} = u^{⊤}R + p^{⊤}B = 0.  (9.107)

Hence, the optimal controller has the form

u = −R^{−1}B^{⊤}p.  (9.108)

Substituting the above into (9.104), we obtain

ẋ = Ax − BR^{−1}B^{⊤}p, x(t_{0}) = x_{0}.  (9.109)

We next find the costate equation,

ṗ = − (\frac{∂H}{∂x} )^{⊤}
= −Qx − A^{⊤}p.  (9.110)

Combining (9.109) and the above yields 2n linear differential equations

[ ẋ ; ṗ ] = [ \begin{matrix}A & −BR^{−1}B^{⊤} \\ −Q & −A^{⊤} \end{matrix} ] [ \begin{matrix}x \\ p \end{matrix} ] .  (9.111)

By Theorem 9.6, the state vector at time t_{f} must satisfy

p(t_{f}) = \frac{1}{2} (∇x^{⊤}F x)|_{t=t_{f}} = F x(t_{f}).  (9.112)

In summary, we reduced the linear quadratic control problem to solving 2n linear differential equations with mixed boundary conditions, which is an example of a two-point boundary value problem, or TPBVP for short. We solve the above TPBVP. If we had the initial conditions x(t_{0}) and p(t_{0}), then the solution to (9.111) would have the form

[ \begin{matrix}x(t_{f}) \\ p(t_{f}) \end{matrix} ] = e^{H(t_{f} −t_{0})} [ \begin{matrix}x(t_{0}) \\ p(t_{0}) \end{matrix} ] ,  (9.113)

where

H = [ \begin{matrix}A & −BR^{−1}B^{⊤} \\ −Q & −A^{⊤} \end{matrix} ] .

However, in this particular TPBVP, p(t_{0}) is unknown and instead we are given p(t_{f}). To proceed, let

[ \begin{matrix}x(t_{f}) \\ p(t_{f}) \end{matrix} ] = e^{H(t_{f} −t)} [ \begin{matrix}x(t) \\ p(t) \end{matrix} ] ,  (9.114)

where

e^{H(t_{f} −t)} = [ \begin{matrix}Φ_{11}(t_{f}, t) & Φ_{12}(t_{f}, t) \\ Φ_{21}(t_{f}, t) & Φ_{22}(t_{f}, t) \end{matrix} ] .

Each of the blocks Φ_{ij}(t_{f}, t), i, j = 1, 2, is n × n. Thus we have

[ \begin{matrix}x(t_{f}) \\ p(t_{f}) \end{matrix} ] = [ \begin{matrix}Φ_{11}(t_{f}, t) & Φ_{12}(t_{f}, t) \\ Φ_{21}(t_{f}, t) & Φ_{22}(t_{f}, t) \end{matrix} ] [ \begin{matrix}x(t) \\ p(t) \end{matrix} ] .  (9.115)

The above matrix equation can be represented as

x(t_{f}) = Φ_{11}(t_{f}, t)x(t) + Φ_{12}(t_{f}, t)p(t),  (9.116)
p(t_{f}) = Φ_{21}(t_{f}, t)x(t) + Φ_{22}(t_{f}, t)p(t).  (9.117)

Using the boundary condition (9.112) and (9.116) gives

p(t_{f}) = F x(t_{f}) = F Φ_{11}(t_{f}, t)x(t) + F Φ_{12}(t_{f}, t)p(t).  (9.118)

Subtracting (9.117) from (9.118), we obtain

0 = (F Φ_{11}(t_{f}, t) − Φ_{21}(t_{f}, t)) x(t) + (F Φ_{12}(t_{f}, t) − Φ_{22}(t_{f}, t)) p(t).  (9.119)

Using the above, we compute

p(t) = (Φ_{22}(t_{f}, t) − F Φ_{12}(t_{f}, t))^{−1} (F Φ_{11}(t_{f}, t) − Φ_{21}(t_{f}, t)) x(t)
= P (t)x(t),  (9.120)

where,

P (t) = (Φ_{22}(t_{f}, t) − F Φ_{12}(t_{f}, t))^{−1} (F Φ_{11}(t_{f}, t) − Φ_{21}(t_{f}, t)) .  (9.121)

Substituting (9.120) into (9.108) yields the optimal state-feedback controller

u(t) = −R^{−1}B^{⊤}P (t)x(t).  (9.122)

We show that P (t) satisfies a matrix differential equation. Indeed, differentiating (9.120) gives

Ṗ x + P ẋ − ṗ = 0.  (9.123)

Substituting into the above (9.109), (9.110), and (9.120) yields

(Ṗ + P A − P BR^{−1}B^{⊤}P + Q + A^{⊤}P ) x = 0.  (9.124)

The preceding must hold throughout t_{0} ≤ t ≤ t_{f}. Hence, P must satisfy

Ṗ = P BR^{−1}B^{⊤}P − A^{⊤}P − P A − Q  (9.125)

subject to the boundary condition

P (t_{f}) = F .  (9.126)

Equation (9.125) is called the matrix Riccati differential equation. Note that because F is symmetric, so is P = P (t). Observe that P = P (t) given by (9.121) is a solution to (9.125) subject to the boundary condition (9.126).

The Riccati differential equation (9.125) associated with a class of one-dimensional plants was solved in Example 9.18.

9.6 Notes

Sussmann and Willems [315] made the following statement, in 1997, regarding the origins of optimal control: “Optimal control was born in 1697—300 years ago—in Groningen, a university town in the north of The Netherlands, when Johann Bernoulli, professor of mathematics at the local university from 1695 to 1705, published his solution of the brachystochrone problem.”

For a serious student of optimal control, the seminal text by Lee and Markus [205] is recommended. Less advanced than Lee and Markus are Kirk [181], Bryson and Ho [45], Owens [260], and Pierre [275]. The classical text by Kwakernaak and Sivan [199] is also worth perusing because of its clarity and its treatment of the stochastic aspects of control problems. Informative, and at the same time entertaining, accounts of the origins of the calculus of variations are given by Dunham [89, Chapter 8], and Sussmann and Willems [315]. Elsgolc [96] provides a lucid introduction, illustrated with many examples, to the calculus of variations. Applications of the calculus of variations in physics and engineering are provided by Weinstock [347]. Interrelations between the calculus of variations and optimal control are discussed by Hestenes [136]. Frequency-domain aspects of optimal control and robust optimal control design is discussed by Burl in [48]. Our presentation of the continuous minimum time regulator problem in Subsection 9.4.3, was based on Boltyanskii [38]. For more numerical examples related to optimal control, see Lewis and Syrmos [210], and Bryson [46]. Exciting applications of optimal control to a number of control problems of technology and science can be found in Lee and Markus [205] .

524 CHAPTER 9. OPTIMAL CONTROL

9.7 Exercises

Exercise 9.1 Find the variations of the functionals:

(i) v(x_{1}, x_{2}) = ∫_{0}^{π/2} (x_{1}^{4} + 2x_{1}^{2}x_{2}^{2} + x_{2}^{2}) dt, (ii) v(x_{1}, x_{2}) = ∫_{0}^{1} (e^{x_{1}−x_{2}} − e^{x_{1}+x_{2}}) dt. Exercise 9.2 Find the equations of the curves that are extremals of the functional

v(x) = ∫_{0}^{t_{f}} (\frac{1}{4}ẋ^{2} − tẋ − x + \frac{1}{2}xẋ) dt

for the boundary conditions (i) x(0) = 1, t_{f} = 2, and x(2) = 10; (ii) x(0) = 1, t_{f} = 10, and x(10) is free. Exercise 9.3 Find the equation of the curve that is an extremal of the functional

v(x) = ∫_{0}^{t_{f}} (\frac{1}{2}ẋ^{2} − tẋ + 2x^{2}) dt

subject to the boundary conditions

x(0) = 3/4, t_{f} = 1, and x(1) is free.

Exercise 9.4 Given a dynamical system model

ẋ = [ \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ] x + [ \begin{matrix}0 \\ 1 \end{matrix} ] u
y = [ 2 0 ] x,

and the associated performance index

J = ∫_{0}^{∞} (y^{2} + u^{2}) dt.

Assuming that u = −kx is such that J is minimized, find the poles of the closed-loop system.

Exercise 9.5 Given a dynamical system model

ẋ = [ \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ] x + [ \begin{matrix}0 \\ 1 \end{matrix} ] u
y = [ 2 0 ] x,

and the performance index

J = ∫_{0}^{∞} (y^{2} + u^{2}) dt.

Use the Kronecker product to represent the associated algebraic Riccati equation as the nonlinear vector algebraic equation. Solve it, and construct the optimal state feedback control law u = −kx. Assume that

x(0) = [ \begin{matrix}1 \\ 2 \end{matrix} ] .

Find the optimal value of J.

Exercise 9.6 Find a state feedback control law u so as to minimize

J = ∫_{0}^{1} (x^{2} + u^{2}) dt

subject to

ẋ = −x + u, x(0) = 2.

Assume J^{∗} = p(t)x^{2}.

Exercise 9.7 Use the principle of optimality to find the set of numbers x_{1}, x_{2}, …, x_{n} satisfying x_{1} + x_{2} + ⋯ + x_{n} = c, where c is a positive constant such that the product x_{1}x_{2} ⋯ x_{n} is maximized.

Exercise 9.8 Use dynamic programming to find the maximum cost path between node a and node t in the routing network depicted in Figure 9.27, where the travel costs are shown beside each segment and toll charges are shown by each node. The network can only be traversed from left to right.

Exercise 9.9 Use dynamic programming to find u[0] and u[1] that minimize

J = (x[2] − 2)^{2} + ∑_{k=0}^{1} u[k]^{2}

subject to

x[k + 1] = x[k] + u[k], x[0] = 1.

526 CHAPTER 9. OPTIMAL CONTROL

Figure 9.27: A routing network of Exercise 9.8.

Exercise 9.10 Consider the following optimal control problem:

minimize J_{2}(u) = \frac{1}{2} ∑_{k=0}^{1} u[k]^{2}
subject to x[k + 1] = x[k] + 2u[k], x[0] = 3, x[2] = 0.

Convert the above optimal control problem into a quadratic optimization problem with an equality constraint using the composite input vector

z = [ \begin{matrix}u[0] \\ u[1] \end{matrix} ] .

Then, employ Lagrange’s multiplier theorem to find the optimal sequence {u^{∗}[0], u^{∗}[1]}. Exercise 9.11 Consider the following optimal control problem:

minimize J_{2}(u) = \frac{1}{2}x[2]^{2} + \frac{1}{2} ∑_{k=0}^{1} (x[k]^{2} + u[k]^{2})
subject to x[k + 1] = x[k] + 2u[k], x[0] = 3.

Convert the above optimal control problem into a quadratic optimization problem with an equality constraint using the composite input-state vector

z = [ \begin{matrix}u[0] \\ x[1] \\ u[1] \\ x[2] \end{matrix} ] .

Then, employ Lagrange’s multiplier theorem to find the optimal sequence {u^{∗}[0], u^{∗}[1]}.

Exercise 9.12 Find u = u(t) so as to minimize

J = ∫_{0}^{1} u^{2}dt

subject to

ẋ = −2x + u, x(0) = 10, x(1) = 0.

Exercise 9.13 For the circuit shown in Figure 9.28, find the input voltage, u(t), that changes the current in the inductor form i = 10 A at t = 0 to i = 0 at t = 1 sec while minimizing the performance index,

J = ∫_{0}^{1} u(t)^{2}dt.

Exercise 9.14 Use the minimum principle to find the input voltage u(t) that charges the capacitor, in Figure 9.29, from x(0) = 2 V at t = 0 to x(10) = 12 V while minimizing the energy dissipated in the resistor. There are no constraints on u(t). Note that, by Kirchhoff’s voltage law,

Ri(t) + x(t) = u(t),

where i(t) = C \frac{dx(t)}{dt}, and the energy dissipated is

J = ∫_{0}^{10} Ri(t)^{2}dt.

Exercise 9.15 Show that if μ is an eigenvalue of the Hamiltonian matrix H, then so is −μ.

528 CHAPTER 9. OPTIMAL CONTROL

Figure 9.28: A circuit for Exercise 9.13.

Exercise 9.16 Consider the Hamiltonian function defined by

H = F + p^{⊤}f.

Show that if the functions f and F do not explicitly depend upon t, then

H|_{u=u^{∗}} = constant, t_{0} ≤ t ≤ t_{f}.

Exercise 9.17 Given the Hamiltonian matrix corresponding to a dynamical system and its associated performance index. The Hamiltonian matrix is

H = [ \begin{matrix}2 & −5 \\ −1 & −2 \end{matrix} ] .

The MATLAB command [V,D] = eig(H) resulted in the following data:

V = [ \begin{matrix}0.9806 & 0.7071 \\ −0.1961 & 0.7071 \end{matrix} ] , D = [ \begin{matrix}3 & 0 \\ 0 & −3 \end{matrix} ] .

(i) Write the equation of the closed loop system driven by the optimal controller u = −kx.

Figure 9.29: A capacitor charging circuit of Exercise 9.14.

(ii) Find the solution to the algebraic Riccati equation corresponding to the optimal controller.

Exercise 9.18 (i) Suppose that s_{i} is an eigenvalue of the matrix

A − BR^{−1}B^{⊤}P ,

and v_{i} ≠ 0 is the corresponding eigenvector, where P is the symmetric positive definite solution of the algebraic Riccati equation. Show that

[ \begin{matrix}v & i \\ P & v & i \end{matrix} ]

is an eigenvector of the associated Hamiltonian matrix corresponding to the eigenvalue s_{i}.

(ii) Show that if Q = Q^{⊤} > 0, then the real parts of the eigenvalues of the matrix

A − BR^{−1}B^{⊤}P

are all negative, where P = P^{⊤} > 0 is the solution of the associated algebraic Riccati equation.

530 CHAPTER 9. OPTIMAL CONTROL Exercise 9.19 Given the following model of a dynamical system:

ẋ_{1} = x_{2} + 5
ẋ_{2} = u,

where

|u| ≤ 1.

The performance index to be minimized is

J = ∫_{0}^{t_{f}} dt.

Find the state-feedback control law u = u(x_{1}, x_{2}) that minimizes J and drives the system from a given initial condition x(0) = [x_{1}(0), x_{2}(0)]^{⊤} to the final state x(t_{f}) = 0. Proceed as indicated below.

(a) Derive the equations of the optimal trajectories.

(b) Derive the equation of the switching curve.

(c) Write the expression for the optimal state-feedback controller. Exercise 9.20 (based on [289]) Given a dynamical system model

ẋ = Ax + bu,

where

A = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & 1 & ⋯ & 0 & 0 \\ ⋮ & & & ⋯ & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & 1 \\ −a_{0} & −a_{1} & −a_{2} & ⋯ & a_{n−2} & a_{n} \end{matrix} ] , b = [ 0 0 ⋮ 0 1 ] ,

and the associated performance index

J = \frac{1}{2} ∫_{0}^{∞} (x^{⊤}Qx + u^{2}) dt, Q = Q^{⊤} ≥ 0.

We label the above problem as (A, b). Consider now the dynamical system model ˙x̃ = Ãx̃ + b̃ũ, where b = b̃,

Ã = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 & 0 \\ 0 & 0 & 1 & ⋯ & 0 & 0 \\ ⋮ & & & ⋯ & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & 1 \\ 0 & 0 & 0 & ⋯ & 0 & 0 \end{matrix} ] ,

and the associated performance index has the form

J̃ = \frac{1}{2} ∫_{0}^{∞} (x̃^{⊤}Q̃x̃ + ũ^{2}) dt,

where Q̃ = Q + DÃ + Ã^{⊤}D + Dbb^{⊤}D, and

D = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 & −a_{0} \\ 0 & 0 & 1 & ⋯ & 0 & −a_{1} \\ ⋮ & & & ⋯ & & ⋮ \\ 0 & 0 & 0 & ⋯ & 0 & −a_{n−2} \\ −a_{0} & −a_{1} & −a_{2} & ⋯ & −a_{n−2} & −a_{n−1} \end{matrix} ] .

We refer to the second problem as (Ã, b̃).

(a) Show that the symmetric matrix P is a solution to the (A, b) problem if, and only if, the symmetric matrix P̃ = P − D is a solution to the (Ã, b̃) problem.

(b) Show that A − bb^{⊤}P = Ã − bb^{⊤}P̃.

Hint: Notice that Ã = A − bb^{⊤}D, and b̃ = b. Write down the algebraic Riccati equation for the (Ã, b̃) problem and perform appropriate substitutions.

Exercise 9.21 Consider the following variation of the brachistochrone problem: A particle of mass m is to slide down a frictionless track connecting two given points that are not located on the vertical line. The particle’s initial speed is v_{0}. The motion of the particle is constrained only by gravity and the shape of the track. Find the curve of the track that minimizes the time travel between the two given points. Solve this brachistochrone problem using the minimum principle of Pontryagin. Illustrate your derivations with a numerical example and simulations using MATLAB.

Exercise 9.22 (a) Consider the ground vehicle model described in Subsection 2.4.2. Let the input signal be u = δ_{f}—the front wheel steer angle, and δ_{r} = 0. Write a MATLAB script that animates the lateral motion Y versus the longitudinal motion X of the vehicle, the steer angle δ_{f}, and the heading angle ψ. Use the steer angle shown in Figure 9.30.

Figure 9.30: Plot of the steer angle δ_{f} versus time for Exercise 9.22.

(b) Use the linear model and the data from Table 2.1 to design a linear quadratic regulator (LQR). In this part, the control has the form

δ_{f} = −kx + Y_{ref},

where Y_{ref} is the command input shown in Figure 9.31. That is, the closed-loop system here has the form,

ẋ = (A − bk)x + bY_{ref}.

You as a designer, will select the weights Q and r. See how the linear vehicle tracks the command input. Observe the sign difference in the command input and the heading angle. Your LQR gain should be calculated for each chosen v_{x}. In other words, if in your interactive simulation, a specific value of v_{x} is chosen, then your program should calculate the corresponding gain for this particular value of v_{x}. Implement the LQR state-feedback employing the vehicle model that uses the nonlinear cornering

Figure 9.31: Plot of the command input Y_{ref} versus time for Exercise 9.22.

characteristics shown in Figure 2.10 and test the performance of the closed-loop system utilizing the command input shown in Figure 9.31. You may need to impose constraints on δ_{f}, for example, |δ_{f}| ≤ 20^{∘}. (Do not forget about converting degrees into radians.) Write a MATLAB script that animates the lateral motion Y versus the longitudinal motion X of the vehicle, the command input, and the heading angle.

Exercise 9.23 For the one-link manipulator from Exercise 5.22, use the linearized model from Part (b) of that exercise to

(a) design the optimal controller that minimizes the performance index

J = \frac{1}{2}Δx(t_{f})^{⊤}F Δx(t_{f}) + \frac{1}{2} ∫_{0}^{t_{f}} (Δx(t)^{⊤}QΔx(t) + Δu(t)^{2}) dt,

where t_{f} = 2 sec, F = 3I_{3}, and Q = 2I_{3}. (Hint: Use the method presented in Subsection 9.5.2);

(b) Implement and perform animation of the closed-loop system comprised of the optimal state-feedback controller driving the nonlinear model of the robot;

(c) Implement and perform animation of the closed-loop system comprised of the combined optimal controller-observer compensator driving the nonlinear model of the robot.

534 CHAPTER 9. OPTIMAL CONTROL

Sliding Modes

As philosopher of science Karl Popper has emphasized, a good theory is characterized by the fact that it makes a number of predictions that could in principle be disproved or falsified by observation. Each time new experiments are observed to agree with the predictions the theory survives, and our confidence in it is increased; but if ever a new observation is found to disagree, we have to abandon or modify the theory. At least that is what is supposed to happen, but you can always question the competence of the person who carried out the observation. [133, page 130]

10.1 Simple Variable Structure Systems

A variable structure system is a dynamical system whose structure changes in accordance with the current value of its state. A variable structure system can be viewed as a system composed of independent structures together with a switching logic between each of the structures. With appropriate switching logic, a variable structure system can exploit the desirable properties of each of the structures the system is composed of. Even more, a variable structure system may have a property that is not a property of any of its structures. We illustrate the above ideas with two numerical examples.

Example 10.1 This example was adapted from Utkin [331]. We consider a dynamical system model

\{ \begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & −ux_{1}, \end{matrix}  (10.1)

Figure 10.1: Figures (a) and (b) are phase-plane portraits of structures making up a variable structure system and Figure (c) is a phase-plane portrait of the variable structure system itself, where the structure of the system is changed any time the system’s trajectory crosses either of the coordinate axis. This figure was adapted from the survey paper by Utkin [331].

having two structures corresponding to u = 1/a and u = a, where a is a positive constant greater than 1; in our simulations we take a = 5. The phase-plane portraits of the structures are shown in Figures 10.1 (a) and (b). The phase-plane portraits of the individual structures are families of ellipses. Neither of the structures is asymptotically stable—they are only stable. However, by choosing a suitable switching logic between the structures, we can make the resulting variable structure system asymptotically stable. Indeed, suppose the structure of the system is changed any time the system’s trajectory crosses either of the coordinate axes of the state plane, that is,

u = \{ \begin{matrix}1/5 & if & x_{1}x_{2} < 0 \\ 5 & if & x_{1}x_{2} > 0. \end{matrix}  (10.2)

A phase plane portrait of the system (10.1) with the switching logic specified by (10.2) is Figure 10.2: Phase-plane portrait of the variable structure system described by (10.1) and (10.2).

shown in Figure 10.2. We add that the trajectories spiral clockwise towards the origin.

Example 10.2 Consider the double integrator

ẍ = u,

whose state-space model is

\{ \begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & u. \end{matrix}  (10.3)

Let u = ±2. Typical trajectories of the above system are shown in Figure 10.3. We have two independent structures, neither of them being stable. Let us use the following switching logic between each of the structures:

u = −2 sign (x_{1} + x_{2}) .  (10.4)

With the above switching logic, the closed-loop system changes its structure anytime the state trajectory crosses the line

x_{1} + x_{2} = 0.

Figure 10.3: Typical trajectories of the double integrator system.

We introduce the notation,

σ(x) = σ(x_{1}, x_{2}) = x_{1} + x_{2}.

Then, we can describe the switching logic action as

u = \{ \begin{matrix}−2 & if & σ(x) > 0 \\ 2 & if & σ(x) < 0. \end{matrix}  (10.5)

Typical trajectories of the closed-loop system (10.3), (10.4) for the initial conditions located “close” to the origin of the state-plane are shown in Figure 10.4. Close to the origin, once the trajectories intercept the switching line they are confined to it thereafter. If we increase the integration step size, then we can observe oscillations, or chattering, of the trajectory about the switching surface as it is explained in Figure 10.5. A plot of two trajectories with an increased integration time is shown in Figure 10.6.

10.2 Sliding Mode

The motion of the system while confined to the switching line or a surface is referred to as sliding. A sliding mode will exist if in the vicinity of the switching surface the state velocity vectors are directed towards the surface. In such a case, the switching surface attracts trajectories when they are in its vicinity and once a trajectory intersects the switching surface it will stay on it thereafter.

10.2. SLIDING MODE 539 Figure 10.4: Typical trajectories of the closed-system (10.3), (10.4) for initial conditions “close” to the origin of the state-plane. The applied control law is given by (10.4).

Figure 10.5: Chattering in a variable structure sliding mode system—a magnified view.

Figure 10.6: An illustration of chattering in a variable structure sliding mode system consisting of the double integrator and the control law, u = −sign(x_{1} + x_{2}). Two trajectories starting from two different initial conditions are plotted.

A surface σ(x) = 0 is attractive if

1. any trajectory starting on the surface remains there, and

2. any trajectory starting outside the surface tends to it at least asymptotically.

Thus, for a sliding motion to occur we need

lim_{σ→0^{+}} σ̇ < 0 and lim_{σ→0^{−}} σ̇ > 0.

The above conditions are illustrated in Figure 10.7. They ensure that the motion of the state trajectory x on either side of the switching surface σ(x) = 0 is towards the switching surface. The two conditions may be combined to give

σσ̇ < 0  (10.6)

in the neighborhood of the switching surface. Note, however, that the closed-loop system is now modeled by differential equations with discontinuous right-hand sides. The modeling differential equations are discontinuous on the switching surface σ(x) = 0. This is the reason we often refer to the switching surface as the discontinuity surface. A rigorous analysis of such systems cannot be carried out using methods of classical theory of differential equations.

Figure 10.7: An illustration of the sufficient condition of the attractivity of the switching surface.

A solution to the differential equation ẋ = f(t, x) with a continuous right-hand side is a function x(t) that has a derivative and satisfies the differential equation everywhere on a given interval. In the case when the right-hand side of ẋ = f(t, x) is discontinuous, we have to use more general methods such as one, for example, developed by Filippov [103]. We illustrate these points with a numerical example taken from Filippov [103, page 1].

Example 10.3 Consider the following differential equation with the discontinuous righthand side,

ẋ = 1 − 2sign(x).  (10.7)

For x < 0, we have ẋ = 3, while for x > 0, ẋ = −1. In Figure 10.8, plots of x versus time for different initial conditions are given. As t increases, the solution x of (10.7) reaches the line x = 0 and stays there. The function x(t), however, does not satisfy (10.7) in the usual sense because if ẋ(t) = 0 and x(t) = 0, the right hand side of (10.7) becomes

1 − 2sign(0) = 1 ≠ 0,

where we used the usual definition of the sign function, where sign(0) = 0. To circumvent the above difficulties, the defintion of the sign function is generalized by definining its value at zero as a point-to-set mapping, that is,

sign(0) ∈ [−1, 1]

542 CHAPTER 10. SLIDING MODES Figure 10.8: Plots of x versus time for different initial conditions for the differential equation (10.7). and as before

sign(z) = \{ \begin{matrix}1 & if z > 0 \\ −1 & if z < 0. \end{matrix}

Filippov [103, pages 2-3] recommends that the generalized solution to an ordinary differential equation with a discontinuous right-hand side should meet the following criteria:

(i) for a differential equation with a continuous right-hand side, the generalized solution should be equivalent with the usual one;

(ii) for the equation ẋ = f(t), the generalized solution should be of the form

x(t) = ∫ f(t)dt + constant;

(iii) for any initial condition x(t_{0}) = x_{0}, the solution must exist at least for t > t_{0} and should be continuable;

(iv) the generalized solution should be applicable to a broad class of equations that model dynamical systems;

(v) the limit of an uniformly convergent sequence of solutions should be a solution;

(vi) after applying to a solution a common change of variables, the result of the transformation should be a solution.

We next describe the proposed generalized solution in the sense of Filippov. We begin with an example. Suppose that we are given a single-input dynamical system model, ẋ = f(x, u) along with a switching surface σ(x) = 0. Let f^{−} = f(x, u^{−}) and f^{+} = f(x, u^{+}), where u = u^{−} if σ < 0 and u = u^{+} if σ > 0. We join the velocity vectors f^{−} and f^{+}. The resulting velocity vector f^{0} is obtained by drawing the line tangent to the switching surface that intersects the line segment joining the velocity vectors f^{−} and f^{+}. Thus,

f^{0} = αf^{−} + (1 − α)f^{+},

where α ∈ [0, 1]. The above construction is illustrated in Figure 10.9. Observe that f^{0} belongs to the smallest convex set containing f^{−} and f^{+}. Recall that a set S is said to be convex if for every x, y ∈ S and every real number α, 0 < α < 1, the point αx+(1−α)y ∈ S.

Figure 10.9: Constructing a solution in the sense of Filippov in ℝ^{2}.

Following Itkis [159], we give a simple interpretation of the above construction. The number α may be interpreted as the time the system’s trajectory remains below the switching surface, while (1−α) may be interpreted as the time the trajectory remains above the switching surface during the trajectory oscillation period about the switching surface. Specifically, if the trajectory performs one oscillation about σ = 0 in a “small” time interval Δτ, then it will traverse a distance ∥f^{−}∥αΔτ in time αΔτ with velocity f^{−}. During the remaining time of oscillation the trajectory will traverse a distance ∥f^{+}∥(1 − α)Δτ in time (1 − α)Δτ with velocity f^{+}. The average displacement of the trajectory during the time Δτ will then be ∥αf^{−} + (1 − α)f^{+}∥Δτ with the average velocity f^{0}.

Suppose that there are m discontinuity surfaces of the form

σ_{i} = {x : σ_{i}(x) = 0}, i = 1, 2, …, m.

Let

H = ⋃_{i=1}^{m} σ_{i}.

Then, we take a solution of the differential equation ẋ = f(x) with the discontinuous righthand side to be a solution of the differential inclusion

ẋ(t) ∈ G(x(t)),

where for each x, the set G(x) is the smallest closed convex set containing the cluster values, limits, of the function f(y) as y → x, y ∉ H, that is, a solution of the dynamical system

Figure 10.10: Constructing a solution in the sense of Filippov in ℝ^{3}.

ẋ = f(x) is an absolutely continuous function x(t), in the sense of Definition A.2, defined on an interval, or segment, L for which ẋ(t) ∈ G(x(t)) almost everywhere on L (Filippov [103]). Observe that if the function f is continuous at a point x, that is, x ∉ H, then G(x) consists of a single point, namely f(x), and the solution satisfies ẋ = f(x) in the usual sense. However, if x ∈ H, then x lies on one or more of the surfaces σ_{1}, σ_{2}, …, σ_{m}. Suppose that x lies on the surfaces σ_{i_{1}}, σ_{i_{2}}, …, σ_{i_{k}}. It then follows that for a sufficiently small δ > 0 the surfaces σ_{i_{1}}, σ_{i_{2}}, …, σ_{i_{k}} partition the δ-neighborhood of x into ℓ ≤ 2^{k} regions R_{1}, R_{2}, …, R_{ℓ}, in each of which the function f is continuous. Let f_{j} denote the function f restricted to the region R_{j}. Then, G(x) is the smallest closed convex set containing the set of points

\{ lim_{{x^{}˜ →x \atop x^{}˜ ∈R_{j}}} f_{j}(x̃) : j = 1, 2, …, ℓ\} .  (10.8)

The velocity vector, ẋ = f^{0}, lies then on the intersection of the tangent plane to the surfaces of discontinuity and the convex set described by (10.8). This is illustrated in Figures 10.9 and 10.10.

10.3 A Simple Sliding Mode Controller

In this section, we discuss the problem of designing sliding mode controllers for single-input plants. The design of a variable structure sliding mode controller consists of two phases.

They are:

• sliding (switching) surface design so as to achieve the desired system behavior, like the stability to the origin, when restricted to the surface, and

• selecting feedback gains of the controller so that the closed-system is stable to the sliding surface.

There is a number of benefits when the system is in sliding. In particular,

• while in sliding the system is not affected by matched uncertainties, and

• while in sliding the system behavior is governed by a reduced set of differential equations. This is what we refer to as the order reduction. It is very useful in designing variable structure sliding mode controllers.

We discuss a sliding surface design phase for linear single-input plants. Without loss of generality, we assume that the plant model is already in its controller companion form,

ẋ = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 \\ 0 & 0 & 1 & ⋯ & 0 \\ ⋮ & & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 1 \\ −a_{0} & −a_{1} & −a_{2} & ⋯ & −a_{n−1} \end{matrix} ] x + [ 0 0 ⋮ 0 1 ] u.

Let the sliding surface be

σ(x) = sx
= [ s_{1} s_{2} ⋯ s_{n−1} 1 ] [ x_{1} ; ⋮ ; x_{n} ]
= s_{1}x_{1} + s_{2}x_{2} + ⋯ + s_{n−1}x_{n−1} + x_{n}
= 0.

Note that, without loss of generality, we assumed that s_{n} = 1. If this was not the case, we could divide both sides of sx = 0 by s_{n} ≠ 0 to ensure that s_{n} = 1. Using the above equation of sliding surface, we can write

x_{n} = −s_{1}x_{1} − s_{2}x_{2} − ⋯ − s_{n−1}x_{n−1}.

We combine the equations of the plant model and that of the sliding surface. First note that

ẋ_{n−1} = x_{n}.

Hence, we can write

ẋ_{1} = x_{2} ; ẋ_{2} = x_{3} ; ⋮ ; ẋ_{n−1} = −s_{1}x_{1} − s_{2}x_{2} − ⋯ − s_{n−1}x_{n−1}, }  (10.9)

or equivalently in matrix form,

ż = [ \begin{matrix}0 & 1 & 0 & ⋯ & 0 \\ 0 & 0 & 1 & ⋯ & 0 \\ ⋮ & & & & ⋮ \\ 0 & 0 & 0 & ⋯ & 1 \\ −s_{1} & −s_{2} & −s_{3} & ⋯ & −s_{n−1} \end{matrix} ] z,  (10.10)

where z = [ x_{1} x_{2} ⋯ x_{n−1} ]^{⊤}. The above equation describes the system dynamics in sliding. Observe the order reduction of the system dynamics in sliding. We now have n − m = n − 1 first-order differential equations. The poles of the reduced-order system are the roots of the equation

λ^{n−1} + s_{n−1}λ^{n−2} + ⋯ + s_{2}λ + s_{1} = 0.

The response of the system in sliding is completely specified by an appropriate choice of the components s_{1}, s_{2}, …, s_{n−1} of the switching surface.

Consider again Example 10.2. The sliding surface for this example can be represented as

σ(x) = [ s_{1} 1 ] x = 0,

or equivalently

s_{1}x_{1} + x_{2} = 0.

The dynamics of the system in sliding are described by

ż = −s_{1}z,

which is just a first-order system. Note that to ensure the reduced-system stability, we select s_{1} > 0.

After designing a sliding surface, we construct a feedback controller. In this section, we discuss a controller design for linear single-input plants. The controller objective is to drive the plant state to the sliding surface and maintain it on the surface for all subsequent time. We use a generalized Lyapunov approach in constructing the controller. Specifically, we use a distance, V = 0.5σ^{2}, from the sliding surface σ as a Lyapunov function candidate. Then, we select the controllers’ gains so that the time derivative of the chosen Lyapunov function candidate evaluated on the solution of the controlled system is negative definite with respect to the switching surface, thus ensuring the motion of the state trajectory to the surface as it is illustrated in Figure 10.7. Our goal is to find u = u(x) so that

\frac{d}{dt} (\frac{1}{2}σ^{2}) = σσ̇ < 0.

We consider the controller structure of the form:

u = k_{1}^{±}x_{1} + k_{2}^{±}x_{2} + ⋯ + k_{n}^{±}x_{n},  (10.11)

where k_{i}^{±} are the feedback gains to be determined so that the condition σσ̇ < 0 is satisfied. To determine the feedback gains, we substitute u, given by (10.11), into the expression σσ̇ and utilize (10.9) to get

σσ̇ = σ [ s_{1} s_{2} ⋯ 1 ] ẋ
= σ (s_{1}ẋ_{1} + ⋯ + s_{n−1}ẋ_{n−1} + ẋ_{n})
= σ (s_{1}x_{2} + ⋯ + s_{n−1}x_{n} + ẋ_{n})
= σ(s_{1}x_{2} + ⋯ + s_{n−1}x_{n} − a_{0}x_{1} − ⋯ − a_{n−1}x_{n} + u)
= σ(s_{1}x_{2} + ⋯ + s_{n−1}x_{n} − a_{0}x_{1} − ⋯ − a_{n−1}x_{n} + k_{1}^{±}x_{1} + ⋯ + k_{n}^{±}x_{n}).

Rearranging the terms yields

σσ̇ = σx_{1} (−a_{0} + k_{1}^{±}) + σx_{2} (−a_{1} + s_{1} + k_{2}^{±}) + ⋯ + σx_{n} (−a_{n−1} + s_{n−1} + k_{n}^{±}) .

A possible way to ensure that σσ̇ < 0 is to select the gains to force each term in σσ̇ to be negative, that is,

σx_{1} (−a_{0} + k_{1}^{±}) < 0 ; σx_{2} (−a_{1} + s_{1} + k_{2}^{±}) < 0 ; ⋮ ; σx_{n} (−a_{n−1} + s_{n−1} + k_{n}^{±}) < 0. }

A possible choice of gains that leads to satisfaction of the attractivity to the surface condition, σσ̇ < 0, is:

k_{1}^{+} < −|a_{0}| if σx_{1} > 0 k_{1}^{−} > |a_{0}| if σx_{1} < 0 ⋮ k_{n}^{+} < −(|a_{n−1}| + |s_{n−1}|) if σx_{n} > 0 k_{n}^{−} > |a_{n−1}| + |s_{n−1}| if σx_{n} < 0. }  (10.12)

We can also use the following controller gains:

k_{1}^{±} = −(|a_{0}| + ε)sign(x_{1}σ) ⋮ k_{n}^{±} = −(|a_{n−1}| + |s_{n−1}| + ε)sign(x_{n}σ), }  (10.13)

where ε > 0 is a design parameter. Note that the controllers’ gains given by (10.13) satisfy conditions (10.12). The larger the value of ε the faster the trajectory converges to the sliding surface. In Figure 10.11, we summarize the above design method in the form of a flow chart.

10.4 Sliding in Multi-Input Systems

In this section, we analyze sliding mode behvior in linear multi-input plants. When the number of the plant inputs is m, we can use sliding surface of the form

{x : Sx = 0, S ∈ ℝ^{m×n}} .

We also use Sx = 0 to denote the above sliding surface. In many applications nonlinear sliding surfaces arise. Here, we use linear sliding surfaces only, although one can use nonlinear sliding surfaces as well. Let σ(x) = Sx. Then,

σ(x) = [ σ_{1}(x) ; σ_{2}(x) ; ⋮ ; σ_{m}(x) ] = [ s_{1}x ; s_{2}x ; ⋮ ; s_{m}x ] ,

where s_{i} ∈ ℝ^{1×n}. Recall, that the system is in sliding mode if σ(x(t)) = 0 and σ̇(x(t)) = 0 for t ≥ t_{0}, where x is the state trajectory and t_{0} is a specified time. It follows that in a sliding mode the velocity vector ẋ(t) is tangent to the switching surface. Equivalently,

(∇σ_{i})^{⊤} ẋ(t) = s_{i}ẋ(t) = 0, for i = 1, 2, …, m,

550 CHAPTER 10. SLIDING MODES Figure 10.11: Summary of a single-input sliding mode controller design. Adapted from DeCarlo et al. [77]. that is,

σ̇(x(t)) = Sẋ(t) = 0.

In summary, the system in sliding mode is described by the equations,

\begin{matrix}σ(x(t)) & = & 0 \\ σ̇(x(t)) & = & 0. \end{matrix} \}  (10.14)

Consider now the plant model ẋ(t) = Ax(t) + Bu(t). Combining Sẋ(t) = 0 and the modeling equation of the plant, we get

SAx + SBu = 0.

If the matrix SB is nonsingular, then we can solve the above algebraic equation for u. The result is called the equivalent control and it is denoted using the symbol u_{eq}. We have

u_{eq} = − (SB)^{−1} SAx.  (10.15)

Substituting u_{eq} into the plant model equation yields

ẋ = (I_{n} − B(SB)^{−1}S) Ax.  (10.16)

The above system is sometimes referred to as the equivalent system. Thus, we can describe the behavior of the system in sliding alternatively as

\begin{matrix}ẋ & = & (I_{n} − B(SB)^{−1}S) Ax \\ Sx & = & 0. \end{matrix} \}  (10.17)

Note that in sliding, the system is governed by a reduced set of differential equations that are obtained by combining the above two sets of equations.

10.5 Sliding Surface Design

We give a sufficient condition for the existence of a sliding surface σ(x) = Sx = 0 such that the plant model ẋ = Ax + Bu restricted to it has arbitrarily prescribed poles. In our discussion, we use the following notation. For a full column rank matrix X, its left inverse will be denoted X^{g}. Thus, X^{g}X = I, where I is the identity matrix whose size is equal to the number of columns of X. The imaginary part of a complex number z is denoted ℑ(z). The set of eigenvalues of a square matrix A, also called its spectrum, is denoted eig(A).

We now state and prove a theorem that can be found in [148]. A general method for designing a sliding surface is contained implicitly in the proof of this theorem.

Theorem 10.1 Consider the plant model

ẋ = Ax + Bu,  (10.18)

where x(t) ∈ ℝ^{n}, u(t) ∈ ℝ^{m}, rank (B) = m, and the pair (A, B) is controllable. Let

{λ_{1}, λ_{2}, …, λ_{n−m}} ∈ ℂ,

where ℂ denotes the set of complex numbers, be an arbitrary set of n − m complex numbers such that any λ_{i} with ℑ(λ_{i}) ≠ 0 appears in this set in conjugate pair. Then, there exist full rank matrices W ∈ ℝ^{n×(n−m)} and W^{g} ∈ ℝ^{(n−m)×n} so that W^{g}W = I_{n−m}, W^{g}B = O, and the spectrum of the matrix W^{g}AW is

eig(W^{g}AW) = {λ_{1}, λ_{2}, …, λ_{n−m}}.

Furthermore, there exists a matrix S ∈ ℝ^{m×n} such that SB = I_{m} and the system (10.18) restricted to the surface Sx = 0 has the poles λ_{1}, λ_{2}, …, λ_{n−m}.

Proof We first prove the existence of the desired matrices W and W^{g}. We can find T ∈ ℝ^{n×(n−m)} so that [ T B ] is invertible because B has full rank. Let

[ T B ]^{−1} = [ \begin{matrix}T & g \\ B & g \end{matrix} ] .

We introduce a new coordinate z such that x = [ T B ] z. Then, in the new coordinates the system ẋ = Ax + Bu has the form

ż = [ \begin{matrix}T^{g}AT & T^{g}AB \\ B^{g}AT & B^{g}AB \end{matrix} ] z + [ \begin{matrix}O \\ I & m \end{matrix} ] u.

Because the pair (A, B) is controllable, we can use the PBH rank test, given on page 228, to conclude that

rank [ λI_{n−m} − T^{g}AT T^{g}AB ] = n − m for all λ ∈ ℂ,

which means that the pair (T^{g}AT , T^{g}AB) is controllable. The set {λ_{1}, λ_{2}, …, λ_{n−m}} is symmetric with respect to the real axis, therefore we can find a matrix F ∈ ℝ^{m×(n−m)} such that

eig (T^{g}AT − T^{g}ABF) = {λ_{1}, λ_{2}, …, λ_{n−m}}.

Let

W^{g} = T^{g}  (10.19)

and

W = T − BF .  (10.20)

Then, because by construction T^{g}T = I_{n−m} and T^{g}B = O, we have W^{g}W = I_{n−m} and W^{g}B = O. Note that

W^{g}AW = T^{g}AT − T^{g}ABF .

Hence,

eig (W^{g}AW) = eig (T^{g}AT − T^{g}ABF) = {λ_{1}, λ_{2}, …, λ_{n−m}}.

We next prove the existence of the sliding surface with the desired properties. We first note that the matrix [ W B ] is invertible because W^{g}W = I_{n−m}, W^{g}B = O, and W, B are full rank. Hence, Range(W) ∩ Range(B) = {0} and we write

[ W B ]^{−1} = [ \begin{matrix}W & g \\ S \end{matrix} ]  (10.21)

for some S ∈ ℝ^{m×n}. Note that SB = I_{m} and SW = O. We introduce new coordinates

z = [ \begin{matrix}z & 1 \\ z & 2 \end{matrix} ] = [ \begin{matrix}W & g \\ S \end{matrix} ] x,  (10.22)

where z_{2} = Sx. In the new coordinates, the equivalent system ẋ = (I_{n} − B(SB)^{−1}S) Ax takes the form

ż = [ \begin{matrix}W & g \\ S \end{matrix} ] (I_{n} − B(SB)^{−1}S) A [ W B ] z
= [ \begin{matrix}W^{g}AW & W^{g}AB \\ O & O \end{matrix} ] z.

The restriction to z_{2} = Sx = 0 is

ż_{1} = W^{g}AW z_{1},

and hence the system restricted to the sliding surface Sx = 0 has the same eigenvalues as the matrix W^{g}AW. The proof of the theorem is now complete.

□

554 CHAPTER 10. SLIDING MODES

Note that by (10.20),

[ W B ] = [ T B ] [ \begin{matrix}I_{n−m} & O \\ −F & I_{m} \end{matrix} ] .

Therefore,

[ W B ]^{−1} = [ \begin{matrix}I_{n−m} & O \\ −F & I_{m} \end{matrix} ]^{−1} [ T B ]^{−1}
= [ \begin{matrix}I_{n−m} & O \\ F & I_{m} \end{matrix} ] [ \begin{matrix}T & g \\ B & g \end{matrix} ] .  (10.23)

Thus the inverse of the matrix [ W B ] has the form

[ W B ]^{−1} = [ \begin{matrix}W & g \\ B^{g} + F T^{g} \end{matrix} ] ,

where we invoked (10.19), that is, W^{g} = T^{g}. Taking into account (10.21), we obtain

S = B^{g} + F T^{g}  (10.24)

We combine the above results into a sliding surface design algorithm.

Algorithm 10.1 Switching Surface Construction

1. Find a matrix T ∈ ℝ^{n×(n−m)} such that [ T B ] is invertible.

2. Compute

[ T B ]^{−1} = [ \begin{matrix}T & g \\ B & g \end{matrix} ] .

3. Find a matrix F ∈ ℝ^{m×(n−m)} such that eig (T^{g}AT − T^{g}ABF) = {λ_{1}, λ_{2}, …, λ_{n−m}}.

4. Construct the sliding surface matrix S using the relation, S = B^{g} + F T^{g}.

10.5. SLIDING SURFACE DESIGN 555

We illustrate the above algorithm with a simple numerical example. Example 10.4 Given a dynamical system model of the form (10.18), where

A = [ \begin{matrix}0 & 1 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} ] , B = [ \begin{matrix}0 & 0 \\ 1 & 0 \\ 0 & 1 \end{matrix} ] .

We design a sliding surface

σ = Sx = 0

so that the system restricted to it will behave like a linear system described by

ż = −z.

Following the steps of the of the design algorithm, we first find a matrix T such that the augmented matrix [ T B ] is invertible. For example, we can select

T = [ 1 0 0 ]^{⊤} .

Note that

[ T B ] = I_{3}.

Hence, the inverse

[ T B ]^{−1} = [ \begin{matrix}T & g \\ B & g \end{matrix} ] = I_{3}.

Next, we find F ∈ ℝ^{m×(n−m)} such that

T^{g}AT − T^{g}ABF = −1.

We have

T^{g}AT − T^{g}ABF = [ 1 0 0 ] [ \begin{matrix}0 & 1 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} ] [ \begin{matrix}1 \\ 0 \\ 0 \end{matrix} ]

− [ 1 0 0 ] [ \begin{matrix}0 & 1 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} ] [ \begin{matrix}0 & 0 \\ 1 & 0 \\ 0 & 1 \end{matrix} ] [ \begin{matrix}f_{11} \\ f_{21} \end{matrix} ]

= −f_{11}
= −1.

We select

F = [ \begin{matrix}1 \\ 0 \end{matrix} ] .

We then compute

S = B^{g} + F T^{g}
= [ \begin{matrix}0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} ] + [ \begin{matrix}1 \\ 0 \end{matrix} ] [ 1 0 0 ] .

Hence,

S = [ \begin{matrix}1 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} ] .

10.6 Sliding Mode and System Zeros

In this section, we show that the poles of the reduced-order system in sliding are the system zeros of the dynamical system described by

\{ \begin{matrix}ẋ & = & Ax + Bu \\ σ & = & Sx, \end{matrix}  (10.25)

where S ∈ ℝ^{m×n} and we assume that det SB ≠ 0. Recall from Section 1.5 that the system zeros, or invariant zeros, are defined using the system matrix. The system matrix, P (s), corresponding to the plant (10.25) has the form

P (s) = [ \begin{matrix}sI_{n} − A & −B \\ S & O \end{matrix} ] .  (10.26)

The system matrix P (s) corresponding to the plant (10.25) is a square matrix of order n+m. The zeros of the determinant of the system matrix (10.26) are the system zeros of (10.25). The system zeros are invariant under the following set of transformations:

1. nonsingular state variable transformation;

2. nonsingular transformation of the inputs;

3. nonsingular transformation of the outputs; 4. state-feedback to the inputs;

5. output-feedback to the rates of change of the states.

To proceed, we evaluate the determinant of the system matrix (10.26). To facilitate this task, we premultiply the system matrix by the matrix

[ \begin{matrix}I_{n} & O \\ −S(sI_{n} − A)^{−1} & I_{m} \end{matrix} ] .

The determinant of the above matrix is unity and so the determinant of the product is the same as the determinant of the system matrix,

det P (s) = det ([ \begin{matrix}I_{n} & O \\ −S(sI_{n} − A)^{−1} & I_{m} \end{matrix} ] [ \begin{matrix}sI_{n} − A & −B \\ S & O \end{matrix} ])
= det [ \begin{matrix}sI_{n} − A & −B \\ O & S(sI_{n} − A)^{−1}B \end{matrix} ]
= det (sI_{n} − A) det (S(sI_{n} − A)^{−1}B) .  (10.27)

We next evaluate the characteristic polynomial of the equivalent system (10.16). While performing manipulations, we use the result of Exercise A.4 to obtain

det (sI_{n} − A + B(SB)^{−1}SA)
= det(sI_{n} − A) det (I_{n} + (sI_{n} − A)^{−1}B(SB)^{−1}SA)
= det(sI_{n} − A) det (I_{m} + SA(sI_{n} − A)^{−1}B(SB)^{−1}) .  (10.28)

In our further manipulations, we use the identity

A (Adj(sI_{n} − A)) = s Adj(sI_{n} − A) − det(sI_{n} − A)I_{n}.  (10.29)

The above follows from the fact that

(sI_{n} − A)(sI_{n} − A)^{−1} = I_{n}.

which is equivalent to

(sI_{n} − A)Adj(sI_{n} − A) = det(sI_{n} − A)I_{n}.  (10.30)

Then we use (10.29) to get

det (I_{m} + SA(sI_{n} − A)^{−1}B(SB)^{−1})

= det (\frac{det(sI_{n} − A)I_{m} + S (s Adj(sI_{n} − A) − det(sI_{n} − A)I_{n}) B(SB)^{−1}}{det(sI_{n} − A)} )

= det (\frac{det(sI_{n} − A)I_{m} + sS Adj(sI_{n} − A) − det(sI_{n} − A)I_{n}(SB)(SB)^{−1}}{det(sI_{n} − A)} )

= det (\frac{sS Adj(sI_{n} − A)B(SB)^{−1}}{det(sI_{n} − A)} )

= s^{m} det (S(sI_{n} − A)^{−1}B) det(SB)^{−1}.  (10.31)

Using (10.31), we represent (10.28) as

det (sI_{n} − A + B(SB)^{−1}SA)
= s^{m} det(SB)^{−1} det(sI_{n} − A) det (S(sI_{n} − A)^{−1}B) .

Taking into account (10.27) gives

det (sI_{n} − A + B(SB)^{−1}SA) = det(SB)^{−1}s^{m} det P (s)

(10.32)

Thus the poles of the equivalent system (10.16) contain m poles at the origin and the remaining n − m poles are the system zeros of the system (10.25) characterized by the triple (A, B, S).

We conclude that the dynamics, that is, the poles, of the system modeled by (10.25) in sliding along σ = Sx are determined by its system zeros. This fact leads to the observation that the switching surface design can be viewed as choosing an output matrix S so that the system (10.25) has a desired set of system zeros locations, which in turn govern the dynamics of the system in sliding along Sx = 0.

10.7 Regulator Design

In this Section, we present a regulator design algorithm for a class of nonlinear dynamical systems using an approach that is rooted in variable structure sliding mode control.

This design methodology consists of two steps: the manifold design and the control law design. We use Algorithm 10.1 on page 554 to construct an asymptotically stable invariant manifold so that all trajectories confined to this manifold converge to the origin. Once a manifold is constructed, the control strategy is obtained by solving the first-order matrix differential equation. The proposed design procedure is tested on the model of a two-link robotic manipulator from Section 2.4.6.

10.7.1 Outline of the Approach

Consider a nonlinear dynamical system modeled as

ẋ = f(x) + B(x)u,  (10.33)

where x is a vector of state variables of dimension n, u is the input vector of dimension m, f(x) is a state-dependent function of dimension n, B(x) is a state-dependent input function matrix of dimension n × m and it has full column rank everywhere.

Let σ(x) = Sx : ℝ^{n} → ℝ^{m} be a vector function of dimension m.

The control objective is to devise a control policy that stabilizes the system (10.33) by steering its trajectories to a manifold,

M = {x : σ(x) = 0}  (10.34)

and forces the trajectories to stay on the manifold thereafter.

We further require that, (i) the system (10.33) confined to the manifold M = {x : σ = 0} is asymptotically stable to the origin, (ii) σ(x) = Sx is constructed so that the square m×m matrix

σx(x)B(x) = SB(x)

is invertible, where σx is the Jacobian matrix of σ(x) with respect to x. We will show that when these two conditions are satisfied, the solution to the above control problem is obtained by solving the associated first-order matrix differential equation,

T σ̇ + σ = 0,  (10.35)

where T ∈ ℝ^{m×m} is a symmetric positive definite matrix, which is a design parameter. Substituting (10.33) into (10.35) yields

T σ̇ + σ = T Sẋ + Sx
= T Sf(x) + T SB(x)u + Sx
= 0.

Solving the above gives

u = −(T SB(x))^{−1} (T Sf(x) + Sx)
= −(SB(x))^{−1}Sf(x) − (SB(x))^{−1}T^{−1}Sx.  (10.36)

The use of the control law (10.36) in the nonlinear dynamical systems (10.33) makes the invariant manifold M = {x : σ = 0} attractive. The control law (10.36) drives closed-loop trajectories of (10.33) from any initial condition towards the invariant manifold M. As stated above, σ is constructed in such a way that all trajectories of (10.33) confined on M converge to the origin. Therefore, the stability of the closed-loop trajectories is achieved in two modes: (i) closed-loop trajectories converging to the manifold M = {x : σ = 0}, and (ii) closed-loop trajectories moving along the manifold M = {x : σ = 0} towards the origin.

The design method employs the following two-step design procedure:

1. Construct an asymptotically stable invariant manifold M = {x : σ = 0}, (i.e. all trajectories of the system confined to M converge asymptotically to the origin).

2. Construct the controller u using (10.36).

This approach shares similarity with that of the variable structure sliding mode control (SMC). This relation was discussed in [294, 295, 254]. In the development of the proposed regulator, we adopt the integral tracking method used in the sliding mode tracking controller design described in [92]. The resulting controller drives the closed-loop dynamics so that the integral of the tracking error converges to zero.

10.7.2 System Description

The class of nonlinear systems that we consider is modeled by

ẋ = f(x) + B(x)u
= A(x)x + B̂ũ,  (10.37)

where we assume that the system (10.37) is transformable into the following “regular form” [333, 92],

[ \begin{matrix}ẋ & 1 \\ ẋ & 2 \end{matrix} ] = [ \begin{matrix}A_{11} & A_{12} \\ A_{21}(x) & A_{22}(x) \end{matrix} ] [ \begin{matrix}x & 1 \\ x & 2 \end{matrix} ] + [ \begin{matrix}O \\ B & u \end{matrix} ] ũ,  (10.38)

where x_{1} is in ℝ^{n−m}, x_{2} is in ℝ^{m}, and the control vector u is in ℝ^{m}. The matrix O ∈ ℝ^{(n−m)×m} is a matrix with zero entries. The matrix B_{u} ∈ ℝ^{m×m} is invertible and the matrices A_{ij}, i = 1, 2, j = 1, 2 have appropriate dimensions. Note that A_{1j}, j = 1, 2, are constant matrices, while A_{2j}, j = 1, 2, are state-dependent matrices. For a constructive algorithm that can be used to transform (10.37) into the regular form given by (10.38), we refer to [92, p. 66].

An example of physical systems that can be modeled using the above non-linear regular form are natural Lagrangian systems. A simple example of a natural Lagrangian system whose model can be represented in the regular form is a planar two-link robot presented in Section 2.4.6. Indeed, let x_{1} = [θ_{1} θ_{2}]^{⊤} be the vector of joint angles, τ = [τ_{1} τ_{2}]^{⊤} be the vector of torques applied to the joints. Then the dynamics of the two-link robotic manipulator can be represented as

H(x_{1})ẍ_{1} + C(x_{1}, ẋ_{1})ẋ_{1} + g(x_{1}) = τ ,  (10.39)

where H(x_{1}) is the manipulator inertia matrix and C(x_{1}, ẋ_{1})ẋ_{1} is the vector of centripetal and Coriolis torques, and g(x_{1}) = [ g_{1}(x_{1}) g_{2}(x_{1}) ]^{⊤} is the vector of gravitational torques. Because the inertia matrix H(x_{1}) is positive definite over the whole workspace, we know that H^{−1}(x_{1}) exists and is also positive definite. Let

G(x_{1}) = [ \begin{matrix}\frac{g_{1}(x_{1})}{x_{1}} & 0 \\ 0 & \frac{g_{2}(x_{1})}{x_{2}} \end{matrix} ] .

Using the above we represent the term g(x_{1}) as

g(x_{1}) = G(x_{1})x_{1}.  (10.40)

Taking into account (10.40), we rewrite (10.39) as

ẍ_{1} = −H^{−1}(x_{1})C(x_{1}, ẋ_{1})ẋ_{1} − H^{−1}(x_{1})G(x_{1})x_{1} + H^{−1}(x_{1})τ .  (10.41)

Let x_{2} = ẋ_{1} and ũ = H^{−1}(x_{1})τ. Then, the dynamics of the planar two-link robot modeled by (10.41) can be represented in the form of (10.38), that is,

[ \begin{matrix}ẋ & 1 \\ ẋ & 2 \end{matrix} ] = [ \begin{matrix}O & I_{2} \\ −H^{−1}(x_{1})G(x_{1}) & −H^{−1}(x_{1})C(x_{1}, x_{2}) \end{matrix} ] [ \begin{matrix}x & 1 \\ x & 2 \end{matrix} ] + [ \begin{matrix}O \\ I & 2 \end{matrix} ] ũ.  (10.42)

10.7.3 Regulator Construction

Suppose that the output y ∈ ℝ^{p} of system (10.38) has the form,

y = [ C_{1} C_{2} ] [ \begin{matrix}x & 1 \\ x & 2 \end{matrix} ] .  (10.43)

Our objective is to design a regulator so that the output y tracks a constant reference vector r ∈ ℝ^{p}. We introduce the vector variable x_{r} defined by the following differential equation,

ẋ_{r} = r − y.  (10.44)

The controller should be designed in such a way so that

lim_{t→∞} ẋ_{r}(t) = 0.

To proceed, we define the augmented state vector,

x̃ = [ \begin{matrix}x & r \\ x \end{matrix} ] = [ \frac{x_{r}}{\underbrace{\{ \begin{matrix}x & 1 \\ x & 2 \end{matrix} \}}_{x}} ] ∈ ℝ^{N},  (10.45)

where N = n + p. The dynamics of the overall augmented system are governed by the state equation,

[ \begin{matrix}ẋ & r \\ ẋ & 1 \\ ẋ & 2 \end{matrix} ] = \underbrace{[ \begin{matrix}O & −C_{1} & −C_{2} \\ O & A_{11} & A_{12} \\ O & A_{21}(x) & A_{22}(x) \end{matrix} ]}_{A^{}}˜[ \begin{matrix}x & r \\ x & 1 \\ x & 2 \end{matrix} ] + \underbrace{[ \begin{matrix}I & p \\ O \\ O \end{matrix} ]}_{B_{r}} r + \underbrace{[ \begin{matrix}O \\ O \\ B & u \end{matrix} ]}_{B^{}}˜  u.  (10.46)

Note that the matrices in the augmented state equation (10.46) are labeled as Ã, B_{r}, and B̃, respectively. Hence, the augmented system model (10.46) can be represented as

˙x̃ = Ãx̃ + B_{r}r + B̃u.  (10.47)

We make the following assumption:

Assumption 1: The pair

([ \begin{matrix}O & −C_{1} \\ O & A_{11} \end{matrix} ] , [ \begin{matrix}−C_{2}B_{u} \\ A & 12 & B & u \end{matrix} ])

is controllable.

An essential element of the regulator design is the construction of an invariant manifold σ = 0. Let the manifold function σ be defined as

σ = σ(x̃) = σ(x_{r}, x_{1}, x_{2}) = −S_{r}x_{r} + S_{1}x_{1} + S_{2}x_{2} = [ −S_{r} S_{1} S_{2} ] x̃ = S̃x̃. (10.48)

The negative sign in front of S_{r} is for the purpose of simplified notation. We construct σ so that the system modeled by (10.46) restricted to the manifold M = {x̃ : σ(x̃) = 0} is asymptotically stable, that is, the trajectories of the system restricted to the manifold M converge asymptotically towards the origin. We can use Algorithm 10.1 to construct such a manifold. We assume that

det (S̃B̃) = det (S_{2}B_{u}) ≠ 0.  (10.49)

Using (10.35), we perform manipulations to obtain,

0 = T σ̇ + σ
= T S̃ ˙x̃ + S̃x̃
= T S̃(Ãx̃ + B_{r}r + B̃ũ) + S̃x̃
= T S̃Ãx̃ + T S̃B_{r}r + T S̃B̃ũ + S̃x̃.  (10.50)

It follows from (10.48) and (10.46) that S̃B_{r} = −S_{r}. Hence, we can re-write (10.50) as

0 = T S̃Ãx̃ − T S̃_{r}r + T S̃B̃ũ + S̃x̃.  (10.51)

Solving (10.51) for u yields

ũ = −(S̃B̃)^{−1}(S̃Ãx̃ + T^{−1}S̃x̃ − S_{r}r)  (10.52)

Substituting (10.52) into (10.46), we obtain the closed-loop system dynamics model,

˙x̃ = (I − B̃(S̃B̃)^{−1}S̃) Ãx̃ − B̃(S̃B̃)^{−1}T^{−1}S̃x̃ + B̃(S̃B̃)^{−1}S_{r}r + B_{r}r.  (10.53)

In the following subsection, we show that the control law given by (10.52) forces the above closed-loop system to track the reference signal r.

10.7.4 Analysis of the Closed-Loop System Behavior

To proceed with our analysis, we need the following lemma, which can be viewed as an extension of Theorem 10.1 to a class of non-liner systems.

Lemma 10.1 Consider the system model given by (10.46), where rank (B̃) = m, Assumption 1 is satisfied, and a given set of N − m complex numbers, {λ_{1}, …, λ_{N−m}} is such that any λ_{i} whose imaginary part is nonzero appears in the above set in a conjugate pair. Then, there exist full-rank matrices W ∈ ℝ^{N×(N−m)} and W^{g} ∈ ℝ^{(N−m)×N} such that

W^{g}W = I_{N−m}, W^{g}B̃ = O,

and the eigenvalues of W^{g}AW are

eig (W^{g}ÃW) = {λ_{1}, …, λ_{N−m}}.

Proof: The proof of this lemma is based on the proof of Theorem 10.1 with an emphasis on the special structure of the non-linear plant. We begin our proof by defining an N ×(N −m) real matrix of the form,

V = [ \frac{I_{N−m}}{O} ] .  (10.54)

Therefore,

[ V B̃ ]^{−1} = [ \begin{matrix}I_{N−m} & O \\ O & B_{u} \end{matrix} ]^{−1} = [ \begin{matrix}I_{N−m} & O \\ O & B_{u}^{−1} \end{matrix} ] = [ \begin{matrix}V & g \\ B̃ & g \end{matrix} ] .

Hence,

[ \begin{matrix}V & g \\ B̃ & g \end{matrix} ] [ V B̃ ] = [ \begin{matrix}V^{g}V & V^{g}B̃ \\ B̃^{g}V & B̃^{g}B̃ \end{matrix} ] = [ \begin{matrix}I_{N−m} & O \\ O & I_{m} \end{matrix} ] = I_{N}.

Note that

V^{g}V = I_{N−m} and V^{g}B̃ = O.  (10.55)

By Assumptions 1, the pair

(V^{g}ÃV , V^{g}ÃB̃) = ([ \begin{matrix}O & −C_{1} \\ O & A_{11} \end{matrix} ] , [ \begin{matrix}−C_{2}B_{u} \\ A & 12 & B & u \end{matrix} ])

is controllable. Therefore, for any set of complex numbers,

{λ_{1}, λ_{2}, …, λ_{N−m}}

symmetric with respect to the real axis, we can always find a matrix F such that

{λ_{1}, λ_{2}, …, λ_{N−m}} = eig (V^{g}ÃV − V^{g}ÃB̃F ) .

The matrix F can be determined using any available method for pole placement. We proceed by performing the following manipulations,

V^{g}ÃV − V^{g}ÃB̃F = V^{g}Ã (V − B̃F ) = W^{g}ÃW ,  (10.56)

where W^{g} = V^{g} and W = V − B̃F. Note that by (10.55),

W^{g}W = V^{g} (V − B̃F ) = V^{g}V = I_{N−m}.  (10.57)

We also have, V^{g}B̃ = W^{g}B̃ = O, which completes the proof of the lemma.

□

Theorem 10.2 Suppose that the assumptions of Lemma 10.1 are satisfied. Then there exists an invariant manifold matrix S̃ ∈ ℝ^{m×N} such that S̃B̃ = I_{m} and the closed-loop system (10.53) asymptotically tracks the reference signal r, that is, y(∞) = r.

Proof: Consider the matrix, [ W B̃ ]. Note that because W = V − B̃F, we have

[ W B̃ ] = [ V B̃ ] [ \begin{matrix}I_{N−m} & O \\ −F & I_{m} \end{matrix} ] .

Thus the matrix [ W B̃ ] is invertible and its inverse has the form

[ W B̃ ]^{−1} = [ \begin{matrix}I_{N−m} & O \\ −F & I_{m} \end{matrix} ]^{−1} [ V B̃ ]^{−1}
= [ \begin{matrix}I_{N−m} & O \\ F & I_{m} \end{matrix} ] [ \begin{matrix}V & g \\ B̃ & g \end{matrix} ]
= [ \begin{matrix}V & g \\ B̃^{g} + F V^{g} \end{matrix} ] .

As in (10.24) in the proof of Theorem 10.1, we take

S̃ = B̃^{g} + F V^{g}
= [ O O B_{u} ] + F [ I_{N−m} O ]
= [ F B_{u}^{−1} ] .

Note that by (10.55),

S̃B̃ = [ F B_{u}^{−1} ] [ \begin{matrix}O \\ O \\ B & u \end{matrix} ] = I_{m}.

Consider the following state-space transformation,

z = [ \begin{matrix}z & 1 \\ z & 2 \end{matrix} ] = [ W^{g} ; S̃ ] x̃ = [ V^{g} ; S̃ ] x̃.

Note that z_{2} = S̃x̃ = σ. We also have

[ W^{g} ; S̃ ] [ W B̃ ] = [ \begin{matrix}W^{g}W & W^{g}B̃ \\ S̃W & S̃B̃ \end{matrix} ] = [ \begin{matrix}I_{N−m} & O \\ O & I_{m} \end{matrix} ] ,  (10.58)

where I_{N−m} is the identity matrix of dimension (N −m)×(N −m). Therefore, the matrices in (10.58) satisfy the following relations,

\begin{aligned}W^{g}W & = & I_{N−m}, \\ S̃W & = & O ∈ ℝ^{m×(N−m)}, \end{aligned} \begin{aligned}W^{g}B̃ & = & O ∈ ℝ^{(N−m)×m}, \\ S̃B̃ & = & I_{m}. \end{aligned} \}  (10.59)

Using matrix relations shown in (10.59), we represent the closed-loop model (10.53) in the z coordinates,

[ \begin{matrix}ż & 1 \\ ż & 2 \end{matrix} ] = [ W^{g} ; S̃ ] (Ã − B̃(S̃B̃)^{−1}S̃Ã − B̃(S̃B̃)^{−1}T^{−1}S̃) [ W B̃ ] [ \begin{matrix}z & 1 \\ z & 2 \end{matrix} ]

− [ W^{g} ; S̃ ] B̃(S̃B̃)^{−1}S̃B_{r}r + [ W^{g} ; S̃ ] B_{r}r

= [ \begin{matrix}W^{g}ÃW & W^{g}ÃB̃ \\ O & −T^{−1} \end{matrix} ] [ \begin{matrix}z & 1 \\ z & 2 \end{matrix} ] + [ W^{g} ; S̃ ] (−B̃(S̃B̃)^{−1}S̃B_{r} + B_{r}) r
= [ \begin{matrix}W^{g}ÃW & W^{g}ÃB̃ \\ O & −T^{−1} \end{matrix} ] [ \begin{matrix}z & 1 \\ z & 2 \end{matrix} ] + [ \begin{matrix}O + W^{g}B_{r} \\ −S̃B_{r} + S̃B_{r} \end{matrix} ] r
= [ \begin{matrix}W^{g}ÃW & W^{g}ÃB̃ \\ O & −T^{−1} \end{matrix} ] [ \begin{matrix}z & 1 \\ z & 2 \end{matrix} ] + [ \begin{matrix}W & g & B & r \\ O \end{matrix} ] r.  (10.60)

Observe that W^{g}B̃(S̃B̃)^{−1}S̃B_{r} = O because W^{g}B̃ = O. Since T is positive definite, the eigenvalues of −T^{−1} are all in the open left-hand complex plane. This implies that the dynamics of z_{2} = σ are asymptotically stable, that is, the invariant manifold, {σ = 0}, is asymptotically attractive. By (10.56), the matrix W can be chosen so that the eigenvalues of the constant matrix W^{g}ÃW are in the open left-hand complex plane. In the steady-state the overall system trajectories converge to the point on the manifold, {z_{2} = σ = 0}, given by

0 = W^{g}ÃW z_{1} + W^{g}B_{r}r.  (10.61)

We partition F as

F = [ F_{1} F_{2} ] ,  (10.62)

where F_{1} ∈ ℝ^{m×p} and F_{2} ∈ ℝ^{m×(n−m)}. We have,

W = V − B̃F = [ \begin{matrix}I_{p} & O \\ O & I_{n−m} \\ −B_{u}F_{1} & −B_{u}F_{2} \end{matrix} ] and W^{g}B_{r} = [ \begin{matrix}I & p \\ O & (n−m)×p \end{matrix} ] .  (10.63)

10.7. REGULATOR DESIGN 567 Recall that W^{g} = V^{g}. Using the above and the fact that

z_{1} = W^{g}x̃ = [ \begin{matrix}x & r \\ x & 1 \end{matrix} ] ,

we express (10.61) in the form,

0 = V^{g} [ \begin{matrix}O & −C_{1} & −C_{2} \\ O & A_{11} & A_{12} \\ O & A_{21}(x) & A_{22}(x) \end{matrix} ] W z_{1} + [ \begin{matrix}I & p \\ O \end{matrix} ] r
= [ \begin{matrix}C_{2}B_{u}F_{1} & −C_{1} + C_{2}B_{u}F_{2} \\ −A_{12}B_{u}F_{1} & A_{11} − A_{12}B_{u}F_{2} \end{matrix} ] W^{g}x̃ + [ \begin{matrix}I & p \\ O \end{matrix} ] r
= [ \begin{matrix}C_{2}B_{u}F_{1} & −C_{1} + C_{2}B_{u}F_{2} \\ −A_{12}B_{u}F_{1} & A_{11} − A_{12}B_{u}F_{2} \end{matrix} ] [ \begin{matrix}x & r \\ x & 1 \end{matrix} ] + [ \begin{matrix}I & p \\ O \end{matrix} ] r,

or, equivalently,

0 = C_{2}B_{u}F_{1}x_{r} − (C_{1} − C_{2}B_{u}F_{2}) x_{1} + r,  (10.64)
0 = −A_{12}B_{u}F_{1}x_{r} + (A_{11} − A_{12}B_{u}F_{2}) x_{1}.  (10.65)

In the steady-state, z_{2} = 0, that is,

S̃x̃ = [ −S_{r} S_{1} S_{2} ] [ \begin{matrix}x & r \\ x & 1 \\ x & 2 \end{matrix} ]
= [ F_{1} F_{2} B_{u}^{−1} ] [ \begin{matrix}x & r \\ x & 1 \\ x & 2 \end{matrix} ]
= 0.

Hence,

S_{r}x_{r} = S_{1}x_{1} + S_{2}x_{2}.  (10.66)

Substituting S_{2} = B_{u}^{−1} and (10.66) into (10.64), we obtain

0 = −C_{2}B_{u}S_{r}x_{r} − C_{1}x_{1} + C_{2}B_{u}S_{1}x_{1} + r
= C_{2}B_{u}(−S_{1}x_{1} − S_{2}x_{2}) − C_{1}x_{1} + C_{2}B_{u}S_{1}x_{1} + r
= −C_{2}x_{2} − C_{1}x_{1} + r
= −y + r,

that is, y(∞) = r, which completes the proof of the theorem.

□

10.7.5 Example: Two-link robotic manipulator

We synthesize a regulator for the two-link robotic manipulator depicted in Figure 2.22 in Section 2.4.6. Recall that the Lagrangian has the form,

L = \frac{1}{2}m_{1}l_{1}^{2}ω_{1}^{2} + \frac{1}{2}m_{2} (l_{1}^{2}ω_{1}^{2} + l_{2}^{2}ω_{2}^{2} + 2l_{1}l_{2} cos (θ_{2} − θ_{1}) ω_{1}ω_{2})
+m_{1}gl_{1} cos(θ_{1}) + m_{2}g (l_{1} cos(θ_{1}) + l_{2} cos(θ_{2})) ,

where ω_{i} = θ̇_{i}, i = 1, 2, are the angular velocities of the links, and m_{i} and l_{i} are the link masses and their lengths, respectively. We assume that the input torques, τ_{1} and τ_{2} are applied at each joint. The parameter values are given in Table 10.1. The Lagrange equations of motion

Table 10.1: Parameter numerical values of the two-link robotic manipulator.

\begin{matrix}m_{1} = m_{2} & 1 kg \\ l_{1} = l_{2} & 0.5 m \\ g & 9.81 m/sec^{2} \end{matrix}

are,

[ \begin{matrix}(m_{1} + m_{2})l_{1}^{2} & m_{2}l_{1}l_{2} cos(θ_{2} − θ_{1}) \\ m_{2}l_{1}l_{2} cos(θ_{2} − θ_{1}) & m_{2}l_{2}^{2} \end{matrix} ] [ θ̈_{1} ; θ̈_{2} ]
= [ \begin{matrix}τ_{1} − τ_{2} + m_{2}l_{1}l_{2}θ̇_{2}^{2} sin(θ_{2} − θ_{1}) − (m_{1} + m_{2})gl_{1} sin(θ_{1}) \\ τ_{2} − m_{2}l_{1}l_{2}θ̇_{1}^{2} sin(θ_{2} − θ_{1}) − m_{2}gl_{2} sin(θ_{2}) \end{matrix} ] . (10.67)

Note that the manipulator inertia matrix, H, on the left-hand side of the above equation is symmetric and positive definite, and its determinant is

det H = (m_{1} + m_{2})m_{2}l_{1}^{2}l_{2}^{2} − m_{2}^{2}l_{1}^{2}l_{2}^{2} cos^{2}(θ_{2} − θ_{1}) > 0.

We represent the above model in state-space format, where the state variables are defined as, x_{1} = θ_{1}, x_{2} = θ_{2}, x_{3} = θ̇_{1}, and x_{4} = θ̇_{2}. The outputs are

y_{1} = x_{1} and y_{2} = x_{2}.

Therefore, C_{1} = I_{2} and C_{2} = O. Using the above notation, we represent (10.67) as

H(x_{1})ẍ_{1} = −C(x_{1}, ẋ_{1})ẋ_{1} − g(x_{1}) + B_{u}τ ,

where

B_{u} = [ \begin{matrix}1 & −1 \\ 0 & 1 \end{matrix} ] ,

and

g(x_{1}) = G(x_{1})x_{1} = [ \begin{matrix}−(m_{1} + m_{2})gl_{1} \frac{sin(x_{1})}{x_{1}} & 0 \\ 0 & −m_{2}gl_{2} \frac{sin(x_{2})}{x_{2}} \end{matrix} ] x_{1}.

We represent the control ũ as in (10.42), where

ũ = H^{−1}B_{u}τ .

We select

T = 0.05I_{2},

and

{λ_{1}, λ_{2}, λ_{3}, λ_{4}} = {−3, −3.5, −4, −4.5}.

Applying the design algorithm, we obtain

S̃ = [ \begin{matrix}−18. & 0 & 8.5 & 0 & 1 & 0 \\ 0 & −10.5 & 0 & 6.5 & 0 & 1 \end{matrix} ] .

With the above data, we construct the regulator (10.52). In our simulations, we select the initial conditions to be,

x_{r}(0) = 0 and x(0) = [ −3/4π π 0 0 ]^{⊤} .

The components of the reference signal vector r are plotted in Figures 10.12 and 10.13, where we show plots of link angles θ_{1} and θ_{2} versus time. In Figures 10.14 and 10.15, we show plots of the time history of the x and y coordinates, respectively, of the end effector of the robotic manipulator. In Figure 10.16, we plot the path of the end effector in the (x, y) plane. As can be seen from the figures the reference signal is tracked as predicted by our analysis.

10.8 State Estimation of Uncertain Systems

So far in our discussion of variable structure sliding mode control system design we assumed that the entire state vector of the plant to be controlled is available to us through measurement. If the state vector cannot be measured, then we can try to determine its suitable 570 CHAPTER 10. SLIDING MODES

Figure 10.12: A plot of θ_{1} and its reference versus time. Figure 10.13: A plot of θ_{2} and its reference versus time.

10.8. STATE ESTIMATION OF UNCERTAIN SYSTEMS 571 Figure 10.14: A plot of the x-coordinate of the end effector versus time. Figure 10.15: A plot of the y-coordinate of the end effector versus time. Figure 10.16: A plot of the path of the end effector in the (x, y)-plane. estimate that can be used by the controller instead of the true state vector. If we use a state vector estimate for the inaccessible state, then we split a control design problem into three phases. The first phase is the design of a controller assuming the availability of the entire state vector of the plant. In the second phase, we construct an observer based on the plant input and output. Finally, in the third phase, we combine the controller and the observer into the combined controller-observer compensator, where the controller uses the plant state estimate instead of the actual state.

We discuss the problem of constructing a state estimator for a class of dynamical systems with matched uncertainties modeled by

ẋ = Ax + B (u + ξ)
y = Cx,

where C ∈ ℝ^{p×n}, and p ≥ m, which means that the number of input channels does not exceed the number of output channels. The vector function ξ models the lumped uncertainties or nonlinearities in the plant. We assume that there is a nonnegative real-valued function ρ = ρ (y) such that

∥ξ∥ ≤ ρ.

We assume that the pair (A, C) is observable. Our goal is to construct the plant state estimator. Let x̂ be an estimate of x. Let e denote the estimation error, that is,

e(t) = x(t) − x̂(t).

The observability of (A, C) implies the existence of a matrix L ∈ ℝ^{n×p} such that A − LC has prescribed (symmetric with respect to the real axis) eigenvalues in the open left-half plane. Because A − LC is asymptotically stable, by the Lyapunov theorem, for any Q = Q^{⊤} > 0, there is a unique P = P^{⊤} > 0 such that

(A − LC)^{⊤} P + P (A − LC) = −Q.  (10.68)

We choose Q, if possible, so that for some F ∈ ℝ^{m×p},

F C = B^{⊤}P .  (10.69)

We need this technical condition for the construction of the observer. This condition has an interesting system theoretic interpretation. Its satisfaction means that the transfer function matrix

G(s) = F C (sI − A + LC)^{−1} B

is positive real, that is, the elements of G(s) are analytic in the open right-half plane and G(s) + G^{∗}(s) ≥ 0 in the open right-half plane. Here the superscript asterisk denotes the conjugate transpose.

An efficient method, using linear matrix inequalities (LMIs), for finding matrices P, L, and F that satisfy (10.68) and (10.69) is presented in Section 12.4.4.

To proceed, define the vector function

E (e, η) = \{ \begin{matrix}\frac{F Ce}{∥F Ce∥}η & for & F Ce ≠ 0 \\ 0 & for & F Ce = 0, \end{matrix}

where η ≥ 0 is a design parameter. Note that if we have a single-input single-output plant, then we can write

E(e, η) = η sign (F Ce) .

We now introduce a sliding mode observer described in [339]. It has the form,

˙x̂ = (A − LC) x̂ + BE(e, η) + Ly + Bu  (10.70)

If η ≥ ρ, the function x̂ is an asymptotic estimate of the state x, that is,

lim_{t→∞} e(t) = lim_{t→∞} (x(t) − x̂(t)) = 0.

To prove the above statement using Lyapunov’s type of arguments, see Exercise 10.9, construct the differential equation satisfied by the estimation error e,

ė = ẋ − ˙ x̂ = (A − LC) e − BE(e, η) + Bξ.  (10.71)

Then, show that

\frac{d}{dt} (e^{⊤}P e) = −e^{⊤}Qe < 0,

which implies

lim_{t→∞} e(t) = 0.

It follows from the above that the estimation error is insensitive to matched uncertainty. A generalization of the sliding mode observer (10.71) is presented in Chapter 12.

10.9 Notes

The idea of controlling a dynamical system through varying its structure was originated in the 1950s. Early pioneers in the field of variable structure systems were S. V. Emelyanov and his co-workers in the Institute of Control Sciences in Moscow in the former Soviet Union. A large body of literature was produced by these researches, unfortunately mostly published in the Russian only. The first widely accessible books in English on variable structure sliding mode systems were books by Itkis [159] and Utkin [332]. A survey paper by Utkin [331] references many early contributions to the field; see also tutorials [77, 76] and a survey by Hung, Gao, and Hung [152]. At about the same time, A. F. Filippov was developing his theory of differential equations with discontinuous right-hand sides [102]. Filippov’s approach is now accepted as the tool for the analysis of sliding mode systems. Sliding modes also found applications in the state estimation of dynamical uncertain systems as we show in Section 10.8. For further analysis of sliding mode observers, we refer the reader to Edwards and Spurgeon [90, 92]. For a stability analysis of uncertain closed-loop systems with the combined variable structure controller-observer compensators we refer to [147]. Output feedback sliding mode controllers are presented in [83, 149, 366, 367, 200, 91].

Impressive applications of sliding modes to solving optimization and computational problems are reported by Bhaya and Kaszkurewicz [33]. Ignaciuk and Bartoszewicz [156], on the other hand, apply discrete-time sliding-mode controllers to congestion control of communication networks.

10.10 Exercises

Exercise 10.1 Consider the following variable structure system that was analyzed in [152] (see also [331] for a similar system):

\begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & −x_{1} + 2x_{2} + u, \end{matrix} \}  (10.72)

where

u = −4x_{1}signσ(x_{1}, x_{2}), σ(x_{1}, x_{2}) = x_{1} (0.5x_{1} + x_{2}) .

(i) Sketch phase plane portraits of two linear subsystems that constitute the given variable structure system (10.72).

(ii) Sketch a phase plane portrait of the variable structure system (10.72).

Exercise 10.2 (i) For the dynamical system model,

ẋ = [ \begin{matrix}1 & 1 \\ 2 & 1 \end{matrix} ] x + [ \begin{matrix}1 \\ −1 \end{matrix} ] u,

10.10. EXERCISES 575

construct the switching surface

σ(x) = sx = 0

such that the system in sliding along the surface has its pole at −2. (ii) Consider the following model of a dynamical system,

ẋ = [ \begin{matrix}2 & 0 & −1 \\ 0 & −2 & 2 \\ −1 & −2 & 3 \end{matrix} ] x + [ \begin{matrix}1 \\ 1 \\ 1 \end{matrix} ] u.

Design a switching surface σ(x) = sx = 0 such that the system restricted to it has poles at −4, −5.

(iii) Given the following dynamical system model,

ẋ = [ \begin{matrix}2 & 0 & −1 \\ 0 & −2 & 2 \\ −1 & −2 & 3 \end{matrix} ] x + [ \begin{matrix}1 \\ 1 \\ 1 \end{matrix} ] u,

along with the switching surface

σ(x) = sx = [ 8 8 −15 ] x = 0.

Determine the reduced-order equations that describe the system behavior in a sliding mode.

Exercise 10.3 Consider a dynamical system modeled by the equations

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}0 & 1 \\ a_{1}(t) & a_{2}(t) \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ 1 \end{matrix} ] u,

where

|a_{1}(t)| ≤ 3, |a_{2}(t)| ≤ 4 for all t.

(i) Design a switching surface

σ(x_{1}, x_{2}) = s_{1}x_{1} + x_{2} = 0

such that the system restricted to it has its pole at −2.

(ii) Design a sliding mode controller

u = k_{1}^{±}x_{1} + k_{2}^{±}x_{2},

so that the switching surface σ = 0 is globally attractive. The gains k_{1} and k_{2} are functions of x_{1}σ and x_{2}σ, respectively.

Exercise 10.4 Consider the following model of an armature controlled DC motor controlling an inverted pendulum:

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ] = [ \begin{matrix}x_{2} \\ 9.8 sin x_{1} + x_{3} \\ −\frac{1}{L_{a}} x_{2} − \frac{1}{L_{a}} x_{3} \end{matrix} ] + [ 0 ; 0 ; \frac{1}{L_{a}} ] u
y = x_{1},

where L_{a} is the armature inductance of the motor.

(i) Derive the reduced-order model assuming L_{a} is negligible.

(ii) For the obtained reduced-order model from the first part, design the switching surface

σ = 0 of the form

σ = σ(x_{1}, x_{2}) = [ s_{1} 1 ] [ \begin{matrix}x_{1} \\ x_{2} \end{matrix} ] = 0

so that the system in sliding will have its pole at −3.

(iii) Use the above sliding surface to design a variable structure controller of the form

u = −k_{1}x_{1} sign(x_{1}σ) − k_{2}x_{2} sign(x_{2}σ).

The smaller the gains k_{1}, k_{2}, the better.

Exercise 10.5 Consider a dynamical system model,

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}A_{1} & A_{2} \\ A_{3} & A_{4} \end{matrix} ] [ \begin{matrix}x & 1 \\ x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ b \end{matrix} ] u  (10.73)
y = [ C_{1} C_{2} ] [ \begin{matrix}x & 1 \\ x_{2} \end{matrix} ] ,  (10.74)

where b and C_{2} are nonzero scalars. Let

σ = [ S_{1} S_{2} ] [ \begin{matrix}x & 1 \\ x_{2} \end{matrix} ] = F [ C_{1} C_{2} ] [ \begin{matrix}x & 1 \\ x_{2} \end{matrix} ] = F y,  (10.75)

where S_{2} and F are nonzero scalars.

(i) Find the reduced-order system that models (10.73) when restricted to the surface σ = 0.

(ii) Find the transfer function H(s) = L(y(t))/L(u(t)) = Y (s)/U(s). What is the relation between the zeros of H(s) and the poles of the reduced-order system found in part (i)?

Exercise 10.6 (based on [200]) Prove the following

Lemma 10.2 Let

ẋ = Ax + Bu,

where the matrix A has negative real distinct eigenvalues, that is,

eig (A) = { −λ_{1}, −λ_{2}, …, −λ_{n} } , λ_{i} > 0, λ_{i} ≠ λ_{j} for i ≠ j.

Let λ_{min} = min { λ_{1}, λ_{2}, …, λ_{n} } and K be a positive constant such that

‖e^{At}‖ ≤ Ke^{−λ_{min}t}

Then,

∥x(t)∥ ≤ w(t),

where w(t) is a solution of

ẇ = −λ_{min}w + K ∥B∥ ∥u∥ , subject to w(0) ≥ K ∥x(0)∥ .

Exercise 10.7 (based on [200]) Consider an n-th order model of an uncertain dynamical system,

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}A_{1} & A_{2} \\ A_{3} & A_{4} \end{matrix} ] [ \begin{matrix}x & 1 \\ x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ b \end{matrix} ] (u + ξ)  (10.76)
y = [ C_{1} C_{2} ] [ \begin{matrix}x & 1 \\ x_{2} \end{matrix} ] ,

where b and C_{2} are nonzero scalars and ξ models system’s uncertainty, where

ξ = D_{1}x_{1} + D_{2}x_{2} + D_{3}u + D_{4}(t).

In the above, D_{1} is a row vector whose size is equal to the dimension of x_{1} ∈ ℝ^{n−1}, D_{2} and D_{3} are scalars, where |D_{3}| < 1 and D_{4} is an unknown bounded function of time whose bound is known to us, that is, |D_{4}(t)| ≤ d_{4} where d_{4} is known to us. Let σ = F y be as defined in Exercise 10.5. Assume that the switching surface,

σ = F y = F [ C_{1} C_{2} ] [ \begin{matrix}x & 1 \\ x_{2} \end{matrix} ] = 0,

was chosen in such a way that the nominal system (10.76) restricted to it has the eigenvalues

{ −λ_{1}, −λ_{2}, …, −λ_{n−1} } , λ_{i} > 0, λ_{i} ≠ λ_{j} for i ≠ j.

(i) Find σ̇ evaluated on the trajectories of the uncertain system model (10.76).

(ii) For the controller

u = (S_{2}b)^{−1} (1 − |D_{3}|)^{−1} (−Hw(t) − G|σ| − δ) sign(σ),

where w(t) is a solution of a first-order differential equation such that ∥x_{1}(t)∥ ≤ w(t), find lower bounds for H, G and δ such that

σσ̇ ≤ −η|σ| for some η > 0. Use the lemma of Exercise 10.6 to find the form of the differential equation whose solution satisfies the condition ∥x_{1}(t)∥ ≤ w(t).

Exercise 10.8 For the dynamical system model

\begin{matrix}ẋ & = & Ax + b (u + f(y)) \\ y & = & cx, \end{matrix} \}

where

A = [ \begin{matrix}−2 & 2 \\ 0 & 1 \end{matrix} ] , b = [ \begin{matrix}0 \\ 1 \end{matrix} ] , c = [ 0 1 ] , and |f(y)| ≤ 10y,

design a sliding mode observer. The observer poles should be placed at −2 and −3. Hint: After constructing the required observer gain l, find a symmetric positive definite matrix Q such that the solution, P, to the Lyapunov matrix equation, (A−lc)^{⊤}P +P (A−lc) = −Q, satisfies the condition, b^{⊤}P = c. Exercise 10.9 Show that the estimation error e of the sliding mode observer (10.70) satisfies the equation

ė = ẋ − ˙ x̂ = (A − LC) e − BE(e, η) + Bξ.

Then show that

\frac{d}{dt} (e^{⊤}P e) = −e^{⊤}Qe < 0,

which implies that

lim_{t→∞} e(t) = 0.

Vector Field Methods

”I wrote this book without a book contract. I decided to just go write the book and then see if I could sell it. When you write a book without a book contract, you have to have a great deal of confidence in the topic and also a great deal of support at home.” Johanna Neuman in [202, page 235]

In this chapter, we discuss methods that use vector fields in the controller construction for a class of nonlinear dynamical systems. The control design process, as in the case of linear systems, involves three steps. The first step is the design of a state-feedback control law. The second step involves the design of an observer. The third step is to combine the first two steps to obtain a combined controller-observer compensator. We use the so-called feedback linearization method to construct a state-feedback control strategy in the first stage of the above design algorithm. The basic idea of the feedback linearization approach is to use a control consisting of two components, one of which cancels out the plant’s nonlinearities and the other controls the resulting linear system. The method is limited to a class of dynamical systems, plants, whose models are sufficiently smooth, that is, the plant models whose righthand sides of the modeling differential equations are sufficiently many times differentiable. In the next section, we acquaint ourselves with some basic notions from differential geometry that we will be using in our further analysis. We then describe the class of plants for which the feedback linearization method is applicable.

11.1 Background Results From Differential Geometry

All functions that we encounter in this Chapter are assumed to have continuous partial derivatives to any order, that is, the functions that we work with are smooth.

Definition 11.1 A vector-valued function of n variables,

f : D → ℝ^{n}, where D ⊆ ℝ^{n} is a domain,

is called a vector field on D. A vector field is an n-dimensional column.

◇

Definition 11.2 The transpose of a vector field is called a covector field. A covector field is an n-dimensional row.

◇

Definition 11.3 A differential of a real-valued function of n variables

h : D → ℝ, where D ⊆ ℝ^{n} is a domain,

is a covector field, denoted dh, and defined as

dh = Dh = ∇xh^{⊤} = [ \frac{∂h}{∂x_{1}} \frac{∂h}{∂x_{2}} ⋯ \frac{∂h}{∂x_{n}} ] ,

where ∇xh is the gradient of h, which is a vector field.

◇

In our controller construction for a class of nonlinear systems, we use three types of Lie derivatives. They are as follows:

1. Derivative of a vector field with respect to a vector field, also known as the Lie bracket. Given the vector valued functions f : ℝ^{n} → ℝ^{n} and g : ℝ^{n} → ℝ^{n}, where f and g are C^{∞} vector fields. Their Lie bracket is defined as

[f, g] = \frac{∂f}{∂x}g − \frac{∂g}{∂x}f,  (11.1)

where \frac{∂f}{∂x} and \frac{∂g}{∂x} are the Jacobian matrices of f and g, respectively. Sometimes in the literature the negative of the above definition is used. For convenience we also use an alternative notation of the form

[f, g] = (ad^{1}f, g).  (11.2)

We define

(ad^{k}f, g) = [f, (ad^{k−1}f, g)],

where

(ad^{0}f, g) = g.

For a single input linear system model, f(x) = Ax and g(x) = b. Hence,

(ad^{0}f, g) = b
(ad^{1}f, g) = Ab
⋮
(ad^{k}f, g) = A^{k}b.

Thus, the controllability matrix of a single-input linear system can be expressed in terms of Lie brackets as

[ b Ab ⋯ A^{n−1}b ] = [ (ad^{0}f, g) (ad^{1}f, g) ⋯ (ad^{n−1}f, g) ] .

We call the n × n matrix

[ (ad^{0}f, g) (ad^{1}f, g) ⋯ (ad^{n−1}f, g) ] ,  (11.3)

the controllability matrix of the single-input nonlinear system modeled by

ẋ = f(x) + g(x)u.  (11.4)

2. Derivative of a function with respect to a vector field. Let h : ℝ^{n} → ℝ be a C^{∞} function on ℝ^{n}. Let Dh = ∇h^{⊤}, where ∇h is the gradient (a column vector) of h with respect to x. Then, the Lie derivative of the function h with respect to the vector field f, denoted Lf h or Lf (h), is defined as

Lf h = Lf (h) = ⟨∇h, f⟩ = Dh f = \frac{∂h}{∂x_{1}} f_{1} + \frac{∂h}{∂x_{2}} f_{2} + ⋯ + \frac{∂h}{∂x_{n}} f_{n}.

We use the following notation:

Lf^{0} h = h
Lf^{1} h = Lf h
Lf^{2} h = Lf Lf h
⋮
Lf^{k} h = Lf Lf^{k−1}h.

3. Derivative of Dh with respect to the vector field.

The Lie derivative of Dh with respect to the vector field f, denoted Lf (Dh), is defined as

Lf (Dh) = (\frac{∂∇h}{∂x}f)^{⊤} + Dh\frac{∂f}{∂x} .  (11.5)

Note that

Lf (Dh) = DLf h = ∇Lf^{⊤} h.  (11.6)

The above three Lie derivatives satisfy the Leibniz formula,

L_{[f ,g]}h = ⟨∇h, [f, g]⟩ = LgLf h − Lf Lgh  (11.7)

11.2 Distributions

In this section, we present notions of a distribution and codistribution that we use in formulating sufficient conditions for a nonlinear model to be transformable into the controller form.

Definition 11.4 Let f_{1}, f_{2}, …, f_{r} be vector fields on D ⊆ ℝ^{n}. Then, for any x ∈ D, f_{1}(x), f_{2}(x), …, f_{r}(x) are vectors in ℝ^{n} and

Δ(x) = span{f_{1}(x), f_{2}(x), …, f_{r}(x)}

is a subspace of ℝ^{n}. To each point x ∈ ℝ^{n}, there corresponds a subspace Δ(x). A distribution Δ is defined as

Δ = span{f_{1}, f_{2}, …, f_{r}}.

That is, Δ is the collection of all vector spaces Δ(x) for x ∈ D.

◇

The dimension of Δ(x) is

dim(Δ(x)) = rank [ f_{1}(x) f_{2}(x) ⋯ f_{r}(x) ] .

This rank may vary with x. If, however, the vectors f_{1}(x), f_{2}(x), …, f_{r}(x) are linearly independent for all x ∈ D, then

dim(Δ(x)) = r for all x ∈ D.  (11.8)

Definition 11.5 The distribution Δ = span{f_{1}, f_{2}, …, f_{r}} is a nonsingular distribution on D if (11.8) is satisfied.

◇

Example 11.1 The distribution,

Δ = span{f_{1}, f_{2}},

where

f_{1}(x) = [ \begin{matrix}e^{x_{2}} \\ 1 \\ 0 \end{matrix} ] and f_{2}(x) = [ \begin{matrix}2 \\ 0 \\ x_{2} \end{matrix} ] , x ∈ ℝ^{3},

is nonsigular over D = ℝ^{3} because dim(Δ(x)) = 2 for all x ∈ ℝ^{3}, that is,

rank [ \begin{matrix}e^{x_{2}} & 2 \\ 1 & 0 \\ 0 & x_{2} \end{matrix} ] = 2 for all x ∈ ℝ^{3}.

If Δ is a nonsingular distribution generated by f_{1}, …, f_{r}, then any vector field g ∈ Δ can be expressed as

g(x) = ∑_{i=1}^{r} c_{i}(x)f_{i}(x),

where c_{i}(·), i = 1, …, r, are smooth functions defined on D.

Definition 11.6 A distribution Δ is involutive if

g_{1} ∈ Δ and g_{2} ∈ Δ then [g_{1}, g_{2}] ∈ Δ
◇

Theorem 11.1 If Δ = span{f_{1}, f_{2}, …, f_{k}} is a nonsingular distribution on D, then it is involutive if and only if

[f_{i}, f_{j}] ∈ Δ for all 1 ≤ i, j ≤ r

Example 11.2 (Kaczorek et al.) [166, p. 448]

Consider the nonsingular distribution of Example 11.1. This distribution is not involutive because

[f_{1}, f_{2}] ∉ span{f_{1}, f_{2}},

that is,

rank [ f_{1} f_{2} ] ≠ rank [ f_{1} f_{2} [f_{1}, f_{2}] ] .

Indeed, the Lie bracket

[f_{1}, f_{2}] = \frac{∂f_{1}}{∂x} f_{2} − \frac{∂f_{2}}{∂x} f_{1}
= [ \begin{matrix}0 & e^{x_{2}} & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} ] [ \begin{matrix}2 \\ 0 \\ x_{2} \end{matrix} ] − [ \begin{matrix}0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 1 & 0 \end{matrix} ] [ \begin{matrix}2 \\ 0 \\ x_{2} \end{matrix} ] [ \begin{matrix}e^{x_{2}} \\ 1 \\ 0 \end{matrix} ]

= [ \begin{matrix}0 \\ 0 \\ 0 \end{matrix} ] − [ \begin{matrix}0 \\ 0 \\ 1 \end{matrix} ]

= [ \begin{matrix}0 \\ 0 \\ −1 \end{matrix} ] for all x ∈ ℝ^{3}.

Therefore

rank [ f_{1} f_{2} [f_{1}, f_{2}] ] = rank [ \begin{matrix}e^{x_{2}} & 2 & 0 \\ 1 & 0 & 0 \\ 0 & x_{2} & −1 \end{matrix} ] = 3

while rank [ f_{1} f_{2} ] = 2.

We now introduce the notion of a codistribution. These objects are dual of distributions and are defined using covector fields. The properties of codistributions are similar to that of distributions, except that now we work with rows rather than columns. One type of codistribution is of special interest to us; it is the annihilator of a distribution Δ denoted Δ^{⊥} and defined as

Definition 11.7 Δ^{⊥} = {w : wv = 0, for all v ∈ Δ(x)}

◇

We also need the notion of complete integrability of a distribution.

Definition 11.8 A nonsingular distribution Δ = span{f_{1}, f_{2}, …, f_{r}} on D is completely integrable if for each x_{0} ∈ D there exists a neighborhood N of x_{0} and n − r real-valued smooth functions h_{1}(·), …, h_{n−r}(·) that satisfy the conditions

dh_{j} [ f_{1} f_{2} ⋯ f_{r} ] = [ 0 0 ⋯ 0 ] , 1 ≤ j ≤ n − r

and

Δ^{⊥} = span{dh_{1}, …, dh_{n−r}},

that is, the covector fields dh_{j}, j = 1, …, n − r, are linearly independent for all x ∈ D.

◇

With the above background, we can state the theorem of Frobenius that gives a necessary and sufficient condition for a nonsingular distribution to be involutive.

Theorem 11.2 (Frobenius) A nonsingular distribution is completely integrable if and only if it is involutive.

11.3 A Nonlinear Plant Model

Dynamical systems, that we consider in this chapter, are modeled by the equations

ẋ = f(x) + G(x)u  (11.9)
y = h(x),  (11.10)

where f : ℝ^{n} → ℝ^{n}, G : ℝ^{n} → ℝ^{n×m}, and the output map h : ℝ^{n} → ℝ^{p}. Note that G(x) may be written as G(x) = [ g_{1}(x) g_{2}(x) ⋯ g_{m}(x) ], where g_{i} : ℝ^{n} → ℝ^{n}, i = 1, 2, …, m. We write f ∈ C^{∞} if f is in all C^{k} for k = 1, 2, . . .. The functions f, g_{i}, and h are assumed to be C^{∞} vector fields on ℝ^{n}. We also assume that f(0) = 0 and h(0) = 0. We note that the linear system model ẋ = Ax + Bu, y = Cx is a special case of the above more general model. Indeed, we have f(x) = Ax, G(x) = B, and h(x) = Cx. A simple example of a nonlinear dynamical system whose model fits into the above class of systems is a model of one-link robot manipulator presented in Subsection 2.4.5.

11.4 Controller Form

One of the objectives of this chapter, is to devise a method for constructing state-feedback stabilizing controllers for a class of dynamical systems modeled by (11.9). To construct such controllers it is advantageous to work with an equivalent system model rather than with the original one. We first discuss a method of transforming a single-input nonlinear system model into an equivalent form that is a generalization of the controller form known from linear systems—see Section 5.4.1. We accomplish this using a transformation, described below, that is a generalization of the similarity transformation for single-input linear systems given by (5.27) on page 241.

A transformation T : D → ℝ^{n} is a diffeomorphism on D if it is invertible on D, that is, there exists a function T^{−1} such that

T^{−1}(T (x)) = x for all x ∈ D,

and both T and T^{−1} are continuously differentiable

If the Jacobian matrix \frac{∂T}{∂x} is nonsingular at a point x_{0} ∈ D, then it follows form the inverse function theorem, see Exercise 11.3, that there is a neighborhood N of x_{0} such that T restricted to N is a diffeomorphism on N.

A transformation T is a global diffeomorphism if it is a diffeomorphism on ℝ^{n} and

T (ℝ^{n}) = ℝ^{n}.

We now present necessary and sufficient conditions for a transformation T to be a global diffeomorphism. This theorem comes from Khalil [178, p. 561].

Theorem 11.3 A transformation T is a global diffeomorphism if and only if

1. the Jacobian matrix \frac{∂T}{∂x} is nonsingular for all x ∈ ℝ^{n};

2. T is proper, that is,

lim_{∥x∥→∞} ∥T (x)∥ = ∞.

Our goal is to construct a diffeomorphism such that

z = T (x), T (0) = 0,

with an inverse x = T^{−1}(z), such that system model ẋ = f(x) + g(x)u in the new coordinates has the form

[ ż_{1} ; ż_{2} ; ⋮ ; ż_{n−1} ; ż_{n} ] = [ z_{2} ; z_{3} ; ⋮ ; z_{n} ; f̃_{n} ] + [ 0 ; 0 ; ⋮ ; 0 ; g̃_{n} ] u,  (11.11)

where f̃_{n} = f̃_{n}(z_{1}, z_{2}, …, z_{n}), g̃_{n} = g̃_{n}(z_{1}, z_{2}, …, z_{n}), and g̃_{n} ≠ 0. The above form is a generalization of the controller form for linear system from Section 5.4.1.

We devise a method for constructing the desired transformation. In our subsequent discussion, we will be using three types of Lie derivatives defined in Section 11.1. Suppose that such a transformation exists. Differentiating z(t) with respect to time t gives

ż(t) = \frac{∂T}{∂x} ẋ = \frac{∂T}{∂x} (f(x) + g(x)u) .  (11.12)

Comparing the right hand sides of (11.11) and (11.12), we obtain

\frac{∂T}{∂x} f(x)|_{x=T^{−1}(z)} = [ z_{2} ; z_{3} ; ⋮ ; z_{n} ; f̃_{n}(z_{1}, z_{2}, …, z_{n}) ] ,  (11.13)

and

\frac{∂T}{∂x} g(x)|_{x=T^{−1}(z)} = [ 0 ; 0 ; ⋮ ; 0 ; g̃_{n}(z_{1}, z_{2}, …, z_{n}) ] .  (11.14)

We first consider (11.13), that is,

[ z_{2} ; z_{3} ; ⋮ ; z_{n} ; f̃_{n}(z_{1}, z_{2}, …, z_{n}) ] = [ T_{2} ; T_{3} ; ⋮ ; T_{n} ; f̃_{n}(z_{1}, z_{2}, …, z_{n}) ]

because z = T (x). Comparing the above with the left-hand side of (11.13) gives

\frac{∂T_{i}}{∂x} f(x) = T_{i+1}, i = 1, 2, …, n − 1,

or equivalently

T_{i+1} = Lf T_{i}, i = 1, 2, …, n − 1.

Therefore, the transformation z = T (x) has the form

T = [ T_{1} ; Lf T_{1} ; ⋮ ; Lf^{n−2}T_{1} ; Lf^{n−1}T_{1} ] .  (11.15)

The above means that the problem of constructing the desired transformation z = T (x) is reduced to finding its first component T_{1}. The remaining components of T can be successively computed using T_{1}. To specify T_{1}, we use (11.14), which we represent componentwise as

LgT_{i} = 0, i = 1, 2, …, n − 1,

and

LgT_{n} = g̃_{n}.

Note that LgT_{1} = 0 can be represented as

\frac{∂T_{1}}{∂x} (ad^{0}f, g) = 0.

Next, consider the relation LgT_{2} = 0. We will show that

LgT_{2} = \frac{∂T_{1}}{∂x} (ad^{1}f, g) = 0.  (11.16)

By the Leibniz formula,

LgT_{2} = LgLf T_{1}
= Lf LgT_{1} + L_{[f ,g]}T_{1}
= L_{[f ,g]}T_{1},  (11.17)

because LgT_{1} = 0, and hence Lf LgT_{1} = 0. Using (11.17), we obtain (11.16). Continuing in this manner, we get

\frac{∂T_{1}}{∂x} (ad^{i}f, g) = 0, i = 0, 1, …, n − 2,

and

\frac{∂T_{1}}{∂x} (ad^{n−1}f, g) = g̃_{n}.

We represent the above n equations in matrix form as

\frac{∂T_{1}}{∂x} [ (ad^{0}f, g) (ad^{1}f, g) ⋯ (ad^{n−1}f, g) ]
= [ LgT_{1} L_{(ad^{1}f ,g)}T_{1} ⋯ L_{(ad^{n−1}f ,g)}T_{1} ]
= g̃_{n} [ 0 0 ⋯ 1 ] .

Thus the derivative of T_{1}, that is, the row vector \frac{∂T_{1}}{∂x}, is the last row of the inverse of the controllability matrix multiplied by g̃_{n}, provided the controllability matrix is invertible. Let q(x) denote the last row of the inverse of the controllability matrix multiplied by g̃_{n}. Then, the problem we have to solve is to find T_{1} : ℝ^{n} → ℝ such that

\frac{∂T_{1}(x)}{∂x} = q(x), T_{1}(0) = 0,  (11.18)

or equivalently

∇T_{1} = q(x)^{⊤}, T_{1}(0) = 0.  (11.19)

The above problem can be viewed as finding a potential function T_{1} for a given vector field q^{⊤}. A necessary and sufficient condition for a given vector field to have a potential function is given by Theorem A.21 on page 1026. The function g̃_{n} is also referred to as an integrating factor.

We will show that if q^{⊤} has a potential function, then the resulting state variable transformation is a diffeomorphism. Before we proceed with the proof, we first illustrate the results obtained so far with a numerical example.

Example 11.3 Consider a nonlinear model of one-link robot manipulator presented in Subsection 2.4.5. The controllability matrix of this system model is

[ (ad^{0}f, g) (ad^{1}f, g) (ad^{2}f, g) ] = [ \begin{matrix}0 & 0 & 10 \\ 0 & 10 & −100 \\ 10 & −100 & 900 \end{matrix} ] .

The above controllability matrix is of full rank on ℝ^{3}. The last row of its inverse is

[ 0.1 0 0 ] .

Suppose that we take g̃_{n} = 1; then q = [ 0.1 0 0 ], and hence we can take,

T_{1} = 0.1 x_{1}.

Having obtained T_{1}, we construct the desired transformation z = T (x), where

T (x) = [ \begin{matrix}T_{1} \\ Lf T_{1} \\ Lf (Lf T_{1}) \end{matrix} ] = [ \begin{matrix}0.1 x_{1} \\ 0.1 x_{2} \\ 0.98 sin x_{1} + 0.1x_{3} \end{matrix} ] .

Note that the inverse transformation x = T^{−1}(z) exists and has the form

T^{−1}(z) = [ \begin{matrix}10 z_{1} \\ 10 z_{2} \\ 10z_{3} − 9.8 sin(10 z_{1}) \end{matrix} ] .

Furthermore,

\frac{∂T}{∂x} = [ \begin{matrix}0.1 & 0 & 0 \\ 0 & 0.1 & 0 \\ 0.98 cos x_{1} & 0 & 0.1 \end{matrix} ] .

Applying the transformation z = T (x) to the model of the one-link robot manipulator yields

ż = \frac{∂T}{∂x} f(x)|_{x=T^{−1}(z)} + \frac{∂T}{∂x} g(x)|_{x=T^{−1}(z)} u

= [ \begin{matrix}z_{2} \\ z_{3} \\ 9.8 sin(10z_{1}) + (9.8 cos(10z_{1}) − 10) z_{2} − 10z_{3} \end{matrix} ] + [ \begin{matrix}0 \\ 0 \\ 1 \end{matrix} ] u.  (11.20)

Our objective now is to show that if there is a potential function for q(x)^{⊤}, then the resulting transformation is indeed a diffeomorphism. By construction, T is of C^{∞} class, and T (0) = 0. We show that T is locally C^{∞}-invertible. By the inverse function theorem, stated in Exercise 11.3, if the Jacobian matrix of T evaluated at 0 is invertible, then T is locally C^{∞}-invertible. The Jacobian matrix of T is

DT = [ DT_{1} ; DLf T_{1} ; ⋮ ; DLf^{n−2}T_{1} ; DLf^{n−1}T_{1} ] .

11.4. CONTROLLER FORM 591 We compute the product of the above Jacobian matrix and the controllability matrix

[ (ad^{0}f, g) (ad^{1}f, g) ⋯ (ad^{n−1}f, g) ]

to get

[ DT_{1} ; DLf T_{1} ; ⋮ ; DLf^{n−2}T_{1} ; DLf^{n−1}T_{1} ] [ (ad^{0}f, g) (ad^{1}f, g) ⋯ (ad^{n−1}f, g) ] .

Performing multiplications yields DT [ (ad^{0}f, g) (ad^{1}f, g) ⋯ (ad^{n−1}f, g) ]

= [ \begin{matrix}LgT_{1} & L_{(ad^{1}f ,g)}T_{1} & ⋯ & L_{(ad^{n−1}f ,g)}T_{1} \\ LgLf T_{1} & L_{(ad^{1}f ,g)}Lf T_{1} & ⋯ & L_{(ad^{n−1}f ,g)}Lf T_{1} \\ ⋮ & ⋮ & & ⋮ \\ LgLf^{n−1}T_{1} & L_{(ad^{1}f ,g)}Lf^{n−1}T_{1} & ⋯ & L_{(ad^{n−1}f ,g)}Lf^{n−1}T_{1} \end{matrix} ]

= [ \begin{matrix}LgT_{1} & L_{(ad^{1}f ,g)}T_{1} & ⋯ & L_{(ad^{n−1}f ,g)}T_{1} \\ LgT_{2} & L_{(ad^{1}f ,g)}T_{2} & ⋯ & L_{(ad^{n−1}f ,g)}T_{2} \\ ⋮ & ⋮ & & ⋮ \\ LgT_{n} & L_{(ad^{1}f ,g)}T_{n} & ⋯ & L_{(ad^{n−1}f ,g)}T_{n} \end{matrix} ]

= [ \begin{matrix}LgT_{1} & L_{(ad^{1}f ,g)}T_{1} & ⋯ & L_{(ad^{n−1}f ,g)}T_{1} \\ L_{(ad^{1}f ,g)}T_{1} & L_{(ad^{2}f ,g)}T_{1} & ⋯ & L_{(ad^{n}f ,g)}T_{1} \\ ⋮ & ⋮ & & ⋮ \\ L_{(ad^{n−1}f ,g)}T_{1} & L_{(ad^{n}f ,g)}T_{1} & ⋯ & L_{(ad^{2n−2}f ,g)}T_{1} \end{matrix} ]

= [ \begin{matrix}0 & 0 & ⋯ & 0 & g̃_{n} \\ 0 & 0 & ⋯ & g̃_{n} & x \\ ⋮ & ⋮ & & ⋮ & ⋮ \\ g̃_{n} & x & ⋯ & x & x \end{matrix} ] ,

where we use x to denote “don’t care” elements and g̃_{n} ≠ 0. It follows from the above that at x = 0 the Jacobian matrix DT is invertible, that is,

rank DT (0) = n.

We summarize the above analysis in the form of the following theorem,

Theorem 11.4 There exists a diffeomorphic state variable transformation, in a neighborhood of x = 0, if

1. in a neighborhood of x = 0,

rank [ (ad^{0}f, g) (ad^{1}f, g) ⋯ (ad^{n−1}f, g) ] = n,

2.

q(x) = g̃_{n} [ 0 0 ⋯ 0 1 ] [ (ad^{0}f, g) (ad^{1}f, g) ⋯ (ad^{n−1}f, g) ]^{−1}

has a potential function in a neighborhood of x = 0 for some g̃_{n} ≠ 0, that is, q(x)^{⊤} is a conservative vector field in a neighborhood of x = 0.

We reformulate the above conditions for the existence of the diffeomorphism transforming the given nonlinear system into the controller form using the notion of distributions that we introduced in Section 11.2.

Theorem 11.5 For the single-input model,

ẋ = f(x) + g(x)u

there exists a diffeomorphism, z = T (x), transforming it into the controller form (11.11), in a neighborhood of x = 0, if

1. in a neighborhood of x = 0,

rank [ (ad^{0}f, g) (ad^{1}f, g) ⋯ (ad^{n−1}f, g) ] = n;

2. the vector fields

(ad^{0}f, g), (ad^{1}f, g), …, (ad^{n−2}f, g)

generate, in the neighborhood of x = 0, a singular involutive distribution.

The above conditions are also necessary—see Khalil [178, pp. 567–568] for a proof of both, necessary and sufficient parts, using the Frobenius theorem.

The above method of transforming a class of single input nonlinear systems into the controller form can be generalized, with some effort, to multi-input systems. We illustrate the method with a numerical example.

Example 11.4 Consider the model of the permanent stepper motor given by (2.61). We first represent the model as

ẋ = f(x) + g_{1}u_{1} + g_{2}u_{2},  (11.21)

where

f(x) = [ \begin{matrix}x_{2} \\ −K_{1}x_{2} − K_{2}x_{3} sin(K_{3}x_{1}) + K_{2}x_{4} cos(K_{3}x_{1}) \\ K_{4}x_{2} sin(K_{3}x_{1}) − K_{5}x_{3} \\ −K_{4}x_{2} cos(K_{3}x_{1}) − K_{5}x_{4} \end{matrix} ] , g_{1} = [ \begin{matrix}0 \\ 0 \\ 1 \\ 0 \end{matrix} ] , and g_{2} = [ \begin{matrix}0 \\ 0 \\ 0 \\ 1 \end{matrix} ] .

We form the controllability matrix for the above model,

[ g_{1} g_{2} (ad^{1}f, g_{1}) (ad^{1}f, g_{2}) (ad^{2}f, g_{1}) (ad^{2}f, g_{2}) (ad^{3}f, g_{1}) (ad^{3}f, g_{2}) ] ,

(11.22)

where, for example,

(ad^{1}f, g_{1}) = [ \begin{matrix}0 \\ −K_{2} sin(K_{3}x_{1}) \\ −K_{5} \\ 0 \end{matrix} ] , (ad^{1}f, g_{2}) = [ \begin{matrix}0 \\ K_{2} cos(K_{3}x_{1}) \\ 0 \\ −K_{5} \end{matrix} ] ,

(ad^{2}f, g_{1})

= [f, (ad^{1}f, g_{1})] = \frac{∂f}{∂x} (ad^{1}f, g_{1}) − \frac{∂ (ad^{1}f, g_{1})}{∂x}f

= [ \begin{matrix}−K_{2} sin(K_{3} x_{1}) \\ K_{1} K_{2} sin(K_{3} x_{1}) + K_{2} K_{5} sin(K_{3} x_{1}) + K_{2} K_{3} x_{2} cos(K_{3} x_{1}) \\ −K_{4} K_{2} sin^{2}(K_{3} x_{1}) + K_{5}^{2} \\ K_{4} K_{2} cos(K_{3} x_{1}) sin(K_{3} x_{1}) \end{matrix} ] , (11.23)

and

(ad^{2}f, g_{2})

= [f, (ad^{1}f, g_{2})] = \frac{∂f}{∂x} (ad^{1}f, g_{2}) − \frac{∂ (ad^{1}f, g_{2})}{∂x}f

= [ \begin{matrix}K_{2} cos(K_{3} x_{1}) \\ −K_{1} K_{2} cos(K_{3} x_{1}) − K_{2} K_{5} cos(K_{3} x_{1}) + K_{2} K_{3} x_{2} sin(K_{3} x_{1}) \\ K_{4} K_{2} cos(K_{3} x_{1}) sin(K_{3} x_{1}) \\ −K_{4} K_{2} cos^{2}(K_{3} x_{1}) + K_{5}^{2} \end{matrix} ] . (11.24)

Observe that

(ad^{1}f, g_{2}) = −K_{5} cot(K_{3} x_{1}) g_{1} − K_{5} g_{2} − cot(K_{3} x_{1}) (ad^{1}f, g_{1}) ,  (11.25)

that is, (ad^{1}f, g_{2}) is a linear combination of g_{1}, g_{2}, and (ad^{1}f, g_{1}). We use this fact when selecting columns to construct a transformation into a controller form. Proceeding from left to right and using (11.25), we select linearly independent columns from the controllability matrix (11.22). The PM stepper motor system is controllable because we have found four linearly independent columns, g_{1}, (ad^{1}f, g_{1}), (ad^{2}f, g_{1}), and g_{2} in its controllability matrix. Thus, the controllability indices of the PM stepper motor system are d_{1} = 3 and d_{2} = 1. Therefore, the controller form of the PM stepper motor model is

[ ż_{1} ; ż_{2} ; ż_{3} ; ż_{4} ] = [ \begin{matrix}z_{2} \\ z_{3} \\ f̃_{3}(z_{1}, z_{2}, z_{3}, z_{4}) \\ f̃_{4}(z_{1}, z_{2}, z_{3}, z_{4}) \end{matrix} ] + [ \begin{matrix}0 & 0 \\ 0 & 0 \\ g̃_{31} & g̃_{32} \\ g̃_{41} & g̃_{42} \end{matrix} ] [ \begin{matrix}u_{1} \\ u_{2} \end{matrix} ] ,  (11.26)

where the elements g̃_{ij} may depend on z_{k}, k = 1, 2, 3, 4, and

det [ \begin{matrix}g̃_{31} & g̃_{32} \\ g̃_{41} & g̃_{42} \end{matrix} ] ≠ 0.  (11.27)

The transformation we are looking for has the form,

z = T (x) = [ \begin{matrix}T_{1} \\ T_{2} \\ T_{3} \\ T_{4} \end{matrix} ] = [ \begin{matrix}T_{1} \\ Lf T_{1} \\ Lf^{2} T_{1} \\ T_{4} \end{matrix} ] ,  (11.28)

where T_{1} and T_{4} are functions that we have to find. There is no unique solution for T_{1} and T_{4}. Differentiating z = T (x) and using (11.21) yields

ż = \frac{∂T}{∂x} ẋ
= \frac{∂T}{∂x} f(x)|_{x=T^{−1}(z)} + u_{1} \frac{∂T}{∂x} g_{1}(x)|_{x=T^{−1}(z)} + u_{2} \frac{∂T}{∂x} g_{2}(x)|_{x=T^{−1}(z)}(11.29)

where

\frac{∂T}{∂x} = DT = [ \begin{matrix}DT_{1} \\ DT_{2} \\ DT_{3} \\ DT_{4} \end{matrix} ] .

Comparing (11.26) and (11.29), we obtain

z_{2} = T_{2} = DT_{1}f = Lf T_{1}, z_{3} = T_{3} = DT_{2}f = Lf T_{2},  (11.30)

and

L_{g_{1}}T_{1} = 0, L_{g_{1}}T_{2} = L_{g_{1}}Lf T_{1} = 0,  (11.31)
L_{g_{2}}T_{1} = 0, L_{g_{2}}T_{2} = L_{g_{2}}Lf T_{1} = 0.  (11.32)

Applying the Leibniz formula (11.7) to the right hand sides of (11.31) and (11.32) gives

−Lf L_{g_{1}}T_{1} = L_{[f ,g_{1}]}T_{1} and − Lf L_{g_{2}}T_{1} = L_{[f ,g_{2}]}T_{1}.

Because by (11.31) and (11.32), L_{g_{1}}T_{1} = 0 and L_{g_{2}}T_{1} = 0, we can write

DT_{1} [ g_{1} (ad^{1}f, g_{1}) g_{2} (ad^{1}f, g_{2}) ] = 0,  (11.33)

that is,

[ \frac{∂T_{1}}{∂x_{1}}  \frac{∂T_{1}}{∂x_{2}}  \frac{∂T_{1}}{∂x_{3}}  \frac{∂T_{1}}{∂x_{4}} ] [ \begin{matrix}0 & 0 & 0 & 0 \\ 0 & −K_{2} sin(K_{3}x_{1}) & 0 & K_{2} cos(K_{3}x_{1}) \\ 1 & −K_{5} & 0 & 0 \\ 0 & 0 & 1 & −K_{5} \end{matrix} ] = [ 0 0 0 0 ] .

By inspection, a nontrivial solution to the above is

DT_{1} = [ \frac{∂T_{1}}{∂x_{1}} 0 0 0 ] ,  (11.34)

where the function T_{1} may only be a function of x_{1} for DT_{1} to be integrable. We specify T_{1} later. To proceed, we note that by (11.28) and (11.29),

[ \begin{matrix}g̃_{31} & g̃_{32} \\ g̃_{41} & g̃_{42} \end{matrix} ] = [ \begin{matrix}(DT_{3}) g_{1} & (DT_{3}) g_{2} \\ (DT_{4}) g_{1} & (DT_{4}) g_{2} \end{matrix} ] ,  (11.35)

Using the structure of the state transformation (11.28) and the Leibniz formula (11.7), we express the element g̃_{31} = (DT_{3}) g_{1} as

(DT_{3}) g_{1} = L_{g_{1}}Lf T_{2}
= L_{[f ,g_{1}]}T_{2} + Lf L_{g_{1}}T_{2}
= L_{[f ,g_{1}]}T_{2},  (11.36)

596 CHAPTER 11. VECTOR FIELD METHODS because L_{g_{1}}T_{2} = 0 and therefore Lf L_{g_{1}}T_{2} = 0. Observing that T_{2} = Lf T_{1}, employing the Leibniz formula, and taking into account the fact that L_{g_{1}}T_{1} = 0, we get

L_{[f ,g_{1}]}T_{2} = L_{[f ,g_{1}]}Lf T_{1}
= L_{[f ,[f ,g_{1}]]}T_{1} + Lf L_{[f ,g_{1}]}T_{1}
= L_{[f ,[f ,g_{1}]]}T_{1} + Lf (L_{g_{1}}Lf T_{1} − Lf L_{g_{1}}T_{1})
= L_{[f ,[f ,g_{1}]]}T_{1}
= DT_{1} (ad^{2}f, g_{1}) .  (11.37)

Hence,

g̃_{31} = DT_{1} (ad^{2}f, g_{1}) .  (11.38)

Similarly, we find

g̃_{32} = DT_{1} (ad^{2}f, g_{2}) .  (11.39)

Substituting (11.23), (11.24), and (11.34) into (11.38) and (11.39), we find

g̃_{31} = −\frac{∂T_{1}}{∂x_{1}} K_{2} sin(K_{3}x_{1}),  (11.40)

and

g̃_{32} = \frac{∂T_{1}}{∂x_{1}} K_{2} cos(K_{3}x_{1}).  (11.41)

Substituting the above expressions into (11.27) yields

g̃_{31}g̃_{42} − g̃_{32}g̃_{41} = (DT_{1} (ad^{2}f, g_{1})) ((DT_{4}) g_{2}) − (DT_{1} (ad^{2}f, g_{2})) ((DT_{4}) g_{1})
= −\frac{∂T_{1}}{∂x_{1}} K_{2} sin(K_{3}x_{1})\frac{∂T_{4}}{∂x_{4}} − \frac{∂T_{1}}{∂x_{1}} K_{2} cos(K_{3}x_{1})\frac{∂T_{4}}{∂x_{3}}
≠ 0.  (11.42)

Suppose that we choose

\frac{∂T_{1}}{∂x_{1}} = \frac{1}{K_{2}} , \frac{∂T_{4}}{∂x_{3}} = cos(K_{3}x_{1}), and \frac{∂T_{4}}{∂x_{4}} = sin(K_{3}x_{1}).  (11.43)

Substituting (11.43) into (11.42) gives

det [ \begin{matrix}g̃_{31} & g̃_{32} \\ g̃_{41} & g̃_{42} \end{matrix} ] = −1.

Using (11.34), (11.43), and (11.28), we obtain

\begin{matrix}T_{1} & = & x_{1}/K_{2} \\ T_{2} & = & Lf T_{1} = x_{2}/K_{2} \\ T_{3} & = & Lf T_{2} = −\frac{K_{1}}{K_{2}} x_{2} − x_{3} sin(K_{3}x_{1}) + x_{4} cos(K_{3}x_{1}) \\ T_{4} & = & x_{3} cos(K_{3}x_{1}) + x_{4} sin(K_{3}x_{1}) \end{matrix} \}  (11.44)

The Jacobian matrix of T is

DT = [ \begin{matrix}1/K_{2} & 0 & 0 & 0 \\ 0 & 1/K_{2} & 0 & 0 \\ −K_{3}x_{3} cos(K_{3}x_{1}) − K_{3}x_{4} sin(K_{3}x_{1}) & −K_{1}/K_{2} & − sin(K_{3}x_{1}) & cos(K_{3}x_{1}) \\ −K_{3}x_{3} sin(K_{3}x_{1}) + K_{3}x_{4} cos(K_{3}x_{1}) & 0 & cos(K_{3}x_{1}) & sin(K_{3}x_{1}) \end{matrix} ]

(11.45)

We have

T (0) = 0 and det DT = −1/K_{2}^{2} ≠ 0 for all x ∈ ℝ^{4},  (11.46)

which means that we were able to find a global transformation into the controller form. The inverse, x = T^{−1}(z) is

\begin{matrix}x_{1} & = & K_{2}z_{1} \\ x_{2} & = & K_{2}z_{2} \\ x_{3} & = & −K_{1}z_{2} sin(K_{3}K_{2}z_{1}) − z_{3} sin(K_{3}K_{2}z_{1}) + z_{4} cos(K_{3}K_{2}z_{1}) \\ x_{4} & = & K_{1}z_{2} cos(K_{3}K_{2}z_{1}) + z_{3} cos(K_{3}K_{2}z_{1}) + z_{4} sin(K_{3}K_{2}z_{1}) \end{matrix} \}  (11.47)

We transform the PM stepper motor model into the new coordinates using (11.28), (11.44), and (11.44) to obtain

[ ż_{1} ; ż_{2} ; ż_{3} ; ż_{4} ] = [ \begin{matrix}z_{2} \\ z_{3} \\ −K_{2}K_{3}z_{2}z_{4} − K_{1}z_{3} − K_{2}K_{4}z_{2} − K_{5}z_{3} − K_{5}K_{1}z_{2} \\ K_{2}K_{3}z_{2}z_{3} + K_{1}K_{2}K_{3}z_{2}^{2} − K_{5}z_{4} \end{matrix} ]

+ [ \begin{matrix}0 & 0 \\ 0 & 0 \\ − sin(K_{3}K_{2}z_{1}) & cos(K_{3}K_{2}z_{1}) \\ cos(K_{3}K_{2}z_{1}) & sin(K_{3}K_{2}z_{1}) \end{matrix} ] [ \begin{matrix}u_{1} \\ u_{2} \end{matrix} ] .  (11.48)

11.5 Linearizing State-Feedback Control

Once the plant model is transformed into the controller form, we can construct a statefeedback controller in the new coordinates and then using the inverse transformation, represent the controller in the original coordinates. While constructing the controller in the new coordinates, a part of the controller is used to cancel nonlinearities thus resulting in a linear system in the new coordinates. This procedure, as we mentioned at the beginning of this chapter, is called the feedback linearization. Then, we proceed to construct the other part of the controller. This part can be designed using linear control methods because the feedback linearized system is linear.

We illustrate the above method of state feedback design on a numerical example involving the one-link manipulator whose model was described in Subsection 2.4.5. The state space model of the manipulator, given by (2.83), was transformed into the controller form in Example 11.3. The controller form of the one-link manipulator model is given by (11.20). It is easy to design a stabilizing state-feedback controller in the new coordinates. It takes the form

u = −f̃_{3} (z_{1}, z_{2}, z_{3}) − (k_{1}z_{1} + k_{2}z_{2} + k_{3}z_{3}),  (11.49)

where f̃_{3} (z_{1}, z_{2}, z_{3}) = 9.8 sin(10z_{1}) + (9.8 cos(10z_{1}) − 10) z_{2} − 10z_{3}. Suppose that the desired closed-loop poles of the feedback linearized one-link manipulator are to be located at {−3.5, −4, −5}. Then, the linear feedback gains k_{i}, i = 1, 2, 3 that shift the poles to these desired locations are:

k_{1} = 70, k_{2} = 51.5, k_{3} = 12.5.

Applying (11.49) with the above values of the linear feedback gains to the model (11.20) gives

ż = [ \begin{matrix}0 & 1 & 0 \\ 0 & 0 & 1 \\ −70 & −51.5 & −12.5 \end{matrix} ] z.

Next, applying the inverse transformation yields the controller in the original coordinates,

u = −7x_{1} − 4.15x_{2} − 0.25x_{3} − 0.98x_{2} cos(x_{1}) − 12.25 sin(x_{1}).  (11.50)

In Figure 11.1, three plots of the manipulator’s link angle, x_{1} = θ, versus time are presented. The initial conditions have the form, x(0) = [ θ(0) 0 0 ]^{⊤}. The control law applied to the manipulator is given by (11.50).

Figure 11.1: Plots of the one-link manipulator’s link angle, x_{1} = θ, versus time for three different initial angles θ(0). The manipulator is driven by the state-feedback control law (11.50).

11.6 Observer Form

In this section, we discuss the problem of transforming a class of nonlinear system models into an observer form, which is then used to construct observers for these systems. We consider a single output nonlinear system model of the form

\begin{matrix}ẋ & = & f(x) + G(x)u \\ y & = & h(x) \end{matrix} \}  (11.51)

First, we analyze the case of unforced system when u = 0. In such a case (11.51) reduces to

\begin{matrix}ẋ & = & f(x) \\ y & = & h(x) \end{matrix} \}  (11.52)

We desire to find a state variable transformation, written as x = T (z) such that the model (11.52) in the z-coordinates is

\begin{matrix}ż & = & [ \begin{matrix}0 & 0 & ⋯ & 0 & 0 \\ 1 & 0 & ⋯ & 0 & 0 \\ ⋮ & & ... & & ⋮ \\ 0 & 0 & ⋯ & 1 & 0 \end{matrix} ] z + [ α_{0}(y) ; α_{1}(y) ; ⋮ ; α_{n−1}(y) ] = Az + α(y) = f̃(z), \\ y & = & [ 0 0 ⋯ 0 1 ] z = cz. \end{matrix} \}  (11.53)

It is easy to verify that the state z̃ of the dynamical system

˙z̃ = (A − lc) z̃ + α(y) + ly,  (11.54)

asymptotically converges to the state z of (11.53) if the matrix A − lc is asymptotically stable. This is because the dynamics of the error, e = z − z̃, are described by

ė = (A − lc) e, e(t_{0}) = z(t_{0}) − z̃(t_{0})  (11.55)

We derive a sufficiency condition for the existence of a state variable transformation that brings (11.52) into the observer form (11.53). Taking the derivative of x = T (z) with respect to time yields

ẋ = \frac{∂T}{∂z} ż = \frac{∂T}{∂z} f̃(z),  (11.56)

where we represent the Jacobian matrix of T (z), denoted \frac{∂T}{∂z}, as

\frac{∂T}{∂z} = DT = [ \frac{∂T}{∂z_{1}} ⋯ \frac{∂T}{∂z_{n}} ] .  (11.57)

Equating the right-hand sides of ẋ = f(x) and (11.56) yields

f(x) = f(T (z)) = \frac{∂T}{∂z} f̃(z).  (11.58)

Taking the partial derivative of (11.58) with respect to z_{k} yields

\frac{∂f(T (z))}{∂z_{k}} = \frac{∂ (\frac{∂T}{∂z} )}{∂z_{k}} f̃(z) + \frac{∂T}{∂z} \frac{∂f̃(z)}{∂z_{k}}  (11.59)

On the other hand, application of the chain rule gives

\frac{∂f(T (z))}{∂z_{k}} = \frac{∂f}{∂x} \frac{∂T (z)}{∂z_{k}}.  (11.60)

Using (11.58) and the fact that x = T (z), the first term on the right-hand side of (11.59) evaluates to

\frac{∂ (\frac{∂T}{∂z} )}{∂z_{k}} f̃(z) = \frac{∂ (\frac{∂T}{∂z_{k}} )}{∂z}f̃(z) = \frac{∂ (\frac{∂T}{∂z_{k}} )}{∂z} \frac{∂x}{∂x}f̃(z) = \frac{∂ (\frac{∂T}{∂z_{k}} )}{∂x} \frac{∂T (z)}{∂z}f̃(z) = \frac{∂ (\frac{∂T}{∂z_{k}} )}{∂x}f(x).

(11.61)

Employing (11.53) and (11.57), we evaluate the second term on the right-hand side of (11.59) to get

\frac{∂T}{∂z} \frac{∂f̃(z)}{∂z_{k}} = \frac{∂T}{∂z_{k+1}} , k = 1, 2, …, n − 1.  (11.62)

Using the above and substituting (11.60) and (11.61) into (11.59) gives

\frac{∂T}{∂z_{k+1}} = \frac{∂f}{∂x} \frac{∂T (z)}{∂z_{k}} − \frac{∂ (\frac{∂T}{∂z_{k}} )}{∂x}f(x), k = 1, 2, …, n − 1  (11.63)

With the help of the Lie-derivative notation introduced in Section 11.4, we express (11.63) in an alternative way. Specifically, using (11.1) and (11.2), we represent (11.63) as

\frac{∂T}{∂z_{k+1}} = [f, \frac{∂T}{∂z_{k}} ] = (ad^{1}f, \frac{∂T}{∂z_{k}} ) , k = 1, 2, …, n − 1.  (11.64)

Observe that

\frac{∂T}{∂z_{2}} = (ad^{1}f, \frac{∂T}{∂z_{1}} ) ,
\frac{∂T}{∂z_{3}} = (ad^{1}f, \frac{∂T}{∂z_{2}} ) = (ad^{2}f, \frac{∂T}{∂z_{1}} ) ,
⋮  ⋮

and in general

\frac{∂T}{∂z_{k}} = (ad^{k−1}f, \frac{∂T}{∂z_{1}} ) , k = 1, 2, …, n  (11.65)

With the aid of the above, we express all the columns of the Jacobian matrix (11.57) of T (z) in terms of the starting vector ∂T /∂z_{1}, that is,

\frac{∂T}{∂z} = [ (ad^{0}f, \frac{∂T}{∂z_{1}} ) (ad^{1}f, \frac{∂T}{∂z_{1}} ) ⋯ (ad^{n−1}f, \frac{∂T}{∂z_{1}} ) ]  (11.66)

To obtain an expression for the starting vector ∂T /∂z_{1}, we use the output equation

y = h(x) = z_{n}.  (11.67)

We use the chain rule when taking the partial derivative of (11.67) with respect to z to get

\frac{∂h(x)}{∂x} \frac{∂T}{∂z} = [ 0 0 ⋯ 0 1 ] .  (11.68)

We utilize (11.5) to express the first component on the left-hand side of the above as

\frac{∂h(x)}{∂x} \frac{∂T}{∂z_{1}} = Lf^{0} (Dh)\frac{∂T}{∂z_{1}} = 0.  (11.69)

Using the Leibniz formula, we represent the second component of the left-hand side of (11.68) as

0 = \frac{∂h(x)}{∂x} \frac{∂T}{∂z_{2}}
= ⟨∇h, [f, \frac{∂T}{∂z_{1}} ]⟩
= L_{\frac{∂T}{∂z_{1}}} Lf h − Lf L_{\frac{∂T}{∂z_{1}}} h
= L_{\frac{∂T}{∂z_{1}}} Lf h,  (11.70)

because \frac{∂h(x)}{∂x} \frac{∂T}{∂z_{1}} = 0 by (11.69). We use (11.6) to represent (11.70) as

0 = L_{\frac{∂T}{∂z_{1}}} Lf h
= DLf h\frac{∂T}{∂z_{1}}
= Lf^{1} (Dh)\frac{∂T}{∂z_{1}} .  (11.71)

By repeated application of the Leibniz formula and (11.66), we express (11.68) equivalently as

[ Lf^{0} (Dh) ; Lf^{1} (Dh) ; ⋮ ; Lf^{n−2}(Dh) ; Lf^{n−1}(Dh) ] \frac{∂T}{∂z_{1}} = [ 0 0 ⋮ 0 1 ] .  (11.72)

The matrix

[ Lf^{0} (Dh) ; Lf^{1} (Dh) ; ⋮ ; Lf^{n−2}(Dh) ; Lf^{n−1}(Dh) ]  (11.73)

is called the observability matrix of the nonlinear system model (11.52). It follows from (11.72) that the starting vector ∂T /∂z_{1} is equal to the last column of the inverse of the observability matrix given by (11.73).

We next discuss the case of transforming a nonlinear system model with inputs, described by (11.51), into its observer form. We proceed as recommended by Krener and Respondek [193, Section 6]. First transform the unforced system into the observer form (11.53). Then, check if the vector field G(x)u is transformed into the form

(\frac{∂T}{∂z} )^{−1} G(x)|_{x=T (z)} u = β(y, u).  (11.74)

If this is the case, then set

γ(y, u) = α(y) + β(y, u).

The above means that the system with inputs is transformed into the observer form,

\begin{matrix}ż & = & Az + γ(y, u) \\ y & = & cz \end{matrix} \}  (11.75)

where the matrices A and c are as in (11.53). If the unforced system cannot be transformed into the observer form or G(x)u does not transform into (11.74), then the original system (11.51) cannot be transformed into the observer form (11.75).

Example 11.5 Consider the one-link robot manipulator model that we analyzed in Subsection 2.4.5. The observability matrix for the system is

[ \begin{matrix}Lf^{0} (Dh) \\ Lf^{1} (Dh) \\ Lf^{2} (Dh) \end{matrix} ] = [ \begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 9.8 cos(x_{1}) & 0 & 1 \end{matrix} ] .  (11.76)

The starting vector ∂T /∂z_{1} is the last column of the inverse of the above observability matrix,

∂T /∂z_{1} = [ \begin{matrix}0 \\ 0 \\ 1 \end{matrix} ] .  (11.77)

We find the Jacobian matrix of the desired transformation:

\frac{∂T}{∂z} = [ (ad^{0}f, \frac{∂T}{∂z_{1}} ) (ad^{1}f, \frac{∂T}{∂z_{1}} ) (ad^{2}f, \frac{∂T}{∂z_{1}} ) ] = [ \begin{matrix}0 & 0 & 1 \\ 0 & 1 & −10 \\ 1 & −10 & 90 \end{matrix} ] .  (11.78)

Therefore, we can take

x = T (z) = [ \begin{matrix}0 & 0 & 1 \\ 0 & 1 & −10 \\ 1 & −10 & 90 \end{matrix} ] z.  (11.79)

Note that T (0) = 0. The inverse transformation is

z = T^{−1}(x) = T^{−1}x = [ \begin{matrix}10 & 10 & 1 \\ 10 & 1 & 0 \\ 1 & 0 & 0 \end{matrix} ] x.  (11.80)

The system in the new coordinates has the form

ż = (\frac{∂T}{∂z} )^{−1} f(x)|_{x=T (z)} + (\frac{∂T}{∂z} )^{−1} g(x)|_{x=T (z)} u = Az + γ(y, u)
y = cz.

Performing simple manipulations yields

ż = [ \begin{matrix}0 & 0 & 0 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{matrix} ] z + [ \begin{matrix}98 sin(y) + 10u \\ 9.8 sin(y) − 10y \\ −10y \end{matrix} ]  (11.81)
y = [ 0 0 1 ] z = z_{3}.  (11.82)

11.7 Asymptotic Observer

Once the nonlinear system model (11.51) is in the observer form, we proceed with the construction of an asymptotic observer using the technique from the linear systems. The observer dynamics in the new coordinates are

˙z̃ = (A − lc) z̃ + ly + γ(y, u),  (11.83)

where the observer gain vector l is chosen so that the matrix (A − lc) has its eigenvalues in the desired locations in the open left-hand complex plane. It is easy to verify that the state estimation error, z − z̃, satisfies the linear differential equation,

\frac{d}{dt}(z − z̃) = (A − lc) (z − z̃) .

Applying the inverse transformation to (11.83), we obtain the state estimator dynamics in the original coordinates.

We illustrate the above method of constructing an observer using the model of the onelink manipulator from Subsection 2.4.5. We first construct an observer for the one-link manipulator model in the observer form given by (11.81) and (11.82). Suppose that we wish the eigenvalues of the observer matrix, A − lc, to be located at {−9, −10, −11}. Then, the observer gain vector is l = [ 990 299 30 ]^{⊤}. With the above choice of the observer design parameters, its dynamics in the new coordinates become

˙ ; z̃ = [ \begin{matrix}0 & 0 & −990 \\ 1 & 0 & −299 \\ 0 & 1 & −30 \end{matrix} ] z̃ + [ \begin{matrix}990 \\ 299 \\ 30 \end{matrix} ] y + [ \begin{matrix}98 sin(y) + 10u \\ 9.8 sin(y) − 10y \\ −10y \end{matrix} ]  (11.84)
y = [ 0 0 1 ] z = z_{3}.  (11.85)

Using transformations (11.79) and (11.80), we represent the observer dynamics in the original coordinates,

˙ ; x̃ = [ \begin{matrix}−20 & 1 & 0 \\ −89 & 0 & 1 \\ 100 & −10 & −10 \end{matrix} ] x̃ + [ \begin{matrix}30 \\ −1 \\ 700 \end{matrix} ] y + [ \begin{matrix}−10y \\ 9.8 sin(y) + 90y \\ −800y + 10u \end{matrix} ]  (11.86)
y = [ 1 0 0 ] x = x_{1}.  (11.87)

Performing simple manipulations, we represent the above observer as:

˙ ; x̃ = [ \begin{matrix}0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & −10 & −10 \end{matrix} ] x̃ + [ \begin{matrix}0 \\ 9.8 sin(y) \\ 0 \end{matrix} ] − [ \begin{matrix}20 \\ 89 \\ −100 \end{matrix} ] (ỹ − y) + [ \begin{matrix}0 \\ 0 \\ 10 \end{matrix} ] u(11.88)
y = [ 1 0 0 ] x = x_{1}  (11.89)
ỹ = [ 1 0 0 ] x̃ = x̃_{1}  (11.90)

11.8 Combined Controller-Observer Compensator

Having constructed an observer, we use the state estimates, rather than the states themselves, in the state-feedback control law implementation. The question now is how the incorporation of the observer affects the stability of the closed-loop system with the combined controller-observer compensator in the loop. Recall that in the case of linear systems, we were able to show that the closed-loop system was asymptotically stable. In the present case, one has to be careful when analyzing the stability of the closed-loop system with the combined controller-observer compensator in the loop. The plant’s nonlinearities may cause Figure 11.2: Plots of the one-link manipulator’s link angle, x_{1} = θ, versus time for three different initial angles θ(0). The manipulator is driven by the combined controller-estimator compensator.

difficulties in stabilizing the closed-loop system using the combined controller-observer compensator. However, for certain “nice” nonlinearities, for example the ones for which the Lipschitz condition is satisfied, one should be able to find stability conditions in terms of the nonlinearity’s Lipschitz constant and the location of the observer’s poles rather easily.

We illustrate the implementation of the combined controller-observer compensator on the one-link manipulator model. In this example, the controller is

u = −7x̃_{1} − 4.15x̃_{2} − 0.25x̃_{3} − 0.98x̃_{2} cos(x̃_{1}) − 12.25 sin(x̃_{1}).  (11.91)

In Figure 11.2, plots of the one-link manipulator output for three different initial conditions are shown. The manipulator is driven by the combined controller-observer compensator consisting of the observer (11.88), (11.89), and (11.90) and the control law (11.91). The initial conditions of the observer were set to zero while the initial conditions of the plant were the same as when we implemented the state-feedback control alone. Comparing plots of Figures 11.1 and 11.2, we can see that the presence of the observer alters the system output in a noticeable way.

11.9 Notes

Friedland [108, page 123] has the following remarks to offer on the method of feedback linearization: “The idea underlying feedback linearization is fairly obvious. The method probably was used for many years in practical control system design without anyone having reported it. Concurrently, a number of mathematical investigations on the use of differential geometry in dynamical systems were conducted without much notice by control system designers.”

Our exposition of feedback linearization is based on Sommer [309] and [368]. The research in applying differential geometric methods to the control design intensified after the papers by Su [311], and by Hunt, Su, and Meyer [153] appeared. Bakker and Nijmeijer [19] showed the equivalence of the Sommer’s approach and that of Hunt, Su, and Meyer. Our treatment of observers follows the approach of Bestle and Zeitz [31], which in turn was presented, using the differential geometry notation, in [338]. Zeitz [372] offers a generalization of the method proposed by Walcott et al. [338]. A short and readable introduction to feedback linearization is [158]. For further study of differential geometric techniques in nonlinear feedback design, we recommend books by Isidori [157], and Marino and Tomei [230].

11.10 Exercises

Exercise 11.1 Verify the Leibniz formula (11.7).

Exercise 11.2 Consider the one-link robot manipulator of Subsection 2.4.5 whose model is given on page 97 by (2.83) and (2.84). Treat this model as a simulation model. In order to obtain a design model, eliminate the armature inductance from the design model. In other words, neglect high-frequency parasitics represented by L_{a} in the simulation model. For this set L_{a} = 0, compute x_{3} from the equation obtained by setting L_{a} = 0 and substitute the result into the second of the state equations. The result will be a second-order design model. Represent the design model in the form

ẋ = Ax + b(u + ξ),

where ξ = 9.8 sin(x_{1}) represents the model nonlinearity. Use the design model to construct:

(a) the sliding surface, σ = sx = 0, so that the design model restricted to this surface will have a pole at -3; (b) construct a control law of the form

u = −k_{1}x_{1}sign(x_{1}σ) − k_{2}x_{2}sign(x_{2}σ)

so that σσ̇ < 0 for all x ∈ ℝ^{2} for which σ(x) ≠ 0. While constructing the controller u use the fact that

−1 ≤ \frac{sin(x_{1})}{x_{1}} ≤ 1.

(c) Apply the above control law to the simulation model in which L_{a} = 10 mH. Plot x_{1}, x_{2} and x_{3} versus time and compare these plots with the plots of x_{1} and x_{2} versus time when the above controller was applied to the design model. Investigate a destabilizing effect of increasing L_{a} on the behavior of the closed-loop system composed of the simulation model and the control law from part (b).

Exercise 11.3 Let S be an open set in ℝ^{n} and let f : S → ℝ^{n}, where

f(x) = [ f_{1}(x) f_{2}(x) ⋯ f_{n}(x) ]^{⊤} .

Recall that if all partial derivatives of all functions f_{i} exist and are continuous in S, then we say that f is a C^{1} function, map, in S. We say that f is a C^{p} function, map, in S if each component f_{i} is continuous and has continuous partial derivatives in S for all k = 1, 2, …, p.

Let f : ℝ^{n} → ℝ^{n} be a C^{p}-map, p > 0, on an open subset S of ℝ^{n}, and let a be a point of S. We say that f is locally C^{p}-invertible at a if there is an open set U ⊂ S containing a and an open set V containing f(a) such that f : U → V has a C^{p} inverse h : V → U so that for all x ∈ U,

h(f(x)) = x.

Figure 11.3 illustrates the sets U and V for one variable case. The Jacobian matrix of f : ℝ^{n} → ℝ^{n}, denoted J(x), is defined as

J(x) = Df(x) = [ ∇f_{1}(x)^{⊤} ; ∇f_{2}(x)^{⊤} ; ⋮ ; ∇f_{n}(x)^{⊤} ] = [ Df_{1}(x) ; Df_{2}(x) ; ⋮ ; Df_{n}(x) ] .

Prove the inverse function theorem:

Theorem 11.6 Let f : ℝ^{n} → ℝ^{n} be a C^{p}-map, p > 0, on an open subset S of ℝ^{n}, and let a be a point of S such that the Jacobian matrix J(a) is invertible, and b = f (a). Then, there exist open sets U and V in ℝ^{n} such that a ∈ U, b ∈ V, f is one-to-one on U, f(U) = V, and f is locally C^{p}-invertible at a.

Figure 11.3: An illustration of the sets U and V that appear in the inverse function theorem. Exercise 11.4 For the dynamical system model

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}x_{1}x_{2} + x_{2} \\ x_{1} + x_{2} \end{matrix} ] + [ \begin{matrix}1 \\ 0 \end{matrix} ] u,

(i) Construct a smooth state-space transformation, z = T (x), that transforms the system into the controller form;

(ii) Transform the system into the controller form; (iii) In the z-coordinates, design a linearizing state-feedback controller so that the linear

closed-loop system has poles at −2 ± 2j; (iv) Represent the control law in the x-coordinates. Exercise 11.5 Consider the following model of a dynamical system:

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}x_{1}^{2} + x_{2} \\ x_{1}^{3} + x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ 1 \end{matrix} ] u.

(i) Transform the system into the controller form; (ii) Design the linearizing state-feedback controller such that the closed-loop system in the

new coordinates has its poles at −1 and −2; (iii) Linearize the closed-loop system about x = 0 and find the poles of the linearized

model;

(iv) Perform numerical simulations of the dynamical behavior of the closed-loop system for a few initial conditions. Plot x_{1} versus time as well as x_{2} versus time.

Exercise 11.6 Consider the following model of a dynamical system,

ẋ = [ \begin{matrix}x_{2} + 2x_{2}^{2} \\ 2x_{2} \end{matrix} ] + [ \begin{matrix}x_{2} \\ 1 \end{matrix} ] u.

(i) Construct a smooth state-space transformation z = T (x) that transforms the system into the controller form, then transform the system into the controller form;

(ii) In the z-coordinates, design a linearizing state-feedback controller so that the linear closed-loop system has poles at −2 ± 2j, then represent the control law in the xcoordinates;

(iii) Perform numerical simulations of the dynamical behavior of the closed-loop system for a few initial conditions. Plot x_{1} versus time as well as x_{2} versus time.

Exercise 11.7 Consider a dynamical system modeled by the equations,

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}−x_{1} \\ x_{1} − x_{2} − x_{2}^{3} \end{matrix} ] + [ \begin{matrix}1 \\ 0 \end{matrix} ] u,
y = x_{2}.

(i) Find a smooth state-space transformation z = T (x) that brings the above system model into the controller form;

(ii) Find the system’s equations in the new coordinates;

(iii) Find a smooth state-space transformation x = T (z) that brings the above system model into the observer form;

(iv) Find the system’s observer form.

Exercise 11.8 Consider a dynamical system modeled by the following equations:

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}x_{1}^{2} + x_{2} \\ x_{1}^{3} + x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ 1 \end{matrix} ] u
y = [ 1 0 ] [ \begin{matrix}x_{1} \\ x_{2} \end{matrix} ] .

Note that this is the same plant as in Exercise 11.5 plus the output map.

(i) Transform the above model into the observer form;

(ii) Design an observer so that the poles of the linearized error dynamics are located at −3 and −4;

(iii) Use the control law developed in Exercise 11.5 and the observer from the previous part of this exercise to construct a combined controller-observer compensator. Simulate the behavior of the closed-loop system for a few different initial conditions. Note that because the state variables of the observer are available to you, you can always set zero initial conditions on the observer if you wish to do so. Plot the output of the plant versus time.

612 CHAPTER 11. VECTOR FIELD METHODS

Unknown Input Observer (UIO) Design

The best piece of advice I got before I wrote this book was from a colleague of mine, David Ignatius, who is editor of the “Outlook” section at The Washington Post. He had written a novel about the Middle East [SIRO], a very good novel. He said, “When you’re writing this book always imagine whether the reader is going to want to turn the page. Whether what you’re writing, how you’re writing, and the anecdotes you’re telling will compel that reader to want to turn the page.” That’s what I tried to keep in mind. Hence the book is a really a combination of analysis of history, of autobiography, of anecdote, put into my own kind of Cuisinart.” Thomas Friedman in [202, p. 288]

In this Chapter, we present design procedures for full- and reduced-order observers for a class of dynamical systems modeled by a triple (A, B, C) where some or all of the inputs are unknown. The unknown input can be a combination of unmeasurable or unmeasured disturbances, unknown control action, and unmodeled system dynamics. The first design method uses a projection operator approach to the state estimation where the state of the system, whose state is to be estimated, is decomposed into known and unknown components. The unknown component is, in general, a skew projection, that is, not necessarily orthogonal, of the whole state along the subspace in which the available state component resides. We then construct a dynamical system to estimate the unknown component. Finally, we combine the output of the dynamical system, which estimates the unknown state component, with the available state information to obtain the observer that estimates the whole state. In the second design method we employ a sliding mode approach combined with the second method of Lyapunov. We include design algorithms and illustrate the results with numerical examples. Portions of this Chapter are based on our paper [151].

12.1 Introduction

Since the early developments of observers for systems without uncertainties, observers for plants with both known and unknown inputs have been developed resulting in the so-called unknown input observer (UIO) architectures, such as, for example, in [141, 346, 313, 32, 197, 198, 360, 142, 148, 71, 57, 68, 56, 143]. Barmish and Galimidi [22] showed that robust stabilization of uncertain systems via the Luenberger observer is possible provided the uncertain parameters do not exceed a certain computable threshold. More recently, observer architectures utilizing the concept of sliding modes were proposed for uncertain systems, see, for example, [339, 338, 340, 146, 369, 367, 370, 92, 330, 365, 123, 186]. Other methods of the observer design for linear systems developed prior 1983 are reported by O’Reilly in [258].

Observers for uncertain systems play an essential role in robust model-based fault detection [92, 56, 93, 291, 162]. The basic idea behind the use of the observers for fault detection is to form residuals from the difference between the actual system outputs and the estimated outputs using an observer. Once a fault occurs, the residuals are expected to react by becoming greater than a pre-specified threshold. When the system under consideration is subject to unknown disturbances or unknown inputs, their effect has to be decoupled from the residuals to avoid false alarms.

12.2 Uncertain System Model

The class of uncertain dynamical systems that we consider is modeled by

ẋ = Ax + Bu  (12.1)
y = Cx,  (12.2)

where the input matrix B ∈ ℝ^{n×m} and the output matrix C ∈ ℝ^{p×n}. We assume that the model parameters (A, B, C) are known. We further assume that some or all of the inputs are unknown and that the last m_{2} inputs are unknown and the remaining m_{1} = m − m_{2} inputs are known. We partition the input matrix B corresponding to the known and unknown inputs as

B = [ B_{1} B_{2} ] ,

where B_{1} ∈ ℝ^{n×m_{1}} and B_{2} ∈ ℝ^{n×m_{2}}. Let

u = [ \begin{matrix}u & 1 \\ u & 2 \end{matrix} ] .

Then, the system model (12.1) can be represented as

ẋ = Ax + B_{1}u_{1} + B_{2}u_{2}  (12.3)

The vector function u_{2} may also model lumped uncertainties or nonlinearities in the plant. The output matrix C ∈ ℝ^{p×n}. We assume that the pair (A, C) is detectable.

12.3 A Projection Operator Approach

12.3.1 Background Results

In this section we employ a projection operator approach to the problem of state observation of uncertain systems. In our further discussion, we assume that the matrix B_{2} has full column rank. We begin our presentation by noticing that because the system output y is known, it would seem reasonable to decompose the state x as

x = (I − MC)x + MCx
= (I − MC)x + My,  (12.4)

where M is an n×p real matrix, and the unknown part of the decomposition is (I −MC)x. Let q = (I − MC)x, then x = q + My, and we have

q̇ = (I − MC)ẋ
= (I − MC)(Ax + B_{1}u_{1} + B_{2}u_{2})
= (I − MC)(Ax + B_{1}u_{1}) + (I − MC)B_{2}u_{2}
= (I − MC)(Aq + AMy + B_{1}u_{1}) + (I − MC)B_{2}u_{2}.

If M is chosen so that (I − MC)B_{2} = O, then the dynamics of q depend only on the known quantities u_{1} and y:

q̇ = (I − MC)(Aq + AMy + B_{1}u_{1}).  (12.5)

Note that if we start the above dynamical system with the initial condition q(0) = (I − MC)x(0), then x = q + MCx = q + My for all t ≥ 0. But since x(0) is assumed to be unknown, then x̃ = q + My is only an approximation of x. To improve convergence rate or to ensure convergence, we add an extra term to the right-hand side of (12.5) to obtain

q̇ = (I − MC)(Aq + AMy + B_{1}u_{1} + L(y − Cq − CMy))
= (I − MC)(Aq + AMy + B_{1}u_{1} + LC(x − q − My)).

Let e = x − x̃. We will show that

ė = (I − MC)(A − LC)e

and e(t) → 0 as t → ∞ under mild conditions.

Since

rank (MCB_{2}) ≤ rank (CB_{2}) ≤ rank (B_{2}),

the equality (I − MC)B_{2} = O makes it necessary that rank (CB_{2}) = rank(B_{2}), which we assume throughout the Chapter. This rank condition also implies that there must be at least as many independent outputs as unknown inputs for the method to work.

For the error dynamics matrix (I − MC)(A − LC) to be asymptotically stable, it is not sufficient for (A − LC) to be asymptotically stable. It is possible for a product of a projection matrix and an asymptotically stable matrix to be unstable as the following simple example shows.

Example 12.1 Let

Π = [ \begin{matrix}1 & 0 \\ 0 & 0 \end{matrix} ] and A = [ \begin{matrix}1 & −3 \\ 3 & −2 \end{matrix} ] .

It is easy to check that A is asymptotically stable while ΠA is unstable. Furthermore, the system ẋ = Ax restricted to the range of Π is governed by ż = z, which is also unstable in this case.

We analyze convergence properties of the proposed full-order observer and then we use the results of our analysis to propose a new type of reduced-order observer for uncertain systems. Consider the following dynamical system model,

q̇ = (I − MC)(Aq + AMy + B_{1}u_{1} + L(y − Cq − CMy))  (12.6)
x̃ = q + My.  (12.7)

We will now show that x̃ → x as t → ∞. To this end let

e(t) = x(t) − x̃(t)

denote the estimation error. Then, using (I − MC)B_{2} = O and y = Cx, we obtain

\frac{de}{dt} = \frac{d}{dt} (x − x̃)
= \frac{d}{dt} (x − q − MCx)
= \frac{d}{dt} ((I − MC)x − q)
= (I − MC)(Ax + B_{1}u_{1} + B_{2}u_{2})
−(I − MC)(Aq + AMy + B_{1}u_{1} + L(y − Cq − CMy))
= (I − MC)(Ax + B_{1}u_{1}) + (I − MC)B_{2}u_{2}
−(I − MC)(Aq + AMCx + B_{1}u_{1} + L(Cx − Cq − CMCx))
= (I − MC)(A − LC)(x − q − MCx)
= (I − MC) (A − LC) e.  (12.8)

Our objective is to specify M and L and a set of initial conditions so that e(t) → 0 as t → ∞. A particular class of solutions to (I − MC)B_{2} = O is given by

M = B_{2} ((CB_{2})^{†} + H_{0} (I_{p} − (CB_{2})(CB_{2})^{†})) ,

where the superscript † denotes the Moore-Penrose pseudo-inverse operation and H_{0} ∈ ℝ^{m_{2}×p} is a design parameter matrix. (See, for example, Kaczorek [165, Section 1.5] or [62, Section 12.5] for more information on the subject of pseudo-inverse matrices.) Because, by assumption, rank (CB_{2}) = rank B_{2} and B_{2} has full rank, we have (CB_{2})^{†}(CB_{2}) = I_{m_{2}}. If CB_{2} is a square matrix, then CB_{2} is invertible by assumption and the above M reduces to B_{2}(CB_{2})^{−1}. Furthermore, it is easy to check that for the above class of M, the product MC is a projection (not necessarily orthogonal):

(MC)^{2} = MC.

It follows that

Π̃ = I − MC

is also a projection.

To proceed further, we need the following lemma.

Lemma 12.1 Let Π̃ : ℝ^{n} → ℝ^{n} be a projection, that is, Π̃^{2} = Π̃, and let rank Π̃ = n − m_{2}. Then Π̃ has (n − m_{2}) eigenvalues equal to 1 while the remaining m_{2} eigenvalues are located at 0 and there is a basis of ℝ^{n} in which the matrix Π̃ relative to this basis has the form

Π = [ \begin{matrix}I_{n−m_{2}} & O \\ O & O \end{matrix} ] ,

that is, there is an invertible matrix T whose columns are eigenvectors of Π̃ such that

T^{−1}Π̃T = Π = [ \begin{matrix}I_{n−m_{2}} & O \\ O & O \end{matrix} ] .  (12.9)

Proof See Smith [304, pp. 156–158 and pp. 194–195].

□

In the next Section, we provide a numerically efficient method for constructing T such that (12.9) holds; see (12.19).

12.3.2 Constructing Full-Order UIO

We begin this section by introducing the following coordinate transformation,

ẽ = T^{−1}e,  (12.10)

where the transformation matrix T is obtained, using Lemma 12.1, from the representation of the projection operator Π̃ in the form

Π̃ = T ΠT^{−1}.  (12.11)

Applying the coordinate transformation (12.10) to the error equation (12.8) gives

˙ ; ẽ = ΠT^{−1}(A − LC)T ẽ
= Π (T^{−1}AT − (T^{−1}L) (CT)) ẽ.  (12.12)

Let

Ã = T^{−1}AT = [ \begin{matrix}Ã_{11} & Ã_{12} \\ Ã_{21} & Ã_{21} \end{matrix} ] , L̃ = T^{−1}L = [ \begin{matrix}L̃ & 1 \\ L̃ & 2 \end{matrix} ] , C̃ = CT = [ C̃_{1} C̃_{2} ] ,

(12.13) where Ã_{11} ∈ ℝ^{(n−m_{2})×(n−m_{2})}, L̃_{1} ∈ ℝ^{(n−m_{2})×p}, C̃_{1} ∈ ℝ^{p×(n−m_{2})}, and the remaining block submatrices are of appropriate dimensions. Using the above notation, we represent (12.12) in the form,

˙ ; ẽ = Π (Ã − L̃C̃) ẽ
= [ \begin{matrix}I_{n−m_{2}} & O \\ O & O \end{matrix} ] ([ \begin{matrix}Ã_{11} & Ã_{12} \\ Ã_{21} & Ã_{21} \end{matrix} ] − [ \begin{matrix}L̃ & 1 \\ L̃ & 2 \end{matrix} ] [ C̃_{1} C̃_{2} ]) ẽ
= [ \begin{matrix}Ã_{11} − L̃_{1}C̃_{1} & Ã_{12} − L̃_{1}C̃_{2} \\ O & O \end{matrix} ] ẽ  (12.14)

Let

ẽ = [ \begin{matrix}ẽ & 1 \\ ẽ & 2 \end{matrix} ] ,  (12.15)

where ẽ_{1} ∈ ℝ^{n−m_{2}}. Note that ˙ ẽ_{2} = 0. Hence if ẽ_{2}(0) = 0, then ẽ_{2} = 0 for all t ≥ 0. Thus if ẽ_{2} = 0 then

˙ẽ_{1} = (Ã_{11} − L̃_{1}C̃_{1}) ẽ_{1},

and so if ẽ_{2} = 0 and ẽ_{1} → 0 then ẽ → 0. Obviously ẽ_{1} → 0 for arbitrary ẽ_{1}(0) if and only if the matrix (Ã_{11} − L̃_{1}C̃_{1}) is asymptotically stable.

We give a condition on q(0) that guarantees that ẽ_{2} = 0. We have,

MC = I − Π̃
= T (I − Π)T^{−1}.

Hence

[ \begin{matrix}0 \\ ẽ & 2 \end{matrix} ] = (I_{n} − Π)T^{−1}e
= T^{−1}MCe
= T^{−1}MC(x − q − MCx)
= T^{−1}(MCx − MCq − (MC)^{2}x)
= −T^{−1}MCq.  (12.16)

Therefore ẽ_{2}(0) = 0 if and only if MCq(0) = 0, which is equivalent to

q(0) = (I − MC)v

for arbitrary v ∈ ℝ^{n}. In particular q(0) = 0 satisfies the above condition.

In summary, we proved the following theorem,

Figure 12.1: Full-order UIO described in Theorem 12.1.

Theorem 12.1 If the following conditions are satisfied:

1. rank (CB_{2}) = rank B_{2};

2. the pair (Ã_{11}, C̃_{1}) defined in (12.13) is detectable;

3. q(0) = (I − MC)v for arbitrary v ∈ ℝ^{n}, then there exists a gain matrix L such that the estimation error, e = x − x̃, of the full-order observer given by

q̇ = (I − MC)(Aq + AMy + B_{1}u_{1} + L(y − Cq − CMy))
x̃ = q + My

converges to 0 as t → ∞.

In Figure 12.1, we show a block diagram of the full-order UIO of Theorem 12.1. Theorem 12.2 The second condition of Theorem 12.1 that states that the pair (Ã_{11}, C̃_{1}) defined in (12.13) is detectable, is equivalent to

rank [ \begin{matrix}sI_{n} − A & B_{2} \\ C & O \end{matrix} ] = n + m_{2} for all s such that Re(s) ≥ 0.

12.3. A PROJECTION OPERATOR APPROACH 621 Proof We begin the proof by considering the projection matrix, Π̃ = I − MC, where

M = B_{2} ((CB_{2})^{†} + H_{0} (I_{p} − (CB_{2})(CB_{2})^{†})) .

To simplify further analysis, we let F = (CB_{2})^{†} + H_{0} (I_{p} − (CB_{2})(CB_{2})^{†}) and S = F C. Note that

SB_{2} = I_{m_{2}}.  (12.17)

Thus, rank S = m_{2} and so we can find a full rank n × (n − m_{2}) matrix W such that

SW = O.  (12.18)

Combining (12.17) and (12.18) we conclude that [W B_{2}] is invertible. Let

[ W B_{2} ]^{−1} = [ \begin{matrix}W & g \\ N \end{matrix} ] ,

where W^{g} is (n − m_{2}) × n and N is m_{2} × n. Then

N [ W B_{2} ] = [ O I_{m_{2}} ] = S [ W B_{2} ] .

Since [W B_{2}] is full rank, we conclude that N = S. Since

[ \begin{matrix}W & g \\ S \end{matrix} ] [ W B_{2} ] = I_{n},

we have

W^{g}W = I_{n−m_{2}} and W^{g}B_{2} = O.

Let

T = [ W B_{2} ] .  (12.19)

Then

T^{−1}Π̃T = [ \begin{matrix}W & g \\ S \end{matrix} ] (I_{n} − BS) [ W B_{2} ]
= I_{n} − [ \begin{matrix}W & g & B & 2 & S \\ S & B & 2 & S \end{matrix} ] [ W B_{2} ]
= [ \begin{matrix}I_{n−m_{2}} & O \\ O & I_{m_{2}} \end{matrix} ] − [ \begin{matrix}W^{g}B_{2}SW & W^{g}B_{2}SB_{2} \\ SB_{2}SW & SB_{2}SB_{2} \end{matrix} ]
= [ \begin{matrix}I_{n−m_{2}} & O \\ O & I_{m_{2}} \end{matrix} ] − [ \begin{matrix}O & O \\ O & I_{m_{2}} \end{matrix} ]
= [ \begin{matrix}I_{n−m_{2}} & O \\ O & O \end{matrix} ] .  (12.20)

We apply the following coordinate transformation to the system modeled by (12.1) and (12.2):

[ \begin{matrix}z \\ σ \end{matrix} ] = [ \begin{matrix}W & g \\ S \end{matrix} ] x = T^{−1}x.

Then

[ ż ; σ̇ ] = [ \begin{matrix}W^{g}AW & W^{g}AB_{2} \\ SAW & SAB_{2} \end{matrix} ] [ \begin{matrix}z \\ σ \end{matrix} ] + T^{−1}B_{1}u_{1} + [ \begin{matrix}O \\ I & m_{2} \end{matrix} ] u_{2}
= [ \begin{matrix}Ã_{11} & Ã_{12} \\ Ã_{21} & Ã_{22} \end{matrix} ] [ \begin{matrix}z \\ σ \end{matrix} ] + T^{−1}B_{1}u_{1} + [ \begin{matrix}O \\ I & m_{2} \end{matrix} ] u_{2}
y = [ CW CB_{2} ] [ \begin{matrix}z \\ σ \end{matrix} ]
= [ C̃_{1} C̃_{2} ] [ \begin{matrix}z \\ σ \end{matrix} ] .

If the trajectory of the system described by the triple (A, B_{2}, F C) resides in the null space of S, then such a motion is described by

ż = Ã_{11}z.

It follows from 1.5 that the poles of the above system are the zeros of the system described by the triple (A, B_{2}, F C), which are the complex numbers s for which the system matrix

[ \begin{matrix}sI_{n} − A & B_{2} \\ F C & O \end{matrix} ]

loses its normal-rank—see Definition 1.5 on page 45. On the other hand, the zeros of the triple (A, B_{2}, C) are also the zeros of the squared-down system (A, B_{2}, F C), that is, the zeros of (A, B_{2}, C) form a subset of the set of the eigenvalues of Ã_{11}. It is well known that zeros are invariant with respect to similarity transformations. Therefore,

rank [ \begin{matrix}sI_{n−m_{2}} − Ã_{11} & −Ã_{12} & O \\ −Ã_{21} & sI_{m_{2}} − Ã_{22} & I_{m_{2}} \\ C̃_{1} & C̃_{2} & O \end{matrix} ] = rank [ \begin{matrix}sI_{n−m_{2}} − Ã_{11} \\ C̃ & 1 \end{matrix} ] + 2m_{2} for s ∈ ℂ,

which means that zeros of the system (A, B_{2}, C) are in the open left-half plane if and only if the pair (Ã_{11}, C̃_{1}) is detectable.

□

12.3.3 Reduced-Order UIO

The error dynamics of the full-order observer that we analyzed above are given by equation (12.14):

˙ẽ = [ \begin{matrix}Ã_{11} − L̃_{1}C̃_{1} & Ã_{12} − L̃_{1}C̃_{2} \\ O & O \end{matrix} ] ẽ

The reader may have noticed that since we choose the initial condition for q to force ẽ_{2}(t) = 0 for t ≥ 0, the dynamics of the error are completely determined by the dynamics of ẽ_{1}, which are given by

˙ẽ_{1} = (Ã_{11} − L̃_{1}C̃_{1}) ẽ_{1},  (12.21)

an (n − m_{2})-dimensional system. This motivates us to apply the transformation from e to ẽ to q:

q̃ = T^{−1}q.

From equation (12.7),

q̇ = (I − MC)(Aq + AMy + B_{1}u_{1} + L(y − Cq − CMy))

and I − MC = T ΠT^{−1}, we obtain

˙ ; q̃ = Π(T^{−1}AT q̃ + T^{−1}AMy + T^{−1}B_{1}u_{1} + T^{−1}L(y − CT q̃ − CMy))
= ΠT^{−1}AT q̃ + ΠT^{−1}AMy + ΠT^{−1}B_{1}u_{1} + ΠT^{−1}L(y − CT q̃ − CMy)
= Π (T^{−1}AT − T^{−1}LCT ) q̃ + Π (T^{−1}AM + T^{−1}L − T^{−1}LCM) y
+ΠT^{−1}B_{1}u_{1}.  (12.22)

Using the notation defined in (12.13), we have

˙q̃ = Π (Ã − L̃C̃) q̃ + ΠT^{−1} [(AM + T L̃(I_{p} − CM)) y + B_{1}u_{1}] .

Let

q̃ = [ \begin{matrix}q̃ & 1 \\ q̃ & 2 \end{matrix} ] ,

where q̃_{1} ∈ ℝ^{n−m_{2}} and q̃_{2} ∈ ℝ^{m_{2}}. Since

Π = [ \begin{matrix}I_{n−m_{2}} & O \\ O & O \end{matrix} ] ,

Figure 12.2: Reduced-order UIO.

we have ˙ q̃_{2}(t) = 0. Therefore, setting q̃_{2}(0) = 0 ensures that q̃_{2}(t) = 0 for t ≥ 0. We thus can remove m_{2} observer states from the observer dynamics. Let G̃ = AM +T L̃(I_{p} −CM). Then the resulting reduced-order observer takes the form,

˙q̃_{1} = (Ã_{11} − L̃_{1}C̃_{1})q̃_{1} + [ I_{n−m_{2}} O_{m_{2}} ] T^{−1} (G̃y + B_{1}u_{1}) , q̃_{1}(0) = 0
x̃ = T [ \begin{matrix}I & n−m_{2} \\ O & m_{2}×(n−m_{2}) \end{matrix} ] q̃_{1} + My,

where the vector x̃ is the estimate of the plant state x. In Figure 12.2, we show a block diagram of the above reduced-order UIO.

We now summarize the above considerations in the form of the following design algorithm.

Algorithm 12.1 Reduced-Order Unknown Input Observer Design For a given a quadruple of matrices (A, B, C) modeling the plant,

1. Check that rank (CB_{2}) = rank B_{2} If rank (CB_{2}) < rank B_{2}, STOP. The observer does not exist.

2. Compute

M = B_{2} ((CB_{2})^{†} + H_{0} (I_{p} − (CB_{2})(CB_{2})^{†})) ,

where the superscript † denotes the Moore-Penrose pseudo-inverse operation and H_{0} ∈ ℝ^{m_{2}×p} is a design parameter matrix.

3. Compute the projector

Π̃ = I_{n} − MC.

4. Compute T = [ W B_{2} ]—see12.19. Note that

Π̃ = T ΠT^{−1},

where

Π = [ \begin{matrix}I_{n−m_{2}} & O \\ O & O \end{matrix} ] .

5. Compute

Ã = T^{−1}AT = [ \begin{matrix}Ã_{11} & Ã_{12} \\ Ã_{21} & Ã_{22} \end{matrix} ] and C̃ = CT = [ C̃_{1} C̃_{2} ] ,

where Ã_{11} ∈ ℝ^{(n−m_{2})×(n−m_{2})} and C̃_{1} ∈ ℝ^{p×(n−m_{2})}

6. Check detectability of the pair (Ã_{11}, C̃_{1})

If the pair (Ã_{11}, C̃_{1}) is not detectable, STOP. The observer does not exist. Note that if the matrix Ã_{11} is asymptotically stable, then the pair (Ã_{11}, C̃_{1}) is detectable for an arbitrary matrix C̃_{1}.

7. If there are eigenvalues of Ã_{11} that are not asymptotically stable, construct L̃_{1} so that the matrix (Ã_{11} − L̃_{1}C̃_{1}) has its eigenvalues in locations as close to the desired eigenvalues as possible.

8. Form

L̃ = [ \begin{matrix}L̃ & 1 \\ O & m_{2}×p \end{matrix} ] ,

where O_{m_{2}×p} is an m_{2} × p zero matrix.

9. Compute matrix

G̃ = AM + T L̃(I_{p} − CM).

10. Construct the observer,

˙q̃_{1} = (Ã_{11} − L̃_{1}C̃_{1})q̃_{1} + [ I_{n−m_{2}} O_{m_{2}} ] T^{−1} (G̃y + B_{1}u_{1}) , q̃_{1}(0) = 0
x̃ = T [ \begin{matrix}I & n−m_{2} \\ O & m_{2}×(n−m_{2}) \end{matrix} ] q̃_{1} + My.

The vector x̃ is the estimate of the state x.

Example 12.2 We consider the fifth-order lateral axis model of an L-1011 fixed-wing aircraft, with the actuator dynamics neglected, at cruise flight conditions. This model can be found in the book by Edwards and Spurgeon [92, pages 122, 123 and pages 179, 180]. We assume that the inputs to the system are unknown and there are no known inputs. We have,

A = [ \begin{matrix}0.0000 & 0.0000 & 1.0000 & 0.0000 & 0.0000 \\ 0.0000 & −0.1540 & −0.0042 & 1.5400 & 0.0000 \\ 0.0000 & 0.2490 & −1.0000 & −5.2000 & 0.0000 \\ 0.0386 & −0.9960 & −0.0003 & −0.1170 & 0.0000 \\ 0.0000 & 0.5000 & 0.0000 & 0.0000 & −0.5000 \end{matrix} ]

and

B_{2} = [ \begin{matrix}0.0000 & 0.0000 \\ −0.7440 & −0.0320 \\ 0.3370 & −1.1200 \\ 0.0200 & 0.0000 \\ 0.0000 & 0.0000 \end{matrix} ] , C = [ \begin{matrix}0 & 1 & 0 & 0 & −1 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 1 & 0 & 0 & 0 & 0 \end{matrix} ] ,

where u_{2} = [ cos(t) sin(t) ]^{⊤}. We first check that rank (CB_{2}) = rank B_{2}. We then compute the matrix M, where in this example we set H_{0} = O,

M = B_{2}(CB_{2})^{†} = [ \begin{matrix}0.0000 & 0.0000 & 0.0000 & 0.0000 \\ 0.9993 & 0.0000 & −0.0265 & 0.0000 \\ 0.0000 & 1.0000 & 0.0008 & 0.0000 \\ −0.0265 & 0.0008 & 0.0007 & 0.0000 \\ 0.0000 & 0.0000 & 0.0000 & 0.0000 \end{matrix} ] .

Then, the projector Π̃ is

Π̃ = I_{5} − MC = [ \begin{matrix}1.0000 & 0.0000 & 0.0000 & 0.0000 & 0.0000 \\ 0.0000 & 0.0007 & −0.0000 & 0.0265 & 0.9993 \\ 0.0000 & −0.0000 & 0.0000 & −0.0008 & 0.0000 \\ 0.0000 & 0.0265 & −0.0008 & 0.9993 & −0.0265 \\ 0.0000 & 0.0000 & 0.0000 & 0.0000 & 1.0000 \end{matrix} ] .

We next use (12.19) to compute T such that

Π̃ = T ΠT^{−1} = T [ \begin{matrix}I_{3} & O_{3×2} \\ O_{3×2}^{⊤} & O_{2×2} \end{matrix} ] T^{−1}.

We have

T = [ \begin{matrix}−0.7169 & −0.0190 & −0.6969 & 0 & 0 \\ −0.4930 & 0.0131 & 0.5069 & −0.7440 & −0.0320 \\ 0.0000 & −0.0008 & 0.0000 & 0.3370 & −1.1200 \\ −0.0138 & 0.9996 & −0.0131 & 0.0200 & 0 \\ −0.4927 & −0.0135 & 0.5072 & 0 & 0 \end{matrix} ] .

Hence,

Ã_{11} = [ \begin{matrix}−0.0124 & −0.0102 & 0.0145 \\ 0.4662 & −0.0863 & −0.5326 \\ −0.0000 & 0.0139 & −0.0004 \end{matrix} ] and C̃_{1} = [ \begin{matrix}−0.0004 & 0.0265 & −0.0003 \\ 0.0000 & −0.0008 & 0.0000 \\ −0.0138 & 0.9996 & −0.0131 \\ −0.7169 & −0.0190 & −0.6969 \end{matrix} ] .

Note that the pair (Ã_{11}, C̃_{1}) is detectable and that the matrix Ã_{11} is unstable with its eigenvalues at {0.0000, −0.0495 + 0.1041j, −0.0495 − 0.1041j}. We proceed to stabilize Ã_{11}. We select the desired eigenvalues to be located at −3, −4, −5. The gain matrix L̃_{1} such that eig(Ã_{11} − L̃_{1}C̃_{1}) = {−3, −4, −5} is

L̃_{1} = [ \begin{matrix}0.3866 & −0.0110 & 14.5695 & −3.5473 \\ 0.2097 & −0.0060 & 7.9007 & −0.2190 \\ −0.4028 & 0.0115 & −15.1794 & −2.0853 \end{matrix} ] .

We obtained the above gain matrix using MATLAB’s command place. We next form the matrix L̃ by adding two zero rows to L̃_{1} and compute

[ I_{3} O_{3×2} ] T^{−1}G̃ = [ \begin{matrix}−0.0787 & −0.7280 & 14.5813 & −3.5473 \\ −0.8007 & −0.0248 & 7.9275 & −0.2190 \\ 0.1035 & −0.6854 & −15.1934 & −2.0853 \end{matrix} ] .

The reduced-order UIO has the form 628 CHAPTER 12. UNKNOWN INPUT OBSERVER (UIO) DESIGN

Figure 12.3: Plots of x_{i}s and their estimates versus time for Example 12.2.

˙q̃_{1} = (Ã_{11} − L̃_{1}C̃_{1})q̃_{1} + [ I_{3} 0 ] T^{−1}G̃y
= [ \begin{matrix}−2.3538 & −14.6523 & −2.2668 \\ 0.4186 & −7.9939 & −0.5818 \\ −1.7052 & 15.1589 & −1.6523 \end{matrix} ] q̃_{1} + [ \begin{matrix}−0.0787 & −0.7280 & 14.5813 & −3.5473 \\ −0.8007 & −0.0248 & 7.9275 & −0.2190 \\ 0.1035 & −0.6854 & −15.1934 & −2.0853 \end{matrix} ] y

x̃ = T [ \begin{matrix}I & 3 \\ O & 2×3 \end{matrix} ] q̃_{1} + My

= [ \begin{matrix}−0.7169 & −0.0190 & −0.6969 \\ −0.4930 & 0.0131 & 0.5069 \\ 0.0000 & −0.0008 & 0.0000 \\ −0.0138 & 0.9996 & −0.0131 \\ −0.4927 & −0.0135 & 0.5072 \end{matrix} ] q̃_{1} + [ \begin{matrix}0.0000 & 0.0000 & 0.0000 & 0.0000 \\ 0.9993 & 0.0000 & −0.0265 & 0.0000 \\ 0.0000 & 1.0000 & 0.0008 & 0.0000 \\ −0.0265 & 0.0008 & 0.0007 & 0.0000 \\ 0.0000 & 0.0000 & 0.0000 & 0.0000 \end{matrix} ] y.

In Figure 12.3, we show plots of the system state variables and their estimates versus time. The initial conditions of the plant were selected randomly to be equal to

x(0) = [ 0.0342 0.3200 0.0178 −0.0287 −0.2497 ]^{⊤} .

The initial conditions of the observer were set to zero.

12.3.4 Relations With Other UIO Architectures

Our analysis, from the previous sections, can be extended to cover the case

q̇ = (I − MC)(Aq + AMy + B_{1}u_{1}) + L(y − Cq − CMy),  (12.23)

where the term L(y − Cq − CMy) is not premultiplied by (I − MC). This case leads to the observer analyzed in [57, 56] even though the approach there is quite different. Indeed, we can equivalently represent the dynamics of the proposed full-order observer as follows:

q̇ = ((I − MC) A − LC) q + ([(I − MC) A − LC] M + L) y + (I − MC) B_{1}u_{1}
= (T A − LC)q + Ky + T B_{1}u_{1},
x̃ = q + My,

where, using the notation similar to that in [57, 56],

T = I − MC, K_{1} = L,
K_{2} = [T A − LC] M, K = K_{1} + K_{2}.

In addition, the conditions for the existence of the full-order observer presented in [57, 56] and our observers are equivalent.

The observer given by (12.23) is also the same as the one proposed by Yang and Wilde [360] and further analyzed by Darouach, Zasadzinski, and Xu [71]. The connections are as follows:

(i) M is called −E in [360, 71], (ii) (I − MC) corresponds to P there, (iii) B_{1} is B and B_{2} is D in [360, 71], (iv) (I − MC)(A − LC) corresponds to N.

A different approach is proposed by Hou and Müller [142] who first transform the system (12.3) into the form

ẋ = Ax + B_{1}u_{1} + [ \begin{matrix}O \\ I & m_{2} \end{matrix} ] u_{2}
= [ \begin{matrix}A_{11} & A_{12} \\ A_{21} & A_{22} \end{matrix} ] [ \begin{matrix}x & 1 \\ x & 2 \end{matrix} ] + [ \begin{matrix}B & 11 \\ B & 12 \end{matrix} ] u_{1}

+ [ \begin{matrix}O \\ I & m_{2} \end{matrix} ] u_{2}.

Note that x_{1} in the new coordinates is independent of u_{2} and we have

ẋ_{1} = A_{11}x_{1} + A_{12}x_{2} + B_{11}u_{1}.  (12.24)

Let

y = Cx = [ C_{1} C_{2} ] [ \begin{matrix}x & 1 \\ x & 2 \end{matrix} ] ,

where C_{2} ∈ ℝ^{p×m_{2}}. Because by assumption, rank(CB_{2}) = rankB_{2} = m, the submatrix C_{2} has a left inverse, C_{2}^{†}. Hence we can compute

x_{2} = −C_{2}^{†}C_{1}x_{1} + C_{2}^{†}y.  (12.25)

Substituting the above into (12.24) gives

ẋ_{1} = (A_{11} − A_{12}C_{2}^{†}C_{1}) x_{1} + A_{12}C_{2}^{†}y + B_{11}u_{1}.

Hou and Müller [142] propose now to construct an observer for x_{1} using only known signals and then substitute the estimate of x_{1} into (12.25) to obtain an estimate of x_{2}.

12.4 Sliding Mode UIO

12.4.1 Sliding Mode UIO Architecture

We present a sliding mode UIO following the exposition in our paper [151]. We assume that u_{2} is bounded, that is, there exists a nonnegative real number, ρ, such that

∥u_{2}∥ ≤ ρ.

Let x̂ be an estimate of x. Let e denote the estimation error, that is,

e(t) = x(t) − x̂(t).

The observability of (A, C) implies the existence of a matrix L ∈ ℝ^{n×p} such that the matrix (A − LC) has prescribed (symmetric with respect to the real axis) eigenvalues in the open left-half plane. Because (A − LC) is asymptotically stable, for any Q = Q^{⊤} > 0, there is a unique P = P^{⊤} > 0 such that

(A − LC)^{⊤} P + P (A − LC) = −Q.  (12.26)

We choose Q, if possible, so that for some F ∈ ℝ^{m_{2}×p},

F C = B_{2}^{⊤}P .  (12.27)

We need this technical condition to ensure the realizablity of the observer.

To proceed, we define the vector function

E (e, η) = \{ \begin{matrix}η \frac{F Ce}{∥F Ce∥_{2}} & for & F Ce ≠ 0 \\ 0 & for & F Ce = 0, \end{matrix}

where η ≥ ρ is a design parameter. In the case of single-input single-output plant, we can write

E(e, η) = η sign (F Ce) .

We note that

Ce = C(x − x̂) = y − ŷ.

The vector function E(e, η) is an essential ingredient of the sliding mode observer that we present next. When implementing the function E, we use the output measurements y and ŷ, that is, instead of using E(e, η), we utilize

E(y, ŷ, η) = \{ \begin{matrix}η \frac{F (y−y^{}ˆ )}{∥F (y−y^{}ˆ )∥_{2}} & for & F (y − ŷ) ≠ 0 \\ 0 & for & F (y − ŷ) = 0. \end{matrix}

Figure 12.4: Sliding mode observer (12.28) can also be used as an unknown input estimator— see (12.33).

Hence for the case of single-input single-output plant, we have

E(ŷ, e, y) = η sign (F (y − ŷ)) .

Using arguments similar to those found in [339], we can show that the state x̂ of the dynamical system

˙x̂ = Ax̂ + B_{1}u_{1} + L(y − ŷ) + B_{2}E(y, ŷ, η)  (12.28)

for η ≥ ρ, is an asymptotic estimate of the state x of the system described by (12.3) and (12.2), that is,

lim_{t→∞} e(t) = lim_{t→∞} (x(t) − x̂(t)) = 0.

In Figure 12.4, we show a block diagram of the sliding mode observer given by (12.28). To prove the above statement using Lyapunov’s type of arguments, first represent (12.28) as

˙x̂ = (A − LC) x̂ + Ly + B_{1}u_{1} + B_{2}E(e, η).

Then construct the differential equation describing the dynamics of the estimation error e,

ė = ẋ − ˙ x̂ = (A − LC) e + B_{2}u_{2} − B_{2}E(e, η).  (12.29)

Then, show that

\frac{d}{dt} (e^{⊤}P e) = −e^{⊤}Qe < 0,

which implies

lim_{t→∞} e(t) = 0.

It follows from the above that the estimation error is insensitive to the uncertainty modeled by the term B_{2}u_{2}. In summary, the design of the sliding mode UIO for a system modeled by the quadruple (A, B_{1}, B_{2}, C) can be thought of as finding a triple of matrices (L, F , P) satisfying (12.26) and (12.27) for some Q = Q^{⊤} > 0. Edwards and Spurgeon [92] (see also Saif and Xiong [291]) present necessary and sufficient conditions for the existence of the above observer, which are:

(i) rank B_{2} = rank CB_{2} = r;

(ii) the system zeros of the triple (A, B_{2}, C) are in the open left-hand complex plane, that is,

rank [ \begin{matrix}sI_{n} − A & B_{2} \\ C & O \end{matrix} ] = n + r for all s such that Re(s) ≥ 0.

It is interesting to note that the above conditions are also necessary and sufficient for the existence of the observers with unknown inputs analyzed by us in the previous sections as well for the Edwards and Spurgeon UIO discussed later in Section 12.5.

The rank condition,

rank B_{2} = rank CB_{2}

is also referred to as the observer matching condition. This condition can be restrictive in some applications. To construct a sliding mode observer for systems that do not satisfy the observer matching condition, Kalsi et al. [171] propose a method where auxiliary outputs are generated using high-gain approximate differentiators and then employ the auxiliary outputs in the design of a sliding mode observer.

12.4.2 Reconstructing the Unknown Input

We show here how the UIO given by (12.28) can be used to reconstruct the unknown input u_{2}. Consider the manifold,

{e : σ(e) = F Ce = 0} .  (12.30)

We can always choose η such that the above manifold is attractive and it will be reached in a finite time. Thus, after the estimation error trajectory, e, enters a sliding mode along the sliding surface, we have σ(e) = 0 and

σ̇ = F C (A − LC) e + F CB_{2}u_{2} − F CB_{2}E(e, η) = 0.  (12.31)

Because e → 0 as t → ∞, we obtain

F CB_{2}u_{2} = F CB_{2}E(e, η).  (12.32)

Recall that F C = B_{2}^{⊤}P. Assuming that B_{2} is of full column rank implies that the square matrix B_{2}^{⊤}P B_{2} is invertible. Hence, the steady-state condition (12.32) can be re-written as

B_{2}^{⊤}P B_{2}u_{2} = B_{2}^{⊤}P B_{2}E(e, ρ).

Therefore, the unknown input estimate is the output of the nonlinear term, E(e, ρ), that is,

û_{2} = E(e, ρ)  (12.33)

See Figure 12.4 for an illustration of (12.33). For a rigorous proof of the above result, see [171].

12.4.3 Sliding Mode UIO Analysis and Construction

We first present a lemma that will serve us as a design platform of the sliding mode observer for uncertain systems. The lemma is a minor modification of Lemma 1 of Corless and Tu [68] who proved it constructively using a singular value decomposition approach. We offer a different constructive proof using the Q-R decomposition.

Lemma 12.2 For a triple (A, B_{2}, C) ∈ ℝ^{n×n} × ℝ^{n×m_{2}} × ℝ^{p×n},

rank B_{2} = rank (CB_{2}) = r,  (12.34)

if and only if there exist nonsingular matrices T and S such that

T AT^{−1} = [ \begin{matrix}A_{11} & A_{12} \\ A_{21} & A_{22} \end{matrix} ] , T B_{2} = [ \begin{matrix}B & 2_{1} \\ O \end{matrix} ] , SCT^{−1} = [ \begin{matrix}I_{r} & O \\ O & C_{22} \end{matrix} ] ,  (12.35)

where A_{11} ∈ ℝ^{r×r}, A_{22} ∈ ℝ^{(n−r)×(n−r)}, B_{2_{1}} ∈ ℝ^{r×m_{2}}, rank B_{2_{1}} = r, and C_{22} ∈ ℝ^{(p−r)×(n−r)}.

Proof (⟹) The proof is constructive. Using the Q-R decomposition applied to B_{2}, we obtain

B_{2} = Q_{B_{2}}R_{B_{2}},

where Q_{B_{2}} ∈ ℝ^{n×n} is an unitary matrix and the matrix R_{B_{2}} ∈ ℝ^{n×m_{2}} is upper triangular where

rank R_{B_{2}} = r.

Let T_{1} = Q_{B_{2}}^{−1}. Then, we obtain

T_{1}B_{2} = [ \begin{matrix}B̃ & 2_{1} \\ O \end{matrix} ] ,

12.4. SLIDING MODE UIO 635 where G̃_{1} ∈ ℝ^{r×m_{2}}. We next partition the matrix CT _{1}^{−1} as follows,

CT _{1}^{−1} = [ C̃_{1} C̃_{2} ] ,

where C̃_{1} ∈ ℝ^{p×r}. Note that

CB_{2} = (CT _{1}^{−1}) (T_{1}B_{2}) = C̃_{1}B̃_{2_{1}}.

By the lemma hypothesis, rank B_{2} = rank (CB_{2}) = r. Hence

rank C̃_{1} = r.

Applying the Q-R decomposition to C̃_{1} yields

C̃_{1} = QC̃_{1}RC̃_{1},

where

RC̃_{1} = [ \begin{matrix}C & 11 \\ O \end{matrix} ] and det C_{11} ≠ 0.

Note that C_{11} ∈ ℝ^{r×r}. Let S = Q−1 ; C̃_{1}, then,

SCT _{1}^{−1} = [ \begin{matrix}C_{11} & C_{12} \\ O & C_{22} \end{matrix} ] .

Postmultiplying SCT _{1}^{−1} by

T _{2}^{−1} = [ \begin{matrix}C_{11}^{−1} & −C_{11}^{−1}C_{12} \\ O & I_{n−r} \end{matrix} ]

gives

SCT _{1}^{−1}T _{2}^{−1} = [ \begin{matrix}C_{11} & C_{12} \\ O & C_{22} \end{matrix} ] [ \begin{matrix}C_{11}^{−1} & −C_{11}^{−1}C_{12} \\ O & I_{n−r} \end{matrix} ] = [ \begin{matrix}I_{r} & O \\ O & C_{22} \end{matrix} ] .

We then have T = T_{2}T_{1}.

(⟸) By inspection.

□

We next show that the systems

\begin{matrix}ẋ & = & Ax + B_{2}u_{2} \\ y & = & Cx \end{matrix} \}

and

\begin{matrix}˙ ; x̃ & = & T AT^{−1}x̃ + T B_{2}u_{2} \\ ỹ & = & SCT^{−1}x \end{matrix} \}

have the same system zeros, that is, their system matrices have the same rank for all s ∈ ℂ, where ℂ is the set of complex numbers. To prove the above statement, we note the following relation between the system matrices of the above models:

[ \begin{matrix}sI_{n} − T AT^{−1} & T B_{2} \\ SCT^{−1} & O \end{matrix} ] = [ \begin{matrix}T & O \\ O & S \end{matrix} ] [ \begin{matrix}sI_{n} − A & B_{2} \\ C & O \end{matrix} ] [ \begin{matrix}T^{−1} & O \\ O & I_{q} \end{matrix} ] .  (12.36)

Suppose now that we have two matrices, X ∈ ℝ^{r×s} and Y ∈ ℝ^{s×t}. Then, we have the following bounds on the rank of the product of these matrices:

rank X + rank Y − s ≤ rank (XY ) ≤ min(rank X, rank Y ).

The above are referred to as the Sylvester’s inequalities—see, for example, Gantmacher [110, pages 65, 66] for their proof or Section A.10 and Exercise A.15 in the Appendix. Applying the Sylvester’s inequalities to (12.36), we conclude that

rank [ \begin{matrix}sI_{n} − T AT^{−1} & T B_{2} \\ SCT^{−1} & O \end{matrix} ] = rank [ \begin{matrix}sI_{n} − A & B_{2} \\ C & O \end{matrix} ] for all s ∈ ℂ.  (12.37)

Lemma 12.3 Assume that rank B_{2} = rank (CB_{2}) = r. Then, the pair (A_{22}, C_{22}) is detectable if and only if

rank [ \begin{matrix}sI_{n} − A & B_{2} \\ C & O \end{matrix} ] = n + r for all s such that Re(s) ≥ 0.  (12.38)

Proof By assumption, rank B_{2} = rank (CB_{2}) = r. By Lemma 12.2, the above condition is equivalent to the existence of nonsingular matrices T and S such that

T AT^{−1} = [ \begin{matrix}A_{11} & A_{12} \\ A_{21} & A_{22} \end{matrix} ] , T B_{2} = [ \begin{matrix}B & 2_{1} \\ O \end{matrix} ] , SCT^{−1} = [ \begin{matrix}I_{r} & O \\ O & C_{22} \end{matrix} ] ,

where B_{2_{1}} ∈ ℝ^{r×m_{2}} and rank B_{2_{1}} = r. Then, by (12.37) for any s ∈ ℂ,

rank [ \begin{matrix}sI_{r} − A_{11} & −A_{12} & B_{2_{1}} \\ −A_{21} & sI_{n−r} − A_{22} & O \\ I_{r} & O & O \\ O & C_{22} & O \end{matrix} ] = rank [ \begin{matrix}−A_{12} & B_{2_{1}} \\ sI_{n−r} − A_{22} & O \\ C_{22} & O \end{matrix} ] + r

= rank [ \begin{matrix}sI_{n−r} − A_{22} \\ C & 22 \end{matrix} ] + 2r.

It follows from the above that the pair (A_{22}, C_{22}) is detectable if and only if the rank condition (12.38) holds.

□

Note that if m_{2} = p = r, then rank [ \begin{matrix}sI_{n} − A & B_{2} \\ C & O \end{matrix} ] = n+r for all s such that Re(s) ≥ 0

if and only if the matrix A_{22} is asymptotically stable.

To proceed, we need the following technical lemma.

Lemma 12.4 For a given matrix V ∈ ℝ^{n×m},

ker(V ) = ker(V^{⊤}V ).

Proof See Exercise 12.5.

□

The following theorem appears in Corless and Tu [68, Lemma 3, page 760]. A related result was obtained by Edwards and Spurgeon [92, Proposition 6.2, page 138]. An algorithm for constructing matrices L, F, and P that are essential ingredients of the sliding mode observer for the uncertain systems is contained in the proof of the theorem.

Theorem 12.3 There exists a triple of matrices (L, F , P) ∈ ℝ^{n×p} × ℝ^{m_{2}×p} × ℝ^{n×n} such that

(A − LC)^{⊤}P + P (A − LC) < 0  (12.39)

and

F C = B_{2}^{⊤}P  (12.40)

if and only if

(i) rank B_{2} = rank (CB_{2}) = r;

(ii) the system zeros of the triple (A, B_{2}, C) are in the open left-hand complex plane, that is,

rank [ \begin{matrix}sI_{n} − A & B_{2} \\ C & O \end{matrix} ] = n + r for all s such that Re(s) ≥ 0.

Proof (⟸) We follow the arguments of Corless and Tu [68]. By Lemma 12.2, the condition, rank B_{2} = rank (CB_{2}) = r, is equivalent to the existence of nonsingular matrices T and S such that

Â = T AT^{−1} = [ \begin{matrix}A_{11} & A_{12} \\ A_{21} & A_{22} \end{matrix} ] , B̂_{2} = T B_{2} = [ \begin{matrix}B & 2_{1} \\ O \end{matrix} ] , Ĉ = SCT^{−1} = [ \begin{matrix}I_{r} & O \\ O & C_{22} \end{matrix} ] ,

where B_{2_{1}} ∈ ℝ^{r×m_{2}} and rank B_{2_{1}} = r. Let

P̂ = T^{−⊤}P T^{−1}, L̂ = T LS^{−1}, and F̂ = F S^{−1}.  (12.41)

To proceed, note that condition (ii) is equivalent to the existence of a matrix L_{22} such that the eigenvalues of (A_{22} − L_{22}C_{22}) are all in the open left-half complex plane. Then, for any symmetric positive positive definite Q_{22}, the symmetric solution P_{22} to the Lyapunov matrix equation,

(A_{22} − L_{22}C_{22})^{⊤}P_{22} + P_{22}(A_{22} − L_{22}C_{22}) = −Q_{22},

is also positive definite. Let

L̂ = [ \begin{matrix}κI_{r} & O \\ O & L_{22} \end{matrix} ] ,

where κ > 0 is a design parameter whose lower bound is determined in the following considerations. We have

Â − L̂Ĉ = [ \begin{matrix}A_{11} & A_{12} \\ A_{21} & A_{22} \end{matrix} ] − [ \begin{matrix}κI_{r} & O \\ O & L_{22} \end{matrix} ] [ \begin{matrix}I_{r} & O \\ O & C_{22} \end{matrix} ]
= [ \begin{matrix}A_{11} − κI_{r} & A_{12} \\ A_{21} & A_{22} − L_{22}C_{22} \end{matrix} ] .

Let

P̂ = [ \begin{matrix}I_{r} & O \\ O & P_{22} \end{matrix} ] and Q̂ = [ \begin{matrix}Q_{11} & Q_{12} \\ Q_{12}^{⊤} & Q_{22} \end{matrix} ] .

Using the above, we obtain

−Q̂ = (Â − L̂Ĉ)^{⊤}P̂ + P̂ (Â − L̂Ĉ)
= [ \begin{matrix}A_{11}^{⊤} − κI_{r} & A_{21}^{⊤} \\ A_{12}^{⊤} & (A_{22} − L_{22}C_{22})^{⊤} \end{matrix} ] [ \begin{matrix}I_{r} & O \\ O & P_{22} \end{matrix} ]

+ [ \begin{matrix}I_{r} & O \\ O & P_{22} \end{matrix} ] [ \begin{matrix}A_{11} − κI_{r} & A_{12} \\ A_{21} & A_{22} − L_{22}C_{22} \end{matrix} ]

= [ \begin{matrix}A_{11}^{⊤} − κI_{r} & A_{21}^{⊤}P_{22} \\ A_{12}^{⊤} & (A_{22} − L_{22}C_{22})^{⊤}P_{22} \end{matrix} ] + [ \begin{matrix}A_{11} − κI_{r} & A_{12} \\ P_{22}A_{21} & P_{22}(A_{22} − L_{22}C_{22}) \end{matrix} ]
= [ \begin{matrix}A_{11}^{⊤} + A_{11} − 2κI_{r} & A_{21}^{⊤}P_{22} + A_{12} \\ A_{12}^{⊤} + P_{22}A_{21} & (A_{22} − L_{22}C_{22})^{⊤}P_{22} + P_{22}(A_{22} − L_{22}C_{22}) \end{matrix} ]
= [ \begin{matrix}−Q_{11} & −Q_{12} \\ −Q_{12}^{⊤} & −Q_{22} \end{matrix} ] ,  (12.42)

where Q_{22} = − ((A_{22} − L_{22}C_{22})^{⊤}P_{22} + P_{22}(A_{22} − L_{22}C_{22})) is positive definite by construction. Our goal is to obtain a lower bound on the parameter κ that would yield a positive definite Q̂. Using the Schur complement of the positive definite Q_{22}, we have that Q̂ is positive definite if and only if

Q_{11} > Q_{12}Q_{22}^{−1}Q_{12}^{⊤}.

Employing the above in (12.42), we obtain

2κI_{r} − (A_{11}^{⊤} + A_{11}) > (A_{21}^{⊤}P_{22} + A_{12}) Q_{22}^{−1} (A_{12}^{⊤} + P_{22}A_{21}) .

Hence, Q̂ is positive definite if

κ > \frac{1}{2}λ_{max} (A_{11}^{⊤} + A_{11} + (A_{21}^{⊤}P_{22} + A_{12}) Q_{22}^{−1} (A_{12}^{⊤} + P_{22}A_{21})) .

Let

F̂ = [ B_{2_{1}}^{⊤} O ] .

Then, it is easy to see that

(Â − L̂Ĉ)^{⊤}P̂ + P̂ (Â − L̂Ĉ) < 0,  (12.43)

and

F̂ Ĉ = B̂_{2}^{⊤}P̂ ,  (12.44)

which are conditions (12.39) and (12.40) in the new basis. Hence, the proof of the sufficiency conditions for the existence of the desired triple of matrices (L, F , P) is complete. (⟹) We assume that there exists a triple of matrices (L, F , P) such that the conditions

(A − LC)^{⊤}P + P (A − LC) < 0 and F C = B_{2}^{⊤}P

are satisfied. Because P is symmetric positive definite, it can be represented as

P = P^{\frac{1}{2}} P^{\frac{1}{2}} ,

where P^{\frac{1}{2}} ∈ ℝ^{n×n} is also symmetric positive definite and hence nonsingular. We refer to P^{\frac{1}{2}} as a square root of P —see, for example, Luenberger [224, page 459] or Section A.4 in the Appendix. We then have

B_{2}^{⊤}P B_{2} = B_{2}^{⊤}P^{\frac{1}{2}} P^{\frac{1}{2}} B_{2} = (P^{\frac{1}{2}} B_{2})^{⊤} (P^{\frac{1}{2}} B_{2}) .

By Lemma 12.4 and the fact that P^{\frac{1}{2}} is nonsingular,

ker (B_{2}^{⊤}P B_{2}) = ker (P^{\frac{1}{2}} B_{2}) = ker(B_{2}).

Therefore,

rank (B_{2}^{⊤}P B_{2}) = rank B_{2}.

Using the above, the condition, B_{2}^{⊤}P = F C, and the Sylvester’s inequality, we obtain

rank B_{2} = rank (B_{2}^{⊤}P B_{2}) = rank (F CB_{2}) ≤ rank (CB_{2}) ≤ rank B_{2}.

Hence,

rank (CB_{2}) = rank B_{2}.

Our goal now is to show that conditions (12.39) and (12.40) imply that the system zeros of the triple (A, B_{2}, C) are in the open left-hand complex plane. Because rank (CB_{2}) = rank B_{2}, by Lemma 12.2, we can construct transformation matrices T and S so that the transformed matrices A, B_{2}, and C take the form given by (12.35). By Lemma 12.3, the system zeros of the triple (A, B_{2}, C) are in the open left-hand complex plane if and only the pair (A_{22}, C_{22}) is detectable. Let matrices P̂, L̂, and F̂ be defined by (12.41), where

P̂ = [ \begin{matrix}P_{11} & P_{12} \\ P_{21} & P_{22} \end{matrix} ] , L̂ = [ \begin{matrix}L_{11} & L_{12} \\ L_{21} & L_{22} \end{matrix} ] , and F̂ = [ F_{1} F_{2} ] .

Because P̂ is symmetric positive definite, the submatrix P_{22} ∈ ℝ^{(n−r)×(n−r)} is symmetric positive definite and hence invertible. We write down the corresponding 2-by-2 block equation of inequality (12.43),

(A_{12} − L_{12}C_{22})^{⊤}P_{12} + (A_{22} − L_{22}C_{22})^{⊤}P_{22} + P_{21}(A_{12} − L_{12}C_{22}) + P_{22}(A_{22} − L_{22}C_{22}) < 0.

(12.45)

Equation (12.44) yields

F_{2}C_{22} = B_{2_{1}}^{⊤}P_{12},

where B_{2_{1}} has a full row rank. Therefore, B_{2_{1}}B_{2_{1}}^{⊤} ∈ ℝ^{r×r} is invertible and we have

P_{12} = (B_{2_{1}}B_{2_{1}}^{⊤})^{−1} B_{2_{1}}F_{2}C_{22}.  (12.46)

Note that

P_{21} = P _{12}^{⊤} = C_{22}^{⊤}F _{2}^{⊤}B_{2_{1}}^{⊤} (B_{2_{1}}B_{2_{1}}^{⊤})^{−1} .

Substituting the above into (12.45), performing manipulations and denoting

L̃_{22} = L_{22} − P _{22}^{−1}(A_{12} − L_{12}C_{22})^{⊤} (B_{2_{1}}B_{2_{1}}^{⊤})^{−1} B_{2_{1}}F_{2},

we obtain

(A_{22} − L̃_{22}C_{22})^{⊤}P_{22} + P_{22}(A_{22} − L̃_{22}C_{22}) < 0.

Because P_{22} is symmetric and positive definite, the above implies that (A_{22} − L̃_{22}C_{22}) is asymptotically stable, which in turn implies that the pair (A_{22}, C_{22}) is detectable. The proof of the theorem is now complete.

□

We now summarize the above considerations in the form of the following design algorithm.

Algorithm 12.2 Sliding-Mode UIO Design

Given a quadruple of matrices (A, B_{1}, B_{2}, C) modeling the plant.

1. Check that the rank condition, rank (CB_{2}) = rank B_{2}, is satisfied.

If rank (CB_{2}) ≠ rank B_{2}, the sliding-mode observer cannot be constructed, STOP.

2. Transform the triple (A, C, B_{2}).

Use the method of the proof of Lemma 12.2 to construct nonsingular matrices T and S and compute

T AT^{−1} = [ \begin{matrix}A_{11} & A_{12} \\ A_{21} & A_{22} \end{matrix} ] , T B_{2} = [ \begin{matrix}B & 2_{1} \\ O \end{matrix} ] , SCT^{−1} = [ \begin{matrix}I_{r} & O \\ O & C_{22} \end{matrix} ] ,

where B_{2_{1}} ∈ ℝ^{r×m_{2}} and rank B_{2_{1}} = r.

3. Check detectability of (A_{22}, C_{22}).

If the pair (A_{22}, C_{22}) is not detectable, the sliding-mode observer cannot be constructed, STOP.

4. Construct a matrix L_{22} so that the eigenvalues of (A_{22} − L_{22}C_{22}) are in the open left-half plane.

5. Choose a positive definite Q_{22} ∈ ℝ^{(n−r)×(n−r)} and solve for positive definite P_{22} the Lyapunov matrix equation,

(A_{22} − L_{22}C_{22})^{⊤}P_{22} + P_{22}(A_{22} − L_{22}C_{22}) = −Q_{22}.

6. Choose κ that satisfies the condition,

κ > \frac{1}{2}λ_{max} (A_{11}^{⊤} + A_{11} + (A_{21}^{⊤}P_{22} + A_{12}) Q_{22}^{−1} (A_{12}^{⊤} + P_{22}A_{21})) .

7. Construct,

L̂ = [ \begin{matrix}κI_{r} & O \\ O & L_{22} \end{matrix} ] , F̂ = [ B_{2_{1}}^{⊤} O ] .

8. Compute

L = T^{−1}L̂S, F = F̂ S.

9. Construct the observer,

˙x̂ = Ax̂ + B_{1}u_{1} + L(y − ŷ) + B_{2}E(y, ŷ, η),

where

E(y, ŷ, η) = \{ \begin{matrix}η \frac{F (y−y^{}ˆ )}{∥F (y−y^{}ˆ )∥_{2}} & for & F (y − ŷ) ≠ 0 \\ 0 & for & F (y − ŷ) = 0. \end{matrix}

Note that Ce = C(x − x̂) = y − ŷ.

Example 12.3 We consider the same fifth-order lateral axis model of an L-1011 fixed-wing aircraft that we considered in Example 12.2. We have rank (CB_{2}) = rank B_{2}. We then use Lemma 12.2 to compute the transformation matrices T and S,

T = [ \begin{matrix}−0.0000 & −0.9106 & 0.4125 & 0.0245 & 0.9106 \\ 0.0000 & −0.4124 & −0.9110 & 0.0103 & 0.4124 \\ −0.9998 & −0.0005 & 0.0000 & −0.0179 & 0 \\ −0.0179 & 0.0265 & −0.0008 & 0.9995 & 0 \\ 0 & 0 & 0 & 0 & 1.0000 \end{matrix} ]

and

S = [ \begin{matrix}−0.9106 & 0.4125 & 0.0245 & −0.0000 \\ −0.4124 & −0.9110 & 0.0103 & 0.0000 \\ 0.0265 & −0.0008 & 0.9996 & −0.0000 \\ 0.0000 & 0.0000 & −0.0000 & 1.0000 \end{matrix} ] .

We next transform the given model into the new coordinates and find the pair(A_{22}, C_{22}), where

A_{22} = [ \begin{matrix}0.0006 & 0.0025 & 0.0178 \\ −0.0368 & −0.0993 & −0.9971 \\ −0.0002 & 0.0133 & −0.0004 \end{matrix} ] and C_{22} = [ \begin{matrix}−0.0179 & 0.9998 & −0.0265 \\ −0.9998 & −0.0179 & −0.0000 \end{matrix} ] .

The pair (A_{22}, C_{22}) is detectable and we use MATLAB’s place function to construct the gain matrix L_{22} so that the eigenvalues of the matrix (A_{22} − L_{22}C_{22}) are located at −3, −4, −5, where

L_{22} = [ \begin{matrix}−0.1441 & −4.0022 \\ 7.5018 & 0.0744 \\ −14.9883 & −0.4586 \end{matrix} ] .

We then solve the Lyapunov matrix equation,

(A_{22} − L_{22}C_{22})^{⊤}P_{22} + P_{22}(A_{22} − L_{22}C_{22}) = −I_{3},

to obtain

P_{22} = [ \begin{matrix}0.1273 & −0.0445 & −0.0236 \\ −0.0445 & 0.9995 & 0.4735 \\ −0.0236 & 0.4735 & 0.3059 \end{matrix} ] .

After that we compute

\frac{1}{2}λ_{max} (A_{11}^{⊤} + A_{11} + (A_{21}^{⊤}P_{22} + A_{12}) Q_{22}^{−1} (A_{12}^{⊤} + P_{22}A_{21})) = 12.6790

and select

κ = 13.6790.

Finally we construct

L = T^{−1}L̂S = [ \begin{matrix}0.0003 & −0.0000 & 0.0100 & 4.0002 \\ 13.2773 & 0.0115 & −15.1363 & −0.4544 \\ 0.0001 & 13.6790 & 0.0044 & −0.0001 \\ −0.1532 & 0.0044 & 7.9048 & 0.1580 \\ −0.3976 & 0.0114 & −14.9831 & −0.4586 \end{matrix} ]

and

F = F̂ S = [ \begin{matrix}−0.7440 & 0.3370 & 0.0200 & −0.0000 \\ −0.0320 & −1.1200 & −0.0000 & 0.0000 \end{matrix} ] .

Let η = 7. We then construct the UIO,

˙x̂ = Ax̂ + L(y − ŷ) + B_{2}Ẽ(y, ŷ, η),

where

Ẽ(y, ŷ, η) = \{ \begin{matrix}η \frac{F (y−y^{}ˆ )}{∥F (y−y^{}ˆ )∥+μ} & for & F (y − ŷ) ≠ 0 \\ 0 & for & F (y − ŷ) = 0, \end{matrix}

644 CHAPTER 12. UNKNOWN INPUT OBSERVER (UIO) DESIGN Figure 12.5: Plots of x_{i}s and their estimates versus time for Example 12.3.

Figure 12.6: Sliding mode observer (12.28) as a part of the combined controller-observer compensator and an unknown input estimator.

where μ = 0.0005. The parameter μ was introduced to smooth out the discontinuity to facilitate simulations. In Figure 12.5, we show plots of the system state variables and their estimates versus time. The initial conditions of the plant were selected randomly to be equal to

x(0) = [ 0.0342 0.3200 0.0178 −0.0287 −0.2497 ]^{⊤} .

The initial conditions of the observer were set to zero.

In the next example, we illustrate the use of the sliding mode UIO as a part of the combined controller-observer compensator and an unknown input estimator as shown in Figure 12.6.

Example 12.4 We consider a plant with unknown input modeled by (12.3), where

A = [ \begin{matrix}0.6733 & −0.9068 & −0.3089 & 1.6703 \\ 1.2736 & −0.4747 & −0.8369 & −1.5417 \\ −1.0398 & −0.6661 & −0.6283 & −0.2720 \\ 0.4880 & −0.5020 & 0.5009 & 0.3416 \end{matrix} ] , B_{1} = [ \begin{matrix}0.4844 & 0.6678 \\ 0.9527 & 0.8982 \\ 0.1314 & −0.5493 \\ −1.7419 & −0.3175 \end{matrix} ] ,

B_{2} = [ \begin{matrix}−0.5827 \\ −0.1642 \\ −0.9351 \\ 1.1881 \end{matrix} ] , C = [ \begin{matrix}−1.2128 & 1.3227 & −0.3618 & 0.3871 \\ −0.3812 & 0.3853 & −0.4264 & 0.9028 \end{matrix} ] .

Let the unknown input be

u_{2} = 3 cos(5t).

We use 12.2 to design the a sliding-mode UIO to reconstruct the unknown input u_{2}. Let the eigenvalues of (A_{22} − L_{22}C_{22}) be placed at {−3, −4, −5}. Using MATLAB’s function place we obtain

L_{22} = [ \begin{matrix}−37.5312 \\ 31.9351 \\ −13.5910 \end{matrix} ] .

We select κ = 3.7417.

The desired closed-loop poles of the system are to be located at {−4, −5, −6, −7}. Using MATLAB’s function place we obtain

K = [ \begin{matrix}−3.6021 & 8.0249 & 8.1459 & −2.3902 \\ −9.3810 & −9.7238 & −38.9690 & −13.7649 \end{matrix} ]

such that eig(A − KB_{1}) = {−4, −5, −6, −7}.

In our simulations, the vector of the initial conditions of the plant is

x(0) = [ 0.6470 −3.4975 −3.4439 0.3146 ]^{⊤}

and we set zero initial conditions for the observer. In Figure 12.7, we show plots of the plant’s states and their estimates. In Figure 12.8, we show an estimate of the unknown input u_{2}, which is the output of the nonlinear element E.

12.4. SLIDING MODE UIO 647

Figure 12.7: Plots of the states and their estimates. Figure 12.8: Plots of the unknown input and its estimate.

12.4.4 Sliding Mode UIO Design Using LMIs

We can use LMIs to design a sliding mode UIO. Xiang, Su, and Chu [356] represent the problem of finding a triple (L, F , P) subject to (12.26) and (12.27) in the form of an LMI feasibility problem.

To proceed, let G be the orthogonal complement of B_{2}, which means that B_{2}G = O, and we write

G = B_{2}^{⊥},

where we assume that the matrix rank B_{2} = m_{2}, that is, the matrix B_{2} has full column rank. Then the problem of finding a triple (L, F , P), P = P^{⊤} > 0 such that

(A − LC)^{⊤}P + P (A − LC) < 0 and B_{2}^{⊤}P = F C  (12.47)

is equivalent to finding matrices

W_{1} = W _{1}^{⊤} ∈ ℝ^{(n−m_{2})×(n−m_{2})},
W_{2} = W _{2}^{⊤} ∈ ℝ^{p×p},
Y ∈ ℝ^{n×p},

such that

GW_{1}G^{⊤} + C^{⊤}W_{2}C > 0  (12.48)

and

GW_{1}G^{⊤}A + C^{⊤}W_{2}CA − Y C + (GW_{1}G^{⊤}A + C^{⊤}W_{2}CA − Y C)^{⊤} < 0. (12.49)

If a feasible solution to (12.48) and (12.49) exists, then

P = GW_{1}G^{⊤} + C^{⊤}W_{2}C,  (12.50)
L = P^{−1}Y ,  (12.51)
F = G^{⊤}C^{⊤}W_{2}.  (12.52)

Example 12.5 We use LMIs to find a triple (L, F , P) satisfying (12.47) for the same system as in Example 12.4. Recall that

A = [ \begin{matrix}0.6733 & −0.9068 & −0.3089 & 1.6703 \\ 1.2736 & −0.4747 & −0.8369 & −1.5417 \\ −1.0398 & −0.6661 & −0.6283 & −0.2720 \\ 0.4880 & −0.5020 & 0.5009 & 0.3416 \end{matrix} ] , B_{2} = [ \begin{matrix}−0.5827 \\ −0.1642 \\ −0.9351 \\ 1.1881 \end{matrix} ] ,

C = [ \begin{matrix}−1.2128 & 1.3227 & −0.3618 & 0.3871 \\ −0.3812 & 0.3853 & −0.4264 & 0.9028 \end{matrix} ] .

We mention that the above matrices were generated using MATLAB’s function randn.

We use MATLAB’s LMI toolbox to solve the LMIs given by (12.49). Then, using (12.50), (12.51), and (12.52), we compute the matrices

P = [ \begin{matrix}178.2434 & −54.7774 & 11.9292 & 66.0388 \\ −54.7774 & 67.3161 & 16.2845 & 16.3716 \\ 11.9292 & 16.2845 & 61.8596 & 6.8944 \\ 66.0388 & 16.3716 & 6.8944 & 154.8361 \end{matrix} ] , L = [ \begin{matrix}−80.7624 & −30.7668 \\ 79.1421 & −578.7866 \\ −194.5307 & 302.2489 \\ 174.7920 & 2.1652 \end{matrix} ] ,

F = [ −28.5969 163.2860 ] .

12.5 The Edwards and Spurgeon Sliding Mode UIO

In this Section, we present an alternative approach to the sliding mode UIO design proposed by Edwards and Spurgeon [92]. The construction of this observer involves a number of state variable transformations that bring the plant model into a special canonical form that is then used to construct the observer. The inverse transformations are performed to obtain the final form of the observer. The necessary and sufficient conditions for the existence of the Edwards and Spurgeon observer are the same as for the previously discussed UIO architectures.

12.5.1 The Canonical Form—The Design Platform

We begin by assuming that

rank B_{2} = m_{2} and rank C = p.

We then construct

T_{c} = [ \begin{matrix}N & {⊤ \atop c} \\ C \end{matrix} ] ,

where N_{c} ∈ ℝ^{n×(n−p)} is the matrix whose columns span the null space of C. The matrix T_{c} is invertible.

We use T_{c} to transform the triple (A, B_{2}, C) into the form,

A_{c} = T_{c}AT _{c}^{−1}, B_{2_{c}} = T_{c}B_{2} = [ \begin{matrix}B & 2_{c_{1}} \\ B & 2_{c_{2}} \end{matrix} ] ,
C_{c} = CT _{c}^{−1} = [ O_{p×(n−p)} I_{p} ] ,

where B_{2_{c_{1}}} ∈ ℝ^{(n−p)×m_{2}} and B_{2_{c_{2}}} ∈ ℝ^{p×m_{2}}.

The next state coordinate transformation, T_{b}, contains two submatrices that require their careful description. First note that by assumption

rank B_{2} = rank(CB_{2}) = m_{2}

and

CB_{2} = C_{c}B_{2_{c}} = B_{2_{c_{2}}}.

Hence

rank B_{2_{c_{2}}} = m_{2},

where recall B_{2_{c_{2}}} ∈ ℝ^{p×m_{2}} and p ≥ m_{2}. Therefore

B_{2_{c_{2}}}^{†} = (B_{2_{c_{2}}}^{⊤}B_{2_{c_{2}}})^{−1} B_{2_{c_{2}}}^{⊤}

is well defined. Next, there exists an orthogonal matrix T_{b_{2}} ∈ ℝ^{p×p} such that

T_{b_{2}}B_{2_{c_{2}}} = [\begin{matrix}O \\ B & 2_{b_{2}} \end{matrix}] ,

where B_{2b} ∈ ℝ^{m_{2}×m_{2}} and rankB_{2b} = m_{2}. We use B_{2_{c_{2}}}^{†} and T_{b_{2}} to construct the coordinate transformation

T_{b} = [\begin{matrix}I_{n−p} & −B_{2_{c_{1}}}B_{2_{c_{2}}}^{†} \\ O_{p×(n−p)} & T_{b_{2}} \end{matrix}] .

Applying the transformation T_{b} to the triple (A_{c}, B_{2c}, C_{c}) gives

B_{2_{b}} = T_{b}B_{2c}, C_{b} = C_{c}T _{b}^{−1}

and

A_{b} = T_{b}A_{c}T _{b}^{−1} = [ \begin{matrix}A_{b11} & A_{b12} \\ A_{b21} & A_{b22} \end{matrix} ] ,

where A_{b11} ∈ ℝ^{(n−m_{2})×(n−m_{2})} and it has the form

A_{b11} = [ \begin{matrix}A_{1111} & A_{1112} \\ A_{1121} & A_{1122} \end{matrix} ] ,

where A_{1111} ∈ ℝ^{(n−p)×(n−p)} and A_{1121} ∈ ℝ^{(p−m_{2})×(n−p)}. If the pair (A_{1111}, A_{1121}) is not observable, we construct a transformation T_{obs} ∈ ℝ^{(n−p)×(n−p)} such that

T_{obs}A_{1111}T _{obs}^{−1} = [ \begin{matrix}A_{11}^{o} & A_{12}^{o} \\ O & A_{22}^{o} \end{matrix} ] and A_{1121}T _{obs}^{−1} = [ O A_{21}^{o} ] ,

where A_{11}^{o} ∈ ℝ^{r×r}, A_{21}^{o} ∈ ℝ^{(p−m_{2})×(n−p−r)}, A_{22}^{o} ∈ ℝ^{(n−p−r)×(n−p−r)}, the pair (A_{22}^{o}, A_{21}^{o}) is observable, and r is number of unobservable states of the pair (A_{1111}, A_{1121}). We embed the transformation T_{obs} into

T_{a} = [ \begin{matrix}T_{obs} & O \\ O & I_{p} \end{matrix} ] .

Applying T_{a} to the triple (A_{b}, B_{2_{b}}, C_{b}) yields

A_{a} = T_{a}A_{b}T _{a}^{−1} = [ \begin{matrix}A_{a11} & A_{a12} \\ A_{a21} & A_{a22} \end{matrix} ] ,

where

A_{a11} = [ \begin{matrix}A_{11}^{o} & A_{12}^{o} \\ A & {m \atop 12} \\ O & A_{22}^{o} \\ O & A_{21}^{o} & A_{22}^{m} \end{matrix} ] ∈ ℝ^{(n−m_{2})×(n−m_{2})},

B_{2_{a}} = T_{a}B_{2_{b}} = [ \begin{matrix}O \\ B & 2_{b_{2}} \end{matrix} ] , B_{2_{b_{2}}} ∈ ℝ^{m_{2}×m_{2}},

and

C_{a} = C_{b}T _{a}^{−1} = [ O T _{b_{2}}^{⊤} ] , T_{b_{2}} ∈ ℝ^{p×p}.

Remark 12.1 The eigenvalues of A_{11}^{o} are the system zeros of the system represented by the triple (A_{a}, B_{2_{a}}, C_{a}). Because system zeros are invariant under similarity transformations, the eigenvalues of A_{11}^{o} are the system zeros of the system represented by the triple (A, B_{2}, C).

◇

By assumption, the system zeros are in the open left-half complex plane. The pair (A_{22}^{o}, A_{21}^{o}) is observable; therefore we can construct a gain matrix L_{a} ∈ ℝ^{(n−p−r)×(p−m_{2})} such that the matrix

A_{22}^{o} − L_{a}A_{21}^{o}

has its eigenvalues in pre-specified locations in the left-half plane. Let

L = [ \begin{matrix}O & r×(p−m_{2}) \\ L & a \end{matrix} ]

Then because the eigenvalues of the matrix A_{11}^{o} are asymptotically stable, the matrix

A_{11} = [ \begin{matrix}A_{11}^{o} & A_{12} \\ O & A_{22}^{o} \end{matrix} ] − [ \begin{matrix}O & r×(p−m_{2}) \\ L & a \end{matrix} ] [ O_{(p−m_{2})×r} A_{21}^{o} ]

has its all eigenvalues in the left-half plane, where A_{11} ∈ ℝ^{(n−p)×(n−p)}.

Let

L̄ = [ L O_{(n−p)×m_{2}} ] .

We construct the transformation

T_{L} = [ \begin{matrix}I_{n−p} & −L̄ \\ O_{p×(n−p)} & T_{b2} \end{matrix} ]

and apply it to the triple (A_{a}, B_{2_{a}}, C_{a}) to obtain

B_{2} = T_{L}B_{2_{a}} = [ \begin{matrix}O \\ B_{2_{2}} \end{matrix} ] , C = C_{a}T _{L}^{−1} = [ O I_{p} ] ,

and

A = T_{L}A_{a}T _{L}^{−1} = [ \begin{matrix}A_{11} & A_{12} \\ A_{21} & A_{22} \end{matrix} ] ,

where A_{11} ∈ ℝ^{(n−p)×(n−p)}, A_{12} ∈ ℝ^{(n−p)×p}, A_{21} ∈ ℝ^{p×(n−p)} and A_{22} ∈ ℝ^{p×p}. The matrix A_{11} has asymptotically stable eigenvalues.

The plant model represented by the quadruple (A, B_{1}, B_{2}, C) takes the following form in the new coordinates

\begin{matrix}ẋ_{1} & = & A_{11}x_{1} + A_{12}y + B_{1_{1}}u_{1} \\ ẏ & = & A_{21}x_{1} + A_{22}y + B_{1_{2}}u_{1} + B_{2_{2}}u_{2} \end{matrix} \}  (12.53)

We refer to the above representation as the Edwards-Spurgeon canonical form.

12.5.2 Observer Architecture

We are now ready to present the Edwards-Spurgeon UIO. Let x̂_{1} denote an estimate of x_{1}, and let ŷ denote an estimate of y. We also let

e_{y} = y − ŷ

to denote the estimation error of y. Edwards and Spurgeon [92, p. 134] propose their UIO for the plant model in the canonical form (12.53),

\begin{matrix}˙x̂_{1} & = & A_{11}x̂_{1} + A_{12}ŷ + B_{1_{1}}u_{1} + A_{12}e_{y} \\ ˙ ; ŷ & = & A_{21}x̂_{1} + A_{22}ŷ + B_{1_{2}}u_{1} + (A_{22} − A_{22}^{s}) e_{y} + ν, \end{matrix} \}  (12.54)

where A_{22}^{s} is a stable design matrix. The discontinuous injection term ν is

ν = \{ \begin{matrix}ρ∥B_{2_{2}}∥\frac{P_{2}e_{y}}{∥P_{2}e_{y}∥} & if e_{y} ≠ 0 \\ 0 & otherwise \end{matrix}

where P_{2} = P _{2}^{⊤} > 0 is the solution to the matrix Lyapunov equation

A_{22}^{s⊤}P_{2} + P_{2}A_{22}^{s} = −Q_{2}  (12.55)

for some symmetric positive definite Q_{2}, and the gain ρ ≥ ∥u_{2}∥.

Let e_{1} = x_{1} − x̂_{1}. The dynamics of the observation error are

ė_{1} = A_{11}ê_{1}
ė_{y} = A_{21}ê_{1} + A_{22}^{s}ê_{y} + B_{2_{2}}u_{2} − ν.

One can show using the Lyapunov second method that the above error system is asymptotically stable.

Let the linear term coefficient G_{l} be defined as

G_{l} = T _{c}^{−1}T _{b}^{−1}T _{a}^{−1}T _{L}^{−1} [ \begin{matrix}A_{12} \\ A_{22} − A_{22}^{s} \end{matrix} ] ,

and the nonlinear term coefficient G_{n} be defined as

G_{n} = T _{c}^{−1}T _{b}^{−1}T _{a}^{−1}T _{L}^{−1} [ \begin{matrix}O & (n−p)×p \\ I & p \end{matrix} ] .

Recall that

e_{y} = y − ŷ = Cx − Cx̂ = Ce.

Then the Edwards-Spurgeon UIO in the original coordinates takes the form

˙x̂ = Ax̂ + B_{1}u_{1} + G_{l}e_{y} + G_{n}ν  (12.56)

12.5.3 Reconstructing the Unknown Input

We show here how the Edwards-Spurgeon observer (12.56) can be used to reconstruct the unknown input u_{2}. Consider the manifold,

{e : σ(e) = Ce = 0} .  (12.57)

Figure 12.9: The Edwards-Spurgeon sliding mode UIO given by (12.56) as a part of the combined observer-controller compensator and an unknown input estimator.

We can always choose ρ such that the above manifold is attractive and it will be reached in a finite time. Thus, after the estimation error trajectory, e, enters a sliding mode along the sliding surface, we have σ(e) = 0, and

σ̇ = C (A − G_{l}C) e + CB_{2}u_{2} − CG_{n}ν(Ce) = 0.  (12.58)

Because e → 0 as t → ∞, we obtain

CB_{2}u_{2} = CG_{n}ν.

From the above, we obtain a least-squares estimate of the unknown input of the form

û_{2} = (CB_{2})^{†}CG_{n}ν  (12.59)

As in the previous case, the Edwards-Spurgeon sliding mode UIO can also be used in the combined controller-observer compensator architecture and as unknown input estimator as depicted in Figure 12.9.

Example 12.6 We construct the Edwards-Spurgeon sliding UIO given by (12.56) for the plant in Example 12.4 and use it in the architecture depicted in Figure 12.9.

We have m_{2} = 1. We start with computing T_{c}, where we employ the command, Nc=null(C), to find N_{c}. We use

ρ∥B_{2_{2}}∥ = 10∥ [ 1.2877 1.6302 ]^{⊤} ∥.

The gain matrix L was selected using the place command so that the eigenvalues of A_{11} are placed at {−5, −6}. The design parameter matrix, A_{22}^{s}, was selected as

A_{22}^{s} = [ \begin{matrix}−6 & 0 \\ 0 & −7 \end{matrix} ] .

For Q_{2} = 2I_{2}, the solution to (12.55) is

P_{2} = [ \begin{matrix}0.1667 & 0 \\ 0 & 0.1429 \end{matrix} ] .

The nonlinear discontinuous term ν was approximated with

ν̃ = ρ∥B_{2_{2}}∥\frac{P_{2}e_{y}}{∥P_{2}e_{y}∥ + μ},

where μ = 0.01.

The feedback matrix K was computed using the place function so that the eigenvalues of (A − B_{1}K) are located at {−4, −5, −6, −7}.

In Figure 12.10, we show plots of the plant’s states and their estimates. In Figure 12.11, we show plots of the unknown input, u_{2} = 3 cos(5t), and its estimate.

12.6 Notes

In his practical guide for the selection and installation of observers in control systems, Ellis [94, page 3] writes: “Observers add complexity to the system and require computational resources. They may be less robust than physical sensors, especially when plant parameters change substantially during operation. Still, an observer applied with skill can bring substantial performance benefits and do so, in many cases, while reducing cost or increasing reliability.” Examples of impressive applications of nonlinear observers to the control of electric machinery can be found in [72, 330, 308]. Röbenack and Lynch [282] presented a method to the observer design for a class of nonlinear plants that yields almost linear observation error dynamics. An application of the sliding mode observer presented in Subsection 12.4 to decentralized dynamic output feedback control of nonlinear interconnected systems is reported in [172].

656 CHAPTER 12. UNKNOWN INPUT OBSERVER (UIO) DESIGN

Figure 12.10: Plots of states and their estimates using the Edwards-Spurgeon UIO. Figure 12.11: Plots of the unknown input and its estimate using the Edwards-Spurgeon UIO.

Sliding-mode observers for dynamical systems with unknown inputs have found applications in robust detection and reconstruction of actuator and sensor faults [319, 357, 318, 359, 358, 8, 9, 320] and fault detection and isolation (FDI) [291]. In particular, the purpose of the FDI scheme is to generate an alarm when a fault, such as a component malfunction, develops in the process being monitored and to identify the location of the fault. The fault signal can often be modeled as an unknown input to the plant. Thus, unknown input observers can be used for FDI purposes. For example, Edwards and Spurgeon (see [92, Section 6.5], Edwards, Spurgeon, and Patton [93]), and Saif and Xiong [291] use sliding mode observers to develop FDI schemes for reconstructing the actuator and sensor fault signals.

12.7 Exercises

Exercise 12.1 Design and test the full-order UIO described in Theorem 12.1 for the system model of Example 12.2.

Exercise 12.2 Design and test the UIO of Hou and Müller, described in Section 12.3.4, for the system model of Example 12.2.

Exercise 12.3 Consider two system models that appear in [360] and [71]. The first system, using our notation, is modeled as

A = [ \begin{matrix}−2 & −2 & 0 \\ 0 & 0 & 1 \\ 0 & −3 & −4 \end{matrix} ] , B_{1} = O, B_{2} = [ \begin{matrix}1 & 0 \\ 0 & 1 \\ 0 & 0 \end{matrix} ]

C = [ \begin{matrix}1 & 0 & 1 \\ 0 & 1 & 0 \end{matrix} ] .

The second system is modeled by

A = [ \begin{matrix}−1 & −1 & 0 \\ −1 & 0 & 0 \\ 0 & −1 & −1 \end{matrix} ] , B_{1} = O, B_{2} = [ \begin{matrix}−1 \\ 0 \\ 0 \end{matrix} ]

C = [ \begin{matrix}1 & 0 & 0 \\ 0 & 0 & 1 \end{matrix} ] .

Construct UIOs developed in Section 12.3.2, 12.3.3, and 12.4.3 for the above systems.

Exercise 12.4 Construct the Hou and Müller UIO, described in Section 12.3.4, for each system of Exercise 12.3.

Exercise 12.5 Prove Lemma 12.4.

Exercise 12.6 Show that the conditions in (12.47) are equivalent to the LMIs given by (12.48) and (12.49).

Exercise 12.7 Use an LMI toolbox to find matrices L, F, and P of Theorem 12.3 for the system of Example 12.3.

Model-based Predictive Control (MPC)

A system—organism, corporate firm, computer program—spends energy feeding the past back into the present because this is an economical way for the system to deal with the future. To see into the future one must see into the past. A constant pulse of the past along feedback loops informs and controls the future. [175, p. 439]

The model-based predictive control (MPC) methodology is also referred to as the moving horizon control or the receding horizon control. The idea behind this approach can be explained using an example of driving a car. The driver looks at the road ahead of him and taking into account the present state and the previous action predicts his action up to some distance ahead, which we refer to as the prediction horizon. Based on the prediction, the driver adjusts the driving direction. The MPC main idea is illustrated in Figure 13.1.

The MPC is constructed using control and optimization tools. The objective of this chapter is to introduce the reader to the linear MPC which refers to the family of MPC schemes in which linear models of the controlled objects are used in the control law synthesis.

In the MPC approach, the current control action is computed on-line rather than using a pre-computed, off-line, control law.

A model predictive controller uses, at each sampling instant, the plant’s current input and output measurements, the plant’s current state, and the plant’s model to

• calculate, over a finite horizon, a future control sequence that optimizes a given performance index and satisfies constraints on the control action; Figure 13.1: The driver predicts future travel direction based on the current state of the car and the current position of the steering wheel.

• use the first control in the sequence as the plant’s input.

The MPC strategy is illustrated in Figure 13.2, where N_{p} is the prediction horizon, u(t+k|t) is the predicted control action at t+k given u(t). Similarly, y(t+k|t) is the predicted output at t + k given y(t).

In Figure 13.3, we show basic structure of an MPC-controlled plant, where we assume that the plant’s state is available to us.

13.1 Simple Discrete-Time MPC

The development a discrete MPC in this section follows that of [342, Chapter 1]. We consider a discretized model of a dynamic system of the form,

x[k + 1] = Φx[k] + Γu[k]  (13.1)
y[k] = Cx[k],  (13.2)

where Φ ∈ ℝ^{n×n}, Γ ∈ ℝ^{n×m}, and C ∈ ℝ^{p×n}.

Applying the backward difference operator, Δx[k + 1] = x[k + 1] − x[k], to (13.1) gives

Δx[k + 1] = ΦΔx[k] + ΓΔu[k],  (13.3)

where Δu[k + 1] = u[k + 1] − u[k].

13.1. SIMPLE DISCRETE-TIME MPC 661 Figure 13.2: Controller action construction using model-based predictive control (MPC) approach.

We now apply the backward difference operator to (13.2) to obtain

Δy[k + 1] = y[k + 1] − y[k]
= Cx[k + 1] − Cx[k]
= CΔx[k + 1].

Substituting into the above (13.3) yields

Δy[k + 1] = CΦΔx[k] + CΓΔu[k].

Hence,

y[k + 1] = y[k] + CΦΔx[k] + CΓΔu[k].  (13.4)

We combine (13.3) and (13.4) into one equation to obtain

[ \begin{matrix}Δx[k + 1] \\ y[k + 1] \end{matrix} ] = [ \begin{matrix}Φ & O \\ CΦ & I_{p} \end{matrix} ] [ \begin{matrix}Δx[k] \\ y[k] \end{matrix} ] + [ \begin{matrix}Γ \\ C & Γ \end{matrix} ] Δu[k].  (13.5)

We represent (13.2) as

y[k] = [ O I_{p} ] [ \begin{matrix}Δx[k] \\ y[k] \end{matrix} ] .  (13.6)

Figure 13.3: State feedback model predictive controller.

We define the augmented state vector,

x_{a}[k] = [ \begin{matrix}Δx[k] \\ y[k] \end{matrix} ] .  (13.7)

Let

Φ_{a} = [ \begin{matrix}Φ & O \\ CΦ & I_{p} \end{matrix} ] , Γ_{a} = [ \begin{matrix}Γ \\ C & Γ \end{matrix} ] , and C_{a} = [ O I_{p} ] .  (13.8)

Using the above notation, we represent (13.5) and (13.6) in a compact format as

x_{a}[k + 1] = Φ_{a}x_{a}[k] + Γ_{a}Δu[k]  (13.9)
y[k] = C_{a}x_{a}[k],  (13.10)

where Φ_{a} ∈ ℝ^{(n+p)×(n+p)}, Γ_{a} ∈ ℝ^{(n+p)×m}, and C_{a} ∈ ℝ^{p×(n+p)}.

Suppose now that the state vector x_{a} at each sampling time, k, is available to us. Our control objective is to construct a control sequence,

Δu[k], Δu[k + 1], …, Δu[k + N_{p} − 1],  (13.11)

where N_{p} is the prediction horizon, such that a given cost function and constraints are satisfied. The above control sequence will result in a predicted sequence of the state vectors,

x_{a}(k + 1|k), x_{a}(k + 2|k), …, x_{a}(k + N_{p}|k),

which can then be used to compute predicted sequence of the plant’s outputs,

y(k + 1|k), y(k + 2|k), …, y(k + N_{p}|k).  (13.12)

Using the above information, we can compute the control sequence (13.11) and then apply u[k] to the plant to generate x[k + 1]. We repeat the process again, using x[k + 1] as an initial condition to compute u[k + 1], until we reach the boundary of the control horizon, N_{c}.

We now present an approach to construct u[k] given x[k]. Using the plant model parameters and the measurement of x_{a}[k] we evaluate the augmented states over the prediction horizon successively applying the recursion formula (13.9) to obtain,

x_{a}(k + 1|k) = Φ_{a}x_{a}[k] + Γ_{a}Δu[k]
x_{a}(k + 2|k) = Φ_{a}x_{a}(k + 1|k) + Γ_{a}Δu[k + 1]
= Φ_{a}^{2}x_{a}[k] + Φ_{a}Γ_{a}Δu[k] + Γ_{a}Δu[k + 1]
⋮
x_{a}(k + N_{p}|k) = Φ_{a}^{N_{p}}x_{a}[k] + Φ_{a}^{N_{p}−1}Γ_{a}Δu[k] + ⋯ + Γ_{a}Δu[k + N_{p} − 1]

We represent the above set of equations in the form,

[ x_{a}(k + 1|k) ; x_{a}(k + 2|k) ; ⋮ ; x_{a}(k + N_{p}|k) ] = [ Φ_{a} ; Φ_{a}^{2} ; ⋮ ; Φ_{a}^{N_{p}} ] x_{a}[k] + [ \begin{matrix}Γ & a \\ Φ_{a}Γ_{a} & Γ_{a} \\ ⋮ & & ... \\ Φ_{a}^{N_{p}−1}Γ_{a} & & ⋯ & Γ_{a} \end{matrix} ] [ Δu[k] ; Δu[k + 1] ; ⋮ ; Δu[k + N_{p} − 1] ] .

(13.13)

We wish to design a controller that would force the plant output, y, to track a given reference signal, r. Using (13.10) and (13.13), we compute the sequence of predicted outputs (13.12),

[ y(k + 1|k) ; y(k + 2|k) ; ⋮ ; y(k + N_{p}|k) ] = [ C_{a}x_{a}(k + 1|k) ; C_{a}x_{a}(k + 2|k) ; ⋮ ; C_{a}x_{a}(k + N_{p}|k) ]

= [ C_{a}Φ_{a} ; C_{a}Φ_{a}^{2} ; ⋮ ; C_{a}Φ_{a}^{N_{p}} ] x_{a}[k]  (13.14)

+ [ \begin{matrix}C & a & Γ & a \\ C_{a}Φ_{a}Γ_{a} & C_{a}Γ_{a} \\ ⋮ & & ... \\ C_{a}Φ_{a}^{N_{p}−1}Γ_{a} & & ⋯ & C_{a}Γ_{a} \end{matrix} ] [ Δu[k] ; Δu[k + 1] ; ⋮ ; Δu[k + N_{p} − 1] ] . (13.15)

We write the above compactly as

Y = W x_{a}[k] + ZΔU,  (13.16)

where

Y = [ y(k + 1|k) ; y(k + 2|k) ; ⋮ ; y(k + N_{p}|k) ] , ΔU = [ Δu[k] ; Δu[k + 1] ; ⋮ ; Δu[k + N_{p} − 1] ] ,

and

W = [ C_{a}Φ_{a} ; C_{a}Φ_{a}^{2} ; ⋮ ; C_{a}Φ_{a}^{N_{p}} ] , and Z = [ \begin{matrix}C & a & Γ & a \\ C_{a}Φ_{a}Γ_{a} & C_{a}Γ_{a} \\ ⋮ & & ... \\ C_{a}Φ_{a}^{N_{p}−1}Γ_{a} & & ⋯ & C_{a}Γ_{a} \end{matrix} ] .

Suppose now that we wish to construct a control sequence, Δu[k], …, Δu[k + N_{p} − 1], that would minimize the cost function

J(ΔU) = \frac{1}{2} (r_{p} − Y )^{⊤} Q (r_{p} − Y ) + \frac{1}{2}ΔU^{⊤}RΔU,  (13.17)

where Q = Q^{⊤} ≥ 0 and R = R^{⊤} > 0 are real symmetric positive semi-definite and positivedefinite weight matrices, respectively. The multiplying scalar, 1/2, is just to make subsequent manipulations cleaner. Finally, the vector r_{p} consists of the values of the command signal at sampling times, k, k + 1, …, k + N_{p} − 1. The selection of the weight matrices, Q and R reflects our control objective to keep the tracking error ∥r_{p} − Y ∥ “small” using the control actions that are “not too large.”

We first apply the first-order necessary condition (FONC) test to J(ΔU),

\frac{∂J}{∂ΔU} = 0^{⊤}.

Then, we solve the above equation for ΔU = ΔU^{∗}, where

\frac{∂J}{∂ΔU} = − (r_{p} − W x_{a} − ZΔU)^{⊤} QZ + ΔU^{⊤}R
= 0^{⊤}.

Performing simple manipulations yields

−r_{p}^{⊤}QZ + x_{a}^{⊤}W^{⊤}QZ + ΔU^{⊤}Z^{⊤}QZ + ΔU^{⊤}R = 0^{⊤}.

Applying the transposition operation to both sides of the above equation and rearranging terms, we obtain

(R + Z^{⊤}QZ) ΔU = Z^{⊤}Q (r_{p} − W x_{a}) .

Note that the matrix (R + Z^{⊤}QZ) is invertible, and in fact, positive definite because R = R^{⊤} > 0 and Z^{⊤}QZ is also symmetric and at least positive semi-definite. Hence, ΔU that satisfies the FONC is

ΔU^{∗} = (R + Z^{⊤}QZ)^{−1} Z^{⊤}Q (r_{p} − W x_{a}) .  (13.18)

Now, applying the second derivative test to J(ΔU), which we refer to as the second-order sufficiency condition (SONC), we obtain

\frac{∂^{2}J}{∂ΔU^{2}} = R + Z^{⊤}QZ
> 0,

which implies that ΔU^{∗} is a strict minimizer of J.

Using (13.18), we compute Δu[k],

Δu[k] = \overbrace{[ I_{m} O ⋯ O ]}^{N_{p} block matrices} (R + Z^{⊤}QZ)^{−1} Z^{⊤}Q (r_{p} − W x_{a})
= Krr_{p} − KxΔx[k] − Kyy[k],  (13.19)

where

Kr = [ I_{m} O ⋯ O ] (R + Z^{⊤}QZ)^{−1} Z^{⊤}Q, Kx = KrW [ \begin{matrix}I & n \\ O \end{matrix} ] ,

and

Ky = KrW [ \begin{matrix}O \\ I & p \end{matrix} ] .

An implementation of the above controller, using a discrete-time integrator, is shown in Figure 13.4.

13.2 MPC With Constraints

An attractive feature of the model-based predictive control approach is that a control engineer can incorporate different types of constraints on the control action. We consider three types of such constraints.

Figure 13.4: Discrete-time MPC for linear time-invariant systems.

13.2.1 Constraints on the Rate of Change of the Control Action

Hard constraints on the rate of change of the control signal can be expressed as

Δu_{i}^{min} ≤ Δu_{i}[k] ≤ Δu_{i}^{max}, i = 1, 2, …, m  (13.20)

Let

Δu^{min} = [ Δu_{1}^{min} ⋯ Δu_{m}^{min} ]^{⊤} and Δu^{max} = [ Δu_{1}^{max} ⋯ Δu_{m}^{max} ]^{⊤} .

Then, we can express (13.20) as

Δu^{min} ≤ Δu[k] ≤ Δu^{max}.  (13.21)

The above can be equivalently represented as

[ \begin{matrix}−I_{m} \\ I & m \end{matrix} ] Δu[k] ≤ [ \begin{matrix}−Δu^{min} \\ Δu^{max} \end{matrix} ] .  (13.22)

Using the above approach we can represent constraints on the rate of change of the control action over the whole prediction horizon, N_{p}, in terms of ΔU[k], by augmenting the above inequality to incorporate constraints for the remaining sampling times. That is, if the above constraints are imposed on the rate of change of the control action for all sampling times within the prediction horizon, then this can be expressed in terms of ΔU[k] as

[ \begin{matrix}−I_{m} & O & ⋯ & O & O \\ I_{m} & O & ⋯ & O & O \\ O & −I_{m} & ⋯ & O & O \\ O & I_{m} & ⋯ & O & O \\ ⋮ & & & & ⋮ \\ O & O & ⋯ & −I_{m} & O \\ O & O & ⋯ & I_{m} & O \\ O & O & ⋯ & O & −I_{m} \\ O & O & ⋯ & O & I_{m} \end{matrix} ] [ Δu[k] ; Δu[k + 1] ; ⋮ ; Δu[k + N_{p} − 2] ; Δu[k + N_{p} − 1] ] ≤ [ −Δu^{min} Δu^{max} −Δu^{min} Δu^{max} ⋮ −Δu^{min} Δu^{max} −Δu^{min} Δu^{max} ] .  (13.23)

On the other hand, if the constraints on the rate of change of the control are imposed only on the first component of ΔU[k], then we express this in terms of ΔU[k] as

[ \begin{matrix}−I_{m} & O & ⋯ & O & O \\ I_{m} & O & ⋯ & O & O \end{matrix} ] ΔU[k] ≤ [ \begin{matrix}−Δu^{min} \\ Δu^{max} \end{matrix} ] .

13.2.2 Constraints on the Control Action Magnitude

Hard constraints on the control action magnitude at the time sampling k have the form

u_{i}^{min} ≤ u_{i}[k] ≤ u_{i}^{max}, i = 1, 2 …, m  (13.24)

We now express constraints on the control action magnitude over the whole prediction horizon in term of ΔU[k]. To accomplish our goal, we first note that

u[k] = u[k − 1] + Δu[k]
= u[k − 1] + [ I_{m} O ⋯ O ] ΔU[k],

where, recall that ΔU[k] = [ Δu[k] Δu[k + 1] ⋯ Δu[k + N_{p} − 1] ]^{⊤}.

Similarly,

u[k + 1] = u[k] + Δu[k + 1]
= u[k − 1] + [ I_{m} O ⋯ O ] ΔU[k] + Δu[k + 1]
= u[k − 1] + [ I_{m} I_{m} ⋯ O ] ΔU[k].

Continuing in this manner, we obtain

[ u[k] ; u[k + 1] ; ⋮ ; u[k + N_{p} − 1] ] = [ I_{m} ; I_{m} ; ⋮ ; I_{m} ] u[k − 1] + [ \begin{matrix}I_{m} & O & ⋯ & O \\ I_{m} & I_{m} & ⋯ & O \\ ⋮ & & ... & ⋮ \\ I_{m} & I_{m} & ⋯ & I_{m} \end{matrix} ] [ Δu[k] ; Δu[k + 1] ; ⋮ ; Δu[k + N_{p} − 1] ] .

(13.25)

Let

E = [ I_{m} ; I_{m} ; ⋮ ; I_{m} ] and H = [ \begin{matrix}I_{m} & O & ⋯ & O \\ I_{m} & I_{m} & ⋯ & O \\ ⋮ & & ... & ⋮ \\ I_{m} & I_{m} & ⋯ & I_{m} \end{matrix} ] .

Then, we can represent (13.25) as

U[k] = Eu[k − 1] + HΔU[k].  (13.26)

Suppose now that we are faced with constructing a control action subject to the following constraints,

U^{min} ≤ U[k] ≤ U^{max}.

The above constraints can be equivalently represented as

[ \begin{matrix}−U[k] \\ U[k] \end{matrix} ] ≤ [ \begin{matrix}−U^{min} \\ U & max \end{matrix} ] ,

Taking into account (13.26), we write the above as

[ \begin{matrix}− (Eu[k − 1] + HΔU[k]) \\ Eu[k − 1] + HΔU[k] \end{matrix} ] ≤ [ \begin{matrix}−U^{min} \\ U & max \end{matrix} ] .  (13.27)

The above, in turn, can be represented as

[ \begin{matrix}−H \\ H \end{matrix} ] ΔU[k] ≤ [ \begin{matrix}−U^{min} + Eu[k − 1] \\ U^{max} − Eu[k − 1] \end{matrix} ] .  (13.28)

In a special case, when a control designer elects to impose constraints only on the first component of ΔU[k], that is, on Δu[k] only, then this scenario is expressed in terms of ΔU[k] as

[ \begin{matrix}−I_{m} & O & ⋯ & O \\ I_{m} & O & ⋯ & O \end{matrix} ] ΔU[k] ≤ [ \begin{matrix}−U^{min} + Eu[k − 1] \\ U^{max} − Eu[k − 1] \end{matrix} ] .  (13.29)

13.2.3 Constraints on the Plant Output

Recall from (13.16) that the predicted plant output is, Y [k] = W x_{a}[k] + ZΔU[k]. Suppose now that the following constraints are imposed on the predicted plant’s output,

Y^{min} ≤ Y [k] ≤ Y^{max}.

We represent the above as

[ \begin{matrix}−Y [k] \\ Y [k] \end{matrix} ] ≤ [ \begin{matrix}−Y^{min} \\ Y & max \end{matrix} ] .

Taking into the account the expression for Y given by (13.16), we represent the above as

[ \begin{matrix}−Z \\ Z \end{matrix} ] ΔU[k] ≤ [ \begin{matrix}−Y^{min} + W x_{a}[k] \\ Y^{max} − W x_{a}[k] \end{matrix} ] .  (13.30)

The above discussion clearly demonstrates that we need an effective method of minimizing a function of many variables, J(ΔU), subject to inequality constraints such as given by (13.22), (13.27), and (13.30), or their combination. In the following, we present a powerful method for solving such optimization problems. This is the method that we shall use to implement our MPCs.

Example 13.1 We formulate the optimization problem that the model predictive controller is to solve at each sampling period for a discrete dynamic system model,

x[k + 1] = [ \begin{matrix}1 & −1 \\ 1 & −1 \end{matrix} ] x[k] + [ \begin{matrix}2 \\ 1 \end{matrix} ] u[k]
y[k] = [ 1 0 ] x[k],

The system output is to track the unit step. The prediction horizon is N_{p} = 2. The constraints on the control signal are to be imposed only on the first component of U[k] and ΔU[k], respectively, and they have the form:

−3 ≤ u[k] ≤ 5 and − 2 ≤ Δu[k] ≤ 1.

Because N_{p} = 2,

ΔU[k] = [ \begin{matrix}Δu[k] \\ Δu[k + 1] \end{matrix} ] .

The constraints on the control magnitude on the first component of U[k] can be expressed as

[ \begin{matrix}−1 & 0 \\ 1 & 0 \end{matrix} ] ΔU[k] ≤ [ \begin{matrix}3 + u[k − 1] \\ 5 − u[k − 1] \end{matrix} ]

The constraints on the first component of ΔU[k] have the form

[ \begin{matrix}−1 & 0 \\ 1 & 0 \end{matrix} ] ΔU[k] ≤ [ \begin{matrix}2 \\ 1 \end{matrix} ] .

Let

Y = [ \begin{matrix}y(k + 1|k) \\ y(k + 2|k) \end{matrix} ] = W x_{a}[k] + ZΔU and r_{p} = [ \begin{matrix}1 \\ 1 \end{matrix} ] ,

where Y is the vector containing predicted plant outputs over the prediction horizon and r_{p} is the vector of the reference signals over the prediction horizon. Then, the optimization problem has the form

min \frac{1}{2} (r_{p} − Y )^{⊤} Q (r_{p} − Y ) + \frac{1}{2}ΔU^{⊤}RΔU

subject to

[ \begin{matrix}−1 & 0 \\ 1 & 0 \\ −1 & 0 \\ 1 & 0 \end{matrix} ] ΔU[k] ≤ [ \begin{matrix}3 + u[k − 1] \\ 5 − u[k − 1] \\ 2 \\ 1 \end{matrix} ] ,

where Q = Q^{⊤} ≥ 0 and R = R^{⊤} > 0. For example, Q = R = I_{2}.

13.3 Solving Constrained Optimization Problems

If follows from the discussion in the previous section that at each sampling time the MPC calls for a solution to a constrained optimization problem of the form,

minimize J(ΔU)
subject to g(ΔU) ≤ 0,

where g(ΔU) ≤ 0 contains inequality constrains given by (13.22), (13.27), and (13.30), or their combination. In this section, we present an iterative methods for solving the above optimization problems. To proceed, we first present the descent gradient method, followed by the Newton’s method, for solving unconstrained optimization problems of the form,

minimize J(ΔU)

13.3.1 Gradient Descent Method

For the sake of simplicity, we denote the argument of a function of many variables as x, where x ∈ ℝ^{N}, and the function that we will be minimizing will be denoted as f, where f : ℝ^{N} → ℝ.

The method of the gradient descent is based on the following property of the gradient of a differentiable function, f, on ℝ^{N}:

Theorem 13.1 At a given point x^{(0)}, the vector

v = −∇f (x^{(0)})

points in the direction of most rapid decrease of f and the rate of increase of f at x^{(0)} in the direction v is − ‖∇f (x^{(0)})‖, equivalently, the rate of decrease of f at x^{(0)} in the direction v is ‖∇f (x^{(0)})‖.

Thus, if we wish to minimize a differentiable function, f, then moving in the direction of the negative gradient is a good direction. The gradient descent algorithm rests on the above observation and has the form,

x^{(k+1)} = x^{(k)} − α∇f (x^{(k)}) ,  (13.31)

where α > 0 is a step size.

13.3.2 Second-Order Necessary Conditions for a Minimum

Suppose we are given a function f of one variable x. Recall a well-known theorem referred to as the second-order Taylor’s formula or the extended law of mean (see [272, p. 1] or [62, Section 5.6]):

Theorem 13.2 Suppose that f(x), f′(x), f′′(x) exist on the closed interval [a, b] = {x ∈ ℝ : a ≤ x ≤ b}. If x^{∗}, x are any two different points of [a, b], then there exists a point z strictly between x^{∗} and x such that

f(x) = f (x^{∗}) + f′ (x^{∗}) (x − x^{∗}) + \frac{f′′(z)}{2} (x − x^{∗})^{2} .

Using the above formula we observe that if

• f′ (x^{∗}) = 0, and

• f′′ (x^{∗}) > 0, then

f(x) = f (x^{∗}) + a positive number

for all x “close” to x^{∗}. Indeed, if f′′(x) is continuous at x^{∗} and f′′ (x^{∗}) > 0, then f′′ (x) > 0 for all x in some neighborhood of x^{∗}. Therefore,

f(x) > f (x^{∗}) for all x close to x^{∗},

which means that x^{∗} is a strict local minimizer of f.

We now extend the above result for functions of many variables. To proceed, we need the second-order Taylor’s formula for such functions that can be found in [272, p. 11].

Theorem 13.3 Suppose that x^{∗}, x are points in ℝ^{N} and that f is a function of N variables with continuous first and second partial derivatives on some open set containing the line segment

[x^{∗}, x] = {w ∈ ℝ^{N} : w = x^{∗} + t (x − x^{∗}) ; 0 ≤ t ≤ 1}

joining x^{∗} and x. Then there exists a z ∈ [x^{∗}, x] such that

f(x) = f (x^{∗}) + ∇f (x^{∗})^{⊤} (x − x^{∗}) + \frac{1}{2} (x − x^{∗})^{⊤} F (z) (x − x^{∗}) ,  (13.32)

where F (·) is the Hessian of f, that is, the second derivative of f.

Hence if

• x^{∗} is a critical point, that is, ∇f (x^{∗}) = 0, and

• F (x^{∗}) > 0,

then using (13.32), we conclude that

f(x) = f (x^{∗}) + 0 + a positive number

for all x in a neighborhood of x^{∗}. Therefore for all x ≠ x^{∗} in some neighborhood of x^{∗}, we have

f(x) > f (x^{∗}) ,

which implies that x^{∗} is a strict local minimizer.

Figure 13.5: Newton’s method minimizes the quadratic approximation of the objective function that utilizes first and second derivatives of the optimized function.

13.3.3 Newton’s Method

The idea behind Newton’s method for function minimization is to minimize the quadratic approximation rather than the function itself as illustrated in Figure 13.5. Newton’s method seeks a critical point, x^{∗}, of a given function. If at this critical point we have F (x^{∗}) > 0, then x^{∗} is a strict local minimizer of f.

We obtain a quadratic approximation q of f at x^{∗} from the second-order Taylor series expansion of f about x^{∗},

q(x) = f (x^{∗}) + ∇f (x^{∗})^{⊤} (x − x^{∗}) + \frac{1}{2} (x − x^{∗})^{⊤} F (x^{∗}) (x − x^{∗}) .

Note that

q (x^{∗}) = f (x^{∗}) , ∇q (x^{∗}) = ∇f (x^{∗}) ,

as well as their Hessians, that is, their second derivatives evaluated at x^{∗} are equal. A critical point of q can be obtained by solving the algebraic equation,

∇q(x) = 0,

that is, by solving the equation

∇q(x) = ∇f (x^{∗}) + F (x^{∗}) (x − x^{∗}) = 0.

Suppose now that we have a quadratic approximation of f at a point x^{(k)}, that is,

q(x) = f (x^{(k)}) + ∇f (x^{(k)})^{⊤} (x − x^{(k)}) + \frac{1}{2} (x − x^{(k)})^{⊤} F (x^{(k)}) (x − x^{(k)}) .

We assume that det F (x^{(k)}) ≠ 0. Denoting the solution to the above equation as x^{(k+1)}, we obtain

x^{(k+1)} = x^{(k)} − F (x^{(k)})^{−1} ∇f (x^{(k)})

The above is known as the Newton’s method for minimizing a function of many variables f. Note that x^{(k+1)} is a critical point of the quadratic function q that approximates f at x^{(k)}.

A computationally efficient representation of Newton’s algorithm has the form,

x^{(k+1)} = x^{(k)} − Δx^{(k)},

where Δx^{(k)} is obtained by solving the equation,

F (x^{(k)}) Δx^{(k)} = ∇f (x^{(k)}) .

13.4 Minimization Subject to Equality Constraints

We now discuss the problem of finding a point x ∈ ℝ^{N} that minimizes f(x) subject to equality constraints,

h_{1}(x) = 0 ; h_{2}(x) = 0 ; ⋮ ; h_{M}(x) = 0 }

where M ≤ N. We write the above equality constraints in a compact form as

h(x) = 0,

where h : ℝ^{N} → ℝ^{M}. We refer to the set of points satisfying the above constraints as the surface.

We now introduce the notion of the tangent plane to the surface S = {x : h(x) = 0} at a point x^{∗} ∈ S . First, we define a curve on the surface S as a family of points x(t) ∈ S continuously parameterized by t for t ∈ [a, b]. The curve is differentiable if ẋ(t) = dx(t)/dt exists. A curve is said to pass through the point x^{∗} ∈ S if x^{∗} = x (t^{∗}) for some t^{∗} ∈ [a, b]. The tangent plane to the surface S = {x : h(x) = 0} at x^{∗} is the collection of the derivatives at x^{∗} of all differentiable curves on S that pass through x^{∗}.

To proceed, we need the notion of a regular point of the constraints. We say that x^{∗} satisfying the constraints, that is, h (x^{∗}) = 0, is a regular point of the constraints if the gradient vectors,

∇h_{1} (x^{∗}) , …, ∇h_{M} (x^{∗})

are linearly independent.

One can show, see, for example [224, p. 298], that the tangent space at a regular point x^{∗}, denoted T (x^{∗}), to the surface {x : h (x) = 0} at the regular point x^{∗} is

T (x^{∗}) = {y : [ ∇h_{1} (x^{∗})^{⊤} ; ⋮ ; ∇h_{M} (x^{∗})^{⊤} ] y = 0} .  (13.33)

With the above notions in place, we are ready to state and prove the following result which is known as the first-order necessary condition (FONC) for function minimization subject to equality constraints.

Theorem 13.4 Let x^{∗} be a local minimizer (or maximizer) of f subject to the constraints h(x) = 0 and let x^{∗} be a regular point of the constraints. Then there exists a vector λ^{∗} such that

∇f (x^{∗}) + [ ∇h_{1} (x^{∗}) ⋯ ∇h_{M} (x^{∗}) ] λ^{∗} = 0.

Proof Let x(t) be a differentiable curve passing through x^{∗} on the surface S = {x : h(x) = 0} such that ẋ (t^{∗}) = y where t^{∗} ∈ [a, b]. Note that the tangent space is given by (13.33). Because x^{∗} is a local minimizer of f on S, we have

\frac{d}{dt}f(x(t))|_{t=t^{∗}} = 0.

Applying the chain rule to the above gives

∇f (x^{∗})^{⊤} y = 0.

Thus ∇f (x^{∗}) is orthogonal to the tangent space T (x^{∗}). That is, ∇f (x^{∗}) is a linear combination of the gradients ∇h_{1} (x^{∗}) , …, ∇h_{M} (x^{∗}). This fact can be expressed as

∇f (x^{∗}) + [ ∇h_{1} (x^{∗}) ⋯ ∇h_{M} (x^{∗}) ] λ^{∗} = 0

for some constant vector λ^{∗} ∈ ℝ^{M}.

□

The vector λ^{∗} is called the vector of Lagrange multipliers.

We now introduce the Lagrangian associated with the constrained optimization problem,

l(x, λ) = f(x) + λ^{⊤}h(x).

Then the FONC can be expressed as

∇xl(x, λ) = 0
∇λl(x, λ) = 0.

Note that the second of the above condition is equivalent to h(x) = 0, that is,

∇λl(x, λ) = h(x) = 0.

Equivalently the FONC can be written as

∇xl(x, λ) = 0
h(x) = 0.

We now apply Newton’s method to solve the above system of equations iteratively,

[ \begin{matrix}x & (k+1) \\ λ & (k+1) \end{matrix} ] = [ \begin{matrix}x & (k) \\ λ & (k) \end{matrix} ] + [ \begin{matrix}d & (k) \\ y & (k) \end{matrix} ] ,

where d^{(k)} and y^{(k)} are obtained by solving the matrix equation,

[ \begin{matrix}L (x^{(k)}, λ^{(k)}) & Dh (x^{(k)})^{⊤} \\ Dh (x^{(k)}) & O \end{matrix} ] [ \begin{matrix}d & (k) \\ y & (k) \end{matrix} ] = [ \begin{matrix}−∇xl(x, λ) \\ −h (x^{(k)}) \end{matrix} ] ,

where L (x, λ) is the Hessian of l(x, λ) with respect to x, and Dh(x) is the Jacobian matrix of h(x), that is,

Dh(x) = [ ∇h_{1} (x)^{⊤} ; ⋮ ; ∇h_{M} (x)^{⊤} ] .

The above algorithm is also referred to as in the literature as sequential quadratic programming (SQP); see, for example [252, Section 15.5].

13.5 Lagrangian Algorithm for Equality Constraints

The first-order Lagrangian algorithm for the optimization problem involving minimizing f subject to the equality constraints, h(x) = 0, has the form,

x^{(k+1)} = x^{(k)} − α_{k} (∇f (x^{(k)}) + Dh (x^{(k)})^{⊤} λ^{(k)})
λ^{(k+1)} = λ^{(k)} + β_{k}h (x^{(k)}) ,

where α_{k} and β_{k} are positive constants. Note that the update for x^{(k)} is a descent gradient for minimizing the Lagrangian with respect to x, while the update for λ^{(k)} is a gradient ascent for maximizing the Lagrangian with respect to λ. For an analysis of the above algorithm, we recommend [62, pp. 522–524]

13.6 Minimization Subject to Inequality Constraints

We discuss the problem of finding a point x ∈ ℝ^{N} that minimizes f(x) subject to inequality constraints,

g_{1}(x) ≤ 0 ; g_{2}(x) ≤ 0 ; ⋮ ; g_{P} (x) ≤ 0 }

We write the above inequality constraints in a compact form as

g(x) ≤ 0,

where h : ℝ^{N} → ℝ^{P}.

Let x^{∗} be a point satisfying the constraints, that is, g (x^{∗}) ≤ 0, and let J be the set of indices j for which g_{j} (x^{∗}) = 0. Then x^{∗} is said to be a regular point of the constraints g(x) ≤ 0 if the gradient vectors,

∇g_{j} (x^{∗}) , j ∈ J,

are linearly independent.

We say that a constraint g_{j}(x) ≤ 0 is active at x^{∗} if g_{j} (x^{∗}) = 0. Thus the index set J, defined above, contains indices of active constraints.

Theorem 13.5 Let x^{∗} be a regular point and a local minimizer of f subject to g(x) ≤ 0. Then there exists a vector μ^{∗} ∈ ℝ^{P} such that

1. μ^{∗} ≥ 0,

2. ∇f (x^{∗}) + [ ∇g_{1} (x^{∗}) ⋯ ∇g_{P} (x^{∗}) ] μ^{∗} = 0,

3. μ^{∗⊤}g (x^{∗}) = 0.

Proof First note that because x^{∗} is a relative minimizer over the constraint set {x : g(x) ≤ 0}, it is also a minimizer over a subset of the constraint set obtained by setting the active constraints to zero. Therefore, for the resulting equality constrained problem, by Theorem 13.4, we have

∇f (x^{∗}) + [ ∇g_{1} (x^{∗}) ⋯ ∇g_{P} (x^{∗}) ] μ^{∗} = 0,  (13.34)

where μ_{j}^{∗} = 0 if g_{j} (x^{∗}) < 0. This means that μ^{∗⊤}g (x^{∗}) = 0. Thus μ_{i} may be non-zero only if the corresponding constraint is active, that is, g_{i} (x^{∗}) = 0.

It thus remains to show that μ^{∗} ≥ 0. We prove this by contraposition. We suppose that for some k ∈ J, we have μ_{k}^{∗} < 0. Consider next a surface formed by all other active constraints, that is, the surface

{x : g_{j}(x) = 0, j ∈ J, j ≠ k}.  (13.35)

Also consider a tangent space to the above surface at x^{∗}. By assumption x^{∗} is a regular point of the active constraints. Therefore, there exists a vector y such that

∇g_{k} (y)^{⊤} y < 0.

Let now x(t), t ∈ [−a, a], a > 0, be a curve on the surface (13.35) such that

ẋ(0) = y.

Note that for small t, the curve x(t) is feasible. We apply the transposition operator to the both sides of (13.34) to obtain

∇f (x^{∗})^{⊤} + ∑_{i=1}^{P} μ_{i}^{∗}∇g_{i} (x^{∗})^{⊤} = 0^{⊤}.

Post-multiplying the above by y an taking into account the fact that y belongs to the tangent space to the surface (13.35) gives

∇f (x^{∗})^{⊤} y = −μ_{k}^{∗}∇g_{k} (x^{∗})^{⊤} y
< 0.

Suppose, without loss of generality, that ∥y∥_{2} = 1. Then, we have that

\frac{df(x(t))}{dt} = ∇f (x^{∗})^{⊤} y
< 0,

that is, the rate of increase of f at x^{∗} in the direction y is negative. This would mean that we could decrease the value of f moving just slightly away from x^{∗} along y while, at the same time, preserving feasibility. But this contradicts the minimality of f (x^{∗}). In summary, if x^{∗} is a relative minimizer then we also have the components of μ^{∗} all non-negative.

□

The vector μ^{∗} is called the vector of the Karush-Kuhn-Tucker (KKT) multipliers.

13.7 Lagrangian Algorithm for Inequality Constraints

We present a first-order Lagrangian algorithm for the optimization problem involving inequality constraints,

minimize f(x)
subject to g(x) ≤ 0,

where g : ℝ^{N} → g : ℝ^{P}. The Lagrangian function is

l(x, μ) = f(x) + μ^{⊤}g(x).

The first-order Lagrangian algorithm for the above optimization problem involving minimizing f subject to the inequality constraints, g(x) ≤ 0, has the form,

x^{(k+1)} = x^{(k)} − α_{k} (∇f (x^{(k)}) + Dg (x^{(k)})^{⊤} μ^{(k)})
μ^{(k+1)} = [μ^{(k)} + β_{k}g (x^{(k)})]_{+} ,

where the operation [·]_{+} = max(·, 0) is applied component-wise.

For an analysis of the above algorithm, we recommend [62, pp. 524–528]

Example 13.2 We illustrate the design of the discrete MPC subject to the constraints on the control input on a plant modeled by the following continuous state-space model,

\begin{matrix}ẋ & = & [ \begin{matrix}−0.1 & −3.0 \\ 1 & 0 \end{matrix} ] x + [ \begin{matrix}1 \\ 0 \end{matrix} ] u \\ y & = & [ 0 10 ] x. \end{matrix} \}  (13.36)

The system with the transfer function as the above system was used by Wang [342, p. 70] to test her discrete MPC designs. We use different optimizer than Wang. The reader can compare the results of two different MPC synthesis methods. The MPC controller’s objective is to force the plant’s output to track the unit step.

We proceed to generate the discrete state-space model employing the MATLAB’s command c2dm. We use the sampling time h = 0.1. The command input is the unit step. The weight matrices in the cost function (13.17) are R = 0.01I_{3} and Q = I_{3}. The prediction horizon is N_{p} = 3. Our objective is to design a discrete-MPC subject to the constraints on the control of the form,

−0.3 ≤ u[k] ≤ 0.5

First, we consider the case when the constraints are imposed only on the first component of U(k). The constraints take the form of (13.29). Thus, in our example, the constraints can be written as,

[ \begin{matrix}−1 & 0 & 0 \\ 1 & 0 & 0 \end{matrix} ] [ \begin{matrix}Δu[k] \\ Δu[k + 1] \\ Δu[k + 2] \end{matrix} ] ≤ [ \begin{matrix}0.3 + u[k − 1] \\ 0.5 − u[k − 1] \end{matrix} ] .

We employed the first-order Lagrangian algorithm for inequality constraints presented in Section 13.7, where α_{k} = β_{k} = 0.005. We set zero initial conditions. Plots of the control effort and the plant’s output are shown in Figure 13.6. We wish to emphasize that the simulations were performed for the discretized plant. Thus, the control and output were evaluated at the sampling instances and marked on the plots using circles. For the purpose of visualization the control action was presented in the form of a staircase function while the plant output values were connected by straight lines.

Next we tested the discrete MPC where the constraints, −0.3 ≤ u[k] ≤ 0.5, were imposed on all the elements of U[k]. We use (13.28) to express the constraints, where

[ \begin{matrix}−1 & 0 & 0 \\ −1 & −1 & 0 \\ −1 & −1 & −1 \\ 1 & 0 & 0 \\ 1 & 1 & 0 \\ 1 & 1 & 1 \end{matrix} ] [ \begin{matrix}Δu[k] \\ Δu[k + 1] \\ Δu[k + 2] \end{matrix} ] ≤ [ \begin{matrix}0.3 + u[k − 1] \\ 0.3 + u[k − 1] \\ 0.3 + u[k − 1] \\ 0.5 − u[k − 1] \\ 0.5 − u[k − 1] \\ 0.5 − u[k − 1] \end{matrix} ] .

Example 13.3 In this example, we apply the discrete MPC controller from Example 13.2 to the continuous plant modeled by (13.36). The MPC controller’s objective is to force the plant’s output to track the unit step. The zero-order hold is applied to the discrete MPC controller’s output sequence resulting in a piece-wise constant input to the continuous plant Figure 13.6: Plots of control effort and the plant’s output versus time of the closed-loop system of Example 13.2 for the case when the constraint was imposed only on the first component of U[k].

Figure 13.7: Plots of control effort and the plant’s output versus time of the closed-loop system of Example 13.2 for the case when the constraints were imposed on all the components of U[k].

Figure 13.8: Plots of control effort and the plant’s output versus time of the closed-loop system of Example 13.3 for the case when the constraints were imposed only on the first component of U(k).

model. In Figure 13.8, we show plots of the plant output as well as the control effort versus time. As in the previous example, the prediction horizon is N_{p} = 3 and the constraints on the control are

−0.3 ≤ u[k] ≤ 0.5

13.8 Observer-Based MPC

If the plant state is not accessible, we need to use an observer in the controller implementation as shown in Figure 13.9.

13.9 Notes

The key reason for huge popularity of the MPC approach is its ability to systematically take into account constraints thus allowing processes to operate at the limits of achievable performance [188]. For some impressive industrial applications of MPCs see [188, 49]. Nonlinear model predictive controllers are presented in [233, 240].

Figure 13.9: MPC implementation when the plant’s state is not available.

An insightful description of model predictive control is offered by Mayne et al. [234, pp. 789–790], where they write, “Model predictive control (MPC) or receding horizon control (RHC) is a form of control in which the current control action is obtained by solving on-line, at each sampling instant, a finite horizon open-loop optimal control problem, using the current state of the plant as the initial state; the optimization yields an optimal control sequence and the first control in this sequence is applied to the plant. This is its main difference from conventional control which uses a pre-computed control law. With the clarity gained by hindsight, it can be recognized that the raison d’être for model predictive control is its ability to handle control problems where off-line computation of a control law is difficult or impossible although other features, such as its capability for controlling multivariable plants, were initially deemed more important. Nearly every application imposes constraints; actuators are naturally limited in the force (or equivalent) they can apply, safety limits states such as temperature, pressure and velocity and efficiency often dictates steadystate operation close to the boundary of the set of permissible states. The prevalence of hard constraints is accompanied by a dearth of control methods for handling them, despite a continuous demand from industry that has had, in their absence, to resort often to ad hoc methods. Model predictive control is one of few suitable methods, and this fact makes it an important tool for the control engineer, particularly in the process industries where plants being controlled are sufficiently ‘slow’ to permit its implementation.”

13.10 Exercises

Exercise 13.1 Derive a closed-form solution of a general quadratic minimization problem subject to a linear equality constraint,

minimize \frac{1}{2}x^{⊤}Qc + c^{⊤}x, x ∈ ℝ^{n}
subject to Ax = b,

where Q = Q^{⊤} > 0, A ∈ ℝ^{m×n}, m < n, and rank A = m.

Exercise 13.2 Find all point that satisfy the KKT conditions for the following quadratic minimization problem subject to a linear equality constraint,

minimize \frac{1}{2}x^{⊤}Qc + c^{⊤}x, x ∈ ℝ^{n}
subject to Ax ≤ b,

where Q = Q^{⊤} > 0, A ∈ ℝ^{m×n}, m < n, rank A = m, and b ≥ 0.

Exercise 13.3 Consider the following optimization problem,

optimize (x_{1} − 2)^{2} + (x_{2} − 1)^{2}
subject to x_{2} − x_{1}^{2} ≥ 0
2 − x_{1} − x_{2} ≥ 0
x_{1} ≥ 0.

The point x^{∗} = [ 0 0 ]^{T} satisfies the KKT conditions.

(i) Does x^{∗} satisfy the FONC for minimum or maximum? What are the KKT multipliers?

(ii) Does x^{∗} satisfy the SOSC? Carefully justify your answer.

Exercise 13.4 (i) Use the KKT conditions to solve the quadratic minimization problem subject to a linear inequality constraint,

minimize \frac{1}{2}x^{⊤}Qc + c^{⊤}x, x ∈ ℝ^{n}
subject to Ax ≤ b,

Q = I_{3}, c = [ \begin{matrix}−2 \\ −3 \\ −1 \end{matrix} ] , A = [ \begin{matrix}1 & 1 & 1 \\ 3 & −2 & −3 \\ 1 & −3 & 2 \end{matrix} ] , and b = [ \begin{matrix}1 \\ 1 \\ 1 \end{matrix} ] .

(ii) Implement in MATLAB the Lagrangian algorithm for inequality constraints of Section 13.7 to solve the above problem. You may wish to compare your solution with that of Wang [342, p. 60], where an active set method was used to solve this numerical problem.

Exercise 13.5 (i) Use the KKT conditions to solve the quadratic minimization problem subject to a linear inequality constraint,

minimize \frac{1}{2}x^{⊤}Qc + c^{⊤}x, x ∈ ℝ^{n}
subject to Ax ≤ b,

Q = [ \begin{matrix}2 & −1 \\ −1 & 1 \end{matrix} ] , c = [ \begin{matrix}−1 \\ 0 \end{matrix} ] , A = [ \begin{matrix}−1 & 0 \\ 0 & −1 \\ 3 & 2 \end{matrix} ] , and b = [ \begin{matrix}0 \\ 0 \\ 4 \end{matrix} ] .

(ii) Implement in MATLAB the Lagrangian algorithm for inequality constraints of Section 13.7 to solve the above problem. You may wish to compare your solution with that of Wang [342, p. 64], where the Hildreth’s quadratic programming algorithm was used to solve this numerical problem.

Exercise 13.6 (i) Use the KKT conditions to solve the quadratic minimization problem subject to a linear inequality constraint,

minimize \frac{1}{2}x^{⊤}Qc + c^{⊤}x, x ∈ ℝ^{n}
subject to Ax ≤ b,

Q = I_{2}, c = [ \begin{matrix}−2 \\ −2 \end{matrix} ] , A = [ \begin{matrix}1 & 0 \\ −1 & 0 \\ 0 & 1 \\ 0 & −1 \end{matrix} ] , and b = [ \begin{matrix}1 \\ 0 \\ 1 \\ 0 \end{matrix} ] .

(ii) Implement in MATLAB the Lagrangian algorithm for inequality constraints of Section 13.7 to solve the above problem. You may wish to compare your solution with that of Wang [342, p. 67], where the Hildreth’s quadratic programming algorithm was used to solve this numerical problem.

Exercise 13.7 Show that if the matrix A in the model (3.27) is invertible, then the matrix Γ in the discretized model (3.31) can also be computed as

Γ = (Φ − I_{n})A^{−1}B.

Exercise 13.8 Design an MPC for the linearized model of the nonlinear model of one-link robot manipulator presented in Subsection 2.4.5. Proceed as follows:

(a) Linearize the nonlinear model about the equilibrium pair

x_{e} = [ 30π/180 0 −4.9 ]^{⊤} , u_{e} = −4.9.

(b) Discretize the linearized model and use it as the basis for your design. You can help yourself using MATLAB’s c2dm. You as a designer select the sampling interval h, for example, h = 0.1.

(c) First design an MPC without any constraints and implement the controller on the nonlinear continuous model. Write a script that animates the behavior of the closedloop system for different initial conditions.

(d) Impose constraints on the input and re-design your MPC. Implement the controller on the nonlinear continuous model. Write a script that animates the behavior of the closed-loop system for different initial conditions.

(e) Implement the combined MPC controller-observer compensator on the nonlinear continuous model. Write a script that animates the behavior of the closed-loop system for different initial conditions.

Exercise 13.9 Design an MPC for the linearized model of Exercise 4.5 of the nonlinear model of a double-link robot manipulator of Subsection 2.4.6. Proceed as follows:

(a) Discretize the linearized model and use it as the basis for your design. You can help yourself using MATLAB’s c2dm. You as a designer select the sampling interval h, for example, h = 0.1.

(b) First design an MPC without any constraints and implement the controller on the nonlinear continuous model. Write a script that animates the behavior of the closedloop system for different initial conditions.

(c) Impose constraints on the input and re-design your MPC. Implement the controller on the nonlinear continuous model. Write a script that animates the behavior of the closed-loop system for different initial conditions.

(d) Implement the combined MPC controller-observer compensator on the nonlinear continuous model. Write a script that animates the behavior of the closed-loop system for different initial conditions.

Proportional-Integral-Derivative (PID) Control

Thomas Stevic, a controls engineer at DMAX, said that when designing a solution to a problem, he finds that he typically has two masters to serve. The first master is the person who is responsible for the output of his design. This is often “the person who pays the bills,” Stevic said. “These people usually do not care how a machine or process works. Their concern is: Does the machine make what it is supposed to make? While exotic solutions may increase quality, decrease production time, make better use of resources and, from an engineering viewpoint, be very cool to use and inch closer to perfection, their use is not often a benefit to the first master.” The second master is described by Stevic as the person or people who will “maintain the system after I am gone. Ultimately, my designs have to address the least capable people who may be required to maintain or modify my design.” Given the requirements of the second master, Stevic says that, given a choice between using adaptive neural network control or a thermostat, if the thermostat will work, that is what he will choose. “If neural nets or fuzzy logic are required to produce the desired result, I will use them, but only after examining the simpler solution,” he said. [121, page 57]

688 CHAPTER 14. PROPORTIONAL-INTEGRAL-DERIVATIVE (PID) CONTROL

In the previous chapters, we analyzed control algorithms that require a mathematical model of the dynamical system to be controlled. The mathematical model is then used to construct a controller. In many practical situations, however, it is not feasible to obtain a quality mathematical model of the controlled system. In this chapter, we present proportional-integral-derivative (PID) controllers that are most useful when a mathematical model of the process to be controlled is not available to the control engineer.

The proportional-integral-derivative (PID) controllers are without a doubt the most widely used controllers in industry today. Åström and Hägglund [15, p. 198] regard the PID controller as the “bread and butter” of control engineering. Knospe [183] estimates that over 90% of control loops employ PID control.

To better appreciate the effectiveness of the PID control strategies, we first discuss a general control design principle referred to as the internal model principle (IMP).

14.1 The Internal Model Principle (IMP)

Consider a block diagram of a closed-loop system shown in Figure 14.1. Using basic rules of

[figure: R(s) = \frac{N_{r}(s)}{D_{r}(s)}; r(t); +; −; E(s); e(t); \frac{N_{c}(s)}{D_{c}(s)}; G_{c}(s); Controller; \frac{N_{p}(s)}{D_{p}(s)}; Plant; G_{p}(s); Y (s); y(t)]

Figure 14.1: Block diagram of a closed-loop control system.

the block diagram algebra yields

E(s) = R(s) − G_{p}(s)G_{c}(s)E(s)
= \frac{N_{r}(s)}{D_{r}(s)} − \frac{N_{p}(s)N_{c}(s)}{D_{p}(s)D_{c}(s)}E(s).

Performing simple manipulations, we obtain

E(s) = \frac{D_{p}(s)D_{c}(s)}{D_{p}(s)D_{c}(s) + N_{p}(s)N_{c}(s)} \frac{N_{r}(s)}{D_{r}(s)}  (14.1)

Our objective is to design a controller G_{c}(s) = \frac{N_{c}(s)}{D_{c}(s)} so that

lim_{t→∞} e(t) = lim_{t→∞} (r(t) − y(t)) = 0,

where e(t) = L^{−1}(E(s)) is the inverse Laplace transform of E(s).

Suppose that the poles of the Laplace transform of the reference signal r(t) are in the closed right half-plane, that is, they belong to the set {s : ℜ(s) ≥ 0}, where ℜ(s) denotes the real part of the complex variable s. The polynomial

P_{c}(s) = D_{p}(s)D_{c}(s) + N_{p}(s)N_{c}(s)

is the closed-loop characteristic polynomial (CLCP) of the closed-loop system depicted in Figure 14.1 and the closed-loop characteristic polynomial zeros are the closed-loop poles. Using (14.1), we can easily prove the following result that we refer to as the Internal Model Principle (IMP).

Theorem 14.1 (Internal Model Principle) In the configuration depicted in Figure 14.1, where the poles of R(s) are in the closed right half-plane,

lim_{t→∞} e(t) = 0

if and only if

1. the closed-loop poles are in the open left-half plane;

2. D_{r}(s) is a factor of the open-loop characteristic polynomial D_{p}(s)D_{c}(s), that is, there is a polynomial, say Q(s), such that D_{p}(s)D_{c}(s) = Q(s)D_{r}(s).

The second condition of the IMP an be interpreted as follows: the controller, G_{c}(s), must be chosen in such a way that the open-loop transfer function, G_{p}(s)G_{c}(s), contains a model of the reference signal to be tracked. If none of the poles of R(s) is a pole of the plant’s transfer function, G_{p}(s), then we can restate the IMP as follows:

Any good tracking controller must stabilize the closed-loop system and must contain a model of the reference signal. For an alternative approach to the IMP, we recommend Wolovich [353, pp. 254–255]. We illustrate the IMP with two numerical examples. We use the final value theorem that

we state next.

[figure: R(s) = \frac{1}{s}; +; −; E(s); e(t); G_{c}(s) =?; G_{c}(s) = \frac{N_{c}(s)}{D_{c}(s)}; Controller; \frac{1}{s+2}; Plant; Y (s); y(t)]

Figure 14.2: A closed-loop control system of Example 14.1.

Theorem 14.2 Suppose that F (s) = L(f(t)) has poles only in the open left half-plane with possible exception of a single-order pole at s = 0. Then

lim_{s→0} sF (s) = lim_{t→∞} f(t).

Example 14.1 For the closed-loop system shown in Figure 14.2, our objective is to construct a transfer function G_{c}(s) such that lim_{t→∞} e(t) = 0. The Laplace transform of the tracking error, e(t), is

E(s) = \frac{1}{1 + \frac{1}{s+2} \frac{N_{c}(s)}{D_{c}(s)}} \frac{1}{s} = \frac{(s + 2)D_{c}(s)}{(s + 2)D_{c}(s) + N_{c}(s)} \frac{1}{s}.

Let N_{c}(s) = 1 and D_{c}(s) = s, that is, let

G_{c}(s) = \frac{1}{s},

Then,

E(s) = \frac{(s + 2)s}{s^{2} + 2s + 1} \frac{1}{s} = \frac{s + 2}{s^{2} + 2s + 1}.

It is easy to check using, for example, the final value theorem that lim_{t→∞} e(t) = 0. Thus, in this case, a simple integral (I) controller does the job.

[figure: R(s) = \frac{1}{s^{2}}; +; −; E(s); e(t); G_{c}(s) =?; G_{c}(s) = \frac{N_{c}(s)}{D_{c}(s)}; Controller; \frac{1}{s+2}; Plant; Y (s); y(t)]

Figure 14.3: A closed-loop control system of Example 14.2.

Example 14.2 Consider the closed-loop system shown in Figure 14.3. We wish to construct a controller G_{c}(s) = \frac{N_{c}(s)}{D_{c}(s)} so that lim_{t→∞} e(t) = 0, where the reference signal is the unit ramp function. In Example 14.1, the simple integrator was all that we needed for the unit step as the reference input. Let us try the same controller here. Then, the Laplace transform of the error signal is

E(s) = \frac{1}{1 + \frac{1}{s+2} \frac{N_{c}(s)}{D_{c}(s)}} \frac{1}{s^{2}}
= \frac{(s + 2)D_{c}(s)}{(s + 2)D_{c}(s) + N_{c}(s)} \frac{1}{s^{2}}
= \frac{s + 2}{s^{2} + 2s + 1} \frac{1}{s}.

Note that now e(∞) = 2. Thus the simple integrator is not enough now to force the steadystate error to zero. Note that in this case the polynomial D_{r}(s) was not a factor of D_{c}(s).

Let us then try a controller that would satisfy the second condition of the IMP, the divisibility condition. If we take the double integrator then the divisibility condition will be satisfied and we obtain

E(s) = \frac{s + 2}{s^{3} + 2s^{2} + 1}.

However, e(t) is divergent because the poles of E(s) are not stable. The three poles of E(s) are located at -2.2056, and 0.1028 ± j0.6655. This time the first condition of the Internal Model Principle is not satisfied.

Finally, let us try a controller of the form

G_{c}(s) = \frac{N_{c}(s)}{D_{c}(s)} = \frac{s + a}{s^{2}},  (14.2)

where a is a design parameter that should be chosen to satisfy the both conditions of the IMP. Note that the above controller contains the model of the reference signal. Thus, we only need to select the design parameter a so that the CLCP, P_{c}(s), has its zeros in the open left half-plane, where

P_{c}(s) = D_{p}(s)D_{c}(s) + N_{p}(s)N_{c}(s)
= (s + 2)s^{2} + s + a
= s^{3} + 2s^{2} + s + a

Selecting a = 1 yields P_{c}(s) with its zeros at

−1.7549, −0.1226 ± j0.7449

and

E(s) = \frac{s + 2}{s^{3} + 2s^{2} + s + 1}

Applying the final value theorem to E(s), we obtain, lim_{t→∞} e(t) = 0.

The selection of a = 1 results in the desired steady-state behavior of the closed-loop system. However, the above controller may, or may not, guarantee the desired transient performance of the closed-loop system. Thus further tuning of the parameter a may be necessary.

In industrial applications, control engineers usually specify the performance of the controlled system based on the system step response. In such a case r(t) is a step function, which sometimes is also referred to as the constant set-point, which can be represented as

r(t) = \{ \begin{matrix}0 & for & t < 0 \\ A & for & t ≥ 0, \end{matrix}

where A is the constant set-point value. Recall that the Laplace transform of the above signal is

R(s) = L(r(t)) = \frac{A}{s}.

Taking the above into account it is easy to see that the integral (I) controller contains the model of the unit step function as well as any scalar multiple of it. If, in addition, the zeros of the CLCP are all in the open left-half plane, then e(∞) = 0 for any step input. For this reason, the I controller is often referred to as as reset controller because it automatically ensures e(∞) = 0 without an operator manually altering or “resetting” the set-point value to achieve a desired output value.

14.2 PID Controller Architecture

We begin our discussion with a basic PID controller architecture, also known as the type A PID controller; it consists of three-terms. Later we discuss two modifications, B and C, of the type A PID control architecture. The three terms of a PID controller attempt to fulfill the transient and steady-state specifications.

To proceed, we need some notation. Let u(t) denote the output of the controller, which is the input signal to the plant. Recall that e(t) is the tracking error, that is, the difference between the desired plant output, r(t), and the actual plant output, y(t). Thus

e(t) = r(t) − y(t).  (14.3)

The three-term type A PID control law can be described as

u(t) = u_{P} (t) + u_{I}(t) + u_{D}(t),  (14.4)

where u_{P} is the proportional term, u_{I} is the integral term, and u_{D} is the derivative term. We now analyze each term one by one.

• The proportional term, u_{P}, has the form

u_{P} (t) = K_{P} e(t),  (14.5)

where K_{P} is the proportional gain.

The proportional term responds immediately to the current tracking error. Typically, however, it cannot achieve the desired tracking accuracy without excessively large gain.

• The integral term, u_{I}, is given by

u_{I}(t) = \frac{K_{P}}{T_{I}} ∫_{0}^{t} e(τ)dτ,  (14.6)

where T_{I} is the integral time constant. The integral term yields a zero steady-state error in tracking a step function (a constant set-point). This term is slow in response to the current tracking error.

• The derivative term, u_{D}, is

u_{D}(t) = K_{P} T_{D} \frac{de(t)}{dt},  (14.7)

where T_{D} is the derivative time constant. The derivative term is especially effective for plants with significant dead-time resulting in a poor representation of previous control actions in the current tracking error. This, in turn, as pointed out by Knospe [183], may lead to large transient error when PI control is used. The derivative action combats this problem by generating its action based on a prediction of future tracking error. But there is a cost to be paid—the derivative term amplifies higher frequencies sensor noise. To alleviate the problem, a filtering of the differentiated signal is employed thus resulting in a PID controller with a derivative filter referred to as a PIDF controller. We present a PIDF controller in a section on implementing PID controllers.

Combining the above terms together, we obtain the type A PID control action, in the time domain,

u(t) = u_{P} (t) + u_{I}(t) + u_{D}(t)
= K_{P} (e(t) + \frac{1}{T_{I}} ∫_{0}^{t} e(τ)dτ + T_{D} \frac{de(t)}{dt}) .  (14.8)

It is advantageous to work in the Laplace domain when analyzing the PID-controlled plant. The Laplace transfer function of the type A PID controller, denoted G_{P ID}(s), is

G_{P ID}(s) = \frac{U(s)}{E(s)} = K_{P} (1 + \frac{1}{T_{I}s} + T_{D}s) ,  (14.9)

where U(s) = L(u(t)) is the Laplace transform of u(t) and E(s) = L(e(t)) is the Laplace transform of the tracking error e(t). In Figure 14.4, we show a PID-controlled system with the type A PID controller in the forward path.

14.3 PID Controller Tuning

The control engineer’s task is to select the PID controller’s parameters to meet given performance specifications. This process is referred to as the controller tuning. The most widely

[figure: Desired output; Reference input; +; −; K_{P} (1 + \frac{1}{T_{I}s} + T_{D}s); Plant; Actual output]

Figure 14.4: A PID-controlled system.

Figure 14.5: Open-loop test on the plant—measuring the step response of the plant.

known PID tuning rules were proposed by Ziegler and Nichols [374] in 1942. They proposed two sets of tuning rules of the type A PID controllers. In both cases the controller’s parameters are selected based on the experimental measurements of the controlled process.

14.3.1 The Open-Loop Method—Tuning Based on Open-Loop Tests on the Plant

The first method of choosing control settings for the PID controller, proposed by Ziegler and Nichols, is based on open-loop tests on the plant as shown in Figure 14.5. There is a restriction on the applicability of this method. The method can only be applied if the step response is S-shaped as shown in Figure 14.6. The S-shaped step response can be characterized using three parameters: the delay time (dead-time) L, the time constant T, and the steady-state gain (process gain) K.

We add that a plant with the S-shaped step response can be approximately modeled by a first-order transfer function with transportation delay,

\frac{Y (s)}{U(s)} = G_{d}(s) = \frac{Ke^{−Ls}}{T s + 1}.  (14.10)

We illustrate the procedure of approximating a system with an S-shaped step response using Figure 14.6: An S-shaped step response and its characterization using three parameters.

a first-order transfer function with transportation delay given by (14.10) with the following example.

Example 14.3 We consider, as in Ogata [255, p. 444], a plant modeled by the second-order transfer function

G_{p}(s) = \frac{10}{(s + 1)(s + 5)} = \frac{10}{s^{2} + 6s + 5}.  (14.11)

Using the procedure illustrated in Figure 14.6, we approximate the second-order transfer function (14.11) using the first-order transfer function with transportation delay,

G_{d}(s) = \frac{Ke^{−Ls}}{T s + 1} = \frac{2e^{−0.053s}}{0.798s + 1}.  (14.12)

In Figure 14.7, we compare the step responses of the above two transfer functions. One can see that approximating the S-shaped step step response of a dynamical system using a firstorder transfer function with transportation delay is a challenging problem. Optimization methods, such as genetic algorithms and particle swarm optimizers are the tools that can be used here effectively to determine parameters L and T that result in G_{d}(s) that “best” approximates a given G_{p}(s).

Ziegler and Nichols proposed a set of tuning rules, formulas, using the parameters that characterize the step response of the plant modeled by the transfer function (14.10). These tuning formulas are shown in Table 14.1.

Figure 14.7: Comparison of step responses of the second-order system and its approximation using a first-order system with transportation delay in Example 14.3.

Table 14.1: Ziegler-Nichols tuning formulas based on the step response of the plant (openloop method).

Controller K_{P} T_{I} T_{D}
P  \frac{T}{L} ∞ 0
PI  0.9\frac{T}{L} \frac{L}{0.3} 0
PID  1.2\frac{T}{L} 2L 0.5L

Substituting the tuning parameters given in Table 14.1 into the transfer function of the PID controller gives

G_{P ID}(s) = K_{P} (1 + \frac{1}{T_{I}} s + T_{D}s)
= 1.2\frac{T}{L} (1 + \frac{1}{2Ls} + 0.5Ls)
= 0.6T \frac{(s + \frac{1}{L})^{2}}{s} .  (14.13)

Thus the type A PID controllers tuned using the formulas of the open-loop method can be parameterized by two parameters, T and L.

The ACT controller manufacturer, as well as Li, Ang, and Chong [211], present slightly Figure 14.8: Experimental setup to determine the critical gain, K_{cr} and critical period of oscillation, P_{cr}.

modified Ziegler-Nichols open-loop tuning formulas shown in Table 14.2. The only difference between the formulas shown in Table 14.1 and those in Table 14.2 is in the first column where all the column elements of the first column of Table 14.2 are obtained by dividing by K the corresponding elements of Table 14.1.

Table 14.2: Modified Ziegler-Nichols tuning formulas based on the step response of the plant (open-loop method).

Controller K_{P} T_{I} T_{D}
P  \frac{T}{LK} ∞ 0
PI  0.9\frac{T}{LK} \frac{L}{0.3} 0
PID  1.2\frac{T}{LK} 2L 0.5L

14.3.2 The Closed-Loop Method—Tuning Based on Closed-Loop Tests on the Plant

The second method of Ziegler and Nichols of tuning PID controllers is also known as the the closed-loop method. This method requires an experiment on the closed-loop system as shown in Figure 14.8.

In this experiment, we have a pure P-controller in the feed-forward path. A step is applied as the input and the response is observed. The test is repeated with increased or decreased controller gain until a sustained oscillation is achieved, as shown in Figure 14.9. This gain is called the “critical gain” and denoted K_{cr}. The period of the sustained oscillation is called the critical period and denoted P_{cr}. Using the parameters K_{cr} and P_{cr}, Ziegler and Nichols proposed tuning formulas depicted in Table 14.3. Substituting the tuning parameters from

Figure 14.9: Measuring the period of the sustained oscillation, P_{cr}.

Table 14.3: Ziegler-Nichols tuning formulas based on critical gain K_{cr} and critical period P_{cr} (closed-loop method).

Controller K_{P}  T_{I}  T_{D}
P  0.5K_{cr}  ∞  0
PI  0.45K_{cr}  \frac{P_{cr}}{1.2}  0
PID  0.6K_{cr} 0.5P_{cr} 0.125P_{cr}

Table 14.3 into the transfer function of the type A PID controller, we obtain

G_{P ID}(s) = K_{P} (1 + \frac{1}{T_{I}} s + T_{D}s)
= 0.6K_{cr} (1 + \frac{1}{0.5P_{cr}s} + 0.125P_{cr}s)
= 0.075K_{cr}P_{cr} \frac{(s + \frac{4}{P_{cr}} )^{2}}{s} .  (14.14)

As in the open-loop method, the type A PID controllers tuned using the formulas of the closed-loop method can be parameterized by two parameters; this time by K_{cr} and P_{cr}.

In summary, both the open-loop and closed-loop Ziegler-Nichols tuning rules allow us to parameterize the type A PID controllers using just two parameters. That is, we can represent a family of type A PID controllers as

G_{A−P ID} = K_{c} \frac{(s + a)^{2}}{s} ,  (14.15)

where K_{c} and a are design parameters. The above representation can be used to computationally obtain parameters’ values that satisfy design specifications—see Exercise 14.2.

14.4 Implementing PID Controllers

If the reference input is a step, then because of the presence of the derivative term, the controller output will involve an impulse function. In addition, the derivative term amplifies high frequency sensor noise. To alleviate the above problems, the pure derivative term is replaced with a derivative filter, which results in the PIDF controller.

14.4.1 PIDF Controller

In the PIDF controller, the pure differentiator is approximated with the pure differentiator cascaded with a first-order low-pass filter. Thus the pure derivative term is replaced by the approximate differentiator and so the derivative term contribution would take the form,

U_{D}(s) = \frac{K_{P} T_{D}s}{γT_{D}s + 1}E(s),  (14.16)

where γ is a small parameter, for example, γ = 0.1. The transfer function of the type A PIDF controller has the form

G_{P IDF} = K_{P} (1 + \frac{1}{T_{I}} s + \frac{T_{D}s}{γT_{D}s + 1}) .  (14.17)

For a step reference, the PIDF controller output will involve a sharp pulse rather than an impulse function. This is referred to as the set-point kick phenomenon.

Example 14.4 In this example, we use the open-loop Ziegler-Nichols tuning formulas given in Table 14.1 to tune the PIDF controller for plant models in Example 14.3. We first apply the PIDF controller to the second-order transfer function given by (14.11). We add that the proportional gain was tuned as

K_{P} = 1.2\frac{T}{L} = 18.068

because for the modified gain, K_{P} = 1.2\frac{T}{LK}, the closed-loop system becomes unstable. The remaining parameters are:

T_{I} = 2L = 0.106, T_{D} = 0.5L = 0.0265, and γ = 0.1.

The plant output and the control signal plots versus time are shown in Figure 14.10. The system output is very oscillatory and takes long time to settle down. We can observe the set-point kick phenomenon in the plot of the control effort versus time.

Figure 14.10: PIDF-controlled second-order system’s step response of Example 14.4, where the reference input is a step function delayed by 0.5 s in order to better observe the set-point kick phenomenon.

Figure 14.11: Step response of the PIDF-controlled first-order system with transportation delay in Example 14.4.

We then applied the type A PIDF controller to the transfer function given by (14.12). Here, on the other hand, we used K_{P} = 1.2\frac{T}{LK} because for K_{P} = 1.2\frac{T}{L} the closed-loop system becomes unstable. The plant output and the control signal plots versus time are shown in Figure 14.11. Here, as in the previous case, we can observe a set-point kick phenomenon. The closed-loop system is much faster here but the overshoot is at unacceptable level.

We conclude that the open-loop Ziegler-Nichols tuning rules do not necessarily yield excellent performance of the PID-controlled system. The controller requires careful additional tuning. To illustrate this point, we doubled the values of K_{P}, T_{I}, and T_{D}. The re-tuned PIDF controller performance applied to the second-order plant is shown in Figure 14.12. This re-tuning was suggested by Ogata [256, p. 165]. However the re-tuned controller does not work for the first-order plant with transportation delay. The closed-loop system driven by this controller is in fact unstable.

As we observed before, in addition to the transient performance, the designer has to deal with the set-point kick phenomenon. We address this issue next.

Figure 14.12: Step response of the closed-loop system with re-tuned PIDF controller and second-order plant in Example 14.4.

14.4.2 Integrator Windup

When implementing a PID controller, we need to take into account physical constraints of the controlled plant. In particular, a control engineer has to remember that all actuators have physical limitations; a motor has a finite limited speed while a valve can only be opened to its limits. Thus a control engineer is faced with the problem of actuator saturation. To illustrate the issue, consider Example 14.4. Suppose that the actuator saturation limits are ±10, that is,

|u(t)| ≤ 10.

These constraints are violated in the above example because of the set-point kick phenomenon. Let us now impose the constraints on the control action and see how this affects the closed-loop system behavior.

Example 14.5 First, we consider the second-order plant (14.11) simulated in the previous example. We impose constraints on the actuator, |u(t)| ≤ 10. The PIDF controller parameters are the same as the ones we used in the simulation shown in Figure 14.12. The output of the closed-loop system is shown in Figure 14.13.

704 CHAPTER 14. PROPORTIONAL-INTEGRAL-DERIVATIVE (PID) CONTROL

Figure 14.13: A PIDF-controlled second-order plant with with constrained actuator. Figure 14.14: Plant output of the PIDF-controlled first-order plant with with transportation delay and constraints on input.

Figure 14.15: A type A PID controller with the integrator anti-windup.

As can be seen in Figure 14.13 and 14.14, in both cases the control signal saturates in the moment the reference input is applied. The input to the plant remains saturated in the sense that the feedback loop is broken because the controller’s output is not responding to its input. The output of the integral term continues to increase because the tracking error is positive. The output of the integral term starts decreasing when the plant’s output becomes larger than the reference input. However, the controller stays saturated for some time because of the excessively large output of the integral term. As a result of the integral term excessive output, the plant’s output has a large overshoot. This phenomenon of excessive integrator term output resulting in the large overshoot of the plant’s output is called the integrator windup.

We can avoid or reduce the integrator windup by reducing the integral term output when the actuator saturates. A possible anti-windup scheme is shown in Figure 14.15. This anti-windup mechanism is referred to as tracking or back calculation. In Figure 14.16, we show the output of the PIDF-controlled second-order plant with the anti-wind, where T_{A} = T_{I}/0.075. In Figure 14.17, we show the output of the first-order plant with transportation delay controlled with a PIDF controller with the integrator windup. The PIDF controller parameters are the same as in the simulation of Figure 14.11, and T_{A} = 0.21.

Figure 14.16: Plant output of the PIDF-controlled second-order plant with the integrator ani-windup circuit.

14.5 Modified PID Control Schemes

We now discuss two alternative schemes to eliminate the set-point kick phenomenon. We first take another look at the type A PID control architecture as shown in Figure 14.18. In order to avoid differentiating a step reference input, it is proposed to operate the derivative action in the feedback as shown in Figure 14.19; hence its name, the PI-D controller also referred to as the type B PID controller. A practical implementation of this controller that we refer to as the type B PIDF controller action can be described as

U(s) = K_{P} (1 + \frac{1}{T_{I}s}) E(s) − \frac{K_{P} T_{D}s}{γT_{D}s + 1}Y (s).

We can go even further and move the proportional and derivative actions into feedback giving us the I-PD control or the type C PID controller shown in Figure 14.20. A practical implementation of this controller that we refer to as the type C PIDF controller action can be described as

U(s) = \frac{K_{P}}{T_{I}s}E(s) − K_{P} (1 + \frac{T_{D}s}{γT_{D}s + 1}) Y (s).

14.5. MODIFIED PID CONTROL SCHEMES 707 Figure 14.17: Plant output of the PIDF-controlled first-order plant with transportation delay and the integrator ani-windup.

Figure 14.18: More detailed view of the type A PID controller. Figure 14.19: Type B PID controller.

Figure 14.20: Type C PID controller.

14.6 Digital Implementation

In this section we present a method of implementing PID controllers using a computer. To obtain a computer implementation, we need to discretize the continuous-time differential equation describing a controller at hand.

We discuss discretization of different terms of the type A PIDF controller. The types B and C PIDF controllers can be easily obtained by modifying the A PIDF architecture. In our discussion, we assume that the sampling instants are equally spaced. We denote the sampling period as h, and sampling instants as t_{k}, where k = 0, 1, 2, ….

• The discrete-time version of the proportional term, u_{P} (t_{k}), has the form

u_{P} (t_{k}) = K_{P} e(t_{k}),  (14.18)

where K_{P} is the proportional gain.

• Recall that the continuous-time integral term, u_{I}(t), is given by

u_{I}(t) = \frac{K_{P}}{T_{I}} ∫_{0}^{t} e(τ)dτ,

Differentiating both sides of the above integral equation yields

\frac{du_{I}(t)}{dt} = \frac{K_{P}}{T_{I}} e(t).

We can use several methods to discretize the above differential equation. Using the forward difference discretization gives

u_{I}(t_{k+1}) = u_{I}(t_{k}) + \frac{K_{P} h}{T_{I}}e(t_{k}).  (14.19)

For other discretization methods, the reader may wish to consult Section 3.7 on numerical techniques.

• Performing cross-multiplication in (14.16) and representing the resulting polynomial equation, in the Laplace variable s, into the time domain gives the continuous-time differential equation describing the filtered derivative term,

γT_{D} \frac{du_{D}(t)}{dt} + u_{D}(t) = K_{P} T_{D} \frac{de(t)}{dt}.  (14.20)

The forward difference discretization of the above differential equation yields

u_{D}(t_{k+1}) = (1 − \frac{h}{γT_{D}} ) u_{D}(t_{k}) + \frac{K_{P}}{γ} (e(t_{k+1}) − e(t_{k})) .  (14.21)

On the other hand, the backward difference discretization of the differential equation (14.20) gives,

u_{D}(t_{k}) = \frac{γT_{D}}{γT_{D} + h}u_{D}(t_{k−1}) + \frac{T_{D}K_{P}}{γT_{D} + h} (e(t_{k}) − e(t_{k−1})) .  (14.22)

Combining (14.18), (14.19), and (14.22), we obtain a discrete-time approximation of the type A PIDF controller,

\begin{matrix}u_{P} (t_{k}) & = & K_{P} e(t_{k}) \\ u_{D}(t_{k}) & = & \frac{γT_{D}}{γT_{D}+h}u_{D}(t_{k−1}) + \frac{T_{D}K_{P}}{γT_{D}+h} (e(t_{k}) − e(t_{k−1})) \\ u(t_{k}) & = & u_{P} (t_{k}) + u_{I}(t_{k}) + u_{D}(t_{k}) \\ u_{I}(t_{k+1}) & = & u_{I}(t_{k}) + \frac{K_{P} h}{T_{I}}e(t_{k}) \end{matrix} \}  (14.23)

To implement the above controller, one can use the zero-order hold—see Exercises 14.5, 14.6, and 14.7.

14.7 Notes

The first PID controllers for industrial applications were introduced in 1939 by the Taylor Instrument Company and the Foxboro Instrument Company. For an account of the PID controller development along with an overview of early process control devises, we recommend Bennett [30]. For an account of recent advances in PID control, see an article by Li, Ang, and Chong [211]. See also Hägglund and Åström for a discussion of automatic tuning of PID controllers [126].

Many engineers are familiar with PID controllers and prefer to enhance the performance of a well-known solution through additional capabilities rather than switch to an untested solution. Knospe [183] observers that even alternative digital control methods have not supplanted PID’s primacy in industrial control with a possible exception of model predictive control, where the control action is calculated on the fly. Knospe [183] adds that PID control still has a role even in this case by handling lower level loops.

Welander states in [348] that about 90% of loops that employ PID control are probably PI control. This is because derivative action is the least understood. “Derivative acts as a brake or dampener on the control effort. The more the controller tries to change the value, the more it counteracts the effort.” Welander concludes his analysis of the D part in PID architecture as follows: “Ultimately, the use of derivative action can be beneficial, but it has to be used in the right situations and in the right amounts. Your starting assumption should be to see if the process will run satisfactorily on PI control alone.” He continues to say that, adding derivative should be done cautiously and with appropriate filtration. “Like cooking with hot sauce, in the right context and in skilled hands, the result can be remarkable. But if it’s used inappropriately, it’s a recipe for disaster” [348].

14.8 Exercises

Exercise 14.1 Propose a simple controller for the system in Example 14.1 that yields e(∞) = 0 when the reference signal is

r(t) = \frac{1}{2}t^{2}1(t),

where 1(t) is the unit step function. Recall that

R(s) = L(r(t)) = \frac{1}{s^{3}} .

Exercise 14.2 (Based on Ogata [257, p. 692]) For the PID-controlled system shown in Figure 14.21, write a MATLAB program that finds a combination of the parameters K_{c} and a such that the maximum overshoot of the system output y(t) is in the range (5%, 10%). Assume the following search region for K_{c} and a,

5 ≤ K_{c} ≤ 20 and 5 ≤ a ≤ 10.

Plot the unit-step response for the first found set of the parameters that satisfy the overshoot specifications.

Exercise 14.3 Compare the performance of the type B PID controller on the plants of Example 14.3 with constraints on the plant input of the form, |u(t)| ≤ 10. Plot u(t) versus time.

[figure: R(s) = \frac{1}{s}; +; −; E(s); K_{c} \frac{(s+a)^{2}}{s}; PID Controller; \frac{10}{s^{2}+6s+5}; Y (s); y(t)]

Figure 14.21: A PID controlled system of Exercise 14.2.

Exercise 14.4 Compare the performance of the type C PID controller on the plants of Example 14.3 with constraints on the plant input of the form, |u(t)| ≤ 10. Plot u(t) versus time.

Exercise 14.5 Compare the performance of the digital PI controller, u(t_{k}) = u_{P} (t_{k})+u_{I}(t_{k}) given by (14.23), on the plants of Example 14.3 with constraints on the plant input of the form, |u(t)| ≤ 10. Use the zero-order hold. Analyze the controller performance with and without the integrator anti-windup. Plot u(t) versus time.

Exercise 14.6 Compare the performance of the digital type B PIDF controller on the plants of Example 14.3 with constraints on the plant input of the form, |u(t)| ≤ 10. Equations describing this controller can be obtained by modifying the derivative term of the architecture (14.23). Use the zero-order hold in the controller implementation. Plot u(t) versus time.

Exercise 14.7 Compare the performance of the digital type C PIDF controller on the plants of Example 14.3 with constraints on the plant input of the form, |u(t)| ≤ 10. Equations describing this controller can be obtained by modifying the proportional and derivative terms of the architecture (14.23). Use the zero-order hold in the controller implementation. Plot u(t) versus time.

Exercise 14.8 For the one-link manipulator from Exercise 5.22, use the linearized model from Part (b) of that exercise to design (a) digital, type A PIDF controller. Implement and perform animation of the closed-loop

system comprised of the PID controller driving the nonlinear model of the robot;

(b) assume that the actuator saturation limits are ±10. Design, implement, and perform animation of the digital, type A PIDF controller with the integrator anti-windup on the nonlinear model of the robot.

Exercise 14.9 For the nonlinear model of a double-link robot manipulator in Subsection 2.4.6, use the linearized model of this robot obtained in Exercise 4.5 to design

(a) digital, type A PIDF controller. Implement and perform animation of the closed-loop system comprised of the PID controller driving the nonlinear model of the robot;

(b) assume that the actuator saturation limits are ±10. Design, implement, and perform animation of the digital, type A PIDF controller with the integrator anti-windup on the nonlinear model of the robot.

Fuzzy Systems

It is in the response to mistakes, of technique or of judgment, that one sees most clearly the human side of science’s practitioners: a mix of pride, stubbornness, humility, honesty, dismay, and regret that one could expect to encounter in any section of the population. There are the inevitable diehards, whose pride forces them to concoct ever-more-elaborate explanations for the facts in order to avoid confession of error. There are the crusaders, determined at all costs to expose the foolishness. And there are the “told-you-so’s,” generally members of the scientific elite, who make little contribution while the debate is raging but are ready to heap scorn on the perpetrators afterward for their “obvious” mistakes. All this, of course, is the antithesis of objective science, but follows as inevitably as night follows day from the fact that science is practiced by people and not by automatons. [20, page 291]

15.1 Motivation and Basic Definitions

In Chapter 14, we present a controller design approach when a mathematical model of the process to be controlled is not available to the control engineer. In this Chapter, we discuss an alternative approach to the controller design when a mathematical model of the controlled system is not available to us. Fuzzy logic control offers a way of dealing with modeling problems by implementing linguistic, non-formally expressed control laws derived from expert knowledge. This approach is especially successful in the case of processes such as washing machines, camera focusing, or drying processes that are being controlled well by

Figure 15.1: An example of a crisp set—fast cars.

human operators. The control of such processes can be captured in the form of rules

IF (process state) THEN (control action).

Thus, by a rule we will understand an IF-THEN statement that defines the set of facts that must be true (the IF part) before a set of actions (the THEN part) can be executed. The basic notion of fuzzy logic is fuzzy set. An ordinary set, or simply a set, is a collection of objects. Ordinary sets have two membership states: inclusion or exclusion. An object is either in the set or it is not. Thus the degree to which an object is a member of a set is either 0 or 1. For example, a car is fast if it is driven faster than 65 mph, and slow if it is driven slower than 65 mph—see Figure 15.1. This is the reason we sometimes refer to ordinary sets as crisp sets. Fuzzy sets, on the other hand, allow for partial membership in a set. The degree to which an object is a member of a fuzzy set can vary between 0 and 1. With a fuzzy set, we have a gradual transition from membership to nonmembership. For example, a car moving with velocity of 65 mph is a member of the fuzzy set FAST to degree 0.5. A car being driven 57.5 mph is FAST to degree 0.25—see Figure 15.2.

We next illustrate the crisp and fuzzy versions of the sets SLOW and FAST. Observe the sudden change of membership in the crisp SLOW and FAST sets at a speed of 65 mph in Figure 15.3. The fuzzy SLOW and FAST sets allow for a gradual change from SLOW to FAST for a car moving between 50 and 80 mph as it is illustrated in Figure 15.4. Notice that a car moving at a speed between 50 and 80 mph has partial membership in SLOW and 15.1. MOTIVATION AND BASIC DEFINITIONS 715

Figure 15.2: A fuzzy set FAST. Figure 15.3: Crisp sets for SLOW and FAST.

Figure 15.4: Fuzzy sets for SLOW and FAST.

partial membership in FAST at the same time. In the above example, the variable SPEED can have two possible linguistic values: SLOW and FAST. For this reason, we refer to the variable SPEED as the linguistic variable because its possible values are the linguistic values: in this example they are SLOW and FAST.

With the above examples in mind, we formally define a fuzzy set. For this, we need to introduce some notation. Let U be a set. We use u ∈ U to denote that u is an element of U. We call U the universe of discourse, or just the universe. We first recall the definition of a function. A function A from a set X to a set Y is a set of ordered pairs in X × Y with the property that if (x, y) and (x, z) are elements of A, then y = z.

Definition 15.1 (Fuzzy set) A fuzzy set A on U is a function that maps U into the interval [0, 1]. Equivalently,

A = {(u, μ_{A}(u)) : u ∈ U},

where μ_{A} is commonly called the membership function of the fuzzy set A.

◇

Note that the concept of a fuzzy set generalizes that of a crisp set in the following way. Let V ⊂ U. Define μ_{V} : U → [0, 1] by μ_{V} (u) = 1 if u ∈ V and μ_{V} (u) = 0 otherwise. Then V can be identified with the fuzzy set associated with μ_{V}. In other words, the subsets of U are identified with the functions from U to the two point set {0, 1}.

Example 15.1 Consider the following universe U = {4, 5, 6, 7, 8}. Let

A = {(4, 0.0), (5, 0.5), (6, 1.0), (7, 0.5), (8, 0.0)}.

Then A is a fuzzy set. One interpretation of A is closeness to 6.

Let A and B be fuzzy sets on U. We define the union, intersection, and complement using the following membership functions:

union: μ_{A∪B}(u) = max{μ_{A}(u) , μ_{B}(u)} for all u ∈ U;

intersection: μ_{A∩B}(u) = min{μ_{A}(u) , μ_{B}(u)} for all u ∈ U;
complement: μĀ(u) = 1 − μ_{A}(u) for all u ∈ U.

Example 15.2 Consider another fuzzy set B, “number close to 4”, given by

B = {(4, 1.0), (5, 0.8), (6, 0.5), (7, 0.2), (8, 0.0)}.

Then,

A ∪ B = {(4, 1.0), (5, 0.8), (6, 1.0), (7, 0.5), (8, 0.0)}.

The intersection of the two fuzzy sets A and B is

A ∩ B = {(4, 0.0), (5, 0.5), (6, 0.5), (7, 0.2), (8, 0.0)}.

The complement of the fuzzy set A is

Ā = {(4, 1.0), (5, 0.5), (6, 0.0), (7, 0.5), (8, 1.0)}.

Definition 15.2 (Fuzzy number) A fuzzy number is a fuzzy set F on a universe U ⊂ ℝ such that

max_{u∈U} μ_{F} (u) = 1,

and

μ_{F} (λu_{1} + (1 − λ)u_{2}) ≥ min(μ_{F} (u_{1}), μ_{F} (u_{2})),

for all u_{1}, u_{2} ∈ U and all λ ∈ [0, 1].

◇

A fuzzy set that satisfies the conditions max_{u∈U} μ_{F} (u) = 1 and μ_{F} (λu_{1} + (1 − λ)u_{2}) ≥ min(μ_{F} (u_{1}), μ_{F} (u_{2})) is called normal and convex, respectively. Thus, a fuzzy number is a fuzzy set that is both normal and convex. Examples of convex and non-convex fuzzy sets are given in Figure 15.5.

Figure 15.5: The first fuzzy set is a fuzzy number, while the second one is not.

15.2 Standard Additive Model

Let X and Y be nonempty sets and let I and J be nonempty index sets. Suppose we are given collections of fuzzy sets {A_{α} : α ∈ I} and {B_{β} : β ∈ J} on X and Y, respectively, and rules of the form

IF A_{α} THEN B_{β}.

The IF-THEN rules along with the collections of fuzzy sets form a function R from {A_{α} : α ∈ I} to {B_{β} : β ∈ J}, which we call the fuzzy system associated with the rules.

The support of a function into ℝ is the subset of the domain where the function is nonzero. The support of the fuzzy sets A_{α} and B_{β} and the IF-THEN rules describe subsets, or patches, in the Cartesian product space X × Y, as illustrated in Figure 15.6. Suppose the support of each A_{α} and each B_{β} is a one point set, say x_{α} ∈ X and y_{β} ∈ Y, respectively. If X = ⋃{x_{α} : α ∈ I}, then R can be identified with a function R̃ from X to Y by

R̃(x_{α}) = y_{α}

if

R(A_{α}) = B_{α}.

One can think of this case as having complete knowledge and the patches are just points.

Figure 15.6: Function approximation using IF-THEN rules.

A fuzzy system can be thought of as a “fuzzified” function from X to Y with fuzzy sets replacing points in the domain. The size of a patch is usually inversely proportional to the knowledge or confidence of the situation. A point x ∈ X is usually in the support of more than one A_{α} with confidence represented by the values of the membership functions μ_{α}. The fuzzy set R(A_{α}) represents the desired action corresponding to A_{α}. This is illustrated in Figure 15.7. Suppose we are given the following two rules:

IF A_{3} THEN B_{3},
IF A_{4} THEN B_{2}.

The point x_{0} is contained in the support of A_{3} and A_{4}. Thus, according to the given rules, both B_{3} and B_{2} need to be taken into account. The process by which we combine these actions is called defuzzification. The process of defuzzification is also known as fuzzy reasoning in the literature. The outcome of defuzzification is a numerical value y ∈ Y. An example of a defuzzification algorithm is a centroidal defuzzifier of the form

y = \frac{∑_{α} μ_{A_{α}}(x)ξ(x, A_{α}, R(A_{α}))}{∑_{α} μ_{A_{α}}(x)},  (15.1)

where ξ(x, A_{α}, R(A_{α})) is a function into the real numbers. We now give an example of ξ.

Let ξ(x, A_{α}, R(A_{α})) be the centroid of the support of R(A_{α}). For example, the centroids of the support of R(A_{3}) = B_{3} is c_{3} and that of R(A_{4}) = B_{2} is c_{2}. A fuzzy system with the centroidal defuzzifier is called the standard additive model (SAM). The term standard additive model was coined by Kosko [187]. A schematic representation of a SAM is shown

Figure 15.7: Generating output fuzzy sets from THEN-part sets of IF-THEN rules.

in Figure 15.8. In the above description of the SAM, we were concerned with a single-input single-output case. The above construction can be extended to a many-inputs case. Suppose we are given IF-THEN rules of the form

IF A_{i} AND B_{i} THEN C_{i},

The SAM corresponding to this case is illustrated in Figure 15.9. We could use a defuzzification algorithm of the form

z = \frac{∑_{l=1}^{M} μ_{A_{l}}(x)μ_{B_{l}}(y)c_{l}}{∑_{l=1}^{M} μ_{A_{l}}(x)μ_{B_{l}}(y)} ,  (15.2)

where M is the number of IF-THEN rules, and c_{l} is the centroid of the fuzzy set C_{l}, l = 1, 2, …, M. This type of defuzzifier is called the center average defuzzifier or the algebraic product defuzzifier.

Another type of defuzzifier that we could use is

z = \frac{∑_{l=1}^{M} min \{μ_{A_{l}}(x), μ_{B_{l}}(y)\} c_{l}}{∑_{l=1}^{M} min \{μ_{A_{l}}(x), μ_{B_{l}}(y)\}} .  (15.3)

The above type of defuzzifier is called the center-of-gravity defuzzifier.

An illustration of fuzzy reasoning using defuzzification scheme (15.3) is given in Figure 15.10. In general, for q inputs, the center average defuzzifier takes the form

z = \frac{∑_{l=1}^{M} ∏_{j=1}^{q} μ_{A_{j}^{l}}(x_{j})c_{l}}{∑_{l=1}^{M} ∏_{j=1}^{q} μ_{A_{j}^{l}}(x_{j})} .  (15.4)

15.2. STANDARD ADDITIVE MODEL 721 Figure 15.8: Single-input single-output standard additive model schematic representation.

Figure 15.9: Two-input single-output standard additive model representation.

Figure 15.10: An illustration of fuzzy reasoning in a two-input fuzzy system.

15.3 Fuzzy Logic Control

A block diagram of a closed-loop system with a fuzzy logic controller (FLC) is shown in Figure 15.11. The controller uses the control error e and the rate of change in the control error, ė or Δe, as its inputs. A typical fuzzy logic control architecture has the structure of a SAM. Designing a fuzzy logic controller can be summarized in the following six-step algorithm:

• Identify ranges of the controller inputs;

• Identify ranges of the controller outputs;

• Create fuzzy sets for each input and output;

• Translate the interaction of the inputs and outputs into IF-THEN rules, and then form the rules matrix;

• Decide on the defuzzifier, then apply it to crisp inputs to produce crisp outputs;

• Implement the controller, test it, and modify it if necessary.

As an example, suppose we use five membership functions for the controller’s inputs and its output. We then generate the rules. A possible set of rules is shown in Table 15.1, where LN denotes large negative, N means negative, ZE is zero, P is positive, and LP refers to large 15.3. FUZZY LOGIC CONTROL 723

Figure 15.11: A diagram of a closed-loop system with a fuzzy logic controller.

Table 15.1: The rules matrix.

change-in-error LN N ZE P LP

error

LN  LN LN LN N ZE
N  LN LN N ZE P
ZE  N N ZE P P
P  N ZE P LP LP
LP  ZE P LP LP LP

positive. The next step involves choosing a defuzzifier. We use the modified center average defuzzifier to obtain

u = \frac{∑_{k=1}^{25} m_{k}^{o}p_{k}/s_{k}^{o}}{∑_{k=1}^{25} p_{k}/s_{k}^{o}} ,

where for k = 1, 2, …, 25, following the rules shown in Table 15.1, we compute

p_{k} = μ_{i_{k}}^{e}μ_{j_{k}}^{ce}, i, j = 1, 2, …, 5.

The functions μ_{i}^{e}, μ_{j}^{ec} are the membership functions corresponding to the linguistic variables ERROR and CHANGE-IN-ERROR, respectively. The parameters m_{k}^{o} and s_{k}^{o} are the m and s parameters of the controller output membership functions—see figures below for their definition. The modified center average defuzzifier takes into account the shape of the membership functions, and in particular their “spread”. As it is pointed out by Wang [344], the sharper the shape of μ, the stronger is our belief that the value should be closer to its m. This observation is reflected in the formula for the defuzzifier we use, where we divide the weights p_{k} by s_{k}^{o}—the spread of the corresponding controller output membership function. We can use triangular membership functions whose equations are given next. The expression for the function shown in Figure 15.12 is

μ(x, m, s) = max (0, 1 − \frac{|x − m|}{s}) .

The expression for the function shown in Figure 15.13 is

μ(x, m, s) = min (1, max (0, 1 + \frac{x − m}{s})) .

Finally, the expression for the function shown in Figure 15.14 is

μ(x, m, s) = min (1, max (0, 1 − \frac{x − m}{s})) .

Example 15.3 Consider the vehicle model given in Subsection 2.4.2. Let the input signal be u = δ_{f}—the front wheel steer angle, and δ_{r} = 0. We design a simple fuzzy logic controller for the above vehicle model. The range for the error is [−0.5, 0.5]. The range for the change-in-error is [−0.06, 0.06]. The range for the controller output is [−3 deg, 3 deg]. The parameters of the membership functions are given in Table 15.2.

Plots of the error, the change-in-error, and controller’s output membership functions are given in Figures 15.15, 15.16, and 15.17, respectively. We created a nonlinear mapping from 15.3. FUZZY LOGIC CONTROL 725

Figure 15.12: A triangular shaped fuzzy set. Figure 15.13: A fuzzy set at the right end of a universe of discourse.

726 CHAPTER 15. FUZZY SYSTEMS Figure 15.14: A fuzzy set at the left end of a universe of discourse. Table 15.2: The m and s-parameters of the membership functions.

error  change-in-error  output
m  s  m  s  m  s
m_{1} = −0.4 s_{1} = 0.1 m_{1} = −0.05 s_{1} = 0.025 m_{1} = −2.5 s_{1} = 1
m_{2} = −0.2 s_{2} = 0.2 m_{2} = −0.025 s_{2} = 0.025 m_{2} = −1 s_{2} = 1
m_{3} = 0 s_{3} = 0.2 m_{3} = 0 s_{3} = 0.025 m_{3} = 0 s_{3} = 1
m_{4} = 0.2 s_{4} = 0.2 m_{4} = 0.025 s_{4} = 0.025 m_{4} = 1 s_{4} = 1
m_{5} = 0.4 s_{5} = 0.1 m_{5} = 0.05 s_{5} = 0.025 m_{5} = 2.5 s_{5} = 1

15.3. FUZZY LOGIC CONTROL 727

Figure 15.15: Fuzzy numbers of the linguistic variable ERROR in Example 15.3. Figure 15.16: Fuzzy numbers of the linguistic variable CHANGE-IN-ERROR in Example 15.3.

Figure 15.17: Fuzzy numbers of the linguistic variable CONTROL in Example 15.3.

the controller’s input space into its output space and formed a look-up table that implements the obtained nonlinear mapping representing fuzzy logic controller. We used SIMULINK’s 2-D Look-Up Table block to implement the controller. The look-up table size was 22 × 22. The output of the look-up table was scaled by the factor −2π/180 before using it as the vehicle’s steering input. The negative sign is to take into account the opposite directions of the lateral axes of the vehicle coordinate system and the fixed system. The factor π/180 converts degrees into radians. We multiplied the output of the look-up table by 2 to increase the control authority. This allows us to use the same controller to track step inputs with large amplitudes. A plot of the control surface is given in Figure 15.18. Next, a reference signal generator was constructed. We selected the following transfer function of the signal generator:

\frac{R(s)}{H(s)} = \frac{1}{s^{3} + 1.75s^{2} + 2.15s + 1},

where R(s) = L(r(t)), H(s) = L(61(t)), and 1(t) is the unit step function. The closed-loop system was simulated using SIMULINK. Plots of Y and r versus X are shown in Figure 15.19.

Example 15.4 This example is based on Example 6.1 found in Driankov, Hellendoorn, and Reinfrank [86, pages 266–272]. We consider a nonlinear dynamical system modeled by the 15.3. FUZZY LOGIC CONTROL 729

Figure 15.18: A plot of the control surface u = f(ė, e) in Example 15.3. Figure 15.19: Plots of Y and r versus X in Example 15.3.

Figure 15.20: A phase-plane portrait of the uncontrolled system in Example 15.4.

differential equation,

ẍ + a_{1}(1 − x^{2})ẋ + a_{2}x = bu,  (15.5)

where |u| ≤ 15, a_{1} = 2.2165, a_{2} = b = 12.7388. Let x_{1} = x and x_{2} = ẋ_{1}. Then, the above system model in state space format has the form

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}x_{1} \\ −a_{2}x_{1} − a_{1} (1 − x_{1}^{2}) x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ b \end{matrix} ] u.

A phase-plane portrait of the uncontrolled system is shown in Figure 15.20. Plots of x_{1} and x_{2} versus time for the initial conditions x_{1}(0) = −1 and x_{2}(0) = 0 are shown in Figure 15.21. The relevant membership functions are shown in Figure 15.22, 15.23, 15.24, and 15.25. The following MATLAB function can be used to generate all the relevant membership functions.

function y = trap_a(x,a1,a2,c,s)
%TRAP_A is an asymmetric trapezoid type membership function,
% also called a ``pi'' function,
% where x is the function domain, a1 is the length of the
% left trapezoid's side
% at the base, a2 is the length of the right side at the base,
% c is the trapezoid's center, and s is a half of the
% trapezoid upper side. Thus the length of the trapezoid upper

15.3. FUZZY LOGIC CONTROL 731 Figure 15.21: Plots of x_{1} and x_{2} versus time of the uncontrolled system of Example 15.4.

Figure 15.22: Membership functions of x_{1} of Example 15.4.

732 CHAPTER 15. FUZZY SYSTEMS Figure 15.23: Membership functions of x_{2} of Example 15.4. Figure 15.24: Membership functions of u of Example 15.4.

Figure 15.25: Magnification of the membership functions of u on the interval [−1.5, 1.5] of Example 15.4.

% side is 2*s. The length of the trapezoid base is then 2*s+a1+a2.
% Set s=0 to get a triangular membership function.
y1=min(0.5,(max((-0.5),(0.5 +((x-c+s)./a1)))));
y2=min(0.5,(max((-0.5),(0.5 -((x-c-s)./a2)))));
y=y1+y2;

We used the rules matrix given in Table 15.3 to produce a nonlinear mapping u = Φ(x_{1}, x_{2}) and to construct a look-up table that implements the obtained mapping representing a fuzzy logic controller. The 2-D Look-Up Table block to implement the controller was used. The look-up table size was 22 × 22. A plot of u = Φ(x_{1}, x_{2}) is shown in Figure 15.26. A SIMULINK diagram of the closed-loop system is given in Figure 15.27. A phase-portrait of the closed-loop system is depicted in Figure 15.28. Plots x_{1} and x_{2} versus time for x_{1}(0) = −1 and x_{2}(0) = 0 for the closed-loop system are shown in Figure 15.29.

Figure 15.26: A plot of u = Φ(x_{1}, x_{2}) of Example 15.4.

Table 15.3: The rules matrix of Example 15.4.

x_{1} LN  N ZE P  LP

x_{2}

LN PVB PM PM PM PVB
N PB  P PS PS PM
ZE P PVS ZE NVS N
P NM NS NS N  NB
LP NVB NM NM NM NVB

15.3. FUZZY LOGIC CONTROL 735 Figure 15.27: A SIMULINK diagram of the closed-loop system of Example 15.4. Figure 15.28: A phase-plane portrait of the closed-loop system of Example 15.4.

Figure 15.29: Plots x_{1} and x_{2} versus time for x_{1}(0) = −1 and x_{2}(0) = 0 for the closed-loop system in Example 15.4.

15.4 Stabilization Using Fuzzy Models

In this section, a systematic Lyapunov based stabilizing control design method for uncertain nonlinear dynamical systems using fuzzy models is presented. The controller is constructed using a design model of the dynamical process to be controlled. Recall that the dynamical process to be controlled is also called the plant. We obtain the design model from the truth model using a fuzzy modeling approach.

15.4.1 Fuzzy Modeling

In many situations there may be human experts that can provide a linguistic description, in terms of IF-THEN rules, of the plant dynamical behavior. Combining available mathematical description of the plant with its linguistic description results in a fuzzy system model. Such an approach to modeling of continuous-time dynamical processes was proposed by Takagi and Sugeno [317] and further developed by Sugeno and Kang [312]. This type of model is called the Takagi-Sugeno (T-S) or Takagi-Sugeno-Kang (TSK) fuzzy model. We mention here that the discrete-time version of the TSK model was analyzed by Tanaka and Sugeno [322], and Tanaka [321], while Wang, Tanaka, and Griffin [341], Cao, Rees, and Feng [51], and Feng et al. [101] used TSK fuzzy models to design stabilizing controllers for nonlinear systems.

One can construct a TSK model if local description of the plant is available in terms of local linear models,

ẋ(t) = A_{i}x(t) + B_{i}u(t), i = 1, 2, …, r,

where the state vector x(t) ∈ ℝ^{n}, the control input u(t) ∈ ℝ^{m}, and the matrices A_{i} and B_{i} are of appropriate dimensions. The above information is then fused with the available IF-THEN rules, where the i-th rule can have the form:

Rule i : IF x_{1}(t) is F_{1}^{i} AND ⋯ AND x_{n}(t) is F_{n}^{i},
THEN ẋ(t) = A_{i}x(t) + B_{i}u(t),

where F_{j}^{i}, j = 1, 2, …, n, is the j-th fuzzy set of the i-th rule. Let μ_{j}^{i}(x_{j}) be the membership function of the fuzzy set F_{j}^{i}, and let

w^{i} = w^{i}(x) = ∏_{j=1}^{n} μ_{j}^{i}(x_{j}).

Then, the resulting fuzzy system model is constructed as the weighted average of the local models and has the form,

ẋ = \frac{∑_{i=1}^{r} w^{i} (A_{i}x + B_{i}u)}{∑_{i=1}^{r} w^{i}}
= ∑_{i=1}^{r} α_{i} (A_{i}x + B_{i}u)
= (∑_{i=1}^{r} α_{i}A_{i}) x + (∑_{i=1}^{r} α_{i}B_{i}) u
= A(α)x + B(α)u,  (15.6)

where for i = 1, 2, …, r,

α_{i} = \frac{w^{i}}{∑_{i=1}^{r} w^{i}} .  (15.7)

Note that, for i = 1, 2, …, r,

α_{i} ≥ 0, ∑_{i=1}^{r} α_{i} = 1 and α = [ α_{1} α_{2} ⋯ α_{r} ]^{⊤} ∈ [0, 1]^{r}.  (15.8)

A system model given by (15.6) and (15.8) is also referred to as in the literature as the polytopic system—see, for example, Takaba [316].

15.4.2 Constructing a Fuzzy Design Model Using a Nonlinear Model

In the stabilization problem of a nonlinear plant, we are concerned with constructing a controller so that starting from an arbitrary point, in some neighborhood of the operating point, the controller forces the closed-loop system trajectory to converge to this operating point. On the other hand, if the starting point coincides with the operating point, the closed-loop system trajectory is expected to stay at this point for all subsequent time. When designing a controller, we use a TSK fuzzy design model obtained by combining a mathematical description of the plant in terms of its truth model and a linguistic description of the plant. The essential components of the TSK model are linear local models, which describe the plant dynamical behavior at its different operating points. We show that using common linearization approach to construct local models may result in a fuzzy design model different from what the designer had in his or her mind. For example, suppose that the truth model of a plant has the form

ẋ = f(x) + G(x)u.  (15.9)

For simplicity of notation, let

F (x, u) = f(x) + G(x)u.

Then, we can represent (15.9) as

ẋ = F (x, u).  (15.10)

Expanding F into a Taylor series around (x_{0}, u_{0}) yields

ẋ = F (x_{0}, u_{0}) + \frac{∂F}{∂x} |_{{x=x_{0} \atop u=u_{0}}} (x − x_{0}) + \frac{∂F}{∂u} |_{{x=x_{0} \atop u=u_{0}}} (u − u_{0}) + higher order terms, (15.11)

where

F (x, u) = f(x) + G(x)u = f(x) + ∑_{k=1}^{m} u_{k}g_{k}(x),  (15.12)

where g_{k} is the k-th column of G. To write expressions for the second and the third terms of (15.11) as functions of f and G, let g_{ij} be the (i, j)-th element of the matrix G. Then,

\frac{∂F}{∂x} |_{{x=x_{0} \atop u=u_{0}}} = \frac{∂f}{∂x}|_{x=x_{0}} + ∑_{k=1}^{m} u_{k} \frac{∂g_{k}}{∂x} |_{{x=x_{0} \atop u=u_{0}}} = \frac{∂f}{∂x}|_{x=x_{0}} + H(x_{0}, u_{0}),  (15.13)

where the (i, j)-th element of the n × n matrix H has the form

∑_{k=1}^{m} u_{k} \frac{∂g_{ik}(x)}{∂x_{j}} |_{{x=x_{0} \atop u=u_{0}}} .

Finally,

\frac{∂F}{∂u} |_{{x=x_{0} \atop u=u_{0}}} = G(x_{0}).  (15.14)

A point (x_{0}^{⊤}, u_{0}^{⊤})^{⊤} ∈ ℝ^{n+m} is an equilibrium point of (15.10) if F (x_{0}, u_{0}) = 0, that is, if at (x_{0}, u_{0}) we have ẋ = 0. Let Δx = x − x_{0} and Δu = u − u_{0}. Note that

\frac{dx_{0}}{dt} = 0.

Then, the linearized model about the equilibrium (x_{0}, u_{0}) is obtained by neglecting higher order terms and observing that at the equilibrium point F (x_{0}, u_{0}) = 0. The linearized model has the form

\frac{d}{dt}Δx = AΔx + BΔu,  (15.15)

where

A = \frac{∂F}{∂x} |_{{x=x_{0} \atop u=u_{0}}} and B = \frac{∂F}{∂u} |_{{x=x_{0} \atop u=u_{0}}} .

Suppose now that we wish to construct a stabilizing controller for the truth model ẋ = f(x)+G(x)u, where f(0) = 0, so that the pair (x = 0, u = 0) constitutes an asymptotically stable equilibrium point of the closed-loop system. Our first step is to construct a fuzzy design model. We start with generating linear local models describing plant’s behavior at selected operating points. It is natural to first construct a local linear model that describes plant’s behavior about the equilibrium state x = 0. We can obtain this first local linear model using the above described linearization technique. The resulting model is given by

ẋ = A_{1}x + B_{1}u.

Next, we construct local linear models describing plant’s behavior at the remaining operating points. Suppose that x = x_{j} is the next operating state of interest. The result of Taylor’s linearization of a nonlinear model about an operating non-equilibrium point is, in general, an affine, rather than a linear model. Even if the operating point is an equilibrium point, Taylor series linearization, in general, will not yield, in general, a local model that is linear. Indeed, suppose that the operating point (x_{j}, u_{j}) is an equilibrium point, that is,

f(x_{j}) + G(x_{j})u_{j} = 0.  (15.16)

By (15.11), the resulting linearized model is

\frac{d}{dt}(x − x_{j}) = f(x_{j}) + G(x_{j})u_{j} + A_{j}(x − x_{j}) + B_{j}(u − u_{j})  (15.17)
= A_{j}(x − x_{j}) + B_{j}(u − u_{j}).  (15.18)

We can represent model (15.18) in the form

ẋ = A_{j}x + B_{j}u − (A_{j}x_{j} + B_{j}u_{j}) .  (15.19)

The term (A_{j}x_{j} + B_{j}u_{j}) does not have to be equal to zero, and hence (15.19) is not a linear model, but rather an affine model. We illustrate, on a numerical example, that the term (A_{j}x_{j} + B_{j}u_{j}) is indeed, in general, not equal to zero.

Example 15.5 Consider a subsystem of model (2.78) of the form

[ ẋ_{1} ; ẋ_{2} ] = [ x_{2}_{\frac{g sin(x_{1})−mlax_{2}^{2} sin(2x_{1})/2}{4l/3−mla cos^{2}(x_{1})}} ] + [ \begin{matrix}0 \\ −\frac{a cos(x_{1})}{4l/3−mla cos^{2}(x_{1})} \end{matrix} ] u,  (15.20)

where g = 9.8 m/sec^{2}, m = 2 kg, M = 8 kg, a = 1/(m + M), l = 0.5 m. We linearize (15.20) about x_{1} = π/4. For x_{1} = π/4 to be a component of an equilibrium state, we have to have x_{2} = 0. Hence, the equilibrium state is

x_{e} = [ π/4 0 ]^{⊤} .

We next compute u = u_{e} so that equation (15.16) holds. Performing simple calculations, we get u_{e} = 98. Then, using (15.13) we obtain

A = [ \begin{matrix}0 & 1 \\ 22.4745 & 0 \end{matrix} ] .  (15.21)

Next, using (15.14) gives

B = [ \begin{matrix}0 \\ −0.1147 \end{matrix} ] .  (15.22)

Note that

Ax_{e} + Bu_{e} = [ \begin{matrix}0 \\ 6.4108 \end{matrix} ] ≠ 0.

The above example illustrates the fact that the system being linear in Δx = x − x_{0} and Δu = u − u_{0} is, in general, only affine, and not linear in x and u because of the presence of the offset term − (Ax_{0} + Bu_{0}). The designer must than be careful when constructing local models using Taylor series linearization because this approach, in general, will not yield local models linear in x and u. We note that Taylor’s linearization yields a linear system in x and u if the equilibrium point is (x_{0}^{⊤}, u_{0}^{⊤}) = (0^{⊤}, 0^{⊤}).

We now describe our method of constructing local models using the truth model of the given plant. We assume that the truth model has the form of (15.9). Suppose that we are given an operating state x_{0} that does not have to be an equilibrium state of (15.9). Our goal is to construct a linear model in x and u that approximates the behavior of (15.9) in the vicinity of the operating state x_{0}, that is, we wish to find constant matrices A and B such that in a neighborhood of x_{0},

f(x) + G(x)u ≈ Ax + Bu,  (15.23)

and

f(x_{0}) + G(x_{0})u = Ax_{0} + Bu for all u.  (15.24)

Because u is arbitrary, we have to have

G(x_{0}) = B.  (15.25)

Thus, we are left with finding a constant matrix A such that in a neighborhood of x_{0},

f(x) ≈ Ax,  (15.26)

and

f(x_{0}) = Ax_{0}.  (15.27)

Let a_{i}^{⊤} denote the i-th row of the matrix A. Then, for the purpose of further analysis, we represent condition (15.26) as

f_{i}(x) ≈ a_{i}^{⊤}x, i = 1, 2, …, n,  (15.28)

and we represent (15.27) as

f_{i}(x_{0}) = a_{i}^{⊤}x_{0}, i = 1, 2, …, n,  (15.29)

where the i-th component of f is f_{i} : ℝ^{n} → ℝ. Expanding the left hand side of (15.28) about x_{0} and neglecting second and higher order terms yields

f_{i}(x_{0}) + ∇f_{i}(x_{0})^{⊤}(x − x_{0}) ≈ a_{i}^{⊤}x,  (15.30)

where ∇f_{i}(x) : ℝ^{n} → ℝ^{n} is the gradient (a column vector) of f_{i} at x. Now, using (15.29), we represent (15.30) as

∇f_{i}(x_{0})^{⊤}(x − x_{0}) ≈ a_{i}^{⊤} (x − x_{0}) ,  (15.31)

where x is arbitrary but “close” to x_{0}. Our task now is to determine a constant vector a_{i} that is as “close as possible” to ∇f_{i}(x_{0}) and satisfies the constraint a_{i}^{⊤}x_{0} = f_{i}(x_{0}). Let

E = \frac{1}{2} ∥∇f_{i}(x_{0}) − a_{i}∥_{2}^{2} .

Then, we can formulate our task as a constrained optimization problem of the form

subject to a_{i}^{⊤}x_{0} = f_{i}(x_{0}).^{minimize E ; a_{i}} }  (15.32)

We note that (15.32) is a convex constrained optimization problem. This means that the first-order necessary condition for a minimum of E is also sufficient, see, for example, Luenberger [224, Section 6.5] or Chong and Żak [62, Chapter 21]. The first-order conditions for the optimization problem given by (15.32) are

∇_{a_{i}}E + λ∇_{a_{i}} (a_{i}^{⊤}x_{0} − f_{i}(x_{0})) = 0,  (15.33)
a_{i}^{⊤}x_{0} = f_{i}(x_{0}),  (15.34)

where λ is the Lagrange multiplier and the subscript a_{i} in ∇_{a_{i}} indicates that the gradient, ∇ is computed with respect to a_{i}. Performing the required differentiation in (15.33) yields

a_{i} − ∇f_{i}(x_{0}) + λx_{0} = 0,  (15.35)
a_{i}^{⊤}x_{0} = f_{i}(x_{0}).  (15.36)

Recall that we consider now the case when x_{0} ≠ 0. Premultiplying (15.35) by x_{0}^{⊤} and substituting (15.36) into the resulting equation yields

λ = \frac{x_{0}^{⊤}∇f_{i}(x_{0}) − f_{i}(x_{0})}{∥x_{0}∥^{2}}.  (15.37)

Substituting λ given by (15.37) into (15.35), we obtain

a_{i} = ∇f_{i}(x_{0}) + \frac{f_{i}(x_{0}) − x_{0}^{⊤}∇f_{i}(x_{0})}{∥x_{0}∥^{2}}x_{0}, x_{0} ≠ 0  (15.38)

We now summarize the above development. Let

J(x_{0}) = \frac{∂f}{∂x}|_{x=x_{0}}

be the Jacobian matrix of f evaluated at the operating state x = x_{0}. Then a linear model in x and u that approximates the behavior of (15.9) in the vicinity of the operating state x_{0} has the form

ẋ = [J(x_{0}) + \frac{(f(x_{0}) − J(x_{0})x_{0}) x_{0}^{⊤}}{∥x_{0}∥^{2}} ] x + G(x_{0})u, x_{0} ≠ 0  (15.39)

On the other hand, if the operating pair (x_{0}, u_{0}) is an equilibrium pair, then Taylor’s linearization of (15.9) would yield the following linearized model;

\frac{d}{dt}(x − x_{0}) = [ \frac{∂f}{∂x}|_{x=x_{0}} + ∑_{k=1}^{m} u_{k} \frac{∂g_{k}}{∂x} |_{{x=x_{0} \atop u=u_{0}}} ] (x − x_{0}) + G(x_{0})(u − u_{0}),

where the matrix

\frac{∂f}{∂x}|_{x=x_{0}} + ∑_{k=1}^{m} u_{k} \frac{∂g_{k}}{∂x} |_{{x=x_{0} \atop u=u_{0}}}

is computed in (15.13).

Example 15.6 We apply (15.38) to construct local linear models of the nonlinear model (15.20) that we analyzed in Example 15.5. This nonlinear model was also considered by Wang et al. [341, page 20]. Suppose that, as in Wang et al. [341], we wish to construct two local models; one describing the plant operation about x_{1} = 0 and the other local model when x_{1} is about ±π/2. In both models, we assume that x_{2} = 0 and u is arbitrary. Note that the nonlinear model (15.20) is in the format given by (15.9). We first construct a local model about the operating state x_{1} = x_{2} = 0. The local model for this case can be obtained using Taylor’s linearization about the equilibrium point (x_{0}, u_{0}) = (0, 0).

We next construct the second local model that corresponds to x_{1} = 88^{∘}π/180^{∘} and x_{2} = 0. Let β = cos(88^{∘}). Then, the input matrix B can be obtained from (15.20) using (15.25),

B = [ \begin{matrix}0 \\ −\frac{aβ}{4l/3−mlaβ^{2}} \end{matrix} ] .

We then compute the matrix A of the second local model using (15.38). We first compute the first row A. We note that f_{1}(x) = x_{2}, and hence

∇f_{1} = [ 0 1 ]^{⊤} .

The operating state is x_{0} = [ 88^{∘}π/180^{∘} 0 ]^{⊤}. We compute a_{1}^{⊤}, the first row of A, using (15.38) to get

a_{1}^{⊤} = ∇^{⊤}f_{1}(x_{0}) + [ 0 0 ] = [ 0 1 ] .

We again use (15.38) to compute a_{2}^{⊤}, the second row of the matrix A, of the second local model,

a_{2}^{⊤} = [ f_{2}(x_{0})/(88^{∘}π/180^{∘}) 0 ] ≈ [ \frac{g}{4l/3−mlaβ^{2}} (\frac{2}{π}) 0 ] .

The above obtained local models are the same as the ones that were proposed by Wang et al. [341] using a heuristic approach.

15.4.3 Stabilizability of Fuzzy Models

In this subsection, we first present sufficient conditions for asymptotic stability in the large of unforced continuous-time fuzzy model

ẋ = ∑_{i=1}^{r} α_{i}A_{i}x,  (15.40)

where α_{i} = α_{i}(x(t)) ≥ 0 for i = 1, 2, …, r, and ∑_{i=1}^{r} α_{i} = 1. The above model can be viewed as an example of unforced continuous-time polytopic model. We next discuss the stabilizability problem of forced fuzzy models using fuzzy state-feedback control strategies.

It is straightforward to show that a sufficient condition for asymptotic stability in the large of the equilibrium state x = 0 of system (15.40) is that there exists a symmetric positive definite matrix P such that for i = 1, 2, …, r,

A_{i}^{⊤}P + P A_{i} < 0.  (15.41)

It is obvious that a necessary condition for the existence of a common symmetric positive definite P satisfying (15.41) is that each A_{i} be asymptotically stable, that is, the eigenvalues of each A_{i} be in the open left-hand complex plane. We now give another necessary condition for the existence of a common P that satisfies (15.41). Theorem 15.1 Suppose that each A_{i}, i = 1, 2, …, r, in the model (15.40) is asymptotically stable and there exists a symmetric positive definite P such that conditions (15.41) hold. Then, the matrices

∑_{k=1}^{s} A_{i_{k}},

where i_{k} ∈ {1, 2, …, r} and s = 1, 2, 3, …, r are asymptotically stable.

Proof Suppose that conditions (15.41) hold. Let

A_{i_{k}}^{⊤}P + P A_{i_{k}} = −Q_{i_{k}},  (15.42)

where Q_{i_{k}} = Q_{i_{k}}^{⊤} > 0 for i_{k} ∈ {1, 2, …, r}. Summing both sides of (15.42) yields

(∑_{k=1}^{s} A_{i_{k}}^{⊤}) P + P (∑_{k=1}^{s} A_{i_{k}}) = − ∑_{k=1}^{s} Q_{i_{k}}.  (15.43)

By assumption P = P^{⊤} > 0, and because Q_{i_{k}} = Q_{i_{k}}^{⊤} > 0 for i_{k} ∈ {1, 2, …, r}, the symmetric matrices

∑_{k=1}^{s} Q_{i_{k}}, s = 2, 3, …, r,

are also positive definite. Hence, by the Lyapunov theorem, the matrices ∑_{k=1}^{s} A_{i_{k}}, s = 2, 3, …, r, are asymptotically stable.

□

The following corollary can be viewed as a continuous-time counterpart of Theorem 4.3 of Tanaka and Sugeno [322, pages 146–147].

Corollary 15.1 Suppose that each A_{i}, i = 1, 2, …, r in fuzzy model (15.40) is asymptotically stable and there exists a symmetric positive definite P such that conditions (15.41) hold. Then, the matrices

A_{i} + A_{j}, i, j = 1, 2, …, r,

are asymptotically stable.

We mention that Tanaka and Sugeno’s Theorem 4.3 in [322] can be generalized to correspond to Theorem 15.1 rather than to the above corollary.

The above corollary can be restated in an equivalent way as follows. If there exist i and j such that a matrix A_{i} + A_{j} is not asymptotically stable, then there is no positive definite matrix P such that A_{i}^{⊤}P + P A_{i} < 0 for i = 1, 2, …, r. The following example illustrates this fact.

Example 15.7 Given the unforced fuzzy system model described by (15.40), where r = 2 and

A_{1} = [ \begin{matrix}−1 & 4 \\ 0 & −2 \end{matrix} ] , A_{2} = [ \begin{matrix}−1 & 0 \\ 4 & −2 \end{matrix} ] .

Obviously each A_{i}, i = 1, 2, is asymptotically stable. However, there is no common positive definite P such that A_{i}^{⊤}P + P A_{i} < 0 for i = 1, 2 because the matrix

A_{1} + A_{2} = [ \begin{matrix}−1 & 4 \\ 0 & −2 \end{matrix} ] + [ \begin{matrix}−1 & 0 \\ 4 & −2 \end{matrix} ] = [ \begin{matrix}−2 & 4 \\ 4 & −4 \end{matrix} ]

is unstable since its eigenvalues are

{1.1231, −7.1231}.

Theorem 15.1 can be used to check if there does not exist a positive definite matrix P satisfying conditions (15.41) because this theorem gives us a sufficient condition for nonexistence of a common P that satisfies (15.41).

We now present a sufficiency condition for asymptotic stabilizability in the large of the model given by (15.6) and (15.8) using fuzzy state feedback control law

u = − ∑_{j=1}^{r} α_{j}K_{j}x.  (15.44)

The closed-loop system has the form

ẋ = ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) x  (15.45)

To obtain the above model we used the fact that ∑_{j=1}^{r} α_{j} = 1. Suppose that the gain matrices, K_{i}, were chosen so that the matrices

A_{i} − B_{i}K_{i}, i = 1, 2, …, r,

have their eigenvalues in the open left-hand complex plane, that is, they are asymptotically stable. Let

G_{ij} = (A_{i} − B_{i}K_{j}) + (A_{j} − B_{j}K_{i}) for i < j ≤ r.

We present sufficient conditions of Wang et al. [341] for the closed-loop system modeled by (15.45) to be asymptotically stable in the large.

Theorem 15.2 If for some symmetric positive definite P the following conditions are satisfied:

(A_{i} − B_{i}K_{i})^{⊤} P + P (A_{i} − B_{i}K_{i}) < 0, i = 1, 2, …, r  (15.46)

and

G_{ij}^{⊤}P + P G_{ij} < 0, i < j ≤ r,  (15.47)

then the closed-loop system modeled by (15.45) is asymptotically stable in the large.

Proof Let P be such that (15.46) and (15.47) are satisfied. We evaluate the time derivative of x(t)^{⊤}P x(t) on the trajectories of (15.45) to get

\frac{d}{dt} (x(t)^{⊤}P x(t)) = 2x^{⊤}P ẋ
= 2x^{⊤}P (∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) x)
= ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j}x^{⊤} ((A_{i} − B_{i}K_{j})^{⊤} P + P (A_{i} − B_{i}K_{j})) x
= ∑_{i=1}^{r} α_{i}^{2}x^{⊤} ((A_{i} − B_{i}K_{i})^{⊤} P + P (A_{i} − B_{i}K_{i})) x
+ ∑_{i=1}^{r} ∑_{j>i}^{r} α_{i}α_{j}x^{⊤} (G_{ij}^{⊤}P + P G_{ij}) x.  (15.48)

Hence, when (15.46) and (15.47) are satisfied, the positive definite quadratic function x^{⊤}P x becomes a Lyapunov function for the system (15.45), which implies asymptotic stability in the large of (15.45). The proof of the theorem is complete.

□

We give a sufficient condition for the stability of system (15.45) for the case when conditions (15.47) are not satisfied. To proceed, we need the following notation. Suppose that there is a positive definite P such that conditions (15.46) are satisfied. Represent conditions (15.46) as

(A_{i} − B_{i}K_{i})^{⊤} P + P (A_{i} − B_{i}K_{i}) = −Q_{i}, i = 1, 2, …, r,  (15.49)

where each Q_{i} is symmetric positive definite. Let λ_{i} denote the minimum eigenvalue of Q_{i}. We have

λ_{i} > 0, i = 1, 2, …, r,  (15.50)

because Q_{i} = Q_{i}^{⊤} > 0. Let

G_{ij}^{⊤}P + P G_{ij} = −Q_{ij}, i < j ≤ r,  (15.51)

where now we do not require Q_{ij} to be positive definite. Note that Q_{ij} = Q_{ij}^{⊤}, and hence the eigenvalues of Q_{ij} are real. Let λ_{ij} denote the minimum eigenvalue of Q_{ij}. We are ready to state and prove the following theorem.

Theorem 15.3 Suppose that each (A_{i} − B_{i}K_{i}), i = 1, 2, …, r, in (15.45) is asymptotically stable and there exists a symmetric positive definite P such that conditions (15.46) hold. Then, the closed-loop fuzzy model (15.45) is asymptotically stable in the large if the matrix

Λ = [ \begin{matrix}λ_{1} & λ_{12}/2 & ⋯ & λ_{1r}/2 \\ λ_{12}/2 & λ_{2} & & λ_{2r}/2 \\ ⋮ & & ... & ⋮ \\ λ_{1r}/2 & λ_{2r}/2 & ⋯ & λ_{r} \end{matrix} ]  (15.52)

is positive definite.

Proof Let

W = W (x(t)) = x(t)^{⊤}P x(t).

be the Lyapunov function candidate for the closed-loop system modeled by (15.45). We then evaluate the time derivative of W on the trajectories of (15.45) to get

Ẇ = 2x^{⊤}P ẋ
= 2x^{⊤}P (∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) x)
= ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j}x^{⊤} ((A_{i} − B_{i}K_{j})^{⊤} P + P (A_{i} − B_{i}K_{j})) x
= − ∑_{i=1}^{r} α_{i}^{2}x^{⊤}Q_{i}x − ∑_{i=1}^{r} ∑_{j>i}^{r} α_{i}α_{j}x^{⊤}Q_{ij}x.  (15.53)

Recall that for any symmetric matrix M = M^{⊤},

λ_{min}(M)∥x∥^{2} ≤ x^{⊤}Mx,

where λ_{min}(M) is the smallest eigenvalue of M. Using this fact in (15.53) yields

Ẇ ≤ − (∑_{i=1}^{r} α_{i}^{2}λ_{i} + ∑_{i=1}^{r} ∑_{j>i}^{r} α_{i}α_{j}λ_{ij}) ∥x∥^{2}

= − ([ α_{1} α_{2} ⋯ α_{r} ] [ \begin{matrix}λ_{1} & λ_{12}/2 & ⋯ & λ_{1r}/2 \\ λ_{12}/2 & λ_{2} & & λ_{2r}/2 \\ ⋮ & & ... & ⋮ \\ λ_{1r}/2 & λ_{2r}/2 & ⋯ & λ_{r} \end{matrix} ] [ α_{1} ; α_{2} ; ⋮ ; α_{r} ]) ∥x∥^{2}

= − (α^{⊤}Λα) ∥x∥^{2}.  (15.54)

The Lyapunov derivative is negative definite if the matrix Λ is positive definite, which completes the proof.

□

15.4.4 A Lyapunov Based Stabilizer

In this subsection, we apply a Lyapunov approach to the fuzzy control design for uncertain dynamical systems of the form

ẋ = A(α)x + B(α)(u + h(t, x)),  (15.55)

where α satisfies (15.8) and the vector-valued function h represents modeling uncertainties or disturbances. The only information about uncertainties available to the designer are their bounds. Specifically, we assume that the uncertainty modeling function is bounded by a non-negative function η = η(t, x), that is,

∥h(t, x)∥ ≤ η(t, x),

where ∥ · ∥ denotes the vector p-norm. Observe that h affects the system dynamics via the input matrix B(α) in a similar fashion as the control input u does. For this reason, we say that the uncertainty h satisfies the matching condition.

Some of the Lyapunov based control strategies, using non-fuzzy controllers, for uncertain systems are documented in Zinober [375]. The method of constructing stabilizing controllers, presented in this section, requires that each of the matrices A_{i}, i = 1, 2, …, r is already asymptotically stable, and that there exists a common symmetric positive definite P = P^{⊤} > 0 such that, for i = 1, 2, …, r,

A_{i}^{⊤}P + P A_{i} < 0.  (15.56)

If this is not the case, we break the design of stabilizing control law, u, into two parts u = u_{1} + u_{2}. We first design u_{1} of the form

u_{1} = − ∑_{j=1}^{r} α_{j}K_{j}x  (15.57)

so that the system

ẋ = ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) x

is asymptotically stable in the large and there exists a common P such that conditions (15.46) and (15.47) or conditions of Theorem 15.3 are satisfied. Note that the system modeled by (15.55) and driven by (15.57) has the form

ẋ = ∑_{i=1}^{r} α_{i}A_{i}x − ∑_{i=1}^{r} α_{i}B_{i} (∑_{j=1}^{r} α_{j}K_{j}x) + ∑_{i=1}^{r} α_{i}B_{i} (u_{2} + h)
= ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) x + ∑_{i=1}^{r} α_{i}B_{i} (u_{2} + h) .  (15.58)

We next proceed with the design of u_{2}. The role of u_{2} is to reject the uncertainty h. Then, u = u_{1}+u_{2} will asymptotically stabilize in the large the uncertain system (15.55). We propose a family of fuzzy controllers u_{2} that reject the uncertainty h. These controllers are obtained by fuzzifying the controllers proposed in [149]. To proceed, we need the following notation. Let

μ = [ μ_{1} μ_{2} … μ_{m} ]^{⊤} = ∑_{i=1}^{r} α_{i}B_{i}^{⊤}P x.  (15.59)

Let ∥·∥_{p} denote the p-norm of a vector, that is, if a vector v ∈ ℝ^{m}, then its p-norm is defined as

∥v∥_{p} = (|v_{1}|^{p} + |v_{2}|^{p} + ⋯ + |v_{m}|^{p})^{1/p} , p ≥ 1.

A well-known result concerning p-norms is the Hölder’s inequality,

|v^{⊤}w| ≤ ∥v∥_{p}∥w∥_{q}, \frac{1}{p} + \frac{1}{q} = 1.

All norms on ℝ^{m} are equivalent in the sense that if ∥ · ∥_{α} and ∥ · ∥_{β} are norms on ℝ^{m}, then there exist positive constants C_{1} and C_{2} such that

C_{1}∥v∥_{α} ≤ ∥v∥_{β} ≤ C_{2}∥v∥_{α} for all v ∈ ℝ^{m}.  (15.60)

Finally, the gradient of ∥μ∥_{p}, where it is defined, has the form

∇∥μ∥_{p} = [ \frac{∂∥μ∥_{p}}{∂μ_{1}}  \frac{∂∥μ∥_{p}}{∂μ_{2}} … \frac{∂∥μ∥_{p}}{∂μ_{m}} ]^{⊤} , p ≥ 1.  (15.61)

Applying the chain rule, we compute

∇∥μ∥_{p} = \frac{1}{∥μ∥_{p}^{p−1}} [ |μ_{1}|^{p−1} sign(μ_{1}) |μ_{2}|^{p−1} sign(μ_{2}) ⋮ |μ_{m}|^{p−1} sign(μ_{m}) ] .  (15.62)

Let ∥h(t, x)∥_{q} ≤ η_{q}, where η_{q} ≥ 0 is a constant. We then show that when

u_{2} = −η_{q}∇∥μ∥_{p}, p ≥ 1, and \frac{1}{p} + \frac{1}{q} = 1,  (15.63)

then the closed-loop system driven by u = u_{1} + u_{2} is asymptotically stable in the large for any uncertainty h such that ∥h∥_{q} ≤ η_{q}.

Theorem 15.4 Suppose that u_{1}, given by (15.57), is such that there exists a common P satisfying conditions (15.46) and (15.47) or conditions of Theorem 15.3. Then, the system (15.55) driven by u = u_{1} + u_{2}, where u_{2} is given by (15.63), is asymptotically stable in the large for any uncertainty h such that ∥h∥_{q} ≤ η_{q}.

Proof Suppose that u_{1}, given by (15.57), was constructed such that (15.46) and (15.47), or conditions of Theorem 15.3 hold. Then, (15.55) driven by u_{1} takes the form (15.58), that is,

ẋ = ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) x + ∑_{i=1}^{r} α_{i}B_{i} (u_{2} + h) .

We now show that

W = x^{⊤}P x,  (15.64)

is the Lyapunov function for the closed-loop system, where P = P^{⊤} > 0 satisfies (15.46) and (15.47) or conditions of Theorem 15.3. For this, we evaluate the Lyapunov derivative of W, that is, we evaluate \frac{d}{dt}W on the trajectories of (15.58), and show that the Lyapunov derivative of W is negative definite. Before we proceed with evaluating \frac{d}{dt}W, observe that using the fact that

sign(μ_{j}) = \frac{|μ_{j}|}{μ_{j}},

and (15.62), we obtain

μ^{⊤}∇∥μ∥_{p} = ∥μ∥_{p}.

Hence,

\frac{d}{dt}W = 2x^{⊤}P (∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) x + ∑_{i=1}^{r} α_{i}B_{i} (u_{2} + h))
< −2η_{q}μ^{⊤}∇∥μ∥_{p} + 2μ^{⊤}h
≤ −2η_{q}∥μ∥_{p} + 2 |μ^{⊤}h| .  (15.65)

Applying Hölder’s inequality to the above yields

\frac{d}{dt}W < −2η_{q}∥μ∥_{p} + 2∥μ∥_{p}∥h∥_{q}
≤ −2η_{q}∥μ∥_{p} + 2∥μ∥_{p} η_{q}.  (15.66)

Therefore, \frac{d}{dt}W < 0 and by Lyapunov’s theorem, the closed loop-system is asymptotically stable in the large.

□

Example 15.8 Using system model (2.79), we propose the following two rules describing the plant dynamics:

Rule 1 : IF x_{1}(t) is about 0

THEN ẋ(t) = A_{1}x(t) + B_{1}(u(t) + h),

and

Rule 2 : IF x_{1}(t) is about ± π/4
THEN ẋ(t) = A_{2}x(t) + B_{2}(u(t) + h),

where h = −f_{c} + mlx_{2}^{2} sin(x_{1}). Local linear models are constructed using the procedure described in the previous section, and in particular relations (15.25) and (15.38). We have

A_{1} = [ \begin{matrix}0 & 1 & 0 & 0 \\ \frac{g}{4l/3−aml} & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ −\frac{amg}{4/3−am} & 0 & 0 & 0 \end{matrix} ] , B_{1} = [ 0 ; −\frac{a}{4l/3−aml} ; 0 ; \frac{4a/3}{4/3−ma} ] .

The matrices A_{2} and B_{2} of the local model corresponding to the second rule are:

A_{2} = [ \begin{matrix}0 & 1 & 0 & 0 \\ \frac{g sin(x_{1})}{4l/3−mla cos^{2}(x_{1})} (\frac{1}{x_{1}} ) & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ \frac{−mag sin(2x_{1})/2}{4/3−ma cos^{2}(x_{1})} (\frac{1}{x_{1}} ) & 0 & 0 & 0 \end{matrix} ]_{|x_{1}=±π/4} and B_{2} = [ 0 \frac{−a cos(x_{1})}{4l/3−mla cos^{2}(x_{1})} 0 \frac{4a/3}{4/3−ma cos^{2}(x_{1})} ]_{|x_{1}=±π/4} .

The parameter values are the same as in Example 15.5. We use weighting functions of the form:

w_{1}(x_{1}) = \frac{1 − 1/ (1 + exp(−14(x_{1} − π/8)))}{1 + exp(−14(x_{1} + π/8))} and w_{2}(x_{1}) = 1 − w_{1}(x_{1}).

The corresponding fuzzy system model is

ẋ = (α_{1}A_{1} + α_{2}A_{2}) x + (α_{1}B_{1} + α_{2}B_{2}) (u − μ_{c}sign(x_{4}) + mlx_{2}^{2} sin(x_{1})),  (15.67)

where α_{i} = w_{i}, i = 1, 2. Note that w_{1} + w_{2} = 1 for all t. The friction coefficient μ_{c} = 0.005. We then proceed to construct a stabilizing controller. We first construct

u_{1} = − (α_{1}K_{1} + α_{2}K_{2}) x

such that the system

ẋ = ∑_{i=1}^{2} ∑_{j=1}^{2} α_{i}α_{j} (A_{i} − B_{i}K_{j}) x  (15.68)

is asymptotically stable in the large. To test our stabilization algorithm, we randomly chose the set of desired poles for (A_{1} − B_{1}K_{1}) to be in the interval [−4, −1]. The set of desired poles is

{−1.0970, −2.1263, −2.5553, −3.9090}.

The desired poles for (A_{2} − B_{2}K_{2}) were also chosen in random fashion to be in the interval [−4, −1]. In this case, we generated the set of poles of the form,

{−1.5794, −1.6857, −2.7908, −3.6895}.

For this choice of the closed-loop poles of the local models, we obtain the following feedback gain matrices:

K_{1} = [ −294.8766 −73.1208 −13.4726 −27.3362 ] ,

and

K_{2} = [ −440.3915 −118.5144 −19.1611 −35.5575 ] .

Let

G_{12} = A_{1} − B_{1}K_{2} + A_{2} − B_{2}K_{1},

and

P = [ \begin{matrix}54.9580 & 15.6219 & 6.9389 & 12.1165 \\ 15.6219 & 4.5429 & 2.1011 & 3.5488 \\ 6.9389 & 2.1011 & 1.3972 & 1.7978 \\ 12.1165 & 3.5488 & 1.7978 & 2.9375 \end{matrix} ] .

Then, it is easy to check that sufficient conditions, given by (15.46) and (15.47), for asymptotic stability in the large of system (15.68) are satisfied. This completes the design of u_{1}.

We now construct u_{2}. We first use (15.59) to construct

μ = (α_{1}B_{1}^{⊤}P + α_{2}B_{2}^{⊤}P ) x.

In this example, the number of inputs is m = 1, and hence (15.63) reduces to

u_{2} = −η_{q}sign(μ),

where we can take η_{q} = |h|. We combine u_{1} and u_{2} to obtain the following stabilizing controller,

u = u_{1} + u_{2} = − (α_{1}K_{1} + α_{2}K_{2}) x − η_{q}sign(μ).  (15.69)

The above controller globally stabilizes the design model. In our simulations we connected the controller to the truth model given by (2.79). Typical responses are shown in Figure 15.30 and Figure 15.31. The controller locally stabilizes the truth model. We observed that when |x_{1}(0)| > 1.3 rads, and x_{2}(0) = x_{3}(0) = x_{4}(0) = 0, then the equilibrium state x = 0 of the closed-loop system consisting of the truth model driven by (15.69) becomes unstable.

We summarize this section as follows. A common approach to the control system design is to use a design model to construct a controller. The design model is a simplified version of the truth model and usually neglects high frequency dynamics of the plant. Thus, the designer has to cope with an incomplete mathematical model of the process to be controlled. To account for modeling uncertainties, we used a two-term controller. The first term of the controller stabilized the nominal plant, that is, the plant model that does not include modeling uncertainties. The role of the second term of the controller was to reject modeling uncertainties. In both cases fuzzy state-feedback control strategies were used. In order to be able to apply the proposed controllers in real-life systems, we have to account for limited authority of actuators, unmatched uncertainties, as well as inaccessibility of state variables.

Figure 15.30: Plots of x_{1} versus time for two different initial values of x_{1}: x_{1}(0) = 1.2 and x_{1}(0) = 0.6. In both cases x_{2}(0) = x_{3}(0) = x_{4}(0) = 0.

Figure 15.31: Plots of x_{3} versus time for two different initial values of x_{1}: x_{1}(0) = 1.2 and x_{1}(0) = 0.6. In both cases x_{2}(0) = x_{3}(0) = x_{4}(0) = 0.

15.5 Stability of Discrete Fuzzy Models

We analyze the model of the form

x[k + 1] = ∑_{i=1}^{r} w_{i}(x[k])A_{i}x[k],  (15.70)

where

w_{i}(x[k]) ≥ 0 and ∑_{i=1}^{r} w_{i}(x[k]) = 1.

To proceed, we need the following notation. Given two n × n matrices A and B. Then, A < B means that

x^{⊤}Ax < x^{⊤}Bx for all x ∈ ℝ^{n}, x ≠ 0,

that is, the quadratic form x^{⊤} (A − B) x is negative definite.

We prove the following matrix inequality lemma that is useful in the stability analysis of discrete fuzzy models.

Lemma 15.1 Let A, B, P ∈ ℝ^{n×n}. If P = P^{⊤} > 0 such that

A^{⊤}P A − P < 0 and B^{⊤}P B − P < 0,

then

A^{⊤}P B + B^{⊤}P A − 2P < 0.

Proof We first represent A^{⊤}P B + B^{⊤}P A − 2P as

A^{⊤}P B + B^{⊤}P A − 2P = −(A − B)^{⊤}P (A − B) + A^{⊤}P A + B^{⊤}P B − 2P
= −(A − B)^{⊤}P (A − B) + (A^{⊤}P A − P)
+(B^{⊤}P B − P).

The first term, −(A − B)^{⊤}P (A − B), is non-positive definite because by assumption P = P^{⊤} > 0, while the second and the third terms are negative definite by assumption. Hence,

A^{⊤}P B + B^{⊤}P A − 2P < 0,

and the lemma is proven.

□

Using Lemma 15.1, we now prove a sufficient condition for global asymptotic stability of model (15.70).

Theorem 15.5 The equilibrium state, x_{e} = 0, of model (15.70) is globally asymptotically stable if there exists a positive definite matrix P = P^{⊤} > 0 such that for i = 1, 2, …, r,

A_{i}^{⊤}P A_{i} − P < 0.  (15.71)

Proof Suppose that there exists a symmetric positive definite matrix P = P^{⊤} > 0 such that for i = 1, 2, …, r inequalities given by (15.71) are satisfied. Consider then the Lyapunov function candidate

V (x[k]) = x[k]^{⊤}P x[k].  (15.72)

To prove the theorem, it is enough to show that

ΔV (x[k]) < 0.

We have

ΔV (x[k]) = V (x[k + 1]) − V (x[k])
= x[k + 1]^{⊤}P x[k + 1] − x[k]^{⊤}P x[k]
= (∑_{i=1}^{r} w_{i}x^{⊤}A_{i}^{⊤}) P (∑_{i=1}^{r} w_{i}A_{i}x) − x^{⊤}P x
= x^{⊤} ((∑_{i=1}^{r} w_{i}A_{i}^{⊤}) P (∑_{i=1}^{r} w_{i}A_{i}) − P ) x.  (15.73)

To proceed, we represent (15.73) in an equivalent form as

ΔV (x) = x^{⊤} ((∑_{i=1}^{r} w_{i}A_{i}^{⊤}) P (∑_{j=1}^{r} w_{j}A_{j}) − P ) x.  (15.74)

By assumption

∑_{i=1}^{r} w_{i} = 1,

and hence

∑_{i=1}^{r} w_{i} ∑_{j=1}^{r} w_{j} = ∑_{i=1}^{r} ∑_{j=1}^{r} w_{i}w_{j} = 1.  (15.75)

Substituting (15.75) into (15.74) yields

ΔV (x) = x^{⊤} ((∑_{i=1}^{r} w_{i}A_{i}^{⊤}) P (∑_{j=1}^{r} w_{j}A_{j}) − ∑_{i=1}^{r} ∑_{j=1}^{r} w_{i}w_{j}P ) x
= x^{⊤} (∑_{i=1}^{r} ∑_{j=1}^{r} w_{i}w_{j} (A_{i}^{⊤}P A_{j} − P )) x.  (15.76)

We rearrange (15.76) to get

ΔV (x[k]) = ∑_{i=1}^{r} w_{i}^{2}x^{⊤} (A_{i}^{⊤}P A_{i} − P ) x + ∑_{i=1}^{r} ∑_{{j=1 \atop j≠i}}^{r} w_{i}w_{j}x^{⊤} (A_{i}^{⊤}P A_{j} − P ) x

= ∑_{i=1}^{r} w_{i}^{2}x^{⊤} (A_{i}^{⊤}P A_{i} − P ) x
+ ∑_{i=1}^{r} ∑_{j>i}^{r} w_{i}w_{j}x^{⊤} (A_{i}^{⊤}P A_{j} + A_{j}^{⊤}P A_{i} − 2P ) x.  (15.77)

The first term in (15.77) is negative definite because of the assumption and the fact that at least one of the weights, w_{i}, is positive because w_{i} ≥ 0 and ∑_{i=1}^{r} w_{i} = 1. The second term in (15.77) is nonpositive definite by virtue of Lemma 15.1. Hence, V = x^{⊤}P x is a Lyapunov function for the fuzzy model (15.70), and by the Lyapunov theorem the equilibrium state x_{e} = 0 of the model is globally asymptotically stable.

□

The above theorem gives no method for finding a common positive definite matrix P that satisfies r inequalities (15.71). All of the matrices A_{i}, i = 1, 2, …, r, are asymptotically stable if there exists a common positive definite P that satisfies (15.71). However, the asymptotic stability of all the matrices A_{i} is only necessary but not sufficient for the existence of a common symmetric positive definite P that satisfies (15.71). The following theorem provides a necessary, but not sufficient, condition for the existence of positive definite P satisfying (15.71).

Theorem 15.6 Given r square n × n matrices A_{i} that are asymptotically stable and nonsingular. If there exists a symmetric positive definite matrix P = P^{⊤} > 0 such that for i = 1, 2, …, r,

A_{i}^{⊤}P A_{i} − P < 0,  (15.78)

then for i, j = 1, 2, …, r, the matrices A_{i}A_{j} are asymptotically stable.

Proof By assumption, there exists a symmetric positive definite matrix P such that for j = 1, 2, …, r,

A_{j}^{⊤}P A_{j} − P < 0.  (15.79)

Premultiplying the above by (A_{j}^{⊤})^{−1} and postmultiplying by A_{j}^{−1} yields

P − (A_{j}^{⊤})^{−1} P A_{j}^{−1} < 0.  (15.80)

Hence,

P < (A_{j}^{⊤})^{−1} P A_{j}^{−1}.  (15.81)

On the other hand, using (15.78) we get

A_{i}^{⊤}P A_{i} < P .  (15.82)

Combining (15.81) and (15.82), we obtain

A_{i}^{⊤}P A_{i} < (A_{j}^{⊤})^{−1} P A_{j}^{−1}.  (15.83)

Premultiplying both sides of (15.83) by A_{j}^{⊤} and postmultiplying by A_{j}, we get, for i, j = 1, 2, …, r,

(A_{i}A_{j})^{⊤} P (A_{i}A_{j}) − P < 0.  (15.84)

Therefore, for i, j = 1, 2, …, r, the matrices A_{i}A_{j} are asymptotically stable, which completes the proof.

□

The above theorem can be stated in an equivalent way as follows. Assume that for i = 1, 2, …, r, the matrices A_{i} are asymptotically stable and nonsingular. If for some i, j = 1, 2, …, r, the matrix A_{i}A_{j} is unstable, then there does not exist a common positive definite P such that

A_{i}^{⊤}P A_{i} − P < 0 for i = 1, 2, …, r.

Example 15.9 Let

A_{1} = [ \begin{matrix}1 & −0.5 \\ 1 & 0 \end{matrix} ] and A_{2} = [ \begin{matrix}−1 & −0.5 \\ 1 & 0 \end{matrix} ] .

Both matrices, A_{1} and A_{2}, are asymptotically stable. Indeed, the eigenvalues of A_{1} are located at {0.5+j0.5, 0.5−j0.5}, while the eigenvalues of A_{2} are located at {−0.5+j0.5, −0.5− j0.5}. However, the matrix A_{1}A_{2} has its eigenvalues located at {−1.866, −0.134}, and hence is unstable. Thus, it follows from Theorem 15.6, that there is no positive definite P such that

A_{i}^{⊤}P A_{i} − P < 0 for i = 1, 2.  (15.85)

15.6 Fuzzy Observer

In this section, we are concerned with the problem of designing an observer using a TSK model of the plant. We first discuss the issue of constructing a TSK model using inputoutput data. To construct a TSK model, we need local description of the plant in terms of linear models,

\{ \begin{matrix}ẋ(t) & = & A_{i}x(t) + B_{i}u(t) \\ y(t) & = & C_{i}x(t), \end{matrix}

where i = 1, 2, …, r, state vector x(t) ∈ ℝ^{n}, the control input u(t) ∈ ℝ^{m}, the output y(t) ∈ ℝ^{p}, and the matrices A_{i}, B_{i}, and C_{i} are of appropriate dimensions. The above information is then fused with the available IF-THEN rules, where the i-th rule can have the form:

Rule i : IF z_{1}(t) is F_{1}^{i} AND ⋯ AND z_{q}(t) is F_{q}^{i},
THEN \{ \begin{matrix}ẋ(t) & = & A_{i}x(t) + B_{i}u(t) \\ y & = & C_{i}x(t), \end{matrix}

where F_{j}^{i}, j = 1, 2, …, q, is the j-th fuzzy set of the i-th rule, and z_{1}(t), …, z_{q}(t) are measurable system variables. Let μ_{j}^{i}(z_{j}) be the membership function of the fuzzy set F_{j}^{i}, and let

w^{i} = w^{i}(z) = ∏_{j=1}^{n} μ_{j}^{i}(z_{j}),

where r is the number of fuzzy rules. The resulting fuzzy system model is the weighted average of the local models and has the form of (15.6), that is,

ẋ = \frac{∑_{i=1}^{r} w^{i} (A_{i}x + B_{i}u)}{∑_{i=1}^{r} w^{i}}
= ∑_{i=1}^{r} α_{i} (A_{i}x + B_{i}u)
= (∑_{i=1}^{r} α_{i}A_{i}) x + (∑_{i=1}^{r} α_{i}B_{i}) u
= A(α)x + B(α)u,

where for i = 1, 2, …, r,

α_{i} = \frac{w^{i}}{∑_{i=1}^{r} w^{i}} .

Similarly,

y = \frac{∑_{i=1}^{r} w^{i}C_{i}x}{∑_{i=1}^{r} w^{i}}
= (∑_{i=1}^{r} α_{i}C_{i}) x
= C(α)x.  (15.86)

The coefficients α_{i}s satisfy (15.8), that is,

α_{i} ≥ 0, ∑_{i=1}^{r} α_{i} = 1 and α = [ α_{1} α_{2} ⋯ α_{r} ]^{⊤} ∈ [0, 1]^{r}.

A fuzzy observer for the system modeled by (15.6) and (15.86) can have the form

˙x̃ = A(α)x̃ + B(α)u + L(α) (y − ỹ) ,  (15.87)

where x̃ is the estimate of the plant state x and

L(α) = ∑_{i=1}^{r} α_{i}L_{i}.

The observer (15.87) can be represented equivalently as

˙x̃ = ∑_{i=1}^{r} α_{i}A_{i}x̃ + ∑_{i=1}^{r} α_{i}B_{i}u + ∑_{i=1}^{r} α_{i}L_{i} (y − ỹ)  (15.88)

We note that the observer (15.88) can be viewed as a result of implementing the following r fuzzy rules:

Rule i : IF z_{1}(t) is F_{1}^{i} AND ⋯ AND z_{q}(t) is F_{q}^{i},
THEN \{ \begin{matrix}˙ ; x̃ & = & A_{i}x̃ + B_{i}u + L_{i} (y − ỹ) \\ ỹ & = & C_{i}x̃, \end{matrix}

where, as before, z_{1}(t), …, z_{q}(t) are measurable system variables. We use the state estimate in the controller implementation to get

u = − ∑_{j=1}^{r} α_{j}K_{j}x̃ + v.  (15.89)

Substituting the above into the plant model yields

ẋ = ∑_{i=1}^{r} α_{i}A_{i}x − ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j}B_{i}K_{j}x̃ + ∑_{i=1}^{r} α_{i}B_{i}v.  (15.90)

Substituting (15.86) and (15.89) into the observer equation (15.88), we obtain

˙ ; x̃ = ∑_{i=1}^{r} α_{i}A_{i}x̃ − ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j}B_{i}K_{j}x̃ + ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j}L_{i}C_{j} (x − x̃) + ∑_{i=1}^{r} α_{i}B_{i}v
= ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) x̃ + ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j}L_{i}C_{j} (x − x̃) + ∑_{i=1}^{r} α_{i}B_{i}v. (15.91)

Equations (15.90) and (15.91) constitute the closed-loop system driven by the combined fuzzy controller-observer compensator. We represent the above two equations modeling the closed-loop system in matrix form as

[ ẋ ; ˙ ; x̃ ] = [ \begin{matrix}∑_{i=1}^{r} α_{i}A_{i} & − ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j}B_{i}K_{j} \\ ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j}L_{i}C_{j} & ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j} − L_{i}C_{j}) \end{matrix} ] [ x ; x̃ ]
+ [ \begin{matrix}∑_{i=1}^{r} α_{i}B_{i} \\ ∑_{i=1}^{r} α_{i}B_{i} \end{matrix} ] v  (15.92)
y = [ ∑_{i=1}^{r} α_{i}C_{i} O ] [ x ; x̃ ] .  (15.93)

15.6.1 The Comparison Method for Linear Systems

We present the comparison method for linear systems. that is used by us to investigate stability properties of the closed-loop system, modeled by (15.92) and (15.93), consisting of a fuzzy model of a given plant and a fuzzy controller-estimator compensator.

Let A be a constant n × n matrix and e^{At} denote the matrix exponential. We give a necessary and sufficient condition for the matrix exponential to have all its elements nonnegative for t ≥ 0. This result appears in [27, page 171].

Lemma 15.2 A necessary and sufficient condition that all elements of e^{At} be non-negative for t ≥ 0 is that

a_{ij} ≥ 0 i ≠ j.  (15.94)

Proof We follow the arguments of Bellman [27, page 172], see also [26]. We first prove the necessary condition. Because

e^{At} = I_{n} + At + \frac{1}{2}A^{2}t^{2} + ⋯

it is clear that (15.94) is necessary for the result to hold for small t, which establishes the necessity condition

To prove the sufficiency part, let c_{1} be a scalar so that all the elements of A + c_{1}I_{n} are non-negative. Then, obviously, all the elements of e^{(A+c_{1}I_{n})t} are non-negative. Also, the elements of e^{−c_{1}I_{n}t} are non-negative because exponentials are always non-negative. The matrices A + c_{1}I_{n} and −c_{1}I_{n} commute. Hence,

e^{At} = e^{(A+c_{1}I_{n})t−c_{1}I_{n}t}
= e^{(A+c_{1}I_{n})t}e^{−c_{1}I_{n}t},

which establishes the desired non-negativity condition and the sufficiency condition.

□

Using the above lemma, we prove the following lemma due to Bellman [28]. We use the following notation: x ≤ y implies component-by-component majorization, that is, x_{i} ≤ y_{i} for i = 1, 2, …, n, where x_{i} and y_{i} are the i-th components of x and y, respectively.

Lemma 15.3 If a_{ij} ≥ 0, i ≠ j, then for t ≥ 0,

\frac{dx}{dt} ≤ Ax, x(0) = c,  (15.95)

implies x ≤ y, where

\frac{dy}{dt} = Ay, y(0) = c.  (15.96)

Proof We can represent (15.95) as

\frac{dx(t)}{dt} = Ax(t) − f(t), x(0) = c,  (15.97)

where f(t) is a vector-valued function whose components are non-negative for all t ≥ 0. The solution of (15.97) is represented as

x(t) = e^{At}c − ∫_{0}^{t} e^{A(t−s)}f(s)ds
= y(t) − ∫_{0}^{t} e^{A(t−s)}f(s)ds.  (15.98)

In the above, s is the integration variable and t ≥ 0. Hence, t ≥ s ≥ 0. Using Lemma 15.2, we conclude that the components of the integrand of the second term in (15.98) are all non-negative for t ≥ s ≥ 0. Therefore,

∫_{0}^{t} e^{A(t−s)}f(s)ds ≥ 0, for t ≥ 0,

which implies

x ≤ y.

The proof is complete.

□

Equation (15.96) is referred to as the equation of comparison for (15.95).

15.6.2 Stability Analysis of the Closed-Loop System

We now perform a stability analysis of the closed-loop system given by (15.92) and (15.93). We first apply a state variable transformation to bring the system into a form convenient for further analysis. The transformation has the form,

[ \begin{matrix}x \\ x̃ − x \end{matrix} ] = [ \begin{matrix}I_{n} & O \\ −I_{n} & I_{n} \end{matrix} ] [ x ; x̃ ] .  (15.99)

The closed-loop system model in the new coordinates is

[ ẋ ; ˙x̃ − ẋ ] = [ \begin{matrix}∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) & − ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j}B_{i}K_{j} \\ O & ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − L_{i}C_{j}) \end{matrix} ] [ \begin{matrix}x \\ x̃ − x \end{matrix} ]
+ [ \begin{matrix}∑_{i=1}^{r} α_{i}B_{i} \\ O \end{matrix} ] v  (15.100)
y = [ ∑_{i=1}^{r} α_{i}C_{i} O ] [ \begin{matrix}x \\ x̃ − x \end{matrix} ] .  (15.101)

Let e = x̃ − x. We now present a sufficient condition for the unforced closed-loop system to be globally uniformly asymptotically stable, that is, we will give a condition for uniform asymptotic stability in the large of the system

[ ẋ ; ė ] = [ \begin{matrix}∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) & − ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j}B_{i}K_{j} \\ O & ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − L_{i}C_{j}) \end{matrix} ] [ \begin{matrix}x \\ e \end{matrix} ] .

(15.102)

This condition was proposed by Ma, Sun, and He [225].

Theorem 15.7 If there exist two scalar functions V (x) : ℝ^{n} → ℝ and Ṽ (e) : ℝ^{n} → ℝ and positive real numbers γ_{i}, γ̃_{i}, i = 1, …, 4 such that

γ_{1}∥x∥^{2} ≤ V (x) ≤ γ_{2}∥x∥^{2}, γ̃_{1}∥e∥^{2} ≤ Ṽ (e) ≤ γ̃_{2}∥e∥^{2},  (15.103)

\begin{matrix}\frac{∂V (x)}{∂x} ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) x ≤ −γ_{3}∥x∥^{2} \\ \frac{∂Ṽ (e)}{∂e} ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − L_{i}C_{j}) e ≤ −γ̃_{3}∥e∥^{2}, \end{matrix} \}  (15.104)

and

‖\frac{∂V (x)}{∂x}‖ ≤ γ_{4}∥x∥, ‖\frac{∂Ṽ (e)}{∂e}‖ ≤ γ̃_{4}∥e∥,  (15.105)

then the system (15.102) is globally uniformly asymptotically stable.

Proof We evaluate the time derivative of V on the trajectories of the system (15.102),

V̇ (x) = \frac{∂V (x)}{∂x}ẋ
= \frac{∂V (x)}{∂x} ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − B_{i}K_{j}) x − \frac{∂V (x)}{∂x} ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j}B_{i}K_{j}e
≤ −γ_{3}∥x∥^{2} + ‖\frac{∂V (x)}{∂x}‖ ∑_{i=1}^{r} ∑_{j=1}^{r} ∥B_{i}K_{j}∥ ∥e∥
≤ −\frac{γ_{3}}{2} ∥x∥^{2} − \frac{γ_{3}}{2} ∥x∥^{2} + γ_{4} ∑_{i=1}^{r} ∑_{j=1}^{r} ∥B_{i}K_{j}∥ ∥e∥∥x∥.  (15.106)

To proceed, we observe that

\frac{γ_{3}}{2} ∥x∥^{2} − γ_{4} ∑_{i=1}^{r} ∑_{j=1}^{r} ∥B_{i}K_{j}∥ ∥e∥∥x∥ + \frac{γ_{4}^{2}}{2γ_{3}} (∑_{i=1}^{r} ∑_{j=1}^{r} ∥B_{i}K_{j}∥)^{2} ∥e∥^{2} ≥ 0.  (15.107)

Indeed, we can represent the left-hand side of the above as

[ ∥x∥ ∥e∥ ] [ \begin{matrix}\frac{γ_{3}}{2} & −\frac{γ_{4}}{2} ∑_{i=1}^{r} ∑_{j=1}^{r} ∥B_{i}K_{j}∥ \\ −\frac{γ_{4}}{2} ∑_{i=1}^{r} ∑_{j=1}^{r} ∥B_{i}K_{j}∥ & \frac{γ_{4}^{2}}{2γ_{3}} (∑_{i=1}^{r} ∑_{j=1}^{r} ∥B_{i}K_{j}∥)^{2} \end{matrix} ] [ \begin{matrix}∥x∥ \\ ∥e∥ \end{matrix} ] .

(15.108)

It is easy to check that the above quadratic form is positive semi-definite—all three principle minors are non-negative. We use this fact and (15.103) to further evaluate (15.106) as

V̇ (x) ≤ −\frac{γ_{3}}{2} ∥x∥^{2} + \frac{γ_{4}^{2}}{2γ_{3}} (∑_{i=1}^{r} ∑_{j=1}^{r} ∥B_{i}K_{j}∥)^{2} ∥e∥^{2}
≤ −\frac{γ_{3}}{2γ_{2}} V (x) + \frac{γ_{4}^{2}}{2γ_{3}γ̃_{1}} (∑_{i=1}^{r} ∑_{j=1}^{r} ∥B_{i}K_{j}∥)^{2} Ṽ (e).  (15.109)

Let a = (∑_{i=1}^{r} ∑_{j=1}^{r} ∥B_{i}K_{j}∥)^{2}. Then, (15.109) can be written as

V̇ (x) ≤ −\frac{γ_{3}}{2γ_{2}} V (x) + \frac{aγ_{4}^{2}}{2γ_{3}γ̃_{1}} Ṽ (e).  (15.110)

Evaluating the time derivative of Ṽ (e) on the trajectories of (15.102) gives

˙Ṽ (e) = \frac{∂Ṽ (e)}{∂e}ė
≤ \frac{∂Ṽ (e)}{∂e} ∑_{i=1}^{r} ∑_{j=1}^{r} α_{i}α_{j} (A_{i} − L_{i}C_{j}) e
≤ −γ̃_{3}∥e∥^{2}
≤ −\frac{γ̃_{3}}{γ̃_{2}} Ṽ (e).  (15.111)

Combining (15.110) and (15.111) into a single matrix differential inequality of the form yields

[ \begin{matrix}V̇ (x) \\ ˙Ṽ (e) \end{matrix} ] ≤ [ \begin{matrix}−\frac{γ_{3}}{2γ_{2}} & \frac{aγ_{4}^{2}}{2γ_{3}γ̃_{1}} \\ 0 & −\frac{γ̃_{3}}{γ̃_{2}} \end{matrix} ] [ \begin{matrix}V (x) \\ Ṽ (e) \end{matrix} ]
= A [ \begin{matrix}V (x) \\ Ṽ (e) \end{matrix} ] .  (15.112)

We then refer to Lemma 15.3 to conclude that the system (15.102) is globally uniformly asymptotically stable. Indeed, because the equation of comparison for (15.112) is globally uniformly asymptotically stable, we have to have

V (x) → 0 and Ṽ (e) → 0 as t → ∞.

Because V and Ṽ are both positive definite decrescent and radially unbounded functions, the above can hold provided x → 0 and e → 0 uniformly with respect to t and irrespective of the initial conditions x(0) and e(0). Therefore, the closed-loop system is globally uniformly asymptotically stable.

□

15.7 Adaptive Fuzzy Control

In this Section, we analyze adaptive fuzzy robust tracking controllers for a class of uncertain dynamical systems. The controllers’ construction and their analysis involves sliding modes.

15.7.1 Plant Model and Control Objective

We consider a class of plants modeled by the differential equation

x^{(n)} = f (x) + g (x) u + η(t, x),  (15.113)

where x = [ x ẋ ⋯ x^{(n−1)} ]^{⊤}, and the functions f = f (x) and g = g (x) are unknown to us. The function η = η(t, x) models the plant’s disturbances. We assume that

|η(t, x)| ≤ d,  (15.114)

where d > 0 is known to us. We further assume that the function g(x) has a positive lower bound, that is, there exists a positive constant \underline{g} such that g(x) ≥ \underline{g} > 0. Let

x_{d} = x_{d}(t) = [ x_{d}(t) ẋ_{d}(t) ⋯ x_{d}^{(n−1)}(t) ]^{⊤}

denote the desired state trajectory and let

e = x − x_{d}
= [ x − x_{d} ẋ − ẋ_{d} ⋯ x^{(n−1)} − x_{d}^{(n−1)} ]^{⊤}
= [ e ė ⋯ e^{(n−1)} ]^{⊤} .

We wish to construct a controller u such that

lim_{t→∞} e(t) = 0.

We analyze two different fuzzy adaptive tracking controllers and show that the plant (15.113) driven by these controllers is stable and the adaptation parameters are bounded.

In the next subsection, we present background results used in the adaptive fuzzy controllers’ construction.

15.7.2 Background Results

Fuzzy Logic System

Recall, from Section 15.2, that a fuzzy logic controller with q inputs, v_{1}, v_{2}, …, v_{q}, and the center average defuzzifier can be represented as

u = \frac{∑_{l=1}^{M} ∏_{j=1}^{q} μ_{F_{j}^{l}}(v_{j})θ_{l}}{∑_{l=1}^{M} ∏_{j=1}^{q} μ_{F_{j}^{l}}(v_{j})} ,  (15.115)

where M is the number of fuzzy rules used to construct the controller, F_{j}^{l} is the j-th fuzzy set corresponding to the l-th fuzzy rule, and c_{l} can be taken as a centroid of the l-th fuzzy set corresponding to the controller’s output, u. To proceed, we represent (15.115) as

u = ∑_{l=1}^{M} (\frac{∏_{j=1}^{q} μ_{F_{j}^{l}}(v_{j})}{∑_{l=1}^{M} ∏_{j=1}^{q} μ_{F_{j}^{l}}(v_{j})}) θ_{l}.  (15.116)

Let

θ = [ θ_{1} θ_{2} ⋯ θ_{M} ]^{⊤}

and

ξ = [ ξ_{1} ξ_{2} ⋯ ξ_{M} ]^{⊤} = [ \frac{∏_{j=1}^{q} μ_{F^{1}_{j}}(v_{j})}{∑_{l=1}^{M} ∏_{j=1}^{q} μ_{F^{l}_{j}}(v_{j})} ⋯ \frac{∏_{j=1}^{q} μ_{F^{M}_{j}} (v_{j})}{∑_{l=1}^{M} ∏_{j=1}^{q} μ_{F^{l}_{j}}(v_{j})} ]^{⊤} .

Then, we can represent (15.116) as

u = θ^{⊤}ξ,  (15.117)

where the vector ξ is referred to as the regressor vector in the control literature.

Example 15.10 Consider an uncertain dynamical system model,

[ ẋ_{1} ; ẋ_{2} ] = [ \begin{matrix}0 & 1 \\ 0 & −1 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \end{matrix} ] + [ \begin{matrix}0 \\ 1 \end{matrix} ] (u + ζ), |ζ| ≤ 5.

Our objective is to design a controller that contains an adaptive fuzzy term so that the state of the above system will track a reference signal generated by the reference model,

ż = [ \begin{matrix}0 & 1 \\ −1 & −1.4 \end{matrix} ] z + [ \begin{matrix}0 \\ 1 \end{matrix} ] r
= A_{d}z + b_{d}r,

where r is a reference input such that |r| ≤ 2. We choose the controller settings and show that the tracking error of the closed-loop system driven by your controller asymptotically decays to zero. The solution to the Lyapunov matrix equation A_{d}^{⊤}P + P A_{d} = −I_{2} is

P = [ \begin{matrix}1.4143 & 0.5000 \\ 0.5000 & 0.7143 \end{matrix} ] .

To proceed, we define the tracking error e_{x}(t) = x(t) − z(t). Note that b = b_{d} and A_{d} = A − bk, where k = [ −1 −0.4 ]. Then, the differential equation that describes the error dynamics is

ė_{x} = ẋ − ż
= Ax + b(u + ζ) − A_{d}z − br
= (A_{d} + bk)x + b(u + ζ) − A_{d}z − br
= A_{d}e_{x} + b(u + ζ − r + kx).

We propose the following control strategy,

u = −u_{fuzzy} − η \frac{b^{⊤}P e_{x}}{|b^{⊤}P e_{x}|} = −u_{fuzzy} − η sign(0.5e_{1} + 0.7143e_{2}),

where η will be determined later, and

u_{fuzzy} = θ^{⊤}ξ,

where the vector ξ is bounded. The role of the fuzzy term is to minimize the effect of uncertainties modeled by the bounded function ζ. Let θ^{∗} denote the “optimal” weight vector such that

ζ = θ^{∗⊤}ξ + e,  (15.118)

where e is the approximation error. We assume that the approximation error is bounded by ρ, that is,

|e| ≤ ρ.  (15.119)

Consider the following Lyapunov function candidate,

V = \frac{1}{2}e_{x}^{⊤}P e_{x} + \frac{1}{2α}ϕ^{⊤}ϕ,

where

ϕ = θ − θ^{∗}.  (15.120)

We note that we are now in the augmented state space (e_{x}, ϕ). The Lyapunov derivative is

V̇ = e_{x}^{⊤}P ė_{x} + \frac{1}{α}ϕ^{⊤}ϕ̇
= e_{x}^{⊤}P (A_{d}e_{x} + b(u + ζ − r + kx)) + \frac{1}{α}ϕ^{⊤}ϕ̇
= −\frac{1}{2}e_{x}^{⊤}e_{x} + e_{x}^{⊤}P bu + e_{x}^{⊤}P b (ζ − r + kx) + \frac{1}{α}ϕ^{⊤}ϕ̇.

Substituting (15.118) and the control strategy into the above gives

V̇ = −\frac{1}{2}∥e_{x}∥^{2} − ηe_{x}^{⊤}P b\frac{b^{⊤}P e_{x}}{|b^{⊤}P e_{x}|} − e_{x}^{⊤}P bθ^{⊤}ξ + e_{x}^{⊤}P b (θ^{∗⊤}ξ + e − r + kx) + \frac{1}{α}ϕ^{⊤}ϕ̇.

Substituting into the above (15.119) and (15.120) and performing manipulations yields

V̇ = −\frac{1}{2}∥e_{x}∥^{2} − η|e_{x}^{⊤}P b| − e_{x}^{⊤}P b (θ − θ^{∗})^{⊤} ξ + e_{x}^{⊤}P b (e − r + kx) + \frac{1}{α}ϕ^{⊤}ϕ̇
≤ −\frac{1}{2}∥e_{x}∥^{2} − η|e_{x}^{⊤}P b| − e_{x}^{⊤}P bϕ^{⊤}ξ + ρ|e_{x}^{⊤}P b| + |e_{x}^{⊤}P b| (|r| + |kx|) + \frac{1}{α}ϕ^{⊤}ϕ̇
= −\frac{1}{2}∥e_{x}∥^{2} − |e_{x}^{⊤}P b| (η − ρ − |r| − |kx|) − ϕ^{⊤} ((e_{x}^{⊤}P b) ξ − \frac{1}{α}ϕ̇) .

We select

η ≥ ρ + |r| + |kx| and ϕ̇ = θ̇ = α (e_{x}^{⊤}P b) ξ

to obtain

V̇ ≤ −\frac{1}{2}∥e_{x}∥^{2}.  (15.121)

Thus V̇ is negative semi-definite in the augmented (e_{x}, ϕ)-space and therefore e_{x} as well as ϕ, and hence θ are bounded. Hence the proposed controller is also bounded, which implies that ė_{x} is bounded. This in turn implies that

\frac{d}{dt}∥e_{x}∥^{2} = 2e_{x}^{⊤}ė_{x}

is bounded and hence ∥e_{x}∥^{2} is uniformly continuous. Integrating both sides of (15.121) gives

∫_{0}^{t} V̇ (τ)dτ = V (t) − V (0) ≤ ∫_{0}^{t} (−\frac{1}{2}∥e_{x}∥^{2}) dτ

and hence

∫_{0}^{t} ∥e_{x}∥^{2}dτ ≤ 2(V (0) − V (t)) ≤ 2V (0)

because V̇ ≤ 0 and V > 0, which means that

lim_{t→∞} ∫_{0}^{t} ∥e_{x}∥^{2}dτ

exists and is finite. By Barbalat’s lemma, stated on page 395, ∥e_{x}∥^{2} → 0 as t → ∞, which implies that

lim_{t→∞} e_{x} = 0.

Projection Operator

Consider a vector-valued function of time,

θ(t) = [ θ_{1}(t) ⋯ θ_{m}(t) ]^{⊤} ∈ ℝ^{m},

where, θ̇_{i}(t) = α_{i}(t), i = 1, ⋯, m. We wish the components θ_{i}(t)s to lie between \underline{θ}_{i} and θ̄_{i}, that is, \underline{θ}_{i} ≤ θ_{i}(t) ≤ θ̄_{i}. One way to achieve this goal is by setting θ̇_{i}(t) = 0 when θ_{i}(t) reaches either of the bounds and tends to go beyond the bound, that is,

θ̇_{i}(t) = \{ \begin{matrix}0 & if θ_{i} = \underline{θ}_{i} and α_{i}(t) < 0 \\ 0 & if θ_{i} = θ̄_{i} and α_{i}(t) > 0 \\ α_{i}(t) & otherwise. \end{matrix}  (15.122)

For compactness, we use the notation Proj for the right hand side of (15.122). Then, we write

θ̇_{i}(t) = P roj_{θ_{i}}(α_{i}), i = 1, …, m,  (15.123)

or in vector notation,

θ̇(t) = P rojθ(α).  (15.124)

Sliding Mode Tracking Controller

Let

s = [ s_{1} s_{2} ⋯ s_{n−1} 1 ] ∈ ℝ^{1×n}

and σ(e) = se. Then, {e : σ(e) = 0} represents a sliding surface in the tracking error, e, space. The coefficients s_{i} of the sliding surface are chosen so that the system (15.113) restricted to the sliding surface is asymptotically stable—see Section 10.5 for a sliding surface design algorithm. Our objective is to construct a fuzzy tracking controller that forces the tracking error to converge to the sliding surface, {σ(e) = 0}, and then stay in some neighborhood of the sliding surface for all subsequent time. If f and g were known exactly to us and η = 0, then the controller

u^{∗} = \frac{1}{g} (−f − μσ + x_{d}^{(n)} − k^{⊤}e) ,  (15.125)

where k = [ 0 s_{1} ⋯ s_{n−1} ]^{⊤} and μ > 0 is a design parameter, would force the tracking error, e, to behave as desired. We can verify this using standard arguments from sliding mode control practice—see Section 10.2. We consider the function V = 0.5σ^{2}. Note that V is positive definite with respect to the sliding surface {σ = 0}. The function V can be viewed as a distance measure to the sliding surface {σ = 0}. The time derivative of V evaluated on the trajectories of the closed-loop system (15.113), (15.125) is

V̇ = σσ̇
= σsė
= σ (e^{(n)} + s_{n−1}e^{(n−1)} + ⋯ + s_{1}ė)
= σ (x^{(n)} − x_{d}^{(n)} + k^{⊤}e)
= σ (f + gu − x_{d}^{(n)} + k^{⊤}e)  (15.126)

Substituting into the equation above the control law given by (15.125) yields

V̇ = −μσ^{2}.  (15.127)

Thus, the sliding surface, {σ = 0}, is asymptotically attractive and the system restricted to the sliding surface can be made asymptotically stable with respect to the origin by an appropriate choice of the parameters, s_{i}, of the sliding surface.

The control law (15.125) requires perfect knowledge of the plant’s model. However, the plant model components f and g are unknown to us. In the following subsection, we describe how to use adaptive fuzzy logic controllers to approximate, as closely as possible, the performance of (15.125). We consider two cases. In the first case, the function f = f(x) is unknown to us while g = g(x) > 0 is known to us. In the second case, both f and g are unknown to us. In both cases, we consider the presence of a bounded disturbance modeled by η.

15.7.3 Controllers

Unknown f(x)

When in our plant model (15.113) only g(x) is known, η ≠ 0, and f(x) is unknown, we propose to approximate f(x) with fuzzy logic system of the form θ_{f}^{⊤}ξ_{f}(x), where the functions θ_{f} and ξ_{f} are described after defining relevant symbols. We use θ_{f}^{∗} to denote an “optimal” vector such that

θ_{f}^{∗} = argmin_{θ_{f}} sup_{x∈Ω} |f(x) − θ_{f}^{⊤}ξ_{f}(x)| ,  (15.128)

where Ω ⊆ ℝ^{n} is a region to which the state x is constrained to reside.

The notation argmin (or arg min) stands for the argument, x^{∗}, that yields the minimal value of the minimized function, f(x). This is well-defined only if the minimal value of f(x) is reached at a single point. Thus, x^{∗} = arg min_{x} f(x) holds if and only if x^{∗} is the unique minimizer of f(x).

We assume that

|f(x) − θ_{f}^{∗}^{⊤}ξ_{f}(x)| ≤ d_{f} for all x ∈ Ω,  (15.129)

where d_{f} > 0 and each element of θ_{f}^{∗} is constant and bounded below and above as follows

\underline{θ}_{fi} ≤ θ_{fi}^{∗} ≤ θ̄_{fi}, for all i = 1, …, m,  (15.130)

or in vector notation,

\underline{θ}_{f} ≤ θ_{f}^{∗} ≤ θ̄_{f}.  (15.131)

We define the adaptation parameter error as

ϕ_{f} = θ_{f} − θ_{f}^{∗}.  (15.132)

We use the adaptation law

ϕ̇_{f}(t) = θ̇_{f}(t) = P roj_{θ_{f}} (γ_{f}σξ_{f}(t)),  (15.133)

where γ_{f} > 0 is a design parameter. It is easy to verify, using the definition of P roj and the fact that \underline{θ}_{fi} ≤ θ_{fi}^{∗} ≤ θ̄_{fi} for each i, that

ϕ_{f}^{⊤} (\frac{1}{γ_{f}} P roj_{θ_{f}} (γ_{f}σξ_{f}) − σξ_{f}) ≤ 0.  (15.134)

We analyze the following fuzzy adaptive control law,

u = \frac{1}{g} (−θ_{f}^{⊤}ξ_{f} − μσ + x_{d}^{(n)} − k^{⊤}e + u_{s})  (15.135)

with u_{s} satisfying

σ (f − θ_{f}^{⊤}ξ_{f} + u_{s} + η) ≤ ϵ,  (15.136)
σu_{s} ≤ 0,  (15.137)

where ϵ > 0 is a design parameter. We state and prove a theorem concerning the dynamical behavior of the closed-loop system driven by the control law (15.135).

Theorem 15.8 For the closed-loop system,

\begin{matrix}x^{(n)} & = & f + gu + η \\ u & = & \frac{1}{g} (−θ_{f}^{⊤}ξ_{f} − μσ + x_{d}^{(n)} − k^{⊤}e + u_{s}) \\ θ̇_{f} & = & P roj_{θ_{f}} (γ_{f}σξ_{f}) \end{matrix} \}  (15.138)

where u_{s} satisfies (15.136) and (15.137), we have:

(i)

σ^{2}(t) ≤ e^{−2μt}σ^{2}(0) + \frac{ϵ}{μ};  (15.139)

(ii) if η = 0 and there exists θ_{f}^{∗} such that f(x) = θ_{f}^{∗}^{⊤}ξ_{f}(x), then the origin of the augmented (σ, ϕ_{f})-space is stable, and hence σ(t) and ϕ_{f}(t) are bounded for all t ≥ 0 and e(t) → 0 as t → ∞.

Proof To prove (i), consider V = 0.5σ^{2}. The time derivative of V evaluated on the solutions of the closed-loop system (15.113), (15.135) is

V̇ = σσ̇
= −μσ^{2} + σ (f − θ_{f}^{⊤}ξ_{f} + u_{s} + η) .

Taking into account (15.136) yields

V̇ ≤ −μσ^{2} + ϵ = −2μV + ϵ.

Applying to the above the comparison theorem, presented in Subsection A.14.2, we obtain

V (t) ≤ e^{−2μt}V (0) + \frac{ϵ}{2μ} (1 − e^{−2μt})
≤ e^{−2μt}V (0) + \frac{ϵ}{2μ}.

Noting in the above that V = 0.5σ^{2}, we obtain (15.139), which proves the first part of the theorem.

To prove (ii), consider the following Lyapunov function candidate,

V = \frac{1}{2} (σ^{2} + \frac{1}{γ_{f}} ϕ_{f}^{⊤}ϕ_{f}) .  (15.140)

We evaluate the time derivative of the above Lyapunov function candidate on the solutions of the closed-loop system (15.138), where η = 0. We obtain,

V̇ = σ (−μσ + f − θ_{f}^{⊤}ξ_{f} + u_{s}) + \frac{1}{γ_{f}} ϕ_{f}^{⊤}ϕ̇_{f}.

Taking into account the assumption that f = θ_{f}^{∗}^{⊤}ξ_{f} and using (15.132), (15.133), (15.134) and (15.137) gives

V̇ = −μσ^{2} + σu_{s} − (θ_{f}^{⊤}ξ_{f} − θ_{f}^{∗T} ξ_{f}) σ + \frac{1}{γ_{f}} ϕ_{f}^{⊤}ϕ̇_{f}
= −μσ^{2} + σu_{s} − ϕ_{f}^{⊤}ξ_{f}σ + ϕ_{f}^{⊤} \frac{1}{γ_{f}} P roj_{θ_{f}} (γ_{f}σξ_{f})
≤ −μσ^{2} + σu_{s}
≤ −μσ^{2}.  (15.141)

Thus, V̇ ≤ 0 in the [ σ ϕ_{f} ]^{⊤}-space, which implies that the zero solution of the closed-loop system (15.138) is stable, and therefore, σ(t) and ϕ_{f}(t) are bounded for t ≥ 0.

To show that σ(t) → 0 as t → ∞, we integrate (15.141) from 0 to t to obtain

∫_{0}^{t} V̇ (τ)dτ = V (t) − V (0) ≤ ∫_{0}^{t} (−μσ^{2}) dτ,  (15.142)

or equivalently,

∫_{0}^{t} μσ^{2}dτ ≤ V (0) − V (t) ≤ V (0).  (15.143)

Thus, lim_{t→∞} ∫_{0}^{t} σ^{2}(τ)dτ exists and is finite. It follows from the first part of the theorem that σ^{2} is bounded and because

\frac{d}{dt}σ^{2} = 2σσ̇ ≤ −μσ^{2} + ϵ,  (15.144)

we conclude that \frac{d}{dt}σ^{2} is also bounded. Therefore, σ^{2} is uniformly continuous. By Barbalat’s lemma, stated on page 395, σ^{2} → 0 as t → ∞. This implies that the tracking error, e(t) → 0 as t → ∞, which concludes the proof of the theorem.

□

Selecting u_{s} in control law (15.135)

We discuss a method that can be used to select the component u_{s} while implementing the control law (15.135)). Let h be a constant such that

h ≥ ‖ξ_{f} ‖ ‖θ̄_{f} − \underline{θ}_{f} ‖ .

Given the design parameter ϵ, choose positive numbers ϵ_{1}, ϵ_{2}, and ϵ_{3} such that ϵ = ϵ_{1}+ϵ_{2}+ϵ_{3}. Then, u_{s} can be chosen as

u_{s} = −k_{s}σ,  (15.145)

where

k_{s} ≥ \frac{d_{f}^{2}}{4ϵ_{1}} + \frac{h^{2}}{4ϵ_{2}} + \frac{d^{2}}{4ϵ_{3}} .  (15.146)

We verify that the above choice of u_{s} guarantees that (15.145) satisfies (15.136) and (15.137). We first consider the left-hand side of (15.136). Adding and subtracting θ_{f}^{∗}^{⊤}ξ_{f} and using (15.132) yields

σ(f − θ_{f}^{⊤}ξ_{f} + u_{s} + η) = σ(f − θ_{f}^{∗}^{⊤}ξ_{f} + θ_{f}^{∗}^{⊤}ξ_{f} − θ_{f}^{⊤}ξ_{f} + u_{s} + η)
= σ(f − θ_{f}^{∗}^{⊤}ξ_{f} − ϕ_{f}^{⊤}ξ_{f} − k_{s}σ + η).  (15.147)

We use (15.146) and group the terms to obtain

σ(f − θ_{f}^{⊤}ξ_{f} + u_{s} + η) ≤ σ (f − θ_{f}^{∗}^{⊤}ξ_{f} − \frac{d_{f}^{2}}{4ϵ_{1}} σ)
−σ (ϕ_{f}^{⊤}ξ_{f} + \frac{h^{2}}{4ϵ_{2}} σ) + σ (η − \frac{d^{2}}{4ϵ_{3}} σ) . (15.148)

Next, we consider each term on the right hand side of the above inequality separately. We start with the first term. Completing the squares, we obtain

σ (f − θ_{f}^{∗}^{⊤}ξ_{f} − \frac{d_{f}^{2}}{4ϵ_{1}} σ) = − (\frac{d_{f}}{2\sqrt{ϵ_{1}}} σ − \frac{f − θ_{f}^{∗}^{⊤}ξ_{f}}{d_{f}/\sqrt{ϵ_{1}}} )^{2} + (\frac{f − θ_{f}^{∗}^{⊤}ξ_{f}}{d_{f}/\sqrt{ϵ_{1}}} )^{2} .

Neglecting the first term in the above and using the assumption (15.129), we get

σ(f − θ_{f}^{∗}^{⊤}ξ_{f} − \frac{d_{f}^{2}}{4ϵ_{1}} σ) ≤ \frac{d_{f}^{2}}{d_{f}^{2}/ϵ_{1}} ≤ ϵ_{1}.  (15.149)

Completing the squares in the second term on the right-hand side of (15.148) gives

−σ (ϕ_{f}^{⊤}ξ_{f} + \frac{h^{2}}{4ϵ_{2}} σ) = − (\frac{h}{2\sqrt{ϵ_{2}}} σ + \frac{ϕ_{f}^{⊤}ξ_{f}}{h/\sqrt{ϵ_{2}}} )^{2} + (\frac{ϕ_{f}^{⊤}ξ_{f}}{h/\sqrt{ϵ_{2}}} )^{2}
≤ (\frac{ϕ_{f}^{⊤}ξ_{f}}{h/\sqrt{ϵ_{2}}} )^{2}
= (\frac{(θ_{f} − θ_{f}^{∗})^{⊤}ξ}{h} )^{2} ϵ_{2}
≤ \frac{‖θ_{f} − θ_{f}^{∗} ‖^{2} ‖ξ_{f} ‖^{2}}{h^{2}} ϵ_{2}
≤ \begin{matrix}‖θ_{f} − θ_{f}^{∗} ‖ & 2 & ‖ξ_{f} ‖ & 2 \\ ‖ξ_{f} ‖ & 2 & ‖θ̄_{f} − \underline{θ}_{f} ‖ & 2 \end{matrix} ϵ_{2}  (15.150)
≤ ϵ_{2}.  (15.151)

Completing the squares in the third term on the right-hand side of (15.148) yields

σ (η(t) − \frac{d^{2}}{4ϵ_{3}} σ) = − (\frac{d}{2\sqrt{ϵ_{3}}} σ − \frac{η}{d/\sqrt{ϵ_{3}}} )^{2} + \frac{η^{2}}{d^{2}} ϵ_{3}
≤ ϵ_{3}.

Thus,

σ (f − θ_{f}^{⊤}ξ_{f} + u_{s} + η) ≤ ϵ_{1} + ϵ_{2} + ϵ_{3} = ϵ.

and hence (15.136) is satisfied.

Also, u_{s} = −k_{s}σ, k_{s} > 0, satisfies (15.137).

Unknown f(x) and g(x)

We assume that g(x) and f(x) are unknown to us and η ≠ 0. We approximate f(x) and g(x) with fuzzy logic systems, θ_{f}^{⊤}ξ_{f}(x) and θ_{g}^{⊤}ξ_{g}(x), respectively. The conditions on θ_{f}^{⊤}ξ_{f}(x) are the same as in the previous case, and the conditions on θ_{g}^{⊤}ξ_{g}(x) are similar, that is, the optimal parameter vector, θ_{g}^{∗}, satisfies

θ_{g}^{∗} = argmin_{θ_{g}}sup_{x∈Ω} |g(x) − θ_{g}^{⊤}ξ_{g}(x)| .  (15.152)

We assume that

|g(x) − θ_{g}^{∗}^{⊤}ξ(x)| ≤ d_{g}, for all x ∈ Ω,  (15.153)

where

\underline{θ}_{gi} ≤ θ_{gi}^{∗} ≤ θ̄_{gi}, for all i = 1, ⋯, m.  (15.154)

The parameter error, ϕ_{g} = θ_{g} − θ_{g}^{∗}, is adapted according to the law,

ϕ̇_{g} = θ̇_{g} = P roj_{θ_{g}}(γ_{g}σξ_{g}u_{a}),

where

u_{a} = \frac{1}{θ_{g}^{⊤}ξ_{g}} (−θ_{f}^{⊤}ξ_{f} + x_{d}^{(n)} − k^{⊤}e) .  (15.155)

The initial values of the components of θ_{g} are chosen to satisfy (15.154), where for each i = 1, …, m, we have θ_{gi} > 0. This ensures that θ_{g}^{⊤}ξ_{g} > 0 because we choose fuzzy sets so that at any t at least one rule fires and so at least one component of ξ is nonzero and in fact positive. Using the definition of P roj, one can verify that

ϕ_{g}^{⊤} (\frac{1}{γ_{g}} P roj_{θ_{g}} (γ_{g}σξ_{g}u_{a}) − σξ_{g}u_{a}) ≤ 0.  (15.156)

We analyze the following fuzzy adaptive control law,

u = \frac{1}{θ_{g}^{⊤}ξ_{g}} (−θ_{f}^{⊤}ξ_{f} + x_{d}^{(n)} − k^{⊤}e) − \frac{1}{\underline{g}}μσ + u_{s}  (15.157)

with u_{s} satisfying the following conditions,

σ (f − x_{d}^{(n)} + k^{⊤}e + gu_{a} + gu_{s} + η) ≤ ϵ,  (15.158)
σu_{s} ≤ 0,  (15.159)

where ϵ > 0 is a design parameter. Using the definition of u_{a}, given by (15.155), we represent (15.157) as

u = u_{a} − \frac{1}{\underline{g}}μσ + u_{s}.  (15.160)

Theorem 15.9 For the closed-loop system,

\begin{matrix}x^{(n)} & = & f + gu + η \\ u & = & \frac{1}{θ_{g}^{⊤}ξ_{g}} (−θ_{f}^{⊤}ξ_{f} + x_{d}^{(n)} − k^{⊤}e) − \frac{1}{\underline{g}}μσ + u_{s} \\ θ̇_{f} & = & P roj_{θ_{f}} (γ_{f}σξ_{f}) \\ θ̇_{g} & = & P roj_{θ_{g}}(γ_{g}σξ_{g}u_{a}) \end{matrix} \}  (15.161)

where u_{s} satisfies (15.158) and (15.159), we have:

(i)

σ^{2}(t) ≤ e^{−2μt}σ^{2}(0) + \frac{ϵ}{μ};  (15.162)

(ii) if η = 0 and there exist θ_{f}^{∗} and θ_{g}^{∗} such that f(x) = θ_{f}^{∗}^{⊤}ξ_{f}(x) and g(x) = θ_{g}^{∗}^{⊤}ξ_{g}(x), then the origin of the [ σ ϕ_{f} ϕ_{g} ]^{⊤}-space is stable and hence σ(t), ϕ_{f}(t), and ϕ_{g}(t) are bounded and e(t) → 0 as t → ∞.

Proof To prove (i), consider V = 0.5σ^{2}. The time derivative of V evaluated on the solutions of the closed-loop system (15.113), (15.157) is

V̇ = σσ̇
= σ (f + gu + η − x_{d}^{(n)} + k^{⊤}e) .

Substituting into the above the alternative expression for the control law given by (15.160) and applying (15.158) gives

V̇ = σ (f + gu_{a} − \frac{g}{\underline{g}} μσ + gu_{s} + η − x_{d}^{(n)} + k^{⊤}e)
≤ −\frac{g}{\underline{g}} μσ^{2} + ϵ
= −2μV + ϵ.

Invoking the comparison theorem, we obtain (15.162).

To prove (ii), consider the following Lyapunov function candidate,

V = \frac{1}{2} (σ^{2} + \frac{1}{γ_{f}} ϕ_{f}^{⊤}ϕ_{f} + \frac{1}{γ_{g}} ϕ_{g}^{⊤}ϕ_{g}) .  (15.163)

By assumption, f = θ_{f}^{∗}^{⊤}ξ_{f}, g = θ_{g}^{∗}^{⊤}ξ_{g}, η = 0 as well as conditions (15.134), (15.156), and (15.159) hold. In addition, it follows from (15.155) that

x_{d}^{(n)} − k^{⊤}e = θ_{f}^{⊤}ξ_{f} + θ_{g}^{⊤}ξ_{g}u_{a}.

Taking into account all of the above in evaluating the time derivative of V on the trajectories of the closed-loop system (15.161), we obtain

V̇ = σ (−\frac{g}{\underline{g}} μσ + f − θ_{f}^{⊤}ξ_{f} + (g − θ_{g}^{⊤}ξ_{g}) u_{a} + gu_{s}) + \frac{1}{γ_{f}} ϕ_{f}^{⊤}ϕ̇_{f} + \frac{1}{γ_{g}} ϕ_{g}^{⊤}ϕ̇_{g}
= σ (f − θ_{f}^{⊤}ξ_{f} + (g − θ_{g}^{⊤}ξ_{g}) u_{a} + gu_{s}) − \frac{g}{\underline{g}} μσ^{2}

+ϕ_{f}^{⊤} \frac{1}{γ_{f}} P roj_{θ_{f}} (γ_{f}σξ_{f}) + ϕ_{g}^{⊤} \frac{1}{γ_{g}} P roj_{θ_{g}}(γ_{g}σξ_{g}u_{a})

≤ −\frac{g}{\underline{g}} μσ^{2} + σgu_{s} − ϕ_{f}^{⊤}ξ_{f}σ + ϕ_{f}^{⊤} \frac{1}{γ_{f}} P roj_{θ_{f}} (γ_{f}σξ_{f})

−σϕ_{g}^{⊤}ξ_{g}u_{a} + ϕ_{g}^{⊤} \frac{1}{γ_{g}} P roj_{θ_{g}}(γ_{g}σξ_{g}u_{a})

≤ −μσ^{2} + gσu_{s}
≤ −μσ^{2}.

It follows from the above that the closed-loop system (15.161) is stable, and therefore, σ(t), ϕ_{f}(t) and ϕ_{g}(t) are bounded for t ≥ 0.

Applying Barbalat’s lemma, similarly the way we did in the proof of Theorem 15.8, we conclude that σ(t) → 0 as t → ∞, which completes the proof of the theorem.

□

Selecting u_{s} in control law (15.157)

The component u_{s} appearing in the control law (15.157) can be chosen similarly as in the previous case. We now give a lower bound on the gain k_{s}. Let

h_{f} ≥ ‖ξ_{f} ‖ ‖θ̄_{f} − \underline{θ}_{f} ‖ and h_{g} ≥ ‖ξ_{g}‖ ‖θ̄_{g} − \underline{θ}_{g}‖ |u_{a}| .

Given the design parameter ϵ, select positive numbers ϵ_{1}, ϵ_{2}, ϵ_{3}, and ϵ_{4} such that ϵ = ϵ_{1} + ϵ_{2} + ϵ_{3} + ϵ_{4}. Then, k_{s} should satisfy the following condition,

k_{s} ≥ \frac{1}{\underline{g}} (\frac{d_{f}^{2} + |u_{a}|^{2} d_{g}^{2}}{2ϵ_{1}} + \frac{h_{f}^{2}}{4ϵ_{2}} + \frac{h_{g}^{2}}{4ϵ_{3}} + \frac{d^{2}}{4ϵ_{4}} ) .  (15.164)

When the gain k_{s} satisfies the above condition, u_{s} = −k_{s}σ guarantees that (15.158) and (15.159) are satisfied. To show that this is indeed the case, consider the left-hand side of (15.158). Use θ_{g}^{⊤}ξ_{g}u_{a} + θ_{f}^{⊤}ξ_{f} = x_{d}^{(n)} − k^{⊤}e from (15.155), and then add and subtract θ_{f}^{∗}^{⊤}ξ_{f} and θ_{g}^{∗}^{⊤}ξ_{g}u_{a} to obtain

σ (f − x_{d}^{(n)} + k^{⊤}e + gu_{a} + gu_{s} + η) = σ (f − θ_{f}^{⊤}ξ_{f} − θ_{g}^{⊤}ξ_{g}u_{a} + gu_{a} + gu_{s} + η)
= σ (f − θ_{f}^{∗}^{⊤}ξ_{f} − ϕ_{f}^{⊤}ξ_{f} + (g − θ_{g}^{∗}^{⊤}ξ_{g})u_{a}
−ϕ_{g}^{⊤}ξ_{g}u_{a} + gu_{s} + η) .

Taking into account (15.164) in the above and rearranging terms gives

σ (f − x_{d}^{(n)} + k^{⊤}e + gu_{a} + gu_{s} + η) ≤ σ (f − θ_{f}^{⊤}ξ_{f} − \frac{d_{f}^{2}}{2ϵ_{1}} σ)

+σ ((g − θ_{g}^{∗}^{⊤}ξ_{g}) u_{a} − \frac{|u_{a}|^{2} d_{g}^{2}}{2ϵ_{1}} σ)

−σ (ϕ_{f}^{⊤}ξ_{f} + \frac{h_{f}^{2}}{4ϵ_{2}} σ) − σ (ϕ_{g}^{⊤}ξ_{g}u_{a} + \frac{h_{g}^{2}}{4ϵ_{3}} σ)

+σ (η − \frac{d^{2}}{4ϵ_{4}} σ) .

Completing the squares, similarly as in the previous case, yields

σ (f − x_{d}^{(n)} + k^{⊤}e + gu_{a} + gu_{s} + η) ≤ (\frac{f − θ_{f}^{⊤}ξ_{f}}{\sqrt{2}d_{f}/\sqrt{ϵ_{1}}} )^{2} + (\frac{(g − θ_{g}^{∗}^{⊤}ξ_{g})u_{a}}{\sqrt{2} |u_{a}| d_{g}/\sqrt{ϵ_{1}}} )^{2}

+ (\frac{ϕ_{f}^{⊤}ξ_{f}}{h_{f}^{2}/\sqrt{ϵ_{2}}} )^{2} + (\frac{ϕ_{g}^{⊤}ξ_{g}u_{a}}{h_{g}^{2}/\sqrt{ϵ_{3}}} )^{2} + (\frac{η}{d/\sqrt{ϵ_{4}}} )^{2}

≤ \frac{1}{2}ϵ_{1} + \frac{1}{2}ϵ_{1} + ϵ_{2} + ϵ_{3} + ϵ_{4}
= ϵ.

Hence, (15.158) is satisfied, and so is (15.159) by the fact that u_{s} = −k_{s}σ, where k_{s} > 0.

15.7.4 Examples

In this Subsection, we present simulation results illustrating the performance of the control strategies discussed above. The controllers are tested on a simplified model of an electric drive system used by Fischle and Schröder [104] to test their fuzzy adaptive controllers. The plant is modeled by the differential equation,

ẍ = −\frac{5}{J} arctan(5ẋ) + \frac{c_{T}}{J} u = f(x, ẋ) + gu,  (15.165)

Figure 15.32: Fuzzy sets for x and ẋ.

where x is the angular position, c_{T} =10 Nm/A, and J = 0.1 kg·m^{2}. We generate a reference signal x_{d} using the reference system whose transfer function is given by \frac{400}{s^{2}+40s+400}. The input signal, w(t), to the reference system is changing its value randomly between -1.5 and 1.5 every 0.5 seconds.

Example 15.11 In this example, we assume that we know g, where g = 100, and that there are no disturbances affecting the system, that is, η = 0. We use the control law given by (15.135), where f(x) is approximated by a fuzzy logic system. We use fuzzy sets for x and ẋ as in Fischle and Schröder [104]. They are shown in Figure 15.32. There are two fuzzy sets for x and six fuzzy sets for ẋ. Thus, we can have twelve fuzzy rules possible. They have the form:

Rule 1: IF x is N AND ẋ is LN THEN y = θ_{1} ⋮ Rule 12: IF x is P AND ẋ is LP THEN y = θ_{12},

where θ_{fi}, i = 1, …, 12 are the adaptation parameters. The bounds on the components of the adaptation parameter vector, θ_{f}, are chosen to be θ̄_{fi} = 200 and \underline{θ}_{fi} = −200. All the Figure 15.33: SIMULINK block diagram of fuzzy adaptive robust control system of Example 15.11.

Figure 15.34: Parameter adaptation block of Example 15.11.

initial values of the components of θ_{f} are set to zero. We chose d_{f} = 50, d_{g} = 10, and γ_{f} = 5000. The values of the remaining design parameters are: μ = 250, ϵ_{1} = 50, ϵ_{2} = 50. As η(t) = 0, we do not need to worry about ϵ_{3}.

SIMULINK is used to simulate the closed-loop system. Its block diagram is depicted in Figure 15.33. The regressor generator block produces ξ using the input x and the parameter adaptation block updates θ_{f}. It is made up of sets of integrators as shown in Figure 15.34. Adaptation rates of the parameters θ_{fi} are computed with the MATLAB function block, depicted in Figure 15.34, using the m-file shown in Figure 15.11. The vector s that defines the sliding surface, {e : se = 0}, was chosen as s = [ 40 1 ]. The simulation results are shown in Figures 15.36 and 15.37. The output x(t), the reference signal x_{d}(t), and the tracking error e(t) are shown in Figure 15.36. The tracking error, e(t), is so small that one cannot distinguish between the actual state x(t) and the desired state x_{d}(t). The plots of the control signal u and the components of θ_{f} versus time are depicted in Figure 15.37. As

%FARC1f_UPDATE.M calculates adapation rates
% (needs 'farc1.mdl' 'farc1_fuzzifier.m', 'af_input_gen.m')
function rate = farc1f_update(x)
% x(1)--x(12) are components of xi,
% x(13) is sigma,  x(14)--x(25) are components of theta
rate = zeros(12,1);
r = 5000;               %adaptation gain(learning rate)
theta_lb = -200;
theta_ub =  200;
for i = 1:12;
   alpha = r*x(13)*x(i);
   rate(i) = alpha*(((x(i+13)>theta_lb)|(alpha>0))&((x(i+13)<...
             theta_ub)|(alpha<0)));
end;

Figure 15.35: An m-file used in the block diagram of Figure 15.34 to compute adaptation rates.

Figure 15.36: Plots of x(t) and error e(t) in Example 15.11.

the fuzzy logic system adapts the parameter θ_{f}, the error, e(t), gets smaller and smaller. If we used smaller ϵ, we would achieve even smaller tracking error, at the expense of higher control effort.

Example 15.12 In this example, we assume that f(x) and g are unknown to us and we employ the control law (15.157). Unknown f(x) and g are approximated by two separate fuzzy logic systems using fuzzy sets shown in Figure 15.32 for both fuzzy logic systems.

The bounds on the adaptation parameters are chosen as: θ̄_{fi} = 200, \underline{θ}_{fi} = −200, θ̄_{gi} = 150, and \underline{θ}_{gi} = 50. All the initial values of the components of the adaptation parameter vector θ_{f} are set to zero. The initial values of θ_{g} are set to 150 to avoid excessively large control signal in the early stages of the simulation run. The selected gains are γ_{f} = 5000 and γ_{g} = 1000. The other design parameters are: ϵ_{1} = 50, ϵ_{2} = 100, ϵ_{3} = 50, and ϵ_{4} = 50. The remaining parameters are the same as in the previous example.

Figure 15.37: Plots of control effort u(t) and the components of the parameter vector θ_{f} versus time in Example 15.11.

Figure 15.38: SIMULINK block diagram of the fuzzy adaptive robust control system of Example 15.12.

786 CHAPTER 15. FUZZY SYSTEMS Figure 15.39: Parameter adaptation block of Example 15.12.

Figure 15.40: Plots of the control effort u(t) and disturbance η(t) versus time in Example 15.12.

SIMULINK block diagrams of the closed-loop system and the parameter adaptation law are depicted in Figure 15.38 and Figure 15.39, respectively. The disturbance, η, is a random signal, whose plot versus time is shown in Figure 15.40. We used d = 100 so that the condition, |η(t)| ≤ d, is satisfied. We chose, as before, s = [ 40 1 ]. The simulation results are shown in Figures 15.41, 15.40, and 15.42. As can be seen in Figure 15.41, the tracking error, e(t), remains very small even in the presence of the disturbance, η. Plots of the time history of the adaptation parameters are shown in Figure 15.42.

15.8 Notes

The reader is encouraged to peruse a text on classical logic to become familiar with basic concepts of logic before undertaking his or her further study of fuzzy logic. An excellent 788 CHAPTER 15. FUZZY SYSTEMS

Figure 15.41: Plots of x(t) and error e(t) versus time in Example 15.12.

15.8. NOTES 789 Figure 15.42: Plots of the components of the parameter vectors θ_{f} and θ_{g} versus time in Example 15.12 introduction to logic is Copi [66]. For an introduction to fuzzy set theory the reader is referred to Klir, St.Clair, and Yuan [182], while for an introduction to fuzzy control, see Driankov, Hellendoorn, and Reinfrank [86]. Kosko [187] is the first book on fuzzy engineering. Graduate texts on fuzzy systems and control are Pedrycz [268] and Wang [345]. Paradigms and practice of fuzzy modeling are the topics of the edited book by Pedrycz [269]. Lee and Lin [215], and Jang, Sun, and Mizutani [161] cover applications of fuzzy systems as well as issues of merging fuzzy logic and neural networks. Mendel [239], on the other hand, presents extensions of the original fuzzy logic to deal with uncertainties.

The first chapter of Kosko’s book [187] contains a thrilling account of the beginnings of fuzzy logic and fuzzy engineering. Kosko [187, page 5] observes that “Fuzziness began as vagueness in the late nineteenth century.” A concept is vague when it has blurred boundaries. For example, the concept mountain is vague because we do not know where a mountain ends and a hill begins. It was Bertrand Russell who in 1923 published a paper in which he described the concept of vagueness in terms of symbolic logic. In the 1920s, Polish mathematician Jan Łukasiewicz invented the multivalued logic. Max Black applied multivalued logic to sets of objects in the 1930s. In the 1950s, a number of logicians worked out a vague set algebra. In his 1965 paper, Zadeh [364] used the term fuzzy to mean vague. According to Kosko [187, page 8], “Zadeh’s 1965 paper applied Łukasiewicz’s logic to each object in a set to work out a complete fuzzy set algebra and to extend the convex separation theorem to pattern recognition.” The beginning of fuzzy engineering, on the other hand, can be marked by the work of Mamdani and Assilian [228] who first used fuzzy calculus to construct a fuzzy logic controller.

The method of constructing fuzzy design models described in Section 15.4 as well as the stabilizability conditions of fuzzy models along with a Lyapunov based stabilizer were first reported in [326]. Teixeira, Assunção, and Avellar [325] present Lyapunov-based design methods of fuzzy regulators and observers using LMIs. The controllers analyzed in Section 15.7 come from Lee et al. [206]. Other fuzzy adaptive control strategies were developed by Wang [343] and by Fischle and Schröder [104]. Fischle and Schröder [104, page 38] observe that a major limitation of all existing stable adaptive fuzzy controllers, including the ones analyzed by us, is that they were developed for systems with unlimited actuators’ authority, that is, for systems with no constraints on the effort level of the actuators. If we are serious about real-life applications of fuzzy adaptive controllers, then the case of the actuators with limited authority has to be rigorously analyzed. Another issue of practical importance is the convergence of the adaptation parameters to their “correct” values. Friedland [108, page 345] states: “One may argue that the failure of a parameter estimate to converge to its correct value is a subsidiary issue as long as the adaptive control works (even with the incorrect parameter estimate). This reasoning might be valid if you can ascertain that the algorithm works for every input to which the system might be subjected. But in most applications the performance of the algorithm is tested only with a limited set of inputs. Satisfactory performance for this set of inputs is not a reliable indicator that its performance will be satisfactory for inputs outside the test set.” In the current state-of-the-art adaptive control algorithms, the convergence of the adaptation parameters is usually ensured through the so called persistently exciting inputs to the adaptation algorithms. However, the injection of the persistently exciting signals into the adaptation algorithms may not be feasible in real life applications. Thus, an alternative approach to the issues of the convergence of the adaptation parameters is needed.

15.9 Exercises

Exercise 15.1 Consider a process whose input u is a function of two variables e and ė. The membership functions for e, ė and u are given in Table 15.4. The control rules are:

Table 15.4: Membership matrix table for e, ė and u for Exercise 15.1.

Fuzzy  Universe of discourse
Numbers -3 -2 -1  0  1  2  3
LN  1 0.5 0.25 0  0  0  0
SN  0.5 1 0.5 0.25 0  0  0
ZE  0 0.25 0.5 1 0.5 0.25 0
SP  0  0  0 0.25 0.5 1 0.5
LP  0  0  0  0 0.25 0.5 1

1. If e is ZE and ė is SP then u is SN

2. If e is ZE and ė is ZE then u is ZE

3. If e is SN and ė is SN then u is LP

4. If e is SP and ė is ZE then u is SN.

Determine the value of u, for e = 1 and ė = −2, using Table 15.5: The rules matrix for Exercise 15.3.

change-in-error LN N ZE P LP

error

LN  LP LP LP LP LP
N  P P P LP LP
ZE  ZE ZE ZE ZE ZE
P  LN LN N N N
LP  LN LN LN LN LN

(a) the algebraic product defuzzifier;

(b) the center-of-gravity defuzzifier.

Exercise 15.2 Use Lemma 15.3 to prove the following result due to Bailey [18]: Let B be a matrix with negative diagonal elements and non-negative off-diagonal elements. If x(t; x_{0}, t_{0}) and y(t; y_{0}, t_{0}) are solutions of ẋ ≤ Bx and ẏ = By, respectively, and x_{0} = y_{0}, then x(t; x_{0}, t_{0}) ≤ y(t; y_{0}, t_{0}) for all t ≥ t_{0}.

Exercise 15.3 Consider the one-link robot manipulator of Exercise 11.2, whose model is given by (2.83) and (2.84). Let

P = s^{⊤} (ss^{⊤})^{−1} s,

where s ∈ ℝ^{1×2} is the row vector that was found in Exercise 11.2. Then, (I_{n} − P) is the orthogonal projector of x onto the sliding surface {x : sx = 0}. The Euclidean distance of the point x to the surface {x : sx = 0} is

∥P x∥_{2}.

Design a fuzzy sliding mode controller whose inputs are:

∥P x(t)∥_{2}sign(σ(x(t)) and Δ∥P x(t)∥_{2} = ∥P x(t)∥_{2} − ∥P x(t − T_{d})∥_{2},

where T_{d} is the time delay that is a design parameter. Use the rules matrix given in Table 15.5.

Exercise 15.4 Consider a dynamical system modeled by (15.6)–(15.8). Let

α_{j}(t) = 1 − ∑_{{i=1 \atop i≠j}}^{r} α_{i}(t).  (15.166)

Represent (15.6) as

ẋ = (α_{j}A_{j} + ∑_{{i=1 \atop i≠j}}^{r} α_{i}A_{i}) x + (α_{j}B_{j} + ∑_{{i=1 \atop i≠j}}^{r} α_{i}B_{i}) u

= (A_{j} + ∑_{{i=1 \atop i≠j}}^{r} α_{i} (A_{i} − A_{j})) x + (B_{j} + ∑_{{i=1 \atop i≠j}}^{r} α_{i} (B_{i} − B_{j})) u. (15.167)

Let

α^{j}(t) = α^{j} = [ α_{1} ⋯ α_{j−1} α_{j+1} ⋯ α_{r} ]^{⊤} .

Observe that α^{j} ∈ [0, 1]^{r−1}, that is, the values of the vector-valued function α^{j}(·) are in the (r − 1)-dimensional unit hypercube. Next, let

ΔA(α^{j}) = ∑_{{i=1 \atop i≠j}}^{r} α_{i} (A_{i} − A_{j}) and ΔB(α^{j}) = ∑_{{i=1 \atop i≠j}}^{r} α_{i} (B_{i} − B_{j}) .

Using the above notation, represent model (15.167), in an equivalent way, as

ẋ = (A_{j} + ΔA(α^{j})) x + (B_{j} + ΔB(α^{j})) u
= A_{j}x + B_{j}u + F (α^{j}, x, u),  (15.168)

where F (α^{j}, x, u) = ΔA(α^{j})x + ΔB(α^{j})u. Let

F (·) = f(·) + B_{j}h(·).  (15.169)

We refer to f as the unmatched uncertainty, while h is referred to as the matched uncertainty. Hence, (15.168) takes the form

ẋ = A_{j}x + B_{j}u + f + B_{j}h.  (15.170)

Assume that

\{ \begin{matrix}∥f∥ & = & ∥f(α^{j}, x)∥ ≤ α_{f}∥x∥ \\ ∥h∥ & = & ∥h(α^{j}, x, u)∥ ≤ β_{x}∥x∥ + β_{u}∥u∥, \end{matrix}  (15.171)

Prove the following theorem:

Theorem 15.10 Suppose that A_{j} is asymptotically stable, and P = P^{⊤} > 0 is the solution to the Lyapunov matrix equation A_{j}^{⊤}P + P A_{j} = −2Q for some Q = Q^{⊤} > 0. Suppose also that

α_{f} < \frac{λ_{min}(Q)}{λ_{max}(P)} and β_{u} < 1.

Then, the state feedback controller u = −γB_{j}^{⊤}P x, where

γ > \frac{β_{x}^{2}}{4 (λ_{min}(Q) − α_{f}λ_{max}(P)) (1 − β_{u})}

globally asymptotically stabilizes the uncertain system for arbitrary f and h that satisfy the norm bounds (15.171).

Exercise 15.5 In this Exercise, the following definition is used:

Definition 15.3 We say that system (15.168) satisfies the matching conditions if there exist continuous matrix functions D(·) : ℝ^{r−1} → ℝ^{m×n} and E(·) : ℝ^{r−1} → ℝ^{m×m} such that

\{ \begin{matrix}ΔA(α^{j}) & = & B_{j}D(α^{j}) \\ ΔB(α^{j}) & = & B_{j}E(α^{j}) \end{matrix}

for all α^{j} ∈ [0, 1]^{r−1}.

◇

Consider the following algorithm for linear controller design for fuzzy system models of the form (15.167) that is the subject of Exercise 15.4. The algorithm is a consequence of Theorem 15.10 of Exercise 15.4.

Algorithm 15.1 Linear Controller Design for Fuzzy Systems Given fuzzy model (15.167)

1. Set j = 1;

2. Check matching conditions (15.172). If the matching conditions are satisfied, calculate matrices D and E, and check if α_{u} < 1. If the matching conditions are not satisfied, express the uncertain element as F = f + B_{j}h, calculate β_{u} and check if β_{u} < 1. If β_{u} > 1, set j = j + 1 and repeat the above computations.

3. Stabilize A_{j}, if necessary. Calculate u_{1} = −K_{1}x so that the matrix A_{j} − B_{j}K_{1} has its eigenvalues in the desired locations. Update the matched uncertainty involving the state vector, that is, replace D with D − EK_{1};

4. Solve the Lyapunov equation A_{j}^{⊤}P + P A_{j} = −2Q for some Q = Q^{⊤} > 0. Choosing Q = I_{n} maximizes the ratio λ_{min}(Q)/λ_{max}(P). Check if α_{f} < λ_{min}(Q)/λ_{max}(P). If α_{f} does not satisfy the condition, set j = j + 1 and go to step 2; 5. Construct u_{2} = −γB_{j}^{⊤}P x;

6. Construct the resulting state feedback linear control law u = u_{1} + u_{2}.

Illustrate the above algorithm on a reduced-order model of an inverted pendulum mounted on a cart. The equations of motion for the pendulum are

\{ \begin{matrix}ẋ_{1} & = & x_{2} \\ ẋ_{2} & = & \frac{g sin(x_{1})−amlx_{2}^{2} sin(2x_{1})/2−a cos(x_{1})u}{4l/3−aml cos^{2}(x_{1})}, \end{matrix}  (15.172)

where x_{1} is the angle of the pendulum from the vertical line, x_{2} is the angular velocity of the pendulum, u is the control force applied to the cart, and g = 9.8 m/sec^{2} is the magnitude of the acceleration due to gravity. Numerical values of the parameters are given in Table 15.6. Consider modeling equations (15.172) as a truth model. Use the truth model

Table 15.6: Parameter numerical values of the model of Exercise 15.5.

m=mass of pendulum 2.0 kg
M=mass of cart  8.0 kg
2l=length of pendulum 1.0 m
a = 1/(m + M)

in your simulations to evaluate the performance of the design. Design a linear stabilizing controller using the above algorithm utilizing a design model. Assume that, by using insight and experience, you obtained the following two rules describing the plant dynamics:

Rule 1 : IF x_{1}(t) is about 0

THEN ẋ(t) = A_{1}x(t) + b_{1}u(t),

and

Rule 2 : IF x_{1}(t) is about ± π/2
THEN ẋ(t) = A_{2}x(t) + b_{2}u(t),

where

A_{1} = [ \begin{matrix}0 & 1 \\ \frac{g}{4l/3−aml} & 0 \end{matrix} ] , b_{1} = [ \begin{matrix}0 \\ −\frac{a}{4l/3−aml} \end{matrix} ] ,

and

A_{2} = [ \begin{matrix}0 & 1 \\ \frac{2g}{π(4l/3−amlb^{2})} & 0 \end{matrix} ] , b_{2} = [ \begin{matrix}0 \\ −\frac{ab}{4l/3−amlb^{2}} \end{matrix} ] ,

where b = cos(88^{∘}). Use the following membership functions:

μ_{1}(x_{1}) = \frac{1 − 1/ (1 + exp(−7(x_{1} − π/4)))}{1 + exp(−7(x_{1} + π/4))} and μ_{2}(x_{1}) = 1 − μ_{1}(x_{1}).

The above membership functions are the same as the ones employed by Cao et al. [51]. The corresponding fuzzy system model is

ẋ = (α_{1}A_{1} + α_{2}A_{2}) x + (α_{1}b_{1} + α_{2}b_{2}) u,  (15.173)

where α_{i} = μ_{i}, i = 1, 2, because μ_{1} + μ_{2} = 1 for all t. Proceed to construct a stabilizing linear controller according to the above algorithm. First, represent model (15.173) as

ẋ = (A_{1} + α_{2} (A_{2} − A_{1})) x + (b_{1} + α_{2} (b_{2} − b_{1})) u.  (15.174)

Check that the uncertain elements of system model (15.174) satisfy the matching conditions. Suppose that the desired eigenvalues of A_{1} − b_{1}k_{1} are {−2, −2}. Construct the controller, and test its performance on the truth model. Generate plots of x_{1} versus time for different initial angular displacements x_{1}(0). You can set x_{2}(0) = 0.

Exercise 15.6 The objective of this exercise is to understand better the dynamical behavior of the nonlinear projector used by us in adaptive fuzzy control and defined by (15.122). For example, take α(t) = sin(0.5πt), \underline{θ} = −0.3, and θ̄ = 0.5. Compare solutions of θ̇ = P roj_{θ}(α) and θ̇ = α for a number of initial conditions. Below is an example of MATLAB’s function that implements the differential equation, θ̇ = P roj_{θ}(α), where low= \underline{θ} and upper= θ̄.

function theta_dot = Proj(t,theta)
low=-0.3;
upper=0.5;
alpha=sin(0.5*pi*t);
if (theta <= low)&(alpha<0)
   theta_dot=0;
elseif (theta >= upper)&(alpha>0)
   theta_dot=0;
else
   theta_dot=alpha;
   end

Exercise 15.7 For the fuzzy model the stick balancer given in Example 15.8 by equation (15.67), where

y = [ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \end{matrix} ] ,

construct a fuzzy observer. (Note that we have C_{1} = C_{2} = C(α).) Implement the combined controller-observer compensator using first the fuzzy model and then the nonlinear model of the stick balancer. Write MATLAB scripts that animate the dynamical behavior of the closed-loop systems along with animation of the angle θ and the input u data.

798 CHAPTER 15. FUZZY SYSTEMS

Neural Networks

We tend to regard forgetting as a failure of memory, the result of an inadequacy in the machinery for remembering, but a moment’s reflection shows that that cannot always be right. If we were to remember every telephone number we have ever used, every scene we have ever looked at, every conversation we have ever heard, every sentence we have ever read, we should be overwhelmed with memories. In fact we transfer to our long-term memory stores only a tiny fraction of the contents of our short-term stores. We retain only what is likely to be useful. [116, page 331]

The motivation behind studying artificial neural networks, commonly referred to as neural networks, or just neural nets, is to build computers whose construction would mimic the organization of the brain. The brain is the most complex natural information processing system. It is capable of organizing its building blocks, called neurons, to perform computations in a very efficient way. A drawing of a typical biological neuron is given in Figure 16.1. The neuron collects signals from other neurons through its dendrites. The signals have the form of electrical pulses. These pulses travel to the neuron’s cell body where they are being processed. The surface of the cell body of the neuron is dotted with numerous receptor sites that can receive messages from other neurons in a fashion similar to the dendrites. The neuron may produce its own pulses in response to excitation from other neurons. The neuron’s own pulses travel through a long, thin strand called an axon, which splits into many end branches. The neuron’s message to the other neurons is delivered via the end branches. The key to the transmission of the neuron’s message is the synapse, shown in Figure 16.2. The end branches of the neuron’s axon terminate in synaptic knobs. The synaptic knob of the first neuron is separated from the receptor site of another neuron by a microscopic distance.

800 CHAPTER 16. NEURAL NETWORKS Figure 16.1: A typical biological neuron.

Figure 16.2: The synapse is a junction point where neuron meets neuron.

The cell body of the first neuron produces chemical substances called neurotransmitters, which are delivered down to the synaptic vesicles. The neurotransmitters are stored in the synaptic vesicles until the neuron fires and a burst of neurotransmitters is released by the vesicles. The neurotransmitters then flow across the synaptic cleft and act on the second neuron. The neurotransmitters of the first neuron may stimulate or inhibit the second neuron activity. The first neuron has many other synaptic knobs elsewhere, while the second neuron has many other receptor sites that receive messages from many other neurons. Each neuron is receiving messages from hundreds or thousands of other neurons. The neuron sets off the nervous pulse, that is, it fires off its own nervous pulse only when it is properly stimulated. It was observed that, in general, the neuron will not fire in response to a single message. To fire, the neuron needs many messages arriving simultaneously or in quick succession from other neurons. In addition, the stimulating messages to fire must outweigh the messages that inhibit the neuron from firing.

In the following sections, we discuss a few mathematical models of the biological neuron. These models are referred to as the artificial neurons. Then, we analyze neural networks that are composed of the artificial neurons, and finally we discuss applications of the artificial neural networks in the controller design of dynamical systems.

16.1 Threshold Logic Unit

A simple model of a biological neuron is the so called the threshold logic unit (TLU), shown in Figure 16.3. The TLU is also called the perceptron. A TLU is parameterized by a set of weights and a threshold (bias). The summation operation is followed by the sign activation function, which models the cell body of a biological neuron. The dendrites are modeled by the inputs and their strengths are modeled by the weights. The axon corresponds to the output y. Suppose that we are given a TLU with n weights, w_{1}, w_{2}, …, w_{n}, and a threshold t. Given an input vector x^{(i)} = [ x_{1}^{(i)} x_{2}^{(i)} ⋯ x_{n}^{(i)} ]^{⊤} ∈ ℝ^{n}, then the corresponding output of the TLU is

y_{i} = sign (∑_{j=1}^{n} w_{j}x_{j}^{(i)} − t) ,  (16.1)

where

sign(z) = \{ \begin{matrix}−1 & if & z < 0 \\ 1 & if & z ≥ 0. \end{matrix}

We note that sometimes in the literature the sign function is undefined at 0, or it is defined as a point to a set mapping, that is, sign(0) ∈ [−1, 1]. Here, we define, sign(0) = 1. Using Figure 16.3: A threshold logic unit (TLU), also called the perceptron, as a model of a biological neuron.

the scalar product notation, we represent (16.1) as

y_{i} = sign (⟨w, x^{(i)}⟩ − t) .

Note that y_{i} ∈ {−1, 1}. Specifically,

y_{i} = \{ \begin{matrix}−1 & if & ⟨w, x^{(i)}⟩ − t < 0 \\ 1 & if & ⟨w, x^{(i)}⟩ − t ≥ 0 \end{matrix} .  (16.2)

The goal of the TLU (perceptron) is to correctly classify the set of given input vectors x^{(1)}, …, x^{(p)} into one of the two classes, 𝒞_{1} or 𝒞_{2}. In other words, if a given vector, say x^{(i)}, belongs to class 𝒞_{1}, then the corresponding output y_{i} should be +1. On the other hand, if x^{(i)} belongs to class 𝒞_{2} then the TLU’s output should be −1. Thus, we are faced with a two-class pattern classification problem. The TLU’s role is to find a hyperplane that separates two decision regions. The separating hyperplane (decision boundary) has the form

⟨w^{∗}, x⟩ − t^{∗} = ∑_{i=1}^{n} w_{i}^{∗}x_{i} − t^{∗} = 0.  (16.3)

In the case of two variables x_{1} and x_{2} the decision boundary is a straight line. An illustration of such a case is given in Figure 16.4. In this case a point that lies above the boundary line is assigned to class 𝒞_{1}, while a point below the boundary line is assigned to class 𝒞_{2}. We present an iterative algorithm for the decision boundary weights and threshold provided that a decision boundary exists. We assume that we are given p training pairs

(x^{(1)}, d_{1}) , (x^{(2)}, d_{2}) , …, (x^{(p)}, d_{p}) ,

where for i = 1, 2, …, p, the input vectors x^{(i)} ∈ ℝ^{n} and d_{i} ∈ {−1, 1} represent the desired responses. We would like to determine a hyperplane (16.3) such that each input vector lies on a pre-assigned side of the hyperplane. If such a hyperplane exists for the given training set, then the set of input vectors is referred to as linearly separable, and the hyperplane is referred to as a separating hyperplane. If no such hyperplane exists, then the training set is referred to as linearly nonseparable. Finding a separating hyperplane is equivalent to finding a corresponding weight vector and a threshold. If we find such a weight vector and a threshold, then we say that the TLU correctly classifies the training set. Thus, designing a TLU that correctly classifies the given training set amounts to finding a weight vector w^{∗} and a threshold t^{∗} such that for i = 1, 2, …, p, we have

\begin{matrix}⟨w^{∗}, x^{(i)}⟩ − t^{∗} < 0 & if & d_{i} = −1 \\ ⟨w^{∗}, x^{(i)}⟩ − t^{∗} ≥ 0 & if & d_{i} = 1. \end{matrix}

In our discussion, we assume the threshold value, t, to be equal zero. We can achieve this by increasing the dimension of every input vector, x^{(i)}, by augmenting it with an entry equal to −1, and by increasing the dimension of the weight vector to n + 1 by augmenting it with t. Indeed,

y_{i} = sign (⟨w, x^{(i)}⟩ − t) = sign (⟨w̃, x̃^{(i)}⟩) ,

where w̃^{⊤} = [ w_{1} w_{2} ⋯ w_{n} t ] and x̃^{(i)⊤} = [ x_{1}^{(i)} x_{2}^{(i)} ⋯ x_{n}^{(i)} −1 ]. In our further discussion, we assume that the given input vectors have been already augmented. In view of the above, we can formulate the training problem of a TLU as follows:

TLU Training Problem

Given p training pairs (x^{(1)}, d_{1}) , …, (x^{(p)}, d_{p}), where x^{(i)} ∈ ℝ^{n+1} and d_{i} ∈ {−1, 1}. Determine a weight vector w ∈ ℝ^{n+1}, if there exists one, such that for i = 1, …, p,

sign (⟨w, x^{(i)}⟩) = d_{i}.

Assume that the given vectors x^{(1)}, …, x^{(p)} are linearly separable, that is, there exists a weight vector w^{∗} such that

⟨w^{∗}, x^{(i)}⟩ ≥ 0 for every input vector x^{(i)} from class 𝒞_{1}

and

⟨w^{∗}, x^{(i)}⟩ < 0 for every input vector x^{(i)} from class 𝒞_{2}.

We can now formulate the algorithm for adapting the weight vector resulting in a separating hyperplane of a given set of linearly separable points. The idea is very simple and we proceed as follows. If the given vector x^{(i)} is classified correctly, do not update the weights, that is,

w[k + 1] = w[k].

Otherwise,

w[k + 1] = w[k] − x^{(i)} if w[k]^{⊤}x^{(i)} ≥ 0 and x^{(i)} ∈ 𝒞_{2}  (16.4)

and

w[k + 1] = w[k] + x^{(i)} if w(k)^{⊤}x^{(i)} < 0 and x^{(i)} ∈ 𝒞_{1}.  (16.5)

Note that if w[k]^{⊤}x^{(i)} ≥ 0, that is, y(k) = 1 and x^{(i)} ∈ 𝒞_{2}, that is, d_{i} = −1, then the error between the actual TLU output and the desired TLU output is d_{i} − y[k] = −2. Therefore, we can represent the weight update in (16.4) as

w[k + 1] = w[k] + \frac{1}{2}(d_{i} − y[k])x^{(i)}.  (16.6)

Analogously, if w[k]^{⊤}x^{(i)} < 0, that is, y[k] = −1, and x^{(i)} ∈ 𝒞_{1}, then d_{i} − y[k] = 2. Here, as in the previous case, we can represent the weight update in (16.5) as (16.6). We now summarize the above considerations in the form of the following algorithm.

Algorithm 16.1 TLU (Perceptron) Learning

Given a set of training pairs {(x^{(1)}, d_{1}), …, (x^{(i)}, d_{i}), …, (x^{(p)}, d_{p})}.

1. Initialize Weight Vector and Threshold Set the values of the components of the initial weight vector w[0] and threshold t[0] to zero or to small random values.

2. Present New Input Vector and Desired Output Present the new input vector, x^{(i)} and the corresponding desired output d_{i}.

3. Calculate Actual Output

y[k] = sign (⟨w[k], x^{(i)}⟩ − t[k]) .

4. Adapt Weight Vector and Threshold

w[k + 1] = w[k] + \frac{1}{2}(d_{i} − y[k])x^{(i)},
t[k + 1] = t[k] − \frac{1}{2}(d_{i} − y[k]).

Note that the weight vector and threshold are unchanged if the correct decision is made by the TLU.

5. Repeat by Going to Step 2

Example 16.1 We used the above algorithm to generate a separating line for eight training pairs. For i = 1, …, 4 and d_{i} = 1,

[ x^{(1)} x^{(2)} x^{(3)} x^{(4)} ] = [ \begin{matrix}−5 & −4 & 3 & 6 \\ −4 & 6 & 4 & 7 \end{matrix} ] .

The above points are labeled “x” in Figure 16.4. The other four training pairs corresponding to d_{i} = −1, i = 5, …, 8, labeled “o” are

[ x^{(5)} x^{(6)} x^{(7)} x^{(8)} ] = [ \begin{matrix}−6 & 0 & 3 & 8 \\ −8 & −1 & −4 & 3 \end{matrix} ]

The weight vector and threshold were initialized randomly, where

w[0] = [ −0.4860 0.8913 ]^{⊤} and t[0] = 0.7621

The TLU learning algorithm generated a separating line, shown in Figure 16.4, corresponding to the weight vector w^{∗} = [ −5.5140 6.8913 ]^{⊤} and threshold t^{∗} = −2.2379.

Our goal is to prove that the TLU learning algorithm can always find a separating hyperplane for linearly separable input vectors. In our proof, we use some technical results that we present next.

Figure 16.4: An example of separable points in two-dimensional space and a separating line.

Lemma 16.1 Let x^{(1)}, x^{(2)}, …, x^{(p)}, x^{(i)} ∈ ℝ^{n}, be a set of p input vectors, w ∈ ℝ^{n} be a weight vector of an n-input TLU, and t ∈ ℝ its threshold. Then, there exists a threshold t′ such that, for i = 1, 2, …, p,

⟨w, x^{(i)}⟩ − t′ ≠ 0,

and

sign (⟨w, x^{(i)}⟩ − t) = sign (⟨w, x^{(i)}⟩ − t′) .

Proof Assume, without loss of generality, that the input vectors are sorted into two sets {x^{(1)}, …, x^{(l)}} and {x^{(l+1)}, …, x^{(p)}} such that the desired response of the vectors from the first set is d_{i} = −1, while the desired response of the vectors from the second set is d_{i} = 1. Let

δ = min {|⟨w, x^{(i)}⟩ − t| : 1 ≤ i ≤ l} .

It follows from (16.2) that δ > 0. Let t′ = t − δ/2. Then,

⟨w, x^{(i)}⟩ − t′ = ⟨w, x^{(i)}⟩ − t + δ/2 ≠ 0 for i = 1, …, p,

and the output of the TLU with the modified threshold, t′, is the same as the output of the TLU with the old threshold, t.

□

By the above lemma, we can always assume that ⟨w, x^{(i)}⟩ − t′ ≠ 0, for i = 1, …, p and that the output of a TLU can be defined as

y_{i} = \{ \begin{matrix}−1 & if & ⟨w, x^{(i)}⟩ − t < 0 \\ 1 & if & ⟨w, x^{(i)}⟩ − t > 0. \end{matrix}

We further simplify the formulation of the TLU learning problem by preprocessing training pairs. Specifically, we replace every training pair, (x^{(i)}, d_{i}) with d_{i} = −1 with their negative, (−x^{(i)}, −d_{i}). After performing the above preprocessing, the TLU learning problem can be reformulated in a simple form that we use while analyzing the TLU learning algorithm.

Simplified TLU Learning Problem

Given a set of input vectors {x^{(1)}, …, x^{(p)}}. Determine a weight vector w, if there exists one, such that for i = 1, …, p,

⟨w, x^{(i)}⟩ > 0.

In view of the above, for a linearly separable set of vectors, {x^{(i)}}, after preprocessing there exists a hyperplane passing through the origin in ℝ^{n+1} such that all the vectors lie on one side of the hyperplane. The normal to the hyperplane is the weight vector w. This is illustrated in Figure 16.5. Taking into account the simplified TLU learning problem, we can state the TLU learning algorithm equivalently as follows:

Simplified TLU Learning Algorithm

1. Initialize Weight Vector Set the values of the components of the initial weight vector w[0] to zero.

2. Test If ⟨w[k], x^{(i)}⟩ ≤ 0, go to Add, else Test next input vector.

3. Add

w[k + 1] = w[k] + x^{(i)}, k = k + 1.

Go to Test.

Figure 16.5: Linearly separable vectors after their preprocessing.

The algorithm is said to converge if for some k < ∞,

⟨w[k], x^{(i)}⟩ > 0, for i = 1, …, p.

The following theorem states that if the set of input vectors is linearly separable, then the TLU learning algorithm converges.

Theorem 16.1 (The TLU Convergence Theorem) If the set of vectors {x^{(1)}, …, x^{(p)}} is linearly separable, then the simplified TLU learning algorithm determines a weight vector w in a finite number of steps, such that ⟨w, x^{(i)}⟩ > 0 for i = 1, 2, …, p

Proof The proof is adapted from Siu, Roychowdhury, and Kailath [299]. We begin by observing that because the set of input vectors is linearly separable, there exists a weight vector w^{∗} and a constant δ > 0 such that for i = 1, 2, …, p, we have

⟨w^{∗}, x^{(i)}⟩ > δ.

To simplify further manipulations, we assume that ∥w^{∗}∥ = 1. We also assume that ‖x^{(i)}‖ ≤ L, for i = 1, 2, …, p, that is, the lengths of all the input vectors are bounded above by L. Let k denote the number of times the weight vector w, in the simplified TLU learning algorithm, has been updated, and let w[k] be the value of the weight vector, w, after the k-th update. Our goal is to show that k is bounded above. To proceed, we note that if an input vector x^{(i)} is used for the k-th update in the algorithm, then

w[k] = w[k − 1] + x^{(i)},

where ⟨w[k − 1], x^{(i)}⟩ = w[k − 1]^{⊤}x^{(i)} ≤ 0. We first derive an upper bound on ∥w[k]∥. We have,

∥w[k]∥^{2} = ∥w[k − 1] + x^{(i)}∥^{2}
= (w[k − 1] + x^{(i)})^{⊤} (w[k − 1] + x^{(i)})
= w[k − 1]^{⊤}w[k − 1] + x^{(i)⊤}x^{(i)} + 2w[k − 1]^{⊤}x^{(i)}
≤ ∥w[k − 1]∥^{2} + L^{2},

because w[k − 1]^{⊤}x^{(i)} ≤ 0 and ∥x^{(i)}∥^{2} ≤ L^{2}. Suppose that for the (k − 1)-st update, the input vector x^{(j)} was used, that is,

w[k − 1] = w[k − 2] + x^{(j)}.

Hence,

∥w[k]∥^{2} ≤ ∥w[k − 1]∥^{2} + L^{2}
= ∥w[k − 2] + x^{(j)}∥^{2} + L^{2}
≤ ∥w[k − 2]∥^{2} + 2L^{2}.

Continuing in this manner, we obtain,

∥w[k]∥^{2} ≤ kL^{2},  (16.7)

because by assumption, w(0) = 0.

We next derive a lower bound on ∥w[k]∥. We have,

⟨w^{∗}, w[k]⟩ = ⟨w^{∗}, w[k − 1]⟩ + ⟨w^{∗}, x^{(i)}⟩
≥ ⟨w^{∗}, w[k − 1]⟩ + δ,

because we assumed that ⟨w^{∗}, x^{(i)}⟩ > δ. Observing that w[k − 1] = w[k − 2] + x^{(j)}, we get

⟨w^{∗}, w[k]⟩ ≥ ⟨w^{∗}, w[k − 1]⟩ + δ
≥ ⟨w^{∗}, w[k − 2]⟩ + 2δ.

Continuing in this manner, we obtain

⟨w^{∗}, w[k]⟩ ≥ kδ.  (16.8)

Applying to the above the Cauchy-Schwarz inequality yields

∥w^{∗}∥∥w[k]∥ ≥ ⟨w^{∗}, w[k]⟩ ≥ kδ.

By assumption ∥w^{∗}∥ = 1. Hence,

∥w[k]∥ ≥ kδ.

Combining the obtained upper and lower bounds on ∥w[k]∥^{2}, given by (16.7) and (16.8), we have

k^{2}δ^{2} ≤ ∥w[k]∥^{2} ≤ kL^{2}.

Therefore,

k ≤ \frac{L^{2}}{δ^{2}} ,  (16.9)

which implies that the TLU learning algorithm converges. The proof of the TLU convergence theorem is complete.

□

We now present a sufficient condition for linear nonseparability.

Theorem 16.2 If there exists a positive linear combination (PLC) that equals 0 of the given set of input vectors, then the given set of input vectors is linearly nonseparable.

Proof We prove the theorem by contradiction. Assume that there exists a PLC of the given input vectors that is equal to 0, that is, there exist nonnegative constants, q_{i}, such that for some j, 1 ≤ j ≤ p, we have q_{j} > 0, and

∑_{i=1}^{p} q_{i}x^{(i)} = 0.

Suppose now that the given input vectors are linearly separable, that is, there is a weight vector, w, such that ⟨w, x^{(i)}⟩ > 0 for i = 1, …, p. We premultiply ∑_{i=1}^{p} q_{i}x^{(i)} = 0 by w^{⊤} to get

w^{⊤} ∑_{i=1}^{p} q_{i}x^{(i)} = ∑_{i=1}^{p} q_{i}⟨w, x^{(i)}⟩ > 0

since ⟨w, x^{(i)}⟩ > 0 and q_{j} > 0. However, we arrived at a contradiction because by assumption ∑_{i=1}^{p} q_{i}x^{(i)} = 0, which completes the proof.

□

Example 16.2 Suppose we are given the following training set:

x^{(1)} = [ \begin{matrix}−1 \\ −1 \end{matrix} ] , d_{1} = −1; x^{(2)} = [ \begin{matrix}1 \\ 1 \end{matrix} ] , d_{2} = −1;
x^{(3)} = [ \begin{matrix}−1 \\ 1 \end{matrix} ] , d_{3} = 1; x^{(4)} = [ \begin{matrix}1 \\ −1 \end{matrix} ] , d_{4} = 1.

The preceding training set represents the 2-input Exclusive-OR (XOR) function. Let us now preprocess the training set so we can apply Theorem 16.2. By preprocessing we mean eliminating the threshold by augmenting the input vectors with −1 and then negating the input vectors for which the desired output is −1. The preprocessed input vectors are:

x̃^{(1)} = [ \begin{matrix}1 \\ 1 \\ 1 \end{matrix} ] , x̃^{(2)} = [ \begin{matrix}−1 \\ −1 \\ 1 \end{matrix} ] , x̃^{(3)} = [ \begin{matrix}−1 \\ 1 \\ −1 \end{matrix} ] , x̃^{(4)} = [ \begin{matrix}1 \\ −1 \\ −1 \end{matrix} ] .

Note that ∑_{i=1}^{4} x̃^{(i)} = 0. It follows from Theorem 16.2 that the given input vectors are not linearly separable.

16.2 Backpropagation

In Section 16.1, we discussed an iterative learning algorithm for a threshold logic unit. This computational unit keeps on iterating until it reaches a solution, if a solution exists. In this section, we present iterative learning algorithms for computational units with differentiable activation functions. This family of algorithms are called the backpropagation algorithms because they back propagate the error between the desired and actual output of the neural network for the purpose of adjusting the network’s weights in order to reduce the learning error. Specifically, a typical backpropagation algorithm consists of two passes: the forward pass during which the input vectors are presented to the network and the actual outputs of the network are calculated. Then, the error between the actual and desired outputs are calculated. In the reverse pass, the error gradient with respect to the network’s weights is calculated by propagating the error backwards through the network. Once the error gradient is calculated, the weights are adjusted using, for example, a descent gradient method given by (13.31) in Subsection 13.3.1.

We first analyze a single computational unit shown in Figure 16.6. It is called the perceptron. We consider a simple case, where we train the perceptron to learn the training pair

(x, d) ,

where x ∈ ℝ^{n+1}, x_{n+1} = (x)_{n+1} = −1, and d ∈ [−1, 1]. Define the error function

E(w) = \frac{1}{2} (d − y)^{2}
= \frac{1}{2} (d − f(net))^{2} ,  (16.10)

Figure 16.6: A single perceptron with a differentiable sigmoidal activation function.

where net = x^{⊤}w. Note that w_{n+1} = t, where t is a threshold, bias. The gradient of E, with respect to w, is

∇E(w) = −(d − f(net))f′(net)x.  (16.11)

Using the descent gradient algorithm with a fixed step size to minimize E, we obtain

w[k + 1] = w[k] − α∇E (w[k])
= w[k] + α (d_{k} − f (net_{k})) f′ (net_{k}) x[k],  (16.12)

where net_{k} = x[k]^{⊤}w[k]. Let

δ^{k} = (d_{k} − f (net_{k})) f′ (net_{k}) .  (16.13)

Then, we represent (16.12) as

w[k + 1] = w[k] + αδ^{k}x[k]  (16.14)

We generalize the above formula for a three-layer feedforward neural network shown in Figure 16.7. We wish to train the network to learn the training pairs

(x^{(1)}, d^{(1)}) , (x^{(2)}, d^{(2)}) , …, (x^{(p)}, d^{(p)}) ,

where x^{(i)} ∈ ℝ^{n} and d^{(i)} ∈ ℝ^{m}. The network has n inputs, which is the dimension of the training vectors, and m outputs. The network’s inputs are connected to a layer of hidden

Figure 16.7: A three-layer feedforward neural network.

units, which in turn is connected to a layer of output units. To construct a network to perform some task, we must set the weights of the connections appropriately. The weights specify the strength of the influence of one unit upon another. We derive rules for adjusting the weights w_{ji} and w_{kj} such that the error function

E(w) = \frac{1}{2} ∑_{s=1}^{m} (d_{s} − y_{s})^{2}
= \frac{1}{2} ∑_{s=1}^{m} (d_{s} − f_{o}(net_{s}))^{2}

is minimized, where

net_{s} = ∑_{j=1}^{l} w_{sj}^{o}z_{j}.  (16.15)

The minimization is performed over the space of weights

{w_{ji}^{h}, w_{sj}^{o} : i = 1, 2, …, n, j = 1, 2, …, l, s = 1, 2, …, m} .

We first give formulas for updating the weights connecting the hidden and output layers. Using the descent gradient algorithm in a similar way as in (16.14), we obtain

w_{sj}^{o}[k + 1] = w_{sj}^{o}[k] − α_{o} \frac{∂E (w[k])}{∂w_{sj}^{o}[k]}
= w_{sj}^{o}[k] + α_{o} (d_{s} − y_{s}) fo^{′}(net_{s})z_{j}[k],  (16.16)

where fo^{′} is the derivative of f_{o}(net_{s}) with respect to net_{s}, and the values of z_{j} are computed by feeding into the network the input vector x and then propagating it forward through the network. We have

z_{j} = f_{h} (∑_{i=1}^{n} w_{ji}^{h}x_{i}) = f_{h}(net_{j}), j = 1, 2, …, l.

To derive the update formulas for the weights in the hidden layer, we use the chain rule to compute ∂E/∂w_{ji}^{h}, and then apply the gradient descent algorithm to obtain

\frac{∂E}{∂w_{ji}^{h}} = \frac{∂E}{∂z_{j}} \frac{∂z_{j}}{∂net_{j}} \frac{∂net_{j}}{∂w_{ji}^{h}} ,  (16.17)

where

\frac{∂net_{j}}{∂w_{ji}^{h}} = x_{i},  (16.18)
\frac{∂z_{j}}{∂net_{j}} = fh^{′}(net_{j}).  (16.19)

Finally, using (16.15) gives

\frac{∂E}{∂z_{j}} = \frac{∂}{∂z_{j}} (\frac{1}{2} ∑_{s=1}^{m} (d_{s} − f_{o} (net_{s}))^{2})
= − ∑_{s=1}^{m} (d_{s} − f_{o} (net_{s})) \frac{∂f_{o} (net_{s})}{∂z_{j}}
= − ∑_{s=1}^{m} (d_{s} − y_{s}) fo^{′} (net_{s}) w_{sj}^{o}.  (16.20)

Combining (16.17)–(16.20), we obtain rules for updating the hidden weights,

w_{ji}^{h}[k + 1] = w_{ji}^{h}[k] − α_{h} \frac{∂E (w[k])}{∂w_{ji}^{h}[k]}
= w_{ji}^{h}[k] + α_{h} (∑_{s=1}^{m} (d_{s} − y_{s}) fo^{′} (net_{s}) w_{sj}^{o}[k]) fh^{′}(net_{j})x_{i}. (16.21)

A commonly used activation function is the sigmoid,

f(v) = \frac{1}{1 + e^{−v}} .  (16.22)

It is easy to check that the activation function (16.22) has the useful property,

\frac{df(v)}{dv} = f(v)(1 − f(v)).  (16.23)

We summarize the above derivations in the following algorithm.

Algorithm 16.2 Backpropagation Training

Given a set of training pairs {(x^{(1)}, d^{(1)}) , …, (x^{(i)}, d^{(i)}) , …, (x^{(p)}, d^{(p)})}.

1. Initialize Weights and Thresholds Initialize the weights and thresholds to small random values.

2. Present New Input Vector and Desired Output Present the new input vector, x^{(i)} and the corresponding desired output d^{(i)}. Calculate actual outputs.

3. Calculate Error Gradients

4. Adapt Weight Vectors and Thresholds

w[k + 1] = w[k] − α∇wE (w[k], t[k]) ,
t[k + 1] = t[k] − α∇tE (w[k], t[k]) .

5. Repeat by Returning to Step 2

Example 16.3 We consider the two-input XOR problem. We showed, in Example 16.2, that this logic function cannot be implemented using a single TLU. Here, we use a two layer neural network with two nodes in the hidden layer and one node in the output layer as shown in Figure 16.8.

In this example, we use the unipolar representation of the two-input XOR function, that is, the elements of the training pairs are from the set {0, 1}. Therefore, the input vectors and the corresponding desired outputs have the form

x^{(1)} = [ \begin{matrix}0 \\ 0 \end{matrix} ] , d_{1} = 0; x^{(2)} = [ \begin{matrix}1 \\ 1 \end{matrix} ] , d_{2} = 0;
x^{(3)} = [ \begin{matrix}0 \\ 1 \end{matrix} ] , d_{3} = 1; x^{(4)} = [ \begin{matrix}1 \\ 0 \end{matrix} ] , d_{4} = 1.

The error function for this example takes the form

E = \frac{1}{2}(d_{i} − y_{i})^{2}.

Figure 16.8: A two-layer feedforward neural network for solving the XOR problem.

The activation function for all three neurons is the sigmoid given by (16.22). The outputs of the hidden neurons are

z_{1} = f(w_{11}^{h}x_{1}^{(i)} + w_{12}^{h}x_{2}^{(i)} − t_{1})  (16.24)
z_{2} = f(w_{21}^{h}x_{1}^{(i)} + w_{22}^{h}x_{2}^{(i)} − t_{2})  (16.25)

The network output is

y = f(w_{11}^{o}z_{1} + w_{12}^{o}z_{2} − t_{o})  (16.26)

The threshold parameters are treated as learning weights. Thus, the above simple network requires updating of nine weights, where, in particular,

\frac{∂E}{∂t_{1}} = (d_{i} − y)y′w_{11}^{o}z′_{1} = (d_{i} − y)y(1 − y)w_{11}^{o}z_{1}(1 − z_{1})
\frac{∂E}{∂t_{2}} = (d_{i} − y)y′w_{12}^{o}z′_{2} = (d_{i} − y)y(1 − y)w_{12}^{o}z_{2}(1 − z_{2})
\frac{∂E}{∂t_{o}} = (d_{i} − y)y′ = (d_{i} − y)y(1 − y).

In calculating the above, we used (16.23). In our simulation the weights of the network were initialized to small random values in the interval [−0.1, 0.1]. The four patterns were presented in a mixed order. We used a fixed learning rate of 0.75 for all nine weights. After 50,000 iterations we obtained the following weight values:

W^{h} = [ \begin{matrix}−4.9288 & −4.9334 \\ −6.7240 & −6.7526 \end{matrix} ] , W^{o} = [ 10.2447 −10.7258 ] ,

Table 16.1: The desired and actual responses of the trained network in Example 16.3.

\begin{matrix}x_{1} & x_{2} & d_{i} & y_{i} \\ 0 & 0 & 0 & 0.0087 \\ 1 & 1 & 0 & 0.0162 \\ 0 & 1 & 1 & 0.9870 \\ 1 & 0 & 1 & 0.9870 \end{matrix}

and t_{1} = −7.3398, t_{2} = −2.7853, and t_{o} = 4.8701. In Table 16.1 we show the desired and actual responses of the trained network. In Figure 16.9 we show a plot of the function that the trained network implements.

Figure 16.9: The function implemented by the trained neural network to solve the two-input XOR problem in Example 16.3.

16.3 Neural Fuzzy Identifier

In this section, we describe a method that can be used to construct identifiers of dynamical systems that in turn could be employed to construct neural-network-based fuzzy logic control strategies. The idea behind the method is to apply the backpropagation algorithm to a fuzzy logic system.

It follows from our discussion in Section 15.2 that a typical fuzzy logic system can be thought of as a nonlinear system described by

f(x_{1}, …, x_{n}) = f(x) = \frac{∑_{l=1}^{M} v_{l} (∏_{i=1}^{n} μ_{il}(x_{i}))}{∑_{l=1}^{M} ∏_{i=1}^{n} μ_{il}(x_{i})},  (16.27)

where, for example,

μ_{il} = exp \{−\frac{(x_{i} − m_{il})^{2}}{2σ_{il}^{2}} \} ,

M is the number of IF-THEN fuzzy rules, x = [ x_{1} x_{2} ⋯ x_{n} ]^{⊤}, and m_{il} and σ_{il} are the parameters of the Gaussian membership functions, while v_{l} is a weight parameter. These parameters are determined using the backpropagation algorithm discussed in the previous section. To facilitate our discussion, we represent (16.27), following Wang [345, page 169], as a feedforward network shown in Figure 16.10. The variables that characterize the feedforward network depicted in Figure 16.10 satisfy the following relations:

\begin{matrix}z_{l} & = & ∏_{i=1}^{n} exp \{−\frac{(x_{i}−m_{il})^{2}}{2σ_{il}^{2}} \} , l = 1, 2 …, M \\ b & = & ∑_{l=1}^{M} z_{l} \\ a & = & ∑_{l=1}^{M} v_{l}z_{l} \\ f & = & a/b \end{matrix} \}  (16.28)

We then use this feedforward network as a component of an identifier of dynamical systems. The identification scheme involving (16.27) is depicted in Figure 16.11. The adaptation algorithm for the identifier parameters is derived using the backpropagation algorithm. The cost function to be minimized is

E = \frac{1}{2} (y − ŷ)^{2} = \frac{1}{2} (y − f)^{2} ,

where y is the output of the identified plant and f is the output of the identifier. We use a gradient algorithm to minimize E over the space of parameters

w = {v_{l}, σ_{il}, m_{il}} .

16.3. NEURAL FUZZY IDENTIFIER 819 Figure 16.10: Fuzzy logic system (16.27) represented as a feedforward network. Figure 16.11: Neural fuzzy identification scheme of a nonlinear dynamical system.

The gradient algorithm that we use has the form

w[k + 1] = w[k] − α∇E (w[k]) ,  (16.29)

where α is a fixed step size. Note that (16.29) is just a simple descent gradient algorithm that we used in our discussion of the backpropagation algorithm.

In order to be able to use the above algorithm, we need to compute partial derivatives of E with respect to each component of w. For this we first fix indices i, and l. We next compute the partial derivatives of E with respect to v_{j}. Then, the parameters v_{j} will be adapted accordingly as

v_{j}[k + 1] = v_{j}[k] − α\frac{∂E (w[k])}{∂v_{j}}.

We note that E depends upon v_{j} only through a, where f = a/b. Hence, using the chain rule, we obtain

\frac{∂E}{∂v_{j}} (w) = \frac{∂}{∂v_{j}} (\frac{1}{2} (y − f)^{2})
= \frac{∂}{∂v_{j}} (\frac{1}{2} (y − a/b)^{2})
= −\frac{(y − f)}{b} \frac{∂a}{∂v_{j}}
= \frac{f − y}{b} z_{j}.

We next compute the partial derivative of E with respect to m_{st}. Observe that E depends upon m_{st} only through z_{t}. Using the chain rule again gives

\frac{∂E}{∂m_{st}} (w) = − (y − f) \frac{∂f}{∂z_{t}} \frac{∂z_{t}}{∂m_{st}}
= − (y − f) (\frac{v_{t}b − a}{b^{2}}) z_{t} \frac{(x_{s} − m_{st})}{σ_{st}^{2}}
= (f − y) (\frac{v_{t} − f}{b} ) z_{t} \frac{(x_{s} − m_{st})}{σ_{st}^{2}}.

Hence, the adaptation formula for m_{st} takes the form

m_{st}[k + 1] = m_{st}[k] − α (f − y) (\frac{v_{t} − f}{b} ) z_{t} \frac{(x_{s} − m_{st}[k])}{σ_{st}^{2}[k]}.

To find the partial derivative of E with respect to σ_{st}, we note that E depends upon σ_{st} only through z_{t}. Applying the chain rule gives

\frac{∂E}{∂σ_{st}} (w) = − (y − f) \frac{∂f}{∂z_{t}} \frac{∂z_{t}}{∂σ_{st}}
= (f − y) (\frac{v_{t} − f}{b} ) z_{t} \frac{(x_{i} − m_{st})^{2}}{σ_{st}^{3}} .

We are now ready to formulate an algorithm for updating on-line the weights of the neural fuzzy identifier.

Algorithm 16.3 On-line Identification

1. Initialize the identifier parameters Set the values of v_{l}, m_{il}, and σ_{il} to small random values.

2. Calculate the output of the identifier at the time instant k Compute

z_{l} = ∏_{i=1}^{n} exp (−\frac{(x_{i} − m_{il})^{2}}{2σ_{il}^{2}} ) ,
b = ∑_{l=1}^{M} z_{l},
a = ∑_{l=1}^{M} v_{l}z_{l},
f = a/b;

3. Calculate the identification error

e[k] = y[k] − f[k];

4. Adapt the identifier parameters

v_{l}[k + 1] = v_{l}[k] + α\frac{e[k]}{b[k]}z_{l}[k],
m_{il}[k + 1] = m_{il}[k] + α\frac{e[k]}{b[k]} (v_{l}[k] − f[k]) z_{l}[k]\frac{(x_{i}[k] − m_{il}[k])}{σ_{il}^{2}[k]},
σ_{il}[k + 1] = σ_{il}[k] + α\frac{e[k]}{b[k]} (v_{l}[k] − f[k]) z_{l}[k]\frac{(x_{i}[k] − m_{il}[k])^{2}}{σ_{il}^{3}[k]} ,
k = k + 1;

5. Repeat by Returning to Step 2

Example 16.4 The plant to be identified is the same as that considered by Wang [345, page 175], and is described by

y[k + 1] = 0.3y[k] + 0.6y[k − 1] + g(u[k]),

where the “unknown” function g(u) has the form

g(u) = 0.6 sin(πu) + 0.3 sin(3πu) + 0.1 sin(5πu),

and u[k] = sin(2πk/200). The identifier has the structure shown in Figure 16.11 and is described by

ŷ[k + 1] = 0.3y[k] + 0.6y[k − 1] + f(u[k]),

where f has the form of (16.27) with M = 10. We chose α = 0.5, y[0] = y[1] = 0, f[1] = f[2] = −3, and we started with u[1] = 0. The identifier’s parameters were initialized in a random fashion using MATLAB’s command rand. After the identifier was trained, we tested its prediction power using the same input signal as before and the initial conditions y[0] = y[1] = 0, f[1] = f[2] = −3, and u[1] = 0. The results of this experiment are shown in Figure 16.13.

16.4 Radial-Basis Function (RBF) Networks

In this section, we present radial-basis function (RBF) neural networks and their applications to interpolation and then to identification of unknown nonlinear dynamical systems using only the systems’ inputs and outputs data. We first show how an RBF network can be used for interpolation purposes. Then, we concern ourselves with the problem of identification of dynamical systems. Our exposition of using RBF neural networks to identify dynamical systems is based on the paper by Liu, Kadirkamanathan, and Billings [218]. The RBF neural identifier is an adaptive system for which the learning algorithm is developed using Lyapunov techniques. The algorithm can add new nodes to the neural identifier to obtain the desired identification accuracy.

Figure 16.12: Plots of the outputs of the plant and its identifier during the learning phase in Example 16.4.

Figure 16.13: Plots of the outputs of the plant and its identifier during the testing phase in Example 16.4.

Figure 16.14: A two-input radial-basis function (RBF) network.

16.4.1 Interpolation Using RBF Networks

A diagram of a radial-basis function (RBF) network is shown in Figure 16.14. The functions ϕ (∥x − x^{(i)}∥), i = 1, 2, …, N, are called the radial-basis functions, where ∥ · ∥ denotes a p-norm that is usually the Euclidean 2-norm. The real parameters w_{i}, i = 1, 2, …, N, are the weights. A common choice for ϕ is

ϕ (∥x − x^{(i)}∥) = exp (−\frac{∥x − x^{(i)}∥^{2}}{2σ_{i}^{2}} ) ,  (16.30)

where x^{(i)} is the center of the basis function and σ_{i} is sometimes called the radius of the radial basis function. The function given by (16.30) is called the Gaussian radial basis function.

The RBF networks are used to perform interpolation. The interpolation problem can be formulated as: For a given set of N different points,

{x^{(i)} : x^{(i)} ∈ ℝ^{n}, i = 1, 2, …, N}

and a corresponding set of N real numbers

{d_{i} : d_{i} ∈ ℝ, i = 1, 2, …, N}

find a function

F : ℝ^{n} → ℝ

such that

F (x^{(i)}) = d_{i}, i = 1, 2, …, N.  (16.31)

We attempt to solve the interpolation problem using the RBF network. The function that the RBF network implements is

F (x) = ∑_{j=1}^{N} w_{j}ϕ (∥x − x^{(j)}∥) .  (16.32)

Substituting the interpolation conditions given by (16.31) into (16.32) gives

[ \begin{matrix}ϕ_{11} & ϕ_{12} & ⋯ & ϕ_{1N} \\ ϕ_{21} & ϕ_{22} & ⋯ & ϕ_{2N} \\ ⋮ & ⋮ & ... & ⋮ \\ ϕ_{N1} & ϕ_{N2} & ⋯ & ϕ_{NN} \end{matrix} ] [ w_{1} ; w_{2} ; ⋮ ; w_{N} ] = [ d_{1} ; d_{2} ; ⋮ ; d_{N} ] ,  (16.33)

where ϕ_{ij} = ϕ (∥x^{(i)} − x^{(j)}∥). Let w = [ w_{1} w_{2} ⋯ w_{N} ], d = [ d_{1} d_{2} ⋯ d_{N} ], and

Φ = [ \begin{matrix}ϕ_{11} & ϕ_{12} & ⋯ & ϕ_{1N} \\ ϕ_{21} & ϕ_{22} & ⋯ & ϕ_{2N} \\ ⋮ & ⋮ & ... & ⋮ \\ ϕ_{N1} & ϕ_{N2} & ⋯ & ϕ_{NN} \end{matrix} ] .

Using the above notation, we represent (16.33) as

Φw = d.  (16.34)

The matrix Φ is called the interpolation matrix. If the interpolation matrix is nonsingular, then we can solve (16.34) for w to obtain

w = Φ^{−1}d.  (16.35)

Example 16.5 We construct an RBF network that interpolates data given in Table 16.2. We have, ϕ_{ij} = exp (−\frac{(x^{(i)}−x^{(j)})^{2}}{2σ_{i}^{2}} ). We set σ_{i} = 1, for i = 1, …, 5, and evaluate (16.35) to obtain

w = [ 0.8000 0.1459 −0.0390 0.5737 0.2227 ]^{⊤} .

The interpolation result is shown in Figure 16.15. In Figure (16.15)(a), we used a solid line to show the result of interpolating using the Gaussian radial-radial basis functions, while a broken line indicates the result of the straight line interpolation. The plots of the basis functions that contribute to the interpolating curve are shown in Figure (16.15)(b).

826 CHAPTER 16. NEURAL NETWORKS

Table 16.2: Interpolation data for Example 16.5.

\begin{matrix}x^{(i)} & d_{i} \\ 0 & 0.8 \\ 6 & 0.2 \\ 7 & 0.4 \\ 8 & 0.6 \\ 10 & 0.3 \end{matrix}

Figure 16.15: Results of interpolating data from Table 16.2 using radial-basis functions.

If the number of radial basis functions was fixed and the rows of the matrix Φ and the elements of the vector d were obtained sequentially, we could use the recursive least squares algorithm, described in Section A.11, to compute the least squares estimate of w.

16.4.2 Identification of a Single-Input Single-State System

Here, we discuss the identification of a single-input single-state dynamical system using neural network. We consider a dynamical system modeled by

ẋ(t) = f(x(t), u(t)), x(0) = x_{0},  (16.36)

where f : ℝ × ℝ → ℝ is an unknown, possibly nonlinear, function of x and u. We will construct a neural network to identify the function f. Adding and subtracting ax(t), where a > 0 is a design parameter, to the right-hand side of (16.36) gives

ẋ(t) = −ax(t) + ax(t) + f(x(t), u(t))
= −ax(t) + g(x(t), u(t)),  (16.37)

where g(x(t), u(t)) = ax(t) + f(x(t), u(t)). The reason for the above manipulations will become apparent when we discuss learning rules of the neural network identifier of the dynamical system (16.36). This identifier is described by

˙x̂(t) = −ax̂(t) + ĝ(x(t), u(t), m_{k}, σ_{k}), x̂(0) = x̂_{0},  (16.38)

where x̂ is the state of the identification network, m_{k} and σ_{k} are adjustable parameters of the identifier. The function ĝ has the form,

ĝ(x(t), u(t), m_{k}, σ_{k}) = ∑_{k=1}^{K} w_{k}ϕ(x, u, m_{k}, σ_{k}),  (16.39)

where for k = 1, 2, …, K, we have m_{k} = [ m_{xk} m_{uk} ], and

ϕ(x(t), u(t), m_{k}, σ_{k}) = exp (−\frac{1}{2σ_{k}^{2}} ((x(t) − m_{xk})^{2} + (u(t) − m_{uk})^{2})) .

Thus, we postulate that the function ĝ be generated using an RBF network.

Suppose that the basis functions are given to us. The problem of choosing basis functions is discussed later. Suppose also that we know the “optimal” weights w_{k}^{∗}, k = 1, 2, …, K,

Figure 16.16: A configuration for the radial-basis function neural identifier.

that result in the least identification error between g in (16.37) and ĝ in (16.38), where the identification error is defined as

e(t) = g(x(t), u(t)) − ∑_{k=1}^{K} w_{k}^{∗}ϕ(x(t), u(t), m_{k}, σ_{k}).  (16.40)

Using the above notation, we represent the dynamical system model (16.37) in the form

ẋ(t) = −ax(t) + ∑_{k=1}^{K} w_{k}^{∗}ϕ(x(t), u(t), m_{k}, σ_{k}) + e(t).  (16.41)

The neural identifier, on the other hand, can be represented as

˙x̂(t) = −ax̂(t) + ∑_{k=1}^{K} w_{k}ϕ(x(t), u(t), m_{k}, σ_{k}).  (16.42)

A schematic of the above identification architecture is depicted in Figure 16.16.

16.4.3 Learning Algorithm for the RBF Identifier

We assume that both the state x and the input u of the dynamical system model (16.36), and hence that of (16.37), are bounded. We define the state estimation error to be

e_{x} = x − x̂,

while the weights’ estimation errors are

ξ_{k} = w_{k}^{∗} − w_{k}, k = 1, …, K.

We assume that the modeling error, e, defined in (16.40), is bounded by a finite constant e_{K}, that is, |e| ≤ e_{K}. However, e_{K} is unknown to us. Next, we use (16.41) and (16.42) to form the differential equation describing the state error dynamics. For simplicity of notation, we omit the argument t of the time-varying functions. We have

ė_{x} = ẋ − ˙ x̂
= −ax + ∑_{k=1}^{K} w_{k}^{∗}ϕ(x, u, m_{k}, σ_{k}) + e + ax̂ − ∑_{k=1}^{K} w_{k}ϕ(x, u, m_{k}, σ_{k})
= −ae_{x} + ∑_{k=1}^{K} ξ_{k}ϕ(x, u, m_{k}, σ_{k}) + e.  (16.43)

We now use the second method of Lyapunov to devise a learning law for the RBF identifier that would yield a stable identification scheme in the sense that the state estimation error e_{x} decreases as the time increases and the weights do not diverge. For this purpose, we use the following Lyapunov function candidate,

V (e_{x}, ξ) = \frac{1}{2} (e_{x}^{2} + \frac{1}{α} ∑_{k=1}^{K} ξ_{k}^{2}) ,  (16.44)

where α > 0 is a design parameter and ξ = [ ξ_{1} ξ_{2} ⋯ ξ_{K} ]^{⊤}. The time derivative of V evaluated on the trajectories of (16.43) is

V̇ = e_{x}ė_{x} + \frac{1}{α} ∑_{k=1}^{K} ξ_{k}ξ̇_{k}
= e_{x} (−ae_{x} + ∑_{k=1}^{K} ξ_{k}ϕ(x, u, m_{k}, σ_{k}) + e) + \frac{1}{α} ∑_{k=1}^{K} ξ_{k}ξ̇_{k}
= −ae_{x}^{2} + \frac{1}{α} ∑_{k=1}^{K} ξ_{k} (αe_{x}ϕ(x, u, m_{k}, σ_{k}) + ξ̇_{k}) + e_{x}e.  (16.45)

Let

ξ̇_{k} = −αe_{x}ϕ(x, u, m_{k}, σ_{k}).  (16.46)

Then (16.45) simplifies to

V̇ = −ae_{x}^{2} + e_{x}e.  (16.47)

Because ẇ_{k} = −ξ̇_{k}, we can represent (16.46) as

ẇ_{k} = αe_{x}ϕ(x, u, m_{k}, σ_{k})  (16.48)

We further evaluate (16.47) to obtain

V̇ ≤ −ae_{x}^{2} + |e_{x}| e_{K}
= −a |e_{x}| (|e_{x}| − \frac{e_{K}}{a} )  (16.49)

It follows from (16.47) that if the state estimation error is zero, that is, e_{K} = 0, then

V̇ = −ae_{x}^{2}.

Thus in this case, V̇ is negative semi-definite in the [ e_{x} ξ^{⊤} ]^{⊤}-space, and hence the origin

of the [ e_{x} ξ^{⊤} ]^{⊤}-space is stable, which guarantees boundedness of e_{x} and the weights w_{k}s. We show, using the Lyapunov-like lemma stated on page 392, that when e = 0, then e_{x}(t) → 0 as t → ∞. We have to check if the assumptions of the Lyapunov-like lemma are satisfied. First, observe that V is bounded below and positive semi-definite. Next, because e_{x}, ξ_{k} as well as ϕ(x, u, m_{k}, σ_{k}) are bounded, the derivative of the state estimation error, given by (16.43) for e = 0, is bounded as well, which implies that

V̈ = −2ae_{x}ė_{x}

is bounded. This in turn implies that V̇ is uniformly continuous. Hence, by the Lyapunovlike lemma, e_{x}(t) → 0 as t → ∞.

In the presence of the identification error, e, if |e_{x}| < e_{K}/a, then it is possible that V̇ > 0, which implies that the weights w_{k}s may drift to infinity with time. To avoid this problem, Liu, Kadirkamanathan, and Billings [218] propose to set ẇ_{k} = 0 if |e_{x}| < e_{K}/a. Also, because e_{K} is unknown, a bound is imposed on the weights as well as on the error. The learning rule becomes

ẇ_{k} = \{ \begin{matrix}αe_{x}ϕ(x, u, m_{k}, σ_{k}) & if & |e_{x}| ≥ e_{0} and ∥w∥ ≤ M \\ 0 & & otherwise \end{matrix} (16.50)

where the bounds e_{0} and M are design parameters, and w = [ w_{1} ⋯ w_{K} ]^{⊤}. With the above modified learning law, whenever |e_{x}| ≥ e_{0} ≥ e_{K}/a, the Lyapunov derivative V̇ is negative semi-definite. However, when e_{0} ≤ |e_{x}| ≤ e_{K}/a, the weights may increase with time because it is possible that V̇ > 0. We impose an upper-bound on the weights. We also impose an upper bound on the estimation error, e_{x}. When any of the two bounds is reached, more basis functions may be needed to improve the approximation of g. New basis functions are added to reduce the error until V̇ < 0. In what follows, we describe a procedure for adding radial-basis functions to the identification network.

16.4.4 Growing RBF Network

As we mentioned above, when V̇ > 0, the identification scheme is no longer stable, but the learning rule (16.50) guarantees that the adjustable parameters are bounded. Once the parameter adjustments can no longer reduce the estimation error and the bounds on the weights or on the estimation error are reached, additional basis functions in the RBF network should be added to reduce the modeling error. This continues until the approximation leads to the error reduction so that V̇ < 0.

In the proposed identification scheme, we use Gaussian radial-basis functions, whose areas of “influence” are determined by the functions’ centers and radii (widths). As the function to be approximated moves further away from a particular basis function, the contribution of that basis function decreases. This is due to the exponential form of the Gaussian basis functions. As |z| increases exp(−z^{2}) decreases. The maximum contribution of a particular basis function to approximating a given function occurs when the center of the basis function and the approximated function are at the same point because in this case we have exp(0) = 1. This heuristic argument leads to the following rule for determining the centers of new basis functions to be added at a time instance t,

m_{K+1} = [ m_{x,K+1} m_{u,K+1} ]^{⊤} = [ δ_{x}round (x(t)/δ_{x}) δ_{u}round (u(t)/δ_{u}) ]^{⊤} , (16.51)

where the round operator rounds the elements on which it operates to the nearest integer. For example, if x = [ −1.8 −0.3 3.7 ], then round(x) = [ −2 0 4 ]. The parameters, δ_{x} and δ_{u} represent the spacing of the centers of the basis functions. We can think of having a grid over the input-state space where the spacing between the vertices of the grid are determined by the parameters δ_{x} and δ_{u}. Each vertex of the grid represents the potential center of a basis function. The parameter σ_{k} of the new basis function is determined from the relation,

σ_{k} = κ ∥m_{k} − m_{k}^{∗}∥ ,  (16.52)

where κ > 0 is a design parameter and

m_{k}^{∗} = arg min_{{i=1,...,K+1 \atop i≠k}} \{∥m_{k} − m_{i}∥\} .  (16.53)

Thus, m_{k}^{∗} is the nearest center of all basis functions to the center of the k-th basis function.

With the above background, we are ready to present the rules for adding a new basis function to the RBF network. Initially, the network contains no basis functions. Then, once the following two conditions are satisfied, a new basis function is added. The first condition is:

min_{k=1,...,K} |x(t) − m_{xk}| > δ_{x}/2 or min_{k=1,...,K} |u(t) − m_{uk}| > δ_{u}/2.  (16.54)

The above is satisfied if the state or input, at time t, is closer to an available “unoccupied” grid vertex than to any of the existing centers of the basis functions. The basis function at the new vertex would be the closest to the existing state or input and would have the greatest influence in the approximation of g. The second condition is

|x(t) − x̂(t)| > e_{max},  (16.55)

where e_{max} is the maximum allowable state estimation error. If (16.55) is satisfied, a new basis function is added to reduce the state estimation error by reducing the modeling error.

Example 16.6 The above technique for identification of dynamical systems using radialbasis function network is illustrated with the system,

ẋ = 1.1 (1 − x + 2xu − x^{2}) exp (−0.5x^{2} − u^{2}) , x(0) = 0,  (16.56)

where u = cos(t/100). This model was adapted from [218]. There are many ways to simulate the above identification scheme. In our simulations, we discretized the above differential equation using Euler’s method with the step size dt = 0.01 sec. The initial weights and the initial state of the identifier were set to zero. We used the following parameter values: e_{0} = 0.001, e_{max} = 0.005, α = 5, κ = 3. In Figure 16.17, plots of the true state and its estimate versus time are shown. A plot of the state estimation error, e_{x}, between the true state and its estimate is shown in Figure 16.18. Initially, the error exceeds the bound e_{max}. Adding additional basis functions drives the error back down. The plots shown in Figure 16.19 depict the true state and its estimate versus the input u. Finally, in Figure 16.20, plots of the derivative of the true state versus the true state and the derivative of the estimated state versus the state estimate are shown.

For a generalization of the above approach to real-time approximation of continuous-time dynamical systems using self-organizing radial basis function networks, we refer to Lian et al. [212].

16.4. RADIAL-BASIS FUNCTION (RBF) NETWORKS 833 Figure 16.17: Plots of the true state and its estimate versus time. The estimated state “hugs” closely the actual state.

Figure 16.18: A plot of the error, e_{x}, between the true state and its estimate.

834 CHAPTER 16. NEURAL NETWORKS

Figure 16.19: A plot of the true state and its estimate versus the input u. Figure 16.20: Plots of the derivative of the true state versus the true state and the derivative of the estimated state versus the state estimate.

16.4.5 Identification of Multivariable Systems

We extend the results of Subsection 16.4.2 to multi-input multi-state dynamical systems. In our derivation of the update law for the weights of the RBF network, we use a result from matrix algebra presented next. Recall the definition of the trace of a square matrix as given by (A.32). If Γ ∈ ℝ^{m×n}, then by (A.79) its Frobenius norm satisfies

∥Γ∥_{F}^{2} = ∑_{i=1}^{m} ∑_{j=1}^{n} γ_{ij}^{2} = trace (ΓΓ^{⊤}) .

If the elements, γ_{ij}, of Γ are functions of time, then

\frac{d}{dt} (trace (ΓΓ^{⊤})) = 2 ∑_{i=1}^{m} ∑_{j=1}^{n} γ_{ij}γ̇_{ij} = 2 trace (ΓΓ̇^{⊤}) ,  (16.57)

where Γ̇ = \frac{dΓ}{dt}. With the above background, we can proceed with an analysis of the RBF neural identifier for the multivariable systems.

The multivariable dynamical systems that we consider are modeled by

ẋ(t) = f(x(t), u(t)), x(0) = x_{0},  (16.58)

where u(t) ∈ ℝ^{m}, x(t) ∈ ℝ^{n}, and f : ℝ^{n} × ℝ^{m} → ℝ^{n} is the function to be identified. Let A ∈ ℝ^{n×n} be a Hurwitz matrix, that is, the matrix whose eigenvalues are all in the open left-half plane. Because A is a Hurwitz matrix, by the theorem of Lyapunov (Theorem 7.1), for any given real symmetric positive definite (r.s.p.d.) matrix Q the solution P to the continuous Lyapunov matrix equation,

A^{⊤}P + P A = −Q  (16.59)

is also (real symmetric) positive definite. We use this fact later when we develop a learning law for the RBF neural identifier.

We represent the model (16.58) in an equivalent form,

ẋ(t) = Ax(t) + g(x(t), u(t)), x(0) = x_{0},  (16.60)

where

g(x(t), u(t)) = −Ax(t) + f(x(t), u(t)).

The k-th basis function can be represented as

ϕ_{k}(x, u) = exp (−\frac{1}{2σ_{k}^{2}} ‖[ \begin{matrix}x \\ u \end{matrix} ] − m_{k}‖^{2}) .

836 CHAPTER 16. NEURAL NETWORKS Let

Φ_{K}(x, u) = [ ϕ_{1}(x, u) ϕ_{2}(x, u) ⋯ ϕ_{K}(x, u) ]^{⊤}

and

ĝ(x, u, m_{k}, σ_{k}) = W_{K}Φ_{K},

where W_{K} is the n × K weight matrix and m_{k} = [ m_{xk}^{⊤} m_{uk}^{⊤} ]^{⊤} is the center of the k-th radial basis function in the [ x^{⊤} u^{⊤} ]^{⊤}-space. The RBF neural identifier has the form,

˙x̂(t) = Ax̂(t) + ĝ(x, u, m_{k}, σ_{k}), x̂(0) = x_{0}.  (16.61)

Let W _{K}^{∗} be the optimal weight matrix that minimizes the modeling error,

e(t) = g(x(t), u(t)) − W _{K}^{∗}Φ_{K}(x(t), u(t)).

Using the above definition, we represent the system model (16.60) as

ẋ(t) = Ax(t) + W _{K}^{∗}Φ_{K}(x(t), u(t)) + e(t).  (16.62)

We define the state estimation error and the weight estimation error as

ex = x − x̂ and Γ_{K} = W _{K}^{∗} − W_{K}.

Next, we form the differential equation describing the state estimation error dynamics,

ėx = ẋ − ˙ x̂
= Ax + W _{K}^{∗}Φ_{K}(x, u) + e − Ax̂ − ĝ(x, u, m_{k}, σ_{k})
= A (x − x̂) + W _{K}^{∗}Φ_{K}(x, u) − W_{K}Φ_{K}(x, u) + e
= Aex + Γ_{K}Φ_{K}(x, u) + e.  (16.63)

We employ the following Lyapunov function candidate,

V = V (ex, Γ_{K}) = \frac{1}{2} (ex^{⊤}P ex + \frac{1}{α}trace (Γ_{K}Γ_{K}^{⊤})) .  (16.64)

Using (16.57), we evaluate the time derivative of V on the solutions of the state estimation error equation to obtain

V̇ = ex^{⊤}P ėx + \frac{1}{α}trace (Γ_{K} Γ̇_{K}^{⊤})
= ex^{⊤} (P Aex + P Γ_{K}Φ_{K}(x, u) + P e) + \frac{1}{α}trace (Γ_{K} Γ̇_{K}^{⊤})
= \frac{1}{2}ex^{⊤} (A^{⊤}P + P A) ex + ex^{⊤}P Γ_{K}Φ_{K}(x, u) + \frac{1}{α}trace (Γ_{K} Γ̇_{K}^{⊤})
+ex^{⊤}P e.  (16.65)

Using the property of the trace operator given by (A.33), on page 965, gives

ex^{⊤}P Γ_{K}Φ_{K}(x, u) = trace (ex^{⊤}P Γ_{K}Φ_{K}(x, u))
= trace (Γ_{K}Φ_{K}(x, u)ex^{⊤}P ) .  (16.66)

Observe that

Ẇ_{K} = −Γ̇_{K}.

Using the above and (16.66) along with the Lyapunov equation (16.59) yields

V̇ = −ex^{⊤}Qex + \frac{1}{α} (trace (αΓ_{K}Φ_{K}(x, u)ex^{⊤}P ) − trace (Γ_{K} Ẇ _{K}^{⊤})) + ex^{⊤}P e. (16.67)

Consider the following weight matrix update law,

Ẇ_{K} = αP exΦ_{K}^{⊤}(x, u)  (16.68)

With the above update law, the Lyapunov derivative (16.67) evaluates to

V̇ = −ex^{⊤}Qex + ex^{⊤}P e
≤ −λ_{min}(Q)∥ex∥^{2} + λ_{max}(P)∥e∥∥ex∥
= −λ_{min}(Q) (∥ex∥ − \frac{λ_{max}(P)}{λ_{min}(Q)}∥e∥) ∥ex∥.  (16.69)

Utilizing the same type of arguments as in the scalar case (analyzed in Subsection 16.4.2) and the Lyapunov-like lemma stated on page 392, we can show that when e = 0, then e_{x}(t) → 0 as t → ∞.

If the magnitude of the state estimation error is such that ∥ex∥ < \frac{λ_{max}(P)}{λ_{min}(Q)}∥e∥, then it may happen that V̇ > 0, which in turn may lead to the weights’ drift to infinity over time. To prevent this, we modify (16.68) following the analysis of the single-input single-state case. The modified weight update law is

Ẇ_{K} = \{ \begin{matrix}αP exΦ_{K}^{⊤}(x, u) & if & ∥ex∥ ≥ e_{0} and ∥W_{K}∥ ≤ M \\ O & & otherwise \end{matrix} (16.70)

Example 16.7 Consider an uncertain dynamical system model,

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ] = [ \begin{matrix}0 & 1 & 0 \\ 0 & 0 & 1 \\ −116 & −69 & −14 \end{matrix} ] [ \begin{matrix}x_{1} \\ x_{2} \\ x_{3} \end{matrix} ] + [ \begin{matrix}0 & 0 \\ 1 & 0 \\ 0 & 1 \end{matrix} ] ([ \begin{matrix}u_{1} \\ u_{2} \end{matrix} ] + [ \begin{matrix}ζ_{1} \\ ζ_{2} \end{matrix} ])
= Ax + B(u + ζ),

where the vector modeling the system uncertainties satisfies the norm bound condition,

∥[ \begin{matrix}ζ_{1} \\ ζ_{2} \end{matrix} ]∥ ≤ 10.

The uncontrolled nominal system ẋ = Ax is asymptotically stable. The solution to the Lyapunov matrix equation A^{⊤}P + P A = −I_{3} is

P = [ \begin{matrix}9.2833 & 4.8451 & 0.0043 \\ 4.8451 & 3.9263 & 0.0775 \\ 0.0043 & 0.0775 & 0.0412 \end{matrix} ] .

Our objective is to design a stabilizing controller, that contains an adaptive neural term employing 10 radial basis functions, so that the state of the closed-loop system driven by your controller asymptotically converges to the origin.

We propose the following stabilizing controller,

u = −u_{neural} − η \frac{B^{⊤}P x}{∥B^{⊤}P x∥},  (16.71)

where

u_{neural} = W Φ

and the gain η will be determined later. The vector Φ is composed of radial basis functions and is bounded. We assume that the “optimal” weights approximate the uncertainties modeled by the function ζ so that

ζ = W^{∗}Φ + e,  (16.72)

where e is the approximation error. We assume that the approximation error is bounded by a known constant ρ, that is,

∥e∥ ≤ ρ.

In further discussion, we use the following notation,

σ = B^{⊤}P x and let Γ = W^{∗} − W .

Employing the above notation, we represent the proposed control strategy (16.71) as

u = −W Φ − η \frac{σ}{∥σ∥}.  (16.73)

Consider the following Lyapunov function candidate,

V = \frac{1}{2}x^{⊤}P x + \frac{1}{2α}trace (ΓΓ^{⊤}) .

The Lyapunov derivative is

V̇ = x^{⊤}P ẋ + \frac{1}{α}trace (ΓΓ̇^{⊤})
= x^{⊤}P (Ax + B(u + ζ)) + \frac{1}{α}trace (ΓΓ̇^{⊤})
= −\frac{1}{2}∥x∥^{2} + x^{⊤}P Bu + x^{⊤}P Bζ + \frac{1}{α}trace (ΓΓ̇^{⊤}) .

Substituting into the above (16.72) and the control strategy given by (16.73) and manipulating gives

V̇ = −\frac{1}{2}∥x∥^{2} − η∥σ∥ − σ^{⊤}W Φ + σ^{⊤} (W^{∗}Φ + e) + \frac{1}{α}trace (ΓΓ̇^{⊤})
= −\frac{1}{2}∥x∥^{2} − η∥σ∥ + σ^{⊤} (W^{∗} − W) Φ + σ^{⊤}e + \frac{1}{α}trace (ΓΓ̇^{⊤})
≤ −\frac{1}{2}∥x∥^{2} − ∥σ∥ (η − ∥e∥) + σ^{⊤}ΓΦ + \frac{1}{α}trace (ΓΓ̇^{⊤})
= −\frac{1}{2}∥x∥^{2} − ∥σ∥ (η − ∥e∥) + trace (σ^{⊤}ΓΦ) + \frac{1}{α}trace (ΓΓ̇^{⊤})
= −\frac{1}{2}∥x∥^{2} − ∥σ∥ (η − ∥e∥) + trace (ΓΦσ^{⊤} + \frac{1}{α}ΓΓ̇^{⊤}) .

We select

η ≥ ∥e∥ and Γ̇ = −ασΦ^{⊤}

to obtain

V̇ ≤ −\frac{1}{2}∥x∥^{2},  (16.74)

which is negative semi-definite in the augmented (x, Γ)-space. Therefore x as well as W are bounded. Hence the proposed controller is also bounded, which implies that ẋ is bounded. This in turn implies that

\frac{d}{dt}∥x∥^{2} = 2x^{⊤}ẋ

is bounded and hence ∥x∥^{2} is uniformly continuous. Integrating both sides of (16.74) gives

∫_{0}^{t} V̇ (τ)dτ = V (t) − V (0) ≤ ∫_{0}^{t} (−\frac{1}{2}∥x∥^{2}) dτ

and hence

∫_{0}^{t} ∥x∥^{2}dτ ≤ 2(V (0) − V (t)) ≤ 2V (0),

which means that

lim_{t→∞} ∫_{0}^{t} ∥x∥^{2}dτ

exists and is finite. By Barbalat’s lemma,

∥x∥^{2} → 0 as t → ∞,

which implies that

lim_{t→∞} x = 0.

16.5 A Self-Organizing Network

Up to now we discussed neural networks that must have a teacher in order to learn presented patterns. This mode of learning is referred to as the supervised mode. In this section, we analyze a neural network that can learn in the unsupervised mode and the network undergoes the self-organization process. Specifically, the network is capable of clustering training patterns. It is assumed that the number of classes, p, is known. The network’s number of weight vectors is equal to the number of classes in the training set. The weights are initialized and then normalized. After initialization, the training patterns, x^{(1)}, x^{(2)}, …, x^{(N)}, are presented. In response to each presented pattern, we compute the winning neuron of the network, that is, we compute the network weight that is closest to the presented pattern. We call this computation the similarity matching. Let x be the presented pattern, where, for simplicity, we omitted its index. Then the similarity matching can be described as

∥x − w^{(i)}∥ = min_{1≤j≤p} {∥x − w^{(j)}∥} ,

where i is the index of the winning neuron. Thus, searching for the winning neuron corresponds to finding the maximum among the p scalar products,

max_{j=1,2,...,p} ⟨w^{(j)}, x⟩ .

To see that this is indeed the case, note that because the weights and the patterns are normalized, that is, ∥w^{(j)}∥ = ∥x∥ = 1, we have

∥x − w^{(j)}∥ = (∥x∥^{2} − 2 ⟨w^{(j)}, x⟩ + ∥w^{(j)}∥^{2})^{1/2}
= (2 − 2 ⟨w^{(j)}, x⟩)^{1/2} .

Hence, minimizing ∥x − w^{(j)}∥ is equivalent to maximizing the scalar product ⟨w^{(j)}, x⟩. Let cos (w^{(j)}, x) denote the cosine of the angle between w^{(j)} and x. Then,

⟨w^{(j)}, x⟩ = ∥w^{(j)}∥ ∥x∥ cos (w^{(j)}, x) ,

which means that the winning neuron is “more parallel” to the input pattern x than the other neurons.

After the winning neuron is declared, it is updated, while the remaining weights are not changed. This is the reason for the name of the network—the winner-take-all (WTA) network. This update algorithm is also known as the Kohonen learning rule after Kohonen who first proposed this learning rule. The role of the update is to reduce the distance between the winning neuron, w^{(i)}, and the presented pattern, x. One way to accomplish this task is to apply the descent gradient method to the objective function \frac{1}{2} ∥x − w^{(i)}∥^{2}. The gradient of \frac{1}{2} ∥x − w^{(i)}∥^{2} with respect to w^{(i)} is

\frac{1}{2}∇w_{(j)} ∥x − w^{(i)}∥^{2} = − (x − w^{(i)}) .

Applying the descent gradient method to minimize \frac{1}{2} ∥x − w^{(i)}∥^{2} with a step size α_{k} yields

w^{(i)}[k + 1] = w^{(i)}[k] + α_{k} (x − w^{(i)}[k]) .

The weights of other neurons are unchanged, that is,

w^{(j)}[k + 1] = w^{(j)}[k], for j ≠ i.

We summarize our discussion in the form of the following algorithm.

Algorithm 16.4 Winner-Take-All (WTA) Learning

1. Initialize Weights Generate the initial weights. Normalize each weight to have unit length: \frac{w}{∥w∥}.

2. Serially Present Training Patterns Normalize each training pattern x^{(j)} to have unit length. The training patterns can be presented in a random order a fixed number of times.

3. Select Winning Neuron For each WTA weight, compute the scalar product with the presented pattern. Select a weight that gives the largest scalar product. Call this the winning weight.

4. Update Weight of Winning Neuron Let i be the index of the winning weight. Set

w^{(i)}[k + 1] = w^{(i)}[k] + α_{k} (x^{(j)} − w^{(i)}[k])
w^{(j)}[k + 1] = w^{(j)}[k], for j ≠ i

Normalize w^{(i)}[k + 1] to have unit length.

5. Repeat by Returning to Step 2 Repeat until stopping criterion is met.

Example 16.8 We take the same set of training patterns as in Zurada [377, page 407]:

[ x^{(1)} x^{(2)} x^{(3)} x^{(4)} x^{(5)} ] = [ \begin{matrix}0.8 & 0.1736 & 0.707 & 0.342 & 0.6 \\ 0.6 & −0.9848 & 0.707 & −0.9397 & 0.8 \end{matrix} ] .

The above patterns are already normalized and are marked in Figure 16.21. As can be seen from Figure 16.21, we can classify the training patterns into two classes, that is, p = 2. The initial normalized weights are

[ w_{0}^{(1)} w_{0}^{(2)} ] = [ \begin{matrix}1 & −1 \\ 0 & 0 \end{matrix} ] .

The initial weights are also marked in Figure 16.21. The training patterns, x^{(1)}, x^{(2)}, …, x^{(5)} are presented in ascending order and then recycled. After 200 presentations of the training set, we obtained the following final weights of the network:

[ w^{(1)} w^{(2)} ] = [ \begin{matrix}0.6628 & 0.2868 \\ 0.7488 & −0.9580 \end{matrix} ] .

The final weights are marked in Figure 16.21 with ∗.

Figure 16.21: Training patterns, initial weights and the final weights of the WTA network of Example 16.8.

16.6 Hopfield Neural Network

16.6.1 Hopfield Neural Network Modeling and Analysis

The analog Hopfield neural network is a dynamical system whose schematic is shown in Figure 16.22. It is a nonlinear interconnected system proposed by Hopfield [140] as a model of biological neural networks. Each node, or neuron, in Figure 16.22 is represented by a circle. The structure of a neuron is depicted in Figure 16.23. The nonlinear amplifier in Figure 16.23 has an input-output characteristic described by a function g : ℝ → [0, 1]. We assume that g is continuously differentiable and strictly monotonically increasing, that is, g(u) > g(u′) if u > u′. The function g is sometimes called, the activation function. A typical choice of g is the sigmoid function given by

x = g(u) = \frac{1}{1 + e^{−u/θ}} ,  (16.75)

where the positive parameter θ controls the slope of g. A circuit implementation of the Hopfield network is shown in Figure 16.24. To derive equations that model the Hopfield neural network, we use an equivalent representation of the single node of the Hopfield network. This equivalent representation is shown in Figure 16.25. Applying Kirchhoff’s current law at the input of each amplifier of the equivalent node representation and taking into account that 844 CHAPTER 16. NEURAL NETWORKS

Figure 16.22: Hopfield neural network model. Figure 16.23: Model of a neuron in the Hopfield neural network.

16.6. HOPFIELD NEURAL NETWORK 845

Figure 16.24: A circuit realization of the Hopfield neural network.

Figure 16.25: Equivalent representation of a single node in the circuit realization of the Hopfield neural network. the current into the amplifier is negligibly small due to the amplifier’s high input resistance, we get

C_{i} \frac{du_{i}}{dt} + \frac{u_{i}}{R_{i}} = \frac{x_{1} − u_{i}}{R_{i1}} + \frac{x_{2} − u_{i}}{R_{i2}} + ⋯ + \frac{x_{n} − u_{i}}{R_{in}} + I_{i}, i = 1, 2, …, n,  (16.76)

where x_{i} = g(u_{i}). Let

T_{ij} = \frac{1}{R_{ij}} , \frac{1}{r_{i}} = ∑_{j=1}^{n} T_{ij} + \frac{1}{R_{i}} .  (16.77)

Substituting (16.77) into (16.76) yields

C_{i} \frac{du_{i}}{dt} + \frac{u_{i}}{r_{i}} = ∑_{j=1}^{n} T_{ij}x_{j} + I_{i}, i = 1, 2, …, n  (16.78)

The system n first-order nonlinear differential equations given by (16.78) constitutes a model of the Hopfield neural network.

The Hopfield neural network can be viewed as an associative memory. This is because it can have asymptotically stable states that attract the neighboring states to develop in time. This behavior can be interpreted as an evolution of an imperfect, corrupted pattern towards the correct, stored pattern—hence the term associative memory.

In further analysis, we use an equivalent representation of (16.78). To obtain this equivalent representation of the Hopfield model, we first note that the activation function g is, by assumption, strictly increasing and differentiable. Therefore g^{−1} exists and is also differentiable—see the inverse function theorem on page 608. In Figure 16.26, we show plots of x = g(u) and u = g^{−1}(x) for θ = 0.2. We re-write equations (16.78) as

C_{i} \frac{dg^{−1}(x_{i})}{dx_{i}} \frac{dx_{i}}{dt} + \frac{g^{−1}(x_{i})}{r_{i}} = ∑_{j=1}^{n} T_{ij}x_{j} + I_{i}, i = 1, 2, …, n  (16.79)

Let T ∈ ℝ^{n×n} be the matrix whose elements are T_{ij} and v = [ I_{1} I_{2} ⋯ I_{n} ]^{⊤} ∈ ℝ^{n}. Using this notation, we write (16.78) in matrix form,

[ \begin{matrix}C_{1} \frac{dg^{−1}(x_{1})}{dx_{1}} & ⋯ & 0 \\ ⋮ & ... & ⋮ \\ 0 & ⋯ & C_{n} \frac{dg^{−1}(x_{n})}{dx_{n}} \end{matrix} ] ẋ(t) = T x + v − [ ⋮_{\frac{g^{−1}(x_{n})}{r_{n}}}^{\frac{g^{−1}(x_{1})}{r_{1}}} ] .  (16.80)

Associated with the Hopfield neural network is its computational energy function, E : ℝ^{n} →

Figure 16.26: Plots of x = g(u) and u = g^{−1}(x).

ℝ, given by

E(x) = −\frac{1}{2} ∑_{i=1}^{n} ∑_{j=1}^{n} T_{ij}x_{i}x_{j} − ∑_{i=1}^{n} I_{i}x_{i} + ∑_{i=1}^{n} ∫_{0}^{x_{i}} \frac{g^{−1}(s)}{r_{i}}ds
= −\frac{1}{2}x^{⊤}T x − v^{⊤}x + ∑_{i=1}^{n} ∫_{0}^{x_{i}} \frac{g^{−1}(s)}{r_{i}}ds.  (16.81)

Suppose now that T = T^{⊤}. Then, the gradient of E is

∇E(x) = −T x − v + [ \frac{g^{−1}(x_{1})}{r_{1}} ⋯ \frac{g^{−1}(x_{n})}{r_{n}} ]^{⊤} .  (16.82)

Let

S(x) = [ \begin{matrix}C_{1} \frac{dg^{−1}(x_{1})}{dx_{1}} & ⋯ & 0 \\ ⋮ & ... & ⋮ \\ 0 & ⋯ & C_{n} \frac{dg^{−1}(x_{n})}{dx_{n}} \end{matrix} ] .  (16.83)

We now combine (16.82) and (16.83) to obtain an equivalent form of the Hopfield network modeling equations,

S(x)ẋ(t) = −∇E(x(t))  (16.84)

We analyze the dynamics of the Hopfield network modeled by (16.84) using the computational energy function. We first show that E is bounded. Taking norms of (16.81) yields

|E(x)| ≤ \frac{1}{2} ∥T ∥ ∥x∥^{2} + ∥v∥∥x∥ + ∑_{i=1}^{n} \frac{1}{r_{i}} |∫_{0}^{x_{i}} g^{−1}(s)ds| .  (16.85)

The components of the vector x are the outputs of the nonlinear amplifiers and they take values from the interval [0, 1]. Hence, ∥x∥ = ∥x∥_{2} ≤ \sqrt{n}. Thus, to show that E(x) is bounded it is enough to show that the integral ∫_{0}^{x_{i}} g^{−1}(s)ds is bounded. We first find the expression for g^{−1}. Recall that g(u) is given by (16.75). Therefore,

\frac{1}{x} = 1 + e^{−u/θ},

or

\frac{1}{x} − 1 = e^{−u/θ},  (16.86)

Taking logarithms of both sides of the above yields

ln (\frac{1 − x}{x}) = −\frac{u}{θ}.

Hence,

u = g^{−1}(x) = θ (ln(x) − ln(1 − x)) .  (16.87)

Using the symmetry property of g^{−1}, we obtain

|∫_{0}^{x_{i}} g^{−1}(s)ds| ≤ ∫_{0}^{x_{i}} |g^{−1}(s)| ds
≤ −2 ∫_{0}^{1/2} g^{−1}(s)ds.  (16.88)

Recall that

∫ ln(s)ds = s ln(s) − s.

Hence,

∫ ln(1 − s)ds = −(1 − s) ln(1 − s) − s.

Using the above, we obtain

∫_{0}^{1/2} g^{−1}(s)ds = ∫_{0}^{1/2} θ (ln(s) − ln(1 − s)) ds
= θ (s ln(s) − s + (1 − s) ln(1 − s) + s)|_{0}^{1/2}
= θ (s ln(s) + (1 − s) ln(1 − s))|_{0}^{1/2} .  (16.89)

Note that

s ln(s) = \frac{ln(s)}{1/s}.

Applying L’Hospital’s rule to the above yields

lim_{s→0} \frac{ln(s)}{1/s} = lim_{s→0} \frac{1/s}{−1/s^{2}} = 0.

Applying the above to (16.89), we get

∫_{0}^{1/2} g^{−1}(s)ds = θ ln(1/2).  (16.90)

Combining (16.88) and (16.90) yields

|∫_{0}^{x_{i}} g^{−1}(s)ds| ≤ −2θ ln(1/2),  (16.91)

which completes the proof of the statement that E is bounded.

Let x(t) denote the trajectory of the Hopfield network model (16.84). We show that if T = T^{⊤}, then

\frac{d}{dt}E(x(t)) ≤ 0

on the Hopfield network trajectories. Using the chain rule gives

\frac{d}{dt}E(x(t)) = ∇E(x(t))^{⊤}ẋ(t).  (16.92)

It follows from (16.84) that

∇E(x(t)) = −S(x(t))ẋ(t).  (16.93)

Substituting (16.93) into (16.92), we obtain

\frac{d}{dt}E(x(t)) = −ẋ^{⊤}(t)S(x(t))ẋ(t)
= − ∑_{i=1}^{n} C_{i} \frac{dg^{−1}(x_{i}(t))}{dx_{i}}ẋ_{i}^{2},  (16.94)

where C_{i} > 0 and

\frac{dg^{−1}(x_{i})}{dx_{i}} = θ (1/x_{i} + 1/(1 − x_{i})) > 0  (16.95)

because x_{i} ∈ (0, 1). Hence,

\frac{d}{dt}E(x(t)) ≤ 0

on the trajectories of the Hopfield model (16.84).

In the following subsection, we use the above result to construct Hopfield network based analog-to-digital converters.

16.6.2 Analog-to-Digital Converter

Our objective now is to show how the Hopfield neural network can be used to solve some engineering problems. Specifically, we will discuss the construction of an analog-to-digital (A/D) converter using the Hopfield neural network. The idea was put forward by Tank and Hopfield [324]. In A/D conversion, an analog value is transformed into its corresponding binary representation. The conversion that we discuss consists of two steps. The first step is to round the given analog value to the nearest integer. The second step is to convert the integer into its binary equivalent. For example, 3.4 is first rounded to 3, then 3 is converted into its binary equivalent “11” if we have a 2-bit converter, “011” if we have a 3-bit converter, or “0011” if we have a 4-bit converter. Therefore, an N-bit A/D converter can be viewed as a device that assigns to a given analog input a corresponding vertex of the hypercube [0, 1]^{N}.

The first step in A/D conversion, rounding of a given number to its nearest integer, can be organized as solving an optimization problem by a neural network. This entails formulation of a suitable cost, or penalty, function to be minimized. In fact, finding a suitable cost function is the first step in constructing a neural A/D converter.

To better understand the idea behind solving A/D conversion problems using neural networks, we first consider constructing a 2-bit A/D converter. Let V_{1} be the most significant bit (MSB), V_{0} be the least significant bit (LSB) and x an analog value to be converted. In the actual neural network, V_{1} and V_{0} are the outputs of the two neurons in the neural A/D converter. We first consider a very simple cost function,

E = \frac{1}{2} (V_{1}2^{1} + V_{0}2^{0} − x)^{2} .  (16.96)

A plot of the above function, for an analog input of x = 1.5, is shown in Figure 16.27(a). Note that E is defined over the two-dimensional hypercube [0, 1]^{2}. However, constructing a circuit minimizing the cost function (16.96) would not yield good results because V_{0} and V_{1} would take values from the interval [0, 1] rather than the set {0, 1}. To overcome the problem, we add to (16.96) two terms whose purpose is to penalize for non-integer values of V_{0} and V_{1}. The modified cost function is

E = \frac{1}{2} (V_{1}2^{1} + V_{0}2^{0} − x)^{2} + aV_{1}(V_{1} − 1) + bV_{0}(V_{0} − 1),  (16.97)

where a and b are weighting coefficients. Tank and Hopfield [324] suggest selecting a and b so that the quadratic terms V_{0}^{2} and V_{1}^{2} are cancelled out. In particular, if we set a = −2 and b = −0.5, then (16.97) becomes

E = \frac{1}{2}x^{2} + 2V_{0}V_{1} − 2V_{1}x − V_{0}x + 2V_{1} + \frac{1}{2}V_{0}.  (16.98)

Figure 16.27: Constructing the cost function for a 2-bit A/D converter—plots of the cost function and its components; (a) plot of (16.96) for an analog input of x = 1.5; (b) plot of the term used to cancel the quadratic terms in (16.96); (c) plot of the cost function (16.98).

The term used to cancel the quadratic terms is plotted in Figure 16.27(b). The final cost function given by (16.98) is plotted in Figure 16.27(c). When an analog input x is converted to its binary equivalent, the cost function (16.98) is minimized. One can interpret this process as follows. A given analog input x defines a corresponding cost function E. We can view E as a landscape. Now, a ball placed somewhere on this landscape will move towards the lowest point, which should correspond to one of the vertices of the hypercube [0, 1]^{2}.

Our next task is to construct differential equations that model a neural network implementing an A/D converter. The parameters of the state equations of the network will be chosen so that the time derivative of E evaluated on the solutions of the equations describing the network is negative. We consider a general case of an N-bit A/D converter. After this, we derive state equations for a 2-bit A/D converter. Note that

\frac{dE}{dt} = \frac{∂E}{∂V_{0}} V̇_{0} + \frac{∂E}{∂V_{1}} V̇_{1} + ⋯ + \frac{∂E}{∂V_{N−1}} V̇_{N−1},

where V_{i} = g_{i}(u_{i}). Hence, similarly as in (16.94), it is enough to set

\frac{∂E}{∂V_{i}} = −C_{i}u̇_{i} = −C_{i} \frac{dg_{i}^{−1}(V_{i})}{dV_{i}} V̇_{i}, i = 0, 1, …, N − 1  (16.99)

to obtain

\frac{dE}{dt} = − ∑_{i=0}^{N−1} C_{i} \frac{dg_{i}^{−1}(V_{i})}{dV_{i}} V̇_{i}^{2} ≤ 0,  (16.100)

where as in (16.95), \frac{dg_{i}^{−1}(V_{i})}{dV_{i}} > 0. Thus, if the state equations of an N-bit A/D converter satisfy (16.99), then dE/dt ≤ 0 on the trajectories of the converter. Using (16.99), we can write the state equations that model the 2-bit A/D converter consisting of two neurons. Let V_{0} = g_{0}(u_{0}) and V_{1} = g_{1}(u_{1}). Then, substituting (16.98) into (16.99) and performing required differentiation, we obtain

\begin{matrix}C_{0}u̇_{0} & = & −\frac{∂E}{∂V_{0}} = − (2V_{1} − x + 0.5) = −(2g_{1}(u_{1}) − x + 0.5) \\ C_{1}u̇_{1} & = & −\frac{∂E}{∂V_{1}} = −2 (V_{0} − x + 1) = −2(g_{0}(u_{0}) − x + 1) \end{matrix} \}  (16.101)

For simplicity, we assume that C_{0} = C_{1} = 1 and that g_{0} = g_{1} = g. Note that because C is present in both state equations, assigning C_{0} = C_{1} = 1 does not distort the dynamical behavior of the network; it only changes the time scale of both equations by the same factor. Observing that for i = 1, 2,

u̇_{i} = \frac{dg_{i}^{−1}(V_{i})}{dV_{i}} V̇_{i},

we represent (16.101) as

\begin{matrix}V̇_{0} & = & − (2V_{1} − x + 0.5) (\frac{dg^{−1}(V_{0})}{dV_{0}})^{−1} \\ V̇_{1} & = & −2 (V_{0} − x + 1) (\frac{dg^{−1}(V_{1})}{dV_{1}})^{−1} \end{matrix} \}  (16.102)

We display a plot of the cost function E for the analog value x = 1.5 in Figure (16.28)(a) and a phase portrait of the 2-bit A/D converter in Figure (16.28)(b). We chose the activation function

V_{i} = g(u_{i}) = \frac{1}{1 + e^{−4u_{i}}} .

Using (16.87), we find

u_{i} = g^{−1}(V_{i}) = \frac{1}{4} (ln(V_{i}) − ln(1 − V_{i})) .

Thus,

\frac{dg^{−1}(V_{i})}{dV_{i}} = \frac{1}{4V_{i}(1 − V_{i})}.

Note that for x = 1.5, the cost function has two minimizers in the hypercube [0, 1]^{2} that are separated by a “ridge”. Initial conditions on each side of this ridge yield a different final value of the state vector [V_{0}, V_{1}]. Depending on the initial conditions of the state vector of the A/D converter, the analog value x = 1.5 will either be converted to the binary “01” or “10”. Thus, initial conditions influence the dynamical behavior of the A/D converter. Also, the activation nonlinearity influences the dynamical behavior of the A/D converter via the multiplicative term (\frac{dg^{−1}(V_{i})}{dV_{i}})^{−1} present in the state equations (16.102). One can see Figure 16.28: A plot of the cost function and a phase portrait of the 2-bit A/D converter for the analog value x = 1.5.

that the multiplicative factor influences the direction in which the converter state trajectory evolves.

Employing the above technique, we can construct higher order A/D converters. For example, to construct a 3-bit A/D converter, we begin with the cost function

E = \frac{1}{2} (V_{2}2^{2} + V_{1}2^{1} + V_{0}2^{0} − x)^{2} + aV_{2}(V_{2} − 1) + bV_{1}(V_{1} − 1) + cV_{0}(V_{0} − 1).

To cancel the quadratic terms, select a = −8, b = −2, c = −1/2. The state equations of the converter are obtained from the equations

V̇_{i} = −\frac{1}{C_{i}} \frac{∂E}{∂V_{i}} (\frac{dg_{i}^{−1}(V_{i})}{dV_{i}})^{−1} i = 0, 1, 2.

In our analysis of A/D neural converters, we observed that while the 2-bit converter yielded correct results, the 3-bit and 4-bit converters showed some deficiency in their performance. The errors observed for these two converters were quite significant and increased with the number of bits and consequently with network size.

16.7 Notes

This chapter gives an introduction to computing using artificial neural networks from a dynamical system perspective. The disclaimer that we found in the book by Hertz, Krogh, and Palmer [135, page xix] also applies to this chapter as well as to the whole field of artificial neural networks, “The adjective “neural” is used because much of the inspiration for such networks comes from neuroscience, not because we are concerned with networks of real neurons. Brain modeling is a different field and, though we sometimes describe biological analogies, our prime concern is with what artificial networks can do, and why.” The readers interested in principles underlying the organization of biological neurons into circuits and networks within different regions of the brain may consult the book edited by Shepherd [298]. For textbooks with an engineering approach to neural networks, we recommend Hassoun [132], Hagan, Demuth, and Beale [125], Zurada [377], Haykin [134], Fausett [99], Khanna [179], or Harvey [131]. Applications of artificial neural networks to signal processing and solving optimization problems are presented in the book by Cichocki and Unbehauen [63]. A psychological slant to the subject is presented in the book by Rumelhart, McClelland, and the PDP Research Group [290], and Anderson [10]. Impatient readers wishing to get a feel of artificial neural networks may consult a tutorial by Lippmann [217], and also by Hush and Horne [154].

According to Haykin [134, page 38], “The modern area of neural networks began with the pioneering work of McCulloch and Pitts (1943). McCulloch was a psychiatrist and neuroanatomist by training; he spent some 20 years thinking about the representation of an event in the nervous system. Pitts was a mathematical prodigy, who joined McCulloch in 1942.” In their paper [236], McCulloch and Pitts, assuming that the activity of the neuron is an “all-or-none” process, developed a logical calculus of neural events. They argued that, with addition of memory, a neural network consisting of a sufficiently large number of their neurons can compute anything that can be computed on any digital computer. McCulloch and Pitts’ paper influenced John von Neumann who used idealized switch-delay elements derived from the McCulloch-Pitts neuron model in the construction of the EDVAC (Electronic Discrete Variable Automatic Computer) that was developed out of the ENIAC (Electronic Numerical Integrator and Computer). The ENIAC was the first general purpose computer and it was invented by J. Presper Eckert and John Mauchly.

More recent investigators of the computational processes in the nervous system compare the brain to a hybrid, analog-digital computer [185]. According to Koch [185, page 210], individual nerve cells communicate between each other via streams of digital pulses. Information is, on the other hand, processed in the analog domain. The resulting signal is then converted back into digital pulses and conveyed to the following neurons.

16.8 Exercises

Exercise 16.1 Write a MATLAB program that implements the TLU (Perceptron) learning algorithm for generating a separating line for points on a plane. Test your program on the following data. For i = 1, …, 4 and d_{i} = 1,

[ x^{(1)} x^{(2)} x^{(3)} x^{(4)} ] = [ \begin{matrix}5 & −7 & 8 & −1 \\ 5 & 6 & −3 & 2 \end{matrix} ] .

The above points constitute class 𝒞_{1}. You can label the above points using, for example, “x”. Class 𝒞_{2} consists of the points corresponding to d_{i} = −1. They are,

[ x^{(5)} x^{(6)} x^{(7)} ] = [ \begin{matrix}−6 & −4 & 7 \\ 1 & −4 & −5 \end{matrix} ]

You can label these points using “o”. The initial weight vector and threshold are

w[0] = [ −5 −1 ]^{⊤} and t[0] = 5.

Plot the initial separating line as well as the final separating line.

Exercise 16.2 Write a MATLAB program that implements the WTA algorithm. Test your program on the following data. The initial weights of the WTA network are:

[ w_{0}^{(1)} w_{0}^{(2)} w_{0}^{(3)} ] = [ \begin{matrix}0.4403 & 0.7411 & −0.8866 \\ 0.8979 & −0.6714 & 0.4625 \end{matrix} ] , α = 0.5.

The training patterns are:

X = [ \begin{matrix}−0.2 & 0.2 & −.1 & 0.98 & 0.98 & .8 & −0.6 & −0.8 & −0.7 \\ 0.98 & 0.98 & .9 & 0.2 & −0.2 & 0.1 & −0.8 & −0.6 & −0.7 \end{matrix} ]

Present the training set 200 times. Find the weights of the network after training.

Exercise 16.3 Use the identification scheme depicted in Figure 16.11 and the neural fuzzy on-line identification algorithm from Section 16.3 to identify the plant described by

y[k + 1] = \frac{y[k](y[k − 1] + 2)y[k] + 2.5)}{8.5 + y[k]^{2} + y[k − 1]^{2}} + u[k],

where

u[k] = 2 cos(2πk/100) for k ≤ 200

and

u[k] = 1.2 sin(2π/20) for k > 200.

The identifier is to be described by

ŷ[k + 1] = f[k] + u[k],

where f has the form of (16.27) with M = 100. Choose α = 0.5, y[0] = y[1] = 0, f[1] = f[2] = 0, and start with u[1] = 0. (The above plant was also used by Lin and Lee [215, page 731] to test their identification scheme.)

Exercise 16.4 Write a MATLAB program that implements the learning algorithm for the RBF identifier discussed in Section 16.4.

Genetic and Evolutionary Algorithms

The word “algorithm” itself is quite interesting; at first glance it may look as though someone intended to write “logarithm” but jumbled up the first four letters. The word did not appear in Webster’s New World Dictionary as late as 1957; we find only the older form “algorism” with its ancient meaning, i.e., the process of doing arithmetic using Arabic numerals.”…“Finally, historians of mathematics found the true origin of the word algorism: it comes from the name of a famous Persian textbook author, Abu Ja‘far Mohammed ibn Mûsâ al-Khowârizmî (c. 825)—literally, “Father of Ja‘far, Mohammed, son of Moses, native of Khowârizm.” ”…“Al-Khowârizmî wrote the celebrated book Kitab al jabr w’al-muqabala (“Rules of restoration and reduction”); another word, “algebra”, stems from the title of his book, although the book wasn’t really very algebraic.”…“The modern meaning for algorithm is quite similar to that of recipe, process, method, technique, procedure, routine, except that the word “algorithm” connotes something just a little different. [184, pages 1–3]

Genetic algorithms, introduced by Holland [139] in the 1960s, have been originally proposed to model adaptive processes, both natural and man-made. In this chapter, we discuss applications of genetic algorithms to solving optimization problems. Genetic algorithms were inspired by biological genetics and evolutionary processes. This is the reason for biological terms being used in the context of genetic algorithms. One of the variants of genetic algorithms, called the evolutionary algorithm is also presented.

17.1 Genetics as an Inspiration for an Optimization Approach

The gene is the basic unit of inheritance. Genes are pieces of DNA, which is the material inside of a nucleus of a cell that carries the genetic instructions for making living organisms. Many traits are determined by genes inherited from the parents of the individual. Consider gene A that determines a certain trait, such as seed coat texture—round or wrinkled. Thus gene A has two alternative forms. These different forms of the same gene are called alleles, or allelic genes. The allelic forms of each gene are symbolized by the uppercase and lowercase of the same letter. In our example, gene A has two alleles, allele A and allele a. One of the alleles controls the round seed coat texture while, the other allele controls the wrinkled texture. Genes are organized into chromosomes, which are small packages of genes in the nucleus of a cell. Different kinds of organisms have different numbers of chromosomes. Humans have 23 pairs of chromosomes including two sex chromosomes. In mating each parent contributes one chromosome to each pair of the chromosomes of an offspring, so the offspring gets half of his or her chromosomes from his or her mother and half from his or her father. The assemblage of genes, encrypted as a string of DNA molecules, is called the genome. The human genome consists of about 35,000 genes. The particular gene constitution is called the genotype. The expression of the genotype is the phenotype, which is the way we look and behave.

Genetic algorithms are a result of an effort to model adaptation phenomena in natural and artificial systems. The term adaptation can be viewed as the composition of two words: ad + aptare meaning “to fit to” [139, page 1]. “Just what are adaptation’s salient features? We can see at once that adaptation, whatever its context, involves a progressive modification of some structure or structures. These structures constitute the grist of the adaptive process, being largely determined by the field of study. Careful observation of successive structural modifications generally reveals a basic set of structural modifiers or operators; repeated action of these operators yields the observed modification sequences.” [139, page 3]

In this chapter, we describe two genetic algorithms and apply them to solving optimization problems. An optimization problem of a real-valued function, f : Ω → ℝ, requires finding an element x ∈ Ω such that f is maximized or minimized. The function f is called the fitness function. Suppose that we wish to maximize f over the parameter space Ω. The parameter space, Ω, of an underlying optimization problem is also called the phenotype space. A classical genetic algorithm works with binary representations of the points from the parameter space, rather than directly with the elements of Ω. We refer to the classical genetic algorithms as the canonical algorithms. The strings of binary digits (1s and 17.1. GENETICS AS AN INSPIRATION FOR AN OPTIMIZATION APPROACH 859

0s) corresponding to these points are called chromosomes. The space of chromosomes is called the genotype space. In general, the chromosomes in the genotype space are abstract mathematical objects on which the so called genetic operators work. After a stopping criterion for the algorithm is satisfied, a decoding function transforms a solution chromosome from the genotype space into the phenotype space. Genetic algorithms are derivative-free optimization algorithms that use only the fitness function. They do not use the fitness function derivatives. The higher the fitness of a chromosome, the better the candidate solution encoded by the chromosome. Genetic algorithms work with populations of chromosomes rather than individual chromosomes. A genetic algorithm is an iterative process that at each iteration uses the information obtained from testing a current population of chromosomes to direct its search into promising areas of the search space. Thus, generating a suitable population of chromosomes at each iteration is the key to the algorithm performance. In the simplest form, creating a new population of chromosomes to be tested involves three genetic operators: selection, crossover, and mutation. These genetic operators work on genotypes that encode phenotypes.

The role of the selection operator is to pick a population of chromosomes for further processing, called reproduction. Chromosomes are selected into a mating pool for reproduction with a probability proportional to their fitness.

The crossover operator randomly exchanges substrings between two parent chromosomes to create two offspring. The simplest crossover operator is the one-point crossover, where a crossing site is randomly picked and the corresponding substrings are exchanged, as illustrated in Figure 17.1. For example, the two parent chromosomes

1010101 and 1111010,

could have the crossing site after the fourth bit in each to produce the two offspring

1010010 and 1111101.

The mutation operator randomly flips an arbitrary bit in a chromosome. For example, the chromosome 010101 might undergo a mutation in its third position to yield a new chromosome 011101.

We now provide a description of the canonical genetic algorithm using the above components.

Algorithm 17.1 Canonical Genetic (GA)

Figure 17.1: The one-point crossover.

For a given optimization problem characterized by a fitness function f: Step 1. Choose a method of encoding candidate solutions into chromosomes, that is, decide upon a map between the phenotype and genotype spaces. (A commonly used encoding method is to concatenate the binary strings representing each real parameter value x_{i}.) Step 2. Start with a randomly generated population of chromosomes. Step 3. Evaluate the fitness of each chromosome in the population. Step 4. Replace the current population with a new population by applying selection, crossover, and mutation operations on the current population. Step 5. If a stopping criterion is satisfied, stop and choose the fittest chromosome from the population as a solution, else go to Step 3.

Another type of a genetic algorithm that we analyze in this chapter attempts to avoid the need for a decoding function by using the chromosome representation as close as possible to their representation in the phenotype space.

17.2 Implementing a Canonical Genetic Algorithm

We now discuss implementation issues of the canonical genetic algorithm (GA) when applied to solving optimization problems. We consider optimization problems of the form:

maximize g(x), x ∈ Ω ⊂ ℝ^{n}.  (17.1)

A maximizing point or maximizer is denoted x^{∗}. We note that there is no loss of generality by considering only maximization problems, for if we are asked to minimize g, we can transform the minimization problem into the maximization problem by means of a simple transformation,

{min \atop x} g(x) = − {max \atop x}(−g(x)).  (17.2)

Also note that if x^{∗} is a minimizer of g(x), then x^{∗} is a maximizer of −g(x), and vice versa. Once an objective function of the maximization problem is given, the next step is to transform the objective function into a fitness function. For example, we can use the transformation,

f(x) = g(x) − g_{min},  (17.3)

where f(x) denotes a fitness function and g_{min} is the minimum value of g(x) for the current population of chromosomes. Note that the constructed fitness function is indeed non-negative on the chromosomes of the current population.

We now discuss the implementation of Step 1 of the canonical GA presented in the previous section. We first analyze the case when the fitness function, f, is a function of only one parameter x. Suppose that

x ∈ [x_{min}, x_{max}] = Ω.

A common encoding method, that transforms points from the phenotype space into a genotype space, is the binary encoding. In our example the phenotype space is the interval [x_{min}, x_{max}]. If we decide to use l bits to represent a point x, then this corresponds to mapping the interval [x_{min}, x_{max}] into the discrete genotype space [0, 2^{l} − 1] consisting of 2^{l} binary strings. Thus, the number of bits l determines the size of the genotype space and hence the resolution, or accuracy, of the subsequent search. We can control the resolution by appropriately choosing the parameter l. Let r denote the desired resolution. Then, the parameter l that results in the required resolution can be found by solving the inequality

r ≥ \frac{x_{max} − x_{min}}{2^{l} − 1} .  (17.4)

If the fitness function depends on many parameters, then we can encode the elements of the phenotype space by concatenating the binary strings representing each component of x = [ x_{1} x_{2} ⋯ x_{n} ]^{⊤}. For example, let f = f(x_{1}, x_{2}), where x_{1} ∈ [x_{1,min}, x_{1,max}] and x_{2} ∈ [x_{2,min}, x_{2,max}]. If we use 10 bits to encode each variable, then the genotype space will consist of 20-bit binary strings and the size of the genotype space is 2^{10} × 2^{10} = 1, 048, 576.

After initializing randomly a population of chromosomes, we evaluate the fitness f(x) of each chromosome x in the population. This requires decoding from the genotype space into the phenotype space. Suppose, as before, that the fitness function depends on two parameters x_{1} and x_{2} and that we used the standard encoding method described above. We now have to convert a given binary string into two real numbers x_{1} ∈ [x_{1,min}, x_{1,max}] and x_{2} ∈ [x_{2,min}, x_{2,max}]. We can perform the required decoding process in three steps. First note that the first l bits of the chromosome x correspond to x_{1} while the remaining bits correspond to x_{2}. We first convert the strings of l bits into integers. Thus, for example, if l = 4 and the chromosome is 10010111, which is an 8-bit string, then the first 4 bits yield

x_{1,int} = 1 × 2^{3} + 0 × 2^{2} + 0 × 2^{1} + 1 × 2^{0} = 9,

while the remaining 4 bits yield

x_{2,int} = 0 × 2^{3} + 1 × 2^{2} + 1 × 2^{1} + 1 × 2^{0} = 7.

Then, we convert x_{1,int} from the range [0, 2^{l} − 1] into the range [0, (x_{1,max} − x_{1,min})] by multiplying x_{1,int} by

\frac{x_{1,max} − x_{1,min}}{2^{l} − 1} .  (17.5)

To obtain the corresponding value of x_{1}, we add x_{1,min} to (17.5) to get

x_{1} = x_{1,min} + x_{1,int} \frac{x_{1,max} − x_{1,min}}{2^{l} − 1} .  (17.6)

We obtain x_{2} proceeding in an analogous manner. In general, if x has n components, the mapping from the genotype space into the phenotype space has the form

x_{i} = x_{i,min} + x_{i,int} \frac{x_{i,max} − x_{i,min}}{2^{l} − 1} , i = 1, 2, …, n (17.7)

The next step of the GA algorithm is the application of the selection operator to pick chromosomes from the current population for reproduction to create the new population. In the canonical GA, a pair of chromosomes is selected from the current population with the probability of selection being proportional to the fitness values of the chromosomes. Selection is done with replacement, which means that the same chromosome can be selected more than once for reproduction. A common method of selecting parent chromosomes is the so called roulette-wheel method. In this method each chromosome is given a slice of a wheel. The area of the wheel slice is proportional to the chromosome’s fitness. The wheel is spun, and the chromosome is selected on whose wheel slice the ball comes to rest. The roulette-wheel method can be implemented in the following steps:

1. Calculate the fitness of each chromosome in the current population;

Figure 17.2: The two-point crossover.

2. Calculate the total fitness of the current population by summing the fitnesses of each chromosome in the current population;

3. Divide the fitness of each chromosome by the total fitness;

4. Calculate the cumulative fitness, labeled for our purposes as cum_fitness, of the current population using the scaled fitness of each chromosome found in the previous step. You can use MATLAB’s command cumsum for this purpose.

5. Return the first population member whose corresponding cumulative fitness is greater than the random number from the interval (0, 1). The following MATLAB command

     can be used here: find(cum_fitness-rand>0), where cum_fitness is the vector of

the scaled cumulative fitnesses found in step 4.

6. Repeat the above step to get the second parent chromosome.

Using the parent chromosomes picked by the selection operator, we form two offspring by crossing over, with probability p_{c}, at a randomly chosen crossing site. The uniform probability p_{c} is called the crossover probability or crossover rate. If no crossover takes place, two offspring chromosomes are formed that are exact copies of their parent chromosomes. One can also use a multi-point crossover operator. The two-point crossover operation is illustrated in Figure 17.2.

The offspring chromosomes obtained from the crossover, are then acted upon by the mutation operator. Mutation is performed at each bit of the offspring chromosomes with probability p_{m}. The probability p_{m} is called the mutation probability or mutation rate. The mutation operator randomly flips some of the bits in a chromosome. For example, the chromosome 101010 might be mutated in its third bit to yield 100010. The mutation probability is usually very small, for example p_{m} = 0.01.

Figure 17.3: A plot of the function given by (17.8).

Each iteration, also called a generation, of the GA replaces the current population with a new one. A typical number of iterations of a GA algorithm is from 50 to 500 or more. The entire set of generations is called a run.

Example 17.1 We apply the canonical GA described above to maximize the function,

f(x) = −15 sin^{2}(2x) − (x − 2)^{2} + 160, |x| ≤ 10.  (17.8)

This function has many local maximizers and minimizers as can be seen from its plot shown in Figure 17.3. The canonical GA was applied to maximize the function given by (17.8) with the chromosomes composed of 12 bits, p_{c} = 0.75, and p_{m} = 0.01. In Figure 17.4 a summary of a typical run is shown. In this run the best candidate solution was x = 1.565324 whose corresponding objective value is f(x) = 159.809259. The best solution that was observed in other runs resulted in f(1.575092) = 159.818346.

In the next section, we attempt to obtain answers to some whys and hows of genetic algorithms.

Figure 17.4: Summary of a sample run of the canonical GA when searching for a maximizer of the function given by (17.8).

17.3 Analysis of the Canonical Genetic Algorithm

In the analysis of genetic algorithms, the notion of a schema is useful. Plural of “schema” is schemas or schemata. A schema is a set of bit strings. It can be described using a template made up of ones, zeros, and asterisks that represent “don’t cares”. Thus, the asterisk ∗ matches either 0 or 1. For example, the schema H = ∗0∗ contains the binary strings 000, 001, 100, and 101. Note that a bit string of length l belongs to 2^{l} different schemas. For example, the string 10 of length l = 2 belongs to the schemas ∗∗, ∗0, 1∗, and 10. Similarly, when l = 3, then any bit string of length l = 3 belongs to 2^{3} = 8 schemas. Elements of a set of N bit strings of length l are members of between 2^{l} and N × 2^{l} schemas. If all the strings are identical, then they belong to 2^{l} schemas, otherwise there may be up to N × 2^{l} schemas to which the N strings of length l can belong.

The parameters that characterize a given schema are its order and defining length. The order of a schema H, denoted o(H), is the number of its fixed bits. For example, the order of H = ∗01 ∗ 1 is o(H) = 3 because H has three bits fixed. The defining length of the schema H, denoted d(H), is the distance between its first and the last fixed bits, which is the difference between the positions of the rightmost fixed symbol and the leftmost fixed symbol. The defining length of H = ∗0 ∗ 1 is d(H) = 4 − 2 = 2. In further discussion, the term schema is used to denote a set of binary strings represented by a template, as well as the template itself.

The capability of a chromosome to solve a given optimization problem, that is, the chromosome performance, is measured by its fitness. Similarly, we can characterize the performance of a schema by its average fitness. We now derive an expression for the average fitness of a given schema. Let x be a chromosome that matches the template H, that is, x ∈ H. Then, we say that “x is an instance of H.” Let H be a schema with at least one instant in the k-th generation. Denote by e(H, k) the number of instances of H in the k-th generation, and let ū(H, k) be the average fitness of H in the k-th generation, that is,

ū(H, k) = \frac{∑_{x∈H} f(x)}{e(H, k)}.  (17.9)

When the genetic algorithm is explicitly evaluating the fitness of N chromosomes from the current generation, it is implicitly evaluating the average fitness of up to N × 2^{l} schemas to which the N chromosomes belong.

We now find E (H, k + 1), the expected number of instances of the schema H in the (k+1)- st generation, given the k-th generation. We first analyze the effect of the selection operator only. Assume that the roulette-wheel method is used in selecting parent chromosomes to the mating pool for reproduction. The probability that the chromosome x is selected to the mating pool, M(k), is

\frac{f(x)}{∑_{i=1}^{N} f (x_{i}^{(k)})},

where x_{i}^{(k)} refers to the i-th chromosome in the k-th generation and f(x) is the fitness of the chromosome x. Therefore, the expected number of chromosomes equal to x that are selected to the mating pool is

N \frac{f(x)}{∑_{i=1}^{N} f (x_{i}^{(k)})} = \frac{f(x)}{f̄(k)},

where

f̄(k) = \frac{1}{N} ∑_{i=1}^{N} f (x_{i}^{(k)})

is the average fitness of the k-th generation. Let m(k) be the number of chromosomes in M(k) that are instances of H. The expected value, M(H, k), of the number of chromosomes in the mating pool that are instances of H is

\frac{∑_{x∈H} f(x)}{f̄(k)} = \frac{∑_{x∈H} f(x)}{f̄(k)} \frac{e(H, k)}{e(H, k)}.  (17.10)

Substituting (17.9) into (17.10) yields an alternative expression for the expected value of the number of chromosomes in the mating pool that are instances of H,

M(H, k) = \frac{ū (H, k)}{f̄(k)}e (H, k)  (17.11)

It follows from the above equation that a schema whose average fitness is greater than the population average, will be found in an increasing number of chromosomes.

We next quantify the effects of the one-point crossover on a given schema. Given two chromosomes, where at least one of them is an instance of the schema H, we say that the schema H survives crossover operation if at least one of the offspring is an instance of H. We will give a lower bound on the probability S_{c}(H) that the schema H will survive the one-point crossover. A schema survives if the crossing site occurs outside the defining length. Crossovers that take place within the defining length can destroy the schema. For example, let H = ∗00 ∗ ∗, and let the parent chromosomes be p_{1} = 10011 and p_{2} = 11100. Note that p_{1} ∈ H, while p_{2} ∉ H. For the crossing site after the third position, there are two offspring chromosomes: s_{1} = 10000, which is an instance of H, and s_{2} = 11111, which is not an instance of H. Thus, the schema H survived this crossover. Suppose now that the crossing site is after the second position, which is within the defining length of the schema H. The offspring chromosomes are s_{1} = 10100 and s_{2} = 11011. None of the offspring chromosome is an instance of H, which means that H was destroyed by the crossover. If both parent chromosomes are instances of H, then the offspring chromosomes are also instances of H irrespective of the location of the crossing site. An upper bound on the probability D_{c}(H) that the schema H will be destroyed by the one-point crossover is

D_{c}(H) ≤ p_{c} \frac{d(H)}{l − 1}.  (17.12)

Therefore, a lower bound on the probability S_{c}(H) that H survives is

S_{c}(H) = 1 − D_{c}(H) ≥ 1 − p_{c} \frac{d(H)}{l − 1}  (17.13)

Thus, the probability of survival under one-point crossover is higher for short schemas.

A schema H survives under mutation of its element, instance, if all defined bits of the instance of H remain unchanged. Let S_{m}(H) denote the probability that the schema H survives under mutation of an instance of H. For each defined bit, the probability that it will not be mutated is 1 − p_{m}. The probability that no defined bit of H will be mutated is obtained by multiplying 1 − p_{m} by itself o(H) times. Therefore,

S_{m}(H) = (1 − p_{m})^{o(H)}  (17.14)

We combine (17.11), (17.13), and (17.14) to obtain a lower bound on the expected number of instances of H in the (k + 1)-st generation, given the k-generation,

E (H, k + 1) ≥ \frac{ū (H, k)}{f̄(k)}e (H, k) (1 − p_{c} \frac{d(H)}{l − 1}) (1 − p_{m})^{o(H)}  (17.15)

The above is known as the schema theorem of Holland. It follows from our discussion that the canonical genetic algorithm implicitly processes, in parallel, schemas. This implicit computation is performed with no operation being explicitly performed on schemas. To illustrate the last statement, consider the string 101. As the fitness of this string is evaluated, partial information is obtained about the expected fitness of all possible schemas to which the evaluated string belongs. In our example, computing the fitness of the string 101 gives partial information about the fitness of strings in the schemas ∗ ∗ ∗, ∗ ∗ 1, ∗0∗, 1 ∗ ∗, ∗01, 1 ∗ 1,….

17.4 Simple Evolutionary Algorithm

Canonical genetic algorithms use binary representations of the elements of the parameter space. The reason behind this is the schema theory, discussed in the previous section, which allows one to analyze canonical genetic algorithms. When using a binary representation of the elements of the parameter space, canonical algorithms use encoding and decoding functions. A binary representation may have some disadvantages though. Indeed, let h : {0, 1}^{l} → Ω denote a decoding function used in the canonical genetic algorithm implementation. Then the original problem of optimizing f : Ω → ℝ is transformed into the problem of optimizing the combined objective function

f(h) : {0, 1}^{l} → ℝ.

The combined objective function to be optimized may be more complex than the original objective function. For example, it may introduce extra nonlinearities or add extra optimizers, which can make the search process for a global optimizer more difficult.

In contrast to the canonical genetic algorithms, in the optimization procedure discussed next, the parameter space is no longer coded using the binary representation. The genetic operators work directly on real-valued vectors. A term used to describe an optimization algorithm using genetic-type operators working on the real-valued vectors is an evolutionary algorithm. In what follows, we present a very simple evolutionary algorithm for solving optimization problems of the form given by (17.1).

Algorithm 17.2 Simple Evolutionary Algorithm (EA)

For a given optimization problem of the form (17.1) perform the following steps: Step 1. Initialize a population of candidate solutions (chromosomes) to the problem. Each chromosome is a real-valued vector of the same length as the vector x of decision variables. Step 2. Evaluate the fitness of each candidate solution in the population. Step 3. Replace the current population with a new population by applying selection, crossover, and mutation operations on the current population. The roulette-wheel selection method can be used for selection. A possible implementation of the crossover operation is to use the average crossover operator. This operator takes two parents, say x_{j} and x_{k} and returns one offspring

x = \frac{1}{2} (x_{j} + x_{k}) .

Another implementation of the crossover operator is a convex combination of two parents. Specifically, let r ∈ (0, 1). Then, the offspring is

x = rx_{j} + (1 − r)x_{k}.

Note that if the feasible set is convex, then the above crossover operators produce chromosomes that are elements of the feasible set. Otherwise conditions should be imposed to ensure that candidate solutions are the elements of the feasible set.

As for the mutation operation, let x be a chromosome to be mutated. Then, an implementation of the mutation operation could have the form,

x′ = x + d,

where d is a randomly generated feasible vector so that x′ is in the feasible set. Step 4. If a stopping criterion is satisfied, terminate the algorithm and choose the fittest chromosome from the population as a solution, else go to Step 2.

In addition, both in the canonical GA, as well as in the evolutionary algorithm, the elitist strategy, introduced by K. DeJong in 1975, can be incorporated. This strategy copies the best chromosomes from the old population into the next population to prevent losing the best solutions in the succeeding iterations. As observed by Mitchell [243], among others, the elitist strategy may lead to the domination of a population by a “super chromosome”. However, the elitist strategy improves genetic and evolutionary algorithms performance. Rudolph [286] showed in 1994 that a canonical GA will never converge to the global optimum unless the best solution in the population, either before or after selection, is copied into the next population.

Figure 17.5: A configuration used to design a fuzzy logic controller using an evolutionary algorithm.

17.5 Evolutionary Fuzzy Logic Controllers

In this section, we employ evolutionary algorithms to design tracking controllers. We describe three design techniques for evolutionary fuzzy logic controllers for the step-lane-change maneuver of a ground vehicle. The presentation of this section is based on [207]. We mention that the step-lane-change maneuver is an important aspect in the design of the Intelligent Vehicle/Highway System (IVHS), the highway of the future.

An essential element in combining an evolutionary algorithm (EA) and a fuzzy logic controller (FLC) is the encoding method used to represent a fuzzy rule base as a chromosome on which the EA operates. Selecting an appropriate encoding procedure is a critical factor that affects the efficiency of the EA and the final solution. Another essential element is the fitness function. Each chromosome defines a fuzzy rule base. After conducting a simulation of the structure depicted in Figure 17.5 and measuring the tracking error, the fitness of the chromosome representing the fuzzy rule base of the simulated FLC is calculated. With all the chromosomes assigned a fitness value, genetic operators yield a new generation and the entire procedure is repeated until a stopping criterion, usually the number of iterations or the fitness value, is satisfied.

17.5.1 Vehicle Model and Control Objective

We use a “bicycle model” of a front-wheel-steering vehicle. Modeling of a ground vehicle is covered in detail in Subsection 2.4.2. The standard coordinate system of the Society of Automotive Engineers (SAE) is used. A schematic of the model used is shown in Figure 17.6. The modeling equations are:

Figure 17.6: Schematic of a bicycle model of a front-wheel-steering vehicle.

Table 17.1: Vehicle data and symbols used in modeling equations.

δ steering angle of the front wheel
F_{yf} lateral force of the front tire
F_{yr} lateral force of the rear tire
m total mass of the vehicle  1280 kg
I moment of the inertia of the vehicle  2500 kg·m^{2}
l_{1} distance from center of gravity to front axle 1.2 m
l_{2} distance from center of gravity to rear axle 1.22 m
v_{x} longitudinal velocity of the vehicle  18.3 m/s

\{ \begin{matrix}ẋ_{1} & = & −v_{x}x_{3} + \frac{1}{m} (F_{yf} + F_{yr}) \\ ẋ_{2} & = & x_{3} \\ ẋ_{3} & = & \frac{2}{I}(l_{1}F_{yf} − l_{2}F_{yr}) \\ ẋ_{4} & = & −v_{x}x_{2} − x_{1}, \end{matrix}

where x_{1} = ẏ is the lateral velocity of the vehicle, x_{2} = θ is the yaw angle, x_{3} = θ̇ is the yaw velocity, and x_{4} = Y is the lateral position of the vehicle. Other parameters and their values are given in Table 17.1. The lateral forces F_{yf} = F_{yf}(α_{f}) and F_{yr} = F_{yr}(α_{r}) are nonlinear functions of respective slip angles with a typical plot of a lateral force versus tire slip angle shown in Figure 2.10. Recall that the slip angles for the front and rear tires are

α_{f} = δ − \frac{l_{1}x_{3} + x_{1}}{v_{x}} and α_{r} = \frac{l_{2}x_{3} − x_{1}}{v_{x}} .

Figure 17.7: SIMULINK block diagram of the closed-loop vehicle system driven by a fuzzy logic controller.

The input is the steering angle δ of the front wheel and the output is the lateral position of the vehicle, x_{4} = Y.

The step-lane-change maneuver along with cruising, traveling path selection, highway exiting and entering is one of the essential elements of automated vehicle control. However, automation of these tasks requires a roadway reference system that involves sensing for lane detection. In our simulation, we assume that the road reference system is already installed. Our objective is to design a fuzzy logic controller that forces the vehicle to track a given reference lateral position signal r. The reference signal is generated by a reference signal generator whose transfer function is 1/(s^{3} + 1.75s^{2} + 2.15s + 1). The tracking error e is the difference between the reference lateral position signal r and the actual lateral position Y of the vehicle. The role of the fuzzy controller is to produce the steering wheel angle δ using the information of the tracking error e and the change in the tracking error ė so that the vehicle tracks the reference signal. A SIMULINK block diagram of the closed-loop vehicle system is depicted in Figure 17.7. In the following, we describe three different methods for an evolutionary algorithm (EA) aided fuzzy logic controller (FLC) design.

17.5.2 Case 1: EA Tunes Fuzzy Rules

In this case, we assume that the membership functions are fixed. In our simulation experiment, we use the membership functions shown in Figure 17.8. The labels of fuzzy sets LN, N, Z, P, and LP denote large negative, negative, zero, positive, and large positive. There are five fuzzy sets for e and five fuzzy sets for ė, hence we can have 25 different rules of the form:

IF e is LN AND ė is LN then δ is SP ⋮ IF e is LP AND ė is LP then δ is SN.

Figure 17.8: Fuzzy membership functions for e, ė, and u.

These rules can be arranged into a rules table as shown in Figure 17.9(a). We mention here that the rules shown in Figure 17.9(a) were arrived at heuristically. Using the rules table, we construct a FLC using the center average defuzzifier given by (15.2). The control surface derived using the rules in Figure 17.9(a) is shown in Figure 17.9(b). Tracking performance for two different initial conditions Y (0) = 5 m and Y (0) = −5 m is illustrated in Figure 17.9(c). We calculated tracking errors for each initial condition and labeled the tracking error for the first initial condition as e_{1} while the tracking error of the second initial condition was labeled as e_{2}. We then constructed the performance index (P I) as the sum of the integrals of the square of the error (ISE) over the simulation time,

P I = ∫_{0}^{50} (e_{1}^{2}(t) + e_{2}^{2}(t)) dt.

The value of the P I in the above experiment is 107.04. The performance index, P I, was evaluated using the circuit shown in Figure 17.10.

Of course, we could include more error trajectories in the expression for the P I. However, for the sake of simplicity, we decided to use only two tracking error trajectories in our simulation experiments.

We next employ an EA to generate a fuzzy rules table automatically. In order to represent the whole fuzzy rules table as a chromosome, we encode the labels of the fuzzy singletons corresponding to the controller outputs as integers, where LN is represented by 1, N by 874 CHAPTER 17. GENETIC AND EVOLUTIONARY ALGORITHMS Figure 17.9: FLC constructed using a fuzzy rule base obtained heuristically: (a) fuzzy rules table (b) control surface (c) tracking performance.

[figure: e; (·)^{2}; \frac{1}{s}; ∫ e^{2}dt]

Figure 17.10: Circuit used to evaluate the performance index, P I.

2, Z by 3, P by 4, and LP by 5. This encoding of the fuzzy singletons of the control signal u is depicted in Figure 17.8. The actual control signal, however, is produced as the weighted sum of the fuzzy singletons using the center average defuzzifier. Because there are 25 rules, each chromosome is a string composed of 25 integers. For example, the rules table of Figure 17.9(a) can be represented as a chromosome of the form,

5 5 5 4 3 5 5 4 3 2 4 4 3 2 2 4 3 2 1 1 3 2 1 1 1

In our experiment, we decided on the population size of 30. The roulette wheel method was used to select chromosomes to the mating pool. We employed the single-point crossover operator with the crossover rate p_{c} = 0.9. The role of the mutation operator was to replace each element of a chromosome randomly with an integer from the set {1, …, 5}. We used the mutation rate p_{m} = 0.05.

We also employed an elitist strategy where two best individuals of the current generation were passed unchanged to the next generation. The fitness function employed was the inverse of the P I discussed above, that is,

F itness = \frac{1}{P I} = \frac{1}{∫_{0}^{50} (e_{1}^{2}(t) + e_{2}^{2}(t)) dt}.  (17.16)

Recall that e_{1}(t) is the tracking error with the initial lateral position at 5 m, and e_{2}(t) is the tracking error with the initial lateral position at −5 m. The input to the reference system was a square wave with the amplitude of 10 m and the frequency of 0.04 Hz. The rules table of the best FLC, after 50 iterations of the EA, is shown in Figure 17.11(a). Figure 17.11(b) shows the fuzzy control surface based on the rules table of Figure 17.11(a). Two different initial conditions, Y (0) = 5 m and Y (0) = −5 m were used to test the tracking performance of the fuzzy controller; Figure 17.11(c) depicts the results. The vehicle lateral position becomes almost indistinguishable from the reference position after about seven seconds. The value of the P I is 97.10, which is an improvement over the performance of the FLC obtained using heuristically generated fuzzy rules.

17.5.3 Case 2: EA Tunes Fuzzy Membership Functions

In this case, fuzzy rules are assumed to be known and fixed. The role of an EA is to tune the input and output membership functions. We use the heuristically obtained rules table of Figure 17.9(a) as the given fuzzy rule base. The fuzzy sets for each input can be completely described using two real numbers if the corresponding fuzzy membership functions are symmetric with respect to the origin and, at each point of their universe of discourse, the 876 CHAPTER 17. GENETIC AND EVOLUTIONARY ALGORITHMS Figure 17.11: The best FLC of the 50-th generation using a fuzzy rule base automatically generated by the EA: (a) fuzzy rules table (b) control surface (c) tracking performance.

Figure 17.12: Input and output fuzzy membership functions of the best FLC of the 50-th generation obtained by tuning fuzzy membership functions.

sum of the values of the membership functions equals 1. The controller output fuzzy sets are also assumed to be symmetric with respect to the origin. Under the above assumptions, the controller output fuzzy singletons can be completely characterized using just two parameters. Hence, six real parameters can completely characterize all the input and output membership functions. So the chromosomes are strings of just six real parameters. For example, the fuzzy sets of Figure 17.8 can be represented as the string,

0.1 0.4 0.1 0.4 2 4

We proceed with a description the EA that operates on the above described fuzzy rule base. The population size, the selection method, the crossover operator and its rate, the mutation operator and its rate, and the fitness function, as well as other parameters, are the same as in the previous case. The resulting fuzzy sets, after 50 iterations of the EA, are shown in Figure 17.12. The control surface and the tracking performance of the closed-loop system driven by the FLC constructed using fuzzy sets of Figure 17.12 are depicted in Figure 17.13. The P I value is 43.51. The tracking error of the lateral position almost disappears in about four seconds.

878 CHAPTER 17. GENETIC AND EVOLUTIONARY ALGORITHMS Figure 17.13: The best FLC of the 50-th generation obtained by tuning fuzzy membership functions: (a) control surface (b) tracking performance.

Figure 17.14: An example of a trapezoidal membership function.

17.5.4 Case 3: EA Tunes Fuzzy Rules and Membership Functions

We observed in the previous cases that changing fuzzy rules or modifying the shape of membership functions can improve the performance of the resulting FLC. So let us analyze an EA that simultaneously operates on fuzzy rules and fuzzy membership functions. This approach is a modification of the technique described in [155, 174].

In general, an i-th fuzzy rule has the form,

IF x_{1} IS F_{1}^{i} AND ⋯ AND x_{n} IS F_{n}^{i} THEN u_{1} IS θ_{1}^{i} AND ⋯ AND u_{m} IS θ_{m}^{i}

where 1 ≤ i ≤ r, x_{j}s are the state variables where 1 ≤ j ≤ n, and u_{k}s being the controller’s outputs, where 1 ≤ k ≤ m. F_{j}^{i}s are fuzzy sets with trapezoidal membership functions, and θ_{k}^{i}s are the controller’s output variables whose membership functions are singletons. A membership function, μ_{F_{j}^{i}}(x_{j}), of the fuzzy set, F_{j}^{i}, can be completely characterized using four parameters c_{ij}, d_{ij}, l_{ij}, and r_{ij}. It can be described as

μ_{F_{j}^{i}}(x_{j}) = \{ \begin{matrix}\frac{x_{j}−l_{ij}}{c_{ij}−l_{ij}−d_{ij}} & if & l_{ij} < x_{j} < c_{ij} − d_{ij} \\ 1 & if & c_{ij} − d_{ij} ≤ x_{j} < c_{ij} + d_{ij} \\ −\frac{x_{j}−r_{ij}}{r_{ij}−c_{ij}−d_{ij}} & if & c_{ij} + d_{ij} < x_{j} < r_{ij} \\ 0 & & otherwise, \end{matrix}  (17.17)

or equivalently,

μ_{F_{j}^{i}}(x_{j}) = max (0, min (1, \frac{x_{j} − l_{ij}}{c_{ij} − l_{ij} − d_{ij}} , −\frac{x_{j} − r_{ij}}{r_{ij} − c_{ij} − d_{ij}} )) .  (17.18)

Figure 17.14 shows a generic fuzzy membership function of the form given by (17.17) or, equivalently by (17.18). The fuzzy singletons, on the other hand, are described as

μ_{θ_{k}^{i}}(z) = \{ \begin{matrix}1 & if z = θ_{k}^{i} \\ 0 & otherwise. \end{matrix}  (17.19)

The controller’s signal u_{k} is calculated using the center average defuzzification formula,

u_{k} = \frac{∑_{i=1}^{r} w_{i}θ_{k}^{i}}{∑_{i=1}^{r} w_{i}} ,  (17.20)

where r is the number of fuzzy rules and the firing strength, w_{i} = μ_{F_{1}^{i}} × μ_{F_{2}^{i}} × ⋯ × μ_{F_{n}^{i}}.

We present a method of representing a given fuzzy rule base as a chromosome. In order to tune fuzzy rules and to alter the shape of membership functions, we require that a chromosome contains all the information pertaining to the structure of the fuzzy rules and the shape of membership functions explicitly. A chromosome is composed of two substructures: one substructure contains the fuzzy rules’ structure matrix while the other substructure contains the information concerning the parameters of the membership functions of the fuzzy rule base. The fuzzy rules’ structure matrix contains the information related to the absence or presence of the premise part of the fuzzy rule corresponding to the particular controller input. Hence the elements of this matrix are from the set {0, 1}.

The parameter matrix contains the parameters that describe fuzzy membership functions. As previously mentioned, there are four parameters for each fuzzy trapezoidal membership function and one for each fuzzy singleton. The parameter matrix has the same number of rows as the number of rules and the number of columns is equal four times the number of the controller inputs plus the number of the controller’s outputs. Figure 17.15(a) shows an example of the rules’ structure matrix and the parameter matrix for the controller with two inputs and one output whose operation is described by two IF-THEN rules. The fuzzy rules and membership functions corresponding to the rules’ structure matrix and the parameter matrix in Figure 17.15(a) are depicted in Figure 17.15(b). Note that a 0 in the second column of the first row of the rules’ structure matrix means that the premise part of the first fuzzy rule corresponding to the second controller input is absent; that is, the first rule does not involve the controller’s second input.

The fitness function has the form,

F itness = \frac{1}{P I + α × (number of rules)}
= \frac{1}{∫_{0}^{50} (e_{1}^{2}(t) + e_{2}^{2}(t)) dt + α × (number of rules)},  (17.21)

where e_{1}(t) is the tracking error with the initial lateral position at 5 m, and e_{2}(t) is the tracking error with the initial lateral position at −5 m. The reference signal used has the amplitude of 10 m and the frequency of 0.04 Hz over 50 seconds. The design parameter α is employed to penalize for the excessive number of fuzzy rules. In our simulations, α = 1.

Figure 17.15: An example of a chromosome structure and its corresponding fuzzy rule base where the controller has two inputs, x_{1} and x_{2}, and one output u.

We next describe genetic operators utilized by the EA. Three genetic operators are used with the probability p_{g}. We first discuss a genetic operator that modifies the number of fuzzy rules. Let r_{new} be the number of fuzzy rules after the genetic operator is applied while r_{old} be the number of fuzzy rules before the genetic operator is applied. Then,

r_{new} = r_{old} + Δr,

where Δr is the correction term that we analyze next. The number of fuzzy rules cannot exceed a prespecified bound r_{max}. It is reasonable to assume that on the average r_{old} is about r_{max}/2. Therefore, we wish to construct the correction term so that

Δr ∈ [−r_{max}/2, r_{max}/2].

Let Ψ be a random variable with standard normal distribution N(0, 1). The probability that Ψ will take its values in the interval (−3, 3) is 0.99. Let integer(·) be the operator that produces the nearest integer value of its argument. Then, the random variable, integer(Ψr_{max}/6), will take its value in the interval (−integer(r_{max}/2), integer(r_{max}/2)) with the probability 0.99. In view of the above, we use the following expression for updating the number of fuzzy rules,

r_{new} = r_{old} + integer(Ψr_{max}/6).  (17.22)

This particular form of the genetic operator ensures that the variation range of the rules’ number is reasonable with respect to the maximum number r_{max} of the rules allowed. If the number of new rules, r_{new}, falls outside the allowed range, then we set r_{new} = r_{old}.

The second genetic operator acts upon the elements of the rules’ structure matrix by taking the binary complement of each element, that is, 1 → 0 and 0 → 1 with probability p_{g}.

The third genetic operator acts upon the parameter matrix as per the following expression,

p_{ij}^{new} = p_{ij}^{old} + η_{ij}Ψe^{1−F_{old}/F_{max}},  (17.23)

where p_{ij}^{old} is the current value of the (ij)-th element of the parameter matrix, p_{ij}^{new} is the updated value of the (ij)-th element of the parameter matrix, Ψ is a random variable with N(0, 1), F_{old} is the fitness value of the chromosome being operated upon, and F_{max} is the maximum fitness value of the chromosomes of the current population. Each design parameter η_{ij} was chosen according to the following recipe,

η_{ij} = (maximal range of p_{ij} − minimal range of p_{ij})/20.

The term e^{1−F_{old}/F_{max}}, in equation (17.23), allows a chromosome with low fitness to vary the elements of the parameter matrix in a larger range than for the chromosomes with high fitness values.

In our implementation of the EA, we employ the elitist strategy to ensure that 20% of the fittest chromosomes in each generation are passed directly to the next generation without being acted upon by the genetic operators. The genetic operators, described above, are applied to the top 20% of the chromosomes in the current generation to produce the remaining 80% of the chromosomes of the new population.

After applying genetic operators to the chromosomes of the current generation, postprocessing may be required to ensure that the resulting chromosomes are feasible. If a zero row in the rules’ structure matrix is created, then that row is deleted and the corresponding row in the parameter matrix is also deleted. The following conditions must be satisfied by the parameter matrix for the chromosome containing this matrix to be feasible,

\begin{matrix}d_{ij} & ≥ & 0 \\ l_{ij} & ≤ & c_{ij} − d_{ij} \\ r_{ij} & ≥ & c_{ij} + d_{ij}. \end{matrix} \}

If the above conditions are violated by a parameter, then that parameter is assigned a boundary value. For example, if d_{ij} < 0, then d_{ij} is set to 0, and if l_{ij} > c_{ij} − d_{ij}, then l_{ij} is set to c_{ij} − d_{ij}, etc.

The above described EA is used to construct a fuzzy logic controller for the front-wheel-steering ground vehicle model, whose dynamics are characterized by a certain type of symmetry that allows us to simplify computations. Specifically, suppose that in the rule base we have a fuzzy rule of the form,

IF x_{1} is F_{1} and x_{2} is F_{2} THEN u is θ,

where the fuzzy sets F_{1} and F_{2} are specified by the membership functions μ_{F_{1}}(c_{1}, d_{1}, r_{1}, l_{1}) and μ_{F_{2}}(c_{2}, d_{2}, r_{2}, l_{2}), respectively. The parameters c_{i}, d_{i}, r_{i}, and l_{i}, i = 1, 2, characterize membership functions of the form shown in Figure 17.14. Because of the symmetry in the vehicle dynamics, the rule base also contains the rule

IF x_{1} is F̃_{1} and x_{2} is F̃_{2} THEN u is −θ,

where F̃_{1} and F̃_{2} have membership functions

μF̃_{1} = μ_{F_{1}}(−c_{1}, d_{1}, −l_{1}, −r_{1}) and μF̃_{2} = μ_{F_{2}}(−c_{2}, d_{2}, −l_{2}, −r_{2}),

respectively.

The size of the population in our simulations was 30, the maximum number of rules was r_{max} = 10, and the probability p_{g} was equal to 0.1. A fuzzy rule base obtained after 100 iterations is shown in Figure 17.16. Only four rules were generated. The fuzzy rule base shown in Figure 17.16 was then used to construct a fuzzy logic controller that was connected to the vehicle model in the closed-loop configuration shown in Figure 17.7. The fuzzy control surface constructed using the obtained fuzzy rule base is shown in Figure 17.17(a). The results of a typical simulation run, the fitness values of the best, average, and worst chromosomes in each generation are shown in Figure 17.17(b), while the tracking performance is illustrated in Figure 17.17(c). As in the previous cases, the simulations were performed for two different initial conditions, and excellent tracking was achieved in both cases. The value of the P I was reduced to 38.06, which is lower than in the two previous designs.

17.6 Notes

Basic principles of genetics have their roots in the experiments performed by a nineteenthcentury monk, Gregor Mendel in his monastery garden in Brno now in the Czech Republic. Mendel worked with the garden pea (Pisum sativum) because it possesses traits that are easily recognizable and appear in widely contrasting forms. In addition the garden pea lends itself well to controlled crosses. Mendel published his findings in 1866 in a paper titled, Figure 17.16: The best fuzzy rule base after 100 iterations of the EA operating on fuzzy membership functions and fuzzy rule base.

“Experiments in Plant Hybridization”. According to Elseth and Baumgardner [95, page 3], “The publication of this paper marked the birth of genetics as a science. The principles stated in the paper still stand today as the cornerstone of modern genetics.” The importance of Mendel’s work went completely unrecognized for 34 years. “In 1900, three botanists, Hugo de Vries (Dutch), Carl Correns (German), and Erich von Tschermak-Seysenegg (Austrian), simultaneously discovered Mendel’s paper on heredity, recognized its importance, and cited it in their own publications.” [95, page 7]. We add that the term gene was coined by W. L. Johannsen in 1903.

A seminal text on genetic algorithms by Holland [139] is definitely worth perusing. However, Holland’s monograph is not intended as an introduction to genetic algorithms. A very well written introductory text to genetic algorithms is Mitchell’s book [243]. A number of engineering design examples using genetic algorithms can be found in Gen and Cheng [113]. Bäck, Hammel, and Schwefel [17] survey the history as well as the current state of evolutionary computation which is the term that encompasses evolution strategies, evolutionary programming, and genetic algorithms. Applications of genetic algorithms to automatic programming using evolving computer programs are described by Koza [189]. Automatic programming with genetic algorithms is called by Koza “genetic programming” (GP). Robust controller design with genetic algorithms are reported by Patton and Liu [266], and Ge, Lee, Figure 17.17: Summary of the simulation run of the closed-loop system driven by the fuzzy logic controller obtained from the fuzzy rule base shown in Figure 17.16: (a) control surface (b) fitness values of the best, average, and worst chromosomes (c) tracking performance.

and Zhu [111]. Kozek, Roska, and Chua [191], Pal and Bhandari [261], and Zamparelli [371] use genetic algorithms to train cellular neural networks. Kim and Myung [180] apply genetic algorithms to solve constrained optimization problems, while Koza et al. [190] employ genetic algorithms for the automatic design of electrical circuits. Yang, Hachino, and Tsuji [362] combine least-squares techniques with a genetic algorithm to on-line identification of timedelay systems. More recently, Loop et al. [220] use genetic algorithms to estimate regions of asymptotic stability of power electronics systems.

17.7 Exercises

Exercise 17.1 Recall that the “elitism” is a genetic operator that forces the GA, or EA, to retain some number of the best individuals from the current population in the new population. Incorporate the elitism operator into your GA program and test it on the function (17.8). You may try, for example, to keep the best two individuals from the current population in the new population.

Exercise 17.2 Implement the simple EA from Section 17.4 and test it on:

(i) the function (17.8);

(ii) the function f(x) = x + | sin(32x)|, where 0 ≤ x < π.

The function in (ii) comes from Mitchell [243, page 9].

Exercise 17.3 Implement the simple EA from Section 17.4 that maximizes the MATLAB’s peaks function

f(x, y) = 3(1 − x)^{2} exp(−x^{2} − (y + 1)^{2}) − 10 (\frac{x}{5} − x^{3} − y^{5}) exp(−x^{2} − y^{2})

−\frac{1}{3} exp(−(x + 1)^{2} − y^{2}),

where −3 ≤ x, y, ≤ 3.

Exercise 17.4 Implement the simple EA from Section 17.4 that maximizes the function

f(x, y) = 3 + y sin(0.7x) + x cos(1.5y);

where −3.0 ≤ x ≤ 12.1 and 4.1 ≤ y ≤ 5.8.

Exercise 17.5 Consider the classic traveling salesman problem (TSP): Suppose a salesman must visit clients in different cities, and then return home. What is the shortest tour through those cities, visiting each one once and only once? Implement in MATLAB a simple variant of the genetic algorithm that optimizes the salesman route for 10 cities. How many different possible paths are there for 10 cities? Following Fogel [105, page 29], use the following representation of any candidate solution: for a given map of the cities to be visited (including the salesman’s home base), assign to each city a positive integer. Thus, for example, if there were six cities, one possible solution might be [ 1 2 3 4 5 6 ]. The above represents an order of progression of the salesman’s trip. Note that because the problem requires a round trip, the first coordinate of the vector representing the first city of the trip is also the last city visited by the salesman. Because the trip is a closed loop, it does not matter from which city the salesman starts his or her trip. As far as a crossover operator implementation is concerned, we recommend using Fogel’s idea [105, page 29], where a singleparent operator produces an offspring by inverting the visiting order between two randomly chosen coordinates of the parent vector. As an example of this implementation of the singleparent crossover operator, suppose that the parent is as above and that the randomly chosen coordinates, that is, the inversion points, are the second and fifth ones. The offspring would then be [ 1 5 3 4 2 6 ].

Plot a map with the cities and mark the obtained salesman’s route. The coordinates of the cities are given in the following table.

Table 17.2: Locations of the cities in Exercise 17.5.

\begin{matrix}x & 0.4306 & 3.7094 & 6.9330 & 9.3582 & 4.7758 & 1.2910 & 4.83831 & 9.4560 & 3.6774 & 3.2849 \\ y & 7.7288 & 2.9727 & 1.7785 & 6.9080 & 2.6394 & 4.5774 & 8.43692 & 8.8150 & 7.0002 & 7.5569 \end{matrix}

Exercise 17.6 Consider a closed-loop system with a fuzzy logic controller (FLC) in the loop shown in Figure 17.18. Suppose that we use five membership functions for the FLC inputs and its output. A possible set of rules is given in Table 17.3, where LN denotes large negative, N means negative, ZE—zero, P—positive, and LP refers to large positive. Implement a variant of the simple EA from Section 17.4 that generates the set of rules to be used to synthesize the FLC. Assign to each fuzzy set a positive integer, for example, let LN be encoded as 1, N as 2, ZE as 3, P as 4, and LP as 5. Then, a candidate solution, that is shown in Table 17.3, can be represented in a vector form as

[ 1 1 2 2 3 1 1 2 3 4 1 2 3 4 5 2 3 4 5 5 3 4 4 5 5 ] ,

that is, a vector representation of the candidate solution given by the rules matrix is obtained using the stacking operation of the columns of the rules matrix. Use the one-point crossover operator similar to the one in the canonical GA. The mutation operation can be implemented by altering a randomly chosen coordinate in the given chromosome, that is, the vector representing a candidate solution. You can use the roulette-wheel method to determine the replacement. For example, if the seventh component was selected to be altered, then you can

Figure 17.18: Closed-loop system with a fuzzy logic controller in the loop.

use the roulette-wheel to determine an integer from the set {1, 2, 3, 4, 5} that will replace the “1” that occupies the seventh coordinate of the above chromosome. Chan, Lee, and Leung [54] used a similar version of the EA to generate fuzzy rules to synthesize a fuzzy logic target tracking algorithm.

Table 17.3: Rules matrix to be used to construct a fuzzy logic controller in Exercise 17.6.

ė LN N ZE P LP
e
LN LN LN LN N ZE
N LN LN N ZE P
ZE N N ZE P P
P N ZE P LP LP
LP ZE P LP LP LP

Chaotic Systems and Fractals

An apparent paradox is that chaos is deterministic, generated by fixed rules which do not themselves involve any elements of change. We even speak of deterministic chaos. In principle, the future is completely determined by the past; but in practice small uncertainties, much like minute errors of measurement which enter into calculations, are amplified, with the effect that even though the behavior is predictable in the short term, it is unpredictable over the long run. [270, page 11]

18.1 Chaotic Systems Are Dynamical Systems With Wild Behavior

The objective of this chapter is to show the reader “the wild things that simple nonlinear equations can do” [232, page 459]. These simple nonlinear models with very complicated dynamics are examples of chaotic systems, which are deterministic nonlinear dynamical systems that exhibit a random-like behavior. A discrete-time model of population dynamics discussed in the next section, Newton’s method for finding roots of certain polynomials, or a jetting balloon after its release are examples of dynamical chaotic systems. Trajectories of chaotic dynamical systems are sensitive to initial conditions in the sense that starting from slightly different initial conditions the trajectories diverge exponentially. Another property of a chaotic dynamical system is loss of information about initial conditions. We follow Moon [246, page 4] to explain this property. Suppose that we are given a state-plane portrait of a chaotic dynamical system in the (x, y)-plane. Assume that we specify the x-coordinate with accuracy Δx and the y-coordinate with accuracy Δy. Then, we could form a grid in Figure 18.1: An illustration of the growth of uncertainty in a chaotic dynamical system.

the (x, y)-plane, shown in Figure 18.1, where each area would be of size Δx Δy. Thus, if we select initial conditions to the stated accuracy, then we know that these initial conditions are located somewhere in the shaded box in the (x, y)-plane corresponding to the initial time t_{0}. For a chaotic system this uncertainty grows in time to, say, N boxes at a later time t_{1}, as illustrated in Figure 18.1. The uncertainty growth, as mentioned above, is exponential and for continuous-time systems can be described by

N(t) ≈ e^{μ(t−t_{0})}N(t_{0}),  (18.1)

where the constant μ is a measure of sensitivity with respect to initial conditions. Lyapunov exponents, that we discuss in a subsequent section, can be used to measure the initial condition sensitivity. Chaotic behavior can arise in low-order nonlinear dynamical systems. In the case of continuous-time systems, chaotic behavior can only occur for systems of dimension three or higher. However, in discrete-time systems, chaos can occur even in one-dimensional systems, as we show in the following section.

18.2 Chaotic Behavior of the Logistic Equation

18.2.1 The Logistic Equation—An Example From Ecology

The so called logistic equation, or logistic map, whose behavior we analyze in this section, arises, for example, in the analysis of the growth of a population.

Definition 18.1 A map is a function whose domain (input) space and range (output) space are the same. Let x(0) be a point in ℝ^{n} and f be a map on ℝ^{n}. The orbit, or trajectory, of x(0) under f is the set of points

{x(0), f(x(0)), f^{2}(x(0)), …, f^{k}(x(0)), …} ,

where f^{k} is the k-th iterate of f. The starting point, x(0), of the orbit is called the initial value of the orbit.

◇

We first analyze a situation where there are no predators and there is an unlimited supply of food. For the sake of argument, we consider a population of rabbits. Assume that on the average each rabbit gives birth to σ new rabbits in one unit of time, and that no rabbit dies. Let N(k) be the number of rabbits at time t = k, then the change in the population over a time period, between k and k + 1 is

N(k + 1) − N(k) = σN(k).  (18.2)

Thus,

N(k + 1) = (1 + σ)N(k).  (18.3)

Equation (18.3) is a very simple linear model of population growth. Given the number N(0) of rabbits at time k = 0, the number of rabbits at time k is

N(k) = (1 + σ)^{k}N(0).  (18.4)

One can see from (18.4) that the linear model given by (18.3) predicts the exponential grow of population. We now try to make the above model more realistic. We assume, as before, that the number of births is proportional to the size of the population. Let b be the birth rate, then the number of births at time k is bN(k). We also assume that the number of deaths is proportional to the population size. Let d be the death rate. Then, the number of deaths at time k is dN(k). The change in population in a time period between times k and k + 1 is

N(k + 1) − N(k) = bN(k) − dN(k).  (18.5)

We rewrite equation (18.5) as

N(k + 1) = (1 + b − d)N(k).  (18.6)

Let σ = b − d. Then, equation (18.6) has the same form as equation (18.3). In both cases, the population grows exponentially. We refer to the constant σ as the unrestricted growth rate. Over short periods of time, the above equation may be sufficient for purposes of an analysis of a population. Over long periods of time, the growth rates may change as the size of a population changes.

We alter the above linear models by introducing the concept of saturation. We first assume that the environment of the population can only support a certain number of species. We denote this number by K, and call it the carrying capacity. Thus, if N(k) > K, then there will not be enough food or space available, and more species will die than will be born. In such a situation the growth rate should be negative. If the population at time k is N(k) < K, then the growth rate should be positive. Finally, if N(k) is much smaller than the carrying capacity, then the growth rate should be close to the unrestricted growth rate σ. A possible function that satisfies the above requirements has the form

σ (1 − \frac{N(k)}{K}) .  (18.7)

The change in the size of population then can be described as

N(k + 1) − N(k) = σ (1 − \frac{N(k)}{K}) N(k),  (18.8)

or equivalently

N(k + 1) = (1 + σ)N(k) − \frac{σ}{K} N^{2}(k).  (18.9)

Equation (18.9) is called the logistic equation. The term \frac{σ}{K} N^{2}(k) is referred to as the damping term, because it dampens the growth of the population. To simplify (18.9), we scale the variable N(k) by introducing the new variable

x(k) = \frac{σ}{ρ K} N(k),

where

ρ = 1 + σ.

Then, equation (18.9) becomes

x(k + 1) = ρx(k)(1 − x(k))  (18.10)

We will show that if 0 < ρ ≤ 4 and 0 ≤ x(0) ≤ 1, then 0 ≤ x(k) ≤ 1 for k = 1, 2, . . .. Let

f(x) = ρx(1 − x).

Then,

\frac{df(x)}{dx} = ρ − 2ρx = 0

for x = 1/2 and f_{max}(x) = ρ/4. Hence, for k = 1, 2, …,

0 ≤ x(k) ≤ 1 if 0 < ρ ≤ 4 and 0 ≤ x(0) ≤ 1.

18.2.2 Stability Analysis of the Logistic Map

We begin our investigation of the logistic equation (18.10) by seeking its equilibrium states (fixed points). These can be found by setting x(k + 1) equal to x(k) in (18.10) to obtain

x(k) (ρx(k) + (1 − ρ)) = 0.

It follows that the equilibrium solutions are

x^{∗} = 0 and x^{∗} = \frac{ρ − 1}{ρ}.

We study attracting properties of the fixed points as a function of the parameter ρ. Roughly speaking, a fixed point x^{∗} = f(x^{∗}) of the system x(k + 1) = f(x(k)) is attracting if, for a starting point close to x^{∗}, the next iterate is still closer. Formally, a fixed point x^{∗} is attracting if there is an ε > 0 such that when |x(0) − x^{∗}| < ε, then lim_{k→∞} x(k) = x^{∗}.

A fixed point x^{∗} is repelling if for any ε > 0 no matter how small, if 0 < |x(0) − x^{∗}| < ε, then |x(k) − x^{∗}| > ε for some k.

Theorem 18.1 Suppose x^{∗} = f(x^{∗}) is a fixed point of x(k + 1) = f(x(k)), where f is continuously differentiable. Then, x^{∗} is attracting if

|f′(x^{∗})| < 1,

and is repelling if

|f′(x^{∗})| > 1.

The case when |f′(x^{∗})| = 1 is inconclusive.

Proof We first show that if x^{∗} = f(x^{∗}) is such that |f′(x^{∗})| < 1, then x^{∗} is attracting. Since by assumption |f′(x^{∗})| < 1 and f′ is continuous, there exists an interval (x^{∗} −ε, x^{∗} +ε) about x^{∗} such that for some L > 0,

|f′(x^{∗})| < L < 1 and |f′(x)| < L for every x ∈ (x^{∗} − ε, x^{∗} + ε).

Let x(k) ∈ (x^{∗} − ε, x^{∗} + ε). We will show that x(k + 1) is closer to x^{∗} than x(k). We have

|x(k + 1) − x^{∗}| = |f(x(k)) − f(x^{∗})|

because x^{∗} = f(x^{∗}) is a fixed point. Using the mean value theorem yields

|x(k + 1) − x^{∗}| = |f(x(k)) − f(x^{∗})| = |f′(x̃)| |x(k) − x^{∗}|

for some x̃ ∈ (x(k), x^{∗}) or x̃ ∈ (x^{∗}, x(k)) if x^{∗} < x(k). Since x(k) ∈ (x^{∗} − ε, x^{∗} + ε), then x̃ ∈ (x^{∗} − ε, x^{∗} + ε) also. Thus, |f′(x̃)| < L < 1, and therefore

|x(k + 1) − x^{∗}| < L |x(k) − x^{∗}| .

From the above, we conclude that if x(0) ∈ (x^{∗} − ε, x^{∗} + ε), then

|x(k) − x^{∗}| < L^{k} |x(0) − x^{∗}| .

Because L < 1, we have lim_{k→∞} |x(k) − x^{∗}| = 0, that is,

lim_{k→∞} x(k) = x^{∗},

which means that x^{∗} is an attracting fixed point.

Similarly, if x^{∗} = f(x^{∗}) is such that |f′(x^{∗})| > 1, then it is a repelling fixed point. Indeed, if |f′(x^{∗})| > 1, then because f′ is continuous, there is an interval (x^{∗} − ε, x^{∗} + ε) about x^{∗} and a constant L > 1 such that |f′(x)| > L for all x ∈ (x^{∗} − ε, x^{∗} + ε). We will now show that if x(k) ∈ (x^{∗} − ε, x^{∗} + ε), then x(k + 1) will be further away from x^{∗} than x(k). Using the mean value theorem, we obtain

|x(k + 1) − x^{∗}| = |f(x(k)) − f(x^{∗})|
= |f′(x̃)| |x(k) − x^{∗}|
> L |x(k) − x^{∗}| .  (18.11)

If x(k + 1) is no longer in (x^{∗} − ε, x^{∗} + ε), we are done. Otherwise, we repeat the process to get

|x(k + 2) − x^{∗}| = |f(x(k + 1)) − f(x^{∗})|
> L |x(k + 1) − x^{∗}|
> L^{2} |x(k) − x^{∗}| .

If x(k + 2) is no longer in the interval (x^{∗} − ε, x^{∗} + ε), we are done. If not, we repeat the process to obtain

|x(k + m) − x^{∗}| > L^{m} |x(k) − x^{∗}| .

Since (x^{∗} − ε, x^{∗} + ε) is a finite interval and L^{m} → ∞ as m → ∞, at some time, x(k + m) must leave the interval, which means that x^{∗} is a repelling fixed point. The proof of the theorem is complete.

□

Equation (18.11) tells us that the trajectory {x(k)} diverges exponentially from x^{∗} as k → ∞. Thus, a trajectory with an initial condition close to x^{∗} for the case when |f′(x^{∗})| = L > 1 will separate from x^{∗} at a multiplicative rate of approximately L > 1 per iteration until it moves significantly away from x^{∗}. In other words, the distance between f(x(k)) starting from x(0) close to x^{∗} will grow exponentially, that is, this distance will be magnified approximately by L for each iteration. To quantify this multiplicative rate of separation of points close to x^{∗}, the Lyapunov exponent term is introduced that we discuss in Section 18.4.

We now consider the logistic equation (18.10). Recall that f = ρx(1 − x) denotes the right hand side of the logistic equation. The derivative of f with respect to x is

f′(x) = ρ(1 − 2x).

Therefore, f′(0) = ρ. Since |f′(0)| < 1 when ρ < 1, by Theorem 18.1, the fixed point x^{∗} = 0 is attracting for ρ < 1 and repelling for ρ > 1. The fixed point x^{∗} = \frac{ρ−1}{ρ} is outside of the range of x for ρ < 1. Thus, if 0 < ρ < 1, we have just one attractive fixed point. For ρ > 1, the fixed point x^{∗} = 0 is repelling and x^{∗} = \frac{ρ−1}{ρ} comes in within the range of the variable x. The derivative of f evaluated at x^{∗} = \frac{ρ−1}{ρ} is

f′ (\frac{ρ − 1}{ρ}) = 2 − ρ.

By Theorem 18.1, the fixed point x^{∗} = \frac{ρ−1}{ρ} is attracting if |2 − ρ| < 1, that is, if

1 < ρ < 3.

Thus, for ρ = 1 there is an exchange of attracting properties from one fixed point to the other. This change in the qualitative behavior at a certain parameter value is called a bifurcation. In Figure 18.2, the fixed points x^{∗} = 0 and x^{∗} = \frac{ρ−1}{ρ} are shown and the ranges of ρ for which they are attracting. For ρ > 3, there are no attracting fixed points, and the solutions of the logistic equation become very complex. We discuss the logistic equation dynamics for ρ > 3 after displaying its solutions for ρ = 0.7, ρ = 1.4, ρ = 2.8, and ρ = 3.6 in Figure 18.3. The initial condition in all four cases was x(0) = 0.9. Another way of displaying a solution of a first-order difference equation is a cobweb. To construct a cobweb for the given first-order discrete system, we first draw the function y = f(x) representing the right-hand side of the difference equation. Then, we draw the straight line y = x. The fixed points correspond to the points of intersection of these two graphs. The solution sequence, represented by the cobweb, staring at x(0) is obtained as follows. Starting from x(0), draw upward vertical line to intersect the graph of f(x). This corresponds to the calculation of x(1). The value of x(1)

896 CHAPTER 18. CHAOTIC SYSTEMS AND FRACTALS

Figure 18.2: Exchange of attracting properties of the fixed points for 0 < ρ ≤ 3.44. Figure 18.3: Solutions of the logistic equation: (a) ρ = 0.7, (b) ρ = 1.4, (c) ρ = 2.8, and (d) ρ = 3.6.

Figure 18.4: Cobwebs for the logistic equation: (a) ρ = 0.7, (b) ρ = 1.4, (c) ρ = 2.8, and (d) ρ = 3.6.

is transferred to the straight line y = x. The x-coordinate of the intersection is x(1) which is then used to generate x(2) using the above procedure. The process is repeated over and over again. In Figure 18.4, the cobwebs corresponding to the solutions displayed in Figure 18.3 are presented.

As we mentioned before, for ρ > 3 there are no attracting fixed points. As ρ increases, the solutions of the logistic equation exhibit increasing complexity. For ρ just slightly larger than 3, the solution sequence settles down into a steady oscillation of period 2 as shown in Figure 18.5. Then, as ρ is further increased, periodic solutions of period 4, 8, 16, … appear. In Figure 18.6, a solution of period 4 is shown corresponding to ρ = 3.5. For ρ = 3.9, the solution is shown in Figure 18.7.

We next analyze the solutions for ρ > 3. To simplify the analysis, we first make the substitution

x(k) = z(k) + \frac{ρ − 1}{ρ},  (18.12)

which is equivalent to shifting the fixed point x^{∗} = \frac{ρ−1}{ρ} to the origin in the (ρ, z)-coordinates. Note that

x(k + 1) = z(k + 1) + \frac{ρ − 1}{ρ}.  (18.13)

898 CHAPTER 18. CHAOTIC SYSTEMS AND FRACTALS Figure 18.5: A two-cycle solution and its cobweb for ρ = 3.3 and x(0) = 0.1. Figure 18.6: A period-four solution and its cobweb for ρ = 3.5 and x(0) = 0.1.

Figure 18.7: A periodic solution and its cobweb for ρ = 3.9 and x(0) = 0.1.

Substituting (18.12) and (18.13) into the logistic equation (18.10) yields

z(k + 1) + \frac{ρ − 1}{ρ} = ρ (z(k) + \frac{ρ − 1}{ρ}) (1 − z(k) − \frac{ρ − 1}{ρ}) .  (18.14)

After performing some manipulations, we represent (18.14) as

z(k + 1) = (2 − ρ)z(k) − ρz^{2}(k)  (18.15)

The fixed points of (18.15) are z^{∗} = 0 corresponding to x^{∗} = \frac{ρ−1}{ρ}, and z^{∗} = \frac{1−ρ}{ρ} corresponding to x^{∗} = 0. We investigate the attracting properties of the fixed point z^{∗} = 0 for ρ ≥ 0. We see from Figure 18.5 that for ρ just greater than 3, the solution of the logistic equation goes to an attracting period-two cycle. Two numbers p and q form a period-2 cycle for a first-order discrete systems if, when x(k) = p, then x(k + 1) = q, x(k + 2) = p, and so on. A period-2 cycle is attracting if there are intervals (p_{1}, p_{2}) and (q_{1}, q_{2}) about p and q, respectively, such that

(i) if x(0) ∈ (p_{1}, p_{2}), then lim_{k→∞} x(2k) = p and lim_{k→∞} x(2k + 1) = q;

(ii) if x(0) ∈ (q_{1}, q_{2}), then lim_{k→∞} x(2k) = q and lim_{k→∞} x(2k + 1) = p.

The fixed points p and q of the period-2 cycle of equation (18.15) can be found by solving the equation

z = g(g(z)),  (18.16)

where g = (2 − ρ)z − ρz^{2}. We can use MATLAB’s function compose to form (18.16), and then use the function solve to solve (18.16). We obtain four roots. Two of them are the fixed points that we analyzed above. The remaining two roots are the fixed points of the period-2 cycle. They are:

p = −\frac{1}{2ρ} (ρ − 3 + \sqrt{ρ^{2} − 2ρ − 3}) ,  (18.17)

and

q = −\frac{1}{2ρ} (ρ − 3 − \sqrt{ρ^{2} − 2ρ − 3}) .  (18.18)

Once we have found the 2-cycle, we next determine the range of ρ for which it is attracting. A 2-cycle p and q of the system z(k + 1) = g(z(k)) is attracting if the fixed points p and q of the map y(k + 1) = h(y(k)) are attracting, where h(y) = g(g(y)). By Theorem 18.1, the fixed points p = h(p) and q = h(q) are attracting if |h′(p)| < 1 and |h′(q)| < 1. Using the chain rule, we obtain

h′(y) = \frac{d}{dy} g(g(y)) = g′(g(y))g′(y).  (18.19)

Since g(p) = q and g(q) = p, we have

h′(p) = g′(g(p))g′(p) = g′(q)g′(p).  (18.20)

Similarly,

h′(q) = g′(p)g′(q).  (18.21)

Therefore, |h′(p)| < 1 and |h′(q)| < 1 when

|g′(p)g′(q)| < 1.  (18.22)

Using Theorem 18.1 and the arguments similar to the ones above, we conclude that the 2-cycle represented by p and q is repelling if

|g′(p)g′(q)| > 1.  (18.23)

We next substitute (18.17) and (18.18) into g′(p)g′(q), and evaluate the resulting expression using MATLAB’s function subs, to obtain

g′(p)g′(q) = (2 − ρ − 2ρp) (2 − ρ − 2ρq) = 4 − ρ^{2} + 2ρ.  (18.24)

Thus, the 2-cycle represented by the pair (p, q) is attracting if

−1 < 4 − ρ^{2} + 2ρ < 1,  (18.25)

Figure 18.8: Bifurcation diagram of the logistic equation.

where ρ > 3. Solving (18.25) yields

3 < ρ < 1 + \sqrt{6}.  (18.26)

When ρ > 1 + \sqrt{6}, an attracting 4-cycle appears. As can be seen from the bifurcation diagram shown in Figure 18.8, that as the value of ρ increases the period doubling process takes place. Each of the branches of the bifurcation diagram produces a period-2 cycle of its own. The process continues as the parameter ρ approaches the value ρ = 3.5699, until no periodic cycles are created. For ρ = 3.5699 the behavior is not cyclic anymore. The behavior of the solutions of the logistic equation for ρ ≥ 3.5699 is complex indeed, as can be seen, for example, by inspecting the bifurcation diagram shown in Figure 18.8. Various windows of ranges of ρ > 3.5699 correspond to different periodicities of solutions as well as to aperiodic solutions. These aperiodic solutions are called chaotic solutions or chaotic motions. Here, we quote a descriptive definition of chaotic motion due to Thompson and Stewart [327, page 197]. “... one way to define chaos is in the negative—recurrent behavior that is not an equilibrium, a cycle or even a quasi-periodic motion—there is more to be said about chaos. Chaotic motion has some aspect that is provably as random as a coin toss. The randomness arises from sensitive dependence on imperfectly known initial conditions, resulting for example in broadband noise in the power spectra of observable time series. This seems remarkable because the dynamical system needs no stochastic input to achieve this”. A precise definition of chaotic motion is given in Section 18.4.

18.2.3 Period Doubling to Chaos

We describe one of the ways in which chaotic motion can arise—by repeated flip bifurcations, that is, through period doubling. Such a behavior arises for 3 ≤ ρ ≤ 3.5699. We show that there is a sequence of numbers

c_{1} < c_{2} < ⋯ < c_{n} < ⋯  (18.27)

such that when

c_{n} < ρ < c_{n+1},

then the logistic equation has an attracting 2^{n}-cycle. Thus, for example, if c_{8} < ρ < c_{9}, then the solution will settle into an attracting 2^{8}-cycle. We already have c_{1} = 3 and c_{2} = 1+\sqrt{6} = 3.4495. We are interested in finding the remaining elements of the sequence (18.27). The sequence (18.27) is convergent, where

lim_{n→∞} c_{n} = c_{∞} = 3.5699.

To find the elements of the sequence (18.27), we could, in principle, form composite maps f^{2^{n}}(x) and use stability results from the previous subsection. We have observed that if the parameter ρ is slightly bigger than 1 + \sqrt{6}, then the 2-cycle is repelling and the solution goes to an attracting 4-cycle. A cobweb of an attracting 4-cycle is shown in Figure 18.9. We could find the four points corresponding to an attracting 4-cycle by solving the algebraic equation

x = f(f(f(f(x)))),  (18.28)

where f(x) = ρx(1 − x). A graph of the function y = f^{4}(x) is shown in Figure 18.10. The solution to (18.28) gives eight points. Four of these points, corresponding to y = f^{4}(x) having negative slopes when intersecting the line y = x, form an attracting 4-cycle. Using results from the previous subsection, we could find the range of ρ resulting in attracting 4-cycle solutions. We could then proceed to find the range of ρ corresponding to solutions going to attracting 8-cycles, then 16-cycles, and so forth. However, this approach is only feasible for n = 1 and n = 2. We have to take a different approach to compute the elements of the sequence (18.27). We use an iteration technique consisting of four steps:

(1) Finding the second iterate, z(k + 1) = g(g(z(k)), of the logistic map (18.15).

(2) Shifting one of the points of the 2-cycle of the second iterate to the origin.

(3) Neglecting cubic and quartic terms in the shifted map.

18.2. CHAOTIC BEHAVIOR OF THE LOGISTIC EQUATION 903

Figure 18.9: Cobweb of an attracting 4-cycle for ρ = 3.5.

Figure 18.10: A graphical solution of x = f^{4}(x), where f(x) = 3.5x(1 − x). Marked points, corresponding to slopes less than one, form an attracting 4-cycle.

(4) Scaling the map obtained in step (3).

Performing iterations consisting of the above four steps will allow us to find the elements of the sequence (18.27).

The second iterate of (18.15) is

z(k + 2) = (2 − ρ)z(k + 1) − ρz^{2}(k + 1)
= (2 − ρ) ((2 − ρ)z(k) − ρz^{2}(k)) − ρ ((2 − ρ)z(k) − ρz^{2}(k))^{2}
= (2 − ρ)^{2}z(k) − ((2 − ρ)ρ + ρ(2 − ρ)^{2}) z^{2}(k)
+2ρ^{2}(2 − ρ)z^{3}(k) − ρ^{3}z^{4}(k).  (18.29)

The points p and q, given by (18.17) and (18.18), are fixed points of the map (18.29). We select one of the points, p or q, to work with. Suppose that we choose the point p. Consider the transformation

v(k) = z(k) − p.  (18.30)

Note that

v(k + 2) = z(k + 2) − p.  (18.31)

Substituting (18.30) and (18.31) into (18.29), performing manipulations, and neglecting terms of order three and four, we obtain

v(k + 2) = (4 + 2ρ − ρ^{2}) v(k) − ρ (ρ^{2} − 2ρ − 3 − 3\sqrt{ρ^{2} − 2ρ − 3}) v^{2}(k).  (18.32)

One can easily verify the above result using MATLAB’s function subs applied to (18.29)– (18.31). We will be studying fixed points of (18.32). We replace k + 2, the argument of v on the left-hand side of (18.32), with k + 1. Let

C_{1}(p) = ρ^{2} − 2ρ − 3 − 3\sqrt{ρ^{2} − 2ρ − 3}.  (18.33)

Then, (18.32) can be represented as

v(k + 1) = (4 + 2ρ − ρ^{2}) v(k) − ρC_{1}(p)v^{2}(k).  (18.34)

Note that we if we used q in (18.30) instead of p, we would get

v(k + 1) = (4 + 2ρ − ρ^{2}) v(k) − ρC_{1}(q)v^{2}(k),  (18.35)

where now C_{1}(q) = ρ^{2} − 2ρ − 3 + 3\sqrt{ρ^{2} − 2ρ − 3}. In the next step, we transform the v-map given by (18.34) into the same format as the z-map given by (18.15). This allows us to apply stability properties of the z-map to the v-map. We introduce an iterated parameter ρ_{1} such that

2 − ρ_{1} = 4 + 2ρ_{0} − ρ_{0}^{2},  (18.36)

and represent (18.34) as

v(k + 1) = (2 − ρ_{1}) v(k) − ρ_{0}C_{1}(p)v^{2}(k).  (18.37)

Finally, we scale (18.37) using the transformation

s(k) = \frac{ρ_{0}C_{1}(p)}{ρ_{1}}v(k).  (18.38)

Substituting (18.38) into (18.37) gives

\frac{ρ_{1}}{ρ_{0}C_{1}(p)}s(k + 1) = \frac{ρ_{1}(2 − ρ_{1})}{ρ_{0}C_{1}(p)}s(k) − \frac{ρ_{1}^{2}ρ_{0}C_{1}(p)}{ρ_{0}^{2}C_{1}^{2}(p)}s^{2}(k).  (18.39)

Simplifying (18.39) yields

s(k + 1) = (2 − ρ_{1}) s(k) − ρ_{1}s^{2}(k)  (18.40)

The above s-map has the same format as the z-map given by (18.15) and analyzed in the previous subsection. It follows from these results that the s-map given by (18.40) has an attracting 2-cycle for 3 < ρ_{1} < 1 + \sqrt{6}, which corresponds to an attracting 4-cycle of the z-map given by (18.15). Using (18.36), we conclude that 3 < ρ_{1} < 1 + \sqrt{6} corresponds to

1 + \sqrt{6} < ρ_{0} < 1 + \sqrt{4 + \sqrt{6}} = 3.5396.

Indeed, representing (18.36) as

ρ_{0}^{2} − 2ρ_{0} − (2 + ρ_{1}) = 0,

and using the quadratic formula gives

ρ_{0} = 1 + \sqrt{3 + ρ_{1}},

where we took the solution that gives ρ_{0} > 0. The above concludes the first iteration. The result of this iteration is the element c_{3} = 1+\sqrt{4 + \sqrt{6}} = 3.5396 of the sequence (18.27) and the scaling parameter C_{1}(p) whose properties will be investigated in the following subsection.

We then proceed with the next iteration. In particular, executing the steps of the iteration method on the s-map yields the new map that we call the t-map. This map has the form

t(k + 1) = (4 + 2ρ_{1} − ρ_{1}^{2}) t(k) − ρ_{1}C_{2}t^{2}(k).  (18.41)

Introducing the iterated parameter, ρ_{2}, such that

ρ_{2} = ρ_{1}^{2} − 2ρ_{1} − 2,  (18.42)

and scaling the t-map using the transformation

u(k) = \frac{ρ_{1}C_{2}}{ρ_{2}} t(k)

yields the u-map of the form

u(k + 1) = (2 − ρ_{2}) u(k) − ρ_{2}u^{2}(k).  (18.43)

The above map has an attracting 2-cycle for 3 < ρ_{2} < 1 + \sqrt{6} which corresponds to an attracting 4-cycle of the s-map, which in turn corresponds to an attracting 8-cycle of the z-map. We calculate the range of the parameter ρ corresponding to attracting 8-cycles of the z-map by working the recursive relation (18.42) to get

c_{4} = 1 + \sqrt{3 + ρ_{1}} = 1 + \sqrt{3 + 1 + \sqrt{3 + ρ_{2}}} = 1 + \sqrt{4 + \sqrt{4 + \sqrt{6}}} = 3.5573.

In general, after i + 1 iterations we obtain

ρ_{i+1} = ρ_{i}^{2} − 2ρ_{i} − 2,  (18.44)

and work backwards to find the term c_{i+3} of the sequence (18.27). We can also compute the elements of the sequence (18.27) using the recursive relation

ρ_{i} = ρ_{i+1}^{2} − 2ρ_{i+1} − 2, i = 1, 2, ….  (18.45)

Solving the above, using the quadratic formula, yields

ρ_{i+1} = 1 + \sqrt{3 + ρ_{i}}.

This means that the elements of the sequence (18.27) satisfy the difference equation

c_{i+1} = 1 + \sqrt{3 + c_{i}}, c(1) = 3.  (18.46)

The equation (18.46) has two equilibrium points that are obtained by solving the algebraic equation

c = 1 + \sqrt{3 + c}.

The equilibrium point

c_{eq} = \frac{3 + \sqrt{17}}{2}

is attracting, and we have

lim_{i→∞} c_{i} = c_{∞} = c_{eq} = 3.5616.

An analysis without truncation when computing second iterates of the logistic equation yields c_{∞} = 3.5699. Such an analysis can be found in the paper by Feigenbaum [100]. After the parameter ρ reaches the limiting value ρ = 3.5699, no stable solution patterns of the logistic equation emerge. We obtain a chaotic behavior.

18.2.4 The Feigenbaum Numbers

The sequence (18.27) and the scaling factors

\frac{ρ_{i}C_{i+1}}{ρ_{i+1}}  (18.47)

have certain properties that are universal for all quadratic maps. Here, we perform an approximate analysis of these properties following Kahn [167, Chapter 16].

We begin by rewriting equation (18.45) as

(ρ_{i+1} − 1)^{2} = 3 + ρ_{i}.  (18.48)

Hence,

ρ_{i+1} − 1 = \sqrt{3 + ρ_{i}},  (18.49)

and

ρ_{i} − 1 = \sqrt{3 + ρ_{i−1}}.  (18.50)

Subtracting (18.50) from (18.49) gives

ρ_{i+1} − ρ_{i} = \sqrt{3 + ρ_{i}} − \sqrt{3 + ρ_{i−1}}.  (18.51)

Summing (18.50) and (18.49) yields

ρ_{i+1} + ρ_{i} − 2 = \sqrt{3 + ρ_{i}} + \sqrt{3 + ρ_{i−1}}.  (18.52)

Multiplying (18.51) and (18.52), we get

(ρ_{i+1} − ρ_{i}) (ρ_{i+1} + ρ_{i} − 2) = ρ_{i} − ρ_{i−1}.  (18.53)

We rewrite (18.53) as

ρ_{i+1} + ρ_{i} − 2 = \frac{ρ_{i} − ρ_{i−1}}{ρ_{i+1} − ρ_{i}} .  (18.54)

Let

δ_{i} = \frac{ρ_{i} − ρ_{i−1}}{ρ_{i+1} − ρ_{i}}  (18.55)

The limiting value of δ_{i} as i tends to infinity is called the Feigenbaum number δ. To find its value, we use (18.55) to represent equation (18.54) as

δ_{i} = ρ_{i+1} + ρ_{i} − 2.  (18.56)

We obtain an approximate value for the Feigenbaum number δ because of the truncation procedure that we used to obtain (18.47). We have

δ_{approx} = lim_{i→∞} (ρ_{i+1} + ρ_{i} − 2) = 2c_{∞} − 2 = 5.1231  (18.57)

The exact value of the Feigenbaum number δ is δ_{exact} = 4.669202 . . ..

The other parameter that characterizes our iteration method is the scaling factor given by (18.47). The limiting value of (18.47) as i tends to infinity is called the Feigenbaum number α. Its approximate value is

α_{approx} = lim_{i→∞} \frac{ρ_{i}C_{i+1}(p)}{ρ_{i+1}}
= \frac{c_{∞}C_{∞}(p)}{c_{∞}}
= c_{∞}^{2} − 2c_{∞} − 3 − 3\sqrt{c_{∞}^{2} − 2c_{∞} − 3}
= −2.24  (18.58)

The exact value of the Feigenbaum number α is α_{exact} = −2.50 . . ..

18.3 Fractals

This section is devoted to modeling of irregular shapes. These non-Euclidean objects are called fractals. The term fractal was coined by Mandelbrot [229] and it comes from the Latin adjective fractus, meaning “irregular”. The corresponding Latin verb is frangere, which means “to break to create irregular fragments.” An example of a simple fractal is the so called the von Koch snowflake curve that is shown in Figure 18.11. In this Figure, an iterative procedure for constructing the curve is also shown. We start with dividing a line

Figure 18.11: The first four stages in constructing the von Koch snowflake curve.

segment into thirds and then replacing the middle segment with two equal segments that form two sides of an equilateral triangle. The outcome of this stage is a shape consisting of four segments. In the next stage, each of the four segments is divided into thirds and the middle segments are replaced with two equal segments that form two sides of equilateral triangle. The process is repeated over and over again. The resulting curve is characterized by self-similarity or scaling.

One of the characteristics of a fractal is its dimension. A line segment is a one-dimensional Euclidean object and we can divide this line segment into N identical parts. In such a case, the individual parts can be viewed as scaled down, by the factor r = 1/N, versions of the original object. Similarly a square, a two-dimensional Euclidean object, can be divided into N self-similar parts. These parts can be viewed as scaled down copies by the factor r = 1/\sqrt{N} of the original square. In general, a D-dimensional object can be divided into N copies of itself. Each copy can be viewed as a scaled down version by the factor

r = \frac{1}{N^{1/D}}  (18.59)

of the parent object. Equation (18.59) can be represented as

r^{D}N = 1.  (18.60)

We now generalize the above arguments to the objects that are not necessarily Euclidean. Taking logarithms of both sides of (18.60) yields the expression for the fractal dimension of

Figure 18.12: The first four iterations in constructing Cantor’s ternary set.

the form

D = \frac{log N}{log (1/r)}  (18.61)

We use (18.61) to compute the dimension of the von Koch curve. Any segment of the curve is composed of four sub-segments, that is, N = 4. Each sub-segment is a scaled down copy of its parent by the factor r = 1/3. Hence, the dimension of the von Koch curve is

D = \frac{log N}{log (1/r)} = \frac{ln 4}{ln 3} = 1.26  (18.62)

We can see from the above that the fractal dimension, unlike the dimension of a common Euclidean pattern, need not be an integer.

Another example of a fractal is the Cantor set, sometimes called Cantor’s ternary set. The Cantor set consists of the points of the unit interval [0, 1] that remain after “middle third” intervals have been successively removed. This set is illustrated in Figure 18.12. It is an example of so called disconnected fractal or fractal dust. For Cantor’s set, we have N = 2 and 1/r = 3. Hence, the fractal dimension of the Cantor set is

D = \frac{log N}{log (1/r)} = \frac{ln 2}{ln 3} = 0.631  (18.63)

18.3.1 The Mandelbrot Set

The Mandelbrot set, denoted M, is the set of all complex c ∈ ℂ for which the sequence

0, c, c^{2} + c, (c^{2} + c)^{2} + c, …  (18.64)

does not tend to ∞ as n tends to ∞. Thus, the Mandelbrot set is generated by the map in the complex plane,

z ↦ z^{2} + c, c ∈ ℂ,  (18.65)

or equivalently,

z(k + 1) = z^{2}(k) + c, c ∈ ℂ.  (18.66)

Let

z(k) = x(k) + jy(k) and c = a + jb.

Then,

z(k + 1) = x(k + 1) + jy(k + 1)
= x^{2}(k) − y^{2}(k) + a + j (2x(k)y(k) + b)
= x^{2}(k) + 2jx(k)y(k) + (jy(k))^{2} + a + jb
= z^{2}(k) + c.

Thus, we can express the first-order quadratic complex dynamical system (18.66) as a secondorder real dynamical system of the form

\begin{matrix}x(k + 1) & = & x^{2}(k) − y^{2}(k) + a \\ y(k + 1) & = & 2x(k)y(k) + b \end{matrix} \}  (18.67)

The representations (18.66) and (18.67) are indeed equivalent.

In terms of the dynamical system (18.67), the Mandelbrot set is the set of parameter values, a and b, for which the solutions of (18.67) starting at x(0) = y(0) = 0 remain bounded. The Mandelbrot set is shown in Figure 18.13.

18.3.2 Julia Sets

A filled Julia set corresponding to the given values of the parameters a and b of the dynamical system (18.67) is the set of initial conditions (x(0), y(0)) resulting in bounded system trajectories. For each value of c = a + jb there is a different Julia set. In order to construct a Julia set, pick first a point (a, b), say in the Mandelbrot set. Then, pick an initial

Figure 18.13: The Mandelbrot set for x(0) = y(0) = 0.

condition (x(0), y(0)) and compute the resulting trajectory of (18.67) starting from the chosen initial condition (x(0), y(0)). Plot only the initial conditions (x(0), y(0)) that give rise to bounded trajectories. The set of plotted points is the filled Julia set corresponding to the particular parameter values (a, b). The boundary of the filled Julia set is the Julia set. Two examples of filled Julia sets are shown in Figures 18.14 and 18.15. The filled Julia set shown in Figure 18.14 is called by Keen ([82, page 61]), “Douady’s Rabbit”. It is observed by Peitgen in [271, page 199] that in a sense the Mandelbrot set can be considered as a one-page-dictionary of all Julia sets. This means that the Mandelbrot set stores infinitely many different Julia sets.

18.3.3 Iterated Function Systems

An iterated function system (IFS) can be used to model natural objects such as clouds, smoke, leaves, or ferns. An IFS uses affine transformations that we discuss next. A twodimensional affine transformation w : ℝ^{2} → ℝ^{2} from ℝ^{2} into itself is defined by

w(x) = Ax + b,  (18.68)

where A ∈ ℝ^{2×2} and b ∈ ℝ^{2}. The affine transformation (18.68) satisfies

∥w(x) − w(y)∥ ≤ L∥x − y∥,  (18.69)

18.3. FRACTALS 913

Figure 18.14: Filled Julia set for c = −0.12256117 + j0.74486177. Figure 18.15: Filled Julia set for c = −0.75. where the Lipschitz constant L is

L = ∥A∥.

The affine transformation w is contractive if L < 1. It is a symmetry if

∥w(x) − w(y)∥ = ∥x − y∥.

The affine transformation w is expansive if L > 1.

We now define an IFS. A two-dimensional IFS consists of a set of N affine transformations denoted

{w_{1}, w_{2}, …, w_{N}},

each transformation mapping ℝ^{2} into itself, together with a set of probabilities

{p_{1}, p_{2}, …, p_{N}},

where each p_{i} ≥ 0 and

p_{1} + p_{2} + ⋯ + p_{N} = 1.

An IFS is the set

{w_{n}, p_{n} : n = 1, 2, …, N}.

Let L_{n}, n = 1, 2, …, N, denote the Lipschitz constant for w_{n}. Then, we say that the IFS obeys the average contractivity condition if

L_{1}^{p_{1}}L_{2}^{p_{2}} ⋯ L_{N}^{p_{N}} < 1.  (18.70)

Interesting mathematical properties of the IFS that satisfy the condition (18.70) are reported by Barnsley [24].

An IFS operates as follows. A set of N affine transformations is chosen and an initial point is picked. Then, the affine transformations are iteratively applied over and over again in a random order with probabilities p_{n}, for example, proportional to the determinants of A_{n}. The result is a sequence of points from ℝ^{2},

x(0), x(1), …

As an example, suppose that N = 4, where

A_{1} = [ \begin{matrix}0.02 & 0 \\ 0 & 0.21 \end{matrix} ] b_{1} = [ \begin{matrix}−0.15 \\ −0.8 \end{matrix} ]
A_{2} = [ \begin{matrix}0.8496 & 0.0255 \\ −0.0255 & 0.8496 \end{matrix} ] b_{2} = [ \begin{matrix}0 \\ 0.15 \end{matrix} ]
A_{3} = [ \begin{matrix}−0.1554 & 0.235 \\ 0.1958 & 0.1865 \end{matrix} ] b_{3} = [ \begin{matrix}0.05 \\ −0.55 \end{matrix} ]
A_{4} = [ \begin{matrix}0.1554 & −0.235 \\ 0.1958 & 0.1865 \end{matrix} ] b_{4} = [ \begin{matrix}−0.35 \\ −0.4 \end{matrix} ] .

The above data comes from Bunde and Havlin [47]. The elements of the set of probabilities

{p_{1}, p_{2}, p_{3}, p_{4}}

were constructed as

p_{n} = \frac{|det(A_{n})|}{∑_{i=1}^{N} |det(A_{i})|} n = 1, 2, 3, 4.

The initial point was chosen to be x(0) = [ −0.1 −0.2 ]^{⊤}. Then, the roulette wheel method was used to select the affine transformation to operate on x(0). The roulette wheel method operates as follows. Each affine transformation is assigned a slice of a wheel, where the slice is proportional to the probability p_{n} of the particular transformation w_{n}. A random number from the interval [0, 1] is generated, which corresponds to spinning the wheel. The transformation under the wheel’s marker is selected. Suppose, w_{k} was selected. Then w_{k} operates on x(0) and the result is

x(1) = A_{k}x(0) + b_{k}.

The roulette wheel is spun again to select the affine transformation to act upon x(1) in order to obtain x(2), etc.

The result of 30, 000 iterations is shown in Figure 18.16. The object is called the Barnsley fern because it was M. Barnsley [24] who introduced the method.

18.4 Lyapunov Exponents

Trajectories of a chaotic dynamical system are characterized by a sensitive dependence upon initial conditions, which means that the distance between the adjacent trajectories grows 916 CHAPTER 18. CHAOTIC SYSTEMS AND FRACTALS

Figure 18.16: The Barnsley fern.

Figure 18.17: The change in distance between adjacent chaotic trajectories that is used to compute the largest Lyapunov exponent. exponentially with time as illustrated in Figure 18.17. For a discrete dynamical system, if x^{∗} is a fixed point of a one-dimensional map f and if |f′ (x^{∗})| = a > 1, then by Theorem 18.1, trajectories starting near x^{∗} will separate from x^{∗} at a multiplicative rate of approximately a per iteration until they move significantly far away from x^{∗}. Let x_{1} be a point that constitutes the k-cycle. Then, for a k-cycle, the trajectory starting from a point close to x_{1}, will separate from x_{1} at a rate approximately equal to the product of the derivatives of the k points of the k-cycle. Let this product be denoted A and suppose that A > 1. Then, it takes k iterations to separate a trajectory starting close to x_{1} by a distance A from x_{1}. We can say that in the case of a k-cycle the average rate of separation is A^{1/k} per iteration. We now are ready to introduce the term Lyapunov number to describe the average multiplicative rate of separation of points close to x_{1}. For a k-cycle, the Lyapunov number L(x_{1}) is

L(x_{1}) = (|(f^{k})′ (x_{1})|)^{1/k}
= (|f′ (x_{1})| |f′ (x_{2})| ⋯ |f′ (x_{k})|)^{1/k}  (18.71)

The concept of Lyapunov number is now extended to non-periodic trajectories.

Definition 18.2 Let f be a differentiable map on the real line ℝ and let {x(0), x(1), …} be a trajectory starting from x(0). Then, the Lyapunov number L(x(0)) of the trajectory starting from x(0) is

L(x(0)) = lim_{N→∞} (|f′(x(0))| ⋯ |f′(x(N))|)^{1/N}

if the limit exists.

◇

The Lyapunov exponent, μ(x(0)), is then defined as the natural logarithm of the Lyapunov number, that is,

μ(x(0)) = ln L(x(0)) = lim_{N→∞} \frac{ln |f′(x(0))| + ⋯ + ln |f′(x(N))|}{N}  (18.72)

if the limit exists.

We now look at the Lyapunov exponent from another perspective. Let d_{0} denote the distance between the two initial states of a continuous dynamical system. Then, for a chaotic motion, at a later time,

d(t) = e^{μt}d_{0}.  (18.73)

The measure of the trajectories’ divergence is obtained by averaging the exponential growth at many points along a trajectory. This is because the chaotic trajectories’ divergence can only be measured locally because if the trajectories are bounded, d(t) in (18.73) cannot go to infinity. To define the measure of the divergence of chaotic trajectories, first a reference trajectory is obtained. Next a point on an adjacent trajectory is selected and d(t)/d_{0} is measured. This is illustrated in Figure 18.17. Then, a new d_{0}(t) is selected on new adjacent trajectory. Using the collected data, the Lyapunov exponent is computed as

μ(x(0)) = lim_{N→∞} \frac{1}{t_{N} − t_{0}} ∑_{k=1}^{N} ln \frac{d(t_{k})}{d_{0}(t_{k−1})}.  (18.74)

There are n Lyapunov exponents for an n-dimensional nonlinear dynamical system, just as there are n eigenvalues that govern behavior of an n-dimensional linear system. To define these Lyapunov exponents for an n-dimensional nonlinear dynamical system model, first choose an n-dimensional sphere centered at a point on a reference trajectory. Let the radius of this sphere be δ_{0}. We can view this sphere as an n-dimensional ellipsoid with n semi-axes of equal length δ_{i}(0) = δ_{0}. Next, at a later time t construct an n-dimensional ellipsoid with the property that all the trajectories emanating from the previously chosen sphere, pass through this ellipsoid. Let the n semi-axes of the ellipsoid be denoted δ_{i}(t). Then, we can compute the Lyapunov exponents as

μ_{i}(x(0)) = lim_{t→∞} ( lim_{δ_{i}(0)→0} \{\frac{1}{t} ln \frac{δ_{i}(t)}{δ_{i}(0)}\}) , i = 1, 2, …, n (18.75)

if the limits exist. It is common to order the Lyapunov exponents so that

μ_{1} ≥ μ_{2} ≥ … ≥ μ_{n}.

Thus, the Lyapunov exponents are numbers that are used to measure the exponential attraction or separation in time of two neighboring trajectories of a dynamical system.

Example 18.1 This example was adapted from Moon [246, page 193]. Suppose that we are given a one-dimensional map

x(k + 1) = f(x(k)).  (18.76)

We demonstrate again that in the regions where f is differentiable, the Lyapunov exponent is determined by |f′|. Indeed, let x(k) and x(k) + ε be two initial conditions. Then, we have

d_{0}(k − 1) = ε and d(k) = |f(x(k) + ε) − f(x(k))| ≈ |f′|_{x=x(k)}| ε.

Using (18.74), we obtain

μ(x(0)) = lim_{N→∞} \frac{1}{t_{N} − t_{0}} ∑_{k=1}^{N} ln \frac{d(t_{k})}{d_{0}(t_{k−1})}
= lim_{N→∞} \frac{1}{N} ∑_{k=1}^{N} ln |f′ (x(k))| ,

which is the same as (18.72)

We use the notion of Lyapunov exponent to formally define chaotic motion. The other term that we use in this definition is an asymptotically periodic trajectory that we define next. This definition is adapted from Alligood et al. [7, page 108].

Definition 18.3 A trajectory {x(0), …, x(N), …} of a smooth map f is called asymptotically periodic if it converges to a periodic trajectory as N → ∞; that is, there exists a periodic trajectory {y_{1}, y_{2}, …, y_{k}, y_{1}, y_{2}, …} such that

lim_{N→∞} ∥x(N) − y(N)∥ = 0.

◇

Using the definition of Lyapunov exponent and the above definition, we can define a chaotic trajectory. This definition is adapted from Alligood et al. [7, page 110].

Definition 18.4 Let f be a map of the real line ℝ, and let {x(0), x(1), …} be a bounded trajectory of the map f. This trajectory is chaotic if

1. the trajectory is not asymptotically periodic,

2. the Lyapunov exponent μ(x(0)) is greater then zero.

◇

For a map on ℝ^{n}, each trajectory has n Lyapunov numbers. The Lyapunov numbers, in this case, measure the rates of separation from the current point of the trajectory along n orthogonal directions. The first direction is the one where the separation between the nearby points is the greatest. The second direction is the one in which the separation is the greatest from all directions perpendicular to the first direction. The third direction will be the one where the separation is the greatest among directions perpendicular to the first two

Figure 18.18: The unit disk is mapped into an ellipse by a two-dimensional map.

directions, and so on. We illustrate the above in two dimensions in Figure 18.18, where a unit disk is centered at x(0). After N iterates of the two-dimensional map f, the disk evolves into an ellipse. The per-iteration changes of the ellipse axes are the Lyapunov numbers of x(0). Thus, the Lyapunov numbers measure the contraction or expansion near the trajectory of x(0) during the first N iterations. The natural logarithm of each Lyapunov number is a Lyapunov exponent.

Let Df(x(0)) be the first derivative of f evaluated at x(0). Let J_{N} = Df^{N}(x(0)) be the first derivative of the N-th iterate of f. We denote by B the unit sphere centered at x(0). With the above notation, we are ready to give the definition of Lyapunov exponents for a map on ℝ^{n}. This definition is adapted from Alligood et al. [7, page 195].

Definition 18.5 Let f be a smooth map on ℝ^{n} and let r_{k}^{N} be the length of the k-th longest orthogonal axis of the ellipsoid J_{N}B for a trajectory with the initial state x(0). The k-th Lyapunov number is

L_{k}(x(0)) = lim_{N→∞} (r_{k}^{N})^{1/N}

if the limit exists. The k-th Lyapunov exponent of x(0) is

μ_{k}(x(0)) = ln L_{k}(x(0)).
◇

Using the above definition, we can extend the definition of chaotic trajectories for maps on ℝ^{n}. This definition is adapted from Alligood et al. [7, page 196] Definition 18.6 Let f be a map on ℝ^{n}, n ≥ 1, and let {x(0), x(1), x(2), …} be a bounded trajectory of the map f. The trajectory is chaotic if

1. it is not asymptotically periodic,

2. no Lyapunov exponent is equal to zero,

3. μ_{1}(x(0)) > 0.

◇

For continuous systems, the above definition of a chaotic trajectory has to be modified— see Alligood at al. [7, pages 385–386].

To numerically compute Lyapunov exponents, a reference trajectory of a given nonlinear system is constructed and the ellipsoid semi-axes lengths r_{k}-s are found using linearized models. We now analyze this process in some detail. For this, suppose that we are given an n-th order continuous dynamical system model

ẋ = f(x),  (18.77)

where x(t) ∈ ℝ^{n} is the system’s state at time t, and f : ℝ^{n} → ℝ^{n} is called the vector field. The solution to (18.77) subject to initial condition x(t_{0}) = x_{0} is called a trajectory and is denoted ϕ_{t}(x_{0}). For every t, the mapping

ϕ_{t} : ℝ^{n} → ℝ^{n},

maps the state space ℝ^{n} into itself. The mapping ϕ_{t} is called the flow of the system (18.77). For example, if (18.77) is linear, that is, f(x) = Ax, where A ∈ ℝ^{n×n}, then

ϕ_{t}(x_{0}) = e^{A(t−t_{0})}x_{0}.

Consider now an equilibrium state x_{e} of (18.77). The local behavior of (18.77) in a neighborhood of x_{e} is determined by the linearized model of (18.77) about the equilibrium state x_{e}. The linearized model is

\frac{d}{dt}Δx = Df(x_{e})Δx,  (18.78)

where Δx = x − x_{e}. We can also linearize the system (18.77) about its trajectory ϕ_{t}(x_{0}). The linearized model about the trajectory ϕ_{t}(x_{0}) is

\frac{d}{dt}Δx = Df(ϕ_{t}(x_{0}))Δx, Δx(t_{0}) = Δx_{0}.  (18.79)

The solution to (18.79) is

Δx(t) = Φ_{t}(x_{0})Δx_{0}.

We call the n × n matrix Φ_{t}(x_{0}) the state transition matrix of the system (18.79)—see Section 6.2 for more information on the state transition matrix properties. The transition matrix satisfies the equation

\frac{d}{dt}Φ = Df(ϕ_{t}(x_{0}))Φ, Φ_{0} = I_{n}.

Let

{m_{i}(t)}_{i=1}^{n}

be the eigenvalues of the transition matrix Φ_{t}(x_{0}). Then, the Lyapunov exponents of x_{0} are

μ_{i} (x_{0}) = lim_{t→∞} \frac{1}{t} ln |m_{i}(t)| , i = 1, 2, …, n  (18.80)

if the limits exist. The above is an alternative to (18.75) representation of the Lyapunov exponents.

Example 18.2 The Lyapunov exponents of an equilibrium state x_{e} of the system (18.77), with t_{0} = 0, can be found by first linearizing (18.77) about the equilibrium state to obtain a linearized model of the form (18.78). The transition matrix of (18.78) is

Φ_{t}(x_{e}) = e^{Df (x_{e})t}.

The i-th eigenvalue of the transition matrix Φ_{t}(x_{e}) is m_{i}(t) = e^{λ_{i}t}, where λ_{i} is the i-th eigenvalue of the matrix Df(x_{e}). Therefore, the Lyapunov exponents of x_{e} are

μ_{i} (x_{e}) = lim_{t→∞} \frac{1}{t} ln |m_{i}(t)|
= lim_{t→∞} \frac{1}{t} ln |e^{ℜλ_{i}t}e^{jℑλ_{i}t}|
= lim_{t→∞} \frac{1}{t} ln (|e^{ℜλ_{i}t}| |cos (ℑλ_{i}t) + j sin (ℑλ_{i}t)|)
= lim_{t→∞} \frac{1}{t}t ℜλ_{i}
= ℜλ_{i}.  (18.81)

Thus, the Lyapunov exponents in this case are the real parts of the eigenvalues of the linearized system about the equilibrium state. Note that if x_{0} is in the basin of attraction of x_{e}, that is, if

lim_{t→∞} ϕ_{t}(x_{0}) = x_{e},

then the Lyapunov exponents of x_{0} are the same as the Lyapunov exponents of x_{e}.

Example 18.3 Suppose that we are given a discrete-time dynamical system model,

x(k + 1) = f(x(k)), x(0) = x_{0}.  (18.82)

Let x_{e} be the equilibrium state of the model (18.82), that is,

x_{e} = f (x_{e}) .

The linearized model is

Δx(k + 1) = Df(x_{e})Δx(k).

The state transition matrix of the linearized model is

Φ_{k}(x_{e}) = (Df (x_{e}))^{k}  (18.83)

If λ_{i} is the i-th eigenvalue of the matrix Df (x_{e}), then λ_{i}^{k} is the i-th eigenvalue of the transition matrix (18.83). The Lyapunov exponents of x_{e}, in this case, are

μ_{i}(x_{e}) = lim_{k→∞} \frac{1}{k} ln |m_{i}(k)|
= lim_{k→∞} \frac{1}{k} ln |λ_{i}|^{k}
= ln |λ_{i}| .  (18.84)

In summary, Lyapunov exponents can be used to characterize a chaotic motion. Specifically, a necessary condition for a chaotic motion to occur in a dynamical system with bounded trajectories is that the largest Lyapunov exponent be positive. An efficient numerical method for evaluating the largest Lyapunov exponent can be found, for example, in the paper by Parker and Chua [263], see also their book [264] on practical numerical algorithms for chaotic systems.

In Figure 18.19, we show a plot of the Lyapunov exponent versus the parameter ρ for the logistic equation (18.10).

Figure 18.19: Lyapunov exponent versus parameter ρ for the logistic equation (18.10).

18.5 Discretization Chaos

In this section, we consider examples of chaotic systems that are discrete-time analogs of continuous-time systems, where the latter systems do not exhibit any chaotic behavior. We begin by considering a method of obtaining discrete models from continuous models. Suppose that we are given a scalar differential equation

ẋ = ax, x(t_{0}) = x_{0}.  (18.85)

The solution to the above differential equation is

x(t) = e^{a(t−t_{0})}x(t_{0}).  (18.86)

Let Δt = t − t_{0}. Then, we can express (18.86) as

x(1) = e^{aΔt}x(0),  (18.87)

where x(1) = x(t_{0} + Δt) and x(0) = x(t_{0}). More generally,

x(k + 1) = e^{aΔt}x(k),  (18.88)

where x(k +1) = x((k +1)Δt) and x(k) = x(kΔt). The discrete model (18.88) of the continuous differential equation (18.85) is exact in the sense that the values of (18.86) and (18.88) are the same at the sampling times. We generalize the above method to a class of nonlinear differential equations. Consider a scalar differential equation

ẋ(t) = x(t)h(x(t)).  (18.89)

Dividing both sides of the above by x and then integrating yields

∫ \frac{dx}{x} = ∫ h(x(t))dt.  (18.90)

Suppose now that we hold h(x(t)) constant on the interval [ kΔt, (k + 1)Δt ). Then, evaluating (18.90) over the above interval gives

ln (\frac{x((k + 1)Δt)}{x(kΔt)}) = h(x(kΔt))Δt.

Using the the notation introduced after (18.88), we represent the above as

x(k + 1) = e^{h(x(k))Δt}x(k)  (18.91)

Example 18.4 Consider the system of differential equations

\begin{matrix}ẋ_{1} & = & γx_{1} (1 − x_{1} − 0.5x_{2}) \\ ẋ_{2} & = & 2γx_{2} (1 − 0.5x_{1} − x_{2}) \end{matrix} \}  (18.92)

where γ = 2.5. The above system models a pair of competing species. For a discussion of modeling of competing species population dynamics, the reader is referred to Subsection 2.4.7. The system (18.92) was analyzed by Grantham and Athalye [119]. A trajectory of the system (18.92) for x_{1}(0) = x_{2}(0) = 0.5 is shown in Figure 18.20. We discretize (18.92) using the formula (18.91). For Δt = 1, we obtain the following discrete-time analog

\begin{matrix}x_{1}(k + 1) & = & x_{1}(k) exp (γ (1 − x_{1}(k) − 0.5x_{2}(k))) \\ x_{2}(k + 1) & = & x_{2}(k) exp (2γ (1 − 0.5x_{1}(k) − x_{2}(k))) \end{matrix} \}  (18.93)

The trajectory of the discrete model (18.93) starting from the same initial condition as its continuous-time analog is shown in Figure 18.21. The continuous-time model trajectory asymptotically converges to the equilibrium state [ 2/3 2/3 ]^{⊤}, while the discrete-time trajectory wanders around chaotically.

926 CHAPTER 18. CHAOTIC SYSTEMS AND FRACTALS Figure 18.20: A trajectory of the system (18.92). Figure 18.21: A trajectory of the system (18.92).

18.6 Controlling Chaotic Systems

A trajectory of a chaotic system behaves in a random-like fashion. The set on which the chaotic motion is taking place attracts trajectories starting from a set of initial conditions. Roughly speaking, this attracting set is called a chaotic attractor. It is not possible to predict when a trajectory will pass through a given point in the chaotic attractor. However, any given point in the chaotic attractor will be visited by the chaotic system’s trajectory in a finite time. This property of a chaotic system is referred to as ergodicity. Using the ergodicity property one can propose a control algorithm whose task it is to bring the chaotic system’s trajectory to a prespecified state in the system’s state space. The idea behind the chaotic control algorithm, that we discuss next, is to set up “fishing nets” in the chaotic attractor in a neighborhood of an operating point to “catch” the system trajectory. Once the trajectory is “caught”, a control law is used to bring the trajectory to the desired operating point. The controller is constructed using linear quadratic regulator (LQR) technique. The chaotic algorithm is illustrated with two examples; the Hénon map and the bouncing ball system.

18.6.1 Ingredients of Chaotic Control Algorithm

Chaotic motion is characterized by extreme sensitivity to initial conditions. This might suggest that it would be impossible, for example, to stabilize a chaotic system with respect to a desired operating point, because any perturbation resulting from applying a control action would lead to a chaotic motion’s exponential grow in time. However, it turns out that precisely this property of a chaotic motion can be used to convert a chaotic attractor to a periodic motion. A chaotic attractor, as observed by Ott, Grebogi, and Yorke [259] typically has imbedded within it an infinite number of unstable periodic orbits. These periodic orbits are dense in the attractor in the sense that they pass through any neighborhood, however small, of any point in the attractor. Ott et al. [259] noticed that these properties could be advantageous in achieving control objectives. They proposed a method, now called the OGY method, which involves converting, using small time-dependent perturbations of a system parameter, a chaotic attractor into a periodic orbit. The idea behind this method is to exploit the ergodicity property of a chaotic system. Specifically, all of the unstable periodic orbits of the chaotic attractor will eventually be visited by the system trajectory. Once the trajectory gets to a neighborhood of the desired unstable periodic orbit, a small perturbation is applied to stabilize the orbit of interest.

The chaotic control algorithm, that we discuss next, can be viewed as a variant of the

Figure 18.22: A pictorial representation of the chaotic algorithm.

OGY method. This algorithm was presented by Vincent [337] and is characterized by two ingredients. They are: a chaotic attractor and a controllable target. These components of the chaotic algorithm are illustrated in Figure 18.22. We assume that either the system under consideration is chaotic or that it can be made chaotic using an open-loop control. Next, we select a fixed point of interest and assume that a feedback law can be constructed such that the closed-loop system is non-chaotic and the chosen fixed point is asymptotically stable. The region of attraction of the fixed point should have a non-empty intersection with the chaotic attractor. A controllable target can be any subset of the region of attraction of the fixed point that has a non-empty intersection with the chaotic attractor. Note that the fixed point does not have to be located in the chaotic attractor. If we start the system from any initial state in the chaotic attractor, the resulting chaotic trajectory will at some time arrive at the intersection region of the chaotic attractor and the controllable target. The chaotic algorithm keeps track when the trajectory reaches this intersection region. When this happens, the open-loop control, that was used to create chaotic motion, is turned off and the feedback control is turned on to drive the system towards the fixed point. The region of attraction intersecting the chaotic attractor plays the role of a “fishing net” for the chaotic trajectory.

18.6.2 Chaotic Control Algorithm

We consider two classes of single-input dynamical systems. The first class is modeled by nonlinear difference equation,

x(k + 1) = f(x(k), u(k)),  (18.94)

and the other class is modeled by nonlinear differential equation,

ẋ(t) = f(x(t), u(t)),  (18.95)

The control u, in both cases, will be bounded. The control u will be either a specified function of time or a specified function of the system state. In the first case, we have an open-loop control, in the latter case we have a closed-loop control. We assume that for a given system there exists an open-loop control such that the system has a chaotic attractor. In addition, we assume that for a specified constant control u = u_{e} there is a corresponding equilibrium state located in a vicinity of the chaotic attractor. The equilibrium state satisfies the equation

x_{e} = f(x_{e}, u_{e})

for the discrete-time system, and it satisfies the relation

0 = f(x_{e}, u_{e})

for the continuous-time system.

We now briefly describe the way we obtain a controllable target for the system of interest. We first linearize the system about the selected equilibrium state. The linearized discretetime system has the form,

Δx(k + 1) = AΔx(k) + bΔu(k),

where Δx(k) = x(k)−x_{e} and Δu(k) = u(k)−u_{e}. The linearized continuous-time system has the form

\frac{d}{dt}Δx(t) = AΔx(t) + bΔu(t),

where Δx(t) = x(t) − x_{e} and Δu(t) = u(t) − u_{e}. Then, the LQR method, described in Chapter 9, is used to construct a state variable feedback control law, Δu(Δx) = −kΔx, that guarantees the origin of the linearized system,

Δx(k + 1) = (A − bk)Δx(k),  (18.96)

or

\frac{d}{dt}Δx(t) = (A − bk)Δx(t),  (18.97)

to be asymptotically stable. This implies that for the corresponding nonlinear system driven by the state feedback controller, the equilibrium state about which the system was linearized, is asymptotically stable. We use a Lyapunov function obtained from the linearized system to estimate the region of asymptotic stability (RAS) of the closed-loop nonlinear system. The Lyapunov function has the form, V (x) = x^{⊤}P_{l}x, where P_{l} is obtained by solving the corresponding Lyapunov equation for the optimal closed-loop linear system given by (18.96) or by (18.97). This Lyapunov function is then used to determine a controllable target for the nonlinear system driven by a bounded controller of the form,

u = sat(kx) + u_{e},  (18.98)

where

sat(v) = \{ \begin{matrix}u_{max} − u_{e} & if & v > u_{max} − u_{e} \\ v & if & u_{min} − u_{e} ≤ v ≤ u_{max} − u_{e} \\ u_{min} − u_{e} & if & v < u_{min} − u_{e}, \end{matrix}

and the bounds u_{min} and u_{max} are specified by a designer. Using the above control law, we then seek the largest level curve V (x) = V_{max} such that for the points {x : V (x) ≤ V_{max}}, for the discrete-time system,

ΔV (k) = V (x(k + 1)) − V (x(k))
= (x(k + 1) − x_{e})^{⊤}P_{l}(x(k + 1) − x_{e}) − (x(k) − x_{e})^{⊤}P_{l}(x(k) − x_{e})
= (f(x(k), u(k)) − x_{e})^{⊤} P_{l} (f(x(k), u(k)) − x_{e}) − (x(k) − x_{e})^{⊤} P_{l} (x(k) − x_{e})
< 0,

or for the continuous-time system,

V̇ (x) = 2 (x − x_{e})^{⊤} P_{l}ẋ
= 2 (x − x_{e})^{⊤} P_{l}f(x, u)
< 0.

We are now ready to describe the chaotic algorithm. When the given nonlinear system trajectory is in the region {x : V (x) > V_{max}}, an open-loop control is applied to keep the system in the chaotic attractor until the system reaches a point in the set {x : V (x) ≤ V_{max}}, where the open-loop control is turned off and the feedback saturating control law is applied to

Figure 18.23: An orbit of the Hénon map.

force the system trajectory to asymptotically approach the equilibrium state. More formally, we state the chaotic algorithm in the form of the following algorithm:

IF V (x) > V_{max}

THEN u = open-loop control to keep the system in the chaotic attractor ELSE u = sat(kx) + u_{e} END

We illustrate the above algorithm when applied to the Hénon map.

Example 18.5 The dynamical system known as the Hénon map is described by the difference equations,

[ \begin{matrix}x_{1}(k + 1) \\ x_{2}(k + 1) \end{matrix} ] = [ \begin{matrix}1 + x_{2}(k) − 1.4x_{1}^{2}(k) \\ 0.3x_{1}(k) \end{matrix} ] .  (18.99)

It was named after the French astronomer M. Hénon who discovered chaotic behavior of the above system in 1974. The orbit of x(0) = [ 1 0 ]^{⊤} of the Hénon map is shown in Figure 18.23. We modify the Hénon map by adding the control input into the first equation so that the map becomes,

[ \begin{matrix}x_{1}(k + 1) \\ x_{2}(k + 1) \end{matrix} ] = [ \begin{matrix}−1.4x_{1}^{2}(k) + x_{2}(k) + 1 + u \\ 0.3x_{1}(k) \end{matrix} ] .  (18.100)

There are two fixed states corresponding to u = u_{e} = 0. They are obtained by solving two algebraic equations,

\{ \begin{matrix}x_{1,eq} & = & −1.4x_{1,e}^{2} + x_{2,e} + 1 \\ x_{2,eq} & = & 0.3x_{1,e} \end{matrix}

Solving the above gives

[ 0.6314 0.1894 ]^{⊤} and [ −1.1314 −0.3394 ]^{⊤} .

We select to work with the first of the above fixed states. We linearize the system (18.100) about the first fixed state to obtain

A = [ \begin{matrix}−2.8x_{1} & 1 \\ 0.3 & 0 \end{matrix} ]|_{x_{1}=0.6314} = [ \begin{matrix}−1.7678 & 1 \\ 0.3 & 0 \end{matrix} ] , b = [ \begin{matrix}1 \\ 0 \end{matrix} ] .

We then use the discrete LQR design method to find the state feedback. The weight matrices of the associated quadratic performance index are chosen to be Q = I_{2} and r = 1. Solving the discrete algebraic Riccati equation yields

P = [ \begin{matrix}3.9351 & −1.5008 \\ −1.5008 & 1.7974 \end{matrix} ] and k = [ −1.5008 0.7974 ] .

Hence,

A_{c} = A − bk = [ \begin{matrix}−0.2670 & 0.2026 \\ 0.3000 & 0 \end{matrix} ] .

Solving the discrete Lyapunov equation,

A_{c}^{⊤}P_{l}A_{c} − P_{l} = −I_{2}

gives

P_{l} = [ \begin{matrix}1.1902 & −0.0686 \\ −0.0686 & 1.0489 \end{matrix} ] .

The next step is to calculate V_{max} corresponding to the largest level curve such that the Lyapunov first difference ΔV < 0 on this level curve for the nonlinear model. We obtain level curves by trial and error as follows. We first evaluate

V = (x − x_{e})^{⊤} P_{l} (x − x_{e})
= [ x_{1} − 0.6314 x_{2} − 0.1894 ] [ \begin{matrix}1.1902 & −0.0686 \\ −0.0686 & 1.0489 \end{matrix} ] [ \begin{matrix}x_{1} − 0.6314 \\ x_{2} − 0.1894 \end{matrix} ]
= 1.1902(x_{1} − 0.6314)^{2} − 0.1372(x_{1} − 0.6314)(x_{2} − 0.1894)
+1.0489(x_{2} − 0.1894)^{2}.  (18.101)

We next choose an initial state x(0) such that V (x(0)) = V_{0}, where V_{0} is a constant selected by the designer. Let x_{2}(0) = 0.1894, which is the value of the second component of the

Figure 18.24: A plot of x_{1} versus time for the controlled Hénon map of Example 18.5.

equilibrium state of interest. Substituting this value into (18.101) allows us to compute the first component of x(0) such that V (x(0)) = V_{0}. We have

x(0) = [ \sqrt{0.8402V_{0}} + 0.6314 0.1894 ]^{⊤} .

Then, a level curve is constructed by solving the differential equation,

∇^{⊤}V ẋ = 0,

subject to the initial condition x(0). To satisfy the above equation, we can select

\{ \begin{matrix}ẋ_{1} & = & \frac{∂V}{∂x_{2}} \\ ẋ_{2} & = & −\frac{∂V}{∂x_{1}} \end{matrix}  (18.102)

After constructing a level curve, we evaluate ΔV at these points and check if ΔV < 0 at every point on the level curve. For the control law

u = 0.3 sat(kx/0.3),  (18.103)

applied to the nonlinear system (18.100), we found V_{max} = 0.0109. Plots of x_{1} and x_{2} versus time for the closed-loop system (18.100), (18.103) are shown in Figures 18.24 and 18.25, respectively, for the initial condition x(0) = [ −0.5 0 ]^{⊤}. The orbit of x(0) = [ −0.5 0 ]^{⊤} of the controlled Hénon map is shown in Figure 18.26, where the controllable target is also depicted as the set of points inside the ellipse V (x − x_{e}) = 0.0109.

Figure 18.25: A plot of x_{2} versus time for the controlled Hénon map of Example 18.5.

Figure 18.26: An orbit of the controlled Hénon map of Example 18.5, where the initial condition is denoted by “∘” and labelled “IC”. The subsequent iterations prior to hitting the controllable target are denoted by “⋆”. The iterations in the controllable target are denoted by “·”.

Figure 18.27: Bouncing ball on a vibrating plate.

Example 18.6 Controlling the bouncing ball on a vibrating plate The bouncing ball on a vibrating plate is an another example of a chaotic system. In this example, we use the chaotic algorithm to design a control law for the vibrating plate, as shown in Figure 18.27, so that the ball, starting at rest on the plate can be bounced up to and then maintained bouncing at a prescribed height. The amplitude of the vibrating plate will be kept fixed. The control input is the frequency of the plate vibrations. To proceed, we need a mathematical model of the ball-plate system, which we derive next.

Modeling a Bouncing Ball on a Vibrating Plate

Let the displacement of the plate in the vertical direction be

y(t) = A sin(ωt),

where A is the amplitude of the plate displacement, ω is the angular frequency of vibrations and t is time. Then, the velocity of the plate is

W (t) = \frac{dy(t)}{dt} = Aω cos(ωt).  (18.104)

We denote by U the velocity of the ball just before impact and let V denote the velocity of the ball just after impact. In our modeling process, we use the principle of conservation of momentum, which states that momentum of the system before and after collisions is constant, that is,

∑(masses × velocities)_{before} = ∑(masses × velocities)_{after}.

Applying the conservation of momentum principle to our system consisting of the bouncing ball and the vibrating plate yields

m_{ball}U + m_{plate}W_{before} = m_{ball}V + m_{plate}W_{after}.  (18.105)

To proceed, we define the so called coefficient of restitution

e = −\frac{V − W_{after}}{U − W_{before}} .  (18.106)

Performing simple manipulations on (18.106), we obtain

W_{after} = Ue − W_{before}e + V.  (18.107)

Substituting the above into (18.105) and rearranging gives

m_{ball}U + m_{plate}W_{before} = (m_{ball} + m_{plate}) V + em_{plate}U − em_{plate}W_{before}.  (18.108)

Dividing both sides of the above by (m_{ball} + m_{plate}) and rearranging, we get

V = \frac{m_{ball} − em_{plate}}{m_{ball} + m_{plate}} U + \frac{m_{plate} + em_{plate}}{m_{ball} + m_{plate}} W_{before}.  (18.109)

Let

M = \frac{m_{ball}}{m_{plate}} .

Substituting the above into (18.109) gives

V = \frac{M − e}{M + 1}U + \frac{e + 1}{M + 1}W_{before}.  (18.110)

The ball is acted upon by a constant acceleration g and therefore V (t) = gt. Hence, from one bounce to the next,

t(k + 1) = t(k) + \frac{2V (k)}{g}.  (18.111)

We assume that the amplitude, A, of the plate vibrations is negligible compared with the height of the bounce. Referring to Figure 18.28, we obtain

U(k + 1) = −V (k).  (18.112)

Figure 18.28: Velocities of the bouncing ball from one bounce to the next.

Let

a_{1} = \frac{e + 1}{M + 1} and a_{2} = \frac{M − e}{M + 1}.

Taking into account the above notation and (18.110), we obtain a relation between velocities of the bouncing ball and the vibrating plate at the bounce at a time t(k + 1),

V (k + 1) = a_{2}U(k + 1) + a_{1}W_{before}(k + 1).  (18.113)

We use (18.112) to represent (18.113) as

V (k + 1) = −a_{2}V (k) + a_{1}W (k + 1).  (18.114)

Furthermore, by (18.104) and (18.111),

W (k + 1) = Aω cos(ωt(k + 1)) = Aω cos (ω (t(k) + \frac{2V (k)}{g})) .

Hence, we can represent (18.114) as

V (k + 1) = −a_{2}V (k) + a_{1}Aω cos (ω (t(k) + \frac{2V (k)}{g})) .  (18.115)

Let

ϕ(k) = ωt(k) and ψ(k) = 2ωV (k)/g,  (18.116)

where ϕ(k) is the current phase angle of the plate and ψ(k) is the phase angle change between the current bounce of the ball and the next. Then, we can write

\begin{aligned}ϕ(k + 1) & = & ϕ(k) + ψ(k) \\ ψ(k + 1) & = & −a_{2}ψ(k) + â_{1}ω^{2} cos(ϕ(k) + ψ(k)), \end{aligned}  (18.117)

Table 18.1: Simulation parameters of the ball map.

\begin{matrix}M & 1/26 \\ e & 0.8 \\ A & 0.013 m \end{matrix}

where â_{1} = 2Aa_{1}/g. The above dynamical system is known as the ball map. In our simulations, we use the same values of the ball map parameters as those used by Vincent [337], which are given in Table 18.1. Using these parameters, we compute a_{1} = 1.73333, â_{1} = 0.004594 sec^{2}, and a_{2} = −0.733333.

There are many possible periodic equilibrium solutions to the ball map (18.117). For instance, the ball can bounce to a fixed height at every n cycles of the plate, or it can bounce to m different heights at nm cycles of the plate. Here, we will only be concerned with the case when m = 1. There are bands of frequencies of the vibrating plate that will not support periodic motion for m = 1. Examples of such frequency bands, as listed by Vincent [337], are:

32.3966 < ω < 33.0782
36.4575 < ω < 38.1954
40.5278 < ω < 42.7038
44.4249 < ω < 46.7796

A frequency of the plate vibration from one of the above ranges will either result in a chaotic motion of the ball or a chaotic transient to a periodic motion with m > 1. In our example, we use ω = 45 rad/sec to generate chaotic motion.

The ball map (18.117) may produce negative values for ψ corresponding to negative bounce heights, which is physically impossible. To overcome this problem, whenever the map produces a negative value for ψ, this value is multiplied by a random negative number between zero and one in order to simulate the fact that the ball cannot go through the plate. In Figure 18.29, an orbit of the modified ball map for the initial conditions ϕ(0) = 0 and ψ(0) = 9.1743 rads and ω = 45 rad/sec is shown. Note that the values of ψ are divided by 2π before plotting. The chaotic attractor is located between the upper and lower bounds as depicted in the figure. The upper bound is given by

ψ_{upper} = 2â_{1}ω^{2} + â_{1}ω^{2} cos(ϕ(k) + ψ(k)),

Figure 18.29: An orbit of the ball map.

while the lower bound is

ψ_{lower} = â_{1}ω^{2} cos(ϕ(k) + ψ(k)),

When ψ_{lower} goes negative it is set equal to zero.

Controller Construction for a Bouncing Ball on a Vibrating Plate To proceed with the controller design, we first find fixed points of the ball map. They

are found by solving the equations,

\{ \begin{matrix}ϕ̄ & = & ϕ̄ + ψ̄ \\ ψ̄ & = & −a_{2}ψ̄ + â_{1}ω^{2} cos(ϕ̄ + ψ̄). \end{matrix}  (18.118)

The fixed points are obtained by evaluating the right-hand side of the first above equation modulo 2nπ, where n is the number of cycles of the plate between bounces. We obtain,

\{ \begin{matrix}ψ̄ & = & 2nπ \\ cos(ϕ̄) & = & \frac{2nπ(1+a_{2})}{â_{1}ω^{2}}. \end{matrix}  (18.119)

Linearizing the ball map about (18.119) gives

[ \begin{matrix}x_{1}(k + 1) \\ x_{2}(k + 1) \end{matrix} ] = [ \begin{matrix}1 & 1 \\ a_{21} & a_{22} \end{matrix} ] [ \begin{matrix}x_{1}(k) \\ x_{2}(k) \end{matrix} ] ,  (18.120)

where

x_{1}(k) = ϕ(k) − ϕ̄
x_{2}(k) = ψ(k) − ψ̄
a_{21} = −â_{1}ω^{2} sin(ϕ̄)
a_{22} = −a_{2} + a_{21}.

We next construct a control law for selecting frequencies of the vibrating plate in such a way that the steady-state constant height of the bouncing ball is reached. Vincent [337] showed that for n = 1 the range of frequencies for which there exists an asymptotically stable equilibrium is [19.0977 rad/sec, 28.9505 rad/sec]. The corresponding range of the bounce heights is [0.0578 m, 0.1327 m]. Thus, for example the height, h, of 0.1 m is in the above range and from the energy conservation law,

\frac{1}{2}m_{ball}V^{2} = m_{ball}gh,

we obtain

V = 1.3994 m/sec^{2}.

Using the above, (18.116), and (18.119), we get

ψ̄ = 2π = \frac{2ωV}{g} and cos(ϕ̄) = \frac{2π(1 + a_{2})}{â_{1}ω^{2}},

from which we compute,

ω = 22 rad/sec and ϕ̄ = 41.1^{∘} = 0.7173 rad.

The linearized ball map about the above equilibrium state is

x(k + 1) = [ \begin{matrix}1 & 1 \\ −1.4617 & −0.7283 \end{matrix} ] x(k),

which has its poles in the open unit circle. Solving the discrete Lyapunov equation, A^{⊤}P A− P = −I_{2}, gives

P = [ \begin{matrix}8.6414 & 4.6978 \\ 4.6978 & 5.9598 \end{matrix} ] .

Employing the same method as in the previous example, we obtain V_{max} = 5.006. The controllable target is

\{[ \begin{matrix}ϕ \\ ψ \end{matrix} ] : V (ϕ − ϕ̄, ψ − ψ̄) = [ ϕ − 0.7173 ψ − 2π ] P [ \begin{matrix}ϕ − 0.7173 \\ ψ − 2π \end{matrix} ] ≤ V_{max}\} .

Figure 18.30: An orbit of the controlled modified ball map.

The controllable target is depicted in Figure 18.30. We use ω = 45 rad/sec when the system trajectory is outside of the controllable target, and apply ω = 22 rad/sec when the system trajectory hits the controllable target, that is, we use the algorithm:

IF V (ϕ − ϕ̄, ψ − ψ̄) > V_{max} THEN ω = 45 rad/sec ELSE ω = 22 rad/sec END

An orbit of ϕ(0) = 0, ψ(0) = 9.1743 of the controlled modified ball map using the above algorithm is shown in Figure 18.30. A plot of ϕ(k) versus time is shown in Figure 18.31. A plot of ψ(k) versus time is shown in Figure 18.32.

18.7 An Application to Secure Communication

The purpose of secure communication is to limit the availability of the transmitted information to a third party even if the communication channel is public or has been compromised. With an ever increasing reliance on wireless and internet communication of sensitive information, including data related to individuals, electronic commerce and other financial transactions, secure communication is of utmost importance.

942 CHAPTER 18. CHAOTIC SYSTEMS AND FRACTALS Figure 18.31: A plot of ϕ(k) versus time of the controlled modified ball map. Figure 18.32: A plot of ψ(k) versus time of the controlled modified ball map.

One approach to secure transfer of signals is to hide them in broad spectrum noise-like signals. Chaotic signals are characterized by spread spectrum, where the power of a chaotic signal is spread across a broad range of frequencies. Thus, chaotic signals are particularly suitable for secure communication because the transmitted signal masked by a chaotic signal may appear to a third party as a noise. The concept of synchronized systems, that we define next, plays important role when using chaotic systems in secure communication.

Definition 18.7 Two dynamical systems,

ẋ = f(x) and ˙ x̃ = g(x̃), x, x̃ ∈ ℝ^{n},

are said to be synchronized if

e(t) = (x(t) − x̃(t)) → 0 as t → ∞.
◇

Cuomo, Oppenheim, and Strogatz [70] used the concept of synchronized chaotic systems (SCS’s) to secure communications with the Lorenz circuit implemented in both the transmitter and receiver. Specifically, a chaotic masking signal was added at the transmitter to the message. Then, at the receiver, the masking was regenerated and subtracted from the received signal.

Synchronization is widely used in electronics, for example, in television, radar, and electronic watches [323]. The general idea behind synchronization is the adaptive generation of a reference signal that is used to estimate the frequency and phase of the received signal. The problem of designing a dynamical system whose behavior mimics that of a given chaotic system is referred to as chaotic synchronization. This is a challenging design problem because chaotic systems are characterized by a sensitive dependence on initial conditions. Thus, one may be tempted to conclude that chaotic synchronization is not feasible because even infinitesimal errors in initial conditions or system parameters will result in the exponential divergence of the systems’ trajectories.

Although chaotic systems would seem to defy synchronization, Pecora and Carroll [267, 52] demonstrated that chaotic system synchronization is possible. Morgül and Solak [247] proposed a systematic approach to chaotic system synchronization using observers. Other observer-based approaches to this problem were presented by Grassi and Masscolo [120], Azemi and Yaz [16], Lin et al. [216], and more recently by Jiang, Tang, and Chen [163]. Lian, Chiu, Chaing, and Liu [214] proposed a systematic design methodology using fuzzy observer-based approach.

In this Section, we show how to combine fuzzy observers, chaos, and LMIs to develop systems for secure communication. Specifically, we construct a fuzzy observer-based secure communication system using a chaotic system. The design is accomplished with LMIs. Our exposition is based on the paper by Lian et al. [214].

18.7.1 Fuzzy Representation of Chaotic Systems

In our construction of a secure communication system, we employ a chaotic system represented as a fuzzy system. We use the Lorentz’s system described by a system of three differential equations,

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ] = [ \begin{matrix}a(−x_{1} + x_{2}) \\ bx_{1} − x_{2} − x_{1}x_{3} \\ −cx_{3} + x_{1}x_{2} \end{matrix} ]
y = x_{1},

where a = 10, b = 28, and c = 8/3; see also Exercise 18.1. To design Lorentz’s system fuzzy representation, we use x_{1} to construct fuzzy rules. We note that x_{1}(t) ∈ [−d, d], where d = 30. We use two fuzzy rules:

Rule i : IF x_{1}(t) is F_{i}(x_{1}(t)) THEN
\{ \begin{matrix}ẋ(t) & = & A_{i}x(t) + b_{i} \\ y(t) & = & c_{i}x(t), i = 1, 2, \end{matrix}

where

F_{1}(x_{1}(t)) = \frac{1}{2} (1 + \frac{x_{1}(t)}{d}) and F_{2}(x_{1}(t)) = \frac{1}{2} (1 − \frac{x_{1}(t)}{d}) ,

A_{1} = [ \begin{matrix}−10 & 10 & 0 \\ 28 & −1 & −d \\ 0 & d & −\frac{8}{3} \end{matrix} ] , A_{2} = [ \begin{matrix}−10 & 10 & 0 \\ 28 & −1 & d \\ 0 & −d & −\frac{8}{3} \end{matrix} ] ,

c_{1} = c_{2} = [ 1 0 0 ] , and b_{1} = b_{2} = 0.

Note that on the interval [−d, d], we have F_{i}(x_{1}(t)) ≥ 0, i = 1, 2. In addition,

F_{i}(x_{1}) + F_{2}(x_{1}) = 1.

The fuzzy Lorentz system takes the form

ẋ(t) = (\frac{F_{1}}{F_{1} + F_{2}} A_{1} + \frac{F_{2}}{F_{1} + F_{2}} A_{2}) x(t)
= (F_{1}A_{1} + F_{2}A_{2}) x(t).  (18.121)

Figure 18.33: Plots of trajectories of the fuzzy Lorentz system (18.121).

In Figure 18.33, we show plots of the fuzzy Lorentz system over the time inerval [0, 100] for the initial condition x(0) = [ −8 −8 24 ]^{⊤} .

18.7.2 Constructing Secure Communication System

We first construct a transmitter. Our approach is to hide the message to be transmitted in a noise-like signal being generated by a fuzzy chaotic system of the form

\begin{matrix}ẋ & = & ∑_{i=1}^{r} μ_{i} (A_{i}x + b_{i} + l_{i}m) \\ y & = & cx + m, \end{matrix} \}  (18.122)

where

μ_{i} = \frac{F_{i}}{∑_{i=1}^{r} F_{i}} ,

m is the message to be transmitted, and l_{i}, i = 1, 2, …, r, are gain vectors selected in such a way that the matrices

A_{i} − l_{i}c, i = 1, 2, …, r,

are asymptotically stable.

The receiver is a fuzzy observer of the transmitter,

\begin{matrix}˙ ; x̃ & = & ∑_{i=1}^{r} μ_{i} (A_{i}x̃ + b_{i} + l_{i}(y − ỹ)) \\ ỹ & = & cx̃, \end{matrix} \}  (18.123)

Note that

y − ỹ = cx + m − cx̃
= c(x − x̃) + m.

Let

e = x − x̃,

be the estimation error of the transmitter’s state x by the receiver’s state x̃. Then

y − ỹ = ce + m  (18.124)

Thus if

e(t) → 0 as t → ∞,

then we can recover the message because

y − ỹ → m as t → ∞.

Our task then is to formulate conditions guaranteeing that e(t) → 0 as t → ∞. Let us analyze the dynamics of the estimation error,

ė = ẋ − ˙ x̃
= ∑_{i=1}^{r} μ_{i}A_{i}x + ∑_{i=1}^{r} μ_{i}b_{i} + ∑_{i=1}^{r} μ_{i}l_{i}m − ∑_{i=1}^{r} μ_{i}A_{i}x̃ − ∑_{i=1}^{r} μ_{i}b_{i} − ∑_{i=1}^{r} μ_{i}l_{i}(y − ỹ)
= ∑_{i=1}^{r} μ_{i}A_{i}e + ∑_{i=1}^{r} μ_{i}l_{i}m − ∑_{i=1}^{r} μ_{i}l_{i}(cx + m − cx̃)
= ∑_{i=1}^{r} μ_{i}A_{i}e − ∑_{i=1}^{r} μ_{i}l_{i}c(x − x̃)
= ∑_{i=1}^{r} μ_{i}(A_{i} − l_{i}c)e.  (18.125)

We are guaranteed that e(t) → 0 as t → ∞ if can select the gains l_{i}, r = 1, 2, …, r such that there exists a common P = P^{⊤} > 0 so that

(A_{i} − l_{i}c)^{⊤}P + P (A_{i} − l_{i}c) < 0, r = 1, 2, …, r.  (18.126)

The problem of solving simultaneously the above r Lyapunov inequalities can be represented as an LMI feasibility problem. Indeed, performing multiplications in (18.126) yields

A_{i}^{⊤}P − c^{⊤}l_{i}^{⊤}P + P A_{i} − P l_{i}c < 0  (18.127)

Let

M_{i} = P l_{i}.

Then we also have that M_{i}^{⊤} = l_{i}^{⊤}P. Taking the above into account, we represent (18.126) in the form of r LMIs,

A_{i}^{⊤}P + P A_{i} − (c^{⊤}M_{i}^{⊤} + M_{i}c) < 0, r = 1, 2, …, r.  (18.128)

We summarize the above development in the form of the following proposition that appears in [214, p. 214].

Proposition 18.1 If there exist P = P^{⊤} > 0 and l_{i}, r = 1, 2, …, r such that for r = 1, 2, …, r,

A_{i}^{⊤}P + P A_{i} − (c^{⊤}M_{i}^{⊤} + M_{i}c) < 0, M_{i} = P l_{i},

then

l_{i} = P^{−1}M_{i}, r = 1, 2, …, r

and the estimation error e(t) → 0 as t → ∞.

Our fuzzy Lorentz-based secure communication system consists of the transmitter (18.122) and the receiver (18.123). The message to be transmitted is

m(t) = 0.01 square(2πt),

where square(·) generates the square wave. In Figure 18.34 we show the transmitted signal, y = cx + m.

In Figure 18.35, we compare the message signal m against its recovered version. As can be seen, after transients decay the recovered signal matches the message signal perfectly.

Finally, in Figure 18.36, we show a state-space trajectory of of the transmitted signal, that is, the signal generated by the transmitter.

We add that the initial condition of the transmitter was x(0) = [ 1 1 1 ]^{⊤}. We set the zero initial condition for the receiver.

948 CHAPTER 18. CHAOTIC SYSTEMS AND FRACTALS

Figure 18.34: Transmitted signal, ỹ.

Figure 18.35: Plots of the message signal, m, and its recovered version at the receiver’s end obtained from the difference, y − ỹ; see (18.124).

Figure 18.36: State-space trajectory of the transmitter (18.121).

18.8 Notes

Sandefur [293] is a lucid elementary introduction to chaotic discrete-time dynamical systems. Peitgen, Jürgens and Saupe [270] and Alligood, Sauer and Yorke [7] are superb introductions to chaos and dynamical systems. Papers by May [232] and Feigenbaum [100] are recommended for their originality and clarity. Introductory texts to nonlinear dynamics and chaos for scientists and engineers are the books by Moon [246] and Thompson and Stewart [327]. Numerical algorithms for chaotic systems can be found in the book by Parker and Chua [264]. Fractals are covered in Mandelbrot [229], Peitgen and Saupe [271], Devaney and Keen [82], and Bunde and Havlin [47]. Barnsley [24], and Peitgen and Saupe [271] are recommended for their coverage of iterated function systems. A reader who is interested in a mathematical approach to chaotic behavior of dynamical systems should consult the book by Arrowsmith and Place [14]. Peterson [273] gives a remarkable account of the development of models of celestial mechanics leading to the discovery of chaos in the solar system. The use of fractals in education, art, music, fashion, chess and medicine is described in an edited volume by Pickover [274]. Other more recent papers on controlling chaotic systems, in addition to the ones already referenced in the previous section, are: [363], [373], [361]. Applications of chaos in information processing are discussed in the papers by Dmitriev, Panas and Starkov [84] and Andreyev et al. [11]. Aihara, Takabe and Toyoda [5] and Adachi and Aihara [3] discuss the presence of chaos in biological neurons and propose mathematical models of chaotic neural networks. Applications of chaotic neural networks in the complex memory search for patterns with a sought feature are discussed by Nara, Davis and Totsuji [251]; see also [53].

Edward Lorenz was a meteorologist at the Massachusetts Institute of Technology (MIT). His discovery of chaotic behavior of his mathematical model of weather is a fascinating one. As reported by Gleick [115, Chapter 1] and Alligood et al [7, Chapter 9], to speed up his computer output which was simulating a weather model, Lorenz used just three significant digits to print out the results of simulations although the calculations were performed using several more digits of accuracy. One day in the winter of 1961, Lorenz came across a simulation that he decided to repeat. He typed in an initial condition for the new simulation from the printout using the three digit format. He then performed the simulation expecting the same behavior as before of the repeated simulation. Instead, “he saw something unexpected, something that planted a seed for a new science.” [115, page 16]. The resulting simulation was totally different from the previous one. The difference between the two solutions was the result of the Lorenz’s weather model sensitive dependence on initial conditions. In the computer memory, according to Gleick [115], the initial condition for the first run was 0.506127, while to save space, Lorenz entered 0.506 for the second run. This seemingly inconsequential difference caused the two simulations to flow apart until there was no resemblance at all between them. Later, Lorenz greatly simplified his weather model to just three nonlinear equations that are given in Exercise 18.1. The phase portrait of these famous equations resemble butterfly’s wings, which became a symbol for the early activity in chaotic systems.

18.9 Exercises

Exercise 18.1 Simulate the system of differential equations

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ] = [ \begin{matrix}a(−x_{1} + x_{2}) \\ bx_{1} − x_{2} − x_{1}x_{3} \\ −cx_{3} + x_{1}x_{2} \end{matrix} ] , x(0) = [ \begin{matrix}−8 \\ −8 \\ 24 \end{matrix} ] ,  (18.129)

where a = 10, b = 28, and c = 8/3. The set of differential equations (18.129) is commonly referred to as the Lorenz equations, after the American meteorologist E. N. Lorenz who first published these equations in 1963. They model fluid flows in the atmosphere. Plot x_{2} versus x_{1}, then x_{3} versus x_{1}, and also x_{3} versus x_{2}. Use MATLAB’s command plot3 to obtain a 3-D plot of the solution of the Lorenz equations.

Exercise 18.2 Simulate the system of the difference equations

[ \begin{matrix}x_{1}(k + 1) \\ x_{2}(k + 1) \end{matrix} ] = [ \begin{matrix}1 − x_{2}(k) − |x_{1}(k)| \\ x_{1}(k) \end{matrix} ] , x(0) = [ \begin{matrix}−0.1 \\ 0 \end{matrix} ] .  (18.130)

The solution “fills” a region that looks like a “gingerbread man”. For this reason, Devaney, in [271, page 149], refers to this example of a chaotic set in the plane as a chaotic gingerbreadman.

Exercise 18.3 Simulate the system of differential equations

[ ẋ_{1} ; ẋ_{2} ; ẋ_{3} ] = [ \begin{matrix}−x_{2} − x_{3} \\ x_{1} + ax_{2} \\ b + x_{3}(x_{1} − c) \end{matrix} ] ,  (18.131)

where a = 0.398, b = 2, and c = 4. The set of differential equations (18.131) is referred to as the Rössler equations, after O. E. Rössler who reported on their chaotic behavior in 1976. Rössler called the chaotic attractor of the above system the “spiral chaos”. This type of chaotic attractor is also called a folded band. Select an initial condition, integrate the equations, then plot x_{2} versus x_{1}, x_{3} versus x_{1}, and x_{3} versus x_{2}. Use MATLAB’s command plot3 to obtain a 3-D plot of the folded band attractor.

Exercise 18.4 For the dynamical system

x(k + 1) = x(k) + r (0.8 − atan(x(k))) ,  (18.132)

plot a bifurcation diagram for the parameter r in the interval [3, 12]. The map (18.132) was analyzed in [150].

Exercise 18.5 For the dynamical system

x(k + 1) = x(k) + r (0.8 − \frac{1 − e^{−x(k)}}{1 + e^{−x(k)}} ) ,  (18.133)

plot a bifurcation diagram for the parameter r in the interval [10, 55].

Exercise 18.6 For the dynamical system

x(k + 1) = x(k) + r (0.8 − \frac{1}{1 + e^{−x(k)}} ) ,  (18.134)

plot a bifurcation diagram for the parameter r in the interval [10, 45].

Exercise 18.7 Write a MATLAB program of the following “chaos game”: Select three points that are vertices of an equilateral triangle. Choose at random one of the vertices as a starting point. Then, choose at random another vertex and move halfway to it from the starting vertex. Mark this halfway point with a dot. This halfway point is your new starting point. Choose at random one of the vertices and move halfway to the chosen vertex from the new starting point. Mark the new halfway point with the dot. Repeat the process. The outcome of this chaos game is a fractal called the Sierpiński gasket. Generate a plot of the Sierpiński gasket. Other methods of constructing the Sierpiński gasket are described in [270] and in Exercise 18.8. The term “chaos game” was coined by M. F. Barnsley, the author of [24].

Exercise 18.8 The result of the chaos game described in Exercise 18.7 is the Sierpiński gasket that can also be constructed using the following procedure. Start with an equilateral triangle. Divide it into four equal sub-triangles, quarters, and throw away the middle one, upside-down, triangle. This step leaves three equilateral triangles half the size of the original triangle. In the second step, divide each of the remaining three triangles into four equal quarter-triangles and throw away the middle, upside-down, triangle in each of the three triangles. This step leaves nine triangles one-quarter the size of the original triangle. Repeat the process. The remaining area as the number of iterations tends to infinity is called the Sierpiński gasket.

(i) Show that the area of the Sierpiński gasket is zero.

(ii) Determine the fractal dimension of the Sierpiński gasket.

Exercise 18.9 In this exercise, we concern ourselves with another fractal, the Sierpiński carpet. Start with a square. Divide it into nine subsquares, each one-third the size of the original square. Remove the central square. Repeat the procedure. The remaining area as the number of iterations goes to infinity is called the Sierpiński carpet.

(i) Show that the area of the Sierpiński carpet is zero.

(ii) Determine the fractal dimension of the Sierpiński carpet.

Exercise 18.10 For the double pendulum system described in Subsection 2.4.6 set the input torques to zero and simulate the system behavior starting from the initial state x(0) = [ 0 0 π .001 ]^{⊤}. Verify that one of the equilibrium states corresponding to zero input

torques is x_{e} = [ 0 0 π 0 ]^{⊤}. Linearize the state-space model about this equilibrium point. Then, use the LQR approach to compute the state feedback matrix K for the weight matrices

Q = [ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 10 & 0 \\ 0 & 0 & 0 & 10 \end{matrix} ] and R = [ \begin{matrix}1 & 0 \\ 0 & 10 \end{matrix} ] .

Solve the Lyapunov equation

(A − BK)^{⊤}P_{l} + P_{l}(A − BK) = −Q.

Suppose that the bound on the input torques are: |T_{1}| ≤ 5 and |T_{2}| ≤ 2. Use the method of Example 18.5 to find V_{max} and apply the chaotic control law to stabilize the closed-loop system with respect to [ 0 0 π 0 ]^{⊤}. Plot x_{1}, x_{2}, and V = (x − x_{e})^{⊤}P_{l}(x − x_{e}) versus time.

954 CHAPTER 18. CHAOTIC SYSTEMS AND FRACTALS

Appendix A

Math Review

Kroemers Lemma of New Technology: “The principal applications of any sufficiently new and innovative technology always have been and will continue to be applications created by that technology.” In other words: The principal applications have not been of the kind where one improves on something that could already be done on an acceptable level; they have been applications that could previously not been met at all or were not even recognized. As a rule, such applications have indeed arisen, although usually not immediately. The problem with truly new applications is that they are extraordinarily hard to predict, and were historically rarely predicted. Ultimately, progress in applications is not deterministic, but opportunistic, exploiting for new applications whatever new science and technology happen to be coming along.—H. Kroemer [194, p. 958]

————————

“The last word, when all is heard: Fear God and keep his commandments, for this is man’s all; because God will bring to judgment every work, with all its hidden qualities, whether good or bad.” Ecclesiastes 12:13–14

The appendix reviews mathematical techniques that are used in the book. The appendix also serves as a glossary of symbols, notation, and concepts employed in the text. For more information on the topics covered here, the reader should consult references supplied at the end of the appendix.

In an attempt to keep the presentation as clear as possible, we employ the theoremproof style whenever appropriate. Major results of independent interest are encapsulated as theorems. A lemma is used to denote an auxiliary result that is used as a stepping stone towards a theorem. We use the term proposition as a way to announce a minor theorem. A corollary is a direct consequence of a theorem or proposition. A hypothesis is a statement that is taken for further reasoning in a proof. For more information on the subject of writing for mathematical or technical sciences, we recommend a very clear and useful book by Higham [137].

A.1 Notation and Methods of Proof

We often use the term set. The great German mathematician G. Cantor (1845–1918), the creator of the theory of sets, defines a set as any collection of definite, distinguishable objects of our thought, to be conceived as a whole. [227, page 1] The objects, in the definition, are the elements or members of the set. Usually, we use capital letters to denote sets and lower case letters to denote elements of the set. Thus, if X is a set, then we write x ∈ X to mean that x is an element of X. When an object x is not an element of a set X, we write x ∉ X. Often, we use the curly bracket notation for sets. For example, {x : x < 7} denotes the set of all x less than 7. The colon, :, following x is read “such that”. The following sets are used in the book:

ℕ = {1, 2, 3, …}, the set of all positive integers, ℤ = {0, 1, −1, 2, −2, …}, the set of all integers, ℝ, the set of all real numbers, ℂ, the set of all complex numbers.

Sometimes, we take a fixed set, and then carry out manipulations with reference to the fixed set. We call this fixed set the universe of discourse, or just the universe, for short.

Throughout the text, we work with statements that are combinations of individual statements. It is necessary to distinguish between sentences and the statements. A sentence is a part of a language. Sentences are used to make different statements. In classical logic, statements are either true or false. Consider the following two statements:

A : x > 7;

B : x^{2} > 49.

We can combine the above two statements into one statement, called a conditional, that has the form: “IF A THEN B”. Thus, a conditional is a compound statement obtained by placing the word “IF” before the first statement and inserting the word “THEN” before the second statement. We use the symbol “⇒” to represent the conditional “IF A THEN B” as A ⇒ B. The statement A ⇒ B also reads as “A implies B”, or “A only if B”, or “A is sufficient for B”, or “B is necessary for A”. Statements A and B may either be true or false. The relationship between the truth or falsity of A and B and the conditional A ⇒ B can be illustrated by means of a diagram called a truth table shown in Table A.1. In the table, T stands for “true”, and F stands for “false”. It is intuitively clear that if A is true,

Table A.1: Truth table for conditional.

A B A ⇒ B
F F  T
F T  T
T F  F
T T  T

then B must also be true for the statement A ⇒ B to be true. If A is not true, then the sentence A ⇒ B does not have an obvious meaning in everyday language. We can interpret A ⇒ B to mean that we cannot have A true and B not true. One can check that the truth values of the statements A ⇒ B and “not (A and not B)” are the same. We can also check that the truth values of A ⇒ B and “not B ⇒ not A” are the same. The above is the basis for three methods, inference processes, of proving statements of the form A ⇒ B. Inference is a process by which one statement is arrived at, using reasoning, on the basis of one or more other statements accepted as a starting point. Reasoning is a special kind of thinking in which inference takes place using the laws of thought. By a proof we mean the process of establishing the truth of a statement by arriving at it from other statements using principles of reasoning. Most of the time we are concerned with the statements of the form A ⇒ B. We use the following types of proofs of such statements:

(i) the direct method;

(ii) the proof by contraposition;

(iii) the proof by contradiction.

In the first case, the direct method, one starts with A, then tries to deduce B. In the proof by contraposition, one starts with “not B”, then deduces a chain of consequences to end with “not A”. This method of proving is based on the fact that the truth values of A ⇒ B and “not B ⇒ not A” are the same. The proof by contradiction is based on the fact that the truth values of A ⇒ B and “not (A and not B)” are the same. In this method, one starts with “A and not B” and tries to arrive at a contradiction, which is a statement that is always false.

It is possible to combine two conditionals to form a biconditional, which is a statement of the form “A if and only if B”. Occasionally, we use the symbol “⇔” to write A ⇔ B to mean “A if and only if B”. Another way of expressing A ⇔ B is “A is equivalent to B”, or “A is necessary and sufficient for B”.

A.2 Vectors

We define an n-dimensional column vector to be an n-tuple of numbers

[ x_{1} ; x_{2} ; ⋮ ; x_{n} ] ,  (A.1)

where n is a positive integer. Thus a vector can be viewed as an ordered set of scalars. We use lower-case bold letters, for example x, to denote such n-tuples. We call the numbers x_{1}, x_{2}, …, x_{n} the coordinates or components of the vector x. The number of components of a vector is its dimension. When referring to a particular component of a vector, we use its component, for example x_{i}, or the subscripted name of the vector, for example (x)_{i} to denote x_{i}. The set of all n-tuples, that is n-vectors, with real coefficients will be denoted ℝ^{n} and called the real n-space. The transpose of the vector x, denoted x^{⊤}, is an n-dimensional row vector

x^{⊤} = [ x_{1} ; x_{2} ; ⋮ ; x_{n} ]^{⊤} = [ x_{1} x_{2} ⋯ x_{n} ] .  (A.2)

The sum of two n-vectors x and y is an n-vector, denoted x + y, whose i-th component is x_{i} + y_{i}. The following rules are satisfied:

1. (x + y) + z = x + (y + z), that is, vector addition is associative.

2. x + y = y + x, which means that vector addition is commutative.

3. Let 0 = [ 0 0 ⋯ 0 ]^{⊤} denote the zero vector. Then,

0 + x = x + 0 = x.

4. Let x = [ x_{1} x_{2} ⋯ x_{n} ]^{⊤} and let −x = [ −x_{1} −x_{2} ⋯ −x_{n} ]^{⊤}. Then,

x + (−x) = 0.

The product of an n-vector x by a scalar c is an n-th vector, denoted cx, whose i-th component is cx_{i}.

For two vectors x and y from ℝ^{n}, we define their scalar product, also called the inner product, to be

⟨x, y⟩ = x^{⊤}y = x_{1}y_{1} + x_{2}y_{2} + ⋯ + x_{n}y_{n}.  (A.3)

The scalar product is commutative,

⟨x, y⟩ = ⟨y, x⟩,

and distributivity over vector addition holds,

⟨x, y + z⟩ = ⟨x, y⟩ + ⟨x, z⟩.

Two vectors x and y are said to be orthogonal if ⟨x, y⟩ = 0.

A vector norm, denoted ∥ · ∥, satisfies the following properties:

VN 1. ∥x∥ > 0 for any nonzero vector.

VN 2. For any scalar c, ∥cx∥ = |c|∥x∥.

VN 3. For any two vectors x and y, ∥x + y∥ ≤ ∥x∥ + ∥y∥—the triangle inequality.

The Euclidean norm, or the 2-norm, of a vector x ∈ ℝ^{n}, denoted ∥x∥_{2}, is defined as

∥x∥_{2} = \sqrt{⟨x, x⟩} = \sqrt{x_{1}^{2} + x_{2}^{2} + ⋯ + x_{n}^{2}}.  (A.4)

Note that ∥x∥_{2}^{2} = ⟨x, x⟩ = x^{⊤}x. The Euclidean norm is a special class of norms called the Hölder or p-norms defined by

∥x∥_{p} = (|x_{1}|^{p} + ⋯ + |x_{n}|^{p})^{1/p} , p ≥ 1.  (A.5)

Other often used norms that are special cases of the above class of norms are the 1-norm and the infinity norm defined respectively as

∥x∥_{1} = |x_{1}| + ⋯ + |x_{n}|  (A.6)

and

∥x∥_{∞} = max_{i} |x_{i}|.  (A.7)

Most of the time, we use the Euclidean norm for vectors. So for the sake of brevity, we write ∥x∥ to mean ∥x∥_{2}.

Theorem A.1 (Pythagoras) If x and y are orthogonal, then

∥x + y∥^{2} = ∥x∥^{2} + ∥y∥^{2}.

Proof We have,

∥x + y∥^{2} = (x + y)^{⊤} (x + y)
= ∥x∥^{2} + 2⟨x, y⟩ + ∥y∥^{2}
= ∥x∥^{2} + ∥y∥^{2},

since by assumption ⟨x, y⟩ = 0. The proof of the theorem is complete.

□

We now introduce the notion of projection. Let x, y be two vectors, where y ≠ 0. Let z = cy, c ∈ ℝ, be the vector such that the vector x − z is orthogonal to y—see Figure A.1 for an illustration of the above construction. Thus,

(x − z)^{⊤} y = 0.  (A.8)

Substituting z = cy into (A.8) yields

(x − z)^{⊤} y = (x − cy)^{⊤} y = 0.  (A.9)

Hence,

c = \frac{⟨x, y⟩}{∥y∥^{2}}.  (A.10)

The number c given by (A.10) is called the component of x along y. The projection of x along y is the vector

z = cy = \frac{⟨x, y⟩}{∥y∥^{2}}y.  (A.11)

[figure: y; x; z = cy; 0; x − z]

Figure A.1: Computing the projection of x along y.

Theorem A.2 (Cauchy-Schwarz Inequality) Let x, y ∈ ℝ^{n}. Then,

|⟨x, y⟩| ≤ ∥x∥∥y∥.  (A.12)

Proof If y = 0, then both sides of (A.12) are equal to 0, and the assertion holds. Suppose that y ≠ 0, and let c denote the component of x along y. Hence, c is given by (A.10). We then write

x = x − z + z = x − cy + cy.  (A.13)

For an illustration of (A.13), consult Figure A.1. Applying the Pythagoras theorem to (A.13), we obtain

∥x∥^{2} = ∥x − cy∥^{2} + ∥cy∥^{2} = ∥x − cy∥^{2} + c^{2}∥y∥^{2}.  (A.14)

Hence,

c^{2}∥y∥^{2} ≤ ∥x∥^{2}.  (A.15)

On the other hand, by (A.10),

c^{2}∥y∥^{2} = \frac{⟨x, y⟩^{2}}{∥y∥^{4}} ∥y∥^{2} = \frac{⟨x, y⟩^{2}}{∥y∥^{2}} .  (A.16)

Combining (A.15) and (A.16) yields

\frac{⟨x, y⟩^{2}}{∥y∥^{2}} ≤ ∥x∥^{2}.  (A.17)

962 APPENDIX A. MATH REVIEW Multiplying both sides of (A.17) by ∥y∥^{2} and taking the square root, concludes the proof.

□

Theorem A.3 (Triangle inequality) Let x, y be two vectors in ℝ^{n}. Then,

∥x + y∥ ≤ ∥x∥ + ∥y∥.  (A.18)

Proof Both sides of (A.18) are nonnegative. Hence, to prove that (A.18) holds it suffices to show that the squares of the both sides satisfy the desired inequality,

(x + y)^{⊤} (x + y) ≤ (∥x∥ + ∥y∥)^{2} .  (A.19)

Expanding (x + y)^{⊤} (x + y) and applying the Cauchy-Schwarz inequality yields

(x + y)^{⊤} (x + y) = ∥x∥^{2} + 2⟨x, y⟩ + ∥y∥^{2}
≤ |x∥^{2} + 2∥x∥∥y∥ + ∥y∥^{2}
= (∥x∥ + ∥y∥)^{2} ,  (A.20)

which completes the proof of the theorem.

□

Theorem A.4 Let x, y be two vectors in ℝ^{n}. Then, |∥x∥ − ∥y∥| ≤ ∥x − y∥.

Proof We have

∥x∥ = ∥ (x − y) + y∥ ≤ ∥x − y∥ + ∥y∥.

Hence,

∥x∥ − ∥y∥ ≤ ∥x − y∥.  (A.21)

The above implies that

∥x − y∥ = ∥y − x∥ ≥ ∥y∥ − ∥x∥ = − (∥x∥ − ∥y∥) .  (A.22)

Combining (A.21) and (A.22) yields

|∥x∥ − ∥y∥| ≤ ∥x − y∥,

thus completing the proof.

□

A.3 Matrices and Determinants

Let m, n be two integers such that m, n ≥ 1. An array of numbers

[ \begin{matrix}a_{11} & a_{12} & ⋯ & a_{1n} \\ a_{21} & a_{22} & ⋯ & a_{2n} \\ ⋮ & ⋮ & ... & ⋮ \\ a_{m1} & a_{m2} & ⋯ & a_{mn} \end{matrix} ]

is called a matrix. Thus a matrix is a set of scalars subject to two orderings. We say that it is an m by n matrix, or m × n matrix. This matrix has m rows and n columns. If m = n, then we say that it is a square matrix. An m × n matrix A whose elements are real numbers will be denoted as A ∈ ℝ^{m×n}. If the elements of the matrix A are complex numbers, then we write A ∈ ℂ^{m×n}. We define the zero matrix, denoted O, to be the matrix such that a_{ij} = 0 for all i, j. We define the n × n identity matrix, denoted I_{n}, as the matrix having diagonal components all equal to 1 and all other components all equal to 0.

Given an m×n matrix A. The n×m matrix B such that b_{ji} = a_{ij} is called the transpose of A and is denoted A^{⊤}. Thus, the transpose of a matrix is obtained by changing rows into columns and vice versa. A matrix A which is equal to its transpose, that is, A = A^{⊤}, is called symmetric.

We now define the sum of two matrices. Given two matrices A and B of the same size. We define A + B to be the matrix whose elements are a_{ij} + b_{ij}, that is, we add matrices of the same size componentwise.

Let c be a number and A be a matrix. We define cA to be the matrix whose components are ca_{ij}. Thus, to obtain cA, we multiply each component of A by c.

We now define the product of two matrices. Let A be an m × n matrix and B be an n × s matrix. Let a_{i}^{⊤} denote the i-th row of A, and b_{j} the j-th column of B, Then, the product of AB is an m × s matrix C defined as

C = AB = [ \begin{matrix}a_{1}^{⊤}b_{1} & a_{1}^{⊤}b_{2} & ⋯ & a_{1}^{⊤}b_{s} \\ a_{2}^{⊤}b_{1} & a_{2}^{⊤}b_{2} & ⋯ & a_{2}^{⊤}b_{s} \\ ⋮ & ⋮ & ... & ⋮ \\ a_{m}^{⊤}b_{1} & a_{m}^{⊤}b_{2} & ⋯ & a_{m}^{⊤}b_{s} \end{matrix} ] .  (A.23)

It is not always true that AB = BA, that is, matrix multiplication is not necessarily commutative.

Let A, B, and C be matrices such that A, B can be multiplied, B, C can be multiplied, and B, C can be added. Then,

A(B + C) = AB + AC.  (A.24)

The above property is called the distributive law. If x is a number, then

A(xB) = x(AB).  (A.25)

Furthermore, matrix multiplication is associative because

(AB)C = A(BC).  (A.26)

Suppose now that A, B can be multiplied. Then,

(AB)^{⊤} = B^{⊤}A^{⊤}.  (A.27)

Thus the transpose of a product is the product of transposed matrices in reverse order. Let A be an n × n matrix. An inverse of A, if it exists, is an n × n matrix B such that

AB = BA = I_{n}.  (A.28)

If an inverse exists, then it is unique. A square matrix that is invertible is said to be nonsingular. We denote the inverse of A by A^{−1}. Hence, we have

A^{−1}A = AA^{−1} = I_{n}.

The transpose of an inverse is the inverse of the transpose,

(A^{−1})^{⊤} = (A^{⊤})^{−1} ,  (A.29)

and we denote it A^{−T}.

We now present the Sherman-Morrison-Woodbury formula for inverting a special kind of matrices.

Lemma A.1 If A, D, and D^{−1} + CA^{−1}B are invertible, then

(A + BDC)^{−1} = A^{−1} − A^{−1}B (D^{−1} + CA^{−1}B)^{−1} CA^{−1}.

Proof We prove the Sherman-Morrison-Woodbury formula by verification. We have

(A + BDC) (A^{−1} − A^{−1}B (D^{−1} + CA^{−1}B)^{−1} CA^{−1})
= (A + BDC) A^{−1}
− (A + BDC) A^{−1}B (D^{−1} + CA^{−1}B)^{−1} CA^{−1}
= I + BDCA^{−1}
− (B + BDCA^{−1}B) (D^{−1} + CA^{−1}B)^{−1} CA^{−1}
= I + BDCA^{−1}
−BD (D^{−1} + CA^{−1}B) (D^{−1} + CA^{−1}B)^{−1} CA^{−1}
= I + BDCA^{−1} − BDCA^{−1}
= I,

as desired.

□

In the case when D = I, the Sherman-Morrison-Woodbury formula takes the form

(A + BC)^{−1} = A^{−1} − A^{−1}B (I + CA^{−1}B)^{−1} CA^{−1}.  (A.30)

Formula (A.30) is sometimes referred to as the matrix inversion formula.

A set of nonzero vectors {q_{1}, q_{2}, …, q_{r}} is said to be orthogonal if

⟨q_{i}, q_{j}⟩ = 0, for i ≠ j.

If each vector in an orthogonal set is of unity length, that is, ∥q_{i}∥ = 1 for all i = 1, 2 …, r, then the set of vectors is also called orthonormal. A square matrix with orthogonal columns is called an orthogonal matrix. Sometimes in the literature, an orthogonal matrix is called an unitary matrix. If Q ∈ ℝ^{n×n} is an orthogonal matrix, then

Q^{⊤}Q = QQ^{⊤} = I_{n}.  (A.31)

For a square, n × n, matrix A, its trace, denoted traceA, is the sum of its diagonal elements,

traceA = ∑_{i=1}^{n} a_{ii}.  (A.32)

If B is also an n × n matrix, then

trace(AB) = trace(BA).  (A.33)

The determinant of a square matrix A, denoted detA, can be evaluated using the Laplace expansion as follows. Let c_{ij} denote the cofactor corresponding to the element a_{ij} of an n×n matrix A. The cofactor c_{ij} is (−1)^{i+j} times the determinant of the (n−1)×(n−1) submatrix of A that is obtained by deleting the i-th row and j-th column of A. Then, for any fixed i, 1 ≤ i ≤ n, we have

det A = ∑_{j=1}^{n} a_{ij}c_{ij}.  (A.34)

This is the expansion of the determinant along the i-th row. In a similar fashion, we can expand the determinant along a column. The determinant has the following properties:

D 1. det A = det A^{⊤}

D 2. Let A and B be square matrices of the same size, then det (AB) = det A det B =

det (BA)

D 3. det O = 0

D 4. det I_{n} = 1.

We now list additional properties of determinants. Let a_{i} denote the i-th column of A. Then, we write

det A = det (a_{1}, a_{2}, …, a_{n}) .

D 5. Suppose that the i-th column a_{i} is written as a sum

a_{i} = b + c.

Then, we have

det (a_{1}, …, a_{i}, …, a_{n}) = det (a_{1}, …, b + c, …, a_{n})
= det (a_{1}, …, b, …, a_{n}) + det (a_{1}, …, c, …, a_{n}) .

D 6. If x is a number, then

det (a_{1}, …, x a_{i}, …, a_{n}) = x det (a_{1}, …, a_{i}, …, a_{n}) .

D 7. If two columns of the matrix are equal, then the determinant is equal to zero.

D 8. If we add a multiple of one column to another, then the value of the determinant does not change. In other words, if x is a number, then

det (a_{1}, …, a_{i}, …, a_{j}, …, a_{n}) = det (a_{1}, …, a_{i} + xa_{j}, …, a_{j}, …, a_{n}) .

Properties D 5–D 8 hold also for rows instead of columns because det A = det A^{⊤}.

Using the cofactors of A and its determinant, we can give a formula for A^{−1}. First, we define the adjugate of A, denoted adjA to be the matrix whose (i, j)-th entry is the cofactor c_{ji}, that is, adjA is the transpose of the cofactor matrix of A. Then,

A^{−1} = (\frac{1}{det A}) adjA = \begin{matrix}adj & A \\ det & A \end{matrix}.  (A.35)

If A is n × n and v is a nonzero n × 1 vector such that for some scalar λ,

Av = λv,  (A.36)

then v is an eigenvector corresponding to the eigenvalue λ. In order that λ be an eigenvalue of A, it is necessary and sufficient for the matrix λI_{n}−A to be singular, that is, det (λI_{n} − A) = 0. This leads to an n-th order polynomial equation in λ of the form

det (λI_{n} − A) = λ^{n} + a_{n−1}λ^{n−1} + ⋯ + a_{1}λ + a_{0} = 0.  (A.37)

This equation, called the characteristic polynomial equation, must have n, possibly nondistinct, complex roots which are the eigenvalues of A. The spectral radius of A, denoted ρ(A) is defined as

ρ(A) = max_{1≤i≤n} |λ_{i}(A)|,

where λ_{i}(A) denotes the i-th eigenvalue of A.

Any m × n matrix A can be represented as

A = U [ S, O ] V^{⊤}, if m ≤ n,  (A.38)

or

A = U [ \begin{matrix}S \\ O \end{matrix} ] V^{⊤}, if m > n,  (A.39)

where U is an m × m orthogonal matrix, V is an n × n orthogonal matrix, and S is a diagonal matrix of the form

S = diag(σ_{1}, σ_{2}, …, σ_{p}), where p = min(m, n) and σ_{i} ≥ 0, i = 1, 2, …, p.

The nonnegative numbers {σ_{i}} are called the singular values of A, and (A.38), or (A.39), is called the singular value decomposition (SVD) of A. Following the convention, we write

σ_{1} ≥ σ_{2} ≥ ⋯ ≥ σ_{p},

so that σ_{1} = σ_{1}(A) denotes the largest singular value of A. The singular values of A are the square roots of the eigenvalues of AA^{⊤} if m ≤ n or of A^{⊤}A if m > n. If A is symmetric, its singular values are the absolute values of its eigenvalues.

A.4 Quadratic Forms

A real quadratic form is a special type of a real-valued function of n variables. It is the product x^{⊤}Qx, where Q is a specified n × n matrix with real coefficients and x is an n × 1 real, column, vector. Without loss of generality, the matrix Q can be taken to be symmetric, that is, Q = Q^{⊤}, for if Q is not symmetric, we can always replace it with a symmetric matrix without changing the quadratic form values. Indeed, because z = x^{⊤}Qx is a scalar, hence z^{⊤} = z, and we have

(x^{⊤}Qx)^{⊤} = x^{⊤}Q^{⊤}x = x^{⊤}Qx.

Therefore, we can write

x^{⊤}Qx = x^{⊤} (\frac{1}{2}Q + \frac{1}{2}Q^{⊤}) x = x^{⊤} (\frac{Q + Q^{⊤}}{2} ) x,

where \frac{1}{2} (Q + Q^{⊤}) is a symmetric matrix. Thus, the quadratic form values are unchanged if Q is replaced with the symmetric matrix \frac{1}{2} (Q + Q^{⊤}). From now on, in our study of quadratic forms, we assume that Q is symmetric.

A quadratic form x^{⊤}Qx, or equivalently, the matrix Q, is called positive semi-definite if x^{⊤}Qx ≥ 0 for all x. It is positive definite if x^{⊤}Qx > 0 for all x ≠ 0. If Q is positive semi-definite, then we write Q ≥ 0, if it is positive definite, we write Q > 0. We write Q_{1} ≥ Q_{2} to mean that Q_{1} − Q_{2} ≥ 0. We say that Q is negative semi-definite if x^{⊤}Qx ≤ 0 for all x, it is negative definite if x^{⊤}Qx < 0 for all x ≠ 0. Note that Q ≤ 0 if and only if −Q ≥ 0.

We next present tests for definiteness properties of symmetric matrices. We state these tests in terms of principal minors of a given real, symmetric, n × n matrix Q with elements q_{ij}. We will also give tests for definiteness in terms of eigenvalues of Q. The principal minors of Q are det Q itself and the determinants of submatrices of Q obtained by removing successively an i-th row and i-th column. Specifically, for p = 1, 2, …, n, the principal minors are

Δ_{p} ( \begin{matrix}i_{1}, & i_{2}, & …, & i_{p} \\ i_{1}, & i_{2}, & …, & i_{p} \end{matrix} ) = det [ \begin{matrix}q_{i_{1}i_{1}} & q_{i_{1}i_{2}} & ⋯ & q_{i_{1}i_{p}} \\ q_{i_{2}i_{1}} & q_{i_{2}i_{2}} & ⋯ & q_{i_{2}i_{p}} \\ ⋮ & ⋮ & ... & ⋮ \\ q_{i_{p}i_{1}} & q_{i_{p}i_{2}} & ⋯ & q_{i_{p}i_{p}} \end{matrix} ] 1 ≤ i_{1} < i_{2} < ⋯ < i_{p} ≤ n.

The above are called the principal minors of order p.

In the following, we use the fact that if Q = Q^{⊤}, then its eigenvalues are all real.

A.4. QUADRATIC FORMS 969 Theorem A.5 The following are equivalent:

(i) The symmetric matrix Q is positive semi-definite. (ii) Principal minors of Q are nonnegative, that is, for p = 1, 2, …, n,

Δ_{p} ( \begin{matrix}i_{1}, & i_{2}, & …, & i_{p} \\ i_{1}, & i_{2}, & …, & i_{p} \end{matrix} ) ≥ 0, 1 ≤ i_{1} < i_{2} < ⋯ < i_{p} ≤ n.

(iii) Eigenvalues of Q are nonnegative,

λ_{i}(Q) ≥ 0, i = 1, 2, …, n.

We give two tests for Q = Q^{⊤} to be positive definite; one in terms of its leading principal minors, and the other in terms of its eigenvalues. The leading principal minors of Q are

Δ_{1} ( \begin{matrix}1 \\ 1 \end{matrix} ) = q_{11}, Δ_{2} ( \begin{matrix}1, & 2 \\ 1, & 2 \end{matrix} ) = det [ \begin{matrix}q_{11} & q_{12} \\ q_{21} & q_{22} \end{matrix} ] ,

Δ_{3} ( \begin{matrix}1, & 2, & 3 \\ 1, & 2, & 3 \end{matrix} ) = det [ \begin{matrix}q_{11} & q_{12} & q_{13} \\ q_{21} & q_{22} & q_{23} \\ q_{31} & q_{32} & q_{33} \end{matrix} ] , …, Δ_{n} ( \begin{matrix}1, & 2, & …, & n \\ 1, & 2, & …, & n \end{matrix} ) = det Q.

Theorem A.6 The following are equivalent:

(i) The symmetric matrix Q is positive definite. (ii) Leading principal minors of Q are positive, that is,

Δ_{p} ( \begin{matrix}1, & 2, & …, & p \\ 1, & 2, & …, & p \end{matrix} ) > 0, p = 1, 2, …, n.

(iii) Eigenvalues of Q are positive,

λ_{i}(Q) > 0, i = 1, 2, …, n.

If a given matrix Q ∈ ℝ^{n×n} is symmetric, then its eigenvalues are all real and there exists an orthogonal matrix U such that

U^{⊤}QU = [ \begin{matrix}λ_{1} & 0 & ⋯ & 0 \\ 0 & λ_{2} & ⋯ & 0 \\ ⋮ & & ... & ⋮ \\ 0 & 0 & ⋯ & λ_{n} \end{matrix} ] = Λ.  (A.40)

Recall that U is orthogonal if U^{⊤}U = I_{n}. Using the above, we show that a symmetric positive semi-definite matrix Q has a positive semi-definite symmetric square root, denoted Q^{1/2} or \sqrt{Q}, that satisfies

Q^{1/2}Q^{1/2} = Q.  (A.41)

Indeed, it follows from Theorem A.5 that if Q = Q^{⊤} ≥ 0 then its eigenvalues are all nonnegative. Hence,

Λ^{1/2} = [ \begin{matrix}λ_{1}^{1/2} & 0 & ⋯ & 0 \\ 0 & λ_{2}^{1/2} & ⋯ & 0 \\ ⋮ & & ... & ⋮ \\ 0 & 0 & ⋯ & λ_{n}^{1/2} \end{matrix} ]  (A.42)

is well defined. Using the fact that U^{⊤}U = I_{n}, we define

Q^{1/2} = UΛ^{1/2}U^{⊤},  (A.43)

which has the desired properties.

If Q = Q^{⊤} is positive semi-definite, and not positive definite, then some of its eigenvalues are equal to zero. Suppose that Q has r nonzero eigenvalues. Then, there is an orthogonal matrix, say U, such that

U^{⊤}QU = [ \begin{matrix}λ_{1} & 0 & ⋯ & 0 & 0 & ⋯ & 0 \\ 0 & λ_{2} & ⋯ & 0 & 0 & ⋯ & 0 \\ ⋮ & & ... & ⋮ & & ... & ⋮ \\ 0 & 0 & ⋯ & λ_{r} & 0 & ⋯ & 0 \\ 0 & 0 & ⋯ & 0 & 0 & ⋯ & 0 \\ ⋮ & & & & & ... & ⋮ \\ 0 & 0 & ⋯ & 0 & 0 & ⋯ & 0 \end{matrix} ] .  (A.44)

Let V ∈ ℝ^{n×r} be a matrix that consists of the first r columns of the matrix U, and let

C = [ \begin{matrix}λ_{1}^{1/2} & 0 & ⋯ & 0 \\ 0 & λ_{2}^{1/2} & ⋯ & 0 \\ ⋮ & & ... & ⋮ \\ 0 & 0 & ⋯ & λ_{r}^{1/2} \end{matrix} ] V^{⊤}.  (A.45)

Note that the matrix C is of full rank. We have

Q = C^{⊤}C.  (A.46)

We refer to (A.46) as a full rank factorization of Q because C is of full rank. We can generalize the above, using (A.38), to obtain a full rank factorization of a non-square matrix. Specifically, given an m × n matrix A of rank r, we can represent it as

A = BC,

where B is an m × r matrix and C is a r × n matrix such that

rank(B) = rank(C) = rank(A) = r.

A.5 The Kronecker Product

Let A be an m × n and B be a p × q matrices. The Kronecker product of A and B, denoted A ⊗ B, is an mp × nq matrix defined as

A ⊗ B = [ \begin{matrix}a_{11}B & a_{12}B & ⋯ & a_{1n}B \\ a_{21}B & a_{22}B & ⋯ & a_{2n}B \\ ⋮ & ⋮ & ... & ⋮ \\ a_{m1}B & a_{m2}B & ⋯ & a_{mn}B \end{matrix} ] ,  (A.47)

where the symbol ⊗ reads “otimes.” Thus, the matrix A ⊗ B consists of mn blocks. Using the definition of the Kronecker product, we can verify the following two properties:

(A ⊗ B) (C ⊗ D) = AC ⊗ BD,  (A.48)
(A ⊗ B)^{⊤} = A^{⊤} ⊗ B^{⊤}.  (A.49)

Let

X = [ x_{1}, x_{2}, …, x_{m} ]

be a n × m matrix, where x_{i}, i = 1, 2, …, m are the columns of X. Each x_{i} consists of n elements. Then, the vectorization operator or stacking operator is defined as

vec(X) = [ x_{1}^{⊤}, x_{2}^{⊤}, …, x_{m}^{⊤} ]^{⊤}
= [ x_{11} x_{21} ⋯ x_{n1} x_{12} x_{22} ⋯ x_{n2} ⋯ x_{1m} x_{2m} ⋯ x_{nm} ]^{⊤}

is the column nm-vector formed from the columns of X taken in order. Let now A be an n × n, and C and X be n × m matrices. Then, the matrix equation

AX = C  (A.50)

can be written as

(I_{m} ⊗ A) vec(X) = vec(C).  (A.51)

Let B be an m × m matrix, then the matrix equation

XB = C  (A.52)

can be written as

(B^{⊤} ⊗ I_{n}) vec(X) = vec(C).  (A.53)

Using the above two facts, we can verify that the Sylvester matrix equation,

AX + XB = C,  (A.54)

where A is n × n, B is m × m, and C is n × m, can be written as

(I_{m} ⊗ A + B^{⊤} ⊗ I_{n}) vec(X) = vec(C).  (A.55)

Let λ_{i}, v_{i} be the eigenvalues and eigenvectors, respectively, of A, and μ_{j} and w_{j} the eigenvalues and eigenvectors of m × m matrix B. Then,

(A ⊗ B) (v_{i} ⊗ w_{j}) = Av_{i} ⊗ Bw_{j}
= λ_{i}v_{i} ⊗ μ_{j}w_{j}
= λ_{i}μ_{j} (v_{i} ⊗ w_{j}) .  (A.56)

Thus, the eigenvalues of A ⊗ B are λ_{i}μ_{j}, and their respective eigenvectors are v_{i} ⊗ w_{j} for i = 1, 2, …, n, j = 1, 2, …, m.

We use the above result to study the Sylvester equation, given by (A.54), that we represent as

M vec(X) = vec(C),  (A.57)

where

M = I_{m} ⊗ A + B^{⊤} ⊗ I_{n}.  (A.58)

The solution to (A.57) is unique if, and only if, the mn × mn matrix M is nonsingular. To find the condition for this to hold, consider the following matrix

(I_{m} + εB^{⊤}) ⊗ (I_{n} + εA) = I_{m} ⊗ I_{n} + εM + ε^{2}B^{⊤} ⊗ A  (A.59)

whose eigenvalues are

(1 + εμ_{j}) (1 + ελ_{i}) = 1 + ε (μ_{j} + λ_{i}) + ε^{2}μ_{j}λ_{i}  (A.60)

because for a square matrix Q,

λ_{i} (I_{n} + εQ) = 1 + ελ_{i} (Q) .

Comparing terms in ε in (A.59) and (A.60), we conclude that the eigenvalues of M are λ_{i} + μ_{j}, i = 1, 2, …, n, j = 1, 2, …, m. Hence, M is nonsingular if and only if

λ_{i} + μ_{j} ≠ 0.  (A.61)

The above is the necessary and sufficient condition for the solution X of the matrix equation AX + XB = C to be unique.

Another useful identity involving the Kronecker product is

vec(ABC) = (C^{⊤} ⊗ A) vec(B)  (A.62)

For more information on the subject of the Kronecker product, we refer the reader to Brewer [42].

A.6 Upper and Lower Bounds

Consider a set S of real numbers. A number M is called an upper bound of the set S if

x ≤ M for all x ∈ S.

A set of real numbers that has an upper bound is said to be bounded above.

The Least Upper Bound Axiom Every nonempty set S of real numbers that has an upper bound has a least upper bound, also called supremum, and is denoted

sup{x : x ∈ S}.

Examples 1.

sup \{\frac{1}{2}, \frac{2}{3}, …, \frac{n}{n + 1}, …\} = 1;

2.

sup \{−\frac{1}{2}, −\frac{1}{8}, −\frac{1}{27}, …, −\frac{1}{n^{3}} , …\} = 0;

3.

sup \{x : x^{2} < 3\} = sup \{x : −\sqrt{3} < x < \sqrt{3}\} = \sqrt{3}.

Theorem A.7 If M = sup{x : x ∈ S} and ε > 0, then there is at least one number x in S such that

M − ε < x ≤ M.

Proof The condition x ≤ M is satisfied by all numbers x in S by virtue of the Least Upper Bound Axiom. We have to show that for an ε > 0 there is a number x ∈ S such that

M − ε < x.

We prove the statement by contradiction. We suppose that there is no such number in S. Then,

x ≤ M − ε for all x ∈ S

and hence M − ε would be an upper bound of S that is less than M which is the least upper bound. This contradicts the assumption, and thus the proof is complete.

□

To illustrate the above theorem, we consider the following set of real numbers:

S = \{\frac{1}{2}, \frac{2}{3}, \frac{3}{4}, …, \frac{n}{n + 1}, …\} .

Note that

sup{x : x ∈ S} = 1.

Let ε = 0.1. Then, for example, the element of the set S that is equal to 99/100 satisfies

1 − ε < \frac{99}{100} ≤ 1.

A number m is called a lower bound of S if

m ≤ x for all x ∈ S.

Sets that have lower bounds are said to be bounded below.

Theorem A.8 Every nonempty set of real numbers that has a lower bound has a greatest lower bound, also called infimum, denoted as

inf{x : x ∈ S}.

Proof By assumption S is nonempty and has a lower bound that we call s. Thus

s ≤ x for all x ∈ S.

Hence

−x ≤ −s for all x ∈ S,

that is, the set {−x : x ∈ S} has an upper bound −s. From the Least Upper Bound Axiom, we conclude that the set {−x : x ∈ S} has a least upper bound, supremum. We call it −s_{0}. Because

−x ≤ −s_{0} for all x ∈ S,

we have

s_{0} ≤ x for all x ∈ S,

and thus s_{0} is a lower bound for S. We now prove, by contradiction, that s_{0} is the greatest lower bound, infimum, of the set S. Indeed, if there was a number x̃ satisfying

s_{0} < x̃ ≤ x for all x ∈ S,

then we would have

−x ≤ −x̃ < −s_{0} for all x ∈ S,

which contradicts the fact that −s_{0} is the supremum of {−x : x ∈ S}.

□

Using arguments similar to those we used in the proof of Theorem A.8, we can prove the following theorem:

Theorem A.9 If m = inf{x : x ∈ S} and ε > 0, then there is at least one number x in S such that

m ≤ x < m + ε.

A.7 Sequences

A sequence of real numbers is a function whose domain is the set of natural numbers 1, 2, …, n, … and whose range is contained in ℝ. Thus, a sequence of real numbers can be viewed as a set of points (1, a_{1}), (2, a_{2}), …, (n, a_{n}), . . .. We denote a sequence of real numbers

a_{1}, a_{2}, …, a_{n}, …

as {a_{n}}.

A sequence {a_{n}} is increasing if a_{1} < a_{2} < ⋯ < a_{n} · · ·. In general, a sequence is increasing if a_{n} < a_{n+1}. If a_{n} ≤ a_{n+1}, then we say that the sequence is nondecreasing. Similarly, we can define decreasing and nonincreasing sequences. Nonincreasing or nondecreasing sequences are called monotone sequences.

Figure A.2: An illustration of the notion of the limit of a sequence.

A number g is called the limit of the infinite sequence a_{1}, a_{2}, …, a_{n}, … if for any positive ε there is a number k = k(ε) such that for all n > k

|a_{n} − g| < ε,

that is, a_{n} lies between g − ε and g + ε for all n > k. In other words, for any ε > 0 we have |a_{n} − g| < ε for sufficiently large n’s, see Figure A.2, and we write

g = lim_{n→∞} a_{n},

or

a_{n} → g.

A sequence that has a limit is called a convergent sequence. A sequence that has no limit is called divergent.

The notion of the sequence can be extended to sequences with elements in ℝ^{p}. Specifically, a sequence in ℝ^{p} is a function whose domain is the set of natural numbers 1, 2, …, n, … and whose range is contained in ℝ^{p}.

Suppose that a_{1}, a_{2}, …, a_{n}, …, denoted {a_{n}}, is a sequence in ℝ^{p}. Then, g ∈ ℝ^{p} is called the limit of this sequence if for each ε > 0 there exists a natural number k = k(ε) such that for all n > k

∥a_{n} − g∥ < ε,

and we write

g = lim_{n→∞} a_{n},

or

a_{n} → g.

Theorem A.10 If a limit of a sequence in ℝ^{p} exists, then it is unique.

Proof We prove this theorem by contradiction. Let us assume that a sequence {a_{n}} has two different limits, say g_{1} and g_{2}. Then, we have

∥g_{1} − g_{2}∥ > 0.

Let

ε = \frac{1}{2} ∥g_{1} − g_{2}∥ .

From the definition of a limit, there exist k_{1} = k_{1}(ε) and k_{2} = k_{2}(ε) such that for n > k_{1} we have ∥a_{n} − g_{1}∥ < ε, and for n > k_{2} we have ∥a_{n} − g_{2}∥ < ε. Let m = max (k_{1}, k_{2}). Then, if n > m, we have ∥a_{n} − g_{1}∥ < ε and ∥a_{n} − g_{2}∥ < ε. Adding ∥a_{n} − g_{1}∥ < ε and ∥a_{n} − g_{2}∥ < ε yields

∥a_{n} − g_{1}∥ + ∥a_{n} − g_{2}∥ < 2ε.  (A.63)

It follows from the triangle inequality that

∥a − b∥ ≤ ∥a∥ + ∥b∥.

Using the above inequality, we obtain

∥−g_{1} + g_{2}∥ = ∥a_{n} − g_{1} − a_{n} + g_{2}∥
= ∥(a_{n} − g_{1}) − (a_{n} − g_{2})∥
≤ ∥a_{n} − g_{1}∥ + ∥a_{n} − g_{2}∥ .  (A.64)

Applying (A.63) to (A.64) yields

∥−g_{1} + g_{2}∥ = ∥g_{1} − g_{2}∥ < 2ε.

However, the above contradicts the assumption that ∥g_{1} − g_{2}∥ = 2ε, which completes the proof.

□

A sequence in ℝ^{p} is bounded if there exists a number M ≥ 0 such that ∥a_{n}∥ ≤ M for all n = 1, 2, . . .. The set of all bounded sequences is sometimes denoted l_{∞}.

Theorem A.11 Every convergent sequence is bounded.

Proof Let

g = lim_{n→∞} a_{n}.

Choose ε = 1. Then, there exists a natural number k = k(ε) such that

∥a_{n} − g∥ < 1 for all n > k.

By Theorem A.4,

∥a∥ − ∥b∥ ≤ ∥a − b∥.

Using the above inequality, we get

∥a_{n}∥ − ∥g∥ ≤ ∥a_{n} − g∥ < 1 for all n > k.

Therefore,

∥a_{n}∥ < ∥g∥ + 1 for all n > k.

Let

M = max (∥a_{1}∥ , ∥a_{2}∥ , …, ∥a_{k}∥ , ∥g∥ + 1) ,

then we have

M ≥ ∥a_{n}∥ for all n,

which means that a convergent sequence is bounded, and the proof is complete.

□

Suppose we are given a sequence a_{1}, a_{2}, … in ℝ^{p}, and an increasing sequence of positive natural numbers

m_{1}, m_{2}, …, m_{n}, ….

The sequence

a_{m_{1}}, a_{m_{2}}, …, a_{m_{n}}, …

is called a subsequence of the sequence a_{1}, a_{2}, … in ℝ^{p}.

We have

m_{n} ≥ n.

Indeed, this is obvious for n = 1, that is, m_{1} ≥ 1 because m_{1} is a positive integer. We now apply the principle of induction. We assume that m_{n} ≥ n for a given n. Then, we have

m_{n+1} > m_{n} ≥ n.

Hence,

m_{n+1} ≥ n + 1.

Thus, indeed m_{n} ≥ n for all n.

We can say that a subsequence is obtained from the given sequence by neglecting a number of elements in the sequence. We denote a subsequence a_{m_{1}}, a_{m_{2}}, …, a_{m_{n}}, … as

{a_{m_{n}}} .

Theorem A.12 A subsequence of a convergent sequence in ℝ^{p} is convergent to the same limit, that is, if

lim_{n→∞} a_{n} = g,

and if

m_{1} < m_{2} < … < m_{n} < …,

then

lim_{n→∞} a_{m_{n}} = g.

Proof Let ε > 0 be given. Then, there exists a natural number k = k(ε) such that ∥a_{n} − g∥ < ε for any n > k. From the fact that m_{n} ≥ n, it follows that

m_{n} > n > k,

and thus

∥a_{m_{n}} − g∥ < ε

for any m_{n} > n > k, which means that

lim_{n→∞} a_{m_{n}} = g.

The proof is complete.

□

We now state, without a proof, an important result due to Bolzano and Weierstrass.

Theorem A.13 (Bolzano-Weierstrass) Every bounded sequence in ℝ^{p} contains a convergent subsequence.

Theorem A.14 (Cauchy) A sequence {a_{n}} in ℝ^{p} is convergent if, and only if for every ε > 0 there exists an r = r(ε) such that

∥a_{n} − a_{r}∥ < ε

holds for all n > r.

980 APPENDIX A. MATH REVIEW Proof (⇒) Let

lim_{n→∞} a_{n} = g,

and let ε > 0 be given. Then, for some l = l(ε), we have

∥a_{n} − g∥ < \frac{1}{2}ε for all n > l.

Let r = l + 1, then

∥a_{r} − g∥ < \frac{1}{2}ε.

Adding the above inequalities we obtain

∥a_{n} − g∥ + ∥a_{r} − g∥ < ε for all n > r.

On the other hand

∥a_{n} − a_{r}∥ ≤ ∥a_{n} − g∥ + ∥a_{r} − g∥ .

Hence,

∥a_{n} − a_{r}∥ < ε for all n > r.

(⇐) We now assume that for every ε > 0 there exists an r = r(ε) such that

∥a_{n} − a_{r}∥ < ε for all n > r.

We then show that the above condition implies

lim_{n→∞} a_{n} = g.

First, we show that {a_{n}} is a bounded sequence. Let ε = 1. Then, an r = r(ε) exists such that ∥a_{n} − a_{r}∥ < 1 for all n > r. Hence,

∥a_{n}∥ − ∥a_{r}∥ ≤ ∥a_{n} − a_{r}∥ < 1,

which means that

∥a_{n}∥ < ∥a_{r}∥ + 1.

Let

M = max {∥a_{1}∥ , ∥a_{2}∥ , …, ∥a_{r−1}∥ , ∥a_{r}∥ + 1} ,

then we have

∥a_{n}∥ ≤ M for all n,

which implies that the sequence {a_{n}} is bounded. By the Bolzano-Weierstrass theorem the sequence {a_{n}} contains a convergent subsequence. Let

lim_{n→∞} a_{m_{n}} = g, m_{1} < m_{2} < ….

We will show that

lim_{n→∞} a_{n} = g.

From the hypothesis of the theorem it follows that for some r = r (ε/3), we have

∥a_{n} − a_{r}∥ < \frac{1}{3}ε for all n > r.  (A.65)

On the other hand, the assumption that the subsequence {a_{m_{n}}} is convergent implies that for some k = k (ε/3),

∥a_{m_{n}} − g∥ < \frac{1}{3}ε for all n > k.  (A.66)

We can select k so that k > r. In such a case, (A.65) and (A.66) are satisfied for all n > k. Since m_{n} ≥ n > r, we have for all n > k,

∥a_{m_{n}} − a_{r}∥ < \frac{1}{3}ε,

or equivalently,

∥a_{r} − a_{m_{n}}∥ < \frac{1}{3}ε.  (A.67)

Adding (A.65), (A.66), and (A.67) yields

∥a_{n} − a_{r}∥ + ∥a_{m_{n}} − g∥ + ∥a_{r} − a_{m_{n}}∥ < ε.

Hence,

∥a_{n} − g∥ = ∥(a_{n} − a_{r}) + (a_{r} − a_{m_{n}}) + (a_{m_{n}} − g)∥
≤ ∥a_{n} − a_{r}∥ + ∥a_{m_{n}} − g∥ + ∥a_{r} − a_{m_{n}}∥
< ε for all n > k,

which implies that

lim_{n→∞} a_{n} = g.

The proof is complete.

□

Theorem A.15 Every monotone bounded sequence in ℝ is convergent.

Proof We will prove the theorem for nondecreasing sequences. The proof for nonincreasing sequences is analogous.

We assume that we are given a sequence {a_{n}} in ℝ that is bounded and nondecreasing. Let Z be the set of numbers belonging to the sequence, and let g be the least upper bound of Z. Hence

g ≥ a_{n}, n = 1, 2, ….

From the properties of the least upper bound—see Theorem A.7, it follows that for any ε > 0 there exists a k such that

g − ε < a_{k}.

Since the sequence {a_{n}} is nondecreasing, the inequality n > k implies that

a_{k} ≤ a_{n},

and thus

g − ε < a_{n}.

We then conclude that for any ε > 0, there exists a k such that for n > k we have

g − ε < a_{n} < g + ε.

Hence,

|a_{n} − g| < ε,

which means that

lim_{n→∞} a_{n} = g.

Thus, a monotone bounded sequence in ℝ is convergent. The proof is complete.

□

Given a sequence {a_{n}} of real numbers. Then the sequence {s_{N}} defined by

s_{N} = ∑_{n=1}^{N} a_{n} = a_{1} + a_{2} + ⋯ + a_{N}

is called the sequence of partial sums of the series

∑_{n=1}^{∞} a_{n}.  (A.68)

If the sequence of the partial sums has a limit, that is, s_{N} → s as N → ∞, then we say that the series converges to the sum s, and write

s = ∑_{n=1}^{∞} a_{n}.

The above formula can only make sense when the series (A.68) converges.

Theorem A.16 If the series ∑_{n=1}^{∞} a_{n} converges, then

a_{n} → 0 as n → ∞.

Proof Let s be the sum of the series ∑_{n=1}^{∞} a_{n}. Then,

s_{N} = ∑_{n=1}^{N} a_{n} → s as N → ∞.  (A.69)

We also have

s_{N−1} → s as N → ∞.  (A.70)

We have,

a_{N} = (a_{1} + a_{2} + ⋯ + a_{N}) − (a_{1} + a_{2} + ⋯ + a_{N−1})
= s_{N} − s_{N−1}.

Therefore, by (A.69) and (A.70),

a_{N} → (s − s) = 0 as N → ∞,

and the proof is complete.

□

The converse of Theorem A.16 is false. The terms of the series

∑_{n=1}^{∞} \frac{1}{n}

tend to zero, but the series diverges.

A.8 Functions

We discuss functions of one and several variables and their basic properties. A function f from a set A to a set B, denoted

f : A → B,

is a rule that assigns to each x ∈ A a unique element y ∈ B. The set A is the domain of f and the range of f is a subset of B, not necessarily the whole of B.

Let f : S → ℝ, where S is a subset of ℝ. Then, f is said to be continuous at z_{0} ∈ S if for every ε > 0, there exists δ = δ(ε, z_{0}) such that z ∈ S, |z − z_{0}| < δ imply |f(z) − f(z_{0})| < ε.

This is also written as f(z) → f(z_{0}) as z → z_{0}. If f is continuous at every point of S, then we say that f is continuous on S.

We say that the function f is bounded on a set S if there is M ≥ 0 such that for any z ∈ S we have

|f(z)| ≤ M.

Lemma A.2 If f is continuous on [a, b] ⊂ ℝ, then f is bounded on [a, b].

Proof Consider the set

{x : x ∈ [a, b], and f is bounded on [a, x]}.

This set is nonempty and bounded above by b. Let

c = sup{x : f is bounded on [a, x]}.

We will show that c = b. Suppose that c < b. From the continuity of f at c it follows that f is bounded on [c − δ, c + δ] for some sufficiently small δ > 0. Being bounded on [a, c − δ] and on [c − δ, c + δ], it is bounded on [a, c + δ]. This contradicts our choice of c. We therefore conclude that c = b. This means that f is bounded on [a, x] for all x < b. From the continuity of f, we know that f is bounded on some interval [b − δ, b]. Hence, f is bounded on [a, b − δ] and on [b − δ, b], which means that f is bounded on [a, b].

□

With the help of the above lemma, we can now prove the maximum-minimum theorem of Weierstrass.

Theorem A.17 (The Maximum-Minimum Theorem) If f is continuous on an interval [a, b], then it takes on both its supremum and infimum values on [a, b].

Proof By Lemma A.2 above, the function f is bounded on [a, b]. Let

M = sup{f(x) : x ∈ [a, b]}.

We show that there is a c ∈ [a, b] such that f(c) = M. Let

g(x) = \frac{1}{M − f(x)}.

If f does not take on the value of M, then g is continuous on [a, b], and by Lemma A.2 the function g is bounded on [a, b]. But g cannot be bounded on [a, b]. Thus, the assumption that f does not take on the value M has led us to a contradiction. The other case that f takes on its infimum value can be proved in a similar manner.

□

A sphere, ball, around x in ℝ^{p} is a set of the form {y : ∥x − y∥ < ε} for some ε > 0. A set Ω in ℝ^{p} is said to be open if around every point x in Ω there is a sphere that is contained in Ω.

If x ∈ ℝ^{p}, then any set that contains an open set containing x is called a neighborhood of x. A sphere around x is a neighborhood of x.

A point x ∈ ℝ^{p} is called a boundary point of a set Ω ⊆ ℝ^{p} if every neighborhood of x contains a point of Ω and a point that is not in Ω. The set of all boundary points of Ω is called the boundary of Ω. A set is said to be closed if it contains its boundary. Equivalently, a set Ω is closed if x_{k} → g with x_{k} ∈ Ω implies g ∈ Ω. A set is bounded if it is contained within some sphere of finite radius. A set is compact if it is both closed and bounded. Any finite set Ω = {x_{1}, x_{2}, …, x_{k}} in ℝ^{p} is compact.

We now generalize Lemma A.2 for real-valued functions of several variables.

Lemma A.3 Let f : ℝ^{p} → ℝ be a continuous function on a compact set Ω ⊂ ℝ^{p}. Then, f is bounded on Ω.

Proof We prove the lemma by contradiction. We assume that f is continuous on a compact set Ω, but f is not bounded on Ω. Without loss of generality, suppose that f is unbounded above on Ω. Since f is unbounded above on Ω, we can find a sequence {x_{n}} ⊂ Ω such that f(x_{n}) > r(n), where r(n) → ∞ as n → ∞, that is, f(x_{n}) → ∞ as n → ∞. The sequence {x_{n}} must be bounded because, by assumption, the set Ω is compact. Therefore, by the Bolzano-Weierstrass theorem, on page 979, there is a convergent subsequence, say, {x_{n_{k}}} that converges to a point x^{∗} ∈ Ω. The function is continuous on Ω. Hence,

lim_{k→∞} f(x_{n_{k}}) = f ( lim_{k→∞} x_{n_{k}}) = f (x^{∗}) .  (A.71)

We arrived at a contradiction because by assumption, the left-hand side of (A.71) is divergent, while f (x^{∗}) < ∞ because f is continuous on Ω. Therefore, a continuous function on a compact set in ℝ^{p} must be bounded.

□

We next prove the theorem of Weierstrass for real-valued functions of several variables.

Theorem A.18 (Weierstrass) If f : ℝ^{p} → ℝ is a continuous function on a compact set Ω ⊂ ℝ^{p}, then f achieves its supremum and infimum.

Proof The proof follows that of Theorem A.17 with the interval [a, b] being replaced with the compact set Ω, and by referring to Lemma A.3 rather than to Lemma A.2.

□

There is a special type of continuity that we will use later.

Definition A.1 (Uniform continuity) Let f : X → Y. Then, f is called uniformly continuous on X if for every ε > 0 there exists δ = δ(ε) > 0, depending only on ε, such that |x − x′| < δ implies |f(x) − f(x′)| < ε, where x, x′ are in X

◇

Every uniformly continuous function is continuous, but not necessarily conversely. Indeed, f(x) = 1/x is continuous on (0, 1) but not uniformly continuous.

Another type of continuous functions that we will be using later are absolutely continuous functions.

Definition A.2 (Absolutely continuous function) Let f be a function defined on the closed interval [a, b]. Then, the function f is said to be absolutely continuous on [a, b] if for every ε > 0, there exists a δ > 0 such that

∑_{k=1}^{n} (b_{k} − a_{k}) < δ  (A.72)

for all numbers a_{1}, b_{1}, …, a_{n}, b_{n} such that a_{1} < b_{1} ≤ a_{2} < b_{2} ≤ ⋯ ≤ a_{n} < b_{n}, we have

∑_{k=1}^{n} |f(b_{k}) − f(a_{k})| < ε.  (A.73)

◇

Note that every absolutely continuous function is continuous, because the case n = 1 is not excluded in the above definition. A function f that satisfies the Lipschitz condition,

|f(x) − f(y)| ≤ L|x − y|,

is absolutely continuous.

A.9 Linear Operators

A function

F : ℝ^{n} → ℝ^{m}

is said to be linear if it satisfies the following two properties:

LM 1. For any x, y in ℝ^{n}, we have

F (x + y) = F (x) + F (y),

LM 2. If c is a number, then

F (c x) = cF (x).

Linear functions are sometimes referred to as linear maps or linear operators.

Linear functions from ℝ^{n} into ℝ^{m} can be identified with the m × n real matrices by

F : ℝ^{n} → ℝ^{m} is linear if and only if F (x) = Ax,

where A ∈ ℝ^{m×n}.

If F : ℝ^{n} → ℝ^{m} is linear, then there exists a constant L ≥ 0 such that

∥F (x)∥ ≤ L∥x∥  (A.74)

for all x ∈ ℝ^{n}. The smallest such constant L for which this is true is denoted ∥F ∥ and called the norm of the linear map F. If F is the linear map induced by the matrix A, then we define ∥A∥ = ∥F ∥. For any matrix A and vector norm ∥ · ∥, we can compute the matrix norm using the formula

∥A∥ = sup_{x≠0} \frac{∥Ax∥}{∥x∥}  (A.75)

The matrix norm, that is, the norm of the linear map F given by (A.75) is equivalent to

∥A∥ = max_{∥x∥=1} ∥Ax∥  (A.76)

A matrix norm produced using (A.75) or (A.76) is called the induced norm. Note that

∥Ax − Ay∥ = ∥A(x − y)∥ ≤ ∥A∥∥x − y∥,

that is, the linear map, F, represented by the matrix A is uniformly continuous.

A couple of comments on the notation used in (A.75) and (A.76) are needed to clarify the matter. First, the same symbol is used for the matrix norm as for the norm of a vector. Second, the norms appearing on the right side of (A.75) and (A.76) are the norms of the vectors x ∈ ℝ^{n}, and Ax ∈ ℝ^{m}. Thus, if ℝ^{n} is equipped with the vector norm ∥ · ∥_{s} while ℝ^{m} is equipped with the vector norm ∥ · ∥_{t}, then (A.76) should be interpreted as

∥A∥ = max_{∥x∥_{s}=1} ∥Ax∥_{t}.

988 APPENDIX A. MATH REVIEW Vector norms and a matrix norm are said to be compatible, or consistent, if they satisfy,

∥Ax∥_{t} ≤ ∥A∥∥x∥_{s}.  (A.77)

We next show that (A.76) is well defined. For this, we need the following lemma. Lemma A.4 A vector norm ∥ · ∥ : ℝ^{p} → ℝ is an uniformly continuous function. Proof By Theorem A.4, on page 962,

|∥x∥ − ∥y∥| ≤ ∥x − y∥.

Hence, for any ε > 0, if we take δ = ε then for any x, y ∈ ℝ^{p}, if ∥x − y∥ < δ then |∥x − y∥| < ε, as was to be shown.

□

By the continuity of the vector norm and the theorem of Weierstrass, on page 985, a vector x_{0} can be found such that ∥x_{0}∥ = 1 and ∥A∥ = ∥Ax_{0}∥ = max_{∥x∥=1} ∥Ax∥. Thus, the expression (A.76) is well defined.

We show that (A.76) satisfies (A.77). Suppose that we are given a vector y ≠ 0. Then,

x = \frac{1}{∥y∥}y

satisfies the condition ∥x∥ = 1. Hence,

∥Ay∥ = ∥A (∥y∥x) ∥ = ∥y∥∥Ax∥.

Applying now (A.76) to the right-hand side of the above yields

∥Ay∥ ≤ ∥y∥∥A∥.

Hence, any matrix norm produced by (A.76) is consistent.

Next, we prove that a matrix norm obtained using (A.76) satisfies the following conditions:

MN 1. ∥A∥ > 0 if A ≠ O and ∥O∥ = 0. MN 2. ∥cA∥ = |c|∥A∥, where c is a constant. MN 3. ∥A + B∥ ≤ ∥A∥ + ∥B∥. MN 4. ∥AB∥ ≤ ∥A∥∥B∥.

Proof of MN 1. Let A ≠ O. Then a vector x, ∥x∥ = 1, can be found such that Ax ≠ 0, and thus ∥Ax∥ ≠ 0. Hence, ∥A∥ = max_{∥x∥=1} ∥Ax∥ ≠ 0. If, on the other hand, A = O, then ∥A∥ = max_{∥x∥=1} ∥Ox∥ = 0.

Proof of MN 2. By definition, ∥cA∥ = max_{∥x∥=1} ∥cAx∥. Using a property of the vector norm, we write ∥cAx∥ = |c|∥Ax∥, and hence

∥cA∥ = max_{∥x∥=1} |c|∥Ax∥ = |c| max_{∥x∥=1} ∥Ax∥ = |c|∥A∥.

Proof of MN 3. For the sum of two matrices A+B, there exists a vector x_{0}, ∥x_{0}∥ = 1, such that

∥A + B∥ = ∥(A + B)x_{0}∥.

Then, using properties of the vector norm, we obtain

∥A + B∥ = ∥Ax_{0} + Bx_{0}∥ ≤ ∥Ax_{0}∥ + ∥Bx_{0}∥.

Using (A.76) yields

∥A + B∥ ≤ ∥A∥∥x_{0}∥ + ∥B∥∥x_{0}∥ = ∥A∥ + ∥B∥.

Proof of MN 4. For the product of two matrices AB, there exists a vector x_{0}, ∥x_{0}∥ = 1, such that ∥AB∥ = ∥ABx_{0}∥. Then, using (A.76), we have

∥AB∥ = ∥A(Bx_{0})∥ ≤ ∥A∥∥Bx_{0}∥.

Using (A.76) again yields

∥AB∥ ≤ ∥A∥∥B∥∥x_{0}∥ = ∥A∥∥B∥.
□

The most frequently used matrix norms are the p-norms

∥A∥_{p} = max_{∥x∥_{p}=1} ∥Ax∥_{p}.

The induced matrix norms corresponding to the one-, two-, and infinity-norm for vectors are:

• ∥A∥_{1} = max_{j} ∥a_{j}∥_{1}, that is, ∥A∥_{1} is the maximum of absolute column sum;

• ∥A∥_{2} = σ_{1}(A), which is the largest singular value of A;

• ∥A∥_{∞} = max_{i} ∥a_{i}^{⊤}∥_{1}, which is the maximum of absolute row sum.

A matrix norm that is not induced by a vector norm is the Frobenius norm, defined for an m × n matrix A as

∥A∥_{F} = (∑_{i=1}^{m} ∑_{j=1}^{n} a_{ij}^{2})^{1/2} .  (A.78)

The Frobenius norm satisfies the relation

∥A∥_{F}^{2} = trace (AA^{⊤}) = trace (A^{⊤}A) .  (A.79)

The Frobenius norm is compatible with the Euclidean vector norm, that is,

∥Ax∥_{2} ≤ ∥A∥_{F} ∥x∥_{2}.  (A.80)

A.10 Vector Spaces

A given set of vectors of the same dimension {x_{1}, x_{2}, …, x_{r}} is linearly dependent if the zero vector, 0, can be written as a non-trivial linear combination of the given vectors,

α_{1}x_{1} + α_{2}x_{2} + ⋯ + α_{r}x_{r} = 0, with α_{i} ≠ 0 for at least one i.

In other words, the set of vectors {x_{1}, x_{2}, …, x_{r}} is linearly independent if

α_{1}x_{1} + α_{2}x_{2} + ⋯ + α_{r}x_{r} = 0, implies α_{1} = α_{2} = ⋯ = α_{r} = 0.

A set S containing vectors of dimension p is a subspace of ℝ^{p} if for any scalars α and β,

x, y ∈ S implies αx + βy ∈ S.  (A.81)

It follows from the above that a subspace must contain the zero vector, 0. The set containing only the zero vector is a subspace. This set, {0}, is called a trivial subspace.

A matrix-vector product

Ax = [ a_{1} a_{2} ⋯ a_{n} ] [ x_{1} ; x_{2} ; ⋮ ; x_{n} ] = x_{1}a_{1} + x_{2}a_{2} + ⋯ + x_{n}a_{n}  (A.82)

is a linear combination of the columns of the matrix. Thus, linear dependence of the columns of A is equivalent to the condition

Az = 0 for some vector z ≠ 0.  (A.83)

Linear independence of the columns of A is equivalent to the condition

Az = 0 implies z = 0.  (A.84)

The set of all vectors that are linear combinations of the columns of A is called the range, or image, of A and will be denoted range(A). Thus,

range(A) = {v : v = Ax for some x}.  (A.85)

For a given m × n matrix A and an m-vector b, we have b ∈ range(A) if and only if there exists an n-vector x such that Ax = b. We say that a system of linear equations represented by Ax = b is compatible if b ∈ range(A).

The range of A is a subspace of ℝ^{m}. Indeed, for any b_{1}, b_{2} ∈ range(A), we have b_{1} = Ax_{1} and b_{2} = Ax_{2} for some x_{1} and x_{2}. Using the fact that A represents a linear map, for any scalars α_{1} and α_{2},

α_{1}b_{1} + α_{2}b_{2} = α_{1}Ax_{1} + α_{2}Ax_{2} = A (α_{1}x_{1} + α_{2}x_{2}) .  (A.86)

Hence, if b_{1}, b_{2} ∈ range(A), then α_{1}b_{1} + α_{2}b_{2} ∈ range(A), which means that the range of A is a subspace of ℝ^{m}.

Similarly, we define the range of A^{⊤} to be the set

range(A^{⊤}) = {y : y = A^{⊤}z for some z} .  (A.87)

For a non-trivial subspace S there is a unique smallest positive integer called the dimension of the subspace S and denoted dim S. The dimension of the subspace is equal to the smallest number of linearly independent vectors of the subspace such that every vector of the subspace can be expressed as a linear combination of these vectors. Any such set of vectors is called a basis of the subspace. The vectors forming a basis are not unique.

The number of linearly independent columns of a matrix A is called the rank of A and denoted rank(A). The rank of A is also equal to the number of linearly independent rows of A.

For the product of a m×n matrix A and a n×p matrix B, we have Sylvester’s inequalities:

rank(A) + rank(B) − n ≤ rank(AB) ≤ min{rank(A), rank(B)}.  (A.88)

For an m × n matrix A, the set of all n-vectors orthogonal to the rows of A is called the kernel, or null space, of A and denoted ker(A), that is,

ker(A) = {x : Ax = 0} .  (A.89)

The kernel of A is a subspace of ℝ^{n}. Indeed, if x_{1}, x_{2} ∈ ker(A), then Ax_{1} = 0 and Ax_{2} = 0. Hence, for any scalars α_{1} and α_{2}, we have

A (α_{1}x_{1} + α_{2}x_{2}) = α_{1}Ax_{1} + α_{2}Ax_{2} = 0.  (A.90)

Thus, if x_{1}, x_{2} ∈ ker(A), then α_{1}x_{1} +α_{2}x_{2} ∈ ker(A), which means that ker(A) is a subspace of ℝ^{n}. Similarly, we define the kernel of A^{⊤} to be the set

ker (A^{⊤}) = {w : A^{⊤}w = 0} .  (A.91)

The intersection of range(A) and ker (A^{⊤}) contains only the zero vector, that is,

range(A) ∩ ker (A^{⊤}) = {0}.  (A.92)

For a given subspace S of ℝ^{n}, the orthogonal complement S^{⊥} of S is defined by

S^{⊥} = {x ∈ ℝ^{n} : x^{⊤}y = 0 for all y ∈ S} .  (A.93)

The vectors in ker (A^{⊤}) are orthogonal to the vectors in range(A), and vice versa. We say that range(A) and ker (A^{⊤}) are orthogonal complements of one another and write

range(A)^{⊥} = ker (A^{⊤}) ,  (A.94)

and

ker (A^{⊤})^{⊥} = range(A).  (A.95)

Similarly,

range (A^{⊤})^{⊥} = ker (A) ,  (A.96)

and

ker (A)^{⊥} = range (A^{⊤}) .  (A.97)

If A is an m × n matrix with linearly independent columns, which implies that m ≥ n, then

rank(A) = n = dim range(A),

and

dim ker (A^{⊤}) = m − n.

We now define a normed space.

Definition A.3 A vector space with a norm is called a normed space.

◇

By the normed vector space ℝ^{n}, we usually mean the Euclidean space, which is ℝ^{n} with the norm ∥x∥_{2} = \sqrt{x_{1}^{2} + ⋯ + x_{n}^{2}}. The vector space ℝ^{n} equipped with any of the p-norm forms a normed vector space. The absolute value is a norm in ℝ.

Let Ω be a compact subset of ℝ^{n} and let C(Ω) denote the set of continuous functions f : ℝ^{n} → ℝ on Ω. Thus, we view functions as vectors in the function space C(Ω). Then, the function ∥ · ∥ : C(Ω) → ℝ defined as

∥f∥ = max_{x∈Ω} |f(x)|,  (A.98)

satisfies the norm axioms. The function space C(Ω), with scalars from ℝ and with the norm (A.98) is a normed linear space. In particular, the space of all real-valued functions, f(x), continuous on some interval [a, b] of the real line with the norm defined by

∥f∥ = max_{x∈[a,b]} |f(x)|,  (A.99)

is an example of a normed linear space. This space is denoted C([a, b]).

We will also be interested in sequences of elements of C([a, b]). Let f_{1}, f_{2}, … be a sequence of elements of C([a, b]). Denote this sequence by {f_{k}}. In the following, f_{k}(x), means the value of the function f_{k} at x, while f_{k} means the function itself. We say that the sequence {f_{k}} converges (pointwise) on the interval [a, b] if there exists a function f such that for every x ∈ [a, b], the sequence {f_{k}(x)} converges to f(x). In other words, the sequence {f_{k}} converges (pointwise) if there exists a function f such that for a given x ∈ [a, b] and a given ε > 0, we can find N = N(x, ε) such that

|f(x) − f_{k}(x)| < ε for k > N.  (A.100)

The sequence {f_{k}} converges uniformly, on the interval [a, b], if there is a function f such that for any ε > 0 there exists N = N(ε), depending on ε but not on x, such that (A.100) holds for every x ∈ [a, b]. The distinction between convergence and uniform convergence is that in the later case, the same N can be used for any value of x to show the convergence of the sequence {f_{k}(x)}. One can show that the limit of an uniformly convergent sequence of continuous functions is continuous. It can be verified that the norm (A.99) defines the uniform convergence because {f_{k}} converges uniformly to f if and only if max_{x∈[a,b]} |f(x) − f_{k}(x)| → 0. This is the reason the norm (A.99) is sometimes called the norm of uniform convergence. It can be shown that there is no norm on C([a, b]) that defines pointwise convergence to an element in C([a, b])—see Debnath and Mikusiński [74, p. 12].

Definition A.4 (Banach Space) A normed linear space is complete, if every Cauchy sequence with elements form the space converges to an element in the space. A complete normed linear space is called a Banach space.

◇

The Euclidean space, as well as C([a, b]) with the norm (A.99), are examples of Banach spaces.

A.11 Least Squares

Consider the system of equations

Ax = b,  (A.101)

where A ∈ ℝ^{m×n}, x ∈ ℝ^{n}, and b ∈ ℝ^{m}. A necessary and sufficient condition for the system (A.101) to have a solution is

rank(A) = rank [ A ⋮ b ] .  (A.102)

In particular, if the matrix A is invertible, then there is a unique solution to (A.101) given by

x^{∗} = A^{−1}b.

Here, we analyze the case when m > n and rank(A) < rank [ A ⋮ b ], that is, b ∉ range(A), which means that the system (A.101) does not have an exact solution. We modify (A.101) by incorporating an error vector e to obtain

Ax + e = b.  (A.103)

This is illustrated in Figure A.3. Because

e = b − Ax,  (A.104)

we have

∥e∥^{2} = e^{⊤}e
= (b − Ax)^{⊤} (b − Ax)
= x^{⊤}A^{⊤}Ax − 2x^{⊤}A^{⊤}b + b^{⊤}b.  (A.105)

The gradient of ∥e∥^{2} with respect to x is

∇x∥e∥^{2} = 2A^{⊤}Ax − 2A^{⊤}b.  (A.106)

Figure A.3: Geometrical interpretation of the least squares solution to a system of linear equations.

Solving

∇x∥e∥^{2} = 0

yields candidate minimizer points of (A.105). If the matrix A has full column rank, then the matrix A^{⊤}A is invertible and the only critical point is

x^{∗} = (A^{⊤}A)^{−1} A^{⊤}b  (A.107)

The above point satisfies the second order sufficient condition to be the minimizer of (A.105) because the Hessian of (A.105) is

2A^{⊤}A,

which is positive definite. The solution given by (A.107) is called the least squares solution of the inconsistent linear system of equations, Ax = b, because it minimizes the sum of squares ∥e∥^{2} = e^{⊤}e = ∑_{i=1}^{m} e_{i}^{2}. For a graphical interpretation of the least squares solution, we refer to Figure A.3.

Suppose now that we interpret the i-th row of the matrix A and the i-th component of the vector b as the i-th data pair. Thus, the i-th data pair has the form (a_{i}^{⊤}; b_{i}). Suppose that the k data pairs are represented by the pair (A; b). We know that the least squares solution to the system of equations represented by the data pair (A; b) is given by (A.107). If the new data pair, (a_{k+1}^{⊤}; b_{k+1}), becomes available, then we could find the least squares solution to the system of equations

[ \begin{matrix}A \\ a & {⊤ \atop k+1} \end{matrix} ] x = [ \begin{matrix}b \\ b_{k+1} \end{matrix} ]  (A.108)

by applying the formula (A.107) to (A.108). Let x_{k+1} denote the least squares solution to the above problem. Then, we have

x_{k+1} = ([ \begin{matrix}A \\ a & {⊤ \atop k+1} \end{matrix} ]^{⊤} [ \begin{matrix}A \\ a & {⊤ \atop k+1} \end{matrix} ])^{−1} [ \begin{matrix}A \\ a & {⊤ \atop k+1} \end{matrix} ]^{⊤} [ \begin{matrix}b \\ b_{k+1} \end{matrix} ] .  (A.109)

In the case of large k the above method of recomputing the new least squares solution from scratch may be time consuming. We present a recursive method of computing least squares solutions as the data pairs become available. To proceed, let

P_{k} = (A^{⊤}A)^{−1} .  (A.110)

In view of the above, we can write

P _{k+1}^{−1} = [ \begin{matrix}A \\ a & {⊤ \atop k+1} \end{matrix} ]^{⊤} [ \begin{matrix}A \\ a & {⊤ \atop k+1} \end{matrix} ]
= [ A^{⊤} a_{k+1} ] [ \begin{matrix}A \\ a & {⊤ \atop k+1} \end{matrix} ]
= A^{⊤}A + a_{k+1}a_{k+1}^{⊤}
= P _{k}^{−1} + a_{k+1}a_{k+1}^{⊤}.  (A.111)

Hence,

P_{k+1} = (P _{k}^{−1} + a_{k+1}a_{k+1}^{⊤})^{−1} .

Applying to the above the matrix inversion formula (A.30), we obtain

P_{k+1} = P_{k} − \frac{P_{k}a_{k+1}a_{k+1}^{⊤}P_{k}}{1 + a_{k+1}^{⊤}P_{k}a_{k+1}}  (A.112)

Using (A.110), we write

x_{k} = (A^{⊤}A)^{−1} A^{⊤}b
= P_{k}A^{⊤}b.  (A.113)

Using (A.111), we can write

x_{k+1} = ([ \begin{matrix}A \\ a & {⊤ \atop k+1} \end{matrix} ]^{⊤} [ \begin{matrix}A \\ a & {⊤ \atop k+1} \end{matrix} ])^{−1} [ \begin{matrix}A \\ a & {⊤ \atop k+1} \end{matrix} ]^{⊤} [ \begin{matrix}b \\ b_{k+1} \end{matrix} ]
= P_{k+1} (A^{⊤}b + a_{k+1}b_{k+1}) .  (A.114)

Computing A^{⊤}b from (A.113), we get

A^{⊤}b = P _{k}^{−1}x_{k}.  (A.115)

Substituting the above into (A.114) yields

x_{k+1} = P_{k+1} (P _{k}^{−1}x_{k} + a_{k+1}b_{k+1}) .  (A.116)

Computing P _{k}^{−1} from (A.111) gives

P _{k}^{−1} = P _{k+1}^{−1} − a_{k+1}a_{k+1}^{⊤}

Substituting the above into (A.116) we have

x_{k+1} = P_{k+1} ((P _{k+1}^{−1} − a_{k+1}a_{k+1}^{⊤}) x_{k} + a_{k+1}b_{k+1}) .  (A.117)

Taking into account that a_{k+1}^{⊤}x_{k} is a scalar while performing simple manipulations, we obtain

x_{k+1} = x_{k} + (b_{k+1} − a_{k+1}^{⊤}x_{k}) P_{k+1}a_{k+1}  (A.118)

The two equations (A.112) and (A.118) constitute the recursive least squares (RLS) algorithm. To initialize the RLS, we can set

P_{0} = (A_{n}^{⊤}A_{n})^{−1} and x_{0} = P_{n}A_{n}^{⊤}b_{n},

where the pair (A_{n}; b_{n}) represents the first n data pairs.

A.12 Contraction Maps

An operator F that maps Euclidean space ℝ^{p} into itself is called a contraction map if for any two elements x and y of ℝ^{p},

∥F (x) − F (y)∥ ≤ L∥x − y∥, 0 < L < 1.  (A.119)

A point x^{∗} ∈ ℝ^{p} is called a fixed point if

F (x^{∗}) = x^{∗}.  (A.120)

The importance of the contraction operators lies in the fact that there is a unique fixed point. Furthermore, this point may be found by successive iterations as stated in the following theorem.

Theorem A.19 Let F : ℝ^{p} → ℝ^{p} be a contraction map with the contraction constant 0 < L < 1. Then F has a unique fixed point, and the sequence

x^{(1)}, x^{(2)}, …

generated by x^{(n+1)} = F (x^{(n)}), n = 0, 1, 2, …, converges to x^{∗} for any initial choice of x^{(0)} ∈ ℝ^{p}.

Proof To show the uniqueness of the fixed point x^{∗}, suppose that there exist two distinct fixed points x^{∗} and y^{∗}. We then have

∥x^{∗} − y^{∗}∥ = ∥F (x^{∗}) − F (y^{∗})∥
≤ L ∥x^{∗} − y^{∗}∥ .

Thus,

∥x^{∗} − y^{∗}∥ ≤ L ∥x^{∗} − y^{∗}∥ , 0 < L < 1,

which is impossible unless ∥x^{∗} − y^{∗}∥ = 0. Hence, x^{∗} = y^{∗} which means that F has at most one fixed point.

We next show that a fixed point exists, and that the sequence

x^{(1)}, x^{(2)}, …

generated by x^{(n+1)} = F (x^{(n)}), n = 0, 1, 2, …, converges to x^{∗} for any initial choice of x^{(0)} ∈ ℝ^{p}. If F (x^{(0)}) = x^{(0)}, then the theorem is proved. In general F (x^{(0)}) ≠ x^{(0)}. For any n, m with m > n, we have

‖x^{(m)} − x^{(n)}‖ = ‖F^{m} (x^{(0)}) − F^{n} (x^{(0)})‖
= ‖F (F^{m−1} (x^{(0)})) − F (F^{n−1} (x^{(0)}))‖
≤ L ‖F^{m−1} (x^{(0)}) − F^{n−1} (x^{(0)})‖
⋮
≤ L^{n} ‖x^{(m−n)} − x^{(0)}‖
= L^{n} ‖x^{(0)} − x^{(m−n)}‖ .  (A.121)

We write

‖x^{(0)} − x^{(m−n)}‖ = ‖x^{(0)} − x^{(1)} + x^{(1)} − x^{(2)} + x^{(2)}−
⋯ − x^{(m−n−1)} + x^{(m−n−1)} − x^{(m−n)}‖ .  (A.122)

Substituting (A.122) into (A.121) and then applying the triangle inequality yields

‖x^{(m)} − x^{(n)}‖ ≤ L^{n} (‖x^{(0)} − x^{(1)}‖ + ‖x^{(1)} − x^{(2)}‖ + ⋯ + ‖x^{(m−n−1)} − x^{(m−n)}‖) .

(A.123)

Observe that

‖x^{(j)} − x^{(j+1)}‖ = ‖F (x^{(j−1)}) − F (x^{(j)})‖
≤ L ‖x^{(j−1)} − x^{(j)}‖
⋮
≤ L^{j} ‖x^{(0)} − x^{(1)}‖ ,  (A.124)

We use the above to further evaluate the right hand side of (A.123) to obtain

‖x^{(m)} − x^{(n)}‖ ≤ L^{n} ‖x^{(0)} − x^{(1)}‖ (1 + L + ⋯ + L^{m−n−1})
≤ L^{n} ‖x^{(0)} − x^{(1)}‖ (1 + L + ⋯ + L^{m−n−1} + L^{m−n} + ⋯)
= \frac{L^{n}}{1 − L} ‖x^{(0)} − x^{(1)}‖

because 0 < L < 1. Recall that a sequence x^{(0)}, x^{(1)}, … is a Cauchy sequence if for each ε > 0 there exists a positive integer n = n(ε) such that ‖x^{(m)} − x^{(n)}‖ < ε for all m > n. Given an ε > 0, let n be such that

L^{n} < \frac{ε(1 − L)}{∥x^{(0)} − x^{(1)}∥}.

Then,

‖x^{(m)} − x^{(n)}‖ ≤ \frac{L^{n}}{1 − L} ‖x^{(0)} − x^{(1)}‖ < ε, for all m > n.  (A.125)

Therefore, the sequence x^{(0)}, x^{(1)}, … is a Cauchy sequence, and we show that the limit g of this sequence is the fixed point x^{∗} of the map F. Note that F is uniformly continuous. Indeed, for each ε > 0 there exists a δ = δ(ε) such that if ∥x−y∥ < δ, then ∥F (x) − F (y)∥ < ε. Since ∥F (x) − F (y)∥ ≤ L∥x − y∥, given an ε > 0, we can choose δ = ε/L. Because F is continuous,

lim_{n→∞} F (x^{(n)}) = F ( lim_{n→∞} x^{(n)}) = F (g).

However, x^{(n+1)} = F (x^{(n)}). Therefore,

g = lim_{n→∞} x^{(n+1)}
= lim_{n→∞} F (x^{(n)})
= F ( lim_{n→∞} x^{(n)})
= F (g).

Hence, g = x^{∗} is a fixed point of F, and the proof is complete.

□

Using the above proof we can give an estimate of the norm of ‖x^{(n)} − x^{∗}‖. Indeed, if we fix n in (A.125) and allow m → ∞, then

‖x^{(n)} − x^{∗}‖ ≤ \frac{L^{n}}{1 − L} ‖x^{(0)} − F (x^{(0)})‖ .  (A.126)

A.13 First-Order Differential Equation

Let ℝ denote the set of real numbers and let D ⊆ ℝ^{2} be a domain, that is, an open connected nonempty subset of ℝ^{2}. Let f be a real valued function defined and continuous on D. Let ẋ = dx/dt denote the derivative of x with respect to t. We call

ẋ = f(t, x)  (A.127)

an ordinary differential equation of the first order. By a solution of the differential equation (A.127) on an open interval I = {t ∈ ℝ : t_{1} < t < t_{2}}, we mean a real valued continuously differentiable function ϕ defined on I such that (t, ϕ(t)) ∈ D for all t ∈ I and

ϕ̇(t) = f(t, ϕ(t)) for all t ∈ I.

Definition A.5 Given (t_{0}, x_{0}) ∈ D, the initial value problem for (A.127) is

ẋ = f(t, x), x(t_{0}) = x_{0}.  (A.128)
◇

A function ϕ is a solution to (A.128) if ϕ is a solution of the differential equation (A.127) on some interval I containing t_{0} and x(t_{0}) = x_{0}.

We can represent the initial value problem (A.128) equivalently by an integral equation,

ϕ(t) = x_{0} + ∫_{t_{0}}^{t} f(s, ϕ(s))ds.  (A.129)

To prove this equivalence, let ϕ be a solution of the initial value problem (A.128). Then x(t_{0}) = x_{0} and

ϕ̇(t) = f(t, ϕ(t)) for all t ∈ I.

Integrating the above from t_{0} to t, we obtain

∫_{t_{0}}^{t} ϕ̇(s)ds = ∫_{t_{0}}^{t} f(s, ϕ(s))ds.

Hence,

ϕ(t) − x_{0} = ∫_{t_{0}}^{t} f(s, ϕ(s))ds.

Therefore, ϕ is a solution of the integral equation (A.129).

Conversely, let ϕ be a solution of the integral equation (A.129). Then ϕ(t_{0}) = x_{0}, and differentiating both sides of (A.129) with respect to t gives ϕ̇(t) = f(t, ϕ(t)). Therefore, ϕ is also a solution of the initial value problem (A.128).

The general ordinary differential linear equation of the first order has the form

ẋ(t) = a(t)x(t) + b(t),  t ∈ (t_{1}, t_{2}),  (A.130)

where a(t) and b(t) are assumed to be continuous on the interval (t_{1}, t_{2}). We associate with equation (A.130) the initial condition

x(t_{0}) = x_{0}, x_{0} ∈ ℝ.

To find the solution to (A.130), we write it in the form

ẋ(t) − a(t)x(t) = b(t).  (A.131)

We then multiply (A.131) by the “integrating factor”

e^{− ∫_{t_{0}}^{t} a(s)ds}  (A.132)

to get

ẋ(t)e^{− ∫_{t_{0}}^{t} a(s)ds} − a(t)x(t)e^{− ∫_{t_{0}}^{t} a(s)ds} = e^{− ∫_{t_{0}}^{t} a(s)ds}b(t).  (A.133)

The above equation can be represented in an equivalent form as

\frac{d}{dt} \{x(t)e^{− ∫_{t_{0}}^{t} a(s)ds}\} = e^{− ∫_{t_{0}}^{t} a(s)ds}b(t).  (A.134)

Integrating both sides of (A.134) yields

x(t)e^{− ∫_{t_{0}}^{t} a(s)ds} − x_{0} = ∫_{t_{0}}^{t} e^{− ∫_{t_{0}}^{τ} a(s)ds}b(τ)dτ.  (A.135)

Hence,

x(t) = e^{∫_{t_{0}}^{t} a(s)ds} [x_{0} + ∫_{t_{0}}^{t} e^{− ∫_{t_{0}}^{τ} a(s)ds}b(τ)dτ] ,  (A.136)

or in an equivalent form

x(t) = e^{∫_{t_{0}}^{t} a(s)ds}x_{0} + ∫_{t_{0}}^{t} e^{∫_{τ}^{t} a(s)ds}b(τ)dτ  (A.137)

A.14 Integral and Differential Inequalities

A.14.1 The Bellman-Gronwall Lemma

Lemma A.5 Let v(t) and w(t) be continuous real functions of t. Let w(t) ≥ 0, and c be a real constant. If

v(t) ≤ c + ∫_{0}^{t} w(s)v(s)ds,  (A.138)

then

v(t) ≤ ce^{∫_{0}^{t} w(s)ds}.  (A.139)

Proof Denote the right-hand side of (A.138) by x(t), that is, let

x(t) = c + ∫_{0}^{t} w(s)v(s)ds,  (A.140)

and let

z(t) = x(t) − v(t).  (A.141)

Note that z(t) ≥ 0, and x(t) is differentiable. Differentiating both sides of (A.140) and using (A.141) yields

ẋ(t) = w(t)v(t)
= w(t)x(t) − w(t)z(t).  (A.142)

We use (A.137) to represent (A.142) in the equivalent integral form as

x(t) = e^{∫_{0}^{t} w(s)ds}x_{0} − ∫_{0}^{t} e^{∫_{τ}^{t} w(s)ds}w(τ)z(τ)dτ.  (A.143)

Since z(t) ≥ 0 and w(t) ≥ 0, we have

x(t) ≤ e^{∫_{0}^{t} w(s)ds}x_{0}.  (A.144)

From the definition of x(t) given by (A.140), we have x(0) = c. Since

v(t) = x(t) − z(t) ≤ x(t),  (A.145)

because z(t) ≥ 0, hence,

v(t) ≤ ce^{∫_{0}^{t} w(s)ds},

and the proof is completed.

□

The generalized Bellman-Gronwall lemma allows us to convert the investigation of integral equations into integral inequalities.

A.14.2 A Comparison Theorem

We state and prove a theorem that allows one to reduce the study of some problems involving differential equations to the study of corresponding problems using differential inequalities instead.

We consider the differential inequality,

ẋ(t) ≤ ω(t, x(t)), x(t_{0}) = x_{0},

where ω : ℝ^{2} → ℝ is a continuous function in a domain Δ ⊂ ℝ^{2}. We assume that in Δ, the function ω satisfies the Lipschitz condition,

|ω(t, x) − ω(t, y)| ≤ L|x − y|,

where L ≥ 0 is a Lipschitz constant. Consider an associated differential equation

˙x̃(t) = ω(t, x̃(t)),  x̃(t_{0}) = x̃_{0}.

We refer to the above equation as the equation of comparison. Denote the solution to the equation of comparison by

x̃(t) = x̃(t; t_{0}, x̃_{0}).

The following theorem, that can be found, for example, in Corduneanu [67, page 49], relates the solutions of the differential inequality and the associated differential equation.

Theorem A.20 If x(t) is a differentiable solution to the differential inequality and x̃(t) is the solution to the associated differential equation with the corresponding initial conditions such that

x_{0} ≤ x̃_{0},

then

x(t) ≤ x̃(t)

for all t ≥ t_{0}.

Proof We prove the theorem by contradiction. We suppose that x_{0} ≤ x̃_{0} but x(t) ≤ x̃(t) does not hold. Thus, there exists t_{1} > t_{0} such that

x(t_{1}) > x̃(t_{1}).

Let

y(t) = x(t) − x̃(t).

Note that y(t_{0}) = x(t_{0}) − x̃(t_{0}) ≤ 0, and y(t_{1}) = x(t_{1}) − x̃(t_{1}) > 0. Denote by τ the largest t ∈ [t_{0}, t_{1}] such that y(τ) = 0. Such a value of t exists by virtue of the continuity of y. Thus,

y(t) > 0 for all t ∈ (τ, t_{1}] .

On the other hand, for t ∈ [τ, t_{1}], we have

ẏ(t) = ẋ(t) − ˙ x̃(t)
≤ ω(t, x(t)) − ω(t, x̃(t))
≤ |ω(t, x(t)) − ω(t, x̃(t))|
≤ L|x(t) − x̃(t)|.

In the interval [τ, t_{1}], we have

y(t) = x(t) − x̃(t) ≥ 0.

Hence,

ẏ(t) ≤ L|x(t) − x̃(t)|
= L(x(t) − x̃(t))
= Ly(t).

Therefore, in the interval [τ, t_{1}], the function y satisfies

y(t) ≤ y(τ)e^{L(t−τ)} = 0

since y(τ) = 0. But this is a contradiction to the previous conclusion that y(t) > 0 for all t ∈ (τ, t_{1}], which completes the proof.

□

A.15 The Laplace Transform

The Laplace transform is an operator that transforms a function of time, f(t), into a new function of complex variable, F (s), where s = σ + jω, as illustrated in Figure A.4. The operator L denotes that the time function f(t) has been transformed to its Laplace transform, denoted F (s). The Laplace transform is very useful in solving linear differential equations

[figure: f(t); L; F (s) = L(f(t))]

Figure A.4: Schematic representation of the Laplace transform operator.

and hence in analyzing control systems.

To obtain the Laplace transform of the given function of time, f(t),

1. multiply f(t) by a converging factor e^{−st}. This is a factor that decreases to zero as time increases to infinity;

2. Integrate f(t)e^{−st} with respect to time between the limits 0^{−} and ∞ to obtain the Laplace transform of f(t),

F (s) = L(f(t)) = ∫_{0^{−}}^{∞} f(t)e^{−st}dt

The lower limit of integration is 0^{−}, rather than 0, to account for the effect of “instantaneous energy transfer,” for example, in electric circuits that contain inductors or capacitors.

The above definition of the Laplace transform is also referred to as the one-sided or unilateral Laplace transform. In the two-sided, or bilateral, Laplace transform, the lower limit is −∞. For our purposes the one-sided Laplace transform is sufficient.

If we wish to reverse the operation and take the inverse transform, back to the time domain, we write

L^{−1}(F (s)) = f(t).

[figure: F (s); L^{−1}; f(t) = L^{−1}(F (s))]

Figure A.5: Schematic representation of the inverse Laplace transform operation.

Taking the inverse Laplace transform is illustrated in Figure A.5.

Because we are using the one-sided Laplace transform, we define all functions, whose Laplace transforms we compute, to be zero for t < 0^{−}. To proceed, we recall the definition of the unit step function denoted 1(t),

1(t) = \{ \begin{matrix}1 & if & t ≥ 0 \\ 0 & if & t < 0. \end{matrix}

The unit step function is also called the Heaviside function.

Example A.1 Find the Laplace transform of

f(t) = e^{−at}1(t),

where a is a real constant. A graph of f(t) for a = 3 is shown in Figure A.6. We have

L(e^{−at}1(t)) = ∫_{0^{−}}^{∞} e^{−at}e^{−st}1(t)dt = ∫_{0^{−}}^{∞} e^{−(a+s)t}dt.

The above integral exists if

ℜ(a + s) > 0.

The region of the s-plane for which the Laplace transform exists is called the Region of Convergence and abbreviated ROC. Since s = σ + jω, the ROC, in our example, is the region in the s-plane where

0 < ℜ(a + s) = ℜ(a + σ + jω) = a + σ,

Figure A.6: A plot of e^{−3t}1(t).

that is, the region where σ > −a. Proceeding with the integration, we obtain

L(e^{−at}1(t)) = (−\frac{1}{a + s}e^{−(a+s)t})|_{0^{−}}^{∞}
= (−\frac{1}{a + s}e^{−(a+s)∞}) − (−\frac{1}{a + s}e^{−(a+s)0^{−}}) .

For s in the ROC, the first term tends to zero. Hence,

L(e^{−at}1(t)) = \frac{1}{s + a}

Using the Laplace transform of the exponential function, we can easily find the Laplace transform of the unit step. Indeed, if a = 0, then f(t) = e^{−at}1(t) = 1(t). Hence,

L(1(t)) = \frac{1}{s}  (A.146)

A.15.1 Linearity Property of the Laplace Transform

The Laplace transform of the sum, or difference, of two functions of time is equal to the sum, or difference, of the transforms of each function, that is,

L (f_{1}(t) ± f_{2}(t)) = L (f_{1}(t)) ± L (f_{2}(t))  (A.147)

Indeed,

L (f_{1}(t) ± f_{2}(t)) = ∫_{0^{−}}^{∞} (f_{1}(t) ± f_{2}(t)) e^{−st}dt
= ∫_{0^{−}}^{∞} f_{1}(t)e^{−st}dt ± ∫_{0^{−}}^{∞} f_{2}(t)e^{−st}dt
= L (f_{1}(t)) ± L (f_{2}(t)) .

The Laplace transform of the product of a real or complex constant K and a time function f(t) is equal to the product of the constant and the transform of the time function, that is,

L (Kf(t)) = KL (f(t)) .  (A.148)

Indeed,

L (Kf(t)) = ∫_{0^{−}}^{∞} Kf(t)e^{−st}dt = K ∫_{0^{−}}^{∞} f(t)e^{−st}dt = KL (f(t)) .

The above two properties can be represented in the form

L (Kf_{1}(t) ± f_{2}(t)) = KL (f_{1}(t)) ± L (f_{2}(t))

Thus the Laplace transform is a linear operator.

Example A.2 To find the Laplace transform of

f(t) = K1(t),

where K is a constant, we can use (A.148) and the Laplace transform of the unit step, given by (A.146), to obtain

L (K1(t)) = KL (1(t)) = \frac{K}{s} .

Example A.3 We use the linear property of the Laplace transform to find the Laplace transform of

f(t) = sin ωt.

First, recall the Euler formula,

e^{jωt} = cos ωt + j sin ωt.

Hence,

e^{−jωt} = cos ωt − j sin ωt.

Subtracting the second of the above expressions from the first one and dividing the result by 2j gives

sin ωt = \frac{e^{jωt} − e^{−jωt}}{2j} .

Therefore,

L (sin ωt) = L (\frac{e^{jωt} − e^{−jωt}}{2j} ) = L (\frac{1}{2j} (e^{jωt} − e^{−jωt})) .

Applying to the above (A.148) yields

L (sin ωt) = \frac{1}{2j} L (e^{jωt} − e^{−jωt}) .

Applying now (A.147), we obtain

L (sin ωt) = \frac{1}{2j} (L (e^{jωt}) − L (e^{−jωt})) .

We then use twice the formula for the Laplace transform of the exponential to get

L (sin ωt) = \frac{1}{2j} (\frac{1}{s − jω} − \frac{1}{s + jω} ) .

Performing algebraic manipulations gives

L (sin ωt) = \frac{ω}{s^{2} + ω^{2}}

Similarly, we obtain

L (cos ωt) = L (\frac{e^{jωt} + e^{−jωt}}{2} ) .

Hence,

L (cos ωt) = \frac{s}{s^{2} + ω^{2}}

A.15.2 Laplace Transforms of Derivatives

We will now show that

L (\frac{df(t)}{dt}) = sF (s) − f (0^{−})

where F (s) = L(f(t)) and f (0^{−}) is the initial value of f, that is, the value of f at 0^{−}. Thus, differentiating in the time domain corresponds to multiplying F (s) by s and then subtracting the initial value of f(t). To derive the above formula, we apply the definition of the Laplace transform,

L (\frac{df(t)}{dt}) = ∫_{0^{−}}^{∞} \frac{df(t)}{dt}e^{−st}dt,  (A.149)

and then evaluate the integral by integrating by parts. Recall that the formula for integrating by parts can easily be derived from the formula for the derivative of the product of two functions. We use the notation,

u′ = \frac{du(t)}{dt}.

Then, we have

(uv)′ = u′v + uv′.

Integrating the above and rearranging gives

∫ u′v = uv − ∫ uv′.  (A.150)

We let

u′ = \frac{df}{dt} and v = e^{−st}.

Then applying (A.150) to (A.149), we obtain

L (\frac{df(t)}{dt}) = e^{−st}f(t)|_{0^{−}}^{∞} − ∫_{0^{−}}^{∞} f(t) (−se^{−st}) dt.

Assuming that f(t) is Laplace transformable, the value of e^{−st}f(t) at t = ∞ is zero. Hence, the right-hand side of the above reduces to

L (\frac{df(t)}{dt}) = −f (0^{−}) + s ∫_{0^{−}}^{∞} f(t)e^{−st}dt = sF (s) − f (0^{−}) .

Thus, we showed that differentiation in the time domain corresponds to an algebraic operation in the s domain.

Using the Laplace transform of the first derivative of a time function, we can easily determine the Laplace transform of higher-order derivatives. Indeed, to find the Laplace transform of \frac{d^{2}f(t)}{dt^{2}}, we let

g(t) = \frac{df(t)}{dt}.

Then,

L(g(t)) = G(s) = L (\frac{df(t)}{dt}) = sF (s) − f (0^{−}) .  (A.151)

Note that

\frac{d^{2}f(t)}{dt^{2}} = \frac{dg(t)}{dt}.

Hence,

L (\frac{df^{2}(t)}{dt^{2}}) = L (\frac{dg(t)}{dt}) = sG(s) − g (0^{−}) .

Substituting into the above (A.151) gives

L (\frac{df^{2}(t)}{dt^{2}}) = s (sF (s) − f (0^{−})) − g (0^{−}) .

Observing that g (0^{−}) = \frac{df(0^{−})}{dt}, we get

L (\frac{df^{2}(t)}{dt^{2}}) = s^{2}F (s) − sf (0^{−}) − \frac{df (0^{−})}{dt}

Successively applying the above arguments, we can obtain the Laplace transform of the n-th time derivative,

L (\frac{df^{n}(t)}{dt^{n}}) = s^{n}F (s) − s^{n−1}f (0^{−}) − s^{n−2} \frac{df (0^{−})}{dt} − ⋯ − s\frac{d^{n−2}f (0^{−})}{dt^{n−2}} − \frac{d^{n−1}f (0^{−})}{dt^{n−1}}

A.15.3 Solving Differential Equations Using Laplace Transform

We now illustrate how to use the Laplace transform and inverse Laplace transform to solve linear ordinary differential equations. After taking the Laplace transform of both sides of a differential equation and performing required manipulations in the s domain, we need to reconstruct the solution in the time domain. This is achieved using the inverse Laplace transform.

Example A.4 We will solve the following differential equation using the Laplace transform,

2\frac{d^{2}x(t)}{dt^{2}} + 7\frac{dx(t)}{dt} + 5x(t) = 10 × 1(t),

subject to the initial conditions

\frac{dx (0^{−})}{dt} = 1, x (0^{−}) = −2,

where 1(t) is the unit step function.

We take the Laplace transform of both sides of the differential equation to get

2L (\frac{d^{2}x(t)}{dt^{2}}) + 7L (\frac{dx(t)}{dt}) + 5L (x(t)) = 10L (1(t)) .

Using Laplace transforms of time derivatives of a function of time and the Laplace transform of the unit step function, we obtain

2 (s^{2}X(s) − sx (0^{−}) − \frac{dx (0^{−})}{dt}) + 7 (sX(s) − x (0^{−})) + 5X(s) = \frac{10}{s}.

Substituting into the above the initial conditions and rearranging gives

(2s^{2} + 7s + 5) X(s) = \frac{10}{s} − 4s − 12 = \frac{10 − 4s^{2} − 12s}{s}.

Hence,

X(s) = \frac{−4s^{2} − 12s + 10}{s (2s^{2} + 7s + 5)} = \frac{−2s^{2} − 6s + 5}{s (s^{2} + 3.5s + 2.5)} = \frac{−2s^{2} − 6s + 5}{s (s + 2.5) (s + 1)}.  (A.152)

We now need to reconstruct the solution of the differential equation in the time domain. In other words, given X(s) we wish to obtain x(t) in which x(t) is zero for t < 0 such that X(s) = L(x(t)). Thus,

x(t) = L^{−1} (X(s)) .

To proceed, note that X(s) as given by (A.152) is a rational function of s, that is, X(s) is a ratio of two polynomials in s, in which the degree of the numerator with respect to s is smaller than the degree of the denominator. Such a rational function is called strictly proper and can be expanded into a sum of partial fractions by writing a term or a series of terms for each zero of the denominator. We obtain

X(s) = \frac{K_{1}}{s} + \frac{K_{2}}{s + 2.5} + \frac{K_{3}}{s + 1}.  (A.153)

We next compute the constants K_{i}’s. We can do so by representing the right-hand side of the above as a rational function and then comparing the resulting numerator with the numerator of (A.152). We represent (A.153) as

X(s) = \frac{K_{1} (s + 2.5) (s + 1) + K_{2}s (s + 1) + K_{3}s (s + 2.5)}{s (s + 2.5) (s + 1)}
= \frac{(K_{1} + K_{2} + K_{3}) s^{2} + (3.5K_{1} + K_{2} + 2.5K_{3}) s + 2.5K_{1}}{s (s + 2.5) (s + 1)}  (A.154)

Comparing coefficients of like powers of the numerators of (A.152) and (A.154) gives three algebraic equations in three unknowns,

\begin{matrix}K_{1} + K_{2} + K_{3} & = & −2 \\ 3.5K_{1} + K_{2} + 2.5K_{3} & = & −6 \\ 5 & = & 2.5K_{1} \end{matrix} \}

A.15. THE LAPLACE TRANSFORM 1013 Solving the above system of equations gives

K_{1} = 2, K_{2} = 2, K_{3} = −6.

Substituting the above into (A.153) yields

X(s) = \frac{2}{s} + \frac{2}{s + 2.5} − \frac{6}{s + 1}.

Applying the inverse Laplace transform to the above, we obtain

x(t) = L (\frac{2}{s} + \frac{2}{s + 2.5} − \frac{6}{s + 1})
= L (\frac{2}{s}) + L (\frac{2}{s + 2.5}) − L (\frac{6}{s + 1})
= (2 + 2e^{−2.5t} − 6e^{−t}) 1(t).

A.15.4 Laplace Transforms of Integrals

We first consider taking the Laplace transform of

∫_{0^{−}}^{t} f(x)dx.  (A.155)

Let F (s) denote the Laplace transform of f(t), that is, F (s) = L(f(t)). We find the Laplace transform of (A.155) using integration by parts to obtain

L (∫_{0^{−}}^{t} f(x)dx) = ∫_{0^{−}}^{∞} (∫_{0^{−}}^{t} f(x)dx) e^{−st}dt.  (A.156)

Recall the formula for integration by parts,

∫ u′v = uv − ∫ uv′.

Let

u = −\frac{1}{s}e^{−st} and v = ∫_{0^{−}}^{t} f(x)dx.

Note that

\frac{dv}{dt} = \frac{d}{dt} ∫_{0^{−}}^{t} f(x)dx = f(t).

Integrating (A.156) by parts gives

L (∫_{0^{−}}^{t} f(x)dx) = −\frac{1}{s}e^{−st} ∫_{0^{−}}^{t} f(x)dx|_{0^{−}}^{∞} + \frac{1}{s} ∫_{0^{−}}^{∞} e^{−st}f(t)dt.

The first term on the right-hand side is zero at both the lower and upper limits. The value at the lower limit, t = 0^{−}, is clearly zero. The value at the upper limit, t = ∞, is zero because we assumed that f(t) is Laplace transformable. Hence,

L (∫_{0^{−}}^{t} f(x)dx) = \frac{F (s)}{s}  (A.157)

We next find the Laplace transform of

∫_{−∞}^{t} f(x)dx.  (A.158)

Applying the definition and using the fact that the Laplace transform is linear yields

L (∫_{−∞}^{t} f(x)dx) = L (∫_{−∞}^{0^{−}} f(x)dx) + L (∫_{0^{−}}^{t} f(x)dx) = F_{1}(s) + F_{2}(s).

The limits of the first integral are constant, therefore the integral will be a constant. Hence,

F_{1}(s) = \frac{∫_{−∞}^{0^{−}} f(x)dx}{s},

while by (A.157),

F_{2}(s) = \frac{F (s)}{s}.

Therefore,

L (∫_{−∞}^{t} f(x)dx) = \frac{F (s)}{s} + \frac{∫_{−∞}^{0^{−}} f(x)dx}{s}  (A.159)

Example A.5 We find the Laplace transform of the voltage, v_{C}(t), across a capacitor C. We have

v_{C}(t) = \frac{1}{C} ∫_{−∞}^{t} i(x)dx.

Using (A.159) gives

L (v_{C}(t)) = \frac{1}{C} \frac{I(s)}{s} + \frac{\frac{1}{C} ∫_{−∞}^{0^{−}} i(x)dx}{s}.

Since ∫_{−∞}^{0^{−}} i(x)dx is the charge, q, on the capacitor at t = 0^{−} and v = q/C, we obtain

L (v_{C}(t)) = \frac{1}{C} \frac{I(s)}{s} + \frac{v_{C} (0^{−})}{s}

More Properties of the Laplace Transform

We will now show that translation in the time domain corresponds to multiplication by an exponential in the s domain, that is,

L (f(t − a)1(t − a)) = e^{−sa}F (s), a > 0

where F (s) = L(f(t)). We have

L (f(t − a)1(t − a)) = ∫_{0^{−}}^{∞} f(t − a)1(t − a)e^{−st}dt = ∫_{a^{−}}^{∞} f(t − a)e^{−st}dt,

because 1(t − a) = 0 for t < a. Next, we change the variable of integration. We let

x = t − a.

Then, x = 0^{−} when t = a^{−}, x = ∞ when t = ∞, and dx = dt. Substituting the above into (A.160) yields

L (f(t − a)1(t − a)) = ∫_{0^{−}}^{∞} f(x)e^{−s(x+a)}dx = e^{−sa} ∫_{0^{−}}^{∞} f(x)e^{−sx}dx = e^{−as}F (s),

which is the desired result.

Next, we prove the time/frequency scaling property,

L(f(at)) = \frac{1}{a}F (\frac{s}{a}) , a > 0

Indeed, applying the Laplace transform yields

L(f(at)) = ∫_{0^{−}}^{∞} f(at)e^{−st}dt.  (A.160)

Let x = at. Then, dt = \frac{1}{a}dx. Substituting the above into (A.160) gives

L(f(at)) = \frac{1}{a} ∫_{0^{−}}^{∞} f(x)e^{−(\frac{s}{a})x}dx = \frac{1}{a}F (\frac{s}{a}) .

Using the sifting property, given by (1.1) on page 15, we show that

L(δ(t)) = 1

Indeed, applying the definition of the Laplace transform and using the sifting property (1.1), we obtain

L(δ(t)) = ∫_{0^{−}}^{∞} δ(t)e^{−st}dt = e^{0} ∫_{0^{−}}^{0^{+}} δ(t)dt = 1.

A.16 The Z-Transform

A.16.1 Definition and Simple Examples

The Z-transform is an operator denoted Z(·) that, when applied to a sequence of numbers produces a function of a variable z.

Definition A.6 For a given sequence of numbers

y(0), y(1), …, y(k), …

its Z-transform is the series defined as

Y (z) = Z(y(k)) = ∑_{k=0}^{∞} \frac{y(k)}{z^{k}}.

◇

[figure: f(kT); Z(·); F (z)]

Figure A.7: The Z-transform is an operator acting on a sequence of numbers resulting in a function of the variable z.

Example A.6 We find the Z-transform of the sampled unit step, f(kT) = 1. The Ztransform of this sequence is

F (z) = ∑_{k=0}^{∞} f(kT)z^{−k}
= 1 + \frac{1}{z} + \frac{1}{z^{2}} + ⋯

A.16. THE Z-TRANSFORM 1017 If z is a complex variable such that |z| > 1, then the above sequence converges and we have

F (z) = \frac{1}{1 − 1/z}
= \frac{z}{z − 1}.

Example A.7 We find the Z-transform of the sampled unit ramp, f(kT) = kT. The Z-transform of this sequence is

F (z) = ∑_{k=0}^{∞} kT z^{−k}
= T (z^{−1} + 2z^{−2} + 3z^{−3} + ⋯)  (A.161)

Multiplying both sides of the above by z gives

zF (z) = T (1 + 2z^{−1} + 3z^{−2} + ⋯)  (A.162)

Subtracting (A.161) from (A.162) yields

zF (z) − F (z) = (z − 1)F (z)
= T (1 + z^{−1} + z^{−2} + ⋯)
= \frac{1}{1 − z^{−1}}
= \frac{z}{z − 1}.  (A.163)

Hence, the Z-transform of f(kT) = kT is

Z(kT) = T \frac{z}{(z − 1)^{2}}  (A.164)

Example A.8 We now find the Z-transform of the geometric sequence

f(k) = a^{kT} .  (A.165)

We obtain

Z(a^{kT}) = ∑_{k=0}^{∞} \frac{a^{kT}}{z^{k}} = \frac{z}{z − a^{T}}  (A.166)

for |z| > |a^{T} |.

A.16.2 Some Properties of the Z-Transform

We analyze now three basic properties that are especially useful when computing transforms of more complicated sequences.

Linearity property: The Z-transform is a linear operator, that is,

(i) if F_{1}(z) = Z(f_{1}(kT)) and F_{2}(z) = Z(f_{2}(kT)) then

Z(f_{1}(kT) + f_{2}(kT)) = F_{1}(z) + F_{2}(z); and

(ii) if F (z) = Z(f(kT)) then

Z(cf(kT)) = cF (z),

where c is a real or complex number.

We represent the above two properties as: if F_{1}(z) = Z(f_{1}(kT)) and F_{2}(z) = Z(f_{2}(kT)), and c is a real or complex number, then

Z(f_{1}(kT) + cf_{2}(kT)) = F_{1}(z) + cF_{2}(z).

Z-transform of a delayed sequence: If the sequence f(kT) has the Z-transform F (z), then the unit delayed sequence

g(kT) = \{ \begin{matrix}f((k − 1)T) & k ≥ 1 \\ 0 & k = 0 \end{matrix}

has the Z-transform

G(z) = z^{−1}F (z).  (A.167)

We use the definition of the Z-transform to prove the above property;

G(z) = ∑_{k=0}^{∞} z^{−k}g(kT)
= ∑_{k=1}^{∞} z^{−k}f((k − 1)T)
= z^{−1} ∑_{k=1}^{∞} z^{−(k−1)}f((k − 1)T)
= z^{−1} ∑_{j=0}^{∞} z^{−j}f(jT)
= z^{−1}F (z).

A.16. THE Z-TRANSFORM 1019

Z-transform of an advanced sequence: If the sequence f(kT) has the Z-transform F (z), then the unit advanced sequence h(kT) = f((k + 1)T), k ≥ 0, has the transform

Z(f((k + 1)T)) = zF (z) − zf(0).  (A.168)

To prove the above, we apply the definition of the Z-transform to obtain,

H(z) = ∑_{k=0}^{∞} z^{−k}h(kT)
= ∑_{k=0}^{∞} z^{−k}f((k + 1)T)
= z ∑_{k=0}^{∞} z^{−k−1}f((k + 1)T)
= z ∑_{j=1}^{∞} z^{−j}f(jT)
= z ∑_{j=0}^{∞} z^{−j}f(jT) − zf(0)
= zF (z) − zf(0).

Example A.9 We find the Z-transform of the delayed geometric sequence,

f(kT) = \{ \begin{matrix}0 & for & k = 0 \\ a^{(k−1)T} & for & k > 0. \end{matrix}  (A.169)

Applying (A.167) to the geometric sequence (A.165) yields

Z(f(kT) = \frac{1}{z − a^{T}}  (A.170)

Solving Difference Equations Using the Z-Transform

The Z-transform is a very effective tool for solving linear, constant-coefficients difference equations. The basis for the method is the Z-transform of an advanced sequence given by (A.168). Repeated application of this property gives

Z(f((k + 2)T)) = z^{2}F (z) − z^{2}f(0) − zf(T).  (A.171)

Applying again (A.168) to (A.171) gives

Z(f((k + 3)T)) = z^{3}F (z) − z^{3}f(0) − z^{2}f(T) − zf(2T)  (A.172)

and so on.

Example A.10 We solve the difference equation

x[k + 1] + 3x[k] = 2^{k}, k = 0, 1, …,

where x(0) = 0. Applying the Z-transform to both sides of the above difference equation gives

zX(z) − zx(0) + 3X(z) = \frac{z}{z − 2}.

Hence,

X(z) = \frac{z}{(z + 3)(z − 2)},

which is a strictly proper rational function of z. We represent X(z) in partial fraction form as

X(z) = \frac{A}{z + 3} + \frac{B}{z − 2},

where

A = (z + 3)X(z)|_{z=−3} = \frac{3}{5} and B = (z − 2)X(z)|_{z=2} = \frac{2}{5}.

Therefore,

X(z) = \frac{3/5}{z + 3} + \frac{2/5}{z − 2}.

We use (A.170) when applying the inverse Z-transform to the above,

x[k] = \{ \begin{matrix}0 & for & k = 0 \\ \frac{3}{5}(−3)^{k−1} + \frac{2}{5}2^{k−1} & for & k ≥ 1 \end{matrix}

Example A.11 We solve the difference equation

x[k + 1] + 3x[k] = 2^{k}, k = 0, 1, …,

where x(0) = 5. Applying the Z-transform to both sides of the above difference equation gives

zX(z) − zx(0) + 3X(z) = \frac{z}{z − 2}.

Hence,

X(z) = \frac{5z^{2} − 9z}{(z + 3)(z − 2)} = \frac{z(5z − 9)}{(z + 3)(z − 2)}.

The above rational function is not strictly proper so we use a different approach than in Example A.10. In this example z is a factor of the numerator so we can represent the above as

\frac{X(z)}{z} = \frac{5z − 9}{(z + 3)(z − 2)}

and express the right hand side in partial fraction format as

\frac{X(z)}{z} = \frac{24/5}{z + 3} + \frac{1/5}{z − 2}.

Thus

X(z) = \frac{24}{5} \frac{z}{z + 3} + \frac{1}{5} \frac{z}{z − 2}.

Taking into account (A.166) when applying the inverse Z-transform yields

x[k] = \frac{24}{5}(−3)^{k} + \frac{1}{5}2^{k}, k ≥ 0

A.17 Curves and Surfaces

In the following, we review selected results from calculus of several variables concerning curves, surfaces, vector fields, and curve integrals.

Consider a point moving along a curve in n-dimensional space. The position of the point at time t can be described as,

x(t) = [ x_{1}(t) ; x_{2}(t) ; ⋮ ; x_{n}(t) ] .  (A.173)

For example, if the point is moving along a straight line passing through fixed points y and z in ℝ^{n}, then

x(t) = y + tz.  (A.174)

As another example, consider

x(t) = [ \begin{matrix}x_{1}(t) \\ x_{2}(t) \end{matrix} ] = [ \begin{matrix}cos t \\ sin t \end{matrix} ] ∈ ℝ^{2}.  (A.175)

Figure A.8: An example of a curve in two-dimensional space.

Then, the point moves around a circle of radius 1 in counterclockwise direction as shown in Figure A.8. We used t as the variable corresponding to the angle corresponding to the position of the point on the circle. We are now ready for a formal definition of a curve.

Definition A.7 Let I ∈ ℝ be an interval. A parameterized curve defined on the interval I is an association which to each point of I associates a vector, where x(t) denotes the vector associated to t ∈ I by x. We write the association t ↦ x(t) as

x : I → ℝ^{n}.

We also call this association the parameterization of a curve. We call x(t), the position vector at time t.

◇

The position vector can be represented in terms of its coordinates as in (A.173), where each component x_{i}(t) is a function of time t. We say that this curve is differentiable if each function x_{i}(t) is a differentiable function of time t. We define the derivative dx(t)/dt to be

ẋ(t) = \frac{dx(t)}{dt} = [ \frac{dx_{1}(t)}{dt} ; \frac{dx_{2}(t)}{dt} ; ⋮ ; \frac{dx_{n}(t)}{dt} ] ,

Figure A.9: Velocity vector of a curve at time t is parallel to a tangent vector at time t.

and call it the velocity vector of the curve at time t. The velocity vector is located at the origin. However, when we translate it to the point x(t), as in Figure A.9, then we visualize it as tangent to the curve x(t).

We define the tangent line to a curve x at time t to be the line passing through x(t) in the direction of ẋ(t), provided ẋ(t) ≠ 0. If ẋ(t) = 0, we do not define a tangent line. The speed of the curve x(t), denoted v(t), is

v(t) = ∥ẋ(t)∥ .

To illustrate the above, we consider the point moving on the circle. The point position at time t is

x(t) = [ cos t sin t ]^{⊤} .

The velocity vector of the curve at time t is

ẋ(t) = [ − sin t cos t ]^{⊤} ,

and the speed of the curve x(t) at time t is

v(t) = ∥ẋ(t)∥ = \sqrt{(− sin t)^{2} + cos^{2} t} = 1.

The set of points in ℝ^{3} such that for some F : ℝ^{3} → ℝ, we have

F (x) = F (x_{1}, x_{2}, x_{3}) = 0,

and

DF (x_{1}, x_{2}, x_{3}) = ∇F (x_{1}, x_{2}, x_{3})^{⊤} = [ \frac{∂F}{∂x_{1}}  \frac{∂F}{∂x_{2}}  \frac{∂F}{∂x_{3}} ] ≠ 0^{⊤},

is called a surface in ℝ^{3}, where ∇ F = grad F. For example, let

F (x_{1}, x_{2}, x_{3}) = x_{1}^{2} + x_{2}^{2} + x_{3}^{2} − 1,

then, the surface is the sphere of radius 1, centered at the origin of ℝ^{3}.

Let

C(t) = [ x_{1}(t) x_{2}(t) ⋯ x_{n}(t) ]^{⊤}

be a differentiable curve. We say that the curve lies on the surface described by

F_{1}(x_{1}, x_{2}, …, x_{n}) = 0 ; ⋮ ; F_{m}(x_{1}, x_{2}, …, x_{n}) = 0, }

if, for all t, we have

F_{1}(x_{1}(t), x_{2}(t), …, x_{n}(t)) = 0 ; ⋮ ; F_{m}(x_{1}(t), x_{2}(t), …, x_{n}(t)) = 0. }

Differentiating the above equations, using the chain rule, we obtain

DF_{i} (C(t)) Ċ(t) = ∇F_{i} (C(t))^{⊤} Ċ(t) = 0, i = 1, 2, …, m.

Let x_{0} be a point of the surface, and let C(t) be a curve on the surface passing through x_{0}, that is, for some t_{0} we have

C(t_{0}) = x_{0}.

For this t_{0} we have

∇F_{i} (x_{0})^{⊤} Ċ(t_{0}) = 0, i = 1, 2, …, m,

where we assumed that Ċ(t_{0}) ≠ 0. Thus, the gradients ∇F_{i} at x_{0} are orthogonal to the tangent vector of the curve at the point x_{0}. This is true for every differentiable curve on the surface passing through x_{0}. For an illustration of the above properties in three-dimensional space, we refer to Figure A.10.

Definition A.8 The tangent space to the surface {x : F (x) = 0} at the point x_{0} is the set of vectors at x_{0} that are tangent to differentiable curves on the surface passing through x_{0}.

◇ Figure A.10: Tangent space to a surface is the set of vectors orthogonal to the gradients of the functions defining the surface.

Vector Fields and Curve Integrals

Definition A.9 Let U be an open subset of ℝ^{n}. By a vector field on U we mean an association

F : U → ℝ^{n}

which to every point of U associates a vector of the same dimension.

◇

We can interpret a vector field as a field of forces. We can visualize a vector field as a field of arrows, which to each point associates an arrow as shown in Figure A.11. If f : U → ℝ is a function, then

F = ∇f

is a vector field, and we have

F (x) = ∇f(x)

for all x ∈ U. We now consider an inverse problem: When is a given vector field in ℝ^{n} a gradient?

Definition A.10 Let F be a vector field on an open set U. If f is a differentiable function on U such that

F = ∇f,

Figure A.11: A vector field.

then we say that f is a potential function for F.

◇

We then can ask the question if the potential functions exist. To answer the question, we recast the problem into the one that involves solving n first order partial differential equations:

\frac{∂f}{∂x_{i}} = F_{i}(x), i = 1, 2, …, n,

with the initial condition

F (x_{0}) = F_{0}.

To answer the above question we need one more definition.

Definition A.11 An open set U is said to be connected if, given two points in U, there is a differentiable curve in U that joins the two points.

◇

If U = ℝ^{n}, then any two points can be joined by a straight line. Of course, it is not always the case that two points of an open set can be joined by a straight line, as it is illustrated in Figure A.12. We say that the function F = [ F_{1} F_{2} ⋯ F_{n} ]^{⊤} is a C^{1} function if all partial derivatives of all functions F_{i} exist and are continuous.

Figure A.12: A connected open set in which there are points that cannot be joined by a straight line.

Theorem A.21 Let F (x) be C^{1} function defined on an open connected region U of ℝ^{n}. Then, there is a solution to the differential equation

∇f(x) = F (x), F (x_{0}) = F_{0},

if and only if

\frac{∂F_{i}}{∂x_{j}} = \frac{∂F_{j}}{∂x_{i}} ,

that is, the Jacobian matrix of F is symmetric.

Proof We prove the theorem for n = 2, where the connected region U is a rectangle. We select a point x_{0} = [ x_{10} x_{20} ]^{⊤} in the rectangle. We assume that F = [ F_{1} F_{2} ]^{⊤} and that

D_{2}F_{1} = D_{1}F_{2}, that is, \frac{∂F_{1}}{∂x_{2}} = \frac{∂F_{2}}{∂x_{1}} .

Our goal is to find a potential function f(x_{1}, x_{2}). We assume that the potential function has the form

f(x_{1}, x_{2}) = ∫_{x_{10}}^{x_{1}} F_{1}(s, x_{2})ds + u(x_{2}),

where u(x_{2}) is a function of x_{2} only and is determined later in the proof. We illustrate the integration path in Figure A.13. Applying the fundamental theorem of calculus, we obtain 1028 APPENDIX A. MATH REVIEW

Figure A.13: Integration path used in the proof of Theorem A.21.

\frac{∂f}{∂x_{1}} = \frac{∂}{∂x_{1}} ∫_{x_{10}}^{x_{1}} F_{1}(s, x_{2})ds + \frac{∂u(x_{2})}{∂x_{1}}
= F_{1}(x_{1}, x_{2})

because

\frac{∂u(x_{2})}{∂x_{1}} = 0.

Hence,

D_{1}f(x_{1}, x_{2}) = F_{1}(x_{1}, x_{2})

as required. We now differentiate f(x_{1}, x_{2}) with respect to x_{2} to get

\frac{∂f}{∂x_{2}} = ∫_{x_{10}}^{x_{1}} \frac{∂F_{1}(s, x_{2})}{∂x_{2}}ds + \frac{∂u(x_{2})}{∂x_{2}}
= ∫_{x_{10}}^{x_{1}} \frac{∂F_{2}(s, x_{2})}{∂x_{1}}ds + \frac{∂u(x_{2})}{∂x_{2}}

because, by assumption, D_{2}F = D_{1}F. Therefore,

\frac{∂f}{∂x_{2}} = F_{2}(s, x_{2})|_{x_{10}}^{x_{1}} + \frac{∂u(x_{2})}{∂x_{2}}
= F_{2}(x_{1}, x_{2}) − F_{2}(x_{10}, x_{2}) + \frac{∂u(x_{2})}{∂x_{2}}.

Let

−F_{2}(x_{10}, x_{2}) + \frac{∂u(x_{2})}{∂x_{2}} = 0,

Figure A.14: A particle moving along a curve in vector field.

that is, \frac{∂u(x_{2})}{∂x_{2}} = F_{2}(x_{10}, x_{2}). Then, D_{2}f = F_{2} as wanted. To conclude the proof, we let

u(x_{2}) = ∫_{x_{20}}^{x_{2}} F_{2}(x_{10}, x_{2})dx_{2},

which yields

f(x_{1}, x_{2}) = ∫_{x_{10}}^{x_{1}} F_{1}(s, x_{2})ds + ∫_{x_{20}}^{x_{2}} F_{2}(x_{10}, t)dt  (A.176)

The proof is complete for n = 2.

□

We note that if the connected region U is an arbitrary open set, then the above proof may not work. This is because we needed to integrate over intervals that were contained in the region U.

Suppose we are given a vector field F on an open set U in the plane. We can interpret F as a field of forces. We wish to calculate the work done when moving a particle from a point C(t_{1}) to a point C(t_{2}) along the curve C(t) in U, as illustrated in Figure A.14. Let

F (x) = [ F_{1}(x_{1}, x_{2}) F_{2}(x_{1}, x_{2}) ]^{⊤} .

We assume that each component of F is differentiable, that is, F is a differentiable vector field. We next assume that the curve C(t) is defined on a closed interval [t_{1}, t_{2}] with t_{1} < t_{2}. For each t ∈ [t_{1}, t_{2}] the value C(t) is a point in ℝ^{2}. The curve C lies in U if C(t) is a point in U for all t ∈ [t_{1}, t_{2}]. We further assume that the curve C is continuously differentiable and its derivative Ċ(t) = \frac{dC(t)}{dt} exists and is continuous, that is, we assume that C ∈ C^{1}.

We define the integral of F along C to be

∫C F = ∫_{t_{1}}^{t_{2}} F (C(t))^{⊤} \frac{dC}{dt} dt
= ∫_{t_{1}}^{t_{2}} F (C(t))^{⊤}dC,

which is the work done against the force field F along the curve C. Let

C(t) = [ x_{1}(t) x_{2}(t) ]^{⊤} .

Then, we can write

∫C F = ∫C F_{1}dx_{1} + F_{2}dx_{2}.  (A.177)

Theorem A.22 (Green) Let the vector field F be given on a region A that is the interior of a closed path C parameterized counterclockwise. Then,

∫C F_{1}dx_{1} + F_{2} dx_{2} = ∫∫_{A} (\frac{∂F_{2}}{∂x_{1}} − \frac{∂F_{1}}{∂x_{2}} ) dx_{1}dx_{2}.

A.18 Matrix Calculus Formulas

Below, we list useful formulas from matrix calculus that involve computing the gradient (with respect to x). Recall that in our convention the gradient is a column vector. We have:

∇ (x^{⊤}y) = ∇ (y^{⊤}x) = y,  (A.178)
∇ (x^{⊤}x) = 2x,  (A.179)
∇ (x^{⊤}Ay) = Ay,  (A.180)
∇ (y^{⊤}Ax) = A^{⊤}y,  (A.181)
∇ (x^{⊤}Ax) = (A + A^{⊤}) x.  (A.182)

Let f : ℝ^{n} → ℝ^{n} and g : ℝ^{n} → ℝ^{n} be two maps, and let \frac{∂f}{∂x} and \frac{∂g}{∂x} be the Jacobian matrices of f and g, respectively. Then,

∇ (f(x)^{⊤}g(x)) = ∇ (g(x)^{⊤}f(x)) = (\frac{∂f}{∂x})^{⊤} g + (\frac{∂g}{∂x})^{⊤} f.  (A.183)

For an n × n symmetric matrix Q,

∇ (f(x)^{⊤}Qf(x)) = 2 (\frac{∂f}{∂x})^{⊤} Qf.  (A.184)

A.19 Notes

For other proof techniques, we refer to Velleman [335]. For further reading on linear algebra, and in particular matrix theory, we recommend classical texts of Gelfand [112] and Gantmacher [110]. Analyses of numerical methods for matrix computations can be found in Faddeeva [98], Householder [144], Golub and Van Loan [117], as well as in Gill, Murray, and Wright [114]. Lang [203] and Seeley [297] are recommended for a review of calculus of several variables. A straightforward approach to mathematical analysis is presented in Binmore [35]. Principles of real analysis can be found in Natanson [253], Bartle [25], and Rudin [285]. Some facts from basic functional analysis that we touched upon in this chapter, are discussed, for example, by Maddox [227], and Debnath and Mikusiński [74]. Lucid expositions of differential equations are in Driver [88], Miller and Michel [242], and in Boyce and DiPrima [40].

The famous mathematician, G. H. Hardy (1877–1947) argues in [130, page 113] that in truly great theorems and their proofs, “there is a very high degree of unexpectedness, combined with inevitability and economy. The arguments take so odd and surprising a form; the weapons used seem so childishly simple when compared with the far-reaching results; but there is no escape from the conclusions.” He further states that “A mathematical proof should resemble a simple and clear-cut constellation, not a scattered cluster in the Milky Way.”

A.20 Exercises

Exercise A.1 The open unit disc with respect to the norm ∥ · ∥_{p}, is the set

Δ_{p} = {x ∈ ℝ^{2} : ∥x∥_{p} < 1}.

The unit discs are quite different for different choices of the norm ∥ · ∥_{p}. Sketch Δ_{1}, Δ_{2}, Δ_{3}, and Δ_{∞}.

Exercise A.2 Let v_{1}, v_{2}, …, v_{n} be non-zero, mutually orthogonal elements of the space V and let v be an arbitrary element of V.

1032 APPENDIX A. MATH REVIEW (i) Find the component of v along v_{j}. (ii) Let c_{j} denote the component of v along v_{j}. Show that the element

v − c_{1}v_{1} − ⋯ c_{n}v_{n}

is orthogonal to v_{1}, v_{2}, …, v_{n}. (iii) Verify the Bessel inequality,

∑_{i=1}^{n} c_{i}^{2} ≤ ∥v∥^{2}.

Exercise A.3 (M.C.M. Teixeira) Given a linear combination of matrices,

M = α_{1}M_{1} + ⋯ + α_{r}M_{r},  (A.185)

where for i = 1, …, r, α_{i} ∈ ℝ, and M_{i} ∈ ℝ^{n×n}. Let m_{i}^{j} be the j-th row of the matrix M_{i}. Thus,

M_{i} = [ m_{i}^{1} ; m_{i}^{2} ; ⋮ ; m_{i}^{n} ] .

Show that

det M = ∑_{s_{1}=1}^{r} ⋯ ∑_{s_{n}=1}^{r} α_{s_{1}} ⋯ α_{s_{n}} det [ m_{s_{1}}^{1} ; ⋮ ; m_{s_{n}}^{n} ] .  (A.186)

Exercise A.4 Let X ∈ ℝ^{n×m} and Y ∈ ℝ^{m×n}. Show that det (I_{n} + XY ) = det (I_{m} + Y X) .

Exercise A.5 Let λ_{i}(A) denote the i-th eigenvalue of a matrix A ∈ ℝ^{n×n} and let α be a scalar. Show that

(i) λ_{i}(αA) = αλ_{i}(A); (ii) λ_{i}(I_{n} ± αA) = 1 ± αλ_{i}(A). Exercise A.6 Show that the formulas (A.75) and (A.76) for matrix norms are equivalent.

Exercise A.7 Show that for any square matrix A ∈ ℝ^{n×n},

∥A∥ ≥ max_{1≤i≤n} |λ_{i}(A)| .  (A.187)

Exercise A.8 Show that if P ∈ ℝ^{n×n} is symmetric, then

λ_{min}(P)∥x∥_{2}^{2} ≤ x^{⊤}P x ≤ λ_{max}(P)∥x∥_{2}^{2}.

Exercise A.9 Show that a sufficient condition for

lim_{k→∞} A^{k} → O.

is

∥A∥ < 1.

Exercise A.10 Let A ∈ ℝ^{n×n}. Show that a necessary and sufficient condition for

lim_{k→∞} A^{k} → O.

is

|λ_{i}(A)| < 1, i = 1, 2, …, n

Exercise A.11 Let eig(M) denote the set of eigenvalues of the square matrix M and let Δ be the open unit disk in the complex plane. Prove that if eig(M) ⊂ Δ∪{1}, then lim_{k→∞} M^{k} exists if and only if the dimension of the eigenspace corresponding to the eigenvalue 1 is equal to the multiplicity of the eigenvalue 1, that is, the geometric multiplicity of the eigenvalue 1 is equal to its algebraic multiplicity.

Exercise A.12 (a) Show that the series

∑_{n=1}^{∞} \frac{1}{n}

diverges to +∞.

(a) Show that if α is a rational number such that α > 1, then the series

∑_{n=1}^{∞} \frac{1}{n^{α}}

converges.

1034 APPENDIX A. MATH REVIEW (c) Use MATLAB’s Symbolic Math Toolbox to compute

∑_{n=1}^{∞} \frac{1}{n^{2}} .

Exercise A.13 (a) Let 0 < x < 1. Show that

ln(1 − x) ≤ −x.

(b) Let 0 < x ≤ 1/2. Show that

ln(1 − x) ≥ −2x.

Exercise A.14 Let 0 < a_{k} < 1 for k ≥ 0. Use a proof by contraposition, Theorem A.16, and Exercise A.13 to show that

∏_{k=0}^{∞}(1 − a_{k}) = 0, if and only if ∑_{k=0}^{∞} a_{k} = ∞.

Exercise A.15 Prove Sylvester’s inequalities (A.88). Exercise A.16 Compute e^{At} for

(a) A = [ \begin{matrix}0 & −3 & 0 \\ 3 & 0 & 0 \\ 0 & 0 & −1 \end{matrix} ] , (b) A = [ \begin{matrix}0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & −2 & −3 \end{matrix} ] .

Exercise A.17 Find x(t) for the dynamical system model,

ẋ(t) = [ \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ] x(t) + [ \begin{matrix}0 \\ 1 \end{matrix} ] u(t), x(0) = [ \begin{matrix}0 \\ 1 \end{matrix} ] ,

where u(t) is the unit step function, that is, u(t) = 1 for t ≥ 0 and u(t) = 0 for t < 0. Exercise A.18 For

A = [ \begin{matrix}−2 & −6 \\ 2 & −10 \end{matrix} ] ,

find constants M > 0 and α > 0 such that ‖e^{At}‖_{2} ≤ Me^{−αt}, t ≥ 0.

Exercise A.19 Show that the function space C([a, b]) of real-valued functions, f : ℝ → ℝ, with the norm defined by (A.99) is a normed linear space. Show that this space is a Banach space.

Exercise A.20 Derive a sufficient condition for the convergence of the iteration process

x^{(k+1)} = Ax^{(k)} + b, k = 0, 1, …,

for any initial x(0), where A ∈ ℝ^{n×n} and b ∈ ℝ^{n}.

1036 APPENDIX A. MATH REVIEW

Bibliography

[1] P. P. Acarnley. Stepping Motors: A Guide to Modern Theory and Practice. IEE Control Engineering Series 19. Peter Peregrinus Ltd., London, 1982.

[2] J. Ackermann. Robust control prevents car skiding. IEEE Control Systems Magazine, 17(3):23–31, June 1997.

[3] M. Adachi and K. Aihara. Associative dynamics in a chaotic neural network. Neural Networks, 10(1):83–98, January 1997.

[4] J. A. Adam and N. Bellomo, editors. A Survey of Models for Tumor-Immune System Dynamics. Birkhäuser, Boston, 1997.

[5] K. Aihara, T. Takabe, and M. Toyoda. Chaotic neural networks. Physics Letters A, 144(6, 7):333–340, 12 March 1990.

[6] A. W. Al-Khafaji and J. R. Tooley. Numerical Methods in Engineering Practice. Holt, Rinehart and Winston, Inc., New York, 1986.

[7] K. T. Alligood, T. D. Sauer, and J. A. Yorke. CHAOS: An Introduction to Dynamical Systems. Springer-Verlag, New York, 1997.

[8] H. Alwi and C. Edwards. Fault detection and fault-tolerant control of a civil aircraft using a sliding-mode-based scheme. IEEE Transactions on Control Systems Technology, 16(3):499–510, May 2008.

[9] H. Alwi, C. Edwards, and C. P. Tan. Sliding mode estimation schemes for incipient sensor faults. Automatica, 45(7):1679–1685, July 2009.

[10] J. A. Anderson. An Introduction to Neural Networks. A Bradford Book, The MIT Press, Cambridge, Massachusetts, 1995.

[11] Y. V. Andreyev, Y. L. Belsky, A. S. Dmitriev, and D. A. Kuminov. Information processing using dynamical chaos: Neural networks implementation. IEEE Transactions on Neural Networks, 7(2):290–299, March 1996.

[12] P. J. Antsaklis and A. N. Michel. Linear Systems. McGraw-Hill, New York, 1997.

[13] P. J. Antsaklis and A. N. Michel. A Linear Systems Primer. Birkhäuser, Boston, 2007.

[14] D. K. Arrowsmith and C. M. Place. An Introduction to Dynamical Systems. Cambridge University Press, Cambridge, Great Britain, 1990.

[15] K. J. Åström and T. Hägglund. PID Control. In W. S. Levine, editor, The Control Handbook, chapter 10.5, pages 198–209. CRC Press in cooperation with IEEE Press, Boca Raton, Florida, 1996.

[16] A. Azemi and E. E. Yaz. Sliding-mode adaptive observer appraoch to chaotic synchronization. Journal of Dynamic Systems, Measurement, and Control, 122:758–765, December 2000.

[17] T. Bäck, U. Hammel, and H.-P. Schwefel. Evolutionary computation: comments on the history and current state. IEEE Transactions on Evolutionary Computation, 1(1):3–17, April 1997.

[18] F. N. Bailey. The application of Lyapunov’s second method to interconnected systems. Journal of SIAM on Control, 3(3):443–462, 1966.

[19] G. J. Bakker and H. Nijmeijer. Comments on ‘Control design for multivariable nonlinear time-varying systems’. Int. J. Control, 41(6):1627–1629, 1985.

[20] P. Ball, editor. A Biography of Water: Life’s Matrix. Farra, Straus and Giroux, New York, 1999.

[21] S. P. Banks. Control Systems Engineering: Modelling and Simulation, Control Theory and Microprocessor Implementation. Prentice-Hall International, Englewood Cliffs, NJ 07632, 1986.

[22] B. R. Barmish and A. R. Galimidi. Robustness of Luenberger observers: Linear systems stabilized via non-linear control. Automatica, 22(4):413–423, July 1986.

[23] S. Barnett. Introduction to Mathematical Control Theory. Clarendon Press, Oxford, [24] M. Barnsley. Fractals Everywhere. Academic Press, San Diego, 1988.

[25] R. G. Bartle. The Elements of Real Analysis. John Wiley & Sons, New York, second edition, 1976.

[26] E. F. Beckenbach and R. Bellman. Inequalities. Springer-Verlag, New York Inc., Berlin, 1965. Second revised printing.

[27] R. Bellman. Introduction to Matrix Analysis. McGraw-Hill Book Company, Inc., New York, 1960.

[28] R. Bellman. Vector Lyapunov functions. Journal of SIAM on Control, 1(1):32–34, 1962.

[29] A. Ben-Zvi, S. D. Vernon, and G. Broderick. Model-based therapeutic correction of hypothalamic-pituitary-adrenal axis dysfunction. PLoS Computational Biology, 5(1):1– 10, January 23, 2009.

[30] S. Bennett. Development of the PID controller. IEEE Conrol Systems Magazine, 13(6):58–65, December 1993.

[31] D. Bestle and M. Zeitz. Canonical form observer design for non-linear time-variable systems. Int. J. Control, 38(2):419–431, 1983.

[32] S. P. Bhattacharyya. Observer design for linear systems with unknown inputs. IEEE Transactions on Automatic Control, AC-23(3):483–484, June 1978.

[33] A. Bhaya and E. Kaszkurewicz. Control Perspectives on Numerical Algorithms and Matrix Problems. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, 2006.

[34] L. Bingzheng, Z. Zhenye, and C. Liansong. A mathematical model of the regulation system of the secretion of glucocorticoids. J. Biological Physics, 17(4):221–233, December 1990.

[35] K. G. Binmore. Mathematical Analysis: A Straightforward Approach. Cambridge University Press, Cambridge, Great Britain, second edition, 1982.

[36] C. Bissell. A history of automatic control. In S. Y. Nof, editor, Springer Handbook of Automation, chapter 4, pages 53–69. Springer-Verlag, Berlin Heidelberg, 2009.

[37] D. L. Boley. On Kalman’s procedure for the computation of the controllable/observable canonical form. SIAM J. Control and Optimization, 18(6):624–626, November 1980.

[38] V. G. Boltyanskii. Mathematical Methods of Optimal Control. Holt, Rinehart and Winston, Inc., New York, 1971.

[39] H. Bourlès and B. Marinescu. Linear Time-Varying Systems: Algebraic-Analytic Approach. Springer-Verlag, Berlin Heidelberg, 2011. Lecture Notes in Control and Information Sciences 410.

[40] W. E. Boyce and R. C. DiPrima. Elementary Differential Equations and Boundary Value Problems. John Wiley & Sons, Inc., New York, sixth edition, 1997.

[41] S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix Inequalities in System and Control Theory. SIAM Studies in Applied Mathematics. Society for Industrial and Applied Mathematics, Philadelphia, 1994.

[42] J. W. Brewer. Kronecker products and matrix calculus in system theory. IEEE Transactions on Circuits and Systems, CAS-25(9):772–781, September 1978.

[43] R. W. Brockett. Finite Dimensional Linear Systems. J. Wiley & Sons, Inc., New York, 1970.

[44] W. L. Brogan. Modern Control Theory. Prentice Hall, Englewood Cliffs, New Jersey 07632, third edition, 1991.

[45] A. E. Bryson and Y. C. Ho. Applied Optimal Control. Blaisdell Publishing Co., Waltham, Mass., 1975.

[46] A. E. Bryson, Jr. Dynamic Optimization. Addison Wesley Longman, Inc., Menlo Park, CA, 1999.

[47] A. Bunde and S. Havlin, editors. Fractals in Science. Springer-Verlag, Berlin, 1994.

[48] J. B. Burl. Linear Optimal Control; ℋ_{2} and ℋ_{∞} Methods. Addison Wesley Longman, Inc., Menlo Park, CA, 1999.

[49] E. F. Camacho and C. Bordons. Model Predictive Control. Springer, London, second edition, 2004.

[50] R. H. Cannon, Jr. Dynamics of Physical Systems. McGraw-Hill, New York, 1967.

[51] S. G. Cao, N. W. Rees, and G. Feng. Quadratic stability analysis and design of continuous-time fuzzy control systems. Int. J. System Science, 27(2):193–203, 1996.

[52] T. L. Carroll and L. M. Pecora. Synchronizing chaotic circuits. IEEE Transactions on Circuits and Systems, 38(4):453–456, April 1991.

[53] H. Y. Chan and S. H. Żak. Chaotic neural fuzzy associative memory. Int. J. Bifurcation and Chaos, 9(8):1597–1617, August 1999.

[54] K. C. C. Chan, V. Lee, and H. Leung. Generating fuzzy rules for target tracking using a steady-state genetic algorithm. IEEE Transactions on Evolutionary Computation, 1(3):189–200, September 1997.

[55] C.-T. Chen. Linear System Theory and Design. Oxford University Press, New York, third edition, 1999.

[56] J. Chen and R. J. Patton. Robust Model-Based Fault Diagnosis for Dynamic Systems. Kluwer Academic Publishers, Boston, 1999.

[57] J. Chen, R. J. Patton, and H.-Y. Zhang. Design of unknown input observers and robust fault detection filters. Int. J. Control, 63(1):85–105, 1996.

[58] M. Chen and W. Min. Unknown input observer based chaotic secure communication. Physics Letters A, 372(10):1595–1600, 3 March 2008.

[59] X. Chen and H. Kano. State observer for a class of nonlinear systems and its application to machine vision. IEEE Transactions on Automatic Control, 49(11):2085–2091, November 2004.

[60] J. Chiasson and B. Vairamohan. Estimating the state of charge of a battery. In Proceedings of the 2003 American Control Conference, Denver, Colorado, pages 2863– 2868, June 4–6, 2003.

[61] J. Chiasson and B. Vairamohan. Estimating the state of charge of a battery. IEEE Transactions on Control Systems Technology, 13(3):465–470, May 2005.

[62] E. K. P. Chong and S. H. Żak. An Introduction to Optimization. Wiley-Interscience, John Wiley & Sons, Inc., New York, third edition, 2008.

[63] A. Cichocki and R. Unbehauen. Neural Networks for Optimization and Signal Processing. John Wiley, Chichester, England, 1993.

[64] M. Conrad, C. Hubold, B. Fischer, and A. Peters. Modeling the hypothalamuspituitary-adrenal system: homeostasis by interacting positive and negative feedback. J. Biological Physics, 35(2):149–162, May 2009.

[65] S. D. Conte and C. de Boor. Elementary Numerical Analysis: An Algorithmic Approach. McGraw-Hill Book Company, New York, third edition, 1980.

[66] I. M. Copi. Introduction to Logic. Macmillan Publishing Co., Inc., New York, fifth edition, 1978.

[67] C. Corduneanu. Principles of Differential and Integral Equations. Allyn and Bacon, Inc., Boston, Massachusetts, 1971.

[68] M. Corless and J. Tu. State and input estimation for a class of uncertain systems. Automatica, 34(6):757–764, June 1998.

[69] M. J. Corless and G. Leitmann. Continuous state feedback guaranteeing uniform ultimate boundedness for uncertain dynamic systems. IEEE Transactions on Automatic Control, AC-26(5):1139–1144, October 1981.

[70] K. M. Cuomo, A. V. Oppenheim, and S. H. Strogatz. Synchronization of Lorenz-based chaotic circuits with applications to communications. IEEE Transactions on Circuits and Systems—II: Analog and Digital Signal Processing, 40(10):626–633, October 1993.

[71] M. Darouach, M. Zasadzinski, and S. J. Xu. Full-order observers for linear systems with unknown inputs. IEEE Transactions on Automatic Control, 39(3):606–609, March 1994.

[72] D. M. Dawson, J. Hu, and T. C. Burg. Nonlinear Control of Electric Machinery. Marcel Dekker, New York, 1998.

[73] J. J. D’Azzo and C. H. Houpis. Linear Control System Analysis and Design: Conventional and Modern. The McGraw-Hill, Inc., New York, fourth edition, 1995.

[74] L. Debnath and P. Mikusiński. Introduction to Hilbert Spaces with Applications. Academic Press, Inc., San Diego, 1990.

[75] R. A. DeCarlo. Linear Systems: A State Variable Approach With Numerical Implementation. Prentice Hall, Englewood Cliffs, New Jersey 07632, 1989.

[76] R. A. DeCarlo, S. H. Żak, and S. V. Drakunov. Variable structure, sliding-mode controller design. In W. S. Levine, editor, The Control Handbook: Control System Advanced Methods, chapter 50, pages 50–1 – 50–22. CRC Press, Boca Raton, Florida, 2011.

[77] R. A. DeCarlo, S. H. Żak, and G. P. Matthews. Variable structure control of nonlinear multivariable systems: A tutorial. Proceedings of the IEEE, 76(3):212–232, March 1988.

[78] V. Del Toro. Electric Machines and Power Systems. Prentice Hall, Inc., Englewood Cliffs, N.J. 07632, 1985.

[79] M. J. Denham. Canonical forms for the identification of multivariable linear systems. IEEE Transactions on Automatic Control, AC-19(6):646–656, December 1974.

[80] A. Derdiyok. Speed-sensorless control of induction motor using a continuous control approach of sliding-mode and flux observer. IEEE Transactions on Industrial Electronics, 52(4):1170–1176, August 2005.

[81] A. Derdiyok, M. K. Güven, H. Rehman, N. Inanc, and L. Xu. Design and implementation of a new sliding-mode observer for speed-sensorless control of induction machine. IEEE Transactions on Industrial Electronics, 49(5):1177–1182, October 2002.

[82] R. L. Devaney and L. Keen, editors. Chaos and Fractals; The Mathematics Behind the Computer Graphics, Providence, Rhode Island, 1989. American Mathematical Society. Proceedings of Symposia in Applied Mathematics, Volume 39.

[83] B. M. Diong and J. V. Medanic. Dynamic output feedback variable structure control for system stabilization. Int. J. Control, 56(3):607–630, September 1992.

[84] A. S. Dmitriev, A. I. Panas, and S. O. Starkov. Storing and recognizing information based on stable cycles of one-dimensional maps. Physics Letters A, 155(8, 9):494–499, 27 May 1991.

[85] R. C. Dorf and R. H. Bishop. Modern Control Systems. Prentice Hall, Upper Saddle River, New Jersey, twelfth edition, 2011.

[86] D. Driankov, H. Hellendoorn, and M. Reinfrank. An Introduction to Fuzzy Control. Springer-Verlag, Berlin, 1993.

[87] M. Driels. Linear Control Systems Engineering. McGraw-Hill, Inc., New York, 1996.

[88] R. D. Driver. Ordinary and Delay Differential Equations. Springer-Verlag, New York, 1977.

[89] W. Dunham. Journey Through Genius: The Great Theorems of Mathematics. Wiley Science Editions, J. Wiley & Sons, Inc., New York, 1990.

[90] C. Edwards and S. K. Spurgeon. On the development of discontinuous observers. Int. J. Control, 59(5):1211–1229, May 1994.

[91] C. Edwards and S. K. Spurgeon. Compensator based output feedback sliding mode controller design. Int. J. Control, 71(4):601–614, 10 November 1998.

[92] C. Edwards and S. K. Spurgeon. Sliding Mode Control: Theory and Applications. Systems and Control Book Series. Taylor & Francis Ltd, London, England, 1998.

[93] C. Edwards, S. K. Spurgeon, and R. J. Patton. Sliding mode observers for fault detection and isolation. Automatica, 36(4):541–553, April 2000.

[94] G. Ellis. Observers in Control Systems; A Practical Guide. Academic Press, San Diego, 2002.

[95] G. D. Elseth and K. D. Baumgardner. Genetics. Addison-Wesley Publishing Company, Reading, Massachusetts, 1984.

[96] L. E. Elsgolc. Calculus of Variations. Pergamon Press Ltd., London, 1961.

[97] K. D. Elworthy, W. N. Everitt, and E. B. Lee, editors. Differential Equations, Dynamical Systems, and Control Science: A Festschrift in Honor of Lawrence Markus. Marcel Dekker, Inc., New York, 1994. Lecture Notes in Pure and Applied Mathematics, Vol. 152.

[98] V. N. Faddeeva. Computationl Methods of Linear Algebra. Dover Publications, Inc., New York, 1959.

[99] L. Fausett. Fundamentals of Neural Networks; Architectures, Algorithms, and Applications. Prentice Hall, Englewood Cliffs, NJ 07632, 1994.

[100] M. J. Feigenbaum. Universal behavior in nonlinear systems. Physica D, 7D(1–3):16–39, May 1983.

[101] G. Feng, S. G. Cao, N. W. Rees, and C. K. Chak. Design of fuzzy control systems with guaranteed stability. Fuzzy Sets and Systems, 85(1):1–10, January 1, 1997.

[102] A. F. Filippov. Differential equations with discontinuous right-hand side. In American Mathematical Society Translations, pages 199–231. American Mathematical Society, Providence, Rhode Island, 1964.

[103] A. F. Filippov. Differential Equations with Discontinuous Righthand Sides. Mathematics and Its Applications (Soviet Series). Kluwer Academic Publishers, Dordrecht, The Netherlands, 1988.

[104] K. Fischle and D. Schröder. An improved stable adaptive fuzzy control method. IEEE Transactions on Fuzzy Systems, 7(1):27–40, February 1999.

[105] D. B. Fogel. What is evolutionary computation? IEEE Spectrum, 37(2):26–32, February 2000.

[106] G. F. Franklin and J. D. Powell. Digital Control of Dynamic Systems. Addison-Wesley Publishing Company, Reading, Massachusetts, 1980.

[107] G. F. Franklin, J. D. Powell, and A. Emami-Naeini. Feedback Control of Dynamic Systems. Prentice-Hall, Inc., Upper Saddle River, New Jersey 07458, fourth edition, 2002.

[108] B. Friedland. Advanced Control System Design. Prentice-Hall, Englewood Cliffs, New Jersey 07632, 1996.

[109] A. T. Fuller. Maxwell’s treatment of governors fitted with differential gears. Int. J. Control, 65(3):385–408, October 1996.

[110] F. R. Gantmacher. The Theory of Matrices, volume 1. Chelsea Publishing Company, New York, second edition, 1990.

[111] S. S. Ge, T. H. Lee, and G. Zhu. Genetic algorithm tuning of Lyapunov-based controllers: An application to a single-link flexible robot system. IEEE Transactions on Industrial Electronics, 43(5):567–574, October 1996.

[112] I. M. Gel’fand. Lectures on Linear Algebra. Interscience Publishers, Inc., New York, [113] M. Gen and R. Cheng. Genetic Algorithms and Engineering Design. A Wiley-

Interscience Publication, John Wiley & Sons, Inc., New York, 1997.

[114] P. E. Gill, W. Murray, and M. H. Wright. Numerical Linear Algebra and Optimization, volume 1. Addison-Wesley Publishing Company, Redwood City, CA, 1991.

[115] J. Gleick. Chaos: Making a New Science. Penguin Books, New York, 1988.

[116] I. Glynn, editor. An Anatomy of Thought: The Origin and Machinery of Mind. Oxford University Press, New York, 1999.

[117] G. H. Golub and C. F. Van Loan. Matrix Computations. The Johns Hopkins University Press, Baltimore, Maryland, 1983.

[118] D. Graham and D. McRuer. Analysis of Nonlinear Control Systems. J. Wiley & Sons, Inc., New York, 1961.

[119] W. J. Grantham and A. M. Athalye. Discretization chaos: Feedback control and transition to chaos. In C. T. Leondes, editor, Control and Dynamic Systems; Advances in Theory and Applications, Vol. 34: Advances in Control Mechanics, Part 1 of 2, pages 205–277. Academic Press, San Diego, California 92101, 1990.

[120] G. Grassi and S. Mascolo. Nonlinear observer design to synchronize hyperchaotic systems via a scalar signal. IEEE Transactions on Circuits and Systems—I: Fundamental Theory and Applications, 44(10):1011–1014, October 1997.

[121] D. Greenfield. When is PID not the answer? Control Engineering, 57(3):54–57, March 2010.

[122] S. Gupta, E. Aslakson, B. M. Gurbaxani, and S. D. Vernon. Inclusion of the glucocorticoid receptor in a hypothalamic pituitary adrenal axis model reveals bistability. Theoretical Biology and Medical Modeling, 4(8):1–12, 14 February 2007.

[123] Q. P. Ha, H. Trinh, H. T. Nguyen, and H. D. Tuan. Dynamic output feedback slidingmode control using pole placement and linear functional observers. IEEE Transactions on Industrial Electronics, 50(5):1030–1037, October 2003.

[124] V. B. Haas. EE 675 Class Notes. Purdue University, School of Electrical Engineering, West Lafayette, IN., Fall 1980.

[125] M. T. Hagan, H. B. Demuth, and M. Beale. Neural Network Design. PWS Publishing Co., Boston, MA, 1996.

[126] T. Hägglund and K. J. Åström. Automatic tuning of PID controllers. In W. S. Levine, editor, The Control Handbook: Control System Advanced Methods, chapter 32, pages 32–1 – 32–20. CRC Press, Boca Raton, Florida, 2011.

[127] W. Hahn. Über die Anwendung der Methode von Ljapunov auf Differenzengleichungen. Mathematische Annalen, 136(1):430–441, 1958.

[128] W. Hahn. Theory and Application of Liapunov’s Direct Method. Prentice-Hall International Series in Applied Mathematics. Prentice-Hall, Englewood Cliffs, N.J., 1963.

[129] W. Hahn. Stability of Motion. Springer Verlag, New York, 1967.

[130] G. H. Hardy. A Mathematician’s Apology. Cambridge University Press, Cambridge, Great Britain, 1992. Canto edition with a foreword by C. P. Snow.

[131] R. L. Harvey. Neural Network Principles. Prentice Hall, Englewood Cliffs, NJ 07632, 1994.

[132] M. H. Hassoun. Fundamentals of Artificial Neural Networks. A Bradford Book, The MIT Press, Cambridge, MA, 1995.

[133] S. W. Hawking. A Brief History of Time: From the Big Bang to Black Holes. Bantam Books, New York, 1988.

[134] S. Haykin. Neural Networks: A Comprehensive Foundation. Prentice Hall, Upper Saddle River, NJ 07458, second edition, 1999.

[135] J. Hertz, A. Krogh, and R. G. Palmer. Introduction to the Theory of Neural Computation. Lecture Notes Volume I in the Santa Fe Institute Studies in the Sciences of Complexity. Addison-Wesley Publishing, Redwood City, CA, 1991.

[136] M. R. Hestenes. Calculus of Variations and Optimal Control Theory. John Wiley & Sons, Inc., New York, 1966.

[137] N. J. Higham. Handbook of Writing for the Mathematical Sciences. SIAM, Philadelphia, 1993.

[138] H. Hochstadt. Differential Equations: A Modern Approach. Dover Publications, Inc., New York, 1975.

[139] J. H. Holland. Adaptation in Natural and Artificial Systems; An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. A Bradford Book, The MIT Press, Cambridge, Massachusetts, 1992.

[140] J. J. Hopfield. Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA, Biophysics, 81:3088–3092, May 1984.

[141] G. Hostetter and J. S. Meditch. Observing systems with unmeasurable inputs. IEEE Transactions on Automatic Control, AC-18(3):307–308, June 1973.

[142] M. Hou and P. C. Müller. Design of observers for linear systems with unknown inputs. IEEE Transactions on Automatic Control, 37(6):871–875, June 1992.

[143] M. Hou, A. C. Pugh, and P. C. Müller. Disturbance decoupled functional observers. IEEE Transactions on Automatic Control, 44(2):382–386, February 1999.

[144] A. S. Householder. The Theory of Matrices in Numerical Analysis. Dover Publications, Inc., New York, 1975.

[145] J. C. Hsu and A. U. Meyer. Modern Control Principles and Applications. McGraw-Hill Book Company, New York, 1968.

[146] S. Hui and S. H. Żak. Control and observation of uncertain systems: A variable structure systems approach. In C. T. Leondes, editor, Control and Dynamic Systems; Advances in Theory and Applications, Vol. 34: Advances in Control Mechanics, Part 1 of 2, pages 175–204. Academic Press, San Diego, California 92101, 1990.

[147] S. Hui and S. H. Żak. Robust control synthesis for uncertain/nonlinear dynamical systems. Automatica, 28(2):289–298, 1992.

[148] S. Hui and S. H. Żak. Low-order state estimators and compensators for dynamical systems with unknown inputs. Systems & Control Letters, 21(6):493–502, December 1993.

[149] S. Hui and S. H. Żak. Robust output feedback stabilization of uncertain dynamic systems with bounded controllers. Int. J. of Robust and Nonlinear Control, 3(2):115– 132, August 1993.

[150] S. Hui and S. H. Żak. The Widrow-Hoff algorithm for McCulloch-Pitts type neurons. IEEE Transactions on Neural Networks, 5(6):924–929, November 1994.

[151] S. Hui and S. H. Żak. Observer design for systems with unknown inputs. Int. J. Appl. Math. Comput. Sci, 15(4):431–446, 2005.

[152] J. Y. Hung, W. Gao, and J. C. Hung. Variable structure control: A survey. IEEE Transactions on Industrial Electronics, 40(1):2–22, February 1993.

[153] L. R. Hunt, R. Su, and G. Meyer. Global transformations of nonlinear systems. IEEE Transactions on Automatic Control, AC-28(1):24–31, January 1983.

[154] D. R. Hush and B. G. Horne. Progress in supervised neural networks: What’s new since Lippmann? IEEE Signal Processing Magazine, 10(1):8–39, January 1993.

[155] H. S. Hwang. Automatic design of fuzzy rule base for modelling and control using evolutionary programming. IEE Proceedings—Control Theory Appl., 146(1):9–16, January 1999.

[156] P. Ignaciuk and A. Bartoszewicz. Discrete-time sliding-mode congestion control in multisource communication networks with time-varying delay. IEEE Transactions on Control Systems Technology, 19(4):852–867, July 2011.

[157] A. Isidori. Nonlinear Control Systems. Springer-Verlag, London, third edition, 1995.

[158] A. Isidori and M. D. Di Benedetto. Feedback linearization of nonlinear systems. In W. S. Levine, editor, The Control Handbook: Control System Advanced Methods, chapter 46, pages 46–1 – 46–15. CRC Press, Boca Raton, Florida, 2011.

[159] U. Itkis. Control Systems of Variable Structure. A Halsted Press Book, a division of John Wiley & Sons, New York, 1976.

[160] M. Jamshidi, M. Tarokh, and B. Shafai. Computer-Aided Analysis and Design of Linear Control Systems. Prentice Hall, Englewood Cliffs, New Jersey 07632, 1992.

[161] J.-S. R. Jang, C.-T. Sun, and E. Mizutani. Neuro-Fuzzy and Soft Engineering: A Computational Approach to Learning and Machine Intelligence. MATLAB Curriculum Series. Prentice-Hall, Upper Saddle River, NJ 07458, 1997.

[162] B. Jiang, M. Staroswiecki, and V. Cocquempot. Fault estimation in nonlinear uncertain systems using robust/sliding-mode observers. IEE Proceedings—Control Theory Appl., 151(1):29–37, February 2004.

[163] G.-P. Jiang, W. K.-S. Tang, and G. Chen. A state-observer-based approach for synchronization in complex dynamical networks. IEEE Transactions on Circuits and Systems—I, 53(12):2739–2745, December 2006.

[164] T. Kaczorek. Teoria Sterowania i Systemów. Wydawnictwo Naukowe PWN—Polish Scientific Publishers, Warszawa, 1993.

[165] T. Kaczorek. Wektory i Macierze w Automatyce i Elektrotechnice. Wydawnictwa Naukowe-Techniczne—Scientific-Technical Publishers, Warszawa, second edition, 1998.

[166] T. Kaczorek, A. Dzieliński, W. Da̧browski, and R. Łopatka. Podstawy Teorii Sterowania. Wydawnictwa Naukowe-Techniczne—Scientific-Technical Publishers, Warszawa, 2005.

[167] P. B. Kahn. Mathematical Methods for Scientists and Engineers: Linear and Nonlinear Systems. Wiley-Interscience, J. Wiley & Sons, New York, 1990.

[168] T. Kailath. Linear Systems. Prentice Hall, Englewood Cliffs, New Jersey 07632, 1980.

[169] R. E. Kalman. Mathematical description of linear dynamical systems. J. SIAM Control, Ser. A, 1(2):152–192, 1963.

[170] R. E. Kalman. On the computation of the reachable/observable canonical form. SIAM J. Control and Optimization, 20(2):258–260, March 1982.

[171] K. Kalsi, J. Lian, S. Hui, and S. H. Żak. Sliding-mode observers for systems with unknown inputs: A high-gain approach. Automatica, 46(2):347–353, February 2010.

[172] K. Kalsi, J. Lian, and S. H. Żak. Decentralized dynamic output feedback control of nonlinear interconnected systems. IEEE Transactions on Automatic Control, 55(8):1964– 1970, August 2010.

[173] E. W. Kamen. Fundamentals of linear time-varying systems. In W. S. Levine, editor, The Control Handbook: Control System Advanced Methods, chapter 3, pages 3–1 – 3–33. CRC Press, Boca Raton, Florida, 2011.

[174] S.-J. Kang, C.-H. Woo, H.-S. Hwang, and K. B. Woo. Evolutionary design of fuzzy rule base for nonlinear system modeling and control. IEEE Transactions on Fuzzy Systems, 8(1):37–45, February 2000.

[175] K. Kelly. Out of Control: The New Biology of Machines, Social Systems and the Economic World. Addison-Wesley Publishing Company, Reading, Massachusetts, 1994.

[176] T. Kenjo. Stepping Motors and Their Microprocessor Controls. Clarendon Press, Oxford, 1984.

[177] T. Kenjo and S. Nagamori. Permanent-Magnet and Brushless DC Motors. Clarendon Press, Oxford, 1985.

[178] H. K. Khalil. Nonlinear Systems. Prentice Hall, Upper Saddle River, NJ 07458, second edition, 1996.

[179] T. Khanna. Foundations of Neural Networks. Addison-Wesley Publishing, Reading, MA, 1990.

[180] E. Kim, M. Park, S. Ji, and M. Park. A new approach to fuzzy modeling. IEEE Transactions on Fuzzy Systems, 5(3):328–337, August 1997.

[181] D. E. Kirk. Optimal Control Theory: An Introduction. Prentice-Hall Inc., Englewood Cliffs, N.J., 1970.

[182] G. J. Klir, U. St.Clair, and B. Yuan. Fuzzy Set Theory: Foundations and Applications. Prentice Hall PTR, Upper Saddle River, NJ 07458, 1997.

[183] C. Knospe. PID control—Introduction to the special section. IEEE Conrol Systems Magazine, 26(1):30–31, February 2006.

[184] D. E. Knuth. The Art of Computer Programming, volume 1. Fundamental Algorithms. Addison-Wesley, Reading, Massachusetts, second edition, 1973.

[185] C. Koch. Computation and the single neuron. Nature, 385(6613):207–210, 16 January 1997.

[186] A. J. Koshkouei and A. S. I. Zinober. Sliding mode state observation for non-linear systems. Int. J. Control, 77(2):118–127, 2004.

[187] B. Kosko. Fuzzy Engineering. Prentice-Hall, Upper Saddle River, NJ 07458, 1997.

[188] B. Kouvaritakis and M. Cannon, editors. Nonlinear Predictive Control: Theory and Practice. The Institution of Electrical Engineers, London, 2001.

[189] J. R. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection. A Bradford Book, The MIT Press, Cambridge, Massachusetts, 1992.

[190] J. R. Koza, F. H. Bennett, III, D. Andre, M. A. Keane, and F. Dunlap. Automated synthesis of analog electrical circuits by means of genetic programming. IEEE Transactions on Evolutionary Computation, 1(2):109–128, July 1997.

[191] T. Kozek, T. Roska, and L. O. Chua. Genetic algorithm for CNN template learning. IEEE Transactions on Circuits and Systems—I: Fundamental Theory and Applications, 40(6):392–402, June 1993.

[192] P. C. Krause and O. Wasynczuk. Electromechanical Motion Devices. McGraw-Hill Book Company, New York, 1989.

[193] A. J. Krener and W. Respondek. Nonlinear observers with linearizable error dynamics. SIAM J. Control and Optimization, 23(2):197–216, March 1985.

[194] H. Kroemer. Nano-whatever: Do we really know where we are heading? Physica Status Solidi (a), 202(6):957–964, May 2005.

[195] M. Krstić, I. Kanellakopoulos, and P. V. Kokotović. Nonlinear and Adaptive Control Design. John Wiley & Sons, Inc., New York, 1995.

[196] V. Kučera. A contribution to matrix quadratic equations. IEEE Transactions on Automatic Control, AC-17(3):344–347, June 1972.

[197] P. Kudva, N. Viswanadham, and A. Ramakrishna. Observers for linear systems with unknown inputs. IEEE Transactions on Automatic Control, AC-25(1):113–115, February 1980.

[198] J. E. Kurek. The state vector reconstruction for linear systems with unknown inputs. IEEE Transactions on Automatic Control, AC-28(12):1120–1122, December 1983.

[199] H. Kwakernaak and R. Sivan. Linear Optimal Control Systems. Wiley-Interscience, a Division of J. Wiley & Sons, Inc., New York, 1972.

[200] C. M. Kwan. On variable structure output feedback controllers. IEEE Transactions on Automatic Control, 41(11):1691–1693, November 1996.

[201] V. Kyrylov, L. A. Severyanova, and A. Vieira. Modeling robust oscillatory behavior of the hypothalamic-pituitary-adrenal axis. IEEE Transactions on Biomedical Engineering, 52(12):1977–1983, December 2005.

[202] B. Lamb. Booknotes: America’s Finest Authors on Reading, Writing, and the Power of Ideas. Times Books, New York, 1997.

[203] S. Lang. Calculus of Several Variables. Springer-Verlag, New York, third edition, 1987.

[204] J. P. LaSalle. The Stability and Control of Discrete Processes. Springer-Verlag, New York, 1986.

[205] E. B. Lee and L. Markus. Foundations of Optimal Control Theory. Robert E. Krieger Publishing Company, Malabar, Florida, 1986.

[206] Y. Lee, J. Q. Gong, B. Yao, and S. H. Żak. Fuzzy adaptive robust tracking control of a class of nonlinear systems. In Proc. 2001 American Control Conf., pages 4040–4045, 2001. Arlington, VA, June 25–27, 2001.

[207] Y. Lee and S. H. Żak. Designing a genetic neural fuzzy antilock-brake-system controller. IEEE Transactions on Evolutionary Computation, 6(2):198–211, April 2002.

[208] J. R. Leigh. Modelling and Simulation. IEE Topics in Control Series 1. Peter Peregrinus Ltd., London, 1983.

[209] Y. Lenbury and P. Pornsawad. A delay-differential equation model of the feedbackcontrolled hypothalamus-pituitary-adrenal axis in humans. Mathematical Medicine and Biology, 22(1):15–33, March 2005.

[210] F. L. Lewis and V. L. Syrmos. Optimal Control. A Wiley-Interscience Publication, John Wiley & Sons, Inc., New York, second edition, 1995.

[211] Y. Li, K. H. Ang, and G. C. Y. Chong. PID control system analysis and design: Problems, remedies, and future directions. IEEE Conrol Systems Magazine, 26(1):32– 41, February 2006.

[212] J. Lian, Y. Lee, S. D. Sudhoff, and S. H. Żak. Self-organizing radial basis function network for real-time approximation of continuous-time dynamical systems. IEEE Transactions on Neural Networks, 19(3):460–474, March 2008.

[213] J. Lian and S. H. Żak. Controller design for a class of uncertain systems with guaranteed performance. SIAM J. Control and Optimization, 49(3):1239–1261, May 2011.

[214] K.-Y. Lian, C.-S. Chiu, T.-S. Chiang, and P. Liu. Secure communications of chaotic systems with robust performance via fuzzy observer-based design. IEEE Transactions on Fuzzy Systems, 9(1):212–220, February 2001.

[215] C.-T. Lin and C. S. G. Lee. Neural Fuzzy Systems: A Neuro-Fuzzy Synergism to Intelligent Systems. Prentice-Hall PTR, Upper Saddle River, NJ 07458, 1996.

[216] J.-S. Lin, T.-L. Liu, J.-J. Yan, and H.-T. Yau. Synchronization of unidirectional coupled chaotic systems with unknown channel time-delay: Adaptive robust observerbased appraoch. Chaos, Solitons and Fractals, 26:971–978, November 2005.

[217] R. P. Lippmann. An introductuion to computing with neural nets. IEEE ASSP Magazine, 4(2):4–22, April 1987.

[218] G. P. Liu, V. Kadirkamanathan, and S. A. Billings. Stable sequential identification of continuous nonlinear dynamical systems by growing radial basis function networks. Int. J. Control, 65(1):53–69, 10 September 1996.

[219] Y.-W. Liu, Z.-H. Hu, J.-H. Peng, and B.-Z. Liu. A dynamical model for the pulsatile secretion of the hypothalamo-pituitary-adrenal axis. Mathematical and Computer Modeling, 29(4):103–110, February 1999.

[220] B. P. Loop, S. D. Sudhoff, S. H. Żak, and E. L. Zivi. Estimating regions of asymptotic stability of power electronics systems using genetic algorithms. IEEE Transactions on Control Systems Technology, 18(5):1011–1022, September 2010.

[221] D. G. Luenberger. Observers for multivariable systems. IEEE Transactions on Automatic Control, AC-11(2):190–197, April 1966.

[222] D. G. Luenberger. An introduction to observers. IEEE Transactions on Automatic Control, AC-16(6):596–602, December 1971.

[223] D. G. Luenberger. Introduction to Dynamic Systems: Theory, Models, and Applications. John Wiley & Sons, New York, 1979.

[224] D. G. Luenberger. Linear and Nonlinear Programming. Addison-Wesley Publishing, Reading, Massachusetts, second edition, 1984.

[225] X.-J. Ma, Z.-Q. Sun, and Y.-Y. He. Analysis and design of fuzzy controller and fuzzy observer. IEEE Transactions on Fuzzy Systems, 6(1):41–51, February 1998.

[226] A. G. J. MacFarlane. The development of frequency-response methods in automatic control. IEEE Transactions on Automatic Control, AC-24(2):250–265, April 1979.

[227] I. J. Maddox. Elements of Functional Analysis. Cambridge University Press, Cambridge, Great Britain, second edition, 1988.

[228] E. H. Mamdani and S. Assilian. An experiment in linguistic synthesis with a fuzzy logic controller. Int. J. Man-Machine Studies, 7(1):1–13, January 1975.

[229] B. B. Mandelbrot. The Fractal Geometry of Nature. W. H. Freeman and Company, New York, 1983.

[230] R. Marino and P. Tomei. Nonlinear Control Design: Geometric, Adaptive and Robust. Prentice Hall Information and System Sciences Series. Prentice Hall Europe, London, 1995.

[231] J. C. Maxwell. On governors. Proceedings of the Royal Society of London, XVI:270–283, March 1868. Printed by Taylor and Francis, London, Red Lion Court, Fleet Street.

[232] R. M. May. Simple mathematical models with very complicated dynamics. Nature, 261(5560):459–467, June 10, 1976.

[233] D. Q. Mayne and H. Michalska. Receding horizon control of nonlinear systems. IEEE Transactions on Automatic Control, 35(7):814–824, July 1990.

[234] D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. M. Scokaert. Constrained model predictive control: Stability and optimality. Automatica, 36(6):789–814, June 2000.

[235] O. Mayr. The Origins of Feedback Control. The M.I.T. Press, Cambridge, Massachusetts, 1970.

[236] W. S. McCulloch and W. Pitts. A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5:115–133, 1943.

[237] B. McEwen with E. Norton Lashley. The End of Stress as We Know It. Joseph Henry Press, Washington, DC, 2002.

[238] M. McIntyre, T. Burg, D. Dawson, and B. Xian. Adaptive state of charge (SOC) estimator for a battery. In Proceedings of the 2006 American Control Conference, Minneapolis, Minnesota, pages 5740–5744, June 14–16, 2006.

[239] J. M. Mendel. Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions. Prentice Hall PTR, Upper Saddle River, NJ 07458, 2001.

[240] H. Michalska and D. Q. Mayne. Moving horizon observers and observer-based control. IEEE Transactions on Automatic Control, 40(6):995–1006, June 1995.

[241] A. N. Michel, J. A. Farrell, and W. Porod. Qualitative analysis of neural networks. IEEE Transactions on Circuits and Systems, 36(2):229–243, February 1989.

[242] R. K. Miller and A. N. Michel. Ordinary Differential Equations. Academic Press, Inc., New York, 1982.

[243] M. Mitchell. An Introduction to Genetic Algorithms. A Bradford Book, The MIT Press, Cambridge, Massachusetts, 1996.

[244] R. R. Mohler. Nonlinear Systems; Volume I, Dynamics and Control. Prentice Hall, Englewood Cliffs, New Jersey 07632, 1991.

[245] C. Moler and C. Van Loan. Nineteen dubious ways to compute the exponential of a matrix. SIAM Review, 20(4):801–836, October 1978.

[246] F. C. Moon. Chaotic Vibrations: An Introduction for Applied Scientists and Engineers. Wiley-Interscience, J. Wiley & Sons, New York, 1987.

[247] Ö. Morgül and E. Solak. Observer based synchronization of chaotic systems. Physical Review E, 54(5):4803–4811, November 1996.

[248] A. G. Nalecz. Sensitivity analysis of vehicle design attributes in frequency domain. Vehicle System Dynamics, 17:141–163, 1988.

[249] A. G. Nalecz and A. C. Bindemann. Analysis of the dynamic response of four wheel steering vehicles at high speed. Int. J. of Vehicle Design, 9(2):179–202, 1988.

[250] A. G. Nalecz and A. C. Bindemann. Investigation into the stability of four wheel steering vehicles. Int. J. of Vehicle Design, 9(2):159–178, 1988.

[251] S. Nara, P. Davis, and H. Totsuji. Memory search using complex dynamics in a recurrent neural network model. Neural Networks, 6(7):963–973, 1993.

[252] S. G. Nash and A. Sofer. Linear and Nonlinear Programming. McGraw-Hill, New York, 1996.

[253] I. P. Natanson. Theory of Functions of a Real Variable. Frederick Ungar, New York, 1955.

[254] Nusawardhana, S. H. Żak, and W. A. Crossley. Nonlinear synergetic optimal control. AIAA Journal of Guidance, Control, and Dynamics, 30(4):1134–1147, July-August 2007.

[255] K. Ogata. Modern Control Engineering. Prentice Hall, Upper Saddle River, New Jersey 07458, third edition, 1997.

[256] K. Ogata. Solutions Manual to Modern Control Engineering. Prentice Hall, Upper Saddle River, New Jersey 07458, third edition, 1997.

[257] K. Ogata. Modern Control Engineering. Prentice Hall, Upper Saddle River, New Jersey 07458, fourth edition, 2002.

[258] J. O’Reilly. Observers for Linear Systems. Academic Press, London, 1983.

[259] E. Ott, C. Grebogi, and J. A. Yorke. Controlling chaos. Physical Review Letters, 64(11):1196–1199, 12 March 1990.

[260] D. H. Owens. Multivariable and Optimal Systems. Academic Press, London, 1981.

[261] S. K. Pal and D. Bhandari. Genetic algorithms with fuzzy fitness function for object extraction using cellular networks. Fuzzy Sets and Systems, 65(2/3):129–139, August 10, 1994.

[262] S. Pang, J. Farrell, J. Du, and M. Barth. Battery state-of-charge estimation. In Proceedings of the 2001 American Control Conference, Arlington, VA, pages 1644– 1649, June 25–27, 2001.

[263] T. S. Parker and L. O. Chua. Chaos: A tutorial for engineers. Proceedings of the IEEE, 75(8):982–1008, August 1987.

[264] T. S. Parker and L. O. Chua. Practical Numerical Algorithms for Chaotic Systems. Springer-Verlag, New York, 1989.

[265] R. V. Patel and M. Toda. Quantitative measures of robustness for multivariable systems. In Proceedings of Joint Automatic Control Conference (JACC), San Francisco, CA, pages TP8–A, 1980.

[266] R. J. Patton and G. P. Liu. Robust control design via eigenstructure assignment, genetic algorithms and gradient-based optimization. IEE Proceedings—Control Theory Appl., 141(3):202–208, May 1994.

[267] L. M. Pecora and T. L. Carroll. Synchronization in chaotic systems. Physical Review Letters, 64(8):821–824, 19 February 1990.

[268] W. Pedrycz. Fuzzy Control and Fuzzy Sytems. Research Studies Press Ltd., Taunton, Somerset, England, second, extended edition, 1993.

[269] W. Pedrycz, editor. Fuzzy Modeling: Paradigms and Practice. Kluwer Academic Publishers, Boston, 1996.

[270] H.-O. Peitgen, H. Jürgens, and D. Saupe, editors. Chaos and Fractals: New Frontiers of Science. Springer-Verlag, New York, 1992.

[271] H.-O. Peitgen and D. Saupe, editors. The Science of Fractal Images. Springer-Verlag, New York, 1988.

[272] A. L. Peressini, F. E. Sullivan, and J. J. Uhl, Jr. The Mathematics of Nonlinear Programming. Springer-Verlag, New York, 1988.

[273] I. Peterson. Newton’s Clock: Chaos in the Solar System. W. H. Freeman and Co., New York, 1993.

[274] C. A. Pickover, editor. Fractal Horizons: The Future Use of Fractals. St. Martin’s Press, New York, 1996.

[275] D. A. Pierre. Optimization Theory With Applications. Dover Publications, Inc., New York, 1986.

[276] L. S. Pontryagin. Ordinary Differential Equations. Addison-Wesley Publishing Company, Inc., Reading, Massachusetts, 1962.

[277] V. M. Popov. Invariant description of linear, time-invariant controllable systems. SIAM J. Control, 10(2):252–264, May 1972.

[278] H. M. Power and R. J. Simpson. Introduction to Dynamics and Control. McGraw-Hill Book Company (UK) Limited, London, 1978.

[279] R. Pratap. Getting Started with MATLAB 7: A Quick Introduction for Scientists and Engineers. Oxford University Press, New York, 2006.

[280] F. H. Raven. Automatic Control Engineering. The McGraw-Hill Book Company, New York, fourth edition, 1987.

[281] Reader’s Digest Practical Problem Solver. Substitutes, Shortcuts, and Ingenious Solutions for Making Life Easier. The Reader’s Digest Association, Inc., Pleasantville, New York, 1991.

[282] K. Röbenack and A. F. Lynch. An efficient method for observer design with approximately linear error dynamics. Int. J. Control, 77(7):607–612, 10 May 2004.

[283] C. E. Rohrs, J. L. Melsa, and D. G. Schultz. Linear Control Systems. McGraw-Hill, Inc., New York, 1993.

[284] B. Ronen and S. Pass. Focused Operations Management: Achieving More With Existing Resources. John Wiley & Sons, Hoboken, New Jersey, 2008.

[285] W. Rudin. Principles of Mathematical Analysis. McGraw-Hill, Inc., New York, third edition, 1976.

[286] G. Rudolph. Convergence analysis of canonical genetic algorithms. IEEE Transactions on Neural Networks, 5(1):96–101, January 1994.

[287] W. J. Rugh. Linear System Theory. Prentice Hall, Englewood Cliffs, New Jersey 07632, 1993.

[288] W. J. Rugh. Linear System Theory. Prentice Hall, Upper Saddle River, New Jersey 07458, second edition, 1996.

[289] W. J. Rugh and G. J. Murphy. A new approach to the solution of linear optimal control problems. Transactions of the ASME: Journal of Basic Engineering, 91(2):149–154, June 1969.

[290] D. E. Rumelhart, J. L. McClelland, and the PDP Research Group. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, volume 1. A Bradford Book, The MIT Press, Cambridge, Massachusetts, 1986.

[291] M. Saif and Y. Xiong. Sliding mode observers and their application in fault diagnosis. In F. Caccavale and L. Villani, editors, Fault Diagnosis and Fault Tolerance for Mechatronic Systems, pages 1–57. Springer-Verlag, Berlin, 2003.

[292] M. G. Salvadori and M. L. Baron. Numerical Methods in Engineering. Prentice-Hall, Inc., Englewood Cliffs, N.J., second edition, 1961.

[293] J. T. Sandefur. Discrete Dynamical Systems; Theory and Applications. Clarendon Press, Oxford, 1990.

[294] E. Santi, A. Monti, D. Li, K. Proddutur, and R.A. Dougal. Synergetic control for DC-DC boost converter: Implementation options. IEEE Transactions on Industry Applications, 39(6):1803–1813, November/December 2003.

[295] E. Santi, A. Monti, D. Li, K. Proddutur, and R.A. Dougal. Synergetic Control for Power Electronics Applications: A Comparison with the Sliding Mode Approach. Journal of Circuits, Systems, and Computers, 13(4):737–760, 2004.

[296] D. Sbarbaro and R. Peña. A non-linear wind velocity observer for a small wind energy system. In Proceedings of the 39-th IEEE Conference on Decsions and Control, pages 3086–3087, Sydney, Australia, December 2000.

[297] R. T. Seeley. Calculus of Several Variables: An Introduction. Scott, Foresman and Company, Glenview, Illinois 60025, 1970.

[298] G. M. Shepherd, editor. The Synaptic Organization of the Brain. Oxford University Press, New York, third edition, 1990.

[299] K.-Y. Siu, V. Roychowdhury, and T. Kailath. Discrete Neural Computation; A Theoretical Fmundation. Prentice Hall PTR, Englewood Cliffs, NJ 07632, 1995.

[300] R. E. Skelton. Dynamic Systems Control: Linear Systems Analysis and Synthesis. John Wiley & Sons, Inc., New York, 1988.

[301] J.-J. E. Slotine and W. Li. Applied Nonlinear Control. Prentice Hall, Englewood Cliffs, New Jersey 07632, 1991.

[302] D. E. Smith and J. M. Starkey. Effects of model complexity on the performance of automated vehicle steering controllers: controller development and evaluation. Vehicle System Dynamics, 23(8):627–645, November 1994.

[303] D. E. Smith and J. M. Starkey. Effects of model complexity on the performance of automated vehicle steering controllers: model development, validation and comparison. Vehicle System Dynamics, 24(2):163–181, March 1995.

[304] L. Smith. Linear Algebra. Springer-Verlag, New York, second edition, 1984.

[305] M. C. Smith. Special issue on control education—The United Kingdom. IEEE Control Systems, 16(2):51–56, April 1996.

[306] W. E. Snyder. Industrial Robots: Computer Interfacing and Control. Prentice Hall, Inc., Englewood Cliffs, New Jersey 07632, 1985.

[307] O. A. Solheim. Design of optimal control systems with prescribed eigenvalues. Int. J. Control, 15(1):143–160, 1972.

[308] J. A. Solsona and M. I. Valla. Disturbance and nonlinear Luenberger observers for estimating mechanical variables in permanent magnet synchronous motors under mechanical parameters uncertainties. IEEE Transactions on Industrial Electronics, 50(4):717– 725, August 2003.

[309] R. Sommer. Control design for multivariable non-linear time-varying systems. Int. J. Control, 31(5):883–891, 1980.

[310] E. D. Sontag. Mathematical Control Theory: Deterministic Finite Dimensional Systems. Springer-Verlag, New York, second edition, 1998.

[311] R. Su. On the linear equivalents of nonlinear systems. Systems & Control Letters, 2(1):48–52, July 1982.

[312] M. Sugeno and G. T. Kang. Structure identification of fuzzy model. Fuzzy Sets and Systems, 28(1):15–33, October 1988.

[313] K. K. Sundareswaran, P. J. McLane, and M. M. Bayoumi. Observers for linear systems with arbitrary plant disturbances. IEEE Transactions on Automatic Control, AC-22(5):870–871, October 1977.

[314] M. Sunwoo and K. C. Cheok. An application of explicit self-tuning controller to vehicle active suspension systems. In Proceedings of the 29th Conference on Decision and Control, Honolulu, Hawaii, pages 2251–2257, December 1990.

[315] H. J. Sussmann and J. C. Willems. 300 years of optimal control: From the brachystochrone to the maximum principle. IEEE Control Systems, 17(3):32–44, June 1997.

[316] K. Takaba. Robust servomechanism with preview action for polytopic uncertain systems. Int. J. Robust and Nonlinear Control, 10(2):101–111, February 2000.

[317] T. Takagi and M. Sugeno. Fuzzy identification of systems and its applications to modeling and control. IEEE Transactions on Systems, Man, and Cybernetics, 15:116– 132, January/February 1985.

[318] C. P. Tan, F. Crusca, and M. Aldeen. Extended results on robust state estimation and fault detection. Automatica, 44(8):2027–2033, August 2008.

[319] C. P. Tan and C. Edwards. Sliding mode observers for robust detection and reconstruction of actuator and sensor faults. Int. J. Robust and Nonlinear Control, 13(5):443–463, 30 April 2003.

[320] C. P. Tan and C. Edwards. Robust fault reconstruction in uncertain linear systems using multiple sliding observers in cascade. IEEE Transactions on Automatic Control, 55(4):855–867, April 2010.

[321] K. Tanaka. Stability and stabilizability of fuzzy-neural-linear control systems. IEEE Transactions on Fuzzy Systems, 3(4):438–447, November 1995.

[322] K. Tanaka and M. Sugeno. Stability analysis and design of fuzzy control systems. Fuzzy Sets and Systems, 45(2):135–156, January 24, 1992.

[323] Y. S. Tang, A. I. Mees, and L. O. Chua. Synchronization and chaos. IEEE Transactions on Circuits and Systems, CAS-30(9):620–626, September 1983.

[324] D. W. Tank and J. J. Hopfield. Simple ’neural’ optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit. IEEE Transactions on Circuits and Systems, CAS-36(5):533–541, 1986.

[325] M. C. M. Teixeira, E. Assunção, and R. G. Avellar. On relaxed LMI-based designs for fuzzy regulators and fuzzy observers. IEEE Transactions on Fuzzy Systems, 11(5):613– 623, October 2003.

[326] M. C. M. Teixeira and S. H. Żak. Stabilizing controller design for uncertain nonlinear systems using fuzzy models. IEEE Transactions on Fuzzy Systems, 7(2):133–142, April 1999.

[327] J. M. T. Thompson and H. B. Stewart. Nonlinear Dynamics and Chaos. John Wiley and Sons, Chichester, 1986.

[328] Y. I. Toptscheev and A. P. Tsyplyakov. Problem Solver in the Theory of Automatic Control. Mashinostroyeniee, Moscow, 1977. In Russian.

[329] J. G. Truxal. Introductory System Engineering. McGraw-Hill Book Company, New York, 1972.

[330] V. Utkin, J. Guldner, and J. Shi. Sliding Mode Control in Electromechanical Systems. Taylor & Francis, London, 1999.

[331] V. I. Utkin. Variable structure systems with sliding modes. IEEE Transactions on Automatic Control, AC-22(2):212–222, April 1977.

[332] V. I. Utkin. Sliding Modes and Their Application in Variable Structure Systems. Mir Publishers, Moscow, 1978.

[333] V. I. Utkin. Sliding Modes in Control and Optimization. Springer-Verlag Berlin, Heidelberg, 1992.

[334] L. Vandenberghe and S. Boyd. Semidefinite programming. SIAM Review, 38(1):49–95, March 1996.

[335] D. J. Velleman. How to Prove it: A Structured Approach. Cambridge University Press, Cambridge, 1994.

[336] M. Vidyasagar. Nonlinear Systems Analysis. Prentice Hall, Englewood Cliffs, New Jersey 07632, second edition, 1993.

[337] T. L. Vincent. Control using chaos. IEEE Control Systems, 17(6):65–76, December 1997.

[338] B. L. Walcott, M. J. Corless, and S. H. Żak. Comparative study of non-linear stateobservation techniques. Int. J. Control, 45(6):2109–2132, 1987.

[339] B. L. Walcott and S. H. Żak. State observation of nonlinear uncertain dynamical systems. IEEE Transactions on Automatic Control, AC-32(2):166–170, February 1987.

[340] B. L. Walcott and S. H. Żak. Combined observer-controller synthesis for uncertain dynamical systems with applications. IEEE Transactions on Systems, Man, and Cybernetics, 18(1):88–104, January/February 1988.

[341] H. O. Wang, K. Tanaka, and M. F. Griffin. An approach to fuzzy control of nonlinear systems: Stability and design issues. IEEE Transactions on Fuzzy Systems, 4(1):14–23, February 1996.

[342] L. Wang. Model Predictive Control System Design and Implementation Using MAT-LAB. Springer, London, 2009.

[343] L.-X. Wang. Stable adaptive fuzzy control of nonlinear systems. IEEE Transactions on Fuzzy Systems, 1(2):146–155, May 1993.

[344] L.-X. Wang. Adaptive Fuzzy Systems and Control; Design and Stability Analysis. PTR Prentice Hall, Englewood Cliffs, New Jersey 07632, 1994.

[345] L.-X. Wang. A Course in Fuzzy Systems and Control. Prentice Hall PTR, Upper Saddle River, NJ 07458, 1997.

[346] S.-H. Wang, E. J. Davison, and P. Dorato. Observing the states of systems with unmeasurable disturbances. IEEE Transactions on Automatic Control, AC-20(5):716– 717, October 1975.

[347] R. Weinstock. Calculus of Variations: With Applications to Physics and Engineering. Dover Publications, Inc., New York, 1974.

[348] P. Welander. Understanding derivative in PID control. Control Engineering, 57(2):24– 27, February 2010.

[349] J. R. Wertz, editor. Spacecraft Attitude Determination and Control. Astrophysics and Space Science Library, Vol. 73. Kluwer Academic Publishers, Dordrecht, The Netherlands, 1997.

[350] A. B. Will and S. H. Żak. Modeling and control of an automated vehicle. Vehicle System Dynamics, 27(3):131–155, March 1997.

[351] J. L. Willems. Stability Theory of Dynamical Systems. John Wiley & Sons, New York, 1970.

[352] W. A. Wolovich. Linear Multivariable Systems. Springer-Verlag, New York, 1974.

[353] W. A. Wolovich. Automatic Control Systems: Basic Analysis and Design. Saunders College Publishing, Harcourt Brace College Publishers, Fort Worth, 1994.

[354] J. Y. Wong. Theory of Ground Vehicles. J. Wiley & Sons, New York, second edition, 1993.

[355] W. M. Wonham. On pole assignment in multi-input controllable linear systems. IEEE Transactions on Automatic Control, AC-12(6):660–665, December 1967.

[356] J. Xiang, H. Su, and J. Chu. On the design of Walcott-Zak sliding mode observer. In Proceedings of the 2005 American Control Conference, Portland, OR, pages 2451–2456, June 8–10, 2005.

[357] X.-G. Yan and C. Edwards. Nonlinear robust fault reconstruction and estimation using a sliding mode observer. Automatica, 43(9):1605–1614, September 2007.

[358] X.-G. Yan and C. Edwards. Robust decentralized actuator fault detection and estimation for large-scale systems using a sliding mode observer. Int. J. Control, 81(4):591– 606, April 2008.

[359] X.-G. Yan and C. Edwards. Robust sliding mode observer-based actuator fault detection and isolation for a class of nonlinear systems. Int. J. Systems Science, 39(4):349– 359, April 2008.

[360] F. Yang and R. W. Wilde. Observers for linear systems with unknown inputs. IEEE Transactions on Automatic Control, 33(7):677–681, July 1988.

[361] T. Yang, C.-M. Yang, and L.-B. Yang. A detailed study of adaptive control of chaotic systems with unknown parameters. Dynamics and Control, 8(3):255–267, July 1998.

[362] Z.-J. Yang, T. Hachino, and T. Tsuji. On-line identification of continuous time-delay systems combining least-squares techniques with a genetic algorithm. Int. J. Control, 66(1):23–42, 10 January 1997.

[363] X. Yu. Controlling chaos using input-output linearization approach. Int. J. Bifurcation and Chaos, 7(7):1659–1664, July 1997.

[364] L. A. Zadeh. Fuzzy sets. Information and Control, 8(3):338–353, June 1965.

[365] S. H. Żak. Systems and Control. Oxford University Press, New York, 2003.

[366] S. H. Żak and S. Hui. On variable structure output feedback controllers for uncertain dynamic systems. IEEE Transactions on Automatic Control, 38(10):1509–1512, October 1993.

[367] S. H. Żak and S. Hui. Output feedback variable structure controllers and state estimators for uncertain/nonlinear dynamic systems. IEE Proceedings—Control Theory Appl., 140(1):41–50, January 1993.

[368] S. H. Żak and C. A. MacCarley. State-feedback control of non-linear systems. Int. J. Control, 43(5):1497–1514, 1986.

[369] S. H. Żak and B. L. Walcott. State observation of nonlinear control systems via the method of Lyapunov. In A. S. I. Zinober, editor, Deterministic Control of Uncertain Systems, chapter 16, pages 333–350. Peter Peregrinus Ltd., London, UK, 1990.

[370] S. H. Żak, B. L. Walcott, and S. Hui. Variable structure control and observation of nonlinear/uncertain systems. In K.-K. D. Young, editor, Variable Structure Control for Robotics and Aerospace Applications, chapter 4, pages 59–88. Elsevier, Amsterdam, 1993.

[371] M. Zamparelli. Genetically trained cellular neural networks. Neural Networks, 10(6):1143–1151, August 1997.

[372] M. Zeitz. Comments on ‘Comparative study of non-linear state-observation techniques’. Int. J. Control, 46(5):1861–1863, 1987.

[373] Y. Zeng and S. N. Singh. Adaptive control of chaos in Lorenz system. Dynamics and Control, 7(2):143–154, April 1997.

[374] J. G. Ziegler and N. B. Nichols. Optimum settings for automatic controllers. Transactions of the American Society of Mechanical Engineers, 64:759–768, November 1942.

[375] A. S. I. Zinober, editor. Deterministic Control of Uncertain Systems. Peter Peregrinus Ltd., London, 1990.

[376] M. Zribi and J. Chiasson. Position control of a PM stepper motor by exact linearization. IEEE Transactions on Automatic Control, 36(5):620–625, May 1991.

BIBLIOGRAPHY 1067 [377] J. M. Zurada. Introduction to Artificial Neural Systems. West Publishing, St. Paul,

MN, 1992.

Index

(n+1)-tuples, 447 1-norm, 960 2-norm, 338, see Euclidean norm

A/D conversion, see analog-to-digital conver-

sion A/D converter

2-bit, 852 dynamical behavior of, 852 higher order, 853

absolutely continuous, 986 absolutely continuous function, 545 accelerating potential, 208 Ackermann’s formula

for pole placement, 253 ACTH, see corticotropin releasing hormone activation function, 843

in Hopfield neural network, 846 of 2-bit A/D converter, 852 sigmoid, 814, 816 strictly increasing and differentiable, 846

active suspension, 410

model of, 111 actual state-feedback control, 269 actuating signal, 9, see error signal actuator authority, 754 actuators

with limited authority, 790 Adams Prize

Routh, 401 Adams-Bashforth

multi-step algorithms, 179 Adams-Moulton

multi-step algorithms, 179 adaptation

progressive modification of structures, 858 adaptation algorithm

for the identifier parameters, 818 adaptation law, 394, 773 adaptation parameter error, 773 adaptation parameters, 781 adaptive fuzzy logic control, 772 adaptive fuzzy robust

tracking controllers, 766 adaptive fuzzy term, 768 adaptive neural term

in stabilizing controller, 838 adding

radial-basis functions, 831 additivity property, 11, 17, 25 adjacent trajectory, 918 adjoint equation, see costate equation adjugate

transpose of the cofactor matrix, 967 adjustable parameters, 394

of neural identifier, 827 admissible control, 281, 493, 509 admissible controllers, 7 adrenal glands, 105 adrenocorticotropic hormone, 105 affine

rather than linear, 739 affine function, 415 affine matrix inequality, 415 affine model, 740 affine transformation, 912

contractive, 914 expansive, 914 symmetry, 914

algebraic

multiplicity, 136 algebraic equation

solving for equilibrium points, 188 algebraic multiplicity, 136

of an eigenvalue, 1033 algebraic product defuzzifier, 720, 792 algebraic Riccati equation, 525, 529, 531

discrete, 932 algebraic Riccati equation (ARE), 455 algorithm, 857

controller design for fuzzy systems, 794 for adapting weight vector, 804 is said to converge, 808

alleles

different forms of the same gene, 858 allelic genes, see alleles allostasis, 105 almost linear observation error dynamics, 655 amplitude

of limit cycle, 205

of the plate vibrations, 936 amplitude constraints, 493 analog input

to A/D converter, 851 analog-digital computer, 854 analog-to-digital (A/D) converter using Hopfield network, 850 analog-to-digital conversion, 850 angular displacement, 61 animation

robot manipulator, 291 using MATLAB, 797 animationhandle graphics, 290 annihilator

of a distribution, 584 anti-windup, 705 aperiodic solutions, 901 appendix

as a glossary of symbols, 955 notation, 955

approximate

the uncertainties, 838 approximate the derivative backward Euler, 170 forward Euler, 166

approximated by

fuzzy logic system, 781 approximating

discontinuous controller, 379 approximating discontinuous term, 655 approximation linear, 191 of discontinuous controller, 377 of nonlinear system, 363

approximation error, 838

ARE, see algebraic Riccati equation (ARE)

solving, 457 arithmetic assignment, 3 armature, 82 armature circuit, 222, 273, 466, 502 armature controlled

DC motor, 501 armature current, 96 armature inductance

elimination from model, 607 armature-controlled

DC motor, 223 artificial networks, 854 artificial neural networks, see neural networks

computing with, 853 artificial neurons, 801 asymptotes, see isocline

equations, 159 asymptotic observer, 260

for nonlinear system, 604 full-order, 287 gain matrix, 262

asymptotic stability global, 313 in the large, 313 Lyapunov equation, 325 necessity condition, 326 of a nonlinear system a local property,

476 of time-varying systems, 389 region of, 406 stability and convergence, 315 sufficiency condition, 326

asymptotic stability in the large

sufficient conditions for, 754 asymptotically stable, 324

equilibrium state, 364 in the large, 746 LTI system, 391

uniformly, 317 asymptotically stable equilibrium, 940 attracting

2-cycle, 899, 905 4-cycle, 901, 902, 905 period-2 cycle, 899 period-two cycle, 899

attracting fixed point, 893 attractive sliding surface, 772 attractive surface, 540 augmented state space, 769 augmented state vector, 562, 662 augmented system dynamics, 562 authority of actuators, 754 automated vehicle control, 872 automatic control, 71 automotive engineer, 112 automotive suspension, 409

model of, 111 auxiliary convex problem, 423 auxiliary outputs, 633 average contractivity

condition of IFS, 914 average fitness

of k-th generation, 866 average multiplicative rate

of separation, 917 averaging

the exponential growth, 917 axon, 799, 801

backlash, 95 backpropagation, 811

algorithm, 811, 818, 820

applied to a fuzzy logic system, 818 training algorithm, 814 backpropagation algorithm

consists of two passes, 811 descent gradient algorithm, 820

backward difference operator, 660 backward Euler, 170 backward pass, 486, 488, 489 ball

uniformly stable, 385, 411 ball map, 938

fixed points of, 939 modified, 941 periodic equilibrium solutions to, 938

Banach space, 1035 complete normed linear space, 994 of continuous functions, 438

bang-bang control, 510 Barbalat’s lemma, 391, 392, 770, 775, 779,

840 alternative form, 395

Barnsley fern, 915 basis

of a subspace, 991 basis function

adding to the RBF network, 832 center of, 824, 831 radial, 824

battery in discharging mode, 296 terminal voltage, 295

battery model, 293, 295 battery SOC

estimate, 295 evaluate, 295

Bellman’s lemma, 763 Bellman’s principle of optimality (PO), 485 Bellman-Gronwall lemma, 1002, 1003 Bendixson’s theorem, 161, 182 benefits of sliding

not affected by matched uncertainties, 546 order reduction, 546

Bessel inequality, 210, 1032 bias, see threshold BIBO stability, see bounded-input bounded-

output stability BIBO stable, 49, 404

definition of, 395 biconditional, 958 bicycle model, 76, see ground vehicle

of a vehicle, 115

of front-wheel-steering vehicle, 870 bifurcation

change in the qualitative behavior, 895 bifurcation diagram, 901, 951 binary encoding

phenotype space into a genotype space,

861 biological genetics, 857 biological neural network

model of, 843 block

diagonal matrix, 133 block diagonal matrix, 133 block diagram

algebra of, 688

of a closed-loop system, 688 block matrix, 36 Bolzano-Weierstrass theorem, 979, 981, 985 bounce heights

of the ball map, 938 bouncing ball, 937

on a vibrating plate, 935 boundary

of a set, 985 boundary condition, 498

DC motor, 512 satisfied, 506

boundary conditions

brachistochrone problem, 514, 516 boundary layer, 377

approximation, 388 thickness of, 380

boundary layer controller, 380 performance, 388 uniformly bounded, 381

boundary point, 985 bounded

equilibrium, 358 sequence, 977

bounded above

a set of real numbers, 973 bounded controller, 930 bounded signal

definition of, 395 bounded-input bounded-output (BIBO) sta-

bility, 395 brachistochrone

from Greek; shortest time, 435 brachistochrone problem, 435

generalized, 531 minimize time travel between two points,

435 minimum principle of Pontryagin, 531 of Johann Bernoulli’s , 512 posed and solved by, 435

brachystochrone problem, see brachistochrone

problem brain

natural information processing system, 799 brain modeling, 854 branches of bifurcation diagram produce period-2 cycles, 901 branches of bifurcation diagram branches produce period-2 cycles, 901

broad spectrum

noise-like signals, 943 butterfly’s wings

symbol for early activity in chaotic systems, 950

c2d

MATLAB’s function, 180 calculate over a finite horizon

future control sequence, 659 calculus

fundamental theorem of, 1027 calculus of variations, 433 origins of, 523 candidate minimizer, 995 candidate solution

stacked columns of the rules matrix, 887 canonical GA

classical GA, 858 canonical GAs

use binary representations, 868 canonical genetic algorithm (GA), 860 canonical representation of LMI, 419 Cantor

creator of the theory of sets, 956 Cantor set, 910

fractal dimension of, 910 Cantor’s ternary set, see Cantor set capacitor, 5, 280, 281

polarization, 294 carrying capacity, 103, 892 cart

with an inverted pendulum, 118 with inverted pendulum, 289

cart friction coefficient, 93 cart with an inverted pendulum, 91 Cartesian coordinates, 60 Cartesian product, 718 Cauchy sequence, 994, 999 Cauchy theorem, 979 Cauchy-Schwarz inequality, 809, 961, 962 causal system, 4, 19 Cayley and Hamilton

theorem of, 138 Cayley and Hamilton theorem, 138, 139 Cayley-Hamilton theorem, 141, 220, 242, 253 center average defuzzifier, 720, 873, 875 center of gravity (CG), 73 center-of-gravity defuzzifier, 720, 792 centrifugal force, 72 centrifugal governor, 71, see Watt’s governor,

279 centrifugal pendulum, 47 centrifugal speed regulator, 9, see Watt’s gov-

ernor centripetal torques, 561 centroid

of the support, 719 centroidal defuzzifier, 719 CFS, 105, see HPA axis CG, see center of gravity chain rule, 601, 1024 change of variables, 364 chaos

in biological neurons, 950 in the solar system, 949

chaos game, 952

chaotic

discrete-time dynamical systems, 949 chaotic algorithm, see chaotic control algo-

rithm, 930 Hénon map, 931

chaotic attractor, 927, 928, 951 of the ball map, 938 chaotic control algorithm, 927 Lyapunov function, 930

chaotic control law

to stabilize the closed-loop system, 953 chaotic gingerbreadman, 951 chaotic motion

aspect of randomness, 901

by repeated flip bifurcations, 902

defintion of, 919

Lyapunov exponent, 919 chaotic motion generation, 938 chaotic motions, see chaotic solutions chaotic signal

power spread across a broad range of fre-

quencies, 943 chaotic signals

characterized by spread spectrum, 943 chaotic solutions, 901 chaotic synchronization definition of, 943

chaotic system bouncing ball on a vibrating plate, 935 represented as a fuzzy system, 944

chaotic systems, 889

uncertainty growth exponential, 890 chaotic trajectories

for maps, 920 sensitive dependence on initial conditions,

915 characteristic equation, 201, 220, 500

roots of, 251 characteristic matrix, 131, 136 characteristic polynomial, 132, 145, 243, 262,

274, 345 desired, 276 of observer, 275

characteristic polynomial equation, 967 charge, 293

of the ion, 209 on the capacitor, 1014

charging resistance, 293 chattering, 377, 538

oscillations of trajectory about switching

surface, 538 chattering elimination, 389 chromosome fitness, 866 chromosome performance

its fitness, 866 chromosomes, 858

as strings of real parameters, 877 in canonical GA strings of binary digits,

859 chronic fatigue syndrome (CFS), 105 chronically stressed, 367 circadian production, 107 circuit

armature, 273 minimizing cost function, 850 state-space model, 281

transfer function, 281 circuit implementation

of the Hopfield network, 843 CLCP, see closed-loop characteristic polyno-

mial clockwise rotation, 95 closed ball

as compact set, 347 closed-loop, 9

characteristic equation, 251 characteristic polynomial, 251

closed-loop characteristic polynomial (CLCP),

689 closed-loop feedback, 47 closed-loop observer, see Luenberger observer

alternative design, 263

gain matrix, 262 closed-loop poles

desired, 251

zeros of the CLCP, 689 closed-loop system, 203, 251, 337 closed-loop system response, 269 closed-loop vehicle system, 872 clustering

of training patterns, 840 cobweb

a way of displaying a solution, 895 of attracting 4-cycle, 902

codistribution, 582

dual of distributions, 584 coefficient of restitution, 936 coercive, see radially unbounded cofactor

of a matrix, 966 collar of the governor, 72 column-by-column method, 41 combine controller

with full-order observer, 268 combined controller-observer

compensator, 250 combined controller-observer compensator, 269,

572, 579, 605, 611, 645, 654, 797 full-order, 276 full-order observer, 275 implementation, 606 one-link manipulator, 606 reduced-order observer, 278

combined fuzzy controller-observer compen-

sator, 762 common

symmetric positive definite P, 744 common P

symmetric positive definite, 749 common positive definite P, 427 communication channel, 941 commutative

scalar product, 959 commutative operation, 21 commutativity property, 26 commute, 300 compact

subset, 993 compact set, 347

closed and bounded, 985 comparison method

for linear systems, 762 comparison theorem, 357, 774, 778 compatible

system of linear equations, 991 compatible matrix norm, 988 compensator, 268, 270

combined controller-observer, 268, 270 competing species

population dynamics, 925 complement

of fuzzy sets, 717 complete integrability

of a distribution, 585 complete normed linear space, 994 completing the squares, 776 complex conjugate, 334 complex conjugate pairs

poles, 251 complex conjugate transpose

of matrix, 334 complex plane, 201 component

of x along y, 960 component-by-component

majorization, 763 components

of the vector, 958 composite maps, 902 computational efficiency, 179 computational energy function

of Hopfield neural network, 846 computing

using artificial neural networks, 853 computing the gradient, 1030 condition for the control optimality

brachistochrone problem, 514 conditional

combination of two statements into one

statement, 957 conflicting objectives

convex combination, 432 conjugate pair, 552 connected

open set, 1026 connected region

a rectangle, 1027 conservation of energy

law of, 109 conservation of momentum

principle of, 936 conservative field

brachistochrone problem, 513 conservative vector field, 109 consistent matrix norm, see compatible ma-

trix norm constant slope, 155 constant-parameter

methods, 207 constrained optimization problem

for constructing MPC, 670

RAS estimate, 480 constraint set, 678

convex, 414 constraints on actuator, 703 constraints on the predicted plant’s output

MPC, 669 constraints on the rate of change of the con-

trol MPC, 666

constructability, 234 constructing sliding controller

generalized Lyapunov approach, 548 continuity

local property, 390 continuous

set of functions, 194 continuous function, 305 on a compact set, 985 continuous functional, 436 continuous functions of time, 293 continuous on interval

continuous at each point, 390 continuous partial derivatives to any order, 579

continuous-time linear, 147

continuously differentiable, 893 continuously parameterized, 674 contraction map, 997 contraction map theorem, 307 contraction operator

unique fixed point, 998 contractive

affine transformation, 914 contraposition, 220, 221 proof by, 228

control admissible, 281, 493 piecewise continuous, 493 unconstrained, 481

control aim control not too large, 433 state “small, 433

control design

three steps, 579 control input

frequency of the plate vibrations, 935 control law, 217

performance of, 431 control laws

linguistic based, 713 control problem, 6

essential elements, 431 control sequence, 217 control strategy, 7

implementation of, 492 optimal, 431 transfers, 224

control surface, 873, 877, 883

of FLC, 728 control system, 6, 8

closed-loop, 9 open-loop, 8

controllability, 219 definition of, 217 invariant under, 287 matrix, 220

controllability index, 244 controllability indices, 246, 256 controllability matrix, 227, 241, 243, 591

in terms of Lie brackets, 581 of one-link robot, 589

controllable, 218 definition, 219 yet not reachable, 218

controllable and observable realization, 409 controllable system

minimum time regulator problem, 520 controllable target

chaotic control algorithm, 928 of the ball map, 941

controlled dynamical system, 146 controlled Hénon map, 933 controlled modified ball map, 941 controller, 6, 8

discontinuous, 377 optimal, 7, 453

controller companion form, 546 controller design

three phases of, 572 controller design for fuzzy systems algorithm,

794 controller form, 241, 243, 255, 374

nonlinear, 609, 610

poor numerical properties, 257 controller gains, 546 controller poles

dominate response, 269 controller tuning

selecting the controller’s parameters, 694 converge

algorithm, 808 convergence rate

of observer, 616 convergent matrix, 335 convergent sequence

is bounded, 978

sequence has a limit, 976 converging factor, 1005 converting

degrees into radians, 533 convex

function, 414 set, 414

convex constrained optimization, 742 convex optimization, 413 convex optimization problem, 414 convex sets

examples, 414 convolution, 20

commutative, 21 convolution integral, 20 coordinate system

of the Society of Automotive Engineers,

870 coordinate transformation, 650 coordinates

of the vector, 958 Coriolis torques, 561 cornering characteristics, 533 cornering stiffness coefficient, 77 corollary

direct consequence of a theorem or propo-

sition, 956 corrector, 172

n-dimensional, 173 corticotropin releasing hormone, 105 cortisol, 105, 107 cortisol concentration, 367 cost function, 818

modified, 850 cost functional, 7, see performance index costate equation, 509

DC motor, 512

minimum time regulator problem, 519 costate equations, 509 counter-clockwise rotation, 95 counterclockwise direction, 1022 covector field, 580 covector fields, 585 CRH, see corticotropin releasing hormone crisp set, see ordinary set critical gain

K_{cr}, 698 critical period P_{cr}, 698

critical point, 672, 673 crossing site, 863 crossover

one-point, 867 single-point, 875 crossover operator, 859, 877 multi-point, 863 two-point, 863 crossover probability, 863 crossover rate, 863, 875 cumulative fitness, 863 current

discharging, 294 curve

in n-dimensional space, 1021 parameterization of, 1022 parameterized, 1022

cyclotron, 208

D’Alembert’s equation, 62

damper

linear, 280 damping coefficient, 280 damping term

in logistic equation, 892 damping torque, 113 DC motor, 95, 119, 222, 272, 276 armature controlled, 501, 576 armature-controlled, 222, 223, 272, 510 field-controlled, 119

dead zone

describing function, 211 decision boundary, see separating hyperplane declare matrix variables

lmiterm, 420 decoding

from genotype space into phenotype space,

861 decomposition theorem

Kalman, 280 decreasing

monotonically, 320 sequence, 975

decrescent, 358 decrescent function, 353, 766

definition of, 351 dees

D-shaped objects, 208 defective, 136 defining length

distance between first and last fixed bits,

865 of schema, 865

deflector plate

cyclotron, 208 defuzzification

fuzzy reasoning, 719 defuzzifier

algebraic product, 720 center average, 720 center-of-gravity, 720 centroidal, 719 modified center average, 724

degree of denominator, 1012 of numerator, 1012

degree of freedom, 61 dendrites, 799 depressed cortisol, 369 derivative

approximate differentiator, 700 computing, 303 piecewise continuous, 199

derivative action

combats large transient error, 694 Derivative of Dh

with respect to the vector field, 582 derivative of a function

with respect to a vector field, 581 derivative of a vector field

with respect to a vector field, 580 derivative time constant

type A PID controller, 694 descent gradient, 841

algorithm, 820 descent gradient algorithm, 813

with a fixed step size, 812 describing function, 193

complex-valued function, 201 dead zone, 211 limiter, 211 relay, 205

describing function method

limit cycle existence, 162 design model, 7, 56, 110, 736

of one-link robot, 607 reduced-order, 119

design parameter matrix, 394 desired

closed-loop poles, 251 desired closed-loop

characteristic polynomial, 256 desired state trajectory, 374, 433 destabilizing effect

of increasing L_{a}, 608 detectability, 625 detectable

definition, 234 detectable pair, 459 determinant

of a square matrix, 966

properties of, 966 determination

of state vector, 230 deterministic nonlinear dynamical systems

with a random-like behavior, 889 diagonal form, 283 diagonal matrix, 342, 967 diagonalization method, 125 diffeomorphic

state variable transformation, 592 diffeomorphism, 586, 589, 590

global, 586 difference equation, 121, 487, 906 difference equation solving

using Z-transform, 1020 different eigenvalues

only sufficient for diagonalizability, 472 differentiable

continuously, 184 differentiable curve, 109, 674, 1024 differential equation, 281

nonlinear, 121 solution of, 1000 with discontinuous right-hand side, 541

differential equations nonlinear, 190 with discontinuous right-hand sides, 574

differential inequality, 1003 differentiator, 23 digital PI controller, 711 digital type B PIDF controller, 711 digital type C PIDF controller, 711 dimension

of a realization, 235 of a subspace, 991 of nonreachable, 228 of the von Koch curve, 910 dimension of a realization, 30 dimension of the eigenspace, 1033 diode

to model charging, 294 to model discharging, 294 Dirac function, see impulse function directions of the field, 157 disc

open unit, 1031 discharge, 293 discharging current, 294 discharging resistance, 294 disconnected fractal, 910 discontinuity

points of, 196

smoothing out, 645 discontinuity surface, see switching surface discontinuous controller, 377 approximation, 377 being approximated, 389 chattering, 377 smoothing out, 377

discontinuous term

approximation, 655 discrete

algebraic Riccati equation, 932 LQR design, 932 Lyapunov equation, 932, 940

discrete (LTI) systems, 333 discrete algebraic Riccati equation, 484, 493 discrete convolution, 26 discrete fuzzy models, 756 discrete linear plant, 481 discrete linear quadratic regulator, 490 discrete Lyapunov equation, 333, 334 discrete Lyapunov matrix equation, 333, 426 discrete model, 149 discrete models

from continuous models, 924 discrete MPC, 660 discrete system

stability of, 401 discrete transfer function, 27 discrete-MPC

subject to constraints on control, 680 discrete-time, 216

linear, 147

sliding-mode controllers, 574

systems, 141 discrete-time analog, 925 discrete-time approximation

of type A PIDF controller, 709 discrete-time system, 25, 407 discrete-time version

of the proportional term, 708 discretization

of type A PIDF controller, 708 discretization chaos, 924 discretized plant, 680 discriminant, 158 distance measure

to the sliding surface, 772 distinct eigenvalues, 124 distributed discrete-time system, 25 distributed system, 6, 22 distribution, 582

annihilator of, 584

complete integrability of, 585 involutive, 583 nonsingular, 583 distributive law, 964 distributivity

over vector addition, 959 disturbance, 111 disturbances

as unknown inputs, 613

uneven road, 409 divergent sequence

sequence has no limit, 976 division, 142 double pendulum, 97

chaotic behavior, 952 kinetic energy, 98 state-space model, 101

dual, 285 dual pair

observer design, 260 dual system, 248 duality, 247 dynamic programming, 485, 525 continuous form of, 493, 496

dynamical system, 5 controlled, 146 dynamical systems

multi-input multi-state, 835 multivariable, 835 dynamics

of the estimation error, 946 dysregulation

of the HPA axis, 105

EA, see evolutionary algorithm, see evolu-

tionary algorithm (EA) simultaneously operating on fuzzy rules

and membership functions, 879 EA aided FLC design, 872 EDVAC, see Electronic Discrete Variable Au-

tomatic Computer Edwards and Spurgeon UIO, 649 Edwards-Spurgeon canonical form, 652 Edwards-Spurgeon UIO, 652 effect of the nonlinearity, 207 eigenspace

dimension of, 1033 eigenvalue, 121, 967 eigenvalue placement, 267 eigenvalue-eigenvector

equation, 125 eigenvalues

on imaginary axis, 364 distinct, 124 repeated, 126

eigenvector, 121, 125, 134, 967

generalized, 127 eigenvector method

can be generalized, 460 eigenvector method for solving ARE, 457 eigenvector test, see Popov-Belevitch-Hautus

test eigenvectors

linearly independent, 463 electric charge, 61 electric drive system

simplified model, 780 electric oscillator, 208 electric power

transformed into mechanical power, 90 electric vehicle, 295 electro-chemical

reaction, 294 electromagnet, 83 electromechanical motion device, see perma-

nent magnet (PM) stepper motor Electronic Discrete Variable Automatic Com-

puter, 854 Electronic Numerical Integrator and Computer,

854 eliminating threshold

by augmenting the input vector, 811 elitism

a genetic operator, 886 elitist strategy, 875, 882

introduced by K. DeJong, 869 ellipsoid

semi-axes of, 918 emergency lane change maneuver, 80 empty set, 414 encirclements, 201 end effector, 569 end point fixed, 447 end point free, 447 end-point condition, 508 end-point conditions, 449

transversality conditions, 449 energy conservation law, 940 energy function

quadratic form, 333 engine-governor

Jacobian matrix, 193 linearized model, 193 Watt’s governor, 192

ENIAC, see Electronic Numerical Integrator

and Computer enterprise

outstandingly successful, 207 epsilonics definition, 314 equality constrained problem, 678 equation of comparison, 764, 766, 1003 equilibrium, 151, 184, 208

asymptotically stable, 312 attractive, 314 constant solution, 313 convergent, 314 isolated, 188, 314 linearization about, 189 locally stable, 312 stable, 311 uniformly attractive, 316 uniformly convergent, 316 uniformly stable, 315 unstable, 317

equilibrium pair, 191 equilibrium point, 188, see equilibrium state,

313, 907 equilibrium state, 188, 290, 313, 740 equilibrium states, see fixed points equilibrium uniformly stable

conditions for, 352 equivalent

related via similarity transformation, 239 equivalent control

solution of algebraic equation, 551 equivalent integral form, 1002 equivalent moment of inertia, 501 equivalent norms, 750 equivalent system, 551 equivalent transformation, 285 ergodicity

property, 927 error

between actual and desired TLU outputs,

804 of state estimation, 828

error dynamics, 375

of full-order observer, 623 error dynamics matrix, 616 error function

of feedforward neural network, 813 error signal, 9 estimatation

of state variables, 370 of stress level, 370

estimate from output and current, 298 of the exact RAS, 476

estimate of the error

Taylor series, 163 estimate of the norm, 1000 estimate of unknown input, 646 estimating SOC

estimating open-circuit voltage, 295 estimation error

insensitive to matched uncertainty, 573 insensitive to uncertainty, 633 of the transmitter’s state, 946

Euclidean norm, 959 Euclidean object, 909 Euclidean space

Banach space, 439 Euler formula, 182, 1008 Euler’s forward, 169 Euler’s method, 832 Euler-Lagrange equation, 445, 448

alternative representation, 445 Euler-Lagrange equation solutions

extremals, 445 evolutionary algorithm (EA), 857, 868, 870 evolutionary algorithms

to design tracking controllers, 870 evolutionary fuzzy logic controllers, 870 evolutionary processes, 857 exact

discrete model, 149 exact linear discrete model, 180 excessive integrator output

resulting in the large overshoot, 705 exchange of attracting properties, 895 Exclusive-OR (XOR) function, 811 existence of a common P, 758 expansion of the determinant

along the i-th row, 966 expansive

affine transformation, 914 expert knowledge

to design a controller, 713 explicit solution, 311 exponential attraction

measure of, 918 exponential factor, 316 exponential matrix, 122 exponential separation

measure of, 918 exponential stability uniform , 317

extract matrix variable values

dec2mat, 420 extract observable

from unobservable, 236 extremal

of a functional, 441 extremal of

functional, 445, 446 extremals

Euler-Lagrange equation solutions, 445 extremals of the functional, 524 extremizer

of a function, 444

false alarms

avoidance of, 614 family

of type A PID controllers, 699 family of curves, 443 family of parabolas

phase portrait, 154 fault, 614 fault detection and isolation (FDI), 657 fault model

as an unknown input, 657 FDI, see fault detection and isolation feasibility problem, 417, 422

feasp, 423 feasible LMI

negative t, 423 feed-forward path, 698 feedback, 9

combination of state variables, 250 feedback controller

design, 250 feedback linearization, 579, 598, 607 feedback loop, 9 feedback loop broken

due to saturated plant input, 705 feedback system, 199 feedforward network, 818 feedforward neural network, 812 Feigenbaum number α, 908 Feigenbaum number δ, 908 field current, 223 field-controlled, 119 Filippov’s approach

analysis of sliding mode systems, 574 Filippov’s generalized solution, 543 filled Julia set, 911 filtering hypothesis, 201, 206 final state

near to 0, 433 final value theorem, 689 final weight matrix, 476 find matrices L, F, and P

using LMIs, 658 finding a common P, 758 finite set

is compact, 985 firing strength, 880 first difference

of quadratic form, 333 first forward difference, 481 first order necessary condition

for a functional, 441 first-order

leading principal minor, 338 first-order differential equation, 163 first-order Lagrangian algorithm, 680 first-order matrix differential equation, 559 first-order necessary condition for a minimum, 742 for unconstrained optimization, 489 first-order necessary condition (FONC), 664,

675 first-order necessary conditions

for unconstrained minimization, 480 first-order transfer function

with transportation delay, 695 fitness

of chromosome, 866 fitness function, 858, 861, 875, 880 fixed point, 998

attracting, 893 repelling, 893, 894 shifting, 897 fixed points, 893

attracting properties of, 893

of the ball map, 939 fixed-wing aircraft

actuator dynamics neglected, 626 cruise flight, 626 model of, 626 reduced-order UIO, 627 FLC, see fuzzy logic controller flow

of the system, 921 fluctuating

engine speeds, 346 flux linkage, 87 flyball governor, 9, see Watt’s governor, 71, see Watt’s governor, 345

flywheel, 72

moment of inertia of, 346 folded band, 951 FONC, see first-order necessary condition force, 57

tangential component, 184 forward difference discretization, 708 forward Euler, 167

rectangular rule, 167 forward loop, 9

transfer function, 403 forward pass, 488, 489

of backpropagation algorithm, 811 forward path

type A PID controller in forward path,

694 Fourier series, 193, 198, 204 Foxboro Instrument Company

first PID controllers, 709 fractal

dimension of, 909

non-Euclidean object, 908

term coined by Mandelbrot, 908

von Koch snowflake, 908 fractal dimension, 909

of the Cantor set, 910 fractal dust, 910 fractals, 949 fractus

irregular, 908 frangere

to break to create irregular fragments, 908 free body diagram, 91 free LTV system, see unforced LTV system frequency domain, 193 friction, 64 friction coefficient, 346 frictionless environment, 153 Frobenius norm, 835, 990 Frobenius theorem, 585, 592 front wheel steered vehicle, 376 front wheel steering, 115 front-wheel-steering

ground vehicle model, 883 front-wheel-steering vehicle bicycle model of, 870

full column rank, 995 full rank factorization, 971 full-order

asymptotic observer, 287 full-order observer, 259, 275, 276, 288, 613

degree of redundancy, 264 error dynamics of, 623

full-order UIO, see full-order unknown input

observer, 657 projection operator approach, 620

function a rule, 436 absolutely continuous, 545, 986 continuous, 305, 984 continuous at a point, 983 converges to a limit, 390 differential of, 580 increment of argument, 436 Laplace transformable, 1010 linear, 438, 986 lower bounded and decreasing, 390 mincx, 423 of one variable, 983 of several variables, 983 periodic, 198 piecewise continuous, 195 positive and strictly increasing, 349 positive semi-definite, 321 real-valued, 438 set of ordered pairs, 716 support of, 718 twice differentiable, 392 uncertainty modeling, 749 uniformly continuous, 986

function differential, 437 functional, 433

continuous, 436 extremal, 445, 446 extremum only on extremals, 445 function of a function, 436 linear, 436 mapping on space of functions, 436 maximal on a function, 441 operator over a Banach space, 438 variation, 436 variation of, 437, 438, 440

functional example

surface of revolution

example of a functional, 435 functional minimization

subject to constraints, 452 functional values

numbers, 433 functions

real-valued, 194 fundamental harmonic, 193, 204, 206

amplitude, 206 fundamental matrix, 299, 302

another one, 302 nonsingular, 307

not unique, 299 fundamental period, 198 fundamental theorem of calculus, 1027 fuzzifying the controllers, 750 fuzziness

began as vagueness, 790 fuzzy adaptive control, 773 fuzzy adaptive control law, 777 fuzzy adaptive tracking, 767 fuzzy calculus, 790 fuzzy chaotic system, 945 fuzzy control, 790 fuzzy control surface, 875 fuzzy controller, 872 fuzzy controller-estimator compensator, 762 fuzzy engineering, 790 fuzzy logic, 714

target tracking algorithm, 888 fuzzy logic control, 713 fuzzy logic controller, 790

constructed using EA, 883 for vehicle model, 724

fuzzy logic controller (FLC), 722, 870, 887 fuzzy logic system, 772, 777, 781 as nonlinear system, 818

fuzzy logic systems

to approximate nonlinearties, 784 fuzzy Lorentz system, 944 fuzzy Lorentz-based

secure communication system, 947 fuzzy model

of the stick balancer, 797 unforced continuous-time, 744

fuzzy modeling, 736 practice of, 790 fuzzy number

definition of, 717 normal and convex fuzzy set, 717

fuzzy observer, 761, 797 chaotic synchronization, 943 design using a TSK model, 760

fuzzy observer design

using LMIs, 790 fuzzy reasoning

defuzzification, 719 fuzzy regulator design

using LMIs, 790 fuzzy rule, 879 fuzzy rule base

as a chromosome, 880 fuzzy rules, 768

electric drive system, 781 fixed, 875 generated automatically, 873 generated heuristically, 875 number of, 880

fuzzy rules table

as a chromosome, 873 fuzzy set, 714

convex, 717 definition of, 716 non-convex, 717 normal, 717

fuzzy set theory, 790 fuzzy sets, 872

electric drive system, 781 fuzzy singleton

description of, 879 fuzzy singletons, 875, 877 fuzzy sliding mode controller, 792 fuzzy stabilizing controller, 753 fuzzy state-feedback, 744 fuzzy system

“fuzzified” function, 719 fuzzy system model

inverted pendulum, 753

mathematical and linguistic descriptions,

736 weighted average of local models, 737

fuzzy term

to minimize effect of uncertainties, 769 fuzzy tracking controller, 771

GA, see genetic algorithm GA generation

iteration of GA, 864 GA run

entire set of generations, 864 gain matrix, 256 Gaussian membership functions, 818 Gaussian radial basis function, 824 Gaussian radial-basis functions, 831 Gaussian radial-radial basis functions, 825 Gear

multi-step algorithms, 179 gear, 95 gear ratio, 96, 115 gene, 884

basic unit of inheritance, 858 generalized coordinate, 61

angular displacement, 61

electric charge, 61 generalized eigenvalue, 425 generalized eigenvector, 127, 132

of grade k, 127 generalized force, 64 generalized solution

to differential equation with discontinu-

ous right-hand side, 543 generic trajectory, 337 genes

organized into chromosomes, 858 pieces of DNA, 858

genetic algorithm (GA), 857 genetic algorithms

derivative-free optimization algorithms, 859 work with populations of chromosomes,

859 genetic operators, 881

crossover operator, 859 mutation operator, 859 selection operator, 859 work on genotypes, 859

genome, 858 genotype

gene constitution, 858 genotype space

discrete, 861 into phenotype space, 862 space of chromosomes, 859

genotypes

encode phenotypes, 859 geodesics, 435 geodesics problem

general procedure for solving, 435 points on a surface, 435

geometric

multiplicity, 136 geometric multiplicity, 136

of an eigenvalue, 1033 gingerbread man

chaotic set, 951 global asymptotic stability

sufficient condition for, 756 global diffeomorphism, 586

conditions for, 586 global optimizer, 868 global stability

conditions for, 357 global transformation

into controller form, 597 globally stable equilibrium, 405 globally uniformly asymptotically stable, 360,

371 glucocorticoid receptor, 106 good tracking controller

conditions for, 689 governor’s linearized model, 345 governor’s stability, 346 GR, see corticotropin releasing hormone gradient

a column vector, 742, 1030

of computational energy function, 847 gradient descent, 671 gradients

orthogonal to, 1024 greatest lower bound

infimum, 974 Green’s theorem, 161 Green’s thereom, 1030 ground vehicle, 73 ground vehicle model

front-wheel-steering, 883 Gulf War illness, 105 GWI, see Gulf War illness, see neuroimmune

disorders Hölder norms, 959, see p-norms Hölder’s inequality, 750, 752 Hénon map

chaotic algorithm, 931

controlled, 933 Hahn’s example, 315

phase portrait, 315 Hamilton principle, 447 Hamilton-Jacobi-Bellman equation, 501 Hamilton-Jacobi-Bellman(HJB) equation, 498 Hamiltonian function, 495, 498, 502, 509

brachistochrone problem, 513

DC motor, 511

minimum time regulator problem, 519 Hamiltonian matrix, 457, 527, 529

determinant of, 465 in a new basis, 462

handle, 168 floating-point number, 168 object’s ID, 168

handle graphics animation, 290 MATLAB, 290

harmonic linearization

describing function, 193 heading angle

of vehicle, 533 Heaviside function, see unit step function Heaviside step function, 22, see step function Hessian, 672 Heun’s method, see predictor-corrector hidden layer, 814 high frequency modes

excited by rapid switching, 377 high-gain

approximate differentiators, 633 higher harmonics, 206 higher order terms neglecting, 741 higher-order terms, 362 nature of, 366

HJB, see Hamilton-Jacobi-Bellman(HJB) HJB equation, 498 HJB equation, 502

with boundary condition, 504 Holland’s schema theorem, 868 homeostasis, 105, see HPA axis homogeneity property, 12, 17, 25 homogenous LTI system, 307 homogenous LTV system, see unforced LTV

system Hooke’s law, 151, 280, 285 Hopfield model, 849 Hopfield network

circuit implementation of, 843

trajectories of, 849 Hopfield neural network, 843, 850 activation function of, 846 as associative memory, 846 computational energy function of, 846 equivalent representation of, 846 model of, 846

Hou and Müller UIO, 630, 657

HPA axis, 105, see hypothalamic-pituitary-

adrenal axis, 108, 367 model of, 107

human experts

provide a linguistic description, 736 human population growth, 103 Hurwitz matrix, 340, 344, 835 Hurwitz theorem, 342, 345 hypercube, 850 hypothalamic-pituitary-adrenal (HPA) axis,

105, 367 hypothalamus dynamics, 107 hypothesis, 956

I-PD controller

type C PID controller, 706 ideal differentiator, 23 ideal operational amplifier, 49 identification

of a single-input single-state system, 827 of unknown systems, 822 scheme, 818 using neural network, 827

identification error

RBF identifier, 830 identifiers of dynamical systems

neural-network-based, 818 identity matrix, 963 identity observer, 264 IF-THEN fuzzy rules, 818 IF-THEN rules, 714, 736, 737

describe subsets, 718 IF-THEN statement, 714 IFS, see iterated function system

Barnsley fern, 915 ill conditioned

numerically, 136 image

of a matric, 991 immune system, 108 IMP, see internal model principle implementation of control strategy, 492 implementing

a PID controller, 703 implementing PID controllers using a computer, 708

implication, 220

implicit feedback control

brachistochrone problem, 516 implicit integration

backward Euler, 170 impulse function, 14 strength of, 15 impulse response, 19 impulse response matrix, 20 impulse sequence, 25 inaccessible state, 572 incomplete mathematical model, 754 increasing

sequence, 975 increment of argument, 436 indefinite, 324 indices

of active constraints, 677 induced matrix norms p-norms, 989 induced norm, 987 induction

proof by, 220 inductor, 280, 281 industrial applications of MPC, 682 inequality constraints

Lagrangian algorithm, 679 inertia of quadratic forms, 345 inference, 957 infimum, 347

greatest lower bound, 974 infinitesimally small distance, 58 infinity norm, 960 initial

perturbations, 311 initial condition, 1001 initial value

of the orbit, 891 initial value problem, 1000

solution of, 1001 initial values of state variables

estimate, 297 inner product, see scalar product input constraints, 711 input pattern, 841 instability corollary, 362 instability test, 360 instantaneous energy transfer, 1005 integrable

on the interval, 199 integral (I) controller, 693 integral equation, 1000 integral equations, 1003 integral inequalities, 1003 integral of the square error (ISE), 330 integral of the tracking error, 560 integral of time multiplied by the squared er-

ror (ITSE), 332 integral time constant

type A PID controller, 694 integrals

the sum of, 197 integrating by parts, 1010 integrating factor, 1001

of potential function, 589 integration constant, 152 integration constants eliminate, 503 integration variable, 763 integrator, 22 integrator anti-windup, 711 integrator windup

excessive integrator output, 705 Intelligent Vehicle/Highway System (IVHS),

870 interacting populations, 103 internal energy, 65 internal model principle (IMP), 688, 689 internal regulatory processes, 370 internal representation getlmis, 425 internal resistance, 293 internet communication, 941 interpolation

using RBF network, 825 interpolation matrix, 825 interpolation problem formulation, 824

intersection of fuzzy sets, 717 of tangent plane and convex set, 545

invariant manifold, 560

invariant zero, 46, see system zero invariant zeros, see system zeros inverse function theorem, 590, 608 inverse Laplace transform, 31, 123, 1006 inverse Z-transform, 1020 inverted pendulum, 118 local models of, 752 on a cart, 289

invertible

locally, 590 invertible matrix, 964 involutive distribution, 583 ion, 208 ion source, 208 ISE, see integral of the square error (ISE) isocline, 155

slope, 158, 159 isolate

sprung mass from road disturbances, 411 isolated equilibrium, 188

definition of, 188 iterated function system (IFS), 912 iterated function systems, 949 iterated parameter, 906 iterative learning algorithm

for TLU, 811 Itkis’interpretation

of solution in the sense of Filippov, 544 ITSE, see integral of time multiplied by the

squared error (ITSE)

Jacobian matrices, 580, 1030 Jacobian matrix, 193, 362, 366, 591, 601, 608,

743, 1027 linearization, 189

Johann Bernoulli’s brachistochrone problem,

512 Jordan block, 131, 134, 145 Jordan canonical form, see Jordan form, 133 Jordan form, 127, 364 Julia set

Douady’s Rabbit, 912

the boundary of the filled Julia set, 912

k-cycle, 917 Kalman decomposition, 280 Karush-Kuhn-Tucker (KKT) multipliers, 679 KCL, see Kirchhoff’s current law kernel, see null space kinetic energy, 58

brachistochrone problem, 513

of particle, 447 Kirchhoff’s current law, 843 Kirchhoff’s current law (KCL), 296 Kirchhoff’s voltage law, 89, 222, 273, 466,

502, 527 KKT multipliers, 679 Kohonen learning rule, 841 Kronecker product, 327, 334, 404, 525, 971,

973 stacking operator, 334

Kučera conditions for solving ARE, 459

L’Hospital’s rule, 849

Lagrange equation of motion, 64 Lagrange equations, 60, 118

for two-link manipulator, 568 Lagrange multiplier, 742 Lagrange multipliers, 675 Lagrange multipliers method RAS estimate, 480

Lagrange stable

equilibrium, 358 Lagrange’s multiplier theorem, 526 to find optimal sequence, 527 Lagrangian, 60, 98, 118, 447, 480, 675

associated with constrained optimization

problem, 675 for two-link manipulator, 568

Lagrangian algorithm for equality constraints, 676 for inequality constraints, 679

Lagrangian function, 60, 679

double pendulum, 100 lane detection, 872 Laplace expansion

to evaluate determinant, 966 Laplace transform, 21, 123, 209, 232, 329,

694, 1005 bilateral, 1005 is linear, 1014 linear operator, 1008 of derivatives, 1009 of higher-order derivatives, 1010 of integrals, 1013 of the exponential function, 1007 of the unit step, 1007 of time derivatives, 1012 of voltage across a capacitor, 1014 one-sided, 1005 solving linear differential equations, 1005 to solve linear ordinary differential equa-

tions, 1011 two-sided, 1005 unilateral, 1005

Laplace transformable, 1014

function, 1010 lateral force, 76 lateral position, 871

tracking error of, 877 lateral velocity

of vehicle, 871 law of conservation of energy, 109 lead screw, 113 leading principal minors, 338, 340, 344, 969 learning

supervised mode, 840 unsupervised mode, 840

learning algorithm

using Lyapunov techniques, 822 learning error

reduce by adjusting weights, 811 learning law

second method of Lyapunov, 829 learning rate, 816 learning rule

RBF identifier, 830 learning rules

of neural identifier, 827 least identification error, 828 least significant bit (LSB), 850 least squares

graphical interpretation, 995 solution, 995 least squares solution

recursive method, 996 least upper bound supremum, 973

Least Upper Bound Axiom, 973–975 least-squares estimate of unknown input by Edwards-Spurgeon UIO, 654

left hand

limits, 199 left outer factor, 420 left-half complex plane, 203 left-hand plane, 270 left-hand side

smaller side of inequality, 420 Leibniz formula, 588, 595, 602, 607 Leibniz’s rule, 305

derive, 309 lemma

an auxiliary result, 956 level curve, 933 Lie bracket, 580 Lie derivatives, 580, 587 Lie-derivative

notation, 601 limit

of infinite sequence, 976 limit cycle, 203

amplitude, 205 definition of, 161 existence, 182 existence of, 162 stable, 203, 205 unstable, 203 van der Pol equation, 160

limit cycles

predict, 193 limiter

describing function, 211 line

segment, 414 linear

approximate nonlinear, 215 linear active suspension, 111 linear approximation, 183 linear combination, 141

of matrices, 141, 1032

of vectors, 990 linear dependence of columns, 990 linear function, 438, 986 linear functional, 436 linear map

norm of, 987 linear maps, see linear functions linear matrix inequalities, 573 linear matrix inequality (LMI), 415 linear model

in x and u, 743

in the vicinity of the operating state, 743 of population growth, 891 predicts the exponential grow of popula-

tion, 891 linear nonseparability

sufficient condition for, 810 linear objective function

mincx, 423 linear operators, see linear functions linear quadratic regulator (LQR), 433, 927 linear saturation, 388 linear spring, 151, 215 linear stabilizing controller for fuzzy systems, 795

linear state-feedback stabilizing, 453 linear system, 13, 183

decomposed as a direct sum, 280

definition of, 13 theory, 279 linear systems, 219

continuous-time, 219 linear term coefficient

in Edwards-Spurgeon UIO, 653 linear time-invariant, 20 linear time-invariant (LTI), 216, 315 linear time-varying (LTV), 293 linearity

concept of, 215 linearization, 183, 406

about equilibrium, 362 of a nonlinear model, 207 when it fails to determine stability, 366

linearized

discrete-time system, 929 linearized ball map

about equilibrium state, 940 linearized model, 921

about the equilibrium, 739 engine-governor, 193

linearizing state-feedback, 609 the ball map, 939

linearizing state-feedback, 610 linearly dependent, 216

set of vectors, 990 linearly independent, 136

set of vectors, 990 vectors, 124

linearly nonseparable, 803 linearly separable, 803

vectors, 811 linearly separable points, 804 linguistic description

provided by human experts, 736 linguistic values, 716 linguistic variable, 716 Lipschitz condition, 606, 986, 1003 Lipschitz constant, 606, 914, 1003 LMI, see linear matrix inequality

auxiliary convex problem, 423 canonical representation of, 419 constant term, 422 feasibility problem, 422 generalized eigenvalue solver, 425 left outer factors, 422 left-hand side of, 422 right outer factors, 422 right-hand side of, 422 structured representation of, 419 symmetrized expression, 422

LMI feasibility problem, 648, 947 LMI feasibility solver

feasp, 423 LMI feasible

negative t, 423 LMI feasible solution

feasp, 420 LMI initialize

setlmis([]), 420 LMI internal representation

getlmis, 420 LMI solver function

mincx, 423 LMI solvers, 419 LMI system

terms of, 421 LMI system description

initialize, 420 LMI toolbox

tutorial, 425 LMI toolbox tutorial

lmidem, 425 LMIs, 573

to design sliding mode UIO, 648 LMIs internal representation

getlmis, 425 LMIs matrix variable values

dec2mat, 420 LMITOOL

Scilab Consortium, 426 Local linear models

inverted pendulum, 752 local minimizer, 675 local models

components of the TSK model, 738

describe plant’s behavior at operating points,

739 pendulum, 743

local tangents

field of, 157 locally C^{p}-invertible, 608 locally C^{∞}-invertible, 590 locus

set of points, 155 logistic equation, 103, 890, 892 logistic map, see logistic equation longitudinal force, 76 longitudinal motion

of vehicle, 533 look-up table

to implement control surface, 733

to implement FLC, 728 Lorentz’s system

fuzzy representation, 944 Lorenz equations, 950 Lorenz’s weather model, 950 lost mechanical energy, 65 Lotka-Volterra equations

predator-prey relationship, 105 low cortisol equilibrium, 369 low-pass filter, 201 lower bound, 974 lower bounded, 974 lower triangular, 341, 344 LQR, see linear quadratic regulator, 532 LQR approach

to compute state feedback matrix, 953 LQR method, 929 LSB, see most significant bit LTI, see linear time-invariant, see linear time-

invariant LTI controlled system, 305 LTI lumped causal system, 23 LTI model

continuous-time, 241 single-input, 241

LTI system, 122 homogenous, 307 seelinear time-invariant, 315 stbility of zero-input response, 395 stbility of zero-state response, 395

LTV, see linear time-varying scalar case, 299 system, 298 uncontrolled system, 298 LTV controlled system, 305 LTV system, 308, 309, 405 LTV systems, 307

analysis and control, 307

observers for, 307 Luenberger observer, 260, 614 Lukasiewicz

inventor of multivalued logic, 790 lumped discrete-time system, 25 lumped mass representation, 67 lumped system, 6, 22, 27 lumped uncertainties, 572

as unknown inputs, 615 Lyapunov

theorem of, 835 Lyapunov based

stabilizer, 790 stabilizing control design, 736

Lyapunov derivative, 751, 837, 839

negative definite, 749 Lyapunov equation, 339, 386, 953

continuous, 322 discrete-time, 333 Kronecker product, 327 observability, 325 solving, 332 uniqueness of solution, 327

Lyapunov exponent, 895, 917, 918

natural logarithm of Lyapunov number,

920 Lyapunov exponents

characterize a chaotic motion, 923 measure the initial condition sensitivity,

890 real parts of the eigenvalues of linearized

system, 922 Lyapunov function, 324, 371

chaotic control algorithm, 930 construction of, 379 for the closed-loop system, 751

Lyapunov function candidate, 380, 393, 394,

748, 769, 774, 778, 829, 836, 839 Lyapunov matrix equation, 342, 408, 417, 638,

835 constructing UIO, 643 continuous, 322

Lyapunov matrix equations, 365 Lyapunov number, 917 Lyapunov numbers

measure the rates of separation, 919 Lyapunov theory

based design, 370

evaluate performance index, 329 Lyapunov’s first method, see Lyapunov’s in-

direct method Lyapunov’s indirect method, 362, 367 Lyapunov’s second method, 614 Lyapunov’s theorem, 322, 752

geometric interpretation, 325 Lyapunov-based design

using LMIs, 790 Lyapunov-like lemma, 392, 394, 830

MacFarlane and Potter method, see eigenvec-

tor method for solving ARE machine tool drive

single axis, 113 magnetic field, 209 main path of flux linking, 87 majorization

component-by-component, 763 Mandelbrot set, 911

one-page-dictionary of all Julia sets, 912 manifold, 559

construction of, 562

invariant, 560 manipulator

inertia matrix of, 568 manipulator inertia, 561 manipulator model

in state-space format, 568 map, 890

on an open subset, 608 Markov parameters, 238 mass-spring-damper

nonlinear, 207 massless shaft, 118 matched uncertainties, 546, 572 matched uncertainty, 336, 793 matching condition, 749 mathematical model

incomplete, 754 not available to control engineer, 713 to construct a controller, 687

mathematical modeling, 55 mathematical models

of biological neuron, 801 mating pool, 859, 875 MATLAB, 168

handle graphics, 290 script, 290 Symbolic Toolbox, 180, 283, 315

MATLAB animation, 533 MATLAB command

[Q,R]=qr(C), 227 eig(H), 528

MATLAB function block, 782 ctrb, 237 dlyap, 404 lyap, 404 obsv, 237 ss(G,’min’), 236 to generate membership functions, 730

MATLAB program

animation, 169 MATLAB’s command

c2dm, 680 place, 627 plot3, 951 rand, 822

MATLAB’s commands, 277 MATLAB’s function

compose, 900 ctrbf, 285 fsolve, 516 obsvf, 285 randn, 649 subs, 900, 904

MATLAB’s handle

object’s ID, 168 MATLAB’s handle graphics, 168 MATLAB’s Symbolic Math Toolbox, 1034 matrix

a set of scalars subject to two orderings,

963 array of numbers, 963 diagonal, 342 diagonal form of, 283 full column rank, 995 invertible, 964 nonsingular, 964 rank of, 991 similar, 137 symmetric positive definite, 432 with negative diagonal and non-negative

off-diagonal elements, 792 matrix 1-norm

maximum of absolute column sum, 989 matrix 2-norm

largest singular value, 989 matrix differential equation, 559

of TPBVP, 522 matrix exponential, 141

non-negative, 762 matrix inequalities, 414 matrix inequality

constraint, 414 matrix inequality lemma, 756 matrix infinity-norm

maximum of absolute row sum, 990 matrix inversion formula, 965, 996 matrix multiplication

is associative, 964

not necessarily commutative, 963 matrix norm, 987

compatible, 988 consistent, 988

matrix of zeros, 137 matrix p-norms, 989 matrix rank

number of linearly independent columns,

991 number of linearly independent rows, 991

matrix variable full rectangular, 421 symmetric block diagonal, 421

matrix variables, 420

three structures, 421 matrix-valued variable, 420 matrix-vector product, 990 maximal range, 218 maximum overshoot, 710 maximum value

of rank, 216 maximum-minimum theorem

of Weierstrass, 984 Maxwell

stability in terms of roots location, 401 Maxwell’s analysis

first work in linear system theory, 279 of Watt’s governor, 279

Maxwell’s problem

Routh’s solution, 401 McCulloch-Pitts neuron, 854 mean value theorem, 893, 894 mechanical energy, 65 membership function, 716 membership functions, 791, 872

Gaussian , 818 their “spread”, 724 tuning using EA, 875

membership states

inclusion or exclusion, 714 memoryless system, 4 Mendel

basic principles of genetics, 883 message

to be transmitted, 947 message to be transmitted

hidden in a noise-like signal, 945 MIMO, see multi-input multi-output minimal

necessary and sufficient conditions for, 235 minimal realization, 43, 235, 236, 285 minimal realizations

equivalent, 237, 240

similarity transformation, 237 minimization

subject to inequality constraints, 677 minimizer, 995 minimizing energy

dissipated in armature resistor, 511 minimizing the Hamiltonian, 505, 509 minimum cost

of transferring, 488 minimum principle, 510, 527 minimum principle of Pontryagin, 506 minimum time regulator, 493 minimum time regulator problem, 519 model

population growth, 103 model nonlinearity

of one-link robot, 607 model predictive control, 710 model reference

plant matrix, 376 model reference system, 376 model-based fault detection, 614 model-based predictive control (MPC), 659 model-reference signal generator, 374 model-reference tracking controller, 373 modeling error, 829, 836 modified ball map, 941 modified center average defuzzifier, 724 modified performance index

change in it, 507 modified threshold, 806 moment of inertia, 346

equivalent, 501 monotone

sequence, 975 monotone bounded sequence

is convergent, 982 Moore-Penrose pseudo-inverse, 617, 624 most significant bit (MSB), 850 motion

animation, 168 motor torque, 113

torque delivered, 223 motor-torque constant, 96 moving away

without returning to equilibrium, 312 moving horizon control, see MPC MPC, see model-based predictive control constraints on the control, 659 control and optimization tools, 659 current control action computed on-line,

659 industrial applications, 682 main idea, 659 performance index, 659 predicted sequence of the state vectors,

662 prediction horizon, 662

MPC sequence

construction of, 662 MPC strategy

illustratraion of, 660 MSB, see most significant bit multi-dimensional case, 300 multi-input, 2

controller form, 244 pole placement, 254 multi-input multi-output, 19 multi-input pole placement

can achieve secondary goals, 257 multi-input pole placement solution

not unique, 257 multi-point

crossover operator, 863 multi-step algorithms, 179 multiplication by an exponential in the s domain, 1015 multiplicative rate

of separation, 895, 917 multivalued logic

of Łukasiewicz, 790 mutation operator, 859, 875, 877 mutation probability, 863 mutation rate, 863, 875 mutually orthogonal, 210

functions, 194

n.s.d., see negative definite, see negative semi-

definite natural Lagrangian system

planar two-link robot, 560 necessary and sufficient condition

of Routh, 343 negative definite, 339, 968

Lyapunov derivative, 389 with respect to the switching surface, 548

negative definite (n.d.), 321 negative semi-definite, 968 negative semi-definite (n.s.d.), 321 negative unity feedback, 402 neglected nonlinearities

effect on stability, 336 neglecting terms

of order three and four, 904 neighborhood, 985

of a point, 442 neighborhood of the origin, 363 neural A/D converter

construction of, 850 neural fuzzy on-line identification, 855 neural identifier

adjustable parameters, 827 architecture of, 828 learning rules, 827 representation of, 828

neural nets, see neural networks neural networks, 799 neuroimmune disorders, 105 neuron’s cell body, 799 neurons

brain’s building blocks, 799 neuroscience, 854 neurotransmitters, 801

stimulate or inhibit, 801 Newton’s equation, 60 Newton’s law, 151 Newton’s method, 673 Newton’s quotient, 75 Newton’s second law, 57, 90, 93, 184, 209

applied to rotor, 90 rotational version, 153

nilpotent, 146 nilpotent matrix, 139 nominal control strategy, 507 nominal system, 372

asymptotically stable, 372 non-anticipatory system, see causal system non-convex

fuzzy set, 717 non-diagonal, 134 non-existence of a common P

sufficient condition, 746 non-feasible LMI, 417 non-interacting species, 104 non-negative matrix exponential, 762 non-periodic trajectories, 917 non-strict inequalities, 418 nonconservative forces, 64, 66 nondecreasing

sequence, 975 nonincreasing

sequence, 975 nonlinear

differential equations, 311 real world, 215

nonlinear amplifier, 843 nonlinear closed-loop system

stability, 201 nonlinear differential equation

HJB equation, 503 nonlinear element, 199

sinusoidal input, 199 nonlinear model

linear approximation, 188 robot manipulator, 290 simple pendulum, 184

nonlinear projector

adaptive fuzzy control, 796 nonlinear system, 13

approximation with linearized system, 363 definition of, 13 linearization, 183 single output, 599 stability, 201 stability analysis, 347

nonlinear term coefficient

in Edwards-Spurgeon UIO, 653 nonlinear time-varying

battery model, 297 nonlinearities

as unknown inputs, 615 nonobservable, 234, 283 nonreachable, 224 nonreachable states, 218 nonrotating coordinate frame, 91 nonsingular distribution, 583

involutive, 583 nonsingular matrix, 964 norm

estimate of, 1000 of a function, 195 of the linear map, 987

norm bound condition, 838 norm bounds, 337, 338 norm of uniform convergence, 993 normal-rank, 45, see polynomial matrix, 622 normalized patterns, 840 normed linear space, 993, 1035 normed space

vector space with a norm, 992 not diagonalizable, 136 notation arg min, see notation argmin notation argmin, 772 notation Proj, 771 null solution

(locally) stable, 352 null space, 991 nullity, 134

of the characteristic matrix, 130 number of fuzzy rules, 880 numerical methods, 163, 179 numerical techniques

approximate solutions, 162

solving differential equations, 162 Taylor series expansion, 162

Nyquist stability criterion, 199 Nyquist’s criterion, 201

objective function convex, 414 minimization, 407 transforming into fitness function, 861

observability, 232 gramian, 298 not zero along nonzero solution, 326

observability index, 249 observability indices, 249 observability matrix, 233

of nonlinear system, 602 observable

definition of, 230 observable pair, 248, 342 observation error, 259

convergence rate, 260 dynamics, 260

observer, 259, 263, 276, 278, 613 closed-loop, 263 convergence rate of, 616 design, 287 deterministic system, 258 dynamics, 288 estimate state vector, 250 full-order, 275 identity, 264 open-loop, 259 reduced-order, 276, 278

observer companion form, 275 observer design

alternative approach, 263 separated from controller design, 269

observer design parameters, 605 observer form, 249, 275, 287, 288, 604

gain matrix construction, 262 nonlinear, 610, 611

of nonlinear system, 599 observer gain vector, 604 observer matching condition, 633 observer poles

location reommendation, 269 observers

as “software” sensors, 258 as “virtual” sensors, 258 deterministic approach, 258 for uncertain systems, 614 state estimators, 258 to control electric machinery, 655

odd function, 204 offset term, 741 offspring, 863 offspring chromosomes, 863 OGY method, see Ott, Grebogi, and Yorke

(OGY) method Ohm’s law, 296 oldest feedback device, see Watt’s governor one-link manipulator

feedback linearization, 598

observer for, 604 one-link robot, 466, 585, 589, 607, 686, 792

observer form, 603 one-point crossover, 887 open

connected region, 1027 open ball, 347 open circuit

voltage, 295 open left-half plane, 345 open set, 985

connected, 1026 open subset, 608 open unit circle, 940 open unit disc, 1031 open-circuit, 293

voltage, 293 open-circuit voltage

compute, 297 open-circuit voltage estimator, 295

model based, 295 open-loop, 8 open-loop control

chaotic control algorithm, 929 open-loop observer, 259

impractical, 260 operating point, 184, see equilibrium

several variables, 186 operating state, 741 operational amplifier, 49 optimal

closed-loop system, 455, 456

control, 431 optimal armature voltage

DC motor, 512 optimal control, 481, 502, 510

candidate for, 455

Lyapunov function, 454

transferring, 496 optimal control problem, 526 optimal control sequence, 490 optimal control system

with prescribed poles, 461 optimal control systems

with prescribed poles, 462 optimal controller, 7, 431, 453

must minimize the Hamiltonian, 521 optimal gain, 466 optimal gain vector, 476 optimal motion of the particle

brachistochrone problem, 516 optimal parameter vector, 777 optimal state-feedback, 501, 530 optimal time

of transferring, 494 optimal trajectories, 530 optimal transfer, 496 orbit, see trajectory order

number of schema’s fixed bits, 865 of schema, 865 order reduction, 546 ordinary differential equation of the first order, 1000 ordinary differential linear equation of the first order, 1001

ordinary set, see set

two membership states, 714 orthogonal

functions, 194 mutually, 210 set of vectors, 965 vectors, 959

orthogonal complement, 648, 992 orthogonal matrix, 650, 965, 967, 969 orthogonal projection, 613 orthonormal, see orthogonal oscillatory

motion about equilibrium, 312 motion steadily decreases, 312

Ott, Grebogi, and Yorke (OGY) method, 928 output

equation, 298 output feedback sliding mode

controllers, 574

P-controller

in feed-forward path, 698 p-norm

for example, Euclidean 2-norm, 824 of a vector, 750

p-norms, 959

of matrix, 989 p.d., see positive definite p.s.d., see positive semi-definite parabola

through three points, 175 parameter adaptation law, 787 parameter space, 858

as phenotype space, 858 parameterization

of a curve, 1022 parameterized curve, 1022 partial membership, 714 partial sums

of the series, 982 particle of a constant mass, 57 partition

of the interval, 196 Patel and Toda lemma, 408 path of quickest descent

upside-down cycloid, 518 pathological, 136 PBH rank test, 552 PBH test, see Popov-Belevitch-Hautus test,

266 controllable, 285 observability, 266 observable, 285 penalty functional, 7, see performance index perceptron, 801, 811 to classify, 802 training of, 811

performance index, 7, 329, 332, 431 convex combination, 432 performance index (P I), 873 performance indices evaluating, 335

period

fundamental, 198 period-2 cycle

attracting, 899 periodic equilibrium solutions to the ball map, 938

permanent magnet (PM) stepper motor, 81 permanent stepper motor controller form, 593

permanent-magnet

stepper motor, 210 persistently exciting inputs, 791 signals, 791

perturb

equilibrium, 191 phase angle change

between current bounce of the ball and

the next, 937 phase plane, see state plane phase portrait, 151, 152, 367, 467, 510

draw interactively, 160

of closed-loop linear system, 470 of closed-loop nonlinear system, 470

phase-plane asymptotes, 181 phase-plane portrait, 181

of uncontrolled system, 730 phase-portrait

of closed-loop system, 733 phenotype

expression of genotype, 858 phenotype space, 858

from genotype space, 862 physical sensors, 655 physical system, 56 PI control, 710 PI controller

digital, 711 PI-D controller

type B PID controller, 706 PID, see proportional-integral-derivative

architecture, 710 PID controller

“bread and butter” of control engineer-

ing, 688 transient and steady-state specifications,

693 type A, 693

PID controller implementation, 703 PID’s primacy

in industrial control, 710 PIDF controller

PID controller with a derivative filter, 694 pure derivative replaced with derivative

filter, 700 piece-wise constant input, 680 piecewise continuous, 196

function, 195 on the interval, 199

piecewise continuous control, 493 Pisum sativum

garden pea, 883 pitch, 73 pituitary gland, 107 planar two-link robot

natural Lagrangian system, 561 plant, 8

process to be controlled, 736 to be identified, 822

plant and full-order observer

2n order system, 268 plant model, 374

in new coordinates, 266 PLC, see positive linear combination PM, see permanent magnet PM stepper motor

controller form, 594, 597 mathematical model, 87

PO, see principle of optimality point, 347

moving around a circle, 1022 point mass, 118, 289 point-to-set mapping, 541 pointwise convergence, 993 pole placement, 241, 252, 564

numerically stable, 257

single-input, 252 pole placement problem

solvable if and only if, 257 poles

complex conjugate pairs, 251 in desired locations, 461

polynomial of n-th degree, 340 with real coefficients, 340

polynomial matrix, 45, see system matrix polynomial zeros, 345 polytopic system, see fuzzy system model Pontryagin’s minimum principle

fixed final state, 506 free final state, 506 generalization of Euler-Lagrange equations,

506 includes constraints on control, 506 minimum time regulator problem, 519

Popov-Belevitch-Hautus (PBH) test, 228 population growth, 103 population size, 875, 877 position vector, 1022 positive definite, 324, 407, 968

definition of, 351 quadratic form, 363 real-valued function, 351 scalar product, 197 with respect to the sliding surface, 772

positive definite (p.d.), 321 positive definite matrix

leading principal minors positive, 342 positive definite over workspace, 561 positive linear combination (PLC), 810 positive real

transfer function matrix, 573 positive semi-definite, 968

definition of, 351 principle minors non-negative, 765 real-valued function, 351 symmetric square root of matrix, 970

positive semi-definite (p.s.d.), 321 posttraumatic stress disorder, 105 potential

accelerating, 208 potential energy, 59

brachistochrone problem, 513 of particle, 447

potential function, 590, 1026 for a vector field, 589 integrating factor, 589

practical voltage source

as battery model, 293 practice of fuzzy modeling, 790 precise positioning, 83 predator-prey relationship, 105 predicted sequence

MPC, 662 prediction horizon

MPC, 663 prediction power

of neural identifier, 822 predictor, 172

n-dimensional, 173 predictor-corrector, 171, 172, 179

n-dimensional, 173

second-order method, 172 preliminary controller, 337 premise part

of the fuzzy rule, 880 preprocessing

training pairs, 807 principal minors, 968

of order p, 968 principle

of conservation of momentum, 936 principle of induction, 978 principle of maximum simplicity, 56 principle of optimality, 525 principle of optimality (PO), 485 principle of the conservation of energy, 65 product

of two matrices, 963 projection, 617, 960

of x along y, 960 orthogonal, 613 skew, 613

projection matrix, 621 projection operator, 613

Proj, 771 projection operator approach

to state observation, 615 prolonged stress, 367 proof, 957

by contradiction, 958, 974, 977 by contraposition, 220, 257, 958 direct, 958

proof by contraposition, 228 proper rational function, 30 proper rational matrix, 35 proper transfer function matrix, 41 property

additivity, 11 change, 168 handle, 168 homogeneity, 12 sifting, 15 superposition, 12, 13

proportional gain

type A PID controller, 693 proportional-integral-derivative (PID), 688 proportional-plus-derivative (PD), 402 proposition

a minor theorem, 956 PTSD, see neuroimmune disorders, see post-

traumatic stress disorder pulse, 17 Pythagoras theorem, 960, 961

Q-R decomposition, 634, 635 quadratic form, 321

a real-valued function of n variables, 968 energy function, 333 first difference, 333 quadratic formula, 905, 906 quadratic indices, 327

performance indices, 327 quadratic maps

universal properties of, 907 quadratic optimization, 526 quarter-car model, 111, 409

r-neighborhood

isolated equilibrium, 188 Rössler equations, 951 radial basis function radius of, 824 radial-basis function

neural networks, 822 radial-basis function (RBF), 822 radial-basis functions adding, 831

radially unbounded, 357, 358, 766 random variable

with standard normal distribution, 881 randomness

from sensitive dependence on initial con-

ditions, 901 range, 218

maximal, 218 of a matric, 991 rank

of a mtrix, 991 RAS, see region of asymptotic stability

estimation of, 479 RAS estimation

for single-input systems, 477 rate of growth

of the population, 103 rational function, 22, 329

converting into differential equations, 329 converting into state-space representation,

330 proper, 30 strictly proper, 30, 1012, 1020

rational proper transfer function, 35 RBF, see radial-basis function RBF identifier, 856

learning law, 829 RBF network, 824, 825

basis functions, 831 for interpolation, 822 neural identifier, 827

RBF networks

perform interpolation, 824 RBF neural identifier, 835, 836

an adaptive system, 822 re-tuning

of PIDF controller, 702 reachability, 219, 229, 283

equivalence to controllability, 219 implies controllability, 219

reachable definition, 219 definition of, 216 pair, 235

separated from nonreachable, 227 reaction

electro-chemical, 294 real Jordan form, 364 real spring, 215 real world

nonlinear, 215 real-valued function, 438 time-dependent, 351

realization controllable and observable, 409 dimesnison of, 30 minimal, 235, 236 of a transfer function, 235

realizations

of different dimensions, 235 reasoning, 957 receding horizon control, see MPC receiver

as a fuzzy observer of the transmitter, 946 reconstruct

entire state, 231 reconstructing

actuator and sensor faults, 657 Reconstructing unknown input, 633 reconstructing unknown input

by Edwards-Spurgeon UIO, 653 recovered signal, 947 rectangular rule

forward Euler, 167 recursive least squares (RLS), 997 reduced-order, 119 reduced-order model

of an inverted pendulum, 795

of one-link robot, 466 reduced-order observer, 265, 276, 288, 289,

613 dynamics, 276 dynamics of, 278 for uncertain systems, 616 method 1, 265 method 2, 265, 267, 278 separation property, 270, 271

reduced-order system

when restricted to the surface, 577 reduced-order UIO, 624 reference

for two-link manipulator, 569 reference input, 8, 700 reference inputscaling, 377 reference model, 768 reference signal, 728

model of, 689 unit ramp, 691

reference signal generator, 872 reference system, 374, 781

changing its value randomly, 781 referred to motor shaft

moment of inertia, 501 region

open connected, 1027 region of asymptotic stability, 406 region of asymptotic stability (RAS), 476 region of attraction, 476, 928 Region of Convergence (ROC), 1006 regressor, 768 regressor generator, 782 regular form, 560

constructive algorithm, 560 regular point, 677 of active constraints, 678 of constraints, 674

regulating

steam engines, 279 regulator, 9, 568

for two-link manipulator, 569 related via similarity transformation

equivalent, 239 relaxed system, 14 relay, 203

describing function, 205 relay output

rectangular wave, 203 remainder

Taylor series, 163 repeated eigenvalues, 126 repeated flip bifurcations lead to chaos, 902

repeated zeros, 143

repelling fixed point, 894 representative point (RP), 151 reset controller, 693 residuals, 614 resistance

charging, 293 discharging, 294 internal, 293

response total, 13 zero-input, 13 zero-state, 13

rest position, 184, see equilibrium restricted to the surface

reduced-order system, 577 reverse pass

of backpropagation algorithm, 811 Riccati differential equation, 523 Riccati equation, 465

discrete, 484 solving, 468

right outer factor, 420 right-hand side

larger side of inequality, 420 RLS, see recursive least squares road roughness, 111 roadway reference system, 872 robot manipulator, 67

combined controller-observer compensator,

291 double-link, 97 lumped mass representation, 67 one-link, 95, 290 state feedback, 290

robot parameters, 467 robust controller

maintains stability for uncertainties in ex-

pected range, 336 robust optimal control, 523 ROC, see Region of Convergence roll, 73 roots of

characteristic equation, 251 Rosenbrock’s system matrix, 45 rotation matrix, 168 rotor, 82, see armature roulette wheel method, 875, 915 roulette-wheel method, 887

selecting parent chromosomes, 862 round operator, 831 rounding of a number to nearest integer

as optimization problem, 850 Routh, 343 Routh array, 345 Routh stability criterion, 345 Routh test

story behind, 401 Routh-Hurwitz criterion, 343, 402 routing problem, 486 row-by-row method, 41 RP, see representative point Rugh’s example, 315 rule base, 877 rules matrix, 722 rules table, 873 Runge method, 182 Runge’s method, 174, 177

first-order differential equations, 177 Runge-Kutta algorithm, see Runge-Kutta method Runge-Kutta method, 178, 182

s-domain, 21 s-map, 905 S-shaped step response

approximation of, 695 SAE, see Society of Automotive Engineers,

see Society of Automotive Engineers SAM, see standard additive model sampling period, 3, 148, 149, 180 satellite

model of, 153 thrusters, 153

saturated plant input

feedback loop broken, 705 saturation, 892 saturation level, 103 scalar

unspecified, 226 scalar block matrix, 421 scalar product, 197, 802, 841, 959

commutative, 959 of two functions, 194 over a function space, 193 positive definite, 197

scaling, 909 scaling reference input

scaling factor, 377 schema

a set of bit strings, 865 as template, 866 probability of survival, 867 survives crossover, 867

schema performance

its average fitness, 866 schema theorem of Holland, 868 schemas

plural of “schema”, 865 schemata

plural of “schema”, 865 Schur complement, 417, 426, 639 Scilab Consortium

LMITOOL, 426 SDP, see semidefinite programming search space, 859 second derivative bounded, 392 second method of Lyapunov, 614 second order sufficiency condition

for minimum, 332, 333 second-order

leading principal minor, 338 second-order differential equation, 164 second-order sufficiency condition (SONC), 665 second-order sufficient condition

for a relative minimizer, 483 second-order Taylor’s formula, 671 secure communication, 941 select

from left to right, 256 selecting parent chromosomes

roulette-wheel method, 862 selection method, 877 selection operator, 859, 863, 866 self-organization, 840 self-similarity, 909 semi-axes

of ellipsoid, 918 Semidefinite programming

extension of linear programming, 414 semidefinite programming

example, 416 semidefinite programming (SDP), 414 semigroup property

state transition matrix, 301 sensitivity

with respect to initial conditions, 890 sentence

a part of a language, 956 sentences

are used to make statements, 956 separating hyperplane, 802, see decision bound-

ary, 803, 804 separating line, 805 separation

multiplicative rate of, 895 separation principle, 269, see combined controller-

observer compensator sequence bounded, 977 decreasing, 975 increasing, 975 limit of, 976 monotone, 975 nondecreasing, 975 nonincreasing, 975 of real numbers, 975

sequence of numbers, 1016 sequence of predicted outputs

MPC, 663 sequential decision making process, 485 sequential quadratic programming (SQP), 676 series

converges, 983 diverges, 983

servomechanism, 11 set

collection of definite, distinguishable ob-

jects, 956 collection of objects, 714 of all complex numbers, 956 of all integers, 956 of all positive integers, 956 of all real numbers, 956 of rules, 722

set of all bounded sequences

denoted l_{∞}, 977 set of eigenvalues of a square matrix

spectrum, 551 set of vectors

linearly dependent, 990 linearly independent, 990

set-point constant, 692 set-point kick phenomenon, 700, 702

elimination of, 706 several variables, 186 shaft

massless, 289 Sherman-Morrison-Woodbury formula, 964, 965 shifting the fixed point, 897 shock absorber, 409 Sierpiński carpet, 952

fractal dimension of, 952

zero area of, 952 Sierpiński gasket, 952

fractal dimension of, 952

zero area of, 952 sifting property, 15 sigmoid

as an activation function, 814 sigmoid function

as activation function, 843 sign activation function, 801 sign changes, 345 sign function, 801 sign nonlinearity, 388 similar

matrices, 341 similar matrix, 137 similarity matching, 840 similarity matrix, 341 similarity transformation, 180, 224, 227, 234 does not affect characteristic polynomial,

257 does not affect controllability, 257 equivalence relation, 235 preserves reachability, 227

simple nonlinear models

with very complicated dynamics, 889 simple pendulum, 66, see pendulum

motion animation, 168 simplified TLU

learning algorithm, 808 learning problem, 807

Simpson’s formula, see Simpson’s rule Simpson’s rule

Runge’s method, 174 simulation model, 7, see truth model, 56 SIMULINK, 50, 290

block diagrams, 787

to simulate the closed-loop system, 782 SIMULINK’s 2-D Look-Up Table block, 728 single axis

machine tool drive, 113 single population

dynamics, 103 single-input, 2 single-input single-output, 17 single-order pole at the origin

final value theorem, 690 single-point crossover, 875 single-step algorithms, 179 singular value, 408 singular value decomposition, 634 singular value decomposition (SVD), 967 singular values, 967 sinusoid

amplitude, 201 sinusoid input

describing function, 193 sinusoidal describing function, 207 sinusoidal signal, 199 sinusoidal transfer function, 205 SISO, see single-input single-output skew projection, 613 sliding

motion confined to switching surface, 538 sliding (switching) surface design, 546 sliding controller design flow chart, 549 multi-input plants, 549 single-input plants, 549

sliding mode, 613

reduced-order, 575 sliding mode control (SMC), 560 sliding mode controller design of, 576 sliding mode observer, 573 design of, 578 gain, 578 Lyapunov matrix equation, 578 poles, 578

sliding mode observer design

essential ingredients of, 637 sliding mode systems, 574 sliding mode UIO, 631 sliding mode UIO design using LMIs, 648

sliding motion

conditions to occur, 540 sliding surface

attractive, 772

design algorithm, 771

in the tracking error space, 771 sliding surface design, 546 small change

in initial conditions, 311

in solution, 311 smallest convex set, 543 smallest singular value, 408 SMC, see sliding mode control smooth functions, 579 smooth state-space transformation, 609, 610 smoothing out discontinuity, 645 SOC, see state of charge Society of Automotive Engineers (SAE), 870 Society of Automotive Engineers (SAE), 73 solution

absolutely continuous function, 545 as time tends to infinity, 311 equilibrium, 313 exists and unique, 313 explicit, 311

solution in the sense of Filippov, 543 solution of

differential equation, 1000 initial value problem, 1001

solving ARE, 457 eigenvector method, 457 Hamiltonian matrix, 457 Kučera conditions, 459 MacFarlane and Potter method, 457

solving simultaneously

Lyapunov inequalities, 947 SONC, see second-order sufficiency condition space

of functions, 193, 196 space of weights

minimization over, 813 spectrum

set of eigenvalues of a square matrix, 551 speed

of the curve, 1023 sphere

ball, 985 spiral chaos, 951 spring

linear, 215, 280

real, 215 spring-mass system, 151, 157 sprung mass, 111

vehicle body, 409 SQP, see sequential quadratic programming square block matrix, 416 square matrix, 963 diagonalize, 136

square of the error (ISE), 873 square root of matrix, 970 stability

asymptotic, 322 classical text on, 401 geometric interpretation, 318 in the sense of Lyapunov, 314 local concept, 317 of discrete systems, 401 of time-varying systems, 389

stability determination

using linearization, 406 stabilizability

of fuzzy models, 790 stabilizable

definition of, 228

pair, 337 stabilizable pair, 459 stabilization problem, 738 stabilizing control design Lyapunov based, 736

stabilizing controller

with adaptive neural term, 838 stabilizing controllers, 586

construction, 749 stabilizing linear state-feedback, 453 stable

asymptotically, 323, 324

based on linearized system, 362 stable limit cycle, 205 stacking operator, 334, 971 standard additive model

fuzzy system with centroidal defuzzifier,

719 standard additive model (SAM), 719 state, 5

not accessible for measurement, 258 state estimate, 250

to close the control loop, 258 state estimation error, 828, 836

dynamics of, 836 RBF identifier, 830

state estimators, see observers state feedback controller

design using LMIs, 427 state of charge (SOC), 294 state plane, 150, 510

methods, 150 state trajectory

desired , 374 state transition matrix, 122, 301, 922

of LTI system, 122

unique, 303

unique solution, 301 state variable

transformation, 229, 241 state variables, 5, 258 change, 269 estimates, 258

state vector, 23

state vector estimate, 572 state-feedback, 241, 250

linear, 250

Lyapunov-based, 411

optimal, 501

preliminary controller, 337 robust control, 336 state-feedback control, 375 state-feedback controller

linear, 339 state-feedback equivalent reference system, 374 state-space description, 23 state-space model

DC motor system, 272 state-space realization, 33, 40 state-transition matrix, 298, 308 state-variable

transformation, 265 statements

are either true or false, 956 steady-state, 193

amplitude, 193 phase, 193 steady-state behavior

of the closed-loop system, 692 steam-engine governor, 47, see Watt’s gover-

nor steepest descent algorithm, 407 steering angle, 76, see ground vehicle step function, 15 step lane change maneuver, 73 step length, 166 step response

delay time (dead-time) L of, 695

S-shaped, 695

steady-state gain (process gain) K of, 695 step-lane-change maneuver, 872

of a ground vehicle, 870 stick balancer, 91, see cart with an inverted

pendulum fuzzy model, 797

straight line

passing through fixed points, 1021 stress management, 105 stress management system

HPA axis, 367 strict inequalities, 418 strict local minimizer, 672 strictly proper rational function, 30 strictly proper transfer function, 33, 37 structure change

with current value of state, 535 structured representation of LMI, 419 subinterval

open, 195 submatrix, 966 subsequence

of a convergent sequence, 979

of the sequence, 978 subspace, 414, 990 subsystem, 2 successive derivatives, 221 successive differentiation, 164 successive iterations

predictor-corrector, 172 successive solutions

of Lyapunov equations, 329 sum

of two matrices, 963 of two vectors, 958 summation operation, 801 summing junction, 9 superposition property, 12, 13, 25 supervised mode

of learning, 840 support

of a function, 718 supremum

least upper bound, 973 surface

attractive, 540 surface of revolution area of, 435 example of a functional, 435 formed by a soap film, 435

suspension, 409 sustained oscillation

using P-controller, 698 SVD, see singular value decomposition switching curve, 510, 530

minimum time regulator problem, 520 switching logic, 535

between each of the structures, 537 switching surface, 380, 540 construction, 575 switching surface design, 546

system zeros locations, 558 Sylvester equation, see Sylvester matrix equa-

tion Sylvester matrix equation, 972 Sylvester’s inequalities, 636, 991, 1034 symmetric block matrices, 420 symmetric matrix, 968 symmetric positive definite matrix, 432 symmetrized expression, 422 symmetry

affine transformation, 914 synapse, 799 synaptic knobs, 799 synaptic vesicles, 801 synchronization

using fuzzy observer, 943 using observer, 943

synchronized systems, 943 system, 1, 11, 276

as an operator, 11 autonomous, 313 causal, 4, 19 continuous-time, 2 control, 6 DC motor, 276 discrete-time, 3, 25, 216 distributed, 6 dynamical, 5 electric motor, 1 in new coordinates, 288 inputs, 2 linear, 13 linear time-invariant, 20 lumped, 6, 22 mass-spring-damper, 207 modeling, 431 multi-input, 2 multi-input multi-output, 19 non-anticipatory, 4 non-autonomous, 313 nonlinear, 13 of p particles, 61 of linear equations, 994 output, 2 reachable, 216 relaxed, 14 single-input, 2 single-input single-output, 17 spring-mass, 151 time-delay, 6 time-invariant, 20, 313 time-varying, 313 two-mass-spring, 118 two-tank pumped storage, 110 unit-sampling-time delay, 27 unit-time delay, 22 system boundaries, 2 system dynamics in sliding, 547 system in sliding

governed by reduced set of differential equa-

tions, 551 reduced-order, 556 system in sliding poles system zeros, 556

system matrix, see Rosenbrock’s system ma-

trix, 556, 622, 636 system model

time-invariant, 188 when controllable, 280

system of LMIs, 415 system restricted

to the sliding surface, 553 system uncertainties

norm bound condition on, 838 system zero, 46 system zeros, 622, 636, 637

invariant under set of transformations, 556 systems

chaotic, 889

t-map, 905 Takagi-Sugeno (T-S) fuzzy model, see Takagi-

Sugeno-Kang (TSK) fuzzy model Takagi-Sugeno-Kang (TSK) fuzzy model, 736 tangent

to switching surface, 543 to the curve, 1023

tangent plane, 187 as a collection of derivatives, 674 to the surface, 674

tangent space, 675, 678

to the surface, 1024 tangent vector

of the curve, 1024 target tracking algorithm

fuzzy logic based, 888 Taylor formula

numerical methods, 163 Taylor Instrument Company

first PID controllers, 709 Taylor series, 163, 166, 182, 449

estimate of the error, 163 linearization, 189 of degree four, 182 of degree three, 182 predictor-corrector, 174 remainder, 163 several variables, 186

Taylor series solution, 163 Taylor’s formula

second-order, 671 Taylor’s linearization, 739, 741, 743 Taylor’s theorem, 362 technical condition

to ensure realizablity of observer, 631 temporal ordering, 3 terms of LMI system

lmiterm, 421 theorem

major result of independent interest, 956 theorem of Frobenius, 585 theorem of Lyapunov, 835 theorem of Weierstrass, 984, 988

for functions of several variables, 985 thereom of Green, 1030 third harmonic, 206 three-layer

feedforward neural network, 812 three-phase motor, 82 threshold

modified, 806 perceptron, 812 TLU, 803

threshold logic unit (TLU), 801 threshold parameters

as learning weights, 816 thrusters

satellite, 153 time

concept of, 293 time derivative

evaluated on trajectories, 365 on system trajectories, 747

time optimal controller, 493 time-delay system, 6, see distributed system time-dependent real-valued function, 351 time-invariant

system, 123 time-invariant system, 20 time-varying model, 293 time/frequency scaling property, 1015 tire dynamics, 108 tire slip, 77 tire slip angle, 871 TLU, see perceptron, see threshold logic unit

learning algorithm, 805

simplified learning problem, 807 training problem, 803

TLU convergence theorem, 808, 810 TLU’s role

find a separating hyperplane, 802 TLU (Perceptron) learning algorithm, 855 toilet flushing device, 47 tooth width, 83 torsional spring, 115 torsional spring coefficient, 115 total mechanical energy, 64 total response, 13 TPBVP, see two-point boundary value prob-

lem trace

of a square matrix, 965

sum of diagonal elements, 965 trace operator, 837 tracking control system

servomechanism, 11 tracking controller, 376 tracking controllers

adaptive fuzzy robust, 766 trained network, 817 training patterns, 841 training set, 803 trajectories’ divergence, 917 trajectory, 151, 890 slope, 159 transducer, 402 transfer function, 21

into state-space representation, 235 invariant under similarity, 235 proper rational, 235 rational proper, 35 sinusoidal, 205

transfer function matrix, 22, 24

positive real, 573 transform

into controller form, 252 transformation

constructing, 256 diffeomorphic, 592 equivalent, 285 into controller form, 245 into observer form, 600 similarity, 227 state-variable, 283 transformation matrix, 126 into controller form, 243

transient performance

of the closed-loop system, 692 transition matrix, 127 translation

in the time domain, 1015 transmission zeros, 45 transmitted signal, 947 transpose of a matrix, 963 of a product of matrices, 964 of the vector, 958

transversality condition, 452 transversality conditions, 451

end-point conditions, 449 trapezoidal rule

of integration, 171 predictor-corrector, 171

travel time

brachistochrone problem, 517 traveling salesman problem (TSP), 886 treatment strategy, 370

moving into negative stress region, 370 triangle inequality, 959, 962, 977 trigonometric identities, 168 trivial observer

for reduced-order method 2, 267 trivial subspace, 990 true state, 250 truth model, 7, 56, see simulation model, 736

to simulate, 795 truth table, 957 TSK fuzzy design model, 738 TSK fuzzy model, see Takagi-Sugeno-Kang

(TSK) fuzzy model TSK model, see Takagi-Sugeno-Kang (TSK)

fuzzy model TSP, see traveling salesman problem tumor dynamics, 108 turbine

producing electric power, 110 turbine supply pump, 110 turning maneuver, 73, see ground vehicle two input PM stepper motor, 91 two stage-by-stage passes, 488 two-class pattern

classification problem, 802 two-dimensional state space, see state plane two-link manipulator regulator for, 568 two-mass-spring, 118 two-point

crossover operator, 863 two-point boundary value problem (TPBVP),

521 two-tank pumped storage, 110 two-term controller, 754 type A PID control three terms, 693 type A PID controller

derivative time constant of, 694 integral time constant of, 694 parameterized by two parameters, T and

L, 697 proportional gain of, 693

type A PID controllers family of, 699 type A PIDF controller

discrete-time approximation, 709 type B PID controller, see PI-D controller,

710 type B PIDF controller digital, 711

type C PID controller, see I-PD controller,

711 type C PIDF controller digital, 711 practical implementation of I-PD, 706

u-map, 906 UIO, see unknown input observer UIO design

as LMI feasibility problem, 648 finding a triple of matrices, 641, 648

UIO of Edwards and Spurgeon, 649 uncertain dynamical systems, 749 uncertain element, 371 uncertain elements, 337, 338 uncertain system, 577 uncertain systems, 614 uncertainties

effect on stability, 336 uncertainty, 577

matched, 336 unmatched, 337

uncertainty growth, 890 uncontrollable

when, 283 uncontrollable system

transfer function of, 283 uncontrolled nominal system, 379 unforced fuzzy

system model, 745 unforced fuzzy model, 744 unforced LTV system, 298 uniform asymptotic stability

in the large, 764 uniform boundedness

definition of, 379 uniform convergence, 360, 993 uniform exponential stability, 317 uniformly asymptotically stable, 317

conditions for, 353 uniformly asymptotically stable in the large

conditions for, 358

definition of, 358 uniformly continuous, 775, 839, 986 size of changes in t, 390 uniformly exponentially stable conditions for, 357 uniformly stable ball, 385, 411 uniformly ultimately bounded (UUB), 379

definition of, 379 union

of fuzzy sets, 717 unipolar representation

of XOR function, 815 unique fixed point, 998 unit impulse function, 15 unit step

input, 306 unit step function, 15, 728, 1034

denoted 1(t), 1006 unit-sampling-time delay, 27 unit-step response, 275 unit-time delay system, 22 unitary matrix, see orthogonal matrix universal properties

of quadratic maps, 907 universe, 716

a fixed set, 956 of discourse, 716

universe of discourse, see universe

a fixed set, 956 unknown input, 109, 613 unknown input observer (UIO), 614 unknown input reconstruction, 633 unmatched uncertainty, 337, 793 unmodeled system dynamics, 613 unrestricted growth rate, 103, 891 unspecified scalar, 226 unsprung mass, 111

tire and axle, 409 unstable

based on linearized system, 362 unstable equilibrium, 365 unsupervised mode

of learning, 840 update formulas

for the weights, 814 update law

for weights of RBF network, 835 upper bound, 973 UUB, see uniformly ultimately bounded UUB with respect to aball, 383

v-map, 905 vagueness

in terms of symbolic logic, 790 van der Pol equation, 160

Bendixson’s theorem, 162 variable force element, 111, 409 variable structure

controller-observer compensators, 574 sliding mode systems, 574

variable structure system, 535, 574 composed of independent structures, 535 phase portrait, 574

variable structure systems

early pioneers, 573 variation, 436 variation in one coordinate, 447 variation of a functional, 437

analog of function differential, 437 variation of the functional, 438 variations of the functionals, 524 vector

an ordered set of scalars, 958

column, 958 dimension of, 958 n-tuple, 958 vector addition

associative, 958 commutative, 959 vector field, 109, 580, 921

as a field of forces, 1025 vector fields

in controller construction, 579 vector norm, 959

uniformly continuous function, 988 vector-valued function, 347 vectorization operator, 971 vectors

linearly independent, 124 vehicle body, 409 vehicle coordinate system, 73 vehicle model, 286, 376, 724 vehicle modeling, 108 vehicle suspension, 285 velocity vector

of the curve, 1023 Verhulst equation, 103, see logistic equation vibrating plate, 937 vicinity, 188 viscous friction, 113 coefficient of, 113

voltage source, 4 von Koch curve

dimension of, 910 von Koch snowflake, 908 Vyshnegradskiy

design rules for governors, 346

Watt’s application

start of modern control, 279 Watt’s governor, 9, 47

engine-governor, 192 linearize the model, 192

Webster’s New World Dictionary, 857 Weierstrass

theorem of, 988 Weierstrass theorem, 347 weight estimation error, 836 weight matrices, 520 weight matrix, 465, 466 weight vector

TLU, 803 weighting functions, 753 well defined

function, 347 Wiener, Norbert, 9

feedback definition, 9 winner-take-all (WTA) network, 841 winning neuron, 840, 841 wireless communication, 941 work, 58

done against the force field, 1030 work-energy theorem, 59 WTA algorithm, 855 WTA network, see winner-take-all (WTA) net-

work

XOR, see Exclusive-OR XOR function

unipolar representation of, 815 XOR problem

two-input, 815 YALMIP

LMIs solving, 426 yaw velocity, 871 Yet Another LMI Package (YALMIP), 426

z-map, 904, 905 Z-transform, 1016

a linear operator, 1018

is a series, 1016

of a delayed sequence, 1018 of an advanced sequence, 1019 of the delayed geometric sequence, 1019 of the geometric sequence, 1017 of the sampled unit ramp, 1017 of the sampled unit step, 1016 solving linear constant-coefficients differ-

ence equations, 1019 z-transform, 26, 28 Z-transfrom

inverse, 1020 zero control, 218 zero matrix

denoted O, 963 zero vector, 959, 990, 992 zero-input response, 13 zero-order hold, 680, 709, 711

in controller implementation, 711 zero-order hold (ZOH), 148 zero-state response, 13, 27, 306 zeros, 44

in open left-half plane, 345 of a transfer function, 44 of rational function and initial conditions,

330 with positive real parts, 345

Ziegler and Nichols closed-loop method, 698 open-loop method, 695

Ziegler and Nichols PID tuning rules, 695 ZOH, see zero-order hold