Prentice Hall Advanced Reference Series ze Engineering Denno Power System Design and Applications for Alternative Energy Sourees Fory, AKERS, AND GREENRICH Ultra Large Scale Integrated Microelectronics Havin, Ep. Selected Topics in Signal Processing JouNsON Lectures on Adaptive Parameter Estimation Muurinovic, Ep Microprocessor Design for GaAs Technology QuacKENBUSH, BARNWELL III, AND CLEMENTS Objective Measures of Speech Quality RorFeL, VERMEER, AND CHIN. Simulation and Implementation of Self-Tuning Controllers Sastry aNd Booson Adaptive Control: Stability, Convergence, and RobustnessPRENTICE HALL INFORMATION AND SYSTEM SCIENCES SERIES Thomas Kailath, Editor ANDERSON & Moone Agro & WITTENMARK GaroxeR Goopwin & Sin Grav & Davison Haves Jaws SoHNSON Kalcati Kono Kono, Wuirenouse, & Kararn, Liuxe Macovsx Mesa & Sac: SastRy & BODsON Spuuker Wittians Optimal Filtering Computer-Controlled Systems: Theory and Design Statistical Spectral Analysis: A Nonprobabilistic Theory Adaptive Filtering, Prediction, and Control Random Provesses: A Mathenatical Approach for Engineers Adaptive Filter Theory Fundamentals of Digital Imege Processing Lectures on Adaptive Parameter Estimation Linear Systems VLSI Array Processors VLSI and Modern Signal Processing System Identification: Theory for the User Medical Imaging Systems ‘An Introduction to Probability and Stochastic Processes Adaptive Control: Stability, Convergence, and Robustness Digital Communications by Satellite Designing Digital Filters ADAPTIVE CONTROL Stability, Convergence, and Robustne: Shankar Sastry University of California at Berkeley Mare Bodson Carnegie Mellon University Prentice Hall, Englewood Cliffs, New Jersey 07632Library of Congress Cataloging Pubic Data St: St Se Sa Sins Gal ae Mare Betas oe toe tps Ha snc ees te) 2 7 SSS iy ty, en nt ems, 1. aon Me Te Ha SF 1099 samo cao de cir Prentice-Hall Advanced Reference Series Prentice-Hall Information and System Sciences Series EG © 1989 by Promicestial, Ine A Division of Simon & Schuster Englewood Chiff, New Jersey 07692 All rights reserved. No part of this hook may be reproduced, in any form or by any meens ‘without permission in writing from the publisher Printed in the United States of America wosT6 54321 ISBN O-13-00432b-5 Prentice-Hall erations (UK) Limited, London Prentice-Hall of Austalia Py Limited, Svdaey Prentice-Hall Canada Ine. Toronto Prentice-Hall Hispanoamerican, S.A, Mexico Prentice-Hall of Inia Private Limited, New Delhi Ponte Hal of pan, Ie Tyo Simon & Schuster Asia Pie LU. Singapore Editors Premice-Hall do Bras Lida. Rio de Janeiro To our parentsCON’ NTS Preface xv Chapter 0. Introduction (0.1 Identification and Adaptive Control 0.2. Approaches to Adaptive Control 0.2.1 Gain Scheduling 0.2.2. Model Reference Adaptive Systems 0.2.3 Self Tuning Regulators 0.2.4 Stochastic Adaptive Control 0.3. A Simple Example Chapter 1 Pretiminaries LLL Notation 1.2. Ly Spaces, Norms 1:3. Positive Definite Matrices 1.4 Stability of Dynamic Systems 1.4.1 Differential Equations 1.4.2. Stability Definitions 1.4.3 Lyapunov Stability Theory LS Exponential Stability Theorems 1.5.1 Exponential Stability of Nonlinear Systems 1.5.2. Exponential Stability of Linear Time-Varying Systems 32 1.5.3 Exponential Stability of Linear Time Invariant Systems 38 1.6 Generalized Harmonie Analysis 39Chapter 2. Identification 2.0 2.8 Introduction Identification Problem Identifier Structure Linear Error Equation and Identification Algorithms 2.3.1 Gradient Algorithms 2.3.2 Least-Squares Algorithms Properties of the Identification Algorithms— Identifier Stability 2.4.1 Gradient Algorithms 4.2 Least-Squares Algorithms 4.3 Stability of the Identifier Persistent Excitation and Exponential Parameter Convergence Model Reference Identifiers—SPR Error Equation 2.6.1, Model Reference Identifiers 2.6.2 Strictly Positive Real Error Equation and Identification Algorithms 2.6.3 Exponential Convergence of the Gradient Algorithms with SPR Error Equations Frequency Domain Conditions for Parameter Convergence 2.7.1 Parameter Convergence 2.7.2 Partial Parameter Convergence Conclusions Chapter 3. Adaptive Control 3.0 3 3.2 33 Introduction Model Reference Adaptive Control Problem Controller Structure Adaptive Control Schemes 3.3.1 Input Error Direct Adaptive Control 3.3.2 Output Error Direct Adaptive Conirol 3.3.3. Indirect Adaptive Control 3.3.4. Alternate Model Reference Schemes 3.3.5. Adaptive Pole Placement Control The Stability Problem in Adaptive Control Analysis of the Model Reference Adaptive Control System Useful Lemmas Stability Proofs 3.7.1 Stability—Input Error Direct Adaptive Control 3.7.2. Stability—Output Error Direct Adaptive Control 3.7.3 Stability—Indirect Adaptive Control 45 45 52 37 58 61 63 63 66 69 1 16 16 85 90 90 95 98 99 99 103 104 10 ut 118. 123, 127 129 130 132 138 142 142 149 ist 3.8 Exponential Parameter Convergence 3.9 Conclusions Chapter 4 Parameter Convergence Using Averaging Techniques 4.0 Introduction 4.1. Examples of Averaging Ana 4.2. Averaging Theory—One-Time Scale 4.3 Application to Identification 4.4 Averaging Theory—Two-Time Scales 4.4.1 Separated Time Scales 4.4.2 Mixed Time Scales 4.5. Applications to Adaptive Control 4.5.1 Output Error Scheme—Linearized Equations 4.5.2 Output Error Scheme—Nonlinear Equations 4.5.3. Input Error Scheme 4.6. Conclusions Chapter 5 Robustness 5.1 Structured and Unstructured Uncertainty 5.2. The Rohrs Examples 5.3. Robustness of Adaptive Algorithms with Persistency of Excitation 5.3.1 Exponential Convergence and Robustness 5.3.2. Robustness of an Adaptive Control Scheme 5.4 Heuristic Analysis of the Rohrs Examples 5.5. Averaging Analysis of Slow Drift Instability 5.5.1 Instability Theorems Using Averaging 5.5.2 Application to the Output Error Scheme 5.6 Methods for Improving Robustness— Qualitative Discussion 5.6.1 Robust Identification Schemes 5.6.2 Specification of the Closed Loop Control Objective—Choice of Control Model and of Reference Input 5.6.3. The Usage of Prior Information 5.6.4 Time Variation of Parameters 5.7 Robustness via Update Law Modifications 5.7.1 Deadzone and Relative Deadzone 5.7.2 Leakage Term (o—Modification) 5.7.3 Regressor Vector Filtering 5.7.4 Slow Adaptation, Averaging and Hybrid Update Laws, 5.8. Conclusions 184 156 158 158, 159 166 175 179 183, 186 187 188. 192 202 207 209 209 215 219 221 225 231 236 236 241 248 248 250 250 251 251 251 233 253 234 254Chapter 6 Advanced Topics in Identification and Adaptive Control 6.1 Use of Prior Information 6.1.1 Identification of Partially Known Systems 6.1.2 Effect of Unmodeled Dynamics 6.2. Global Stability of Indirect Adaptive Control Schemes, 6.2.1 Indirect Adaptive Control Scheme 6.2.2 Indirect Adaptive Pole Placement 6.2.3 Indirect Adaptive Stabilization— The Factorization Approach 6.3 Multivariable Adaptive Control 6.3.1 Introduction 6.3.2. Preliminaries 6.3.2.1 Factorization of Transfer Function Matrices 6.3.2.2 Interactor Matrix and Hermite Form 6.3.3 Model Reference Adaptive Control— Controller Structure 6.3.4 Model Reference Adaptive Control— Input Error Scheme 6.3.5 Alternate Schemes 6.4 Conclusions Chapter 7 Adaptive Control of a Class of Nonlinear Systems 7.1 Introduction 7.2. Linearizing Control for a Class of Nonlinear Systems— A Review 7.2.1 Basic Theory 7.2.2 Minimum Phase Nonlinear Systems 7.2.2.1 The Single-Input Single-Ovtput Case 7.2.2.2 The Multi-Input Multi-Ouiput Case 7.2.3. Model Reference Control for Nonlinear Systems 7.3. Adaptive Control of Linearizable Minimum Phase Systems 7.3.1. Single-Input Single-Output, Relative Degree One Case 7.3.2. Extensions to Higher Relative Degree SISO Systems 7.3.3 Adaptive Control of MIMO Systems Decouplable by Static State Feedback 7.4 Conclusions Chapter 8 Conclusions 8.1 General Conclusions 8.2 Future Research Appendix References IndexPREFACE The objective of this book is to give, in a concise and unified fashion, the major results, techniques of analysis and new directions of research in adaptive systems. Such a treatment is particularly timely, given the rapid advances in microprocessor and multi-processor technol- ogy which make it possible to implement the fairly complicated non- linear and time varying control laws associated with adaptive control Indeed, limitations to future growth can hardly be expected to be com putational, but rather from a lack of a fundamental understanding of the methodologies for the design, evaluation and testing of the algorithms. Our objective has been to give a clear, conceptual presentation of adap- tive methods, to enable a critical evaluation of these techniques and sug- gest avenues of further development. Adaptive control has been the subject of active research for over three decades now. There have been many theoretical successes, includ- ing the development of rigorous proofs of stability and an understanding of the dynamical properties of adaptive schemes. Several successful applications have been reported and the last ten years have seen an impressive growth in the availability of commercial adaptive controllers. In this book, we present the deterministic theory of identification and adaptive control. For the most part the focus is on linear, continu- fous time, single-input single-output systems. The presentation includes the algorithms, their dynamical properties and tools for analysis— including the recently introduced averaging techniques, Current research in the adaptive control of multi-input, multi-output linear systems and axvi Preface class of nonlinear systems is also covered. Although continuous time algorithms occupy the bulk of our interest, they are presented in such a way as to enable their transcription to the discrete sime case. A brief outline of the book is as follows: Chapter 0 is a brief his- torical overview of adaptive control and identification, and an introduc- tion to various approaches. Chapter I is a chapter of mathematical pre- liminaries containing most of the key stability results used later in the book. In Chapter 2, we develop several adaptive identification algo- rithms along with their stability and convergence properties. Chapter 3 is a corresponding development for model reference adaptive control. In Chapter 4, we give a self contained presentation of averaging techniques and we analyze the rates of convergence of the schemes of Chapters 2 and 3. Chapter 5 deals with robustness properties of the adaptive schemes, how to analyze their potential instability using averaging tech- niques and how to make the schemes more robust. Chapter 6 covers some advanced topics: the use of prior information in adaptive identification schemes, indirect adaptive control as an extension of robust non-adaptive control and multivariable adaptive control. Chapter 7 gives a brief introduction to the control of a class of nonlinear systems, explicitly linearizable by state feedback and their adaptive con- trol using the techniques of Chapter 3. Chapter 8 concludes with some of our suggestions about the areas of future exploration, This book is intended to introduce researchers and practitioners to the current theory of adaptive control, We have used the book as a text several times for a one-semester graduate course at the University of California at Berkeley and at Carnegie-Mellon University. Some back- ground in basic control systems and in linear systems theory at the gra- duate level is assumed, Background in stability theory for nonlinear sys- tems is desirable, but the presentation is mostly sel-contained, Acknowledgments It is a pleasure to acknowledge the contributions of the people who helped us in the writing of this book. We are especially appreciative of the detailed and thoughtful reviews given by Charles Desoer of the orii- nal Ph.D. dissertation of the second author on which this book is based. His advice and support from the beginning are gratefully acknowledged. Brian Anderson and Petar Kokotovic offered excellent critical comments, that were extremely helpful both in our research, and in the revisions of the manuscript. We also thank Karl Astrom and Steve Morse for their enthusiasm about adaptive control, and for fruitful interactions, The persistently exciting inputs of students at Berkeley and Came- gic Mellon have helped us refine much of the material of this book. We are especially thankful to those who collaborated with us in research, Preface xvii and contributed to this work: Erwei Bai, Stephen Boyd, Michel de Mathelin, Li-Chen Fu, Ping Hsu, Jeff Mason, Niklas Nordstrom, Andy Packard, Brad Paden and Tim Salcudean. Many of them have now adapted to new environments, and we wish them good luck, We are indebted to many colleagues for stimulating discussions at conferences, workshops and meetings. They have helped us broaden our view and understanding of the field. We would particularly like to met tion Anu Annaswamy, Michael Athans, Bob Bitmead, Soura Dasgupta, Graham Goodwin, Petros loannou, Alberto Isidori, Rick Johnson, Ed Kamen, Bob Kosut, Jim Krause, Rogelio Lozano-Leal, Iven Mareels. Sanjoy Mitter, Bob Narendra, Dorothee Normand-Cyrot, Romeo Ortega. Laurent Praly, Brad Riedle, Charles Rohrs, Fathi Salam and Lena Val We acknowledge the support of several organizations, including NASA (Grant NAG-243), the Army Research Office (Grant DAAG 29- 85-K0072), the IBM Corporation (Faculty Development Award), and the National Science Foundation (Grant ECS-8810145). Special thanks are due to George Meyer and Jagdish Chandra: their continuous support of our research made this book possible. We are also grateful for the logistical support received from the administration of the Department of Electrical Engineering and Com- puter Sciences at the University of California at Berkeley and of the Blectrical and Computer Engineering Department of Carnegie Mellon University. Part of our work was also done at the Laboratory for Infor- mation and Decision Systems, in the Massachussetts Institute of Tech- nology, thanks to the hospitality of Sanjoy Miter. We wish to express our gratitude to Carol Block and Tim Burns for their diligent typing and layout of the manuscript in the presence of uncertainty. The figures were drafted by Osvaldo Garcia, Cynthia Bit- brey and Craig Louis. at the Electronics Research Laboratory at Berke- ley. Simulations were executed using the package SIMNON, and we thank Karl Astrom for providing us with a copy of this software pack- age. We also acknowledge Bernard Goodwin of Prentice Hall for his friendly management of this enterprise and Elaine Lynch for coordinat- ing production matters. Last, but not least, we would like to express our sincere apprecia- tion to Nirmala and Cecilia for their patience and encouragement Despite distance, our families have been a source of continuous support and deserve our deepest gratitude. Shankar Sastry Berkeley, California Mare BodsonCHAPTER 0 INTRODUCTION 0.1 IDENTIFICATION AND ADAPTIVE CONTROL Most current techniques for designing control systems are based on a good understanding of the plant under study and its environment. How- ever, in a number of instances, the plant to be controlled is too complex and the basic physical processes in it are not fully understood. Control design techniques then need to be augmented with an identification tech- nique aimed at obtaining a progressively better understanding of the plant to be controlled. It is thus intuitive to aggregate system identification and control. Often, the two steps will be taken separately If the system identification is recursive—that is the plant model is periodically updated on the basis of previous estimates and new data— identification and control may be performed concurrently. We will see adaptive control, pragmatically, as a direct aggregation of a (non. adaptive) control methodology’ with some form of recursive system identification. Abstractly, system identification could be aimed at determining if the plant to be controlled is linear or nonlinear, finite or infinite dimen- sional, and has continuous or discrete event dyniamics. Here we will res. trict our attention to finite dimensional, single-input single-output linear plants, and some classes of multivariable and nonlinear plants. Then, the Primary step of system identification (structural identification) has already been taken, and only parameters of a fixed type of model need to be determined. Implicitly, we will thus be limiting ourselves to parametric system identification, and parametric adaptive control.2 Introduction Applications of such systems arise in several contexts: advanced flight control systems for aircraft or spacecraft, robot manipulators, process control, power systems, and others. Adaptive control, then, is a technique of applying some system identification technique to obtain a model of the process and its environ- ‘ment from input-output experiments and using this model to design a controller. The parameters of the controller are adjusted during the operation of the plant as the amount of data available for plant identification increases. For a number of simple PID (proportional + integral + derivative) controllers in process control, this is often done manually. However, when the number of parameters is larger than three or four, and they vary with time, automatic adjustment is needed. The design techniques for adaptive systems are studied and analyzed in theory for unknown but fixed (shat is, time invariant) plants. In practice, they are applied to slowly time-varying and unknown plants. Overview of the Literature Research in adaptive control has a long and vigorous history. In the 1950s, it was motivated by the problem of designing autopilots for air- craft operating at a wide range of speeds and altitudes, While the object of a good fixed-gain controller was to build an autopilot which was insensitive to these (large) parameter variations, it was frequently observed that a single constant gain controller would not suffice. Conse- quently, gain scheduling based on some auxiliary measurements of airspeed was adopted. With this scheme in place several rudimentary model reference schemes were also attempted—the goal in this scheme was to build a self-adjusting controller which yielded a closed loop transfer function matching a prescribed reference model. Several schemes of self-adjustment of the controller parameters were proposed, such as the sensitivity rules and the so-called MIT. rule, and were verified to perform well under certain conditions. Finally, Kalman [1958] put on a firm analytical footing the concept of a general self- tuning controller with explicit identification of the parameters of linear, single-input, single-output plant and the usage of these parameter estimates to update an optimal linear quadratic contreller, The 1960s marked an important time in the development of con- trol theory and adaptive control in particular. Lyapunov's stability theory was firmly established as a tool for proving convergence in adap- tive control schemes, Stochastic control made giact strides with the understanding of dynamic programming, due to Bellman and others. Learning schemes proposed by Tsypkin, Feldbaum and others (see Tsyp- kin [1971] and [1973}) were shown to have roots in a single unified framework of recursive equations. System identification (offline) was ‘Section 0.1 Identification and Adaptive Control 3 thoroughly researched and understood. Further, Parks [1966] found a way of redesigning the update laws proposed in the 1950s for model reference schemes so as to be able to prove convergence of his controller, In the 1970s, owing to the culmination of determined efforts by several teams of researchers, complete proofs of stability for several ‘adaptive schemes appeared. State space (Lyapunov based) proofs of sta- bility for model reference adaptive schemes appeared in the work of Narendra, Lin, & Valavani [1980] and Morse [1980]. In the late 1970s, input output (Popov hyperstability based) proofs appeared in Egardt [1979] and Landau [1979]. Stability proofs in the discrete time deter- ministic and stochastic case (due to Goodwin, Ramadge, & Caines [1980}) also appeared at this time, and are contained in the textbook by Goodwin & Sin [1984]. Thus, this period was marked by the culmina- tion of the analytical efforts of the past twenty years, Given the firtu, analytical footing of the work to this point, the 1980s have proven to be a time of critical examination and evaluation of the accomplishments to date. It was first pointed out by Rohrs and co- workers [1982] that the assumptions under which stability of adaptive schemes had been proven were very sensitive to the presence of unmo- deled dynamics, typically high-frequency parasitic modes that were neglected to limit the complexity of the controller. This sparked a flood of research into the robustness of adaptive algorithms: a re-examination of whether or not adaptive controllers were at least as good as fixed gain controllers, the development of tools for the analysis of the transient behavior of the adaptive algorithms and attempts at implementing the algorithms on practical systems (reactors, robot manipulators, and ship steering systems to mention only a few). The implementation of the ‘complicated nonlinear laws inherent in adaptive control has been greatly facilitated by the boom in microelectronics and today, one can talk in terms of custom adaptive controller chips. All this flood of research and. development is bearing fruit and the industrial use of adaptive control is srowing, Adaptive control has a rich and varied literature and it is impossi- ble to do justice to all the manifold publications on the subject. It is a tribute to the vitality of the field that there are a large number of fairly recent books and monographs. Some recent books on recursive estima tion, which is an important part of adaptive control are by Eykhoft [1974], Goodwin & Payne (1977), Ljung & Soderstrom [1983] and Ljung [1987]. Recent books dealing with the theory of adaptive control are by Landau [1979], Egardt [1979], loannou & Kokotovic [1984], Goodwin & Sin [1984], Anderson, Bitmead, Johnson, Kokotovie, Kosut, Mareels, Praly, & Riedle [1986], Kumar and Varaiya [1986], Polderman [1988] and Caines [1988]. An attempt to link the signal processing viewpoint4 Introduction with the adaptive control viewpoint is made in Johnson [1988]. Surveys Of the applications of adaptive control are given in a book by Harris & Billings [1981], and in books edited by Narendra & Monopoli [1980] and Unbehauen {1980}. As of the writing of this book, two other books on adaptive control by Astrom & Wittenmark and Narendra & ‘Annaswamy are also nearing completion, In spite of the great wealth of literature, we feel that there is a need for a “toolkit” of methods of analysis comparable to non-adaptive linear time invariant systems. Further, many of the existing results concern cither algorithms, structures or specific applications, and a great deal more needs to be understood about the dynamic behavior of adaptive systems. This, we believe, has limited practical applications more than it should have, Consequently, our objective in this book is 10 address fundamental issues of stability, convergence and robustness. Also, we hope to communicate our excitement about the problems and potential of adaptive control. In the remainder of the introduction, we will review some common approaches to adaptive control systems and introduce the basic issues studied in this book with a simple example. 0.2 APPROACHES TO ADAPTIVE CONTROL 0.2.1 Gain Scheduling One of the earliest and most intuitive approaches to adaptive control is, gain scheduling. It was introduced in particular in the context of flight control systems in the 1950s and 1960s. The idea is to find auxiliary process variables (other than the plant outputs used Zor feedback) that correlate well with the changes in process dynamics, It is then possible to compensate for plant parameter variations by changing the parame- ters of the regulator as functions of the auxiliary variables, This is illus- rated in Figure 0.1 in Avaany scheduar — wehSERBB comtano /contnonten SIGNAL F TF ourTPUT Conroe | CONTRO pn i Figure 04: Gain Scheduling Controle Section 0.2 Approaches to Adaptive Control 5 The advantage of gain scheduling is that the parameters can be changed quickly (as quickly as the auxiliary measurement) in response to changes in the plant dynamics. It is convenient especially if the plant dynamics depend in a well-known fashion on a relatively few easily measurable variables, In the example of flight control systems, the dynamics depend in relatively simple fashion on the readily available dynamic pressure—that is the product of the air density and the relative velocity of the aircraft squared, Although gain scheduling is extremely popular in practice, the disadvantage of gain scheduling is that it is an open-loop adaptation scheme, with no real “learning” or intelligence. Further, the extent of design required for its implementation can be enormous, as was illus- ‘trated by the flight control system implemented on a CH-47 helicopter. The fight envelope of the helicopter was divided into ninety flight condi- tions corresponding to thirty discretized horizontal flight velocities and three vertical velocities. Ninety controllers were designed, correspond- ing to each flight condition, and a linear interpolation between these controllers (linear in the horizontal and vertical flight velocities) was Programmed onto a flight computer. Airspeed sensors modified the con- trol scheme of the helicopter in flight, and the effectiveness of the design was corroborated by simulation, 0.2.2 Model Reference Adaptive Systems Again in the context of flight control systems, two adaptive control schemes other than gain scheduling were proposed to compensate for changes in aircraft dynamics: a series, high-gain scheme, and a parallel scheme. Series Figure 0.2 shows a schematic of the series high-gain scheme. Reredence outed SIGNAL | i ain Lint cyct : Asjstment |" Oatector Figure 0.2: Model Reference Adaptive Control—Series, High-Gain Scheme6 Introduction The reference model represents a pilot's desired command-response characteristic. It is thus desired that the aircraft response, that is, the ‘output yp, matches the output of the reference model, that is, Yn The simple analysis that goes into the scheme is as follows: con- sider P(s) to be the transfer function of the linear, time invariant plant and k the constant gain of the servo, The transfer function from jy 10 Jy is KP(s\/1+k Pls). When the gain k is sufficiently large, the transfer function is approximately 1 over the frequencies of interest, so that yn ~ Jp ‘The aim of the scheme is to let the gain k be as high as possible, so that the closed-loop transfer function becomes close to 1, until the onset of instability (e limit cycle) is detected. If the limit cycle oscillations exceed some level, the gain is decreased. Below this level, the gain is increased. The limit cycle detector is typically just a rectifier and low- pass filter The series high-gain scheme is intuitive and simple: only one parameter is updated. However, it has the following problems 8) Oscillations are constantly present in the system. b) Noise in the frequency band of the limit eyele detector causes the tain to decrease well below the critical value, ©) Reference inputs may cause saturation due 10 the high-gain, 4) Saturation may mask limit cycle oscillations, allowing the gain to increase above the critical value, and leading to instability. Indeed, tragically, an experimental X-15 aircraft flying this control sys- tem crashed in 1966 (cf. Staff of the Flight Research Center [1971), owing panilly to the saturation problems occurring in the high-gain Scheme. ‘The roll and pitch axes were controlled by the right and left rear ailerons, using differential and identical commands respectively. ‘The two axes were assumed decoupled for the purpose of control design, However, saturation of the actuators in the pitch axis caused the aircraft to lose controllability in the roll axis (since the ailerons were at max- imum deflection). Due to the saturation, the instability remained undetected, and created aerodynamic forces too great for the aircraft to withstand, Parallel Scheme As in the series scheme, the desired performance of the closed-loop sys- tem is specified through a reference model, and the adaptive system attempts to make the plant output match the reference model output asymptotically. An early refere.ice to this scheme is Osburn, Whitaker, & Kezer [1961]. A block diagram is shown in Figure 0.3. The controller Section 0.2 Approaches to Adaptive Control 7 Params] MECN o outea 00 Figure 0.3: Model Reference Adaptive Control—Parallel Scheme can be thought of as having two loops: an inner or regulator loop that is an ordinary control loop consisting of the plant and regulator, and an Outer or adaptation loop that adjusts the parameters of the regulator in Such a way as to drive the error between the model output and plant output to zero, ‘The key problem in the scheme is to obtain an adjustment mechan- ism that drives the output error eo y,~Ym to zero. In the earliest applications of this scheme, the following update, called the gradient update, was used. Let the vector 6 contain the adjustable parameters of the controller. The idea behind the gradient update is to reduce e? (0) by adjusting @ along the direction of steepest descent, that is de a a” (8) 02.1) ~28e00 5 (eo) = -2e2 (4) 02.2) where g is a positive constant called the adaptation gain. The interpretation of e9(@) is as follows: it is the output error (also a function of time) obtained by freezing the controller parameter at 9. The gradient of eo(@) with respect to 0 is equal to the gradient of y, with respect to 4, since ym is independent of 4, and represents the sensitivity of the output error to variations in the controller parameter @8 Introduction Several problems were encountered in the usage of the gradient update. The sensitivity function dy, (0)/0 usually depends on the unk- nown plant parameters, and is consequently unavailable. At this point the so-called M.LT. rule, which replaced the unknown parameters by their estimates at time 1, was proposed. Unforturately, for schemes based on the M.LT. rule, it is not possible in general to ‘prove closed- loop stability, or convergence of the output error to zéro. Empirically, it was observed that the M.LLT. rule performed well when the adaptation gain g and the magnitude of the reference input were small (a conclusion later confirmed analytically by Mareels et a/ (1986)). However, examples of instability could be obtained otherwise (cf. James [1971)). Parks [1966] found a way of redesigning adaptive systems using Lyapunov theory, so that stable and provably convergent model refer- fence schemes were obtained. The update laws were similar to (0.2.2), with the sensitivity dy, (0)/28 replaced by other functions, The stability and convergence properties of model reference adaptive systems make them particularly attractive and will occupy a lot of our interest in this book. 0.2.3 Self Tuning Regulators In this technique of adaptive control, one starts from a control design method for known plants, This design method is summarized by a con troller structure, and a relationship between plant parameters and con- troller parameters. Since the plant parameters are in fact unknown, they are obtained using a recursive parameter identification algorithm. The controller parameters are then obtained from the estimates of the plant parameters, in the same way as if these were the true parameters. This is usually called a certainty equivalence principle, The resulting scheme is represented on Figure 0.4. An explicit separation between identification and control is assumed, in contrast to the model reference schemes above, where the parameters of the con- troller are updated directly to achieve the goal of model following. ‘The self tuning approach was originally proposed by Kalman [1958] and clarified by Astrom & Wittenmark [1973]. The controller is called self tuning, since it has the ability to tune its own parameters. Again, it can be thought of as having two loops: an inner loop consisting of a conven- tional controller, but with varying parameters, and an outer loop consist- ing of an identifier and a design box (representing an on-line solution to a design problem for a system with known parameters) which adjust these controller parameters, The self tuning regulator is very flexible with respect to its choice of controller design methodology (linear quadratic, minimum variance, ‘gain-phase margin design, ...), and to the choice of identification scheme Section 0.2 Approaches to Adaptive Control 9 Aeterance 2] consrowen PUNT INNER LOOP Convote 9 Beramoter VoeniFieR Conraowen]|_ouren voor] dentin, besiGn fo aE Figure 0.4: Self-tuning Controller (east squares, maximum likelihood, extended Kalman filtering, ..). The analysis of self tuning adaptive systems is however more complex than the analysis of model reference schemes, due primarily to the (usually nonlinear) transformation from identifier parameters to controller parameters. Direct and Indirect Adaptive Control While model reference adaptive controllers and self tuning regulators were introduced as different approaches, the only real difference between them is that model reference schemes are direct adaptive control schemes, whereas self tuning regulators are indirect. The self tuning regulator first identifies the plant parameters recursively, and then uses these estimates to update the controller parameters through some fixed transformation. The model reference adaptive schemes update the con- troller parameters directly (no explicit estimate or identification of the plant parameters is made). It is easy to see that the inner or control loop of a self tuning regulator could be the same as the inner loop of a model reference design. Or, in other words, the model reference adap- tive schemes can be seen as a special case of the self tuning regulators, with an identity transformation between updated parameters and con. troller parameters. Through this book, we will distinguish between direct and indirect schemes rather than between model reference and self ‘tuning algorithms.10 Introduction 0.2.4 Stochastic Control Approach Adaptive controller structures based on model reference or self tuning approaches are based on heuristic arguments. Yet, it would be appealing to obtain such structures from a unified theoretical framework. This can >be done (in principle, at least) using stochastic control. The system and its environment are described by a stochastic model, and a criterion is, formulated to minimize the expected value of a loss function, which is a scalar function of states and controls. It is usually very difficult to solve stochastic optimal control problems (a notable exception is the linear quadratic gaussian problem). When indeed they can be solved, the optimal controllers have the structure shown in Figure 0.5: an identifier (estimator) followed by a nonlinear feedback regulator, Figure 0.5: “Generic” Stochastic Controller The estimator generates the conditional probability distribution of the state from the measurements: this distribution is called the Ayperstate (usually belonging to an infinite dimensional vector space). The self tuner may be thought of as an approximation of this controller, with the hyperstate approximated by the process state and process parameters estimate, From some limited experience with stochastic control, the following interesting observations can be made of the optimal control law: in addition to driving the plant output to its desired value, the controller introduces probing signals which improve the identification and, there- fore future control. This, however, represents some cost in terms of con- trol activity. The optimal regulator maintains a balance between the control activity for learning about the plant it is controlling and the activity for controlling the plant output to its desired value. This pro- perty is referred to as dual control. While we will rot explicitly study stochastic control in this book, the foregoing trade-off will be seen repeatedly: good adaptive control requires correct identification, and for the identification to be complete, the controller signal has to be sufficiently rich to allow for the excitation of the plant dynamics. The Section 0.2 Approaches to Adaptive Control ul presence of this rich enough excitation may result in poor transient per- formance of the scheme, displaying the trade-off between learning and control performance. 0.3. A SIMPLE EXAMPLE Adaptive control systems are difficult to analyze because they are non- linear, time varying systems, even if the plant that they are controlling is linear, time invariant, This leads to interesting and delicate technical problems. In this section, we will introduce some of these problems with a simple example. We also discuss some of the adaptive schemes of the previous section in this context. We consider a first order, time invariant, linear system with transfer function ky sta 1) where a>0 is known. The gain k, of the plant is unknown, but its sign is known (say k,>0). The control objective is to get the plant output to match a model output, where the reference model transfer function is 1 sea Ps) = (0. Ms) = (0.3.2) Only gain compensation—or feedforward control—is necessary, namely a gain @ at the plant input, as is shown on Figure 0.6. vw Figure 0.6: Simple Feedforward Controller Note that, if kp were known, would logically be chosen to be 1/ky. We will call- ¢ (0.3.3) the nominal value of the parameter 0, that is the value which realizes the output matching objective for all inputs. The design of the various12 Introduction adaptive schemes proceeds as follows. Gain Sched Let v(e) € IR be some auxilis-y measurement that correlates in known, fashion with Kp, say k(t) = f(v(0)). Then, the gain scheduler chooses at time ¢ 1 4) = Fay (0.3.4) ‘Model Reference Adaptive Control Using the M.L-T. Rule To apply the MALT. rule, we need to obtain d¢p(8}/ = dyp(0)/ 30, with the understauding that 6 is frozen. From Figure 0.6, it is easy to see that ayp(6) ky b a” «Sea " We see immediately that the sensitivity function in (0.3.5) depends on the parameter k, which is unknown, so that ay, /<0 is not available. However, the sign of k, is known (K,> 0), so that we may merge the con- stant k, with the adaptation gain. The MIT rule becomes KyYm (0.3.5) § = -ge0m — >0 (0.3.6) Note that (0.3.6) prescribes an update of the parameter 4 in the direction opposite to the “correlation” product of eo and the model output Yr. ‘Model Reference Adaptive Control Using the Lyapunoy Redesign ‘The control scheme is exactly as before, but the parameter update law is chosen to make a Lyapunov function decrease along the trajectories of the adaptive system (see Chapter 1 for an introduction to Lyapunov analysis). The plant and reference model are described by Jn = ayy + kyr (03.7) Jn =~ Oy th = ym + yr 03.8) Subtracting (0.3.8) from (0.3.7), we get, with €9 = Yp ~ Yon bo = -aeyt ky(0-0)r 0.39) Since we would like # to converge to the nominal value 6° = 1 /k,, we define the parameter error as = 0- (0.3.10) Section 0.3 A Simple Example B Note that since 6” is fixed (though unknown), é = 4. The Lyapunov redesign approach consists in finding an update law so that the Lyapunov function Veo) = 6b + ke (03.11) is decreasing along trajectories of the error system fy = ~aeot or @ = update law to be defined (0.3.12) Note that since k,>0, the function v(¢q.¢) is a positive definite fune- tion. The derivative of v along the trajectories of the error system (0.3.12) is given by ¥(e0,8) = -2ae} + 2kpeodr + 2kyod — (0.3.13) Choosing the update law G6 = 6 = -er (03.14) yields (e406) = -2ae§ $0 (0.3.15) 4 thereby guaranteeing that ef + ky@? is decreasing along the trajectories of (0.3.12), (0.3.14) and that eo, and ¢ are bounded. Note that (0.3. is similar in form to (0.3.6), with the difference that eg is correlated with r rather that yy. An adaptation gain g may also be included in (0.3.14). Since v(eo,@) is decreasing and bounded below, it would appear that ¢->0 as 1-+G0. This actually follows from further a vided that r is bounded (cf. Barbalat’s lemma 1.2.1). Having concluded that ¢9-+0 as +00, what can we say about 0? Does it indeed converge to #*= I/k,? The answer is that one can not conclude anything about the convergence of # to 6° without extra condi- tions on the reference input, Indeed, if the reference input was a con- stant zero signal, there would be no reason to expect # to converge to 4”. Conditions for parameter convergence are important in adaptive control and will be studied in great detail. An answer to this question for the simple example will be given for the following indirect adaptive control scheme. ysis, pro4 Introduction Indirect Adaptive Control (Self Tuning) To be able to effectively compare the indirect scheme with the direct schemes given before, we will assume that the control objective is still model matching, with the same model as above. Figure 0.7 shows an indirect or self tuning type of model reference adaptive controller. ayn Figure 0.7; A Simple Indirect Controller ‘The identifier contains an identifier parameter «(t) that is an esti- mate of the unknown plant parameter k,. Therefore, we define x* = ky The controller parameter is chosen following the certainty equivalence principle: since 0° = 1/k, and x*= ky, we let )=1/x(2). The hope is that, a8 1 + 00, x(t) + ky, $0 that (0) I/Ky ‘The update law now is an update law for the identifier parameter x(t). There are several possibilities at this point, and we proceed to derive one of them. Define the identifier parameter error YO) = O-* (0.3.16) and let TO) = ae ‘The signal w may be obtained by stable filtering of the input w, since a>Qis known. The update las ‘based on the identifier error eis Rwy, (0.3.18) Equation (0.3.18) is used in the actual implementation of the algorithm. For the analysis, note that w (0.3.17) Section 0.3 A Simple Example 1s yp Hen tw 03.19) so that nye (0.3.20) Consider the update law eae ~gew gro (0.3.21) and let the Lyapunov function yay (03.22) ‘This Lyapunov function has the special form of the norm square of the identifier parameter error. Its derivative along the trajectories of the adaptive system is bom -eyrwt (0.3.23) Therefore, the update law causes a decreasing parameter error and all signals remain bounded. ‘The question of parameter convergence can be answered quite sim- ply in this case, Note that (0.3.20), (0.3.21) represent the first order linear time varying system = -ewy (0.3.24) which may be explicitly integrated to get v(t) = (0) exp (- 8 [winery (0.3.25) It is now easy to see that if [vtore + 0 as +00 (0.3.26) then ¥(¢)-+0, so that x(t)-r x" and O(t)-+1/kpy yielding the desired controller. The condition (0.3.26) is referred to as an identifiability con- dition and is much related to the so-called persistency of excitation that will be discussed in Chapter 2. It is easily seen that, in particular, it excludes signals which tend to zero as ¢ -» oo. The difficulty with (0.3.26) is that it depends on w, which in turn depends on u and therefore on both @ and r, Converting it into a condi- tion on the exogenous reference input r(¢) only is another of the prob- lems which we will discuss in the following chapters.16 Introduction ‘The foregoing simple example showed that even when simple feed- forward control of a linear, time invariant, first order plant was involved, the analysis of the resulting closed-loop dynamics could be involved: the equations were time-varying, linear equations. Once feed- back control is involved, the equations become nonlinear and time vary- ing. CHAPTER 1 PRELIMINARIES ‘This chapter introduces the notation used in this book, as well as some basic definitions and results. The material is provided mostly for refer- ence. It may be skipped in a first reading, or by the reader familiar with the results. ‘The notation used in the adaptive systems literature varies widely. We elected to use a notation close to that of Narendra & Valavani [1978], and Narendra, Lin, & Valavani [1980], since many connections exist between this work and their results. We will refer to texts such as, Desoer & Vidyasagar [1975], and Vidyasagar [1978] for standard results, and this chapter will concentrate on the definitions used most often, and on nonstandard results. 11 NOTATION Lower case letters are used to denote scalars or vectors. Upper case letters are used to denote matrices, operators, or sets. When w(t) is a function of time, i(s) denotes its Laplace transform. Without ambi- guity, we will drop the arguments, and simply write w and i. Rational transfer functions of linear time invariant (LTI) systems will be denoted using upper case letters, for example, H(s) or H. Polynomials in s will be denoted using lower case letters, for example, fi(s) or simply A Thus, we may have = fi/d, where H is both the ratios of polyiomials in s and an operator in the Laplace transform domain, Sometimes, the time domain and the Laplace transform domain will be mixed, and 1718 Preliminaries Chapter 1 parentheses will determine the sense to be made of an expression. For example, /1(u) or Hi is the output of the LTI system /7 with input w, Hi(u)v is H(u) multiplied by v in the time domain, while Hi(uv) is ‘operating on the product u(t) v(0), 1.2L, SPACES, NORMS We denote by |x| the absolute value of x if x is a scalar and the euclidean norm of x if x is a vector. The notation | || will be used to denote the induced norm of an operator, in particular the induced matrix norm Wall = sup, 14x (2.1) and for functions of time, the notation is used for the Ly norm lap = Cf luca rari (1.2.2) for pe[ 1,00), while I lgg = sup, | mE) (1.2.3) and we say that weL, when | uly exists. When p is omitted, || ul denotes the L; norm. Truncated functions are defined as La) = £0) tss =0 tps 1.24) and the extended L. spaces are defined by Lye = (S| forall < 00, fz € Lp) (125) For example, e' does not belong to L, we Ly, we have or but ee Ly. When Milly = supl eG)! (1.2.6) A function f may belong to L and not be bounded. Conversely, a bounded function need not belong to L1. However, if fel) NL... then feZy for all pe[1, 0] (ef. Desoer & Vidyasagar [1975], p. 17) Also, f € Ly does not imply that +0 as ta9. This is not even guaranteed if f is bounded. However, note the following results. Section 1.2 L, Spaces, Norms 19 Lemma 1.2.1 Barbalat’s Lemma f ——_f(0) isa uniformly continuous function, such that lim [foe exists and is finite, Then f(t) 0.as roo. Proof of Lemma 1.2.1 cf. Popov [1973] p. 211 Corollary 1.2.2 Bb € Lay and g € Ly, for some p € [1,00), Then g(t)-+0a5 +00 Ln, arr Proof of Corollary 1.2.2 C Direct from lemma 1.2.1, with f = | g|’, since g,@ bounded implies that f is uniformly continuous. O 13 POSITIVE DEFINITE MATRICES Positive definite matrices are frequently found in work on adaptive sys- tems. We summarize here several facts that will be useful. We consider real matrices. Recall that a scalar u, or a function of time u(t), is said to be positive if w 0, or u(t) > 0 for allt. It is strictly positive if u>0, or, for some a>0, u(t) 2 a for all ¢. A square matrix A € R™" is posi- tive semidefinite if x" A x & 0 for all x. It is positive definite if, for some a>0, xx 2 ax" x = a| x|? for all x. Equivalently, we can require 74x 2a for all x such that |x| = 1. The matrix is negative semidefinite if ~A is positive semidefinite and for symmetric matrices, we write A = B if A-B > 0. Note that a matrix can be neither posi- tive semidefinite nor negative semidefinite, o that this only establishes a partial order on symmetric matrices ‘The eigenvalues of a positive semidefinite matrix lie in the closed right-half plane (RHP), while those of a positive definite matrix lie in the open RHP. If 420 and 4=A?, then A is symmetric positive semidefinite. In particular, if A > 0, then A +47 is symmetric positive semidefinite. The eigenvalues of a symmetric matrix are all real, ‘Such a matrix also has » orthogonal eigenvectors, so that we can decompose 4 as 4=UTAU (3.1) where U is the matrix of eigenvectors satisfying U? U = I (that is, U is a unitary matrix), and A is a diagonal matrix composed of the20 Preliminaries Chapter t eigenvalues of A. When A 2 0, the square root matrix A* is a diagonal ‘matrix composed of the square roots of the eigenvalues of 4, and. 4% = uTA*U (13.2) is the square root matrix of 4, with A = 4%: 4 and(A®)? = 4%, If A 20 and B > 0, then 4 +B = 0 but it is not true in general that A-B 20. However, if A, B are symmetric, positive semidefinite ‘matrices, then AB—although not necessarily symmetric, or positive semidefinite—has all eigenvalues real positive. Another property of symmetric, positive semidefinite matrices, fol- lowing from (1.3.1), is Amin()| x]? S xTAX S Ama(A)| x]? (3.3) This simply follows from the fact that x74 x = x7UTAUx = 27 Az and |z|?= 27z = |x|, We also have that All = Amat) (1.3.4) and, when A is positive definite WAT = 1/\min(A) (1.3.5) 14 STABILITY OF DYNAMIC SYSTEMS 1.4.1 Differential Equations This section is concerned with differential equations cf the form X= flux) xlto) = x0 (at) where x € RY): 20. The system defined by (1.4.1) is said to be autonomous, or time- invariant, if f does not depend on 1, and non awionomous, or time- varying, otherwise. It is said to be linear if f(t,x) = A(x for some A(): IR, > IR"" and nonlinear otherwise. We will always assume that /(¢,x) is piecewise continuous with respect tot. By this, we mean that there are only a finite number of discontinuity points in any coripact set. We define by By the closed ball of radius h centered at 0 in IR”, Properties will be said to be true: © locally, if true for all xq in some ball By. Section 1.4 ‘Stability of Dynamic Systems 21 © globally, if true for all xo € IR", ‘© imany closed ball, if true for all xo @ By, with h arbitrary. © uniformly, if true for all to = 0. By default, properties will be true locally, Lipschitz Conc mn and Consequences The function f is said to be Lipschitz in x if, for some h>0, there exists / > 0 such that [fx - feed] S Nx; =22] (1.4.2) for all x1, x2 € By, £20. The constant / is called the Lipschitz con- stant, This defines locally Lipschitz functions. Globally Lipschitz func- tions satisfy (1.4.2) for all x1, x2 € IR", while functions that are Lipschitz in any closed ball satisfy (1.4.2) for all x1, x; © By, with | possibly depending on /. The Lipschitz property is by default assumed to be satisfied uniformly, that is, ! does not depend on ¢. If f is Lipschitz in x, then it is continuous in x. On the other hand, if f has continuous and bounded partial derivatives in x, then it is Lipschitz, More formally, we denote #| (143) ax, so that if | D2 < J, then f is Lipschitz with constant From the theory of ordinary differential equations (ef. Coddington & Levinson [1955)), it is known that f locally bounded, and locally Lipschitz in x imply the existence and uniqueness of the solutions of (1.4.1) on some time interval (for as long asx € By). Dif = Definition Equilibrium Point x is called an equilibrium point of (1.4.1), if f(t,x) = 0 for all ¢ 2 0. By translating the origin to an equilibrium point xo, we can make the origin 0 an equilibrium point. This is of great notational help, and we will assume henceforth that 0 is an equilibrium point of (1.4.1) Proposition 1.4.1 Y x = Vis an equilibrium point of (1.4.1), f is Lipschitz in x with constant / and is piecewise, continuous with respect to 1 Then the solution x(0) of (1.4.1) satisfies2 Preliminaries Chapter 1 Irate!" & [xO] & Jal eH 1.44) as long as x(t) remains in By Proof of Proposition 1.4.1 Note that |x|? = x?x implies that gee ie last] = att |r] rd | sapeda|sra[£e] ass so that [Gra] s [4e] (1.46) Since f is Lipschitz d mtixt s Aixt s dai (14) and there exists a positive function s(t) such that fix = -Uxi+s (1.4.8) Solving (1.4.8) [etn] = [get + pcan te 2 [ret ay 4 The other inequality follows similarly from (1.7). Proposition 1.4.1 implies that solutions starting inside By will remain inside By for at least a finite time interval. Or, conversely, given a time interval, the solutions will remain in B, provided that the initial conditions are sufficiently small. Also, f globally Lipschitz implies that xeL,,,. Proposition 1.4.1 also says that x cannot tend to zero faster than exponentially The following lemma is an important result generalizing the well- known Bellman-Gronwall lemma (Bellman [1943]). The proof is similar to the proof of proposition 1.4.1, and is left to the appendix. Section 14 ‘Stability of Dynamic Systems 23 Lemma 1.4.2 Bellman-Gronwall Lemma Let x(),a(.), w() IRR. Let 720. y #10 [atoninde + 00) (1.4.10) forall: © (0,7) Then fee x0) s [amuiner dr + ult) (4.1) for allt « [0,7] When u(.) is differentiable inde taste x(t) Ix) 0, there exists 8(¢0), such that [xo] <6 => lim |x(0)| = 0 Definition Uniform Asymptotic Stability (u.a.s.) x =0 is called a uniformly asymptotically stable (u.as,) equilibrium point of (1.4.1), if (@) x = isa uniformly stable equilibrium point of (1.4.1), (b) the trajectory x(¢) converges to 0 uniformly in fo. More pre- cisely, there exists. 5>0 and a function y(r,x0) IR, x R"+IR,, such that lim y(r,xo) = 0 for all xo and [xl <8 => [x(0 < y(t ~t0, x0) foray’ ZO The previous definitions are local, since they concern neighbor- hoods of the equilibrium point. Global asymptotic stebility is defined as, follows, Definition Global Asymptotic Stability x =0 is called a globally asymptotically stable equilibrium point of (14.1), if it is asymptotically stable and lim|x(0)| = 0, for all Xo e RY Global u.a.s. is defined likewise, Note that the speed of conver- gence is not quantified in the definitions of asymptotic stability. In the following definition, the convergence to zero is required to be at least exponential. Definition Exponential Stability, Rate of Convergence x= 0 is called an exponentially stable equilibrium point of (1.4.1) if there exist m, a >0 such that the solution x(¢) satisfies [x] sme | x9] (1.4.13) Section 1.4 Stability of Dynamic Systems 25 for all xp € By, = fo 0. The constant a is called the rate of conver- gence. Global exponential stability means that (1.4.13) is satisfied for any Xo € IR". Exponential stability in any closed ball is similar except that ‘m and a may be functions of fh. Exponential stability is assumed to be uniform with respect to fo. It will be shown that uniform asymptotic stability is equivalent to exponential stability for linear systems (see Sec- tion 1.5.2), but it is not true in general. 1.4.3 Lyapunov Stability Theory We now review some of the key concepts and results of Lyapunov stabil- ity theory for ordinary differential equations of the form (1.4.1). A more complete development is available, for instance, in the texts by Hahn [1967] and Vidyasagar {1978}. The so-called Lyapunov second method enables one to determine the nature of stability of an equilibrium point of (1.4.1) without expli- city integrating the differential equation. The method is basically a gen- eralization of the idea that if some “measure of the energy” associated with a system is decreasing, then the system will tend to its equilibrium, To make this notion precise, we need to define exactly what we mean by a “measure of energy,” that is, energy functions. For this, we first define class K functions (Hahn [1967], p. 7). Definition Class K Funetions A function a(¢) : IR, -rIR, belongs to class K (denoted al.) € K), if it is continuous, strictly increasing, and a(0) = 0. Definition Locally Positive Definite Functions A continuous function v(¢,x): IR, x IR" > IR, is called a locally posi- tive definite function (Lp.d.f) if, for some h >0, and some a(.) © K (0) =0 and v(4,x)2a(lx]) — forallx © By 20 ‘An Lp.d.f. is locally like an “energy function.” Functions which are globally like “energy functions” are called positive definite functions (p.d.f) and are defined as follows. Definition -Positive Definite Functions A continuous function v(¢,.r): IR, x IR" -» IR, is called a positive definite function (pdf) ,if for some a(.) ¢ K (1,0) =0 and v(t,x)2a(/x}) forallx ¢ R02026 F eliminaries Chapter 1 ‘and the function «(p)-» 60 as p» 00 In the definitions of Lp.df. and p.d.f. functions, the energy like functions are not bounded from above as ¢ varies, This follows in the next definition, Definition Decrescent Function The function v(t,.x) is called decrescent, if there exists a function A) © K, such that vx) B(\Ix1) forall x € Byt 20 Examples Following are several examples of functions, and their membership in the various classes: v(t,x) = |x|?: p.d.f, decrescent v(t,x) = x7 Px, with P>O: p.d.f., decrescent v(t,x) = (t+ Dx? pds. (t,x) =e! |x]? : decrescent v(t,x) = sin?(|x|?): Lp.d.f,, decrescent Lyapunoy Stability Theorems Generally speaking, the theorems state that when v(t,x) is a p.d.f,, or an Ip, and dv/di(t,x) $0, then we can conclude the stability of the equilibrium point. The derivative of v is taken along the trajectories of (1.4.1); that is, dv(tx 2 ED, LD yy ‘a 7 Go foxy (4.14) ay Theorem 1.4.3 Basie Theorems of Lyapunov Let v(t,x) be continuously differentiable: Then Section 14 Stability of Dynamic Systems n Conditions on | Conditions on Conclos ED ux) onclusions | ipa %0 locally | stable Lpa.t, deerescent | 20 locally | uniformly stable ipa, Inde asymptotically stable Ips, decrescent | Lp.af | uniformly asymptotically stable pa, decrescent | pdt | globally uas Proof of Theorem 1.4.3 cf. Vidyasagar (1978), p. 148 and after. Example We refer to Vidyasagar [1978] for examples of application of the theorems. An interesting example is the second order system. Hy = mxP +49 = 1-22 y= x) + xQ(xf +x} - 1) (1.4.13) with the pd. vOe) = xP +3 (1.4.16) ‘The derivative of v is given by HX x2) = Qed +9) Oc? +27 - 1) (1.4.17) so that -¥ is an Lp.d.f. (in By, where <1), establishing the local asymptotic stability of the origin. The origin is, in fact, not globally asymptotically stable: the system can be shown to have a circular limit cycle of radius 1 (by changing to polar coordinates). From (1.4.17), we can also show that for all x € By i < -2(1-A*)y (1.4.18) so that v() S v(Qper2t-M ir [xO] Sx) eC foralez0 (1.4.19) and, in fact, x = 0 is a locally exponentially stable equilibrium point. Note that the Lyapunov function is in fact the squared norm of the state, a situation that will often occur in the sequel,28 Preliminaries Chapter 1 Comments, ‘The theorems of Lyapunov give sufficient conditions guaranteeing the stability of the system (1.4.1). It is a remarkable fact that the converse of theorem 1.4.3 is also true: for example, if an equilibrium point is stable, there exists an Lp. v(t,x) with W(x) $0. The usefulness of theorem 1.4.3 and its converse is limited by the fact that there is no gen- eral (and computationally non-intensive) prescription for generating the Lyapunov functions. A significant exception to this concerns exponen- tially stable systems, which are the topic of the following section, 1.5 EXPONENTIAL STABILITY THEOREMS We will pay special attention to exponential stability for two reasons. When considering the convergence of adaptive algorithms, exponential stability means convergence, and the rate of convergence is a useful measure of how fast estimates converge to their nominal values. In Chapter 5, we will also observe that exponentially stable systems possess at least some tolerance to perturbations, and are therefore desirable in ‘engineering applications. 1.5.1 Exponential Stability of Nonlinear Systems ‘The following theorem will be useful in proving several results and relates exponential stability to the existence of a specific Lyapunov func- tion, ‘Theorem 1.5.1 Converse Theorem of Lyapunov Assume that /(¢,x) : IR, xIR" + IR" has continuous and bounded first partial derivatives in x and is piecewise continuous in ¢ for all xX € By, £20. Then, the following statements are equivalent: (@) x = Oisan exponentially stable equilibrium point of & = f(t.x) xl) = x (1.5.1) (b) There exists @ function v(¢,x), and some stictly positive con- stants fi’, ay,a,a3,a%4, such that, for all x € By, £20 yx]? < v(t,x) S ag] x|? (1.5.2) ae ay 27BIEP (1.5.3) | zen C2) | sal xt (54) Section 1.5 Exponential Stability Theorems 29 Comments Again, the derivative in (1.5.3) is a derivative taken along the trajec- tories of (1.5.1), that is dv(t x) HUE), HED Fey 15.39) Gas ae GeO fem) (1.5.5) This means that we consider x to be a function of ¢ to calculate the derivative along the trajectories of (1.5.1) passing through x at /. It does not require of x to be the solution x(t) of (1.5.1) starting at x(to). Theorem 1.5.1 can be found in Krasovskii [1963] p. 60, and Hahn [1967] p. 273. It is known as one of the converse theorems. The proof of the theorem is constructive: it provides an explicit Lyapunov func- tion v(¢,x). This is a rather unusual circumstance, and makes the theorem particularly valuable. In the proof, we derive explicit values of the constants involved in (1.5.2)-(1.5.4). Proof of Theorem 1.5.1 (a) implies (8) ( Denote by p(x, x, 1) the solution at time + of (1.5.1) starting at x(0), 1, and define yx) = f iptr,x,0)2dr (1.5.6) where T'>0 will be defined in (i). From the exponential stability and the Lipschitz condition mx} ee & [plex] & |x| el? as.) and inequality (1.5.2) follows with ay is (1 YT) / 20 a = m2(1-e787)/2a (1.5.8) Gi) Differentiating (1.5.6) with respect to ¢, we obtain wen = [p+T x07 - | [email protected],0/? wr d 2 : + [lpn dr (1.5.9) Note that d/dt is a derivative with respect to the initial time ¢ and is taken along the trajectories of (1.5.1) By definition of the solution p, POX +40, 6440 = pOx,0) (1.5.10)30 Preliminaries Chapter 1 for all A‘, so that the term in the integral is identically zero over t,1+T7}. The second term in the right-hand side of (1.5.9) is simply |.x1?, while the first is related to |x|? by the assumption of exponential stability. It follows that BOR) gy ment) gf? HSA GS 7 mte |x] (15.11) Inequality (1.5.3) follows, provided that 7'>(1/a)Inm and ay t= Lome et (15.12) * (ii) Differentiating (1.5.6) with respect to x;, we have PED 2 Soren PE asin 1 fal 7 Under the assumptions, the partial derivative of the solution with respect to the initial conditions satisfies d [atexo) a fa #\e7]-¢ |g tease} = [sexo] - Ze I e Sates) 0319 (except possibly at points of discontinuity of f(r,x)). Denote Qilr.x,0) 2= api(r,x 0) (ax; Aylest) 2» alex) fax, (15.15) so that (15.16) ecome Leen =40eson-eten 516 Equation (1.5.16) defines Q(r,x,1), when integrated from r=t to r= t+T, with initial conditions Q(t ,x,1)=J, Thus, Q(r,x,1) is the state transition matrix (ef. Section 1.5.2) associated with the time vary- ing matrix A(p(r,x,4),7). By assumption, || A. ,.)|| 0 (0) implies (a) ‘This direction is straightforward, using only (1.5.2)-(1.5.3), and we find (1.5.20) a Comments ‘The Lyapunov function v(¢, x) can be interpreted as an average of the squared norm of the state along the solutions of (1.5.1). This approach is actually the basis of exact proofs of exponential convergence presented in Sections 2.5 and 2.6 for identification algorithms, On the other hand, the approximate proofs presented in Chapter 4 rely on methods for averaging the differential system itself. Then the norm squared of the state itself becomes a Lyapunov function, from which the exponential ‘convergence can be deduced. Theorem 1.5.1 is mostly useful to establish the existence of the Lyapunov function corresponding to exponentially stable systems. To establish exponential stability from a Lyapunov function, the following theorem will be more appropriate. Again, the derivative is to be taken along the trajectories of (1.5.1) Theorem 1.5.2 Exponential Stability Theorem Uf there exists a function v(t ,x), and strictly p 2, a5, and 8, such that for all x € By, = 0 ve constants a, - ai[x|? < v(x) S ag]? (1.5.21) da aod, oa32 Preliminaries Chapter 1 drs -aslxit)l? (1.5.23) sn) A vee.xt) a Then x(t) converges exponentially to 0. Proof of Theorem 1.5.2 From (1.5.23) ¥(E (0) = 9 (E45, x00 +8) 2 (ayer) VEU) (1.5.24) for all ¢ 2 0, so that (C48, x(€+8) S (I~ a /az) ¥(0,x(0) (1.5.25) for all ¢ > 0, From (1.5.22) vena) S$ ¥(Cx() — forall ty [40 #5] (1.5.26) Choose for # the sequence tp, fo+8, f+28,... so that v(¢,x(0)) is bounded by a staircase v(to,x(tg)), v(to+8,x(f9+8),... where the steps are related in geometric progression through (1.5.24). It follows that vtxt) sme v(t, x(t) (1.5.27) for allt 2 f 2 0, where oo 1.5.28) ™ ale) on Silay, (1.529) where js 1 1 oe 1.5.30) rae)? «+ gerbe] oom 1.5.2. Exponential Stability of Linear Time-Varying Systems We now restrict our attention to linear time-varying (LTV) systems of the form X= AMX —xlla) = Xo (1.3.31) where A(1) € IR"*" is a piecewise continuous function belonging to L Section 1.5 Exponential Stability Theorems 33 Definition State-Transition Matrix The state-transition matrix (1,19) € IR"*" associated with A(t) is, by definition, the unique solution of the matrix differential equation Leet ta) = AM FEt) — Blood =F (15.32) Note that linear systems with A(.) ¢ L,,, automatically satisfy the Lipschitz condition over any finite interval, so that the solutions of (1.5.31) and (1.5.32) are unique on any time interval, It is easy to verify that the solution of (1.5.31) is related to that of (1.5.32) through (0) = ®(C 10) xU00) (1.5.33) In particular, this expression shows that trajectories are “proportional” to the size of the initial conditions, so that local and global properties of LTV systems are identical. The state-transition matrix satisfies the so-called semigroup property (cf. Kailath [1980], p. $99): PU to) = BE) B(rt0) forall 2729 (1.5.34) and its inverse is given by Bt)! = Plto,0) (1.5.35) ‘A consequence is that d d 1 Gig MD = Feud) = Plo)" "A () ® (C051) ®(Co,1) - 810A) (1.5.36) The following propositions relate the stability properties of (1.5.31) to properties of the state-transition matrix. Proposition 1.5.3 Uniform Asymptotic Stability of LTV Systems is @ uniformly asymptotically stable equilibrium point of (1.5.31) Wand only if x =0 is stable, which is guaranteed by xe BP, CSUP BEDI) < 00 3.37) and x = 0 is attractive, which is guaranteed by 1 2U,t0)|| +0 as ¢-200 uniformly in fo. (1.5.38)