EE531 (Semester II, 2010)
1. Introduction
basic concept system identication methods procedures in system identication examples
1-1
Basic concept
objective: how to build static/dynamical models from experimental data
Disturbance
v(t)
Input System
u(t)
Output
y(t)
estimation of system description can serve for many purposes: obtain a mathematical model for controller design explain/understand observed phenomena (e.g., machine learning) forecast events in the future (e.g., time series analysis in econometrics) obtain a model of signal in lter design (e.g., signal processing)
Introduction 1-2
System Identication for Controller Design
Disturbance
Reference +
Output Controller Plant
Sensor
noise
for controller design, the plant is assumed known in system identication, we aim to estimate the parameters in a model
Introduction 1-3
System Identication for prediction
31
1 USD in Thai Baht
30.5
30
29.5
Dec 2010
Jan 2011
Feb 2011 Mar 2011
how to forecast the Thai Baht in Apr, May,... ? need a model for prediction, e.g. xApr = a1xMar + a2xFeb
Introduction 1-4
Models
a description of the system, or a relationshop among observed signals a model should capture the essential information about the system Types of Models graph and tables, e.g., bode plots and step response mathematical models, e.g., dierential and dierence equations System identication is a process of obtaining models based on a data set collected from experiments input and output signals from the system are recorded and analyzed to infer a model
Introduction 1-5
System identication methods
Nonparametric approach aim at determining a (time/frequency) response directly without rst selecting a possible set of models gives basic information about the sytsem and is useful for validation examples are transient analysis, frequency analysis, correlation analysis, and spectral analysis Parametric approach require assumptions on a model class/structure the search for the best model within the candidate set becomes a problem of determining the model parameters typically more complicated than the nonparametric approach results can be further used for controller design, simulation, etc.
Introduction
1-6
Procedures in System Identication
Linear models of order n
Model Class
Model/Complexity Noise
Training data
AR models of order n
Prediction error method Leastsquares
True Description
Linear/Nonlinear Timevarying/Timeinvariant
Estimation
Model Fit
quadratic loss likelihood
Model selection determine n
No Validation Model Accept ? End Yes
Validation data
Introduction
1-7
Parametric Estimation
Model classication: SISO/MIMO, Linear/Nonlinear, Time-invarian/Time varying, Discrete/Continuous searching the best model within a candidate set becomes a problem of determining the model parameters the selected parameter x from a model class M is optimal in some sense, i.e., x = argmin f (x, D),
xM
where f is a measure of goodness of t (or loss function) and is a function of information data (D) examples of f are quadratic loss, likelihood, entropy function, etc.
Introduction 1-8
Estimation methods
Linear least-squares method (LS) simple to compute, no assumption on noise model Statistical estimation methods, e.g., Maximum likelihood, Bayes use prior knowledge about the noise Instrumental-variable method a modication of the LS method for correlated noise Prediction-error method model the noise, applicable to a broad range of models
Introduction
1-9
Model selection
Principle of parsimony: one should pick a model with the smallest possible number of parameters that can adequately explain the data one can trade o between
Goodness of t VS Complexity
related to the concept of bias VS variance in statistics examples of model selection criterions are FPE, AIC, BIC, etc.
Introduction
1-10
Example: Polynomial tting
20 True curve 15 data n=1 n=3 10 n=10
p(x)
5 0 5 2
1.5
0.5
0.5
1.5
x a curve tting problem of polynomial of order n the true order is n = 3
Introduction 1-11
Example: Trade-o curve
45 40 35
tting error
30 25 20 15 10 5 0 1 2 3 4 5 6 7 8 9 10
polynomial order shows the minimized loss as a decreasing function of model complexity the error begins to decrease as the model picks up the relevant features as the model order increases, the model tends to over t the data In practice, the model order is determined by the knee in the curve
Introduction 1-12
Model Validation
a parametric estimation procedure picks out the best model a problem of model validation is to verify whether this best model is good enough test the estimated model (obtained from training data), with a new set of data (validation set) The tests verify whether the dynamic from the input and the noise model are adequate
Introduction
1-13
Numerical Example
10
Output
10 0 1
50
100
150
200
250
Time
Input
1 0
50
100
150
200
250
Time feed a known input to the system and measure the output the input should contain rich information to excite the system
Introduction 1-14
t the measured output to the model (1 + a1q 1 + . . . + anq n)y(t) = (b1q 1 + . . . + bnq n)u(t) + (1 + c1q 1 + . . . + cnq n)(t) with unknown parameters a1, . . . , an, b1, . . . , bn, c1, . . . , cn this model is known as Autoregressive Moving Average with Exogenous input (ARMAX) (t) represents the noise that enters to the system n is the model order, which is selected via model selection the parameters are estimated by the Prediction-error method (PEM)
Introduction
1-15
Example of output prediction
15
n=1
FIT = 30.87%
15
n=2
FIT = 65.37%
10
10
y(t)
y(t)
5 model measured
5 model measured
10
10
15 0
20
40
60
80
100
120
15 0
20
40
60
80
100
120
15
t n=3
FIT = 73.85%
15
t n=4
FIT = 75.98%
10
10
y(t)
y(t)
5 model measured
5 model measured
10
10
15 0
20
40
60
80
100
120
15 0
20
40
60
80
100
120
t (estimated by PEM and validated on a new data set)
Introduction
1-16
Example of Zero-Pole location
1
n=2
1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1
0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1
n=6
0.8
0.6
0.4
0.2
0.2
0.4
0.6
0.8
1 1
0.8
0.6
0.4
0.2
0.2
0.4
0.6
0.8
: zeros, : poles red: true system, blue: estimated models chance of zero-pole cancellation at higher order
Introduction 1-17
References
Chapter 1,2 in L. Ljung, System Identication: Theory for the User, Prentice Hall, Second edition, 1999 Chapter 1-3 in T. Sderstrm and P. Stoica, System Identication, Prentice Hall, 1989 o o L. Ljung, Perspective on System Identication, http://www.control.isy.liu.se/ljung/
Introduction
1-18
