Download - System Identification: Black Box
![Page 1: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/1.jpg)
System Identification: Black Box
Robert GreplInstitute of Solid Mechanics, Mechatronics and Biomechanics
Faculty of Mechanical Engineering, Brno University of Technology
2010 - 2015
![Page 2: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/2.jpg)
Black box - agenda
1. Prerequisities: Least squares.
Discrete dynamic system.
2. How to use LS to estimate parameters of DDS? Example: mechanical oscillator
3. Problem: Noise.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
2
![Page 3: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/3.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
3
Methods: Least Squares 1/2
generally: LS is Linear Optimization method
assume function
(linear)(ordinary) LS requires linear parametrization:example:
consider m measurements... written in matrix form:
if m=n, then Problem: Find parameters of
line if 2 points are given.
![Page 4: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/4.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
4
Methods: Least Squares 2/2
in real apps, the measurement is not ideal
task is: how to solve this overdetermined problem? (m equations and n unknowns)
answer: minimize criterion
result:
Note: Penrose pseudoinversion
![Page 5: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/5.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
5
LS example: regression 1D function
fit 1D function
(this task = the fitting of static model linear in parameters)
noise added to measurement
regression matrix:
[E015]
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
t
y
noisy data
original
fit
![Page 6: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/6.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
6
Recursive Least Squares (RLS) (Plackett's algorithm)
online estimation
gives identical result as OLS
assume model
in matrix form:
![Page 7: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/7.jpg)
Black box - agenda
1. Prerequisities: Least squares.
Discrete dynamic system.
2. How to use LS to estimate parameters of DDS? Example: mechanical oscillator
3. Problem: Noise.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
7
![Page 8: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/8.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
8
OLS & RLS example: Estimation of mass+spring+damper system 1/3
1) Discretization
![Page 9: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/9.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
9
OLS & RLS example: Estimation of mass+spring+damper system 2/3
2) Create matrix X and vector Y
Y(k) = y(k);
X(k,:) = [y (k-1), y(k-2), u(k-2)];
{E003}
![Page 10: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/10.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
10
OLS & RLS example: Estimation of mass+spring+damper system 3/3
3) Solve LS/RLS problem– both LS and RLS provide the same results
– convergence of RLS/precision of LS depends on excitation signal
{E003}0 2 4 6 8 10
-10
0
10
t [s]
Oscilator position
0 2 4 6 8 10-50
0
50
t [s]
inpu
t u
Input excitation force [N]
0 2 4 6 8 10-2
0
2convergence of Recursive LS (RLS)
t [s]
continuous sim
ARX sim
a1
a2
b2
But: there is no noise
considered in our
simulation!
![Page 11: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/11.jpg)
Black box - agenda
1. Prerequisities: Least squares.
Discrete dynamic system.
2. How to use LS to estimate parameters of DDS? Example: mechanical oscillator
3. Problem: Noise.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
11
![Page 12: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/12.jpg)
Problem = noise: simple example study – setup 1/4
consider first order system
discretization:
for simplicity, consider NO INPUT
TASK: estimate one parameter a.
simulation data :
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
12
Ts
1k kx ax
0 2 4 6 8 100
0.5
1
1.5
2
tx
-1 0 1 2 3
0.5
1
1.5
x(k)
x(k
+1
)
![Page 13: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/13.jpg)
Problem = noise: simple example study – without noise 2/4
Consider ini value x_0 and simulate equation:
On lower Fig. we see the plot x(k) vs. x(k+1).From the equation, it is obvious, that it should be a line.
The slope of this line is a.
With data WITHOUT noise, the estimation of a is precise.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
13
1k kx ax
0 2 4 6 8 100
0.5
1
1.5
2
t
x
-1 0 1 2 3
0.5
1
1.5
x(k)
x(k
+1
)
![Page 14: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/14.jpg)
Problem = noise: simple example study – noise on output 3/4
Consider further the noise on output (e.g. noise of sensor). Equation: (w is random number with normal distribution (Gauss)
We can observe a problem with estimation of parameter a.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
14
1k k
k k
x ax
y x w
0 2 4 6 8 10-2
-1
0
1
2
3
4
5
6
t
x
-1 0 1 2 3 4 5
-1
0
1
2
3
4
5
x(k)
x(k
+1
)
Example:
SI 004
% simulation
x(1) = x0;
for i = 2:length(t)
x(i,1) = a*x(i-1);
y(i,1) = x(i) + randn*0.2;
end
0 2 4 6 8 10-2
-1
0
1
2
3
4
5
6
t
x
-1 0 1 2 3 4 5
-1
0
1
2
3
4
5
x(k)
x(k
+1
)
![Page 15: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/15.jpg)
Problem = noise: simple example study
Next, consider noise „inside“ the systemwhich modifies the state of x(k).Equation:
This noise can be considered as a randomdisturbance (input) influencing the system.
Using the simulation, we can observe, that the estimation ofparameter a is perfect.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
15
1k k
k k
x ax w
y x
Example:
SI 004
% simulation
for i = 2:length(t)
x(i,1) = a*x(i-1) + randn*0.2;
y(i,1) = x(i) ;
end
0 2 4 6 8 10-8
-6
-4
-2
0
2
4
6
8
10
12
t
x
-5 0 5 10
-6
-4
-2
0
2
4
6
8
10
x(k)
x(k
+1
)
0 2 4 6 8 10-8
-6
-4
-2
0
2
4
6
8
10
12
t
x
-5 0 5 10
-6
-4
-2
0
2
4
6
8
10
x(k)
x(k
+1
)
![Page 16: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/16.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
16
Linear Models for System Identification
q = forward shift operatore.g.: q-1 x(k) = x(k-1)
linear models = relation between y(),u(), and noise v(k) through transfer functions
general form:
![Page 17: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/17.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
17
Linear Models for System Identification
AR – autoregressive model
MA – moving average
example: smoothing
![Page 18: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/18.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
18
Linear Models for System Identification
ARMA
ARX
ARMAX
OE
BJExample: system completely deterministic,
noise added only to output (e.g. position
sensor = potentiometer)
Very popular – linear in parameters
OLS
![Page 19: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/19.jpg)
Black-box identification- the use of a tool System Identification/GUI
Robert GreplInstitute of Solid Mechanics, Mechatronics and Biomechanics
Faculty of Mechanical Engineering, Brno University of Technology
2010 - 2015
![Page 20: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/20.jpg)
System Identification Toolbox - GUI
System Identification Toolbox very (very) complex tool, author L. Ljung
you can work using
– command line
– GUI
how to run it?: ident
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
20
![Page 21: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/21.jpg)
System Identification Toolbox – GUI: Import data
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
21
![Page 22: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/22.jpg)
System Identification Toolbox – GUI: Preprocessing
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
22
preprocessing of the data check data
select estimationand verification data
Check
imported data
divide
Estimation and
Verification
data
![Page 23: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/23.jpg)
System Identification Toolbox – GUI: Select data
Select data for Estimation and for Verification
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
23
Data for
estimation
Data for
verification
![Page 24: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/24.jpg)
System Identification Toolbox – GUI:
Estimate order of the system
You can estimateseveral ARX models atonce.
Think about noise...
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
24
Example:
ARX
Example:
OE
![Page 25: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/25.jpg)
Exercise
S005
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
25
![Page 26: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/26.jpg)
Dodatky
• Offline vs. online odhad parametrů
• IV (Instrumental variable) method
• Nelineární nejmenší čtverce
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
34
![Page 27: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/27.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
35
Offline vs. online odhad parametrů
batch estimation– measurement in time (0,T)
– data processing
– parameter estimation using all data
online estimation– parameter estimation during system operation
– recursive algorithm
– advantages:
» memory requirements (often) reduced – OLS vs. RLS
» the slow changes of parameters can be captured
![Page 28: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/28.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
36
Recursive Least Squares (RLS) (Plackett's algorithm)
online estimation
gives identical result as OLS
assume model
in matrix form:
![Page 29: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/29.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
37
OLS & RLS example: Estimation of mass+spring+damper system 3/3
Solve LS/RLS problem– both LS and RLS provide the same results
– convergence of RLS/precision of LS depends on excitation signal
{E003}0 2 4 6 8 10
-10
0
10
t [s]
Oscilator position
0 2 4 6 8 10-50
0
50
t [s]
inpu
t u
Input excitation force [N]
0 2 4 6 8 10-2
0
2convergence of Recursive LS (RLS)
t [s]
continuous sim
ARX sim
a1
a2
b2
No noise considered in our
simulation!
![Page 30: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/30.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
38
OLS & RLS example: Estimation of mass+spring+damper system with noise
Assume noise added to measurement (e.g. if potentiometer used)
noise_stdvar = 0.1;
(later, we name this noise model “Output Error” = OE model)
{E003} 5.5 6 6.5 7 7.5 8
-6-4-202
t [s]
Oscilator position
0 2 4 6 8 10-50
0
50
t [s]
inpu
t u
Input excitation force [N]
0 2 4 6 8 10
-1
0
1
2
convergence of Recursive LS (RLS)
t [s]
continuous sim
ARX sim
a1
a2
b2
![Page 31: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/31.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
39
Estimation of OE: Instrumental Variable (IV) Method
solution for consistency problem
1. Estimate ARX
2. Simulate model with estimated parameters
3. Create instrumental matrix Z
4. Estimate
repeat steps 2-4 (usually upto 3-4 steps) [E009]
![Page 32: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/32.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
40
Nonlinear Least Squares 1/2
motivation: need to find parameters for functions which are nonlinear in parameters
assume vector field
goal: minimize criterion
Taylor series
– gradient:
![Page 33: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/33.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
41
Nonlinear Least Squares 2/2
all NLS methods are iterative
methods for computation of h:– Steepest descent
– Newton’s method
– Gauss-Newton
– Levenberg-Marquardt
![Page 34: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/34.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
42
Example NLS: Solution of overdetermined system
Assume vector field
clearly, the system is overdetermined
x = 1 / 0.4115
Task: Find solution which has minimal mean square error.
[E002]0 2 4 6 8 10 12 14 16 18 20
0
0.2
0.4
0.6
0.8
1
convergence of iteration method
iterations
valu
es o
f x
0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.30
0.2
0.4
0.6
0.8
variable x
Plot of square error
LM lambda = 2
GN alpha = 0.5
solution of individual equations
f12
f22
f12 + f
22
![Page 35: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/35.jpg)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
43
References
[Iserman 2011] Isermann, R., Muenchhof, M.: Identification of Dynamics Systems, Springer 2011
[Nelles, 2001] O. Nelles: Nonlinear system identification: from classical approaches to neural networks and fuzzy models, Springer-Verlag, 2001google books
[Noskievič, 1999] P. Noskievič: Modelování a identifikace systémů, Montanex, 1999
[Ljung 2008] L. Ljung: Perspectives on System Identification, Seoul 2008, http://www.control.isy.liu.se/~ljung/
[Denery 2005] T. Denery: Modeling Mechanical, Electric and Hydraulic Systems in Simulink, Matlab webinars, 2005
[Blaha] doc. Ing. Petr Blaha, PhD., FEKT, CAKhttp://sites.google.com/site/modelovaniaidentifikace/
[Gander, 1980] Gander, W.: Algorithms for QR-Decomposition, research report, 1980
http://www.emeraldinsight.com/journals.htm?articleid=1943785&show=html
http://courses.ae.utexas.edu/ase463q/design_pages/fall02/AWE_web/secondorder.htm
http://www.ni.com/white-paper/4028/en/
![Page 36: System Identification: Black Box](https://reader033.vdocuments.mx/reader033/viewer/2022042702/626549580ef2242a65130fa3/html5/thumbnails/36.jpg)
Kontakt: www.mechlab.czÚstav mechaniky těles, mechatroniky a biomechaniky
Fakulta strojního inženýrstvíVysoké učení technické v Brnědoc. Ing. Robert Grepl, Ph.D.