brian do anderson the australian national university and national ict australia limited
DESCRIPTION
Two Decades of Adaptive Control Pitfalls KUL System Identification and Data Modelling Lennart Ljung Symposium. Brian DO Anderson The Australian National University and National ICT Australia Limited. Outline. Adaptive Control MIT Rule Bursting - PowerPoint PPT PresentationTRANSCRIPT
Two Decades of Adaptive Control Pitfalls
KUL System Identification and Data Modelling
Lennart Ljung Symposium
Brian DO Anderson
The Australian National University
and
National ICT Australia Limited
Lennart Ljung Symposium Oct 20042
Outline
Adaptive Control
MIT Rule
Bursting
Good Models, Bad Models and Changing the Controller
Multiple Model Approach to Adaptive Control
Conclusions
Lennart Ljung Symposium Oct 20043
Plant is initially unknown or partially known, or is slowly varying.
There is an underlying performance index, eg
—Reference
r
+Controller
Input
uPlant
Output
y
Disturbanced
Adaptive Control
minimize T → ∞lim 1
T ∫ 0T u2 + y−r( )2
[ ]dt
Lennart Ljung Symposium Oct 20044
A non-adaptive controller maps the error signal r-y into u in a causal, time-invariant way eg
An adaptive controller is one where parameters are adjusted.
Adaptive Control (continued)
˙ x c =Acxc +bc r −y( )
u=ccxc
with Ac,bc,cc, constant, xc a vector
—Reference
r
+Controller
Input
uPlant
Output
y
Disturbanced
Lennart Ljung Symposium Oct 20045
Other ways of doing this exist Often 3 time scales:
Underlying plant dynamics (with fixed parameters) Time scale for identifying plant Time scale of plant parameter variation
ControlLaw Calculation
IdentifierPlant Parameters
Controller Plant
disturbanceControllerParameters
yu
One Formof Adaptive Controller
Lennart Ljung Symposium Oct 20046
Outline
Adaptive Control
MIT Rule
Bursting
Good Models, Bad Models and Changing the Controller
Multiple Model Approach to Adaptive Control
Conclusions
Lennart Ljung Symposium Oct 20047
MIT Rule Problem
k m(t)
+
-
• Zp(s) is known, km is known, k p is positive and unknown, but kc(t) is known and adjustable
• Problem is to find a rule using e(t) to adjust kc(t) to cause e(t) to go to zero
• Problem source: k p depends on dynamic air pressure for aircraft.
kc(t) kpZp(s)
kmZp(s)
r(t)yp(t)
e(t)
ym(t)
Lennart Ljung Symposium Oct 20048
MIT Rule Intuition and Performance
Use gradient descent to try to drive e(t) to 0:
with g a gain constant
€
˙ k c = −g∂[
1
2e2(t)]
∂kc
€
˙ k c = −g y p − ym[ ]ym
Equivalently,
Sometimes this worked, sometimes it did not work.Why?
Lennart Ljung Symposium Oct 20049
Example of performance
•Unshaded regionis stable
•Sine wave input at frequency
•Plant is (s+1)-1
g
Lennart Ljung Symposium Oct 200410
Underlying differential equation for kc is Mathieu equation. Solution regions of this equation are depicted.
One instability mechanism is interaction of excited plant dynamics with adaptive dynamics, made worse at high gain g
Explaining Instability
High (adaptive) gain instability for some Zp(s) : consider a
constant input R to display phenomenon. The MIT rule
leads to a characteristic equation; high g may give RHP zero
zero:€
˙ k c = −g y p − ym[ ]ym
s + gkmkpR2Zp(s) = 0From derivative of
kc
From PlantDynamics
Lennart Ljung Symposium Oct 200411
Explaining Instability II
k m(t)kmZm(s)
+
-
Zm(s) is known, km is known, k p is positive and unknown, but kc(t) is known and adjustable
A second instability mechanism comes from modelling errors, here errors between Zp(s) and Zm(s)
Following two figures show case where plant and model are the same and where they are different.
kc(t) kpZp(s)r(t)
yp(t)e(t)
ym(t)
Lennart Ljung Symposium Oct 200412
Example of performance
•Unshaded regionis stable•Sine wave input at frequency •Plant and model are (s+1)-1
g
Lennart Ljung Symposium Oct 200413
Performance: another example
g
•Unshaded regionis stable
•Sine wave input at frequency
•Plant is e-s(s+1)-1
while model is still(s+1)-1
Lennart Ljung Symposium Oct 200414
Averaging theory is the general analysis tool usable given separation of time scales of the plant dynamics and the learning/adaptation rate
Rescuing the MIT Rule: Averaging
€
˙ k c = −g y p − ym[ ]ymZp(s)kpkc(t)r(t)
Averaging theory treats kc slowly-varying : kc* is approximately kc for
small g where .
kc*=-g{Zp(s)kp r(t)}{Zm(s)kmr(t)} kc* + terms indep of kc* Stability is ensured if the average value of
{Zp(s)kp r(t)}{Zm(s)kmr(t)}
is positive--and if Zp is like Zm at frequencies where r(t) is concentrated, then stability is achieved.
Lennart Ljung Symposium Oct 200415
Performance: another example
g
•Unshaded regionis stable
•Sine wave input at frequency
•Plant is e-s(s+1)-1 iswhile model is still
(s+1)-1
Lennart Ljung Symposium Oct 200416
Rescuing the MIT Rule: Averaging
Lennart Ljung used averaging when he explained how to analyse the behaviour of a discrete time adaptive algorithm with the aid of an ordinary differential equation
The adaptation rate became slower and slower as time evolved--achieving the time scale separation.
Lennart Ljung Symposium Oct 200417
Adaptive Control
MIT Rule
Bursting
Good Models, Bad Models and Changing the Controller
Multiple Model Approach to Adaptive Control
Conclusions
Outline
Lennart Ljung Symposium Oct 200418
Bursting Phenomenon
Lennart Ljung Symposium Oct 200419
Bursting Phenomenon (continued)
Bursting phenomena were seen in an experimental adaptive control system - sometimes after 1 week of successful operation
Why do they occur? How could they be stopped?
bs+c
u(t) y(t)
˙ y +cy=bu
From measurements of u(•), y(•), one should be able to identify b and c If u = constant, can only identify b/c-the DC gain Adaptive controllers contain adaptive identifier of b and c
Lennart Ljung Symposium Oct 200420
Bursting Phenomenon (Explanation)
Control law is designed based on estimates of b,c. Hence could accidentally implement unstable closed loop.
Instability then enriches the signals, giving improved identification.
α2I > ss+Tφ t( )φT t( )dt>α1I
Identification process is robust if T such that for all s and
some positive 1, 2, regression vector (t) obeys:
or ∑ ss+Tφ k( )φT k( ) in place of integral[ ]
normally involves inputs and outputs. Need to convert to input-only condition
Lennart Ljung Symposium Oct 200421
Rich Excitation
If there are p scalar parameters to be identified, input needs to have a complexity related to p: (p/2 sinusoidal frequencies).
Practical issue: unless adaptation is turned off, must drive the system with “rich” input. [Some algorithms turn adaptation off at 1/t rate]
Lennart Ljung Symposium Oct 200422
Adaptive Control
MIT Rule
Bursting
Good Models, Bad Models and Changing the Controller
Multiple Model Approach to Adaptive Control
Conclusions
Outline
Lennart Ljung Symposium Oct 200423
Good Models, Bad Models and Changing the Controller
In adaptive control, at each time instant
• There is a model of the plant (which may be a good model)
• There is a certain controller attached to the plant
• If the plant model is a good one, a simulation of the model and controller will perform like the actual plant and controller
In adaptive control
• The controller may be changed to better reflect a control objective
• The calculation of the new controller is based on the current model--applying with the current controller
Lennart Ljung Symposium Oct 200424
Good Models, Bad Models and Changing the Controller (continued)
This presents a fundamental challenge in adaptive control Consider:
True plant:
Model:
[Transfer functions are and ]
0.1̇ ̇ y +1.1˙ y +y=u
˙ y +y=u
1s+1( ) 0.1s+1( )
1
s+1
Lennart Ljung Symposium Oct 200425
P1 =1
s+1Similar open-loop behaviours: and P2 =
1(s+1)(0.1s+1)
open-loop closed-loop
K =100(−)/ K =1(−−−)
Good Models, Bad Models and Changing the Controller (continued)
Lennart Ljung Symposium Oct 200426
P1 =1
s+1Different open-loop behaviours: and P2 =
1s
Plants in open-loop Plants in closed-loop with K =100
Good Models, Bad Models and Changing the Controller (continued)
Lennart Ljung Symposium Oct 200427
Good Models, Bad Models and Changing the Controller (continued)
Moral: changing the controller may turn a good model into a bad one, or vice versa
Changing the controller is like changing the experimental condition--and Lennart Ljung always told us to watch the experimental conditions!
“Goodness of fit of a model” is a term which only makes sense for a particular set of experimental conditions OR
Don’t overgeneralise what you have learnt
If you change the controller significantly, you might produce instability with the real plant, while it works fine with the model (=estimate of plant)
Lennart Ljung Symposium Oct 200428
A frequently advanced approach to adaptive control design is iterative identification and controller redesign.
One iteration comprises (re) identifying the plant with the current controller redesigning the controller to achieve the design
objective on the basis of the identified model, and implementing it on the real plant
Iterative Identification and Controller Redesign
This can lead to instability!
One needs algorithms which will move performance with the model and the new controller towards the design objective--but not change the controller too much.Same issue for IFT, VRFT Safe adaptive control.
Lennart Ljung Symposium Oct 200429
Adaptive Control
MIT Rule
Bursting
Good Models, Bad Models and Changing the Controller
Multiple Model Approach to Adaptive Control
Conclusions
Outline
Lennart Ljung Symposium Oct 200430
Multiple Model Adaptive Control
Imagine a bus on a city street. The equations of motion of the bus have parameters depending on
• the load• The friction between tyres and road
Many plants have equations in which a (frequently small) number of physically-originating parameters are changeable/unknown. Call such a plant p(). Here = physical parameter vector
Learning from measurements with an equation of the form
may be too hard, especially for nonlinear plants
ˆ ˙ λ = f ˆ λ ,measurements( )ˆ λ → λ
true
Lennart Ljung Symposium Oct 200431
Multiple Model Adaptive Control (continued)
An alternative approach (MMAC) is as follows:
• Suppose that the unknown parameter lies in a bounded simply connected region. Call the unknown plant .
• Choose a set of values in this region, with associated plants P1,.......,PN.
• Design (in advance) nice controllers for P1,.......,PN.
• Call them C1,......,CN .
• Run an algorithm which at any instant of time estimates (via the measurements) the particular Pi which is the best model to explain the measurements from . Call the associated parameter
• Connect up
P
P C
λ ˆ i
λ1,λ2,....λN
λˆ i
Lennart Ljung Symposium Oct 200432
Supervisor
noisereference + +
Controller i u Unknown or + y partially known
input - Plant P
Supervisor studies effect of using present controller and decides whether or not to switch controller
Desirable outcome: after a finite number of switchings, the best controller for the plant is obtained.
Unknown or Partially known
Plant P
Supervisor
Controller i
Multiple Model Adaptive Control(continued)
Lennart Ljung Symposium Oct 200433
Why the name “multiple model”?
Underlying precept is that the plant coincides with or is near one of N nominal plants P1,.......,PN
P
Controller i, denoted Ci, is a good controller for Pi
(and possibly plants “near” Pi)
Lennart Ljung Symposium Oct 200434
Deciding the Best Model Pi for P
u y1
Multi-y estimator yN
+r + Controller k Plant P -
Multiestimator is a device which produces N outputs if (and only if with complicated signals)
(The controller is irrelevant)
Controller k
Multi-estimator
yi =y
P =Pi
_Plant P
Lennart Ljung Symposium Oct 200435
Multi- y1
estimatory yN
r + Controller J Plant P
-
Idea of algorithm: study
for some small a > 0, and k=1,…,N. If the smallest occurs for
k = I, say that P is best modelled by
Switch in
_Plant P
Early Approach to Supervision: Using Multiestimator
Controller J
Multi-estimator
CI
PI
0t y−yk( )
2dt or
0te
−at−s( )2
y−yk( )2ds
u
∫
This may lead to switching in a destabilising controller!
Lennart Ljung Symposium Oct 200436
Example
Plant is 3rd order, stable, with non-minimum phase zero in [1,10] and DC gain in [.2,2].
Control objective is to extend bandwidth beyond open loop plant, with closed loop transfer function close to 1 in magnitude. Non-minimum phase zero is a limiter.
441 plant models chosen, with DC gain and non-minimum phase zero each in 20 logarithmically space intervals
Reference signal is wideband noise Measurement noise and process (input) noise are present
Lennart Ljung Symposium Oct 200437
Example of Temporary Instability
Figure 7a: Example of Temporary Instability (without safety)
Lennart Ljung Symposium Oct 200438
Example of Temporary Instability
Figure 7b: Example of Temporary Instability (without safety)
Lennart Ljung Symposium Oct 200439
Multiple Model Adaptive Control-Difficulties
How can one avoid the instabilities?Should there be 7, 70 or 7000 models? How
should one actually choose the models?
These questions are actually linked.
Lennart Ljung Symposium Oct 200440
How does one choose N? How does one choose 1,.........., N?
Idea of solution: Pick 1 . Design C1 for P( 1). Figure the plant set P() around P1 = P( 1) such that C1 is a good controller. Pick 2 near the boundary. Figure the plant set P() around P2 = P( 2) such that C2 is a good controller. Pick 3 near the boundary of union of these two sets, etc.
The set is then covered by a set of balls indexed by
1,.........., N and this determines N.
Choosing the Multiple Models
Metrics (Vinnicombe) help with this in the linear case
Lennart Ljung Symposium Oct 200441
Safe Switching
Difficulty is that P may be best modelled by PI when CJ is connected, but may be best modelled by PK when CI is connected. PI may be a terrible model of P when CI is connected.
The index of the best model of P (out of P1,........PN) with controller CJ connected is NOT NECESSARILY the index
of the best controller to connect to P. Nontrivial fact: using crude estimation techniques one can obtain set of controllers {CK } which, when used to replace CJ, are guaranteed to retain stability (and even retain similarity of performance). Vinnicombe metric is used.
Even if PI is the best model of P when CJ is connected, CI
may not be in the safe set of {CK } . Only switch if it is safe.
Lennart Ljung Symposium Oct 200442
OVERVIEW OF RESULTS
With safety constraint, controller switching is less frequent, convergence to the “best” controller was slower.
With safety constraint, performance could be poor but never unstable.
Without safety constraint, most runs exhibited poor performance, some yielded instability
With use of possibly more nominal plants and controllers, one can probably get “performance safety” as well as “stability safety”
Lennart Ljung Symposium Oct 200443
Safe Controller Switching
Figure 3a:(Safe) Controller Switching
Lennart Ljung Symposium Oct 200444
Safe Controller Switching
Figure 3b:(Safe) Controller Switching
Lennart Ljung Symposium Oct 200445
Adaptive Control
MIT Rule
Bursting
Good Models, Bad Models and Changing the Controller
Multiple Model Approach to Adaptive Control
Conclusions
Outline
Lennart Ljung Symposium Oct 200446
Conclusions
Keeping adaptation and plant time scales different is good practice
Modelling as well as you can is a good idea--even with an adaptation capability.
Having lots more parameters than you need could be dangerous
• Bursting• Satisfactory learning occurs only for a limited
set of experimental conditions
If you want to be able to keep learning (accurately) , you need to continue excitation
Lennart Ljung Symposium Oct 200447
Conclusions (continued)
A good model is only good for a particular set of experimental conditions. If you change the controller, it may cease to be good.
Picking representative models from an infinite set can often be done scientifically
Abrupt changes of a controller can introduce instability -even if on the basis of having a good model, the new controller looks good.
Safe adaptive control should be contemplated--to avoid temporary connection of a controller which can destabilise the (unknown) plant
Need Vinnicombe metric ideas for nonlinear problems