theory and applications of optimal control problems … · theory and applications of optimal...
TRANSCRIPT
Theory and Applications of Optimal ControlProblems with Time-Delays
Helmut Maurer
University of MunsterInstitute of Computational and Applied Mathematics
South Pacific Optimization Meeting (SPOM)
Newcastle, CARMA, 9–12 February 2013
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
EXPECT DELAYS
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Challenges for delayed optimal control problems
Theory and Numerics for non-delayed optimal control problemswith control and state constraints are rather complete:
1 Necessary and sufficient conditions,
2 Stability and sensitivity analysis,
3 Numerical methods:Boundary value methods, Discretization and NLP,Semismooth Newton methods,
4 Real-time control techniques for perturbed extremals.
Challenge: establish similar theoretical and numerical methodsfor delayed (retarded) optimal control problems.
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Book on SSC for regular and bang-bang controls
This book is devoted to the theory and applications of second-order necessary and sufficient optimality conditions in the calculus of variations and optimal control. The authors develop theory for a control problem with ordinary differential equations subject to boundary conditions of both the equality and inequality type and for mixed state-control constraints of the equality type. The book is distinctive in that
• necessary and sufficient conditions are given in the form of no-gap conditions,• the theory covers broken extremals where the control has finitely many points
of discontinuity, and• a number of numerical examples in various application areas are fully solved.
This book is suitable for researchers in calculus of variations and optimal control and researchers and engineers in optimal control applications in mechanics; mechatronics; physics; chemical, electrical, and biological engineering; and economics.
Nikolai P. Osmolovskii is a Professor in the Department of Informatics and Applied Mathematics, Moscow State Civil Engineering University; the Institute of Mathematics and Physics, University of Siedlce, Poland; the Systems Research Institute, Polish Academy of Science; the University of Technology and Humanities in Radom, Poland;
and of the Faculty of Mechanics and Mathematics, Moscow State University. He was an Invited Professor in the Department of Applied Mathematics, University of Bayreuth, Germany (2000), and at the Centre de Mathématiques Appliquées, École Polytechnique, France (2007). His fields of research are functional analysis, calculus of variations, and optimal control theory. He has written fifty papers and four monographs.
Helmut Maurer was a Professor of Applied Mathematics at the Universität Münster, Germany (retired 2010) and has conducted research in Austria, France, Poland, Australia, and the United States. His fields of research in optimal control are control and state constraints, numerical methods, second-order sufficient conditions, sensitivity analysis, real-time control techniques, and various applications in mechanics, mechatronics, physics, biomedical and chemical engineering, and economics.
For more information about SIAM books, journals, conferences, memberships, or activities, contact:
Society for Industrial and Applied Mathematics 3600 Market Street, 6th Floor
Philadelphia, PA 19104-2688 USA +1-215-382-9800 • Fax +1-215-386-7999
[email protected] • www.siam.org
DC24
DC24
Nikolai P. O
smolovskii
Helm
ut Maurer
ISBN 978-1-611972-35-1
Applications to Regular and Bang-Bang ControlSecond-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control
Applications to R
egular and Bang-B
ang Control
Second-Order N
ecessary and Sufficient Optim
ality C
onditions in Calculus of V
ariations and Optim
al Control
Nikolai P. OsmolovskiiHelmut Maurer
N. P
. Osm
olov
skii
H. M
aure
r
DC24_Osmolovskii-Maurer_cover.indd 1 10/2/2012 9:58:56 AM
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Overview
1 Formulation of optimal control problems withstate and control delays.
2 Example: Two-stage continuous stirred tankreactor (CSTR).
3 Minimum Principle.
4 NLP methods: discretize and optimize.
5 Optimal control of the innate immune response.
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Delayed Optimal Control Problem with State Constraints
State x(t) ∈ Rn, Control u(t) ∈ Rm, Delays dx , du ≥ 0.
Dynamics and Boundary Conditions
x(t) = f (x(t), x(t − dx), u(t), u(t − du)), a.e. t ∈ [0, tf ] ,
x(t) = x0(t), t ∈ [−dx , 0] ,
u(t) = u0(t), t ∈ [−du, 0),
ψ(x(tf )) = 0
Control and State Constraints
u(t) ∈ U ⊂ Rm, S(x(t)) ≤ 0, t ∈ [0, tf ] (S : Rn → Rk) .
Minimize
J(u, x) = g(x(tf )) +
∫ tf
0f0(x(t), x(t − dx), u(t), u(t − du)) dt
All functions are assumed to be sufficiently smooth.Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Two-Stage Continuous Stirred Tank Reactor (CSTR)
Time delays are caused by transport between the two tanks.
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Two Stage CSTR
Dadebo S. and Luus R. Optimal control of time-delay systemsby dynamic programming, Optimal Control Applications andMethods 13, pp. 29–41 (1992).
A chemical reaction A⇒ B is processed in two tanks.
State and control variables:
Tank 1 : x1(t) : (scaled) concentration
x2(t) : (scaled) temperature
u1(t) : temperature control
Tank 2 : x3(t) : (scaled) concentration
x4(t) : (scaled) temperature
u2(t) : temperature control
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Dynamics of the Two-Stage CSTR
Reaction term in Tank 1 : R1(x1, x2) = (x1 + 0.5) exp(
25x21+x2
)Reaction term in Tank 2 : R2(x3, x4) = (x3 + 0.25) exp
(25x41+x4
)Dynamics:
x1(t) =−0.5− x1(t)− R1(t),
x2(t) =−(x2(t) + 0.25)− u1(t)(x2(t) + 0.25) + R1(t),
x3(t) = x1(t − d)− x3(t)− R2(t) + 0.25,
x4(t) = x2(t − d)− 2x4(t)− u2(t)(x4(t) + 0.25) + R2(t)− 0.25.
Initial conditions:
x1(t) = 0.15, x2(t) = −0.03, −d ≤ t ≤ 0,
x3(0) = 0.1, x4(0) = 0.
Delays d = 0.1, d = 0.2, d = 0.4 in the state variables x1, x2 .
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Optimal control problem for the Two-Stage CSTR
Minimizetf∫0
( x21 + x22 + x23 + x24 + 0.1u21 + 0.1u22 ) dt (tf = 2) .
Hamiltonian with yk(t) = xk(t − d), k = 1, 2 :
H(x , y , λ, u) = f0(x , u) + λ1x1+λ2(−(x2 + 0.25)− u1(x2 + 0.25) + R1(x1, x2) )
+λ3(y1 − x3 − R2(x3, x4) + 0.25)
+λ4(y2 − 2x4 − u2(x4 + 0.25) + R2(x3, x4) + 0.25)
Advanced adjoint equations:
λ1(t) = −Hx1(t)− χ [ 0,tf−d ] λ3(t + d),
λ2(t) = −Hx2(t)− χ [ 0,tf−d ] λ4(t + d),
λk(t) = −Hxk (t) (k = 3, 4).
The minimum condition yields Hu = 0 and thus
u1 = 5λ2(x2 + 0.25), u2 = 5λ4(x4 + 0.25).
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Two-Stage CSTR with free x(tf ) : x1, x2, x3, x4
-0.02 0
0.02 0.04
0.06 0.08
0.1 0.12
0.14 0.16
0 0.5 1 1.5 2
concentration x1
d=0.1d=0.2d=0.4
-0.03
-0.02
-0.01
0
0.01
0.02
0 0.5 1 1.5 2
temperature x2
d=0.1d=0.2d=0.4
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
0 0.5 1 1.5 2
concentration x3
d=0.1d=0.2d=0.4
0 0.005
0.01 0.015
0.02 0.025
0.03 0.035
0.04 0.045
0 0.5 1 1.5 2
temperature x4
d=0.1d=0.2d=0.4
Delays d = 0.1, d = 0.2, d = 0.4.
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Two-Stage CSTR with free x(tf ) : u1, u2, λ1, λ2
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
control u1
d=0.1d=0.2d=0.4
0
0.05
0.1
0.15
0.2
0.25
0.3
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
control u2
d=0.1d=0.2d=0.4
-0.02 0
0.02 0.04
0.06 0.08
0.1 0.12
0.14 0.16
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
adjoint variable λ1
d=0.1d=0.2d=0.4
-0.4
-0.3
-0.2
-0.1
0
0.1
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
adjoint variable λ2
d=0.1d=0.2d=0.4
Delays d = 0.1, d = 0.2, d = 0.4.
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Two-Stage CSTR with x(tf ) = 0 : x1, x2, x3, x4
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0 0.5 1 1.5 2
concentration x1
d=0.1d=0.2d=0.4
-0.03
-0.02
-0.01
0
0.01
0.02
0 0.5 1 1.5 2
temperature x2
d=0.1d=0.2d=0.4
-0.02
0
0.02
0.04
0.06
0.08
0.1
0 0.5 1 1.5 2
concentration x3
d=0.1d=0.2d=0.4
0
0.005
0.01
0.015
0.02
0.025
0.03
0 0.5 1 1.5 2
temperature x4
d=0.1d=0.2d=0.4
Delays d = 0.1, d = 0.2, d = 0.4.
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Two-Stage CSTR with x(tf ) = 0 : u1, u2, λ1, λ2
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
control u1
d=0.1d=0.2d=0.4
0
0.05
0.1
0.15
0.2
0.25
0.3
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
control u2
d=0.1d=0.2d=0.4
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
adjoint variable λ1
d=0.1d=0.2d=0.4
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
adjoint variable λ2
d=0.1d=0.2d=0.4
Delays d = 0.1, d = 0.2, d = 0.4.
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Two-Stage CSTR with x(tf ) = 0 and x4(t) ≤ 0.01
0
0.002
0.004
0.006
0.008
0.01
0.012
0.014
0 0.5 1 1.5 2
temperature x4
d=0.1d=0.2d=0.4
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
control u1
d=0.1d=0.2d=0.4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 0.5 1 1.5 2
multiplier µ for x4 <= 0.01
d=0.1d=0.2d=0.4
0.05
0.1
0.15
0.2
0.25
0.3
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
control u2
d=0.1d=0.2d=0.4
Delays d = 0.1, d = 0.2, d = 0.4.
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Delayed Optimal Control Problem with State Constraints
State x(t) ∈ Rn, Control u(t) ∈ Rm, Delays dx , du ≥ 0.
Dynamics and Boundary Conditions
x(t) = f (x(t), x(t − dx), u(t), u(t − du)), a.e. t ∈ [0, tf ] ,
x(t) = x0(t), t ∈ [−dx , 0] ,
u(t) = u0(t), t ∈ [−du, 0),
ψ(x(tf )) = 0
Control and State Constraints
u(t) ∈ U ⊂ Rm, S(x(t)) ≤ 0, t ∈ [0, tf ] (S : Rn → Rk) .
Minimize
J(u, x) = g(x(tf )) +
∫ tf
0f0(x(t), x(t − dx), u(t), u(t − du)) dt
All functions are assumed to be sufficiently smooth.Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Literature on optimal control with time-delays
State delays and pure control constraints:
Kharatishvili (1961), Oguztoreli (1966), Banks (1968),Halanay (1968), Soliman, Ray (1970, chemical engineering),Warga (1972, abstract theory, optimization in Banach spaces),Guinn (1976, transform delayed problems to standard problems),Colonius, Hinrichsen (1978), Clarke, Wolenski (1991),Dadebo, Luus (1992), Mordukhovich, Wang (2003–).
State delays and pure state constraints: Angell, Kirsch (1990).
State and control delays and mixed control–state constraints:
Gollmann, Kern, Maurer (OCAM 2009),
Gollmann, Maurer: ”Theory and applications of optimal controlproblems with multiple time-delays,” to appear in JIMO 2013.
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Optimal control problems with state constraints
Use the transformation method of Guinn (1976) and transform anoptimal control problem with delays and state constraints to astandard non–delayed optimal control problem with stateconstraints. Then apply the
necessary conditions for non-delayed problems:
Jacobson, Lele, Speyer (1975): KKT conditions in Banachspaces.
Maurer (1979) : Regularity of multipliers for state constraints.
Hartl, Sethi, Thomsen (SIAM Review 1995): Survey onMaximum Principles.
Vinter (2000): (Nonsmooth) Optimal Control
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Hamiltonian
Hamiltonian (Pontryagin) Function
H(x , y , λ, u, v) :=λ0f0(t, x , y , u, v) + λf (t, x , y , u, v)
y variable with y(t) = x(t − dx)
v variable with v(t) = u(t − du)
λ ∈ Rn, λ0 ∈ R adjoint (costate) variable
Let (u, x) ∈ L∞([0, tf ],Rm)×W1,∞([0, tf ],Rn) be a
locally optimal pair of functions. Then there exist
an adjoint function λ ∈ BV([0, tf ],Rn) and λ0 ≥ 0,
a multiplier ρ ∈ Rq (associated with terminal conditions),
a multiplier function (measure) µ ∈ BV([0, tf ],Rk),
such that the following conditions are satisfied for a.e. t ∈ [0, tf ] :Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Minimum Principle
(i) Advanced adjoint equation and transversality condition:
λ(t) =tf∫t
(Hx(s) + χ[0,tf−dx ](t)Hy (s + dx) ) ds +tf∫tSx(x(s)) dµ(s)
+ (λ0g + ρψ)x(x(tf )) ( if S(x(tf )) < 0 ),
where Hx(t) and Hy (t + dx) denote evaluations along the optimaltrajectory and χ[0,tf−dx ] is the characteristic function.
(ii) Minimum Condition:
H(t) + χ[0,tf−du ](t)H(t + du)
= min w∈U [ H(x(t), y(t), λ(t),w , v(t))
+χ[0,tf−du ](t)H(t + du)H(x(t + du), y(t), λ(t + du), u(t + du),w) ]
(iii) Multiplier condition and complementarity condition:
dµ(t) ≥ 0,
tf∫0
S(x(t)) dµ(t) = 0
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Minimum Principle
(i) Advanced adjoint equation and transversality condition:
λ(t) =tf∫t
(Hx(s) + χ[0,tf−dx ](t)Hy (s + dx) ) ds +tf∫tSx(x(s)) dµ(s)
+ (λ0g + ρψ)x(x(tf )) ( if S(x(tf )) < 0 ),
where Hx(t) and Hy (t + dx) denote evaluations along the optimaltrajectory and χ[0,tf−dx ] is the characteristic function.
(ii) Minimum Condition:
H(t) + χ[0,tf−du ](t)H(t + du)
= min w∈U [ H(x(t), y(t), λ(t),w , v(t))
+χ[0,tf−du ](t)H(t + du)H(x(t + du), y(t), λ(t + du), u(t + du),w) ]
(iii) Multiplier condition and complementarity condition:
dµ(t) ≥ 0,
tf∫0
S(x(t)) dµ(t) = 0
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Minimum Principle
(i) Advanced adjoint equation and transversality condition:
λ(t) =tf∫t
(Hx(s) + χ[0,tf−dx ](t)Hy (s + dx) ) ds +tf∫tSx(x(s)) dµ(s)
+ (λ0g + ρψ)x(x(tf )) ( if S(x(tf )) < 0 ),
where Hx(t) and Hy (t + dx) denote evaluations along the optimaltrajectory and χ[0,tf−dx ] is the characteristic function.
(ii) Minimum Condition:
H(t) + χ[0,tf−du ](t)H(t + du)
= min w∈U [ H(x(t), y(t), λ(t),w , v(t))
+χ[0,tf−du ](t)H(t + du)H(x(t + du), y(t), λ(t + du), u(t + du),w) ]
(iii) Multiplier condition and complementarity condition:
dµ(t) ≥ 0,
tf∫0
S(x(t)) dµ(t) = 0
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Regularity conditions for dµ(t) = η(t)dt if du = 0
Boundary arc : S(x(t)) = 0 for t1 ≤ t ≤ t2.
Assumption : u(t) ∈ int(U) for t1 < t < t2 .
Under certain regularity conditions we have dµ(t) = η(t) dtwith a smooth multiplier η(t) for all t1 < t < t2 .
Adjoint equation and jump conditions
λ(t) = −Hx(t)− χ[0,tf−dx ]Hy (t + dx)− η(t)Sx(x(t))
λ(tk+) = λ(tk−)− νkSx(x(tk)) , νk ≥ 0
at each contact or junction time tk , νk = µ(tk+)− µ(tk−)
Minimum condition on the boundary
Hu(t) = 0 .
This condition allows to compute the multiplier η = η(x , λ).
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Model of the immune response
Dynamic model of the immune response:
Asachenko A, Marchuk G, Mohler R, Zuev S,Disease Dynamics, Birkhauser, Boston, 1994.
Optimal control:
Stengel RF, Ghigliazza R, Kulkarni N, Laplace O,Optimal control of innate immune response,Optimal Control Applications and Methods 23, 91–104 (2002),
Lisa Poppe, Julia Meskauskas: Diploma theses, UniversitatMunster (2006,2008).
L. Gollmann, H. Maurer: Optimal control problems with multipletime-delays, to appear in JIMO 2013.
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Innate Immune Response: state and control variablesState variables:
x1(t) : concentration of pathogen(=concentration of associated antigen)
x2(t) : concentration of plasma cells,which are carriers and producers of antibodies
x3(t) : concentration of antibodies, which kill the pathogen(=concentration of immunoglobulins)
x4(t) : relative characteristic of a damaged organ( 0 = healthy, 1 = dead )
Control variables:
u1(t) : pathogen killer
u2(t) : plasma cell enhancer
u3(t) : antibody enhancer
u4(t) : organ healing factor
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Innate Immune Response: state and control variablesState variables:
x1(t) : concentration of pathogen(=concentration of associated antigen)
x2(t) : concentration of plasma cells,which are carriers and producers of antibodies
x3(t) : concentration of antibodies, which kill the pathogen(=concentration of immunoglobulins)
x4(t) : relative characteristic of a damaged organ( 0 = healthy, 1 = dead )
Control variables:
u1(t) : pathogen killer
u2(t) : plasma cell enhancer
u3(t) : antibody enhancer
u4(t) : organ healing factor
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Generic dynamical model of the immune response
x1(t) = (1− x3(t))x1(t)− u1(t),
x2(t) = 3A(x4(t))x1(t − d)x3(t − d)− (x2(t)− 2) + u2(t) ,
x3(t) = x2(t)− (1.5 + 0.5x1(t))x3(t) + u3(t) ,
x4(t) = x1(t)− x4(t)− u4(t) .
Immune deficiency function triggered by target organ damage
A(x4) =
{cos(πx4) , 0 ≤ x4 ≤ 0.50 0.5 ≤ x4
}.
For 0.5 ≤ x4(t) the production of plasma cells stops.
State delay d ≥ 0 in variables x1 and x3
Initial conditions (d = 0) : x2(0) = 2, x3(0) = 4/3, x4(0) = 0
Case 1 : x1(0) = 1.5, decay, requires no therapy (control)Case 2 : x1(0) = 2.0, slower decay, requires no therapyCase 3 : x1(0) = 3.0, diverges without control (lethal case)
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Generic dynamical model of the immune response
x1(t) = (1− x3(t))x1(t)− u1(t),
x2(t) = 3A(x4(t))x1(t − d)x3(t − d)− (x2(t)− 2) + u2(t) ,
x3(t) = x2(t)− (1.5 + 0.5x1(t))x3(t) + u3(t) ,
x4(t) = x1(t)− x4(t)− u4(t) .
Immune deficiency function triggered by target organ damage
A(x4) =
{cos(πx4) , 0 ≤ x4 ≤ 0.50 0.5 ≤ x4
}.
For 0.5 ≤ x4(t) the production of plasma cells stops.
State delay d ≥ 0 in variables x1 and x3
Initial conditions (d = 0) : x2(0) = 2, x3(0) = 4/3, x4(0) = 0
Case 1 : x1(0) = 1.5, decay, requires no therapy (control)Case 2 : x1(0) = 2.0, slower decay, requires no therapyCase 3 : x1(0) = 3.0, diverges without control (lethal case)
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Generic dynamical model of the immune response
x1(t) = (1− x3(t))x1(t)− u1(t),
x2(t) = 3A(x4(t))x1(t − d)x3(t − d)− (x2(t)− 2) + u2(t) ,
x3(t) = x2(t)− (1.5 + 0.5x1(t))x3(t) + u3(t) ,
x4(t) = x1(t)− x4(t)− u4(t) .
Immune deficiency function triggered by target organ damage
A(x4) =
{cos(πx4) , 0 ≤ x4 ≤ 0.50 0.5 ≤ x4
}.
For 0.5 ≤ x4(t) the production of plasma cells stops.
State delay d ≥ 0 in variables x1 and x3
Initial conditions (d = 0) : x2(0) = 2, x3(0) = 4/3, x4(0) = 0
Case 1 : x1(0) = 1.5, decay, requires no therapy (control)Case 2 : x1(0) = 2.0, slower decay, requires no therapyCase 3 : x1(0) = 3.0, diverges without control (lethal case)
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Generic dynamical model of the immune response
x1(t) = (1− x3(t))x1(t)− u1(t),
x2(t) = 3A(x4(t))x1(t − d)x3(t − d)− (x2(t)− 2) + u2(t) ,
x3(t) = x2(t)− (1.5 + 0.5x1(t))x3(t) + u3(t) ,
x4(t) = x1(t)− x4(t)− u4(t) .
Immune deficiency function triggered by target organ damage
A(x4) =
{cos(πx4) , 0 ≤ x4 ≤ 0.50 0.5 ≤ x4
}.
For 0.5 ≤ x4(t) the production of plasma cells stops.
State delay d ≥ 0 in variables x1 and x3
Initial conditions (d = 0) : x2(0) = 2, x3(0) = 4/3, x4(0) = 0
Case 1 : x1(0) = 1.5, decay, requires no therapy (control)Case 2 : x1(0) = 2.0, slower decay, requires no therapyCase 3 : x1(0) = 3.0, diverges without control (lethal case)
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Optimal control model: cost functional
State x = (x1, x2, x3, x4) ∈ R4, Control u = (u1, u2, u3, u4) ∈ R4
L2-functional quadratic in control: Stengel et al.
Minimize J2(x , u) = x1(tf )2 + x4(tf )2
+tf∫0
( x21 + x24 + u21 + u22 + u23 + u24 ) dt
L1-functional linear in control
Minimize J1(x , u) = x1(tf )2 + x4(tf )2
+tf∫0
( x21 + x24 + u1 + u2 + u3 + u4 ) dt
Control constraints: 0 ≤ ui (t) ≤ umax, i = 1, .., 4
Final time: tf = 10Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Optimal control model: cost functional
State x = (x1, x2, x3, x4) ∈ R4, Control u = (u1, u2, u3, u4) ∈ R4
L2-functional quadratic in control: Stengel et al.
Minimize J2(x , u) = x1(tf )2 + x4(tf )2
+tf∫0
( x21 + x24 + u21 + u22 + u23 + u24 ) dt
L1-functional linear in control
Minimize J1(x , u) = x1(tf )2 + x4(tf )2
+tf∫0
( x21 + x24 + u1 + u2 + u3 + u4 ) dt
Control constraints: 0 ≤ ui (t) ≤ umax, i = 1, .., 4
Final time: tf = 10Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Optimal control model: cost functional
State x = (x1, x2, x3, x4) ∈ R4, Control u = (u1, u2, u3, u4) ∈ R4
L2-functional quadratic in control: Stengel et al.
Minimize J2(x , u) = x1(tf )2 + x4(tf )2
+tf∫0
( x21 + x24 + u21 + u22 + u23 + u24 ) dt
L1-functional linear in control
Minimize J1(x , u) = x1(tf )2 + x4(tf )2
+tf∫0
( x21 + x24 + u1 + u2 + u3 + u4 ) dt
Control constraints: 0 ≤ ui (t) ≤ umax, i = 1, .., 4
Final time: tf = 10Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
L2–functional, d = 0 : optimal state and control variables
State variables x1, x2, x3, x4 and optimal controls u1, u2, u3, u4 :
second-order sufficient conditions via matrix Riccati equation
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
L2–functional, d = 0 : state constraint x4(t) ≤ 0.2
State and control variables for state constraint x4(t) ≤ 0.2 .Boundary arc x4(t) ≡ 0.2 for t1 = 0.398 ≤ t ≤ t2 = 1.35
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
L2–functional, multiplier η(t) for constraint x4(t) ≤ 0.2
Compute multiplier η as function of (x , λ) :
η(x , λ) = λ2 3π sin(πx4)x1x3 − λ1 + 2λ4 − 2x3x1 + 2x4
Scaled multiplier 0.1 η(t) and boundary arc x4(t) = 0.2
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
L2–functional, delay d > 0 , constraint x4(t) ≤ α
Dynamics with state delay d > 0
x1(t) = (1− x3(t))x1(t)− u1(t),
x2(t) = 3 cos(πx4) x1(t − d)x3(t − d)− (x2(t)− 2) + u2(t) ,
x3(t) = x2(t)− (1.5 + 0.5x1(t))x3(t) + u3(t) ,
x4(t) = x1(t)− x4(t)− u4(t)
x4(t) ≤ α ≤ 0.5
Initial conditions
x1(t) = 0 , −d ≤ t < 0, x1(0) = 3,x3(t) = 4/3 , −d ≤ t ≤ 0,x2(0) = 2, x4(0) = 0.
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
L2–functional : delay d = 1 and x4(t) ≤ 0.2
State variables for d = 0 and d = 1
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
L2–functional : delay d = 1 and x4(t) ≤ 0.2
Optimal controls for d = 0 and d = 1
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
L2–functional, d = 1 : multiplier η(t) for x4(t) ≤ 0.2
Compute multiplier η as function of (x , λ) :
η(x , y , λ) = λ2 3π sin(πx4)y1y3 − λ1 + 2λ4 − 2x3x1 + 2x4
Scaled multiplier 0.1 η(t) and boundary arc x4(t) = 0.2 ;
η(t) is discontinuous at t = d = 1
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
L1–functional and state delay d ≥ 0
Minimize
J1(x , u) = p11x1(tf )2 + p44x4(tf )2
+tf∫0
( p11x21 + p44x
24 + q1 u1 + q2 u2 + q3 u3 + q4 u4 ) dt
Dynamics with delay d and control constraints
x1(t) = (1− x3(t))x1(t)− u1(t),
x2(t) = 3A(x4(t))x1(t − r)x3(t − r)− (x2(t)− 2) + u2(t) ,
x3(t) = x2(t)− (1.5 + 0.5x1(t))x3(t) + u3(t) ,
x4(t) = x1(t)− x4(t)− u4(t) ,
0 ≤ ui (t) ≤ umax , 0 ≤ t ≤ tf (i = 1, .., 4)
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
L1–functional : non-delayed problem d = 0 : umax = 2
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
L1–functional : delayed problem d = 1 : umax = 2
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
L1–functional : non-delayed, time–optimal control forx1(tf ) = x4(tf ) = 0, x3(tf ) = 4/3
umax = 1: minimal time tf = 2.2151, singular arc for u4(t)
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Further applications and future work
1 Optimal oil extraction and exploration : state delay (Bruns,Maurer, Semmler)
2 Biomedical applications: optimal protocols in cancertreatment and immunology
3 Vintage control problems
4 Delayed control problems with free final time
5 Optimal control problems with state-dependent delays
6 Verifiable sufficient conditions
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays
Thank you for your attention !
Helmut Maurer [2mm] University of Munster Institute of Computational and Applied Mathematics [-4mm]Optimal Control Problems with Time-Delays