pontryagin's maximum principle for optimal control of spdes talks workshop... · 2012. 1....

Problem formulationPontryagin stochastic maximum principle

Plan of the proof

Pontryagin’s maximum principle for optimalcontrol of SPDEs

AbdulRahman Al-Hussein

Department of Mathematics, College of Science, Qassim University,P. O. Box 6644, Buraydah 51452, Saudi Arabia

E-mail: alhusseinqu@hotmail.com

Workshop on Stochastic Control Problems forFBSDEs and Applications

Marrakesh, December 13-18, 2010

AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs

Plan of the proof

StatementAssumptionsThe adjoint equation

Outline

1 Problem formulationStatementAssumptionsThe adjoint equation

2 Pontryagin stochastic maximum principle

3 Plan of the proofEstimatesVariational inequality

ProofTwo more lemmasProof of main result

Plan of the proof

Consider the SPDE on a sep. Hilbert space K :dX u(·)(t) = (A(t) X u(·)(t) + F (X u(·)(t),u(t)) )dt + G(X u(·)(t))dM(t),X u(·)(0) = x .

M is a continuous martingale, << M >>t =∫ t

0 Q(s) ds,for a predictable process Q(·) s.t. Q(t) ∈ L1(K ) is symmetric,positive definite, Q(t) ≤ Q, where Q ∈ L1(K ) (positive definite).

F : K ×O → K (O is a sep. Hilbert space).

G : K → L2(K0,K ), K0 = Q−1/2(K ).

A(t , ω) is a predictable unbounded linear operator on K .

Plan of the proof

. We shall derive a stochastic maximum principle for this controlproblem by using the adjoint equation (BSPDE).

u(·) : [0,T ]× Ω→ O is admissible if u(·) ∈ L2F (0,T ;O) and

u(t) ∈ U a.e., a.s. ( U is a nonempty convex subset of O ) .

The set of admissible controls Uad .

L2F (0,T ; E) := ψ : [0,T ]× Ω→ E ,predictable,

E [∫ T

0 |ψ(t)|2Edt ] <∞ .

Plan of the proof

. We shall derive a stochastic maximum principle for this controlproblem by using the adjoint equation (BSPDE).

u(·) : [0,T ]× Ω→ O is admissible if u(·) ∈ L2F (0,T ;O) and

u(t) ∈ U a.e., a.s. ( U is a nonempty convex subset of O ) .

The set of admissible controls Uad .

L2F (0,T ; E) := ψ : [0,T ]× Ω→ E ,predictable,

E [∫ T

0 |ψ(t)|2Edt ] <∞ .

Plan of the proof

Definethe cost functional:

J(u(·)) := E [

0g(X u(·)(t), u(t) ) dt + φ(X u(·)(T )) ],

J∗ := infJ(u(·)) : u(·) ∈ Uad.

The control problem for this SPDE (1) is to find a control u∗(·) and thecorresponding solution X u∗(·) of (1) s.t.

J∗ = J(u∗(·)).

u∗(·) is an optimal control.

X u∗(·) (or briefly X ∗) is an optimal solution.

The pair (X ∗ ,u∗(·)) is an optimal pair.

Plan of the proof

Definethe cost functional:

J(u(·)) := E [

0g(X u(·)(t), u(t) ) dt + φ(X u(·)(T )) ],

J∗ := infJ(u(·)) : u(·) ∈ Uad.

The control problem for this SPDE (1) is to find a control u∗(·) and thecorresponding solution X u∗(·) of (1) s.t.

J∗ = J(u∗(·)).

u∗(·) is an optimal control.

X u∗(·) (or briefly X ∗) is an optimal solution.

The pair (X ∗ ,u∗(·)) is an optimal pair.

Plan of the proof

(H1) F ,G,g, φ are C1 w.r.t. x , F is C1 w.r.t. u,the derivatives Fx , Fu, Gx ,gx are uniformly bounded.

Also |φx |K ≤ C1 (1 + |x |K ), some constant C1 > 0.

(H2) gx satisfies Lipschitz condition with respect to u uniformly inx .

(H3) A(t , ω) is a predictable linear operator on K , belongs toL(V ; V ′) ( (V ,K ,V ′) is a Gelfand triple ),

1 2⟨A(t , ω) y , y

⟩+ α |y |2

V≤ λ |y |2 a.e. t ∈ [0,T ] , a.s. ∀ y ∈ V ,

for some α, λ > 0.

2 ∃ C2 ≥ 0 s.t. |A(t , ω) y |V ′ ≤ C2 |y |V ∀ (t , ω), ∀ y ∈ V .

Plan of the proof

Define the Hamiltonian:

H : [0,T ]× Ω× K ×O × K × L2(K )→ R,

H(t , x ,u, y , z) := −g(x ,u) +⟨F (x ,u) , y

⟩+⟨G(x)Q1/2(t) , z

⟩2 .

The (adjoint) BSPDE:

−dY u(·)(t) = [ A∗(t) Y u(·)(t) +∇xH(X u(·)(t),u(t),Y u(·)(t),Z u(·)(t)Q1/2(t)) ]dt−Z u(·)(t)dM(t)− dNu(·)(t), 0 ≤ t ≤ T ,

Y u(·)(T ) = −∇φ(X u(·)(T )),

A∗(t) is the adjoint operator of A(t).

Plan of the proof

Define the Hamiltonian:

H : [0,T ]× Ω× K ×O × K × L2(K )→ R,

H(t , x ,u, y , z) := −g(x ,u) +⟨F (x ,u) , y

⟩+⟨G(x)Q1/2(t) , z

⟩2 .

The (adjoint) BSPDE:

−dY u(·)(t) = [ A∗(t) Y u(·)(t) +∇xH(X u(·)(t),u(t),Y u(·)(t),Z u(·)(t)Q1/2(t)) ]dt−Z u(·)(t)dM(t)− dNu(·)(t), 0 ≤ t ≤ T ,

Y u(·)(T ) = −∇φ(X u(·)(T )),

A∗(t) is the adjoint operator of A(t).

Plan of the proof

M2,c[0,T ](K ) the space of continuous square integrable

martingales in K .

Two elements M and N ofM2,c[0,T ](K ) are very strongly orthogonal

(VSO) ifE [M(τ)⊗ N(τ)] = E [M(0)⊗ N(0)],

for all [0,T ] - valued stopping times τ.

In fact: M and N are VSO⇔ << M,N >> = 0.

Λ2(K ;P,M) the space of integrands w.r.t. M s.t.

Φ(t , ω)Q1/2(t , ω) ∈ L2(K ), for every h ∈ K

the K - valued process Φ Q1/2(h) is predictable,

E [∫ T

0 ||(Φ Q1/2)(t)||22 dt ] <∞.

Plan of the proof

martingales in K .

(VSO) ifE [M(τ)⊗ N(τ)] = E [M(0)⊗ N(0)],

E [∫ T

0 ||(Φ Q1/2)(t)||22 dt ] <∞.

Plan of the proof

martingales in K .

(VSO) ifE [M(τ)⊗ N(τ)] = E [M(0)⊗ N(0)],

E [∫ T

0 ||(Φ Q1/2)(t)||22 dt ] <∞.

Plan of the proof

DefinitionA solution of a BSPDE: −dY (t) =

(A(t)Y (t) + f (t ,Y (t),Z (t)Q1/2(t))

−Z (t)dM(t)− dN(t), 0 ≤ t ≤ T ,Y (T ) = ξ,

is (Y ,Z ,N) ∈ L2F (0,T ; V )× Λ2(K ;P,M)×M2,c

[0,T ](K ) s.t. ∀ t ∈ [0,T ] :

Y (t) = ξ +

(A(s) Y (s) + f (s,Y (s),Z (s)Q1/2(s))

−∫ T

tZ (s)dM(s)−

tdN(s),

N(0) = 0, N is VSO to M.

Plan of the proof

. The following theorem gives the unique solution to the adjointequation.

Theorem 1 (Existence & uniqueness of the solution of the adjointBSPDE)

Assume (H1)–(H3). There exists a unique solution (Y u(·),Z u(·),Nu(·) )

to the adjoint BSPDE in L2F (0,T ; V )× Λ2(K ;P,M)×M2,c

[0,T ](K ).

The proof of this theorem can be found in:[ Al-Hussein, A., Backward stochastic partial differential equations driven byinfinite dimensional martingales and applications, Stochastics, Vol. 81, No. 6,2009, 601-626 ].

Denote the solution of the (adjoint) BSPDE corresponding to u∗(·) by(Y ∗,Z ∗,N∗).

Plan of the proof

[0,T ](K ).

Plan of the proof

[0,T ](K ).

Plan of the proof

Outline

Plan of the proof

Theorem 2 (Pontryagin stochastic maximum principle)

Suppose (H1)–(H3). Assume (X ∗,u∗(·)) is an optimal pair for ourcontrol problem.

Then there exists a unique solution (Y ∗,Z ∗,N∗) to the correspondingBSPDE s.t. the following inequality holds:

⟨∇uH(t ,X ∗(t),u∗(t),Y ∗(t),Z ∗(t)Q1/2(t)),u − u∗(t)

⟩O ≤ 0,

∀ u ∈ U, a.e. t ∈ [0,T ], a.s.

Plan of the proof

EstimatesVariational inequalityTwo more lemmasProof of main result

Outline

Plan of the proof

. Let u∗(·) be an optimal control and X ∗ be the corresponding

solution of SPDE (1). Let u(·) be s.t. u∗(·) + u(·) ∈ Uad .

For 0 ≤ ε ≤ 1 consider the variational control:

uε(t) = u∗(t) + εu(t), t ∈ [0,T ].

The convexity of U ⇒ uε(·) ∈ Uad .

. Get the corresponding Xε of (1).

. Let p be the solution of the linear equation: dp(t) = (A(t)p(t) + Fx (X ∗(t),u∗(t))p(t))dt+ Fu(X ∗(t),u∗(t))u(t)dt + Gx (X ∗(t))p(t)dM(t),

p(0) = 0.

Plan of the proof

. Let u∗(·) be an optimal control and X ∗ be the corresponding

solution of SPDE (1). Let u(·) be s.t. u∗(·) + u(·) ∈ Uad .

For 0 ≤ ε ≤ 1 consider the variational control:

uε(t) = u∗(t) + εu(t), t ∈ [0,T ].

The convexity of U ⇒ uε(·) ∈ Uad .

. Get the corresponding Xε of (1).

. Let p be the solution of the linear equation: dp(t) = (A(t)p(t) + Fx (X ∗(t),u∗(t))p(t))dt+ Fu(X ∗(t),u∗(t))u(t)dt + Gx (X ∗(t))p(t)dM(t),

p(0) = 0.

Plan of the proof

. We obtain:

Lemma 1Assume (H1), (H3). Let

ηε(t) =Xε(t)− X ∗(t)

ε− p(t), t ∈ [0,T ].

1 supt∈[0,T ]

E [ |p(t)|2 ] <∞,

2 supt∈[0,T ]

E [ |Xε(t)− X ∗(t)|2 ] = O(ε2),

3 limε→0+

supt∈[0,T ]

E [ |ηε(t)|2 ] = 0.

Plan of the proof

Lemma 2 (Variational inequality)

Suppose (H1)–(H3). Then ∀ ε > 0,

J(uε(·))− J(u∗(·)) = ε E [φx (X ∗(T )) p(T ) ]

+ ε E [

0gx (X ∗(s),u∗(s)) p(s) ds ]

(g(X ∗(s),uε(s))− g(X ∗(s),u∗(s))

+ o(ε).

Plan of the proof

Sketch proof of Lemma 2

J(uε(·))− J(u∗(·)) = I1(ε) + I2(ε),

I1(ε) = E [φ(Xε(T ))− φ(X ∗(T )) ],

I2(ε) = E [

(g(Xε(s),uε(s))− g(X ∗(s),u∗(s))

)ds ].

Then apply Lemma 1 (2), (3).

Plan of the proof

Lemma 3If (H1)–(H3) hold, then

− ε E⟨

Y ∗(T ),p(T )⟩

+ ε E [

0gx (X ∗(s),u∗(s)) p(s) ds ]

0(−δεH(s) +

⟨Y ∗(s) , δεF (s)

⟩) ds ] ≥ o(ε),

δεF (s) = F (X ∗(s),uε(s))− F (X ∗(s),u∗(s)),

δεH(s) = H(X ∗(s),uε(s),Y ∗(s),Z ∗(s)Q1/2(s))

−H(X ∗(s),u∗(s),Y ∗(s),Z ∗(s)Q1/2(s)).

Plan of the proof

Proof of Lemma 3

. Since u∗(·) is an optimal control, J(uε(·))− J(u∗(·)) ≥ 0.

. Next apply Lemma 2 and the definition of the Hamiltonian H.

Next need the following relation:

Plan of the proof

Proof of Lemma 3

. Since u∗(·) is an optimal control, J(uε(·))− J(u∗(·)) ≥ 0.

. Next apply Lemma 2 and the definition of the Hamiltonian H.

Next need the following relation:

Plan of the proof

Lemma 4

Y ∗(T ),p(T )⟩

0gx (X ∗(s),u∗(s)) p(s) ds ]

⟨Y ∗(s),Fu(X ∗(s),u∗(s))u(s)

⟩ds ].

Proof of Lemma 4

. Use Ito’s formula together with⟨∇xH(X ∗(t),u∗(t),Y ∗(t),Z ∗(t)Q1/2(t)),p(t)

⟩= − gx (X ∗(t),u∗(t)) p(t) +

⟨Fx (X ∗(t),u∗(t)) p(t) ,Y ∗(t)

⟩+⟨

G(X ∗(t))Q1/2(t) ,Z ∗(t)Q1/2(t)⟩

2 a.s. ∀ t ∈ [0,T ].

Plan of the proof

Lemma 4

Y ∗(T ),p(T )⟩

0gx (X ∗(s),u∗(s)) p(s) ds ]

⟨Y ∗(s),Fu(X ∗(s),u∗(s))u(s)

⟩ds ].

Proof of Lemma 4

. Use Ito’s formula together with⟨∇xH(X ∗(t),u∗(t),Y ∗(t),Z ∗(t)Q1/2(t)),p(t)

⟩= − gx (X ∗(t),u∗(t)) p(t) +

⟨Fx (X ∗(t),u∗(t)) p(t) ,Y ∗(t)

⟩+⟨

G(X ∗(t))Q1/2(t) ,Z ∗(t)Q1/2(t)⟩

2 a.s. ∀ t ∈ [0,T ].

Plan of the proof

Proof of main resultConsider the adjoint equation. From Lemma 3, Lemma 4 get

⟨Y ∗(s), δεF (s)− εFu(X ∗(s),u∗(s))u(s)

⟩ds ]

− E [

0δεH(s)ds ] ≥ o(ε).

The continuity, boundedness of Fu in (H1) and the DCT give

1εE [

⟨Y ∗(s), δεF (s)− ε Fu(X ∗(s),u∗(s)) u(s)

⟩ds ]

= E[ ∫ T

⟨Y ∗(s),

(Fu(X ∗(s),u∗(s) + θ(uε(s)− u∗(s)))

−Fu(X ∗(s),u∗(s)))

u(s) dθ⟩

ds]→ 0,

as ε→ 0+.

Plan of the proof

Proof of main resultConsider the adjoint equation. From Lemma 3, Lemma 4 get

⟨Y ∗(s), δεF (s)− εFu(X ∗(s),u∗(s))u(s)

⟩ds ]

− E [

0δεH(s)ds ] ≥ o(ε).

The continuity, boundedness of Fu in (H1) and the DCT give

1εE [

⟨Y ∗(s), δεF (s)− ε Fu(X ∗(s),u∗(s)) u(s)

⟩ds ]

= E[ ∫ T

⟨Y ∗(s),

(Fu(X ∗(s),u∗(s) + θ(uε(s)− u∗(s)))

−Fu(X ∗(s),u∗(s)))

u(s) dθ⟩

ds]→ 0,

as ε→ 0+.

Plan of the proof

continued proof

In particular

⟨Y ∗(s), δεF (s)− εFu(X ∗(s),u∗(s)) u(s)

⟩ds ] = o(ε).

∴ − E [

0δεH(s) ds ] ≥ o(ε).

∴ by dividing this inequality by ε and letting ε→ 0+ :

0∇uH(t ,X ∗(t),u∗(t),Y ∗(t),Z ∗(t)Q1/2(t)),u(t)

⟩dt ] ≤ 0.

The theorem then follows.

Plan of the proof

Thank you

pontryagin's maximum principle for optimal control of spdes talks workshop... · 2012. 1....

Documents

optimal control: the maximum principle

maximum principle for infinite-horizon optimal control

principle and maximum shearing stresses1 principle and...

using the maximum entropy production principle to

pearsonthe principle of maximum hardness

01 parr - principle of maximum hardness

the maximum principle: continuous...

optimal control and maximum principle -...

maximum-principle-satisfying space-time conservation element...

maximum modulus principle:

stochastic global maximum principle for optimization with...

principle maximum entropy

chapter 8 the maximum principle: discrete time

maximum-principle-satisfying and positivity-preserving

petr veverka - infinite time maximum principle

objective bayesianism and the maximum entropy principle

maximum principle pontreyagin

ad 267 860 - dticafosr tn 1489 division of engineering...

the maximum principle for beltrami color ﬂow

a biharmonic maximum principle for hyperbolic surfaces