optimal control and mean field games (part 3)€¦ · optimal control and mean field games (part 3)...
TRANSCRIPT
![Page 1: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/1.jpg)
Optimal Control and Mean Field Games(Part 3)
P. Cardaliaguet
Paris-Dauphine
“NEW TRENDS IN OPTIMAL CONTROL"Ravello, Italy
September, 3-7, 2012
Joint work with P.-M. Cannarsa (in progress)
P. Cardaliaguet (Paris-Dauphine) MFG 1 / 63
![Page 2: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/2.jpg)
Aim of the lectures
√ Describe the model and its interpretations(Part 1)
√ Existence of the MFG system by fixed point arguments(Non local MFG - Part 2)
→ The MFG system as optimality condition for an optimal control problemof HJ equations(Local MFG - Part 3)
P. Cardaliaguet (Paris-Dauphine) MFG 2 / 63
![Page 3: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/3.jpg)
Part 3
The MFG system and optimal control problems for PDE equations
P. Cardaliaguet (Paris-Dauphine) MFG 3 / 63
![Page 4: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/4.jpg)
The Mean Field Game system
We are interested in the MFG system
(MFG)
(i) −∂tu + H(x ,Du) = f (x ,m(x , t))(ii) ∂tm − div(mDpH(x ,Du)) = 0(iii) m(0) = m0, u(x ,T ) = uT (x)
where f : Rd × [0,+∞)→ [0,+∞) is a local coupling term.
Problem : Existence/uniqueness of a solutions - smoothness properties.
P. Cardaliaguet (Paris-Dauphine) MFG 4 / 63
![Page 5: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/5.jpg)
Similar systems
The Monge-Kantorovitch optimal transport problems : minimize the costto transport a probability density m0 onto a probability density m1. Thedual problem reads
max∫
Rdu(x)d(m1 − m0)(x), u : Rd → R 1−Lipschitz continuous
.
If u is optimal, the system of necessary conditions reads|Du| ≤ 1 in Rd , |Du| = 1 in m > 0−div(mDu) = m1 − m0 in Rd
where m is the transport density.−→ Existence of solutions, uniqueness of m and uniqueness (up toadditive constants) of u on each connected component of m > 0.(Evans-Gangbo 1999, Feldman-Mc Cann 2002, Ambrosio 2003)
P. Cardaliaguet (Paris-Dauphine) MFG 5 / 63
![Page 6: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/6.jpg)
Sandpile model : Introduced by Hadeler, Kuttler (1999). Ω ⊂ R2 is abounded table, on which one pours sand with a rate f = f (x). In thestationary regime, the heap of sand consists in a standing layer u and arolling layer m. |Du| ≤ 1 in Ω, |Du| = 1 in m > 0
−div(mDu) = f in Ωu = 0 on ∂Ω
−→ existence and representation of solutions, uniqueness of m anduniqueness of u in m > 0.(Feldman (1999), Carnarsa-C. (2004), Crasta-Finzi Vita (2008),Cannarsa-C.-Sinestrari (2009)).
P. Cardaliaguet (Paris-Dauphine) MFG 6 / 63
![Page 7: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/7.jpg)
Adjoint methods for Hamilton-Jacobi PDE : analysis of the vanishingviscosity limit :
∂tuε + H(Duε) = ε∆uε,−∂tmε − div
(DH(Duε)
)= ε∆mε,
uε(0, x) = u0(x), mε(1, .) = m1.
−→ better understand the convergence of the vanishing viscositymethod when H is nonconvex.(Evans (2010))
P. Cardaliaguet (Paris-Dauphine) MFG 7 / 63
![Page 8: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/8.jpg)
A congestion model : An optimal transport problem related to congestionyields to the following system of PDEs : for α ∈ (0,1),
∂tu +α
4mα−1|Du|2 = 0,
∂tm + div(1
2mαDu
)= 0,
m(0, .) = m0, m(1, .) = m1.
(Benamou-Brenier formulation of Wasserstein distance : α = 1 -Dolbeault-Nazaret-Savaré, 2009)−→ analysis of the system : Existence and uniqueness of a weaksolution(C.-Carlier-Nazaret 2012.)
P. Cardaliaguet (Paris-Dauphine) MFG 8 / 63
![Page 9: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/9.jpg)
Back to the MFG system
(MFG)
(i) −∂tu + H(x ,Du) = f (x ,m(x , t))(ii) ∂tm − div(mDpH(x ,Du)) = 0(iii) m(0) = m0, u(x ,T ) = uT (x)
under the assumptions that
f : Rd × [0,+∞)→ [0,+∞) is a local coupling term.
all the data are Zd−periodic in space : denoted by ].
P. Cardaliaguet (Paris-Dauphine) MFG 9 / 63
![Page 10: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/10.jpg)
Difficulties for the MFG with local coupling
For nonlocal coupling, existence/regularity of the MFG system comefrom
semi-concavity estimates for HJ equations with C2 RHS,preservation of the absolute continuity of m0 by Kolmogorovequation for gradient flows of semi-concave functions.
For local equations,
the RHS is a priori only measurable (or integrable),the HJ with discontinuous RHS is poorly understood,the Kolmogorov equation with Du ∈ L∞ is ill-posed as well.
−→ requires a completely different approach.
P. Cardaliaguet (Paris-Dauphine) MFG 10 / 63
![Page 11: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/11.jpg)
Difficulties for the MFG with local coupling
For nonlocal coupling, existence/regularity of the MFG system comefrom
semi-concavity estimates for HJ equations with C2 RHS,preservation of the absolute continuity of m0 by Kolmogorovequation for gradient flows of semi-concave functions.
For local equations,
the RHS is a priori only measurable (or integrable),the HJ with discontinuous RHS is poorly understood,the Kolmogorov equation with Du ∈ L∞ is ill-posed as well.
−→ requires a completely different approach.
P. Cardaliaguet (Paris-Dauphine) MFG 10 / 63
![Page 12: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/12.jpg)
Difficulties for the MFG with local coupling
For nonlocal coupling, existence/regularity of the MFG system comefrom
semi-concavity estimates for HJ equations with C2 RHS,preservation of the absolute continuity of m0 by Kolmogorovequation for gradient flows of semi-concave functions.
For local equations,
the RHS is a priori only measurable (or integrable),the HJ with discontinuous RHS is poorly understood,the Kolmogorov equation with Du ∈ L∞ is ill-posed as well.
−→ requires a completely different approach.
P. Cardaliaguet (Paris-Dauphine) MFG 10 / 63
![Page 13: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/13.jpg)
Idea for the MFG system
Inspired with similar problems in optimal transport, we look for optimizationproblems with the MFG system as necessary condition.
This is the case
For an optimal control problem for an Hamilton-Jacobi equation
For an optimal control problem for a Kolmogorov equation
The second one is the dual of the first one.
P. Cardaliaguet (Paris-Dauphine) MFG 11 / 63
![Page 14: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/14.jpg)
Idea for the MFG system
Inspired with similar problems in optimal transport, we look for optimizationproblems with the MFG system as necessary condition.
This is the case
For an optimal control problem for an Hamilton-Jacobi equation
For an optimal control problem for a Kolmogorov equation
The second one is the dual of the first one.
P. Cardaliaguet (Paris-Dauphine) MFG 11 / 63
![Page 15: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/15.jpg)
Idea for the MFG system
Inspired with similar problems in optimal transport, we look for optimizationproblems with the MFG system as necessary condition.
This is the case
For an optimal control problem for an Hamilton-Jacobi equation
For an optimal control problem for a Kolmogorov equation
The second one is the dual of the first one.
P. Cardaliaguet (Paris-Dauphine) MFG 11 / 63
![Page 16: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/16.jpg)
Idea for the MFG system
Inspired with similar problems in optimal transport, we look for optimizationproblems with the MFG system as necessary condition.
This is the case
For an optimal control problem for an Hamilton-Jacobi equation
For an optimal control problem for a Kolmogorov equation
The second one is the dual of the first one.
P. Cardaliaguet (Paris-Dauphine) MFG 11 / 63
![Page 17: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/17.jpg)
Outline
1 Preliminary estimates on HJ equations
2 Optimal control of the HJ equation
3 Optimal control of the Kolmogorov equation
4 Existence/uniqueness of solutions for the (MFG) system
5 Conclusion
P. Cardaliaguet (Paris-Dauphine) MFG 12 / 63
![Page 18: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/18.jpg)
Outline
1 Preliminary estimates on HJ equations
2 Optimal control of the HJ equation
3 Optimal control of the Kolmogorov equation
4 Existence/uniqueness of solutions for the (MFG) system
5 Conclusion
P. Cardaliaguet (Paris-Dauphine) MFG 12 / 63
![Page 19: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/19.jpg)
Outline
1 Preliminary estimates on HJ equations
2 Optimal control of the HJ equation
3 Optimal control of the Kolmogorov equation
4 Existence/uniqueness of solutions for the (MFG) system
5 Conclusion
P. Cardaliaguet (Paris-Dauphine) MFG 12 / 63
![Page 20: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/20.jpg)
Outline
1 Preliminary estimates on HJ equations
2 Optimal control of the HJ equation
3 Optimal control of the Kolmogorov equation
4 Existence/uniqueness of solutions for the (MFG) system
5 Conclusion
P. Cardaliaguet (Paris-Dauphine) MFG 12 / 63
![Page 21: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/21.jpg)
Outline
1 Preliminary estimates on HJ equations
2 Optimal control of the HJ equation
3 Optimal control of the Kolmogorov equation
4 Existence/uniqueness of solutions for the (MFG) system
5 Conclusion
P. Cardaliaguet (Paris-Dauphine) MFG 12 / 63
![Page 22: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/22.jpg)
Preliminary estimates on HJ equations
Outline
1 Preliminary estimates on HJ equations
2 Optimal control of the HJ equation
3 Optimal control of the Kolmogorov equation
4 Existence/uniqueness of solutions for the (MFG) system
5 Conclusion
P. Cardaliaguet (Paris-Dauphine) MFG 13 / 63
![Page 23: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/23.jpg)
Preliminary estimates on HJ equations
Aim
We study the optimal control of Hamilton-Jacobi equations :
(HJ)
−∂tu + H(x ,Du) = α in (0,T )× Rd
u(T , x) = uT (x) in Rd
by the control α = α(t , x).
−→ Find estimates for the solution u in terms of integral norms of α.
L∞ estimates
Hölder estimates
P. Cardaliaguet (Paris-Dauphine) MFG 14 / 63
![Page 24: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/24.jpg)
Preliminary estimates on HJ equations
Discussion
The viscosity solution of
(HJ)
−∂tu + H(x ,Du) = α in (0,T )× Rd
u(T , x) = uT (x) in Rd
is represented by the optimal control problem
u(t , x) = infξ(·), ξ(t)=x
∫ T
t(L(ξ(s), ξ′(s)) + α(s, ξ(s))) ds + uT (ξ(T ))
where L(x , y) := supz〈z, y〉 − H(x ,−z) is as smooth as H.
The optimal control avoids places where α is large...
but if α << −1 even at some small places, then u << −1. So smallnessof α leads to instabilities.
P. Cardaliaguet (Paris-Dauphine) MFG 15 / 63
![Page 25: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/25.jpg)
Preliminary estimates on HJ equations
Assumptions
The Hamiltonian H : Rd × Rd → R is of class C1 and has a superlineargrowth in the gradient variable : there is r > 1 such that
1C|ξ|r − C ≤ H(x , ξ) ≤ C(|ξ|r + 1) ∀(x , ξ) ∈ Rd × Rd .
x → H(x , z) is Zd periodic for any z.
α ∈ Lp] ([0,T ]× Rd ) where p > 1 + d/r .
P. Cardaliaguet (Paris-Dauphine) MFG 16 / 63
![Page 26: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/26.jpg)
Preliminary estimates on HJ equations
Upper bounds
Let α ∈ C1] ([0,T ]× Rd ) and u be a viscosity subsolution to
−∂tu + H(x ,Du) ≤ α in (0,T )× Rd
Lemma (Cannarsa, C.)There is a universal constant C such that
u(t1, x) ≤ u(t2, x) + C(t2 − t1)p−1−d/r
p+d/r ‖(α)+‖p
for any 0 ≤ t1 < t2 ≤ T (where p − 1− d/r > 0 by assumption).
Remark : the result actually holds if u ∈ BV], Du ∈ Lr and α ∈ Lp] , where the
inequality holds in the sense of distribution.
P. Cardaliaguet (Paris-Dauphine) MFG 17 / 63
![Page 27: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/27.jpg)
Preliminary estimates on HJ equations
Proof : We assume that u and α are of class C1.Fix β ∈ (1/r , 1
d(q−1) ). For σ ∈ Rd with |σ| ≤ 1, we define the arc
xσ(s) =
x + σ(s − t1)β if s ∈ [t1, t1+t2
2 ]x + σ(t2 − s)β if s ∈ [ t1+t2
2 , t2]
Let L be the convex conjugate of p → H(x ,−p), i.e., L(x , ξ) = H∗(x ,−ξ).Then
ddt
[u(s, xσ(s))−
∫ t2
sL(xσ(τ), x ′σ(τ))dτ
]= ∂tu(s, xσ(s)) + 〈Du(s, xσ(s)), x ′σ(s)〉+ L(xσ(s), x ′σ(s))≥ ∂tu(s, xσ(s))− H(xσ(s),Du(s, xσ(s))) ≥ −α(s, xσ(s))
P. Cardaliaguet (Paris-Dauphine) MFG 18 / 63
![Page 28: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/28.jpg)
Preliminary estimates on HJ equations
Soddt
[u(s, xσ(s))−
∫ t2
sL(xσ(τ), x ′σ(τ))dτ
]≥ −α(s, xσ(s))
We integrate in time on [t1, t2] :
u(t2, x)− u(t1, x) +
∫ t2
sL(xσ(τ), x ′σ(τ))dτ ≥ −
∫ t2
sα(τ, xσ(τ))dτ
Then we integrate on σ ∈ B1 :
u(t1, x) ≤ u(t2, x) +1|B1|
∫B1
∫ t2
t1[L(xσ(s), x ′σ(s)) + α(s, xσ(s))] dsdσ .
By the growth assumption on L (=coercivity assumption on H), we have
1|B1|
∫B1
∫ t2
t1L(xσ(s), x ′σ(s)) dsdσ ≤ C
[∫B1
∫ t2
t1|x ′σ(s))|r
′dsdσ + (t2 − t1)
]≤ C(t2 − t1)1−r ′(1−β)
where 1− r ′(1− β) > 0 since β > 1/r .P. Cardaliaguet (Paris-Dauphine) MFG 19 / 63
![Page 29: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/29.jpg)
Preliminary estimates on HJ equations
Using Hölder’s inequality, we get, on another hand,∫B1
∫ t2
t1α(s, xσ(s))dsdσ ≤ C(t2 − t1)(1−dβ(q−1))/q‖α‖p
where 1− dβ(q − 1) > 0 since β < 1d(q−1) .
For a suitable choice of β ∈ (1r,
1d(q − 1)
), we obtain finally that
u(t1, x) ≤ u(t2, x) + C(t2 − t1)p−1−d/r
d+d/r ‖α‖p .
P. Cardaliaguet (Paris-Dauphine) MFG 20 / 63
![Page 30: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/30.jpg)
Preliminary estimates on HJ equations
Regularity
Theorem (C.-Silvestre, 2012)Let u be a bounded viscosity solution of
−∂tu + H(x ,Du) = α in (0,T )× Rd
u(T , x) = uT (x) in Rd
where α ≥ 0, α ∈ Lp with p > 1 + d/r .
Then, for any δ > 0, u is Hölder continuous in [0,T − δ]× Rd :
|u(t , x)− u(s, y)| ≤ C|(t , x)− (s, y)|γ
whereγ = γ(‖u‖∞, ‖α‖p,d , r), C = C(‖u‖∞, ‖α‖m,d , r , δ).
P. Cardaliaguet (Paris-Dauphine) MFG 21 / 63
![Page 31: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/31.jpg)
Preliminary estimates on HJ equations
Related results
Capuzzo Dolcetta-Leoni-Porretta (2010), Barles (2010) : stationaryequations, bounded RHS,
C. (2009), Cannarsa-C. (2010), C. Rainer (2011) : evolution equations,bounded RHS,
Dall’Aglio-Porretta (preprint) : stationary setting, unbounded RHS,
C.-Silvestre (2012) : evolution equations, unbounded RHS.
P. Cardaliaguet (Paris-Dauphine) MFG 22 / 63
![Page 32: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/32.jpg)
Preliminary estimates on HJ equations
Idea of proofs :
For the stationary setting, by comparison with suitable test functions.(Capuzzo Dolcetta-Leoni-Porretta, Barles, Dall’Aglio-Porretta)
For evolution equations, two techniques
By representation formula and reverse Hölder inequalities,(C., Cannarsa-C., C. Rainer)Comparison + improvement of oscillations techniques(C.-Silvestre)
P. Cardaliaguet (Paris-Dauphine) MFG 23 / 63
![Page 33: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/33.jpg)
Preliminary estimates on HJ equations
Summary
Let (u, α) solve −∂tu + H(x ,Du) = α in (0,T )× Rd
u(T , ·) = uT in Rd
with α = α(t , x) ≥ 0. Then
(Upper bound)
u(t1, ·) ≤ u(t2, ·) + C(t2 − t1)p−1−d/r
d+d/r ‖α‖p a.e.
for any 0 ≤ t1 < t2 ≤ T (where r − d(q − 1) > 0 by assumption (H3)).
(Lower bound) u(t , x) ≥ uT (x)− C(T − t).
(Regularity) u is locally Hölder continuous in [0,T )× Rd with a modulusdepending only on ‖α‖p and ‖uT‖∞.
P. Cardaliaguet (Paris-Dauphine) MFG 24 / 63
![Page 34: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/34.jpg)
Optimal control of the HJ equation
Outline
1 Preliminary estimates on HJ equations
2 Optimal control of the HJ equation
3 Optimal control of the Kolmogorov equation
4 Existence/uniqueness of solutions for the (MFG) system
5 Conclusion
P. Cardaliaguet (Paris-Dauphine) MFG 25 / 63
![Page 35: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/35.jpg)
Optimal control of the HJ equation
Back to the MFG system
(MFG)
(i) −∂tu + H(x ,Du) = f (x ,m(x , t))(ii) ∂tm − div(mDpH(x ,Du)) = 0(iii) m(0) = m0, u(x ,T ) = uT (x)
under the assumptions that
f : Rd × [0,+∞)→ [0,+∞) is a local coupling term.
all the data are Zd−periodic in space : denoted by ].
P. Cardaliaguet (Paris-Dauphine) MFG 26 / 63
![Page 36: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/36.jpg)
Optimal control of the HJ equation
We study the optimal control of the HJ equation :
(HJ− Pb) inf
∫ T
0
∫Q
F ∗(x , α(t , x)) dxdt −∫
Qu(0, x)dm0(x)
where u is the solution to the HJ equation−∂tu + H(x ,Du) = α in (0,T )× Rd
u(T , ·) = uT in Rd
and F ∗(x ,a) = supm∈R
(am − F (x ,m)) where
F (x ,m) =
∫ m
0f (x ,m′)dm′ if m ≥ 0
+∞ otherwise
Note that F ∗(x ,a) = 0 for a ≤ 0.
P. Cardaliaguet (Paris-Dauphine) MFG 27 / 63
![Page 37: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/37.jpg)
Optimal control of the HJ equation
We study the optimal control of the HJ equation :
(HJ− Pb) inf
∫ T
0
∫Q
F ∗(x , α(t , x)) dxdt −∫
Qu(0, x)dm0(x)
where u is the solution to the HJ equation−∂tu + H(x ,Du) = α in (0,T )× Rd
u(T , ·) = uT in Rd
and F ∗(x ,a) = supm∈R
(am − F (x ,m)) where
F (x ,m) =
∫ m
0f (x ,m′)dm′ if m ≥ 0
+∞ otherwise
Note that F ∗(x ,a) = 0 for a ≤ 0.
P. Cardaliaguet (Paris-Dauphine) MFG 27 / 63
![Page 38: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/38.jpg)
Optimal control of the HJ equation
Heuristics :
α must be small in order to minimize∫ T
0
∫Q
F ∗(x , α(t , x)) dxdt .
However, if α is small, then u is also small (by comparison).
This contradicts the smallness of the term −∫
Q u(0, x)dm0(x) (becausem0 ≥ 0).
P. Cardaliaguet (Paris-Dauphine) MFG 28 / 63
![Page 39: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/39.jpg)
Optimal control of the HJ equation
In order to prove existence of optimal solutions
We look for estimates on minimizing sequences,
and derive from them a relaxed problem.
P. Cardaliaguet (Paris-Dauphine) MFG 29 / 63
![Page 40: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/40.jpg)
Optimal control of the HJ equation
Assumptions
(H1) f : Rd × [0,+∞)→ R is smooth and increasing with respect to thesecond variable m with f (x ,0) = 0,
(H2) There exists q > 1 and a constant C such that
−C +1C|m|q−1 ≤ f (x ,m) ≤ C(1 + |m|q−1) ∀m .
(H3) The Hamiltonian H : Rd × Rd → R is of class C1 and has a superlineargrowth in the gradient variable : there is r > d(q − 1) such that
1C|ξ|r − C ≤ H(x , ξ) ≤ C(|ξ|r + 1) ∀(x , ξ) ∈ Rd × Rd .
P. Cardaliaguet (Paris-Dauphine) MFG 30 / 63
![Page 41: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/41.jpg)
Optimal control of the HJ equation
(H4) We also assume that there is θ ∈ [0, rd+1 ) such that
|H(x , ξ)− H(y , ξ)| ≤ C|x − y | (|ξ| ∨ 1)θ ∀x , y , ξ ∈ Rd
(H5) uT : Rd → R is a smooth, periodic map, while m0 : Rd → R is a smooth,periodic map, with m0 ≥ 0 and
∫Q m0dx = 1.
Remark : Recall that F ∗(x ,a) = supm∈R
F (x ,m) where
F (x ,m) =
∫ m
0f (x ,m′)dm′ if m ≥ 0
+∞ otherwise
With our assumptions, F ∗ is C1, nondecreasing, with F ∗(x ,a) = 0 for a ≤ 0.
P. Cardaliaguet (Paris-Dauphine) MFG 31 / 63
![Page 42: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/42.jpg)
Optimal control of the HJ equation
(H4) We also assume that there is θ ∈ [0, rd+1 ) such that
|H(x , ξ)− H(y , ξ)| ≤ C|x − y | (|ξ| ∨ 1)θ ∀x , y , ξ ∈ Rd
(H5) uT : Rd → R is a smooth, periodic map, while m0 : Rd → R is a smooth,periodic map, with m0 ≥ 0 and
∫Q m0dx = 1.
Remark : Recall that F ∗(x ,a) = supm∈R
F (x ,m) where
F (x ,m) =
∫ m
0f (x ,m′)dm′ if m ≥ 0
+∞ otherwise
With our assumptions, F ∗ is C1, nondecreasing, with F ∗(x ,a) = 0 for a ≤ 0.
P. Cardaliaguet (Paris-Dauphine) MFG 31 / 63
![Page 43: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/43.jpg)
Optimal control of the HJ equation
Estimates on the minimizing sequence
Let (un, αn) be a minimizing sequence for the optimal control of HJ equation :∫ T
0
∫Q
F ∗(x , αn) dxdt −∫
Qun(0, x)dm0(x)
→ inf
where (un, αn) is the solution to the HJ equation−∂tun + H(x ,Dun) = αn in (0,T )× Rd
un(T , ·) = uT in Rd
P. Cardaliaguet (Paris-Dauphine) MFG 32 / 63
![Page 44: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/44.jpg)
Optimal control of the HJ equation
Let αn = αn ∨ 0 and un solve−∂t un + H(x ,Dun) = αn in (0,T )× Rd
un(T , ·) = uT in Rd
Then :
by comparison, un ≤ un,
F ∗(x , αn(t , x)) ≥ F ∗(x , αn(t , x)) because F ∗(x ,0) ≤ F ∗(x ,a) for all a.
Therefore∫ T
0
∫Q
F ∗(x , αn)−∫
Qun(0)m0 ≤
∫ T
0
∫Q
F ∗(x , αn)−∫
Qun(0)m0
and (un, αn) is still a minimizing sequence.
P. Cardaliaguet (Paris-Dauphine) MFG 33 / 63
![Page 45: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/45.jpg)
Optimal control of the HJ equation
So we now assume that (un, αn) be a minimizing sequence for the optimalcontrol of HJ equation with αn ≥ 0.
P. Cardaliaguet (Paris-Dauphine) MFG 34 / 63
![Page 46: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/46.jpg)
Optimal control of the HJ equation
We use the results of the first part :
(Upper bound)
un(t1, ·) ≤ un(t2, ·) + C(t2 − t1)p−1−d/r
d+d/r ‖αn‖p
for any 0 ≤ t1 < t2 ≤ T (where r − d(q − 1) > 0 by assumption (H3)).
(Lower bound) un(t , x) ≥ uT (x)− C(T − t).
(Regularity) un is locally Hölder continuous in [0,T )× Rd with a modulusdepending only on ‖αn‖p and ‖uT‖∞.
P. Cardaliaguet (Paris-Dauphine) MFG 35 / 63
![Page 47: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/47.jpg)
Optimal control of the HJ equation
In particularun(0, ·) ≤ uT + C‖αn‖p
By assumption on F , we have
1C|a|p − C ≤ F ∗(x ,a) ≤ C(1 + |a|p) ∀a ≥ 0 ,
(where 1/p + 1/q = 1). Therefore
C ≥∫ T
0
∫Q
F ∗(x , αn)−∫
Qun(0)m0 ≥
∫ T
0
∫Q|αn|p − C‖αn‖p − C
This shows that (αn) is bounded in Lp.
P. Cardaliaguet (Paris-Dauphine) MFG 36 / 63
![Page 48: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/48.jpg)
Optimal control of the HJ equation
Estimates on the minimizing sequence
PropositionIf (un, αn) is a minimizing sequence, then
(αn) is bounded in Lp.
(uniform bounds on the un)
uT (x)− C(T − t) ≤ un(t , x) ≤ uT (x) + C(T − t)p−1−d/r
d+d/r
(Regularity) the un are uniformly locally Hölder continuous in [0,T )× Rd .
(Integral bounds) Dun is bounded in Lr] and (∂tun) is bounded in L1
] .
Remark : In particular, (Dun) is bounded in BV].
P. Cardaliaguet (Paris-Dauphine) MFG 37 / 63
![Page 49: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/49.jpg)
Optimal control of the HJ equation
Explanation of the integral boundFrom the equation satisfied by the (un) we have∫ T
0
∫Q
H(x ,Dun) =
∫ T
0
∫Q
(αn + ∂tun)
≤ ‖αn‖p +
∫Q
(un(T )− un(0)) ≤ C
As H is coercive, (Dun) is bounded in Lr].
Now∂tun = H(x ,Dun)− αn
where (H(x ,Dun)) is bounded in L1] while (αn) is bounded in Lp
] . So (∂tun) isbounded in L1
] .
P. Cardaliaguet (Paris-Dauphine) MFG 38 / 63
![Page 50: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/50.jpg)
Optimal control of the HJ equation
The relaxed problem
Let K be the set (u, α) ∈ BV]((0,T )× Rd )× Lp] ((0,T )× Rd ) such that
Du ∈ Lr]((0,T )× Rd )
α ≥ 0 a.e.
u(T , x) = uT (x) and
−∂tu + H(x ,Du) ≤ α in (0,T )× Rd .
holds in the sense of distribution.
Note that K is a convex set.
The relaxed problem is
(HJ− Rpb) inf(u,α)∈K
∫ T
0
∫Q
F ∗(x , α(t , x)) dxdt −∫
Qu(0, x)dm0(x)
P. Cardaliaguet (Paris-Dauphine) MFG 39 / 63
![Page 51: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/51.jpg)
Optimal control of the HJ equation
The relaxed problem
Let K be the set (u, α) ∈ BV]((0,T )× Rd )× Lp] ((0,T )× Rd ) such that
Du ∈ Lr]((0,T )× Rd )
α ≥ 0 a.e.
u(T , x) = uT (x) and
−∂tu + H(x ,Du) ≤ α in (0,T )× Rd .
holds in the sense of distribution.
Note that K is a convex set.
The relaxed problem is
(HJ− Rpb) inf(u,α)∈K
∫ T
0
∫Q
F ∗(x , α(t , x)) dxdt −∫
Qu(0, x)dm0(x)
P. Cardaliaguet (Paris-Dauphine) MFG 39 / 63
![Page 52: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/52.jpg)
Optimal control of the HJ equation
Theorem
The relaxed problem has (HJ-Rpb) at least one minimum (u, α), whereu is continuous and satisfies in the viscosity sense
−∂tu + H(x ,Du) ≥ 0 in (0,T )× Rd
The value of the relaxed problem (HJ-Rpb) is equal to the value of theoptimal control of the HJ equation (HJ-Pb) .
P. Cardaliaguet (Paris-Dauphine) MFG 40 / 63
![Page 53: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/53.jpg)
Optimal control of the Kolmogorov equation
Outline
1 Preliminary estimates on HJ equations
2 Optimal control of the HJ equation
3 Optimal control of the Kolmogorov equation
4 Existence/uniqueness of solutions for the (MFG) system
5 Conclusion
P. Cardaliaguet (Paris-Dauphine) MFG 41 / 63
![Page 54: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/54.jpg)
Optimal control of the Kolmogorov equation
The optimal control problem of HJ equation (HJ-Pb) can be rewritten as
inf
∫ T
0
∫Q
F ∗(x ,−∂tu + H(x ,Du)) dxdt −∫
Qu(0, x)dm0(x)
with constraint u(T , ·) = uT .
This is a convex problem.
−→ Suggests a duality approach.
P. Cardaliaguet (Paris-Dauphine) MFG 42 / 63
![Page 55: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/55.jpg)
Optimal control of the Kolmogorov equation
The optimal control problem of HJ equation (HJ-Pb) can be rewritten as
inf
∫ T
0
∫Q
F ∗(x ,−∂tu + H(x ,Du)) dxdt −∫
Qu(0, x)dm0(x)
with constraint u(T , ·) = uT .
This is a convex problem.
−→ Suggests a duality approach.
P. Cardaliaguet (Paris-Dauphine) MFG 42 / 63
![Page 56: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/56.jpg)
Optimal control of the Kolmogorov equation
The optimal control problem of HJ equation (HJ-Pb) can be rewritten as
inf
∫ T
0
∫Q
F ∗(x ,−∂tu + H(x ,Du)) dxdt −∫
Qu(0, x)dm0(x)
with constraint u(T , ·) = uT .
This is a convex problem.
−→ Suggests a duality approach.
P. Cardaliaguet (Paris-Dauphine) MFG 42 / 63
![Page 57: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/57.jpg)
Optimal control of the Kolmogorov equation
The Fenchel-Rockafellar duality Theorem
Let
E and F be two normed spaces,
Λ ∈ Lc(E ,F ) and
F : E → R ∪ +∞ and G : F → R ∪ +∞ be two lsc proper convexmaps.
TheoremUnder the qualification condition : there is x0 such that F(x0) < +∞ and Gcontinuous at Λ(x0), we have
infx∈EF(x) + G(Λ(x)) = max
y∗∈F ′−F∗(Λ∗(y∗))− G∗(−y∗)
as soon as the LHS is finite.
P. Cardaliaguet (Paris-Dauphine) MFG 43 / 63
![Page 58: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/58.jpg)
Optimal control of the Kolmogorov equation
Proof of one inequality : for any x ∈ E and y∗ ∈ F ′ we have
F(x) + F∗(Λ∗(y∗)) ≥ 〈Λ∗(y∗), x〉
andG(Λ(x)) + G∗(−y∗) ≥ 〈−y∗,Λ(x)〉 = −〈Λ∗(y∗), x〉
Adding both inequalities gives
F(x) + F∗(Λ∗(y∗)) + G(Λ(x)) + G∗(−y∗) ≥ 0 .
Rearranging
F(x) + G(Λ(x)) ≥ −F∗(Λ∗(y∗)) + G∗(−y∗)
Therefore
infx∈EF(x) + G(Λ(x)) ≥ sup
y∗∈F ′−F∗(Λ∗(y∗))− G∗(−y∗)
P. Cardaliaguet (Paris-Dauphine) MFG 44 / 63
![Page 59: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/59.jpg)
Optimal control of the Kolmogorov equation
Proof of one inequality : for any x ∈ E and y∗ ∈ F ′ we have
F(x) + F∗(Λ∗(y∗)) ≥ 〈Λ∗(y∗), x〉
andG(Λ(x)) + G∗(−y∗) ≥ 〈−y∗,Λ(x)〉 = −〈Λ∗(y∗), x〉
Adding both inequalities gives
F(x) + F∗(Λ∗(y∗)) + G(Λ(x)) + G∗(−y∗) ≥ 0 .
Rearranging
F(x) + G(Λ(x)) ≥ −F∗(Λ∗(y∗)) + G∗(−y∗)
Therefore
infx∈EF(x) + G(Λ(x)) ≥ sup
y∗∈F ′−F∗(Λ∗(y∗))− G∗(−y∗)
P. Cardaliaguet (Paris-Dauphine) MFG 44 / 63
![Page 60: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/60.jpg)
Optimal control of the Kolmogorov equation
Proof of one inequality : for any x ∈ E and y∗ ∈ F ′ we have
F(x) + F∗(Λ∗(y∗)) ≥ 〈Λ∗(y∗), x〉
andG(Λ(x)) + G∗(−y∗) ≥ 〈−y∗,Λ(x)〉 = −〈Λ∗(y∗), x〉
Adding both inequalities gives
F(x) + F∗(Λ∗(y∗)) + G(Λ(x)) + G∗(−y∗) ≥ 0 .
Rearranging
F(x) + G(Λ(x)) ≥ −F∗(Λ∗(y∗)) + G∗(−y∗)
Therefore
infx∈EF(x) + G(Λ(x)) ≥ sup
y∗∈F ′−F∗(Λ∗(y∗))− G∗(−y∗)
P. Cardaliaguet (Paris-Dauphine) MFG 44 / 63
![Page 61: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/61.jpg)
Optimal control of the Kolmogorov equation
Proof of one inequality : for any x ∈ E and y∗ ∈ F ′ we have
F(x) + F∗(Λ∗(y∗)) ≥ 〈Λ∗(y∗), x〉
andG(Λ(x)) + G∗(−y∗) ≥ 〈−y∗,Λ(x)〉 = −〈Λ∗(y∗), x〉
Adding both inequalities gives
F(x) + F∗(Λ∗(y∗)) + G(Λ(x)) + G∗(−y∗) ≥ 0 .
Rearranging
F(x) + G(Λ(x)) ≥ −F∗(Λ∗(y∗)) + G∗(−y∗)
Therefore
infx∈EF(x) + G(Λ(x)) ≥ sup
y∗∈F ′−F∗(Λ∗(y∗))− G∗(−y∗)
P. Cardaliaguet (Paris-Dauphine) MFG 44 / 63
![Page 62: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/62.jpg)
Optimal control of the Kolmogorov equation
Remark : If x and y∗ are respectively optimal in
infx∈EF(x) + G(Λ(x)) and sup
y∗∈F ′−F∗(Λ∗(y∗))− G∗(−y∗)
thenF(x) + F∗(Λ∗(y∗)) = 〈Λ∗(y∗), x〉
andG(Λ(x)) + G∗(−y∗) = 〈−y∗,Λ(x)〉
i.e.,Λ∗(y∗) ∈ ∂F(x)
andΛ(x) ∈ ∂G∗(−y∗)
P. Cardaliaguet (Paris-Dauphine) MFG 45 / 63
![Page 63: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/63.jpg)
Optimal control of the Kolmogorov equation
Remark : If x and y∗ are respectively optimal in
infx∈EF(x) + G(Λ(x)) and sup
y∗∈F ′−F∗(Λ∗(y∗))− G∗(−y∗)
thenF(x) + F∗(Λ∗(y∗)) = 〈Λ∗(y∗), x〉
andG(Λ(x)) + G∗(−y∗) = 〈−y∗,Λ(x)〉
i.e.,Λ∗(y∗) ∈ ∂F(x)
andΛ(x) ∈ ∂G∗(−y∗)
P. Cardaliaguet (Paris-Dauphine) MFG 45 / 63
![Page 64: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/64.jpg)
Optimal control of the Kolmogorov equation
The optimal control problem of HJ equation (HJ-Pb)
inf
∫ T
0
∫Q
F ∗(x ,−∂tu + H(x ,Du)) dxdt −∫
Qu(0, x)dm0(x)
(with constraint u(T , ·) = uT ) can be rewritten as
infu∈EF(u) + G(Λ(u))
where
E = C1] ([0,T ]× Rd ) ,
F = C0] ([0,T ]× Rd ,R)× C0
] ([0,T ]× Rd ,Rd ) ,
F(u) = −∫
Qm0(x)u(0, x)dx if u(T , ·) = uT (+∞ otherwise).
G(a,b) =
∫ T
0
∫Q
F ∗(x ,−a(t , x) + H(x ,b(t , x))) dxdt .
Λ(u) = (∂tu,Du).
P. Cardaliaguet (Paris-Dauphine) MFG 46 / 63
![Page 65: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/65.jpg)
Optimal control of the Kolmogorov equation
Then F is convex and lower semi-continuous on E while G is convex andcontinuous on F . Moreover Λ is bounded and linear.
The qualification condition is satisfied by u(t , x) = uT (x).
By Fenchel-Rockafellar duality theorem we have
infu∈EF(u) + G(Λ(u)) = max
(m,w)∈F ′−F∗(Λ∗(m,w))− G∗(−(m,w))
where F ′ is the set of Radon measures (m,w) ∈ M]([0,T ]× Rd ,R× Rd ) andF∗ and G∗ are the convex conjugates of F and G.
P. Cardaliaguet (Paris-Dauphine) MFG 47 / 63
![Page 66: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/66.jpg)
Optimal control of the Kolmogorov equation
Then F is convex and lower semi-continuous on E while G is convex andcontinuous on F . Moreover Λ is bounded and linear.
The qualification condition is satisfied by u(t , x) = uT (x).
By Fenchel-Rockafellar duality theorem we have
infu∈EF(u) + G(Λ(u)) = max
(m,w)∈F ′−F∗(Λ∗(m,w))− G∗(−(m,w))
where F ′ is the set of Radon measures (m,w) ∈ M]([0,T ]× Rd ,R× Rd ) andF∗ and G∗ are the convex conjugates of F and G.
P. Cardaliaguet (Paris-Dauphine) MFG 47 / 63
![Page 67: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/67.jpg)
Optimal control of the Kolmogorov equation
Then F is convex and lower semi-continuous on E while G is convex andcontinuous on F . Moreover Λ is bounded and linear.
The qualification condition is satisfied by u(t , x) = uT (x).
By Fenchel-Rockafellar duality theorem we have
infu∈EF(u) + G(Λ(u)) = max
(m,w)∈F ′−F∗(Λ∗(m,w))− G∗(−(m,w))
where F ′ is the set of Radon measures (m,w) ∈ M]([0,T ]× Rd ,R× Rd ) andF∗ and G∗ are the convex conjugates of F and G.
P. Cardaliaguet (Paris-Dauphine) MFG 47 / 63
![Page 68: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/68.jpg)
Optimal control of the Kolmogorov equation
Recall that
F(u) = −∫
Qm0(x)u(0, x)dx if u(T , ·) = uT (+∞ otherwise)
and
G(a,b) =
∫ T
0
∫Q
F ∗(x ,−a(t , x) + H(x ,b(t , x))) dxdt .
Lemma
F∗(Λ∗(m,w)) =
∫
QuT (x)dm(T , x) if ∂tm + div(w) = 0, m(0) = m0
+∞ otherwise
and
G∗(m,w) =
∫ T
0
∫Q−F (x ,m)−mH∗(x ,−w
m) dtdx if (m,w) ∈ L1
]
+∞ otherwise
P. Cardaliaguet (Paris-Dauphine) MFG 48 / 63
![Page 69: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/69.jpg)
Optimal control of the Kolmogorov equation
Consequence of the Lemma :
max(m,w)∈F ′
−F∗(Λ∗(m,w))− G(m,w)
= max
∫ T
0
∫Q−F (x ,m)−mH∗(x ,−w
m) dtdx −
∫Q
uT (x)m(T , x) dx
where the maximum is taken over the L1] maps (m,w) such that m ≥ 0 a.e.
and∂tm + div(w) = 0, m(0) = m0
P. Cardaliaguet (Paris-Dauphine) MFG 49 / 63
![Page 70: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/70.jpg)
Optimal control of the Kolmogorov equation
Idea of proof :
F∗(Λ∗(m,w)) = supu(T )=uT
〈Λ∗(m,w),u〉 − F(u)
= supu(T )=uT
〈(m,w),Λ(u)〉+
∫Q
m0(x)u(0, x)dx
= supu(T )=uT
∫ T
0
∫Q
(m∂tu + 〈w ,Du〉) +
∫Q
m0(x)u(0, x)dx
“ = supu(T )=uT
∫ T
0
∫Q−u(−∂tm + div(w))
+
∫Q
m(T )uT +
∫Q
(m0 −m(0))u(0)dx”
=
∫
QuT (x)dm(T , x) if ∂tm + div(w) = 0, m(0) = m0
+∞ otherwise
P. Cardaliaguet (Paris-Dauphine) MFG 50 / 63
![Page 71: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/71.jpg)
Optimal control of the Kolmogorov equation
The dual of the optimal control of HJ eqs
TheoremThe dual of the optimal control of HJ (HJ-Pb) equation is given by
(K− Pb) inf
∫ T
0
∫Q
mH∗(x ,−wm
) + F (x ,m) dxdt +
∫Q
uT (x)m(T , x)dx
where the infimum is taken over the pairs(m,w) ∈ L1
]((0,T )× Rd )× L1]((0,T )× Rd ,Rd ) such that
∂tm + div(w) = 0, m(0) = m0
in the sense of distributions. Moreover this dual problem has a uniqueminimum.
P. Cardaliaguet (Paris-Dauphine) MFG 51 / 63
![Page 72: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/72.jpg)
Optimal control of the Kolmogorov equation
(K-Pb) as an optimal control problem for Kolmogorov equation :
Set v = w/m. Then (K-Pb) becomes
inf
∫ T
0
∫Q
mH∗(x ,−v) + F (x ,m) dxdt +
∫Q
uT (x)m(T , x)dx
where the infimum is taken over the pairs (m, v) such that
∂tm + div(mv) = 0, m(0) = m0
in the sense of distributions.
P. Cardaliaguet (Paris-Dauphine) MFG 52 / 63
![Page 73: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/73.jpg)
Optimal control of the Kolmogorov equation
(K-Pb) as an optimal control problem for Kolmogorov equation :
Set v = w/m. Then (K-Pb) becomes
inf
∫ T
0
∫Q
mH∗(x ,−v) + F (x ,m) dxdt +
∫Q
uT (x)m(T , x)dx
where the infimum is taken over the pairs (m, v) such that
∂tm + div(mv) = 0, m(0) = m0
in the sense of distributions.
P. Cardaliaguet (Paris-Dauphine) MFG 52 / 63
![Page 74: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/74.jpg)
Existence/uniqueness of solutions for the (MFG) system
Outline
1 Preliminary estimates on HJ equations
2 Optimal control of the HJ equation
3 Optimal control of the Kolmogorov equation
4 Existence/uniqueness of solutions for the (MFG) system
5 Conclusion
P. Cardaliaguet (Paris-Dauphine) MFG 53 / 63
![Page 75: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/75.jpg)
Existence/uniqueness of solutions for the (MFG) system
Back to the (MFG) system
We now study the weak solutions of the (MFG) system
(MFG)
(i) −∂tu + H(x ,Du) = f (x ,m(x , t))(ii) ∂tm − div(mDpH(x ,Du)) = 0(iii) m(0) = m0, u(x ,T ) = uT (x)
and explain the relation with the two optimal control problems
for the HJ equation (problem (HJ-Pb))
for the Kolmogorov equation (problem (K-Pb))
P. Cardaliaguet (Paris-Dauphine) MFG 54 / 63
![Page 76: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/76.jpg)
Existence/uniqueness of solutions for the (MFG) system
DefinitionA pair (m,u) ∈ L1((0,T )× Rd )× BV ((0,T )× Rd ) is a weak solution of(MFG) if
(i) u is continuous in [0,T ]× Rd with Du ∈ Lr ((0,T )× Rd ,m) andmDpH(x ,Du) ∈ L1
]
(ii) Inequality −∂tu + H(x ,Du) ≤ f (x ,m) holds in the sense ofdistribution, with u(T , x) = uT (x) in the sense of trace,
(iii) Equality ∂tm − div(mDpH(x ,Du)) = 0 holds in the sense ofdistribution in (0,T )× Rd and m(0) = m0,
(iv) Equality∫ T
0
∫Q
m (∂tuac − 〈Du,DpH(x ,Du)〉) =
∫Q
m(T )uT −m0u(0)
holds.
(where ∂tuac is the a.c. part of the measure ∂tu).
P. Cardaliaguet (Paris-Dauphine) MFG 55 / 63
![Page 77: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/77.jpg)
Existence/uniqueness of solutions for the (MFG) system
DefinitionA pair (m,u) ∈ L1((0,T )× Rd )× BV ((0,T )× Rd ) is a weak solution of(MFG) if
(i) u is continuous in [0,T ]× Rd with Du ∈ Lr ((0,T )× Rd ,m) andmDpH(x ,Du) ∈ L1
]
(ii) Inequality −∂tu + H(x ,Du) ≤ f (x ,m) holds in the sense ofdistribution, with u(T , x) = uT (x) in the sense of trace,
(iii) Equality ∂tm − div(mDpH(x ,Du)) = 0 holds in the sense ofdistribution in (0,T )× Rd and m(0) = m0,
(iv) Equality∫ T
0
∫Q
m (∂tuac − 〈Du,DpH(x ,Du)〉) =
∫Q
m(T )uT −m0u(0)
holds.
(where ∂tuac is the a.c. part of the measure ∂tu).
P. Cardaliaguet (Paris-Dauphine) MFG 55 / 63
![Page 78: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/78.jpg)
Existence/uniqueness of solutions for the (MFG) system
DefinitionA pair (m,u) ∈ L1((0,T )× Rd )× BV ((0,T )× Rd ) is a weak solution of(MFG) if
(i) u is continuous in [0,T ]× Rd with Du ∈ Lr ((0,T )× Rd ,m) andmDpH(x ,Du) ∈ L1
]
(ii) Inequality −∂tu + H(x ,Du) ≤ f (x ,m) holds in the sense ofdistribution, with u(T , x) = uT (x) in the sense of trace,
(iii) Equality ∂tm − div(mDpH(x ,Du)) = 0 holds in the sense ofdistribution in (0,T )× Rd and m(0) = m0,
(iv) Equality∫ T
0
∫Q
m (∂tuac − 〈Du,DpH(x ,Du)〉) =
∫Q
m(T )uT −m0u(0)
holds.
(where ∂tuac is the a.c. part of the measure ∂tu).
P. Cardaliaguet (Paris-Dauphine) MFG 55 / 63
![Page 79: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/79.jpg)
Existence/uniqueness of solutions for the (MFG) system
DefinitionA pair (m,u) ∈ L1((0,T )× Rd )× BV ((0,T )× Rd ) is a weak solution of(MFG) if
(i) u is continuous in [0,T ]× Rd with Du ∈ Lr ((0,T )× Rd ,m) andmDpH(x ,Du) ∈ L1
]
(ii) Inequality −∂tu + H(x ,Du) ≤ f (x ,m) holds in the sense ofdistribution, with u(T , x) = uT (x) in the sense of trace,
(iii) Equality ∂tm − div(mDpH(x ,Du)) = 0 holds in the sense ofdistribution in (0,T )× Rd and m(0) = m0,
(iv) Equality∫ T
0
∫Q
m (∂tuac − 〈Du,DpH(x ,Du)〉) =
∫Q
m(T )uT −m0u(0)
holds.
(where ∂tuac is the a.c. part of the measure ∂tu).
P. Cardaliaguet (Paris-Dauphine) MFG 55 / 63
![Page 80: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/80.jpg)
Existence/uniqueness of solutions for the (MFG) system
DefinitionA pair (m,u) ∈ L1((0,T )× Rd )× BV ((0,T )× Rd ) is a weak solution of(MFG) if
(i) u is continuous in [0,T ]× Rd with Du ∈ Lr ((0,T )× Rd ,m) andmDpH(x ,Du) ∈ L1
]
(ii) Inequality −∂tu + H(x ,Du) ≤ f (x ,m) holds in the sense ofdistribution, with u(T , x) = uT (x) in the sense of trace,
(iii) Equality ∂tm − div(mDpH(x ,Du)) = 0 holds in the sense ofdistribution in (0,T )× Rd and m(0) = m0,
(iv) Equality∫ T
0
∫Q
m (∂tuac − 〈Du,DpH(x ,Du)〉) =
∫Q
m(T )uT −m0u(0)
holds.
(where ∂tuac is the a.c. part of the measure ∂tu).
P. Cardaliaguet (Paris-Dauphine) MFG 55 / 63
![Page 81: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/81.jpg)
Existence/uniqueness of solutions for the (MFG) system
DefinitionA pair (m,u) ∈ L1((0,T )× Rd )× BV ((0,T )× Rd ) is a weak solution of(MFG) if
(i) u is continuous in [0,T ]× Rd with Du ∈ Lr ((0,T )× Rd ,m) andmDpH(x ,Du) ∈ L1
]
(ii) Inequality −∂tu + H(x ,Du) ≤ f (x ,m) holds in the sense ofdistribution, with u(T , x) = uT (x) in the sense of trace,
(iii) Equality ∂tm − div(mDpH(x ,Du)) = 0 holds in the sense ofdistribution in (0,T )× Rd and m(0) = m0,
(iv) Equality∫ T
0
∫Q
m (∂tuac − 〈Du,DpH(x ,Du)〉) =
∫Q
m(T )uT −m0u(0)
holds.
(where ∂tuac is the a.c. part of the measure ∂tu).
P. Cardaliaguet (Paris-Dauphine) MFG 55 / 63
![Page 82: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/82.jpg)
Existence/uniqueness of solutions for the (MFG) system
Remarks
If (ii) holds with an equality and if u is in W 1,1, then (iii) implies (iv).
Conditions (ii) and (iv) imply that−∂tuac(t , x) + H(x ,Du(t , x)) = f (x ,m(t , x)) holds a.e. in m > 0.
P. Cardaliaguet (Paris-Dauphine) MFG 56 / 63
![Page 83: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/83.jpg)
Existence/uniqueness of solutions for the (MFG) system
Existence for the (MFG) system
TheoremThere is a weak solution (m,u) of (MFG) such that u is locally Höldercontinuous in [0,T )× Rd and which satisfies in the viscosity sense
−∂tu + H(x ,Du) ≥ 0 in (0,T )× Rd .
Idea of proof :
Let (m,w) is a minimizer of (K-Pb) and (u, α) is a minimizer of (HJ-Rpb)such that u is continuous. Then one can show that (m,u) is a solution ofmean field game system (MFG) and w = −mDpH(·,Du) whileα = f (·,m) a.e..
Conversely, any solution of (MFG) such that u is continuous is such thatthe pair (m,−mDpH(·,Du)) is the minimizer of (K-Pb) while (u, f (·,m)) isa minimizer of (HJ-Rpb).
P. Cardaliaguet (Paris-Dauphine) MFG 57 / 63
![Page 84: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/84.jpg)
Existence/uniqueness of solutions for the (MFG) system
Existence for the (MFG) system
TheoremThere is a weak solution (m,u) of (MFG) such that u is locally Höldercontinuous in [0,T )× Rd and which satisfies in the viscosity sense
−∂tu + H(x ,Du) ≥ 0 in (0,T )× Rd .
Idea of proof :
Let (m,w) is a minimizer of (K-Pb) and (u, α) is a minimizer of (HJ-Rpb)such that u is continuous. Then one can show that (m,u) is a solution ofmean field game system (MFG) and w = −mDpH(·,Du) whileα = f (·,m) a.e..
Conversely, any solution of (MFG) such that u is continuous is such thatthe pair (m,−mDpH(·,Du)) is the minimizer of (K-Pb) while (u, f (·,m)) isa minimizer of (HJ-Rpb).
P. Cardaliaguet (Paris-Dauphine) MFG 57 / 63
![Page 85: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/85.jpg)
Existence/uniqueness of solutions for the (MFG) system
Existence for the (MFG) system
TheoremThere is a weak solution (m,u) of (MFG) such that u is locally Höldercontinuous in [0,T )× Rd and which satisfies in the viscosity sense
−∂tu + H(x ,Du) ≥ 0 in (0,T )× Rd .
Idea of proof :
Let (m,w) is a minimizer of (K-Pb) and (u, α) is a minimizer of (HJ-Rpb)such that u is continuous. Then one can show that (m,u) is a solution ofmean field game system (MFG) and w = −mDpH(·,Du) whileα = f (·,m) a.e..
Conversely, any solution of (MFG) such that u is continuous is such thatthe pair (m,−mDpH(·,Du)) is the minimizer of (K-Pb) while (u, f (·,m)) isa minimizer of (HJ-Rpb).
P. Cardaliaguet (Paris-Dauphine) MFG 57 / 63
![Page 86: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/86.jpg)
Existence/uniqueness of solutions for the (MFG) system
Uniqueness for the (MFG) system
Theorem
Let (m,u) and (m′,u′) be two weak solutions of (MFG). Then m = m′ andu = u′ in m > 0.
Moreover, if u satisfies the additional condition
(∗) − ∂tu + H(x ,Du) ≥ 0 in (0,T )× Rd ,
in the viscosity sense, then u ≥ u′.
Remark : In particular, if we add condition (*) to the definition of weaksolution of (MFG), then the weak solution exists and is unique.
P. Cardaliaguet (Paris-Dauphine) MFG 58 / 63
![Page 87: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/87.jpg)
Existence/uniqueness of solutions for the (MFG) system
Uniqueness for the (MFG) system
Theorem
Let (m,u) and (m′,u′) be two weak solutions of (MFG). Then m = m′ andu = u′ in m > 0.
Moreover, if u satisfies the additional condition
(∗) − ∂tu + H(x ,Du) ≥ 0 in (0,T )× Rd ,
in the viscosity sense, then u ≥ u′.
Remark : In particular, if we add condition (*) to the definition of weaksolution of (MFG), then the weak solution exists and is unique.
P. Cardaliaguet (Paris-Dauphine) MFG 58 / 63
![Page 88: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/88.jpg)
Existence/uniqueness of solutions for the (MFG) system
Uniqueness for the (MFG) system
Theorem
Let (m,u) and (m′,u′) be two weak solutions of (MFG). Then m = m′ andu = u′ in m > 0.
Moreover, if u satisfies the additional condition
(∗) − ∂tu + H(x ,Du) ≥ 0 in (0,T )× Rd ,
in the viscosity sense, then u ≥ u′.
Remark : In particular, if we add condition (*) to the definition of weaksolution of (MFG), then the weak solution exists and is unique.
P. Cardaliaguet (Paris-Dauphine) MFG 58 / 63
![Page 89: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/89.jpg)
Existence/uniqueness of solutions for the (MFG) system
Idea of proof :
In a suitable sense,
u(t , x) = infξ(·), ξ(t)=x
∫ T
t(L(ξ(s), ξ′(s)) + α(s, ξ(s))) ds + uT (ξ(T ))
where L(x , y) := supz〈z, y〉 − H(x ,−z) is as smooth as H.
the optimal solutions of the above problems can be built by the pair(m,w) where w = −mDpH(·,Du).
P. Cardaliaguet (Paris-Dauphine) MFG 59 / 63
![Page 90: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/90.jpg)
Conclusion
Outline
1 Preliminary estimates on HJ equations
2 Optimal control of the HJ equation
3 Optimal control of the Kolmogorov equation
4 Existence/uniqueness of solutions for the (MFG) system
5 Conclusion
P. Cardaliaguet (Paris-Dauphine) MFG 60 / 63
![Page 91: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/91.jpg)
Conclusion
Summary : Throughout these lectures, we have seen
how the MFG system is a natural formulation for Nash equilibria fordifferential games with infinitely many players,
that existence/uniqueness of MFG systems are well understood forsecond order problems,
the relation between the 1rst order, nonlocal MFG systems andsemiconcavity,
the relation between the 1rst order, local MFG systems and optimalcontrol of PDEs
P. Cardaliaguet (Paris-Dauphine) MFG 61 / 63
![Page 92: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/92.jpg)
Conclusion
Open problems
Regularity of solutions for 1rst order, local MFG systems.
Existence/uniqueness for the MFG system of congestion type(α ∈ (0,2))
(i) −∂tu +|Du|2
2mα= f (x ,m(x , t))
(ii) ∂tm − div(m1−αDu)) = 0(iii) m(0) = m0, u(x ,T ) = uT (x)
Application to N−player games,
Long-time behavior as T → +∞,
Periodic solutions...
P. Cardaliaguet (Paris-Dauphine) MFG 62 / 63
![Page 93: Optimal Control and Mean Field Games (Part 3)€¦ · Optimal Control and Mean Field Games (Part 3) P. Cardaliaguet Paris-Dauphine “NEW TRENDS IN OPTIMAL CONTROL" Ravello, Italy](https://reader033.vdocuments.mx/reader033/viewer/2022050611/5fb2c46e640c4147930de6ea/html5/thumbnails/93.jpg)
Conclusion
Thank you !
P. Cardaliaguet (Paris-Dauphine) MFG 63 / 63