Controlling propagation of epidemics:Mean-field SIR games

Stanley Osher

Joint work with Wonjun Lee, Siting Liu, Hamidou Tembine andWuchen Li

COVID 19

As of May 30, 2020, the total case of COVID 19 has reached:

2

COVID 19 in US

3

Goals

Fight against COVID-19 by optimal transport and mean field games.

4

Classic Epidemic Model

The classical Epidemic model is the SIR model (Kermack andMcKendrick, 1927) 8

>>>>><

>>>>>:

dS

dt= ��SI

dI

dt= �SI � �I

dR

dt= �I

where S, I,R : [0, T ] ! [0, 1] represent the density of the susceptiblepopulation, infected population, and recovered population, respectively,given time t. The nonnegative constants � and � represent the rates ofsusceptible becoming infected and infected becoming recovered.

5

Spatial SIR

To model the spatial e↵ect of virus spreading ,the spatial SIR model isconsidered:8>>>>>>><

>>>>>>>:

@t⇢S(t, x) + �⇢S(t, x)

Z

⌦K(x, y)⇢I(t, y)dy �

⌘2S

2�⇢S(t, x) = 0

@t⇢I(t, x)� �⇢I(x)

Z

⌦K(x, y)⇢S(t, y)dy + �⇢I(t, x)�

⌘2I

2�⇢I(t, x) = 0

@t⇢R(t, x)� �⇢I(t, x)�⌘2R

2�⇢R(t, x) = 0

Here ⌦ is a given spatial domain and K(x, y) is a symmetric positivedefinite kernel modeling the physical distancing. E.g.

RKd⇢I is the

exposure to infectious agents.6

Optimal control of population behaviors

Optimal control of population behaviors have been widely considered inoptimal transport and mean field games. Long story short, it refers to anoptimal control problem in density space:

min Running cost of a population

s.t.Evolution of population dynamics

E.g.

7

S

I

S

I

R

MFG Related

I Introduced by Jovanovic & Rosenthal[JR88], M. Huang, P. Caines,R. Malhame [HMC06] and P.-L. Lions, J.-M. Lasry [LL06a, LL06b]to model huge populations of identical agents playingnon-cooperative di↵erential games.

I Wide applications to various fields: in economics, Finance, crowdmotion, industrial engineering, data science, material dynamics, andmore [GNP15, BDFMW13, LLLL16, AL19].

I Computational methods developed to solve high dimensionalproblems. [BC15, BnAKS18, EHL18, LFL+20, ROL+19, LJL+20].

Figure: Divers with Schooling Fish

9

Related Study on COVID-19

I Study traveling waves to understand the propagation of epidemics.In [BRR20], they introduce a SIRT model to study the e↵ects of the

presence of a road on the spatial propagation the epidemic.

I Optimal control with control measures on medicare (vaccination)

I Machine Learning, Data Driven + Epidemic model

Figure: Social Distancinghttps://www.wfla.com/news/by-the-numbers/tampa-bay-counties-earn-d-and-f-grades-for-social-distancing/https://s.hdnux.com/photos/01/12/06/10/19423760/5/1024x1024.jpg

Understand connection between the society (global) and the individuals(local) .

10

Spatial SIR variational problems

Construct the following variational problem to balance virus spreadingand “social” cost.

min⇢i,vi

E(⇢I(T, ·)) +

ZT

0

Z

⌦

X

i=S,I,R

↵i

2⇢ikvik

2 +c

2(⇢S + ⇢I + ⇢R)

2dxdt

subject to8>>>>>>>>><

>>>>>>>>>:

@t⇢S +r · (⇢SvS) + �⇢S⇢I �⌘2S

2�⇢S = 0

@t⇢I +r · (⇢IvR)� �⇢S⇢I + �⇢I �⌘2I

2�⇢I = 0

@t⇢R +r · (⇢RvR)� �⇢I �⌘2R

2�⇢R = 0

⇢S(0, ·), ⇢I(0, ·), ⇢R(0, ·) are given.

11

Mean-field game SIR systems

8>>>>>>>>>>>>>>>>>>>>>>>>><

>>>>>>>>>>>>>>>>>>>>>>>>>:

@t�S �↵S

2|r�S |

2 +⌘2S

2��S + c(⇢S + ⇢I + ⇢R)

+ � (K ⇤ (�I⇢I)� �SK ⇤ ⇢I) = 0

@t�I �↵I

2|r�I |

2 +⌘2I

2��I + c(⇢S + ⇢I + ⇢R)

+ � (�IK ⇤ ⇢S �K ⇤ (�S⇢S)) + �⇢(�R � �I) = 0

@t�R �↵R

2|r�R|

2 +⌘2R

2��R + c(⇢S + ⇢I + ⇢R) = 0

@t⇢S �1

↵S

r · (⇢Sr�S) + �⇢SK ⇤ ⇢I �⌘2S

2�⇢S = 0

@t⇢I �1

↵I

r · (⇢r�I)� �⇢IK ⇤ ⇢S + �⇢I �⌘2I

2�⇢I = 0

@t⇢R �1

↵R

r · (⇢Rr�R)� �⇢I �⌘2R

2�⇢R = 0.

12

Review on PDHG method

Consider a saddle point problem

minx

supy

{L(x, y) := hAx, yi+ g(x)� f⇤(y)} .

Here, f and g are convex functions with respect to a variable x, A is acontinuous linear operator. For each iteration, the algorithm finds theminimizer x⇤ by gradient descent method and the maximizer y⇤ bygradient ascent method. Thus, the minimizer and maximizer arecalculated by iterating

(xk+1 = argmin

xL(x, yk) + 1

2⌧ kx� xkk2

yk+1 = argmaxyL(xk+1, y) + 1

2�ky � ykk2

where ⌧ and � are step sizes for the algorithm.

13

Review on G-Proximal

Here G-Prox PDHG is a modified version of PDHG that solves theminimization problem by choosing the most appropriate norms forupdating x and y. Choosing the appropriate norms allows us to chooselarger step sizes. Hence, we get a faster convergence rate. In details,

(xk+1 = argmin

xL(x, yk) + 1

2⌧ kx� xkk2H

yk+1 = argmaxyL(xk+1, y) + 1

2�ky � ykk2G

where H and G are some Hilbert spaces with the inner product

(u1, u2)G = (Au1, Au2)H.

14

Algorithm: Primal-Dual updates

In particular, we use G-Prox PDHG to solve the variational SIR model by

x = (⇢S , ⇢I , ⇢R,mS ,mI ,mR), g(x) = F (⇢i,mi)i=S,I,R,

f(Ax) =

(0 if Ax = (0, 0, �⇢I)

1 otherwise.

Ax = (@t⇢S +r ·mS �⌘2S

2�⇢S + �⇢SK ⇤ ⇢I ,

@t⇢I +r ·mI �⌘2

2�⇢I � �⇢IK ⇤ ⇢S + �⇢I ,

@t⇢R +r ·mR �⌘2

2�⇢R).

15

Variational formulation

Denote mi = ⇢ivi. Define the Lagrangian functional for Mean field gameSIR problem by

L((⇢i,mi,�i)i=S,I,R)

=P (⇢i,mi)i=S,I,R �

ZT

0

Z

⌦

X

i=S,I,R

�i

✓@t⇢i +r ·mi �

⌘2i

2�⇢i

◆dxdt

+

ZT

0

Z

⌦��I⇢IK ⇤ ⇢S � ��S⇢SK ⇤ ⇢I + �⇢I(�R � �I)dxdt.

Using this Lagrangian functional, we convert the minimization probleminto a saddle problem.

inf(⇢i,mi)i=S,I,R

sup(�i)i=S,I,R

L((⇢i,mi,�i)i=S,I,R).

16

Algorithm

Algorithm: PDHG for mean field game SIR systemInput: ⇢i(0, ·) (i = S, I, R)Output: ⇢i,mi,�i (i = S, I, R) for x 2 ⌦, t 2 [0, T ]

While relative error > tolerance

⇢(k+1)i

= argmin⇢L(⇢,m(k)

i,�(k)

i) + 1

2⌧ik⇢� ⇢(k)

ik2L2

m(k+1)i

= argminmL(⇢(k+1),m,�(k)

i) + 1

2⌧ikm�m(k)

ik2L2

�(k+ 1

2 )i

= argmax�L(⇢(k+1),m(k+1)

i,�)� 1

2�ik�� �(k)

ik2H2

�(k+1)i

= 2�(k+ 1

2 )i

� �(k)i

end

17

Examples I

19

Small recovery rate

Example II

20

Large recovery rate

Example III

21

Small recovery rate

Example IV

22

Large recovery rate

Discussions

Importance of spatial SIR variational problems.

I Consider more status of populations, going beyond S, I, R.

I Construct discrete spatial domain model, including airport, trainstation, hospital, school etc.

I Propose inverse mean field SIR problems. Learning parameters inthe model by daily life data.

I Combine mean field game SIR models with AI and machine learningalgorithms, including APAC, Neural variational ODE, NeuralFokker-Planck equations, etc.

22

References

W. Lee, S. Liu, T. Tembine, W. Li, S. Osher.Controlling Propagation of epidemics via mean-field games, 2020.

23

Yves Achdou and Jean-Michel Lasry.Mean field games for modeling crowd motion.In Contributions to partial di↵erential equations and applications,volume 47 of Comput. Methods Appl. Sci., pages 17–42. Springer,Cham, 2019.

J.-D. Benamou and G. Carlier.Augmented Lagrangian methods for transport optimization, meanfield games and degenerate elliptic equations.J. Optim. Theory Appl., 167(1):1–26, 2015.

Martin Burger, Marco Di Francesco, Peter Markowich, andMarie-Therese Wolfram.Mean field games with nonlinear mobilities in pedestrian dynamics.arXiv preprint arXiv:1304.5201, 2013.

L. M. Briceno Arias, D. Kalise, and F. J. Silva.Proximal methods for stationary mean field games with localcouplings.SIAM J. Control Optim., 56(2):801–836, 2018.

Henri Berestycki, Jean-Michel Roquejo↵re, and Luca Rossi.

23

Propagation of epidemics along lines with fast di↵usion.arXiv preprint arXiv:2005.01859, 2020.

Weinan E, Jiequn Han, and Qianxiao Li.A Mean-Field Optimal Control Formulation of Deep Learning.arXiv:1807.01083 [cs, math], 2018.

Diogo A Gomes, Levon Nurbekyan, and Edgard A Pimentel.Economic models and mean-field games theory.IMPA Mathematical Publications. Instituto Nacional de MatematicaPura e Aplicada (IMPA), Rio de Janeiro, 2015.

M. Huang, R. P. Malhame, and P. E. Caines.Large population stochastic dynamic games: closed-loopMcKean-Vlasov systems and the Nash certainty equivalenceprinciple.Commun. Inf. Syst., 6(3):221–251, 2006.

Boyan Jovanovic and Robert W. Rosenthal.Anonymous sequential games.Journal of Mathematical Economics, 17(1):77 – 87, 1988.

23

Alex Tong Lin, Samy Wu Fung, Wuchen Li, Levon Nurbekyan, andStanley J. Osher.Apac-net: Alternating the population and agent control via twoneural networks to solve high-dimensional stochastic mean fieldgames, 2020.

Siting Liu, Matthew Jacobs, Wuchen Li, Levon Nurbekyan, andStanley J Osher.Computational methods for nonlocal mean field games withapplications.arXiv preprint arXiv:2004.12210, 2020.

Jean-Michel Lasry and Pierre-Louis Lions.Jeux a champ moyen. I. Le cas stationnaire.C. R. Math. Acad. Sci. Paris, 343(9):619–625, 2006.

Jean-Michel Lasry and Pierre-Louis Lions.Jeux a champ moyen. II. Horizon fini et controle optimal.C. R. Math. Acad. Sci. Paris, 343(10):679–684, 2006.

Aime Lachapelle, Jean-Michel Lasry, Charles-Albert Lehalle, andPierre-Louis Lions.

23

E�ciency of the price formation process in presence of highfrequency participants: a mean field game analysis.Mathematics and Financial Economics, 10(3):223–262, 2016.

Lars Ruthotto, Stanley Osher, Wuchen Li, Levon Nurbekyan, andSamy Wu Fung.A machine learning framework for solving high-dimensional meanfield game and mean field control problems, 2019.

23