Optim LettDOI 10.1007/s11590-012-0565-6

ORIGINAL PAPER

Existence of open loop Nash equilibria in certain typesof nonlinear differential games

Atle Seierstad

Received: 16 November 2011 / Accepted: 18 September 2012© Springer-Verlag Berlin Heidelberg 2012

Abstract This paper yields sufficient conditions for existence of open loopNash equilibria in certain types of nonlinear differential games satisfying certainmonotonicity and/or convexity conditions.

Keywords Differential games · Existence of open loop Nash equilibria

Mathematics Subject Classification 91A23

1 Introduction

Certain types of existence of Nash equilibria in differential games have beenpresented (see the references on differential games), partly based on problems witha special structure, e.g. linearity in the state equations. For example, Nikol’skii [8]considers linear problems with so-called programmed strategies. Scalzo and Williams[9] study δ-strategies, with controls appearing linearly. In general nonlinear problems,also Buckdahn et al. [3] and Tolwinski et al. [12] study types of trigger strategies.Azevedo-Perdicoúlis and Jank [2] study existence in affine quadratic differential gamesand Cardaliaguet and Plaskacz [4] discuss existence in a specific simple game. Forcertain nonlinear problems, “direct” existence results for open loop controls can beestablished that to the best of my knowledge is not covered by results in the literature.Below, three theorems are stated. Theorem 3 contains convexity and monotonicity

I am very grateful for comments received from two referees. It made it possible to improve the expositionand remove errors.

A. Seierstad (B)Department of economics, University of Oslo, Box 1095 Blindern, 0317 Oslo, Norwaye-mail: atle.seierstad@econ.uio.no

123

A. Seierstad

conditions that are very easily checked. Theorem 3 is a consequence of more generalresults (Theorems 1 and 2), which are based on certain convexity conditions thatmay be more difficult to check, (when the simple conditions of Theorem 3 are notavailable). In Theorems 2 and 3, but not in Theorem 1, it is assumed that only thecontrol of the player himself enter “his” state equation and criterion..

2 Description of the dynamic game

Let m be the number of players, x (i) ∈ Rni the state of player i, i = 1, . . . , m. u(i) ∈

Rki be the control of player i, taking values in a given compact set Ui ,

x := (x (1), . . . , x (m)), u := (u(1), . . . , u(m)), x0 := (x (1)0 , . . . , x (m)

0 ) given,f (i)(t, x, u) = f (i)(t, x (1), . . . , x (m), u(1)(.), . . . , u(m)(.)) be the instantaneousreward to player i, x (i) =g(i)(t, x, u) = g(i)(t, x (1), . . . , x (m), u(1)(.), . . . , u(m)(.)),

x (i)(0) = x (i)0 , be the state equation of player i.

Below, on vectors, indices that refer to players occur in parentheses.For any vector q, let [q]k be its k’th component. Let k∗

i and k∗∗i be given natural

numbers, k∗i ≤ k∗∗

i ≤ ni and let x ik, k = 1, . . . , k∗∗

i be given real numbers. For eachplayer i, define

Ai :={x ∈ Rni : [x]k = x i

k for k =1, . . . , k∗i , [x]k ≥ x i

k for k =k∗i +1, . . . , k∗∗

i }.(1)

So, for x ∈ Ai , there are no conditions on [x]k, k > k∗∗i . Let M, M ′′ and M

be positive constants, and define the two balls around the origin B := B(0, (|x0| +3M/M ′′)eM ′′T ) ⊂ R

∑i ni and B := B(0, (|x0| + 2M/M ′′)eM ′′T ) ⊂ R

∑i ni . We

assume that f (i), g := (g(1), . . . , g(m)), f (i)x and gx (derivatives with respect to x)

are continuous in J × B ×U, where J := [0, T ], U := U1 ×· · ·×Um, that |g| ≤ M +M ′′|x | for all (t, x, u) ∈ J ×B×U and that | f (i)

x | ≤ M and |gx | ≤ M for all (t, x, u) ∈J × B × U (some such constant M automatically exists). Note that, by Gronwall’sinequality (see e.g. [6], 10.5.1.3), B contains (x(t) = (x (1)(t), . . . , x (m)(t)) for allsolution x (i) of the state equations.

Let a(i) ∈ Rni be a fixed vector for which [a(i)]k = 0, k ≤ k∗∗

i , let T be a fixedend-time, let Ui be the set of all measurable functions with values in Ui and considerthe problem

maxu(i)(.)∈Ui W u(1)(.),...,u(m)(.)i , where (2)

W u(1)(.),...,u(m)(.)i := a(i)xu(1)(.),...,u(m)(.)(i)(T )

+T∫

0

f (i)(t, xu(1)(.),...,u(m)(.)(t), u(1)(.), . . . , u(m)(.)(t))dt,(3)

123

Existence of open loop Nash equilibria

subject to

x ( j) = g ( j)(t, x, u(1)(.), . . . , u(m)(.)), x(0) = x0 (x0 given), j = 1, . . . , m,(4)

x u(1)(.),...,u(m)(.)( j)(T ) free for j �= i, xu(1)(.),...,u(m)(.)(i)(T ) ∈ Ai (5)

Here xu(1)(.),...,u(m)(.)(.) is the solution of the m vector equations (4) andxu(1)(.),...,u(m)(.)( j)(.) is the solution of the j’th vector equation. So player i seeks acontrol u(i)(.) ∈ Ui such that, given u( j)(.) ∈ U j , j �= i, the criterion in (2) is max-imized, subject to the condition x (i)(T ) ∈ Ai . He/she is forced to have x (i)(T ) ∈ Ai

satisfied, but he/she disregards all conditions x ( j)(T ) ∈ A j , j �= i, (to have the latterconditions satisfied is not his/her problem but the problem of the other players!) Thisis called case (α). In another case discussed, however, (case (β)), player i respects allconditions x ( j)(T ) ∈ A j . Player i takes into account the influence of u(i) on every x ( j).

We shall also consider problems where there are no end conditions, a case referred toas Ai = R

ni .

Let p(i) = (p(i)(1), . . . , p(i)

(m)) ∈ R

∑ni , pi

0 ∈ {0, 1}, Hi := pi0 f (i) + p(i)g and

for given x(.) = xu(.), u(.) = (u(1)(.), . . . , u(i)(.)), let the function p(i)(t) =(p(i)

(1)(t), . . . , p(i)(m)(t)) satisfy the following equation

p(i)(t) = −Hix (t, x(t), u(t), p(i)) a.e, (6)

together with the following condition: Define

Qij := {p ∈ R

n j : For k = k∗j + 1, . . . , k∗∗

j : [p]k ≥ 0, [p]k = 0 if [xu(.)( j)(T )]k > x jk },

Q j :={p ∈ Rn j : For k > k∗∗

j , [p]k =0}, Qi := {p ∈ Rni : for k > k∗∗

i , [p]k = pi0[ai ]k}.

Then

p(i)(i)(T ) ∈ Qi

i ∩ Qi . Furthermore,

for j �= i, p(i)( j)(T ) ∈ Qi

j ∩ Q j in case (β), p(i)( j)(T ) = 0 in case (α). (7)

Note that if u(i)(.) is optimal given u( j), j �= i, then for some pi0 ∈ {0, 1} and some

p(i)(.) satisfying (6) and (7), u(i)(.) satisfies the maximum condition a.e.

Hi (t, x(t), u(t), p(i)(t))

= maxui ∈Ui

Hi (t, x(t), u(1)(t), . . . , u(i−1)(t), u(i), u(i+1)(t), . . . , u(m)(t), p(i)(t)) (8)

Denote by x−i the collection (x (1), . . . , x (i−1), x (i+1), . . . , x (m)), a similar notationis used for other collections of m entities.

Let C(U ) be the space of continuous functions on U normed by sup-norm, let Ui :=P(Ui ), the set of probability measures on Ui and let W i := {wi := t → wi

t : wit ∈ Ui

123

A. Seierstad

for all t, t → wit measurable}.1 Furnish W := W 1×· · ·×W m with the weak* topology

defined by the scalar product (φ,w) = ∫ T0

∫U φ(t, u)dwt (u), φ ∈ L1(J, C(U )). Let

f (i)(t, x, wt ) = ∫U f (i)(t, x, u)wt (du) and g(t, x, wt ) = ∫

U g(t, x, u)wt (du) and

note that fx (t, x, wt ) and gx (t, x, wt ) are uniformly continuous in x ∈ B, uniformlyin t and in w := t → wt , because fx (t, x, u) and gx (t, x, u) are uniformly continuousin (t, x, u) ∈ J × B × U . Moreover, | f (i)

x (t, x, wt )| ≤ M, |gx (t, x, wt )| ≤ M forx ∈ B, t ∈ J. Let xw be the solution of xw = g(t, x, wt ), x(0) = x0, let x (i)

w (T ) ∈ Ai

be the end restriction for player i in case (α) (x ( j)w (T ) ∈ A j for all j in case (β)),

and let ai x (i)w (T ) + ∫ T

0 f (i)(t, xw(t), wt )dt be the criterion of player i . (Note thatxw(t) ∈ B for all t, all w.) Let W∗ be the set of controls w ∈ W such that xw(T ) ∈A := A1 × · · · × Am . Define, for w ∈ W,

W i (w) := {wi ∈ W i : For w = (w1, . . . , wi−1, wi , wi+1, . . . , wm),

x ( j)w

(T ) ∈ A j for all j in case (β), for j = i in case (α)}.

3 Results

For both cases (α) and (β), we have:

Theorem 1 Assume that W i (w) is nonempty for all i, w ∈ W . Given w =(w1, . . . , wm), define W i (w) := {wi ∈ W i (w) : wi is optimal for player i , givenw j , j �= i} and assume that W i (w) is convex. Moreover, in case Ai �= R

ni , foreach w ∈ W , assume for each wi ∈ W i (w) that the maximum principle is satis-fied only for pi

0 = 1, not pi0 = 0, (i.e. the above conditions (6), (7), (8) hold when

Ui , ui , f (i), g, u j , j �= i, are replaced by Ui , wi , f (i), g, w j , j �= i). Then some

w := (w1, . . . , wm) ∈ W exists such that wi ∈ W i (w) for all i, i.e. a Nash equilib-rium consisting of open loop relaxed controls exists.2

Remark 1 Assume W∗ �= ∅. Then Wi := {wi ∈ W i : wi ∈ W i (w) for some w ∈ W }is nonempty. In this case, if Wi is also convex, Wi and W ∗ := W1 × · · · × Wm canreplace W i and W in Theorem 1.

Proof We give a proof only in the case f i = 0, k∗i = k∗∗

i . Using auxiliary states, thegeneral case can be derived from this special case. The set W i is compact by standardarguments, (see e.g. [5], (43.3), Proposition). By assumption W i (w) is convex. ThenW (w) := W 1(w)×· · ·× W m(w) ⊂ W is nonempty and convex. Now, W i (w) has theclosed graph property at any w ∈ W by the assumed property pi

0 �= 0 and Lemma 3in Appendix (let C ′ = W −i ), so W (w) has the closed graph property. By Kakutani’stheorem (see Ch.6, Sec 4 in [1]), a fixed point w∗ ∈ W exists, such that w∗ ∈ W (w∗).

��1 Ui has the weak∗ topology. The weak∗ topology on the unit ball in the dual of C(U ) related to thescalar product

∫U γ (u)w(du), γ (.) continuous, can be metrizised, so this measurability is actually a metric

measurability.2 It is a consequence of the assumptions in the Theorem that W i (w) (by standard existence theorems) andW∗ are nonempty.

123

Existence of open loop Nash equilibria

Remark 2 Add the assumption that even W i (w) is convex, but assume that the prop-erty pi

0 �= 0 only holds for w ∈ W i (w) for w ∈ W∗. Now, by continuity ofw → xw(T ), W (w) := W 1(w) × · · · × W m(w) has a closed graph and henceis upper semicontinuous. Let V∗ be the closure of the graph of W (w), and letV (w) = {w : (w, w) ∈ V∗}. Then V (w) and clcoV (w) ⊂ W (w) are nonemptyand upper semicontinuous. By Kakutani’s theorem, a fixed point w∗ ∈ W exists, suchthat w∗ ∈ clcoV (w∗) ⊂ W (w∗). Hence, w∗ ∈ W∗. Now, W (w∗) has the closed graphproperty at w∗, so V (w∗) ⊂ W (w∗), and by convexity clcoV (w∗) ⊂ W (w∗). Hence,w∗ ∈ W (w∗). (Here convexity of W (w) was only used for w ∈ W∗).

Below, we need the assumption that f (i) and g(i) depend only on u(i), not onu( j), j �= i, this is called owns control dependence.

Theorem 2 Assume in the situation of Theorem 1 [both cases (α) and (β)] that wehave owns control dependence and that, for each i, for all (t, x), x ∈ B,

Ni (t, x) = {( f (i)(t, x, ui ) + γi , g(i)(t, x, ui )) : ui ∈ Ui , γi ≤ 0} is convex. (9)

Then a Nash equilibrium in ordinary open loop controls exists.3

Proof In this case it is wellknown that to each relaxed control wi there correspondsan ordinary control ui . Below, using the identification wi ↔ ui , we write W i (u) andW i (u), these sets now consisting of ordinary controls. In Theorem 2 the assumptionis then that for each u(i) = u(i)(.) ∈ W i (u) the maximum principle (6)–(8) (withu(i) = u(i)(.) replaced by u(i)), is only satisfied for pi

0 = 1, not for pi0 = 0. ��

Remark 3 If, for each i, W i (u) contains a single element, and if, for some norm onL1(J, R

∑i ki ), u(.) = u → W (u) is a contraction, (u(i)(.) ∈ Ui for all i), then the

Nash equilibrium in Theorem 2 is unique. The “one-line” proof is virtually the sameas for Theorem 2.5 in [7].

Theorem 3 Assume in the situation of Theorem 2 that no equality terminal restrictionsare present and that all [ai ]k ≥ 0. The conclusion of the theorem still holds if the twoconditions (9) and convexity of W i (w) are replaced by the conditions that f (i) andg(i) are nondecreasing in x for each (u, t), that f (i) and g(i) are concave in (x, ui )

for each i, and that Ui is convex.4

Remark 4 Provided, for each component number k, g(i)k = hi

k + aik(t)x (i)

k , aik(t) con-

tinuous, ≤ 0, we have that the monotonicity of g(i) in x can be replaced by themonotonicity of hi (t, x, u) in x ∈ B. Theorem 3 also holds if f (i) and g(i) arenondecreasing in x (i) and nonincreasing in x−i .

Proof We can assume f (i) ≥ 0, g(i) ≥ 0, (otherwise f (i), [g(i)]k can be replacedf (i)(t, x − 1Mt, ui ) + M, [g(i)(t, x − 1Mt, ui )]k + M), 1 = (1, . . . , 1). Then

3 Without owns control dependence, to obtain this conclusion, we would have to assume instead of (9) that,in an obvious notation, Ni (t, x, w) = {(∫ f (i)(t, x, ui , u−i )w(du−i ) + γi ,

∫g(t, x, ui , u−i )w(du−i ) :

ui ∈ Ui , γi ≤ 0} is convex for all w = w−i , t, x .

4 Using Remark 2, note that pi0 �= 0 need only hold for u ∈ W (u), u ∈ W∗, as W i (w) is convex (see the

proof below).

123

A. Seierstad

{vi ( f (i)(t, x, ui ), g(i)(t, x, ui )) : ui ∈ Ui , vi ∈ [0, 1]} is convex and we can

apply Theorem 2. Let us check that the convexity condition on the set of optimalcontrols holds. Let u = (v, u+) = ((v1, u1+), . . . , (vm, um+)), ui = (vi , ui+) ∈W i (u), ui = (vi , ui+) ∈ W i (u), λ ∈ (0, 1). By monotonicity and optimal-ity (and the lack of equality terminal constraints), we can assume that vi =vi ≡ 1. Define u∗ by ui∗ = (1, λui+ + (1 − λ)ui+), u j∗ = (v j , u j

+), j �= i.

It suffices to show that xu∗(t) ≥ λx (1,ui+),u−i(t) + (1 − λ)x (1,ui+),u−i

(t) wherex (1,ui+),u−i

(t) and x (1,ui+),u−i(t) are the solutions corresponding to (1, ui+), u−i and

(1, ui+), u−i , respectively. This follows if we can show that xu∗(t) ≥ λx (1,ui+),u−i(t)+

(1 − λ)x (1,ui+),u−i(t) �⇒ xu∗(t) ≥ λx (1,ui+),u−i

(t) + (1 − λ)x (1,ui+),u−i(t). The

latter inequality holds because the former inequality implies by monotonicity thatxu∗(i)(t)i ≥ g(i)(λx (1,ui+),u−i

(t) + (1 − λ)x (1,ui+),u−i(t), λui+(t) + (1 − λ)ui+(t)) ≥

λx (1,ui+),u−i (i)(t) + (1 − λ)x (1,ui+),u−i (i)(t) (the last inequality by concavity), and,for j �= i, xu∗( j)(t) ≥ v(t)g( j)(λx (1,ui+),u−i

(t) + (1 − λ)x (1,ui+),u−i(t), u j (t)) ≥

λx (1,ui+),u−i ( j)(t) + (1 − λ)[x (1,ui+),u−i ( j)(t).Of course, a Nash equilibrium {(vi , ui )}i can be assumed to satisfy vi ≡ 1 for all i.This type of argument also implies that W i (w) is convex: When wi

1 ∈ W i (w),

wi2 ∈ W i (w), then for some rapidly switching ordinary controls ui

1, ui2, u, xui

k ,u−1(.) ≈

xwik ,w

−i (.), k = 1, 2, and

xλ(wi1,w

−i )+(1−λ)(wi2,w

−i )(.) ≈ xλ(ui1,u

−i )+(1−λ)(ui2,u

−i )(.)

≥ λxui1,u

−i(.) + (1 − λ)xui

2,u−i

(.)

≈ λxwi1,w

−i (.) + (1 − λ)xwi2,w

−i (.).

��

4 Example

Let xi , i = 1, 2, be the states (xi real capital belonging to player i). The capital xi

develops according to

x i = gi (x1, x2) − ui , ui ∈ [0, Mi ], xi (0) = xi0 > 0, xi

0 fixed, (10)

where ui is the consumption of player i (the control of player i), Mi are given positivenumbers and gi is concave , Lipschitz continuous, C1 for (x1, x2) ∈ R

2 with gix j > 0,

i, j ∈ {0, 1}. Let xiT , i = 1, 2, be given numbers > 0. For any u1, u2 write xu1,u2(.) for

the solution of (10). For any u1, u2 we assume5 that x10,u2(T ) > x1

T and x2u1,0

(T ) > x2T .

The problem of player i is

5 This assumption can be weakened, see next footnote.

123

Existence of open loop Nash equilibria

max

T∫

0

vi (ui )dt subject to (10), xi (T ) ≥ xi

T , x j (T ) free, j �= i,

where vi (u), defined on [0,∞), is continuous, increasing and concave, and is C1 on(0,∞), with limC→0v

′i (C) = ∞.

We have

Hi = pi0vi (Ci ) + p1[g1(x1, x2) − u1] + p2[g2(x1, x2) − u2].

and, for pi = (pi1.p

i2) corresponding to problem i, we have

pij = −pi

1g(1)

x j − pi2g(2)

x j . (11)

We shall prove the existence of a Nash equilibrium in open-loop controls by meansof Theorem 3. The monotonicity and concavity of ( f (i), g(i)) is satisfied, as well ascompactness of Ui . From now on consider i = 1, the case i = 2 has a completelysymmetric treatment. Note that p1(t) ≥ 0 everywhere by (11). To prove p1

0 = 1, notethat, for any u, if 0 ∈ W 1(u), then by assumption x1

0,u2(T ) > x1T . For u1 ≡ 0 ∈

W 1(u), then (p11(T ), p1

2(T )) = 0 and p10 = 1, but then u1 ≡ 0 cannot satisfy the

maximum condition (then Hiu1 = ∞). Thus, for any u1 ∈ W 1(u), u ∈ W∗ we have

that u1(t) > 0 for some Lebesgue point t. At some Lebesgue point t ′ < t, arbitrarilynear t, u1(t ′) ∈ (0, M1] and at this point Hu1 ≥ 0. Now, (p1

0, p11(t), p1

2(t)) �= 0, and(p1

1(t), p12(t)) �= 0, p1

0 = 0 implies by the adjoint equation that p11(t

′) > 0, whichcontradicts Hu1 ≥ 0 at t ′. So p1

0 = 0 is impossible. All conditions in Theorem 3 arethen satisfied, so a Nash equilibrium (u1, u2) exists. The Nash equilibrium found is ofcourse also a “case (β) Nash equilibrium”. (Assume g1(0, x2), g2(x1, 0) ≤ 0 for allx1, x2. Then Lipschitz continuity implies that for some constant γ > 0, x0,0(T ) ≥ xT

only if x0,0(s) ≥ xT /γ = x for all s, in fact, for all u1, u2, xu1,u2(s) ≥ x if xu1,u2(T ) ≥xT . Hence gi need only be defined for x1, x2 ≥ 0, and be C1 for x1, x2 > 0).6

5 Appendix

In proofs in Appendix, Seierstad [10] shall be used. To save space, definitions andassumptions in that paper are used without recalling them here. Below any entity insquare brackets is found in [10].

From [Theorem 1] the following lemma trivially follows.

6 Because W i (u) is convex, by Remark 2, we can operate with a weaker assumption, namely thatx0,u2 (T ) ≥ xT ⇒ x1

0,u2 (T ) > x1T and xu1,0(T ) ≥ xT ⇒ x2

u1,0(T ) > x2

T . If gi ≥ 0 for x1, x2 ≥ 0,

g1(0, x2), g2(x1, 0) ≤ 0 for all x1, x2, and xiT < xi

0, then x0,u2 (T ) ≥ xT automatically implies

x10,u2 (T ) > x1

T .

Note that if 0 ∈ W 1(u), u ∈ W∗, then x10,u2 (T ) ≥ x1

T and x20,u2 (T ) ≥ x2

u1,u2 (T ) ≥ x2T .

123

A. Seierstad

Lemma 1 Assume that the functions [y(a)] and [y+(a)] and the entities [Ad , A, a, ∂]appearing in [Theorem 1] depend on a parameter c ∈ C for some given set C, (wethen write yc(a), y+

c (a), Acd , Ac, ac, ∂c, Ac metricized by ∂c). Assume that a → yc(a)

is continuous, Ac is complete, and that [(A)], [(D)], [(B)] and [(C)] and B( pc, 2e) ⊂y+

c (Ac) hold for the entities yc(a), y+c (a), Ac

d , Ac, ∂c, for each c in C, [d0] and [e(.)]in [(D)] independent of c. Then for some d ′ > 0, for each c in C, B(d pc, de) +yc(ac) ⊂ yc(clAc

d), d ∈ (0, d ′].In the next lemma we need the following entities. Let π := x = (x1, . . . , xn) →(x1, . . . , xn∗), n∗ < n, f (x, t, u, c) : R

n × J × U × C ′ → Rn, U , C ′ metric

spaces, let U = {u(.) : u(t) ∈ U for all t, u(.) measurable}, and let σ(u, u′) :=meas {t : u(t) �= u′(t)}. For some open convex set B, let f (x, t, u, c) be Borelmeasurable in (t, u) and continuously differentiable in x ∈ B, the continuity offx being uniform in x , uniformly in t, u, c. For some constant M, assume that| f (x, t, u, c)| ≤ M, | fx (x, t, u, c)| ≤ M for (x, t, u, c) ∈ B × J × U × C ′. Letxc

u(.), u = u(.), be defined by x cu = f (x, t, u(t), c), x(0) = x0 (assumed to exist for

all c, u), and assume for some δ∗ > 0 that B(xcu(t), δ∗) ⊂ B for all t, c, u. For given

u, u ∈ U , qcu,u is defined by

qcu,u = fx (xc

u(t), t, u(t), c)qcu,u + f (xc

u(t), t, u(t), c) − f (xcu(t), t, u(t), c),

qcu,u(0) = 0. (12)

Let δ be a metric on U and assume, for any continuous x(t) taking values in B, andfor any continuous q(t), that (u, c) → ∫ t

0 f (x(s), s, u(s), c), (u, c) ∈ U × C ′ and(u, c) → ∫ t

0 fx (x(s), s, u(s), c)q(t) are continuous for all t and that the continuity ofc → ∫ t

0 f (x(s), s, u(s), c) is uniform in u(.). The continuity of the first integral thenalso holds if δ is replaced by σ, as is easily seen.

Note that it follows from Gronwall’s inequality that (u, c) → xcu(.) (both for δ and

σ) and c → qcu,u(.) are continuous for sup-norm on xc

u(.) and qcu,u(.), the continuity

of qcu,u(.) being uniform in u(.). See Lemma 4 below.

Lemma 2 Let (u(.), c) be any given element in U ×C ′. If B( p, 3e) ⊂ clco{πqcu,u(T ) :

u ∈ U }, then for some d∗ > 0, d∗ > 0, for any c ∈ B(c, d∗), πxcu(T )+ B(d p, de) ⊂

{πxcu(T ) : u ∈ U , σ (u(.), u(.)) ≤ dT }, d ∈ (0, d∗).

Proof It suffices to prove this for T = 1. Let us use the identifications ac =f (., ., u(.), c), a = f (., ., u(.), c), y+

c (a) = πqcu,u(T ), yc(a) = πxc

u(T ), Ac =Fc = { f (., ., u(.), c) : u(.) ∈ U }, ∂c( f (., ., u1(t), c), f (., ., u2(t), c)) =meas{t : f (., t, u1(t), c) �= f (., t, u2(t), c)}. Because c → qc

u,u(1) is continuous, uni-

formly in u(.) ∈ U , then B( p, 2e) ⊂ clco{πqcu,u(T ) : u ∈ U } for all c close to c,

say c ∈ B(c, d∗) =: C (see Lemma 11.1 in [11]). The uniform continuous differen-tiability with respect to x ∈ B, uniformly in t, u, c, implies that [(8) p. 485] holdsfor f (x, t) = f (x, t, u(t), c) for [ f ], [g] in B||.||∗( f (., ., u(.), c), d) ⊂ Fc for any

123

Existence of open loop Nash equilibria

c, [e(d)] independent of c for ||.||∗ see [(7)], with [B ′] equal to B × [0, 1]).7 Thisimplies that [(A)],[(B)],[(C)] and [(D)] holds in the manner described in Lemma 1,(for [e(d)] = [2Me(2Md)]), and from that Lemma the conclusion in Lemma 2 fol-lows. (Note that Fc is automatically closed under switching and is complete in themetric σ, for the last fact see [11]). ��

Lemma 3 For a fixed vector a, with [a]k = 0, k ≤ n∗, for any given c ∈ C ′ let Z∗(c)be the set of optimal controls u(.) ∈ U maximizing axc

u(T ) subject to πxcu(T ) = z, this

set assumed to be nonempty. For a given c ∈ C ′, assume for any u(.) ∈ Z∗(c) that thenecessary conditions (maximum principle) are satisfied for p0 = 1, not p0 = 0, p0 ∈{0, 1} being the multiplier in the transversality condition [p(T )]k = p0[a]k, k > n∗(no information on [p(T )]k, k ≤ n∗). Then Z∗(c) has the closed graph propertyat c.

Proof To prove the closed graph property, assume that un ∈ Z∗(cn), un → u ∈ U inδ, cn → c ∈ C ′, cn ∈ C ′, and let us show that u ∈ Z∗(c). By continuity, πxc

u(T ) = z.Take any u∗ in Z∗(c). The fact that the necessary conditions are not satisfied forp0 = 0 means that for some ε > 0, B(0, 3ε) ⊂ clco{πqc

u,u∗(T ) : u ∈ U }. To see this,note that the origin 0 belongs to clcoπqc

U ,u∗(T ). If 0 were a boundary point, then for

some nonzero p∗, p∗ clco πqcU ,u∗

(T ) ≤ p∗0 = 0, which implies that the maximum

principle is satisfied for p0 = 0. So 0 is an interior point. By Lemma 2, for somed∗ > 0, d∗ > 0, for all c ∈ B(c, d∗), all d ∈ (0, d∗],

B(0, dε) ⊂ {πxcu(T ) − πxc

u∗(T ) : u ∈ U , σ (u, u∗) ≤ dT }. (13)

Now, for any natural number m such that 1/m < min{d∗, d∗}, a number nm ≥m exists, such that dist(cnm , c) ≤ 1/m (⇒ cnm ∈ B(c, d∗)) and such thatαnm < ε/m, where αn = |xc

u∗(T ) − xcnu∗(T )|, by continuity. Then, πxc

u∗(T ) −πx

cnmu∗ (T ) ∈ clB(0, αnm ) ⊂ B(0, ε/m) ⊂ B(0, d∗ε). Thus, by (13), for some

unm ∈ U , σ (unm , u∗) ≤ T/m, πxcnmunm (T ) − πx

cnmu∗ (T ) = πxc

u∗(T ) − πxcnmu∗ (T ),

hence πxcnmunm (T ) = πxc

u∗(T ) = z. Now, by optimality, axcnmunm

(T ) ≥ axcnmunm (T ) and,

by continuity, axcu(T ) = limm→∞ax

cnmunm

(T ) ≥ limm axcnmunm (T ) = axc

u∗(T ). Hence,u ∈ Z∗(c). ��

Lemma 4 (u, c) → xcu(.) and c → qc

u,u(.) are continuous, the continuity of qcu,u(.)

being uniform in u(.).

7 [e(d)] equals [sup f ∈B||.||∗ ( f (.,.),d)|Dx f (.) − Dx f (.)|], which in [10] is small by the continuity of

x → fx . When this continuity is uniform in x, uniformly in c (for f = f (., ., u(.), c)), then [e(d)] is smalluniformly in c. For more details, see Lemma 5.

123

A. Seierstad

Proof Gronwall’s inequality is used below. In a shorthand notation,

∣∣∣xc′

u′(t) − xcu(t)

∣∣∣ =

∣∣∣

t∫

0

f (xc′u′ , u′, c′)dr −

t∫

0

f (xcu, u, c)dr

∣∣∣

=∣∣∣

t∫

0

f (xc′u′ , u′, c′)dr −

t∫

0

f (xcu, u′, c′)dr +

t∫

0

f (xcu, u′, c′)dr

−t∫

0

f (xcu, u, c)dr

∣∣∣

≤ M

t∫

0

∣∣∣xc′

u′ − xcu |dr + |

t∫

0

f (xcu, u′, c′)dr −

t∫

0

f (xcu, u, c)dr

∣∣∣,

hence

|xc′u′(t) − xc

ut)| ≤∣∣∣

t∫

0

f (xcu, u′, c′)dr −

t∫

0

f (xcu, u, c)dr

∣∣∣

+ eMt

t∫

0

∣∣∣

s∫

0

f (xcu, u′, c′)dr −

s∫

0

f (xcu, u, c)dr

∣∣∣ds

By boundedness of f, the right hand side → 0 when (u′, c′) → (u, c).Define q∗c

u,u(.) by (12) for fx (xcu(t), t, u(t), c) replaced by fx (xc′

u (t), t, u(t), c).Then, by | fx | ≤ M,

|q∗cu,u(t) − qc′

u,u(t)| ≤ M

t∫

0

|qc∗u,u(r) − qc′

u,u(r)|dr

+ |t∫

0

[ f (xcu(r), r, u(r), c) − f (xc′

u (r), r, u(r), c′)]dr

−t∫

0

[ f (xcu(r), r, u(r), c) + f (xc′

u (r), r, u(r), c′))]dr |,

so |q∗cu,u(t)−qc′

u,u(t)| is small when c′ is close to c, uniformly in u, by boundedness off (for each r, by Lipschitz continuity in x , the two last integrands are small uniformlyin u when c is close to c′). Finally (shorthand notation), for

123

Existence of open loop Nash equilibria

φ(t) :=∣∣∣

t∫

0

( fx (xcu, r, u, c)qc

u,u − fx (xc′u , r, u, c))q∗c

u,udr∣∣∣

=∣∣∣

t∫

0

fx (xcu, r, u, c)(qc

u,u − q∗cu,u)dr + η(t)

∣∣∣,

where

η(t) :=t∫

0

( fx (xcu, r, u, c) − fx (xc′

u , r, u, c))q∗cu,udr |,

we have

|qcu,u(t) − q∗c

u,u(t)| = φ(t) ≤ M∫

|qcu,u − qc∗

u,u | + |η(t)|

so |qcu,u(t) − q∗c

u,u(t)| ≤ |η(t)| + eMt∫ t

0 |η(s)|ds is small uniformly in u when c isclose to c′ by boundedness, because the integrand in η(t) converges for each r to 0uniformly in u when c → c′. ��Lemma 5 Denote by qc

f, f(.) the function obtained from (12) when f (., ., u(.), c)

and f (., ., u) are replaced by f, and f , respectively, f ∈ linspanFc, f ∈ Fc. Then|qc

f, f(.) − qc

f, f(.)|1 is small, uniformly in c and in f, || f ||∗ ≤ 1, when || f − f ||∗ is

small, f ∈ linspanFc, f ∈ Fc.

Proof Writing xcf(.) for the solution of x = f (x, t, c), x(0) = x0, f ∈ linspan Fc,

note that for any f ∈ linspanFc,

| f (xcf, t, c) − f (xc

f, t, c)| ≤ sup

y∈[xcf:xc

f]| fx (y, t, c)|xc

f− xc

f|∞

= | fx (z(t), t, c)|xcf− xc

f|∞

for some measurable z(t) ∈ B, so (approximating z(t) by a continuous function in B)| f (xc

f, ., c)− f (xc

f, t, c)|1 ≤ || f ||∗|xc

f− xc

f|∞. Now, |xc

f(.)− xc

f(.)|∞ ≤ eMT || f −

f ||∗ for f ∈ Fc, f ∈ linspanFc, and |qcf, f

| ≤ || f − f ||∗(1 + || f ||∗ )e|| f ||∗ , f, f ∈linspan Fc, both by Gronwall’s inequality. Then | fx (xc

f, ., c)− fx (xc

f, ., c)|∞ is small

when || f − f ||∗ is small ( f ∈ Fc, f ∈ linspan Fc), uniformly in c and f ∈ Fc, bythe uniform continuity assumption on x → f (x, t, u, c). From this, by Gronwall’sinequality, for any given K > 0, it easily follows that |qc

f, f(.) − qc

f, f(.)|∞ and hence

also |qcf, f

(.) − qcf, f

(.)|1 is small when || f − f ||∗ is small, f ∈ linspanFc, f ∈ Fc,

123

A. Seierstad

uniformly in c and f ∈ linspanFc, || f ||∗ ≤ K . To give some details, in a shorthandnotation,

φ(t) := qcf, f

(t) − qcf, f

(t)

= fx (xcf)qc

f, f(.) + f (xc

f) − f (xc

f) − fx (xc

f)qc

f, f(.) − f (xc

f) + f (xc

f)

= fx (xcf)(qc

f, f(.) − qc

f, f(.)) + f (xc

f) − f (xc

f) + f (xc

f) − f (xc

f)

+( fx (xcf)− fx (xc

f))qc

f, f(.)+( fx (xc

f)− fx (xc

f))qc

f, f(.)+ f (xc

f)− f (xc

f)|.

Hence

α(t) := |t∫

0

φ(r)dr | ≤ M

t∫

0

α(r)dr + a, where

a := || f ||∗|xcf− xc

f|∞ + || f − f ||∗ + |qc

f, f(.)|∞|| f − f ||∗

+ | fx (xcf) − fx (xc

f)|∞|qc

f, f(.)|∞ + M |xc

f− xc

f|∞.

Thus, α(t) ≤ aeM , where a is small when || f − f ||∗ is small, || f ||∗ ≤ K . ��

References

1. Aubin, J.P., Ekeland, I.: Applied Nonlinear Analysis. Wiley, New York (1984)2. Azevedo-Perdicoúlis T-P., Jank G.: Existence and uniqueness for disturbed open-loop Nash equilibria

for affine-quadratic differential games. In: Advances in dynamic games, pp 25–39. Ann. Internat. Soc.Dynam. Games 11, Birkhäuser/Springer, New York (2011)

3. Buckdahn, R., Cardaliaguet, P., Rainer, C.: Nash equilibrium payoffs for nonzero-sum stochasticdifferential games. SIAM J. Control Optim. 43, 624–642 (2004)

4. Cardaliaguet, P., Plaskacz, S.: Existence and uniqueness of a Nash equilibrium feedback for a simplenonzero-sum differential game. Int. J. Game Theory 32, 33–71 (2003)

5. Davis, M.H.A.: Markov Models and Optimization. Chapman and Hall, London (1993)6. Dieudonné, J.: Foundations of Modern Analysis. Academic Press, New York (1960)7. Friedman, J.W.: Game Theory with Applications to Economics. Oxford University Press, New York

(1986)8. Nikol’skii, M.S.: On the existence of Nash equilibrium points for linear differential games in

programmed strategies (Russian), Problems in dynamic control No1 222–229, Mosk. Gos Univ. im.Lomonosova, Fak. Vychisl. Mat. Kibern, Moscow (2005)

9. Scalzo, R.C., Williams, S.A.: On the existence of a Nash equilibrium point of N-person differentialgames. Appl. Math. Optim. 2(3), 271–278 (1976)

10. Seierstad, A.: A local attainability property for control systems defined by nonlinear ordinary differ-ential equations in a Banach space. J. Differ. Equ. 8, 475–487 (1970)

11. Seierstad, A.: An extension to Banach space of Pontryagin’s maximum principle. J. Optim. TheoryAppl. 17, 293–335 (1975)

12. Tolwinski, A., Haurie, A., Leitmann, G.: Cooperative equilibria in differential games. J. Math. Anal.Appl. 119, 182–202 (1986)

123