differential game theory for traffic flow modelling

Modelling (Active) Traffic

An Introduction to Applications of Optimal Control and Differential Game Theory Prof. dr. Serge Hoogendoorn

1

2

Pedestrian modeling questions in practise… • Redesign of the Al Mataf Mosque

(testing for safety and throughput) • Station design (e.g. Amsterdam Zuid)

to show if LOS remains acceptable • Investigating building evacuations

(e.g. impact of counterflow WTC 9/11) • Testing crowd management plans

during SAIL tallship event in Amsterdam

For each of these (and many others) applications, the key question is: “Can we provide predictively valid models that can predict LOS, throughput, and safety pinch points in case of regular or irregular conditions”

“A model is as good as the predictions it provides”

• Questionable if from this engineering perspective, ped models are predictively valid (in contrast to models for car traffic)

• Why? In our field, DATA is key in the development of theory and models

• Pedestrian theory (and cyclists) has suffered from lack of data and has been “assumption rich and data poor”

• Some examples of different data collection exercises that we have performed from say 2000 onward

3

Understanding transport begins and ends with data

Let’s start with the pedestrians…

Speed density relation pedestrian flow

• Fundamental diagram for pedestrian flow stemming from 2002 experiments

• Non-increasing relation between speed and density showing reduction in speed as density increases (or increase in density as speed reduces?)

• Strong impact of flow composition (e.g. purpose, age, gender, etc.)

• Can you interpret the relation? That is, can you explain the shape?

5

6Example shared-space region Amsterdam Central Station

Empirical facts of self-organisation (2002 experiment) • Bi-directional flow experiment revealed self-organisation of lanes • Process has chaotic features (e.g. equilibrium states formed depend critically on initial and

boundary conditions, ill-predictable) • Collected microscopic data allowed for unique quantification self-organised features


Empirical facts of self-organisation (2002 experiment) • Bi-directional flow experiment revealed self-organisation of lanes • Process has chaotic features (e.g. equilibrium states formed depend critically on initial and

boundary conditions, ill-predictable) • Collected microscopic data allowed for unique quantification self-organised features

1 2 3 40

0.2

0.4

0.6

Number of lanes formed

Rela

tive

frequ

ency

Density = 0 to 0.25 Ped/m2

1 2 3 40

0.2

0.4

0.6


Rela

tive

frequ

ency

Density = 0.25 to 0.5 Ped/m2

1 2 3 40

0.2

0.4

0.6


Rela

tive

frequ

ency


1 2 3 40

0.2

0.4

0.6


Rela

tive

frequ

ency

Density = 0.75 to 1 Ped/m2

0 0.2 0.4 0.6 0.80.9

1

1.1

1.2

1.3

density (Ped/m2)

spee

d (m

/s)

1234

1 2 3 40

0.2

0.4

0.6


Rela

tive

frequ

ency

Density = 0 to 0.25 Ped/m2

1 2 3 40

0.2

0.4

0.6


Rela

tive

frequ

ency


1 2 3 40

0.2

0.4

0.6


Rela

tive

frequ

ency


1 2 3 40

0.2

0.4

0.6


Rela

tive

frequ

ency

Density = 0.75 to 1 Ped/m2

• Examples show how likelihood of certain number of lanes formed depends on density• Right picture shows how speed-density relation is (to a limited extent) dependent on

the number of lanes that is formed

Further experimentation supports and extends our findings • Example shows results by Prof. Seyfried (downloadable via the Juelich Centre website)

http://www.fz-juelich.de/ias/jsc/EN/Research/ModellingSimulation/CivilSecurityTraffic/PedestrianDynamics/Activities/database/


Traffic Flow Phenomena • Other self-

organised patterns found for other experiments

• Example shows formation of diagonal stripes

• Other examples include zipper effect, viscous fingering, and faster is slower effect

• Self-organisation also occurs in practise…

10Example shared-space region Amsterdam Central StationSelf-organisation during LowLands…

Modelling challenge…

• To come up with a model (and underlying theory) that can predict observed relations (e.g. speed-density) and phenomena for different situations

• In 2003, we opted for differential game theory to model the behaviour of pedestrians that are competing for the use of (scarce) space

• Some motivation? - We know that under specific conditions, differential game theory predicts

occurrence of (meta-) stable equilibrium state, which (hopefully) are our self-organised patterns…

- Moreover, research from the late Seventies and Eighties provides us with evidence that could be used as a basis for a game-theoretical model…

• Let us have a brief look at the different behavioural assumptions (the theory) that underly the game theoretical model…

11

Microscopic pedestrian modelling

• Main assumption “pedestrian economicus” based on principle of least effort: For all available options (accelerating, changing direction, do nothing) he chooses option yielding smallest predicted disutility (predicted effort)

• When predicting walking effort, he values and combines predicted attributes characterising available options (risk to collide, walking too slow, straying from intended walking path, etc.)

12

Six additional behavioural assumptions…

• Pedestrians are feedback-oriented, reconsidering their decisions based on current situation

• They anticipate behaviour of others by predicting their walking behaviour according to non-co-operative or co-operative strategies

• Their predicting abilities are limited, reflected by discounting effort of their actions over time and space

• Pedestrians are anisotropic in that react mainly to stimuli in front of them

• They minimise predicted discounted costs resulting from: (a) straying from planned path; (b) vicinity of other peds (+ obstacles); (c) applying control

• Pedestrians are more evasive encountering a group than a single pedestrian

The Math…

• State equation describes ‘mental model’ peds to predict from where the state describes the positions and velocities of ped p and his opponents q (e.g. ), and were the control is the acceleration that a pedestrian can apply

• Prediction model describes kinematics of pedestrians, e.g. and • Ped p chooses control (accel.) minimising effort for

x(t) = f(t, x, u) x(tk) = xksubject to

r(t) v(t)x(t)rp(t) u(t)

r = v v = u

[tk, tk + T )

Jp =

Z tk+T

tk

e

�⌘sLp(s, x(s), u(s))ds+ e

�⌘(tk+T )�p(tk + T, x(tk + T ))

t = tk

u[tk,tk+T )

Using the assumptions to specify model

• Most behavioural assumptions are specified via running cost where: and with the proximity cost equal to:

Lp(t, x, u)

Lp

= Lstray

p

+ Lprox

p

+ Laccel

p

Lstrayp =

1

2(v0p � vp)

2 Laccelp =

1

2u2p

Lprox

p

=

X

q2Q

e�dpq/RP

✓ p

+ (1� p

)

1 + cos ✓pq

2

◆

and



Lp(t, x, u)

Lp

= Lstray

p

+ Lprox

p

+ Laccel

p

Lprox

p

=

X

q2Q

e�dpq/RP

✓ p

+ (1� p

)

1 + cos ✓pq

2

◆

andLstrayp =

1

2(v0p � vp)

2

Straying cost describe the impact of not walking in the desired direction and at the desired speed

Laccelp =

1

2u2p

Acceleration cost describe the cost of applying the control acceleration in

long. and lat. direction

Lstrayp =

1

2(v0p � vp)

2 Laccelp =

1

2u2p



Lp(t, x, u)

Lp

= Lstray

p

+ Lprox

p

+ Laccel

p

and

Proximity cost shows spatial discounting of cost impact using distance

Lprox

p

=

X

q2Q

e�dpq/RP

✓ p

+ (1� p

)

1 + cos ✓pq

2

◆

Impact of ‘groups’ by adding proximity costs over opponents

Anisotropy is reflected by making cost

dependent on angle ✓pqp

q

✓pq

rq � rp

vp

dpq = ||rq � rp||

Solving the problem?

• Problem can be solved using the Minimum Principle of Pontryagin• Without going into details…• Define Hamiltonian function:

and use it for necessary conditions for optimality of control signal• Next to the state equation + initial conditions, we can derive an equation for

the co-states (a.k.a. marginal costs) + terminal condition and optimality conditions to determine optimal acceleration

18

Hp = e�⌘tLp + �0p · f

u⇤[tk,tk+T )

��p = @Hp/@xp �p(tk + T ) = @�p/@xand

u⇤p = argminH(t, x, u,�p)

Finding solutions?

• Assume non-cooperative behaviour: pedestrian p optimises own cost function while begin aware that the opponents will do the same

• Novel iterative approach for mixed initial-terminal boundary condition problem can be applied to small test cases due to computational burden

• Example shows ‘self-organisation’ crossing flow case: intersection without signals?

Can we come up with a simpler model?

• The trick we used: simplify prediction model of pedestrian p • Pedestrian p assumes that opponents do not change their speed or direction

(i.e. ) during the prediction period with • Time-invariant infinite horizon discounted cost problem has closed-form

solution which is similar to the social-forces model of Helbing:

[tk, tk + T )uq = 0 T ! 1

up =

v0p � vp⌧p

�Ap

X

q 6=p

e�dpq/Rp

✓ p + (1� p)

1 + cos ✓pq2

◆npq

p

q

✓pq

vp

npq

Acceleration towards desired velocityPush away from ped q

+ …

Speed-density relation?

• Assume pedestrians walking in straight line• Equilibrium: no acceleration, equal

distances R between pedestrians• We can easily determine equilibrium speed

for pedestrian q (q > p means q is in front)• Speed-density diagram looks reasonable

for positive values of anisotropy factor

21

0 2 4 6density (P/m)

0

0.5

1

1.5

spee

d (m

/s)

0 2 4 6flow (P/s)

0

2

4

6

8

10

spee

d (m

/s)

� = 0

� = 1

� = 0.6

vep = v0p � ⌧pAp

X

q>p

e�(q�p)d/Rp � �X

q<p

e�(p�q)d/R

!

Characteristics of the simplified model • Simple model captures macroscopic characteristics of flows well • Also self-organised phenomena are captured, including dynamic lane formation, formation of diagonal stripes, viscous fingering, etc. • Does model capture ‘faster is slower effect’? • If it does not, what would be needed to include it?

Application of differential game theory: • Pedestrians minimise predicted walking cost, due

to straying from intended path, being too close to others / obstacles and effort, yielding:

• Simplified model is similar to Social Forces model of Helbing

Face validity? • Model results in reasonable macroscopic flow characteristics• What about self-organisation? 22

Characteristics of the simplified model • Simple model captures some key

relations (e.g. speed-density curve) reasonable well!

• Also self-organised phenomena are captured, including dynamic lane formation, formation of diagonal stripes, viscous fingering.

• Self-organisation depends critically on paramaters and variance: freezing by heating

• Do you know other features we have not discussed? Would the model be able to reproduce these?

• Self-organisation of bi-directional lanes fails if demand becomes too high

• Parameters determine threshold

23

• Faster = slower effect states that the faster people try to get out, the longer an evacuation will take • Different experiments showed substantial reduction in outflow when evacuees rush • Capacity reduction is caused by friction / arc-formation in front of door due to increased pressure

Introducing the Faster is Slower effect

• What could you do to reproduce the capacity drop phenomenon / faster = slower effect?

Introducing the Faster is Slower effect

• What could you do to reproduce the capacity drop phenomenon / faster = slower effect?

• Analogy with squash balls:• Pedestrians are ‘compressible circles’• Normal force pushing away the other pedestrian• Tangential friction force increasing with reduction

of distance of pedestrian centers• Capacity decreases / evacuation time increases with

increasing pressure / haste

frictionnormal force

Calibration and validation• Capacity could be reproduced with 4% error

26

empirical

simulation

1 2 3 40

0.2

0.4

0.6


Rela

tive

frequ

ency

Some lessons… • Importance of sensitivity

analysis to see which parameters are most relevant

• Different average parameter values are found when calibrating per situation (bi-dir flow, cross flow, bottleneck)

• Reasonable average parameter set was found for all situations

• Improvement when including explicit delay

• Large differences in microscopic valuesCalibration and validation

Calibration and validation

• Novel estimation technique (based on ML) allowed using the microscopic data from experiments to estimate parameter distribution and correlations

• Table shows results of estimation procedure for illustration purposes: note that for some parameters, variance is substantial as is the correlation!


Fascination with active-mode traffic modelling Amsterdam cycling during rush hour

Versatility of the game-theory approach?

• Modelling cyclist behaviour • Approach is very similar to pedestrian

modelling, however the kinematics of cyclists is different:

• First results of application to cycle flow • Example shows application to mixed

pedestrian / cyclist flow (shared space)29

xp = vp cos↵p yp = vp sin↵p

vp = ap ↵p = !p

Successful shared-space implementation


Shared-space: interaction between bikes and peds… • Area behind

Amsterdam Central Station

• Mix between pedestrians going to / coming from station to ferries and crossing pedestrians

• Self-organisation yields reasonable flow operations

• Modelling using game-theory

Successful shared-space implementation

31

Example simulation results for shared-space situation Shared-space: interaction between bikes and peds… • Area behind

Amsterdam Central Station

• Mix between pedestrians going to / coming from station to ferries and crossing pedestrians

• Modelling using game-theory

• Depending on parameter choices, self-organisation occufs…

Use of game theory for C-ACC systems

• Use optimal control approach to minimise joint cost of vehicles in platoon by controlling few vehicles

• Cost function can reflect efficiency, comfort, emissions:

Follower 2 -Human-driven

vehicle

Follower 1-Cooperative vehicle

Leader – Human-driven

vehicle

s1, Δv1 s2, Δv2

J = J(u[t ,T ) )= c1Jsafety + c2Jefficiency + c3J fuel + ...

Use of game theory for C-ACC systems

• Differences no control, ACC and C-ACC

Autonomous cars? What about bikes?

Applications of game theory to modelling traffic flows Contributions of talk

• Importance of data & examples of typical features of traffic flow • Motivate use of differential game theory • Apply game theory to pedestrian modelling• Show model features and discuss possible further improvements• Calibration and validation issues• Show other applications (cooperative vehicles, bicycle flow and shared space)

Work is based on research contributions from 2002 onward with Winnie Daamen, Mario Campanella, Dorine Duives, Meng Wang, and others!

9th Workshop on the Mathematical Foundations of Traffic @ TU Delft

Co-organised by the TU Delft Transport Institute

& TU Delft Institute for Computational Science and Engineering

Dates soon to follow!36