Game DynamicsOut of Sync

Michael Schapira(Yale University and UC Berkeley)

Joint work withAaron D. Jaggard

and Rebecca N. Wright

Motivation: Internet Routing

Establish routes between Autonomous Systems (ASes).

Currently handled by the Border Gateway Protocol (BGP).

AT&T

Qwest

Comcast

Sprint

Internet Routing as a Game[Levin-S-Zohar]

• Internet routing is a game!– players = ASes – players’ types = preferences over routes– strategies = outgoing edges

• BGP = Best-Response Dynamics– each AS constantly selects its best

available route to each destination– … until a “stable state” (= PNE) is reached.

But…

• Challenge I: No synchronization ofplayers’ actions– players can best-reply simultaneously.– players can best-reply based on outdated

information.

• Challenge II: Are players incentivized to follow best-response dynamics?– Can a player benefit from not best-replying?

this talk

[Nisan-S-Valiant-Zohar]

Game Dynamics and Asynchrony

• Dynamic environments– Internet protocols– large-scale markets– social networks– multi-processor computer architectures

• Game theory provides useful tools to analyze these interactions, but….

• … has so far primarily concentrated on synchronous environments (simultaneous, sequential).

2,1 0,0

1,20,0

RowPlayer

ColumnPlayer

Illustration

Illustration

2,1 0,0

1,20,0

RowPlayer

ColumnPlayer

But…

2,1 0,0

1,20,0

RowPlayer

ColumnPlayer

•Model for asynchronous game dynamics

• Impossibility result

•Circumventing our impossibility result

•Complexity of asynchronous game dynamics

•Directions for future research

Agenda

• n nodes 1,…,n

• Node i has action space Ai

– A=A1•…•An

– A-i=A1•…•Ai-1•Ai+1•…•An

• Node i has reaction function fi:A→Ai

– f=(f1,…,fn)

Simple Model: Nodes Interacting

• Infinite sequence of discrete time steps t=1,…

• Initial state a0, Schedule :{1,…} →2[n]

– fair schedule

• The (a0,)-dynamics– Start at the initial state a0

– In each time step t let the nodes in (t) react.

Simple Model: Dynamics

•Defn: an action profile a=(a1,…,an) is a stable state if fi(a)=ai for all i.– that is, a is a fixed point of f.

•Defn: The system is convergent if the (a0,)-dynamics converges to a stable state for all choices of a0 and (fair) .

Simple Model: Convergence

• Defn: f is node independent if, for each node i, fi:A-

i→Ai

• Thm: If f is node independent, and there exist multiple stable states, then the system is not convergent.

• Can be generalized to reaction functions that are– randomized– bounded-recall– non-stationary

Guaranteed Convergence?

• Internet protocols– Internet routing [Sami-S-Zohar]– congestion control [Godfrey-S-Zohar-Shenker]

• Best-response dynamics– with consistent tie-breaking– orthogonal to the results of Hart and Mas-Colell

• Diffusion of technologies in social networks– 2 technologies {A,B}. Each node wants to be consistent with the majority of

its neighbours.

• Circuit design

Applications

• Example 1: (node-dependent reactions)Each fi is such that for every a=(a1,…,an) it holds that fi(a)=ai.

“Tightness” of Our Result

• Example 1: (node dependent reactions)Each fi is such that for every a=(a1,…,an) it holds that fi(a)=ai.

• Example 2: (unbounded recall)– 2 nodes, 1 and 2, each with action space {a,b}. – Node 2 wants to match node 1’s action.– Node 1 selects b if node 2 changed its action from a

to b in the past, and a otherwise.– What happens at the initial state (b,a)?

“Tightness” of Our Result

• Thm: If f is node independent, andthere exist multiple stable states, thenthe system is not convergent.

• Interesting connections to fundamental results in distributed computing theory.– the Fischer-Lynch-Patterson impossibility result for

consensus protocols (1983)

• But, neither result is a special case of the other.

Proving Our Result

The Dynamics Graph

action vector aS=(aS1,… aS

n)knowledge vector bS=(bS

1,…

bSn)

StateR

knowledge transition

i-transition

StateT

StateS

1. aT:=aS

2. bT:=aS

1. aR:=aS except aR

i:=fi(bS)

2. bR:=bS

• The dynamics graph captures all dynamics.

• The scenario where– the initial state is a0.– nodes 1 and 3 react simultaneously.– then nodes 2 and 3 react simultaneously.

is captured as follows:

Visualising Dynamics

• The dynamics graph captures all dynamics.

• The scenario where– the initial state is a0.– nodes 1 and 3 react simultaneously.– then nodes 2 and 3 react simultaneously.

is captured as follows:

Visualising Dynamics

State SaS=bS=a0

• The dynamics graph captures all dynamics.

• The scenario where– the initial state is a0.– nodes 1 and 3 react simultaneously.– then nodes 2 and 3 react simultaneously.

is captured as follows:

Visualising Dynamics

State SaS=bS=a0

1-transition 3-transition k-transition

• The dynamics graph captures all dynamics.

• The scenario where– the initial state is a0.– nodes 1 and 3 react simultaneously.– then nodes 2 and 3 react simultaneously.

is captured as follows:

Visualising Dynamics

State SaS=bS=a0

1-transition 3-transition k-transition 2-transition 3-transition k-transition

• Defn: A state S in the dynamics graph is stable if every outgoing edge from S leads to S.

• Defn: A fair path in the dynamics graph is a path that (1) for each i, contains an i-transition; and (2) also contains a knowledge transition.

Stability and Fairness

• Defn: The attractor region of a stable state S are all states from which any (long enough) fair path reaches S.

Attractor Regions

• Claim: A fair cycle in the dynamics graph implies an oscillation in our model.

• Proposition: If there are multiple stable states then there are states in the dynamics graph that are not in any attractor region (“neutral states”).

Proof Sketch (Cont.)

• Colour each

attractor region in a different colour – red, blue, etc.

• Colour the neutral states in purple.

Colouring States

•Key idea: We show that from every purple state there is a fair path that leads to another purple state.

•The number of purple states is finite and so this implies a fair cycle.

Creating Oscillations

• Lemma: There cannot be two edges leading from a purple state to two non-purple states that do not have the same colour.

• Intuition: We can swap the order of activations without affecting the outcome.

Proof Sketch (Cont.)

?

: different transitions

• Fix a purple state p.

• Let R be a “maximal” fair path from p to another purple state.

Proof Sketch (Cont.)

p ……

q

R

• Let be a transition that is not on R.

• Observe that at q takes us to a non-purple state.

p ……

q

R

Proof Sketch (Cont.)

• Because q is purple it must have a fair path to a non-purple non-red state.

p ……

q

R

……

u

Proof Sketch (Cont.)

• Now, we prove that at u must take us to a red state --- a contradiction!

p ……

q

R

……

u

Proof Sketch (Cont.)

• Our result holds for randomized reaction functions.– adversarially-chosen schedule

• What if the schedule is randomized?– our impossibility result breaks …– … but no general possibility result either

Circumventing Our Impossibility Result: Randomness

• Defn: A schedule is r-fair if each node is activated at least once within every r consecutive time steps.

• Can we prove our impossibility result for schedules that are r-fair? If so, for what values of r?

• We present positive and negative results.

Circumventing Our Impossibility Result: r-Fair Schedules

•Thm: Determining if a system with n nodes, each with two actions, is convergent requires exponential communication (in n).

• The proof requires reaction functions to be of exponential size.

• Combinatorial proof: a “Snake in the Box” construction

Complexity Results

• What if the reaction functions can be succinctly described?

•Thm: Determining if a system with n nodes is convergent is PSPACE-Complete.

• Hence, there is no “short” characterization of asynchronous convergence!

Complexity Results

•Other notions of asynchrony

•Other reaction functions– fictitious play, regret minimization– Observation: regret minimization is much more

resilient to asynchrony (different framework…).

•Other restrictions on schedules– random schedules– r-fair schedules– more

Directions for Future Research

THANK YOU