applying the repeated game framework to multiparty networked applications mike afergan july 22, 2005...
TRANSCRIPT
Applying the Repeated Game Framework to Multiparty Networked Applications
Mike AferganJuly 22, 2005
Joint work with Dave Clark, Rahul Sami and John Wroclawski
My Thesis
Repeated games can be an important and practical tool for the design of networked
applications.
3 of 59
Talk Overview
Fundamental Motivations Background on Repeated Games Example: Incentive-Based Routing Research Overview and Concluding
Thoughts
4 of 59
Initial Assumptions
Networked applications are important
Incentives are a concern for a large class of networked applications.
Routing Peer-to-Peer
Network application developers need tools to build systems robust to user incentives.
Properties Fundamental to Networked ApplicationsProperty #1: Multiple interacting self-interested
parties Direct communication or shared network Motivates the use of game theory
Property #2: Interactions are repeated. Causal relationship between one time period and
the next Examples:
ISPs in near identical BGP sessions Users in similar interactions with similar users (e.g.,
web, wireless, P2P)
Suggests that the repeated context should be considered to use game theory effectively.
Repeated Games are Important Repeated games are a well-studied area
of game theory. The outcome of the repeated game can
significantly differ from the outcome of the one-shot game.
This research is the first to consider repeated games as a tool for networked applications.
However, most relevant prior work considers only the one-shot game.
7 of 59
A Practical Fit
Importantly, in each example we derive practical results
These practical results stem from further relationships between networked applications and repeated games.
Networked
Applications
RepeatedGame Theory
8 of 59
Repeated Games are Practical
Property #3: Networked applications face multiple constraints
Example Constraints: Need to realize system objectives Cost, privacy, shared network
Impact of Constraints: May not be able to realize a one-shot solution Provides explanation for real-world
phenomena
Repeated games work well with practical models of networked applications.
9 of 59
Repeated Games are Practical
Property #4: Actions in Networked Applications Are Highly Parameterized
Parameter value is important More interestingly, parameter granularity
is also important In repeated games, the granularity of the
action qualitatively impacts the equilibrium The freedom permitted can be a first order
concern
MultipleParties
Properties Fundamental to Networked Applications
These four properties apply to a large class of networked applications
Repeated games are an important and practical tool for the design of networked applications.
Repeated Dynamics
Constraints Parameterized
Repeated games are important and practical
Areas of Contribution
Exposition of Thesis Introduce the concept of using repeated games Demonstration of a fundamental relationship
between repeated games and networked applications
Present approaches and techniques
Application to Important Networked Problems1. Inter-ISP Relationships with User-Directed Routing
(Chapter 3)
2. Design of Incentive-Based Routing Systems (Chapter 4)
3. Application-Layer Multicast Overlays (Chapter 5)
Later in this talk, I will present #2 in depth.
12 of 59
Talk Overview
Fundamental Motivations Background on Repeated Games Example: Incentive-Based Routing Research Overview and Concluding
Thoughts
One-Shot Prisoner's Dilemma
P2 Cooperate Defect
Cooperate (5,5) (0,9)
Defect (9,0) (1,1)
Static EquilibriumOutcome
In the one-shot game, (D,D) is the outcome of the unique Nash Equilibrium.
P1
Repeated Prisoner's Dilemma
$$$ or $+$+$+ $+ $ + S
Key Takeaway: The equilibrium of the repeated game may differ from the equilibrium of the
corresponding one-shot game.
Example Strategy: 1. Play C 2. If the other player defects, play D forever
Outcome ofthe RepeatedGame
P2 Cooperate Defect
Cooperate (5,5) (0,9)
Defect (9,0) (1,1)
P1
15 of 59
Sample Analysis
$$$ or $+$+$+ $+ $ + S
Parameterized by discount factor () Patience Factor (infinite game) Probability of game ending (finite game with unknown horizon)
Example: Strategy is an equilibrium of the game iff: (Playing forever) (One-time “defect”) + (Resulting
payoffs)
“Play C forever. If other plays D, play D forever” is an equilibrium iff:
10
)1(95t
t
t
t ½
P2 Cooperate Defect
Cooperate (5,5) (0,9)
Defect (9,0) (1,1)
P1
Repeated Equilibria Under General Conditions
“Folk theorem” results show the feasibility of a large set of potential outcome payoffs
Repeated equilibria feasible under a variety of practical assumptions: Imperfect Information [Green-Porter ’84, Fudenberg-
Levine-Maskin ’94] Players of different horizons [Fudenberg-Levine ’94]
Anonymous random matching [Ellison ’93]
In practice, this means many repeated outcomes are possible under a broad class of restrictions.
17 of 59
Talk Overview High Level Argument Background on Repeated Games Specific Example: Incentive-Based Routing
Problem Overview The Problem of Repeated Dynamics Finding Key Protocol Parameters Generalizing the Results Summary
Research Overview and Concluding Thoughts
18 of 59
The ContextIncentive-Based Interdomain Routing
Architecture Overview Routes as goods Applied specifically and deployed incrementally
Well-motivated by: Economic realities of today’s Internet Increasingly prevalent technology (User-Directed Routing)
[A., Wroclawski ’04] This talk does not defend such an architecture.
s t
A
B
PriceC
PriceA
C
PriceB
Protocol Design Question We consider a single
competitive interchange
Our Question: How should one design a protocol for conveying pricing information for routes?
Protocol Designer Does Control Protocol Designer Does Not Control
Protocol period (time between updates)
Number of networks
Unit of Measure (Mbps vs. MBps)
Network Cost
Width of protocol fields (number of bits)
Strategies used by Networks
s t
Our Analytical Framework:Repeated Games
1. Routing is inherently a repeated process
2. The outcome of the repeated game can differ qualitatively from that of the one-shot game
Our research is the first to consider routing as a repeated game.
21 of 59
Our ContributionsPractical Conclusions
Although routing is repeated, important properties of prior models do not hold in the repeated setting.
We find newfound importance for several parameters1.The length of the protocol period2.The granularity of the unit-of-measure
(e.g., Mbps, MBps, or Gbps)3.The width of the price field
These provide practical insight for protocol designers.
It is possible to upper-bound prices using these parameters.This helps designers (to the extent desired) control the uncertainty presented by the repeated game.
22 of 59
Talk Overview High Level Argument Background on Repeated Games Specific Example: Incentive-Based Routing
Problem Overview The Problem of Repeated Dynamics Finding Key Protocol Parameters Generalizing the Results Summary
Research Overview and Concluding Thoughts
23 of 59
Problem of Repeated Routing
An interconnect is A repeated game Between a small number of players (ISPs)
The repeated game may cause artificially higher prices
Standard pricing technique: Strategyproof Mechanisms
Truthtelling is at least as good as any other strategy Benefits: Reduced strategizing and potential oscillation Standard mechanism: Vickrey-Clark-Groves (VCG) Feigenbaum, Papadimitriou, Sami, and Shenker (FPSS ’02)
show how to apply this to an Internet-like network efficiently
s t
24 of 59
Applying VCG to a Network [FPSS ’02]
1
10
10
11
1t2
t1
A
Bs
Each node, i, on the Least Cost Path (LCP) paid: pi = (LCP avoiding i) – LCP + ci
Applying VCG to a Network [FPSS ’02]
1
10
10
11
1t2
t1
A
Bs
Each node, i, on the Least Cost Path (LCP) paid: pi = (LCP avoiding i) – (LCP) + ci
Example: s -> t1: A is paid (10 + 1) – (1 + 1) + 1 = 10 s -> t2: B is paid (10 + 1) – (1 + 1) + 1 = 10
In the one-shot game, this is strategyproof.
26 of 59
The Repeated Version
In the repeated game A and B could both bid 20: A is paid (10 + 20) – (1 + 20) + 20 = 29 B is paid (10 + 20) – (1 + 20) + 20 = 29
1
10
10
11
1t2
t1
A
Bs
Conclusion #1: Although Internet routing is a repeated setting, the VCG mechanism (and thus the FPSS implementation) is not strategyproof in the repeated routing game.
27 of 59
Questions
1. What determines the equilibrium price?
2. What can be done to control, bound, or influence prices (if so desirable)?
28 of 59
Talk Overview High Level Argument Background on Repeated Games Specific Example: Incentive-Based Routing
Problem Overview The Problem of Repeated Dynamics Finding Key Protocol Parameters Generalizing the Results Summary
Research Overview and Concluding Thoughts
A Full Model of Routing
We: Prove that particular parameters may significantly impact price Formally analyze that impact (by looking at the derivatives)
given a model with: Repeated interactions Asynchronous interactions Heterogeneous networks Multi-hop paths and multiple destinations Confluent (BGP-like) routing Large class of strategies
This talk focuses on a simple model: Repeated Incentive Routing Game (RIRG)
Intuition and analysis is similar for more general models Will later briefly discuss generalizations (more details in thesis)
30 of 59
Repeated Incentive Routing Game (RIRG): Topology
A particular interchange: Single Source Single Destination Multiple homogenous networks offering connectivity Networks compete for traffic on price (Bertrand
competition) Route is the market good
s…
t
Direction of Traffic
Strategic Player
31 of 59
RIRG: Key Assumptions
Key Assumption #1: The game is played via a networked protocol.
Protocol runs in a series of synchronized rounds (of length d)
There is a minimum bid granularity size (b).
Key Assumption #2: The game is not infinite. Players only know length in expectation (D) Note: D and d define : = 1- d/D
Additional Assumptions that can be Relaxed Traffic is fixed Networks have fixed per unit cost Networks have infinite capacity Minimum bid becomes common knowledge Traffic is splittable
FPSS-like network
RIRG: Play of the Game
In each round:1. All N players announce their bids2. Traffic is evenly split among the
provider(s) with the lowest price3. Provider is paid for the volume of
traffic at the price bid (1st price auction)
Key Decision: In each round, each network can either:1. Try to be the low-price provider2. Split the market with other firms at a higher price
33 of 59
Equilibrium Notion The potential strategy space is quite large
An equilibrium notion refines the strategy space Subgame perfect equilibrium (SPE) is natural and
standard for repeated games
A strategy is subgame perfect if i) is a Nash equilibrium for the entire
game and ii) is a Nash equilibrium for each subgame.
34 of 59
Price Matching For the purposes of this talk, I will focus on Price
Matching (PM) Strategies Informally: “Bid the lowest price seen in the prior period” Results generalize, for example:
“Match price and then raise later” “Punish by doubling initial deviation”
Price Matching Strategy:
1. At t0, offer p*
2. For all t>t0, pi =
p* is the largest p such that PM is SPE
1min,max t
jjpc
Defining Price MatchingSolving for p*
Term Meaning
(pi, p-i) Profit function
Period probability of game ending (discount factor)
One Stage Deviation Principle (Abridged): is subgame perfect if and only if no player can gain by deviating from in a single stage and conforming to thereafter.
(1)
11
,,,t
it
it
it bpbppbppp
bpbppbppp iii ,,1, (2)
p* is the maximum p such that the inequality holds.
Solving for Equilibrium
)1)(1(
N
NNbp
bpbpNp 1
(3)
Variable Meaning
b Minimum bid size
N Number of players
Period probability of game ending (discount factor)
N
bpTbpT
N
Tp 1
(4)
Theorem: In the RIRG, the unique equilibrium price from Price Matching is:
bpbppbppp iii ,,1, (2’)
Deriving Practical Intuition
Theorem: When playing Price Matching:
where d is the length of the protocol period.
0d
p
Conclusion #2: A longer period may lead to lower prices
“A longer period may lead to lower prices”
$$$ or $+$+$+ $+ $ + S
Lowering price leads to:
Big payoff now
Higher payoffs later
Period of protocol1sec 1 month
$ $$$ $
39 of 59
“A longer period may lead to lower prices”
Longer protocol period
More benefit to deviating
Lower prices
Longer time before competitors react
More Practical Intuition Theorem: When playing PM:
where b is the minimum bid size. “Minimum bid size” is not a protocol
parameter. But:
Unit-of-measure (Megabits, Megabytes, Terabits) Width of price field (number of bits in protocol)are protocol parameters
0b
p
Conclusion #3: A wider price field and a more granular unit of measure may reduce price.
Profit Margin vs Delta (N=2)
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1Delta
Pro
fit M
argi
n
b=0.01
b=0.05
b=0.1
Sensitivity to Parameters
Observations:1. Sensitivity to delta is large, especially in the relevant range
2. Impact of b is qualitative, not just precision
Result Summary
Example Takeaways:1) Using Megabytes instead of Megabits can lead to lower
prices.2) A system that runs faster may lead to higher prices.
As [Variable] Increases… Prices
# of players Decreases
Width of price field Increases
Unit-of-Measure Granularity
Decreases
Protocol period Decreases
Topology Stability Increases
A priori, some of these parameters seem benign or at most only having impact as “rounding error.”
43 of 59
Constraining Prices Sensitivity to parameters means:
This insight must be considered They can help “solve the problem” of the
repeated dynamics (to the extent desirable)
Theorem: For all >0, there exists protocol parameter settings such that pR
pS + , where: pR is the equilibrium price in the repeated
game pS is the equilibrium price of the stage game.
44 of 59
Talk Overview High Level Argument Background on Repeated Games Specific Example: Incentive-Based Routing
Problem Overview The Problem of Repeated Dynamics Finding Key Protocol Parameters Generalizing the Results
Generalizing the Strategy Space Generalizing the Game
Multiple Destinations and Confluent Flows Heterogeneous Costs
Summary Research Overview and Concluding Thoughts
45 of 59
Proportional Punishment (PP) Strategies Price Matching has two weaknesses:
1. Prices never rise2. Punishment limited to matching price
Proportional Punishment Strategies are SPE Punishment is bound by some constant k Class is very large (perhaps too large)
If is a one-stage deviation from h when playing at t0, then for PPk iff:
'0ˆˆ pk t
h
t
h
th
h
46 of 59
Visualizing PPk
Time
Pricep
p’
p-k(p-p’)
Price Matching
Match then Raise
Punish by Doubling
t0
47 of 59
Analyzing PPk
Theorem 3: For any PPk, the maximal price obtained by is bound by
Further, this bound is tight.
Other results follow similar to the simple Price Matching case Impact of b and Bounds on pR
)1)(1(
N
kNbp
48 of 59
A More General ModelMultiple Destinations and Confluent Flows
Multiple Destinations Multiple goods, multiple markets Provides for cooperation even with
confluent flows
c
c
t2
t1A
Bs
A wins traffic for t1
B wins traffic for t2
49 of 59
A More General ModelHeterogeneous Networks
Assume c > c’ Potential for a repeated equilibrium at p*(c’) Requires that |c – c’| is sufficiently small
Equilibria may involve only a subset of the N players Does not necessarily imply repeated equilibria
More general graph presents more options A robust protocol must consider such conditions
c’
c
t2
t1A
Bs
A wins traffic for t1
B wins traffic for t2
50 of 59
Talk Overview High Level Argument Background on Repeated Games Specific Example: Incentive-Based Routing
Problem Overview The Problem of Repeated Dynamics Finding Key Protocol Parameters Generalizing the Results Summary
Research Overview and Concluding Thoughts
51 of 59
Summary1. The repeated setting is a vitally important
setting to consider.2. Our analysis provides insight into the
importance of several protocol parameters3. These parameters are:
Under the control of the protocol designer Unavoidable
4. Consideration of these parameters can help build a robust system
5. Suggests that repeated game analysis can be important and practical
52 of 59
Talk Overview
Fundamental Motivations Background on Repeated Games Example: Incentive-Based Routing Research Overview and Concluding
Thoughts
53 of 59
Benefits and Feasibility of Incentive Based Routing (Chapter 3)
Problem: User-directed routing (e.g., overlays) transforms inter-domain routing into a meaningfully repeated game
Sample Contributions: Exposition of the problem Consideration of principles for why and how incentives (i.e.,
prices) should be integrated to various routing architectures
Traffic PatternsBusiness
RelationshipsTraffic Policies
54 of 59
Application-Layer Multicast Overlays (Chapter 5)
Problem: Selfish users can degrade system performance
Contribution: A repeated model of cooperation Contribution: Use model and simulation to descry
practical techniques and parameters that can aid in building more robust systems
……
Selfish Nodes able to alter the topologyFaithful Nodes create an efficient tree
……
55 of 59
Meaningful Themes
For each problem considered: The repeated dynamic plays a vital
role in defining the system equilibrium
Our model is the first to capture the repeated dynamic
We are able to derive practical insight into how to build more robust systems.
56 of 59
Exogenous Types vs Endogenous Motivations
Some models use exogenous types: Network type: business relationships
(e.g, [GaoRexford00])
Node type: cheater/not [Mathy et al ‘04], generosity parameter [Feldman et al ‘04]
Repeated game models can capture these factors in an endogenous fashion
57 of 59
Concluding Thoughts The repeated dynamic must be considered
in modeling networked applications.
Repeated Games can provide practical results
Relevance of repeated games stems from properties fundamental to networked applications
58 of 59
Thank you for coming!
Questions?
Thesis (and slides) will be available athttp://www.mit.edu/~afergan/thesis/