The Computational Complexityof Finding Nash Equilibria
Edith ElkindIntelligence, Agents, Multimedia group (IAM)
School of Electronics and CS
U. of Southampton
Games and Strategies
• Games: strategic interactions between rational entities
• Solution concepts: what’s going to happen?– dominant strategies– Nash equilibrium– ….
• Can it be computed?– if your computer cannot find it, the market
probably cannot either
Matrix (Normal Form) Games
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
• finite set of players {1, …, n}
• each player has k actions
(pure strategies): 1, …, k
• payoffs of the ith player: Pi: {1, …, k}n → R
Nash Equilibrium
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
• Nash equilibrium: a strategy profile such that
noone wants to deviate given other players’ strategies, i.e., each player’s strategy is a best response to others’ strategies:– (0, 0) and (1, 1) are both NE
Pure vs. Mixed Strategies
1 -1
-1 1
-1 1
1 -1
Row player:
Column player:
H
T
H T H T
H
T
• NE in pure strategies may not exist!– “matching pennies”
• Mixed strategy: a probability distribution over actions– 50% tail, 50% head
Existence of NE
• Theorem (Nash 1951): any n-player k-action game in normal form has an equilibrium in mixed strategies
can we find one in poly-time?
Plan of the Talk
2 players, k actions
n players, 2 actions
other cool stuff
2 (r=const) players, k actions
• Input representation:– 2 players: two k x k matrices– r players: r k x k x … x k matrices
• poly-size for constant r
• Output representation: – for 2 players all NE are in Q– but not for 3 and more players…
• Checking for pure NE: easy – at most k2 strategy profiles
2 players, k actions: mixed NE
• Naïve approaches: exp(k)
• Simplex-like approach (Lemke-Howson algorithm):– works well in practice– exp(k) in the worst case (2004)
• Is it time to give up?– maybe the problem is NP-hard?
Is Finding NE NP-hard?
• Reminder: a problem P is NP-hard if you can reduce 3-SAT to it:– “yes”-instance 3-SAT → “yes”-instance of P– “no”-instance 3-SAT → “no”-instance of P
• Problem: each instance of NASH is a “yes”-instance!– every game has a NE
• Formally: if NASH is NP-hard then NP = coNP• Need: complexity theory for
total search problems
Reducibility Among Search Problems
• S associates x in X with a solution set S(x)• Total search problem: for any x, S(x) is not empty
S: X Y
T: X’ Y’
If T is easy, so is S• S is reducible to T if:
– f, g easy to compute– g(T(f(x))) is in S(x)
f g
Completeness Results?
• Can we prove that any total search problem is reducible to r-NASH?
• Not really: the class T of all total search problems is a semantic class– not known how to find complete problems for these
• Want to pick a large subclass S of T s.t.– S includes some natural problems– there are problems that are complete for S– in particular, r-NASH is complete for S
• Input: Boolean circuits S (Successor), P (Predecessor):– n inputs, n outputs– S(0n) ≠ 0n, P(0n) = 0n
• Output: x ≠ 0n s.t. – S(P(x)) ≠ x or P(S(x)) ≠ x
Intuition: G=(V, E): – V = n; – E = {(x,y) | y=S(x), x=P(y)}
END OF THE LINE
00000
01011
11001
01011
PPAD
• PPAD: Polynomial Parity Argument, Directed version
• PPAD is the class of all search problems that are reducible to END OF THE LINE
search problem solution
circuits S, T “end of the line”f g
r-NASH is in PPAD
• Proof on Nash’s theorem:– existence of NE reduces to Brouwer’s fixpoint
theorem– Brouwer’s fixpoint theorem reduces to
Sperner’s lemma– Sperner’s lemma is proven by a parity
argument (similar to END OF THE LINE)
• Reduction of r-NASH to END OF THE LINE can be extracted from these proofs (Papadimitriou 94)
Brouwer’s Fixpoint Theorem
• Brouwer’s Theorem: Any continuous mapping from the simplex to itself has a fixpoint.
• Nash Brouwer proof sketch:– set of all strategy profiles → simplex– mapping: (s1, …, sn) → (s1+1, …, sn+n),
where i is a shift in the direction of best response to (s1, …, si-1, si+1, …, sn)
– NE is a point where noone wants to deviate, i.e., a fixpoint
• Proper coloring:– vertices on BC are not blue– vertices on AC are not green– vertices on AB are not yellow
• Sperner’s Lemma: there exists a trichromatic triangle
• Brouwer’s theorem Sperner’s Lemma:– x is blue if the grad(F) at x points away from A, etc.– trichromatic triangle “has no direction” – repeat at increased resolution…
Sperner’s Lemma
A
B
C
Reductions (Papadimitriou 1994)
END OF THE LINE is PPAD-complete
TRICHROMATIC TRIANGLE is PPAD-complete
3D-BROUWER is PPAD-complete
r-NASH is in PPAD
r-NASH vs 3D BROUWER
• Existence of NE follows from Brouwer’s fixpoint theorem
• NE are special cases of Brouwer’s fixpoints– just how special?
• Can any fixpoint be represented as a NE of a game?– Is there a reduction
from 3D BROUWER to r-NASH?
Hardness Reductions: the Timeline
• 3D-BROUWER is PPAD-complete (Papadimitriou, 1994)
• 4-NASH is PPAD-complete (Daskalakis,Goldberg, Papadimitriou, Sep 2005)
• 3-NASH is PPAD-complete (Daskalakis, Papadimitriou, Oct 2005, Chen, Deng, Nov 2005)
• 2-NASH is PPAD-complete !!! (Chen, Deng, Dec 2005)
n players, 2 actions
• representation: payoffs to each player for every action profile (vector in {0, 1}n): n2n numbers
• graphical games:– players are vertices of a graph– V’s payoff depends on actions of W in N(V) U V– n players, max degree d => n2d+1 numbers
TU
V
W t=0, u=0, v=0, w=0: 12t=1, u=0, v=0, w=0: 31 ….t=1, u=1, v=1, w=1: -6
W’s payoffs(16 cases):
Algorithms: What Was Known
• Bounded-degree trees:– Exp-time algorithm/poly-time approximation
algorithm to find all NE (Kearns, Littman, Singh, UAI 2001)
– ??? poly-time algorithm to find a single NE (Kearns, Littman, Singh, NIPS’2001)
• Heuristics for graphs with cycles
Our Results (E., Goldberg, Goldberg’06)
• Algorithm in NIPS’01 paper is incorrect (does not always output a NE)
• We fix the NIPS’01 algorithm, but…– our algorithm runs in poly-time on paths– with a trick, also on cycles
• There is a graph of pathwidth 2 on which our algorithm runs in exp time– true for all algorithms that use the basic
approach of the UAI’01 paper
Warm-up: 2-player 2-action games
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
Suppose R plays 1 w.p. r
EP(C) from playing 0: (1-r)*1
EP(C) from playing 1: r*3
1-r > 3r iff r < ¼
Suppose C plays 1 w.p. c
EP(R) from playing 0: (1-c)*2
EP(R) from playing 1: c*1
(1-c)*2 > c iff c < 2/3
1/4 1
r
BR(C)c1
2/3BR(R)
mixed NE: r=1/4, c=2/3
• Potential best response: v is a PBR to w iff when W plays w, there is a NE for TV in which V plays v.
• upstream pass: construct PBRV(w) from PBRU1(v), PBRU2(v) and PBRU3(v)
• downstream pass: root selects its strategy based on the children’s PBR’s; propagates to leaves
Algorithm for Trees (KLS’01)
TV
W
V
U1U2
U3
v
w
Computing PBR on a Path
• E0 = EP(V) from playing 0: (1-u)(1-w)*v000+(1-u)w*v001+u(1-w)*v100+uw*v101 = auw+bu+cw+d
• E1 = EP(V) from playing 1: (1-u)(1-w)*v010+(1-u)w*v011+u(1-w)*v110+uw*v111 = a’uw+b’u+c’w+d’
• E0 = E1 iff w = (Au+B)/(Cu+D) = f(u)
U V W
.5 1
1u
v
v
1
1
.5
.1 .9 w
(v, u) → (f(u), v)
PBRU(v) PBRV(w)
Trees: too many segments
v v w
ut v
v1 v2 v1 v2
v1
v2
KLS (NIPS’01): can “trim” PBR
Incorrect!
W
V
T U
(v,t), (v,u) → (f(u,t), v)
u2
u1
t2
t1
Solutions?
• Solution 1 (for paths): algorithm of UAI’01 paper, careful analysis– the number of segments/rectangles in each
PBR is O(n2)– running time O(n3)
• Solution 2 (for paths): can pick a subset of each PBR consisting of O(n) segments– O(n2) running time
Extension to trees? V0 V1 V2 Vn-1 Vn
U1
T1
Un-1
T2
U2
Tn-1 Tn
Un
Graphical games: hardness results
• NP-hard?– no: total search problem
• PPAD-hard?– yes!– in fact, this is how the hardness result for
4-player games was obtained (Goldberg, Papadimitriou, Aug 2005)
Equivalences: GP’05r-player game G NE of G
deg 3 graphical game G’ NE of G’
f g
d2-player game G’ NE of G’
deg d graphical game G NE of G
f g
Combining Reductions: GP’05
r-player game G NE of G
9-player game G’ NE of G’
f g
Finding NE in a 4-player game is as hard as
finding NE in a r-player game for any constant r
X4
PPAD-hardness: missing details
• 3D-Brouwer is PPAD-complete (Papadimitriou, 1994)
• 4-NASH is as hard as deg 3-GG (Goldberg, Papadimitriou, Aug 2005)
• deg 3-GG is PPAD-complete and hence 4-NASH is PPAD-complete
(Daskalakis,Goldberg, Papadimitriou, Sep 2005) • 3-NASH is PPAD-complete
(Daskalakis, Papadimitriou, Oct 2005, Chen, Deng, Nov 2005)
• 2-NASH is PPAD-complete !!! (Chen, Deng, Dec 2005)
NE with special properties
Pure NE:
• easy for constant number of players
• NP-hard for general graphical games – even if max degree = 3– NP vs. PPAD: pure NE may not exist!
• poly-time on trees (KLS algorithm)– also on graphs with bounded treewidth
Welfare-Maximizing NE
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
Nash equilibria: • (0, 0): total payoff is 3• (1, 1): total payoff is 4• (1/4, 2/3): total payoff is 17/12
not all NE are created equal…
Algorithms for Good NE
• 2-player games: checking for NE with total payoff > T is NP-hard (Gilboa Zemel 89, Conitzer, Sandholm 03)
• Graphical games: for any algebraic , deg() = n, there is a GG with int payoffs on a path of length O(n) in which in the best NE player 1 plays approximation algorithms for any (E., Goldberg, Goldberg 07)
Approximate NE
• -Nash equilibrium: a strategy profile such that noone can gain > by deviating
• Graphical games on trees: poly-time algorithms for any (KLS’01)
• 2-player games ( utilities in [0, 1] ): – PPAD-complete for =O(1/n)– Approximation for constant :
• 0.5 WINE’06 (Dec 2006)• 0.382 ( =1-1/) ACM EC’07 (June 2007)• 0.364 WINE’07 (Dec 2007)• 0.339 WINE’07 (Dec 2007)
Conclusions
• Computational aspects of game-theoretic questions are crucial
• Lots of cool open problems– computing NE in graphical games on trees– finding -Nash in 2-player games for small
• A rich set of techniques
• Talk to me if you want to know more…
Mixed strategies and payoffs
• Payoff matrices:
• the row player plays a = (a1, …, an)• the column player plays b = (b1, …, bn)• expected payoff of R when playing i: (Ri, *, b)• expected payoff of C when playing i: (C*, j, a)
R11 R12 … R1n
R21 R22 … R2n
…Rn1 Rn2 … Rnn
C11 C12 … C1n
C21 C22 … C2n
…Cn1 Cn2 … Cnn
R: C:
• if 1st player’s strategy a supported on I N ai ≠ 0 iff i I
2nd player’s strategy b supported on J N bj ≠ 0 iff j J
• then I BR(b): (b, Ri, *) ≥ (b, Rk, *) for all i I, k N
J BR(a): (a, C*, j) ≥ (a, C*, k) for all j J, k N– LP on variables a1, …, an, b1, …, bn
– solutions to LP ↔ Nash equilibria
• running time: 22kpoly(k)
2 players, k actions: support guessing
linear inequalities!
Reminder: 2-player 2-action games
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
Suppose R plays 1 w.p. r
EP(C) from playing 0: (1-r)*1
EP(C) from playing 1: r*3
1-r > 3r iff r < ¼
Suppose C plays 1 w.p. c
EP(R) from playing 0: (1-c)*2
EP(R) from playing 1: c*1
(1-c)*2 > c iff c < 2/3
1/4 1
r
BR(C)c1
2/3BR(R)
mixed NE: r=1/4, c=2/3
Computing PBR on a path
• f(u) = (au+b)/(cu+d)• a, b, c, d are determined by V’s payoffs
U V W
.5 1
1u
v
v
1
1
.5
.1 .9 w
(v, u) → (f(u), v) + “tails”
PBRU(v) PBRV(w)