microeconomics 2. game theory · description backward induction imperfect info mixed and behav. str...

Microeconomics - 2.2 Extensive form games

Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications

Microeconomics

2. Game Theory

Alex Gershkov

http://www.econ2.uni-bonn.de/gershkov/gershkov.htm

18. November 2008

1 / 36



Dynamic games

Time permitting we will cover

2.a Describing a game in extensive form (efg)

2.b Imperfect information

2.c Mixed and behavioural strategies; Kuhn’s Theorem

2.d Nash equilibrium

2.e Credible strategies and Subgame Perfection

2.f Backward induction

2.g Applications

2 / 36



2.a Describing a game in Extensive form

We want to capture dynamic aspects of a game where timing isimportant. Consider the following game

1. 2 parties {P1,P2} are trying to share two indivisible units ofa good yielding positive utility (say one util each)

2. suppose that P1 makes a take-it-or-leave-it offer to P2

3. P2—after having observed this offer—decides whether toaccept or reject this offer

4. if no agreement is reached (ie. if P2 rejects the offer), bothP1 and P2 get nothing.

A game’s Extensive form (efg) is a description of the sequentialstructure of dynamic games such as above.

3 / 36




Features:

A1 moves occur in sequence,

A2 all previous moves are observed before a move is chosen, and

A3 payoffs and structure of the game are common knowledge.

DefinitionGames satisfying (A1–A3) are called finite efg ofperfect information.

4 / 36




The efg of a perfect information game consists of

E1. the set of players N

E2. the set of sequences (finite or infinite) H, that satisfies thefollowing three properties:

0.1 ∅ ∈ H0.2 if {ak}K

k=1∈ H and K > L, then {ak}L

k=1∈ H for every L

0.3 if an infinite sequence {ak}∞k=1 satisfies {ak}Lk=1 ∈ H for all

finite L, then {ak}∞k=1∈ H.

5 / 36




Each member of H is a history; each component of a history ak isan action taken by a player. A history {al}k

l=1is terminal if it is

infinite or if there is no aK+1 such that {al}K+1

l=1∈ H. The set of

terminal histories is denoted Z .

E3. a player fn P : H−Z → N where P(h) is the label of theplayer who is supposed to choose an action after history h ∈ H

E4. for each player i ∈ N, a payoff fn ui : Z → R.

6 / 36




DefinitionA game is called finite if the number of stages is finite and thenumber of feasible actions at any stage is finite.

We denote, for every h ∈ H−Z such that P(h) = i , the set ofactions available to i by Ai (h) = {si |(h, si ) ∈ H}.

DefinitionA perfect information efg Γ consists of Γ = {N,H,P , u}.

7 / 36



2.a Describing a game in Extensive formTypically, an efg of perfect info can be described by the ”gametree”The game tree T consisting of the set of nodes (including decisionand terminal nodes) and the branches which are directedconnections between nodes; T must satisfy the ‘tree conditions’

◮ there is one node without incoming branches called the ‘initialnode’ (the open circle)

◮ for any given node, there is a unique path connecting it to theinitial node.

◮ any non-terminal node corresponds to the player that choosesthe action there.

◮ Each branch from a node corresponds to an action a availableto the player at this info set.

◮ Payoffs are given for all i ∈ N at the terminal nodes

The tree captures the temporal structure of how events unfold overtime. 8 / 36




The key definition of the efg is the strategy. It is a complete,contingent plan of action specifying a choice for the concernedplayer at every possible history.

DefinitionA strategy for player i ∈ N is a sequence si = {si (h)}h∈Hi

, whereHi = {h|h ∈ H −Z and P(h) = i} and si (h) ∈ Ai(h).

9 / 36



2.g Backward induction

Finite games satisfying (A1–A3) can be solved byBackward Induction. Roughly speaking this is

◮ replacing each choice set which only leads to terminal nodesin Z by the corresponding NE’qm outcome

◮ notice that this creates a new set of terminal nodes Z ′ in the‘shortened’ game

◮ now again replace all choice sets which only lead to terminalnodes in Z ′ by the corresponding NE’qm outcome

◮ repeat the above until the game is reduced to the initial nodeand a set of choices leading to terminal nodes only; theNE’qm outcome of this game is the backward inductionoutcome of the original game.

10 / 36



2.g Backward induction

TheoremAny perfect information game with a finite number of strategiesand players can be solved backwards and therefore has a purestrategy eq’m.

11 / 36




To introduce information imperfections we require the followingadditions to the perfect information requirements (E1–E4)

E3.’ the opportunity for chance moves by the additional playerNature (N)

E3.” a players fn P : H−Z → 2N assigns to each nonterminalhistory a set of players

E4.’ we assume that the ui : Z → R satisfy the axioms of vN-Mexpected utility theory

E4.” for every h ∈ H −Z such that N∈ P(h), we assume thatthere exists a probability measure fN(·|h) defined on AN(h),where each such probability measure is independent of everyother such measure

12 / 36




E5. for every i ∈ N, Di is a partition of {h ∈ H|i ∈ P(h)} withthe property that Ai(h) = Ai(h

′) whenever h and h′ are in thesame element di ∈ Di

DefinitionA set di ∈ Di is called information set of player i .

Interpretation: any given member of di is indistinguishable toplayer i .

DefinitionA game is of perfect recall if no player ever forgets any informationhe once knew and all players know the actions they have chosenpreviously.

In perfect information games of perfect recall all information setsare singletons.

13 / 36




DefinitionAn imperfect information efg Γ consists of Γ = {N,H,P , f ,D, u}.

DefinitionThe probability measures fN(·|h) over Nature’s moves AN(h) ath ∈ H are called prior probabilities over A(h).

14 / 36



2.c Mixed & behavior strategiesConvexity considerations lead us to allow for mixed actions.

DefinitionPlayer i ’s mixed strategy σi is a probability distribution over a setof (pure) strategies.

DefinitionA behavior strategy for player i , βi is an element of the Cartesianproduct ×di∈Di

△ (A(di )).

So the difference is that◮ a mixed strategy is a mixture over complete, contingent plans:

a pure strategy is selected randomly before play starts◮ a behavioral strategy specifies a probability distribution over

actions at each di and the probability distribution at differentinfo sets are independent.

15 / 36



2.c Mixed & behavioural strategies

These two objects are different

◮ the mixed strategy selects one pure complete, contingent planrandomly at the beginning of the game

◮ the behavioural strategy specifies a randomisation over theavailable actions for each point of choice

but the difference only matters in games of imperfect recall.

DefinitionTwo strategies σi and σ′

i are equivalent if they lead to the sameprob distribution over outcomes for all σ−i .

Theorem(Kuhn 1953) In a game of perfect recall, mixed and behaviorstrategies are equivalent.

16 / 36




DefinitionWe denote the terminal histories when each player i followssi , i ∈ N by o(s) ∈ Z.

DefinitionA NE’qm of an efg is a strategy profile σ∗ such that for everyplayer i ∈ N and all σi ∈ ∆(Si)

ui(o(σ∗)) ≥ ui(o(σi , σ∗−i )).

We can solve an efg for NE’qa by transforming the game into areduced strategic form game (rsfg) and solving this game in theusual way.

17 / 36



2.d Nash equilibriumConsider the following efg

du

1

r1

(0,0)

l1

(2,1)

2r2

(3,2)

l2

(-1,1)

2

N =

[u, (l1, l2)] ,[d , (l1, r2)] ,[d , (r1, r2)]

◮ A1 = {u, d}

◮ S1 = {u, d}

◮ A2 = {l1, r1, l2, r2}

◮ S2 = {(l1, l2), (l1, r2),(r1, l2), (r1, r2)}

The rsfg for this game is{N,S , u(o(s))}

l1, l2 l1, r2 r1, l2 r1, r2u 2,1 2,1 0,0 0,0d -1,1 3,2 -1,1 3,2

18 / 36




Notice that

◮ there are several efg’s for the same sfg

◮ the set of NE’qa of the efg can be found by looking at the setof NE’qa of the sfg

◮ there is a serious problem with NE’qa in efg: there may beactions in a strategy which do not affect an (e’qm) outcomewhich are inconsistent with what the associated player wouldchoose if moving at that node.

19 / 36



2.e Subgame Perfection

Definition: A proper subgame G of an efg Γ consists of a singlenode and all its successors in Γ, with the following properties:

1. if node x ′ ∈ G and for some i ∈ N x ′′ ∈ di(x′), then x ′′ ∈ G .

That is x ′ and x ′′ are in the same information set in thesubgame if and only if they are in the same information set inthe original game

2. the payoff function of the subgame is just the restriction of theoriginal payoff function to the terminal nodes of the subgame

20 / 36




Less formally, part of an efg is called a subgame if

◮ it starts from a singleton information set d

◮ it contains all successors to d (until the end of the game)

◮ no node outside the set of successors to d is contained in anyof the subgame’s information sets.

Notice that

◮ by definition, the entire game is a subgame of itself

◮ after different histories follow different subgames

◮ players know that they are in the same subgame.

21 / 36




Definition (Selten): A NE’qm s∗ of an efg Γ is calledsubgame perfect e’qm (SGPE’qm) iff it induces a NE’qm for every(proper) subgame Γ(h) for every h ∈ H−Z.

22 / 36




Consider the games such that

1. in each stage k, every player knows all the actions, includingthose by Nature, that were taken at any previous stage

2. each player moves at most once within a given stage

3. no information set contained in stage k provides anyknowledge of play in that stage

these games are called multi-stage games with observed actions.

Notice that multi-stage games with observed actions can be bothfinite and infinite horizon games.

23 / 36




Definition: A strategy profile s satisfies the one-stage-deviationcondition if no player i can gain by deviating from s in a singlestage and conforming to s thereafter.

Theorem(one-stage-deviation principle): In a finite multi-stagegame with observed actions, strategy profile s is subgame perfect ifand only if it satisfies the one-stage-deviation condition. Moreprecisely, profile s is subgame perfect iif there is no player i andstrategy si that agrees with si except at a single stage t andhistory ht , and such that si is a better response to s−i than siconditional on ht being realized.

24 / 36




Definition: A game is continuous at infinity if for each player i theutility function ui satisfies

suph,h s.t. ht=ht

|ui (h) − ui (h)| → 0 as t → ∞

Theorem(one-stage-deviation principle): In an infinite multi-stagegame with observed actions that is continuous at infinity, strategyprofile s is subgame perfect if and only if it satisfies theone-stage-deviation condition.

25 / 36




Example: Bargaining model

◮ two players must agree on how to share a pie of size 1

◮ in periods 0,2,4,... P1 proposes sharing rule (x , 1 − x) and P2can accept or reject. If P2 accepts, the game ends

◮ if P2 rejects in period 2k, then in period 2k + 1 P2 proposessharing rule (x , 1 − x) and P1 can accept or reject.

◮ if (x , 1 − x) is accepted at date t, the payoffs are(δtx , δt(1 − x)), where δ < 1 is the players’ discount factor

26 / 36




The SGPE’qm of this game is:

◮ Pi always demands a share (1 − δ)/(1 − δ2) when it is histurn to make an offer.

◮ He accepts any share greater or equal to δ(1 − δ)/(1 − δ2).

27 / 36



2.f The value of commitment

Tournament (Dixit 87)

◮ 2 players choose how much effort to exert (e1, e2) ≥ 0

◮ player i wins the tournament with probabilitypri (ei , ej ) = ei

ei+ejwhere i ∈ {1, 2} and j = 3 − i if (e1, ej) 6= 0

and 1

2otherwise

◮ the winner gets the prize of K > 0

◮ the costs of exerting effort e ≥ 0 is c1(e) = e for player 1 andc2(e) = 2e for player 2

◮ agent i ’s expected utility is given by

Kpri (ei , ej ) − ci (ei )

28 / 36




Without commitment:

Nash equilibrium (e∗1 , e∗2 ) should satisfy

e∗1 = argmaxe1Kpr1(e1, e

∗2 ) − c1(e1)

e∗2 = argmaxe2Kpr2(e2, e

∗1 ) − c2(e2)

Which is given by (e∗1 , e∗2 ) = (2

9K , 1

9K )

while equilibrium utilities are (2

9K , 1

9K )

29 / 36




With commitment: Assume that player 1 is able to commit tosome effort level e1.

P2 will choose effort

e2 = argmaxe2Kpr2(e1, e2) − c2(e2)

The solution is given by e2(e1) =√

K2e1 − e1

Hence, P1 should commit to the effort level e1

e1 = argmaxe1Kpr1(e1, e2(e1)) − c1(e1)

The solution is given by e1 = K2.

Hence, the vector of the chosen actions is (K2, 0) and of utilities

(K2, 0) ⇒ the ability to commit increases the utility.

30 / 36



2.f Repeated games

Repeated ”prisoner’s dilemma”. Current payoffs gi (a∗) are given by

cooperate defectcooperate 1,1 -1,2

defect 2,-1 0,0

◮ players’ discount factor is δ

◮ the utility of a sequence (a0, ..., aT ) is

1 − δ

1 − δT+1

T∑

t=0

δtgi (at)

31 / 36



2.f Repeated gamesStage game

◮ finite game (just for simplicity)◮ N players◮ simultaneous move (just for simplicity) game◮ Ai action space for i◮ stage-game payoff function gi : A → R with A = ×iAi

Repeated game◮ history ht = (a0, ..., at−1) is the realized choice of actions

before t with aτ = (aτ

1 , ..., aτ

N )◮ a mixed (behavior) strategy σi : Ht −→ ∆(Ai ) where

Ht = (A)t is the space of all possible period t histories◮ payoff function, for instance

1 − δ

1 − δT+1

T∑

t=0

δtgi (at) or

1

T

T∑

t=0

gi (at)

32 / 36



2.f Repeated games

Equilibria of repeated ”prisoner’s dilemma”

◮ The game is played only once, (defect, defect) is the onlyequilibrium.

◮ The game is repeated finite number of times. The uniquesubgame perfect eq. is for both players to defect every period.

◮ The game is played infinitely often. The profile (defect, defect)in all periods remains a subgame perfect equilibrium.

33 / 36



2.f Repeated games

For infinitely repeated game with δ > 1/2 the following strategyprofile is a subgame-perfect eq as well:

◮ cooperate in the first period and continue to cooperate aslong as no player has ever defected.

◮ if any player has ever defected, then defect for the rest of thegame

34 / 36



2.f Repeated games

Two repetition of the stage game:

L M RU 0,0 3,4 6,0M 4,3 0,0 0,0D 0,6 0,0 5,5

If played once, there are three equilibria: (M,L), (U,M), andmixed equilibrium ((3/7U, 4/7M), (3/7L, 4/7M)) withcorresponding payoffs (4, 3), (3, 4) and (12/7, 12/7).

35 / 36



2.f Repeated games

Assume that δ > 7/9 Consider the following strategy profile:

◮ P1: Play D in the first stage. If the first stage outcome is(D,R), then play M in the second stage, otherwise, play(3/7U, 4/7M)

◮ P2: Play R in the first stage. If the first stage outcome is(D,R), then play L in the second stage, otherwise, play(3/7L, 4/7M)

It constitutes a subgame-perfect equilibrium.

36 / 36

microeconomics 2. game theory · description backward induction imperfect info mixed and behav. str...

Documents