Finite Iterated Prisoner’s Dilemma Revisited: Belief
Change and End Game Effect
Jiawei Li (Michael) & Graham KendallUniversity of Nottingham
2
Outline
• Iterated prisoner’s dilemma (IPD)• Probability vs. uncertainty• A new model of IPD• End game effect• Conclusions
Jubilee Campus, University of Nottingham
3
Iterated Prisoner’s Dilemma (IPD)
• Prisoner’s DilemmaTwo suspects are arrested by the police. They are separated and offered the same deal. If one testifies (defects from the other) for the prosecution against the other and the other remains silent (cooperates with the other), the betrayer goes free and the silent accomplice receives the full 10-year sentence. If both remain silent, both prisoners are sentenced to only six months in jail for a minor charge. If each betrays the other, each receives a five-year sentence. Each prisoner must choose to betray the other or to remain silent.
4
Iterated Prisoner’s Dilemma (IPD)
Nash equilibrium: (Defect, Defect)
5
Iterated Prisoner’s Dilemma (IPD)
• Finite IPD– n-stages;– n is known;– End game effect;– Backward induction;– Nash equilibrium;
6
Iterated Prisoner’s Dilemma (IPD)
• Finite IPD experiments
Each of 15 pairs of subjects plays 22-round IPD. Average cooperation rates are 44.2% in known and 55.0% in unknown. (from Andreoni and Miller (1993))
7
Iterated Prisoner’s Dilemma (IPD)
• Incomplete information (Kreps et al (1982))– ‘Tit for Tat (TFT)’ type or ‘Always defect (AllD)’ type;– Assign probability to
TFT type and 1- to AllD type;– Mutual cooperation can be
sequential equilibrium;– End game effect;
8
Iterated Prisoner’s Dilemma (IPD)
• A new model
– Assumption that both players may be either AllD or TFT type;
– Repeated game with uncertainty;
– Changeable beliefs;
9
Probability vs. uncertainty
• Probability: Tossing a coin;
50% -- 50%
• Uncertainty: Tossing a what?50% -- 50%?
10
Probability vs. uncertainty
• Probability– Tossing a coin repeatedly;– 50% -- 50%
• Uncertainty– Tossing the dice repeatedly;– Expected probability may
change according to the outcome of past playing.
11
Probability vs. uncertainty
• Repeated games with uncertainty– Bayes theorem:
P(Head) = 0.5;p(Head|1Head) = 0.683;p(Head|1Tail) = 0.317;p(Head|2Head) = 0.888;p(Head|2Head+1Tail) = 0.565;......
12
A new model of IPD
• Let ROW and COL denote the players in a N-stage IPD game.
where R, S, T, and P denote, Reward for mutual cooperation, Sucker’s payoff, Temptation to defect, and Punishment for mutual defection, and
13
A new model of IPD
• We refer to COL’s belief (Row’s type) by
that denote the probabilities that, if ROW is a TFT player, ROW
chooses to cooperate at the 1,…,n stage. Similarly, we define
ROW’s belief about COL’s type by that
denotes the probabilities that COL chooses to cooperate at
each stage if COL is a TFT player. {δi} and {θi} represent the
beliefs of player COL and ROW respectively.
14
A new model of IPD
Two Assumptions:
• A1:
• A2: If either player chooses to defect at stage i, there will be
15
A new model of IPD
16
A new model of IPD
17
A new model of IPD
18
A new model of IPD
(D, D) is always Nash equilibrium at stage i; but it is not necessarily the only Nash equilibrium. When (2) is satisfied, both (C, C) and (D, D) are Nash equilibrium. (2) is a necessary and sufficient condition.
19
A new model of IPD
A sufficient condition for (C,C) to be Nash equilibrium:
This condition denotes a depth-one induction, that is, if both players are likely to cooperate at both the current stage and the next stage, it is worth each player choosing to cooperate at the current stage. For example, when T=5, R=3, P=1, and S=0, the condition for (C,C) to be Nash equilibrium is,
20
End game effect
• ?
• Unexpected hanging paradox.– A condemned prisoner;– Will be hanged on one weekday
in the following week;– Will be a surprise;
21
End game effect
22
End game effect
P4 = 1
P5 = 1
P3 = 1
P2 = 1
P1 = 1
23
End game effect
24
End game effect
• Process of belief change
P1 = 1/5P2 = 1/5P3 = 1/5P4 = 1/5P5 = 1/5
P1 = 0P2 = 1/4P3 = 1/4P4 = 1/4P5 = 1/4
P1 = 0P2 = 0P3 = 1/3P4 = 1/3P5 = 1/3
P1 = 0...P4 = 1/2P5 = 1/2
P1 = 0...P5 = 1
25
End game effect
• Backward induction is not suitable for repeated games with uncertainty because uncertainty can be decreased during the process of games.
• End game effect has limited influence on the players’ strategies since it cannot be backward inducted.
26
End game effect
• Why does the rate of cooperation in finite IPD experiments decrease as the game goes toward the end?
27
Conclusions
• 1. We develop a new model for finite IPD that takes into consideration belief change.
• 2. Under the new model, the conditions of mutual cooperation are deduced. The result shows that, if the conditions are satisfied, both mutual cooperation and mutual defection are Nash equilibrium. Otherwise, mutual defection is the unique Nash equilibrium.
28
Conclusions
• 3. This model could also deal with indefinite IPD and infinite IPD.
• 4. The outcome of this model conforms to experimental results.
29
Conclusions
• 5. Backward induction is not suitable for repeated games with uncertainty when the beliefs of the players are changeable.
• 6. This model has the potential to apply to other repeated games of incomplete information.
30
Thank you.