1

Coinduction Principle for Games

Michel de Rougemont

Université Paris II & LRI

2

Coinduction (inspired by D. Kozen)

• Naive notionInduction: F monotone, F has a unique least-fixed point.Coinduction: F is antimonotone, unique smallest greatest fixed-point.Example: Even numbers over N

• General PrincipleFunction F defined by an equation, admits a fixed-point F*.We want to show that F* satisfies a property: it suffices to show that the property is preserved by the equation.

• Foundation: Let B a Banach Space, R is a linear Operator, of spectral radius <1. It admits a fixed point.

• Applications to Computer Science:1. Evolutionary Games2. Combinatorics3. Stochastic processes

3

1. Evolutionary Dynamics

1 2+a 0

0 1 2+a

2+a 0 1

Example: Rock-Scissor-Paper:

Mixed strategy= density of agents playing pure strategiesReplicator Strategy:

( , ) ( , )ii i i

xx g e x g x x x

t

( , ) (0,...0,1,0,..0) . .tig e x A x

1 1 2 1

2 2 3 1

1 3 1 1

(2 )

(2 )

(2 )

t

t

t

x x a x x Ax x

x x a x x Ax x

x x a x x Ax x

0Case a

( , ) ( ) *

1 1 1* ( , , )

3 3 3

tT x Min x t x

x

2

c Show T(x, ) ?

4

2. Enumerative Combinatorics

=( (0), (1), (2),....)= (0)+X ' Coinductive Counting (J. Rutten), Stream Calculus

Male Bees=Drones D (Q,D)QFemale Bees=Queens Q QD

How many Q ancestors at level k?

Q D

Q

Q D

D

Q

Q

D

Q

D

Q

kq0 q1

0 0 1 (0)=0 ; ' =

1 1 1 0 (0)=1 ; ' =

1 02 2

1 X = ; =

1-X-X 1-X-X

5

3. Probabilistic Processes

•Equivalence of Markov Chains ?

•Metric Analogue of Bisimulation (Desharnais and al.)

•D-bar measure in Statistics

•Approximate Equivalence

•Property of Markov Chains:

….

p

1-p

( ) E [ ( , ) ]c

T n Time s tn

6

Coinduction to compare 2 processes

1 1 2 2 E [ ( , ) ]; E [ ( , ) ]T Time s t in P T Time s t in P

•Property of Markov Chains:

Examples from D. Kozen, Lics 2006

Generation of a biased coin (q) given a biased coin (p).

Consider 3 different processes.

1 2 We want to show that T T

7

Biased coin simulation

Algorithm : qflip(q):If q >p( if pflip=head) return head else return qflip(q-p/1-p) )

else ( if pflip=head) return qflip(q/p) else return tail )

p q0

1

q’0

1

Given: pflip a biased coin (head, tail) with probability (p,1-p).

Task(q) : Generate a biased coin with probability (q,1-q).

q p0

1

q’0

p q

q p

8

Strategy : qflip(q)

Convergence: output with probability Min(p,1-p) at each step. Halts with probability 1.

H(q)=Probability qflip(q)=head. ( / ) if

( )

(1 ). (1 (1 /1 )) if

p H q p q p

H q

p p H q p q p

p q0

1

q’0

1

*( ) is a solution.H q q

9

Time estimation of qflip(q)

Estimated Time:

0

0

0

1 . ( / ) if

( )

1 (1 ). (1 (1 /1 )) if

p E q p q p

E q

p E q p q p

p q0

1

Algorithm : qflip(q):If q >p( if pflip=head) return head else return qflip(q-p/1-p) )

else ( if pflip=head) return qflip(q/p) else return tail )

q’0

1

0

1 *( )

1

q qE q

p p

10

Strategy 1: qflip(q)

Estimated Time:

1

q/p if

1- ( ) 1- if 1-

1-

11 if 1-

q p

qf q p q p

p

qp q

p

p q0 1

Algorithm 1 : qflip(q): If ( q>1-p) ( if pflip=head) return qflip((q-1+p)/p) else return head

Else if q >p( if pflip=head) return head else return qflip(q-p/1-p) )else ( if pflip=head) return qflip(q/p) else return tail )

1-p

1 if p<q 1-p ( )

o.w.

pr q

p

1 1 1 ( ) 1 ( ) ( ( ))E q r q E f q

1-p p

11

Comparison between Strategies 0 and 1

1

4p

1

4p

0 1 Experimental Graph *( ), *( ) 0 1E q E q q

1 Difficulty: no analytical representation of *( ) E q

12

Bounded Linear Operators

1

1

( ) 1 ( ) ( ( )) .

. . ( ) ( ( ))

E q r q E f q a R E

R E q r q E f q

B, Banach Space, R is a linear Operator, of spectral radius <1.Affine Operator τ(e) =a+ReΦ closed non empty region preserved by τ.

Conclusion: there exists a fixed point e* (τ(e*) =a+Re*) s.t e* is in Φ.

Example: E(q) is bounded.

closed non-empty region preserved by

B

Φ

E*

is the property we want to prove on E*

13

Coinduction principle

Co induction principle:

*

( ) ( ) ( ( ))

---------------------------------------

( )

e e e e e

e

(e)=a+Re where R has Spectral radius < 1

closed region preservev by B

* Unique solution of e=a+Re satisfisfies e

14

Application

Co induction principle1

( ) : ( )1

q qE q E q

p p

1 1 ( ) .(1 ( ) ( ( ))) radius 1-pE q r q E f q

*1

1 We want to show: ( )

1

q qq E q

p p

1(1) E ( ) take E= q.0

1

q qq E q

p p

1 1(2) E[ ( ) ) ( ( )( ) ))]

1 1

q q q qq E q q E q

p p p p

1 11 1

( ) 1 ( ) 1(4) ( ( )) ) 1 ( ) ( ( ))

1 1

f q f q q qE f q r q E f q

p p p p

1 11

( ) 1 ( ) 1(3) It suffices to show: ( ( )) ) ( )( )

1 1

f q f q q qE f q E q

p p p p

Take cases : , 1 ,1q p p q p p q

15

Case 1

Co induction setting1

( ) : ( )1

q qE q E q

p p

1 1 ( ) .(1 ( ) ( ( ))) radius 1-pE q r q E f q

*1

1 We want to show: ( )

1

q qq E q

p p

1 11 1

( ) 1 ( ) 1(4) ( ( )) ) 1 ( ) ( ( ))

1 1

f q f q q qE f q r q E f q

p p p p

1Case 1: ( ) / and ( )q p f q q p r q p

2

1 / 1(6) 1 ( / ) 1 ( )

1 1

q q p q qpE q p p

p p p p

2

11

(5) ( ) 1 ( )1 1

qq q q q qp

E pEp p p p p p

16

Strategy 2: qflip(q)

Estimated Time:

1

q/p if

1-1- if 1-

1-

( ) if 1/2 1

1

11 if 1-

q p

qp q p

p

f q qq p

p

qp q

p

p q0 1

Algorithm 2 : qflip(q): If ( q>1-p) ( if pflip=head) return qflip((q-1+p)/p) else return head Else If ( q>0.5) ( if pflip=head) return tail else qflip(q/1-p) Else if q >p( if pflip=head) return head else return qflip(q-p/1-p) )

else ( if pflip=head) return qflip(q/p) else return tail )

1-p

1 if p<q 1-p ( )

o.w.

pr q

p

2 2 2 ( ) 1 ( ) ( ( ))E q r q E f q

1-p p

1/2

17

New Application

2 2 2 Second trial: break point at (1- ,1 (1 ) ) if (1 )c Max p p p p

Co induction principle on pairs (E,E’)

First trial. ( , ') : ( ) '( ). !!E E q E q E q Incorrect

( , ') [ .(1 ( ) ( ( ))), .(1 ( ) ' ( '( )))]E E q r q E f q q r q E f q

* *2 1 We want to show: ( ) ( )q E q E q

Second trial. ( , ') : ( ) '( ) .....E E q E q E q

* * * *2 1 2 1 ( ), ( ) are nowhere differentiable but ( ) ( )E q E q E q E q

18

Probabilistic Processes

•Equivalence of Markov Chains ? Are M1 ,M2 ε-close ?

•Metric Analogue of Bisimulation (Desharnais and al.)•Distance d between distributions obtained by iterations, also the maximum fixed-point of a Functional F.

•Property of Markov Chains:

….

p-ε

1-p+ ε

….

p

1-p

M1

M2

19

Conclusion

General Principle: Given τ linear bounded operator of spectral radius <1,Φ closed non empty region preserved by τ, we conclude that there exists a fixed

point e* in Φ.

Applications:1. Stochastic processes. Compare Expected time between two fractal

processes.

2. Evolutionary Games. Compare convergence time.

3. Analysis of Streams.