the multiple-urn ehrenfest model - willamette …...the multiple-urn ehrenfest model a look into...

The Multiple-Urn Ehrenfest ModelA Look into Eigenanalysis and Hitting Times

Jacquelyn Combellick, Katie Patterson, Daniel RabanAdvised by Yung-Pin Chen

Willamette Mathematics Consortium REU

July 30, 2016

Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 1 / 42

What will we cover today?



(1) Markov chains



(1) Markov chains

(2) The Ehrenfest urn model



(1) Markov chains


(3) Eigenanalysis



(1) Markov chains


(3) Eigenanalysis

(4) Mixing times



(1) Markov chains


(3) Eigenanalysis

(4) Mixing times

(5) Hitting times



(1) Markov chains


(3) Eigenanalysis

(4) Mixing times

(5) Hitting times

(6) Summary


The Ehrenfest urn model example

Urn 0 Urn 1

1 4

3

M 2


What is a Markov chain?

A Markov chain is a sequence of random variables X0,X1,X2, ... such thatPr[Xn+1 = sn+1 |X0 = s0,X1 = s1, ...,Xn = sn]=Pr[Xn+1 = sn+1 |Xn=sn]where s0, s1, ..., sn, sn+1 are elements in the state space of the Markovchain.




This is known as the memoryless property.




This is known as the memoryless property.

Let Xn denote the number of balls in Urn 1 at time n = 0, 1, 2.... Then(X0,X1, ...) forms a Markov chain with the state space S = {0, 1, ...,M}.


The Ehrenfest urn model example

Urn 0 Urn 1

1 4

3

M 2


Transition matrix

A k × k matrix P is said to be a transition matrix of a Markov chain(X0,X1, ...) with state space S = {s1, ..., sk} if

Pr[Xn+1 = sj | Xn = si ] = Pi ,j .


Transition matrix

A k × k matrix P is said to be a transition matrix of a Markov chain(X0,X1, ...) with state space S = {s1, ..., sk} if

Pr[Xn+1 = sj | Xn = si ] = Pi ,j .

For the Ehrenfest urn model with M=5 balls, the transition matrix is:

0 1 2 3 4 5

0 0 1 0 0 0 01 1/5 0 4/5 0 0 02 0 2/5 0 3/5 0 03 0 0 3/5 0 2/5 04 0 0 0 4/5 0 1/55 0 0 0 0 1 0


Stationary distribution

Let (X0,X1, ...) be a Markov chain with state space S = {s1, ..., sk} andtransition matrix P. A row vector π=(π1, ..., πk ) is said to be a stationary

distribution for the Markov chain, if it satisfies

(i)πi ≥ 0 for i = 1, ..., k, and∑k

i=1 πi = 1, and

(ii)πP=π, meaning that∑k

i=1 πiPi ,j = πj for j = 1, ..., k.


The 3-urn Ehrenfest model

Urn 0 Urn 1 Urn 2

1 4

3

M 2


The d -urn Ehrenfest model

Urn 0 Urn 1 Urn d-1

1

M-1

3

M 2

. . .


Motivation for studying the multiple-urn Ehrenfest model



Real-world applications:




(1) Population migration




(1) Population migration(expected hitting time)





(2) Statistical mechanics, thermodynamics, and diffusion





(2) Statistical mechanics, thermodynamics, and diffusion(stationary distribution)





(2) Statistical mechanics, thermodynamics, and diffusion(stationary distribution)

(3) Treatment allocation


Why is eigenanalysis important?



(1) Eigenvalues can tell how a Markov chain behaves.



(1) Eigenvalues can tell how a Markov chain behaves.

For example, a finite Markov chain converges to a stationarydistribution if and only if its transition matrix has eigenvalue 1 withmultiplicity 1 and all other eigenvalues are of modulus less than 1.



(2) We can diagonalize a transition matrix P and compute Pn easily,

provided P is diagonalizable.





That is, we can find an invertible matrix A such that

P = A−1

D A,

where D is a diagonal matrix with all the eigenvalues of P on its maindiagonal.





That is, we can find an invertible matrix A such that

P = A−1

D A,

where D is a diagonal matrix with all the eigenvalues of P on its maindiagonal.

This helps us compute Pn:

Pn = A

−1D

nA.


Eigenvalues of the 3-urn model

For the 3-urn model, we started by observing the eigenvalues λ for variousvalues of M, the number of balls in the model.




M = 1: λ = 1, −12 , −1

2 .




M = 1: λ = 1, −12 , −1

2 .

M = 2: λ = 1, 14 , 1

4 , −12 , −1

2 , −12 .




M = 1: λ = 1, −12 , −1

2 .

M = 2: λ = 1, 14 , 1

4 , −12 , −1

2 , −12 .

M = 3: λ = 1, 12 , 1

2 , 0, 0, 0, −12 , −1

2 , −12 , −1

2 .




M = 1: λ = 1, −12 , −1

2 .

M = 2: λ = 1, 14 , 1

4 , −12 , −1

2 , −12 .

M = 3: λ = 1, 12 , 1

2 , 0, 0, 0, −12 , −1

2 , −12 , −1

2 .

M = 4: λ = 1, 58 , 5

8 , 14 , 1

4 , 14 , −1

8 , −18 , −1

8 , −18 , −1

2 , −12 , −1

2 , −12 , −1

2 .




M = 1: λ = 1, −12 , −1

2 .

M = 2: λ = 1, 14 , 1

4 , −12 , −1

2 , −12 .

M = 3: λ = 1, 12 , 1

2 , 0, 0, 0, −12 , −1

2 , −12 , −1

2 .

M = 4: λ = 1, 58 , 5

8 , 14 , 1

4 , 14 , −1

8 , −18 , −1

8 , −18 , −1

2 , −12 , −1

2 , −12 , −1

2 .

Our observation:

For the 3-urn model with M balls, the transition matrix has (M + 1)distinct eigenvalues equally distanced between 1 and −1

2 , and the kthlargest eigenvalue has multiplicity k.




M = 1: λ = 1, −12 , −1

2 .

M = 2: λ = 1, 14 , 1

4 , −12 , −1

2 , −12 .

M = 3: λ = 1, 12 , 1

2 , 0, 0, 0, −12 , −1

2 , −12 , −1

2 .

M = 4: λ = 1, 58 , 5

8 , 14 , 1

4 , 14 , −1

8 , −18 , −1

8 , −18 , −1

2 , −12 , −1

2 , −12 , −1

2 .

Our observation:

For the 3-urn model with M balls, the transition matrix has (M + 1)distinct eigenvalues equally distanced between 1 and −1

2 , and the kthlargest eigenvalue has multiplicity k. Let α = 3

2M . Then the eigenvaluesare given by

1, (1 − α)2, (1 − 2α)3, ...,

(

−1

2

)

M+1

.


Mark Kac’s results

In 1947, mathematician Mark Kac examined the Ehrenfest urn model with2 urns and found that the eigenvectors and eigenvalues of the transitionmatrix are determined by a system of linear equations using the generatingfunction

f (z) =

∞∑

k=0

xkzk

where xk is the kth component of the eigenvector x.


Mark Kac’s results

Using Kac’s function as a model, we define the following polynomial togenerate the row eigenvectors for the 3-urn model:

f M(z1, z2) =

M∑

s1=0

M−s1∑

s2=0

a(s1,s2)zs11 z s2

2

where a(s1,s2) is the (s1, s2)th component of the eigenvector a from thetransition matrix P.


Transition probabilities

The possible states for the 3-urn model are given by s = (s1, s2). Thetransition probabilites are then




Pr[(s1, s2) → (s1 − 1, s2)] =s1

2M




Pr[(s1, s2) → (s1 − 1, s2)] =s1

2M

Pr[(s1, s2) → (s1 + 1, s2)] =M − s1 − s2

2M

Pr[(s1, s2) → (s1 + 1, s2 − 1)] =s2

2M

Pr[(s1, s2) → (s1 − 1, s2 + 1)] =s1

2M

Pr[(s1, s2) → (s1, s2 + 1)] =M − s1 − s2

2M

Pr[(s1, s2) → (s1, s2 − 1)] =s2

2M.


Derived partial differential equation

Using the fact that aP = λa for any eigenvector a with eigenvalue λ, weare able to substitute in our transition probabilities, multiply each term byz s11 z s2

2 , and sum over all of s1 and s2.


Derived partial differential equation

Using the fact that aP = λa for any eigenvector a with eigenvalue λ, weare able to substitute in our transition probabilities, multiply each term byz s11 z s2

2 , and sum over all of s1 and s2.

After collecting like terms and simplifying, we obtain the partial differentialequation

(1 − z1)(1 + z1 + z2)∂f M

∂z1+ (1 − z2)(1 + z1 + z2)

∂f M

∂z2

+ M(z1 + z2 − 2λ)f M(z1, z2) = 0.


Generating function and eigenvalues for d = 3 urns

Theorem

The coefficients of the functions

f M(r1,r2)

(z1, z2) = (1 − z1)r1(1 − z2)

r2(1 + z1 + z2)r3 ,

where r1, r2, and r3 are nonnegative integers and r1 + r2 + r3 = M, define a

set of linearly independent eigenvectors of the 3-urn M-ball transition

matrix.


Generating function and eigenvalues for d = 3 urns

Theorem


f M(r1,r2)

(z1, z2) = (1 − z1)r1(1 − z2)

r2(1 + z1 + z2)r3 ,

where r1, r2, and r3 are nonnegative integers and r1 + r2 + r3 = M, define a

set of linearly independent eigenvectors of the 3-urn M-ball transition

matrix.

Corollary

The eigenvalues of the 3-urn M-ball transition matrix are λ = 32M r3 −

12

for r3 ∈ {0, · · · ,M}. Eigenvalue λ has multiplicity 2M3 (1 − λ) + 1.


Example: The row eigenvector matrix A for M = 3 ballsConsider the function f 3

(r1,r2)(z1, z2) = (1 − z1)

r1(1 − z2)r2(1 + z1 + z2)

r3 .

For r1 + r2 = 2M3 (1 − λ), we have:



(r1,r2)(z1, z2) = (1 − z1)

r1(1 − z2)r2(1 + z1 + z2)

r3 .

For r1 + r2 = 2M3 (1 − λ), we have:

0 = 0 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1



(r1,r2)(z1, z2) = (1 − z1)

r1(1 − z2)r2(1 + z1 + z2)

r3 .

For r1 + r2 = 2M3 (1 − λ), we have:

0 = 0 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1

1 = 1 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1

2

1 = 0 + 1 = r1 + r2 = 2M3 (1 − λ), λ = 1

2



(r1,r2)(z1, z2) = (1 − z1)

r1(1 − z2)r2(1 + z1 + z2)

r3 .

For r1 + r2 = 2M3 (1 − λ), we have:

0 = 0 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1

1 = 1 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1

2

1 = 0 + 1 = r1 + r2 = 2M3 (1 − λ), λ = 1

2

2 = 2 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 0

2 = 1 + 1 = r1 + r2 = 2M3 (1 − λ), λ = 0

2 = 0 + 2 = r1 + r2 = 2M3 (1 − λ), λ = 0



(r1,r2)(z1, z2) = (1 − z1)

r1(1 − z2)r2(1 + z1 + z2)

r3 .

For r1 + r2 = 2M3 (1 − λ), we have:

0 = 0 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1

1 = 1 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1

2

1 = 0 + 1 = r1 + r2 = 2M3 (1 − λ), λ = 1

2

2 = 2 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 0

2 = 1 + 1 = r1 + r2 = 2M3 (1 − λ), λ = 0

2 = 0 + 2 = r1 + r2 = 2M3 (1 − λ), λ = 0

3 = 3 + 0 = r1 + r2 = 2M3 (1 − λ), λ = −1

2

3 = 2 + 1 = r1 + r2 = 2M3 (1 − λ), λ = −1

2

3 = 1 + 2 = r1 + r2 = 2M3 (1 − λ), λ = −1

2

3 = 0 + 3 = r1 + r2 = 2M3 (1 − λ), λ = −1

2


Example: The row eigenvector matrix A for M = 3 ballsWith these values for (r1, r2), we can assess f 3

(r1,r2)(z1, z2):

f 3(0,0)(z1, z2) = (1 + z1 + z2)

3 λ = 1



(r1,r2)(z1, z2):

f 3(0,0)(z1, z2) = (1 + z1 + z2)

3 λ = 1

f 3(1,0)(z1, z2) = (1 − z1)

1 (1 + z1 + z2)2 λ = 1

2

f 3(0,1)(z1, z2) = (1 − z2)

1(1 + z1 + z2)2



(r1,r2)(z1, z2):

f 3(0,0)(z1, z2) = (1 + z1 + z2)

3 λ = 1

f 3(1,0)(z1, z2) = (1 − z1)

1 (1 + z1 + z2)2 λ = 1

2

f 3(0,1)(z1, z2) = (1 − z2)

1(1 + z1 + z2)2

f 3(2,0)(z1, z2) = (1 − z1)

2 (1 + z1 + z2)1 λ = 0

f 3(1,1)(z1, z2) = (1 − z1)

1(1 − z2)1(1 + z1 + z2)

1

f 3(0,2)(z1, z2) = (1 − z2)

2(1 + z1 + z2)1



(r1,r2)(z1, z2):

f 3(0,0)(z1, z2) = (1 + z1 + z2)

3 λ = 1

f 3(1,0)(z1, z2) = (1 − z1)

1 (1 + z1 + z2)2 λ = 1

2

f 3(0,1)(z1, z2) = (1 − z2)

1(1 + z1 + z2)2

f 3(2,0)(z1, z2) = (1 − z1)

2 (1 + z1 + z2)1 λ = 0

f 3(1,1)(z1, z2) = (1 − z1)

1(1 − z2)1(1 + z1 + z2)

1

f 3(0,2)(z1, z2) = (1 − z2)

2(1 + z1 + z2)1

f 3(3,0)(z1, z2) = (1 − z1)

3 λ = −12

f 3(2,1)(z1, z2) = (1 − z1)

2(1 − z2)1

f 3(1,2)(z1, z2) = (1 − z1)

1(1 − z2)2

f 3(0,3)(z1, z2) = (1 − z2)

3 .


Example: The row eigenvector matrix A for M = 3 ballsExpanding the two polynomials for λ = 1

2 gives us the coefficients andtherefore the components of the corresponding eigenvectors:




f 3(1,0)(z1, z2) = (1 − z1)(1 + z1 + z2)

2

= 1 + z1 + 2z2 − z21 + 0z1z2 + z2

2 − z31 − 2z2

1z2 − z1z22 + 0z3

2




f 3(1,0)(z1, z2) = (1 − z1)(1 + z1 + z2)

2

= 1 + z1 + 2z2 − z21 + 0z1z2 + z2

2 − z31 − 2z2

1z2 − z1z22 + 0z3

2

and

f 3(0,1)(z1, z2) = (1 − z2)(1 + z1 + z2)

2

= 1 + 2z1 + z2 + z21 + 0z1z2 − z2

2 + 0z31 − z2

1z2 − 2z1z22 − z3

2 .




f 3(1,0)(z1, z2) = (1 − z1)(1 + z1 + z2)

2

= 1 + z1 + 2z2 − z21 + 0z1z2 + z2

2 − z31 − 2z2

1z2 − z1z22 + 0z3

2

and

f 3(0,1)(z1, z2) = (1 − z2)(1 + z1 + z2)

2

= 1 + 2z1 + z2 + z21 + 0z1z2 − z2

2 + 0z31 − z2

1z2 − 2z1z22 − z3

2 .

The row eigenvectors corresponding to each of these polynomials are

(1, 1, 2,−1, 0, 1,−1,−2,−1, 0)




f 3(1,0)(z1, z2) = (1 − z1)(1 + z1 + z2)

2

= 1 + z1 + 2z2 − z21 + 0z1z2 + z2

2 − z31 − 2z2

1z2 − z1z22 + 0z3

2

and

f 3(0,1)(z1, z2) = (1 − z2)(1 + z1 + z2)

2

= 1 + 2z1 + z2 + z21 + 0z1z2 − z2

2 + 0z31 − z2

1z2 − 2z1z22 − z3

2 .

The row eigenvectors corresponding to each of these polynomials are

(1, 1, 2,−1, 0, 1,−1,−2,−1, 0)

and(1, 2, 1, 1, 0,−1, 0,−1,−2,−1).


Example: The row eigenvector matrix A for M = 3 balls

So for f 3(r1,r2)

(z1, z2), we have the row eigenvector matrix

1 z1 z2 z21 z1z2 z2

2 z31 z2

1z2 z1z22 z3

2

(0, 0)(1, 0)(0, 1)(2, 0)(1, 1)(0, 2)(3, 0)(2, 1)(1, 2)(0, 3)

1 3 3 3 6 3 1 3 3 11 1 2 −1 0 1 −1 −2 −1 01 2 1 1 0 −1 0 −1 −2 −11 −1 1 −1 −2 0 1 1 0 01 0 0 −1 −1 −1 0 1 1 01 1 −1 0 −2 −1 0 0 1 11 −3 0 3 0 0 −1 0 0 01 −2 −1 1 2 0 0 −1 0 01 −1 −2 0 2 1 0 0 −1 01 0 −3 0 0 3 0 0 0 −1

= A .


Generating function and eigenvalues for d urns

Theorem


f Mr (z) =

(

d−1∏

k=1

(1 − zk)rk

)(

1 +d−1∑

k=1

zk

)rd

,

where all the components of r are nonnegative integers and∑d

k=1 rk = M,

define a set of linearly independent eigenvectors of the M-ball d-urn

transition matrix.


Generating function and eigenvalues for d urns

Theorem


f Mr (z) =

(

d−1∏

k=1

(1 − zk)rk

)(

1 +d−1∑

k=1

zk

)rd

,

where all the components of r are nonnegative integers and∑d

k=1 rk = M,

define a set of linearly independent eigenvectors of the M-ball d-urn

transition matrix.

Corollary

The eigenvalues of the d-urn M-ball transition matrix are

λ = dM(d−1) rd − 1

d−1 for rd ∈ {0, · · · ,M}. Eigenvalue λ has multiplicity(

M(d−1)d

(1−λ)+d−2d−2

)

.


Finding the inverse matrix A−1 for d = 3 urns



While we cannot expect there to be a pattern in the inverse of theeigenvector matrix, a remarkable pattern emerges upon inspection.




d = 3, M = 1:

1 1 11 −1 01 0 −1

−1

=1

3

1 1 11 −2 11 1 −2




d = 3, M = 1:

1 +z1 +z2

1 −z1 +0z2

1 +0z1 −z2

1

3

1 +z1 +z2

1 −2z1 +z2

1 +z1 −2z2



This pattern suggests that the rows of the inverse matrix A−1 are

generated by

f̃ M(s1,s2)

(z1, z2) =1

3M(1 − 2z1 + z2)

s1(1 + z1 − 2z2)s2(1 + z1 + z2)

s3 ,

where s1, s2, s3 are nonnegative integers such that s1 + s2 + s3 = M.s1, s2, s3 are the same as the r1, r2, r3 used to generate the correspondingrow of the original eigenvector matrix A.



This pattern suggests that the rows of the inverse matrix A−1 are

generated by

f̃ M(s1,s2)

(z1, z2) =1

3M(1 − 2z1 + z2)

s1(1 + z1 − 2z2)s2(1 + z1 + z2)

s3 ,

where s1, s2, s3 are nonnegative integers such that s1 + s2 + s3 = M.s1, s2, s3 are the same as the r1, r2, r3 used to generate the correspondingrow of the original eigenvector matrix A.

This function is also a valid solution to our partial differential equation.


The d -urn inverse matrix

Theorem

The rows of the matrix A−1 are the coefficients of

f̃ Ms (z) =

1

dM

d−1∏

k=1

(1 − (d − 1)zk +∑

i 6=k

zi)sk

(

1 +

d−1∑

k=1

zk

)M−Pd−1

k=1 sk

,

where the row corresponding to the vector s in f̃ Ms (z) is in the same

position as the row corresponding to r in f Mr (z).


Proof: f̃Ms (z) generates A

−1

To prove that f̃ Ms (z) generates A

−1, let the coefficients of f Mr (z) and

f̃ Ms (z) be a

(M)r,s and 1

dM b(M)s,t , respectively, so that 1

dM B is the matrix

generated by f̃ Ms (z).


Proof: f̃Ms (z) generates A

−1

To prove that f̃ Ms (z) generates A

−1, let the coefficients of f Mr (z) and

f̃ Ms (z) be a

(M)r,s and 1

dM b(M)s,t , respectively, so that 1

dM B is the matrix

generated by f̃ Ms (z).

It is sufficient to prove that:

1

dM

∑

s

a(M)r,s b

(M)s,t =

{

1 if r = t,

0 otherwise.

This condition can be proved by induction on M.


Computational complexity

Using diagonalization, we can compute the n-step transition matrix Pn

more efficiently. Assuming d << M,





Ordinary matrix multiplication: ∼ nM3(d−1) operations

Diagonalization: ∼ nM(d−1) + 2M3(d−1) operations





Ordinary matrix multiplication: ∼ nM3(d−1) operations

Diagonalization: ∼ nM(d−1) + 2M3(d−1) operations

Steps

Balls

100 200 400

10 0.637 0.786 0.806

20 26.861 29.771 34.201

(Ordinary matrix multiplication)

Steps

Balls

100 200 400

10 0.215 0.229 0.234

20 9.017 9.453 9.655

(Diagonalization)


Mixing times


Mixing times

The total variation distance between a probability distribution p and thestationary distribution π is given by

dTV (p,π) =1

2

∑

i

|pi − πi |.


Mixing times

The total variation distance between a probability distribution p and thestationary distribution π is given by

dTV (p,π) =1

2

∑

i

|pi − πi |.

The mixing time of a Markov chain is the minimum number of steps ittakes for the total variation distance to be less than a given ǫ:

tmix(ǫ) = min{n : dTV (Pn,π) ≤ ǫ}


Mixing time bounds

We can bound the mixing time of the Ehrenfest urn model as follows:

ln( 12ǫ

)( 11−λ∗

− 1) ≤ tmix(ǫ) ≤ −ln(ǫπmin)(1

1−λ∗

)

where

λ∗ = 1 − d(d−1)M

is the second-largest eigenvalue of P and

πmin = 1dM .


Mixing times: example

Consider d = 3 urns, M = 20 balls, ǫ = 0.25. If we start with all the ballsin one urn, tmix = 30.


Mixing times: example

Consider d = 3 urns, M = 20 balls, ǫ = 0.25. If we start with all the ballsin one urn, tmix = 30.

The bounds on the mixing time are approximately

8 ≤ tmix ≤ 312.


Mixing times for the multiple-urn Ehrenfest model

Using the eigenanalysis, we can precisely quantify the mixing times for themultiple urn Ehrenfest model.




Beginning with all balls in an urn, and with the choice of ǫ = 0.25, we have

M tmix(ǫ) dTV (Pn,π)

5 5 0.2305

10 13 0.2177

20 30 0.239

40 70 0.2407

80 158 0.2476




Beginning with all balls in an urn, and with the choice of ǫ = 0.25, we have

M tmix(ǫ) dTV (Pn,π)

5 5 0.2305

10 13 0.2177

20 30 0.239

40 70 0.2407

80 158 0.2476

When M = 80, the dimensions of the transition matrix P are(82

2

)

×(82

2

)

,or 3321 × 3321. Even equipped with the eigenanalysis, it took R morethan 5 hours to compute the total variation distance.


Hitting times

For a Markov chain Xn, n ≥ 0, the hitting time (or first passage time)from state r to state s is the minimum number of steps the chain takes toreach state s for the first time when the chain initially starts at state r.The expected value of such a hitting time is denoted by Er [Ts ], where

Ts = min{n ≥ 0 : Xn = s,Xi 6= s, for all i < n}.


Hitting times


Hitting times

We have investigated the following hitting times associated with a multipleurn Ehrenfest model.


Hitting times


(1) How long does it take to empty a full urn?


Hitting times



(2) How long does it take to fill a specific empty urn?


Hitting times



(2) How long does it take to fill a specific empty urn?

(3) How long does it take to transfer all balls in a full urn to an emptyurn?


Computing expected hitting times

General method for computing expected hitting times:




Er [Ts] = E[Ts |X0 = r]




Er [Ts] = E[Ts |X0 = r]

= 1 +∑

k

E[Ts |X1 = k] × Pr,k




Er [Ts] = E[Ts |X0 = r]

= 1 +∑

k

E[Ts |X1 = k] × Pr,k

If we label τr,s = Er [Ts], it amounts to solve the linear system

τr,s = 1 +∑

k

τk,s × Pr,k,

and the number of unknowns τr,s equals the square of the size of the statespace of the Markov chain.


Emptying a full urn, our method



Given a specific urn containing M − k balls, let Tk be the first time ittakes for the urn to have M − k − 1 balls.



Given a specific urn containing M − k balls, let Tk be the first time ittakes for the urn to have M − k − 1 balls.

Tk =

1 with probability M−kM

,

1 + T ′k with probability k(d−2)

M(d−1) ,

1 + Tk−1 + T ′k with probability k

M(d−1) ,

where T ′k is a random variable with the same distribution as Tk .


Emptying a full urn

We find that

E[Tk ] =1

(

M−1k

)

k∑

j=0

(

Mj

)

(d − 1)k−j


Emptying a full urn

We find that

E[Tk ] =1

(

M−1k

)

k∑

j=0

(

Mj

)

(d − 1)k−j= M

∫ 1

0xM−k−1

(

d − x

d − 1

)k

dx ,


Emptying a full urn

We find that

E[Tk ] =1

(

M−1k

)

k∑

j=0

(

Mj

)

(d − 1)k−j= M

∫ 1

0xM−k−1

(

d − x

d − 1

)k

dx ,

which gives us

Efull [Tempty ] =

M−1∑

k=0

E[Tk ]


Emptying a full urn

We find that

E[Tk ] =1

(

M−1k

)

k∑

j=0

(

Mj

)

(d − 1)k−j= M

∫ 1

0xM−k−1

(

d − x

d − 1

)k

dx ,

which gives us

Efull [Tempty ] =

M−1∑

k=0

E[Tk ] = M

M−1∑

k=0

(

dd−1

)k

k + 1.


Filling an empty urn

Redefining Tk as the time to have k + 1 balls in an urn initially containingk balls,

E[Tk ] =d − 1(

M−1k

)

k∑

j=0

(

M

j

)

(d − 1)k−j .




E[Tk ] =d − 1(

M−1k

)

k∑

j=0

(

M

j

)

(d − 1)k−j .

Using the same method, we find that the expected time to fill a specificempty urn is

Eempty [Tfull ] = M(d − 1)M−1∑

k=0

dk

k + 1.




E[Tk ] =d − 1(

M−1k

)

k∑

j=0

(

M

j

)

(d − 1)k−j .

Using the same method, we find that the expected time to fill a specificempty urn is

Eempty [Tfull ] = M(d − 1)M−1∑

k=0

dk

k + 1.

Starting with a full urn, the time to fill any of the other urns is

Efull1 [Tfullany] = M

M−1∑

k=0

dk

k + 1.


Hitting time graphical representation

blue: time to empty a specific urnorange: time to fill a specific urn


Summary of results


Summary of results

(1) Developed a generating function f Mr for the eigenvectors of the

transition matrix P, first for the 3-urn model, then generalized for d

urns. f Mr also allows us to solve for the eigenvalues λ.


Summary of results




(2) Proved that f̃ Ms generates the inverse of the eigenvector matrix A,

both for the 3-urn model and a generalized d-urn model.


Summary of results






(3) Explored an application of the eigenanalysis involving bounding themixing time of the Ehrenfest urn model.


Summary of results






(3) Explored an application of the eigenanalysis involving bounding themixing time of the Ehrenfest urn model.

(4) Solved for general formulae for hitting times for various scenarios ofthe Ehrenfest urn model.


Works cited

Blom, Gunnar. ”Mean Transition Times for the Ehrenfest urn Model.”Advances in Applied Probability. 21.2 (1989): 479-480. Web.

Blum, Avrim, John Hopcroft, and Ravindran Kannan. ”Chapter 5:Random Walks and Markov Chains.” Foundations of Data Science.

Unpublished.Grinstead, Charles M., and J. Laurie Snell. ”Chapter 11: Markov Chains.”

Introduction to Probability. Unpublished.Haggstrom, Olle. Finite Markov Chains and Algorithmic Applications.

New York: Cambridge UP, 2003.Kac, Mark. Probability and Related Topics in Physical Sciences. New

York: Interscience Publishers Inc, 1959. Print.Karlin, Samuel, and James McGregor. ”Ehrenfest Urn Models.” Journal of

Applied Probability 2.2 (1965): 352-376. Web.Levin, Davlid A., Yuval Peres, and Elizabeth L. Wilmer. Markov Chains

and Mixing Times. Unpublished.


the multiple-urn ehrenfest model - willamette …...the multiple-urn ehrenfest model a look into...

Documents