1

Russell and Norvig, AIMA : Chapter 15Part A: 15.1 , 15.2

Probabilistic Reasoning over Time

Presented to: Prof. Dr. S. M. Aqil Burney

Presented by: Zain Abbas (MSCS-UBIT)

2

Agenda

Temporal probabilistic agents

Inference: Filtering, prediction,

smoothing and most likely

explanation

Hidden Markov models

Kalman filters

3

Agenda

Temporal probabilistic agents

Inference: Filtering, prediction,

smoothing and most likely

explanation

Hidden Markov models

Kalman filters

4

environmentagent

?

sensors

actuators

t1, t2, t3, …

Temporal Probabilistic Agents

5

Time and Uncertainty

The world changes, we need to track and predict it

Examples: diabetes management, traffic monitoring

Basic idea: copy state and evidence variables for each

time step

Xt – set of unobservable state variables at time t

e.g., BloodSugart, StomachContentst

Et – set of evidence variables at time t

e.g., MeasuredBloodSugart, PulseRatet, FoodEatent

Assumes discrete time step

6

States and Observations

Process of change is viewed as series of snapshots, each

describing the state of the world at a particular time

Each time slice involves a set or random variables indexed

by t:

1. the set of unobservable state variables Xt

2. the set of observable evidence variable Et

The observation at time t is Et = et for some set of values et

The notation Xa:b denotes the set of variables from Xa to Xb

7

Markov Assumption Current state depends on only a finite history of

previous states

i.e. Xt depends on some bounded set of X0:t-1

First-order Markov process

P(Xt|X0:t-1) = P(Xt|Xt-1)

Second-order Markov process

P(Xt|X0:t-1) = P(Xt|Xt-2, Xt-1)

Markov processes

8

Sensor Markov assumption

P(Et|X0:t, E0:t-1) = P(Et|Xt)

Assume stationary process

transition model P(Xt|Xt-1) and sensor model

P(Et|Xt) are the same for all t

In a stationary process, the changes in

the world state are governed by laws that

do not themselves change over time

Markov processes

9

Example

10

Complete Joint Distribution

Given: Transition model: P(Xt|Xt-1)

Sensor model: P(Et|Xt)

Prior probability: P(X0) Then we can specify complete joint

distribution of over all variables :

t

iiiiitt XEPXXPXPEEXXXP

110110 )|()|()(),...,,,...,,(

11

Agenda

Temporal probabilistic agents

Inference: Filtering, prediction,

smoothing and most likely

explanation

Hidden Markov models

Kalman filters

12

Inference in Temporal Models

Having set up the structure of a generic temporal model, we can formulate the basic inference tasks that must be solved.

They are as follows: Filtering or Monitoring Prediction Smoothing or Hindsight Most likely explanation

13

What is the probability that it is raining today, given all the umbrella observations up through today?

P(Xt|e1:t) - computing current belief state, given all evidence to date

Filtering or Monitoring

14

Filtering or Monitoring

Transition modelPosterior

distribution at time t

Prediction

Sensor model

Filtering

15

Proof

Forward Algorithm

Bayes Rule

Chain Rule

Conditional Independence

Marginal Probability

Chain Rule

Conditional Independence

Dividing the evidence

t

t

t

xtttttt

ttx

ttttt

xttttt

tttt

ttttt

ttt

ttttt

exPxXPXeP

exPexXPXeP

exXPXeP

eXPXeP

eXPeXeP

eeXP

eeXPeXP

)|()|()|(

)|(),|()|(

)|,()|(

)|()|(

)|(),|(

)|,(

),|()|(

:1111

:1:1111

:1111

:1111

:11:111

:111

1:111:11

16

What is the probability that it will rain the day after tomorrow, given all the umbrella observations up through today?

P(Xt+k|e1:t) computing probability of some future state

Prediction

17

Example

18

Example

Let us illustrate the filtering process for two steps in the basic umbrella example

Assume that security guard has some prior belief as to whether it rained on day 0, just before the observation sequence begins. Let's suppose this is P(R0)=<0.5,0.5>

On day 1, the umbrella appears, so U1= true. The prediction from t=0 to t=1 is

19

Example … continued

Updating the evidence for t=1 gives

Prediction from t=1 to t=2 is

Updating the evidence for t=2 gives

or

o rPrRPRPo

11

5.0,5.0

182.0,818.01.0,45.0

373.0,627.0182.07.0,3.0818.03.0,7.0

117.0,883.0075.0,565.0

20

Example… continued

21

What is the probability that it rained yesterday, given all the umbrella observations through today?

P(Xk|e1:t) 0 ≤ k < tcomputing probability of past state (hindsight)

Smoothing or Hindsight

22

Smoothing or Hindsight

Smoothing computes P(Xk|e1:t), the posterior distribution of the state at some past time k given a complete sequence of observations from 1 to t

23

Smoothing or Hindsight

bk+1:t

Bayes Rule

Chain Rule

Conditional Independence

Dividing the evidence

)|()|(

)|(),|(

)|,(

),|()|(

:1:1

:1:1:1

:1:1

:1:1:1

kkktk

kkkktk

ktkk

tkkktk

eXPXeP

eXPeXeP

eeXP

eeXPeXP

f1:k

24

Smoothing or Hindsight

Sensor modelBackward

message at time

k+1

Sensor model

Backward Message at time k

25

Smoothing or Hindsight

Backward Algorithm

11111:2

111:21

111:1

111:1

11:1:1

)|()|()|(

)|()|,(

)|()|(

)|(),|(

)|,()|(

kkkkkktk

kkkktkk

kkkktk

kkkkktk

kkktkktk

XxPxePxeP

XxPxeeP

XxPxeP

XxPxXeP

XxePXeP

Bayes Rule

Chain Rule

Conditional Independence

Marginal Probability

Conditional Independence

26

Example

27

If the umbrella appeared the first three days but not on the fourth, what is the most likely weather sequence to produce these umbrella sightings?

arg maxx1:xtP(x1:t|e1:t)given sequence of observation, find sequence of states that is most likely to have generated those observations.

Most likely explanation

28

Most likely explanation

Enumeration Enumerate all possible state sequence Compute the joint distribution and find the

sequence with the maximum joint distribution

Smooth Calculate the posterior distribution

for each time step k In each step k, find the state with maximum

posterior distribution Combine these states to form a sequence

)|( :1 tk eXP

29

Most likely explanation

Most likely path to each xt+1 = most likely path to some xt plus one more step

Identical to filtering except f1:t replaced by

)|,(max)|(max)|(

)|,(max

:11:1111

1:11:1

1:1

:1

tttx

ttx

tt

tttx

eXxPxXPXeP

eXxP

tt

t

)|,(max :11:1:11:1

tttx

t eXxPmt

30

Proof… Viterbi Algorithm

)|,(max)|(max)|(

)|,()|(max)|(

)|,()|(max)|(

)|,(),,|(max

)|,,(max

),|,(max

)|,(max

:11:1111

:11:1111

:11:1111

:11:11:1:11

:111:1

1:11:1

1:11:1

1:1

:1

:1

:1

:1

:1

:1

tttx

ttx

tt

tttttx

tt

tttttx

tt

tttttttx

ttttx

ttttx

tttx

exxPxXPXeP

exxPxXPXeP

exxPxXPXeP

eXxPXxeeP

eeXxP

eeXxP

eXxP

tt

t

t

t

t

t

t

Bayes Rule

Chain Rule

Conditional Independence

Divide Evidence

Chain Rule

31

Example