1

Chapter 15 Chapter 15 Probabilistic Reasoning over TimeProbabilistic Reasoning over Time

2

Outline Outline

• Time and UncertaintyTime and Uncertainty

• Inference: Filtering, Prediction, SmoothingInference: Filtering, Prediction, Smoothing

• Hidden Markov modelsHidden Markov models

• Brief Introduction to Kalman FiltersBrief Introduction to Kalman Filters

• Dynamic Bayesian networksDynamic Bayesian networks

• Particle FilteringParticle Filtering

3

Time and uncertaintyTime and uncertainty

• The world changes; we need to track and predict it

• Diabetes management vs vehicle diagnosis

• Basic idea: copy state and evidence variables for each time step

4

Markov processes (Markov chains)Markov processes (Markov chains)

5

ExampleExample

6

Inference tasksInference tasks

tt

7

FilteringFiltering

8

Filtering exampleFiltering example

0

1 1 0 0

1 1 1 1 1

Day 1, U1=true

( ) ( | ) ( )

0.7,0.3 0.5 0.3,0.7 0.5 0.5,0.5

( | ) ( | ) ( ) 0.9,0.2 0.5,0.5

0.45,0.1 0.818,0.182

r

p R p R r p r

p R u p u R p R

Rt P(Ut)

t 0.9

f 0.2

Rt-1 P(Rt)

t 0.7

f 0.3

9

Filtering exampleFiltering example

1

2 1 2 1 1 1

2 1 2 2 2 2 1

Day 2, U2=true

( | ) ( | ) ( | )

0.7,0.3 0.818 0.3,0.7 0.182 0.627,0.373

( |, , ) ( | ) ( | ) 0.9,0.2 0.627,0.373

0.565,0.075 0.883,0.117

r

p R u p R r p r u

p R u u p u R p R u

Rt-1 P(Rt)

t 0.7

f 0.3

Rt P(Ut)

t 0.9

f 0.2

10

SmoothingSmoothing

11

Smoothing exampleSmoothing example

2

2 1 2 2 2 2 1(u | R ) ( | ) (| )p( | )

=0.9 1 0.7,0.3 0.2 1 0.3,0.7 0.69,0.41

r

p p u r p r r R

Rt-1 P(Rt)

t 0.7

f 0.3

Rt P(Ut)

t 0.9

f 0.2

12

Most likely explanationMost likely explanation

13

Viterbi exampleViterbi example

14

Hidden Markov modelsHidden Markov models

:

: :

( | ) ( | ) ( | ) ( | )

( | )

1 1 1 1 1 1

1 1

tt t t t t t t tx

t t t

P X e P e X P X x P x e

f P x e

15

Country dance algorithmCountry dance algorithm

16

Country dance algorithmCountry dance algorithm

17

Country dance algorithmCountry dance algorithm

18

Country dance algorithmCountry dance algorithm

19

Kalman FiltersKalman Filters

20

Updating Gaussian distributionsUpdating Gaussian distributions

21

Simple 1-D exampleSimple 1-D example

22

General Kalman updateGeneral Kalman update

23

2-D tracking example: Filtering2-D tracking example: Filtering

24

2-D tracking example: smoothing2-D tracking example: smoothing

25

Where it breaksWhere it breaks

26

27

Dynamic Bayesian networksDynamic Bayesian networks

28

DBNs vs. HMMsDBNs vs. HMMs

29

DBNs vs Kalman FiltersDBNs vs Kalman Filters

30

Exact inference in DBNsExact inference in DBNs

31

Likelihood weighting for DBNsLikelihood weighting for DBNs

32

Particle FilteringParticle Filtering

33

Particle Filtering contd.Particle Filtering contd.

34

Particle ltering performanceParticle ltering performance

35

Chapter 15, Sections 1-5 Chapter 15, Sections 1-5 SummarySummary