solutions homework #2 - eth z · pdf file1 solutions homework #2 carlos cotrini october 27,...

1

Solutions homework #2Carlos Cotrini

October 27, 2017

Probabilistic foundations of artificial intelligence

2

1. Bayesian Networks: d-separation● Solutions task 1

A

B

C

D

F

E

G

H

I

J

B indep. I given F?

3

Principle of d-separation● Given a set of observed variables, if there is no

active trail between two variables, then they are independent.

B

A

C

D

B

A

C

D

B indep from C | A Unclear if B indep from C | A, D

4

1. Bayesian Networks: d-separation● Recap on active trails. Case 1. (Mountain)

Observed variable

Hidden variable

Observed orhidden variable

Inactive!

5

1. Bayesian Networks: d-separation● Recap on active trails. Case 2a. (Downhill)

Inactive!

6

1. Bayesian Networks: d-separation● Recap on active trails. Case 2b. (Uphill)

Inactive!

7

1. Bayesian Networks: d-separation● Recap on active trails. Case 3. (Valley)

Inactive!

trail

8

1. Bayesian Networks: d-separation● Recap on active trails. Case 3.

Inactive!

trail

9


Inactive!

trail

10


Inactive!

trail

11


A

B

C

D

F

E

G

H

I

J

12


A

B

C

D

F

E

G

H

I

J

A indep. F?

13


A

B

C

D

F

E

G

H

I

J

A indep. F?

14


A

B

C

D

F

E

G

H

I

J

A indep. F?

15


A

B

C

D

F

E

G

H

I

J

A indep. F?

INCONCLUSIVE

16


A

B

C

D

F

E

G

H

I

J

A indep. G?

17


A

B

C

D

F

E

G

H

I

J

A indep. G?

18


A

B

C

D

F

E

G

H

I

J

A indep. G?

19


A

B

C

D

F

E

G

H

I

J

A indep. G?

YES!

20


A

B

C

D

F

E

G

H

I

J

B indep. I given F?

21


A

B

C

D

F

E

G

H

I

J

B indep. I given F?

22


A

B

C

D

F

E

G

H

I

J

B indep. I given F?

INCONCLUSIVE

23


A

B

C

D

F

E

G

H

I

J

D indep. J given G, H?

24


A

B

C

D

F

E

G

H

I

J


25


A

B

C

D

F

E

G

H

I

J


26


A

B

C

D

F

E

G

H

I

J


YES!

27


A

B

C

D

F

E

G

H

I

J

I indep. B given H?

28


A

B

C

D

F

E

G

H

I

J

I indep. B given H?

YES!

29


A

B

C

D

F

E

G

H

I

J

D indep. J?

30


A

B

C

D

F

E

G

H

I

J

D indep. J?

INCONCLUSIVE

31


A

B

C

D

F

E

G

H

I

J

I indep. C given H, F?

32


A

B

C

D

F

E

G

H

I

J

I indep. C given H, F?

INCONCLUSIVE

33

Be careful with the direction of the arrows!

B

A

C

D

B indep from C ?

34

Be careful with the direction of the arrows!

B

A

C

D

A indep from D ?

35

2. Variable elimination● Compute

P(J = j)

AB

C

D

F

E

G

H

I

J

36


P(J = j)

AB

C

D

F

E

G

H

I

J

37


P(J = j)

AB

C

D

F

E

G

H

I

J

38


P(J = j)

AB

C

D

F

E

G

H

I

J

39


P(J = j)

AB

C

D

F

E

G

H

I

J

40


P(J = j)

AB

C

D

F

E

G

H

I

J

41


P(J = j)

AB

C

D

F

E

G

H

I

J

1

3. An algorithm for d-separation● Given a Bayesian Network, a variable X, and a

set of variables E with observed values e, compute all variables Z that are indep. from X given E.

2

3. Algorithm for d-separation

W

S

U

X

I

A

P

Q

EB

R

K

XXXX

3


W

S

U

X

I

A

P

Q

EB

R

K

XXXX

D-reachable: if it can be reached with an active trail from X

d-reachable

4


W

S

U

X

I

A

P

Q

EB

R

K

XXXX

5


W

S

U

X

I

A

P

Q

EB

R

K

XXXX

6


Key insight: Subtrails of active trails are also active!

7


Key insight: Subtrails of active trails are also active!

Compute d-reachable variables with a depth-first (or breadth-first) search.

8


W

S

U

Y

I

A

P

Q

EB

R

KtoVisit = [X]

X

d-reachable = {}

9


W

S

U

Y

I

A

P

Q

EB

R

KtoVisit = []

X

d-reachable = {X}

10


W

S

U

Y

I

A

P

Q

EB

R

KtoVisit = [A]

X

d-reachable = {X}

11


W

S

U

Y

I

A

P

Q

EB

R

KtoVisit = [S, A]

X

d-reachable = {X}

12


W

S

U

Y

I

A

P

Q

EB

R

KtoVisit = [I, S, A]

X

d-reachable = {X}

13


W

S

U

Y

I

A

P

Q

EB

R

KtoVisit = [S, A]

YX

d-reachable = {X}

14


W

S

U

Y

I

A

P

Q

EB

R

KtoVisit = [S, A]

YX

d-reachable = {X}

15


W

S

U

I

A

P

Q

B

R

KtoVisit = [A]

Y

E

YX

d-reachable = {X, S}

16


W

S

U

I

A

P

Q

B

R

KtoVisit = [W, A]

Y

E

YX


17


W

S

U

I

A

P

Q

B

R

KtoVisit = [B, W, A]

Y

E

YX


18


W

S

U

I

A

P

Q

B

R

KtoVisit = [U, B, W, A]

Y

E

YX


19


W

S

U

I

A

P

Q

B

R

KtoVisit = [W, B, A]

E

YYX

d-reachable = {X, S, U}

20


W

S

U

I

A

P

Q

B

R

KtoVisit = [B, A]

E

YYX


21


W

S

U

I

A

P

Q

B

R

KtoVisit = [B, A]

E

YYX


22


W

S

U

I

A

P

Q

B

R

KtoVisit = [A]

E

YYX


23


W

S

U

I

A

P

Q

B

R

KtoVisit = [R, A]

E

YYX


24


W

S

U

I

A

P

Q

B

R

KtoVisit = [A]

E

YYX

d-reachable = {X, S, U, R}

25


W

S

U

E

I

A

P

Q

B

R

KtoVisit = []

E

YX

d-reachable = {X, S, U, R, A}

26


W

S

U

E

I

A

P

Q

B

R

KtoVisit = []

E

YX


27


W

S

U

E

I

A

P

Q

B

R

KtoVisit = [K]

E

YX


28


W

S

U

E

I

A

P

Q

B

R

KtoVisit = []

E

YX

d-reachable = {X, S, U, R, A, K}

29


def get_reachable(X, E):toVisit = [X], visited = {}

while toVisit != []:V = toVisit.pop()

“visit V”

add V to visited

for Y an unvisited neighbor of V:if …. then push Y

Loop’s invariant: Z is in toVisit iff there is an active trail from X to Z

30


def get_reachable(X, E):toVisit = [X], visited = {}, reachable = {}


if V is not obs then add V to reachable

add V to visited



31

What neighbors to append?● Let V be the node currently visited and Y be an

unvisited neighbor.● By our invariant, there is an active trail t from X

to V.● What we need to decide if t.append(Y) active...

32




– V - -> Y or V <- - Y?

33




– V - -> Y or V <- - Y?

– Is V or any of its descendants observed?

34




– V - -> Y or V <- - Y?


– W - -> V or W <- - V? (W is V’s predecesor in t)

35




– V - -> Y or V <- - Y?

Easy to check



36




– V - -> Y or V <- - Y?

Easy to check


Compute in advance the ancestors of all observed variables


37




– V - -> Y or V <- - Y?

Easy to check


Compute in advance the ancestors of all observed variables


Keep track how you reached V

38





add V to visited



39



ancestors_E = computeAncestors(E)



add V to visited



40


def get_reachable(X, E):toVisit = [(<--,X)], visited = {}, reachable = {}

ancestors_E = computeAncestors(E)

while toVisit != []:(dir, V) = toVisit.pop()


add V to visited



41

for Y an unvisited neighbor of V:● If dir == <-- :

● If dir == --> :

42


–

● If dir == --> :– If V --> Y:

●

– If V <-- Y:●

43


– If V is not observed, then push (dir’, Y)

● If dir == --> :– If V --> Y:

● If V is not observed, then push (-->, Y)

– If V <-- Y:● If V is in ancestors_E, then push (<--, Y)

solutions homework #2 - eth z · pdf file1 solutions homework #2 carlos cotrini october 27,...

Documents