Daphne Koller

Decision Making

MaximumExpectedUtility

ProbabilisticGraphicalModels

Acting

Daphne Koller

Simple Decision MakingA simple decision making situation D:• A set of possible actions Val(A)={a1,

…,aK}• A set of states Val(X) = {x1,…,xN}• A distribution P(X | A)• A utility function U(X, A)

Daphne Koller

Expected Utility

• Want to choose action a that maximizes the expected utility

Daphne Koller

Simple Influence Diagram

Market Found

U

0.30.5

m1m0

0.2

m2

f1f0

000 -7m

0

20m2

5m1

Daphne Koller

IntelligenceDifficulty

Grade

Job

Study

VS

VQ

LetterVG

More Complex Influence Diagram

Daphne Koller

Survey

Market

Found

U

0.30.5

m1m0

0.2

m2

f1f0

000 -7m

0

20m2

5m1

Information Edges

0.10.30.6 0.

3m

0

0.4m2

0.4

m1

0.10.50.3

s0 s1 s2

Decision rule at action node A is a CPD: P(A | Parents(A))

Daphne Koller

Expected Utility with Information

• Want to choose the decision rule A that maximizes the expected utility

Daphne Koller

Survey

Market

Found

U

Finding MEU Decision Rules

Daphne Koller

Survey

Market

Found

U

Finding MEU Decision Rules

0.30.5

m1m0

0.2

m2

f1f0

000 -

1.25s0

2.1s2

1.15

s1

0.10.30.6 0.

3m

0

0.4m2

0.4

m1

0.10.50.3

s0 s1 s2 f1f0

000 -7m

0

20m2

5m1

Daphne Koller

More Generally

Daphne Koller

MEU Algorithm Summary• To compute MEU & optimize decision at A:– Treat A as random variable with arbitrary CPD– Introduce utility factor with scope PaU

– Eliminate all variables except A, Z (A’s parents) to produce factor (A, Z)

– For each z, set:

Daphne Koller

Decision Making under Uncertainty

• MEU principle provides rigorous foundation• PGMs provide structured representation for

probabilities, actions, and utilities• PGM inference methods (VE) can be used for

– Finding the optimal strategy– Determining overall value of the decision situation

• Efficient methods also exist for:– Multiple utility components– Multiple decisions

Daphne Koller

END END END