1 value and utility functions - dartmouth collegecushman/courses/engs41/mark borsuk's...

5/11/2015

1

Value and Utility Functions

Probability Theory

Value Functions Definition of a value function

Eliciting value functions

Utility Functions Definition of a utility function

Eliciting utility functions

1

Introduction• Probabilities in Decision Analysis▫ Example: Influence diagram for basic risky decision

5/11/2015

2

Introduction• Probabilities in Decision Analysis▫ Example: Decision tree for basic risky decision

• Probability is the most common and theoretically-supported method for describing uncertainty

• Formulated by Kolmogorov’s axioms (1933):

▫ Event space

▫ Probability function P: () [0,1] :

1. P(A) ≥ 0 A

2. P() = 1

3. P(A B) = P(A) + P(B) if A B =

Basics of Probability Theory

5/11/2015

3

• Definitions

▫ Conditional probability:

▫ Independence:)(

)()(

BP

BAPBAP

)()( APBAP

)()()( BPAPBAP


• Reversing conditional probabilities:

▫ Conditional probability:

▫ Re-formulation:

▫ Bayes’ Rule:

)(

)()(

BP

BAPBAP

)()()()()( APABPBPBAPBAP

)(

)()()(

BP

APABPBAP

n

iii APABPBP

1

)()()(


5/11/2015

4

▫ To apply the mathematical theory of probability to aspects of the real world we need a “physical” or “psychological” interpretation.

▫ At least two different interpretations have been suggested (both of which adhere to Kolmogorov’s axioms):

Frequentist interpretation

Personalist (subjective or Bayesian) interpretation

Interpretations of Probability

• Frequentist Interpretation

▫ The probability of an event represents the long-run frequency of similar events over repeated observations

▫ Useful for statistical inference about underlying regularities in random systems based on repeated measurements

▫ Strictly applies only to learningfrom data


5/11/2015

5

• But what is the relevant population of similar events for:

▫ Tomorrow’s weather?

▫ Next week’s playoff game?

▫ Next month’s court case?

▫ An earthquake at a nuclear waste storage facility?


• Subjective Interpretation

▫ The probability of an event represents the “degree of belief” that a person has that the event will occur, given all relevant information currently known to that person.

▫ Explicitly a function of the state of information

▫ Frequencies may be one useful piece of information.

▫ Two or more people can rationally disagree.

▫ “Probability does not exist.” –Bruno de Finetti


5/11/2015

6

• Subjective Interpretation (cont’d)

▫ Still adheres to Kolmogorov’s axioms (must be coherent)

▫ Bayes’ rule becomes a method for updating beliefs after obtaining new data (information, evidence, etc.)

)(

)()()(

eP

APAePeAP

Constant gNormalizin

PriorLikelihoodPosterior


• Your aging pet cat seems to not be feeling well. After speaking with some friends you become concerned that she may have chronic kidney disease. If so, then you would need to begin to feed her a special diet. • Based on the comments of your friends, you assess a 10%

probability that she has the disease, but would like to take her to the vet for a test.• The vet performs a blood test and tells you that it came back

positive.• You ask the vet how accurate the test is and he tells you that

when a cat actually has the disease, a positive result will occur 99% of the time, while if the cat does not have the disease a false positive may occur 15% of the time.• What is the probability that your cat has kidney disease?

10%? 99%? 15%? Or something else?

Example of Bayes’ Rule

5/11/2015

7

What is the probability that your cat has kidney disease given the test results?D = has disease + = test positiveD’ = no disease - = test negative

• We want P(D|+), which by Bayes’ rule is:P(+|D) = 0.99P(D) = 0.10

P(+|D’) = 0.15

)'()'|()()|(

)()()(1

DPDPDPDP

DPDPPn

iii

)(

)()()(

P

DPDPDP



423.0135.0099.0

099.0

)10.01(15.010.099.0

10.099.0

)(

)()()(

P

DPDPDP

What is the probability that your cat has kidney disease given the test results?

5/11/2015

8

• Subjective Interpretation (cont’d)

▫ The subjective interpretation of probability includes the frequentist interpretation as a limiting case, because for random events it is reasonable to believe that the chance for the next outcome is given by the frequentist probability.

▫ On the other hand, the subjective interpretation has a much wider field of application, as it can be used to describe knowledge or belief about non-repeatable and non-random events.


Eliciting Probabilities• Elicitation of probabilities

▫ Subjective probabilities can be assessed by elicitation.

▫ They can be made more tractable by comparison with hypothetical bets on lotteries of outcomes.

▫ Formally describes existing knowledge

▫ Can be used to describe knowledge or belief about non-repeatable and non-random events.

▫ Constitutes a “prior”, to be used either on its own or updated using data through a Bayesian process.

5/11/2015

9

Reintroduced tiger will survive in the wildTiger survives $X

Lottery A

Tiger does not survive $0

25% $100

Lottery B

75% $0

Method

1. Set up Lottery B as a reference lottery with a well-specified probability mechanism

2. Adjust the value of the winnings until the decision maker is indifferent

p x X = 0.25 x 100 + .75 x 0

p = 25/X

Fixed Probability Method

Method (cont’d)

3. Use wide brackets and converge slowly

4. Check for consistency

Reintroduced tiger will survive in the wildTiger survives $X

Lottery A


25% $100

Lottery B

75% $0

Fixed Probability Method

5/11/2015

10

Reintroduced tiger will survive in the wildTiger survives $100

Lottery A


p $100

Lottery B

(1-p) $0

Method

1. Set up Lottery B as a reference lottery with a well-specified probability mechanism

2. Adjust the probability of the reference lottery until the decision maker is indifferent

3. Use wide brackets and converge slowly

4. Check for consistency

Fixed Value Method

Reintroduced tiger will survive in the wildTiger survives $100

Lottery A


p $100

Lottery B

(1-p) $0

Problems

1. People may not like the idea of a game

2. People may have a hard time understanding the game

3. Problems such as anchoring may affect results

Fixed Value Method

5/11/2015

11

• Find a partner such that at least one of you is actively looking for a job or summer internship. This person will be the ‘expert’ and the other person will be the ‘analyst’.

• The analyst should elicit from the expert the probability that he/she will have found and committed to a job by June 1, 2014.

• Probability Density Function (PDF)

Probability distribution of X describes the probability of possible results of X

variablerandom continuous a is ifd)(

variablerandom discrete a is if)(

],[Xxxf

XxP

xxP r

l

ril

x

x

xxxi

rlX

Distributions

5/11/2015

12

Distributions• Cumulative Distribution Function (CDF)

Distribution function: Probability for values less than or equal to x:

variablerandom continuous a is if'd)'(

variablerandom discrete a is if)(

Xxxf

XxP

xF xxx

i

X

i

How do we elicit subjective probabilities?

Continuous quantities1. Assess a number of cumulative probabilities to

derive a CDF

2. Assess “ends” and middle of distribution, then a number of midpoints of segments

3. Calculate the derivative of the CDF to get the PDF

Eliciting Probabilities

5/11/2015

13

Climate Sensitivity

• Climate sensitivity, S, is defined as the equilibrium global temperature change in response to a doubling in radiative forcing.• Climate sensitivity is a

variable about which there is much uncertainty.

1. Use reference lotteries to assess a number of cumulative probabilities to derive a CDF

S < 2.0 deg C $100

Lottery A

S > 2.0 deg C $0

p $100

Lottery B

(1-p) $0

Eliciting ProbabilitiesContinuous Quantities

5/11/2015

14

2. Assess “ends” of distribution (0.10 and 0.90 fractiles) and middle (median) by fixing p and assessing corresponding S. Then assess a number of midpoints of segments.

S < x deg C $100

Lottery A

S > x deg C $0

0.90 $100

Lottery B

0.10 $0


P(S < 1.25) = 0.10P(S < 2.0) = 0.30P(S < 2.25) = 0.5P(S < 2.30) = 0.70P(S < 2.50) = 0.90

P (Delta T < deg)

0.00

0.25

0.50

0.75

1.00

0.50

1.00

1.50

2.00

2.50

3.00

Degrees

Cu

m.

Pro

b.


Can then take the derivative to get a probability density function.

P(S < x degrees)

5/11/2015

15

Elicitation ExerciseAssess a continuous probability distribution for the amount of snow we will receive in Hanover, NH, next winter (in whatever units you prefer). Do this with a partner using the reference lottery approach. Use either the fixed probability or fixed values approach (you and your partner might each try a different method). Draw your CDF on graph paper. Using slopes, convert this to an approximate PDF. Your challenge is for your distribution to be neither too narrow (i.e., overconfident) nor too wide (underconfident).


Probability Theory





30

5/11/2015

16

• Preferences and Value Functions

21 yy means that the outcome y1 is preferredto the outcome y2.

We want to analyse preferences of possible consequences of management alternatives.

The alternative leading to the most preferred consequences is the one to be chosen.

To find the preferred alternative we need:

an outcome model for decision alternatives, and

a characterization of preferences over consequences.

Preferences are defined for pairs of possible consequences.

Value Functions

31


Definition of a value function:

An value function is a function which assigns each outcome, y1, a real number v(y1), which is larger then v(y2) if y1 is preferred to y2.

Note that in economics, value functions are often called “utility functions.” In the decision sciences, the name utility function is reserved for the description of preferences under uncertainty/risk.

Value Functions

32

5/11/2015

17

313221 thenandif yyyyyy

211221 ororeither yyyyyy

Existence of a value function:

For each preference that is

transitive ( ) and

complete ( )

there exists an ordinal value function representing the preference.


Value Functions

33

• Preferences and Value FunctionsNon-uniqueness of value functions: Value functions are unique up to an

affine transformation (i.e., linear and shift).

Typically scaled to range from 0 to 1.

Interpretation of value functions: Existence as a mathematical

description of preferences, not as a model of thinking of a human decision maker (decision makers do not explicitly maximise values).

Construction of value functions by analysis of preferences.

Value Functions

34

5/11/2015

18


Human preferences are not always transitive and complete; the assumptions can be questioned for a behavioural decision theory. However, the assumptions seem reasonable for a prescriptive theory.

Completeness may limit the field of application of decision theory (no alternatives with “extreme” outcomes).

Value Functions

35

• Elicitation of a Value Function

Single Attribute Case:

Ask for preferences between exemplary outcomes, and construct the value function by interpolation or smoothing.

Do not ask for shape parameters of the value function directly (value functions are mathematical representations of preferences not “stored” in this form by the decision maker).

Value functions are often normalised to the interval [0,1].

After eliciting a value function, consistency checks have to be performed.

Value Functions

36

5/11/2015

19

• Elicitation of a Measurable Value Function


Direct Rating (not recommended)Ordering of outcomes; specification of values compatible with ordering and with preferences about switching between outcomes.

Difference Standard Sequence TechniqueSet value of worst outcome to zero; set value of outcome with attribute value about 20% of span larger to unity; determine subsequent outcome values so that value increments are unity; normalise value function.

Bisection Method (Midvalue Splitting Technique)Determine worst and best outcome, set values to 0 and 1; identify outcome with value of 0.5; identify outcomes with values of 0.25 and 0.75.

Value Functions

37

Direct Rating – Continuous Case

Ordering of outcomes; specification of values compatible with ordering and with preferences about switching between outcomes

Value Functions

38

5/11/2015

20

Direct Rating – Discrete Case

For a job choice, the location alternatives (from worst to best) are:

Ordering of outcomes; specification of values compatible with ordering and with preferences about switching between outcomes

Value Functions

39

Difference Standard Sequence Technique

1.0

1.0

1.0

1.0

1.0

1

0.8

0.6

0.4

0.2

0

Set value of worst outcome to zero; set value of outcome with attribute value about 20% of span larger to unity; determine subsequent outcome values so that value increments are unity; normalise value function.

Value Functions

40

5/11/2015

21

Difference Standard Sequence Technique to assessValue function for “time spent on a cruise ship”

1

0.857

0.714

0.571

0.429

0.286

0.143

0

Value Functions

41

Bisection Method (Midvalue Splitting Technique)

Determine worst and best outcome, set values to 0 and 1; identify outcome with value of 0.5; identify outcomes with values of 0.25 and 0.75.

Value Functions

42

5/11/2015

22

Typical shape of a single attribute value function:

Decreasing marginal value:

Of special importance for attributes measured in monetary units

Value Functions

43

• Other Parametric Shapes

Value Functions

44

5/11/2015

23

Break and Practice Exercise• Assess a continuous value function for the amount of

sleep you need per day. • Do this with a partner using the Difference Standard

Sequence Technique or the Bisection Method.

45

Summary of Value Functions• Decisions under Certainty with Value Functions

Assumptions:▫ An outcome prediction model gives certain results for all

attributes for all decision alternatives.▫ The value function is elicited from the decision maker as a

function of the attributes (preferences fulfil axioms).

Values can be calculated for all decision alternatives.

Decreasing values lead to a preference ranking of the alternatives.

46

5/11/2015

24

Utility functions should be considered when there is uncertainty and a non-neutral attitude toward risk

Decisions nearly always have uncertainty, however value functions can still be useful:

Value functions are easier to elicit then utility functions;

The application of value functions already leads to a clear separation of predictions from valuations;

Development of the theory for decisions under certainty is a useful step towards the theory under uncertainty/risk;

Uncertainty can still be (partially) taken into account through sensitivity analysis.

Value Functions

47


Probability Theory





48

5/11/2015

25

a probabilistic outcome model (yielding a probability of each possible outcome for each decision alternative)

appropriate characterization of preferences under uncertainty (accounting for attitudes toward risk)

Most decisions are made under uncertainty, therefore we need:

Utility Functions

49

Lotteries

p = 0.5 $1000

p = 0.5 - $1000

?

Would you pay to participate in this lottery?

If not, how much would you need to be paid?

Utility Functions

50

5/11/2015

26

• Preferences and Utility Functions

Definition of a utility function:

A utility function describes preferences for both outcomes and risk such that overall preference between actions with uncertain outcomes is reflected by the expected values of the utilities for each outcome.

This is done by replacing outcomes in the value function setting with lotteries of outcomes, designated Li

p y1

1 – p y2

Utility Functions

51

• Axioms for Expected Utility:

1. Preferences among uncertain events called lotteries Li

can be specified based on consequences and associated probabilities.

2. These preferences fulfill the following axioms:

1. Transitivity and completeness

2. Continuity

3. Independence (substitutability)

313221 thenandIf LLLLLL

211221 ororeither Also, LLLLLL

312321 )1(:thenIf LpLpLpLLL

3323121 )1()1(thenIf LLpLpLpLpLL

Utility Functions

52

5/11/2015

27

• Expected Utility Theory

If these assumptions hold, then…

1. It is possible to construct a utility function to represent your preferences for uncertain outcomes, such that…

2. Your overall preference between lotteries of uncertain outcomes is reflected by the expected utilities of each lottery

p1 U(y1)

1 – p1 U(y2)

p2 U(y3)

1 – p2 U(y4)

Utility Functions

53

• Interpretation of Utility Functions

Existence as a mathematical description of preference under uncertainty, not as a model of thinking of a human decision maker (decision makers do not explicitly maximize utility)

Construction of utility functions by analysis of preferences under uncertainty for an individual

The shape of U(y) is a result of both the shape of the value function and risk attitudes

Utility Functions

54

5/11/2015

28

• Typical risk averse utility function

0

1

Utility

Attributeattribute range

Utility Functions

55

0.96

0.36

0.66

• Risk aversion as a positive risk premium

CertaintyEquivalent

ExpectedOutcome

Risk Premium

Outcome ($000)

CE < E(X) Risk Aversion

ExpectedUtility

Utility Functions

56

5/11/2015

29

• Other attitudes toward risk

Outcome

Utility Functions

57

• Non-uniqueness of utility functions

Utility functions are only ordinal (they do not express strengthof preference)

Utility functions are only unique up to an affine transformation (linear and shift).

Utilities are not absolute, but depend on the attribute ranges

Utility Functions

58

5/11/2015

30

• Elicitation of a Utility Function

Principles:

Ask for preferences between reference lotteries and certain outcomes

Do not ask for shape parameters of the utility function directly

Utility functions are usually normalised to the interval [0,1]

After eliciting a utility function, consistency checks should be performed

It is often useful to compose a utility function as a function of single attribute “sub-utility functions”

Utility Functions

59



Certainty Equivalent Method

1. Assign outcomes of a 50/50 reference lottery to the best and worst possible levels of the attribute; determine the attribute level for which the decision maker would be indifferent between this level with certainty and the reference lottery; this level is called the “certainty equivalent” for the reference lottery and is assigned a utility of 0.5.

2. Repeat by replacing the worst outcome with the outcome determined in step 1 (with utility = 0.5); the resulting certainty equivalent is assigned a utility of 0.75.

3. Repeat step 1 replacing the best outcome with the outcome with utility = 0.5; the resulting certainty equivalent is assigned a utility of 0.25.

Utility Functions

60

5/11/2015

31

• Certainty equivalent method

p = 0.5 $100

Lottery A

p = 0.5 $ 10

Certainty BCE

U(CE) = 0.5 U(100) + 0.5 U(10)

= 0.5 (1) + 0.5 (0)

= 0.5

Utility Functions

61


p = 0.5 $100

Lottery A

p = 0.5 CE’

Certainty BCE

U(CE) = 0.5 U(100) + 0.5 U(CE’)

= 0.5 (1) + 0.5 (0.5)

= 0.75

Utility Functions

62

5/11/2015

32


0

1

Utility


0.75

0.5

0.25

Utility Functions

63



Probability Method

1. Assign probabilities of a reference lottery values of p and 1-p for the outcomes with the best and worst possible levels of the attribute respectively; set the certainty equivalent to an intermediate value; determine the value of p for which the decision maker would be indifferent between the certainty equivalent and the reference lottery; the value of p is equal to the utility of the certainty equivalent.

2. Repeat with other intermediate values for the certainty equivalent.

Utility Functions

64

5/11/2015

33

p $100

Lottery A

1-p $10

Certainty B

• Probability method

$65

U(65) = p U(100) + (1-p) U(10)

= p (1) + (1-p) (0)

= p

Utility Functions

65

• Probability method

0

1

Utility


0.8

0.6

0.4

0.2

Utility Functions

66

5/11/2015

34

• Functional Forms

U(x) = log(x)

U(x) = 1-exp(-x/R)

U(x) = sqrt(x)

0

1

Utility


Each of which would then need to be normalized to a [0,1] utility range by application of a linear transformation.

Utility Functions

67

Discrete Attributes• Utility Function Assessment▫ Generalized procedure

68

5/11/2015

35

p San Francisco

Lottery A

1-p Ann Arbor

Certainty B Boston

U(BO) = p U(SF) + (1-p) U(AA)

= p (1) + (1-p) (0)

= p

Discrete Attributes• Utility Function Assessment▫ Probability method

69

Practice Exercise

• With a partner, assess each other’s utility function for money over the range $0 to $1000 using the Certainty Equivalent Method or Probability Method.

70

1 value and utility functions - dartmouth collegecushman/courses/engs41/mark borsuk's...

Documents