judgments and decisions psych 253 using decision analysis to answer questions about the value of...

Judgments and DecisionsPsych 253

• Using Decision Analysis to Answer Questions about the Value of Getting Information

• Perfect vs. Imperfect Information• Signal Detection Theory• Medical Example• Jury Decision Making

You can select A, B, or C. Below are the payoffs for each depending on the throw of a die. Which one do you want to have?

Die Roll1 2 3 4 5 6

A $1 $2 $3 $4 $5 $6

B $6 $2 $5 $2 $2 $2

C $7 $5 $4 $4 $2 $1

What should you choose?

Die Roll1 2 3 4 5 6 EV

A $1 $2 $3 $4 $5 $6 $3.50

B $6 $2 $5 $2 $2 $2 $3.17

C $7 $5 $4 $4 $2 $1 $3.83

Which one would you select if you knew what number would come up on the die?

Die Roll1 2 3 4 5 6

A $1 $2 $3 $4 $5 $6

B $6 $2 $5 $2 $2 $2

C $7 $5 $4 $4 $2 $1

What is the expected value of knowing the number that would come up?

1/6*[ $7 + $5 + $5 + $4 + $5 + $6 ] = $5.33

Die Roll1 2 3 4 5 6

A $1 $2 $3 $4 $5 $6

B $6 $2 $5 $2 $2 $2

C $7 $5 $4 $4 $2 $1

The value of the information is

$5.33 - $3.83 = $1.50Value of the

decision WITH

perfect information

Value of the decision

WITHOUT perfect

information

Miracle Movers rents out trucks with a crew of 2 people. One day, Miracle discovers it is a truck short. When this happens, Miracle rents a truck from a local firm. Small trucks cost $130 per day, and large ones cost $200. The advantage of the small truck may vanish if the crew has to make 2 trips. The extra cost of a second trip is $150, and the probability of 2 trips is 40%.

$200 $200

Small Truck

Need Small Truck $130$130

Need Large Truck $280$280

P(NS) =.60

P(NL) =.40

What would it be worth to Michelle to know for sure whether she would need a small truck?

Large Truck

$200 $200

Small Truck

$130$130

$280$280

P(NS) =.60

P(NL) =.40

What would it be worth to Michelle to know for sure whether she would need a small truck?

Large Truck

$200 $200

Small Truck

What would it be worth to Michelle to know for sure whether she would need a small truck?

Large Truck

EV= $190280

Collect Information(that happensto be perfect)

“NS”

“NL”

P(NS) =.60

P(NL) = .40

Small Truck $190

Need Large Truck

Don’t Collect Information

Need Small Truck

Need Large Truck

Need Small Truck

Collect PerfectInformation

“NS”

“NL”

P(NS) =.60

P(NL)=.40Need Large Truck $200

What does Perfect Information Mean?

Need Small Truck $130

P (“NL”|NL) = 1 and P (“NL”|NS) = 0

P(“NS”|NS) = 1 and P (“NS”|NL ) = 0

Value of the decision with perfect information

= p(“NL” and NL)*$200 + p(“NS” and NS)*$130

+ p(“NL” and NS)*$200 + p(“NS” and NL)*$280

But because the information is perfect,

p(“NL” and NS) and p(“NS” and NL) never occur.

So the value of the decision with perfect information is

= p(“NL” and NL)*$200 + p(“NS” and NS)*$130

What is P(“NL” and NL) and P(“NS” and NS)?

P(“NL” and NL) = P(“L”|NL)*P(NL) = 1.0*(.4) =.4

P(“NS” and NS) = P(“S”|NS)*P(NS) = 1.0*(.6)=.6

Value of the decision with perfect info = p(“NL” and NL)*$200 + p(“NS” and NS)*$130

= .4*$200 + .6*$130 = $158

Value of the Decision – Value of the Decision

with perfect info = Value of the Information

Value of the information = $190 - $158 = $32

Perfect information means this…

P(“L”|NS)=0

P(“S”|NL) =0

P(“S”|NS) =1

P(“L”|NL) =1

“S”

NS

NL

“L”Information

Actual

If the information is imperfect, we need to quantify that uncertainty.

P(“L”|NS)=.1

P(“S”|NL)=.2

P(“S”|NS) =.9

P(“L”|NL) =.8

“S”

NS

NL

“L”Information

Actual

Signal Detection Theory:A Theory of Repeated Decision Making

when Information is Fuzzy

-deciding which customers would default on a loan-deciding whether to hire an employee-determining whether someone is lying or not-and any other decision that is made repeatedly using ambiguous evidence

Imagine a doctor who treats patients for a disease. A test for the number of goodies in the patient’s blood is the only predictor or clue he has to predict whether or not the patient has the disease.

From past research, the doctor knows that goodies are normally distributed in the blood of Healthy and Sick people. The average # of goodies in the blood of Healthy people is 100, and the average number of goodies in the blood of Sick people is 115.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

# Goodies in the Blood

Pro

babi

lity

of D

isea

se

Healthy Sick

We can analyze the doctor’s ability to diagnose a patients with Signal Detection Theory. This theory has two parameters:

d´ = d prime - the ability to distinguish Healthy people from Sick people with the blood test

B = beta (also called the threshold or cutoff) - the doctor’s tendency to call a patient “Healthy” or “Sick”

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

# Goodies in the Blood

Pro

babi

lity

of D

isea

se

Healthy Sick

d’

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

# Goodies in the Blood

Pro

bab

ilit

y o

f D

isea

se

HealthySick

B “Sick”“Healthy”

No matter where the doctor sets his cutoff, there are two errors and two correct decisions.

P(“H”|H) P(“S”|H)

P(“H”|S) P(“S”|S)Sick

Healthy

“Healthy” “Sick”

State of the World

Decision

Each outcome has a name.

CorrectRejection

FalseAlarm

Miss HitSick

Healthy

“Healthy” “Sick”

State of the World

Decision

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

# Goodies in the Blood

Pro

bab

ilit

y o

f D

isea

se

HealthySick

B “Sick”“Healthy”

Which error is worse? It depends.

If the decision is whether to treat a patient who may have cancer, a miss may be worse than a false alarm.

If the decision is whether to attack an unidentified plane, a false alarm may be worse than a miss.

Where should the doctor set his threshold?

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

# Goodies in the Blood

Pro

bab

ilit

y o

f D

isea

se

“Sick”

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

# Goodies in the BloodP

rob

abil

ity

of

Dis

ease SickHealthy

“Healthy” “Sick”“Healthy”

Healthy Sick

B B

As we move B from the left to the right, we decrease P(“S”|H) (false alarms), but increase P(“H”|S) (misses)

Where shouldthe doctor to put his cutoff (B)?

SDT tells the decision maker where to locate B, the cutoff, for a given objective. In this sense, it is a normative theory.

But SDT can also be used to estimate where someone puts his or her B. In this sense, it is a descriptive theory.

The optimal location for B depends on the base rate for the event (in this case, the disease) and the utilities associated with the four possible outcomes.

d’

Healthy Sick

0

0.05

0.10.15

0.2

0.25

0.30.35

0.4

0.45

Values of Y hat

Pro

bab

ilit

yd’

Healthy Sick

Increasing accuracy requires one to find variables that strengthen the predictability of the criterion.

Using those variables in a consistent, statistical fashion improves predictability.

Using a fixed (and optimal) response threshold reduces noise further improves predictability.

Perceived Culpability

Pro

babi

lity

“Convict”“Acquit”

Innocent Guilty

Another Example: Jury Decision Making

What are acceptable probabilities of errors?

Convicting the innocent - False Alarm

Acquitting the guilty - Miss

Within this framework, how good must the juror be to achieve reasonable rates of error ?

Assuming normal distributions and equal variances,

If false alarms and misses are 1%, d’ = 4.7.

If false alarm and miss rates are 5%, d’ = 3.3.

If false alarm and miss rates are 10%, d’= 2.6.

What are common values of d’?

Legal Settings d’ Detecting liars in a mock crime 0Detecting liars with polygraphs 0.5 - 1.0

Personnel Selection d’

Job placement with the Armed Forces Qualification Test 0.6 - 0.8

Weather Forecasting d’

Fog-risk in Canberra, Australia24 hours earlier 0.8 18 hours earlier 1.0 12 hours earlier 1.2

Tornados near Kansas City 1.0Rain in Chicago 1.5Minimum temp in Albuquerque 1.7

Medical Settings d’

Detecting breast cancer w mammograms 1.3 Detecting brain lesions with RN Scans 1.4 - 1.7 Experts detecting cervical cancer 1.6Detecting cervical cancer with computer based systems 1.8Detecting prostate cancer with PSA tests 2.0Detecting brain lesions with CT Scans 2.4 - 2.9

Signal detection theory and empirical estimates of d’ (with all possible technological advances) tells us we are not achieving errors that are even remotely desirable.

What might this imply for policy?