judgments and decisions psych 253 using decision analysis to answer questions about the value of...
TRANSCRIPT
Judgments and DecisionsPsych 253
• Using Decision Analysis to Answer Questions about the Value of Getting Information
• Perfect vs. Imperfect Information• Signal Detection Theory• Medical Example• Jury Decision Making
You can select A, B, or C. Below are the payoffs for each depending on the throw of a die. Which one do you want to have?
Die Roll1 2 3 4 5 6
A $1 $2 $3 $4 $5 $6
B $6 $2 $5 $2 $2 $2
C $7 $5 $4 $4 $2 $1
What should you choose?
Die Roll1 2 3 4 5 6 EV
A $1 $2 $3 $4 $5 $6 $3.50
B $6 $2 $5 $2 $2 $2 $3.17
C $7 $5 $4 $4 $2 $1 $3.83
Which one would you select if you knew what number would come up on the die?
Die Roll1 2 3 4 5 6
A $1 $2 $3 $4 $5 $6
B $6 $2 $5 $2 $2 $2
C $7 $5 $4 $4 $2 $1
What is the expected value of knowing the number that would come up?
1/6*[ $7 + $5 + $5 + $4 + $5 + $6 ] = $5.33
Die Roll1 2 3 4 5 6
A $1 $2 $3 $4 $5 $6
B $6 $2 $5 $2 $2 $2
C $7 $5 $4 $4 $2 $1
The value of the information is
$5.33 - $3.83 = $1.50Value of the
decision WITH
perfect information
Value of the decision
WITHOUT perfect
information
Miracle Movers rents out trucks with a crew of 2 people. One day, Miracle discovers it is a truck short. When this happens, Miracle rents a truck from a local firm. Small trucks cost $130 per day, and large ones cost $200. The advantage of the small truck may vanish if the crew has to make 2 trips. The extra cost of a second trip is $150, and the probability of 2 trips is 40%.
$200 $200
Small Truck
Need Small Truck $130$130
Need Large Truck $280$280
P(NS) =.60
P(NL) =.40
What would it be worth to Michelle to know for sure whether she would need a small truck?
Large Truck
$200 $200
Small Truck
$130$130
$280$280
P(NS) =.60
P(NL) =.40
What would it be worth to Michelle to know for sure whether she would need a small truck?
Large Truck
$200 $200
Small Truck
What would it be worth to Michelle to know for sure whether she would need a small truck?
Large Truck
EV= $190280
Collect Information(that happensto be perfect)
“NS”
“NL”
P(NS) =.60
P(NL) = .40
Small Truck $190
Need Large Truck
Don’t Collect Information
Need Small Truck
Need Large Truck
Need Small Truck
Collect PerfectInformation
“NS”
“NL”
P(NS) =.60
P(NL)=.40Need Large Truck $200
What does Perfect Information Mean?
Need Small Truck $130
P (“NL”|NL) = 1 and P (“NL”|NS) = 0
P(“NS”|NS) = 1 and P (“NS”|NL ) = 0
Value of the decision with perfect information
= p(“NL” and NL)*$200 + p(“NS” and NS)*$130
+ p(“NL” and NS)*$200 + p(“NS” and NL)*$280
But because the information is perfect,
p(“NL” and NS) and p(“NS” and NL) never occur.
So the value of the decision with perfect information is
= p(“NL” and NL)*$200 + p(“NS” and NS)*$130
What is P(“NL” and NL) and P(“NS” and NS)?
P(“NL” and NL) = P(“L”|NL)*P(NL) = 1.0*(.4) =.4
P(“NS” and NS) = P(“S”|NS)*P(NS) = 1.0*(.6)=.6
Value of the decision with perfect info = p(“NL” and NL)*$200 + p(“NS” and NS)*$130
= .4*$200 + .6*$130 = $158
Value of the Decision – Value of the Decision
with perfect info = Value of the Information
Value of the information = $190 - $158 = $32
Perfect information means this…
P(“L”|NS)=0
P(“S”|NL) =0
P(“S”|NS) =1
P(“L”|NL) =1
“S”
NS
NL
“L”Information
Actual
If the information is imperfect, we need to quantify that uncertainty.
P(“L”|NS)=.1
P(“S”|NL)=.2
P(“S”|NS) =.9
P(“L”|NL) =.8
“S”
NS
NL
“L”Information
Actual
Signal Detection Theory:A Theory of Repeated Decision Making
when Information is Fuzzy
-deciding which customers would default on a loan-deciding whether to hire an employee-determining whether someone is lying or not-and any other decision that is made repeatedly using ambiguous evidence
Imagine a doctor who treats patients for a disease. A test for the number of goodies in the patient’s blood is the only predictor or clue he has to predict whether or not the patient has the disease.
From past research, the doctor knows that goodies are normally distributed in the blood of Healthy and Sick people. The average # of goodies in the blood of Healthy people is 100, and the average number of goodies in the blood of Sick people is 115.
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
# Goodies in the Blood
Pro
babi
lity
of D
isea
se
Healthy Sick
We can analyze the doctor’s ability to diagnose a patients with Signal Detection Theory. This theory has two parameters:
d´ = d prime - the ability to distinguish Healthy people from Sick people with the blood test
B = beta (also called the threshold or cutoff) - the doctor’s tendency to call a patient “Healthy” or “Sick”
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
# Goodies in the Blood
Pro
babi
lity
of D
isea
se
Healthy Sick
d’
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
# Goodies in the Blood
Pro
bab
ilit
y o
f D
isea
se
HealthySick
B “Sick”“Healthy”
No matter where the doctor sets his cutoff, there are two errors and two correct decisions.
P(“H”|H) P(“S”|H)
P(“H”|S) P(“S”|S)Sick
Healthy
“Healthy” “Sick”
State of the World
Decision
Each outcome has a name.
CorrectRejection
FalseAlarm
Miss HitSick
Healthy
“Healthy” “Sick”
State of the World
Decision
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
# Goodies in the Blood
Pro
bab
ilit
y o
f D
isea
se
HealthySick
B “Sick”“Healthy”
Which error is worse? It depends.
If the decision is whether to treat a patient who may have cancer, a miss may be worse than a false alarm.
If the decision is whether to attack an unidentified plane, a false alarm may be worse than a miss.
Where should the doctor set his threshold?
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
# Goodies in the Blood
Pro
bab
ilit
y o
f D
isea
se
“Sick”
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
# Goodies in the BloodP
rob
abil
ity
of
Dis
ease SickHealthy
“Healthy” “Sick”“Healthy”
Healthy Sick
B B
As we move B from the left to the right, we decrease P(“S”|H) (false alarms), but increase P(“H”|S) (misses)
Where shouldthe doctor to put his cutoff (B)?
SDT tells the decision maker where to locate B, the cutoff, for a given objective. In this sense, it is a normative theory.
But SDT can also be used to estimate where someone puts his or her B. In this sense, it is a descriptive theory.
The optimal location for B depends on the base rate for the event (in this case, the disease) and the utilities associated with the four possible outcomes.
d’
Healthy Sick
0
0.05
0.10.15
0.2
0.25
0.30.35
0.4
0.45
Values of Y hat
Pro
bab
ilit
yd’
Healthy Sick
Increasing accuracy requires one to find variables that strengthen the predictability of the criterion.
Using those variables in a consistent, statistical fashion improves predictability.
Using a fixed (and optimal) response threshold reduces noise further improves predictability.
Perceived Culpability
Pro
babi
lity
“Convict”“Acquit”
Innocent Guilty
Another Example: Jury Decision Making
What are acceptable probabilities of errors?
Convicting the innocent - False Alarm
Acquitting the guilty - Miss
Within this framework, how good must the juror be to achieve reasonable rates of error ?
Assuming normal distributions and equal variances,
If false alarms and misses are 1%, d’ = 4.7.
If false alarm and miss rates are 5%, d’ = 3.3.
If false alarm and miss rates are 10%, d’= 2.6.
What are common values of d’?
Legal Settings d’ Detecting liars in a mock crime 0Detecting liars with polygraphs 0.5 - 1.0
Personnel Selection d’
Job placement with the Armed Forces Qualification Test 0.6 - 0.8
Weather Forecasting d’
Fog-risk in Canberra, Australia24 hours earlier 0.8 18 hours earlier 1.0 12 hours earlier 1.2
Tornados near Kansas City 1.0Rain in Chicago 1.5Minimum temp in Albuquerque 1.7
Medical Settings d’
Detecting breast cancer w mammograms 1.3 Detecting brain lesions with RN Scans 1.4 - 1.7 Experts detecting cervical cancer 1.6Detecting cervical cancer with computer based systems 1.8Detecting prostate cancer with PSA tests 2.0Detecting brain lesions with CT Scans 2.4 - 2.9
Signal detection theory and empirical estimates of d’ (with all possible technological advances) tells us we are not achieving errors that are even remotely desirable.
What might this imply for policy?