the impact of discredited evidence david lagnado nigel harvey evidence project, ucl
TRANSCRIPT
The impact of discredited evidence
David LagnadoNigel HarveyEvidence project, UCL
Discredited evidence
How do people revise their beliefs once an item of evidence is discredited? For example, in a murder trial, when the
testimony of a key witness is shown to be fabricated, how does this affect juror’s beliefs about the testimony of other witnesses, or even other forensic evidence?
OJ Simpson trial
Normative Models
Bayesian network models Normative model for combining probabilistic
evidence E.g., forensic DNA evidence; paternity cases
(Dawid) Formal modelling of ‘manipulated evidence’
(Baio) Starting to be applied to crime cases
But a lot depends on network construction No ‘normative’ method for this?
Crime case (Leucari, 2005)
Explaining away (Pearl, 1988)
SS = Suspect commits crimeP(S|C) > P(S)
Finding out C raises probability of S
CC = Suspect confesses
FF = Police force confession
P(S|C&F) < P(S|C)
Finding out F too lowers the probability of S
Despite its simplicity and ubiquity, this pattern of inference is hard to capture on most psychological models of inference (e.g., associative models, connectionist, mental models, mental logic)
Psychological models
Belief-adjustment model Hogarth & Einhorn (1992)
Story model Pennington & Hastie (1981, 1988, 1992,
1993)
Belief adjustment model
For Evaluation tasks Evidence encoded as +ve or –ve relative to
hypothesis Adding model
Sk = Sk-1 + wk s (xk)
Sk = degree of belief in hypothesis given k items of evidence
Sk-1 = prior opinion
S (xk) = subjective evaluation of kth item (-1 ≤ s (xk) ≤ +1)
wk = adjustment weight (0 ≤ wk ≤1)
Online vs. Global Processing
Two processing modes Online (step-by-step)
Belief adjusted incrementally with each item of evidence
Global (end-of-sequence) Belief adjusted by aggregate impact of all
items
Sk = S0 + wk [s (x1,…, xk)]
When is each process used?
Online Global
Online All tasksComplex items
and/or long series
Global ImpossibleSimple items
and short series
PROCESS
RESPONSE MODE
Processing load - Aggregation can be costly in terms of mental resources whereas step-by-step integration makes minimal demands
Evidence for Belief Adjustment Order effects in online processing
Evaluation mode (adding not weighted average) None with consistent evidence (e.g, ++ or --) Recency with mixed evidence (-+ > +-) over-weight
last item Supported in Exps 1-5
Model is quite flexible – designed to account for rich patterns of primacy and recency evidence
But does not address relations between evidence items (assumes independence?)
Story model
Evidence evaluated through story construction Stories involve network of causal relations
between events Causal narratives not arguments
People represent events in the world, not inference process
Stories constructed prior to judgment or decision Stories determine verdicts, and are not post hoc
justifications
Evidence for story model
Verbal protocols 85% of events causally linked
Verdicts covaried with story models Recognition memory tests
More likely to falsely remember items consistent with story model
Story vs witness order More likely to convict when prosecution evidence
in causal order, defence in witness order, and vice-versa
Current experiments
Investigate effect of discredited evidence Look at relations between items of evidence Do these modulate how people revise their
beliefs? Once an item of evidence is discredited, do people
simply return to their prior level of belief? Or does this change permeate their belief network?
What factors affect this? Relations between evidence Order of presentation of evidence
HYPOTHESIS: Suspect S did it
Scenario: House burglary, local suspect S apprehended
EVIDENCE 1
Neighbour 1 says that S often loiters in area
EVIDENCE 2
Neighbour 2 says S was outside house on night of crime
Neighbour 2 is lying because he dislikes S
?P(S)
Under-discounting
Over-discounting
Generalisation
When do people generalize from the discrediting of one item to other items?
Dependent on relatedness of generating mechanisms?
SAME E.g. two statements from same neighbour
SIMILAR E.g. two statements from two different
neighbours DIFFERENT
E.g., one statement and one blood test
HYPOTHESIS: Suspect S did it
DIFFERENT Scenario: House burglary, local suspect S apprehended
EVIDENCE 1
Footprints outside house match suspect’s
EVIDENCE 2
Neighbour says S was outside house on night of crime
Neighbour is lying because he dislikes S
?P(S)
Under-discounting
Over-discounting
Experiment 1
Each subject completes 12 problems (4 scenario types x 3 levels of relatedness)
‘Relatedness’: SAME, SIMILAR, DIFFERENT
Four probability judgments (of guilt)1. Background information2. Evidence 1 (E1)3. Evidence 2 (E2)4. Discredit evidence 2 (D2)
Compare 2 and 4 (E2 vs. D2) If D2 > E2 then under-discounting If D2 < E2 then over-discounting
Example of SAME condition
BackgroundYou are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene
Evidence 1The police have a statement from the current wife of the suspect, confessing that the suspect had previously revealed a desire for the victim to be dead
Evidence 2The same police station has a confession from the suspect, admitting that he killed the victim
Discredit 2The confession from the suspect was made under extremely pressured circumstances at the police station
Example of DIFF condition
BackgroundYou are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene
Evidence 1Laboratory tests revealed that blood found at the crime scene matched the blood type of the suspect.
Evidence 2The police station has a confession from the suspect, admitting that he killed the victim
Discredit 2The confession from the suspect was made under extremely pressured circumstances at the police station
Exp 1: Results
Significant ‘over’-discounting (D2 < E1) in all conditions SAME: t(23)=3.74, p<0.05; SIM: t(23)=2.71, p<0.05; DIFF: t(23)=3.13,
p<0.05
Amount of over-discounting greater in SAME vs. SIM, t(23)=1.91, p=0.07; no differences with SAME vs. DIFF, or SIM vs. DIFF
Main effect of DIFF due to physical test as E1
0
10
20
30
40
50
60
70
80
90
100
B E1 E2 D2
Evidence
Pro
ba
bili
ty o
f gu
ilt
DIFFSIMSAME
Individual analysis
Over-discount (D2 < E1)
None (D2 = E1)
Under-discount (D2 > E1)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
SAME SIM DIFF
Relatedness
% p
art
icip
an
ts
Over
None
Under
Conclusions
Difficult to interpret on either BA or Story model
BA model Does not predict effect of relatedness Predicts recency effect with mixed evidence ++- (overweight last item) Asymmetric rebound effect?
Story model Predicts story construction only with global
judgment Does not predict over-discounting
Experiment 2
Better test of two modelsOrder of evidence
LATE - discrediting info presented after both itemsB E1 E2 D
EARLY – discrediting info presented after first itemB E2 D E1
Relatedness SAME, DIFFERENT
HYPOTHESIS: Suspect S did it
DIFFERENT & EARLY
EVIDENCE 2
Neighbour says S was outside house on night of crime
Neighbour is lying because he dislikes S
EVIDENCE 1
Footprints outside house match suspect’s
?P(S)
Model predictions
BA predicts recency Final judgment for early > late Because +-+ > ++- (overweight last item)
Story model predicts recency with online but not global SAME ≠ DIFF for global condition (take account
of relatedness)
Example of SAME and EARLY
BackgroundYou are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene
Evidence 2 The police station has a confession from the suspect, admitting that he killed
the victim Discredit 2
The confession from the suspect was made under extremely pressured circumstances at the police station
Evidence 1The police have a statement from the current wife of the suspect, confessing
that the suspect had previously revealed a desire for the victim to be dead
Example of DIFF and EARLY condition
BackgroundYou are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene
Evidence 2The police station has a confession from the suspect, admitting that he killed the victim
Discredit 2The confession from the suspect was made under extremely pressured circumstances at the police station
Evidence 1Laboratory tests revealed that blood found at the crime scene matched the blood type of the suspect.
Online judgments:Early vs. Late
EARLY
0102030405060708090
100
B1 E1 U1 E2
EVIDENCEPr
obab
ility o
f guil
t SAME
DIFF
Early condition –more sensitivity to relatedness
NB no diff between B1 & U1 in EARLY rules out asymmetric rebound effect
LATE
0102030405060708090
100
B1 E2 E1 U1
EVIDENCE
Prob
abilit
y of g
uilt SAME
DIFF
Late condition – less sensitivity to relatedness (but note that over-discounting (E1 > U1) only sig for SAME not DIFF)
Problematic for both models
BA cannot explain early condition because does not consider relations between evidence
Story model needs to be applied/adapted to online processing, and somehow explain difference between early and late
Any other models? Needed: online model that takes relations
between items into account, but can also explain early/late difference
Speculations
Even with online processing people construct network fragments
As evidence is accumulated it is compactly stored /integrated
Natural to integrate items according to valence (+ve or –ve wrt hypothesis)
E.g., group +ve evidence together
Late condition
Positive evidence A and B integrated
GUILT
A
+
B
+
GUILT
A&B
+
C discredits both A and B (irrespective of relatedness)
C
Early condition DIFF
B unaffected by C’s discredit of A
GUILT
A
+
B
+
C
GUILT
A
+
C
Early condition SAME
GUILT
A
+
A*
A* discredited by C too (because similar to A)
C
GUILT
A
+
C
Ongoing research
Look at both witness and alibi statements E.g., How does discrediting of an alibi affect
evaluation of a positive witness? Are there asymmetries in dealing with positive
vs. negative evidence? More generally, look at positive and
negative evidence (including forensic tests) Are there differential affects of discrediting? Can evidence integration idea explain these?
Manipulate deception vs error