signal-detection theory and receiver operating characteristic (roc) analysis
DESCRIPTION
Signal-detection theory and receiver operating characteristic (roc) analysis. Psych 272. Eyewitness Memory and Wrongful Convictions. Since the 1990s, DNA testing has overturned 318 wrongful convictions - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/1.jpg)
SIGNAL-DETECTION THEORY AND RECEIVER OPERATING CHARACTERISTIC (ROC) ANALYSIS
Psych 272
![Page 2: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/2.jpg)
Eyewitness Memory and Wrongful Convictions• Since the 1990s, DNA testing has overturned 318 wrongful convictions
• Eyewitness misidentifications – which were invariably made with high confidence in a court of law – played a role in 75% of these cases
• One of the most famous cases involved the misidentification of a man named Ronald Cotton
![Page 3: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/3.jpg)
The Case of Ronald Cotton• In 1984, a college student named Jennifer Thompson was raped • Shortly thereafter, she picked Ronald Cotton out of a photo lineup• "I was absolutely, positively, without-a-doubt certain he was the man who
raped me when I got on that witness stand and testified against him. And nobody was going to tell me any different."
• Cotton was sentenced to life in prison plus 54 years, and he served almost 11 years in jail before being exonerated by DNA testing
![Page 4: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/4.jpg)
Eyewitness Identification Procedures
Simultaneous Lineup
Suspect:Innocent or
Guilty?
Fillers:All are known to be
innocent
![Page 5: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/5.jpg)
Eyewitness Identification Procedures
Sequential LineupSimultaneous Lineup
Suspect:Innocent or
Guilty?
![Page 6: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/6.jpg)
Lindsay & Wells (1985)
Simultaneous lineup Correct ID rate = 0.58 False ID rate = 0.43
Sequential lineup Correct ID rate = 0.50 False ID rate = 0.17
.58 / .43 = 1.35
.50 / .17 = 2.94
Diagnosticity Ratio
![Page 7: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/7.jpg)
![Page 8: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/8.jpg)
When does signal detection theory apply?
1. There are two true states of the world• An enemy plane is either present or absent in the sky• A disease is either present or absent in a patient• A guilty suspect is either present or absent in a lineup
2. An imperfect diagnostic procedure is used to make a decision (the target is "present" or "absent")
• An air-defense radar system• A medical test• An eyewitness presented with a lineup
![Page 9: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/9.jpg)
Hit(Correct ID)
TrueState
Present
Absent
Present Absent
DiagnosticDecision
Miss
CorrectRejection
False Alarm(False ID)
2 X 2 Table
![Page 10: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/10.jpg)
Hit(Correct ID)
TrueState
Present
Absent
Present Absent
DiagnosticDecision
Miss
CorrectRejection
False Alarm(False ID)
2 X 2 Table
![Page 11: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/11.jpg)
Hit(Correct ID)
TrueState
Present
Absent
Present Absent
DiagnosticDecision
Miss
CorrectRejection
False Alarm(False ID)
2 X 2 Table
![Page 12: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/12.jpg)
Hit(Correct ID)
TrueState
Present
Absent
Present Absent
DiagnosticDecision
Miss
CorrectRejection
False Alarm(False ID)
2 X 2 Table
![Page 13: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/13.jpg)
Hit(Correct ID)
TrueState
Present
Absent
Present Absent
DiagnosticDecision
Miss
CorrectRejection
False Alarm(False ID)
2 X 2 Table
![Page 14: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/14.jpg)
Hit(Correct ID)
TrueState
Present
Absent
Present Absent
DiagnosticDecision
Miss
CorrectRejection
False Alarm(False ID)
2 X 2 Table
![Page 15: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/15.jpg)
Hit(Correct ID)
TrueState
Present
Absent
Present Absent
DiagnosticDecision
Miss
CorrectRejection
False Alarm(False ID)
2 X 2 Table
Innocence Project
Ronald Cotton
Ted Bundy
![Page 16: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/16.jpg)
Hit(Correct ID)
TrueState
Present
Absent
Present Absent
DiagnosticDecision
Miss
CorrectRejection
False Alarm(False ID)
2 X 2 Table
Innocence Project
Ronald Cotton
Ted Bundy
![Page 17: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/17.jpg)
Hit(Correct ID)
TrueState
Present
Absent
Present Absent
DiagnosticDecision
Miss
CorrectRejection
False Alarm(False ID)
2 X 2 Table
![Page 18: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/18.jpg)
Signal Detection Theory
Weak Signal Strong Signal
Continuous diagnostic signal
Power of the reflected radio signalBlood glucose levelMemory strength
![Page 19: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/19.jpg)
Signal Detection Theory: Response Bias
Weak Signal Strong Signal
“the guilty suspect is probably in the lineup”
![Page 20: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/20.jpg)
Signal Detection Theory: Response Bias
Weak Signal Strong Signal
“present”“absent”
“the guilty suspect is probably in the lineup”
Liberal response bias:Identify even if confidence is low
![Page 21: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/21.jpg)
Signal Detection Theory: Response Bias
Weak Signal Strong Signal
“the guilty suspect may or may not be in the lineup”
![Page 22: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/22.jpg)
Signal Detection Theory: Response Bias
Weak Signal Strong Signal
“present”“absent”
“the guilty suspect may or may not be in the lineup”
Neutral response bias:Identify if confidence is fairly high
![Page 23: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/23.jpg)
Signal Detection Theory: Response Bias
Weak Signal Strong Signal
“too many innocent suspects have been misidentified”
![Page 24: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/24.jpg)
Signal Detection Theory: Response Bias
Weak Signal Strong Signal
“present”“absent”
“too many innocent suspects have been misidentified”
Conservative response bias:Identify only if confidence is very high
![Page 25: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/25.jpg)
Signal Detection Theory: Discriminability
Weak Signal Strong Signal
![Page 26: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/26.jpg)
Signal Detection Theory: Discriminability
Weak Signal Strong Signal
Target Absent(Innocent suspects)
![Page 27: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/27.jpg)
Signal Detection Theory: Discriminability
Weak Signal Strong Signal
Target Absent(Innocent suspects)
![Page 28: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/28.jpg)
Signal Detection Theory: Discriminability
Weak Signal Strong Signal
Target Absent(Innocent suspects)
Target Present(Guilty suspects)
![Page 29: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/29.jpg)
Weak Signal Strong Signal
Target Absent(Innocent suspects)
Target Present(Guilty suspects)
Signal Detection Theory: Discriminability
Discriminability
The degree to which the memory signals associated with innocent and guilty suspects are separated using a particular diagnostic procedure
![Page 30: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/30.jpg)
Weak Signal
Target PresentTarget Absent
"Present""Absent"
Strong Signal
"Present""Absent"
Target AbsentTarget Present
Target PresentTarget Absent
Weak Signal Strong Signal
Weak Signal Strong Signal
"Present""Absent"
Liberal
Neutral
Conservative
![Page 31: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/31.jpg)
Weak Signal
Target PresentTarget Absent
"Present""Absent"
Strong Signal
"Present""Absent"
Target AbsentTarget Present
Target PresentTarget Absent
Weak Signal Strong Signal
Weak Signal Strong Signal
"Present""Absent"
Liberal: “the guilty suspect is probably in the lineup”
![Page 32: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/32.jpg)
Correct ID Rate = 0.98
Weak Signal
Target PresentTarget Absent
"Present""Absent"
Strong Signal
"Present""Absent"
Target AbsentTarget Present
Target PresentTarget Absent
Weak Signal Strong Signal
Weak Signal Strong Signal
"Present""Absent"
Liberal: “the guilty suspect is probably in the lineup”
![Page 33: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/33.jpg)
Correct ID Rate = 0.98
False ID Rate = 0.50
Weak Signal
Target PresentTarget Absent
"Present""Absent"
Strong Signal
"Present""Absent"
Target AbsentTarget Present
Target PresentTarget Absent
Weak Signal Strong Signal
Weak Signal Strong Signal
"Present""Absent"
Liberal: “the guilty suspect is probably in the lineup”
![Page 34: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/34.jpg)
Weak Signal
Target PresentTarget Absent
"Present""Absent"
Strong Signal
"Present""Absent"
Target AbsentTarget Present
Target PresentTarget Absent
Weak Signal Strong Signal
Weak Signal Strong Signal
"Present""Absent"
Neutral: “the guilty suspect may or may not be the lineup”
![Page 35: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/35.jpg)
Correct ID Rate = 0.84
Weak Signal
Target PresentTarget Absent
"Present""Absent"
Strong Signal
"Present""Absent"
Target AbsentTarget Present
Target PresentTarget Absent
Weak Signal Strong Signal
Weak Signal Strong Signal
"Present""Absent"
Neutral: “the guilty suspect may or may not be the lineup”
![Page 36: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/36.jpg)
False ID Rate = 0.16
Correct ID Rate = 0.84
Weak Signal
Target PresentTarget Absent
"Present""Absent"
Strong Signal
"Present""Absent"
Target AbsentTarget Present
Target PresentTarget Absent
Weak Signal Strong Signal
Weak Signal Strong Signal
"Present""Absent"
Neutral: “the guilty suspect may or may not be the lineup”
![Page 37: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/37.jpg)
Weak Signal
Target PresentTarget Absent
"Present""Absent"
Strong Signal
"Present""Absent"
Target AbsentTarget Present
Target PresentTarget Absent
Weak Signal Strong Signal
Weak Signal Strong Signal
"Present""Absent"
Conservative: “do not make an ID unless you are certain of being correct”
![Page 38: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/38.jpg)
Correct ID Rate = 0.50
Weak Signal
Target PresentTarget Absent
"Present""Absent"
Strong Signal
"Present""Absent"
Target AbsentTarget Present
Target PresentTarget Absent
Weak Signal Strong Signal
Weak Signal Strong Signal
"Present""Absent"
Conservative: “do not make an ID unless you are certain of being correct”
![Page 39: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/39.jpg)
Correct ID Rate = 0.50
False ID Rate = 0.02
Weak Signal
Target PresentTarget Absent
"Present""Absent"
Strong Signal
"Present""Absent"
Target AbsentTarget Present
Target PresentTarget Absent
Weak Signal Strong Signal
Weak Signal Strong Signal
"Present""Absent"
Conservative: “do not make an ID unless you are certain of being correct”
![Page 40: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/40.jpg)
False ID Rate
0.0 0.2 0.4 0.6 0.8 1.0
Cor
rect
ID R
ate
0.0
0.2
0.4
0.6
0.8
1.0
Receiver Operating Characteristic (ROC)
Correct ID Rate = 0.98
False ID Rate = 0.50
False ID Rate = 0.16
Correct ID Rate = 0.84
Correct ID Rate = 0.50
False ID Rate = 0.02
Weak Signal
Target PresentTarget Absent
"Present""Absent"
Strong Signal
"Present""Absent"
Target AbsentTarget Present
Target PresentTarget Absent
Weak Signal Strong Signal
Weak Signal Strong Signal
"Present""Absent"
![Page 41: Signal-detection theory and receiver operating characteristic (roc) analysis](https://reader035.vdocuments.mx/reader035/viewer/2022062309/56813992550346895da12729/html5/thumbnails/41.jpg)
Receiver Operating Characteristic Analysis
Target AbsentTarget Present
Weak Signal Strong Signal
False ID Rate
0.0 0.2 0.4 0.6 0.8 1.0
Cor
rect
ID R
ate
0.0
0.2
0.4
0.6
0.8
1.0
Weak Signal
Target PresentTarget Absent
Strong Signal
False ID Rate
0.0 0.2 0.4 0.6 0.8 1.0
Cor
rect
ID R
ate
0.0
0.2
0.4
0.6
0.8
1.0
Highdiscriminability
Lowdiscriminability