upstream remedies to prevent wrongful convictions · pdf fileupstream remedies to prevent...

34
SAMSI PROGRAM IN FORENSICS 2015 RESEARCH TRIANGLE PARK, NC PETER J. NEUFELD CO-DIRECTOR INNOCENCE PROJECT The Critical Need for Research Assessing the Probative Value of Forensic Evidence Upstream Remedies to Prevent Wrongful Convictions:

Upload: lecong

Post on 11-Mar-2018

224 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

SAMSI PROGRAM IN FORENSICS 2015RESEARCH TRIANGLE PARK, NC

PETER J. NEUFELDCO-DIRECTORINNOCENCE PROJECT

The Critical Need for Research Assessingthe Probative Value of Forensic Evidence

Upstream Remedies to Prevent Wrongful Convictions:

Page 2: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

330 POST-CONVICTION DNA EXONERATIONS IN USA

Misapplication of forensic science contributed to approximately half of wrongful convictions overturned by DNA

Page 3: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

MISAPPLICATION OF FORENSIC SCIENCE

74 of the total DNA exonerations included erroneous hair microscopy evidence as demonstrated by DNA testing (included both state and FBI hair comparisons)

Presenter
Presentation Notes
But the reason its hair evidence as opposed to other types of impression or pattern evidence, is serendipitous. The same over the top statements are made by forensic examiners in other disciplines lacking sufficient data but because there is no dna in those cases, there is no way to get to ground truth. It’s much more challenging to vacate those convictions. It would be extremely useful to design a study on fingerprints from actual cases. Where the lifting of prints is done in such a way as to preserve the dna for testing. And then like in the hair cases, compare the results.
Page 4: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

A CRITICAL MASS OF EXONERATIONS

Page 5: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

3 EXONERATIONS, 3 YEARS, 3 EXAMINERS

Donald Gates exonerated 2009

Kirk Odom exonerated in 2012

Santae Tribbleexonerated in 2012

Presenter
Presentation Notes
Three cases litigated by Sandra Levick of DC PDS revealed that three different FBI examiners were responsible for invalid testimony suggesting that positive associations between suspect hairs and the questioned hairs were extraordinarily rare.
Page 6: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

In 2012, Spencer Hsu began an investigative series highlighting flawed forensic work by the FBI and the hair review project, including:• “Convicted defendants left uninformed of forensic flaws found by

Justice Dept.” (April 16, 2012)• “Forensic science not as reliable as you may think”

(April 17, 2012)• “Flawed forensics prompt case reviews” (July 10, 2012)

Page 7: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

MICROSCOPIC HAIR COMPARISON

Presenter
Presentation Notes
I’m using hair because it’s biological evidence and so we can reopen some old convictions because there is dna to test.
Page 8: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

HISTORY OF MICROSCOPIC HAIR COMPARISON

FBI began performing hair comparison long before 1970s in effort to make probative match of unknown hairs to hairs connected to known suspect or victim.

• FBI Manual on Hair Microscopy (1977), at https://www.ncjrs.gov/pdffiles1/Digitization/78957NCJRS.pdf.

• Oien (2009) (stating that “[t]hese comparisons have been routinely conducted in forensic laboratories and accepted both in the scientific community and in the legal community for the past 75 years."

Page 9: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

HISTORY OF MICROSCOPIC HAIR COMPARISON

Two relevant questions:Can a properly trained examiner determine that a questioned sample hair and a known suspect hair are similar (sometimes referred to as a positive association)? Can a properly trained examiner provide a scientifically valid estimate of the rareness or frequency of the similarity that you would find between two differently sourced samples?

Page 10: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

HISTORY OF MICROSCOPIC HAIR COMPARISON

Presenter
Presentation Notes
Discuss training and proficiency and/or competency testing. One year. Rigorous. Last 5 bags of hair, required 100% correct to graduate to case work.
Page 11: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

HISTORY OF MICROSCOPIC HAIR COMPARISON Myth:

Hair comparison is a 1:1 match where all characteristics are present on both the questioned and known hair.

Fact:There is rarely, if ever, a 1:1 match. Most cases involve a questioned hair being compared to dozens of hairs (“the sample hair”) from a suspect due to intra-individual variation. Thus, certain characteristics of the questioned hair might be present in one sample hair from a suspect and not in others, while other characteristics might be present in a different subset of sample hair(s) from the suspect.

Presenter
Presentation Notes
This important distinction was rarely communicated by the FBI hair examiner to the jury. This can become an issue in ballistics where you may not have “identical” striations but the examiner is making a determination about “sufficient agreement” within a range of variation.
Page 12: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

2009 NAS REPORT: SCIENTIFIC LIMITS OF HAIR COMPARISON

“No scientifically accepted statistics exist about the frequency with which particular characteristics of hair are distributed in the population. There appear to be no uniform standards on the number of features on which hairs must agree before an examiner may declare a “match.” (p.160)

“The committee found no scientific support for the use of hair comparisons for individualization in the absence of nuclear DNA. Microscopy and mtDNA analysis can be used in tandem and may add to one another’s value for classifying a common source, but no studies have been performed specifically to quantify the reliability of their joint use.” (p.161)

Strengthening Forensic Science in the United States: A Path Forward, Committee on Identifying the Needs of the Forensic Science Community, The National Academies Press (2009)

Page 13: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

FBI RESEARCH ILLUSTRATES THE PROBLEM

M. Houck and B. Budowle. Correlation of microscopic and mitochondrial DNA hair comparisons. Journal of Forensic Sciences 47(5), at 964-967 (2002)

11 percent of the cases in which the FBI hair examiners declared an “association”, mtDNA testing revealed that the hairs did not have a common source. 54 percent of the cases in which FBI hair examiners declared an “inconclusive” decision, mtDNA testing revealed were true exclusions

Presenter
Presentation Notes
The research indicates that hair microscopy has the potential to have some modest value, but to date, there have not been adequate studies to estimate what that value is. Given the disparity between the FBI hair examiners’ proficiency/competency testing where they needed to get 100% correct, and the results of the Houck & Budowle study Query: Is non-blinded proficiency testing enough for any discipline to estimate sensitivity and specificity?
Page 14: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

FBI HAIR REVIEW

Page 15: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

REVIEW PROCESS

FBI / DOJ / NACDL / Innocence Project Collaboration (2012- ):

FBI Identified 21,000 cases where analysis was done by FBI before 2000; 3000 cases where FBI made positive associations between questioned hair and known sample;FBI conducts review of transcripts, shares results with NACDL/Innocence Project, FBI certifies results;Criteria for evaluating testimony was developed by FBI Lab with the input of statisticians. FBI makes final decision on presence of errors in each case.

Presenter
Presentation Notes
The question as to the validity of examining hairs beneath a microscope and declaring known and a crime scene samples to be similar was not part of the review. In a sense, its validity was assumed – only the interpretation of the probative value of the evidence was the subject of this review.
Page 16: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

APPROPRIATE NON-MISLEADING TESTIMONY

The examiner’s testimony appropriately reflected the fact that hair comparison could not be used to make a positive identification, but that it could indicate, at the broad class level, that a contributor of a known sample could be included in a pool of people of unknown size, as a possible source of the hair evidence (without in any way giving probabilities, as an opinion to the likelihood or rareness of the positive association, or the size of the class) or that the contributor of a known sample could be excluded as a possible source of the hair evidence based on the known sample provided.

Presenter
Presentation Notes
The FBI agreed that it’s not sufficient to testify in a manner that may be technically accurate if in fact, there is a substantial danger of misleading the jury or judge.
Page 17: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

TYPES OF INAPPROPRIATE TESTIMONY

Error Type 1: The examiner stated or implied that the evidentiary hair could be associated with a specific individual to the exclusion of all others.

Page 18: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

ERROR TYPE 1

Page 19: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

ERROR TYPE 1

Page 20: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

ERROR TYPE 1

Page 21: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

ERROR TYPE 1

Page 22: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

TYPES OF INAPPROPRIATE TESTIMONY

Error Type 2: The examiner assigned to the positive association a statistical weight or probability or provided a likelihood that the questioned hair originated from a particular source, or an opinion as to the likelihood or rareness of the positive association that could lead the jury to believe that valid statistical weight can be assigned to microscopic hair association.

Page 23: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

ERROR TYPE 3 & 2

Type 3

Type 2

Page 24: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

ERROR TYPE 2

Page 25: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

TYPES OF INAPPROPRIATE TESTIMONY

Error Type 3: The examiner cites the number of cases or hair analyses worked in the lab and the number of samples from different individuals that could not be distinguished from one another as a predictive value to bolster the conclusion that a hair belongs to a specific individual.

Page 26: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

ERROR TYPE 3

Presenter
Presentation Notes
Explain 1. It allows jurors to think that he has compared the questioned hair or a reference hair to the hairs of 10,000 people. No logs, no notes on comparisons. Numbers stayed at 10000 over a decade of testing. The examiner continues to look at hundreds of hair samples each year but his denominator never changes. 2. how this was further distorted in prosecutor’s summation
Page 27: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

ERROR TYPES 2 & 3

Presenter
Presentation Notes
Several different fbi examiners all used the 10000 number without any logs or data to support the number. And as I said none of them compared a crime scene hair or a reference hair in a case to hairs examined the week before or the year before.
Page 28: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

INITIAL RESULTS

• As of March 2015:• FBI reviewed approximately 500 cases•Of the 268 transcripts where examiners provided testimony to inculpate the defendant at trial, 96% (N=257) contained one or more types of testimonial errors. • All but two of 28 FBI examiners provided testimony that contained erroneous statements or authored lab reports with such statements. •The review showed that FBI examiners testified in 41 states.• 94% (N=33) of capital case testimony where associations were made had errors.

• 9 of these capital defendants executed• 5 of these capital defendants died of other causes

Presenter
Presentation Notes
Page 29: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

2015 Spencer Hsu article reveals FBI also trained thousands of state and local level hair microscopists.

These state and local examiners were trained by the FBI to use the same methods to reach the same conclusions and to give testimony containing the same errors as the FBI.How pervasive is the problem? Estimates of 500-1000 state and local microscopic hair comparison examiners were trained by the FBI to testify the same way.

Page 30: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

KEY ISSUES:WHAT IS TO BE DONE?

Page 31: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

KEY ISSUE

We are counting on your community to devise

1. Statistically appropriate - valid and reliable methods to assess the similarity or dissimilarity of two samples

2. Statistically appropriate - valid and reliable methods for interpreting and expressing to the jury the probative value of the forensic evidence

Presenter
Presentation Notes
Page 32: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

KEY ISSUE

We need to communicate the value of the evidence to lay jurors and judges in a way people will understand the evidence, and not distort it.

Presenter
Presentation Notes
There is some data indicating that lay people do not accurately comprehend statistical evidence. The problem is present in the expression of random match probabilities, likelihood ratios, and their verbal equivalents. There is a need for studies that explore the impact of cognitive biases in jurors. The fields of statistics and psychology should work together to design appropriate studies.
Page 33: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

KEY ISSUE

What should be required of the expert when life and liberty are at stake?

Presenter
Presentation Notes
Is a statistical method that is technically accurate but not terribly robust adequate for application to the criminal justice system when the stakes are so high?
Page 34: Upstream Remedies to Prevent Wrongful Convictions · PDF fileUpstream Remedies to Prevent Wrongful Convictions: ... biological evidence and so we can reopen some old convictions because

THANK YOU