m.res programme critical appraisal bob lightowlers, wt centre for mitochondrial research
DESCRIPTION
M.Res Programme CRITICAL APPRAISAL Bob Lightowlers, WT Centre for Mitochondrial Research. r [email protected]. NOT EVERYTHING THAT IS PUBLISHED IS CORRECT!!. NOT EVERYTHING THAT IS PUBLISHED IS CORRECT!!. ‘The statistical error that keeps on coming’ - PowerPoint PPT PresentationTRANSCRIPT
M.Res Programme
CRITICAL APPRAISAL
Bob Lightowlers, WT Centre for Mitochondrial Research
NOT EVERYTHING THAT IS PUBLISHED IS CORRECT!!
NOT EVERYTHING THAT IS PUBLISHED IS CORRECT!!
‘The statistical error that keeps on coming’
Ben Goldacre – The Guardian 9th Sept 2011
• Numbers of mice not likely to generate significance• No randomisation• No blinded evaluations • Bias in reporting (no reporting of negative data)• Variable genetic background
• Broader endpoints such as life expectancy/behaviour too crude
Problem transferring ‘effective’drugs from mouse model toclinic
Nonhuman primate models ?
Sept 2011 Bayer Pharmaceuticals assessed reproducibility of 67 published projects
Only 20-25% of published data in line with in-house findings
Jan 2012Amgen tried to confirm research from 53 published ‘landmark’ studies
Jan 2012Amgen tried to confirm research from 53 published ‘landmark’ studies
ONLY 6 (11%) could be confirmed
Jan 2012Amgen tried to confirm research from 53 published ‘landmark’ studies
ONLY 6 (11%) could be confirmed
What causes this lack of reproducibility ?
What causes this lack of reproducibility ?
What causes this lack of reproducibility ?
What causes this lack of reproducibility ?
What causes this lack of reproducibility ?
NOT EVERYTHING THAT IS PUBLISHED IS CORRECT!!
ONLY 15% OF PUBLICATIONS ARE TRUSTWORTHY
• Be critical
• Guilty until proven innocent
The peer review process
What should we be looking for ?
• Incorrect statistical analysis• Power of study• Absence of essential controls• Incorrect methodology• Over/mis interpretation of data• Lack of reference to any conflicting data
What should we be looking for ?
• Incorrect statistical analysis• Power of study• Absence of essential controls• Incorrect methodology• Over/mis interpretation of data• Lack of reference to any conflicting data
• Falsification of data
Somatic Cell Nuclear Transfer
Peer review cannot guarantee scientific integrity
Peer review cannot guarantee scientific integrity
http://www.biochem.arizona.edu/classes/bioc568/papers.htm
Critical Evaluation - a worked example
Human mtDNA
• An autosomally replicating genome
• Found in mitochondrial matrix
• Comprises app. 0.1% of total cell DNA
• Varies enormously in copy number/cell Approx. 700 in fibroblasts to >200,000 in some mammalian oocytes
• Maternally inherited
• Often heteroplasmic in the diseased state
16,569 bp
• Encodes 13 proteins, all of which are OXPHOS components
Hypothesis:
1. Alzheimers Disease could be caused by defects in activity of the respiratory chain complex cytochrome c oxidase
2. Alzheimers Disease is due to mutations in the mitochondrial genome
Why ?
• Lack of FH is a negative risk factor
Why ?
• Lack of FH is a negative risk factor • Risk of AD increases with affected maternal relative (mtDNA?)
Why ?
• Lack of FH is a negative risk factor • Risk of AD increases with affected maternal relative (mtDNA?)
• Mutations in mtDNA can lead to defective OXPHOS
Why ?
• Lack of FH is a negative risk factor • Risk of AD increases with affected maternal relative (mtDNA?)
• Mutations in mtDNA can lead to defective OXPHOS
• Neurons may be particularly susceptible to such defects
DiabetesThyroid Disease
Myopathy
Peripheral Neuropathy
Deafness
CVA / Seizures /
Developmental delay
Respiratory Failure Optic Atrophy / Retinitis Pigmentosa
Cardiomyopathy
Short StatureMarrow Failure
Liver Failure
NeurologicalNon-Neurological
Why ?
• Lack of FH is a negative risk factor• • Risk of AD increases with affected maternal relative (mtDNA?)
• Mutations in mtDNA can lead to defective OXPHOS
• Neurons may be particularly susceptible to such defects
• COX activity reported to decrease in brain of AD patients
Methods used
• MtDNA isolation and sequencing in patients, asymptomatic relatives and controls
Methods used
• MtDNA isolation and sequencing in patients, asymptomatic relatives and controls
• All three COX genes sequenced
Methods used
• MtDNA isolation and sequencing in patients, asymptomatic relatives and controls
• All three COX genes sequenced
• Platelet fusion from AD patients to neuronal cells lacking mtDNA (rho0)
Generation of transmitochondrialcybrids
Biopsy
EthBr Enucleation
Methods used
• MtDNA isolation and sequencing in patients, asymptomatic relatives and controls
• All three COX genes sequenced
• Platelet fusion from AD patients to neuronal cells lacking mtDNA (rho0)
• Analysis of respiratory enzyme activity in the cybrids
Methods used
• MtDNA isolation and sequencing in patients, asymptomatic relatives and controls
• All three COX genes sequenced
• Platelet fusion from AD patients to neuronal cells lacking mtDNA (rho0)
• Analysis of respiratory enzyme activity in the cybrids
• Analysis of ROS production in cybrids
Results
506 Patients and 95 controls
Results
506 Patients and 95 controls
10 clones of all three COX genes sequence
Results
506 Patients and 95 controls
10 clones of all three COX genes sequence
6 mutations found in COI and COII
Results
506 Patients and 95 controls
10 clones of all three COX genes sequence
6 mutations found in COI and COII
Different levels of heteroplasmy but levels significantly greater in the AD cohort
Results
506 Patients and 95 controls
10 clones of all three COX genes sequence
6 mutations found in COI and COII
Different levels of heteroplasmy but levels significantly greater in the AD cohort
No disease-associated mutations in COIII gene
Results
506 Patients and 95 controls
10 clones of all three COX genes sequence
6 mutations found in COI and COII
Different levels of heteroplasmy but levels significantly greater in the AD cohort
No disease-associated mutations in COIII gene
AD cybrids but not controls had low COX activity
Results
506 Patients and 95 controls
10 clones of all three COX genes sequence
6 mutations found in COI and COII
Different levels of heteroplasmy but levels significantly greater in the AD cohort
No disease-associated mutations in COIII gene
AD cybrids but not controls had low COX activity
Increased production of ROS in AD cybrids
Critical evaluation:
How appropriate and robust are the methods ?
Is the data and evaluation robust ?
Are the conclusions valid, based on the reported data ?
Has the paper been validated by other labs and do the authors self-cite excessively ?
How does the paper stand the test of time ?
Is there any conflict of interest ?
Why has this not been reported before ?
mtDNA is destroyed by boiling
We’re looking at pseudogenes!
How can an apparent increase in nuclear pseudogene mutationspredispose to Alzheimers Disease and cause respiratory chaindeficiency in transmitochondrial cybrids ?
Critical evaluation:
How appropriate and robust are the methods ?
Is the data and evaluation robust ?
Are the conclusions valid, based on the reported data ?
Has the paper been validated by other labs and do the authors self-cite excessively ?
How does the paper stand the test of time ?
Is there any conflict of interest ?
How can an apparent increase in nuclear pseudogene mutationspredispose to Alzheimers Disease and cause respiratory chaindeficiency in transmitochondrial cybrids ?
