evidence based neurology maurizio a. leone neurologic clinic university hospital “maggiore della...
DESCRIPTION
On the need of Evidence Based Medicine 1.Few medical decisions are based on evidenceTRANSCRIPT
Evidence Based Evidence Based NeurologyNeurology
Maurizio A. Leone
Neurologic Clinic University Hospital
“Maggiore della Carità”,Novara, Italy
I International Course on Neuroepidemiology in Eastern Europe, Chisinau, Moldova. September 24-28, 2012
• The need for EBM
• How can I practice EBM ?
• What is EBM ?
• The story
• What are the “products” of EBM ?• Limitations of and criticisms to EBM
On the need of Evidence Based Medicine
1. Few medical decisions are based on evidence
Why do we need RANDOMIZED CONTROLLED TRIALS ?
In the early 1980s newly introduced antiarrhythmics were found to be highly successful at suppressing arrhythmias.
Not until a RCT was performed was it realized that, although these drugs suppressed arrhythmias, they actually increased mortality.
The CAST trial revealed Excess mortality of 56/1000.
By the time the results of this trial were published, at least 100,000 such patients had been taking these drugs.
Caveat
Many medical decisions are taken on the basis of:
• etiopathogenetic theories • experience• other factors (pharma industry, other
interests, ideology…)
Probability of a result favouring study drug in industry-sponsored trials OR = 4.05 (2.98-5.51)
On the need of Evidence Based Medicine
1. Few medical decisions are based on evidence
2. Deterioration of knowledge after graduation
Performance deteriorates ….
1. Level of blood pressure.2. Patient’s age.3. The physician’s year of
graduation from medical school.
4. The amount of target-organ damage.
Determinants of the clinical decision to treat some, but not other, hypertensives:
On the need of Evidence Based Medicine
1. Few medical decisions are based on evidence
2. Deterioration of knowledge after graduation
3. Delay in transferring results from research to clinical practice
Systematic review of bed rest after spinal tap
• 10 trials of bed rest after spinal puncture – no change in headache with bed rest– Increase in back pain
• Protocols in UK neurology units - 80% still recommend bed rest after LP
Serpell M, BMJ 1998;316:1709–10
• …evidence of harm available for 17 years preceding... Allen, Glasziou, Del Mar. Lancet, 1999
On the need of Evidence Based Medicine
1. Few medical decisions are based on evidence
2. Deterioration of knowledge after graduation
3. Delay in transferring results from research to clinical practice
4. Limited resources
Information epidemic: how to keep up-to-date ?
MEDLINE 20102,000 articles / day
approx 75 new trials published every day
Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)
Median minutes/week spent reading about my patients (self report):
• Medical Students: 90 minutes• House Officers: 0 (up to 70%=none)• Senior House Officers: 20 (up to 15%=none)• Registrars: 45 (up to 40%=none)• Senior Registrars 30 (up to 15%=none)• Consultants:
– Grad. Post 1975: 45 (up to 30%=none)– Grad. Pre 1975: 30 (up to 40%=none)
• The need for EBM• What is EBM ?• The story• How can I practice EBM ?• What are the “products” of EBM ?• Limitations of and criticism to EBM
Evidence (from Latin “evidentia”)
In Italian (Rumanian-Moldovan):• Anything that is clear and obvious, that
doesn’t need any further demonstration
In English:• The available body of information
indicating whether a belief or a proposition is true or valid (= proof, testimony, sign).
‘Background’ Questions
• About the disorder, test, treatment, etc.
2 components:a. Root* + Verb: “What causes …”b. Condition: “… stroke?”
• * Who, What, Where, When, Why, How
Foreground learning: Clinical decision-making, history taking, examining, diagnosing, and therapeutic intervening.
Background learning: basic neurosciences, neuroanatomy, neurophysiology, neuropharmacology, amd neuropathology
Evidence-Based Medicine:The Practice
1. Translation to an answerable question (patient/manoeuvre/outcome).
2. Efficient track-down of the best evidence3. Critical appraisal of the evidence for its validity
and clinical applicability4. Integration of that critical appraisal with clinical
expertise and the patient’s unique biology and beliefs.
5. Evaluation of one’s performance.
• The need for EBM• What is EBM ?• The story• How can I practice EBM ?• What are the “products” of EBM ?• Limitations of and criticism to EBM
Dave Sackett
Evidence-based Medicine: the storyPierre Charles Alexandre Louis (Paris, 1830): “A true science is merely a summary of facts which have no value unless they are expressed in numbers ... The statistics represent the only and fundamental basis of all the medical studies ‘. He was the promoter of “La Societé d’Observation Medicale”, a cultural movement supporting the concept that knowledge about a disease, its history, clinical presentation and treatment, could be derived from aggregated patient data rather than from individual experience.
Archibald Cochrane (1972): “It is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials.”
1986: The focus of Sackett and coll. gradually moves from "how to read the biomedical literature" to "how to use the biomedical literature to solve clinical problems."
1981: (McMaster Medical School, Canada): “How to read clinical journals", a series of articles describing the strategies of critical approach to the biomedical literature. These articles are among the top reprinted in the history of biomedical literature.
1993 JAMA: first article of the movement of Evidence-Based Medicine
1993: founding of the Cochrane Collaboration, an international network to “prepare, update and disseminate systematic reviews of controlled trials on the effects of health care and, where there is no controlled trials, systematic reviews of existing evidence, however.”
http://www.cochrane.org
• Back• Consumers and
communication• Dementia & cognitive
improvement• Incontinence • Infectious diseases• Injuries• Movement disorders• Multiple sclerosis
• Muskuloskeletal • Musculoskeletal injuries• Neuromuscular disease• Pain, palliative and
supportive care• Drugs and alcohol• Epilepsy• Eyes and vision• Stroke
Review Groups
• The need for EBM• What is EBM ?• The story• How can I practice EBM ?• What are the “products” of EBM ?• Limitations of and criticism to EBM
Evidence-Based Medicine:The Practice
1. Translation to an answerable question (patient/manoeuvre/outcome).
????
1st part PopulationClinical characteristics of the patient or of the reference patients population
2nd part
3rd part
Structure of the clinical question
Courtesy of Luca Vignatelli
1st part PopulationClinical characteristics of the patient or of the reference patients population
2nd part InterventionTherapy: drug / device /
procedureDiagnosis: test
ComparatorAlternative clinical act (placebo / no therapy / other therapy / other test)
Structure of the clinical question
1st part PopulationClinical characteristics of the patient or of the reference patients population
2nd part InterventionTherapy: drug / device /
procedureDiagnosis: test
ComparatorAlternative clinical act (placebo / no therapy / other therapy / other test)
3rd part OutcomeClinical outcome of interest (to be increased/reduced)
TimeTime dimension of the observation of the outcome
Structure of the clinical question
Can coffee reduce daytime drowsiness?
In healthy adults engaging in normal activity, does coffee reduce daytime drowsiness compared with de-caffeinated coffee?
Control
PopulationIntervention
Outcome
Formulate a research question that can be
answered
Components: PICO(T)• Population• Intervention• Control • Outcome • T (Time)
Evidence-Based Medicine:The Practice
1. Translation to an answerable question (patient/manoeuvre/outcome).
2. Efficient track-down of the best evidence
• The expert (colleague): lack of obiectivity and completeness of information.
• Medical books (printed): often incomplete, outdated, lack of quantitative data. Selection of scientific evidence is not systematic neither explicit; generally organized by disease and not by clinical presentations.
• Journals: too many, fragmentation of topics, rarity of definitive studies to be transferred to clinical practice.Traditional (narrative) reviews: same methodological shortcomings of the medical books: selection bias, self-quotation and harmony with the opinion of the author.Randomized Clinical Trials (RCTs): difficult to adapt their results to the individual patient. Selected populations, complex patients excluded. Ideal world and peculiar competence and motivation of doctors. Often surrogate rather than clinically significant end-points, often relative and not absolute (NNT) measures are reported.
Traditional information instruments
GUIDELINES / HTA
SECONDARY LITERATURE
PRIMARY LITERATURE
CLINICAL STUDIES
Pyramid of the productionof medical literature
Courtesy of Luca Vignatelli
DATA-BANKS OF GUIDELINES / HTA
SECONDARY DATA-BANKS
PRIMARY DATA-BANKS
RCTs DATA-BANKS
How to organize the searchof evidence in the literature
Clinical research, synthesis of studies and recommendations
Primary Studies
Results ofresearch
. .. .
.
. .. .
.
. .. .
.
. .. .
.
. .. .
.
. .. .
.
Systematic ReviewsMeta-analyses
Sinthesis
HealthTechnology Assessments (HTA)
Guidelines Recommendations
Evidence-Based Medicine:The Practice
1. Translation to an answerable question (patient/manoeuvre/outcome).
2. Efficient track-down of the best evidence3. Critical appraisal of the evidence for its validity
and clinical applicability
We needn’t always carry out all 5 steps to provide E-B Care
• The need for EBM• What is EBM ?• The story• How can I practice EBM ?• What are the “products” of EBM ?• Limitations of and criticism to EBM
Guidelines are:
• Recommendations of clinical practice,• produced through a systematic process,• to assist physicians and patients• in deciding which are the most
appropriate method of care• in specific clinical circumstances.
Aims of the guidelines
To ensure the highest degree of appropriateness of the interventions, reducing the possible variability in clinical decisions (due to lack of knowledge and subjectivity in the definition of strategies of care).
Basic methodological elements
• Multidisciplinarity• Use of systematic reviews• Explicit evaluation of quality of
evidence • Strenght of recommendations
Requirements of a guideline
• Validity• Reproducibility• Representativeness • Applicability• Flexibility• Clarity• Documentation• Update
Who produce guidelines?
• National and Regional Agencies
• Scientific Societies (EFNS,….)
• Research Institutions• Ad hoc Associations• ..
Data Bank IstituzioneNational Guideline Clearinghouse™ Agency for
Healthcare Research and Quality (USA)
National Library of Guidelines (GuidelinesFinder)
NHS Evidence (UK)
CMA Infobase Canadian Medical Association (CMA)
International Guidelines Library G-I-N – the Guidelines International Network
Where to find guidelines?
The quality of evidence (levels of evidence) depends on:
• Appropriatness of study design• Quality of study conduction and analysis• Effect size• Appropriatness and relevance of outcomes
The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable effects of a management strategy outweigh undesirable effects.
EFNS Guidelines: classes of evidence • Class I: An adequately powered prospective, randomized, controlled clinical trial with
masked outcome assessment in a representative population or an adequately powered systematic review of prospective randomized controlled clinical trials with masked outcome assessment in representative populations. The following are required:
• (a) randomization concealment• (b) primary outcome(s) is/are clearly defined• (c) exclusion/inclusion criteria are clearly defined• (d) adequate accounting for dropouts and crossovers with numbers sufficiently low to
have minimal potential for bias• (e) relevant baseline characteristics are presented and substantially equivalent among
treatment groups or there is appropriate statistical adjustment for differences• Class II: Prospective matched-group cohort study in a representative population with
masked outcome assessment that meets a–e above or a randomized, controlled trial in a representative population that lacks one criteria a–e
• Class III: All other controlled trials (including well-defined natural history controls or patients serving as own controls) in a representative population, where outcome assessment is independent of patient treatment
• Class IV: Evidence from uncontrolled studies, case series, case reports, or expert opinion
• Level A: (established as effective, ineffective, or harmful) requires at least one convincing class I study or at least two consistent, convincing class II studies
• Level B: (probably effective, ineffective, or harmful) requires at least one convincing class II study or overwhelming class III evidence
• Level C: (possibly effective, ineffective, or harmful) rating requires at least two convincing class III studies
EFNS Guidelines: rating of recommendations
Grades of Recommendation Assessment,
Development and Evaluation
Consensus Conference
Drafting of recommendations by a jury at the end of a presentation and consultation of experts summarizing scientific knowledge on a given topic.Agreement among different figures regarding controversial and complex health issues, to provide patients the best quality of care in relation to available resources.
Clinical pathway: result of guidelines adaptation to local situations, with their specific organizational and management characteristics.
Protocol: the detailed outline of the steps to be followed in the treatment or the diagnosis of a patient.
What is a systematic review?
• Is a research, conducted with an explicit and repeatable method to determine what is the best available evidence on the effectiveness of an intervention or the accuracy of a diagnostic test.
• The meta-analysis is the most well-known statistical tool associated with the systematic reviews. It is used to comprehensively analyze all the observations of controlled clinical trials and to provide a summary estimate.
FEATURES TRADITIONAL REVIEWS
SYSTEMATIC REVIEWS
Cook DJ et al Ann Int Med 1997; 126:320, adapted
Question often large often focused
Protocol no yes
Literature search strategy not specified explicit and reproducible
Selection of studies criteria not specified inclusion/exclusion criteria
Evaluation of studies variable rigorous and critical
Synthesis often qualitative summary quantitative summary (meta-analyses)
Inferences not always evidence-based generally evidence-based
Systematic Reviews seek to:
• Identify all relevant published and unpublished primary studies• Select studies or reports for inclusion• Critically appraising the quality of each study or report• Extract data• Quantitatively synthetise findings from individual studies or reports in an unbiased way• Interpret the findings and present a balanced and impartial summary with due consideration of any flaws.• SR may examine quantitative or qualitative evidence.
If we believe it,does it apply to our patient?
• Is our patient (or population) so different from those in the primary studies that the results may not apply?
• consider differences in: - time - many things change. - culture - both treatments and values of
outcomes can be different. - stage of illness or prevalence can affect results.
We believe it ! But —>> does it matter?
• Is the benefit worthwhile to my patient?Ask the patient about cultural values.
• Are the outcomes relevant for my patient?Think about Relative Risk Reduction vs. Absolute Risk to your patient. Potential benefit is the Absolute risk avoided in our patient!
Event Rate = 1 relapse at 2 years
RRR ARR NNT
Control Event Rate (CER)
Experimental Event rate (EER)
(CER-EER) /CER
(CER-EER)
1/AAR
IFNBMSG 76% 64%
(76-64)/76 = 16%
(76-64)=12%
100/12=8
Event Rate = 1 relapse at 2 years
RRR ARR NNT
Control Event Rate (CER)
Experimental Event rate (EER)
(CER-EER) /CER
(CER-EER)
1/AAR
IFNBMSG 76% 64% (76-64)/76 = 16%
(76-64)= 12%
100/12= 8
Hypothesis A
7,6% 6,4% (7,6-6,4)/7,6 = 16%
(7,6-6,4) = 1,2%
100/1,2= 83
Event Rate = 1 relapse at 2 years
RRR ARR NNT
Control Event Rate (CER)
Experimental Event rate (EER)
(CER-EER) /CER
(CER-EER)
1/AAR
IFNBMSG 76% 64% (76-64)/76 = 16%
(76-64)= 12%
100/12= 8
Hypothesis A
7,6% 6,4% (7,6-6,4)/7,6 = 16%
(7,6-6,4) = 1,2%
100/1,2= 83
Hypothesis B
0,76% 0,64% (0,76-0,64) /0,76= 16%
(0,76-0,64) = 0,12%
100/0,12= 833
Infezioni nosocomialiProfilassi vs controllo
Risk ratio.01 .05 .2 1 5
Study
Risk ratio (95% CI)
0.29 (0.17,0.49) Abele 0.07 (0.01,0.52) Aerdts 0.33 (0.18,0.62) Blair 0.82 (0.49,1.37) Boland 0.33 (0.11,0.99) Cockerill 0.69 (0.23,2.01) Finch 0.11 (0.01,2.05) Jacobs 1 0.15 (0.07,0.36) Kerver 0.39 (0.21,0.73) Palomar 0.32 (0.15,0.68) Rocha 0.57 (0.40,0.81) Sanchez 0.61 (0.47,0.78) Stoutnbek 0.28 (0.13,0.59) Ulrich 0.53 (0.33,0.85) Verwaest1 0.72 (0.47,1.10) Verwaest 2 0.18 (0.05,0.59) Winter
For(r)est plot
Punctual estimate
Favours treatment Favours control
No difference line
Confidence Intervals
Not statistically significant results
The pooled analysis of the results of all studies is represented by a diamond (weighted average).If the confidence interval includes the unity, this
means that we have not found a statisticallysignificant difference between the two treatments.
Total (95% CI) 410/1194 301/925 100.0 0.96 (0.85, 1.08)Test for heterogeneity 2=18.11 df=14 p=0.2020 I2=23.3%Test for overall effect z=0.68 p=0.50
Standard Meta-analysis (Left)&Cumulative Meta-analysis (Right)
Cochrane Database of Systematic Reviews
DARE: http://www.crd.york.ac.uk/crdweb
Where to find systematic reviews?
• The systematic evaluation of properties, effects, and/or impacts of health care technology
• It may address the direct, intended consequences of technologies as well as their indirect, unintended consequences
• Its main purpose is to inform technology-related policymaking in health care
• HTA is conducted by interdisciplinary groups using explicit analytical frameworks drawing from a variety of methods
(HTA glossary - www.htaglossary.net)
Health Technology Assessment
Were to find Health Technology Assessments
http://www.crd.york.ac.uk/crdweb/
• The need for EBM• What is EBM ?• The story• How can I practice EBM ?• What are the “products” of EBM ?• Limitations of and criticism to EBM
Limitations of EBM
1. Shortage of consistent scientific evidence2. Difficulties to apply evidence to the careof
individual patient3. Economic barriers to the practice of high-
quality medicine4. Need to develop new skills5. Limited time and resources6. Paucity of evidence that EBM “works”
Misperceptions of EBM
1. EBM denigrates clinical expertise2. It ignores patients’ values and preferences3. It promotes a cookbook approach to medicine4. It is simply a cost-cutting tool5. It is an ivory-tower concept6. It is limited to medical research7. It leeds to therapeutic nihilism in the absence of
evidence from randomized clinical trials
Patients’ values and
expectations
EBM for clinical decision-making
Sackett et al. 2000
Best available evidence
from medical research
Individual clinical
expertise
Available resources
Best Evidence Resources for Neurology
Guidelines (free access): American Academy of Neurology www.aan.com/go/practice/guidelines
• European Federation of Neurological Societies (EFNS) www.efns.org
CATs Data bank: Evidence Based Neurology, University of Western Ontario www.uwo.ca/cns/ebn/
Courtesy of Luca Vignatelli