gordon guyatt department of health research … guyatt department of health research methods...
TRANSCRIPT
Gordon GuyattDepartment of Health Research MethodsEvidence and ImpactMcMaster University
¡ Co-chair of the GRADE working group
¡ Have written about dangers of conflict of interest – therefore, when it comes to intellectual conflict of interest, I have intellectual conflict of interest
¡ Current problems in health care
¡ What is a guideline?
¡ What makes a trustworthy guideline?§ Collect and summarize evidence§ Move from evidence to recommendations
¡ Slow dissemination of innovations
¡ Related problems especially U.S§ Over diagnosis, overtreatment§ Low value care, waste§ Runaway costs
¡ Trustworthy guidelines could help
Clinical guidelines - Not guidelines on how to do tests in the laboratory
¡ Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.
¡ Most guidelines deal with therapeutic issues
¡ Guidelines in diagnosis present special challenges
¡ GOBSAT or trustworthy
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
¡ Experts§ Concern about conflict of interest
¡ Methodologists§ Concern about conflict of interest
¡ Front-line clinicians
¡ Patients§ Concern about representativeness
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
¡ Types§ Financial, Intellectual, Professional
¡ Possible approaches§ Ignore - GOBSAT§ Declare § Forbid § Manage
▪ Balance▪ Selectively forbid
¡ 500 recommendations, 14 panels
¡ COI tension
¡ COI financial § Grant funding, speaker’s bureaus, § Consultancy, advisory board, ownership
¡ COI intellectual§ Authorship paper directly bearing recommendations
¡ Conflict, no participation make recommendation
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
¡ Partial thromboplastin time accurate test (more or less) § Pre-op PTT in major surgery improves outcomes
¡ D-dimer accurate test (more or less)¡ Suspected DVT, high probability improves outcomes¡ Suspected DVT, low probability improves outcomes
¡ ELISA for HIT accurate test (more or less)¡ ELISA any pt suspected HIT improves outcomes¡ ELISA in patients with intermediate or high probability 4T
score improves outcomes
¡ Very low prevalence§ False positive harm outweigh benefits
¡ Clinicians misinterpret test result¡ Clinicians just as good without the test¡ Test results don’t lead to change in
management¡ Clinicians don’t offer optimal management¡ Optimal management doesn’t improve
outcomes
¡ PICO format¡ Pt: Suspect heparin induced thrombocytopenia
¡ Intervention: 4T + ELISA § Low 4T heparin; Intermediate or high on ELISA result
¡ Comparator: 4T § heparin for low, argatroban for intermediate and high
¡ Outcomes:§ Thrombosis § Bleeding§ Mortality§ Resource use (cost)
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
¡ Systematic reviews of best evidence
¡ What makes a systematic review systematic?§ Explicit eligibility criteria§ Comprehensive search§ Explicit consideration of evidence limitations
▪ Risk of bias, Imprecision, Inconsistency
§ Duplicate judgements (eligibility, RoB)§ Appropriate analysis
Study designs RCTs
Studies inform mortality, morbidity, symptoms, quality of life, resource use
Randomised Trial orObservational Study
Accuracy Study
Target population
New test(s)Old test(s) New test(s) + Reference test
One
step
inference
Twostepinference
Managementdependingon results
Patient-important outcomes
TP + FP FN + TN TP FP FN TN
Assumptionsor indirect evidence aboutmanagementof patients correctly or incorrectly classified aspositive or negative with the newor old test(s)
Judgements about patient-important outcomeswith a new test and a reference test
Managementdependingon results
Managementdependingon results
Patient-important outcomes
TP + FP FN + TN
Managementdependingon results
Target population
Study designs Linked EvidenceLook for diagnostic test accuracy studies
And then draw inferences from other evidence
Randomised Trial orObservational Study
Accuracy Study
Target population
New test(s)Old test(s) New test(s) + Reference test
One
step
inference
Twostepinference
Managementdependingon results
Patient-important outcomes
TP + FP FN + TN TP FP FN TN
Assumptionsor indirect evidence aboutmanagementof patients correctly or incorrectly classified aspositive or negative with the newor old test(s)
Judgements about patient-important outcomeswith a new test and a reference test
Managementdependingon results
Managementdependingon results
Patient-important outcomes
TP + FP FN + TN
Managementdependingon results
Target population
Strategy4(4Tsscore)
PatientswithsuspectedHIT(n=1000)
Lowprob (n=558)
T(n=1)MB(n=0)
Totaladverseoutcomes:Thrombosis(n=40)Majorbleeds(n=30)
Argatroban Argatroban
Intermediateprob (n=360) Highprob (n=82)
HIT(n=1)
NotHIT(n=557)
HIT(n=50)
NotHIT(n=310)
HIT(n=52)
NotHIT(n=30)
Notx
T(n=18)MB(n=3)
T(n=9)MB(n=3)
T(n=3)MB(n=19)
T(n=9)MB(n=3)
T(n=0)MB(n=2)
Strategy5(4Tsscore+PolyspecificELISA):Testpatientswithanintermediate/high4Tsscore
andtreatifpositive
PatientswithsuspectedHIT(n=1000)
Lowprob (n=558)
T(n=1)MB(n=0)
Totaladverseoutcomes:Thrombosis(n=48versus40)Majorbleeds(n=12versus30)
Intermediateprob (n=360) Highprob (n=82)
HIT(n=1)
NotHIT(n=557)
HIT(n=52)
NotHIT(n=30)
Notx
T(n=18)MB(n=3)
HIT(n=50)
NotHIT(n=310)
ELISA+(n=48)
ELISA-(n=2)
ELISA+(n=41)
ELISA-(n=269)
T(n=18)MB(n=6)
ELISA+(n=50)
ELISA-(n=2)
ELISA+(n=4)
ELISA-(n=26)
T(n=11)MB(n=3)
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
¡ Many available
¡ Cause of confusion, dismay
¡ GRADE (Grades of recommendation, assessment, development and evaluation)
¡ Guidance: BMJ 2004, 6 part series 2008
>100organizationshaveadoptedGRADE
AssessingQualityoftheEvidencebyOutcome
Table:'GRADE's'approach'to'rating'quality'of'evidence'(aka'confidence'in'effect'estimates)'For$each$outcome$based$on$a$systematic$review$and$across$outcomes$(lowest$quality$across$the$outcomes$critical$for$decision$making)
1.''Establish'initial'level'of'certainty'
' 2.''Consider'lowering'or'raising'
level'of'certainty'
' 3.''Final'level'of''
certainty'rating'
Study&design& Initial&certainty&&in&the&evidence&
' Reasons&for&considering&lowering&&or&raising&certainty&&
' Certainty&&in&the&evidence&&
across&those&considerations&''! 'Lower'if' ''" 'Higher'if*'
Randomized&trials! & High'certainty'
Risk'of'Bias!
Inconsistency!
Indirectness!
Imprecision'
Publication'bias'
Large'effect'
Dose'response'
All'plausible''confounding'&'bias'• would!reduce!a!demonstrated!effect!!
'''or'• would!suggest!a!spurious!effect!if!no!effect!was!observed!
High!⊕⊕⊕⊕'
! ' Moderate'⊕⊕⊕!'
Observational&studies! & Low'certainty'
Low'⊕⊕!!'
! ' Very'low'⊕!!!'
*upgrading criteria are usually applicable to observational studies only.
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
Whose values?: Patients
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
¡ strong recommendation§ benefits clearly outweigh risks/hassle/cost§ risk/hassle/cost clearly outweighs benefit
¡ What can downgrade strength?
¡ Low quality evidence
¡ Close balance between up and downsides
¡ Variability in patient preference§ strong, almost all same choice (> 90%)§ weak, choice varies appreciably
¡ Interaction with patient§ strong, just inform patient§ weak, ensure choice reflects values
¡ Use of decision aid§ strong, don’t bother; weak, use the aid
¡ Quality of care criterion§ strong, consider; weak, don’t consider
¡ Look to trustworthy guidelines – reject GOBSAT
¡ Panel: experts, methodologists, clinicians, patients
¡ Deal effectively with conflict of interest
¡ Gather and summarize the best evidence§ GRADE
¡ Rate strength of recommendations§ GRADE