does journal endorsement of reporting guidelines influence ... · does journal endorsement of...
TRANSCRIPT
Does journal endorsement of reporting
guidelines influence the completeness
of reporting of health research?
Larissa Shamseer, MSc Clin Epi
Reporting Guideline Research Coordinator
11 October 2012, Freiburg, DE
Project Investigators
• David Moher, OHRI, Canada
• Doug Altman, The EQUATOR Network, Centre for Statistics in
Medicine (CSM), University of Oxford, UK
• Allison Hirst, CSM, University of Oxford, UK
• John Hoey, Queen’s University, Kingston, Canada
• Anita Palepu, St. Paul’s Hospital, Vancouver, Canada
• Ken Schulz, FHI360, North Carolina, USA
• Iveta Simera, The EQUATOR Network, Centre for Statistics in
Medicine (CSM), University of Oxford, UK
• Project Funding: CIHR (FRN 111750)
Research Staff
Staff Leads: Adrienne Stevens & Larissa Shamseer
Hadeel AlYacoob
Raymond Daniel
Mary Gauthier
Mandi Kaur
Misty Pratt
Kavita Singh
Becky Skidmore
Justin Thielman
Sophia Tsouros
Lucy Turner
Erica Weinstein
Fatemeh Yazdi
Review Objectives
To evaluate whether journal endorsement of reporting
guidelines influences the completeness of the reported
literature by comparing:
1. Completeness of reporting of studies published in
journals endorsing reporting guidelines before and
after endorsement.
2. Completeness of reporting of studies published in
journals that have and have not endorsed reporting
guidelines.
Methods
• Search: (1) reporting guidelines (2) evaluations of RGs
• Screening: determine relevant evaluations, journal
endorsement status for comparisons
• Validity assessment: 6 pre-specified criteria
• Outcome extraction and analyses
• obtain data for all included studies
• Completeness of reporting – RRs/SMDs (99% CIs),
where possible
(by item, total score, flow diagram, descriptive)
• Methodological quality - narrative
• Unwanted effects - narrative
Methods (2)
• Statistical heterogeneity: I2 statistic
• Subgroup analyses:
• Extent of endorsement
• Variations in checklist items
• Official and unofficial extensions
• Sensitivity analyses:
• Six-month endorsement period
• Study outliers
Eligibility criteria - inclusions
• Primary intent: completeness of reporting
• Exclude methodologic quality / quality assessment
• Exclude SRs addressing clinical Q
• Evaluates one of 101 RGs of interest
• Exclude those that do not explicitly state assessing RG
• At least one study in journal that endorses RG
• Exclude evaluations of conference abstracts
• Contacted authors
• Follow-up with journals
Eligibility criteria - exclusions
• Inappropriate use of RG
• Eg, STROBE for abstracts
• Eg, STARD for reviews
• Assesses studies before RG published
• Eg, used PRISMA on reviews published before 2009
• Interrupted series designs, controlled before-after
analyses
PRISMA flow - Included RGs
• General – 5
• Animal – 2
• Basic science – 1
• Lab/pre-clinical – 10
• Case reports – 1
• Observational – 9
• Non-randomized – 1
• Prospective studies – 5
• General, clinical trials – 14
• Various content areas – 13
• RCTs – 18
• RCT + observational – 1
• RCTs, quasi, SRs, m/a – 1
• SRs + m/a – 3
• Diagnostic studies – 4
• Economics – 5
• Evaluation studies – 2
• Qualitative – 2
• QI studies – 1
• Standardized patient reports – 1
• Methodology – 1
• Validation studies – 1
98 potential RGs
(EQUATOR search)
76 excluded
• Not reporting guidance (12)
• Methods not described (56)
• Not explicitly consensus–based (4)
• Duplicates/companion reports (3)
• CONSORT (1) 79 RGs (Moher 2011) 22 RGs +
101 included RGs
PRISMA flow –
Included Evaluations
Records screened (n = 15240) Records excluded (n = 14096)
Full-text - round 1 (n = 1144) Full-text excluded, with reasons (n = 826)
Full-text - round 2 (n = 318) Full-text excluded, with reasons (n = 159)
Duplicate & multiple reports quarantined (n=22)
Contact authors & journals / Modified checklists (n = 159)
Full-text excluded, with reasons (n = 139)
Included (n=20)
Other source records (n = 40)
Database records (n = 17225)
After duplicates removed (n = 15262)
Included (n=20)
Included studies table only
(n=10) Analysis (n=10)
(7/101 RGs evaluated)
Final included
evaluations
1. CONSORT for harms, 2004 (n=3): Haidich 2011, Turner 2011, Lee 2008
2. CONSORT for herbal medicines, 2006 (n=1): Ernst 2011
3. BMJ economic checklist, 1996 (n=2): Herman 2005, Jefferson 1998
4. QUOROM, 1999 (n=3): Hind 2007, Biondi Zoccai 2006, Poolman 2007
5. STARD, 2003 (n=8): Freeman 2009, Mahoney 2007, Selman 2011, Smidt
2006, Coppus 2006, Johnson 2007, Krzych 2009, Paranjothy 2007
6. STRICTA, 2002 (n=1): Hammerschlag 2011
7. STROBE, 2007 (n=2): Parsons 2011, Delaney 2010
Characteristics of studies
• 7 countries:
• UK, US, Canada, Netherlands, Greece, Italy, Poland
• Study funding:
• Not reported, Government, Charity, Industry, No funding,
Other
• Range of interventions
• Diagnostic studies, Complementary medicine, Drug
therapies, Therapeutic interventions (general), Surgery,
Biochem and Lab Methods, Unspecified
• Range of medical specialties
• Unspecified, Medicine (General/Internal), Obstet/Gyne,
Orthopedics, Cardiac/Cardiovascular, Endocrinology &
Metabolism, Hematology, Neurology, Reproductive,
Urology/Nephrology
Included evaluations
Guideline Endorsers v. Non-endorsers Before v. After Endorsement
# evals # journals # studies # evals # journals # studies
BMJ econ 2 2 vs. 11 3 vs. 16 1 1 1 vs. 8
CONSORT
- Harms
3 8 vs. 108 31 vs. 267 1 1 1 vs. 2
CONSORT
- Herbals
1 1 vs. 3 1 vs. 4 - - -
QUOROM 2 2 vs. 11 2 vs. 12 1 1 13 vs. 15
STARD 8 26 vs. 71 133 vs.
168
2 8 98 vs. 79
STRICTA
(2002)
1 3 vs. 64 17 vs. 129 1 2 11 vs. 4
STROBE 2 3 vs. 9 10 vs. 42 1 2 9 vs. 11
Endorser vs. Non-endorser
BMJ Economics guidelines
Item # studies Risk ratio and 99% CI (Random)
RR (99%CI)
Economic importance of question 1 13 1.18 (0.52, 2.67)
Clearly describe comparisons 1 13 1.18 (0.52, 2.67)
Form of economic evaluation 1 13 2.86 (0.75, 10.89)
Justify choice of economic evaluation 1 13 1.33 (0.03, 63.52)
Source(s) of effectiveness estimates 1 13 1.00 (0.51, 1.97)
Design and results of effectiveness study (single study) 1 12 1.00 (0.50, 1.99)
Primary economic evaluation outcome measure(s) 1 13 0.95 (0.46, 1.96)
Subjects from whom valuations obtained 1 4 3.00 (0.08, 114.59)
Quantities of resources separate from unit costs 1 13 3.00 (0.12, 77.64)
Methods for estimating quantities and unit costs 1 13 1.05 (0.49, 2.26)
Currency of price adjustments for inflation or currency conversion 1 13 6.67 (0.84, 53.19)
Time horizon of costs and benefits 1 13 1.00 (0.51, 1.97)
Statistical tests and CIs for stochastic data 1 12 1.00 (0.50, 1.99)
Compare relevant alternatives 1 13 1.38 (0.17, 11.13)
Incremental analysis 1 4 1.67 (0.33, 8.48)
Major outcomes in aggregated and dissaggregated forms 1 13 0.95 (0.46, 1.96)
Answer to study question 1 13 1.05 (0.49, 2.26)
Conclusions follow from data 1 13 1.00 (0.51, 1.97)
Conclusions with appropriate caveats 1 13 0.61 (0.10, 3.82)
0.1 0.2 0.5 1 2 5 10
Favours
non-endorsement
Favours
endorsement
# evals
Endorser vs. Non-endorser
CONSORT for Harms
Item # evals Risk ratio and 99% CI (Random) RR (99%CI)
Title or abstract 1 102 0.94 (0.66, 1.34)
Introduction 1 102 1.00 (0.54, 1.85)
Outcomes - List adverse events and definitions 2 296 1.24 (0.42, 3.64)
Outcomes - How information collected 2 296 1.18 (0.95, 1.46)
Statistical methods - Plans for presenting/analyzing harms 2 296 0.89 (0.45, 1.78)
Participant flow 2 296 0.90 (0.47, 1.72)
Numbers analyzed 2 296 0.98 (0.68, 1.41)
Absolute risk for each adverse event and appropriate metrics 1 102 0.98 (0.79, 1.22)
Subgroup and exploratory analyses 1 102 0.98 (0.38, 2.53)
Discussion 1 102 1.08 (0.85, 1.37)
0.1 0.2 0.5 1 2 5 10
Favours
non-endorsement
Favours
endorsement
# studies
Endorser vs. Non-endorser
STARD
Item # evals # studies Risk ratio and 99% CI (Random) RR ( 99% CI)
Title, abstract, keywords 2 62 1.03 (0.49, 2.15)
Introduction 3 88 1.07 (0.82, 1.39)
Participants - Study population 3 88 1.36 (0.56, 3.34)
Participants - Recruitment 3 88 1.07 (0.68, 1.68)
Participants - Sampling 3 88 0.73 (0.41, 1.28)
Participants - Data collection 3 88 0.94 (0.74, 1.19)
Test methods - Reference standard 3 88 1.03 (0.81, 1.33)
Test methods - Technical specifications 2 38 1.00 (0.61, 1.64)
Test methods - Readers blinded to other result? 2 38 0.83 (0.02, 42.02)
Statistical methods - Measures and uncertainty 2 60 2.19 (0.93, 5.14)
Statistical methods - Test reproducibility 2 38 0.56 (0.12, 2.64)
Results - Recruitment 3 87 0.83 (0.54, 1.28)
Results - Participant characteristics 3 88 1.77 (1.16, 2.71)
Results - Participant flow 4 227 1.35 (0.84, 2.17)
Results - Time interval from test to reference 3 88 0.51 (0.20, 1.32)
Results - Disease severity 3 88 0.73 (0.08, 6.84)
Results - Cross-tabulation of test by reference results 3 88 1.28 (0.70, 2.34)
Results - Any adverse events 3 68 1.42 (0.34, 6.03)
Results - Diagnostic accuracy estimates and uncertainty 2 60 1.82 (0.95, 3.50)
Results - How indeterminate results, missing data, outliers handled 3 79 1.44 (0.86, 2.41)
Results - Variability between subgroups 2 29 1.32 (0.75, 2.31)
Results - Test reproducibility 2 38 0.56 (0.12, 2.64)
Discussion 3 88 1.00 (0.89, 1.12)
0.1 0.2 0.5 1 2 5 10
Favours
non-endorsement
Favours
endorsement
Endorser vs. Non-endorser
STROBE
Item # evals Risk ratio and 99% CI (Random)
Title or Abstract 1 47 0.40 (0.08, 2.09)
Abstract 1 47 1.02 (0.73, 1.43)
Introduction - Background and rationale 1 47 1.02 (0.82, 1.26)
Introduction - Objectives 1 47 1.09 (0.64, 1.86)
Methods - Study design 1 47 0.94 (0.68, 1.29)
Methods - Setting, location, dates 1 47 1.06 (0.53, 2.11)
Methods - Participants eligibility 1 47 0.99 (0.71, 1.39)
Methods - Participant matching 1 24 1.50 (0.55, 4.10)
Methods - Outcome, exposure, other variables 1 47 0.88 (0.38, 2.03)
Methods - Data sources and measurement 1 47 0.91 (0.67, 1.24)
Methods - Addressing sources of bias 1 47 0.84 (0.15, 4.87)
Methods: - Study size 1 47 1.58 (0.73, 3.43)
Methods - Handling of quantitative variables 1 47 0.92 (0.56, 1.51)
Methods - Statistical methods 1 47 0.92 (0.56, 1.51)
Methods - Subgroups and interactions 1 47 1.54 (0.48, 4.93)
Methods - Loss to follow-up, matching, sampling 1 42 0.81 (0.14, 4.76)
Methods - Sensitivity analyses 1 47 1.21 (0.19, 7.53)
Results - Participant flow 1 47 0.99 (0.71, 1.39)
Results - Reasons for nonparticipation 1 47 0.84 (0.06 , 12.01)
Results - Flow diagram 1 47 2.81 (0.33 , 24.14)
Results - Participant characteristics 1 47 1.61 (0.99, 2.61)
Results - Missing data 1 47 0.70 (0.13, 3.93)
Results - Follow-up time (cohort studies) 1 10 1.08 (0.56, 2.08)
Results - Outcome data 1 47 0.95 (0.58, 1.56)
Results - Estimates and precision 1 47 0.82 (0.43, 1.56)
Results - Boundaries for continuous variable categories 1 44 0.49 (0.04, 6.27)
Results - Translating relative into absolute risk estimates 1 35 1.04 (0.02 , 61.88)
Results - Other analyses 1 46 1.23 (0.30, 5.02)
Discussion - Key results 1 47 0.99 (0.81, 1.21)
Discussion - Limitations 1 47 1.51 (1.05, 2.18)
Discussion - Interpretation 1 47 1.26 (0.93, 1.70)
Discussion - Generalizability 1 47 1.35 (0.88, 2.07)
Other - Funding 1 47 1.47 (0.93, 2.32)
0.1 0.2 0.5 1 2 5 10
Favours
non-endorsement
Favours
endorsement
RR (99% CI) # studies
Endorser vs. Non-endorser
Total sum score of items
Guideline Evaluation Studies SMD (99% CI)
BMJ economics
guideline, 1996
1 2 vs. 11 3.47 (0.58 to 6.35)
CONSORT for
harms, 2004
2 30 vs. 266 -0.06 (-0.59 to 0.47)
STARD, 2003 3 88 0.55 (-0.08 to 1.18)
STRICTA, 2002 1 17 vs. 129 1.35 (-0.11 to 2.81)
STROBE, 2007 1 9 vs. 38 0.33 (-0.63 to 1.29)
After vs. Before Endorsement
QUOROM
Item # studies Risk ratio and 99% CI (Random)
RR (99%CI)
Flow diagram 1 28 1.01 (0.41, 2.49)
0.1 0.2 0.5 1 2 5 10
Favours
non-endorsement
Favours
endorsement
# evals
After vs. Before Endorsement
STARD
Item # studies Risk ratio and 99% CI (Random) RR (99%CI)
Flow diagram 1 173 11.49 (0.82 ,160.60)
0.01 0.1 1 10 100
Favours
non-endorsement
Favours
endorsement
# evals
After vs. Before Endorsement
STROBE
Item # evals # studies Risk ratio and 99% CI (Random) RR (99% CI)
Title or Abstract 1 20 1.22 (0.12, 12.20)
Abstract 1 20 0.98 (0.66, 1.45)
Introduction - Background and rationale 1 20 1.00 (0.78, 1.28)
Introduction - Objectives 1 20 0.86 (0.51, 1.45)
Methods - Study design 1 20 0.89 (0.61, 1.29)
Methods - Setting, location, dates 1 20 1.83 (0.56, 6.05)
Methods - Participants eligibility 1 20 0.89 (0.61, 1.29)
Methods - Participant matching 1 15 1.00 (0.38, 2.62)
Methods - Outcome, exposure, other variables 1 20 0.76 (0.31, 1.88)
Methods - Data sources and measurement 1 20 0.98 (0.66, 1.45)
Methods - Addressing sources of bias 1 20 0.81 (0.11, 6.31)
Methods: - Study size 1 20 1.83 (0.56, 6.05)
Methods - Handling of quantitative variables 1 20 0.86 (0.51, 1.45)
Methods - Statistical methods 1 20 1.07 (0.55, 2.08)
Methods - Subgroups and interactions 1 20 4.89 (0.35, 68.31)
Methods - Loss to follow-up, matching, sampling 1 19 0.74 (0.10, 5.65)
Methods - Sensitivity analyses 1 20 2.44 (0.13, 45.99)
Results - Participant flow 1 20 1.09 (0.68, 1.75)
Results - Reasons for nonparticipation 1 20 3.60 (0.06, 208.53)
Results - Flow diagram 1 20 6.00 (0.13 , 277.77)
Results - Participant characteristics 1 20 1.63 (0.75, 3.53)
Results - Missing data 1 20 6.00 (0.13, 277.77)
Results - Follow-up time (cohort studies) 1 5 1.00 (0.44, 2.28)
Results - Outcome data 1 20 0.86 (0.51, 1.45)
Results - Estimates and precision 1 20 0.68 (0.37, 1.25)
Results - Boundaries for continuous variable categories 1 20 0.41 (0.03, 6.31)
Results - Translating relative into absolute risk estimates 1 18 0.41 (0.01, 23.24)
Results - Other analyses 1 20 0.92 (0.19, 4.48)
Discussion - Key results 1 20 1.00 (0.78, 1.28)
Discussion - Limitations 1 20 1.34 (0.80, 2.24)
Discussion - Interpretation 1 20 1.09 (0.78, 1.53)
Discussion - Generalizability 1 20 0.98 (0.66, 1.45)
Other - Funding 1 20 1.40 (0.72, 2.72)
0.1 0.2 0.5 1 2 5 10
Favours
non-endorsement
Favours
endorsement
After vs. before endorsement
Total sum score of items
Guideline Evaluation Studies SMD (99% CI)
STRICTA,
2002
1 11 vs. 4 0.63 (-0.91 to 2.17)
STROBE, 2007 1 9 vs. 11 0.26 (-0.90 to 1.42)
Secondary outcomes
• Methodological quality of evaluations’ included studies
• 6 of 20 studies reported information
• 2 studies used Oxman-Guyatt
• 1 studies used Cochrane Risk of Bias tool
• 1 study assessed generalizability and reporting
• 1 study used QUADAS but could not extract as per
our comparisons
• 1 study assessed internal and external validity
• Unwanted effects from use of guideline
• No studies assessed
Results
• Only 7/101 RGs evaluated
• Few existing evaluations, each assessing completeness
of reporting of a small number of studies; subgroup &
sensitivity analyses not possible
• Few studies assessed methodological quality
• No studies assessed unwanted effects
vs.
• CONSORT SR (50 evaluations, >16000 RCTs)
• 25 of 27 items favoured endorsement vs. non-
endorsement with 5 items statistically significant
• 5 items stat sig for after-before endorsement
Strengths and Limitations
• Strengths:
• First cumulative evaluation of RG effectiveness
• Used best comparisons from CONSORT SR
• Reworked authors’ data to enable comparisons
• Limitations:
• Were all potential evaluations located?
• For some, ‘partial’ reporting may have been included
in ‘complete’ data by authors
• Not able to obtain all data – authors’ availability to
answer queries, some items considered ‘not
applicable’
Discussion
• Why so few evaluations?
• We couldn’t put a drug to market without evaluating it!
• Potentially more evaluations that we wished to
include, but could not
• What is the intervention?
• Journal endorsement (not just guideline publication) is
where the rubber-meets-the-road
• Extent of endorsement – needs to be clear and have
an implementation strategy
Published protocol
Shamseer L, Stevens A, Skidmore B, Turner L, Altman DG, Hirst A,
Hoey J, Palepu A, Simera I, Schulz KF and Moher D. Does journal
endorsement of reporting guidelines influence the completeness of
reporting of health research? A systematic review protocol. Systematic
Reviews 2012, 1:24 doi:10.1186/2046-4053-1-24
http://www.systematicreviewsjournal.com/content/1/1/24/abstract
28
Changes from protocol
• Did not include CONSORT
• Refer to CONSORT SR
• Did not include variations in checklist items
• Changes to validity assessment
• 2 items clarified wording
• Confounding
Collate reports / Contact authors,
journals/ Assess modified
checklists (n=159)
Full-text excluded (n=139): • Evaluates CONSORT (n=45) • Multiple report of excluded study (n=27) • No response from author to determine eligibility (n=19) • No journals endorsed the RG (n=14) • Authors used modified checklist (n=9) • List of studies or journals not provided (n=6) • Inappropriate use of RG (n=3) • Comparison not possible given study dates and endorsement information (n=3) • Assessed studies before RG published (n=2) • Full report or information not in English or French (n=2) • Could not locate author contact information (n=2) • All journals endorsed the RG, comparison not relevant (n=2) • Did not address completeness of reporting (n=1) • Full report or information not available from the author (n=1) • Reporting guideline publication (n=1) • Could not locate journal website or instructions to authors (n=1) • Not comparison of interest (n=1)
Included (n=20)
Validity – Endorsing vs Non-endorsing Author, Year Studies Two or more
assessors for
reporting
No evidence
of selective
reporting
Comprehensive
search strategy
Balance of
studies per
journal
BMJ economic guidelines, 1996
Herman, 2005 2 vs. 11 Unclear High Low High
Jefferson, 1998 1 vs. 5 Unclear Unclear High High
CONSORT extension for harms, 2004
Haidich, 2011 25 vs. 77 High High High Low
Lee, 2008 1 vs. 1 High High High High
Turner, 2011 5 vs. 189 Low High Low Low
CONSORT extension for herbal interventions, 2006
Ernst, 2011 1 vs. 4 Unclear High Low High
QUOROM, 1999
Biondi-Zoccai, ‘06 1 vs. 6 High High High High
Poolman, 2007 1 vs. 6 High Unclear Low High
STRICTA, 2002
Hammerschlag, ‘11 17 vs. 129 Low High Low Unclear
STROBE, 2007
Delaney, 2010 1 vs. 4 High Unclear Low High
Parsons, 2011 9 vs. 38 Low Unclear Low Low
Validity – Endorsing vs Non-endorsing
Author, Year Studies Two or more
assessors for
reporting
No evidence
of selective
reporting
Comprehensive
search strategy
Balance of
studies per
journal
STARD, 2003
Coppus, 2006 8 vs. 19 Low High Low High
Freeman, 2009 3 vs. 9 Unclear High High High
Johnson, 2007 1 vs. 10 High High Low High
Krzych, 2009 4 vs. 21 Unclear High Low High
Mahoney, 2007 6 vs. 20 High High Low High
Paranjothy, 2007 1 vs. 8 High High Low High
Selman, 2011 15 vs. 35 High Low Low Low
Smidt, 2006 95 vs. 46 High High Low Low
Validity – After vs. before endorsement
Author, Year Studies Two or more
assessors for
reporting
No evidence
of selective
reporting
Comprehensive
search strategy
Balance
of studies
per
journal
Sampling
took place
after RG
publication
BMJ economic guidelines, 1996
Jefferson, 1998 1 vs. 8 Unclear Unclear High High High
CONSORT extension for harms, 2004
Lee, 2008 1 vs. 2 High High High Low Low
QUOROM, 1999
Hind, 2007 13 vs. 15 Low High Unclear High High
STARD, 2003
Selman, 2011 3 vs. 1 High Low Low Low High
Smidt, 2006 95 vs. 78 High High Low Low Low
STRICTA, 2002
Hammerschlag,
2011 11 vs. 4 Low High Low High Low
STROBE, 2007
Parsons, 2011 9 vs. 11 Low Unclear Low Low Low
CONSORT SR – by item
Endorser vs. Non After vs. Before
CONSORT Checklist Item # of Evals
# of RCTs
Pooled Risk ratios and 99% CI
RR 99% CI
Title and Abstract 3 532 1.41 (0.63, 3.16) Introduction 2 457 1.04 (1.00, 1.08) Participants 4 622 0.98 (0.88, 1.09) Interventions 4 630 1.02 (0.97, 1.07) Objectives 2 517 1.04 (0.91, 1.18) Outcomes 5 716 1.43 (0.85, 2.40) Sample Size 6 983 1.30 (0.71, 2.37) Sequence Generation 8 1,085 1.46 (0.88, 2.44) Allocation Concealment 6 855 1.23 (0.55, 2.75) Implementation 2 517 1.94 (0.15, 24.7) Blinding of Paticipants 1 75 0.77 (0.45, 1.31) Blinding of Intervention 1 75 0.26 (0.09, 0.74) Blinding of Outcome Assessor 1 75 0.66 (0.34, 1.30) Blinding of Data Analyst 1 75 0.27 (0.02, 3.71) Blinding Any description 4 926 0.96 (0.61, 1.51) Statistical Methods 5 1,111 0.86 (0.62, 1.19) Participant Flow 8 992 1.33 (0.95, 1.87) Recruitment 3 828 1.77 (0.48, 6.49) Baseline Data 2 529 1.42 (1.24, 1.62) Numbers Analysed 6 1,005 1.72 (1.18, 2.50) Outcomes and Estimation 3 532 1.35 (0.73, 2.50) Ancillary Analyses 1 442 3.46 (2.47, 4.84) Adverse Events 3 507 1.39 (1.12, 1.73) Interpretation 1 442 1.01 (0.98, 1.04) Generalisability 1 442 1.77 (2.12, 1.48) Overall Evidence 2 517 1.31 (0.99, 1.73)
0.5 1 2
Favours non-endorsement
Favours endorsement
# of Evals
# of RCTs
Pooled Risk ratios and 99% CI
RR 99% CI
7 1,233 1.13 (0.96, 1.33) 5 513 1.07 (1.01, 1.14) 6 683 0.95 (0.56, 1.62) 6 638 1.00 (0.95, 1.05) 5 540 1.01 (0.96, 1.06) 8 1,302 1.17 (0.95, 1.44) 11 1,843 1.61 (1.13, 2.29) 14 2,231 1.59 (1.38, 1.84) 16 2,396 1.81 (1.25, 2.62) 5 498 1.47 (0.65, 3.32) 5 711 1.39 (0.87, 2.22) 5 710 1.25 (0.74, 2.12) 5 719 1.72 (0.69, 4.30) 3 497 3.56 (0.40, 31.8) 8 1,851 1.23 (0.93, 1.62) 9 894 1.03 (0.90, 1.18) 16 2,461 1.16 (0.94, 1.44) 6 959 1.03 (0.75, 1.41) 5 529 1.07 (0.94, 1.22) 13 2,145 1.23 (0.98, 1.55) 6 617 1.00 (0.95, 1.05 4 378 1.31 (0.48, 3.58) 8 911 1.14 (0.86, 1.52) 5 540 1.01 (0.96, 1.06) 5 540 1.22 (0.88, 1.70) 4 317 1.03 (0.91, 1.17)
0.5 1 2
Favours non-endorsement
Favours endorsement
• Stat sig: Introduction, Baseline data, Numbers
analyzed, Ancillary analyses, Adverse events,
Generalizability, Sum score
• Stat sig: Alloc conceal, Introduction,
Sample size, Sequence generation,
Sum score