cochrane diagnostic test accuracy reviews introduction to meta-analysis jon deeks and yemisi...
TRANSCRIPT
![Page 1: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/1.jpg)
Cochrane Diagnostic test accuracy reviews
Introduction to meta-analysis
Jon Deeks and Yemisi TakwoingiPublic Health, Epidemiology and BiostatisticsUniversity of Birmingham, UK
![Page 2: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/2.jpg)
Outline Analysis of a single study Approach to data synthesis Investigating heterogeneity Test comparisons RevMan 5
![Page 3: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/3.jpg)
Test accuracy
What proportion of those with the disease does the test detect? (sensitivity)
What proportion of those without the disease get negative test results? (specificity)
Requires 2×2 table of index test vs reference standard
![Page 4: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/4.jpg)
2x2 Table – sensitivity and specificity
Disease (Reference test)
Present Absent
Indextest
+ TP FP TP+FP
- FN TN FN+TN
TP+FN FP+TN TP+FP+FN+TN
sensitivityTP / (TP+FN)
specificityTN / (TN+FP)
![Page 5: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/5.jpg)
Heterogeneity in threshold within a study
diseasednon-diseased
specificity=100% sensitivity=100%
0 40 80 120 160
test measurement
diagnostic threshold
![Page 6: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/6.jpg)
Heterogeneity in threshold within a study
TN FN FP TP
specificity=99% sensitivity=69%
diseasednon-diseased
0 40 80 120 160
test measurement
diagnostic threshold
![Page 7: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/7.jpg)
Heterogeneity in threshold within a study
TN FN FP TP
specificity=98% sensitivity=84%
diseasednon-diseased
0 40 80 120 160
test measurement
diagnostic threshold
![Page 8: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/8.jpg)
Heterogeneity in threshold within a study
TN FNFP TP
specificity=93% sensitivity=93%
diseasednon-diseased
0 40 80 120 160
test measurement
diagnostic threshold
![Page 9: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/9.jpg)
Heterogeneity in threshold within a study
TN FN FP TP
specificity=84% sensitivity=98%
diseasednon-diseased
0 40 80 120 160
test measurement
diagnostic threshold
![Page 10: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/10.jpg)
Heterogeneity in threshold within a study
TN FN FP TP
specificity=69% sensitivity=99%
diseasednon-diseased
0 40 80 120 160
test measurement
diagnostic threshold
![Page 11: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/11.jpg)
Threshold effect
0.0
0.2
0.4
0.6
0.8
1.0
sen
sitiv
ity
0.00.20.40.60.81.0
specificity
Increasing threshold decreases sensitivity but increases specificity
Decreasing threshold decreases specificity but increases sensitivity
Threshold Sensitivity Specificity
65 0.99 0.69
70 0.98 0.84
75 0.93 0.93
80 0.84 0.98
85 0.69 0.99
![Page 12: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/12.jpg)
Ex.1 Distributions of measurements and ROC plotno difference, same spread
diseasednon-diseased
0 40 80 120
test measurement
0.0
0.2
0.4
0.6
0.8
1.0
sen
sitiv
ity
0.00.20.40.60.81.0
specificity
Uninformative test
![Page 13: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/13.jpg)
Ex.2 Distributions of measurements and ROC plotsmall difference, same spread
diseasednon-diseased
0 40 80 120
test measurement
0.0
0.2
0.4
0.6
0.8
1.0
sen
sitiv
ity
0.00.20.40.60.81.0
specificity
line of symmetry
![Page 14: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/14.jpg)
Diagnostic odds ratios
FNFP
TNTPORDiagnostic
veLR
veLR
yspecificityspecificit
ysensitivitysensitivit
DOR
1
1
Ratio of the odds of positivity in the diseased to the odds of positivity in the non-diseased
![Page 15: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/15.jpg)
Diagnostic odds ratiosSensitivity
Specificity 50% 60% 70% 80% 90% 95% 99%
50% 1 2 2 4 9 19 99
60% 2 2 4 6 14 29 149
70% 2 4 5 9 21 44 231
80% 4 6 9 16 36 76 396
90% 9 14 21 36 81 171 891
95% 19 29 44 76 171 361 1881
99% 99 149 231 396 891 1881 9801
![Page 16: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/16.jpg)
Symmetrical ROC curves and diagnostic odds ratios
As DOR increases, the ROC curve moves closer to its ideal position near the upper-left corner.
(1)
line of symmetry
uninformative test
(2)
(5)
(16)
(81)(361)
0.0
0.2
0.4
0.6
0.8
1.0
sens
itivi
ty
0.00.20.4 0.6 0.81.0
specificity
![Page 17: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/17.jpg)
Asymmetrical ROC curve and diagnostic odds ratios
diseasednon-diseased
0 40 80 120
test measurement
0.0
0.2
0.4
0.6
0.8
1.0
sen
sitiv
ity
0.00.20.40.60.81.0
specificity
ROC curve is asymmetric when test accuracy varies with threshold
LOW DOR
HIGH DOR
![Page 18: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/18.jpg)
Challenges There are two summary statistics for each study –
sensitivity and specificity – each have different implications
Heterogeneity is the norm – substantial variation in sensitivity and specificity are noted in most reviews
Threshold effects induce correlations between sensitivity and specificity and often seem to be present Thresholds can vary between studies The same threshold can imply different sensitivities
and specificities in different groups
![Page 19: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/19.jpg)
Approach for meta-analysis
Current statistical methods use a single estimate of sensitivity and specificity for each study
Estimate the underlying ROC curve based on studies analysing different thresholds
Analyses at specified threshold Estimate summary sensitivity and
summary specificity
Compare ROC curves between tests Allows comparison unrestricted to
a particular threshold
0.0
0.2
0.4
0
.60
.81
.0
sens
itivi
ty
0.00.20.40.60.81.0
specificity
![Page 20: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/20.jpg)
ROC curve transformation to linear plot Calculate the logits of TPR and FPR Plot their difference against their sum
Moses-Littenberg statistical modelling of ROC curves
![Page 21: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/21.jpg)
Moses-Littenberg SROC method Regression models used to fit straight lines to model
relationship between test accuracy and test threshold
D = a + bS Outcome variable D is the difference in the logits Explanatory variable S is the sum of the logits Ordinary or weighted regression – weighted by sample
size or by inverse variance of the log of the DOR
What do the axes mean? Difference in logits is the log of the DOR Sum of the logits is a marker of diagnostic threshold
![Page 22: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/22.jpg)
Producing summary ROC curves
Transform back to the ROC dimensions
where ‘a’ is the intercept, ‘b’ is the slope when the ROC curve is symmetrical, b=0 and the
equation is simpler
![Page 23: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/23.jpg)
Example: MRI for suspected deep vein thrombosis
Study
Fraser 2003Fraser 2002Sica 2001Jensen 2001Catalano 1997Larcom 1996Laissy 1996Evans 1996Spritzer 1993Evans 1993Carpenter 1993Vukov 1991Pope 1991Erdman 1990
TP
204940
344
1516269
2749
27
FP
14431205233000
FN
03360601000103
TN
34453
188
1916
43265271586
Sensitivity
1.00 [0.83, 1.00]0.94 [0.84, 0.99]0.57 [0.18, 0.90]0.00 [0.00, 0.46]1.00 [0.90, 1.00]0.40 [0.12, 0.74]1.00 [0.78, 1.00]0.94 [0.71, 1.00]1.00 [0.87, 1.00]1.00 [0.66, 1.00]1.00 [0.87, 1.00]0.80 [0.28, 0.99]1.00 [0.66, 1.00]0.90 [0.73, 0.98]
Specificity
0.97 [0.85, 1.00]0.92 [0.80, 0.98]0.43 [0.10, 0.82]0.86 [0.64, 0.97]0.89 [0.52, 1.00]0.99 [0.96, 1.00]1.00 [0.54, 1.00]0.90 [0.77, 0.97]0.93 [0.76, 0.99]0.95 [0.85, 0.99]0.96 [0.89, 0.99]1.00 [0.48, 1.00]1.00 [0.63, 1.00]1.00 [0.54, 1.00]
Sensitivity
0 0.2 0.4 0.6 0.8 1
Specificity
0 0.2 0.4 0.6 0.8 1
Sampson et al. Eur Radiol (2007) 17: 175–181
![Page 24: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/24.jpg)
Transformation linearizes relationship between accuracy and threshold so that linear regression can be used
0.0
0.2
0.4
0
.60
.81
.0
sens
itivi
ty
0.00.20.40.60.81.0
specificity
weighted
unweighted
-10
12
34
56
78
D-5 -4 -3 -2 -1 0 1 2 3
S
Linear transformation
SROC regression: MRI for suspected deep vein thrombosis
![Page 25: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/25.jpg)
The SROC curve is produced by using the estimates of a and b to compute the expected sensitivity (tpr) across a range of values for 1-specificity (fpr)
weighted
unweighted
-10
12
34
56
78
D
-5 -4 -3 -2 -1 0 1 2 3
S
Inverse transformation
SROC regression: MRI for suspected deep vein thrombosis
![Page 26: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/26.jpg)
The SROC curve is produced by using the estimates of a and b to compute the expected sensitivity (tpr) across a range of values for 1-specificity (fpr)
weighted
unweighted
-10
12
34
56
78
D
-5 -4 -3 -2 -1 0 1 2 3
S0
.00
.20
.4
0.6
0.8
1.0
sens
itivi
ty0.00.20.40.60.81.0
specificity
Inverse transformation
697.01
697.01
697.01721.4 11
1
1
697.0,721.4
FPRFPR
e
TPR
ba
SROC regression: MRI for suspected deep vein thrombosis
![Page 27: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/27.jpg)
The SROC curve is produced by using the estimates of a and b to compute the expected sensitivity (tpr) across a range of values for 1-specificity (fpr)
weighted
unweighted
-10
12
34
56
78
D
-5 -4 -3 -2 -1 0 1 2 3
S0
.00
.20
.4
0.6
0.8
1.0
sens
itivi
ty0.00.20.40.60.81.0
specificity
Inverse transformation
SROC regression: MRI for suspected deep vein thrombosis
![Page 28: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/28.jpg)
Poor estimation Tends to underestimate test accuracy due to zero-cell
corrections and bias in weights
Problems with the Moses-Littenberg SROC method
![Page 29: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/29.jpg)
Problems with the Moses-Littenberg SROC method: effect of zero-cell correction
0.0
0.2
0.4
0
.60
.81
.0
sen
sitiv
ity
0.00.20.40.60.81.0
specificity
![Page 30: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/30.jpg)
0.0
0.2
0.4
0
.60
.81
.0
sen
sitiv
ity
0.00.20.40.60.81.0
specificity
Problems with the Moses-Littenberg SROC method: effect of zero-cell correction
![Page 31: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/31.jpg)
Problems with the Moses-Littenberg SROC method
Poor estimation Tends to underestimate test accuracy due to zero-cell
corrections and bias in weights Validity of significance tests
Sampling variability in individual studies not properly taken into account
P-values and confidence intervals erroneous Operating points
knowing average sensitivity/specificity is important but cannot be obtained
Sensitivity for a given specificity can be estimated
![Page 32: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/32.jpg)
Mixed models Hierarchical / multi-level
allows for both within (sampling error) and between study variability (through inclusion of
random effects) Logistic
correctly models sampling uncertainty in the true positive proportion and the false positive proportion
no zero cell adjustments needed Regression models
used to investigate sources of heterogeneity
![Page 33: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/33.jpg)
33
Investigating heterogeneity
![Page 34: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/34.jpg)
CT for acute appendicitis
0.0
0.2
0.4
0
.60
.81
.0
sens
itivi
ty
0.00.20.40.60.81.0
specificityTerasawa et al 2004
(12 studies)
![Page 35: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/35.jpg)
Sources of Variation
Why do results differ between studies?
![Page 36: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/36.jpg)
Sources of Variation
I. Chance variationII. Differences in (implicit) thresholdIII. BiasIV. Clinical subgroupsV. Unexplained variation
![Page 37: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/37.jpg)
Sources of variation: ChanceChance variability:
total sample size=100
Sen
sitiv
ity
0.0
0.2
0.4
0.6
0.8
1.0
Specificity1.0 0.8 0.6 0.4 0.2 0.0
Chance variability:total sample size=40
Sen
sitiv
ity
0.0
0.2
0.4
0.6
0.8
1.0
Specificity1.0 0.8 0.6 0.4 0.2 0.0
![Page 38: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/38.jpg)
May be investigated by:– sensitivity analyses – subgroup analyses or – including covariates in the modelling
Investigating heterogeneity in test accuracy
![Page 39: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/39.jpg)
Example: Anti-CCP for rheumatoid arthritis by CCP generation (37 studies)
(Nishimura et al. 2007)
![Page 40: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/40.jpg)
Anti-CCP for rheumatoid arthritis by CCP generation: SROC plot
0.0
0.2
0.4
0
.60
.81
.0
sens
itivi
ty
0.00.20.40.60.81.0
specificity
Generation 1 Generation 2
![Page 41: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/41.jpg)
00.10.20.30.40.50.60.70.80.910
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Example: Triple test for Down syndrome (24 studies, 89,047 women)
Sen
sitiv
ity
Specificity
![Page 42: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/42.jpg)
00.10.20.30.40.50.60.70.80.910
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Studies of the triple test ( = all ages; =aged 35 and over)
Sen
sitiv
ity
Specificity
![Page 43: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/43.jpg)
Verification bias
Down's Normal
Test +ve (high risk) 50 250
Test -ve (low risk) 50 4750
100 5000
Down's Normal
50 250
50 4750
100 5000
AMNIO
AMNIOSensitivity = 50%
Specificity = 95%
Follow-up = 100%
Down's Normal
Test +ve (high risk) 50 250
Test -ve (low risk) 50 4750
100 5000
AMNIO
BIRTH
Down's Normal
50 250
34 4513
84 4763
Sensitivity = 60%
Specificity = 95%
Follow-up = 95%
16 lost (33%)
237 lost (5%)
Participants recruited Participants analysed
Participants recruited Participants analysed
![Page 44: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/44.jpg)
00.10.20.30.40.50.60.70.80.910
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1S
ensi
tivity
Specificity
Studies of the triple test ( = all ages; =aged 35 and over)
![Page 45: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/45.jpg)
00.10.20.30.40.50.60.70.80.910
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
= all verified by amniocentesis
Sen
sitiv
ity
Specificity
Studies of the triple test ( = all ages; =aged 35 and over)
![Page 46: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/46.jpg)
Limitations of meta-regression
Validity of covariate information poor reporting on design features
Population characteristics information missing or crudely available
Lack of power small number of contrasting studies
![Page 47: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/47.jpg)
Which test is best?
The same approach used to investigate heterogeneity can be used to compare the accuracy of alternative tests
![Page 48: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/48.jpg)
![Page 49: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/49.jpg)
Comparison between HRP-2 and pLDH based RDT Types: all studies
75 HRP-2 studies and 19 pLDH studies
![Page 50: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/50.jpg)
Comparison between HRP-2 and pLDH based RDT Types: paired data only
10 comparative studies
![Page 51: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/51.jpg)
![Page 52: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/52.jpg)
Issues in test comparisons Some systematic reviews pool all available studies that have assessed
the performance of one or more of the tests. Can lead to bias due to confounding arising from heterogeneity
among studies in terms of design, study quality, setting, etc
Adjusting for potential confounders is often not feasible
Restricting analysis to studies that evaluated both tests in the same patients, or randomized patients to receive each test, removes the need to adjust for confounders.
Covariates can be examined to assess whether the relative performance of the tests varies systematically (effect modification)
For truly paired studies, the cross classification of tests results within disease groups is generally not reported
![Page 53: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/53.jpg)
![Page 54: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/54.jpg)
![Page 55: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/55.jpg)
![Page 56: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/56.jpg)
![Page 57: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/57.jpg)
![Page 58: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/58.jpg)
![Page 59: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/59.jpg)
![Page 60: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/60.jpg)
![Page 61: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/61.jpg)
![Page 62: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/62.jpg)
Summary Different approach due to bivariate correlated data
Moses & Littenberg method is a simple technique useful for exploratory analysis included directly in RevMan should not be used for inference
Mixed models are recommended Bivariate random effects model Hierarchical summary ROC (HSROC) model
![Page 63: Cochrane Diagnostic test accuracy reviews Introduction to meta-analysis Jon Deeks and Yemisi Takwoingi Public Health, Epidemiology and Biostatistics University](https://reader034.vdocuments.mx/reader034/viewer/2022051110/5518c42b550346881f8b5743/html5/thumbnails/63.jpg)
RevMan DTA tutorial included in version 5.1
Handbook chapters and other resources available at:
http://srdta.cochrane.org