1 meta-analysis with missing data: metamiss ian white and julian higgins mrc biostatistics unit,...
Post on 18-Dec-2015
217 views
TRANSCRIPT
1
Meta-analysis with missing data: metamiss
Ian White and Julian HigginsMRC Biostatistics Unit, Cambridge, UK
Stata users’ group, London
10 September 2007
2
Motivation
• Missing outcome data compromise trials
• So they also compromise meta-analyses
• We may want to – correct for bias due to missing data– down-weight trials with more missing data
• NB missing data within trials, not missing trials
3
Plan
Meta-analysis of binary data
• Haloperidol example
• Standard approaches to missing data
• Imputation methods
• IMORs
• Methods that allow for uncertainty
• Demonstration
4
Haloperidol meta-analysis Haloperidol Placebo
% missing r1 f1 m1 n1 r2 f2 m2 n2
Arvanitis 25 25 2 52 18 33 0 51 2%
Beasley 29 18 22 69 20 14 34 68 41%
Bechelli 12 17 1 30 2 28 1 31 3%
Borison 3 9 0 12 0 12 0 12 0%
Chouinard 10 11 0 21 3 19 0 22 0%
Durost 11 8 0 19 1 14 0 15 0%
Garry 7 18 1 26 4 21 1 26 4%
Howard 8 9 0 17 3 10 0 13 0%
Marder 19 45 2 66 14 50 2 66 3%
Nishikawa 82 1 9 0 10 0 10 0 10 0%
Nishikawa 84 11 23 3 37 0 13 0 13 6%
Reschke 20 9 0 29 2 9 0 11 0%
Selman 17 1 11 29 7 4 18 29 50%
Serafetinides 4 10 0 14 0 13 1 14 4%
Simpson 2 14 0 16 0 7 1 8 4%
Spencer 11 1 0 12 1 11 0 12 0%
Vichaiya 9 20 1 30 0 29 1 30 3%
r=successes
f=failures
m=missing
n=total
5
Standard approaches to missing data
• Available cases (complete cases): ignore the missing data– assumes MAR: missingness is independent of
outcome given arm
• Assume missing=failure– implausible, but not too bad for health-related
behaviours
• Neither assumption is likely to be correct
6
Other ideas• Sensitivity analyses, e.g. do both
missing=failure and available cases– but these could agree by chance
• Explore best / worst cases• Use reasons for missingness• Explicit assumptions about informative
missingness (IM)– IM: missingness is dependent on outcome
7
metamiss.ado• Processes data on successes, failures and missing by arm
& feeds results to metan• Available cases analysis (ACA)• Imputed case analyses (ICA):
– impute as failure: ICA-0– impute as success: ICA-1– best-case: ICA-b (missing=success in E, failure in C)– worst-case: ICA-w– impute with same probability as in control arm: ICA-pC– impute with same probability as in experimental arm: ICA-pE– impute with same probability as in own arm: ICA-p (agrees with
ACA)– impute using IMORs: ICA-IMOR (see next slide)
8
More general imputation: IMORs• Measure Informative Missingness using the
Informative Missing Odds Ratio (IMOR):– Odds ratio between outcome and missingness
• Can’t estimate IMOR from the data, but given any value of IMOR, we can analyse the data
• Generalises other ideas: e.g. – ICA-0 uses IMORs 0, 0– ICA-1 uses IMORs , – ICA-b uses IMORs , 0– ICA-p uses IMORs 1, 1– ICA-pC uses IMORs OR, 1 where OR is odds ratio
between arm and outcome in available cases
9
Getting standard errors (weighting) right
• Weight 1: treat imputed data as real• Weight 2: use standard errors from ACA• Weight 3: scale imputed data to same sample size
as available cases• Weight 4: algebraic standard errors
– same as weight 1 for ICA-0, ICA-1, ICA-b, ICA-w– same as weight 2 for ICA-p– uses Taylor expansion for ICA-IMOR– for ICA-pC & ICA-pE, we condition on the IMOR (I
can explain…)
10
11
12
Overall (I-squared = 41.4%, p = 0.038)
Arvantis
Vichaiya
Selman
Chouinard
Nishikawa_84
Howard
Serafetinides
Durost
Simpson
Mander
Reschke
Study ID
Garry
Borison_92
Spencer
Beasley
Bechelli
Nishikawa_82
1.57 (1.28, 1.92)
1.42 (0.89, 2.25)
19.00 (1.16, 311.96)
1.48 (0.94, 2.35)
3.49 (1.11, 10.95)
9.20 (0.58, 145.76)
2.04 (0.67, 6.21)
8.40 (0.50, 142.27)
8.68 (1.26, 59.95)
2.35 (0.13, 43.53)
1.36 (0.75, 2.47)
3.79 (1.06, 13.60)
ES (95% CI)
1.75 (0.58, 5.24)
7.00 (0.40, 122.44)
11.00 (1.67, 72.40)
1.05 (0.73, 1.50)
6.21 (1.52, 25.35)
3.00 (0.14, 65.90)
100.00
18.86
0.52
19.11
3.10
0.53
3.27
0.51
1.09
0.48
11.37
2.48
Weight
3.37
%
0.49
1.14
31.22
2.05
0.42
1.57 (1.28, 1.92)
1.42 (0.89, 2.25)
19.00 (1.16, 311.96)
1.48 (0.94, 2.35)
3.49 (1.11, 10.95)
9.20 (0.58, 145.76)
2.04 (0.67, 6.21)
8.40 (0.50, 142.27)
8.68 (1.26, 59.95)
2.35 (0.13, 43.53)
1.36 (0.75, 2.47)
3.79 (1.06, 13.60)
ES (95% CI)
1.75 (0.58, 5.24)
7.00 (0.40, 122.44)
11.00 (1.67, 72.40)
1.05 (0.73, 1.50)
6.21 (1.52, 25.35)
3.00 (0.14, 65.90)
100.00
18.86
0.52
19.11
3.10
0.53
3.27
0.51
1.09
0.48
11.37
2.48
Weight
3.37
%
0.49
1.14
31.22
2.05
0.42
1.1 1 10 100
ACA
13
Overall (I-squared = 25.8%, p = 0.158)
Study ID
Borison_92
Chouinard
Durost
Vichaiya
Selman
Beasley
Nishikawa_82
Howard
Simpson
Nishikawa_84
Mander
Spencer
Garry
Arvantis
Reschke
Bechelli
Serafetinides
1.90 (1.51, 2.39)
ES (95% CI)
7.00 (0.40, 122.44)
3.49 (1.11, 10.95)
8.68 (1.26, 59.95)
19.00 (1.16, 312.42)
2.43 (1.19, 4.96)
1.43 (0.90, 2.27)
3.00 (0.14, 65.90)
2.04 (0.67, 6.21)
2.65 (0.14, 49.42)
8.47 (0.53, 134.46)
1.36 (0.74, 2.47)
11.00 (1.67, 72.40)
1.75 (0.58, 5.27)
1.36 (0.85, 2.17)
3.79 (1.06, 13.60)
6.20 (1.51, 25.40)
9.00 (0.53, 152.93)
100.00
Weight
0.65
4.06
1.42
0.68
10.42
%
25.01
0.56
4.29
0.62
0.70
14.75
1.50
4.38
24.38
3.26
2.67
0.66
1.90 (1.51, 2.39)
ES (95% CI)
7.00 (0.40, 122.44)
3.49 (1.11, 10.95)
8.68 (1.26, 59.95)
19.00 (1.16, 312.42)
2.43 (1.19, 4.96)
1.43 (0.90, 2.27)
3.00 (0.14, 65.90)
2.04 (0.67, 6.21)
2.65 (0.14, 49.42)
8.47 (0.53, 134.46)
1.36 (0.74, 2.47)
11.00 (1.67, 72.40)
1.75 (0.58, 5.27)
1.36 (0.85, 2.17)
3.79 (1.06, 13.60)
6.20 (1.51, 25.40)
9.00 (0.53, 152.93)
100.00
Weight
0.65
4.06
1.42
0.68
10.42
%
25.01
0.56
4.29
0.62
0.70
14.75
1.50
4.38
24.38
3.26
2.67
0.66
1.1 1 10 100
ICA-0
14
Overall (I-squared = 60.3%, p = 0.001)
Spencer
Study ID
Beasley
Serafetinides
Borison_92
Garry
Selman
Mander
Nishikawa_82
Durost
Howard
Chouinard
Nishikawa_84
Bechelli
Arvantis
Reschke
Vichaiya
Simpson
1.16 (1.04, 1.29)
11.00 (1.67, 72.40)
ES (95% CI)
0.93 (0.77, 1.12)
4.00 (0.51, 31.46)
7.00 (0.40, 122.44)
1.60 (0.60, 4.25)
1.12 (0.95, 1.32)
1.31 (0.75, 2.28)
3.00 (0.14, 65.90)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
3.49 (1.11, 10.95)
10.68 (0.68, 167.43)
4.48 (1.42, 14.15)
1.47 (0.93, 2.32)
3.79 (1.06, 13.60)
10.00 (1.36, 73.33)
1.00 (0.11, 9.44)
100.00
0.35
Weight
35.81
0.29
0.15
1.29
47.38
4.01
0.13
0.33
0.99
0.94
0.16
0.93
5.95
0.75
0.31
0.24
%
1.16 (1.04, 1.29)
11.00 (1.67, 72.40)
ES (95% CI)
0.93 (0.77, 1.12)
4.00 (0.51, 31.46)
7.00 (0.40, 122.44)
1.60 (0.60, 4.25)
1.12 (0.95, 1.32)
1.31 (0.75, 2.28)
3.00 (0.14, 65.90)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
3.49 (1.11, 10.95)
10.68 (0.68, 167.43)
4.48 (1.42, 14.15)
1.47 (0.93, 2.32)
3.79 (1.06, 13.60)
10.00 (1.36, 73.33)
1.00 (0.11, 9.44)
100.00
0.35
Weight
35.81
0.29
0.15
1.29
47.38
4.01
0.13
0.33
0.99
0.94
0.16
0.93
5.95
0.75
0.31
0.24
%
1.1 1 10 100
ICA-1
15
Overall (I-squared = 25.9%, p = 0.156)
Study ID
Bechelli
Chouinard
Borison_92
Simpson
Nishikawa_84
Mander
Beasley
Nishikawa_82
Serafetinides
Reschke
Durost
Howard
Selman
Spencer
Arvantis
Garry
Vichaiya
2.42 (1.95, 3.00)
ES (95% CI)
6.72 (1.65, 27.28)
3.49 (1.11, 10.95)
7.00 (0.40, 122.44)
2.65 (0.14, 49.42)
10.68 (0.68, 167.43)
1.50 (0.84, 2.69)
2.51 (1.69, 3.73)
3.00 (0.14, 65.90)
9.00 (0.53, 152.93)
3.79 (1.06, 13.60)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
4.00 (2.09, 7.65)
11.00 (1.67, 72.40)
1.47 (0.93, 2.32)
2.00 (0.69, 5.83)
21.00 (1.29, 342.93)
100.00
Weight
2.37
3.57
0.57
0.54
0.62
13.68
30.05
%
0.49
0.58
2.86
1.25
3.76
11.08
1.31
22.59
4.07
0.60
2.42 (1.95, 3.00)
ES (95% CI)
6.72 (1.65, 27.28)
3.49 (1.11, 10.95)
7.00 (0.40, 122.44)
2.65 (0.14, 49.42)
10.68 (0.68, 167.43)
1.50 (0.84, 2.69)
2.51 (1.69, 3.73)
3.00 (0.14, 65.90)
9.00 (0.53, 152.93)
3.79 (1.06, 13.60)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
4.00 (2.09, 7.65)
11.00 (1.67, 72.40)
1.47 (0.93, 2.32)
2.00 (0.69, 5.83)
21.00 (1.29, 342.93)
100.00
Weight
2.37
3.57
0.57
0.54
0.62
13.68
30.05
%
0.49
0.58
2.86
1.25
3.76
11.08
1.31
22.59
4.07
0.60
1.1 1 10 100
ICA-B
16
Overall (I-squared = 74.2%, p = 0.000)
Serafetinides
Arvantis
Durost
Beasley
Study ID
Simpson
Reschke
Selman
Borison_92
Mander
Nishikawa_82
Vichaiya
Chouinard
Bechelli
Nishikawa_84
Spencer
Garry
Howard
0.94 (0.79, 1.12)
4.00 (0.51, 31.46)
1.36 (0.85, 2.17)
8.68 (1.26, 59.95)
0.53 (0.39, 0.72)
ES (95% CI)
1.00 (0.11, 9.44)
3.79 (1.06, 13.60)
0.68 (0.48, 0.95)
7.00 (0.40, 122.44)
1.19 (0.67, 2.10)
3.00 (0.14, 65.90)
9.00 (1.21, 66.70)
3.49 (1.11, 10.95)
4.13 (1.29, 13.20)
8.47 (0.53, 134.46)
11.00 (1.67, 72.40)
1.40 (0.51, 3.85)
2.04 (0.67, 6.21)
100.00
0.72
13.99
0.82
33.33
Weight
0.60
%
1.87
26.57
0.37
9.35
0.32
0.76
2.33
2.26
0.40
0.86
2.98
2.46
0.94 (0.79, 1.12)
4.00 (0.51, 31.46)
1.36 (0.85, 2.17)
8.68 (1.26, 59.95)
0.53 (0.39, 0.72)
ES (95% CI)
1.00 (0.11, 9.44)
3.79 (1.06, 13.60)
0.68 (0.48, 0.95)
7.00 (0.40, 122.44)
1.19 (0.67, 2.10)
3.00 (0.14, 65.90)
9.00 (1.21, 66.70)
3.49 (1.11, 10.95)
4.13 (1.29, 13.20)
8.47 (0.53, 134.46)
11.00 (1.67, 72.40)
1.40 (0.51, 3.85)
2.04 (0.67, 6.21)
100.00
0.72
13.99
0.82
33.33
Weight
0.60
%
1.87
26.57
0.37
9.35
0.32
0.76
2.33
2.26
0.40
0.86
2.98
2.46
1.1 1 10 100
ICA-W
17
Overall (I-squared = 42.0%, p = 0.035)
Spencer
Nishikawa_82
Study ID
Simpson
Serafetinides
Chouinard
Bechelli
Borison_92
Mander
Arvantis
Vichaiya
Selman
Reschke
Nishikawa_84
Garry
Beasley
Durost
Howard
1.53 (1.24, 1.88)
11.00 (1.67, 72.40)
3.00 (0.14, 65.90)
ES (95% CI)
2.35 (0.13, 43.53)
8.40 (0.50, 142.27)
3.49 (1.11, 10.95)
6.03 (1.47, 24.68)
7.00 (0.40, 122.44)
1.35 (0.74, 2.45)
1.40 (0.88, 2.23)
18.42 (1.12, 302.65)
1.30 (0.76, 2.23)
3.79 (1.06, 13.60)
8.55 (0.54, 135.71)
1.72 (0.57, 5.16)
1.03 (0.72, 1.49)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
100.00
1.21
0.45
Weight
0.51
0.54
3.29
2.17
0.53
%
12.05
19.89
0.55
14.85
2.64
0.56
3.57
32.56
1.15
3.47
1.53 (1.24, 1.88)
11.00 (1.67, 72.40)
3.00 (0.14, 65.90)
ES (95% CI)
2.35 (0.13, 43.53)
8.40 (0.50, 142.27)
3.49 (1.11, 10.95)
6.03 (1.47, 24.68)
7.00 (0.40, 122.44)
1.35 (0.74, 2.45)
1.40 (0.88, 2.23)
18.42 (1.12, 302.65)
1.30 (0.76, 2.23)
3.79 (1.06, 13.60)
8.55 (0.54, 135.71)
1.72 (0.57, 5.16)
1.03 (0.72, 1.49)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
100.00
1.21
0.45
Weight
0.51
0.54
3.29
2.17
0.53
%
12.05
19.89
0.55
14.85
2.64
0.56
3.57
32.56
1.15
3.47
1.1 1 10 100
ICA-pc
18
Overall (I-squared = 41.4%, p = 0.038)
Garry
Vichaiya
Howard
Borison_92
Selman
Durost
Nishikawa_82
Reschke
Mander
Chouinard
Simpson
Spencer
Nishikawa_84
Bechelli
Serafetinides
Study ID
Beasley
Arvantis
1.57 (1.28, 1.92)
1.75 (0.58, 5.24)
19.00 (1.16, 311.96)
2.04 (0.67, 6.21)
7.00 (0.40, 122.44)
1.48 (0.94, 2.35)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
3.79 (1.06, 13.60)
1.36 (0.75, 2.47)
3.49 (1.11, 10.95)
2.35 (0.13, 43.53)
11.00 (1.67, 72.40)
9.20 (0.58, 145.76)
6.21 (1.52, 25.35)
8.40 (0.50, 142.27)
ES (95% CI)
1.05 (0.73, 1.50)
1.42 (0.89, 2.25)
100.00
3.37
0.52
3.27
0.49
19.11
1.09
0.42
2.48
11.37
3.10
0.48
1.14
0.53
2.05
0.51
Weight
31.22
18.86
%
1.57 (1.28, 1.92)
1.75 (0.58, 5.24)
19.00 (1.16, 311.96)
2.04 (0.67, 6.21)
7.00 (0.40, 122.44)
1.48 (0.94, 2.35)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
3.79 (1.06, 13.60)
1.36 (0.75, 2.47)
3.49 (1.11, 10.95)
2.35 (0.13, 43.53)
11.00 (1.67, 72.40)
9.20 (0.58, 145.76)
6.21 (1.52, 25.35)
8.40 (0.50, 142.27)
ES (95% CI)
1.05 (0.73, 1.50)
1.42 (0.89, 2.25)
100.00
3.37
0.52
3.27
0.49
19.11
1.09
0.42
2.48
11.37
3.10
0.48
1.14
0.53
2.05
0.51
Weight
31.22
18.86
%
1.1 1 10 100
ICA-p
19
Overall (I-squared = 48.2%, p = 0.014)
Vichaiya
Nishikawa_82
Garry
Bechelli
Simpson
Beasley
Howard
Selman
Borison_92
Nishikawa_84
Reschke
Mander
Durost
Chouinard
Spencer
Study ID
Serafetinides
Arvantis
1.42 (1.19, 1.69)
18.73 (1.14, 306.64)
3.00 (0.14, 65.90)
1.74 (0.59, 5.16)
6.12 (1.51, 24.87)
2.14 (0.12, 38.74)
1.00 (0.74, 1.35)
2.04 (0.67, 6.21)
1.32 (0.93, 1.86)
7.00 (0.40, 122.44)
9.57 (0.61, 151.28)
3.79 (1.06, 13.60)
1.35 (0.75, 2.45)
8.68 (1.26, 59.95)
3.49 (1.11, 10.95)
11.00 (1.67, 72.40)
ES (95% CI)
7.91 (0.47, 132.79)
1.43 (0.90, 2.28)
100.00
0.40
0.33
2.64
1.59
0.37
35.19
2.52
26.22
0.38
0.41
%
1.91
8.89
0.84
2.39
0.88
Weight
0.39
14.67
1.42 (1.19, 1.69)
18.73 (1.14, 306.64)
3.00 (0.14, 65.90)
1.74 (0.59, 5.16)
6.12 (1.51, 24.87)
2.14 (0.12, 38.74)
1.00 (0.74, 1.35)
2.04 (0.67, 6.21)
1.32 (0.93, 1.86)
7.00 (0.40, 122.44)
9.57 (0.61, 151.28)
3.79 (1.06, 13.60)
1.35 (0.75, 2.45)
8.68 (1.26, 59.95)
3.49 (1.11, 10.95)
11.00 (1.67, 72.40)
ES (95% CI)
7.91 (0.47, 132.79)
1.43 (0.90, 2.28)
100.00
0.40
0.33
2.64
1.59
0.37
35.19
2.52
26.22
0.38
0.41
%
1.91
8.89
0.84
2.39
0.88
Weight
0.39
14.67
1.1 1 10 100
ICA-IMOR 2 2
20
Overall (I-squared = 35.0%, p = 0.077)
Spencer
Durost
Simpson
Study ID
Beasley
Nishikawa_82
Reschke
Bechelli
Garry
Howard
Serafetinides
Selman
Vichaiya
Mander
Borison_92
Arvantis
Chouinard
Nishikawa_84
1.70 (1.37, 2.11)
11.00 (1.67, 72.40)
8.68 (1.26, 59.95)
2.49 (0.13, 46.31)
ES (95% CI)
1.12 (0.74, 1.70)
3.00 (0.14, 65.90)
3.79 (1.06, 13.60)
6.23 (1.52, 25.49)
1.75 (0.58, 5.26)
2.04 (0.67, 6.21)
8.68 (0.51, 147.44)
1.74 (0.97, 3.12)
19.06 (1.16, 313.20)
1.36 (0.75, 2.47)
7.00 (0.40, 122.44)
1.40 (0.88, 2.23)
3.49 (1.11, 10.95)
8.91 (0.56, 141.28)
100.00
1.35
1.28
0.56
Weight
27.47
0.50
2.94
2.41
3.96
3.87
%
0.60
14.11
0.61
13.34
0.58
22.12
3.66
0.63
1.70 (1.37, 2.11)
11.00 (1.67, 72.40)
8.68 (1.26, 59.95)
2.49 (0.13, 46.31)
ES (95% CI)
1.12 (0.74, 1.70)
3.00 (0.14, 65.90)
3.79 (1.06, 13.60)
6.23 (1.52, 25.49)
1.75 (0.58, 5.26)
2.04 (0.67, 6.21)
8.68 (0.51, 147.44)
1.74 (0.97, 3.12)
19.06 (1.16, 313.20)
1.36 (0.75, 2.47)
7.00 (0.40, 122.44)
1.40 (0.88, 2.23)
3.49 (1.11, 10.95)
8.91 (0.56, 141.28)
100.00
1.35
1.28
0.56
Weight
27.47
0.50
2.94
2.41
3.96
3.87
%
0.60
14.11
0.61
13.34
0.58
22.12
3.66
0.63
1.1 1 10 100
ICA-IMOR 1/2 1/2
21
Allowing for reasons (ICA-R)
• Specify number of missing individuals in each arm to be imputed by each scheme ICA-0, ICA-1, ICA-pC, ICA-pE, ICA-p, ICA-IMOR.
• Can take these data from a different outcome: metamiss scales to #missing
• If missing in a particular study, metamiss imputes using combined studies
22
23
Allowing for uncertainty
• So far we have pretended we really know the IMORs
• This is never really correct
• Now we allow them to be unknown but from a user-specified distribution
24
Bayesian approach allowing for uncertain IMORs (Rubin, 1977)
E C
2E E E E
2E
E
E
Prior for , = log(IMOR) in experimental, control arm:
N ,
, measure your best guess about IM;
, measure your uncertainty about IM;
measu
C
C C C C
C
C
E Cres how similar you think , are:
0 is most conservative, 1 often allows little impact of IM on results.
25
Bayesian analysis
• Elicit prior for E, C or use N(0,12) or N(0,22)• Get posterior distribution by integrating over the 2-
dimensional distribution of E, C. • metamiss does this fast & accurately by:
1. Standard normal approximation to posterior given E, C 2. Integrate using Gauss-Hermite quadrature.
• Alternatives: – Taylor expansion (inaccurate for large SD of log IMOR)– Full Bayesian Monte Carlo (slow, little gain in accuracy)
26
Understanding priors for log IMOR: implied prior for P(success | missing)
when P(success | observed) = 1/2D
ensi
ty
0 .25 .5 .75 1P(success | missing)
N(0,0.5^2) N(0,2^2) N(-1,0.5^2) N(-1,2^2)
27
28
Overall (I-squared = 31.4%, p = 0.106)
Study ID
Borison_92
Simpson
Chouinard
Mander
Bechelli
Selman
Serafetinides
Spencer
Nishikawa_84
Garry
Durost
Nishikawa_82
Beasley
Reschke
Arvantis
Vichaiya
Howard
1.75 (1.38, 2.22)
ES (95% CI)
7.00 (0.40, 122.44)
2.27 (0.12, 42.39)
3.49 (1.11, 10.95)
1.35 (0.74, 2.47)
6.14 (1.50, 25.13)
1.54 (0.82, 2.88)
8.17 (0.48, 139.06)
11.00 (1.67, 72.40)
9.25 (0.58, 146.78)
1.74 (0.58, 5.23)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
1.06 (0.61, 1.85)
3.79 (1.06, 13.60)
1.42 (0.89, 2.25)
18.74 (1.14, 308.29)
2.04 (0.67, 6.21)
100.00
Weight
%
0.68
0.65
4.26
15.46
2.80
14.02
0.69
1.57
0.73
4.60
1.49
0.58
18.07
3.41
25.78
0.71
4.49
1.75 (1.38, 2.22)
ES (95% CI)
7.00 (0.40, 122.44)
2.27 (0.12, 42.39)
3.49 (1.11, 10.95)
1.35 (0.74, 2.47)
6.14 (1.50, 25.13)
1.54 (0.82, 2.88)
8.17 (0.48, 139.06)
11.00 (1.67, 72.40)
9.25 (0.58, 146.78)
1.74 (0.58, 5.23)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
1.06 (0.61, 1.85)
3.79 (1.06, 13.60)
1.42 (0.89, 2.25)
18.74 (1.14, 308.29)
2.04 (0.67, 6.21)
100.00
Weight
%
0.68
0.65
4.26
15.46
2.80
14.02
0.69
1.57
0.73
4.60
1.49
0.58
18.07
3.41
25.78
0.71
4.49
1.1 1 10 100
logimor ~ N(0,1)
29
Overall (I-squared = 23.6%, p = 0.181)
Serafetinides
Vichaiya
Study ID
Nishikawa_84
Selman
Durost
Nishikawa_82
Arvantis
Bechelli
Simpson
Garry
Howard
Borison_92
Reschke
Mander
Chouinard
Beasley
Spencer
1.87 (1.44, 2.41)
7.61 (0.43, 133.33)
17.81 (1.06, 298.38)
ES (95% CI)
9.32 (0.59, 148.16)
1.60 (0.67, 3.80)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
1.42 (0.89, 2.26)
5.98 (1.45, 24.71)
2.10 (0.11, 40.70)
1.73 (0.57, 5.22)
2.04 (0.67, 6.21)
7.00 (0.40, 122.44)
3.79 (1.06, 13.60)
1.35 (0.74, 2.47)
3.49 (1.11, 10.95)
1.08 (0.51, 2.32)
11.00 (1.67, 72.40)
100.00
0.80
0.83
Weight
%
0.86
8.77
1.77
0.69
30.37
3.28
0.75
5.40
5.32
0.81
4.04
18.04
5.05
11.36
1.86
1.87 (1.44, 2.41)
7.61 (0.43, 133.33)
17.81 (1.06, 298.38)
ES (95% CI)
9.32 (0.59, 148.16)
1.60 (0.67, 3.80)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
1.42 (0.89, 2.26)
5.98 (1.45, 24.71)
2.10 (0.11, 40.70)
1.73 (0.57, 5.22)
2.04 (0.67, 6.21)
7.00 (0.40, 122.44)
3.79 (1.06, 13.60)
1.35 (0.74, 2.47)
3.49 (1.11, 10.95)
1.08 (0.51, 2.32)
11.00 (1.67, 72.40)
100.00
0.80
0.83
Weight
%
0.86
8.77
1.77
0.69
30.37
3.28
0.75
5.40
5.32
0.81
4.04
18.04
5.05
11.36
1.86
1.1 1 10 100
logimor ~ N(0,2^2)
30
Proposal: 4 sensitivity analysesIMORs Options
(e.g.)Sensitive to: Works via:
fixed equal
imor(2 2) Imbalance in missingness
Point estimates
fixed opposite
imor(2 1/2) Amount of missing data
random equal
sdlogimor(2) corr(1)
Imbalance in missingness
Weightings
random uncorrelated
sdlogimor(2) corr(0)
Amount of missing data
31
Summary• Tool for sensitivity analysis• Requires thought about plausible missing data
mechanisms• Would be nice to overlay sensitivity analysis
with ACA• Further work includes combining uncertainty
with reasons• I also have a program mvmeta for
multivariate meta-analysis
32
References• 1st part: Higgins JPT, White IR, Wood A. Imputation methods
for missing outcome data in meta-analysis of clinical trials. Clinical Trials, submitted.
• 2nd part: White IR, Higgins JPT, Wood AM. Allowing for uncertainty due to missing data in meta-analysis. 1. Two-stage methods. Statistics in Medicine, in press.
• Related: White IR, Welton NJ, Wood AM, Ades AE, Higgins JPT. Allowing for uncertainty due to missing data in meta-analysis. 2. Hierarchical models. Statistics in Medicine, in press.
• metamiss.ado available from http://www.mrc-bsu.cam.ac.uk/BSUsite/Software/Stata.shtml
33
Extra slides
34
Overall (I-squared = 16.7%, p = 0.257)
Vichaiya
Howard
Nishikawa_82
Spencer
Reschke
Serafetinides
Selman
Garry
Borison_92
Mander
Bechelli
Simpson
Nishikawa_84
Study ID
Durost
Beasley
Chouinard
Arvantis
2.02 (1.51, 2.70)
19.00 (1.13, 318.79)
2.04 (0.67, 6.21)
3.00 (0.14, 65.90)
11.00 (1.67, 72.40)
3.79 (1.06, 13.60)
8.40 (0.50, 140.57)
1.48 (0.37, 5.90)
1.75 (0.52, 5.92)
7.00 (0.40, 122.44)
1.36 (0.68, 2.72)
6.21 (1.35, 28.50)
2.35 (0.12, 44.54)
9.20 (0.52, 162.90)
ES (95% CI)
8.68 (1.26, 59.95)
1.05 (0.34, 3.24)
3.49 (1.11, 10.95)
1.42 (0.86, 2.33)
100.00
1.05
6.75
0.88
2.36
5.13
1.05
4.39
5.63
1.02
17.36
3.60
0.97
1.01
Weight
2.24
6.58
6.40
33.58
%
2.02 (1.51, 2.70)
19.00 (1.13, 318.79)
2.04 (0.67, 6.21)
3.00 (0.14, 65.90)
11.00 (1.67, 72.40)
3.79 (1.06, 13.60)
8.40 (0.50, 140.57)
1.48 (0.37, 5.90)
1.75 (0.52, 5.92)
7.00 (0.40, 122.44)
1.36 (0.68, 2.72)
6.21 (1.35, 28.50)
2.35 (0.12, 44.54)
9.20 (0.52, 162.90)
ES (95% CI)
8.68 (1.26, 59.95)
1.05 (0.34, 3.24)
3.49 (1.11, 10.95)
1.42 (0.86, 2.33)
100.00
1.05
6.75
0.88
2.36
5.13
1.05
4.39
5.63
1.02
17.36
3.60
0.97
1.01
Weight
2.24
6.58
6.40
33.58
%
1.1 1 10 100
Gamble-Hollis