marie kassapian 1,2, toufik zahaf 3, fabian tibaldi 3 1 university of hasselt 2 frontier science...
TRANSCRIPT
![Page 1: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/1.jpg)
Marie Kassapian1,2, Toufik Zahaf3, Fabian Tibaldi3
1 University of Hasselt2 Frontier Science Foundation Hellas
3 GlaxoSmithKline (GSK) Vaccines
Tel Aviv, 22.04.2013
![Page 2: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/2.jpg)
The disease Herpes Zoster After a varicella (chicken-pox) incident, the virus
may be expressed again after several years. Basically in ages above 60 years old. Can turn out very severe in terms of pain.
2Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 3: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/3.jpg)
Zoster Brief Pain Inventory (ZBPI) Questionnaire: A set of questions to determine the level of pain
interfering with functional status & quality of life Scale from 0 to 10 Filled in every day during follow-up period (182
days) Score=0 Non-incident case & Score>0 Incident case Final score: Sum of worst daily scores (182-1820)
3Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 4: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/4.jpg)
The resulted data after the end of the follow-up period contain many zeros.
These zeros belong to the scores of those individuals that did not experience zoster.
Need for methods capable of handling such datasets.
Important to account both for the reduction in the total number of cases as well as for the reduction in the severity of pain.
4Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 5: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/5.jpg)
Burden-of-Illness (BOI) Measure - Chang et al. (1994)
Test accounting for: Disease incidence Disease severity
Assign a score to each patient and create the Burden-of-Illness score by adding them.
5Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 6: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/6.jpg)
6
Statistic:
where :nj represents the total number of pts. in each group.mi represents the number of infected pts. in each group.Wji represents the BOI score of the ith patient in the jth group.
For the groups: 0:placebo group & 1:vaccine group
Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 7: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/7.jpg)
Choplump test - Follmann et al. (2009) Sort the scores in each group. Toss out the same number of zeros in both groups. 1 group with no zeros + 1 group with few zeros.
Statistic:
7Comparison of Statistical Tests in Presence of Many Zeros Data
n=number of pts randomized in each group m=max(m0,m1) S2
m=pooled variance based on the m largest W’s in each group
Calculation of the p-value can be: Exact or Approximate
![Page 8: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/8.jpg)
Comparison between the test suggested by Chang et al. (1994) and the one suggested by Follmann et al. (2009).
8Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 9: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/9.jpg)
Comparison of Statistical Tests in Presence of Many Zeros Data 9
![Page 10: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/10.jpg)
10
No real data
Simulated dataset based on assumptions for the sample size, the incidence rate and the risk reduction.
Number of cases: Placebo: Incidence rate * N0* years of follow-up Vaccine: Incidence rate * N1 * Risk * years of follow-up
Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 11: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/11.jpg)
11
N MeanStd.
Dev.Median Min. Max.
All cases (W* ≥ 0)
Placebo (Z=0) 8,000 28.69 195.92 0 0 1431Vaccine (Z=1) 8,000 4.01 50.58 0 0 690
Zoster cases only (W* > 0)
Placebo (Z=0) 168 1366.20 21.60 1366 1320 1431Vaccine (Z=1) 50 641.54 21.02 641 597 690
*W: the Burden-of Illness score of a patient
Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 12: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/12.jpg)
Normality tests to observe the distribution of the patients’ BOI scores.
All cases:
12
p-value<0.01 (both groups)
Z=0 Z=1
Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 13: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/13.jpg)
13
Zoster cases only:
p-value=0.128 (placebo)
p-value=0.15 (vaccine)
Z=0
Z=1
Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 14: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/14.jpg)
14
Area Under the Curve for the two groups based on the mean daily severity (BOI) scores.
Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 15: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/15.jpg)
Implementation of Chang et al. method:
Findings: P-value from Chang et al. method much more
significant than those yielded for the separate tests.
Both methods (Choplump & Chang) reject H0.
15
Test Statistic p-value
Incidence Rate 63.87 <0.001
Severity score per case 209.49 <0.001
Burden-of-illness score 11.22 <0.0001
Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 16: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/16.jpg)
1st case: Exact p-value
H0: No difference in B.O.I. scores between placebo and vaccine group
p-value=0.047
16
Patient ID 1 2 3 4 5 6 7 8 9 10
W=score 0 1326 1369 1387 1374 0 0 0 0 650
Z=group 0 0 0 0 0 1 1 1 1 1
Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 17: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/17.jpg)
Conclusion: The treated groups differ in 2 ways: Difference in the number of incidents per group Difference in the mean severity scores per
group
17Comparison of Statistical Tests in Presence of Many Zeros Data
Note: •N=10 patients and M=5 incident cases: 252 permutations•N=20 patients and M=10 incident cases: 182,756 permutations
![Page 18: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/18.jpg)
2nd case: Approximate p-value
Simulated dataset (RR=70% , Incidence rate=0.7%) :N=16,000 pts. N0=N1=8,000 pts.
M=218 cases M0=168 cases M1=50 cases
K=15,782 zeros K0=7,732 zeros K1=7,950 zeros
H0: No difference in B.O.I. scores between placebo
and vaccine group
p-value=2.72*10-31
18Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 19: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/19.jpg)
Conclusion: Again, the groups differ in 2 ways: Difference in the number of incidents per group Difference in the mean severity scores per group
19Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 20: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/20.jpg)
Chang method cannot compute very small p-values. Comparison between the tests not
straightforward. Implementation of power analysis in order to find
the most powerful test.
Building of different scenarios based on: Sample size (1,000 , 2,000 , 5,000 , 10,000 , 20,000) Risk reduction (30% , 50% , 70%) Severity reduction (Yes , No)
Simulation of 1,000 datasets for each scenario.20Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 21: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/21.jpg)
Hypothesis Sample sizeRisk
ReductionSeverity
ReductionH0
N
0% No
HA(1)
30%Yes
HA(2)
NoHA
(3)
50%Yes
HA(4)
NoHA
(5)
70%Yes
HA(6)
No
Comparison of Statistical Tests in Presence of Many Zeros Data 21
RR=0% RR=30% RR=50% RR=70%
Placebo 1-10 4-10 4-10 4-10
Vaccine 1-10 3-9 2-8 1-7
Ranges for severity scores:
![Page 22: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/22.jpg)
22
Boxplots of scores under the different hypotheses (N=10,000)
Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 23: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/23.jpg)
Comments based on the summary statistics of the resulted p-values:
The alternative hypotheses that also account for severity reduction, apart from risk reduction, present incredibly small distances between the minimum and the maximum values.
More obvious in the case of the Choplump test. As N increases, the mean p-values decrease much
faster especially for the Choplump test.
23Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 24: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/24.jpg)
Estimated type I error probabilities for each test:
Estimated power:
N 1,000 2,000 5,000 10,000 20,000
Chang 0.01 0.011 0.013 0.02 0.026
Choplump 0.02 0.027 0.025 0.025 0.032
N
Chang Choplump
30% 50% 70% 30% 50% 70%
Yes No Yes No Yes No Yes No Yes No Yes No
1,000 0.001 0.001 0.003 0.001 0.21 0.17 0.09 0.003 0.24 0.18 0.35 0.44
3,000 0.005 0.002 0.25 0.16 0.39 0.31 0.21 0.035 0.36 0.24 0.74 0.61
5,000 0.43 0.01 0.58 0.13 0.68 0.56 0.51 0.12 0.65 0.39 0.81 0.77
10,000 0.77 0.66 0.86 0.71 0.91 0.80 0.78 0.54 0.88 0.57 0.93 0.85
20,000 0.93 0.89 0.95 0.91 0.99 0.94 0.95 0.92 0.97 0.94 0.98 0.97
![Page 25: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/25.jpg)
Both tests represent adequate approaches to the issue of handling a lot of zeros.
The Choplump test is dominant over its competitor only in cases when the efficacy of the vaccine is reflected by both risk and severity reduction.
25Comparison of Statistical Tests in Presence of Many Zeros Data
![Page 26: Marie Kassapian 1,2, Toufik Zahaf 3, Fabian Tibaldi 3 1 University of Hasselt 2 Frontier Science Foundation Hellas 3 GlaxoSmithKline (GSK) Vaccines Tel](https://reader036.vdocuments.mx/reader036/viewer/2022062417/551c1e21550346a84f8b5a03/html5/thumbnails/26.jpg)
Thank you
26Comparison of Statistical Tests in Presence of Many Zeros Data