safety, can you paradigm? a statistical lament janet turk wittes statistics collaborative
TRANSCRIPT
Safety, Can You Paradigm?
A Statistical Lament
Janet Turk WittesStatistics Collaborative
2
Harms identified late
•fenflurmine-phentermine (Fen Phen)
•Rofecoxib (Vioxx)
•Troglitazone (Rezulin)
•HRT (Premarin and PremPro)
•Celecoxib (Celebrex)
•Telithromycin (Ketek)
•Rosiglitazone (Avandia)
•Antidepressants, anti-epileptics….
3
Could we have identified these harms earlier?
•Troglitazone (Rezulin) -removed from market in 2000
Lots of liver abnormalities
Severe toxicities noted in 1997
Other equally effective drugs didn’t have same problems
4
Could we have identified these harms earlier?
•Troglitazone (Rezulin) -removed from market in 2000
• Rofecoxib (Vioxx) -removed from market in 2004
Every study showed excess heart attack
Attributed to benefit of naproxen
5
Could we have identified these harms earlier?
•Troglitazone (Rezulin) – removed from market in 2000
•Rofecoxib (Vioxx) -removed from market in 2004
•HRT (Premarin/PremPro)-major label change 2006
Heart attacks in Puerto Rican girls on oral contraception -1960’s
Men on estrogens had higher event rates – 1970’s
6
Could we have identified these harms earlier?
•Troglitazone (Rezulin) – removed from market in 2000
•Rofecoxib (Vioxx) -removed from market in 2004
•HRT (Premarin and PremPro)-label change 2006
•Celecoxib (Celebrex) – paper published 2005
•Telithromycin (Ketek) – major label change 2007
7
“CELEBRATE :: CELEBREX”
December 2004
8
How we statisticians help to save drugs
•We find safety boring
9
For efficacy we think hard about…
Outcomes
Population to study
Protocol
Analysis of primary outcome
Control of Type I error rate
Other outcomes
Missing data
Sensitivity analyses
10
How we statisticians save drugs
•Because we find safety boring….
We don’t look at preclinical and early Phase data
We don’t ask about
•Chemistry
•Biology
•What PK/PD studies show
•Safety part of analysis plan is an afterthought
11
How the statistical -police protect drugs
•We test hypotheses
•Put events in correct body system
•Give precise definitions
•No data dredging
•Too many type 1 errors if we dredge
12
And we divide and…
•conquer
•obfuscate
13
e.g. Neuropathy
Event T C
Neuropathic pain 1 0
Neuropathy 1 0
Neuropathy NOS 5 2
Neuropathy peripheral 2 0
… 2 1
…
…
14
e.g. Neuropathy
Event T C
…
Parathesia 3 2
Parathesia NOS 4 0
Parathesia other 0 1
…
Peripheral motor neuropathy 6 0
Peripheral sensory neuropathy 3 2
15
True(ish) data from a coxib
C TCardiac disorders 42 46Respiratory 33 29Vascular disorders 7 9
16
True(ish) data from a coxib
C TCardiac disorders 42 46•Angina 2 2•Angina aggravated 0 2•Angina unstable 0 3•…•Cardiac arrest 0 1•Cardiac failure congest 2 0•Coronary artery disease 4 7•…•Myocardial infarction 5 10
17
True(ish) data from a coxib
Respiratory 33 29•Dyspnea 1 3
Vascular disorders 7 9•Cerebral infarction 0 1
•Pulmonary embolism 0 2
•TIA 2 0
18
If you combined…
No. of people with at least one serious thromboembolic event or evidence of heart failure
Placebo Coxib
16 27
19
Other ways to save drugs
Modified Daley’s Rule:
Censor early and often
20
e.g., Rofecoxib- short follow-up
21
Through 36 months
22
With denominators (Bresalier et al. NEJM 2005 352:1092) (And see Adam Boyd’s poster!)
23
Known or suspected adverse events
•Monitor them
•Look at events, their (near) synonyms, labs
Are they real?
Are they too frequent?
24
Hierarchical multiplicity
•Think of biology
•Order hierarchy by decreasing
Biological plausibility
Objectivity
•Look for monotone decreasing hazard ratio
25
Which dose of celecoxib do you want?
26
APC Study (Placebo vs high dose)
Outcome n------------------------------------------------CV death 6 +MI 19+Stroke 26+CHF 29+Angina 34+CV procedure 46-----------------------------------------------Other CV 62
27
Adenoma Prevention with Celecoxib (APC) Study
HRCV death 5.1 +MI 3.8 +Stroke 3.4 +CHF 3.2 +Angina 2.1 +CV procedure 1.7 -------------------------------------------------Other CV 1.1
28
APC Study
CV death 5.1 ( 0.6, 43.2) +MI 3.8 ( 1.3, 11.4) +Stroke 3.4 ( 1.4, 8.3)
+CHF 3.2 ( 1.4, 7.4) +Angina 2.1 ( 1.0, 4.3) +CV procedure 1.7 ( 1.0, 3.1) ------------------------------------------Other CV 1.1 ( 0.7, 1.8)
29
APC Study
CV death 5.1 ( 0.6, 43.2) 0.14 +MI 3.8 ( 1.3, 11.4) 0.015 +Stroke 3.4 ( 1.4, 8.3)
0.007+CHF 3.2 ( 1.4, 7.4) 0.006+Angina 2.1 ( 1.0, 4.3) 0.05+CV procedure 1.7 ( 1.0, 3.1) 0.05-------------------------------------------------Other CV 1.1 ( 0.7, 1.8) 0.7
Solomon (2006). Circulation 114:1028
30
Unknown harms: usual approach
•Respond by
Agonizing
Checking informed consent document
Asking for more frequent looks
Asking for more thorough analyses
•Worry about falsely discovered harm
31
Sentinel events
•Identify
•Follow in the next patients
•Invent formal statistical methods
32
Single sentinel event
•Childhood vaccine
•30 day follow-up for serious adverse events
•1 death occurred
•DSMB: did the vaccine cause the death?
33
Women’s Health Initiative
•Early in the trial, DSMB noted:
Increase in stroke
Increase in pulmonary embolism
Increase in myocardial infarction
•Possible sentinel events
Myocardial infarction
The big meanies: stroke, PE, MI
34
Proposal
1. Identify sentinel event (or cluster or rate)
2. Monitor for subsequent occurrence(s)
Have reasonable power
Be statistically unbiased (exclude sentinel)
Type 1 error rate may be large (~0.2)
Lachenbruch, Wittes: 2007
35
Safety report sample –abnormal lab values
•Time A B Total •Point [N= 150] [N= 148] [N= 298] •_______________________________________________________________ SCREENING 0 0 0 •RANDOM 0 0 0 •WEEK 2 0 0 0 •WEEK 3 0 0 0 •WEEK 4 0 0 0 •WEEK 5 0 0 0 •WEEK 6 0 0 0 •WEEK 7 0 0 0 •WEEK 8 0 0 0 •___________________________________________________________________
36
But wait! You also get:
• Time A B Total • Point [N= 150] [N= 148] [N= 298] •_______________________________________________________________• SCREENING 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • RANDOM 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 2 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 3 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 4 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 5 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 6 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 7 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 8 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) •_______________________________________________________________
37
And 150 pages of where’s Waldo
• Time A B Total • Point [N= 150] [N= 148] [N= 298] •_______________________________________________________________• SCREENING 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • RANDOM 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 2 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 3 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 4 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 5 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 6 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 7 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 8 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • EARLY TERM 0 (0.00 %) 0 (0.00 %) 0 (0.00 %)UNSCHEDULED 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) •_______________________________________________________________
38
And if this isn’t enough…
•Change from baseline where missing is counted as zero (change in HR=64????)
•Values out of temporal order
•Lots and lots of decimal places
•P-values to 3 and 4 significant digits
•Etc., etc. etc.
39
We need to change our habits
•Current statistical approach
One variable at a time
Template applied to all studies
No wonder the docs don’t ask us to work with them!
•Simple change in attitude
Safety parameters aren’t separable
Focus first from biological insights and previous hints
Then scan the other variables
Then refocus
40
Conclusions
•Worry about multiplicity, but not too much
Listen to Joe Heyse’s talk this afternoon
•Beware the censor-happy protocol and analysis
•Don’t be too much the statistician
•But don’t forget randomness