safety throughout the life cycle of vaccines 2012 russek... · 2015-02-02 · safety throughout the...
TRANSCRIPT
![Page 1: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/1.jpg)
1
Safety Throughout the Life Cycle of Vaccines
Estelle Russek-Cohen, PhDActing Director, Division of Biostatistics Office of Biostatistics & Epidemiology
Center for Biologics Evaluation and ResearchFDA
![Page 2: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/2.jpg)
2
DisclaimerThe findings and conclusions in this presentation have not been formally disseminated by the Food and Drug
Administration and should not be construed to represent any Agency
determination or policy.
![Page 3: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/3.jpg)
3
FDA Organization
Center for Biologics
Evaluation and Research
Center for Devices and Radiological
Health
Center for Drug Evaluation and
Research
Center for Food Safety and Applied
Nutrition
Center for Veterinary Medicine
Center for Tobacco Products
National Center for Toxicological
Research
Office of Regulatory
Affairs
![Page 4: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/4.jpg)
4
Division of BiostatisticsOrganizational Chart
Division Director
TherapeuticEvaluation Branch
Vaccine Evaluation Branch
Viral and BioassayTeam
Bacterial/AllergenicTeamHematology Team
Diagnosticsand
Screening Team
Cell and GeneTherapy Team
![Page 5: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/5.jpg)
5
Outline• Vaccines 101• Safety in Pre-Market• Postmarket and SCCS• Passive Surveillance and VAERS• Active Surveillance
VSD, PRISM, Federal Partners• Open methodological questions• Conclusions
![Page 6: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/6.jpg)
6
Vaccine 101
![Page 7: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/7.jpg)
7
Vaccines are not drugs!• Administered to healthy people• Designed to prevent disease• Importance of herd immunity
Need high buy-inMaintaining confidence in vaccines is key
![Page 8: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/8.jpg)
8
Vaccines are not drugs!! • Like other biologics, vaccines are licensed• FDA: premarket review• Vaccines in post-market:
HHS: FDA, CDC, NVPOGlobal: WHO, EMA, …..
• Pandemic and Seasonal flu Vaccines:New variants keep coming
• Lot release review
![Page 9: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/9.jpg)
9
http://www.flu.gov/professional/federal/fed-plan-to-mon-h1n1-imm-safety.pdf Advisory Committees in Red
![Page 10: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/10.jpg)
10
Immunological Assay• Required for all vaccines• Carefully reviewed at FDA• Immune response endpoint
Correlate of protection(?)see papers by Gilbert et alExample: Antibody to HBV
Surrogate endpoint (?)see papers by Gilbert or Prentice
![Page 11: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/11.jpg)
11
Lot-to-Lot Consistency• Three lots of vaccine• Used in a 3 arm study
….often sub-study of bigger trial• Three lots must be comparable
….similar to a bioequivalence criteria• Immunological assay variability can be an
issue….Important for design
![Page 12: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/12.jpg)
12
Vaccine Development• Phase 1 Safety studies• Phase 2 Different doses and schedules• Hundreds of patients• Characterize very common A.E. • Phase 3 Pivotal studies for licensure• Plan for pharmacovigilance • Unsafe vaccines don’t get to next phase!• Sometimes additional safety registry
![Page 13: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/13.jpg)
13
Efficacy Trials • First of a kind vaccines:
Endpoint based on case definition• Usually has a placebo arm• Cannot control disease exposure• Large trials and super-superiority • VE=Vaccine Efficacy=
1- (disease rate, V)/(disease rate, C)• Flu VE>40% (95% Lower Conf Bound)
![Page 14: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/14.jpg)
14
Several trials: common • Different age classes (e.g. flu)
Infants, kids, adults, elderly• Common concomitant vaccines• International trials
Higher background ratesRelevance to US? (e.g. strain types)
• Safety data are captured in all
![Page 15: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/15.jpg)
15
Immunogenicity trials • 2nd of a kind• Disease prevalence is now lower• Active control arm• Non-inferiority of an immune response
endpoint (NI Margin: talk with OVRR)• Multiple serotypes : multiple endpoints
trivalent flu, Prevnar 13• Interpretation of safety data ?
![Page 16: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/16.jpg)
16
Plan for Pharmacovigilance : During BLA review (DE)
• International Conference on Harmonization (ICH) Guidance for industry: E2E Pharmacogivilance Planning format
• Early consideration of FDAAA 2007 options• Postmarketing studies are informed by:
– Experience with post marketing surveillance strengths and limitations
– Experience with similar products – Safety issues identified by Clinical and Statistical
reviewers during pre-licensure review
![Page 17: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/17.jpg)
17
Vaccines are not drugs !!!• Some rare but serious events:
linked to vaccine use • Guillian Barre Syndrome
(Swine Flu 1970s)• Intussusception (Rotavirus 1990s)• Severe allergic reactions (Vaccines & eggs)
• May trigger additional studies or spur serious post-market surveillance
![Page 18: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/18.jpg)
18
Inference for safety in phase 3• Most studies use 1:1 allocation• Some expose more to new vaccine• Flu guidance for established mfg:
Rule out 1 in 300 adverse event• Inference with very big N, very small p
Most articles assume rate>.01 or 1%
![Page 19: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/19.jpg)
19
Inference Methods Study: Pre-specified adverse events
• Exact methods:Computationally burdensome in phase 3For safety: symmetric methods 95% confidence intervalWant appropriate one sided values
• Score methods are compromise (Newcombe)• Wald and related methods are poor• Farrington and Manning for NI not implemented
the same across packages.
![Page 20: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/20.jpg)
20
Risk difference (RD) orRelative Risk (RR)
• Very rare events, RR exaggerates risk • RR when control has zero events?• Deeks et al: RR more stable across studies• RD provides excess risk estimate
number of cases per 100,000• Reporting both makes sense (SPERT, 2009)• Control of alpha: rarely done if small number of
pre-specified events.
![Page 21: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/21.jpg)
21
21 RotaShield• As of 1998, rotavirus was the most common cause of
severe gastroenteritis in infants and children less than 5 y.o. in the U.S.– 500,000 physician visits, 50,000 hospitalizations, 20
deaths/year in the U.S.– 600,000 deaths/year worldwide
• The first rotavirus vaccine, RotaShield was licensed in August 1998
• RotaShield was voluntarily taken off the market in 1999
![Page 22: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/22.jpg)
22
22RotaShield and intussusception
• Very soon after licensure, reports of intussusception temporally associated with RotaShield began appearing in VAERS
• Intussusception is a potentially life-threatening bowel obstruction– Background incidence in infants ~ 0.0004 cases / year
• Following investigation, CDC determined that 1 – 2 additional cases of intussusception would be caused by RotaShield per 10,000 infant-years
![Page 23: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/23.jpg)
23
23 The REST trial (1)• Thus, intussusception was a major concern for future
rotavirus vaccine candidates• Development of the RotaTeq vaccine included the
Rotavirus Efficacy and Safety Trial (REST)• 69,625 subjects were vaccinated (n=34,837) or placebo
(n=34,788) • The primary efficacy endpoint was based on cases of
disease– But this was only assessed in 5,673 (8%) of subjects
![Page 24: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/24.jpg)
24
24 The REST trial (2)• The sample size was driven by the safety endpoint of
intussusception– Subjects actively monitored for potential
intussusception at 7, 14 and 42 days post each dose, then every 6 weeks for 1 year
– Primary safety win criterion was upper bound of the 95% CI for RR to be ≤ 10 without hitting safety stopping boundaries
– Group sequential design• Initial analysis at n = 60,000• Subsequent analyses after each 10,000 subjects up
to 100,000• Study concluded with 6 cases in the vaccine arm, 5 in
the placebo arm
![Page 25: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/25.jpg)
25
25
Lessons learned from a huge safety trial
• A huge trial may only be possible with a relatively easy-to-ascertain primary safety endpoint
• Embedding efficacy and detailed safety subsets in the overall safety trial improves efficiency– Always a good idea to collect whatever efficacy and
safety information is feasible• With a background incidence of ~ 1/10,000, even 60,000
– 100,000 subjects will only permit ruling out an RR of 10.– Safety trials have to operate within realistic constraints
![Page 26: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/26.jpg)
26
Vaccines vs Drugs in Postmarket
![Page 27: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/27.jpg)
27
Key differences • Fewer possible confounders with vaccines• Drugs: indications and duration can vary• Vaccines: limited exposures• Larger premarket studies imply:
Looking for very rare A.E.s in postmarket
![Page 28: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/28.jpg)
28
Self controlled case series• Tutorial in Stat in Med (see references)• Developed methods for vaccine safety• Each subject serves as own control• Efficient signal detection• Doesn’t formally address who is at risk• OMOP methodology comparisons….this
can outperform many other methods.
![Page 29: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/29.jpg)
2929
Vaccination
-------------------------------------------------------------------------------------------Days -56 -15 0 42 84
Comparison Risk window Comparisonwindow (pre) window
(post-post)
Self-Controls
![Page 30: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/30.jpg)
30
Vaccine Adverse Event Reporting (VAERS)
![Page 31: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/31.jpg)
31
Passive Surveillance: VAERS • Voluntary reports• Patients, physicians, others• Most fields are publicly available• Case of no denominators…
how many are exposed to product?• CBER review: Division of Epidemiology• Use Empirica Signal Detection Software
![Page 32: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/32.jpg)
32
Passive Surveillance: AERS and VAERS • STRENGTHS:
– Open-ended for hypothesis generation– Potential detection of new or rare adverse events– Timeliness– Geographic diversity– Capability to monitor production lots
• LIMITATIONS: – Missing and inaccurate data– Under-reporting/Stimulated reporting– Absence of controls and denominators– Inability to assess causation– Low likelihood of detection for long latency events
![Page 33: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/33.jpg)
33
CBER research initiative• Text mining of narratives in VAERS• 2 stage process:• using natural language processing to
extract features from text• use supervised learning methods to
develop classification rule. Can evaluation of narratives improve
yield rate of anaphylaxis?
![Page 34: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/34.jpg)
34
• Manual search and review of case reports for H1N1 anaphylaxis (10/12/2009-06/30/2010).
• Important to automate:– the whole process, but step 2 requires MO (<=>pdf files) review.– at least step 1 and provide MOs with the low number of reports
for further review in step 2.
Review by Medical Officers
ConfirmedN=100
A B C
All flu reports N=6034
Manual searchN=237
Search by PT term and/or keyword
Case categorization by reviewing MO
1 2
1* 2
![Page 35: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/35.jpg)
35
Training set: Classification ResultsMOs’ review
TotalsPos Neg
Text Miner
Pos 183 352 535
Neg 54 5445 5499Totals 237 5797 6034
Sensitivity: 77.2%Specificity: 94.0%
PPV: 34.2%NPV: 99.0%
![Page 36: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/36.jpg)
36
Independent validation Independent validation of algorithm, truth
determined by manual review (N=689):PPV=30% NPV=99%
PPV hurt by low prevalence, but NPV suggests text miner can be used to enrich the dataset.
![Page 37: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/37.jpg)
37
Vaccine Safety DataLink CDC and FDA
![Page 38: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/38.jpg)
38
CDC Vaccine Safety Datalink (1991)
• Eight geographically diverse health maintenance organizations that participate in a large linked database representing approximately 3% of U.S. population
• Surveillance and “Hypothesis testing” studies can be conducted– Vaccination (exposure)– Outpatient, emergency department, hospital and
laboratory coding data (health outcomes)– Demographic variables (confounders)– Accessible medical chart review
![Page 39: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/39.jpg)
39
VSD Rapid Cycle Analysis
• Method motivated by Wald SPRT:Wald: Simple vs Simple Hypotheses
• Near continuous monitoring (weekly)• Extension to composite alternative• Two variants:
Poisson: #events vs expected countsBinomial: event rates exposed vs not exp
![Page 40: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/40.jpg)
40
Max SPRT Kulldorff et al (2011)
• Likelihood ratio statistic • (Poisson or binomial)
• Length of surveillance fixed (e.g. 2 yrs)• Time is expected counts not calendar time• Rapid detection important
![Page 41: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/41.jpg)
41
Sentinel InitiativePublic-private partnership
Mini-Sentinel• Collaboration of data partners, academia, non-profit organizations• Covers all FDA regulated medical products
Federal Partners Collaboration• Department of Defense (DoD)• Department of Veterans Affairs (VA)• Centers for Medicaid & Medicare Services (CMS)
OMOP*
Brookings Institution
Postlicensure Rapid Immunization Safety Monitoring program (PRISM)
Blood Safety Continuous Active Surveillance Network (Blood-SCAN)
Mini-Sentinel is Part of FDA’s Sentinel Initiative
* Observational Medical Outcomes Partnership
![Page 42: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/42.jpg)
42
PRISM Basics• Mini-Sentinel program dedicated to
vaccine safety• Claims based system with data from 4
national health plans– Aetna, HealthCore (Wellpoint), Humana,
Optum (United Healthcare)– Data linked to 8 vaccine registries in USA
• Access to medical records and pharmacy data
Pharmacoepidemiol Drug Saf. 2012 Jan;21 Suppl 1:291‐7.
![Page 43: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/43.jpg)
43
Analytic Modular Programs• Represents next step in standardization
– From quick query to standardized analytic programs
• Designed to address 2 problems– Facilitates simultaneous monitoring of numerous FDA approved
medical products– Reduces start-up time and resources of customized analyses
• Semi-automated product safety assessments– Predefined algorithms to identify exposures, outcomes,
comparators– Standardized confounding control– Analytic choices chosen to cover most scenarios
![Page 44: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/44.jpg)
44
Prespecified event: 3 Methods Initially Selected
1. Self-controlled design– Useful for single or short-term exposures or when no
independent comparator group is available– When between-person confounding is large but within-person
confounding is modest2. Exposure match cohort
– Uses propensity or disease risk scores in fixed or variable ratio– Provides flexible choices of effect measures, multiple endpoints
and broad range of alerting rules3. Full cohort design with regression
– Permits a high degree of analytic flexibility (e.g., the ability to simultaneously evaluate interactions, multiple comparison groups, and subgroups)
![Page 45: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/45.jpg)
45
PRISM Methods: Improving Causal Inference
• Improve on design-based confounding control– Traditionally use matching (age, site, sex),
stratification– Limited by number of confounders or high
dimensionality– Loss of efficiency (cannot use entire cohort)– Method like Lunceford and Davidian (2004)Group sequential element for surveillanceSee ms by Cook et al (Mini-sentinel site)
![Page 46: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/46.jpg)
46
Data Mining Development• Test whether it is possible to detect adverse
events without pre-specifying them a priori• Develop statistical approach to simultaneously
evaluate hundreds of different adverse events– Advantage: detect unexpected adverse events– Disadvantages: not possible to adjust for all possible
confounders, as they vary by disease outcomesFinding optimum risk window for all events is hardHierarchy of events imperfect.
Pilot phase: can we detect known signals?
![Page 47: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/47.jpg)
47
Mining: 3 Methods Being Evaluated
• Project led by Martin Kulldorff– DuMouchel’s Gamma Poisson Shrinker– Tree-based scan statistic with population based controls– Tree-based scan statistic with self-controls
• Basics of Tree Scan algorithm– Use a hierarchical tree– Evaluate cuts on the tree (assess observed vs. expected at each
leaf) – Control for multiple testing
Kulldorff M, Fang Z, Walsh S. A tree‐based scan statistic for database disease surveillance. Biometrics, 2003,59:323‐331.
![Page 48: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/48.jpg)
48
Example of a Small Tree
Myocardial Infarction
Cardiac Arrhythmia
Cardio-myopathy
Acute Renal Failure
Kidney Infection
Cut
![Page 49: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/49.jpg)
49
Tree-based SCAN Statistic1. Scan the tree by considering all possible cuts on any
branch2. For each cut, calculate the likelihood3. Denote the cut with the maximum likelihood as the most
likely cut (cluster)4. Generate 9999 Monte Carlo replications under H0.5. Compare the most likely cut from the real data set with
the most likely cuts from the random data sets6. If the rank of the most likely cut from the real data set is
R, then the p-value for that cut is R/(9999+1).
![Page 50: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/50.jpg)
50
Tree-based SCAN Statistic1. Scan the tree by considering all possible cuts on any
branch2. For each cut, calculate the likelihood3. Denote the cut with the maximum likelihood as the most
likely cut (cluster)4. Generate 9999 Monte Carlo replications under H0.5. Compare the most likely cut from the real data set with
the most likely cuts from the random data sets6. If the rank of the most likely cut from the real data set is
R, then the p-value for that cut is R/(9999+1).
Helps answer, “Has FDA observed any new safety issues?” without pre‐specifying a particular outcome
![Page 51: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/51.jpg)
51
Implementation in distributed environment
• Methods assessed– Empirical Bayes Gamma Poisson Shrinker
(DuMouchel)– Tree-based scan statistic (Kulldorff)Open challenges:
Multiplicity of risk windows or age classesConstellations of eventsConcomitant vaccines or drugs
![Page 52: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/52.jpg)
52
Post-marketing Vaccine Safety Research: Federal Partners
• Claims datasets– Near-real time monitoring– Centers for Medicare and Medicaid (CMS)
• Population >35 million• Comprehensive datasets
– Electronic data for near-real time monitoring – Access to medical records for diagnosis verification and hypothesis
confirmation – > 1,000,000 beneficiaries in each dataset
• Indian Health Service • Department of Defense• Veterans Administration
![Page 53: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/53.jpg)
53
A fictional vaccine…..
![Page 54: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/54.jpg)
54
PRISM-Enhanced Pharmacovigilance PlanHealth Outcome Action PlanImportant Identified Risks
1. Anaphylaxis2. Syncope causing
injury
• Quick queries to follow up any safety signals from passive surveillance• Routine pharmacovigilance
Important Potential Risks
• Febrile seizures• Immune
thrombocytopenic purpura
• Myocarditis
• Prospective sequential surveillance with self controlled analysis• Current vs. historical surveillance for rare events• PMC observational study 50,000 subjects •Routine pharmacovigilance
Important Missing Information
1. Safety in pregnant women, older adults (>64 years)
2. Unanticipated adverse events
• Retrospective pregnancy safety study at 3 years postlicensure • Data mining• Routine pharmacovigilance
![Page 55: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/55.jpg)
55
FDA-CMS Project SafeVax: Rapid Assessment of Vaccine Safety
2009–2010 season: monitored safety of seasonal and H1N1 pandemic influenza vaccines Approximately 45 million CMS
beneficiaries and more than 3 million H1N1 pandemic vaccinations monitored
Monitoring of GBS after seasonal influenza vaccine now routine
More and better data for safety: other adverse events, improved access to medical records, possible exploration of Medicaid data
Developed a novel approach to near real-time safety surveillance adjusting for delay in claims in collaboration with CMS
![Page 56: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/56.jpg)
56
FDA Next Steps • Better integrated safety summaries.
Reviewing role critical. • Data mining in premarket RCTs
Pediatric vaccines…• Gaining more hands-on experience with
active surveillance datasets. • Active engagement of DB/DE in best use
of Passive Surveillance Data.
![Page 57: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/57.jpg)
57
References• Text mining for the Vaccine Adverse Event Reporting System: medical text
classification using informative feature selectionBotsis T, Nguyen MD, Woo EJ, Markatou M, Ball R (on line June 2011)
• Tutorial in Biostatistics: The self controlled case series method SIM 200625: 1768-1797
Whitaker, HJ, Farrington CP, Spiessens B, Musonda, P.• Post-Licensure Rapid Immunization Safety Monitoring AJE (2012)
Yih WK, Lee G, Lieu T, Ball R, Kulldorff M, et al • Bayesian Data Mining in Large Frequency Tables with an Application to
FDA Spontaneous Reporting System. Am. Stat. 1999 53:177-190DuMouchel, W.
• Mini-Sentinel Web Site: Several references from the Statistics Working Group (on-line)
• Crowe BJ, Xia HA, Berlin JA et al Recommendations for safety planning, data collection, evaluation and reporting during drug, biologic, and vaccine development: a report of the safety planning evaluation and reporting team. Clinical trials 2009 6:430-440
![Page 58: Safety Throughout the Life Cycle of Vaccines 2012 Russek... · 2015-02-02 · Safety Throughout the Life Cycle of Vaccines Estelle Russek-Cohen, PhD ... Nutrition Center for Veterinary](https://reader034.vdocuments.mx/reader034/viewer/2022042420/5f37fc3a0daece373b05035e/html5/thumbnails/58.jpg)
5858
References (VSD)1. Lieu TA, Kulldorff M, Davis RL, et al. Real-time vaccine safety
surveillance for the early detection of adverse events. Med Care 2007;45:S89-95.
2. Klein NP, Fireman B, Yih WK, et al. Measles-mumps-rubella-varicella combination vaccine and the risk of febrile seizures. Pediatrics 2010;126:e1-8.
3. Kulldorff M, Davis RL, Kolczak M, et al. A maximized sequential probability ratio test for drug and vaccine safety surveillance. Seq Anal, 2011 30:58-78
4. Li L, Kulldorff M. A conditional maximized sequential probability ratio test for pharmacovigilance. Stat Med 2010;29:284-95.
5. Yih WK, Kulldorff M, Fireman BH, et al. Active surveillance for adverse events: The experience of the Vaccine Safety Datalink Project. Pediatrics, in press.