Framework for FDA’sReal-World Evidence Program

M. Khair ElZarrad, PhD., M.P.H.Deputy Director, Office of Medical Policy

Center for Drug Evaluation and Research

Food and Drug Administration

March 28, 2019The views in this presentation do not necessarily represent the policies of FDA

Disclosures: None

2

For today…• FDA’s real-world evidence (RWE) Program.

• RWE and randomized controlled trials (RCTs)

• Demonstration projects that will help inform the use of RWD and RWE

• Current work

Throughout….

• Technology as a corner stone

• Opportunity and challenges

3

21st Century Cures Deliverables

• FDA shall establish a program to evaluate the potential use of real world evidence (RWE) to:

– Help support approval of a new indication for a drug approved under section 505(c)

– Help satisfy post-approval study requirements

• Program will be based on a framework that was to be issued by 2018

Real-World Data (RWD) are data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources.

Real-World Evidence (RWE) is the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD.

Real world evidence means data regarding the usage, or the potential benefits or risks, of a drug derived from sources other than traditional clinical trials

4* Figure minimally modified from: https://www.thrombosisadviser.com/differences-RCTs-real-world-studies/https://rwe-navigator.eu/use-real-world-evidence/model-effectiveness-in-the-real-world/

RWE and RCTs

• Data generated from the clinical trial as specified in the protocol for research purposes

• High level of pre-specified controls over variables

• Random distribution of known and unknown confounders

• Variable data, from many sources, not always generated for research purpose.

• Post-hoc controls

• Typically, no random distribution of potential confounders – may require a more intense analyses to identify and attempt to ameliorate confounders.

Broad selection

*

5

• Intended for drug and biological products

• Outlines FDA’s plan to implement the RWE program

• Multifaceted program

– Internal processes

– Guidance development

– Stakeholder engagement

– Demonstration projects

• Comment period closes April 16, 2019

https://www.fda.gov/downloads/ScienceResearch/SpecialTopics/RealWorldEvidence/UCM627769.pdf

6

Technology is key for all phases of drug development using RWD

• Identifying relevant data sets• Providing an assessment of

data quality• Identifying potential gaps

and other data sets for potential linkage

• Identifying missing data and potential covariates

• Determining appropriate analytical methods

• Informed consent considerations

• Ensuring that data include the appropriate study population

• Ensuring the existence of audit trail - All transactions (data modification, data analyses, data set changes, and other) are all accounted for and recorded

• Ensuring that privacy and confidentiality is protected throughout the study.

• Ensuring that appropriate controls are used

• Accounting for potential confounders that could impede accurate inference

• Detailing limitations associated with areas of unknown confounders or bias or the nature of data sets used

• Help augment data from RCTs

• Other

Hypothesis generation and feasibility

Data cleaning, aggregation & analyses

Inference

7

Example of Current RWE Workstreams

• The use of EHR and claims data

• Digital health tools

• Observational data and studies

• The use of current health care infrastructure to conduct clinical trials

• Data Standards and Implementation

• Regulatory considerations

8

Foundation for Use of Electronic Source Data

9

Framework for Evaluating RWD/RWE for Use in Regulatory Decisions

Considerations

• Whether the RWD are fit for use

• Whether the trial or study design used to generate RWE can provide adequate scientific evidence to answer or help answer the regulatory question

• Whether the study conduct meets FDA regulatory requirements

10

RWD Fitness for Use

• Data reliability (data accrual and data quality control) and relevance

– Data must be collected and maintained in a way that provides an appropriate level of reliability

– Data must be suitable to address specific regulatory question of interest• Challenges of capturing clinical effectiveness outcomes

• FDA does not endorse any one type of RWD

• Challenge: A single source of RWD may not capture all data elements, and multiple data sources may be needed

– How to integrate data sources and address duplication

11

Data Standards and Implementation

• Identifying and assessing data standards and implementation strategies required to use RWD/ RWE

• Identifying gaps between RWD/ RWE data standards and existing FDA systems

• Collaborating with stakeholders to adapt or develop standards and implementation strategies

RAPID MOBILE APPS

RAPID CLOUD ENVIRONMENT

EXISTING ANALYTICAL TOOLS

FDA COMMUNICATIONS

CHIO Cloud Infrastructure

• Taha Kass-Hout (CHIO)

• Jim Milto

• AWS Contractor Team

Security

• Lewis Watson (CISO)

• Shawn Porter

MedDRA / Data SMEs

• Sonja Brajovic

• Roger Goetsch

• Krishna Chary (E2B)

• Mitra Roca (HL7)

Web Services, Database,

Dashboards

• FDA RAPID Contractor Team

FDA RAPID LEADERSHIP

• Dr Henry Francis

• Richard Zhang

• Bruce Weaver

• Syed Haider

3500A / MEDWATCH

• Joseph Tonning

• Dr Robert Ball

MOBILE DEVELOPMENT

• NIH National Library of

Medicine: George Thoma,

Sameer Antani, Stacey Arnesen

• FDA RAPID Contractor Team:

Booz Allen Hamilton, Program

Manager Dinesh Kolla

Office of

Communications

• Paul Buckman

• Sherunda Lister

• Kim Rawlings

OIM (Streaming)

• Josh Lehman

ArcGIS (Geolocation from

Mobile Devices, Heat Maps)

•Martha O’Connor

•Wayne Gorski

•Nathan Beck

•Newland Agbenowosi (RAPID

Contractor Team)

Empirica (Signal Detection)

• Ana Szarfman

• Marilyn Pitts

• Michael Johnston

1

3

4

5

2

1

2

3

4

5Mobile Data Collection:

Clinicians / Reporters enter

MCM AE, attach digital file (ie,

photo) and app auto captures

geolocation (lat/long)

Data Transferred to FDA: Data

is submitted from mobile device

and sent over Cellular network

or WiFi to FDA’s Cloud

Environment

Data Processed in Cloud: Data

is processed and stored in FDA

Cloud. Adhering to FDA

Security and Data best

practices. Dashboards provide

leadership, comm staff and QC

views into MCM AE Data.

Response Sent to Reporter:

Within 24 hours a targeted response

is sent via email containing link to

digital file (ie, Podcast) with

additional information

Perform Analytics : Utilize

existing tools for location-based

analysis (ArcGIS) and signal

detection (Empirica) of captured

MCM AE data.

Real Time Application for Programable Interactive Devices (RAPID)

Division of Drug

Information (DDI)

• Mary Kremzer

• Catherine Chew

DMEP (Emergency)

• Mary Beth Roberts

Slide from presentation by Henry “Skip” Francis, M.D. Data Mining Research and Evaluation Team, FDA/CDER

13

Sources of Patient-Centric RWD Beyond Health Care Records

There is a need to explore the use of digital technology tools, electronic PROs, and wearables to potentially fill gaps.

14

Potential for Study Designs Using RWD to Support Effectiveness

• Transparency about study design and analysis before execution is critical for ensuring confidence in the result

• What should transparency for observational studies look like?

• How can technology help in ensuring transparency?

• Ensuring the appropriate application of informed consent and other regulations

Observational studies

15

16

Demonstration Projects

• Relevancy

• Quality

• Linkage

• Security

• Common data models

• Digital technology tools

• Advance analytical techniques

• Randomized trials

• Assessment of observational studies

DataTools Study

Design

EHRs: Greatest Potential and Challenge

EHR data have advantages of:

• Presenting a more complete and granular clinical picture

• Including labs/imaging/pathology reports

Challenges include:

•Data in pathology/ radiologyand clinical notes are often unstructured (80%)

• Typing ≠ consistency/complete documentation

• Clinical outcome measures for drug approvals may not be used or consistently recorded in practice 17

18

Creating Quality Clinical/Research Records – Design for Multiuse

• OneSource: “enter the right clinical data once, use many times”

• FDA collaboration with Dr. Laura Esserman (UCSF)

• Integration of standards based tools into the EHR to bring together health care and research

• Demonstration in breast cancer clinical trials

Courtesy of Dr. Laura Esserman and Susan Dubman

19

Demonstration Project –Impact AFib – Large Randomized Trial

• Implementation of an individually randomized controlled trial within the FDA-Catalyst distributed database environment

• Test the ability of an education intervention to increase the appropriate use of oral anticoagulants in a patient population with atrial fibrillation (afib) at high risk of stroke

• Intervention materials include letter from health plan to describe project, patient brochure (additional information on AF and OACs), and patients pocket card (tool to facilitate conversation between patients and providers)

• Enrollment of approximately 80,000 individuals in the early and late intervention arm

• Protocol available at:

https://www.sentinelinitiative.org/FDA-catalyst/projects/implementation-randomized-controlled-trial-improve-treatment-oral-anticoagulants-patients

20

Ultimately, we need to have confidence that data is reliable and fit-for use

• Audit trails are essential

• Transparency in every step is needed - transactional accountability and tracking

• Ensuring compliance with consent and HSP

• Facilitating data aggregation, exchange and transfer

21

Blockchain for Drug Supply Chain and Beyond

https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm630942.htm

22

FDA is evaluating the use of and management of health data from a variety of sources, including EHR, , genomic data, data from clinical trials, data from registries, and data collected via mobile devices, wearables and other digital sensors.

23

Conclusion

• FDA is committed to exploring the potential use of RWE to fully incorporate useful evidence into the regulatory paradigm

• Maximizing the use of RWD/RWE requires a converge of multiple skills and resources

• Multi-stakeholder efforts and collaborations will benefit everybody

• Technology is the corner stone that will enable the optimization of data collection, data analysis, and evidence generation

24

Continued External and Internal Engagement

September 13, 2017

25

Acknowledgements

• Jacqueline Corrigan-Curay

• David Martin

• Dianne Paraoan

• FDA RWE Committee