Applicant Town HallJanuary 14th, 2021

Cycle 1 2021 Pragmatic Clinical Studies (PCS)

2

Agenda

• About PCORI

• Programmatic Overview

• Patient and Stakeholder Engagement

• Administrative Overview

• Merit Review Process

• Resources

• Questions and Answers

Submit questions via the Question box in GoToWebinar

3

Today’s Presenters

Julie Kennedy

Lesch, MPA

Senior Engagement

Officer

Public and Patient

Engagement

Iris Giggetts,

MSW

Senior Contract

Administrator

Program Support

& Information

Management

Cary Scheiderer,

PhD

Senior Merit Review

Officer

Program Support

& Information

Management

Brigette Scott,

MA

Senior Project

Manager

Program Support

& Information

Management

Anne Trontell,

MD, MPH

Associate

Director,

Clinical

Effectiveness and

Decision Science

Penny Mohr, MA

Senior Advisor,

Healthcare Delivery

and Disparities

Research

4

Patient Centered Outcomes Research Institute (PCORI)

• An independent, non-profit research institute to support informed health

decisions of diverse stakeholders via research & dissemination activities

• Funds comparative clinical effectiveness research (CER) of 2 or more options to

prevent, treat, or manage a clinical condition or to improve health care delivery

‒ More than 770 research and research-related awards

‒ Funding to-date in excess of $2.6 billion

• Funds research contracts using milestones and deliverables to assess progress

5

Our Mission

PCORI helps people make informed health care decisions, and improves health care delivery and outcomes, by producing and promoting high integrity, evidence-based information that comes from research guided by patients, caregivers and the broader health care community throughout the entire research process.

6

PCORI Funds Comparative Clinical Effectiveness Research (CER)

What is CER?

• Generates and synthesizes evidence comparing benefits and harms of at least two

different methods to prevent, diagnose, treat, and monitor a clinical condition or

improve care delivery

• Measures patient-centered outcomes in real-world populations

• Helps consumers, clinicians, purchasers, and policy makers make informed

decisions that will improve care for individuals and populations

• Informs a specific clinical or policy decision

• Describes results in subgroups of people

7

Research PCORI Does Not Fund

PCORI does not fund research whose findings will include

• Development of clinical practice guidelines

• Coverage recommendations

• Payment or policy recommendations

8

Research PCORI Does Not Fund

PCORI does not fund studies of cost-effectiveness analysis (CEA).

For more info, please see FAQ here

https://help.pcori.org/hc/en-us/sections/200565804-Cost-and-Cost-Effectiveness-Analysis

9

PCORI Methodology Standards

Research funded by PCORI must adhere to the PCORI Methodology Standards, which represent minimal requirements for the design, conduct, analysis, and reporting of patient-centered outcomes research.

The 65 standards can be grouped into 2 broad categories and 16 topic areas.

Cross-Cutting Standards

• Formulating Research Questions

• Patient Centeredness

• Data Integrity & Rigorous Analyses

• Preventing/Handling Missing Data

• Heterogeneity of Treatment Effects

Design-Specific Standards

• Data Registries

• Data Networks

• Causal Inference Methods*

• Adaptive & Bayesian Trial Designs• Studies of Medical Tests

• Systematic Reviews

• Research Designs Using Clusters

• Studies of Complex Interventions

• Qualitative Methods• Mixed Methods Research

• Individual Participant-Level Data Meta-Analysis

(IPD-MA)*The first standard for Causal Inference Methods (CI-1) is

considered cross-cutting and applicable to all PCOR/CER studies.

https://www.pcori.org/research-results/about-our-research/research-methodology/pcori-methodology-standards

10

PCORI’s National Patient-Centered Clinical Research Network – PCORnet

• A collaborative network of health systems, researchers, patients, and clinicians with EHRs from more than 100 health systems and 70 million patients

• Use of existing PCORnet data resources or health systems may facilitate observational studies and randomized trials proposed under this PFA.

• Additional information on PCORnet can be found at PCORnet.org

• To leverage this PCORI resource, contact the PCORnet Front Door to schedule a consultation and ensure the proposed use of PCORnet is appropriate and feasible, and that budget materials accurately reflect the cost of using PCORnet resources.

‒ Initiate consultations by completing the intake form located here (https://pcornet.org/contact-us/).

http://pcornet.org/http://www.pcornet.org/contact-us/

PCS PFA: Overview

12

Goals of PCS PFA

• To address critically important evidence gaps where insufficient comparative effectiveness information is a major barrier to clinical or healthcare system decision making about 2 or more healthcare options

• To gain insight into the heterogeneity of treatment effects (HTE) for subgroups and individuals through the study of large, diverse, and heterogeneous populations

• To support research conducted under conditions that can be generalized to real-world populations and settings as reflected by

‒ The study of actual choices currently faced by patients, clinicians, or policy-makers

‒ Protocols which reflect usual clinical care delivery settings and practices

‒ Study designs, outcome measures, and follow up that are as simple as possible without sacrificing scientific rigor

13

PCS PFA Overview

• Seeks to fund clinical trials, large simple trials, or large-scale observational studies comparing two or more healthcare alternatives that address PCORI national priority areas

‒ Assessment, prevention, diagnosis, treatment, or management of a disease or symptom

‒ Improving healthcare system-level approaches to managing care

‒ Health or healthcare disparities

‒ Maternal morbidity and mortality

‒ Intellectual and developmental disabilities

• Randomized study designs are strongly encouraged but not required.

Available Funds:

Up to $90 million

Total Direct Cost:

Up to $10 million

Maximum Project

Period:

5 years

14

Pragmatic Design Features

• Seek simplified yet rigorous designs, outcome measures, and follow up

• Avoid design features considered to be more explanatory or mechanistically-focused in their research question

‒ Highly selected and homogeneous populations relatively free of comorbidities

‒ Specialized research settings and conditions

‒ Rigidly controlled research protocols unlike typical care delivery conditions

‒ Comparisons made to a placebo

15

PCORI Expectations forPragmatic Study Designs

• Research design features should be “fit for purpose” in answering the research questions of stakeholders

• Pragmatic elements should serve the research question not maximal pragmatism

• Tabular PRECIS summary in submitted applications should emphasize how design choices best inform future implementation of study findings

• Important supplementary references to PRECIS citations in the PFA

‒ PCORI Methodology Standards, especially for Studies of Complex Interventions to help define allowable adaptations or “guardrails” in how flexibly the protocol can be interpreted and carried out

‒ PCORI Guidance on the Design and Conduct of Trials in Real-World Settings

https://www.pcori.org/research-results/about-our-research/research-methodology/pcori-methodology-standardshttps://www.pcori.org/sites/default/files/PCORI-Guidance-Design-and-Conduct-of-Trials-Real-World-Settings-Factors-to-Consider-Pragmatic-PCOR.pdf

16

PCS PFA Requirements

• Must answer a critical choice among healthcare options, each with established efficacy

• Support testing of potential differences in effectiveness among relevant patient subgroups (heterogeneity of treatment effects)

• Deploy interventions that are sufficiently well defined to be replicable in their dissemination and implementation in US health care

• Adhere to PCORI Methodology Standards establishing minimum requirements for patient-centered outcomes research

17

PCS PFA Requirements

• Focus on clinical comparative effectiveness

‒ Explain how the research is comparative

‒ Name the comparators

‒ State the importance of the comparison(s)

• Describe the potential for findings to be adopted and improve clinical practice and delivery of care

‒ Identify end-users and how study findings will support their information needs

‒ Address patient and practice/organizational barriers and facilitators to rapid adoption

‒ Demonstrate applicability to diverse practices and delivery systems

‒ Show study endorsement by key patient, physician, and other stakeholder groups

18

Comparisons to Usual Care

• “Usual care” is typically a suboptimal comparator for CER studies

‒ It is ill-defined, difficult to quantify, and subject to geographic and temporal variations that limit its interpretability, applicability, and reproducibility

• If proposing “usual care” as a comparator

‒ Describe it in detail

‒ Justify its use rather than an active comparator and how it is a legitimate and coherent comparator (e.g. it is guidelines-based)

‒ Explain how the care given in the usual care group will be measured for each individual and how appropriate inferences will be drawn from its inclusion

19

Considerations for Complex Interventions

If proposing a multi-component intervention

• Components of the intervention should each have evidence of effectiveness

• If the bundled components have not been tested for effectiveness as a package

‒ Justify why the package of all components is likely to be more effective than a single component or subset of the components

‒ Discuss issues that might arise in the dissemination and sustainability of the package as a whole

• Refer to PCORI Methodology Standards for Complex Interventions

20

Considerations for Interventions Targeting Disparities

Evidence-based interventions shown effective in the general population may lack evidence of effectiveness in populations at risk for disparities

• PCORI is interested in studies that tailor and test effectiveness in populations at risk of disparities

• If necessary, interventions with documented efficacy or effectiveness in similar situations may be used with some adaptation

‒ Efficacy should be well-documented by prior research or systematic review

‒ A strong rationale should be provided for expecting the intervention to be efficacious in its new setting(s) and/or population(s)

‒ Efforts to adapt and validate an intervention should be minor with most of the proposed effort, time, and budget aimed at assessing comparative effectiveness

21

Pilot Funding Initiative 2021 PCS PFA Cycle 1

• PCORI will entertain applicant requests for funding support to cover critical and unmet needs for coverage of patient care costs that fall upon study participants

• Patient care costs may include study interventions that constitute the procedures, treatments, interventions, or other standard clinical care (patient care) provided

• Interested applicants must provide well-justified requests addressing 3 criteria

‒ The strategic importance of the research question as shown by a systematic review, national PCORI priority area, and/or its potential for significant impact upon clinical decision-making or policy formation

‒ Evidence of significant barriers to study completion in the absence of coverage

‒ Evidence of commitment by key stakeholders to support the future uptake of study results without ongoing subsidies, provided via letters of support or contributions of funding/other resources

22

What does PCORI look for when reviewing Letters of Intent (LOIs)?

• Importance and relevance of the CER question as demonstrated by

‒ a health decision of high relevance and critical importance to stakeholders

‒ a research need or gap documented in a systematic review or other high quality evidence syntheses

‒ Readiness of findings to be implemented into clinical practice or care delivery

• Clarity and credibility of applicants’ responses to the LOI questions about the study population, comparators, outcomes, and proposed research methods (e.g., study design, sample size, effect size)

• Programmatic fit and balance

23

Impact of COVID-19

• Due to the multiple risks and uncertainties of COVID-19, applications should include an explicit assessment of potential risks and risk management plans/contingencies for the proposed research as it may be affected by COVID-19

‒ Provisions in their leadership and staffing plan to have backup or “understudy” staff who can readily assume key duties and assure study continuity if necessary

• PCORI is receptive to LOIs and applications related to COVID-19 CER and its long-term impacts on health

‒ The expected timeframe for PCS studies to generate research results may not be suitable to address acute research needs of the COVID-19 pandemic

About PCORIpcori.org

Patient and Stakeholder Engagement

25

Patients and Other Stakeholders

PCORI

Community

Patient/

ConsumerCaregiver/

Family Member of

Patient

Clinician

Patient/

Caregiver Advocacy

Org

Hospital/

Health System

Training Institution

Policy

Maker

Industry

Payer

Purchaser

Definitions can be found here: https://www.pcori.org/about-us/our-programs/engagement/public-and-patient-engagement/pcoris-stakeholders

26

Patient-Centeredness vs. Patient Engagement

• Patient-Centeredness‒ Addresses outcomes (both benefits and harms) that are important to patients

‒ The interventions proposed for comparison are currently available to patients

‒ Study design is low-burden to participants

• Patient and Stakeholder Engagement‒ Patients and stakeholders are partners in research, not only “subjects”

‒ leverages the expertise of patients, clinicians and other stakeholders for the purpose of improving study design and conduct

‒ Includes a range of engagement approaches: input, consultation, collaboration, shared leadership

‒ Can occur by building upon existing relationships, or developing new ones

27

“Research Done Differently”

PCORI’s research has shown that engagement can impact studies:

Forsythe LP, Carman KL, et al. Patient Engagement In Research: Early Findings From The Patient-Centered Outcomes Research Institute. Health Aff (Millwood). 2019 Mar;38(3):359-367;

Acceptability:studies in which patients and clinicians will be willing to participate based on burden, usability,

and alignment with preferences, values, and needs

Feasibility:mitigating potential or actual roadblocks—particularly to making interventions, enrollment,

retention, and data collection more doable in real-world settings

Rigor:choices that minimize bias and enhance data quality

Relevance:applicability and importance of the research for patients and clinicians making decisions,

including the importance of research questions and outcomes, generalizable study samples, interpretations of findings in a real-world context, and more effective dissemination

https://www.healthaffairs.org/doi/full/10.1377/hlthaff.2018.05067?url_ver=Z39.88-2003&rfr_id=ori%3Arid%3Acrossref.org&rfr_dat=cr_pub++0pubmed&

28

Continuum of Engagement Practices

29

Engagement Requirements for LOI

Two main engagement objectives for the LOI:

1. Justify why the research question is important to patients and other stakeholders,

2. Briefly indicate how engagement may have already occurred, and with whom, how engagement will take place within the study if funded, the currently involved stakeholders and stakeholders who will invited to partner if funded

30

Engagement Requirements for Application

1. List of proposed patient and other healthcare research partners (include names and affiliations, if available), and justification for their inclusion

2. Goals for working with stakeholders, which may include affecting the acceptability, feasibility, rigor, and/or relevance of the study

3. Description of the approaches the team will use to engage stakeholders at key decision points over the course of the study

4. Proposed ideas for how stakeholder advisory committees and/or governance committees that include stakeholders will be structured and their responsibilities

Funded awardees are required to submit a more detailed engagement plan six months after contract execution.

Applications should include:

https://www.pcori.org/sites/default/files/PCORI-Updated-Engagement-Plan-Template.pdf

Administrative Requirements

32

Using the PCORI Online System

• Register as a New User and create your LOI as soon as possible

• Please note that the PI and AO cannot be the same person

• Please only use Chrome, Safari, and Firefox browsers to access the system

https://pcori.force.com/engagement

PCORI Online Training Slides

PCORI Online Application Cheat Sheet

PCORI Online Training Resources

https://pcori.force.com/engagementhttp://www.pcori.org/sites/default/files/PCORI-Online-Pre-Award-Applicant-Training-Slides.pdfhttp://www.pcori.org/sites/default/files/PCORI-Online-Application-Submission-CheatSheet.pdfhttps://www.pcori.org/sites/default/files/PCORI-Online-Pre-Award-User-Guide.pdf

33

Letters of Intent to Apply: Eligibility to Submit

• Any private sector (non-profit or for-profit) research organization.

• Any public sector research organization (university or college hospital or healthcare system, laboratory or manufacturer, unit of local, state, or federal government).

• Non-domestic components of organizations based in the US and foreign organizations may apply, as long as there is demonstrable benefit to the US healthcare system and US efforts in the area of patient-centered research can be clearly shown.

• Individuals are not permitted to apply.

34

Letter of Intent (LOI): Requirements

• An LOI is required and must be submitted prior to the deadline.

• To submit an LOI, download the PFA-specific Letter of Intent Template from the

Funding Center to begin your LOI.

• You must answer all questions.

• Do not upload additional documents as part of your LOI. Letters of endorsements or

support are not accepted at this stage.

• Only those LOIs deemed most responsive (programmatically and administratively) to this

PFA will be invited to submit a full application.

https://www.pcori.org/funding-opportunities/announcement/broad-pcori-funding-announcements-cycle-3-2020

35

From LOI to Application

• Full applications are invited based on the information provided in the LOI.

• Changes to the following require PCORI’s approval:

‒ Principal investigator

‒ Institution

‒ Research question(s)

‒ Specific aims

‒ Study design

‒ Comparators

‒ Budget/period of performance

36

Invited to Apply: Tips for Success

• Adhere to the Submission Instructions for the appropriate PFA and funding cycle

• Start and submit application early

• Have a copy of your approved LOI readily accessible

• Ensure that all team members can see the application in the system (check during the LOI stage)

• Inform your AO of your intent to submit

• Clearly describe comparators for the study

• Submit the completed application on/before the due date by 5:00 PM ET

37

Application Components: Research Plan

• Research Strategy: 15 pages (upload as a single pdf file)

‒ Provide all the information requested, as outlined in the template:

*Specific Aims *Significance

*Background *Study Design or Approach

• Research Team & Environment: 3 pages

‒ Describe the research team’s capabilities to accomplish the goals of the proposed research project and the appropriateness of the research environment to conduct the study.

• Dissemination & Implementation: 1 page

‒ Describe how you will make study results available to study participants after you complete the analyses, and possible barriers to disseminating and implementing the results of this research in other settings.

• Return of Aggregate Study Results: 1 page

‒ Describe the approach that will be used to communicate the research findings to study participants once the study is complete.

• Protection of Human Subjects: 5 pages

‒ PCORI follows the Federal Policy for the Protection of Human Subjects (45 CFR part 46), including the Common Rule.

38

Application Components: Research Plan (cont.)

• Consortium Contractual Arrangements: 10 pages

‒ Describe the proposed components of the research project that will be performed by subcontracted organizations.

‒ Explain the strengths that these partners bring to the overall project to ensure successful submission of contract deliverables in accordance with the milestone schedule.

• References Cited: 10 pages

‒ List the full bibliographical citation for each reference. Each reference must include the names of all authors the article title and journal or book title, the volume number, the page numbers, and the year of publication

39

Application Components: Research Plan (cont.)

• Appendices: 10 pages (optional)

‒ Applicants can include additional materials that they believe are useful, but reviewers are not required to review the appendix materials in evaluating the application.

• Appendix: 2 pages (To be completed only if requesting PCORI funding of patient care costs).

‒ Description and justification of request for Patient Care Costs coverage

• Methodology Standards Checklist: no page limit (upload as excel file)

‒ Applicants must complete each column of this checklist, as appropriate, and include it with the Research Plan PDF upload

40

Additional Application Components

• Milestones/Deliverables: no page limit (upload as excel file)

• People & Places Template (upload as a pdf file)

‒ Professional/Patient/Stakeholder Biosketch: 5 pages per individual

‒ Project/Performance Site(s) and Resources: 15 pages

• Leadership Plan Template (Dual PI Applications only): 5 pages

• Subcontractor Detailed Budget Template: (upload as a excel file)

• Budget Justification Template: no page limit (upload as a pdf file)

• Letters of Support: no page limit (upload as a pdf file)

• Resubmission letter: 1 page (if applicable) (upload as a pdf file)

41

How to Request Coverage of Patient Care Costs

In the LOI and full application (if invited)

• Describe the procedures, treatment, interventions, or clinical personnel services for which funding is requested and provide their estimated costs.

In submitting an application

• Limit the description and justification to 2 pages

• Provide stakeholder Letters of support

Merit Review

43

Merit Review Process

• pcori.org/content/merit-review-process

44

Application Review

Applications are reviewed against six criteria:

1. Potential for the study to fill critical

gaps in evidence

2. Potential for study findings to be

adopted into clinical practice and

improve delivery of care

3. Scientific merit (research design,

analysis, and outcomes)

4. Investigator(s) and environment

5. Patient-centeredness

6. Patient and stakeholder engagement

• Each application is reviewed by

three scientists, one patient,

and one other stakeholder.

• PCORI’s Board of Governors

makes funding decisions based

on merit review and staff

recommendations.

45

Submission and Key Dates

What When

LOI Deadline February 2, 2021 by 5:00 pm ET

Application Deadline May 4, 2021 by 5:00 pm ET

Merit Review Dates July 2021

Awards Announced November 2021

Earliest Start Date March 2022

46

Resources

Refer to the funding opportunities page in our Funding Center (http://www.pcori.org/funding-opportunities) for the following resources:

• PFA and Submission Instructions

• PCORI Online User Manuals

• General Applicant FAQs: https://help.pcori.org/hc/en-us/categories/200010230-Applicant-Resources

• PCORI Online: https://pcori.force.com/engagement/

• Research Methodology: http://www.pcori.org/node/4020

https://help.pcori.org/hc/en-us/categories/200010230-Applicant-Resourceshttps://pcori.force.com/engagement/http://www.pcori.org/node/4020

47

Specific Engagement Resources

Please visit PCORI's website for updated content and resources on the Public and Patient Engagement pages: https://www.pcori.org/engagement/engagement-resources

• The value of engagement and planning for effective engagement in research

• Definitions and examples of different stakeholders

• Engagement Resources

‒ Engagement Rubric

‒ Compensation Framework

‒ PCORI in Practice webinars

‒ Research Fundamentals training for lay person partners

• Engagement in Health Research Literature Explorer

https://www.pcori.org/engagement/engagement-resourceshttps://www.pcori.org/engagement/engagement-literature

48

Where can I find help?

Visit pcori.org/apply

• Submission Instructions

• FAQs

• PCORI Online User Manuals

• Sample Engagement Plans

Schedule a Call with a Program Officer

• Submit a request at

pcori.org/content/research-inquiry

• E-mail sciencequestions@pcori.org

Contact our Helpdesk

• E-mail pfa@pcori.org

49

Q&A

Ask a question via the question box in GoToWebinar

If we are unable to address your question during this time, e-mail the Helpdesk

at pfa@pcori.org.

mailto:pfa@pcori.org

Thank you