pcori update: lenore arab, phd, ms value of information: starting the discussion doug bell, md, phd...
TRANSCRIPT
PCORI Update: Lenore Arab, PhD, MS
Value of Information: Starting the Discussion
Doug Bell, MD, PhD12/04/2012
Director, Dr. Lenore Arab
Project Coordinator, Robin FariaStudy Coordinator, Erica Sasman
The Rapid Response Team:
Who We Are
Outline
PCORI Evolving Structure PCORI Status of Current Applications Currently Open RFAs: January & February
2013 LoIs Future PCORI Disease Priority Areas Value of Information Analyses- why you
should care, and what it might entail
PCORI Organization: Divisions
Status of Current Proposals
Topic Improving Methods for Conducting Patient-
Centered Outcomes Research Due Dates
LOI (required): 1/15/2013; 6/15/2013; 10/15/2013 Application: 3/13/2013; 8/15/2013; 12/17/2013
Funding $250,000 direct per year for three years (shorter
duration encouraged)
PCORI Funding Opportunity 5: Inaugural Cycle
Specific Questions of Interest1.Development of methods for patient centeredness2.Research in methods to conduct systematic reviews of
patient-centered comparative effectiveness research.3.Development of methods for generating, selecting, and
prioritizing topics for research and for including patients and stakeholders in the peer‐review process.
4.Development and refinement of general analytic methods.
5.Development and refinement of design‐specific analytic methods.
Improving Methods for Conducting Patient-Centered Outcomes
Research I
Specific Questions of Interest, Continued6. Research that determines the validity and
efficiency of data sources commonly used in PCOR.7. Research related to Patient-Centered Outcomes
(PCOs) and Patient-Reported Outcomes (PROs).8. Research in methods to enhance the
reproducibility, transparency, and replication of PCOR research.
9. Research that evaluates and compares strategies for training researchers, patients, and other stakeholders in the methods of patient‐centered outcomes research.
Improving Methods for Conducting Patient-Centered Outcomes
Research II
PCORI Funding Opportunity, Cycle III – 3rd Opportunity
Topics Assessment of Prevention, Diagnosis, and Treatment
Options Improving Healthcare Systems Communication and Dissemination Research Addressing Disparities
Due Dates LOI (required): 2/15/2013 Application: 4/15/2013
Funding $500,000 direct cost per year for xx years?
Purpose of Survey: to actively engage faculty interested in PCORI research and find ways to further help the research community in areas of interest
Incentive: two winners per week for $50 gift card Preliminary Results on Week 1 of Survey:
34 respondents First winners to be announced Thursday Cycle II and Cycle III Interest:
PCORI Interest Survey
Assessment of
Prevention
Communication &
Dissemination
Improving Healthcare Systems
Addressing Disparities
Accelerating Patient-Centered Outcomes
Yes 3 0 1 2 1 1No 17
Undecided 13 3 2 3 6 5Total 33 3 3 5 7 6
PCORI Staff Choice of Disease Priority Areas
Douglas Bell, MD, PhD
Value of Information: Starting the Discussion
Parameter Treatment A(95% CI)
Treatment B(95% CI)
Cure rate 94%(86.0 to 98.6%)
90%(83.5 to 95.0%)
Life expectancy if cured 20 years
Life expectancy if treatment fails
5 years
Costs of managing treatment failure
$50,000
Overall complication rate 20%(10.0 to 27.5%)
5%(2.1 to 10.0%)
Mortality rate after complication 10%(8.2 to 12.0%)
Cost of complication $10,000
Costs associated with fatal complication
$50,000
Input Variables and Values for Treatment A and B
Outcome Treatment A
Treatment B
Mean life expectancy 18.72 years 18.40 years
Mean costs $10,940 $10,725
Mortality from complications of treatment
2.0% 0.5%
Expected Values of Outcomes of Interest Given
Treatment A or B
Outcomes from 10 Simulations of Treatment A and B Assuming a WTP
Threshold of $750Simulation Number
Net Benefits Treatment A
Net Benefits Treatment B
Maximum Net Benefits
Preferred Strategy
Opportunity Cost
1 $4,180 $4,306 $4,306 B $0
2 $2,273 $2,415 $2,415 B $0
3 $7,095 $4,507 $7,095 A $2,588
4 $3,186 $4,017 $4,017 B $0
5 $3,504 $3,433 $3,504 A $72
6 $5,698 $6,740 $6,740 B $0
7 $4,762 $3,718 $4,762 A $1,044
8 $3,960 $1,919 $3,960 A $2,041
9 $5,071 $5,964 $5,964 B $0
10 $1,904 $5,123 $5,123 B $0
Expected value (mean of simulations 1-10)
$4,163 $4,214 $4,789 $575
An approach to research prioritization which uses Bayesian methods to estimate the potential benefits of gathering further information (through more research) before making a decision (Meyers, et al., 2012)
A tool of the decision sciences, estimates the potential economic losses associated [with] choosing suboptimal policies when that decision is made with uncertain information (Rhein, 2012)
A decision analytic technique that explicitly evaluates the benefit of collecting additional information to reduce or eliminate uncertainty. (Yokota and Thompson, 2004)
Value of Information
General Challenges to the Use of VOI for Research
Prioritization Resources needed to develop appropriate models
Personnel Time Computing Resources
Scope of VOI Prioritizing across disease areas Prioritizing within a condition Prioritizing specific comparative effectiveness
research Stakeholder Engagement
Lack of familiarity with the methodology Timing of VOI
Personnel Time Computing Resources
“minimal modeling appears to be most useful when an intervention affects quality of life alone” 3
Resources Needed to Conduct VOI
Difference between the expected value given perfect information and the value given current information
i.e. the upper bound of the opportunity cost of making a wrong decision; any effort to improve the quality of available data that costs less than the EVPI is worth pursuing
Expected Value of Perfect Information (EVPI)
Calculating EVPI
EVPI is calculated as the expected value of a decision made with perfect information minus the expected value of a decision made with the existing information:
EVPI = EmaxjNB(j,) - maxjE NB(j,)
Go/no go measurement to determine if additional research is appropriate
Comparison for cost effectiveness of research across different interventions
Using EVPI for Research Prioritization
The value of being able to know the outcome of another related uncertainty instead of the original value itself before making a decision. It is quantified as the highest price the decision-maker is willing to pay for being able to know the uncertainty before making a decision. Note that it is essentially the value of perfect information on the second variable.
Expected Value of Partial Perfect Information (EVPPI)
Using VOI by Research Sponsor, such as PCORI
Perform VOI prior to solicitation for research proposals and only request areas identified as having acceptable EVPI/EVPPI
Perform VOI after initial review of proposals as part of funding decisions with higher EVPI/EVPPI receiving higher priority for funding
Research sponsor could encourage or require investigators to include VOI analysis as part of rationale for proposals
Treatment Decision Tree
Justify additional research in advance of adapting specific interventions
Identifying which areas contribute the most to uncertainty
Common uses of VOI
Patient-Centered Outcomes Research Institute Board of Governors Meeting, www.pcori.org, 2012.
Minimal Modeling Approaches to Value of Information Analysis for Health Research, Metlzer, et al. www.ahrq.gov, 2011.
Systematizing the Use of Value of Information Analysis in Prioritizing Systematic Reviews. Hoomans, et al. www.ahrq.gov, 2012.
Value of Information and Research Prioritization. Rein. NORC at the U of Chicago, 2012.
Value of Information Literature Analysis: A Review of Applications in Health Risk Management. Yokota and Thompson, MDM, 2004.
Value of Information on Preference Heterogeneity and Individualized Care, Basu and Meltzer, MDM, 2007.
Value-of-Information Analysis for Patient-Centered Outcomes Research Prioritization, Myers et al., www.pcori.org, 2012.
References
Thank You!
Contact us at: [email protected] or (310) 267-4258