Value of Information Calculations to Inform and Prioritize

Clinical Research Investments

David Meltzer MD, PhDThe University of Chicago

Overview

• Cost-effectiveness analysis has long been used to assess the value of medical treatments and the information that comes from diagnostic tests– Cost-effectiveness measured in Cost/QALY– Net health benefits

= gain in QALYs – opportunity cost of spending in QALYs– Net monetary benefit

= $ value of improved health - costs• Newer value of information techniques have extended

these tools to assess the value of medical research

Research as Value of Information: Analogy to Diagnostic Testing

Test

Don’t Test

S

H

S

H

Max{pU(T|S)+(1-p)U(T|H), pU(N|S)+(1-p)U(N|H)}

U(T|S)

U(N|H)

pU(T|S)+(1-p)U(N|H)

Value of Information Approach to Value of Research

• Without information– Make best compromise choice not knowing true state of the

world (e.g. don’t know if intervention is good, bad)• With probability p: get V(Compromise|G)• With probability 1-p: get V(Compromise|B)

• With information– Make best decision knowing true state

• With probability p: get V(Best choice|G)• With probability 1-p: get V(Best choice|B)

• Value of information = E(outcome) with information - E(outcome) w/o information

= {p*V(Best choice|G) + (1-p)*V(Best choice|B)} - {p*V(Compromise|G) + (1-p)*V(Compromise|B)}

= Value of Research

Practical Applications of Value of Information

• VOI requires modeling population value of information

where

• VOI based on decision models– IVOI modeled with decision model– UK (NICE): Alzheimer’s Disease Tx, wisdom teeth removal

• Minimal modeling approaches to VOI– IVOI comes (nearly) directly from clinical trial– US (NIH): CATIE Trial of atypical antipsychotics

• Bound with more limited data (burden of illness)

( ) ( )tt

t

VOI D t I t N IVOI ( )

( )

in

t

t

is time preference discount factor

D t is depeciation of knowledge over time

I t is extent of implementation

N is number of eligible individuals each cohort

IVOI is individual VOI

“Bayesian Value of information analysis: An application to a policy model of Alzheimer's disease.”

Uncertainty in Incremental Net Benefits

Cost-Effectiveness Acceptability Curve

Value of Research by Value of Health

Value of Research by Time Horizon

Contributors to Value of Research

Practical Applications of Value of Information

• VOI requires modeling population value of information

where

• VOI based on decision models– IVOI modeled with decision model– UK (NICE): Alzheimer’s Disease Tx, wisdom teeth removal

• Minimal modeling approaches to VOI– IVOI comes (nearly) directly from clinical trial– US (NIH): CATIE Trial of atypical antipsychotics

• Bound with more limited data (burden of illness)

( ) ( )tt

t

VOI D t I t N IVOI ( )

( )

in

t

t

is time preference discount factor

D t is depeciation of knowledge over time

I t is extent of implementation

N is number of eligible individuals each cohort

IVOI is individual VOI

Expected Value of Researchon the Comparative Cost-Effectiveness

of Antipsychotics Drugs

David Meltzer MD PhD

Department of Medicine, Department of Economics, Harris School of Public Policy, &

University of Chicago CERTChicago IL

(Joint work with Anirban Basu PhD, University of Chicago & Dr. Herbert Y. Meltzer, Vanderbilt University)

The Clinical Antipsychotic Trials in Intervention Effectivness (CATIE)

• $42.6 million, NIMH-funded randomized trial of Atypical Antipsychotic Drugs (A-APDs) and a Neuroleptic (Perhphenazine) in patients with established schizophrenia

• Major findingso Discontinuation rates similar with A-APDs and Perphenazineo Perphenazine cost-effective first-line treatment

• Impacto Frequently discussed in coverage decisionso Some have argued results should be considered definitive

The Clinical Antipsychotic Trials in Intervention Effectivness (CATIE)

• Limitationso Continuation was major endpointo Limited precision in estimates of effectiveness, costso Small differences in effectiveness/costs across many

persons could be of great value

• Important to know value of potential future researcho Help prioritize individual research opportunitieso Facilitate rational investment decisions

CATIE Cost-Effectiveness Results

Monthly Costs Mean (sd) ($)

QALY Mean (sd)

ICER ($/QALY)

Perphenazine 817 ( 728) 0.722 (0.0064) -

Olanzapine 1619 (1442) 0.723 (0.0063) 9,624,000

Risperidone 1635 (1457) 0.706 (0.0066) Dominated

Quetiapine 1680 (1497) 0.721 (0.0065) Dominated

(Ref: Rosenheck et al , 2006; Private Communications with Dr. Rosenheck)

Only statistically significant difference:

QALYPerphenazine > QALYRisperidone (p-val < 0.001)

Aims

Primary Aim: To determine the expected value of more precise determination of effects of AAPDs and Perphenazine on costs and QALYs.

Secondary Aim: To determine the optimal sample size for a future trial of the effects of AAPDs and Perphenazine on costs and QALYs

Methods Used for Value of Research

Expected value calculated based on the welfare of the prevalent cohort over their lifetimes and the welfare of next 20 incident cohorts over their lifetimes

3% discounting was used

Simulated Distribution of Mean QALYS(Based on uncertainty around CATIE results)

Den

sity

.65 .7 .75 .8 .85E(QALY)/per patient per year

Olanzapine: 0.723 (0.0063)Quetiapine: 0.721 (0.0065) Risperidone: 0.706 (0.0066) Perphenazine: 0.722 (0.0064)

Simulated Distribution of Mean Costs(Based on uncertainty around CATIE results)

Den

sity

0 5000 10000 15000E(QALY)/per patient per year

Olanzapine: $1606 (1421)Quetiapine: $1685 (1485) Risperidone: $1621 (1439) Perphenazine: $ 810 ( 723)

Realizations of Value of Research Over Time

02

46

81

01

2V

alu

e (

in B

illio

n $

)

2007 2017 2027 2037 2047 2057 2067 2077 2087YEAR

Incident in 2012-2036Incident in 2011Incident in 2010Incident in 2009

Incident in 2008Incident in 2007Prevalent Cohort

Value of Future Research to Prevalent and Incident Cohortsat $50k/QALY

Total Value to Prevalent Cohort: $207 billionTotal Value to Each Incident Cohort: $6.6 billion

Total Value to Prevalent & Next 20 Incident Cohorts: $342 billion

Value of Research and Acceptability Profile

.5.5

2.5

4.5

6.5

8.6

Pr

300

320

340

360

380

400

Va

lue

(in b

ilio

n $)

10000 30000 50000 70000 90000 110000 130000 150000Marginal WTP for a QALY

Max VofRPr(CATIE decision is incorrect)

Optimal Sample Size for A Future Trial• Traditional (Deterministic) Power Calculations (at $50K/QALY)

• Largest effect size in NMB between an atypical & perphenazine based on CATIE results: $15,680 (sd=$315,000) vs. $26,296 (sd=$140,000)

• Sample size required for alpha = 0.05 & power = 0.80: 8,300 for each arm

• Power associated with n of CATIE = 400/arm & alpha = 0.05: 10%

Net Expected Value of Sample Information(at $50K, $100K and $150K/QALY)

|

|

|

|

|

||v3

003

253

50E

VS

I (in

Bill

ion

s)

5000 10000 15000 20000 25000 30000 35000 40000 45000 50000Sample size for each arm

at $50K/QALYat $100K/QALY

at $150K/QALY

Cost of Research: $3 mill + (sample size*4)*($5000/month)*18 months

Optimal sample size for each arm = 22,500

CATIE VOI Conclusions

• The value to more precisely establishing the cost-effectiveness of typical/atypical antipsychotics is enormous.

• The results of CATIE should not be viewed as definitive.

• Further studies of the comparative cost-effectiveness of typical/atypical antipsychotics with adequate sample size to answer such questions have high expected value.

• Optimal sample sizes may be exceptionally large, raising interesting questions as to how clinical trials on such a scale might be executed.

• Large scale social experiments may provide an interesting model for such studies.

Conclusions• Value of information (VOI) analysis can be used to develop

prospective estimates of the value of research• VOI can be used to prioritize areas of clinical research or choose

among study designs• Decision models or minimal modeling approaches

– Additional minimal model applications in progress– Erlotinib (Tarceva) (+gemcitabine) in advanced pancreatic CA– Azithromycin vs. Augmentin in acute sinusitis

• Incorporating VOI into research prioritization is a work in progress– Use by funding agencies?– Use by investigators?

Acknowledgments

• Collaborators: Anirban Basu Ph.D., Jeanette Chung Ph.D., Ties Hoomans Ph.D., Herbert Meltzer M.D.

• Funding: AHRQ BCBS Evidence-Based Practice Center, AHRQ Hospital Medicine and Economics Center for Education and Research in Therapeutics, Best Practice Inc, National Institute of Aging, National Institute of Mental Health