cadth 2015 a4 15.04.04 workshop mike paulden
TRANSCRIPT
Transforming the cost-effectiveness threshold into a ‘value threshold’
Initial findings from a simulation modelMike Paulden and Christopher McCabe
Problem• The conventional cost-effectiveness (CE) threshold represents “an
estimate of health forgone as other [services] are displaced to
accommodate the additional costs of new technologies”
(Claxton et al. 2013)
• Plotted as a straight line on the CE plane (Drummond et al. 2005)
• Numerous limitations and assumptions:
• Assumes constant marginal returns and divisibility of technologies
• No account for aspects of ‘value’ beyond those considered by the QALY
• Impact of imperfect information is not explicitly considered, nor the possibility
that new interventions represent net disinvestments
• No account for multiple decision makers with conflicting objectives
• Recently, NICE has applied ‘modifiers’ to its baseline threshold to
account for aspects of ‘value’ beyond the QALY (NICE 2009, 2014)
• Resulted in inconsistencies in NICE’s methodology (Paulden et al. 2014)
Objective
• Our objective is to transform the conventional CE threshold
into a ‘value threshold’ of greater use to decision makers
• In doing so we aim to address the limitations previously
described
• As a first step we have developed a simulation model in order to
understand how a ‘value threshold’ may differ from a CE threshold
• Of key interest are the implications of:
i. Relaxing conventional assumptions such as constant marginal
returns to scale and perfect divisibility of technologies
ii. Incorporating imperfect information and ‘value’ considerations within
a complex health system with multiple decision makers
iii. Extending the threshold so that it may be used for net disinvestments
Pool of initial technologiesThe cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a ‘value’ attribute and a specific health production function ‘shape’ (applies
only if marginal returns are diminishing)
Model schematic
Pool of initial technologiesThe cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a ‘value’ attribute and a specific health production function ‘shape’ (applies
only if marginal returns are diminishing)
Divisibility of technologiesTechnologies in the pool are either
all divisible or all indivisible
Marginal returns to scaleTechnologies in the pool either all have constant marginal returns to
scale or all have diminishing marginal returns to scale
Model schematic
Pool of initial technologiesThe cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a ‘value’ attribute and a specific health production function ‘shape’ (applies
only if marginal returns are diminishing)
Initial budgetUpon the establishment of the health system, an initial budget
is assigned for purchasing technologies from the pool
Divisibility of technologiesTechnologies in the pool are either
all divisible or all indivisible
Marginal returns to scaleTechnologies in the pool either all have constant marginal returns to
scale or all have diminishing marginal returns to scale
Model schematic
Pool of initial technologiesThe cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a ‘value’ attribute and a specific health production function ‘shape’ (applies
only if marginal returns are diminishing)
Initial allocator
Imperfect informationEach decision maker has one of
four levels of information regarding the effectiveness of technologies:
none, poor, good, or perfect
Other value considerationsEach decision maker assigns one
of four possible weights to ‘value’ considerations beyond the QALY:
none, small, medium, or large
Initial budgetUpon the establishment of the health system, an initial budget
is assigned for purchasing technologies from the pool
1. The initial allocator purchases technologies from the pool until the
initial budget is exhausted
Divisibility of technologiesTechnologies in the pool are either
all divisible or all indivisible
Marginal returns to scaleTechnologies in the pool either all have constant marginal returns to
scale or all have diminishing marginal returns to scale
Model schematic
Pool of initial technologiesThe cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a ‘value’ attribute and a specific health production function ‘shape’ (applies
only if marginal returns are diminishing)
New interventionEach new intervention represents either a net investment or net disinvestment
Net investments impose costs on the health system, requiring that resources
be released from other technologies
Net disinvestments release resources, allowing these to be spend on other
technologies from the pool
Initial allocator
Imperfect informationEach decision maker has one of
four levels of information regarding the effectiveness of technologies:
none, poor, good, or perfect
Other value considerationsEach decision maker assigns one
of four possible weights to ‘value’ considerations beyond the QALY:
none, small, medium, or large
Initial budgetUpon the establishment of the health system, an initial budget
is assigned for purchasing technologies from the pool
1. The initial allocator purchases technologies from the pool until the
initial budget is exhausted
Divisibility of technologiesTechnologies in the pool are either
all divisible or all indivisible
Marginal returns to scaleTechnologies in the pool either all have constant marginal returns to
scale or all have diminishing marginal returns to scale
Model schematic
Pool of initial technologiesThe cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a ‘value’ attribute and a specific health production function ‘shape’ (applies
only if marginal returns are diminishing)
Initial allocator Agent
Imperfect informationEach decision maker has one of
four levels of information regarding the effectiveness of technologies:
none, poor, good, or perfect
Other value considerationsEach decision maker assigns one
of four possible weights to ‘value’ considerations beyond the QALY:
none, small, medium, or large
Initial budgetUpon the establishment of the health system, an initial budget
is assigned for purchasing technologies from the pool
1. The initial allocator purchases technologies from the pool until the
initial budget is exhausted
Value thresholdUsed by the agent to determine whether or not to recommend
the new intervention
2. The agent recommends the new intervention if its
expected value exceeds the agent’s value threshold
Divisibility of technologiesTechnologies in the pool are either
all divisible or all indivisible
Marginal returns to scaleTechnologies in the pool either all have constant marginal returns to
scale or all have diminishing marginal returns to scale
Model schematicNew intervention
Each new intervention represents either a net investment or net disinvestment
Net investments impose costs on the health system, requiring that resources
be released from other technologies
Net disinvestments release resources, allowing these to be spend on other
technologies from the pool
Pool of initial technologiesThe cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a ‘value’ attribute and a specific health production function ‘shape’ (applies
only if marginal returns are diminishing)
Initial allocator Reallocator Agent
Imperfect informationEach decision maker has one of
four levels of information regarding the effectiveness of technologies:
none, poor, good, or perfect
Other value considerationsEach decision maker assigns one
of four possible weights to ‘value’ considerations beyond the QALY:
none, small, medium, or large
Initial budgetUpon the establishment of the health system, an initial budget
is assigned for purchasing technologies from the pool
1. The initial allocator purchases technologies from the pool until the
initial budget is exhausted
Value thresholdUsed by the agent to determine whether or not to recommend
the new intervention
2. The agent recommends the new intervention if its
expected value exceeds the agent’s value threshold
Divisibility of technologiesTechnologies in the pool are either
all divisible or all indivisible
Marginal returns to scaleTechnologies in the pool either all have constant marginal returns to
scale or all have diminishing marginal returns to scale
Model schematicNew intervention
Each new intervention represents either a net investment or net disinvestment
Net investments impose costs on the health system, requiring that resources
be released from other technologies
Net disinvestments release resources, allowing these to be spend on other
technologies from the pool
Pool of initial technologiesThe cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a ‘value’ attribute and a specific health production function ‘shape’ (applies
only if marginal returns are diminishing)
Initial allocator Reallocator Agent
Imperfect informationEach decision maker has one of
four levels of information regarding the effectiveness of technologies:
none, poor, good, or perfect
Other value considerationsEach decision maker assigns one
of four possible weights to ‘value’ considerations beyond the QALY:
none, small, medium, or large
Initial budgetUpon the establishment of the health system, an initial budget
is assigned for purchasing technologies from the pool
1. The initial allocator purchases technologies from the pool until the
initial budget is exhausted
Value thresholdUsed by the agent to determine whether or not to recommend
the new intervention
3. If the agent recommends a net investment, the reallocator must contract adopted NE/NW technologies and/or expand non-exhausted SE/SW technologies. Alternatively, if the agent recommends a net disinvestment, the reallocator may
expand non-exhausted NE technologies and/or contract adopted SW technologies
Divisibility of technologiesTechnologies in the pool are either
all divisible or all indivisible
Marginal returns to scaleTechnologies in the pool either all have constant marginal returns to
scale or all have diminishing marginal returns to scale
Model schematic
2. The agent recommends the new intervention if its
expected value exceeds the agent’s value threshold
New interventionEach new intervention represents either a net investment or net disinvestment
Net investments impose costs on the health system, requiring that resources
be released from other technologies
Net disinvestments release resources, allowing these to be spend on other
technologies from the pool
Pool of initial technologiesThe cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a ‘value’ attribute and a specific health production function ‘shape’ (applies
only if marginal returns are diminishing)
Initial allocator Reallocator Agent
Imperfect informationEach decision maker has one of
four levels of information regarding the effectiveness of technologies:
none, poor, good, or perfect
Other value considerationsEach decision maker assigns one
of four possible weights to ‘value’ considerations beyond the QALY:
none, small, medium, or large
Initial budgetUpon the establishment of the health system, an initial budget
is assigned for purchasing technologies from the pool
1. The initial allocator purchases technologies from the pool until the
initial budget is exhausted
Value thresholdUsed by the agent to determine whether or not to recommend
the new intervention
Divisibility of technologiesTechnologies in the pool are either
all divisible or all indivisible
Marginal returns to scaleTechnologies in the pool either all have constant marginal returns to
scale or all have diminishing marginal returns to scale
Agent’s authorityAgent may have mandate to
consider reallocation and/or an alternative to the intervention
Model schematic
3. If the agent recommends a net investment, the reallocator must contract adopted NE/NW technologies and/or expand non-exhausted SE/SW technologies. Alternatively, if the agent recommends a net disinvestment, the reallocator may
expand non-exhausted NE technologies and/or contract adopted SW technologies
2. The agent recommends the new intervention if its
expected value exceeds the agent’s value threshold
New interventionEach new intervention represents either a net investment or net disinvestment
Net investments impose costs on the health system, requiring that resources
be released from other technologies
Net disinvestments release resources, allowing these to be spend on other
technologies from the pool
Pool of initial technologiesThe cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a ‘value’ attribute and a specific health production function ‘shape’ (applies
only if marginal returns are diminishing)
Initial allocator Reallocator Agent
Imperfect informationEach decision maker has one of
four levels of information regarding the effectiveness of technologies:
none, poor, good, or perfect
Other value considerationsEach decision maker assigns one
of four possible weights to ‘value’ considerations beyond the QALY:
none, small, medium, or large
Initial budgetUpon the establishment of the health system, an initial budget
is assigned for purchasing technologies from the pool
1. The initial allocator purchases technologies from the pool until the
initial budget is exhausted
Value thresholdUsed by the agent to determine whether or not to recommend
the new intervention
4. Prior to making its recommendation, the agent places its own valuations on both the new intervention and the reallocator’s preferred reallocation. If the agent has the authority to mandate a reallocation and/or propose an alternative to the new intervention then it also places a valuation upon this. The optimal value threshold is that which ensures that a new intervention is only recommended if doing so maximizes the expected value to the agent
Divisibility of technologiesTechnologies in the pool are either
all divisible or all indivisible
Marginal returns to scaleTechnologies in the pool either all have constant marginal returns to
scale or all have diminishing marginal returns to scale
Agent’s authorityAgent may have mandate to
consider reallocation and/or an alternative to the intervention
Model schematic
3. If the agent recommends a net investment, the reallocator must contract adopted NE/NW technologies and/or expand non-exhausted SE/SW technologies. Alternatively, if the agent recommends a net disinvestment, the reallocator may
expand non-exhausted NE technologies and/or contract adopted SW technologies
2. The agent recommends the new intervention if its
expected value exceeds the agent’s value threshold
New interventionEach new intervention represents either a net investment or net disinvestment
Net investments impose costs on the health system, requiring that resources
be released from other technologies
Net disinvestments release resources, allowing these to be spend on other
technologies from the pool
Conventional Assumptions
-$50m
-$40m
-$30m
-$20m
-$10m
$0m
$10m
$20m
$30m
$40m
$50m
-2,000 -1,000 0 1,000 2,000
Expe
nditu
re o
n ne
w te
chno
logy
Value of new technology (QALY equivalents)
Lowerbudget
Higherbudget
Lowerbudget
Higherbudget
Threshold'kinks'
All decision makers haveperfect information
All decision makers placeno weight on other
‘value’ considerations
Technologies are divisible
Technologies exhibit constant returns to scale
Agent cannot reallocate
-$50m
-$40m
-$30m
-$20m
-$10m
$0m
$10m
$20m
$30m
$40m
$50m
-2,000 -1,000 0 1,000 2,000
Expe
nditu
re o
n ne
w te
chno
logy
Value of new technology (QALY equivalents)
Lowerbudget
Higherbudget
Lowerbudget
Higherbudget
All decision makers haveperfect information
All decision makers placeno weight on other
‘value’ considerations
Technologies are divisible
Technologies exhibit diminishing returns to scale
Agent cannot reallocate
Diminishing Returns to Scale
-$50m
-$40m
-$30m
-$20m
-$10m
$0m
$10m
$20m
$30m
$40m
$50m
-2,000 -1,000 0 1,000 2,000
Expe
nditu
re o
n ne
w te
chno
logy
Value of new technology (QALY equivalents)
Lowerbudget
Higherbudget
Lowerbudget
Higherbudget
(overlap)
All decision makers haveperfect information
All decision makers placeno weight on other
‘value’ considerations
Technologies are indivisible
Returns to scale irrelevant if technologies are indivisible
Agent cannot reallocate
Indivisible Technologies
-$50m
-$40m
-$30m
-$20m
-$10m
$0m
$10m
$20m
$30m
$40m
$50m
-2,000 -1,000 0 1,000 2,000
Expe
nditu
re o
n ne
w te
chno
logy
Value of new technology (QALY equivalents)
Lowerbudget
Higherbudget
Lowerbudget Higher
budget
Threshold'kinks'
Reallocator and agent haveperfect information
and initial allocator haspoor information
Reallocator and agentplace small weight and initial allocator placeslarge weight on other ‘value’ considerations
Technologies are divisible
Technologies exhibit diminishing returns to scale
Agent cannot reallocate
Imperfect Information andOther ‘Value’ Considerations
-$50m
-$40m
-$30m
-$20m
-$10m
$0m
$10m
$20m
$30m
$40m
$50m
-2,000 -1,000 0 1,000 2,000
Expe
nditu
re o
n ne
w te
chno
logy
Value of new technology (QALY equivalents)
Lowerbudget
Higherbudget
Lowerbudget
Higherbudget
Threshold'kinks'
Initial allocator and agent have perfect information
and reallocator haspoor information
Initial allocator and agentplace small weight and
reallocator placeslarge weight on other ‘value’ considerations
Technologies are divisible
Technologies exhibit diminishing returns to scale
Agent cannot reallocate
-$50m
-$40m
-$30m
-$20m
-$10m
$0m
$10m
$20m
$30m
$40m
$50m
-2,000 -1,000 0 1,000 2,000
Expe
nditu
re o
n ne
w te
chno
logy
Value of new technology (QALY equivalents)
Lowerbudget
Higherbudget
Lowerbudget
Higherbudget
Threshold'kink'
Initial allocator and agent have perfect information
and reallocator haspoor information
Initial allocator and agentplace small weight and
reallocator placeslarge weight on other ‘value’ considerations
Technologies are divisible
Technologies exhibit diminishing returns to scale
Agent can reallocate
Agent Has Authority to Reallocate
Conclusions
• The conventional ‘CE threshold’ model is merely a special case
among many approaches for determining a value threshold
• Departing from this special case allows for consideration of:
• Differences in the information available to, the values held by, and the
objectives pursued by, multiple interacting decision makers
• The specific value characteristics of each technology
• This has potentially significant implications for the appropriate
specification of value thresholds used for decision making
• Our findings provide insights for future theoretical work, as well
as a rich source of potential hypotheses for researchers
conducting empirical research in this area
Questions
1. Why should value considerations be accounted for within the
threshold used for CE analysis? Isn’t it sufficient to simply apply
weights to new technologies or to consider ‘values’ separately?
2. Why might differences in information, values and objectives
across multiple interacting decision makers result in:
a) Different thresholds for net investments and net disinvestments?
b) Thresholds that cross into the SE and NW quadrants of the CE plane?
3. Why is the threshold dependent upon the agent’s authority?
Are there any implications for the recommendations made by
CADTH or for the decisions of Canadian policy makers who
depend upon CADTH’s guidance?
References• Claxton et al. (2013). Methods for the Estimation of the NICE Cost
Effectiveness Threshold. CHE Research Paper 81. York: University of
York.
• Drummond et al. (2005). Methods for the Economic Evaluation of Health
Care Programmes. Third Edition. Oxford: Oxford University Press.
• Sendi et al. (2002). Opportunity costs and uncertainty in the economic
evaluation of health care interventions. Health Economics, 11(1), 23–31.
• National Institute for Health and Care Excellence (2009). Appraising life-
extending, end of life treatments. London: NICE.
• National Institute for Health and Care Excellence (2014). Consultation
Paper: Value Based Assessment of Health Technologies. London: NICE.
• Paulden et al. (2014). Some Inconsistencies in NICE’s Consideration of
Social Values. PharmacoEconomics. November 2014, 32(11), 1043-1053.