preference measurement in cea: are we capturing values or creating them?

Preference measurement in CEA: Preference measurement in CEA: Are we capturing values or Are we capturing values or

creating them?creating them?

Peter A. Ubel, M.D. Peter A. Ubel, M.D.

Program for Improving Health Care Decisions Program for Improving Health Care Decisions

Ann Arbor VAMC Ann Arbor VAMC

University of Michigan Health System University of Michigan Health System

A Policy DilemmaA Policy Dilemma Imagine Medicaid program is Imagine Medicaid program is

choosing a colon cancer screening choosing a colon cancer screening testtest

Test #1Test #1 Inexpensive - can offer to everyoneInexpensive - can offer to everyone Less effective - saves 1,000 livesLess effective - saves 1,000 lives

Test #2Test #2 More expensive - can offer to 1/2 of More expensive - can offer to 1/2 of

the peoplethe people More effective – saves 1,100 livesMore effective – saves 1,100 lives

Which test would you choose?Which test would you choose?

If choosing according to CEAIf choosing according to CEA

You would choose Test #2You would choose Test #2– The one that saves 1,100 livesThe one that saves 1,100 lives

CEA helps identify health care CEA helps identify health care interventionsinterventions– That maximizeThat maximize– The average health of a populationThe average health of a population

What do people actually choose?What do people actually choose? Test #1 (saving 1,000 lives) chosen byTest #1 (saving 1,000 lives) chosen by

55% of general public55% of general public 55% of medical ethicists55% of medical ethicists 45% of CEA experts45% of CEA experts

In This TalkIn This Talk

Present evidence demonstrating that CEA Present evidence demonstrating that CEA does not capture people’s allocation does not capture people’s allocation preferencespreferences

Show that people’s allocation preferencesShow that people’s allocation preferences– While at odds with CEAWhile at odds with CEA– Are often at odds with themselvesAre often at odds with themselves

» Internally inconsistentInternally inconsistent» Susceptible to irrelevant informationSusceptible to irrelevant information» Occasionally downright confusedOccasionally downright confused

Another Another DilemmaDilemma

A transplant allocation choiceA transplant allocation choice Imagine there is a blood test that predicts the outcome of Imagine there is a blood test that predicts the outcome of

liver transplantliver transplant 200 patients200 patients

Divided into two groups based on prognosisDivided into two groups based on prognosis Only 100 organs availableOnly 100 organs available

Subjects received one of five allocation choicesSubjects received one of five allocation choices 80% vs. 70%80% vs. 70% 80% vs. 50%80% vs. 50% 80% vs. 20%80% vs. 20% 40% vs. 25%40% vs. 25% 40% vs. 10%40% vs. 10%

Allocation decisionsAllocation decisions

% organs to better

prognostic group 80/70 80/50 80/20 40/25 40/10 Total

<50

50

51-75

76-99

100


% organs to better


<50 3 0 0 9 3 3

50 53 33 26 40 14 33

51-75 22 27 21 14 29 22

76-99 9 6 29 11 37 19

100 13 33 24 26 17 22


% organs to better


<50 3 0 0 9 3 3

50 53 33 26 40 14 33

51-75 22 27 21 14 29 22

76-99 9 6 29 11 37 19

100 13 33 24 26 17 22 100 13 33 24 26 17 22


% organs to better


<50 3 0 0 9 3 3

50 53 33 26 40 14 33

51-75 22 27 21 14 29 22

76-99 9 6 29 11 37 19

100 13 33 24 26 17 22

50 53 33 26 40 14 33

What do these two studies prove?What do these two studies prove? Colon cancer studyColon cancer study

– That the public values That the public values equityequity in allocating in allocating resourcesresources

– Disagree with CEA’s emphasis on Disagree with CEA’s emphasis on efficiencyefficiency Transplant studyTransplant study

– Preference for equity over efficiency is not Preference for equity over efficiency is not absoluteabsolute

» ““Consensus view” strikes a balanceConsensus view” strikes a balance– People’s preferences seem really damn People’s preferences seem really damn

reasonablereasonable» When more to gain from efficiency, more preference for itWhen more to gain from efficiency, more preference for it

What values should What values should guide allocation guide allocation

decisions?decisions?

1. Priority for treating severely 1. Priority for treating severely ill patientsill patients

Imagine an illness A that causes severe Imagine an illness A that causes severe health problemshealth problems treatment will help patients a littletreatment will help patients a little

Imagine illness B that causes moderate Imagine illness B that causes moderate problemsproblems treatment will help patients considerablytreatment will help patients considerably

The cost of treatment is the same in The cost of treatment is the same in both casesboth cases

What do you believe?What do you believe?

Most funding should be allocated to illness A, Most funding should be allocated to illness A, involving severe health problems that improve involving severe health problems that improve a littlea little

Most funding should be allocated to illness B, Most funding should be allocated to illness B, involving moderate health problems that involving moderate health problems that improve considerablyimprove considerably

Severity Study - Treatment Severity Study - Treatment ChoicesChoices

Most funding should be allocated to Most funding should be allocated to illness A, involving severe health illness A, involving severe health problems that improve a littleproblems that improve a little 4040

Most funding should be allocated to illness B, Most funding should be allocated to illness B, involving moderate health problems that involving moderate health problems that improve considerably improve considerably 6060

2. Avoiding Discrimination 2. Avoiding Discrimination Against People With DisabilitiesAgainst People With Disabilities

1

2

3

4

5

6

7Death

Disability levelFull health

Death

Paraplegia

How Valuable is Life with Paraplegia?How Valuable is Life with Paraplegia?

Asked PeopleAsked PeopleHow many lives of people with paraplegia How many lives of people with paraplegia

would need to be savedwould need to be savedTo be equally beneficial as saving 100 To be equally beneficial as saving 100

lives of people who could be returned to lives of people who could be returned to perfect health perfect health

65% of people said the number should be 65% of people said the number should be 100100

3. Age discrimination is OK3. Age discrimination is OK

Allocating 100 transplantable livers Allocating 100 transplantable livers amongamong– 100 35 year olds100 35 year olds– 100 65 year olds100 65 year olds

Even distributionEven distribution– Favored by 40% of peopleFavored by 40% of people

Priority to younger patientsPriority to younger patients– Favored by 57%Favored by 57%

Fine-tuning CEAFine-tuning CEA

Tweaking the numbersTweaking the numbers Proposals to adjust numerical weights in Proposals to adjust numerical weights in

CEA to account for people’s valuesCEA to account for people’s values– Nord: severity weightsNord: severity weights– Williams: age-adjustment of QALYsWilliams: age-adjustment of QALYs

Basic ideaBasic idea– Current measure:Current measure: QALY = PQALY = P11 U U11 + P + P2 2 UU22 +… +…

Revised measure: QALY = PRevised measure: QALY = P11 U U11 X X11 + P + P22 UU22 X X22 +… +…

Assumptions underlying the fine-Assumptions underlying the fine-tuning proposalstuning proposals

Public has stable allocation preferencesPublic has stable allocation preferences These preferences are quantifiableThese preferences are quantifiable CEA is amenable, methodologically, to CEA is amenable, methodologically, to

incorporating these preferencesincorporating these preferences

It’s time to shatter It’s time to shatter some assumptionssome assumptions

1. People Get Confused1. People Get Confused

Reminder of Transplant Reminder of Transplant Allocation decisionsAllocation decisions

% organs to better


<50 3 0 0 9 3 3

50 53 33 26 40 14 33

51-75 22 27 21 14 29 22

76-99 9 6 29 11 37 19

100 13 33 24 26 17 22

Do people understand their Do people understand their allocation choices?allocation choices?

After people made their allocation choices, After people made their allocation choices, we asked them we asked them – What distribution of organs would maximize What distribution of organs would maximize

survivalsurvival

In all cases, correct answer = all 100 organs In all cases, correct answer = all 100 organs to group with better prognosisto group with better prognosis

Majority of people did not give correct answerMajority of people did not give correct answer– E.g. 80/20 group E.g. 80/20 group → 80/20 distribution→ 80/20 distribution

Do confused people make different Do confused people make different allocation choices?allocation choices?

Percent of subjects making choicePercent of subjects making choice

Allocation choice

Those who understood

(n=71)

Those who did not understand

(n=96)

<50 0

50 14

51-75 18

76-99 18

100 49

3

48

26

20

3

Even when people aren’t confused . . .Even when people aren’t confused . . .

Suppose 200 transplant candidates can Suppose 200 transplant candidates can be ranked from 1 – 200be ranked from 1 – 200– By prognosisBy prognosis– Based on a blood testBased on a blood test

Would you give organs to top 100 Would you give organs to top 100 patients?patients?– Majority say yes!Majority say yes!

2. People hate saying no to a whole 2. People hate saying no to a whole group of patientsgroup of patients

Blood test ranks people 1 – 200Blood test ranks people 1 – 200– Okey DokeyOkey Dokey

Blood test divides patients into two Blood test divides patients into two groupsgroups– No one wants to abandon second groupNo one wants to abandon second group

What if you could ignore blood test?What if you could ignore blood test? 41% would choose to do so!41% would choose to do so!

3. People like “easy outs”3. People like “easy outs” Reminder of “severity study”Reminder of “severity study”

Most funding should be allocated to illness A, Most funding should be allocated to illness A, involving severe health problems that improve involving severe health problems that improve a littlea little 4040

Most funding should be allocated to illness B, Most funding should be allocated to illness B, involving moderate health problems that involving moderate health problems that improve considerably improve considerably 6060

People like “easy outs”People like “easy outs” Reminder of “severity study”Reminder of “severity study”

Most funding should be allocated to illness A, Most funding should be allocated to illness A, involving severe health problems that improve a involving severe health problems that improve a littlelittle

Most funding should be allocated to illness B, Most funding should be allocated to illness B, involving moderate health problems that improve involving moderate health problems that improve considerably considerably

Equal $ to A and BEqual $ to A and B

10

15

75

4. People often refuse to make tradeoffs4. People often refuse to make tradeoffs

The Person Tradeoff (PTO) preference The Person Tradeoff (PTO) preference measuremeasure

to be equallygood as

Curing 100 people with severe shortness of breath?

<How many> people with mild shortness of breath would need to be

cured

Types of PTO RefusalsTypes of PTO Refusals Two types of refusals:Two types of refusals:

– Equality RefusalsEquality Refusals» The choices are equally goodThe choices are equally good» Curing Curing 100100 people of quadriplegia = curing people of quadriplegia = curing 100100

people of foot numbnesspeople of foot numbness

– High RefusalsHigh Refusals» Extremely high indifference pointsExtremely high indifference points

Curing Curing 100100 people of quadriplegia = curing people of quadriplegia = curing 300,000,000300,000,000 people of foot numbness people of foot numbness

Frequency of PTO RefusalsFrequency of PTO Refusals

27%

17%

0%

10%

20%

30%

40%

50%

Equality Refusals High Refusals

Most prevalent

Is the problem the Decision-maker Is the problem the Decision-maker Perspective?Perspective?

Imagine that YOU ARE THE EXECUTIVE Imagine that YOU ARE THE EXECUTIVE DIRECTOR of a regional health system …you DIRECTOR of a regional health system …you have only enough money to fund one have only enough money to fund one treatment program….THE FINAL DECISION treatment program….THE FINAL DECISION IS UP TO YOU.IS UP TO YOU.

– … … you must choose between two treatment you must choose between two treatment programs… programs… who would you curewho would you cure??

Study QuestionStudy Question

Does perspective matter?Does perspective matter?– Will a non-decision-making perspective Will a non-decision-making perspective

encourage more people to make tradeoffs?encourage more people to make tradeoffs?» Less negative emotionLess negative emotion» Less pressureLess pressure» EasierEasier

Evaluator PerspectiveEvaluator Perspective Imagine two regional health systems…the Imagine two regional health systems…the

Executive Director of each system had only Executive Director of each system had only enough money to fund one treatment program…enough money to fund one treatment program…The health systems …were the same in every The health systems …were the same in every way except for the treatment program each way except for the treatment program each Executive Director decided to fund.Executive Director decided to fund.

– The Directors made the following decisions… The Directors made the following decisions… who made a better decisionwho made a better decision??

Results Results

Perspective:Perspective:

Decision-makerDecision-maker

EqualityEquality

RefusalsRefusals 21%21%

HighHigh


Results Results


Decision-makerDecision-maker EvaluatorEvaluator

EqualityEquality

RefusalsRefusals 21%21% 32%32%

HighHigh


6. What do people mean 6. What do people mean by “equity”?by “equity”?

Reminder of Colon Cancer Reminder of Colon Cancer Study DesignStudy Design

Imagine Medicaid program is choosing Imagine Medicaid program is choosing a colon cancer screening testa colon cancer screening test

Test #1Test #1 Inexpensive - can offer to everyoneInexpensive - can offer to everyone Less effective - save 1,000 livesLess effective - save 1,000 lives

Test #2Test #2 More expensive - can offer to 1/2 of the More expensive - can offer to 1/2 of the

peoplepeople More effective - save 1,100 livesMore effective - save 1,100 lives

Why Do People Value Equity?Why Do People Value Equity?

Test 1 can be offered toTest 1 can be offered to moremore people people than Test 2than Test 2

Test 1 can be offered to Test 1 can be offered to everyoneeveryone and and Test 2 cannotTest 2 cannot

Do people’s preferences for equity over efficiency persist when neither test can

be offered to the entire population?

Is Equity All or Nothing?Is Equity All or Nothing?

A B %1,000 lives 1,100 lives choosing A

1. 100% 50%2. 90% 40%3. 50% 25%

Is Equity All or Nothing?Is Equity All or Nothing?


1. 100% 50% 56

2. 90% 40% 27

3. 50% 25% 28

Isn’t 100% Arbitrary?Isn’t 100% Arbitrary?

Now imagine that the situation has changed Now imagine that the situation has changed in the following way:in the following way:Because of an unusually weak economy, the Because of an unusually weak economy, the number of people poor enough to qualify for number of people poor enough to qualify for Medicaid is doubled. That means twice as many Medicaid is doubled. That means twice as many people will be enrolled in Medicaid as had been people will be enrolled in Medicaid as had been predicted. However, there’s no change in the predicted. However, there’s no change in the budget for colon cancer screening. . .budget for colon cancer screening. . .

Vice versaVice versa

Arbitrary DesignArbitrary Design


4. 100% 50%50% 25%

5. 50% 25%100% 50%

Arbitrary ResultsArbitrary Results


4. 100% 50% 6250% 25% 64

5. 50% 25% 24100% 50% 40

Colon Cancer ThoughtsColon Cancer Thoughts

Preferences for equity vs. efficiency are Preferences for equity vs. efficiency are fragilefragile

Preferences depend on whether more Preferences depend on whether more effective tests can be offered to 100% of effective tests can be offered to 100% of a populationa population

People are only moderately sensitive to People are only moderately sensitive to the “arbitrariness” with which the “arbitrariness” with which populations are definedpopulations are defined

7. Revisiting attitudes toward 7. Revisiting attitudes toward paraplegiaparaplegia

Prior resultPrior result– Saving 100 people with paraplegiaSaving 100 people with paraplegia– Equally good as saving 100 non-disabled Equally good as saving 100 non-disabled

peoplepeople Could conclude thatCould conclude that

– When saving lives, disabilities like When saving lives, disabilities like paraplegia don’t matterparaplegia don’t matter

– Or Or pre-existingpre-existing disabilities don’t matter disabilities don’t matter

Pre-existing versus new paraplegiaPre-existing versus new paraplegia

Asked about pre-existing paraplegiaAsked about pre-existing paraplegia– 100 non-disabled = 100 paraplegia100 non-disabled = 100 paraplegia

Then asked about onset of paraplegiaThen asked about onset of paraplegia– 100 non-disabled = 126 paraplegia100 non-disabled = 126 paraplegia

Conclusion?Conclusion?– Care more about saving lives of people Care more about saving lives of people

with pre-existing paraplegia?with pre-existing paraplegia?– But still don’t think paraplegia is too bad?But still don’t think paraplegia is too bad?

New versus pre-existing paraplegiaNew versus pre-existing paraplegia

Asked about onset of paraplegiaAsked about onset of paraplegia– Instead of saying 100 non-disabled = 126 Instead of saying 100 non-disabled = 126

paraplegiaparaplegia– Said it = 1000Said it = 1000

Then asked about pre-existing paraplegiaThen asked about pre-existing paraplegia– Instead of saying 100 non-disabled = 100 Instead of saying 100 non-disabled = 100

paraplegiaparaplegia– Said it = 200Said it = 200

People’s preferences/values seem . . .People’s preferences/values seem . . . Flippy FloppyFlippy Floppy

8. Revisiting age-based rationing8. Revisiting age-based rationing

Remember: people favor distributing life Remember: people favor distributing life saving resources tosaving resources to– 35 year olds over 65 year olds35 year olds over 65 year olds

And some CEA experts say this should And some CEA experts say this should lead to age-weighted QALYslead to age-weighted QALYs

But what happens when we explore But what happens when we explore these values more thoroughly?these values more thoroughly?

Preference for young versus oldPreference for young versus old

Lifesaving Treatment Preference

5,004,003,002,001,00

Pe

rce

nt

50

40

30

20

10

0

Lifesaving Treatment

Preference for young versus oldPreference for young versus old

Lifesaving Treatment Preference

5,004,003,002,001,00

Pe

rce

nt

50

40

30

20

10

0

Palliative Care Treatment Preference

5,004,003,002,001,00

Pe

rce

nt

60

50

40

30

20

10

0

Lifesaving Treatment Palliative Care

Direct Assessments of Age ImportanceDirect Assessments of Age Importance

Intervention Intervention Type Type

MeanMean

InfertilityInfertility 6.46.4Lifesaving Lifesaving 4.34.3PalliativePalliative 3.83.8Reconstr. SurgReconstr. Surg 3.33.3Pain ReliefPain Relief 3.13.1DepressionDepression 2.62.6

(1-10)

Direct Assessments of Age Direct Assessments of Age ImportanceImportance

Intervention Intervention TypeType

MeanMean %“Not at all %“Not at all Important”Important”

%“Very %“Very Important”Important”

InfertilityInfertility 6.46.4 1313 4040

Lifesaving Lifesaving 4.34.3 3434 2121

PalliativePalliative 3.83.8 4040 2121

Reconstr. SurgReconstr. Surg 3.33.3 3939 99

Pain ReliefPain Relief 3.13.1 5252 1818

DepressionDepression 2.62.6 5656 1212

(1-10)

Is there any value in Is there any value in measuring public measuring public

values?values?

Consider the alternativeConsider the alternative

Before QALYsBefore QALYs– We had $ / life yearWe had $ / life year

That meant we had no abilityThat meant we had no ability– To capture the value of interventions that To capture the value of interventions that

improve QOLimprove QOL– To compare life-saving treatments to life To compare life-saving treatments to life

improving onesimproving ones

CEA is not perfectCEA is not perfect

There is no single $ / QALY figure for There is no single $ / QALY figure for any interventionany intervention

There is no exact $ / QALY threshold for There is no exact $ / QALY threshold for societysociety– Not $50KNot $50K– Not $100KNot $100K

CEA is a tool to guide decisionsCEA is a tool to guide decisions

A glimmer of hope for finding A glimmer of hope for finding stable valuesstable values

People place a different value on saving People place a different value on saving the lives ofthe lives of– Non-disabled peopleNon-disabled people– People with pre-existing paraplegiaPeople with pre-existing paraplegia– People with onset of paraplegiaPeople with onset of paraplegia

And those various values are messyAnd those various values are messy But why do they think paraplegia is so But why do they think paraplegia is so

bad?bad?

Failure to consider Failure to consider emotional adaptationemotional adaptation

People consistently misestimate the People consistently misestimate the QOL of chronic illness/disabilityQOL of chronic illness/disability– Patients report high levels of well-beingPatients report high levels of well-being– Public imagines miseryPublic imagines misery

But public can be prodded to think But public can be prodded to think about adaptationabout adaptation– Think back to bad event > 6 months ago. . .Think back to bad event > 6 months ago. . .

How thinking about adaptation How thinking about adaptation stabilized valuesstabilized values

1. Pre-existing1. Pre-existing

2. Onset2. Onset

No AdaptationNo Adaptation

100100

126126

1. Onset1. Onset

2. Pre-existing2. Pre-existing

10001000

200200

AdaptationAdaptation

100100

102102

102102

100100

Revisiting the Decision-maker II Revisiting the Decision-maker II Perspective: A follow up studyPerspective: A follow up study

Made the hot-seat even hotter:Made the hot-seat even hotter:– ““The money cannot be split between the two The money cannot be split between the two

programs”programs”– ““You will only be able to fund one of them “You will only be able to fund one of them “– ““The other program will not be funded “The other program will not be funded “

» ““People who have the condition covered by that People who have the condition covered by that program will go untreated.”program will go untreated.”

– ““Who would you cure,Who would you cure, thereby leaving the thereby leaving the other group without treatment?” other group without treatment?”

Method: Benefits PerspectiveMethod: Benefits Perspective

Focus on benefits:Focus on benefits:– ““Imagine that two groups of patients Imagine that two groups of patients

received medical treatment for a condition received medical treatment for a condition from which they were suffering…”from which they were suffering…”

– ““Which group received the greatest overall Which group received the greatest overall benefitbenefit? “? “

ResultsResults

PerspectivePerspective::Decision-maker IDecision-maker I

EqualityEquality


HighHigh


ResultsResults

PerspectivePerspective::Decision-maker IDecision-maker I Decision-maker IIDecision-maker II

EqualityEquality


HighHigh


ResultsResults

PerspectivePerspective::Decision-maker IDecision-maker I Decision-maker IIDecision-maker II BenefitsBenefits

EqualityEquality

RefusalsRefusals 21%21% 12%12% 43%43%

HighHigh

RefusalsRefusals 19%19% 10%10% 4%4%

Why do the non-decision-maker Why do the non-decision-maker perspectives INCREASE equality refusals?perspectives INCREASE equality refusals?

A prioriA priori, we thought, we thought– People would find the decision-maker People would find the decision-maker

perspective perspective more difficultmore difficult

Yes, they did…Yes, they did…

……but, people who thought the survey was but, people who thought the survey was hard were LESS likely to refuse to tradehard were LESS likely to refuse to trade


Decision-maker IIDecision-maker II BenefitsBenefits

Survey Survey was hardwas hard 46%46% 37%37%

11%20%

10%

34%

50%47%

0%

20%

40%

60%

80%

100%

1-Disagree 4-Neutral 7-Agree

Decision-maker II

Benefits

Equality Refusals & Perceived Equality Refusals & Perceived DifficultyDifficulty

Why do the non-decision-maker Why do the non-decision-maker perspectives INCREASE equality refusals?perspectives INCREASE equality refusals?

A prioriA priori, we thought, we thought– People would find the decision-maker People would find the decision-maker

perspective perspective more difficultmore difficult

– Higher negative emotion Higher negative emotion Less likely to Less likely to tradeofftradeoff

Emotion: Follow-up StudyEmotion: Follow-up StudyPerspective:Perspective:

Decision-Decision-maker IImaker II

BenefitsBenefits

Outraged Outraged about about

rationingrationing50%50% 57%57%

Emotion: Follow-up StudyEmotion: Follow-up StudyPerspective:Perspective:

Decision-Decision-maker IImaker II

BenefitsBenefits


rationingrationing50%50% 57%57%


questionsquestions12%12% 8%8%

9%

20%12%

35%

50%44%

0%

20%

40%

60%

80%

100%

1-Disagree 4-Neutral 7-Agree

Decision-maker II

Benefits

Equality Refusals & EmotionEquality Refusals & Emotion

Perspective mattersPerspective matters

Do not worry about making people sweatDo not worry about making people sweat It is not a matter of making elicitations It is not a matter of making elicitations

easy – but, why the question is easy – but, why the question is importantimportant– More engagedMore engaged

Final, inconclusive thoughtsFinal, inconclusive thoughts Future AgendaFuture Agenda

– Develop methods for measuring public values that Develop methods for measuring public values that areare

» More stableMore stable» More consistentMore consistent

– Recognize that those valuesRecognize that those values» Are not fully formed before measurementAre not fully formed before measurement

– Remember thatRemember that» Even though value measurement flawedEven though value measurement flawed» Measuring cost-effectiveness without QALYs is Measuring cost-effectiveness without QALYs is not an not an

option!option!

www.med.umich.edu/pihcd

preference measurement in cea: are we capturing values or creating them?

Documents

moderate health problems

perfect health

lives of people

allocation choices80

littleimagine illness

organs availablesubjects

moderate problemstreatment

littlemost funding