Multiple Challenges in Capturing Utilities for Paediatric Conditions

Andrew Lloyd, DPhil (Workshop Chair), Cicely Kerr, PhD

ICON Patient Reported Outcomes

John Brazier, PhD Health Economics and Decision Science (HEDS)

School of Health and Related Research The University of Sheffield, UK

Andrew Walker, PhD

Clinical Trials Unit, Glasgow University and Scottish Medicines Consortium (SMC)

Workshop Agenda

1. Introduction – Andrew Lloyd

2. Methodological issues in paediatric utility assessment – John Brazier

3. Case study: eliciting utilities in Fragile X Syndrome – Cicely Kerr

4. Paediatric utilities: the view from within an HTA agency – Andrew Walker

5. Chaired Discussion and Comment – Andrew Lloyd

Methodological Issues in Paediatric Utility Assessment

Katherine Stevens and John Brazier

Health Economics and Decision Science (HEDS)

School of Health and Related Research The University of Sheffield, UK

Prepared for ISPOR conference, Dublin, 3-6 November 2013

Overview

• The use of QALYs

• Why are children different?

• Challenges

• Options available

Use of QALYs in Economic Evaluation

• Quality Adjusted Life years combine survival and quality of life into a single measure

• They do this by assigning a score to health related quality of life where one is full health and zero states as bad as being dead

• Establishment of organisations such as NICE, PBAC, PHARMAC.......

• There is now an increased demand for QALY measurement

Why are children different?

Challenges in the measurement and valuation of child and

adolescent health

• What should be measured?

• Who should measure it?

• Who should value it?

What should be measured? • Dimensions that are relevant to the

purpose of the measure (health care resource allocation decisions)

• Dimensions that are relevant to children • Some adult measures may contain irrelevant

dimensions (e.g. Work)

Dimensions relevant to children – of all ages?

• Teenagers, younger children, babies • Confounding with development in younger

children, particularly under 5 years of age • HRQoL measures may demonstrate an

improvement but this could be due to natural development, not to an improvement in QoL

Taken from the HUI2

Unable to walk at all

Required the help of another person to eat, bathe, dress or use the toilet

Who should measure it? • Growing recognition children have their own unique

views and a right to express them in matters affecting them

• Increasingly recognized in clinical trials and HSR that descriptions of the experience of a health state should be elicited from the patients – to reflect the actual experience of the disease and its treatment

Who should value it?

• Publicly funded health care system • Tax payers • Adults as rational and informed individuals • Children

• cognitive complexities of valuation tasks • ethical issues

How should values be elicited from children?

• Cognitive and ethical problems in using conventional time trade-off and standard gamble techniques

• Interest in using either: 1) adults to value states or 2) using pairwise choices or best worst scaling

Current options

• Currently 3 existing preference based measures available and 1 in development

• The Health Utilities Index 2 (HUI2) • The Child Health Utility 9D (CHU9D) • The AQoL 6D • The EQ 5D Y

• Use vignettes

CHU9D HUI2 AQoL-6D EQ-5D-Y Worried

Emotion Mental Health Anxiety and depression

Sad

Annoyed

Tired Pain Pain Pain Pain/discomfort Sleep

Daily Routine Self Care Independent Living Self care

School work Cognition Usual activities Joining in activities Mobility

Mobility Usual activities

Sensation Senses Relationships Coping

Comparison of measures - dimensions

Comparison of measures

How was the content generated?

Who completes it?

Whose values?

CHU9D Interviews with children

Child or proxy UK adult general population Australian adolescent population

HUI2 Literature review followed by views of parent/child

Child or proxy •Parents (Canada) •Adult general population (UK)

AQoL 6D

Adaptation of adult instrument

Adolescent Adolescents

EQ 5D Y

Adaptation of adult instrument

Child/Adolescent None available

What do the guidelines say?

• NICE – require the use of QALYs • “It is recognised that the current version of the EQ-5D has not been

designed for use in children. When necessary, consideration should be given to alternative standardised and validated preference-based measures of HRQL, such as the Health Utility Index 2 (HUI 2), that have been designed specifically for use in children.”

(5.4.10) Guide to the methods of technology appraisal. NICE 2008

• PBAC – also mention the HUI2 • PHARMAC – no mention of child instruments

Future Directions

• Is it feasible to obtain young people’s preferences using DCE and BW methods?

• Adult vs. young people valuing health • Adults as proxies • QALY age weightings – link to Value

Based Pricing • Validation of utility measures

Summary

• QALYs • Why they are different for children • There are options available • Many issues outstanding for further

research

Advertisement: www.chu9d.org

Case Study: Eliciting Utilities in Fragile X Syndrome

Cicely Kerr, PhD Lead Outcomes Researcher

ICON Patient Reported Outcomes

Acknowledgements This work was sponsored by Novartis Pharmaceuticals and conducted with the help of: • Donald B. Bailey Jr., RTI International • Elizabeth Berry-Kravis, Departments of Pediatrics, Neurological

Sciences, and Biochemistry, Rush University Medical Center, USA

• Jonathan Cohen, Fragile X Alliance Inc, Australia; Centre for Developmental Disability Health Victoria, Monash University, Australia

• Walter Kaufmann, Harvard Medical School, USA • Angela Hassiotis, University College London, UK • John Brazier, ScHARR, The University of Sheffield, UK • Koonal Shah and Nancy Devlin, Office of Health Economics, UK • Mohsen Tavakol, University of Nottingham, UK

• Patricia Sacco and Jennifer Petrillo, Novartis Pharmaceuticals Corporation

Fragile X Syndrome (FXS)[1-4]

• X-linked genetic condition caused by a mutation on the FMR1 gene

• Most common inherited form of intellectual disability • Affects children and into adulthood • Males - more prevalent and typically worse affected • Large caregiver burden, many patients unable to live

independently • Characteristics include:

• Aggression • Hyperarousal • Hyperactivity • Irritability • Below average IQ • Attention difficulties

• Stereotyped movements e.g. hand flapping

• Self injurious and avoidant behaviour

• Emotional problems, particularly anxiety

FXS Context and Challenges • FXS drug development

• Potential new treatments emerging • New area for HTA/payer assessment of value

• Utility/HRQL Measurement in FXS • Limited FXS HRQL data • Proxy assessments commonly used, challenge for capturing

HRQL data[5]: • Subjective aspects of patients’ HRQL (e.g. mood, psychological state)

hard for proxy to assess • Proxy assessments of patients’ HRQL can be influenced by

relationship between the proxy and the patient, level of burden experienced by the proxy and the proxy’s own HRQL

• Generic HRQL/utility measures may not capture key aspects of FXS[6]

New approach

Case Study Approach

• Adapt FXS outcome measure for utility estimation • Measure: Aberrant Behavior Checklist (ABC)[7-10]

• Proxy completed instrument developed for rating maladaptive and inappropriate behaviours of individuals with intellectual disabilities

• Sensitive in FXS and commonly adopted as a primary outcome measure in FXS clinical trials

• Caregivers/proxies assess observable aspects of the condition rather than subjective HRQL domains

Aberrant Behavior Checklist (ABC)[7]

Standard Methodological Steps

• Standard approach for developing condition-specific preference-weighted scoring algorithm[11]:

• Develop health states from items

• Item reduction/selection

• Classification system from selected items/levels

• Health state valuation

• Model health state data

• Develop scoring algorithm from model parameters

Case Study Challenges

• Health state development from ABC items

• Behavioural vs. HRQL focus

• Instrument length (58 items)

• FXS ABC health state valuation

• Adult vs. child health states

• Severity of worst health states

Health State Development

• Development of health state classification from ABC items • Aimed to identify items capturing

• Important aspects of FXS • Items associated with important HRQL impact

• Iterative process combining information from multiple sources • Clinical experts • Statistical analysis of FXS ABC dataset

• Item performance, Factor Analysis, Rasch Analysis, correlation with other measures (IQ, attention problems, global assessment of HRQL)

• Piloting • Cognitive debrief interviews

• Development of health states for valuation • Fractional factorial orthogonal design

Example ABC FXS Health State

Valuation Study Methods

• FXS ABC health state valuation • Adult participants valuing health states by imagining they were

experiencing them (i.e. as adult patient) • Combined TTO/Lead Time TTO valuation

• Sample • UK general population (n=349)

• Analysis • Regression models fitted to individual level and mean health

state values • OLS, RE MLE, Mean OLS

• Assessment of model performance • consistency and predictive performance, RMSE, MAE, AIC, BIC

• Algorithm developed from values from best performing model • ABC items (health state dimension) weighted by parameter

coefficients (decrements) associated with levels (scores)

Results

• Observed mean utilities for FXS ABC health states: • 0.92 best FXS health state • 0.16 worst FXS health state

• Model predictive performance (RE MLE)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

observed

prediction

FXS ABC Health States Valued

Mea

n ut

ility

val

ue

Discussion • Strengths

• Careful health state development • Good model performance • ABC-UI able to estimate a wide range of utility values from

patient-level FXS ABC data

• Limitations • ABC-UI performance compared with other utility measures? • ABC

• Proxy measure of behaviour • Limitations for capturing HRQL and non-behavioural

characteristics of FXS

Is it HRQL/a QALY?

Whose HRQL/values?

References 1. Crawford, D. C., Acuña J. M., & Sherman, S. L. (2001). FMR1 and the fragile X syndrome: human genome

epidemiology review. Genetics in Medicine, 3, 359-371. 2. Hagerman, R. J. & Hagerman, P. J. (2002). Fragile X syndrome: Diagnosis, treatment, and research. Johns

Hopkins Univ Pr., Baltimore, MD 3. Chonchaiya, W., Schneider, A., & Hagerman, R. J. (2009). Fragile X: a family of disorders. Advances in

Pediatrics, 56, 165. 4. Hartley, S. L., Seltzer, M. M., Raspa, M., Olmstead, M., Bishop, E., & Bailey Jr, D. B. (2011). Exploring the adult

life of men and women with fragile x syndrome: results from a national survey. American Journal of Intellectual and Developmental Disabilities, 116, 16-35.

5. Addington-Hall, J. & Kalra, L. (2001). Who should measure quality of life? BMJ, 322, 1417-1420. 6. Krabbe, P. F. M., Stouthard, M. E. A., Essink-Bot, M. L., & Bonsel, G. J. (1999). The effect of adding a cognitive

dimension to the EuroQol multiattribute health-status classification system. Journal of Clinical Epidemiology, 52, 293-301.

7. Aman, M. G., Singh, N. N., Stewart, A. W., & Field, C. J. (1985). The Aberrant Behavior Checklist: A behavior rating scale for the assessment of treatment effects. American Journal of Mental Deficiency. 89(5); 485-491

8. Erickson, C. A., Stigler, K. A., Wink, L. K., Mullett, J. E., Kohn, A., Posey, D. J. et al. (2011). A prospective open-label study of aripiprazole in fragile X syndrome. Psychopharmacology, 216, 85-90.

9. Jacquemont, S., Curie, A., des, P., V, Torrioli, M. G., Berry-Kravis, E., Hagerman, R. J. et al. (2011). Epigenetic modification of the FMR1 gene in fragile X syndrome is associated with differential response to the mGluR5 antagonist AFQ056. Science Translational Medicine, 3.

10.Paribello, C., Tao, L., Folino, A., Berry-Kravis, E., Tranfaglia, M., Ethell, I. et al. (2010). Open-label add-on treatment trial of minocycline in fragile X syndrome. BMC Neurology, 10, 91.

11.Brazier, J.E., Rowen, D., Mavranezouli, I., Tsuchiya, A., Young, T., Yang, Y., Barkham, M., Ibbotson, R. (2012). Developing and testing methods for deriving preference-based measures of health from condition specific measures (and other patient based measures of outcome). Health Technology Assessment, 16(32), 1366-5278.

The view from within an HTA agency

Dr Andrew Walker Clinical Trials Unit, Glasgow University

And Scottish Medicines Consortium

I am a consultant to SMC and their views may not reflect mine

Utility values @ SMC

• Economic evaluation required in all full submissions, and CUA is preferred (but CMA is allowed)

• Less specific about a ‘hierarchy of evidence’ for utility values e.g. role of EQ-5D, level of criticism of ‘mapping’ from disease-specific scales

• Plus point: more pragmatic and flexible • Minus point: scope for companies to pick best case

HTA of medicines for children & adolescents • SMC support paediatric licensing so when medicine

is accepted for adults, only require a (highly) ‘abbreviated’ submission

• But in principle we want consistent approach when there is a full submission, ‘usual rules’ (economics evidence)

• Acknowledge difficulties, especially regarding utilities

• E.g. use of proxies not seen as a reason for rejecting the case, familiar to us from other diseases e.g. dementia, severe mental illness

Last 10 full submissions

• Where children / adolescents / paediatrics mentioned in the license

• Had to look back over 3 years • And even then 5 of them had marketing

authorisation for “adults and children” – in these cases, children were ignored in the economics

• Of the remainder (all CUAs): • 1 case: survival was an important issue • 1 case: acute illness, rarely life-threatening • 3 cases: chronic illness mainly affecting QoL

Utilities: sources and standardisation • Sources of utilities:

• 1 mapped from a disease-specific outcome – algorithm for adult patients

• 2 used EQ-5D completed by proxies • Parents • GP/paediatrician • Valued using population TTO values

• 2 drew on catalogues of values for all ages

• Only 1 applied age-standardisation and in this case the value used was for everyone under 25 years of age

QALY gain

• The net QALY gain estimated for each medicine was as follows:

• Where survival was a factor (n=1): gain 1.34 QALYs over lifetime

• In acute illness (n=1): gain of 0.007 QALYs per 100 children

• In chronic illness (n=3): gains of 0.01 and 0.004 over one year and of 0.32 over lifetime

SMC’s response

• Tests applied by SMC to the utilities used: • Sensitivity analysis – look for reassurance, does cost per

QALY change dramatically with a simple change in utility?

• Comparison with values used in other credible studies – e.g. NICE MTA, where this was adapted and applied in Scotland

• Opinion of clinical experts on values

• Opinion of clinicians on the SMC

SMC guidance and the role of utilities • Of the 5 medicines, 3 were accepted for use, 2

were not recommended (NR)

• All acknowledged difficulties for company

• NRs were not because of utility values used

• But there were critical comments: • Collecting utility data and not using it (3 of 5) • Utilities should not be applied outside age range (3 of 5)

Lessons when planning submissions • Cost per QALY is required, but we are prepared to

show some pragmatism

• If you collect QoL data, then use it!

• Use sensitivity analysis to reassure (threshold values helpful)

• Compare to other reputable sources (not the same as a systematic literature review)

Issues

• Is SMC ‘too pragmatic’?

• How often do the utility values matter in sensitivity

analysis?

• If you were in my shoes, what would you do in response to the submissions described?