Accepted Manuscript

Measuring What Matters: Top-Ranked Quality Indicators for Hospice and PalliativeCare from the American Academy of Hospice and Palliative Medicine and Hospiceand Palliative Nurses Association

Sydney Morss Dy, MD, MSc, Kasey B. Kiley, MPH, Katherine Ast, MSW, LCSW, DaleLupu, PhD, Sally A. Norton, PhD, RN, FAAN, Susan C. McMillan, PhD, ARNP, FAAN,Keela Herr, PhD, RN, AGSF, FAAN, Joseph D. Rotella, MD, MBA, FAAHPM, David J.Casarett, MD, MA

PII: S0885-3924(15)00073-1

DOI: 10.1016/j.jpainsymman.2015.01.012

Reference: JPS 8834

To appear in: Journal of Pain and Symptom Management

Received Date: 4 October 2014

Revised Date: 10 January 2015

Accepted Date: 21 January 2015

Please cite this article as: Dy SM, Kiley KB, Ast K, Lupu D, Norton SA, McMillan SC, Herr K,Rotella JD, Casarett DJ, Measuring What Matters: Top-Ranked Quality Indicators for Hospice andPalliative Care from the American Academy of Hospice and Palliative Medicine and Hospice andPalliative Nurses Association, Journal of Pain and Symptom Management (2015), doi: 10.1016/j.jpainsymman.2015.01.012.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service toour customers we are providing this early version of the manuscript. The manuscript will undergocopyediting, typesetting, and review of the resulting proof before it is published in its final form. Pleasenote that during the production process errors may be discovered which could affect the content, and alllegal disclaimers that apply to the journal pertain.

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

Special Article 14-00532R1

Measuring What Matters: Top-Ranked Quality Indicators for Hospice and Palliative Care

from the American Academy of Hospice and Palliative Medicine and Hospice and

Palliative Nurses Association

Sydney Morss Dy, MD, MSc, Kasey B. Kiley, MPH, Katherine Ast, MSW, LCSW, Dale Lupu,

PhD, Sally A. Norton, PhD, RN, FAAN, Susan C. McMillan, PhD, ARNP, FAAN, Keela Herr,

PhD, RN, AGSF, FAAN, Joseph D. Rotella, MD, MBA, FAAHPM, and David J. Casarett, MD,

MA

Johns Hopkins Sidney Kimmel Cancer Center (S.M.D.), Baltimore, Maryland; Johns Hopkins

Hospital (K.B.K.), Baltimore, Maryland; American Academy of Hospice and Palliative Medicine

(K.A., D.L.), Chicago, Illinois; University of Rochester School of Nursing (S.A.N.), Rochester,

New York; University of South Florida (S.C.M.), Tampa, Florida; University of Iowa College of

Nursing (K.H.), Iowa City, Iowa; Hosparus (J.D.R.), Louisville, Kentucky; and University of

Pennsylvania Perelman School of Medicine (D.J.C.), Philadelphia, Pennsylvania, USA

Address correspondence to:

Sydney Morss Dy, MD MSc

Room 609

624 N. Broadway, Baltimore, MD 21205, USA

E-mail: sdy@jhsph.edu

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

2

Abstract

Context. Measuring quality of hospice and palliative care is critical for evaluating and

improving care, but no standard U.S. quality indicator set exists.

Objectives. The Measuring What Matters (MWM) project aimed to recommend a

concise portfolio of valid, clinically relevant, cross-cutting indicators for internal measurement

of hospice and palliative care.

Methods. MWM was a sequential consensus project of the American Academy of

Hospice and Palliative Medicine (AAHPM) and Hospice and Palliative Nurses Association

(HPNA). We identified candidate indicators mapped to National Consensus Project (NCP)

Palliative Care Guidelines domains. We narrowed the list through a modified Delphi rating

process by a Technical Advisory Panel and Clinical User Panel and ratings from AAHPM and

HPNA membership and key organizations.

Results. We narrowed the initial 75 indicators to a final list of 10. These include one in

the NCP domain Structure and Process (Comprehensive Assessment), three in Physical Aspects

(Screening for Physical Symptoms, Pain Treatment, Dyspnea Screening and Management), one

in Psychological and Psychiatric Aspects (Discussion of Emotional or Psychological Needs), one

in Spiritual and Existential Aspects (Discussion of Spiritual/Religious Concerns), and three in

Ethical and Legal Aspects (Documentation of Surrogate, Treatment Preferences, and Care

Consistency with Documented Care Preferences). The list also recommends a global indicator of

patient/family perceptions of care, but does not endorse a specific survey instrument.

Conclusion. This consensus set of hospice and palliative care quality indicators is a

foundation for standard, valid internal quality measurement for U.S. settings. Further

development will assemble implementation tools for quality measurement and benchmarking.

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

3

Key Words: Palliative care, pain measurement, hospice care, quality of health care, quality

indicators, advance care planning, patient satisfaction

Running Title: Measuring What Matters in Hospice and Palliative Care

Accepted for publication: January 21, 2015.

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

4

Introduction

Far too many patients and families in the United States experience unnecessary physical

and emotional suffering during serious and life-threatening illnesses. 1 Recent studies have

demonstrated gaps in quality of care in domains such as pain and symptom management,

communication, and care planning across settings such as hospitals, nursing homes and

ambulatory care. 2-4 Palliative care is defined as care that provides relief from symptoms and

supports quality of life for patients with serious advanced illness and their families, and hospice

care is an approach focused on patients with limited life expectancy and their families. 1

Accumulating research demonstrates that interventions such as ambulatory palliative care clinics,

structured goals of care discussions in critical care, and outpatient nurse-led interventions

targeting patient/family pain management can improve outcomes such as patient and family

satisfaction and health care utilization. 5

Measuring the quality of care delivery is integral to hospice and palliative care programs,

as specified by the National Consensus Project (NCP) Clinical Practice Guidelines for Palliative

Care, 6 the Joint Commission Advanced Certification Program for Palliative Care and, for

hospice programs, the Centers for Medicare & Medicaid Services (CMS). 7 A critical first step in

improving care is demonstrating where quality deficits exist compared to national benchmarks

and determining where quality improvement initiatives are most needed and most likely to be

beneficial, but no national U.S. (or universal) standards for measurement or databases that would

support benchmarking yet exist. Quality indicator sets for a variety of relevant populations and

settings have been developed and tested 8-11 and the National Quality Forum (NQF) has endorsed

a group of palliative care indicators 11 suitable for accountability, such as public reporting.

However, many of the indicators intended for accountability may not be the most appropriate for

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

5

internal use in clinical programs, and a standard, concise, cross-cutting set of indicators that can

be used internally for benchmarking, comparison across programs and quality improvement is

needed. Many palliative care programs are not yet routinely measuring quality, many are using

locally developed, non-validated indicators, and for those who wish to measure quality, there are

no nationally used sets with benchmarking that apply across populations and settings.

The Quality and Practice Standards Committee of the American Academy of Hospice and

Palliative Medicine (AAHPM), therefore, initiated the Measuring What Matters (MWM)

consensus project, which was joined by the Hospice and Palliative Nurses Association (HPNA)

Research Advisory Council. The overall goal of the project was to recommend a concise

portfolio of valid, clinically relevant, cross-cutting performance indicators for internal

measurement for hospice and palliative care programs. The intent was to develop a common

core set from which programs could select, to help create standards for quality measurement of

palliative care in the U.S. and allow for national benchmarking. The aspirational goal was an

initial set of process and outcome indicators that apply regardless of diagnosis, organizational

structure or setting, although the MWM team recognized that such indicators may not always be

appropriate or not yet exist and that some included indicators as currently defined may not apply

across populations and settings. The MWM team acknowledged that promoting high-quality

hospice and palliative care indicators for accountability with CMS and other groups is also an

important goal, 1 but may be less relevant for program development or for improving the patient

and family experience of care; this was, therefore, not a goal of the MWM project. In this paper,

we describe the MWM methodology and the initial core consensus set of quality indicators.

Methods

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

6

MWM was a sequential consensus project, directed by a partnership between AAHPM

and HPNA with a modified Delphi rating process by first a Technical Advisory Panel and then a

Clinical User Panel, followed by input solicited from AAHPM and HPNA membership and from

external organizations and patient advocacy groups to obtain the final indicator set (Table 1).

The project goals described above guided the MWM process. In particular, the MWM team

aligned the set with the NCP guidelines, including organizing by the eight domains, and other

existing quality initiatives whenever possible. The team envisioned the indicator set for

population-level measurement, and it may be insufficient for measurement targeting specific

settings or populations. The MWM process also evaluated whether denominators of existing

indicators should be considered for future modification to be more inclusive and cross-cutting

across settings or populations.

Identifying Indicators

The MWM project began by identifying existing U.S. process and outcome indicators

relevant to hospice and palliative care, available in the public domain as of October 2013 and

specified for U.S. data sources and developed through a rigorous process and/or tested for

reliability and validity in English. Sources included indicators endorsed by the NQF and/or in the

Agency for Healthcare Research and Quality (AHRQ) Quality Measures database, as well as

indicators included in the PEACE Palliative Care and Hospice, 8, 12 Cancer Quality-ASSIST

supportive oncology, 2, 13 American Society for Clinical Oncology (ASCO) Quality Oncology

Practice Initiative (QOPI®) Care at the End of Life, 14 and Assessing Care of Vulnerable Elders

(ACOVE) Palliative Care and End of Life 15, 16 sets, indicators developed by the National

Hospice and Palliative Care Organization (NHPCO) 17, 18 and American Medical Association-

convened Physician Consortium for Performance Improvement (PCPI), 19 and indicators used by

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

7

the CMS. Indicators were considered relevant to hospice and palliative care based on the

numerator. If the numerator or topic of the indicator was related to the NCP domains or preferred

practices of palliative care, the indicator was considered for inclusion. Members of the project

steering committee then reviewed the compiled list and identified any missing indicators. From

these sources, the MWM team identified 75 existing indicators that met criteria, and classified

them by NCP domain and compiled existing specifications and evidence about each indicator.

Almost all indicators had reliability and validity testing in palliative care populations.

Panel Ratings and Public Comment

Technical Advisory Panel (TAP). The key purpose of the TAP was to judge the technical

strength of existing indicators. The MWM team recruited 11 key researchers on quality

measurement in hospice and palliative medicine for the TAP and co-chairs from AAHPM and

HPNA. After an introductory instructional webinar, all members of the TAP were given detailed

information on the indicators and associated evidence, including their reliability and validity

testing, and asked to review and rate the indicators on a scale from 1 to 9, where 1 indicated low

and 9 high scientific soundness (Appendix). When rating each indicator, in addition to using the

specifications and evidence provided about each indicator, TAP members considered the AHRQ

desirable attributes of quality indicators:

• Importance

• Scientific soundness: clinical logic

• Scientific soundness: indicator properties, including evidence for reliability and

validity

• Feasibility

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

8

TAP members rated each indicator twice: once for the indicator as specified and once for the

indicator concept (the numerator only), and also could provide free-text comments. The purpose

of the indicator concept rating was to determine if the numerator was a good candidate for future

indicator development using a global all-palliative care population denominator to be developed

in the future.

TAP members then participated in a conference call to discuss the indicators, focusing

particularly on those with disagreement, defined as at least one rating <4 OR at least three ratings

<7. Members could discuss issues raised and advocate for or against indicators. TAP members

then rated the indicators a second time using the same process as the first round. This process

identified 34 indicators sufficiently highly rated for scientific soundness to advance to the

Clinical User Panel (CUP) rating process.

Clinical User Panel (CUP). The purpose of the CUP process was to select the indicators

that were the most clinically important and usable. The goal was to form a concise measurement

set of approximately 10 indicators, although it was recognized that the final number of indicators

would be responsive to the concerns of both comprehensiveness and brevity. The MWM team

recruited co-chairs from AAHPM and HPNA and 27 members for the CUP to represent diversity

in settings where hospice and palliative services are provided, patient populations and illnesses,

and providers including physicians, nurses, social workers, chaplains and grief counselors, as

well as consumer groups and quality organization groups. The CUP reviewed the 34 indicators

forwarded by the TAP using a similar process as the TAP. The CUP rated the indicators on

meaningfulness for patients and families, actionability for providers and organizations (including

both feasibility of measurement and possibility of making changes based on results), and

potential impact (prevalence of problem and/or amount of possible improvement) (Appendix).

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

9

Following the survey and discussion webinar and further consolidation/prioritization from the

CUP co-chairs considering all feedback and comments, consensus was reached on 12 cross-

cutting indicators from a variety of sources to submit for the last steps in the process.

In the final steps, the MWM team distributed information on the project and on the 12

indicators to the AAHPM and HPNA membership and their teams and to key relevant

stakeholders for further ratings and comment on which indicators were highest priority and

which should not be included, before winnowing down to a final list of 10 indicators. In total,

feedback was received from 264 individuals, including 131 physicians, 106 nurses, 10 social

workers, four pharmacists and three chaplains, and 27 organizations, including the American

Academy of Neurology, American Academy of Pain Medicine, American Academy of

Pediatrics, American College of Emergency Physicians, American Geriatrics Society,

Association of Professional Chaplains, Center to Advance Palliative Care, NHPCO, National

Palliative Care Research Center, Oncology Nursing Society, Social Work Hospice & Palliative

Network and The Joint Commission.

Results

Description of Selected Indicators

The final MWM quality indicator set includes 10 indicators (Table 2). These include one

in the NCP domain of Structure and Process of Care (Comprehensive Assessment), three in the

domain of Physical Aspects of Care (Screening for Physical Symptoms, Pain Treatment, and

Dyspnea Screening and Management), one in Psychological and Psychiatric Aspects of Care

(Discussion of Emotional or Psychological Needs), one in Spiritual and Existential Aspects of

Care (Discussion of Spiritual/Religious Concerns), and three in Ethical and Legal Aspects of

Care (Documentation of Surrogate, Treatment Preferences, and Care Consistency with

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

10

Documented Care Preferences). In addition, the MWM list recommends that a global indicator of

care from the patient/family perspective be included, but the MWM process did not endorse a

specific measure. Measures of the patient/caregiver experience that were considered and met the

initial MWM criteria for the candidate set are described in Table 3. No indicators were included

in the domains of Social Aspects of Care, Cultural Aspects of Care, and Care at the End of Life,

highlighting gaps in areas where indicators may not be supported by current evidence or may

potentially not be appropriate for evaluating care.

None of the MWM indicators are disease-specific and almost all apply to any patients

receiving palliative or hospice care services, although one is for vulnerable elders (people over

the age of 65 at increased risk for death or functional disability). Most are from the PEACE

dataset and apply to hospitalized or hospice patients; one (Discussion of Spiritual/Religious

Concerns) is for hospice only and one (Dyspnea Screening and Treatment) is for ambulatory care

(by physicians). Most address processes of care. All indicators would require medical record

review or duplicate data entry into a quality database, except for patient/family perceptions of

care. Only two are currently endorsed by the NQF (Discussion of Spiritual/Religious Concerns

and Treatment Preferences).

Discussion

In summary, the MWM project, led by the AAHPM and HPNA with input from a wide

variety of stakeholder and patient advocacy groups, assembled a consensus-based concise set of

process and outcome quality indicators for internal current use for quality measurement and

improvement in settings caring for hospice and palliative care patients. These indicators cover

five of the eight domains of the NCP Palliative Care Guidelines and most are applicable to broad

hospice and palliative care populations and more than one setting (hospital and hospice care);

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

11

only one is for the ambulatory setting. Most are process indicators and require medical record

abstraction or separate data entry. The MWM list also includes a global indicator for

patient/family experience of care, although no consensus developed for a specific measurement

tool (the measure for hospices, the Family Evaluation of Hospice Care (FEHC), was being

replaced at the time of this project by the Consumer Assessment of Healthcare Providers and

Systems (CAHPS) hospice survey) and the group recommended that additional measure

development was needed.

Some of these indicators have been used and tested more widely than others, and further

refinement and evaluation are needed. Many supporting tools already exist for the PEACE

indicators,12 and further development of accompanying tools for the indicator set, including

details on standardizing data abstraction, measurement specifications, benchmarking, and

evidence-based quality improvement, as well as potential recommendations for modifications of

the indicators based on experience with them, is needed. The MWM panel recognized that

quality indicators are needed both to evaluate the quality of hospice and palliative care programs

and to evaluate the quality of palliative care provided more broadly within settings by all

providers to patients with palliative care needs, and further testing should evaluate both of these

applications. Because the purpose of the project was to define cross-cutting indicators, initiatives

targeting quality measurement in specific populations (e.g., cancer) or settings (e.g., nursing

homes) may benefit from using more targeted indicators. In addition, the purpose of the MWM

project was not to identify or test indicators for public reporting, which requires a different

evaluation process; the NQF, which evaluates indicators for public reporting, has endorsed a

number of indicators relevant to palliative and hospice care (only two of the MWM set are

endorsed by the NQF). 11 Finally, the MWM team emphasizes that quality measurement is only

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

12

one small component of ensuring the best hospice and palliative care – attention to structure,

program development, evidence-based practice, education and quality improvement are all also

key to improving care for patients and families.

The MWM list includes cross-cutting indicators in five of the eight NCP domains,

leaving social aspects of care, cultural aspects of care, and care of the imminently dying without

any defined indicators in this set. Although a number of indicators in these domains were

considered by the expert panels, the lack of included indicators in these domains in the final

MWM set reflects the lack of scientific evidence and insufficient documentation in the medical

record for feasible abstraction. In addition, some of these aspects, such as cultural aspects of

care, are less practical for process and outcome quality measurement, which requires issues that

can be objectively measured in standard ways that reflect quality across populations. Potentially,

structural indicators, such as the availability or penetration of chaplaincy services, may be more

appropriate for these domains. Some of the domains, such as physical and ethical and legal

aspects of care (which have the most indicators), are more likely to be documented and easier to

objectively measure than others, such as social aspects of care. For example, whether or not a

patient has life-sustaining treatment preferences documented in their chart is easier to objectively

measure than whether or not the provider developed a comprehensive social care plan,

addressing issues such as relationships and caregiver stress, as recommended by the NCP

guidelines. 6

The lack of included indicators in the domain of care of the imminently dying is a clear

gap critical to hospice and palliative care, where more objective markers exist but further

development is needed. Eight of the nine indicators considered in this domain were outcome

measures of health care utilization (the ninth was family evaluation of bereavement services; the

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

13

global indicators discussed above also address this domain from the patient/caregiver

perspective.). In particular, two NQF-endorsed Quality Oncology Practice Initiative indicators

for cancer patients, whether or not a patient received hospice for less than three days 14, 20 and

proportion with more than one emergency room visit in the last 30 days of life, 20 were

considered throughout the MWM process. However, many concerns about these two Quality

Oncology Practice Initiative indicators were raised by the panels, membership and stakeholder

groups. These measures were tested using Medicare claims data, and have been benchmarked

only for cancer patients; they require knowing about patients’ outcomes outside a health system

and who were within the last 30 days of life, which is generally challenging data to obtain for

palliative care providers not in hospice settings. Outcome indicators also are generally risk-

adjusted and have exclusions, which have not yet been developed for these indicators and could

significantly affect outcomes. They also could be very dependent on local structure, such as

availability of urgent care and the efficiency of hospice admission processes. Finally, these

indicators may not accurately reflect patient and family perceptions of quality of care, especially

across cultures, as studies have found that a majority of caregivers feel that the length of stay in

hospice was sufficient 21 and that caregivers in Japan did not rank emergency department use

highly as an indicator of quality of care. 22

Ideal indicators for care at the end of life could include patient and family preferences for

hospice care and the timing of initial hospice referral and admission accounting for whether any

lag between referral and admission was patient preference or a system performance issue. On the

other hand, outcomes such as hospice length of stay can be integrative and identify potential

issues within a setting that need to be addressed (e.g., very low hospice stays could indicate that

information is not being provided early enough, referral processes are too slow, or coordination

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

14

with hospices is poor). However, these would need further detailed evaluation to identify the

source of the issue before attempts to improve quality can be made.

The MWM process and the indicator set have a number of limitations. The MWM set is

based on an extensive process of expert consensus and diverse input, but will require ongoing

efforts to further evaluate feasibility for routine data collection, usability for quality

improvement projects, links between process and outcome indicators, and updates as new and

revised indicators become available. Using indicators from the set will require careful

monitoring for unintended negative consequences, such as focusing initiatives only on such areas

where indicators are robust and neglecting important areas where performance may be more

difficult to measure. Not all indicators may apply to all settings or populations, and programs

will need to select those that are most appropriate to their internal population, needs, and

opportunities for improvement. Further research with benchmarking and evaluation of the impact

of contextual factors, such as electronic health record system, academic status or quality program

structure, is needed to improve the usability of the indicators. None of the indicators have been

evaluated in pediatrics, and evaluation specific to geriatric populations is limited.

Patient/caregiver evaluations of care are critical to hospice and palliative care, as medical

record review process indicators address only a small fraction of care and cannot measure many

important issues – including the quality of communication, coordination, respect, and attention to

patient preferences. However, there is no universally agreed-upon measure in palliative care;

different measures have been developed and are used in different settings, and no current

indicators we identified that met eligibility criteria crossed settings. The consensus and

stakeholder input processes, therefore, did not endorse any specific measure, and further measure

development is needed, but some process for assessing relevant patient/caregiver input should be

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

15

included in settings providing care to palliative or hospice patients.

Conclusion

In conclusion, the Measuring What Matters Project of the AAHPM and HPNA has

assembled a list of 10 consensus indicators for internal measurement of quality in settings

serving hospice and palliative patients. Although this set serves as an initial starting place for

standardizing U.S. measurement and benchmarks for care, more development and partnerships

with organizations to develop benchmarking is needed, and additional or revised indicators may

be needed for particular settings or populations. Further research is needed to evaluate how best

to define palliative care populations of interest for quality measurement, and how to measure the

quality of hospice and palliative care both overall in different settings and for patients served by

hospice and palliative care services. Finally, as evaluation of this national U.S. indicator set

occurs, consideration of including structural indicators and newly developed indicators and

coordination with lessons learned from individual hospice and palliative care programs in the

U.S., relevant quality programs such as the ASCO QOPI®14 and other emerging international

palliative care quality measurement projects, such as those in Australia,23 Belgium 24 and Te

Netherlands,25 will help to maximize the value of quality measurement in the U.S.

Disclosures and Acknowledgments

This project was funded by the American Academy of Hospice and Palliative Medicine,

with additional support from the Hospice and Palliative Nurses Association.

The authors acknowledge the members of the Technical Advisory Panel: Marie Bakitas,

Teresa Craig, Mary Ersek, Chris Feudtner, Laura Hanson, Arif Kamal, Lisa Lindley, Karl

Lorenz, Carol Spence, Martha Tecca, and Joan Teno; and Clinical User Panel: Michael Balboni,

Patricia Berry, Cynthia Boyd, Caitlin Brennan, Janet Bull, Ira Byock, Barb Daley, Kenneth

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

16

Doka, Jennifer Eurek, Joy Goebel, Elizabeth Gunderson, Krista Harrison, Joan Harrold, Jean

Kutner, Thomas Lee, Suzana Makowski, Kelly McCutcheon Adams, Deirdre Mylod, Marsha

Nelson, Ritika Oberoi-Jassal, Lynn Reinke, Michael Reynolds, Christine Ritchie, Eugenia

Smither, Lisa Stephens, Rodney Tucker, Deborah Waldrop, and Joanne Wolfe. The authors also

acknowledge June Lunney and Steve Smith for their support and guidance.

References

1. Institute of Medicine. Dying in America: Improving quality and honoring individual

preferences near the end of life. Washington, DC: The National Academies Press, 2014.

2. Walling AM, Tisnado D, Asch SM, et al. The quality of supportive cancer care in the veterans

affairs health system and targets for improvement. JAMA Intern Med 2013;173:2071-2079.

3. Fisch MJ, Lee JW, Weiss M, et al. Prospective, observational study of pain and analgesic

prescribing in medical oncology outpatients with breast, colorectal, lung, or prostate cancer. J

Clin Oncol 2012;30:1980-1988.

4. Teno JM, Clarridge BR, Casey V, et al. Family perspectives on end-of-life care at the last

place of care. JAMA 2004;291:88-93.

5. Dy S, Aslakson R, Wilson R, et al. Improving health care and palliative care for advanced and

serious illness. closing the quality gap: revisiting the state of the science. Evidence Report

No. 208. (Prepared by the Johns Hopkins University Evidence-based Practice Center under

Contract No. 290-2007-10061-I.) AHRQ Publication No. 12(13)-E014-EF. Rockville, MD:

Agency for Healthcare Research and Quality, 2012.

6. National Consensus Project for Quality Palliative Care. Clinical Practice Guidelines for

Quality Palliative Care, 3rd ed. 2013. Available at: http://www.nationalconsensusproject.org.

Accessed September 1, 2014.

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

17

7. Center for Medicare & Medicaid Services. Hospice quality reporting. Available at:

http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Hospice-

Quality-Reporting/index.html. Accessed September 23, 2014.

8. Hanson LC, Scheunemann LP, Zimmerman S, Rokoske FS, Schenck AP. The PEACE project

review of clinical instruments for hospice and palliative care. J Palliat Med 2010;13:1253-

1260.

9. Nelson JE, Mulkerin CM, Adams LL, Pronovost PJ. Improving comfort and communication in

the ICU: a practical new tool for palliative care performance measurement and feedback.

Qual Saf Health Care 2006;15:264-271.

10. Campion FX, Larson LR, Kadlubek PJ, Earle CC, Neuss MN. Advancing performance

measurement in oncology: quality oncology practice initiative participation and quality

outcomes. J Oncol Pract 2011;7:31s-35s.

11. National Quality Forum. Palliative and end-of-life care performance measures. Available at:

www.qualityforum.org. Accessed September 1, 2014.

12. The Carolinas Center for Medical Excellence (CCME). PEACE Hospice and Palliative Care

quality measures. Available at: http://www.med.unc.edu/pcare/resources/PEACE-Quality-

Measures. Accessed May 23, 2014.

13. Dy SM, Lorenz KA, O'Neill SM, et al. Cancer quality-ASSIST supportive oncology quality

indicator set: feasibility, reliability, and validity testing. Cancer 2010;116:3267-3275.

14. Campion FX, Larson LR, Kadlubek PJ, Earle CC, Neuss MN. Advancing performance

measurement in oncology: Quality Oncology Practice Initiative participation and quality

outcomes. J Oncol Pract 2011;7:31s-35s.

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

18

15. Lorenz KA, Rosenfeld K, Wenger N. Quality indicators for palliative and end-of-life care in

vulnerable elders. J Am Geriatr Soc 2007;55:S318-326.

16. Walling AM, Asch SM, Lorenz KA, et al. The quality of care provided to hospitalized

patients at the end of life. Arch Intern Med 2010;170:1057-1063.

17. Connor SR, Tecca M, LundPerson J, Teno J. Measuring hospice care: The National Hospice

and Palliative Care Organization national hospice data set. J Pain Symptom Manage

2004;28:316-328.

18. Connor SR, Teno J, Spence C, Smith N. Family Evaluation of Hospice Care: results from

voluntary submission of data via website. J Pain Symptom Manage 2005;30:9-17.

19. Wenger N, Dy SM, Casarett D, et al. American Medical Association - Physician Consortium

on Performance Improvement palliative and end of life care physician performance

measurement set. 2008. Available at: http://www.ama-assn.org/ama/pub/physician-

resources/physician-consortium-performance-improvement/pcpi-measures.page. Accessed

September 4, 2014.

20. Earle CC, Park ER, Lai B, et al. Identifying potential indicators of the quality of end-of-life

cancer care from administrative data. J Clin Oncol 2003;21:1133-1138.

21. Teno JM, Shu JE, Casarett D, Spence C, Rhodes R, Connor S. Timing of referral to hospice

and quality of care: length of stay and bereaved family members' perceptions of the timing of

hospice referral. J Pain Symptom Manage 2007;34:120-125.

22. Miyashita M, Morita T, Ichikawa T, et al. Quality indicators of end-of-life cancer care from

the bereaved family members' perspective in Japan. J Pain Symptom Manage 2009;37:1019-

1026.

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

19

23. Currow DC, Allingham S, Yates P, et al. Improving national hospice/palliative care service

symptom outcomes systematically through point-of-care data collection, structured feedback

and benchmarking. Support Care Cancer 2015;23:307-315.

24. Leemans K, Deliens L, Francke AL, et al. Quality indicators for palliative care services:

mixed-method study testing for face validity, feasibility, discriminative power and

usefulness. Palliat Med 2015;29:71-82.

25. Claessen SJ, Francke AL, Sixma HJ, de Veer AJ, Deliens L. Measuring relatives'

perspectives on the quality of palliative care: the Consumer Quality Index Palliative Care. J

Pain Symptom Manage 2013;45:875-884.

26. Hanson LC, Rowe C, Wessell K, et al. Measuring palliative care quality for seriously ill

hospitalized patients. J Palliat Med 2012;15:798-804.

27. Schenck AP, Rokoske FS, Durham D, Cagle JG, Hanson LC. Quality measures for hospice

and palliative care: piloting the PEACE measures. J Palliat Med 2014;17:769-775.

28. Teno JM, Clarridge B, Casey V, Edgman-Levitan S, Fowler J. Validation of Toolkit After-

Death Bereaved Family Member Interview. J Pain Symptom Manage 2001;22:752-758.

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

20

Table 1. Steps in the Measuring What Matters (MWM) Process for Selecting the Initial Consensus Set of Quality Indicators 1. The MWM team identified existing indicators a. Relevant to U.S. hospice and palliative care, b. Available in the public domain as of October 2013, and c. Developed through a rigorous process and/or tested for reliability and validity 2. The Technical Advisory Panel (TAP) rated indicators on their scientific soundness and

referred a set of indicators (n=34) for review by the Clinical User Panel (CUP). 3. The CUP rated those indicators based on three dimensions: a. How MEANINGFUL is this for patients/families? b. How ACTIONABLE is this for providers/organizations? c. How large is the POTENTIAL IMPACT? 4. The CUP achieved consensus on the top 12 indicators for further input. 5. The draft set of 12 indicators was posted on the American Academy of Hospice and Palliative

Medicine (AAHPM)'s website to elicit feedback from AAHPM and Hospice and Palliative Nurses Association (HPNA) members and their interdisciplinary teams, and asked them to reduce the list of 12 measures down to 10 (or less) of the best measures.

6. The draft set of 12 indicators was distributed to key relevant external organizations and patient advocacy groups for a final set of ratings.

7. The MWM team compiled the input from the membership, teams, organizations, and advocacy groups for the final consensus set of 10 indicators.

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

21

Table 2. Measuring What Matters Top 10 Quality Indicators Set

National Consensus Project Domain

Indicator Source

1. Structure and Processes of Care

Comprehensive Assessment Hospice: Percentage of patients enrolled >7 days for whom a comprehensive assessment was completed within 5 days of admission (documentation of prognosis; functional assessment; screening for physical and psychological symptoms; and assessment of social and spiritual concerns). Seriously ill patients receiving specialty palliative care in an acute hospital setting: Percent of all patients admitted for >1 day who had comprehensive assessment (screening for physical symptoms and discussion of the patient/family’s emotional or psychological needs) completed within 24 hours of admission.

PEACE Set 26, 27

2. Physical Aspects of Care

Screening for Physical Symptoms Percentage of seriously ill patients receiving specialty palliative care in an acute hospital setting >1 day or patients enrolled in hospice >7 days who had a screening for physical symptoms (pain, dyspnea, nausea, and constipation) during the admission visit.

PEACE Set 26, 27

Pain Treatment For seriously ill patients receiving specialty palliative care in an acute hospital setting >1 day or patients enrolled in hospice >7 days who screened positive for moderate to severe pain on admission, the percent with medication or non-medication treatment, within 24 hours of screening.

PEACE Set 26, 27

Dyspnea Screening and Management Percentage of patients with advanced chronic or serious life threatening illnesses that are screened for dyspnea. For those that are diagnosed with moderate or severe dyspnea, a documented plan of care to manage dyspnea exists. (Ambulatory physician care)

AMA-PCPI/ NCQA 19

3. Psychological and Psychiatric Aspects of Care

Discussion of Emotional or Psychological Needs Percentage of seriously ill patients receiving specialty palliative care in an acute hospital setting >1 day or patients enrolled in hospice >7 days with chart documentation of a discussion of emotional or psychological needs.

PEACE Set 26, 27

4. Social Aspects of Care

(No indicators)

5. Spiritual, Religious and Existential Aspects of Care

Discussion of Spiritual/Religious Concerns Percentage of hospice patients with documentation of a discussion of spiritual/religious concerns or documentation that the patient/caregiver/family did not want to discuss.

Deyta, LLC/ NQF # 1647

6. Cultural Aspects of Care

(No indicators)

7. Care of the Patient at the End of Life

(No indicators)

8. Ethical and Legal Aspects of

Documentation of Surrogate Percentage of seriously ill patients receiving specialty

PEACE Set 26, 27

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

22

ACOVE = Assessing Care of Vulnerable Elders 15,

16 End of Life; AMA-

PCPI =American Medical Association Physician Consortium on Performance Improvement; NCQA = National Committee on Quality Assurance; NQF = National Quality Forum; DNR= Do Not Resuscitate. More details on the measures are available at http://aahpm.org/quality/measuring-what-matters. The PEACE measures and supporting materials are available at http://www.med.unc.edu/pcare/resources/PEACE-Quality-Measures.

Care palliative care in an acute hospital setting >1 day or patients enrolled in hospice >7 days with name and contact information for surrogate decision-maker in the chart or documentation that there is no surrogate. Treatment Preferences Percentage of seriously ill patients receiving specialty palliative care in an acute hospital setting >1 day or patients enrolled in hospice >7 days with chart documentation of preferences for life sustaining treatments.

PEACE Set 26, 27

/ NQF # 1641

Care Consistency with Documented Care Preferences IF a vulnerable elder has specific treatment preferences (for example, a DNR order, no tube feeding, or no hospital transfer) documented in a medical record, THEN these treatment preferences should be followed.

ACOVE Palliative Care and End of Life 15, 16

Global Measure

Although no specific global measure was endorsed by the MWM process, the committee, panels, membership, and stakeholders agreed that patient and/or family assessments of the quality of care are a key part of measuring quality for any setting caring for palliative or hospice patients. Global measures that met the MWM selection criteria are described in Table 3.

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

23

Table 3. Candidate Global Measures of Patient and Family Experience Considered by the Measuring What Matters Project Measure Description Source Comments Family Evaluation of Hospice Care 18 (FEHC) (www.nhpco.org/fehc-survey-materials)

The FEHC survey is an after-death survey administered to bereaved family caregivers of individuals who died while enrolled in hospice

NHPCO / NQF # 0208

A new measure for CMS reporting is currently being developed

Family Evaluation of Palliative Care (FEPC) (www.nhpco.org/fepc-survey-materials)

A post-death survey that captures family members’ perceptions about the quality of the palliative care that their loved ones received – by a hospital-based consult service or by a hospice program offering palliative care

NHPCO The questions on the FEPC survey are based on those in the FEHC survey, with wording modifications appropriate to palliative care service delivery.

Bereaved Family Survey (BFS) (www.cherp.research.va.gov/ PROMISE/)

The purpose of this measure is to assess families´ perceptions of the quality of care that Veterans received from the VA in the last month of life. The BFS consists of 19 items (17 structured and 2 open-ended).

PROMISE Center/ NQF # 1623

Used throughout the VA; includes questions specific to Veterans.

Consumer Assessments and Reports of End of Life 28 (CARE) (www.chcr.brown.edu/pcoc/ toolkit.htm)

The CARE survey is a mortality follow-back survey that is administered to the bereaved family members of adult persons (age 18 and older) who died of a chronic progressive illness receiving services for at least 48 hours from a home health agency, nursing homes, hospice, or acute care hospital.

Center for Gerontology and Health Care Research, Brown University/ NQF # 1632

Requires administration by an interviewer.

NHPCO = National Hospice and Palliative Care Organization; NQF = National Quality Forum; PROMISE =

Performance Reporting and Outcomes Measurement to Improve the Standard of care at End-of-life; CMS = Centers

for Medicare & Medicaid Services; VA=Veterans Administration.

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

24

Appendix

Example Questions from the TAP and CUP Surveys