using vignettes to measure and encourage adherence … the forefront (burns 2012). how-ever, what...

56 MANAGED CARE / OCTOBER 2015

Using Vignettes to Measure and Encourage Adherence to Clinical Pathways in a Quality-Based Oncology Network: An Early Report on the Moffitt Oncology Network InitiativeTimothy Kubal, MD, MBA;1 John W. Peabody, MD, PhD, FACP;2 Eliot Friedman, MD;3 Rich Levine, MD;4 Steve Pursell, MD;5 Doug G. Letson, MD1

1Moffitt Cancer Center, Tampa, FL; 2QURE Healthcare, San Francisco, CA and the University of California–San Francisco and Los Angeles; 3Division of Hematology-Medical Oncology, Lehigh Valley Health Network, Allentown, PA; 4Space Coast Cancer Center, Titusville, FL; 5Norton Cancer Institute, Louisville, KY

INtRODUCtION As health care delivery and financing shifts from volume-based to value-based, the aspiration is that financial returns will be achieved by offering services with the best possible quality and outcomes for the lowest possible cost across the continuum of patient care. It is in-creasingly recognized that when value (defined as the patient health outcomes achieved per dollar spent) improves, patients, providers, and payers alike derive benefit from the improvement (Porter 2010).

In accountable care organizations (ACOs), a cornerstone of value-based care, the physician incentive to provide a certain level of quality is ideally based on patient benefit and alignment with economic efficiencies. Provider payment is thus being tied to a defined set of results for quality, access, and efficiency agreed upon by all providers that form the network. When threshold performance levels of quality and cost are met, provid-ers may benefit from shared savings negotiated with payers. Conversely, when threshold performance levels are not met, hospitals and physicians are potentially at risk for reduced pay-ment, no payment, or exclusion from a network. With the promise of higher quality and lower costs, accountable, unified networks and partnerships increasingly have replaced adver-sarial relationships in delivering care (McDonald 2007, Porter 2007). Large centers have shown that integration in care can be done successfully with

Original Research

care coordinated from diagnosis through therapy and follow-up. For specialty care, this shift has driven the importance of building networks of physicians and other health care practitioners who are jointly respon-sible for care (Peabody 2013).

A focus on networks is not a novel idea; in fact, the managed care move-ment in the 1990s brought this idea to the forefront (Burns 2012). How-ever, what was largely missing in the 1990s was an ability to align providers

based on quality, exacerbated by a lack of technological means to measure quality, give feedback to providers in a meaningful way, and link quality to reimbursement. We now know that focusing on quality not only may in-tegrate care effectively but also bring substantial cost savings to the larger health system, as in the experiences of Intermountain Health and Cleveland Clinic (Bohmer 2011, James 2011).

To focus on quality, however, newer provider-oriented strategies

ABstRACt Introduction: The Moffitt Oncology Network (MON) Initiative demon-

strates a way to form a value-based network based upon clinical pathways across a broad geographical area.

Methods: Moffitt Cancer Center (MCC) has developed various cancer- specific pathways. MCC pathways translate evidence-based guidelines into personalized cancer treatment and set a care standard for evaluation and personalized treatment. MCC is using these pathways with other hospital sys-tems and physician groups throughout the MON. Clinical Performance and Value Vignettes, which are virtual patient cases related to the specific clinical pathways, are used to improve the uptake of pathways in the MON. We report here on the baseline data of 66 breast cancer care providers who took 132 breast cancer vignettes. Using the vignettes, variation in care practice is examined, with special attention to use of clinical breast cancer pathways.

Results: Pathway-based clinical care was measured at baseline across MON sites. The mean distributions at baseline varied across all sites and were not statistically significantly different (P>.05). Scores varied by domain across sites, although history and physical scores tended to be higher than work-up, diagnosis, and treatment scores. Pathway adherence also varied for specific diagnostic evaluations or treatments: surgery; sentinel/axillary lymph node dissection; radiation therapy; chemotherapy; and hormonal therapy, and also for the prevalence of unnecessary testing.

Conclusion: Our study suggests that fostering the adoption of breast can-cer clinical pathways into an oncology network is feasible; however, adher-ence to pathways in breast cancer is varied and reducing such variation is a priority as oncology networks continue to grow in popularity.

OCTOBER 2015 / MANAGED CARE 57

Measuring Adherence to Clinical Pathways in a Quality-Based Oncology Network

treatment into specific clinical path-ways based on a patient’s unique clinical profile, using current clinical standards and associated medical evi-dence and integrating Moffitt’s clinical research priorities into an interactive Web-based application. These path-ways also include cost information for chemotherapy regimens and radiation therapy.

To overcome the challenges of adherence to clinical pathways and link network providers with a com-mon quality metric, MCC joined with QURE Healthcare to develop and ad-minister vignettes related to specific clinical pathways in breast cancer. The vignettes were given to participating providers in a serial measurement and

that engage physicians in system-wide initiatives of routine measure-ment, feedback, and learning are needed (Mountford 2012, Meyer 2012, Biller-Andorno 2013). Simula-tions of a patient-provider interac-tion, such as Clinical Performance and Value (CPV) Vignettes (hereaf-ter “vignettes”) have been shown to measure real practice data and offer promise in this area as rapid feedback on simulated patients, done serially, promotes continuous learning and resonates with professional purpose (Peabody 2000, Dresselhaus 2000, Peabody 2004, Peabody 2011, Pea-body 2014).

Moffitt Cancer Center (MCC), located in Tampa, Fla., is a National Cancer Institute Comprehensive Cancer Center that has developed a collaborative network consisting of hospitals, health care systems, and community-based practices that agree to follow Moffitt’s Clinical Pathways, collaborate on quality initiatives, and participate in clinical research. Each site emulates Moffitt’s model of care through interdisciplinary, disease-focused clinics and prospective peer review. To further standardize clini-cal care and quality, vignettes were developed to measure provider per-formance against explicit, evidence-based standards in Moffitt’s Clinical Pathways. The tool outlines a patient’s optimal clinical care based on best practices specific to each patient’s situ-ation and treatment aims.

As part of the Moffitt Oncology Network (MON) initiative, vignette measurement and feedback of clinical pathway compliance was launched at MCC and then at 3 partner-network hospitals/facilities: Norton Cancer Institute (Louisville, Ky.), Lehigh Valley Health Network (Allentown, Pa.), and Space Coast Cancer Cen-ter (Titusville, Fla.). At these institu-tions, measurement and feedback to the multidisciplinary team of medi-cal oncologists, radiation oncologists,

surgeons, and advanced practice pro-viders occurs every 4 months, starting with breast cancer vignettes.

We report here on our baseline findings in breast cancer at 4 Moffitt Oncology Network sites and discuss how the measurement and feedback system could improve clinical qual-ity and decrease costs in the care of patients with breast cancer.

MEthODs/APPROAChSetting: MCC collaborates with other hospital systems and physician groups through the Moffitt Oncology Net-work. The network uses Moffitt’s Clinical Pathways, a decision-making tool that translates evidence-based guidelines for personalized cancer

KEy POINts• Value-based payment (for example, shared savings) to ACOs and other

organizations tie provider compensation to outcomes and quality of care.

• Getting clinicians to follow evidence-based clinical pathways is one way to improve outcomes and quality of care and to manage costs, but the experience with clinical pathways has been mixed.

• Computerized vignettes that simulate different patient scenarios and ask clinicians to make decisions can be used to gauge adherence to clinical pathways and to educate physicians.

• For this study, 66 providers at 4 different sites agreed to complete 2 vignettes every 4 months for 6 rounds over a period of 20 months. The vignettes consisted of 12 different cases that required the provider to evaluate a simulated female patient for breast cancer or suspected breast cancer.

• This study reports baseline results after the providers had gone through just the first 2 vignettes (132 total).

• Scores measuring adherence to the pathway adherence varied by domain across the 4 sites, although the scores for history taking and physical examination tended to be higher than those for work-up, diagnosis, and treatment scores.

• Pathway adherence for evaluation and treatment varied widely by site. For example, for radiation, 83% of the providers at one site were adherent but at another site only 11% were. The range for chemotherapy was narrower: from 50% at one site to 40% at another.

• At all 4 sites, 50% or more of the providers ordered unnecessary tests or procedures as they went the vignettes.

• Future analyses will examine the effects of vignettes using data from rounds 2 to 6 in breast cancer and other clinical areas, such as lung and gastrointestinal cancers.



REsUlts At baseline, 66 breast cancer care providers took 132 breast cancer vi-gnettes. The provider characteristics (Table 1) demonstrate a broad range of training and experience, with some providers exclusively treating breast cancer patients and others having a mixed oncology practice.

The CPV vignettes consisted of 12 different online cases that required the provider to evaluate a simulated female patient for breast cancer or suspected breast cancer. Providers were required to take a history and physical exami-nation; they were then asked to order whatever test they felt was indicated, make a diagnosis, and then provide treatment, any follow-up testing they required, and follow-up care. Upon completing the history, the physical, and the testing, clinical responses (eg, the result of a biopsy) were provided in real time. Providers completed their cases, on average, in less than 25 min-utes across all the sites. Once all the vignettes in any given site were com-pleted, the free-text responses were scored by a team of trained physicians using explicit predetermined criteria. Scoring was done by 2 physicians blinded to the caregiver and to each other to check for accuracy. Individual feedback forms, which were always provided confidentially, were prepared for each provider for each case they completed. Feedback forms bench-marked where individual providers scored in relation to other providers in

criteria for each case. The provider re-ceived confidential feedback on each vignette, which included an overall score and domain scores, as well as recommendations for improvement and links to relevant clinical guide-lines and medical literature.

Objectives: We examine the base-line variation in breast cancer care as captured by the vignettes, paying special attention to the use of un-necessary tests and provision of care that deviate from MCC’s clinical path-ways. We discuss the implications of this initiative on improving adherence to pathways as MCC begins to use this platform to align providers and build a network based on quality.

Analysis: We examined overall distributions of the vignette scores and then evaluated distributions by domain score. The small sample size limited our ability to disaggregate data by provider type.

Using the vignettes, we also ex-amined adherence to MCC breast cancer clinical pathways in the fol-lowing 5 areas: surgery; sentinel/axillary lymph node management; radiation therapy; chemotherapy, and hormonal therapy.

We also examine the prevalence of unnecessary testing, defined as ordering a test whose result would not change clinical management as identified in the breast cancer clinical pathways. From our vignettes, we are able to estimate the average number of unnecessary diagnostic tests ordered.

feedback system developed and vali-dated by QURE Healthcare (Peabody 2000, Dresselhaus 2000, Peabody 2004). Vignettes are administered ev-ery 4 months. First launched in breast cancer in March 2013 at MCC, breast cancer vignettes were then introduced to interested partner institutions in July 2013 (Lehigh Valley Health Net-work, Norton Cancer Institute, Space Coast Cancer Center).

Study population: As part of this initiative, all breast cancer providers (69 medical oncologists, radiation oncologists, surgeons, and advanced practice providers) at the participat-ing sites were asked to complete 2 vi-gnettes every 4 months for 6 rounds over a period of 20 months. Sixty-six providers at 4 sites agreed to par-ticipate and completed the vignettes (96% participation).

Measurement and feedback system: Using the MCC clinical pathways, 12 vignettes were written to address di-agnostic, therapeutic, and cost chal-lenges in cancer care.

Vignettes have been previously validated as a measure of the provid-er’s ability to evaluate, diagnose, and treat specific diseases and conditions (Peabody 2000, Dresselhaus 2000, Peabody 2004). Each vignette takes approximately 25 minutes to com-plete and asks the provider to respond online to open-ended questions as they proceed through a patient visit. Participation was not compensated. We report data from the first round of administration within the network. Trained physician abstractors, blinded to the vignette-taker’s identity, scored each vignette using MCC pathways to determine the prevalence of on- and off-pathway care and domain mea-sures of overall care in history tak-ing, physical exam, lab and imaging studies ordered, diagnostic accuracy, and treatment plan (domain score). Scores generated from the cases were reported as the percentage of criteria correct divided by the total number of

tABlE 1Participating provider characteristics

site #1 site #2 site #3 site #4Number of providers who participated 17 17 22 10• Medical oncologist 7 16 12 8• Radiation oncologist 0 1 5 2• Surgeon 4 0 2 0• Advanced practice providers 6 0 3 0Average number of years of experience treating breast cancer 12.6 21.1 11.7 13.7

Average number of breast cancer cases seen per week 34.1 17.6 12.9 10.8



delivery models, such as specialty ACOs, shared risk payments, and oncology patient-centered medical homes (Porter 2007).

In 2009, Moffi tt Cancer Center (MCC) launched an intensive eff ort to introduce evidence-based clinical pathways among its own providers and among providers in its oncol-ogy network. Th e key goals were to advance internal and cross-system discussions of quality and value, which included assuring that patients receive the most appropriate person-alized care to optimize their desired outcomes, identifying and reducing practice variation, streamlining pa-tient-care processes, increasing the use of generic drugs when available for chemotherapy, and advancing sup-portive care.

Th e challenge for MCC and other

ways in the following 5 areas: surgery; sentinel/axillary lymph node dissec-tion; radiation therapy; chemother-apy, and hormonal therapy, along with the prevalence of unnecessary testing.

DIsCUssIONIn its seminal 2013 report Delivering High Quality Cancer Care: Charting a New Course for a System in Crisis, the Institute of Medicine called for linking payment to quality measures. Th rough its recomendations, the IOM envisioned a system that rewards can-cer care networks for the provision of high-quality, patient-centered care and discourages ineff ective interven-tions that contribute to costs in the fee-for-service environment. Payers and providers across the country are experimenting with a variety of ways to measure quality as they shift to new

their sites. Th e feedback also provided customized suggestions on where the provider might improve in the care of the simulated patient. Unnecessary testing was also fl agged and fed back to the provider (see summaries of the 12 CPV vignette cases in Box 1).

We examined overall score distri-butions of the vignettes by site (Figure 1) and then by domain score (Figure 2, page 62). Th e mean distributions var-ied across all sites and were not statis-tically signifi cantly diff erent (P>.05). Scores varied by domain across sites, although history and physical scores tended to be higher than work-up, di-agnosis, and treatment scores. As seen in Figure 2, there was wide variation in each domain. Pathway adherence for specifi c diagnostic evaluations or treatments (Table 2, page 62) shows the adherence to MCC clinical path-

FIGURE 1Distributions of breast cancer CPV vignette scores, 2013

Note: CPVs are scored based on whether providers followed MCC breast clinical pathways. Scores generated from the cases are reported as the percentage of criteria correct divided by the total number of criteria for each case. Scoring of the vignettes was completed by a team of trained physicians who were blinded to the identities of the providers taking the vignettes.

0

1

2

3

4

5

6

7

8

9

20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

0

1

2

3

4

5

6

7

8

9

20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

0

1

2

3

4

5

6

7

8

9

20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

0

1

2

3

4

5

6

7

8

9

20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

site 1

site 3

site 2

site 4

Num

ber o

f vig

nett

esN

umbe

r of v

igne

ttes

CPV scoreCPV score

CPV scoreCPV score

Num

ber o

f vig

nett

esN

umbe

r of v

igne

ttes



BOx 1Breast cancer CPV vignette case description

CaseAge of patient Description Diagnosis

Diagnostic teaching points

treatment teaching points Unnecessary items

1 52 Abnormal mammogram (BIRADS 3) but no abnormal findings on history and PE; patient on hormone replacement

Dense breasts, benign

• No biopsy, further imaging (breast ultrasound or MRI) appropriate but not necessary

• Reassurance• Advise to stop hormone

replacement

• Axillary work-up• Core needle biopsy• Surgery/RT/chemo-

therapy• Genetic testing/coun-

seling

2 55 Abnormal mammogram (BIRADS 4), no abnormal findings on history and PE

Atypical ductal hyperplasia, benign

• Core needle biopsy required to rule out cancer

• Excisional biopsy • No lymph node biopsy/

dissection• Chemoprophylaxis with

tamoxifen

• Breast ultrasound• Breast MRI• Axillary ultrasound

and FNA• Metastatic work-up• Mastectomy• Sentinel node biopsy/

axillary node dissec-tion

• Aromatase inhibitors• Radiation therapy• Chemotherapy

3 56 Presents with new breast mass, apparent breast cancer with enlarged axillary node

Diffuse large B cell NHL, Stage IIIAE No breast cancer

• Mammogram and breast ultrasound, axillary ultrasound needed

• Chest/Abd CT, PET CT for staging

• Excisional biopsy with immunohistochem-istry

• Rituximab-CHOP regimen• CNS prophylaxis• Repeat CT scans for

monitoring

• FNA of breast mass (nodal FNA showed lymphoma)

• Genetic testing• Breast irradiation• Axillary node dissec-

tion

4 60 Apparent metastatic disease in a woman s/p Stage II at 24-month follow-up

Hyper para-thyroid ism from adenomaNo progression of breast cancer

• Bilateral mammo-gram to rule out recurrence (negative)

• Recognize elevated serum calcium

• Check parathyroid hormone levels

• Bone densitometry

• Parathyroidectomy or cinacalcet

• Bisphosphonates• Continue anastrozole

• Breast ultrasound and MRI

• Axillary ultrasound• Metastatic work-up• Chemotherapy for

breast cancer

5 57 Presents with a small (<2cm) mass with abnormal mammogram, HER2 equivocal

Stage IA breast cancer

• Breast and axillary ultrasound necessary

• Need for HER2 FISH due to equivocal IHC result

• Oncotype DX neces-sary to determine risk for recurrence/need for cytotoxic chemo

• Lumpectomy/MRM with SLN (negative)

• Post-op RT and hormonal therapy with anastrozole

• No chemotherapy (low Oncotype DX score)

• Metastatic work-up (PET/CT of chest, abdomen, pelvis)

6 63 Diagnosed with CHF, presents with new moderate size, 2.5 cm mass with abnormal mammogram

Stage II breast cancer (clinical)

• Mammogram and axillary ultrasound/FNA (due to palpable axillary mass) neces-sary

• Need for metastatic workup

• Cardiac evaluation

• MRM or lumpectomy with ALND showed 4/20 posi-tive nodes

• Docetaxel and cyclophos-phamide (no anthracycline due to cardiac disease)

• Post-op RT and hormonal therapy with anastrozole

• Breast MRI• Sentinel lymph node

biopsy • Oncotype DX (due to

positive nodes)• Neoadjuvant chemo-

therapy

Key: PE=physical examination FNA=fine needle aspiration BIRADS=Breast Imaging Reporting and Data System (categories 0-4)RT=radiation therapyAbd=abdominal NHL=non-Hodkin lymphoma

CNS=central nervous system FISH=Fluorescence in situ hybridizationIHC=ImmunoHistoChemistryOncotype DX=Oncotype DX Breast Cancer AssayALND=axillary lymph node dissection MRM=modified radical mastectomy



BOx 1Breast cancer CPV vignette case description

CaseAge of patient Description Diagnosis

Diagnostic teaching points

treatment teaching points Unnecessary items

7 38 Presents with new moderate size, 3 cm mass, no history of mammography HER2+ Disease

Stage II breast cancer (clinical)

• No palpable axillary nodes

• Mammogram positive but axillary ultra-sound negative

• Indications for ge-netic testing

• Patient positive for BRCA2 mutation

• MRM with SLN (positive) --> ALND (2/20 positive nodes)

• Contralateral mastectomy (BRCA mutation carrier)

• ACTH (TCH is the ac-cepted alternate)

• Nodal FNA (no enlarged node on ultrasound)

• Metastatic work-up• Lumpectomy• Radiation therapy

8 38 Presents with moderate size, 2.5 cm mass, triple-negative disease, s/p incomplete excisional biopsy

Stage II multifocal breast cancer

• Mammogram, axillary ultrasound necessary

• Sentinel node biopsy positive

• Metastatic work-up necessary

• MRM, multiple tumors, ALND (5/16 nodes posi-tive)

• Post-op radiation• Fertility counseling• Chemotherapy with

dose dense Adriamycin, cyclophosphamide then paclitaxel

• Breast MRI• Neoadjuvant

chemotherapy• Bilateral

oophorectomy• Contralateral

mastectomy

9 42 Treated for HER2+ Stage II breast cancer (chemo RT) presents at 36 months with headache and supraclavicular node

Stage II breast cancer with recurrent disease

• Normal mammogram with metastasis in supraclavicular node

• Metastatic workup showed hilar and liver metastases

• Cardiac evaluation• Preferred chemo with

pertuzumab, trastuzum-ab, docetaxel

• No hormonal regimens

• Breast biopsy (no apparent local recurrence)

• Ado-trastuzumab• Hormonal therapy

10 42 Presents with new onset >5cm mass which on biopsy is ER/PR+, HER2- invasive breast cancer

Stage III breast cancer (clinical)

• Tumor >5cm with negative axillary ultrasound

• Sentinel lymph node biopsy needed (positive)

• Need for mets workup (negative)

• Neoadjuvant chemotherapy with ACT followed with lumpectomy and ALND (3/22 positive nodes)

• Post-op radiation therapy• Delayed breast

reconstruction preferred• Follow-up mammography

q6 months

• Breast MRI• Bone scan• Tumor markers• RT if patient decides

on mastectomy over lumpectomy

• Bilateral oophorectomy

• Contralateral mastectomy

11 60 Status post Stage III 10 years ago with new-onset back pain

Recurrent breast cancer, metastatic to thoracic spine and ribs with compression fracture but no cord compression

• Metastatic workup (includes full spine MRI) positive for T-5 and rib disease

• Other met workup negative

• Cardiac evaluation required

• Consultations with neurosurgery, intervention radiology and radiation oncology

• Treatment options for spine metastasis with no apparent cord compression: vertebro/kyphoplasty, radiation, bisphosphonates

• Hormonal therapy with another aromatase inhibitor (previously on anastrozole

• Patient/physician decision on chemotherapy

• Lumbar tap• Work-up for

pulmonary embolism or anemia (anemia consistent with chronic disease)

• Steroids (no cord compression/neurologic deficits)

12 42 Treated for Stage IIIB HER2+ breast cancer 2 years ago, now presents with new onset headaches

Stage IIIB with recurrent disease to brain, liver

• Normal neurologic exam

• Cranial MRI/CT shows new parietal mass, Abdominal CT 2–3 cm mets to the liver, other met workup negative

• Tissue confirmation with CT-guided biopsy of liver, followed by complete staging necessary

• Treatment for brain mets: dexamethasone and RT

• Offer systemic treatment with pertuzumab, trastuzumab, docetaxel

• Emphasize palliative and supportive care

• Surgical resection of brain and liver metastasis

• Third line chemo: lapatinib, capecitabine

continued



scores revealed a similar distributions across all of the institutions, with large variation in the work-up, diagnosis, and treatment. The large variation in treatment scores is notable, as is the variation in adherence to MCC pathways in the 4 different sites. Fu-ture analyses will examine data from rounds 2 to 6 in breast cancer and other clinical areas, such as lung and gastrointestinal cancers. We did not make statistical comparisons across the sites, as the goal of this initiative was to examine and improve pathway adherence collectively at all sites.

Other studies, such as a recently published report among cardiologists in a large Northwestern U.S. health care system, have shown that the use of CPVs to measure variability among providers is similarly feasible and successful in demonstrating the variability of care practice. Like this study, provider engagement is high,

health care systems has been to en-gage providers to use their pathways. Research suggests oncologists tend to find pathways influential and favor their use (Farina 2012). However, the rate of actual adoption of pathways into practice is variable, is often quite low and is the subject of active inves-

tigation (Casey 2013). To address this challenge, CPV vi-

gnettes were introduced to encour-age adherence to pathways, determine their feasibility, and promote a culture fostering measurement and learning improvement throughout MCC and its network sites. At baseline, CPV

tABlE 2Unnecessary testing and adherence to MCC clinical pathways

site #1 (n=17)

site #2(n=17)

site #3(n=22)

site #4(n=10)

Percent of providers ordering unnecessary tests/procedures 50% 55% 81% 62%

Average number of unnecessary/tests procedures (among those who ordered unnecessary tests)

1.6 1.7 1.7 1.8

Adherence to MCC pathways• Surgery 69% 33% 52% 40%• Axillary/sentinel lymph node dissection 31% 42% 39% 50%• Radiation therapy 83% 11% 57% 50%• Chemotherapy 40% 28% 50% 50%• Hormonal therapy 79% 67% 71% 60%

FIGURE 2Distributions of breast cancer CPV vignette scores by domains

020

4060

8010

0Pe

rcen

t (%

)

Total score History Physical exam Workup Diagnosis Treatment


020

4060

8010

0

020

4060

8010

0Pe

rcen

t (%

)

excludes outside values


020

4060

8010

0Pe

rcen

t (%

)

excludes outside values


Note: Boxplots show the variation in scores. The bottom of the box is the first quartile (Q1) to the third quartile (Q3). Within the box, the line denotes the median of the data. The ends of the whiskers denote the minimum and maximum values of the data.

site 1

site 3

site 2

site 4



oncology offer a method to reduce unnecessary and costly treatment variation, their success relies on physician participation. In this study, strong leadership and institutional commitment to pathway adherence helped to make voluntary participa-tion very high. The perception among some physicians that pathways rep-resent “cookbook medicine” is a bar-rier to acceptance of pathways, as are constraints on physcians’ time and discomfort with changing practice patterns. The counter point is that without accountability, providers may not—and, typically in other studies, have not—complied with pathway programs, suggesting a need for a vignette-based or similar approach and other research on how to intro-duce collaborative efforts that engage physicians.

sUMMARyWe present an initiative that uses evidence-based pathway care to build a network based on quality and a cul-ture of measurement and feedback. Contemporary quality improvement initiatives have begun calling for in-novative ways to approach health care delivery and teaching. At the core of this shift is increased attention to learning how quality-improvement efforts can engage physicians to ex-amine their practice performance in a systematic way. This movement has also affected academic continuing medical education (CME) and would link quality improvement to provider education so they would not be pur-sued as separate activities.

In this study, we used measurement and feedback with the vignettes as a way to identify areas of quality gaps and as a bridge to education. With this initiative, the MCC worked with 3 network member institutions to iden-tify areas of improvements in quality for breast cancer; this is now being expanded to lung and gastrointesti-nal cancers. As we continue to collect

require uniform IT systems, as they operate on their own data platform, and can be written to address such network-related issues as referrals and adoption of other quality initiatives. With the successful implementation of the CPVs, the MON is providing feedback to providers in 2 ways: in-dividually and confidentially to the provider on pathway adherence for each case completed, and to the group as a whole in a conference setting to discuss the areas (e.g. work-up of the axilla) where there is the most dis-parity. By doing feedback repeatedly, providers have a chance to absorb the individual and group feedback and improve their performance.

Evidence suggests that improving pathway adherence reduces costs and improves outcomes (Neubauer 2010, Smith 2011, Butcher 2010, Hoverman 2011). The Highmark Blue Cross Blue Shield program, for example, showed that the total cost of care for patients under pathways practices grew at a slower rate than did the cost for patients who were not in pathways program (Butcher 2010). Outpatient costs were 35% lower for patients with non–small-cell lung cancer treated on a clinical pathway compared with those who received nonpathway treat-ment. CareFirst reduced its costs by 15% using a clinical pathways pro-gram for breast, lung, and colon can-cers, resulting primarily from a 7% decline in emergency room visits, shorter hospital stays, increased use of generic medications, and more ap-propriate use of chemotherapy (Hov-erman 2011). Newer reports from a large NIH-funded experimental study indicate that when CPV vignettes are used serially, they change practice and result in better patient outcomes (Pea-body 2011, Peabody 2014). Further research is needed to determine if this is the case in the MON with the MCC pathways.

Implementation challenges ex-ist. First, while clinical pathways in

with comments on the feedback, the quality criteria, and differences among colleagues dominating dis-cussions (Peabody 2015). Other large cancer care centers are looking for ways to transform the culture of measuring and improving evidence-based care. City of Hope, for example, is implementing a continuing medical education (CME)-based initiative that aims to link education with improving performance to meet current stan-dards of care (Uemura 2013). Much like CME, MCC pathways are a learn-ing tool, offering the best evidence for therapeutic equivalence or superior-ity by highlighting benefits and costs. They are intended to manage patient care in real time by documenting local practice patterns and by prospectively defining specific treatment strategies.

With their use in the MON, the initiative to use vignettes went a step further, requiring buy-in from other institutions. With the measurement system in place, we hope to create the groundwork for quality as the basis for forming partnerships with other hospitals to provide high-quality care to ever-larger numbers of pa-tients. Vignettes are ideally suited for large-scale cross-system mea-surement in networks. They do not

Corresponding author

John W. Peabody, MD, PhD, FACP President, QURE Healthcare 450 Pacific Avenue San Francisco, CA 94123 (415) 226-4332 [email protected]

Funding source: Moffitt Cancer Center

Disclosure statement: Peabody is presi-dent of QURE Healthcare, which owns the quality measurement used in the initiative described.

Acknowledgements: The authors wish to acknowledge Riti Shimkhada, PhD, for her exceptional assistance with the drafting and preparation of this manu-script.



serial data, our next steps will be to examine whether this initiative has improved clinical care, increased adherence to clinical pathways, reduced unnecessary testing, become integrated into the culture of practice at MCC and its partners, and demonstrated value to payers.

REFERENCEsBiller-Andorno N, Lee TH. Ethical physician incentives—from carrots and sticks to

shared purpose. N Engl J Med. 2013;368(11):980–982.Bohmer RM. The four habits of high-value health care organizations. N Engl J Med.

2011;365(22):2045–2047.Burns LR, Pauly MV. Accountable care organizations may have difficulty avoiding

the failures of integrated delivery networks of the 1990s. Health Aff (Millwood). 2012;31(11):2407–2416.

Butcher L. Will clinical pathways work? Biotechnol Healthc. 2010;7(3):16–20.Casey DE Jr. Why don’t physicians (and patients) consistently follow clinical practice

guidelines? JAMA Intern Med. 2013;173(17):1581–1583.Dresselhaus TR, Peabody JW, Lee M, et al. Measuring compliance with preventive

care guidelines: standardized patients, clinical vignettes, and the medical re-cord. J Gen Intern Med. 2000;15(11):782–788.

Farina K. How payers and oncologists really feel about oncology pathways. Am J Manag Care. 2012;18(4):SP188.

Hoverman JR, Cartwright TH, Patt DA, et al. Pathways, outcomes, and costs in co-lon cancer: retrospective evaluations in two distinct databases. J Oncol Pract. 2011;7(3 suppl):52s–59s.

IOM (Institute of Medicine). Delivering High Quality Cancer Care: Charting a New Course for a System in Crisis. Washington, DC: IOM. 2013.

James BC, Savitz LA. How Intermountain trimmed health care costs through robust quality improvement efforts. Health Aff (Millwood) 2011;30(6):1185–1191.

Meyer GS, Nelson EC, Pryor DB, et al. More quality measures versus measuring what matters: a call for balance and parsimony. BMJ Qual Saf. 2012;21(11): 964–968.

McDonald KM, Sundaram V, Bravata DM, et al. Care coordination. In: Shojania KG, McDonald KM, Wachter RM, Owens DK, eds. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies, vol 7. Technical Review 9. AHRQ Publication No. 04(07)-0051-7. Rockville, MD: Agency for Healthcare Research and Quality; 2007.

Mountford J, Shojania KG. Refocusing quality measurement to best support quality improvement: local ownership of quality measurement by clinicians. BMJ Qual Saf. 2012;21(6):519–523.

Neubauer MA, Hoverman JR, Kolodziej M, et al. Cost effectiveness of evidence-based treatment guidelines for the treatment of non–small-cell lung cancer in the community setting. J Oncol Pract. 2010;6(1):12–18.

Peabody J, Shimkhada R, Quimbo S, et al. Financial incentives and measurement improved physicians’ quality of care in the Philippines. Health Aff (Millwood). 2011;30(4):773–781. Erratum in: Health Aff (Millwood). 2011;30(6):1217.

Peabody JW, Huang X. A role for specialists in resuscitating accountable care organi-zations. Harvard Business Review. November 2013.

Peabody JW, Huang X, Shimkhada R, Rosenthal M. Managing specialty care in an era of heightened accountability: emphasizing quality and accelerating savings. Am J Manag Care. 2015;21(4):284–292.

Peabody JW, Luck J, Glassman P, et al. Comparison of vignettes, standardized pa-tients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000;283(13):1715–1722.

Peabody JW, Luck J, Glassman P, et al. Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Ann Intern Med. 2004;141(10):771–780.

Peabody JW, Shimkhada R, Quimbo S, et al. The impact of performance incentives on child health outcomes: results from a cluster randomized controlled trial in the Philippines. Health Policy Plan. 2014;29(5):615–621.

Porter ME. What is value in health care? N Engl J Med. 2010; 363(26):2477–2481.Porter ME, Teisberg EO. How physicians can change the future of health care.

JAMA. 2007;297(10):1103–1111.Smith TJ, Hillner BE. Ensuring quality cancer care by the use of clinical practice

guidelines and critical pathways. J Clin Oncol. 2011;19(11):2886–2897.Uemura M, Morgan R Jr, Mendelsohn M, et al. Enhancing quality improvements in

cancer care through CME activities at a nationally recognized cancer center. J Cancer Educ. 2013;28(2):215–220.

using vignettes to measure and encourage adherence … the forefront (burns 2012). how-ever, what...

Documents