INTERVENTIONS TO REDUCE INAPPROPRIATE TEST UTILIZATION

Diagnostic Error in Medicine

12 November 2012

Paul L Epner

2

TRENDS SUGGEST INCREASED DIAGNOSTIC ERRORS

Aging population means more diagnoses

Increasing chronic comorbidities mean increased diagnostic complexity

Decreasing number of primary care physicians combined with emphasis on “cost effectiveness” means less time with patients

Anecdotal evidence of reduced skills in taking history and conducting physical

Diagnosis is an evolving term

3

©2

01

2 Pa

ul E

pner LLC

DEFINITION OF DIAGNOSIS IS EXPANDING

The cause of symptoms (traditional) The condition’s subtype (for best treatment)

Antimicrobial susceptibility testing Tumor typing

The body’s likely response to treatments The stratification of risk

4

©2

01

2 Pa

ul E

pner LLC

THE ROLE OF LABORATORY TESTING IN DIAGNOSIS IS LIMITED BUT IMPORTANT AND LIKELY INCREASING

In a study of 248 hospitalized patients, 246 had definitive diagnosis within 3 months of hospitalization.

The primary determinant of diagnosis for 215 with “exact” in-hospital diagnosis was: History and Physical – 48.4% Radiologic exam – 33.5% Blood test or culture – 9.8%

Study limitations did not examine diagnostic error did not examine time to diagnosis did not examine appropriate use of diagnostic

toolsSource: Wahner-Roedler, D. L.et al. (2007). Who makes the diagnosis? The role of clinical skills and diagnostic test results. Journal of evaluation in clinical practice, 13(3)

5

©2

01

2 Pa

ul E

pner LLC

OLDER STUDIES YIELD COMPARABLE RESULTS

80 prospective outpatient cases Final diagnosis made

Following history - 61 (76%) Following physical – 10 (12%) Following laboratory – 9 (11%)

Confidence in diagnosis rose with more information Following history – 7.1 (scale of 1 to 10) Following physical – 8.2 Following laboratory – 9.3

Some evidence that skill in conducting history and physical is decreasing while reliance on data is increasing

Source: M.C. Peterson, J.H. Holbrook, D. Von Hales, N.L. Smith, and L.V. Staker, “Contributions of the history, physical examination, and laboratory investigation in making medical diagnoses.,” The Western journal of medicine, vol. 156, Feb. 1992.

6

©2

01

2 Pa

ul E

pner LLC

N= 583 Cases

G. D. Schiff et al., “Diagnostic error in medicine: analysis of 583 physician-reported errors.,” Archives of internal medicine, vol. 169, no. 20, pp. 1881-7, Nov. 2009.

THE ROLE OF TESTING IN DIAGNOSTIC ERRORS IS SIGNIFICANT

7

©2

01

2 Pa

ul E

pner LLC

U.S. MALPRACTICE CASES CONFIRM SIGNIFICANCE

Source: T. GANDHI, A. KACHALIA, E.J. Thomas, A.L. Puopolo, C. Yoon, T. Brennan, and D. Studdert, “Missed and delayed diagnoses in the ambulatory setting: A study of closed malpractice claims.,” Annals of internal medicine, vol. 145, 2006.

Of 307 closed cases (ambulatory) studied because they alleged missed or delayed diagnosis, 181 did involve diagnostic errors that harmed patients

8

©2

01

2 Pa

ul E

pner LLC

TRADITIONAL LABORATORY QUALITY MEASURES ARE NOT SPECIFIC FOR PATIENT HARM OR DIAGNOSTIC ERRORS

Prolonged turn-around time Error logs

Missing ID, Hemolysis, Short fills, Interface error logs, Incomplete requisitions, uncollected samples, order entry errors, lost specimens, contaminated specimens

Incident reports Corrected result reports

9

A FRAMEWORK FOR LABORATORY-RELATED DIAGNOSTIC ERRORS HAS BEEN DEFINED*

Inappropriate test is ordered Appropriate test is not ordered Appropriate test result utilization is delayed Appropriate test result is not properly utilized

Knowledge deficit Failure of synthesis Misleading result Systematic failure

Appropriate test result is wrong

*Adapted from P Epner and M Astion, “Focusing on Test Ordering Practices to Cut Diagnostic Errors,” Clinical Laboratory News, vol. 38, no. 7, July 2012

©2

01

2 Pa

ul E

pner LLC

10

THE FRAMEWORK GUIDES INTERVENTIONS

Inappropriate test ordered or appropriate test not ordered CPOE design and monitoring Algorithms, clinical pathways, guidelines Reflex testing Data mining Inter-physician variance analysis Resource utilization committee

©2

01

2 Pa

ul E

pner LLC

11

THE FRAMEWORK GUIDES INTERVENTIONS

Test result not utilized properly or fully Interpretive comments EMR interface Real-time triggers

Test result delayed or not retrieved Process monitor Discharge monitor

Appropriate test result is wrong Delta checks Controls/Calibrations Autoverification Second read (AP)

©2

01

2 Pa

ul E

pner LLC

12

REQUISITION DESIGN

Design changes focused on medical necessity, reduction in panels, test groupings linked to specialty, etc.

Reduction in tests per visit occurred

No assessment of impact on Dx errors was made

Source: J.F. Emerson and S.S. Emerson, “The impact of requisition design on laboratory utilization,” American Journal of Clinical Pathology, vol. 116, Dec. 2001.

13

©2

01

2 Pa

ul E

pner LLC

CLINICAL DECISION SUPPORT/BEST PRACTICE ALERTS

Source: Jones, Jay, “Lab Enterprise Analytics,” Executive War College 2009

14

©2

01

2 Pa

ul E

pner LLC

DIAGNOSTIC ALGORITHMS

Clinical variables drive six distinct but potentially overlapping algorithms for prolonged PTT

Evaluation preoperatively of an asymptomatic prolonged PTT Evaluation of a persistently prolonged PTT with bleeding Evaluation of a persistently prolonged PTT without bleeding Evaluation of an elderly patient without bleeding history

accompanied by sudden development of soft tissue hematomas and/or persistent and significant gastrointestinal or genitourinary hemorrhage

Evaluation of hospitalized newborn with prolonged PTT Evaluation of a unexplained prolonged PTT following multiple,

appropriate workups; searching for rare diagnoses

Source: Tcherniantchouk, O., Laposata, M., & Marques, M. B. (2012). The isolated prolonged PTT. American journal of hematology.

15

Developed by the Centers for Disease Control with the support of the Algorithm Subgroup of CLIHC™

16

17

18

©2

01

2 Pa

ul E

pner LLC

REFLEX AND REFLECTIVE TESTING

Creating protocols for the sequential addition of tests based on earlier results reduces diagnostic delays and patient inconvenience while reducing test volume

Reflex testing can improve diagnostic accuracy

The improvement in diagnostic accuracy is linked to the threshold criteria and varies with the clinical scenario

Source: R. Srivastava, W. a Bartlett, I.M. Kennedy, A. Hiney, C. Fletcher, and M.J. Murphy, “Reflex and reflective testing: efficiency and effectiveness of adding on laboratory tests.,” Annals of clinical biochemistry, vol. 47, May. 2010.

19

©2

01

2 Pa

ul E

pner LLC

DATA MINING Data mining is the process of nontrivial extraction

of implicit, previously unknown and potentially useful information from data stored in repositories.1

Strategies can be driven by published guidelines Retrospective study2 of more than 450,000 HPV

tests against new guideline published in 2004 HPV testing is contraindicated in women under age 21 HPV testing is contraindicated without positive cytology.

Study showed multi-year improvements in compliance

Data mining is a tool that identifies opportunities for education or other interventions

1Lee, S.J. and Siau,K., “A review of data mining techniques,” Industrial Management & Data Systems, Vol. 101, January 2001.2B.H. Shirts and B.R. Jackson, “Informatics methods for laboratory evaluation of HPV ordering patterns with an example from a nationwide sample in the United States, 2003-2009.,” Journal of pathology informatics, vol. 1, Jan. 2010.

20

©2

01

2 Pa

ul E

pner LLC

PHYSICIAN-LEVEL PERFORMANCE FEEDBACK When physicians are given feedback on their test

ordering patterns compared to colleagues or guidelines, test ordering behavior changes.

In one study1, clinicians were educated about the laboratory tests needed to monitor patients on antihypertensive medication. Additionally, they were given feedback on their testing patterns. Appropriate testing improved.

In another study2, quarterly feedback of practice requesting rates for nine laboratory tests, enhanced with educational messages were provided to primary care physicians which proved to be an effective strategy for reducing inappropriate testing

1Lafata, J.E. et al, “Academic detailing to improve laboratory testing among outpatient medication users.,” Medical care, vol. 45, Oct. 2007.2Thomas, R.E. et al, “Effect of enhanced feedback and brief educational reminder messages on laboratory test requesting in primary care: a cluster randomised trial.,” Lancet, vol. 367, Jun. 2006.

©2

01

2 Pa

ul E

pner LLC

21

RESOURCE UTILIZATION COMMITTEE Typically involves locally driven consensus One study is noteworthy for assessment of

patient impact.*

*Neilson, E. G., Johnson, K. B., Rosenbloom, S. T., Dupont, W. D., Talbert, D., Giuse, D. A., Kaiser, A., et al. (2004). The impact of peer management on test-ordering behavior. Annals of internal medicine, 141(3), 196–204.

22

©2

01

2 Pa

ul E

pner LLC

FOCUS ON SYSTEMATIC ERROR REDUCTION

Many laboratory professionals routinely drive initiatives to reduce systematic errors. Tools in use

Lean 6 Sigma Root Cause Analysis Failure Mode & Effect Analysis

Bias in problem selection may exist Within the laboratory walls Within the control or shared control of the laboratory

Evidence for the use of these tools to eliminate diagnostic errors is difficult to find

23

©2

01

2 Pa

ul E

pner LLC

INTERPRETIVE COMMENTS

Criteria for providing interpretive comments have been described*

a decision on treatment is indicated by the results in combination with the clinical details provided

a result is unexpected a specific question has been posed but it is not

obvious whether the results provide the answer a clinician has requested a test with which he/she is

not likely to be familiar Areas where Interpretive reports are most relevant

*E. Piva and M. Plebani, “Interpretative reports and critical values.,” Clinica chimica acta; international journal of clinical chemistry, vol. 404, 2009.

24

©2

01

2 Pa

ul E

pner LLC

DIAGNOSTIC MANAGEMENT TEAMS AT VANDERBILT ENSURE APPROPRIATE CONSULTATIVE SERVICES

25

©2

01

2 Pa

ul E

pner LLC

PENDING LAB RESULTS: PROCESS MONITORING Shifts the focus from catching failures e.g.,

clinical event monitors to workflow process control

Some efforts are ongoing: MSTART (Multi-Step Task Alerting, Reminding, and Tracking)

*Tarkan, S., Plaisant, C., Shneiderman, B., & Hettinger, A. (2010). Improving Timely Clinical Lab Test Result Management: A Generative XML Process Model to Support Medical Care.

26

©2

01

2 Pa

ul E

pner LLC

PENDING LAB RESULTS: DISCHARGE MONITOR

Several attempts to create automated tools have been tried with limited success

Positive results were obtained with a system of email notifications1

A computer-based antimicrobial monitoring (CBAM) system has been used to ensure positive microbiology cultures receive attention with improved outcomes2

Discharge systems need to alert both hospital-based and primary care physician

1Dalal, A. K., Schnipper, J. L., Poon, E. G., Williams, D. H., Rossi-Roh, K., Macleay, A., Liang, C. L., et al. (2012). Design and implementation of an automated email notification system for results of tests pending at discharge. Journal of the American Medical Informatics Association : JAMIA, 19(4), 523–8.2Wilson, J. W., Marshall, W. F., & Estes, L. L. (2011). Detecting delayed microbiology results after hospital discharge: improving patient safety through an automated medical informatics tool. Mayo Clinic proceedings. Mayo Clinic, 86(12), 1181–5. doi:10.4065/mcp.2011.0415

27

©2

01

2 Pa

ul E

pner LLC

TOOLS EXIST; PROVING VALUE IS MORE DIFFICULT

Robust research on the role of laboratory services does not exist

Research on the effectiveness of available tools is limited

28

©2

01

2 Pa

ul E

pner LLC

IMPROVEMENTS IN TEST SELECTION AND RESULTS INTERPRETATION (ITSRI) – A RESEARCH AGENDA

Strategic Intent Establish empirically the optimum role for

laboratory medicine’s physicians and scientists to maximize positive patient outcomes Appropriate testing Appropriate interpretation

Identify evidence-based interventions that support the optimum role

29

©2

01

2 Pa

ul E

pner LLC

ITSRI STATUS

Narrowed scope to diagnostic errors Seeking to catalyze research

Diagnostic Process Variation Chief complaint specific Diagnosis specific Test domain specific

Intervention effectiveness Building awareness Recruiting collaborators

NorthShore University HealthSystemVirginia Commonwealth UniversityKaiser Permanente

30

©2

01

2 Pa

ul E

pner LLC

OTHER EFFORTS ONGOING

Diagnostic errors and the clinical laboratoryAHRQ ACTION IICLIHC™

Significant challenges remainLack of funding and resourcesShifting the focus from laboratory costs

31

©2

01

2 Pa

ul E

pner LLC

AHRQ FUNDED RESEARCH

Awarded to RTI in August, 2011; 18 month effort

Developing risk assessment tools which will be tested in three sites: Vanderbilt Emory Seattle Children’s

32

©2

01

2 Pa

ul E

pner LLC

REFERRAL LABORATORY RISK ASSESSMENT

33

©2

01

2 Pa

ul E

pner LLC

IDENTIFICATION AND PRIORITIZATION OF RISK

34

©2

01

2 Pa

ul E

pner LLC

CLINICAL LABORATORY INTEGRATION INTO HEALTHCARE COLLABORATIVE – CLIHC™ CDC sponsored Seeking to break down the barriers between care

providers and laboratory professionals Key initiatives are moving forward

A survey of medical schools to understand curricular changes since 1992 involving laboratory medicine

A survey of pathology residency programs quantifying time spent teaching consultation

A survey of primary care clinicians to quantify the barriers to appropriate laboratory utilization

An initiative to define nomenclature issues and investigate technology strategies for addressing them

An initiative that will develop and publish algorithms to guide clinicians in the use of complex tests (with iPhone app)

An initiative that seeks to experimentally determine the effectiveness of laboratory interventions on diagnostic error reduction (ITSRI)

35

©2

01

2 Pa

ul E

pner LLC

KEY MESSAGES

Diagnostic error is a major patient safety problem

The total testing process is a significant source of diagnostic errors

Laboratory-directed interventions are available and can be effective in reducing errors

Laboratory physicians and scientists will realize other benefits from leading collaborative efforts Improve patient outcomes Strengthen relationships with clinicians Reduce the level of risk in the health system Become indispensable stewards of clinical data

36

©2

01

2 Pa

ul E

pner LLC

FINAL THOUGHT: SHIFTING THE GOAL

The clinical lab’s mission should not be:

To provide accurate, timely, low cost test results

Although necessary, it is not sufficient

The clinical lab’s mission should be:

To rapidly and efficiently enable the accurate diagnosis of conditions, the selection of appropriate treatments and the effective monitoring of health status*

* Epner, Paul, “Impact of Laboratory Services on Diagnostic Errors,” ThinkLab ‘11