appropriate use criteria: past, present, future
TRANSCRIPT
Editorial Comment
Appropriate Use Criteria: Past, Present, Future
Pamela S. Douglas, MD, FASE, Durham, North Carolina
From Duke U
North Carolina
Reprint reque
Medical Cent
(E-mail: pame
0894-7317/$3
Copyright 201
http://dx.doi.o
1176
Reports that cardiologists overuse procedures are common in the laypress, with concerns ranging from legitimate differences in judgmentto outright fraud. Clinical practice guidelines were created to assistphysicians in practicing evidence-based medicine, but in 2004, onthe basis of mounting evidence of unexplained geographic variationin use and increasing cost and regulatory pressures, the AmericanCollege of Cardiology decided to create a new category of standards,appropriate use criteria (AUC). AUC are constructed aroundcommon patient scenarios and intended to inform bedside decisionmaking and provide guidance regarding overall patterns of care.AUC blend all levels of evidence gleaned from relevant randomizedclinical trials, clinical practice guidelines, and expert consensus. AUCare based on levels of evidence A (randomized controlled trials) and B(observational studies) when available (and including level ofevidence C, expert opinion). The evidence is considered followinga rigorous, prospective process adapted from an existing Delphi meth-odology used by the RAND Corporation for similar purposes, whichgenerates ratings regarding the reasonableness of performing a test orprocedure in a specific clinical situation.1 Importantly, the technicalpanel rating the indications in terms of appropriateness is composedof <50% of specialists in that area.
The reactions to these early documents ranged from embracingthem as a useful and needed tool to guide practice to outright rejec-tion (‘‘How dare anyone imply that I practice inappropriate medi-cine!’’). The first applications of AUC to actual patients providedinteresting and important insights into current patterns of care, high-lighted areas of possible overuse, and indicated that quality improve-ment was possible by means of targeted intervention, especiallybecause most inappropriate use was clustered into just a few indica-tions. However, there were also questions: the complexities of clinicalmedicine meant that the AUC class often could not be determined,and when it could, reproducibility among observers was modest.There were subtle differences in indications and rankings betweensimilar modalities such as stress echocardiography and stress nuclearmedicine that could be confusing. Those wishing to adopt AUC wereuncertain as to the ‘‘correct’’ level of inappropriateness and what theoptimal relationship should be between AUC ranking and the likeli-hood of a new finding or a change in care. Clearly, this was an effortin its infancy.
With time, experience, and a broader awareness of the need forsuch documents, AUC have matured. Revisions of virtually all the im-aging AUC have increased the documents’ coverage of the full rangeof clinical scenarios, harmonized indications and methods, and in thecase of echocardiography created a single combined document foreasier reference.2 By including an updated evidence base andaddressing omissions in prior documents,3,4 the revised
niversity Medical Center, Duke Clinical Research Institute, Durham,
.
sts: Pamela S. Douglas, MD, MACC, FASE, FAHA, Duke University
er, 7022 North Pavilion DUMC, PO Box 17969, Durham, NC 27715
6.00
2 by the American Society of Echocardiography.
rg/10.1016/j.echo.2012.09.016
echocardiographic AUC nearly doubled the number of indicationsfrom 110 to 202, with 97 indications rated as appropriate (up from66, a 47% increase), 34 as uncertain (from 11, a 209% increase),and 71 as inappropriate (from 33, a 115% increase). In addition, therevised document developed a guiding set of principles that broadlydefine an approach to testing on the basis of five general types ofindications: those encompassing initial diagnosis, guidance oftherapy, or evaluation of a change in clinical status were generallyranked as appropriate, while early follow-up was more likely to re-ceive uncertain or inappropriate ratings than late follow-up, especiallyin asymptomatic patients (i.e., surveillance).
A second wave of studies examining the applicability of the AUCto clinical echocardiography is now being published; this issue of JASEprovides three such examples,5-7 which together demonstratesignificant progress as well as a continued need for refinement.Both studies by Bhatia et al.5,6 and that by Mansour et al.7 comparethe application of the older AUC with the new revision and demon-strate substantial improvement in the ability to classify the real-worlduses of all types of echocardiography (transthoracic, transesophageal,and stress) into AUC rankings. Indeed, in these reports, 97% to 100%of all indications can now be mapped to the revised AUC, dependingon modality, with robust reproducibility. The remaining ‘‘missing’’transthoracic echocardiographic indications identified by the presentstudies are largely limited to those encountered in high-end referralcenters (transplantation and adult congenital heart disease) and infre-quently seen by the vast majority of echocardiographers. The gapsalso reflect ongoing innovations in care, such as transcatheter inter-ventions for structural heart diseases, for which there are few dataor little consensus on optimal imaging use. For transesophageal echo-cardiography, areas such as perioperative echocardiography are out-side the scope of the present documents and may be specializedenough to merit stand-alone AUC. Thus, aided by the identificationof gaps when applying the older AUC to clinical practice by theseand other authors, a major concern has been addressed, which hasmade the AUC much more robust and enhanced their utility and rel-evance. Similarly, future revisions will benefit from the gaps identifiedin the present documents.
Interestingly, all three reports also note decreases in the percentageof studies that are ranked as appropriate across modalities. Althoughinitially counterintuitive (has our decision making really gottenworse?), all three reports are consistent in noting that nearly two thirdsof echocardiographic studies that are newly classifiable by the revisedAUC are ranked as uncertain or inappropriate. This is not surprising,as the update added nearly twice as many indications in these cate-gories than were added to the appropriate category. Because nearlyall echocardiographic studies (96%) classifiable in both sets of AUCmapped to the same category of appropriateness in both, both setsof authors suggest that the earlier documents may have underesti-mated the prevalence of the uncertain and inappropriate categoriesin clinical practice and overestimated the prevalence of the appropri-ate category.
One of the most important contributions of AUC is the identifica-tion of the common inappropriate indications, because this in turnprovides the basis for education and quality improvement efforts.In applying the 2011 criteria, Bhatia et al.6 (for transthoracic
Journal of the American Society of EchocardiographyVolume 25 Number 11
Douglas 1177
echocardiography) found that the most common inappropriate usesfor outpatient transthoracic echocardiography centered on the rou-tine reevaluation of chronic conditions in the absence of a changein status, including mild valvular stenosis, hypertension, left ventricu-lar function in those with prior results showing normal function, as-cending aortic dilation, and completely repaired congenital heartdisease in adults. For inpatients, suspected endocarditis, pulmonaryembolism, and syncope in patients clinically at low riskwere also com-mon inappropriate indications. For transesophageal echocardiogra-phy, they found the most common inappropriate indications to bediagnosis of endocarditis in patients with low pretest likelihood, useof transesophageal echocardiography instead of transthoracic echo-cardiography that was likely to provide adequate data, or when nochange in treatment was anticipated regardless of the findings. Unfor-tunately Mansour et al.7 do not provide information on the most com-monly used inappropriate indications in their report, as theirrecognition can provide powerful guidance to ordering providers re-garding situations in which echocardiographymay not be useful in themajority of patients and in designing focused interventions to improvetest ordering and sharpen our appreciation of the proper uses of im-aging. However, this must be done with caution. As the AUC docu-ments themselves note, these are intended as only as guides ratherthan rules, and AUC are not intended to be used to deny needed im-aging care in individual patients. Conversely, a rating as appropriateshould not be misunderstood as a recommendation to image everypatient who might fit that scenario but rather that it would be reason-able to do so if the information derived will assist in care.
Although there is general agreement that providers should beknowledgeable about their ordering patterns and should minimizethe use of imaging for inappropriate indications, there is no evidenceas yet as to whether that would improve processes of care, costs, and/or outcomes. This is a complex question, especially for a diagnostictest, because test performance alone cannot directly affect outcomes;moreover, the science of outcomes research in imaging is still a matterof debate.8 However, this question can be explored by examining thelikelihood of new ormajor findings for eachmodality according to theAUC ranking, as Mansour et al.7 and Bhatia et al.5,6 have done. By farthe highest likelihood of a new or major finding was inechocardiographic studies performed for appropriate indications,with a lower likelihood in studies performed for uncertainindications and the lowest likelihood for studies done forinappropriate indications. Although this gradient is reassuring, thecorrect and much harder evidentiary standard to achieve isdocumentation that an echocardiographic exam improvesoutcomes and does not merely confirm the presence of knownmajor findings. Even the discovery of new findings is at best anintermediate measure; to be of value, echocardiographic findingsshould directly influence care in some way. Also, we must becareful not to equate new findings with imaging value, for thisignores the importance of negative results, which may excludesuspected disease or reassure patients and their physicians thatdisease is not progressing and that current care is adequate. Indeed,Mansour et al. note that more than twice as many transesophagealechocardiographic studies influenced medical care than had newfindings. Although quite complex, defining the value of imaging isa critical question, especially in this era of constrained resources.AUC may provide essential standards to guide future research inthis area, and subsequent revisions would certainly benefit fromsuch studies.
Both Bhatia et al.5,6 and Mansour et al.7 document new or majorfindings in a handful of inappropriate studies. These ‘‘positive’’ studies
appear to question the validity of an inappropriate rating; however,perhaps the answer lies in the knowledge that any imaging studymay yield an incidental finding but that their incidence and signifi-cance are generally low. As with all incidental findings, knowledgeof their presence may not improve care but instead lead to furtherevaluation, which may be unnecessary. Indeed, the constancy ofthe relative incidences of major findings in each of the AUC categoriesin the original and revised AUC lends validity to this hypothesis aswell as to the categories themselves.
Another concern regarding studies based on ranking echocardio-graphic studies that have already been or will be performed is thatit cannot address the appropriateness of those studies that were notperformed, whether through underuse of needed imaging, consulta-tion resulting in a revised diagnostic plan as may be commonly thecase with transesophageal echocardiography, or administrative denialor some other mechanism. This in turn highlights an important aspectof care, underuse, that is often not well addressed by AUC. This isa very difficult question to address, because the indirect link betweena diagnostic test and outcomes is also relevant in this situation, whenimportant findings are missed and not available to guide care whena needed test is not performed. Thus, it is difficult to trace the connec-tion between a missed test and a poor outcome. Nevertheless, under-use clearly exists, as shown by two recent studies. Ballo et al.9 appliedthe revised 2011 echocardiographic AUC to hospitalized patientswho were discharged without having undergone transthoracic echo-cardiography and concluded that in 16% of them, it would have beenappropriate to have ordered transthoracic echocardiographic evalua-tions. In another study, Curtis et al.10 demonstrated that more thanone third of patients with newly diagnosed heart failure failed to re-ceive imaging tests that could quantify ejection fraction, which isone of the handful of American College of Cardiology and AmericanHeart Association performance measures identifying ‘‘must-do’’ care.Furthermore, the proper use of testing was associated with decreasedmortality. However desirable it might be to do so to ensure optimalimaging care, AUC, as currently designed, are unable to address the‘‘appropriateness’’ of not performing echocardiography.
Additional important concerns are raised by past and current appli-cations of AUC: what is the ‘‘correct’’ level of appropriate (or inappro-priate) use, and how much variation is acceptable? This is a criticalissue, because reimbursement is increasingly tied to demonstratingquality of care, and in the absence of more robust outcomes datafrom imaging, process-of-care measures such as AUC are likely to be-come the default yardstick. Bhatia et al.’s6 comparison (for transtho-racic echocardiography) of academic and community practicesdemonstrates that patient mix may affect a provider’s AUC ‘‘perfor-mance’’: a higher rate of inappropriate use was found in an academicmedical center compared with a community practice. This was felt toreflect differences in patient populations, with the academic practicehaving a higher proportion of patients with established cardiovasculardiagnoses who were more likely to have been tested previously, butone could argue that some of these repeat studies may have been un-necessary. Furthermore, side-by-side comparison of the reports in thisissue of JASE reveals differing levels of appropriate use even at twoacademic medical centers, a difference found in applying both theoriginal and revised AUC. The data suggest that physicians in Bostonmay be more inappropriate than those in Chicago (22% of transtho-racic echocardiographic studies performed in Boston were inappro-priate vs 11% performed in Chicago; for transesophagealechocardiography, these percentages were 8% and 3%). Certainlythis is of a piece with the notoriously rowdy Red Sox Nation, but per-haps enthusiastic Cubs fans might take exception.
1178 Douglas Journal of the American Society of EchocardiographyNovember 2012
Together, Bhatia et al.’s5,6 andMansour et al.’s7 reports demonstratesubstantial improvement in the echocardiographic AUC, reflectingmaturation consistent with the growing acceptance and adoption ofAUC. Nevertheless, the future still holds many questions and chal-lenges. There is no doubt that themethods, nomenclature, indications,and application of AUC will continue to be refined. Continued re-search on the optimal use of imaging will also inform this field, al-though given the difficulties in randomized trials in this area, theevidence base will likely grow largely because of observational studiesand registries. These efforts need to be prioritized and funded. Newresearch findings will enable upgrading the level of evidence asmore robust data are generated and will serve to resolve the rankingsof some indications that are currently classified as uncertain. Most im-portant, research is critically needed to validate indication rankings anddemonstrate that use of AUC really does improve outcomes.
We also urgently need guidance as to how to best use AUC in clin-ical care. Implementation research can evaluate potential AUC qual-ity metrics and how they may be used to evaluate and guide care.Designed as tools to track overall patterns of care and to identify areasfor education and improvement, AUC have perhaps been most fre-quently applied in case-by-case decision-making, whether by physi-cians or payers. Certainly the need for guidance at the bedside isreal, and the extent to which AUC can contribute to this is of greatvalue. Their utility for this purpose would be enhanced by the useof decision support tools and other positive interventions, ratherthan by care denials. Development and implementation testing ofsuch tools is under way,11 and it is critical that this continue.
The current push to value-based purchasing and away from thefee-for-service model opens up another area in which the applicationof AUC may be of substantial value, but evidence is lacking. As carepathways and episodes of care are designed to enable bundled pay-ments, echocardiography will be seen as a cost center and no longera potential source of revenue. AUC rankings can provide needed doc-umentation of the value of echocardiography and could help preserveaccess for patients who would benefit from testing.
In conclusion, the three studies in this issue of JASE together dem-onstrate that the revision of the echocardiographic AUC show muchprogress over the original documents. AUC have become an impor-tant set of clinical and policy standards at a time when these are sorelyneeded. However, they have not achieved perfection. Many, if not all,of these shortcomings are being addressed by our clinical, research,and policy communities, which, together with patients, are all stake-holders who will benefit from the improvements in AUC.
REFERENCES
1. Patel MR, Spertus JA, Brindis RG, Hendel RC, Douglas PS, Peterson ED,et al. ACCF proposed method for evaluating the appropriateness of car-diovascular imaging. J Am Coll Cardiol 2005;46:1606-13.
2. Douglas PS, Garcia MJ, Haines DE, Lai WW, Manning WJ, Patel AR, et al.ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011appropriate use criteria for echocardiography. A report of the American
College of Cardiology Foundation Appropriate Use Criteria Task Force,American Society of Echocardiography, American Heart Association,American Society of Nuclear Cardiology, Heart Failure Society ofAmerica, Heart Rhythm Society, Society for Cardiovascular Angiographyand Interventions, Society of Critical CareMedicine, Society of Cardiovas-cular Computed Tomography, Society for Cardiovascular Magnetic Reso-nance American College of Chest Physicians. J Am Soc Echocardiogr2011;24:229-67.
3. Douglas PS, Khandheria B, Stainback RF, Weissman NJ, Brindis RG,Patel MR, et al. ACCF/ASE/ACEP/ASNC/SCAI/SCCT/SCMR 2007appropriateness criteria for transthoracic and transesophageal echocardi-ography: a report of the American College of Cardiology FoundationQuality Strategic Directions Committee Appropriateness Criteria Work-ing Group, American Society of Echocardiography, American College ofEmergency Physicians, American Society of Nuclear Cardiology, Societyfor Cardiovascular Angiography and Interventions, Society of Cardiovas-cular Computed Tomography, and the Society for CardiovascularMagnetic Resonance. Endorsed by the American College of Chest Physi-cians and the Society of Critical Care Medicine. J Am Soc Echocardiogr2007;20:787-805.
4. Douglas PS, Khandheria B, Stainback RF, Weissman NJ, Peterson ED,Hendel RC, et al. ACCF/ASE/ACEP/AHA/ASNC/SCAI/SCCT/SCMR2008 appropriateness criteria for stress echocardiography: a report ofthe American College of Cardiology Foundation Appropriateness CriteriaTask Force, American Society of Echocardiography, American College ofEmergency Physicians, American Heart Association, American Society ofNuclear Cardiology, Society for Cardiovascular Angiography and Inter-ventions, Society of Cardiovascular Computed Tomography, and Societyfor Cardiovascular Magnetic Resonance endorsed by the Heart RhythmSociety and the Society of Critical Care Medicine. J Am Coll Cardiol2008;51:1127-47.
5. Bhatia RS, Carne DM, Picard MH, Weiner RB. Comparison of the 2007and 2011 appropriate use criteria for transesophageal echocardiography.J Am Soc Echocardiogr 2012;25:1170-5.
6. Bhatia RS, Carne DM, Picard MH, Weiner RB. Comparison of the 2007and 2011 appropriate use criteria for transthoracic echocardiography invarious clinical settings. J Am Soc Echocardiogr 2012;25:1162-9.
7. Mansour IN, Razi RR, Bhave NM, Ward RP. Comparison of the updated2011 appropriate use criteria for echocardiography to the original criteriafor transthoracic, transesophageal, and stress echocardiography. J Am SocEchocardiogr 2012;25:1153-61.
8. Douglas PS, Taylor A, Bild D, Bonow R, Greenland P, Lauer M, et al. Out-comes research in cardiovascular imaging: report of a workshop spon-sored by the National Heart, Lung, and Blood Institute. J Am SocEchocardiogr 2009;22:766-73.
9. Ballo P, Bandini F, Capecchi I, Chiodi L, Ferro G, Fortini A, et al. Applica-tion of 2011 American College of Cardiology Foundation/American Soci-ety of Echocardiography appropriateness use criteria in hospitalizedpatients referred for transthoracic echocardiography in a community set-ting. J Am Soc Echocardiogr 2012;25:589-98.
10. Curtis LH, Greiner MA, Shea AM, Whellan DJ, Hammill BG,Schulman KA, et al. Assessment of left ventricular function in older Medi-care beneficiaries with newly diagnosed heart failure. Circ CardiovascQual Outcomes 2011;4:85-91.
11. AmericanCollege of Cardiology. Imaging in ‘‘FOCUS.’’ Available at: http://www.cardiosource.org/science-and-quality/quality-programs/imaging-in-focus.aspx. Accessed September 10, 2012.