it I

Interpretation of HospitalMortality Rates: The CurrentState of the Art

In 1986, the Health Care Financing Adminis­tration startled the medical community in theUnited States with the publication of mortalityrates for individual hospitals. 1 These rates,the publication of which was prompted by a"freedom of information" request from the NewYork Times and consumer advocate and recentMcArthur Award winner Dr. Sidney Wolfe, wereintended to prompt greater disclosure and ac­countability from American hospitals. Manyobservers hoped that the widespread availabil­ity of these rates, which were adjusted only fordifferences in dismissal diagnoses and patientage, would be useful in identifying hospitalswith poor-quality care.

Five years later, the release of hospital-spe­cific mortality rates has become an annual "non­event." It no longer attracts front-page news­paper coverage and is largely ignored by boththe press and the hospital community.f TheHealth Care Financing Administration hasstated that they are improving their patientadjustment procedures. A few state legislatureshave passed laws requiring hospitals to collectdata under the guidance of companies that claimto make mortality results more meaningful.Despite this rhetoric, not a single study hasconvincingly linked variations in hospital mor­tality rates to underlying differences in qualityof care," and widespread disillusionment pre­vails with what is largely viewed and increas­ingly presented as a political rather than ascientific process.

This study was supported in part by the Agency for HealthCare Policy and Research (HS 05787) and The John A.Hartford Foundation (87267).

Address reprint requests to Dr. W. A. Knaus, ICU ResearchUnit, George Washington University Medical Center, 2300K Street NW, Washington, DC 20037.

From this perspective, the article by Marshand colleagues in this issue of the Proceedings(pages 1549 to 1557) is a step in a different but,we believe, more fruitful direction. Unlike theHealth Care Financing Administration's com­parison and other recent attempts at modifica­tion of the large data-base approach," the Mayoinvestigators used a clinically valid adjustmentfor severity of patient risk, prospectively col­lected on a small patient cohort. They concludedthat much of the variation in hospital deathrates could be accounted for by appropriatepretreatment risk adjustment. They also found,however, that some ofthe variation in predictedversus actual death rates was still related tounmeasured differences in patient characteris­tics rather than variations in the quality ofcare.

Before we consider the implications of theseresults, what exactly did the Mayo investigatorsdo? They analyzed consecutive admissions tofour general medical-surgical intensive-careunits at two Mayo-affiliated hospitals and, dur­ing the initial 24 hours, collected data thatenabled them to evaluate each patient's risk ofdying before hospital dismissal. This individualrisk assessment was possible because the riskfactors used-diagnosis and APACHE II (acutephysiology and chronic health evaluation)score-were linked to a data base of approxi­mately 5,000 other admissions to intensive-careunits where the presence of these risk factorshad been previously related to the patient'soutcome at hospital dismissal. 4 This enabled theinvestigators to calculate a predicted mortalityrate within each major diagnostic group and tocompare it with the actual rate as a standard­ized mortality ratio.

They found that this method worked well forall surgical diagnoses and for most medicaladmissions. It underestimated the mortalityrate for a group of medical patients admitted tothe hematology-oncology and hepatology ser­vices. The Mayo investigators concluded thatthis result occurred because the patients fromthese services admitted to the intensive-care

Mayo Clin Proc 65:1627-1629, 1990 1627

1628 EDITORIAL

unit had severe chronic health problems thatincreased their risk of death more than theirAPACHE II scores would have indicated.

The prediction of patient outcome with use ofclinical data and a reference data base is a newundertaking for clinical medicine, and the mostappropriate methods for both collecting patientdata and developing the data base are still evolv­ing. Ideally, one would want to develop a modelthat contained all the patient characteristicsand other elements that influenced the patient'srisk of hospital death for a sufficiently largegroup of patients so that all the various combi­nations of such factors as diagnosis, severity ofdisease, and chronic health problems were wellrepresented.

The first task is, by itself, a substantial under­taking. Consider all the various factors thatcould influence a patient's risk of death. Amongthese factors are the well-recognized influence ofthe patient's disease, the severity ofthat diseaseas determined by physiologic abnormalities, andthe chronic state of the patient's health. Theseare the major elements that constitute theAPACHE system, but, as indicated in the reportby Marsh and associates, important nuancesmay be present in each of these elements. Thepatient's diagnosis, for example, can be recordedas generally as sepsis or as specifically as bacter­emia related to urinary tract infection. The in­creased precision of the latter designation willresult in an increased explanatory power foroutcome, but all such designations must becarefully described so that they can be reliablyreproduced.

Likewise, knowing the exact degree and ex­tent of the patient's chronic health status-forexample, how extensive a case of cirrhosis is andwhether it is accompanied by portal hyperten­sion-might also help distinguish those withslightly different risks. How can these designa­tions, however, be made accurate and reliable,and how many of these subtle nuances are nec­essary for accurate prediction of risk?

In our recent comprehensive effort to reviseand improve the APACHE classification sys­tem, we have been carefully evaluating each ofthese elements within a nationally representa-

Mayo Clin Proc, December 1990, Vol 65

tive data base of 17,500 admissions to intensive­care units." We have concluded that an increasein the number of possible diagnostic categoriesfrom the 50 used in APACHE II to 75 is rea­sonable and will improve the precision of thepredictions. We have also confirmed the findingby the Mayo investigators that chronic liverdisease is indeed an important risk factor fordeath among acutely ill patients in the inten­sive-care unit, and two designations of chronicliver disease-simple cirrhosis and cirrhosisassociated with past episodes of hepatic fail­ure-will be part of the APACHE III prognosticsystem.

In these investigations, we have also discov­ered that, despite a precise recording of disease,chronic health, and physiologic data, the pa­tient's location immediately before admission tothe intensive-care unit (emergency room, hospi­tal floor, another intensive-care unit, or anotherhospital) remains an important independentpredictor of the patient's risk of hospital mor­tality. When all these elements are taken intoaccount, however, we can achieve accurate andreliable predictions of hospital death for individ­ual patients in the intensive-care unit-both atthe time they initially are admitted for intensivecare and also over time. As recently pointed outby Silverstein," one future challenge presentedby this accurate prognostic ability is whetherprediction instruments can be used to provideless costly and more effective care for patientswho are likely to benefit from intensive-careunits.

If we are proposing that prognostic data aresufficiently accurate to begin to supplementclini­cal judgment at the bedside, should not thesedata be sufficiently accurate to use in the com­parison of hospitals? The answer is no-not yet.The reason for the caution is that, as accurate asthese predictions may be for individual patients,aggregating them for institutional evaluationpresents another level of complexity.

In the analysis of patients treated withinintensive-care units at two Mayo-affiliated hos­pitals, Marsh and his colleagues determinedthat subtle differences related to the prior chronichealth status of patients admitted to hematol-

Mayo Clin Proc, December 1990, Vol 65

ogy-oncology and hepatology services were aconfounding factor. Much of that variation, webelieve, is related to the selection or referralof patients for care at one institution versusanother.

Indeed, our research on APACHE III empha­sizes that if such subtle differences in selectionof patients occur between two hospitals thatshare common medical staffs and treatmentapproaches such as Saint Marys Hospital andRochester Methodist Hospital, the variationexisting among very dissimilar institutions (forexample, a small rural hospital in Minnesotaand a large teaching institution in New YorkCity) may be substantial. We believe that wesimply do not yet have the understanding tostate with confidence that we have measuredand accurately recorded all sources of potentialvariation in selection of patients to contrastdirectly the Minnesota hospital's death ratewith the one in New York and conclude that dif­ferences in standardized mortality ratios arecaused by differences in quality of care. Weagree that it is premature to relate the processto the outcome until we have adequately con­trolled for patient input-especially for a tertia­ry-care institution such as the Mayo Clinic,which attracts patients worldwide for special­ized care.

Nonetheless, progress is being made. Stan­dardized mortality ratios over time within thesame institution would not be threatened byinterinstitutional selection biases and could beuseful for tracking and evaluating performance.Comparisons among hospitals within homoge­neous diagnostic groups are also less susceptibleto unmeasured patient variations or referralbias. The Mayo study showed, for example, thatmortality ratios for surgical patients and formost of the medical diagnoses were predictable

EDITORIAL 1629

at both Rochester Methodist Hospital and SaintMarys Hospital.

We are convinced that, with continued mul­tidisciplinary research and institutional com­mitment, as demonstrated by this Mayo analy­sis, even further progress will occur. Mostimportantly, the American medical, nursing,and hospital community will be responding tothe political challenge raised by the federalgovernment's publication of hospital mortalitydata not with rhetoric but with accurate andclinically meaningful scientific investigations.These studies could provide an answer to thequestion of whether significant variations inmortality rates exist among US hospitals. Theyshould also point the way, then, toward address­ing the ultimate question-why?

William A. Knaus, M.D.Douglas P. Wagner, Ph.D.ICU Research UnitGeorge Washington University

Medical CenterWashington, DC

REFERENCES1. Brinckly J: New York Times, Mar 12, 1986, p 12. BerwickDM, WaldDL: Hospitalleaders'opinionsofthe

HCFA mortality data. JAMA 263:247-249,19903. Park RE, Brook RH, KosecoffJ, Keesey J, Rubenstein L,

Keeler E, Kahn KL, Rogers WH, Chassin MR: Explain­ing variations in hospital death rates: randomness,severity of illness, quality of care. JAMA 264:484-490,1990

4. Knaus WA, Draper EA, Wagner DP, Zimmerman JE:APACHE II: a severity of disease classification system.Crit Care Med 13:818-829, 1985

5. Zimmerman JE (ed): APACHE III study design: ana­lytic plan for evaluation of severity and outcome. CritCare Med 17 (Suppl):S169-S221, 1989

6. Silverstein MD: Prediction instruments and clinicaljudgement in critical care (editorial). JAMA 260:1758­1759, 1988