1

Emergency Surgery Score

Variable Points

Age>60 years 2 Albumin <3.0 U/L 1 Alkaline phosphatase >125 U/L 1 Ascites 1 Body mass index <20 kg/m2 1 BUN > 40 mg/dL 1 History of COPD 1 Creatinine > 1.2 mg/dL 2 Disseminated cancer 3 Dyspnea 1 Functional dependence 1 Hypertension 1 INR >1.5 1 Platelets <150 X 103 /µL 1 SGOT > 40 U/L 1 Sodium > 145 mg/dL 1 Steroid use 1 Transfer from outside ED or acute care hospital 1 Ventilator requirement 48 hours pre-operatively 3 WBC < 4.5 X 103 /µL or 15-25 X 103 /µL 1 WBC>25 X 103 /µL 2 White race 1 >10% weight loss in last 6 months 1

2

3

Functional dependence assessment

1

EAST MULTICENTER STUDY

DATA COLLECTION FORM

ESS as a tool to predict postoperative mortality and complications after emergency general surgery

Enrolling Center: __________________________________

Enrolling Co-investigator: __________________________________

Demographics

Age: ____________ Gender: _______________

Admission source

Emergency department Transfer

Inpatient consultation Acute care hospital

Medicine Nursing home

Surgery Other

OB/GYN

Other

Diagnosis (check all that applies)

Hollow viscus organ perforation Complicated Diverticulitis

Foreign body/ Caustic ingestion Endoscopic complication

Pancreatitis Other surgical complications

non-resolving Small bowel obstruction Colonic obstruction

Bowel ischemia Volvulus

Gastrointestinal bleeding Incarcerated/ strangulated obstructed abdominal wall hernia

Incarcerated/ strangulated inguinal hernia (with exploratory laparotomy)

Fulminant clostridium difficile colitis Abdominal Compartment Syndrome

2

Internal hernia or closed loop bowel obstruction Others: _____________________________

Organ associated (check all that apply)

Esophagus Colon

Stomach Appendix

Small bowel Hepatobiliary and pancreas

ESS score variables

Demographic score

Age > 60 years Y/N ___

Race _____________ ___

Transfer from outside ED Y/N ___

Transfer from an acute care hospital inpatient facility Y/N ___

Comorbidities

Ascites Y/N ___

BMI < 20 kg/m2 Y/N ___

Dyspnea at baseline Y/N ___

Functional dependence Y/N ___

History of COPD Y/N ___

Hypertension Y/N ___

Steroid use Y/N ___

Ventilator requirement with 48 h preoperatively Y/N ___

Weight loss > 10% in the preceding 6 mo Y/N ___

History of disseminated cancer Y/N ___

Laboratory values

Albumin < 3.0 U/L Y/N ___

Alkaline phosphatase > 125 U/L Y/N ___

Blood urea nitrogen > 40mg/dl Y/N ___

Creatinine > 1.2 mg/dl Y/N ___

3

International normalized ratio > 1.5 Y/N ___

Platelets < 150 x 103 µL Y/N ___

SGOT > 40 U/L Y/N ___

WBC ___ x 103 µL ___

Total score ______

Operative variable at the time of surgery

Intraoperative resuscitation Crystalloid ___________ml

Colloid ___________ml

PRBC ___________ml

FFP ___________ml

Platelet ___________ml

Inotropic drug Y/N Number: ______

Operative intervention at the time of laparotomy (check all that applies)

Exploratory laparotomy

Small bowel resection Location: _________________

Large bowel resection

Gastrectomy

Simple suture/ repair

Pancreatic Necrosectomy

Abdominal lavage/ drainage

Postoperative variables

ICU admission (initial or anytime during hospitalization) Y/N Indication:

SOFA score POD 0 ______ POD 1 _______

Length of ICU stay ______days

Postoperative inotropic drug Y/N Number: ______

30-day mortality Y/N

4

Day from operation to mortality _____days

Length of hospital stay ______days

Complication within 30-days postoperatively (check all that apply)

Superficial surgical site infection Deep incisional surgical site infection

Organ/space surgical site infection Wound dehiscence

Pneumonia Unplanned intubation

Pulmonary embolism Ventilator requirement > 48 hours

Renal insufficiency Acute renal failure

Urinary tract infection Stroke/ cerebrovascular accident

Cardiac arrest requiring CPR Myocardial infarction

Bleeding requiring transfusion Graft/ prosthesis/ flap failure

Deep vein thrombosis/ thrombophlebitis Sepsis

Septic shock Anastomosis leak

Reoperation _ Planned _ Unplanned

At the time of discharge

Outcome

Complete recovery

Decreased functional status compared to preoperative baseline level (See Katz scale for guidance)

Death

End organ failure at discharge (new onset from this admission)

Respiratory system On tracheostomy Y/N

5

Home O2 Y/N

Ventilator dependence Y/N

Cardiovascular system Congestive heart failure Y/N

Atrial Fibrillation Y/N

Gastrointestinal system Feeding tube Y/N

Enterostomy Y/N

Renal system On Dialysis Y/N

Neurology Functional dependence Partial Total

Destination of discharge

Home Home with nursing services

Rehabilitation LTAC

Nursing home Hospice

AAST 2015 PLENARY PAPER

Derivation and validation of a novel EmergencySurgery Acuity Score (ESAS)

Naveen F. Sangji, MD, MPH, Jordan D. Bohnen, MD, MBA, Elie P. Ramly, MD, Daniel D. Yeh, MD,DavidR.King,MD,MarcDeMoya,MD,KathrynButler,MD, Peter J. Fagenholz,MD,GeorgeC.Velmahos,MD, PhD,

David C. Chang, MBA, MPH, PhD, and Haytham M.A. Kaafarani, MD, MPH, Boston, Massachussetts

Sub

Fro

Thi

Ad

DO

J TrVol

BACKGROUND: T

mitted: October 20, 2015, R22, 2016, Published online: Mam the Division of Trauma, EmeJ.D.B., E.P.R., D.D.Y., D.R.K.,of Surgery, and Codman CenterC., H.M.A.K.), Massachusettss study was presented at the 74tthe Surgery of Trauma, Septemdress for reprints: Haytham MEmergency Surgery, and Surgiand Harvard Medical School, 1email Haytham.kaafarani@mgh

I: 10.1097/TA.00000000000010

auma Acute Care Surgume 81, Number 2

here currently exists no preoperative risk stratification system for emergency surgery (ES). We sought to develop an EmergencySurgery Acuity Score (ESAS) that helps predict perioperative mortality in ES patients.

METHODS: U

sing the 2011 American College of Surgeons' National Surgical Quality Improvement Program (ACS-NSQIP) database (deriva-tion cohort), we identified all surgical procedures that were classified as “emergent.” A three-step methodology was then per-formed. First, multiple logistic regression models were created to identify independent predictors (e.g., patient demographics,comorbidities, and preoperative laboratory variables) of 30-day mortality in ES. Second, based on the relative impact of each iden-tified predictor (i.e., odds ratio), using weighted averages, a novel score was derived. Third, using the 2012 ACS-NSQIP database(validation cohort), the score was validated by calculating its C statistic and evaluating its ability to predict 30-day mortality.

RESULTS: F

rom 280,801 NSQIP cases, 18,439 ES cases were analyzed, of which 1,598 (8.7%) resulted in death at 30 days. The multiplelogistic regression analyses identified 22 independent predictors of mortality. Based on the relative impact of these predictors,ESAS was derived with a total score range of 0 to 29. ESAS had a C statistic of 0.86; the probability of death at 30 days graduallyincreased from 0% to 36% then 100% at scores of 0, 11, and 22, respectively. In the validation phase, 19,552 patients were in-cluded, the mortality rate was 7.2%, and the ESAS C statistic stayed at 0.86.

CONCLUSION: W

e have therefore developed and validated a novel score, ESAS, that accurately predicts mortality in ES patients. Such a scorecould prove useful for (1) preoperative patient counseling, (2) identification of patients needing close postoperative monitoring,and (3) risk adjustment in any efforts at benchmarking the quality of ES. (J Trauma Acute Care Surg. 2016;81: 213–220. Copyright© 2016 Wolters Kluwer Health, Inc. All rights reserved.)

LEVEL OF EVIDENCE: P

rognostic/epidemiologic study, level III. KEYWORDS: E mergency surgery; mortality score; quality benchmarking; perioperative mortality; outcomes.

I n the 1940s, emergency surgery (ES) was defined as “a surgi-cal procedure which, in the surgeon's opinion, should be per-

formed without delay.”1 In the past decade, under the umbrellaof acute care surgery, ES has been recognized as a surgical sub-specialty along with trauma surgery and critical care.2 In2013, the American Association for the Surgery of Trauma(AAST) attempted to define ES in a data-driven, evidence-based manner and identified a list of 98 International Classi-fication of Diseases—9th Rev. (ICD-9) codes constitutingemergency general surgery.3 The burden of emergency surgicaldisease is substantial and has been increasing during the past de-cade. With the use of the AAST definitions, the estimated bur-den of ES disease in the United States between 2001 and 2010

evised: February 9, 2016, Accepted: Februaryrch 30, 2016.rgency Surgery, and Surgical Critical Care (N.F.S.,M.D., K.B., P.J.F., G.C.V., H.M.A.K.), Departmentfor Clinical Effectiveness in Surgery (N.F.S., D.C.General Hospital, Boston, Massachussetts.h annual meeting of the American Association forber 9–12, 2015, in Las Vegas, Nevada..A. Kaafarani, MD, MPH, Division of Trauma,cal Critical Care, Massachusetts General Hospital65 Cambridge St, Suite 810, Boston, MA 02114;.harvard.edu.

59

Copyright © 2016 Wolters Kluwer H

was determined at 27,668,807 admissions and 7.1% of all hos-pitalizations, with 28.8% (7,979,578) of the admitted patientsrequiring an operation.4 ES carries a higher risk of morbidityand mortality compared with nonemergent or elective proce-dures.5,6 In fact, ES has been shown to be an independent riskfactor for postoperative complications and death even whencontrolling for preoperative variables and procedure type.6

Estimating the risk of postoperative morbidity and mortal-ity for ES as an entity by itself is crucial for appropriate counsel-ing of the patient needing surgical intervention and for obtainingawell-informed consent. In addition, adequate risk adjustment isindispensable for any efforts aimed at benchmarking and im-proving the quality of care of ES. There exist several general pre-operative risk assessment tools that allow determination of themortality and morbidity risk to patients undergoing surgery.The American Society of Anesthesiologists' (ASA) classificationdeveloped in 1941, the Physiological and Operative SeverityScore for enUmeration of Mortality and morbidity (POSSUM),and the Surgical Risk Scale (SRS), examples of such tools, allpredict outcomes for surgical patients.1,7,8 There are also othertools available to assess the preoperative risk in patients undergo-ing certain specific elective procedures such as colorectal sur-gery.9 Recently, efforts were made to classify the severity ofdisease in ES using anatomic scales.10–12 Recognizing the “chal-lenges for outcome assessment, research, and surgical training” inES, with its wide range of pathology, the Committee on Patient

213

ealth, Inc. All rights reserved.

TABLE 1. ES Patient Characteristics in the ACS-NSQIP 2011 (Score Development) and 2012 (Score Validation) Data Sets

Variable Derivation Cohort (2011) Validation Cohort (2012)

n 18,439 19,552

Mortality, % 8.7 7.2

Demographics Percentage in 2011 ES patients Percentage in 2012 ES patients

Age > 60 y 50.7 52.16

White race 69.8 68.91

Female 50.4 50.46

Transfer from outside emergency department 9.2 10.13

Transfer from an acute care hospital inpatient facility 8.8 6.67

Current smoker within 1 y 23.34 22.16

Comorbidities Percentage in 2011 ES patients Percentage in 2012 ES patients

Ascites 3.5 2.8

BMI < 20 kg/m2 8.9 8.13

Chemotherapy for malignancy 2.52 2.9

Coma > 24 h 0.77 0.69

Current pneumonia 2.86 2.79

CVA/stroke with neurologic deficit 4.64 4.81

CVA/stroke with no neurologic deficit 3.32 3.67

Diabetes mellitus with oral agents or insulin 17.36 17.39

Disseminated cancer 3.5 3.69

Dyspnea 13.1 7.65

Esophageal varices 0.36 0.36

EtOH > 2 drinks per day in 2 wk before admission 4.13 4.42

Functional dependence 13.0 7.89

Hemiplegia 2.37 1.87

History of angina in 1 mo before surgery 2.15 1.83

History of congestive heart failure in 30 d before surgery 3.32 2.88

History of COPD 8.5 8.14

History of myocardial infarction 6 mo before surgery 3 2.54

History of revascularization/amputation for peripheral vascular disease 6.14 6.04

History of transient ischemic attacks 3.25 3.05

Hypertension 51.5 50.43

Impaired sensorium 5.06 5.28

Paraplegia 0.87 0.95

Steroid use 6.6 6.15

Ventilator requirement within 48 h preoperatively 7.8 5.03

Weight loss > 10% in the preceding 6 mo 3.1 2.77

Laboratory values Percentage in 2011 ES patients Percentage in 2012 ES patients

Albumin < 3.0 U/L 47.0 43.89

Alkaline phosphatase > 125 U/L 12.5 11.52

Bilirubin > 1 mg/dL 19.56 18.58

Blood urea nitrogen > 40 mg/dL 9.6 8.81

Creatinine > 1.2 mg/dL 25.1 24.07

Hematocrit <38% 55.53 52.3

International normalized ratio > 1.5 10.3 9.37

Platelets < 150 103/μL 15.0 15.17

SGOT > 40 U/L 15.8 14.78

Sodium < 135 mg/dL 19.36 18.28

Sodium > 145 mg/dL 2.3 1.96

WBC, 103/μL<4.5 5.0 4.56

>11 and ≤15 24.37 24.91

>15–25 20.7 19.34

>25 3.8 3.31

CVA, cerebrovascular accident; SGOT, serum glutamic oxaloacetic transaminase.

Sangji et al.J Trauma Acute Care Surg

Volume 81, Number 2

214 © 2016 Wolters Kluwer Health, Inc. All rights reserved.

Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.

J Trauma Acute Care SurgVolume 81, Number 2 Sangji et al.

Assessment and Outcomes of AAST defined severity (of disease)grading systems for each eight common ES problems, such as in-fectious colitis, perirectal abscesses, and soft tissue infections, toenable better measurement of risk-adjusted outcomes.11,12

Besides a limited tool specifically estimating the mor-bidity and mortality of octogenarians undergoing emergencycolectomy,9,13 there currently exists no mortality risk predictiontool specifically for the high-risk, often in extremis, patients un-dergoing ES. We sought to derive and validate a novel Emer-gency Surgery Acuity Score (ESAS) that captures both thepatient comorbidities and the acuity of disease at presentationand thus helps predict, a priori, the risk of postoperative mortal-ity in patients undergoing ES.

PATIENTS AND METHODS

To derive ESAS and measure its baseline predicting per-formance, we used the American College of Surgeons' NationalSurgical Quality Improvement Program (ACS-NSQIP) 2011 da-tabase. To validate ESAS, we used the 2012 ACS-NSQIP data-base. The ACS-NSQIP is a prospective database that collectsmore than 150 preoperative and intraoperative clinical vari-ables as well as 30-day postoperative morbidity and mortality

TABLE 2. Multivariate Analysis With ORs for the 22 Predictors of Mo

Predictor OR

Demographics

Age > 60 y 2.65

White race 1.21

Transfer status

From outside emergency department 1.49

From an acute care hospital inpatient facility 1.26

Comorbidities

Ascites 1.63

BMI < 20 kg/m2 1.68

Disseminated cancer 3.09

Dyspnea 1.42

Functional dependence 1.76

History of COPD 1.53

Hypertension requiring medications 1.27

Steroid use 1.33

Ventilator requirement within 48 h preoperatively 3.33

Weight loss > 10% in the preceding 6 mo 1.58

Laboratory values

Albumin < 3.0 U/L 1.41

Alkaline phosphatase > 125 U/L 1.38

Blood urea nitrogen > 40 mg/dL 1.25

Creatinine > 1.2 mg/dL 2.04

International normalized ratio > 1.5 1.74

Platelets < 150 103/μL 1.53

SGOT > 40 U/L 1.48

Sodium > 145 mg/dL 1.77

WBC count, 103/μL<4.5 1.71

>15 and ≤25 1.31

>25 1.91

SGOT, serum glutamic oxaloacetic transaminase.

© 2016 Wolters Kluwer Health, Inc. All rights reserved.

Copyright © 2016 Wolters Kluwer H

outcomes for patients undergoing both inpatient and outpatientsurgical procedures. The ACS-NSQIP structure and methodol-ogy (i.e., data collection, sampling, variables collected, out-comes tracked, analyses performed) have been well describedand repeatedly validated in the surgical literature.14–17

Patient PopulationFor each of the derivation and validation phases, all patients

undergoing ES in 2011 as defined by the ACS-NSQIP variable“emergency case” were identified. ACS-NSQIP defines “emer-gency case” as one that is “performed as soon as possible andno later than 12 hours after the patient has been admitted tothe hospital or after the onset of related preoperative symptom-atology.”15 Cases simultaneously categorized as “elective sur-gery” were excluded.

Preoperative VariablesAll preoperative variables collected in the ACS-NSQIP

data fields were identified, including demographics (e.g., age,race, sex), comorbidities (e.g., chronic obstructive pulmonarydisease [COPD], hypertension, ascites), functional status, andpreoperative laboratory variables (e.g., sodium, albumin, andwhite blood cell [WBC] count). Laboratory values were divided

rtality in ES Patients in the 2011 ACS-NSQIP Data Set

95% CI p

1 2.293–3.066 <0.0001

7 1.055–1.404 0.007

1 1.250–1.778 <0.0001

4 1.067–1.497 0.007

3 1.307–2.039 <0.0001

1 1.407–2.007 <0.0001

0 2.481–3.848 <0.0001

0 1.234–1.635 <0.0001

1 1.538–2.016 <0.0001

2 1.302–1.803 <0.0001

8 1.117–1.463 <0.0001

7 1.110–1.610 0.002

1 2.845–3.901 <0.0001

2 1.226–2.040 <0.0001

0 1.245–1.596 <0.0001

6 1.189–1.615 <0.0001

7 1.070–1.476 0.005

8 1.785–2.349 <0.0001

8 1.512–2.020 <0.0001

6 1.339–1.764 <0.0001

4 1.289–1.709 <0.0001

0 1.366–2.293 <0.0001

6 1.379–2.135 <0.0001

8 1.140–1.524 <0.0001

6 1.535–2.391 <0.0001

215

ealth, Inc. All rights reserved.

TABLE 3. Development of the ESAS Using RegressionCoefficients of the Variables Significant for Mortality

Variable Points*

Demographics

Age > 60 y 2

White race 1

Transfer from outside emergency department 1

Transfer from an acute care hospital inpatient facility 1

Comorbidities

Ascites 1

BMI < 20 kg/m2 1

Disseminated cancer 3

Dyspnea 1

Functional dependence 1

History of COPD 1

Hypertension 1

Steroid use 1

Ventilator requirement within 48 h preoperatively 3

Weight loss > 10% in the preceding 6 mo 1

Laboratory values

Albumin < 3.0 U/L 1

Alkaline phosphatase > 125 U/L 1

Blood urea nitrogen > 40 mg/dL 1

Creatinine > 1.2 mg/dL 2

International normalized ratio > 1.5 1

Platelets < 150 103/μL 1

SGOT > 40 U/L 1

Sodium >145 mg/dL 1

WBC, 103/μL<4.5 1

>15 and ≤25 1

>25 2

Maximum score 29

Pseudo R2 0.2567

ROC 0.8647

z95% CI for ROC 0.8567–0.8726

*The OR for each variable was divided by the lowest OR (as the common denominator)and then rounded to the nearest integer to arrive at the number of points.

Sangji et al.J Trauma Acute Care Surg

Volume 81, Number 2

into low, normal, and high (where applicable) using clinically rel-evant cutoffs. Demographic factors and comorbid conditionsweredichotomized using ACS-NSQIP definitions of normal and ab-normal. Age was dichotomized into younger than 60 years or60 years and older; race, into white and nonwhite; “partially” or“totally” dependent was deemed as functional dependence; anddyspneawith moderate exertion or rest was classified as dyspnea.Body mass index (BMI) was divided into less than 20, 20 to 35,or greater than 35 kg/m2. Laboratory valueswere divided into low,normal, and high using the NSQIP definitions. WBC was furtherdivided into equal to or less than 4.5, greater than 4.5 and equal toor less than 11, greater than 11 and equal to or less than 15,greater than 15 and equal to or less than 25, and greater than25 103/μL.Missing datawere coded asmissing and not imputed.Only variables with 55% or greater capture in the ES subset of pa-tients were used for statistical analyses. Variables that incorporatemultiple preoperative characteristics such as ASA classificationand sepsis were excluded to prevent colinearity and erroneousinclusion of the variables included in these broader categoriesbecause their contribution may have already been accountedfor in the broader variables (e.g., WBC count and sepsis).

OutcomesThe primary outcome of interest assessed was 30-day

mortality using the ACS-NSQIP variable “YRDEATH.” Thisvariable reports if the patient is alive 30 days postoperatively re-gardless of admission status.

Derivation of the ScoreThe 2011 ACS-NSQIP data set was used to derive ESAS,

using a three-step methodology.First, univariate analyses of 30-day mortality were per-

formed using all available preoperative variables (e.g., patient de-mographics, comorbidities, preoperative laboratory variables).Variables with a p < 0.2 were then included in the multiple logisticregression models created to identify independent predictorsof 30-day mortality in ES. Second, stepwise logistic regressionmodels were constructed for 30-day mortality. Both forward andbackward regressions were performed with a p < 0.05 as the cut-off for statistical significance. Third, based on the relative impactof each identified predictor (i.e., odds ratio [OR]), using weightedaverages, a novel score was derived. The coefficients, namely,OR, were divided by the lowest common denominator androunded off to the nearest half integer or nearest integer in severaliterations to develop a score that would be easy to use. The re-ceiver operating characteristic (ROC) curve was examined foreach iteration to ensure consistency in theC statistic. TheC statis-tic is a measure of the ability to discriminate the outcome of inter-est (death) from cases without the outcome of interest (survival)and has been used as a measure of model success in multiplescore development efforts.13,18–20

Validation of the ScoreWith the use of the 2012 ACS-NSQIP database, the score

was validated with evaluation of its C statistic and its ability topredict mortality at 30 days in a different data set. The coeffi-cients of the score derived from 2011 data were applied to 2012data for validation. The model fit of ESAS was also compared

216

Copyright © 2016 Wolters Kluwer H

with the ASA classification through comparison of their respec-tive C statistics in the derivation and validation cohorts.

Statistical AnalysesAll data analyses were performed in STATAversion 13.1

(StataCorp, College Station, TX), as described earlier.

RESULTS

From 280,801 NSQIP cases in the 2011 data set, 44,618met our inclusion criteria, of which 18,439 ES cases capturedall the variables that were significant for mortality. Of these,1,598 (8.7%) resulted in postoperative death at 30 days. The de-mographics, comorbidities, and laboratory characteristics ofboth our derivation and our validation patient populations are in-cluded in Table 1. Of all the ES patients analyzed, approximatelyhalf were older than 60 years in both the score development andvalidation data sets; half were women; and approximately 70%were of white race.

© 2016 Wolters Kluwer Health, Inc. All rights reserved.

ealth, Inc. All rights reserved.

J Trauma Acute Care SurgVolume 81, Number 2 Sangji et al.

Derivation of the ScoreMultiple logistic regression analyses identified 22 inde-

pendent predictors of mortality in the ES derivation cohort pa-tients (Table 2). These included 3 demographic variables, 10comorbidities, and 9 laboratory variables. High BMI as a pre-dictive variable was noted to be protective and was thus ex-cluded from the stepwise regression. Based on the relativeimpact of these 22 predictors, using weighted averages, ascore was derived that ranges from 0 to 29 points (Table 3).Multiple iterations of the score to simplify the coefficientsyielded unchanged ROCs. Therefore, we propose the simplestpossible score, with an integer point scale, as the ESAS. Thisscore has a C statistic of 0.86 for mortality (95% confidenceinterval [CI], 0.8567–0.8726). The observed probability of30-day mortality gradually increased from 0% at a score of0 to 36% at a score of 11 and 100% at a score of 22 (Fig. 1).There were no observations for scores greater than 22.

Validation of the ScoreThe 2012 validation data set included 19,552 patients,

with an overall mortality rate of 7.2%. ROC curves were com-puted for both 2011 and 2012 data sets. The C statistic of ESASwas unchanged at 0.86 when applied to the 2012 ACS NSQIPdata set, with a 95% CI that overlaps with the 95% CI for ourderivation cohort (0.8456–0.8638) (Fig. 2). The observed andexpected percentage mortality for each point in the score isdepicted in Figure 3.

For the derivation cohort, 18,401 of the 18,439 patients hadan ASA score reported. TheC statistic for ASA classification as apredictor of mortality was 0.82 (95% CI, 0.8189–0.8354) in thisgroup. In the validation cohort, 19,525 of the 19,552 patientshad an ASA score reported. TheC statistic for ASA classification

Figure 1. ESAS. Observed mortality rates per ESAS points.

© 2016 Wolters Kluwer Health, Inc. All rights reserved.

Copyright © 2016 Wolters Kluwer H

as a predictor of mortality was unchanged at 0.82 (95% CI,0.8174–0.8348) in the validation cohort. The 95% CIs for the Cstatistic of ASA and ESAS do not overlap in the derivation or val-idation cohorts, suggesting a statistically significant difference be-tween the ESAS and ASA C statistics.

DISCUSSION

We have therefore derived and validated a novel preoper-ative risk assessment tool specifically for ES patients, the ESAS.In our evaluation and analyses, we believe that ESAS (1) is user-friendly, (2) is comprehensive (high ROC), and (3) impressivelypredicts mortality in a stepwise progression. Moreover, it islikely to have a much greater discriminatory power for ES pa-tients compared with more generic surgical risk assessmenttools.6,21 We demonstrate this higher discriminatory power inthe comparison of ESAS with ASA. Such discriminatory poweris crucial for the ES patient population because of its especiallyincreased risk of mortality a priori. In fact, ES patients have beenconsistently shown to have worse outcomes compared with pa-tients undergoing nonemergency general surgery even whencontrolling for preoperative variables and procedure type.6 Alarge retrospective analysis of the ACS-NSQIP database from2008 to 2012 revealed that the overall mortality was sixfoldhigher for patients undergoing emergency versus nonemergencygeneral surgery.6 The difference between ES and non-ES de-creases but does not disappear when accounting for preoperativepatient variables such as age, race, and sex. The acuity of diseaseat presentation, the inability to optimize preoperative status be-fore surgery, and the inherent time sensitivity implying an imme-diate, imminent, and real risk of complications with or without

217

ealth, Inc. All rights reserved.

Figure 2. Comparison of ROC curves using the development(A) and the validation (B) data sets. A, ESAS data set (0.86).B, ACS-NSQIP 2012 data set (0.855).

Sangji et al.J Trauma Acute Care Surg

Volume 81, Number 2

surgery all potentially account for this increased risk of deathfollowing ES.

Existing tools such as SRS, POSSUM, and the Ports-mouth predictor equation (p-POSSUM) all provide valuable in-formation for patients undergoing operations. However, they havesignificant limitations when applied to ES patients. The SRS wasvalidated on a cohort with a very low observed mortality.7 POS-SUM has been shown to overpredict mortality by at least twofoldand is quite complex to use.7,8,22 The p-POSSUM, developedusing in-hospital (rather than 30-day) mortality, adjusts the POS-SUM mortality overprediction but remains complex, requiringdetermination of intraoperative variables as well as physiologiccharacteristics.22

ESAS has a higher than or comparable C statistic at0.86 to that of widely accepted medical and surgical scoringsystems.7–9,22–26 The only large study evaluating existingscoring systems on ES patients found C statistics ranging

218

Copyright © 2016 Wolters Kluwer H

from 0.71 to 0.90 for 30-day mortality using four scores in-cluding POSSUM and SRS in a cohort of 2,349 patients in asingle, large (700-bed) UK hospital.27 ESAS offers compara-ble discrimination. However, compared with surgical scoringsystems such as SRS and the p-POSSUM, ESAS allows formortality prediction for ES patients using just preoperativevariables. ESAS contains no subjective variables such asASA class in the SRS or complexity of procedure in bothp-POSSUM and SRS. This partly explains the higher discrim-inatory power of ESAS compared with ASA, which we havedemonstrated. ESAS hence provides critical information thatcan help guide decision making for both physicians and pa-tients as they discuss the need for an emergency procedure.

The need for risk adjustment and grading models specifi-cally for ES is highlighted by the efforts underway at the AASTto develop such benchmarking tools.10–12 The current scoringsystems offer disease specific grades, but those require a combi-nation of clinical, radiographic, endoscopic, and pathologicfindings to determine a grade, and the scores currently do not ac-count for patient comorbidities or preoperative physiologicderangements.3,10–12 These scores also require validation studiesacross larger and more heterogeneous populations. We offerESAS not only as a comprehensive and validated tool but alsoas one that accurately predicts postoperative mortality progres-sively across its range of scores based on easily available preoper-ative characteristics and physiologic derangements. We envisionin the near future a freely available online calculator that allowscomputation of the ESAS. At the bedside, this can translate intopatient counseling and informed consent preoperatively for pa-tients undergoing a procedure urgently or emergently.28 At an in-stitutional or national level, this score will allow comparison andhence benchmarking of the quality of care provided for this com-plex patient population.

Our study has a few limitations. First, while the ACS-NSQIP is a large database with rich clinical information, theES subset from the 2011 data set contained several variables thathad lower than 55% capture. Those variables were excludedfrom our multivariate regressions but perhaps may be significantif captured at a higher rate. Second, the stepwise regressionmodel may not capture variables that may be significant butget discounted as other variables are selected. Third, the tooldoes not, at this time, assess for morbidity. Applying ESAS tomajor morbidity is an interesting and important next step inour research work. There is also an opportunity to generatemore targeted prediction scores for either disease-specific orprocedure-specific outcomes. While those may increase the dis-criminatory power of the score, the development of multiple sys-tems will add complexity to an already complex health careenvironment. A single, user-friendly, comprehensive score thataccurately predicts mortality in all ES patients based solely onpreoperative variables is both practical and feasible, whether inquality improvement efforts or at the bedside.

CONCLUSION

We have therefore developed and validated a novel score,ESAS, that accurately predicts postoperative mortality in ES pa-tients. Such a score could prove useful for (1) preoperative

© 2016 Wolters Kluwer Health, Inc. All rights reserved.

ealth, Inc. All rights reserved.

Figure 3. ESAS expected and observed percentage mortality.

J Trauma Acute Care SurgVolume 81, Number 2 Sangji et al.

patient counseling, (2) identification of patients needing closepostoperative monitoring, and (3) risk adjustment in any effortsat benchmarking the quality of ES.

AUTHORSHIP

N.F.S., J.D.B., E.P.R., D.C.C., and H.M.A.K. designed this study. N.F.S., J.D.B., E.P.R.,and D.C.C. performed the data collection. N.F.S., J.D.B., E.P.R., D.C.C.,and H.M.A.K. performed the data analysis/interpretation. N.F.S., J.D.B., D.C.C., and H.M.A.K. performed the statistical analysis. N.F.S. performedthe literature search. N.F.S., J.D.B., D.C.C., H.M.A.K., D.D.Y., D.R.K., M.D., K.B., P.J.F., and G.C.V. contributed to the writing and critical revisions.

ACKNOWLEDGMENT

WethankDr.MatthewHutter (from theDepartment of Surgery,MassachusettsGeneral Hospital, and Director, Codman Center for Clinical Effectivenessin Surgery, Massachusetts General Hospital) and Ms. Donna Antonelli(Codman Center for Clinical Effectiveness in Surgery, Massachusetts Gen-eral Hospital) for their guidance in using the ACS-NSQIP database.

DISCLOSURE

The authors declare no conflicts of interest.

REFERENCES1. Saklad MDM. Grading of patients for surgical procedures. Anesthesiology.

1941;2(3):281–284.2. American Association for the Surgery of Trauma (AAST), Acute Care

Surgery: Introduction. Available at: http://www.aast.org/AcuteCareSurgery.aspx. Last accessed August 20, 2015.

3. Shafi S, AboutanosMB, Agarwal S Jr, Brown CV, Crandall M, FelicianoDV,Guillamondegui O, Haider A, Inaba K, Osler TM, et al. Emergency generalsurgery: definition and estimated burden of disease. J Trauma Acute CareSurg. 2013;74(4):1092–1097.

4. Gale SC, Shafi S, Dombrovskiy VY, Arumugam D, Crystal JS. The publichealth burden of emergency general surgery in the United States: a 10-yearanalysis of the Nationwide Inpatient Sample—2001 to 2010. J TraumaAcute Care Surg. 2014;77(2):202–208.

© 2016 Wolters Kluwer Health, Inc. All rights reserved.

Copyright © 2016 Wolters Kluwer H

5. Ingraham AM, Cohen Mark E, Bilimoria KY, Raval MV, Ko CY, NathensAB, Hall BL. Comparison of 30-day outcomes after emergency generalsurgery procedures: potential for targeted improvement. Surgery. 2010;148(2):217–238.

6. Havens JM, Havens JM, Peetz AB, Do WS, Cooper Z, Kelly E, Askari R,Reznor G, Salim A. The excess morbidity and mortality of emergencygeneral surgery. J Trauma Acute Care Surg. 2015;78(2):306–311.

7. Sutton R, Bann S, Brooks M, Sarin S. The Surgical Risk Scale as animproved tool for risk-adjusted analysis in comparative surgical audit. Br JSurg. 2002;89(6):763–768.

8. Copeland GP, Jones D, Walters M. POSSUM: a scoring system for surgicalaudit. Br J Surg. 1991;78(3):355–360.

9. Kwok AC, Lipsitz SR, Bader AM, Gawande AA. Are targeted preoperativerisk prediction tools more powerful? A test of models for emergency colonsurgery in the very elderly. J Am Coll Surg. 2011;213(2):220–225.

10. Savage SA, Klekar CS, Priest EL, Crandall ML, Rodriguez BC, Shafi S.Validating a new grading scale for emergency general surgery diseases. JSurg Res. 2015;196(2):264–269.

11. Shafi S, Aboutanos M, Brown CV, Ciesla D, Cohen MJ, Crandall ML, InabaK, Miller PR, Mowery NT. Measuring anatomic severity of disease inemergency general surgery. J Trauma Acute Care Surg. 2014;76(3):884–887.

12. Crandall ML,Agarwal S,Muskat P, Ross S, Savage S, Schuster K, TominagaG, Shafi S. Application of a uniform anatomic grading system to measuredisease severity in eight emergency general surgical illnesses. J TraumaAcute Care Surg. 2014;77(5):705–708.

13. Cohen ME, Bilimoria KY, Ko CY, Hall BL. Development of an AmericanCollege of Surgeons National Surgery Quality Improvement Program:morbidity and mortality risk calculator for colorectal surgery. J Am Coll ofSurg. 2009;208(6):1009–1016.

14. ACS NSQIP, User Guide for the 2012 ACS NSQIP Participant Use DataFile. 2012. Available at: https://www.facs.org/~/media/files/quality%20programs/nsqip/ug12.ashx. Accessed February 22, 2015.

15. ACS NSQIP, User Guide for the 2011 Participant Use Data File. 2011.Available at: https://www.facs.org/~/media/files/quality%20programs/nsqip/ug11.ashx. Accessed February 22, 2015.

16. ACS NSQIP, Data Collection, Analysis, and Reporting. Available at: https://www.facs.org/quality-programs/acs-nsqip/program-specifics/data. AccessedAugust 15, 2015.

17. ACS NSQIP, About ACS NSQIP. Available at: https://www.facs.org/quality-programs/acs-nsqip/about. Accessed August 15, 2015.

18. Hanley JA, McNeil BJ. The meaning and use of the area under a receiveroperating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.

219

ealth, Inc. All rights reserved.

Sangji et al.J Trauma Acute Care Surg

Volume 81, Number 2

19. Easterlin MC, Chang DC, Wilson SE. A practical index to predict 30-daymortality after major amputation. Ann Vasc Surg. 2013;27(7):909–917.

20. Wong CH, Khin LW, HengKS, Tan KC, LowCO. The LRINEC (LaboratoryRisk Indicator for Necrotizing Fasciitis) score: a tool for distinguishingnecrotizing fasciitis from other soft tissue infections. Crit Care Med.2004;32(7):1535–1541.

21. Shah AA, Haider AH, Zogg CK, Schwartz DA, Haut ER, Zafar SN,Schneider EB, Velopulos CG, Shafi S, Zafar H, et al. National estimates ofpredictors of outcomes for emergency general surgery. J Trauma AcuteCare Surg. 2015;78(3):482–490; discussion 490–1.

22. Whiteley MS, Prytherch DR, Higgins B, Weaver PC, Prout WG. Anevaluation of the POSSUM surgical scoring system. Br J of Surg. 1996;83(6):812–815.

23. Alvarado A. A practical score for the early diagnosis of acute appendicitis.Ann Emerg Med. 1986;15(5):557–564.

24. Kamath PS,Wiesner RH,MalinchocM, KremersW, Therneau TM, KosbergCL, D'Amico G, Dickson ER, Kim WR. A model to predict survival inpatients with end-stage liver disease. Hepatology. 2001;33(2):464–470.

25. Gawande AA, Kwaan MR, Regenbogen SE, Lipsitz SA, Zinner MJ. AnApgar score for surgery. J Am Coll Surg. 2007;204(2):201–208.

26. Meredith JW, Evans G, Kilgo PD, MacKenzie E, Osler T, McGwin G, CohnS, Esposito T, Gennarelli T, Hawkins M, et al. A comparison of the abilitiesof nine scoring algorithms in predicting mortality. J Trauma. 2002;53(4):621–628; discussion 628–9.

27. Neary WD, Prytherch D, Foy C, Heather BP, Earnshaw JJ. Comparison ofdifferent methods of risk stratification in urgent and emergency surgery. BrJ Surg. 2007;94(10):1300–1305.

28. Bernat JL, Peterson LM. Patient-centered informed consent in surgicalpractice. Arch Surg. 2006;141(1):86–92.

DISCUSSIONDr. James Davis (Fresno, California): The study repre-

sents an important step into better defining the risk andbenchmarking emergency surgery. I only have a few questions.

The authors used age as a dichotomous variable, but weheard in the presentations yesterday that there are significant dif-ferences between being a 60-year-old and a 75-year-old. And asI approach that first number ever faster I would like to be risk-adjusted differently.

Why did you choose not to divide age by quartiles or evendeciles? I suspect that this increased granularity would increasethe robustness of your scoring system.

You noted an increased risk for patients transferred fromother facilities for their operations. Is this related to the com-plexity of the surgical problem that necessitated the transferand is it a surrogate for that? Or is it a function of time frompresentation at an emergency department someplace to theoperating room?

Further, you did not address this time issue—time from theED or other hospital until they had definitive surgical care—asa risk factor for adverse outcome. Is this a variable that was notexamined or one that did not prove to be significant?

I would recommend a careful reading of this well-writtenmanuscript to the audience. And I commend the authors for thisexcellent contribution. Thank you.

Dr. Ronald Simon (NewYork, NewYork): I just have onequick question. You can already do a mortality calculator fromNSQIP and I’m wondering whether or not you looked atwhether or not you are better or worse. Thank you.

220

Copyright © 2016 Wolters Kluwer H

Dr. Garth Utter (Sacramento, California): Of course thisis a very heterogeneous population of patients. Might there bedisease-specific predictors that are more important to tease out?

And, ultimately, is there any real difference between out-come prediction in EGS patients versus in all non-EGS patients?

Also, I would point out there are many, many patients ex-cluded. Could there be something about these missing patientsthat makes prediction among the rest less reliable?

Dr. Naveen F. Sangji (Boston, Massachusetts): Thankyou, Dr. Davis, for your feedback and your comments. Ad-dressing your first question about using age as a dichotomousvariable rather than deciles, and that is something that we de-bated within our group while we were developing this score.

We decided to use a dichotomous cut-off for age just tokeep the score more user-friendly rather than using quartilesand deciles. But you are correct that that may yield some moreimportant information.

Your next question about time of transfer to the hospital,whether outcomes were related to the time to the OR, the timespent at the outside facility, and that is a great question.

Those variables are unfortunately currently not capturedby theNSQIP, at least 2011–2012 databases. In the future if thosevariables are captured, as I understand TQIP is starting to capturetime to the emergency department of the hospital where the pa-tient receives ultimate treatment, then that would be an importantstep to study as well.

Moving on the question by Dr. Simon about the NSQIPcalculator, that is a great tool. The calculator uses demographicand comorbidities to predict outcomes.

The calculator was developed on the entire cohort of theNSQIP patients. And as the paper by Dr. Cliff Coe and Dr. BillMoria mentions when they presented the score, it is meantmostly for use for elective surgery patients.

Now that calculator does have a variable for emergencysurgery, however, when we move on to Step 2 of that calculatorthere is an area where a surgeon is allowed to use a surgeon-ad-justment score—so it’s a subjective assessment of whetherthey think the patient is more or less sick than the average pa-tient presenting with that condition, which is a difficult assess-ment to make.

What our score does with the inclusion of lab variables,which the NSQIP calculator doesn’t have, is that it allows for ob-jective assessment of those acuity factors.

Moving on to Dr. Utter’s question about disease-specificpredictions— you are absolutely correct, that is also somethingthat we are considering as a next step of our study.

The American College of Surgeons calculator for electiveprocedures has shown very little change in the predicting powerof disease-specific versus a universal risk calculator. So, for ex-ample, they compared their universal calculator to the colorectalcalculator and did not find much difference. But you are correct.That may have some impact.

However, to keep this simple and user-friendly we decidedto stick with the universal calculator but we will be exploringthat in the future.

© 2016 Wolters Kluwer Health, Inc. All rights reserved.

ealth, Inc. All rights reserved.