ATsAm

F

EECTECGa

B

Rytmspotinptthhht

i

i

Pp1

APM

Pers

pect

ives

0d

PM Perspectiveshe Association of Professors of Medicine (APM) is the national organization of departments of internal medicine at the US medicalchools and numerous affiliated teaching hospitals as represented by chairs and appointed leaders. As the official sponsor of Themerican Journal of Medicine, the association invites authors to publish commentaries on issues concerning academic internaledicine.

or the latest information about departments of internal medicine, please visit APM’s website at www.im.org/APM.

xperience with Faculty Supervision of anlectronic Resident Sign-out System

hristopher Nabors, MD, PhD,a Stephen J. Peterson, MD,a Wei-Nchih Lee, MD, MPH,b Arif Mumtaz, MD,a

ushar Shah, MD,a Sachin Sule, MD,a Andrew H. Gutwein, MD,c Leanne Forman, MD,a Etta Eskridge, MD,a

ric Wold, MD, MS,a Gary W. Stallings, MD, MPH,a Kathleen Kelly Burak, MD,a Carol Karmen, MD,a

aren F. Behar, MD,a Christine Carosella, MD,a Shick Yu, MD,a Kausik Kar, MD,a Melissa Gennarelli, MD,a

ail Bailey-Wallace, MD,a Randy Goldberg, MD,a Gary Guo, MD, PhD,a William H. Frishman, MDa

Department of Medicine, New York Medical College/Westchester Medical Center, Valhalla, NY; bStanford Center for

iomedical Informatics Research, Stanford, Calif; and cDepartment of Medicine, Jacobi Medical Center, Bronx, NY.

dorwtabs

eohoremsaHircdmwhs

eductions in residency work hours during the past 20ears have resulted in a corresponding increase in bothhe number of transitions of care and the chances ofedical errors related to such transitions.1 Despite con-

iderable efforts to improve the house staff handoffrocess,2-13 problems remain.7,14 Recent studies dem-nstrate that an alarming proportion of handoffs con-inue to omit important information, contain erroneousnformation, and lack a standardized structure.14,15 Ofote, although lack of supervision was one of the mainroblems that led to the implementation of 405 regula-ions in New York and similar regulations elsewhere,he idea of faculty supervision of the sign-out processas received little attention.5,14 In the past, it wouldave been difficult for faculty members, each of whomad only a few patients in the hospital at a given time,o oversee the sign-out process. However, most aca-

Funding: None.Conflict of Interest: None of the authors have any conflicts of

nterest associated with the work presented in this manuscript.Authorship: All authors had access to the data and played a role

n writing this manuscript.Reprint requests should be addressed to Christopher Nabors, MD,

hD, New York Medical College, Westchester Medical Center, De-artment of Medicine, Room 529 Munger Pavilion, Valhalla, NY0595.

oE-mail address: christopher_nabors@nymc.edu

002-9343/$ -see front matter © 2010 The Association of Professors of Meoi:10.1016/j.amjmed.2009.12.010

emic medical centers now operate under fully devel-ped hospitalist care models where the faculty typicallyemain in-house throughout the day, manage teamsith fairly uniform schedules, and are available at the

ime of afternoon sign-outs.1 Given the technologicdvances and the changes in the care structure, weelieve that the time has come for faculty to assume fullupervision of the sign-out process.

More than a year ago, we began using the comput-rized Patient Documentation Transfer System (PDTS)n the medicine service to improve on our preexistingandoff system that already included face-to-face sign-uts at standard locations and times and where inter-uptions were minimized. Like similar systems in uselsewhere, the PDTS database stores key patient infor-ation and allows printing of standardized sign-out

heets that are relayed from primary teams to “on call”nd “night float” teams (5 PM and 9 PM, respectively).owever, during the past year we enhanced PDTS to

nclude a faculty “oversight module” by which facultyeview and approve all handoff information before itan be signed out to coverage teams and through whichata regarding sign-out were collected.1 We supple-ented the data collected from the oversight moduleith data derived from surveys of both the faculty andouse staff using the system. The current report de-cribes our experience with the new computerized sign-

ut system.

dicine. All rights reserved.

MAsd

rBtac(dtefwmmafigaPchpsPdwibd

HImwosam5ttehpapm

FIts

fiistt

pempotct4smrmfit

care

377Nabors et al Electronic Resident Sign-out System

ATERIALS AND METHODSttending physician hospitalists began supervision of all

ign-outs on the general medicine service in June 2008 usingrop-down and free text boxes as shown in Figure 1.

Interns completed entry of sign-out data by 4 PM, aftereview by the supervising resident.etween 4 and 4:30 PM, the hospi-

alists made corrective notations viany hospital (or Internet capable)omputer. The supervising residentsin cooperation with the chief resi-ent) then reviewed all oversight at-ending evaluations and contacted rel-vant team members to ensure thataculty corrections to the sign-outere entered. In the event that infor-ation was considered to be of im-ediate import to patient care, the

ttending physician personally noti-ed the primary intern or resident re-arding the desired change of plans inddition to correcting the sign-out onDTS. Only after ensuring that allhanges to their patient informationad been approved by the attendinghysician did the interns complete theign-out process between 4:30 and 5M in the presence of the chief resi-ent. The attending oversight dataere maintained in the PDTS admin-

strative database and reviewed dailyy chief residents and the programirector.

ouse Staff Surveyn March 2009, we surveyed the 24embers of our 30-person intern classho were not off-service using a seriesf 29 questions that had been pre-creened for clarity by 3 chief residentsnd the Program Director. Of the 29 questions, there were 15ultiple choice, 3 Likert-scale, 5 Likert-style, 1 ordinal, andopen-comment type questions. These assessed the percep-

ions of interns regarding the functionality and usefulness ofhe PDTS system, as well as their perceptions regarding theffects that attending oversight and adverse events reportingad on the quality of patient care and safety, and the sign-outrocess itself. The data regarding “adverse events” reportingnd the “peer-to-peer” sign-out evaluation process will beresented elsewhere. No citation is available at present as theanuscript is in final stages of preparation.

aculty Surveyn July 2009, each of the 8 hospitalists who supervisedhe general medicine service house staff during this

PERSPECTIVE

● Reductionshave increastions of careerrors relate

● Transitions ofditionally invo

● The hospitawith technolsible facultysign-out pro

● Surveyed hosan academiccorporates fout within tsoftware) fouand practicasafety.

● A handheldconcerns reversions of s

● A handheld vware permitsand multiplestaff patient

tudy responded to a survey, which consisted of 19 s

ll-in-the-blank and multiple choice questions regard-ng the general functionality of PDTS, the faculty’supervision of handoffs before and after the implemen-ation of PDTS, and the faculty’s perceptions regardinghe usefulness of the oversight system.

RESULTS

Attending OversightDuring February 2009,1225 sign-outs on the gen-eral internal medicine ser-vice were reviewed by fac-ulty for accuracy andcompleteness of demo-graphics, code status, aller-gies, history of present ill-ness, recent clinical events(daily update, “very sick”status), medication list, di-agnosis, and plan of action(“to-do’s”) for the on-calland night-float teams. Ofthe 1225 sign-outs, 7%were considered to be incor-rect. Of those, Figure 2shows the percentage ofhandoffs that were incorrectwith respect to each definedcategory. Faculty rated6.5% of handoffs to be in-correct for Department ofMedicine house staff com-pared with 12.4% for intern“rotators” on the medicineservice.

House Staff SurveyFunctionality. Sixty-three

ercent of interns thought that the PDTS system wasither good or excellent in terms of its overall quality inanaging patient sign-out information. Eighty-four

ercent stated that they would use this system over anyther method of sign-out. Eighty-nine percent believedhe system took either the same time or saved timeompared with other sign-out methods. With respect toime spent updating sign-out information each day, 5%,8%, 19%, 24%, and 5% of interns reported using theystem 0 to 10 minutes, 11 to 20 minutes, 21 to 30inutes, 31 to 40 minutes, and 41 to 50 minutes,

espectively. No interns reported using more than 50inutes to update patient information. Comments af-rmed the intern classes’ overall approval of the sys-

em’s functionality but requested improved speed and

WPOINTS

sidency work hourse number of transi-

he chances of medicaluch transitions.

sign-outs) have not tra-attending supervision.

are model, combinedadvances, makes pos-ight of the house staff

sts and house staff atical center (which in-

supervision of sign-omputerized sign-oute system to be usefulto enhance patient

m eliminates privacyto misplaced paper

ut.

n of the sign-out soft-ty to oversee sign-outer features of housein real time.

S VIE

in reed thand td to s

care (lved

list cogicoverscess.

pitalimed

acultyhe cnd thl, and

systelatedign-o

ersiofaculoth

ought a handheld version of the system.

POpytrm1swiwgomow1fwbo

att

FOnsttppastbiafPt“[tc(rcfloomarstta

attend

378 The American Journal of Medicine, Vol 123, No 4, April 2010

atient Safety, Attending Oversight, Adverse Events.f surveyed interns, 67% believed that PDTS improvedatient safety. When asked, “with what frequency doou believe having readily available informationhrough PDTS prevents an adverse event from occur-ing?” 24% responded greater than 10 instances peronth, 29% answered 5 to 10 instances per month,

8% stated 1 to 5 instances per month, and 29% re-ponded 1 or less instances per month. The intern classas asked, “how much is the quality of [your] sign-out

nformation improved by attending oversight?” tohich 84% thought that it was improved to some de-ree. In response to the statement, “An attending’sversight evaluation results in a change in my patient’sanagement with respect to what percentage of sign-

uts,” 23% of the interns responded “almost never,”hereas the remaining interns believed that at least0% of his/her patients’ management was altered byaculty supervision of sign-outs. When asked, “withhat frequency would you estimate that attending feed-ack on a sign-out prevents an adverse event fromccurring . . .?” 36% believed that oversight prevented

Figure 1 Snapshot of

1%

0%

21%

5%

0%

15%

33%

25%

Inadequate: Did Not BehaveAs OutlinedInadequate: Very Sick notFlaggedInadequate: HPI

Inadequate: Dx

Inadequate: Allergy

Inadequate: Med List

Inadequate: To Do

Inadequate: Others

Figure 2 Percentage of handoffs noted to be defectiveby faculty with respect to each defined PDTS sign-outdata category. HPI � history of present illness; Dx �

pdiagnosis.

n adverse event 50% of the time or more, and 65%hought that oversight prevented adverse events 25% ofhe time or less.

aculty Surveyf the faculty responding to our survey, the averageumber of years in practice was 12. All physicians (8/8)urveyed believed that PDTS “improved the informa-ion available to [him/her] regarding [his/her] patients’ransfers of care.” Only 1 faculty member (1/8) re-orted having previously reviewed (before PDTS im-lementation) his/her patients’ sign-out information,nd none had previously received a written copy ofign-out information. However, since the date on whichhe PDTS system was implemented, each faculty mem-er stated that s/he now routinely reviews all sign-outnformation. The average time spent per patient byttendings in overseeing sign-outs was 2 minutes. Eachaculty member believed that the implementation ofDTS allowed him/her to feel more confident (than was

he case prior to PDTS inception) that he/she isaware of what information is being signed out onhis/her] patients?” The mean percent of handoffshat faculty estimated they “correct” was 12. Of theorrected sign-outs, the faculty estimated that 7%mean) of those sign-outs “represent a serious matterelated to patient safety or quality of care.” Over theourse of an average month on the general medicineoors, faculty estimated that on approximately 1 to 2ccasions (range 0-5), their oversight of PDTS sign-ut information prevented the occurrence of a seriousedical error or adverse event. Each physician

greed that the implementation of PDTS oversightegarding “very sick” patients made it easier to en-ure that “nothing [was] overlooked.” The facultyhought that with the addition of PDTS oversighthey had improved “control” of sign-out informationnd better access to information about what hap-

ing oversight module.

ened with their patients overnight.

DTaiCoTctrbtshibfotasAatssis

AWorwbwdloepvq“lifOoposrgi

spacchnrcfwipfldvhrpibtmfwmfosAwet

FTPbtbahhtTgenwssoat

379Nabors et al Electronic Resident Sign-out System

ISCUSSIONhe characteristics of an ideal patient handoff systemre not certain. However, several features have beendentified as being desirable.3-9,12,16 In 2006, the Jointommission made a “standardized approach to ‘handff’’ communications” a National Patient Safety Goal.he 2009 Safety Goals state that effective hand-offommunication includes interactive communicationhat allows for questioning between the giver and theeceiver of patient information. The handoff should beased on up-to-date information and include a methodo verify the accuracy of received information andhould allow an “opportunity for the receiver of theand-off information to review relevant patient histor-cal data . . . . ” Interruptions during hand-offs shoulde limited to minimize the possibility that informationails to be conveyed or is forgotten.17 Standardizationf the sign-out format and order and use of writtenemplates for sign-out improve sign-out consistencynd accuracy.2,4,5,9,11,13,16 Specialized training of housetaff in the art of sign-out also has been recommended.5

lthough supervision of house staff has been identifieds importantly related to patient safety,18,19 actual at-ending supervision of information signed out by housetaff seems to be unusual.5,14,20 Whether such supervi-ion might prove to be of benefit to the handoff processs not yet known. However, the results of this studyuggest that this must be the case.

ttending Oversighte now provide formal attending oversight of all sign-

uts on the general medicine service. Of 1225 sign-outseviewed by faculty during a 1-month time period, 7%ere found to be deficient and were modified on theasis of the attending’s feedback. Our results agreeith earlier studies showing that sign-out data are ofteneficient.14,21-23 As expressed by Borowitz and col-eagues,15 the “sign-out between resident physicians isften inadequate and incomplete . . . possibly due toxchange of the wrong information during the sign-outrocess.” In our study, although 65% of interns sur-eyed stated that attending oversight improved theuality of sign-out information either “not at all,”somewhat,” or “a little,” 77% of the interns neverthe-ess stated that attending oversight resulted in a changen patient management 10% of the time or more. Ouraculty estimated that they “correct” 12% of sign-outs.f those corrected sign-outs, faculty estimated that 7%f the sign-outs “represent a serious matter related toatient safety or quality of care.” Faculty estimated thatn 1 to 2 occasions per month their oversight of PDTSign-out information prevented the occurrence of a se-ious medical error or adverse event. These data sug-est that faculty supervision has already made a signif-

cant contribution to patient safety in our hospital. f

On the basis of survey results, attending review ofign-out requires only approximately 2 minutes peratient and requires interns to await feedback fromttendings for approximately 15 to 30 minutes beforeompleting sign-out. Thus, the attending oversight pro-ess does engender some inconvenience to faculty andouse staff. However, the benefits are potentially sig-ificant. First and foremost, the attending physician isesponsible for ensuring that information relayed toovering physicians is accurate and useful. It is there-ore reasonable to expect that the attending physicianould personally approve of such information before it

s passed along to call teams. If only a small additionalercentage of handoff problems are “caught” by suchormal oversight, the effect could be substantial on aarger scale. Attending feedback at the end of the workay also may promote a sense of team unity and pro-ide valuable guidance to interns in terms of learningow to formulate optimal sign-outs. Notably, the interneceiving faculty-approved handoff information is em-owered to “trust” the information s/he receives, allow-ng him/her to perform more effectively.7 As discussedy Johnson and Arora,7 “. . . a resident who does notrust information received during handover will ulti-ately check each detail of each patient to obtain in-

ormation needed to provide the best patient care,”asting valuable time that could have been spent onore important tasks.7 Given the potential benefits of

aculty sign-out oversight and the modest commitmentf time required for the process, it seems that facultyupervision of handoffs is both practical and desirable.dditional studies are necessary to demonstratehether attending oversight actually prevents adverse

vents (as suggested by our survey results) or improveshe sign-out technique by house staff.

uture Direction: Handheld-based Sign-outshe experience gained during the modifications of ourDTS sign-out system led to generalizations that haveeen useful in guiding further developments with thisechnology. First, the sign-out computer system muste easy to use and must contain complete, reliable,ccurate, and up-to-date information. To this end, weave transferred most of the functions of the system toandheld devices, whereas personal computer worksta-ions will back up the features located on handhelds.he new system allows sorting of patient information touide the on-call physician through work duties. Forxample, at that beginning of a call shift the softwareow automatically generates a complete list of patientsithin each on-call intern’s coverage area and prompts

ign-out of high-risk patients first. As each patient isigned out, the covering intern checks that patient offn his/her coverage list (ensuring the actual sign-out ofll patients). Thereafter, the default screen appearing onhe handheld displays the intern’s “to-do” list compiled

rom all sign-outs received. The “to-do’s” are ordered

baccrinctitattfch

ticglemrta

buprpmsvssc

gihadpBeap“iap

CTotimWssntoi

ATaisrlscWdPit

R

1

380 The American Journal of Medicine, Vol 123, No 4, April 2010

y priority and due times so that the most urgent mattersre always at the top of the intern’s “to-do” list. During theall shift, the intern is guided by the “to-do” list andhecks off each “to-do” as it is completed, compiling aecord of all work done for later sign-out. When the interns notified of a new clinical event, s/he finds the patient’same on a list and taps on a “called on” button andompletes drop-down and free text boxes to make a no-ation of the event. New “to-do’s” placed by a coveringntern and new patients admitted are automatically addedo the intern’s on-call report. Should the intern requiredditional information about a patient, the coverage list orhe entire patient list is accessed and the particular pa-ient’s name is tapped, linking the intern to that patient’sull set of information. At the end of the call shift, PDTSompiles a “sign-out” report by which the intern signs outis/her coverage activities.

By organizing an intern’s on-call work around theasks requiring active intervention (to-do’s) and keep-ng close at hand (but in the background) the keylinical information the intern might need for contin-encies, the PDTS system is tailored to compensate forimitations of the human mind and to enhance physicianffectiveness in handling coverage duties. This infor-ation is available to the attending on call and for

eview by the attending physician of record. This helpshe attending prioritize which patient to see first onrrival for rounds the next morning.

Other important innovations found in our handheld-ased system include instant messaging, which allows aser to send an automatically routed message to theerson caring for a particular patient. For example, aadiologist can instant message the intern caring for aatient with a pulmonary embolus on a computed to-ography scan without needing to refer to a coverage

chedule or to make a phone call.4 Critical laboratoryalues also are messaged directly to the covering phy-ician’s handheld device. We anticipate using the mes-aging function for rapid recognition events and foralling codes in the near future.

PDTS now also has an “autopopulation” function thatenerates a new patient identification whenever an arriv-ng patient is placed by the admitting office into theospital’s order entry system. This sends a “to-do” to thedmitting resident’s handheld device, prompting the resi-ent to admit the patient or to transfer and sign-out theatient to the correct house staff member for admission.y having all admissions routed through this system, wensure that no patient can slip through “the cracks” and bedmitted to a hospital bed without proper notice to ahysician. Once the patient is entered into the system, allto-do’s,” messages, and other tasks are tracked and ver-fied, with sequential messaging to supervising house staffnd attending physicians in the event a task is not com-

leted timely or a message is not timely answered.

ONCLUSIONShe sign-out systems of the future will rely increasinglyn innovative technologies that permit rapid access tohe most relevant patient information, guide cliniciansn their work duties, and incorporate quality improve-ent features that do not increase physician work load.e also believe the time has come for faculty to as-

ume responsibility for the accuracy and safety of theign-out process and believe strongly that PDTS tech-ology provides a suitable means for accomplishinghis goal. Successful implementation of this technologyn a large scale could go a long way in reducing errorsn patient care related to sign-outs in teaching hospitals.

CKNOWLEDGMENTShe authors thank Drs Harry Steinberg, Cindy Baskin,nd Howard Kerpen of North Shore-Long Island Jew-sh Health System for introducing us to the PDTSoftware program. The authors also thank Reggie Car-ion for the significant software enhancements that al-ow for attending supervision and correction of theign-out process. The PDTS software package is nowommercially available. New York Medical College atestchester Medical Center Internal Medicine Resi-

ency Training Program is an Education Innovationsroject designee. The program is a by-product of the

nnovations process fostered by the Education Innova-ions Project.

eferences1. Conigliaro J, Frishman WH, Lazar EJ, Croen L. Internal medi-

cine housestaff and attending physician perceptions of the impactof the New York State Section 405 regulations on workingconditions and supervision of residents in two training programs.J Gen Intern Med. 1993;8:502-507.

2. Arora V, Johnson J. A model for building a standardized hand-off protocol. Jt Comm J Qual Patient Saf. 2006;32:646-655.

3. Berthold J. Improving handoffs through better communication.ACP Hospitalist. July 2009.

4. Chacko V, Varvarelis N, Kemp DG. eHand-offs: an IBM LotusDomino application for ensuring patient safety and enhancingresident supervision in hand-off communications. AMIA AnnuSymp Proc. 2006:874.

5. Chu ES, Reid M, Schulz T, et al. A structured handoff programfor interns. Acad Med. 2009;84:347-352.

6. Horwitz LI, Moin T, Green ML. Development and implementa-tion of an oral sign-out skills curriculum. J Gen Intern Med.2007;22:1470-1474.

7. Johnson JK, Arora VM. Improving clinical handovers: creatinglocal solutions for a global problem. Qual Saf Health Care.2009;18:244-245.

8. Kemp CD, Bath JM, Berger J, et al. The top 10 list for a safe andeffective sign-out. Arch Surg. 2008;143:1008-1010.

9. Lee LH, Levine JA, Schultz HJ. Utility of a standardized sign-outcard for new medical interns. J Gen Intern Med. 1996;11:753-755.

0. Smorenburg S, Johnson J. Spotlight on the night: a 5-star pro-gramme to improve safety at night and weekends. InternationalForum on Quality and Safety in Health Care, March 20, 2009,

Berlin.

1

1

1

1

1

1

1

1

1

2

2

2

2

381Nabors et al Electronic Resident Sign-out System

1. Van Eaton EG, Horvath KD, Lober WB, Pellegrini CA. Orga-nizing the transfer of patient care information: the developmentof a computerized resident sign-out system. Surgery. 2004;136:5-13.

2. Wayne JD, Tyagi R, Reinhardt G, et al. Simple standardizedpatient handoff system that increases accuracy and completeness.J Surg Educ. 2008;65:476-485.

3. Van Eaton EG, Horvath KD, Lober WB, et al. A randomized,controlled trial evaluating the impact of a computerized roundingand sign-out system on continuity of care and resident workhours. J Am Coll Surg. 2005;200:538-545.

4. Horwitz LI, Moin T, Krumholz HM, et al. What are coveringdoctors told about their patients? Analysis of sign-out amonginternal medicine house staff. Qual Saf Health Care. 2009;18:248-255.

5. Borowitz SM, Waggoner-Fountain LA, Bass EJ, Sledd RM.Adequacy of information transferred at resident sign-out (in-hospital handover of care): a prospective survey. Qual Saf HealthCare. 2008;17:6-10.

6. Petersen LA, Orav EJ, Teich JM, et al. Using a computerizedsign-out program to improve continuity of inpatient care and

prevent adverse events. Jt Comm J Qual Improv. 1998;24:77-87.

7. The Joint Commission on Accreditation of Healthcare Organiza-tions. National Patient Safety Goals, Elements of Performance;2009. Available at: http://www.jointcommission.org/NR/rdonlyres/31666E86-E7F4-423E-9BE8-F05BD1CB0AA8/0/HAP_NPSG.pdf.Accessed March 11, 2010.

8. Hobgood C, Hevia A, Tamayo-Sarver JH, et al. The influence of thecauses and contexts of medical errors on emergency medicine res-idents’ responses to their errors: an exploration. Acad Med. 2005;80:758-764.

9. Shojania KG, Fletcher KE, Saint S. Graduate medical educationand patient safety: a busy—and occasionally hazardous—inter-section. Ann Intern Med. 2006;145:592-598.

0. Horwitz LI, Krumholz HM, Green ML, Huot SJ. Transfers ofpatient care between house staff on internal medicine wards: anational survey. Arch Intern Med. 2006;166:1173-1177.

1. Arora V, Johnson J, Lovinger D, et al. Communication failures inpatient sign-out and suggestions for improvement: a critical in-cident analysis. Qual Saf Health Care. 2005;14:401-407.

2. Gandhi TK. Fumbled handoffs: one dropped ball after another.Ann Intern Med. 2005;142:352-358.

3. Sutcliffe KM, Lewton E, Rosenthal MM. Communication fail-ures: an insidious contributor to medical mishaps. Acad Med.

2004;79:186-194.