discharge documents: how well do they support care

© 2018 ECRI INSTITUTE© 2018 ECRI INSTITUTE

Discharge Documents: How well do they

support care coordination?

Erin Sparnon, MEng, Engineering Manager

Polly Tremoulet, PhD, Human Factors Scientist


• Power Point Slides viewed here• Today’s session is recorded• Today’s slides and recording will be

posted to all PSO web portals.


• Please submit your text questions andcomments using the Questions Panel

Remember…

How to Ask Questions


Download Slides


This activity has been approved for 1.0 California State Nursing contact hour

by the provider, Debora Simmons, who is approved by the California Board

of Registered Nursing, Provider Number CEP 13677.

All faculty members involved in ECRI Institute’s July 19, 2018 live webinar,

Discharge Documents – How Well Do They Support Care Coordination?

have disclosed in writing that they do not have any relevant conflicts or

financial affiliations.


To qualify for credit:

Credit will only be issued to attendees that are individually

registered and attend the entire program. Each individual

participant must logon prior to the start of the program and remain

on the line for the entirety of the program. This is how individual

timed attendance is verified. In addition, you must complete an

attestation survey included in the post webinar evaluation at the

conclusion of the webinar. Once all that information is verified, a

certificate will be e-mailed to you within 60 days of today’s

program.


Today’s Speaker: Dr. Patrice Tremoulet

Dr. Patrice Tremoulet is an Assistant Professor of Psychology at Rowan University and a

Human Factors consultant for Children’s Hospital of Philadelphia (CHOP) and ECRI

Institute. Before joining ECRI as a consultant, she conducted a broad range of human-

machine interaction research at Bell Laboratories, Lockheed Martin, and Drexel

University. She is currently leading a CHOP effort exploring how to ensure that children

can ride safely in self-driving vehicles and is supporting multiple ECRI projects by helping

to assess the usability of different medical devices.

Dr. Tremoulet received her Ph.D. in Psychology and a certificate in Cognitive Science

from Rutgers University, an M.S. in Operations Research from Stanford University and a

B.S.E. in Operations Research and a certificate in Engineering Management Systems

from Princeton University.


Agenda

Learning Objectives

Motivation

■ Patient discharge documents* used for clinical information

Approach

■ Expert review to assess how well they support this use

Recommendations

Future directions

*Referred to by many names : discharge instructions, patient instructions, clinical summaries, after visit summaries, summary of care documents


Learning Objectives

1. Describe a care coordination challenge & a common ‘work-around’

2. Share our method for evaluating usability of medical documentation

3. Identify best practices for improving patient discharge documents

Terminology: human factors, usability, heuristic evaluation, heuristics, after visit summary, discharge summary


Physicians are using patient-facing documents to

obtain information about acute care visits

Discharge summaries generated by inpatient providers’ EHR systems

often do not reach outpatient providers.

■ Poor integration/lack of interoperability

■ Inaccurate or missing contact information

■ No required timeframe for sending them & no confirmation of receipt when sent

No quick fix in sight

Patients often bring their discharge documents to follow-up visits

■ These often give more information than patients could recall on their own


Why Study Physician Use of Patient Discharge Documents?

Patient-facing documents used for care coordination more often than

inpatient providers realize*

■ Problems that prevent discharge summaries from being shared with outpatient providers

are not going to be resolved quickly

Other studies* assess them from the perspective of patients and caregivers

■ few have considered usability from the perspective of outpatient providers

Patient discharge documents must be improved to better support care

coordination AND to be more usable by patients and caregivers

*Gorry, T., Personal communication with ECRI Institute. 2017.

**Sarzynski, E., et al., 2016; Federman, A.D., et al., 2017; Unnewehr, M., et al., 2015; Newnham, H., et al., 2017.


ECRI Team’s Novel Approach

Develop ‘medical document usability heuristics’ to enable expert reviews of

documents produced by electronic health record (EHR) systems

Conduct collaborative expert reviews with human factors and clinical experts

Assess simulations of discharge documents produced by EHR systems


Four step process

1. Create simulated patient

discharge documents

2. Develop ‘medical

documentation heuristics’

3. Experts apply heuristics to

identify & rate usability issues

4. Generate recommendations

to improve usability


What did we assess?

2 Patient discharge

document examples From hospitals w/

different EHR vendors

2 pediatric test patients,Created by National Institutes of

Standards and Technology (NIST)

■ 2 year old with ear infection

■ 16 year old with injured knee

1. Simulated patient discharge

documents

4 simulated

documents


Two different approaches1. Simulated

patient discharge documents

Hospital partner 2 created an

anonymized version of a discharge

document from an actual patient

(replacing patient information with

fictitious data)

ECRI created simulations, replacing

the anonymized patient data with

NIST test patient data, and

replacing hospital and doctor

names with fictitious ones

Hospital partner 1 created an

electronic health record for each of

the NIST patient cases

Hospital partner 1 used its EHR

system to produce discharge

documents for these test patients

ECRI created simulations of those

documents (replacing hospital and

doctor names with fictitious ones)


Developing heuristics

Reviewed previously validated heuristics for:

■ software user interfaces

■ medical devices

■ online documentation usability heuristics

Eliminated those that don’t apply (most!)

Consulted literature on “good” writing

■ Generic writing guidelines

■ Medical documentation specific guidelines (extensive)

Extracted relevant recommendations

Removed duplicates & transformed recommendations into heuristics

2. Medical Documentation

Heuristics


Developing Heuristics, Continued

Organized heuristics into 5 general categories*

■ Readability

■ Minimalism

■ Comprehensibility

■ Content

■ Organization

Developed examples for each retained candidate

* Easier to perform evaluations & communicate results


Heuristics


Heuristic Category Definitions

Readability: Information is presented in a manner that is easy to read.

Content: All presented information is relevant to either a clinical expert or the

patient/caregiver and no needed information is missing.

Comprehensibility: It is easy for readers to make sense of presented information.

Minimalism: Information is presented as simply and succinctly as possible.

Organization: Information is ordered logically and grouped into reasonably-sized

sections with prominent and meaningful headings and subheadings.


Heuristics


Heuristic Evaluation

Four teams, each comprised of a clinician and a human factors engineer

■ Clinical experts all experienced as primary care providers

■ Human factors experts all currently working in healthcare

Human Factors experts introduced heuristics and explained evaluation

process to clinical experts

Clinical experts provided severity ratings for each potential issue:

■ 0= not an issue; 1=cosmetic, 2=minor, 3=major, 4=catastrophic

3. Apply heuristics to identify & rate usability issues


23

Content, Organization issues

Readability & Content issues


24

Organization issue

Minimalism & Readability issues


INSERT H2P1 screenshots here….


Content & Readability issues

Comprehensibility issues


Content & Readability issues


Usability issue counts and average severity ratings for each evaluated document, grouped by heuristic category

Hospital 1, Patient 1 (severity)




Total (severity)

Readability 15 (2.33) 15 (2.2) 9 (2.66) 8 (2.31) 47 (1.9)

Minimalist 4 (2.25) 3 (2.67) 8 (2.75) 8 (2.88) 23 (2.11)

Comprehensibility 6 (2.33) 5 (2.2) 12 (3.08) 8 (2.88) 31 (2.1)

Content 15 (2.47) 15 (2.63) 17 (2.94) 25 (2.96) 72 (2.2)

Organization 9 (1.89) 10 (1.7) 16 (2.73) 16 (2.78) 51 (1.82)

TOTAL 49 (2.25) 48 (2.28) 62 (2.83) 65 (2.76) 224 (2.57)

Severity scale: 1 = cosmetic, 2 = minor, 3 = major, 4 = catastrophic

RESULTS


Clear and consistently formatted section labels and sub-section headings, using white space to ensure the document is legible

Patient and provider information up front and clearly labelled


Clear, organized table explains how to arrange follow-up appointments

Medication information and instructions clearly labeled and listed

© 2018 ECRI INSTITUTE© 2018 ECRI INSTITUTE

Best Practice Recommendations

Given that EHR-generated patient discharge documentation may be the primary

information source for the outpatient provider, how can we make them more

usable and understandable for that provider?

4. Generated recommendations



Detailed Recommendations: Content

1. Include 6 items Joint Commission mandates for

hospital discharge summaries:

■ Reason for Hospitalization

■ Significant Findings

■ Procedure and Treatment provided

■ Patient’s discharge condition

■ Patient and family instructions

■ Attending physician’s signature

2. Ensure any information relevant to outpatient physician

and patient follow-up care can be included in document




Detailed Recommendations: Content

3. Ensure headers and footers contain meaningful information

4. Ensure proper use of medical, nonmedical, and billing terminology




Detailed Recommendations: Organization

5. Establish consensus across stakeholders on standardized order and format

for presenting information




Detailed Recommendations: Organization

6. If multiple diagnoses are present, make sure they are

clearly defined and differentiated, and that primary

diagnosis is explained up front.

7. Ensure that content matches headings and sub-

headings in each section




Detailed Recommendations:

Readability

8. Use consistent font size and type, indents, and

spacing throughout document

9. Ensure sufficient & consistent use of color and contrast

10. Emphasize important information in each section




Detailed Recommendations: Readability

11. Within each section, present discrete ideas and information as

separate bullets or paragraphs

12. Balance text and white space




Detailed Recommendations: Comprehensibility

& Minimalism

13. Present information clearly and concisely

14. Use consistent terminology

15. Remove irrelevant information




Detailed Recommendations: Comprehensibility

16. Explain any abbreviations




It’s Not Just patient discharge documents:

Improving care coordination in the long term

Establish policies on timeliness of distributing discharge documents.

Configure EHR systems so they provide confirmations or failure notices

about delivery of information to outpatient providers.

Pressure vendors to make systems inter-operable

Adopt Joint Commission mandate on discharge summary components


Summary

Spread the word: patient-facing discharge

documents are being used for clinical

communication

ECRI offers guidance to improve discharge

documents so they support clinical communication

AND are easier for patients to use


Summary: What can Hospitals do?

Learn how information from EHR gets pulled into generated documents

■ Ensure headings and subheadings match the content

■ Emphasize important information in each section

Work with vendors to modify EHR modules used to generate documents

■ Add new section(s) specifically directed towards clinical readers


Summary: What can EHR vendors do?

Use our heuristics to guide the redesign of generic templates used to generate

clinical documentation

■ Establish standardized, logical order and format to present information

Important information upfront

Get feedback from document recipients

Follow Joint Commission’s Standard IM.6.10

Give acute care provider organizations more flexibility

■ Enable users to add supplementary notes or additional content into automatically

generated sections


Conclusions

Long term: better EHR

interoperability will help improve

coordination of care

Short term, improve discharge

documents

■ Make them more usable for both

providers and patients


Next Steps: Future Directions

Expand scope

■ Consider care coordination needs of different types of outpatient providers

■ Assess usability from patient perspective

Apply heuristics to assess discharge summaries & other medical documents

Collect stakeholder inputs & develop user requirements

■ Work with EHR vendors and inpatient and outpatient providers to redesign discharge

document templates

https://www.google.com/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwj-hpv3n6HbAhXSx1kKHRdJBbMQjRx6BAgBEAU&url=https://www.youtube.com/intl/en-GB/yt/about/policies/&psig=AOvVaw2AbHJ6PyRb4SfkLjPVLteo&ust=1527349643631762


Questions

Polly Tremoulet

Human Factors Scientists, Health Devices Group

ECRI Institute

[email protected]

Erin Sparnon

Engineering Manager, Health Devices Group

ECRI Institute

[email protected]

mailto:[email protected]

mailto:[email protected]


References

ECRI Institute. Postacute care, aging services, primary care: hospitals look beyond their walls. Contin Care Risk

Manage [online]. 2016 Nov 18 [cited 2017 Sep 14].

Australian Commission on Safety and Quality in Health Care (2011), Electronic Discharge Summary Systems Self-

Evaluation Toolkit, ACSQHC, Sydney. https://www.safetyandquality.gov.au/wp-content/uploads/2012/01/EDS-self-

eval-toolkit-sept2011.pdf

Health Information and Quality Authority (2013), National Standard for Patient Discharge Summary Information, HIQA,

Ireland. https://www.hiqa.ie/system/files/National-Standard-Patient-Discharge-Summary.pdf

Maher B., Drachsler H., Kalz M., et al. Use of mobile applications for hospital discharge letters – improving handover

at point of practice. https://pdfs.semanticscholar.org/6d6c/4af54b15e26f167f7be6db106094f54c4bd5.pdf

National Academy of Sciences. Overview of issues involved in creating better discharge instructions. In: Facilitating

patient understanding of discharge instructions: workshop summary. Washington (DC): National Academies Press;

2014 Dec 1.

Sarzynski, E., et al., Opportunities to improve clinical summaries for patients at hospital discharge. BMJ Qual Saf,

2016: p. bmjqs-2015-005201.

Federman, A.D., et al., Patient and clinician perspectives on the outpatient after-visit summary: a qualitative study to

inform improvements in visit summary design. Journal of the American Medical Informatics Association, 2017. 24(e1):

p. e61-e68.

https://www.safetyandquality.gov.au/wp-content/uploads/2012/01/EDS-self-eval-toolkit-sept2011.pdf

https://www.hiqa.ie/system/files/National-Standard-Patient-Discharge-Summary.pdf

https://pdfs.semanticscholar.org/6d6c/4af54b15e26f167f7be6db106094f54c4bd5.pdf


References, continued

Unnewehr, M., et al., Optimizing the quality of hospital discharge summaries–a systematic review and practical

tools. Postgraduate medicine, 2015. 127(6): p. 630-639.

Newnham, H., et al., Discharge communication practices and healthcare provider and patient preferences,

satisfaction and comprehension: A systematic review. Int J Qual Health Care, 2017: p. 1-17.

Solan LG, Sherman SN, DeBlasio D, Simmons JM. Communication challenges: a qualitative look at the relationship

between pediatric hospitalists and primary care providers. Academic pediatrics. 2016 Jul 31;16(5):453-9.

Coghlin DT, Leyenaar JK, Shen M, et al.. Pediatric discharge content: a multisite assessment of physician

preferences and experiences. Hospital pediatrics. 2014 Jan;4(1):9.

Shen MW, Hershey D, Bergert L, et al. Pediatric hospitalists collaborate to improve timeliness of discharge

communication. Hospital Pediatrics. 2013 Jul 1;3(3):258-65.

Ruth JL, Geskey JM, Shaffer ML, Bramley HP, Paul IM. Evaluating communication between pediatric primary care

physicians and hospitalists. Clinical pediatrics. 2011 Oct;50(10):923-8.

Nguyen OK, Kruger J, Greysen SR, Lyndon A, Goldman LE. Understanding how to improve collaboration between

hospitals and primary care in postdischarge care transitions: A qualitative study of primary care leaders'

perspectives. Journal of hospital medicine. 2014 Nov 1;9(11):700-6.


Next PSO Webinar

September 20th

Keeping Staff Safe in Acute Care and in the

Community

Partnering with Virginia PSO and their providers

■ Carilion Clinic Home Health

■ UVa Medical Center