ASSESSING THE READABILITY OF ELECTRONIC HEALTH RECORDSLan VoBa

SI 561 Natural Language Processing

Overview of EHRs• What is an EHR?• How are they being used?• What are the benefits?• What are the challenges?

Motivation• How “patient-friendly” are doctors’ notes, letters, etc.?• How difficult to read and understand those documents?• Related work

• “Applying Multiple Methods to Assess the Readability of a Large Corpus of Medical Documents” (Wu et al)

• Readability formulas used to assess other health-related texts• Automated Readability Index• New Dale-Chall Formula

The Data• UMHS

• 120 clinic locations and offices• 45,000 inpatient hospital stays• 1.8 million outpatient visits and surgeries

• The records• Freetext• Preparation• 4,133 Letters• 18,217 non-letters

• Challenges• Limited access• Poor metadata

Automated Readability Index

Age Group Grade Level

5 – 6 years old Kindergarten

6 – 7 years old First Grade

7 – 8 years old Second Grade

8 – 9 years old Third Grade

9 – 10 years old Fourth Grade

10 – 11 years old Fifth Grade

11 – 12 years old Sixth Grade

Age Group Grade Level

12 – 13 years old Seventh Grade

13 – 14 years old Eighth Grade

14 – 15 years old Ninth Grade

15 – 16 years old Tenth Grade

16 – 17 years old Eleventh Grade

17 – 18 years old Twelfth Grade

18 – 22 years old College

New Dale-Chall Formula

Score Meaning

4.9 and Below Grade 4 and Below

5.0 to 5.9 Grades 5 – 6

6.0 to 6.9 Grades 7 – 8

7.0 to 7.9 Grades 9 – 10

8.0 to 8.9 Grades 11 – 12

9.0 to 9.9 Grades 13 – 15 (College)

10 and Above Grades 16 and Above (College Graduate)

Method

Metadata

LettersAutomated Readability

Index

New Dale-Chall

Formula

Non-Letters

Scores

Results

Future Work• Wu et al• More documents• Remove “headers”• Compare scores to human evaluators to confirm

assessment

Thank you!Questions? Comments?