classification of emergency department ct imaging reports using natural language processing and...
TRANSCRIPT
Classification of Emergency Department CT Imaging Reports
using Natural Language Processing and Machine Learning
Efsun Sarioglu, Kabir Yadav, Meaghan Smith, Hyeong-Ah Choi
This project supported by the NIH National Center for Research Resources(UL1RR031988 and KL2RR031987)
Background, Objective & Methods
Use of electronic medical record data for clinical research and quality improvement requires free-text data interpretation for outcomes of interest. Natural language processing has shown
promise for this purpose
To demonstrate real-world performance of a hybrid NLP-machine learning system for automated classification of radiology reports
Approach Overview Multicenter review of consecutive CT
reports obtained for facial trauma using a trained reference standard Medical Language Extraction and Encoding (MedLEE) WEKA 3.7.5 Salford Systems CART 6.6
Results
Total reports: 3710
Positive cases: 460 (12.4%)
Manual coding had high reliability Kappa=0.97 [95% CI 0.94-0.99]
CART Decision Trees (50:50)Raw Text (8-node) NLP (9-node)
Classification Performance
Raw Text
NLP
Precision
0.949 0.968
Recall 0.932 0.964
F-score 0.940 0.966
Unexpectedly high performance of machine learning without NLP
Comparable to inter-rater performance and prior studies of inter-physician agreement
Comparable to prior real-world and simulation studies
Concluding Remarks How’s it novel?
One of only a handful of real-world NLP studies using validated reference standard
Translating existing NLP and machine learning technologies to support CER
Next step: validation Test approach using other clinical cases Evaluate different features or
classification algorithms