Use of natural language processing and machine learning to derive clinical

benefit from electronic patient data. A PhD research project

Principal Investigator: Dr. Robert ChallenDepartment of Mathematics & Living Systems Institute, University of Exeter

The challenge:

To deliver the best care possible the NHS must be able to learn from itself.

Acute NHS trusts in slow transition to electronic hospital records

● Primary care nearly 100% digital and has been for last 10-15 years

Acute NHS trusts deliver the majority of care using unstructured text records, either on paper or as electronic documents.

Clinical benefits of digitisation not yet realised.

Electronic hospital records: The problem

Limited benefits seen by end users

● Lack of clinician engagement

Cultural resistance due to poor user experience

● Change in ways of capturing information● Change in workflows● Hard to represent complexity of medicine in structured data

Mixed paper and electronic records are unsatisfactory

● Transition painful● Fragmentation of record & poor data quality

Limited engagement leads to limited data quality

DIGITAL PAPER

Electronic patient records: The opportunity

Electronic clinical documents are a rich, and largely untapped data seam, stretching back over the last 20 years or more.

● Outpatient clinic letters, radiology reports, discharge summaries, primary care referrals

Natural language processing tools for analysing unstructured data are maturing

Machine learning techniques for making predictions on incomplete data improving

Mining clinical value from existing documents may deliver benefit quicker than changing the clinical workflow to collect better data for analysis.

Research objectives:

Prove natural language processing (NLP) and machine learning (ML) can make clinically useful predictions using the existing electronic

documents in Taunton.

Demonstrate a clinical benefit from existing digitisation to further engage healthcare professionals on their journey to a full digital record.

Challenge and address the ethical, legal, technical or cultural issues that might influence our ability to apply NLP and machine learning to

patient data.

ref Vorhies, W. (n.d.). Machine Learning – Can We Please Just Agree What This Means - Data Science Central. Retrieved January 12, 2018, from https://www.datasciencecentral.com/profiles/blogs/machine-learning-can-we-please-just-agree-what-this-means

Machine learning applications

Non medical example Potential medical application

Classification ● Spam filtering● Credit risk profiling● Photo assistants

● Detection of radiology test abnormality

● Clinical risk scoring

Similarity ● Recommendation engines ● Protocols / guideline development

● Patients like me

Outlier detection ● Fraudulent claim identification

● Adverse drug reactions● Misdiagnosis identification

Clustering ● Market segmentation ● Clinical subgroup detection● Outcome based / precision

medicine

Deep reinforcement learning

● Learning game playing strategies

● Power systems control

● Optimising ventilator strategy● Optimising infusions

Clinical scope - examples

Can we predict from the medical record which patients will be failed discharges so that we can improve our monitoring of those patients, or increase their step down care?

Can we predict from the medical record which tests need to be done on a patient before they are admitted for a procedure or an interventional investigation?

Could we identify children who might be at risk of non accidental injury from their (and their relatives) medical records?

What is most useful and practical thing to focus on in Taunton?

Recent advances in machine learning in medicine

Medical testing

Sensitivity

the probability, given a patient with disease, that the test will be positive.

(the true positive rate of the test)

Specificity

the probability, given a patient without disease, that the test will be negative

(1 - the false positive rate of the test)

sensitivity

specificity

AUC

Receiver operating characteristic

for a continuous test variable

ref Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542.

Esteva et al.: Classification of skin cancer

Trained a deep convolutional neural net on 129,450 clinical images which had been biopsied and had known histopathology.

Model’s binary classification of benign or malignant “outperformed” dermatologists, but training set had bias, and conditions not exactly the same

ref Walsh, C. G., Ribeiro, J. D., & Franklin, J. C. (2017). Predicting Risk of Suicide Attempts over Time through Machine Learning. Clinical Psychological Science, 1–13. https://doi.org/10.1177/2167702617691560

Walsh et al.: Predicting suicide risk in patients who self harm

Patients who self harm are at high risk of suicide, but unpredictably so.

Trained random forest model from extracts of patient record for A&E attendances for self harm.

Model prediction has sensitivity 94%, specificity 71% for suicide within the next week (better than clinicians):

ref Price SJ, Stapley SA, Shephard E, Barraclough K, Hamilton WT. Is omission of free text records a possible source of data loss and bias in Clinical Practice Research Datalink studies? A case-control study. BMJ Open [Internet]. 2016 May 13 [cited 2017 Apr 21];6(5):e011664. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27178981

Technical constraints

Data quality:

● Even looking at scan images or photos you are not seeing the whole clinical picture

● In UK primary care ~40% of relevant cases identified only by free text● Conclusions based on structured data alone => are inherently biased by

what is captured

Machine learning approach:

● Selection of cases and controls must reflect real life, otherwise:○ performance measurement is biased○ model will not be accurate for real data

Medical text

Medical text - investigation request clinical details

Admitted with N+V shortness of breath and signs of sepsis ?chest

AML ON CHEMO FLAG IDA SPIKED TEMP 38.2 . TO NEUTROPENIC FOR SEPSIS CLEARANCE STAT DOSE TAZ GIVEN POST CULUTED ON CVC LINE

chest/epigastric pain sob ?perforation

Collapse ? cause

Fell 2m from ladder, hypoxic ?pneumothorax. Chest clear on clinical exam

Isolated dysphasia starting abruptly ~3 weeks ago

?recent Stroke

Motocross accident ~7 hours ago. Flipped off bike and bike hit him. Right ankle effusion and tenderness laterally ?#

New NHL. Pre chemotherapy bloods. Having R-CHOP 09/10/17.

Patient fell this evening and hit his head. Complaining of tenderness around occipital part of head and cervical spine. On apixaban.

Patient with right iliac fossa pain and tenderness on examination. USS requested for ?appendicitis ?other abdominal pathology leading to her pain

Medical text - admission clerking (part)

CHIEF COMPLAINT: "I have had trouble breathing for the past 3 days"HISTORY: 69-year-old Caucasian male complaining of difficulty breathing for 3 days. He also states that he has been coughing accompanying with low-grade type fever. He also admits to having intermittent headaches and bilateral chest pain that does not radiate to upper extremities and jaws but worse with coughing. Patient initially had this type of episodes about 10 months ago but has intermittently getting worse since.PMH: DM, HTN, COPD, CADPSH: CABG, appendectomy, tonsillectomyFH: Non-contributorySOCH: Divorce and live alone, retired postal worker, has 3 children, 7 grandchildren. He smokes 1 pack a day of Newport for 30 years and is a social drinker. He denies any illicit drug use.TRAVEL HISTORY: Denies any recent travel overseasALLERGIES: Denies any drug allergiesHOME MEDICATIONS: Advair 1 puff bid Lisinopril 10 mg qd Lopressor 50 mg bid Aspirin 81 mg qd Plavix 75 mg qd Multivitamins Feso4 1 tab qd Colace 100 mg qdREVIEW OF SYSTEMS REVEALS: Same as above

NLP - investigation request clinical details

Fell 2m from ladder, hypoxic ?pneumothorax. Chest clear on clinical exam

AML ON CHEMO FLAG IDA SPIKED TEMP 38.2 . TO NEUTROPENIC FOR SEPSIS CLEARANCE STAT DOSE TAZ GIVEN POST CULUTED ON CVC LINE

Clinical terminology: SNOMED CT

SNOMED CT: Logical modelling and post-coordination

ref

Word embeddings

Train a neural network to predict probability of surrounding words in a large corpus of text.

The weights in the trained neural network represent the meaning of the word as a vector.

“Word2Vec Tutorial - The Skip-Gram Model · Chris McCormick.” [Online]. Available: http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/. [Accessed: 15-Jan-2018].

ref T. Mikolov et al., “Distributed Representations of Words and Phrases and their Compositionality,” in Advances in Neural Information Processing Systems 26, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2013, pp. 3111–3119.

Pros and cons of alternate approaches

Concept identification

● More mature in medical domain● Curated knowledge model● Complex to analyse● Little prior art on combining description logic with machine learning● Logical inferencing possible (e.g. diabetic retinopathy)

Word vectorisation

● No medical domain word embeddings● Does not perform well with polysemy / complex phrases● Natural fit for ML processes● Active focus of research for uses such as machine translation

For any NLP method you need to assume the output is substantially

inaccurate.

What can you use it for?

In summary….

Ultimate aim is to develop prediction models that work on the clinical text

Initial research on de-identification of text needed

Close collaboration with my NHS industrial sponsor is key