1

The Role of Big Data Analytics in

Predicting Patients' Outcome

Prof Marcus Ong Eng Hock

Senior Consultant and Clinician Scientist

Dept of Emergency Medicine, Singapore General Hospital

Director, Health Services and Systems Research (HSSR), Duke-NUS Medical School

Vice Chair (Research), Emergency Medicine Academic Clinical Program

Director, Health Services Research Center, Health Services Research Institute

Director, Unit for Prehospital Emergency Care (UPEC), Senior Consultant, Ministry of Health, Hospital Services Division

2

Data is the New Oil of Healthcare and Biomedicine

Harnessing and Using the Data

Disease and Biological Insights Improve Hospital

Efficiencies and Processes

Improve Patient Outcomes and

Experiences

New Tools for Healthcare

2

Lower Healthcare

Costs

Data Liquidity!

Data Rich with INformation and Knowledge (DRINK!)

4

MEASURED REALITY

Intermountain’s

4-steps to a LHS

Data is Key to a Learning Healthcare System

Experience of CarePopulation Health

Cost Per Capita Work life of Health

Care Providers

The Quadruple Aim

66

Undergirding EnablersCase Management | Operations | HR | Finance | IT | Education and Research Infrastructure

Regional Health System (RHS)

Changi

General

Hospital

Our Patients

Community / Primary Acute / Secondary Tertiary / Quaternary Intermediate / Long Term

Chinatown

FMC, Tiong

Bahru

CHC

NHGP

GP

SHPPrimary Care

Integrated

Community

Primary Care

Lung

Diabetes and

Metabolism

Breast

Future

SDDCs

Blood

Cancer

Head &

Neck

Liver

Transplant

Sengkang

General

Hospital

SGH

Campus

MSK

Digestive

Disease

Transplant

Dental Heart

Eye

Neuro-

scienceCampus DEM

Campus Inpatient

Campus ASC

KKH

Campus

Bright

Vision

Hospital

Outram Community HospitalPost-Acute & Continuing Care

SengKang

Community

Hospital

Nursing Homes Palliative

Care

HomecareCommunity

Partners

Seamless Transfer of Patients Across Care

Continuum

* Collaboration with Pearl’s Hill Care Home

77

Data Science (not just) Data Analytics

• Data Analytics can produce operational insights, but

• Data Analytics Data Science!• Data Science requires rigorous

scientific thought processes– “Data-driven Science” – “Evidence-based”– “Statistical Science”– “Statistics” – Jeff Wu of

Michigan U– Etc…

Max {f:f (IS, SM, AM)>1} [(Health Services Research X Data Science)f]

Scientific Thinking (ST)

Analytical Methods

(AM)

Implementation Science (IS)

Translational Impact =

Data Assets + Scientific Thinking

8

Roles of HSRI and HSRC

Long term goal- Learning, Integrated Health System

9

* Modified from Best Care at Lower Cost, Institute of Medicine 2012

TECHNOLOGY

TALENT

GOVERNANCE

TALENTData Literarcy, Manpower Training

TECHNOLOGYAI/ Data Science Supporting Infrastructure

GOVERNANCEData QualityAI/ Data GovernancePolicies and Processes

USER EMPOWERMENT

1010

Data Governance Framework

Master Data Metadata Data Quality Data Security

Create and Maintain

Definitions

Review and Approve Changes/ Additions

RemediateResolve Conflicts

Train and Communicate

Technology Enablers (e.g. workflow engines, corporate portals and etc.)

Key ProcessesKey Processes

Data Governance Pillars

Technology enablers can assist the management of the governance but these may only be feasible when the rest of the components are already established

Sound policies and standards must be in place for the governance framework to be effective

are required for the maintenance/ enhancement of the standards and policies of the Data Governance Pillars

are required for the maintenance/ enhancement of the standards and policies of the Data Governance Pillars

DATA GOVERNANCE ENABLES DATA SHARING AND RESEARCH

Data Sharing

•Data Quality

•Data Governance

•Data Security

•Open-Access

•Inter-institutional

•Common Platform

•DEDUCES

•TriNetX

•Deidentification

•Data Warehouse

•Database/ Data Science Infrastructure (RSD)

Data Sharing

DataSharingGAPGAP

GAP

HSRC Data Science works closely with SingHealth RICE and OIA

GAP

Access Restricted Cluster (External Collaborations)

Cluster Deidentification Process and Training

Within GOVERNANCE:Training – Systems – Policies and Processes

A comprehensive tender should be called to provide a rigorous evaluation

of privacy-preserving software and technologies

DATA GOVERNANCE INFRASTRUCTURE

Enhanced Privacy Preserving System (For IT Evaluation)

• Privacy Preserving System technology is available and rapidly evolving

• UK National Health Service called for a tender to develop a Data Services Platform (DSP) which includes various components, including De-ID services

• Key objectives:– Enhance safety and security

– Improve timeliness and utility

– Remove duplication and drive efficiency

– Etc …

• Major contenders: IBM, Privitar, A*STAR etc

N-CRiPT: https://ncript.comp.nus.edu.sg/

TALENT DEVELOPMENT

Data Producers

Data Users

Data Analysts / Scientists

REDCap

Data Visualisation; Design Thinking, R/ Python for Data Preprocessing, Predictive Modellin, Optimization and Simulation

Practical Citizen Data Science Program for Health Services

Beginners / Advanced

ResearchersData Analytics ProfessionalsSenior Residents/Clinicians

User Group Course Target AudienceConducted by

HSRC

OIA/HSRC

HSRC/OIA

Data QualityManagement TBD After FY19

Intro Health Services Research

HSRC

HBRA Deidentification HSRCInstitutional Deidentification 3rd Party

Everyone

Everyone

Clinical Data Warehouse (DW) at SingHealth

Research Standing Database

eHINTS - SingHealth Data Warehouse• Sample Data Sources ingested:

• LAB, OAS, OPEC; SAPISH; MAXCARE; SCM-ED; OTMS; RIS; REDCap

• Structured DW that facilitates logical data consumption from disparate data sources

HBRA/ PDPA Compliant

Multi-layer Privacy and Security Policies

Shared Roles and Responsibilities

RSD Value Proposition and Design Parameters

1. Purpose-built for Research and Innovation: Structured and Cleaned Data

Pipelines (to ensure provenance)

2. HBRA Compliant Deidentified Database. Transparent Data Governance

3. Robust IT security

4. Linkage to other data sets approved for research (OMICS, Images, Hospice,

external data sources)

5. Decoupled from clinical/operations needs No impact on

clinical/operational effectiveness in using the data-warehouse

6. Rapid Response for Research and Innovation with no bottleneck

7. User-centric design. Adaptable to different analysis needs and

requirements (from basic statistics to advanced data science and AI)

8. Facilitate research collaborations across disciplines and institutions

DATA SHARING INFRASTRUCTURE for RESEARCH

• SingHealth joined the TriNetX Consortium in 2017

• HSRC currently hosts the internal TriNetX hardware servers and software

• Pending feasibility studies and IT Security clearance for full pilot

• Only aggregate results can be obtained by Pharma / CRO

• Collaborative research is also possible between hospitals

• Hospitals’ data always remains within the internal system

TriNetX - 46.2M patients with 10.4B clinical facts in TriNetX network

DEDUCES – Collaboration between SingHealth – Duke NUS – Duke Health

Proposal for a SingHealth-A*STAR Partnership on “High-Profile Use

Cases and Registries”

Institution Pilot Use Case

SGH a) Peri-operative databaseb) Surgical databasec) ED Database

NHCS a) Nuclear databaseb) Echo database

NCCS a) Pathology databaseb) Radiology database

PRISM PRISM Cohort

Multiple Institutions

a) Diabetes Registry b) Respiratory Medicinec) Population Health

Name Instn Title

1 Ecosse Lamoureux SERI/SNEC The Clinical And Economic Effectiveness Of Extending

Diabetic Eye Screening In Singapore.

2 Aung Tin SERI/SNEC Singapore angle closure glaucoma program:

characterization, prevention and management

3 Pierce Chow Kah

Hoe

NCCS A patient-specific diagnostic and predictive platform for

precision medicine in HCC

4 John Chia Whay

Kuang

NCCS Aspirin for dukes c and high risk dukes b colorectal

cancers - an international, multi-centre, double blind,

randomised placebo controlled phase III trial (ASCOLT)

5 Kenneth Kwek /

Sng Ban Leong

KKH Improving the Outcomes of Women and Children Health In

Singapore - An Intergative Translational Research

Programme (II) [CG Main Pot]

6 Kenneth Kwek /

Sng Ban Leong

KKH Integrated Platform for Research in Advancing Metabolic

Health Outcomes in Women and Children (I-PRAMHO)

(Collaborative Pot)

7 Wong Tien Yin SERI/SNEC DYNAMO: Diabetes studY in Nephropathy And other

Microvascular cOmplications

8 Poon Choy Yoke NDCS Transforming the Future of Oral Healthcare - A Clinical,

Translational and Health Services Research Approach

9 Leopold

Schmetterer

SERI/SNEC Singapore Imaging Eye Network

(SIENA)

20

SELENA for DR Screening

Total retinal images : 493,667• 76,370 and 112,648 for DR • 125,189 and 71,896 for Glaucoma• 71,616 and 35,948 for AMDfor training and validation

Precision Diagnostics by Multi-Modal Data Collection

Scanadu

Scout

Autographer

LifeLogger

Fitbit

Charge

iHealth

Pulse Ox

Athos

Smart Shorts

Withings

Smart Scale

DexCon

Glucose Sensor

Genetic

Sequence

Body

Imaging

Metabolite

Profiling

Clinical

Records

Slide 22

Integrating Multi-Modal Data with EHRs

• Collaboration with Health Services Research Centre (HSRI)

Integrate PRISM data with

longitudinal electronic

medical records (EMR)

Supported by ARTS (Analytics and Research Technologies for SingHealth) program

Development and Implementation of Nationwide Predictive

Model for Admission Prevention

HOSPITAL CARE COMMUNITY CARE

Patient admitted to

hospital

Risk prediction generated

H2H Care team assesses these

high risk patients

Provide suitable intervention to

patients

Patient discharged timely and remains

in community

Any patient with an unscheduled admission into

the hospital

Utilizes routine data found in the NEHR to build a nationwide FA

prediction tool.The tool flags out the risk of having a subsequent or

multiple readmissions

Clinical team enrolls into 3 tiers for

timely/safe transition care from Hospital to

Home (H2H)

Arranges for post-discharge medical,

nursing & social community supports

H2H Community teams review, track, monitor and report outcomes. This will

enhance the predictive model

The predictive model was Launched in all Public Hospitals in Singapore since April 2017To date more than 12,000 patients have benefitted

24

Singapore Cardiac Longitudinal OUtcomes Database

(SingCLOUD)

Reporting and Analytics Services

Manage Calls and Dispatch Manage EMS Transition and Return

Emergency Call Dispatch Monitoring ConveyanceLocate/Treat/Deliver

Handover to ED Return to Service

Operations Centre Mobile DevicesSmart Ambulance Device Participating EDs (SGH, KTPH and TTSH)

995 call/ situational

data

Electronic Case Record

Incident Details on

IBCR

Clinical Data

Capture

My Responder

Heartsave forms

IBCR

Dispatch Tape Review

HDG (MOHH)

MHA Firewall

Data Warehouse

Extract, Transform, Load

Hospital Firewall

PEC IT Blueprint and Analytics

Potential Area of Analytics

• Research on OHCA, Trauma, Stroke and STEMI

Comms between Paramedics and

ED

ePCRNEHR

Dial 995 and send your geo-location at the same time

Know where the nearest AED is located

Sign up as a volunteer responder

1

2

3

myResponder app

A novel model for predicting inpatient mortality after emergency admission to hospital in Singapore

XIE Feng

Health Services & Systems Research (HSSR)

Duke-NUS Medical School

2019.4.12

Predictive variables

Demographics ED administrative data Clinical data

• Age• Gender• Nationality• Race

• Triage class (PACS score)• Consultation waiting time • ED boarding time • Day of week • Shift time

• Blood gas • Pulse • Respiration rate • FiO2 • SPO2 • Diastolic BP • Systolic BP • Bicarbonate • Creatinine • Potassium • Sodium

Novel Triage Model For Predicting Inpatient Mortality

CART (Baseline model) Our model

AUC 0.71 0.83

Sensitivity 0.730 0.770

Specificity 0.561 0.733

Score threshold 9 0.035

M I N D T O M A R K E T

AiTriage™ enabledTriage Pathway for Risk Stratification

of Low Risk MACE Patients in the

Emergency Department

Confidential

34

PRODUCT: AI ALGORITHM

InputECG, BP, SPO2

5 min

OuputRisk score

> CO score: High Risk< CO score: Low Risk

CO: cut off

aiTriage™ ML Algorithm trained with 39 variables

ML Algorithm based on 39 variables from HRV parameters, ECG 12-Leads and Vital signsAll parameters are objective and measured or derived from acquired signals.

Patents Granted

1. Method of predicting acute cardiopulmonary events and survivability of a patient | US 8668644

2. System and Method of Determining A Risk Score For Triage | US 13/791,764

35

To Improve Patient Care and Clinical Outcomes for Asthma and

COPD Patients through Data Linkages and Analytics

© 2019 AI Singapore. Confidential & Proprietary.

JARVISDHL: Transforming Chronic Care for

Diabetes, Hypertension and hyperLipidemia with AI

NUS – Wynne Hsu, See-Kiong Ng, Mong Li Lee, Chee Yong Chan

SingHealth – Marcus Ong, Ngiap Chuan Tan, Tien-Yin Wong, Ming-Ming Teh, Khung Keong Yeo

JARVISDHL (“Just” A Rather Very Intelligent System)

Physical activity

DietJ.A.R.V.I.S.

(personalisedintelligent

management)

Blood glucose

Clinical history

Current state

Previous experience

Patient updates

Adherence to Rx

Drug responses

Current diseases

Family history

Biomarkers

Genetics

Adherence to appts

Admission/ED visit

App inputs

Chatbot

Anthropometrics

Appointment reminders

Medication reminders

Physical activity reminders

Dietary recommendation

Voice recognition

Face recognition

Insulin dose recommendation

Treatment suggestions

Glucose prediction

Complications detection

Complications prediction

Health services planning

AI System

AI/ Data Science models need to go beyond validation to IMPLEMENTATION

1. Moons et al. Heart. 2012;98(9):691-6982. Moons et al. Heart. 2012;98(9):683-6903. Amarasingham et al. Health affairs 2014;33(7):1148-54

Scale up

• Implement

• Sustain

Test-bedding

• Explore context

• Adoption

Impact assessment

• Quantify impact on behaviour and decision making, health outcomes & cost-effectiveness

• Comparative designs

Model updating

• To adjust/improve performance for other settings or populations

• Need to undergo further external validation

External validation

• Temporal

• Geographical

• Domain (different population)

Assessing incrementa

l value of new

(bio)marker

• C-statistic

• Net reclassification improvement

Internal validation

• From same sample

• Random split

•Bootstrapping

Development

• Data source

• Quality

• Missing data

• Variable selection

Research Implementation

Our Goal is Implementation!