an open science approach to medical evidence …...–safety surveillance (e.g., identifying new...
TRANSCRIPT
![Page 1: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/1.jpg)
An Open Science Approach to Medical Evidence
Generation: Introducing Observational Health Data Sciences and Informatics
Jon Duke, MD MS Regenstrief Institute
Academy Heath June 14 2015
Slide Credits: Patrick Ryan
![Page 2: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/2.jpg)
What is OHDSI?
• The Observational Health Data Sciences and Informatics (OHDSI) program is a multi-stakeholder, interdisciplinary collaborative
• The goal of OHDSI is to bring out the value of observational health data through large-scale analytics and evidence generation
• All our software and other products are released as open-source
2 http://ohdsi.org
![Page 3: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/3.jpg)
OHDSI: a global community
OHDSI Collaborators: • >140 researchers in academia,
industry and government • >10 countries OHDSI Data Network: • >50 databases standardized to
OMOP common data model • >680 million patients
![Page 4: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/4.jpg)
OHDSI Evidence Generation
• Clinical characterization: – Descriptive statistics (e.g., natural history of a disease or
patterns of medication use)
– Quality improvement (e.g., performance measures)
• Population-level estimation – Safety surveillance (e.g., identifying new adverse event
risks for drugs)
– Comparative effectiveness (e.g. comparing interventional to non-interventional treatment of chronic back pain
• Patient-level prediction – Incorporating patient medical history to provide
personalized recommendations for therapy selection, adverse event risk, high value diagnostic studies
![Page 5: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/5.jpg)
The odyssey to evidence generation
Patient-level data in source
system/ schema
evidence
![Page 6: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/6.jpg)
Open Science through Standardization
• The OHDSI community has standardized core components of the research process in order to
– Promote transparent, reproducible science
– Reveal data quality issues
– ‘Calibrate’ datasets
– Bring skillsets together from across the community (clinical, epi, stats, compSci)
![Page 7: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/7.jpg)
Opportunities for standardization in the evidence generation process
• Data structure : tables, fields, data types • Data content : vocabulary to codify clinical domains • Data semantics : conventions about meaning • Cohort definition : algorithms for identifying the set of
patients who meet a collection of criteria • Covariate construction : logic to define variables
available for use in statistical analysis • Analysis : collection of decisions and procedures
required to produce aggregate summary statistics from patient-level data
• Results reporting : series of aggregate summary statistics presented in tabular and graphical form
Pro
toco
l
![Page 8: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/8.jpg)
Concept
Concept_relationship
Concept_ancestor
Vocabulary
Source_to_concept_map
Relationship
Concept_synonym
Drug_strength
Cohort_definition
Stand
ardize
d vo
cabu
laries
Attribute_definition
Domain
Concept_class
Cohort
Dose_era
Condition_era
Drug_era
Cohort_attribute
Stand
ardize
d
de
rived
ele
me
nts
Stan
dar
diz
ed
clin
ical
dat
a
Drug_exposure
Condition_occurrence
Procedure_occurrence
Visit_occurrence
Measurement
Procedure_cost
Drug_cost
Observation_period
Payer_plan_period
Provider
Care_site Location
Death
Visit_cost
Device_exposure
Device_cost
Observation
Note
Standardized health system data
Fact_relationship
Specimen CDM_source
Standardized meta-data
Stand
ardize
d h
ealth
e
con
om
ics
Drug safety surveillance
Device safety surveillance
Vaccine safety surveillance
Comparative effectiveness
Health economics
Quality of care Clinical research One model, multiple use cases
Person
![Page 9: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/9.jpg)
Preparing your data for analysis
Patient-level data in source
system/ schema
Patient-level data in
OMOP CDM
ETL design
ETL implement
ETL test
WhiteRabbit: profile your source data
RabbitInAHat: map your source
structure to CDM tables and
fields
ATHENA: standardized vocabularies for all CDM
domains
ACHILLES: profile your CDM data;
review data quality
assessment; explore
population-level summaries
OH
DSI
to
ols
bu
ilt t
o h
elp
CDM: DDL, index,
constraints for Oracle, SQL
Server, PostgresQL;
Vocabulary tables with loading
scripts
http://github.com/OHDSI
OHDSI Forums: Public discussions for OMOP CDM Implementers/developers
Usagi: map your
source codes to CDM
vocabulary
![Page 10: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/10.jpg)
Standardized large-scale analytics tools under development within OHDSI
Patient-level data in
OMOP CDM
http://github.com/OHDSI
ACHILLES: Database profiling
CIRCE: Cohort
definition
HERACLES: Cohort
characterization
OHDSI Methods Library: CYCLOPS
CohortMethod SelfControlledCaseSeries
SelfControlledCohort TemporalPatternDiscovery
Empirical Calibration HERMES:
Vocabulary exploration
LAERTES: Drug-AE
evidence base
HOMER: Population-level
causality assessment
PLATO: Patient-level
predictive modeling
CALYPSO: Feasibility
assessment
![Page 11: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/11.jpg)
OHDSI Software
• Community developed
• Apache 2.0 licensed
• Available on GitHub
• Common frameworks
– Java
– HTML5 / Javascript
– R
– Oracle / SQL Server / Postgres / Redshift / Netezza
![Page 12: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/12.jpg)
Motivating example to see the OHDSI tools in action
![Page 13: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/13.jpg)
![Page 15: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/15.jpg)
![Page 16: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/16.jpg)
Use ACHILLES to see if the databases have the required data elements
![Page 17: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/17.jpg)
Also use ACHILLES to check for any data quality issues
![Page 18: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/18.jpg)
Use HERMES to figure out how to find a particular condition, drug, procedure, or other concept
![Page 19: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/19.jpg)
Use CIRCE to define the cohort of interest
![Page 20: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/20.jpg)
Use CALYPSO to conduct feasibility assessment to evaluate the impact of study inclusion criteria
![Page 21: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/21.jpg)
Use HERACLES to characterize the cohorts you developed
![Page 22: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/22.jpg)
Use HERACLES to characterize the cohorts you developed
![Page 23: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/23.jpg)
Use LAERTES to summarize evidence from existing data sources
![Page 25: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/25.jpg)
Open-source large-scale analytics through R
Why is this a novel approach? • Large-scale analytics,
scalable to ‘big data’ problems in healthcare: • millions of patients • millions of covariates • millions of questions
• End-to-end analysis, from
CDM through evidence • No longer de-coupling
‘informatics’ from ‘statistics’ from ‘epidemiology’
![Page 26: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/26.jpg)
Standardize Analysis and Results Reporting
![Page 28: An Open Science Approach to Medical Evidence …...–Safety surveillance (e.g., identifying new adverse event risks for drugs) –Comparative effectiveness (e.g. comparing interventional](https://reader034.vdocuments.mx/reader034/viewer/2022042416/5f3215a704477a6cb03f237e/html5/thumbnails/28.jpg)
Concluding Thoughts
• Open science requires optimized technical infrastructure, community infrastructure, and dedication
• But open science is not charity!
– The payoff can be both for individual participants and the community
• A diversity of skillsets brings value to all and greatly accelerates generation of high quality evidence