clinical trial data wants to be free: lessons from the immport immunology data and analysis portal
DESCRIPTION
Presentation to the Clinical and Research Ethics Seminar, Clinical and Translational Science Center, Buffalo, January 21, 2014 https://immport.niaid.nih.gov/ http://youtu.be/booqxkpvJMgTRANSCRIPT
![Page 1: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/1.jpg)
Clinical Trial Data Wants to be Free
Barry Smith
![Page 2: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/2.jpg)
Publish all drug trial results, says Dr Ben Goldacre, 19 June 2013:
http://www.bbc.co.uk/news/uk-politics-22957195
![Page 3: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/3.jpg)
John Holdren, Director of the Office of Science and Technology Policy, “has directed Federal agencies with more than $100M in R&D expenditures to develop plans to make the published results of federally funded research freely available to the public within one year of publication and requiring researchers to better account for and manage the digital data resulting from federally funded scientific research.”
![Page 4: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/4.jpg)
Increasingly, these data will be digitalized
![Page 5: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/5.jpg)
Can we take care of the problems here, at least prospectively?
![Page 6: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/6.jpg)
Paper is giving way to digitalized data• As more studies come on-line, the problems involved
in making them available to automated analysis will only get worse
• What we need is prospective standardization of a useful sort – using standards that will make the life of trialists easier and also increase the value of the data they produce for secondary use
![Page 7: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/7.jpg)
Ought implies can
![Page 8: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/8.jpg)
Ought implies can
Complete clinical trial data can be made freely available, in de-identified form
![Page 9: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/9.jpg)
https://immport.niaid.nih.gov/
![Page 10: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/10.jpg)
10
Complete, deidentified data for 89 trials
![Page 11: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/11.jpg)
![Page 12: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/12.jpg)
![Page 13: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/13.jpg)
DAIT-funded Projects Depositing Data Into ImmPort
• Immune Tolerance Network (ITN)• Atopic Dermatitis Research Network (ADRN)• Population Genetics Analysis Program• HLA Region Genetics in Immune-Mediated Diseases• Clinical Trials in Organ Transplantation in Children (CTOT-
C)• Consortium of Food Allergy Research (CoFAR)• Renal and Lung Living Donors Evaluation Study (RELIVE)• The Inner City Asthma Consortium (ICAC)
![Page 14: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/14.jpg)
ImmPort TeamNorthrop Grumman Information Technology Health Solutions
Stanford UniversityAtul Butte (PI)Mark Davis (co-PI)Garry Nolan (co-PI)Ravi Shankar
University of Buffalo Barry Smith (co-PI)Alex Diehl Alan Ruttenberg
Technion Israel Institute of TechnologyShai Shen-Orr
![Page 15: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/15.jpg)
Why do we want the data to be free?• Education• Replication of results • Scientific scrutiny / economy • Secondary use
• New biomedical discovery, including DIY science• New –omics/ Big Data start-ups• Reanalysis of original results
oLinking existing trial data to new bioinformatics discoveriesoHarvesting existing trial data by creating new, virtual meta-
trials
![Page 16: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/16.jpg)
Tutorial and WorkshopOntology and Imaging
Informatics
• Would the training of pathologists (or other professionals) change if hundreds/thousands of trial-labeled images were publicly available?
![Page 17: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/17.jpg)
Cooperative Clinical Trials in Pediatric Transplantation (CCTPT): Four studies
2001 2003 2004 2005 2006 20072002
CN01
SW01
SNS01
PC01
# Arms
Participate Centers
Length of follow up (years)
Transplant years
CN01 1 4 3 2001-03
SW01 2 19 3 2001-04
SNS01 2 12 3 2004-06
PC01 1 4 2 2005-07
PRELIMINARY
![Page 18: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/18.jpg)
PRELIMINARY
![Page 19: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/19.jpg)
Common follow-up time points for 4 studies
5 time points: time 0, 3, 6, 12, and 24 months post-transplant are in common
PRELIMINARY
![Page 20: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/20.jpg)
HLA data (purple)
Flow Cytometry data (yellow)
PCR data (green)
Study Protocol,Operational data,Clinical data (blue)
ITN Data (with thanks to Ravi Shankar)
SpecimenManagementData (green)
21
![Page 21: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/21.jpg)
What is in a visit name?Visit 0, v0, v 0, 0, Day 0, Transplant
ProtocolGroup
Assay Group
Schedule of Events
SpecimenTable
0
0
![Page 22: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/22.jpg)
What is in a visit name?Visit 0, v0, v 0, 0, Day 0, Transplant
ProtocolGroup
Assay Group
CRO
Schedule of Events
SpecimenTable
CRF
Day 0, Transplant
0
0
![Page 23: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/23.jpg)
What is in a visit name?Visit 0, v0, v 0, 0, Day 0, Transplant
ProtocolGroup
Assay Group
CRO
OperationsGroup
Schedule of Events
SpecimenTable
TubeTable
CRF
v 0
0
0
Day 0, Transplant
![Page 24: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/24.jpg)
What is in a visit name?Visit 0, v0, v 0, 0, Day 0, Transplant
ProtocolGroup
Assay Group
CRO
CimarronOperationsGroup
Tube Manufacturer
Schedule of Events
SpecimenTable
TubeTable
CRF
ImmunoTrak
KitReport
Day 0, Transplant
0
0
v 0
v 0v0, Visit 0
![Page 25: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/25.jpg)
What is in a visit name?Visit 0, v0, v 0, 0, Day 0, Transplant
ProtocolGroup
Assay Group
CRO
CimarronOperationsGroup
Schedule of Events
SpecimenTable
TubeTable
CRF
ImmunoTrak
KitReport
CoreLabs
Assays
0
0
Day 0, Transplant
v0
v0, Visit 0v 0
Tube Manufacturer v 0
![Page 26: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/26.jpg)
What is in a visit name?Visit 0, v0, v 0, 0, Day 0, Transplant
CRO
ProtocolGroup
Assay Group
CimarronOperationsGroup
Data Center
Schedule of Events
SpecimenTable
TubeTable
CRF
ImmunoTrak
KitReport
Database
CoreLabs
Assays
Day 0, Transplant
v0
0
0
v 0v0, Visit 0
Tube Manufacturer v 0
![Page 27: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/27.jpg)
What is in a visit name?Visit 0, v0, v 0, 0, Day 0, Transplant
28
CRO
ProtocolGroup
Assay Group
CimarronOperationsGroup
Data Center
Schedule of Events
SpecimenTable
TubeTable
CRF
ImmunoTrak
KitReport
Database
CoreLabs
Assays
Day 0, Transplant
v0
0
0
v 0v0, Visit 0
Tube Manufacturer v 0
![Page 28: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/28.jpg)
29
Allergy Score ( Study Collection Day) Lab Tests ( Study Time collected)
Microarray Data ( Only Visit ) Flow ( Collection_Study_day and Visit)
Mappings between protocol, lab tests and mechanistic assays were missing
![Page 29: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/29.jpg)
How are these problems currently being solved?
Hard work
Problems with hard work:•Does not scale•Does not comport with the vision underlying ImmPort – that we can transform clinical medicine from an art into an (information-driven) science, based on repeatable processes documented in advance
![Page 30: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/30.jpg)
Goals of ImmPort• Accelerate a more collaborative and coordinated
research environment• Create an integrated database that broadens the
usefulness of scientific data• Advance the pace and quality of scientific discovery • Integrate relevant data sets from participating
laboratories, public and government databases, and private data sources
• Promote rapid availability of important findings• Provide analysis tools to advance immunological
research
![Page 31: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/31.jpg)
pipeline
32
perform study &collect data
analyze data(SAS …)
submit data toImmPort
process & de-identify, data in ImmPort
discover, aggregate, analyze,data inImmPort
![Page 32: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/32.jpg)
Pipeline
33
PIs, hospitals, biostatisticians NorthropGrumman
Max & Mindy
![Page 33: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/33.jpg)
Alternatives to the strategy of hard work
perform study &collect data
analyze data(SAS …)
submit data toImmPort
process & de-identify, data in ImmPort
discover, aggregate, analyze,data inImmPort
PIs, hospitals, biostatisticians NorthropGrumman
Stanford (Max & Mindy)
![Page 34: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/34.jpg)
What we have currently
PIs, hospitals, biostatisticians, CROs …
35
NorthropGrumman
Stanford (Max & Mindy)
Lots of free text, local formats, local standards, local terminologies operating here
![Page 35: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/35.jpg)
Semantic Web strategy of post-coordination via arms-length enhancement of data
PIs, hospitals, biostatisticians, CROs …
36
NorthropGrumman
Stanford (Max & Mindy)
Lots of free text, local formats, local standards, local terminologies operating here
uniform standards and ontologies applied post hoc
![Page 36: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/36.jpg)
The problem with this post hoc strategy is that it still requires the same amount of hard work
PIs, hospitals, biostatisticians, CROs …
37
NorthropGrumman
Stanford (Max & Mindy)
Lots of free text, local formats, local standards, local terminologies operating here
uniform standards and ontologies applied post hoc
![Page 37: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/37.jpg)
A preferred but much more ambitious strategy: Pre-coordination
PIs, hospitals, biostatisticians, CROs …
38
NorthropGrumman
Stanford (Max & Mindy)
Identify uniform standards that can be applied already here
![Page 38: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/38.jpg)
ImmPort data is already being taggedFor example•where data is prepared to meet FDA requirements•where data is published to meet NIH mandates for reusability•in the post-submission phase, where data is analyzed by third partiesBut this tagging is •partial•uncoordinated•uses ontologies and analysis tools of varying quality
![Page 39: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/39.jpg)
Ought implies can
Complete clinical trial data can be made freely available, in de-identified formBut to be useful these need need to be discoverable and analyzableWhich means: standardization
![Page 40: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/40.jpg)
Two alternative strategies for standardization
• 1. via consensus-based ontologies adapted to the needs of trialists
• 2. via FDA (CDISC) standards
![Page 41: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/41.jpg)
Advantages of pre-coordination with ontologies
• Better quality of data for all Maxes and Mindies• Enhanced discoverability of data• Cost-free submission of data to ImmPort• Works even for those trials which have nothing to do with
FDA• Allows incremental strategy • Leads to immediate integration with bioinformatics data
sources
![Page 42: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/42.jpg)
Immune-Related Ontologies (examples) Protein Ontology (PRO)Gene Ontology (GO) Cell Ontology (CL)Immune Epitope Ontology Beta Cell Genomics Ontology Infectious Disease Ontology
Allergy OntologyAntibody OntologyCDISC2RDFCL+ (for CyTOF)Cytokine OntologyImmunology OntologyVDJ Ontology
http://ncorwiki.buffalo.edu/index.php/Immunology_Ontologies43
![Page 43: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/43.jpg)
The very same ontological framework will work not just for BISC but also for the NIAID BRCs
44
![Page 44: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/44.jpg)
An Example of his this will work: The ImmPort Antibody Registry/Ontology
Experimental methods typically report antibody clones or target markers using non-standardized terminology:
CD3e, CD3E, CD3ɛ, CD3 epsilon (protein names)
HIT3e vs. UCHT1 (antibody clones for CD3e)
550367 vs. 300401 (catalog numbers for anti-CD3e antibody reagents)
Even catalog numbers have a half-life as concerns the information they provide
![Page 45: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/45.jpg)
ImmPort Antibody Registry (Diehl, et al)
from BD Lyoplate Screening Panels Human Surface Markers46
![Page 46: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/46.jpg)
Semantic Query / Discoverability
Find all experiments in which IL2 mRNA levels were quantified
Infer that IL2 mRNA is analyte and SAGE, QPCR and microarrays are appropriate measurement techniques
Find all experiment samples that include samples from subjects with diseases like Type 1 diabetes
Infers that the source of the biological sample used must be a human subject with Type 1 diabetes mellitus, Grave’s disease or other autoimmune diseases of endocrine glands
![Page 47: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/47.jpg)
Second strategy: coordination through FDA (CDISC) standards
Currently, PIs may need to reformat twice, once for ImmPort, once for FDACoordination via ontologies would require mappings from these ontologies to CDISC standardsBut there is an alternative strategy: have all trialists use CDISC standards •Map the CDISC standards to common ontologies
![Page 48: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/48.jpg)
Problems with the FDA (CDISC) strategy• They have not been developed to support computation across
biological data• They are very slow to evolve (> 14 years so far)• They are designed to meet the needs of data managers rather than
bioinformaticians• They lack compositionality (hard to integrate with other data• They are very complicated and so typically not in fact used by
trialists; rather they are generated by software (for example Medidata) – with some loss in data quality (?) (through hard work?)
![Page 49: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/49.jpg)
![Page 50: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/50.jpg)
BRIDG 3.2 Domain Analysis Model
![Page 51: Clinical trial data wants to be free: Lessons from the ImmPort Immunology Data and Analysis Portal](https://reader034.vdocuments.mx/reader034/viewer/2022051108/5461b64daf79595e398b64d0/html5/thumbnails/51.jpg)
Strategy
• Identify useful standards and build them into the clinical trial management systems, laboratory information management systems, such as LabKey that the PIs will be using in any case?
• Join with Ravi Shankar and with the PHUSE (EU, Roche, AstraZeneca, FDA, …) project to incorporate ontology technology into CDISC