delivering innovation to the clinic: the path from problem
TRANSCRIPT
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
1
Delivering Innovation to the Clinic:The Path from Problem to PracticeMark Gooding, DPhil
Chief Science and Technology Officer, Mirada Medical
@SciChief
©Copyright Mirada Medical Ltd. 2019
The problem?
1
2
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
2
©Copyright Mirada Medical Ltd. 2019
The problem?
300 years?
©Copyright Mirada Medical Ltd. 2019
The problem?Disappointed patient
Unhappy doc
3
4
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
3
©Copyright Mirada Medical Ltd. 2019
The problem?
©Copyright Mirada Medical Ltd. 2019
From problem to practice?
5
6
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
4
©Copyright Mirada Medical Ltd. 2019
An example
©Copyright Mirada Medical Ltd. 2019
Do we have a problem?
7
8
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
5
©Copyright Mirada Medical Ltd. 2019
Do we have a problem?
©Copyright Mirada Medical Ltd. 2019
Do we have a problem?
9
10
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
6
©Copyright Mirada Medical Ltd. 2019
Do we have a problem?
©Copyright Mirada Medical Ltd. 2019
Do we have a problem?
• Body
• Femur_L
• Femur_R
• Bladder
• Rectum
• SeminalVesc
• Prostate
de novo 26 mins
Langmack KA, Perry C, Sinstead C, Mills J, Saunders D. The utility of atlas‐assisted segmentation in the male pelvis is dependent on the interobserver agreement of the structures segmented. The British journal of radiology. 2014 Oct 3;87(1043):20140299.
11
12
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
7
©Copyright Mirada Medical Ltd. 2019
But, there are solutions …
Sharp, Gregory, et al. "Vision 20/20: Perspectives on automated image segmentation for radiotherapy." Medical physics 41.5 (2014).
©Copyright Mirada Medical Ltd. 2019
But, there are solutions …
Yes78%
No22%
Yes28%
No50%
Don't have22%
… with room for improvement
Does your institution havean auto‐contouring system?Does your institution use an auto‐contouring system?
Informal poll during talk at AAMD 2017N = ~70
13
14
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
8
©Copyright Mirada Medical Ltd. 2019
Auto‐contouring problems
1.) Workflow Integration
©Copyright Mirada Medical Ltd. 2019
Auto‐contouring problems
1.) Workflow Integration
2.) Performance
Editing took longer than de novo!
Barely saved time
Good efficiency gain
Langmack KA, Perry C, Sinstead C, Mills J, Saunders D. The utility of atlas‐assisted segmentation in the male pelvis is dependent on the interobserver agreement of the structures segmented. The British journal of radiology. 2014 Oct 3;87(1043):20140299.
15
16
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
9
©Copyright Mirada Medical Ltd. 2019
Auto‐contouring problems
1.) Workflow Integration
2.) Performance
3.) Clinical Integration
Which contour is quickest to fix?
S. Nikolov et al. Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy. arXivpreprint arXiv:1809.04430. 2018 Sep 12.
©Copyright Mirada Medical Ltd. 2019
Auto‐contouring solutions
1.) Workflow Integration
2.) Performance
3.) Clinical Integration
Improved product design
Improved product design?Education/training
R&D Innovation
17
18
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
10
©Copyright Mirada Medical Ltd. 2019
Innovation
Perform
ance
Atlas contouringCirca 2008
Multi‐atlas fusionCirca 2012
Probablistic atlasCirca 2011
Model‐basedCirca 2013
©Copyright Mirada Medical Ltd. 2019
Incremental innovation
• Less risk in research
• Perceived as less risky by customers
19
20
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
11
©Copyright Mirada Medical Ltd. 2019
Incremental Innovation ≠ Breakthrough
Perform
ance
©Copyright Mirada Medical Ltd. 2019
Research Why didn’t it work?
21
22
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
12
©Copyright Mirada Medical Ltd. 2019
Limitation of atlas contouring
©Copyright Mirada Medical Ltd. 2019
Rethinking auto‐contouring
Dosimetrist
Atlas contouring
23
24
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
13
©Copyright Mirada Medical Ltd. 2019
Model‐based contouring
Model‐based contouring
Dosimetrist
©Copyright Mirada Medical Ltd. 2019
Horizontal Innovation
25
26
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
14
©Copyright Mirada Medical Ltd. 2019
Simple Neural Networks
The artificial neuron ( the Perceptron [Rosenblatt 1958]) models this as a combination of linear and non‐linear operations
A biological neuron “fires” the output signal to connected cells when the input signals reach a certain level
©Copyright Mirada Medical Ltd. 2019
Artificial Neural Networks
Artificial neurons grouped into layers
• Input (Blue)
• Hidden (Red)
• Output (Yellow)
Each connection represents a tunable parameter – ‘weight’
Multi‐layer Perceptron [Rosenblatt 1961]
27
28
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
15
©Copyright Mirada Medical Ltd. 2019
Deep Networks
Stack many hidden layers to get ‘depth’
Each connection represents a tunable parameter – ‘weight’
©Copyright Mirada Medical Ltd. 2019
Applying Neural Networks to images
Pass in entire image as a single vector
– Images can be big! Can end up with too many parameters to tune
29
30
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
16
©Copyright Mirada Medical Ltd. 2019
Convolutional layers
[Fukushima, 1980, Matan, Le Cun, 1991]
Images have structure
– Structure is mainly ‘local’
– Local structure can appear anywhere in the image
©Copyright Mirada Medical Ltd. 2019
Types of model
Layers can be stacked together in many different ways, like construction blocks
The way of connecting the blocks are known as architectures
Different architectures can be designed for different tasks
31
32
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
17
©Copyright Mirada Medical Ltd. 2019
Deep Learning Contouring
Cat!
©Copyright Mirada Medical Ltd. 2019
Research validation
Bakker H, Peressutti D, Aljabar P, Van Dijk LV, Van den Bosch L, Gooding M, Brouwer CL. OC‐0418: Quantitative evaluation of deep learning contouring of head and neck organs at risk. Radiotherapy and Oncology. 2018 Apr 1;127:S217‐8.
33
34
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
18
©Copyright Mirada Medical Ltd. 2019
Building a product
©Copyright Mirada Medical Ltd. 2019
Business Case
Business Case
$Your problem
35
36
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
19
©Copyright Mirada Medical Ltd. 2019
Building a product
©Copyright Mirada Medical Ltd. 2019
Current Good Manufacturing Process (cGMP)
• Requirements
• Risk Analysis
• Testing
– Validation
– Verification
• Documentation
37
38
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
20
©Copyright Mirada Medical Ltd. 2019
Requirements
On getting DICOM data
the system delineates the visible organs‐at‐risk
the system saves the contours as RTSS
the system sends the RTSS to the user
RTSS DICOM?
How? How well?
How to configure number of contour points?
How to handle user system being disconnected?
©Copyright Mirada Medical Ltd. 2019
Intended use / Indications for use
• What it is
• What it is for
Intended use Indications for use
• Who it is for
• What are the limitations
… is a system designed to allow users to route DICOM‐compliant data to and from automated processing components…
… includes processing components for automatically contouring imaging data using machine learning
… is intended to be used by trained medical professionals
… must be used in conjunctions with appropriate software to review and edit results generated automatically….
… may be used as an input to clinical workflows including, but not limited to, radiation therapy treatment planning
… is NOT intended to automatically detect lesions
39
40
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
21
©Copyright Mirada Medical Ltd. 2019
Product classification
Class I Class IIa Class IIb Class III
Intended use
RISK
Indications for use
©Copyright Mirada Medical Ltd. 2019
Risk analysis
Product Function Configuration
Originating Requirement XXXXXXXX
Effect (failure mode) Nuclear war started
Hazard (potential source of
harm)
Human race annihilated Hazard
relates to
usability?
No
Cause of failure User interface?
Risk Evaluation
User sends data to server
System becomes sentient
System becomes scared of being turned off
System launches nuclear missile strike against Russia…
No Occurrence Probability
Severity Risk Area
Negligible Extreme 162
Safety by Design Protective measures Responsible
Learning system should be separate from operation system
System should not be connected to nuclear missile launch systems
AN
41
42
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
22
©Copyright Mirada Medical Ltd. 2019
Testing
Did we build the thing right?Verification:
On getting DICOM data
the system delineates the visible organs‐at‐risk
the system saves the contours as RTSS
the system sends the RTSS to the user
Input test data
Check output matches expectation
©Copyright Mirada Medical Ltd. 2019
DLC automatically generates contours!DLC automatically generates contours!
Testing
Did we build the thing right?Verification:
Did we build the right thing?Validation:
Simple functional testDLC is better than atlas contouring!
Quantitative comparisonClinical evaluation of time savingBlind comparison / Turing TestDLC is better than atlas contouring!DLC saves you time when contouring!DLC saves you time when contouring!
DLC is similar quality to clinical
contouring!
DLC is similar quality to clinical
contouring!
DLC is so good, the contours never need looking at!
Prove it never goes wrong! Ever!DLC is so good, the contours never need looking at!
43
44
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
23
©Copyright Mirada Medical Ltd. 2019
Why regulatory clearance is perceived as a blocker?
• Time to recruit enough patients• Size of effect• Time for clinical response• Long term effects / follow‐up
Research
Breakthrough Breakthrough PR
©Copyright Mirada Medical Ltd. 2019
It’s all about safety
Class I Class IIa Class IIb Class III
RISK
Evidence
45
46
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
24
©Copyright Mirada Medical Ltd. 2019
Product released!
©Copyright Mirada Medical Ltd. 2019
Deployment
The Customer
47
48
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
25
©Copyright Mirada Medical Ltd. 2019
Why does the customers prevent innovation?!
©Copyright Mirada Medical Ltd. 2019
So where are we on the technology adoption curve?
Atlas‐based auto‐contouring
Yes78%
No22%
Deep Learning Contouring
49
50
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
26
©Copyright Mirada Medical Ltd. 2019
What drives adoption?
©Copyright Mirada Medical Ltd. 2019
What hinders adoption?
The Customer
51
52
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
27
©Copyright Mirada Medical Ltd. 2019
An example?
2011
2015
2013
2019
2017
Number of PET/M
RI systems
10
70
30
100
2009
Commercial
release
Innovators
Early adopters
Early majority?
©Copyright Mirada Medical Ltd. 2019
Summary
Learner Outcomes:
1. Drivers for innovation, and why interesting research may or may not make it out of the lab
2. Why regulation is important, and what is required to market a medical device
3. How software products are developed, and how this impacts your practice
It’s not there to prevent innovation, but to ensure there’s evidence of safety and efficacy.
Does it solve a problem?Will someone pay for it to be solved?
Good process and risk analysis are part of the regulatory approach for clinical software
Safety
Business
Process
Summary: What I should have said
53
54
AAMD 44th Annual MeetingJune 16 – 20, 2019
6/18/2019
28
©Copyright Mirada Medical Ltd. 2019
Summary: What I did say
1. Research
2. Regulatory
3. Reality
Risk
Risk
Risk
Regulation it the to lower riskAnd to ensure there’s evidence of safety and efficacy.
Real innovation requires researchResearch can be risky and time consuming
But adopting innovative technology can significantly improve practiceRewardAdoption can be slow, because of perceived risk
Worldwide HeadquartersMirada Medical Ltd.Oxford Centre for InnovationNew Road, Oxford, OX1 1BY United Kingdominfo@mirada‐medical.com
+44 (0)1865 261410
USA OfficeMirada Medical USA, Inc.999 18th Street, 2230SDenver, CO 80202 United Statesinfo@mirada‐medical.com
+1 877.872.2617
©Copyright Mirada Medical Ltd. 2018
mirada‐medical.com
@SciChief
Mark.gooding@mirada‐medical.com
55
56