delivering innovation to the clinic: the path from problem

AAMD 44th Annual MeetingJune 16 – 20, 2019

6/18/2019

1

Delivering Innovation to the Clinic:The Path from Problem to PracticeMark Gooding, DPhil

Chief Science and Technology Officer, Mirada Medical

@SciChief

©Copyright Mirada Medical Ltd. 2019

The problem?

1

2


6/18/2019

2


The problem?

300 years?


The problem?Disappointed patient

Unhappy doc

3

4


6/18/2019

3


The problem?


From problem to practice?

5

6


6/18/2019

4


An example


Do we have a problem?

7

8


6/18/2019

5





9

10


6/18/2019

6





• Body

• Femur_L

• Femur_R

• Bladder

• Rectum

• SeminalVesc

• Prostate

de novo 26 mins

Langmack KA, Perry C, Sinstead C, Mills J, Saunders D. The utility of atlas‐assisted segmentation in the male pelvis is dependent on the interobserver agreement of the structures segmented. The British journal of radiology. 2014 Oct 3;87(1043):20140299.

11

12


6/18/2019

7


But, there are solutions …

Sharp, Gregory, et al. "Vision 20/20: Perspectives on automated image segmentation for radiotherapy." Medical physics 41.5 (2014).


But, there are solutions …

Yes78%

No22%

Yes28%

No50%

Don't have22%

… with room for improvement

Does your institution havean auto‐contouring system?Does your institution use an auto‐contouring system?

Informal poll during talk at AAMD 2017N = ~70

13

14


6/18/2019

8


Auto‐contouring problems

1.) Workflow Integration




2.) Performance

Editing took longer than de novo!

Barely saved time

Good efficiency gain

Langmack KA, Perry C, Sinstead C, Mills J, Saunders D. The utility of atlas‐assisted segmentation in the male pelvis is dependent on the interobserver agreement of the structures segmented. The British journal of radiology. 2014 Oct 3;87(1043):20140299.

15

16


6/18/2019

9




2.) Performance

3.) Clinical Integration

Which contour is quickest to fix?

S. Nikolov et al. Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy. arXivpreprint arXiv:1809.04430. 2018 Sep 12.


Auto‐contouring solutions


2.) Performance

3.) Clinical Integration

Improved product design

Improved product design?Education/training

R&D Innovation

17

18


6/18/2019

10


Innovation

Perform

ance

Atlas contouringCirca 2008

Multi‐atlas fusionCirca 2012

Probablistic atlasCirca 2011

Model‐basedCirca 2013


Incremental innovation

• Less risk in research

• Perceived as less risky by customers

19

20


6/18/2019

11


Incremental Innovation ≠ Breakthrough

Perform

ance


Research Why didn’t it work?

21

22


6/18/2019

12


Limitation of atlas contouring


Rethinking auto‐contouring

Dosimetrist

Atlas contouring

23

24


6/18/2019

13


Model‐based contouring

Model‐based contouring

Dosimetrist


Horizontal Innovation

25

26


6/18/2019

14


Simple Neural Networks

The artificial neuron ( the Perceptron [Rosenblatt 1958]) models this as a combination of linear and non‐linear operations

A biological neuron “fires” the output signal to connected cells when the input signals reach a certain level


Artificial Neural Networks

Artificial neurons grouped into layers

• Input (Blue)

• Hidden (Red)

• Output (Yellow)

Each connection represents a tunable parameter – ‘weight’

Multi‐layer Perceptron [Rosenblatt 1961]

27

28


6/18/2019

15


Deep Networks

Stack many hidden layers to get ‘depth’

Each connection represents a tunable parameter – ‘weight’


Applying Neural Networks to images

Pass in entire image as a single vector

– Images can be big! Can end up with too many parameters to tune

29

30


6/18/2019

16


Convolutional layers

[Fukushima, 1980, Matan, Le Cun, 1991]

Images have structure

– Structure is mainly ‘local’

– Local structure can appear anywhere in the image


Types of model

Layers can be stacked together in many different ways, like construction blocks

The way of connecting the blocks are known as architectures

Different architectures can be designed for different tasks

31

32


6/18/2019

17


Deep Learning Contouring

Cat!


Research validation

Bakker H, Peressutti D, Aljabar P, Van Dijk LV, Van den Bosch L, Gooding M, Brouwer CL. OC‐0418: Quantitative evaluation of deep learning contouring of head and neck organs at risk. Radiotherapy and Oncology. 2018 Apr 1;127:S217‐8.

33

34


6/18/2019

18


Building a product


Business Case

Business Case

$Your problem

35

36


6/18/2019

19


Building a product


Current Good Manufacturing Process (cGMP)

• Requirements

• Risk Analysis

• Testing

– Validation

– Verification

• Documentation

37

38


6/18/2019

20


Requirements

On getting DICOM data

the system delineates the visible organs‐at‐risk

the system saves the contours as RTSS

the system sends the RTSS to the user

RTSS DICOM?

How? How well?

How to configure number of contour points?

How to handle user system being disconnected?


Intended use / Indications for use

• What it is

• What it is for

Intended use Indications for use

• Who it is for

• What are the limitations

… is a system designed to allow users to route DICOM‐compliant data to and from automated processing components…

… includes processing components for automatically contouring imaging data using machine learning

… is intended to be used by trained medical professionals

… must be used in conjunctions with appropriate software to review and edit results generated automatically….

… may be used as an input to clinical workflows including, but not limited to, radiation therapy treatment planning

… is NOT intended to automatically detect lesions

39

40


6/18/2019

21


Product classification

Class I Class IIa Class IIb Class III

Intended use

RISK

Indications for use


Risk analysis

Product Function Configuration

Originating Requirement XXXXXXXX

Effect (failure mode) Nuclear war started

Hazard (potential source of

harm)

Human race annihilated Hazard

relates to

usability?

No

Cause of failure User interface?

Risk Evaluation

User sends data to server

System becomes sentient

System becomes scared of being turned off

System launches nuclear missile strike against Russia…

No Occurrence Probability

Severity Risk Area

Negligible Extreme 162

Safety by Design Protective measures Responsible

Learning system should be separate from operation system

System should not be connected to nuclear missile launch systems

AN

41

42


6/18/2019

22


Testing

Did we build the thing right?Verification:

On getting DICOM data

the system delineates the visible organs‐at‐risk

the system saves the contours as RTSS

the system sends the RTSS to the user

Input test data

Check output matches expectation


DLC automatically generates contours!DLC automatically generates contours!

Testing

Did we build the thing right?Verification:

Did we build the right thing?Validation:

Simple functional testDLC is better than atlas contouring!

Quantitative comparisonClinical evaluation of time savingBlind comparison / Turing TestDLC is better than atlas contouring!DLC saves you time when contouring!DLC saves you time when contouring!

DLC is similar quality to clinical

contouring!

DLC is similar quality to clinical

contouring!

DLC is so good, the contours never need looking at!

Prove it never goes wrong! Ever!DLC is so good, the contours never need looking at!

43

44


6/18/2019

23


Why regulatory clearance is perceived as a blocker?

• Time to recruit enough patients• Size of effect• Time for clinical response• Long term effects / follow‐up

Research

Breakthrough Breakthrough PR


It’s all about safety

Class I Class IIa Class IIb Class III

RISK

Evidence

45

46


6/18/2019

24


Product released!


Deployment

The Customer

47

48


6/18/2019

25


Why does the customers prevent innovation?!


So where are we on the technology adoption curve?

Atlas‐based auto‐contouring

Yes78%

No22%

Deep Learning Contouring

49

50


6/18/2019

26


What drives adoption?


What hinders adoption?

The Customer

51

52


6/18/2019

27


An example?

2011

2015

2013

2019

2017

Number of PET/M

RI systems

10

70

30

100

2009

Commercial

release

Innovators

Early adopters

Early majority?


Summary

Learner Outcomes:

1. Drivers for innovation, and why interesting research may or may not make it out of the lab

2. Why regulation is important, and what is required to market a medical device

3. How software products are developed, and how this impacts your practice

It’s not there to prevent innovation, but to ensure there’s evidence of safety and efficacy.

Does it solve a problem?Will someone pay for it to be solved?

Good process and risk analysis are part of the regulatory approach for clinical software

Safety

Business

Process

Summary: What I should have said

53

54


6/18/2019

28


Summary: What I did say

1. Research

2. Regulatory

3. Reality

Risk

Risk

Risk

Regulation it the to lower riskAnd to ensure there’s evidence of safety and efficacy.

Real innovation requires researchResearch can be risky and time consuming

But adopting innovative technology can significantly improve practiceRewardAdoption can be slow, because of perceived risk

Worldwide HeadquartersMirada Medical Ltd.Oxford Centre for InnovationNew Road, Oxford, OX1 1BY United Kingdominfo@mirada‐medical.com

+44 (0)1865 261410

USA OfficeMirada Medical USA, Inc.999 18th Street, 2230SDenver, CO 80202 United Statesinfo@mirada‐medical.com

+1 877.872.2617


mirada‐medical.com

@SciChief

Mark.gooding@mirada‐medical.com

55

56

delivering innovation to the clinic: the path from problem

Documents