automated detection of change in serial mr brain studies julia patriarche, ph.d. november 18, 2008

Automated Detection of Change in Serial MR Brain Studies

Julia Patriarche, Ph.D.

November 18, 2008

Brain Tumor Suspected


Neurologic exam, Imaging, Biopsy

Resection, radiation, chemotherapy



Resection, radiation, chemotherapy

Serial Imaging



Baseline Followup

Change

or No Change?

FollowupBaseline

FollowupBaseline

Scanner has changedScanner Software has changedPulse Sequences have changed

Acquisition parameters have changedGradient Coils have changed

RF Inhomogeneities have changed Registration has changed

…

Change Detection is not so simple

1. Separation of acquisition changes from disease related changes 2. Information overload

3. Change Blindness / Inattentional Blindness

Change Blindness

Our visual systems have evolved to control our sensitivity to change

What we see is in the moment is richly detailed, but what is held in memory from one moment to the next is not the detailed visual information – it is abstracted.

Mirror Example

Ronald A. Rensink, University of British Columbia http://www.psych.ubc.ca/~rensink/flicker/download/

Changing case of letters

McConkie, G.W. and Zola, D. “Is visual information integrated across successive fixations in reading?” Percept. Psychophysiol. 1979; 25: 221-224.

AITeRnAtEd CaSe

Changing case of letters

McConkie, G.W. and Zola, D. “Is visual information integrated across successive fixations in reading?” Percept. Psychophysiol. 1979; 25: 221-224.

AITeRnAtEd CaSe

altErNaTeD cAsE

Change Blindness

There really isn’t a mechanism for changes in this abstracted information to grab your attention.

Change Blindness


If you don’t specifically attend to some feature, or some aspect of that feature at both time-points and then consciously compare them, there is a very good chance you’ll

miss changes.

Change Blindness


If you don’t specifically attend to some feature, or some aspect of that feature at both time-points and then consciously compare them, there is a very good chance you’ll

miss changes.

Trying to find subtle changes, in extent, or character is very difficult.

Change Blindness

There is a whole area of study devoted to change blindness

Change Blindness


People tend not to be aware of the extent of change blindness (i.e. change blindness-blindness is a topic of study)

Change Blindness


People tend not to be aware of the extent of change blindness (i.e. change blindness-blindness is a topic of study)

Visual change detection is so difficult that there are even games where you’re supposed to find the differences.

Change Blindness

The point: side-by-side presentation is very poorly matched to the human visual system. Unregistered images, inhomogeneity effects, information overload make

this problem even more challenging.

Change

or No Change?

Baseline Follow-up

Other reasons radiologists might miss changes:

Satisfaction of SearchChange occurs in an unexpected location

Change is in a sub-part of a complicated lesionEtc.

Not surprisingly…

Change detection is thought to be not very reproducible

What is desired:

1. Systems which reduce the quantity of data presented to the radiologist,

2. Systems which help to separate acquisition related change from disease related change,

3. Systems which present data in a way matched to the human visual system

4. Systems which produce objective, reproducible, and accurate metrics

Something which draws the radiologist’s attention to those changes.

Something which presents what is of interest – changes – to the radiologist.

It’s easier to understand the nature of the changes as a whole, if you can view an image of the changes.

Change Detector System Flow

Inhomogeneity Correction

Registration

Change Detector

Presentation of Results

Image Acquisition Consistent With Objective

Acquisition Considerations

To the Greatest Degree Possible:

Reduce slice thicknessEliminate inter-slice gaps

3D acquisitionEquivalent scanner

Equivalent scanner softwareEquivalent pulse sequences

Equivalent acquisition parametersCorrect for decay of gradient coils

Correct for inhomogenetiesIdentical administration of contrast

As with an experiment,

Control the variables, to isolate the changes of interest

Change Detector Algorithm Organization

Define Imaging Properties for: NAWM, Gray Matter, CSF,Lesion, and Enhancing Lesion

Define Brain Parenchyma

Compute Lesion and Enhancing Lesion Memberships On aVoxel-by-voxel Basis

Significant Region Detection

Automated Sample PointsGeneration / Automated

Parenchyma MaskGeneration

Lesion Finder

Noise Reduction

Change Detector Compute Measures of Change On a Voxel-by-voxel Basis,expressed in Δ membership of lesion and enhancing lesion

The algorithm is designed to detect changes which are:

Subtle in extentSubtle in degree

The algorithm produces both:

Quantitative summariesQualitative color change maps

Baseline

Followup

Baseline Followup

Color Change Map

Original Clinical Interpretation: Stable / Time to Progression

Diagnosis = 7 months

Baseline Followup

Baseline Followup

Color Change Map


Diagnosis = 3.25 months

Baseline Followup

Baseline Followup

Color Change Map


Diagnosis = 4.75 months

Baseline Follow-up

Baseline Follow-up

Color Change Map


Diagnosis > 18 months

Determining that something hasn’t changed, can be as

tricky as determining that it has

Study #1: Assisted Serial MRI Examination

Sensitivity: Fraction of the actually positive cases which were correctly identified

Specificity: Fraction of the actually negative cases which were correctly identified

Study #1: Assisted Serial MRI Examination

•6 neuroradiologists examined a series of 50 serial MR pairs from 28 patients•Asked to rate each case as stable or progressing•Double cross over study with one month separation

Study #1 Results

• All but one neuroradiologist had improved sensitivity

• Two NRs achieved perfect sensitivity with the change detector, none without

• Mean accuracy improved from 0.859 to 0.891

• Mean sensitivity was largely unchanged, from 0.892 to 0.891

• Mean specificity improved moderately from 0.848 to 0.874

Interobserver Agreement

• Without the change detector, there were 17 cases in which all the neuroradiologists agreed with the gold standard

• With the change detector, there were 23

Study #1 Results:

• Automated system examined 88 serial comparisons• Divided cases into: stable or progressing

• Automated diagnosis was compared to original clinical diagnosis

Study #2: Automated Serial MRI Examination

Suggests that the change detector may be able to detect changes earlier than expert neuroradiologists.

Patriarche JW, Erickson BJ. "Part 2. Automated Change Detection and Characterization Applied to Serial MR of Brain Tumors May Detect Progression Earlier Than Human Experts". Journal of Digital Imaging 2007; 20(4): 321-328.

Reasons for good outcome:

• No objective definition of progression.



• Subtle changes difficult to see • Especially when images displayed side-by-side • Especially when images not registered




• Changes may not be obvious looking at only one pulse sequence, but might present across pulse sequences





• Changes unexpected in location, etc.






• Satisfaction of Search






• Satisfaction of Search

• Information overload - impractical or impossible to wade through every possible comparison manually

The Point:

• Computers are great at methodically wading through large amounts of data, looking for needles in hay-stacks, integrating large amounts of data at once, applying serial processing steps, analyzing data mathematically, and bringing noteworthy observations to the attention of the neuroradiologist.

Eric Kischell has implemented a very streamlined workflow• Provides full automation, end-to-end

Liqin Wang has implemented a web-based interface • Neuroradiologists can review change detector output from any computer with a web-browser• Allows sophisticated interaction with output, e.g. via two color map modes, linked cursors, and ‘flicker mode’ viewing of serial images.

Current Work

Clinical Deployment:

• Facilitate studies with more patients and neuroradiologists• Facilitate more varied study designs• Directly aid patients

Other Developments:

• Full automation / full reproducibility - removes barriers to widespread deployment.

Other Developments:

• Full automation / full reproducibility - removes barriers to widespread deployment.

• Greatly improved sensitivity and specificity

Layered AI Architecture:

Change detector possesses multiple layers of automated reasoning / information gathering.

Starts with simplistic but highly reliable knowledge, and uses this to acquire case-specific information.

Uses this case-specific information to acquire more complex information, etc. up the hierarchy.

SignificantRegion Detector


AutomatedSample Points Algorithm

Parenchyma MaskAlgorithm




Lesion Finder




Lesion Finder

Change Detector




Lesion Finder

Change Detector

?




Lesion Finder

Change Detector

?

?

Components are modular and interchangeable.

Obviously:If everything is built on the lower levels – if the reasoning is based on the knowledge gleaned by those lower levels -- those levels must produce very reliable information.

Information Overload:

• The phenomenon is extremely wide-spread.

• The problem / opportunity will almost definitely continue to grow.

Quantitative & Automated Analyses:

• Makes information overload, information affluence

Change Detector:

• May help to train new practitioners

• May serve as a model of how computer can liberate neuroradiologist from doing mundane things sub-optimally, and instead have computer wade through mountains of analysis and present the clinician with refined and concise data tailored to their purpose.

• May serve as a model of how computer can perform a mountain of peripheral analyses and bring to clinician’s attention things they weren’t looking for (but which are important).

• May serve as a model of ‘layered analyses’ which are impractical or impossible without computer assistance.

All of this:

Designed to translate what is acquired, into what is desired – data tailored to the task at hand, by using processing and domain specific knowledge.

All of this:


Don’t ask the radiologist to do the routine / protocol-driven work – define the procedure, and build a machine to perform that procedure.

All of this:



Use the computer’s ability to do routine pre-defined tasks precisely, every time, and to handle complex quantitative and layered analyses with ease; to do massive numbers of procedure-driven analyses on each case, to present highly processed information tailored to the task at hand, and to direct attention to identified features which require it.

All of this:



Use the computer’s ability to do routine pre-defined tasks precisely, every time, and to handle complex quantitative and layered analyses with ease; to do massive numbers of procedure-driven analyses on each case, to present highly processed information tailored to the task at hand, and to direct attention to identified features which require it.

Use the computer as a smart helper, who goes away and does tons of analysis, and who brings to the attention of the radiologist things which should be attended to.

Some clinical reasons the change detector is of value:

Create a quantitative definition of progression

Possible improved outcomes through earlier interventions

Potentially facilitate screening

Potentially facilitate personalized therapy

Compare novel therapies, through automatic, reproducible, and quantitative measures of change.

Future Directions:

Many possibilities.

Future Directions:

Many possibilities.

Add additional pulse sequences (e.g. diffusion, perfusion, etc.).

Future Directions:

Many possibilities.


Add more complex anatomical knowledge (e.g. so change detector knows what is white matter, gray matter, CSF, and can act appropriately.

Future Directions:

Many possibilities.



Identify and quantify different kinds of changes, ex. attempt to differentiate between character change and mass effect; identify and quantify atrophy.

Future Directions:

Many possibilities.




Apply technique to different anatomical regions (e.g. breast, etc.)

Future Directions:

Many possibilities.




Apply technique to different anatomical regions (e.g. breast, etc.)

A variety of clinical studies.

Acknowledgements

Christopher Wood, M.D.

Norbert Campeau, M.D.

Edward Lindell, M.D.

Vladimir Savcenko, M.D.

Norman Arslanlar, D.O.

Bradley Erickson M.D., Ph.D.

Acknowledgements

Bradley Erickson, M.D., Ph.D. – Director, Mayo Clinic Radiology Informatics Lab

Brian O’Neill, M.D. – Director, Mayo Neurology Fellowship Program

Christopher Chute, M.D., Dr. P.H. – Director, Mayo Informatics Fellowship Program

Eric Kischell – Programmer, implemented change detector integrated system

Liqin Wang – Programmer, implemented registration algorithm, web interface

Mayo Brain Computation Workgroup

University of Hawaii – Department of Information and Computer Sciences

NIH T32 NS07494-03

NIH T15 LM07041-23

automated detection of change in serial mr brain studies julia patriarche, ph.d. november 18, 2008

Documents