Content-Based Image Retrieval System for Breast Dynamic Contrast-Enhanced

(DCE)-MRI: Role of Kinetic Texture Features

Anant MadabhushiDirector, Laboratory for Computational Imaging and Bioinformatics

Assistant Professor, Dept. of Biomedical Engineering,Rutgers The State University of New Jersey

Member, Cancer Institute of New JerseyAdjunct Assistant Professor of Radiology, UMDNJ-Robert Wood

Johnson Hospital.

Statement of Disclosure

Collaborating Institutions:

Prior to beginning my presentation, I would like to inform you that I

have no conflicts of interest with respect to this work.

Laboratory for Computational Imaging and Bioinformatics

Lab Director: • Anant Madabhushi, PhD

Postdocs: • James Monaco, PhD• Gaoyu Xiao, PhD• Jun Xu, PhD

Graduate Students:• Jonathan Chappelow• Scott Doyle• Satish Viswanath• Pallavi Tiwari• George Lee • Shannon Agner• Ajay Basavanhally• Rob Toth• Andrew Janowczyk

Undergraduate Students • Jay Naik• Hussain Fatakdawala• Amod Jog

Clinical Collaborators • Michael D. Feldman, MD, PhD• John E Tomaszewski, MD• Mark Rosen, MD, PhD• Shridar Ganesan, MD, PhD• Nicholas Bloch, MD• Steven Master, PhD, MD• Mitch Schnall, MD, PhD• Salil Soman, MD,• John Nosher, MD

http://lcib.rutgers.edu

Breast MRI and Quantitative Lesion Descriptors

• Breast MRI recognized as useful complement to X-ray mammography and ultrasound

• Crucial to understand observable features of breast MRI in terms of:

• Current breast MRI evaluation methods (e.g., BI-RADS) have low specificity and high interobserver variability

CBIR: Why and How?• Retrieve images based on quantitative descriptors

associated with the lesion images• Clinical applications – training students/diagnosis

Contributions of this work

• CBIR system for Breast MRI

• Novel quantitative features– Kinetic texture

features

1 2 3 4 550

100

150

200

Time (min)

Sig

nal I

nten

sity

1 2 3 4 595

100

105

110

115

120

125

130

135

140

Time (min)

Gre

y Le

vel T

extu

re V

alue

Kinetic Intensity Curve

Kinetic Texture Curve

3TP map

Kinetic Texture Parametric Map

1 2 3 4 550

100

150

200

Time (min)

Wash-in

Wash-out

Rationale for Kinetic Texture Features

Kinetic Signal Intensity

Kinetic Texture

Sample Images Kinetic Profiles for Multiple Lesions

Sign

al In

tens

ity

Time

Time

Mea

n 2n

dor

der s

tatis

tical

feat

ure

Raw Image

CBIR WorkflowEM MapFinal Segmentation

Database

Query

Query and RetrievalBased on Image

Similarity

Kinetic Texture

Kinetic SignalIntensity

Morphology

SimilarityCalculation

EuclideanDistance

Canberra

Chi Square

FeatureExtraction

Lesion Classification in Reduced Dimensional Space

2nd Order Kinetic Texture, GE1st Order Kinetic Texture, LLE

0.1562

0.1562

0.1562

0.1562

-0.4-0.2

00.2

0.4-0.6

-0.4

-0.2

0

0.2

0.4

-4 -20 2

4

x 10-3-5

0

5

10

x 10-3

-6

-4

-2

0

2

4

6

x 10

Kinetic Signal Intensity, GE Morphology, GE

?

Quantitative Classification by Features

73

88

53

63

90

25

6367

59

8379

88

Accuracy Sensitivity Specificity

Morphological Pre-contrast Texture Kinetic Signal Intensity Kinetic Texture

Best combination of accuracy, sensitivity, and specificity

•SVM classifier using leave one out cross validation

Feature Evaluation in CBIR Context

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.58

0.6

0.62

0.64

0.66

0.68

0.7

0.72

0.74

0.76

Recall

Pre

cisi

on

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

0.7

Recall

Pre

cisi

on

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Recall

Pre

cisi

on

•Precision-Recall (P-R) curves allow for the assessment of CBIR system performance•An ideal P-R curve looks like a backward ROC curve

•All images in the database that are matches are retrieved first:

Kinetic TextureMorphology Kinetic Signal Intensity

P

R

CBIR Demonstration

Step 1:Open a query image

Step 2: Decide on a feature set to use for image comparison

Step 3: Retrieve images similar to query image

Step 4: Review the retrieved images and their characteristics

Stratifying Molecular BC subtypes –Triple Negative Breast Cancer

• Extremely aggressive phenotype • Do not respond to targeted hormonal therapies

Agner et al., Intl. Symposium on Biomedical Imaging, 2009.

Triple Negative (TN+)*

Other Malignant

Benign

Pathology CIBR

Query ImageLow-Dimensional Data Representation Calculate distance to other images to find nearest neighbors

Sort nearest neighbors in ascending distance from query

1

2

3

1

2

321 3

First image is most similar, second is second-most similar, and so on

Query

CBIR Results

Naik, Doyle et al., SPIE Medical Imaging, 2009.

Acknowledgements

• The Society for Imaging Informatics in Medicine

• The Coulter Foundation• The Aresty Research Center• The New Jersey Commission on Cancer

Research• The National Cancer Institute, • The Life Science Commercialization Fund,

Rutgers University• Department of Defense

Questions?

Pathology CBIR

Different distance metrics yield different embeddings

Query

CBIR Results

Classification Methodology

Lesion Segmentation

Morphological Feature

Extraction

Textural Feature

Extraction

DimensionalityReduction

Support VectorMachine

Classification

Kinetic TexturalFeature

Extraction

Segmentation

• Use of kinetic contrast information• Use of expectation maximization and

magnetostatic active contours

Original post-contrast image

EM map MAC segmentation of lesion and breast

Final lesion segmentation

result

Rationale for Developing Kinetic Texture

Sign

al In

tens

ity

Time TimeMea

n 2n

dor

der s

tatis

tical

feat

ure