strategic organization of image processingpcosman/nelson2010.pdf · 2010-05-13 · uc san diego...

Thomas R. Nelson, Ph.D. Professor of Radiology and Bioengineering

UC San Diego [email protected]

Applications of Image Processing and Registration in Medical Imaging Applications of Image Processing and Registration in Medical ImagingApplications of Image Processing and Registration in Medical Imaging

Strategic Organization of Image Processing

CREDITS:• Philips Ultrasound• GE Medical Systems• Siemens Ultrasound• de Korte et.al., Erasmus University Rotterdam

Filtering

Anatomic Reference

Image series

Patient AtlasRegistration

Visualization

Parameter Estimates

Segmentation

Applications of Image Processing and Registration in Medical Imaging

• Anatomic– Intensity

– Shape

– Speckle pattern

• Functional– Motion

– Temporal variation

– Tissue properties

3

Applications of Image Processing and Registration in Medical Imaging Applications of Image Processing and Registration in Medical Imaging

Breast Density (%)

6.0

5.0

4.0

3.0

2.0

1.0

0.0

0 1-10 >10-25 >25-50 >50-75 >75

5.3

3.4

2.42.2

1.21.0

Rel

ativ

e R

isk

Breast Cancer Risk


Background:

• One in 6 women will develop breast cancer• Early detection is the most important factor for survival • Most cancers arise in dense ductal tissue• Mammography is the “gold standard” but dense breasts are a challenge• Volume imaging (bCT or VBUS) improves lesion localization• Precision localization improves biopsy results, especially for small lesions• Robotic devices may provide more precise device placement

Applications of Image Processing and Registration in Medical ImagingApplications of Image Processing and Registration in Medical Imaging Applications of Image Processing and Registration in Medical Imaging

Breast Imaging

•Tomographic slices

•Pendant breast

•less discomfort

•better resolution

Building a Breast Imaging Machine

Breast CT - Design / Construction System Modules Acquisition ReconstructionBreast Ultrasound - Design / Construction System Modules Acquisition ReconstructionMammogram Simulation Classification & Extraction CompressionTissue Parameter EstimationImage-Guided Robotic Biopsy


40 cm

14 cm


Breast CT Scanner Design - CAD Engineering


System Control Modules

• interface all sub-systems to master control

• position synchronization

• control timing

• control x-ray on

• interlocks

• acquisition control


X-ray tube & power supply

• 100 kVp 6.4 mA --> 0.7 kW generator

• 0.4 mm x 0.4 mm focal spot

• Be window

• water cooled heat exchanger

• high voltage cables

• grounding rod

47.5 mm


X-ray detector

• Solid-state Cs(I)

• TFT detector

• 1x1: 2048 x 1536

• 0.194 mm pixels

• 60 frames per second

• 2x2: 1024 x 768

• 0.388 mm pixels


• 2x4: 1024 x 384

• Dual Gain Mode



Gantry motion

• motor control (angle & motion)

• interface with x-ray

• interface with detector

• support x-ray tube

• support detector

• precise angular control

Kollmorgen Servo Motor

DDR D081M

13.0 ft-lb continuous torque

32.0 ft-lb peak torque


Patient support

• provide comfortable support for patient

• provide close access to chest wall

• support weight of large patients


Data acquisition and analysis

• acquire images through 2! geometry

• incorporate system calibration parameters

• correct images errors

• non-linearity

• pixel drop-out

• perform weighted back-projection algorithm

• correct slices to CT numbers

• provide image orientation

• provide image position


stationary stand

Pb shot in foam board

BB tracker

path of BBs in sinogram

BB paths are fit to analytical functions

that define CT geometry

central ray

rotation of detector



768 x 1024 pixels500 views

512 x 512 pixels300 slices ~(0.5 mm)

Reconstruction Based on Feldkamp Algorithm(written in C)


Subjective Assessment

resolution

noise

uniformity

distortion


The Broccoli Test


Breast CT voxels are 16x Smaller Than Mammography

0.10 mm x 0.10 mm x 50 mm

= 0.50 mm3

mammography breast CT

0.25 mm x 0.25 mm x 0.50 mm

= 0.03 mm3


E x p o s u re (m A )

Noise

0 . 2 6 0

0 . 1 3 3

0 . 0 9 60 . 0 8 1

0 . 0 5 6

0 . 0 4 4

0 . 0 3 6

0 . 5 1 1 .5 2 3 . 5 5 7

Air Scans

Noise Assessment

Noise


70 µm nickel-chromium wire

Spatial Resolution

detector pixel size

geometric magnification


Spatial Resolution

Sp a tia l Fre q u e n c y (1 /m m )

0.0 0.2 0.4 0.6 0.8 1 .0 1 .2 1 .4

MT

F(f)

0.0

0.2

0.4

0.6

0.8

1 .0

1 .2

4 cm8 cm IC

1000 views recon to 5122 matrix

influence of FOV position

Shepp-Logan Filter


Spatial Resolution

1000 viewsinfluence of reconstruction matrix

S p a tia l F re q u e n c y (1 /m m )

0.0 0.2 0.4 0.6 0.8 1 .0 1 .2 1 .4 1 .6 1 .8 2.0 2.2

MT

F(f)

0.0

0.2

0.4

0.6

0.8

1 .0

1 .2

5122

8002

10242

Shepp-Logan Filter


Visualization

• slices

• rendering

• interactivity


CTA0302


CTA0296 - improved structural detail

mammogram


CTA0316

Implant

expanded views from adjacent slices

axial coronalsagittal

MLO CC


CTA0316 - implant + cancer

Implant

expanded views from adjacent slices

axial coronalsagittal

MLO CC

µ-calc


CTA0415


CTA0415 - DCIS


Visualization

• slices

• volume rendering

• interactivity




• Mammography is “gold standard” for breast cancer diagnosis but

uncomfortable and uses ionizing radiation

• Ultrasound is diagnostically useful in dense breasts

• poor standardization limits clinical use

• ultrasound provides valuable biopsy sampling guidance

• Most cancers arise in dense ductal tissue

• ultrasound performs better than mammography

• Small lesions (~3-5 mm) are difficult to biopsy

38


Design Strategy:

• Image pendant breast

• Use clinical US scanner with high frequency probe

• Stabilize breast without compression

• Automate scan acquisition

• Integrate with breast CT scanner

Performance Parameters Evaluated:

• Spatial and contrast resolution

• Image acquisition time

• Image compounding and speckle reduction

• Volumetric data display

• Evaluate series of test objects

• Evaluate volunteers


VBUS Master Control

Image Display

Image Correction

Image Reconstruction

Image Control

Image Storage

US Scanner Setup

Image Acquisition

Acquisition Control

Breast Stabilization

Gantry Rotation

US Probe Positioning

Mechanical Control


Coupling Fluid

Breast Stablizing Cup

Gantry Rotation

Imaging Transducer Gantry

Rotation


Imaging Transducer

Coupling Fluid


Motor Control, Acquisition &

Reconstruction Computers

Rotation and

Elevation Motors

Scanning Tank,

Laser Alignment

and TransducerWater Reservoir, Pumps

and Control Valves

Pendant Breast Access PortPatient Cushion

Reflector,Transducer, Laser Alignment

and Scanning Tank,

Design Strategy:

• Image pendant breast

• Use clinical US scanner

• Stabilize breast without compression

• Automate scan acquisition

• Volume data review on workstation

Performance Summary:

• Scan acquisition fully automated

• Scan time 20 sec per slice; slice reconstruction time < 100 ms

• Pendant geometry visualizes entire breast

• Images compare favorably with breast CT


Results:

• Scan acquisition fully automated

• Scan time 18 sec at 20°/sec

• Pendant geometry provides good visualization of entire

breast

• Slice reconstruction time < 100 ms using GPU

• Volume reconstruction time < 5 sec using GPU

• Slice images compare favorably with Breast CT

• Spatial resolution ~ 1 mm

• Volume breast data reviewed on volume workstation


Gantry Rotation


Imaging Transducer

Coupling Fluid

1 2

4 8 300

Original Scan Opposing Views Reconstructed Slice


1 cm grid of 2 mm spaghetti in gelatin


Original Scan ROI Reconstructed Slice

1 cm grid of 2 mm spaghetti in gelatin


Horizontal ScanVertical Scan

Horizontal

Vertical


Original Scan Profiles & MTF


Original US Scan VBUS Scan

Gelatin Test Object

Low Contrast Test Objects


Cystic MassesCystic Masses

Original US Scan Breast CTVBUS Scan

Breast Test Object (051, CIRS, Norfolk, VA)

VBUS Comparison to Breast CT


VBUS Scan

Urethane (CIRS, Norfolk, VA) Test Object (courtesy Dr. M. Andre)

High US attenuation (-10 dB)

High US attenuation (-5 dB)

3 mm

8 mm

5 mm

2 mm

5 mm

5 mm

Low Contrast Test Objects


bCTVBUS - HStandard VBUS - V

Volume Breast Ultrasound


53

CC CCMLO MLO

Fatty Replacement Breast Fibro-Glandular Breast

Volunteer Scans- Volume Acquisition (~3 mm spacing)


54Breast Test Objects

Solidified Mineral Oil Breast Test Objects:

• sound speed ~1460 m/s

• attenuation modified by addition of corn starch

• lesions created by H2O bubble injection

• lesion sizes range from 2-15 mm

• skin simulated by rubber compound

• can be imaged by mammography (with compression), CT and ultrasound


55

VBUS VBUSCTCTMammo

Gel Breast Test Object Modality Comparison


Fibro-Glandular Breast with TumorMRI

(sagittal)Breast CT

(sagittal)

Breast CT(axial)

chest wall

Mammo Volume SlicesBreast CT Slice

VBUS SliceLCC

nipple

P0003 (VBUS) - P0133 (bCT)


Volume Slices Volume Slices

chest wall

nipple

chest wall

nipple

P0003 (VBUS) - P0133 (bCT)

OriginalDeconvolution

(Gaussian and point-spread-function)





Fat

Gland

Skin

Analysis

• segmentation & classification

• histogram analysis

• region growing

• quantification

• skin

• fat

• gland


60

Fat Gland Skin

CTA0321 - classify & segment


61

Original Segmented


62


63


64


65


66

Original Segmented




Force

Stationary

Primary challenges to simulate mechanical compression of the breast:

• describe the mechanical properties of

breast tissue• tissues present • proper classification • mechanical coefficients for each tissue

• select the appropriate numerical model

• describe the contact between the• compression paddle and the breast skin• mammography unit and the breast skin


Numerical Compression Methods:

• Evaluated various approaches to compression

• Forward approach – Finite Element Analysis

• Inverse approach – registration / transform

F


Numerical Compression Methods: Finite Element Analysis

• Challenges

• Mesh generation

• Mesh complexity

• Computational time

• Benefits

• Accurate simulation

• Results may be generalized for rapid computation

• Physical parameters may be extracted from solution


Methods

• Finite Element Analysis

• Challenges

• Computational time

• Mesh generation

• Mesh complexity

• Use techniques of Garboczi et.al. to solve the equations in a random linear elastic material, subject to an applied macroscopic strain, using the finite element method.


Sphere shell (Young’s Modulus - 2 kPa - Medium 2 kPa)

Sphere shell (Young’s Modulus - 90 kPa Meduim 2 kPa)

30507090100

30507090100

Simple Compression - Sphere Shell


Breast(segmented) (Young’s Modulus - skin 90 kPa,

Gland 10 kPa, Fat 1 kPa, Meduim 1 kPa)

50607080100

Simple Compression - Breast (segmented)


Breast segmented Young’s Modulus


• Surface is fit by tetrahedral elements

• Used voxel data threshold value

• Mesh is generated by marching tetrahedra algorithm– http://en.wikipedia.org/wiki/Marching_tetrahedrons

– G. M. Treece, R. W. Prager, and A. H. Gee. Regularised marching tetrahedra: improved iso-surface extraction. Computers and Graphics, 23(4):583--598, 1999


• Interior treated as incompressible fluid• volume preserved

• Chest wall vertices remain in a plane

• Impose various constraints

• Pointwise:

• Fixed vertices

• Level set constraints

• One-sided constraints

• nonpositive or non negative

vertex function

• Global:

• Volume constraint

• Apply compression


• Compression plates can be simulated in different ways

• Top: entire breast is compressed

• Bottom: major portion of breast compressed


• Surface point displacements

• Known from mesh surface vertex points

• Interior point displacements

• Inferred by interpolation from mesh surface vertex points and their compression displacement

?


• (IWD Interpolation).

• One of the most commonly used methods to interpolate a series of scatter points.

• Assumption: interpolated value is most influenced by nearby points and less by more distant points.

• The simplest form of IWD Interpolation is "Shepard's method"

D. Shepard, A two-dimensional interpolating function for irregularly

spaced data. Proc. ACM. nat. Conf., 517--524, 1968


n: number of scatter

points in the set

fi: prescribed function

values at scatter pointswi: weight functions assigned to

each scatter point

di: distance from scatter point to

interpolation point

Interpolated value

Interpolation point location

p: real number > 1.0 (here best

results with p=4.0)


• Mesh generation• Marching tetrahedrons

• 1283 voxel volume input• Triangles in mesh

• Vertices: 1620 Edges: 4854 Facets: 3236

• Interpolation• 1620 surface points and displacements

• Compression Computation Time• Surface Mesh Generation

• 0.04 sec

• Surface Compression• 1.15 sec

• Voxel Interpolation• 63.13 sec (1283)• 504.3 sec (2563)


before compression after compression (CC)





85

VBUS VBUSCTCTMammo

Gel Breast Test Object Modality Comparison


Sound speed and attenuation:• Measure time delay (spatial shift) of reflections from back wall

• delay represents relative change in speed from baseline as sound propagates through different materials

• Time delay converted into per view time delay line

• no delay equals baseline sound speed

• shift towards transducer represents shorter time delay

• shift away from transducer represents longer time delay

time delay = 2 * x / v

x = distance (m)

v = speed of sound assumed

by US machine (m/s)

x x

86


Sound speed and attenuation:

• Use filtered back-projection to construct sound speed image

• use delays from 360o of acquisition

• Mathematical considerations: • Back wall deviations give us time delay

• Time delay is a “projection” of image “slowness”

• Use filtered back projection to reconstruct slowness image

• Invert slowness image to get sound speed image

time_delay(x,!) = 2 * slowness(x, y)dy0

D

"

time_delay(x,!) =back _wall _ shift(x,!) + D

SOUND _SPEED

slowness(x, y) = time_delay(x cos! + ysin! " t)0

w

#0

2$

# dtd!

speed(x, y) =1

slowness(x, y)

87


reflection speed-of-sound attenuation

Sound Speed and Attenuation


89

Parametric Breast Ultrasound Imaging - Reflectivity, Sound Speed and Attenuation

reflection

speed attenuation

composite


90

Breast Test Object Scans- Vascular Flow




• Improved detection and biopsy of small lesions is essential for improving

breast cancer diagnosis, management and survival.

• Minimally invasive robotic devices potentially can assist physicians

perform more precise biopsies thereby improving breast diagnosis and

management.

• To design and construct a dedicated Volume Breast Biopsy System

(VBBS) integrating volume ultrasound images with a compact robotic

biopsy device to provide more precise image-guided breast lesion biopsy

capability.

92


• The Volume Breast UltraSound (VBUS) system is

integrated with a compact robotic device having a 6-DOF

articulated arm to reach any breast location within ±1.0

mm carrying up to a 3.0 kg load.

• A load sensor measured force (Fx, Fy, Fz), and torque (Tx, Ty,

Tz) providing real-time data for biopsy device insertion forces.

• A linear actuator was used for final device insertion

• Volume US or CT data provided 3-dimensional lesion

coordinates.

• Targeting and guidance algorithms optimized the path for

insertion of a Mammotome™ vacuum biopsy device.

• Physician guidance was used to direct robot motion and

device insertion.

93


94

Volume Breast Biopsy System

Robot Arm

Stabilization Support Arm

Video Camera

Breast

Stabilization

Device

Breast Test Object

MammotomeTM

Biopsy System

Force/Torque Sensor

Patient Table

Biopsy Device Insertion Drive

System

Biopsy Needle

VBUS(axial)

chest wall

nipple

VBUS Slices (axial)

Biopsy Device Table and Robot


• System performance was evaluated by scanning a variety of

breast test objects with simulated lesions in a pendant breast

position.

• We measured targeting accuracy and reproducibility in:

• air using acrylic spheres (3, 6, 9, 12, and 15 mm)

• solidified mineral oil based breast test objects with (lesion

sizes 2-15 mm).

95

courtesy of Ron Truong - MAE 156


96

Detected Property Range of SensitivityRange of SensitivityForce (X-direction) 0 to 15 pounds 0 to 67 Newtons

Force (Y-direction) 0 to 15 pounds 0 to 67 Newtons

Force (Z-direction) 0 to 50 pounds 0 to 222 Newtons

Torque (X-direction) 0 to 50 in-pounds 0 to 5650 Newton-mm

Torque (Y-direction) 0 to 50 in-pounds 0 to 5650 Newton-mm

Torque (Z-direction) 0 to 50 in-pounds 0 to 5650 Newton-mm


97

10 mm 1 mm

Robotic Targeting Accuracy Motion Path


98

10 mm 1 mm

x displacement error (mm) y

0.6 average absolute value 0.6

0.5 standard deviation 0.5

1.9 maximum 2.6

0.0 minimum 0.0

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

4

0 10 20 30 40 50 60 70 80

radia

l er

ror

(mm

)

radius (mm)

+/- 1 mm


99

Multi-modality Imaging of Fibro-Glandular Breast with Tumor

MRI (sagittal)

Breast CT(sagittal)

LCC

MammoVolume Slices

(axial)

Breast US(axial)

Breast CT(axial)


100


• VBUS volume data were acquired in 20 sec/slice (volume < 20 min.)

showing ~1 mm spatial resolution with lesions clearly identified.

• Lesion targeting and guidance algorithms showed insertion trajectory

prior to device motion.

• Initial positioning located the device adjacent to skin surface;

• insertion was under physician guidance.

• Lesion targeting accuracy was to within ±1 mm.

• Reproducibility was excellent.

• Gel test objects provided force feedback data regarding object

deformation to improve targeting small lesions.

101


• Volume ultrasound imaging improved lesion localization

• assisting biopsy device position guidance, especially for small lesions.

• Robotic devices may provide more precise device placement assisting

physicians with biopsy procedures.

• This work demonstrates the potential to translate the capabilities of

two rapidly developing areas of medicine:

• volumetric imaging devices

• particularly automated volume ultrasound scanners

• robotic devices

• using compact and economical sizes

into a fully-functional clinical volume image-guided, physician-directed

precision breast biopsy system.

102


Strategic Organization of Image Processing

CREDITS:• Philips Ultrasound• GE Medical Systems• Siemens Ultrasound• de Korte et.al., Erasmus University Rotterdam

Filtering

Anatomic Reference

Image series

Patient AtlasRegistration

Visualization

Parameter Estimates

Segmentation

Thomas R. Nelson, Ph.D. Professor of Radiology and Bioengineering

UC San Diego [email protected]


strategic organization of image processingpcosman/nelson2010.pdf · 2010-05-13 · uc san diego...

Documents