Patch-based Nonlocal Denoising for MRI and Ultrasound Images

Xin Li

Lane Dept. of CSEE

West Virginia University

Outline• I come to see and be seen• Motivation: nonlocal (symmetry-related) dependency in

medical images• Technical Approach

– Patch-based image modeling and geometric resampling – From locally linear embedding (LLE) to locally linear transform

(LLT) – Nonlocal denoising algorithm

• Experimental results– Synthetic images, Gaussian noise– MRI images, Rician noise– Ultrasound images, speckle noise

Big Picture: Computational ImagingQuality

Cost

Physical

Examples: SMASH/SENSE for fast MRISuper-resolution in PET imaging

High-dynamic-range (HDR) imaging

Computational

Motivation: Modeling Human-related Prior

Bilateral symmetry Shape boundary regularity

Patch-based Image Modeling • To overcome the curse of

dimensionality, we have to work at the middle ground between pixel-level and image-level

• An old concept with renewing interest– Vector quantization is

patch-based, JPEG used 8-by-8 patches (SP community)

– Patch-based recognition (CV community)

– Nonlinear dimensionality reduction (ML community)

P

P

Nonparametric: patch-basedvs.

Parametric: wavelet-based

Nonlocal Dependency

reflective symmetry translational symmetry

Beyond the reach of any localized models (MRF, wavelet-based, PDE-based)

Redundant Representation by Geometric Resampling

fliplr(x)x flipud(x) flipud(fliplr(x))

Collection of P-by-P patches

Exploiting Manifold Constraint

B4

B2

B3

B0

B1

RPP

k

tttw

10 BB

Nonlinear Dimensionality ReductionBy Locally Linear Embedding (LLE) Roweis and Saul, Science’2000

0WD

},...,,1{ 1 kwwdiag W],...,,[ 10 kBBBD

Sparsifying transform

t

Artificial third dimension t records the location information

Nonlocal Sparse Representation (NSR)

0FD

Approximated solution(3D FFT/DCT)

0WD

Optimal sparsifyingtransform (KLT)

B0 BkB1 Pack into3D Array D 3D-FFT

Thresholding

…

3D-IFFTPack into

3D Array D

B0 BkB1 …^ ^ ^

NSR Image Denoising Algorithm

Experimental Results on NSR• Computer-generated toy images, additive

White Gaussian noise– Illustrate the algorithm procedure and verify the

benefit of resampling

• MRI images, Rician noise– Benchmark: PDE-based scheme (total-variation

denoising)

• Ultrasound images, speckle noise– Benchmark: local schemes (SRAD, SBF, PDE)

Denoising Procedure Illustration by Toy Example

Noisy image Search similar patchesNoisy 3D array

LLT Thresholding

denoised 3D arrayDenoised image denoised patches

Benefit of Resampling

Translation only

Translation and 1 reflection

Translation and 2 reflections

Translation and 3 reflections

original noisy

NSR (ISNR=17.5dB)GSM (ISNR=13.3dB)

GSM: Gaussian Scalar Mixture in Wavelet space (state-of-the-art denoising scheme)

MRI Image Denoising

original Noisy (Rician, =30)

PDE scheme NSR scheme

Ultrasound Despeckling

Field-IISimulation

SBF(local)

11.2ˆ Q

40.2ˆ Q02.2ˆ Q

NSR (nonlocal) 2

ˆˆˆ NSRSBF xxx

Q̂ Ultrasound Despeckling Assessment Index (USDSAI)**Tay, P.C.; Acton, S.T.; Hossack, J.A., “A stochastic approach to ultrasound Despeckling,”ISBI’2006

Other (Non-medical) Applications of Nonlocal Sparse Representation

original Randomly-sampled(20% data)

RUPScheme*

griddatascheme

EM+NSRscheme

*Candes, E.J.; Romberg, J.; Tao, T., “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency Information,” IEEE Trans. on Infor. Theory, pp. 489- 509, Feb. 2006

Concluding Remarks

• Symmetry – an important piece of prior information about human subjects

• Patch-based models enable us to better distinguish signal (pattern of interest) from noise using the tool of nonlocal sparsity

• Our experiments have shown the effectiveness of such models in a variety of imaging modalities and noise conditions

• Interest in NIH RFP: Innovations in Biomedical Computational Science and Technology (R01)