Non-local Sparse Models for Image Restoration

Julien Mairal, Francis Bach, Jean Ponce, Guillermo Sapiro and Andrew Zisserman

ICCV 2009

Presented by: Mingyuan ZhouDuke University, ECE

April 09, 2010

Image restoration

Two different approaches to image restoration:

• Dictionary learning for sparse image representation: decomposing each image patch into a linear combination of a few elements from a basis set (dictionary).

• Non-local means approach: explicitly exploiting the self-similarities of natural images.

• Simultaneous sparse coding is proposed as a framework for combining these two approaches in a natural manner, achieved by Jointly decomposing groups of similar signals on subsets of the learned dictionary. It imposes that similar patches share the same dictionary elements in their sparse decomposition.

Representative approaches

• Non-local Mean

• Sparse coding

• Dictionary learning

Dictionary Learning

BM3D

Reference: K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3D transform-domain collaborative filtering,” IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080-2095, August 2007.

Simultaneous Sparse Coding• Sparse coding is too flexible: similar patches

sometimes admit very different estimates due to the potential instability of sparse decompositions

• Constraint: forcing similar patches to admit similar decompositions

Practical Formulation and Implementation

Demosaicking

This is a file from the Wikimedia Commons.

Denoising• SC: sparse coding, use the online dictionary learning

approach to train a global dictionary from 2 × 10^7 natural image patches.

• LSC: learned sparse coding

• LSSC: learned simultaneous sparse coding

Denoising

Demosaicking

Denoising + Demosaicking