BM3D-AMP: A New Image

Recovery Algorithm Based on

BM3D Denoising

Chris Metzler, Richard Baraniuk

Rice University

Arian Maleki

Columbia University

Compressive Sensing Problem

Solution: Assume Structure

• Sparse: OMP [Tropp 04], IST [Figueiredo et al 07], AMP [Donoho et al 09]

• Minimal Total Variation: TVAL3 [Li et al 09]

• Tree-Sparse: Model-CoSaMP [Baraniuk et al 10],Turbo-AMP [Som and Schniter 12]

• Group-Sparse: NLR-CS [Dong et al 14]

Using Structure is Hard

• Write as penalty or constraint

• How to efficiently solve for non-convex R(x)?

• What is R(x) for natural images?

• What is R(x) for RF, microscopy, and other applications?

This Talk

• Develop algorithm that can easily use almost any structure

• Predict performance with accurate state evolution

• Derive theoretical guarantees• Measurements required

• Robust to noise

• Demonstrate state-of-the-art performance• 10dB better than wavelet sparsity

• 50x faster than group-sparsity

Insight: Denoisers Use Structure

• Gaussian Kernel• Smooth

• Soft Wavelet Thresholding [Donoho and Johnstone 94]

• Sparse wavelet representation

• BLS-GSM [Portilla et al 03]

• Coefficients follow Gaussian Mixture Model

• NLM [Baudes et al 05]

• Correlated structures

• BM3D [Dabov et al 07]

• Group-sparse in DCT/Wavelet representation

• BM3D-SAPCA [Dabov et al 09]

• Group-sparse in adaptive basis

Denoisers as Black Boxes

Denoiser

Denoisers as Projections

C

Naïve Algorithm: Denoising-based

Iterative Thresholding (D-IT)

Naïve Algorithm: D-IT

Naïve Algorithm: D-IT

Naïve Algorithm: D-IT

Naïve Algorithm: D-IT

Naïve Algorithm: D-IT

Failure of D-IT: Systematic Errors

Systematic Errors: Overshooting

Systematic Errors: Overshooting

Too High

Systematic Errors: Overshooting

Too High Too Low

Too High Too Low

Systematic Errors: Overshooting

Too High Too Low

Too High Too Low

Systematic Errors: Overshooting

Too High Too Low

Too High Too Low

Systematic Errors: Overshooting

Too High Too Low

Too High Too Low

Use residuals from

previous iterations to

avoid overshooting

New Algorithm: D-AMP

Onsager Correction

D-AMP Benefits

D-IT

• Updates proportional to residual (P controller).

• 5dB improvement over L1

D-AMP

• Updates use rate residuals change (PD controller).

• 10dB improvement over L1

(state-of-the-art)

• Onsager Correction

Onsager Correction:

• Where did it come from?• Approximation of message passing algorithm

Onsager Correction:

• Where did it come from?• Approximation of message passing algorithm

• Why does it help?• zt stores residuals over many iterations (momentum)

• Corrects for bias in denoiser solutions

• Makes errors uncorrelated (Gaussian) and thus easy to remove

Onsager Correction:

• Where did it come from?• Approximation of message passing algorithm

• Why does it help?• zt stores residuals over many iterations (momentum)

• Corrects for bias in denoiser solutions

• Makes errors uncorrelated (Gaussian) and thus easy to remove

• How is it calculated?• Approximation from Monte Carlo SURE [Ramani et al 08]

D-AMP Avoids Systematic Errors

D-AMP Theoretical Properties

• Predictable performance

• Denoiser determines phase transition

• Robust to noise

• Easy to tune

State Evolution

• Errors are uncorrelated → Gaussian noise at each iteration

• Variance of noise determines MSE

• State evolution predicts noise variance

State Evolution of D-IT and D-AMP

Main Theoretical Results

• Denoiser Performance

• Phase Transition: Determined by denoiser

• Noise Sensitivity: Graceful failure

Parameter Tuning

• Denoiser parameters

• Problem: Tune denoiser parameters over multiple iterations

• Result: Greedy parameter selection is optimal

3x Under-Sampling

20x Under-Sampling

Wavelet Sparse (L1) BM3D-AMP (our algorithm)

10x Under-Sampling with Noise

NLR-CS BM3D-AMP

Performance without Noise

Computation Time

30x Faster

70x Faster

Performance with Noise

D-AMP Summary

• Arbitrary denoiser• NLM

• BM3D

• Useful state evolution

• State-of-the-art performance

• Resilient to noise

• >97% reduction in average computation time