non-local compressive sampling recovery
DESCRIPTION
Compressive sampling (CS) aims at acquiring a signal at a sampling rate below the Nyquist rate by exploiting prior knowledge that a signal is sparse or correlated in some domain. Despite the remarkable progress in the theory of CS, the sampling rate on a single image required by CS is still very high in practice. In this presentation, a non-local compressive sampling (NLCS) recovery method is proposed to further reduce the sampling rate by exploiting non-local patch correlation and local piecewise smoothness present in natural images. Two non-local sparsity measures, i.e., non-local wavelet sparsity and non-local joint sparsity, are proposed to exploit the patch correlation in NLCS. An efficient iterative algorithm is developed to solve the NLCS recovery problem, which is shown to have stable convergence behavior in experiments. The experimental results show that our NLCS significantly improves the state-of-the-art of image compressive sampling.TRANSCRIPT
Non-Local Compressive Sampling Recovery
Xianbiao Shu1, Jianchao Yang2, Narendra Ahuja1
1{xshu2,n-ahuja}@[email protected]
2 1
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 2
Outline• Introduction to Compressive Sampling (CS)
• Overview of CS recovery methods
• Proposed recovery method----Non-Local CS (NLCS)
• Experimental results
• Summary
ICCP2014
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 3
Introduction• Nyquist Sampling
– Map one object point to one image point (one measurement) – The sampling rate , which is very challenging in many applications:
• To reduce the sampling rate– Low-Resolution Sampling (LRS), but it suffers loss of information– Compressive Sampling (CS)[Donoho06]
ICCP2014
Heavy encoding burden before transmission
Long sensing timein medical imaging (e.g. MRI)
High-resolution sensor in Infrared thermal imaging
Introduction
– If , then this L0-norm problem has unique solution.
– If , then L0-norm solution = L1-norm solution.
Random matrix:
ICCP2014 Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 4
Restricted Isometry Property [Candes08]
If there exists , s.t.
holds for all -sparse signals
Then, this sampling matrix satisfies RIP condition at isometry constant
Problem Formulation
where is -sparse signal is sampled data is random sampling matrix
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 5
Image Compressive Sampling
• Sensing – Random sampling in single-pixel camera [Duarte08]
multiple the image with a 2D random matrix
exactly satisfies RIP condition but costly– Fourier sampling in MRI [Lustig07]
also satisfies RIP condition and efficient– Circulant sampling [Romberg09][Yin10] convolute the image with a random kernel
efficient and the same performance as random sampling
we use circulant sampling in this paper.
ICCP2014
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 6
Image Compressive Sampling• 2DCS (state-of-the-art recovery) [Lustig07][Duarte08][Ma08]
– Sparsity in -transform domain, e.g. wavelet sparsity– Total Variation (TV), i.e., gradient sparsity
It still requires a high sampling rate for real-word images.
• RIP condition:
When sampling rate r=M/N=25%
ICCP2014
2DCS (30.04dB) LRS (26.29dB)
Whichone
isbetter?
Cameraman
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 7
Proposed Method• Spatial correlation in CS, which is motivated by
– Temporal correlation in video CS [Shu11]
– Non-local mean methods (BM3D) [Egiazarian07] in image restoration
• Non-local CS (NLCS)
denotes an image
denotes n patch groups
contains the patch coordinates in i-th group
is the 3D stack of patches in i-th group
– L1-norm based TV[Shu10][Li10]
– Non-Local Sparsity (NLS).
patch correlation within the same group
wavelet sparsity to keep image sharpness
ICCP2014
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 8
• Non-Local Sparsity (NLS) measures– Non-Local Wavelet Sparsity (NLWS): sum of 3D wavelet sparsity
– Non-Local Joint Sparsity (NLJS): sum of Joint Sparsity (JS) [Baron05]:
Proposed Method
ICCP2014
NLWS
NLJS
2D wavelet
Common patch
Sparse errorstack
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 9
Iterative Algorithm• “Chicken-and-egg” problem: recover image and • Two iterative steps:
– Non-local grouping– Non-local recovery
ICCP2014
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 10
Non-Local Grouping• Patch grouping on image
– Divide the image into reference patches– For each , search for up to best matched patches in its neighborhood
ICCP2014
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 11
• Non-local joint sparsity (NLJS)
• Lagrangian Function auxiliary variables
Lagrangian multipliers
• Alternating-Direction-Method-of-Multipliers (ADMM) [Boyd11]
Non-Local Recovery Algorithm
ICCP2014
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 12
• Lagrangian Function
• Three iterative steps of ADMM– Estimate of auxiliary variable
– Joint reconstruction of image
– Update Lagragian multipliers
Non-Local Recovery Algorithm
ICCP2014
B
g1
g2
I
R
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 13
Experimental Results• Barbara (r=20%) (PSNR)
ICCP2014
Ground truth
NLJS_ideal(32.12dB)
2DCS (24.85dB) NLWS (28.06dB) NLJS (30.60dB)
2DCS error map NLWS error map NLJS error map
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 14
Experimental Results
ICCP2014
1 2 3 4 5 6 7 8
NLWS NLJS
SamplingRate r=20%
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 15
Experimental Results
ICCP2014
1 2 3 4 5 6 7 8
On average, NLWS-2DCS = 2.56dB NLJS-2DCS = 3.80dB
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 16
Cameraman(r =20%)
ICCP2014
Ground truth 2DCS (28.52dB)
NLWS (30.10dB) NLJS (31.02dB)
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 17
Cameraman(r =25%)
ICCP2014
Ground truth 2DCS (30.04dB)
NLWS (30.10dB) NLJS (32.32dB)LRS (26.29dB)
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 18
Brain (r =20%)
ICCP2014
Ground truth 2DCS (28.54dB)
NLWS (29.83dB) NLJS (30.91dB)
Ground truth
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 19
Train Station (r =10%)
ICCP2014
Ground truth 2DCS (29.33dB)
NLWS (34.35dB) NLJS (36.07dB)
Required sampling rate=10%
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 20
Summary• Non-Local Compressive Sampling (NLCS) recovery
– NLCS exploits a new prior knowledge---nonlocal spatial correlation,
in addition to conventional local piecewise smoothness and wavelet sparsity– Two non-local sparsity measures (NLJS >NLWS) is proposed to seek spatial
correlation in compressive sampling.– An efficient algorithm is given to recover the image and its patch grouping in
NLCS by iterating between non-local grouping and non-local recovery.– NLCS significantly improves the recovery quality and reduces the sampling rate
to a practical level (e.g. 10%)
ICCP2014
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 21
Sources/Modalities Used in My Research• Dell WorkStation• Matlab Simulation • MRI dataset• Common test images
ICCP2014
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 22
Reference• [Baron05]: D. Baron, M. B.Wakin, M. F. Duarte, S. Sarvotham, and R. G. Baraniuk. Distributed compressed sensing. 2005
• [Donoho06]: D. Donoho. Compressed sensing. IEEE Transaction on Information Theory, 2006.
• [Egiazarian07]: K. Egiazarian, A. Foi, and V. Katkovnik. Compressed sensing image reconstruction via recursive spatially adaptive filtering. In ICIP, 2007.
• [Lustig07]: M. Lustig, D. Donoho, J. Santos, and J. Pauly. Compressed sensing MRI. IEEE Sig. Proc. Magazine, 2007.
• [Candes08]: E. J Candes. The restricted isometry property and its implications for compressed sensing. Comptes Rendus Mathematique 346.9 (2008): 589-592.
• [Duarte08]: M. F. Duarte, M. A. Davenport, D. Takhar, and et al. Single-pixel imaging via compressive sampling. IEEE Signal Processing Magazine, 25(2):83–91, 2008.
• [Ma08]: S. Ma, W. Yin, Y. Zhang, and A. Chakraborty. An efficient algorithm for compressed MR imaging using total variation and wavelets. In CVPR, 2008.
• [Romberg09]: J. Romberg. Compressive sensing by random convolution. SIAM Journal on Imaging Science, 2009.
• [Li10]: C. Li, W. Yin, and Y. Zhang. TVAL3: TV minimization by Augmented Lagrangian and ALternating direction Algorithms, 2010.
• [Shu10]: X. Shu and N. Ahuja. Hybrid compressive sampling via a new total variation TVL1. In ECCV, 2010.
• [Shu11]: X. Shu and N. Ahuja. Imaging via three-dimensional compressive sampling (3DCS). In ICCV, 2011.
• [Boyd11]: S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1):1122, 2011.
• [Yin10] W. Yin, S. P. Morgan, J. Yang, and Y. Zhang. Practical compressive sensing with toeplitz and circulant matrices. Rice University CAAM Technical Report TR10-01, 2010.
ICCP2014
Non-Local Compressive Sampling Recovery X. Shu, J. Yang, N. Ahuja 23
Thanks
And
Questions?
ICCP2014