image deblurring with optimizations qi shan leo jiaya jia aseem agarwala university of washington...
Post on 20-Jan-2016
218 views
TRANSCRIPT
Image Deblurring with Optimizations
Qi ShanQi ShanLeo Jiaya Jia Leo Jiaya Jia Aseem AgarwalaAseem Agarwala
University of WashingtonThe Chinese University of Hong KongAdobe Systems, Inc.
2
The Problem
An Example
4
Previous Work (1)
Hardware solutions:
[Raskar et al. 2006]
[Ben-Ezra and Nayar 2004]
[Levin et al. 2008]
5
Previous Work (2)
Multi-frame solutions:
[Petschnigg et al. 2004]
[Jia et al. 2004] [Rav-Acha and Peleg 2005]
[Yuan et al. 2007]
6
Previous Work (3)
Single image solutions:
[Jia 2007][Fergus et al. 2006]
[Levin et al. 2007]
Most recent work on Single Image Deblurring
Qi Shan, Jiaya Jia, and Aseem AgarwalaHigh-Quality Motion Deblurring From a Single Image. SIGGRAPH 2008
Lu Yuan, Jian Sun, Long Quan and Heung-Yeung ShumProgressive Inter-scale and intra-scale Non-blind Image Deconvolution. SIGGRAPH 2008.
Joshi, N., Szeliski, R. and Kriegman, D. PSF Estimation using Sharp Edge Prediction, CVPR 2008.
A. Levin, Y. Weiss, F. Durand, W. T. Freeman Understanding and evaluating blind deconvolution algorithms. CVPR 2009
Sunghyun Cho and Seungyong Lee, Fast Motion Deblurring.SIGGRAPH ASIA 2009
And many more...
Some take home ideas
1. Using hierarchical approaches to estimate kernel in different scales
2. Realize the importance of strong edges
3. Bilateral filtering to suppress ringing artifacts
4. RL deconvolution is good, but we've got better chioces
5. Stronger prior does a better job
6. Deblurring by assuming spatially variant kernel is a good way to go
Today's topic
How to apply natural image statistics, image local smoothness constraints, and kernel sparsity prior in a MAP process
Short discussion on
1. the stability of a non-blind deconvolution process
2. noise resistant non-blind deconvolution and denoising
10
Image Global Statistics
…
11
…
Image Global Statistics
12
Image Global Statistics
13
LI
Image Local Constraint
14
LI
Image Local Constraint
15
LI
Image Local Constraint
16
LI
12 )( |) ( 0,i ii N dL dIp L
p2
Image Local Constraint
17
exponentially distributed
) ( jfj ep f
Kernel Statistics
18
Combining All constraints
1 2min ( , ) min log[ ( ) ( ) ( ) ( )]E L f p n p dL p L p f
L f n
Two-step iterative optimization• Optimize L• Optimize f
19
Idea: separate convolution
22'( ) \ (|| * || )E L Sum L f I
log ( )p n
Optimize L
1log ( )p dL 2log ( )p L
22 2(\ ( || || ))i i iSum m dL dI
Optimization Process
idLreplace with i
1 1 1|| log ( ) ||p dL
20
22'( ) \ (|| * || )E L Sum L f I
log ( )p n
Idea: separate convolution
Optimize L
1log ( )p dL 2log ( )p L
1 1 1|| lo g ( ) ||p 22 2(\ ( || || ))i iiSum m dI
Optimization Process
idLreplace with i
21
22'( ) \ (|| * || )E L Sum L f I
1 1 1|| log ( ) ||p 22(|| || )dL 2
2 2(\ (|| || ))i iSum dI
22arg min \ (|| * || )opt LL Sum dL f dI 2
2(|| || )dL
Adding a new constraint to makeRemoving terms that are not relevant to
~ dLL
Updating L
An easy quadratic optimization problem with a closed form solution in the frequency domain
22
Updating
Removing terms that are not relevant to
22'( ) \ (|| * || )E Sum L f I
1 1 1|| lo g ( ) ||p 22(|| || )dL 2
2 2(\ (|| || ))i iSum dI
21 1 1 2 2arg min || log ( ) || (\ (|| || ))opt i ip Sum dI
22(|| || )dL
23
each only contains a single variable Ψi'i
E
21 1 1 2 2arg m in || lo g ( ) || (\ ( || || ))op t i ip Sum dI
22(|| || )dL
arg min(\ ( ' ))i
Sum E
It is then a set of easy single variable optimization problems
24
Iteration 0 (initialization)
25
Time: about 30 seconds for an 800x600 image
Iteration 8 (converge)
26
A comparison
RL deconvolution
27
A comparison
Our deconvolution
28
Two-step iterative optimization• Optimize L• Optimize f
22 1( ) || * || || ||E f L f I f
1 2min ( , ) min log[ ( ) ( ) ( ) ( )]E L f p n p dL p L p f
Optimization with a total variation regularization
29
Results
30
Results
31
32
33
More results
34
More results
Today's topic
How to apply natural image statistics, image local smoothness constraints, and kernel sparsity prior in a MAP process
Short discussion on
1. the stability of a non-blind deconvolution process
2. noise resistant non-blind deconvolution and denoising
Stability
Considering the simplest case: Wiener Filtering
T
T
FX B
F F I
How about if *B B n
**
T
T T
F XX X n
F F I F F I
* *T
T
FX B
F FAnd
Stability
Thus
* 2 2 22 2|| || || ||
PX X C
PP
Pwhere is the frequency domain representation of
is the variance of the noise
Observation: the noise in the blur image is magnified in
the deconvolved image. And the Noise Magnification
Factor (NMF) is solely determined by the filter
F2
F
Some examples
Some examples
Dense kernels are less stable for deconvolution than sparse ones
40
Noise resistant deconvolution and denoising
With Jiaya Jia, Singbing Kang and Zenlu QinIn CVPR 2010
Blind and non-blind image deconvolution softwareis available online and will be updated soon!
See you in San Francisco!