1 removing camera shake from a single photograph rob fergus, barun singh, aaron hertzmann, sam t....
TRANSCRIPT
1
Removing Camera Shake from a Single Photograph
Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis and William T. Freeman
ACM SIGGRAPH 2006, Boston, USA
2
Outline
Overview
Proposed method for camera shake removing
Simulation results
3
Image blur due to camera shake
Desired image Degraded image
4
Image formation process
=
Blurry image Sharp image Blur kernel
Input to algorithm Desired output
Convolutionoperator
+
Gaussian noise
5
Image formation process
A linear imaging model is assumed in this paper, that is:
where
: Desired image
: Observation
: Blur kernel
: Gassian noise
Posterior probability :
( , | )} ( | , ) ( , ) ( | , ) ( ) ( )
B K L N
L
B
K
N
p L K B p B L K p L K p B L K p L p K
Prior for blur kernel
Prior for image
Observation model
6
Prior model for nature image (1)
Characteristic distribution of sharp image with heavy tails
L
Histogram of image gradient ( )L i
7
Prior model for nature image (2)
Use parametric model for sharp image statistics
L
1
Mixture of Gaussian model for the gradient at pixel i :
( ( )) ( ( ) | 0, )C
c cc
p L i N L i v
1
Pdf for image gradients :
( ) ( ( ) | 0, )C
c cci
p L N L i v
8
Prior model for blur kernel (1)
The characteristics of blur kernel are positive and sparse
=
Blurry image Sharp imageBlur
kernel
9
Prior model for blur kernel (2)
Assume the probability distribution of the element of blur kernel is the mixture of exponential distributions
Exponential distribution
1
1
Mixture of Exponential model
for kernel element:
( ) ( | )
Pdf for blur kernel:
( ) ( | )
D
j d j dd
D
d j ddj
p K E K
p K E K
10
Model transformation
The imaging model need to be transformed before we using the image gradient prior, that is:
2
11
Original formulation:
( , | ) ( | , ) ( ) ( )
Modified formulation:
'
( | , )
{ ( ( ) | (
( )
{
( ( )
{ ( ( ) | 0, )}
, | )
), ( | )= ) *}C
c ccii
D
d j dd
B K L N
p L K B p B L K p L p K
B K L N
p p p K
E K
B L K L
N L i
p L K
N B i K v
B
L i
} j
11
Variational Baye (1)
Illustration for Bayesian mean squared error estimator (Minimum mean squared error estimator, MMSE)
Ө1
Ө2
Ө3Process
Parameter space Ө
d
Observed data
1 1 2 2 3 3
MMSE for (Weighted aveage):
* ( | ) * ( | ) * ( | )p d p d p d
12
Variational Baye (2)
Apply MMSE estimator for blur kernel estimation
K
K
MMSE for Blur kernel:
' * ( | )
{ ( , | ) }
dd dL
K K p K B
K p L K B
It may be difficult to find the integration result of the posterior probability
13
Variational Baye (3)
Factorize the previous posterior probability for blur kernel inference operation
( , | )p L K B
K K
K K ( )
The factorization of posterior probability:
( , | ) ( ) ( )
MMSE for Blur kernel now becomes:
' * ( | ) { ( , | ) }
{ ( ) ( ) } ( )
d d dd d d
L
L q K
p L K B q K q L
K K p K B K p L K B
K q K q L Kq K K
14
Variational Baye (4)
The factorization of the posterior probability could be modeled as an optimization problem
, , ,
, ,
Assume
( ) ~ ( ; , )
( ) ~ ( ; , )
where
, , , are unknown parameters
Distribution approximation:
', ', ', ' arg min Distance( ( ) ( ), ( , | ))
arg minK K L L
K K
L L
rectified K K
L L K K
K K L L m v m v
m v m
q L N L m v
q K N K m v
m v m v
m v m v q L q K p L K B
,
_ ( ( ) ( ), ( , | ))L LvKL divergence q L q K p L K B
15
Original photograph
16
Blur kernel
Proposed method
17
Original photograph
Matlab’s deconvblind
18
Original photograph
19
Matlab’s deconvblind
20
Proposed method Blur kernel
21
Original photograph
22
Proposed method
Blur kernel
23
Original photograph Proposed method