old and new algorithms for blind...
TRANSCRIPT
![Page 1: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/1.jpg)
Old and New Algorithms for BlindDeconvolution
Yair Weiss
Hebrew University of Jerusalemjoint work with Anat Levin, WIS
(thanks to Meir Feder, EE, TAU)
![Page 2: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/2.jpg)
Blind Deblurring
y = x * k
![Page 3: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/3.jpg)
Much Recent Progress
(Fergus et al. 06, Cho et al. 07, Jia 07, Joshi et al. 08, Whyte et al.10, Shan et al. 07, Harmeling et al. 10, Sroubek and Milanfar 11 · · · )
![Page 4: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/4.jpg)
MAP using sparse derivative prior
−100 −50 0 50 10010
0
105
dx
log
coun
ts
log P(x , k |y) = C +∑
i
|xi |α + λ‖k ∗ x − y‖2
• MAPxk : (x∗, k∗) = arg max P(x , k |y). Guaranteed to failwith global optimization. Often works well in practice.
• MAPk : (k∗) = arg maxk∫
x P(x , k |y). Guaranteed tosucceed if images sampled from prior. Can be difficult tooptimize.
(Levin et al. 09)
![Page 5: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/5.jpg)
This talk:
• Old algorithms for blind deconvolution from communicationsystems. 6= MAP. Rigorous correctness proofs.
• Can explain success of some MAPx ,k algorithms.
![Page 6: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/6.jpg)
Blind Deconvolution in Communication
• x(t) the transmitted signal.• y(t) =
∑τ kτx(t − τ) received signal.
• y=x * k
![Page 7: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/7.jpg)
Blur makes signals Gaussians (Central Limit Theorem)Orig. Signal (IID) Histogram (κ=1)
−1 −0.5 0 0.5 10
100
200
300
400
500
600
Blurred Histogram (κ = 2.683)
0 1 2 3 4 50
10
20
30
40
50
κ(y) =1N
∑i
y4i , yi = yi/std(y)
(Shalvi and Weinstein 1990)
![Page 8: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/8.jpg)
Blur makes signals Gaussians (Central Limit Theorem)
Orig. Signal (IID) Histogram (κ=26.43)
−4 −2 0 2 40
100
200
300
400
Blurred Histogram (κ = 5.45)
−4 −2 0 2 40
20
40
60
80
100
![Page 9: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/9.jpg)
Simple Blind Deconvolution Algorithm (Shalvi andWeinstein 1980)
Assume xi IID. y = x ∗ k . Solve for the “inverse filter” e suchthat e ∗ y = x .
• xi sub-Gaussian (κ < 3):
e∗ = arg mineκ(e ∗ y)
(Godard 1970)• xi super-Gaussian (κ > 3):
e∗ = arg maxe
κ(e ∗ y)
Claim: Guaranteed to find the correct e: e∗ ∗ y = x . No localminima.
![Page 10: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/10.jpg)
Simple Proof
Assume xi IID. xi super-Gaussian (κ > 3): y = x ∗ k .
e∗ = arg maxe
κ(e ∗ y)
Proof: x = e ∗ y = e ∗ k ∗ x = (e ∗ k) ∗ x⇒ Unless e ∗ k = δ, x is more Gaussian than x so κ(x) < κ(x)
![Page 11: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/11.jpg)
Proof by pictures:Orig. Signal (IID) Histogram (κ=26.43)
−4 −2 0 2 40
100
200
300
400
Blurred Histogram (κ = 5.45)
−4 −2 0 2 40
20
40
60
80
100
deblurred (wrong e) Histogram (κ = 14.14)
−3 −2 −1 0 1 2 30
20
40
60
80
100
120
140
160
180
![Page 12: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/12.jpg)
Blind Deconvolution by Maximizing non Gaussianity
Assume xi IID. xi super-Gaussian. (κ > 3): y = x ∗ k .
e∗ = arg maxe
κ(e ∗ y)
• Universal proof of correcteness (don’t need to know Pr(xi),just IID and sub/super Gaussianity).
• Proofs of global convergence of iterative algorithms.• Used in microwave radio transmissions (1980s), cable
set-top boxes (mid 1990s) and wireless communication(late 1990s-today) (Johnson et al. 1998)
• Can we use this for blind image deblurring?
![Page 13: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/13.jpg)
Blur makes derivatives more Gaussian
Sharp image dx Histogram (κ=20.02)
−100 −50 0 50 10010
0
105
Blurred image dx Histogram (κ = 17.99)
−100 −50 0 50 10010
0
105
blurredsharp
![Page 14: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/14.jpg)
New class of image blind deblurring alglrithms
y : blurred (deriv) image Histogram (κ = 17.99)
−40 −20 0 20 4010
0
105
x sharp (deriv) image Histogram (κ=20.02)
−100 −50 0 50 10010
0
105
Find (x∗, k∗) such that:• y = x∗ ∗ k∗
• x∗ as non Gaussian as possible.If derivatives are IID, this algorithm provably finds the correctblur kernel.
![Page 15: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/15.jpg)
Measuring non Gaussianity using normalizedmoments
1N
∑i
xαi , xi = xi/std(x)
−5 0 5−2.5
−2
−1.5
−1
−0.5
0
x
log
prob
−5 0 5−5
−4
−3
−2
−1
0
x
log
prob
−5 0 5−25
−20
−15
−10
−5
0
x
log
prob
∑i x4
i = 25.73∑
i x4i = 5.8
∑i x4
i = 3∑i x1/2
i = 0.60∑
i x1/2i = 0.74
∑i x1/2
i = 0.81
![Page 16: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/16.jpg)
A new(?) algorithm
y : Blurred image Histogram (κ = 17.99)
−40 −20 0 20 4010
0
105
(x∗, k∗) = arg minx ,k
∑i
|xi |α + λ‖k ∗ x − y‖2
xi = xi/std(x)
Guaranteed to succeed.
![Page 17: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/17.jpg)
MAPx ,k algorithm
y : Blurred image Histogram (κ = 17.99)
−40 −20 0 20 4010
0
105
(x∗, k∗) = arg minx ,k
∑i
|xi |α + λ‖k ∗ x − y‖2
Guaranteed to fail.
![Page 18: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/18.jpg)
Normalized Sparsity (Krishnan et al. 11)
minx ,k
λ‖x ∗ k − y‖2 +‖x‖1‖x‖2
+ Ψ‖k‖1
Can be seen as special case of “new” algorithm.
![Page 19: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/19.jpg)
MAPxk with bilateral filtering (Cho and Lee 09, Hirschet al. 11)
x ← BilateralFilter(x)
k ← arg mink‖y − k ∗ x‖
x ← arg minxλ‖y − k ∗ x‖2 + sparsity(x)
Can be seen as approximating special case of “new” algorithm.
![Page 20: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/20.jpg)
A new(?) algorithm
y : Blurred image Histogram (κ = 17.99)
−40 −20 0 20 4010
0
105
(x∗, k∗) = arg minx ,k
∑i
|xi |α + λ‖k ∗ x − y‖2
xi = xi/std(x)
Guaranteed to succeed.
![Page 21: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/21.jpg)
So what does this buy us?
⇔• Understanding recent algorithms. Proof of success.• Improving recent algorithms. Filters with better
independence properties. Iterative algorithms with globalconvergence.
![Page 22: Old and New Algorithms for Blind Deconvolutionwebdav.is.mpg.de/pixel/workshops/mlmcp-nips2011/sli… · · 2012-01-13Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew](https://reader031.vdocuments.mx/reader031/viewer/2022022502/5aaa98ea7f8b9a6c188e6cf2/html5/thumbnails/22.jpg)
Conclusions
• MAP algorithms for blind image deconvolution. May workeven when not supposed to.
• Old algorithms for blind deconvolution in communications.Universal guarantees, global convergence, used in millionsof devices.
• Can help us understand and improve image deblurringalgorithms.