04/12/10 SIAM Imaging Science 2010 1

Superresolution and Blind Deconvolution

of Images and Video

Institute of Information Theory and AutomationAcademy of Sciences of the Czech RepublicPrague

Filip Šroubek, Jan Flusser, and Michal Šorel

04/12/10 SIAM Imaging Science 2010 2

Traffic surveillance Can we read license plates?

04/12/10 SIAM Imaging Science 2010 3

Empirical observation

• One image is not enough– ill-posed problem

• Solution– strong prior knowledge of blurs and/or the

original imageOR– more images– techniques how to combine them

04/12/10 SIAM Imaging Science 2010 4

Outline

• Mathematical model

• Algorithm

• Examples

• Extension to the space-variant case

04/12/10 SIAM Imaging Science 2010 5

original image

u(x) + nk(x)

+

noise

acquired images

= zk(x)

Multichannel Acquisition Model

channel K

channel 2

channel 1

D[u * hk](x)

04/12/10 SIAM Imaging Science 2010 6

Multichannel Deconvolution

Super-resolution

04/12/10 SIAM Imaging Science 2010 7

Misregistration

04/12/10 SIAM Imaging Science 2010 8

Misregistration

• Incorporating between-image shift

original image PSF degraded image

04/12/10 SIAM Imaging Science 2010 9

Superresolution & Blind Deconv.

• Acquisition model

• Optimization problem

Dataterm

Imageregularization

term

BlurRegularization

term

04/12/10 SIAM Imaging Science 2010 10

Regularization Terms

04/12/10 SIAM Imaging Science 2010 11

0

u

h2*u uh1 *= =z1 z2

z1 h2* *u h1= h2* z2*h1h2 u* =h1 *

PSF Regularization

04/12/10 SIAM Imaging Science 2010 12

• Alternating minimizations of F(u,{hk})

• Input: blurred LR images and

estimation of PSF size

• Output: HR image and PSFs

• Blind deconvolution in the SR framework

AM Algorithm

04/12/10 SIAM Imaging Science 2010 13

Blind Deconvolution

04/12/10 SIAM Imaging Science 2010 14

04/12/10 SIAM Imaging Science 2010 15Superresolved image (2x)

Optical zoom (ground truth)

rough registration

Superresolution

04/12/10 SIAM Imaging Science 2010 16

Space-variant Case

04/12/10 SIAM Imaging Science 2010 17

Space-variant Case

• Video with local motion

• Space-variant PSFs and/or misregistered images

interpolatedSR

interpolatedSR

t t+1 t+2t-2 t-1

interpolatedSR

SR + masking

t t+1 t+2t-2 t-1

04/12/10 SIAM Imaging Science 2010 21

Out-of-focus Blur

04/12/10 SIAM Imaging Science 2010 22

Camera-motion Blur

04/12/10 SIAM Imaging Science 2010 23

Space-variant Superresolution

04/12/10 SIAM Imaging Science 2010 24

04/12/10 SIAM Imaging Science 2010 25

Close-up

Input LR

Space-variantReconstruction

Original

Space-invariantReconstruction

04/12/10 SIAM Imaging Science 2010 26

Misregistered Images

04/12/10 SIAM Imaging Science 2010 27

Misregistered Images - Results

Space-variant Space-invariant

04/12/10 SIAM Imaging Science 2010 28

MATLAB Application

zoi.utia.cas.cz/download

04/12/10 SIAM Imaging Science 2010 29

Thank You for

Your Attention