ee565 advanced image processing copyright xin li 20081 different frameworks for image processing...

EE565 Advanced Image Processing Copyright Xin Li 2008 1

Different Frameworks for Image Processing

Statistical/Stochastic Models:

Wiener’s MMSE estimation (e.g., denoising filtering)

Transform-Based Models:

Fourier/Wavelets transform (e.g., denoising thresholding)

Variational PDE Models:

Evolve image according to local derivative/geometric info,

(e.g. denoising diffusion)

Concepts are related mathematically:

Brownian motion – Fourier Analysis --- Diffusion Equation


PDE-based Image Processing

Image as a surface Image Interpolation

Implication of artifacts on surface area Minimal surface solution via mean curvature

diffusion Image inpainting

Variational formulation Energy minimization solution

Image denoising From linear to nonlinear diffusion Perona-Malik diffusion


PDE-based Image Interpolation*

Bilinear Interpolation

PDE-based post-processing

Low-resolution image

Intermediate result

High-resolution image


Image as a Surface

3D visualizationsingle-edge image

If image can be viewed as a surface, it is then naturalto ask: can we apply geometric tools to process thissurface (or its equivalent image signals)?


Geometric Formulation

Image I: R2→R may be viewed as a two-dimensional surface in three-dimensional space, i.e.,

3)),(,,(),(: RyxIyxyxS

2222

2222222

)1(2)1(

)(

dyIdxdyIIdxI

dyIdxIdydxdIdydxds

yyxx

yx

2

22

1

1,][

yyx

yxx

III

IIIG

dy

dxGdydxds

G: symmetric and positivedefinite matrix


Key Motivation

Why these concepts are useful for image processing? Image surface containing artifacts do

not have minimal surface

dxdyIIdxdyGMS yx221)det()(

minimize S(M) leads toEuler-Lagrange Equation:

011 2222

yx

y

yx

x

II

I

yII

I

x (A)


Minimal Surface

221

)1,,(

||||),(

yx

yx

yx

yx

II

II

SS

SSyxN

Unit normal of this surface is

Mean curvature is

2/322

22

)1(2

)1(2)1(),(

yx

xyyxyyxyxx

II

IIIIIIIyxH

TheoremSurfaces of zero mean curvature have minimal areas

0)1(2

)1(2)1(2/322

22

yx

xyyxyyxyxx

II

IIIIIII (B)

Exercise: Derive (B) from (A) by direct calculation


Mean Curvature Diffusion

2/322

22

)1(2

)1(2)1(

yx

xyyxyyxyxxt II

IIIIIIII

Diffusion equation

Discrete Implementation

http://www.cmla.ens-cachan.fr/Cmla/Megawave/index.html

NOT straightforward!

Reference: MegaWave 2.0 software

We will discuss more numerical implementation next


Experiment Result

Before post-processing After post-processing


Further Diffusion

After 3 iterations After 10 iterations


Image Inpainting

I

EI

EI Extended inpainting domain

Assumption: inpainting domain is local and does notcontain texture (complimentary to texture-synthesisbased inpainting techniques)

Image example


Total Variation

Key idea: it is L1 instead of L2 norm(minimizing L2 will not preserve edges)

0 50 100 150 200 250 3000

20

40

60

80

100

120

140

160

180

200

0 50 100 150 200 250 300-50

0

50

100

150

200

250

Clean (TV small) noisy (TV large)


Variational Problem Formulation

IEI

EI

dxdyuudxdyuuJ\

2|ˆ|2

|ˆ|)ˆ(min

u Restored image u degraded image

Rational:

The first term describes the smoothness constraintwithin the extend inpainting domain

The second term describes the observation constraint

Total variation (TV)


How to obtain the corresponding PDE?

0)ˆ(|ˆ|

ˆ

uuu

uE

Euler-Lagrangian Equation

Where

I

IEI

Eyx

yx

),(0

\),(

)ˆ(|ˆ|

ˆˆuu

u

u

t

uE

TV-inpainting:


Numerical Implementation of PDEs

h

nmInmI

x

I

nm 2

),1(),1(

),(

m

n n+1n-1

m+1

m-1

)2

)1,1()1,1(2

)1,1()1,1((

2

1

2

),1(),1(

),(

h

nmInmIh

nmInmI

h

nmInmI

x

I

nm

2),(

),(4)1,()1,(),1(),1(

h

nmInmInmInmInmII

nm

2

2),(

),(4)1,1()1,1()1,1()1,1()1(

),(4)1,()1,(),1(),1(

h

nmInmInmInmInmIh

nmInmInmInmInmII

nm

or

or


Inpainting Example

(Courtesy: Jackie Shen, UMN MATH)


Geometry-driven PDEs

x

y I(x,y)

image I

image I viewed as a 3D surface (x,y,I(x,y))


Simplest Case: Laplace Equation

2

2

2

2 ),,(),,(),,(

),,(

y

tyxI

x

tyxItyxI

t

tyxI

Linear Heat Flow Equation:

)()0,,(),,( tGyxItyxI

scale A Gaussian filterwith zero mean and variance of t

Isotropic diffusion:


Example

t=0 t=1 t=2


Example (Cont.)

t=4 t=8 t=16


From Isotropic to Anisotropic

Gaussian filtering (isotropic diffusion) could remove noise but it would blur images as well

Ideally, we want Filtering (diffusion) within the object

boundaryNo filtering across the edge orientation

How to achieve such “anisotropic diffusion”?


Perona-Malik’s Idea

][),,(

Idivt

tyxI

Isotropic diffusion:

]||)(||[),,(

IIgdivt

tyxI

edge stopping function


Pursuit of Appropriate g

]|)(|[),(

xx IIgdivt

txI

Define

1D case:

xxx IIgI )()(

xxxx II

x

I

t

txI

)('

)(),(

Encourage diffusion: 0)(' xIDiscourage diffusion: 0)(' xI

Edge slope decreases

Edge slope increases


Examples

K

)( xI

xI

22 /)( Kxexg

Choice-I

2)(1

1)(

Kx

xg

Choice-II


Discrete Implementation

IN

Is

IEIW

jijiN III ,,1

jijiS III ,,1

jijiE III ,1,

jijiW III ,1,

][,1

, IcIcIcIcII WWEESSNNt

jit

ji

WESNdIgc dd ,,,||),(||


Numerical Examples

100

100 110

100

100

10 Id

38.3d

dd Ic

125.0,16 K

100

100 200

100

100

100 Id

0d

dd Ic

125.0,16 K

22 /)( Kxexg


Scale-space with Anisotropic Diffusion

original P-M filter (K=16,100 iterations)


P-M Filter for Image Denoising

Noisy image(PSNR=28.13)

P-M filtered image(PSNR=29.83)


Variational Interpretation

x

xxgIIgdiv

t

txIxx

)(')(],|)(|[

),(

dI ||)(||min

]||||

||)(||'[),,(

I

IIdiv

t

tyxI


Comparison between Wavelet -based and PDE-based denoising

Wavelet theory Strength: offers a basis to distinguish

signals from noise (signal behaves as significant coefficients while noise will not)

Weakness: ignore geometry Diffusion theory

Strength: geometry-drivenWeakness: localized model (poor to

realize global trend in the signal)

ee565 advanced image processing copyright xin li 20081 different frameworks for image processing...

Documents