ee565 advanced image processing copyright xin li 20081 why do we need image model in the first...

EE565 Advanced Image Processing Copyright Xin Li 2008

1

Why do we Need Image Model in the first place? Any image processing algorithm has to

work on a collection (class) of images instead of a single one

Mathematical model gives us the abstraction of common properties of the images within the same class

Model is our hypothesis and images are our observation data In physics, can F=ma explain the

relationship between force and acceleration? In image processing, can this model fit this class of images?


2

Statistical vs. Deterministic They are different languages invented

by mathematicians to facilitate the communication of scientific results (just like English vs. Chinese spoken by people in different countries)

None is better than other – pick up the one you feel most comfortable with

We adopt a statistical language most of the time in this class


3

The Curse of Dimensionality

Even for a small-size image such as 64-by-64, we need to model it by a random process in 4096-dimensional space (R4096) whose covariance matrix is sized by 4096-by-4096

More importantly, we ask ourselves: do we need to consider all pixels simultaneously?


4

A Simple Idea: Locality

The conditional pdf is determined by a local neighborhood

),...,|(),...,|( 111 Nkkkkk XXXPXXXP N past samples


5

Parametric vs. Nonparametric

N

nknknk wXaX

1

non-parametricsampling

Input image

Xk1

2 3 4

5

678

Parametric model


6

Spatial vs. Wavelet


7

Complete vs. Overcomplete


8

Marginal PDF of wavelet coefficients

where

Laplacian

Gaussian

P: shape parameter: variance parameter


9

Joint PDF of Wavelet Coefficients

Neighborhood I(Q): {Left,Up,cousin and aunt}

X=

Y=

Joint pdf of two correlated random variables X and Y

Can you use this model to interpret why EZW works?


10

Good Bad

Spatially Fixed vs. Adaptive Models


11

Locality Revisited

Input image

),...,|(),...,|( 111 Nkkkkk XXXPXXXP N past samples

The definition of local neighborhood has to be relative


12

Application I: Image Denoising

Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering

Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering

From local to nonlocal denoising


13

Linear Frequency Weighting

),(),(

),(),(

),,(),(),(ˆ

2121

2121

212121

wwSwwS

wwSwwH

wwYwwHwwX

WX

X

22

2

,ˆwx

xaaYX

FT

Power spectrum |X|2


14

Spatially Adaptive Wiener Filtering of Wavelet Coefficients

Basic assumption: image source is modeled by a nonstationary Gaussian process

Signal variance is locally estimated from the windowed noisy observation data

]1

,0max[ˆ 2

1

22w

N

iix y

N

T

TN=T2

Recall


15

Wavelet Thresholding

DWT IWTThresholdingY X

~

otherwise

TnYifnYnX

0

|][|][][

~Hard thresholding

Soft thresholding

TnY

TnYTnY

TnYTnY

nX

|][|0

][][

][][

][~

Noisysignal

denoisedsignal


16

Spatially Adaptive Wiener Filtering in Wavelet Domain Wavelet high-band coefficients are

modeled by a Gaussian random variable with zero mean and spatially varying variance

Apply Wiener filtering to wavelet coefficients, i.e.,

][][

][][

~22

2

nYn

nnX

estimated in the same wayas spatial-domain (Slide 15)


17

Translation Invariant Denoising

Noisy image

Tce Tce-1ThresholdingWD =

shift(mK,nK) WD shift(-mK,-nK)

shift(m1,n1) WD shift(-m1,-n1)

Avg

denoised image

(mk,nk): a pair of integers, k=1-K (K: redundancy ratio)


18

Further Improvements Gaussian scalar mixture (GSM) based

denoising (Portilla et al.’ 2003) Instead of estimating the variance, it

explicitly addresses the issue of uncertainty with variance estimation

Hidden Markov Model (HMM) based denoising (Romberg et al.’ 2001) Build a HMM for wavelet high-band

coefficients (refer to the posted paper)


19

Nonlocal Patch-based Denoising

WD

T T-1ThresholdingWD =

Noisy patches Denoised patches


20

Application II: Texture Syntehsis

Spatial-domain models Parametric autoregressive model Nonparametric resampling based

Wavelet-domain models Histogram matching based Parametric models based joint-

statistics


21

Spatial-DomainParametric Texture Synthesis

Gaussian noise image -- w Gabor Filter -- g

Output of Gabor Filter,x=g**w Synthesis Result from AR model,Xn(z)=W(z)/A(z)


22

Nonparametric Texture Synthesis


23

Wavelet-domain Histogram Matching


24

Wavelet-DomainParametric Texture Models

original

synthesized


25

Other Applications

Interpolation Spatial-domain covariance-based models PDE-based (nonlinear diffusion) models

Coding Statistical modeling of wavelet

coefficients Dual to wavelet-based image denoising

Data hiding DCT-domain human vision model


26

Summary on Theory Image models are at the foundation of

any image processing algorithm Statistical models help us deal with the

uncertainty in observation data Appropriate image representation (e.g.,

prediction/transform) facilitates the modeling task

Spatial adaptation is important – to have a good model for a wide class of images

Localized models are popular and powerful but nonlocal models might prevail later


27

Summary on Practice MATLAB provides a user friendly

platform for testing your ideas You can see what you have done

Experimental efficiency is important Avoid loops and test small-size images

C/C++ programming skills are a plus Efficient implementation could make a

difference


28

Beyond Image Processing I will discuss something important than

Wiener filtering or wavelet coding It is about you and your career

If you are a MS student, your master thesis will be your selling point in your job hunting

If you are a PhD student, you need to have a desire for first-class research

It all depends on your perspective - how you want to look at it


29

Where is Your Talent?

Outsider advantage: EZW, Turbo codes, Youtube, …

A tradeoff among mathematical capabilities, physics intuitions, programming skills, management style …

Selling your work could be even more important than doing the work itself


30

Follow your heart and

enjoy what you do!

Final Words

ee565 advanced image processing copyright xin li 20081 why do we need image model in the first...

Documents

image model

image processing algorithm

image source

wavelet slide

class slide

smallsize image

parametric model slide

overcomplete slide