ee565 advanced image processing copyright xin li 2008 1 statistical modeling of natural images in...

EE565 Advanced Image Processing Copyright Xin Li 2008

1

Statistical Modeling of Natural Images in the Wavelet Space

Parametric models of wavelet coefficients Univariate i.i.d. models Spatially adaptive models

Application into texture synthesis Pyramid-based scheme

(Heeger&Bergen’1995) Projection-based scheme

(Portilla&Simoncelli’2000)


2

Recall: Transform Facilitates Modeling

x1

x2 y1y2

x1 and x2 are highly correlated

p(x1x2) p(x1)p(x2)

y1 and y2 are less correlated

p(y1y2) p(y1)p(y2)


3

Empirical Observation

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

100

200

300

400

500

600

700

800

900

1000

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

100

200

300

400

500

600

700

800

900

1000

H1

A single peak at zero


4

Univariate Probability Model

Laplacian:

Gaussian:


5

Gaussian Distribution


6

Laplacian Distribution


7

Statistical Testing

How do we know which parametric model better fits the empirical distribution of wavelet coefficients?

In addition to visual inspection (which is often subjective and less accurate), we can use various statistical testing tools to objectively evaluate the closeness of an empirical cumulative distribution function (ECDF) to the hypothesized one

One of the most widely used techniques is Kolmogorov-Smirnov Test (MATLAB function: >help kstest).


8

Kolmogorov-Smirnov Test*

The K-S test is based on the maximum distance between empirical CDF (ECDF) and hypothesized CDF (e.g., the normal distribution N(0,1)).


9

Example

Usage: [H,P,KS,CV] = KSTEST(X,CDF)If CDF is omitted, it assumes pdf of N(0,1)

x: computer-generated samples(0<P<1, the larger P, the more likely)

Accept the hypothesis

Reject the hypothesis

d: high-band wavelet coefficientsof lena image (note the normalizationby signal variance)


10

Generalized Gaussian/Laplacian Distribution

where

Laplacian

Gaussian

P: shape parameter: variance parameter


11

Model Parameter Estimation*

Maximum Likelihood EstimationMethod of momentsLinear regression method

[1] Sharifi, K. and Leon-Garcia, A. “Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video,”IEEE T-CSVT, No. 1, February 1995, pp. 52-56.

[2] www.cimat.mx/reportes/enlinea/I-01-18_eng.pdf

Ref.


12

I.I.D. Assumption Challenged

If wavelet coefficients of each subband are indeed i.i.d., then random permutation of pixels should produce another image of the same class (natural images)

The fundamental question here: does WT completely decorrelate image signals?


13

Image Example

High-band coefficientspermutation

You can run the MATLAB demo to check this experiment


14

Another Experiment

-4 -3 -2 -1 0 1 2 3 4-4

-3

-2

-1

0

1

2

3

4

5

Joint pdf of two uncorrelated random variables X and Y

X

Y


15

Joint PDF of Wavelet Coefficients

Neighborhood I(Q): {Left,Up,cousin and aunt}

X=

Y=

Joint pdf of two correlated random variables X and Y


16

Heeger&Bergen’1995: Histogram-based

Pyramid-based (using steerable pyramids) Facilitate the statistical modeling

Histogram matching Enforce the first-order statistical constraint

Texture matching Alternate histogram matching in spatial and wavelet

domain

Boundary handling: use periodic extension Color consistency: use color transformation

Basic idea: two visually similar textures will also have similar statistics


17

Histogram MatchingGeneralization of histogram equalization (the target is the histogramof a given image instead of uniform distribution)


18

Histogram Equalization

x

t

thLy0

)(Uniform

Quantization

L

t

th0

1)(Note:

L

1

x

t

ths0

)(

x

L

y

0

cumulative probability function


19

MATLAB Implementation

function y=hist_eq(x)

[M,N]=size(x);for i=1:256 h(i)=sum(sum(x= =i-1));End

y=x;s=sum(h);for i=1:256 I=find(x= =i-1); y(I)=sum(h(1:i))/s*255;end

Calculate the histogramof the input image

Perform histogramequalization


20

Histogram Equalization Example


21

Histogram Specification

ST

S-1*T

histogram1 histogram2

?


22

Texture Matching

Objective: the histogram of both subbands and synthesized image matches the given template

Basic hypothesis: if two texture images visually look similar, then theyhave similar histograms in both spatial and wavelet domain


23

Image Examples


24

Portilla&Simoncelli’2000: Parametric

Instead of matching histogram (nonparametric models), we can buildparametric models for wavelet coefficients and enforce the synthesizedimage to inherit the parameters of given image

Model parameters: 710 parameters were used in Portilla and Simoncelli’s experiment (4 orientations, 4 scales, 77 neighborhood)

Basic idea: two visually similar textures will also have similar statistics


25

Statistical Constraints

Four types of constraints Marginal Statistics Raw coefficient correlation Coefficient magnitude statistics Cross-scale phase statistics

Alternating Projections onto the four constraint sets Projection-onto-convex-set (POCS)


26

Convex Set

A set Ω is said to be convex if for any two point yx ,We have 10,)1( ayaax

Convex set examples

Non-convex set examples


27

Projection Operator

f

g

Projection onto convex set C

C ||}||min||||{ fxfxCxPfgCx

In simple words, the projection of f onto a convex set C is theelement in C that is closest to f in terms of Euclidean distance


28

Alternating Projection

X0

X1

X2

X∞

Projection-Onto-Convex-Set (POCS) Theorem: If C1,…,Ck areconvex sets, then alternating projection P1,…,Pk will convergeto the intersection of C1,…,Ck if it is not empty

Alternating projection does not always converge in the caseof non-convex set. Can you think of any counter-example?

C1

C2


29

Convex Constraint Sets

● Non-negative set

}0|{ ff

● Bounded-value set

}2550|{ ff

● Bounded-variance set

}||||{ 2 Tgff

A given signal

}|{ BfAf or


30

Non-convex Constraint Set

Histogram matching used in Heeger&Bergen’1995

Bounded Skewness and Kurtosis

skewness kurtosis

The derivation of projection operators onto constraint sets are tediousare referred to the paper and MATLAB codes by Portilla&Simoncelli.


31

Image Examples

original

synthesized


32

Image Examples (Con’d)

original

synthesized


33

When Does It Fail?

original

synthesized


34

Summary

Textures represent an important class of structures in natural images – unlike edges characterizing object boundaries, textures often associate with the homogeneous property of object surfaces

Wavelet-domain parametric models provide a parsimonious representation of high-order statistical dependency within textural images

ee565 advanced image processing copyright xin li 2008 1 statistical modeling of natural images in...

Documents

gaussian distribution

laplacian distribution

empirical cdf ecdf

hypothesized cdf

normal distribution

correlated py

correlated px

empirical observation