patch matching under non gaussian noise - u-bordeaux.frjaujol/isis/121205/deledalle.pdf · gdr isis...

GDR ISIS – Modélisation mathématique des texturesTélécom ParisTech, France, December 05, 2012

Patch matching under non Gaussian noise

Charles Deledalle1, Florence Tupin2, Löıc Denis3, Martin Royer2

1 CNRS, Institut de Mathématiques de Bordeaux, Université Bordeaux 1, France2 Institut Telecom, Telecom ParisTech, CNRS LTCI, Paris, France

3 Telecom Saint-Etienne, France

December 05, 2012

C. Deledalle (CNRS/IMB) Patch matching December 05, 2012 1 / 24

Motivation

Increasing use of patches to model images

Texture synthesis,

Inpainting,

Image editing,

Denoising,

Super-resolution,

Image registration,

Stereo vision,

Object tracking.

Image model based on the natural redundancy of patches


Motivation


Texture synthesis,

Inpainting,

Image editing,

Denoising,

Super-resolution,

Image registration,

Stereo vision,

Object tracking.

(a) Mitochondrion in microscopy

c©Chandra

(b) Supernova in X-ray imagery

c©DLR

(c) Polarimetric SAR imagery


Motivation


Texture synthesis,

Inpainting,

Image editing,

Denoising,

Super-resolution,

Image registration,

Stereo vision,

Object tracking.

How to compare noisy patches together?

︸︷︷︸?

︸︷︷︸?

︸︷︷︸?

How to take into account the noise model?


Motivation


Texture synthesis,

Inpainting,

Image editing,

Denoising,

Super-resolution,

Image registration,

Stereo vision,

Object tracking.

How to compare noisy patches to a noise-free ones?

︸︷︷︸?

︸︷︷︸?

︸︷︷︸?

How to take into account the noise model?


Table of contents

1 A similarity criterion to compare noisy patches

2 Compare noisy patches to noise-free ones with contrast invariance

3 Conclusions and perspectives


Table of contents





Motivation

(a) Microscopy

c©Chandra

(b) Astronomy

c©DLR

(c) SAR polarimetry

︸︷︷︸?

︸︷︷︸?

︸︷︷︸?

How to compare noisy patches?


Patch-similarity from the squares differences

How to compare noisy patches?

Assume noise is additive and Gaussian such that:

︸︷︷︸v1

= ︸︷︷︸u1

+ ︸︷︷︸n1

and ︸︷︷︸v2

= ︸︷︷︸u2

+ ︸︷︷︸n2

Compare patches by the use of the squares differences:

when u1 = u2 :

(−

)2= is low⇒ decide “similar”

when u1 6= u2 :(

−)2

= is high⇒ decide “dissimilar”

What about non-Gaussian noise?


Limits of the squares differences

Beyond the Gaussian noise assumption

Noise can be non-additive and/or non-Gaussian, e.g., for Poisson noise:

︸︷︷︸v1

= ︸︷︷︸u1

+ ︸︷︷︸n1

and ︸︷︷︸v2

= ︸︷︷︸u2

+ ︸︷︷︸n2

The squares differences is no longer discriminant:

when u1 = u2 :

(−

)2=

when u1 6= u2 :(

−)2

=

The squared differences are not discriminant when noise departs from Gaussian


Similarity through variance stabilization

Variance stabilization approach

Use an application s which stabilizes the variance for a specific noise model

Evaluate the squared differences between the transformed patches:(s

( )− s

( ))2=

(−

)2,

Example

Gamma noise (multiplicative) and the homomorphic approach:

s(V ) = log V

Poisson noise and the Anscombe transform:

s(V ) = 2

√V +

3

8


Similarity with variance stabilization

Limits

Only heuristic

No optimality results

Does not take into account the statistics of the transformed data

Does not apply to all noise distributionse.g., multi-modal distributions like interferometric phase distribution

(a) Image with impulse noise

c©ONERA c©CNES

(b) SAR interferometric phase


Similarity in a detection framework

Similarity in the light of detection theory

Similarity can be defined as an hypothesis test (i.e., a parameter test):

H0 : u1 = u2 ≡ u12 (null hypothesis)H1 : u1 6= u2 (alternative hypothesis)

Its performance can be measured as:

PFA = P(decide “dissimilar” | u12,H0) (false-alarm rate)PD = P(decide “dissimilar” | u1,u2,H1) (detection rate)

The likelihood ratio (LR) test maximizes PD for any PFA:

L(v1,v2) =p(v1,v2 | u12,H0)p(v1,v2 | u1,u2,H1)

← given by the noise distribution model

→ Problem: u12, u1 and u2 are unknown



Generalized likelihood ratio (GLR)

Replace u12, u1 and u2 with maximum likelihood estimates (MLE)

Define the (negative log) generalized likelihood ratio test:

− logLG(v1,v2) = − logsupt p(v1,v2 | u12 = t,H0)

supt1,t2 p(v1,v2 | u1 = t1,u2 = t2,H1)

= − logp(v1 | u1 = t̂12) p(v2 | u2 = t̂12)p(v1 | u1 = t̂1) p(v2 | u2 = t̂2)

Illustration

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

Noise−free values space (domain of t)

Possib

ility

mesure

of u

1=

u2

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

Noise−free values space (domain of t)

Possib

ility

mesure

of u

1=

u2







supt1,t2 p(v1,v2 | u1 = t1,u2 = t2,H1)


Equal self similarity

Assume v1 = v2, then:

− logp

(v1 =

∣∣∣∣∣ u1 =)p

(v2 =

∣∣∣∣∣ u2 =)

p

(v1 =

∣∣∣∣∣ u1 =)p

(v2 =

∣∣∣∣∣ u2 =) = 0







supt1,t2 p(v1,v2 | u1 = t1,u2 = t2,H1)


Maximal self similarity

Assume v1 6= v2, then:

− logp

(v1 =

∣∣∣∣∣ u1 =)p

(v2 =

∣∣∣∣∣ u2 =)

p

(v1 =

∣∣∣∣∣ u1 =)p

(v2 =

∣∣∣∣∣ u2 =) > 0


Patch similarity criteria – Generalized likelihood ratio

name pdf − logLG Stabilization Sq. diff

Gaussian e− (v−u)

2

2σ2√2πσ

(v1−v2)2

Poisson uve−u

v!log

(2v1+v2v1

v1v2v2

(v1+v2)v1+v2

) (√v1+3/8−

√v2+3/8

)2

Gamma LLvL−1e−

Lvu

Γ(L)uLlog(√

v1v2

+√v2v1

)−log 2

(log

v1v2

)2

The three criteria for three noise models

Illustration with Gamma noise

1 When u1 = u2 = u12, the residue is statistically small:

− logLG

(,

)= is low⇒ decide “similar”

2 Then, when u1 6= u2, the residue is statistically higher:

− logLG

(,

)= is high⇒ decide “dissimilar”


Evaluation of similarity criteria – Detection performance

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Probability of false alarmP

rob

ab

ility

of

de

tectio

n

Gamma

L GSGQ GGeneralized likelihood ratio

Variance stabilization

Squared differences

Maximum joint likelihood [Alter et al., 2006]

Mutual information kernel [Seeger, 2002]

Bayesian likelihood ratio [Minka, 1998, Minka, 2000]

Bayesian joint likelihood [Yianilos, 1995, Matsushita and Lin, 2007]



0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Probability of false alarmP

rob

ab

ility

of

de

tectio

n

Poisson

L GK BSL BGQ BQ GGeneralized likelihood ratio

Variance stabilization

Squared differences

Maximum joint likelihood [Alter et al., 2006]

Mutual information kernel [Seeger, 2002]

Bayesian likelihood ratio [Minka, 1998, Minka, 2000]

Bayesian joint likelihood [Yianilos, 1995, Matsushita and Lin, 2007]


Evaluation on denoising – Non-local filtering with GLRG

am

ma

c©ONERA c©CNES

Po

isso

n

c©Chandra

(a) Noisy image (b) Euclidean distance (c) GLR


Stereo-vision

(a) Noisy imageG

am

ma

(b) Euclidean distance (c) GLR

(d) Ground truth

Po

isso

n

(e) Euclidean distance (f) GLR


Motion tracking – Glacier monitoring with a stereo pair of SAR images

(g) Noisy image (h) Euclidean distance (i) GLR

Glacier of Argentière. With GLR, the estimated speeds matches with the ground truth: average over the surface

of 12.27 cm/day and a maximum of 41.12 cm/day in the areas with crevasses.


Comparison of noisy patches beyond Gaussian noise

Conclusion

Similarity between noisy patches expressed as an hypothesis test

Among 7 similarity criteria, GLR provides the best performance

Apply even when variance stabilization is not possible

Easy to derive as long as the MLE is known in closed form

Offers good theoretical properties:

Max. self sim. Eq. self sim. Id. of indiscernible Invariance Asym. CFAR Asym. UMPI

Euclidean kernel√ √ √

× × ×

Stabilization transform ×

Bayesian joint lik. × × × × × ×

Maximum joint lik. × × × × × ×

Bayesian lik. ratio × × ×√

× ×

Mutual info. kernel√ √ √ √

× ×

GLR√ √ √ √ √ √

[Deledalle et al., 2012] Deledalle, C., Denis, L. , Tupin, F. (2012).

How to compare noisy patches? Patch similarity beyond Gaussian noise.

International Journal of Computer Vision, vol. 99, no. 1, pp. 86-102, 2012


Table of contents





Motivation

(a) Microscopy (b) Dictionnary

︸︷︷︸?

How to compare noisy patches to noise-free ones?


Compare noisy patches to noise-free ones with contrast invariance

︸︷︷︸v1

= ︸︷︷︸u1

+ ︸︷︷︸n1

where ︸︷︷︸v1

matches , and

First intuition: use the correlation

Compute the modulus of the correlation

C(v1,u2) =

∣∣∣∣∣∑k(v1,k − v̄1)(u2,k − ū2)√∑

k(v1,k − v̄1)2∑k(u2,k − ū2)2

∣∣∣∣∣such that

u1 = αu2 + β : C(

,

)=

∣∣∣∣∑ ∣∣∣∣ = 0.97 ⇒ decide “similar”u1 6= αu2 + β : C

(,

)=

∣∣∣∣∑ ∣∣∣∣ = 0.07 ⇒ decide “dissimilar”

Is the correlation a good detector?



Detection theory

Similarity can be defined as an hypothesis test (i.e., a parameter test):

H0 : u1 = αu2 + β (null hypothesis)H1 : u1 6= αu2 + β (alternative hypothesis)

The likelihood ratio (LR) test maximizes PD for any PFA:

L(v1,u2) =p(v1 | αu2 + β,H0)p(v1 | u1,H1)

← given by the noise distribution model

→ Problem: α, β, u1 are unknown


Replace α, β and u1 with maximum likelihood estimates (MLE)


− logLG(v1,u2) = − logsupα,β p(v1 | u1 = αu2 + β,H0)

supt1 p(v1 | u1 = t1,H1)

= − logp(v1 | u1 = α̂u2 + β̂)

p(v1 | u1 = t̂1)



GLR for Gaussian noise

Obtain α̂ and β̂ by minimizing − log p(v1 | u1 = α̂u2 + β̂) ∝∥∥∥v1 − α̂u2 − β̂∥∥∥2

⇒ α̂ =[∑

k(v1,k − v̄1)(u2,k − ū2)]2∑

k(u2,k − ū2)2and β̂ = v̄1 − α̂ū2

Injecting α̂ and β̂ gives

− logLG(v1,u2) =[1− C2(v1,u2)

] ∑k(v1,k − v̄1)2

2σ2

For a fixed v1, maximizing GLR is equivalent in max. the correlation or the likelihood





⇒ α̂ =[∑

k(v1,k − v̄1)(u2,k − ū2)]2∑

k(u2,k − ū2)2and β̂ = v̄1 − α̂ū2


− logLG(v1,u2) =[1− C2(v1,u2)

] ∑k(v1,k − v̄1)2

2σ2


Illustration

The correlation does not take into account the noise, while GLR does

− log C(

,

)> − log C

(,

)

while − logLG

(,

)� − logLG

(,

)

In fact 0× + β “explains” better than .





⇒ α̂ =[∑

k(v1,k − v̄1)(u2,k − ū2)]2∑

k(u2,k − ū2)2and β̂ = v̄1 − α̂ū2


− logLG(v1,u2) =[1− C2(v1,u2)

] ∑k(v1,k − v̄1)2

2σ2


GLR for non-Gaussian noise

For gamma and Poisson distributions: no closed-form expressions

Redefine contrast invariance, for instance, as u1 = β̂u2α̂

Estimate α̂ and β̂ iteratively by Newton’s method (generally, only few iterations are required)

Evaluate the negative log-likelihood ratio at α̂ and β̂



0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Probability of false alarm

Pro

ba

bili

ty o

f d

ete

ctio

n

Gaussian

LG

CGeneralized likelihood ratio

Correlation

+ Stabilization

GLR Gaussien + Stabilization

GLR Gaussien



0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Pro

ba

bili

ty o

f d

ete

ctio

n

Gamma

LGSGC + SGeneralized likelihood ratio

Correlation + Stabilization


GLR Gaussien



0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Pro

ba

bili

ty o

f d

ete

ctio

n

Poisson

LGSGC + SGeneralized likelihood ratio

Correlation + Stabilization


GLR Gaussien


Evaluation on denoising – Patch dictionnary based denoisingG

am

ma

Ga

mm

a

c©ONERA c©CNES

(a) Noisy image (b) Result (c) Dictionnary


Table of contents





Conclusions and perspectives

Take home messages

Comparison of patches can be expressed as an hypothesis testRobust noisy patch comparisonRobust noisy versus noise free patch comparison with contrast invariance

Perform slightly better than variance stabilization

Apply even when variance stabilization is not possible

Can be computed in closed-form in many cases (or in few iterations)

Perspectives

Inject geometric invariance

Inject priors

Deal with blur, subsampling, masking, . . .

Patch dictionary learning based patch matching


Thanks for your attention

[email protected]

http://www.math.u-bordeaux1.fr/~cdeledal/

http://www.math.u-bordeaux1.fr/~cdeledal/

[Alter et al., 2006] Alter, F., Matsushita, Y., and Tang, X. (2006).An intensity similarity measure in low-light conditions.Lecture Notes in Computer Science, 3954:267–280.

[Deledalle et al., 2009] Deledalle, C.-A., Denis, L., and Tupin, F. (2009).Iterative Weighted Maximum Likelihood Denoising with Probabilistic Patch-Based Weights.IEEE Trans. Image Process., 18(12):2661–2672.

[Matsushita and Lin, 2007] Matsushita, Y. and Lin, S. (2007).A Probabilistic Intensity Similarity Measure based on Noise Distributions.In IEEE Comput. Vis. and Pattern Recognition (CVPR), pages 1–8. IEEE.

[Minka, 1998] Minka, T. (1998).Bayesian Inference, Entropy, and the Multinomial Distribution.Technical report, Carnegie Mellon University.

[Minka, 2000] Minka, T. (2000).Distance measures as prior probabilities.Technical report, Carnegie Mellon University.

[Seeger, 2002] Seeger, M. (2002).Covariance kernels from Bayesian generative models.In Advances in Neural Inf. Process. Syst. (NIPS), volume 2, pages 905–912. MIT Press.

[Yianilos, 1995] Yianilos, P. (1995).Metric learning via normal mixtures.Technical report, NEC Research Institute, Princeton, NJ.


A similarity criterion to compare noisy patchesCompare noisy patches to noise-free ones with contrast invarianceConclusions and perspectives

patch matching under non gaussian noise - u-bordeaux.frjaujol/isis/121205/deledalle.pdf · gdr isis...

Documents