texture analysis - nus computing - home

Texture Analysis

CS4243 Computer Vision and Pattern Recognition

Leow Wee Kheng

Department of Computer ScienceSchool of Computing

National University of Singapore

(CS4243) Texture Analysis 1 / 47

Texture

Texture

Texture: spatial arrangement of colors or intensities in an image.

(a) artificial texture (b) natural texture


Texture

An image of a complex scene can contain many textured regions.

The same texture can appear in different intensities and contrastsdue to varying lighting conditions.

The same texture can appear in different scales and orientationsdue to varying view points.


Texture

Basic types of computational texture features:

Structuraldescribe arrangment of texture elements,e.g., texton model [JB83], texel model [TJ90].

Statisticalcharacterize texture in terms of statistical features,e.g., co-occurrence matrix [HSD73], local statistics [TMY78],multiresolution simultaneous autoregressive model (MRSAR)[MJ92], Markov random field (MRF) [CJ83].

Spectralbased on analysis in spatial-frequency domain,e.g., Fourier transform, Gabor filters [BCG90], Gaussianderivatives [JP90], wavelets.


Statistical Features Co-occurrence Matrix

Co-occurrence Matrix

Co-occurrence Matrix Cd [HSD73, SS01]

Specified with a displacement vector d = (row, column).

Entry Cd[i, j] indicates how many times a pixel with gray level i isseparated from a pixel of gray level j by the displacement vector d.

Usually use normalized version of Cd:

Nd[i, j] =Cd[i, j]

∑

i

∑

j

Cd[i, j](1)

Sometimes use symmetric version of Cd:

Sd[i, j] = Cd[i, j] + C−d[i, j] (2)



Examples:

= (0,1)dCd Cd = (1,0)d Cd = (1,1)d

1 01 0

1 0 01

2 2

2 2

00

0 0 2

1

4 0 2

2 2 0

0 0 2

0 2

2 2 0

0 0 2

0 2

2 1 1

0 0

4 2

1

0 1 2

i

j

ji i

j

Image I

i

j

0 0

2 2

22

11

10 01 j j

0

i i



Co-occurrence matrix captures spatial distribution of gray levels.

Consider the following example (black = 1, white = 0):

Image I Cd,d = (1, 1) Cd,d = (0, 1)

25 0

0 24

0 28

28 0

For d = (1, 1), the only non-zero entries are at [0,0] and [1,1].Captures diagonal structure.

For d = (0, 1), the only non-zero entries are at [0,1] and [1,0].Captures horizontal structure.



What if the image does not have a fixed structure?

Use entropy H to measure randomness of gray-level distribution:

H = −∑

i

∑

j

Nd(i, j) log2Nd(i, j) (3)

Entropy has the largest value when all the Nd(i, j) are equal.What does this means?



In addition, these features are also useful:

Energy =∑

i

∑

j

N2d(i, j) (4)

Contrast =∑

i

∑

j

(i− j)2Nd(i, j) (5)

Homogeneity =∑

i

∑

j

Nd(i, j)

1 + |i− j| (6)

Correlation =

∑

i

∑

j

(i− µi)(j − µj)Nd(i, j)

σiσj(7)



µi is mean and σi is standard deviation of row sum Nd(i):

Nd(i) =∑

j

Nd(i, j) (8)

µj is mean and σj is standard deviation of column sum Nd(j):

Nd(j) =∑

i

Nd(i, j) (9)

One last question: How to choose a good d?

Suggestion: Use χ2 test to select the values of d that have the moststructure, i.e., minimize the value:

χ2(d) =∑

i

∑

j

N2d(i, j)

Nd(i)Nd(j)− 1 (10)


Statistical Features Markov Random Field

Markov Random Field

Discrete Markov Random Field (MRF) model describes therelationship between a pixel and its neighbors [CJ83]:

f(p) =∑

q∈N

θ(q)f(p+ q) + ε(q) (11)

f(p) : intensity of pixel at p

θ(q) : model parameters

ε(q) : noise, usually a Gaussian random variable with zero meanand variance σ2.

N : neighborhood, e.g., 3×3 neighborhood centered at p.


Statistical Features Markov Random Field

How to extract MRF features from an image?

Given an image, estimate θ(q) and ε(q) that fit the image well.

Can estimate using least square method:Find θ(q) and ε(q) that minimize error E

E =∑

p

f(p)−

∑

q∈N

θ(q)f(p+ q) + ε(q)

2

(12)


Statistical Features Multiresolution Simultaneous Autoregressive Model

Simultaneous Autoregressive Model

Simultaneous Autoregressive Model (SAR) [MJ92]

a kind of Markov Random Field model

models the relationship between a pixel and its neighbors as

f(p) = µ+∑

q∈N

θ(q)f(p+ q) + ε(q) (13)

f(p) : intensity of pixel at p

θ(q) : model parameters

µ : bias, usually taken as the mean intensity of the image

ε(q) : noise, usually a Gaussian random variable with zero meanand variance σ2.

N : neighborhood, e.g., 3×3 neighborhood centered at p.



Rotation-Invariant SAR

Basic Ideas:

Model relationship of pixels on circular grid instead of square grid.

white dot: center pixel p

black dots: neighbors q of center pixel p

Input image has square grid.

To obtain intensity of pixels on circular grid from those on squaregrid, use bilinear interpolation(see lecture notes for Image Registration and Mosaicking).



Multiresolution SAR

Basic Ideas [MJ92]:

Perform SAR or RISAR at various image resolution.

e.g., apply 5×5, 7×7, 9×9 windows on input image.Can also construct image pyramid:

SAR/RISAR

SAR/RISAR

SAR/RISAR

image pyramid

parametersmodel

input image

and apply the same window size on images of different resolutions.


Spectral Features Fourier Transform

Fourier Transform

Fourier transform [Bra86] is a very useful tool for analyzing patterns.

1-D Fourier Transform

Given a function f(x), its 1-D Fourier transform F (u) is:

F (u) =

∫ ∞

−∞

f(x) exp(−i2πxu)dx (14)

i : the imaginary number√−1

u : frequency

Given F (u), the original f(x) can be recovered by the inverse FT:

f(x) =

∫ ∞

−∞

F (u) exp(i2πxu)du (15)



Some properties of FT:

domain spatial frequency

property f(x) F (u)

scaling f(ax)1

|a|F(u

a

)

addition f(x) + g(x) F (u) +G(u)

shift f(x− a) exp(−i2πau)F (u)

convolution f(x) ∗ g(x) F (u)G(u)



More properties of FT [Bra86]:

In general, if f(x) is a complex function, F (u) is also complex.

If f(x) is real and even (i.e., symmetric), F (u) is also real andeven.



If f(x) is real and odd (i.e., anti-symmetric), F (u) is imagineryand odd.



Fourier Transform Pairs [Bra86]

f(x) F (u)

If f(x) contains a single frequency, then the frequency is shown inthe FT.




f(x) F (u)



sinc x = (sinx)/x

If f(x) contains 0 frequency, then the FT has a non-zero valueonly at frequency 0.

If f(x) contains many frequencies, then the FT is non-uniformwith peaks at strong frequency components.

FT of Gaussian is still a Gaussian. Increasing the width (σ) of theorignal Gaussian decreases the width of its FT.



2-D Fourier Transform

Given a function f(x, y), its 2-D FT F (u, v) is

F (u, v) =

∫ ∞

−∞

∫ ∞

−∞

f(x, y) exp(−i2π(ux+ vy)) dx dy (16)

u, v : frequencies along orthogonal axes

The inverse FT of F (u, v) is

f(x, y) =

∫ ∞

−∞

∫ ∞

−∞

F (u, v) exp(i2π(ux+ vy)) du, dv (17)



2-D FT has similar properties as 1-D FT:

domain spatial frequency

Property f(x, y) F (u, v)

scaling f(ax, by)1

|ab|F(u

a,v

b

)

addition f(x, y) + g(x, y) F (u, v) +G(u, v)

shift f(x− a, y − b) exp(−i2π(au+ bv))F (u, v)

convolution f(x, y) ∗ g(x, y) F (u, v)G(u, v)




f(x, y) F (u, v)

Orientation of sinusoidal pattern is reflected in the 2-D FT.




f(x, y) F (u, v)



Sample texture images and their Fourier transforms:

(a) structured (b) oriented (c) granular (d) random

Question: What happen to the Fourier transform patterns when thetexture scale and orientation are changed?


Spectral Features Gabor Filters

Gabor Filters

Gabor function is a complex sinusoidal (i.e., sine and cosine) functionmodulated by an oriented Gaussian [BCG90]:

h(x, y) = g(x′, y′) exp(2πjfx′) (18)

Oriented Gaussian:

g(x′, y′) =1

2πλσ2exp

[

−(x′/λ)2 + y′2

2σ2

]

(19)

(x′, y′) = (x cos θ + y sin θ,−x sin θ + y cos θ) are rotatedcoordinates

θ is the orientation of the Gaussian from the x-axis

λ is the aspect ratio

σ is the standard deviation of the Gaussian



Examples:

-2 -1

0 1

2

x -2

-1

0 1

2

y

0

0.1

0.2

0.3

0

0.1

0.2

0.3

-2

-1 0

1 2

x -2

-1 0

1 2

y

0

0.1

0.2

0.3

0

0.1

0.2

0.3

(a) λ = 0.5, σ = 1, θ = 0 (b) λ = 2, σ = 0.5, θ = 0

Usually use λ < 1.



Complex sinusoid:

exp(2πjfx′) = cos(2πfx′) + j sin(2πfx′) (20)

The real (cosine) and imaginary (sine) parts of h(x, y) are

hc,fθ = g(x′, y′) cos(2πfx′)

hs,fθ = g(x′, y′) sin(2πfx′)(21)



cosine part sine part

Side view:

Top view:

Gaussiancosine sine Gaussian



Let’s analyze the Gabor function in the frequency domain.

The Fourier transform of h(x, y) is

H(u, v) = exp

−2π2σ2[

(u′ − f)2λ2 + v′2]

(22)

H(u, v) is a Gaussian function

(u′, v′) = (u cos θ+ v sin θ,−u sin θ+ v cos θ), i.e., rotated version of(u, v)

θ is the orientation of the Gaussian from the u-axis



Gaussiancosine

sine Gaussian

B

v

fΩ

u

half-peak support



Let U and V denote the spatial frequencies along the u and v-axes.

v

f

U

Vα u

f =√

U2 + V 2 (23)

θ = tan−1 V

U(24)

Frequency bandwidth B and orientation bandwidth Ω are given by

B = log2

[

πfλσ + α

πfλσ − α

]

(25)

Ω = 2 tan−1

(

α

πfσ

)

(26)

where α =√

(ln 2)/2.(CS4243) Texture Analysis 35 / 47


So, can design a set of Gabor filters with 4 parameters λ, σ, f, θ orλ, σ,B,Ω or other combinations.

Notes:

Usually, want to cover the frequency domain with a small numberof filters.

That is, minimize the overlap between the half-peak supports ofthe filters.



Example Gabor filters (in Fourier domain):



To obtain the Gabor features of an image I, convolve the image withthe Gabor filters:

ec,fθ(x, y) = I(x, y) ∗ hc,fθ(x, y)

es,fθ(x, y) = I(x, y) ∗ hs,fθ(x, y)(27)

The energy E and amplitude A of Gabor features are

Efθ(x, y) = e2c,fθ(x, y) + e2s,fθ(x, y) (28)

Afθ(x, y) =√

Efθ(x, y) (29)

The mean energy Efθ over the whole image is

Efθ =1

n

∑

x

∑

y

Efθ(x, y) (30)

where n is the number of pixels in the image.



Let’s arrange the mean energy (or mean amplitude) in a 2-D form.

f

θ

texture image Gabor features



Different types of texture have different Gabor patterns.

structured oriented granular random

structured: localized pattern

oriented (or directional): column pattern

granular: row pattern

random: random pattern



Change of texture scale and orientation result in shift of Gabor pattern.


Least Square Method

Appendix A

Least Square Method

Basic Ideas:

Form an error function E(w1, . . . , wn) of parameters wi.

Differentiate E with respect to each parameter, set to 0, and solvefor optimal parameter values.

∂E

∂wi

= 0, for i = 1, . . . , n (31)

Many times, it is difficult to solve the above equations directly.Then, use numerical algorithms.


Least Square Method

Simplest case: gradient descent or steepest descent.

1 Set initial parameter values wi(0).

2 Change wi iteratively as follows:

wi(t+ 1) = wi(t)− η∂E

∂wi

∣

∣

∣

∣

wi(t)

(32)

where η is a constant update rate.

Because wi is changed according to negative of error gradient,changing wi always decrease E.

Have many sophisticated methods.Check Numerical Receipes in C or use Matlab.

For brief introduction to numerical methods, checkwww.comp.nus.edu.sg/∼cs6240/numalgo.html.


Reference

Reference I

A. C. Bovik, M. Clark, and W. S. Geisler.Multichannel texture analysis using localized spatial filters.IEEE Trans. on Pattern Analysis and Machine Intelligence,12(1):55–73, 1990.

R. N. Bracewell.The Fourier Transform and Its Applications.McGraw-Hill, 2nd, revised edition, 1986.

G. R. Cross and A. K. Jain.Markov random field texture models.IEEE Trans. on Pattern Analysis and Machine Intelligence,5:25–39, 1983.


Reference

Reference II

R.M. Haralick, K. Shanmugam, and I. Dinstein.Texture features for image classification.IEEE Trans. on Systems, Man, and Cybernetics, 3:610–621, 1973.

B. Julesz and J. R. Bergen.Textons: The fundamental elements in preattentive vision andperception of textures.Bell System Technology Journal, 62(6):1619–1645, 1983.

M. Jitendra and P. Pietro.Preattentive texture discrimination with early vision mechanisms.Journal of the Optical Society of America, A, 7(5):923–932, 1990.


Reference

Reference III

J. C. Mao and A. K. Jain.Texture classification and segmentation using multiresolutionsimultaneous autoregressive models.Pattern Recognition, 25:173–188, 1992.

L. G. Shapiro and G. C. Stockman.Computer Vision.Prentice-Hall, 2001.

M. Tuceryan and A. K. Jain.Texture segmentation using voronoi polygons.IEEE Trans. on Pattern Analysis and Machine Intelligence,12(2):211–216, 1990.


Reference

Reference IV

H. Tamura, S. Mori, and T. Yamawaki.Textural features corresponding to visual perception.IEEE Trans. on Systems, Man, and Cybernetics, 8(6):460–473,1978.


texture analysis - nus computing - home

Documents