texture - university of washington · 2010. 4. 2. · • texture is a local neighborhood property....
TRANSCRIPT
![Page 1: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/1.jpg)
Texture
Texture is a description of the spatial arrangement of color orintensities in an image or a selected region of an image.
Structural approach: a set of texels in some regular or repeated pattern
![Page 2: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/2.jpg)
Problem with Structural Approach
How do you decide what is a texel?
Ideas?
![Page 3: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/3.jpg)
Natural Textures from VisTex
grass leaves
What/Where are the texels?
![Page 4: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/4.jpg)
The Case for Statistical Texture
• Segmenting out texels is difficult or impossible in real images.
• Numeric quantities or statistics that describe a texture can becomputed from the gray tones (or colors) alone.
• This approach is less intuitive, but is computationally efficient.
• It can be used for both classification and segmentation.
![Page 5: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/5.jpg)
Some Simple Statistical Texture Measures
1. Edge Density and Direction
• Use an edge detector as the first step in texture analysis.
• The number of edge pixels in a fixed-size region tells ushow busy that region is.
• The directions of the edges also help characterize the texture
![Page 6: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/6.jpg)
Two Edge-based Texture Measures
1. edgeness per unit area
2. edge magnitude and direction histograms
Fedgeness = |{ p | gradient_magnitude(p) ≥ threshold}| / N
where N is the size of the unit area
Fmagdir = ( Hmagnitude, Hdirection )
where these are the normalized histograms of gradientmagnitudes and gradient directions, respectively.
![Page 7: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/7.jpg)
Original Image Frei-Chen ThresholdedEdge Image Edge Image
Example
![Page 8: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/8.jpg)
Local Binary Pattern Measure
100 101 10340 50 8050 60 90
• For each pixel p, create an 8-bit number b1 b2 b3 b4 b5 b6 b7 b8,where bi = 0 if neighbor i has value less than or equal to p’svalue and 1 otherwise.
• Represent the texture in the image (or a region) by thehistogram of these numbers.
1 1 1 1 1 1 0 0
1 2 3
4
57 6
8
![Page 9: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/9.jpg)
Fids (Flexible Image DatabaseSystem) is retrieving imagessimilar to the query imageusing LBP texture as thetexture measure and comparingtheir LBP histograms
Example
![Page 10: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/10.jpg)
Low-levelmeasures don’talways findsemanticallysimilar images.
Example
![Page 11: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/11.jpg)
Co-occurrence Matrix Features
A co-occurrence matrix is a 2D array C in which
• Both the rows and columns represent a set of possibleimage values.
• C (i,j) indicates how many times value i co-occurs withvalue j in a particular spatial relationship d.
• The spatial relationship is specified by a vector d = (dr,dc).
d
![Page 12: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/12.jpg)
1 1 0 01 1 0 00 0 2 20 0 2 20 0 2 20 0 2 2
j
i
1
3
d = (3,1)
0 1 2
012
1 0 32 0 20 0 1
Cd
gray-toneimage
co-occurrencematrix
From Cd we can compute Nd, the normalized co-occurrence matrix,where each value is divided by the sum of all the values.
Co-occurrence Example
![Page 13: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/13.jpg)
Co-occurrence Features
sums.
What do these measure?
Energy measures uniformity of the normalized matrix.
![Page 14: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/14.jpg)
But how do you choose d?
• This is actually a critical question with all thestatistical texture methods.
• Are the “texels” tiny, medium, large, all three …?
• Not really a solved problem.
Zucker and Terzopoulos suggested using a χ2 statisticaltest to select the value(s) of d that have the most structurefor a given class of images.
![Page 15: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/15.jpg)
Example
![Page 16: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/16.jpg)
Laws’ Texture Energy Features
• Signal-processing-based algorithms use texture filtersapplied to the image to create filtered images from whichtexture features are computed.
• The Laws Algorithm• Filter the input image using texture filters.• Compute texture energy by summing the absolute
value of filtering results in local neighborhoods around each pixel.
• Combine features to achieve rotational invariance.
![Page 17: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/17.jpg)
Law’s texture masks (1)
![Page 18: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/18.jpg)
Law’s texture masks (2)Creation of 2D Masks
E5L5
E5L5
![Page 19: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/19.jpg)
9D feature vector for pixel• Subtract mean neighborhood intensity from (center) pixel• Apply 16 5x5 masks to get 16 filtered images Fk , k=1 to 16
• Produce 16 texture energy maps using 15x15 windowsEk[r,c] = ∑ |Fk[i,j]|
• Replace each distinct pair with its average map:• 9 features (9 filtered images) defined as follows:
![Page 20: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/20.jpg)
Laws Filters
![Page 21: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/21.jpg)
Laws Process
![Page 22: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/22.jpg)
water
tiger
fence
flag
grass
small flowers
big flowers
Is there aneighborhoodsize problemwith Laws?
Example: Using Laws Features to Cluster
![Page 23: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/23.jpg)
Features from sample images
![Page 24: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/24.jpg)
Gabor Filters
• Similar approach to Laws• Wavelets at different frequencies and different
orientations
![Page 25: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/25.jpg)
Gabor Filters
![Page 26: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/26.jpg)
Gabor Filters
![Page 27: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/27.jpg)
Segmentation with Color and Gabor-Filter Texture (Smeulders)
![Page 28: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/28.jpg)
A classical texture measure:
Autocorrelation function• Autocorrelation function can detect repetitive patterns of texels
• Also defines fineness/coarseness of the texture
• Compare the dot product (energy) of non shifted image with a shifted image
![Page 29: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/29.jpg)
Interpreting autocorrelation
• Coarse texture function drops off slowly• Fine texture function drops off rapidly• Can drop differently for r and c• Regular textures function will have peaks and
valleys; peaks can repeat far away from [0, 0]• Random textures only peak at [0, 0]; breadth
of peak gives the size of the texture
![Page 30: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/30.jpg)
Fourier power spectrum
• High frequency power fine texture• Concentrated power regularity• Directionality directional texture
![Page 31: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/31.jpg)
Blobworld Texture Features
• Choose the best scale instead of using fixed scale(s)
• Used successfully in color/texture segmentation in Berkeley’s Blobworld project
![Page 32: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/32.jpg)
Feature Extraction
• Input: image• Output: pixel features
– Color features– Texture features– Position features
• Algorithm: Select an appropriate scale for each pixel and extract features for that pixel at the selected scale
Pixel Features PolarityAnisotropyTexture contrast
featureextraction
Original image
![Page 33: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/33.jpg)
Texture Scale• Texture is a local neighborhood property.
• Texture features computed at a wrong scale can lead to confusion.
• Texture features should be computed at a scale which is appropriate to the local structure being described.
The white rectangles show some sample texture scales from the image.
![Page 34: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/34.jpg)
Scale Selection Terminology
• Gradient of the L* component (assuming that the image is in the L*a*b* color space) :▼I
• Symmetric Gaussian : Gσ (x, y) = Gσ (x) * Gσ (y)
• Second moment matrix: Mσ (x, y)= Gσ (x, y) * (▼I)(▼I)T
Notes: Gσ (x, y) is a separable approximation to a Gaussian.
σ is the standard deviation of the Gaussian [0, .5, … 3.5].
σ controls the size of the window around each pixel [1 2 5 10 17 26 37 50].
Mσ(x,y) is a 2X2 matrix and is computed at different scales defined by σ.
Ix2 IxIy
IxIy Iy2
IxIy
![Page 35: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/35.jpg)
Scale Selection (continued)• Make use of polarity (a measure of the extent to which the gradient
vectors in a certain neighborhood all point in the same direction) to select the scale at which Mσ is computed
Edge: polarity is close to 1 for all scales σTexture: polarity varies with σUniform: polarity takes on arbitrary values
![Page 36: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/36.jpg)
Scale Selection (continued)
• n is a unit vector perpendicular tothe dominant orientation.
• The notation [x]+ means x if x > 0 else 0
The notation [x]- means x if x < 0 else 0
• We can think of E+ and E- as measuresof how many gradient vectors in thewindow are on the positive side andhow many are on the negative side of the dominant orientation in thewindow.
n=[1 1]
x = [1 .6]
x’ = [-1 -.6]
Example:
polarity pσ
![Page 37: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/37.jpg)
Scale Selection (continued)
• Texture scale selection is based on the derivative of the polarity with respect to scale σ.
• Algorithm:
kσkσ
1. Compute polarity at every pixel in the image for σk = k/2, (k = 0,1…7).
2. Convolve each polarity image with a Gaussian with standarddeviation 2k to obtain a smoothed polarity image.
3. For each pixel, the selected scale is the first value of σfor which the difference between values of polarity at successive scales is less than 2 percent.
![Page 38: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/38.jpg)
Texture Features Extraction
• Extract the texture features at the selected scale– Polarity (polarity at the selected scale) : p = pσ*
– Anisotropy : a = 1 – λ2 / λ1
λ1 and λ2 denote the eigenvalues of Mσ
λ2 / λ1 measures the degree of orientation: when λ1 is largecompared to λ2 the local neighborhood possesses a dominant orientation. When they are close, no dominant orientation.When they are small, the local neighborhood is constant.
– Local Contrast: C = 2(λ1+λ2)3/2
A pixel is considered homogeneous if λ1+λ2 < a local threshold
![Page 39: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/39.jpg)
Blobworld Segmentation Using Color and Texture
![Page 40: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/40.jpg)
Application to Protein Crystal Images
Original image in PGM (Portable Gray Map ) format
• K-mean clustering result (number of clusters is equal to 10 and similarity measure is Euclidean distance)
• Different colors represent different textures
![Page 41: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/41.jpg)
Application to Protein Crystal Images
• K-mean clustering result (number of clusters is equal to 10 and similarity measure is Euclidean distance)
• Different colors represent different textures
Original image in PGM (Portable Gray Map ) format
![Page 42: Texture - University of Washington · 2010. 4. 2. · • Texture is a local neighborhood property. • Texture features computed at a wrong scale can lead to confusion. • Texture](https://reader035.vdocuments.mx/reader035/viewer/2022063022/5fea5959d0a4e169f13a3c4b/html5/thumbnails/42.jpg)
References
• Chad Carson, Serge Belongie, Hayit Greenspan, and Jitendra Malik. "Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying." IEEE Transactions on Pattern Analysis and Machine Intelligence 2002; Vol 24. pp. 1026-38.
• W. Forstner, “A Framework for Low Level Feature Extraction,”Proc. European Conf. Computer Vision, pp. 383-394, 1994.