digital image processing & pattern analysis (csce 563) introduction to pattern analysis prof....
TRANSCRIPT
Digital Image Processing&
Pattern Analysis(CSCE 563)
Introduction to Pattern Analysis
Digital Image Processing&
Pattern Analysis(CSCE 563)
Introduction to Pattern Analysis
Prof. Amr Goneid
Department of Computer Science & EngineeringThe American University in Cairo
Prof. Amr Goneid, AUC 2
Introduction to Pattern Analysis
Patterns & Pattern Analysis
Pattern Classes
Minimum Distance (Nearest Neighbor) Classifiers
Structural Matching Classifiers
Information Measure Classifiers
Learning-Based Classifiers
Prof. Amr Goneid, AUC 3
1. Patterns & Pattern Analysis
Patterns are Object Descriptors
The objective of pattern analysis is to classify objects into distinct and separable classes
A pattern consists of a finite set of “features” or “attributes”
Such set is usually called a “Feature Vector”
The “Dimesionality” of a pattern is the number (n) of features in the feature vector
Prof. Amr Goneid, AUC 4
Patterns & Pattern Analysis
Features can be:
Quantitative or Qualitative
Numerical or Binary
Ordinal or Non-Ordinal
The process of mapping object descriptors
to a Feature Vector is called “Feature Extraction”
The process of reducing the number of attributes while maximizing the seprability between classes is called “Dimensionality Reduction”
Prof. Amr Goneid, AUC 5
Feature Vector
object Mapper
Dimensionality
Reduction
Feature
Extraction
k-dimensional
Feature Vector
n-dimensional
Feature Vectorx1
x2
.
.
xn
X = n < k
Prof. Amr Goneid, AUC 6
Feature Vector (Example)
Boundary
Fourier
Descriptors
Sort &
Truncate
Boundary
Pixels
k-dimensional
Feature Vector
n-dimensional
Feature Vectorx1
x2
.
.
xn
X = n < k
Prof. Amr Goneid, AUC 7
Dimensionality Reduction
Removal of features with insignificant weights (ordering or sorting then truncation)Removal of non-orthogonal features (discovering association)Selecting features that maximize inter-class to intra-class variances.Discriminant AnalysisPCA
Prof. Amr Goneid, AUC 8
2. Pattern Classes
Classifiersw1
w2
wM
Classifierx
Prof. Amr Goneid, AUC 9
Feature Space & Seprability
Feature Space
Clusters (Classes)
Linearly Separable Classes
Non-Separable Classes
w1w2
w1w2
w1w2
Prof. Amr Goneid, AUC 10
3. Minimum Distance (Nearest Neighbor) Classifiers
Distance Measures:
Search for minimum
distance to class “center”
“Nearest Neighbor”
w1w2
**x
Prof. Amr Goneid, AUC 11
Class Means
n Features: x1 , x2 , ….., xn
Feature Vector of unknown pattern: X
M Pattern Classes: w1 , w2 , …. wM
Sample Patterns for class (j) L = 1,2,…, Nj have sample feature vectors SjL
Class Means for class (j):
mj = (1/ Nj) L SjL = =
m1
m2
.
mnj
<x1>
<x2>
.
<xn> j
Prof. Amr Goneid, AUC 12
Euclidean Distance
For the unknown pattern X , the Euclidean Distance from the “center” of class wj is:
Dj2(X) = || X – mj||2 = (X – mj)T (X – mj)
= XT X - 2 { XT mj – (1/2) mjT mj }
The “Affinity Function”:
aj(X) = XT mj – (1/2) mjT mj
changes with class (j) and should be maximum
Prof. Amr Goneid, AUC 13
Decision Boundary
The decision boundary between class wi and class wj is:
aij(X) = ai(X) – aj(X) = 0
If aij(X) > 0 pattern X belongs to class wi
If aij(X) < 0 pattern X belongs to class wj
Prof. Amr Goneid, AUC 14
Example:
w1: Ballet dancers, w2 : weight lifters
Features: x1= weight in kg, x2 = height in cm
Samples have means:
m1 = ( 60 170)T , m2 = ( 120 160)T
Unknown pattern X = (x1 x2)T
The affinity functions are:
a1(X) = 60 x1 + 170 x2 – 16250
a2(X) = 120 x1 + 160 x2 - 20000
Prof. Amr Goneid, AUC 15
Example (continued)
The decision boundary is:
a12(X) = - 60 x1 + 10 x2 + 3750 = 0
For the unknown pattern X,
if a12(X) > 0 , it belongs to ballet dancers
if a12(X) < 0 , it belongs to weight lifters
e.g. if x1 = 100 kg , x2 = 165 cm
then it is a weight lifter
weight
height
w1w2
a12(X) = 0
Prof. Amr Goneid, AUC 16
4. Structural Matching Classifiers
String Matching
Two boundaries are coded as strings A , B.
A match occurs at position k if A(k) = B(k).
M = # of matches
Number of mismatches is:
Q = max (length(A) , length(B)) – M
Q = 0 if A and B are identical
Degree of matching is R = M / Q
Prof. Amr Goneid, AUC 17
Example: Matching Polygons
Boundaries can be approximated by polygons
Traversing anti-clockwise, we compute the interior angles between segments.
Angles are quantized into regions. Quantized angles are coded by symbols.
Successive angles of a polygon are represented by a string of symbols that can be matched to another string (polygon)
Prof. Amr Goneid, AUC 18
5. Information Measure Classifiers(Information Content)
For Binary Attributes
Information Content:
A set of n objects with only one attribute:
a = # of objects having attribute
(n-a) = # of objects missing attribute
Information Content is:
I = n log n – { a log a + (n-a) log (n-a)} for n > 1
= 0 for n = 1
Prof. Amr Goneid, AUC 19
Diversity & Similarity
A set of n objects i = 1 , 2 , .. N
and m attributes j = 1 , 2 , … , m
Bij = 1 if object (i) has attribute (j) and zero otherwise
aj = # of objects having attribute (j)
(n – aj) = # of objects missing attribute (j) = i Bij
Information Content:
I (m,n,a) = m n log n - j {aj log aj + (n – aj) log (n – aj)}
for n > 1
I (m,n,a) = 0 for n = 1
Prof. Amr Goneid, AUC 20
Diversity & Similarity
Let m = n
For complete similarity, aj = n for all j
I = n2 log n - j { n log n + 0 } = 0
Least info content
For complete diversity, aj = 1 for all j
I = n2 log n - n(n-1) log (n-1) > 0
Max info content
e.g. for n = m = 2, I = 4
Prof. Amr Goneid, AUC 21
Fusion & Division
Two object sets A , B
Objects in A are similar. Same for B
IA = IB = 0
Fusing A,B into C = AB we gain info by
I = IC – [IA + IB]
Dividing C into A and B with AB = 0, we lose info by I = IC – [IA + IB]
Prof. Amr Goneid, AUC 22
Selecting Division Attributes
Dividing set C on attribute (j) produces two sets A with nj
+ = aj objects and B with nj- = (n-aj) objects.
Let Ij+ = I (m , nj
+ , a) , Ij- = I (m , nj
- , a) , then the gain from fusing A and B into C is
Ij = IC – [IA + IB] = I (m,n,a) – {Ij+ + Ij
- }
The division attribute (j) corresponds to max(Ij )
The process is repeated for the net attribute after removing the first one.
Prof. Amr Goneid, AUC 23
Classification Tree
Recursive computation of info gain produces a classification tree
4 5 6 7
2 3
11
1 0
0
1 0
Prof. Amr Goneid, AUC 24
6. Learning-Based Classifiers(Neural Networks)
Neurons
Learning
Feed-Forward NN
Backpropagation NN
Other NN