1 pattern recognition: statistical and neural lonnie c. ludeman lecture 12 sept 30, 2005 nanjing...
TRANSCRIPT
![Page 1: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/1.jpg)
1
Pattern Recognition:Statistical and Neural
Lonnie C. Ludeman
Lecture 12
Sept 30, 2005
Nanjing University of Science & Technology
![Page 2: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/2.jpg)
2
Lecture 11 Topics
1. M-Class Case and Gaussian Review
2. M-Class Case in Likelihood Ratio Space
3. Example Vector Observation M-Class
![Page 3: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/3.jpg)
3
if equality then decide x from the boundary classes by random choice
MPE and MAP Decision Rule: M-Class Case
Select class Ck
if p( x | Ck ) P(Ck ) > p( x | Cj ) P(Cj )
for all j = 1, 2, … , M j = k
for observed x
Review 1
![Page 4: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/4.jpg)
4
yi(x) = Cij p(x | Cj) P(Cj)j=1
M
if yi(x) < yj(x) for all j = i
Then decide x is from Ci
Bayes Decision Rule: M-Class Case
Review 2
![Page 5: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/5.jpg)
5
if- (x – M
1)TK
1
-1(x – M1) + (x – M
2)TK
2
-1(x – M2)
><
C1
C2
T1 = 2 ln(T ) = 2 lnT + ln - ln
K2
1 2
K1
1 2
T1
Optimum Decision Rule: 2-Class Gaussian
where
And T is the optimum threshold for the typeof performance measure used
K1
K2
Quadratic Processing
Review 3
![Page 6: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/6.jpg)
6
if ( M1 – M
2)T K-1 x >
<C
1
C2
T2
2-Class Gaussian: Special Case 1: K1 = K
2 = K
And T is the optimum threshold for the typeof performance measure used
T2 = ln T + ½ ( M
1
T K-1 M1 – M
2
T K-1 M2)
Equal Covariance Matrices
Linear Processing
Review 4
![Page 7: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/7.jpg)
7
if ( M1 – M
2)T x >
<C
1
C2
T3
2-Class Gaussian: Case 2: K1 = K
2 = K = s2 I
And T is the optimum threshold for the typeof performance measure used
T3 = s2 ln T + ½ ( M
1
T M1 – M
2
T M2)
Equal Scaled Identity Covariance Matrices
Linear Processing
Review 5
![Page 8: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/8.jpg)
8
Q
i(x) = (x – M
j)TK
j
-1(x – Mj) } – 2 ln P(C
j) + ln | K
i |
dMAH
(x , Mj) Bias
Quadratic Operation on observation vector x
2
M- Class General Gaussian - Continued
Select Class Cj if Q
j(x) is MINIMUM
Equivalent statistic: Qj(x) for j = 1, 2, … , M
Review 6
![Page 9: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/9.jpg)
9
Select Class Cj if L
j(x) is MAXIMUM
Lj(x) = M
j
TK-1x – ½ Mj
T
K-1M
j
+ lnP(Cj)
Equivalent Rule for MPE and MAP
M-Class Gaussian – Case 1: K1 = K
2 = … = K
M = K
Dot Product Bias
Linear Operation on observation vector x
Review 7
![Page 10: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/10.jpg)
10
where yi(x) = Cij p(x | Cj) P(Cj)j=1
M
if yi(x) < yj(x) for all j = i
Then decide x is from Ci
Bayes Decision Rule in Likelihood ratio space: M-Class Case derivation
We know that Bayes Decision Rule for the M-Class Case is
![Page 11: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/11.jpg)
11
LM
(x) = p(x | CM
) / p(x | CM
) = 1
Dividing through by p(x | CM) gives sufficient
statistics vi(x) as follows
Therefore the decisin rule becomes
![Page 12: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/12.jpg)
12
Bayes Decision Rule in the Likelihood Ratio Space
The dimension of the Likelihood Ratio Space is always one less than the number of classes
![Page 13: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/13.jpg)
13
Given: Three Classes C1, C
2, and C
3
Example: M-Class case
Nk are statistically independent all classes
Nk ~ N(0,1)
![Page 14: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/14.jpg)
14
Determine:
This problem is an abstraction of a tri-phase communication system
(a) Find the minimum probability of error (MPE) decision rule
(b) Illustrate your decision regions in the observation space
(c) Use your decision rule to classify the observed pattern vector
x =[ 0.4, 0.7]T
(d) Calculate the probability of error P(error)
![Page 15: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/15.jpg)
15
Solution:
Problem is Gaussian with equal scaled identity Covariance Matrices so the optimum decision rule is as follows
(a) Find MPE decision Rule
![Page 16: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/16.jpg)
16
Select class with minimum Li(x)
/
for our example we have
![Page 17: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/17.jpg)
17
Droping the -½ + ln 1/3 as it appears in all the Li(x),
the new statistics s1(x), s
2(x), and s
3(x)can be defined
as
and an equivalent decision rule becomes
![Page 18: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/18.jpg)
18
This decision rule can be rewritten in terms of the observation x as follows
where in the observation space X, R
k is the region where
Ck is decided
![Page 19: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/19.jpg)
19
Decision Region in the Observation Space
X Observation Space
![Page 20: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/20.jpg)
20
(c) the pattern vector x
x = [ x1, x
2 ]T = [ 0.4, 0.7 ]T
x Is a member of R1 therefore x is classified
as coming from class C1
![Page 21: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/21.jpg)
21
(d) Determine the probability of error
P(error) = 1- P(correct)
= 1 - P(correct |C1)P(C
1 )
- P(correct |C2)P(C
2 )
- P(correct |C3)P(C
3 )
![Page 22: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/22.jpg)
22
![Page 23: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/23.jpg)
23
![Page 24: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/24.jpg)
24
P(error) = 0.42728
![Page 25: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/25.jpg)
25
Summary
1. M-Class Case and Gaussian Review
2. M-Class Case in Likelihood Ratio Space
3. Example Vector Observation M-Class
![Page 26: 1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology](https://reader035.vdocuments.mx/reader035/viewer/2022081420/56649f495503460f94c6ac72/html5/thumbnails/26.jpg)
26
End of Lecture 12