face recognition and biometric systems eigenfaces (3)
TRANSCRIPT
Face Recognition and Biometric Systems
Plan of the lecture
Eigenfaces-based methods Fisherfaces Bayesian Matching Local PCA
Face relevance mapsError function minimisation Eigenfaces – feature extraction definition of recognition error optimal masks and weights
Face Recognition and Biometric Systems
Eigenfaces – drawbacks
Main drawbacks: holistic method face topology not taken into
account statistical analysis of differences
between images in the training set character of differences not taken
into account
Face Recognition and Biometric Systems
Fisherfaces
PCA finds main directions of variance class identity not utilised
Methods based on PCA which utilise class identity: Linear Discriminant Analysis (LDA) Fisherfaces
Face Recognition and Biometric Systems
Fisherfaces
Principal Component Analysis: training set covariance matrix
Linear Discriminant Analysis: classified training set two covar.
matrices within-class covariance matrix between-class covariance matrix
orthogonal basis from two matrices
Face Recognition and Biometric Systems
Fisherfaces
Between-class matrix
CB – between-class covariance matrix
C – number of classesMi – number of images in i-th class
– average imagei – average image of i-th class
C
iiiiB M
1)(( μμμμC
Face Recognition and Biometric Systems
Fisherfaces
Within-class covariance matrix
CW – within-class covariance matrix
C – number of classesXi – set of images which belong to i-th class
xk – k-th image which belongs to i-th class
i – average image of i-th class
C
i XikikW
ik1)((
xμxμxC
Face Recognition and Biometric Systems
Fisherfaces
PCA:
- eigenvectors matrix (vectors in columns)
LDA:
vv C
||maxarg TCψψψψ
||
||maxarg
ψCψ
ψCψψ
ψW
TB
T
vv WB CC vvBW CC 1
Face Recognition and Biometric Systems
Fisherfaces
LDA – hard to find inverse matrixFisherfaces – improved approach: PCA for dimensionality reduction LDA for finding optimal orthogonal
basis
|''|
|''|maxarg
ψψCψψ
ψψCψψψ
ψPCAW
TPCA
TPCAB
TPCA
T
Face Recognition and Biometric Systems
Fisherfaces
Feature extraction in the Fisherfaces:
1. Feature vector calculated by PCA normalised image as an input dimensionality reduction
2. Feature vector calculated by LDA PCA feature vector as an input rotation of feature vector no dimensionality reduction
Face Recognition and Biometric Systems
Bayesian Matching
Vectors similarity based on probability of their difference classification
I – set of intra-personal pairs E – set of extra-personal pairs
)|()(),( 21 II PPIIS
21 II
Face Recognition and Biometric Systems
Bayesian Matching
)()|()()|(
)()|()|(
EEII
III PPPP
PPP
P(|) – probability of observing a given difference in a defined set of differences function of PCA back projection error –
() 2)(~)|( eP
Face Recognition and Biometric Systems
Bayesian Matching
Two classes of image pairs intra- and extra-personal
Differences generated from pairs two classes of pairs
PCA used for both classes separately two image spaces
Face Recognition and Biometric Systems
Bayesian Matching
Image difference recognition Dual Eigenfaces
Difference distance from two image spacesBayesian Matching – a slow method image difference calculated for every
comparison possibility of applying other method for
selecting candidates (n most similar images)
Face Recognition and Biometric Systems
Local PCA
Based on detected features eyes, nose, mouth
PCA for features small part of face image analysis of small images (eigeneyes,
eigennoses, etc.)
Less dimensional spacesLower effectiveness, but supports the Eigenfaces
Face Recognition and Biometric Systems
Other methods
Local Feature Analysis2D PCA, 2D LDAIndependent Component Analysis
Face Recognition and Biometric Systems
Face relevance map
Face topology eyes & nose – extra-personal
differences mouth & cheeks – intra-personal
differences
Nature of features concerned with location
Face Recognition and Biometric Systems
Face relevance map
Face relevance map enhance influence of extra-personal
features decrease influence of intra-personal
features
Feature extraction with a map (m)
N
jjjiji mxuw
1x ii uw
Face Recognition and Biometric Systems
Face relevance map
„T” map artificial map for eyes and nose binary values
Results: FeretA: 423 -> 445 (3,7%)
Conclusion: good approach, need for better map generation methods
Face Recognition and Biometric Systems
Face relevance mapDifference map – statistical analysisPairs of images: intra-personal extra-personal
Average differences between images: average intra-personal difference average extra-personal difference
Map obtained by subtracting intra-personal difference from extra-personal oneResults for FeretA: 423 -> 462 (6,4%)
Face Recognition and Biometric Systems
Face relevance map
Colour data information lost during conversion
to GS low distinctiveness can be used for map generation
Colour used for detection eye and mouth map masks based on detection maps
Face Recognition and Biometric Systems
Face relevance map
Desired effect: higher values around eyes and nose lower values in the area of mouth
Maps deliver information about features locationTwo possible approaches: image -> feature maps -> face relevance
map image -> feature maps -> features -> f.r.m.
Face Recognition and Biometric Systems
Face relevance map
Maps from pointsNose location derived from eye & mouth weighted mean
eye(R): (15, 24)eye(L): (49, 24)mouth: (32, 58)
Face Recognition and Biometric Systems
Face relevance map
Single point influence
r – radius, mmax – maximal map value
Map – summed influence of the points eye, nose – positive weights mouth – negative weights
20
20
),(max
)()(),(
),(
yydxxdyxD
emyxm
yx
yxDr
Face Recognition and Biometric Systems
Face relevance map
Maps from colour improvement comparable to
difference maps colour data carry information
concerning nature of face areas generated for every image
Map may be imposed during normalisation
Face Recognition and Biometric Systems
Face relevance map
Back-projection based dynamic map dynamic – created for every image
Back projection: map of local projection error higher error = lower importance map should be smoothed
Good for occluded images
Face Recognition and Biometric Systems
Face relevance map
Back-projection based dynamic map examples of occluded face images
Face Recognition and Biometric Systems
Recognition errorMaps take into account difference nature basing on face topologyDifferences not concerned with location lighting
Eigenfaces – appearance interpretation various types of information some responsible for lighting
Weights assigned to eigenvectors:
N
jjjijii mxuw
1
Face Recognition and Biometric Systems
Recognition error
Eigenvectors weights lower values for intra-personal
directions of variance
How to obtain the weights? visual assessment – may be
incorrect the same procedure as in the case
of difference masks
Face Recognition and Biometric Systems
Recognition error
A better method for obtaining maps and eigenvector weights: error function minimisation
Face Recognition and Biometric Systems
Recognition error
Definition of recognition problem: M vectors, C classes and C base vectors
(ui1) Mi vectors in i-th class (uij) classification of non-base vectors (j > 1)
Single comparison similarity to home class and foreign class classes represented by base vectors
Face Recognition and Biometric Systems
Recognition error
Single comparison error:
uij – a vector which is being recognised
ui1 – home class base vector
uk1 – foreign class base vectorS – similarity between vectors
),(
),(),(
1
11
iij
kijkij S
Sd
uu
uuuu
Face Recognition and Biometric Systems
Recognition error
Single comparison:
correct if
incorrect if
1),( 1 kijd uu
1),( 1 kijd uu
Face Recognition and Biometric Systems
Recognition error
Error for comparison with all classes:
Error for the whole set:
C
i
M
j
C
ikk
ij
i
kdD1 2 1
),( 1uu
C
ikk
ijij kdD1
),()( 1uuu
Face Recognition and Biometric Systems
...
Scalar products betweennormalised image and
eigenvectors
...
K1
K2
K3
Feature vector
Eigenfaces: feature extraction
Face Recognition and Biometric Systems
Feature vector element ( ):
- dimensionality of feature vector
- normalised face image- i-th eigenvector
Improvements to the Eigenfaces face relevance masks eigenvector weights
xv Tiiu
Eigenfaces: feature extraction
li ,1lx
iv
iu
Face Recognition and Biometric Systems
Eigenfaces: feature extraction
Improved feature extraction:
- i-th eigenvector weight- j-th element (pixel) of the mask- j-th element of the i-th
eigenvector- i-th element of the feature
vector
N
jjjijii mxvu
1 li ,1
ijmijviu
Face Recognition and Biometric Systems
Eigenfaces: feature extraction
Similarity based on Euclidean distance:
l
iii uu
S
1
221
21
)(1
1),( uu
l
ppkijp
l
ppiijp
kij
uu
uud
1
21
1
21
1
)(1
)(1),( uu
Face Recognition and Biometric Systems
Error minimisation
Recognition error is a function of mask and eigenvector weightsThe function may be minimised optimal mask optimal eigenvector weights
Example of mask optimisation...
Face Recognition and Biometric Systems
Error minimisation
Optimised dataset problem of overfitting
How to avoid overfitting? large datasets optimisation can be stopped
Advantages of overfitting overfitting to a group of people
40%45%50%55%60%65%70%75%80%85%90%95%
100%
-1 9 19 29
Iteration
Cla
ss
ific
ati
on
eff
ec
tiv
en
es
s Optimised set
FeretA
FeretC
Notre-Dame -connected images
Notre-Dame - imagesnot connected
Face Recognition and Biometric Systems
Summary
There are many methods derived from the EigenfacesError is a function of masks and eigenvectors weightsClassification parameters can be optimisedImprovement aims at: including face topology feature analysis difference classification