robust face via sparse representation
TRANSCRIPT
![Page 1: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/1.jpg)
Robust Face Recognitionvia Sparse Representation
Authors: John Wright, Allen Y. Yang, Arvind Ganesh,S. Shankar Sastry, and Yi Ma
Presented byHu Han
Jan. 30 2014
1For CSE 902 by Prof. Anil K. Jain: Selected Topics in Recognition by Machine
![Page 2: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/2.jpg)
Outline
• Background• Formulation of face recognition via sparse representation
• Experimental results• Summary and discussion
2
![Page 3: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/3.jpg)
What is automatic face recognition (AFR)?
• Automatically identify or verify a person from still images, video sequences or even sketches
3
Probe Gallery
The same person or not?
Verification
Who is this person?
Identification
Query Target
1 vs. 1Comparison
![Page 4: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/4.jpg)
Formulation of face identification
• Traditional formulation: Nearest neighbor classifier–
…,
– , is a similarity measure between the probe image and a gallery image
– Image is of vectorized form ( )
4
The traditional formulation is to assign the identity of themost similar face image in the gallery set to a probe faceimage.
![Page 5: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/5.jpg)
Formulation of face identification
• Can we formulate the face identification problem based on spare representation?
– where a signal is coded over a dictionary as , and is a sparse coefficient vector
5
Probe Gallery
Who is the person in a probe ?
![Page 6: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/6.jpg)
Formulation of face identification
• We can, provided a probe face image can be represented using a linear combination of the gallery face images
6
![Page 7: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/7.jpg)
Mathematical fundamentals
• Mean (the average face)
–
• Covariance matrix of the gallery set
–
– (The normalized gallery set)
7
![Page 8: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/8.jpg)
Mathematical fundamentals
• Rank– The maximum number of linearly independent column or row vectors in a matrix
– One of the important properties of real matrices
8
has the same number ( ) of non‐zero eigenvalues as
![Page 9: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/9.jpg)
Mathematical fundamentals
• Eigenvectors and eigenvalues of the gallery set
• Principal component analysis (PCA)
9
Is the second term also a linear combination of all the gallery samples ?
The first term is a linear combination of all the gallery samples
![Page 10: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/10.jpg)
Rationality analysis
• is reasonable or not?
• Denote ; is real and symmetric, and thus it has non‐zero eigenvalues
10
![Page 11: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/11.jpg)
Rationality analysis
11
=
![Page 12: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/12.jpg)
Rationality analysis
•– The eigenvectors of a set of gallery face images
is the linear combination of its original data , and is the coefficients for linear combination
• PCA model
12
The first term is a linear combination of all the gallery samples
The second term si also a linear combination of all the gallery samples
![Page 13: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/13.jpg)
Formulation of face identification
• Therefore, it is reasonable to represent a probe face image using a simple linear combination of gallery face images
• We can also reversely solve the linear combination coefficients based on a sparse representation formulation
13
![Page 14: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/14.jpg)
Formulation of face identification
• The sparse representation itself is not a classifier
• But a classifier can be built based on the solved sparse coefficients
14
![Page 15: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/15.jpg)
Classification using the sparse coefficients
• One classification scheme that we can naturally think of
15
12x10downsampling
argmax…
The extended YaleB database: ~1200 training images of 38 subjects
, , … ,
Magnitude
![Page 16: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/16.jpg)
Classification using the sparse coefficients
• The classification scheme proposed in Wright et al.’s paper:
16
argmin…
argmin…
∙ selects the coefficients associated with the i‐th subject(totally subjects)
12x10downsampling
, , … ,
Magnitude
![Page 17: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/17.jpg)
Sparse Representation‐based Classification (SRC) Algorithm
17• In SRC, the training set is also the gallery set. All face images must be aligned.• I am using I_p instead of y for a probe image.
![Page 18: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/18.jpg)
Why sparse representation?
• In practice, the linear system is typically underdetermined, because there are fewer equations ( ) than unknowns ( )
18
, , ,, , ,
, , ,
m= 12x10=120downsampling
n=1200
![Page 19: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/19.jpg)
Why sparse representation?
• An underdetermined system either has no unique solution or has no solution– An example with no solution
– But for the linear system in face recognition, we assume there is at least one solution
19
Inconsistent
![Page 20: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/20.jpg)
Why sparse representation?
• If there is no sparsity constraint
20
…
m= 12x10=120downsampling
is not discriminative, and it does not indicate any identity of the probe image;The nonzero entries in are very unstable;
![Page 21: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/21.jpg)
Experimental evaluations
• Face recognition on Extended Yale B database – 2,414 images (192x168) of 38 subjects– Half (~ 32) of the images per subject used for training (gallery), the other half for testing
– Thus, the dictionary size (n) is ~1200– Features: Eigenfaces, Laplacianfaces, Randomfaces, Fisherfaces, and raw intensities with m= 30, 56, 120, and 504, respectively
– SRC is compared with Nearest Neighbor, Nearest Subspace, and Linear SVM
21
![Page 22: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/22.jpg)
A brief description of different feature extraction methods
• Eigenfaces– Project a face image into a low‐dimensional subspace that keeps the largest the data variations
• Laplacianfaces– Project a face image into a low‐dimensional subspace that keeps the local relationship in high‐dimensional domain
• Randomfaces– Project a face image into a new subspace using a Gaussian random matrix
• Fisherfaces– Project a face image into a low‐dimensional subspace that best separates individual classes
22
![Page 23: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/23.jpg)
Face recognition on Extended Yale B
• Example images of Extended Yale B
23http://vision.ucsd.edu/~leekc/ExtYaleDatabase/ExtYaleB.html
Extreme illumination
![Page 24: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/24.jpg)
Face recognition on Extended Yale B
24
Good feature extraction method is critical for SVM
![Page 25: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/25.jpg)
Experimental evaluations
• Face recognition on AR– 14,00 images (165x120) of 100 subjects (50 males & 50 females)
– 7 images per subject used for training (gallery), 7 images for testing
– Thus, the dictionary size (n) is ~700– Features: Eigenfaces, Laplacianface, Randomfaces, Fisherfaces, and raw intensities with m= 30, 56, 120, and 504, respectively
– SRC is compared with Nearest Neighbor, Nearest Subspace, and Linear SVM
25
![Page 26: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/26.jpg)
Face recognition on AR
• Example images of AR
26http://www2.ece.ohio‐state.edu/~aleix/ARdatabase.html
Illumination, expression, occlusions
![Page 27: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/27.jpg)
Face recognition on AR
27
![Page 28: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/28.jpg)
Experimental evaluations
• Face recognition on Extended YaleB with random pixel corruption– 1,170 images (96x84) of 38 subjects– 717 images used for training (gallery) and 453 images for testing
– Thus, the dictionary size is ~717– Percentage of random pixel corruption: 0%, 10%, 20%,…,90%
• Replace a pixel intensity with uniformly sampled value from [0, max‐pixel intensity]
– Feature: raw intensity (8064 dimensions)
28
![Page 29: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/29.jpg)
Face recognition on Extended YaleBwith random pixel corruption
• Example images with random pixel corruption
29
30%
50%
70%
![Page 30: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/30.jpg)
Face recognition on Extended YaleBwith random pixel corruption
30
![Page 31: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/31.jpg)
Experimental evaluations
• Face recognition on Extended YaleB with random block occlusion– 1,170 images (96x84) of 38 subjects– 717images for training (gallery) and 453 images for testing
– Thus, the dictionary size is ~717– Percentage of random block size: 0%, 10%, 20%,…,50%
• The location of occlusion square block is random– Feature: raw intensity (8064 dimensions)
31
![Page 32: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/32.jpg)
Face recognition on Extended YaleBwith random block occlusion
• Example images with random block occlusion
32
30%
30%
30%by LS
![Page 33: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/33.jpg)
Face recognition on Extended YaleBwith random block occlusion
33
![Page 34: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/34.jpg)
Experimental evaluations
• Face recognition on Extended Yale B database using partial face (component) features
34
The holistic faces arealready aligned witheach other. Therefore,component images arealso aligned
Feature: raw intensity
![Page 35: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/35.jpg)
Experimental evaluations
• Face recognition on AR database using block based features
35Feature: raw intensity
Partition is based on the aligned holistic images, and thus each block is aligned
![Page 36: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/36.jpg)
Extension of SRC
• Sparse representation based rejection of invalid face images– The subject in a probe image is not enrolled in the gallery set
– A probe image is not a face image
• Sparsity concentration index (SCI):
36
…
![Page 37: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/37.jpg)
Sparsity concentration index (SCI)
• Rejection criterion: • Physical explanation: If a testing face image can be well represented by a single subject in the gallery set, the testing subject has been enrolled in the system; otherwise the subject has not been enrolled in the gallery
37
![Page 38: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/38.jpg)
Experimental evaluations
• Invalid face rejection on Extended YaleB with random block occlusion– 1,170 images (96x84) of 38 subjects– 360 images of 19 subjects for training (gallery) and 453 images of 38 subjects for testing
– Percentage of random block size: 0%, 10%, 30%, and 50%
• The location of occlusion square block is random
– Feature: raw intensity
38
![Page 39: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/39.jpg)
Invalid face rejection on Extended YaleB with random block occlusion
39
0% 10%
30%50%
![Page 40: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/40.jpg)
Summary and discussion
• Sparsity constraint in SRC assures the discriminative characteristics of solved coefficients for face recognition
• SRC is robust to various feature extraction methods, such as Eigenfaces, Laplacianface, Randomfaces, Fisherfaces, and raw intensities. SVM achieves similar performance as SRC but SVM is sensitive to the feature extraction methods
• In the linear system , the dictionary should be significantly overcomplete ( ) in order to get good performance
• The SRC method requires that the face images are aligned 40
![Page 41: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/41.jpg)
Resources• L‐1 minimization algorithms (code)
– http://www.eecs.berkeley.edu/~yang/software/l1benchmark/
• Talks on sparse representation– http://www.cs.technion.ac.il/~elad/talks/
• Discussions on face recognition using SRC– R. Rigamonti, et al., Are sparse representations really relevant for image classification?, CVPR’2011
– Qinfeng Shi, et al, Is face recognition really a Compressive Sensing problem?, CVPR’2011
– Lei Zhang, et al. Sparse representation or collaborative representation: Which helps face recognition? ICCV’2011.
![Page 42: Robust Face via Sparse Representation](https://reader030.vdocuments.mx/reader030/viewer/2022020703/61fb2792fe39fa2216217084/html5/thumbnails/42.jpg)
Q&AThanks!
42