fundamentals of media processingresearch.nii.ac.jp/~satoh/fmp/fmp_2012_10_23.pdffisherface method...
TRANSCRIPT
![Page 1: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/1.jpg)
Fundamentals of Media Processing
Shin'ichi SatohKazuya KodamaHiroshi MoDuy-Dinh Le
![Page 2: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/2.jpg)
Course web page
http://research.nii.ac.jp/~satoh/fmp/
![Page 3: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/3.jpg)
Today's topics
Face detection and recognition PCA and Eigenface method LDA and Fisherface method Relation to Bayes decision rule assuming
normal distribution Application of Eigenvalue decomposition Searching document by the latent semantic
information: Latent Semantic Indexing (LSI) Estimating 3-D structure from video:
Factorization method
![Page 4: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/4.jpg)
Face detection and recognition
Assume that we have a set of face images (with identity)
Face detection: decide if given unknown image is face or not
Face recognition: decide the identity of given face image
![Page 5: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/5.jpg)
![Page 6: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/6.jpg)
Image as intensity data
![Page 7: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/7.jpg)
Image conversion
original gray scale crop
discretize quantize
x=[x1 x2 x3 ... xn]
![Page 8: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/8.jpg)
The Space of Faces
An image is a point in a high dimensional space An N x M image is a point in RNM
We can define vectors in this space as we did in the 2D case
+=
[Thanks to Chuck Dyer, Steve Seitz, Nishino]
![Page 9: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/9.jpg)
Multivariate normal distribution
We typically assume normal distribution Let's assume p(x|non-face) yields uniform
distribution, and p(x|face) yields normal distribution
multivariate normal distribution
)()(
21exp
)2(1)( 1T
2/12/μxμxx
dp
![Page 10: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/10.jpg)
Normal distribution
![Page 11: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/11.jpg)
Eigenface method
Key idea: observe distribution of face images in the image space
find principal components by principal component analysis (PCA)
project unknown images onto the space spanned by the obtained principal components
face detection: decide face if it's close enough to the space
face recognition: decide the identity of the closest face image within the space
![Page 12: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/12.jpg)
Eigenface method
Assume we have samples of faces:X=[x1 x2 x3 ... xn]
We can then obtain covariance matrix:
n
in 1
1E ixxμ
TT ))((1)(E μμμμ)(xx XXn
--
![Page 13: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/13.jpg)
Eigenface method
We then apply eigenvalue decomposition to the covariance matrix
where λi are eigenvalues and φi are eigenvectors Retain m eigenvectors (φi , i=1... m) corresponding
to the m largest eigenvalues Then each image x can be converted into m-
dimensional vector
iii
m1
)(T μxx'
![Page 14: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/14.jpg)
Covariance matrix and its algebraic/geometric interpretation What is the quadratic form ?
x1
x2x3
y1 y2 y3
μ
φ
![Page 15: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/15.jpg)
Covariance matrix and its algebraic/geometric interpretation How to maximize wrt ?
Lagrange multipliers method
very good exercise. pls try!
![Page 16: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/16.jpg)
J. M. Rehg © 2002
Principle Component Analysis
PCA is an orthonormal projection of a random vector X onto a lower-dimensional subspace Y that minimizes mean square error.
YUXeXUY
uuuU
uuΣXXΣ
,
},{
21
21
Tm
n
iiiT
nm
E
1x
2x1y
2y
![Page 17: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/17.jpg)
J. M. Rehg © 2002
Principle Component Analysis
Equivalently, PCA yields a distribution for Y with Uncorrelated components Maximum variance
YUXeXUY
uuuU
uuΣXXΣ
,
},{
21
21
Tm
n
iiiT
nm
E
1x
2x1y
2y
![Page 18: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/18.jpg)
J. M. Rehg © 2002
Classification Using PCA
Detection of faces based on distance from face spaceRecognition of faces based on distance within face space
YUXeXUY
uuuU
uuΣXXΣ
,
},{
21
21
Tm
n
iiiT
nm
E
1x
2x1y
2y
?DTe
?Ri Tyy
![Page 19: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/19.jpg)
Eigenfaces PCA extracts the eigenvectors of A
Gives a set of vectors v1, v2, v3, ... Each one of these vectors is a direction in face space
what do these look like?
![Page 20: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/20.jpg)
Projecting onto the Eigenfaces
The eigenfaces v1, ..., vK span the space of faces
A face is converted to eigenface coordinates by
![Page 21: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/21.jpg)
Fisherface method
Eigenface finds principal components which maximize the variance of face distribution
What happens if we use identity information? it would be reasonable idea to maximize the
variance between different people, while minimize the variance within the same people
find such components by Linear Discriminant Analysis (LDA)
project unknown images onto the space spanned by the obtained components
face recognition (only): decide the identity of the closest face image within the space
![Page 22: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/22.jpg)
Fisherface -2
Poor Projection Good Projection
![Page 23: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/23.jpg)
FisherFace - 3
N Sample images: C classes:
Average of each class:
Total average:
Nxx ,,1
c ,,1
ikx
ki
i xN
1
N
kkx
N 1
1
![Page 24: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/24.jpg)
FisherFace - 4
Scatter of class i: Tikx
iki xxSik
c
iiW SS
1
c
i
TiiiBS
1
BWT SSS
Within class scatter:
Between class scatter:
Total scatter:
![Page 25: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/25.jpg)
FisherFace - 5
After projection:
Between class scatter (of y’s):Within class scatter (of y’s):
kT
k xWy
WSWS BT
B ~
WSWS WT
W ~
![Page 26: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/26.jpg)
FisherFace - 6
Good separation
2S
1S
BS
21 SSSW
![Page 27: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/27.jpg)
FisherFace - 7
The wanted projection:
WSW
WSW
SS
WW
TB
T
W
Bopt WW
max arg~~
max arg
miwSwS iWiiB ,,1
How is it found ?
![Page 28: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/28.jpg)
Experimental Results - 1
Variation in Facial Expression, Eyewear, and Lighting
Input: 160 images of 16 people Train: 159 images Test: 1 image
With glasses
Without glasses
3 Lighting conditions
5 expressions
![Page 29: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/29.jpg)
Experimental Results - 2
![Page 30: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/30.jpg)
Normal distribution
![Page 31: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/31.jpg)
Singular Value Decomposition
orthogonalorthogonal
diagonal
![Page 32: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/32.jpg)
Some Properties of SVD
![Page 33: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/33.jpg)
Some Properties of SVD
• That is, Ak is the optimal approximation in terms of the approximation error measured by the Frobenius norm, among all matrices of rank k
• Forms the basics of LSI (Latent Semantic Indexing) in informational retrieval
![Page 34: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/34.jpg)
Low rank approximation by SVD
![Page 35: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/35.jpg)
Applications of SVD
Pseudoinverse Range, null space and rank Matrix approximation Other examples
http://en.wikipedia.org/wiki/Singular_value_decomposition
![Page 36: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/36.jpg)
LSI (Latent Semantic Indexing)
Introduction Latent Semantic Indexing
LSI Query Updating
An example
![Page 37: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/37.jpg)
Problem Introduction
Traditional term-matching method doesn’t work well in information retrieval
We want to capture the concepts instead of words. Concepts are reflected in the words. However, One term may have multiple meaning Different terms may have the same meaning.
![Page 38: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/38.jpg)
LSI (Latent Semantic Indexing)
LSI approach tries to overcome the deficiencies of term-matching retrieval by treating the unreliability of observed term-document association data as a statistical problem.
The goal is to find effective models to represent the relationship between terms and documents. Hence a set of terms, which is by itself incomplete and unreliable, will be replaced by some set of entities which are more reliable indicants.
![Page 39: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/39.jpg)
LSI, the Method
Document-Term M Decompose M by SVD. Approximating M using truncated SVD
![Page 40: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/40.jpg)
LSI, the Method (cont.)
Each row and column of A gets mapped into the k-dimensional LSI space, by the SVD.
![Page 41: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/41.jpg)
Query
A query q is also mapped into this space, by
Compare the similarity in the new space Intuition: Dimension reduction through LSI
brings together “related” axes in the vector space.
1 kkT
k Uqq
![Page 42: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/42.jpg)
Example
![Page 43: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/43.jpg)
Example (cont.)
![Page 44: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/44.jpg)
Example (cont. Mapping)
![Page 45: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/45.jpg)
Example (cont. Query)
Query: Application and Theory
![Page 46: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/46.jpg)
Example (cont. Query)
![Page 47: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/47.jpg)
How to set the value of k?
LSI is useful only if k << n. If k is too large, it doesn't capture the
underlying latent semantic space; if k is too small, too much is lost.
No principled way of determining the best k.
![Page 48: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/48.jpg)
How well does LSI work?
Effectiveness of LSI compared to regular term-matching depends on nature of documents. Typical improvement: 0 to 30% better precision. Advantage greater for texts in which synonymy and ambiguity
are more prevalent. Best when recall is high.
Costs of LSI might outweigh improvement. SVD is computationally expensive; limited use for really large
document collections Inverted index not possible
![Page 49: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/49.jpg)
References
Mini tutorial on the Singular Value Decomposition http://www.cs.brown.edu/research/ai/dynamics/tutorial/Post
script/SingularValueDecomposition.ps
Basics of linear algebra http://www.stanford.edu/class/cs229/section/section_lin_alg
ebra.pdf
![Page 50: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/50.jpg)
Factorization
C. Tomasi and T. Kanade. Shape and motion from image streams under orthography---a factorization method. International Journal on Computer Vision, 9(2):137-154, November 1992. also as Technical Report TR-92-1270, Cornell University, March 1992.
![Page 51: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/51.jpg)
Motivation
Given: an image sequence of a particular object
Automatically extract the 3-D structure of the object
Useful for object identification, 3-D model construction for CG, CAD, etc.
![Page 52: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/52.jpg)
Example
![Page 53: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/53.jpg)
First Step: Feature Point Selection and Tracking
Feature Point SelectionFeature Point Tracking
![Page 54: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/54.jpg)
Feature Point Tracking
Select good points to track We regard a given image sequence as the
following spatio-temporal function: Pixel correspondence:
If we define:
...
),,( tyxI
),,(),,( tyxItyxI
),,()( tyxIJ x),,()( tyxII dx
![Page 55: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/55.jpg)
),,( tyxI ),,( tyxI
),( d
The problem is then to find that minimizes:d
W
wJI xxdx d)]()([ 2
W
![Page 56: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/56.jpg)
If we assume that the displacement vector is small,
dgxdx )()( II
where ),(yI
xI
g
then what we need to solve becomes:ed G
W
wG xgg dT W
wJI xge d)(where and
![Page 57: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/57.jpg)
Feature Point Selection
Good feature points should provide stable solution for tracking: from
G should have numerically stable inverse Assume two eigenvalues of G be The criterion for feature point selection is:
d ed G
21,
),min( 21
![Page 58: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/58.jpg)
Measurement Matrix
f-th frame (f+1)-th framep-th point p-th point),( fpfp vu ),( )1()1( pfpf vu
)(
)(~~~
fpfp
fpfp
vv
uuVUW
P
F
F
f=1,…,Fp=1,…,P
![Page 59: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/59.jpg)
The Rank Theorem
![Page 60: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/60.jpg)
The Rank Theorem
pT
ffpfp
pT
ffpfp
vv
uu
sj
si
under orthographic projection,
P
F
FRSW ss
j
ji
i
1
T
T1
T
T1
~
thus
![Page 61: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/61.jpg)
The Rank Theorem
How we decompose ? by using the singular value decomposition
(SVD)
then
W~
BAW ~
BS
AR2/1
2/1
ˆ
ˆ
![Page 62: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/62.jpg)
Results
![Page 63: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/63.jpg)
Results
![Page 64: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/64.jpg)
Results
![Page 65: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/65.jpg)
Schedule (topics subject to change) 10/9(today) orientation, Bayes decision theory, probability
distribution, normal distribution 10/16 random vector, linear algebra, orthogonal expansions,
principal component analysis(PCA) 10/23 Applications of Eigenvalue decomposition (or Singular Value
Decomposition): Latent Semantic Indexing and the Factorization method
10/30 no class 11/6 advanced pattern recognition #1 11/13 advanced pattern recognition #2 11/20 Nonparametric methods (Parzen windows, k-nearest neighbor
estimate, k-nearset neighbor classification) 11/27 advanced pattern recognition #3 12/4 no class 12/11 Nonparametric methods contd., Clustering (k-Means,
agglomerative hierarchical clustering)
![Page 66: Fundamentals of Media Processingresearch.nii.ac.jp/~satoh/fmp/FMP_2012_10_23.pdfFisherface method Eigenface finds principal components which maximize the variance of face distribution](https://reader034.vdocuments.mx/reader034/viewer/2022042112/5e8d3dd3b03bda20307b51b8/html5/thumbnails/66.jpg)
Schedule cont'd
12/18 advanced pattern recognition #4 12/25 advanced pattern recognition #5 2013/1/8 no class 1/15 (Fundamentals of) Signal processing 1/22 (Fundamentals of) Image processing 1/29 3-D/4-D Image processing 2/5 Project 2/12 Discussion