18 cv mil_style_and_identity
TRANSCRIPT
Computer vision: models, learning and inference
Chapter 18 Models for style and identity
Please send errata to [email protected]
2
Identity and Style
2Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Identity differs, but images similar
Identity same, but images quite
different
3
Structure
3Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications
4
Factor analysis review
4Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Generative equation:
Probabilistic form:
Marginal density:
5
Factor analysis
5Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
6
Factor analysis review
6Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
E-Step:
M-Step:
7
Factor analysis vs. Identity model
7Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Each color is a different identity• multiple images lie in similar part of subspace
8
Subspace identity model
8Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Generative equation:
Probabilistic form:
Marginal density:
9
Subspace identity model
9Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
10
Factor analysis vs. subspace identity
10Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Factor analysis Subspace identity model
11
Learning subspace identity model
11Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
E-Step:
Extract moments:
12
Learning subspace identity model
12Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
M-Step:
E-Step:
13
Subspace identity model
13Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
14
Subspace identity model
14Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
15
Inference by comparing models
15Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Model 1 – Faces match (identity shared):
Model 2 – Faces dont match (identities differ):
Both models have standard form of factor analyzer
16
Inference by comparing models
16Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Compute likelihood (e.g. for model zero)
where
Compute posterior probability using Bayes rule
Face Recognition TasksPROBE
…
GALLERY
? 1. CLOSED SET FACE IDENTIFICATION
…
GALLERY PROBE
?NO MATCH
2. OPEN SET FACE IDENTIFICATION
PROBE
?NO MATCH
3. FACE VERIFICATION
4. FACE CLUSTERING?
17Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
18
Inference by comparing models
18Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
19
Relation between models
19Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
20
Structure
20Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications
21
Probabilistic linear discriminant analysis
21Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Generative equation:
Probabilistic form:
22
Probabilistic linear discriminant analysis
22Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
23
Learning
23Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
E-Step – write out all images of same person as system of equations– Has standard form of factor analyzer– Use standard EM equation
M-Step – write equation for each individual data point– Has standard form of factor analyzer– Use standard EM equation
24
Probabilistic linear discriminant analyis
24Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
25
Inference
25Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Model 1 – Faces match (identity shared):
Model 2 – Faces dont match (identities differ):
Both models have standard form of factor analyzer
Compute likelihood in standard way
26
Example results (XM2VTS database)
26Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
27
Structure
27Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications
28
Non-linear models (mixture)
28Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Mixture model can describe non-linear manifold.
Introduce variable ci which represents which cluster
To be the same identity, must also belong to the same cluster
29
Non-linear models (kernel)
29Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Pass hidden variable through non-linear function f[ ].• Leads to kernelized algorithm• Identity equivalent of GPLVM
30
Structure
30Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications
31
Asymmetric bilinear model
31Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Introduce style variable sij
• indicates conditions in which data was observed • Example: lighting, pose, expression face recognition
Asymmetric bilinear model
• Introduce style variable sij
• indicates conditions in which data was observed • Example: lighting, pose, expression face recognition
32
Asymmetric bilinear model
32Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
33
Asymmetric bilinear model
33Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Generative equation:
Probabilistic form:
Marginal density:
34
Learning
34Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
E-Step:
M-Step:
35
Asymmetric bilinear model
35Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
36
Inference – inferring style
36Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Likelihood of style
Prior over style
Compute posterior over style using Bayes’ rule
37
Inference – inferring identity
37Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Likelihood of identity
Prior over identity
Compute posterior over identity using Bayes’ rule
38
Inference – comparing identities
38Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Model 1 – Faces match (identity shared):
Model 2 – Faces dont match (identities differ):
Both models have standard form of factor analyzer
Compute likelihood in standard way, combine with prior in Bayes rule
39
Inference – Style translation
39Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Compute distribution over identity
• Generate in new style
40
Structure
40Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications
41
Symmetric bilinear model
41Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Generative equation:
Probabilistic form:
Mean can also depend on style...
42
Symmetric bilinear model
42Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
43
Inference – translating style or identity
43Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
44
Multilinear models
44Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Extension of symmetric bilinear model to more than two factors
e.g.,
45
Structure
45Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications
46
Face recognition
46Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
47
Tensortextures
47Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
48
Synthesizing animation
48Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
49
Discussion
49Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Generative models• Explain data as combination of identity and
style factors • In identity recognition, we build models where
identity was same or different• Other forms of inference such as style
translation also possible