face detection, pose estimation and landmark localization ... · face detection, pose estimation...
TRANSCRIPT
![Page 1: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/1.jpg)
Face detection, pose estimation and landmark
localization in the wild
Presenter: Shuai Zheng (Kyle)
Paper: X. Zhu and D. Ramanan in CVPR 2012
![Page 2: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/2.jpg)
Many Applications of Face Det, Pose Est, Landmarks Loc.
![Page 3: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/3.jpg)
Microsoft Face Game
Google Picasa’s Face Movie
Face.com App (Facebook)
Hot Area
Face
Apps
and Facial expression recognition, etc…
…...
![Page 4: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/4.jpg)
Face Recognition Pipeline
How far is our technique from 100% accuracy
face recognition (object recognition) system?
Name: Andrea? Gender: Male Age: 24? Has beard?
Assume the previous step is perfect.
Overly optimistic!
R. Jenkins and A. M. Burton, 100% accuracy in automatic face recognition, Science, 25 Jan, 2008.
![Page 5: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/5.jpg)
Face recognition in the wild
• Face presents different appearances and
shapes under different viewpoints;
![Page 6: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/6.jpg)
Face recognition in the wild
• Face presents different appearances and
shapes under different elastic deformation.
![Page 7: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/7.jpg)
Problems about Face App in the wild
• Optimizing all isolated components in a
computer vision system is very difficult.
• Viewpoints problem
• Elastic deformation problem
• Do we need to collect billions of
low-quality data to get state-of-the-
art?
![Page 8: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/8.jpg)
Structured
SVM with
mixtures of
trees
Joint Approach
Joint Detection, landmarks localization and pose
estimation.
![Page 9: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/9.jpg)
Model viewpoints with mixtures of trees
![Page 10: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/10.jpg)
Model viewpoints with mixtures of trees
![Page 11: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/11.jpg)
Model elastic deformations with trees
![Page 12: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/12.jpg)
Pictorial Structured Model
![Page 13: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/13.jpg)
Inference
𝑚∗ : the estimated viewpoint.
𝐿∗ : the estimated landmark locations.
Search over scales using an image pyramid.
![Page 14: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/14.jpg)
Learning
![Page 15: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/15.jpg)
Chow-Liu algorithm is an efficient method for
constructing a second-order product
approximation of a joint distribution.
Learning tree with Chow-Liu Alg
Joint probability distribution 𝑃 𝑋1, . . , 𝑋𝑛 can
be described as a product of second-order
conditional and marginal distributions. As
shown in the figure,
![Page 16: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/16.jpg)
Learning with structured SVM
![Page 17: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/17.jpg)
Problem Formulation
Given labeled positive examples {𝐼𝑛, 𝐿𝑛, 𝑚𝑛} and negative examples {In}, Lets write zn = {Ln, 𝑚𝑛}. Score function in is linear in the part templates 𝑤 , spring parameters (a, b, c, d) and mixture biases 𝛼. Concatenated all the parameters into 𝛽. We can formulate the problem as
![Page 18: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/18.jpg)
Experimental Results
![Page 19: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/19.jpg)
Experimental Results
![Page 20: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/20.jpg)
Experimental Results
![Page 21: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/21.jpg)
Experimental Results
![Page 22: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/22.jpg)
Experimental Results
![Page 23: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/23.jpg)
Experimental Results
![Page 24: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/24.jpg)
Experimental Results
![Page 25: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/25.jpg)
Experimental Results
![Page 26: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/26.jpg)
Experimental Results
![Page 27: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/27.jpg)
Conclusions
Pros:
• Model the view-specific within
mixtures of trees.
• Joint method to do face detection,
pose estimation, and landmarks
localization for face images with
viewpoint variations and elastic
deformation.
![Page 28: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/28.jpg)
Conclusions
Cons:
• Slow in the inference, given one
image (80*80), it takes more than 20
seconds to process.
• Cannot handle large size images.
![Page 29: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/29.jpg)
Conclusions
Messages:
• Tree-structure elastic model can do
many jobs together.
• Matching small patch is much easier
than matching the object of interest.
• Training model on selective
supervised data is the key to
success.
![Page 30: Face detection, pose estimation and landmark localization ... · Face detection, pose estimation and landmark localization in the wild Presenter: Shuai Zheng (Kyle) Paper: X. Zhu](https://reader033.vdocuments.mx/reader033/viewer/2022051409/601c40ad1b05691d212d39d8/html5/thumbnails/30.jpg)
END