registration of 3d faces leow wee kheng cs6101 ay2012-13 semester 1
DESCRIPTION
Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1. Main Paper. T. J. Hutton, B. F. Buxton and P. Hammond. Automated Registration of 3D Faces using Dense Surface Models. In Proc. British Machine Vision Conference , 2003. Motivation. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/1.jpg)
1
Registration of 3D Faces
Leow Wee KhengCS6101 AY2012-13 Semester 1
![Page 2: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/2.jpg)
Main Paper T. J. Hutton, B. F. Buxton and P. Hammond.
Automated Registration of 3D Faces using Dense Surface Models.In Proc. British Machine Vision Conference, 2003.
2
![Page 3: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/3.jpg)
Motivation 3D face model useful for many applications:
animationmotion trackingface recognitionface reconstructionsurgery planning & simulationforensic reconstruction…
3
![Page 4: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/4.jpg)
Motivation Build 3D face model from training samples:
Need to align them: registration.
4
![Page 5: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/5.jpg)
Motivation Can’t just align spatially:
Everything is messed up!
Need to align nose to nose, eyes to eyes, …
5
![Page 6: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/6.jpg)
Motivation
Two general kinds of registration: Rigid registration
Objects differ by scale, rotation, translation.No change in shape during registration.Easy to solve.
Non-rigid registrationObjects differ by scale, rotation, translation, shape.Must change shape during registration.Harder to solve.
6
![Page 7: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/7.jpg)
Motivation One possibility: manually mark landmark points.
Very tedious and time-consuming!
Need automatic method!
7
![Page 8: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/8.jpg)
Focus 3D model has shape and texture. Focus on shape, leave out texture
8
![Page 9: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/9.jpg)
Related Work ICP [Besl92, Feldmar96]
Global alignment, not landmark correspondence.
Mesh parameterisation [Brett97,98; Lorenz99,00; Praun01, Davies02]
Re-mesh, rearrange mesh points consistentlyTheir landmark = re-parameterised mesh points
facial landmark.
Shape features [Johnson99, Wang00, Yamany02, ]
Surface curvature, geodesic distance, spin image;not landmark correspondence.
9
![Page 10: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/10.jpg)
3D Face Registration Main ideas of Hutton et al.:
Manually place 10 landmarks on training samples.Use landmark correspondence to compute mapping.Interpolate other points: thin-plate spline.
10
?
![Page 11: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/11.jpg)
Mean Landmarks Compute mean landmarks of training samples. Procrustes alignment:
Compute best alignment by similarity transformation,i.e., scaling, translation, rotation.
Align landmarks of all training samples.Compute mean of landmarks.
11
![Page 12: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/12.jpg)
Dense Correspondence Main steps:
Warp mesh by thin-plate spline so thatlandmarks coincide with mean landmarks.
12
![Page 13: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/13.jpg)
Resample warped mesh using reference mesh.Unwarp resampled mesh.Now, training samples have consistent mesh vertices.Some mesh vertices are facial landmarks.Now, can apply PCA on all mesh vertices.
Dense Correspondence
13
![Page 14: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/14.jpg)
Statistical face model Main steps:
Align all resampled training samples.Perform PCA.Keep top principal components.Normally,
Hutton et al. used
14
Φbxx shape parameters
ΦWbxx
k ,,diag 1 W
unwhitening matrix
![Page 15: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/15.jpg)
Model Fitting Fit mean shape x to input shape y.
Apply ICP to align x to y (align global pose).Repeat until convergence:
○ Map vertices on x to closest surface points on y.
○ New x1 has similar shape as y.
○ Align x1 to x giving x2.
○ Find shape parameters b of x3 wrt face model:
○ Restrict b to probable values b’ according to model.
○ Generate new shape x3 with b’ from
15
)( 2 xxΦWb T1
no facial landmark
'3 ΦWbxx for generating yclose to y
![Page 16: Registration of 3D Faces Leow Wee Kheng CS6101 AY2012-13 Semester 1](https://reader035.vdocuments.mx/reader035/viewer/2022062518/5681431a550346895daf7962/html5/thumbnails/16.jpg)
Questions Can it work for skulls? How many skull landmarks? Strengths? Weaknesses?
16