augmented reality: object tracking and active appearance model presented by pat chan 01/03/2005...
Post on 15-Jan-2016
218 views
TRANSCRIPT
![Page 1: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/1.jpg)
Augmented Reality:Object Tracking and Active Appearance Model
Presented by Pat Chan01/03/2005Group Meeting
![Page 2: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/2.jpg)
Outline
Introduction to Augmented Reality Object Tracking Active Appearance Model (AAM) Object Tracking with AAM Future Direction Conclusion
![Page 3: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/3.jpg)
Introduction
An Augmented Reality system supplements the real world with virtual objects that appear to coexist in the same space as the real world
Properties : Combine real and virtual objects in a real
environment Runs interactively, and in real time Registers(aligns) real and virtual objects with each
other
![Page 4: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/4.jpg)
Introduction
Display Presenting virtual objects on real environment
Tracking Following user’s and virtual object’s
movements by means of a special device or techniques
3D Modeling Forming virtual object
Registration Blending real and virtual objects
![Page 5: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/5.jpg)
Object Tracking
Visual content can be modeled as a hierarchy of abstractions.
At the first level are the raw pixels with color or brightness information.
Further processing yields features such as edges, corners, lines, curves, and color regions.
A higher abstraction layer may combine and interpret these features as objects and their attributes.
![Page 6: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/6.jpg)
Object Tracking
Accurately tracking the user’s position is crucial for AR registration
The objective is to obtain an accurate estimate of the position (x,y) of the object tracked
Tracking = correspondence + constraints + estimation
Tracking objects is a sequence of video frames is composed of two main stages: Isolation of objects from background in each frames Association of objects in successive frames in order
to trace them
![Page 7: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/7.jpg)
Object Tracking
Object Tracking in image processing is usually based on reference image of the object, or properties of the objects.
Tracking techniques: Kalman filtering Correlation-based tracking, Change-based tracking 2D layer tracking tracking of articulated objects
![Page 8: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/8.jpg)
Object Tracking
Object Tracking can be briefly divides into following stages: Input (object and camera) Finding correspondence Motion Estimation Corrective Feedback Occlusion Detection
![Page 9: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/9.jpg)
Input
Tracking algorithms can be classified into Single object & Single Camera Single object & Multiple Cameras Multiple object & Single Camera Multiple objects & Multiple Cameras
![Page 10: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/10.jpg)
Single Object & Single Camera
Accurate camera calibration and scene model
Suffers from Occlusions Not robust and object dependant
![Page 11: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/11.jpg)
Single Object & Multiple Camera
Accurate point correspondence between scenes Occlusions can be minimized or even avoided Redundant information for better estimation Multiple camera Communication problem
![Page 12: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/12.jpg)
Possible Solution
ObjectIdentification
ObjectTracking
Check Position(X1-X2) <(Y1-Y2) <
ChooseCameraView
ObjectIdentification
ObjectTracking
![Page 13: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/13.jpg)
Static Point Correspondence The output of the tracking stage is A simple scene model is used to get real
estimation of coordinates Both Affine and Perspective models were
used for the scene modeling Static corresponding points were used for
parameter estimation Least mean squares was used to improve
parameter estimation
)(),( nYnX ii
)(ˆ),(ˆ nYnX ii
![Page 14: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/14.jpg)
Dynamic Point Correspondence
Affine modelusing A(n)
B(n)
Affine modelusing A(n)
Add this point to AAdd this point to A
Check Position(X1-X2) < T(Y1-Y2) < T
![Page 15: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/15.jpg)
Block-Based Motion Estimation
Typically, in object tracking precise sub-pixel optical flow estimation is not needed.
Motion can be in the order of several pixels, thereby precluding use of gradient methods.
A simple sum of squared differences error criterion coupled with full search in a limited region around the tracking window can be applied.
2
),(
)),,(),,(( tyxsttnymxsSSD cyx
cerror
![Page 16: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/16.jpg)
Adaptive Window Sizing Although simple block-based motion
estimation may work reasonably well when motion is purely translational
It can lose the object if its relative size changes. If the object’s camera field of view shrinks, the SSD
error is strongly influenced by the background. If the object’s camera field of view grows, the
window fails to make use of entire object information and can slip away.
![Page 17: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/17.jpg)
Four Corner Method
This technique divides the rectangular object window into 4 basic regions - each one quadrant.
Motion vectors are calculated for each subregion and each controls one of four corners.
Translational motion is captured by all four moving equally, while window size is modulated when motion is differential.
Resultant tracking window can be non-rectangular, i.e., any quadrilateral approximated by four rectangles with a shared center corner.
![Page 18: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/18.jpg)
Example: Four Corner Method
Synthetically generated test sequences:
![Page 19: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/19.jpg)
Correlative Method
Four corner method is strongly subject to error accumulation which can result in drift of one or more of the tracking window quadrants.
Once drift occurs, sizing of window is highly inaccurate.
Need a method that has some corrective feedback so window can converge to correct size even after some errors.
Correlation of current object features to some template view is one solution.
![Page 20: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/20.jpg)
Correlative Method (con’t)
Basic form of technique involves storing initial view of object as a reference image.
Block matching is performed through a combined interframe and correlative MSE:
where sc’(x0,y0,0) is the resized stored template image. Furthermore, minimum correlative MSE is used to
direct resizing of current window.
,))0,,(),,(()1(
)),,(),,((1
200
'
),(1
2
),(1
21
yxsttnymxsa
tyxsttnymxsann
MSE
c
yxc
cyx
cerror
![Page 21: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/21.jpg)
Example: Correlative Method
![Page 22: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/22.jpg)
Occlusion Detection Each camera must possess an ability to assess
the validity of its tracking (e.g. to detect occlusion).
Comparing the minimum error at each point to some absolute threshold is problematic since error can grow even when tracking is still valid.
Threshold must be adaptive to current conditions.
One solution is to use a threshold of k (constant > 1) times the moving average of the MSE.
Thus, only steep changes in error trigger indication of possibly wrong tracking.
![Page 23: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/23.jpg)
Improvements
Things can be improved Good filtering algorithms Adequate dynamical models Shape/appearance models need
work
![Page 24: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/24.jpg)
Active Appearance Models (AAMs)
Active Appearance Models are generative models commonly used to model faces
Can also be useful for other phenomena Matching object classes Deformable appearance models
![Page 25: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/25.jpg)
Active Appearance Models (AAMs)
2D linear shape is defined by 2D triangulated mesh and in particular the vertex locations of the mesh.
Shape s can be expressed as a base shape s0.
pi are the shape parameter. s0 is the mean shape and the matrices si are the eigenvectors cor
responding to the m largest eigenvalues
![Page 26: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/26.jpg)
Active Appearance Models (AAMs)
The appearance of an independent AAM is defined within the base mesh s0. A(u) defined over the pixels u ∈ s0
A(u) can be expressed as a base appearance A0(u) plus a linear combination of l appearance
Coefficients λi are the appearance parameters.
A0(u) A1(u) A2(u) A3(u)
![Page 27: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/27.jpg)
Active Appearance Models (AAMs)
The AAM model instance with shape parameters p and appearance parameters λ is then created by warping the appearance A from the base mesh s0 to the model shape s.
Piecewise affine warp W(u; p):(1) for any pixel u in s0 find out which triangle it lies in,(2) warp u with the affine warp for that triangle.
M(W(u;p))
![Page 28: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/28.jpg)
Fitting AAMs
Minimize the error between I (u) and M(W(u; p)) = A(u).
If u is a pixel in s0, then the corresponding pixel in the input image I is W(u; p).
At pixel u the AAM has the appearance
At pixel W(u; p), the input image has the intensity I (W(u; p)).
Minimize the sum of squares of the difference between these two quantities:
uu u u
![Page 29: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/29.jpg)
Object Tracking with AAM
Objects can be tracked with the trained AAM
3-D face tracking with AAM search Pose estimation with AAM
![Page 30: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/30.jpg)
Example
The training set consisted of five images of a DAT tape cassette
DAT cassette was annotated using 12 landmarks
Upon the five training images, a two-level multi-scale AAM was built.
aam_tracking_mpeg4.avi
![Page 31: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/31.jpg)
Future Direction
Propose a general object tracking algorithm with the help of AAM
Improve the accuracy of the object tracking algorithm
Improve the fitting speed of the AAM
![Page 32: Augmented Reality: Object Tracking and Active Appearance Model Presented by Pat Chan 01/03/2005 Group Meeting](https://reader036.vdocuments.mx/reader036/viewer/2022062322/56649d615503460f94a430ef/html5/thumbnails/32.jpg)
Conclusion
Introduction on Augmented Reality Survey on Object Tracking Introduction Active Appearance
Model Improve the accuracy of object
tracking by AAM Proposed our future research
direction