determining planar translation and rotation by optical flow david hong ncssm, mini-term 2008
TRANSCRIPT
![Page 1: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/1.jpg)
Determining Planar Translation and Rotation
by Optical Flow
David HongNCSSM, Mini-term 2008
![Page 2: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/2.jpg)
Example Problem 1
In:
Out: 4 0 0s s sX Y
![Page 3: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/3.jpg)
Example Problem 2
In:
Out: 0 4 0s s sX Y
![Page 4: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/4.jpg)
Example Problem 3
In:
Out: 0 0 6s s sX Y
![Page 5: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/5.jpg)
Problem (Formal Statement)
In: , n nf g R
Out: , , such that
translated by and and
rotated by is
s s s
s s
s
X Y
f X Y
g
![Page 6: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/6.jpg)
Motivation
Many Applications:– Special Effects (Movie)
– Video Compression
– Pattern Recognition
– Image Stabilization (Digital Cameras)
– Dead-reckoning (Mobile Robotics)
![Page 7: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/7.jpg)
State of the Art
Block Motion Estimation
Demir-Ertűrk (2007)
Optical Flow
Lucas-Kanade (1985)
Optical Flow with Smoothness Constraint
Horn-Schunck (1980)
![Page 8: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/8.jpg)
X
Y
u(x,y,t)
U(X,Y)
xy
Idea
![Page 9: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/9.jpg)
Idea
0 0, , ,U X Y u x y t
Then the floor-coordinate is (X0,Y0) and the sensor-coordinate is (x,y) at time t.
Let us consider a point on the plane.
From there, we can see:
![Page 10: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/10.jpg)
Idea
0 0, , ,
0
U X Y u x y t
d dU u
dt dtdx
u u udtdyx y t
dt
Differentiating on time gives us:
![Page 11: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/11.jpg)
Idea
0
0
cos sin
sin coss s s
s s s
X Xx
Y Yy
Expressing (x,y) in terms of (X0,Y0) and the mouse position (Xs,Ys,Θs) gives us:
![Page 12: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/12.jpg)
Idea
s
s
s
dX
dtdYu u u u u
x yx y y x dt t
d
dt
Putting the two together, we get:
This is underdetermined!
![Page 13: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/13.jpg)
Idea
, 1 , 1 1, 1,1, 1, , 1 , 1 , ,'
2 2 2 2
s
i j i j i j i ji j i j i j i j i j i js
s
X
tu u u uu u u u u uY
i x j yx y y x t t
t
We choose a convenient unit so Δx= Δy=Δt=1
Discretizing, we get:
1, 1, , 1 , 1 , 1 , 1 1, 1, , ,2 's
i j i j i j i j i j i j i j i j s i j i j
s
X
u u u u u u i u u j Y u u
![Page 14: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/14.jpg)
Idea
This is normally over-determined!
Now, using all points, we get:
1, 1, , 1 , 1 , 1 , 1 1, 1, , ,2 's
i j i j i j i j i j i j i j i j s i j i j
s
X
u u u u u u i u u j Y u u
Use least-squares!
![Page 15: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/15.jpg)
Algorithm
1, 1, , 1 , 1 , 1 , 1 1, 1, , ,2 's
i j i j i j i j i j i j i j i j s i j i j
s
X
u u u u u u i u u j Y u u
uu’
![Page 16: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/16.jpg)
Improvement by Iteration
u
u’
Iteration 1:
3
2
0
s
s
s
X
Y
u’’
Iteration 2:
1
1
0
s
s
s
X
Y
Overall Estimate:
4
1
0
s
s
s
X
Y
u’’’
It is more difficult with rotation!Lagrange Interpolation!
![Page 17: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/17.jpg)
Improvement by Iteration
u
u’
![Page 18: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/18.jpg)
Performance of Algorithm
Surface:
![Page 19: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/19.jpg)
Performance of Algorithm
Good Surface: Bad Surface:
Algorithm Fails!
![Page 20: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/20.jpg)
Performance of AlgorithmSurface:
First iteration:
Second iteration:
![Page 21: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/21.jpg)
Performance of AlgorithmSurface:
![Page 22: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/22.jpg)
Performance of AlgorithmSurface:
![Page 23: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/23.jpg)
Future Work
• Smoothness Constraint
• Pre-processing the image
• Condition for Convergence
• Phase Correlation
![Page 24: Determining Planar Translation and Rotation by Optical Flow David Hong NCSSM, Mini-term 2008](https://reader035.vdocuments.mx/reader035/viewer/2022062518/56649f175503460f94c2e9f8/html5/thumbnails/24.jpg)
Thank You!