cs 6825: motion part 2 – optical flow
DESCRIPTION
CS 6825: Motion Part 2 – Optical Flow. Recall: Optical Flow. is an approximation of the 2D motion field. Motion in the world usually occurs in 3D, but, we have a 2D image sensor. So, we see the results as movement across the 2D image plane. - PowerPoint PPT PresentationTRANSCRIPT
CS 6825: Motion Part 2 – CS 6825: Motion Part 2 – Optical FlowOptical Flow
Recall: Optical FlowRecall: Optical Flow is an approximation of the 2D motion is an approximation of the 2D motion
field. field. Motion in the world usually occurs in 3D, Motion in the world usually occurs in 3D,
but, we have a 2D image sensor. So, we but, we have a 2D image sensor. So, we see the results as movement across the see the results as movement across the 2D image plane. 2D image plane.
Hence we are seeing the projection of the Hence we are seeing the projection of the 3D moving points onto the image plane. 3D moving points onto the image plane.
00
00
10
0
00
ˆby torelated
imagein induces , velocity has point Object
zr
rrrr
rv
rv
vv
f
dt
d
dt
d
P
ii
i
i
Motion Field and Optical Flow Motion Field and Optical Flow FieldField
Motion field: projection of 3D motion vectors on image planeMotion field: projection of 3D motion vectors on image plane
Optical flow field: apparent motion of brightness patternsOptical flow field: apparent motion of brightness patterns We equate motion field with optical flow fieldWe equate motion field with optical flow field
Brightness Constancy EquationBrightness Constancy Equation
Let P be a moving point in 3D:Let P be a moving point in 3D:• At time t, P has coords (X(t),Y(t),Z(t))At time t, P has coords (X(t),Y(t),Z(t))• Let p=(x(t),y(t)) be the coords. of its Let p=(x(t),y(t)) be the coords. of its
image at time t.image at time t.• Let E(x(t),y(t),t) be the brightness at p at Let E(x(t),y(t),t) be the brightness at p at
time t.time t. Brightness Constancy Assumption:Brightness Constancy Assumption:
• As P moves over time, E(x(t),y(t),t) As P moves over time, E(x(t),y(t),t) remains constant.remains constant.
Brightness Constancy Brightness Constancy EquationEquation
Taking derivative wrt time:Taking derivative wrt time:
Brightness Constancy EquationBrightness Constancy Equation
LetLet(Frame spatial gradient)(Frame spatial gradient)
(optical flow)(optical flow)
andand (derivative across frames)(derivative across frames)
Brightness Constancy EquationBrightness Constancy Equation
Becomes:Becomes:
vvxx
vv
yy
rr E E
The Optical Flow The Optical Flow is CONSTRAINED is CONSTRAINED to be on a line !to be on a line !
-E-Ett/|/|rr E| E|Can calculate these: Different techniques to figure these out.
E is the spatial change in brightness in image i
Et is the difference in the brightness at (x,y) between image i and image i+1
We want to calculate v = [dx/dt dy/dt]