APPLICATION OF FEATURE POINT MATCHING TO VIDEO STABILIZATION

By

Nikhil Prathapani

Student Member, IEEE

INTRODUCTION

Along with advancement, digital video has introduced new

problems like video noising, video de-stabilization and video

jitter.

In order to overcome these problems, new techniques like video

enhancement and video stabilization have been proposed.

Of the proposed video stabilization techniques, all most all of

them require prior knowledge of prominent frame.

But the proposed technique is based on RANSAC*, SSD and SIFT,

it does not require any erstwhile knowledge of prominent video

frame for stabilization purpose.

*Tordoff, B; Murray, DW. "Guided sampling and consensus for motion estimation."European Conference n Computer Vision, 2002.

Why estimate visual motion? Visual Motion can be annoying

Camera instabilities, jitter Measure it; remove it (stabilize)

Visual Motion indicates dynamics in the scene Moving objects, behavior Track objects and analyze trajectories

Getting six parameters

SIFT algorithm – Find corresponding pairs At time k

It needs three pairs to determine a unique solution

Y X A

SIFT correspondence from frame 200,201 in outdoor sequence STREET

The fundamental matrix F

C C’T=C’-C

Rp p’

TRp'p Two reference frames are related via the extrinsic parameters

The fundamental matrix F The fundamental matrix is the algebraic

representation of epipolar geometry

The fundamental matrix satisfies the condition that for any pair of corresponding points x↔x’ in the two images

0Fx'xT 0lxT

RANSAC (Random Sampling and Consensus )

repeatselect minimal sample (8 matches)compute solution(s) for Fdetermine inliers

until (#inliers,#samples)>95% or too many times

compute F based on all inliers

SSD (sum of squared differences) surface – textured area

SSD surface – edge

SSD – homogeneous area

ALGORITHM

ANALYSIS AND RESULTS

Step 1- Reading frames from a movie file

Step 2- Collecting Salient Points from Each Frame

SIFT

Step 3- Selecting Correspondences Between Points

SSD

Step 4-. Estimating Transform from Noisy Correspondence

RANSACaffine transform will be a 3-by-3 matrix:[a_1 a_3t_r;a_2 a_4t_c;

0 0 1]The parameters ‘a’ define scale, rotation, and sheering effects of the transform, while the parameters ‘t’ are

translation parameters.

Step 5- Transform Approximation and Smoothing

Step 6- Run on the Full Video

Raw input mean and Corrected sequence mean images

PSNR MSE

RAW INPUT MEAN 22.5406 3.62

CORRECTED SEQUENCE MEAN

25.5725 3.59

CONCLUSION

The paper presents a comprehensive and thorough

approach to video stabilizing videos using MATLAB.

This kind of novel approach to video stabilizing [6, 7, 8]

without prior knowledge of prominent features in the frames

has targeted many applications in the fields of motion

estimation, remote sensing, and airborne applications

REFERENCES

[1]P. A. Keller, The cathode-ray tube: technology, history and applications,Palisades Press, 1991, ISBN 0963155903.[2]W. C. O’Mara, Liquid crystal flat panel display: manufacturing science andtechnology, Van Nostrand Reinhold, 1993, ISBN 0442014287.[3]J. Hutchison, “Plasma display panels: the colorful history of an Illinois tech-nology”, ECE alumni news, university of Illinois, vol. 36(1), 2002.[4]C. Poynton, Digital video and HDTV algorithms and interfaces, MorganKaufmann, 2003, ISBN 1558607927.[5] Tordoff, B; Murray, DW. "Guided sampling and consensus for motion estimation."European Conference n Computer Vision, 2002.[6] Lee, KY; Chuang, YY; Chen, BY; Ouhyoung, M. "Video Stabilization using Robust Feature Trajectories." National Taiwan University, 2009.[7] Litvin, A; Konrad, J; Karl, WC. "Probabilistic video stabilization using Kalman filtering and mosaicking." IS&T/SPIE Symposium on Electronic Imaging, Image and Video Communications and Proc., 2003.[8] Matsushita, Y; Ofek, E; Tang, X; Shum, HY. "Full-frame Video Stabilization." Microsoft® Research Asia.CVPR 2005.

Acknowledgements

I am deeply indebted to my parents who have always backed me equally during all times.

THANK YOU

Any Queries?

For research articles, papers and projects in the fields of Image Processing and Nanoelectronics,

you can connect to my research profile:

http://jntuhcej.academia.edu/NikhilPrathapani

SIFT detector proposed considers local image characteristic and retrieves feature points that are invariant to image rotation, scaling, translation, partly illumination changes and projective transform.

The scale-invariant feature extractor detects feature points through a staged filtering approach that identifies stable points in the scale-space.

Scale Invariant Feature Transform

Why Features? A brief yet comprehensive representation of

the image Can be used for:

Image alignment Object recognition 3D reconstruction Motion tracking Indexing and database search More…

Desired Feature Properties• Robustness => Invariance to changes in

illumination, scale, rotation, affine, perspective • Locality => robustness to occlusion and clutter.

• Distinctiveness => easy to match to a large database of objects.

• Quantity => many features can be generated for even small objects

• Efficiency => computationally “cheap”, real-time performance

Algorithm1. Scale-space extrema detection2. Keypoint localization3. Orientation assignment4. Keypoint descriptor