handgesturerecognitionusingopencv

Dept. Of CSE,RNSIT 2015-2016 1

CONTENTS

• Introduction

• Algorithm’s Flow chart

• Colour Profile

• Binary Representations of Contours

• Analysis of Contours

• Finding Length and Angle

• Conclusion

• References


Hand Gesture Recognition using OpenCV

INTRODUCTION

WHAT IS A GESTURE?

A movement of a limb or the body as an expressionof thought or feeling.(Oxford English Dictionary)



Wireless & Flexible No specialized hardwareSingle Camera Real-time



Get frame

from video

information

Information about

fingertips, number

of fingers etc

Basic flow of hand recognition

Filter out irrelevant polygonDescribe contours with

convex polygons

and find convexity defects

Sample Color

Make binary Get n biggest representation contours

of hand

Dismiss contours

not representing

hands

Flowchart



COLOUR PROFILE


Median 1

Median 2Median 3

Median 4

Median 5

Median 6

Median 7

Figure: Making a color profile of hand, based on median color value sampled from different areas of the hand

Colour Profile


MAKE BINARY REPRESENTATIONS AND EXTRACT RELEVANT CONTOURS


Extract hand based on color recognition. Compute one binary image based on each sampled median and sum the binary images together. Finally filter the result with the nonlinear median blur filter.


ANALYZING CONTOUR GEOMETRIC APAROACH



FINDING LENGTH AND ANGLE

• 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 = (𝑥1 − 𝑥2)2+(𝑦1 − 𝑦2)2

• 𝜃 = cos−1(𝑎𝑥∗𝑏𝑥)+(𝑎𝑦∗𝑏𝑦)

𝑎 ∗|𝑏|



RESULT OF HAND ANALYSIS

The result of data can be further used in:

• Fingertip position

• Hand area

• Drawing Patterns from the tip of finger



THINGS THAT NEEDS IMPROVEMENTS

• Still need to tune the threshold for extracting skin colour.

• For finger tracking one would need to identify each finger individually.

• Can be modified to work with a moving background.



LIMITATIONS

• Inability to differentiate between objects of similar color as that of sample

• Overlapping objects.

• Noise and lighting sensitive

• Manual caliberation



CONCLUSION

• We try to implement a Hand Gesture Recognition tool which can be helpful to read and decipher the sign language.

• This can also be extended to perform certain activities:• such as tracking the finger tips and using it as a pointing object which can

perform the functions of mouse.

• Using the finger tips to draw certain figures on the computer screen



REFERENCES

[1] Higher Mathematics by ML Khanna

[2] Learning OpenCV by Gary Bradski and Adrian Kaehler.

[3] K. Murphy, Hidden Markov Model (HMM) Toolbox

http://www.cs.ubc.ca/ murphyk/Software/HMM/hmm.html.

[4] Computer Vision Application Programming Cook Book by

Robert Laganiere.

[5] OpenCV documentation .



QUESTIONS?



handgesturerecognitionusingopencv

Documents