presentation
DESCRIPTION
PRESENTATION . ON. “ GESTURE RECOGNITION ”. SUBMITTED BY :. SUBMITTED TO:. PRESENTATION OUTLINE :. INTRODUCTION STEPS OF GESTURE RECOGNITION TRACKING TECHNOLOGIES SPEECH WITH GESTURE APPLICATIONS. WHAT ARE GESTURES ???. Gestures are expressive, meaningful body motions – - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/1.jpg)
PRESENTATION
SUBMITTED TO: SUBMITTED BY:
ON
“ GESTURE RECOGNITION ”
![Page 2: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/2.jpg)
PRESENTATION OUTLINE :
INTRODUCTION STEPS OF GESTURE RECOGNITION TRACKING TECHNOLOGIES SPEECH WITH GESTURE APPLICATIONS
![Page 3: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/3.jpg)
WHAT ARE GESTURES ???
Gestures are expressive, meaningful body motions –i.e., physical movements of the fingers, hands, arms,head, face, or body with the intent to conveyinformation or interact with the environment.
![Page 4: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/4.jpg)
GESTURE RECOGNITION:
•Mood and emotion are expressed by body language.•Facial expressions.•Tone of voice.•Allows computers to interact with human beings in a more natural way.•Allows control without having to touch the device.
![Page 5: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/5.jpg)
Human Computer Interface using Gesture• Replace mouse and keyboard.• Pointing gestures.• Navigate in a virtual environment.• Pick up and manipulate virtual
objects.• Interact with a 3D world.• No physical contact with computer.• Communicate at a distance.
![Page 6: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/6.jpg)
STEPS OF GESTURE RECOGNITION:
![Page 7: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/7.jpg)
TRACKING TECHNOLOGIES:
1. DATAGLOVES / CYBERGLOVES
- Use of gloves equipped
with sensors. - Use of fiber optic cables.
![Page 8: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/8.jpg)
SIGN LANGUAGE RECOGNITION
• 5000 gestures in vocabulary.
• each gesture consists of a hand shape, a handmotion and a location in 3D space.
A FC
![Page 9: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/9.jpg)
Datagloves
![Page 10: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/10.jpg)
![Page 11: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/11.jpg)
Colour Segment
NoiseRemoval
Scale by Area
THE PROCESS
![Page 12: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/12.jpg)
TRACKING TECHNOLOGIES:
2. COMPUTER-VISION TECHNOLOGY.
• USE OF CAMERAS
- DEPTH CAMERAS.- STEREO CAMERAS.- NORMAL CAMERAS.
![Page 13: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/13.jpg)
THE VIDEOPLACE :
Here the index finger is recognized and when extended, becomes a drawing tool.
Here, text is entered by pointing at the character desired
Here the index fingers and thumbs of the two hands are recognized and are used to control the shape of the object being defined
![Page 14: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/14.jpg)
A B C Y
Yes/No? Yes/No? Yes/No?
Yes/No?
![Page 15: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/15.jpg)
Hierarchical Search• We need to search thousands of images.• How to do this efficiently?• We need to use a “coarse-to-fine”search
strategy.
![Page 16: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/16.jpg)
Original image Blurring Factor = 1
Blurring Factor = 2Blurring Factor = 3
![Page 17: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/17.jpg)
Factor = 3.0
Factor = 2.0
Factor = 1.0
• Multi-scale Hierarchy
![Page 18: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/18.jpg)
Hidden Markov Model ( HMM ) --- time sequence of images modeling
Motion Recognition
HMM1 (Hello)
HMM2 (Good)
HMM3(Bad)
HMM4 (House)
P(f |HMM1)f
P(f |HMM2)
![Page 19: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/19.jpg)
Prediction and Tracking•Given previous
frames we can predict what will happen next•Speeds up search.•occlusions -
![Page 20: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/20.jpg)
Co-articulation
In fluent dialogue signs are modified by preceding and following signs.
intermediate forms
AB
![Page 21: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/21.jpg)
Face recognitionSingle pose
Standard head-and-shoulders view with uniform background
Easy to find face within image
![Page 22: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/22.jpg)
Aligning ImagesAlignment
• Faces in the training set must be aligned with each other to remove the effects of translation, scale, rotation etc.
• It is easy to find the position of the eyes and mouth and then shift and resize images so that are aligned with each other
![Page 23: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/23.jpg)
Nearest Neighbour• Once the images have been aligned you
can simply search for the member of the training set which is nearest to the test image.
• There are a number of measures of distance including Euclidean distance, and the cross-correlation.
![Page 24: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/24.jpg)
Principal Components Analysis
• PCA reduces the number of dimensions and so the memory requirement is much reduced.
• The search time is also reduced
![Page 25: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/25.jpg)
Problems with PCA• The same person may sometimes
appear differently due to• Beards, moustaches• Glasses,• Makeup
• These have to be represented by different ellipsoids.
![Page 26: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/26.jpg)
Facial Expressions
• There are six types of facial expression
• We could use PCA on the eyes and mouth – so we could have eigeneyes and eigenmouths
Anger Fear Disgust Happy Sad Surprise
![Page 27: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/27.jpg)
Multiple Poses• Heads must now be aligned in 3D world
space.• Classes now form trajectories in feature
space.• It becomes difficult to recognise faces
because the variation due to pose is greater than the variation between people.
![Page 28: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/28.jpg)
Model-based Recognition• We can fit a model directly• to the face image
• Model consists of a mesh which is matched to facial features such as the eyes, nose, mouth and edges of the face.
• We use PCA to describe the parameters of the model rather than the pixels.
![Page 29: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/29.jpg)
Speech with Gesture
• Voice and gesture compliment each other and form a powerful interface that either a modality alone.
• Speech and gesture make a more interactive interface.
• Combining gesture and voice increase recognition accuracy.
![Page 30: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/30.jpg)
MEDIA ROOMWithin the media room user can use gesture ,speech ,eye movements or combination of all three.
Example:One application allowed user to manage color coded ship against a map of a carribean . A user just need to point the location and need to say “create a large blue tank”.A blue tank will appear on the location.
Media room
![Page 31: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/31.jpg)
Applications
Sign language recognition: gesture recognition software can transcribe the
symbols represented through sign language into text. Control through facial gestures: Controlling a computer through facial gestures is a
useful application of gesture recognition for users who may not physically be able to use a mouse or keyboard.
Immersive game technology: Gestures can be used to control interactions within
video games to try and make the game player's experience more interactive or immersive.
![Page 32: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/32.jpg)
A person playing game.
Computer is responding as per user instruction.
A girl is instructing the computer from her body movements .
![Page 33: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/33.jpg)
Applications Virtual controllers: For systems where the act of finding or acquiring a
physical controller could require too much time, gestures can be used as an alternative control mechanism.
Affective computing: In affective computing, gesture recognition is used in
the process of identifying emotional expression through computer systems.
Remote control: Through the use of gesture recognition, “remote control with
the wave of a hand” of various devices is possible. The signal must not only indicate the desired response, but also which device to be controlled.
![Page 34: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/34.jpg)
Future Work:
• Occlusions (Atid).• Grammars in Irish Sign
Language.--- Sentence Recognition.
• Body Language.
![Page 35: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/35.jpg)
References
Wu yang,vision based gesture recognition lecture notes in artificial intelligence 1999.
Wikipedia .
![Page 36: PRESENTATION](https://reader036.vdocuments.mx/reader036/viewer/2022081604/5681613f550346895dd0aabb/html5/thumbnails/36.jpg)
THANK YOU !!!!ANY
QUERIES ???