eye/gaze tracking in video; identify the user’s “focus of attention” omihaela romanca –...
TRANSCRIPT
Eye/gaze tracking in video; Eye/gaze tracking in video; identify the user’s “focus of identify the user’s “focus of
attention”attention”
oMihaela Romanca – Technical University of Cluj-NapocaoPeter Robert - Technical University of Cluj-NapocaoVilius Matiukas - Vilnius Gediminas Technical University oBrigitta Nagy – University of Debrecen
Introducing the team
SSIP 2009 2
Mihaela Romanca
SSIP 2009
Student from Technical University of Cluj-Napoca
Hobbies: Sports and ecologyE-mail: [email protected]
3
Robert Peter
SSIP 2009
Masters student from Technical University of Cluj-Napoca
Hobbies: PC games, football and movies/musicE-mail: [email protected]
4
Vilius Matiukas
SSIP 2009
PhD student from Vilnius Gediminas Technical University, Faculty of Electronics, Department of Electronic Systems
Hobbies: Image Processing and fishingE-mail: [email protected]
5
Brigitta Nagy
SSIP 2009
Student from University of Debrecen, Faculty of Informatics
Hobbies: Image processing, Wing-Tsun Kung-fu, Reading and PuzzlesE-mail: [email protected]
6
Test subject
SSIP 2009
Uneducated peace of paper
Hobbies: Staring at the same direction.Address: computer laboratory
7
SSIP 2009
Problem Description
• Input: video of a user sitting in front of the computer
• Goal: Detect the focus of attention and the modification of the region of interest of the user.
8
Equipment and software
• Genius Slim 321c webcamera.• Language: C#• IDE: Microsoft Visual Studio 2005• EMGU CV: Wrapper for C# of
OpenCV
SSIP 2009 9
SSIP 2009
Tasks to do
1. Face Detection2. Detection of the eyeregion3. Pupil Detection4. Eye Corner Detection5. Determine the focus of attention
10
SSIP 2009
1. Face Detection• We used Haar-like features for
face detection.• Haar-like features are digital
image features used in object recognition.
• Then we reduced the face region and split it to region of eyes.
11
Example
SSIP 2009
• Our test subject
12
SSIP 2009
2. Detection the region of the eye
• To detect the region of eyes we used also Haar-like features.
• Then contrast enchancement on the detected eye region was applied.
13
Example
• The test subject
SSIP 2009
Approximation With Haar-like features
14
SSIP 2009
3. Pupil detection
• Circular Hough transformation was applied for detection of the pupil.
• The Hough transform is a feature extraction technique.
• The classical Hough transform was concerned with the identification of lines in the image, but later the Hough transform has been extended to identifying positions of arbitrary shapes, most commonly circles or ellipses.
15
SSIP 2009
4. Eye Corner Detection
• We choose the one closest corner to the nose.
16
Calibration for gaze detection
• Wait until the user sits in a position, where 80% of the frames detect the iris center and the corner also.
• Put circles in the center and the four extremities of the screen, and wait until at least 15 pupil and eye corners are detected in both region of eye.
• Calculate the average of eye corner and center coordinates in all the positions (center, topleft, topright…).
SSIP 2009 17
SSIP 2009
5. Focus of attention
• As the users moves the eyes the mouse cursor moves in the corresponding direction.
18
Statistics
Spot
Variance (pixel^2)Test1
Variance (pixel^2)Test2
Variance (pixel^2)Test3
Variance (pixel^2)Test4
Variance (pixel^2)Test5
Left pupil 437 309 292 300 314
Left corner 577 555 567 570 556
Right pupil 346 288 341 294 296
Right corner
256 293 274 178 220
SSIP 2009
The numbers represent the variance of coordinates during calibration.
19
Future development
• Imitating left and right mouse clicks with blink detection
• Recognition even when face is in different angle
• Expression detection for different focus regions
• Higher precision for full control for people with disabilities
SSIP 2009 20
Thank you!
SSIP 2009 21