Real time robust eye tracking in a large working space

Maria MikhisorApril 2013

Outline

• Why is this problem worth studying?• The Watching Window project• The eye tracking algorithm requirements• Eye detection and tracking methods• What I have done so far

Why is this problem worth studying?

• It hasn’t been done before (as far as I know)

• The resulting algorithm can be used in deep-immersion virtual reality systems (e.g. the Watching Window project) and possibly other applications

The concept of the Watching Window

The concept of the Watching Window

Eye tracking algorithm requirements

• Accurate• Fast (25-30 fps, <40 ms for 1 frame)• It should allow the user to move freely in the

working area (0-5 meters)• No head-mounted devices• Independent or only slightly dependent on

background and lighting conditions• Tolerant to glasses, diff. hair styles, skin color,

etc.

Eye detection and tracking methods

• Shape methods• Appearance methods• Features methods• IR reflection• … The main disadvantage:

Almost all these methods need high-resolution images of the face area in the frontal pose.

What I have done so far?

• Stereo cameras calibration for two fisheye cameras

Fisheye camera calibration

For one camera calibration, I used the Scaramuzza Matlab toolbox.

Reprojection error

• The average reprojection error is 0.692924

Stereo calibration

The essential matrix: E=R[t]x

The epipolar constraint: xRTExL=0

Triangulation

Simple way: find the middle of the common perpendicular