eye tracking: principles and applications 廖文宏 wen-hung liao [email protected] 12/10/2009
TRANSCRIPT
![Page 2: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/2.jpg)
Outline
Eye Tracking Device Application areas State-of-the-art technology
Eye tracker @NCCUCS 1.0 (wearable) Eye tracker @NCCUCS 2.0 (remote) Gaze-based HCI Demo Conclusions
![Page 3: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/3.jpg)
Eye Tracking Device
An eye tracker is a device for measuring eye positions and eye movements.
The most popular variant uses video images from which the eye position is extracted.
Input source: visible spectrum vs. infrared
![Page 4: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/4.jpg)
Eye Movements Eye movements are typically divided into
fixations (when the eye gaze pauses in a certain position) and saccades (when it moves to another position).
The resulting series of fixations and saccades is called a scanpath.
![Page 5: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/5.jpg)
![Page 6: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/6.jpg)
Web Design (U. of Manchester)
Heat map Gaze plot
![Page 7: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/7.jpg)
Usability Study:Google Search Evaluation
![Page 8: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/8.jpg)
Gaming
![Page 9: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/9.jpg)
Driving Behavior
[Andrew T. Duchowski]
![Page 10: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/10.jpg)
Character Input
EyeWrite [Andrew T. Duchowski]
![Page 11: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/11.jpg)
Media Research
The Poynter Institute : http://www.poynter.org Published first eye track study in 1991. More results published in 2000, 2004 and
2007. Eyetrack ’07:
http://www.poynter.org/content/content_view.asp?id=105035
![Page 12: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/12.jpg)
Research Communities
ACM SIG/CHI Eye Tracking Research and Applications
(ETRA) Symposium: http://www.e-t-r-a.org/ COGAIN: Communication by Gaze
Interaction http://www.cogain.org References:
http://www.cogain.org/downloads/
![Page 13: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/13.jpg)
State-of-the-art Eye tracking Technology
Hi-Speed (SMI iView X™ Hi-Speed) Head-Mounted System Remote tracking Integrated with LCD monitor
![Page 14: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/14.jpg)
Mobile Eye Tracker 1.0 @ NCCUCS
eye camera
scene camera
![Page 15: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/15.jpg)
Key Objectives
Low-cost (NTD 10,000 vs. 1,000,000) Mobility Easily customized for specific applications On-line processing Sampling rate? Accuracy?
![Page 16: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/16.jpg)
System Architecture
Eye image
Preprocessing
Pupil detection
Gaze point projection
Scene image
Calibration
9 pairsof points
Calibration process
![Page 17: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/17.jpg)
Pupil Detection (I):extracting feature points
![Page 18: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/18.jpg)
Pupil Detection (II): Ellipse fitting using RANSAC
![Page 19: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/19.jpg)
Pupil Detection (III):checking fitness measure
Ellipse model:
20,sin
coscossinsincos cy
cxba
yxp
n
bfScore
m
afScore
n
iiin
in
m
iiout
out
11
)(,
)(
,0
,)(,1
otherwise
PaThresholdapiff outbrightout
,0
,)(,1
otherwise
PbThresholdbpiff indarkin
Number of bright vs. dark points inside and out the perimeter
![Page 20: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/20.jpg)
Results
![Page 21: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/21.jpg)
HCI Application: Eye Scrolling
![Page 22: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/22.jpg)
HCI Application: Eye Gaming
![Page 23: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/23.jpg)
HCI Application: Eye Typing
![Page 24: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/24.jpg)
Mobile Eye Tracker 2.0 @ NCCUCS
Improve the pupil detection algorithm to alleviate corneal reflection problem.
Enhance the accuracy by compensating for head movement.
Construct and test a remote eye tracker. More HCI applications using the remote eye
tracker. Use the eye tracking device to assist mobile
user interface design.
![Page 25: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/25.jpg)
Improved Pupil Detection
![Page 26: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/26.jpg)
Allowing Head Movement
![Page 27: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/27.jpg)
Remote Eye Tracker
![Page 28: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/28.jpg)
Experimental Results (Wearable)
Calibration
point
Original error Compensate for head
movement
(error/standard deviation)
1 98.17 32.91 (12.7)
2 137.54 55.89 (10.94)
3 108.20 29.69 (9.57)
4 76.30 28.79 (21.19)
5 113.30 22.49 (7.79)
6 117.86 34.83 (10.96)
7 116.53 33.31 (8.01)
8 112.56 16.03 (6.78)
9 146.73 26.49 (9.66)
1 cm = 38 Pixels
![Page 29: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/29.jpg)
Accuracy (Wearable)
![Page 30: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/30.jpg)
Experimental Results (Remote)Calibration point Original error Compensate for head movement
(error/standard deviation)
1 238.73 39.69 (24.6)
2 227.51 63.56 (22.82)
3 132.97 44.30 (21.49)
4 222.97 51.66 (30.94)
5 306.58 31.15 (16.71)
6 280.79 51.66 (21.83)
7 311.89 69.03 (41.57)
8 344.24 69.88 (31.71)
9 347.03 64.66 (24.41)
![Page 31: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/31.jpg)
Accuracy (Remote Eye Tracker)
![Page 32: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/32.jpg)
Demo: Web Browsing
![Page 33: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/33.jpg)
Demo: Photo Viewing
![Page 34: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/34.jpg)
Demo: Interactive Story Telling
![Page 35: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/35.jpg)
Demo: Tic-Tac-Toe
![Page 36: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/36.jpg)
Summary
Eye tracking device Low cost (<$10,000 NTD) Head-mounted with mobile functionality Remote eye tracking Allow slight head movements Accuracy: suitable for some HCI applications,
image viewing tasks, not high enough for reading HCI applications
![Page 37: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/37.jpg)
http://www.cs.nccu.edu.tw/~whliao/dct/
![Page 38: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/38.jpg)
More materials
![Page 39: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/39.jpg)
Gaze Path Analysis
Recursive intersection Find the similarity between two scan paths Order of scanning is irrelevant Suitable for processing fixation data
Modified dynamic time warping Order of scanning is taken into account Can handle both fixation and saccade data
![Page 40: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/40.jpg)
Recursive Intersection
Path 2
Path 1
![Page 41: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/41.jpg)
Recursive Intersection: Example
![Page 42: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/42.jpg)
Dynamic Time Warping (DTW)
![Page 43: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/43.jpg)
Modified DTW (MDTW)
X-axisY-axis
time
![Page 44: Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009](https://reader035.vdocuments.mx/reader035/viewer/2022062322/56649dbe5503460f94ab2636/html5/thumbnails/44.jpg)
MDTW Result