Eye-Based Interaction in Graphical Systems: Eye-Based Interaction in Graphical Systems: Theory & PracticeTheory & Practice

Eye-Based Interaction in Graphical Systems: Eye-Based Interaction in Graphical Systems: Theory & PracticeTheory & Practice

Part IIIPart III

Potential Gaze-Contingent ApplicationsPotential Gaze-Contingent Applications

G: Applications—IntroductionG: Applications—IntroductionG: Applications—IntroductionG: Applications—Introduction

• Wide variety of eye tracking applications Wide variety of eye tracking applications exist, each class increasingly relying on exist, each class increasingly relying on advanced graphical techniquesadvanced graphical techniquesPsychophysicsPsychophysics Human FactorsHuman Factors

AdvertisingAdvertising DisplaysDisplays

Virtual RealityVirtual Reality HCI & Collaborative SystemsHCI & Collaborative Systems

• Two broad categories: Two broad categories: diagnosticdiagnostic or or interactiveinteractive

H: Psychology, Psychophysics, and H: Psychology, Psychophysics, and NeuroscienceNeuroscienceH: Psychology, Psychophysics, and H: Psychology, Psychophysics, and NeuroscienceNeuroscience

• Applications range from basic research in Applications range from basic research in vision science to investigation of visual vision science to investigation of visual exploration in aesthetics (e.g., perception exploration in aesthetics (e.g., perception of art)of art)

• Examples:Examples:• psychophysics: spatial acuity, contrast sensitivity, ...psychophysics: spatial acuity, contrast sensitivity, ...• perception: reading, natural scenery, ...perception: reading, natural scenery, ...• neuroscience: cognitive loads, with fMRI, ...neuroscience: cognitive loads, with fMRI, ...

H: Psychology, Psychophysics, and H: Psychology, Psychophysics, and Neuroscience Neuroscience (cont’d)(cont’d)

H: Psychology, Psychophysics, and H: Psychology, Psychophysics, and Neuroscience Neuroscience (cont’d)(cont’d)

Fig.48: Perception of artFig.48: Perception of art• small but visible differences in scanpathssmall but visible differences in scanpaths• similar sets of fixated image featuressimilar sets of fixated image features

(a) aesthetic group(a) aesthetic group (b) semantic group(b) semantic group

I: Ergonomics and Human FactorsI: Ergonomics and Human FactorsI: Ergonomics and Human FactorsI: Ergonomics and Human Factors

• Applications range from usability studies Applications range from usability studies to testing effectiveness of cockpit displaysto testing effectiveness of cockpit displays

• Examples:Examples:• evaluation of tool icon groupingsevaluation of tool icon groupings

• comparison of gaze-based and mouse interactioncomparison of gaze-based and mouse interaction

• organization of click-down menusorganization of click-down menus

• testing electronic layout of pilots’ visual flight rulestesting electronic layout of pilots’ visual flight rules

• testing simulators for training effectivenesstesting simulators for training effectiveness

I: Ergonomics and Human Factors I: Ergonomics and Human Factors (cont’d)(cont’d)I: Ergonomics and Human Factors I: Ergonomics and Human Factors (cont’d)(cont’d)

Fig.49: Virtual aircraft cargo-bay Fig.49: Virtual aircraft cargo-bay environmentenvironment

• examination of visual search patterns of experts during examination of visual search patterns of experts during aircraft inspection tasksaircraft inspection tasks

• 3D scanpaths: gaze/wall intersection points3D scanpaths: gaze/wall intersection points

J: Marketing / AdvertisingJ: Marketing / AdvertisingJ: Marketing / AdvertisingJ: Marketing / Advertising

• Applications range from assessing ad Applications range from assessing ad effectiveness (copy testing) in various effectiveness (copy testing) in various media (print, images, video, etc.) to media (print, images, video, etc.) to disclosure research (visibility of fine print) disclosure research (visibility of fine print)

• Examples:Examples:• eye movements over print media (e.g., yellow eye movements over print media (e.g., yellow

pages)pages)

• eye movements over TV ads, magazines, ...eye movements over TV ads, magazines, ...

J: Marketing / Advertising J: Marketing / Advertising (cont’d)(cont’d)J: Marketing / Advertising J: Marketing / Advertising (cont’d)(cont’d)

Fig.50: Scanpaths over magazine adsFig.50: Scanpaths over magazine ads

K: DisplaysK: DisplaysK: DisplaysK: Displays

• Applications range from perceptually-Applications range from perceptually-based image and video display design to based image and video display design to estimation of corrective display functions estimation of corrective display functions (e.g., gamma, color spaces, etc.)(e.g., gamma, color spaces, etc.)

• Examples:Examples:• JPEG/MPEG (no eye tracking JPEG/MPEG (no eye tracking per seper se, but , but

perceptually based, e.g., JPDs)perceptually based, e.g., JPDs)

• gaze-contingent displays (e.g., video-telephony, …)gaze-contingent displays (e.g., video-telephony, …)

• computer (active) visioncomputer (active) vision

K: Displays K: Displays (cont’d)(cont’d)K: Displays K: Displays (cont’d)(cont’d)

Fig.51: Gaze-based foveo-peripheral image codingFig.51: Gaze-based foveo-peripheral image coding• 2 Regions Of Interest (ROIs)2 Regions Of Interest (ROIs)• smooth degradation (wavelet interpolation)smooth degradation (wavelet interpolation)

(a) Haar HVS reconstruction(a) Haar HVS reconstruction (b) wavelet acuity mapping(b) wavelet acuity mapping

L: Graphics and Virtual RealityL: Graphics and Virtual RealityL: Graphics and Virtual RealityL: Graphics and Virtual Reality

• Applications range from eye-slaved foveal Applications range from eye-slaved foveal Region Of Interest (ROI) VR simulators to Region Of Interest (ROI) VR simulators to gaze-contingent geometric modelinggaze-contingent geometric modeling

• Examples:Examples:• flight simulators (peripheral display degradation)flight simulators (peripheral display degradation)

• driving simulators (driver testing)driving simulators (driver testing)

• gaze-based dynamic Level Of Detail modelinggaze-based dynamic Level Of Detail modeling

• virtual terrainsvirtual terrains

L: Graphics and Virtual Reality L: Graphics and Virtual Reality (cont’d)(cont’d)L: Graphics and Virtual Reality L: Graphics and Virtual Reality (cont’d)(cont’d)

Fig.52: Gaze-Fig.52: Gaze-contingent contingent Martian terrainMartian terrain

• subdivided subdivided quad meshquad mesh

• per-block LODper-block LOD

• resolution resolution level based on level based on viewing viewing direction and direction and distancedistance

L: Graphics and Virtual Reality L: Graphics and Virtual Reality (cont’d)(cont’d)L: Graphics and Virtual Reality L: Graphics and Virtual Reality (cont’d)(cont’d)

L: Graphics and Virtual Reality L: Graphics and Virtual Reality (cont’d)(cont’d)L: Graphics and Virtual Reality L: Graphics and Virtual Reality (cont’d)(cont’d)

L: Graphics and Virtual Reality L: Graphics and Virtual Reality (cont’d)(cont’d)L: Graphics and Virtual Reality L: Graphics and Virtual Reality (cont’d)(cont’d)

L: Graphics and Virtual Reality L: Graphics and Virtual Reality (cont’d)(cont’d)L: Graphics and Virtual Reality L: Graphics and Virtual Reality (cont’d)(cont’d)

M: Human-Computer Interaction and M: Human-Computer Interaction and Collaborative SystemsCollaborative SystemsM: Human-Computer Interaction and M: Human-Computer Interaction and Collaborative SystemsCollaborative Systems

• Applications range from eye-based Applications range from eye-based interactive systems to collaborationinteractive systems to collaboration

• Examples:Examples:• intelligent gaze-based informational displays (text intelligent gaze-based informational displays (text

scroll window synchronized to gaze)scroll window synchronized to gaze)

• ““self-disclosing” display where digital characters self-disclosing” display where digital characters responded to user’s gaze (e.g., blushing)responded to user’s gaze (e.g., blushing)

• multiparty VRML environmentsmultiparty VRML environments

M: Human-Computer Interaction and M: Human-Computer Interaction and Collaborative Systems Collaborative Systems (cont’d)(cont’d)

M: Human-Computer Interaction and M: Human-Computer Interaction and Collaborative Systems Collaborative Systems (cont’d)(cont’d)

Fig.53: GAZE Groupware displayFig.53: GAZE Groupware display

Fig.54: GAZE Groupware interfaceFig.54: GAZE Groupware interface

• multiparty tele-multiparty tele-conferencing and conferencing and document sharing document sharing systemsystem

• images rotate to images rotate to show gaze direction show gaze direction (who is talking to (who is talking to whom)whom)

• document “lightspot” document “lightspot” (deictic “look at this” (deictic “look at this” reference)reference)

Eye-Based Interaction in Graphical Systems: Eye-Based Interaction in Graphical Systems: Theory & PracticeTheory & Practice

Eye-Based Interaction in Graphical Systems: Eye-Based Interaction in Graphical Systems: Theory & PracticeTheory & Practice

For further information:For further information:

http://www.vr.clemson.edu/eyetrackinghttp://www.vr.clemson.edu/eyetracking

• SIGGRAPH course notesSIGGRAPH course notes

• Eye Tracking Research & Applications SymposiumEye Tracking Research & Applications Symposium

Eye Tracking Research & ApplicationsEye Tracking Research & ApplicationsSymposium 2000Symposium 2000

November 6th-8th 2000, Palm Beach Gardens, FL, USA

Sponsored by ...

With corporate sponsorship from …

http://www.vr.clemson.edu/eyetracking/et-conf/http://www.vr.clemson.edu/eyetracking/et-conf/

Don’t forget to attend:Don’t forget to attend:Don’t forget to attend:Don’t forget to attend:

LC Technologies Unilever Corp. SensoMotoric Instr.

Eye-Based Interaction in Graphical Systems: Eye-Based Interaction in Graphical Systems: Theory & PracticeTheory & Practice

Eye-Based Interaction in Graphical Systems: Eye-Based Interaction in Graphical Systems: Theory & PracticeTheory & Practice

DemonstrationDemonstration

GAZE Groupware SystemGAZE Groupware System