towards interaction models derived from eye-tracking data

Interaction Models Derived From Eye-tracking Data .

Jacek Gwizdka & Michael ColeRutgers University, USA

jacek@gwizdka.com

http://jsg.tel

April 19, 2012

Towards

Eye-tracking?

2

Eye-trackers © Tobii

3

Eye-movement in UX Research

There is a Lot More Eye-tracking Datacan offer UX / HCI / IA

4

Eye-tracking Data

5

State2State1

State3

Patterns

Eye-movement Patterns

New methodology to analyze eye-movement patterns◦Model reading and Measure cognitive effort

◦Correlate with higher-level constructs

user task characteristics, user knowledge, etc.

6

7

Eye-tracking – Fundamentals

8

Reading Model Origins

Based on E-Z Reader model Rayner , Pollatsek, Reichle

◦ Serial reading

◦ Words can be identified in parafovial region

◦ Early lexical access (word familiarity) + Complete lexical processing (word identification)

2o (70px) foveal region parafoveal region

a bit MORE…

9

Two-State Reading Model

◦Filter fixations < 150ms (min time required for lexical processing)

◦Model states characterized by: probability of transitions; number of lexical fixations; duration length of eye-movement trajectory, amount of text covered

ScanRead

1-q

p

1-p

q

a bit MORE…

isolated fixationsfixation

sequences

10

Example Reading Sequence

Fixation sequence: (F F F) F (F F F) F F F F (F F F F F F) FReading model states: R S R S S S S R S

Reading state – R | Scanning state – S

Cognitive Effort Measures of Reading

Reading Speed

Fixation Regression

Perceptual Span

Fixation Duration (“lexical processing excess”)

11

foveal region

a b c d

Perceptual span = Mean(a,b,c,d)

regression

excess

User Study 1: Cognitive Effort and Tasks

12

OBI: advanced obituaryINT: interview preparationCPE: copy editingBIC: background information

N=32

MORE…

Journalists’Information Search

Eye-data and Cognitive Effort Measures

Cognitive effort measuresreading speedmean fixation durationperceptual spantotal fixation regressions

Task complexity by designCopy Editing (CPE) Advance Obituary (OBI)

Search effort task timepages visitedqueries entered

Subjective Task Difficulty

CPE INT BIC OBI

As expected: Copy Editing CPE easiestAdvance Obituary OBI most difficultSig: Kruskal-Wallis χ2 =46.1, p<.0001

13

Eye-data and Task Characteristics

14

Measure Related Task Characteristics

Frequency of reading state transitions

SR bias to readAdvanced obituary and Interview preparation tasks: search for document; task goal not specific

RS bias to scanCopy Editing task: search for segment and task goal specific

ScanRead

1-q

p

1-p

q

MORE…

Copy Editing Interview preparation

User Study 2: Assessing User’s Knowledge

Search in Genomics Domain

15

N=40

MORE…

Rate own domain knowledge

Results: Modeling Domain Knowledge

16

Reading Model features & cognitive effort measures

Eye-tracking Data

Domain knowledge MeSH-based self-ratings

predicted

self-rated

Results: Modeling Domain Knowledge

17

predicted

self-rated

Reading Model features & cognitive effort measures

reading seq length and total durationperceptual spanfixation durationregressions…

Reading Model

Eye-tracking Data Random Forest Model

Domain knowledge MeSH-based self-ratings

m

tk=PDK

m

iii

*5

)*(1

For each user predict

build model

agglomerative hierarchical clustering (Ward’s)

PDK: Participants’ domain knowledge

MORE…

Tobii

Eye-tracking is Coming to Us!

18

Eye-tracker © Tobii | Laptop © Lenovo

From Eye-tracking Data to Interaction Models

Measures derived from eye-movement patterns

Macro use task characteristics, cognitive effort, domain knowledge

Meso reading patterns

Micro eye-gaze positions + timing

19

From Real-time Interactions to Applications

Cognitive Load

Domain Knowledge

Information Relevance

Adapt presentation& content

Enable Interaction(e.g., disabilities)

Task Aspects

Eye-TrackingData

Standard input devices (mouse, keyboard)

other psycho-physiological devices (EEG, SCR, HRV)

Better understand interaction

Micro-level

Macro-level

Applications

Cognitive Load Model

ReadingModel

Task Model

Models

20

…

…

…

21

Thank You! Dziekuje!

Funding: Google, HP, IMLS (now funded by IMLS CAREER)Collaborators: Drs. Nicholas Belkin, Art Chaovalitwongse (U Wash), Xiangmin Zhang,

Ralf Bierig (Post Doc); PhD students: Michael Cole (co-author), Chang Liu, Jingjing Liu, Irene Lopatovska

+ many Master and undergraduate students …

Acknowledgements:

22

Pytania?

More info & contact http://jsg.tel