State of the IT University

EyeGrip: Detecting Targets in a Series of Uni-directional Moving Objects Using Optokinetic Nystagmus Eye Movements

Shahram Jalaliniya - Diako MardanbegiIT University of CopenhagenPervasive Interaction Technology Lab

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN

1

MotivationInformation age & overwhelming users with dataData is getting more visual (e.g. Web, Facebook)Scrolling among visual data is becoming more popular

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN

2

Scrolling includes:ScrollingStopping the pageBringing the desired content back (not always an easy task)

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN EYEGripEyeGrip automatically detects moving images that seem to be interesting for a user among other scrolling images by monitoring and analyzing users eye movements.

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN

4

EYEGRIP works based on OKN eye movementsOKN is an eye movement that tend to track the motion of one element at a time in a set of unidirectional moving stimuliOKN: Optokinetic nystagmus is a combination of saccadic and smooth pursuit eye movements.

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN how does EyeGrip work?When one of the images grabs our attention we follow that image for a longer time that creates a peak in the OKN signal.

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN Experiment GoalTesting the feasibility of EyeGrip in different scrolling conditions:Different speedsMaximum number of visible images on the screen (manipulated by changing image width)

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN Experimental design (3 2 )

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN ApparatusHead-mounted eye tracker with the Haytham open source gaze tracker (20 Hz sampling rate)Laptop to display the scrolling images & collect eye data

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN Experiment TASKVisual search among faces: Participants should press space key as soon as they see Bill Clintons picture among other facesParticipants repeated the task for all 6 conditions40 random images of famous people is displayed in each condition where 7 was Bill Clintons photos

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN User errorThe error rate: total missing target images by participants divided by total number of targets

User error rate (%)Cond.123456SpeedslowslowmedmedfastfastImage width smallbigsmallbigsmallbig

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN Data AnalysisCleaning data: Removing 5 participants with less than 75% dataNormalization: finding left & right eye coordinates during the experiment by displaying 2 red circles a the beginning of each task. We used these coordinates to bring all the data in the same range (min-max normalization)Aggregation: we aggregated the data from all 15 participantsLabeling data: we used the space key to label the collected data

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN Event Detection algorithmWe used the default setting for the Multilayer perceptron algorithm in the WEKA with a single hidden layerHorizontal coordinate of the pupil center was the only featureSliding window is selected based on maximum performance30 frames for conditions 1,2,4 20 frames for conditions 3,616 frames for condition 5

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN Effect of speed & max no. of images10 cross-fold validation is usedRepeated measure ANOVASignificant effect of image width F(1,14) = 34.9, p < :0001No significant effect for speedBut higher speed caused more error (missing targets)

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN Design GuidelinesImages should move in a one direction at a certain speedThere should be a balance between moving speed & max number of imagesThe visual search task should not be very complex otherwise all images will draw equally high attention that increases falls positives

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN Study 1: a Picture selection system8 participantsSpeed: 37 secImage width: 18 Selecting Clinton pictures

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN Study 2: Mind reading Game10 participantsSelect 1 of 4 charactersParticipants are asked to count repetitions of selected personAccuracy:100%

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN Other suggested applicationsInteraction with scrolling menus (e.g. scrolling cards on Google Glass)EyeGrip for browsing Facebook pageAdvertisement on public displays Text reading assistant for small displays (slowing down text when a user has problem with a word)Assistant for visual inspection in production lines (automatically detecting unqualified products)

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN Related workPursuit [1]: is a calibration-free technique to detect limited number of moving objects on the screen using smooth pursuit eye movements.EyeGrip detects unlimited number of unidirectional moving objects.

[1] Melodie Vidal, Andreas Bulling, and Hans Gellersen. 2013. Pursuits: Spontaneous Interaction with Displays Based on Smooth Pursuit Eye Movement and Moving Targets. In Proceedings of UbiComp 13. ACM, 439448.

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN ConclusionsEyeGrip is a calibration-free and implicit eye interaction technique to select an object among other moving unidirectional objects (top-down attention)EyeGrip can be used in gaze-contingent user interfaces to detect what draws users attention (bottom-up attention)Simpler algorithms (e.g. threshold-based method) can be applied to detect the event in EyeGrip

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN

IT UNIVERSITY OF COPENHAGEN