do we need eye trackers to tell where people look?

CHI 2006 Work-in-Progress

April 22-27, 2006 Montral, Qubec, Canada

Do We Need Eye Trackers to Tell Where People Look?

Sune Alstrup Johansen IT University of Copenhagen Rued Langgaards Vej 7, DK-2300 Copenhagen S, Denmark [email protected] John Paulin Hansen IT University of Copenhagen Rued Langgaards Vej 7, DK-2300 Copenhagen S, Denmark [email protected]

AbstractWe investigated the validity of two low-cost alternatives to state-of-the-art eye tracking technology: 1) prompting users to report from memory on their own eye movements, and 2) asking experienced web designers to predict the eye movements of a typical user. Users could reliably remember 70 % of the web elements they had actually seen. Web designers could only predict 46 % of the elements typically seen. Users were not particularly good at remembering the order of their fixations. We discuss how to further improve the validity of self-reported gaze patterns and suggest new areas that it may be used in.

KeywordsEye Tracking, Usability Evaluation, Visual Attention, Web Design, Cognition, User Centered Design.

ACM Classification KeywordsCopyright is held by the author/owner(s). CHI 2006, April 2227, 2006, Montral, Qubec, Canada. ACM 1-59593-298-4/06/0004.

H.5.2 User Interfaces: Evaluation/methodology, H.1.2 User/Machine Systems: Human information processing

IntroductionVisual perception is an essential part of users interaction with interfaces. Modern eye tracking

923



equipment makes it possible to record and analyze parts of this process. Which elements are actually seen? Where do users look first? What do users look at the most? Did modifications of the graphic design lead to a wanted change in user gaze patterns? Eye tracking has been criticized for being costly and tedious [1,9]. Older generations of eye tracking equipment did not deliver sufficient value for usability professionals. Difficulties calibrating the equipment to users with glasses, contact lenses, heavy make-up, or even dark/brown eyes were common. Precision was low, and tiny head movements could jeopardize the validity of the recorded eye tracking data. State-of-the-art eye tracking equipment has solved most of these problems, and accurate recordings of eye movements can be made without obtrusive headmountings or unnatural fixations of the head. However, some barriers for deploying eye tracking in usability studies still remains: Data analysis can be very time consuming, even with the use of advanced software tools that come with modern eye trackers. Furthermore, state-of-the-art eye trackers are relatively expensive, (especially when purchasing both the hardware and analysis software) with market prices above 20,000 $ (as of January 2006). Last but not least, eye tracking equipment may reduce the test validity, for instance by notably slowing the system response or by requiring users to re-calibrate between tasks. These barriers may well disappear with improvements of technology. Until this happens, there will be a need for alternatives to eye tracking by costly equipment. Even when tracking technology becomes mature, there

could still be room in the usability toolbox for a simple and fast method that everybody could use. In a similar way, paper and pen are often the preferred tools by usability professionals, although they have lots of advanced systems available for data collection and notation. For instance, stick notes and paper cards have survived over sophisticated digital tools for e.g. cardsort analysis of information architectures. Peoples patterns of fixation are known to be highly predictive of what they can remember afterwards [3], and people tend to have a consistent viewing pattern when they re-visit something that they have seen previously, cf. the scan path theory [10]. It is an open issue to what degree people can reliably remember where they have looked. Obviously, they can tell from moment to moment what they look at, and users can explain their own eye movements in details during retrospections of gaze recordings [4], but can they keep the attended elements in their memory even for a short while - before reporting on them?

ProcedureIn the first experiment, state-of-the-art eye tracking equipment (Tobii 1750 with Clearview 2.5.1 analysis software, cf. Figure 1) was used to track the actual eye movements of 10 users, while they searched for an answer to simple questions on 8 different web pages. For instance, they were asked: Where would you click on this web page to find the shop nearest to your residence? The user first pressed a button to get the question, and then again to view the webpage and search for an answer. Immediately after answering, the users would report their remembered eye movements by repeating

Figure 1: A Tobii eye tracker recording visual attention. Afterwards, the user reproduced gaze patterns from memory.

924



them as accurately as possible by looking at the screen with the original webpage shown again. This made it possible to track remembered eye movements, and later to analyze how well they compared to the original ones (cf. Figure 2). To analyze the data, each webpage was divided into a number of Areas of Interest (AOIs) on basis of the

gestalt laws (e.g. law of good continuation, law of proximity, law of similarity, low of closure, and law of symmetry). Each AOI was furthermore given a unique identifier and classified from a list of common web page elements, including banners, contact information, drawings, email address, input fields, logos, mixed content, navigation elements, pictures, search fields, text blocks, and URLs.

Green boxes indicate the defined Areas of Interest (AOIs).

Black text indicate the unique identifier given to each AOI for analysis purposes, and a text code according to the nature of the element.

Red circles indicate fixations. The sizes of the circles illustrate time spent looking at a particular point, e.g. bigger circles means longer time spent. Numbers in circles indicate the order of fixations.

Figure 2: Plot of a user repeating from memory his gaze pattern associated with the question Where would you click to find out if this hotel has a pool?

925



Seen 0 7 14 17 20 22 24 28 28 5

Remembered 10 15 20

In the second experiment, 17 web designers (all of them with more than 18 months of experience in web design) were asked to predict the eye movements by marking a typical user scan path on print-outs of the 8 web pages that had been used in the first experiment. The designers predictions were compared with typical user scan paths constructed by n-gram analysis of the total eye tracking data from the 10 subjects. N-grams are sequences of characters from the ASCII character set, where a bi-gram (N=2) consists of 2 characters, a tri-gram (N=3) consists of 3 characters, etc. Based on the unique identifiers of AOIs, each user scan path could be expressed as a sequence of ASCII characters. Typical user scan paths of each web page was constructed from the bi-grams with the highest appearance, matched together in a continuing order to create a complete sequence. The first and last AOI that had been seen by most people would define the beginning and the end of the sequence.

On average, users had 1.9 false memories per web page (SD=1.4), with a non-significant tendency that web pages with many AOIs would introduce more false memories than simple ones. For instance, on the simplest web page with only 7 pre-defined AOIs, users had only 0.5 false memories, while on the most complex one with 28 pre-defined AOIs, they had an average of almost 3 false memories. The users were not particularly good at remembering the actual order by which they had looked at AOIs. We measured this by calculating the Levensthein Distance (LD) between the sequence of AOIs seen [5,8] and the sequence remembered. On average, the LD was 16 (SD = 12.8) and this was highly dependent on the length of the sequence to remember (R2 = 0.91, p

do we need eye trackers to tell where people look?

Documents