Analysis of Eye-tracking on Students Searching for Information in a Heterogeneous Resource Web Portal

Jason HolmesKent State University

Heather BryanKent State University

Introduction

This short paper will present an overview of eye-tracking metrics and present an analysis of

eye-tracking data collected from 38 students (4th - 12th grade) searching a pre-release version of

a heterogeneous resource web portal called SchoolRooms®.

SirsiDynix®, a major library system vendor, has partnered with INFOhio, Ohio’s statewide

cooperative school library and information network, to develop a new product called

SchoolRooms®. SchoolRooms® is an online education portal that includes resources selected by

teacher-librarian teams which meet national and state academic content standards. These web

resources are organized into a browseable, categorical hierarchy divided into topical sections

called “Rooms.” Using “Explore a Room” provides structured access to teacher-librarian selected

web resources. SchoolRooms® also features a federated or single-search capability that can

simultaneously search library catalogs, electronic databases, web sites selected by the

teacher-librarian teams, and the generic web using a commercial search engine.

Five tasks were developed to study the effectiveness and efficiency of students’ interaction with

SchoolRooms in searching for information. As part of this study we were able to track eye

movements of participants using a Tobii® 1750 eye-tracking monitor. This monitor unobtrusively

tracks the position of the subjects’ eyes on the screen using an infrared camera built into the

bezel of the monitor. The monitor records samples of several variables, including eye position

(X-Y coordinates for each eye independently), pupil dilation, and distance from screen every 20

milliseconds. Using this data, Clearview® software can be employed to represent the fixations,

scanpaths, and areas of interest (AOIs) for each student.

Overview of eye-tracking metrics

As a relatively new tool, eye tracking metrics are still evolving. Current metrics for eye tracking

include fixations (duration, rate, and total count), scanpaths, saccades, areas of interest, and

pupil dilation (Granka et al., 2004; Duchowski, 2006). Bojko (2006) compared the number of

fixations associated with identical tasks across two interfaces. Measures of pupil dilation are

being studied as indicators of affective processing (Partala & Surakka, 2003) and cognitive

processing (Iqbal, Zheng & Bailey, 2004).

Another common method of analyzing eye tracking data is the use of areas of interest (AOIs). An

AOI is a section of an interface; interfaces can be divided into numerous AOIs to suit different

studies. The eye tracking data within an AOI can be compared with that of other AOIs. Granka et

al. (2004) created an AOI (called “LookZones” in their study) for each displayed result of a search

engine to study how users view the results. Russell (2005) split interfaces into numerous AOIs

and examined the duration and order of attention given to each; results were used to suggest

users’ expected locations of interface features.

Duchowski (2006) advocates using the metrics above to place eye tracking into the usability

contexts of satisfaction, efficiency, and effectiveness. Fixations and scanpaths allow verbal

expressions of satisfaction to reference the area of the interface that was viewed as the

comment was made. Metrics such as fixation time and number of fixations may isolate areas

with understandability and efficiency problems. Aggregated scanpaths and “hot spots” can show

the efficiency of an interface by portraying the visual area coverage of an interface at a point in

time.

“Hot-spot” images demonstrate the powerful qualitative analysis made possible with

eye-tracking technology (see figure 1). “Hot-spot” images are a static representation of the

fixation points aggregated into one view. The “hotter” the area, indicated by more red color, the

longer that area was viewed. Fixation data presented in “hot-spots” can be aggregated based on

demographics or other independent variables. For example, males and females can be

aggregated into separate images in order to analyze whether there are gender differences in

fixation. Similarly, we might want to look at the difference in fixation patterns between 5 th

graders and 12th graders.

Results

Due to the intensely graphical nature of eye-tracking analysis, a complete report is not possible

given the space limitations of this short paper. More visualization of eye-tracking data will be

provided in the poster session format. Figure 1 provides a sample visualization of aggregate

fixation data on the homepage of the elementary school level of SchoolRooms®. Scanpaths and

AOI data will also be presented.

Conclusion

The great benefit of integrating eye-tracking into user studies is that it provides data that no

amount of asking, probing, or thinking aloud protocols can get. It is possible to visualize and

quantitatively analyze precisely where students are looking on a computer screen at any given

time. Through scanpath analysis, it is possible to see and analyze the paths users take to

accomplish a searching goal. AOI analysis can provide quantitative data about the percentage of

time a user looks at a particular area of a screen in relation to the other areas. The

measurement of pupil dilation has the potential to give hard data on cognitive load experienced

when searching for information. This technology opens a whole new horizon of questions about

human-computer interaction, particularly in information seeking and using. In combination with

already established user observation techniques, it is now possible to triangulate the conscious

motivation of users with the perceptual data provided by eye-tracking. We can do more than ask

people what they are looking at, we can see what they are looking at. Other methods are still

needed to help determine why a person is looking at a particular spot, or what they are thinking

as they move around an interface, but eye-tracking provides the answer to “where are they

looking?”.

Figure 1: Sample “hot-spot” image of fixation data.

References

Bojko, A. (2006). Using eye tracking to compare web page designs: a case study. Journal of Usability Studies, , 3, 112-120.

Duchowski, A.T. (2006). High-level eye movement metrics in the usability context. Paper presented at the 2006 ACM Conference on Human Factors in Computing Systems. Retrieved

October 28, 2006 from: www.amber-light.co.uk/CHI2006/pos_papr_duchowski.pdf

Granka, L.A., Joachims, T., & Gay, G. (2004). Eye-tracking analysis of user behavior in

WWW-search. In Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 478 - 479). New York: ACM Press.

Iqbal, S.T., Zheng, X.S., & Bailey, B.P. (2004). Task-evoked papillary response to mental

workload in human-computer interaction. In CHI '04 extended abstracts on Human factors in computing systems (pp. 1477 - 1480). New York: ACM Press. Retrieved October 28, 2006 from:

http://www.interruptions.net/literature/Iqbal-CHI04-p1477-iqbal.pdf

Partala, T. & Surakka, V. (2003). Pupil size variation as an indication of affective processing.

International Journal of Human-Computer Studies, 59, 185-198.