Behavior Research Methods & Instrumentation1974, Vol. 6 (6),525·530

METHODS & DESIGNSHigh-speed data processing and unobtrusive

monitoring of eye movements

ROBERT H. LAMBERT and RICHARD A. MONTYu.s. Army Human Engineering Laboratory

Aberdeen Proving Ground, Maryland 21005

and

ROBERT J. HALLUniversity ofNevada. Las Vegas, Nevada

A system has been developed that monitors eye movements without imposing any unnaturalconstraints or mechanical attachments on the subject. The system also features high-speed on-line dataprocessing.

SYSTEM CONFIGURATION

Figure l. Artist's illustration of the laboratory.

capability that provides immediate feedback to theexperimenter or to the subject if desired.

in an armchair locatedfront of a polarized,

It is convenient to think of the oculometer asconsisting not only of the hardware for monitoring theeyes and reducing the data, but also of the environmentand the stimulus-projection room. Conceived of in thisway, the system can be described as consisting of aviewing studio separated from a camera room by arear-projection screen, and a control room, all shown inthe artist's illustration of Figure 1 and represented in thefunctional block diagram of Figure 2.

The viewing studioThe subject is seated

a p p ro ximately 1.5 m in

This research may be reproduced in full or in part for anypurpose of the United State Government. Most of thedevelopment of the instrumentation was done while the first andthird authors were with the Special Projects Division, EG&G,Inc., Las Vegas; Nevada under contract for the U.S. ArmyHuman Engineering Laboratory.

When the range of eye behaviors are viewed in theirentirety, it seems fair to say that the state-of-the-art ofeye-movement recording is still in its infancy. Atpresent, no single technique existing or underdevelopment can adequately record the entire range ofeye-movement behavior without compromising orinterfering with normal visual processes. Therefore, oncethe experimenter has defined his objective, he must lookat tradeoffs between a method's accuracy and the extentto which it distorts and constrains the behavior hewishes to study (Hall & Cusack, 1972). For example, ifthe experimenter wishes to observe eye movementswithout inducing severe postural constraints orrestricting head movement, he must be willing tosacrifice some degree of tracking accuracy. In manysituations, however, lower accuracy can be compensatedfor by partitioning and distributing the stimulussituation under investigation.

The present paper describes an eye-movement cameraor oculometer designed to overcome some of theinadequacies of earlier systems, many of which havebeen described by Young (1963) and Young and Sheena(1975). Specifically, the oculometer described hereimposes no constraints on the subject that mightinterfere with comfort or normal viewing behavior; noris it necessary that the subject be aware that his eyes arebeing monitored. In addition, the system measures awide spectra of visual responses that include basicmeasures such as variability of fixation times and scanpatterns. and provides a data-reduction capacity forhandling large volumes of data with an on-line processing

525

526 LAMBERT, MONTY, ANDHALL

Ne. 4 IMAGE INTENSIFIER & LOWN•. 5 PROCESSOR I---t N•. 7 DATA INDEXER

LIGHT LEVEL TV CAMERA f----t TUCItU 11101 D(fHTO., lOCATIOIil Of N•. 6 VIDEO TAPE RECORDER I+- ElHIlMlIlT

'UPH (flnu oUID Il16HlIGH'- onU.'UlIO. SU'JHT

Of 'OIIT Of GAH ,'U'H Sm,AND ILtI' .... n STIMUli

r 1 1FUMf MUilIU

N•. 2 OPTICAL RELAY~

No. B TV MONITOR & MANUAL

& TRACKING SYSTEM TRACKING CONTROLS

~... 3 AUTOMATIC TRAUING ---------------sliva (OMItOllfO MIIiOH AIIO

FOCUSING O'TlCS DATA DISPLAY PANEL

i T

N. I SUBJECT STATION

lI'UllUIliUL 100M fOI VIUIU' mMUlI No.9 COMPUTER PDP 11 DATA fORMATlNG.uD Dun".G SUIJHT'S vaUAl UHAYIOI ON-LINE PROCESSING fEEDBAU TO SUBJECT &

--------------- UPERIMENTER

~ Ii 1 1 1

N•. 10 EXPERIMENTEn STATION .-- CRT DISPLAY & COPY UNIT 1.2M WORD DISK TAPE RECORDER J I TAPE RECORDER J 2

Figure 2. Schematic representation of the system.

rear-projection screen measuring approximately 1 m on aside as shown in Figure 3 and represented schematicallyby Block 1 of Figure 2. No constraints are imposed onthe subject except that he remain seated. Generallyspeaking, as long as the subject's head remains within aspace of approximately 1 cu ft, his eye movements aretracked automatically. Polarized lighting surrounds theviewing screen providing a comfortable level of roomillumination as well as suff1cient illumination to allowadequate TV viewing of the eye. The resulting ambientillumination of the subject ranges from 1 to 3 fc. Lightfrom a small unpolarized section of the surround whichis reflected by the front surface of the cornea is allowedto pass through the optical system, producing a brightspot (highlight) at the camera. It is the apparent positionof this highlight relative to the center of the pupil that isdirectly related to eye orientation, or line of sight. Thelow-light-level TV camera (Westinghouse ModelNo. STV 614) is concealed in what appears to be aspeaker box located directly beneath the screen.

The camera roomThe camera room houses a variety of devices used to

present stimulus materials to the subject, such as KodakCarousel or random-access projectors (Massey DickinsonCo, Inc., 9-11 Elm Street, Saxonville, Mass. 01701). It ispossible to utilize other readily available devicessuch asmotion-picture cameras or CRT projection devices. Inaddition, the camera room houses the concealed opticalrelay and tracking system (Block 2), the controller forautomatic eye tracking (Block 3), and the TV camera(Block 4).

The optical relay and tracking system, along with itscontroller, performs two major functions. The first isone of relaying information to the TV camera. In thisinstance, the image of the eye and highlight go to theimage intensifier and camera. The second function,tracking the movement of the eye, is performedautomatically by a system of mirrors and lens-focusingmechanisms. These mechanisms keep the eye in focuswithin the field of the stationery TV camera. The con­trol information for the servo-motors that track the eye

Figure 3. Subject seated comfortably in the studio. Thecamera is concealed behind the speaker grill, and the highlight isgenerated by the bright spot between the grill and the screen.

Figure 4. The operator's console and the data-reductionfacility. The CRT and hard-copy printer can be seen on the farright.

comes from a processor (Block 5) which determines thelocation of the highlight and the location of the pupil.The relative position of the highlight and the pupil isused to determine point of gaze. A signal proportional tothe position of the pupil is sent to the mirror-servoswhich continually maintain an image of the eye in theapproximate center of the field of view of the camera.

The control roomThe control room, shown in Figure 4, houses the

electronic processor, experimenter console,data-reduction equipment, etc., as represented by Blocks5-10 of Figure 2. The processor in Block 5 is essentiallyan analog computer housed in the experimenter'sconsole. It computes point of gaze by determining thedistance between the center of the pupil and thelocation of the highlight. This information is displayedon the data panel to the right of the TV monitor(Block 8), and it can also be recorded in digital form(Block 6) on the video tape recorder (Sony ModelNo. EV 200), shown on the left side of the operator'sconsole. This last feature is particularly important fromthe experimenter's point of view-should the digitalcomputer (Block 9), which normally processes the dataon-line, be shut down for any reason, the experimentcan still continue as the data can be stored indefinitelyon the video tape for processing at a later time.

The experimenter sits at the console and views thepupil. The joy stick in front of him is used tosuperimpose a circular cursor on the image of the pupil,i.e., acquire the target. Once this is accomplished, theexperimenter switches out of the manual mode and theautomatic tracker assumes control and tracks the eyeuntil the pupil is obscrued, in which case the target mustbe reacquired manually. Normally, tracking the pupil

MONITORING EYE MOVEMENTS 527

presents no problem, but the target can be lost if thesubject sneezes, laughs violently, rubs his eyes, etc.

Also displayed on the upper right side of the consoleis information concerning S (i.e., his number), thestimulus or slide number, the frame number and the X-Ycoordinates (Block 7). All of this information is eitherrecorded on video tape, sent directly to the computer, orboth.

Information concerning the eye's position and otherbehavior such as blinking and changes in pupil size canbe fed to the Digital Equipment Corp (DEC) PDP 11/20computer (Block 9), from either the electronic processorif real-time data processing is required, or from the videotape recorder if one wishes to process data from anearlier experiment. The computer, which interfaces witha DEC 1.2-million-word disk and two DEC industrialcompatible tape decks, performs several functions. First,it processes the data in real time so that information(such as pattern of fixations) can be provided to theexperimenter on the high-speed CRT computer terminal(Tektronix Model 4010-1). This information can also befed directly to projection devices such as therandom-access projector or a video projector in the eventone wants to change the material being shown to thesubject based on his previous performance. Secondly, itrelays the processed raw data to one of the tape decksfor storage and subsequent statistical or mathematicalanalysis. Thus. it is possible to go directly fromcalibration of the raw data to analysis of group datawithout any manual manipulation of that data. Thesame CRT terminal, together with its keyboard is usedto program the computer with the sequence of eventsthat constitutes an experiment. Finally, at any time, theCRT terminal can be used to provide a hard copy ofgraphics (scan patterns) or processed data, such asaverage fixation duration or frequency of fixations of aparticular stimulus, on the Tektronix Inc. Hard CopyUnit (Model No. 4610).

DEFINING A FIXAnON

The basic data consist of the X and Y positions ofeach eye fixation and the duration of those fixations.

The program used to define a fixation operates like acomplex filter, which determines: first, what data areusable (i.e., data that contain sufficient information todetermine the eye position); and secondly, whether asufficient number of frames are within the boundaryconditions used to define a fixation. The shortest timeused to define a fixation is six frames or 100 msec.

Data may be determined unusable for a variety ofreasons such as (a) highlight loss (i.e., highlight is belowthe threshold set for defining its presence; for example,blinking produces highlight loss) (b) severe tracking errorresulting from failure of the cursor used to outline andtrack the pupil to overlap the pupil exactly, (c) printingerrors stemming from video noise, etc., and (d) gross

528 LAMBERT, MONTY, AND HALL

head movements (e.g., laughing or sneezing) that maycause the camera to lose its view of the subject. Whenthis happens, the tracking device is brought back intolock by the operator's manual override.

Continuous point-of-gaze measurements arepartitioned into a temporal series of fixation coordinateswith the aid of the complex filter program mentionedearlier. X and Y refer to the measured horizontal andvertical coordinates of the subject's point of gaze on therear-projection screen. This program first collects a seriesof six measurements (representing a time span of100 msec) and determines whether the standarddeviations (in X and .Y) are less than preselectedallowable deviations and whehter the computed meanvalues of these coordinates indicate that the group maylogically be associated with the previously establishedfixation. If both conditions are satisfied, the programconcludes that the onset of a fixation has been detected,and the spatial coordinates and starting times arerecorded. As subsequent measurements are taken, X andY values are subtracted from these coordinates, yieldingLiX and LiY, respectively. Whether the new data shouldbe interpreted as a continuation of the establishedfixation or the possible initiation of the next fixation isdetermined as follows: if the absolute values of LiX andLiY are less than preset values (boundary conditions) thefixation time is extended accordingly. If not, theprogram will continue this test until a total of foursequential measurements fail to meet the boundarycondition criteria. Any data point measured during thissequence which falls within the acceptance boundariesautomatically extends the fixation to that point in timeand the test reinitiated with the following measurement.If the average of four sequential measurements, whichare individually rejected, lies within the same bound­aries, the fixation time is again extended and testingreinitiated as before. Failing this, two additionalmeasurements are taken and the standard deviation ofthe six collected values is computed. The six stored datapoints are then systematically replaced by incomingmeasurements until the standard deviation of the groupbecomes sufficiently small to indicate that a newfixation has been detected.

CALIBRATING THE SYSTEM

Two techniques have been effectively utilized togather calibration data when the subject first enters thestudio, without alerting him to the fact that his eyes arebeing monitored. Both techniques are performed underthe guise that the experimenter is making sure that theprojection devices are properly focused. The firsttechnique consists of projecting Landolt Cs in thecorners and center of the viewing screen and having thesubject systematically scan the Cs and report theposition of the opening. The second technique requires

the subject to view a Troxler calibration slide whichprojects a dark background with four dots, one in eachcorner of the viewing screen. This task provides excellentcalibration data, since the disappearance and fading ofthe dots in the peripheral position of the visual field is avery real phenomenon known as the Troxler effect(Troxler, 1804). The subject is instructed to stare at oneparticular colored dot and then to report thedisappearance or change in any of the other three. Thisprocedure is repeated for each of the dots in the fourcorners of the screen. Either procedure can becompleted in a matter of seconds.

STATISTICAL ANALYSIS

It is possible to go directly from monitoring of thesubject's eye movements to statistical analysis of groupdata without manual manipulation of the data. A singlesubject's data is readied for analysis on a given criterion(such as number of fixations, average duration' offixations, etc.) as it is collected and stored on themagnetic tape unit until all subjects in the experimenthave been tested. The 24K of core allows for areasonably sophisticated degree of processing. A popular.program has been the analysis of variance developed byButler, Karnlet, and Monty (1969) which enables anycombination of up to eight within and eight betweenvariables to be assessed. Results of these analyses aresimilarly displayed on the CRT and hard-copiesproduced on the printer when desired.

APPLICATIONS

The potential applications of the system arenumerous. Any material that can be projected on arear-projection screen can be displayed. In addition,modification of the stimulus display can be directlycontingent upon the subject's preceding visual or motorresponses. Further, the on-line capability can easily beused to simultaneously collect data on physiologicalparameters such as heart rate, EEG, etc., as well as dataon motor responses such as reaction time.

To date, the system has been used to study a varietyof problems. For example, Hall, Rosenberger, andMonty (1974) compared the eye movements of heroinaddicts and matched controls engaged in word andobject recognition tasks. Representative data are shownin Figure 5. While there was little difference in theaverage duration of fixations for addicts with old objects(i.e., objects they had seen earlier in the test session),regardless of whether they were looking at drug-relatedobjects or neutral objects, in all other cases (i.e., withobjects seen for the first time or with the nonaddictedsubjects) drug objects led to more fixations than neutralobj ects. These differences were attributed tomotivational or interest factors associated with theimportance of the material shown and basic differences

MONITORING EYEMOVEMENTS 529

Figure 5. Average duration of fixation of heroin addicts andmatched controls engaged in an object-recognition task (AfterHall, Rosenberger, & Monty, 1974).

in psychological and central nervous system processesthat regulate eye movement. The series of studiesindicated that addiction may lead to an altered sensorycapacity in the temporal domain associated with thegating and subsequent scanning of stimuli.

A different application of the oculometer systeminvolved examining eye movements during a simpledetection and threat-evaluation task (Hall, 1974). Theobjective was to select a representative problem thatwould demonstrate how the oculometer system mightcontribute to display research. A 75-min display taskwas devised that involved surveillance, tracking, andthreat evaluation. During this task, the amount ofeye-movement data generated and analyzed for a singlesubject was 270,000 frames, or a total of 2,970,000frames for all 11 subjects. The analysis of mean distancebetween successive fixations and mean fixationdurations provided no indication of fatigue or impairedperformance over time. However, fixation durationswere highly uniform during search periods when notargets were present and highly variable when the subjectwas observing and evaluating a target.

More recently Fisher (1974) has utilized the system instudies of reading and search behavior with children andadults. In a typical experiment, subjects view printedtext projected with a computer-controlledrandom-access slide projector. Sometimes the text isnormal; at other times it may be perturbed byeliminating spaces between words, alternating type, etc.The subjects are asked to read as quickly and accuratelyas possible (reading condition) or to search for a targetword embedded in the text (search condition).Representative adult data are shown in Table 1. Whenreading and searching normal text, fixation durations are

30

Alternating CaselNo Space

Normal CaselNormal Space

Rate180 256 36 148(Words Per Minute)

Fixation Duration266 260 417 310(in Milliseconds)

Words66 47 116 59(Read or Searched)

Words Per Fixation 1.05 1.9 .27 1.00Character Spaces

351 270 511 260(Read or Searched)Character Spaces

5.6 11.8 1.2 4.4Per Fixation

LIMITAnONS AND ADVANTAGES OF THE SYSTEM

Reading Search Reading Search

Table IRepresentative Data of Adult Subjects Engaged in

Reading and Search (After Fisher, 1974)

The advantages of the system are clear. First andforemost, unlike earlier systems, it does not interferewith or restrict the subject's visual behavior. The extentto which other systems do may now be determinedobjectively by comparing a subject's performances ontwo or more systems. Second, it is unnecessary to callthe SUbject's attention to the fact that his' eyemovements are of interest; thus problems of selfconsciousness. etc., can be avoided. Third, the systemcan handle voluminous quantities of data in real time. Amillion frames of data can be reduced for statisticalanalysis of a given criterion in less than 4 h. a task whichif performed manually or even semimanually, wouldtake months. And finally, when studying reading orsearch, each fixation can be identified by location andduration so that progressive and regressive saccades areeasily discernible.

The principal drawback of the system is its initialcost, but cost is directly dependent upon thesophistication of the data-reduction capabilities desired.A complete system such as the one described here wouldcost at least $100,000 if purchased commercially. Ofcourse, the time saved quickly compensates for the'

about the same. When the text is perturbed, readingfixations increase in duration dramatically whilesearch-fix ation durations increase only slightly.Likewise, reading speed decreases more dramaticallywhen the text is perturbed than does search speed.Further, perceptual span (character spaces per fixation)is smaller for reading than search and decreases furtherin both tasks when spatial cues are perturbed. Thisindicates that reading requires more fixations becauseless information is absorbed on each fixation than duringsearch. In short, these data show that both taskdemands, c.g., comprehension and spatial factors, canreduce the functional visual field of view.

,D/

//

/P

/

//

/ // /

/ / CONTROL

IT / / OLD a acI NEW 0- - - -()

ADDICTSOLD 0 0

NEW 0---~

NEUTRAL DRUG

TYPE OF OBJECT

u>10.12ct 700:LL

Z

zO~~ 60I-u«10.1xu>LL-

LL • 50au>10.1

z2Oct;=0:«LL0: a 40i5<DLLJ(!)

«0:LLJ>«

530 LAMBERT, MONTY, AND HALL

initial investment. A second limitation is the system'smonitoring accuracy, which is a direct result of notrestraining the subject's head movements in anydimension. Currently, the system can perform withapproximately 2 deg of positioning accuracy, but thiscould probably be improved with modification of thesystem software. Further, the system does not track wellwith individuals wearing glasses or with subjects whohave low-contrast pupils. Poor tracking results in the lossof data. Solutions to these two problems are well withinthe state of the art, and are currently underdevelopment.

REFERENCESButler, D. H., Karnlet , A. S., & Monty, R. A. A multi-purpose

analysis of variance FORTRAN IV computer program.Psychonomic Monograph Supplements, 1969, 2, 301·319(whole No. 32).

Fisher, D. F. Spatial factors in reading and search: The case forspace. Paper presented at a Specialists' Meeting on EyeMovements and Psychological Processes, Princeton, NewJersey, 1974.

Hall, R. J. Eye movement fixations and gating processes. Paperpresented at a Specialists' Meeting on Eye Movements andPsychological Processes, Princeton, New Jersey, 1974.

Hall, R. J .. & Cusack, B. L. The measurement of eye behavior:critical and selected reviews of voluntary eye movements andblinking. Tech Memo 18-72, U.S. Army Human EngineeringLaboratory, Aberdeen Proving Ground, Maryland, 1972.

Hall, R. J., Rosenberger, M. A., & Monty, R. A. An experimentalinvestigation of the visual behavior of young heroin addictsand matched controls. JSAS Catalog of Selected Documentsin Psychology, 1974, 4, 7.

Troxler, D. Uber das verschwmden gagebener gegenstandeinnerhalb unsers gesichtskreises. In K. Himly and J. A.Schmidt (Eds.), Opthalmologische Bibliothek 1804, 2, 51-53.

Young, L. R. Measuring eye movements. American Journal ofMedical Electronics, 1963, 2, 300·307.

Young, L. R., & Sheena, D. Survey of eye movement recordingtechniques. Behavior Research Methods & Instrumention,1975, in press.

(Received for publication August 29, 1974;revision received October 3,1974.)