Applied Ergonomics 34 (2003) 499–509

ARTICLE IN PRESS

*Correspondin

E-mail addres

0003-6870/$ - see

doi:10.1016/j.ape

Alarm mistrust in automobiles: how collision alarm reliabilityaffects driving

James P. Blissa,*, Sarah A. Actonb

a Psychology Department (MGB 244B), Old Dominion University, Norfolk, VA23529, USAb Foster-Miller, Inc., Boston, MA, USA

Received 19 November 2002; received in revised form 2 June 2003; accepted 11 July 2003

Abstract

As roadways become more congested, there is greater potential for automobile accidents and incidents. To improve roadway

safety, automobile manufacturers are now designing and incorporating collision avoidance warning systems; yet, there has been

little investigation of how the reliability of alarm signals might impact driver performance. We measured driving and alarm reaction

performances following alarms of various reliability levels. In Experiment One, 70 participants operated a driving simulator while

being presented console emitted collision alarms that were 50%, 75%, or 100% reliable. In Experiment Two, the same participants

were presented spatially generated collision alarms of the same reliability levels. The results were similar in both experiments: alarm

and automobile swerving reactions were significantly better when alarms were more reliable; however, drivers still failed to avoid

collisions following reliable alarms. These results emphasize that alarm designers should maximize alarm reliability while minimizing

alarm invasiveness.

r 2003 Elsevier Ltd. All rights reserved.

Keywords: Alarm; Driving; Collision

1. Introduction

Recently, the federal government has designed andtested Intelligent Transportation System (ITS) technol-ogies in an effort to decrease collision rates by creating‘‘intelligent’’ roadways and enabling safer automobileinteractions (National Highway Traffic Safety Admin-istration, 1997). As part of that effort, many automobilemanufacturers have studied and included collisionavoidance warning systems (CAWS) in new car models(Tilin, 2002; NHTSA, 2002). Properly designed andimplemented, CAWS would notify drivers about poten-tial dangers from roadway departures and otherautomobiles, particularly in rear-end collisions (Hirstand Graham, 1997; Chen et al., 1997; Araki et al., 1996).As noted by Parasuraman et al. (1997), rear-endcollisions constitute a sizeable proportion of automobileaccidents.

Aviation was the first transportation division toimplement CAWS. Because of increasing numbers ofnear-collisions, the Traffic Collision Avoidance System

g author. Fax: +1-757-683-5087.

s: jbliss@odu.edu (J.P. Bliss).

front matter r 2003 Elsevier Ltd. All rights reserved.

rgo.2003.07.003

(TCAS) was implemented to notify pilots of impendingcollisions with other aircraft. Although the TCASsystem improved overall safety, pilots voiced concernsabout false alarms and improper directives (Shapiro,1994). Since its initial development, the algorithmscontrolling aviation TCAS alarms have improved.However, the false alarm issue has not been resolvedcompletely (Bliss et al., 1999).

Researchers have studied the technological challengesof implementing automotive CAWS. Currently, severaldifferent versions of automotive CAWS are available(see Mazzae et al., 1995; Jansson et al., 2002). Thetechnology incorporated in CAWS ranges from head-updisplays that simply notify the operator about impend-ing collisions to intelligent computers that could takeover vehicle operations. One type of CAWS works onthe basis of a turn signal onset rule, where a warningsignal occurs if the driver activates the turn signal whenthere is another vehicle blocking the intended path. Sucha system may be an effective deterrent for collisions, butonly if drivers reliably use their turn signals (Tijerniaand Hetrick, 1997).

Other systems employ radar technology to initiatewarnings when the driver’s automobile is close to

ARTICLE IN PRESSJ.P. Bliss, S.A. Acton / Applied Ergonomics 34 (2003) 499–509500

another vehicle, regardless of turn signal use (Tijerniaand Hetrick, 1997). Crash avoidance potential is high inthese situations, but the probability of nuisance alarms(those signals that are invalid for the current taskoperation context) is also high (Xiao and Seagull, 1999).Alarm signals may frequently occur in ‘‘normal’’contexts such as parking lots, or when cars are followinga close, parallel course.

In addition to technical challenges, lack of warningcompliance is a concern. Researchers have proposedmany ways to improve operator detection of alarmsignals, such as using auditory stimuli (Lilliboe, 1963;Bronkhorst et al., 1996) and manipulating signalparameters to heighten perceived urgency. Haasand Edworthy (1996) successfully increasedperceived signal urgency by raising fundamental fre-quency, raising sound pressure level, and reducinginterpulse intervals.

Although such changes may improve signal detection,they do not guarantee signal compliance. Even whensignals are clearly detectable, unreliable automationmay cause mistrust (Parasuraman and Riley, 1997).Furthermore, Bliss (1993) demonstrated that alarmmistrust could cause operators to ignore alarms often.As noted within Xiao and Seagull’s (1999) usefultaxonomy, a variety of alarm problems are possible,including false alarms (alarm stimuli without corre-sponding system problems), nuisance alarms (alarmstimuli indicating a potential problem in an unrelatedcontext), and inopportune alarms (cascading alarmstimuli indicating minor problems). False and nuisancealarms are a particular concern for automotive CAWS.Research suggests that drivers may ignore or disableCAWS if its alarms do not reliably signal danger(Tijerina and Garrott, 1997). Such disregard, a directconsequence of mistrust, may be compounded underhigh levels of workload (Bliss and Dunn, 2000).

Researchers have studied operator trust in manycontexts. Muir (1989) presented a broad theory ofmachine trust based on social trust theories from Barber(1993) and Rempel et al. (1985). Barber (1993) claimedthat social trust relies on one’s belief that a partnerfollows natural and moral laws, that the partner hastechnical competence, and that the partner acts tobenefit others. Rempel et al. (1985) stated that humansshow trust because a partner behaves predictably, has adependable disposition, and follows moral laws consis-tently. Muir (1989) aggregated these theories andapplied them to the human–machine relationship.Recently, researchers have applied Muir’s theory ofalarm mistrust to the human–automation relationship.They have validated Muir’s theory, and have documen-ted alarm related performance decrements in a variety ofcomplex task areas such as process control (Lee andMoray, 1992), medicine (Bitan et al., 2000), aviation(Pritchett, 2001) and driving (MacKinnon et al., 1993).

In the laboratory, researchers have improved variousaspects of alarm responsiveness by increasing alarmurgency (Haas and Casali, 1995; Hellier et al., 2002),advertising high reliability rates to operators (Bliss et al.,1995), and incorporating redundant information dis-plays (Selcon et al., 1995). Each of these manipulationsheightened operator situation awareness, resulting infaster reaction times and increased compliance rates. Itis important that these manipulations be validated usingan automotive task scenario, because of the potential forfalse alarms to be generated by automotive CAWS(Tijerina and Garrott, 1997). Farber and Paley (1993)judged the impact of false alarms on driver behavior tobe a critical human factor research issue.

1.1. Goals and hypotheses

The purpose of this research was to document theeffects of alarm unreliability in an automotive context.Alarm systems may be unreliable in many ways,including failing to emit signals when such emissionsare warranted (missed alarms), and emitting signalswhen emissions are not warranted (false alarms).Because of the implications of false alarms in theautomotive cab, we chose to study false alarms in thisresearch. As demonstrated by Baber (1994), a significantissue with false alarms is their potential for driverdistraction, causing operators to redirect their attentionaway from the primary driving task. False alarms arealso ubiquitous, because alarm manufacturers have alegal predisposition to warn.

Using laboratory-based research paradigms, research-ers have demonstrated performance deficits resultingfrom frequent false alarms in active collision avoidancesystems, where collisions might result from actionstaken by the driver. For example, Dingus et al. (1997)studied the impact of false collision avoidance alarms onyoung and old drivers, finding that young driversexhibited signs of alarm mistrust in a rear-end collisionscenario. Such research usually involves incorporatingfalse alarms in an attempt to manipulate aspects ofMuir’s trust framework; degrading the predictabilityand dependability of the alarm system. This approach isalso followed in the current research; however, thecollision avoidance system was a passive one, wherethreats approached the driver. The driver had to takeevasive action to avoid the collision. Such situationscommonly occur on multi-lane highways, when a carbegins to change lanes or approaches from the rear, notrealizing another car is in the way. We presented driverswith alarms of varying reliability levels signaling thethreat of a collision from the rear. In the firstexperiment, the alarm signals originated from the centerconsole. In the second, signals originated from variousspatial locations within the automotive cabin, corre-sponding to the location of the approaching vehicle. In

ARTICLE IN PRESSJ.P. Bliss, S.A. Acton / Applied Ergonomics 34 (2003) 499–509 501

each case, the driver and alarm system operated inparallel, as described by Tijerina and Garrott (1997).

In past research, alarm mistrust has consistently beenassociated with reduced alarm response frequency (Bliss,1993; Sorkin, 1988). Furthermore, in their final reportfor the Automotive Collision Avoidance Systems(ACAS) Program, NHTSA suggests that drivers willlikely ignore collision alarms that are not reliable(NHTSA, 2000). Therefore, participants were expectedto respond less frequently to alarm systems of lowreliability. We also measured alarm reaction time, alarmreaction appropriateness, and appropriateness of driv-ing actions following the alarms. However, due toinconsistent results regarding these variables in thealarm mistrust literature, we made no hypothesesregarding them.

Fig. 1. The HMMWVt driving simulator.

2. Experiment One

2.1. Method

2.1.1. Experimental design

For this experiment, we manipulated auditory alarmreliability among three groups (50%, 75%, or 100% truealarms). Dependent measures reflected driving perfor-mance and reactions to alarms. Upon hearing an alarm,drivers were to determine whether it actually signaled anapproaching car (true alarm) or not (false alarm). If thealarm was true, drivers were told to swerve to avoidbeing struck from behind by the approaching car. Inaddition to transcribing videotapes to capture steeringmovements, experimenters relied on vocalizations bydrivers to determine whether and in what direction theyswerved following the alarm signals. Magnitude ofswerving, while not measured directly, was inferredfrom collision rate. If drivers did not swerve enough,they would collide with the approaching car.

2.1.2. Measurements

Driving performance measures included driving reac-tion appropriateness (the proportion of trials, out of 12,where drivers swerved in the proper direction after truealarms and did not swerve after false alarms), andcollision frequency (the proportion of trials during theexperiment where reactions to alarms resulted incollisions with approaching vehicles). Collision fre-quency was calculated by dividing collision occurrencesby the number of threats (6, 9, or 12, for reliabilitygroup 50%, 75%, or 100%, respectively).

Measured reactions to auditory alarms included thefrequency with which drivers swerved to avoid ap-proaching cars (after true and false alarms); the timetaken by drivers to swerve, in seconds from the onset ofthe auditory alarm stimulus to the beginning of theswerve; and the proportion of trials when participants

appropriately reacted to true alarms and ignored falsealarms. Because of the importance of swerving, wecoded it visually from videotapes and confirmed it fromthe drivers’ descriptions of their actions.

Participants were randomly assigned to a 50%, 75%,or 100% reliability group. Following a procedure usedin previous alarm mistrust research (see Bliss et al.,1995), participants were told the alarm system’sreliability before performing the experimental task.Because alarm mistrust accumulates as exposure to falsealarms continues (see Bliss and Kilpatrick, 2000), thisensured that participants adopt stable and appropriatetrust levels within each experimental session. It alsoreflected real-world situations where drivers may havepreconceived notions (from the media or other drivers)about CAWS reliability.

2.1.3. Participants

To ensure statistical power greater than 0.80 at thep ¼ 0:05 level (Cohen, 1988), data were collected from70 undergraduate volunteers who received course creditstoward their general psychology classes at The Uni-versity of Alabama in Huntsville. All were licenseddrivers living in the Huntsville, AL area. Fortyparticipants were male, and 30 were female (this ratiowas similar within all groups). The average age ofparticipants was 22.1 years. The driving experience ofparticipants ranged from 1 to 28 years, with an averageof 6.6 years.

2.1.4. Materials

A driving simulator owned by the DistributedSimulation Group at the US Army Aviation and MissileCommand Center (AMCOM—Redstone Arsenal, AL)was used as a vehicular platform for the primary,driving task (see Fig. 1). The simulator was a militaryHMMWVt (multipurpose vehicle) modified to serve asan automotive simulator. The front window of the

ARTICLE IN PRESS

Fig. 2. A sample scene from the driving environment.

J.P. Bliss, S.A. Acton / Applied Ergonomics 34 (2003) 499–509502

HMMWV faced three large (238.76 cm high� 177.8 cmwide) screens. The total horizontal field of view was135�, each individual screen’s horizontal field of viewwas 45�, and the vertical field of view was 33.75�.Although there was no motion base, physical andfunctional fidelity were high (see Hays and Singer,1989). The steering wheel, brakes, accelerator, andgearshift controls were functional, and visual andauditory stimuli faithfully represented environmentalchanges according to driver actions. In addition to thefront view screens, participants also viewed a rear-viewmirror display inside the vehicle cabin. This display wasa 1400 flat-panel display that showed a rearward view ofterrain that had already been passed.

The simulated driving environment was created usingMulti-Gen (Creator)t on a Silicon Graphics computer.The environment faithfully represented a 1609.34 km2

area from Decatur, AL (West) to Huntsville, AL (East);and from the Tennessee State Line (North) to Arab, AL(South). The corridor contained the length of Interstate565, was approximately 32.19 km from East to West,and 80.47 km from North to South. The environmentwas initially developed at Boeing’s Advanced Comput-ing Laboratory in Huntsville, AL and translated at theArmy-NASA Virtual Innovations Laboratory (AN-VIL), Marshall Space Flight Center, AL. The environ-ment was refined and rendered by personnel atAMCOM’s Distributed Simulation Center. After theinitial environment was completed, AMCOM personnelincorporated it into existing distributed interactivesimulation software to allow representation of othervehicles and terrain.

To create a simulated environment of I-565, theexperimenter selected and placed polygons and lines tocreate the road, background, overpasses, exit ramps,road signs, and landmarks. Land topography wasdetermined by referring to topographical maps andGlobal Positioning System elevation estimates. Therewere approximately 255 stationary polygons in theenvironment plus moving objects such as vehicles.Objects such as road surfaces, signs, overpasses andtrees were given a realistic appearance by placing texturemaps on polygon surfaces. This technique facilitatedreal-time model rendering (see Fig. 2).

During the simulation, the view of the computer-generated imagery was continuously updated at a rate ofapproximately 20 Hz to match the simulated vehicle’smovement. The update rate for the rear-view mirrordisplay was 60 Hz.

While driving in the simulation at 112.65 kph,participants were required to react to 12 intermittentauditory alarms spaced randomly throughout thesession. The alarms originated from a center frontconsole located behind the automobile cabin firewall.The alarm stimulus, audible for 4 s, consisted of regular1000 Hz sine wave pulses at approximately 90 dB(A).

The signal was obtained from the Huntsville ChryslerElectronics Division, was digitized, and then incorpo-rated into the driving simulation to coincide with thepassage of particular landmarks. Designed initially foruse in Dodge Neont automobiles, the signal was ingeneral accordance with Haas et al.’s (1996) guidelinesfor high urgency alarms, and was the prototype signalfor collision avoidance systems under development byChrysler. The 4-s audibility duration was chosen toensure that participants detected the signal, and wasvalidated during a pilot study.

True alarms signaled the appearance and approach ofa car from behind the participant’s automobile. Theapproaching cars appeared in the driver’s rear viewmirror display at the same time as the alarm signalsounded. Approaching cars appeared 91.4 m (300 feet)behind the participant’s car, traveled at 144.84 kph(90 mph), and could approach from the right rear, rear,or left rear. The 12 alarms were scheduled to appearrandomly according to simulated distance traveled(interstimulus distance ranged from 0.40 to 4.43 km(0.25–2.75 mile), with an average of 2.41 km (1.50mile)).Drivers were dependent upon the rear view mirrordisplay to determine whether the alarm was true or false,and to determine the proper action to take. If a carapproached from the left or right rear areas, it remainedhalfway in the participant’s lane, so that the participantneeded to swerve left or right to avoid a collision.

Participants were told to swerve left to avoid carscoming from the right rear, and right to avoid carscoming from the left rear or rear. They were told toverbally indicate their action while taking it, to facilitatedata recording. If an alarm was false, the signal wouldsound but there would be no approaching car. Therewere no missed alarms (an approaching car without analarm signal) included in the scenario.

ARTICLE IN PRESS

Fig. 3. Frequency of console alarm reactions as a function of signal

reliability.

J.P. Bliss, S.A. Acton / Applied Ergonomics 34 (2003) 499–509 503

In addition to the simulated alarms and drivingenvironment, the experimenters also used an informedconsent form and two data collection forms. Theinformed consent form described the purpose ofresearch, risks, benefits, and withdrawal procedures.The background questionnaire included informationabout age, driving experience, sex, vision, hearing, andcomputer game familiarity.

2.1.5. Procedure

After completing the informed consent form, partici-pants completed the demographics questionnaire andreviewed task instructions with the experimenter.

The experimenter then led each participant into thesimulator room. At this time, participants were shownthe driving simulator. The participant started thevehicle, engaged the transmission, and drove down theroad for approximately 3.22 km (2 mile). After reachingthe 3.22 km point, participants made a left hand turn, aright hand turn, accelerated, and braked. Participantswere encouraged to check their rear view mirrorfrequently. Participants were presented with the alarmsignal during familiarization. Upon hearing an alarm,participants were told to scan the rear view display tolocate the approaching vehicle, and then to respondappropriately (swerve left or right to avoid theapproaching vehicle).

After familiarization, the experimenter then repeatedthe experimental task instructions. Each participant wastold the percentage of true alarms to expect during theexperiment (50%, 75% or 100%). After the participantindicated that he or she understood the instructions, theexperimental task began.

The experimental task consisted of a 20-min session ofdriving and reacting to collision warning alarms. Theroute began at the on-ramp to I-565 from WashingtonStreet in downtown Huntsville, AL. The driver traveledwest from that point to the intersection of I-565 and I-65near Decatur, AL. Drivers were encouraged to maintaina maximum speed of 112.65 kph (70 mph). Whiledriving, the participants were presented alarms accord-ing to a predetermined random schedule, so that alarmonset could not be predicted from time passage orgeographic location. True alarms signaled the approachof a car from the right rear, rear, or left rear of theparticipant’s automobile. False alarms sounded withoutan approaching car being present.

Personnel at AMCOM programmed a semi-auto-mated computer interface to assist the experimenter withdata collection. The computer interface was responsiblefor generating alarm signals, starting and ending thesimulation, and storing alarm response frequency andappropriateness data recorded by the experimenter aftereach alarm signal. As a supplementary data collectiontool, an 8-mm video camera was placed just outside of

the simulator’s driver’s-side door to record the alarmsand each participant’s vocalizations and movements.

After participants had completed the 20-min drivingtask, they rested for 10 min before starting the secondexperiment.

2.2. Results

After collating and entering the task performancedata, we analyzed the videotaped performances todetermine swerving reaction times to alarms. Todetermine if console alarms affected driver behavior,we calculated one-way analyses of variance (ANOVAs)for alarm reaction frequency, alarm reaction time,appropriateness of alarm-instigated driving reactions,appropriateness of alarm reactions, and collision fre-quency. The results of those ANOVAs are reportedbelow.

As shown in Fig. 3, participants responded morefrequently when alarms reliably indicated the presenceof an approaching car in the rear view mirror display,F(2,67)=20.603, po:001: Tukey post-hoc comparisonsindicated that average alarm response frequencies weredifferent across all three reliability levels (po0:05).

There was no effect of alarm reliability on swervingreaction time (p > 0:05). The average time taken byparticipants to swerve following 50% reliable consolealarms was 5.82 s (SD=0.54), the average time taken toswerve following 75% reliable console alarms was 5.91 s(SD=0.83), and the average time taken to swervefollowing 100% reliable alarms was 6.01 s (SD=0.67).

Alarm reliability did have an effect on whetherparticipants swerved correctly to avoid a collisionfollowing the alarms, F(2,67)=12.216, po0:001 (seeFig. 4). Tukey post-hoc tests indicated that both the75% and the 50% groups swerved correctly less oftenthan the 100% group; however, there was no significantdifference between the reactions of the 50% and 75%groups.

ARTICLE IN PRESS

Fig. 4. Appropriateness of driving reactions as a function of console

alarm reliability.

Fig. 5. Console alarm reaction collision rates as a function of signal

reliability.

J.P. Bliss, S.A. Acton / Applied Ergonomics 34 (2003) 499–509504

The ANOVA for alarm reaction appropriateness(whether drivers properly reacted to true alarms andignored false alarms) was not significant, p > 0:05:

Participants in different reliability groups differedwith regard to collision rate, F(2,67)=5.68, p ¼ 0:005(see Fig. 5). Tukey post-hoc tests indicated that the 50%group collided with fewer cars than the 100% group(po0:05).

2.3. Discussion

We were not surprised to see lower alarm reactionfrequency and driving reaction appropriateness forlower reliability alarms. This supports our expectationthat the cry-wolf phenomenon would occur, even in acomparatively realistic task setting. It is encouragingthat dependent measures such as response frequency aresensitive indicators of alarm mistrust regardless of theveridicality of task structure. This shows the applic-

ability of Muir’s machine trust theory to automotivetask situations.

Previously, researchers have shown that alarm re-sponse rate and reaction appropriateness are sensitive tofluctuations in alarm system reliability (Bliss, 1993).However, primary task performance has rarely sufferedas participants would typically ignore unreliable alarmsand continue to perform the primary task efficiently.However, in the current research the primary task wasintricately tied to the alarms, and so it is reasonable thatappropriateness of the driving task (swerving response)suffered. One explanation for the lack of effect for alarmreaction appropriateness is the fact that drivers reliedheavily on the rear view mirror display when reacting.Therefore, participants in all groups were able toconfirm the appropriateness of their reactions prior tomaking them.

The frequency of alarm reactions in this experimentresembled the frequency matching pattern demonstratedin previous research (Bliss, 1993); participants mirroredtheir reaction rates to the reliability of the alarm signals.However, in the past, participants generally over-matched their response rates (Bliss, 1993) whereas inthe current experiment reaction rates were lower thanthe reliability of the signals. This may suggest that theparticipants experienced significant cognitive load whiledriving the simulator, and so had less spare cognitivecapacity to react to alarms. Additionally, participantsmay have devoted more attention to the primary task,because it was realistic and somewhat novel.

One interesting finding concerns the observed colli-sion rate for the groups. Although we made nohypothesis concerning collision rate, logic might suggestthat a higher reliability alarm system would result infewer collisions. However, the 50% reliability groupcollided with significantly fewer approaching vehiclesthan the 100% group. Although all participants checkedthe rear view mirror display before swerving, membersof the 50% group likely did so more deliberately, and sowere more successful at avoiding collisions. In contrast,members of the 100% group appeared to fall into a sortof routine, swerving more carelessly as time progressed.

In the next experiment, we presented participants withspatially generated signals, to determine whether addi-tional signal information would mitigate the alarmmistrust demonstrated in the first experiment.

3. Experiment Two

In recent years, researchers have demonstrated manyways to improve alarm compliance, including increasingalarm urgency (Haas et al., 1995; Hellier et al., 2002)advertising high reliability rates to operators (Bliss et al.,1995), and requiring operators to respond verbally(Bliss, 1997). One particularly successful method studied

ARTICLE IN PRESSJ.P. Bliss, S.A. Acton / Applied Ergonomics 34 (2003) 499–509 505

was to relay additional information about the alarmsystem, so that operators could refer to supplementalevidence of alarm validity or alarm system reliabilitybefore reacting (Bliss et al., 1996; Selcon et al., 1995). Byincreasing operator situation awareness in this fashion,reaction appropriateness increased and response fre-quency was optimized.

In automotive environments, one way to relayadditional information might be to design alarms thatspatially mirror the dangers they signal. Previousresearch in the aviation domain suggests that perceivedreliability might be greater for spatial alarms than foralarms generated from a central console because spatialalarms provide additional information about potentialcollisions (Lee and Patterson, 1993). However, researchis needed to determine the value of spatial alarms fordrivers (Parasuraman et al., 1997).

In the aviation domain, Lee and Patterson (1993)found that fighter pilots preferred spatial signals forwingman communication and threat information. Rud-mann and Strybel (1999) showed that using centered andspatial auditory cues for a search task in a virtualenvironment led to quicker response times over auditorycues generated in front of participants.

Some automobile designers have consideredpresenting auditory alarm signals spatially to increaselocalization ability. The expectation is that driverreactions may improve because the threshold at whichsignals are discriminated is lowered (Bronkhorst et al.,1996).

Spatially generated auditory signals may indeed bepromising for automotive environments, because driverswould not have to look at a head-up display, console, orrear view mirror to know from which direction anothercar is approaching. Instead, information about the car’slocation could be embedded into the spatial auditorysignals. Research suggests that drivers may also perceivespatial signals as more reliable than centralized consolesignals, because of their closer correspondence to theevent signaled (Bliss et al., 1996; Bronkhorst et al.,1996).

We intended this experiment to partially replicate thefirst experiment, while providing additional data aboutspatial alarm reactions. A direct comparison of spatialand console alarm reactions is not possible because allparticipants experienced central alarms prior to spatialalarms. However, we were interested to know howparticipants would react to the introduction of spatialalarms, after experiencing console alarms. Few research-ers have considered whether alarm respondents adaptwell to changes in technology. However, such changesare commonplace in many environments, such as theautomobile industry: With the numerous technologies inplace controlling CAWS function, people who drive avariety of cars may be required to adjust to a variety ofsignal types and origins.

Because of the potential interaction between spatialsignal source and perceived reliability, it is important toexamine alarm mistrust in conditions where signals aregenerated spatially. In Experiment Two, we replicatedthe first experiment, using spatially representativesignals. We expected that alarm reliability would impactalarm reactions and driving performance as it did in thefirst experiment. However, we expected better overallperformance because of our use of spatial signals, andbecause the experimental participants had alreadyreceived exposure to console based signals in Experi-ment One.

3.1. Method

3.1.1. Experimental design

The approach was the same as in Experiment One.Alarm reliability was manipulated between groups(50%, 75% or 100% true alarms), and we measuredalarm reaction frequency (to both true and false alarms),time, and appropriateness, driving appropriateness, andcollision rate. Participants experienced the same level offalse alarms in both experiments.

3.1.2. Participants

The same participants were used for this experimentas were used in Experiment One. After participantscompleted the first experiment, they rested for 10 min,and then began the second experiment.

3.1.3. Materials

All materials used were the same as in the firstexperiment, except for the origin of alarm signals andthe direction of travel (participants drove east instead ofwest). In this experiment, the 12 alarms were spatiallyrepresentative of the direction of the approaching car.Alarm signals sounded from the left rear, rear, or rightrear of the automotive cabin. A preliminary studyindicated that participants could effectively localizealarm sounds within the cabin. All other aspects of thealarms remained the same as in Experiment One.

3.1.4. Procedure

After completing the simulated driving task inExperiment One, participants rested for 10 min. Theythen began to drive east from the intersection ofInterstate 65 and Interstate 565 to Huntsville. Thisroute brought them back to their origination point atthe beginning of Experiment One.

Along the way, 12 alarms were generated in a mannersimilar to the alarms during the first experiment:reliability of the alarm system was the same as it wasin the first experiment, and the validity of individualalarms (true or false) was randomly determined. Asbefore, participants were told to react to the alarms byignoring them if they were false and swerving in the

ARTICLE IN PRESS

Fig. 8. Spatial alarm reaction collision rates as a function of signal

reliability.

J.P. Bliss, S.A. Acton / Applied Ergonomics 34 (2003) 499–509506

proper direction (away from the approaching car) if theywere true. As before, participants were instructed tomaintain a speed of 112.65 kph (70 mph).

After they completed the simulated scenario, theystopped the vehicle, exited, and completed the opinionquestionnaire. After ensuring that participants were notsuffering from simulator sickness, we debriefed anddismissed them.

3.2. Results

To determine if the spatial alarms affected driverbehavior, we calculated one-way analyses of variance(ANOVAs) using the same variables as in ExperimentOne. The results of those ANOVAs are reported below.

As shown in Fig. 6, there was a significant effect ofspatial alarm reliability on alarm response frequency,F(2,67)=21.448, po0:001: Tukey post-hoc comparisonsindicated that each increase in reliability was associatedwith a significant increase in alarm responses (po0:05).

As in the first experiment, there was no statisticallysignificant effect of alarm reliability on swervingreaction times (p > 0:05). Participants averaged 5.73 s(SD=0.79) to swerve following 50% reliable spatialalarms; 5.83 s (SD=0.69) to swerve following 75%

Fig. 6. Frequency of spatial alarm reactions as a function of signal

reliability.

Fig. 7. Appropriateness of driving reactions as a function of spatial

alarm reliability.

reliable spatial alarms; and 5.95 s (SD=0.53) to swervefollowing 100% reliable spatial alarms.

Alarm reliability did influence the way that partici-pants drove following alarms, F(2,67)=16.089, po0:001(see Fig. 7). Tukey post-hoc tests indicated that the 75%group and the 50% group each swerved in a significantlyless appropriate manner than the 100% group; however,there was no significant difference between the 50% and75% groups.

Alarm reliability did not significantly affect alarmreaction appropriateness, p > 0:05: There was, however,a statistically significant effect of reliability on collisions,F(2,67)=6.694, p ¼ 0:002 (see Fig. 8). Tukey post-hoctests indicated that the 50% group collided with fewercars than the 75% and 100% groups (po0:05); however,there was no difference between the 75% and 100%groups.

3.3. Discussion

As in the first experiment, we observed performancedeficits in the 50% and 75% reliability groups.Participants in those groups responded to alarms lessfrequently, and performed the driving task less appro-priately following alarms. Yet, as before, participants inthe 50% group collided with the fewest approachingcars, compared to the 75% or 100% groups.

A rigorous comparison of console and spatial alarmsacross the two experiments is not advisable because ofalarm origin and task experiences were confounded.However, an informal consideration of the data showsthat spatial alarm reliability affected performancemeasures the same way that console alarm reliabilityhad. An examination of performance means suggeststhat participants exhibited generally better perfor-mances when the alarm signals were spatially generatedthan when they were generated from the center console.The lone exception to this is driving appropriateness.Participants made less appropriate driving reactionsfollowing spatial alarms than they had following console

ARTICLE IN PRESSJ.P. Bliss, S.A. Acton / Applied Ergonomics 34 (2003) 499–509 507

alarms. While this seems contradictory to the results forthe other variables, an examination of the videotapedperformances revealed some indecision on the part ofparticipants. In some cases, participants’ recorded initialreactions were inappropriate, but they ultimatelyavoided the oncoming car after correcting the initialreaction.

In both the console and spatial experiments, weexpected that participants would rely upon the rear-viewmirror to detect the position of the approaching car.Indeed, an examination of mirror checking behaviorconfirmed this. Similar to Experiment One, as Experi-ment Two progressed some members of the 75% and100% groups seemed to fall into a comfortable routineof swerving slightly or making a ‘‘token’’ reaction. Incontrast, many members of the 50% group questionedevery alarm and paid close attention to the rear viewmirror, perhaps because they were not biased to believethat the alarms would be true. Therefore, they weremore deliberate about swerving, and more effective atavoiding collisions.

An additional factor that influenced collision rateswas that true alarms appeared to create a startle effect.Upon hearing the alarms, some drivers would swervecorrectly, but then swerve back again. As a result, eventhough their initial swerving response was technicallyappropriate, they ultimately collided with the approach-ing car. The possibility of such reactions has beendiscussed before by Baber (1994).

An interesting aspect of performance in ExperimentTwo concerns the driving appropriateness data. As isevident from Fig. 7, drivers generally made poor drivingreactions following spatial alarms. There are two likelyexplanations for this. The first is that scanning the rearview mirror display made it difficult for participants tolocalize the spatial alarm signals. The second explana-tion is that during the second experiment, drivers weremore fatigued and frustrated with the driving and alarmreaction tasks. As a result, they may have reacted lessappropriately in general.

4. General discussion

Past research has determined that response frequencyis a stable indicator of alarm mistrust (Bliss, 1997). Forthe current research, we increased the realism of theexperimental paradigm by requiring drivers to avoidother traffic were realistic and task-relevant. It isencouraging that dependent measures such as responsefrequency are sensitive indicators of alarm mistrustregardless of the veridicality of task structure. Asmentioned previously, this lends support to existingtheories of machine trust (Muir, 1989), because aspectsof the alarm system such as predictability and depend-

ability clearly impact performance, regardless of taskrealism.

Perhaps one of the most important measures in thisresearch was collision rate. Because the alarm systemswere designed to prevent collisions, it is of concern thata 50% alarm system should result in fewer collisions. Itis likely that such low reliability levels led participants tofollow a wise course of action, confirming the existenceand nature of any imminent threats before reacting. Yet,the urgency of the alarms seemed to lead members of the100% group to be distracted, so that while their drivingreactions began correctly, they often overcompensatedor reversed their action. Dingus et al. (1997) havediscussed the possibility that improperly designedcollision avoidance alarms may actually lead to morecollisions. Our findings may lend support to thatnotion.

Considering the results of the present experiment,there are several recommendations that may be made.First, it is important to replicate these findings in avariety of other traffic conditions, as the driving task inthis experiment was admittedly fabricated and simplis-tic. If future replications reveal similar results, alarmdesigners would be well advised to consider interven-tions to counter the potential for false alarms.

In previous alarm mistrust research, researchers havefound that making participants aware of high alarmsystem reliability rates improved responsiveness (Blisset al., 1995). Such a strategy may be applicable forautomotive collision avoidance alarm systems as well.Conversely, if drivers are made aware of low reliabilityrates, their responsiveness may decline.

Another intervention shown by Bliss (1993) todecrease response times was to heighten alarm signalurgency. However, in the current experiment partici-pants reacted poorly to alarm urgency, becomingdistracted and confused. Urgent, reliable alarms evokedresponses that, while appropriate, led to a greaternumber of collisions. For this reason, advocating quick,reflexive reactions to automated alarm systems may notbe a wise course of action. Furthermore, the negativeimpact of such reflexive behavior may well be com-pounded in situations where task workload is heigh-tened, or where there are a number of collateral alarmsystems.

As noted in the Section 1, introduction of spatialalarm systems has been considered as a way to improvealarm responsiveness. However, the effectiveness of suchalarms may be limited. Even when spatial alarms wereused, participants still showed clear signs of alarmmistrust. Furthermore, as indicated previously, alimitation of this research is that there is no way toseparate the effects of alarm origin and task training.Because of this, future researchers are encouraged tomanipulate alarm reliability and origin together, todetermine the robustness of our findings.

ARTICLE IN PRESSJ.P. Bliss, S.A. Acton / Applied Ergonomics 34 (2003) 499–509508

The current research provides an interesting view ofCAWS. Past research has shown the importance ofredundant information sources in situations wherealarm reliability is suspect (Bliss et al., 1996). In thecurrent research, it was clear that participants relied onthe rear view mirror display, even when spatial locationof the threats was coded within the alarm signal.Designers of alarm systems may do well to heed thisinformation. It seems that any degradation of alarmreliability will likely cause drivers to verify alarmvalidity through other information sources. For thisreason, the implementation of CAWS (particularlythose that feature spatially generated signals) shouldbe considered with caution.

As this project was the first step toward a programof research concerning CAWS, there are manyavenues to pursue. The driving environment createdfor this research was intentionally simplistic, toallow rigorous manipulation of reliability and signalsource location and to facilitate the measurement ofdriving performance. Following these results, it isimportant that research be conducted that includes thepresence of other traffic (besides cars overtaking theexperimental vehicle). It is also important that investi-gators examine alarm responsiveness using trafficscenarios that may yield high numbers of false alarms,such as parking lots, two lane roads, and interstateinterchanges.

In the current experiments, we took care toensure that participants could definitely tell whetheralarms were true or false, by presenting no carswhen false alarms sounded. Other researchers havechosen to present less threatening cars when falsealarms sound. This may make the differencebetween true and false alarm situations less pro-nounced (see Gupta et al., 2002). Both approacheshave advantages (see NHTSA, 2000). Situationswhere true and false alarm situations are less distin-guishable may be more externally valid. However, wepresented clearly distinguishable true and false alarmsituations to avoid any ambiguity concerning alarmvalidity perception.

The current collision avoidance warning system was apassive one, generating alarms when other cars en-croached upon the experimental vehicle. Such situationsare important to consider, because of their prevalence.However, many prototype CAWS operate according toan active algorithm, where alarms are generated afterthe driver activates the turn signal or begins to changelanes. Such alarm systems must also be evaluated forfeasibility and reliability. Drivers may be more alert insuch situations because they instigated the collisionsituation. Such situations may also generalize morereadily to collision situations involving objects otherthan automobiles (i.e., car–pedestrian accidents androadway departures).

Acknowledgements

This project, funded by the University TransportationCenter of Alabama (UTCA Grant No. 03UTCA-003)could not have been completed without the assistance ofmany people and entities. We wish to thank Frank Craigand Stephen Tanner from The Boeing Company, MarkBlasingame and Chris Daniel from the Army-NASAVirtual Innovations Laboratory, Robert Lock ofChrysler (Huntsville Electronics Division), Laurie Fra-ser, Greg Tackett, and Tim McKelvy from the US ArmyAviation Missile Command—Distributed SimulationCenter, Greg Lee and Skip Clay from Nichols Research,and Doug Barclay, Christy Bates, and Jimmy Moorefrom Computer Sciences Corporation.

References

Araki, H., Yamada, K., Hiroshima, Y., Toshio, I., 1996. Development

of rear-end collision avoidance system. Proceedings of the 1996

IEEE Intelligent Vehicle Symposium, Tokyo, Japan, pp. 652–657.

Baber, C., 1994. Psychological aspects of in-car warning devices. In:

Stanton, N. (Ed.), Human Factors in Alarm Design. Taylor &

Francis, London.

Barber, B., 1993. Logic and the Limits of Trust. Rutgers University

Press, New Brunswick, NJ.

Bitan, Y., Meyer, J., Shinar, D., Zmora, E., 2000. Staff actions and

alarms in a neonatal intensive care unit. Proceedings of the IEA

2000/HFES 2000 Congress, San Diego, CA, July 29–August 4.

Bliss, J.P., 1993. The cry-wolf phenomenon and its effect on alarm

response. Unpublished Doctoral Dissertation, University of

Central Florida, Orlando.

Bliss, J.P., 1997. Alarm reaction patterns by pilots as a function of

reaction modality. Int. J. Aviat. Psychol. 7 (1), 1–14.

Bliss, J.P., Dunn, M.C., 2000. The behavioral implications of alarm

mistrust as a function of task workload. Ergonomics 43 (9),

1283–1300.

Bliss, J.P., Dunn, M., Fuller, B.S., 1995. Reversal of the cry-wolf

effect: an investigation of two methods to increase alarm response

rates. Perceptual Motor Skills 80, 1231–1242.

Bliss, J.P., Freeland, M., Millard, J., 1999. Alarm related incidents in

aviation: a survey of the aviation safety reporting system database.

Proceedings of the 43rd Annual Meeting of the Human Factors

and Ergonomics Society, Houston, TX, September 27–October 1.

Bliss, J.P., Jeans, S.M., Prioux, H.J., 1996. Dual-task performance as a

function of individual alarm validity and alarm system reliability

information. Proceedings of the Human Factors and Ergonomics

Society 40th Annual Meeting, Santa Monica, CA, October 2–8.

Human Factors and Ergonomics Society, Philadelphia, PA,

pp. 1237–1241.

Bliss, J.P., Kilpatrick, F., 2000. The influence of verbal content on

alarm mistrust. Proceedings of the 2000 Human Factors and

Ergonomics Society Annual Meeting, San Diego, CA, July 30–

August 4.

Bronkhorst, A.W., Veltman, J.A., van Breda, L., 1996. Application of

a three-dimensional auditory display in a flight task. Human

Factors 38 (1), 23–33.

Chen, C., Quinn, R.D., Ritzmann, R.E., 1997. A crash avoidance

system based on the cockroach escape response circuit. Proceedings

on the 1997 IEEE International Conference on Robotics and

Automation, Albuquerque, USA, pp. 2007–2012.

ARTICLE IN PRESSJ.P. Bliss, S.A. Acton / Applied Ergonomics 34 (2003) 499–509 509

Cohen, J., 1988. Statistical Power Analysis for the Behavioral Sciences,

2nd Edition.. Erlbaum, Hillsdale, NJ.

Dingus, T.A., McGehee, D.V., Manakkal, N., Jahns, S.K., Carney, C.,

Hankey, J.M., 1997. Human factors field evaluation of automotive

headway maintenance/collision avoidance devices. Human Factors

39 (2), 216–229.

Farber, E., Paley, M., 1993. Using freeway traffic data to estimate the

effectiveness of rear end collision countermeasures. Proceedings of

the Third Annual IVHS America Meeting, Washington, DC, April.

Gupta, N., Bisantz, A.M., Singh, T., 2002. The effect of adverse

condition warning system characteristics on driver performance: an

investigation of alarm signal type and threshold level. Behav. Inf.

Technol. 21 (4), 235–248.

Haas, E.C., Casali, J.G., 1995. Perceived urgency and response time to

multi-tone and frequency-modulated warning signals in broadband

noise. Ergonomics 38 (11), 2281–2299.

Haas, E.C., Edworthy, J., 1996. Designing urgency into auditory

warnings using pitch, speed, and loudness. Comput. Control Eng.

J. 7 (4), 193–198.

Hays, R.T., Singer, M.J., 1989. Simulator Fidelity in Training System

Design. Springer, New York.

Hellier, E., Edworthy, J., Weedon, B., Walters, K., Adams, A., 2002.

The perceived urgency of speech warnings: semantics vs. acoustics.

Human Factors 44 (1), 1–17.

Hirst, S., Graham, R., 1997. The format and presentation of collision

warnings. In: Ian, N. (Ed.), Ergonomics and Safety of Intelligent

Driver Interfaces. Lawrence Erlbaum Associates, Mahwah, NJ.

Jansson, J., Johansson, J., Gustafsson, F., 2002. Decision making for

collision avoidance systems. SAE Paper No. 2002-01-0403, Society

of Automotive Engineers.Washington, DC.

Lee, J., Moray, N., 1992. Trust, control strategies, and allocation of

function in human–machine systems. Ergonomics 35 (10), 1243–1270.

Lee, M.D., Patterson, R.W., 1993. The application of three-dimen-

sional audio displays to aircraft cockpits: user requirements,

technology assessment, and operational recommendations. Pro-

ceedings of the Beyond Speech/Virtual Reality/Teleoperation 1993.

SIG—Advanced Applications, New York, NY, November.

Lilliboe, M.L., 1963. Final Report: evaluation of Astropower, Inc.

auditory information display installed in the VA-3B airplane.

Technical Report ST 31-22R-63. US Naval Air Station, Naval Test

Center, Patuxent River, MD.

MacKinnon, D.P., Bryan, A.D., Barr, A., 1993. Four studies on the

effects of multiple warnings: The false alarm effect and over-

warning. Technical Report, Contract # AA8547. Project ABLE,

Arizona State University.

Mazzae, E.N., Garrott, W.R., Flick, M., 1995. Human factors

evaluation of existing side collision avoidance system driver

interfaces. SAE Paper No. 952659. National Highway Traffic

Safety Administration, Washington, DC.

Muir, B.M., 1989. Operators’ trust in and percentage of time spent

using the automatic controllers in a supervisory process control

task. Doctoral Thesis, University of Toronto.

National Highway Traffic Safety Administration, 1997. Report to

congress on the national highway traffic safety administration ITS

program. ITS Electronic Document Library Paper No. 2683. US

Department of Transportation, Washington, DC.

National Highway Traffic Safety Administration, 2000. Automotive

Collision Avoidance Systems (ACAS) Program. Final Report

No. DOT HS 809 080. US Department of Transportation

Washington, DC.

National Highway Traffic Safety Administration, 2002. Automotive

collision avoidance system field operational test. Final Report

No. DOT HS 809 462. US Department of Transportation,

Washington, DC.

Parasuraman, R., Hancock, P.A., Olofinboba, O., 1997. Alarm

effectiveness in driver-centred collision-warning systems. Ergo-

nomics 40 (3), 390–399.

Parasuraman, R., Riley, V., 1997. Humans and automation: use,

misuse, disuse, abuse. Human Factors 39, 230–253.

Pritchett, A.R., 2001. Reviewing the role of cockpit alerting systems.

Human Factors Aerospace Safety 1 (1), 5–38.

Rempel, J.K., Holmes, J.G., Zanna, M.P., 1985. Trust in close

relationships. J. Personality Social Psychol. 49, 95–112.

Rudmann, D.S., Strybel, T.Z., 1999. Auditory spatial facilitation of

visual search performance : effects of cue precision and distractor

density. Human Factors 41 (1), 146–160.

Selcon, S.J., Taylor, R.M., McKenna, F.P., 1995. Integrating multiple

information sources: using redundancy in the design of warnings.

Ergonomics 38 (11), 2362–2370.

Shapiro, N., Executive Producer, 1994. NBC Dateline. National

Broadcasting Corporation, New York.

Sorkin, R.D., 1988. Why are people turning off our alarms? J. Acoust.

Soc. Am. 84 (3), 1107–1108.

Tijerina, L., Garrott, W.R., 1997. A reliability theory approach to

estimate the potential effectiveness of a crash avoidance system to

support lane change decisions. Proceedings of the SAE Interna-

tional Congress and Exposition, Detroit, MI.

Tijernia, L., Hetrick, S., 1997. Analytical evaluation of warning onset

rules for lane change crash avoidance systems. Proceedings of the

Human Factors and Ergonomics Society 40th Annual Meeting,

Philadelphia, USA, pp. 949–953.

Tilin, A., 2002. You are about to crash. Wired Magazine, Issue 10.04.

Online: http://www.wired.com/wired/archive/10.04/.

Xiao, Y., Seagull, F.J., 1999. An analysis of problems with

auditory alarms: defining the roles of alarms in process moni-

toring tasks. Proceedings of the 43rd Annual Meeting of

the Human Factors and Ergonomics Society, Houston, USA,

pp. 256–260.