© 2009 The Psychonomic Society, Inc. 118

Air traffic control (ATC) simulations are frequently used for both applied and basic research. There is a growing need for ATC simulations, to identify factors that influence the workload and performance of air traffic controllers (Athenes, Averty, Puechmorel, Delahaye, & Collet, 2002; Lamoureux, 1999) and to build theories of the representa-tions and processes that underlie performance on specific control tasks (Gronlund, Ohrt, Dougherty, Perry, & Man-ning, 1998; Rantanen & Nunes, 2005). In addition, ATC simulations are frequently used to address more basic is-sues of human cognition, such as the associative learning mechanisms involved in relative judgment (Loft, Neal, & Humphreys, 2007), the processes that underlie memory in the performance of intended actions (Stone, Dismukes, & Remington, 2001), the effects of time pressure on process-ing load (Hendy, Liao, & Milgram, 1997), and individual differences in complex skill acquisition (Ackerman, 1992). Consequently, ATC simulations are effective tools for eval-uating the generalizability of broader theories about basic cognitive processes and capacities, thus explaining human performance more generally. In this article, we describe a new ATC simulation package called ATC-labAdvanced that can be used for both applied and basic research. In doing so, we highlight the improvements it offers over currently available ATC simulators.

Existing ATC simulators have typically been developed so as to have the level of realism and experimental control

required to investigate specific research questions. Real-ism refers to the extent to which experiences encountered in the simulation occur in the field of interest (DiFonzo, Hantula, & Bordia, 1998; Ehret, Gray, & Kirschenbaum, 2000). Experimental control refers to the degree to which a simulation can provide control over variables and thus support the conclusion that the effects obtained are due to experimental manipulations (Boring, 1954; Brehmer & Dorner, 1993). To maximize efficiency, existing simula-tors have typically been designed to compromise between the extent to which they can mimic field experience (real-ism) and the experimental control that they can provide. High-fidelity ATC simulators typically have high realism but lack experimental control. Medium-/low-fidelity sim-ulators can provide this control but often lack realism.

This trade-off between realism and experimental con-trol presents a problem when both are required. For ex-ample, many research groups are developing theories and models designed to predict controller performance in field settings (for a review, see Loft, Sanderson, Neal, & Mooij, 2007). For this type of research, it is crucial to use simu-lations that are representative of the environmental con-text in which experts make decisions (Brunswick, 1956; Simon, 1956). At the same time, experimental control is required in order to isolate the effects of independent variables on specific ATC control tasks. In contrast, the purpose of more basic research may be to test a specific

ATC-labAdvanced: An air traffic control simulator with realism and control

SELINA FOTHERGILLUniversity of Queensland, Brisbane, Queensland, Australia

SHAYNE LOFTUniversity of Western Australia, Perth, Western Australia, Australia

AND

ANDREW NEALUniversity of Queensland, Brisbane, Queensland, Australia

ATC-labAdvanced is a new, publicly available air traffic control (ATC) simulation package that provides both realism and experimental control. ATC-labAdvanced simulations are realistic to the extent that the display features (including aircraft performance) and the manner in which participants interact with the system are similar to those used in an operational environment. Experimental control allows researchers to standardize air traffic scenarios, control levels of realism, and isolate specific ATC tasks. Importantly, ATC-labAdvanced also provides the program-ming control required to cost effectively adapt simulations to serve different research purposes without the need for technical support. In addition, ATC-labAdvanced includes a package for training participants and mathematical spreadsheets for designing air traffic events. Preliminary studies have demonstrated that ATC-labAdvanced is a flex-ible tool for applied and basic research.

Behavior Research Methods2009, 41 (1), 118-127doi:10.3758/BRM.41.1.118

S. Fothergill, selina@psy.uq.edu.au

ATC-LABADVANCED 119

labAdvanced display resembled the ATC operational envi-ronment in as many ways as possible (Schiff, Arnone, & Cross, 1994). To achieve this, the ATC-labAdvanced display was based on the Australian Air Traffic Management Sys-tem and was developed in close collaboration with subject matter experts.

Figure 1 illustrates a generic example of the display used in a high-fidelity ATC-labAdvanced simulation. The sector that the participant controls (the active sector) is made up of a series of flight paths, waypoints, and airports presented against a light gray background. The surround-ing darker gray background represents adjacent and ap-proach sectors (sectors that contain airports). Small green circles symbolize aircraft track symbols, and each aircraft has a data block label that displays the call sign, aircraft type, ground speed, current flight level, and cleared flight level. These aircraft track symbols and data blocks can be fully customized. ATC-labAdvanced uses nautical miles for distance, knots for ground speed, and feet for altitude.

Every 5 sec, each aircraft’s position and data block label information is updated. Aircraft enter the active sector on inbound flight paths from adjacent sectors or take off from airports in approach sectors. They then proceed as denoted in their flight plan through the series of waypoints and either land at an airport or exit to adjacent sectors on outbound flight paths. Aircraft that cruise at flight levels below or above the sector flight level boundary of the active sec-tor (over flights) can also be simulated. Importantly, ATC-labAdvanced simulates aircraft performance data (e.g., climb and descent rate, speed rate) accurately for commercial jets, turbo propeller aircraft, and military aircraft. As a re-sult, aircraft can transit through sectors in a manner similar to that for an ATC operational environment.

The notification system used to denote transitions in aircraft states can be closely based on ATC operational environments. That is, the attributes (e.g., colors, flash-ing) of aircraft track symbols and data block labels can be set to represent different phases of flight, which change dynamically as aircraft move though sectors. For exam-ple, an aircraft approaching an active sector from an ad-jacent sector may be set to turn from black to blue when it reaches a certain distance (e.g., 10 nm) from the active sector. As the aircraft travels closer to the active sector, it may be set to flash orange until the controller officially “accepts” the aircraft, using a specific sequence of ac-tions, at which point it would turn green to denote that it is under the jurisdiction of that controller. When the air-craft is handed off to the adjacent sector or approaches the airport, it would turn black to indicate that it is no longer under the jurisdiction of that controller.

Response-system realism. The second important re-quirement for achieving realism was to ensure that partici-pants performing control tasks would be able to interact with the ATC-labAdvanced system as similarly as possible to how controllers would interact with ATC systems in the field (Schiff et al.,1994). ATC-labAdvanced can be custom-ized to provide simulations of the major control tasks pre-viously identified in cognitive task analyses of ATC (Cox, 1994; Rodgers & Drechsler, 1993). These control tasks

theoretical issue that is prevalent in a range of applied set-tings in which individuals monitor dynamic multi-item displays (e.g., military command, radar system operators). In these circumstances, it may be desirable to have low correspondence (cf. Gray, 2002) between the simulation and the operational environment, so that the research can be generalized to other systems (Berkowitz & Donner-stein, 1982; Mook, 1983). In other circumstances, ATC simulations may be conducted to assess the effectiveness of controller team performance or training programs, and increased experimental control would add little to improv-ing the outcomes of the research.

This highlights a need for an ATC simulation package in which realism and control can be systematically varied according to the research question(s) under investigation. In the present article, we present a new ATC simulator called ATC-labAdvanced that provides this. Importantly, ATC-labAdvanced also provides the programming control required for researchers to customize the exact levels of realism and control they require in their simulations. The aim of the present article is to introduce ATC-labAdvanced and indicate how it can be used for research. First, we will detail the features of ATC-labAdvanced that provide realism, experimental control, and programming control. These features will then be compared with those of existing simulators. We will then provide examples of applied and basic research programs that have used ATC-lab Advanced. Next, we will outline the training package available to fa-miliarize participants with ATC-labAdvanced simulations. Finally, data logging features and system requirements will be provided.

Realism in ATC-labAdvanced

The primary duties for air traffic controllers are to en-force separation standards between aircraft and ensure that aircraft reach their destinations in an orderly and expedi-tious manner. One of the more common separation stan-dards set by the International Civil Aviation Organization (ICAO) is that aircraft are required to maintain either a 1,000-ft vertical separation or 5 nautical miles horizontal separation from all other aircraft. Consequently, a pair of aircraft is considered to be in conflict if they will, given their current speeds, flight levels (altitudes), and bearings, simultaneously violate vertical and horizontal separation standards in the future. Controllers are required to perform a range of control activities to ensure the safe and efficient flow of aircraft. When logical, practical, or logistical con-siderations constrain field experimentation or observation in an applied work context (DiFonzo et al., 1998; Gray, 2002), high-fidelity simulators can be used to simulate these tasks. Examples include the FAA Academy Training Simulator (Jones & Endsley, 2000), TRACON (Acker-man, 1992), the EUROCONTROL Simulation Capability and Platform for Experimentation, ATCoach (UFA Inc., n.d.), and FIRSTplus (Raytheon, 2005). ATC-labAdvanced

simulations can also be designed so that participants per-form tasks in a manner similar to field controllers.

Display realism. The first requirement for achieving realism was to ensure that the components of the ATC-

120 FOTHERGILL, LOFT, AND NEAL

level by a certain distance). Flight levels or speeds can be altered by clicking on the data block label where these val-ues are displayed and then choosing new values from drop-down menus. An example of how to change a flight level is illustrated in Figure 2. Similarly, heading changes can be chosen from drop-down menus. Headings of aircraft can be changed by selecting a predetermined heading func-tion on the keyboard, clicking on the aircraft, and dragging a line to a new destination point. Level requirements can be issued by pressing designated keys and entering into text boxes the distances by which aircraft are required to reach certain flight levels. Participants can accept aircraft by pressing designated keys and clicking on aircraft track

include accepting and handing off aircraft from adjacent sectors or airports, assigning boundary and cruise alti-tudes, monitoring air traffic to detect potential conflicts, resolving conflicts, and traffic sequencing.

The intervention methods participants use to modify air-craft trajectories in ATC-labAdvanced, the way participants accept and hand off aircraft, and how they use prediction tools were designed on the basis of structured interviews with controllers (Fothergill & Neal, 2005, 2006) and analy-ses of the ATC literature (Callantine, 2002; Späth & Ey-ferth, 2001). Examples of aircraft intervention methods include changing flight levels, speeds, or headings and as-signing flight-level requirements (e.g., reaching a flight

Figure 1. A generic example of a display used in an ATC-labAdvanced simulation. The screen shot displays one active and six (four adjacent, two approach) nonactive sectors, various route structures, and aircraft in their different phases of flight. All the aircraft have probe minute vectors to indicate their position in 1 min’s time; the route for SIA16 is displayed; a scale marker is available in the top left corner; and a bearing and range line has been attached to VOZ555. To resolve the potential conflict between VOZ555 and VOZ892, VOZ555 is being vectored away from its planned route. The clock is paused, and the mode display shows that a vector solution is being used.

Figure 2. Changing the cleared flight level of an aircraft. By clicking on the current cleared flight level, a new cleared flight level can be selected from the menu. The new level will be displayed in the aircraft’s label in the next 5-sec update.

ATC-LABADVANCED 121

generally have low display realism and low system re-sponse realism. For example, these simulators do not all have the capability to simulate changes in aircraft altitude, do not allow participants to interact with the ATC system in order to modify aircraft trajectory only in limited ways, and do not provide access to prediction tools.

This lack of realism does not present a problem when the intent of the research is to test a theoretical idea by mapping the functional relations between variables in a simulation, rather than generalizing to a specific domain. ATC-lab, for example, has been successfully used to de-velop general theories (Loft, Humphreys, & Neal, 2004; Loft, Neal, & Humphreys, 2007; Yeo & Neal, 2004) and computational models (Kwantes, Neal, & Loft, 2004) of the processes by which individuals make decisions about the movement of objects on radar displays. However, the lack of realism is problematic when one is building theo-ries and models of performance that apply directly to ATC operations (Kopardekar & Magyarits, 2003; Laudeman, Shelden, Branstrom, & Brasil, 1998), since a lack of real-ism poses a substantial threat to the external validity of results. For example, a researcher may be interested in examining the processes underlying ATC conflict detec-tion. Here, it would be essential that aircraft performance is accurately simulated so that aircraft transit through sec-tors as they would in the field. Controllers must also have access to their regular prediction tools, so they are able to make aircraft trajectory predictions in a way that is similar to how they would make them in the field.

There are many high-fidelity ATC simulators that can provide levels of display realism and response system re-alism that are similar to (or better than) those in ATC-labAdvanced. These include but are not limited to the FAA Academy Training Simulator (Jones & Endsley, 2000), the EUROCONTROL Simulation Capability and Platform for Experimentation, FIRSTplus (Raytheon, 2005), and the Total Airport and Airspace Modeler (TAAM) (Jeppesen, 2007). For example, TAAM runs real gate-to-gate traf-fic extracted from the Australian Air Traffic Management System, and FIRSTplus replicates all the features of mod-ern ATC radar situation displays and can even emulate future operational ATC display types. However, as will be discussed in the sections below, many of these high- fidelity simulators are not made freely available for re-search, nor do they necessarily provide experimental con-trol or programming control.

Experimental and Programming Control in ATC-labAdvanced

ATC-labAdvanced provides the experimental control re-quired to make definitive conclusions regarding the ef-fects of independent variables on dependent variables. Standardized air traffic scenarios can be presented that control extraneous variables and separate confound-ing variables. Programming control refers to the extent to which the researcher can control what is presented in simulations. ATC-labAdvanced provides high programming control over a wide range of task features. These task fea-tures include display realism, response system realism, trial presentation, and presentation of rating scales. This

symbols. Similar to ATC operational environments, hand-offs can be designed to occur automatically at a set dis-tance (e.g., 5 nm) beyond the sector boundary.

Prediction tools in ATC-labAdvanced include scale mark-ers, bearing and range lines, probe vectors, route displays, and history dots. These tools are regularly used by control-lers in the field. Scale markers are moved around the screen to measure distance. Bearing and range lines indicate dis-tance (in nautical miles), bearing (in degrees), and the time (in minutes) to a future waypoint or another aircraft. An example of how to use the bearing and range line function is illustrated in Figure 3. Route displays indicate the planned routes of aircraft, punctuated by the times at which the air-craft are predicted to reach waypoints, on the basis of their current nominal trajectory. History dots are displayed be-hind aircraft and represent the routes that aircraft have trav-eled. Probe vectors display the predicted position of aircraft (in a specified number of minutes) in the horizontal plane, on the basis of their current nominal trajectory.

Realism: Comparison with existing ATC simu-lators. A significant limitation of existing low- and medium- fidelity ATC simulators is that they lack display realism and response system realism. One prototypical example is our medium-/low-fidelity predecessor to ATC-labAdvanced, which we called ATC-lab (Loft, Hill, Neal, Humphreys, & Yeo, 2004). ATC-lab simulations are real-istic for participants to the extent that they involve and af-fect participants and to the extent that participants take the simulations seriously (DiFonzo et al., 1998). However, a major limitation of ATC-lab is that it simulates very selec-tive aspects of ATC. ATC-lab has low display realism be-cause it does not simulate features such as aircraft altitude, does not use real aircraft performance profiles, does not present adjacent/approach sectors, and does not provide any notification system for denoting aircraft transition states. In addition, ATC-lab has low response system real-ism because it simulates very few control tasks (conflict detection/resolution only), provides a very limited number of intervention methods for modifying aircraft trajectory (speed change only), and provides no prediction tools. The medium-fidelity ATC simulators used by Metzger and Parasuraman (2001) and Remington, Johnston, Ruthruff, Gold, and Romera (2000; also see Stone et al., 2001) also

Figure 3. The bearing and range line tool. This shows the dis-tance between the aircraft and the selected end point (in nautical miles), the bearing (in degrees), and the time that it would take the aircraft to reach the selected end point (in minutes) based on its indicated speed.

122 FOTHERGILL, LOFT, AND NEAL

researchers were required to wait for the developer (for up to 10 min) to generate starting x- and y-coordinates. Second, it did not allow the calculation of vertical dis-tance, which is essential for ATC-labAdvanced. With the mathematical spreadsheets, researchers enter the desired spatial and temporal characteristics of aircraft events, and hard-coded formulae contained in these spreadsheets pro-vide starting x- and y-coordinates for aircraft in the lateral plane. These spreadsheets are accompanied by a report documenting the underlying formulae. A scenario tester is also included in the ATC-labAdvanced simulation pack-age, which enables researchers to view (at a faster speed) the air traffic scenarios that are being developed.

Programming control over task features. Due to high levels of display realism and system response re-alism, ATC-labAdvanced simulates a much wider array of potential task features than do many existing simulators. Furthermore, a significant advantage of ATC-labAdvanced

is that the XML scripting language and code base archi-tecture provide the researcher with programming control over task features. First, researchers can control the real-ism of the display, which includes specifying the type of sector (e.g., approach, en route, tower), active and inactive sectors, route structures, position of waypoints, position of airports, and weather patterns. Trials can be constructed so that different sector maps with different traffic patterns can be presented within the same experiment. Researchers can control settings of the aircraft transition notification system, such as the specific color used to denote aircraft transitional states and the positions in sectors where air-craft automatically begin climbing or descending. Aircraft performance can also be modified. Second, researchers can control response system realism features, such as the type of prediction tools available to participants and the manner in which they are used, the type of methods that participants can use to modify aircraft trajectory, and the timing/content of instructions and questionnaire items (e.g., workload ratings, motivation ratings). Third, re-

programming control of ATC-labAdvanced is an important advance, since it allows simulations to be adapted quickly and cost effectively to serve different research purposes without the need for technical support.

Standardized air traffic scenarios. ATC-labAdvanced experimental scripts are used to specify aircraft events that occur during experimental trials. An example is illus-trated in Figure 4. These scripts are written using the Ex-tensible Markup Language (XML) Version 1.0. This is a free-to-use general purpose markup language, which can be used as a generic framework for storing any amount of text or any data whose structure can be represented as a tree. In contrast to the text files used in ATC-lab, XML scripts can be screened for errors before they are used in experiments. Aircraft details specified in the scripts in-clude call sign, type, minimum and maximum speed and flight level, current speed, current flight level, starting x- and y- coordinates, planned route, position (if any) for automatic start of climb or descent, and climb and descent rate. The values for aircraft call sign, aircraft type, ground speed, current flight level (altitude), and cleared flight level are derived from these scripts and are displayed on aircraft data blocks. When participants intervene during trials, these values are updated.

ATC-labAdvanced provides a set of mathematical spread-sheets to control the spatial (e.g., minimum separation, angle of convergence) and temporal (e.g., time to mini-mum separation) characteristics of aircraft events. These spreadsheets were developed to replace the script devel-oper provided in the ATC-lab simulation package (Loft, Hill, et al., 2004). The script developer represented a substantial improvement over existing medium- and low- fidelity simulators because it improved the degree to which air traffic scenarios could be standardized (see Loft, Hill, et al., 2004, for a detailed description), and eliminated the need for manual calculation or trial-by- error script-ing. However, the ATC-lab script developer had two major limitations. First, it was time consuming to use because

Figure 4. Specifications for an aircraft using XML scripting language. The aircraft’s type, call sign, starting altitude, starting velocity, starting coordinates, cleared flight level, and flight path are scripted.

ATC-LABADVANCED 123

In the next section of this article, we will provide ex-amples of applied and basic research programs in which ATC-labAdvanced simulations have been used. The degree of realism and control used in the three research programs were specifically tailored to the research question(s) under investigation, demonstrating the flexibility of ATC-labAdvanced as a tool for cognition research.

Illustrative Examples of ATC-labAdvanced Simulations

The three main studies that have used ATC-labAdvanced simulations to date are summarized in Table 1. Fothergill and Neal (2008) used ATC-labAdvanced to examine the ef-fect of workload on the selection of conflict resolution strategies. Participating controllers managed traffic in their sector and resolved potential conflicts as efficiently as possible. The purpose was to inform the development of a computational model that could simulate how con-trollers resolve conflicts in the field (Bolland, Fothergill, & Humphreys, 2007). The key finding was that control-lers were less likely to implement optimal conflict resolu-tion strategies under a high workload than under a low workload, but only in situations in which these strategies were more difficult to calculate (see Table 1). To obtain applicable results, the simulations were required to be representative of ATC, especially in terms of (1) aircraft performance, (2) sector structure, (3) aircraft transition notification, (4) controller intervention methods, and (5) prediction tool use. In order to systematically manipu-late independent variables, a high degree of experimental control was also required to vary configurations of air traffic. For example, high-workload scenarios contained configurations that produced more tasks (e.g., conflicts, acceptances and handoffs, aircraft sequencing) than did lower workload scenarios.

A recent issue raised in the experimental literature con-cerns how to capture expert performance across different task domains (Ericsson & Williams, 2007). Loft, Bol-land, and Humphreys (2007) recently developed a theory of expertise for ATC conflict detection. ATC-labAdvanced

simulations were then used to test a series of predictions from this theory that concerned the factors that affect the likelihood of controllers intervening to ensure separation between aircraft. In addition, data were used to test the development of a computational model that simulates how controllers detect conflicts in the field (Loft, Bol-land, & Humphreys, 2007). Thus, it was essential for ATC- labAdvanced to simulate the environmental context in which controllers make conflict detection decisions. In particular, it was critical that controllers have access to their regular prediction tools, such as range and bearing lines, in order to ensure that they acquire aircraft trajec-tory information in a realistic manner.

However, in contrast to Fothergill and Neal (2008), ATC-labAdvanced was programmed in such a way that controllers performed only conflict detection. By using the program-ming control available in ATC-labAdvanced to remove other ATC control tasks, Loft, Bolland, and Humphreys (2007) isolated conflict detection by eliminating visual search requirements and competing demands on attention (see

searchers can control general features, such as the order of presentation of trials, the timing and length of task breaks, and when scenarios are paused.

Control: Comparison with existing ATC simula-tors. There are a handful of ATC simulators that provide some level of experimental control. For example, both ATC-lab (Loft, Hill, et al., 2004) and TRACON (Acker-man, 1992) can present standardized air traffic scenarios. However, in comparison with ATC-labAdvanced, they pro-vide little programming control. Ackerman noted that in order to adapt TRACON to the study of skill acquisition, the features of TRACON simulations needed to be con-siderably modified, which resulted in high programming costs. This is the case with the original ATC-lab (Loft, Hill, et al., 2004) as well. As a result, researchers using simulators such as ATC-lab or TRACON would need to hire a technical specialist to implement changes to simu-lation features. In addition, as was discussed previously, many of these simulators have low realism.

Despite high realism, a significant limitation of many existing high-fidelity ATC simulators (such as the FAA Academy Training Simulator) is that they lack the experi-mental control required to make definitive conclusions re-garding the effects of independent variables on dependent variables (see Loft, Hill, et al., 2004). Furthermore, many of these simulators and other high-fidelity simulators that do provide better experimental control are not made freely available for research (e.g., EUROCONTROL Simulation Capability and Platform for Experimentation; ATCoach).

There are at least two ways in which experimental con-trol is restricted in some high-fidelity simulators. First, although general task conditions, such as the number of aircraft, type and mix of aircraft, and flight paths, can be controlled, little control is provided over the spatial and temporal properties of aircraft events. A lack of standard-ization in air traffic scenarios makes it difficult to control extraneous variables or to separate confounding variables. This can present a problem, such as when the effects of task demands on the time taken to complete specific con-trol tasks are assessed. Task demands may include average distance between aircraft, number of aircraft in altitude transition, and number of potential conflicts. Without control, researchers would be forced to extract values for task demands from historic flight data in ATC simula-tions and correlate those values with performance on a post hoc basis (e.g., Laudeman et al., 1998). This method would make it difficult to determine how unique factors and combinations of factors influence performance dif-ferentially (Loft, Sanderson, et al., 2007).

Second, the programming architecture underlying many existing high-fidelity simulators is typically based on an all-or-none philosophy, in that it does not provide substantial experimental control over what is displayed (e.g., altitude, maps), what specific ATC control tasks are conducted (e.g., accepting aircraft, conflict detection), or the manner in which participants interact with the ATC system (e.g., intervention methods, prediction tools). A consequence of this is that it is difficult to test predictions about processing mechanisms underlying performance on control tasks or to test specific theoretical questions.

124 FOTHERGILL, LOFT, AND NEAL

margins around the projected trajectory of aircraft as a function of experience could closely predict these inter-vention decisions (see Table 1).

In addition to these applied research programs, ATC-labAdvanced has been used when more basic research has been conducted. Prospective memory refers to remember-ing to perform an action in the future and is traditionally studied using verbal task paradigms (Einstein & McDan-iel, 1990). In the real world, highly practiced tasks make up much of the work of experts, meaning that in order to execute intentions, people must remember to deviate from routine (Dismukes, 2008). In addition, prospec-tive memory demands often occur in visuospatial, rather than verbal, contexts. Exploring prospective memory in the context of routine visuospatial tasks is thus of both

Remington et al., 2000). Experimental control was also required in order to systematically vary factors such as (1) the minimum separation of aircraft pairs, (2) the angles of intersection, and (3) the times to minimum separation. For example, for vertical problems, one aircraft was cruis-ing and the other climbing, with lateral separation set at 0 nm. On the basis of current speeds and climb rates, the vertical separation distance when the aircraft violated lat-eral separation ( 5 nm) varied from 0 ft to 4,000 ft. As is illustrated in Figure 5, one of the key findings was that the probability of controller intervention decreased with increases in the minimum lateral separation of the aircraft. Furthermore, expert controllers were significantly more likely to intervene than were trainees. A computational model that assumed that controllers place different safety

Table 1 Summary of the Three Main Studies That Have Used ATC-labAdvanced Simulations

Research Independent Dependent Results/Authors Participants Questions Variables Variables Conclusions

Fothergill & Neal (2008)

16 endorsed air traffic controllers

1. What is the effect of workload on conflict reso-lution decisions?

2. Can we computation-ally model conflict resolution heuristics as a function of workload?

1. Workload level of scenario (high vs. low)

2. Difficulty of calculating the op-timal solution (dif-ficult vs. easy)

1. Conflict resolu-tion strategy

1. When the optimal solution was difficult to calculate, controllers were less likely to select the optimal solution under high workload than under low workload.*

2. When the optimal solution was easy to calculate, control-lers were likely to select the optimal solution under both levels of workload.*

3. These results can be incor-porated into the development of a computational model that simulates how controllers re-solve conflicts in the field.

Loft, Bolland, & Humphreys (2007)

13 endorsed air traffic controllers and 7 trainee con-trollers (1 year training)

1. What aircraft geometry factors affect the proba-bility that controllers will intervene to ensure sepa-ration between aircraft?

1. Distance of minimum lat-eral separation (0 nm–20 nm)

2. Controller ex-perience (experts vs. trainees)

1. Probability of controller intervention

1. Controllers were more likely to intervene with in-creases in minimum lateral separation.

2. Experts were more likely to intervene than trainees.

3. A computational model that assumes controllers place safety margins around the projected trajectory of aircraft can account for both expert and trainee intervention decisions.

2. Will intervention deci-sions differ as a function of controller experience?

3. Can the psychologi-cal processes underlying these intervention deci-sions be captured by a computational model?

Loft, Campbell, & Remington (2008)

32 undergradu-ate psychology students

1. Will participants find it more difficult to remem-ber to deviate from strong routines, as compared with weak routines?

Routine strength 1. Probability of performing a rou-tine action instead of an intended action

1. Participants were more likely to forget to deviate from strong routines, as com-pared with weak routines.

2. No effect of routine strength on ongoing task performance.

2. Will ongoing task performance decrease when participants have to remember to deviate from strong routines, as com-pared with weak routines?

2. Ongoing task performance; aircraft accep-tance and conflict detection

*Since the dependent variable in this study was qualitative (solution type), categorical difference tests (McNemar tests) were used to determine whether participants switched their conflict resolution strategy preferences under different levels of workload and as a function of the difficulty of calculating the optimal solution.

ATC-LABADVANCED 125

ules to the training program. The amount of emphasis on each training module will depend on the realism of the simulation and the expertise of the participants (e.g., con-trollers, university students). The first module provides a general overview of the task. The second module de-scribes the human–machine interface, which includes the general display, maps, and aircraft flight strips. For the third module, participants are instructed on and practice how to use the prediction tools. For the fourth module, participants are instructed on and practice how to accept and hand off aircraft, how to assign cruise or boundary levels, and where the top of descent points are on sector maps. The fifth module instructs participants on how to answer questions that may appear during the experiment. For the sixth and final training module, participants are instructed on and practice how to intervene to modify aircraft trajectories. The duration of the ATC-labAdvanced training is approximately 30 min, although there is some variance with respect to how long it takes participants to familiarize themselves with the intervention methods and prediction tools.

The contents of data log files recorded at the end of ex-perimental sessions vary according to the type of experi-ment. Nevertheless, these files generally collect two types of data. The first type consists of the details of the air traf-fic scenarios that were presented on each trial, including the type, timings, and durations of aircraft events. Partici-pants’ actions are the second source of data. These actions include the timing of interventions to aircraft trajectories, subjective ratings, timing of aircraft acceptances and handoffs, and the timing and type of prediction tool use. ATC-labAdvanced also records all mouse movements made by participants in x-, y-coordinates, allowing researchers to make inferences regarding participant attention. Log files generated for each participant can be imported into statistical packages such as Microsoft Excel and SPSS.

ATC-labAdvanced was written using Qt Widget Library, owned by Troltech. Microsoft Visual C6 compiler was used to build the source code. ATC-labAdvanced can be run on desktops or laptop computers that run Microsoft Win-dows. No additional software or hardware is required. The program will update and display each aircraft’s position, speed, and level in the sector once every 5 sec, on the basis of the aircraft’s current speed, average climb/descent rates, and heading. These values are preset in a simulation script that specifies the series of x-, y-coordinates through which the aircraft will travel at various flight levels and speeds. In simulations in which participants are asked to resolve potential conflicts and assign boundary and cruise altitudes, participants may change these parameters dur-ing a trial.

ConclusionsATC simulators are frequently used in a variety of ap-

plied and basic research programs. Existing ATC simula-tors typically compromise between the extent to which they can mimic field experience (realism) and the experi-mental control that they can provide. In addition, very few ATC simulators are made publicly available to research-

practical and theoretical importance, and ATC-labAdvanced provides a useful platform for conducting such investiga-tions. Loft, Campbell, and Remington (2008) used ATC-labAdvanced to investigate individuals’ ability to remember to deviate from routine. Participants accepted aircraft into their sector and intervened to prevent the occurrence of conflicts by changing the flight levels of aircraft. Routine strength was manipulated by varying the number of times the participants performed a specific sequence of actions when accepting aircraft. At test, prospective memory in-structions asked the participants to substitute a different key for the standard key when accepting aircraft. The par-ticipants were more likely to forget to deviate from their strong routines (M .17), as compared with weak ones (M .08). Although experimental control was required to present standardized air traffic scenarios, the realism of the simulation was minimized in order to allow partici-pating first-year psychology students to quickly become highly practiced on a small number of ATC control tasks.

ATC-labAdvanced also has the potential to be more broadly used in basic and applied experimental research contexts. For example, we are currently using ATC-labAdvanced to ex-amine the motivational processes responsible for the regu-lation of task-directed effort, using a variety of behavioral, physiological, and self-report measures. The simulation is suited to the analysis of psychological phenomena at both the within- and between-persons levels of analysis, using both experimental and correlational methods (e.g., growth curve modeling; Bliese & Ployhart, 2002). Other types of phenomena that can be examined include the effects of fatigue, alcohol, and caffeine on attention, reaction time, and decision-making processes.

Training Manual, Data Logging, and System Requirements

The ATC-labAdvanced simulation package includes a training manual and practice scenarios. There are six mod-

0

.2

.4

.6

.8

1

Minimum Lateral Separation (nm)

Pro

bab

ility

of I

nte

rven

tio

n

ExpertsTrainees

0 1 2 4 6 8 10 16 14 16 18 20

Figure 5. The probability of intervention by controllers across the minimum lateral separation of aircraft pairs.

126 FOTHERGILL, LOFT, AND NEAL

Ericsson, K. A., & Williams, A. M. (2007). Capturing naturally occur-ring superior performance in the laboratory: Translational research on expert performance. Journal of Experimental Psychology: Applied, 13, 115-123.

Fothergill, S., & Neal, A. (2005). Managing the airspace: A task anal-ysis of Australian air traffic control. Australian Journal of Psychology Supplement, 57, 109-110.

Fothergill, S., & Neal, A. (2006). Decision making in air traffic control: How contextual factors influence conflict resolution choices. Australian Journal of Psychology Supplement, 59, 3.

Fothergill, S., & Neal, A. (2008). An evaluation of the effect of workload on conflict decision making in air traffic control. Australian Journal of Psychology Supplement, 60, 4.

Gray, W. D. (2002). Simulated task environments: The role of high-fidelity simulations, scaled worlds, synthetic environments, and mi-croworlds in basic and applied cognitive research. Cognitive Science Quarterly, 2, 205-227.

Gronlund, S. D., Ohrt, D. D., Dougherty, M. R. P., Perry, J. L., & Manning, C. A. (1998). Role of memory in air traffic control. Jour-nal of Experimental Psychology: Applied, 4, 263-280.

Hendy, K. C., Liao, J., & Milgram, P. (1997). Combining time and intensity effects in assessing operator information processing load. Human Factors, 39, 30-47.

Jeppesen (2007). TAAM solutions. Retrieved April 8, 2008, from www .preston.net/products/TAAM.htm.

Jones, D. G., & Endsley, M. R. (2000). Overcoming representational errors in complex environments. Human Factors, 42, 367-378.

Kopardekar, P., & Magyarits, S. (2003, June). Measurement and pre-diction of dynamic density. Paper presented at the 5th USA/Europe ATM Research and Development Seminar, Budapest.

Kwantes, P. J., Neal, A., & Loft, S. (2004). Developing a formal model of human memory in a simulated air traffic control conflict detection task. In Proceedings of the Human Factors and Ergonom-ics Society 48th Annual Meeting (pp. 391-395). Santa Monica, CA: Human Factors & Ergonomics Society.

Lamoureux, T. (1999). The influence of aircraft proximity data on the subjective mental workload of controllers in the air traffic control task. Ergonomics, 42, 1482-1491.

Laudeman, I., Shelden, S., Branstrom, R., & Brasil, C. (1998). Dynamic density: An air traffic management metric (NASA-TM-1988- 11226). Moffett Field, CA: NASA Ames Research Center.

Loft, S., Bolland, S., & Humphreys, M. (2007). Modelling the human air traffic controller. Expert–trainee differences in conflict detection. In R. Jensen (Ed.), Proceedings of the 14th International Symposium on Aviation Psychology (pp. 409-414). Dayton, OH: Association for Aviation Psychology.

Loft, S., Campbell, L. & Remington, R. W. (2008, March). Failure to deviate from routine and task interference in an air traffic control task. Paper presented at the 34th Australasian Experimental Psychology Conference, Perth, Australia.

Loft, S., Hill, A., Neal, A., Humphreys, M., & Yeo, G. (2004). ATC-lab: An air traffic control simulator for the laboratory. Behavior Re-search Methods, Instruments, & Computers, 36, 331-338.

Loft, S., Humphreys, M., & Neal, A. (2004). The influence of memory for prior instances on performance in a conflict detection task. Journal of Experimental Psychology: Applied, 10, 173-187.

Loft, S., Neal, A., & Humphreys, M. (2007). The development of a general associative learning account of skill acquisition in a conflict detection task. Journal of Experimental Psychology: Human Percep-tion & Performance, 33, 938-959.

Loft, S., Sanderson, P., Neal, A., & Mooij, M. (2007). Modeling and predicting mental workload in en route air traffic control: Critical review and broader implications. Human Factors, 49, 376-399.

Metzger, U., & Parasuraman, R. (2001). The role of the air traffic controller in future air traffic management: An empirical study of ac-tive control versus passive monitoring. Human Factors, 43, 519-528.

Mook, D. G. (1983). In defense of external validity. American Psycholo-gist, 38, 379-387.

Rantanen, E. M., & Nunes, A. (2005). Hierarchical conflict detection in air traffic control. International Journal of Aviation Psychology, 15, 339-362.

Raytheon (2005). FIRSTplus tower and radar simulator. Retrieved

ers who wish to use or adapt them. The present article has presented a new, publicly available ATC simulation package called ATC-labAdvanced.1 The realism and experi-mental control provided by ATC-labAdvanced represents an advance over many currently available simulators. In ad-dition, ATC-labAdvanced has the programming control to allow systematic variation of realism and control in order to investigate specific research questions of interest in a cost-effective manner.

AUTHOR NOTE

This research was supported in part by Linkage Grant LP0453978 from the Australian Research Council. The authors thank Phillip Waller for his C programming of the ATC-labAdvanced program. Thanks also go to Peter Lindsay for his contribution to the formulae that underlie the 2-D (lateral) dynamics of ATC-labAdvanced. Please contact Peter (p.lindsay@uq.edu.au) for further information regarding how these formulae were derived. Correspondence concerning this article should be addressed to S. Fothergill, School of Psychology, University of Queensland, Brisbane 4072, QSLD, Australia (e-mail: selina@psy.uq.edu.au).

REFERENCES

Ackerman, P. L. (1992). Predicting individual differences in complex skill acquisition: Dynamics of ability determinants. Journal of Ap-plied Psychology, 77, 598-614.

Athenes, S., Averty, P., Puechmorel, S., Delahaye, D., & Col-let, C. (2002). Complexity and controller workload: Trying to bridge the gap. In Proceedings of the 2002 International Conference on Human–Computer Interaction in Aeronautics (HCI-Aero 2002) (pp. 56-60). Cambridge, MA: MIT.

Berkowitz, L., & Donnerstein, E. (1982). External validity is more than skin deep: Some answers to criticisms of laboratory experiments. American Psychologist, 37, 245-257.

Bliese, P. D., & Ployhart, R. E. (2002). Growth modeling using ran-dom coefficient models: Model building, testing, and illustration. Or-ganizational Research Methods, 5, 362-387.

Bolland, S., Fothergill, S., & Humphreys, M. (2007). Modelling the human operator: Part II. Emulating controller intervention. In R. Jensen (Ed.), Proceedings of the 14th International Symposium on Aviation Psychology (pp. 57-62). Dayton, OH: Association for Avia-tion Psychology.

Boring, E. G. (1954). The nature and history of experimental control. American Journal of Psychology, 67, 573-589.

Brehmer, B., & Dorner, D. (1993). Experiments with computer- simulated microworlds: Escaping both the narrow straights of the lab-oratory and the deep blue sea of the field study. Computers in Human Behavior, 9, 171-184.

Brunswick, E. (1956). Perception and the representative design of psy-chological experiments. Berkeley: University of California Press.

Callantine, T. J. (2002). CATS-based air traffic controller agents (NASA Technical Memorandum 211856). Moffett Field, CA: NASA Ames Research Center.

Cox, M. (1994). Task analysis of selected operating positions within UK air traffic control (Rep. No. 749). Farnborough, U.K.: DRA/Institute of Aviation Medicine.

DiFonzo, N., Hantula, D. A., & Bordia, P. (1998). Microworlds for experimental research: Having your (control and collection) cake, and realism too. Behavior Research Methods, Instruments, & Computers, 30, 278-286.

Dismukes, R. K. (2008). Prospective memory in aviation and everyday settings. In M. Kliegel, M. A. McDaniel, & G. O. Einstein (Eds.), Prospective memory: Cognitive, neuroscience, developmental, and applied perspectives (pp. 411-428). Mahwah, NJ: Erlbaum.

Ehret, B. D., Gray, W. D., & Kirschenbaum, S. S. (2000). Contend-ing with complexity: Developing and using a scaled world in applied cognitive research. Human Factors, 42, 8-23.

Einstein, G. O., & McDaniel, M. A. (1990). Normal aging and pro-spective memory. Journal of Experimental Psychology: Learning, Memory, & Cognition, 16, 717-726.

ATC-LABADVANCED 127

UFA Inc. (n.d.). The experts in air traffic control simulation: Products. Retrieved on April 8, 2008, from www.atcoach.com/products1.html.

Yeo, G., & Neal, A. (2004). A multilevel analysis of effort, practice and performance: Effects of ability, conscientiousness and goal orienta-tion. Journal of Applied Psychology, 89, 231-247.

NOTE

1. Research groups interested in using ATC-labAdvanced for noncom-mercial purposes can download the program from www.psy.uq.edu.au/directory/index.html?id=25. The following materials will be available for download: the ATC-labAdvanced base code; an example XML script based on a representative sample of the published studies; instructions on how to use the programming control features of the XML scripts; math-ematical formulae, spreadsheets, and instructions; the training modules and instructions; and the practice scenarios. Questions regarding any of these materials can be directed to S. Fothergill (selina@psy.uq.edu.au) at the University of Queensland.

(Manuscript received January 14, 2008; revision accepted for publication September 26, 2008.)

April 8, 2008, from www.ray.ca/external/home.nsf /(Webpages)/ Products_FIRSTplus? OpenDocument.

Remington, R. W., Johnston, J. C., Ruthruff, E., Gold, M., & Romera, M. (2000). Visual search in complex displays: Factors af-fecting conflict detection by air traffic controllers. Human Factors, 42, 349-366.

Rodgers, M. D., & Drechsler, G. K. (1993). Conversion of the CTA Inc, En Route operations concepts database into a formal sentence outline job task taxonomy (FAA Rep. DOT/FAA/AM-93/1). Washing-ton, DC: FAA Office of Aviation Medicine.

Schiff, W., Arnone, W., & Cross, S. (1994). Driving assessment with computer-video scenarios: More is sometimes better. Behavior Re-search Methods, Instruments, & Computers, 26, 192-194.

Simon, H. A. (1956). Rational choice and the structure of environments. Psychological Review, 63, 129-138.

Späth, O., & Eyferth, K. (2001). Conflict resolution in en route traffic: A draft concept for an assistance system compatible with solutions of air traffic controllers. MMI-Interaktiv, 5, 1-11.

Stone, M., Dismukes, K., & Remington, R. (2001). Prospective mem-ory in dynamic environments: Effects of load, delay, and phonological rehearsal. Memory, 9, 165-176.