partitioning complexity in air traffic management tasks

This article was downloaded by: [Georgia Tech Library]On: 13 November 2014, At: 03:02Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK

The International Journal of Aviation PsychologyPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/hiap20

Partitioning Complexity in Air Traffic Management TasksM. L. Cummings & Chris G. TsonisPublished online: 13 Nov 2009.

To cite this article: M. L. Cummings & Chris G. Tsonis (2006) Partitioning Complexity in Air Traffic Management Tasks, TheInternational Journal of Aviation Psychology, 16:3, 277-295

To link to this article: http://dx.doi.org/10.1207/s15327108ijap1603_3

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/hiap20

http://dx.doi.org/10.1207/s15327108ijap1603_3

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

Partitioning Complexity in Air TrafficManagement Tasks

M. L. Cummings and Chris G. TsonisAeronautics and Astronautics Department

Massachusetts Institute of Technology

Cognitive complexity is a term that appears frequently in air traffic control researchliterature, yet there is little principled investigation of the potential sources of cogni-tive complexity. Three distinctly different sources of cognitive complexity are pro-posed: environmental, organizational, and display. Two experiments were conductedto explore whether these proposed components of complexity could be effectivelypartitioned, measured, and compared. The findings demonstrate that sources of com-plexity can be decomposed and measured and furthermore, the use of color in dis-plays, a display design intervention meant to reduce environmental complexity, canactually contribute to it.

Addressing the difference between environmental and innate human complexity(often referred to as cognitive complexity), Simon described an ant’s path as itnavigates across a beach. The ant eventually reaches its destination, but becausethe ant must constantly adapt its course as a result of obstacles, the path seemsirregular, laborious, and inefficient. Simon pointed out that although the ant’spath seems complex, the ant’s behavior is relatively simple as compared to thecomplexity of the environment. Simon (1996) proposed the following hypothe-sis as a result: “Human beings, viewed as behaving systems, are quite simple.The apparent complexity of our behavior over time is largely a reflection of thecomplexity of the environment in which we find ourselves” (p. 53).

This distinction between innate, or cognitive complexity and environmentalcomplexity is especially relevant in light of the considerable research conducted inair traffic controller cognitive complexity. Several studies have investigated airtraffic control (ATC) information complexity issues (for a review, see Hilburn,

THE INTERNATIONAL JOURNAL OF AVIATION PSYCHOLOGY, 16(3), 277–295Copyright © 2006, Lawrence Erlbaum Associates, Inc.

Requests for reprints should be sent to M. L. Cummings, Massachusetts Institute of Technology, 77Massachusetts Avenue, 33–305, MIT, Cambridge, MA 02139. Email: [email protected]

Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

2004; Majumdar & Ochieng, 2002). In this literature, several common cognitivecomplexity factors have emerged to include traffic density, traffic mix, aircraftspeeds, and transitioning aircraft. However, in light of Simon’s ant parable, thesefactors really represent environmental complexity factors that influence cognitivecomplexity, as compared to other factors such as separation standards, which areintentional organizational factors externally imposed on controllers. This is an im-portant distinction because as can be seen in Figure 1, there are several sources ofcomplexity that can affect an individual’s cognitive complexity level.

Figure 1 illustrates the decomposition of complexity as it applies to complexsociotechnical systems in which operators interact with their environment, medi-ated by procedures and technology, such as ATC. Cognitive complexity is actuallythe lowest form of complexity in this notional model that can be affected by some orall of the preceding levels, although not necessarily in a linear fashion. A true, objec-tive state of complexity exists in the world (environmental complexity), as per-ceived by one or more humans (cognitive complexity). In ATC, factors that affectenvironmental complexity include number of aircraft, weather, congestion, and soon, which are essentially the same elements identified in ATC cognitive complexityliterature. As illustrated in Figure 1, the operator’s perceived complexity can be mit-igated through two strategies, organizational and displays. Organizations like theFederal Aviation Administration (FAA) and Department of Defense (DoD) will in-stitutepoliciesandprocedures tomitigateenvironmental complexity such thatoper-ations are both safe and meet specified goals. For example, separation standardsexist to mitigate complexity for controllers (and promote safety); however, whenairspace becomes saturated, the need to maintain these organization-driven con-straints causes situation complexity, and thus workload, to increase.

278 CUMMINGS AND TSONIS

FIGURE 1 Human supervisory control complexity chain.

Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

In human supervisory control domains, displays are nominally designed to fur-ther reduce complexity by representing the environment so that a correct mentalmodel can be formed and suitable interactions can take place (Woods, 1991). Thusdisplays should reduce complexity and hence workload through transforminghigh-workload cognitive tasks such as mental computations into lower workloadtasks through direct perception (i.e., visually; Miller, 2000). However, in complexand dynamic sociotechnical domains such as ATC, it is not always clear whetherdisplays and organizational interventions actually alleviate or contribute to theproblem of complexity.

Although previous research has been critical in addressing controller workloadas a function of cognitive complexity due to environmental and, to a lesser degree,organizational factors, significantly less attention has been paid to the role of dis-plays in either adding to or mitigating complexity in the ATC environment. As il-lustrated in Figure 1, only examining environmental factors as sources ofcomplexity is necessarily limited and does not capture the actual complexity per-ceived by operators. In an attempt to partition and quantify the influence of envi-ronmental, organizational, and display complexity factors on cognitivecomplexity, two experiments were conducted to determine if these componentscould be independently measured, and if so, determine possible implications fordecision support design.

EXPERIMENT 1

In addition to traffic density and related factors, it has also been hypothesizedthat the underlying airspace structure is a critical complexity factor (Histon etal., 2002). In theory, airspace structure provides the basis for mental abstractionsthat allow controllers to reduce complexity and maintain situation awareness.Histon et al. (2002) proposed that these mental abstractions, known as struc-tured-based abstractions, can be generalized to standard flows (reminiscent ofPawlak, Brinton, Crouch, & Lancaster’s, 1996, “streams”), groupings, and criti-cal points. Providing air traffic controllers with these interventions, either ex-plicitly through airspace design or implicitly through policy, in theory shouldhelp controllers improve through mental models, reduce overall complexity, andreduce perceived workload.

In a study investigating judgment and complexity, Kirwan, Scaife, and Ken-nedy (2001) determined that airspace sector design was second only to traffic vol-ume in terms of contributing to cognitive complexity. In terms of the model inFigure 1, airspace sector design straddles both the organizational and display com-plexity categories. Designed by humans to mitigate environmental complexity,airspace structure is an organizational policy. However, airspace structure con-tains significant visual components represented through displays; thus, it is an en-

PARTITIONING COMPLEXITY 279

Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

vironmental complexity intervention both from an organizational and displayperspective.

Including interventions in airspace sector design such as critical points (pointsthrough which aircraft must pass) and designated standard flows (e.g., jetways)can increase order and improve predictability, and thus lower cognitive complex-ity. However, it is also possible that when uncertainty levels increase, usually as afunction of dynamic environmental factors such as changes in weather and avail-able airspace, these same airspace structures could actually add to complexity.Airspace structure and procedures mitigate complexity in what are termed nomi-nal situations, but when an off-nominal condition occurs, such as an emergency orunexpected weather phenomena, the resultant increasing uncertainty causes com-plexity to grow (Athenes, Averty, Puechmorel, Delahaye, & Collet, 2002). To ad-dress whether airspace structure as an organizational or display factor is asignificant contributor to cognitive complexity, an experiment was conducted asdescribed next.

Method

Apparatus, participants, and procedure. To objectively investigate theimpact of complexity sources described in Figure 1 on controller performance, ahuman-in-the-loop simulation test bed was programmed in MATLAB® (Figure 2).Because the participant pool consisted primarily of college students, it was neces-sary to devise a simplified and abstract task that represented fundamental elementsof enroute ATC control. Participants as controllers were only concerned with pro-viding heading commands to the aircraft, and aircraft velocities were held constant.Twenty egress points were located in the periphery of the airspace (Figure 2), and


FIGURE 2 Experiment 1 test bed.

Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

each incoming aircraft was assigned one of these egress points. The primary goalwas to direct the incoming aircraft to the assigned egress, and when an aircraft ex-ited correctly, participants received points. Additional waypoints were locatedthroughout the airspace sector by which participants could collect more points bydirecting aircraft through these points.

A path connecting the waypoints with the highest values was the optimal paththat encouraged controllers to fly aircraft on a specified route and not directly totheir exits. To discourage controllers from flying aircraft through all waypoints(which is unrealistic in actual enroute ATC control), scores were penalized basedon an aircraft’s total time transiting the airspace sector. A final component of thescore was a penalty for flying through a no-fly zone, which represented restrictedairspace. Maximization of total score was the participants’ goal, which could beachieved by flying the optimal path that was not necessarily the most direct path.Participants’ total scores were displayed in real time. To provide a strong incentivefor participants to perform well and attempt to maximize their total score, a $50prize was offered to the participant who attained the highest score.

Two sources of complexity were investigated using this test bed: dynamic air-space sectors (environmental complexity factor) and airspace structure (displaycomplexity factor). To investigate display complexity as a function of airspacestructure, in certain scenarios participants were given structure through the displayof the actual optimum paths (those that maximized the score as a function ofwaypoints and time; Figure 2a). As demonstrated in Figure 2b, some participantswere not given this same visual path structure, although the waypoints were thesame, so the optimal paths were identical, but just not clearly linked with lines.

The second potential source of complexity was the use of dynamic as opposedto static airspace sectors, termed no-fly zones. In the dynamic case, the no-flyzones moved at rates of approximately two fifths the aircraft velocity and weremeant to represent areas of airspace that can change rapidly such as military oper-ating areas or areas of severe weather conditions. In these cases, air traffic control-lers must develop new routes for affected aircraft under time pressure. Similarly,in this test bed, as airspace changed, participants were required to reroute aircraftbut still attempt to maximize their overall score. For those participants with thedisplayed paths between optimal waypoints, the displayed lines were still the opti-mum, but only in cases where they were not obstructed, and thus participants hadto replan for a new best case.

The primary purpose of the experiment was to investigate whether visual struc-ture in an airspace sector, in combination with changes in the external airspace en-vironment, added to or mitigated perceived complexity as measured throughperformance. In theory, under nominal conditions (static airspace), participants’performance should be higher with the displayed lines than no lines, as the optimalpath is clearly laid out. However, in the off-nominal case when airspace is dy-namic, the visually compelling paths may no longer be optimal and could provide


Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

a distraction to participants who must mentally generate new possible paths. In thiscase, the more unstructured airspace without the lines may be easier for partici-pants to use to generate contingency plans.

Training and testing was conducted using a Dell personal computer with a19-in. color monitor with a screen area of 1024 × 768 pixels, 16-bit high-color res-olution, and a Pentium 4 processor. During testing, all user responses were re-corded in separate files specific to each participant and scenario. A Visual Basicscript was then written that scored and compiled the data into a single spreadsheetfile for the subsequent statistical analysis. After signing required consent forms,participants completed a tutorial that discussed the nature of the experiment, ex-plained the context and use of the interface, and gave them the opportunity to un-derstand the scoring mechanism. Participants completed four practice scenariosthat exposed them to every combination of independent variables. They then be-gan the randomly ordered four test sessions, which lasted until all aircraft had ex-ited the airspace (approximately 6 to 7 min).

Design of experiment. As discussed earlier, two independent variableswere investigated: (a) structure of airspace (either with lines or no lines), and (b) theenvironment state (either static or dynamic no-fly zones). A single dependent vari-able of total performance score was used. As described earlier, the score was a lin-ear weighted function of correct aircraft egress actions, bonus waypoints, incurredpenalties for no-fly-zone violations, and total time transiting in sector. In the case ofegress scores, aircraft did not received points for exiting through the wrong egressand the maximal egress score was obtained by exiting near the center of the egress,with a minimal score awarded toward the edge. The four experimental scenarioswere 90° rotations of each other, and the rotation was automated in a scenario gen-eration tool (Tsonis, Cunha, & Histon, 2005). The statistical model used was a 2 × 2fully crossed analysis of variance (ANOVA), and the four scenarios were randomlypresented to a total of 20 participants.

Results and Discussion

The ANOVA linear model revealed that the environmental factor of dynamicairspace was highly significant, F(1, 74) = 54.55, p < .001, all α < .05 whereas,the displayed airspace structure and the Environment × Displayed Structure in-teraction were not significant. Figure 3 depicts the average performance scoresacross all four conditions. It can be seen on inspection that the performancescores were clearly higher in the static environment scenario as opposed to thedynamic environment phase. Whether participants had less or more airspacestructure did not significantly affect their scores. These results show that for thisrepresentative ATC task, the environment was a significant contributor to com-plexity and performance, causing lower scores. Thus as environmental complex-


Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

ity increased through changing airspace, workload, and thus cognitive complex-ity, increased, with no apparent effect, either positive or negative, fromincreasing airspace structure. This is not to say aircraft structure was not an im-portant contributor, just that its effects could not be measured due to the over-whelming effect of changing airspace.

In terms of the model in Figure 1, this experiment demonstrated for this repre-sentative ATC task that the main component of complexity associated with con-troller workload was environment, and not organizational or display related.Dynamically changing airspace structure was far more influential than the designof the airspace itself. Thus, although sector design may be a contributing factor toair traffic controller performance, environmental complexity factors such as thun-derstorms and special-use airspace (SUA) that intermittently become availablecan be significantly larger contributors to individual cognitive complexity.

Furthermore, these results provide quantitative support for previous subjectiveassessments of controllers that active SUA increases complexity and would bene-fit from some display intervention (Ahlstrom, Rubinstein, Siegel, Mogford, &Manning, 2001). In light of the results reported here, it is likely that SUA, an orga-nizationally driven constraint, can increase workload for controllers, not becauseof the actual structure of the airspace, but because the status can change. SUA ex-ists to mitigate environmental complexity in that, for example, it separates militaryand commercial traffic. However, when SUAs cycle between active and inactive,especially in short time periods, situation complexity can increase.

EXPERIMENT 2

Whereas Experiment 1 addressed complexity concerns due to airspace design, asecond experiment was conducted to examine more specifically complexity due


FIGURE 3 Experiment 1 re-sults.

Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

to display design in terms of a generic timeline. Timelines are currently in usefor the Center-TRACON Automation System (CTAS) Traffic Management Ad-visor (TMA) to provide for coordination of arrival traffic between Center andTRACON airspace (Hansman & Davison, 2000). In addition, although not yetoperational, the Surface Management System integrates both tactical and strate-gic timelines; however, results of exploratory and operational trials have beenmixed (Atkins et al., 2004; Brinton, Carniol, Butler, Duley, & Hitt, 2002;Spencer, Smith, Billings, Brinton, & Atkins, 2002). For example, in one recentoperational trial, local and ground controllers rejected timelines; whereas trafficmanagers felt they were very useful (Atkins et al., 2004). However, in all trials,evaluation of the timelines has been subjective based on controllers’ opinions, sono performance data have yet been published that objectively demonstratewhether timeline displays, or features of these timelines, actually help control-lers in their tasks.

One specific concern in the design of timelines is the use of color. With advanceddisplay technologies that allow designers to use hundreds of color conventions withno added system cost, there has been a recent increase in the amount of color used inATC displays. However, there has been little, if any, consideration of how muchcolor might be too much from an information processing perspective. Xing andSchroeder (2006) documented the extensive and inconsistent color use in ATC dis-plays. The FAA has issued no formal requirements for the use of color in ATC dis-plays, and consequently manufacturers of ATC technologies are free to decide theirown color schemes. Indeed even ATC facilities and individual users are allowed todetermine their own color preferences in some tools, such as the Enhanced TrafficManagement System and the Information Display System. Although guidelines ex-ist for the general use of color in ATC display technologies (Cardosi & Hannon,1999; Reynolds, 1994), they generally address optimal perception conditions andnot how the use of color could improve or degrade task performance.

Color in ATC displays is typically used for three primary task reasons: (a) to drawattention, (b) to identify information so that search time is minimized, and (c) to or-ganize information through segmentation such as airspace boundaries (Xing &Schroeder, 2006). Research has shown that color is superior to achromatic visual at-tributes (e.g., luminance, shapes,and text) insearchandorganization tasksprimarilybecause color-coded information can be processed more quickly (Christ, 1975). Forexample, the use of red in displays to convey warning information allows operatorssuch as pilots and ATC personnel to quickly assess an urgent problem state.

Although color can provide improvements in search and organization tasks,previous research has shown that although participants believed that color im-proved their performance, color did not improve target detection or identification(Jeffrey & Beck, 1972). In addition, the use of color can cause cognitive tunnelingor inattentional blindness (Simons, 2000), in which operators may miss other im-portant information on a display because they fixate on the more salient color


Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

change. Another possible problem with the current use of color in ATC displays isthat many displays use the same color to identify different types of information,such as using blue to indicate both an airline type, as well as a particular phase offlight. As the number of display colors increases, with possible multiple meanings,both the perceptual and cognitive load of controllers is increased, subsequentlyadding to cognitive complexity.

To address whether display complexity could be captured due to the use ofcolor categories in a generic timeline, an experiment was conducted using theframework in Figure 1 for partitioning sources of complexity.

Method

Apparatus, participants, and procedure. To objectively investigate theimpact of environmental and display complexity factors associated with a timeline,a human-in-the-loop simulation test bed was developed in MATLAB® (Figure 4).Because the participant pool consisted of college students, a simplified task wasneeded that contained realistic decision support tools, yet did not require years ofexpertise to operate. Thus the subjects’ task was that of a low-level surface managerof incoming and outgoing traffic, responsible for assisting a supervisor in manag-ing personnel for baggage handling, ground personnel, and galley service.

The radar screen in Figure 4 represents incoming and outgoing traffic in the ter-minal control area. The circle in the middle represents the airport area. The


FIGURE 4 Experiment 2 interface.

Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

timeline contains two essential elements, much like what is used in actual ATCtimelines, incoming traffic (left) and outbound traffic (right). The incoming side ofthe timeline represents the time until the expected aircraft gate arrival. The outgo-ing side of the timeline represents the time that an aircraft begins loading passen-gers and baggage at the gate until it becomes airborne. Each aircraft label containsthe flight number, number of passengers, number of baggage items, assigned gatenumber, velocity (when airborne), and altitude (when airborne).

After signing consent forms, participants completed a tutorial and three prac-tice scenarios. They then began the randomly assigned 18 test sessions, whichlasted approximately 3.5 min each. Participants were required to monitor both thespatial display and timeline, and answer questions from their superior throughdatalink (text message) communication. Participants were also required to notifytheir superior when aircraft of a particular airline entered the outermost circle.

Design of experiment. Two independent variables were used to representenvironmental complexity, number of aircraft (10, 20, 30) and arrival sequence (se-quential vs. nonsequential.) Aircraft number has been cited throughout the litera-ture as a leading source of complexity (Hilburn, 2004), and it has been previouslydemonstrated in an ATC study that number of aircraft is the primary driver of oper-ator performance (Kopardekar, 2003). Arrival sequence was included as an addi-tional environmental complexity factor because in the context of a visual lineartimeline display, the sequence could affect a controller’s ability to effectivelysearch for information. Those aircraft that maintain their relative positions in thetimeline are easier to track than those aircraft that appear to “jump” on the timeline,such as those aircraft that are put into holding patterns, disrupting the expected flowof traffic. This could be problematic for situation awareness as a timeline does notconvey the uncertainty and dynamic changes of nonsequential aircraft arrivals.Thus, we examined whether sequential arrivals versus nonsequential arrivals asdisplayed by the timeline influenced operator performance.

The third and final factor investigated was the number of color categories usedto represent information about incoming and outgoing aircraft (3, 6, 9). It was im-portant to ensure that as the levels of color increased, the additional colors addedmore detailed information for the general category so the meaning of the colorswould not be a confound. In the three-color condition, aircraft were generallygrouped into arrival, taxiing, and gate status categories. Thus as more colors wereadded, more refinements were made to the specific categories. For example, in thethree-color condition, a single color (white) represented arrivals; whereas in thenine-color condition, arrivals were further subdivided into enroute both in and out-side of terminal area, as well as on final approach. As in actual timeline displays,colors were sometimes duplicated across arrival and departing aircraft (e.g., yel-low on the left side of the timeline meant taxiing in, whereas yellow on the righttimeline meant taxiing out; thus yellow always meant taxiing).


Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

The primary dependent variable was answer accuracy, measured through ques-tions in an embedded datalink tool, a strategy that has been useful in developingworkload metrics in military command and control (Cummings & Guerlain,2004). The questions introduced through the datalink window fell into two catego-ries: (a) search questions that relied on perceptual information processing; for ex-ample, participants had to locate a single piece of information such as the gate callsign of a certain aircraft; and (b) problem-solving questions, in which participantswere required to calculate or derive information from multiple sources; for exam-ple, the number of aircraft at their gates. In each of the 18 scenarios, participantswere asked six questions that were generally evenly split between prob-lem-solving and search categories. In addition to answering questions, participantswere required to notify the supervisor when they first noted that a flight of a certaincarrier entered the outermost radial circle on the spatial display. Failures to recog-nize this situation resulted in an error of omission, which is the other dependentvariable. The statistical model used was a 3 × 3 × 2 fully crossed ANOVA and the18 scenarios were randomly presented to a total of 29 participants.

Results

Accuracy results. For this experiment, participants were classified as gen-erally accurate if they answered at least four out of six test questions correctly in aparticular scenario. An overall comparison of participant performance for thesearch and problem-solving question accuracy in each of the 18 test sessions re-


TABLE 1Color Categories

No. of Colors

Flight Status 3 6 9

Scheduled to arrive WhiteEn route BlueEn route outside airspace BlueEn route inside airspace CyanOn final approach Orange OrangeTaxiing in Yellow Yellow YellowOn runway RedReady to dock with gate PurpleAt gate Green Green GreenReady for pushback Pink PinkTaxiing out Yellow Yellow YellowIn final queue (holding short) Orange OrangeDeparted White White White

Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

veals intriguing results. For the search questions, additional color categories in-creased inaccuracy, most predominantly from three to six color categories; how-ever, the general participant population still answered accurately overall (Figure 5;Pearson chi-square showed marginal significance, p = .069).

For the problem-solving questions, the increase in color categories actually im-proved the answer accuracy, although there was no difference between the use ofsix or nine color categories (Figure 6, Pearson chi-square test showed p < .001, be-tween three colors vs. six and nine). In general for both question types, wrong an-swers increased with increasing plane levels. Measures of association usingCramer’s V are reflected in Table 2. For the search questions, arrival patterns andnumber of planes affected correct answers; whereas increasing planes and colorcategories were both moderately associated with wrong answers.

Errors of omission. A known negative consequence of high human supervi-sory control workload is increase in human error, as illustrated in the previous sec-tion. The incorrect answers given by the participants to datalink queries representerrors of commission, but errors of omission are also prevalent and of particular


FIGURE 6 Accuracy for problem-solving questions.

FIGURE 5 Accuracy for search questions.

Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

concern in the ATC community. To this end, the influence of the three display met-rics on omission error occurrences was investigated. Participants were told thatwhenever a plane marked as BAW (British Airways) entered the area between thetwo dashed lines on the radar display in Figure 4, they were to notify their supervi-sor by clicking a single button on the lower right corner, marked “Warn groundmanager.” For all independent variables, correct notifications exceeded errors ofomission (unlike accuracy for problem-solving questions); however, there wereimportant trends in error rates due to the display components.

Figure 7 represents the overall correct notifications as compared to the errors ofomission for the color categories, which was significant (Wilcoxon, p = .009). AMann–Whitney test between the errors for the six and nine color categories alsowas significant (p = .001). Figure 8 demonstrates a similar omission error increas-ing trend between 20 and 30 planes (Mann–Whitney, p < .001). The arrival patterneffect is seen in Figure 9. Participants’ errors of omission increased when the ar-rival patterns were nonsequential, and the difference between sequential andnonsequential was significant (Mann–Whitney, p = .036).

Given that all three independent variables showed significance throughnonparametric testing, further investigation was warranted to determine the mag-nitude of any significant relation. Association testing using the Kendall tau-b sta-tistic revealed significant associations for all three variables, as shown in Table 3.As color categories increased from three to nine and as planes increased from 10 to30, errors of omission increased, as indicated by the positive association. Partici-pants who experienced sequential arrival patterns experienced lower errors ofomission as indicated by the negative sign. All associations were moderate, but theplane factor, a complexity factor driven by the external environment, contributedonly slightly more to error rates than the color category, a display design interven-tion typically included to mitigate complexity.

Discussion

In this second experiment that investigated whether complexity from differentsources could be effectively measured and compared, we identified two sources


TABLE 2Accuracy Measure of Association

Search Accuracy Problem Solving Accuracy

Color categories .102* .240**Planes .269** .275**Arrival pattern .327** ns

*p < .01. **p < .001.

Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

FIGURE 7 Omission errors: Colors.

FIGURE 8 Omission errors: Planes.

FIGURE 9 Omission errors: Arrival pattern.

290

Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

of environmental complexity (arrival sequence and number of aircraft), andcomplexity from displays (color coding). Using a generic timeline and groundtraffic management task, we demonstrated that these sources of complexity canbe partitioned, measured for increasing levels, and then compared to one anotherto determine which factors contributed the most to cognitive complexity as mea-sured through performance. Although the following sections contain more de-tails about the analysis of environmental and display complexity sources, in gen-eral we determined that increasing aircraft number is a primary driver ofcomplexity. However, increasing use of color categories contributed onlyslightly less to poor performance.

Environmental complexity. The aircraft number factor was included be-cause it represents a primary source of environmental complexity for controllers. Ingeneral, answer accuracy decreased and errors of omission increased as the numberof aircraft increased. Increasing number of entities is a known source of cognitivecomplexity (Edmonds, 1999), and aircraft count has been identified as the primaryenvironmental driver of ATC cognitive complexity (Kopardekar, 2003), so the re-sult is expected.

Traffic flow is another commonly cited source of ATC complexity (Majumdar& Ochieng, 2002), and although arrival sequence was not a significant factor foranswer accuracy, it did significantly affect participants’ errors of omission but to alesser degree than the plane and color categories. Arrival sequence did not directlycause errors of omission, but nonsequential patterns increased search time and thusincreased overall workload, which diverted attention from the monitoring task.The negative influence of nonsequential arrivals on operator performance illus-trates the consequences of change blindness, which occurs when operators fail todetect a change in a visual display. This blindness typically occurs if a change oc-curs in concert with another visual event, such as animation or an eye blink, andhas been shown to occur even for large changes in a visual display (Rensink,O’Regan, & Clark, 1997).

The arrival sequence result suggests that although arrival sequencing may be anenvironmental complexity factor, there is a display component associated with it,in the sense that there was no visual indication to users when aircraft unexpectedly


TABLE 3Measures of Omission Error Association

Factor Association Significance

Color category .330 p < .001Planes .355 p < .001Arrival sequence –.293 p = .001

Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

changed sequence, which is typical of most timelines. Had some kind of situationawareness recovery tool been implemented for aircraft jumps in the timeline, thiseffect may have been mitigated.

Display complexity. For search questions that relied primarily on perceptualcuing, color coding was generally helpful; however, there was a slight drop in accu-racy as color categories increased. For the problem-solving answer accuracies,color coding significantly improved correct answers from three to six colors butreached a plateau at six, so nine color categories provided no additional benefit.However, one area where increasing color categories appeared to be detrimentalwas in the analysis of errors of omission, which significantly increased from six tonine color categories. This is an important finding because color coding is a designintervention meant to mitigate complexity, not add to it.

In this generic timeline task, increasing color usage beyond six color categoriesappeared to promote inattentional blindness or cognitive tunneling. Although in-creasing the number of color categories does not cause people to forget actions, itdoes require that controllers spend more time in search and mapping tasks, andthus takes away time from other tasks and increases the likelihood that a subse-quent task will be forgotten. In addition, although number of aircraft exhibited aslightly stronger association, increasing color categories was a significant contrib-utor to errors of omission, more so than arrival rate. This result that suggests colorusage should not exceed six color categories is in keeping with both previous ATCresearch (Cardosi & Hannon, 1999) and general display design principles(Wickens & Hollands, 2000).

CONCLUSIONS

Complexity, as it applies to operators and the ATC and management environ-ments, or any other human supervisory control domain, cannot be simply cate-gorized as cognitive complexity. We proposed there are different components ofcomplexity (Figure 1) and demonstrated through two experiments that thesesources can be partitioned and the impact on human performance can be mea-sured. Specifically, we partitioned two sources of complexity, environmentaland display, in a basic enroute ATC control task and a simplified ground trafficmanagement task. In both experiments, the environmental complexity factorswere predominant; however, in the case of the timeline task, the use of color inthe displays added to, instead of reducing, environmental complexity.

In the enroute experiment, the environmental complexity source of changingairspace was far more significant in influencing overall controller performance ascompared to the complexity display factor caused by visual routes. These resultssupport air traffic controllers’ subjective opinions that SUA is a source of com-plexity, and that more work is needed for better display representation. Because of


Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

the temporal and cyclic nature of SUA, possible design interventions could includesome kind of timeline display for SUA scheduling, as well as intelligent decisionsupport agents that can predict in advance when airspace could become availableor deactivated, and possible courses of action.

In the ground traffic management task, investigation of environmental and dis-play sources of complexity revealed that increasing numbers of planes affectedparticipant performance only slightly more than color categories; however, trafficarrival sequence patterns were not as strongly associated with degraded perfor-mance. Although aircraft count is a known primary contributor to cognitive com-plexity, this research demonstrates that the increasing use of color is a likelysource as well. Color has been shown to aid operators in search and organizationtasks, but this experiment generated evidence that beyond six color categories,performance was not aided and overall task performance degraded through an in-crease in errors of omission. These research results highlight the importance of un-derstanding sources of complexity in human supervisory control domains.Displays are the primary visual mechanisms through which controllers build men-tal models, but as demonstrated in this study, even a simple addition of color cate-gorization can significantly influence controller cognitive complexity.

The proposed components of environmental, organizational, and display com-plexity may not contribute in a linear, additive, or consistent manner to either cog-nitive complexity or performance and will likely be task and technologydependent. As new technologies and procedures are introduced in ATC and man-agement tasks, if the different sources of complexity are not well understood, it islikely that the interventions themselves could add to overall complexity and not re-duce it, as evidenced by our color category results. Thus it is important to be able topartition the sources of complexity and measure their impact so that areas for pos-sible design interventions can be identified. Once the potential design interven-tions are identified and implemented, further testing can compare the new resultsto the earlier complexity tests to determine if the perceived complexity on the partof the operator has actually been mitigated.

ACKNOWLEDGMENTS

This research was sponsored by the Civil Aerospace Medical Institute. We thankDaniel Crestian Cunha, a visiting student from Instituto Tecnologico deAeronautica, as well as Kathleen Voelbel for their assistance.

REFERENCES

Ahlstrom, U., Rubinstein, J., Siegel, S., Mogford, R. H., & Manning, C. (2001). Display concepts for enroute air traffic control (DOT/FAA/CT–TN01/06). Washington, DC: U.S. Department of Trans-portation.


Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

Athenes, S., Averty, P., Puechmorel, S., Delahaye, D., & Collet, C. (2002, October). ATC complexityand controller workload: Trying to bridge the gap. Paper presented at HCI–Aero, Cambridge, MA.

Atkins, S., Jung, Y., Brinton, C. R., Stell, L., Carniol, T., & Rogowski, S. (2004, September). Surfacemanagement system field trial results. Paper presented at the AIAA 4th Aviation Technology, Inte-gration, and Operations (ATIO) Forum, Chicago.

Brinton, C. R., Carniol, T., Butler, T., Duley, J., & Hitt, J. M. (2002). Air traffic management system de-velopment and integration (ATMSDI) acquisition CTO–05—Surface management system (SMS)CTOD 5.11–1: Phase 1 report. Moffett Field, CA: Raytheon.

Cardosi, K., & Hannon, D. (1999). Guidelines for the use of color in ATC displays(DOT/FAA/AR–99/52). Washington, DC: Office of Aviation Research.

Christ, R. E. (1975). Review and analysis of color coding research for visual displays. Human Factors,7, 542–570.

Cummings, M. L., & Guerlain, S. (2004, March). Using a chat interface as an embedded secondarytasking tool. Paper presented at the 2nd Annual Human Performance, Situation Awareness, and Au-tomation Conference, Daytona Beach, FL.

Edmonds, B. (1999). What is complexity? The philosophy of complexity per se with application to someexamples in evolution. In F. Heylighen & D. Aerts (Eds.), The evolution of complexity (pp. 1–18).Dordrecht, The Netherlands: Kluwer.

Hansman, R. J., & Davison, H. J. (2000, June). The effect of shared information on pilot/controller andcontroller/controller interactions. Paper presented at the 3rd USA/Europe Air Traffic ManagementR&D Seminar, Napoli, Italy.

Hilburn, B. (2004). Cognitive complexity in air traffic control: A literature review (EEC Note No.04/04). Bretigny-sur-Orge, France: Eurocontrol.

Histon, J. M., Hansman, R. J., Gottlieb, B., Kleinwaks, H., Yenson, S., Delahaye, D., et al. (2002,October). Structural considerations and cognitive complexity in air traffic control. Paper presentedat the 19th IEEE/AIAA Digital Avionics Systems Conference, Irvine, CA.

Jeffrey, T. E., & Beck, F. J. (1972). Intelligence information from total optical color imagery. Washing-ton, DC: U.S. Army Behavior and System Research Laboratory.

Kirwan, B., Scaife, R., & Kennedy, R. (2001). Investigating complexity factors in UK Air TrafficManagament. Human Factors and Aerospace Safety, 1, 125–144.

Kopardekar, P. (2003, June). Measurement and prediction of dynamic density. Paper presented at theATM 2003 R&D Seminar, Budapest, Hungary.

Majumdar, A., & Ochieng, W. Y. (2002, January). The factors affecting air traffic controller workload:A multivariate analysis based upon simulation modeling of controller workload. Paper presented atthe 81st Annual Meeting of the Transportation Research Board, Washington, DC.

Miller, C. (2000). The human factor in complexity. In T. Samad & J. Weyrauch (Eds.), Automation, con-trol, and complexity: New developments and directions (pp. 35–57). New York: Wiley.

Pawlak, W. S., Brinton, C. R., Crouch, K., & Lancaster, K. M. (1996, July). A framework for the evalua-tion of air traffic control complexity. Paper presented at the AIAA Guidance Navigation and ControlConference, San Diego, CA.

Rensink, R. A., O’Regan, J. K., & Clark, J. J. (1997). To see or not to see: The need for attention to per-ceive changes in scenes. Psychological Science, 8, 368–373.

Reynolds, L. (1994). Colour for air traffic control displays. Displays, 15, 215–225.Simon, H. (1996). The sciences of the artificial (3rd ed.). Cambridge, MA: MIT Press.Simons, D. J. (2000). Attentional capture and inattentional blindness. Trends in Cognitive Sciences, 4,

147–155.Spencer, A., Smith, P., Billings, C., Brinton, C. R., & Atkins, S. (2002, October). Support of traffic man-

agement coordinators in an airport air traffic control tower using the surface management system.Paper presented at the International Conference on Human–Computer Interaction in Aeronautics,Cambridge, MA.


Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

Tsonis, C., Cunha, D. C., & Histon, J. (2005, September). Developing visualization tools for the designof air traffic control experiments. Paper presented at the 49th Annual Meeting of the Human Factorsand Ergonomics Society, Orlando, FL.

Wickens, C. D., & Hollands, J. G. (2000). Engineering psychology and human performance (3rd ed.).Upper Saddle River, NJ: Prentice-Hall.

Woods, D. D. (1991). The cognitive engineering of problem representations. In G. R. S. Weir & J. L.Alty (Eds.), Human–computer interaction and complex systems (pp. 169–189). London: Taylor &Francis.

Xing, J., & Schroeder, D. J. (2006). Reexamination of color vision standards: 1. Status of color use inATC displays and demography of color-deficit controllers (DOT/FAA/AM–06/2) Washington, DC:Federal Aviation Administration.

Manuscript First Received: February 2005


Dow

nloa

ded

by [

Geo

rgia

Tec

h L

ibra

ry]

at 0

3:02

13

Nov

embe

r 20

14

partitioning complexity in air traffic management tasks

Documents