attachment 1 to rma hdbk-006c capacity, delay, and availability categorization for … · 2015. 11....

FAA Attachment 1 to RMA HDBK-006C Capacity, Delay, and Availability Categorization for NAS

Services and Systems

Version 1.17

U.S. Department of Transportation

Federal Aviation Administration

Attachment 1 to RMA HDBK-006C

Capacity, Delay, and Availability

Categorization for NAS Services and Systems

November 19, 2015

Version 1.17

Select Sections from ANG-B White Paper

Federal Aviation Administration

800 Independence Avenue, SW

Washington, DC 20591



Version 1.17

2

ABSTRACT

Currently National Airspace System (NAS) requirements are assigned to one of four availability

categories based on the criticality of the service the requirement supports. However there is no

method for delineating the criticality between Efficiency-Critical and Essential NAS services.

Additionally NAS level requirements are overly restrictive by designating NAS services as

Efficiency-Critical for the entire NAS. A primary focus of this white paper is to present a

methodology for delineating Essential and Efficiency-Critical service level requirements on a

site by site or area by area basis.

The original release of this whitepaper also examined enterprise Reliability, Maintainability,

Availability (RMA) aspects of leased NAS services as well as the use of FAA-HDBK-006B

Availability categories to analyze the consequences of major outages caused by unpredictable

events.



Version 1.17

3

Table of Contents

INTRODUCTION ............................................................................................................. 4

UNDERSTANDING THE PROBLEM ........................................................................... 5

2.1 Contributors to Availability ................................................................................................ 6

RMA RELATED NAS FACILITY CHARACTERISTICS AFFECTING

CAPACITY ................................................................................................................................ 9

3.1 Role of Demand ................................................................................................................ 10

3.1.1 Role of Delay ................................................................................................................. 11

3.1.2 Economic Impact ........................................................................................................... 13

3.1.3 Other Contributing Factors ............................................................................................ 13

ENTERPRISE AND LEASED SERVICES .................................................................. 14

USING AVAILABILITY HIERARCHY TO STUDY VULNERABILITIES .......... 15

CONCLUSIONS AND RECOMMENDATIONS ........................................................ 16

6.1 Preliminary Economic Impact Graphs .............................................................................. 17

References ......................................................................................................................... 19

Appendix A DEFINITIONS ............................................................................................... 20

Appendix B APPLICABLE DOCUMENTS ..................................................................... 21

B.1 Specifications, Standards, and Handbooks ....................................................................... 21

Appendix C ATCT DATA FOR CLUSTER ANALYSIS ................................................ 22

Appendix D LEASED FLIGHT SERVICE PERFORMANCE MEASURES ............... 24

Appendix E ECONOMIC COST FACTORS ................................................................... 25



Version 1.17

4

INTRODUCTION

This whitepaper discusses methodologies for prescribing appropriate and achievable Reliability,

Maintainability, Availability (RMA) requirements for the National Airspace System (NAS)

Enterprise and individual Air Traffic Control (ATC) systems and facilities. This whitepaper

release only covers one of the three original focus areas; methods for delineating Essential and

Efficiency-Critical criticality designations for NAS services, and relating them to impacts on

NAS capacity.

.



Version 1.17

5

UNDERSTANDING THE PROBLEM

The NAS Requirements Document (NAS RD-2013) defines Capacity as: “The number of aircraft

that can be accommodated in a given time period by the system or one of its components.” The

RD does not impose any particular capacity metrics, but does designate numerous requirements

associated with maintaining capacity as “Efficiency-Critical”.

This definition of Capacity implies that a system or facility operating at or below that limit

imposes no delay on aircraft. Delays resulting from lack of capacity are a major economic factor

in airline operations and are well documented. Proper selection of systems, infrastructure, and

system reliability can reduce the contribution of NAS Services to delays due to capacity

reductions.

The NAS is comprised of a complex network of ATC facilities servicing unevenly distributed

air traffic demand. Since the traffic is non-uniform, some ATC facilities are more critical than

others. A loss of service at one facility may have a detrimental effect on the NAS, while a loss at

another may be trivial. The following are NAS-RD criticality categories used to classify the

consequences of service reduction or loss on NAS operations:

Safety Critical – A key service in the protection of human life. Loss of a Safety-Critical

service increases the risk in the loss of human life.

Efficiency Critical– A key service that is used in present operation of the NAS. Loss of

an Efficiency Critical Service has a major impact in the present operational capacity.1

Essential – A service that if lost would significantly raise the risk associated with

providing efficient NAS operations.

Routine – A service which, if lost, would have a minor impact on the risk associated with

providing safe and efficient NAS operations.

NAS-RD criticality categories are assigned based on the consequence of risk incurred when a

NAS service is lost. This risk is assessed during the transition period that occurs when a

service is lost. During this period, ATC personnel transition to pre-established manual

procedures for the service reducing capacity. This transition period is illustrated by Figure

2-1.

During the transition period an increased risk to either safety or capacity can occur. Safety risk

depends on whether service loss poses an increased risk to loss of life. If so, the service is

designated Safety-Critical and the system and infrastructure supporting the service must

require a high availability. The impact of a service loss on efficiency determines the severity

of a loss of service and determines the designated NAS criticality category.

Multiple factors can be used to assess the severity of a loss of service including the air traffic

demand at the facility or area where the outage occurs and the intricacies of operations,

airspace complexity, and regional dependencies. When delineating Essential and Efficiency-

Critical services the potential for delay propagation needs to be assessed.

1 In the RMA context, Efficiency-Critical is generally considered to be a category of service loss which impacts the

NAS over a wide area.



Version 1.17

6

Figure 2-1 Effect of Service Interruptions on NAS Capacity

2.1 Contributors to Availability

The availability of NAS services depends on the availability of multiple contributing local

systems and external services. In preparation for the deployment of the Standard Terminal

Automation Replacement System (STARS) in the TRACON, Futron Corporation prepared a

study of TRACON availability risk factors [2]. In this study Futron presented an approach for

deriving “Airspace Vulnerability” and “Workload Vulnerability” measures from availability

parameters of the systems and services utilized in the TRACON.



Version 1.17

7

Level 1

Level 3

Level 3

Level 6

Level 2

Level 5

Level 2Level 1

STARS

Workstation

Radar

Power

Data

Comm

Radar

Common

ASR-9 ICSS

FSL

TRACON

Power

FDIO

PRM

Mode-S

Voice

Comm

Repair time + 1 minute 5 min

ESL Mode

ESL Mode

Enroute Radar

No Radar

** AVI contribution

incorporated into

S10 result

0% 0%

66/390 0

S3

0% 25%

66/390 0.62

S1

0% 10%

66/390 0.20

S4

30% 25%46/27

0.09 0.17

S2

30% 30%46/27

0 0

S6

50% 10%33/20

2.84 0

S5

85% 100%

10/62.58 0.07

S11

85% 100%

10/60 0

S11

30% 30%

46/27** 0.01

S10.1

25% 10%50/29

0.90 0.13

S13

25% 10%50/29

0.06 0.07

S14

25% 10%50/29

0.39 0.07

S15

85% *10/6

0 0

S16

Full

Service

66/39

Full

Service

Legend:

S: State Identif ier

1: % Aircraft Reduction

2: % Workload Increase

3: Arrivals per Hour

4: Airspace Capacity

4: AVI Result

5: WKI Result

1 2

3/45 6

S

STARS Scenario Values

Scenario Characteristic Value

Functional Level 5 (severe impact) Reduction of traffic in airspace 85% Aircraft Capacity (# a/c) 6 Time to reach new state (min) 29 Time to identify failure (min) Immediately Increase in workload 100% Time to implement new procedures (min) Immediately

Figure 2-2 TRACON State Transition Diagram Template for Separation Services



Version 1.17

8

Figure 2-2 shows Futron’s availability model for an STARS equipped TRACON. Futron’s study

identified multiple service loss contingencies, shown as “Levels” in the diagram.

The Futron study addressed safety, capacity, and contingencies, but only at two airports. The

methodology of that study is extremely useful and should be recommended as an approach to

modeling future Safety-Critical innovations in the NAS. However, for initial studies of capacity

impacts on the NAS as a whole a simpler approach is needed. Section 3 below presents such an

approach.



Version 1.17

9

RMA RELATED NAS FACILITY CHARACTERISTICS AFFECTING

CAPACITY

Assessing if a loss of a service is a risk to safety is relatively straightforward. However there is

not an established methodology to determine if loss of a service poses an increased risk to

efficiency or if a service loss qualifies as a significant impact to NAS efficiency. A NAS service

is designated Efficiency-Critical, if losing the service leads to widespread delays and results in

significant economic consequences. The question remains, what magnitude of impact justifies

designating a service as Efficiency-Critical?

Because of the limited impact of service outages at facilities with low demand, those services can

be considered essential. But at the busiest facilities or regions in the NAS, loss of a service can

lead to severe delays and disruption across major portions of the NAS. Loss of services at these

facilities leads to widespread flight delays and cancellations, resulting in severe economic

consequences.

Additionally, similar problems exist for the NAS, as FAA engineers and managers face major

cost-benefit and life cycle decisions when specifying characteristics of individual system

instances and the NAS enterprise services they support. Figure 3-1 illustrates the differences in

systems, system availability, and infrastructure deployed among three typical terminal facilities,

representing low, medium, and heavy traffic volume areas.

Figure 3-1 Availability Related Parameters for Terminal Facilities with Varying Demand



Version 1.17

10

This section will show how facilities criticality categories could be determined based on

published or derivable metrics to assist decisions regarding appropriate availability requirements.

Candidate metrics include:

Operations count

Delays

Economic impact

The following sections discuss quantifiable parameters that may be utilized to “scale” the various

facilities in the NAS; thereby providing a means to differentiate between Essential and

Efficiency-Critical services provided by these facilities.

3.1 Role of Demand

Perhaps the most obvious metric to assess the severity of a loss of service is demand. This study

examines airport demand, measured by the number of operations at an airport during the Fiscal

Year (FY) of 2013. Airport operations count varies dramatically as shown in Figure 3-2 and

provides some insight into the volume of traffic that would be immediately affected if a service

failure occurs. A service thread failure can result in delays, flight diversions and flight

cancellations. However, the situation is often exacerbated when these impacts propagate to other

facilities. Facilities with a higher operations count intuitively have a higher potential impact and

as such will necessitate more stringent RMA requirements.



Version 1.17

11

Figure 3-2 ATCT Itinerant Operations2

3.1.1 Role of Delay

Delays often propagate through a network of airports, compounding their effect. Utilizing delay

propagation multipliers produced by the Aviation System Performance Metrics (ASPM) the

relationship between original and propagated delay of airports can be depicted in Figure 3-3

below [3].

Figure 3-3 Original vs Propagated Delay

Delay propagation is a key parameter that can assist in differentiating between Efficiency-

Critical and Essential criticalities. This parameter can not only be quantified in terms of time, but

also in the number of facilities affected. Consequently, delay propagation provides an

understanding of how extensive a service failure at a particular facility can cause to the NAS.

Naturally, service threads are more critical in ATC facilities that would have a greater

propagated effect, such as large hubs and therefore would merit higher availability.

2 “Itinerant Operations” are defined as operations originating or terminating at another facility, as opposed to “Local

Operations” which originate and end within a single facility’s airspace.



Version 1.17

12

Figure 3-4 Projected Service Loss Impact at Major Airports

To provide a basis for a future classification to rank the potential impact of a facility on NAS

efficiency, a categorization was developed. Airports were categorized according to their impact

to NAS efficiency. Data was partitioned using a k-means clustering algorithm. The output of the

algorithm converged to three clusters using the squared Euclidean distance measure. Results of

the analysis are shown above in Figure 3-4. Select facilities are annotated within Figure 3-4 and

a full list of facilities is provided in Appendix C. The mean of each cluster or centroid is shown

for each of the 3 regions. Future economic studies could assess the projected economic impact of

each region using the centroids as test points.

Figure 3-4 illustrates how potential NAS service loss impacts may be derived from an

assessment of the parameters of interest. The relationship between facility demand and

anticipated delay propagation indicates where the potential for severe disruption to NAS

efficiency due to a service loss is highest. For facilities with high operations count and the

potential for large delay propagation, loss of services is anticipated to have more significant

impact on overall NAS efficiency. Facilities within the low region are not as critical to

maintaining overall NAS efficiency.



Version 1.17

13

3.1.2 Economic Impact

The cost of delay is normally included in FAA cost benefit analyses. These calculations are

detailed and usually particular to specific systems and circumstances. The FAA’s review of the

Chicago Center Fire [5] states: “The fire at Chicago Center and it’s after effects inconvenienced

passengers; reportedly cost the airlines over $350 million dollars…”

The report does not go into any detail of this cost calculation, but appears to explicitly exclude

any non-airline costs. Traditionally FAA cost/benefit calculations include passengers’ value of

time (PVT) as well (but not secondary effects such as canceled hotel and rental car bookings, or

loss of economic value of the business traveler’s intended work).

The Department of Transportation (DOT) requires airlines to file delay reports, part of which is

their accounting of delay costs which Airlines for America reports at $8,064,000,000 for CY

2013 [6]. ANG-5 performs economic and cost benefit analyses for FAA programs and

represents a resource for programs requiring specific facility or service analyses. Early cost

model development should inform the RMA analysis of architectural alternatives for new

systems.

3.1.3 Other Contributing Factors

A primary contributor to delay in the NAS is extreme or severe weather conditions [7]. For

weather related services, the impact of weather on delay propagation on criticality should be

considered.

It is interesting to note that in Figure 3-4 the airports in the Moderate cluster divide into two

populations along the lower left – upper right diagonal. This is indicative of higher delay impact

on those below the line. In every case the airports below the line are those with more severe

weather conditions. This would indicate that these facilities would be better candidates for

systems that mitigate weather related delays.



Version 1.17

14

ENTERPRISE AND LEASED SERVICES

Deleted for the purposes of this reprint.



Version 1.17

15

USING AVAILABILITY HIERARCHY TO STUDY

VULNERABILITIES

Deleted for the purposes of this release.



Version 1.17

16

CONCLUSIONS AND RECOMMENDATIONS

This white paper has studied a number of aspects of the impact of RMA on NAS capacity related

metrics. Propagated Delay was found to be a useful metric which can be calculated on a facility

basis and relates directly to airline and passenger costs. The use of cluster analysis to group

facilities according to their potential to cause enterprise-wide impact on NAS capacity was a

useful discriminator between Essential and Efficiency-Critical status for services and systems on

a per-facility basis. Recommendations arising from this study

Calculation of the Delay Propagation multipliers should be updated and extended to

additional airports including all focus airports. Propagated delay is calculated yearly, but

the multipliers are based on 2009 data.

Figure 6-1 Propagated Delay for Top 50 Airports and Extrapolation for the Next 4653

Figure 6-1 shows our extrapolation of the contribution of 465 airports below the top 50’s

contribution to total propagated delay. As can be seen it is non negligible and could

reward further study, particularly in completion of analysis of the Focus Airports.

3 Estimate for 465 airports made on basis of 2nd order polynomial fit to 50 airport data in Figure 6-1 constrained at

0,0.



Version 1.17

17

Additional modeling and simulation should be conducted to better understand factors

influencing delay propagation to increase the use of this metric in determine NAS

efficiency critical decisions.

6.1 Preliminary Economic Impact Graphs

Based on preliminary reviews of this paper the authors were asked to work with ANG-5 to

develop a set of graphs illustrating the rough economic impact of the total outage of a NAS

facility for use in right-sizing decisions.

Figure 6-2 Preliminary ATCT Economic Impact Graph

Figure 6-2 illustrates the impact of total outage of ATC Services at a wide range of ATCT

serviced airports. This graph is based on the same data used in Figure 3-4, broken down by

month over a five year period. Economic cost was calculated based on a one month outage using

the tables in Appendix E, and includes aircraft operator’s direct operational costs (ADOC) and

passenger’s value of time (PVT); indirect costs are not included.

ANG-B7 does more detailed economic analyses for cost benefit studies of particular program

benefits, the intent of these graphs is strictly as guidance to engineers in making RMA related

decisions.



Version 1.17

18

Figure 6-3 Preliminary TRACON Economic Impact Graph

Figure 6-3 is based on the same data as Figure 6-2, but is aggregated into TRACON impacts by

adding up the major airports serviced by a TRACON, and by averaging the impact over the full

five year period. As in Figure 3-4, we have clustered the data to show major economic impact

groupings.

As stated previously, these graphs are for total outages (ATC Zero). For measuring the economic

impact of individual services or systems it would be necessary to apply the Futron methodology

illustrated in Figure 2-2 to estimate the individual contribution of a system or service to

availability.



Version 1.17

19

References

All references used through this document are listed in Table 7-1.

Table 7-1 References

Reference # ID Title

[1] FLIGHT RADAR

Flightradar24. (2014). Flightradar24 Live Air Traffic.

Retrieved September 26, 2014, from

http://www.flightradar24.com

[2] FUTRON Joseph P. Carrigan and Clayton A. Smith, TRACON Risk

Study, Futron Corporation, June 2000

[3] ASPM

Federal Aviation Administration. (2009). FAA Operations

& Performance Data. Retrieved December 2, 2014, from

Federal Aviation Administration :

https://aspm.faa.gov/aspm/ASPMframe.asp

[4] NAS-RD

Federal Aviation Administration. (2012). National

Airspace System Requirements Document. District of

Columbia: Department of Transportation.

[5] ZAU FIRE

(2014). Chicago Center Fire Contingency Planning and

Security Review . District of Columbia : Federal

Aviation Adminstration .

[6] AIRLINES

Airlines for America . (2014). Airlines for America.

Retrieved December 10, 2014, from

http://airlines.org/data/per-minute-cost-of-

delays-to-u-s-airlines/

[7] DOT

Department of Transportation . (2012, March 2). United

States Department of Transportation . Retrieved

December 8, 2014, from Bureau of

Transportation Statistics :

http://www.transtats.bts.gov/OT_Delay/ot_delay

cause1.asp?type=3&pn=1

[8] WRIGHT Wright, G. (2014, September 28). System Analysis

Recording Data Imaged. Arlington , Virginia .

[9] RMA HDBK

Federal Aviation Administration. (2014). Reliability,

Maintainability, and Availability (RMA)

Handbook. District of Columbia : Federal

Aviation Administration .

[10] MITRE

Welman, S., Williams, A., & Hechtman, D. (2010).

Calculating Delay Propagation Multipliers for

Cost-Benefit Analysis. McLean: MITRE.

[11] CONTRACT

Federal Aviation Administration . (2007). Controls Over

the Federal Aviation Administration's

Conversion of Flight Service Stations to

Contract Operations . District of Columbia :

Federal Aviation Administration .

https://aspm.faa.gov/aspm/ASPMframe.asp



Version 1.17

20

Appendix A DEFINITIONS

Capacity: The number of aircraft that can be accommodated in a given time period by the system

or one of its components.

Delay Propagation: Delay that has been transmitted over more than a single flight leg for a given

aircraft itinerary.

Service threads: Service threads are strings of systems/functions that support one or more of the

NAS EA Functions. These service threads represent specific data paths (e.g. radar surveillance data)

to air traffic specialists or pilots.

Severity: A relative measure of the consequence of a failure mode, sensitivity to outage downtime,

and its frequency of occurrence.



Version 1.17

21

Appendix B APPLICABLE DOCUMENTS

B.1 Specifications, Standards, and Handbooks

1. NAS-RD-2012

2. NAS-SR-1000

3. NAS-HDBK-006B



Version 1.17

22

Appendix C ATCT DATA FOR CLUSTER ANALYSIS

ATCT Delay Propagation

(Minutes)

Total

Operations NAS Impact Category

ORD 3,144,148 596,056 Extreme

ATL 2,563,861 785,306 Extreme

DFW 1,822,954 556,682 Extreme

SFO 1,758,539 328,688 Extreme

DEN 1,498,391 445,142 Moderate to Severe

LAX 1,144,325 440,120 Moderate to Severe

EWR 1,134,614 235,472 Moderate to Severe

IAH 881,101 358,997 Moderate to Severe

LGA 839,446 194,529 Moderate to Severe

MDW 827,028 174,969 Moderate to Severe

BWI 814,785 199,012 Moderate to Severe

MCO 762,074 228,359 Moderate to Severe

JFK 722,772 210,839 Moderate to Severe

LAS 675,746 276,406 Moderate to Severe

DTW 663,513 292,936 Moderate to Severe

BOS 662,621 213,087 Moderate to Severe

PHX 613,164 352,298 Moderate to Severe

CLT 553,966 283,508 Low to Moderate

PHL 542,573 158,467 Low to Moderate

FLL 538,444 133,356 Low to Moderate

IAD 534,594 145,882 Low to Moderate

HOU 533,127 114,626 Low to Moderate

DAL 465,544 92,255 Low to Moderate

BNA 458,512 114,091 Low to Moderate

SAN 455,507 151,362 Low to Moderate

MSP 449,489 272,685 Low to Moderate

SLC 438,205 223,042 Low to Moderate



Version 1.17

23

ATCT Delay Propagation

(Minutes)

Total

Operations NAS Impact Category

STL 418,748 113,068 Low to Moderate

TPA 412,627 131,468 Low to Moderate

MCI 396,061 99,874 Low to Moderate

MIA 387,486 159,944 Low to Moderate

SEA 383,375 202,182 Low to Moderate

DCA 377,724 144,194 Low to Moderate

AUS 344,568 92,776 Low to Moderate

CLE 335,439 90,666 Low to Moderate

RDU 319,403 93,056 Low to Moderate

MKE 275,559 70,597 Low to Moderate

MSY 273,155 83,394 Low to Moderate

SAT 267,906 79,512 Low to Moderate

HNL 253,246 106,358 Low to Moderate

PDX 244,955 104,493 Low to Moderate

OAK 244,417 88,085 Low to Moderate

PIT 237,395 67,299 Low to Moderate

SMF 232,893 84,680 Low to Moderate

CMH 223,172 56,124 Low to Moderate

IND 195,779 59,241 Low to Moderate

CVG 194,759 75,834 Low to Moderate

PBI 166,065 44,314 Low to Moderate

MEM 149,317 66,669 Low to Moderate

SNA 140,279 79,461 Low to Moderate

FAA Attachment 1 to RMA HDBK-006C Capacity, Delay, and Availability Categorization for NAS Services and Systems

Version 1.17

24

Appendix D LEASED FLIGHT SERVICE PERFORMANCE MEASURES 1

2

3

4

Deleted for the purposes of this reprint.5

FAA Attachment 1 to RMA HDBK-006C Capacity, Delay, and Availability Categorization for NAS Services and Systems

Version 1.17

25

Appendix E ECONOMIC COST FACTORS 6

The following Economic Cost Tables provided by ANG-5 were used as input to the calculations in Figures 6-2 and 6-3: 7

8

FY13$ Cancellations*

(ADOC) Diversions

(ADOC)**

Cancellations*

(PVT)

(5 Hours)

Diversions

(PVT)**

Air Carrier - Passenger $ - $ 19,911 $ 20,412 $ 20,412

Air Carrier - Cargo $ 5,611.27 $ 15,345 N/A N/A

Air Carrier - TAF $ 383 $ 19,599 $ 19,107 $ 19,018

Air Taxi - TAF $ - $ 5,919 $ 6,235 $ 6,733

General Aviation3 TBD N/A N/A N/A

Military4 TBD N/A N/A N/A

* Cancellations ADOC & PVT still under research

** Diversions based on ASQP (BTS Airlines) Data (AC -PAX = 5 hrs, AC-Cargo = 2 hrs, AT-TAF = 5.4 hrs)

9

10

attachment 1 to rma hdbk-006c capacity, delay, and availability categorization for … · 2015. 11....

Documents