care/asas activity 3: airborne separation minima ... · relationship between hazard brainstorm and...

CARE/ASAS Activity 3: Airborne Separation Minima: Extension Study ASAS in CARE

CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002

CARE/ASAS Action

CARE/ASAS Activity 3:Airborne Separation Minima:

Extension Study

Lessons learntfrom the OSED/OHA activities

and hazard brainstorm


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page i

DOCUMENT REVIEW

Version Date Description of evolution Modifications

0.1 11/03/02 Initial report structure and objectivesfor Kick-Off meeting

All

0.2 22/03/02 Integration of comments from Kick-off meeting All

0.3 05/04/02 Integration of comments from NLR dated 29/03/02,Analysis the OSA methodology applied to ASAS

applications

Sections 1 and 7,Section 2 and

Annex A

0.4 19/04/02 Analysis of the OSED/OHA steps performedwithin the ASM study, by Sofréavia and EECfollowing audio-conference held on 03/04/02

Section 3

0.5 30/04/02 Analysis of hazard brainstorm results by NLR,Update following comments from partners during

the audio-conference held on the 23/04/02

Section 4 andAnnex F,

Sections 2 and 3 andAnnex B

0.6 17/05/02 Updates following internal review and discussionsduring the audio-conference held on the 14/05/02

Sections 3 and 4,Annex D deleted

0.7 31/05/02 Updates following comments from Progress ReportMeeting held on the 24/05/02,

Relationship between hazard brainstorm and OSAmethodology

All,Annexes C, D and E

deleted

0.8 04/05/02 Integration of updated Analysis of hazard brainstormresults by NLR,

Executive summary and conclusions

Sections 4, 6 and 7

1.0 07/06/02 Editorial changes following partners review,Proposed issue of the final report

All

1.1 17/09/02 Minor changes following CARE-ASASManagement Board comments

Sections 1, 2,3, and4


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page ii

CONSORTIUM

Representatives Organisations

Authors Anne Cloerec,Béatrice Raynaud

SOFREAVIA

Authors Eric HoffmanFranck-Olivier Ripoll

EEC

Authors Henk Blom,Mariken Everdij

NLR

Reviewer Colin Goodchild University Of Glasgow

Reviewers Jean-Marc Loscos,Thierry Miquel

CENA


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page i

Executive summary

Background and context

The CARE/ASAS Activity 3: ASM projects consisted of the investigation of safetyinfluencing factors that may affect airborne separation minima.

In that perspective, the use of the RTCA/EUROCAE OSA guidelines, at least the initialsteps of OSED and qualitative OHA, were investigated on two selected ASAS applications.As part of the TOPAZ methodology, a hazard brainstorm was also organised to identifyadditional hazards that were due to the mitigation means identified in the autonomousoperations OHA.

Based on the experience gained through the ASM study, this extension study aims atdeveloping lessons learnt from the OSED/OHA activities and the hazard identificationbrainstorm.

Analysis of the OSA methodology for ASAS applications

Fisrt, the characteristics of ASAS applications; the type of safety assessment required forASAS applications; and the relevance of the OSA guidelines for ASAS applications arediscussed, and lessons learnt from the use of the OSA guidelines in the ASM study areproposed.

In the perspective of performing an end-to-end safety assessment, the RTCA/EUROCAEOSED/OSA guidelines were found particularly relevant for ASAS applications. However,some amendments would be required to the better address the characteristics and safetyissues of these ASAS applications.

In particular, the OSED guidance material should be amended to allow the adequatedescription of the airborne CNS and ASAS characteristics, as well as the operational use ofthese ASAS applications. The Hazard Classification Matrix (HCM) developed by theRTCA/EUROCAE should be amended with proper definition of the safety marginsassociated with ASAS operations. And, appropriate ASOR guidance material should bedeveloped to help people taking into account the essential role of airborne separationcriteria when determining the safety performance requirements.

Analysis of the OSED/OHA steps for identifying safety-influencing factors

A critical analysis of the OSED/OHA approach performed during the ASM study in theperspective of identifying safety-influencing factors is performed, and lessons learntthrough the OSED/OHA performed in the ASM study are developed.

The main OSED items discussed are the operating environment characteristics, theoperating method with and without ASAS, the operational service description, and thefunctional characteristics description. Lessons learnt from the OSED developmentperformed in the ASM study would be:

• to support an iterative approach for the OSED development taking into account thedevelopment and validation stage of the ASAS application;

• to better adapt the OSED information depending on the ASAS application category;


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page ii

• to better adapt the type of information and level of detail in the OSED to the objectivesof the safety assessment.

The main OHA items analysed are the failure scenarios either related to technical or humanerrors, the relationship between failure scenarios and operational hazards (OHs), therelationship between OHs and phases of operations, the development of the risk mitigationstrategy (on qualitative basis), and the implicit relationship between OHs.

Lessons learnt from of both OHA performed in the ASM study would be:

• to combine both top-down approach (deductive) and bottom-up approach (inductive)within an iterative OHA process including both qualitative and quantitative assessment;

• to better determine and validate, through the OHA, the additional mitigation means thatactually need to be put in place;

• to better assess the impact of human factors in ASAS operations;

• to allow effective clustering of adverse scenarios, and identification of relevant safetyinfluencing factors, at different stages of development of the ASAS applications.

Analysis of hazard identification brainstorm results

The hazards from the different sources (i.e. OHA and hazard brainstorm) have beencompared on two aspects: types of individual hazards and overlap between group ofhazards.

The example results of the ASM hazard type and overlap analysis indicate that theTOPAZ-based brainstorms and the OHA’s have identified hazards that werecomplementary to each other. In particular, hazards that will not easily be identified withan OHA approach, i.e. OHA-unimaginable hazards, are typically the hazards that go-beyond the functional level of human tasks. However, the results of such a brainstorm is alist of hazards that are not analysed and clustered. A further analysis and clustering couldbe necessary after the brainstorm is finished. On the other side, hazard identificationthrough brainstorming can be done at a very early stage of the operational conceptdevelopment process.

Relationship between the OSA methodology and brainstorm based hazardidentification

Based on the lessons learnt from the CARE-ASAS ASM study, an integrated and iterativeapproach for the safety assessment of ASAS applications, based on the RTCA/EUROCAEOSA methodology and the use of hazard brainstorm, is proposed.

This approach supports the iterative assessment of operational hazards related to the ASASapplication, with early identification through brainstorm and stepwise analysis of hazards atdifferent level of detail. Successive OHA steps would consist in the identification andseverity assessment of operational hazards; the preliminary frequency assessment ofoperational hazards, and finally the assessment of failure conditions leading to operationalhazards.

Along this iterative process, any refinement of the OSED, related or not to some previoussafety assessment, would imply another iteration in the safety assessment of the ASASapplication.


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page iii

Recommendations

On the basis of the overall work performed during the CARE-ASAS ASM study, thefollowing recommendations need to be considered to support future work related toOperational Safety Assessment of ASAS applications:

• RTCA/EUROCAE Operational Safety Assessment (OSA) guidelines should be adaptedto better address the characteristics of ASAS applications;

• Guidelines should be developed to support an iterative development of the OSED of anASAS application along its development life cycle and safety assessment;

• Guidelines should be developed to support an iterative OHA process combining bothtop-down (or deductive) and bottom-up (or inductive) approach, as well as severity andfrequency assessment.


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page iv

Table of Content

1. INTRODUCTION.................................................................................... 11.1. DOCUMENT OBJECTIVES .....................................................................................11.2. BACKGROUND AND CONTEXT...........................................................................1

1.2.1. CARE/ASAS ACTIVITY 3 ...............................................................................11.2.2. CARE-ASAS ACTIVITY 3: THE ASM STUDY..................................................21.2.3. OTHER RELATED WORK IN EUROPE..................................................................41.2.4. FAA/EUROCONTROL ACTION PLAN 1 COORDINATION ....................................4

1.3. DOCUMENT OVERVIEW .......................................................................................4

2. ANALYSIS OF OSA METHODOLOGY FOR ASASAPPLICATIONS...................................................................................... 62.1. THE OSA METHODOLOGY ...................................................................................62.2. APPLICABILITY OF THE OSA METHODOLOGY TO ASAS

APPLICATIONS........................................................................................................72.2.1. CHARACTERISTICS OF ASAS APPLICATIONS....................................................72.2.2. WHICH SAFETY ASSESSMENT FOR ASAS APPLICATIONS..................................92.2.3. RTCA/EUROCAE OSA GUIDELINES APPLIED TO ASAS

APPLICATIONS................................................................................................102.3. LESSONS LEARNT ABOUT RTCA/EUROCAE OSA APPLIED TO

ASAS APPLICATIONS ..........................................................................................12

3. ANALYSIS OF OSED/OHA FOR IDENTIFYING SAFETYINFLUENCING FACTORS.................................................................... 143.1. WHAT HAS BEEN DONE IN THE ASM STUDY (AND WHY).........................143.2. CRITICAL ANALYSIS OF THE WORK DONE IN THE ASM STUDY.............17

3.2.1. ANALYSIS OF THE OSED DEVELOPMENT WITHIN THE ASM STUDY...............173.2.2. ANALYSIS OF BOTH OHA PERFORMED WITHIN THE ASM STUDY..................20

3.3. LESSONS LEARNT FROM THE OSED/OSA ACTIVITIES................................26

4. ANALYSIS OF HAZARD IDENTIFICATION BRAINSTORMRESULTS .............................................................................................. 294.1. WHAT HAS BEEN DONE ON HAZARD IDENTIFICATION IN THE

ASM STUDY (AND WHY) ....................................................................................294.2. COMPARATIVE ANALYSIS OF THE HAZARDS IDENTIFIED

DURING BRAINSTORM, OHA AND TOPAZ HAZARD DATABASE.............31


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page v

4.3. LESSONS LEARNT FROM TOPAZ-BASED HAZARDBRAINSTORMS......................................................................................................34

5. RELATIONSHIP BETWEEN THE OSA METHODOLOGY ANDBRAINSTORM BASED HAZARD IDENTIFICATION.......................... 36

6. CONCLUSIONS .................................................................................... 38

7. RECOMMENDATIONS......................................................................... 39

8. REFERENCES ....................................................................................... 41

ANNEX A: RTCA/EUROCAE OPERATIONAL SAFETYASSESSMENT METHODOLOGY OVERVIEW ...................................... 1

ANNEX B: SPECIFIC USE OF THE OSA GUIDELINES IN THECARE-ASAS ACTIVITY 3: ASM STUDY ............................................... 6


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page vi

List of figures

Figure 1: RTCA/EUROCAE OSA methodology overview...............................................................6

Figure 2: High Level Conceptual Safety Model [15].........................................................................8

Figure 3: Various shared risk mitigation strategies of ASAS applications ..........................................8

Figure 4: High Level Conceptual Safety Model applied to the OHA step......................................... 11

Figure 5: Interaction between airborne separation criteria and safety objectives and requirements in theASOR step........................................................................................................................... 12

Figure 6: OSED step conducted in the ASM study ......................................................................... 15

Figure 7: OHA step conducted in the ASM study........................................................................... 16

Figure 8: Operating environment characteristics ............................................................................ 18

Figure 9: Various failure modes analysed during the OHA ............................................................. 21

Figure 10: Illustration of the relationship between failure scenarios and OHs................................... 22

Figure 11: Illustration of relationship between OHs and phases of operations .................................. 23

Figure 12: Illustration of alternative risk mitigation strategies based on the OHA............................. 24

Figure 13: Illustration of implicit relationship between different OHs (depending on environmentconditions) ........................................................................................................................... 25

Figure 14: Illustration of implicit relationship between different OHs (depending on mitigationeffectiveness)....................................................................................................................... 26

Figure 15: Input-output scheme for identification of the AA hazards within ASM project................. 29

Figure 16: Analysis of overlap between identified AA-hazards within the ASM project ................... 33

Figure 17: Co-ordinated Requirements Determination process in RTCA/EUROCAE methodology [1].............................................................................................................................................1

Figure 18: Relationship between hazard classification and greatest likelihood of occurrences [1] ........3

Figure 19: Concept Model of Operational Environment, the Relationships Among Faults, Failures,Procedural Errors, and Airspace Characteristics, and the Risk Mitigation Strategy [1] .................4


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page vii

List of Tables

Table 1: The four groups of hazards identified during the ASM project........................................... 31

Table 2: Summary of analysis of hazard types identified during the ASM project ............................ 32

Table 3: Operational Safety Assessment hazard classification matrix [1] ...........................................3

Table 4: Sequence in ASAS operations ...........................................................................................7

Table 5: Air/ground operations (and communications) during sequence in ASAS operations ..............8

Table 6: ASAS and other CNS functions .........................................................................................8

Table 7: Activated functions during the sequence in ASAS operations ..............................................8

Table 8: Operational failure scenario template ............................................................................... 10

Table 9: Operational hazard description table ................................................................................ 11


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page viii

Acronym list

AA Autonomous Aircraft

ADS Automatic Dependent Surveillance

ADS-B Automatic Dependent Surveillance-Broadcast

ASAS Airborne Separation Assurance System

ASM Airborne Separation Minima

ATC Air Traffic Control

ATM Air Traffic management

ATS Air Traffic Service

CARE Co-operative Actions of Research and development in EUROCONTROL

CD&R Conflict Detection and Resolution

CDTI Cockpit Display of Traffic Information

CENA Centre d’Etude de la Navigation Aérienne

CNS Communication, Navigation, Surveillance

EEC EUROCONTROL Experimental Centre

EMERTA EMERging Technologies opportunities, issues and impact on ATM

EUROCAE EURopean Organisation for Civil Aviation Equipment

FAA Federal Aviation Administration

FD Flight Deck

FFAS Free-Flight AirSpace

FMS Flight Management System

ICAO International Civil Aviation Organisation

MAS Managed Airspace

MASPS Minimum Aviation System Performance Standards

NLR National Aerospace Laboratory

OH Operational Hazard

OHA Operational Hazard Analysis

OSED Operational Service and Environment Description

PO-ASAS Principles of Operations in the use of ASAS

R&D Research and Development


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page ix

RCP Required Communication Performances

RNP Required Navigation Performances

RSP Required Surveillance Performances

TBS Time Based Sequencing

TIS-B Traffic Information Service Broadcast

TOPAZ Traffic Organization and Perturbation AnalyZer


CARE/ASAS/Sofréavia/02-041 - Version 1.1 - September 17, 2002 page 1

1. INTRODUCTION

1.1. Document objectives

The document aims at developing lessons learnt from the OSED/OHA activities and thehazard brainstorm performed during the “CARE-ASAS Activity 3: Airborne SeparationMinima” (ASM) study.

More precisely, based on the experience gained through the CARE-ASAS ASM study oftwo ASAS applications (i.e., “Autonomous Aircraft” in FFAS, and “Time-BasedSequencing” in MAS), the document discusses the following items:

• Adequacy of the RTCA/EUROCAE Operational Safety Assessment (OSA)methodology (that include the OSED/OHA steps), initially developed for D/Lapplications, for safety assessment of Airborne Surveillance and Separationapplications, in general;

• Adequacy of the OSED/OHA steps for the identification of safety influencing factorsthat may impact the airborne separation minima, and critical analysis of the tailoredOSED/OHA methods used in the CARE-ASAS ASM study;

• Contribution of hazard identification brainstorm towards identifying safety criticalhazards of ASAS applications; and

• Relationship between hazard identification brainstorm and the OSAmethodology, in the perspective of an overall framework allowing for the assessmentof operational hazards and allocation of safety requirements.

The first and second items consolidate the lessons learnt through the work packages 1(WP1: Identification of ASAS operational scenarios) and 2 (WP2: Identification of safetyinfluencing factors) of the ASM study; the third item does the same for Task 3.1 (T3.1:Hazard brainstorm) of the WP3. The last item benefits from the relationship experiencedbetween the work packages 2 and Task 3.1 of the ASM study.

Based on the TOPAZ methodology, the ASM study (WP3: First estimation of a safeseparation minima) also performed a risk assessment and evaluation of the AutonomousAircraft application. However, the lessons learned from this work are not considered in thisdocument.

Finally, conclusions and recommendations relative to operational safety assessment ofASAS applications are provided.

1.2. Background and context

1.2.1. CARE/ASAS Activity 3

In the prospect of giving flight crews some responsibilities related to aircraft separation, asenvisaged with some ASAS applications, it is essential to identify all the equipment andoperational elements required for the provision of safe separation by flight crews, definedas airborne separation.



Therefore the major issue is the establishment of the airborne separation minima so as toachieve safe flight operations. Optimistic views are that they could be much smaller thanradar separation and contributed to the publicity of autonomous separation. Other views aremuch more reserved and warn that minima might even be much larger.

The CARE/ASAS Activity 3 deals with the assessment of the magnitude of the airborneseparation minima as it directly affects the usefulness of ASAS applications. Aninvestigation of experience in modelling and determining separation minima has beenconducted initially, followed by the CARE/ASAS Activity 3: Airborne Separation Minima(ASM) study, that was carried out from November 2000 to December 2001.

1.2.2. CARE-ASAS Activity 3: the ASM study

As part of this activity, the CARE-ASAS Activity 3: Airborne Separation Minima (ASM)project consists of the investigation of safety influencing factors that may affect airborneseparation minima, and of a first attempt of quantification [4] , [5], [6] , [7]. Nevertheless,since the work has been performed for R&D purposes and not with any implementationobjectives, the results shall not be taken as definitive ones.

This project was carried out by a consortium of organisations, composed of Sofréavia,NLR, and the EUROCONTROL Experimental Centre (EEC) and with the participation ofCENA and the University of Glasgow.

Methodologies and scope

In the prospect of investigating the airborne separation minima, the project adopted astepwise approach using recognised methodologies, tailored to meet the objectives of theCARE-ASAS Activity 3: ASM project.

The first two steps, i.e. Operational Service and Environment Description (OSED) andOperational Hazards Analysis (OHA) are derived from the RTCA SC-189/EUROCAEWG-53 guidance. For the other steps, i.e. the likelihood of occurrence for hazards andconsequence modelling, the risk estimation and evaluation and an ICAO Target Level ofSafety, the TOPAZ methodology was used.

Two ASAS applications, i.e. the “Time-Based Sequencing” (TBS) application and the“Autonomous Aircraft” (AA) application, were selected as case studies in the project.These ASAS applications, both challenging in terms of safety, were considered ascomplementary since they allowed to identify safety influencing factors that could moregenerally apply to two groups of ASAS applications: Airborne Self-Separation in FFASand Co-operative Separation in MAS.

OSED/OHA of the two selected ASAS application

During the project, a customised OSED template and a tailor-made OHA method havebeen defined to meet the CARE/ASAS Activity 3: ASM objectives. These methods haveevolved during the project based on the experience gained through the assessment of eachASAS application.

The OSED objective was to obtain the relevant information for the safety assessment. TheOHA allowed for the identification of operational hazards, their severity assignment anddetermination of safety influencing factors from a qualitative perspective. At the sametime, a list of safety assumptions, requirements and recommendations used to mitigate theoperational consequences of these hazards was identified.



Although based on different operational and technical characteristics, some commonaltieshave emerged from the safety influencing factors identified through the OHA of the twoselected ASAS applications.

Among the common operational safety influencing factors, both ASAS applications rely onclear and unambiguous procedures, and the protagonists have to be qualified and welltrained. Regarding the system safety influencing factors, the correct and safe execution ofboth ASAS applications is supported by on-board assistance that computes and providesthe relevant information to the flight crew. This on-board assistance and the CNScapabilities (e.g. ADS-B, TIS-B) will have to respect minimum performances in order toreduce the likelihood of occurrence of system-related hazards to an acceptable level.

TOPAZ based accident risk assessment for autonomous aircraft

The next step was to exploit the TOPAZ risk assessment methodology to provide insightinto estimating safe separation criteria for ASAS-based operational concepts, and inparticular for the Autonomous Aircraft application. The influencing factors and operationalhazards identified in the associated OHA have been taken into account.

First, a hazard brainstorming has been organised to identify additional hazards that are dueto the mitigation means identified in the autonomous operations OHA. The subsequent taskwas to develop a mathematical model for the ASAS operation considered (by re-using amodel previously developed within TOPAZ for a similar ASAS-based operation) andperform accident risk assessment for this mathematical model. The accident risk-spacingcurve that was based on this accident risk model was used as a reference curve.

The next stage was to identify and assess the differences between the mathematicalaccident risk model and the Autonomous Aircraft application considered in the ASM study.These differences were identified and assessed on risk bias and uncertainty, and the model-based reference curve was compensated for this risk bias and uncertainty. Theseassessments resulted in a first estimation of safe separation criteria for the AutonomousAircraft operational concept.

Suggestions for future work

In the prospect of determining airborne separation minima, and on the basis of the overallwork performed during the ASM study, some items appeared to particularly deserve furtherinvestigations. These include:

• a more in-depth analysis of both operational and technical factors that may influencethe airborne separation minima;

• an improvement of the mathematical models for risk assessment of ASAS applications;• a more integrated and iterative approach allowing for the assessment of operational

hazards, the allocation of safety operational requirements and the determination of safeairborne separation minima

• and finally, to support the previous approach, the selection of the most appropriatemethods and tools.



1.2.3. Other related work in Europe

Some Operational Service and Environment Descriptions and Operational HazardAssessments of ASAS applications, also based on the RTCA/EUROCAE guidelines, havealready been conducted within the scope of the NUP (NEAN Update Programme) of theEuropean Commission [10] , [11] , [12], [13].

More recently, a preliminary OHA of ASAS Case Studies inspired from NUP have alsobeen developed by the EUROCONTROL ADS Programme [14]. Operational HazardAssessment of other ASAS applications should be performed within the European MFF(Mediterranean Free-Flight) program.

Although based on the same methodology, the level of detail of the associated OSEDs, andconsequently the OHAs, vary depending on the project scope and objectives.

Related TOPAZ work on ASAS includes the projects EMERTA [18], Initial Free Flight[19] and Extended Free Flight [16] , in which different ASAS-based operational conceptshave been studied on safety-separation. In addition, reference [17] gives an overview ofNLR free flight projects.

1.2.4. FAA/Eurocontrol Action Plan 1 coordination

The FAA/EUROCONTROL R&D committee Action Plan 1 has been tasked to review thesafety assessment methodologies under development in USA, Europe and ICAO, and findand agree on a common approach to safety assessment methodology for ASASapplications.

In that context, the CARE-ASAS Activity 3: ASM study, as well as the lessons learnt fromthat work, are expected to provide relevant information on the use of theRTCA/EUROCAE OSA methodology in general, the OSED/OHA steps in particular, andon the TOPAZ methodology.

1.3. Document overview

This first section describes the purpose of the document and presents the scope of theCARE-ASAS Activity 3: ASM Extension study.

Section 2 analyses the OSA methodology (that include the OHA step), initially developedfor D/L applications, for Airborne Surveillance and Separation applications, in general.

Section 3 more specifically and thoroughly analyses the OSED/OHA steps for theidentification of safety influencing factors that may impact the airborne separation minima.Based on the two OSED/OHA conducted during the CARE-ASAS ASM study, thisanalysis of the work done highlights the characteristics of the tailored OSED/OHAmethods used and discusses their benefits and drawbacks in performing qualitativeoperational hazard assessment of ASAS applications.

Section 4 analyses the contribution of the hazard identification brainstorm performedwithin of the CARE-ASAS Activity 3: ASM study of the Autonomous Aircraft application.This analysis is particularly focused on the overlap and differences with the hazardsidentified during the OHA, those identified through the hazard brainstorm and thosealready available in the TOPAZ database.



Section 5 evaluates the possible complementary contribution of hazard identificationbrainstorm to the OSA methodology, and more generally discusses an overall frameworkrelying on these two methodologies, allowing for the assessment of operational hazards andallocation of safety requirements.

Finally, sections 6 and 7 provide conclusions and recommendations relative to safetyassessment of ASAS applications based on the experience gained within the CARE-ASASActivity 3: ASM study.

The annexes provide additional information about the background of the study:

• ANNEX A: provides an overview of the RTCA/EUROCAE Operational SafetyAssessment methodology;

• ANNEX B: describe the specific use of the OSA guidelines in the CARE-ASASActivity 3: ASM study;



2. ANALYSIS OF OSA METHODOLOGY FOR ASASAPPLICATIONS

2.1. The OSA methodology

The initial purpose of the RTCA SC189/EUROCAE WG53 Operational Safety Assessment(OSA) methodology is to identify safety requirements and allocate them between thevarious ATM segments of Data-Link applications (Cf. ANNEX A:).

In the context of the OSA methodology, the OSED objective is to obtain the relevantinformation for the safety assessment of the considered CNS/ATM system.. In a widerscope, as considered in the RTCA/EUROCAE guidance, the OSED is also used as a basisfor assessing and establishing the performances and interoperability requirements.

The purpose of the OHA step is to develop an end-to-end qualitative assessment ofpotential operational hazards. The next step is the establishment and Allocation of SafetyObjectives and Requirements (ASOR) to stakeholders and air/ATS segments. The OHAand ASOR constitute two interrelated processes, which development may be iterative anddependent on the degree to which the operational concept has been developed.

Operational Safety

Assessment OSA

Operational Service Environment Definition OSED

Operational Hazard Analysis OHA

Allocation of Safety Objectives and Requirements ASOR

ATS Segment Airborne Segment

Figure 1: RTCA/EUROCAE OSA methodology overview

The OHA is a qualitative assessment of the operational hazards associated with the OSED.For the OHA, operational functions are examined to identify and classify hazards thatcould adversely affect those functions. Hazards are classified according to a standardisedclassification scheme based on hazard severity and taking into account human factors.Overall safety objectives are assigned to the identified hazards according to a riskclassification matrix: the most severe the hazards are, the less frequently they are tolerated.

Based on the OHA results, the ASOR allocates safety objectives to organisations, developsand validates risk mitigation strategies that are shared by multiple organisations, andallocates safety requirements to those organisations.



2.2. Applicability of the OSA methodology to ASAS applications

Based on the ASM study experience, this section discusses the use of the OSAmethodology for ASAS applications. The main items developed are the followings:

• Characteristics of ASAS applications;

• Type of safety assessment required for ASAS applications; and

• Relevance of the OSA guidelines for ASAS applications.

2.2.1. Characteristics of ASAS applications

Even if already being used for the assessment of ASAS applications in various context (e.g.European and US programs like NUP II, MFF or Safe-flight 21), one should remind thatthe OSA methodology was initially developed for assessing ATS services based on datacommunications.

Airborne surveillance and separation requirements

In general terms, ASAS applications can be considered as more challenging than most ofD/L applications since the scope of the safety assessment has to address not only theair/ground communications requirements, but also the airborne surveillance andseparation capabilities requirements.

From that perspective, some similarity can be found between the assessment of ASASapplications and ADS services supported by data communications. However, ASASapplications are raising rather new safety issues, since they relate to the procedural use byflight crew of airborne surveillance data to support separation between aircraft (whichis a new paradigm for ATM).

Since the separation between aircraft plays a major role in the safety of the ATM system,both the human and technical components of ASAS applications would probably have tomeet minimum levels of performance to achieve an agreed target level of safety.

These safety requirements should support the definition of Minimum Aviation SystemPerformance Standards (MASPS) for ASAS systems, including minimum performancesfor the separation functions on-board. More basically, these safety requirements mayhave an impact of the Required Surveillance Performances (RSP) expected from the ASASsystem, as well as (typically in case of an ASAS based on ADS-B), on the RequiredNavigation Performances (RNP) of the aircraft involved and the Required CommunicationPerformances (RCP) between the various segments involved.

Air/ground shared risk mitigation strategy

The different levels of flight deck implication in the risk mitigation strategy of ASASapplications is illustrated here after using the following high level safety model forassessing safety of ATM operational improvements developed by EUROCONTROL SPF(Strategic Performance Framework) Safety Group [15]:



CRITICAL EVENT INCIDENT ACCIDENT

Air/groundcomms, Detection/ correction means ATC/flight deck procedures

RESOLUTION

Safety nets See& Avoid Chance

INCIDENT RECOVERY

Figure 2: High Level Conceptual Safety Model [15]

Typically, different sharing of risk mitigations with respect to aircraft separation canbe identified depending on the ASAS application category [8]. Essential elements of thatrisk mitigation strategy for ASAS applications are the requirements on the separationcriteria applied by flight crew, and the capabilities of either the ATS or the airbornesegments to detect and solve critical events related to aircraft separation.

CRITICAL EVENT

INCIDENTS

RESOLUTION

Enhanced Visual separation Hazards

Airborne Spacing Hazards

Airborne Separation Hazards

Airborne Self-separation Hazards

(Primarily) By flight deck

(Primarily) By ATC



Loss of Visual separation

Loss of ATC separation

Loss of Airborne separation

Loss of Airborne separation

FOUR ASAS APPLICATION CATEGORIES

Figure 3: Various shared risk mitigation strategies of ASAS applications

This figure is only intended to illustrate the main differences between the various ASASapplication categories, and not to be exhaustive. In particular, the resolution of criticalevents occurring during an ASAS application, even if mainly under the responsibility ofeither ATC or the flight deck, may involve both air and ground mitigation measures.

Incidents related to airborne separation standards

Another major characteristic of ASAS applications is the possible change in the evaluationof critical event consequences. Indeed, for ASAS applications that will require the flightcrew to comply with airborne separation standards, infringements of these new standardswill constitute new operational incidents related to the loss of separation.



2.2.2. Which safety assessment for ASAS applications

The safety of an ASAS application should be assessed from the early stages ofdevelopment. And, this should be an iterative process in which the granularity of theassessment is increased as the operational, technical and environmental characteristics ofthe ASAS application are being defined.

When assessing the safety of an ASAS application, it is essential to first define theobjectives of the safety assessment to be performed. In particular, depending on thedevelopment stage of the ASAS application, focus may be put in one of the followingobjectives (although not exclusive one from each other):

• Development of safe operational procedures for the use of ASAS, including safeairborne separation criteria 1 applicable during these procedures;

• Allocation of safety objectives and requirements to all segments involved in the ASASapplication (as advocated by the OSA methodology);

• Assessment of the impact on the level of safety (i.e. no adverse effect on safety orincrease in safety) of the ASAS application.

Depending on the objectives of the safety assessment, different methods of safetyassessment may be used. For instance, different methods for assessing the safety ofcomplex systems, such as Hazard Identification and Fault/Event Trees, are already widelyused in the assessment of changes to ATM systems. In other respects, separation modellingand risk collision assessment methods are being used within ICAO to establish separationstandards in ATM.

In the perspective of developing safe ASAS applications, the OSA methodology shouldprovide an adequate framework allowing for critical events (i.e. operational hazardsaccording to the RTCA/EUROCAE terminology) identification and assessment through theOHA step, and a more in-depth analysis of these critical events generation through theASOR step.

Nevertheless, for those ASAS applications that require the establishment of airborneseparation minima, relationship between the use of the OSA methodology (to allocatesafety objectives and requirements) and of collision risk modelling and assessmentmethods will have to be put in place. Indeed, a combination of both methodologies wouldprovide an overall framework for the establishment of airborne separation standards andassociated air/ground safety requirements.

In addition, for ASAS applications claiming for additional safety benefits, specific methodsmay have to be used allowing for relative assessment of safety compared to an existingone.

1 Examples of such airborne separation criteria in the “Time-Based Sequencing” application studiedin the ASM study include: the airborne separation minima the flight crew have to comply withduring the procedure (and which depend on their CNS/ASAS performances), as well as, the timeseparation value requested by ATC (which is not necessarily compatible with the ATC radarseparation minima, but which has to be compatible with the airborne separation minima).



2.2.3. RTCA/EUROCAE OSA guidelines applied to ASAS applications

As an R&D contribution in early stages of ASAS applications development, the purpose ofthe CARE-ASAS Activity 3: ASM study was to identify safety-influencing factors thatmay have an impact on the airborne separation minima. In that perspective, it was plannedto investigate the use of the OSA guidelines, at least the initial steps of OSED andqualitative OHA on two selected ASAS applications.

These OHA consisted in the identification of operational hazards, their severity assignmentand determination of safety influencing factors through the analysis of operational andfunctional failure scenarios.

Although this specific approach were outside the defined scope of the RTCA/EUROCAEmethodology [1] , some general conclusions can be stretched out with regard to theapplicability of the OSED and OSA guidelines to ASAS applications.

Operational Service and Environment Definition

In the perspective of performing an end-to-end safety assessment of ASAS applications,the OSED guidelines (Cf. Annex C of reference [1]) were found particularly relevant todefine the scope and main characteristics of the applications to be assessed. Nevertheless,some amendments would be required to the better address the characteristics and safetyissues of ASAS applications.

In particular, the guidance material should be amended to allow the description of not onlyD/L communication characteristics, but also those related to airborne surveillance andseparation capabilities, and to the operational use of these ASAS functions .

The approach used in the ASM study to describe ASAS applications and their operationalenvironment is further discussed in section 3.

The OSA guidance material (Cf. Annex E of reference [1]) on how to define safetyrequirements was also found particularly relevant in the context of ASAS applicationsdevelopment.

Operational Hazard Analysis

In a first stage, the OHA should allow to enumerate operational hazard events that couldpertain to an ASAS application, to identify the mitigation measures that support safety incase of these events and to assess the resulting operational consequences.



OperationalHazard

INCIDENT ACCIDENTAir/groundcomms, Detection/ correction means ATC/flight deck procedures

HAZARD RESOLUTION

Safety nets See& Avoid Chance

INCIDENT RECOVERY

Airspace & procedures design Air/ground system perfos.

HAZARD PREVENTION

Major

Minor

Hazardous

RISK MITIGATION STRATEGY

Undesired Operational Consequences

Catastrophic

Avoidance and mitigation factors

Figure 4: High Level Conceptual Safety Model applied to the OHA step

The Hazard Classification Matrix (HCM) developed by RTCA/EUROCAE alreadyprovides a basis for such assessment of operational hazards. Nevertheless, properdefinition of the “safety margins” (as referred in the HCM) associated with ASASoperations should be developed to allow the adequate evaluation of the effects of systemfailures and procedural errors on ASAS operations. Indeed, the HCM was foundparticularly oriented towards the assessment of D/L applications in which the separation isprovided by ATS. Since the operational use of ASAS introduces a new paradigm forseparation provision in ATM, appropriate and agreed classification of operational incidentsrelated to ASAS operations will have to be defined.

The approach followed in the ASM study to identify and qualitatively assess operationalhazards related to the selected ASAS applications is further discussed in section 3. Inaddition, the relationship with the results of the hazard brainstorm conducted as part of theTOPAZ study of airborne separation criteria is discussed in section 4.

Allocation of Safety Objectives and Requirements

Even if the ASOR step was not performed as such during the ASM study, the associatedRTCA/EUROCAE guidelines were also considered of particular interest in the perspectiveof identifying, through quite in-depth OHA (cf. section 3), both technical and operationalsafety influencing factors relevant to ASAS applications.



Nevertheless, it appears that appropriate ASOR guidance material should be developed tohelp people taking into account the essential role of the airborne separation criteria in thesafety of ASAS applications.

Indeed, the major issue when allocating the safety objectives and requirements for ASASapplications would probably consist in finding an acceptable combination of the safetyperformance requirements put on the air/ground segments and the airborneseparation criteria requirements. In that perspective, different approaches can beenvisaged:

• The airborne separation criteria are first established taking into account operationalconsiderations (for instance, airspace capacity constraints). And, safety performancerequirements for the ASAS procedures and systems are derived so as to achieve thesafety objectives derived from the OHA.

• The safety performances that can be reasonably expected from the air/ground segmentsduring ASAS applications are established first taking into account existing limitationsfrom technologies or procedures. Then, the airborne separation criteria are establishedso as to achieve the safety objectives derived from the OHA, while taking into accountthe level of performances required from ASAS procedures and systems.

Actually, both approaches would probably have to be investigated depending on the ASASapplications, and potentially a combination of these approaches in an iterative process forsome ASAS applications.

SAFETY

OBJECTIVES (in terms of accidents, hazardous, major and minor incidents)

AIRBORNE SEPARATION

CRITERIA (used either by ATC or the Flight Deck)

SAFETY & PERFORMANCE REQUIREMENTS

(in terms of RNP, RCP, RSP, …)

Figure 5: Interaction between airborne separation criteria and safety objectives andrequirements in the ASOR step

2.3. Lessons learnt about RTCA/EUROCAE OSA applied to ASASapplications

This sub-section presents the main lessons learnt from the use of the OSA guidelines withinthe CARE-ASAS Activity 3: ASM study, and briefly discusses possible amendments of theRTCA/EUROCAE guidelines to support the future Operational Safety Assessment ofASAS applications.



In the perspective of performing an end-to-end safety assessment, the RTCA/EUROCAEOSA guidelines were found particularly relevant for ASAS applications, which claim for agreater involvement of the flight deck in air traffic services and separation provision.Nevertheless, some amendments would be required to the better address the characteristicsand safety issues of these ASAS applications.

Operational Service and Environment Definition

First, the scope of the safety assessment of ASAS applications has to address not only theair/ground communications requirements, but also the airborne surveillance and separationcapabilities requirements.

Therefore, the OSED guidance material should be amended to allow the adequatedescription of the airborne CNS and ASAS characteristics, the operational use of theseASAS functions, as well as the expected performances of both the human and technicalcomponents of ASAS applications.

Such description of the “operating method with ASAS” and their functional characteristicsshould support the analysis of operational hazards related to ASAS-based aircraftseparation, as well as the definition of safety requirements related to surveillance andseparation functions on-board.

Operational Hazard Analysis

Since the operational use of ASAS introduces a new paradigm for separation provision inATM, appropriate and agreed classification of operational incidents related to ASASoperations will have to be defined.

As a consequence, the Hazard Classification Matrix (HCM) developed byRTCA/EUROCAE should be amended with proper definition of the safety marginsassociated with ASAS operations. Indeed, the HCM was found particularly orientedtowards the assessment of D/L applications in which the separation is provided by ATS.

Allocation of Safety Objectives and Requirements

Since the separation between aircraft plays a major role in the safety of the ATM system,both the human and technical components of ASAS applications would probably have tomeet minimum levels of performance to achieve an agreed target level of safety.

Another major issue when allocating the safety objectives and requirements for ASASapplications would probably consist in finding the best compromise between the safetyperformance requirements put on the air/ground segments and the airborne separationcriteria requirements.

Therefore, appropriate ASOR guidance material should be developed to support taking intoaccount the essential role of the airborne separation criteria in the safety of ASASapplications.



3. ANALYSIS OF OSED/OHA FOR IDENTIFYING SAFETYINFLUENCING FACTORS

This section aims developing lessons learnt from the OSED/OHA activities performedduring the ASM study, in the perspective of identifying safety influencing factors. As such,the section is organised as follows:

• Section 3.1 briefly describes the OSED/OHA activities performed in the ASM projectand their rational. More detailed description of the OSED template and customisedOHA method used is given is ANNEX B:.

• Section 3.2 provides a critical analysis of the work done, and develops lessons learntfrom the ASM study about these OSED/OHA steps, though examples derived fromeither of the two ASAS applications studied within the ASM study;

• Section 3.3 summarises the lessons learned from the OSED/OHA activities, and brieflydiscusses possible improvements of the approach, in the prospect of furtherinvestigating safety influencing factors related to ASAS applications using theOSED/OHA framework.

3.1. What has been done in the ASM study (and why)

Within the scope of the CARE-ASAS Activity 3: ASM project, Operational Service andEnvironment Definition and preliminary Operational Hazard Assessment of the twoselected ASAS applications (i.e., “Autonomous Aircraft” in FFAS, and “Time-BasedSequencing” in MAS) have been performed.

Rationale for the OSED/OHA steps

Within the ASM study, the rationale for conducting OHA of the two selected ASASapplications, using the RTAC/EUROCAE methodology, was to identify safety-influencing factors that may have an impact on the airborne separation minima.

Although the ASOR step was not intended to be developed in the ASM study, the need foridentification of both operational and technical safety influencing factors was clearlyidentified during the study. Indeed, the elements potentially affecting the separationminima applicable during ASAS operations include the operational procedures them-selves, the Communication, Navigation, Surveillance and ASAS separation capabilities onboard, as well as the human capabilities to conduct such operations.

For that purpose, customised OSED template and tailored-made OHA method have beendefined, which have evolved during the project (Cf. ANNEX B:). This framework alsopermitted to get common understanding of OHA process to be performed between thevarious participants of the ASM project.

OSED approach overview

Originally oriented in a data link and communication aspect, the RTCA/EUROCAE OSEDtemplate used was adapted to cope with the specificity of ASAS applications studied withinthe CARE-ASAS ASM study.

The objective of the OSED was to provide an operational perspective of these ASASapplications in a defined anticipated environment of use:



• “Autonomous Aircraft” operations in a Free Flight Airspace, where the operations giveto flight crew the complete responsibility of the flight, and

• “Time Based Sequencing” application, in a Managed Airspace, where the operationscould be defined as an application based on the “extended visual clearance procedure”in current control practice, but with specific procedures and using suitable instrumentsto comply with airborne separation minima in IMC.

One of the challenges of the “Time Based Sequencing” OSED was to describe theoperations with a level of details allowing the extrapolation of the results to a generalcontext of co-operative ASAS operations.

For both ASAS applications, operational environment and procedures were described,together with the different functional characteristics of the system to properly perform theprocedures (see Figure 6). The environment description aimed at being generic as far aspossible to avoid the specificity of an airspace. However to be representative, it wasderived from a real airspace.

Once the broad characteristics of the environment and ASAS operations were jointlydefined, the capture of the detailed information required to fulfil the OSED template [2]was performed in parallel by two members of the ASM project. Then, the functionalcharacteristics of both CNS and ASAS systems supporting the operations were described.

Operational Service

Environment Definition OSED

Operational Hazard Assessment OHA

Environment characteristics description

Operations with (and without) ASAS

Functional characteristics description

Figure 6: OSED step conducted in the ASM study

Whereas the “Autonomous Aircraft” OSED provided a quite detailed description of thefunctional characteristics of the ASAS system, the description of the “Time-BasedSequencing” OSED identified more precisely the operations (including communications)between controllers and pilots. This more operational-oriented approach advocated limitedairborne and ground system/support tool assumptions in the OSED. The purpose was tofacilitate the identification of operational safety-influencing factors, in addition toenvironmental and technical factors .



OHA approach overview

Quite in-depth OHA based on failure scenarios analysis were performed, which wassupported by, high level, but as much as possible unambiguous and exhaustive, descriptionof ASAS operations and functions (Cf. ANNEX B.2.1 ).

This ASAS application modelling was used to split the analysis of failure scenarios relatedto the various operations (or functions) between the people involved (either 2 or 3depending on the studied ASAS application).

Whereas the first OHA was rather focused on the analysis of the CNS/ASAS functionsinvoked during the ‘Autonomous Aircraft’ operations, the second OHA were supported bya description of the ‘Time-Based Sequencing’ application that identified more precisely theoperations (including communications) performed by both ATC and the flight deck. Thepurpose was to facilitate the identification of operational safety-influencing factors, andalso to assess the safety of the air/ground cooperation during the ASAS procedures.

OperationalHazard

AssessmentOHA

Operational Service Environment DefinitionOSED

Sequence in ASAS operationsdescription

Consolidation ofOperational Hazards

Operational Failure ScenariosAnalyses

Figure 7: OHA step conducted in the ASM study

The failure scenarios analyses allowed identifying some human-related and system-relatederrors leading to operational hazards, and which may actually influence more directly theairborne separation minima (Cf. ANNEX B.2.2 ). Nevertheless, since the ASM study wasconducted only for R&D purposes with limited efforts within limited timeframe, thesefailure scenario analyses were not fully completed.

Two or three study participants, each of whom was considering failure conditions related toa set of operations or functions, were involved in these analyses.

The consolidation of the various failure scenarios having the same effects on theoperations resulted in a list of operational hazards (either related to functional systemfailures or procedural errors), their severity assignment and determination of risk mitigationmeasures (Cf. ANNEX B.2.3 ). These mitigation measures consisted in the variousenvironmental, operational and technical means introduced to mitigate or justify theseverity assigned to the operational hazards.

This consolidation was done by cross-review of the failure scenarios analysed by eachOHA participant, and discussion/agreement on the analysis and additional mitigations(when required).



This OHA process resulted in the identification of various safety-influencing factorswith different levels of assessment. These factors, which influence directly or indirectly thesafety of ASAS applications under assessment, consisted in:

• The various failure events (“low-level” hazards) analysed during the OHA and that mayresult in operational hazards;

• The operational hazards (hazards at the level of operations) themselves which werequalitatively assessed, and

• The safety assumptions and requirements, which were necessary to achieve the severityassigned to the OH in which they play a role.

It should be noted that the hazards that could result from the lack of, or the erroneous,application of these safety assumptions or requirements were not further assessed duringthe OHA, since this would have required further efforts not compatible with the ASMstudy.

3.2. Critical analysis of the work done in the ASM study

This section develops a critical analysis of the OSED/OHA performed during the ASMstudy in the perspective of identifying safety influencing factors. Main advantages,limitations and drawbacks of the approach used are discussed and illustrated usingexamples extracted from the two studied ASAS applications.

3.2.1. Analysis of the OSED development within the ASM study

This section develops a critical analysis of the work carried out to capture the operationalservice and environment information associated with both ASAS applications. The mainOSED items discussed are the followings:

• Operating environment characteristics;

• Operating method with and without ASAS;

• Operational service description;

• Operational procedures versus functional characteristics.

Operating environment characteristics

The typical various elements listed in the figure below were captured when describing theoperational environment of the two studied ASAS applications.



ENVIRONMENTCHARACTERISTICS

AIRSPACECHARACTERISTICS

TRAFFICCHARACTERISTICS

- Type of Air Traffic Services

- Airspace class - Separation minima

- Airspace configuration andcomplexity

- Sectorization

- Special Use of Airspace - Weather

- Ground CNS coverage andequipment

- Throughput/track occupancy

- Sector Traffic Density

- Aircraft mix

- Aircraft performance

- Aircraft CNS equipment

Figure 8: Operating environment characteristics

Although widely opened and sufficiently general to allow for an ad hoc adaptation, some ofenvironment items appeared either not appropriate for the purpose of the CARE/ASASASM study, or not sufficiently detailed to cope with the specificity of ASAS applications:

• Airspace configuration and complexity, Type of ATS, sectorisation, airspace class:

It appeared that the environment issues to be considered differed with the type of ASASapplications. Whereas the Time-Based Sequencing (TBS) is defined in a similarairspace environment as today that adheres to the propose guidelines, the AutonomousAircraft (AA) application is defined in a new type of airspace environment (e.g.segregated airspace without current ATS support) and raises new issues (e.g.MAS/FFAS transitions). These quite different airspace issues were not well underlinedin the OSED template, which was therefore subject to an ad hoc adaptation.

• Airborne Separation Minima (ASM):

As it was the purpose of the study, this issue was not investigated during theenvironment description stage. However, in the perspective of future operational safetyassessment of ASAS applications, it will be essential to precisely define the separationminima applicable during ASAS applications, as well as their relationship withenvironment characteristics like the traffic level or the CNS/ASAS level of services.

• Air/ground CNS characteristics:

Since CNS information is quite sensitive within ASAS environment description, it wasintended to identify the CNS aspects which ASAS applications may have to cope with(e.g. existing surveillance means, TIS-B infrastructure), as well as those which may beimpacted by the ASAS applications. However, the ASAS applications were not matureenough to give details about CNS characteristics. Without clear view of the relationshipbetween CNS and ASAS, only qualitative information at a functional level were given.

For the “Autonomous Aircraft” TOPAZ-based spacing/accident risk assessment,additional and more detailed information were provided for some CNS characteristics(e.g. ADS-B performances).



In the perspective of an “end-to-end” safety assessment (spanning the pertinent aspectsof the aircraft segment and the ATS provider segment), the OSED was also intended toidentify external ground ATC tools that could impact the execution of ASASapplication (e.g. Arrival manager for the TBS environment). High-level qualitativedescription was provided, with the objective of identifying how it could interfere withASAS operations.

• Throughput/track occupancy, sector traffic density:

Whereas one can anticipated that such information might have an impact on the safetyof an ASAS application, the required level of detail necessary for the following steps ofthe study was not well identified.

As a first step, indicative figures were given to provide an overview of the trafficcharacteristics. In a second step, additional information was provided for the“Autonomous Aircraft” TOPAZ-based spacing/accident risk assessment.

• Aircraft mix and aircraft performance:

Similarly to previous item, the approach was to provide indicative figures. The issueabout percentage of aircraft equipage for ASAS was only evoked, but it should be animportant issue in the ASAS implementation.

Operating method with and without ASAS

Part of the OSED methodology, the description of the operating method with and withoutthe new D/L services appeared to be incomplete or not adapted within the ASM study.Actually, the imbalance between the “Operating method with ASAS” and “Operatingmethod without ASAS” descriptions turned out to be an issue with the lack of comparisonsbetween current procedures and ASAS ones.

For the “Autonomous aircraft” application, this comparison was almost meaningless,whereas for the co-operative ASAS application, the major risk was to “re-define” the rulesused by the current air traffic operations during the study of operational failure scenarios.And a more detailed parallel and a comparison between the two parts (with and withoutASAS) would have been required in this case.

Operational service description

The OSA process implies refinement of the OSED through OHA results. One of thedifficulties in the ASM study was to identify some relevant level of detail for the OSED,and to appreciate the consequences of developing them items more than others before thesafety assessment.

• Sequence in ASAS operations:

Following the first operational safety assessment, it appeared that the ASAS applicationneeded to be defined in general phases, where the different interactions between thevarious participants (i.e. controllers and pilots) have to be defined. This approachallowed a better view of ASAS operations and their supporting functions.For example, in the “Time-based Sequencing” application, three main phases wereidentified:

1- Initiation of the procedure, with the transfer of separation responsibility from ATC topilot between the delegated and the target aircraft,

2- Execution of the procedure, with the total flight crew responsibility of the separationwith the target

3- End of the procedure, when the separation responsibility goes back to ATC.



• Capturing time constraints:

Another point to be highlighted is the inadequacy of the time diagrams for theprocedure description and study. In effect, the given ones in the original templateappeared to be in accordance with a study of an implementation of a Data link (verticaldiagram as a coded program).

• Air/ground operations during sequence of ASAS operations:

During the study, it appeared that the whole document needed to be improved withfunctional description of the air/ground operations associated with the ASASapplication. All aspects concerning the system were reduced to the minimum, in orderto later identify the system functional requirements (e.g. the pilot had the necessaryinformation to perform the application, displayed in a specific interface; no systemassumptions were made).

At this stage, the objective was not to strictly identify the Stakeholders, but to define themain protagonists and their respective roles and responsibilities: the ATC and the flightdeck.

• Activated functions during ASAS operations:

Some improvements in the description of the links between ASAS functions and thephases of operations were also added to better understand (and better identify hazardsrelated to) the operational use of ASAS.

In particular, there was a need to link system functions invoked during ASASoperations, and their use in the procedures (which function at which step of theprocedure, using which part of the system).

Such description of the whole mechanism allowed characterising links from input datato processed or displayed ones (which imply human or system reactions), giving the“opportunity” to detail interactions between functions, systems and human.

This approach should facilitate the identification of the required performances andsafety requirements, which are done during the following steps of theRTCA/EUROCAE methodology.

3.2.2. Analysis of both OHA performed within the ASM study

This section develops a critical analysis of the OHA performed in the ASM study, step bystep and using examples extracted from the two studied ASAS applications. The mainitems discussed are the followings:

• Failure scenarios either related to technical or human errors;

• Relationship between failure scenarios and Operational Hazards (OHs);

• Relationship between OHs and phases of operations;

• Development of the risk mitigation strategy (on qualitative basis);

• Implicit relationship between different OHs.



Operational failure scenarios

Based on the work that had already been performed in NUP [10], [11], [12] , [13] before theCARE-ASAS Activity 3: ASM study, it was decided to support the OHA of the twoselected ASAS applications by the analysis of failure scenarios.

Since a restricted view of functional characteristics related to systems would not allowidentifying all the potential Operational Hazards, the analysis of failure scenarios (leadingto operational hazards) was intended to investigate procedural errors, in addition tofunctional failures.

For that purpose, the set of descriptors applied to the hazards related to functionalcharacteristics was extended to the hazards related to human involvement in ASASoperations, thus allowing the identification of failure modes either related to systemfunctions and human operations (Cf. ANNEX B.2.2 ).

Failures modesrelated to system functions

Failure modesrelated to human operations

Undetected loss of function (or data) Lack of operation (or human action)

Detected loss of function (or data) Inability (for human action)

Incorrect function (erroneousdetected)

Incorrect operation (or humanaction)

Misleading function (erroneousundetected)

Misleading operation (or humanaction)

Figure 9: Various failure modes analysed during the OHA

Considering the limited amount of time and efforts associated with the ASM study, thedetailed description of the functions invoked during the “Autonomous aircraft” operationsprecluded almost any analysis of human failure conditions.

On the contrary, the limited OSED assumptions about airborne and ground systems, as wellas more the appropriate level of description of the “Time-Based Sequencing” operations(including communications), did facilitate the identification of hazards related to ATC andflight deck (FD) operations, which actually may either derive from human or systemfailures. And, this level of detail did allow to better identify operational safety-influencingfactors.

Nevertheless, the approach clearly did not easily allow identifying the non-expected humanbehaviour that could adversely impact on the development of the ASAS procedures.Although such safety analysis of human factors might not have necessarily come out withnew operational hazards, it would have made the analysis more rigorous and exhaustive.

Taking into account these considerations, there is an obvious need for amendments to theapproach to better identify and assess the impact of human factors on the safety ofASAS operations. In that perspective, the complementarities between the failure scenariosanalysis and the hazard brainstorm session is further discussed in section 4.



Relationship between failure scenarios and OHs

Although not exhaustive, the failure scenario analyses allowed the identification of a list ofoperational hazards, together with the mitigation means that were taken into account toassess their severity.

More precisely, the relationship between the failure scenarios and the operational hazardswas as follows:

OH = Set of failure scenarios leading to the same effects on operations, and having to thesame operational consequences on the basis of on the same overall risk mitigation strategy

As an example extracted from the “Time-Based Sequencing” OHA, the “Incorrect targetaircraft identification” was an operational hazard identified through various failurescenarios either related to ATC or flight deck operations or functions invoked during theTBS procedure initialisation phase. In all these scenarios, detection and correction meanswere identified, which allowed limiting the operational consequences to an increase incontrollers’ and pilots’ workload.

Incorrect target a/c

identification

ATC/flight deck

cross-check procedures

HAZARD RESOLUTION

Workload increase

HAZARD CAUSES

ATC related failure scenarios

Erroneous

identification on CWP,

Erroneous instruction by

ATC, …

FD related failure scenarios

Erroneous

selection into ASAS by pilot,

Erroneous information from

ASAS, …

Figure 10: Illustration of the relationship between failure scenarios and OHs

Since the failure scenario analyses were first performed in parallel by different people(with different background and different matter of interest), the consolidation of theanalysis allowed to enlarge the scope of the safety-influencing factors identified throughthe various failure scenarios. Nevertheless, the approach was time-consuming andprecluded the identification of all possible causes of failures, which was not anyway thepurpose of the study.

This was particularly the case for the ‘Autonomous aircraft’ application in which thedetailed breakdown of the sequence of ASAS operations and detailed description of thefunctions invoked resulted in a large number of scenarios to be analysed.

In the perspective of performing an efficient end-to-end safety analysis, particularly in theearly stages of development of an ASAS application, there would be a need for aneffective clustering of adverse scenarios related to the ASAS application underassessment in order to speed up the analysis of operational hazards.



Relationship between OHs and phases of operations

The failure scenarios analyses were broken down into the various phases of operationsidentified for each ASAS application under assessment. This approach allowed to analysethe operational consequences of the same undesired event depending on the time the eventoccurs, and to identify operational hazards associated with these various phases ofoperations, as illustrated with the following examples extracted from the “Autonomousaircraft” OHA [5].

Phase ofoperation

Failurescenario

OperationalHazard

Surrounding trafficsurveillance andmonitoring phase

Erroneous airbornesurveillance data (onboard an aircraft)

Aircraft has incorrectknowledge of surroundingtraffic in FFAS

Priority determinationphase


Aircraft provided withincorrect information aboutconflicting aircraft in FFAS

Conflict resolutionphase


Incorrect conflict resolutionby aircraft in FFAS

Figure 11: Illustration of relationship between OHs and phases of operations

Nevertheless, the approach was quite repetitive and both safety and operational expert’sjudgement was required to focus the analysis to the most critical conditions (includingphases of operations and environmental conditions) in which the undesired event couldresult in the most severe operational hazard.

• Abnormal phases of ASAS operations:

Another characteristic of the work performed was the identification of OHs related tomultiple air/ground failures through the analysis of abnormal phases of operations (e.g.failure scenarios during abnormal conflict resolution in FFAS, or failure scenarios duringcontingency procedures associated to “Time-Based Sequencing” operations).

Although the analysis of these abnormal phases of operations would not have been requiredin a preliminary OHA, it allowed within the scope of the ASM study to identify OHsresulting from a combination of hazards occurring simultaneously onboard two distinctaircraft, or on board an aircraft and within an ATC unit.

As a side effect, these “multiple” OHs also gave the opportunity to better define thecontingency procedures, to assess the impact of the new ASAS operations on existingmitigation strategies, and sometimes to propose additional mitigation means that could beput in place, if required.

For the co-operative ASAS application, the challenge when analysing these abnormalphases of operations, was to focus the safety assessment on the possible impact of thenew operating method on current mitigation strategies, and to avoid assessment ofcurrent ATM operations under abnormal conditions.



The challenge was quite different for the “Autonomous aircraft” application since thecomparison with the existing ATM operations did not apply. The major issue was in thatcase to develop the mitigation strategies (for those OHs related to abnormal phases ofoperations in FFAS) without investigating into details all the combination of hazards thatmay occur on board both aircraft.

Development of the risk mitigation strategy (on qualitative basis)

At different level, the OHA of the two selected ASAS applications allowed to furtherdevelop the risk mitigation strategies more or less defined in the OSED:

• During the “Autonomous Aircraft” OHA for which the OSED already included a lot ofmitigations, this was mainly done through the analysis of the phase of abnormaloperations.

• During the “Time-Based Sequencing” OHA, the appropriate level of description of theASAS operations (including communications), did facilitate the identification throughthe OHA of additional requirements (or alternatives) necessary to support the safety ofthe ASAS application.

Nevertheless, since both OHA were only based on a qualitative assessment of hazards, itwas not always possible to conclude on the effective need for these additional means.

As an example extracted from the “Time-Based Sequencing” OHA, the “Misleading targetaircraft identification” was an operational hazard identified through various failurescenarios, and for which various alternative resolution strategies (based on either ATC orthe flight deck) have been proposed to reduce the severity of its operational consequences.However, neither the need for these alternatives, nor their relative impact on safety, havenot been assessed within the scope of the ASM study.

Major

Serious

Alternative solutions based on:

Enhanced

STCA, or

Monitoring by target a/c

or ASAS alerting

Misleading target a/c

identification

HAZARD RESOLUTION

Depending on the risk mitigation strategy put in place

Figure 12: Illustration of alternative risk mitigation strategies based on the OHA

Lessons learnt from the second OHA on the “Time-Based Sequencing” application includethe need for a preliminary evaluation of frequencies of occurrence of operationalhazards , early in the OHA process, to better generate options for and evaluate theadditional mitigations that actually need to be put in place.

Indeed, for the more severe operational hazards, either the related scenarios will beevaluated (in first approximation) to be extremely remote, or the most appropriate riskmitigation strategy (possibly selected among different options) will have to be developed.



Such preliminary quantitative assessment of the operational hazards, which could beperformed through judgement-based analysis, would also support more in-depthinvestigation of environmental, operational and technical safety influencing factors.

Implicit relationship between different OHs

As previously discussed, the operational hazards identified during the OHA resulted fromthe consolidation of operational failure scenarios analyses. Although adapted to theidentification of “low-level” safety influencing factors, this bottom-up approach did notfacilitate the identification of the relationship that might exist between the various OHs.

The following examples extracted from the “Autonomous Aircraft” OHA illustrate thelinks that were identified between some OHs potentially resulting from the same failureconditions (e.g. erroneous airborne surveillance data), but for which traffic conditions (e.g.conflicting aircraft in vicinity or not) did affect their effects on operations.

Phase ofoperation

OperationalHazard

Possible effects(other OHs)

Mitigations

Surrounding trafficsurveillance andmonitoring phase

Aircraft has incorrectknowledge ofsurrounding traffic inFFAS

Induced conflict by aircraftmanoeuvring in, orundetected conflict byaircraft that has tomanoeuvre according topriority rules

Prioritydeterminationphase

Aircraft provided withincorrect informationabout conflicting aircraftin FFAS

Incorrect conflict resolutionby aircraft in FFAS

Conflict resolutionphase

Incorrect conflictresolution by aircraft inFFAS

Flight crews’ agreementon a common resolutionstrategy (using voicecommunications)

Figure 13: Illustration of implicit relationship between different OHs (depending onenvironment conditions)

As another example extracted from the “Time-Based Sequencing” OHA, the “Misleadingtarget aircraft identification” was an operational hazard that could result from the failure ofthe cross-check procedures already defined in the OSED, and used to resolve the “incorrecttarget aircraft identification” by either ATC or the flight deck. During the operationalfailure scenario analysis, unfavourable traffic situations were identified in which thesecross-check procedures could be made inefficient, thus resulting in a more severeoperational hazard.



Misleading target a/c

identification

Incorrect target a/c

identification

Success

Failed

Unfavorable traffic situation (e.g. similar SSR codes on the

same arrival stream)

ATC/flight deck

cross-check procedures

HAZARD RESOLUTION

Figure 14: Illustration of implicit relationship between different OHs (depending onmitigation effectiveness)

In the perspective of a preliminary OHA, lessons learnt from the ASM study include theneed for a complementary approach for the operational hazard identification andassessment. Such an approach would consist in identifying the hazards at the level ofoperations and assessing their various relationship and consequences, without necessarilygoing into the detailed analysis of the failure scenarios leading to those operational hazards.

This would allow the identification of the various risk mitigations that may affect the finalconsequences of same initial undesired event, depending on the environmental conditionsand the effectiveness (or not) of the successive mitigation measures.

3.3. Lessons learnt from the OSED/OSA activities

This sub-section presents main lessons learned from the OSED/OHA activities performedwithin the CARE-ASAS Activity 3: ASM study, and briefly discusses possibleimprovements of the approach, in the prospect of further investigating safety influencingfactors related to ASAS applications.

Main characteristics of the OSED development

Because of the prospective nature of the CARE-ASAS ASM study, the approach adoptedwhen developing the OSED of both selected ASAS applications was:

• To highlight all the potential issues (e.g. issues related to ASAS operations and systems,but also environmental ones like the definition of new rules of the flight or newairspace) that would be further detailed, or on the contrary kept aside, according to thedevelopment of the ASAS applications and their safety assessment;

• To identify and describe the environment, operational and technical characteristics thatcould affect the robustness of the ASAS applications.



Nevertheless, it should be recognised that:

• Some of the OSED items were not mature enough and were defined during the study(e.g. FFAS/MAS transition issues, time criteria for the TBS application);

• Without a clear view of the next steps of the study, it was difficult to anticipate theamount of OSED information, as well as the appropriate level of detail, required for thesafety assessment;

• To better identify any “interference” with the current operations, particularly for co-operative ASAS applications, the OSED should have been developed with adequatereferences and parallels to the standard operations and rules.

Possible improvements in the OSED development

In the prospect of further investigating the safety of ASAS applications using the OSEDframework, the main conclusions from the ASM study experience would be:

• To support an iterative approach for the OSED development taking into account thedevelopment and validation stage of the ASAS application (in terms of operationalprocedures, ATC and FD operations and air/ground functional characteristics, andfinally ATM system description).

• To better adapt the information provided in the OSED depending on the ASASapplication category and the different issues raised, in particular with respect to airspaceenvironment and operating method;

• To better adapt the type of information, and level of detail, in the OSED to theobjectives of the safety assessment.

Main characteristics of the customised OHA approach

With respect to the OHA activity, the customised approach used the ASM study, whichactually evolved during the study, could be characterised as a:

• Systematic approach for identifying OHs through operational failure scenarios analysis;

• Bottom-up approach for early identification of both procedural errors and functionalfailures leading to these OHs;

• Simplistic approach for taking into account time aspects (through the analysis of sameinitial hazard depending on the phase in ASAS operations);

• Identification of different levels of safety influencing factors (i.e. some system andhuman failure events, operational hazards and risk mitigation means) and theirrelationship.

The OHA approach also had major limitations and drawbacks including the followings:

• Time consuming approach due to the complexity of the system under assessment andthe large number of scenarios to be analysed;

• Judgement-based approach for the identification of safety mitigation measures (eitherassumptions from the OSED or additional requirements defined during the OHA);

• Judgement-based approach for identification of relevant combination of undesiredevents and environmental factors (i.e. operational failure scenarios) leading to OHs;

• Inadequate approach for the identification of the relationship between OHs associatedwith similar operational failure scenarios, but within different phases of operations orwith different environmental conditions.



Possible improvements in the OHA approach

In the prospect of further investigating safety influencing factors related to ASASapplications using the OSED/OHA framework, the main conclusions from the ASM studyexperience would be:

• To better determine and validate, through the OHA, the additional mitigations thatactually need to be put in place. This could be supported by an early OHA based on theanalysis of the sequence of operations, and focused on the more critical sequences ofevents and conditions, allowing for comparison of various risk mitigation strategies.

This comparative analysis could also be complemented by a preliminary frequencyevaluation (i.e. levels of order) of operational hazards (with judgment-based assessmentof likelihood of occurrences of various OHs, including probabilities of unfavourableenvironment conditions affecting effectiveness of mitigations).

• To better assess the impact of human factors in ASAS operations. In that perspective,the possible results achievable through the use of hazard brainstorms are discussed insection 4.

• To allow effective clustering of adverse scenarios, and identification of relevant safetyinfluencing factors, at different stages of development of the ASAS applications. Thiscould be achieved through an early qualitative OHA based on the analysis of thepossible failure of air/ground operations and communications, and their effects onoperations.

Then, taking into account refinement of the OSED (possibly impacted by thepreliminary OHA), the analysis could be completed step-by-step by the “top-down” (ordeductive) determination of possible causes of OHs, and also the “bottom-up” (orinductive) analysis of various failure scenarios (with different aggravating factors linkedto the environment).

Such iterative approach for the stepwise analysis of operational hazards at differentlevel of details is proposed in section 5.



4. ANALYSIS OF HAZARD IDENTIFICATION BRAINSTORMRESULTS

This section analyses the complementarity of the hazard identification results obtained withthe TOPAZ-based hazard identification approach used in work package 3 of the ASMstudy [6] and the ASM-customised OSED/OHA approach, in order to identify lessonslearnt from the brainstorm approach. The section is organised as follows:

• Subsection 4.1 describes what has been done in the ASM project on hazardidentification, and why. This description includes the OHA approach identified hazards,since these were used by the TOPAZ-based hazard identification approach.

• Subsection 4.2 provides a comparative analysis of the hazard identification resultsobtained with all methods used within ASM.

• Subsection 4.3 presents some lessons learnt regarding the TOPAZ-based hazardbrainstorm approach.

Note that in this section, the term hazard is used in a broad sense: the term includes anyhazard or non-nominal event or situation. Therefore, it includes the hazards that the OHArefers to as "(failure) scenarios". This also means that there are "hazards" of various levelof detail. This factor should be taken into account when comparing counted numbers ofhazards.

4.1. What has been done on hazard identification in the ASM study (andwhy)

This subsection describes what has been done within the ASM project to identify hazardsand why. The figure below presents an input-output scheme for how the AA-identifiedhazards were obtained. The dates given are the dates of the document in which the resultsare described.

AA- OSED( 30 -3-01)

AA-OSED(5-7-01)

TOPAZ used AA operational

concept description(19-10-01)

TOPAZ database(Okt 01)

Group 3:AA-haz ards fromTOPAZ database

(22-10-01)

Group 2:TOPAZ brainstorm

AA identified hazards(31- 7-01)

AA-safetyrequirements

(19-6-01)

Group 1:AA-OHA

identified hazards(19-6-01)

1

3

4

AA-safety ass umptions

(19-6-01)

2

5

Minutes from previ ous

brains torms

: Done outside CARE-ASAS Activity 3 project

Figure 15: Input-output scheme for identification of the AA hazards within ASMproject



Action 1 (identification process of the AA-OHA hazards, the safety assumptions, the safetyrequirements, etc) has been described in section 3. Actions 2, 3 and 4 are standard practicewhen performing TOPAZ-based accident risk assessment. Normally, the sequence is 3, 4,2, but in this specific situation, action 2 could be performed prior to action 3 due to theexplicit limitation of the brainstorm scope to safety requirements. Next, we describe inmore detail how the TOPAZ hazards were obtained within the ASM project, followingactions 2, 3 and 4 in Figure 15.

Action 2: TOPAZ hazard brainstorm for ASM:

As shown in the previous sections, the AA-OHA has identified additional safetyrequirements that can be used to mitigate the effects of identified operational hazards. Theintroduction of new safety requirements into an operational concept may lead to newhazards, hence to new safety influencing factors. To stress the importance of this issue, theASM project explicitly included a hazard brainstorm to identify these new hazards.

The hazard brainstorm meeting was organised to identify as many hazards and non-nominal events as possible. The participants were explicitly discouraged to analyseidentified hazards on e.g. consequences, severity, frequency, level of detail. The reason forthis is that during the brainstorm, an expert should not feel dissuaded from bringingforward new hazards, even if their effects seem minor, since new ideas may trigger new,more interesting, hazards from another expert. Notice the difference with some other typesof hazard brainstorms, like HAZOP (Hazard and Operability study), where the analysis andthe development of "solutions" is part of the brainstorm. In order to avoid confusion, thetype of brainstorm used within ASM is referred to as TOPAZ-based hazard brainstorm.

This hazard brainstorm for ASM was focused on identifying hazards due to AA-OHAidentified safety requirements only. Note that this is not a necessary restriction for TOPAZ-based hazard brainstorm. Obviously, due to the previous remark, AA-applicable hazardsthat are not necessarily due to safety requirements only, might also have been identified.The result of the brainstorm can be found in Appendix E of the ASM WP3 report [6].

Action 3: Consolidation of OSED/OHA results into AA-operational concept for TOPAZneeds:

In this action, the operational concept described by the AA-OSED was written in such away that it satisfied TOPAZ needs: operational aspects that were missing in the AA-OSEDwere identified, and level of technical detail was intentionally kept limited. Result of thisstep is an operational concept description that is high level on a broad front. Next, the listsof safety assumptions and safety requirements identified by the AA-OHA were added. Theresult can be found in Appendix A of the ASM WP3 report [6].

Action 4: Identification of AA-applicable hazards from TOPAZ hazard database:

Next, the latest version of the TOPAZ hazard database was retrieved and was scanned forhazards applicable to AA operations. The resulting list was composed of:

• The list of hazards that had been previously identified for the Extended Free Flightstudy [16]; these have been identified through three brainstorms.

• The list of hazards identified during the EMERTA WP3 project (which also considereda free flight operational concept) [18], through one brainstorm.



Note that the TOPAZ hazard database may also have contained hazards due to AA-OHAidentified safety requirements: it may have happened that some of these safetyrequirements were covered by the Extended Free Flight or the EMERTA WP3 operationalconcepts.

4.2. Comparative analysis of the hazards identified during brainstorm,OHA and TOPAZ hazard database

This subsection provides the results of a comparative analysis of all hazards identifiedduring the ASM project. This analysis was conducted as part of the ASM Extension study[20].

First, each of the four available lists of hazards has been scanned for hazards that appearedto occur more than once in the list. The result is given in the table below.

Group Short name Description of original list Number of hazards that havenot been taken into account foroverlap and type analysis

Effectivenumber ofhazards

1 AA-OHAhazards

The AA-OHA has identified 30hazards which the AA-OHArefers to as 'scenarios', and fromthis has identified 17 operationalhazards.

Five failure scenarios appearedto be equal to theircorresponding operationalhazard, hence five hazardsoccurred twice in the list

42

2 TOPAZbrainstormedAA-hazards

The TOPAZ hazard brainstormhas identified 55 hazards thatmay occur due to AA-OHA-identified safety requirements.

One hazard appeared to occurtwice in the list

54

3 AA-hazardsfrom TOPAZdatabase

2182 hazards (applicable to AA)have been selected from theTOPAZ hazard database(obtained in four previousbrainstorms).

Nine hazards appeared to occurtwice in the list

209

4 TBS-OHAhazards

The TBS-OHA has identified 92hazards which the TBS-OHArefers to as 'scenarios', and fromthis has identified 18 operationalhazards.

No double hazards wereidentified

110

Table 1: The four groups of hazards identified during the ASM project

Next, the hazards from the different sources have been compared on two aspects:

• Types of the individual hazards

• Overlap between groups of hazards

2 Note that the list used for the ASM study contained 233 hazards. For the analysis in this extensionstudy, 15 of these hazards have been omitted, since they appeared to be applicable for Extended FreeFlight, but not for AA-OSED.



Types of hazards identified

For the type analysis, each of the identified hazards in the four groups (see Table 1) hasbeen classified as one of the following types:

• T: technical (e.g. R/T; Navigation; Surveillance; CD&R; distribution of weather details;FMS; other aircraft systems)

• E: human error (which can be considered their "fault", e.g. human mistakes; incorrectlyfollowing procedures, confusion due to unawareness of correct procedures, lack oftraining)

• H: other human related hazards (which can be considered as not necessarily their fault,e.g. situational awareness problems; increased workload; human reaction times)

• P: procedures (e.g. confusing or conflicting procedures; absence of procedures forsituation)

• A: "alien" elements (e.g. weather, intruding military traffic)

The analysis results are given below:

Type AA-OHA hazards TOPAZ brainstormdue to AA-mitigatingmeasures

AA-applicablehazards from TOPAZhazard database

TBS-OHA hazards

T 42 100% 9 17% 88 41% 54 49%E 0 0% 12 22% 60 29% 46 42%H 0 0% 16 30% 21 10% 10 9%P 0 0% 17 31% 22 11% 0 0%A 0 0% 0 0% 18 9% 0 0%Total 42 100% 54 100% 209 100% 110 100%

Table 2: Summary of analysis of hazard types identified during the ASM project

Obviously, these results should be regarded as an example illustration of outcomes for thehazard identification approaches used. Application to other operational concepts, orapplication with other focus in mind (see the difference between the AA-OHA and theTBS-OHA results) may lead to different distributions of hazard types. However, theseresults do indicate that the TOPAZ-based hazard identification approach for ASM hasidentified hazards of types that are complementary to the OHA-identified hazards forASM: The TOPAZ-based approach appears to have identified hazards of types P and A,and more hazards of type H than the OHA approach has.

A likely explanation for this effect is that the purpose of an OHA is to identify hazards atthe service level, e.g.:

− System failures (e.g. Aircraft flying in FFAS with degraded navigation capability),

− Failure to follow procedure correctly (e.g. Crew starts solving conflict before priorityis determined),

− Human failure to respond to these (e.g. Pilot misunderstands ATCo instruction),

− Human error or omission (e.g. Pilot is listening on the wrong frequency).

Hazards that will not easily be identified with an OHA approach, i.e. OHA-unimaginablehazards, are typically the hazards that go beyond the functional level of human tasks.Examples are:



− Pilot is confused by too many rules are mixes up rules with normal operation,

− Flight crews query the (lack of) clearances more frequently due to presence of ASASdisplay,

− Danger of conflict underestimated,

− Pilot cannot explain where he is due to lack of language,

− Confusion due to many sources that provide controller/pilot with information.

Since hazard brainstorms with capable experts as considered in this section incorporate, butdo not restrict to functional failures, they can be used to identify many of these OHA-unimaginable hazards. Of course, there will always be expert or OHA-imaginable hazardsthat stay unidentified until the next brainstorm, or worse: until the next incident or accident.

Hazards overlap

Subsequently, we analysed overlap between the lists of hazards from the different groups.The analysis is restricted to groups 1, 2 and 3 (AA-hazards).

A summary of the results is given in the following figure: The number between bracketsindicates the number of Non-Technical hazards (i.e. of Type E, H, P or A) in the subset.

AA-TOPAZBrainstorm

AA-OHA

AA-TOPAZDatabase

31(0)

0(0)

46 (42)

1(0)

7(3)

10(0)

191(118)

Figure 16: Analysis of overlap between identified AA-hazards within the ASM project

From Figure 16, it appears that there is hardly any overlap between the different lists ofhazards: the approaches appear to provide complementary results. Below, we will evaluateif this complementarity was according to prior expectation or not:

• Overlap between AA-TOPAZ brainstorm hazards and AA-OHA hazards:

No overlap was expected since the AA-OHA has identified hazards due to AA-operations, before the mitigating measures were developed, while the brainstorm hasidentified hazards due to mitigating measures.

• Overlap between AA-TOPAZ database hazards and AA-OHA hazards:

The AA-relevant hazards from the TOPAZ hazard database were based on brainstormsfor ASAS-like operational concepts that had some differences but also many similaritieswith the AA-OSED defined operational concept. Hence, a significant overlap of the setof AA-OHA hazards with the TOPAZ hazard database hazards had been expected.



• Overlap between AA-TOPAZ brainstorm hazards and AA-TOPAZ database hazards:

Although identified using the same method, i.e. brainstorming with experts, the AA-TOPAZ brainstorm and the AA-TOPAZ database hazards had different scopes. Inaddition, it was not anticipated at the time of the brainstorm, to what extent the ASAS-like operational concepts from the TOPAZ database would cover the AA-OHAidentified mitigating measures (on which the AA-TOPAZ brainstorm was focused).

4.3. Lessons learnt from TOPAZ-based hazard brainstorms

This subsection gathers the lessons learnt during the ASM project regarding theidentification of hazards.

Complementarity of hazards

The example results of the ASM hazard type and overlap analysis indicate that theTOPAZ-based brainstorms and the OHA's have identified hazards that werecomplementary to each other. In particular, it occurred that

• The OHA works well in identifying hazards of technical type and can be stretched toalso identify hazards of human error type.

• In addition to technical and human error typed hazards, the TOPAZ-based hazardidentification covers many hazards of the other types as well.

• There appeared to be less overlap between the different hazard lists than expected priorto the TOPAZ-based hazard identification.

The lesson learnt here would be that the use of TOPAZ-based hazard brainstorm would bea very valuable extension to the OSED/OHA approach.

Clustering of hazards

An important lesson learnt regarding the TOPAZ hazard brainstorm is that it was able toidentify a long list of hazards, using the input of only a few operational experts during afew hours only.

However, in order to make use of the fact that hazards identified by one expert may triggerthe identification of other hazards from another expert, the experts were discouraged toanalyse or discard hazards during the brainstorm. A side effect is that the result of such abrainstorm is a list of hazards that are not analysed and clustered.

Depending on what is required for the next step in the safety analysis, a further analysisand clustering could be necessary, after the brainstorm is finished. Note that this clusteringstep was not required during WP3 of the ASM study, though it is a normal step within theTOPAZ methodology. Experience with this clustering step within other projects teachesthat the clustering of hazards of types H, P and A is much more difficult than the clusteringof hazards of types T and E. This is because hazards of types H, P and A often cannot beuniquely connected to functions of technical systems or humans.



Early identification of hazards

Another strong point of hazard identification through brainstorming is that it can be done ata very early stage of the operational concept development process. The operational conceptdescription does not have to be specified in too much technical detail as long as theoperation is clear. Big advantage is that the further operational concept development cantake the identified hazards into account at an early stage, which avoids the need of addinglater on many risk mitigating measures and safety requirements.



5. RELATIONSHIP BETWEEN THE OSA METHODOLOGYAND BRAINSTORM BASED HAZARD IDENTIFICATION

Based on the lessons learnt from the CARE-ASAS ASM study, this section proposes anintegrated and iterative approach for the safety assessment of ASAS applications, based onthe RTCA/EUROCAE OSA methodology and the use of hazard brainstorm.

This approach supports the iterative assessment of operational hazards related to an ASASapplication, with early identification through brainstorm, and stepwise analysis of hazardsat different level of details through successive OHA steps as follows:

• Identification and severity assessment of operational hazards;

• Preliminary frequency assessment of operational hazards;

• Assessment of failure conditions leading to operational hazards.

Along this iterative process, any refinement of the OSED, related or not to some previoussafety assessment, would imply another iteration in the safety assessment of the ASASapplication.

Preliminary assessment of operational hazards

In the first steps of the ASAS application development, the assessment of hazards shouldbe done at the level of operational procedures, based on an OSED focused on the definitionof the air/ground operations and their environment, rather than going into the detail oftechnical systems.

As an early stage, hazard identification brainstorm could be organised to support theidentification of various types of operational hazards, and in particular human-relatedhazards and hazards related to unfavourable environmental conditions.

Due to the unstructured nature of the hazards identified through brainstorm, some furtheranalysis and clustering would then be required to take them into account in a morestructured Operational Hazard Analysis. Note that for many human related hazards, thisclustering cannot be done on a functional basis. Hence, depending on what is required forthe next step in the safety analysis, appropriate methods may have to be studied for thesehazards.

In the perspective of allocating safety operational requirements, preliminary OHA shouldidentify and assess the severity of sequence of events and conditions at the level of ASASoperations, thus allowing for:

• Identification of operational safety influencing factors and successive mitigationsagainst loss of separation (or risk of collision) during ASAS operations;

• Identification of various environmental factors affecting the sequence of operationalhazards;

• If required, the development of the risk mitigation strategy at the level of ASASoperations, with possible impact on the operational procedures or air/ground operationssupporting these procedures.



Preliminary frequency evaluation of operational hazards

If required, preliminary frequency evaluation (e.g. levels of order) of operational hazardscould also be performed in order to support:

• Identification of OHs that would require further investigation (i.e. those OH for whichthe evaluated frequency of occurrence is not compatible with their assigned severity);

• Possible comparison of various risk mitigation strategies, with more detailed analysis ofOHs along the most critical sequences of operational events and conditions;

Assessment of failure conditions leading to operational hazards

Taking into account refinement of the ASAS application and its OSED, the operationalsafety assessment should follow with determination of possible causes of OHs, and theirprobability of occurrences, thus allowing for:

• Identification of both human and technical safety influencing factors (at a lower level ofdetail);

• Possible amendments of the OHA through the more precise evaluation of theirlikelihood of occurrences;

• If required, further development of the risk mitigation measures, with possible impacton the functional characteristics of the air/ground systems.

According to the OSA methodology, this iteration of the OHA should result in theAllocation of Safety Objectives and Requirements for all segments involved in therealisation of the ASAS application.



6. CONCLUSIONS

A critical analysis of the work carried out during the CARE/ASAS Activity 3: ASM studyhas been made focused on the OSED/OHA activities, and hazard identification brainstormperformed as part of the TOPAZ methodology.

On the basis of this analysis, it has been possible to raise some conclusions relative to theapplicability of the RTCA/EUROCAE OSA methodology for ASAS applications, andmore specifically to the adequacy of the OSED/OHA steps for safety assessment of ASASapplications. Contribution of hazard identification brainstorm has also been discussedthrough comparative analysis of hazards identified in the ASM study.

In the perspective of performing an end-to-end safety assessment, the RTCA/EUROCAEOSA guidelines were found particularly relevant for ASAS applications. Nevertheless,some amendments would be required to better address the characteristics and the safetyissues of these ASAS applications. Furthermore, it has been shown that a hazardbrainstorm could identify complementary hazards, already in the early phases of safetyassessment.

Based on the lessons learnt from the CARE-ASAS ASM study, an integrated and iterativeapproach for the safety assessment of ASAS applications, based on the RTCA/EUROCAEOSA methodology and the use of hazard brainstorm has been proposed.

This approach supports for a stepwise analysis of operational hazards, in a first step, at thelevel of the ASAS operations in a given environment, and in a second step, at the level ofsupporting CNS and ASAS functions.

To conclude, some recommendations relative to RTCA/EUROCAE OSED/OSA guidelineshave been made to support future work related to operational safety assessment of ASASapplications.



7. RECOMMENDATIONS

On the basis of the overall work performed during the CARE-ASAS ASM study, thefollowing recommendations need to be considered to support future work related toOperational Safety Assessment of ASAS applications:

R1 RTCA/EUROCAE Operational Safety Assessment (OSA) guidelines should beadapted to better address the characteristics of ASAS applications

R1.1 The OSED guidance material should be amended to allow the description ofnot only D/L communication characteristics, but also those related to airborne surveillanceand separation capabilities, and those related to the operational use of the ASAS functions.

R1.2 In support to the OHA step, proper definition of the “safety margins” (asreferred in the HCM) associated with ASAS operations should be developed to allow theadequate evaluation of the effects of system failures and procedural errors on ASASoperations.

R1.3 In support to the ASOR step, appropriate guidance material should bedeveloped to help people taking into account the essential role of airborne separationcriteria when determining the safety performance requirements.

R2 Guidelines should be developed to support an iterative development of the OSEDof an ASAS application along its development life cycle and safety assessment

R2.1 Initial OSED should be focused on the definition of ASAS operations andtheir environment, since there is no need for too much technical detail during the firststages of the ASAS application development.

R2.2 Taking into account feedback from the preliminary OHA, the OSED should beupdated with any required changes in the operational procedures or air/ground operationssupporting the procedures.

R2.3 When the risk mitigation strategy at the level of ASAS operations isconsidered mature, only then the OSED should be refined with detailed description of thefunctional systems characteristics in order to support the Allocation of Safety Objectivesand Requirements.

R2.4 Taking into account feedback from the more in-depth OHA, the OSED shouldbe updated with any required changes in the functional characteristics of the air/groundsystems supporting ASAS operations.



R3 Guidelines should be developed to support an iterative OHA process combiningboth top-down (or deductive) and bottom-up (or inductive) approach, as well asseverity and frequency assessment

R3.1 Preliminary OHA should be based on the analysis of the sequence of ASASoperations, possible failure of air/ground operations and communications during thesequence and their effects on operations (possibly depending on various alternative riskmitigation strategies).

R3.2 To support identification of human related hazards, preliminary OHA shouldinvolve hazard identification brainstorm session. And, a suitable method should bedeveloped to properly cluster all these hazards.

R3.3 A preliminary frequency assessment (e.g. levels of order) of operationalhazards could also be performed consisting in judgment-based assessment of likelihood ofoccurrences of various OHs, including likelihood of unfavourable environment conditionsaffecting effectiveness of mitigations.

R3.4 Taking into account refinement of the ASAS application and its OSED, theOHA should follow with the “top-down” (or deductive) determination of possible causes ofoperational hazards.

R3.5 More in-depth OHA should rely on the “bottom-up” (or inductive) analysis ofvarious failure conditions related to human and system functions (with identification ofaggravating factors linked to the environment).



8. REFERENCES

[1] RTCA SC-189 DO 264/EUROCAE ED 78 A - “Guidelines for Approval of theProvision and Use of Air Traffic Services Supported by Data Communications” –ED78A – June 2000

[2] CARE-ASAS Activity 3, “OSED Template”, technical note, January 2001

[3] CARE-ASAS Activity 3 - “OHA Template”, Technical Note, version 1.0 – May 2001

[4] CARE-ASAS Activity 3 – Airborne Separation minima – WP1 Technical Results,version 1.3, January 2002

[5] CARE-ASAS Activity 3 – Airborne Separation minima – WP2 Technical Results,Reference CARE/ASAS/Sofréavia/01-016, version 1.1, January 2002

[6] CARE-ASAS Activity 3 – Airborne Separation minima – WP3 report – Estimating safeseparation criteria”, Reference CARE/ASAS/NLR/01-017, version 2.0, January 2002

[7] CARE-ASAS Activity 3 – Airborne Separation minima – Final Report, ReferenceCARE/ASAS/ Sofréavia /01-018, version 2.0, February 2002

[8] FAA/EUROCONTROL COOPERATIVE R&D, “Principles of Operation for the Useof Airborne Separation Assurance Systems”, Version7.1, June 2001

[9] ATM R&D Seminar, “Managing criticality of ASAS applications”, Napoli, June 2000

[10] NUP – Operational Hazard Assessment - Delegated Airborne Separation Approach andClimb-Out, Stockholm-Arlanda, Version 1.0

[11] NUP – Operational Hazard Assessment – Extended Visual Acquisition applied to finalapproach throughput enhancement, Frankfurt, Version 1.0

[12] NUP – Operational Hazard Assessment – Pilot Delegated In-Trail Procedure (ITP) InNon-Radar Oceanic Airspace, Reikjavik, Version 1.0

[13] NUP – Operational Hazard Assessment - Delegated Airborne Separation ClusterControl (DAS-CC) En-Route, Maastricht UAC, Version 1.0

[14] EUROCONTROL ADS Programme – Stage 1 CBA Case Studies – PreliminaryOperational Hazard Assessment, Version 1.0

[15] EUROCONTROL/SCS/SPAF, “Strategic Performance Analysis and Forecast Service –SPF SAFETY report”, Version 1.0, January 2001

[16] Daams, J., Bakker, G.J., Blom, H.A.P. (1999). Safety evaluation of encounters betweenfree-flight equipped aircraft in a dual route structure, NLR-TR-99577

[17] Hoekstra, J.M., Ruigrok, R.C.J., van Gent, R.N.H.W., Visser, J., Gijsbers, B., ValentiClari, M.S.C.V., Heesbeen, W.W.M., Hilburn, B.G., Groeneweg, J.G., Bussink, F.J.L.(2000), Overview of NLR Free Flight project 1997-1999, NLR Technical Paper TP2000-227, May 2000.

[18] Everdij, M.H.C., Bakker, G.J., Blom, H.A.P., EMERTA WP3.2, "Safety/separationmodelling of a particular ASAS application", EMERTA/WP3.2/NLR/MAIN/2.0,Version 2.0, 5 March 2001

[19] Daams, J., Bakker, G.J., Blom, H.A.P., "Safety evaluation of an initial free flightscenario with TOPAZ (Traffic Organization and Perturbation AnalyZer)", NLR-TP-98098, 1998

[20] Everdij, M.H.C., Blom, H.A.P., "Details on hazard type and overlap analysis for theCARE-ASAS activity 3 extension study", NLR memorandum, 2002



ANNEX A: RTCA/EUROCAE OPERATIONAL SAFETYASSESSMENT METHODOLOGY OVERVIEW

A.1 Operational Safety Assessment framework

The RTCA SC189/EUROCAE WG53 methodology [1] provides guidance material:

• To establish the operational, safety, performance, and interoperability requirements forAir Traffic Services supported by data communications,

• To assess their validity, and

• To qualify the related CNS/ATM system.

The methodology considers the allocations of the safety, performance, and interoperabilityrequirements to the elements of the CNS/ATM system. These include ground-basedelements, operational procedures, including human elements, and aircraft equipage.

Evidence ofCoordinated

RequirementsDetermination

Evidence ofApprovalPlanning

Coordinated RequirementsDetermination

OSED(Updated)

OSA• Identification•Coordination•Allocation•Validation

OSEIC• Capture

information• Coordination

IA• Identification•Coordination•Allocation•Validation

OPA•Identification•Coordination•Allocation•Validation

SPR

AllocatedAircraft

Requirements

AllocatedATS ProviderRequirements

AllocatedOperator

OperationalRequirements

Evidence ofAnalysis,

Coordination,& Validation

INTEROP

ExternalInfluences

Approval Plan

Evidence ofOther Processes

(Feedback)

Development& qualification

data

MASPS& MOPS

OSED(Initial)

Figure 17: Co-ordinated Requirements Determination processin RTCA/EUROCAE methodology [1]

In particular, the purpose of the Operational Safety Assessment (OSA) methodology is toidentify safety requirements and allocate them between the various ATM segments ofData-Link applications.

A.2 Operational Service and Environment Definition (OSED)

The OSED is used as a basis for assessing and establishing the safety requirements of theconsidered CNS/ATM system. Its purpose is to obtain the relevant information for the nextOperational Safety Assessment.



In a wider scope, as considered in the RTCA/EUROCAE guidance, the OSED is also usedas a basis for assessing and establishing the performances and interoperabilityrequirements.

The OSED provides a description of the application with its intended operationalenvironment, its associated procedures and the operational and functional expectations ofthe related CNS and ATM means. It may also contain performance expectations andselected technologies of the related CNS/ATM system.

The OSED is an evolving document, updated as necessary to integrate the refinementresulting from the safety assessment (and in a larger scope those resulting from the co-ordinated requirement determination process capturing the performances andinteroperability requirements).

A.3 Operational Hazard Assessment (OHA)

The OHA is part of the Operational Safety Assessment (OSA) which purpose is to establishand allocate the safety objectives and requirements to stakeholders and elements of theCNS/ATM system.

The purpose of the OHA step is to develop an end-to-end qualitative assessment ofpotential hazards that result from a malfunction or failure of one component (either systemor human) of the system.

Based on a high level, but unambiguous and complete, description of the operationalprocedures and airborne/ground functional characteristics, the identification of operationalhazards should be supported by considerations including:

• Functional failure

• Human failure to respond appropriately to functional failure

• Human error or omission during normal use

• Transitional hazards (when changing from exiting to new operations)

• External factors.

The impact of these undesired events has then to be assessed at the aircraft, air trafficservices and operations level. At the same time, mitigation means either procedural orsystem that would reduce the risk of collision in case of an operational hazard, have to beconsidered.

The OSA hazard classification matrix in Table 3 provides a scheme for the severityassignment of each hazard.

HazardClass

1 (most severe) 2 3 4 5 (least severe)

Effect onAircraft

Normally with hullloss. Total loss offlight control, mid-air collision, flightinto terrain or highspeed surfacemovement collision.

Large reduction insafety margins oraircraft functionalcapabilities.

Significant reductionin safety margins oraircraft functionalcapabilities.

Slight reduction insafety margins oraircraft functionalcapabilities.

No effect onoperationalcapabilities orsafety



HazardClass

1 (most severe) 2 3 4 5 (least severe)

Effect onOccupants

Multiple fatalities. Serious or fatalinjury to a smallnumber ofpassengers orcabin crew.

Physical distress,possibly includinginjuries.

Physical discomfort. Inconvenience.

Effect onAir crew

Fatalities orincapacitation.

Physical distress orexcessiveworkload impairsability to performtasks.

Physical discomfort,possibly includinginjuries orsignificant increasein workload.

Slight increase inworkload.

No effect on flightcrew.

Effect onAir TrafficService

Total loss ofseparation.

Large reduction inseparation or atotal loss of airtraffic control for asignificant periodof time.

Significant reductionin separation orsignificant reductionin air traffic controlcapability.

Slight reduction inseparation or in ATCcapability.Significant increasein air trafficcontroller workload.

Slight increase inair traffic controllerworkload.

Table 3: Operational Safety Assessment hazard classification matrix [1]

It is important to consider the severity separately from the event’s likelihood of occurrence.The severity depends only on the effects that the hazard could cause and upon the presenceor absence of mitigating factors.

There is a standard, qualitative relationship illustrated in Figure 18 that gives the greatestlikelihood of occurrence allowed for each level of hazard’s severity (naturally, the moresevere the hazard, the less frequently it is tolerated).

HazardClass

SafetyObjectives Extremely Extremely

Probable Remote Remote Improbable

1

2

3

4

5

Risk Acceptance Cases

Unacceptable Acceptable with Review

Acceptable

Acceptable with Review - Unacceptable withSingle Point Failures and Common-Cause Failures

Figure 18: Relationship between hazard classification and greatest likelihood ofoccurrences [1]



A.4 Allocation of Safety Objectives and Requirements (ASOR)

Based on the OHA results, the ASOR allocates safety objectives to organizations, developsand validates risk mitigation strategies that are shared by multiple organizations, andallocates safety requirements to those organizations.

The concept of the ASOR is to start with the Safety Objectives derived from the OHA anddevelop an agreed strategy to implement these objectives, taking into account proceduraland architectural mitigation to derive a set of safety requirements.

Safety Objectives derived from the OHA level relate to the operational environment andthe required operational functionality and are generally independent of implementation.The ASOR process increases the detail of the risk mitigation strategy to includeconsideration of how the defined safety functions and objectives can be allocated to variousstakeholders.

ATSU

AOCU

ATSUSupporting

ATS

Aircraft

Airspace characteristicsëSeparation minimaëTraffic densityëTraffic complexity

Operationalboundary

ATSboundary

5

5

Failure

Effect on ATS

Procedural error5

5

Hazard

Denotes institutional boundaries

ATS Procedures

OperationalEnvironment

CNS/ATM System

Systemboundary

5

5

Fault

5Fault

Riskmitigationstrategy

Figure 19: Concept Model of Operational Environment, the Relationships AmongFaults, Failures, Procedural Errors, and Airspace Characteristics, and the Risk

Mitigation Strategy [1]

Guidance for allocating safety objectives and requirements is as follows:

a. Identify and assess CNS/ATM system failure relationships. This includes therelationships of system failures, procedural errors, combinations thereof, and theeffects on air traffic services based on the CNS/ATM architecture and the proceduralrequirements provided in the OSED.



b. Identify risk mitigation strategies that are shared by multiple elements of theCNS/ATM system, including mitigation from effects of common cause failures orerrors occurring across system elements.

c. Allocate safety objectives and requirements to stakeholders and elements of the CNSATM System. The level to which allocated safety requirements can be derived fromthe safety objectives is dependent on the degree to which the operational concept hasbeen developed.

d. Coordinate shared safety objectives and requirements across organizationalboundaries. Indeed, one of the principal reasons for conducting the OSA process is toallow for the fact that the various stakeholders implement systems to mitigate a riskthat arises from another stakeholder.



ANNEX B: SPECIFIC USE OF THE OSA GUIDELINES INTHE CARE-ASAS ACTIVITY 3: ASM STUDY

Within the CARE-ASAS Activity 3: ASM project, customised OSED template andtailored-made OHA method have been defined, which have evolved during the project.This framework permitted to get common understanding of OHA process to be performedbetween the various participants of the ASM project in order to identify safety influencingfactors that may impact affect airborne separation minima.

B.1 Operational Service and Environment Definition

From the RTCA/EUROCAE OSED guidance, a tailored template [2] has been derived inorder to provide a common structure to be used in the description of both applications. Thepurpose is to provide the basis for an assessment of system failures, procedural errors andairspace characteristics from which to determine risk mitigation strategy that protectagainst the hazards of defined operational capabilities.

The OSED is composed of the following sections:

1. Operating environment characteristics where the airspace and traffic characteristicsare described. Any relevant information about the structure of the airspace, the ATCservices, the required ground and airborne equipment and the traffic patterns in whichthe application is expected to be defined are provided.

2. Application description, including

- Expected benefits, anticipated constraints, human factors issues associated with theapplication.

- Operating method without and with ASAS sections, where the operationalprocedures are defined.

- Functional characteristics of the airborne and ground systems associated with theapplication.

- Capturing time constraints section, where the sequence of operations are detailed.

- Information exchanges section describing the information exchanged betweencontroller and flight crew or between flight crews. It contains the phraseology andany other messages exchanged.

- Exception handling section describing the human and system failures andemergency/contingency procedures.

- Training section highlighting the issues related to the training of the protagonists.

In the context of the study, no assumptions about infrastructure or technologicalarchitecture are made.

Taking into account the experience gained from the OHA based safety assessment of theautonomous aircraft application, the cooperative separation assurance application has beendescribed at a higher system level with an emphasis put on operation description, in orderto address more in detail human factor issues.

To end, each OSED was updated to integrate the refinements resulting from the safetyassessment conducted on the corresponding application.



B.2 Operational Hazard Assessment method

This section provides an overview of the customised Operational Hazard Assessment(OHA) method [3] based on the RTCA/EUROCAE guidance, and used within the CARE-ASAS Activity 3: ASM study as a basis for conducting the OHA on the two ASASapplications.

B.2.1 ASAS operations description

In order to conduct the OHA based upon sound basis, high level, but as much as possibleunambiguous and exhaustive, description of the sequence in ASAS operations is firstprovided through:

• Sequence definition: When required, this sequence is split into high-level phases ofoperations (or flight).

• Sequence table: This table (as defined in Table 4) provides a formal identification ofdifferent phases (or sub-phases) in the sequence, and conditions that trigger for the nextphase (or sub-phase).

Phase Nextphase

Condition Title Comment

Pi Phase Pi definition

Pj Cn Condition Cn definition

Pk Cm Condition Cm definition

Table 4: Sequence in ASAS operations

• Sequence diagram: For better understanding, a sequence diagram of each ASASapplication is also provided. These diagrams are only intended to provide an overallunderstanding, and should not be considered as complete and detailed state diagrams ofthe ASAS applications under assessment.

Taking into account experience gained from the assessment of the first ASAS application,the OHA method has slightly evolved during the CARE-ASAS Activity 3: ASM project.The main adaptation was on the level of description of the ASAS operations. For thesecond ASAS application, this description identifies more precisely the operations(including communications) of the various ATM components. The purpose was to facilitatethe identification of operational safety-influencing factors and also to assess the air/groundcooperation.

Therefore, the description of second ASAS application under assessment includes:

• Formal identification of the operations (OP) including communications (CO) performedby the various ATM components (or agents), independently from the interactionbetween the human and the airborne/ground systems.

Among the involved ATM components, the following ones have been particularlydistinguished: Controller, ATC (both controller and his tools), flight crew, ASASsystem and flight deck (both flight crew and airborne systems).

• Relationship between these operations (and communications) and the phases in ASASoperations (as defined in Table 5).



Phase(s) No Agent Process (or Communication) Input (or Message)

Pi OPj AK Operational Process OPj definition. Input data description

COm An Communication COm definition Message content description

Table 5: Air/ground operations (and communications) during sequence in ASASoperations

In order to allow the identification of technical safety-influencing factors, the description ofeach ASAS application under assessment also includes:

• Formal identification of the functions (and sub-functions) invoked during ASASoperations, either CNS or ASAS functions, but also functions (or actions) performedmanually by the operators (i.e. flight crews or air traffic controllers).

Function Sub-function

Title Comment

Fi Function Fi definition

Fij Sub-function Fij definition

Table 6: ASAS and other CNS functions

• Relationship between the ASAS/CNS functions (and sub-functions) and the phases inASAS operations (as defined in Table 7). For the second cooperative ASAS application,this relationship has been defined at the level of operations performed by each ATMactor.

Functions

Sub-function

Fi

Phases(oroperationalprocesses)) Fij Fik

Pi Function in use (X) or possibly in use(P) or not used.

Pj (+ Cj)

Pk (+ Ck)

Table 7: Activated functions during the sequence in ASAS operations



B.2.2 Operational Hazard Assessment

The operational hazard identification, severity assignment and determination of safetyinfluencing factors from operational failure scenarios as then made by considering bothfunctional system failures, and procedural errors, such as that involving transactionsbetween the flight crews and ATC, or between the flight crews themselves.

This has been done through an analysis of operational failure scenarios based on thefunctions (or actions) invoked during ASAS operations. More precisely, a set of descriptorsere applied to the hazards related to those functional characteristics and operationalprocedures, so that the effect of various failure modes either system or human-orientedcould be considered.

In particular, it was proposed to distinguish between the following descriptors whendescribing the various failure modes related to system functional characteristics:

• Detected loss or unavailability of function (or information): The information is lost, thefunction is unavailable, but the user can detect the error;

• Undetected loss or unavailability of function (or information): The information is lost,the function is unavailable, but the user does not detect the error;

• Incorrect (or erroneous) function (or information): The function (or information) is notcorrect, but the user can detect the error;

• Misleading function (or information): The function (or information) is not correct, butthe user does not detect the error and uses it as a correct function (or information).

Similarly, it was proposed to distinguish between the following descriptors whendescribing the various failure modes related to operational procedures:

• Lack or inability of action: the action is not performed by the user;

• Incorrect (or erroneous) action: The action is not correct, but either the system or theuser can detect the error;

• Misleading action: The action is not correct, but neither the system nor the user detectsthe error and the user proceeds as if the action was correct.

The impact of these undesired events has then be assessed at the aircraft, air traffic servicesand operations level. At the same time, mitigation means either procedural or system thatwould reduce the risk of collision in case of an operational hazard, have been considered.



PHASE OF OPERATION:Title of the phase (or sub-phase)

PHASE REF: Pp = Reference of the phase (orsub-phase)

OPERATION TITLE:Title of the operation (or communication)

OPERATION REF: OPi = Reference of theoperation (or communication)

OPERATIONAL FAILURE TITLE:Explicit description of the operational failure

OPERATIONAL FAILURE REF: n = Index ofeach operational failure

SCENARIO TITLE:Possible cause (internal or external factors) of the operational failure

SCENARIO REF: p-i-n = Reference of eachoperational failure scenario

Effects

1.Aircraft

2.ATC

3.Operations

The effects of the failures are split on:

1. Effects on the aircraft, its flight crew or its occupants.

2. Effects on air traffic controllers, ATC ground systems or ATC services.

3. Effects on air traffic operations, and the neighbour aircraft.

Detection means

1.aircraft

2.ATC

3.Other aircraft

Possible ways to detect the failure include.

1. Detection by the airborne systems or the flight crew.

2. Detection by the ATC ground systems or the air traffic controller.

3.Detection by the neighbour aircraft or its flight crews.

AssociatedSR, REC andASS

Correctiveactions

1.aircraft

2.ATC

3. Other aircraft

After detection of the failure the corrective action is executed by:

1. The airborne systems or the flight crew.

2. The ATC ground systems or by the air traffic controller.

3. The neighbour aircraft or their flight crew.

AssociatedSR, REC andASS

SafetyClassification

From 5 to 1 according the matrix in Table 3: Operational Safety Assessment hazardclassification matrix

Safety-Requirements(SR)Recommendations(REC)Assumptions(ASS)

List of the safety requirements, assumptions and recommendations used to mitigate orjustify the safety classification.

Operationalhazard

Operational hazard resulting from the operational failure scenario.

Table 8: Operational failure scenario template

After consolidation of the operational failure scenario analysis related to each ASASapplication under assessment, the list of operational hazards identified and assessed duringthe OHA was provided.

OHn Title of operational hazard (to be further described in next section).



Then, each operational hazard is described with:

• Severity assignment according to the OHA classification matrix given in Table 3 (5 =least severe, 1 = most severe).

• List of safety assumptions (ASS), requirements (SR) and recommendations (REC) usedto mitigate (mitigation means) or justify (contributing or avoiding factors) the severityassigned to the operational hazard.

• List of operational failure scenarios that may result in the operational hazard. Actually,the safety requirements and recommendations have been established when analysingthese operational failure scenarios.

During the OHA, distinction is made between safety assumptions, requirements orrecommendations that are introduced to mitigate or justify the operational hazardassessment. Assumptions are directly extracted from the OSED under assessment;requirements and recommendations were developed during the OHA and would requirefurther investigation and agreement before being integrated in an updated version of theOSED.

Title of thehazard

OHn

HazardSeverity

Description of the operational effects RelatedRequirements

RelatedScenarios

Description of the effects of the operational hazard related to thedifferent participants:

1. Effects on the flight crew, the aircraft (and the occupants).

2. Effects on the controller, the ground systems and the Air TrafficServices.

3. Effects on air traffic operations and other aircraft.

ASSi

SRj

RECk

Cf. Table 8

List of assumptions (ASS), requirements (SR) and recommendations (REC) used to mitigate (mitigation means) orjustify (contributing or avoiding factors) the severity assigned to the operational hazard.

ASSi Title of the assumption extracted from the Operational Service and Environment Description(either environmental (ASS/ENV), operational (ASS/OPS) or technical [ASS/SYS] assumption).

SRj Title of the safety requirement (either environmental (SR/ENV), operational (SR/OPS) ortechnical [SR/SYS] requirement).

RECk Title of the safety recommendation (either environmental (REC/ENV), operational(REC/OPS) or technical (REC/SYS) recommendation).

Table 9: Operational hazard description table



Note: Considering that ASAS applications involve at least two aircraft, it is worthwhile todistinguish between the effects on own aircraft, and the neighbour aircraft. The severityassigned to an operational hazard results from the most severe judgement of its effects onair traffic services and operations.

Note: The severity of each operational hazard has been determined taking into account allthe safety assumptions and safety requirements identified during the OHA process. Thesafety recommendations that further mitigate the risk are not necessary to justify theassigned severity.

B.2.3 Safety influencing factors

Within the CARE-ASAS Activity 3: ASM study, the rationale for conducting OHA of thetwo selected ASAS applications is to identify safety-influencing factors that may have animpact on the airborne separation minima.

These safety influencing factors were identified at different levels during the OHA andconsisted in the followings:

• List of operational failure scenarios (per phases of operations) analysed during theOHA, and that may result in operational hazards;

• List of operational hazards (OHs) identified and assessed during the OHA;

• List of safety assumptions (ASSi) extracted from the OSED used to mitigate or justifythe severity assigned to some operational hazards;

• List of safety requirements (SRj) identified during the OHA process and used tomitigate or justify the severity assigned to some operational hazards.

• List of safety recommendations (RECk) identified during the OHA process to furthermitigate the risk, but not used to justify the severity assigned to the operational hazards.

• Overview of the risk mitigation strategy further developed during the OHA process.

*************** END OF DOCUMENT ***************

care/asas activity 3: airborne separation minima ... · relationship between hazard brainstorm and...

Documents