toulouse a-smgcs verification and validation results · emma toulouse a-smgcs verification and...

Contract No. TREN/04/FP6AE/SI2.374991/503192

Project Funded by European Commission, DG TREN The Sixth Framework Programme Strengthening the competitiveness

Contract No. TREN/04/FP6AE/SI2.374991/503192

Project Manager M. Röder

Deutsches Zentrum für Luft und Raumfahrt Lilienthalplatz 7, D-38108 Braunschweig, Germany

Phone: +49 (0) 531 295 3026, Fax: +49 (0) 531 295 2180 email: [email protected]

Web page: http://www.dlr.de/emma

© 2007, - All rights reserved - EMMA Project Partners The reproduction, distribution and utilization of this document as well as the communication of its contents to other without explicit authorization is prohibited. This document and the information contained herein is the property of Deutsches Zentrum für Luft- und Raumfahrt and the EMMA project partners. Offenders will be held liable for the payment of damages. All rights reserved in the event of the grant of a patent, utility model or design. The results and findings described in this document have been elaborated under a contract awarded by the European Commission.

Toulouse A-SMGCS verification and validation results

MONGENIE Olivier & PAUL Stéphane

DSNA & Thales ATM

Document No: D6.4.1 Version No. 1.0

Classification: Public Number of pages: 115

EMMA


Save date: 2007-03-08 Public File Name: D641_Results-TLS_V1.0.doc Version: 1.0

Distribution list

Member Type No. Name POC Distributed1

Internet http://www.dlr.de/emma Web

Intranet https://extsites.dlr.de/fl/emma 1 DLR Joern Jakobi 2 AENA Mario Parra Martínez 3 AIF Marianne Moller 4 SELEX Giuliano D'Auria 5 ANS_CR Miroslav Tykal 6 BAES Stephen Broatch 7 STAR Max Koerte 8 DSNA Thierry Laurent 9 ENAV Antonio Nuzzo 10 NLR Luc de Nijs 11 PAS Alan Gilbert 12 TATM Stéphane Paul 13 THAV Alain Tabard 14 AHA David Gleave 15 AUEB Konstantinos G. Zografos 16 CSL Libor Kurzweil 17 DAV Rolf Schroeder 18 DFS Klaus-Ruediger Täglich 19 EEC Stéphane Dubuisson 20 ERA Jan Hrabanek 21 ETG Thomas Wittig 22 MD Phil Mccarthy 23 SICTA Claudio Vaccaro

Contractor

24 TUD Christoph Vernaleken CSA Karel Muendel

Sub-Contractor N.N.

Customer EC Morten Jensen Additional EUROCONTROL Paul Adamson

1 Please insert an X, when the PoC of a company receives this document. Do not use the date of issue!

EMMA



Document control sheet Project Manager Michael Roeder Responsible Authors MONGENIE Olivier, PAUL Stéphane DSNA, TATM

SAINI Luca TATM MARCOU Nicolas DSNA MONTEBELLO Philippe DSNA

Additional Authors

Subject / Title of Document: Toulouse A-SMGCS verification and validation results Related Task('s): WP6.4 Deliverable No. D6.4.1 Save Date of File: 2007-03-06 Document Version: 1.0 Reference / File Name D641_Results-TLS_V1.0.doc Number of Pages 115 Dissemination Level Public Target Date 2006-01-31

Change control list

Date Release Changed Items/Chapters Comment 2005-06-23 0.01 - Initial draft. 2005-09-02 0.02 §2.3.2 Comments by Holger Neufeldt

and by Marianne Moller. 2005-09-12 0.03 §2.3.2 and §2.3.3 Update of Airbus analysis by

Marianne Moller. 2005-12-07 0.04 New §2.4 Contribution by Holger Neufeldt. 2006-01-16 0.05 New §2.5 Contribution by Luca Saini and

Paolo Gervasoni. 2006-01-20 0.06 New §2.5.2 Answer of FAA on RTCA issue. 2006-03-01 0.07 New §2.6 Temporary internal results, to be

updated. 2006-03-31 - Parallel D641 document created by

DSNA for validation results. Initial draft validation results by DSNA. Release v0.01 is delivered to TATM for review.

2006-04-04 0.08 New §2.7 Time stamp issue. 2006-04-14 - The TATM editorial comments to the

parallel D641 document created by DSNA, release v0.01, are implemented. The results of the shadow-mode trials are completed. A fast-time simulations chapter is added.

By DSNA. Release v0.02 is delivered to TATM for review.

2006-04-21 - Editorial and content improvement. By DSNA. Release v0.03 is

EMMA



delivered to DLR for review and to TATM for integration of the verification results.

2006-04-27 0.09 Integrated parallel D641 document created by DSNA for validation results in its release v0.03. Transfer of the analysis tables for the shadow-mode validation trials in an annex. Editorial improvement. Check of acronym list. Suppression of figure 15. New verification tests, re-organisation of chapters and consolidation with V&V test plan.

Submission to DSNA and Luca Saini for review.

2006-05-09 0.10 Small comment about MOGADOR. Editorial improvements.

Submission to DLR for general assembly review and/or submission to EC.

2006-12-07 0.11 Update of all the document by DSNA following EC review.

Submission by DSNA to TATM.

2007-02-26 0.12 Update of all the document by Luca Saini following EC review.

2007-03-06 0.13 Processing of remaining comments from EC by Philippe Montebello and Stéphane Paul.

2007-03-08 1.0 Approval by EC.

EMMA



Table of contents Distribution list........................................................................................................................................ 2 Document control sheet........................................................................................................................... 3 Change control list .................................................................................................................................. 3 Table of contents ..................................................................................................................................... 5 1 Scope .................................................................................................................................................... 7

1.1 Identification ................................................................................................................................. 7 1.2 Project overview............................................................................................................................ 7 1.3 Document overview ...................................................................................................................... 7

1.3.1 Purpose ................................................................................................................................... 7 1.3.2 Intended audience................................................................................................................... 8 1.3.3 Document structure ................................................................................................................ 8

1.4 Relationships to other EMMA documents .................................................................................... 8 2 Verification........................................................................................................................................... 9

2.1 Mosquito position precision .......................................................................................................... 9 2.1.1 System configuration.............................................................................................................. 9 2.1.2 Results .................................................................................................................................... 9 2.1.3 On-the-fly analysis ................................................................................................................. 9 2.1.4 Conclusion............................................................................................................................ 10 2.1.5 RTCA DO-260A vol-1 bug on ADS-B surface position reporting ...................................... 10

2.2 ADS-B ground station coverage.................................................................................................. 11 2.2.1 Results .................................................................................................................................. 12 2.2.2 On-the-fly analysis ............................................................................................................... 12

2.3 Unstable track positions for some aircraft................................................................................... 12 2.3.1 Results .................................................................................................................................. 12 2.3.2 On-the-fly analysis ............................................................................................................... 13 2.3.3 Conclusion............................................................................................................................ 14

2.4 MAGS preliminary verification results ....................................................................................... 14 2.4.1 System configuration............................................................................................................ 14 2.4.2 Result of step (e) – system calibration ................................................................................. 15 2.4.3 Result of step (f) – localisation of test transmitter................................................................ 16 2.4.4 Results of step (g) – localisation of real targets ................................................................... 16

2.5 Surveillance latency and position precision ................................................................................ 18 2.5.1 First test: latency & position verification with a vehicle...................................................... 18 2.5.2 Second test: latency verification with an aircraft ................................................................. 20 2.5.3 Third test: dispersion verification......................................................................................... 21 2.5.4 On-the-fly analysis of the 3 tests .......................................................................................... 22 2.5.5 Conclusion............................................................................................................................ 22

2.6 ADS-B stations synchronisation: time drift test .......................................................................... 23 2.6.1 The test set-up ...................................................................................................................... 23 2.6.2 On-the-fly analysis ............................................................................................................... 24 2.6.3 Detailed analysis and issue solving ...................................................................................... 24 2.6.4 Conclusion and more (interesting) results ............................................................................ 24

2.7 ADS-B latency test ...................................................................................................................... 24 2.8 Test of the probability of detection of a stationary vehicle ......................................................... 24

2.8.1 System configuration............................................................................................................ 25 2.8.2 Findings ................................................................................................................................ 25

2.9 ADS-B cost-benefit test............................................................................................................... 25 2.9.1 System configuration............................................................................................................ 26 2.9.2 Findings ................................................................................................................................ 26

2.10 Consolidation of verification results ......................................................................................... 26 2.10.1 Surveillance ........................................................................................................................ 26 2.10.2 Control................................................................................................................................ 27

EMMA



2.10.3 Human machine interface................................................................................................... 27 3 Validation session – shadow mode trials............................................................................................ 28

3.1 Introduction ................................................................................................................................. 28 3.2 Data description and data collection methods ............................................................................. 28

3.2.1 Raw data ............................................................................................................................... 28 3.2.2 Additional data ..................................................................................................................... 29

3.3 Data analysis................................................................................................................................ 30 3.3.1 Data analysis method............................................................................................................ 30 3.3.2 Analysis tables for the shadow-mode validation trials......................................................... 31

3.4 Results ......................................................................................................................................... 31 3.4.1 Operational feasibility: acceptance of technical performances ............................................ 31 3.4.2 Operational feasibility: procedures....................................................................................... 43 3.4.3 Operational improvements ................................................................................................... 46

4 Validation session – fast-time simulations ......................................................................................... 52 4.1 Introduction ................................................................................................................................. 52 4.2 Data description and data collection methods ............................................................................. 52

4.2.1 Raw data ............................................................................................................................... 52 4.2.2 Additional data ..................................................................................................................... 53

4.3 Data analysis................................................................................................................................ 54 4.3.1 Data analysis method............................................................................................................ 54 4.3.2 Measured indicators ............................................................................................................. 56

4.4 Results ......................................................................................................................................... 60 4.4.1 Operational improvements ................................................................................................... 60

5 Digest and conclusions....................................................................................................................... 68 6 Annexes.............................................................................................................................................. 69

6.1 Annex A: analysis tables for the shadow-mode validation trials ................................................ 69 6.1.1 Operational feasibility: acceptance of technical performances ............................................ 70 6.1.2 Operational feasibility: procedures....................................................................................... 91 6.1.3 Operational improvements ................................................................................................. 110

6.2 Annex B: references .................................................................................................................. 113 6.2.1 Applicable documents ........................................................................................................ 113 6.2.2 Referenced documents ....................................................................................................... 113

6.3 Annex C: abbreviations ............................................................................................................. 113

EMMA



1 Scope

1.1 Identification This document provides the Toulouse A-SMGCS verification and validation results. • Document Name: Toulouse A-SMGCS verification and validation results • EMMA No.: D6.4.1 • Revision: 0.13 • File Name: d641v013.doc

1.2 Project overview The project is named “European Airport Movement Management by A-SMGCS” with the acronym EMMA. The duration of the project is 2 years, with a follow-up in EMMA-2 (another 2 years). The project is organised in six different sub-projects. There are three ground-related sub-projects and one onboard-related sub-project. Based on an advanced operational concept, three functional level III advanced surface movement, guidance and control systems (A-SMGCS) will be implemented at three European airports: Prague-Ruzynĕ, Toulouse-Blagnac and Milano-Malpensa. The systems are to be tested operationally (i.e. with live traffic). The three ground-related sub-projects and the onboard-related sub-project are autonomous, but are inter-linked with the sub-projects ‘concept’ and ‘validation’ to guarantee that the different systems are based on a common A-SMGCS interoperable air-ground co-operation concept and that all are validated with the same criteria. On-site long-term trials (i.e. 6 months to one year) are to ensure the assessment of benefit estimations. The results of the test phase shall feed back to the concept of operations, and are intended to set standards for future implementation in terms of: (a) common operational procedures, (b) common technical and operational system performance, (c) common safety requirements, and (d) common standards of interoperability with other ATM systems. These standards shall feed the relevant documents of international organisations involved in the specification of A-SMGCS, i.e. mainly ICAO, EUROCAE, and EUROCONTROL. This document is produced in the scope of work package 6.4 entitled "Verification and Validation at Toulouse". The activities performed in this work package 6.4 include: • preparation of the infrastructure at Toulouse-Blagnac airport for verification and validation; • technical tests (i.e. verification) of the A-SMGCS installed in Toulouse-Blagnac; • training of Toulouse-Blagnac controllers to the A-SMGCS; • shadow-mode trials with trained controllers on the A-SMGCS HMI installed in the Toulouse-

Blagnac tower to evaluate the operational feasibility of the A-SMGCS and assess the operational improvements brought by the system;

• fast-time simulations of several traffic and environment conditions in Toulouse-Blagnac to quantify the operational improvements potentially brought by the A-SMGCS.

1.3 Document overview

1.3.1 Purpose The ICAO manual [15] on Advanced Surface Movement Guidance and Control Systems (A-SMGCS) and the EUROCAE WG-41 MASPS [16] for A-SMGCS contain operational and performance requirements that are considered to be necessary in the process of selection, development and introduction of A-SMGCS. These manuals have been defined for those aerodromes where current SMGCS needs to be upgraded, or for aerodromes which currently have no SMGCS, but where the traffic density and/or aerodrome layout requires so. The objective of this document is to report on the collected data aimed at supporting the assessment of the surveillance, surface conflict alerting and routing functions as implemented in Toulouse-Blagnac

EMMA



against the aforementioned ICAO and EUROCAE requirements, in conformance to [3], [4], [7] and [14]. Due to some delays in the implementation of the Toulouse-Blagnac A-SMGCS test bed (cf. [2]) it was not possible to run all the tests foreseen in [5]. The corresponding implications for the overall project plan, as well as the plans foreseen by the project, in general, to cope with such a non-conformance are described within each test description. However, many tests were run, and some significant results were obtained, as documented within.

1.3.2 Intended audience The dissemination of this document is public.

1.3.3 Document structure Chapter 1 defines the scope of this document, the intended audience, and the document structure. Chapter 2 presents the verification results. Verification is testing against predefined technical specifications, i.e. technical functional testing: "did we build the system right?" Each section of this chapter presents a short description of the experiment set-up, an overview of the raw data that has been gathered, and different stages of analysis by the different partners involved. Chapter 3 and 4 present the validation results obtained respectively through shadow-mode trials and fast-time simulations. Validation is testing against operational requirements as defined by stakeholders and written down in [6], i.e. "did we build the right system?" The chapters describe the data, data collection methods, data analysis and results measured in terms of operational feasibility and operational improvements. Chapter 5 provides the conclusions drawn from the V&V activities in Toulouse-Blagnac airport. Chapter 6 provides details on referenced documents and some definitions.

1.4 Relationships to other EMMA documents Apart from the EMMA documents already referenced above, the author would like to bring the reader's attention to: • similar result collection reports for Prague-Ruzynĕ [8] and Milano-Malpensa airports [9], • a similar result collection report for airborne systems [10], • the verification and validation analysis report [11], • the verification and validation recommendation report [12]. Public EMMA documents can be found on www.dlr.de/emma.

EMMA



2 Verification Due to delays in the A-SMGCS implementation at Toulouse-Blagnac, it was not possible to thoroughly follow the V&V test plan, as documented in D6.1.3 [5]. However, some of the verification tests provided below in §2.1 to §2.9 and the site acceptance tests performed for each of the A-SMGCS sub-system elements (cf. D4.1.1, D4.12, D4.1.3, D4.2.1, D4.3.1, D4.4.1 and D4.4.2) allow us to take a stand on a number of technical hypotheses, as described below in §2.10, in particular where the surveillance, control and human-machine interface hypotheses are concerned. The D6.1.3 Toulouse-Blagnac V&V Test Plan [5] planned that the MOGADOR tool would be used in order to assess detection and identification indicators on a long-term basis from EMMA A-SMGCS recorded data. The recorded EMMA A-SMGCS SDF data (i.e. one week sample recorded in February 2006) was input into MOGADOR. However, the results obtained were abnormally bad and a very low number of tracks could be actually analysed. This is explained by the important number of lack of identifier and the poor continuity of the correlation function (i.e. MOGADOR uses the flight call sign as a key element to associate and identify tracks) and the need for tuning of MOGADOR to Toulouse-Blagnac platform that would necessitate additional time and effort. Consequently, since the results of the MOGADOR analysis do not reflect the reality, it was decided not to publish them in this document. The impossibility to perform a long-term analysis has a significant impact on the verification phase. In particular, the detection and identification performance of EMMA A-SMGCS will not be fully verified in Toulouse-Blagnac. Therefore, an additional campaign of measures will have to be performed as a preliminary to EMMA2 validation activity.

2.1 Mosquito position precision In June 2005, Laurent Volkmann (DSNA) calculated the A-SMGCS reported position accuracy (RPA), by comparing the Mosquito reported position to the D-GPS reported position.

2.1.1 System configuration For the test, the following set-up was prepared: • full automatic dependant surveillance broadcast (ADS-B) system installed, up and running; • ADS-B ground station (GS) configured to emit ASTERIX reports as soon as they receive 1090ES

message updates from ADS-B targets; • Mosquito installed on a vehicle, up and running; • the same vehicle mounted with a D-GPS independent receiver, used as a reference; • ELVIRA recording system connected to the system LAN.

2.1.2 Results When both Mosquito and D-GPS track trajectories are superimposed on the ELVIRA display (cf. Figure 1), the reported position accuracy (RPA) appeared to be very good, but the calculation gave bad results because of the ADS-B GS time stamp. See picture below.

2.1.3 On-the-fly analysis Fact 1: The D-GPS has an update rate of 5 per second. The GPS inside Mosquito has an update rate of 1 per second. When the mobile is moving, it is assumed that the position sent by the GPS inside Mosquito is an average position over the last second. This introduces some latency.

EMMA



Fact 2: Mosquito does not time stamp as no time stamp can be sent in ADS-B position messages. The first time stamp is performed during the ADS-B ASTERIX message formatting by the ADS-B ground station and reflects the ASTERIX message emission date. Fact 3: During the tests and in accordance with the ADS-B standard, each Mosquito/GPS position data was sent twice per second (with 0.5s interval), whilst the odd and even coding/decoding of the position data differed. This explains on Figure 1: Mosquito RPA on ELVIRA display the presentation of Mosquito positions as tight-couples (i.e. in fact the same data) separated by 0.5 seconds. Fact 4: In June 2005, the Mosquito equipment did not perform any position extrapolation to compensate the latency.

2.1.4 Conclusion Thales ATM implemented a position prediction algorithm in order to extrapolate the 1st message position and the 2nd message position by different factors. The position prediction algorithm was derived from the ADS-B standard (RTCA-260A).

Figure 1: Mosquito RPA on ELVIRA display

2.1.5 RTCA DO-260A vol-1 bug on ADS-B surface position reporting Thales ATM implemented inside the Mosquito vehicle ADS-B transmitter an algorithm suggested by RTCA D0-260A for GPS position prediction by ADS-B transmitting mobiles. This algorithm aims at compensating some latency effects that can affect the reported position accuracy. During the implementation activities, Luca Saini and Paolo Gervasoni identified a bug on a formula in the RTCA D0-260A volume 1 document. The bug details are reported below. Document reference: Minimum operational performance standards for 1090 MHz extended squitter, automatic dependant surveillance broadcast (ADS-B) and traffic information service - broadcast (TIS-B), RTCA DO-260A volume 1, 10th April 2003, Section: 2.2.3.2.4: ADS-B Surface Position Message,

EMMA



Sub-sections: 2.2.3.2.4.8.2 and 2.2.3.2.4.8.3 Surface Longitude Position, Extrapolation/Estimation, precision and non-precision case. In the above mentioned sections there is twice reported a prediction formula of a longitude value in order to compensate for the time latency between the time of last received update position fix and the time of the applicability of the ADS-B surface position message. In such a formula the variable called "approximate longitude" is defined as the "longitude at the time of the fix". The mistake is found in that this must be the latitude at the time of the fix and not the longitude. Note that the corresponding formulas, as reported in the sections for the airborne position messages longitude extrapolation/estimation are correctly typed. Two simple examples can describe some effects of the mistake. Case 1: a target moving along the equator in the E-W direction at a constant velocity. In such a case it's expected to obtain a constant longitude predicted delta for the same delta time. Applying the formula would result in delta longitude predicted values that strongly depend on the target longitude position. In world areas close to plus or minus 90° this delta becomes… infinite! Case 2: a target is moving in world areas with longitude > + 90° or < - 90°. In such a case the formula is predicting a negative deltas of longitude (i.e. longitude decreasing) when the target is moving towards east (longitude increasing) and vice-versa. RTCA and EUROCAE were contacted to communicate the finding. The following message was received from FAA, acknowledging our finding. -----Original Message----- From: [email protected] [mailto:[email protected]] Sent: Thursday, January 19, 2006 6:11 PM To: […] Subject: Re: DO-260A bug on ADS-B Surface Position Reporting Gentlemen, After a review of the problem that was initially reported below, we agree that there is a typographical error in the text of RTCA/DO-260A in the two places indicated for Surface Position reporting in the "Commentary" sections of 2.2.3.2.4.8.2.1 and 2.2.3.2.4.8.3.1, where longitude is indicated in the definition of the symbol "Phi" when in fact the latitude should have been indicated. I deeply regret that these errors were entered into the DO-260A document, and cannot understand how a simple cut-and-paste from DO-260 to DO-260A could have occurred, but we are pleased that the error has been found and can be easily corrected. To the end of making the correction to these, and other errors identified in DO-260A, I have added the corrected text for these two errors into the proposed "Change 1 to DO-260A" document that was initially reviewed during Meeting #19 of the RTCA SC-186 Working Group 3 (WG3) in December 2005 as Working Paper 1090-WP19-06. After discussions of how the proposed changes to both DO-260 and DO-260A might effect the 1090ES SARPs Technical Manual, during the ICAO SCRSP TSG meeting in Fort Lauderdale 2 - 10 February 2006, WG3 will finalize and submit "Change 1" documents for both DO-260 and DO-260A to RTCA SC-186 membership for their Final Review and Comment period, which ends with the SC-186 Plenary meeting in Washington on 20 April, where WG3 expects that both "Change 1" documents will be approved for publication on the RTCA web site. […] Gary Furr L-3 Communications / The Titan Group FAA Technical Center

2.2 ADS-B ground station coverage In June 2005, Luca Saini performed some ADS-B ground station output recordings. At that time, the equipment configuration was as follows: two ADS-B ground stations, one at the GBAS location and one at the top of the new tower.

EMMA



2.2.1 Results The main results were as follows: • Both ADS-B ground stations showed good surveillance coverage relating to aircraft; • The new tower ADS-B ground station showed good performance, with a few systematic holes

(probably due to reflections), concerning vehicle tracking with Mosquito; • The GBAS ADS-B ground station showed very poor coverage performance concerning vehicle

tracking with Mosquito.

2.2.2 On-the-fly analysis The poor performance of the GBAS ground station relating to vehicle tracking is difficult to understand. Indeed the GBAS ground station is more powerful and has an additional low noise amplifier compared to the new tower ADS-B ground station. It is currently thought that the low height of the antenna makes it more subject to reflection problems.

2.3 Unstable track positions for some aircraft In June 2005, Luca Saini performed some ADS-B ground station output recordings. At the time, the equipment configuration was as follows: two ADS-B ground stations, one at the GBAS location and one at the top of the new tower.

2.3.1 Results A surprising result was the unstable track positions reported for some aircraft, mainly Air France aircraft, whilst the EMMA Mosquito track showed very stable trajectories, as well as some other aircraft (EasyJet, Airbus A380, etc.) – see figures below.

Figure 2: Mosquito (stable) vehicle tracking

EMMA



Figure 3: Unstable aircraft tracks during pushback

Figure 4: Unstable aircraft tracks during taxi

2.3.2 On-the-fly analysis For aircraft with unstable target reports, there seemed to be a systematic jump between odd and even target position reports. This is thought to be related to the encoding of the positions, using different algorithms for odd and even reports. But why is this phenomenon related only to some aircraft?

2.3.2.1 Comments by Thales ATM Regarding the accuracy of aircraft, it is to be noted that within the ADS-B ASTERIX cat 21 message, the ground stations forward a Figure of Merit / Position Accuracy indicator (FOM/PA). The higher this number, the better the accuracy. A FOM/PA of 7 or above is good, 6 or less is poor. The reason

EMMA



for poor quality is usually the current GPS reception quality or constellation as perceived by the onboard GPS receiver. Some older types do not deliver adequate position quality. It is also to be noted that there is a number of other misbehaving equipment sets flying around (including sudden jumps backwards along track, jumps to the north pole or another distant position etc.) These are known issues in some manufacturers equipment that are caused by, for example: • onboard equipment that – whilst outputting ADS-B formats – is not certified for ADS-B use, • a problem in the internal time synchronisation function of the transponder. When analysing ADS-B reports one must therefore be extremely careful in describing what actually happened. ADS-B is just a data link mechanism. The positions are measured by avionics and the ADS-B receiver depends on this information (hence the name "dependant surveillance").

2.3.2.2 Comments by Airbus The above results were provided to Airbus, who gave the following preliminary analysis. Air France has not yet fully equipped all its airliners with the package enabling GPS data transmission via ADS-B. In such a case, the resolution of the position transmitted via ADS-B can lead to jumps around the aircraft trajectory. The observed jumps should progressively disappear2 whilst Air France completes the installation.

2.3.3 Conclusion Thales ATM decided to suppress all ADS-B tracks whose Figure of Merit / Position Accuracy indicator (FOM/PA) was strictly lower than 7. In addition, this result is interesting on a safety assessment point of view. The observed behaviour is linked to an intermediate configuration in the frame of the ADS-B deployment. It would be interesting to know if an on-board equipment failure of the final configuration might lead to a similar fallback behaviour. If so, it might be interesting to analyse it in future releases of the EMMA functional hazard assessment [13].

2.4 MAGS preliminary verification results An important part of the EMMA installation in Toulouse is the MAGS multilateration. Multilateration is a complex technology whose implementation on an actual airport requires a great deal of tuning. For the installation of MAGS at Toulouse-Blagnac airport within the frame of the EMMA programme, the initial tuning phase has yielded some preliminary verification results that are presented in this section. The system installation and tuning consists of the following steps: (a) installation of equipment; (b) setting up of communication links; (c) tuning of ground station sensitivity for the intended coverage area; (d) verification of probability of detection of ground stations; (e) configuration of calibration mechanism to establish a common time base for all ground stations; (f) parameter tuning to locate the fixed test transmitter; (g) parameter tuning to locate targets; (h) definition of areas and their properties; (i) optimisation of plot validation and tracking parameters; (j) integration with sensor data fusion. The results presented correspond to steps (e), (f) and (g). For this purpose – after an initial on-site tuning session - data were recorded from the MAGS and taken to the lab. These data were then replayed in real time to the reference system in the lab and its parameters were configured to obtain the best possible performance. The following sections present some intermediate results.

2.4.1 System configuration For the MAGS system in Toulouse, a set of five ADS-B/MLAT ground stations were installed as depicted in Figure 5. One ground station was additionally equipped with a calibrator module to 2 If it were not the case, a new analysis would be required.

EMMA



provide system synchronisation, and another ground station was additionally equipped with an interrogator module to allow interrogation of aircraft. A fixed test transmitter based on a Mosquito ADS-B transmitter was installed as a permanent, fixed test target.

1

2

3

5

6

1

2

3

4

5

6

GSC(TWR)

GSR(Old TWR) GSR

(GBAS)

GSI(SMR)

GSR(PC)

Test Transmitter(DF)

1

2

3

5

6

1

2

3

4

5

6

GSC(TWR)

GSR(Old TWR) GSR

(GBAS)

GSI(SMR)

GSR(PC)

Test Transmitter(DF)

Figure 5: MAGS Configuration at Toulouse Blagnac

2.4.2 Result of step (e) – system calibration Calibration results are best illustrated by locating the calibrator transmitter using multilateration. The result is depicted in Figure 6. The filled red square indicates the ground station calibrator (GSC) position, while the outlined red squares are 10m, 20m, and 30m, respectively, around the GSC position.

Figure 6: Measured position of MAGS calibrator station (GSC) at control tower

The GSC position measured by multilateration is shown as sequence of target plots on the MAGS technical display. Due to ASTERIX cat-10 quantisation, the position resolution is limited to 1m (i.e. distance between adjacent plots). All position reports are within 3 metres of the exact GSC position.

EMMA



2.4.3 Result of step (f) – localisation of test transmitter Figure 7 shows the resulting plot sequence of locating the fixed test transmitter (TT) by multilateration. Again the minimum distance between adjacent plots is determined by the ASTERIX position resolution of 1m. All position reports are within 4 metres of the exact TT position.

Figure 7: Measured position sequence of fixed test transmitter

2.4.4 Results of step (g) – localisation of real targets The following two screenshots in Figure 8 and Figure 9 show real targets detected via multilateration together with the GSC and the test transmitter. Track continuity was not ensured. Indeed, no tracker was yet configured (as this is a task within the definition of area properties). Therefore tracks were neither smoothed nor fully regular in their update rate. It should be noted that, at that time, MAGS did not interrogate aircraft, so that multilateration relies purely on passive reception. Active interrogation of targets can be used to enhance update rates and to fill gaps.

EMMA



Figure 8: Targets, GSC (label “3C3D0C”) and test transmitter (label “3C3D0D”) detected by MAGS

Missing detections can be caused by coverage gaps. It is clear that five ground stations in this configuration are not ideal in terms of redundancy. For example, if one ground station (e.g. the one installed at the SMR, cf. Figure 5) does not contribute to a plot data set, the geometrical constellation is not good enough for parts of the coverage area.

Figure 9: Targets, GSC (label “3C3D0C”) and test transmitter (label “3C3D0D”) detected by MAGS

The next screenshot, in Figure 10, shows one target that taxies onto the apron area. It should be noted that no tracker was yet configured to smoothen the track and to provide regular updates.

EMMA



Figure 10: One target picked out

2.5 Surveillance latency and position precision On Thursday 23rd February 2006, a DSNA/DTI team (headed by Mr. Philippe Soleilhac) performed a surveillance latency and reported position accuracy verification session. The test scope included the vehicle tracking system (Mosquito), the automatic dependant surveillance broadcast sensor (AS-680) and the sensor data fusion (SDF).

2.5.1 First test: latency & position verification with a vehicle The verification was performed using the latest Mosquito equipment delivered week 7 of year 2006. Tests were performed by driving a Mosquito-equipped car on service roads. The car was detected only by the automatic dependant surveillance broadcast (ADS-B) sensor and the surface movement radar (because the multilateration system was shut down). A very precise differential global positioning system (D-GPS) was used as time and position reference.

EMMA



Figure 11: Mosquito latency & precision verification test at ADS-B sensor output

The comparison between the D-GPS reports and the Mosquito reports as output directly by the ADS-B sensor located at the new control tower (cf. Figure 11) showed a 1.2s latency and a position error of 12 metres.

EMMA



Figure 12: Mosquito latency & precision verification test at sensor data fusion output

The comparison between the differential global positioning system (D-GPS) reports and the Mosquito reports as output by the sensor data fusion (cf. Figure 12) showed a 2s latency and a position error of 6 metres.

2.5.2 Second test: latency verification with an aircraft This test was performed with a flying aircraft. The approach secondary surveillance radar (called DACOTA) was used as reference system.

EMMA



Figure 13: Air surveillance latency & precision verification test

The comparison between the DACOTA target reports and the EMMA target reports as output by the ADS-B sensor located at the new control tower and the sensor data fusion showed a 0.3s latency.

2.5.3 Third test: dispersion verification The EMMA sensor data fusion (SDF) introduced some target report dispersion, which is not originating from any sensor.

EMMA



Figure 14: Dispersion test with flying aircraft

2.5.4 On-the-fly analysis of the 3 tests The offset of 12 m between the ADS-B output trajectory of Mosquito and the D-GPS recording is due to a bug inside the Mosquito software. The latency of 1.2 seconds is affected by the same error. A new software release has been produced. The "improvement" performed by the SDF output on the trajectory (from 12 m to 6 m) is only due to the fact that the SMR output is much more precise than the ADS-B output for Mosquito. Fixing the Mosquito problem should allow to reduce this offset. The crazy behaviour of the SDF output for the flying aircraft has an explanation and is due to two probable contributions: 1) The SDF does not fuse the ADS-B output (which is very good in this case) because the ADS-B emitting transponder does not deliver a figure of merit better than 6 (i.e. 7 or 8 or 9). 2) The EMMA SDF, as far as the DACOTA GTW tracker (alfa, beta and gamma filter) is concerned, is configured in such a way that aircraft entering pre-defined geometrical areas (cones) associated to each RWY are kept with a stronger weight on the current direction, at the time the aircraft enters the cone. This setting is performed in order, for a landing airborne aircraft, to "follow" the RWY centreline. The case described above was a test flight entering the cone in a perpendicular direction from what could be expected. As this setting is also creating problems in case aircraft land in "baïonnette" mode, the tracker's filter values were changed. The new proposed values are not giving any "preferred" direction on the tracker.

2.5.5 Conclusion In case a test aircraft will be used for EMMA verification, it should be better that this aircraft is delivering a good Figure of Merit, otherwise the SDF will not fuse the ADS-B output.

EMMA



2.6 ADS-B stations synchronisation: time drift test On Thursday 9th March 2006, a DSNA/DTI team (headed by M Philippe Soleilhac) performed timestamp tests related to Mosquito and ADS-B stations. With Mosquito, the position is elaborated onboard the vehicle (using the GPS), but the time is stamped by the ADS-B ground station. The ADS-B standard does not allow for the transmission of the time together with the position.

2.6.1 The test set-up Tests were performed by driving a D-GPS and Mosquito-equipped car on service roads. Positioning was performed via the D-GPS, the ADS-B ground stations and the mode S multilateration.

OTWR PCBLD NTWRSMR GBAS

ADS-B GTW

multilateration

SDFELVIRA

SAF Clock

NTP Synchro

Asterix 62

Asterix 10

Asterix 62Asterix 21

Asterix 21

GPS

Vehicle TX

1090ES ADS-B

Precision GPS RX

GPS Position & TimeOTWROTWROTWR PCBLDPCBLDPCBLD NTWRNTWRNTWRSMRSMRSMR GBASGBASGBAS

ADS-B GTWADS-B GTW

multilaterationmultilateration

SDFELVIRAELVIRA

SAF Clock

NTP Synchro

Asterix 62

Asterix 10

Asterix 62Asterix 21

Asterix 21

GPS

Vehicle TXVehicle TX

1090ES ADS-B

Precision GPS RXPrecision GPS RX

GPS Position & Time

Figure 15: Position and timestamp test set-up

ASTERIX-21 messages are time-stamped by ELVIRA with a tenth of a second accuracy. The following table shows timestamp discrepancies between the ADS-B ground stations.

ADS-B ground station

Time in the ASTERIX messages stamped by the ADS-B stations

Time stamped by ELVIRA upon message reception

New TWR 12.34.02 12.14.68 PC 2 12.15.40 12.14.77 SMR 12.14.67 12.14.68

Old TWR 12.15.26 12.14.75

Table 2-1: Timestamp measurements

Times in the ASTERIX-21 messages, stamped by the ADS-B stations, are in advance, with a dramatic 20 seconds for the ADS-B ground station located at the new control tower. Time in the ASTERIX-62 messages, stamped by the SDF, is also in advance, but with a smaller amount (between 0.7s and 1s).

EMMA



2.6.2 On-the-fly analysis The ADS-B ground stations are normally synchronised, between themselves and with the real time, using NTP. It seems that the implementation of NTP in this case was not effective, with a detrimental effect on latency compensation (at ADS-B gateway level) and position accuracy.

2.6.3 Detailed analysis and issue solving According to Luca Saini, the poor latency performances are essentially related to three sub-systems: • Mosquito; • the ADS-B GS; • the SDF (at ADS-B GTW level). With respect to Mosquito, the main problem is due to the fact that the internal processing cycle is not synchronised with the GPS receiver output. TATM is working on this feature. However, to solve definitively the full latency problem, it is required to introduce additional features that at least compensate each known delay. Specific requirements have been added to the software requirements specification (SRS). See below for more details. With respect to the ADS-B GS, the main problem is the synchronisation using NTP (SDF being the master). After investigating the network time protocol (NTP) effect, it was discovered that the issue was in fact a human error: the NTP server was not reachable on the network. The NTP service, initially located on the ADS-B gateway PC had been transferred to the SDF PC. Thus, the NTP server IP address had changed. The ground stations were re-configured to synchronise with the current NTP server. With respect to the ADS-B GTW, one additional source of latency has been identified, i.e. the ADS-B GTW stamped anew the ASTERIX messages without making any change on the reported position. The proposed correction consists in forwarding the original time stamp given by the ADS-B ground station to the SDF and letting the SDF extrapolate the position at the time of SDF output as done for any other sensor track.

2.6.4 Conclusion and more (interesting) results Beyond the aforementioned test, which reflects human error more than system performance, one should not discard the results so easily. First, it is to be noted that the built-in test equipment did not make it evident, neither to the end-user nor to technical staff, that the ground stations were not synchronised. This is a typical case of slightly corrupted surveillance data, which unnoticed, results in a severe "misuse of surveillance data" hazard (cf. [13]). Secondly, the aforementioned test results provide us with the opportunity to validate another two requirements, which were not initially in our test plan: • When not synchronised on external time signal, the automatic dependant surveillance broadcast

(ADS-B) ground subsystem time shall have a maximum drift of 4.4 sec/day. • When not synchronised on the time reference system, the system time shall have a maximum drift

of 20 ms per day.

2.7 ADS-B latency test On Wednesday 5th April 2006, Holger Neufeldt, Volker Seidelmann and Bernd Doleschal tested the internal latency of ADS-B processing under real load conditions at Toulouse-Blagnac. They found that the total latency between the reception of a radio frequency (RF) signal and the output of the corresponding ADS-B report on the network (in ADS-B pipeline mode) was between 6 and 35 ms.

2.8 Test of the probability of detection of a stationary vehicle The objective of this verification test was to measure the probability of detection of a stationary vehicle at the output of the following components: • MLAT;

EMMA



• ADS-B GTW; • SDF. This test was performed in January 2006 on the fixed test transmitter located at the direction finder shelter.

2.8.1 System configuration The initial equipment conditions were as follows: • full ADS-B system installed, up and running; • ADS-B system synchronised with the airport time reference; • Mosquito unit, installed within the direction finder shelter, up and running; • ELVIRA recording system connected to the system LAN. In a stationary condition the Mosquito emits the following 1090ES messages with associated emission rates: • surface position: once every 5 seconds; • identity and type: once every 10 seconds. The ADS-B ground stations emit ASTERIX CAT 021 target reports as soon as they receive updates from ADS-B targets. The ADS-B GTW (only for stationary targets) adapts the received input rate from ground stations to a one second output rate (ASTERIX CAT 062) towards the SDF. The SDF emits target reports (ASTERIX CAT 062) towards the CWP at one-second output-rate.

2.8.2 Findings The results are provided in Table 2-2.

Toulouse ADS-B MLATSystem MLAT ADS-B GTW SDF

Total measure time sec 5127 5141 5140Total detected events 1 sec 5114 5141 5140Detection Probability % 99,75 100,00 100,00

X-pos average m -456,7 -479,0 -478,5Y-pos average m -478,3 -470,4 -470,4X-pos stdev m 0,85 0,27 0,27Y-pos stdev m 1,25 0,77 0,772D average deviation m 1,30 0,63 0,632D standard deviation m 1,51 0,82 0,822D deviation 50%-tile m 1,31 0,37 0,372D deviation 95%-tile m 2,81 1,63 1,632D deviation 99%-tile m 3,78 2,01 2,002D deviation 99.99%-tile m 5,74 2,01 2,002D max deviation m 5,76 2,01 2,00

Test trasmitterUnit

Table 2-2: Test transmitter measured probability of detection

The detection probability at the output of the sensor data fusion is 100%. It has to be noted that the Toulouse-Blagnac implementation of the ADS-B sub-system (made of 5 ADS-B ground stations) is over-redundant. Please refer to §2.9 for more details. Note: the reported accuracies in Table 2-2 are referred to the average measured position of the test transmitter GPS receiver. They are not referred to any other external reference and therefore they represent more an indication of the position sample dispersions rather than an absolute accuracy measure.

2.9 ADS-B cost-benefit test The objective of this verification test was to assess the number of ADS-B ground stations required at Toulouse-Blagnac to provide adequate vehicle surveillance coverage and probability of detection.

EMMA



2.9.1 System configuration The test was performed by measuring the probability of detection of moving vehicles and aircraft along taxiways and runways centrelines (and runways borders) at the output of the following components: • each ADS-B ground station; • ADS-B GTW; • SDF. This test was performed in January 2006. The initial equipment conditions were as follows: • full ADS-B system installed, up and running; • ADS-B system synchronised with the airport time reference; • Mosquito units installed, up and running; • some real life traffic; • ELVIRA recording system connected to the system LAN. Considering the Mosquito emission rate, for a recording period of N seconds of each Mosquito equipped target and 100% of detection probability the following maximum detected events can be obtained: • each ground station: 2*N+N/5; • ADS-B GTW: N; • SDF: N.

2.9.2 Findings The overall probability of detection at SDF output was 100%. The 5 ground stations installed at Toulouse-Blagnac define an over redundant system, but one single ground station is not enough to have a complete and reliable coverage. For a single ground station, the detection probability (with exception of the one located at GBAS shelter) ranges from 70 to 90%, depending on the airport area.

Table 2-3: Detection probability for triples and couples of ADS-B ground stations

As shown in Table 2-3, a combination of 3 ground stations provides an adequate 99.88% probability of detection of moving vehicles and aircraft along taxiways and runways centrelines (and runways borders). Combined with other sensors (e.g. SMR), this performance is sufficient to reach a 100% probability of detection at the output of the sensor data fusion.

2.10 Consolidation of verification results In the tables below, the hypothesis numbers and hypothesis titles as documented in the Toulouse-Blagnac verification and validation test plan (D6.1.3) are recalled, followed by the verification result. The remaining hypotheses will be verified in the scope of EMMA-2, when the proposed system improvements will have been implemented. Please refer to the verification and validation results analysis document (D6.7.1) for more details on the proposed improvements.

2.10.1 Surveillance

EMMA



Hypothesis N°

Hypothesis title Verification result

Ver.sur.2

Coverage Verified.

Ver.sur.3 Target position accuracy

Partially verified, cf. D6.7.1 for an in-depth analysis. At the current date (i.e. March 2007), the vehicle position accuracy issue has been solved and the test can be considered "verified".

Ver.sur.5 Update rate Verified. Ver.sur.6 Latency Requirements not met, cf. D6.7.1 for an in-depth analysis. At the current date (i.e. March

2007), the vehicle position latency issue has been solved and the test can be considered "verified".

Ver.sur.7 Efficiency Partially verified, cf. D6.7.1 for an in-depth analysis. At the current date (i.e. March 2007), the efficiency issue has been solved and the test can be considered "verified".

2.10.2 Control Hypothesis

N° Hypothesis

title Verification result

Ver.con.1 Capability Verified as part of D4.2.1 Site acceptance procedure for the Surface Conflict Alert (SCA) system at Toulouse airport.

Ver.con.2 Boundary Verified as part of D4.2.1. Ver.con.5 Latency Verified as part of D4.2.1 to range between 0 ms and 2 ms (depending on the number of

conflicts).

2.10.3 Human machine interface Hypothesis N° Hypothesis title Verification result

Ver.hmi.1 Surveillance Verified. Ver.hmi.2 Control Verified. Ver.hmi.3 Update capability Verified.

EMMA



3 Validation session – shadow mode trials

3.1 Introduction The validation session subject of the current chapter aims at assessing both the operational feasibility of A-SMGCS in Toulouse-Blagnac airport and the operational improvements brought by this system. Although the operational feasibility and operational improvements are defined as two different stages of the validation activities, they have been evaluated during the same validation session. The validation session consisted in a series of shadow-mode trials performed with controllers of Toulouse-Blagnac airport. They were asked to fill in a questionnaire while assessing the A-SMGCS HMI installed in the tower. The questionnaire was derived from the one developed by the DLR for the Prague test site and adapted to Toulouse-Blagnac local specificities. It addressed two aspects of the operational feasibility (i.e. the acceptance of technical performance and the procedures) and the operational improvements. For each topic, a series of statements were presented to the reader who was asked to point out his / her level of agreement over 6 possibilities ranging from ‘strongly disagree’ to ‘strongly agree’. All the controllers of Toulouse-Blagnac airport (i.e. 65) have followed a 45 minutes training course on the SMGCS HMI installed in the tower for operational trials. Since the installation of the SMGCS in September 2005, the controllers had many opportunities to use and evaluate the HMI during operations. The A-SMGCS HMI has the same ‘look-and-feel’ as the SMGCS HMI. However, in order to inform the controllers working at Toulouse-Blagnac about the differences between both systems, 44 of them have followed a one-hour briefing about the A-SMGCS between the end of January and the beginning of February 2006. Then a questionnaire was distributed to all the controllers. They were asked to fill it in by themselves whenever they wanted and, if possible, in front of the A-SMGCS HMI in the tower. Some of them were also asked to participate to interviews during shadow-mode trials on a voluntary basis. The interviews took place in front of the A-SMGCS HMI in the tower and lasted 1h30 on average. The questionnaire was filled in with each controller individually. In all cases, emphasis was put on the importance of collecting controllers’ feedback and remarks in order to better understand their answers. The shadow-mode trials took place between the beginning of February and mid March 2006.

3.2 Data description and data collection methods

3.2.1 Raw data Thirty-five questionnaires in paper format and in French have been collected, representing more than 50 percent of the 65 controllers in Toulouse-Blagnac. Amongst them, 17 were filled in during interviews with controllers in 4 different sessions and 18 controllers filled them in on their own. The initial version of the questionnaire, used for Prague test site validation activities, had to be adapted to the trial constraints and local particularities. Unlike the Prague airport A-SMGCS, Toulouse A-SMGCS was not used in operations. The questionnaire was adapted to take this into account. A human factor expert reviewed the questionnaire in order to delete some minor inconsistencies and redundancies. However, in order to allow comparison between the different test sites, it was decided to minimize the differences between the questionnaires. Three different versions of the questionnaire have been used during shadow-mode trials: • The first version (V003) was too long (116 statements). After the first questionnaires had been

filled in, it appeared that some questions were not relevant for Toulouse-Blagnac or were redundant, and therefore needed to be deleted or modified. That version was not distributed to all the controllers and only 6 V003 questionnaires were filled in, both during interviews and directly by controllers.

EMMA



• The number of statements was reduced to 92 in version V005. A more direct grammatical style was also adopted in order to ease the understanding of the statements and make the questionnaire faster to fill in. That version was only used for 6 interviews.

• Version V006 only differs from V005 by one statement. The controllers did not understand the statement ‘the EMMA display enables to detect a loss of accuracy of the surveillance’ (number 90 of version V005). Therefore, it was decided to delete it in version V006. This last version contains 91 statements. It was distributed to all the controllers that had not filled a questionnaire in yet. Twenty-three V006 questionnaires were filled in both during interviews and directly by controllers.

Version V005 is more complete than V006 but as the two versions are very similar, it has been decided to keep version V005 as the reference questionnaire for the data analysis. In the following, questionnaire V005 will be referred to as ‘the questionnaire’. Each paper version has been converted to an individual sheet in an Excel file, allowing a correspondence between the different versions of the questionnaire and facilitating the data analysis. All the comments were translated from French to English before they were reported in the Data Analysis section.

3.2.2 Additional data Before starting the analysis of the data collected through the questionnaires and in order to better understand some controllers’ answers and comments, it is worth highlighting that because of delays in the technical implementation of the A-SMGCS, the controllers had a limited time period to discover and master the A-SMGCS. As a consequence, they often replied to the statements with a conservative eye. They sometimes even preferred not to give any opinion, not knowing the system enough to evaluate the statement correctly. Another issue was the lack of reliability of some of the functions of the system. In particular, the identification of the aircraft was not continuous because of technical (e.g.: no identification coverage on the apron, important latency time for the identification of departing aircraft, label swap, loss of identification) and operational (e.g.: depending on each airline’s procedures, some pilots do not switch the transponder on before they have reached the runway threshold) problems. An important number of false plots or track duplication were regularly observed during all the shadow-mode trials, especially when a wide body aircraft taxied in the vicinity of the SMR or during heavy rain. Some of the false reports triggered false alarms, for example when they appeared on the runway while an aircraft was approaching. These technical limitations have been addressed in §3.4.3.1.4. In this context, the controllers had a very limited trust in the system, which was reinforced by their lack of experience with the A-SMGCS. The A-SMGCS HMI has been installed on the V2 position in the tower (see Figure 1), on the left-hand side of the pre-flight working position. The location was convenient to perform shadow-mode trials and interviews without disturbing the normal operations but it is a bit too far from the working positions used in normal operations to allow the controllers to look at the HMI from time to time during periods of reduced traffic.

EMMA



A-SMGCS HMI

A-SMGCS HMI

Figure 16: Layout of the Working Positions in the Toulouse-Blagnac Tower

3.3 Data analysis

3.3.1 Data analysis method In order to perform the data analysis on the answers given to the 92 statements of the questionnaire, a figure has been associated to each level of agreement as follows: • 1 – strongly disagree; • 2 – disagree; • 3 – slightly disagree; • 4 – slightly agree; • 5 – agree; • 6 – strongly agree. Then, a statistical analysis using the indicators described hereafter has been performed on the 33 collected questionnaires after they have been converted in MS-Excel sheets. The results have been gathered in result sheets of the same file. For each statement, the total number of answers has been counted. A low number of answers may indicate that the controllers did not clearly understand the statement but they may also consider that the system is not mature enough or they are not experienced enough with A-SMGCS to evaluate the statement correctly. Comments made by the controllers often give helpful information on the reasons for a low number of answers to a statement. For each statement, the average of the answers has been calculated. The result, a number between 1 and 6, reflects the global opinion of the controllers on the statement. For each statement, the standard deviation of the answers has been calculated. A low number for the standard deviation indicates that the controllers’ opinions are close the ones from the others and consequently close to the average while a high number reveals the disparity of the controllers’ opinions. For each statement, the P parameter of the one sample t-test has been calculated. The one sample t-test is a statistical hypothesis test, which compares the mean score of a sample to a known value,

EMMA



usually the population mean. It allows the determination of the significance of the mean score of the tested sample. In the particular case of the questionnaire, the aim was to know if the hypothesis ‘the average of the answers to each statement is significant’ was true. This hypothesis was tested with the one sample t-test against the ‘null hypothesis’ (i.e.: ‘the average of the total population of controllers is 3.5’). If the P parameter of the one sample t-test for the tested hypothesis was smaller than 0.05 then the hypothesis was accepted, which means that the average of the answers of the tested statement was significant. If the P parameter was greater than 0.05, the hypothesis was rejected, which means that the average of the answers to the tested statement was not significant.

3.3.2 Analysis tables for the shadow-mode validation trials Annex A presents the results of the analysis in tables ordered according to the three topics assessed in the questionnaire: acceptance of technical performances for operational feasibility, procedures for operational feasibility and operational improvements.

3.4 Results This section presents the results of the validation session in terms of operational feasibility and operational improvements. For each stage the results of the metrics primarily defined in the Verification and Validation Test Plan Document [5] are detailed. The validation of the hypotheses and high-level objectives specified in that document are derived from the measured metrics.

3.4.1 Operational feasibility: acceptance of technical performances

3.4.1.1 Metrics The results of the individual metrics related to the operational feasibility and acceptance of technical performance listed in this section were measured during the shadow-mode trials.

3.4.1.1.1 General

3.4.1.1.1.1 Level of automation acceptance The controllers are positive in principle about the level of automation of the A-SMGCS. Through the detection and identification functions, the system provides assistance for surveillance activities on the airport, even in good visibility conditions. Automation of the tracking of the aircraft is well accepted when there is no need for high position accuracy (e.g.: surveillance of an aircraft at the runway threshold or at the beginning of its takeoff). There is even a demand for an adaptation of the procedures to allow more automation in non-critical areas (e.g.: procedures requesting pilots to switch the aircraft transponder before pushing back, reduction of the number of position reports by pilots on taxiways). See section 3.4.2 Operational feasibility: procedures for further details. The controllers consider that some functions like identification and alerts are not reliable enough yet and they do not trust the system enough to exercise control with it.

3.4.1.1.1.2 Modularity acceptance The modularity of the A-SMGCS is well accepted by the controllers. They consider that, after a short adaptation period, the HMI can easily be customised to the user’s preferences.

3.4.1.1.1.3 Assignment of responsibilities acceptance The A-SMGCS has only been implemented for shadow-mode trials in Toulouse-Blagnac. However, the controllers have not foreseen any major modifications of the current responsibilities of the different partners already present in Toulouse-Blagnac airport. For instance, the controllers noted that the pilot should still be responsible for ensuring a safe push back and taxi, even when an A-SMGCS is implemented.

EMMA



The main modifications foreseen by the controllers would be the share of responsibility between the controller and the A-SMGCS. As indicated in the questionnaires, the controllers seem to be ready to trust the system to determine the position and identification of mobiles in low visibility conditions.

3.4.1.1.1.4 Visibility transition procedures acceptance The A-SMGCS has been implemented for shadow-mode trials in Toulouse-Blagnac. No A-SMGCS procedures have been defined. Therefore, the acceptance of visibility transition procedures could not be measured. Nevertheless, the controllers consider that the transition from normal operations to low visibility operations would be easier with the A-SMGCS and that the system helps them to reduce their workload during low visibility operations. Even if several controllers suggested that the current visibility limits could be reduced, the controllers did not agree on modifying the visibility limits when using the A-SMGCS. It must be stressed that the current visibility limit for the transition from normal operations to low visibility operations (i.e. 1000 m) is very high in Toulouse-Blagnac. A reduction of this limit to the standard 550 m limit, for instance, would bring a significant gain in terms of capacity and efficiency. See section 3.4.2.10 Visibility transition for further details.

3.4.1.1.1.5 Fallback procedures acceptance The A-SMGCS has been implemented for shadow-mode trials in Toulouse-Blagnac. No A-SMGCS procedures have been defined. Therefore the acceptance of fallback procedures could not be measured. Nevertheless, the controllers consider that if A-SMGCS procedures were defined, they would not have to return to SMGCS procedures for all the traffic if only one mobile could not comply with the technical requirements for A-SMGCS operations. See section 3.4.2.12 Contingency procedures for further details.

3.4.1.1.1.6 Movement area acceptance The definition of the manoeuvring and movement areas were agreed prior to the experimentation sessions, as recommended by the ICAO manual on A-SMGCS. It was ensured that no inactive or closed areas were to be excluded from the manoeuvring and movement areas. It was also ensured that a common understanding of the different areas was reached prior to the assessment of other technical performance indicators, such as detection and identification coverage.

3.4.1.1.1.7 Movement rate acceptance The A-SMGCS has been implemented for shadow-mode trials in Toulouse-Blagnac. No A-SMGCS procedures have been defined and the movement rate has not been modified. Therefore, the acceptance of increased movement rates could not be measured operationally. Nevertheless, the controllers considered that the use of A-SMGCS could increase the airport capacity.

3.4.1.1.2 Surveillance

3.4.1.1.2.1 Controller acceptance of detection coverage Despite their lack of experience with the A-SMGCS, the controllers consider that the detection coverage is rather acceptable on the manoeuvring area. It must be noted that on the apron, aircraft are only detected by the primary radar, which may create confusion between the different types of mobiles.

3.4.1.1.2.2 Controller acceptance of identification coverage No consensus was reached on the identification coverage. Several controllers considered that because of their lack of experience with the A-SMGCS and the poor continuity of the identification function they could not take a stand on the identification coverage.

EMMA



Nevertheless, some of the controllers regretted that identification was not possible on the apron (in order provide traffic information).

3.4.1.1.2.3 Controller acceptance of position report accuracy Despite their lack of experience, the controllers consider that the accuracy of the position report is rather acceptable. However, some of the controllers noted that it was not the responsibility of the controller to take into account the accuracy of the system in order to perform control with the A-SMGCS (e.g. to ensure separation between aircraft on taxiways, to check that an aircraft has crossed a stop bar). Therefore, accuracy margins, taking into account the static accuracy of the system and aircraft latency and speed should be defined and directly integrated in the procedures. The controllers would accept, for instance, a procedure stating that an aircraft can be considered as having crossed a stop bar when its report is displayed 50 m in front of the stop bar on the HMI.

3.4.1.1.2.4 Controller acceptance of heading and velocity accuracy When the SMGCS was implemented for operational trials, the controllers requested that the heading and velocity were not displayed on the HMI. For consistency reasons, it has been decided to reproduce this characteristic on the A-SMGCS HMI. Therefore the acceptance of heading and velocity accuracy could not be measured.

3.4.1.1.2.5 Controller acceptance of report delay The controllers could not easily evaluate the system latency because of their lack of experience. However, they consider that the global consistency between displayed and actual traffic is good. Some controllers experienced an important latency with the mode S equipped vehicles. Therefore, these controllers did not accept the report delay for the vehicles concerned.

3.4.1.1.2.6 Controller acceptance of probability of detection The controllers could not easily evaluate the probability of detection because of their lack of experience. However, they consider that there are too many missing reports to control the traffic in a safe and efficient way.

3.4.1.1.2.7 Controller acceptance of probability of identification No consensus was reached on the probability of identification. Several controllers considered that because of their lack of experience with the A-SMGCS and the poor quality of the identification function (poor continuity of service and correlation with flight plan data), they could not take a stand on the probability of identification.

3.4.1.1.2.8 Controller acceptance of probability of false detection The controllers consider that the probability of false detection is not acceptable. They had difficulties to evaluate this metric because of their lack of experience.

3.4.1.1.2.9 Controller acceptance of probability of false identification The controllers consider that the probability of false identification is not acceptable. They had difficulties to evaluate this metric because of their lack of experience.

3.4.1.1.3 Routing The routing function has not been implemented on the A-SMGCS HMI installed in the tower. Therefore, the related routing metrics could not be measured. It should be noted that the routing system installed could only be used for technical testing with the objective of further developing it during the EMMA 2 project. The routing function and corresponding metrics will be assessed in EMMA 2.

EMMA



3.4.1.1.4 Control

3.4.1.1.4.1 Controller acceptance of the short-term conflict alert The controllers could not easily evaluate the short-term conflict alert because of their lack of experience. Nevertheless, they are interested in principle in an efficient A-SMGCS alert service. They consider that the A-SMGCS can already help them to detect and prevent incursions on the runway and into restricted areas and stop bar crossings.

3.4.1.1.4.2 Controller acceptance of the alert response time No consensus was reached on the runway alert response time. Many controllers considered that because of their lack of experience with the A-SMGCS they could not take a stand on the runway incursion alert response time. Several controllers noted that the runway incursion alert is triggered too late to let them take the appropriate action. A few controllers consider that information alerts are often popping up too late to take an appropriate action before an alarm is triggered.

3.4.1.1.4.3 Controller acceptance of the number of false and nuisance alerts No consensus was reached on the number of false and nuisance alerts. Many controllers considered that because of their lack of experience with the A-SMGCS they could not take a stand on the number of false and nuisance alerts. However, it should be noted that, during the shadow-mode trials, many false alerts due to false detections were observed.

3.4.1.1.5 Human Machine Interface

3.4.1.1.5.1 HMI efficiency The controllers have a globally positive opinion of the HMI efficiency. In particular: • They consider that the zoom function is easy to use, even if a short adaptation period was

necessary for some of them. • They consider that it is easy to display several windows. • They have a rather positive opinion of the size of the pop up windows and the place where they

appear. However, several controllers noted that the alert texts were too small. • They consider that the windows are rather conveniently arranged on the A-SMGCS HMI. • They consider that the HMI is rather easy to customise to everyone’s own preferences. • They consider that the level of interaction with the HMI is satisfactory. No consensus was reached on whether pop-up windows sometimes obscure aircraft that should be visible and whether the manual modification of the content of mobile reports on the HMI can be performed quickly and efficiently. Many controllers considered that because of their lack of experience with the A-SMGCS they could not take a stand on these issues. Nevertheless, some of the controllers noted that the manual modification of the label content was not easy to use because of the not intuitive label selection mode (i.e. click on the mobile instead of the label) and the difficulty to ‘catch’ a quickly moving mobile. Note: most of the time, the controllers preferred to disable the automatic label anti-overlapping function. Indeed, when there is a risk of overlap between two labels, the function systematically makes the leader line of the concerned labels longer. As a result the HMI, quickly full of very long leader lines (some of the labels even falling outside the display), looses its readability and usability.

EMMA



3.4.1.1.5.2 HMI information usability The controllers have a globally positive opinion of the HMI information usability. In particular: • They are rather satisfied with the amount of information about airborne traffic in the vicinity of

the airport. However, several controllers noted that this kind of information being available in other tools, it is not useful in a ground tool like A-SMGCS.

• They generally consider that no important function is missing but several of them found unfortunate that the reliability of the identification function and correlation with flight plan data was so poor. Others asked for an automatic tracking of a selected aircraft in a dedicated window and the possibility to delete the third line in a departure label while keeping the aircraft type.

• They consider that the colours used in the HMI are appropriate. • They consider that the contrast between the windows and their background is sufficient. • They consider that the height and width of the characters on the HMI are sufficient. • They consider that the texts in the HMI are rather easy to read, except for the alert text that is too

small. • They consider that the labels, signs and symbols in the HMI are easy to interpret. • They consider that information is conveniently arranged in the HMI. • They consider that symbols can easily be read under different angles of view in the HMI. • They consider that the screen size is appropriate for a daily work but several of them are

concerned with the integration of the screen in almost full working positions. Only three controllers evaluated the statement ‘the EMMA display enables to detect a loss of accuracy of the surveillance’. The statement was deleted in version V006 of the questionnaire, the controllers not being able to evaluate the loss of accuracy of the surveillance during the shadow-mode trials.

3.4.1.1.5.3 HMI information usefulness The controllers have a globally positive opinion of the HMI information usefulness. In particular: • They consider that it is useful to represent the closed taxiways, runways and apron areas on the

HMI. • They consider that the actions that can be performed on the windows are relevant. • They consider that the manual anti-overlapping of labels is rather useful, even if they would prefer

a more efficient automatic anti-overlapping function (cf. note in the HMI efficiency metric section).

• They consider that the use of colour characterisation for the different types of mobiles is useful. • They consider that the raw video is useful. • They consider that the function allowing the manual modification of label content is useful, even

if many of them expected an improvement of its usability (Cf. HMI information usability metric).

3.4.1.2 Hypotheses In this section, the operational feasibility hypotheses described in the Verification and Validation Test Plan Document [5] are validated or invalidated according to the results of the metrics measured during the shadow-mode trials. The characters between parentheses in each hypothesis title indicate the reference of the indicator used in [5].

3.4.1.2.1 General

3.4.1.2.1.1 Level of automation (Opf.pre.1) Hypothesis: EMMA A-SMGCS should clearly identify the level of automation of each of these functions at level I/level II. Validation: The hypothesis has been validated.

EMMA



3.4.1.2.1.2 Modularity (Opf.pre.2) Hypothesis: An A-SMGCS shall be composed of different modules required for particular user needs or technological choices. The design principle of an A-SMGCS should permit modular enhancements. A-SMGCS for each aerodrome will comprise a different mix of modular components dependent on operational factors as categorized in Appendix A. For example, some modules of an A-SMGCS will be required when one or more of the following conditions exist: a) heavy traffic density; b) visibility condition 2, 3 or 4; and c) complex aerodrome layout. The required modules should be defined for each set of conditions (visibility, traffic density). Validation: The hypothesis has been validated.

3.4.1.2.1.3 Assignment of responsibilities (Opf.pre.3) Hypothesis: Although the responsibilities and functions may vary, they should be clearly defined for all users of A-SMGCS. An A-SMGCS should be designed so that the responsibilities and functions may be assigned to the following:

a) The automated system; b) Controllers; c) Pilots; d) Vehicle drivers; e) Marshallers; f) Emergency services; g) Airport authorities; h) Regulatory authorities; and i) Security services.

With the automation of safety-critical functions, responsibilities should be carefully assigned and shared between equipment and human operators, at level I&II and for visibility conditions 1, 2 and 3. Procedures should reflect this share of responsibilities. Validation: The hypothesis has been validated.

3.4.1.2.1.4 Visibility transition procedure (Opf.pre.4) Hypothesis: Transition limits between visibility conditions 1, 2, 3 and 4 should be carefully defined in terms of RVR and ceiling. Transition procedures should be defined in order to ensure smooth transitions between different visibility conditions. Taking into account the evolution of visibility, the use of transition buffers may help to prepare the traffic for operations in the next visibility conditions. Validation: The hypothesis could not be tested during the shadow-mode trials. A-SMGCS visibility transition procedures had not been defined when the validation sessions took place. This hypothesis should be tested during operational trials, when the system is mature enough to define experimental visibility transition procedures adapted to A-SMGCS.

3.4.1.2.1.5 Fallback procedure (Opf.pre.5) Hypothesis: Fallback procedures should be defined to recover from the failure of one or more A-SMGCS equipment. In EMMA and EMMA2, in Toulouse airport, these fallback procedures will be defined to

EMMA



prevent the hazards due to failure of equipment, identified in D1.3.9 ‘Functional Hazard Analysis and Very Preliminary System Safety Assessment’. Validation: The hypothesis could not be tested during the shadow-mode trials. A-SMGCS fallback procedures had not been defined when the validation sessions took place. This hypothesis should be tested during operational trials, when the system is mature enough to define experimental fallback procedures adapted to A-SMGCS.

3.4.1.2.1.6 Definition of Movement rate (Opf.pre.6) Hypothesis: An A-SMGCS should be capable of operating at a specified movement rate in visibility conditions down to the aerodrome visibility operational level (AVOL). When visibility conditions are reduced to below AVOL an A-SMGCS should provide for a reduction of surface movements of aircraft and vehicles to a level acceptable for the new situation. The acceptable maximum movement rate should be defined in level I & II and for visibility conditions 1, 2 and 3. Validation: The hypothesis could not be tested during the shadow-mode trials. Movement rates had not been modified when the validation sessions took place. This hypothesis should be tested during operational trials, when the system is mature enough to define experimental movement rates with A-SMGCS.

3.4.1.2.1.7 Definition of movement area (Opf.pre.7) Hypothesis: In order to resolve the problem of vehicle control/segregation on a specific stand, the concept is introduced whereby the role of that stand may change from active to passive and vice versa. Hence, the use of movement area in this manual excludes passive stands, empty stands and those areas of the apron(s) which are exclusively designated to vehicle movements. The movement area should be clearly defined, in terms of included/excluded taxiways, runways, vehicle routes, stands and other specific areas. The protected areas should also be fully defined. Validation: The hypothesis has been validated. The movement area has been fully defined prior to the experimentations.

3.4.1.2.2 Surveillance

3.4.1.2.2.1 Target reference point (Opf.sur.1) Hypothesis: The target reference point of aircraft and vehicle should be carefully defined. For operations in Toulouse-Blagnac airport, the target reference point is defined as the mid-point of longitudinal axis of the aircraft or vehicle. Validation: The hypothesis has been validated. Note: the target reference point verification test had not been performed when the validation sessions took place.

3.4.1.2.2.2 Acceptable detection coverage (Opf.sur.2) Hypothesis:

EMMA



Although all the movement area should be covered, the verification phase may reveal some shadow detection areas. The impact of these shadow areas on ground traffic control should be carefully determined, depending on the criticality of the area (final approach, initial departure, runway, runway entrance/exit, taxiway, intersection, apron, protected area, etc.), on the visibility conditions (the area is visible/not visible from the tower) and on the traffic density. Procedures related to these areas should be adapted (for instance, by limiting/forbidding all traffic through these areas in visibility condition 2). Validation: The hypothesis has been validated. The detection coverage is considered as acceptable. Therefore, no modification of A-SMGCS procedures is required. The apron is only covered by the primary radar and procedures have not been adapted.

3.4.1.2.2.3 Acceptable identification coverage (Opf.sur.3) Hypothesis: Although all the movement area should be covered, the verification phase may reveal some shadow identifications areas. The impact of these shadow areas on ground traffic control should be carefully determined, depending on the criticality of the area (final approach, initial departure, runway, runway entrance/exit, taxiway, intersection, apron, protected area, etc.), on the visibility conditions (the area is visible/not visible from the tower) and on the traffic density. Procedures related to these areas should be adapted (for instance, by limiting/forbidding all traffic through these areas in visibility condition 2). Validation: The hypothesis has not been validated. The controllers did not reach any consensus on the acceptance of the identification coverage. The apron is only covered by the primary radar and procedures have not been adapted.

3.4.1.2.2.4 Acceptable target position accuracy (Opf.sur.4) Hypothesis: Although the static and dynamic accuracy should be inferior to 7.5m, the verification phase may reveal some problems: • The report is located near the aircraft MLAT antennas, different from the reference point,

(decrease of static accuracy), • The system extrapolates the trajectory when it loses a track (decrease of dynamic accuracy), • The resolution is too low, leading to a systematic bias for the positioning of a mobile. The impact of this loss of static and dynamic accuracy should be carefully determined, depending on the criticality of the concerned area (final approach, initial departure, runway, runway entrance/exit, taxiway, intersection, apron, protected area, etc.), on the visibility conditions (the mobile is visible/not visible from the tower) and on the traffic density. The impact of this loss of accuracy on the performances of the control function should also be determined. Validation: The hypothesis has been partly validated. The controllers consider that the target position accuracy is acceptable. Nevertheless, many controllers require an adaptation of the procedures to take into account accuracy margins. Due to the lack of technical data, this adaptation could not be performed prior to the validation activities.

3.4.1.2.2.5 Acceptable reported velocity and heading accuracy (Opf.sur.5) Hypothesis: The reported velocity and heading accuracy will have been measured during the verification phase. Too low velocity and heading accuracies may have some unwanted technical and operational effects,

EMMA



in level II, but especially in higher levels of A-SMGCS, where more automated functions will rely on the determination of correct velocity and headings: • Nuisance short term conflict alerts, • Nuisance route conformance monitoring alerts, • Wrong route definition by the routing function, due to wrong heading of aircraft. The impact of these lacks of accuracy should be determined, in different visibility and traffic density conditions. A distinction should be made between direct effects (display of wrong velocities and headings on the HMI) and indirect effects (nuisance alerts). Validation: The hypothesis could not been tested. It has been chosen not to display the heading and velocity on the A-SMGCS HMI. This hypothesis can only be tested in Toulouse-Blagnac if the need for heading and velocity information on the HMI is identified.

3.4.1.2.2.6 Acceptable discrimination accuracy (Opf.sur.6) Hypothesis: The discrimination between two mobiles will have been measured during the verification phase. A too low discrimination accuracy may reduce the controller’s situation awareness, by leading to a confusion between mobiles or by considering only one mobile, when several mobiles are near the same location. This is especially true when two vehicles or when an aircraft and a vehicle are involved. The impact of a too high discrimination distance should be carefully determined, depending on the visibility conditions and on the traffic density (aircraft + vehicles). Validation: The hypothesis could not be tested. The position discrimination verification test had not been performed when the validation sessions took place. This hypothesis should be tested during shadow mode trials, when the discrimination accuracy has been measured (e.g. in EMMA2).

3.4.1.2.2.7 Acceptable accuracy margins (Opf.sur.7) Hypothesis: In order to increase movement efficiency, accuracy margins on the HMI may be defined to allow a controller to check (if the pilot still reports) or to state (if he does not) that a mobile has cleared the runway, crossed an intersection, pushed back, etc. These accuracy margins should take into account the reported size of the mobile on the HMI, and the dynamic accuracy. The measurement of these values during the verification phase will help to define acceptable accuracy margins with the controller. Also, a loss of accuracy, if the controller is not informed, may decrease safety, by encouraging him to use procedures based on accuracy margins when these margins are not applicable anymore. Fallback procedures or technical requirements should then be defined to prevent this case. Validation: The hypothesis could not be tested. Accuracy margins on the HMI had not been defined when the validation sessions took place.

3.4.1.2.2.8 Acceptable report delay (Opf.sur.8) Hypothesis: The update rate, its variations, and the latency will have been measured during the verification phase. This will lead to the determination of the maximum report delay, i.e. the maximum delay between a given situation and its display on the controller’s HMI. This delay has a negative effect on the controller’s situation awareness, especially when no outside view is available from the tower, and may lead to late reactions in case of safety critical situations.

EMMA



The impact of this report delay should be carefully examined, depending on the visibility conditions and on the traffic density. Validation: The hypothesis has been validated. The controllers consider that the consistency between displayed and actual traffic is good.

3.4.1.2.2.9 Acceptable probability of detection (Opf.sur.9) Hypothesis: The definition of acceptable detection coverage will address the issue related to fixed and permanent shadow areas. However, the verification phase may reveal some other dynamic, not fixed, losses of detection. The impact of these losses on ground traffic control should be carefully determined, depending on the criticality of the area (final approach, initial departure, runway, runway entrance/exit, taxiway, intersection, apron, protected area, etc.), on the visibility conditions and on the traffic density. Quantitative and qualitative criteria, determined with the help of controllers, such as, the acceptable number of detection gaps in a track, the size of these gaps, the acceptable number of tracks with a gap, etc. will help compute acceptable probabilities of detection for each visibility conditions and possibly different for the different areas of the airport. Validation: The hypothesis has not been validated. The controllers consider that there are too many missing reports to control the traffic in a safe and efficient way.

3.4.1.2.2.10 Acceptable probability of identification (Opf.sur.10) Hypothesis: The definition of acceptable identification coverage will address the issue related to fixed and permanent shadow areas. However, the verification phase may reveal some other dynamic, not fixed, losses of identification. The impact of these losses on ground traffic control should be carefully determined, depending on the criticality of the area (final approach, initial departure, runway, runway entrance/exit, taxiway, intersection, apron, protected area, etc.), on the visibility conditions and on the traffic density. Quantitative and qualitative criteria, determined with the help of controllers, such as, the acceptable number of identification gaps in a track, the size of these gaps, the acceptable number of tracks with a gap, etc. will help compute acceptable probabilities of identification for each visibility conditions and possibly different for the different areas of the airport. Validation: The hypothesis has not been validated. The controllers did not reach any consensus on the acceptance of the probability of identification.

3.4.1.2.2.11 Acceptable probability of false detection (Opf.sur.11) Hypothesis: The verification phase may reveal some fixed and not fixed false target reports. False target reports are reports of not existing solid targets (rain, grass, reflection, etc.). These target reports have negative on the controller’s situation awareness, as they create some additional workload to determinate their non-significance. The impact of these false reports on ground traffic control should be carefully determined, depending on the criticality of the area (especially on or near the runway, where they can cause false short term conflict alerts), on the visibility conditions (the determination of the significance of the target is more difficult in visibility 2 or more) and on the traffic density (high traffic densities may cause more confusion between false and real targets).

EMMA



Validation: The hypothesis has not been validated. The controllers consider that the probability of false detection is not acceptable.

3.4.1.2.2.12 Acceptable probability of false identification (Opf.sur.12) Hypothesis: The verification phase may reveal some target reports with incorrect identification. These target reports, labelled with incorrect identification, have negative on the controller’s situation awareness, as they create some additional workload to re-create a correct mental association between the position of the mobiles and their identification. The impact of these reports on ground traffic control should be carefully determined, depending on the criticality of the area (especially on or near the runway, where they can cause false short term conflict alerts, due to confusion between mobile types), on the visibility conditions (a correct mental association is more difficult to re-create in visibility 2 or more) and on the traffic density (high traffic densities may cause more confusion between mobiles). Validation: The hypothesis has not been validated. The controllers consider that the probability of false identification is not acceptable.

3.4.1.2.3 Routing The routing function has not been implemented on the A-SMGCS HMI installed in the tower. Therefore the related routing hypotheses could not be tested. These hypotheses should be tested during shadow mode trials, when the routing function has been implemented and routing performance has been measured (e.g. in EMMA2).

3.4.1.2.4 Control

3.4.1.2.4.1 Short-term conflict alert validation (Opf.con.1) Hypothesis: The short-term conflict alerts produced by the SCA should help to prevent safety critical events. The SCA rule implementation and parameters should be consistent with this requirement. Validation: The hypothesis has been partly validated. The hypothesis has been validated for the incursion rules but not for the conflict rules. The controllers consider that the A-SMGCS helps them to detect and prevent incursions on the runway and into restricted areas and stop bar crossings.

3.4.1.2.4.2 Acceptable alert response time (Opf.con.2) Hypothesis: The short-term conflict alert function should allocate sufficient time to the controller to take the necessary actions to solve the safety critical situation. For the validation of this hypothesis, it should be considered that: • The short term conflict alert delay should be sufficiently low, • The SCA rule implementation and parameters should take into account the response time, possibly

by providing several levels of alerts. Validation: The hypothesis has not been validated. The controllers did not reach any consensus on the acceptance of the alert response time.

EMMA



3.4.1.2.4.3 Acceptable probability of false or nuisance alert (Opf.con.3) Hypothesis: The short-term conflict alert system should provide an acceptable number of false and nuisance alerts. False alerts are alerts caused by incorrect surveillance data (false reports, etc.). Nuisance alerts are alerts caused by an inappropriate definition of alert rules and parameters. False and nuisance alerts, not related to a safety critical event, have a negative impact on the situation awareness of the controller and on the trust he/she has in the SCA system. Validation: The hypothesis has not been validated. The controllers did not reach any consensus on the acceptance of the number of false and nuisance alerts.

3.4.1.2.5 Human Machine Interface

3.4.1.2.5.1 Efficiency (Opf.hmi.1) Hypothesis: A-SMGCS shall enable users to interface efficiently. Validation: The hypothesis has been validated.

3.4.1.2.5.2 Usefulness of information (Opf.hmi.2) Hypothesis: The information provided to the controller though the A-SMGCS display is useful for the operations. Validation: The hypothesis has been validated.

3.4.1.2.5.3 Usability of information (Opf.hmi.3) Hypothesis: The presentation of the information provided through the A-SMGCS display is adapted to user needs. Validation: The hypothesis has been validated.

3.4.1.3 High-level objectives In this section, the operational feasibility high-level objectives described in the Verification and Validation Test Plan Document [5] are validated or invalidated according to the results of the hypotheses validation.

3.4.1.3.1 High-level objective 1 Objective: EMMA A-SMGCS meets the operational requirements expressed in the ICAO manual on A-SMGCS [15] for each set of conditions. Validation: The objective has been partly validated. This is mainly explained by the lack of maturity of the surveillance function of EMMA A-SMGCS installed in Toulouse-Blagnac. It does not allow the operational requirements for the other functions to be fulfilled. Please refer to the D6.7.1 Verification and Validation Analysis Report [11] for a detailed review of each operational requirement defined in EMMA.

EMMA



3.4.1.3.2 High-level objective 2 Objective: EMMA A-SMGCS meets the operational requirements expressed by local end-users for each set of conditions. Validation: The objective has been partly validated.

3.4.1.3.3 High-level objective 3 Objective: For each set of conditions, a full set of performance requirements is defined for each automated service. Validation: This objective has not been validated. Some performance requirements have not been fulfilled by the system.

3.4.1.3.4 High-level objective 4 Objective: EMMA A-SMGCS supports adequate procedures to recover from the possible failures of A-SMGCS equipment. Validation: This objective has not been validated.

3.4.2 Operational feasibility: procedures No metrics were primarily defined in the Verification and Validation Test Plan Document [5] for the measurement of the operational feasibility of A-SMGCS procedures. Nevertheless, this section presents the results of the controllers’ evaluation of statements related to the operational feasibility of A-SMGCS procedures.

3.4.2.1 Start-up clearance No consensus was reached on whether the A-SMGCS allows the controllers to set up a more efficient start up sequence. It appeared from the comments made by several controllers that the A-SMGCS has no direct impact on the setting up of the start up sequence. It is rather considered as a support tool that helps the controllers by offering a global synthetic picture of the platform.

3.4.2.2 Push back clearance The controllers consider that the information on the A-SMGCS HMI helps them to decide whether a push back clearance should be delayed. It was noted that it would be even more helpful if the identification function were more reliable and if identification was possible on the apron. The controllers slightly agree with the statement that the A-SMGCS is sufficient to determine that an aircraft is on the stand or has left the stand. They generally agree that it can be used to determine that an aircraft has left the stand but not to check that it is at the stand. Too many permanent false reports are observed on the apron, in particular on the Echo area where the terminal building creates interferences on the contact stands, to be able to know whether a stand is free or occupied. Until the system is more accurate, the controllers consider that it cannot be used as the unique information source to deliver a push back clearance (i.e. they need the pilot’s confirmation and / or the external view).

EMMA



3.4.2.3 Taxi clearance The controllers consider that when they want to give a taxi clearance they can rely on the information provided by the A-SMGCS, even when visual reference is not possible on the manoeuvring area provided that the system has been improved in terms of reliability. The controllers consider that the surveillance of the longitudinal spacing between aircraft on taxiways is easier with the A-SMGCS when visual reference is not possible on the manoeuvring area. Several controllers explained that they are missing the actual size of each aircraft, accuracy of displayed aircraft position and accuracy margins in order to use the A-SMGCS for the surveillance of the longitudinal spacing. Mainly because that information is missing, no consensus was reach on whether the longitudinal spacing between aircraft on taxiways could be reduced with the A-SMGCS when visual reference is not possible on the manoeuvring area. Moreover, the controllers often claimed that ground traffic on the taxiways is not the limiting factor during low visibility operations, the departing aircraft having to queue at the runway holding point anyway. They do not feel the need to modify the current procedures (i.e. one aircraft per taxiway) in order to reduce longitudinal spacing on taxiways during low visibility operations. The controllers consider that the A-SMGCS will not necessarily prevent them from making records of the vehicles on the manoeuvring area. As for the use of paper strips, it depends on each controller’s working habit. Nevertheless, even if some controllers consider that it may not be necessary to maintain a list of the vehicles on the manoeuvring area during low visibility operations if the reliability of the A-SMGCS is improved, many of them insisted on the importance of having a paper backup in case of system failure.

3.4.2.4 Co-ordination The controllers consider that the coordination and the hand over of a taxiing aircraft between the control working positions in the tower would be more efficient with the A-SMGCS. Some controllers noted that the ground and tower working positions should be equipped with A-SMGCS HMIs and that it could be useful for the hand over, for example when the activation sequence is different from the off-block sequence.

3.4.2.5 Taxiing on the runway The controllers consider that the number of pilot position reports could be reduced when using the A-SMGCS. Nevertheless, they do not agree on suppressing the pilot position report to confirm that the aircraft has vacated the runway because it is safety critical. The controllers consider that with the A-SMGCS an aircraft may be permitted to taxi on the runway in use even when visual reference is not possible on the manoeuvring area. However, the interest of making aircraft taxi on the runway in use is limited in Toulouse because the taxiway network is developed enough to avoid it.

3.4.2.6 Line-up clearance The controllers consider that line-up of an aircraft at the runway threshold is easier to control with the A-SMGCS whether visual reference is possible or not on the manoeuvring area. In normal operations it is particularly true at night and when runway 32R is used for departures because the threshold is far from the tower. No consensus was reached on whether multiple line-ups could be performed with the A-SMGCS when visual reference is not possible on the manoeuvring area. Many controllers noted that the A-SMGCS is not mature enough (in terms of reliability and accuracy) to perform multiple line-ups in low visibility conditions. Several of them were reluctant on the idea of modifying the current working habits to encourage multiple line-ups in low visibility conditions.

3.4.2.7 Takeoff clearance The controllers consider that the A-SMGCS is rather helpful for better monitoring an aircraft commencing the takeoff roll.

EMMA



No consensus was reached on whether rolling takeoff clearances could be given more frequently with the A-SMGCS. Many controllers consider that the A-SMGCS is not a key tool when making the decision about a rolling takeoff clearance. The controllers consider that the A-SMGCS is not sufficient to decide when a takeoff clearance should be issued. Several of them explained that other parameters have to be considered (e.g. inbound traffic in final approach, readiness of the pilot).

3.4.2.8 Landing clearance No consensus was reached on whether the A-SMGCS is sufficient to decide if the runway in use has been vacated in order to deliver a landing clearance when visual reference is not possible on the manoeuvring area. Many controllers noted that the A-SMGCS is not mature enough (in terms of reliability and accuracy) to use it as the unique decision tool to give landing clearances.

3.4.2.9 Conditional clearances The controllers consider that when visual reference is possible on the manoeuvring area, the A-SMGCS is rather helpful to deliver conditional clearances on the runway in use and it is helpful to deliver conditional clearances on taxiways in a safe and efficient way. They also consider that it is rather helpful to deliver conditional clearances on taxiways in a safe and efficient way even when visual reference is not possible on the manoeuvring area. No consensus was reached on whether the A-SMGCS may help them to deliver conditional clearances on the runway in use in a safe and efficient way when visual reference is not possible. Several controllers were reluctant to modify there working habits (i.e.: they do not give conditional clearances in low visibility conditions) in such a safety critical case, because they do not trust the system enough.

3.4.2.10 Visibility transition The controllers consider that the transition from normal operations to low visibility operations is easier with the A-SMGCS. They also consider that when low visibility procedures are activated, the A-SMGCS helps them to reduce their workload. No consensus was reached on whether the visibility limits for the transition to low visibility procedures could be redefined for the A-SMGCS. Several controllers noted that the A-SMGCS has no influence on the definition of the visibility limits.

3.4.2.11 Transponder procedures The controllers consider that it would be appropriate that pilots switch the transponder on before they request a push back clearance, in order to automatically associate a label to a departing aircraft.

3.4.2.12 Contingency procedures The controllers consider that if only one mobile fails to comply with the technical requirements for A-SMGCS operations, they do not have to return to SMGCS procedures for all the traffic. They generally agree that if only one mobile cannot be automatically identified, they can easily manage it either by identifying it manually or by asking for position reports to the pilot.

3.4.2.13 Conclusion on procedures The controllers consider that the A-SMGCS helps them whether visual reference is possible or not on the manoeuvring area. They consider that they can rely on the A-SMGCS instead of the outside view when visual reference is not possible on the manoeuvring area provided that the reliability of the current system is improved. They consider that there are not too many inconsistencies between the A-SMGCS HMI and actual traffic. They slightly agree with the fact that the mix of cooperative and non-cooperative mobiles could severely restrict the use of A-SMGCS when visual reference is not possible on the manoeuvring area.

EMMA



Nevertheless, some of them argued that the severity of the restriction depends on the number of non-cooperative mobiles.

3.4.3 Operational improvements

3.4.3.1 Metrics The results of the individual metrics related to operational improvements listed in this section were measured during the shadow-mode trials. The characters between parentheses in each metric title indicate the reference3 of the indicator used in D6.1.3: Verification and validation test plan for Toulouse-Blagnac airport, version 1.02, issued on the 15th February 2006 [5].

3.4.3.1.1 Capacity

3.4.3.1.1.1 Assessment of departure and arrival runway capacity in visibility conditions 2 and 3 No direct measurement of the runway capacity could be performed during the shadow-mode trials. Nevertheless, the results of the evaluation of the operational feasibility of A-SMGCS procedures seem to show that the potential gain in runway capacity would be limited. Indeed, the controllers do not want to suppress the position report of the pilot when the aircraft has vacated the runway. There is consensus neither on the use of multiple line-up, rolling takeoff and conditional clearances on the runway in use in low visibility conditions, nor on the use of A-SMGCS only to decide to give a landing clearance in low visibility conditions, nor on the redefinition of the visibility limits. See section 3.4.2 Operational feasibility: procedures for further details. Therefore, the controllers do not see the introduction of the A-SMGCS as an opportunity to modify deeply the procedures and their working habits during low visibility operations in order to increase the runway capacity. This is explained mainly by the immaturity of the system (in terms of reliability and accuracy) and, as a consequence, by the lack of confidence of the controllers to perform control with it. The controllers also seem to consider that the interest of introducing changes during low visibility operations and increasing the runway capacity is very limited because most of the time the traffic demand can be absorbed without high delays during low visibility operations. Nevertheless, several controllers stressed that a validated A-SMGCS and an adapted lighting on the platform could allow both runways to be used during low visibility operations, which would increase the runway throughput and the capacity of the airport.

3.4.3.1.1.2 Assessment of departure and arrival runway throughput in visibility conditions 2 and 3 (CA01 & CA02)

The controllers have a positive feeling about the increase of departure and arrival throughput during low visibility operations. The A-SMGCS may also help them to manage more traffic when visual reference is not possible on the manoeuvring area. As suggested by some controllers, if both runways are used in low visibility operations with the A-SMGCS, the throughput of each runway may be increased because the time between two movements could be reduced. See section 3.4.2 Operational feasibility: procedures for further details. A quantitative assessment of the runway throughput has been performed during the fast-time simulation. See section 4.4.1.1.1.1 Assessment of departure and arrival runway throughput in visibility conditions 2 and 3 (CA01 & CA02) for further details.

3.4.3.1.1.3 Assessment of mean and maximum number of aircraft simultaneously taxiing in visibility conditions 2 and 3 (CA09 & CA10)

No direct measurement of the taxiway capacity could be performed during the shadow-mode trials.

3 These figures are unfortunately not always consistent with D6.2.2: Indicators and Metrics for A-SMGCS [7].

EMMA



Nevertheless, the results of the evaluation of the operational feasibility of A-SMGCS procedures seem to show that there would be a gain in taxiway capacity if the A-SMGCS were used during low visibility operations. The gain would be mainly due to the use of A-SMGCS by the controllers to deliver taxi clearances and the reduction of the number of position reports by the pilots on the taxiways. Conditional clearances on taxiways could also be authorised in low visibility operations, making the ground traffic more fluid. However, the controllers did not agree to reduce the longitudinal spacing between aircraft on taxiways in low visibility conditions because of their lack of trust in the A-SMGCS and the limited interest of such a modification of the current procedures compared with the traffic demand during low visibility operations. See section 3.4.2 Operational feasibility: procedures for further details. A quantitative assessment of the maximum number of aircraft simultaneously taxiing has been performed during the fast-time simulation. See §4.4.1.1.1.2, Assessment of mean and maximum number of aircraft simultaneously taxiing in visibility conditions 2 and 3 (CA09 & CA10), for further details.

3.4.3.1.1.4 Assessment of mean and maximum number of push back clearances in visibility conditions 2 and 3 (CA07 & CA08)

No direct measurement of the apron capacity could be performed during the shadow-mode trials. Nevertheless, the results of the evaluation of the operational feasibility of A-SMGCS procedures seem to show that there would be a gain in apron capacity if A-SMGCS were used during low visibility operations. However, the controllers do not seem to consider that A-SMGCS would be of any help in the preparation of the start up sequence. They believe that A-SMGCS could help them in the decision to deliver a push back clearance and to check that an aircraft has left a stand. Then, A-SMGCS could have a slightly positive impact on the apron capacity but this impact is restrained by the lack of identification coverage on the apron. See section 3.4.2 Operational feasibility: procedures for further details. A quantitative assessment of the maximum number of aircraft pushing back per 10 minutes time slot has been performed during the fast-time simulation. See section 4.4.1.1.1.3 Assessment of mean and maximum number of push back clearances in visibility conditions 2 and 3 (CA07 & CA08) for further details.

3.4.3.1.2 Efficiency

3.4.3.1.2.1 Minimum, mean and maximum departure and arrival taxi times in visibility conditions 2 and 3 (EF10, EF11 & EF12)

No direct measurement of the taxi time could be performed during the shadow-mode trials. Nevertheless, from the results of the evaluation of the operational feasibility of A-SMGCS procedures, a reduction of the taxi time on average during low visibility operations can be expected for the same reasons as for a potential increase of the taxiway capacity (i.e. use of A-SMGCS for taxi clearance delivery, reduced number of position reports by pilots, possibility of giving conditional clearances). See sections 3.4.2 Operational feasibility: procedures and 3.4.3.1.1.3 Assessment of mean and maximum number of aircraft simultaneously taxiing in visibility conditions 2 and 3 for further details. A quantitative assessment of the mean departure and arrival taxi times has been performed during the fast-time simulation. See section 4.4.1.1.2.8 Minimum, mean and maximum departure and arrival taxi times in visibility conditions 2 and 3 (EF10, EF11 & EF12) for further details.

3.4.3.1.2.2 Assessment of number of communications in visibility conditions 2 and 3 (EF20) No direct measurement of the number of communications could be performed during the shadow-mode trials. Nevertheless, the results of the evaluation of the operational feasibility of A-SMGCS procedures seem to show that the average number of pilot / controller communications could decrease if the A-SMGCS

EMMA



was used during low visibility operations. The controllers consider indeed that the number of position reports by pilots could be reduced in low visibility conditions. See section 3.4.2 Operational feasibility: procedures for further details.

3.4.3.1.3 Human Factors

3.4.3.1.3.1 Controller situational awareness (HF02) The controllers consider that their situational awareness is improved with the A-SMGCS. In particular, they consider that the A-SMGCS HMI allows them to have a better awareness of the respective position and identification of each mobile, a better understanding of the traffic situation and to better anticipate the evolution of traffic. Several controllers noted that an A-SMGCS HMI should be installed at the approach control working position to help the controller to better anticipate the evolution of traffic.

3.4.3.1.3.2 Controller acceptance of the A-SMGCS HMI (HF03) The controllers have a good global acceptance of the A-SMGCS HMI. In particular, they consider that it provides the right information at the right time without being unnecessarily complex. The HMI helps them to reduce their mental workload. See section 3.4.1.1.5 Human Machine Interface for further details.

3.4.3.1.3.3 Usability of the A-SMGCS HMI (HF05) The controllers consider that the global usability of the A-SMGCS HMI is satisfactory. See section 3.4.1.1.5.2 HMI information usability for further details.

3.4.3.1.4 Safety

3.4.3.1.4.1 Detected incorrect display (SA07) The opinion of the controllers concerning the detected incorrect display events is not obvious. On the one hand they consider that the A-SMGCS HMI and the actual traffic are quite consistent and that the A-SMGCS does not endanger safety at the airport. On the other hand, they think that the amount of incorrectly identified mobiles on the A-SMGCS HMI is a problem to control the traffic in a safe and efficient way. Many incorrect display events were observed during the shadow-mode trials. The main categories of incorrect display event are described hereafter: • In most cases, false plots appeared somewhere on the airport, sometimes on the runway while an

aircraft was about to land. This triggered false runway incursion alerts. Interferences caused by a wide-body aircraft (e.g. Beluga, A380) taxiing on taxiway Whisky in the vicinity of the SMR triggered some of the observed false plots.

• Since the system was used in shadow-mode trials when these events happened, there was no safety impact on the operations.

• This kind of event may have a very negative impact on the acceptance of the A-SMGCS by the controllers, even as an information system.

• During one of the shadow-mode trial sessions, a storm brought heavy rain in the southeast part of the airport, over runway 32L and 32R thresholds. It caused a lot of false plots in this area on the HMI, making it unusable during the event while visibility suddenly reduced.

• This kind of event may have a negative impact on the acceptance of the A-SMGCS by the controllers, even as an information system.

• Several cases of inaccurate line-up on the runway axis occurred for aircraft in final approach. A discontinuity was observed in the trajectory of approaching flights, suddenly passing from lined up on the runway axis to shifted several metres to the right or to the left of the runway axis just before landing. This inconsistency is probably due to a gap between the track detected by the approach radar and the track detected by the SMR. In some cases the two trajectory were displayed at the same time on the HMI, causing false runway incursion alerts.

EMMA



• This kind of event may have a very negative impact on the acceptance of the A-SMGCS by the controllers, even as an information system.

Finally, despite these problems that need to be addressed, the controllers seem to accept the A-SMGCS as an additional information tool. However, major improvements, mainly in terms of reliability of detection and identification, will have to be made for the controllers to trust the A-SMGCS enough and accept it as a control tool.

3.4.3.1.4.2 Detected loss of display (SA08) No total loss of A-SMGCS HMI display was observed during the shadow-mode trials. Only a few cases of mobiles loosing their synthetic plots for a very limited time were observed. The severity of the losses was mitigated by the continuity of the primary radar track. Although the controllers consider that there are too many missing reports to control the traffic in a safe and efficient way, they seem to accept the level of detected loss of display for the A-SMGCS as an additional information tool.

3.4.3.1.4.3 Detected lack of identifier (SA09) Many cases of lack of identifier were observed during the shadow-mode trials. The main categories of lack of identifier events are described hereafter: • In many cases for departing flights, pilots did not switch the transponder on before they lined up

on the runway threshold for takeoff. As a consequence the aircraft could not be identified before takeoff. As mitigation, the controller in charge of the flight could ask for the pilot to switch the transponder on. Nevertheless, it was chosen not to do it systematically not to disturb the controllers and pilots. Another solution would be a manual identification of the unidentified mobile. This kind of event could easily be avoided by modifying the transponder procedures for pilots.

• Several cases of lack of identifier were observed for departing flights where although the transponder was on (cross-check with the pilot), the aircraft was identified several tenths of seconds later. A latency time of several tenths of seconds was also observed on mode S equipped vehicles. As mitigation, the controller could identify the mobile manually.

• In many cases for departing and arriving flights, temporary or permanent losses of identification were observed. As mitigation, the controller could identify the mobile manually.

• In many cases, mainly for departing aircraft, the transponder code was the only information displayed on labels because flight plan data were not available. No mitigation was possible during the shadow-mode trials.

All the observed cases of lack of identifier may have a very negative impact on the acceptance of the A-SMGCS by the controllers, even as an information system. In summary, the controllers do not seem to accept the level of detected lack of identifier.

3.4.3.1.4.4 Detected incorrect identifier (SA10) Several cases of incorrect identifier were observed during the shadow-mode trials. They concerned label swaps between mobiles. More precisely, the label of an aircraft was ‘caught’ by an unidentified mobile taxiing in its vicinity (e.g. bus, push back truck). The controllers who witnessed these cases tried to manually dissociate the label from the incorrectly identified mobiles but most of the time this action was not possible. The controllers do not accept the level of detected incorrect identifier for the A-SMGCS as a control tool but it seems to be more acceptable if it is seen as an additional information tool.

3.4.3.1.4.5 Controller errors induced by A-SMGCS in visibility conditions 2 The ‘controller errors induced by A-SMGCS in visibility conditions 2’ metric could not be measured during the shadow-mode trials.

EMMA



3.4.3.2 Hypotheses In this section, the operational improvement hypotheses described in the Verification and Validation Test Plan Document [5] are validated or invalidated according to the results of the metrics measured during the shadow-mode trials.

3.4.3.2.1 Capacity

3.4.3.2.1.1 Capacity hypothesis 1 (Opi.cap.1) Hypothesis: Using A-SMGCS, the runway departure and arrival capacity will be maintained or increased, especially in visibility conditions 2 and 3. Validation: The hypothesis has been validated. The runway departure and arrival capacity should at least be maintained.


3.4.3.2.2.1 Efficiency hypothesis 1 (Opi.eff.1) Hypothesis: The number of pilot reports will be maintained or decreased, especially in visibility conditions 2 and 3. Validation: The hypothesis has been validated. The number of pilot reports should be decreased.

3.4.3.2.3 Human Factors

3.4.3.2.3.1 Human factors hypothesis 1 (Opi.hf.1) Hypothesis: Using A-SMGCS, the situation awareness of the controller will be improved, in particular the mental association between position and identification of mobiles. Validation: The hypothesis has been validated.

3.4.3.2.3.2 Human factors hypothesis 2 (Opi.hf.2) Hypothesis: The A-SMGCS display will be accepted by the controller. Validation: The hypothesis has been validated.

3.4.3.2.3.3 Human factors hypothesis 3 (Opi.hf.3) Hypothesis: The controller will state that A-SMGCS is usable. Validation: The hypothesis has been validated.

EMMA



3.4.3.2.4 Safety

3.4.3.2.4.1 Safety hypothesis 1 (Opi.saf.1) Hypothesis: The detected corruption of the A-SMGCS HMI won't have a significant negative impact on the safety of operations. The following events should be considered: • Detected incorrect displays, • Detected loss of display, • Detected lack of identifier, • Detected incorrect identifier. Validation: The hypothesis has been partly validated. The controllers consider that the detected incorrect displays, detected loss of display and detected incorrect identifier are acceptable to use A-SMGCS as an additional information tool but not as a control tool. They consider that the detected lack of identifier is not acceptable.

3.4.3.2.4.2 Safety hypothesis 2 (Opi.saf.2) Hypothesis: When using A-SMGCS, the number of controller errors will be reduced, especially under adverse weather conditions. Validation: The hypothesis could not be tested. This hypothesis should be tested during shadow-mode trials, when all the experimental A-SMGCS procedures have been performed.

3.4.3.3 High-level objectives In this section, the operational improvement high-level objectives described in the Verification and Validation Test Plan Document [5] are validated or invalidated according to the results of the hypotheses validation.

3.4.3.3.1 High-level objective 2 Objective: With use of EMMA A-SMGCS, the level of safety of airports will be maintained or even increased, especially under adverse weather conditions and in congested traffic situations. A full safety assessment of the system installed in Toulouse will be performed in EMMA2. A FHA and a very preliminary safety assessment are available in D1.3.9 of EMMA. Validation: The objective has been partly validated. The controllers consider that the A-SMGCS is acceptable from the safety point of view as an additional information source but not as a control tool.

3.4.3.3.2 High-level objective 4 Objective: With use of EMMA A-SMGCS, the Human Factors situation will be improved, especially under adverse weather conditions and in congested traffic situations. Section 2.6.14 and 2.6.15 of the ICAO manual on A-SMGCS [15] refer to operational requirements on the HMI and state that ‘where automation is available the automated systems should demonstrate an acceptable level of HMI efficiency’. It is expected that the assessment of these requirements will demonstrate that the Human Factors situation has been improved.

EMMA



Validation: The objective has been validated.

4 Validation session – fast-time simulations

4.1 Introduction The validation session subject of the current chapter provides a quantitative assessment of the potential operational improvements induced by the use of A-SMGCS at Toulouse-Blagnac airport. Since the system was not used operationally, no direct measurement of operational improvement indicators could be performed. Two methods were used in order to assess capacity and efficiency indicators. First, the evolution of some indicators could directly be derived from the results of the questionnaires and the conclusions of the use of A-SMGCS procedures, provided in §3, Validation session – shadow mode trials. For instance, the taxi time in low visibility conditions could be directly derived from the current measurement of the taxi time in low visibility operations, from the average time length of a position report by a pilot, and from the fact that the controllers were ready to reduce the number of pilot reports using A-SMGCS. The other indicators were assessed using a very simple fast-time simulator. Due to the fact that the capacity and efficiency indicators were not directly measured, the results given in this section should be considered with great caution. They can only be seen as ‘potential’ improvements induced by the use of A-SMGCS. Furthermore, as the fast-time simulation only partially reflects the complexity of traffic management on the Toulouse platform, the results provided by the simulation should not be considered as ‘absolute’ values: only the relative evolution of an indicator can give an indication of the corresponding operational improvement.

4.2 Data description and data collection methods

4.2.1 Raw data The fast-time simulation has been performed with two traffic scenarios, derived from a day with high traffic on Toulouse-Blagnac airport (25th June 2004): • The first hour (i.e. from 07:00 to 08:00 UT) is a period of very high traffic with 14 departing and

11 arriving flights, • The second hour (i.e. from 16:00 to 17:00 UT) is a period of intermediate traffic, with 9 departing

and 6 arriving flights. During these hours, runways 32R and 32L were active. Each of these traffic scenarios has been run with four different configuration files, each corresponding to different operational conditions: • Normal visibility operations (with or without A-SMGCS, two runways), • Low visibility operations without A-SMGCS (one runway), • Low visibility operations with A-SMGCS (one runway), • Low visibility operations with A-SMGCS (two runways). A detailed description of these four operational conditions and assumptions made for each of them is provided in the next section.

EMMA



4.2.2 Additional data

4.2.2.1 Description of the operational conditions

4.2.2.1.1 Normal visibility operations During normal visibility conditions, both runways can be used. Runway 32R is dedicated to departures. There is on average a two-minute waiting time between two takeoffs. Runway 32L is dedicated to arrivals. The time separations between two landings are dependent on the aircraft types and ICAO separation standards. Based on observation of real traffic during peak hours, it can be reasonably assumed that the average time separation between two landing aircraft is two minutes. The runways are dependent. However, it should be noted that: • there is no delay between a landing and a following takeoff; • there is a two minutes time separation between a takeoff and the following landing. The runway capacity is 46 movements/hour. The average unimpeded taxi time is 4 minutes, for departures and arrivals.

4.2.2.1.2 Low visibility operations without A-SMGCS Currently, no A-SMGCS is used in low visibility operations at Toulouse-Blagnac. Only runway 32R, equipped with ILS cat. III, is used. For this reason, compared to normal visibility conditions, a supplementary constraint is added: takeoffs become dependent on landings. The average time separation is about two minutes. In low visibility operations, only one aircraft at a time is allowed on the main taxiway, which generates a strong constraint on the ground traffic flow. The average unimpeded taxi time is 5 minutes, due to the fact that a pilot report is necessary at each taxiway intersection. Between three and four taxiway intersections are crossed from the stand to the runway threshold, or from the runway exit to the stand. The airport capacity reduces to 12 aircraft per hour (i.e. one movement every five minutes). The controllers make records of every vehicle on the manoeuvring area.

4.2.2.1.3 Low visibility operations with A-SMGCS (one runway) Following the results and analysis of the questionnaires filled in by the controllers, several conclusions can be made about the use of A-SMGCS procedures during low visibility operations: • The controllers were not ready to trust A-SMGCS in order to reduce horizontal separations. They

considered that the rule ‘one aircraft on the main taxiway at a time’ would still be valid when using the A-SMGCS. They would also continue to make records of every vehicle on the manoeuvring area.

• The controllers do not demand to be able to perform multiple line-up or rolling takeoffs in low visibility operations. They also insist on still having a pilot report at runway exit for arrivals. No significant increase of the unique runway capacity is expected (i.e. still 12 aircraft per hour).

• However, they agree on the fact that the number of pilot position reports could be reduced. If only one aircraft is present on the main taxiway, it could be assumed that the pilot would not be forced to give a position report at each runway intersection. In the current scenario, it was assumed that the taxi time (without holding time) could be reduced to 4 minutes, i.e. the estimated average unimpeded value in normal visibility conditions.

The current visibility transition limit is 1000m. A lot of controllers claim that the limit could be reduced because there are situations when low visibility procedures are activated while pilots can still see the other aircraft. Some controllers suggested a reduction to 800m or 600m. It could result in a great increase of taxiway capacity in visibility 2 conditions. However, the results of the questionnaires are not statistically significant, so this potential improvement was not considered for the fast-time simulations scenarios.

EMMA



4.2.2.1.4 Low visibility operations with A-SMGCS (two runways) Several controllers suggested that during low visibility operations, both runways could be used simultaneously, as in normal visibility operations. The reason why the second runway is not activated in low visibility procedures is related to the absence of adapted ground lighting and stop bars. Some controllers suggest that an A-SMGCS HMI could make up for this lack. A safety case should be performed in order to confirm this hypothesis. However, considering the fact that a full implementation of A-SMGCS would imply the installation of appropriate ground lighting and stop bars, this scenario has been chosen in order to assess the potential capacity and efficiency improvements in low visibility operations when both runways are active. The separation rules on the runways are the same as in normal visibility operations. The other parameters are the same as in the ‘low visibility operations with A-SMGCS (one runway)’ scenario.

4.2.2.2 Fast-time simulation assumptions Two types of fast-time simulators have been developed, following two different types of traffic management rules: • The first version is based on the ‘first come, first served’ rule. If two pilots (e.g. of a departing

flight and an arriving flight) request a taxi clearance at the same time, priority is given to the one with the earliest estimated off-block time (EOBT) for departures or estimated landing time (ELDT) for arrivals.

• The second version gives priority to arrivals whether the conflict concerns the runway or the request for taxi clearance.

The first version, which is more equitable, has been tested first. In low visibility operations, the results lead to a high waiting time of arriving aircraft after landing and before taxi to stand. On Toulouse-Blagnac airport it is not realistic to assume that several aircraft would be held back near runway exits while waiting for a taxi clearance. For this reason, it has been assumed that the controller would give priority to arriving aircraft and that he would try to expedite the arriving traffic to the parking stands. The second version has then been chosen for the fast-time simulations. However, this assumption should also be qualified, because this traffic management method could easily lead to a saturation of the apron. In the fast-time simulator, no management of stand allocation has been performed. It is likely that the controller confronted to low visibility conditions would choose a more subtle traffic management method, taking into account stand allocations, departures requesting push back and arrivals requesting taxi clearance to the stands. However, this subtle management method could not be easily simulated. This is one of the limits of these fast-time simulations. The first simulation sessions were performed on a whole day of traffic. It appeared that during low visibility operations, the departure and arrival delays quickly became unacceptable. It is likely that, if low visibility conditions were to last several hours, Toulouse control would request a regulation to the Central Flow Management Unit (CFMU). For this reason, it has been decided to focus only on one hour of traffic for each traffic scenario. This simulates the case of a sudden reduction of visibility conditions, when the traffic situation can only be solved tactically, using the A-SMGCS and procedural control. This is also consistent with the fact that priority is given to arriving flights, since they could not have been regulated earlier.

4.3 Data analysis

4.3.1 Data analysis method A description of the fast-time simulation algorithm is provided in the D6.1.3 Toulouse V&V Test Plan [5]. The algorithm that was described in this document was the ‘first come, first served’ algorithm. The ‘arrivals first’ algorithm is slightly different: it systematically gives priority to arrivals, if an arriving aircraft is about to land or request a taxi clearance. The fast-time simulator is run: • For each traffic scenario described in section 4.2.1 Raw data;

EMMA



• For each set of environmental conditions described in section 4.2.2.1 Description of the operational conditions.

The values for each considered capacity and efficiency indicator are directly provided by the fast-time simulator. A comparative analysis is then performed on these values. For each scenario and environmental condition, the following indicators have been evaluated: • The departure runway throughput. It is stated in number of aircraft taking off from the airport

per hour. • The arrival runway throughput. It is stated in number of aircraft landing at the airport per hour. • The maximum number of aircraft simultaneously taxiing per hour. Both arriving and

departing aircraft have been considered. A departing aircraft is considered as taxiing between its actual off-block time (AOBT) and actual takeoff time (ATOT – i.e. AOBT plus actual taxi-out time). An arriving aircraft is considered as taxiing between its actual landing time (ALDT) and actual in-block time (AIBT – i.e. ALDT plus actual taxi-in time).

• The maximum number of aircraft simultaneously waiting in the departure queue. A departing aircraft is considered as waiting in the departure queue when it has left the stand (i.e. AOBT has been recorded), the average unimpeded taxi time has been exceeded and it has not taken off yet (i.e. ATOT has not been recorded yet).

• The maximum number of aircraft pushing back per 10 minutes time slot. A departing aircraft is considered as pushing back in a given 10 minutes time slot if its AOBT is recorded during that time period.

• The minimum, mean and maximum departure taxi times. They are stated in minutes. • The minimum, mean and maximum arrival taxi times. They are stated in minutes. • The mean push back delay. The push back delay of an aircraft is calculated as the difference, in

minutes, between its Actual Off-Block Time (AOBT) and its EOBT. • The mean departure queuing delay. The departure queuing delay of an aircraft is calculated as

the difference, in minutes, between its actual taxi time (i.e. ATOT minus AOBT) and the average unimpeded taxi time.

• The mean departure delay. The departure delay of an aircraft is calculated as the difference, in minutes, between its ATOT and its estimated takeoff time (ETOT – i.e. EOBT plus average taxi-out time). This is only the ground ATC delay. No other delay has been taken into consideration.

• The mean arrival sequencing delay. The arrival queuing delay of an aircraft is calculated as the difference, in minutes, between its ALDT and ELDT.

• The mean arrival queuing delay. The arrival queuing delay of an aircraft is calculated as the difference, in minutes, between its actual taxi time (i.e. AIBT minus ALDT) and the average unimpeded taxi time.

• The mean arrival delay. The arrival delay of an aircraft is calculated as the difference, in minutes, between its ALDT and ELDT. This delay is only composed of the sequencing and ground ATC delays. No other delay has been taken into consideration.

• The number of aircraft departing on time. A departing aircraft is considered as ‘on time’ if its ATOT is equal to its ETOT.

• The departure punctuality index. It is calculated as the ratio between the number of aircraft taking off on time or with a less than 10 minutes delay and the total number of departing aircraft.

• The number of aircraft arriving on time. An arriving aircraft is considered as ‘on time’ if its ALDT is equal to its ELDT.

• The arrival punctuality index. It is calculated as the ratio between the number of aircraft landing on time or with a less than 10 minutes delay and the total number of arriving aircraft.

The following section presents the raw results of the fast-time simulation. The results of each measured indicator are presented in tables. All the tables are organised as follows: • Each ‘Environmental Conditions’ column corresponds to a set of environmental condition where

NVO stands for normal visibility operations, LVO stands for low visibility operations and RWY stands for runway.

• The line entitled ‘High Traffic’ corresponds to the first scenario (i.e. one hour of high traffic from 07:00 to 08:00 UT on the 25th June 2004).

EMMA



• The line entitled ‘Intermediate Traffic’ corresponds to the second scenario (i.e. one hour of intermediate traffic from 16:00 to 17:00 UT on the 25th June 2004).

4.3.2 Measured indicators

4.3.2.1 General data

4.3.2.1.1 Departure and arrival runway throughput Environmental Conditions

Indicator Scenario NVO 2 RWYs

LVO without A-SMGCS

1 RWY

LVO with A-SMGCS 1 RWY

LVO with A-SMGCS 2 RWYs

High Traffic 11 DEP / 10 ARR

2 DEP / 11 ARR

3 DEP / 11 ARR

4 DEP / 11 ARR

Departure / Arrival Runway Throughput

(in Aircraft per Hour) Intermediate Traffic 9 DEP / 6 ARR 5 DEP / 6 ARR 5 DEP / 6 ARR 5 DEP / 6 ARR

Table 4: Fast-Time Simulations Indicators – Departure and Arrival Runway Throughput

4.3.2.1.2 Maximum number of aircraft simultaneously taxiing Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 5 1 1 1 Maximum Number of Aircraft

Simultaneously Taxiing per Hour Intermediate Traffic 4 1 1 1

Table 5: Fast-Time Simulations Indicators – Maximum Number of Aircraft Simultaneously Taxiing

4.3.2.1.3 Maximum number of aircraft simultaneously waiting in the departure queue

Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 3 1 1 0 Maximum Number of Aircraft

Simultaneously Waiting in the

Departure Queue Intermediate Traffic 0 1 2 0

Table 6: Fast-Time Simulations Indicators – Maximum Number of Aircraft Simultaneously Waiting in the Departure Queue

EMMA



4.3.2.1.4 Maximum number of aircraft pushing back per 10 minutes time slot Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 5 1 1 1 Maximum Number of Aircraft Pushing

Back per 10 Minutes Time Slot Intermediate Traffic 3 1 1 1

Table 7: Fast-Time Simulations Indicators – Maximum Number of Aircraft Pushing Back per 10 Minutes Time Slot

4.3.2.1.5 Minimum departure taxi time Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 4 5 4 4 Minimum Departure Taxi Time (in

Minutes) Intermediate Traffic 4 5 4 4

Table 8: Fast-Time Simulations Indicators – Minimum Departure Taxi Time

4.3.2.1.6 Mean departure taxi time Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 6.57 6.07 5.29 4.14 Mean Departure Taxi Time (in Minutes) Intermediate Traffic 4.44 5.11 4.67 4.11

Table 9: Fast-Time Simulations Indicators – Mean Departure Taxi Time

4.3.2.1.7 Maximum departure taxi time Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 14 20 15 5 Maximum Departure Taxi Time (in


Table 10: Fast-Time Simulations Indicators – Maximum Departure Taxi Time

4.3.2.1.8 Minimum arrival taxi time Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 4 5 4 4 Minimum Arrival Taxi Time (in


Table 11: Fast-Time Simulations Indicators – Minimum Arrival Taxi Time

EMMA



4.3.2.1.9 Mean arrival taxi time Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 4 12.09 12.36 12.36 Mean Arrival Taxi Time (in Minutes) Intermediate Traffic 4 9 6.5 4.83

Table 12: Fast-Time Simulations Indicators – Mean Arrival Taxi Time

4.3.2.1.10 Maximum arrival taxi time Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 4 22 20 20 Maximum Arrival Taxi Time (in


Table 13: Fast-Time Simulations Indicators – Maximum Arrival Taxi Time

4.3.2.2 Departure delays

4.3.2.2.1 Mean push back delay Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 0 67.21 46.43 46.43 Mean Push Back Delay (in Minutes) Intermediate Traffic 0 16.22 10.67 11.22

Table 14: Fast-Time Simulations Indicators – Mean Push Back Delay

4.3.2.2.2 Mean departure queuing delay Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 2.57 1.07 1.29 0.14 Mean Departure Queuing Delay (in

Minutes) Intermediate Traffic 0.44 0.11 0.67 0.11

Table 15: Fast-Time Simulations Indicators – Mean Departure Queuing Delay

4.3.2.2.3 Mean departure delay Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 2.57 68.29 47.71 46.57 Mean Departure Delay (in Minutes) Intermediate Traffic 0.44 16.33 11.33 11.33

Table 16: Fast-Time Simulations Indicators – Mean Departure Delay

EMMA



4.3.2.3 Arrival delays

4.3.2.3.1 Mean arrival sequencing delay Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 1.27 0.36 0.36 0.36 Mean Arrival Sequencing Delay (in


Table 17: Fast-Time Simulations Indicators – Mean Arrival Sequencing Delay

4.3.2.3.2 Mean arrival queuing delay Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 0 7.09 8.36 8.36 Mean Arrival Queuing Delay (in

Minutes) Intermediate Traffic 0 4 2.50 0.83

Table 18: Fast-Time Simulations Indicators – Mean Arrival Queuing Delay

4.3.2.3.3 Mean arrival delay Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 1.27 7.45 8.73 8.73 Mean Arrival Delay (in Minutes) Intermediate Traffic 0 4 2.50 0.83

Table 19: Fast-Time Simulations Indicators – Mean Arrival Delay

4.3.2.4 Departure punctuality

4.3.2.4.1 Number of aircraft departing on time Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 5/14 1/14 1/14 1/14 Number of Aircraft Departing On Time vs. Total Number of Departing Aircraft

per Hour Intermediate Traffic 7/9 1/9 2/9 3/9

Table 20: Fast-Time Simulations Indicators – Number of Aircraft Departing On Time

EMMA



4.3.2.4.2 Departure punctuality index Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 1 0.07 0.14 0.21 Departure Punctuality Index Intermediate Traffic 1 0.44 0.44 0.55

Table 21: Fast-Time Simulations Indicators – Departure Punctuality Index

4.3.2.5 Arrival punctuality

4.3.2.5.1 Number of aircraft arriving on time Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 4/11 1/11 1/11 1/11 Number of Aircraft

Arriving On Time vs. Total Number of

Arriving Aircraft per Hour

Intermediate Traffic 6/6 0/6 2/6 4/6

Table 22: Fast-Time Simulations Indicators – Number of Aircraft Arriving On Time

4.3.2.5.2 Arrival punctuality index Environmental Conditions


LVO without A-SMGCS

1 RWY



High Traffic 1 0.73 0.64 0.64 Arrival Punctuality Index Intermediate Traffic 1 1 1 1

Table 23: Fast-Time Simulations Indicators – Arrival Punctuality Index

4.4 Results This section presents the results of the validation session in terms of operational improvements. For each stage, the results of the metrics primarily defined in the Verification and Validation Test Plan Document [5] are detailed. The validation of the hypotheses and high-level objectives specified in that document are derived from the measured metrics.


4.4.1.1 Metrics The results of the individual metrics related to operational improvements listed in this section were measured during the fast-time simulation. The characters between parentheses in each metric title indicate the reference of the indicator used in D6.1.3: Verification and validation test plan for Toulouse-Blagnac airport, version 1.02, issued on the 15th February 2006 [5]. These figures are unfortunately not always consistent with D6.2.2: Indicators and Metrics for A-SMGCS [7].

EMMA



4.4.1.1.1 Capacity

4.4.1.1.1.1 Assessment of departure and arrival runway throughput in visibility conditions 2 and 3 (CA01 & CA02)

The fast-time simulation only partially reflects the complexity of traffic management on the Toulouse-Blagnac platform. Therefore, the departure and runway throughput measured during the simulation cannot be used in absolute terms. Nevertheless, the relative evolutions of the departure and arrival runway throughputs from normal operations to the various scenarios in low visibility conditions show that: • The departure throughput is slightly improved (i.e. one more aircraft per hour, representing a 7%

increase) in low visibility conditions, high traffic situation and with one runway activated when the A-SMGCS used, compared to the same situation when the A-SMGCS is not used.

• The departure throughput is slightly improved (i.e. one more aircraft per hour, representing a 7% increase) in low visibility conditions, high traffic situation and when A-SMGCS is used with two runways activated, compared to the same situation with only one runway active.

• The departure throughput is significantly reduced in low visibility conditions, high traffic situation, whether the A-SMGCS is used or not and whether the number of active runways.

• The departure throughput is unchanged in low visibility conditions, intermediate traffic situation, whether the A-SMGCS is used or not and whether the number of active runways.

• The arrival throughput is unchanged in low visibility conditions, whether the traffic situation, whether the A-SMGCS is used or not and whether the number of active runways. The arrival throughput is even slightly increased (i.e. one more aircraft per hour) in low visibility conditions and high traffic situation compared to normal operations. This is due to the fact that arrivals are given the priority in the simulator’s algorithm.

The use of the A-SMGCS may improve the runway throughput by increasing the departure runway throughput during periods of high traffic. It seems that it will have an actual operational impact only if both runways can be use in low visibility conditions. This is consistent with the controllers’ opinion. See section 3.4.3.1.1.2 Assessment of departure and arrival runway throughput in visibility conditions 2 and 3 (CA01 & CA02) for further details. It must also be noted that the gain in the departure runway throughput obtained by the use of the A-SMGCS is limited compared to the dramatic decrease of the departure runway throughput in low visibility conditions. It shows the limits of the benefits brought by the A-SMGCS when it is not used as a replacement of the outside view.

4.4.1.1.1.2 Assessment of mean and maximum number of aircraft simultaneously taxiing in visibility conditions 2 and 3 (CA09 & CA10)

The mean number of aircraft simultaneously taxiing in visibility conditions 2 and 3 has not been assessed during the fast time simulation. The relative evolutions of the maximum number of aircraft simultaneously taxiing from normal operations to the various scenarios in low visibility conditions show that: • The maximum number of aircraft simultaneously taxiing is strongly reduced during low visibility

conditions compared to normal operations. • The use of the A-SMGCS does not bring any improvement in the number of aircraft

simultaneously taxiing. These results are explained by ‘one aircraft on the main taxiway at a time’ rule, which is a strong constraint on the ground traffic in low visibility conditions. While this rule will be applied when the A-SMGCS is used (i.e. while the controllers’ trust in the system will be limited), the number of aircraft simultaneously taxiing will not be increased. See section 3.4.3.1.1.3 Assessment of mean and maximum number of aircraft simultaneously taxiing in visibility conditions 2 and 3 (CA09 & CA10) for further details.

EMMA



4.4.1.1.1.3 Assessment of mean and maximum number of push back clearances in visibility conditions 2 and 3 (CA07 & CA08)

The mean number of push back clearances in visibility conditions 2 and 3 has not been assessed during the fast time simulation. The relative evolutions of the maximum number of aircraft pushing back per 10 minutes time slot from normal operations to the various scenarios in low visibility conditions show that: • The maximum number of aircraft pushing back per 10 minutes time slot is strongly reduced during

low visibility conditions compared to normal operations. • The use of the A-SMGCS does not bring any improvement in the maximum number of aircraft

pushing back per 10 minutes time slot. The ‘one aircraft on the main taxiway at a time’ rule has a direct impact on the number of aircraft authorised to push back. Therefore, while the rule will be applied, the number of push back clearances will not be increased.


4.4.1.1.2.1 Taxiing delays (EF01) The metrics entitled ‘taxiing delays’ (EF10) corresponds to the indicators ‘mean departure queuing delay’ and ‘mean arrival queuing delay’ measured during the fast time simulation. The relative evolutions of the mean departure and arrival queuing delays from normal operations to the various scenarios in low visibility conditions show that: • The mean departure queuing delay is decreased by 84 to 89 % in low visibility conditions when

the A-SMGCS is used and two runways are active, compared to the same situation with only one runway active.

• When only one runway is active in low visibility conditions, the A-SMGCS does not help reduce the mean departure queuing delay.

• The A-SMGCS does not help reduce the mean arrival queuing delay. Note: when examining the mean departure and arrival queuing delays, it must be kept in mind that the average unimpeded taxi time is 5 minutes in low visibility conditions without A-SMGCS against 4 in all the other scenarios and environmental conditions. The ‘one aircraft on the main taxiway at a time’ rule is a strong constraint on the ground traffic in low visibility conditions. It is reflected in the mean departure and arrival queuing delays that should not be improved with the use of the A-SMGCS while the rule applies, except for departures if both runways were active.

4.4.1.1.2.2 Gate delays (EF02) The metrics entitled ‘gate delays’ (EF02) corresponds to the indicator ‘mean push back delay’ measured during the fast time simulation. The relative evolutions of the mean push back delay from normal operations to the various scenarios in low visibility conditions show that: • Important mean push back delays are observed in low visibility conditions (i.e. from 16 to 67

min.), while no push back delays were observed in normal operations. • The use of the A-SMGCS allows the mean push back delay to be reduced by 31 to 34% in low

visibility conditions. • The number of active runways has no impact on the mean push back delay in low visibility

conditions. The A-SMGCS has a positive impact on the mean push back delays in low visibility conditions, even in an intermediate traffic situation.

EMMA



4.4.1.1.2.3 Departure queuing delays (EF05) The relative evolutions of the mean departure queuing delay from normal operations to the various scenarios in low visibility conditions show that: • The mean departure queuing delay is lower in low visibility conditions than during normal

operations. This is most probably explained by the ‘one aircraft on the main taxiway at a time’ rule, which limits the ground traffic in low visibility conditions while several aircraft can taxi at the same time and then queue at the runway takeoff in normal conditions.

• The mean departure queuing delay is improved by 2 to 32 % in low visibility conditions, high traffic situation when the A-SMGCS is used and both runways are active compared to the other scenarios in low visibility conditions.

The A-SMGCS should not have a significant impact on the departure queuing delays in low visibility conditions (except if both runways were used). This is mainly due to the ‘one aircraft on the main taxiway at a time’ rule, which is a strong constraint on the ground traffic.

4.4.1.1.2.4 Arrival sequencing delays (EF06) The relative evolutions of the mean arrival sequencing delay from normal operations to the various scenarios in low visibility conditions show that: • In an intermediate traffic situation, no arrival sequencing delay has been measured in any of the

scenarios and environmental conditions. • In a high traffic situation, the mean arrival sequencing delay is less important by 72% in low

visibility conditions than during normal operations. This is most probably due to the choice made to give priority to arriving flights in low visibility conditions.

• In a high traffic situation and low visibility conditions, the use of the A-SMGCS has no influence on the mean arrival sequencing delay.

The A-SMGCS does not seem to improve the arrival sequencing delays in low visibility conditions. Nevertheless, it is mitigated by the deliberate choice to run the simulation in a context where priority is given to arriving flights. See section 4.2.2.2 Fast-time simulation assumptions for further details.

4.4.1.1.2.5 Mean departure delays (EF07 & EF204) The relative evolutions of the mean departure delays from normal operations to the various scenarios in low visibility conditions show that: • The mean departure delay is strongly increased (i.e. multiplied by 18 to 37) in low visibility

conditions compared to normal operations. • The use of the A-SMGCS improves the mean departure delay by 30 to 32 % in low visibility

conditions in both traffic situations and whether one or two runways are active. The mean departure delay is a key metrics for evaluating the efficiency of the A-SMGCS. The results obtained show that the A-SMGCS could help significantly reduce the departure delays in low visibility conditions, especially during periods of high traffic. The metrics also show that the use of both runways would not change the results. This is most probably due to the ‘one aircraft on the main taxiway at a time’ rule, which is very limitative for the ground traffic in low visibility conditions. It must also be noted that the gain in the mean departure delay obtained by the use of the A-SMGCS is limited compared to the dramatic increase of the mean departure delay in low visibility conditions. It shows the limits of the benefits brought by the A-SMGCS when it is not used as a replacement of the outside view.

4.4.1.1.2.6 Mean arrival delays (EF08 & EF205) The relative evolutions of the mean arrival delays from normal operations to the various scenarios in low visibility conditions show that: • The mean arrival delay increases in low visibility conditions compared to normal operations.

4 The number of communications (EF20) could not be evaluated during the validation sessions. 5 The number of communications (EF20) could not be evaluated during the validation sessions.

EMMA



• The mean arrival delay increases in low visibility conditions, high traffic situation whether one or two runways are active when the A-SMGCS is used, compared to the same situation without A-SMGCS. The increase happens when aircraft have already landed (Cf. mean arrival queuing delay). The constraint of the ‘one aircraft on the main taxiway at a time’ rule being stronger than the benefit brought by the use of the A-SMGCS (i.e. reduction of the number of position reports by pilots), leading to a similar taxi time with or without A-SMGCS.

• The mean arrival delay is improved in low visibility conditions, intermediate traffic situation when one runway is active and A-SMGCS is used, compared to the same situation without A-SMGCS.

• The mean arrival delay is improved in low visibility conditions, intermediate traffic situation when the A-SMGCS is used and both runways are active, compared to the same situation with only one runway active.

The fast-time simulation shows that the A-SMGCS could help reduce arrival delays at least during intermediate traffic periods. The efficiency of the system is not proven when the traffic is high. Note: the computed mean departure and mean arrival delays are influenced by the priority given to arrivals in the simulator's algorithm. An algorithm, based on the ‘first come, first served’ principle, was also tested. This algorithm gives priority to the most delayed aircraft, whether it is a departure or an arrival. The results obtained were more balanced between departure and arrival delays than those obtained with the ‘arrivals first’ algorithm. However, it is interesting to notice that the global reduction of delays (i.e. departure and arrival delays) is equivalent with both algorithms. This tends to suggest that this global reduction of ground ATC delay could be expected, whatever the traffic control method. Only the share between departure and arrival delays would change.

4.4.1.1.2.7 Punctuality index – number of mobiles on time (EF09 & EF19) The relative evolutions of the departure and arrival punctuality indexes from normal operations to the various scenarios in low visibility conditions show that: • The departure punctuality index strongly decreases in low visibility conditions compared to

normal operations. • The departure punctuality index is improved in low visibility conditions, high traffic situation

when one runway is active and A-SMGCS is used, compared to the same situation without A-SMGCS.

• Whatever the traffic situation, the departure punctuality index is improved in low visibility conditions when the A-SMGCS is used and both runways are active, compared to the same situation with only one active runway.

• The departure punctuality is unchanged in low visibility conditions, intermediate traffic situation and with one active runway whether the A-SMGCS is used or not.

• The arrival punctuality is slightly decreased in low visibility conditions, high traffic situation whether the A-SMGCS is use or not and whether one or two runways are activated, compared to normal operations.

• The arrival punctuality is unchanged in intermediate traffic situation whatever the scenario and environment conditions.

The A-SMGCS seems to have a positive impact on the departure punctuality index in low visibility conditions. This is mainly perceptible during periods of high traffic and / or if both runways can be used. However, the gain in the departure punctuality obtained by the use of the A-SMGCS is limited compared to the dramatic decrease of the departure punctuality in low visibility conditions. It shows the limits of the benefits brought by the A-SMGCS when it is not used as a replacement of the outside view. The impact of the A-SMGCS on the arrival punctuality index is not proven, keeping in mind that it has been deliberately chosen to give priority to arriving traffic in the simulation algorithm.

EMMA



4.4.1.1.2.8 Minimum, mean and maximum departure and arrival taxi times in visibility conditions 2 and 3 (EF10, EF11 & EF12)

The relative evolutions of the minimum, mean and maximum departure and arrival taxi times from normal operations to the various scenarios in low visibility conditions show that: • In all the scenarios and environmental conditions, the minimum departure taxi time is equal to the

average unimpeded taxi time. • Whatever the traffic situation, the mean departure taxi time is improved in low visibility

conditions when one runway is active and the A-SMGCS is used, compared to the same situation without A-SMGCS.

• Whatever the traffic situation, the mean departure taxi time is improved in low visibility conditions when the A-SMGCS is used and both runways are active, compared to the same situation with only one active runway.

• The maximum departure taxi time is improved in low visibility conditions, high traffic situation when one runway is active and A-SMGCS is used, compared to normal operations.

• Whatever the traffic situation, the maximum departure taxi time is improved in low visibility conditions when the A-SMGCS is used and two runways are active, compared to the same situation with only one active runway.

• The maximum departure taxi time increases in low visibility conditions, intermediate traffic situation when one runway is active and A-SMGCS is used compared to the same situation without A-SMGCS.

• The minimum arrival taxi time increases in low visibility conditions, intermediate traffic situation when one runway is active and without A-SMGCS, compared to normal operations. In all the other scenarios and environmental conditions, it is equal to the average unimpeded taxi time.

• The mean and maximum arrival taxi times strongly increase in low visibility conditions compared to normal operations.

• In high traffic situation, the mean and maximum arrival taxi times are unchanged in low visibility conditions whether the A-SMGCS is used or not and whatever the number of active runways.

• In intermediate traffic situation, the mean and maximum arrival taxi times are improved in low visibility conditions when one runway is active and the A-SMGCS is used, compared to the same situation without A-SMGCS.

• In intermediate traffic situation, the mean and maximum arrival taxi times are improved in low visibility conditions when the A-SMGCS is used and both runways are active, compared to the same situation with only one active runway.

The A-SMGCS may help reduce the departure taxi time in low visibility conditions, even more if both runways may be used. However; the improvement of arrival taxi time seems to be limited to intermediate traffic situations. These results are consistent with the controllers’ expectations. See section 3.4.3.1.2.1 Minimum, mean and maximum departure and arrival taxi times in visibility conditions 2 and 3 (EF10, EF11 & EF12) for further details.

4.4.1.1.2.9 Mean and maximum number of aircraft in the departure queue (EF13 & EF14) The mean number of aircraft waiting in the departure queue has not been assessed during the fast time simulation. The relative evolutions of the maximum number of aircraft simultaneously waiting in the departure queue from normal operations to the various scenarios in low visibility conditions show that: • In high traffic situation, the maximum number of aircraft simultaneously waiting in the departure

queue is lower in low visibility conditions than in normal operations. This is most probably due to the ‘one aircraft on the main taxiway at a time’ rule, which limits the number of aircraft on the taxiways in low visibility conditions and consequently the number of aircraft in the departure queue.

• In intermediate traffic situation, the maximum number of aircraft simultaneously waiting in the departure queue does not significantly vary whatever the scenarios and environmental conditions.

EMMA



The A-SMGCS does not seem to have a significant influence on the number of aircraft in the departure queue. The ‘one aircraft on the main taxiway at a time’ rule may bias the result for this metrics since it strongly limits the number of aircraft on the taxiing simultaneously.

4.4.1.2 Hypotheses In this section, the operational improvements hypotheses described in the Verification and Validation Test Plan Document [5] are validated or invalidated according to the results of the metrics measured during the shadow-mode trials.

4.4.1.2.1 Capacity

4.4.1.2.1.1 Capacity hypothesis 2 (Opi.cap.2) Hypothesis: Using A-SMGCS, the taxiway capacity will be maintained or will increase, especially in visibility conditions 2 and 3. The following parameters will be considered: • Mean number of simultaneously taxiing aircraft, • Maximum number of simultaneously taxiing aircraft. Validation: The hypothesis has been validated. The taxiway capacity should at least be maintained.

4.4.1.2.1.2 Capacity hypothesis 3 (Opi.cap.3) Hypothesis: Using A-SMGCS, the apron and gate capacity will be maintained or will increase, especially in visibility conditions 2 and 3. The following parameters will be considered: • Mean number of push back clearances, • Maximum number of push back clearances. Validation: The hypothesis has been validated. The apron and gate capacity should at least be maintained.

4.4.1.2.1.3 Capacity hypothesis 4 (Opi.cap.4) Hypothesis: Using A-SMGCS, the runway departure and arrival throughput will be maintained or increased, especially in visibility conditions 2 and 3. Validation: The hypothesis has been validated. The departure runway throughput should be maintained in intermediate traffic situations and increased in high traffic situations. The arrival runway throughput should be maintained.


4.4.1.2.2.1 Efficiency hypothesis 2 (Opi.eff.2) Hypothesis: The taxi time will be maintained or decreased, especially in visibility conditions 2 and 3. The following parameters will be considered: • Mean departure taxi time, • Mean arrival taxi time, • Minimum and maximum taxi times.

EMMA



Validation: The hypothesis has been validated. The departure taxi time should be decreased. The arrival taxi time should be maintained in high traffic situations and decreased in intermediate traffic situations.

4.4.1.2.2.2 Efficiency hypothesis 3 (Opi.eff.3) Hypothesis: Related to the reduction of taxi time, the delays due to ATC ground operations will be maintained or decreased, especially in visibility conditions 2 and 3. The following delays will be considered: • Taxiing delays, • Gate delays, • Departure queuing delays, • Arrival sequencing delays, • Mean departure delays, • Mean arrival delays. Validation: The hypothesis has been validated.

4.4.1.3 High-level objectives In this section, the operational improvements high-level objectives described in the Verification and Validation Test Plan Document [5] are validated or invalidated according to the results of the hypotheses validation.

4.4.1.3.1 High-level objective 1 Objective: With use of EMMA A-SMGCS, the level of capacity of airports will be maintained or even increased, especially under adverse weather conditions and in congested traffic situations. Section 2.2.3 of the ICAO manual on A-SMGCS states that ‘an A-SMGCS should be capable of operating at a specified movement rate in visibility conditions down to the aerodrome visibility operational level (AVOL). When visibility conditions are reduced to below AVOL an A-SMGCS should provide for a reduction of surface movements of aircraft and vehicles to a level acceptable for the new situation’. It is expected that the assessment of this requirement will help define levels of acceptable traffic in different visibility conditions, and that these levels will be higher than the current ones. Validation: The objective has been partly validated. The level of capacity of airports should be maintained and maybe increased but levels of acceptable traffic in different visibility conditions could not be defined.

4.4.1.3.2 High-level objective 3 Objective: With use of EMMA A-SMGCS, the efficiency of traffic movements will be increased, especially under adverse weather conditions and in congested traffic situations. Along with capacity assessment, it is expected to demonstrate that the efficiency will also be increased, especially in Visibility 2 conditions. Validation: The objective has been validated.

EMMA



5 Digest and conclusions Both shadow-mode trials and fast-time simulations performed during the validation sessions of the A-SMGCS implemented in Toulouse-Blagnac airport permitted to evaluate the operational feasibility of the system. The potential operational improvements brought by the A-SMGCS were also assessed during the validation sessions. The main findings are summarised in this conclusion section. The controllers are convinced that an A-SMGCS would be useful in Toulouse-Blagnac airport. This is why they are supporting the operational implementation of an A-SMGCS in Toulouse-Blagnac. The controllers are satisfied with the A-SMGCS HMI and the utility of the information displayed. However, the controllers are expecting improvements in terms of reliability for the current installed A-SMGCS to be used as a control tool and not only as an additional information source. The controllers would be ready to adapt some procedures to take the A-SMGCS into account (e.g. reduction of the number of position reports by the pilots in low visibility conditions) provided that some minor technical malfunctions are solved. The controllers seem to be conservative regarding the A-SMGCS in Toulouse-Blagnac. This is due to both their lack of experience with this new system and its lack of technical maturity. It prevents the A-SMGCS in Toulouse-Blagnac to be fully validated. However, the feelings expressed by the controllers of Toulouse-Blagnac airport about the A-SMGCS are close to the findings of the validation sessions performed in the other test-sites of the EMMA project. Therefore, they participate to the global validation of the A-SMGCS concept. The controllers are expecting the A-SMGCS to bring significant benefits in terms of safety and human factors. In particular, the controllers are very interested in the surface conflict alert function of the A-SMGCS even if they are expecting some improvements in its implementation before operational use. The shadow-mode trials revealed that the controllers did not generally express expectations for the potential benefits brought by the A-SMGCS in terms of capacity and efficiency. In normal visibility conditions, no benefits can be expected from the A-SMGCS in terms of capacity and efficiency. The fast-time simulations confirmed that the impact of low visibility conditions on a low or even an intermediate traffic situation is rather low. Therefore, minor benefits can be expected from the A-SMGCS in terms of capacity and efficiency in these situations. The fast-time simulations showed that the impact of low visibility conditions on a heavy traffic is much more significant. In Toulouse-Blagnac airport, periods of high traffic in low visibility conditions are currently managed by issuing CFMU regulations and creating stacks for the incoming traffic. The use of the A-SMGCS would help reduce delays in this kind of situation. Many controllers considered that the improvement would be even more significant if both runways could be used in low visibility conditions instead of one nowadays. This was confirmed by the results of the fast-time simulations. However, such a change would necessitate strong modifications of the infrastructures and procedures at Toulouse-Blagnac airport. The potential benefits that the A-SMGCS would bring if it were used in operations in Toulouse-Blagnac airport would remain limited while the system is seen as an additional information tool and not as a control tool by the controllers. In order to measure actual benefits of the A-SMGCS in low visibility conditions and high traffic situations, a lot of effort is still needed. In particular, the reliability of the current system has to be improved and the current procedures and working habits have to be adapted to the improved system. However, the frequency of such traffic situations should be assessed before deciding to go a step further in the development of the A-SMGCS in Toulouse-Blagnac.

EMMA



6 Annexes

6.1 Annex A: analysis tables for the shadow-mode validation trials The following sections present the results of the analysis in tables ordered according to the three topics assessed in the questionnaire: acceptance of technical performances for operational feasibility, procedures for operational feasibility and operational improvements. For each statement, the reference number and the English version of the statement are recapped. The total number of answers, the average and standard deviation of the answers are presented and all the comments made by the controllers are listed with a reference to the originator by using the coding ‘ATCOn’, where ATCO stands for air traffic controller and n is a number between 1 and 33 allowing anonymous reference to the author of the comment. Lines with a grey background correspond to statements where the controllers’ answers are not significant according to the P parameter of the one sample t-test.

EMMA Toulouse A-SMGCS verification and validation results


6.1.1 Operational feasibility: acceptance of technical performances

6.1.1.1 Surveillance

Ref. Statement Tot. Nb Answers Average SD P Comments

34

When visual reference is not possible on the manoeuvring area, the detection coverage (area in which the mobiles are detected) of the EMMA display is sufficient to control the traffic in a safe and efficient way

28 4.36 1.31 0.002

• ATCO1: Replace ‘manoeuvring area’ with ‘movement area’ in the statement. Totally disagree on the movement area because the apron is missing (in order to provide traffic information). But coverage is OK on the manoeuvring area.

• ATCO8: To be confirmed (thresholds, arrival on the runway, parking stands: OK).

• ATCO9: In principle, yes. • ATCO10: In principle, yes. • ATCO11: I do not know. • ATCO20: If the accuracy is guaranteed. • ATCO23: Too early to take a stand. • ATCO29: In principle, yes. • ATCO30: Not tested. • ATCO33: Technically OK but we do not use EMMA to exercise control. • ATCO35: If all the mobile are surely detected on the global view of the

airport: yes.




35

When visual reference is not possible on the manoeuvring area, the identification coverage (area in which the mobiles are identified) of the EMMA display is sufficient to control the traffic in a safe and efficient way.

25 3.96 1.46 0.110

• ATCO7: Not enough experience to judge. • ATCO8: The identification coverage is not OK. • ATCO9: Not possible to answer. • ATCO10: Not possible to test because no flight plan correlation for

departures. But arrivals OK. • ATCO11: I do not know. • ATCO12: Even if there are losses of identification, OK (comforted by the

pilot). • ATCO21: Not enough experience. • ATCO22: The departures are not often enough identified. • ATCO23: Not enough experience to take a stand. • ATCO29: I cannot answer. • ATCO30: Not tested. • ATCO33: We do not use EMMA to exercise control. • ATCO35: No but it is not a problem. Yes if improvements.

36

When visual reference is not possible on the manoeuvring area, the displayed position of a mobile on the manoeuvring area is accurate enough to control the traffic in a safe and efficient way.

30 4.21 1.24 0.004

• ATCO1: No precise knowledge. It is not pertinent to separate the runway sensitive area and manoeuvring area because there is no difference in the accuracy.

• ATCO2: Associated with specific use instructions. • ATCO8: Pilot’s help is necessary (not 100% confidence in the system). • ATCO10: To be tested. • ATCO11: Visibly, yes. • ATCO13: Except if there is a bug! • ATCO18: Not tested enough. It should be close to a control position. • ATCO20: It is too early to answer. I do not know the accuracy of the image. • ATCO21: Not enough experience. • ATCO23: Cf. question on the runway vacancy. • ATCO29: In principle the ground radar is OK (the analog looks accurate). But

EMMA is not reliable enough to work with (e.g.: problems of label jumps). • ATCO34: No for vacating the runway (3). Yes for the rest of the manoeuvring

area. • ATCO35: It depends on the protection volume decided.

If inaccuracy = 15m. then strongly agree.




37

The current update rate of the EMMA display is sufficient to control the traffic in a safe and efficient way when visual reference is not possible on the manoeuvring area.

31 5.10 0.79 0.000

• ATCO1: But the latency has to be known (approx. 5m. delay). Could be better. • ATCO8: I cannot answer. • ATCO9: I cannot answer. • ATCO11: I do not know. • ATCO12: In principle, yes. • ATCO21: Not enough experience.

38

When visual reference is not possible on the manoeuvring area, the current amount of missing reports on the EMMA display is not a problem to control the traffic in a safe and efficient way.

21 2.67 1.56 0.044

• ATCO1: Problem with the SMR. • ATCO7: Too early. • ATCO8: Rephrase the question. If there are missing reports, it is a problem. • ATCO9: I cannot answer. • ATCO10: Not experienced. • ATCO11: I do not know. • ATCO12: The primary radar is infallible. • ATCO14: Not enough experience to judge. • ATCO15: I did not observed the missing reports and their amount, therefore I

have difficulty in answering the question. Thus I will be number 1. • ATCO20: Not experienced. • ATCO21: Not enough experience. • ATCO22: Lack of practical experience. • ATCO23: Too early as things stand at the moment. • ATCO29: As soon as a report is missing, it is a problem.

There is always a problem when the system is not reliable. • ATCO31: ? • ATCO32: I cannot answer. • ATCO33: Because we do not use EMMA to exercise control. • ATCO34: We have to be sure to see everybody. • ATCO35: Knowing the comfort it brings, I am already very happy.

Any improvement of the accuracy is positive.




39

When visual reference is not possible on the manoeuvring area, the current amount of unidentified mobiles on the EMMA display is not a problem to control the traffic in a safe and efficient way.

29 2.86 1.53 0.080

• ATCO7: Lack of experience. • ATCO8: It is not a problem even without EMMA but it is less efficient. • ATCO10: Not experienced. • ATCO21: Not enough experience. • ATCO23: Too early as things stand at the moment. • ATCO33: Because we do not use EMMA to exercise control. • ATCO34: We have to be sure to see everybody.

40 When visual reference is not possible on the manoeuvring area, identifying a mobile is more efficient when using the EMMA display.

34 5.18 0.72 0.000

• ATCO1: Not enough trust in the system currently. • ATCO8: If EMMA works properly, yes. • ATCO22: It depends on the performance of the system. • ATCO28: If correlated, yes. • ATCO34: If we cannot see anything, EMMA is obviously more efficient.

41 Even when visual reference is possible on the manoeuvring area, identifying a mobile is more efficient when using the EMMA display.

34 4.59 1.21 0.000

• ATCO1: When it works. • ATCO3: That depends on its position related to the tower. • ATCO19: The information provided by EMMA is a confirmation. • ATCO20: Except night time. Useful when visual reference is not possible. • ATCO22: Because we can look outside. • ATCO28: If correlated, yes. • ATCO29: In non-visible areas in particular. • ATCO32: EMMA is a good confirmation mean. OK for the non-visible areas. • ATCO35: To remove a doubt if there are several vehicles.

42 Recognition of the aircraft type is more efficient with the EMMA display. 33 2.85 1.75 0.099

• ATCO1: Nothing guarantees the correctness of the information in the label. • ATCO9: If the aircraft has a label and if there is no error in the flight plan. • ATCO12: I hope so. Looking forwards to improved flight plan correlation. • ATCO20: Except if the aircraft type is incorrect. • ATCO22: At night or when the aircraft is not visible. • ATCO23: I am not expecting that from EMMA. • ATCO28: If the information comes from the transponder in the aircraft. • ATCO29: Problem if the aircraft type is incorrect.




43

When visual reference is not possible on the manoeuvring area, the amount of false reports on the EMMA display is not a problem to control the traffic in a safe and efficient way.

32 2.34 1.33 0.000

• ATCO1: Lots of false reports (problems SCA and SMR). • ATCO7: If experienced. • ATCO8: False reports are a problem. • ATCO10: Depends on the safety limit. • ATCO12: It may be annoying. • ATCO20: False reports are more annoying on the runway than on the

taxiways. • ATCO29: It is an important problem. • ATCO33: Because we do not use EMMA to exercise control. • ATCO35: Providing that the tool is known and real false reports are identified.

44

When visual reference is not possible on the manoeuvring area, the amount of incorrectly identified mobiles on the EMMA display is not a problem to control the traffic in a safe and efficient way.

29 2.10 1.29 0.000

• ATCO1: Already experienced before. • ATCO7: If experienced. • ATCO10: Problems if too many mobiles are not identified. • ATCO12: Very annoying. • ATCO20: Not experienced. • ATCO30: Assessment on the ground radar. Not enough experience with

EMMA. • ATCO33: Because we do not use EMMA to exercise control. • ATCO35: Personally not observed.

If exists: big problem.




45 Very frequently, I experienced swapping of labels between two mobiles. 18 3 1.19 0.072

• ATCO1: Problem when two mobiles pass each other (no continuous correlation).

• ATCO4: Not experienced. • ATCO5: Not tested. • ATCO7: I cannot answer. • ATCO9: Already experienced (not very frequently). • ATCO10: Not tested. • ATCO11: Not experienced. • ATCO12: Experienced on this day only. I do not know. • ATCO18: Experienced, but not enough. • ATCO20: Already experienced but not frequently enough. • ATCO29: Already experienced on the ground radar.

It should not happen. • ATCO30: Not tested. • ATCO31: Experienced once. • ATCO33: Not experienced. • ATCO34: And even dangerous ones!

Table 24: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Acceptance of Technical Performance – Surveillance



6.1.1.2 Alerts


46 When using the EMMA display, I experienced too many false alerts to use the conflict alert system.

16 3 1.41 0.207

• ATCO5: Not tested. • ATCO7: In principle interested in the alerts but not experienced. • ATCO10: False alerts not experienced. • ATCO12: Not seen. • ATCO13: Not experienced. • ATCO14: Not experienced. • ATCO15: Not tested. • ATCO17: No opinion. • ATCO18: Not experienced. • ATCO20: To be fine-tuned. • ATCO26: Not enough experience time. • ATCO29: Not seen… • ATCO30: Not tested. • ATCO31: Not seen. • ATCO33: Not experienced. • ATCO34: One observed. • ATCO35: Not experienced.




47 While using the EMMA display, I experienced traffic situations when an alarm should have been triggered but it has not been.

13 2,15 0,99 0,000

• ATCO1: I did not experience this situation. • ATCO2: Not experienced. • ATCO5: Not tested. • ATCO7: In principle interested in the alerts but not experienced. • ATCO9: Not experienced. • ATCO10: Not experienced. • ATCO12: Not experienced enough. • ATCO13: Not experienced. • ATCO14: Not experienced. • ATCO15: Not tested. • ATCO17: No opinion. • ATCO18: Not experienced. • ATCO20: Not experienced (but I have not used EMMA enough). • ATCO26: Not enough experience time. • ATCO29: Not seen… • ATCO30: Not tested. • ATCO31: Not seen. • ATCO34: Not enough experienced on EMMA to answer. • ATCO35: Not experienced.




48 The runway incursion alerts are triggered at the right moment. 13 3 1.68 0.422

• ATCO4: Too late. They should appear when the aircraft passes the holding point (setting up of a virtual holding point).

• ATCO5: Not tested. • ATCO7: In principle interested in the alerts but not experienced. • ATCO8: The alert is triggered too late (to be triggered at the holding point at

the latest). • ATCO9: Too late. • ATCO12: Not seen. • ATCO14: Not experienced. • ATCO15: Not seen. • ATCO18: Not experienced. • ATCO20: Not enough time. The experienced incursion alert is not much

relevant (arrival / departure distance too short). • ATCO22: A bit late (only one case experienced). • ATCO26: Not experienced. • ATCO28: There are still errors remaining. • ATCO31: Not seen. • ATCO32: Not tested enough. • ATCO34: Alert triggered too late (it is already on the runway). • ATCO35: Not observed.

49 The EMMA display can help me to detect and prevent runway incursions 27 4.81 0.83 0.000

• ATCO5: Not tested. • ATCO7: In principle interested in the alerts but not experienced. • ATCO8: EMMA would be better if the alerts were triggered earlier. • ATCO10: The presentation of the alerts is not good enough. • ATCO14: Not experienced. • ATCO18: In LVP, yes. • ATCO21: I hope so. • ATCO29: True even without the alerts. • ATCO31: General case.




50 The EMMA display can help me to detect and prevent incursions into restricted areas (e.g.: ILS area in LVP).

28 4.86 0.97 0.000

• ATCO7: In principle interested in the alerts but not experienced. • ATCO14: Not experienced. • ATCO21: I hope so. • ATCO30: A bit thin. • ATCO31: General case.

51 The EMMA display can help me to detect lit stop bar crossings. 25 5.04 1.02 0.000

• ATCO1: Because the obstacle free zone depends on the stop bars in LVP. • ATCO7: In principle interested in the alerts but not experienced. • ATCO14: Not experienced. • ATCO19: And especially to know which stop bar has been crossed. • ATCO22: EMMA helps only. • ATCO29: Yes for permanent stop bars not for remote controlled ones. • ATCO30: Not tested. • ATCO31: General case.

52 Information alerts are often popping up too late to take an appropriate action before an alarm is triggered.

13 4.23 0.83 0.018

• ATCO4: Especially for runway incursions. • ATCO5: ? • ATCO7: In principle interested in alerts but not experienced. • ATCO8: Not seen. • ATCO9: Not experienced. • ATCO10: Not tested. • ATCO12: To be confirmed after it has been used. • ATCO14: For an opposite runway heading. • ATCO18: Not experienced. • ATCO19: In case of opposite runway heading, an alert should be triggered as

soon as the aircraft is on taxiway P20 if it taxis to runway 32R and if the runway in use is 14L.

• ATCO20: Problems of setting: late short final alert. • ATCO22: Not experienced. • ATCO30: Not tested. • ATCO31: Not seen. • ATCO32: Not enough experience. • ATCO33: Not experienced. • ATCO35: Not observed.




53 Too many unnecessary information alerts are triggered. 9 3.11 1.27 0.460

• ATCO1: I did not observe any information alert. • ATCO2: Not experienced. • ATCO5: ? • ATCO7: In principle interested in the alerts but not experienced. • ATCO8: Not seen. • ATCO9: Not experienced. • ATCO10: Not tested. • ATCO12: I cannot answer. • ATCO14: Not experienced. • ATCO15: Not seen. • ATCO18: Not experienced. • ATCO19: I would like to be allowed to acknowledge manually some of the

alarms. • ATCO22: Not experienced. • ATCO30: Not tested. • ATCO31: Not seen. • ATCO32: No direct practical experience. • ATCO33: Not experienced. • ATCO35: Not observed.

Table 25: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Acceptance of Technical Performance – Alerts

6.1.1.3 HMI – efficiency


65 The image scaling function (zoom in / zoom out) is easy to use. 32 4.69 0.82 0.000

• ATCO1: The HMI is the same as the SMR one. • ATCO7: Problem to find the zoom function. • ATCO8: Once one has become used to this function, OK. • ATCO24: Not really tested. • ATCO28: Yes but the pitch is too high! • ATCO29: A priori efficient HMI that can surely be improved after operational

use.




66 It is easy to display several windows. 32 5 0.67 0.000 • ATCO24: Not really tested.

67 The pop-up windows appear at the expected place and size. 26 4.19 1.02 0.002

• ATCO5: To be improved. • ATCO7: Not experimented. • ATCO10: I do not know. • ATCO11: Not enough experience. • ATCO12: To be tested. • ATCO14: Text is too small in the alert windows. • ATCO15: The alert windows are too small. • ATCO18: I do not know. • ATCO19: The alert windows are a bit too big. • ATCO22: The alert texts should be bigger. • ATCO24: Not really tested. • ATCO29: Except the size of the alert windows. • ATCO32: Size is OK. Place is OK, especially because the preferred place may

be chosen the first time the windows appear. • ATCO33: Not tested so much. • ATCO34: The alert does not catch the attention enough.

68 The windows are conveniently arranged on the EMMA display. 28 4.46 0.96 0.000

• ATCO7: Not enough experience to judge. • ATCO9: The windows are neither at the right place nor at the right size. • ATCO10: There is too much overlap between the windows. Could be better. • ATCO11: I do not know. • ATCO14: Is not it customisable? • ATCO24: Not really tested. • ATCO29: Anyway the HMI is customisable. • ATCO34: I believe that the air window cannot replace the IRMA display at the

ground position.




69 Pop-up windows sometimes obscure aircraft that should be visible. 20 4.2 1.44 0.119

• ATCO8: I do not know. • ATCO12: Not seen. • ATCO18: Not experienced. • ATCO20: Not seen. • ATCO23: See user rules. • ATCO24: Not really tested. • ATCO28: The aircraft labels are too big. Smaller characters would be better. • ATCO29: An area should be dedicated to alerts / information. • ATCO30: Not tested.

70 The manual modification of the content of mobile reports on the EMMA display can be performed quickly and efficiently.

30 2.93 1.55 0.087

• ATCO1: This function is useless because the menu follows the moving mobile. • ATCO7: Not enough experience to judge. • ATCO9: I did not manage to modify the content. • ATCO12: Not experienced. • ATCO19: It would be easier to click on the labels instead of the position

reports. • ATCO20: I do not know. • ATCO24: Not really tested. • ATCO29: You have to click on the position report while you have to click on

the label in IRMA 2000. The buffer zone (where you remain in the menu even if the cursor moves) is not wild enough.

• ATCO35: It at least has the merit to exist but it is a bit heavy.

71 The EMMA display layout is easy to customise to my own preferences. 30 4.5 0.82 0.000

• ATCO10: Not tested. • ATCO19: A reset function of the controller preferences is missing. • ATCO24: Not really tested. • ATCO25: Not needed. • ATCO29: It is a bit long to customise the HMI. Preset customised

configurations should be implemented.




72 Too much interaction with the EMMA display is needed. 28 2.71 0.90 0.000

• ATCO8: I do not know (in principle no but to be checked). • ATCO9: If EMMA has been initially set up correctly, no. • ATCO10: A lot of interaction is need. • ATCO11: Not enough time. • ATCO14: There is no other solution (as in IRMA). • ATCO21: In principle no but lack of practical experience. • ATCO29: In principle no. The HMI is a bit long to customise but OK once it

has been adjusted (except if identification problems occur).

Table 26: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Acceptance of Technical Performance – HMI – Efficiency

6.1.1.4 HMI – usefulness


73 It is useful to represent the closed taxiways, runways and apron areas on the EMMA display.

32 4.94 1.24 0.000 • ATCO30: Problem: it has to be kept up to date.

74 The actions that can be performed on the windows (display / hide, manipulation, scaling) are relevant.

32 4.69 0.69 0.000 • ATCO7: Little practical experience. • ATCO9: There should be less choice left (preset windows). • ATCO11: Not enough time.

75 The manual anti-overlapping of labels (possibility to change the label position manually) is useful.

32 4.53 1.14 0.000

• ATCO1: Because the automatic anti-overlapping does not work. • ATCO7: In principle. • ATCO14: Could it be automatic? • ATCO20: We may not have time to use the manual anti-overlapping. • ATCO28: I would have preferred an automatic change… • ATCO29: It should be used as few as possible. • ATCO35: It is a positive thing.

76 The use of colour characterisation for the different types of mobiles is useful. 32 5.06 0.76 0.000 • ATCO30: Not tested.




77 The raw video display (provided by the primary surface radar) is useful. 30 5.1 0.88 0.000

• ATCO1: Support of the information. • ATCO3: In order to know that a parking stand is occupied. • ATCO5: In case of failure of the secondary radar yes, but not useful otherwise. • ATCO9: Ideally no (if everybody is identified). • ATCO20: Good backup in case of failure but not useful in normal operations. • ATCO22: Until a better solution is implemented. • ATCO24: Too rough.

78 The function allowing the manual modification of label content is useful. 30 4.7 1.06 0.000

• ATCO1: But the function is not implemented correctly. • ATCO7: Not experienced. • ATCO8: The modification should be automatic. • ATCO14: May be useful but not handy at all. • ATCO15: Useful but not easy to use. • ATCO20: Yes if exists. • ATCO26: Not easy to use. • ATCO27: Answer between 3 and 4. • ATCO28: Yes but beware of erroneous identifications. To be handled with

care. • ATCO30: Useful if everything is not identified. But it is heavy to use.




79 The EMMA display gives me information that I missed before. If yes, which ones? 27 5.04 1.06 0.000

• ATCO1: Flight plan information and parking stands. • ATCO3: Identification. • ATCO4: SID. Type. • ATCO5: A lot of information. • ATCO7: Callsign, position. • ATCO8: CANARI, flight plan information. • ATCO9: Labelled mobiles, dynamic airfield map (+ flight plan when the

information is available). • ATCO10: Not seen. • ATCO12: Strip information. • ATCO13: Identification. • ATCO15: Conflict alert. • ATCO16: Hidden parking stands. • ATCO18: Information in the labels. • ATCO19: No but EMMA links information coming from different sources. • ATCO20: Better identification of the vehicles. • ATCO21: Flight plan correlation. • ATCO22: Alarms, helicopter area, parking stands F. • ATCO23: The point is to gather scattered information together and present it

in an ergonomic way. • ATCO24: Critical / sensitive areas, etc. • ATCO26: A lot! • ATCO27: Before, we did not have anything!!! • ATCO28: Obvious answer in LVP conditions!!! • ATCO29: Information in labels and alerts.

Equipped vehicles (but it is already available on the ground radar). • ATCO30: Note tested. • ATCO32: Information is already available somewhere else (strip). • ATCO33: Alerts (e.g.: runway incursion, stop bar crossing). • ATCO34: Closed taxiways and runways.

Alerts. • ATCO35: Already an additional safety net.

Accuracy of the information at t time, without request.




80 The EMMA display contains too much information that I regard as not useful. 31 2.29 0.90 0.000

• ATCO9: Except perhaps the EMMATT label. • ATCO12: To be confirmed. • ATCO14: One can choose to keep only what one regards as to be useful. • ATCO23: Selective choice + profile reset function. • ATCO28: The vehicle information in the apron is not useful because it is not

usable (image polluted by tugs, etc.). • ATCO30: Not tested. • ATCO34: Fix runways.

Mobiles around the aircraft (refuellers, etc.). • ATCO35: Except some two or three fix reference reports.

81 The EMMA display provides the right information at the right time. 27 4.58 0.80 0.000

• ATCO1: Problems of latency before identification (up to 2 min. 37 sec. this day) and loss of identification.

• ATCO7: To be checked. • ATCO8: In the end, yes. • ATCO11: I do not know. • ATCO12: Yes, when it works… • ATCO30: Not tested. • ATCO34: E.g.: the alert is triggered too late. • ATCO35: I hope so.

According to what I have observed: yes.

Table 27: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Acceptance of Technical Performance – HMI – Usefulness



6.1.1.5 HMI – usability


82 The EMMA display gives me sufficient information about airborne traffic in the vicinity of the airport.

29 4.24 1.24 0.002

• ATCO7: E.g.: no information about a tail wind VFR. • ATCO14: Except for short finals runway and FATO. • ATCO16: Not useful. • ATCO17: Useless. • ATCO18: It is not the purpose of EMMA. • ATCO19: To be used on the ground. • ATCO22: But not the appropriate way (e.g.: colours are missing in the

approach window). • ATCO29: There is even too much information (e.g.: information about transit

flights). • ATCO30: The arrivals are visible. • ATCO33: The approach is not that useful. • ATCO34: No colours.

A background is missing on the map.




83 Some important functions are missing on the EMMA display. If yes, which ones? 23 2.48 0.95 0.000

• ATCO1: The type of information is OK but MOSQUITO is missing. • ATCO7: To be tested. • ATCO8: It would be OK if each flight was identified and correlated to its flight

plan. • ATCO9: No for the time being. • ATCO11: I do not know. • ATCO12: To be checked in a few months. • ATCO14: The aircraft type. • ATCO19: Information about the closed taxiways.

Runway in use. • ATCO22: In principle no. • ATCO23: ? • ATCO26: TOTAL identification of all the mobiles.

The automatic tracking of a selected aircraft in a dedicated window. • ATCO28: Being able to delete the third line of the departing aircraft

(clearance) while keeping the aircraft type. • ATCO29: A link to dynamic maps (dynamic information on closed runways

and taxiways). A direct display of the ground lighting, or even a remote control of the ground lighting from EMMA. The goniometric information. The identification feedback (sending of identification by the pilot in reply to a controller request).

• ATCO33: Not enough time.

84 The colours used in the EMMA display are appropriate. 30 4.8 0.76 0.000

• ATCO11: I do not know. • ATCO34: No for the information (yellow).

Perhaps too many colours. Why not making the alarms flash?

85 The contrast between the windows and their background is sufficient. 31 4.90 0.54 0.000




86 The height and width of the characters on the EMMA display are sufficient. 30 4.63 0.89 0.000

• ATCO12: Smaller police would be as convenient. • ATCO14: Answer between 3 and 4.

Labels OK. Windows NO. • ATCO22: Texts should be bigger (alarm texts mainly). • ATCO23: Except for the alert windows. • ATCO26: Except for the alert windows. • ATCO27: Almost too big. • ATCO28: Divide at least by two the height of the characters in the aircraft

labels. • ATCO29: A bit big (labels).

87 All the texts in the EMMA display are easy to read. 32 4.19 1.15 0.001

• ATCO1: The alerts are too small. • ATCO4: Alert window. • ATCO7: The names of the taxiways are small in large-scale views but OK if

zoom. • ATCO11: The names of the taxiways are not easy to read. • ATCO13: Yes except for the alarms. • ATCO14: Not the alert windows. Not the pull-down menus. • ATCO21: The names of the taxiways are too small. • ATCO22: The texts should be bigger (especially the alarm ones). • ATCO23: Except for the alert windows. • ATCO26: Except for the alert windows. • ATCO34: Alert window. • ATCO35: Alert windows to make bigger.

Perhaps make text in the windows thicker.

88 Labels, signs and symbols in the EMMA display are easy to interpret. 32 4.75 0.76 0.000 • ATCO18: Except for the ‘Ge SF’ button.

89 Information is conveniently arranged in the EMMA display. 31 4.81 0.65 0.000

• ATCO10: Except for the alert window. • ATCO11: A bit messy. • ATCO19: Except for the runway in use for example. True in most cases. • ATCO34: Except label and window overlapping.




90 The EMMA display enables to detect a loss of accuracy of the surveillance. 3 2 0 N/A

• ATCO5: What is this gibberish (what is the meaning)? • ATCO7: No information provided on the loss of accuracy. • ATCO8: Unclear. • ATCO9: I do not know. • ATCO10: I do not know. I do not understand the question. • ATCO11: I do not know. • ATCO12: I do not understand the statement.

91 Symbols can easily be read under different angles of view in the EMMA display. 29 4.76 0.64 0.000 • ATCO11: It is better to stand in front of the screen.

• ATCO13: Not applicable.

92 The EMMA display size is appropriate for daily work. 31 4.65 0.88 0.000

• ATCO1: A bigger screen is needed. • ATCO5: A slightly bigger screen would be great. • ATCO7: I would like a slightly bigger screen. • ATCO8: Bigger, if possible (but that feats in the working positions). • ATCO10: A bigger screen would be good but it has to feat in the working

position. • ATCO21: It depends on where it is. This size means that it has to be close to

the controller. • ATCO22: A bit too big to be integrated easily.

Table 28: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Acceptance of Technical Performance – HMI – Usability



6.1.2 Operational feasibility: procedures

6.1.2.1 Start-up clearance


01

The EMMA display allows me to set up a more efficient start up sequence when visual reference is possible on the manoeuvring area and the apron.

35 3.23 1.50 0.291

• ATCO1: EMMA is not of any use for the start up. • ATCO7: Yes if the apron is visible. • ATCO8: EMMA has no influence on the start up sequence. But it is useful for

push back approvals. • ATCO17: Excellent plan. • ATCO19: EMMA is a confirmation. • ATCO23: General remark: I assume that the system has been validated and it

assumed to be in operations a t time for the whole questionnaire. After the interview it is clear that for the moment EMMA is not reliable enough to be operational.

• ATCO24: General remark: as EMMA has been installed on a remote position, it has been tested just a little bit. In the end, excellent tool.

• ATCO29: There is no added value compared with the ground radar. It is easier to set up a start up sequence with EMMA but it is not more efficient.

• ATCO31: EMMA does not enable that. The map in EMMA enables that.

02

The EMMA display allows me to set up a more efficient start up sequence when visual reference is not possible on the manoeuvring area and the apron.

35 3.57 1.74 0.438

• ATCO1: There is no need to have a view of the airfield for the start up. • ATCO8: EMMA has no influence on the start up sequence. But it is useful for

push back approvals. • ATCO19: EMMA is a confirmation. • ATCO20: There is no need for the view. • ATCO22: EMMA is not useful to set up a start up clearance. • ATCO29: There is no added value compared with the ground radar. It is easier

to set up a start up sequence with EMMA but it is not more efficient. • ATCO31: There is no correlation at the parking stands but there is a map of

the parking stands. It prevents from having scattered tools. • ATCO35: In LVP if the sequence is long, if need be…

Table 29: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Start Up Clearance



6.1.2.2 Push back clearance


03 The traffic information on the EMMA display helps me to decide whether a push back clearance should be delayed.

35 5.03 1.15 0.000

• ATCO1: Yes if the mobiles are identified on the taxiways. • ATCO7: Yes if the apron is visible. • ATCO12: In theory we do not have to exercise control on the apron. • ATCO18: In particular on parking stands F and E10/E20. • ATCO19: EMMA helps. • ATCO21: In principle yes, if the system is reliable. • ATCO31: EMMA helps but it must not be use systematically.

OK for the arrivals but not for the departures because they are not correlated. • ATCO35: Only on hidden parts of the manoeuvring area.

04 The EMMA display is sufficient to determine that an aircraft is on the stand or has left the stand.

33 4.30 1.26 0.000

• ATCO1: Yes to determine that an aircraft has left the parking stand, no to determine that it is at the parking stand.

• ATCO2: It depends on the accuracy. Is if it is accurate. • ATCO7: I do not trust it enough for the time being, to be discussed (if trust,

yes). • ATCO8: Yes if it is identified. • ATCO18: In particular on parking stands F, E10/E20, G and in the CEV area. • ATCO19: EMMA is not sufficient. • ATCO20: Yes to know that it has left the parking stand. • ATCO21: The external view is needed. • ATCO29: That depends on the accuracy of the system. • ATCO31: EMMA helps but it is not reliable enough.

DISCUSS and EMMA are complementary. • ATCO32: EMMA is not sufficient; there is a need for the external view or the

contact with the pilot. • ATCO35: It is a confirmation.

Table 30: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Push Back Clearance



6.1.2.3 Taxi clearance


05

When I want to give a taxi clearance, I can rely on the information provided by the EMMA display, even when visual reference is not possible on the manoeuvring area.

31 4.68 1.11 0.000

• ATCO1: Yes if there is a (continuous) identification. • ATCO2: Depends on the accuracy. • ATCO7: Yes if collective trust. Yes if put in operations. • ATCO8: In the end, it would be good (especially on the Fox area, which is not

visible from the tower). • ATCO11: If EMMA is reliable. • ATCO12: Let’s be careful. • ATCO14: If the system is validated. • ATCO15: If the system is validated. • ATCO18: Yes if there are no false reports and all the aircraft have a label. • ATCO19: On moving aircraft. • ATCO21: If the system is reliable. • ATCO22: Depends on the reliability of the system. • ATCO23: If operational. • ATCO28: To be checked. • ATCO29: Depends on the accuracy of the system.

In principle yes, if it works. • ATCO30: We hope so… • ATCO31: I do not know if I can trust the tool, so today no.

It is not the most convenient tool to manage the apron (especially in non visible areas like in F, E10, E20, E21…).

• ATCO32: And by asking for a confirmation to the pilot or by informing him that we do not know all the traffic.

• ATCO34: Yes if it is reliable (e.g.: label swapping…). • ATCO35: Yes if validated!




06

Surveillance of longitudinal spacing between aircraft on taxiways is easier with the EMMA display when visual reference is not possible on the manoeuvring area.

35 4.6 1.46 0.000

• ATCO1: No because the aircraft size and position of the synthetic plot against the aircraft are unknown.

• ATCO5: Is there a definition of longitudinal spacing on taxiways in the official regulation? I do not think so…

• ATCO8: No without any identification or correlation with the flight plan. • ATCO12: It depends on the standards. • ATCO19: Easier. • ATCO20: The actual position of the aircraft is not guaranteed. • ATCO22: If the system is reliable. • ATCO29: It is done more through the use of the raw video than EMMA.

EMMA does not bring anything more. • ATCO31: Yes, even without correlation. • ATCO32: EMMA is not enough, visual confirmation of the pilot is needed.

To be confirmed following practical experience and once trust in EMMA has been reached.

• ATCO34: Yes if reliable. • ATCO35: It depends on the spacing margin e.g. between two taxiways.




07

When visual reference is not possible on the manoeuvring area, longitudinal spacing between aircraft on taxiways could be reduced with the EMMA display.

32 3.72 1.61 0.206

• ATCO1: No because the aircraft size and position of the synthetic plot against the aircraft are unknown.

• ATCO5: This is not the purpose of the ground radar! I will not ask a pilot to follow another aircraft close behind just because I have a ground radar.

• ATCO6: Answer between 3 and 4. • ATCO8: If the aircraft are identified. • ATCO11: OK if EMMA is reliable. • ATCO20: The actual position of the aircraft is not guaranteed. The size of the

aircraft is not indicated in EMMA. To be confirmed.

• ATCO21: If the system is reliable. • ATCO22: If the system is reliable. • ATCO25: ? No spacing standard. • ATCO28: Not reliable when taxiing near the radar. • ATCO29: Standard to be defined. • ATCO30: It depends on the official regulation. • ATCO31: There is no problem with longitudinal spacing in LVP because most

of the time the pilots can see each other. Anyway, there is no need to make them queue on taxiways since they will have to wait at the holding point.

• ATCO32: EMMA is not enough, visual confirmation of the pilot is needed. To be confirmed following practical experience and once trust in EMMA has been reached.

• ATCO33: We do not perform any separation during the taxi phase. • ATCO34: Precautions taken in LVP must not be reduced by this tool.




08 With the EMMA display, it is no longer necessary to make records of the vehicles on the manoeuvring area.

35 2.8 1.45 0.013

• ATCO1: Yes if all the vehicles are equipped with MOSQUITO. • ATCO4: If all the vehicles are equipped. • ATCO5: Good question! I do not have any opinion yet… • ATCO8: OK if there is only one vehicle but no if there are several ones. • ATCO10: In case of failure of EMMA, there may be a problem. • ATCO11: It is safer to keep the list. • ATCO18: It depends on each controller. Some use a strip to list all the vehicles

present. EMMA is not sufficient. • ATCO19: Problem of trust in the HMI: need for a paper backup in case of

failure or loss of correlation or detection of the vehicles. • ATCO21: In the end, yes. • ATCO22: If not all the vehicles are equipped, a list is needed. • ATCO27: Without any technical failure! • ATCO29: Lack of reliability of the system and trust in the system.

Problem in case of failure / missing update / additional information on the paper strip not included in the system.

• ATCO30: Depends on the reliability. • ATCO31: If the system is reliable and if there is a correlation with the flight

plan, OK But in case of failure an emergency solution is needed (e.g.: include the vehicles in DISCUSS).

• ATCO32: If the system is reliable for the identification. • ATCO34: Precautions taken in LVP must not be reduced by this tool. • ATCO35: It is the same as for DISCUSS.

Table 31: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Taxi Clearance



6.1.2.4 Co-ordination


09 Coordination between the control working positions in the tower would be more efficient with the EMMA display.

35 4.97 0.95 0.000

• ATCO1: The ground and tower working positions should be equipped. • ATCO5: Including for the evaluation of the taxi phase and departure

sequencing. • ATCO8: No observed cases for the moment. It would be useful in LVP mainly. • ATCO19: Confirmation of the sequence of taxiing aircraft, if not activated in

the right sequence. E.g.: an aircraft leaving from B1 will be number one at the holding point even if the taxi clearance has been delivered to an aircraft in B2 first.

• ATCO21: If the tower controller has an EMMA display too. • ATCO29: It must be kept in mind not to simplify too much the communications

for the coordination, but OK. • ATCO31: If the traffic is correctly seen, there is no need for any coordination.

EMMA would enable more tacit coordination. • ATCO33: E.g.: for the vehicles.

10 With the EMMA display, hand over of a taxiing aircraft between the different control working positions would be more efficient.

35 4.89 1.05 0.000

• ATCO1: Replace ‘flight’ by ‘taxiing aircraft’ in the statement. • ATCO5: In order to know the situation of the departures, an A-SMGCS display

should be implemented at the approach working position. • ATCO22: EMMA helps to be more confident about the position.

Table 32: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Coordination



6.1.2.5 Taxiing on the runway


11

The number of position reports can be reduced when using the EMMA display (e.g.: to make sure that an aircraft has vacated the runway in use).

34 4.62 1.52 0.000

• ATCO1: Yes with an accuracy tolerance margin on the runway (not necessary at the parking stand).

• ATCO8: Especially in LVP. • ATCO11: The confirmation of the pilot is needed. • ATCO18: If there is no bug and if I can trust the system. • ATCO20: The exact position of the aircraft is not known precisely. It depends

on the position of the aircraft on the airfield. • ATCO22: Not for the given example, but OK for other ones. • ATCO28: No in the particular case of an aircraft vacating the runway in use. • ATCO29: OK to reduce the number of reports on the taxiways but not on the

runways in use. • ATCO30: It helps but it does not replace pilot reports. • ATCO31: The example is not appropriate because the pilot has to report in

that particular case. • ATCO32: OK for the reports on taxiways, but the pilot confirmation is needed

in the case of the runway. • ATCO33: The example is not appropriate because the controller has to contact

the pilot to instruct him to change of frequency. • ATCO34: Are we sure that the runway has been actually vacated?




12

With the EMMA display, an aircraft may be permitted to taxi on the runway in use even when visual reference on the manoeuvring area is not possible.

31 4.68 1.22 0.000

• ATCO2: Depends on the procedures. • ATCO5: To be considered… • ATCO7: Ambiguous question: EMMA or not, there is no difference. • ATCO8: Unclear. • ATCO19: Not relevant in Blagnac if all the taxiways are opened. Useful if M11

is not accessible, EMMA could allow the aircraft to taxi on the runway to the threshold from M10.

• ATCO20: This is already in use but EMMA brings more comfort. • ATCO22: Yes if the system is reliable. • ATCO27: The runway is under the responsibility of the tower controller. • ATCO29: The ground radar may help, it is easier with EMMA but the tool is

not necessary. • ATCO31: It is not relevant in Blagnac because the whole runway is used in

LVP. But EMMA would help to increase the capacity in LVP by instructing the aircraft to vacate the runway earlier and by using runway 14L.

• ATCO32: E.g.: taxi on runway 32R from taxiway N2 to the threshold while an aircraft is at the holding point in N1.

• ATCO34: In LVP, no taxi on the runway in use is a vital precaution.

Table 33: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Taxiing on the Runway

6.1.2.6 Line-up clearance


13

Even when visual reference on the manoeuvring area is possible, line-up of an aircraft at the runway threshold is easier to control with the EMMA display.

34 4.68 1.49 0.000

• ATCO1: Yes because the parallax is disturbing (e.g.: holding point at runway 32R is hardly visible from the tower).

• ATCO5: It is better to look outside in good visibility conditions!!! • ATCO8: Yes because, for example, the runway 32R threshold is hard to see at

nightfall. • ATCO9: Yes, in 32. • ATCO11: E.g.: at night.




14 When the runway threshold is not visible line-up of an aircraft is easier to control with the EMMA display.

34 5.41 0.82 0.000

• ATCO5: In order to know where it is in the takeoff phase. • ATCO8: Yes because, for example, the runway 32R threshold is hard to see at

nightfall. • ATCO9: Yes, on every runway. • ATCO34: Does not replace the confirmation of the pilot.

15 Even when visual reference is not possible, multiple line-ups could be performed with the EMMA display.

32 3.59 1.72 0.379

• ATCO1: Yes if the distance between the taxiway is long enough (e.g.: 500m on 14R: OK; 150m on 32L: NOK).

• ATCO2: Depends on the accuracy. • ATCO5: I believe that such a use of the radar should not be promoted. • ATCO8: I do not know. No specific case in mind. EMMA may help but it is

tricky. • ATCO10: It is risky. To be considered. • ATCO14: I do not usually do multiple line-up in LVP. • ATCO15: I do not do multiple line-up in LVP. • ATCO18: Technically OK but I would not do it for safety reasons. It would

necessitate an appropriate ground lighting (i.e. not all the runway entry points are marked out today). It also raises procedure issues.

• ATCO19: Considering today’s operational rules. • ATCO20: The tool is not accurate enough (accuracy of the position). • ATCO22: It is not possible from the safety point of view. • ATCO23: Double-check with phraseology. • ATCO26: In accordance with official rules. • ATCO29: It depends on the reliability of the system: if the system is reliable,

OK. But today it is not feasible because there are only two stop bars. • ATCO31: I can do it but it is not very interesting. Better not to do when the

weather is not good. Not used often here. It depends on the working methods. If the pilots can see each other, OK.

• ATCO33: It is better to do multiple line-up if the weather conditions are good. • ATCO34: Surely not. • ATCO35: Reactionary again… Old reflex.

Table 34: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Line-Up Clearance



6.1.2.7 Takeoff clearance


16 The EMMA display is helpful for better monitoring an aircraft commencing the takeoff roll.

33 4.27 1.35 0.003

• ATCO1: Yes, at the beginning of the rolling for takeoff. • ATCO5: It is better to look outside in good visibility conditions!!! • ATCO11: EMMA is mainly a ground tool. • ATCO12: But not decisive. • ATCO18: No if the weather is good. Yes in LVP. • ATCO22: Because we look outside. • ATCO29: The ground radar is enough, EMMA does not bring anything more.

In case of unauthorised takeoff, the alert system helps a lot. • ATCO33: I would rather look outside.

17 I can give rolling takeoff clearances more frequently with the EMMA display. 34 3.47 1.40 0.898

• ATCO1: The exact distance of the closer approaching aircraft to the runway threshold has to be known.

• ATCO5: It is better to look outside in good visibility conditions!!! • ATCO8: EMMA is not decisive in that particular case. • ATCO10: Because it depends on the arrivals. • ATCO12: But not only EMMA. • ATCO18: EMMA may help (in addition to the camera and external view). • ATCO20: It is not the purpose of EMMA. • ATCO29: Rolling takeoff clearances can be delivered more easily (thanks to

the identification function) but not more frequently. The criteria for a rolling takeoff are unchanged with EMMA.

• ATCO31: Not relevant. • ATCO32: EMMA is not that helpful for the rolling takeoff clearance (in

principle when a controller gives a rolling takeoff clearance it implies that the pilot has to hurry because of the incoming traffic).

• ATCO33: There is no link with EMMA. • ATCO34: On runway 32 we can see that it is near the holding point (not

related to the view of the arrival). • ATCO35: We can do it without EMMA.




18 The EMMA display is sufficient to decide when a takeoff clearance should be issued. 35 2.8 1.32 0.008

• ATCO2: Depends on the situation. • ATCO5: It is better to look outside in good visibility conditions!!! • ATCO8: In the case separation with approaching traffic has to be performed,

additional information is needed. • ATCO14: About ground constraints only. • ATCO18: In good weather conditions EMMA is not used. Yes in LVP. • ATCO20: Not sure (yes and no). We have to trust the pilot. • ATCO29: EMMA is not decisive, does not bring anything in comparison to the

ground radar. • ATCO30: EMMA is not enough. • ATCO31: EMMA helps but it is not enough (one of the parameters considered

to give the clearance). • ATCO32: The takeoff clearance also depends on the final approaches on the

other runway. Therefore, EMMA cannot be the only tool used. • ATCO35: We can do it without EMMA.

Table 35: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Takeoff Clearance



6.1.2.8 Landing clearance


19

When visual reference is not possible on the manoeuvring area, The EMMA display is sufficient to decide if the runway in use has been vacated in order to deliver a landing clearance.

34 3.74 1.76 0.413

• ATCO1: Depends on the accuracy tolerance margin. • ATCO2: Depends on the accuracy. • ATCO5: Yes if the ground radar is validated as reliable. • ATCO8: In LVP the pilot confirmation is needed. • ATCO17: Plus pilot confirmation. • ATCO18: Yes if the system is reliable. • ATCO20: The accuracy of the system is not known (position). • ATCO21: Not for the moment because the system reliability has not been

proved. • ATCO22: Need for pilot report. • ATCO29: It depends on the trust in the system. Today EMMA is not sufficient. • ATCO30: EMMA is not sufficient. • ATCO32: Pilot acknowledgement is needed. • ATCO35: Yes if validated.

Table 36: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Landing Clearance

6.1.2.9 Conditional clearances


20

When visual reference is possible on the manoeuvring area, the EMMA display may help me to deliver conditional clearances on the runway in use in a safe and efficient way.

34 4.41 1.08 0.000

• ATCO1: Nothing guarantees that what the pilot sees is the same as what we see. EMMA is not needed to give conditional clearances.

• ATCO5: I think that a systematic or too frequent use of the ground radar can be dangerous.

• ATCO12: It is a support. • ATCO18: EMMA is not used for conditional clearances (=> answer between 3

and 4). • ATCO22: EMMA helps only. • ATCO29: It helps. • ATCO: Not applicable.




21

When visual reference is possible on the manoeuvring area, the EMMA display may help me to deliver conditional clearances on taxiways in a safe and efficient way

34 4.53 1.08 0.000

• ATCO1: No because I do not look at the HMI if I can see outside. EMMA is not needed to give conditional clearances.

• ATCO5: I consider it as a help when the manoeuvring area is visible but not as the only local control means.

• ATCO12: It is a support. • ATCO18: EMMA is not used for conditional clearances (=> answer between 3

and 4). • ATCO22: EMMA only helps. • ATCO29: The identification function helps a lot in that case. • ATCO35: In some specific cases, e.g. T10 and P yes.

22

When visual reference is not possible on the manoeuvring area, the EMMA display may help me to deliver conditional clearances on the runway in use in a safe and efficient way.

35 3.86 1.44 0.088

• ATCO5: Is it to be desired from the safety point of view? I do not think so. • ATCO8: If the controller cannot see the aircraft, then no conditional

clearance. • ATCO10: To be assessed. • ATCO11: Not enough. • ATCO18: A pilot request is needed anyway. • ATCO20: This is not possible in LVP today (only one entry point and one exit

point on the runway). • ATCO22: Yes, but we need the pilot. • ATCO28: Lacks of reliability when the aircraft is located near the antenna. • ATCO29: The identification function helps a lot in that case. • ATCO34: Because uncertainty on the aircraft position.

23

When visual reference is not possible on the manoeuvring area, the EMMA display may help me to deliver conditional clearances on taxiways in a safe and efficient way

35 4.31 0.96 0.000

• ATCO1: Not sure because not tested. • ATCO5: Yes but only at the intersections of the taxiways. • ATCO8: In the end, yes. • ATCO18: A pilot request is needed anyway. • ATCO20: It depends on the accuracy of the system. • ATCO22: Yes, but we need the pilot. • ATCO29: The identification function helps a lot in that case. • ATCO34: If reliable.

Table 37: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Conditional Clearances



6.1.2.10 Visibility transition


24 The transition from normal operations to low visibility operations is easier with the EMMA display.

35 5.29 0.75 0.000 • ATCO1: Not sure because not tested. • ATCO29: The identification function helps a lot in that case. • ATCO35: It is obviously more comfortable.

25 When low visibility procedures (LVP) are activated, the EMMA display helps me to reduce my workload.

35 5.29 0.89 0.000

• ATCO1: If everything is identified, then I strongly agree. • ATCO5: The level of stress is also reduced because the respective positions of

the aircraft are known. • ATCO22: If the system is reliable. • ATCO29: The identification function helps a lot in that case. • ATCO35: And my level of stress!




26

With the EMMA display, it would make sense to redefine the visibility limits for the transition to low visibility procedures (if yes, please indicate your suggestions).

30 4.03 1.73 0.101

• ATCO1: Today the limit is 1000m. I hope to reduce it to 800m. with the ground radar.

• ATCO2: Depends on the accuracy of EMMA: if it is validated at the technical level, yes.

• ATCO5: They have changed at the CHEA but not yet at LFBO. Why? • ATCO8: There is no direct relation between the use of EMMA and visibility

limits. Moreover, modifying visibility limits is a heavy process. • ATCO9: Today the LVP limit is given by the meteo and not by a tool in the

tower. The use of runway 14L could be authorised in LVP. • ATCO10: Yes in the particular case of a low ceiling and good horizontal

visibility conditions. • ATCO11: I do not know. • ATCO17: I think that these limits are not defined locally and there is no

possible justification for the 1000m. limit this day. • ATCO18: The visibility limit, 1000m. today, could be reduced to 600m. with

EMMA. • ATCO19: Both runways could be used in LVP with EMMA. • ATCO21: Reduction of spacing only. • ATCO22: Capacities could be increased instead of visibility limits. • ATCO23: E.g.: lowest visibility accepted by pilots in CAT I. • ATCO26: 550m. • ATCO29:It has got nothing to do with it! OK to redefine airport capacities

(use of both runways in LVP). • ATCO30: It could be changed to 800m. • ATCO32: today’s limit at 1000m. is a bit too high (in particular to forbid all

the non authorised vehicles to circulate). In principle, it could be lowered to 800 to 600 m.

• ATCO34: Slightly agree if EMMA is reliable to replace the external view. • ATCO35: It would be very interesting but I do not have all the clues to give a

value (see what happens in other places).

Table 38: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Visibility Transition



6.1.2.11 Transponder procedures


27

It would be appropriate that pilots switch the transponder on before they request a push back clearance (condition to automatically associate a label to a departing aircraft).

35 5.6 0.81 0.000 • ATCO29: At start up.

Table 39: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Transponder Procedures

6.1.2.12 Contingency procedures


28

If only one mobile fails to comply with the technical requirements for A-SMGCS operations (e.g.: on-board Mode S transponder failure), I have to return to SMGCS procedures (e.g.: position report by the pilot) for all the traffic.

33 2.52 1.25 0.000

• ATCO9: Yes if the mobile declares it. • ATCO14: We do neither know the methods to be used nor their limits. • ATCO15: The decision lies on the technical side. • ATCO18: The mobile would be identified with a label including digits anyway. • ATCO19: Manual identification of the mobile. • ATCO21: It depends on the traffic. • ATCO22: Only the unidentified mobile needs to report. • ATCO28: Yes in LVP, no in normal operations. • ATCO29: Not for all the traffic. • ATCO32: In this particular case, we request position reports to the

unidentified mobile. • ATCO35: With only one vehicle, we can cope with it, by characterizing it.

Table 40: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Contingency Procedures



6.1.2.13 Conclusion on procedures


29 The EMMA display helps me when visual reference is not possible on the manoeuvring area.

35 5.6 0.60 0.000

30 I can rely on the EMMA display instead of the outside view when visual reference is not possible on the manoeuvring area.

30 4.5 1.61 0.002

• ATCO1: There is an issue with the current technical limits of the system. Only a few things are missing in order to become a control tool.

• ATCO2: Depends on the accuracy of the system. • ATCO8: EMMA may help but it does not take over from the outside view. • ATCO11: Not as a substitution. • ATCO20: EMMA does not take over from the outside view. There is no

comparison. • ATCO21: If the system is reliable. • ATCO22: Yes if the system is reliable. • ATCO29: Yes if the system is reliable. • ATCO32: I would like to, if the system is certified as being reliable. It also

depends on the official texts. Visual or pilot confirmation is still needed.

• ATCO33: I would rather give the responsibility to the pilot. • ATCO34: Yes if reliable. • ATCO35: It depends on the validation.

31 The EMMA display helps me when visual reference on the manoeuvring area is possible. 29 5.34 0.77 0.000

• ATCO10: For example at night, on remote areas and for small aircraft. • ATCO20: At night mainly. • ATCO35: It MAY help.




32 I think that there are too many inconsistencies between the EMMA display and the actual traffic.

27 2.37 1.01 0.000

• ATCO8: If the transponders would work, yes. • ATCO9: No opinion (not enough traffic). • ATCO10: If EMMA works, OK. • ATCO11: Not enough experience. • ATCO13: Not experienced. • ATCO18: Today there are too many inconsistencies. • ATCO19: Some false reports. • ATCO20: Lack of time of practical experience to be able to answer. • ATCO21: Lack of practical experience. • ATCO22: The primary radar plots are disturbing. • ATCO25: ? • ATCO29: Not experienced enough. • ATCO30: Not tested. • ATCO34: For the time being! • ATCO35: I have never observed big inconsistencies (only 1 or 2 false reports).

33

The mix of cooperative and non-cooperative mobiles could severely restrict the use of A-SMGCS when visual reference is not possible on the manoeuvring area.

35 4.14 1.52 0.042

• ATCO9: Every mobile has to be equipped. • ATCO14: The false reports are more disturbing. • ATCO17: ! • ATCO20: Anyway, in LVP the traffic of vehicles is restricted. It is better if all

the vehicles are cooperative. • ATCO25: It depends on the amount of non-cooperative vehicles. • ATCO28: Depends on the procedures applied. • ATCO29: It depends on the amount of non-cooperative mobiles, but it does not

severely restrict the use of A-SMGCS.

Table 41: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Conclusion on Procedures




6.1.3.1 Capacity


54 The EMMA display may enable to increase traffic throughput at the airport. 34 4.82 1.11 0.000

• ATCO5: May enable a bit more taxi in LVP. • ATCO17: By using runway 14L for departures in LVP as in former times (a

200m. visibility is sufficient. • ATCO19: By using both runways in LVP. • ATCO32: Yes in LVP. No in normal operations. • ATCO35: In LVP yes. Or in normal operations for the hidden parts.

55 The EMMA display helps me to manage more traffic when visual reference is not possible on the manoeuvring area.

34 4.82 1.19 0.000

• ATCO17: By using runway 14L for departures in LVP as in former times (a 200m. visibility is sufficient.

• ATCO19: Improvement of the instructions in LVP: it will enable us to check that an arriving aircraft is at the parking stand, does not get lost.

Table 42: Results of the Shadow-Mode Trials Questionnaires – Operational Improvements – Capacity

6.1.3.2 Efficiency


56 The EMMA display allows me to provide the pilots with a better level of service. 34 5.09 0.83 0.000

• ATCO5: Above all a better level of safety in LVP. • ATCO12: Also for the parking stand. • ATCO22: For example during the push back phase. • ATCO30: Not obvious. • ATCO32: Not obvious (e.g.: if the frequency occupation rate is high or the

controller has to deal with another issue, the provided service will not be better).

Table 43: Results of the Shadow-Mode Trials Questionnaires – Operational Improvements – Efficiency



6.1.3.3 Human factors


57 The EMMA display allows me to have a better awareness of the respective position and identification of each mobile.

27 5.41 0.64 0.000 • ATCO34: 4 for the position and 3 for the identification.

58 The EMMA display allows me to have a better understanding of the traffic situation. 33 5.15 0.87 0.000 • ATCO5: ‘Understanding’ = improper word.

• ATCO14: On the ground.

59 The EMMA display allows me to better predict the evolution of traffic. 32 4.53 1.46 0.001

• ATCO1: Yes if EMMA is implemented on the approach working positions so then the controller could see the number of aircraft waiting at the holding point.

• ATCO2: Yes if there is an HMI available in the IFR room. • ATCO3: Yes if HMI in the IFR room. • ATCO4: If HMI in the IFR room. • ATCO6: Yes if the HMI is in the IFR room. • ATCO7: It is hard to answer (rephrase). Lack of time. • ATCO8: I do not know. • ATCO9: Yes in the short term (3 min.). • ATCO20: Real time image but not anticipation. • ATCO34: For areas that we cannot see from the tower. • ATCO35: It is an additional thing but not vital in normal operations.

60 The EMMA display is unnecessarily over-complicated. 31 2.26 0.86 0.000

• ATCO5: ? • ATCO9: I do not know. • ATCO18: Not tested enough. But some information is not immediately

accessible. • ATCO22: A bit complex. • ATCO29: The different HMIs should be harmonised. • ATCO32: May seem complex because it is new but it is OK after initial handle.

61 The EMMA display helps me to reduce my mental workload. 35 4.77 1.06 0.000

• ATCO7: Lack of practical experience. • ATCO12: If everything works. • ATCO30: Reassures. • ATCO35: It relieves it!

Table 44: Results of the Shadow-Mode Trials Questionnaires – Operational Improvements – Human Factors



6.1.3.4 Safety


62 The information in the EMMA display helps me to avoid conflicts. 35 4.91 0.82 0.000

• ATCO6: Taxi parking F – facing South and facing North. • ATCO30: Especially the names of the mobiles. • ATCO33: I do not use EMMA to avoid conflicts. • ATCO34: If reliable. • ATCO35: Any additional equipment is useful!

We could do our job without it, but it is an additional help.

63 The EMMA display endangers safety at the airport. 34 1.68 0.84 0.000

• ATCO13: Cf. reliability! • ATCO23: If it has been validated and put in operation at t time. • ATCO29: Yes as it is. No when EMMA is finalised! • ATCO34: Yes if it is reliable!

64 The EMMA display makes the detection of pilot errors easier. 35 5.09 1.09 0.000 • ATCO16: Yes in LVP

• ATCO34: E.g.: on areas St Martin when St Martin is closed.

Table 45: Results of the Shadow-Mode Trials Questionnaires – Operational Improvements – Safety

EMMA



6.2 Annex B: references

6.2.1 Applicable documents [1] European Airport Movement Management by A-SMGCS, EU FP6 contract

6.2.2 Referenced documents

6.2.2.1 Emma deliverables [2] D4.0.2: Sub-project 4 installation and test plan [3] Convention n° 02 034 Aéroport de Toulouse Blagnac Installation des matériels relatifs à

l’expérimentation EMMA ref. 455/JJD/OP/TE/4, annexed to D4.0.2. [4] D6.1.1: Generic verification and validation master plan [5] D6.1.3: Verification and validation test plan for Toulouse-Blagnac airport, version 1.02,

issued on the 15th February 2006 [6] D1.3.5: Operational requirements document [7] D6.2.2: Indicators and Metrics for A-SMGCS [8] D6.3.1: Test results Prague [9] D6.5.1: Milano-Malpensa A-SMGCS Verification and Validation report [10] D6.6.1: Airborne verification and validation results [11] D6.7.1: V&V analysis report [12] D6.8.1: V&V recommendation report [13] D1.3.9: Functional hazard assessment and very preliminary system safety assessment

6.2.2.2 Other documents [14] Quality Manual, Thales ATM, QM-03, Commercial-in-Confidence [15] ICAO manual on Advanced Surface Movement, Guidance And Control System (A-SMGCS),

doc 9830 AN 452 - 2004. [16] EUROCAE WG-41, MASPS for A-SMGCS, Edition ED-87A, Jan. 2001.

6.3 Annex C: abbreviations ADS-B Automatic Dependent Surveillance – Broadcast AENA ente público Aeropuertos Españoles y Navegación Aérea AIBT Actual In-Block Time AIF Airbus France ALDT Actual Landing Time ANS_CR Air Navigation Services Czech Republic AOBT Actual Off-Block Time ARR Arrival A-SMGCS Advanced Surface Movement Guidance and Control System ASTERIX All purpose STructured EUROCONTROL Radar Information eXchange ATC Air Traffic Control ATCO Air Traffic Controller ATM Air Traffic Management ATOT Actual Takeoff Time AUEB Athens University of Economics and Business AVOL Aerodrome Visibility Operational Level CFMU Central Flow Management Unit CWP Controller Working Position DACOTA Dispositif d'Association, de COrrélation et de Traitement radar pour les

Approches DEP Departure

EMMA



DFS German ATC Corporation (Deutsche Flugsicherung GmbH) D-GPS Differential Global Positioning System DG-TREN Directorate General Transport and Energy DLR Deutsches Zentrum für Luft- und Raumfahrt e.V. / German Aerospace Centre

(German Aerospace Research Institute) DSNA Direction des Services de la Navigation Aérienne DSNA/DTI Direction des Services de la Navigation Aérienne / Direction de la Technique et

de l’Innovation EC European Commission ELDT Estimated Landing Time EMMA European Airport Movement ENAV Ente Nazionale di Assistenza al Volo (Italian ANSP) EOBT Estimated Off-Block Time ERC EUROCONTROL ETG EuroTelematik AG ETOT Estimated Takeoff Time EUROCAE European Organisation for Civil Aviation Equipment manufacturers EUROCONTROL European Organisation for the Safety of Air Navigation FAA Federal Aviation Administration FHA Functional Hazard Assessment FOM/PA Figure of Merit / Position Accuracy GBAS Ground Based Augmentation System GS Ground Station GPS Global Positioning System GSC Ground Station Calibrator GTW Gateway ICAO International Civil Aviation Organisation LAN Local Area Network LFBO Toulouse-Blagnac LVO Low Visibility Operation LVP Low Visibility Procedure MAGS Mode S Airport Ground Sensor MASPS Minimum Aviation System Performance Standards MLAT Multilateration MXP Milano-Malpensa Nb Number NLR Nationaal Lucht- en Ruimtevaart Laboratorium (NL) NTP Network Time Protocol NVO Normal Visibility Operations POC Point of Contact R&D Research and Development RF Radio Frequency RPA Reported Position Accuracy RTCA Requirements and Technical Concepts for Aviation RWY Runway SCA Surface Conflict Alert SD Standard Deviation SDF Sensor Data Fusion SICTA Sistemi Innovativi per il Controllo del Traffico Aereo SMR Surface Movement Radar SRS Software Requirements Specification TATM Thales ATM THAV Thales Avionics TIS-B Traffic Information Service - Broadcast

EMMA



TT Test Transmitter TUD Technische Universitaet Darmstadt TWR Tower UT(C) Universal Time (Co-ordinated) V&V Verification and Validation WG Working Group WP Work Package

toulouse a-smgcs verification and validation results · emma toulouse a-smgcs verification and...

Documents