development of a collision avoidance ... - atm seminar › seminarcontent › seminar13 ›...
TRANSCRIPT
![Page 1: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/1.jpg)
Sybert Stroeve, Henk BlomCarlos Hernandez Medel, Carlos Garcia Daroca
Alvaro Arroyo Cebeira, Stanislaw Drozdowski
13th USA/Europe ATM R&D Seminar, Vienna, 17-21 June 2019
Development of a Collision Avoidance Validation and Evaluation Tool (CAVEAT)
Addressing the intrinsic uncertainty in TCAS II and ACAS X
![Page 2: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/2.jpg)
2
Contents
• Context
• Agent-based sociotechnical model
• CAVEAT software tool
• Illustrative simulation results
• Conclusions
![Page 3: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/3.jpg)
ContextUncertainty in encounter scenarios for ACAS design
3
![Page 4: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/4.jpg)
4
TCAS II & ACAS X
Airborne Collision Avoidance System (ACAS)– Provides advisories to pilots for avoiding potential collisions– Traffic advisories (TAs): support pilot situation awareness on nearby traffic– Resolution advisories (RAs): Level off, Climb, Descend, Do not climb, etc.
Traffic Collision Avoidance System II (TCAS II)– Current commercial implementation, version 7.1 required by ICAO Annex 10– Filtering of transponder data and a fixed set of rules for alert generation
Recent development of ACAS X– Modular architecture for various surveillance sources– Optimization of logic tables for alert generation– Variants: ACAS Xa / ACAS Xo / ACAS Xp / ACAS Xu
![Page 5: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/5.jpg)
5
Evaluation of ACAS designs
USEurope
ACAS advisories TCAS II v7.1 ACAS XaPilot response Standard 20% no response
P(NMAC) ACAS type Altitude Pilot responseP(RA types) Reversal Crossing Strengthening
1
encN
1
encN
![Page 6: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/6.jpg)
6
Elements in an encounter scenario
![Page 7: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/7.jpg)
7
Intrinsic uncertainty (each aircraft)
Pilot performance variability• Probability of response• Time of response• Strength of response
Sensor errors• Static errors (bias)• Variable errors (jitter)
![Page 8: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/8.jpg)
8
Resulting uncertainty in ACAS alerting
As a result of uncertainty in ACAS surveillance, ACAS alerting is a stochastic process• No single unique RA for a scenario• Probability density for RA timing • Probability distribution RA types
P(RA)
![Page 9: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/9.jpg)
9
Uncertainty in outcomes of an encounter scenario
![Page 10: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/10.jpg)
10
ACAS evaluation with Monte Carlo simulation
MCN
1 MCN
1
encN
1
encN
ACAS typeSensor error models Ownship state estimation Othership measurementPilot response models Stochastic
![Page 11: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/11.jpg)
Agent-based Sociotechnical Model
11
![Page 12: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/12.jpg)
12
Agent-based sociotechnical model
![Page 13: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/13.jpg)
13
Pressure altimetry system
0
0
0
,, , ,,
, , ,, , ,
( ;0, ) if { }
( ;0, ) if
N z bias biasPAS i PAS iz bias
t PAS i L z bias biasPAS i PAS i
f Normal
f Laplaceτ
ττ
σ κε
σ κ≥
⋅ =
⋅ =
Static errors (bias)• Normal or Laplace distribution• Altitude dependent variance
Dynamic errors (jitter)• First-order auto-regressive noise process
,
, , 0,,, ,
, , 0, ,
if if sample
PAS i
noiset PAS iz jitterauto z jitter noiset PAS iPAS t PAS it T PAS i
tt
ε τε α ε ε τ
−
== + >, 2
, , ,{ } (0, 1 ( ) )noise N z jitter autot PAS i PAS PAS ifε σ α−
, ,, , , , , , , ,z z z bias z jittert PAS i t AC i t PAS i t PAS is s ε ε= + +
Resulting pressure altitude
![Page 14: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/14.jpg)
14
Slant range measurement
ACAS MOPS• RMS jitter ≤ 50 ft (mode S or C)• RMS bias ≤ 125 ft (mode S) or 250 ft (mode C)
0 ,
, ,, , , ,
{ } ( ;0, )jTS i
r bias N r biasTS i TS m i
fτε σ⋅
, , ,, , , , ,, ,sample
TS
r jitter auto r r jitter noiset TS i TS i t TS it T TS iε α ε ε
−= +
,
, , 2, , ,, ,
{ } ( ;0, 1 ( ) )jTS i
noise N r jitter auto rt TS i TS iTS m i
fε σ α⋅ −
, ,, , , , , , , , , ,j r bias r jitter
t TS i t AC i t AC j t TS i t TS ir ε ε= − + +s s
Error modelStatic and variable noise processes with mode dependent variance
• Gaussian bias
• 1st order autoregressive jitter
• Resulting slant range
![Page 15: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/15.jpg)
15
ACAS algorithms in CAVEAT
TCAS II• Versions 7.0, 7.1, 7.2 stem from InCAS 3.3, based on MITRE
implementation (v7.0) and adjustments by Evosys (v7.1, v7.2)
ACAS Xa• Presented results: R15V2 in Julia with interface to C++ • Current implementation: R15V4 in C++ code from Honeywell
ACAS Xu• Expected in 2020/2021
![Page 16: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/16.jpg)
16
Pilot model: response probability
ICAO standard model• Response probability: 100%• Delay
5 s (initial RA) 2.5 s (modified RA/COC)
• Acceleration 0.25g (normal) 0.35g (reverse / increase)
CAVEAT response probability model• P(initial RA | context)
altitude rate reversal parallel approach
• P(modified RA | previous response)• P(COC)• 13 conditional probabilities
Response probability measurements• Londner and Moss (2017)• P(response) <<100%• P(response | context)
rate reversal / parallel approach
![Page 17: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/17.jpg)
17
Pilot model: delay
ICAO standard model• Response probability: 100%• Delay
5 s (initial RA) 2.5 s (modified RA/COC)
• Acceleration 0.25g (normal) 0.35g (reverse / increase)
CAVEAT delay model• Log-normal probability distribution• Advisory dependent parameters
initial / modified / COC
Delay measurements• Cloerec (2005)
![Page 18: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/18.jpg)
18
Pilot model: acceleration
ICAO standard model• Response probability: 100%• Delay
5 s (initial RA) 2.5 s (modified RA/COC)
• Acceleration 0.25g (normal) 0.35g (reverse / increase)
CAVEAT acceleration model• Log-normal probability distribution• Linear relation between mean
acceleration and vertical speed
Acceleration measurements• Cloerec (2005)• Relation between acceleration
and vertical speed
1 1 1
, , ,, , , , , ,
a RA z RA a zPF i PF i PF iv vτ τ τµ α β= + −
![Page 19: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/19.jpg)
CAVEAT Software Tool
19
![Page 20: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/20.jpg)
20
High-level design of CAVEAT software
![Page 21: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/21.jpg)
21
HMI allows setting of all agent parameters
Pilot response Ownship state estimation
![Page 22: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/22.jpg)
22
Horizontal / vertical views of a simulation run
![Page 23: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/23.jpg)
23
Statistics report (for MC simulation)
![Page 24: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/24.jpg)
Illustrative Simulation Results
24
![Page 25: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/25.jpg)
25
Encounter scenarios
Encounter Horizontal Vertical HMD (ft) VMD (ft)
E1 Crossing AC1 climbsAC2 is level 3038 200
E2 Crossing AC1 is levelAC2 is level 0 0
Scenario Sensor errors
Pilot Dynamic Variability
Pilot Response Probability
D none none (standard) 100%
S1 MOPS none (standard) 100%
S2 MOPS delay, rate, acc. 100%
S3 MOPS delay, rate, acc. 80%
Stochastic
Deterministic
![Page 26: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/26.jpg)
26
Encounter scenario E1-D with TCAS II v7.1Vertical viewHorizontal view
HMD: 3038 ft VMD: 200 ft (original) / 522 ft (modified)
![Page 27: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/27.jpg)
27
Encounter scenario E2-D with TCAS II v7.1Vertical viewHorizontal view
HMD: 0 ft VMD: 0 ft (original) / 975 ft (modified)
DE
CL
![Page 28: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/28.jpg)
28
Timing of advisories (TCAS II v7.1) Encounter scenario E1-S1 Encounter scenario E2-S1
![Page 29: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/29.jpg)
29
Number & types of advisories (TCAS II v7.1)
AC RADeterministic Stochastic (S1)
Sense P(RA) P(Sense | RA)
1
RA-1 Level off 100% Level off 97%Descend 3%
RA-2 - 6% Level off 53%Descend 47%
RA-3 - 3% Level off 100%
2
RA-1 Climb 100%Climb 94%Do not
descend 6%
RA-2 Level off 94% Level off 94%Climb 6%
RA-3 - 1% Level off 1%
Encounter E1 (climb x level)AC RA
Deterministic Stochastic (S1)
Sense P(RA) P(Sense | RA)
1RA-1 Descend 100% Climb 51%
Descend 49%
RA-2 Level off 100% Level off 100%
2RA-1 Climb 100%
Descend 49%
Climb 51%
RA-2 Level off 100% Level off 100%
Encounter E2 (same level)
![Page 30: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/30.jpg)
30
VMD empirical PDF of Encounter E1 (climb x level)
Sensor errors only (S1) Sensor errors & pilot var. (S3)522 723
Deterministic: TCAS II v7.1 ACAS Xa V15R2
TCAS II v7.1mean: 519 ftSD: 111 ft
ACAS Xa V15R2mean: 691 ftSD: 96 ft
522 723
TCAS II v7.1mean: 466 ftSD: 143 ft
ACAS Xa V15R2mean: 639 ftSD: 162 ft
![Page 31: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/31.jpg)
31
VMD empirical PDF of Encounter E2 (same level)
Sensor errors only (S1) Sensor errors & pilot var. (S3)
Deterministic: TCAS II v7.1 ACAS Xa V15R2
975 1025 975 1025
TCAS II v7.1mean: 845 ftSD: 85 ft
ACAS Xa V15R2mean: 906 ftSD: 78 ft
TCAS II v7.1mean: 916 ftSD: 230 ft
ACAS Xa V15R2mean: 916 ftSD: 230 ft
![Page 32: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/32.jpg)
Conclusions
32
![Page 33: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/33.jpg)
33
Conclusions
Evaluation of ACAS designs requires assessment of all relevant sources of variability and uncertainty
– Variability in encounter geometries / altitude layers– Variability in equipage types– Uncertainty due to pilot response– Uncertainty due to sensor errors
Development of an agent-based sociotechnical model for Monte Carlo simulation of encounter scenarios
– Various sensor error (bias & jitter) models– Pilot performance models– Supports deterministic (single run) simulations– Systematic evaluation of the impact of uncertainty in encounter scenarios– Extendible for new agents: ACAS Xu, UAV, ATC,...
![Page 34: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/34.jpg)
34
Conclusions
Illustrative simulation results– Variability in timing and types of RAs, and dispersion in VMD
• Considerable differences with deterministic simulation results – Mean MC simulation results can differ considerably from deterministic simulation results
Use for prospective analysis: What will be the impact of changes? – Impact on RA types / VMD / P(NMAC)– MC simulation of sets of encounter scenarios – Evaluation of ACAS designs / ACAS related systems / system compatibility– Evaluation of changes in airspace, ATM, and regulations
Use for retrospective analysis: What happened / could have happened? – MC simulation of a single encounter scenario– Likelihood of observed RAs (type / timing)– Likelihood of observed CPA
![Page 35: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/35.jpg)
35
Conclusions
CAVEAT software development for EUROCONTROL– Extensive HMI gives wide control over parameters of the agent-based model– TCAS II v7.0, v7.1, v7.2, ACAS Xa R15V4– Detailed views for incident investigation of single encounters– Powerful simulation core for analysis of large sets of encounters – Integration of ACAS Xu (when available)
Future research options– Required sizes of encounters sets and number of MC simulation runs– Sensitivity analysis for understanding the impact of the range of uncertainty sources– Detailed analysis of the impact of new ACAS designs (ACAS Xa, ACAS Xu)
![Page 36: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/36.jpg)
36
Acknowledgements
The R&D in this paper has been conducted with support from EUROCONTROL in the scope of SESAR 2020 Project 11
We thank MITRE and Honeywell for making available their TCAS II and ACAS Xa/Xo libraries, respectively
We thank Carmelo Javier Villanueva Cañizares, Daniel Rubio Garcia, Cristina Midori Fukuda Leon, Eduardo Pablos Ruiz, Ángela Merino Pérez, Volker Huck, David Phu, Garfield Dean, Busso Gellert, Frank Bussink and Bert Bakker for their support of the CAVEAT development
![Page 37: Development of a Collision Avoidance ... - ATM Seminar › seminarContent › seminar13 › presentations › … · Sybert Stroeve, Henk Blom Carlos Hernandez Medel, Carlos Garcia](https://reader033.vdocuments.mx/reader033/viewer/2022060406/5f0f53517e708231d4439a45/html5/thumbnails/37.jpg)
NLR AmsterdamAnthony Fokkerweg 21059 CM AmsterdamThe Netherlands
p ) +31 88 511 31 13 e ) [email protected] i ) www.nlr.org
NLR MarknesseVoorsterweg 318316 PR MarknesseThe Netherlands
p ) +31 88 511 44 44 e ) [email protected] i ) www.nlr.org
Fully engagedRoyal Netherlands Aerospace Centre