- eurocontrol | - driving excellence in atm ... agent systems –multi unmanned vehicles 9...

www.onera.fr

Applicability of Artificial Intelligence and Decision to Air Traffic and Unmanned Vehicles

Jean-Loup FargesInformation Processing and Systems

Department

Outline

3

IntroductionSystemsSolving problemsLearning strategiesHuman factorsConclusion


4

Introduction

• Artificial Intelligence: To describe precisely human intelligence in order to implement it has a computer program

• Decision theory: To study the reasoning underlying choices

• Purpose of the talk: To present some instances of artificial intelligence and decision that are or can be applied to:

• Air traffic

• Unmanned vehicles


5

Systems:• Multi-agents systems• Discrete event systems


Multi-agents systems - multi stations systems

6

C2 node Function

Supervisor Decompose high level tasks in smaller tasks (Protect = Deploy ->

Patrol -> DRIL -> Engage ->BDA), allocate tasks to most capable

stations

UAV station Task allocation to controlled UAV, path planning

UGV station Task allocation to controlled UGV, path planning

ML station Task allocation to controlled ML, path planning

UGV stationUGV station UAV stationUAV station ML station ML station

SupervisorSupervisor

6 Applicability of Artificial Intelligence and Decision to Air Traffic and Unmanned Vehicles

Possible agent architecture for sharing of work in Air Traffic Control centers?

7

Multi agents systems - multi unmanned vehiclesDeliberative distributed architecture:• Executes a hierarchical plan• Reacts to perturbations and disruptive events


Lesire, Charles, Guillaume Infantes, Thibault Gateau, and Magali Barbier. "A distributed architecture for supervision of autonomous multi-robot missions." Autonomous Robots 40, no. 7 (2016): 1343-1362.

Multi-agents systems – multi unmanned vehicles

8

Assessment of cooperation between air and ground autonomous robots� In field demonstration in a fight training village

� Area control mission

� Search and track intruders

� Actual or simulated disruptive events


Multi agent systems – multi unmanned vehicles

9

Robustness validation of real time supervision and planning functions� Respecting planned rendezvous

� Contingent strategies applied� Plan repair after disruptive event

Efficiency of blending partial order planning and hierarchical planning for multi-robots problems

Demonstration of trade off between performance and data privacy for temporal consistency of multi-robots execution


Bechon, Patrick, Magali Barbier, Charles Lesire, Guillaume Infantes, and Vincent Vidal. "Using hybrid planning for plan reparation." In Mobile Robots (ECMR), 2015 European Conference on, pp. 1-6. IEEE, 2015

Multi-agents systems

10

Algorithms for Multi-agent Simple Temporal Network

• Synchronization between agents• Extension taking into account uncertainty on dates

• Do not take into account disjunctions: too simple for separation assurance

Casanova, Guillaume, Cédric Pralet, Charles Lesire, and Thierry Vidal. "Solving Dynamic Controllability Problem of Multi-Agent Plans with Uncertainty." (2016)

Multi-agents patrol problem

• Assessment of strategies with different levels of centralizationOthmani-Guibourg, Mehdi, Amal El Fallah-Seghrouchni, Jean-Loup Farges, and Maria Potop-Butucaru. "Multi-agent patrolling in dynamic environments." In Agents (ICA), 2017 IEEE International Conference on, pp. 72-77. IEEE, 2017


Multi-agents systems – multi unmanned vehicles

Distributed Guidance• Distribution of control laws and estimators• Communication triggered in function of the state es timation

uncertaintyViel, Christophe, S. Bertrand, H. Piet-Lahanier, and M. Kieffer. "New state estimator for decentralized event-triggered

consensus for multi-agent systems." IFAC-PapersOnLine 49, no. 5 (2016): 365-370

Source location mission using gradient or response surface

11

Gathering

detection & identification

reconfiguration– Distributed estimation of source location

– Detection and identification of faulty agents

– Reconfiguration without faulty agents


Discrete Event Systems

Petri nets– Supervision and execution control for autonomous vehicles

– ProCoSA tool• Hierarchical interpreted Petri nets

Barbier, Magali, Claude Barrouil, Jean-François Gabard, and Guy Zanon. "ProCoSA: a Petri Net based software package for autonomous system supervision." In International conference on application and theory of Petri nets and other models of concurrency (ATPN). 2006

Altarica language– Components

– State flows

– For model based safety analysis• Contributing to drone certification• Analysis of collision risk between drone and aircraft


Bieber, Pierre , Seguin, Christel, Louis, Vincent, and Florian Many. "Model Based Safety Assessment of Concept of Operations for Drones." 20th Congrès de maîtrise des risques et de sûreté de fonctionnement, 2016

13

Solving problems:• Planning• Combinatorial algorithms• Evolutionary algorithms• Operation research


Planning: Action planning - search in state space

14

Heuristic search → Yet Another Heuristic Search Planner (YAHSP)

Variants :

• Parallel implementation on many core computers

• Use in YAHSP of landmarks generated by a best first meta search

• Solving multi-criteria planning problems and generating Pareto frontiers: YAHSP is integrated in evolutionary algorithms


Number of solved problems

Advantage of parallel computingfor heuristic search

Com

puta

tion

time

Vidal, Vincent, Lucas Bordeaux, and Youssef Hamadi. "Adaptive k-parallel best-first search: A simple but efficient algorithm for multi-core domain-independent planning." In Third Annual Symposium on Combinatorial Search. 2010

Hybrid architecture for actions and motion planning• Pre conditions interpretation

Symbolicpre conditions

verified

Attitudepre conditions

verified

Behaviorpre conditions

verified

Computation of symbolicpre conditions

noSearch for satisfying attitude

pre conditions

yes

no

Search for satisfyingbehavior pre conditions

yes

Reduction of theattitude search space

no yesApply action effects

Start : tested action

End : impossible action

End : impossible action

End : possible action

Motion plannerRapidly-exploring Random Trees

Symbolic plannerHierarchical task network

Planning: Actions and motion


Guitton, Julien, and Jean-Loup Farges. "Taking into account geometric constraints for task-oriented motion planning." Proc. Bridging the gap Between Task And Motion Planning, BTAMP 9 (2009): 26-33

Rapidly-exploring Random Trees (RRT) with cellular partition of space

Planning: Motion

16

Restriction of configuration space to a corridor

* Partition of the space with a set of cells

* Estimation of cell traversability* Cell selection with A* -> corridor* RRTs developed in the corridor* Assessment of traversability and

new selection in case of failure

Space partition

Com

puta

tion

time

Optimal RRTsUsed for flight of aerial vehicles in heterogeneous environment


Guitton, Julien, Jean-Loup Farges, and Raja Chatila. "Cell-RRT: Decomposing the environment for better plan." In Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on, pp. 5776-5781. IEEE, 2009

Pharpatara, Pawit, Bruno Hérissé, Romain Pepy, and Yasmina Bestaoui. "Shortest path for aerial vehicles in heterogeneous environment using RRT." In Robotics and Automation (ICRA), 2015 IEEE International Conference on, pp. 6388-6393. IEEE, 2015

Planning: Trajectories

Find a safe and short path in a 3D space• 3D occupancy map• Several location modes

Availability mapModel for propagation of location

error• Several guidance modes

Availability mapModel for propagation of execution

error• Uncertainty corridor function of safety distance and standard deviation of execution error• Safe path = no interception between uncertainty corridor and occupied voxels


Watanabe, Yoko, Aurélien Veillard, and Caroline Chanel. "Navigation and Guidance Strategy Planning for UAV Urban Operation." In AIAA Science and Technology Forum and Exposition Forum (SciTech 2016), pp. pp-1. 2016

Collision

Planning: Trajectories

Search in a graph in a 5D space– 3D + location mode + guidance mode

– Neighbor nodes = 26-neighbors in 3D space x any mode transition

– Valid connection between two nodes:• Location and guidance modes are available

• Safe path

– Transition cost = Volume of uncertainty corridor• Minimizes length and width

– Tree search using A* algorithm• Heuristic function :

Null execution error and distance

18

Booking the uncertainty corridor thought the UTM -> obstacles for other drones?


Watanabe, Yoko, Aurélien Veillard, and Caroline Chanel. "Navigation and Guidance Strategy Planning for UAV Urban Operation." In AIAA Science and Technology Forum and Exposition Forum (SciTech 2016), pp. pp-1. 2016

Planning: Decision under uncertainty - Reactive planning

19

Anytime planner managing several queries in background

Reactive execution of planned actionsTests with algorithms for Markov Decision Process and with Real Time Dynamic Programming

• Reduces mission duration for large scale problems

• Performs mission in case the 'plan and then act' approach is not able to do it

• Outperforms greedy method for observation planning

Applicability to FMS trajectory planning?Applicability of Artificial Intelligence and Decision to Air Traffic and Unmanned Vehicles

Chanel, Caroline Ponzoni Carvalho, Charles Lesire, and Florent Teichteil-Königsbuch. "A Robotic Execution Framework for Online Probabilistic (Re) Planning." In ICAPS. 2014

Planning: Decision under uncertainty - Use for perception and mission planning

Partially Observable Markov Decision ProcessTaking into account a measurement vector with contin uous variables* Discrete observation model not enough reliable for vision based robotics

* With a continuous observation model no classifier between image processing and decision

* Improvement with respect to a classical approach for a search and classification of objects in areas mission

Use of possibilities* Smaller algorithmic complexity than for probabilities

* Takes into account poorly validated measurement models

* Transformation of probabilistic POMDP in probabilistic MDP using transformations probabilities ↔ possibilities


Drougard, N., Teichteil-Königsbuch, F., Farges, J. L., & Dubois, D. (2014, July). Structured Possibilistic Planning Using Decision Diagrams. In AAAI (pp. 2257-2263)

Combinatorial algorithms: Constraint Programming

21

InCell library � Constraint based local search

� Model:� Decision variables� Constraints� Criteria

� Sequence of changes:� Searching an optimal solution� Adding or deleting tasks� Incremental evaluation

� Set of constraints:� Time distance� No overlapping� Resource loads� Logical� Arithmetic


Pralet, Cédric, and Gérard Verfaillie. "Dynamic Online Planning and Scheduling Using a Static Invariant-Based Evaluation Model." In ICAPS. 2013

Combinatorial algorithms: Constraint Programming

22

Use of InCell library for projects� Compute acquisition and downloading plans for an autonomous surveillance satellite� Produce plans for a robotic mission where a set of robots make acquisitions and deploy a

communication network to send the acquisitions to a ground station� Solve acquisition task scheduling problem for a set of heterogeneous robots� Solve mission decomposition problem for a set of heterogeneous robots


Pralet, Cédric, and Charles Lesire. "Deployment of mobile wireless sensor networks for crisis management: A constraint-based local search approach." In International Conference on Principles and Practice of Constraint Programming, pp. 870-885. Springer, Cham, 2014.

Combinatorial algorithms: Logic, SAT problem, SMT problem – Application to model checking

SAT: Boolean satisfiability problem– As a formula with Boolean variables a solution?– NP-Complete decision problem

Use of SAT solvers for code verificationSMT – SAT modulo theories – more expressive with, for

instance:– Integer arithmetic– Arrays (read and write inside)– Not interpreted functions, equality, inequality

Program (SCADE or Simulink) with properties -> Pred icatesFalsification on a trace of length nContributing to certification of avionics software


Delmas, Rémi, A. Fernandes Pires, and Thomas Polacsek. "A verification and validation process for model-driven engineering." In Progress in Flight dynamics, guidance, navigation, control, fault detection, and avionics, vol. 6, pp. 455-468. EDP Sciences, 2013.

Evolutionary algorithms: Airspace management

Delahaye, Daniel, Jean-Marc Alliot, Marc Schoenauer, and Jean-Loup Farges. "Genetic algorithms for partitioning air space." In AI 1994, 10th Conference on Artificial Intelligence for Applications, pp. pp-291. IEEE, 1994.

Genetic algorithms used to compute a balanced sectoring of airspace� Increase ATC capacity in high density areas

Chromosome Algorithm iteration

Airspace


Evolutionary algorithms: Assignment and traffic flow management

Delahaye, Daniel, Jean-Marc Alliot, Marc Schoenauer, and Jean-Loup Farges. "Genetic algorithms for air traffic assignment." In Proceedings of the European Conference on Artificial Intelligence. ECAI. 1994.Delahaye, Daniel, Jean-Marc Alliot, Marc Schoenauer, and Jean-Loup Farges. "Genetic algorithms for automatic regroupment of air traffic control sectors." In EP1995, 4th Annual Conference on Evolutionary Programming. 1995.

Genetic algorithms used to compute:

� a traffic assignment on the network to increase ATC capacity in high density areas

� a balanced grouping of sectors to optimally reduce the number of controller teams during daily low flow periods

Chromosome

Chromosome

Operators


Operation research: Assignment and traffic flow manag ement

Deschinkel, Karine, Jean-Loup Farges, and Daniel Delahaye. "Optimisation of prices for air traffic control." In 9th IFAC Symposium Control in Transportation Systems. GMA, Braunschweig, pp. 174-179. 2000.Deschinkel, Karine, Jean-Loup Farges, and Daniel Delahaye. "Pricing policies for air traffic assignment." PROGRESS IN ASTRONAUTICS AND AERONAUTICS 193 (2001): 143-160.Deschinkel, Karine, Jean-Loup Farges, and Daniel Delahaye. "Optimizing and assigning price levels for air traffic management." Transportation Research Part E: Logistics and Transportation Review 38, no. 3 (2002): 221-237.

Dynamic pricing policies:� airline modify departure times and routes� minimizes the en-route congestion� Restricted the number of price levels and assignment of one price level to

each sector at each time periodLogit discrete choice model:

� option = departure time x route� utility : flying cost, cost of ground delay and prices of crossed sectors

Optimization of policy:� minimizes the quadratic difference between desired and expected flows� iterations of:

� simulated annealing for assignment of price levels� gradient values for price levels

No prices Prices


Operation research: Tactical separation assurance

Omer, Jérémy, and Jean-Loup Farges. "Automating air traffic control through nonlinear programming." In 5th International Conference on Research in Air Transportation, ICRAT, Berkeley, USA. 2012.Omer, Jeremy, and J-L. Farges. "Hybridization of nonlinear and mixed-integer linear programming for aircraft separation with trajectory recovery." Intelligent Transportation Systems, IEEE Transactions on 14, no. 3 (2013): 1218-1230.

Avoid collisions on a short time noticeTrajectory planning with collision avoidance:

� Bolza problem� Transcribed into a nonlinear program� Feasible domain non-convex� Nonlinear programming methods may

converge to a local optimumLinearization based on the use of binary variables:

� Mixed Integer Linear Programming� computable global optimum� used to initialize the resolution of the

nonlinear program in order to get a goodsolution


28

Learning strategies


29

Learning strategies

Missile

STT controller

BTT controller

Angles, angle ratesaccelerations

Guidance

Controlled accelerations + constraints

Speed and position

TargetSensors

Line of sight and itsvariation, distance,

closing speed

Commutation or blending

strategyBWT controller

Measured accelerations

Guidance loop is more likely unstable in BTT and BWT modes!BTT > BWT > STT

Speed and position

azT

ayT

azTmax = aTmax

ayTmax


Farges, Jean-Loup, Patrick Fabiani, and Stéphane Le-Menec. "Blending of missile control modes with neural networks." In Neural Networks for Signal Processing, 2003. NNSP'03. 2003 IEEE 13th Workshop on, pp. 141-150. IEEE, 2003.

Learning strategies

30

Minimisation for a finite number of scenarios

c

ayc

azc

-13.4

9.1

0.6

11.7 7.1

-10.8

6.6

-3.4µBWT

9.5

-20.8

0.5

-5.6

0.3

0.0

ayc

ay -8.2

-12.0

1.6ay

-12.0

c=STT

> 8

> 80

azc > -0.05

18 parameters

Three networks: 35 parameters

Cloning expert rules

0,53

0,55

0,57

0,59

0,61

0,63

0 20 40 60

Itérations

dist

ance

de

pass

age

moy

enne

Réseau neuronal

Règles


Ave

rage

mis

sdi

stan

ce

Neural networkRules

Farges, Jean-Loup, Patrick Fabiani, and Stéphane Le-Menec. "Blending of missile control modes with neural networks." In Neural Networks for Signal Processing, 2003. NNSP'03. 2003 IEEE 13th Workshop on, pp. 141-150. IEEE, 2003.

31

Human factors

• Modeling and knowledge representation• Data mining• Display


32

Activity description languages allow the modeling of procedural knowledge

• Activity model• Hierarchical decomposition of tasks ->

elementary operator actions

• Parallelism

• Temporal constraints

• Perception of:

• activities or

• sets of activities or

• absence of activities or set of activities


Maille, Nicolas. "Modeling airline crew activity to improve flight safety analysis." Aerospace Conference, 2017 IEEE. IEEE, 2017

Human Factors: Modeling and knowledge representation

Human factors: Data mining

33

• Comparison with flight data– To recognize procedures

actually used and analyze adherence to Standard Operating Procedures

– Search for regularities and irregularities

• Using a large number of aircraft procedures

– Data mining: Multi-Kernel Anomaly Detection from NASA

• Kernels for discrete and continuous parameters

An atypical Go Around


Maille, Nicolas. "Modeling airline crew activity to improve flight safety analysis." Aerospace Conference, 2017 IEEE. IEEE, 2017.

Human factors: Display


Maille, Nicolas. "Modeling airline crew activity to improve flight safety analysis." Aerospace Conference, 2017 IEEE. IEEE, 2017

Use of display tool FromDady from ENACTrace from a given eventVisual detection of atypical flights

700 flights

35

Conclusion

• Artificial Intelligence and Decision theoretically widely applicable for air traffic and unmanned vehicles

• Possibility for development of systems raise questions:• interaction with humans: operators, pilots, controllers…

• safety

• Artificial Intelligence and Decision gives also some answers• Data mining and display for flight analysis

• Automated code analysis and fault tree building


- eurocontrol | - driving excellence in atm ... agent systems –multi unmanned vehicles 9...

Documents