bloem defense.v16.slides

Optimization and Analytics

for Air Traffic Management

Enabling Decision-Support Tools

Michael Bloem

Stanford University NASA Ames Research Center

Outline

• Air traffic management background

• Research topics

• Research objective and approach

• Case study:

decision-support tool for area supervisors

• Summary of contributions

2

Air Traffic Management is Important

3

Air Traffic Management is Important

civil aviation

was responsible for

5.4% of US GDP

in 2012

[FAA’s 2014 "The Economic Impact of Civil Aviation on the US Economy"]

3

Air Traffic Management is Important

civil aviation

was responsible for

5.4% of US GDP

in 2012

837 million passengers

carried by airlines

operating in US airspace

in 2012

[FAA’s 2014 "The Economic Impact of Civil Aviation on the US Economy"]

3

Air Traffic Management is Complex

4

Air Traffic Management is Complex

∼50,000 flights/day

5,000+ flights simultaneously

4

Air Traffic Management is

Accomplished Largely by

Human Decision Makers

5

Air Traffic Management is

Accomplished Largely by

Human Decision Makers

5

Air Traffic Management is

Accomplished Largely by

Human Decision Makers

5

Air Traffic Management is

Accomplished Largely by

Human Decision Makers

5

Air Traffic Management is

Accomplished Largely by

Human Decision Makers

5

Research Topics: ATM Decisions

6

Research Topics: ATM Decisions

1 Area supervisors:area configuration selection

– Motivation: prescriptive decision model

6

Research Topics: ATM Decisions

1 Area supervisors:area configuration selection

– Motivation: prescriptive decision model

2 Operations managers:assignment of flights to slots

– Motivation: insight into airline delay costs

6

Research Topics: ATM Decisions

1 Area supervisors:area configuration selection

– Motivation: prescriptive decision model

2 Operations managers:assignment of flights to slots

– Motivation: insight into airline delay costs

3 Flow managers:Ground Delay Program implementation

– Motivation: predictive capability and insights

6

Research Objective

decision-

support

tool

7

Research Objective

solution

algorithm

decision-

support

tool

decision

model

constraints

objective

7

Research Approach

expert input

& feedback

solution

algorithm

decision-

support

tool

decision

model

constraints

objective

8

Research Approach

expert input

& feedback

operational

decision

data

solution

algorithm

decision-

support

tool

decision

model

constraints

objective

8

Research Approach

fast-time

simulations

expert input

& feedback

operational

decision

data

solution

algorithm

decision-

support

tool

decision

model

constraints

objective

8

Research Approach

fast-time

simulations

human-in-

the-loop

simulations

expert input

& feedback

operational

decision

data

solution

algorithm

decision-

support

tool

decision

model

constraints

objective

8

Outline

• Air traffic management background

• Research topics

• Research objective and approach

• Case study:

decision-support tool for area supervisors

• Summary of contributions

9

En-Route Air Traffic Control Centers

10

En-Route Air Traffic Control Centers

10

Cleveland Center

11

Cleveland Center Area 4

4645

12

Cleveland Center Area 4

47

48

49

4546

36,000

31,000

24,000Longitude

Latitude

Altitude

13

Sample Area of Specialization Resources

Sectors

47

48

49

4546 4546

4749

48

14

Sample Area of Specialization Resources

Sectors

Controllers

available to fill

operating

positions

47

48

49

4546 4546

4749

48

09:00–09:30 :

09:30–11:00 :

14

Sample Area of Specialization Resources

Sectors

Controllers

available to fill

operating

positions

Workstations

47

48

49

4546 4546

4749

48

09:00–09:30 :

09:30–11:00 :

48

14

Modeling Decisions and Developing Algorithms

fast-time

simulations

human-in-

the-loop

simulations

expert input

& feedback

operational

decision

data

solution

algorithm

decision-

support

tool

decision

model

constraints

objective

15

Modeling Decisions and Developing Algorithms

fast-time

simulations

15

Sample Sectors, Configuration, and Traffic

sample sectors:

16

Sample Sectors, Configuration, and Traffic

sample sectors: at time step k:

Ck

16

Sample Sectors, Configuration, and Traffic

Tk

sample sectors: at time step k:

Ck

16

Decision Model

Configuration Schedule Advisory Problem (CSA)

minimize

K∑

k=1

gk(Ck−1, Tk−1, Ck, Tk)

subject to Ck ∈ Ck, k = 0,1,2, . . . , K

17

Decision Model

Configuration Schedule Advisory Problem (CSA)

minimize

K∑

k=1

gk(Ck−1, Tk−1, Ck, Tk)

subject to Ck ∈ Ck, k = 0,1,2, . . . , K

gk: single-time step advisory cost

17

Decision Model

Configuration Schedule Advisory Problem (CSA)

minimize

K∑

k=1

gk(Ck−1, Tk−1, Ck, Tk)

subject to Ck ∈ Ck, k = 0,1,2, . . . , K

gk: single-time step advisory cost

Ck: set of valid configurations at k

17

Sample Scenario

0timestep1 2 3 4 5

18

Sample Scenario

0timestep1 2 3 4 5

18

Sample Scenario

0timestep1 2 3 4 5

18

Sample Scenario

0timestep1 2 3 4 5

18

Sample Scenario

0timestep1 2 3 4 5

18

Sample Scenario

0timestep1 2 3 4 5

18

Configuration Constraints

0timestep1 2 3 4 5

19

Configuration Constraints

0timestep1 2 3 4 5

19

Configuration Constraints

0timestep1 2 3 4 5

C2

19

Possible Algorithm Advisory

0timestep1 2 3 4 5

20

Possible Algorithm Advisories

0timestep1 2 3 4 5

21

Possible Algorithm Advisories

0timestep1 2 3 4 5

25 = 32 possibilities

21

Advisory Cost: Static Cost

gSk(Ck, Tk)

20% 40% 60% 80% 100% 120%0

2

4

6

8

10

Open Sector Load[aircraft count/Monitor Alert Parameter]

Two−operating positionstatic costgS

k(Ck, Tk)

22

Advisory Cost: Static Cost

gSk(Ck, Tk)

20% 40% 60% 80% 100% 120%0

2

4

6

8

10

Open Sector Load[aircraft count/Monitor Alert Parameter]

Two−operating positionstatic cost

One−operating positionstatic cost

gSk(Ck, Tk)

22

Advisory Cost: Reconfiguration Cost

gRk(Ck−1, Tk−1,Ck, Tk)

(1)

(1)

(2)

(1)

(2)

(2)

(1)

Ck−1, Tk−1 Ck, Tk

23

Advisory Cost: Reconfiguration Cost

gRk(Ck−1, Tk−1,Ck, Tk)

sector & flights moving

(1)

(1)

(2)

(1)

(2)

(2)

(1)

Ck−1, Tk−1 Ck, Tk

between workstations

23

Advisory Cost: Reconfiguration Cost

gRk(Ck−1, Tk−1,Ck, Tk)

open sector gaining

sector & flights moving

(1)

(1)

(2)

(1)

(2)

(2)

(1)

Ck−1, Tk−1 Ck, Tk

between workstations

2nd operating position

23

Advisory Cost: Reconfiguration Cost

gRk(Ck−1, Tk−1,Ck, Tk)

open sector gaining

open sector losingsector & flights moving

(1)

(1)

(2)

(1)

(2)

(2)

(1)

Ck−1, Tk−1 Ck, Tk

between workstations 2nd operating position

2nd operating position

23

Advisory Cost: Reconfiguration Cost

gRk(Ck−1, Tk−1,Ck, Tk)

open sector gaining

open sector losingsector & flights moving

(1)

(1)

(2)

(1)

(2)

(2)

(1)

Ck−1, Tk−1 Ck, Tk

between workstations 2nd operating position

2nd operating position

single-time step advisory cost:

gk(Ck−1, Tk−1, Ck, Tk) = gSk(Ck, Tk)+β

RgRk(Ck−1, Tk−1, Ck, Tk)

23

Advisory Cost: Reconfiguration Cost

gRk(Ck−1, Tk−1,Ck, Tk)

open sector gaining

open sector losingsector & flights moving

(1)

(1)

(2)

(1)

(2)

(2)

(1)

Ck−1, Tk−1 Ck, Tk

between workstations 2nd operating position

2nd operating position

single-time step advisory cost:

gk(Ck−1, Tk−1, Ck, Tk) = gSk(Ck, Tk)+β

RgRk(Ck−1, Tk−1, Ck, Tk)

operational decision data & fast-time simulations

leveraged to select βR

23

Minimum-Cost Advisory

0timestep1 2 3 4 5

0

00 3

0 1

0 0

8

5

0

24

Minimum-Cost Advisory

0timestep1 2 3 4 5

0

00 3

0 1

0 0

8

5

0

gSk(C1, T1)

24

Minimum-Cost Advisory

0timestep1 2 3 4 5

0

00 3

0 1

0 0

8

5

0

gRk(C3, T3, C4, T4)

gSk(C1, T1)

24

Decision Model and Solution Algorithm

Configuration Schedule Advisory Problem (CSA)

minimize

K∑

k=1

gSk(Ck, Tk) + βRgR

k(Ck−1, Tk−1, Ck, Tk)

subject to Ck ∈ Ck, k = 0,1,2, . . . , K

25

Decision Model and Solution Algorithm

Configuration Schedule Advisory Problem (CSA)

minimize g(C,T)

subject to Ck ∈ Ck, k = 0,1,2, . . . , K

25

Decision Model and Solution Algorithm

Configuration Schedule Advisory Problem (CSA)

minimize g(C,T)

subject to Ck ∈ Ck, k = 0,1,2, . . . , K

Shortest path problem → use A∗ algorithm

25

Decision Model Relative to Previous Research:

Objectives and Constraints

AdvisoryCharacteristic

matchconfigurations to

traffic

few disruptivereconfigurations

number of opensectors

number ofoperatingpositions

26

Decision Model Relative to Previous Research:

Objectives and Constraints

AdvisoryCharacteristic

Cano et al.(2007)

MB et al.(2008–2009)

Tien et al.(2010–2012)

matchconfigurations to

traffic

constraintand

optimizeconstraint constraint

few disruptivereconfigurations

constraintnot

consideredconstraint

number of opensectors

minimize minimizenot

considered

number ofoperatingpositions

notmodeled

not modeled minimize

26

Decision Model Relative to Previous Research:

Objectives and Constraints

AdvisoryCharacteristic

Cano et al.(2007)

MB et al.(2008–2009)

Tien et al.(2010–2012)

matchconfigurations to

traffic

constraintand

optimizeconstraint constraint

few disruptivereconfigurations

constraintnot

consideredconstraint

number of opensectors

minimize minimizenot

considered

number ofoperatingpositions

notmodeled

not modeled minimize

26

Decision Model Relative to Previous Research:

Objectives and Constraints

AdvisoryCharacteristic

Cano et al.(2007)

MB et al.(2008–2009)

Tien et al.(2010–2012)

DecisionModel

matchconfigurations to

traffic

constraintand

optimizeconstraint constraint optimize

few disruptivereconfigurations

constraintnot

consideredconstraint optimize

number of opensectors

minimize minimizenot

consideredconstraint

number ofoperatingpositions

notmodeled

not modeled minimize constraint

26

Decision Model Relative to Previous Research:

Objectives and Constraints

AdvisoryCharacteristic

Cano et al.(2007)

MB et al.(2008–2009)

Tien et al.(2010–2012)

DecisionModel

matchconfigurations to

traffic

constraintand

optimizeconstraint constraint optimize

few disruptivereconfigurations

constraintnot

consideredconstraint optimize

number of opensectors

minimize minimizenot

consideredconstraint

number ofoperatingpositions

notmodeled

not modeled minimize constraint

26

Decision Model vs. Operational Decisions:

Problem Instances for Fast-Time Simulations

47

48

49

4546 4546

4749

48

• traffic and configurations from

231 days in 2011 and 2012

• 6 am to midnight local time

• rolling horizon: implement first

hour of two-hour advisories

• five-minute time steps

27

Decision Model vs. Operational Decisions:

Few Disruptive Reconfigurations

3 6 9 12 150

500

1000

1500

2000

2500

Number ofOpen Sector

Instances

Duration [hours]

operational

model

28

Decision Model vs. Operational Decisions:

Match Configurations to Traffic

20% 40% 60% 80% 100% 120%0

2

4

6

8

10

Open Sector Load[aircraft count/Monitor Alert Parameter]

Low HighJust Right

gSk(Ck, Tk)

29

Decision Model vs. Operational Decisions:

Match Configurations to Traffic

Low Just Right High0

20

40

60

80

100

Percent ofOpen Sector-

Minutes

operational

model

30

Modeling Decisions and Developing Algorithms

fast-time

simulations

31

Modeling Decisions and Developing Algorithms

fast-time

simulations

31

Experts’ Feedback and Suggestion

Limitations of decision model

32

Experts’ Feedback and Suggestion

Limitations of decision model

1 difficult to quantify safe and efficient operations

32

Experts’ Feedback and Suggestion

Limitations of decision model

1 difficult to quantify safe and efficient operations

2 does not model individual human controllers

32

Experts’ Feedback and Suggestion

Limitations of decision model

1 difficult to quantify safe and efficient operations

2 does not model individual human controllers

Suggestions for solution algorithm

32

Experts’ Feedback and Suggestion

Limitations of decision model

1 difficult to quantify safe and efficient operations

2 does not model individual human controllers

Suggestions for solution algorithm

• return ∼ 3 good advisories

32

Experts’ Feedback and Suggestion

Limitations of decision model

1 difficult to quantify safe and efficient operations

2 does not model individual human controllers

Suggestions for solution algorithm

• return ∼ 3 good advisories

• one advisory: optimal for decision model

32

Experts’ Feedback and Suggestion

Limitations of decision model

1 difficult to quantify safe and efficient operations

2 does not model individual human controllers

Suggestions for solution algorithm

• return ∼ 3 good advisories

• one advisory: optimal for decision model

• other advisories: not too sub-optimal

32

Experts’ Feedback and Suggestion

Limitations of decision model

1 difficult to quantify safe and efficient operations

2 does not model individual human controllers

Suggestions for solution algorithm

• return ∼ 3 good advisories

• one advisory: optimal for decision model

• other advisories: not too sub-optimal

• distinct advisories

32

Advisory Difference Metric

Φ(C,C′)

Φ(C,C′) =

K∑

k=1

ϕ(Ck, C′k)

33

Advisory Difference Metric

Φ(C,C′)

Φ(C,C′) =

K∑

k=1

ϕ(Ck, C′k)

Configuration difference metric

ϕ(Ck , C′k) =

¨

1 if Ck and C′kcombine sectors differently

0 else

33

Multiple-Advisories Problem Statement

M ϵ-Optimal d-Distinct Configuration Schedule

Advisories Problem (M-ϵ-d-CSAs)

minimize

M∑

m=1

g(Cm, T)

subject to |CM| =M

Cmk∈ Ck k = 0,1,2, . . . , K, m = 1,2, . . . ,M

C1 ∈ C⋆

CSA( = optimal advisories for CSA)

g(Cm, T)− g(C1, T)

g(C1, T)≤ ϵ m = 2,3, . . . ,M

Φ(Cm, Cm′

) ≥ d ∀m,m′ where m 6=m′

34

Multiple-Advisories Problem Statement

M ϵ-Optimal d-Distinct Configuration Schedule

Advisories Problem (M-ϵ-d-CSAs)

minimize

M∑

m=1

g(Cm, T)

subject to |CM| =M

Cmk∈ Ck k = 0,1,2, . . . , K, m = 1,2, . . . ,M

C1 ∈ C⋆

CSA( = optimal advisories for CSA)

g(Cm, T)− g(C1, T)

g(C1, T)≤ ϵ m = 2,3, . . . ,M

Φ(Cm, Cm′

) ≥ d ∀m,m′ where m 6=m′

M-ϵ-d-CSAs is NP-complete

34

Algorithms for M-ϵ-d-CSAs

35

Algorithms for M-ϵ-d-CSAs

1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)

• novel algorithm based on A∗

35

Sequential Distinct A∗ (SDA∗)

Finding advisory C1:

C, T C1shortest path

algorithm

36

Sequential Distinct A∗ (SDA∗)

Finding advisory Cm:

C, T Cm

{C1, C2, . . . , Cm−1}

constrained

shortest path

algorithm

36

Sequential Distinct A∗ (SDA∗)

Finding advisory Cm:

C, T Cm

{C1, C2, . . . , Cm−1}

constrained

shortest path

algorithm

constrained shortest path problem is NP-complete

36

Constrained Shortest Path Algorithm:

Forward Distinct A∗ (FDA∗)

For some λ ≥ 0, find C̃2 that minimizes Lagrangian

L(C2, λ) = g(C2, T)︸ ︷︷ ︸

advisorycost

+λ�

d− Φ(C1, C2)�

︸ ︷︷ ︸

similaritycost

37

Constrained Shortest Path Algorithm:

Forward Distinct A∗ (FDA∗)

For some λ ≥ 0, find C̃2 that minimizes Lagrangian

L(C2, λ) = g(C2, T)︸ ︷︷ ︸

advisorycost

+λ�

d− Φ(C1, C2)�

︸ ︷︷ ︸

similaritycost

Another shortest path problem → use A∗

37

FDA∗ and Duality

Proposition

If M = 2, ϵ =∞, and |C⋆CSA| = 1,

then FDA∗ implements the dual objective:

h(λ) =minimizeC2∈C

L(C2, λ)

38

FDA∗ and Duality

Proposition

If M = 2, ϵ =∞, and |C⋆CSA| = 1,

then FDA∗ implements the dual objective:

h(λ) =minimizeC2∈C

L(C2, λ)

Corollary: If M = 2, ϵ =∞, |C⋆CSA| = 1,

λ = λ⋆, and strong duality holds,

then C̃2 satisfies a necessary condition for C2⋆

(C̃2 = second advisory returned by FDA∗)

38

Sequential Distinct A∗ (SDA∗)

Finding advisory Cm:

C,T Cm

{C1,C2, . . . ,Cm−1}

constrained

shortest path

algorithm

constrained shortest path problem is NP-complete

39

Sequential Distinct A∗ (SDA∗)

Finding advisory Cm:

C,T Cm

{C1,C2, . . . ,Cm−1}

FDA∗

λ

39

Algorithms for M-ϵ-d-CSAs

1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)

• novel algorithm based on A∗

40

Algorithms for M-ϵ-d-CSAs

1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)

• novel algorithm based on A∗

2 Forward Backward Value Iterationwith Sequential Advisory Search (FBVISAS)

• novel algorithm based on value iteration

40

Forward Backward Value Iteration

with Sequential Advisory Search (FBVISAS)

0timestep1 2 3 4 5

41

Forward Backward Value Iteration

with Sequential Advisory Search (FBVISAS)

0timestep1 2 3 4 5

41

Forward Backward Value Iteration

with Sequential Advisory Search (FBVISAS)

0timestep1 2 3 4 5

41

Forward Backward Value Iteration

with Sequential Advisory Search (FBVISAS)

0timestep1 2 3 4 5

41

Theoretical Guarantees for FBVISAS:

the Good News

42

Theoretical Guarantees for FBVISAS:

the Good News

Simple M-ϵ-d-CSAs instance: M = 2, ϵ =∞, |C⋆CSA| = 1,

C1 = C2 = · · · = CK, and C0 ∈ C1

42

Theoretical Guarantees for FBVISAS:

the Good News

Simple M-ϵ-d-CSAs instance: M = 2, ϵ =∞, |C⋆CSA| = 1,

C1 = C2 = · · · = CK, and C0 ∈ C1

Reconfiguration cost-dominated M-ϵ-d-CSAs instance:

reconfiguring never decreases the advisory cost

42

Theoretical Guarantees for FBVISAS:

the Good News

Simple M-ϵ-d-CSAs instance: M = 2, ϵ =∞, |C⋆CSA| = 1,

C1 = C2 = · · · = CK, and C0 ∈ C1

Reconfiguration cost-dominated M-ϵ-d-CSAs instance:

reconfiguring never decreases the advisory cost

Proposition

If one exists, FBVISAS finds an optimal C2⋆ for

simple reconfiguration cost-dominated instances

42

Theoretical Guarantees for FBVISAS:

the Bad News

Static cost-dominated M-ϵ-d-CSAs instance: βR = 0

43

Theoretical Guarantees for FBVISAS:

the Bad News

Static cost-dominated M-ϵ-d-CSAs instance: βR = 0

Proposition

If d > 1, FBVISAS does not return any C2 for

simple static cost-dominated instances

43

Algorithms for M-ϵ-d-CSAs

1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)

• novel algorithm based on A∗

2 Forward Backward Value Iterationwith Sequential Advisory Search (FBVISAS)

• novel algorithm based on value iteration

44

Algorithms for M-ϵ-d-CSAs

1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)

• novel algorithm based on A∗

2 Forward Backward Value Iterationwith Sequential Advisory Search (FBVISAS)

• novel algorithm based on value iteration

3 Lowest-Cost Paths (LCP)

• finds optimal solution to relaxation• many efficient algorithms

44

Algorithms for M-ϵ-d-CSAs

1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)

• novel algorithm based on A∗

2 Forward Backward Value Iterationwith Sequential Advisory Search (FBVISAS)

• novel algorithm based on value iteration

3 Lowest-Cost Paths (LCP)

• finds optimal solution to relaxation• many efficient algorithms

4 Value Iteration Fraction Optimalwith Exhaustive Advisory Search

• finds optimal solution• not computationally efficient

44

Modeling Decisions and Developing Algorithms

fast-time

simulations

45

Modeling Decisions and Developing Algorithms

fast-time

simulations

45

Evaluation of Algorithms on Small Instances

47

48

49

4546 4546

4749

48

• traffic and configurations from

two days in December 2011

• nine two-hour instances per day

• 18 total instances

• five-minute time steps

• C requires same number of open

sectors as were used operationally

• request two advisories (M = 2)

• ϵ = 0.2 constraint on cost of C2

• d requires 30 minutes of different

airspace configurations

46

Small Instances: Properties of C2

SDA∗-SC FBVISAS LCP0

20

40

60

80

100

Percent ofInstances

optimal C2⋆

47

Small Instances: Properties of C2

SDA∗-SC FBVISAS LCP0

20

40

60

80

100

Percent ofInstances

optimal C2⋆

feasible C2

47

Evaluation of Algorithms on Realistic Instances

47

48

49

4546 4546

4749

48

• traffic and configurations from

231 days in 2011 and 2012

• 6 am to midnight local time

• rolling horizon: implement first

hour of two-hour advisories

• 4158 total instances

• five-minute time steps

• C only requires same initial

configuration as was used

operationally

• request three advisories (M = 3)

• ϵ = 0.5 constraint on cost

• d requires 30 minutes of different

airspace configurations

48

Realistic Instances: Advisory Costs

0.7 0.8 0.9 1.1 1.2 1.30

20

40

60

80

100

0.99 1.01

Percent ofInstances

Cost Ratio�g(C2 ,T)+g(C3 ,T) for SDA∗-SC

g(C2 ,T)+g(C3 ,T) for FBVISAS

�

SDA∗-SCworse

SDA∗-SCbetter

mean = 1.02

49

Realistic Instances: Computation Times

(on a Workstation)

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Computation

Times

[seconds]

SDA∗-SC FBVISAS

50

Modeling Decisions and Developing Algorithms

fast-time

simulations

51

Modeling Decisions and Developing Algorithms

51

Operational Airspace Sectorization

Integrated System (OASIS)

52

OASIS Screenshot: Multiple Advisories

53

Human-in-the-Loop Simulations Set Up

• eight retired FAA

personnel

• four simulated

scenarios

54

Human-in-the-Loop Simulations Set Up

• eight retired FAA

personnel

• four simulated

scenarios

Three experimental conditions in which users

1 generate configuration schedule

2 select from among algorithm-generated advisories

3 select from among algorithm-generated advisories

and then modify

54

Human-in-the-Loop Simulations Results

[Lee et al., 2013]

55

Human-in-the-Loop Simulations Results

[Lee et al., 2013]

Advisories enable safe and efficient operations

• average acceptability of selected

algorithm-generated advisories: > 4 out of 5

• user modifications were minor and led to

no significant improvement in acceptability

55

Human-in-the-Loop Simulations Results

[Lee et al., 2013]

Advisories enable safe and efficient operations

• average acceptability of selected

algorithm-generated advisories: > 4 out of 5

• user modifications were minor and led to

no significant improvement in acceptability

Providing multiple advisories added value

• in more than 60% of instances, users selected

second or third advisory

• when asked how many advisories they wanted the

tool to provide, users requested an average of 2.8

55

Human-in-the-Loop Simulations Results

[Lee et al., 2013]

Advisories enable safe and efficient operations

• average acceptability of selected

algorithm-generated advisories: > 4 out of 5

• user modifications were minor and led to

no significant improvement in acceptability

Providing multiple advisories added value

• in more than 60% of instances, users selected

second or third advisory

• when asked how many advisories they wanted the

tool to provide, users requested an average of 2.8

Computation times were rated as highly acceptable

55

Modeling Decisions and Developing Algorithms

56

Outline

• Air traffic management background

• Research topics

• Research objective and approach

• Case study:

decision-support tool for area supervisors

• Summary of contributions

57

Summary of Contributions

58

Summary of Contributions

1 Area supervisors:area configuration selection

– developed prescriptive decision model– designed efficient algorithms to findlow-cost and distinct paths

– implemented algorithm in decision-support tool

58

Summary of Contributions

1 Area supervisors:area configuration selection

– developed prescriptive decision model– designed efficient algorithms to findlow-cost and distinct paths

– implemented algorithm in decision-support tool

2 Operations managers:assignment of flights to slots

– designed maximum-likelihood-based algorithm that– generated insights from operational decision data

58

Summary of Contributions

1 Area supervisors:area configuration selection

– developed prescriptive decision model– designed efficient algorithms to findlow-cost and distinct paths

– implemented algorithm in decision-support tool

2 Operations managers:assignment of flights to slots

– designed maximum-likelihood-based algorithm that– generated insights from operational decision data

3 Flow managers:Ground Delay Program implementation

– deployed supervised learning and inversereinforcement learning models that

– produced predictive capability and insights

58

Relevant peer-reviewed conference papers1 M. Bloem and P. Gupta, "Configuring Airspace Sectors with Approximate Dynamic

Programming," ICAS, September 2010.

2 M. Bloem and H. Huang, "Evaluating Delay Cost Functions with Airline Actions inAirspace Flow Programs," USA/Europe ATM R&D Seminar, June 2011.

3 M. Bloem and N. Bambos, "Coordinated Tactical Air Traffic and Airspace Management,"IEEE CDC, December 2011.

4 M. Bloem, M. Drew, C. F. Lai, and K. Bilimoria, "Advisory Algorithm for Scheduling OpenSectors, Operating Positions, and Workstations," AIAA ATIO, September 2012.

5 M. Bloem, H. Huang, and N. Bambos, "Approximating the Likelihood of Historical AirlineActions to Evaluate Airline Delay Cost Functions," IEEE CDC, December 2012.

6 M. Bloem and N. Bambos, "An Approach for Finding Multiple Area of SpecializationConfiguration Advisories," AIAA ATIO, August 2013.

7 M. Bloem and N. Bambos, "Ground Delay Program Analytics with Behavioral Cloningand Inverse Reinforcement Learning," AIAA ATIO, June 2014.

8 M. Bloem and N. Bambos, "Infinite Time Horizon Maximum Causal Entropy InverseReinforcement Learning," IEEE CDC, December 2014.

Relevant journal articles1 M. Bloem, P. Gupta, and P. Kopardekar, "Algorithms for Combining Airspace Sectors," Air

Traffic Control Quarterly, Vol. 17, No. 4, 2009.

2 M. Bloem, M. Drew, C. F. Lai, and K. D. Bilimoria, "Advisory Algorithm for Scheduling OpenSectors, Operating Positions, and Workstations," AIAA Journal of Guidance, Control, andDynamics, Vol. 37, No. 4, July–August 2014.

3 M. Bloem and N. Bambos, "Air Traffic Control Configuration Advisories fromNear-Optimal Distinct Paths," AIAA Journal of Aerospace Information Systems, Vol. 11,No. 11, November 2014.

4 M. Bloem and N. Bambos, "Ground Delay Program Analytics with Behavioral Cloningand Inverse Reinforcement Learning," AIAA Journal of Aerospace Information Systems,accepted on 21 December 2014, available online on 03 March 2015.

5 M. Bloem and N. Bambos, "Stochastic Models of Ground Delay ProgramImplementation for Prediction, Simulation, and Insight," Journal of AerospaceOperations, invited submission to special issue, expected publication in late 2015.

59

Acknowledgments

• Stanford

• NASA

• Family & friends

60

Questions?

Backup Slides

62

Why Improve Area Configuration Planning?

1 more efficient flights

2 fewer disruptions to area supervisors and

traffic managers

3 better controller staff management

4 prepare for increased flexibility in future operations

→ more complex planning

63

Modeling Decisions and Developing Algorithms

fast-time

simulations

64

Operational Decision Data Set and

Fast-Time Simulation Setup

47

48

49

4546 4546

4749

48

• traffic and configurations from

230 days in 2011 and 2012

• 6 am to midnight local time

• rolling horizon: implement first

hour of two-hour advisories

• five-minute time steps

• only airspace configurations

• constraint: only use typical

configurations

65

Static Cost versus Reconfiguration Cost

700 800 900 1000 11000

50

100

150

200

Total

Reconfiguration

Cost

Total Static Cost

operational

βR = 0.5

βR = 15

ratio(βR) =total reconfiguration cost(βR)

total static cost(βR)

error(βR) =��ratio(operational)− ratio(βR)

��

66

Selecting βR

0 1.75 5.0 10 155

10

15

20

25

Total Error

βR

set βR = 1.75

67

Uncertainty in Traffic Predictions:

Why Not Incorporated?

• Experts understand prediction errors

• Avoid inconsistency with other deterministic tools

68

Uncertainty in Traffic Predictions:

Uncertain Future Aircraft Counts

0 5 10 15 200

0.05

0.1

0.15

0.2

0.25

Aircraft Count

Probability

prediction

distribution

69

Uncertainty in Traffic Predictions:

Impact on Advisory Costs

Advisories generated using predicted traffic with realistic errors

are typically 2.5%–12.5% costlier

than advisories generated with perfectly-predicted traffic

70

Uncertainty in Traffic Predictions:

Approximate Dynamic Programming Approach

• Finite time horizon Markov decision process (MDP)

– state: sector aircraft counts and predictions thereof– action: current configuration (not a schedule)– state transitions: based on prediction errors inoperational tool

– objective: similar to decision model (CSA)

• Rollouts algorithm performs 15% better than a

heuristic and within 2% of optimal

• Rollouts algorithm computes solutions fast enough

for real-time implementation

M. Bloem and P. Gupta, "Configuring Airspace Sectors withApproximate Dynamic Programming," ICAS, September 2010.

71

Uncertainty in Traffic Predictions:

Other Approaches

• improve predictions with a statistical model

• discount contribution to cost by predicted aircraft

• stochastic shortest path problem

• risk-averse advisory

• larger MDP formulation

• robust set of advisories

72

SDA∗ with Shortcuts (SDA∗-SC)

Objectives:

1 computation time ≤ that of A∗ for each advisory

→ use data from reverse A∗ for C1

to find "shortcuts"

2 no re-tuning of λ per instance or advisory

→ normalize advisory cost and similarity cost terms

73

Forward Backward Value Iteration

with Sequential Advisory Search (FBVISAS)

• Use value iteration to find minimum-cost advisory

through each Ck ∈ Ck, k = 1, . . . ,K

• Sort these advisories from low to high cost

• Initialize set of advisories to return: CM ← ∅

• Loop:

– select next advisory– if advisory is sufficiently different from otheradvisories in C

M, add it to CM

– if |CM| =M, stop

74

Multiple Advisories Problem is NP-Complete

For an arbitrary instance of the independent set

problem on G(V,E), construct M-ϵ-d-CSAs instance:

• M = required number of independent nodes

• gk() = 0

• K = 1 and C1 = V

• d = 1

• configuration difference metric:

ϕ(Ck,C′k) =

¨

1 if Ck and C′kindependent in G(V,E)

0 else

∃ solution to the independent set instance

⇔ ∃ solution to the M-ϵ-d-CSAs instance

75

Computational Complexity

of Multiple Solutions Problem Algorithms

Algorithm Complexity

VIFOEAS O(n3K)a

FBVISAS O(n2K + nK log(nK + 1))

SDA∗-SC O(Mn2K log(nK +1))

Eppstein lowest-cost paths O(n2K + nK log(nK + 1) +M)

Suurballe lowest-cost node-disjoint paths O(Mn2K log(nK +1))b

a This does not include the complexity of searching through�|Cϵ |

M

�advisory

subsets.b This assumes that Dijkstra’s algorithm is used as a subroutine for findingshortest paths.

76

Airline Delay Cost Model Evaluation

decision-

support

tool

77

Airline Delay Cost Model Evaluation

Motivation: how costly is a flight delay?

78

Airline Delay Cost Model Evaluation

Motivation: how costly is a flight delay?

Operational decision data:

default assignment

ABC1

10:00

ABC2

10:30

Slot #60

11:00

Slot #90

12:00

78

Airline Delay Cost Model Evaluation

Motivation: how costly is a flight delay?

Operational decision data:

airline-selected assignment

ABC1

10:00

ABC2

10:30

Slot #60

11:00

Slot #90

12:00

78

Airline Delay Cost Model Evaluation

Motivation: how costly is a flight delay?

Operational decision data:

airline-selected assignment

ABC1

10:00

ABC2

10:30

Slot #60

11:00

Slot #90

12:00

Contribution: novel algorithm enables determination of

delay cost model and cost noise parameters that

maximize an approximation of likelihood of data

78

Airline Delay Cost Model Evaluation

• developed novel algorithm for finding cost noise

parameters that maximize an approximation of the

likelihood of minimum-cost matching problem

instance solutions, given a candidate cost model

79

Airline Delay Cost Model Evaluation

• developed novel algorithm for finding cost noise

parameters that maximize an approximation of the

likelihood of minimum-cost matching problem

instance solutions, given a candidate cost model

• evaluated more than ten proposed airline delay

cost models using hundreds of slot swap decisions

79

Airline Delay Cost Model Evaluation

• developed novel algorithm for finding cost noise

parameters that maximize an approximation of the

likelihood of minimum-cost matching problem

instance solutions, given a candidate cost model

• evaluated more than ten proposed airline delay

cost models using hundreds of slot swap decisions

• determined delay cost model that maximizes

approximate likelihood of data for each airline

79

Airline Delay Cost Model Evaluation

• developed novel algorithm for finding cost noise

parameters that maximize an approximation of the

likelihood of minimum-cost matching problem

instance solutions, given a candidate cost model

• evaluated more than ten proposed airline delay

cost models using hundreds of slot swap decisions

• determined delay cost model that maximizes

approximate likelihood of data for each airline

• estimated cost noise parameters for each delay

cost model for each airline

79

Ground Delay Program (GDP)

Implementation Modeling

decision-

support

tool

80

Ground Delay Program (GDP)

Implementation Modeling

Operational decision data:

arrival rate control arrivals

start end per

airport time time hour

SFO 9:00 am noon 30

81

Ground Delay Program (GDP)

Implementation Modeling

Operational decision data:

arrival rate control arrivals

start end per

airport time time hour

SFO 9:00 am noon 30

Motivation: predict & understand GDP implementation

81

Ground Delay Program (GDP)

Implementation Modeling

Operational decision data:

arrival rate control arrivals

start end per

airport time time hour

SFO 9:00 am noon 30

Motivation: predict & understand GDP implementation

Contribution: produced predictive models and insights

by developing behavioral cloning and inverse

reinforcement learning models of GDP implementation

81

Ground Delay Program (GDP)

Implementation Modeling

• evaluated behavioral cloning and inverse

reinforcement learning models of GDP

implementation at EWR and SFO using hundreds of

days of operational decision data

82

Ground Delay Program (GDP)

Implementation Modeling

• evaluated behavioral cloning and inverse

reinforcement learning models of GDP

implementation at EWR and SFO using hundreds of

days of operational decision data

• demonstrated that a behavioral cloning model can

produce superior predictions of GDP

implementation

82

Ground Delay Program (GDP)

Implementation Modeling

• evaluated behavioral cloning and inverse

reinforcement learning models of GDP

implementation at EWR and SFO using hundreds of

days of operational decision data

• demonstrated that a behavioral cloning model can

produce superior predictions of GDP

implementation

• determined that neither class of model provides

much evidence that conditions beyond those in the

next two hours impact GDP implementation

82

Ground Delay Program (GDP)

Implementation Modeling

• evaluated behavioral cloning and inverse

reinforcement learning models of GDP

implementation at EWR and SFO using hundreds of

days of operational decision data

• demonstrated that a behavioral cloning model can

produce superior predictions of GDP

implementation

• determined that neither class of model provides

much evidence that conditions beyond those in the

next two hours impact GDP implementation

• estimated a reward function that provides insight

into metrics guiding GDP implementation

82