evidence for change in flight crew training - halldale · evidence for change in flight crew...

APATS Bangkok 17.09.2013 1

Evidence for change in flight crew training

developed by Thomas LeoffIAAPS chairman


Follow Me….


Evolution in Aviation

Where do we come from?

• Today s training system is based on the seventies

• ICAO s SARPs on training stem from that time with the exception of the MPL

• The operation philosophies of aircraft like the B707 which where a core part of that huge progress in aviation at that time are still a cornerstone in today s training



The seventies….


… are the scenario for today's legal requirements for training programs

They are:• Based on traditional developments• Engineering driven• Knowledge based• Based on numbers like requirements of

flight hours of instruction or landings, legs etc.


The major deficiencies are:

• Syllabus structures from the 70ies and 80ies

• No feedback from the real world OPS• Human Factors training outdated and not

related to actual needs • No feedback system to control the

efficiency of the training in place


International activities by IATA, ICAO and EASA

There are 3 international activities which are relevant for the corrective action:• IATA´s ITQI initiative• ICAO´s developments regarding EBT• EASA´s planned Rulemaking Activity for

Evidence Based Training

IATA´s Data Report on EBT (soon to be published)



The 4 Ws of change in training:• What – training, testing & legal requirements

therefore• Why? – number of new pilots in the next 20

years, introduction of SMS for ATOs, pilot qualification as key factor for safety improvements and the frequent technical improvements in the system

• Who? – ICAO, EASA, NAAs &Stakeholders• When? – ASAP!


The „What?“

• The legal requirements• The scenarios for the training• The Training Programs and their

supervised effectiveness• The form of examination• The qualification of instructors and

examiners


The „Why?“• Pilot Demand forecasted for the next 20

years• The new concept of Competency- &

Evidence Based Training• Introduction of SMS in ATOs and other

sources of operational data and the new opportunities from that

• Current “flat” trend lines of accident rates


Pilot Demand Forecast


The actual accident trend lines

Two practical samples for human deficiencies

They stem from a Research Project in which several German Universities from Bremen, Berlin and Munich cooperate with Lufthansa.The goal is, to develop valid Safety Performance Indicators for the SMS


FINISHED FILES ARE THE RESULT OF YEARS OF

SCIENTIFIC STUDY COMBINED WITH THE

EXPERIENCE OF YEARS

Kategorie Fehler MTBF

Task Related Probability Of Errors MTBFs

Prof. Bubb TU-Munich

Simple and regularely performed tasks at a low stress level.

1 . 10-3

Complex tasks in unusual situations at a high stress level and / or time pressure.

Complex , regularely performed tasks in a well known working environment at a low stress level.

1 · 10-1

1 · 10-2

~30 min

~5 min

~30 sec

Category Error probability MTBF


Today we fly…


… and that´s today´s A380!


… and here the new B787


What shall we update?

• A simple update of the actual set of Training Programs is not enough!

• We must initiate a more sophisticated approach!

• How could we achieve that?• We should start with the newly defined

Pilot Core Competencies and Behavioral Indicators….


The new set of Pilot Competenciesas defined by IATA/ICAO/IFALPA February 2012


How can we adapt the training to that new approach?

• Integration of the SMS of operators and the ATOs

• Defined feedback loops based on data from the SMS, incident reports & analysis

• This feedback loop must cover the entire system from initial to recurrent training

• Research to support the SMS utilization

An Example from our Research Project

Simulator study measuring manual flying skills regarding Performance Shaping Factors


Technische Universität MünchenLehrstuhl für Ergonomie

Simulatorstudie zu manuallen Flugfertigkeiten

Simulator study measuring manual flying skills regarding Performance Shaping Factors

Dipl.-Ing. Andreas Haslbeck

Dipl.-Ing. Ekkehart Schubert, Cpt. Manfred Müller



Captains A346 FOs A320

Age MW 50,0 SD 4,1 MW 30,4 SD 3,1

2. Demographic Data

2.5622.093

14.851

3.374

0

2.000

4.000

6.000

8.000

10.000

12.000

14.000

16.000

18.000

20.000

A346 A320

Flugerfahrung in Flugstunden

Flugstunden Typ Flugstunden gesamt

3

17

0

5

10

15

20

A346 A320

Landungen als PF in 30 Tagen

Simulator Study: manual flying skills

Flight hours Landings as PF within last 30 days

on type Total hours

„Quality nine

three kilo“

„Continental

one ooh six“„Whitestar

one eight“

„Lufthansa

one seven eight niner“

„Singapore

three two eight“

„Eurotrans

zero seven“

„Lufthansa

seven three one“

„Lufthansa

seven two seven“

Traffic Scenarios



Scenario 1 (summary)

EDDMILS APP 08L with >15Kts tailwind (WX: VIS >3.000m OVC 800ft)

G/A Standard Missed APP

Raw Data ILS APP 26R (WX >non-precision: VIS 2.000m OVC 350ft)

Simulator study manual flying skills


Simulatorstudie zu manuallen FlugfertigkeitenSimulatorstudie zu manuallen Flugfertigkeiten

Measuring basic (manual) flying performance

The Simulator crew measures deviations:•from Localizer•altitude and sinkrate (Glideslope)•approach speed•touchdown point

A Computer program integrates all measured deviations.

In addition pilot performance assessment by an experienced training pilot.

Technische Universität Berlin

VP Level Loc GS speed visual landing average grade

1 6 1 1 2 1 1 2 2

2 5 3 2 2 2 4 3 3 weiblich

3 2 1 1 1 2 1 1,33333 1

4 1 1 1 1 1 1 1 1

5 2 1 1 2 2 1 1,5 2

6 2 1 1 1 2 1 1,33333 1

7 3 1 1 1 2 2 1,66667 2

8 1 1 1 1 1 1 1 1

9 1 1 1 2 2 1 1,33333 1

10 5 1 2 2 1 1 2 2

11 1 1 1 1 2 2 1,33333 1

12 1 1 1 1 1 2 1,16667 1

13 2 1 1 1 3 3 1,83333 2

14 3 2 2 2 2 2 2,16667 2

15 3 1 1 2 2 1 1,66667 2

16 1 1 3 2 4 2 2,16667 2

17 1 2 2 2 3 3 2,16667 2 weiblich

18 2 1 1 1 2 2 1,5 2

19 4 1 1 2 2 1 1,83333 2

20 2 1 1 2 2 1 1,5 2

21 1 1 1 1 2 1 1,16667 1

22 2 1 1 2 2 1 1,5 2

23 3 1 2 2 2 2 2 2

24 3 1 1 2 1 1 1,5 2

25 2 1 1 1 2 2 1,5 2

26 1 1 1 1 1 1 1 1

27 3 1 1 1 2 1 1,5 2

28 6 6 6 4 6 3 5,16667 5 weiblich

29 3 1 1 1 1 1 1,33333 1

30 2 1 1 2 2 1 1,5 2

A320-Pilots Grading


-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2

x 105

-5

0

5

10

15

20x 10

4

X [ft]

Y [

ft]

Topansicht

EDDM 26R

MIQ

DM431

DM429

GUDEGMAGAT

WLD

BURAM

XERUM

A320A340-600

34

Datarecording(A340 & A320 FFS)

eye tracking camera

FMA-Checks

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

4 8 9

10

12

13

14

15

16

17

18

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

49

50

51

52

53

54

55

56

58

60

Ø C

PT

Ø F

O

An

teil

ge

täti

gte

rF

MA

-Ch

eck

s

CPT FO

Performance variability (all kinds of FMA-Checks)

Speed-ChecksSpeed/vert.Speed-Checks unter 100ft

19,0%

9,5%

64,0%60,0%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Speed überprüft zwischen 100ft und TD vert. Speed überprüft zwischen 100ft und TD

An

teil

de

r P

ilo

ten

CPT FO

F/O A320

CPT A346

F/O A320

CPT A346

Speed / Vertical Speed Checks below 100ft RA

Speed checked below 100ft and TD V/S checked below 100ft and TD

19.05.2014Seite 38

Bitte diesen Text mit dem Präsentationstitel überschreibenDeutsche Lufthansa AG

SaMSys Simulator Experiments

Measuring…

Economical Flight Performance

FMS Mode Awareness / Automation

Handling unclear situations (Warsawa)

Redundancy and team-interaction

Go-Around-decision and -performance

Basic Flying Skills (AI and Boeing)

Predictability of pilot performance (DLR-test)

Simulator Study: Manual Pilot Skills

Test Scenario 2

• The tower reported wind is within limits. However the actual surface wind is beyond the limit (18Kts instead of max. 10Kts)

• A Go-Around should be performed!

• In 70ft a Go-Around is triggered by the PM

• Runway change after the Go-Around.


39% of all pilots discussed the tailwind situation during the approach briefing.

89% of all pilots talked about the strong tailwind during approach above 1000ft AGL.

50% of all pilots checked the wind indication during final approach below 1000ft AGL.

64% of all pilots would have landed without the wind information given by the PM (TW of 18 Kts).

14% of all pilots decided to land, knowing the tail wind was 18Kts.

44% of all A340 pilots performed the Go-Around only after the PM gave the command in ~70ft AGL.

Go-Around Anslysis(MSc of Alexander Schmidt)

40

19.05.2014Seite 41

Bitte diesen Text mit dem Präsentationstitel überschreibenDeutsche Lufthansa AG

SaMSys Simulator Experiments

Measuring…

Economical Flight Performance

FMS Mode Awareness / Automation

Handling unclear situations (Warsawa)

Redundancy and team-interaction

Handling the Go-Around

Basic Flying Skills (AI and Boeing)

Predictability of pilot performance (DLR-test)


Go-Around Analysis(MSc of Alexander Schmidt)

a) 1 A340-600 captain (3,7%) performs the Go-Around correctly.10 A320-F/O (33%) perform the Go-Around correctly.

b) In general more than one SOP-violation per Go-Around Average: A340-600 Cpt.: 2 , A320 F/O: 1,3.

c) Common SOP violations:• „Missed Appr. Proc.“ not correct („NAV-Mode“ not activated).• Flaps not operated according SOP.• Thrust Reduction to late and/or MCT is selected instead of CLB thrust.• Vertical speed is not reduced during the last 1000ft before reaching

Go-Aroud Altitude.

42

Masterarbeit Alexander Schmidt


Vertical Speed within the last 1.000ft before reach ing the

level off altitude

0,0%

12,9%

35,5%

51,6%

3,2%

67,7%

16,1% 12,9%

0,0

10,0

20,0

30,0

40,0

50,0

60,0

70,0

80,0

< 1000 1000-1500 1500-2000 >2000

rela

tive H

äufig

keit

[%]

Steiggeschwindigkeit [ft/min]

A340

A320

Go-Around Analysis

v/s in ft/min

Go Around CampaignFRA CF, Annina Müller23. Februar 2010

Fligth Recorder Data Analysis

�Only 5% of all Go-Arounds are flown correctly.

� In average 2,7 SOP-violations can be observed per Go-Around.

� Analysis of 150 Go-Arounds showed 415 SOP-violations .

�Maximum number of violations per Go-Around: 10.

But a word of BIG CAUTION• Research is necessary but must be

performed by competent researchers• Human monitoring of data acquisition is a

must • Computerized data analysis of OPS data

has risks of misleading results• Identification of root causes is paramount

but difficult


What are the next steps?

• A Process for data driven revision and update of training programs must be implemented – the “Inner Loop”

• Control the effectiveness of the training in an “Outer Loop” by measuring the performance in the real OPS through valid Safety Performance Indicators (SPIs)

• Develop such valid SPIs for the Airline Operations and the related training


Main topics for improvement

• Full application of all available training media backed up by a regular training transfer control process

• Global harmonization and joint efforts in the development of the new Evidence Based Training System

• Safety shall be the globally agreed driving factor for this new training system


Conclusion• Initiate a human centered approach to

training, teaching and learning by using all available means of training know-how & technology and validate the efficiency

• Design the testing of knowledge, capabilities and the resulting performance as close as possible to the requirements of the aviation system

• Ensure regular updates driven by data


Thank you for your attention!

evidence for change in flight crew training - halldale · evidence for change in flight crew...

Documents