copy of overheads
TRANSCRIPT
© D
édal
e &
EU
RO
CO
NTR
OL
Systemic Occurrence Analysis Methodology
Tony LICUEUROCONTROL
Glasgow – Aug 2005
SOAM
A Reason-Based Organisational Methodology
A tool for the analysis of safety occurrences
(accidents & Incidents)
EUROCONTROL SOAM
SOAM Antecedents
The Reason Model ~ circa 1990 Developed from Professor James Reason’s
work on human error and “organisational accidents”
Tripod Delta ~ circa 1994 Developed for Shell Petroleum, based on
Reason Model ICAM ~ circa 2000
Developed for BHP Billiton, based on Reason Model and Tripod Delta
EUROCONTROL SOAM
“Recent” fatal accidents in Europe with ATM
contribution
Überlingen, 1 July 2002
Linate, 8 October 2001
EUROCONTROL SOAM
Call for Action
Aviation Safety is still at a good level Traffic however is increasing Two major accidents involving Air
Traffic Management in 2001 and 2002 …..
In well developed countries in the heart of Europe …
From which many lessons can be learnt Wake up call for actionWake up call for action
EUROCONTROL SOAM
AGAS/ SSAP Priority Areas
1. Safety related human resources in ATM2. Incident reporting and data sharing3. ACAS/TCAS4. Ground-based safety nets5. Runway safety6. Enforcement of ESARRs and implementation
monitoring7. Awareness of safety matters8. Safety and human factors research & development
An Action Group of European aviation safety experts An Action Group of European aviation safety experts identified the following areas as needing immediate identified the following areas as needing immediate focus: focus: 1. Safety related human resources in ATM2. Incident reporting and data sharing3. ACAS/TCAS4. Ground-based safety nets5. Runway safety6. Enforcement of ESARRs and implementation monitoring7. Awareness of safety matters8. Safety and human factors research & development
EUROCONTROL SOAM
Why anotherinvestigation tool?
Support ESARR 2 implementation and Strategic Safety Action Plan
Higher quality reports and AST returns ~ a need to:
clearly identify causes and report them concisely
go beyond the human errors, to find systemic causes
use a simple, consistent approach for events of all severity levels
ensure recommendations are relevant and effective
EUROCONTROL SOAM
Current investigation methods
What can we improveabout the way we conduct
safety occurrence investigations?
EUROCONTROL SOAM
How SOAM can help
A methodology that includes structured processes to:
identify and classify a range of contributing factors
sort out irrelevant, non-contributing facts move from a focus on human error/s to
identify systemic causes ~ support for ‘Just Culture’
analyse simple events through to high severity incidents and accidents
clearly link recommendations to the facts of the analysis
EUROCONTROL SOAM
Elements of Organisational Occurrences
Organisational Factors: Latent system failures that produce or allow
conditions under which accidents are possible Contextual Conditions:
Situational factors involving characteristics of the task, the environment or human limitations
Human Involvement: Errors and/or violations which have an immediate
adverse affect (“active failures”) Inadequate or absent barriers/defences:
Failure to identify and protect the system against human errors or violations, local conditions
EUROCONTROL SOAM
The Reason Model Organisational Error Chain
Organisational and System Factors
“Unsafe
Acts”
Latent Conditions (adapted from Reason, 1990)
ActiveFailures
Contextual Conditions Human
Involvement
Limited window/sof opportunity
Absent or Failed Barriers
ACCIDENT
People, Task, Environment
EUROCONTROL SOAM
Accident “Causes” A man has a bad argument with his wife. He storms out of the house to the nearest bar and
drinks four whiskies. He then decides to go for a drive. It is night-time, there is a skim of snow on the
ground, and the tyres on our victim’s car are smooth.
In rounding a poorly banked curve at excessive speed, the right front tyre blows out, the car leaves the road and is demolished.
What is the cause of the accident?
(Johnston, 1996)
EUROCONTROL SOAM
SOAM Worked Example
Runway Overrun, Bangkok
September 1999
EUROCONTROL SOAM
Accident Summary
On 23 September 1999, at about 2247 local time,a Boeing 747-438 aircraft overran runway 21 Left (21L) while landing at Bangkok International Airport, Thailand.
EUROCONTROL SOAM
Accident Summary
The aircraft sustained substantial damage during the overrun. None of the three flight crew, 16 cabin crew or
391 passengers reported any serious injuries.
The overrun occurred after the aircraftlanded long and aquaplaned on a runway which was
affected by water following very heavy rain.
EUROCONTROL SOAM
SOAM analysis key steps
Review the Facts
Identify the Organisational Factors
Identify the Contextual Conditions
Identify the Human Involvement
Identify the Absent or Failed Barriers
Validate the OFs against the Occurrence
“CHECKQUESTIONS”
HELP TOSORT ANDCLASSIFY
FACTS
EUROCONTROL SOAM
The SHEL Model(after Edwards, 1972)
LIVEWARE
Operators
L
HARDWARE
Equipment, vehicles, tools,controls, switches, levers,
workplace design, seating etc
H
L LLIVEWARE/LIVEWARE
Interface between people.Operators, controllers,
managers, etc
EENVIRONMENT
Site, terrain,weather, roads,
traffic,remoteness
etc
SSOFTWARE
Procedures,checklists,manuals,training
materials,charts etc
First Officer
Other pilots
Captain
PEOPLE
Crew employed flaps 25/ idle reverse landing
configuration
Very heavy rainfall, runway surface
affected by water
Captain awake 21 hours at time of
accident
Importance of reverse thrust as stopping force on
water-affected runways not known
Qantas B747s generally operated in
good weather & toaerodromes with long,good quality runways
FO awake for 19 hoursat the time of the
accidentConfusion after
thrust levers retarded, in high
workload situation
Most pilots not fullyaware about 'aquaplaning'
Crew did not use an adequate risk mgt
strategy for approachand landing
No formal risk assessment conducted when changed landing procedure researched
“Landing on SlipperyRunways” (Boeing
doc) not distributed in Qantas since 1977
Captain & FO quite low levels of flying prior 30
days
No policies or procedures for maintenance of
recency for management pilots
Normal practice to use flaps 25/idle
reverse
Documents unclear (eg., key terms not
well defined)
FO did not fly theaircraft accurately
during final approach
No formal review of new procedures after
'trial' periodAbsence of reverse thrust during
landing roll not noticed, not used
Captain cancelled go-around decision by
retarding thrust levers
SOFTWAREHARDWARE ENVIRONMENT ORGANISATION
Raw Data Collection AC1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway.
Captain did not order ago-around earlier
Recent crew experience using full reverse thrust lacking
No appropriatelydocumented info,
procedures regarding operations on water-
affected runways
No policies, procedures on duty
or work limits for pilots with flying & non-flying duties
Reduced visibility & distraction: rain andwindscreen wipers
High workload situation, distraction or
inexperience
Contaminated runwayissues not covered in recent years during crew endorsement,
promotional or recurrent training
Cost-benefit analysisof new landing
procedure was biased
Gather data relevant to the occurrence
Partial loss of external visual reference due to
heavy rain
Revised approach/ landing procedure introduced in 1996:
flaps 25, idle reverse thrust
Boeing advised that if idle reverse technique is
adopted, it should be the exception
rather than the rule
Most pilots disagreed they had adequate
training on landing on contaminated
runways
Introduction of newlanding procedure
poor
Bangkok runway was resurfaced in 1991
First Officer
Other pilots
Captain
PEOPLE
Crew employed flaps 25/ idle reverse landing
configuration
Very heavy rainfall, runway surface
affected by water
Captain awake 21 hours at time of
accident
Importance of reverse thrust as stopping force on
water-affected runways not known
Qantas B747s generally operated in
good weather & toaerodromes with long,good quality runways
FO awake for 19 hoursat the time of the
accidentConfusion after
thrust levers retarded, in high
workload situation
Most pilots not fullyaware about 'aquaplaning'
Crew did not use an adequate risk mgt
strategy for approachand landing
No formal risk assessment conducted when changed landing procedure researched
“Landing on SlipperyRunways” (Boeing
doc) not distributed in Qantas since 1977
Captain & FO quite low levels of flying prior 30
days
No policies or procedures for maintenance of
recency for management pilots
Normal practice to use flaps 25/idle
reverse
Documents unclear (eg., key terms not
well defined)
FO did not fly theaircraft accurately
during final approach
No formal review of new procedures after
'trial' periodAbsence of reverse thrust during
landing roll not noticed, not used
Captain cancelled go-around decision by
retarding thrust levers
SOFTWAREHARDWARE ENVIRONMENT
Raw Data Refinement
Captain did not order ago-around earlier
Recent crew experience using full reverse thrust lacking
No appropriatelydocumented info,
procedures regarding operations on water-
affected runways
No policies, procedures on duty
or work limits for pilots with flying & non-flying duties
Reduced visibility & distraction: rain andwindscreen wipers
High workload situation, distraction or
inexperience
Contaminated runwayissues not covered in recent years during crew endorsement,
promotional or recurrent training
Cost-benefit analysisof new landing
procedure was biased
Partial loss of external visual reference due to
heavy rain
Revised approach/ landing procedure introduced in 1996:
flaps 25, idle reverse thrust
Boeing advised that if idle reverse technique is
adopted, it should be the exception
rather than the rule
Most pilots disagreed they had adequate
training on landing on contaminated
runways
Introduction of newlanding procedure
poor
Bangkok runway was resurfaced in 1991
Sort out the non-contributing facts of the investigation
Boeing advised that if idle reverse technique is
adopted, it should be the exception
rather than the rule
Bangkok runway was resurfaced in 1991
ORGANISATION
AC1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway.
First Officer
Other pilots
Captain
PEOPLE
Crew employed flaps 25/ idle reverse landing
configuration
Very heavy rainfall, runway surface
affected by water
Captain awake 21 hours at time of
accident
Importance of reverse thrust as stopping force on
water-affected runways not known
Qantas B747s generally operated in
good weather & toaerodromes with long,good quality runways
FO awake for 19 hoursat the time of the
accidentConfusion after
thrust levers retarded, in high
workload situation
Most pilots not fullyaware about 'aquaplaning'
Crew did not use an adequate risk mgt
strategy for approachand landing
No formal risk assessment conducted when changed landing procedure researched
“Landing on SlipperyRunways” (Boeing
doc) not distributed in Qantas since 1977
Captain & FO quite low levels of flying prior 30
days
No policies or procedures for maintenance of
recency for management pilots
Normal practice to use flaps 25/idle
reverse
Documents unclear (eg., key terms not
well defined)
FO did not fly theaircraft accurately
during final approach
No formal review of new procedures after
'trial' periodAbsence of reverse thrust during
landing roll not noticed, not used
Captain cancelled go-around decision by
retarding thrust levers
SOFTWAREHARDWARE ENVIRONMENT
Raw Data Refinement
Captain did not order ago-around earlier
Recent crew experience using full reverse thrust lacking
No appropriatelydocumented info,
procedures regarding operations on water-
affected runways
No policies, procedures on duty
or work limits for pilots with flying & non-flying duties
Reduced visibility & distraction: rain andwindscreen wipers
High workload situation, distraction or
inexperience
Contaminated runwayissues not covered in recent years during crew endorsement,
promotional or recurrent training
Cost-benefit analysisof new landing
procedure was biased
Partial loss of external visual reference due to
heavy rain
Revised approach/ landing procedure introduced in 1996:
flaps 25, idle reverse thrust
Most pilots disagreed they had adequate
training on landing on contaminated
runways
Introduction of newlanding procedure
poor
Use the remaining factors to build the Analysis chart
ORGANISATION
AC1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway.
First Officer
Other pilots
Captain
PEOPLE
Crew employed flaps25/ idle reverse landing
configuration
Captain awake 21hours at time of
accident
Importance ofreverse thrust asstopping force on
water-affectedrunways not known
Qantas B747sgenerally operated in
good weather & toaerodromes with long,good quality runways
FO awake for 19 hoursat the time of the
accidentConfusion after
thrust leversretarded, in high
workload situation
Most pilots not fullyaware about'aquaplaning'
Crew did not use anadequate risk mgt
strategy for approachand landing
No formal riskassessment conductedwhen changed landingprocedure researched
“Landing on SlipperyRunways” (Boeing
doc) not distributed inQantas since 1977
Captain & FO quite lowlevels of flying prior 30
days
No policies orprocedures formaintenance of
recency formanagement pilots
Normal practice touse flaps 25/idle
reverse
Documents unclear(eg., key terms not
well defined)
FO did not fly theaircraft accurately
during final approach
No formal review ofnew procedures after
'trial' period
Absence of reversethrust during
landing roll notnoticed, not used
Captain cancelled go-around decision by
retarding thrust levers
SOFTWAREHARDWARE ENVIRONMENT
Raw Data QF1 overruns runway at Bangkok after landing long,recent heavy rainfall, and water on runway.
Captain did not order ago-around earlier
Recent crewexperience using fullreverse thrust lacking
No appropriatelydocumented info,
procedures regardingoperations on water-
affected runways
No policies,procedures on duty
or work limits forpilots with flying &non-flying duties
Reduced visibility &distraction: rain andwindscreen wipers
High workloadsituation, distraction
or inexperience
Contaminated runwayissues not covered inrecent years duringcrew endorsement,
promotional orrecurrent training
Cost-benefit analysisof new landing
procedure was biased
Partial loss of externalvisual reference due to
heavy rain
Revised approach/landing procedureintroduced in 1996:
flaps 25, idle reversethrust
Most pilots disagreedthey had adequatetraining on landingon contaminated
runways
Introduction of newlanding procedure
poor
ORGANISATION
Very heavy rainfall,runway surface
affected by water
ACCIDENTABSENT OR FAILED BARRIERS
HUMANINVOLVEMENT
CONTEXTUALCONDITIONS
ORGANISATIONAL FACTORS
Very heavy rainfall, runway surface
affected by water
Building the Analysis Chart
Very heavy rainfall, runway surface
affected by water
Very heavy rainfall, runway surface
affected by water
?
Very heavy rainfall, runway surface
affected by water
?
EUROCONTROL SOAM
Absent or Failed Barriers
Describe the “last minute” measures which failed or were missing, and therefore did not prevent the accident
Check Question:
“Does the item describe a work procedure, aspect of human awareness, physical obstacle, warning or control system, or protection measure designed to prevent an occurrence or lessen its consequences?”
EUROCONTROL SOAM
Human Involvement
Describe the errors or violations (actions or omissions) by operators at the scene which “triggered” the accident
Check Question:
“Does the item describe an action or non-action (error or violation) that immediately contributed to the occurrence?”
First Officer
Other pilots
PEOPLE
Captain awake 21hours at time of
accident
Importance ofreverse thrust asstopping force on
water-affectedrunways not known
Qantas B747sgenerally operated in
good weather & toaerodromes with long,good quality runways
FO awake for 19 hoursat the time of the
accidentConfusion after
thrust leversretarded, in high
workload situation
Most pilots not fullyaware about
'aquaplaning'
Crew did not use anadequate risk mgt
strategy for approachand landing
No formal riskassessment conductedwhen changed landingprocedure researched
“Landing on SlipperyRunways” (Boeing
doc) not distributed inQantas since 1977
Captain & FO quite lowlevels of flying prior 30
days
No policies orprocedures formaintenance of
recency formanagement pilots
Normal practice touse flaps 25/idle
reverse
Documents unclear(eg., key terms not
well defined)
FO did not fly theaircraft accurately
during final approach
No formal review ofnew procedures after
'trial' period
Absence of reversethrust during
landing roll notnoticed, not used
Captain cancelled go-around decision by
retarding thrust levers
SOFTWAREHARDWARE ENVIRONMENT ORGANISATION
Raw Data QF1 overruns runway at Bangkok after landing long,recent heavy rainfall, and water on runway.
Captain did not order ago-around earlier
Recent crewexperience using fullreverse thrust lacking
No appropriatelydocumented info,
procedures regardingoperations on water-
affected runways
No policies,procedures on duty
or work limits forpilots with flying &non-flying duties
Reduced visibility &distraction: rain andwindscreen wipers
High workloadsituation, distraction
or inexperience
Contaminated runwayissues not covered inrecent years duringcrew endorsement,
promotional orrecurrent training
Cost-benefit analysisof new landing
procedure was biased
Partial loss of externalvisual reference due to
heavy rain
Revised approach/landing procedureintroduced in 1996:
flaps 25, idle reversethrust
Most pilots disagreedthey had adequatetraining on landingon contaminated
runways
Introduction of newlanding procedure
poor
Very heavy rainfall,runway surface
affected by water
ACCIDENTABSENT OR FAILED BARRIERS
HUMANINVOLVEMENT
CONTEXTUALCONDITIONS
ORGANISATIONAL FACTORS
Building the Analysis Chart
Crew employed flaps 25/ idle reverse landing
configuration
Crew employed flaps25/ idle reverse landing
configuration
Crew employed flaps 25/ idle reverse landing
configuration
Very heavy rainfall, runway surface
affected by water
Crew employed flaps 25/ idle reverse landing
configuration
?
EUROCONTROL SOAM
Contextual Conditions
Describe the context of the event ~ the conditions existing immediately prior to, or at the time of the accident
Check Question:
“Does the item describe an aspect of the workplace, local organisational climate, or a person’s attitudes, personality, performance limitations, physiological or emotional state that helps explain their actions?”
EUROCONTROL SOAM
Organisational Factors
Describe the organisational and system factors (failures) which created, or allowed, the prevailing contextual conditions
Check Question:
“Does the item describe an aspect of an organisation’s culture, systems, processes or decision-making that existed before the occurrence and which resulted in the contextual conditions or allowed those conditions to continue?”
•Aircraft overranrunway afterlanding long
•No serious injuries(391 pax, 19 crew)
•Potential for moreserious outcome
•Aircraft repair cost:$100,000,000 (?)
•Damage tocompanyreputation
Very heavy rainfall, runwaysurface affected by water
Crew not aware of criticalimportance of reverse thrust
as stopping force onwater-affected runways
FO awake for 19 hoursat the time of the accident
CM No formal review of newprocedures after 'trial' period
Most pilots not fullyaware about 'aquaplaning'
PP No appropriatelydocumented info,
procedures re operations onwater-affected runways
WM No policies, procedures onduty or work limits for pilots
with flying & non-flying duties
PP Regulationscovering emergency
procedures &EP training
were deficient
AC CASAsurveillance ofairline flight
operations deficient
Recent crew experience usingfull reverse thrust lacking
RM No formal risk assessmentconducted when changed
landing procedure researched
SOAM ChartAircraft Accident Boeing 747-438
Bangkok, ThailandSeptember 1999
ACCIDENTABSENT OR
FAILEDBARRIERS
HUMANINVOLVEMENT
CONTEXTUALCONDITIONS
ORGANISATIONAL FACTORS
OTHER SYSTEM FACTORS
First Officerdid not fly the
aircraft accuratelyduring the
final approach
Captain cancelledgo-around decision
by retardingthe thrust levers
New 1996 approach/ landingprocedure inappropriate
CO Documents unclear (eg., key terms not well defined)
CO “Landing on SlipperyRunways” (Boeing doc) not
distributed in Qantas since 1977
Absence ofreverse thrustduring landingroll not noticed,reverse thrust
not used
Flight crew did notuse an adequate
risk managementstrategy forapproach
and landing
Reduced visibility & distraction: rain andwindscreen wipers
Qantas B747s generallyoperated in good weather
& to aerodromes with long,good quality runways
Captain awake 21 hoursat time of accident
High workload situation
TR Contaminated runwayissues not covered during crewendorsement, promotional or
recurrent training in recent years
Captain & FO quite lowlevels of flying prior 30 days
WM No policies or proceduresfor maintenance of recency
for management pilots
Normal practice to useflaps 25/idle reverse
CM Introduction of newlanding procedure poor
CG Cost-benefit analysisof new landing procedure
was biased
PP Regulationscovering
contaminated runwayoperations deficient
Crew employedflaps 25/idle
reverse landingconfiguration
Captain did notorder a go-
around earlier
Landingprocedure
inappropriate
Crew ResourceManagement
deficient
OC Mgt decisions informal,“intuitive”, “personality-driven”
EUROCONTROL SOAM
Provide recommendationsthat will preventrecurrence of this scenario
Recommendations shouldbe directed to theresponsible position,and must addressall identified:
1 Absent or Failed Barriers2 Organisational Factors
Recommendations
•Aircraft overranrunway afterlanding long
•No serious injuries(391 pax, 19 crew)
•Potential for moreserious outcome
•Aircraft repair cost:$100,000,000 (?)
•Damage tocompanyreputation
Very heavy rainfall, runwaysurface affected by water
Crew not aware of criticalimportance of reverse thrust
as stopping force onwater-affected runways
FO awake for 19 hoursat the time of the accident
CM No formal review of newprocedures after 'trial' period
Most pilots not fullyaware about 'aquaplaning'
PP No appropriatelydocumented info,
procedures re operations onwater-affected runways
WM No policies, procedures onduty or work limits for pilots
with flying & non-flying duties
PP Regulationscovering emergency
procedures &EP training
were deficient
AC CASAsurveillance ofairline flight
operations deficient
Recent crew experience usingfull reverse thrust lacking
RM No formal risk assessmentconducted when changed
landing procedure researched
SOAM ChartAircraft Accident Boeing 747-438
Bangkok, ThailandSeptember 1999
ACCIDENTABSENT OR
FAILEDBARRIERS
HUMANINVOLVEMENT
CONTEXTUALCONDITIONS
ORGANISATIONAL FACTORS
OTHER SYSTEM FACTORS
First Officerdid not fly the
aircraft accuratelyduring the
final approach
Captain cancelledgo-around decision
by retardingthe thrust levers
New 1996 approach/ landingprocedure inappropriate
CO Documents unclear (eg., key terms not well defined)
CO “Landing on SlipperyRunways” (Boeing doc) not
distributed in Qantas since 1977
Absence ofreverse thrustduring landingroll not noticed,reverse thrust
not used
Flight crew did notuse an adequate
risk managementstrategy forapproach
and landing
Reduced visibility & distraction: rain andwindscreen wipers
Qantas B747s generallyoperated in good weather
& to aerodromes with long,good quality runways
Captain awake 21 hoursat time of accident
High workload situation
TR Contaminated runwayissues not covered during crewendorsement, promotional or
recurrent training in recent years
Captain & FO quite lowlevels of flying prior 30 days
WM No policies or proceduresfor maintenance of recency
for management pilots
Normal practice to useflaps 25/idle reverse
CM Introduction of newlanding procedure poor
CG Cost-benefit analysisof new landing procedure
was biased
PP Regulationscovering
contaminated runwayoperations deficient
Crew employedflaps 25/idle
reverse landingconfiguration
Captain did notorder a go-
around earlier
Landingprocedure
inappropriate
Crew ResourceManagement
deficient
OC Mgt decisions informal,“intuitive”, “personality-driven”
CM No formal review of newprocedures after 'trial' period
PP No appropriatelydocumented info,
procedures re operations onwater-affected runways
WM No policies, procedures onduty or work limits for pilots
with flying & non-flying duties
PP Regulationscovering emergency
procedures &EP training
were deficient
AC CASAsurveillance ofairline flight
operations deficient
RM No formal risk assessmentconducted when changed
landing procedure researched
CO Documents unclear (eg., key terms not well defined)
CO “Landing on SlipperyRunways” (Boeing doc) not
distributed in Qantas since 1977
TR Contaminated runwayissues not covered during crewendorsement, promotional or
recurrent training in recent years
WM No policies or proceduresfor maintenance of recency
for management pilots
CM Introduction of newlanding procedure poor
CG Cost-benefit analysisof new landing procedure
was biased
PP Regulationscovering
contaminated runwayoperations deficient
OC Mgt decisions informal,“intuitive”, “personality-driven”
Absence ofreverse thrustduring landingroll not noticed,reverse thrust
not used
Landingprocedure
inappropriate
Crew ResourceManagement
deficient
EUROCONTROL SOAM
Questions?
EUROCONTROL SOAM