a glimpse at archie: the losa archive ashleigh merritt, ph.d. the university of texas human factors...
TRANSCRIPT
A Glimpse at Archie: The A Glimpse at Archie: The LOSA ArchiveLOSA Archive
Ashleigh Merritt, Ph.D.Ashleigh Merritt, Ph.D.
The University of Texas Human The University of Texas Human Factors Research Project (UT)Factors Research Project (UT)
Third ICAO/IATA LOSA/TEM ConferenceThird ICAO/IATA LOSA/TEM Conference
September 13-14, 2005, Kuala LumpurSeptember 13-14, 2005, Kuala Lumpur
What is Archie?What is Archie?
““Archie” is the LOSA Archive Archie” is the LOSA Archive A database of de-identified airline data A database of de-identified airline data
which is maintained and updated by UTwhich is maintained and updated by UT
Airlines that do a LOSA with The LOSA Airlines that do a LOSA with The LOSA Collaborative contribute their data to Collaborative contribute their data to Archie for research purposes Archie for research purposes
How old is Archie?How old is Archie?
Five LOSAs in five years to Five LOSAs in five years to develop the methodology develop the methodology
Data in the Archive starts in Data in the Archive starts in 20002000
Had to wait for Archie to grow Had to wait for Archie to grow before doing any industry-before doing any industry-wide analyses.wide analyses. If we think of Archie as a dog, he If we think of Archie as a dog, he
would be a young bloodhoundwould be a young bloodhound
How big is Archie?How big is Archie?
As of August 2005, Archie has data fromAs of August 2005, Archie has data from 20 airlines (including 3 repeat LOSAs)20 airlines (including 3 repeat LOSAs) 4800 flights4800 flights
4800 flight narratives written by trained 4800 flight narratives written by trained observersobservers
4800 ratings of TEM countermeasure 4800 ratings of TEM countermeasure performanceperformance
17,500+ threats17,500+ threats 12,500+ errors12,500+ errors 2,400+ undesired aircraft states2,400+ undesired aircraft states
Archie’s Attributes 2004Archie’s Attributes 2004
1.1. Archie is loyal and Archie is loyal and faithful faithful
2.2. Archie alerts of dangerArchie alerts of danger
3.3. Archie is a tracker Archie is a tracker
4.4. Archie plays well with Archie plays well with othersothers
2005 Update on Archie2005 Update on Archie
Archie has remained loyal and Archie has remained loyal and faithful to its aviation masterfaithful to its aviation master
No airline has been publicly No airline has been publicly identifiedidentified
No individual pilots can be or ever No individual pilots can be or ever have been identifiedhave been identified
Archie continues to play well Archie continues to play well with otherswith others
2005
2004
Queries from / data-Queries from / data-sharing with: sharing with:
NOSS (ATC group)NOSS (ATC group) IATA/ICAO (ITA)IATA/ICAO (ITA) Boeing Boeing Airlines Airlines ASAP ASAP ATSBATSB
2005: Archie is learning to 2005: Archie is learning to tracktrack
Airline threats Airline threats ATC threats ATC threats Intentional Intentional
NoncomplianceNoncompliance
Airline threats: Airline threats: Simple TrackingSimple Tracking
Airline threats include aircraft malfunctions, time Airline threats include aircraft malfunctions, time pressure, errors/distractions/interruptions from pressure, errors/distractions/interruptions from Cabin, Dispatch, Maintenance, Ground, Ramp. Cabin, Dispatch, Maintenance, Ground, Ramp.
Airline threats occur on 2/3 of all flights, adding Airline threats occur on 2/3 of all flights, adding to overall workload. to overall workload.
Range: 55% - 81% across different airlinesRange: 55% - 81% across different airlines Three-quarters of airline threats occur before the Three-quarters of airline threats occur before the
aircraft has even left the gate (predeparture).aircraft has even left the gate (predeparture).
TARGETS FOR ENHANCEMENT?TARGETS FOR ENHANCEMENT?
ATC threat chains: ATC threat chains: Complex trackingComplex tracking
10 LOSAs – 2426 Flights – 9450 threats 2350 ATC threats
= ¼ of all threats = about 1 per flight
236 mismanaged ATC threats = 30% of all mismanaged threats 10% of all ATC threats were
mismanaged
Mismanaged ATC threatsMismanaged ATC threats
2426 Flights – 9450 threats
236 mismanaged ATC threats
Challenging clearances
(53%)
Runway Changes (14%)
Language difficulty (8%)
Similar call signs (5%)
ATC error (7%)
ATC threats -> ATC threats -> ErrorsErrors
2426 Flights – 9450 threats
236 mismanaged ATC threats
280 errors
Automation errors (20%) – wrong
MCP/FCU altitude setting dialed
Communication errors (27%) -
misinterpret ATC instructions
Aircraft handling (24%) –
unintentional speed, lateral, vertical
deviations
Cross-verification errors (7%)
Ground navigation errors (6%)
ATC threats -> ATC threats -> Errors -> Errors -> UASUAS
2426 Flights – 9450 threats
236 mismanaged ATC threats
280 errors
56 Automation errors
76 Communication errors
67 Aircraft handling errors
19 Cross-verification errors
16 Ground navigation errors
2UAS
58UAS
5UAS
9 UAS
21UAS
ATC threats ATC threats -> UAS-> UAS
2426 Flights – 9450 threats
236 mismanaged ATC threats
280 errors
106 UAS
Lateral deviation 17%
Speed too high 14%
Vertical deviation 10%
Unstable Approach 10%
Continued Landing 9%
Incorrect Automation configuration 9%
Taxiway/runway incursion 6%
Speed too low 6%
Incorrect Aircraft configuration 5% Other 14%
Bottom line: 4% of flights had a
UAS arising from an ATC threat that was
mismanaged. ¾ occurred during
descent/approach/land
Archie tracks intentional Archie tracks intentional noncompliancenoncompliance
Across 22 LOSAs, the average is 40% of Across 22 LOSAs, the average is 40% of flights have one or more noncompliance flights have one or more noncompliance errorserrors
Range: 23% of flights at one airline to 90% (!!) Range: 23% of flights at one airline to 90% (!!) at another at another
Most common noncompliance errors:Most common noncompliance errors: checklist performed from memory / checklist performed from memory /
nonstandard checklist usenonstandard checklist use failure to cross-verify MCP/FCU altitude alerter failure to cross-verify MCP/FCU altitude alerter
changes changes PF makes own MCP/FCU changesPF makes own MCP/FCU changes
Noncompliance correlates Noncompliance correlates with..with..
Based on 22 LOSAs (4800 flts), airlines Based on 22 LOSAs (4800 flts), airlines that have more flights with intentional that have more flights with intentional noncompliance errors also have more noncompliance errors also have more flights with:flights with:
Mismanaged threats (r = .7) Mismanaged threats (r = .7) Mismanaged handling errors (r = .9)Mismanaged handling errors (r = .9) Undesired aircraft states (r = .9)Undesired aircraft states (r = .9) Mismanaged UASs (r =.8)Mismanaged UASs (r =.8)
Archie was shocked…
Archie’s FutureArchie’s FutureArchie is maturing and getting smarter – more LOSAs allow for more analyses at the airline level, which is where the greatest differences are emerging.
Soon a sub-Archive for regional airlines
Archie is getting involved in more industry projects
If LOSA and ASAP overlap, then focusing on LOSA targets should reduce ASAP incidents…
ATC is a big threat to pilots, are pilots a big threat to ATC?
Archie is recognizing his Archie is recognizing his strengthstrength
LOSA is a macro LOSA is a macro approach for Safety approach for Safety Management Systems Management Systems
LOSA detects “gross” LOSA detects “gross” effects – strengths and effects – strengths and weaknesses in the weaknesses in the systemsystem
Management uses the Management uses the informationinformation
Self-corrections, training, Self-corrections, training, procedural changes,procedural changes, … …
If and when certain problems persist, micro If and when certain problems persist, micro approaches can investigate more closely…approaches can investigate more closely…
The University of TexasThe University of TexasHuman Factors Research ProjectHuman Factors Research Project
www.psy.utexas.edu/HumanFactors