emerging results from the “safer” project
DESCRIPTION
Emerging Results From the “SAFER” Project. Dr. Wayne A. Dornan Middle Tennessee State University Murfreesboro, TN. Collaborators. Dr. Paul Craig (Principal Investigator) Dr. John Bertrand (Co-Principal Investigator) Steve Gossett Kim Thorsby. - PowerPoint PPT PresentationTRANSCRIPT
Emerging Results From the Emerging Results From the “SAFER” Project“SAFER” Project
Dr. Wayne A. Dornan Dr. Wayne A. Dornan Middle Tennessee State Middle Tennessee State
UniversityUniversityMurfreesboro, TN Murfreesboro, TN
CollaboratorsCollaborators
Dr. Paul Craig (Principal Dr. Paul Craig (Principal Investigator)Investigator)
Dr. John Bertrand (Co-Principal Dr. John Bertrand (Co-Principal Investigator) Investigator) Steve GossettSteve GossettKim ThorsbyKim Thorsby
PILOT ERROR PILOT ERROR CONTINUES TO PLAGUE CONTINUES TO PLAGUE
THE INDUSTRYTHE INDUSTRY
Contributing FactorsContributing Factors
Pilots Missing a Problem Pilots Missing a Problem
Recognizing a Problem to LateRecognizing a Problem to Late
Misidentifying a Problem Misidentifying a Problem
Not Recognizing that there is an Not Recognizing that there is an Immediate Need to Deal with the ProblemImmediate Need to Deal with the Problem
Not Utilizing Their ResourcesNot Utilizing Their Resources
Another Contributing Another Contributing FactorFactor
In aviation, the importance of critical In aviation, the importance of critical decision making skills decision making skills can notcan not be over- be over-emphasizedemphasizedIndeed, the safety of each flight depends Indeed, the safety of each flight depends on it !on it !
Factors Influencing Critical Factors Influencing Critical Decision Making SkillsDecision Making Skills
One possible contributing factor something One possible contributing factor something psychologists have known for years.psychologists have known for years.
Humans consistently are found not to give enough Humans consistently are found not to give enough importance to new evidenceimportance to new evidence
Can you think of an example? Can you think of an example? Inadvertent IMC (176 seconds to live)Inadvertent IMC (176 seconds to live)
Effective Pilot Training Effective Pilot Training Programs Make Pilots Aware of Programs Make Pilots Aware of
this Phenomenonthis Phenomenon
The Decide Model The Decide Model
Was developed to help pilots organize Was developed to help pilots organize their thoughts to prevent overlooking their thoughts to prevent overlooking factors that may be important factors that may be important
Recent Issues That are Emerging Recent Issues That are Emerging
““Out-of-the-Loop” phenomenon Out-of-the-Loop” phenomenon
ComplacencyComplacency
Lack of Knowledge of Automation Lack of Knowledge of Automation
Poor Situational AwarenessPoor Situational Awareness
Poor Situational AssessmentPoor Situational Assessment
HOW DO THE AIRLINES HOW DO THE AIRLINES DEAL WITH THIS ?DEAL WITH THIS ?
MEMORY ITEMS MEMORY ITEMS
CHECKLISTSCHECKLISTS
SCENARIO BASED TRAININGSCENARIO BASED TRAINING
““FITS”FITS”FAA Industry Training StandardsFAA Industry Training Standards
The initiative has been guided by a visionary group The initiative has been guided by a visionary group of FAA administrators, educators, and industry of FAA administrators, educators, and industry
““SAFER”SAFER”
SSATS ATS AAerospaceerospaceFFlight light EEducation ducation RResearchesearch
NASA Research Cooperative AgreementNASA Research Cooperative AgreementNCCI - 03032NCCI - 03032
“FAA Industry Training StandardsScenario Based
Private / Instrument Syllabus forTechnically Advanced Piston Aircraft”
was approved by FAA Headquartersin Washington D.C. and at the Nashville FAA Flight Standards
District Office, as a part of MTSU’s existingAir Agency Certificate
FAA Exemption 8456 to 61.65(a)(1)
Sixteen MTSU Sixteen MTSU students began their students began their flight training in flight training in “glass” in DA-40 “glass” in DA-40 aircraft using the aircraft using the FAA-approved FAA-approved training program in training program in August 2004August 2004
Private Pilot BottlenecksPrivate Pilot BottlenecksArchival data 1999-2004Archival data 1999-2004
00.5
11.5
22.5
33.5
44.5
Target TimeAverage Time
Instrument Pilot BottlenecksInstrument Pilot BottlenecksArchival data 1999-2004Archival data 1999-2004
0
1
2
3
4
5
6
7
IFR1 IFR4 IFR7 IFR10 IFR13 IFR16 IFR19 IFR22 IFR25 IFR28
Target Time
Average Time
Setbacks ComparisonSetbacks ComparisonCaution: Preliminary data using small numbersCaution: Preliminary data using small numbers
0
24
68
1012
14
Start Presolo Pvt X-C IFR
SAFER
Traditional
Traditional SAFER Pre Solo 77 of 449 17.1% 59 of 97 60.8% Pvt & X-C 169 of 449 37.6% 15 of 97 15.4% Instrument 203 of 449 45.2% 23 of 97 23.7%
The results are still The results are still preliminary! preliminary!
However, early data tends to However, early data tends to indicate that pilots have fewer indicate that pilots have fewer
setbacks over the entire VFR/IFR setbacks over the entire VFR/IFR training using new technology training using new technology
and a scenario-based syllabusand a scenario-based syllabus
The first student in our second cohort The first student in our second cohort passed on her combined check ride on passed on her combined check ride on
June 15June 15thth , 2005 , 2005
She had a total of 55 airplane hours !She had a total of 55 airplane hours !
IF WE ARE GOING TO BE IF WE ARE GOING TO BE CERTIFICATING PILOTS WITH CERTIFICATING PILOTS WITH
FEWER HOURSFEWER HOURS
WHAT KIND OF PILOTS WILL THEY WHAT KIND OF PILOTS WILL THEY
BE ?BE ?
PHASE TWOPHASE TWO“ “ THE EFFECTS OF A “FITS” THE EFFECTS OF A “FITS” TRAINING PROGRAM THAT TRAINING PROGRAM THAT
EMPHASIZES SCENARIO BASED EMPHASIZES SCENARIO BASED FLYING ON PILOT DECISION FLYING ON PILOT DECISION
MAKING SKILLS “ MAKING SKILLS “
METHODOLOGYMETHODOLOGY
16 Students enrolled in the “SAFER” 16 Students enrolled in the “SAFER” project: COMBINED GROUPproject: COMBINED GROUP24 Pilots who had obtained their 24 Pilots who had obtained their instrument rating within the last 3 months instrument rating within the last 3 months TRADITIONAL GROUPTRADITIONAL GROUP
METHOLODOLOGY METHOLODOLOGY
The “combined” group were administered The “combined” group were administered pencil and paper tests throughout their pencil and paper tests throughout their flight training flight training
The “traditional group were administered The “traditional group were administered pencil and paper tests within three months pencil and paper tests within three months following the successful completion of their following the successful completion of their check ride check ride
METHODOLOGYMETHODOLOGYAll “SAFER” students also enrolled in a ground class that All “SAFER” students also enrolled in a ground class that combined both private pilot and instrument knowledgecombined both private pilot and instrument knowledgeThis class placed a major emphasis on This class placed a major emphasis on Automation (e.g. mode awareness, automation traps)Automation (e.g. mode awareness, automation traps)Situational AwarenessSituational AwarenessGPS technologyGPS technologyGPS programming skillsGPS programming skillsCBT using real flight scenario’s CBT using real flight scenario’s Critical thinking skills using NTSB reports of fatal aircraft Critical thinking skills using NTSB reports of fatal aircraft accidents as examplesaccidents as examples
DEMOGRAPHICSDEMOGRAPHICS
AGE: AGE: CombinedCombined 19.6 19.6 ++ 1.6 1.6
TraditionalTraditional 20.7 + 1.720.7 + 1.7
DEMOGRAPHICSDEMOGRAPHICS
Total Airplane Hours:Total Airplane Hours:CombinedCombined 90.8 90.8 ++ 5.0 *** 5.0 ***TraditionalTraditional 176.8 176.8 ++ 12 12
Total Flight Hours: Total Flight Hours: CombinedCombined 90.8 90.8 ++ 5.0 *** 5.0 ***TraditionalTraditional 188.75 188.75 ++ 10 10
*** = p <0.01*** = p <0.01
DEMOGRAPHICSDEMOGRAPHICS
Total Instrument Hours:Total Instrument Hours:CombinedCombined 44.3 44.3 ++ 2.0 2.0TraditionalTraditional 38.3 38.3 ++ 3.0 3.0
Total Actual Hours: Total Actual Hours: CombinedCombined 6.1 6.1 ++ 1.8 1.8TraditionalTraditional 4.6 4.6 ++ 0.7 0.7
DEMOGRAPHICSDEMOGRAPHICS
Total PIC Hours:Total PIC Hours:
CombinedCombined 6.1 6.1 ++ 0.6 *** 0.6 ***
TraditionalTraditional 125.6 125.6 ++ 14.0 14.0
*** = p < 0.01*** = p < 0.01
Personal IFR Comfort Personal IFR Comfort Questionnaire Questionnaire
How comfortable are you to fly alone in How comfortable are you to fly alone in the IFR environment? the IFR environment?
How comfortable are you to fly alone in How comfortable are you to fly alone in IMC ? IMC ?
How comfortable are you to shoot an ILS How comfortable are you to shoot an ILS approach to minimums ? approach to minimums ?
What are your “personal minimums” ? What are your “personal minimums” ?
ResponsesResponses
1 = Not comfortable 1 = Not comfortable
2 = Somewhat comfortable2 = Somewhat comfortable
3 = comfortable3 = comfortable
4 = very comfortable4 = very comfortable
5 = Absolutely comfortable, no problem!5 = Absolutely comfortable, no problem!
RESULTSRESULTS
Not Comfortable flying alone in the IFR Not Comfortable flying alone in the IFR environmentenvironment
Traditional 18 %Traditional 18 %
Combined 0 % ****Combined 0 % ****
*** = p < 0.01*** = p < 0.01
RESULTSRESULTS
Not Comfortable flying alone in IMCNot Comfortable flying alone in IMC
Traditional 38 %Traditional 38 %
Combined 14 % ****Combined 14 % ****
*** = p < 0.01*** = p < 0.01
RESULTSRESULTS
Not Comfortable shooting an ILS to Not Comfortable shooting an ILS to minimumsminimums
Traditional 16 %Traditional 16 %
Combined 0 % ****Combined 0 % ****
*** = p < 0.01*** = p < 0.01
RESULTSRESULTS
Would you feel comfortable using a Would you feel comfortable using a GPS when flying IMCGPS when flying IMC
Traditional = Yes (48 %) ***Traditional = Yes (48 %) ***
Combined = Yes (100 %) Combined = Yes (100 %)
**** = p < 0.01 **** = p < 0.01
RESULTSRESULTS
What are your personal minimums ? What are your personal minimums ?
Visibility: Visibility:
Traditional 2.07 + .18Traditional 2.07 + .18
Combined 3.50 + .06 ****Combined 3.50 + .06 ****
*** = p < 0.01*** = p < 0.01
RESULTSRESULTS
What are your personal minimums ? What are your personal minimums ?
Clouds: Clouds:
Traditional 1,350 + 223Traditional 1,350 + 223
Combined 2,100 + 585 ****Combined 2,100 + 585 ****
*** = p < 0.01*** = p < 0.01
RESULTSRESULTS
What are your personal minimums ? What are your personal minimums ?
Visibility: Never Thought about it Visibility: Never Thought about it !!
Traditional 68 %Traditional 68 %
Combined 18 % ****Combined 18 % ****
*** = p < 0.01*** = p < 0.01
Self-Efficacy: TAA Self-Efficacy: TAA
I have confidence in my ability to fly a TAAI have confidence in my ability to fly a TAA1 = to no extent1 = to no extent 5 = to a great extent 5 = to a great extent
Students were assessed at: Beginning of Students were assessed at: Beginning of flight training, Mid-flight training, and at flight training, Mid-flight training, and at their completiontheir completion
RESULTSRESULTS
Beginning 4 – 5 = 58 % *** (3.70)Beginning 4 – 5 = 58 % *** (3.70)Mid-training 4 – 5 = 100 % (4.75)Mid-training 4 – 5 = 100 % (4.75)EndEnd 4 – 5 = 100 % (4.75)4 – 5 = 100 % (4.75)
Did not complete the training: 3.10 *Did not complete the training: 3.10 *Did complete the training: 3.70Did complete the training: 3.70
*** = p < 0.01*** = p < 0.01* = p > 0.05 * = p > 0.05
Self-Efficacy: Single Pilot Resource Self-Efficacy: Single Pilot Resource Management (SRM)Management (SRM)
I have confidence in my ability to perform SRMI have confidence in my ability to perform SRM
1 = to no extent1 = to no extent 5 = to a great extent 5 = to a great extent
Students were assessed at: Beginning of flight Students were assessed at: Beginning of flight training, Mid-flight training, and at their training, Mid-flight training, and at their completioncompletion
RESULTSRESULTS
Beginning 4 – 5 = 65 % *** (3.80)Beginning 4 – 5 = 65 % *** (3.80)
Mid-training 4 – 5 = 100 % (4.75)Mid-training 4 – 5 = 100 % (4.75)
EndEnd 4 – 5 = 80 % (4.16)4 – 5 = 80 % (4.16)
Did not complete the training: 2.70 *Did not complete the training: 2.70 *
Did complete the training: 3.80Did complete the training: 3.80
*** = p < 0.01*** = p < 0.01
* = p > 0.05 * = p > 0.05
Other Data Analysis Pending Other Data Analysis Pending
Any relationship between cognitive Any relationship between cognitive demands and “choke points” demands and “choke points”
Any relationship between self-efficacy and Any relationship between self-efficacy and quality of TAA pilotquality of TAA pilot
Follow up study at 3-6 months Follow up study at 3-6 months
SUMMARYSUMMARY
1)1) Are results indicate that pilots have fewer Are results indicate that pilots have fewer setbacks over the entire VFR/IFR training using setbacks over the entire VFR/IFR training using
the FITS syllabus the FITS syllabus 2)2) Pilots trained using a combined private / Pilots trained using a combined private /
instrument ground school that focuses on instrument ground school that focuses on decision making, automation issues, and decision making, automation issues, and
navigational issues in TAA aircraft, in navigational issues in TAA aircraft, in combination with a FITS flight training syllabus combination with a FITS flight training syllabus
emphasizing scenario based flying, are:emphasizing scenario based flying, are:
1)1) More comfortable with their automationMore comfortable with their automation2)2) More comfortable with their IFR skillsMore comfortable with their IFR skills
3)3) More conservative with IFR decision making More conservative with IFR decision making
QUESTIONSQUESTIONS