amp meeting | february 20, 2014gama | washington, dc g. oda improvement action plan
TRANSCRIPT
AMP Meeting | February 20, 2014 GAMA | Washington, DC
G. ODA Improvement Action Plan
GAMA/FAA Leadership Review | February 19, 2014
• Improving FAA Certification Capacity– Continue to Improve Timeliness of FAA Actions with Expected Flat
or Reduced Budget Levels
• Ensuring ODA Provides Expected Efficiency for FAA & Industry – Beginning ODA Projects in a Timely Manner– Fully Utilizing Available Delegations
• FAA/Industry Collaboration toward Improvements in Certification Process Effectiveness & Efficiency– Strengthen Applicant Processes and Increase Robustness – FAA Resources Focus on Systems Oversight and
Risk Based Level of Project Involvement
Objective
GAMA/FAA Leadership Review | February 19, 2014
ODA Action Plan: Status
• Proper Communications• Unit Member (UM) Selection• Certification Project Initiation• Procedures Manual Revisions• Full & Efficient Use • Use of ODA Data
3
Improvement Category Actions
OnTrack
5
13
7
1
3
21
14 15 16 17 18 19 20
6
8 9 10 11 12
4
2
22
ImplementedClosed
23
GAMA/FAA Leadership Review | February 19, 2014
• Sequencing for ODA (Red)– Revised procedures issued for public comment in 2013– Revised SOP planned for December 2014– Assess after implementation
• Risk Based Resource Targeting for ODA– FAA considering approach for ODA projects
• Consider Allowing CAA Design Organizations to Support ODA
• Standardized UM Training – Suppliers to Multiple ODAs– Industry/FAA to establish joint team
• Enhance ODA Approval Activities Prior to Project Start– Industry work with OMT to implement
ODA Action Plan: Open Items
GAMA/FAA Leadership Review | February 19, 2014
• Order 8100.15A issued June 2011– Streamlined FAA Review Process and Project Start
• Order 8100.15B issued May 2013– Facilitate Full Utilization and Cultural Shift• ODA Recommend Areas for FAA Involvement• FAA Document Reason for Involvement
• Change 1 issued Feb 2014– Certification Plan Improvements
ODA Action Plan: Implementation
GAMA/FAA Leadership Review | February 19, 2014
ODA MetricsCert Plan Response Time (goal 30 days)
Avg. Working Days
Accepted Accepted w/ comment
Not accepted
2012 – 1Q 18 50% 39% 11%
2012 – 2Q 44 55% 11% 34%
2012 – 3Q 22 52% 26% 22%
2012 – 4Q 16 58% 24% 18%
2013 – 1Q 30 58% 31% 11%
2013 – 2Q 52 68% 22% 10%
2013 – 3Q 28 64% 26% 10%
• Considerable improvement from 2011 Avg. of 60-90 Days (industry data)
GAMA/FAA Leadership Review | February 19, 2014
ODA MetricsManual Review Response Time (goal 30 days)
Avg. Working Days
Accepted Accepted w/ comment
Not accepted
2012 – 1Q 24 50% 29% 21%
2012 – 2Q 71 41% 6% 53%
2012 – 3Q 28 65% 25% 10%
2012 – 4Q 37 61% 27% 12%
2013 – 1Q 25 65% 20% 15%
2013 – 2Q 38 78% 7% 15%
2013 – 3Q 24 86% 6% 8%
GAMA/FAA Leadership Review | February 19, 2014
ODA MetricsUM Review Response Time (goal 30 days)
Prescreen Avg. Working Days
Selection Review Avg. Work Days
Concur Further Action Req’d
2012 – 1Q 7 10 82% 18%
2012 – 2Q 3 12 93% 7%
2012 – 3Q 3 10 80% 20%
2012 – 4Q 2 8 88% 12%
2013 – 1Q 7 15 85% 15%
2013 – 2Q N/A 17 88% 12%
2013 – 3Q N/A 8 97% 3%
GAMA/FAA Leadership Review | February 19, 2014
• Trending Shows FAA Meeting Goals (on avg.)• Does Not Reflect Some Experiences/Potential Issues– Large Program Response Time is Much Longer (130 day avg)
– Certain Disciplines Often Remain Open Until Late in Program
• Next Steps– Analyze Turn Around Metrics to Identify Issues
• Examples: First pass yield on cert plans and number >90-days• Industry to Provide Details on Outliers
– Implement UM Appointment Authorization– Assess Effectiveness of ODA Improvements
• Implementation of Key Improvements in January 2014• Industry/FAA Workshop in June 2014
ODA Metrics - Conclusions
GAMA/FAA Leadership Review | February 19, 2014
• The Delegated Organization Inspection Program (DOIP)– Assesses system-level procedures and compliance, – Inspects how the ODA holder manages the ODA unit, and – Evaluates the ODA unit's technical proficiency
• Currently 87 organizations are ODA holders– For the 2-year period between 2011 and 2012: 79 DOIP
Audits were conducted resulting in 799 discrepancies– 31% of discrepancies involve regulatory violations, primarily ODA
holders not complying with their procedures manual– 16% of discrepancies were technical discrepancies found in
design/substantiation data or airworthiness certification packages
ODA Performance – DOIP Results
GAMA/FAA Leadership Review | February 19, 2014
• FAA Continued Focus on Systems Approach to Oversight– Documented Level of Involvement– Restructured FAA Oversight Procedures– Audit Training for FAA personnel (ACPRR Recommendation)
• Industry Commitment to ODA System Performance – Adherence to Procedures and Corrective Action– Strengthen Self-Audit
• Increases Robustness and Confidence in Systems Approach
• Continuing ODA Policy Improvements - Expand Delegation– ICA Functions– Noise and Emissions Functions
Continuing ODA Initiatives
AMP Meeting | February 20, 2014 GAMA | Washington, DC
ODA Improvement – Next Steps
Analysis of ODA Turn Around Metrics Analyze Metrics to Identify Issues and Determine Resolution Process to Address Significant Exceedences (i.e. 60-90 days) Report to GAMA/FAA Leadership Team
Reinforce Importance Program Management CPI Principles – PSP and PSCP
Communications, Program Planning and Accountability Issues Resolution
Consider Actions to(GAMA/FAA PACT)