1

Report of the Controller and Auditor-GeneralReport of the Controller and Auditor-GeneralCivil Aviation Authority: Certification and Civil Aviation Authority: Certification and

Surveillance FunctionsSurveillance Functions

CAA Progress ReportCAA Progress ReportTo To

Select Committee on Transport & Industrial Select Committee on Transport & Industrial RelationsRelations

25 May 200625 May 2006

2

BackgroundBackground

The Auditor-General’s Report completed in The Auditor-General’s Report completed in June 2005.June 2005.

Follows two previous reports completed in Follows two previous reports completed in 1997 and 2000.1997 and 2000.

The relationship between the Office of the The relationship between the Office of the Auditor-General and the CAA can be described Auditor-General and the CAA can be described as open, cooperative and cordial.as open, cooperative and cordial.

3

Audit TimelineAudit Timeline

11stst meeting with the CAA Executive November 2003. meeting with the CAA Executive November 2003.

Audit proper commenced December 2003.Audit proper commenced December 2003.

11stst draft report received November 2004. draft report received November 2004.

CAA provided list of proposed remedial actions CAA provided list of proposed remedial actions

January 2005.January 2005.

22ndnd draft report received March 2005. draft report received March 2005.

Updated list of proposed CAA remedial actions Updated list of proposed CAA remedial actions

provided April 2005.provided April 2005.

Final Report 17 June 2005 and tabled on 21 June Final Report 17 June 2005 and tabled on 21 June

2005.2005.

4

Audit FindingsAudit Findings

This report covers two of the CAA’s core This report covers two of the CAA’s core regulatory functions:regulatory functions: CertificationCertification SurveillanceSurveillance

Some areas of improvement noted from 1997 Some areas of improvement noted from 1997 and 2000, but some persistent areas of and 2000, but some persistent areas of concern as well, particularly with respect to concern as well, particularly with respect to client risk analysis.client risk analysis.

Six key findings and 10 recommendations Six key findings and 10 recommendations have been made in the report.have been made in the report.

5

Notes to FindingsNotes to Findings

A significant proportion of the findings had A significant proportion of the findings had been noted by CAA internal audit enabling a been noted by CAA internal audit enabling a rectification plan to be quickly drawn up rectification plan to be quickly drawn up against the Auditor-General’s against the Auditor-General’s recommendations.recommendations.

At the time of the 1997 and 2000 audits a At the time of the 1997 and 2000 audits a mature risk analysis system simply did not mature risk analysis system simply did not exist. Even now, the systems developed by exist. Even now, the systems developed by the CAA are just reaching a useful level of the CAA are just reaching a useful level of maturity.maturity.

6

Key Findings 1Key Findings 1 Certification of clients in the Airline sector is generally sound,

but needs to be more rigorous in the General Aviation sector.

Significant concerns with the surveillance function noted in 1997, 2000 and this Report, includes the following:

Risk analysis and risk assessment processes need to be more effective.

Risk analysis needs to “feed through” to the surveillance process.

High risk clients need to be appropriately targeted in the terms of both audit depth and frequency.

During audits, Finding Notices are not always raised for non-compliances or non-conformances, and closure of Findings by auditors is not always timely.

7

Key Findings 2Key Findings 2 CAA Inspectors have not logged all their hours worked

on surveillance duties. This affects the resourcing of the CAA and the accuracy of the risk assessment systems.

Because of financial pressure, resource demands and the high cost of specialised technical training, only essential training of CAA staff has been carried out over the last three years.

While CAA internal audits had raised many of the issues noted in the Report, the operational Groups did not always “buy in” to the internal auditors recommendations. Issues raised by internal audit included the following:

Raising Finding Notices where appropriate Consistency in certification and surveillance

activities between operational Groups Operational Group resourcing Logging all time spent on surveillance

activities

8

Recommendation ActionsRecommendation Actions

The 10 Recommendations made by the The 10 Recommendations made by the Auditor-General have resulted in 29 action Auditor-General have resulted in 29 action items for the CAA.items for the CAA.

10 action items are complete or are now 10 action items are complete or are now absorbed into CAA’s day-to-day operational absorbed into CAA’s day-to-day operational business processes.business processes.

The remaining action items have been The remaining action items have been absorbed into major projects that address the absorbed into major projects that address the aspects of the CAA’s core business. aspects of the CAA’s core business.

9

CAA Response to RecommendationsCAA Response to RecommendationsThe CAA has approached the Auditor-General’s The CAA has approached the Auditor-General’s recommendations in the context of its life cycle recommendations in the context of its life cycle

approach to regulating the aviation industry.approach to regulating the aviation industry.

“Life Cycle” approach where Civil Aviation Rules = Safety Boundary

ENTRY CONTROL (CERTIFICATION)

CONTINUEDOPERATION

CORRECTIVEACTIONS

CONFIDENCE

SURVEILLANCE

SUPPORTTO MAINTAIN

GOOD RELATIONS

RULES FOREXIT CONTROL

10

ProjectsProjects

The CAA has commenced three major The CAA has commenced three major projects as follows:projects as follows:

Certification ReviewCertification Review Surveillance ReviewSurveillance Review Risk Assessment and Intervention Risk Assessment and Intervention

These projects address CAA core These projects address CAA core regulatory functions that have not regulatory functions that have not changed substantially since 1995.changed substantially since 1995.

11

Project TimelinesProject Timelines

Surveillance Review Project:Surveillance Review Project: User test June – mid AugustUser test June – mid August Training June - NovemberTraining June - November Technical full functionality mid Technical full functionality mid

DecemberDecember Operational full functionality Operational full functionality

November – mid JanuaryNovember – mid January

12

Project TimelinesProject Timelines

Certification Review ProjectCertification Review ProjectManual functionality November – DecemberManual functionality November – DecemberAutomated functionality mid 2007Automated functionality mid 2007

Risk Assessment & Intervention Risk Assessment & Intervention ProjectProjectComplete early AugustComplete early August

13

Improvement in Industry Safety Improvement in Industry Safety PerformancePerformance

Since the re-organisation in 2000 the CAA has Since the re-organisation in 2000 the CAA has put major effort into building relationships with put major effort into building relationships with the industry and increasing the safety the industry and increasing the safety performance overall.performance overall.

The results of these efforts by both the CAA The results of these efforts by both the CAA and the industry are only now becoming and the industry are only now becoming apparent. apparent.

14

Reduction in Annual Accident RateReduction in Annual Accident Rate

YearYear AccidentsAccidents Hours Hours FlownFlown

Accidents/Accidents/100,000 hrs100,000 hrs

20032003 6969 465,276465,276 14.7614.76

20042004 5959 474,243474,243 12.3712.37

20052005 5252 487,044487,044 10.6810.68

20062006 5050 492,624492,624 10.2210.22

15

This is a This is a 30.77%30.77% reduction in the reduction in the General Aviation accident rate in General Aviation accident rate in

period 2003 - 2006period 2003 - 2006

16

Accident Rates v Social CostAccident Rates v Social Cost

Using accident rates can be misleadingUsing accident rates can be misleading Consequence of accidents can be very Consequence of accidents can be very

differentdifferent

This can be demonstrated by looking at This can be demonstrated by looking at previous information in terms of social costprevious information in terms of social cost

17

Social Cost of GA AccidentsSocial Cost of GA Accidents

YearYear Social CostSocial Cost

20032003 $44.30M44.30M

20042004 $45.10M$45.10M

20052005 $38.60M$38.60M

20062006 $35.43M$35.43M

18

In terms of Social Cost, General Aviation accidents have reduced by 20.02% in the period

2003 - 2006

19

Safety Targets for 2010Safety Targets for 2010

Safety targets have been set in terms of Safety targets have been set in terms of social cost/unit of exposure - $ per seat hoursocial cost/unit of exposure - $ per seat hour

13 safety outcome target groups redefined to 13 safety outcome target groups redefined to range from:range from:

Airline Operations – Large Aeroplanes $0.10 per seat hour$0.10 per seat hour

Private Operations – Sport $20.00 per seat hour$20.00 per seat hour

20

Safety Outcome Targets 2010Safety Outcome Targets 2010Target GroupTarget Group TargetTarget Set-up Est Set-up Est 1 Jul 051 Jul 05 1 Apr 061 Apr 06

Airlines – Large aeroplanesAirlines – Large aeroplanes $0.10$0.10 $0.13$0.13 $0.03$0.03 $0.03$0.03

Airlines – Medium Airlines – Medium aeroplanesaeroplanes

$0.10$0.10 $0.13$0.13 $0.04$0.04

Airlines – Small aeroplanesAirlines – Small aeroplanes $6.50$6.50 $65.18$65.18 $85.91$85.91 $0.00$0.00

Airlines – HelicoptersAirlines – Helicopters $6.50$6.50 $55.46$55.46 $0.00$0.00

Sport TransportSport Transport $13.00$13.00 $113.21$113.21 $11.00$11.00

Commercial Ops AeroplanesCommercial Ops Aeroplanes $6.50$6.50 $6.44$6.44 $21.31$21.31 $21.41$21.41

Commercial HelicoptersCommercial Helicopters $6.50$6.50 $36.76$36.76 $0.07$0.07 $1.87$1.87

Ag Ops AeroplanesAg Ops Aeroplanes $14.00$14.00 $141.90$141.90 $109.05$109.05 $338.53$338.53

Ag Ops HelicoptersAg Ops Helicopters $14.00$14.00 $85.44$85.44 $18.45$18.45 $0.00$0.00

Ag Ops Sport AircraftAg Ops Sport Aircraft $28.00$28.00 $0.00$0.00

Private Ops – AeroplanesPrivate Ops – Aeroplanes $10.00$10.00 $115.51$115.51 $0.22$0.22 $48.99$48.99

Private Ops – HelicoptersPrivate Ops – Helicopters $10.00$10.00 $98.31$98.31 $454.22$454.22

Private Ops – Sport Private Ops – Sport $10.00$10.00 $174.06$174.06 $137.27$137.27 $30.45$30.45

21

Agricultural Operations - AeroplaneAgricultural Operations - AeroplaneAgricultural Operations - Aeroplane

$-

$50.00

$100.00

$150.00

$200.00

$250.00

$300.00

$350.00

$400.00

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Year

Social Cost Target Trend

22

Agricultural Operations - HelicopterAgricultural Operations - HelicopterAgricultural Operations - Helicopter

$ -

$ 50.00

$ 100.00

$ 150.00

$ 200.00

$ 250.00

$ 300.00

$ 350.00

$ 400.00

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Y ear

Social Cost Target Trend

y = -7.1672x + 145.15

Cur r ent tr end

indicates that tar get

l ine wi l l be

inter cepted in the year

2015.

23

Private Operations - HelicopterPrivate Operations - HelicopterPrivate Operations - Helicopter

$-

$50.00

$100.00

$150.00

$200.00

$250.00

$300.00

$350.00

$400.00

$450.00

$500.00

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Year

Social Cost Target Trend

24

ConclusionConclusion

The CAA intends to maintain its current The CAA intends to maintain its current momentum on improving industry momentum on improving industry safety performance while introducing safety performance while introducing better, more efficient and effective better, more efficient and effective operational systems and processes. operational systems and processes.

The CAA Vision: The CAA Vision: ““New Zealand aviation free from safety New Zealand aviation free from safety and security failure and contributing to and security failure and contributing to an integrated, responsive and an integrated, responsive and sustainable transport system.”sustainable transport system.”