csi copyright © 2008 certification services, inc. ascsa 2008: the agnostic hazard 1 the agnostic...

1

CSI

Copyright © 2008 Certification Services, Inc.

aSCSa 2008: The Agnostic Hazard

The agnostic hazard

Frank [email protected]

CERTIFICATION SERVICES, INC.

aSCSa 2008, Canberra

3

CSI



A distinctive contrast

• Aircraft: little or no discretion– Safety assessment and design assurance driven by

comprehensive, transparent standards, notably SAE ARP4761, DO-178B, DO-254, DO-160x

• Publicly owned and operated infrastructure on ground and in space: wide discretion– Safety assessment and design assurance varies

greatly by contract

4

CSI



SAE ARP4761

• Society of Automotive Engineers is active in aerospace standards• “Guidelines and Methods for Conducting the Safety Assessment

Process on Civil Airborne Systems and Equipment”– Functional Hazard Assessment– Preliminary System Safety Assessment– Failure Modes and Effects Analysis– Failure Modes and Effects Summary– Zonal Safety Analysis– Particular Risks Analysis– Common Mode Analysis– System Safety Assessment

5

CSI



Example: Particular Risks Analysis

• Fire• Rotor burst

– Engine

– APU

• High pressure bottles• High pressure air duct• High temp air duct• Leaking fluids

• Hail, ice, snow• Bird strike• Tire burst, flailing tread• Wheel rim release• Lightning strike• HIRF• Flailing shafts• Bulkhead rupture

6

CSI



Publicly owned CNS/ATM

• MIL-STD-882x?• Safety case?• IEEE 12207? MIL-STD-2167A or -498?• CMMI?• Other?

• ADF worth noting: 7001.054, “Airworthiness Design Requirements Manual”, more comprehen-sive and prescriptive than public-sector average

7

CSI



Treatment in practice

• Private sector– Requirements for civil airborne network device– Handling of failure of cockpit display

• Public sector– Use of software-intensive COTS hardware– Handling of UAV crash

8

CSI



Assurance of digital component in airborne data bus needed for dispatch

• HW & SW planning: certification issues, safety assessment, development, verifi-cation, CM, QA, special considerations

• HW & SW verifica-tion: reviews, analyses, testing, inspections

• HW & SW develop-ment: requirements, design, implementa-tion, integration

• HW & SW CM• HW & SW QA• HW & SW cert liaison• HW & SW accomp-

lishment summaries

9

CSI



Private avionics

• Failure of primary display• Prompt FAA response in

Airworthiness Directive– Flight Manual update– Dispatch prohibition– MMEL update– Software change– Functional test– Flight Manual reversion

10

CSI



Public COTS

• Network control aboard International Space Station and Space Shuttle for primary data link between ground and orbit

• Black-box only– Functional testing

– Performance testing

• Later serves as baseline or authoritative reference for CNS/ATM systems

11

CSI



Unintentionally autonomous

• UAV: General Atomics, Predator B– Loss of contact and subsequent crash near

Nogales, Arizona: 25 April 2006– Wingspan: 66 feet (approx. 20 meters)– Weight: 10,000 lb (approx. 4500 kg)– Speed: 220 knots– Ceiling: 50,000 feet (approx. 15,200 meters)– Endurance: 30 hours

12

CSI



Report of theNational Transportation Safety Board

• COTS software• Weekly “lockups”• Two lockups just

before accident flight• Confusing operator

controls (same lever can be engine thrust or camera position, depending on mode)

13

CSI



NTSB recommendations

• Better transponders on UAVs• Communications recorded• Periodic meetings between UAVers and ATC• Manned-aircraft emergency procedures

applied to UAVs• Manned-aircraft reporting requirements

applied to UAVs• FAA to consider recommendations

14

CSI



Other examples

• Closure of Problem Reports via “procedural mitigations” that were never implemented

• Use of bogus parts in maintenance of state aircraft

• Reductions in assigned criticality levels based on budget constraints or absence of data due to COTS status

15

CSI



The contrast revisited

• Do as I say…– Highly structured– Detailed– Mandatory– Transparent

• …Not as I do– Flexible and malleable– Broadly sketched as goals or intentions– Discretionary– Obscure

16

CSI



A single assurance standard

• FAA Designees support development and operation of digital systems in aviation

• Work often spans public and private sectors

• Would greatly prefer a regime in which assurance is determined by the nature of the hazard rather than who owns the gadgetry

17

CSI



RTCA / DO-264

Guidelines for Approval of the Provision and Use of Air Traffic

Services Supported by Data Communications

18

CSI



DO-264 extends SSA

• No longer talking about what happens to a stricken individual airplane

• Failure of CNS/ATM infrastructure can affect many aircraft simultaneously

• 4761-style safety assessment inappropriate

• Larger environment and players must be considered

19

CSI



Core contributions of DO-264

• OSED: Operational Services and Environment Description

• Approval processes and plans

• SPR: Operational, Safety, and Performance Requirements

• INTEROP: Interoperability Requirements

• Large additional supporting framework

20

CSI



System vs Operations

• Fly-by-wire flight controls– Single thread?– Dual channel?– Triple channel?– Dual-dual?

• SSA: Can judge flight-controls suitability for manned aircraft but not for UAVs

• OSA: Most relevant issue is mission profile

21

CSI



“Communications error wreaks havocin Los Angeles air control system”

• IEEE Spectrum: November 2004• “Lost Radio Contact Leaves

Pilots On Their Own”• Primary failure, then failure of

backup one minute later• 800 flights disrupted, five close

calls, many TCAS alerts• UNIX-to-Windows switch?• 30-day reboots required?• FAA blames its personnel• Little information shared

publicly

22

CSI



“FAA grounds unknown number of flights”

• MSNBC: September 25, 2007

• Loss of all communications

• “Major telephone line…went out”

• World’s busiest cargo hub, >4m tons/year

• All traffic cleared within 250nm radius of Memphis center

• Little information shared publicly

23

CSI



Lone rat kills rail traffic

• April 5, 2008• Stockholm Central Station• One rat in signal box• Three-hour standstill

– Intercity

– Commuter

– Subway

24

CSI



Whither the automobile?

• Automotive Engineering International, August 2008

• The digital car– Steering

– Brakes

– Engine control

– Automatic navigation

– Much more

25

CSI



The goal

• Unified, uniform standard for evaluating safety of system that poses risks to public

• Policy support and enforcement mechanisms• “Early warning system” for attempts to solve

technical problems through political or administrative means

• Protection of internal critics and whistleblowers• Make it easier for public servants to do the right

thing

csi copyright © 2008 certification services, inc. ascsa 2008: the agnostic hazard 1 the agnostic...

Documents

certification services

csi copyright

safety assessment process

certification issues

safety case

canberra slide

design assurance

south africa slide