Gérard LadierAirbus France
11/2003
DO-178B / ED-12B
Software Aspect of Certificationin the Aerospace sector
Page 2
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Equipment Rules (JAR/FAR 25-1309)
• “Essential” equipment must be designed to perform its intended functions
• The airplane systems and associated components, considered separately and in relation to other systems, must be designed so that :
The occurrence of any failure condition which would prevent the continued safe flight and landing of the airplane is extremely improbable, and
The occurrence of any other failure conditions which would reduce the capability of the airplane or the ability of the crew to cope with adverse operating conditions is improbable
• ...
Page 3
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
An official application document (AMJ 25.1309/AC 25.1309-1A):
How can we comply with the rules? The logic.
• Requires that system design be "fail-safe"
• Classifies failure conditions according to the severity of their consequences
• Defines the acceptable probabilities for catastrophic failures
• Associates acceptable probabilities with the various categories of failure conditions
• Defines the acceptable means of conformity for the various contributory elements.
Page 4
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Classification of failure conditions according to the severity of their effects
How can we comply with the rules? Details
• Catastrophic: multiple loss of life, usually with loss of aircraft
• Hazardous: the aircraft, or crew, is less capable to deal with adverse operating conditions, which may result in:
Large reduction in safety margins or functional capabilities Physical distress or a workload of such magnitude that the crew is unable to
perform its tasks Serious injury or death concerning a relatively small number of passengers/crew
members
• Major: the aircraft, or crew, is less able to deal with unfavorable operational conditions, which may result in:
Significantly reduced safety margins Significantly reduced functional capabilities A significant increase in the crew’s work load, or conditions which reduce the
effectiveness of their work Discomfort for passengers and crew members, including injury
• Minor: no significant reduction in the aircraft’s level of safety
• No effect on safety
Page 5
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Definition of the acceptable probabilities for catastrophic failures
How can we comply with the rules? Details
• Fact (“historical evidence”) : 1 serious accident per 106 flight hours
• Fact: 10% are due to aircraft system problems
• First conclusion: “It seems reasonable that serious accidents caused by systems should not be allowed a higher probability than this in new airplane
designs” => acceptable threshold for serious accidents caused by system failure : 10 -7 per flight hour
• Arbitrarily, it is considered that there are about one hundred (102) potential failure conditions in an airplane which would be catastrophic and that each one of these is equally likely to occur.
• We can thus define the acceptable probability of ONE catastrophic failure condition per flight hour as being : 10-9
Page 6
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Extremely improbable
Catastrophic
10-9
Extremely remote
Hazardous
10-7
Remote
Major
10-5
Probable
Minor
10-3
Acceptable
Unacceptable
How can we comply with the rules? Details
Associating the acceptable probabilities with the various categories of failure conditions
Page 7
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Since dependability cannot be guaranteed from an assessment of the software product, it is necessary to have assurance on its development process
You can’t deliver clean water in a dirty pipe
•It is not feasible to assess the number or kinds of software errors, if any, that may remain after the completion of system design, development, and test.
Means of conforming to the rules
The AMJ 251309 / AC 25.1309-1A presents acceptable means of demonstrating conformity with JAR/FAR 25.1309 requirements applicable to equipment. But software is a “special case”, and cannot be treated in the same way as other equipment items:
Page 8
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
In order to detect and correct specification, design and production errors for complex systems, we use the systems’ Development Assurance.
Planned, systematic actions necessary to provide anadequate confidence and evidence that a product or process satisfies given requirements
How can we comply with the rules?
The level of Assurance required is determined by the severity of the potential consequences of the errors concerned.
Page 9
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Catastrophic
Hazardous
Major
MinorNo safety effect
Failure Condition
A
B
C
DE
DAL System(Development
Assurance Level)
Once the DAL has been determined through a safety analysis, the ED-12B/DO-178B requirements must be observed for this level
From the rules to the ED-12B/DO-178B : the levels
Page 10
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
SOFTWARE CONSIDERATIONS IN AIRBORNE SYSTEMS
AND EQUIPMENT CERTIFICAION
RTCA
DOCUMENT NO. RTCA/DO-178B
December 1, 1992Prepared by: SC-167
“Requirements and Technical Concepts for Aviation”
The ED-12B/DO-178B - Anatomy
• One section establishes the link with the “system” aspects
• Several sections specify the requirements (Assurance objectives and means of achieving them) for each process
• One section summarizes the life cycle data requests (~ doc)
• One section provides “additional considerations"
• An appendix defines, for each software level:The applicable objectives and the products requested per process (with
cross-references to the rest of the document for precise descriptions)
The degree of independence of the process’ activitiesThe control category for the data generated by the process’
activities.
Page 11
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
The DO-178B/ED-12B – Life cycles and processes
• No mandatory life cycle
• Definition of separate processes that will be combined for a given project to describe its life cycle:
Planning process (organization/plans rather than scheduling)
Development process (specification, design, coding, integration)
Integral processes (verification, configuration management, quality assurance, certification liaison process).
Page 12
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
The ED-12B/DO-178B – Processes
• Define for each process:The Assurance objectives
(e.g. : detect any errors introduced during development) The means of achieving those objectives
(e.g : combination of reviews, analysis and tests)The process input data
(e.g. : specifications, source code, verification plan)The process activities
(e.g. : various reviews and analysis, various tests based on requirements)The process products
(e.g. : test sets, procedures and results)
The transition criteria, which must be met in order to proceed
• Generally speaking, there is no specific definition of the methods or means to be used (for example, it does not impose unit tests).
Page 13
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
The ED-12B/DO-178B - Verification
• The largest section of the document in terms ofvolume: 13 pages of descriptions (the others contain an average of
5 pages)
Work loads (and justification) generated....
• Basic principles:“Integral” process => applies to all the development processes
A combination of reviews, analyses and tests to detect and identify errors introduced during development
Functional test (based on the requirements)
NO TEST BASED ON THE CODE STRUCTURE
“Functional” and “structural” coverage analyses.
Page 14
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
The ED-12B/DO-178B - certification liaison
• Objective: ensure effective communication/understanding between the applicant and the certification authorities
• Means: The Plan for Software Aspects of Certification, given to the
Authorities as early as possibleReviews carried out by the certification
authorities “software” specialists at their own discretion
Software Accomplishment Summary and Software Configuration Index.
Page 15
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
The requirements are more or less equal to those of the level of software generated
Reduced requirements (3 lines as opposed to 53), limited to tool validation, which are nonetheless subject to a large number of different interpretations depending on the certification authority representative.
The ED-12B/DO-178B - S. 12 – Tool Qualification
• Necessary when processes required by the rest of the document are eliminated, reduced or automated by the use of a software tool (deterministic) whose outputs are not verified
• 2 categories of tools, defined according to the risk of error in the avionics code :
Development tools (e.g.: code generator)
Verification tools (e.g.: emulators, simulators)
Page 16
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
ED-12B/DO-178B -Variations by level
• Levels A and B are very similar They have the same number of objectives (except for one, concerning
structural coverage)The main difference between them is the degree of independence required
in the meeting of objectives (40 % with independence for level A, 20% for level B)
• Level C ~ 85 % of levels A/B (no. of objectives)The variation is particularly sensitive as regards the verifiability of
specification, design, code, the test coverage of structure and the independence
• Level D ~ 50 % of level C (no. of objectives)No requirements on design, coding & test coverage of structure
• Level E: No requirement “once the certification authority has confirmed that a software is level E"
Page 17
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Questions about the ED-12B/DO-178B and its application
• Is the ED 12-B/DO-178B too expensive to apply?
•Can the software alone be certified?
•Can COTS be used in avionics?
•Can Object Oriented approach be used?
•Can the test be replaced by proof?
Page 18
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
What about the future?
• The proof of the pudding is in the eating: the development assurance approach is effective
• But it remains a default approach: “I can’t ensure the quality of my product, but at least I can ensure the quality of its development process”
• There are promising signs that a change/return to a “product approach” may be possible
To be developed… to make the pudding easier to digest!
Page 19
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Page 20
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Is the ED 12-B/DO-178B too expensive to apply?
• 1997, SSAC : "make recommendations to the FAA to reduce the cost and time associated with software aspects of certification for both airborne and ground-based software while maintaining or improving safety”.
• Survey carried out on US industry. 240 questions asked, 300 questionnaires returned. Questions concerning DO-178B:
$ Independence ?
$ MC/DC ?
$ Quality assurance ?
$ Traceability ?
$ Unreasonable requests for documentation ?
$ Tool qualification ?
$ Another specific question was also asked on the connection between DO-178B/ED-12B and safety.
Page 21
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Is the ED 12-B/DO-178B too expensive to apply?
• But the answers received were rather surprising :
Independence: 82 % think it is extremely or very valuable
Traceability: considered effective, even if it is expensive
Quality assurance: between 57% and 79% (depending on the question) think it is extremely or very valuable
Tools qualification: everybody found some errors during qualification, which is less expensive than you might think
Analysis of MC/DC structural coverage: 12% of industrials have never found an error with this technique, which is very expensive.
Documentation was the only area in which the assumption was validated by the study, but a more detailed analysis showed that people had little understanding of the real requirements in this field.
Page 22
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Can the software alone be certified?
• 1996. Certain members of the FAA (>free flight) wanted to change the way certification was carried out : currently, computer hardware and software has to be certified as a "system" for every installation. We're suggesting that standards be developed so that software can be certified as a discrete appliance, irrespective of the operating system or hardware platform.(…)
• This « plug & play » approach was eventually abandoned (computer software cannot be certified alone – the whole system has to be certified).
• But the concept of integrated modular avionics for the A380 and its “incremental qualification” may represent a (small) step in this direction.
See you in 2006!
Page 23
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
The ED-12B/DO-178B and COTS:
•COTS ? Business as usual !
2.4/f “Off-the-shelf software products: Software products of this type included in on board systems and equipment must meet the objectives set out in this document”
2.4/g “If commercial off-the-shelf software products have any missing life cycle data, it is necessary to complete this data in order to meet the objectives set out in this document…”
Can COTS be used in avionics?
• Contradiction between the industrial need to use COTS and the process approach (as opposed to the product approach) of the ED-12B/DO-178B.
• May turn out to be counter-productive as far as dependability is concerned...
Page 24
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Can the Object approach be used?
• The subject is still under debate, but some applications already exist.
• Mike Dewalt summed up the situation:
Page 25
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Can the test be replaced by proof?
• The ED-12B/DO-178B is prescriptive as far as verification means are concerned
• Proof of properties is temptingExhaustiveSource or binary analysis Extra design effortFinancial gain:
– No costly test case identification phase – No specific material resources – Extensive or total automation
• First pilot industrial application for the A380
• If the application is extended, the contradiction with the ED-12B/DO-178B must be resolved (it is necessary to return to the objective of the verification, while the current document prescribe a means, the testing)
Page 26
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
Introduction
6768 4697071727374 237576777879
200k - A300FF
23k - A300B
4 k - Concorde
2 M - A3105 M - A320
12 M - A330/340
1
10
100
1000
10000
65 66 6768 69 70 71 7273 74 75 76 7778 79 80 81 8283 84 85 86 8788 89 90 91 9293 94 95
year
vo
lum
e (
k.o
cte
ts)
Page 27
© A
IRB
US
FR
AN
CE
S.A
.S.
Tou
s dr
oits
rés
ervé
s. D
ocum
ent
conf
iden
tiel.
This document and all information contained herein is the sole property of AIRBUS S.A.S. No intellectual property rights are granted by the delivery of this document or the disclosure of its content. This document shall not be reproduced or disclosed to a third party without the express written consent of AIRBUS S.A.S. This document and its content shall not be used for any purpose other than that for which it is supplied.
The statements made herein do not constitute an offer. They are based on the mentioned assumptions and are expressed in good faith. Where the supporting grounds for these statements are not shown, AIRBUS S.A.S. will be pleased to explain the basis thereof.