3.4-3.6 do-178b and do-254
Post on 14-Apr-2016
124 Views
Preview:
DESCRIPTION
TRANSCRIPT
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
1
MGA-855 Certification des systèmes embarqués
d’aéronefs Maîtrise en génie : Concentration en génie aérospatial
It’s Only Software (and Hardware)!
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
2
MGA-855 Certification des systèmes embarqués
d’aéronefs Maîtrise en génie : Concentration en génie aérospatial
Chapitres 3.4, 3.5,3.6 RTCA/DO-178B V&V, Traceability, Testing,
Structural Coverage, Tool Qualification and PDS, RTCA/DO-254 Design Assurance Level and AC20-152
présenté par :
Maxence Vandevivere cellmaxence@gmail.com
Professeur responsable : René Jr. Landry Poste : 8506 Porte : 2950
Email : rlandry@ele.etsmtl.ca Site web : www.etsmtl.ca/rlandry
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
3
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
This module is designed to show the application of the certification principles contained in the earlier chapters. As such, it touches upon many aspects of the certification process, but the material is not complete, comprehensive, or necessarily current with the latest regulations and guidance materials. For these reasons, the analysis contained in the following slides should not be used as the basis of a certification program for the system under investigation.
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
4
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
Overview
3.4 to 3.6 RTCA/DO-178B Wrap-up and DO-254 • 3.4.1 Verification & Validation • 3.4.2 Traceability • 3.4.3 Verification and Testing Methods • 3.4.4 Structural coverage • 3.4.5 Testing • 3.4.6 Conformity Review • 3.4.7 Delivery • 3.5.1 Tools • 3.5.2 Additional Considerations (tool qualification, PDS, alternative methods) • 3.6.1 DO-254 • 3.6.2 DO-254 Design Assurance Level • 3.6.3 AC20-152
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
5
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.1 – V&V
• Recap: • Verification means ensuring that what was done in a
process was correct (meetings requirements, matches process, etc.)
• Validation means ensuring that the assumptions, requirements are actually correct and complete
• Verification takes place throughout the entire life cycle • Validation takes place at the requirements level
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
6
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.1 – V&V (cont’d)
• Requirements are the foundation we build on
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
7
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.1 – V&V (cont’d)
• What happens when a requirement changes, or becomes invalid in some way?
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
8
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.1 – V&V (cont’d)
• Requirement validation is a critical process • If the requirement is wrong, there is a ripple effect that
could extend through the entire project • If a requirement changes at any level we need to:
– Update all connected requirements (up & down stream) – Update all data items connected – Verify changes
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
9
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.1 – V&V (cont’d)
• Question: How do we know what data (requirements, code, documentation, etc.) needs to change or be updated when something in our project changes?
• Answer: Traceability
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
10
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.2 – Traceability
• Traceability is: “The evidence of an association between items, such as between process outputs, between an output and its originating process, or between a requirement and its implementation.” (DO-178B, glossary)
• DO-178B: – does not prescribe any particular method of showing traceability – Does require that traceability can be demonstrated in some way
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
11
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
• Traceability also helps us avoid this situation:
3.4.2 – Traceability (cont’d)
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
12
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.2 – Traceability (cont’d)
• What needs to be traceable? • Strictly speaking, DO-178B states we need to show
traceability for: – System requirements to software requirements – High-level to low-level software requirements – Low-level software requirements to code – Evidence of process outputs tracing to the process activity and its inputs
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
13
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.2 – Traceability (cont’d)
• Generalizing, the big items that must be traceable are: – Requirements – Design – Code – Test & test results
• However, breaking it down, really everything traces:
– Reports need to trace to the process action it satisfies. Reports also need to show/trace to the artifacts that are being reported on
– Audits, reviews, checklists need to uniquely identify what was reviewed
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
14
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.2 – Traceability (cont’d)
System requirements (System level documentation and SRD)
Software high-level requirements (SRD)
Software low-level requirements (SDD)
Code
Test
Test results
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
15
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.2 – Traceability (cont’d)
• What does it mean if something doesn’t trace? – For code? – For requirements?
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
16
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.2 – Traceability (cont’d) Derived Requirements
• Derived requirement – Definition: “Additional requirements resulting from the software development process, which may not be directly traceable to higher level requirements.” (DO-178B glossary)
• Example of a derived requirement: – In a graphics library, that has high level requirements for drawing
primitives, in an organized way, one example might be: – “The graphics library shall externalize line styles that may be used to
render primitives.” – There is no top-level SRD requirement for a line style external definition.
However the software design may do this for flexibility, future expandability, or other benefits
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
17
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.2 – Traceability (cont’d)
• If a requirement doesn’t trace down to a child (either another requirement, or code, or test) our traceability is broken and needs to be fixed
• If a requirement does not trace upward, we must check if it is a derived requirement
– NB: Of course, all requirements will follow the rules defined in the SRS which will show us how to label derived requirements differently than other requirements
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
18
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.2 – Traceability (cont’d)
• If code doesn’t trace up to a requirement, we need to: – add a requirement, or – add design to support it, – or remove the code
• If code doesn’t trace down to test, we need to: – Analyze why it wasn’t tested in the first place – did we miss a
requirement in our testing? And/Or, – Add more test cases to test the code more completely
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
19
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.3 – Verification and Testing Methods
• Question: How do we ensure our requirements are implemented in our code/software?
• Answer: Testing – specifically, requirements-based testing
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
20
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.3 – Verification and Testing Methods (cont’d)
• Types of testing (from DO-178B):
– Hardware/software integration testing: To verify the correct operation of the software on the target computer environment
– Software integration testing: To verify the interrelationships between software requirements and components and to verify the implementation of the software requirements and software components within the software architecture
– Low-level testing: To verify the implementation of software low-level requirements
• Formal methods: discrete math, proofs, logic grammars checked by computers, can also provide exhaustive analysis to compliment testing
• Exhaustive input testing
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
21
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.3 – Verification and Testing Methods (cont’d)
• How? – Base our test cases on the requirements – Requirements coverage analysis to verify all requirements tested – Structural coverage analysis to verify all code was tested/exercised
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
22
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.3 – Verification and Testing Methods (cont’d)
• Two main types of test cases are used:
• Normal cases: Tests normal operations, or ranges of inputs
• Robustness cases: Tests that the code responds correctly to:
– Invalid input – Abnormal conditions – Failure modes – Timing/scheduling issues – Etc.
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
23
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.3 – Verification and Testing Methods (cont’d)
• Robustness?
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
24
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.3 – Verification and Testing Methods (cont’d)
• Data coupling: The dependence of a software component on data not exclusively under the control of that software component
• Control coupling: The manner or degree by which one software component influences the execution of another software component
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
25
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.4 – Structural Coverage
• Question: How do we know if our code represents our requirements?
• Answer: Traceability, requirements-based testing, and Structural coverage
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
26
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.4 – Structural Coverage (cont’d)
• Definition: structural coverage is an analysis of what code is executed by the requirements-based testing procedures
• Typically done during testing • Code is instrumented, and a tool reports all lines in the
code executed during an execution of the software • All lines not executed then need to be analyzed • Should confirm data and control coupling (note: this
should have been identified during design, code review, etc.)
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
27
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.4 – Structural Coverage (cont’d)
• The level of structural coverage detail/granularity depends on the software criticality:
– Level A: » Byte code – each part of the binary code created must be justified
and traceable to a requirement » Level A also requires MC/DC (not AC/DC) – modified condition /
decision coverage » MC/DC means: Every condition in a decision in the software has
taken all possible outcomes at least once, and the condition has been shown to independently affect the decision outcome
– All other levels (if required) : Statement coverage – or put another way – traceability to a line of code
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
28
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.4 – Structural Coverage (cont’d)
• Possible reasons for finding code that is not executed: – Tests were not complete – need to add more tests – Requirements were not complete – need to add more requirements and
tests – Code that is not used and never will be – “dead code”
» Does not link to requirements » Is not reachable (cannot be executed or used) in operational
configuration – Code that is not used in normal circumstances – “deactivated code”
» Like a maintenance routine, or data-loader routines used only on the ground
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
29
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.5 – Testing
• Before we start testing… what do we need to do?
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
30
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.5 – Test for score
• Definition: “test for score” : testing for certification credit • All “test for score” must be done on a representative
system or it cannot be used for credit • Question: What do we need to do before we start
testing?
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
31
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.5 – Test for score (cont’d)
• Before we start testing, we need to complete a TRR – a test readiness review
• What is a TRR? – A verification of our transition criteria to advance into the testing phase – Makes sure we have completed all items before we begin testing – If something is not complete, we are testing a moving target
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
32
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.5 – Test for score (cont’d)
• What needs to be complete to begin testing? – All plans reviewed, audited, and complete – All requirements and design, reviewed, audited, and complete – All test procedures, test cases reviewed, audited, and complete – All traceability reviewed, audited and complete – A baseline of the software and all life-cycle data created – Is the test environment ready for testing? (calibration, configuration as
per plans, etc.) – All CRs, PRs should be closed (or at least open items contain mitigation
or have been reviewed to ensure no impact on testing)
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
33
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.5 – Test for score (cont’d)
• What will we test? – A controlled baseline
• Where will it be tested? – in a representative target environment
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
34
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.6 – Conformity Review
• Question: What do we do when we have finished testing? • Answer: Conformity review!
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
35
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.6 – Conformity Review (cont’d)
• Definition: Conformity review: A QA (Quality Assurance) process that reviews everything we did, that we did it, and that everything has been completed as per the plans
• In general, a conformity review, ensures that: – All plans, process activities for certification credit have been completed – That all life-cycle data has been retained, and under CM – That all life-cycle data complies with the plans & standards – Traceability has been completed (especially up to the system and safety
requirements) – All problem reports have been closed, or have been dealt with according
to the CMP – Any deviations are recorded and approved – All executable object code can be re-created from what is stored under
CM using the instructions created
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
36
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.6 – Conformity Review (cont’d)
• Conformity review cont’d: – If applicable, the software can be loaded using the instructions created – Any problem reports or change requests deferred from previous
releases have been re-evaluated and reviewed to incorporate into the software or determine necessary status
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
37
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.4.7 – Delivery
• Now what? – Deliver to the regulatory authority concerned (TCCA or FAA) – part of
your certification liasion process – Deliver to your customer (if any) – Take a nap
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
38
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.1 – Tools
• What tools are available to help achieve our goals? • For documentation, tracking, etc:
» Excel for tracking » Word for documents » Filemaker or Access for databases » cvs, svn, git, for CM
• Some more integrated examples: – IBM Rational toolset – Rose, ClearCase, ClearQuest, etc. – MKS Integrity & Source – Polarion
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
39
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.1 – Tools (cont’d)
• Tools to help development – other than compilers, linkers, etc.
• Static and dynamic analysis tools (a few examples): – OpenSource: splint, lint, cpplint, cppcheck – PClint (Gimpel software) – Klockwork
• Code complexity: – SourceMonitor – Klockwork
• Code coverage: – Bcov, gcc/g++, – lots of compiler-specific tools
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
40
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.2 – Additional Considerations
• Tool qualification • Previously developed software (PDS) • Alternative methods
– Exhaustive testing – Multi-version dissimilar software – Product service history – equivalent level of safety
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
41
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.2 – Additional Considerations : Tool Qualification
• What if we want to use a tool to automate part of the certification process?
• We can. • DO-178B classifies the use of a tool in two ways:
– Tools we use with a human-in-the-loop – the output is verified – Purely automated, no output verification
• For the former, we need to make sure the tool is fit for the purpose
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
42
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.2 – Additional Considerations : Tool Qualification (cont’d)
• Definition: “Tool qualification” is needed when: “a processes of [DO-178B] are eliminated, reduced or automated by the use of a software tool without its output being verified as [part of the software verification process].” (section 12.2 of DO-178B)
• Tools are put into one of the following categories: – Software development tools – a tool that can introduce errors, e.g.: a
tool that automatically generates code for a developer – Software verification tools – a tool that cannot introduce errors but may
fail to detect them (analysis tools, etc.)
• Any tool that will be used needs to be controlled (CM)
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
43
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.2 – Additional Considerations : Tool Qualification (cont’d)
• Tool qualification for software development tools: – Typically (unless you can convince the cert authorities otherwise) you
need to certify the tool to the same criticality level as the software you’re using it with. e.g.: A certified code compiler for a Level B project would need to be Level B certified
– Need to show compliance with tool operational requirements – In addition, you need to show:
» that the operational requirements are “good” – i.e.: correct, consistent, complete – and that the requirements are all covered via analysis
» demonstrate operation in normal and robust testing conditions » Structural coverage analysis » Analysis of the potential errors produced by the tool
– Software verification tools – a tool that cannot introduce errors but may fail to detect them (analysis tools, etc.)
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
44
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.2 – Additional Considerations : Tool Qualification (cont’d)
• Tool qual for software verification tools: – Show that the tool complies with its operational requirements
• Quick tip: if you need a certified development tool, see if one exists on the market for purchase
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
45
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.2 – Additional Considerations: PDS
• What if we want to use pre-existing software in a new certification project?
– Code that was previously used on another certification program – Usually applies to certified code
• There are complexities that are specifically discussed in DO-178B in doing this
• For instance: – Modifying existing code for a new project (adding requirements,
features) – Changing the development environment (porting the code) – Changing the target environment (aircraft, avionics, etc.)
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
46
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.2 – Additional Considerations: PDS
• Some items and risks to be aware of: – SSA should be revised to include new
environment/target/modifications/changes – Impact analysis of new/modified requirements, design, code, etc.
(especially on data & control coupling) – Re-verification of modified areas – including areas related by data and
control – New target environment considerations need to include target processor,
memory, other hardware and software used, data busses, etc – If a different compiler is used – are different options used? The code
output will probably be different too, so need to test all over again – For COTS software, need to reverse engineer supporting data life-cycle
data – Need to show traceability from PDS to new baseline, and support
tracking changes for multiple versions of the software
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
47
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.2 – Additional Considerations: Alternative methods
• Alternative methods – can be used as alternative ways of satisfying objectives of DO-178B
• Methods outlined in DO-178B is not an exhaustive list – other methods do, and may exist in the future
• Must get early buy-in and acceptance of alternative method from certification authorities
– Exhaustive testing – Multi-version dissimilar software – Product service history – equivalent level of safety
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
48
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.2 – Additional Considerations: Alternative methods (cont’d)
• Examples of alternative methods: – Exhaustive testing – testing using logic flow, discrete math, etc. to
improve the specification and verification of the software – Multi-version dissimilar software – system design technique that uses
two or more parts of software to provide the same functionality to attempt to avoid common errors in the components. Techniques are usually a combination of some of the following:
» Using different languages to write the components » Use different compilers to generate binaries with same code » Using different processors to run the same code » Software requirements, design, code created by two separate
teams (note, system and functional requirements are identical!) Can also use different code and design standards
Note: Testing approach must take dissimilar s/w into account – Product service history – equivalent level of safety
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
49
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.5.2 – Additional Considerations: Alternative methods (cont’d)
• Examples of alternative methods, cont’d: – Product service history – some certification credit may be granted by
showing an equivalent level of safety with service history evidence. – This approach depends on demonstrating quality of processes used to
manage the software and attributes like: » Effectiveness of change tracking, problem reporting » Control/configuration of the software » Stability and maturity of the software » What environment the software is used in (aerospace, auto industry,
gaming?) » Error rates from service history » Impact of modifications to the software
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
50
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.6.1 – DO-254
• What is DO-254? – Hardware certification guideline – Also known as EUROCAE ED-80 – Released in 2000 (was not really in use – required – by the FAA until
about 2005) – Provides guidance for the design assurance of complex electronic
hardware (CEH) for airborne use in aircraft equipment and systems. – Structure of the document is based on DO-178B
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
51
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.6.1 – DO-254 (cont’d)
• What is Complex Electronic Hardware (CEH)? – DO-254 defines a piece of hardware as simple if: “a comprehensive
combination of deterministic tests and analyses appropriate to the design assurance level can ensure correct functional performance under all foreseeable operating conditions with no anomalous behavior.” (DO-254, section 1.6)
– Or put another way, hardware is classified as simple if you can fully test it
– If the hardware is not simple, it’s complex
Simple, right? Not quite: this distinction is still subject to heated debates on a fairly regular basis
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
52
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.6.1 – DO-254 (cont’d)
• A simple example: – If we have a 4-bit controller with discrete I/O (two input, two output) – This could easily be exhaustively (completely) tested
• Another example: – A 16-bit controller with discrete I/O (8 input, 8 output) – Again, we can exhaustively test all inputs and outputs
• Not so simple: – A 1000 gate FPGA with 100 pins – We can no longer easily (and deterministically) exhaustively test all
inputs and outputs
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
53
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.6.1 – DO-254 (cont’d)
• Document structure is basically the same DO-178B: – Need to define integral processes: Verification, CM, QA, cert. liaison. – We have a life-cycle – but it is a Hardware design life cycle instead of
software – Planning process – Hardware design process
» requirements capture, » high-level & low-level design » implementation, » production, and » testing).
– Design assurance level instead of software criticality level – Like DO-178B, there is no prescribed process you must follow
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
54
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.6.1 – DO-254 (cont’d)
• One difference from DO-178B is production transition: – Basically covers the transition from development to production. – Intended to ensure:
» Actual physical production of hardware has been planned » Process to procure and handle hardware is in place » Safety considerations are known and documented
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
55
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.6.2 – DO-254 – Design Assurance Level
Design Assurance Level
Failure Conditions Probability
Level E No Effect < 1 x10-5 Level D Minor > 1 x10-5 Level C Major 1 x 10-5 < 1 x 10-7 Level B Hazardous/Severe-Major 1 x 10-9 < 1 x 10-7 Level A Catastrophic < 1 x 10-9
* If level is Level D or below, you do not need to use DO-254
Note: DAL uses the same scale as software criticality
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
56
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.6.1 – DO-254 (cont’d)
• FFP analysis– functional failure path analysis – Appendix B of DO-254
– Iterative analysis to identify which components have a critical level (Level A or B DAL)
– Similar to a fault tree analysis – Only applies to critical (Level A or B) components – Used when components of different levels are connected together – Intent is to identify critical components so that they can be assigned an
appropriate DAL
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
57
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.6.1 – DO-254 (cont’d)
• For DO-254, need to deliver the following documents to cert authority:
– Plan for Hardware Aspects of Certification – PHAC – Hardware Verification Plan (HVP) – Top Level Drawing
» Contains the configuration information (like a configuration index with both hardware and software information)
– Hardware Accomplishment Summary (HAS)
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
58
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.6.1 – DO-254 (cont’d)
• COTS hardware – DO-254 is interested in their pedigree, and service history – Can use COTS chips as long as can show:
» Wide use (the wider, the better) » Manufacturer is known to have good engineering processes » At least some documentation (technical, specs, etc.) » Look for quality control, mil spec, etc.
– Don’t reinvent the wheel if you don’t need to – Examples:
» ARINC429 bus controller » MIL-STD 1553 bus controller
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
59
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.6.3 – AC20-152
• AC20-152 - a typical AC • Released in 2005 • A lengthy document – 2 pages long! • Purpose: Defines more specific scope and details about
the application of DO-254
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
60
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
3.6.3 – AC20-152 (cont’d)
• AC20-152 – highlights: – Typical scope is for “application specific integrated circuits (ASIC),
programmable logic devices (PLD), field programmable gate arrays (FPGA)” (1.a)
– “When designing level D devices, manufacturers may choose to use RTCA, Inc. document RTCA/DO-254, Design Assurance Guidance For Airborne Electronic Hardware, dated April 19, 2000, or continue to use their existing design assurance practices.” (1.b, emphasis added)
– “We don’t intend that you apply RTCA DO-254 to every type of electronic hardware” (section 2)
– “we don’t intend that you apply RTCA/DO-254 to COTS microprocessors. There are alternative methods or processes to ensure that COTS microprocessors perform their intended functions and meet airworthiness requirements. Coordinate your plans for alternative methods or processes with us early in the certification project.” (3.b, emphasis added)
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
61
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
Summary
• Traceability is a tool and evidence • Tools used with no human-in-the-loop must be qualified • Development tools must be “certified” & comply with ops
requirements • Verification tools must comply with ops requirements only • Before testing, must be ready (TRR) • Must test software in both normal and robustness cases • Test for score (credit) must take place:
– on target hardware, – with an identified baseline
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
62
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
2.3.6 Summary
• DO-254 has same basic structure as DO-178B • Don’t over-apply DO-254 • DO-254 targets complex hardware (PLDs, ASICs,
FPGAs) • COTS hardware can be used, but be careful to select
hardware with a history
• For both software and hardware: Contact cert authorities early to coordinate efforts (and decrease the amount of surprises)
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
63
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
Questions?
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
64
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
References
• RTCA/DO178B • RTCA/DO-254 • AC 20-152
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE
65
3.4, 3.5, 3.6 DO-178B Wrap up & DO-254
Certification des systèmes embarqués d’aéronefs MGA-855: Chapitre 3
Images References
• All images creative commons, unless otherwise noted.
top related