functional safety standards, status updates

Alexander Mirmilstein & Mark Costin | March 24, 2021

FUNCTIONAL SAFETY STANDARDS, STATUS & UPDATES

2

SOTIF ISO 21448

▪ What is ISO/DIS 21448 SOTIF?

▪ ISO working group status and plan

▪ PAS 21448 vs. DIS 21448 vs. ISO 26262

▪ Examples of new ISO 21448 content for the DIS

ISO 26262, Third Edition

Machine Learning Safety Activities

AGENDA

3

WHAT IS ISO 21448 SOTIF?

▪ ISO 26262 excludes risks due to nominal performance of the sensors and algorithms from scope

▪ ISO 26262 limited to malfunctioning behavior

▪ New vehicle functionalities are being introduced based on complex sensors and algorithms

▪ Safety Of The Intended Functionality (SOTIF) deals with:

▪ The safety of the required/defined behavior of the system

▪ The reduction of additional risk due to limitations in the performance of sensors and algorithms

Why Do We Need?

4

ISO 26262 VS. SOTIFInformal Comparison

Component/Deliverable ISO 26262 SOTIF

Electronic components Considered major source of

failure

Considered minor source of failure

Sensor limitations, normal

operation

Done by product safety Major consideration, i.e., dirt on

camera, fog, etc.

Algorithms Part 6: Assume correct via

process

Assumed to contain limitations

Requirements Complete, testable High level, hard to verify, e.g., never

pass on right

Verification Good coverage due to complete

requirements

Less coverage in structured tests, only

‘known’ knowns

Validation Additional coverage to

verification

Critical, targets risk-based; derive from

traffic statistics; demonstrate system

below target risk

5

ISO WORKING GROUP STATUS AND PLAN

▪ The ISO/TC22/SC32/WG8 working group (responsible for ISO 26262) proposed a “New Work Item” in March 2016

▪ ISO/PAS 21448 published January 2019

▪ ISO 21448, second edition

▪ International standard under development

▪ ISO/DIS 21448, first publicly available draft, released January 2021

▪ In DIS commenting phase

▪ Comments can be submitted via national bodies

History & Status of SOTIF – ISO/PAS 21448

6

ISO WORKING GROUP STATUS AND PLANISO 21448, Second Edition Timing

3rd Mtg.April 20-24, 2020

4th Mtg.Oct, 12-16, 2020

5th Mtg.TBD 2021

7

ISO/PAS 21448 VS. ISO/DIS 21448

▪ Scope

▪ Enlarged to cover all levels of autonomy

▪ Clause 4 overview of activities in the development process

▪ Existing content expanded & clarified

▪ New content: Sense-Plan-Act model

▪ New content: Management of SOTIF activities & supporting processes

DIS Updated Content – Supporting Clauses

Content expanded and clarified

8

ISO/PAS 21448 VS. ISO/DIS 21448DIS Updated Content - Main Body

Content expanded and clarified

9

ISO/PAS 21448 VS. ISO/DIS 21448DIS Updated Content - Main Body

New content to support the updated scope

Content expanded and clarified

New Content

10

ISO/PAS 21448 VS. ISO/DIS 21448DIS Updated Content - Main Body

New content to support the updated scope

Content expanded and clarified

Existing Content• Updated for expanded scope• Content clarified and improved

11

ISO/PAS 21448 VS. ISO/DIS 21448

Annex A: General guidance on SOTIF

Annex B: Guidance on scenario and system analyses

Annex C: Guidance on SOTIF verification and validation

Annex D: Guidance on specific aspects of SOTIF

New Annex Structure

NVIDIA sub-team lead and co-lead

12

ISO/PAS 21448 VS. ISO/DIS 21448

Annex A.3: Examples of the application of SOTIF activities

Annex BC.2: Example for definition and validation of an acceptable false alarm rate in AEB systems

Annex C.3: Validation of SOTIF applicable systems

Annex DC.4: Automotive perception systems verification and validation

Annex EB.1: Method for deriving SOTIF misuse scenarios

Annex FB.2: Example construction of scenario for SOTIF safety analysis method

Annex GD.2: Implications for offline trainings

Annex Re-Numbering

13

ISO/PAS 21448 VS. ISO/DIS 21448

▪ Annex A: General guidance on SOTIF

▪ A.1: Goal structuring notation (GSN) example

▪ A.2: Explanations regarding the interaction between functional safety according to the ISO 26262 series and this document

▪ Annex B: Guidance on scenario and system analyses

▪ B.3: Examples of adaptation of safety analyses to identify and evaluate potential trigger conditions and functional insufficiencies

▪ B.4: Applying STPA in the context of SOTIF for ADAS and automated vehicles

Annexes A & B – New Content

14

ISO/PAS 21448 VS. ISO/DIS 21448

▪ Annex C: Guidance on SOTIF verification and validation

▪ C.1: Purpose of the validation strategy

▪ C.5: Guidance on scenario parameterization and sampling

▪ C.6: Considerations for reducing validation testing

▪ Annex D: Guidance on specific aspects of SOTIF

▪ D.1: Guidance for driving policy specification

▪ D.3: SOTIF considerations for maps

▪ D.4: SOTIF considerations for V2X

Annexes C & D – New Content

15

ISO/PAS 21448 VS. ISO/DIS 21448

▪ Iterative concept of the development

▪ Scenario classification known/unknown, safe/hazardous (Areas 1, 2 & 3)

▪ Overall structure of the document

Unchanged Since ISO/PAS 21448

16

ISO/PAS 21448 VS. ISO/DIS 21448

Main goal of SOTIFis to reduce Area 2 and Area 3 and to increase Area 1

Unchanged Since ISO/PAS 21448

17

ISO 26262, THIRD EDITION

18

ISO 26262, THIRD EDITIONMajor Topics of Discussion

Topic Leader

Integrated approach for new energy vehicle (NEV) Li Bo (CATARC)

Automated driving Rami Debouk (GM)

Connected vehicle, including end-to-end safety Hugues Bonnin (Continental)

Link to SOTIF Nicolas Becker (PSA)

Safety demonstration for AI/DL Mark Costin (NVIDIA)

Predictive maintenance Karl Greb (NVIDIA)

Qualification of pre-existing SW Simon Fürst (BMW)

19

MACHINE LEARNING/AI

20

ISO 26262, THIRD EDITION

▪ Part 6

▪ Alignment of existing requirements to ML

▪ New requirements related to ML

▪ Tailor application of Part 6 to ML

▪ Extend Annex C for configuration of ML (e.g., NN structures, weights parameters)

▪ How to handle training data as a potential source of harm

▪ Other parts potentially affected

▪ Part 4: clause 8.4 validation

▪ Hardware topics: part 5 & 11

▪ Part 8: clause 11.4, confidence in the use of software tools

Updates for ML/AI Content

21

ISO 26262, THIRD EDITION

▪ Many standards with overlapping content on ML/AI

▪ ISO 26262

▪ ISO 21448

▪ ISO/TR 4804 – 5083

▪ ISO/TR 5469 (IEC 61508)

▪ Activities underway to harmonize all standards

ML/AI Content – Coordination with Other Standards

22

QUESTIONS

23

BACKUP

24

ROLES FOR ISO 26262 & ISO 21448ML/AI

Topic Reference

Tools for offline training ISO 26262-8:2018, Clause 11ISO 21448 D.2.4

Hardware - random and systematic faults ISO 26262-5:2018

Hardware limitations ISO 21448

SW compute (e.g., NN structure) ISO 26262-6:2018

SW limitations ISO 21448

SW calibrations (e.g., NN weight values) ISO 26262-6:2018, Annex C

25

ISO 21448 Figure D.4 — Example of Offline ML Training Process Flow

26

ISO 21448 Steps in Analysis of Offline Training Process of ML Algorithms

Figure D.5