Managing Assurance Cases in Model Based Software Systems

Sahar Kokaly McMaster University and University of Toronto, Canada

kokalys@mcmaster.ca

Supervised by: Tom Maibaum (McMaster University) and Marsha Chechik (University of Toronto)

ICSE’17 Doctoral Symposium May 23, 2017

The “thesis” of my thesis

¤  The state of practice in compliance management can be made more effective using model management.

2

The “thesis” of my thesis

¤  The state of practice in compliance management can be made more effective using model management.

3

Context

4

Software Safety Standards

¤  Regulations and standards, e.g., ISO 26262 (Functional Safety in Road Vehicles) to demonstrate compliance.

Hazardous Events (HE)

Safety Goals (SG)

Functional Safety Requirements (FSR)

Technical Safety Requirements (TSR)

Hardware Safety Requirements

(HWSR)

Software Safety Requirements

(SWSR)

defined by

refines

refines

decomposed

5

Safety Cases (a special kind of assurance case)

¤  A Safety Case is an argument which demonstrates that each of the safety goals has been met, by eventually linking them to evidence in the system.

¤  Evidence can come in many forms: e.g., test results, analyses, model checking results, expert opinion, etc.

6

Example: Power Sliding Door

7

The “thesis” of my thesis

¤  The state of practice in compliance management can be made more effective using model management.

8

Model Based Software Systems

¤  Models as first class citizens

¤  Model Driven Engineering “MDE” and the ultimate goal of automatic code generation

¤  Advantages: ¤  Reduce accidental complexity by working at a higher level

of abstraction

¤  Minimize development cost

9

Example: Power Sliding Door System

requestDoorOpen() requestDoorClose()

State: {open, closed}

requestSpeed() sensed_speed: Real

request Speed() closed: Boolean sensed_speed: Real

getSpeed(sensed_speed) sensed_speed: Real

openDoor() closeDoor() powered: Boolean activated: Boolean

powers controls

communicatesWith communicatesWith

communicatesWith

controls

vs_ecu:VSECU ac_ecu:ACECU a:Actuator ds:DriverSwitch s:RedundantSwitch

ds.requestDoorOpen() ac_ecu.requestSpeed()

ac_ecu.sensed_speed

[if ac_ecu.sensed_speed<=15 and a.powered and s.closed] a.activated = True, a.openDoor()

ds.requestDoorClose()

[if ac_ecu.sensed_speed<=15 and a.powered and s.closed] a.activated = True, a.closeDoor()

s.requestSpeed()

[if s.sensed_speed<=15] s.closed else s.open

par

R: CD-SD

PSD: CD

PSD: SD

ac_ecu.requestSpeed()

ac_ecu.sensed_speed

Power Sliding Door System

10

The Model Management Toolbox

11

lift slice

match (bidirectional) model transformation

merge

+

diff

+ MegaModel Management Operators (map, filter, reduce) and lifted operators

The “thesis” of my thesis

¤  The state of practice in compliance management can be made more effective using model management.

12

let’s consider the following compliance management scenario…

13

Problem: Safety Case and System Co-Evolution

SafetyCase

SystemModel SystemModel’

SafetyCase’

Change

R R’

?

Problem: Can we aid the safety engineer in constructing a safety case for an evolved system by reusing the components of the original safety case as much and as soundly as possible, thus reducing the overall revision cost incurred?

14

Necessary step: Impact assessment to identify how changes in the system affect the safety case.

Model Based Safety Case Impact Assessment due to System Changes

ModelChecking

PSD:CD

TestResults

PSD:SD

!

!!! !

!!!

✓

✓

✓

✓

✓

!

reuse recheck revise

Goal

Context

Sol.

Str.IsSupportedBy

IsSolvedBy

InContextOf

✓ ✓ ✓

✓

✓

𝐴

𝐶3

recheck𝐶2

6 7

𝑆 𝑆′

𝐴′complete

change 𝐶1𝑎𝑚𝐶0𝑚𝐶0𝑎

𝐶1𝑑𝑚𝐶0𝑚𝐶0𝑑

𝐷

𝑅 𝑅′𝑅𝐴′1

23

44

5

revise 8,9

System Megamodel

Annotated Safety Case

Model Slicers

Delta (change)

Safety Case

Traceability

Model-Based Impact Assessment Algorithm

[MODELS’16]: original approach

✓ ! reuse recheck revise

15 [SafeComp’17]: improved approach, safety case slicer, cost-savings analysis

Research Objectives

¤  RO1: Assurance Case Modeling.

¤  RO2: Modeling the Compliance Ecosystem.

¤  RO3: Assurance Case Operators.

¤  RO4: Model Management Workflows for Compliance Scenarios.

¤  RO5: Application in the Automotive Domain.

¤  RO6: Tool Building.

¤  RO7: Validation.

16

Research Objectives

¤  RO1: Assurance Case Modeling.

¤  RO2: Modeling the Compliance Ecosystem.

¤  RO3: Assurance Case Operators.

¤  RO4: Model Management Workflows for Compliance Scenarios.

¤  RO5: Application in the Automotive Domain.

¤  RO6: Tool Building.

¤  RO7: Validation.

17

Research Contributions

¤  RO1: Assurance Case Modeling à [SafeComp’17]

¤  RO2: Modeling the Compliance Ecosystem à [MiSE’16]

¤  RO3: Assurance Case Operators à [SafeComp’17]

¤  RO4: Model Management Workflows for Compliance Scenarios à [MiSE’16, MoDELS’16, SafeComp’17]

¤  RO5: Application in the Automotive Domain à partially in [SafeComp’17]

¤  RO6: Tool Building à [planned]

¤  RO7: Validation à [planned]

18

Related Work: Using Modeling for Compliance

¤  Compliance Management Frameworks ¤  e.g., [Hamou-Lhadj], [DLVara], [Habli2008]

¤  Algorithms and Operators for Compliance ¤  e.g., [Nejati], [Ghanavati]

¤  Modeling Standards and Assurance Cases ¤  e.g., [Kelly2004], [Luo] [Panesar-Walawege], [Ghanavati] [Bandur]

¤  Model-based Approaches for Compliance ¤  e.g., [Habli2010], [Gallina]

¤  Safety Case Construction and Maintenance ¤  e.g., [Kell2001], [Li], [Jaradat]

19

Our Work:

¤  Uses model management to construct workflows that address compliance scenarios especially for automotive systems.

¤  Uses specific operators defined in related work to construct larger and more impactful workflows.

¤  Out of scope: we do not focus on safety case construction, instead, we assume presence of a safety case and focus on constructing model-based workflows for scenarios such as safety case impact assessment and reuse.

20

Next Steps: Tool Support

•  RO6: Tool Support •  Q1: How do we best provide tool support for our approaches?

•  Q2: How can the compliance management workflows we propose be implemented on top of an existing model management tool? If so, what type of adaptation is needed?

•  Q3: What are the missing components that will allow us to work with and manage compliance ecosystems which are heterogeneous in nature?

21

Next Steps: Validation

•  RO7: Validation •  Q1: Given the tool support, how do we validate the correctness of

the approaches presented as model management workflows?

•  Q2: How do we evaluate their effectiveness with respect to efficiency, cost savings and applicability (e.g., how they fit into a real-world software development and safety assessment process)?

22

Summary

¤  The “thesis” of my thesis: The state of practice in compliance management can be made more effective using model management.

¤  So, what is the “impact”? ¤  Industrial:

¤  reduce cost of managing assurance cases as model-based systems evolve (e.g., composition, incrementality, product lines)

¤  Academic: ¤  many open problems stated in “Model Management for Regulatory

Compliance: a Position Paper” [MiSE’16] ¤  MMINT: Model Management INTeractive tool

https://github.com/adisandro/MMINT/

23

Managing Assurance Cases in Model Based Software Systems

Sahar Kokaly McMaster University and University of Toronto, Canada

kokalys@mcmaster.ca

Supervised by: Tom Maibaum (McMaster University) and Marsha Chechik (University of Toronto)

ICSE’17 Doctoral Symposium May 23, 2017

Questions?

Bibliography [DLVara] J. L. de la Vara and R. K. Panesar-Walawege, “Safetymet: A Metamodel or Safety Standards,” in Proc. of MODELS’13. Springer, 2013

[Habli2008] I. Habli and T. Kelly, “A Model-Driven Approach to Assuring Process Reliability,” in Proc. of ISSRE’08. IEEE, 2008.

[Hamou-Lhadj] A. Hamou-Lhadj and A. Hamou-Lhadj, “Towards a Compliance Support Framework for Global Software Companies,” in Proc. of SEA’07, 2007

[Nejati] S. Nejati, M. Sabetzadeh, D. Falessi, L. Briand, and T. Coq, “A SysMLbased Approach to Traceability Management and Design Slicing in Support of Safety Certification: Framework, Tool Support, and Case Studies,” Information and Software Technology, vol. 54, no. 6

[Ghanavati] S. Ghanavati, A. Rifaut, E. Dubois, and D. Amyot, “Goal-Oriented Compliance with Multiple Regulations,” in Proc. of RE’14. IEEE, 2014

[Luo] Y. Luo, M. van den Brand, L. Engelen, J. Favaro, M. Klabbers, and G. Sartori, “Extracting Models from ISO 26262 for Reusable SafetyAssurance,” in Proc. of ICSR’13. Springer, 2013, pp. 192–207

[Jaradat] Jaradat, O., Bate, I.: Systematic Maintenance of Safety Cases to Reduce Risk. In: Proc. of SafeComp'16. pp. 17{29. Springer (2016)

25

Bibliography ctd. [Panesar-Walawege] R. K. Panesar-Walawege, M. Sabetzadeh, and L. Briand, “Supporting the verification of compliance to safety standards via model-driven engineering: Approach, tool-support an empirical validation,” Information and Software Technology, vol. 55, no. 5, pp. 836–864, 2013.

[Brunel ] J. Brunel and J. Cazin, “Formal Verification of a Safety Argumentation and Application to a Complex UAV System,” in Prof. of SAFECOMP Workshops. Springer, 2012, pp. 307–318.

[Kelly2004] T. Kelly and R. Weaver, “The Goal Structuring Notation – A Safety Argument Notation,” in Proc. of DSN’04, 2004.

[Habli2010] Habli, I., Ibarra, I., Rivett, R.S., Kelly, T.: Model-Based Assurance for Justifying Automotive Functional Safety. Tech. rep., SAE (2010)

[Gallina] Gallina, B.: A Model-Driven Safety Certication Method for Process Compliance. In: Proc. of ISSRE'14 Workshops. pp. 204{209. IEEE (2014)

[Kelly2001] Kelly, T.P., McDermid, J.A.: A Systematic Approach to Safety Case Maintenance. J. Rel. Eng. & System Safety 71(3), 271{284 (2001)

[Li] Li, Z.: A Systematic Approach and Tool Support for Assessing GSN-Based Safety Case. Master's thesis, Technische Universiteit Eindhoven (2016)

26

Key Contributions

1.  Model Management for Regulatory Compliance [MiSE’16]

2.  A Model Management Approach for Assurance Case Reuse Due to System Evolution [MoDELS’16, SafeComp’17]

3.  Safety Case Impact Assessment in Automotive Software Systems: An Improved Model-Based Approach [SafeComp’17]

4.  Tool Support and Application in the Automotive Domain [SafeComp’17 and planned SoSym’17]

27

Other Contributions 1.  “Mapping-Aware Megamodeling: Design Patterns and Laws.” Zinovy

Diskin, Sahar Kokaly, Tom Maibaum [SLE’13]

2.  “Enriching Megamodel Management with Collection-Based Operators.” Rick Salay, Sahar Kokaly, Alessio Di Sandro, Marsha Chechik [MODELS’15]

3.  “MMINT: A Graphical Tool for Interactive Model Management.” Alessio Di Sandro, Rick Salay, Michalis Famelis, Sahar Kokaly, Marsha Chechik [Demo Track at MODELS'15]

4.  “Elementary Model Management Patterns.” Sahar Kokaly, Zinovy Diskin, Tom Maibaum, Hamid Gholizadeh [PAME'15]

5.  “Heterogenous Megamodel Slicing for Model Evolution.” Rick Salay, Sahar Kokaly, Marsha Chechik, Tom Maibaum [ME’16]

6.  “Managing Heterogeneous Collections of Related Models.” Rick Salay, Sahar Kokaly, Alessio Di Sandro, Marsha Chechik, Tom Maibaum [submitted]

28