agile practices proven in highly regulated environments by craig langenfeld

© 2011 Rally Software Development and Leffingwell, LLC.

Agile Practices Proven in High

Assurance and Highly

Regulated Environments


Define high assurance?

“High assurance software systems are unique because they must satisfy basic functional service properties that the system intends to deliver, as well as guarantee desirable system properties such as security, safety, timeliness, and reliability.”

2


Regulating bodies…

FDA – Federal Drug Administration ISO – International Standards European Union MEDDEV Drug Controller General of India

– Central Drugs Standard Control Organisation (CDSCO – Medical Devices Division

Health Canada Ourselves (CMMI)

Global Harmonization Task Force – guidance docs IEC, although not a regulation is recognized as a

good standard when developing such things as medical devices 4


“Although the waterfall model is a useful tool for introducing design controls, its usefulness in practice is limited… for more complex devices, a concurrent engineering model is more representative of the design processes in use in the industry. “

From [FDA CDRH 1997] Design Control Guidance for Medical Device Manufacturers

6


Surprise? FDA and IEC Guidance does NOT recommend waterfall

[FDA CDRH 2002] It is important to note, that neither this document, nor CFR820.30 itself, constrains development to single pass, stage-gated, waterfall activities.

From General Principles of Software Validation….. [FDA CDRH 2002]: While this guidance does not recommend any specific life cycle model or any specific technique or method, it does recommend that software validation and verification activities be conducted throughout the entire software life cycle.

From [FDA CDRH 1997] Design Control Guidance for Medical Device Manufacturers : Although the waterfall model is a useful tool for introducing design controls, its usefulness in practice is limited… for more complex devices, a concurrent engineering model is more representative of the design processes in use in the industry.

IEC 62304 medical device standard states: … these activities and tasks may overlap or interact and may be performed iteratively or recursively. It is not the intent to imply that a waterfall model should be used.


Industry myth perpetuated by our own waterfall past?


Regulated environment

Lean / Agile

Software engineering

& SDLC

Craig Langenfeld PMP, [email protected]


Dean Leffingwell

10

Author Executive

Founder/CEO Requisite, Inc. Makers of RequisitePro

Senior VPRational SoftwareResponsible for Rational Unified Process

Founder and CEOProQuo, Inc., Internet identity

Agile Enterprise Coach and Mentor To some of the worlds larger software enterprises

Coach

Cofounder/AdvisorPing Identity, Roving Planet, Silver Creek Systems, Rally Software

Chief Methodologist Rally Software

Founder and CEORELA/Colorado MedtechMedical Device Development


Waterfall Story

11


Where are we going?

➵ Agile Proven within High Assurance➵ Healthcare Example

➵ How?➵ Agile Framework for High Assurance

➵ High Assurance Requirements Model

➵ Artifact generation

➵ Updated Quality Management Systems

13


Agile is already in high assurance

Abbott Laboratories –

– 20 – 30 % fewer defects were found

– availability of working software early on was a significant factor

– “This experience has convinced us that an agile approach is the approach best suited to development of FDA-regulated devices.”

GE Healthcare Goes Agile – Dr. Dobbs article 2010 – “we are making progress and feel that the benefits of our Agile adoption have

been worth the effort. Because of this we are rolling out Agile globally within GE Healthcare”

AFEI DoD Agile Development Conference

– “Agile Methods are in widespread use by the U.S. DoD, Prior … the commercial industry and DoD contractors believed the U.S. DoD was not committed to Agile , an enormously incorrect assumption…”

Sources: Abbott Labs whitepaper: http://www.computer.org/portal/web/csdl/doi/10.1109/AGILE.2009.50. AAMI report: See http://www.aami.org/applications/search/details.cfm?WebID=P1541_D6110DoD Association for Enterprise Information (AFEI): See http://www.afei.org/Pages/default.aspx.(See http://davidfrico.com/afei-2010.doc)

http://www.computer.org/portal/web/csdl/doi/10.1109/AGILE.2009.50

http://www.afei.org/Pages/default.aspx

http://davidfrico.com/afei-2010.doc


Whitepapers and other references

Association for Advancement Medical Instrumentation– TIR - Guidance on the use of AGILE practices in the

development of medical device software

– “Since AGILE is a highly INCREMENTAL/EVOLUTIONARY approach, it can therefore be mistakenly assumed that AGILE is incompatible with the expectations for a medical device software process. “

Blogs…– Scott Ambler – Agile Scaling Model

– Tom Grant – Forrester Analyst

– Dean Leffingwell – Scaling Software Agility Blog15


Agile gets results

16

37-50% faster to market − QSM research

“We experienced a 20-50% increase in productivity.” − BMC Case Study

“ makes the work more enjoyable, helps us work together, and is empowering”

− Medtronic

Quality


Agile drives quality, safety, efficacy

1717

User Stories

CollectiveOwnership

CodingStandards

Refactoring

ContinuousIntegration

SimpleDesign

PairProgramming

Test-DrivenDevelopment

AutomatedTesting

…fewer defects were found − Abbott Labs

Helps us find bugs earlier − Medtronic

… of 131 respondents, 88% said quality was better or significantly better − Shine Technology Survey


AN AGILE, HIGH ASSURANCE LIFECYCLE FRAMEWORK

18


But high assurance development has additional requirements

User Needs

Design Input

Design Process

Design Output

Medical device

Review

Validation

Verification

Source: FDA CDRH 1997 Design Control Guidance for Medical Device Manufacturers

Medical device exemplar: US FDA mandates Software Verification and Validation


So we have additional mandates

VerificationProvides objective evidence that the design outputs of a particular phase of the software development life cycle meet all of the specified requirements for that phase.

20

Sources:Regulation: Code of Federal Regulations 21 Part 830, Subpart C Design ControlsGuidance: General Principles of Software Validation

ValidationConfirmation …… that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled….Since software is usually part of a larger hardware system, the validation … includes evidence that all software requirements have been implemented correctly and completely and are traceable to system requirements.

Code of Federal Regulations CFR 21 Part 830, Subpart C Design Controls mandates device design verification and validation.

You built it right

You built the right

thing


And

Requirements SpecificationA documented software requirements specification (SRS) provides a baseline for both validation and verification. The software validation process cannot be completed without an established software requirements specification (Ref: 21 CFR 820.3(z) and (aa) and 820.30(f) and (g

21

Sources:Regulation: Code of Federal Regulations 21 Part 830, Subpart C Design ControlsGuidance: General Principles of Software Validation

TraceabilityFDA guidelines describe traceability and a primary mechanism to assure that verification and validation are complete and consistent. Traceability. The degree to which a relationship can be established between two or more products of the development process, especially products having a predecessor-successor or master-subordinate relationship to one another; e.g., the degree to which the requirements and design of a given software component match [IEEE]

Code of Federal Regulations CFR 21 Part 830, Subpart C Design Controls mandates a requirements specification.


ProductBacklog

Iteration Backlog

Define

ProductIncrement

ITERATIONMECHANICS

Backlog Grooming

Iteration Planning

Iteration Demo, Review

& Retrospective

Daily Standup

Build

Verify


ProjectInception

Set up ProjectInfrastructure

Design Transfer

VerificationIteration



ValidationIteration N

AG

ILE

PROJECTLIFECYCLE

SystemIncrement

Verification and Validation activities and artifacts driven by QMS

Planning, Analysis,

Architecture, QMS

Production Code

High Assurance Scaled Agile Framework


The User Story

As a <role>I can <activity>

So that <business value>

As an EPAT (Extracorporeal Pulse Activation Technology) technician, (<role>) I can adjust the energy delivered (<what I do with the system>) in increments so as to deliver higher or lower energy pulses to the patient’s treatment area (<value the patient receives from my action>).

Acceptance Criteria

Definition of Done


Implemented byCode

1

User Story

1..*

StoryAcceptance Test

Unit Test

1 1

Verified by

Software Requirements Specification

SCM Repository

Verified by

Test Repository1..*

1..*

Traceability from User Story to Code and Story Acceptance Test


Product Requirements Document

User Story

Feature

Traced to

Software Requirements Specification

Pulse amplitude is adjustable from 1-5 bar

As an operator, I always see the current setting on the display

in .1 bar increments, so I can be confident I’m

delivering the right energy

As an operator, rotating the energy knob past the point where the system is delivering 5 bar

will have no further effect

As an operator, I can adjust the pulse amplitude in .1 bar increments so as to

be able to make small changes to change energy delivered to patient

area

Validating Product Claims

User Story

User Story


High Assurance Agile Backlog Model

Source: Leffingwell. Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise. Addison-Wesley 2011.

© 2011 Leffingwell, LLC.


Validating Features and System Qualities

29


Agile and Quality Management Systems (QMS)

Continuous improvement or (re-)write from scratch

Establish cross-functional QMSscrum team

Run releases and sprints to refine / establish QMS

Design Controls needs to provide flexibility

Software Development Life Cycle (SDLC), Tools, etc. should be specified in the Design and Development Plan (DDP), not in the QMS


Validation Sprint Activities

31

April 11, 2023

Quality Management Strategy

Matt Anderson

Director Program Management


MethodQ/SLIM Overview

IterationsIterations

Release PlanRelease Plan

ReleaseRelease

Initial “Design Input” Signature• Roadmap• User Stories/Capabilities (Epics)• Acceptance Criteria

Final “Design Input” Signature• Solution Level Requirement Document(s)

Final “Design Output” Signature• Asset design artifacts, code, traceability

Tasks to update for each User Story •Solution Level Requirements •Design artifacts


Change Record Management

Release CRRelease CR

ReleaseRelease

Capability CRCapability CR

Initial “Design Input” Signature• Roadmap


Final “Design Input” Signature• Current Solution Level Requirements

Final “Design Output” Signature• Solution Level Test Scenarios• Test Evidence• Solution Level Technical Artifacts


Change Record Management


ReleaseRelease


Initial “Design Input” Signature• User Stories• Acceptance Criteria• Initial Visual Design


Final “Design Input” Signature• Updated Solution Level Requirements• Visual Design

Final “Design Output” Signature• Test Scenarios• Test Evidence• Code/Code Review• Technical Artifacts as needed


Capability CRCapability CRCapability CRCapability CR

Parent/Child Relationships

Capabilities cannot span releases, but can span iterations CR can be both a Child and a Parent Each CR must have completed Design Input and Output

– Initial Design Input for Child can be covered by the Parent



Capability CRCapability CR Capability CRCapability CR


Agile extremism does not help (working software over documentation)


Agile and most regulating bodies are not at odds

38


Satisfy compliance while preserving

Agility.


Implement the appropriate degree of rigor


Agile – Perfect for High Assurance

“Agile isn’t just good for High Assurance development – it’s

better than traditional methods.”

- Tom Grant, Forrester Group


Live long and prosper!

Craig Langenfeld

[email protected]

cameo by Matt Anderson

43