moving to agile in an fda environment an experience report august 27, 2009
TRANSCRIPT
Moving to Agile in an FDA Environment
An Experience ReportAn Experience Report
August 27, 2009August 27, 2009
An Experience ReportSlide 2
Introduction
• Available for download at http://agile2009.agilealliance.org/
• Agile Resources– Agile Manifesto http://www.agilemanifesto.org/– Agile Alliance http://www.agilealliance.org
• Agile Books – http://www.agiletek.com/thoughtleadership/books
“Agile Software Development” by Alistair Cockburn
“Agile Project Management” by Jim Highsmith
An Experience ReportSlide 3
Outline
• The Abbott Experience
• Lessons Learned1. The least burdensome approach?
2. A Risk Based Approach
3. Control what you don’t know, don’t let it control you
4. Dispense with ceremony
• Results
An Experience ReportSlide 4
The Abbott Experience
• Comparing two FDA-regulated medical device projects– One not agile, one agile
– Class III devices (most stringent)
– Results of agile adoption• Lower cost
• Shorter duration
• Better, less prescriptive test cases
• Accommodated change
• Higher quality
• See paper “Adopting Agile in an FDA Regulated Environment”
An Experience ReportSlide 6
Software Development Lifecycle
An Experience ReportSlide 7
Iteration Model
Product Definition
Feature SummaryFeature DescriptionsSoftware Requirements ...
Planning
Software Development PlanSoftware Process DefinitionSoftware Engineering Development PracticeSoftware Test PlanSoftware Configuration Management PlanSoftware Quality Assurance PlanSoftware Measurement PlanCapability Allocation MatrixSoftware Schedule...
High Level Design
High Level Design DescriptionSoftware Architecture DescriptionUser Interface Design DocumentUser Interface Framework...
Software Construction
Design
Feature Detailed DesignPresentationBusiness RuleData AccessPersistence
Test Design...
Implementation
PresentationBusiness Data AccessDatabaseTest Cases...
Verification
Unit TestIntegration TestCode InspectionDesign Review...
Stakeholder Review
ApplicationRequirementsFeature DescriptionsScenario Descriptions...
Software Confirmation
System Test
Regression AnalysisIntegration TestAcceptance TestFunctional Testing...
Software Release
Version Description (Release Notes)Traceability MatrixInputs -> OutputsValidation / Verification Data...
Software Characterization
Iteration Model
An Experience ReportSlide 8
QSR Ref. ISO Ref. Design Control Element Documents 1. 820.30(b) 7.3.1 Design and development
planning Design and Development Plan
2. 820.30(c) 7.3.2 Design input System Requirements Specification Safety Risk Analysis
3. 820.30(d) 7.3.3 Design output System Design Description Source code Drawings/Schematics
4. 820.30(e) 7.3.4 Design review System review reports Design review reports
5. 820.30(f) 7.3.5 Design verification Verification Test Procedures 6. 820.30(g) 7.3.6 Design validation Validation Test Procedures
Test Summary Report Requirements traceability matrix
7. 820.30(h) NA Design transfer Design transfer procedure 8. 820.30(i) 7.3.7 Design changes Change control procedures 9. 820.30(j) 4.2.3-4 Design history file Document control procedures
Sample Design Control Documents
Courtesy of: Certified Compliance Solutions, Inc.
Lessons Learned
An Experience ReportSlide 10
Least Burdensome
We [FDA] are defining the term “least burdensome” as a successful means of addressing a premarket issue that involves the most appropriate investment of time, effort, and resources on the part of industry and FDA.
--The Least Burdensome Provisions of the FDA Modernization Act of 1997: Concept and Principles; Final Guidance for FDA and Industry
An Experience ReportSlide 11
A Risk Based Approach
Courtesy of: Certified Compliance Solutions, Inc.
An Experience ReportSlide 12
• IEC 62304 section 5.1.1, page 31: Note 1: The software model can identify different activities for different software items according to
the safety classification • FDA General Principles of Software
Validation, page 7, section 3.1.2: The level of confidence, and therefore the level of software validation, verification, and testing effort needed, will vary depending on the safety risk (hazard) posed by the automated functions of the device.
Risk Based Approaches
Courtesy of: Certified Compliance Solutions, Inc.
An Experience ReportSlide 13
SOFTWARE DOCUMENTATION
MINOR CONCERN
MODERATE CONCERN
MAJOR CONCERN
Verification and Validation Documentation
Software functional test plan, pass / fail criteria, and results.
Description of V&V activities at the unit, integration, and system level. System level test protocol, including pass/fail criteria, and tests results.
Description of V&V activities at the unit, integration, and system level. Unit, integration and system level test protocols, including pass/fail criteria, test report, summary, and tests results.
FDA Reviewer Guidance 2005
Courtesy of: Certified Compliance Solutions, Inc.
An Experience ReportSlide 14
Control What You Don’t Know, Don’t Let It Control You
• What you do know:– A typical medical device is developed over a 3-5 year
time horizon– It is a myth that you can predict in detail your end
product requirements up-front– Start with a core set of features that you must implement– Implement the core features first– Defer the most volatile features as long as possible
• Iterative approach allows the team to:– Manage scope and limit feature creep– Negotiate scope and tradeoffs with key stakeholders
• “At time of commercial launch, a number of features, once thought to be essential, were not included. Some were deferred as long as three years. Nonetheless, the product was considered highly successful and trading off nice to have features for three years of sales is an easy choice.”
An Experience ReportSlide 15
Dispense With Ceremony• If it is not adding value, and it is not required,
do not do it• The design history files should contain the
minimum set of documentation that satisfies the regulatory requirements
• There will be other activities that you will want to document, no need to include in design history file, make sure they add value and do it in a least burdensome way
• Avoid doing things because “that’s the way we’ve always done it”
• If it feels like you are wasting your time you probably are
Results
An Experience ReportSlide 17
Results
• High visibility– Easier to manage and control– Far fewer surprises
• Lower cost and shorter duration– Estimated schedule and team size reduction of 20%-
30%– Estimated cost savings of 35%-50%
• Higher quality– Availability of working software facilitated continuous
testing instead of back loaded V&V– Resulted in fewer overall defects, especially at the end
of the project• Better work life balance and team morale
– Project death marches are rarer because the issues are surfaced as you go and are managed accordingly, not all saved up for the end of the project
Q & A
An Experience ReportSlide 19
Tim Hughes J.R. JenksManaging Partner Managing [email protected] [email protected] 847-699-2250
Thank You
John SkachSenior Technology [email protected]
Rod RasmussenDirector, Informatics & Software Systems [email protected]