6th Annual Event –
Development Lifecycle
Management For Medical
Device Summit
orcanos 2013
From Generation Based Product
to a Brand New One
Accelerating Scrum through advanced
methodologies, testing & collaboration
Boaz Ben Nahum
Senior Software
Engineer
Hagai Livni
Head of Software
Validation
May 2013
− 3 −
Agenda
• Given Imaging Products and the PLCM process
• What Agile process was used in the Generation based
product
• Moving Forward to a new product using SCRUM
methodology “by the book”
• User stories
• Agile Planning
• Continuous integration
• Test planning and executing
• Tools in use
• How do we move forward in future projects
(opportunities for improvement)
− 4 −
Agenda
• Given Imaging Products and the PLCM process
• What Agile process was used in the Generation based
product
• Moving Forward to a new product using SCRUM
methodology “by the book”
• User stories
• Agile Planning
• Continuous integration
• Test planning and executing
• Tools in use
• How do we move forward in future projects
(opportunities for improvement)
− 5 −
Given Imaging Products
• Established in 1998, and started selling our products in
2001
• Given develops, produces and distributes medical
systems for the gastrointestinal community, which
helps diagnose pathologies through visualization of the
GI tract as well as manometry measuring products
• Given products are distributed in more than 75
countries
• Just released our 8th generation of the software
• This project was developed for the first time in an
incremental approach (SCRUM)
• Started a completely new big project
− 6 −
Given Imaging Products
� PillCam COLON 2
� Safe, minimally invasive,
sedation-free, patient-
friendly modality to
visualize the colon and
rectum
� PillCam ESO 2
� Capsule endoscopy
procedure for visualization
of the esophagus
� The only capsule-
based pH test for
GERD
� Ambulatory pH testing
gold standard
� High-Resolution
Manometry with
physiology
visualization and data
analysis software for
identification of
motility disorders
Fully-integrated GI solutions
� Ingestible, wireless
motility capsule using
sensor technology to
measure pH, pressure
and temperature from
within the entire GI
tract
� Complete reflux
diagnostic solution for
both capsule and
catheter-based testing
4
� PillCam SB 2
� Most Widely used capsule
endoscpe for visualization
of the entire small bowel
� The standard of care in
numerous conditions
− 7 −
Given Imaging Workflow
Define features & effort
PM
MKT
Clinical analysis
Feasibility &
Fine-tuning features
SW R&D
Algorithms\HW
Clinical analysis
PM
MKT
SQA
CS
Extensive QA &
Final fine-tuning
SW R&D
V&V= SQA + Clinical
analysis (Alpha)
PM
MKT
3rd level support &
Field questions
CS
PM
MKT
− 8 −
The “Traditional” Process
Implementation – R&D
Verification - SQA
Internal Val.
(clinical staff)
Clinical Trials
‘last feature in’
full testing cycle
vs. regression
when to start? when to end?
− 9 −
The Agile Work Flow
Implementation + Verification
R&D, PM, and SQA
Bug fixing + Verification
R&D, PM, and SQA
Clinical Trials
‘last feature in’
full testing cycle
vs. regression
Internal Val.
(clinical staff)
− 10 −
Agenda
• Given Imaging Products and the PLCM process
• What Agile process was used in the Generation based
product
• Moving Forward to a new product using SCRUM
methodology “by the book”
• User stories
• Agile Planning
• Continuous integration
• Test planning and executing
• Tools in use
• How do we move forward in future projects
(opportunities for improvement)
− 11 −
Previous* Status of SW at Given Imaging
• The 8th generation of our main software product,
“RAPID for PillCam®”, was developed, for the first
time, using an iterative approach, adopting Scrum
methodology
• 3-week long sprints were executed, practicing:
• Sprint planning using developer tasks
• Bi-Weekly stand-up meetings
• Task-oriented test execution using exploratory testing
method
• Sprint “retrospective” meetings
• Releasing sprint SW version to internal clients (marketing),
using Citrix XenApp
But…
* Previous = End of 2011
− 12 −
What can be improved?
• Plans & Specifications
Development Plans
• The planning unit is a
“Development Task”
• Development Tasks are
detailed using MS Project
and GANTT charts
• Tasks are “engineering-
centric” and can be hard to
test
SW Specifications
• The specification unit is a
“Software Requirement”
• SW requirements are detailed in
a monolithic SRS MS-Word
document
• SRS written by a single person
and available too late in the
process
The two types of entities are not connected!
− 13 −
Agenda
• Given Imaging Products and the PLCM process
• What Agile process was used in the Generation based
product
• Moving Forward to a new product using SCRUM
methodology “by the book”
• User stories
• Agile Planning
• Continuous integration
• Test planning and executing
• Tools in use
• How do we move forward in future projects
(opportunities for improvement)
− 14 −
Moving Forward to a New Product
• Using SCRUM methodology “by the book”:
• User stories
• Agile planning
• Continuous Integration
• Test Planning and Execution
• Tools
• ALM – Application Lifecycle Management system
• Project Portal
• The most significant change was the introduction of
user stories into the process.
− 15 −
Why User Stories?
• User stories
• Emphasize the user’s goals, not the system’s attributes
• Can be verified by SQA
• Can be demoed to stake holders
• Bridge the communication gap between PM, Dev & SQA
• Support and encourage iterative development
• Provide means for flexible and efficient planning
• Each user story satisfies two goals at the same time:
• It describes the software requirement in detail
• It also describes a piece of development work for planning
purposes
• User stories bridge the gap between plans and
specifications!
− 16 −
How User Stories Are Applied?
• They are written collaboratively & incrementally by entire team
• Initial form may include a single sentence.
• That sentence may be refined during sprint planning
• Details emerge during the sprint by discussions between
developers, SQA & PM, and are written inside the User Story as
“Conditions of Satisfaction”
• The detailed user story serves as a first class Software
Requirement within the ALM system
• The list of “Done” user stories forms the software specification
As a <role> I want to <goal> so that <benefit>
Conditions of Satisfaction
1. …
2. …
3. …
− 17 −
Agile Planning Phases
• Backlog grooming
• Sprint planning
− 18 −
Backlog Grooming
• User stories are added to the backlog periodically by
individuals (developers, team leader, product
manager), in “New” status.
• We hold regular “backlog grooming” meetings where
we:
• Review new stories and either approve or reject them.
• Questionable stories are moved to “Pending Decision” status
for later discussion with PM
• Estimate user stories using story
points – playing "Planning Poker".
• Participants: SW team leader,
SW Director, Scrum Master
− 19 −
Backlog Grooming - How to Estimate Stories?
• User stories are estimated relatively using abstract
measures: story points
• Story point have no meaning per se, only relatively to other
stories – a 20 point story represents twice amount of work than
a 10 point story
• Sprint velocity
• Measured empirically (using points) for all past sprints
• It is then used to determine sprint capacity during planning
(with some adjustments)
• Estimation may be inaccurate for single story & single
developer• But tend to be more accurate for a team of developers over a
sprint
− 20 −
Sprint Planning
• Sprint Planning Meeting is held before each sprint
Participants: SW team leader, SW Director, Scrum Master, Product
Manager
Agenda:
• Decide on sprint goals.
• Set the number of story points for the coming sprint (based on
velocity, team’s availability, vacations, etc.).
• Allocate stories for the coming sprint.
• Assign stories to developers.
− 21 −
User Story Workflow
New
Open
In Work
Implemented
Done
Review
& Approve
Allocate
& Assign
Implement
& Test
Verify
− 22 −
Continuous Integration - Daily Process
• Daily automated process includes:
• Building the latest version of the software
• Deploying the built software on a dedicated server
• Executing the following test sets:
• Unit level tests
• Component level tests
• System level tests
• Notifying all relevant parties with the results
• The daily updated server is used for exploratory testing
and verification of user stories during the sprint
− 23 −
Continuous Integration – On Demand
• Team City System – by JetBrains• For every source code commit, performs full build and runs unit tests
• Average duration: 4 minutes
• If any unit test fails, developer is instantly emailed, and can fix the
problem even before it reaches daily build.
• Allows also monitoring build & tests stability over time
• Visual Studio add-in allows “remote run”: sending code changes for
build & test on the server, before committing to source control
− 24 −
Agenda
• Given Imaging Products and the PLCM process
• What Agile process was used in the Generation based
product
• Moving Forward to a new product using SCRUM
methodology “by the book”
• User stories
• Agile Planning
• Continuous integration
• Test planning and executing
• Tools in use
• How do we move forward in future projects
(opportunities for improvement)
− 25 −
Test Planning and Execution
• Test Planning and Designing
• Exploratory testing > design within execution
• What to execute per sprint / where to focus
• Automation testing implementation
• Automation testing advantages
• Constrains and limitations
• Identifying the tests to be automated
• When to start automate (cost effective)
• Creating different test sets for different purposes
• Using KDT test design methodology
− 26 −
Test Planning and Designing
• Exploratory testing > design within execution during the
sprint
• Specific module testing (Exploratory testing - linked to the user
stories) on the daily build
• SQA-development interaction during iterations
• What to execute per sprint released version / where to
focus
• Pre-determined Baseline testing for the sprint’s release version
• Risk based tests execution
• Execution of all other tests in a FIFO method (executing the
earliest previous test first)
• Executing a full test cycle (all test suite) in pre determined
milestones within project
− 27 −
Automation Testing - Advantages & Limitations
• Automation testing advantages
• Exposes defects in the unit level at early stages
• Reduces testing cycles duration
• Overnight test run
• Ideal for having a ‘green light’ indication to enter a full testing
cycle
• ‘Clears’ the manual testers time to do more complex tests
• Constraints and limitations
• In the unit testing level – time allocation within the
development process
• In the components testing level – prof. resources are needed
• In the system testing level – execute the exact scenario vs. the
randomness which exists in manual tests
− 28 −
Automation Testing – What to Automate
• Identifying the tests to be automated
• Good test cases for automation are ones that are running
frequently and require large amounts of data to perform the
same action
• Test cases that require multiple data sets
• Frequently used functionality that introduces high risk
conditions
• Test cases that are impossible (or almost) to be performed
manually
• Test cases that are exposed in their execution to human error
• Test cases that take a lot of effort and time when executed
manually
− 29 −
Automation Testing –When Do We Automate
• Unit testing level
• As part of the development
• Unit tests can be created based on found and treated defects
• Components testing level
• Design in parallel to the components development
• Execute once the component was implemented AND test data is
available
• System testing level – once a GUI objects' properties
has been finalized
− 30 −
Automation Testing Implementation
• Creating different test sets for different purposes
• Unit level
• Creating the tests as part of the implementation of the relevant
function (part of the developer’s tasks)
• Using Nunit ,unit-testing framework, enabling execution of unit
tests in two modes: GUI Runner, and Console Runner
• Running the tests as a task on the build machine and generating a
report with email
• Component level
• Executed a set of the tests on the daily build
• Executing the full test suite at the end of the sprint
• System level
• Test Objects’ Repository qualifying
• Functional - Sanity test set
• Functional - Full test suite
− 31 −
Automation Testing Implementation - cont.
• Using KDT test design methodology
• QTP with KDT - KDT (Keyword Driven Testing) tool, running a
dedicated engine within the QTP which call other QTP
functions
• Bridging between the ‘tool experts’ and the ‘domain experts’
• Helping in design and execute tests sequences and test
scenarios
• Enabling ‘manual testers’ with basic QTP understanding to
design automatic tests
• QTP manage the tests’ infrastructure - object and function level
• KDT manage the tests’ design - sequence and test scenario level
− 32 −
Tools - ALM
Planning• User stories
• Developer’s Tasks
• Design Specifications
• Test cases
• Tests Sets for execution
Managing• Tasks and Stories ‘burn down’
status
• Tests Planning status (workflow)
• Product requirement’s coverage
Risks vs. SR
SR vs. SDD
SR vs. TC
• Test execution
• Recording automation test
execution results
• Defects / change requests
Records generating for the DHF
− 33 −
Tools – Project Portal
• A SharePoint-based portal is used for collaboration of
team members
• Allows anyone to contribute using Wiki-style web pages
• Includes:
• Educational materials & Best Practices
• Description of software architecture
• Deployment guide
• Description of Scrum Process & Ceremonies
• Primary usage: maintain sprint information
• Sprint Schedule
• Sprint Goals
• Sprint Retrospective Feedback
− 34 −
Tools – Project Portal
− 35 −
Agenda
• Given Imaging Products and the PLCM process
• What Agile process was used in the Generation based
product
• Moving Forward to a new product using SCRUM
methodology “by the book”
• User stories
• Agile Planning
• Continuous integration
• Test planning and executing
• Tools in use
• How do we move forward in future projects
(opportunities for improvement)
− 36 −
How do we move forward
• Enforcing Done, Done approach
• Fully implemented
• Tests approved (after R&D review), executed and passed
• All defects are closed
� Shaping the process of test cases review by Dev. Team
� Improved feedback throughout the process by
� End Users
� Marketing
• Level of test coverage of a feature / function – where do
we stop?
• Building up automation test within the test design by
manual testers (“domain experts”) using the KDT
platform
− 37 −
How do we move forward – cont.
� Using automatic tools to measure code coverage (unit
and system level testing)
� Static code analysis tools use
� Adopting Test driven development approach
• Using the BI analysis tool for better planning,
controlling and reflecting progress to management
� Integration between the source control system and the
ALM system
• Shortening formal SQA phase (stabilization period) –
time from feature freeze to product release