agile endgames

8/14/2019 Agile Endgames

1/84

Releasing Agile Productsin the Enterprise

Exploring the Challenges as Agility Scales

Bob Galen

President & Principal Consultant,

RGCG, LLC Leading you down the path of agility

www.rgalen.com [email protected]
http://www.rgalen.com/mailto:[email protected]:[email protected]://www.rgalen.com/


2/84

v1.1 -- November 2008Copyright 2009 RGalen Consulting Grp,

LLC 2

Introduction

Bob Galen Somewhere north of 20 years experience

Various lifecycles Waterfall variants, RUP, Agile, Chaos, etc.

Various domains SaaS, Medical, Financial, Computer Systems, andTelecommunications

Developer first, then Project Management / Leadership, then Testing

Pieces of Scrum in late 90s; before Agile was Agile

Agility @ Lucent in 2000 2001 using Extreme Programming

Formally using Scrum since 2000 Most recently at ChannelAdvisor as Dir. Product Development and Agile

Architect/Coach (2007-2008)

Connect w/ me via LinkedIn if you wish

Bias Disclaimer:Agile is THE BEST Methodology for Software Development

However, NOT a Silver Bullet!


3/84


LLC 3

Outline

Introduction1. Agile Scaling & Scrum

2. The Evolving Role of the Product Owner

3. Criteria

4. Scrum-in-Testing

5. Testing Challenges

6. Q&A


4/84


LLC 4

Introduction


5/84


LLC 5

Scrum

Well be using Scrum as the Agile reference framework for

the class

Agile Project Management Framework

Roles:

Scrum Master, Product Owner & Team

Sprint iteration (15 30 days)

Product Backlog a prioritized list of: requirements, work to do, user

stories Sprint Planning mapping PB to tasks

Daily stand-up (15 minutes, Pigs & Chickens)

Sprint Review (demo) & Retrospective


6/84


LLC 6

Process Overview

30 days

24 hours

Product Backlog

As prioritized by Product Owner

Sprint Backlog

Backlog tasks

expanded

by team

Potentially Shippable

Product Increment

Daily Scrum

Meeting

Source: Adapted from AgileSoftware Development withScrumby Ken Schwaber andMike Beedle.


7/84


LLC 7

Core Agility

Not always an Enterprise Play Short, time boxes; typically 2 weeks

Small, co-located teams; typically 7 +/- 2

100% automated testing run during each sprint

Continuous Customer collaboration

Artifact-lite; collaboration & conversation heavy

Velocity based; not plan-driven Forecasting at a high level; scope always compromised

Change friendly

High levels of transparency (reality)


8/84


LLC 8

Challenge Points

Larger scale implementations: larger & distributed teams

Regulated environments with documentation requirements

Enterprise Architectural consistency

Shared team members

Subject Matter Expertise; Specialized domain expertise

Full-time doesnt make sense Not a prescriptive process (little guidance)

Larger scale manual testing environments (scripted)

PMO-based organizations (portfolio based decisions, earned value,

gated project commitments)

Cost-Schedule-Quality fixed; not Scope!

Agile Customer fully engaged; in real-time


9/84


LLC 9

Aspects of Enterprise Agile

There are really 4 aspects to Scaling Agility within the Enterprise:

Organizational larger teams, distributed teams, and outsourced teams

Products larger scale projects, interoperability, usability &consistency, and forecasting

Development dependencies & integration, varying processes, and

cross group collaboration

Testing & Deployment regression, regulatory, integration, productmaturation visibility, and production deployment readiness

This section focuses on these areas from all four perspectives, with

a strong emphasis on Products & Testing


10/84


LLC 10

Aspects of Enterprise Agile

Industry lessons have lagged in larger scale Agile implementations

Its not the sweet spot for them and we seem to be mostly on our own

In the Enterprise, Scrum is leading the way in providing guidancetowards scaling, but so far its sparse in nature

Ken Schwaber published Enterprise Scrumin 2007

Dean Leffingwell published Scaling Software Agilityin 2007 Jeff Sutherland has taken the lead in defining and sharing lessons

learned

There are still gaps from a Product Owner perspective although the

Certified Scrum PO class tries to help

Larger scale implications have been ignored to-date thus thissection


11/84


12/84


LLC 12

Siemens Medical Systems

Size Case Study

1200 developers

300 testers

Distributed across North

America, Europe, India

At any one time: 100 120Sprints going on at once

3 major product lines

150 Scrum Masters

Matrix - Product Owner & QA

organizations

(S1) - Scrum Teams Focused on the work

(S2) - Scrum of Scrums

Focused on cross-team, crossproduct issues & integration

(S3) - Scrum of Scrums of Scrums

Focused on cross-product line

integration, compliance, PMO,

processes

(S4) Focused on Leadership of

Scrum adoption

Agile strategy


13/84


LLC 13

ChannelAdvisor

Size Case Study

60-70 developers

15-20 testers

Raleigh NC, Atlanta GA,

Limerick IE, Russia

At any one time: 8-10 Sprintsgoing on at once

3 major product lines; 1 SaaS

deployment instance

12 Scrum Masters; 6-8 ProductOwners

Separate Product, QA, and

Development organizations

(S1) - Scrum Teams Focused on the work (daily)

(S2) - Scrum of Scrums

Mostly active at integrationrelease points (daily, normallyweekly)

(S3) - Scrum of Scrums of Scrums

Focused on staffing &

contention; forward looking x 2-

3 sprints, (on demand)

(S4) Focused on Leadership of

Scrum adoption

Agile strategy, (on demand)


14/84


LLC 14

Aspects of Enterprise Scaling

Behind the Scrum of ScrumsAnd there are roles not defined behind the SoSoS, for example

What are the sorts of conversations & activities that occur at eachlevel? Are there even levels?

Who guides the process, tools, & techniques for consistency? Or doyou even care?

How do you integrate agile processes with traditional business and

operational processes?


15/84


LLC 15

Aspects of Enterprise Scaling

Behind the Scrum of Scrums What are the hierarchies behind the SoSoS?

Scrum Master(s)

Product Owner(s)

Lines of business

Team collaboration resource management, allocation, and budgeting

Labs & equipment sharing

Reporting & release coordination

Quality, measurement, and governance

And how do they operate, integrate, and provide consistency w/o command-

and-control?


16/84


LLC 16

Typical S2

Charter

Coordinate amongst various Scrum teams at an integrated product level

Product integration, testing

Resource sharing & constraints (people)

Resource sharing (hardware & tools)

Operationally - Scrum dynamics

Scrum Master of the S2

Scrum Masters gather, potentially daily stand-up

Discuss cross-team progress, dependencies, impediments

Focused towards the next, integrated product release point

Managing towards short term roadmap deliverables


17/84


LLC 17

Typical S3

Charter

Product Backlog / Roadmap definition, 6-12+ months

Product integration, testing, and consistency checks (Compliance,Usability, UAT)

Resource sharing & constraints (people) and (hardware & tools)

Budget / cost management

Operationally - Scrum dynamics Periodic (monthly, quarterly) meetings

Rotating Scrum Master of the S3, from within the leadership team

Rarely daily; typically weekly meetings

Discuss adjustments required to execute to the roadmap; cross-team &cross-product line trade-offs

Focused towards to effectively managing the portfolio


18/84


LLC 18

Typical S4

Charter

Evangelizing & Guiding the Agile adoption practices, evolution, meta-

retrospectives Organizational focus:

Corporate leadership understanding & adaptation

Cross-functional understanding & adaptation (ALL functions)

Operationally - Scrum dynamics

Typically monthly meetings; Evolution Backlog / road-map

Rolling up retrospectives; enabling change

Training; Scrum Master & Product Owner focus groups

Focused towards to effectively managing the evolution of agility

Also deriving KPAs and metrics for soft/hard ROI


19/84


LLC 19

Exercise

Gather in small groups; better that you are from the same

company/organization OR those with similar characteristics

Using the Scrum of Scrums notion and Scrum as your framework,please define:

1) A hypothetical SoS hierarchy for your organization2) Detail the format (timing, attendees, focus, etc.) for each levelsort

of the charter

3) Detail some of the conversations that might occur at each level

1) Status or state; progress & plans

2) Impediments

3) Improvement

4) Identify your primary impediment's to implementing this SoS


20/84


LLC 20

What Part do Tools Play

in the Enterprise?

Enterprise Drivers Compliance / artifacts Sheer team size

Distributed teams

Project Management / PMO

Budgeting

Project complexity

Communications

Status, metrics, and reporting

Point #1 Agile

Methodologies are not toolscentric

Point #2 Enterprise Agility needs something to helpcoordinate larger scale teams

Point #3 So, Use tools that are as simple

as possible

And, that dont get in the

way of your face-to-facecollaboration

Go out of your way toreinforce collaboration


21/84


LLC 21

Agile Tools

Low Fidelity 3x5 Cards, Post-it Notes

planning on walls

Web cameras, conference

phones, skype

Webex, GotoMeeting, Wiki

Whiteboards, Corkboards

Swim lane management

Team rooms

Continuous Integration

(CruiseControl), Automated

testing FitNesse, xUnit

Medium Fidelity Jira or other issue/defect tracking

Advanced wiki functionality

SharePoint

PlanningPoker, CardMeeting,

ScrumWorks, TargetProcess

High Fidelity Rally

VersionOne

Mingle

Microsoft Team Foundation

Server (TFS)


22/84


LLC 22

Aspects of Agile Scaling

Jeff Sutherland Parallel Scrum Sutherland is leading the way toward modifying Scrum towards

greater efficiency

Scrum levels A, B, and C

Sutherland used Type C Scrum for 3-4 years at PatientKeeper

Sort of the Nirvana Scrum state, anyone else at Type C?

The key point is the organizational change dynamics

Sprint transition time compression

Forward thinking towards staging & delivery Parallel --- Everything!

Organization-wide change implications including Testing & Quality


23/84


LLC 23

Scrum Evolution

Parallel Pipelining Type A Scrum

Isolated cycles of work

Type B Scrum Overlapping Iterations

Backlog continuously

refreshed Reduced staging times

Type C Scrum All at once, multiple

simultaneous releases

Organization of a Meta-Scrum


24/84


LLC 24

Evolving Role of the Product Owner


25/84


LLC 25

Product Owners

Their Evolving Role Within each Scrum, the PO is typically narrowly focused on crafting

the Backlog, engaging in progress, and reviewing sprint results

However, as Scrum scales, the PO team needs to become morefocused on:

Product Line Evolution Meta Backlog and coordination across Sprint

teams, strategy development & execution, resource load-balancing, andbudgeting

Cross-Team Planning SoS coordination, linked goals and backlogwork, delivery integration, and staged (forward-thinking) planning

Delivery Dynamics timing, marketing, packaging, interrelationships,

customer feedback, and achieving production quality

All of course with the teams delivering the product


26/84


27/84


LLC 27

Product Owners

Coordination Workflow Scrum of Scrum of Scrums

The PO organization mustcoordinate

Feature timing & workflow

Quality & integration workflow

Product maturation andrelease readiness

Production deployment

In conjunction with technology andproject leadership

While often interfacing to operations andcustomer facing organizations

Product Families

Applets

Applications

Products

Stories

Features

DevelopmentWorkflow

Testing &Evolving

Maturation

ProductIntegration


28/84


LLC 28

Product Owner

Planning Levels in Large-Scale Agile Projects

Release Plans

Iteration Plans

Daily Plans

Product Vision

Product Roadmap

Yearly by Product Owner (s)

Semi-Annually by the Product Owner (s)

Quarterly by the Product Owner & Team (s)

Monthly or bi-weekly by the Team (s)

Daily by the team members

SmallTo

Large


29/84


LLC 29

Release Train Management

Iterative model with a releasetarget

Product centric

Focused on a production

push/release

Synchronized Sprints acrossteams

Some teams are un-synchronized, but leads to lessefficient cross-team (product)interactions

Continuous Integration is theglue

Including automated unit andfeature tests; partial regression

Notion of a Hardening Sprint Focused more on Integration &

Regression testing

Assumption that its mostlyautomated

Environment promotion

Define a final Hardening Sprintwhere the product is readied

for release Documentation, Support,

Compliance, UAT, Training


30/84


LLC 30


Motivational Driving Forces Vision

Schedule for delivery

Feature set(s)

Goals for each team

Themes

Which teams will be working on what components

Packages of User Stories

Prioritized by business value & need

End-to-End Use Cases

Integration focused execution


31/84


LLC 31


Internal Driving Forces Customers ability to accept the release timing

Value being delivered within the release purely scope

Hardening Sprint (production readiness reality)

Time, Complexity & Size

Levels of Test Automation

Compliance & Industry

Internal team readiness

Customer support

Sales & Marketing readiness

Overall documentation & training


32/84


LLC 32

Meta-Backlog to Backlog Translation

---------------

Enterprise

Architecture Product line

Features sets andThemes

Targeted releasepoints

Integration &testing

Documentation,support training,

OPS readiness

Regulatory steps---------------

------

---

---

---

---

------

---

---

---

---

---

---

------

---

---

---

---

------

---

---

---

---

---

------

---

---

---

---

------

---

---

---

---

---

------

---

---

---

---

---

---

Meta Backlogs are typically focused at multiple teams, working on a

unified set of work, targeted for release

It can also coordinate across separate product dependencies in ashared codebase

Or simply manage the shared codebase

Sprint

Sprint

Sprint

Sprint


33/84


LLC 33

The Agile Release Train

Synchronized

Iterate

Iterate

Team 1

Team 2

Team 3

Team 4

Iterate

Iterate

Harden Iterate Iterate Iterate

X-teamHarden

Harden

Harden

Harden

Iterate Iterate

Iterate Iterate

Iterate Iterate

Iterate Iterate Iterate Iterate

Iterate

Iterate

Iterate

InternalRelease

ExternalRelease

Docs,Training,

Support,UAT,

Comp.

Team n






34/84


LLC 34

The Agile Release Train

Example: eCommerce / SaaS Model

Iterate

Iterate

Team 1

Team 2

Team 3

Team 4

Iterate

Iterate

Harden

X-team

Harden

Docs,Training

Harden

Harden

Harden

Iterate Iterate

Iterate Iterate

ExternalRelease

Team 8





10 days 10 days 5 + 2 days

Rinse

&

Repeat

Environment

EvolutionDev + QA Dev + QA QA -> Staging Production


35/84


36/84


LLC 36

Establishing Goals & Criteria


37/84


LLC 37

Why its Crucial?

Agile teams are essentially self-directed, so plans dont drive

behavior or success

People do and Goals drive the Team!

The teams then swarm around the goals, using their creativity andteamwork to figure out:

Whats most important

How to achieve it

Always looking for simple & creative20% solutions


38/84


LLC 38

Lean Principles

Most important aspects first

KISS principle; no Gold-plating

Small deliverables; worked on serially

Deliver End-to-End behavior

In thin Slices

UI to Backend DB Driven to done, into inventoryas completed components, as soon as

possible!

Avoiding 90% done syndrome

Integration collaboration

Rarely re-visited; mindset is to ruthlessly minimize rework


39/84


40/84


LLC 40

Story Acceptance

Each User Story should have acceptance criteria as part of the card

They should focus on the verifiable behavior, core business logic,key requirements for the story

Typically, they are crafted by the Product Owner Leveraging skills of Business Analysts and Testers

Story acceptance tests are normally automated and run as part of

feature acceptance AND regression FIT & FitNesse are among the Open Source tools that enable this


41/84


LLC 41

User Story

Examples

As a dog owner, I want to sign-up

for a kennel reservation over

Christmas so that I get a

confirmed spot

Verify individual as a registered pet owner

Verify that preferred members get 15% discount on basic service

Verify that preferred members get 25% discount on extended servicesand reservation priority over other members

Verify that past Christmas customers get reservation priority

Verify that declines get email with discount coupon for future services


42/84


LLC 42

Iteration / Goal Acceptance

Each Scrum Sprint has a Product Owner determined goal

Usually sprint success is not determined by the exact number ofcompleted stories or tasks

Instead, what most important is meeting the spirit of the goal

Deliver a 6 minute demonstration of the software that

demonstrates our most compelling value features andachieves venture capital investment interest


43/84


LLC 43

Release Criteria

Goals and objectives for the entire project release

Usually they are multi-faceted, defining a broad set of conditions

Required artifacts & levels of detail

Testing activities or coverage levels

Quality or allowed defect levels

Results or performance metrics achievement levels

Collaboration with other groups dependency management

Compliance levels

That IF MET would mean the release could occur.


44/84


LLC 44

Levels of Criteria

Activity Criteria Example

Basic Team

Work Products

Doneness criteria Pairing or pair inspections of code prior to check-in; or

development, execution and passing of unit tests.

User Story or

Theme Level

Acceptance Tests

Development of FitNesse based acceptance tests with the

customer AND their successful execution and passing.

Developed toward individual stories and/or themes for sets

of stories.

Sprint or

Iteration Level

Doneness criteria Defining a Sprint Goal that clarifies the feature

development and all external dependencies associcated with

a sprint.

Release Level Release criteria

Defining a broad set of conditions (artifacts, testing

activities or coverage levels, results/metrics, collaborationwith other groups, meeting compliance levels, etc.) that IF

MET would mean the release could occur.


45/84


LLC 45

Exercise



In this section, we discussed the various levels of criteria that areimportant within agile teams.

1) Amongst your team, discuss how youve established goals & criteria

in your Agile (or non-Agile) teams

1) What levels are the most important?

2) What levels dont matter as much?

2) What part does the team play in definition? Should play?3) Does defining what done means really matter? How?

4) If you had only one to pick, which would it be? And why?


46/84


LLC 46

Scrum-in-Testing


47/84


LLC 47

Process Overview

30 days

24 hours

Product Backlog

As prioritized by Product Owner

Sprint Backlog

Backlog tasks

expanded

by team

Potentially Shippable

Product Increment

Daily ScrumMeeting

Source: Adapted from AgileSoftware Development with

Scrumby Ken Schwaber andMike Beedle.

The Agile intent is toperform all testing within

the iteration usually

via automation.

Unit & Acceptance are

the typical focus, butwhat about other forms

of testing?

Legacy regression,

interoperability across

sprints, performance,usability, etc. Thus the

rub!


48/84


LLC 48

Inherently Narrow Focus

of Agile TestingTypical Agile Team

Testing Focus

Unit Testing

Automated Builds Smoke Testing

Focused Customer

Acceptance Testing

Test Driven Development(TDD)

Very limited Integration

& Regression Testing Focused Towards

Automation

Limited ExploratoryTesting

Whats Missing for Larger ScaleOrganizations?

Integration Testing

Functional Testing

System Testing

Regression Testing Performance Testing

Load Testing

Scenario Based Testing User Acceptance Testing (UAT)

Usability Testing, Other ility Testing

Exploratory Testing

Large-scale Automation


49/84


50/84


LLC 50

Scrum-in-Testing

Transitional Drivers High levels of traditional testing

Manual regression burden; Legacy systems; Habits across the business

stakeholder team; External pressures or expectations PO requires this as part of the Backlog

Early Scrum implementations

Development teams struggled with gaining testing traction not meeting

Agile code quality goals or core practices

Testing teams falling into traditional behaviors artifact based,

gatekeeper mentality, and low adaptability or flexibility

Insufficient testing resources to staff Sprints and post Development

Sprint testing requirements (thinking that Agile teams inherently need

lesstesters)


51/84


52/84


LLC 52

Scrum-in-Testing

Strategies Extending testing activity within the Sprint to include as much

coverage as possible:

Of course, unit & acceptance for the current iteration features Re-running existing unit & acceptance tests

Maintaining existing automation

Running partial integration & partial regression testing as possible

Cross-team collaboration

Which usually requires a different staffing mix for each Sprint

Orextendingthe iterative model to accommodate testing via Stabilization Sprints

Skewed Development & Testing focused Sprints


53/84


LLC 53

Scrum-in-Testing

Stabilization Sprints

30 day

Dev.Sprint(s)

Variable length

Sprint(s)

Product Owner defined Product Backlogs

And coordinates between development sprints

Production

Product Release

Stablilzation Sprint(s) focused on integrating developmentrelease forward towards production release

30 day

Dev.Sprint(s)

Testing Activities:1. Full regression

2. Overall integration3. Performance

4. Usability

5. Bug fixing

6. Production

promotion steps

S i T i


54/84


LLC 54

Scrum-in-Testing

Stabilization Sprints This is a modified

version of theScrum modelwhere Sprintsevolve from

Note: iterationresource mixchanges as you

move fromdevelopmenttowardsstabilization

1. Pure Development focused Sprintsdelivering features to PO to

2. Early Integration focused Sprintscoordinating a building product story andshaking our interoperability dependencies to

3. Pure Testing focused Sprints performingmore traditional testing activity leading towardsproduction release.

4. Finally and potentially Testing Infrastructuralfocused Sprints automation maintenance,next iteration readiness, and customer / usabilitycollaboration

S i T i


55/84


LLC 55

Scrum-in-Testing

Stabilization Sprint Sample Workflow

4 Development SprintsNormal team composition

Early transition to

Stabilization Sprints50/50 Development +Testing focus

2 Stabilization Sprints

Strongly connected todevelopment, led bystrong testing team

Next development

Sprint beginnings

Iteration #0, gainingtraction, Sprint #1

Repeat

There is a resource transitioncycle from full DevelopmentSprints forward to full

Stabilization Sprints.The Art is in effectivecollaboration, resource sharing& communication forward &

backward

S i T i


56/84


LLC 56

Scrum-in-Testing

Stabilization Sprint Content Pressure Release Think of Stabilization Sprint(s) as a feature content pressure release

mechanism. As your content increases, so does its integration risk.Youll want to use them

When you have many Development Sprints running in parallel

As an interim integration mechanism

When Stabilization Sprint threads run in parallel it creates the needfor S2 & S3 oriented interactions

The model typically is used for larger numbers of DevelopmentSprints contributing to a large-scale enterprise product

Duration and focus can vary from one Stabilization Sprint to the next

Although consistency matters

S i T i


57/84


LLC 57

Scrum-in-Testing

Skewed Testing Sprints

(n) day Test

Sprint(s)

Product Owner & test team members coordinate

bug feedback into current Development Sprint

Backlog and Product Backlog

Interim or Integration

Product Releases

Skewed Testing Sprint(s) focused on providing more formaltesting feedback by virtually running development & testing in

parallel / skewed Sprints

30 day

Dev.Sprint(s)

Testing Activities:

1. Partial regression2. Limited integration

3. Early performance

4. Bug fixing

5. QA promotion

steps

S i T i


58/84


LLC 58

Scrum-in-Testing

Skewed Testing Sprints This model balances some traditional testing post Development

Sprint against bogging downthe sprint w/too broad a level of testing

Usually the testing is focused towards traditional regression andintegration testing

May also be used for performance, compliance and other non-functionaltesting activity

The model typically is used for domains with an existing large scalelegacy testing burden (or requiring larger scale testing practices)

In this case, the skew allows the development activity to progress whiletesting is performed

Sometimes this is viewed as Waterfall testing within Scrum; butnecessary if you cant perform ALL testing within the iteration

S i T i


59/84


LLC 59

Scrum-in-Testing

Skewed Testing Sprints Advantage of handling defects and integration issues close to the

originating Sprint

Can reserve Sprint Backlog time so that changes can be incorporatedimmediately in the next Development Sprint

Testing Sprints are usually staffed solely with testers

In later phases, the testing can turn into more of a Stabilizationsprint effort so a morphing of the two strategies

Over time as your Agile experience grows, youll want to narrow theskew perhaps with overlapping iterations

T i i S


60/84


LLC 60

Testing-in-Scrum

Challenges

T ti i S


61/84


LLC 61

Testing-in-ScrumTypical Challenges Alignment

Its important to try and align Scrum iteration tempo across your

SoSoS Sprints, putting pressure on your:

Cross-sprint Meta Backlog management & planning

Sprint Review results & feedback

Testing Sprint alignments

Dependency management

Lab support scheduling & deployment phasing

3rd party integrations


62/84

T ti i S


63/84


LLC 63

Testing-in-ScrumTypical Challenges Skill-sets

Agile skill-sets and collaborative expectations are WAY different!

Do Define agile testing behavior guideline (role & responsibilities)

Encourage pairing and strong collaboration

Assess your technical capabilities and begin to aggressively fill in any

gaps: Technical domain understanding and direct programming experience

Open source automation tools experience

Customer collaborative and UAT experience

Establish guidelines for balancing between Agility & corporate qualityand governance expectations

Dont underestimate the impact that Agility will have towards yourtraditional teams capabilities, approaches and capacity for change

T ti i S r m


64/84


LLC 64

Testing-in-ScrumTypical Challenges Quality Influence

Working with the Product Owner (Customer)

Becoming a collaborative partner; defining & automating acceptancetests

Actively representing the Voice of the Customer (VOC)

Its important to setup clear & broad release criteria for each Sprint

and the overall Release Feature goals; usability goals; acceptance confirmation

Quality criteria (defects, coverage, types of testing)

Process artifact goals (for example: SOX or other compliance,traceability)

Establishing release readiness (features, quality, interoperability) forPO during Sprint Review (Pass/Fail goals met?)

Testing in Scr m


65/84


LLC 65

Testing-in-ScrumTypical Challenges Metrics & Visibility

As the number of Scrums grow with application size, feedback isgathered more so at the SoS level via

Cross Sprint burndowns and feature tracking

Feedback from the testing team in integration issues

Product Owner Sprint Review(s) experience

Product Roadmap progress across all of the relevant Sprints

The Meta ScrumMasters & Meta Product Owners are activelyengaged in defining goals and measuring progress against them

Test teams can / should provide some traditional metrics focusedtowards

Defects, test coverage & traceability, workflow defect patterns, Sprintrelease acceptance / goal attainment levels

Testing in Scrum


66/84


LLC 66

Testing-in-ScrumTypical Challenges Defects

In pure Agile teams (small teams & projects, quality & testingfocused, automation centered) there is little need for traditionaldefect tracking techniques

They test first by developing unit tests and continuously integratingchanges;

Establish automated acceptance tests and fixing bugs as theyre found

However, in evolving or large-scale Agile teams

There is a need for defect coordination across the various product(s) andteam(s);

Traditional triage, and targeting repairs to specific teams & iterations

Product Owner(s) and ScrumMaster(s) are involved in this coordinationand delegation

Testing teams are at the centerof these efforts; guiding the teamstowards their Sprint & Product Release goals

Exercise


67/84


LLC 67

Exercise



The context here is your personal experience with any aspect ofAgile Testing

1) Describe some of the biggest challenges youve faced.

2) Can you envision some of the iterative model changes working in

your domains? What adjustments are needed?

3) Often a challenge is legacy code bases. Describe yours and, more

importantly, how have you attacked it to improve your agility?4) Often the test teams themselves struggle moving towards agility,

describe your experiences with this. How have you improved these

situations?

Agile Endgames


68/84


LLC 68

Agile Endgames

Triage


69/84


LLC 69

Triage

Even in an agile environment there are bugs that surface late in the

game

Always bugs are

Immediately made visible to the Product Owner

Team explores impacts and repair options

3 options: Fix, Defer to Product Backlog, Ignore

Triage is based on the (pre-established) Release Criteria view for

User Stories, the Sprint, and the overall Release

Always without the traditional fanfareassociated with traditionalbugs

Role of Continuous Integration


70/84


LLC 70

Role of Continuous Integration

From the Release Train, youll remember that continuous Integration

is sort of the gluefor the entire process

Realize that you may have to make adjustments to the purist CI view

that everything moves on the trunk

Short lived branches might be required to be efficient in handling content

that is WIP and not releasing

Architectural implications

Ability to release products or components separately Ability to hold back features or components that arent done

CI Information Radiator


71/84


LLC 71

CI Information RadiatorChannelAdvisor

Environment Management


72/84


LLC 72

Environment ManagementProduct Maturation

Establishing an Environment & Promotion model

Development -> QA -> Staging -> Production

When and where are transitions made Establish clear criteria for each transition; with back-out capabilities

Supplement the model with automated scripting to replicate

environments

Can be particularly challenging in DB intensive environments stored

procedures & structure changes

Strongly consider coupling this to your CI efforts

Code Freeze Dynamics still Apply!


73/84


LLC 73

Code Freeze Dynamics still Apply!

Try and front-load major features and high priority work

Develop milestones for coalescing your code towards a freeze point Enter your hardening Sprints with a specific freeze target

During hardening have a coding halt target

May need layered freezes

UI versus Back-end Database layers

Database deployment script development

Still need to triage bugs to see what fitsusually in daily release

Scrum of Scrums

Microsoft - Code Complete Model


74/84


LLC 74

Microsoft - Code Complete Model

Microsoft employs a code complete strategy as defined below

Various project

milestonesFeature

CompleteVisual Freeze Code Complete

Stabilization leading to a

zero bug fix release

Release to

manufacturing

Typical Activities:

Pilots Alpha &Beta testing

Time buffer

Iterative testing and rework

Schoor Example of Repair Code Freeze


75/84


LLC 75

Schoor Example of Repair Code Freeze

Bruce Schoor has an extension to the Microsoft code completemodel as defined below

Code Complete

MilestoneLBF

Limited Bug Fix

UBF

Unlimited Bug Fix

RC

Release Candidate

SystematicTest Passes

Reduce # of High

Priority Defects

Production &Regression Tests

Acceptance Tests

Drive to ZeroDefects

No More Repairs

& Release!

Testing Focus

Release Goals

UBF LBF

Gate

LBF - RC

Gate


76/84

Deployment Readiness


77/84


LLC 77

Deployment Readiness

Focused on:

Support readiness

Production environment readiness Customer readiness

Training

Usually achieved including different skills in Hardening or Pre-

Release Sprints

Brings up the point of Sprint team composition. Does it change over the

life of the Agile Release Train?

What are some examples?

Release Retrospective


78/84


LLC 78

Release Retrospective

Should be conducted for each Release!

Apply your traditional format & facilitative rules I like the What Worked, Needs to Change, Want to Trymodel for

gathering thoughts

Much broader engagement For example, in a SaaS model engage Operations, Support, Customer

Account Managers, etc. for overall effectiveness

And stakeholdersperceptions? Calls from customers?

Log them and make action plans visible. Our CI manager servedas ourRelease Coordinatorto focus our attention on the entirecycle.

Wrap-up


79/84


LLC 79

Wrap up

Traditional Scrum (and other Agile methods) struggle to operate in:

Non-green field

Legacy based or compliance focused Enterprise level or large-scale team

environments.

Primary focus points for successfully Being Agilein these contextsinclude

Adapting your Customers towards Agility

Adapting Agility towards your Environment in a pragmatic fashion Focus on the basics of the Agile Manifesto and basics of Scrum

Questions?


80/84


LLC 80

Questions?

Thank you!

Core Agile References


81/84


LLC 81

Core Agile References Augustine, Sanjiv, Managing Agile Projects, Addison Wesley, (2005) Beck, Kent, Extreme Programming Explained Embrace Change, Addison

Wesley, (2005) Beck, Kent and Fowler, Martin, Planning Extreme Programming, Addison

Wesley, (2001) Cockburn, Alistair, Agile Software Development The Cooperative Game, 2nd

Edition, Addison Wesley, (2006) Cockburn, Alistair, Crystal Clear A Human-Powered Methodology for Small

Teams, Addison Wesley, (2005) Cohn, Mike, User Stories Applied For Agile Software Development, Addison

Wesley, (2004) Cohn, Mike, Agile Estimating & Planning, Addison Wesley, (2006) Derby, Esther and Larsen, Diane, Agile Retrospectives Making Good Teams

Great, Pragmatic Bookshelf, (2006) Highsmith, Jim, Agile Project Management, Addison Wesley, (2004) Larman, Craig, Agile & Iterative Development A Managers Guide, Addison

Wesley, (2004)

Leffingwell, Dean, Scaling Software Agility Best Practices for LargeEnterprises, Addison Wesley, (2007) Manns, Mary Lynn and Rising, Linda, Fearless Change Patterns for

Introducing New Ideas, Addison Wesley, (2004)



82/84


LLC 82


Poppendieck, Mary & Tom, Lean Software Development An Agile Toolkit,Addison Wesley, (2003)

Poppendieck, Mary & Tom, Implementing Lean Software Development FromConcept to Cash, Addison Wesley, (2006)

Schwaber, Ken and Beedle, Mike, Agile Software Development with Scrum,

Prentice Hall Publishing, (2002) Schwaber, Ken, Agile Project Management with Scrum, Microsoft Press,

(2004)

Schwaber, Ken, The Enterprise and Scrum, Microsoft Press, (2007)

Tabaka, Jean, Collaboration Explained Facilitation Skills for Software ProjectLeaders, Addison Wesley, (2006)

Core Agile Web References


83/84


LLC 83

Core Agile Web References

Mike Cohn Scrum of Scrums page/picture:http://www.mountaingoatsoftware.com/scrum/scrumteam.php

Jeff Sutherland: http://jeffsutherland.com Agile 2005 paper on Type A, B, C Scrums: http://www.agile2005.org/RP10.pdf Agile 2005 presentation on Scrum II:

http://jeffsutherland.com/scrum/ScrumIIAgile2005.pdf#search=%22scrum%20of%20sc

rums%22

Jeff Sutherland Agile 2006 presentation on Distributed Scrum Teams:www.scrumalliance.org/index.php/content/download/4836/49524/file/SutherlandDistributedScrumAgile2006_v4_28_Feb_2006.pdf

Annual Scrum Gatherings and Agile conferences www.agile2008.org Alternate perspective on XP - http://www.softwarereality.com/ExtremeProgramming.jsp Firms using Scrum - http://scrumalliance.pbwiki.com/Firms-Using-Scrum?doneSave=1 Planning Poker cards - http://contrado.com.au/PP_Cards.pdf

The history of the term SCRUM dates back to a 1986 article by Takeuchi and Nonaka inHarvard Business Review. In it they connected the Rugby term to an iterative model forproduct development. It's worth a read to simply see the history and connection back toLean Manufacturing practices - http://apln-richmond.pbwiki.com/f/New%20New%20Prod%20Devel%20Game.pdf

Contact Info
http://www.mountaingoatsoftware.com/scrum/scrumteam.phphttp://www.mountaingoatsoftware.com/scrum/scrumteam.phphttp://jeffsutherland.com/http://www.agile2005.org/RP10.pdfhttp://jeffsutherland.com/scrum/ScrumIIAgile2005.pdf#search=%22scrum%20of%20scrums%22http://jeffsutherland.com/scrum/ScrumIIAgile2005.pdf#search=%22scrum%20of%20scrums%22http://jeffsutherland.com/scrum/ScrumIIAgile2005.pdf#search=%22scrum%20of%20scrums%22http://www.scrumalliance.org/index.php/content/download/4836/49524/file/SutherlandDistributedScrumAgile2006_v4_28_Feb_2006.pdfhttp://www.scrumalliance.org/index.php/content/download/4836/49524/file/SutherlandDistributedScrumAgile2006_v4_28_Feb_2006.pdfhttp://www.scrumalliance.org/index.php/content/download/4836/49524/file/SutherlandDistributedScrumAgile2006_v4_28_Feb_2006.pdfhttp://www.agile2008.org/http://www.softwarereality.com/ExtremeProgramming.jsphttp://scrumalliance.pbwiki.com/Firms-Using-Scrum?doneSave=1http://contrado.com.au/PP_Cards.pdfhttp://apln-richmond.pbwiki.com/f/New%20New%20Prod%20Devel%20Game.pdfhttp://apln-richmond.pbwiki.com/f/New%20New%20Prod%20Devel%20Game.pdfhttp://apln-richmond.pbwiki.com/f/New%20New%20Prod%20Devel%20Game.pdfhttp://apln-richmond.pbwiki.com/f/New%20New%20Prod%20Devel%20Game.pdfhttp://apln-richmond.pbwiki.com/f/New%20New%20Prod%20Devel%20Game.pdfhttp://apln-richmond.pbwiki.com/f/New%20New%20Prod%20Devel%20Game.pdfhttp://contrado.com.au/PP_Cards.pdfhttp://scrumalliance.pbwiki.com/Firms-Using-Scrum?doneSave=1http://www.softwarereality.com/ExtremeProgramming.jsphttp://www.agile2008.org/http://www.scrumalliance.org/index.php/content/download/4836/49524/file/SutherlandDistributedScrumAgile2006_v4_28_Feb_2006.pdfhttp://www.scrumalliance.org/index.php/content/download/4836/49524/file/SutherlandDistributedScrumAgile2006_v4_28_Feb_2006.pdfhttp://jeffsutherland.com/scrum/ScrumIIAgile2005.pdf#search=%22scrum%20of%20scrums%22http://jeffsutherland.com/scrum/ScrumIIAgile2005.pdf#search=%22scrum%20of%20scrums%22http://www.agile2005.org/RP10.pdfhttp://jeffsutherland.com/http://www.mountaingoatsoftware.com/scrum/scrumteam.php


84/84

Co c o

Robert Galen

RGalen Consulting Group, L.L.C.

PO Box 865, Cary, NC 27512

919-272-0719www.rgalen.com

[email protected]

Related ST&P articles in June 2006 and 2007 issues,

www.stpmag.com

Software Endgames: Eliminating Defects, ControllingChange, and the Countdown to On-Time Delivery

published by Dorset House in Spring 2005.www.rgalen.com for order info, misc. related

presentations, and papers.
http://www.rgalen.com/mailto:[email protected]://www.stpmag.com/http://www.rgalen.com/http://www.rgalen.com/http://www.stpmag.com/mailto:[email protected]://www.rgalen.com/

agile endgames

Documents