Protecting the irreplaceable | f-secure.com
Evolution of Longer-Term Planning in a Large Scale Agile Project – F-Secure’s experienceGabor Gunyho & Juan GutierrezImprovement Coach Manager, Agile Practices
2011-05-09
What is this presentation all about?
• Many publications in the
SW development literature
offer ready-made “recipes”
for solving a certain
problem
• We believe this is way too
often not helpful as the
context varies a lot
© F-Secure Public2011-05-092
Text source: http://easteuropeanfood.about.com/od/hungariansoups/r/gulyasleves.htm
What is this presentation all about?
• Instead we offer a story. This is how we did it in a big project.
• The story is told through the project's major events called “Release Planning”
• We hope that you‟ll learn some ideas on project planning and steering on a larger scale … even if it does not offer a ready-made recipe
© F-Secure Public2011-05-093
Image source: http://lorabetht.blogspot.com/2010/11/violence-and-sophocles-antigone.html
About F-Secure
© F-Secure Public2011-05-094
• Company
• Founded in 1988, listed on NASDAQ OMX Helsinki
• Market cap ca 350 m€, annual revenue ca 130 m€ (2010)
• Headquartered in Helsinki, 18 country offices, presence in more than 100 countries
• 850 people, 300+ in R&D, 5 R&D offices in 4 countries, Agile since 2003-2005
• Products and Services
• Online Security: Anti-Malware, e-mail filter, Browsing Protection, Parental Control
• Content Protection: Online Backup, Anti-Theft
• Online Storage and Services: Storage Platform, Sharing, Social Media Access
• Multiple OS platforms (Win, Mac, Linux, mobile), 20+ language versions
• Customers
• Consumers (retail, reseller, e-store), millions of homes
• Network operators (ISP, mobile), world leader with 200+ operator partners
• Corporate
About the Authors
© F-Secure Public2011-05-095
Gabor Gunyho, Improvement Coach with the “R&D
Global Methods” team at F-Secure,
experienced Agile and Lean product
development expert, contributor and
reviewer of books on scaling Agile and
Lean SW development
Juan Gutierrez PlazaCurrently „Agile Practices Manager‟ at F-
Secure‟s Storage & Digital Content unit,
focusing on the R&D transformation of the site.
Experienced coach who has helped different
teams to improve in engineering and process
practices. Active member in different agile
forums and member of the board of Agile Spain
About the project
• Major new product, significant changes in
• Business model
• Architecture
• Method for Longer-Term Planning, including new backlog tooling
• About 10 teams
• Mostly in Helsinki, some in Kuala Lumpur, later also one in Poland
• Mostly feature teams
• Fairly mature in basic Scrum [1] and Agile engineering practices
• Some experience in multi-team projects [2] [3] but not on this scale
© F-Secure Public2011-05-096
Longer-Term Planning: Why?
• Plan for developing a complex system with lots of unknowns
and lots of dependencies, both in features and in architecture
• Provide estimates and visibility to progress on a higher level of
abstraction of the content and timeline
• Handle the inherent organizational complexity in multi-team
setup
We need to get alignment in teams to a common
objective
© F-Secure Public2011-05-097
“We need to improve the way how we manage our requirements, and especially
how we create concept (release) plans and link longer term architecture into our
short term plans“ Pirkka Palomäki, CTO
Longer-Term Planning in brief – the context
• Basic, Scrum-based Agile methodology does not cover scaling [1]
• Dean Leffingwell„s “Agile Release Train” [4] [5] covers multiple layers of
abstraction in all key dimensions of the project:
content, timeline and organization.
© F-Secure Public2011-05-098
Product BacklogProduct
Owner 2Team B
I1 I2 I3 I4
Beta1
B
I
P
B
I
PI5 I6 I7 I8
Beta2
B
I
PI9 I10 I11 I12
Release
Epic Feature Story
Reporting Aggregate
Reports
As an user I want to see a list of my average
spending for each of my budget-lines so that I
can get a fast control of my average expenses
Reporting Aggregate
Reports
As a end-user I can get a summary report my
total spending on a selected set of accounts
Reporting List Report As a end-user I can get a summary report my
total spending on a selected set of accounts
Reporting List Report As a end-user I can get a summary report my
total spending on a selected set of accounts
Logging ...
Fea
ture
Sto
ryE
pic
Team A
Business Iteration
Product
Owner 1
BIP = Business Iteration Planning
PM
PM
AM
AM
Longer-Term Planning in brief – the event
© F-Secure Public2011-05-099
Day 2
Status check
Planningteam breakout sessions
Final plan review
Risk review
Confidence vote
Retrospective
Day 1
Introduction
Project setup
Business Vision
Architecture Vision
User experience and UI
Engineering practices
Planning process intro
Planning team breakout sessions
Draft Plan review
The outcome: Plans vs. Planning
Plans are of little importance, but planning is essential
– Winston Churchill
Plans are nothing; planning is everything
– Dwight D. Eisenhower
No battle plan survives contact with the enemy
– Helmuth von Moltke the Elder
A good plan, violently executed now, is better than a
perfect plan next week
– George S. Patton
© F-Secure Public2011-05-0910
Source: http://en.wikipedia.org/wiki/Plan
The project journey – from the Longer-Term Planning perspective
© F-Secure Public2011-05-0911
New Method for the New Project
© F-Secure Public2011-05-0912
• New planning method: “Agile Release Train”
• Project kick-off and first “Release Planning” event: December 2009
• Planning Scope = 90 days
• 6 sprints, 2 weeks each, plus next planning
• Program for the "release” planning event
• One day training on Scaling SW agility
• “Release” Planning: 2 days +1 day reserve (that had to be used after all)
• Attendants: ~120, including all functions (R&D, Product Mgmt,
business, etc)
• Everybody (almost) in one space
• Facilitator: Dean Leffingwell, supported by internal improvement
coaches
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011
Biz Iteration #1P
Learning from the Business Iteration #1
• Beta releases: 2 internal (though delayed), 1 failed
• Huge amount of content becomes apparent, many dependencies and risks
• Feature vs. story concept is not clear to many
• Planning event can be done in 2 days (keep a 3rd day in reserve though)
• The 90-day “Business Iteration” with mid-term re-planning is not meaningful
• Change the cadence to 8-week Business Iterations with proper planning for each
• Changes in the planning process:
• Internal event facilitator takes over from external guru
• Moving towards continuous “Flow” in planning e.g., “Draft plan review” scrapped, instead, synchronize planning work of teams in every hour
• Special-skill experts (business, architecture, UX, etc) available throughout the event
© F-Secure Public2011-05-0913
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011
Biz Iteration #1P P
1 2 3
Planning event #2 (End of March 2010)
• Preceded by a ½ day retrospective done with Open Space [6]
• Slow feedback in development
• Improve Build System, Test Automation and reduce release effort
• “Release” Planning fixes: focus on dependencies, limit sprint load to team‟s
capacity
• Architecture challenges: different levels of abstraction needed, establish
virtual team for architects
• Changes for “Release” Planning event #2
• Imbalance between scope and velocity recognized and
acknowledged
• Change project setup:
Full system release June 2010 ->
Variant-A release by Aug (+2 months), Variant-B by Dec (+ 6 months)
• Plan for releasing a beta (for customer preview) in every sprint
© F-Secure Public2011-05-0914
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011
Biz Iteration #1 Biz Itrtn #2P P
1 2 3
Planning event #3 (early June 2010)
• Beta releases: 2 of 4 successful
“Based on R&D analysis of project scope and schedule
it is apparent that the current roadmap is not feasible”
Project split
• Spin off “mini-project” for downscaled Variant-A
• Variant-B development continues, with staged delivery,
Variant-B basic release Oct 2010, full set release Jan 2011
• Changes in the planning process
• At every hour synchronization meetings alternate,
one for the planning work and another for architecture issues
• Helpers for planning the improvement items rotate in teams
• Risk, impediment and dependency handling in short cycles (along with the
synch meetings), and not at the end of the planning event
• “Feature wall “ (or “master wall”) to depict which feature will be done by
whom and when, also visualizing dependencies
© F-Secure Public2011-05-0915
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011
Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3P P P
54 6 71 2 3
Example of a Master Wall
• Master wall that shows which team works on which
feature in which iteration
• List of identified risks and impediments
• ROAMed - Resolved, Owned, Accepted, Mitigated
© F-Secure Public2011-05-0916
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011
Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3P P P
54 6 71 2 3
Planning event #4 (early Sept 2010)
• Beta releases: 3 of 6 successful
• Full-time ScrumMasters for each team
• Event delayed by 1 week, time used for
• hardening (bug killing)
• re-teaming event: moving towards more feature teams
• ScrumMaster training
• Changes in the planning process
• Separate intro briefing session, including solution demo
• All input in physical tokens (all features printed)
• Improved master wall, physical visualization of dependencies
• Clear priorities for planning: “honesty over precision”
• Smoother planning in general
© F-Secure Public2011-05-0917
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011
Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3 Biz Iteration #4P P P P
54 6 7 13 14 15 16 17 181 2 3
Planning event #4 - memories
© F-Secure Public2011-05-0918
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011
Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3 Biz Iteration #4P P P P
54 6 7 13 14 15 16 17 181 2 3
Planning event #5 (early Dec 2010)
• Beta releases: 3 of 5 successful
• 1st public beta release
• “Last mile” before release
• Company restructuring
• Feature leaks (slipping content)
Last-minute change in the planning process
“We believe the last mile before release is better executed with feature focused planning.”
• Changes in the planning process
• New internal event facilitator
• New “feature planning” (see next)
• More emphasis in backlog grooming within sprints and Product Backlog preparation by Product Owners
• Release when last good build is deemed by Product Owners as good to go
Business Iteration (BI) Planning method is not abandoned, it’s just not applied for the last mile of the project
© F-Secure Public2011-05-0919
BI #4
BI #3
BI #2
BI #1
Planned
Done
Nr of
features
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011
Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3 Biz Iteration #4 “Last mile”P P P P P
54 6 7 13 14 15 16 17 18 19 20 21 221 2 3 18.5 23
Feature Planning for the “last mile” before release
© F-Secure Public2011-05-0920
Team
X
Team
Y
Develop
Feature A
Develop
Feature D
I(n) I(n+1) I(n+2) I(n+3) I(n+4) I(n+5)
Develop
Feature B
Develop
Feature C
Beta(n) Beta(n+1)
Develop
Feature E Backlog grooming
is about:- User needs and
requirements
- Architecture
- Dependencies
- UX
- Risks
of the next featureDevelop
Feature F
Features prepared
or refined by
Product Owners
in Product Backlog
regularly
Grooming for Feature D Grooming for Feature E
Grooming for Feature C Grooming for Feature F
Product
Owners,
etc
Iterations
Beta(n+2) Beta(n+3) Beta(n+4) Beta(n+5) Beta releases
(public)
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011
Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3 Biz Iteration #4 “Last mile”P P P P P
54 6 7 13 14 15 16 17 18 19 20 21 221 2 3 18.5 23
Towards the end of the project
• As of mid-Feb, 2011 for the first time the burndown shows that the remaining
work can be done by the planned release date
© F-Secure Public2011-05-0921
Note: data on picture was adjusted for work in progress items not reflected on picture
when the above conclusion was made
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011
Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3 Biz Iteration #4 “Last mile”P P P P P
54 6 7 13 14 15 16 17 18 19 20 21 221 2 3 18.5 23 24 2625 27 28
… and at the end of the project
• As of March 30, 2011 CR1 (Commercial Release #1) was released
© F-Secure Public2011-05-0922
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011
Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3 Biz Iteration #4 “Last mile”P P P P P
54 6 7 13 14 15 16 17 18 19 20 21 221 2 3 18.5 23 24 2625 27 28 29 30 31
Summary of the method
• New method for handling layers of abstraction in all key dimensions
• Business Iteration for steering in mid-term time scale
• Levels of abstraction in the “Value item” hierarchy: epics, features, stories
• Planning for the Business Iteration with the features and stories in a multi-
team setting
• Essential to pay attention to quality and the engineering practices like
Continuous Integration and Test Automation
• Never sacrifice quality, never
• Every bug found invokes adding a new test case to the Test Automation
suite
• No extra hardening outside of sprints, every sprint results in a customer
beta
© F-Secure Public2011-05-0923
Conclusions - the morale of the story
Better visibility and steerability for business
management,
helping making tough decisions
• Inspect and adapt, ie, evolve the method as you go
• Get real feedback after almost every sprint, even if product is big
• Get help from an experienced “process master”, ie, event facilitator
• Prepare the content well and do it continuously, don‟t overload the system
• Synchronization meetings within the planning, later extended to continuous
handling of risks, dependencies and impediments
• Presence of all stakeholders (business, architects, user experience)
• Master wall and feature coverage tracking
• Move to “feature planning”, a precursor to continuous planning
© F-Secure Public2011-05-0924
References
[1] Schwaber, K., Beedle, M.: Agile Software Development with Scrum. Prentice Hall (2001)
[2] Larman, C., Vodde, B.: Scaling Lean & Agile Development: Thinking and Organizational Tools for
Large-Scale Scrum. Addison-Wesley Professional (2008)
[3] Larman, C., Vodde, B.: Practices for Scaling Lean & Agile Development: Large, Multisite, and
Offshore Product Development with Large-Scale Scrum. Addison-Wesley Professional (2010)
[4] Leffingwell, D.: Scaling Software Agility: Best Practices for Large Enterprises. Addison-Wesley
Professional (2007)
[5] Leffingwell, D.: Agile Software Requirements: Lean Requirements Practices for Teams, Programs,
and the Enterprise. Addison-Wesley Professional (2011)
[6] http://en.wikipedia.org/wiki/Open_Space_Technology
© F-Secure Public2011-05-0925
Questions?
© F-Secure Public2011-05-0926
Contact [email protected]@f-secure.com
This presentation, and the project behind it are the work of many people whom all we can‟t possibly list here.
We‟d like to thank all who made this possible, most notable the whole team of this project.
Special thanks to F-Secure‟s R&D Management for supporting the work and this presentation
The authors did their best to attribute the authors of texts and images, and to recognize any copyrights, see more
details of copyrights, license terms and conditions for each source under the reference link provided. If you think
that anything in this material should be changed, added or removed, please contact the authors at the addresses
above
© F-Secure Public2011-05-0927
http://creativecommons.org/licenses/by-nd/3.0/
Background slides(actual fact sheets)
2011-05-0929 © F-Secure Public
Schedule look before planning event #1.5 (re-plan)
© F-Secure Public2011-05-0930
Week Iteration Comments
50 7.12. Iteration 1
51 18.12.
52 21.12. Iteration 2
53 1.1.
1 4.1. Iteration 3
2 15.1.
3 18.1. Iteration 4 PSI-1 re-planning End-to-end testable
internal (LAB) release4 29.1. beta-1
5 1.2. Iteration 5
6 12.2. beta-1 (12.2.2010)
7 15.2. Iteration 6
8 26.2. beta-2 (26.2.2010)
9 1.3. Iteration 7
10 12.3. beta-3 & PSI-1
(12.3.2010)
11 15.3. Release-retrospective & PSI2 planning
12 19.3. PSI-2 iterations begin…
Schedule
100%
25
%
30.6.
7.12.
29.1.
∼60% of
PSI-1
PSI = Potentially Shippable Increment, earlier term for Business Iteration
Scoping before planning #2
• Number of features before planning #2
2011-05-0931 © F-Secure Public
Nice to have
Important
Mandatory
Variant-A
Common
Variant-B
Schedule and scope adjustment before planning #2
© F-Secure Public2011-05-0932
Var-A
Var-B
RTM
Var-A
RTM
Var-B
RTM
Original
Current approved
+2 months +6 months
PSI planning
PSI planning
PSI planning
PSI planning
PSI planning
PSI-1 PSI-2 PSI-3 PSI-4 PSI-5 PSI-6
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan
Planning
for this now
2010
PSI = Potentially Shippable Increment, earlier term for Business Iteration
RTM = Release to Manufacturing
2011
Trade-off matrix for Variant-A as of planning #2
© F-Secure Public2011-05-0933
PERFORMANCE of the back-end Improve Keep Sacrifice
PERFORMANCE of the client Improve Keep Sacrifice
USER EXPERIENCE Improve Keep Sacrifice
USABILITY Improve Keep Sacrifice
BUSINESS support for F-Secure Improve Keep Sacrifice
SECURITY Improve Keep Sacrifice
FEATURE SET of the client Improve Keep Sacrifice
FEATURE SET of the back-end Improve Keep Sacrifice
INTEROPERABILITY Improve Keep Sacrifice
RELIABILITY Improve Keep Sacrifice
SUPPORTABILITY Improve Keep Sacrifice
TESTABILITY in R&D Improve Keep Sacrifice
Legend: Comparing to our client & backend offer of August 2009
• IMPROVE this area
• KEEP this area roughly equally good
• SACRIFICE things in this area to succeed in others
Trade-off matrix for Variant-B as of planning #3
© F-Secure Public2011-05-0934
PERFORMANCE of the client Improve Keep Sacrifice
USER EXPERIENCE Improve Keep Sacrifice
USABILITY Improve Keep Sacrifice
BUSINESS support for F-Secure Improve Keep Sacrifice
SECURITY Improve Keep Sacrifice
FEATURE SET of the client Improve Keep Sacrifice
FEATURE SET of the back-end Improve Keep Sacrifice
INTEROPERABILITY Improve Keep Sacrifice
RELIABILITY Improve Keep Sacrifice
SUPPORTABILITY Improve Keep Sacrifice
TESTABILITY in R&D Improve Keep Sacrifice
Legend: Comparing to our client & backend offer of August 2009
• IMPROVE this area
• KEEP this area roughly equally good
• SACRIFICE things in this area to succeed in others