agile testing in a formal organisation · mapping cobit at the agile/scrum process 39 • include...
TRANSCRIPT
![Page 1: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/1.jpg)
Agile Testing in aFormal Organisation
Egbert Bouman & Robin Esman
Twitter: @egbertbouman
Valori & De Nederlandsche Bank
![Page 2: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/2.jpg)
About us
Robin Esman
• 20+ years in IT, 9 years at DNB
• Test community lead
• Business Analist
Egbert Bouman
• Valori
• Author SmarTEST
• Certified risk auditor
• Test automation consultant
• Agile test coach & trainer
![Page 3: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/3.jpg)
Our message today
YES…
… formal organisations do agile
• Because it’s mainstream now
• And rightly so: agility is good for everyone
BUT…
… this poses extra challenges
• Be aware of that challenges
• And adopt your agile strategy accordingly
SO…
… take advantage of our story
• With seven best practices we’ve selected in retrospective
• They worked for us, and may work for you
3
In the context of better testing!
![Page 4: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/4.jpg)
DNB INTRO
In
Behind Closed Doors
![Page 5: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/5.jpg)
Collectors value >> Gold value Collectors value >> Gold value
![Page 6: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/6.jpg)
DNB Working on trust
6
![Page 7: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/7.jpg)
Het Agile Manifesto
77
![Page 8: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/8.jpg)
1. Our highest priority is to satisfy the
customer with early and continuous
delivery of valuable software.
2. Welcome changing requirements, even
late in development.
3. Deliver working software frequently,
from a couple of weeks to a couple of
months.
4. Business people and developers must
work together daily.
5. Build teams with motivated
individuals. Give them the
environment and support, and trust
them.
6. Most efficient and effective is face-
to-face conversation.
7. Working software is the primary
measure of progress.
8. Promote sustainable development. Agile
teams should be able to maintain a
constant pace indefinitely.
9. Continuous attention to technical
excellence and good design.
10.Simplicity: the art of maximizing the
amount of work not done.
11. The best results emerge from self-
organizing teams.
12.Regularly, the team reflects on how
to become more effective, then adjusts
its behavior accordingly.
The 12
‘A
gile Principles’
![Page 9: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/9.jpg)
How to merge agile into
a formal organisation’s DNA?
9
vs
Risk
aversion
& control
How did we
address the
challenge?
![Page 10: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/10.jpg)
We implemented a lot of bright
(and not so bright) idea’s
10
And selected the top-7 for You
![Page 11: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/11.jpg)
Our advice: 7 ‘agile test essentials’
11
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
![Page 12: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/12.jpg)
Beware of scrumdamentalism
Maximise agility, but stick to ‘good old’ best test practices
• Risk analysis
• Refinement: critical thinking, ‘what if’
• Exceptions, rainy days, negative testing
• Sound defect management
• Overall test management
These and other proven practices are OK
• Don’t treat them with contempt
• Beware of short sighted labels like SINO, AINO, Zombie Scrum, …
12
![Page 13: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/13.jpg)
But be clear about the tough choices
More trust, less ‘governance’
More exploratory, less detail in advance
More collaboration, less toll gates
13
![Page 14: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/14.jpg)
Our advice: 7 ‘agile test essentials’
14
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
![Page 15: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/15.jpg)
Started with the
‘Agile Growth Accelerator’
and Agile Scorecard
MaakMaakBuild
Architecture
Six improvement
areas
![Page 16: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/16.jpg)
Testing still the bottleneck
Gartner:
“Testing repeatedly emerges as the most formidable constraint
impacting velocity”
Forrester:
“As organizations adopt (…) Agile and DevOps, testing has
become the biggest constraint impacting quality and speed ”
16
![Page 17: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/17.jpg)
Continued with the
‘Test Growth Accelerator’,
and ‘SmarTEST scorecard
Seven improvement areas:
1.Agile test strategy
2.Scope, risks and test goals
3.Test design & execution
4.People & culture
5.Test environments
6.Automated testing
7.Planning & control
![Page 18: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/18.jpg)
The SmarTEST scorecard in more detail
Best of breed ‘waterfall’ frameworks
• Test Process Scorecard (SmarTEST®)
• TPI (TMap®), CMMi, TMMi
Made agile
• Address all 12 ‘agile principles’
• Kniberg self assessment checklist
Fit for non-hardcore software companies
• Fully agile
• Respecting pre-agile proven practices
‘Challenged Self Assessment’
• Team decides upon improvement focus and required maturity level
• Supported and challenged!! by agile coach18
![Page 19: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/19.jpg)
Real Growth AccelerationPresented 2016Q4 by the pilot teams themselves
Deposito guarantee system
WSS Wall Street System F&A Peoplesoft(no improvement data)
Cash Operations(late boarding)
CRM Financial Supervision
![Page 20: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/20.jpg)
Scorecard process retrospective
Management happy
• Control
• Direction
• Clear & concise
Teams happy
• Challenging whilst helpful
• Freedom of choice
• ‘Do-able action items’
20
![Page 21: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/21.jpg)
Our advice: 7 ‘agile test essentials’
21
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
![Page 22: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/22.jpg)
DeveloperQuality & business aware
Tester(More or less) independent
CustomerFinal proof & verdict
Respect separation of duties: DTC model
No walls, only roles!!
![Page 23: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/23.jpg)
Design&
Build
Operations
Developer Tester Customer
The test ‘Shift Left’
Unit & integration test (CI)
Acceptancetest
Independent functional (chain)test new & existing
(ST, SIT, FT)
![Page 24: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/24.jpg)
Developer Tester
Unit Test
Customer
A succesfull shift left path
Test New & OldAcceptance
testWAS
Unit Test & CI
Test New
Test Old
AcceptancetestNOW
Unit test & CI
Test New
Test Old
Acctest
GOAL
Insufficient coverage, production incidents.
Beter coverage,testing still on critical path.
Faster, same coverage,less business capacity.
![Page 25: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/25.jpg)
Our advice: 7 ‘agile test essentials’
25
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
![Page 26: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/26.jpg)
WoW cards
Lean Way of Work (WoW!)
• Two-sided with wow-effect
• Kanban: Japanese for ‘Visual card’
Lean, accessible, visual
• Color & appearance
• No bulky documents
Keep them alive
• Proces en ownership
• Expiration date26
![Page 27: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/27.jpg)
27
Visualize
Kanban = visual card!
WoW: make testing colourful again!
![Page 28: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/28.jpg)
Our advice: 7 ‘agile test essentials’
28
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
![Page 29: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/29.jpg)
Tool does the job
Faster, cheaper
Dependable, repeatable
Everybody does it
Demo impressed
me!
29
Top-down driver: bosses like it
![Page 30: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/30.jpg)
Fiddling with tools
I like hightechstuff
Manual testingis boring
Automationis cool
Mymarketvalue
30
Bottom-up driver: boys will be boys (girls too)
![Page 31: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/31.jpg)
Manage your Test Automation ‘just enough’!
Choose what to automate
• And what not to automate
Manage the top of the pyramid
• And choose ‘middle & top of the pyramid tools’
Let developers do it their way
• Trust & let go
• But be aware of ‘too many tools’
31
![Page 32: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/32.jpg)
32
![Page 33: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/33.jpg)
Our advice: 7 ‘agile test essentials’
33
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation tool landscape
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
![Page 34: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/34.jpg)
Improve your improvement
Problem: continuous improvement is immature
Solution: agile working secures continuous improvement
Unfortunately it doesn’t work that way
34
![Page 35: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/35.jpg)
What did we do?
We solved this problem by continuously organising ‘just do Scrum’
Stimulating, Challenging, Facilitating
Use guilds, audits, events, coaches => time to focus
Build a community
Keep looking for leverage opportunities (e.g. Cobit audit)
And make it fun!
35
![Page 36: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/36.jpg)
Make it fun
Test community:40 members
‘pepernoten’ test‘broodje kroket’
![Page 37: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/37.jpg)
Our advice: 7 ‘agile test essentials’
37
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
![Page 38: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/38.jpg)
The Cobit
audit framework:
Overview
38
Still a popular framework!
DNB uses a sub-set
Level 3: formalise & proof
Level 4: continuous improvement
![Page 39: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/39.jpg)
Mapping Cobit at the agile/scrum process
39
• Include non-functional acceptance criteria
• Like Security and Performance
• Atention for out-of-the-box risks
Refinement and Sprint planning
• Security and Performancetest done
• Fall back plan available
• Anticipated and demo ’ed with your DoD
Review(= productevaluation)
• Did we follow the proces agreed upon?
• Do we do a sound risk management job?
• Is testing sufficiently Independent (separation of duties)?
Retro(= process evaluation)
This
example:
AI7.6
‘Testing of
Changes’
![Page 40: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/40.jpg)
40
![Page 41: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/41.jpg)
So far our story
41
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
![Page 42: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/42.jpg)
![Page 43: Agile Testing in a Formal Organisation · Mapping Cobit at the agile/scrum process 39 • Include non-functional acceptance criteria • Like Security and Performance • Atention](https://reader034.vdocuments.mx/reader034/viewer/2022042303/5ece6350a274d538646fc2a0/html5/thumbnails/43.jpg)
43
Thank you
www.valori.nl
www.dnb.nl