20141113 devoxx2014 jochim van dorpe testing in agile
DESCRIPTION
Testing in an agile context - Beyond the urban legend Talk from #Devoxx 2014 on November 13 by @jochimvandorpe on software testing in an agile context? What does it mean for testing methodology when switching from waterfall to agile? Have testers become redundant or rather an essential part of a software development team? How do you keep the pace when functional requirements are changing every two weeks? This presentation is giving no-nonsense advice from hands-on testing experience in a large ICT organisation. #agile # testing #Java #DevoxxTRANSCRIPT
@ Smals_ICT#DV14 # AgileContextTesting @Smals_ICT#DV14 #AgileContextTesting
Beyond the urban legend
–
Testing in an agile contextJochim Van Dorpe
@ Smals_ICT#DV14 # AgileContextTesting
In-house ICT shared services for e-government
• Focus on social security and e-health services
• For federal, regional, local & European institutions
• Based in Brussels
• Software development, ICT-operations & staffing
• >1700 people
Introduction: What is Smals?
@ Smals_ICT#DV14 # AgileContextTesting
Introduction: Who Am I?
(QA) ( (lead) (technical) test(er)
analyst/coördinator/manager) (automator/engineer)
I am a tester
@ Smals_ICT#DV14 # AgileContextTesting
Introduction: What do I have to tell?
@ Smals_ICT#DV14 # AgileContextTesting
What is testing?
It's not about proving that the software works, it's about finding bugs
@ Smals_ICT#DV14 # AgileContextTesting
Why is testing necessary?
To prevent:
• Death
• (Serious) injury
• Loss of business
• Loss of reputation
@ Smals_ICT#DV14 # AgileContextTesting
Why do we test?
@ Smals_ICT#DV14 # AgileContextTesting
The test project in waterfall
@ Smals_ICT#DV14 # AgileContextTesting
Agility
Agility is the continuous
delivery of prosperity (value) to
stakeholders of a system in a
sustainable and balanced
manner -Schalk Cronjé (@ysb33r)-
@ Smals_ICT#DV14 # AgileContextTesting
Agile fundamentalists are like …
Did you even read
our manifesto?
@ Smals_ICT#DV14 # AgileContextTesting
Agile manifestoOf course …
• Individuals & interactions over
processes and tools
• Working software over
comprehensive documentation
• Customer collaboration over
contract negotiation
• Responding to change over
following a plan
@ Smals_ICT#DV14 # AgileContextTesting
Urban legends
• Agile doesn’t need testers
• Agile = de facto good code, built-in quality
• Testing is dead
• Agile = TDD, ATDD, BDD
• Testers should be able to code
• Agile hasn’t time for testing
• 100% automation
• Faster, shorter, better!
• Developers and Testers are like oil and water
• You only need to unit test
• User Acceptance Testing is no longer necessary
@ Smals_ICT#DV14 # AgileContextTesting
So…
• The Agile manifesto shouldn’t stand between you &
good quality
• No documentation, few documentation or poor
documentation is no excuse for “not testing”
• Don’t put an Agile stamp on your team, just to avoid
the things you dislike, but find a way of doing them
so you like them …
@ Smals_ICT#DV14 # AgileContextTesting
No difference at all?
@ Smals_ICT#DV14 # AgileContextTesting
Many ways to be agile …
@ Smals_ICT#DV14 # AgileContextTesting
… but I’m no salesman
@ Smals_ICT#DV14 # AgileContextTesting
Principles of testing in an agile
context
-Elisabeth Hendrickson (@testobsessed)-
@ Smals_ICT#DV14 # AgileContextTesting
1. Testing moves the project forward
Shifting focus:
No longer the last line of defense
@ Smals_ICT#DV14 # AgileContextTesting
2. Testing is not a phase
Integrated & continuous:
develop & test together
@ Smals_ICT#DV14 # AgileContextTesting
3. QA = responsability of the team
@ Smals_ICT#DV14 # AgileContextTesting
4. Reduce feedback latency
No long gaps between development
& testing feedback
@ Smals_ICT#DV14 # AgileContextTesting
5. Don’t let bugs crawl around
Deal with bugs as soon as possible
@ Smals_ICT#DV14 # AgileContextTesting
6. Test should represent requirements
Don’t ASSUME, it makes an ASS of
U & ME
SCHOOLBUS
@ Smals_ICT#DV14 # AgileContextTesting
7. Reduce overhead
Think high level: don’t waste time on
too much detail
@ Smals_ICT#DV14 # AgileContextTesting
8. Don’t believe in fairy tales
If something
seems to good to be true, it probably is
@ Smals_ICT#DV14 # AgileContextTesting
9. Done means done
Done means implemented and tested
@ Smals_ICT#DV14 # AgileContextTesting
Concrete practices
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate More
• The testing triangle: This will drown your testers:
Manual tests
Acceptance tests
System &
integration tests
Comp.
tests
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate More
• The testing triangle: Make sure your testers can
cope with the pace
Component tests
System &
Integration tests
Acc.
Tests
Manual
testsManual tests
Acceptance tests
System &
integration tests
Comp.
tests
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate Testing, not only tests
Test
execution
ch
ec
kin
g
Load
Data-
sets
Reset
DB
Continuous integration
Logging
resultsGenerate
reports
Live
dashboards
Calculating
‘metrics’
Use the tools that fit your purpose
AND NOT THE OTHER WAY AROUND!
@ Smals_ICT#DV14 # AgileContextTesting
B. Explore
Leave some room for
exploratory testing
@ Smals_ICT#DV14 # AgileContextTesting
C. Diversify
• …in the test levels
• Unit-tests only won’t find
everything
• System test only, and you’ll
drown
• … in the way of testing
• Automation is good, automation
is fun, but in the end, an
automated test isn’t better than a
human based test (manual test)
@ Smals_ICT#DV14 # AgileContextTesting
D. Test the tests
• GREEN == GOOD ?
• DO: Review tests
• DONT: No test coverage fairy tales
• DONT: Mock lasagna
• DONT: I should adapt the test cases so it’s green again
@ Smals_ICT#DV14 # AgileContextTesting
E. Crowdsource …
• Crowdsource in your team to find
the good & passionate tester
• Accept that there are learning curves
• Testing is a craftmanship
• A passionate tester will tend to be more
effective than the chinese-volunteer-
tester
• Encourage collaborative testing
@ Smals_ICT#DV14 # AgileContextTesting
… or ask a testjumper
• Ask a ‘test jumper’ to join the
team
• May spend some hours as a consultant
• Or months as an ordinary tester
• Coaches or contributes himself
• Encourages testability
• Help the developers think
productively about testing
@ Smals_ICT#DV14 # AgileContextTesting
The test project in agile
@ Smals_ICT#DV14 # AgileContextTesting
What should the PL not expect?
• Metrics such as:
• # of bugs found
• Bugs per functionality
• Bugs per 1000 lines of code
• DDP (defect detection percentage)
• We can however show our QA-efforts
• QA-survey
• Test case progress
• Fixed test cases
@ Smals_ICT#DV14 # AgileContextTesting
What should the PL not expect?
• Qualification (good / bad) of the devs
• 100% green charts
• Upfront fixed number of test cases
• Automated self-generating auto-code-correcting
tests
@ Smals_ICT#DV14 # AgileContextTesting
conclusion
There are no agile testers,
but we need testers who
adapted their mindset to an
agile context
@ Smals_ICT#DV14 # AgileContextTesting
Read & experience more 1/2• Books:
• Lisa Crispin, Janet Gregory (2009). Agile Testing: A Practical Guide for Testers and
Agile Teams
• Lisa Crispin, Janet Gregory (2014), More Agile Testing: Learning Journeys for the
Whole Team
• Blogs:
• Huibschoots.nl
• LisaCrispin.com
• http://testobsessed.com : Agile testing overview
• Http://www.satisfice.com : Test jumpers: one vision of Agile testing
• pascaldufour.wordpress.com
• agile-and-testing.chriss-baumann.de
@ Smals_ICT#DV14 # AgileContextTesting
Read & experience more 2/2• Presentations:
• 40 agile methods in 40 minutes by Craig Smith
• Agile testing quadrants explained by @RubyTester
• Certification:
• CAT-training: Certified Agile Tester
• ISTQB foundation level add-on: Agile tester
• Conferences:
• Agile testing day(s)
• More references:
• Word clouds generated on wordle.net
• Images found with google image search
@ Smals_ICT#DV14 # AgileContextTesting
Questions