![Page 1: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/1.jpg)
The Changing Roles of Testers
@pau
l_ge
rrard
Paul Gerrard [email protected]
gerrardconsulting.com
Programme Chair 2014
![Page 2: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/2.jpg)
Helping clients transform their testing through
INNOVATION, COACHING and LEADERSHIP
Our CLIENTS
– Want to be agile rather than follow Agile dogma
– Have a pragmatic approach and are focused on delivery
– Want a solution that fits, not a badly fitting suit.
![Page 3: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/3.jpg)
Typical tester job spec?
• Skills & Requirements – Thorough C# knowledge including Linq
– Good .Net experience including charting and winforms applications.
– Experience building and maintaining automated test systems
– Ability to interact with the developers to anticipate future needs
– Ability to influence to ensure good test procedures are developed and implemented
• Desirable Skills – Some knowledge of Java for mobile testing
– An interest in photography.
![Page 4: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/4.jpg)
Agenda
• Pressure to Change
• Three Innovations
• Where is this Leading?
• A New Model for Testing
• Consequences
• Why Shift-Left is More Important than Agile
![Page 5: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/5.jpg)
What's happening?
• Stampede to mobile computing
• Big Data
• Internet of (Every)Thing, Pervasive Computing
• Continuous Delivery, DevOps
• (Test) Analytics, Data-Driven Development
• Shift-left, embedded testers, no test team
• Did you really think Agile was the last word?
![Page 6: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/6.jpg)
Three Innovations That
Are Driving Change
Being agile
Shift Left
Analytics
![Page 7: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/7.jpg)
#1
Agile doesn’t work
but being agile might
Agile is no longer innovative.
Challenging Agile and being agile
might be
Is continuous delivery for you?
![Page 8: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/8.jpg)
Agile doesn’t work?
• It works for some
• But many organisations have difficulties
• Mostly a cultural problem
– Requirements take months to elicit and agree
– Complex stakeholder relationships
– Stakeholders with ‘all the knowledge’ can’t be spared to be the on-site customer/product owner
– Lack of confidence => big system/acceptance tests
• We are Agile but ...
![Page 9: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/9.jpg)
Pragmatism rules (again)
• Pragmatic companies use the things that work
and discard those that don’t
• Let's be honest: Most are doing some form of
hybrid approach:
– Chaotic interfaces between Agile teams and
structured businesses
– TDD, BDD, ATDD, SBE today…
– Continuous Delivery, DevOps… tomorrow?
![Page 10: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/10.jpg)
#2
Shift Left
Developers in test?
Testers in development?
Testing (testers?) everywhere
![Page 11: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/11.jpg)
Testing BDD/Test-Driven) Trusted
Requirements + Stories
Shift left (redistributed testing)
Functional Requirements
Build (+little testing) but lots of rework
Testing (lots of checking + bugs)
Trusted Requirements +
Stories Testing
SAVED TIME
BDD/Test-Driven
Who does the testing now? BAs, Developers or Testers? Stepping stone to continuous delivery...
![Page 12: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/12.jpg)
#3
Test Analytics
Analytics from production
Analytics from test
An opportunity for testing
![Page 13: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/13.jpg)
From traditional delivery…
… to Continuous Delivery
![Page 14: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/14.jpg)
Deployment into
Production is not the
Same as ‘Going Live’
Release to production and test
environments on-demand
Go Live weekly/monthly/quarterly…
![Page 15: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/15.jpg)
Analysis Insight Decision Development and Testing
Production Testing
Introducing Test Analytics
“Introducing Test Analytics”, Paul Gerrard:
http://gerrardconsulting.com/index.php?q=node/630
Julian Harty, "Symbiotic Relationships between Testing and SMAC":
http://blog.bettersoftwaretesting.com/2013/07/symbiotic-relationships-between-
testing-and-social-mobile-analytics-cloud-computing/
“The capture, integration and analysis of test and
production monitoring data to inform business and
software development decision-making”.
![Page 16: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/16.jpg)
Data silos for test analytics (an illustration)
Stakeholder Requirements Assurance (Production)
Application
Monitoring
Production
Environment
Monitoring
Stakeholders
Business Goals
and measures
Stakeholder
Involvement/
Engagement
Risk
Requirements
Stories/Feature
Descriptions
Glossary of terms
and Term Usage
Processes
Process Paths
(Workflows)
Manual Tests
Generated Test
Code (unit,
integration,
system level)
Application
Instrumentation
Automated Test
Execution History
Test Status
Feature Code
Change History
Application
Process Flows
User Accesses/
Activities
Feature Calls,
Response times
Feature use cases
(data usage)
Application
Alerts/Failures
Database Accesses
Database Content
Production
failures
System assets
Resource Usage
Log
Performance data
System Events
System Alerts/
Failures/Incidents
Outages
![Page 17: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/17.jpg)
Where is this leading?
If I knew that, I wouldn't be here...
![Page 18: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/18.jpg)
Three development patterns
Structured
Agile Continuous
Autonomous
![Page 19: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/19.jpg)
But there aren't three
patterns
There are many
![Page 20: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/20.jpg)
From high process to DIY process
• In the past, high process was seen to be the
technical solution to the software problem
• But technical solutions cannot solve human
problems
• Agile promised flexibility, but a lot of Agile is
really 'high process-in-the-small'
• True agility means a team can adapt or select
its process on-the-fly.
![Page 21: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/21.jpg)
The old ways won't
work in the future
We need a New Model of Testing
(free from logistics)
![Page 22: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/22.jpg)
A New Model for
Testing
Don't throw stones. Yet.
I'm working on a refined version (with others).
Want to help/review?
![Page 23: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/23.jpg)
Forget Logistics (for the time being)
Document or not?
Automated or manual?
Agile v waterfall?
Planned v exploratory?
This technology v that technology?
![Page 24: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/24.jpg)
ALL Testing is
Exploratory
We explore sources of knowledge ...
... to build test models ...
... that inform our testing.
![Page 25: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/25.jpg)
All Testing is Based on
Models
![Page 26: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/26.jpg)
Examples of models
![Page 27: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/27.jpg)
Models are innate, essential, human
![Page 28: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/28.jpg)
Judgement, exploring and testing
Testing (the system)
Our model(s) are adequate
Our model(s) are not adequate
Exploring (sources)
Judgement
Creates test models
Uses test models
We explore sources of knowledge to build test models that inform our testing
BTW – Do Developers explore the same way? I think so.
![Page 29: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/29.jpg)
Exploration process
Exploration Definitions specs/stories
People (& you)
Sources
Require-ments
Test Models
Enquiring
Challenging
Sources: People, documents, experience, system under test
Modelling
Test Models: Can be documented
or mental models
Predicting
System under test
![Page 30: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/30.jpg)
Testing process
Testing System
Under Test
Refining
Informing Applying
Interpreting
Test Models
Revise the System
More exploring Reporting
![Page 31: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/31.jpg)
New Model Testing
29 page paper: http://dev.sp.qa/download/newModel
![Page 32: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/32.jpg)
Some Consequences
![Page 33: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/33.jpg)
Do developers and
testers think the same
way?
Similar processes, not thinking the same
Easier to embed testers in development
Developer testing will improve
![Page 34: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/34.jpg)
Covers functional and
non-functional testing?
Ignoring logistics means we can have
the same foundational process for
ALL testing
(I think)
![Page 35: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/35.jpg)
Relation to
TDD and BDD?
TDD is not testing
BDD is modelling using stories
![Page 36: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/36.jpg)
Testing v Checking?
I submit that a Check is "a test that
can be applied by a human or tool"
But only humans can interpret
![Page 37: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/37.jpg)
Test automation from a
different perspective
Automation uses different test models
Is that why automation efforts fail so often?
![Page 38: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/38.jpg)
Developer Testing
The model works (for me)
![Page 39: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/39.jpg)
Capabilities
Enquiring, Modelling, Predicting, Challenging
Informing, Applying, Interpreting, Refining
Reporting and Logging
![Page 40: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/40.jpg)
• Analysis, enquiry and elicitation
• Modelling
• Creation of custom models, using
heuristics, guesses, brainstorming,
ideation, creative thinking
• Custom test design techniques
• Comparison of models, value, advantages,
disadvantages, compromises
• Identification, validation and use of
oracles
• Predicate logic and proof
• Hypothesis and inference
• Socratic method
• Rapid Review and Inspection techniques
• Test case design
• Test models and the meaning of coverage
• Testing as controlled experiment
• Observation, Note taking, recording
A very different skillset • Basic data analysis and statistics
• Decision-making with incomplete data
• Computer forensics
• Fault tree analysis
• Failure diagnosis
• Bug advocacy, triage processes and
negotiation
• Meaningful software and test metrics
• Visual presentation of data
• Reporting and presentation skills
• Understanding stakeholders
• Test analytics
• Risk management, risk-based testing and
decision-making
• Critical Thinking
• Interpersonal skills
• Dealing with uncertainty/fallibility
![Page 41: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/41.jpg)
Testing Career Development (speculative)
Foundations
Technical
Management
Strategic
Test Strategy Project Intelligence
Test Assurance
Exploration Forensics Interpretation
Scripting/ Programming
Test Automation Technical (Excel, SQL, OS utils etc)
Stakeholder management
Analytics & visualisation
Managing uncertainty
Critical Thinking
ISTQB etc...
Supplier Management
Test Process Management
Methodology
![Page 42: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/42.jpg)
Why Shift-Left is More
Important than Agile
![Page 43: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/43.jpg)
Developers write code;
testers test because
developers can’t
No they don’t
![Page 44: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/44.jpg)
Goal: End Manual
Feature Checking
Let the tools do it and
free people to do testing please!
![Page 45: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/45.jpg)
Testing doesn’t just
apply to existing
software, at the end
We test systems, not software.
We test early, we test often but it
might not be called testing
![Page 46: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/46.jpg)
Testers Need to Learn
How to Code?
Some, perhaps many will have to
![Page 47: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/47.jpg)
Testing is about
measuring achievement,
not quality
Align systems delivery with project
goals and risks and you’ve got a seat
on the board
![Page 48: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/48.jpg)
Testers don’t own
testing anymore
Be a TestMaster
"Assurance through the lifecycle"
![Page 49: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/49.jpg)
New Model Testing
changes what testers &
developers need to
learn
Roles of developers/testers become blurred
Perhaps they are not different disciplines after all?
Why have a tester certification?
![Page 50: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/50.jpg)
The phase after
development is
REWORK, not Testing
Testing must align with development,
not compete with it or rescue it
![Page 51: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/51.jpg)
Agile was for some
Shift-left is for ALL
That is why it's more important.
![Page 52: 20140916 Software Testing Seminar_Changing Roles of Testers_ New Model_Paul Gerrard_EuroSTAR 2014 Conference Chair](https://reader033.vdocuments.mx/reader033/viewer/2022042713/546c3026af79595d298b4f22/html5/thumbnails/52.jpg)
The Changing Roles of Testers
@pau
l_ge
rrard
Paul Gerrard [email protected]
gerrardconsulting.com
Programme Chair 2014