a new model for testing · a new model for testing @ d paul gerrard ... agile v waterfall? ... •...
TRANSCRIPT
A New Model for Testing
@pau
l_ge
rrard
Paul Gerrard [email protected]
gerrardconsulting.com
Programme Chair 2014
Agenda
• The Old Ways won't work in the future
• All Testing is based on models
• A New Model for Testing
• Tester Capabilities and Skills
• What next?
• Some Observations (themes that inspired this
work)
Typical tester job spec?
• Skills & Requirements – Thorough C# knowledge including Linq
– Good .Net experience including charting and winforms applications.
– Experience building and maintaining automated test systems
– Ability to interact with the developers to anticipate future needs
– Ability to influence to ensure good test procedures are developed and implemented
• Desirable Skills – Some knowledge of Java for mobile testing
– An interest in photography.
What's happening?
• Stampede to mobile computing
• Big Data
• Internet of (Every)Thing, Pervasive Computing
• Continuous Delivery, DevOps
• (Test) Analytics, Data-Driven Development
• Shift-left, embedded testers, no test team
• Did you really think Agile was the last word?
ALL Testing is
Exploratory
We explore sources of knowledge ...
... to build test models ...
... that inform our testing.
Forget Logistics (for the time being)
Document or not?
Automated or manual?
Agile v waterfall?
Planned v exploratory?
This technology v that technology?
Judgement, exploring and testing
Testing (the system)
Our model(s) are adequate
Our model(s) are not adequate
Exploring (sources)
Judgement
Creates test models
Uses test models
We explore sources of knowledge to build test models that inform our testing
BTW – Do Developers explore the same way? I think so.
Exploration process
Exploration Definitions specs/stories
People (& you)
Sources
Require-ments
Test Models
Enquiring
Challenging
Sources: People, documents, experience, system under test
Modelling
Test Models: Can be documented
or mental models
Predicting
System under test
Testing process
Testing System
Under Test
Refining
Informing Applying
Interpreting
Test Models
Revise the System
More exploring Reporting
New Model Testing
Paper: http://dev.sp.qa/download/newModel
Consequences of the model
• ALL technologies, methods and logistics
• Aligns with/supports (tentatively): – Waterfall, Agile, Continuous Delivery, TDD/BDD etc.
– The Context-Driven View
– Functional, (Possibly all) Non-Functional types
• A different perspective on: – "Testing v Checking"
– Developer Testing, Embedded Testers, Shift Left
– Test automation
– Skills, capabilities and Certification.
Capabilities
Enquiring, Modelling, Predicting, Challenging
Informing, Applying, Interpreting, Refining
Reporting and Logging
New Model and Skills
• The 'New Model suggests that the core skills
of testers are somewhat different
– Somewhat different to those promoted by
'thought leaders'
– Very different to those promoted by the
certification schemes
• Of course interpersonal skills are, perhaps, the
most critical.
• Analysis, enquiry and elicitation
• Modelling
• Creation of custom models, using
heuristics, guesses, brainstorming,
ideation, creative thinking
• Custom test design techniques
• Comparison of models, value, advantages,
disadvantages, compromises
• Identification, validation and use of
oracles
• Predicate logic and proof
• Hypothesis and inference
• Socratic method
• Rapid Review and Inspection techniques
• Test case design
• Test models and the meaning of coverage
• Testing as controlled experiment
• Observation, Note taking, recording
A very different skillset • Basic data analysis and statistics
• Decision-making with incomplete data
• Computer forensics
• Fault tree analysis
• Failure diagnosis
• Bug advocacy, triage processes and
negotiation
• Meaningful software and test metrics
• Visual presentation of data
• Reporting and presentation skills
• Understanding stakeholders
• Test analytics
• Risk management, risk-based testing and
decision-making
• Critical Thinking
• Interpersonal skills
• Dealing with uncertainty/fallibility
Testing Career Development (speculative)
Foundations
Specialist/Technical
Management
Strategic
Test Strategy Project Intelligence
Test Assurance
Exploration Forensics Interpretation
Scripting/ Programming
Test Automation Technical (Excel, SQL, OS utils etc)
Stakeholder management
Analytics & visualisation
Managing uncertainty
Critical Thinking
ISTQB etc...
Supplier Management
Test Process Management
Methodology
People Skills for Better Business (courtesy of Susan Windsor)
Foundations
Specialist
Management
Strategic
Critical Thinking
Supplier Selection and Management
Stakeholder Identification
Creating Shared Goals
Collaborative Team Working
Managing Cultural Differences
Managing Challenging Conversations
Enhance Your Communication Style
Managing Self Assessment
Define Team Qualities and Competencies
Implementing Assurance
What can we do with this?
• A broader tester skills framework?
– New syllabi
– New skills and training
– A competitive certification scheme? (No, thanks)
• Alignment with BDD, ATDD, TDD...
• Flexibility (agility) to cope with any dev/test
process?
Testing doesn’t just
apply to existing
software, at the end
We test systems, not software.
We test early, we test often but it
might not be called testing
The phase after
development is
REWORK, not Testing
Testing must align with development,
not compete with it or rescue it
A New Model for Testing
THANK YOU!
@pau
l_ge
rrard
Paul Gerrard [email protected]
gerrardconsulting.com
Programme Chair 2014