andy glover - testing is evolving, but where is the evidence - eurostar 2012
DESCRIPTION
EuroSTAR Software Testing Conference 2012 presentation on the evolution of testingTRANSCRIPT
Andy Glover, Exco Intouch
Testing is evolving, but where is the
evidence?
www.eurostarconferences.com
@esconfs #esconfs
www.excointouch.com
Andy Glover
EuroSTAR 2012
BUT WHERE IS THE EVIDENCE?
TESTING IS EVOLVING,
www.excointouch.com
It needs to work!
Highly Regulated Environments
There’s a reason
It’s Safety Critical
www.excointouch.com
Verify and Validate
Verified
Show evidence the requirements have been met in the product
Validated
The product has been built as intended for use
www.excointouch.com
What’s the problem
Testers mind-set:
Pass an audit!
• Need to document [all] actions and decisions
www.excointouch.com
Tester’s Mind Set
• Pass all tests (if possible)
• Reluctant to try new processes
• Reluctant to try new tools
• Functional testing
– Requirements based testing
• Involved late in the development process
www.excointouch.com
Reality
A formal process V. An informal process
• Formal test process near the end of the project. Highly detailed and rigid process.
• Informal test process throughout project. Flexible, low detail.
www.excointouch.com
Reality: Informal Test Process
Hard to describe
Not managed
Seems random at times
www.excointouch.com
Reality: Formal Test Process
• Test Plan & Test Scripts & Traceability Matrix
• Review documents and sign off
– Execute tests
– Sign & Date
• Record screenshots of testing
• Test Report & Issue Logs
• Review documents & test evidence and sign off
• Issue Management
www.excointouch.com
Every project is different
FDA do not prescribe a test approach or techniques
Every software development project is different
There are no best practices, only good practices
www.excointouch.com
Testing Starts From Day One
Testing starts from day one
• Request testability requirements
– Help with test evidence recording and logging
• Review user requirements
• Consider developing test automation
www.excointouch.com
Manage all the testing
Manage the informal testing
Discover where testing happens
Does it add value?
Can it be improved or shared with other testers?
If beneficial, document it
www.excointouch.com
Add Variety
Do more than one type of testing Not just Requirements based testing
Consider group testing sessions
Collaborate with developers on unit testing
www.excointouch.com
Merge the [testing] phases
Agile
• Code, build, test & ‘validate’ in a sprint!
www.excointouch.com
Merge the [testing] phases
Agile
• Code, build, test & ‘validate’ in a sprint!
Hard to fit in with legacy systems
Need automation for regression testing
And… Need to change the whole process!
www.excointouch.com
Agile
Perform a Cost Benefit Analysis. For example:
• Positives:
– Well documented (e.g. books, articles, etc)
• Negatives:
– Change!
– Documentation is still very important in the industry
www.excointouch.com
Agile
Example
2 main testing activities:
Exploratory Testing Sessions
Build and execute Test Automation
www.excointouch.com
Agile
User Stories
Test Ideas
Exploratory Testing
Design Test
Automation
Previous Automation
Automated Regression
Suite
Subset
Review
Test Report
www.excointouch.com
Agile
Exploratory sessions
• Whole team to review & update user stories
• Use the user stories as charters during testing
• Video record the testing, adding comments or tags
• Traceability between sessions and user stories
• Review sessions and follow up with more testing or annotated videos
www.excointouch.com
Session Based Testing
• Manage the informal testing in sessions
• Focus testers on testing (mind set)
– Focus on learning
– Finding issues quickly
– Document as you go
www.excointouch.com
Agile
Automation
• Automate as you go within the sprint
• Automate only what needs automating
– Focus on checking the software
• Add to the automated regression suite
• Work with the developers as one team
• Use the automation to validate product:
– Build reviews/checks around the automation
– Produce screenshots & reports from the automation
www.excointouch.com
Pilot
• Try Agile in a pilot project
• Develop the process as you go
• Allow team to own the process
• Set the challenge to complete all tasks in each sprint
– Can use the hardening sprint for documentation
www.excointouch.com
It’s all about the evidence
Highly detailed test scripts are common in the industry
• To prove the testing was performed
• As it’s hard to prove the software works!
www.excointouch.com
It’s all about the evidence
Video shows software working in specific scenarios
• Less need for detailed test scripts
• Argument tests need to be self explanatory are no longer valid
– Reviewing sessions and video evidence requires skill and time
– The tests are more powerful (re-use, variable inputs).
www.excointouch.com
Slide layout for large images NOT to be used for text
www.excointouch.com
Learn
Learning doesn’t stop at the end of conferences
Learn from other testers in the same industry as well as other industries
Books, magazines/articles, blogs, forums, exercises/group sessions
www.excointouch.com
Skill
As a tester, focus on testing skills
• Ability to learn quickly
• Thinking Skills (critical, lateral)
• Technical knowhow and/or programming skills
• Training/exercises
www.excointouch.com
Summary
Highly regulated environments can develop and learn from other industries
Suggestions on new processes/tools
Testers must keep learning and developing skills
THANK YOU
www.excointouch.com
Andy Glover
Head of Testing at Exco InTouch
Software Testing community:
Blog: http://cartoontester.blogspot.com
Twitter: @cartoontester
I regularly contribute on STC’s The Testing Planet
www.excointouch.com
EXCO INTOUCH
Our mission is to improve clinical and healthcare outcomes through simple, effective and engaging
patient communication
Exco InTouch is a privately owned & managed company, established in 2004
Operational offices in UK, USA and Canada
Studies conducted in 85 countries with over 750,000 patients
www.excointouch.com
Questions