exploratory testing with jira | qasymphony webinar

Exploratory Testing + JIRAMay 18 2016

Kyle McMeekin - QASymphony

Join the conversation – use the hashtag #exploratorytesting on Twitter

OUR PRESENTER

Kyle McMeekin, Senior Product Specialist at QASymphony

Kyle McMeekin is a Senior Product Specialist at QASymphony focusing his time on customer demonstrations of product offerings and technical support.

He previously spent time as a tester at Cognizant Technology Solutions and has recently moved to the Atlanta area after growing up near Washington, DC and then spending a few years in the arctic known as Minnesota. Graduate of the University of Michigan, Kyle is excited to watch Michigan win the National Title in 2016.


AGENDA

JIRA’s ET Suggestions

Problems With Traditional Testing

Exploratory Testing (ET) Explained

Tools for ET


JIRA AND EXPLORATORY TESTING

• “As the product grows, the amount of testing grows exponentially–and QA invariably struggles to keep up”


https://www.atlassian.com/landing/software-testing?tab=exploratory-software-testing


JIRA AND EXPLORATORY TESTING

• You want exploratory testing to be product focused

• You need a thorough way of capturing test evidence… print screen to Paint to an email/doc no longer works

• Exploratory testing is not just for Testers




WHY SHOULD WE DO ET?

Controlled study link:: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.167.3696&rep=rep1&type=pdf

Testing with test cases vs. exploratory

testing take almost 7x longer

7xTIME SAVED MORE BUGSTesting with test cases

vs. exploratory testing finds more

defects

ACCURACYTesting with test cases

vs. exploratory testing finds more

defects


http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.167.3696&rep=rep1&type=pdf

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.167.3696&rep=rep1&type=pdf

PROBLEMS IN TRADITIONAL TESTING

Prescribing detailed test steps, and test planning, can cloud

interactions.

Focus on compressive documentation blocks cognitive thinking abilities

Customer collaboration in testing remains in silo or

nonexistent. Comprehensive test plans create a

tunnel vision affect

Individuals and interactions over processes

and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan


PROBLEMS IN TRADITIONAL TESTING

Function A

Function B

Function C

Function D

• Step1• Step2

• Step1• Step2

• Step1• Step2

• Step1• Step2Scripted Testing

Function A

Function B

Function C

Function D

Exploratory Testing

• Objective1

• Objective2


WHAT IS EXPLORATORY TESTING?

1. Parallel test planning, test design, and test execution

2. Specific yet flexible

3. Aligned towards investigation of potential opportunities

4. Values depth and attention to detail during testing

5. Fosters knowledge sharing and accountability


PARALLEL PLANNING, DESIGNING, EXECUTION

Unlike traditional testing techniques, planning, design, and execution happen concurrently, allowing efficiencies of time as well as flexibility in approach

PLAN

DESIGN

EXECUTE

REPORT

PLAN

DESIGN

EXECUTE

REPORT

Traditional Scripted Testing Exploratory Testing


Exploratory Testing provides a specific lends through which to perform testing – whether that be a user person, functionality, criteria (i.e. Localization), ect.

However, it allows testers to use the tool as an end user would, not necessarily as the product owner envisioned it.

SPECIFIC YET FLEXIBLE

I tested the application as the end user would

I tested the application as the script prescribed


INVESTIGATE OPPORTUNITIES

Exploratory testing rewards testers who identify unknown areas of “opportunity” within the application, as they are essential in maintaining a backlog of future test charters

Traditional Scripted Testing

Exploratory Testing


EXPLORATORY KNOWLEDGE SHARING

Exploratory testing relies on knowledge sharing to reach full potential – developing testers who understand the impact of more areas of the application allows them to identify more areas of risk and opportunity

Plan

DesignExecute

Report Transfer Learning

Example Questions to Ask

• Have you seen this before?

• What am I not considering?

• Why would someone do this?

• How would you have tested this?


BRINGING IT ALL TOGETHER: CAR SHOPPING

Scripted Test Drive Exploratory Test Drive


WHAT EXPLORATORY TESTING IS NOT

1.Exploratory testing is not unstructured

2.Exploratory testing is not the only type of testing


Successful exploratory testing still contains parameters such as:

- A goal of the exploration

- A log of the activity performed

- A lens through which the testing is performed (user persona)

Performing exploratory testing without involving some parameters such as the above allows a greater risk of unsuccessful implementation of exploratory testing

EXPLORATORY TESTING UNSTRUCTURED TESTING


HOW TO STRUCTURE ET: SESSION BASED

Session Based Test Management is a popular framework, because it tracks important data on the testing, while also adding some much needed structure.

• Session charter (includes a mission statement, and areas to be tested)

• Tester name(s) • Date and time started • Task breakdown (the TBS metrics) • Data files • Test notes • Issues • Bugs

Session Report

More info on SBTM: http://www.satisfice.com/articles/sbtm.pdf


http://www.satisfice.com/articles/sbtm.pdf

EXAMPLE: SESSION REPORTDon’t forget the debrief!

A debrief is a short discussion between the manager and tester (or testers) about the session report. Jonathan Bach uses the acronym PROOF to help structure his debriefing. PROOF stands for:

• Past: What happened during the session?

• Results: What was achieved during the session?

• Obstacles: What got in the way of good testing?

• Outlook: What still needs to be done?

• Feelings: How does the tester feel about all this?

More info on SBTM: http://www.satisfice.com/articles/sbtm.pdf


http://www.satisfice.com/articles/sbtm.pdf

UAT WITH EXPLORATORY TESTING

UAT Challenge ET Benefit

UATer’s are unfamiliar test case syntax and need continual clarification

Allow UATer’s to perform the business flows they know well without test scripts

UATer’s are not trained on test case management, automation tools, etc.

Focus UATer’s time on learning how to document proper defects, reduce time to ramp

UATer’s have a shorter attention span – they are not used to testing 6-8 hrs. per day

Allow UATer’s to veer off the rails from time to time and investigate areas of interest

UATer’s have a short period of time in which to provide feedback

Ensure that as much of the UATer’s time as possible is dedicated to ET


Keep your Best Testing Talent

IMPROVE JOB SATISFACTION MOVE FROM TESTER TO SME

WHY DO EXPLORATORY TESTING?


TOOLS FOR THE JOB


JIRA CAPTURE

JIRA Capture• Capture screen shots• Annotate and share• Create and record

exploratory testing sessions

• Submit issues in to JIRA


qTest Explorer• Applicable on any

application; including mobile, desktop, and web

• Automatically generate TCs from your sessions

• Capture environment details

• All sessions stored in central repository

• Submit test cases and defects directly to qTest or JIRA.

QTEST EXPLORER


REFERENCES / THOUGHT LEADERS

James Bach - http://www.satisfice.com/

Jonathan Bach - https://jonbox.wordpress.com/

Michael Bolton - http://www.developsense.com/

Paul Holland - http://testingthoughts.com/blog/author/testthought

Keith Klain - http://qualityremarks.com/

Brian Osman - https://bjosman.wordpress.com/

Exploratory Testing White Board Friday Series by QASymphony


http://www.satisfice.com/

https://jonbox.wordpress.com/

http://www.developsense.com/

http://testingthoughts.com/blog/author/testthought

http://qualityremarks.com/

https://bjosman.wordpress.com/

https://bjosman.wordpress.com/

http://www.qasymphony.com/blog/whiteboard-friday-video-series-exploratory-testing/

http://www.qasymphony.com/blog/whiteboard-friday-video-series-exploratory-testing/

Thank you

exploratory testing with jira | qasymphony webinar

Education