test plan documentation strategies - mind maps, test scripts, and more

Documenting Software Testing Instructions - A Survey of

Successful Approaches

STP Con October 25th, 2011Justin Hunter

Topics

Tradeoffs

Considerations

Documents

Idea Triggers

Context Objectives

Mind Maps“Straw Man”

ChecklistsPersonas

“Kan Ban Cards”

Combinations

Topics

3

Context is Critical

4

Tester’s experience

level / Exploratory

Testing skills

Importance of keeping detailed records (e.g., for regulatory compliance or lawsuit defense)

Context is Critical

5

Tester’s experience

level / Exploratory

Testing skills

Importance of keeping detailed records (e.g., for regulatory compliance or lawsuit defense)

Relatively littledetail

Enough detail to satisfy lawyers

Very detailed

Enough detail to avoid tester confusion

Appropriate Amt. of Detail

6Sources: Key Terms (“Cloud Level” / “Clam Level”, etc.)

http://alistair.cockburn.us/Use+casesImage: accelerateddeliveryplatform.com

Objectives

7

Objectives & Assumptions

8

Who? W

hen?

Prescriptive vs. Exploratory?

Tradeoffs

9

Practical Tradeoffs

10

ManyFew

Reqs Docs and the SUT

Reqs Docs Only

Test ideas from

looking at...

Detailsin tests

Practical Tradeoffs

11

ManyFew

Reqs Docs and the SUT

Reqs Docs Only

Test ideas from

looking at...

Detailsin tests

Prescriptive /Designed Before

Exploratory /Designed During

Documentation Strategy

12

“Straw Man”

13

Every test scriptMUST have:

• Test Name• Test Number• Tester’s Name • Tester’s Dept.• Type of Test • Requirement Number• Pre-conditions• Configuration Details• Test Data• Specific Steps• Designer• Approver• Execution Time (est.)• Expected Results

Time Management

14

Overly-Detailed Documentation

15

ActualTesting

Mind Maps

16

Mind Maps - Test Plan Template

17http://www.softwaretestingclub.com/forum/topics/the-two-

page-test-plan?commentId=751045%3AComment%3A105353

18http://www.softwaretestingclub.com/forum/topics/the-two-page-

test-plan?commentId=751045%3AComment%3A105353

Clear Communication Example

Evolution of a Mind Map

19

“1. Identify Properties you're interested in”

http://testerkiwi.blogspot.com/2011/04/building-exploratory-test-plan-redux.html

Evolution of a Mind Map

20

“2. Start Collecting Intel”

http://testerkiwi.blogspot.com/2011/04/building-exploratory-test-plan-redux.html

Evolution of a Mind Map

21

3. After Talking to Developer...

http://testerkiwi.blogspot.com/2011/04/building-exploratory-test-plan-redux.html

Mind Maps

22

Benefits of mind maps so far... Benefits of mind maps so far...

Creation

Quick and Easy Maintenance

Quick and Easy Collaboration

Quick and Easy

Reporting

Quick and Easy

But would mind maps be detailed enough...

... For junior testers?

... For testers who will join team in six months?

“MindMapping 101 for Testers”

23

http://www.bettertesting.co.uk/content/?p=956

Test Details into Test Mgmt Tool

24

25

Collaboration is Easy

Planning & Prioritizing Risks

26

http://www.bettertesting.co.uk/content/wp-content/uploads/2011/01/Test-Phase-3.4.jpg

27

http://www.bettertesting.co.uk/content/wp-content/uploads/2011/01/Test-Phase-3.4.jpg

Determining Scope - What’s In

28

http://www.bettertesting.co.uk/content/wp-content/uploads/2011/01/Test-Phase-3.4.jpg

Determining Scope - What’s Out

Documentation

29

Kan Ban Boards

30

31

http://www.flickr.com/photos/roxocubes/5786672702/

Paul Holland - (Testing-Focused)

32

Paul HollandAlcatel-Lucent

(personal photo of histesting board, used with permission)

Paul Holland - (Testing-Focused)

33

Each note represents a half-day of testing activity (via Session-Based Testing)

Highly recommended reading: For a detailed description of Paul’s experiences with this approach and his lessons learned, check out Michael

Bolton’s “A Sticky Situation” in an upcoming version of STP Magazine. Also, Google “Session-based Test Management”

Kan Ban-ish Tool: Pivotal Tracker

34

More Details on Hover-Overs

35

More Detail Still by Clicking

36

Including Recent Test Results

37

Test Descriptions / Size Estimates

38

39

Soap Opera Testing

40

http://www.logigear.com/logi_media_dir/Documents/Soap_Opera_Testing.pdf

Tests should be fun and aggressive.

Write scenarios that are:

• Reality-based• Exaggerated, and • Condensed

Soap Opera Testing

41

http://www.logigear.com/logi_media_dir/Documents/Soap_Opera_Testing.pdf

42

http://www.logigear.com/logi_media_dir/Documents/Soap_Opera_Testing.pdf

Soap Opera Testing

Soap Opera Example

43

http://www.logigear.com/logi_media_dir/Documents/Soap_Opera_Testing.pdf

Personas and “Substruction”

44

http://www.software-testing.com.au/blog/2006/07/30/personas-substruction-and-other-trades-tricks/

Behavior-Driven Development

45

Behavior-Driven Development

46

47

How to Select?

48

49

... How to Handle More Details?

50

... and still MORE Details?

How to Document?

51

Test inputs from all 3

mind maps are entered.

jklhlkjhkjhkjh

52

Pairwise coverage in 19 highly-varied tests

Auto-populate Mad Libs-like tests

53

Turned into Detailed Test Scripts

54

Detailed test scripts: quickly generated by Hexawise.

Checklists

55

Checklists

56

• Cem Kaner - The Value of Checklists and the Danger of

Scripts: What Legal Training Suggests for Testers http://

www.kaner.com/pdfs/ValueOfChecklists.pdf

• Justin Hunter - http://hexawise.wordpress.com/2009/11/04/

checklists-good-test-scripts-bad/

• Thoughts from the Test Eye - Software Quality

Characteristics

• Michael Hunter, “You Are Not Done Yet”

• Elizabeth Hendrickson - Testing Cheat Sheet

Software Quality Characteristics

57

http://thetesteye.com/blog/2010/11/software-quality-characteristics-1-0/

Rapid Reporter (Free Tool)

58

http://testing.gershon.info/reporter/

Checklists

59

www.kaner.com/pdfs/ValueOfChecklists.pdf

“Easy-Access Executive Summary”

60

http://bitly.com/bundles/hexawise/2

An organized summary of all the links referred to in this presentation is

available in the link below:

Invitation to SlideShare Viewers

61

https://app.hexawise.com/signup-1-5

If you have read this far, odds are very good that you’re the kind of tester who is interested in trying out promising new testing tools.

Hexawise is a powerful test design tool that allows testers to zero in on a small set of unusually-powerful tests. We’d like you to try it and let us

know what you think about it.

We have an unusual - some have even said bizarre - pricing policy. While enterprise licenses of Hexawise start at $50,000 / year, we make fully-featured versions of Hexawise available at no cost for teams of 1 to 5

testers. It’s like having a rich uncle with a $50,0000 BMW who gave you the keys to his car indefinitely. And he wouldn’t get upset even if you kept it or ruined it. Wouldn’t you at least want to take it out for a test drive?

Sign up now