oredev 2014 knjohnson: ux & software testing

A VENN DIAGRAM OF UX & SOFTWARE TESTING Karen N. Johnson Oredev 2014

The team needs to build a product. The team readily recognizes the testers need to work

with the developers but the same team often doesn’t consider that the testers also need to

work with the UX staff. Often the UX staff is tucked away in a different part of the office,

working with multiple teams and yet, rarely working directly with the testers. Why? How

can testers review a product without a good understanding of the design? Testers need

closer access to UX and UX would benefit from working directly with the testers. Karen

highlights specific design areas where UX and testing pair well.

About this talk …

Oredev - UX & Software Testing © Karen N. Johnson 2014

•  Software Test Consultant •  Published Author (Beautiful Testing) •  Co-founder of WREST: Workshop on Regulated Software Testing •  Website: www.karennicolejohnson.com or www.karennjohnson.com •  Twitter: @karennjohnson

Karen N. Johnson


So what’s a Venn diagram???


A/B Testing A/B testing refers to a marketing experiment when two options are offered. An example is two versions of a web page to see which page results in more successful progression, successful typically is referring to customers progressing to the next step in a sales progression of an ecommerce site.


A/B Testing from UX perspective

¨  Look and feel ¨  Control test ¨  Funnel optimization ¨  Increased sales


A/B Testing from Software Testing perspective

¨  Don’t break the flow ¨  Test paths ¨  Work with UX to build

accounts to control and test the paths.



UX Page design Market feedback Conversion improvement

Testing

A/B split handled

Both “paths” Don’t break overall flow

Look & Feel Nav Process flow

A/B “Testing” with UX & Testing

Multivariate testing Multivariate testing is a technique for testing changes to many different elements all at the same time on one web page.


Funnel Optimization ¨  Conversion rate is a common

metric for gauging customers taking the next step.

¨  UX may design with funnel optimization in mind but testing does not typically approach a product from this mindset.


Storyboarding


The premise … that UX and Test can benefit by

storyboarding together.

storyboarding


A storyboard visually

tells the story

panel by panel.


UX designers use storyboarding.

Designers focus on the flow

of a feature story.


Testers focus on where and how a story can breakdown.


Let’s try another example. Let’s try this one together.


But if you only learn one question to ask, let the question be …

What if?


•  What if? Is a challenge question.

•  What if? Is the type of question that begs for investigation and

discovery.

•  What if? Does not require technical training or knowledge.


When UX & Test collaborate on storyboarding,

together they can think through a process flow

and also consider how the process can break.

The result: Risks revealed. Testing understood.

Stronger design built. More complete testing.


State Transitions


State transition when defined in

academic terms can be long and

(frankly) dull.

A shorter more practical explanation is:

identify states and the events that

trigger the transition.


Let’s go back to the ecommerce

shopping cart flow and see

where the state transitions

occur.


“State transitions provide a

Bermuda-Triangle-like moment in time

where “things” can go wrong for a user

or system. And once a user or system

is in a bad state, they can be awkwardly

trapped or lost.”


When UX & Test map state transitions in a

process, it becomes visible where points of

vulnerability lie. These potential problem spots

need design and test considerations.


Using personas


The premise … that UX and Test can work these

two concepts together to arrive to a better end.

personas & tours


The concept of using a

more precise model helps

to envision designing a

product for someone, not

just for a “user.”

UX designers use the persona concept


The tour concept represent specific, focused testing advice that guides the tester through paths of the software. The tours help a tester think through inputs, data and environment settings that satisfy the goals of the tour. It's a more methodical and purposeful way to test.

Exploratory testers use the tour concept


Exploratory testing & the tour concept

James Whittaker

At Microsoft a group of us test folk from around the division and around the company are experimenting with tour-guided testing. We have the Supermodel Tour in which all we do is window shop … look at the interface and find superficial (no intended disrespect to any actual supermodel who might be reading this blog) bugs. We have the Money Tour where we watch salespeople demo the product and create variations of those demos as test cases. We have the Intellectual Tour where we ask the software hard questions. We have the Garbage Collector’s Tour, the Competitor’s Tour, the Museum Tour, the After Hours Tour and a host more that we are experimenting with. All of these tours represent specific, focused testing advice that guides the tester through paths of the software. The tours help a tester think through inputs, data and environment settings that satisfy the goals of the tour. It's a more methodical and purposeful way to test.

Source: http://blogs.msdn.com/b/james_whittaker/archive/2009/04/06/tour-of-the-month-the-landmark-tour.aspx


Persona + Context + Exploratory Testing + Tours


Imagine a persona combined with a context (such as a mobile banking) app combined with a “money” exploratory testing tour.


If we design by persona,

why not test by persona?

When UX & Test collaborate on personas and

exploratory tour testing together, they can both

think through not just from a user perspective but

through a model. This combination can help

clarify design needs as well as streamline testing.



Accessibility: Consider an accessibility persona Browsers: Be fluent in browser settings and coach your UX designers Review browser stats on an intermittent basis with your UX designer Mobile: Review stats Watch app feedback and help loop feedback into design


Testers can: •  Give feedback on look & feel •  Share our test lab & devices •  Test with different resolutions, browsers, devices. •  Learn UX language for better collaboration. •  Test by persona. •  Highlight error and boundary conditions where design is needed. •  A/B testing (and multivariate testing) offer new forms of “white box testing.”


UX can: •  Be open to feedback on look & feel •  Share design strategy •  Highlight concerns with different resolutions, browsers, devices. •  Learn testing language for better collaboration. •  Suggest testing by persona. •  Ask testers to explore error and boundary conditions to ensure design “holds up” to expectations. •  Highlight where A/B testing (and multivariate) testing is setup so testers can explore.

Email: [email protected] Twitter: @karennjohnson

Karen N. Johnson


oredev 2014 knjohnson: ux & software testing

Technology

ab testing ab testing

testing changes

testing pair

ux designers

ux staff

ux perspective look

karennjohnson karen

software test consultant