AGILE -FR IENDLY USER RESEARCHU X P E O P L E 2 0 1 3

Nina BelkLead Research StrategistDigitasLBi

Twitter: @ninabelkLinkedIn: uk.linkedin.com/in/ninabelk/

DISCLAIMERThis is a workshop about research,

not about agile project delivery.

“It costs too much.”“It takes too long.”“We don’t need it. We’re building something new, people will just ask for a faster horse!”

Objections:

“We can just optimise based on analytics data once we’ve launched.”

SOLUTION:

A cautionary tale…

Sorry, I can’t show you slides to illustrate the or tell you who the client is.

Here’s the summary: when we did some post launch user research and discovered the client assumptions were badly wrong, and then redesigned some key pages and tools on the site, conversion went through the roof.

The Agile Manifesto (for software development)“We are uncovering better ways of developingsoftware by doing it and helping others do it.Through this work we have come to value:

Individuals and interactions over processes and toolsWorking software over comprehensive documentationCustomer [client] collaboration over contract negotiationResponding to change over following a planThat is, while there is value in the items onthe right, we value the items on the left more.”

Source: http://agilemanifesto.org/

Source: http://agilemanifesto.org/

The 12 Principles of Agile (software development)1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable

software. 2. Welcome changing requirements, even late in development. Agile processes harness change for

the customer's competitive advantage.3. Deliver working software frequently, from a couple of weeks to a couple of months, with a

preference to the shorter timescale. 4. Business people and developers must work together daily throughout the project. 5. Build projects around motivated individuals. Give them the environment and support they need,

and trust them to get the job done. 6. The most efficient and effective method of conveying information to and within a development

team is face-to-face conversation. 7. Working software is the primary measure of progress. Agile processes promote sustainable

development. 8. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.9. Continuous attention to technical excellence and good design enhances agility. 10. Simplicity--the art of maximizing the amount of work not done--is essential. 11. The best architectures, requirements, and designs emerge from self-organizing teams. 12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts

its behavior accordingly.

My Brief for an Agile-Friendly Approach:

– Fast– Collaborative– Minimal documentation

The Traditional Approach to User Experience Research

Discussion Guide

Discussion Guide DocumentYou draft a discussion guide, the

client reviews it and provides feedback. You then make

amendments and then send it back to the client for approval.

Clientactivities

Youractivities

User Research SessionsParticipants will be given scenarios that

encourage a natural exploration of the key pages we’re researching.

Client and design team observe.

Discussion Guide

Discussion Guide

Participants recruitedAgainst an agreed recruitment brief.

Brief

Discussion Guide

!

The Traditional Approach to User Experience Research

Data analysisYou analyse the data for patterns.

Clientactivities

Youractivities

Report PresentationYou present your findings and

recommendations to the client and discuss the implications of these with them and the

design team before agreeing on actions.

Detailed Report

Report PreparedYou put together a lengthy

PowerPoint deck and it’s sent to the client and design teams

Report Presentation:

Findings and recommendations

presented & discussed.

Detailed Report

The Agile-Friendly Approach

Participants recruitedAgainst an agreed recruitment brief.

Brief

Discussion Guide

Workshop:Agree areas to explore in the

research.

Discussion Guide

Discussion Guide Document

Workshop to agree discussion guide topics and structure, the output of which we will document for reference not

approval

Collaborative Note TakingObservers note their observations on post-it notes and stick to wall.

Findings Workshop:

Review notes captured during

research sessions &

agree actions.

Findings Summary

Client & Design Teamactivities

Youractivities

User Research SessionsParticipants will be given scenarios that encourage a natural exploration of the

key pages we’re researching.

OutputsFollowing the workshop we will put together a short summary

of the agreed actions for reference not approval.

Recruitment

Don’t give the recruitment criteria away in the screener questions!

Try to word questions in an open manner

TASK ONE:Fix the recruitment screener to prevent professional research participants getting into your study.

TIP TO REMEMBER:Avoid giving the criteria away in the question.

Discussion Guide Workshops

The Discussion Guide WorkshopDiscussion Guide Workshop Agenda:1. Explain the purpose of the workshop2. Explain how you will run the research sessions:

– There’s no script because it feels unnatural – You start with contextual questions to help you build a picture of participants and warm them up to the

content of the session– You then look at what you want to evaluate – Finally you wrap-up with some overall evaluation of what the participants have seen during the course of

the session.

3. With the designers, walkthrough the journeys they would like you to evaluate – these would ideally be printed and stuck on the wall.

4. Brainstorm the research questions you have specific to the experience together and write these on post-it notes and stick alongside the relevant parts of the journey.

5. Create realistic scenarios that allow the participants to get into the right frame of mind and ensure the research questions will be addressed.

6. Brainstorm contextual research questions that can be asked at the start of the research sessions7. Review the standard wrap-up questions and amend where needed:

– Overall what did you think of what we’ve looked at today?– Was there anything you particularly liked that you want to highlight?– Is there anything you would change or add to improve it for you?– Any final comments?

FINALLY: Document what you agree (no client review).

TASK TWO:Create your discussion guide

POINTS TO REMEMMBER:• Walkthrough the journeys you want to evaluate• Together, brainstorm the research questions you have specific to the experience and write these on

post-it notes. Where they are screen specific stick the post-it alongside, where they’re not create a group of questions that apply to all screens.

• Write a realistic scenario(s) that allows the participants to get into the right frame of mind and ensure the research questions will be addressed.

• Brainstorm contextual research questions that can be asked at the start of the research sessions.• Review the standard wrap-up questions and amend where needed.• Document on the template provided.

Facilitating

Research labs can be intimidating places

for participants.

Image sourced from: http://www.flickr.com/photos/kellysue

Help them relax by making them feel

at home.

Introducing Participants to the Session

– An introduction to you– An overview of what the session is about– Explain the tech set-up– Outline the structure– What you’d like them to do.

Facilitation

“I’d like you to imagine that…”

“What’s going on here?”

“What’s this page about?”

“What can you do here?” Top Tip

Repeat the last thing they said in a rising intonation.

“What did you find difficult about that?”

Open with your scenario

Encourage them to think aloud

Asking ‘why’ a lot can feel like an interrogation

It’s hard not to ask leading questions, but don’t panic…

“Can you give me an example

of when?”

…you can always qualify them.

“What do you think will

happen next?”

“Would you find that useful?”

Collaborative Analysis

“Can I have a faster horse

please Mr. Ford?”

Watch, listen and interpret. Don’t just report!

It’s seriously slow getting from A to B

on this horse and cart.

Image sourced from: Detroit Publishing Company photo via Library of Congress website.

Setting up the Observation Room and the Observers

1. Have all pages printed and stuck on brown paper

2. Have an area dedicated to ‘key observations’

3. Give observers a guide to effective note taking

Guide For Observers

1. Watch what they do2. Listen to what they say, and how they say it3. Compare what you see and what you hear

If they struggled but said it was easy trust their actions not their words.

4. Try and interpret what’s going on rather than just reporting. Goal – get a “because” on every post-it note

5. Mark-up your post-it notes with the participant number and the screen they were on (post-it rain is a pain).

6. Discuss observations together after each session 7. Pull out the three most important issues to address. Summarise

these on a post-it not for each participant and place on the key observations sheet.

TASK THREE:Run a short research session

TIPS FOR FACILITATORS:• Put your participant at ease• Open questions and then explore interesting

avenues

TIPS FOR OBSERVERS:• Watch what they do• Listen to what they say and how.• Compare what they do with what they say• Observation + because (try and interpret)

Findings Workshops

Findings Workshop Preparation

Group observations

Summarise key findings for each

screen

The Findings Workshop

Agenda:• Recap on the key research questions• Nominate someone as the actions scribe• Walk through the experience with the discussion guide

as your agenda• Highlight the key findings and discuss the actions the

team could take to address these.• Agree the priority of the design changes• Review the personas and discuss if these need to be

updated based on anything you’ve learned about during the research.

TASK FOUR:Prepare and run a findings workshop

POINTS TO REMEMBER• Group similar observations• Summarise the key findings• Use your discussion guide as a workshop

agenda• Discuss findings and agree actions

“It costs too much.”“It takes too long.”“We don’t need it. We’re building something new, people will just ask for a faster horse!”

Objections:

– It can cost you more not to do research (remember my cautionary tale?)

– If it’s collaborative with minimal documentation, it’s faster and cheaper.

– The real skill of a researcher is interpretation not facilitation!

Thank You