make it fast: delivering ux research to agile teams
TRANSCRIPT
User Research for Agile Teams
One of the biggest challenges facing UX designers working with agile teams is providing user research in a quick, effective
way. Design sprints take less time than in the past and development makes it difficult to slip user feedback into the mix.
Traditional research takes time to design, set up, recruit for, run and analyze. Since that could span several sprints,
“traditional” research simply doesn’t work in today’s rapid pace development, and the user experience suffers. Many
organizations are tackling this challenge. We’ve brought together 4 panelists who are using methods to address the issue of
rapid UX research. Panelists come from both in-house teams and agencies. We’ll share our approaches and offer practical
advice about how to do it, why it works and what could be improved. We’ll cover both unmoderated tests and more
traditional moderated tests. You’ll learn some new approaches and get a chance to ask questions or share your own
experiences.
Panel Discussion
Moderator: Sarah Bloomer, SVP UX
GfK
Chris Chiusano Senior Manager UX
athenahealth
Luis Valencia UX Specialist
GfK
Jen McGinn Director of User Experience
Veracode
Jennifer Fabrizi Experience Design Consultant
Slalom LLC
At athenahealth we believe healthcare is broken and we
have the ambitious mission to make it work the way it should.
Agile has become the most efficient path to delivering value
and true customer success to our users. But, how canone1
scrum team experience designer possibly fit in research,
design and iteration into a 2 week sprint? We can share a
couple of things that make it possible. Including, being
embedded on a single scrum team, UX review as part of the
definition of done for every story, listening to customers
constantly, and measuring success early and often. We look
forward to sharing some proven tips!
Chris Chiusano Senior Manager UX, athenahealth
OVERVIEW OF A RELEASE
Each release at athena is 16 weeks long from
development kick-off to GA release, but the Planning
phase should start about 6 weeks before Sprinting to
ensure developers can immediately start working on
correct plans. At the end of the cycle, 4 weeks of
sandbox & alpha testing occurs between completion of
regular build sprints and GA release.
CONCEPT DEFINITION
From a scrum team designer perspective, there are 3
phases to each release:
• Concept definition
• Detailed design
• Refinement & readiness
DETAILED DESIGN
REFINEMENT & READINESS
OVERVIEW OF A RELEASE - IDEAL CASE
Numerous types of activities need to be completed by designers throughout the release cycle:
Business problem framing,
alignment and approval
Build understanding and empathy through
research, analytics and testing
Conceptual / architectural system
design
Roadmapping from high-level
backcasts to sprint plans
Detailed design in collaboration
with PO’s and developers.
Supporting developers in
execution.
Our client is creating a new digital product for one of their insurance lines-of-
business. They’re using an enterprise variation of agile called “SAFe Agile”
(Scaled Agile Framework) consisting of a Product Team and 6 scrum teams.
They wanted a complete set of tested wireframes before launching the
program, which is really evidence of the vestige of waterfall thinking. But
what they actually needed was some light-weight user research and a way to
jump start concept designs to enable the scrum teams to work on detailed
designs collaboratively in the sprint cadence instead of the old Big Design
Upfront approach. To meet the real need while still delivering the client
“enough design upfront” to reduce angst about the designs, I used a Slalom
Design Sprint in the runway leading up to the program launch. The Design
Sprint consisted of research, a series of workshops, plus lightening usability
tests. As a byproduct, the Design Sprint started the program’s path to
transformation by creating a new, common “experience of design” for
everyone on the team.
Jen Fabrizi Experience Design Consultant, Slalom LLC
http://www.scaledagileframework.com/
SAFe is an agile framework that our corporate clients use.
Balancing Enterprise Ops with Agile Principles & Behaviors
1
2
3
4
http://www.scaledagileframework.com/
Where to Add UX Research & Design in SAFe
1 Portfolio & Strategic Themes Example • Corporate XD Strategist at the Portfolio level to
support business strategists by layering in customer/user research
• Slalom Design Sprints for new product ideation & visualization
• Product validation research
Value Stream Example • Portfolio XD Strategist to support/influence Value
Stream engineers with outside-in PoVs • Slalom Design Sprints for new product refinement &
visualization • Product validation research
Program/”Agile Release Train” Example • XD Lead on Product Teams (2-3) to support Product
Manager with detailed user research, usability testing, & UX strategy at the product level
• Work with Product Owners Increment Planning to schedule sprints for usability testing
XD Information Architects/Interaction Designers/Visual Designers/Front-end Developers • XD Delivery Resources on Scrum Teams for detailed
design & team support
2
3
4
At Veracode, our UX scrum team supports 18 dev teams.
Because our PM and Dev teams create a 12-month product
roadmap, we often have the ability to design a quarter ahead
of development. In that case, we conduct lightweight
research during design “discovery” and then lightweight
validation of the designs prior to implementation. When we
need to pivot quickly, we conduct rapid Design Studios that
get our stakeholders involved and sketching, but bring us to
results in 4 hours, flat.
Jen McGinn Director of User Experience, Veracode (CA Technologies)
What are the situations? I need to test A changed workflow in existing software A new workflow A new UI component
Inside of a sprint cycle Outside of a sprint cycle
That has to make sense to
New users Existing users
Types of Agile User Research
Concept validation
RITE+Krug testing
Lobby evaluations
Baseline testing
Concept validation RITE+Krug testing Lobby evaluations Baseline testing
Changed workflow x x
New workflow
x x
New UI component
x
Inside a sprint cycle
x x x
Outside a sprint cycle
x x x x
New users
x x x x
Existing users
x x x x
But that’s not all ….
Other methods we use are:
Surveys
Analytics
Interviews
Card sorts
Tree testing
Cognitive walkthroughs
User and task analysis
Competitive analysis
Our client’s new app is being rapidly designed during a pre-
development design sprint. They wanted to test their designs
as it evolved. We test every week to get enough people
overall, yet we don’t want to miss the deeper insights. To
meet this demand we combined both unmoderated and
moderated research, introduced “Quick Tests” in combination
with monthly in-person tests. An unmoderated Quick Test is
ideally completed in 5 business days. We then hold monthly,
moderated in-person tests to dig into the larger issues
identified during the quick tests. Unmoderated tests have
their own challenges, as well as ongoing tests over a long
period of time on the same products. And then there’s
working remotely with the design team.
Luis Valencia UX Specialist, GfK
LEARN & LAUNCH
LEARN & LAUNCH
Quick testing provides
rapid user feedback to
meet the tight deadlines of
weekly design sprints.
QUICK
TESTS
GfK Creates Test Plans Pilot Un-moderated test, then LAUNCH Design Teams’ Proposals
Present Findings Summarize & Begin Again Analyze Tests