more research, more frequently: how to sell your stakeholders on research
TRANSCRIPT
RESEARCH
FREQUENTLY
I
oooooooooooooooooop
noooooooooooooooooomore
more
TUUUUUUUUUUUUUUUUUUUUUUVChris Avore @erova #confabMN
#WOCINTECH CHAT
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Why we should talk to our customers, users, clients, etc
• Best practices to conduct research
• Research deliverables
What we won’t be talking aboutINTRO
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrWhat we won’t be talking aboutINTRO
RESEARCH
FREQUENTLY
I
oooooooooooooooooop
noooooooooooooooooomore
more
TUUUUUUUUUUUUUUUUUUUUUUVChris Avore @erova #confabMN
#WOCINTECH CHAT
During Investor Day, we also said that at NASDAQ our fundamental mission first and
foremost is to serve our customers. We accomplished this through intensive engagement and feedback with
these customers.
We also received much feedback from the early adopters, most of it
overwhelmingly positive, but this feedback loop is also helping us to develop and quickly
deploy innovative new features.
Bob Greifeld CEO, Nasdaq Inc.
NASDAQ INC Q1 2016 EARNINGS CONFERENCE CALL APRIL 27, 2016 8:00 AM ET
The first thing we have to do is listen to our customers, and in fact, great design …
starts with a really deep listening to the customer.
What are the biggest problems they have in doing their job today, or what do they wish they could do better, or where do they wish they
could take their relationships with their own clients?
We’ve found things that when we’ve shown it to the client, they said, oh, that’s exactly what I need, but I didn’t even know I needed it.
So that’s great design.
Adena Friedman COO & President, Nasdaq Inc.
OPENING REMARKS, PRODESIGN CONFERENCE JANUARY 30 2015
RESEARCH
FREQUENTLY
I
oooooooooooooooooop
noooooooooooooooooomore
more
TUUUUUUUUUUUUUUUUUUUUUUVChris Avore @erova #confabMN
#WOCINTECH CHAT
SURVEY SAYS…
SURVEY RESULTS rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
Location RoleEmployer Team size
SURVEY RESULTS rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
Do you have access to artifacts, transcripts, and other assets from any research activities?
SURVEY RESULTS rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
How frequently is research included in your process or projects?
SURVEY RESULTS rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
Select the barriers that prevent you from conducting research:
SURVEY RESULTS rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrsDo you have a governance strategy in place for research outcomes, assets, artifacts, and transcripts?
LAGGARD MATUREPROGRESSING
Exec Attitude
EARLY
• Limited to user feedback • Market research
• Ad-hoc • Late-stage testing existing products w/
clients • User/advisory groups • Log analysis • Limited to one business or product
• Discovery research w/ client base • Consistent, iterative usability testing • Monitored Beta & Pilot programs • Comprehensive analytics & data collection • Participatory design w/ clients, users, etc
• Discovery research w/ clients, non-clients • Consistent evaluation of current products • Explores new markets, audiences • Multi-channel touchpoints • Spans across organization • Not limited to functionality: includes customer
experience, recruiting & onboarding, etc
• Find what customers don’t like • Usability test to predict the future (CYA) • Confirm current product decisions are sound • Establish shared understanding of customers
and markets by product team
• Prioritize where to focus new innovations • Inform customer/market segmentation • Learn where to adjust product strategy • Identify unmet needs
• No dedicated research team • Product management or marketing
interacts with clients • May hire perception/market research
firm
• Designers lead guerrilla usability tests • May outsource research to agency
• Dedicated research person or team within organization
• Product management & design support the research team
• Management level position to lead alongside product, marketing, commercial etc
• Resources available for non-research teams to conduct research, report findings
• Individual product team • Product management • Marketing
• Unconvinced • Cautious • Enabling • Expecting
• Product management • Marketing • Design • Engineering
• Research processes are managed, consistent • Follow best-practices • Findings are available for wider teams
• Little pre-research planning • Reports are emailed to stakeholders • Little awareness of best practices
• none • Decisions are made consulting existing design research or create need for more research
• Strategy, tactics, findings are consistently applied across projects, orgs, businesses
• C-suite (CEO, CMO, etc) • Commercial leaders • Corporate Strategy • External partners
What does research cover?
What are some of the methods that are being used?
How far does research reach in the organization?
Why is the organization investing in design research?
Who makes up the research team in the organization?
Who uses the findings that come from design research?
What are the policies to how research will be conducted and consistently applied across teams, projects, and the organization?
+
+
+
+
+
Scope
Staffing
Purpose
Governance
Audience
WHAT DO WE DO AS DESIGNERS THAT PREVENT US FROM DOING ?MORE RESEARCH
MORE FREQUENTLY
NO CLEAR RESEARCH STRATEGY
NO CLEAR RESEARCH STRATEGY
ASSUME ONLY WE CAN DO THE RESEARCH
ASSUME ONLY WE CAN DO THE RESEARCH
ASSUME ONLY NEW RESEARCH WILL TELL US WHAT WE NEED
TO KONW
ASSUME ONLY NEW RESEARCH WILL TELL US WHAT WE NEED
TO KNOW
DOGMADOGMA
SELF-SERVING PRIORITIESSELF-SERVING PRIORITIES
NO USEFUL CONCLUSIONNO USEFUL CONCLUSION
IGNORE WHAT CUSTOMERS TELL YOU WHAT THEY WANTIGNORE WHAT CUSTOMERS
TELL YOU WHAT THEY WANT
• No clear strategy why to research • Assume only we can do the research • Demand new research • dogma
• self-serving priorities • No useful conclusions • ignore customers
methods are lemonsRESEARCH
Some
You’re Doing It
Well, actually,
WRONG
MAKING THE BEST OF IT rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrOutsource research to 3rd parties
• Little knowledge sharing beyond hiring
client team
• Difficult to adjust logistics, script,
scope, reach on the fly without
incurring new costs, contracts, etc
• Much insight leaves with the agency
when the project is over
• Check-the-research-box mentality:
one shot to do everything
• Make a case to reallocate funds in-
house or to *your* team
• Create a cross-project, cross-team
repository of reports/assets/etc
• Build elasticity into SOW
Yes,
and…
MAKING THE BEST OF IT rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrOther internal teams do all the research
• Research and findings exist in silos
• Teams who need research the most
may not be able to contribute
• Research priorities may be different
than yours
• Show connections between their
research and your work
• Invite those teams to your meetings
where design decisions, prioritization,
etc are made
• Find ways to help—reporting, recruiting,
managing research assets
Yes,
and…
MAKING THE BEST OF IT rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrFocus groups
• Groupthink
• Self-conscious or deliberately
secretive around strangers
• Little exploration or elaboration
• No additional research necessary
• Get in the room
• Influence the script
• Get leads for more interviews
• No inadmissible evidence:Don’t fully discount what you hear
Yes,
and…
MAKING THE BEST OF IT rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrSurveys
• May give false sense of security
• Can be written to skew one way
• Harder to write than they seem
• No observable human behavior
• Provide opt-in to be contacted for more
comprehensive interviews later
• Mine for verifiable information
• Write questions that are appropriate for
surveys (duh)
Yes,
and…
MAKING THE BEST OF IT rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrOld Research
• Org may think research has been done
and doesn’t need more
• May not address evolving marketplace,
expectations, reality
• May have been discovery research, but
you need usability testing
• Show gaps in understanding or
where things have changed
• Don’t discount research just
because it’s old
• If available, talk to people who
did or paid for the research to
ask what’s missing
Yes,
and…
• Be involved
• Influence the script
• Use any outreach as an opportunity to
establish a relationship for later
• Don’t dismiss the value just if the Internet
doesn’t think it’s the best way
Make THe best of it
DChange
MANAGEMENT- -
klllll lllllm
1Find an
advocate
#WOCINTECH CHAT
IMPO
RTAN
CE Political support
Control over resources
Expertise
Same team
#WOCINTECH CHAT
Selecting the right challenge is paying attention to
who else cares about it
klllll lllllm
ESTABLISH A SENSE OF
URGENCY
klllll lllllmEstablish a sense of urgency
• Too much past success
• Lack of visible crises
• Low performance standards
• Insufficient feedback from
external, trusted sources
Traits of complacent orgs
klllll lllllm
Find external data to refute a comfortable status quo
Talk to unhappy or former customers
Show how profitable future opportunities are unobtainable
Establish a sense of urgency
klllll lllllmEstablish a sense of urgency
There were 70,000 searches for that topic last
quarter—people can’t find this content!
klllll lllllmCelebrate short-term wins
klllll lllllm
• Visible to outsiders
• Unambiguous
• Tied directly to change effort, vision
Celebrate short-term wins
klllll lllllm
• Provide evidence you’re on the right track
• Help hone the vision and long-term strategies
• Build momentum
• Keep bosses/advocates on your side
Celebrate short-term wins
klllll lllllmCelebrate short-term wins
We interviewed over 50 customers this quarter!
klllll lllllmCelebrate short-term wins
We researched patterns across 80% of our product portfolio—almost
everyone is on board!
klllll lllllmCelebrate short-term wins
Our beta program uncovered over 50 bugs we never
found in QC! Good thing we caught them when we did!
klllll lllllmCelebrate short-term wins
Our prototypes saved over $1MM of at-risk
business after those clients saw what was coming!
DChange
MANAGEMENTFind Advocates Create UrgencY
Celebrate The Wins- -
tactics
Tactics
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrPlan for iterative researchTACTICS
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Address research in the project plan
• Plan to iterate, redo, refine, based on the research
• Align research with what’s getting built and when
• Be as consistent as makes sense across projects
Plan for iterative researchTACTICS
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Frame why you’re
researching for yourself,
your team, and stakeholders
• Applies focus
• Sets expectations
Transparent research strategyTACTICS
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Who’s who
• Themes to explore
• Duration of campaign
• Qualifying criteria for
participants
Transparent research strategyTACTICS
• Confirmed participants
• Methods to be used
• Script, if available
• Estimated date to deliver
findings
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
What do we want to learn
and what will we do with it?
Transparent research strategyTACTICS
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Most effective research doesn’t happen in a vacuum
• If dev informs the campaign upfront, they’re more prepared
for outcomes of campaign
• Marketing teams will be more receptive to your findings if
they know ahead of time
Encourage Dev/Product/Marketing collaborationTACTICS
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Critical these teams know about your efforts so you don’t
contact same people
• Gold mine for referrals/leads for recruiting
• Expect to educate these teams on why you want to talk to
“their” clients
Service, sales, and support are alliesTACTICS
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Show you’re on the right track
• Share recordings & video clips
• Don’t sweat cherry-picking
Show execs the good stuffTACTICS
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Before/After case study
• Show evolution of features/products based on user research
• Include quotes/metrics showing success
• Consider audience skeptical and unaware; don’t use jargon
or assume buy-in by default
Keep a case-study nearbyTACTICS
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Big data
• Machine Learning
• Network analysis
• Perception studies
Tie campaigns to larger business projectsTACTICS
}• Listening campaigns
• Sentiment analysis
• New people to talk to
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Benefits of research
• How to:
• conduct an interview
• synthesize research
• prepare a report
Coach your organizationTACTICS:
WORKSHOPS RESOURCES
• Script templates
• Mad-Libs cheat sheet
• Report templates
• Persona templates
tt tt tt tt
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Market segmentation | Buyers | Promoters, saboteurs, etc
Expand the scopeTACTICS:
Plan for iterative research
Transparent research strategy
Encourage Product/Dev involvement
Sales, Service, & Support are allies
Show execs the good stuff
Keep a case study nearby
Tie campaigns to larger business projects
Train & coach to scale research
Expand the scope
Rethink your presentation
abab
abab
ITransparency Collaboration
Planning PromotionI
ONE MORE THING
NO RESEARCH GOVERNANCE STRATEGY
MOSAIQFree Wordpress theme to organize & manage all
your research activities and assets
Perfect for teams and agencies
Tie transcripts, notes, and participants to findings,
personas, and more
Download today at NasdaqDesign.com/Mosaiq
RESEARCH
FREQUENTLY
I
oooooooooooooooooop
noooooooooooooooooomore
more
TUUUUUUUUUUUUUUUUUUUUUUVChris Avore @erova #confabMN
#WOCINTECH CHAT