user research fast & cheap
DESCRIPTION
You don’t need a big budget, weeks of time or special labs to get user insights quickly and inexpensively. We’ll discuss how you can meet your goals, improve your products and make informed decisions through user research. Usability testing (remote & in-person), interviews, surveys and analytics are a few methods we’ll review, particularly in the context of your own business challenges and user questions.TRANSCRIPT
user research fast & cheap
John DouglassPhoto: ahearnecyclespdx.blogspot.com
Google+
Agenda
METHODS
Measuring Success
Why User Research?
6 Fast & Cheap Methods
Applications
Questions
INTRO
IMPLEMENTATION
How do you measure the success of your sites and apps?
A process to study behavior
Why is it important?
Pain Squad: keeping sick kids focused
Image: aiga.com
90% compliance rate (unheard of!)Higher quality reportsScaling internationally
Image: aiga.com
Amazon’s $300 million button
# of customers went up by 45%extra $15 million the first month $300 million by end of year
Quantitative - how many or how much
Flickr: Anirudh Koul
Qualitative - why or how to fix a problem
All this sounds expensive and time-consuming.
Hidden Gems
Stuart Chalmers, flickr
Hidden GemsDig up user insights floating around the organization.
Go hunting for* Search logs / analytics* Dept or agency reports - marketing, PR, comms* CRM application data (Salesforce)* Logs from call center* Information surrogates - volunteers, fundraisers
Ask others* What’s surprising to you? What do our users want?
Hidden GemsBenefits* Little preparation needed* Makes allies, gets you closer to stakeholders
Pro Tips* Set time limits for each source, break up reviews * Put findings on stickies, organize into groups over time
Heuristics Reviews
Flickr: Official U.S. Air Force
Heuristics ReviewsEnsure optimal usability by assessing against a predefined set of guidelines.
Method * Determine your scope: entire app, forms, content, etc.* Choose a set of guidelines. Ex: http://goo.gl/GmAUi * Document applicable guidelines and whether your product conforms* Use “yes/no” or scales of severity (1-5)
Heuristics ReviewsBenefits Can conduct quickly: from a few hours to 1-2 days tops.
Pro Tips * Have a colleague conduct a separate review. Compare and discuss. * Conduct the review (or a part of it) in front of key stakeholders. * Focus on “top 5” list and assign to responsible parties.
Cognitive Walkthroughs
Flickr: seantoyer
Cognitive WalkthroughsGroup inspection to discover how easy it is for new users to accomplish tasks with the system
Method (1) Scope a task - donation flow, sign-up, searching, etc.(2) Write down the “happy path”(3) Recruit 2-4 colleagues not on the web team. New employees, volunteers, support staff work well. (4) Schedule 1-hour session w/ colleague participants and key stakeholders (designer, developer and/or PM) as observers . (5) Ask stakeholders to look out for 2 things...
Cognitive Walkthroughs
* Do participants know what to do at this step?
* If the participants do the right thing, do they know that they did the right thing, and are making progress towards their goal?
Cognitive WalkthroughsBenefits Can conduct at any stage of development: wireframes or design mocks work well.
Pro Tips * Keep the scope manageable, no more than 1-2 tasks.* Hold pre-briefing for stakeholder observers to deflect design defensiveness, establish leadership of moderator. * Brief after the session on what people took away.* Focus on top 5 issues
Lightning Surveys
Flickr: Thomas Hawk
Lightning SurveysUse Google Consumer Surveys to find preferences, opinions, behavior and beliefs.
Method (1) Scope question(s): “which logo is better?”, “how many people have heard of our product?”, “which services are most important to our key demographic?” (2) Scope an audience: general US population, doctors only, people who are familiar with your product, etc. (3) Write questions following best practices, see goo.gl/ohPHY
Lightning SurveysBenefits * Can take as little as 10 minutes to deploy. Days to get results. * Awesome graphs and data manipulation. * Statistically accurate.
Pro Tips * Field questions with key stakeholders before the test for buy-in.* Print graphs and put around the office to socialize findings.
Remote Recruiting
Flickr: A.Davey
Remote RecruitingIntercept users of your site or immediately respond to mentions on social media. All in real time. Create a participant database.
Method (1) Determine the type of user you’d like to target. Write questions to screen out broader audiences.(2) Use Ethn.io to add screener to your site or you can send out a link via email, social channels. (3) Use your database of users for whatever you want: live remote interviews, surveys, in-person testing, etc.
Remote RecruitingBenefits * MUCH cheaper than recruiting agencies. Can use database for other methods: interviews, testing, etc.* Recruit people who are real users of your sites/apps, not just those who say they are.
Pro Tips * Check with your legal dept to clear compensation and non-disclosure documents, when you interview/test. * Read the book Remote Research (Bolt, Tulathimutte) goo.gl/uIvxe. Good guides/scripts and tech advice.
Unmoderated Testing
Flickr: Ed Yourdon
Unmoderated TestingHave users complete tasks and answer questions on your site or on competitors. Recommend Loop11.com.
Method (1) Determine which tasks / sections your team wants usability feedback on.(2) Write 5-10 task scenarios that have users attempting realistic tasks in which they interact with the parts of the application you want to test.(3) Send link to recruits (from Ethnio, other databases). (4) Launch project and analyze results.
Unmoderated Testing - Loop11Benefits * Support available throughout the process. * Available for iPad and iPhone. * Can run a test on ANY website. No code to embed. * Pricing for non-profits ~$158 / month or 1-time use $350.
Pro Tips * Setup monthly benchmarking to evaluate changes over time. Let it run on “auto-pilot”, analyze for a few hours each month.
Hidden Gems - uncovering insights in own org
Heuristics Reviews - expert review using guidelines
Cognitive Walkthrough - group review by task
Lightening Surveys - consumer surveys in minutes
Remote Recruiting - Ethnio to recruit a database
Unmoderated Testing - Loop11 for testing tasks
Be Inclusive
Dump Brains
Stay Childlike
Use Snack Sizes
Principles to get most from methods
It’s Our Research (Sharon)
Going Deeper
Remote Research (Bolt, Tulathimutte)
Observing the User Experience (Goodman et. al.)
More remote research tools: http://remoteresear.ch/tools/
Questions?
Thank you! Get at me...
John DouglassGoogle+ http://goo.gl/DwvXHTwitter @JDouglass