![Page 1: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/1.jpg)
“Users are Losers!”“They’ll Like Whatever we Make” and Other FallaciesPresented by Carol Smith @carologic
CodeMash 2013
![Page 2: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/2.jpg)
Many names…
User Experience Ethnography Customer Insight Usability Interaction Design User Research
![Page 3: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/3.jpg)
“Users are Losers!”
![Page 4: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/4.jpg)
Plan A: R-E-S-P-E-C-T
![Page 5: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/5.jpg)
Plan B: Stay
Let’s find out about those losers users! Share what is known Existing users = usability study Observations and interviews Web site – use analytics Social listening
![Page 6: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/6.jpg)
Observations & Interviews Learn about:
User’s environment Real process Interruptions Attitudes and
opinions Problems Goals
![Page 7: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/7.jpg)
Preparation
Plan with a goal/hypothesis Questions
1. Make a guide2. Review3. Test4. Start study
![Page 8: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/8.jpg)
Watch Closely
Share little Related tasks Wait for patterns Save questions Stay out of their
“space” Don’t interrupt
![Page 9: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/9.jpg)
Interview
Clarify observations Why doing? Goal? How typical was this?
Use prepared questions Don’t lead the witness Do listen closely Use their language
![Page 10: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/10.jpg)
http://www.flickr.com/photos/heygabe/ via http://creativecommons.org/licenses/by-nc-sa/2.0/Actual Photo: http://www.flickr.com/photos/heygabe/47206241/
Artifacts!
![Page 11: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/11.jpg)
Logistics
Explicit consent Record video, photo, audio Take notes Give incentives
![Page 12: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/12.jpg)
“We Know Our Users”
![Page 13: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/13.jpg)
How well?
When do they think about your product? In what context? Most important to them? Most like to change?
Web sites used most frequently? Phone? What kind? Etc. Etc.
![Page 14: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/14.jpg)
“I don’t know”
Let’s find out! Market research / segments are a start Go where (they *think*) they are ▪ Starbucks▪ Wal-Mart*▪ Conferences/User Groups
Card sort to test organization of info
![Page 15: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/15.jpg)
Card Sorting
Use to determine: Order of
information Relationships Labels for
navigation Verify correct
audience
http://www.flickr.com/photos/rosenfeldmedia/ via http://creativecommons.org/licenses/by-nc-sa/2.0/
![Page 16: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/16.jpg)
Card Sorting
Maximize probability of users finding content
Explore how people are likely to group items
Identify content likely to be: Difficult to categorize Difficult to find Misunderstood
Gaffney, Gerry. (2000) What is Card Sorting? Usability Techniques Series, Information & Design. http://www.infodesign.com.au/usabilityresources/design/cardsorting.asp
![Page 17: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/17.jpg)
http://www.flickr.com/photos/richtpt via http://creativecommons.org/licenses/by-nc-sa/2.0/
![Page 18: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/18.jpg)
Preventive Care Guidelines
36
One title/subjectConcise and clear
Printed stickers
Numbered for analysis
Short description on back of card if needed
![Page 19: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/19.jpg)
Practice session Allow 1 hr for 50 items - Total of 30 –
100 Name groups of cards Moderated (in-person or remote) Un-moderated (online)
Conducting Study
![Page 20: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/20.jpg)
Conversation (if moderated)
Ask to Describe overall rationale for grouping
cards Show best example What was difficult? What was easy? Happy with final outcome?
![Page 21: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/21.jpg)
Analysis
Code cards = faster data analysis Look for patterns
Excel Spreadsheet (Donna Spencer) Online tools - limited analysis
Screenshot of OptimalSort online tool’s analysis - http://www.optimalworkshop.com/optimalsort.htm
![Page 22: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/22.jpg)
“They’ll Like Whatever we Make”
![Page 23: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/23.jpg)
Really?
Let’s test that Usability test prototypes Rapid, iterative cycles of design and
evaluation Web - feedback from on-site tools Customer feedback/Help desk
![Page 24: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/24.jpg)
Usability Testing
Real users doing real tasks
Using prototypes or live products
Doing assigned tasks without guidance
Observed closely
http://creativecommons.org/licenses/by-sa/2.0/http://www.flickr.com/photos/raphaelquinet/513351385/sizes/l/in/photostream/http://www.flickr.com/photos/raphaelquinet/
![Page 25: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/25.jpg)
Rapid Iterative Testing & Evaluation (RITE)
Qualitative – not quantitative actions + comments
Series of small usability tests 3 participants each day At least 3 days of testing Changes made between testing days
![Page 26: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/26.jpg)
RITE Process
Test Update Test
1
2
3
High
Medium
Low
Priority& Level of
Effort
Day 1 Day 2 Day 3
![Page 27: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/27.jpg)
Recap Sessions
End of each day - after the last session Room with a whiteboard About 30 minutes Discuss
trends seen concerns recommendations prioritize changes for the next round list lower priority changes for future
iterations
![Page 28: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/28.jpg)
RITE Results
Final prototype Vetted with users Base for recommendations
Light Report: “Caterpillar to Butterfly” Screenshots show progressions What changes were made and why
![Page 29: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/29.jpg)
Testing
Traditional Testing In-Person Remote
Moderated or Un-moderated
![Page 30: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/30.jpg)
Habitual Testing
(Yes, this is an old idea; a great one!)
![Page 31: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/31.jpg)
Bring it On!
Small focused tests Reduce waiting for recruitment Once per week/sprint Same day mid-week Less users, shorter sessions: analyze
at lunch 3 or more participants recommended Half hour to 1 hour each
![Page 32: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/32.jpg)
User Testing Day!
Make team aware Invite everyone Recurring meeting invites for
stakeholders
![Page 33: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/33.jpg)
What could I test?
Work in Progress Multiple projects Prototypes Concepts, rough ideas, brainstorming Competing designs, (A/B testing) Comparative studies across market Conduct interviews to inform research More…
![Page 34: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/34.jpg)
“Teams should stretch to get work into that day’s test and use the cadence to drive productivity.”
- Jeff Gothelf - http://blog.usabilla.com/5-effective-ways-for-usability-testing-to-play-nice-with-agile/
![Page 35: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/35.jpg)
Why Regular?
Team becomes accustomed to steady stream of
qualitative insight ensures quick decisions lines up with business and user goals
Adapted from Jeff Gothelf - http://blog.usabilla.com/5-effective-ways-for-usability-testing-to-play-nice-with-agile/
![Page 36: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/36.jpg)
Include PWD
“We are all only temporarily able-bodied. Accessibility is good for us all.”
Spirit of the law WCAG 2.0 Country specific (Section 508)
-@mollydotcom at #stirtrek 2011 via @carologic
![Page 37: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/37.jpg)
Make it Repeatable
![Page 38: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/38.jpg)
Pre-Book Your Rooms
Test & Observation Rooms Any location will do
Conference rooms Offices Quiet corner of cafeteria Remote
Purchase software - always ready
![Page 39: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/39.jpg)
Create Reusable Templates Screener
Technology use/experience Knowledge of topic
Scripts/Guides Consent Forms Data Collection
![Page 40: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/40.jpg)
“We have a survey set up and are getting data from it.”“Why would we need anything more?”
![Page 41: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/41.jpg)
Surveys
Great way to get quantitative information
Questions Words can have multiple meanings Un-intended meanings
Less people participate now than in past
People save face “It’s not that bad”, “It’s my fault”
Vendors requesting Perfect 10
![Page 42: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/42.jpg)
“We’ll just ask employees to save time and money”
![Page 43: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/43.jpg)
Employees
Too close to the project Know things others wouldn’t about
product Concerns about ego, job, co-workers,
etc. Not the intended user!
![Page 44: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/44.jpg)
“Don’t we need to test 100s of users to get real results?”
![Page 45: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/45.jpg)
5-6 Participants
Studies have shown that testing 5-6 representative users of each user type will reveal 80% of usability issues.
http://www.useit.com/alertbox/20000319.htmlJakob Nielsen’s Alertbox. Why You Only Need to Test with 5 Users. March 19, 2000.
![Page 46: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/46.jpg)
Look for Patterns
Identify repetition After pattern is
found, continuation of
study: Adds cost Delays reporting Low probability of
many new findings
![Page 47: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/47.jpg)
Does Not Mean That…
Testing five users is always enough Can test anyone and have the same
results Smaller groups equate better
findings
![Page 48: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/48.jpg)
“Our design has won awards. Why would we want to change it?”
![Page 49: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/49.jpg)
Why Change?
Visual appearance is important
Must also be usable Designed for users Tasks able to be
completed Organized well
http://www.brainjuicer.com
![Page 50: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/50.jpg)
“We Know it’s Difficult, We Have a Training Program!”
http://www.flickr.com/photos/kaptainkobold/5181464194/sizes/o/in/photostream/http://www.flickr.com/photos/kaptainkobold/
![Page 51: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/51.jpg)
Training
Costs more time and money How long will product be used? Less costly to find and correct issues
than provide training to work around the problem
![Page 52: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/52.jpg)
Other Arguments Against UX
Time Money Can’t talk to our Customers Liability Not needed Invisible ROI
![Page 53: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/53.jpg)
Prepare for Arguments
Be armed with Facts Questions
Don’t just pick a method What do you need to know? What will the stakeholders respond to?
![Page 54: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/54.jpg)
Recommended Readings
54
![Page 55: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/55.jpg)
Contact Carol
@carologic
slideshare.net/carologic
speakerrate.com/speakers/15585-caroljsmith
![Page 56: Users are Losers! They’ll Like Whatever we Make! and Other Fallacies](https://reader031.vdocuments.mx/reader031/viewer/2022013101/554bd95db4c905706a8b536c/html5/thumbnails/56.jpg)
References
Albert, Bill, Tom Tullis, and Donna Tedesco. Beyond the Usability Lab Albert, Bill, Tom Tullis. Measuring the User Experience Beyer, Hugh. User-Centered Agile Methods (Synthesis Lectures on Human-
Centered Informatics) Gothelf , Jeff. http://blog.usabilla.com/5-effective-ways-for-usability-testing-to-
play-nice-with-agile/ Bias, , Randolph G. and Deborah J. Mayhew Cost-Justifying Usability: An Update
for the Internet Age. Henry, S.L. and Martinson, M. Evaluating for Accessibility, Usability Testing in
Diverse Situations. Tutorial, 2003 UPA Conference. Krug, Steve. Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and
Fixing Usability Problems. Molich, Rolf. A Critique of “How to Specify the Participant Group Size for Usability
Studies: A Practitioner’s Guide” by Macefield. Journal of Usability Studies. Vol. 5, Issue 3, May 2010. pg. 124-128.
Nielsen, Jakob’s Alertbox. Why You Only Need to Test with 5 Users. March 19, 2000. and Usability Evangelism: Beneficial or Land Grab? by Jakob Nielsen, Ph.D
Ratcliffe, Lindsay and Marc McNeill. Agile Experience Design: A Digital Designer's Guide to Agile, Lean, and Continuous.
Rubin, Jeffrey and Dana Chisnell. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. John Wiley & Sons, Inc.
The $300 Million Button by Jared Spool