Research Methods in UXUXMad 9/19/13

Schedule

● Socializing/Mingling● Intros/Announcements

○ Jobs/Opportunities● Brief Presentation

○ Why research?○ What research is○ A few methods

● Workshop!● Afterparty?

About Me

Brad Orego

me@bradorego.com@bradorego

B.S. in Computer Science, B.A. in PsychologyUX Lead at NexttUXMad Meetup Co-OrganizerEntrepreneur

Dancer, Curler, Ultimate-er, Cyclist, Beer-batter pancake blogger

Why research?

Why?

● UX is important● UX sells● UX is the key differentiator● People matter● We aren’t our users● Build a solution that people want● Solve a problem people have

How?

● Qualitative○ Focus Groups/Interviews○ Design Ethnography○ Usability Studies○ Diary Studies

● Quantitative ○ Surveys○ Metrics○ Usability studies

Details!

Qualitative

Qualitative

● Goals○ Meaning - how people see the world○ Context - the world in which people act○ Process - what actions and activities people do○ Reasoning - why people behave the way they do

● Attributes○ Creativity - what might be interesting to see?○ Subjective - influence by personal opinions○ Inductive reasoning - few examples, then extrapolate

● Data○ field notes, recordings, diary entries, …○ thoughts, views, assumptions, feelings

Focus Groups & Interviews

● Benefits○ may be structured, directed, detailed○ put real people/faces/stories behind data

● Problems○ highly subjective; open to bias by interviewer○ not in-situ

● Tools○ Flow - keep them talking when it’s interesting○ Non-direction - let them meander○ Transition - close current; open new conversation○ Depth - tell me more

Design Ethnography

● Comes from anthropology○ Science!

● In-situ○ observation, interviews○ get the real story

● How○ What people say, do, and use○ Why they do it this way. Why not another way?

● Be…○ Non-disruptive, non-interventionist, unbiased

Usability Studies

● More appropriate for actual artifacts○ mock-ups, existing systems, prototypes, etc

● Methods○ Heuristic Evaluation - rate it against a set of criteria○ Conceptual Modelling - show & tell○ Direct Observation - watch people play with it

● Interviews○ Retrospective - record it then show them○ Critical Incidence - the parts that stood out

● Data○ audio/video recordings, handwritten notes○ needs to be codified, grouped, summarized

Diary Studies

● Participants answer over a period of time○ Same questions? Different questions? Open/closed?

● Pros○ Longitudinal, in-depth self-report data○ Highly detailed; very realistic○ Trends over time

● Cons○ Expensive; difficult to get right○ Self-report data○ Availability Bias

Numbers!

Quantitative

Quantitative

● Goals○ Numbers○ Trends in data○ Read between the lines○ Answer specific questions

● Attributes○ Large(r) sample sizes○ Phone, online, in-person○ Open vs. closed questions

● Data○ Means, medians, SDs, regressions, correlations○ Scalar values, differentials, ratios, ...

Surveys

● Questionnaires with specific measures, hypotheses, and results○ SPECIFIC

● Types of Questions○ Multiple Choice; Scalar; Matching/Association○ Open-ended

● Things to consider○ What to ask? How to ask it?○ Ordering effects○ Validity threats

Metrics

● Measuring real behavior, interactions, use○ Doesn’t have to be on a real system

● Numbers!○ Click-through rates○ Time spent on an activity○ Error rate

● Method○ Have a specific thing to measure○ Figure out how to measure it○ Figure out why the results are the way they are

Usability Studies

● Not as hard-and-fast as other Qualitative Data○ In-person, so you can dive deeper

● Be specific (numbers!)○ Measure success/failure rate○ Measure time○ A/B test

● Continuous Evaluation○ Measure real systems in real use○ Metrics, analytics, log files

How?

● Qualitative○ Focus Groups/Interviews○ Design Ethnography○ Usability Studies○ Diary Studies

● Quantitative ○ Surveys○ Metrics○ Usability studies

Now what?

Tonight's Meetup

● Socializing/Mingling● Intros/Announcements

○ Jobs● Brief Presentation

○ Why research?○ What research is○ A few methods

● Workshop!● Afterparty?

Workshop!

References/Further Reading● http://carmster.com/hci/● http://www.cs.rochester.edu/courses/212/fall2012/● http://www.upassoc.

org/usability_resources/usability_in_the_real_world/roi_of_usability.html● http://www.youtube.com/watch?v=O94kYyzqvTc● http://uxmag.com/topics/research-methods-and-techniques● http://www.usability.gov/● http://www.nngroup.com/articles/which-ux-research-methods/● https://speakerdeck.com/tsharon/validating-assumptions-with-12-ux-

research-methods● http://www.uxbooth.com/articles/complete-beginners-guide-to-design-

research/● http://www.slideshare.net/dgcooley/introduction-to-ux-research-methods● http://www.nngroup.com/articles/discount-usability-20-years/