QualitativeResearchinInteractionDesignandUXMattiasArvola
DepartmentofComputerandInformationScience
EthnographyinDesign
2
• http://vimeo.com/6038262
Researchfordesign
3
• Good design requires insight • Insight requires research • Empathy – The ability to take someone else's
perspective • Get an empirical benchmark that everybody agree on,
so that you after the design can show impact • Do it yourself as a designer
4
By Bill Verplank.
KindsofScientificResearch
5
• Basic research • Applied research • Contract research
ScientificResearch
6
• Systematic search for knowledge • The choice of method depends on what the research
question is • New knowledge • Replication • Systematic and critical review of previous research
ScientificResearchvs.ResearchforDesign 7
• Scientific research is: • Statement by statement,
verified by carefully described observations, or references to such
• Criticisable and verifiable • Reproducible • aimed to distinguish more
likely conclusions from less so
• As objective and without local and temporary prejudices as possible
• Cumulative • For common good
• Research for design is: • Good enough based on
resources • Check that you are on the
right track • Find a baseline to show that
you have made improvements
• Inspiration • Find out what you need to
know to move on or get unstuck
Qualityandtrustworthiness
8
• Adds new original knowledge
• Clear conditions and starting point • Purpose & questions • Methods explained
in relation to questions
• Consideration of method alternatives
• Systematic and critical analysis of carefully gathered data
• Possible sources of errors and limitations identified and discussed
• Arguments clearly articulated, and relevant for conclusions
• Not over-generalise
Ethicconsiderationsatthestartofaproject
9
• Choose the method with the least harmful consequences for people, if otherwise equal
• Expected benefits and potential scientific value of the research should should be weighed against harmful consequences
• Benefits and harm for whom? • Openness is the lodestar • Make dependencies and interests clear
Ethics
10
• Completely voluntary participation • Informed consent: verbal or written • Explain risks and possibilities of participating • Respect participants’ integrity and privacy • Confidentiality: coding and masking
• What do participants get out of it? • Let the participants read/see your results
InformedConsent
11
• Awareness of researcher’s identity and purpose • In general announcements inform about purpose,
research topic, and data gathering methods • Seek permission from gatekeepers • Make sure that there is no pressure to participate • If possible, get informed consent from every
individual
Videorecording
12
• Can intrude on private life and integrity, since people can be identified
• Use only when the same data cannot be obtained by other methods (eg. masked still images)
• Respectful and responsible recording • If minors (<15) are recorded: • Consent from both legal guardians • Informed so that the child understands
• Inform: What will be analysed in the video?
EthicsinParticipantObservation
13
• Long term immersion in everyday activities of the people being studied to record conduct under the widest range of possible settings
• Information is the result of dialogical interaction between the researcher and the informants: any interaction may constitute some form of data gathering
• Informed consent process should be dynamic and continuous
• Unfamiliar settings can be sensitive, and have clashes in ethical principles
Datacollection
14
• Take notes in a hardback notebook • Preferably two persons • Specifically if you are a novice
• Sound recording is important • Look out for tech trouble
• Take photos • Memory aid and details about the context of use
15
Inpractice Overarchingprocess
16
1. Kickoff meeting 2. Stakeholder interviews 3. Competitive analysis and literature study 4. User interviews and observations
Stakeholders
17
• People in the organisation that: • Sponsor, build, test, market, sell, and support the
product • Plus: Other people that affect the design of the
product
Kickoffmeeting
18
• Gather project members and stakeholders • Discuss • What is the product? • Who will use it? • What do the users need? • What customers and users are the most
important? • What challenges do we face?
Stakeholderinterviews
19
• General questions • What is your role in relation to this product? • What did you do before you had this role? • What should this product or service be? • Whom is it for? • When should the version we are designing for be
released? Why is it important that it is released then?
• What do you worry about in this project? What is the worst thing that could happen?
Stakeholderinterviews
20
• Cont. • What is effect should this project have one the
business or organisation? • How would you, personally, define success for this
project? • Is there anyone we should talk to that isn’t on this
list? Who are they? • How would you like to be involved in this project,
and how should we contact you?
Stakeholderinterviews
21
• Marketing: • Who are your customers and users today, and how
would you like it to be in five years? • How does this product fit into the overarching
product strategy? • Who are you biggest competitors and what is it
about them that worry you?
Stakeholderinterviews
22
• Cont. • What 3-4 qualities would you like for people to
associate with your company and your product? • What is the current status of your identity? Could
we have a copy of your graphical profile if there is one and examples of how it has been applied?
23
• Engineering questions • What technology decisions have already been
made? What are the reasons for them? • How many engineers and other tech staff are
allocated to this project, and what are their competencies?
• Can you draw a diagram and broadly explain how the existing system works?
Userinterviewsandobservations
24
• Try to understand what is done, why, what goals there are, what mental models people have, how they feel about things
• Search for similarities and differences between people, what different goals they have and if (how) they solve the same task in different ways
• Interview people in different roles • Make sure you have a protocol
Asktheuserswhattheywant?
25
Asktheuserswhattheywant?
30
• Compare to the physician who asks: • What medicin would you like to have?
• Or the architect that just make the finished drawings from the clients’ sketches?
Generalquestions31
Productorservice
Badgeneralquestions Goodgeneralquestions
Business-orientedemailsystem
Whatemailsystemdoyouuse? Weareinterestedinwhatkindofcommunicationyouhavewithotherpeopleduringatypicaldayatwork.Couldyoustartbygoingthroughyourworkdayyesterday?
Digitalcameraforhobbyuse
Whatkindofcameradoyouhave?
Whatroledoesphotographyplayinyourlife?Whatdoyouusuallytakephotosofandhowoften?
Websiteforcamerabusiness
Howdoyouuseourwebsite? 1. Tellmeaboutthelasttimeyouboughtacamera.
2. Tellmeaboutsomeoccasionwhenyouafterafterbuyingacamerafeltthatyouhaveneededextrainfoorhelp.
Serviceforhelpingstudentsfindrelevantsummerjobs
Haveyouhadajobthissummer?Whatkindofjobhaveyouhadthissummer?
?
Interviewthemes
32
• Objects • Find important nouns • Objects from the mental
model • ”I have seen you work
with three types of forms so far: travel order, travel expenses, and purchase order. What other kinds of forms do you use and why?
• Attributes of the objects
• What information does a travel order contain?
• How many fields are there in a travel order?
• Relations between objects • What is the difference
between a travel order and a travel expense form?
• Quantity • How many travel
orders do you usually handle every week?
33
• Actions • Find verbs
• So every time you get a travel order, you first take a copy for archiving before you contact the owner for to verify.
• The reason for an action • Why should the travel
order be archived? • How is an action performed
and described? • The administrator
looks at a post-it note with codes for accounts
when entering the travel order into the accounting system.
• Frequency and priority • Do you ever wait with
archiving a travel order?
• The photocopier is not warm early in the mornings, so it takes forever to take a copy for archiving, so I usually wait with all the copying until later in the day. There’s usually no hurry anyway.
34
• The role of the present product • I have to enter all
information manually into the the accounting system. since travel orders are handed in on paper.
• Relations to other people: • The cost owner • The one who hands in the
travel order • Archivist • …
• Workarounds • Do you ever not archive a
travel order?
• There is this professor who always changes his mind in the last second. I usually don’t archive his travel orders until he actually have left. I would otherwise have to retrieve them every time he changes his mind.
35
• Frustrations • Sighs, facial expressions, body language, gestures. etc.
• It’s a drag with travel orders since you have to feed them into a PDF, but you cannot save the file, just print it, so if you afterwards realise that you’ve done something wrong you have to start all over.
• Skills and experience • How long have you used this product? • You seem to know quite a bit about this technology.
What did you do before starting here? • Study how the interviewee expresses things. Choice
of terminology, etc.
36
• Goals • What characterises a
good experience at work?
• When someone recognises that you’ve done a good job.
• Why is someone doing an action? • I take photos since she
grows up so fast. Now that she is two, I can hardly remember what she was like when she was one. • Track changes
• I want to take a photo that communicates how amazing it is when I see this really striking landscape. • Capture an
experience of something
• When I design a garden, I take photos of what it looks like for future reference for the design work. When the new design is done I take photos of the result for my portfolio to show people that I can do a good work. • Support work and sales
37
EthnographicObservations
UnderstandingUXRequiresQualitativeResearch
43
FreeImages.com/kukirosen
UXResearchInvolvesInterpretation 44
FreeImages.com/JuliaFreeman-Woolpert
TheHermeneuticalCircle45
Parts
Whole
FreeImages.com/francescofattore
CharacteristicsofQualitativeResearch
46
• No sharp boundaries of study objects • Careful contextualisation of the instance • Transferability rather than generalisability • Cooperation between researcher and participants • Purposeful sampling • Alternative interpretations • Suspicion of interpretations • Going beyond what is said and done to what is
intended and motivated
MeasuresforTrustworthynessinQualitativeResearch
47
• Prolonged engagement and observation • Building trust • Learning the culture • Checking misinformation
and distortions • Finding the focus
• Triangulation • Researcher • Theory • Method
• Peer review, debriefing and external audits
• Clarifying what subjective things that have likely shaped the the interpretation • past experiences, • biases • prejudices • orientations
• Member checking • Rich and thick description • Revise working hypothesis • Negative cases with
disconfirming evidence
DataAnalysisbyAffinityDiagramming
48
A special case of thematic analysis conducted in group
1.All workshop participants read the notes, mark important passages, and notes codes in the margin 2.Every person reads their markings aloud and write
them down on a sticky note (code notes on origin and person) 3.Put notes on wall or table and move them around so
that related notes are placed together 4.Discuss what binds the groups of notes together 5.Name the groups: you now have themes/categories
AQualitativeandInterpretativeStance
52
• What is surprising here? • What did we expect to see that did not appear? • What does the data tell us that we didn’t already
know? • What in the data makes us see things differently? • Why does a pattern or anomaly arise? • What theoretical concepts can we use to make sense
of our data?
DevelopUXThemes
53
• Form Statements about: • What do people do?
• How do they do it? What do their actions lead to? • What do people know?
• What is their frame of reference? What habits of mind does it lead to?
• What do people feel? • How are things experienced? What do their experiences mean
for them? • Order them according to their affinity to form themes • What is similar across participants? • What is different? Why?
54
By Bill Verplank.
Insummary
55
• There are differences between scientific research and research for design
• Ethics including informed consent is critical • Understanding UX requires qualitative research into
the interactions, mental models, and experiences people have
• Qualitative research methods includes both observations and interviews
• Analysis and interpretation into UX themes can be made with affinitiy diagrams
www.liu.se
@mattiasarvola