the researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in research
TRANSCRIPT
The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in research
UX Australia 2016, MelbourneRuth Ellison, Principal User Researcher, PwC’s Experience Centre
@RuthEllison from PwC’s Digital Services
@RuthEllison from PwC’s Digital Services
$7$3 $6.50
@RuthEllison from PwC’s Digital Services
$7$3
@RuthEllison from PwC’s Digital Services
$7$3 $6.50
@RuthEllison from PwC’s Digital Services
@RuthEllison from PwC’s Digital Services
The Decoy Effect is a Cognitive Bias
@RuthEllison from PwC’s Digital Services
What is a cognitive bias?Mental shortcut
@RuthEllison from PwC’s Digital Services
We are all human
@RuthEllison from PwC’s Digital Services Image source: http://www.businessinsider.com.au/what-new-coke-looked-like-in-1985-2013-3
@RuthEllison from PwC’s Digital Services
Applying it to design research
PreparingRunning
Analysing
@RuthEllison from PwC’s Digital Services
Preparing for research
@RuthEllison from PwC’s Digital Services
@RuthEllison from PwC’s Digital Services
Selection bias
@RuthEllison from PwC’s Digital Services 15
@RuthEllison from PwC’s Digital Services
Multiple channels - mix of recruitment companies, social media, trusted networksAvoid professional respondentsCareful screenersBehavioural based recruitment
Dealing with
it….
@RuthEllison from PwC’s Digital Services
Running research sessions
Photo available under a CC by 2.0 licence: https://www.flickr.com/photos/gdsteam/20649386153
@RuthEllison from PwC’s Digital Services
Observer-Expectancy effect
@RuthEllison from PwC’s Digital Services
The way we frame our questions matter
@RuthEllison from PwC’s Digital Services
Loftus, E. F., & Palmer, J. C. Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 1974, 13, 585-589.
McLeod, S. A. (2014). Loftus and Palmer. Retrieved from www.simplypsychology.org/loftus-palmer.html
About how fast were the cars going when they smashed into
each other?
@RuthEllison from PwC’s Digital Services
About how fast were the cars going when they collided into each other?
About how fast were the cars going when they bumped into each other?
About how fast were the cars going when they contacted into each other?
@RuthEllison from PwC’s Digital Services
Triangulate researchUse observational methodsKeeping positive-neutral body language, watch the tone of your voiceAvoid leading questions
Dealing with
it….
@RuthEllison from PwC’s Digital Services 23
I need a volunteer please
24
248
PwC’s Digital Services
What’s going on?
@RuthEllison from PwC’s Digital Services
Confirmation bias
Take the opposing view
Don’t just confirm your hypothesis, see if you can prove it wrong
Analysis of Competing Hypotheses (ACH)
Use open ended questions
Some people think that soft drinks are bad for you. What do you think?
What’s your opinion about soft drinks?
@RuthEllison from PwC’s Digital Services
List assumptionsBe skeptical, especially if everyone agrees with youRemain open Consider all evidence equallyMultiple user researchersLeave your ego by the door.
Dealing with
it….
@RuthEllison from PwC’s Digital Services
@RuthEllison from PwC’s Digital Services
Groupthink & the Bandwagon effect
@RuthEllison from PwC’s Digital Services 32
https://youtu.be/MDD4IkVZWTM?t=52s
@RuthEllison from PwC’s Digital Services
Triangulate with other research methods (e.g. observational)Use a mixture of individual and group exercisesAvoid stating preferences and expectations at startGive someone the devil’s advocate role to question assumptions
Dealing with
it….
@RuthEllison from PwC’s Digital Services
Anchoring bias
@RuthEllison from PwC’s Digital Services 35
https://www.flickr.com/photos/naotakem/17105160150/
@RuthEllison from PwC’s Digital Services
Consider the order of questions and designs carefullyOpen ended questions Alternate order in which participants are shown concept or design versions
Show version A first1 3 5
2 4 6
Show version B first
Dealing with
it….
@RuthEllison from PwC’s Digital Services
Analysing
@RuthEllison from PwC’s Digital Services 39
@RuthEllison from PwC’s Digital Services 40
Apophenia
Source: https://au.pinterest.com/pin/249738741807276910/
@RuthEllison from PwC’s Digital Services
Clustering illusion & reporting bias
@RuthEllison from PwC’s Digital Services
Breadcrumbs
@RuthEllison from PwC’s Digital Services
Sample sizes – it’s about the WHYConsider evidence equally – not just the ones that confirm your belief/assumptionCollaborative analysis sessionsStrive for objectivity
Dealing with
it….
@RuthEllison from PwC’s Digital Services 44
How vulnerable are you to these biases?
@RuthEllison from PwC’s Digital Services
What percentage of the population will be more biased than the average
person?
@RuthEllison from PwC’s Digital Services
Are you more biased than the average person?
@RuthEllison from PwC’s Digital Services
Blind spot bias
48@RuthEllison from PwC’s Digital Services
Listen with an open mind
Become more rational, but less rationalising
Continuous learning
Always assess your method, your analysis and yourself
for bias
@RuthEllison from PwC’s Digital Services 49
We are all human
@RuthEllison from PwC’s Digital ServicesImage source: https://en.wikipedia.org/wiki/Twenty_Questions#/media/File:20_questions_1954.JPG
Thank you:)
Say hello @RuthEllison
@RuthEllison from PwC’s Digital Services 52
Further reading
List of cognitive biasesYou are not so smart: a celebration of self delusion9 Biases In Usability Testing