behavioural research bloopers & how to avoid them · behavioural research bloopers june 20th...
Post on 02-Jun-2020
2 Views
Preview:
TRANSCRIPT
Dr Jon Dodd
BEHAVIOURAL RESEARCH BLOOPERS
June 20th 2019
& HOW TO AVOID THEM
2
1999-2019 CEO Bunnyfoot
Hi - DR JON DODD
1990-1999 Neuroscience researcher (and Guinee pig)
POSITIVE, PROFITABLE & REWARDING
Since 1999 a leading specialist UX consultancy
WE HELP OUR CLIENTS CREATE
Experts in:
+ Providers of UX related services:
UX STRATEGY
SERVICE DESIGN
SERVICE INNOVATION
USER/CUSTOMER RESEARCH
USER CENTRED DESIGN
DESIGN THINKING
USABILITY TESTING
PERSUASION DESIGN
EYE TRACKING
ACCESSIBILITY
USER EXPERIENCES
Some of the clients we are currently
working with (May/June 2019)
A 40 strong team of psychologists, researchers, experience designers, usability & customer experience specialists
based out of London, Oxford, and Sheffield
▪ LEADING UX TRAINING PROVIDER
▪ LAB HIRE & LAB BUILD SERVICE
▪ FIELDWORK RECRUITMENT
4
LTM & FORGETTING
Learning requires rehearsal or conscious repetition
Hermann Ebbinghaus 1886
5
RESEARCH IS INHERENT TO GOOD DESIGN
e.g. in User Centred Design
6
IN USER CENTRED DESIGN
learn about your
Business
Business requirements/ stakeholder workshop
Participatory design/concept testing
Strategy and goal workshops
Requirement analysis planning
Business to customer uncovery
Desk research
learn about your
Competitors
Customer research/ benchmarking
Best practice analysis/feature analysis
learn about your
Customers
Behaviour observation
▪ Observed browsing
▪ Customer Inquiry
▪ Ethnography
▪ Competitor testing
Diary studies
Audience surveys
Depth interviews
User/Customer requirement workshop (focus groups)
define the
Information
architecture
Information architecture development
Content strategy
develop the
Interaction design
Paper prototyping
Iterative wireframe development
Clickable prototype development
document with
Blueprints /instructions
Information architecture blueprints
Interaction design documentation
▪ Wireframes
▪ Templates
▪ Elements
long term
Support ‘Rent-a-guru’ ongoing ‘’on
call’ immediate access to our experts
Workshop, training and seminars
Policy documentation
regular
Monitoring Advanced visitor analysis
Benchmark quantitative testing
Competitor monitoring/trend spotting
Social media monitoring
Satisfaction surveys and interviews
ongoing
Optimisation A/B testing
Multi-variate testing
Updated customer intelligence
Design Modification updates
Persuasion auditing
define
Customer Models
Persona development
Scenario and use case setting
explore
Customer interaction
Content audit and analysis
Ideation/concepting
Participatory design workshop
Mental modeling
Mental model/content model
develop
Testable concepts
Concept rationalisation
Information architecture concept generation
Storyboarding
Lo-fi prototype sketching
research
Creative concepts Creative research
exploration
Creative workshop
evolve and
Develop the creative Creative concept
generation
Creative application
▪ Templates
▪ Elements
develop
Design guides/assets Design guide production
Design asset production
▪ Layered psd, png, etc,
▪ Presentation layer code
RESEARCH MODEL ARCHITECT DESIGN OPTIMISE
Expert usability eval
User testing
Accessibility audit
Prototype testing
Eye tracking
Card sorting
Navigation testing
Look and feel testing
Remote testing
Analytics
…and more tailor made for you
TESTING AND EVALUATION
7
IN SERVICE DESIGN
Management expectations
Customer
expectationsKnowledge gap Do your research to reduce this
gap
Management expectations
Servicespecification
Try to set spec to meet customer
expectations
Policy gap
Service Specification
Servicedelivery
Attempt to deliver the spec
(resources etc.)
Delivery gap
Service delivery
Customer coms
Communication realistically and
truthfully
– help set expectations
Communication gap
Note: this is a model we use called the gap model to help frame service design
Customer
perception
Overall goal:
Reduce this gap!
Research is essential to reduce the knowledge gap this is the difference between what the org thinks its users/customers want and need – and what they actually need.
8
4D OR DOUBLE DIAMOND
9
DESIGN THINKING…
10
11
DESIGN SPRINTS
Define the problem & start the UX
Canvas
Learn from others &
develop ideas
Rapidly evolve ideas & produce
visualisation(s)
Build suitable prototype(s)
Test & determine next steps
Monday Tuesday Wednesday Thursday Friday
12
ANYONE RECOGNISE THIS?
13
THE BIG PICTURE
Kruger & Dunning 1999
…people who lack the knowledge or wisdom to perform well are often unaware of this fact.
We attribute this lack of awareness to a deficit in metacognitive skill. That is, the same incompetence that leads them to make wrong choices also deprives them of the savvy necessary to recognize competence, be it their own or anyone else’s.
14
PERHAPS THE POSTER CHILD OF THE MOMENT…
15
16
A NICE ONE
17
…EASIER TO REMEMBER AND MORE POINTED
18
AND HAS BEEN APPLIED TO MANY INDUSTRIES
19
IT’S ACTUALLY A BIT MORE COMPLICATED THAN THAT…
…which is a bit ironic…
20
BUT A USEFUL TOOL…
Essentially all models are wrong – but some are useful
George E P Box
21
ASK YOURSELF REGULARLY – WHERE ARE YOU?
UX and research etc.
I hope I am here – every
day a school day
I did stand-up comedy
I’m about to
do my
second ultra
marathon
22
This proves this
RESEARCH: SOME THINGS I HEAR REGULARLY
We have loads of
[survey] data
ask them this…
We need more
people (a bigger
sample)
I want [eyetracking, focus group,
interviews…]
You’re an expert –
why do we need
research
That wasn’t a good test…
(moderator didn’t say enough)
(a child kept running in)
But we know our
customers
Just ask them what
they think about
this
23
…AND IN DESIGN (PARTICULARLY INTERACTION & SERVICE DESIGN)
24
RESEARCH: WHAT WE WOULD LIKE TO SEE MORE OF
What’s the best
approach(s)…
These are our
research
objectives
How can we
improve…
This is our problem
Every day is a
school day
25
You are not your audience/user/customer *
A FUNDAMENTAL TRUTH
You do not…
See things like they do
Know what they know
Want what they want
Work how they work
This is critical information when designing a product or service
…and btw your bosses have even less clue than you do…
* delete as appropriate!
…that you will often have to remind yourself and your colleagues about
26
27
MORE FROM WHAT KIDS CAN TEACH US…
28
ANYONE KNOW THIS CHAP?
Don’t Make Me Think (Revisited), 2014
It’s only natural to assume that everyone uses the Web the same way we do, and - like everyone else – we tend to think that our own behaviour is much more orderly and sensible than it really is
Steve Krug, 2014
29
WORTH READING…. AND GIVING TO YOUR BOSS?
30
ROLF MOLICH’S COMMON USABILITY EVALUATION (CUE) SERIES
Look it up – it might scare you
The latest one was about skilful
moderation
Participants were some leading lights
from usability evaluation (e.g.
president of UXPA, writers of popular
usability and usability testing books…)
and others
A wide variety of ‘mistakes’ and less
than best practice practice observed
Every day a school day
31
JUNK IN – JUNK OUT
It is essential to choose the right methodology
….good reminder – or a story you can tell those you might need to
!
32
THE STORY OF THE HEALTHY HAMBURGER?
33
THE STORY OF THE HEALTHY HAMBURGER!
Q?So what would
be the correct
approach?
34
ANOTHER EXAMPLE
If I asked you what you looked
at – would you lie?
Be truthful!
35
YOUR CHALLENGE
Which is from 30 Men?
Which is from 30 women?
36
ITS JUST PAST MIDDAY...
37
FILL IN THE BLANKS
38
ULTIMATELY...
People don’t remember...
People lie (confabulate)...
People post rationalise...
People hate to appear stupid...
People are bad at introspection...
People will be kind to you and tell you what they think you want to
hear
39
and…most of our behaviour including ‘complex’ behaviour is non-conscious (sub-conscious)- So we need to design (and design our research) with this in mind
40
EXERCISE: WHICH METHODS GO WHERE…
Shout some out…
41
HERE IS ONE THEY PREPARED EARLIER…
42
LETS LEARN FROM SOME BLOOPERS & HORROR STORIESEvery day is a school day…
43
RESEARCHING THE WRONG PEOPLE…
Under-specification
Over-specification
Bias
What about the ‘unknown unknowns’?
44
RESEARCHING THE WRONG PEOPLE…
‘Professional’ research participants
Fakes
Dunning Kruger sufferers (usage, expertise)
The lovelorn
(the smelly)
45
RESEARCHING THE RIGHT PEOPLE…
Take recruitment very seriously
Test screeners and modify appropriately
Know the flex points
Go where they flock
46
FALLING IN LOVE WITH A METHOD
The tyranny of eyetracking
The ease of the survey deployment
The fame of the focus group
47
= BE PROMISCUOUS WITH METHODS
Start with the research objectives
Use a mix of methods
Be innovative
48
TALKING TOO MUCH
Asking too many questions
Interrupting flow
(leading Qs, closed Qs etc.)
49
= WATCH, LISTEN & LEARN
Think very carefully before speaking
Power of silence
Practice, checklists etc.
50
ROBOTIC ADHERENCE TO A SCRIPT
Soulless Q & A
No empathy
No Engagement
No discovery
51
= BE FLEXIBLE, BE INTERESTED
Be prepared (and confident) to explore
Be interested (and look like you are)
Have empathy
Respond appropriately
52
THE WRONG NUMBER
Often misinterpreted
Too many = wasted ROI = too much to
analyse (paralysis through analysis)
Scientific significance v trend v
confidence
53
THE RIGHT NUMBER…
Rules of thumb for qual
Calculators for quant
Repeats
54
AND LOTS MORE TO WATCH OUT FOR
Confirmation bias (selective nature for research particularly important)
Expectation (experimentors/observer) bias (along with confirmation bias dangerous for the truth)
Availability heuristic (take notes, independent observers, group analysis)
Courtesy bias (look out for people doing this – occurs a lot in focus groups, at the end of user testing)
Curse of knowledge (especially important in research – domain experts lack empathy for users)
Group think (focus groups especially)
Interoceptive bias (well fed etc. changes outlook of researcher/judge)
Mere exposure effect (familiarity breeds… in this case positivity – important in ratings etc. e.g. satisfaction)
False consensus effect (you are not your user… and neither is your husband, wife, mother etc…)
55
CONCLUSIONS
Assess where you are on the curve
Every day is a school day
Observe behaviour where possible
Use mix methods and be flexible
Avoid and learn from bloopers
56
FOLLOW US
LONDONT: 0207 608 1670
5th Floor,
54 St John's Square,
Farringdon, London,
EC1V 4JL
OXFORDT: 01235 838 514
Harwell Innovation Centre,
173 Curie Avenue
Harwell, Oxfordshire, OX11 0QG
SHEFFIELDT: 0114 478 2950
The Old Chapel
8 Mortimer Street
Sheffield, S1 4SF
THANK YOU!
Questions…?
top related