tips to radically improve your surveys
TRANSCRIPT
Tips to radically improve your
Surveys @stephbeadell
Stephanie BeadellVP of Marketing
[email protected]@StephBeadell
? + =
@stephbeadell
Surveys are Powerful Tools
…unless the data is bad.
@stephbeadell
Radically Improve Your Data
3@stephbeadell
steps
Typical Survey Question
@stephbeadell
Do you feel like part of our community?
Do you feel like part of our community?
@stephbeadell
Yes
No
I Don't Know
1@stephbeadell
Use a 5 or 7-point
Scale@stephbeadell
A Better Question
@stephbeadell
I fit in with the community.
Strongly
AgreeStrongly
Disagree
Neutral
I Fit in with the Community
@stephbeadell
0
20
40
60
80
100
120
140
160
180
200
Strongly Disagree
Disagree Neutral Agree Strongly Agree
IFitinwiththeCommunity
Average: 3.67
“I Fit in with the Community” Score Over Time
@stephbeadell
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
IFitinwiththeCommunity
“Likert Scale”
@stephbeadell
Likert Scale Basics
@stephbeadell
Strongly
Agree
NeutralStrongly
Disagree
5 or 7 points
Likert Scale Basics
@stephbeadell
BraveShy
Unipolar
(Don’t Do This)
Analysis Opportunities
@stephbeadell
Likert Scale Question
@stephbeadell
I fit in with the community.
Strongly
AgreeStrongly
Disagree
Neutral
Scoring the Answers
@stephbeadell
I fit in with the community.
1 2 3 4 5
Correlation Analysis & More
@stephbeadell
“Heavy forum users are more likely to
be satisfied with their membership.”
Sat
isfa
ctio
nFrequency of Forum Visits
Forum Usage vs. Satisfaction
Use a 5 or 7-point
Scale
When in Doubt:
@stephbeadell
2@stephbeadell
@stephbeadell
into multiple questions
Break down
Big Concepts
Typical Survey Question
@stephbeadell
Are you happy with the community?
… it could mean
@stephbeadell
Do you like the people in the community?
… it could mean
@stephbeadell
Does the community meet your needs?
… it could mean
@stephbeadell
Does being a member make you feel good?
@stephbeadell
?
?? ?
Try this instead
@stephbeadell
1. Being a member is valuable to my career.
2. I enjoy being a member.
3. The company cares about the community.
Report it Like This
@stephbeadell
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Valuable to Career
Enjoy Being Part
Company Cares
Member Happiness Survey
Average Happiness
Score:
11 out of 15
Track it Over Time
@stephbeadell
“Our team increased member happiness by 3 points this quarter.”
0
2
4
6
8
10
12
14
16
Month 1 Month 2 Month 3
Happiness
@stephbeadell
Break down
Big Concepts
3@stephbeadell
Typical Survey Results
@stephbeadell
020406080100120140160180200
Strongly Disagree
Disagree Neutral Agree Strongly Agree
IFitinwiththeCommunity2.
Freelancer
Small Business
Enterprise
BusinessSize1.
Compare and
Contrast@stephbeadell
Comparing Answers
@stephbeadell
EnterpriseSmall Business
01020304050607080
IFitinwiththeCommunity
01020304050607080
IFitinwiththeCommunity
Typical Survey Results
@stephbeadell
020406080100120140160180200
Strongly Disagree
Disagree Neutral Agree Strongly Agree
IFitinwiththeCommunity2.
Freelancer
Small Business
Enterprise
BusinessSize1.
Comparing Answers
@stephbeadell
EnterpriseSmall Business
01020304050607080
IFitinwiththeCommunity
01020304050607080
IFitinwiththeCommunity
How to Do It in Excel: Pivot Tables
@stephbeadell
Export Raw Data
And in SurveyMonkey: Filter/Compare
@stephbeadell
Filter or Compare
@stephbeadell
Plan segments in advance
Comparing Answers
@stephbeadell
Ask identifying questions
3@stephbeadell
steps
Use a 5 or 7-point
Scale
When in Doubt:
@stephbeadell
@stephbeadell
Break down
Big Concepts
Compare and
Contrast@stephbeadell
How Not to Screw it Up
@stephbeadell
BiasIntroducing
@stephbeadell
Priming
Bargh, Chen, and Burrows (1996)
@stephbeadell
Sentence Scramble Task
Rudeaggressively, bold, rude, bother, disturb, intrude, annoyingly, interrupt,
audaciously, brazen, impolitely, infringe, obnoxious, aggravating, bluntly
Politehonor, considerate, appreciate, patiently, cordially, yield, polite, cautiously,
courteous, graciously, sensitively; discreetly, behaved, unobtrusively
Neutralexercising, flawlessly; occasionally rapidly, gleefully, practiced,
optimistically, successfully, normally, send, watches, encourages. gives,
clears, prepares
Bargh, Chen, and Burrows (1996)
@stephbeadell
People who Interrupted After the Task
Polite Neutral Rude
Introducing Bias: Priming
@stephbeadell
When are people prompted to take the survey?
@stephbeadell
Order Bias
Order Bias
@stephbeadell
How important is content marketing?
Order Bias
@stephbeadell
How important is content marketing?
How much content marketing do you plan to do this year?
Solving Order Bias with Randomization
@stephbeadell
Question Randomization
Solving Order Bias with Randomization
@stephbeadell
Shuffle Question Order
@stephbeadell
Stereotype Threat
Stone, Lynch, Sjomeling, and Darley (1999)
@stephbeadell
“natural athletic ability”
“sports intelligence”
Inzlicht (2000)
@stephbeadell
Female Math Test Performance When Others Present
2Women 2Men
Solving for Stereotype Threat
@stephbeadell
Demographic data always
goes at the end of a survey
Thanks!