how to read your members’ minds - stephanie beadell

@stephbeadell

(with surveys)

How to

Get into Your Members’

Heads

Stephanie BeadellVP of Marketing

[email protected]

@StephBeadell

? + =

@stephbeadell

Surveys are Powerful Tools

…unless the data is bad.

@stephbeadell

Radically Improve Your Data

3@stephbeadell

steps

Typical Survey Question

@stephbeadell

Do you feel like part of our community?

Do you feel like part of our community?

@stephbeadell

Yes

No

I Don't Know

1@stephbeadell

Use a 5 or 7-point

Scale@stephbeadell

A Better Question

@stephbeadell

I fit in with the community.

Strongly

Agree

Strongly

Disagree

Neutral

I Fit in with the Community

@stephbeadell

0

20

40

60

80

100

120

140

160

180

200

Strongly Disagree

Disagree Neutral Agree Strongly Agree


Average: 3.67

“I Fit in with the Community” Score Over Time

@stephbeadell

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5


“Likert Scale”

@stephbeadell

Likert Scale Basics

@stephbeadell

Strongly

Agree

NeutralStrongly

Disagree

5 or 7 points

Likert Scale Basics

@stephbeadell

BraveShy

Unipolar

(Don’t Do This)

Analysis Opportunities

@stephbeadell

Likert Scale Question

@stephbeadell


Strongly

Agree

Strongly

Disagree

Neutral

Scoring the Answers

@stephbeadell


1 2 3 4 5

Correlation Analysis & More

@stephbeadell

“Heavy forum users are more likely to

be satisfied with their membership.”

Sa

tisf

act

ion

Frequency of Forum Visits

Forum Usage vs. Satisfaction

Use a 5 or 7-point

Scale

When in Doubt:

@stephbeadell

2@stephbeadell

@stephbeadell

into multiple questions

Break down

Big Concepts

Typical Survey Question

@stephbeadell

Are you happy with the community?

… it could mean

@stephbeadell

Do you like the people in the community?

… it could mean

@stephbeadell

Does the community meet your needs?

… it could mean

@stephbeadell

Does being a member make you feel good?

@stephbeadell

?

?? ?

Try this instead

@stephbeadell

1. Being a member is valuable to my career.

2. I enjoy being a member.

3. The company cares about the community.

Report it Like This

@stephbeadell

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Valuable to Career

Enjoy Being Part

Company Cares

Member Happiness Survey

Average Happiness

Score:

11 out of 15

Track it Over Time

@stephbeadell

“Our team increased member happiness by 3 points this quarter.”

0

2

4

6

8

10

12

14

16

Month 1 Month 2 Month 3

Happiness

@stephbeadell

Break down

Big Concepts

3@stephbeadell

Typical Survey Results

@stephbeadell

020406080100120140160180200

Strongly Disagree


I Fit in with the Community2.

Freelancer

Small Business

Enterprise

Business Size1.

Compare and

Contrast@stephbeadell

Comparing Answers

@stephbeadell

EnterpriseSmall Business

01020304050607080


01020304050607080



@stephbeadell

020406080100120140160180200

Strongly Disagree



Freelancer

Small Business

Enterprise

Business Size1.

Comparing Answers

@stephbeadell


01020304050607080


01020304050607080


How to Do It in Excel: Pivot Tables

@stephbeadell

Export Raw Data

And in SurveyMonkey: Filter/Compare

@stephbeadell

Filter or Compare

@stephbeadell

Plan segments in advance

Comparing Answers

@stephbeadell

Ask identifying questions

3@stephbeadell

steps

Use a 5 or 7-point

Scale

When in Doubt:

@stephbeadell

@stephbeadell

Break down

Big Concepts

Compare and

Contrast@stephbeadell

How Not to Screw it Up

@stephbeadell

BiasIntroducing

@stephbeadell

Priming

Bargh, Chen, and Burrows (1996)

@stephbeadell

Sentence Scramble Task

Rudeaggressively, bold, rude, bother, disturb, intrude, annoyingly, interrupt,

audaciously, brazen, impolitely, infringe, obnoxious, aggravating, bluntly

Politehonor, considerate, appreciate, patiently, cordially, yield, polite, cautiously,

courteous, graciously, sensitively; discreetly, behaved, unobtrusively

Neutralexercising, flawlessly; occasionally rapidly, gleefully, practiced,

optimistically, successfully, normally, send, watches, encourages. gives,

clears, prepares

Bargh, Chen, and Burrows (1996)

@stephbeadell

People who Interrupted After the Task

Polite Neutral Rude

Introducing Bias: Priming

@stephbeadell

When are people prompted to take the survey?

@stephbeadell

Order Bias

Order Bias

@stephbeadell

How important is content marketing?

Order Bias

@stephbeadell

How important is content marketing?

How much content marketing do you plan to do this year?

Solving Order Bias with Randomization

@stephbeadell

Question Randomization

Solving Order Bias with Randomization

@stephbeadell

Shuffle Question Order

@stephbeadell

Stereotype Threat

Stone, Lynch, Sjomeling, and Darley (1999)

@stephbeadell

“natural athletic ability”

“sports intelligence”

Inzlicht (2000)

@stephbeadell

Female Math Test Performance When Others Present

2 Women 2 Men

Solving for Stereotype Threat

@stephbeadell

Demographic data always

goes at the end of a survey

@stephbeadell

Double-Barreled Questions

Double-Barreled Question

@stephbeadell

Do you follow us on

Facebook and Twitter?

@stephbeadell

Leading Questions

Leading Question

@stephbeadell

Do you believe you deserve a

higher salary?

A Better Way to Ask

@stephbeadell

Are you satisfied with your

current salary?

Why People Lie

@stephbeadell

SensitiveInformation

How to Handle Sensitive Information

@stephbeadell

Put tough questions toward

the end of your survey

@stephbeadell

OverloadInformation

How to Prevent Information Overload

@stephbeadell

Limit the amount of questions

in your survey

How to Prevent Information Overload

@stephbeadell

Avoid overwhelming tables

@stephbeadell

Boredom

How to Prevent Boredom

@stephbeadell

Set Time Expectations

“This survey will take about 5 minutes”

How to Prevent Boredom

@stephbeadell

Use a Progress Bar

Powerful Data Analysis

AnalysisCorrelation

@stephbeadell

Correlation Analysis

@stephbeadell

“Heavy forum users are more likely to

be satisfied with their membership.”

Sa

tisf

act

ion

Frequency of Forum Visits

Forum Usage vs. Satisfaction

Correlation Quick Tips

@stephbeadell

Use the CORREL() Function in Excel

Scores range from -1.0 to 1.0

Correlation Quick Tips

@stephbeadell

at least a 0.3 to show correlation

AnalysisAdvanced

@stephbeadell

Regression Analysis

@stephbeadell

Outcome

Variable 1

Variable 2

Variable 3

Cluster Analysis

@stephbeadell

K-Means Cluster Analysis

Excel Analysis Toolpak Add-In

@stephbeadell

bit.ly/excelanalysistoolpak

AnalysisCross Tabs

@stephbeadell


@stephbeadell

020406080100120140160180200

Strongly Disagree



Freelancer

Small Business

Enterprise

Business Size1.

Comparing Answers

@stephbeadell


01020304050607080


01020304050607080


Thanks!