Download - WPAS Are We Engaged Yet
People talk. We listen.
Are we engaged yet? A framework for measuring engagementA presentation to WPAACrystal City, VAMay 2009 Katie Delahaye PaineCEOKDPaine & Partners, LLCDurham & Berlin, NHkdpaine.blogs.comwww.measuresofsuccess.com
People talk. We listen.
Eyeball
counting
HITS
Engagement
MSM
Online
Social
Media
A measurement timeline
Page 2
People talk. We listen.
The Social Media Decision TreeDefined Goal
Sales:Memberships
Donations Applications
Traffic/Users
Influence
Web analyt
ics
LeadsLoyalty
Frequency
of visits
LeadsTime on site
Comments
LinksTime on site
Messaging
Visibility
Tonality
Loyalty/
renewal
SalesClickthru
People talk. We listen.
The Engagement Decision TreeAwareness
Consideration
Preference
Trial
Purchase
FindObserve/
LurkParticipate Engagemen
t
Purchase/Act/Link/
WOM
Page 4
People talk. We listen.
A Proposed Engagement Index
ClickthruDonations/
ordersSignups
Time on siteRepeat visits
Forwards/links
/comments
RelationshipsTone/content of
conversationMembership
One Number?
Output Outtake Outcome
+ +
Page 5
People talk. We listen.
Engagement Indices
Forrester: Involvement
Website visitsTime spentPage views
InteractionCommentsReviews
IntimacySentimentPositioning
InfluenceLikelihood to recommendBrand affinityForwardsLinks
Others: Stowe Boyd:
Posts/commentsNielsen – time on
siteQuantcast –
Frequency of return
Comscore – visits per visitor
Eric Petersen
Page 6
People talk. We listen.
What do you need to measure?
Outputs?Did you get the coverage you wanted?
Did you produce the promised materials on time and on budget?
Outtakes?Did your target audience see the messages?
Did they believe the messages?
Outcomes?Did audience behavior change?
Did the right people show up?
Did your relationship change?
Did sales increase?
7
People talk. We listen.
Goals, Actions and Metrics Goal Action Output
Metric Outtake Metric
Outcome Metric
Recruitment
Twitter campaign
Number of tweets
% inclined to support% improvement in perception of organization
% increase in online requests for information % increase traffic to recruitment site
Increase in support
Flickr photo contest
Number of entrants
% likely to support the cause
Improvement in relationship scores
Message consistency
Blogger/Speaker training
% postings/articles containing one or more key messages
Share of positioning on key issues
% hearing message% believing message
% consistency in messaging between external and internal communicationsMarket shareRatio of on-message to off-message quotes
8
People talk. We listen.
Metrics that show engagement with your own site
% increase or decrease in unique visits Change in page rank - i.e a list of the top ten most popular areas and how it has changed in the last week How many sessions on our blog or web site represent more than 5 page views In the past month, what % of all sessions represent more than 5 page views % of sessions that are greater than 5 minutes in duration % of visitors that come back for more than 5 sessions % of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand % of visitors that become a subscriber % of visitors that download something from the site % of visitors that provide an email address
Courtesy of Eric Peterson
Page 9
People talk. We listen.
Metrics that show engagement with other sites
Conversation IndexLinksForwardsDigg, StumbleuponFacebook, Twitter, Pownce, Utterz, SeesmicCommentsRatingsRankingTime between postsTone of discussion
Page 10
People talk. We listen.
Share of conversation vs share of engagement
Page 11
2
2
1
2
1
6
5
3
1
1
1
1
1
1
1
1
1
1
2
2
2
1
2
1
2
1
4
2
1
4
2
1
1
4
1
6
7
6
2
2
2
2
1
3
2
3
1
0 2 4 6 8 10 12 14 16 18 20
Faculty
Students
Research, Physical Sciences
Courses
Research, Earth Sciences
Projects, Non - Research
Financials
Alumni Topics
Research, Life Sciences
Staff
Admissions
Legal News
Other
Research, Agriculture
Policies
Institution, Overall
Campus Life
Research, Social Sciences
Share of Subject
Peer 1
Michigan State
Peer 2
Peer 3
Peer 4
15.3%
68.7%
100.0%
4.4%
33.3%
96.8%
28.6%
34.9%
12.5%
43.3%
28.6%
13.0%
38.3%
100.0%
23.6%
66.7%
6.3%
28.6%
20.8%
2.3%
95.6%
33.2%
5.8%
28.6%
100.0%
86.8%
13.0%
31.0%
22.1%
3.2%
71.4%
43.5%
18.8%
94.2%
56.7%
14.2%
13.2%
53.2%
28.4%
21.1%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Admissions
Alumni Topics
Campus Life
Community Relations
Courses
Events
Faculty
Financials
Institution, Overall
Inventions
Legal News
Other
Partnerships
Policies
Projects, Non - Research
Research, Agriculture
Research, Earth Sciences
Research, Life Sciences
Research, Other
Research, Physical Sciences
Research, Social Sciences
Staff
Students
Share of Engagement by Subject - ,External Blogs
Peer 1
Michigan State
Peer 2
Peer 3
Peer 4
People talk. We listen.
The vast majority of discussion in external blogs is neutral.
Page 12
23
29
12
14
20
5
8
4
1
4
0
5
10
15
20
25
30
University of Michigan Purdue University Penn State Michigan State Arizona State
Share of Tone
Negative
Neutral
Positive
71%
3%
29%
94%
83%
42%
58%
6%
14%
58%
42%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Arizona State Michigan State Penn State Purdue University University of Michigan
Share of Engagement by Tone - External Blogs
Negative
Neutral
Positive
People talk. We listen.
For all institutions, most postings were simply making an observation or distributing media.
Page 13
3
6
1
1
7
36
1
29
5
15
14
2
16
1
2
12
7
2
6
2
24
787
3
2
203
12
12
46
11
1
3
2
1
4
1
4
3
6
2
1
13
2
2
1
13
2
6
18
4
1
1
5
35
3
17
2
8
9
1
1
1
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Acknowledging receipt of information
Advertising Something
Answering a question
Asking a question
Augmenting a previous post
Calling for action
Disclosing personal information
Distributing media
Expressing criticism
Expressing support
Expressing surprise
Giving a heads-up
Giving a shout-out
Making a suggestion
Making an observation
Offering an opinion
Playing a game
Rallying support
Recruiting people
Showing dismay
Share of Conversation Types
Arizona State
Michigan State
Penn State
Purdue University
University of Michigan
44.2%
6.5%
30.9%
49.5%
100.0%
100.0%
100.0%
1.6%
53.9%
100.0%
26.9%
23.1%
10.8%
38.7%
72.7%
10.9%
15.5%
46.1%
66.6%
27.3%
35.1%
39.7%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Acknowledging receipt of information
Advertising Something
Answering a question
Asking a question
Augmenting a previous post
Calling for action
Disclosing personal information
Distributing media
Expressing criticism
Expressing support
Expressing surprise
Giving a heads-up
Giving a shout-out
Making a suggestion
Making an observation
Offering an opinion
Playing a game
Rallying support
Recruiting people
Showing dismay
Share of Engagement by Conversation Type - Institutional Blogs
Arizona State
Michigan State
Penn State
Purdue University
University of Michigan
cx
People talk. We listen.
Aspects of relationships
Control mutualityTrustSatisfactionCommitmentExchange relationshipCommunal relationship
14
People talk. We listen.
Components of a Relationship Index
Control mutuality In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)This organization really listens to what people like me have to say.
TrustThis organization can be relied on to keep its promises.This organization has the ability to accomplish what it says it will do.
SatisfactionGenerally speaking, I am pleased with the relationship this organization has established with people like me.Most people enjoy dealing with this organization.
CommitmentThere is a long-lasting bond between this organization and people like me.Compared to other organizations, I value my relationship with this organization more
Exchange relationshipEven though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.This organization will compromise with people like me when it knows that it will gain something.This organization takes care of people who are likely to reward the organization.
Communal relationshipThis organization is very concerned about the welfare of people like me.I I think that this organization succeeds by stepping on other people. (Reversed)
15
People talk. We listen.
How to implement relationship metrics
Step 1: Conduct a benchmark relationship studyStep 2: Implement PR programStep 3: Conduct a follow up relationship studyStep 4: Look at what’s changed
People talk. We listen.
Data mining the numbers you have
Look for failures firstCheck to see what the competition is doing Then look for exceptional successCompare to last month, last quarter, last yearFigure out what worked and what didn’t work
People talk. We listen.
Best Practices:
Engagement leads to bottom-line impact
ASPCA: Donations Memberships
War Child: 38% increase in
donations 300 requests to be
voluntersNWA:
65000 new wildlife spotted
Using SMM engagement for planning
Define the market/topic you want to focus onDefine the bloggers that influence the marketplace
Benchmarking against your peers
Looking at what the best doSet goals accordinglyUse data to persuade the recalcitrant
Social Media in CrisisListen instantly to a wide range of influencersIdentify weaknesses in communications, image or reputationCommunicate directly to people who need to hear you
People talk. We listen.
Correlation exists between traffic to the ASPCA web site and the organization’s
overall media exposure
-
100,000
200,000
300,000
400,000
500,000
600,000
700,000
0
50,000,000
100,000,000
150,000,000
200,000,000
250,000,000
300,000,000
350,000,000
Web
Sit
e Vi
sito
rs
Expo
sure
Overall Exposure
Web Traffic
People talk. We listen.
Tying activity to development/marketing goals
0
50,000,000
100,000,000
150,000,000
200,000,000
250,000,000
300,000,000
350,000,000
Exposure
$0
$200,000
$400,000
$600,000
$800,000
$1,000,000
$1,200,000
$1,400,000
$1,600,000
$1,800,000
Donations
Overall exposure
Online donations
20
People talk. We listen.
Case study USO
People talk. We listen.
Anatomy of a perfect measurement program
Start with message map Create measurement map Start measuring traditional media Establish relationship and awareness benchmark Start measuring social media Conduct social media program Measure relationships again
People talk. We listen. 23
42
15 7
45
12
67
6
88
17
5
30 33 45
34
116
33
22
12
43
9
7
12
-
20
40
60
80
100
120
140
Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
2008 2009
Opp
ortu
nitie
s to
See
in M
illio
nsKey Message Communication Over Time
Connect troops with home Delivers comforts of home Enable meaningful involvement
Good business practices Home away from home
People talk. We listen. 24
10 15 21 15 7
35 21
12 12
12 20
27 13 14
26 47
8
13 8
24 12
13
20
15 53
4
4
47 51
15
12
42 32
44
126
16 23
112
25
18
21
28 13
15
62 112
47
-
20
40
60
80
100
120
140
160
180
Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
2008 2009
Opp
ortu
nitie
s to
See
in M
illio
nsExposure Over Time by Subject
Centers Corporate Partners Development Donations
Historical Operations Organizational Programs
Thought Leadership Tours Volunteers
People talk. We listen.
Key Indicators Over Time
People talk. We listen. 26
Conversation Type January February March
Making an observation 143 152 6
Expressing support 20 40 42
Offering an opinion 18 1 0
Advertising something 10 0 0
Rallying support 4 4 1
Distributing media 4 2 0
Making a suggestion 2 3 0
Calling for action 1 2 0
Giving a shout-out 1 2 0
Asking a question 1 2 0
Disclosing personal information 1 1 0
Recruiting people 1 0 0
Putting out a wanted ad 0 1 0
Expressing criticism 0 1 1
Expressing agreement 0 1 0
Answering a question 1 0 0
Expressing surprise 0 1 0
Grand Total 207 213 50 0 10 20 30 40 50
Expressing criticism
Rallying support
Making an observation
Expressing support
1
37
Mentions
Conversation Type by ToneMarch 2009
Positive
Neutral
Negative
0 10 20 30 40 50
Expressing criticism
Rallying support
Making an observation
Expressing support
4
1
1
2
42
Mentions
Conversation Type by Message SaturationMarch 2009
Contains no message
Contains 1+ Messages
People talk. We listen.
Tonality of social media towards USO over time
27
63 6138
143 151
11
0
50
100
150
200
250
Jan Feb Mar
2009
Me
nti
on
s
Tonality Over Time
Positive Neutral Negative
People talk. We listen. 28
130 137
43
1215 12
42 55
7
0
50
100
150
200
250
Jan Feb Mar
2009
Men
tion
sMentions by Subject Over Time
Entertainment Fund raising Local USO Military Support Organizational
What people are talking about
People talk. We listen.
Social Media OverviewMarch 2009
29
131
47
4
76
166
46
0
50
100
150
200
250
Jan Feb Mar
2009
Men
tion
s
Key Message Saturation Over Time
Contains no message Contains one or more full messages
5
67
149
45
413
0
20
40
60
80
100
120
140
160
180
Jan Feb Mar
2009
Men
tion
s
Key Message Communication Over Time
Connect troops with home Delivers comforts of home Enable meaningful involvement
Good business practices Home away from home
People talk. We listen.
For more information on measurement, check out our blog: kdpaine.blogs.com orSubscribe for free to The Measurement Standard, www.themeasurementstandard.comTo start developing your own dashboard or for a copy of this presentation go to: http://www.kdpaine.comOr call me at 1-603-868-1550Or give us your business card and we'll be happy to send it to you
Thank you
Page 30