Social Media Measurement Techniques and Tips  October 14 , 2009Katie Delahaye PaineCEOkdpaine@kdpaine.comwww.kdpaine.comhttp:/kdpaine.blogs.comMember, IPR Measurement Commissionwww.instituteforpr.org

What Matters?

To P&G: EngagementTo the Humane Society:

DonationsTo ComCast: Happier

customers To Best Buy: Better informed

employeesTo WMUR: Faster, more

complete, more relevant stories To Dell: SalesTo Molson: Better messaging

What Doesn’t Matter?

AVEsEyeballsHITS (How Idiots Track Success)Couch Potatoes# of Twitter Followers (unless you’re a celebrity)# of Facebook Friends/Fans (unless they donate money)

Page 3

Page 4

Old School 21st Century

You are a party planner, not a communicator

The definition of timely has changedThe definition of reach has changed

GRPs & Impressions are impossible to count (an irrevelvant) in social media

The definition of success has changedThe answer isn’t how many you’ve reached, but how those you’ve reached have responded Page 5

Old School PR 21st Century Role of PR

Social Media renders everything you know about measurement obsolete

Some really scary numbers* if you’re an ad agency

More respondents spend more time on daily personal Intent usage than watching TV 53% of DVR owners watch at least 50% of content on replay – skipping ads all together Consumer Internet ad spend outpaces TV spend by 3X26% of US respondents have already contributed content to social networking sites. 32% said they follow recommendations from friends2 out of 3 ad execs expect ad revenue to shift from impression-based to impact based metrics within three years

*The End of Advertising as we Know it , IBM 2009

Signs that it’s the end of measurement as we know it 1. 11 Moms make a bigger difference than 11

million 2. 1 person on Twitter changed the reputation of

Comcast 3. 1 CEO’s blog is changing the face of

healthcare in Boston4. Facebook USERS translated the site from

English to Spanish via a Wiki in less than 4 weeks and cost Facebook $0

5. Dell has made more money on Twitter than Twitter has

6. Ex-employee networks are helping companies lower costs and speed response times

7. The Epping, NH police department is on Twitter

The New Rules of Communications

You aren’t in control and never have beenThere is no market for your message You become what you measureShe/he with the most data winsBehind every Tweet or Post is a person Empower employees, rely on customersEnable the conversations—it’s going on, with or without youSpin is dead, long live transparency – you can’t fake it so be who you are and see who is pleased Crowdsourcing will beat outsourcing every time

The Engagement Decision Tree

Awareness

Consideration

Preference Trial Purchas

e

FindObserv

e/Lurk

Participate

Engagement

Purchase/Act/Link/WOM

Goals for Social Media

1.Marketing/leads/sales/2.Mission/safety/civic

engagement3.Relationship/reputation/

positioning To fix this Or get to this

Goals drive metrics, metrics drive results

11

Reputation/Relationships

Relationship scores

Recommendations

Positioning

Engagement

Get the word out

% hearing

% believing

% acting

Sales

Engagement Index

Cost per customer

acquisition

Web analytics

Sales leads

Marketing Mix Modeling

Goal

Metrics

Change the conversation, improve your reputation

Improve your reputation

Listen first, then respondStop doing stupid things

Negative coverage over time

Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr2006 2007 2008

0

5

10

15

20

25

21

2

5

21 1

4 42

1 12 2 2

1

10

9

4

1418

21

12

10

15

14

7

26

2

10

4

12

2

3 1

1

1

4

2

2

1

2

5

3

2

2

2

Entr

ies

Goals, Actions and Metrics Goal Action Output Metric Outtake

MetricOutcome Metric

Increased on-line reservations

Revamp website

Amount of content on web site

% perceiving state as a destination

% increase in web traffic and reservations

#1site for visitors to NH

Increase staffing and resources for communications

Increased exposure of “visit NH” message

Increased perception of NH as an an extreme destination

% increase in agreement with the statement

Website is preferred site for information

Add content, features to web site, keep up to date

% increase in traffic

% agreeing with the statement

# 1 rankings, and time spent on site

The 7 steps to Social Media ROI

1. Define the “R” – Define the expected results?

2. Define the “I” -- What’s the investment?

3. Understand your audiences and what motivates them

4. Define the metrics (what you want to become)

5. Determine what you are benchmarking against

6. Pick a tool and undertake research7. Analyze results and glean insight,

take action, measure again

Step 1: Define the “R”

What return is expected? – Define in terms of the business or mission.

What were you hired to do? What difference are you expected to make?

If you are celebrating complete 100% success a year from now, what is different about the organization?

If your department was eliminated, what would be different?

16

Step 2: Define the “I”

What is the investment? PersonnelAgency compensationSenior Staff time Opportunity costRaw costs/hr costs vs material costs.

17

Step 3: Define your audiences and how you impact them

There is no “audience.” There are multiple constituencies Should you blog or Twitter? Don’t ask me, ask your customers List every stakeholder

Where do they go for information?What’s important to them?What is the benefit of having a good relationship with that stakeholder group?What’s important to them?Where do they go for information?What do you want them to know?

Understand your role in getting the audience to do what you want it to do

Raise awarenessIncrease preferenceIncrease engagement

18

Step 4: Define your Key Performance Indicators (KPIs)

19

The Perfect KPIGets you where you want to go (achieves corporate goals)Is actionableContinuously improves your processesIs there when you need it

KPIs should be developed for: Your own propertiesDifferent tacticsOther influential sites

Step 4: Define your Key Performance Indicators (KPIs) carefully because you become what you measure

Cost savingsEfficiency

Cost per message communicatedCost per new lead/customer acquired

Productivity: Increase in employee engagement/moraleLower turnover/recruitment costs

Engagement: Ratio of posts to comments% of repeat visitors% of 5+min visitors% of registrations

Trust:Improvement in relationship /reputation scores with customers and communities (Loyalty/Retention)

Thought leadership: Share of quotesShare of opportunities

Message penetrationPositioning on key issuesImprovement in favorable/unfavorable ratioImprovement in Optimal Content Score (OCS)

20

KPIs for External blogs and other Consumer Generated Media

Share of positioningShare of rants vs. ravesShare of positives/negativesShare of visibilityShare of quotesShare of brand benefits

mentionedTypes of conversationsEngagement – ratio of posts to

comments Optimal content score

Revenue KPIs

Cost savingsCost per click thru, downloads, engagement vs other marketing channelsCost per message communicated vs other channels

Lifetime value of engagementCost per customer acquisition

Engagement metrics

% increase or decrease in unique visits In the past  month,  what % of all sessions represent more than 5 page views % of sessions that are greater than 5 minutes in duration % of visitors that come back for more than 5 sessions % of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand % of visitors that become a subscriber % of visitors that download something from the site % of visitors that provide an email addressRatio of posts to comments

Courtesy of Eric Peterson

For all institutions, most postings were simply making an observation or distributing media.

Page 24

3

6

1

1

7

36

1

29

5

15

14

2

16

1

2

12

7

2

6

2

24

787

3

2

203

12

12

46

11

1

3

2

1

4

1

4

3

6

2

1

13

2

2

1

13

2

6

18

4

1

1

5

35

3

17

2

8

9

1

1

1

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Acknowledging receipt of information

Advertising Something

Answering a question

Asking a question

Augmenting a previous post

Calling for action

Disclosing personal information

Distributing media

Expressing criticism

Expressing support

Expressing surprise

Giving a heads-up

Giving a shout-out

Making a suggestion

Making an observation

Offering an opinion

Playing a game

Rallying support

Recruiting people

Showing dismay

Share of Conversation Types

Arizona State

Michigan State

Penn State

Purdue University

University of Michigan

44.2%

6.5%

30.9%

49.5%

100.0%

100.0%

100.0%

1.6%

53.9%

100.0%

26.9%

23.1%

10.8%

38.7%

72.7%

10.9%

15.5%

46.1%

66.6%

27.3%

35.1%

39.7%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Acknowledging receipt of information

Advertising Something

Answering a question

Asking a question

Augmenting a previous post

Calling for action

Disclosing personal information

Distributing media

Expressing criticism

Expressing support

Expressing surprise

Giving a heads-up

Giving a shout-out

Making a suggestion

Making an observation

Offering an opinion

Playing a game

Rallying support

Recruiting people

Showing dismay

Share of Engagement by Conversation Type - Institutional Blogs

Arizona State

Michigan State

Penn State

Purdue University

University of Michigan

cx

Share of conversation vs share of engagement

Page 25

2

2

1

2

1

6

5

3

1

1

1

1

1

1

1

1

1

1

2

2

2

1

2

1

2

1

4

2

1

4

2

1

1

4

1

6

7

6

2

2

2

2

1

3

2

3

1

0 2 4 6 8 10 12 14 16 18 20

Faculty

Students

Research, Physical Sciences

Courses

Research, Earth Sciences

Projects, Non -Research

Financials

Alumni Topics

Research, Life Sciences

Staff

Admissions

Legal News

Other

Research, Agriculture

Policies

Institution, Overall

Campus Life

Research, Social Sciences

Share of Subject

Peer 1

Michigan State

Peer 2

Peer 3

Peer 4

15.3%

68.7%

100.0%

4.4%

33.3%

96.8%

28.6%

34.9%

12.5%

43.3%

28.6%

13.0%

38.3%

100.0%

23.6%

66.7%

6.3%

28.6%

20.8%

2.3%

95.6%

33.2%

5.8%

28.6%

100.0%

86.8%

13.0%

31.0%

22.1%

3.2%

71.4%

43.5%

18.8%

94.2%

56.7%

14.2%

13.2%

53.2%

28.4%

21.1%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Admissions

Alumni Topics

Campus Life

Community Relations

Courses

Events

Faculty

Financials

Institution, Overall

Inventions

Legal News

Other

Partnerships

Policies

Projects, Non - Research

Research, Agriculture

Research, Earth Sciences

Research, Life Sciences

Research, Other

Research, Physical Sciences

Research, Social Sciences

Staff

Students

Share of Engagement by Subject - ,External Blogs

Peer 1

Michigan State

Peer 2

Peer 3

Peer 4

The vast majority of discussion in external blogs is neutral.

Page 26

23

29

12

14

20

5

8

4

1

4

0

5

10

15

20

25

30

University of Michigan Purdue University Penn State Michigan State Arizona State

Share of Tone

Negative

Neutral

Positive

71%

3%

29%

94%

83%

42%

58%

6%

14%

58%

42%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Arizona State Michigan State Penn State Purdue University University of Michigan

Share of Engagement by Tone - External Blogs

Negative

Neutral

Positive

Emerging benchmarks Past PerformanceThink 3

PeerUnderdog nipping at your heelsStretch goal

Whatever keeps the C-suite up at night

Step 5: Define your benchmarks

27

28

Non-Profit industry benchmarks in social media % of Desirable Coverage 96%

% of Key Message Communication

37%

Number of Messages tracked 11.5

Words per Key Message 14

Number of Key Messages tracked

11.5

Most frequent conversation types

Express support (69%) Making an observation (28%)

% rallying support 3%

% asking a question 4%

% of exclusive mentions 17%

% mentioning brand in title 8%

% of discussions in Blogs 51%

% of discussion on Twitter 30%

% of visibility from YouTube & Flickr

10%

Overview of Key Metrics

Bookmark.

Facebook

Ext. Blogs

Inst. Blogs YouTube MSM

SOV 2% — 8% 9% 11% 7%

Popularity

230 bkmks

500/mo. — 20 links

150k views —

Engagement 59 cmts 1 day 13 cmts

2-12 cmts 2 cmts —

% Positive 20% 32% 54% 50% 15% 15%

% Negative 0% 0% 4% 0% 1% 2%

Strat. Mess. 40%† 18%† 42% 42%† 18% 38%

Peer 1 was the competitive leader in all but YouTube, where Peer 4 and Peer 3 led.Actions attributed to individuals were responsible for most content, except on YouTube.

Rank Order

Facebook YouTube Social Bookmarking

External Blogs

Institutional Blogs

1 Campus Life

Events Courses Faculty Campus Life

2 Sports Campus Life

Projects, Non-Research

Research, Physical Sciences

Events

3 Technology Faculty Research, Physical Sciences

Institution Overall

Institution Overall

4 Product Services

Courses Events Expert Commentary

Institution Sub-Groups

5 Events Institution Overall

Faculty Events Admissions

Few subjects appear across all forms of social media, so tailor outreach accordingly

Benchmarks put numbers in perspective

Twitter grade

Followers

Twitter rank

98.3

1295

43395

96.4

622

93718

99.99

15291

398

Shel Israel U of Penn Carnegie Mellon

Page 31

Step 6: Pick a tool

1. Content Analysis2. Survey3. Web Analytics

Step 6: Selecting a measurement tool

Objective KPI Tool

Increase inquiries, web traffic, recruitment

% increase in traffic#s of clickthrus or downloads

Google Analytics, Omniture, Web trends

Increase awareness/preference

% of audience preferring your brand to the competition

SurveyMonkey, Zoomerang

Engage marketplace Conversation index greater than .8Rankings % increase in engagement

TypePad, Technorati Omniture, Google Analytics

Communicate messages

% of articles containing key messagesTotal opportunities to see key messagesCost per opportunity to see key messages

Media content analysis –Dashboards

% aware of or believing in key message

Survey

33

Content Analysis requires:

A content source: Google News/Google Blogs, RSS feedsTechnorati, Social Mention, Twazzup, Cyberalert, CustomScoop, e-WatchRadian 6, Techrigy, Sysymos, Visible Technologies, Scout Labs

34

A way to analyze content

1. Automated vs. Manual

2. Census vs random sample

3. The 80/20 rule – Measure what matters because 20% of the content influences 80% of the decisions

4. Dashboards to aggregate data

35

A coding methodology

TonalityWhat messages were communicatedHow you’re positioned on key issuesDominance/Prominence/VisibilitySubject of the article/postingWho was quoted?Products, events, initiatives, battles mentioned

Standard classifications of discussion

• Acknowledging receipt of information

• Advertising something• Answering a question• Asking a question• Augmenting a previous

post• Calling for action• Disclosing personal

information• Distributing media• Expressing agreement• Expressing criticism• Expressing support• Expressing surprise• Giving a heads up

• Responding to criticism• Giving a shout-out• Making a joke• Making a suggestion• Making an observation• Offering a greeting• Offering an opinion• Putting out a wanted ad• Rallying support• Recruiting people• Showing dismay• Soliciting comments• Soliciting help• Starting a poll• Validating a position

Standard classifications of videos

AdvertisementAnimationDemonstrationEvent/PerformanceFictionFilmHome VideoInstructional VideoInterviewLecture

MontageMusic VideoNews BroadcastPromotional VideoSightseeing/TourSlideshowSpeechTelevision ShowVideo Log

Why an Optimal Content Score?

You decide what’s important:Benchmark against peers and/or competitorsTrack activities against OCS over time Positive:

Mentions of the brandKey messagesPositioningVisibility

Negative OmittedNegative toneNo key message

39

How to calculate Optimal ContentQuality score +1 0 -1

Score Score ScoreTonality Positive 3 Neutral 0 Negative -3

Positioning Contains 2 Doesn't contain 0

Positions the competition favorably or positions Sargento negatively -2

Messaging Contains 3 partially contains 0

Does not contain or miscommunicates key message (neg mess) -1

Quotes Contains 1 Does not contain -1Competitive mention

Does not mention Competition 1

Competition mentioned prominently -3

Total Score 10 0 -10

Visibility Score+1 0 -1

Score Score Score

Brand Photo Contains 3 Doesn't contain 0Contains competitive photo -5

Dominance Focal point 3 Not a focal point -1Visibility Headline mention 2 Top -20 % of story 0 Minor mention -2Target publication Top Tier 2 2nd tier 0 Not on target list -2

Total Score 10 0 -10

Optimal Content Score

Building Measures

Units of contentOverall theme (when many messages are combined)Entire message (e.g. article, blog post, etc.)Message parts (brand mentions, paragraphs, sentences)

Types of contentManifest: on the surfaceLatent: the meaning or interpretation of the contentLatent Pattern: meaning determined by surface observationsLatent Projective: meaning determined by coder interpretation

Building Measures (cont.)

Variables are exhaustive/ mutually exclusive

One can always be selected, and only one

Types of variablesNominal: categories, “buckets”

Brands mentioned, organizations mentioned, messages communicated

Ordinal: categories with an order or scale

Tonality, prominence, dominance

Numbers: number of words (zero means no words), number of brand mentions

Surveys require:

A defined sampleA list – a way to get to that sampleAgreement on what questions you need to answerA survey instrument/questionnaire A testA way to analyze data SPSS SAS

43

Aspects of relationships

Control mutualityTrustSatisfactionCommitmentExchange relationshipCommunal relationship

44

Control Mutuality

The degree to which parties agree on who has the rightful power to influence one another. Although some imbalance is natural, stable relationships require that organizations and publics each have some control over the other.

45

Questions that test Control Mutuality

This organization and people like me are attentive to what each other says.This organization believes the opinions

of people like me are legitimate.In dealing with people like me, this

organization has a tendency to throw its weight around. (Reversed)This organization really listens to what

people like me have to say.The management of this organization

gives people like me enough say in the decision-making process.

46

Measuring Trust

One party’s level of confidence in and willingness to open oneself to the other party. Includes:

Integrity: the belief that an organization is fair and justDependability: the belief that an organization will do what it says it will doCompetence: the belief that an organization has the ability to do what it says it will do.

47

Questions to measure trust

This organization treats people like me fairly and justly.Whenever this organization makes

an important decision, I know it will be concerned about people like me.This organization can be relied

upon to keep its promises.I believe that this organization

takes the opinions of people like me into account when making decisions.I feel very confident about this

organization’s skills.This organization has the ability to

accomplish what it says it will do.

48

Measuring satisfaction

The extent to which each party feels favorably toward the other because positive expectations about the relationship are reinforced. A satisfying relationship is one in which the benefits outweigh the costs.

49

Questions to Measure Satisfaction

I am happy with this organization.Both the organization and people like me

benefit from the relationship.Most people like me are happy in their

interactions with this organization.Generally speaking, I am pleased with the

relationship this organization has established with people like me.Most people enjoy dealing with this

organization.50

Measuring commitment

The extent to which each party believes and feels that the relationship is worth spending energy to maintain and promote.

51

Commitment

I feel that this organization is trying to maintain a long-term commitment to people like me.I can see that this organization wants

to maintain a relationship with people like me.There is a long-lasting bond between

this organization and people like me.Compared to other organizations, I

value my relationship with this organization more.I would rather work together with this

organization than not.52

Measuring relationships

Exchange Relationship In an exchange relationship, one party gives benefits to the other only because the other has provided benefits in the past or is expected to do so in the future.

Communal Relationship In a communal relationship, both parties provide benefits to the other because they are concerned for the welfare of the other -- even when they get nothing in return.

53

Exchange Relationships

Whenever this organization gives or offers something to people like me, it generally expects something in return.Even though people like me have had a

relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.This organization will compromise with

people like me when it knows that it will gain something.This organization takes care of people who

are likely to reward the organization. 54

Communal Relationships

This organization does not especially enjoy giving others aid. (Reversed)This organization is very concerned about

the welfare of people like me.I feel that this organization takes advantage

of people who are vulnerable. (Reversed)I think that this organization succeeds by

stepping on other people. (Reversed)This organization helps people like me

without expecting anything in return.

55

How to implement relationship metrics

Step 1: Conduct a benchmark relationship studyStep 2: Implement PR programStep 3: Conduct a follow up relationship studyStep 4: Look at what’s changed

Web Analytics Require:

Google Analytics/Web Trends/OmnitureUnique URLsData delivered in parallel with content analysisAbility to correlate and integrate data SPSS/SAS

57

Look for failures firstCheck to see what the competition is doing Then look for exceptional successCompare to last month, last quarter, 13-month averageFigure out what worked and what didn’t workMove resources from what isn’t working to what is

Step 7: Analysis - -Research without insight is just trivia

58

Ask for money Get Commitment Manage Timing Influence decisions Get Outside help Just Say No

Actionable Conclusions

59

Overall Comparison of Georgia Tech Social Media Outlets

60/17

0%

5%

10%

15%

20%

25%Share of Youtube

Share of Social Bookmarking

Share of FacebookShare of Institutional

Blogs

Share of External Blogs

Georgia Tech

Competitor Average

• Based on 2007 data, Georgia Tech outperformed its peers in Facebook presence, but significantly lagged peers on other social media.

• Post-2007 media monitoring has not included a social media dimension due to funding constraints, but this will be important to trend as feasible in the future.

Definitions: YouTube: a video sharing site. Social Bookmarking: a site where members can display media they have found on the web. Facebook: a social networking site. Institutional Blogs: blogs hosted and owned by schools studied. External Blog: any blog post that is not hosted by an institution.

Share of All Coverage

Best Practices:

Correlations to bottom-line impact

DonationsMembershipsSign-upsLeads

Using SMM for planning

Define the time frame, market/topic you want to studyUse Google News, Technorati or Radian6 to identify the conversations around the topic Analyze the conversations for type, tone and positioningLook at share of positioning, tone or conversation

Benchmarking against your peers

Looking at what the best doSetting goals accordinglyUse data to persuade recalcitrant spokespeople

Social Media in CrisisListen instantly to a wide range of influencersIdentify weaknesses in communications, customer service, or in the product

Improve your reputation

Listen first, then respondStop doing stupid things

Using SMM for planning

The environmental scanDefining issues in a marketSelecting a positioning that works

Benchmarks put numbers in perspective

Twitter grade

Followers

Twitter rank

98.3

1295

43395

96.4

622

93718

99.99

15291

398

Shel Israel U of Penn Carnegie Mellon

Page 63

Diversity dominates C-M discussions in Social Media

Page 64

Where people get the content they share on Facebook

Sources of content

Genre of content

Understanding brand ownership of online video content

N=2,555,691

Peer Organizations

4.33%

Your Organization0.18% Other

Organizations8.65%

Individual Users86.84%

Use ownership to signal brand participation

Provide alerts for possible brand management issues

8 ways to do research without a budget

1. Become someone’s research project

2. Involve your board of directors and volunteers

3. Research something that HAS a budget

4. Take advantage of free offers5. Become a case study6. Team up with peer organizations7. Analyze data that already exists8. Use blogs and social networks to

listen to conversations

67

Thank You!

For more information on measurement, read my blog: http://kdpaine.blogs.com or subscribe to The Measurement Standard:

www.themeasurementstandard.comFor a copy of this presentation

go to: http://www.kdpaine.comFollow me on Twitter: KDPaineFriend me on Facebook: Katie

Paine Or call me at 1-603-868-1550