taming the stream, tweetreach at internet summit

Internet Summit 11Taming the Stream | #tamethestream | @hayesdavis

Taming the StreamIntegrating and Standardizing Social Analytics


About me

Hayes DavisCEO of TweetReachLover of bacon, analytics@hayesdavis


What’s on the agenda?

• About measurement

• Generally accepted metrics

• Evolving metrics

• Black box metrics

• Putting it all together


About Measurement


Accuracy vs Precision

• AccuracyMeasurements reflect the True State of the World at the time the measurement was taken

• PrecisionThe same measurements under the same circumstances are consistent


Accuracy in social media

• In most computer systems larger than a single machine, There is no Truth

• CAP Theorem says: Choose two of Consistency, Availability and Partition Tolerance

• For social networks, consistency gets the axe


Data collection still matters

• Data collection usually done via APIs

• Some APIs give better data than others:

• Twitter Search excludes certain users and is limited to 1500 tweets over 7 days

• Twitter’s Streaming API provides full coverage in real time if connected 24x7

• Know how your vendor collects data for each channel


How to be precise

• Collect data the same way every time

• The channels often change the rules, be aware of these changes

• If you’re relying on vendors for measures, understand any algorithm changes the make

• Measure the same thing repeatedly and watch it trend over time. If there are wild swings, investigate all parts of the “data chain”


On to the metrics...


A simple framework

AccountCentric

TopicCentric

Owned MediaYour ContentYour Network

Earned MediaAll Content

Whole Network


Generally Agreed Metrics


Accepted counting metrics

• “Graph” metrics like Fans/Followers/Friends

• Interactions such as Likes/Retweets/Replies/Comments

• Mention volume

• Clickshttp://muppet.wikia.com/wiki/Count_von_Count


Some accepted KPIs

• Share of Voice(Your Mentions) / (Industry Mentions)

• Velocity(Volume in a Time Period) / (Length of Time Period)

• Unique Contributors|Participants|

• Conversion Rate(Completed Actions) / (Audience Size)


Evolving Metrics


“You keep using that word. I do not think it means what you think it means.”

- Inigo Montoya


Same name, different meaning


Some evolving metrics

• Reach: Size of the potential audience. Sometimes applied to accounts, sometimes to overall conversation. May contain heuristics.

• Amplification: How much is the reach of a message or user being increased by sharing?

• Impact: What effect is a user having on a conversation?


Black Box Metrics


Data in, answer out

?

Step 1. Get data

Step 2. ???

Step3. Profit


Black boxes are valuable...

• Specialized providers

• “Big picture” algorithms

• Immense data collection

• Simplified communication

• Increased efficiency


... If everyone believes

$1.00 is worth a soda because we believe it’s worth a soda.

If people lose faith in a black box, it becomes worthless.


Putting it all together

• Aim for precision with as much accuracy as possible

• Use commonly agreed definitions where possible

• For evolving metrics, pick a definition and stick with it

• Use black box metrics sparingly

• Research the algorithm

• Supplement your analysis, don’t replace


Questions?

Hayes DavisCEO of TweetReach@[email protected]

taming the stream, tweetreach at internet summit

Technology