measuring for engagement
TRANSCRIPT
Measuring For Engagement
1
John Egan Lead Growth Engineer @ Pinterest
2
New Users
MAUs
Dormant Users
3
New Users
MAUs
Dormant Users
4
New Users
Dormant Users
Core
Casual
Marginal
5
New Users
MAUs
Dormant Users
6
• Retail shopping rewards app • Led Growth Engineering team • Grew from 1MM to 8MM users in 3 years
• Acquired for $200MM
• Visual discovery & bookmarking • Eng lead for R&R team • Increased mobile WAUs by 20% • Increased mobile signups by 10%
Activation
7
Retained MAUs• Percentage of new signups that are still using the app a month later • Figure out what a good retention number is for your industry
8
1d7s• Percentage of new signups that use the app 1 or more times in the week
following signup • Quicker to run experiments with • At Pinterest this is highly correlated with user’s long-term retention
9
Cohort Heatmap
10Source: Placeholder Numbers
Cohort Heatmap
11Source: Placeholder Numbers
Cohort Heatmap
12Source: Placeholder Numbers
Engagement
13
One Key Engagement MetricMedium: Total Time Spent Reading (TTR) Twitter: MAUs with 7+ visits a month (7d28s) Uber: Weekly Trips Facebook: WAUs with 6+ visits a week (6L7s) Pinterest: Weekly Active Repiners or Clickers (WARCs)
14
Finding Your Key Engagement Metric1) What are the actions a user has to take to get value from your product?
2) What is the frequency someone need to use your product
15
User States
Core: Active multiple times a week Casual: Active once or twice a week Marginal: Active a couple times a month New: Joined in the past 28 days Dormant: Not active for 28 days Resurrected: Was dormant, but became active again in the past 28 days
16
Badging Holdout Experiment
2:50 PM 100%
17
Measuring Effect on Engagement
18
*AU Ratios
19
20
DAU MAU DAU/MAU ratioKnighthood 30,014 389,583 7.70%
RockYou Pets 50,482 582,720 8.66%Tetris Friends 100,404 841,223 11.94%
Superpoke Pets 299,505 2,296,175 13.04%Mobsters 539,012 3,431,063 15.71%
Vampire Wars 687,067 3,794,869 18.11%Country Story 1,647,763 8,097,356 20.35%
Fish World 2,084,358 8,581,587 24.29%Pet Society 5,046,347 20,480,397 24.64%Mafia Wars 6,473,504 25,431,622 25.45%
Restaurant City 5,033,468 17,250,419 29.18%Cafe World 7,409,711 22,062,364 33.59%
Courtesy Social Times & Nabeel Hyatt
Retention & Resurrection
21
Measuring Engagement for Emails/Push
• Positive measures of quality - Must lift key engagement metric - Minimum required CTR rates
• Negative measures of quality - App deletions - Spam reports - Unsubscribes
22
Experiment Segmentation
23
All Users
24
Marginal Users (~1 month)
25
Core Users (multiple times a week)
26
Wrap Up• 1d7s to experiment on activation
• Cohort heatmaps to track new user retention curve
• Define your key engagement metric
• Use user states to understand how engaged users are
• Make sure emails/notifications for true engagement
• Use segmentation for experiments & metrics in general
Confidential 27
John EganGrowth Blog: jwegan.com
Email: [email protected]
Twitter: @jwegan_com
Pinterest: pinterest.com/jwegan
28