Analytics Toolbox

What can Analytics do for me?

Retention Engineering

● How can we better understand Retention?○ Survival Analysis

■ Well understood branch of analysis■ Used heavily in social and life sciences

○ Survival vs Returns■ Returns - return frequency per cohort■ Survival - active users per cohort

○ Hazard Ratios■ Relative risk of an event■ Helps us decide which events are correlated with

survival and which are correlated with churn■ Used in biomedical field to understand drug risks

Retention Engineering

Survival Curves - better resolution for cohort differences

Retention Engineering

● How can this be applied to games?○ Hazard Ratio by Level○ Hazard Ratio for New User Flow

■ Identify drop off steps for dynamic user funnels■ Find key steps in users first session

○ Monetization Funnel■ Pivot our point of origin from the days since install to

the days since first purchase■ Active users are defined by their last purchase date

○ Interpret AB test results■ Understand impact of AB test changes by observing

differences in Hazard Ratios○ Expected User Lifetime

■ Build expectations using user level, days since install, activity within game, etc.

Retention Engineering

Key Points: First Gate Level 7, Second Gate Level 25

User Surveys

● Why do people play our games? Let's ask...○ Orthogonal Data Source

■ Independent method to characterize users■ Even more powerful when used with behavior data

○ Net Promoter Score (NPS)■ How likely are you to recommend Product X to a

friend or colleague? - Response [0-10]■ Simple way to track customer satisfaction

○ Gamer Profiles■ Break users into groups based on their motives■ Bartle Test :Killer, Socializer, Explorer, Achiever

○ Addiction versus Engagement■ Are our Whales addicted or highly engaged?

User SurveysCalculated Net Promoter Scores

User Surveys

● Advancement○ Acquiring rare items that most players will never have.

● Escapism○ How often do you play to relax from the day's work?

● Addiction vs Engagement (Yes or No)○ I often feel that I spend more money than I can afford on

Game X.○ I would hate to go without playing Game X for more than

a few days.○ I often fail to get enough sleep because of playing Game

X.○ I feel happy at the thought of playing Game X.

● Role-Playing○ How often do you make up stories and histories for

your characters?

User Surveys

We interpret results using Principal Components Analysis. Breaks the survey results into simple, interpretable pieces.

User SurveysMy Bartle Profile

Motivations of Play in MMORPGs

User SurveysAsheron's Call Survey Results

Segmentation

● Why segment users? The Flaw of Averages.○ Purchase Behavior

■ Whales, Sharks, Minnows, Guppies○ Retention

■ First Day Drops versus Sticky Users○ Engagement

■ Number of Clicks as a proxy for Activity Level○ Game Behavior

■ Affinities to particular Game Components○ Facebook Profiles

■ Break up users based on keywords from profiles○ Prizm Codes

■ Marketing Segments defined by Nielsen

Segmentation Example Prizm Codes

Segmentation

● We segmented them ... so what?○ Valuable way to interpret AB test results

■ Not all groups react the same to changes■ An average would leave that uncovered

○ Logistic Regression■ Identify factors that are predictive for each group■ Odds ratio

■ Relative measure of group probability ○ Clustering (similar to PCA)

■ Algorithmically detect patterns among segments○ Bayes Networks�

■ Network of IF-THEN/ELSE conditions■ Understand connections between segments

Segmentation

Discover what variables best separate our user segments

Economy Balancing

● How do detect economy imbalances?○ Sources & Sinks

■ Basic Accounting track debits and credits■ Segment by relevant time measures

○ Fluid Dynamics■ Analogy: Water flowing in & out of a bucket■ Track the velocity of flow

Economy Balancing Fluid Mechanics Analogy● Segment by progression

Economy Balancing

Segment by Level, Days Since Install, Minutes of Gameplay, ...

Economy Balancing Tune specific sources & sinks

Everything Else

● Troubleshooting (Game Change Log)● Taxonomies

○ Qualitative way to compare intra & inter game○ Item Sales Classification○ Viral Message Classification

● Path Analysis○ State Space Models

■ Each user has a unique sequence of events, DNA■ Compress the event sequence to know probabilities■ Know what the probability of a buy event at a given

point in time given a user's DNA

Path Analysis

Uses well understood mathematics to reduce user event sequences to event probabilities, very actionable data.

El Fin.

Analytics is insight.