Lean LaunchPad Workshop:Defining an Analytics Strategy

Ryan Jung

Haas MBA 2014

ryan_jung@haas.berkeley.edu

Why Are Analytics Important?

• Failure to define an analytics strategy can be a fatal error for a startup in 2015.

• Analytics has changed the landscape

• A great analytics strategy is tightly integrated with the overall business strategy

Why You Need an Analytics Strategy

• Learn faster by creating feedback loops

• More clarity based on behavior

• Consensus on future action

There exists a host of tools to help you with these objectives.

History of Analytics

• 1990s – Web counters

• 2000s – Click Analytics and SEO

• 2010s – Behavioral and Predictive Analytics

Keys to a Great Analytics Strategy

1. Tightly integrated with overall business strategy

2. Iterative process

3. Measurable set of hypotheses, results, and revisions

The Modern Data-Driven Lean Startup

Goal is to optimize a set of business objectives in a logical

progression leveraging quantitative and qualitative facts in order to delight customers in a scalable,

repeatable fashion

Most Important Reports

• Segmentation (Cohorting)

• Retention

• Funnels

• Revenue Tracking

• Marketing Campaign Effectiveness

• Path Analysis

• Notifications

Segmentation / Cohorting

What segments are getting what value out of your product?

Value Proposition / Customer Segment

Who is our customer?

What problem are we reallysolving for them?

Will they buy from us?

How do we reach them?

• Build customer archetypes• Add properties to define the user• Use segmentation to look at differences in customers• Good for looking at actions, but need to understand causation to be actionable

Using Analytics

Segmentation Example

• Look at aggregated events and then segment by properties

• See who is doing particular actions and identify trends

• Want to segment as far as possible

• Point you to needs and how your product adds value

Google Analytics

Retention

Who gets the most value out of your solution?

How Churn affects LTV

Lifetime Value

Monthly Churn

Source: David Skok Matrix Partners

Thinking Through Retention

Get –> Keep –> Grow = Activation –> Retention –> Engagement

Understanding key features

Understanding core users and testing their needs

Identifying most effective channels

Retention ReportsIn-session retention In-app retention

Key Question(s) Where do users spend their time in your app? What features are valuable?

Are users coming back and using the app repeatedly? Who are users that are more likely to come back?

Value Proposition Features that are most valuable Users that get most value out of product

Tool Addiction Recurring or Segmented Retention

Mixpanel

BIG IDEA:LTV drives CAC which drives channel

selection

Increasing Sales Complexity

Log(

Acq

uis

itio

nC

ost

)

CAC < LTV

Funnels

How are users interacting with your solution?

Sales Funnels

Where are we losing customers?

How do we know if we are doing well or not well in sales?

How can we do better?

Core Idea: Track conversion rates between levels of funnel to see where “leakage” occurs and create strategies to minimize this loss.

Is my marketing spend being used efficiently?

Funnel Reports

Localytics

Funnel Reports

KISSMetrics

Tying funnels to revenues

Revenue = installs x [signups / installs] x [purchases / signups ] x [revenue / purchase]

Back-end tells you this

Analytics tells you this

Analytics can tell you this

You control this

The main point here is that you can break revenue into measureable components• Tie how you earn revenue to what you measure• Then understand where you are doing well and not well• Then use your analytics solution to design tests to figure out how to drive

more lifetime value

Mathematically:

Pitfalls to AvoidProblem Explanation

Search vs. Execution Metrics

Are we measuring KPIs or are we testing hypotheses?

Vanity metrics If it only goes “up and to the right” and / or if it’s not actionable, it’s a waste of time to measure it.

Biased tests Be sure that the hypotheses that you are testing are not set up to confirm your assumptions. Take the approach of trying to disprove your hypothesis.

Data overload “Measuring everything and then mining for insights” creates too much noise for most to get any real value from.

Summary

• You need to be thinking about analytics because your competition probably already is

• Analytics is evolving, so keeping up is imperative

• Analytics needs to be tied to your overall business strategy, should be hypothesis-driven, and is an iterative process

Case Studies

Airbnb

• Challenge: Initially wanted to optimize booking flow

• Allowed them to identify to distinct classes of users

• Can better target users and their needs

More info: https://mixpanel.com/case-study/airbnb/

Khan Academy

• Challenge: increase engagement and the rate at which people learn

• Used funnels to optimize search and registration processes

• Start with a definition for “user engagement”

More info: https://mixpanel.com/case-study/khanacademy/

Jawbone

• Challenge: Assess the viability of Jawbone UP

• Used Segmentation reporting to better understand their users

• Helps to build customer archetypes

• Faster iterations and faster time to product-market fit

More info: https://mixpanel.com/case-study/jawbone/

Cohort analysis

Renewal and upsell rates

Return on marketing investment

Revenue by Cohort – Each Year Builds on a Stronger Base

Note: Excludes inorganic growth.

201120102009200820072006

Highly Loyal Customers

2007 Cohort

Earlier Cohorts

Revenue by Cohort – Each Year Builds on a Stronger Base

2006

2008 Cohort

2009 Cohort

2010 Cohort

2011 Cohort

20112010200920082007

Highly Loyal Customers

Note: Excludes inorganic growth.

2007 Cohort

Earlier Cohorts