Analytics frameworks and startup stages

Farley Millano - farleymillano@gmail.com

1

Agenda [1/2]Recap

Analytics frameworks

Startup stages

Empathy

Stickiness

Virality

Revenue

Scale

2

Agenda [2/2]

Start the development of proposed work

Analysis of a commercial game regarding:

Balancing + Monitoring

http://www.di.ubi.pt/~palmeida/Balanceamento_Jogos_15_16/Balanceamento_Jogos_15_16.htm

3

RecapFundamentals: Lean Startups (Strategic) + Agile (Operational)

Business Model: Intro + Canvas

Intro to Digital Product Metrics

Types (Quali vs. Quanti, Vanity vs. Actionable, etc.)

Analysis techniques (A/B Testing, Segmentation, etc.)

4

Analytics frameworks

5

IntroThere’s a myriad of factors that define which metrics to watch, but we mostly it relies on:

Business model type

Startup stage

Some authors defines a set of categories to organize this task

Frameworks

6

Analytics frameworksHelp understand startups functioning in terms of:

Lifecycle

Grow

Find market

Costumer acquisition

Revenue streams

7

Pirate Metrics - AARRR

Created by Dave McClure, startup investor and founder of business accelerator 500 startups

Its name comes from a acronym for Acquisition, Activation, Retention, Revenue and Referral (AARRR)

A user generates value not only from revenue, but also from referral and content creation

8

Pirate Metrics - AARRR

Lean Analytics (Croll & Yoskovitz, 2013)9

Dave McClure - Pirate Metrics

http://developers.magmic.com/metrics-track-mobile-game/10

Engines of growthCreated by Eric Ries, author of Lean Startup

Startup growth comes from the works of 3 engines:

Sticky

Virality

Paid

Each is associated with specific KPIs (key process indicator)

11

Engines of growthSticky engine: focused in user engagement

Returning active users

Key metrics: costumer retention, churn rates and usage frequency

Secondary metrics: time since last visit, click-through rate (email, notifications)

Examples: Game notifications, email reminder, Ads, etc.

12

Engines of growthVirality engine: focused in user virality

Users bringing other users

Key metrics: viral coefficient

Secondary metrics: Viral cycle time, number of connected accounts, number of invitations, number of answered invitation

Examples: Facebook Connect, invite users to speed progression (Farm Ville), etc.

13

Engines of growthPaid engine: focused in user monetization

Key metrics: Costumer lifetime value / costumer acquisition cost

Caution: focusing on receiving money before sustainability (stickiness and virality) is dangerous

Examples: number of subscriptions, inventory availability, etc.

14

Lean Canvas

15

Lean CanvasLean Canvas was created by Ash Maurya (detailed last class)

It can be interpreted as a current snapshot of your game / product

Hypothesis and Results

Its lightness should be used to provide dynamism to strategic decisions

16

Lean Canvas

Video: https://vimeo.com/39687297 Lean Analytics (Croll & Yoskovitz, 2013)17

Startup stages

18

Startup stagesIt is usually hard to have a specific "hard number” metric telling which stage a startup is at or should be do next

Lean Analytics authors created a five stage gate model to help that out

Empathy; Stickiness; Virality; Revenue and Scale

Draws some aspects of models presented previously and puts emphasis on metrics for transitioning

19

20

EmpathyIt is the time to get into your user’s head

Discover and validate a problem/idea/need

Qualitative data is key in this stage

Observe patterns that emerge

Methodology is extremely important

Tools, methods => Collect, analyse and report data

Avoid bias and unwanted effects

21

EmpathyDecouple problem from solution:

Painful enough?

Enough people care?

Are they trying to solve it themselves?

What it takes to make them aware of the problem?

22

EmpathyPrepare for interviews

Face to face

Neutral location

Have a script (Ash Maurya’s Running Lean)

Set the stage: put the interviewee in the right frame of mind

Segment: collect demographics

Problem context: tell the story

Test the problem: rank problems and others that might come up with

Test the solution: how do they solve it today

Ask for something: schedule a solution interview and refer other people

23

Empathy

Choose an approach

Convergent: some specific problems to be ranked

Divergent: diving into a more broad problem space

24

Empathy

How do I know if I have found the right problem?

The metric here is Pain (or Fun)

Ideally, results from your interview should be scored

25

Empathy

Did the interviewee ranked the problems presented?26

Empathy

Is the interviewee actively trying to solve the problems (or already did so)?27

Empathy

Was the interviewee engaged and focused throughout the interview?28

Empathy

Did she agree on a follow with you (possibly with a solution)?29

Empathy

Did she refer other people for you to interview?30

Empathy

Did she offer to pay you for the solution?31

EmpathyAt TechStars, LikeBright founders were asked to talk to 100 women to understand better their “problem"

Frustrations on dating

Report

Used Mechanical Turk from Amazon and Google Voice

$2 per interview and got 100 responses in 4 hours

This gave a better understanding and insights of their problem and had the company accepted

32

Empathy - SummaryGoal: identify a need that people want to pay at scale.

Conduct qualitative, exploratory discussions early on

Finding the right questions, scale to more quantitative surveys, reach more people

Use existing tools to collect data and simulate your solution

Balsamiq, Google Docs, Twitter, Linkedin, etc.

33

Empathy - Moving to next stage

34

StickinessOnce the problem is known and realistic, start build something

Iterate your MVP: methodical work

Improve core metrics

Avoid premature scaling

The goal at this stage is retention.

35

StickinessPrioritise features according to the following:

Develop estimates on why things will be better with that feature

Measure the impacts (metrics), avoid scope creep

Estimate time to develop vs. expected results

Usability: avoid complexity

Anticipate risks: technical and user response

Level of innovation: it is allowed to bake big bets on this stage

User voice: listen (carefully), take deeper look to their actions.

36

StickinessPrioritise features according to the following:

Develop estimates on why things will be better with that feature

Measure the impacts (metrics), avoid scope creep

Estimate time to develop vs. expected results

Usability: avoid complexity

Anticipate risks: technical and user response

Level of innovation: it is allowed to bake big bets on this stage

User voice: listen (carefully), take deeper look to their actions.Problem-solution canvas: weekly reality-check tool

37

Stickiness

Problem-solution canvas: weekly reality-check tool38

StickinessSimple tool for conducting focused surveys for a small group

First iteration of the product required that respondents first signed-up and then answered the questions

Problem: low response rate: 10-25%

Test: What if the user is already considered signed up if they answer a question?

They prioritised this feature (based on email) instead of building a mobile version

Stickiness

10-25% 70-90%

Stickiness - Moving to next stage

Are people using the product as expected?

What is an active user?

What is current percentage? Can this be higher?

Evaluate product roadmap against presented criteria

Evaluate user complaints

41

ViralityStands for the phase that users shares your product with other users

There are three types

Inherent: function of use

GDocs: share documents

Artificial: stimulated through reward

Dropbox: Space Race

Word-of-mouth: natural, happens from users exchange of experience

Blog post, Influentials, Spontaneous media, etc.

42

Virality

Most important metric: Viral coefficient

It is obtained by combining (multiplying):

Invitation rate: number of invites sent / number of users

Acceptance rate: number of signups / number of invites

43

Virality

Second most important metric: Viral cycle time

Time taken for a user to invite another one

It can make a huge difference when combined with viral coefficient

44

ViralityLeading indicator

Social engagement (links to friends)

Content creation (post, share, likes)

Return frequency (days since last visit, time on site, etc.)

Tied to part of a business model (revenue, daily traffic, etc.)

Come early on user lifecycle: increase data points

Early extrapolation = Sooner prediction

45

Virality

Circle of Moms

"Being a mom" was a leading indicator of engagement.

Facebook / Linkedin

Friends suggestion / invitation early in user lifecycle leads to higher spreadability

46

ViralityTimehop is a startup focused on recalling/reposting previous posts from social networks.

They already had an extremely engaged audience: 40-50% emails open rate

It was needed to go viral

Through pixel tracking on emails: 50% of emails were on iOS

As email is no inherently sharable, led them to develop an iOS version of their app

Virality - Moving to next stage

Which types of virality are you employing?

What is your viral coefficient? Is it enough to sustain business growth?

What is viral cycle time? Can it be speed up?

48

RevenueThis stage is focused in making money

After showing you are solving a real problem and the solution is sticky and viral

Charging up-front ≠ Focusing on revenue and margins

Most startups spend sometime until being self-sustainable

49

RevenueRevenue per costumer is a better metric than raw revenue

Clearer perspective and actionable

Other important metrics:

Ad revenue

Conversion rate

Shopping cart size

Subscription

Costumer lifetime value

Costumer acquisition cost

50

RevenueGoal: “more stuff to more people for more money more often more efficiently”

Focus on your strength:

Selling physical, per-transaction cost: focus on more efficiently

High viral: more people for every dollar

Loyal costumer: buy more often

One-time big ticket: more money for each purchase

Recurring: more stuff through higher-capacity packages

51

RevenueCostumer lifetime value > Costumer acquisition cost

Oversimplified math: delay between acquiring and paying costumers

Balance between:

Investment

Amount spent on costumer acquisition each month

Revenue brought from each user per month

Churn rate

52

RevenuePivoting

Changing part of a product and its business model in order to fit a new market.

If business model is not working: avoid the urge to build more features

It might be easier to change a market (target users) than a product

But it is difficult anyway53

Revenueparse.ly is focused on giving some tools for publishers in order to track their metrics

Their first product was a reader for end user

An extremely engaging product and earned really good media attention

All the metrics were fine, except one, revenue.

"They love, but they don’t pay.”

They built an entirely new product, using part of the knowledge and architecture from the past

This new offering uses a trial mode (one month)

Revenue - Moving to next stage

Breakeven on variable costs:

Costumer revenue > costumer acquisition + delivering service

Time to costumer breakeven

Investment made by the company until a user pays itself

EBITDA Breakeven

Ignores large investments and previous debts

Hibernation Breakeven

No new marketing is spent, only minimum

Lights on, servicing existing costumers, word of mouth, no new features

55

Scale

Represents a stage for wider audience, new market entrance

A product/service can stand on its own

High order metrics come into play

Channels, regions and marketing campaigns

56

ScaleRepresents a stage for wider audience, new market entrance

A product/service can stand on its own

High order metrics come into play

Channels, regions and marketing campaigns

Compensation, API traffic, channel relationship and competitors

Efficiency vs. Differentiation: reduce cost vs. increase margins

57

Scale

Strategy, tactics and implementation must be aligned regarding their goals

It might be harder to innovate at this stage

“Big company” feelings

58