11 ways humans kill good analysis (kevin ertell)

#monetatesummit

11 Ways Humans Kill

Good AnalysisKevinErtell, Sur LaTable

FOCUSED

FOCUSEDACTIONABLE

FOCUSEDACTIONABLEMANAGEABLE

FOCUSEDACTIONABLEMANAGEABLEENLIGHTENING

FOCUSED

ACTIONABLE

MANAGEABLE

ENLIGHTENING

FOCUSED

ACTIONABLE

MANAGEABLE

ENLIGHTENING

11 Ways Humans Kill Good Analysis

1. We hire reporters not analysts

Logical

Sequential

Rational

Objective

2. We turn analysts into reporters

Why is conversion down on Google

paid search?

Why is conversion down on Google

paid search?What’s'the'op+mal'marke+ng'mix'to'use'to'launch'Brand'X?

Why is conversion down on Google

paid search?What’s'the'op+mal'marke+ng'mix'to'use'to'launch'Brand'X?

Why'are'return'rates'growing?

3. We expect the data to be perfect and the analysis to be flawless

A man with one watch knows what time it is; a man with two watches is never quite sure.

4. We fail to define objectives and state our assumptions

5. We want numbers for number’s sake

KPIs

Suppor+ng'Metrics

Forensic'Metrics

Supporting metrics

KPIsSupporting metrics

6. We insist on simplicity

How likely are you to recommend this business?

• Large margins of error

• Low precision

• Low detection of movement • Interpretation problems

• Not very predictive

How likely are you to recommend this business?

All'we'did'was'quan+fy'this'common'sense'in'a'way'that'made'sense'to'business'leaders—the'target'audience'for'my'book.'These'prac+cal'leaders'have'liJle'interest'in'advanced'sta+s+cal'methods.'

All'we'did'was'quan+fy'this'common'sense'in'a'way'that'made'sense'to'business'leaders—the'target'audience'for'my'book.'These'prac+cal'leaders'have'liJle'interest'in'advanced'sta+s+cal'methods.'

These practical leaders havelittle interest in advancedstatistical methods

correlations

7. We just want the number

“Plans based on average assumptions are wrong on average.” -Sam Savage

8. We aren’t multilingual in the languages of business & statistics

Standard deviations

Standard deviations

Variances

9. We expect answers immediately

You’re approaching a Coast Guard security zone. … If you don’t stop your vessel, you will be fired upon. Stop your vessel immediately.

You’re approaching a Coast Guard security zone. … If you don’t stop your vessel, you will be fired upon. Stop your vessel immediately.

bang,'bang,'bang,'bangBang! Bang! Bang! Bang!

regression to the mean

10. We ignore our guts

Prefrontal cortex

11. We blow the presentation

Questions?Kevin'Ertell'

@kevinertell

Kevin.Ertell@SurLaTable.com