Building an Insight MachineMatthew GranadeStrata DDBD 2015

September 29, 2015

Big Data, Big Spending

50%

10%

• Market growing at 10%1

• Big Data portion growing at 50%1

• $60 billion1 on information management software

• Shortfall of 150,000 data scientists2

1 http://www.ovum.com/press_releases/ovum-forecasts-big-data-software-to-grow-by-50/2 McKinsey & Company

Common Question“I know clearly how much I’m spending on my data initiatives – it’s millions of dollars. But I don’t feel like our company is that much smarter.

… Where’s all the insight I was promised?”

How do I generate insight from all this data?

Data Insight

Three questions1. What is insight?

2. Where does it come from?

3. How do I build a machine to generate it?

What is insight?

Better understanding.

Vx Vx + 1

Where does insight come

from?

Lone genius?

Lone genius?

Lone genius?

Lone genius?

“If I have seen further, it is by standing on the shoulders of giants.”

-- Sir Isaac Newton

Compounding

Where does insight come from?

0

5000

10000

15000

20000

1 11 21 31 41 51

5% 6%

Compounding interest

“My wealth has come from a combination of living in America, some lucky genes, and compound interest.”

-- Warren Buffett

Checklists

Academic Compounding

Compounding Code

“Reporters, investors, and others are always asking me for the secret ingredient. There is no one secret. We just worked every day to make the math better and express it well in code.” -- Early RenTech Employee

Compounding Investments

What if you don’t?• People waste time doing things that have

already been done.

• It’s difficult or impossible to make things better. There is no place where ideas go.

 • Idea generation suffers because people

aren’t exposed to the thinking that came before them … they have no one to stand on.

Codify Do Learn

1

2Invest in technology and related processes to make analysts’ work accessible and reproducible

Build an Insight Machine: Two Steps

Codify Do Learn

Codify Do Learn

Write down your procedure

Follow your procedure

Compare outcomes, diagnose, improve

The Full Circle

Codify Do Learn

1

2

Invest in technology and related processes to make analysts’ work accessible and reproducible

Build an Insight Machine: Two Steps

Invest in technology and related processes to make analysts’ work accessible and reproducible

2. Version control system

3. Create building blocks

1. Clarity standard

Building the insight machine

Follow on twitter: @dominodatalab@matthewgranade

Questions / Feedback?

Booth at Strata NYC: P3matthew@dominodatalab.com

dominodatalab.com