Chase McMichael CEO @chasemcmichael @infinigraph

Big Data @ Cloud ScaleTuesday, June 26, 2012

SVForum Silicon Valley Cloud Center

InfiniGraph solves a problem for marketers drowning in data by enabling them to

identify what's relevant to their customers.

We simplify trend identification empowering brands to create resonating content

marketing campaigns. 

We’re drinking from the social media fire hose.

Big Data is Big Opportunity

Content insight involves a staggering amount of data:

3.2 billion pieces of content are shared each week between a billion + consumers.

How do you make sense of all this data?

Large Scale Industry Insights

Who is driving the greatest engagement in your business. What content is working and why?

How Big?

6

7

Trend Identification - Hypercuration™

Example: Real-Time trending content from 25 retail brands in one view. Know what your audience is resonating around.

Our Biggest Challenges

1. Data Store Scale (I/O & Performance)

2. Distributed Data Gathering (Drone / Broker Mesh)

3. Changing Landscape of Data (Changing API / Changing Data "Types" & Sources)

4. Categorization / Taxonomy Handling (Dealing with lots of disparate data)

Scale, Scale, Scale

@chasemcmichel @infinigraph

Big Data Problems Big Solutions

Quick Poll –

Who is doing Big Data Projects?

Open to Innovation

“It’s not information overload. It’s filter failure.” ― Clay

Shirky

@chasemcmichel @infinigraph

Big Data Requires People

• Deep expertise in statistics

• Machine learning

• Managers

• Analysts who know how to operate companies by using insights from Big Data,

13

People with:

Big Data is more than just technology

Data Scientist Must Have Four Key Skills

• They can take a data set and model it mathematically and

• Understand the math required to build those models; they

• Can actually do that, which means they have the engineering skills…

14

Someone who can find insights and tell stories from their data. Able to ask the right questions, and that is usually the hardest piece.

Batch or Real Time

15

Data can be a blessing or a curse

• Need the systems to monitor at a whole new level,

• Looking at usage patterns and

• Predicting future trouble before the trouble even happens

• Know a server pool needs tuning

• Need to provision additional services

The future of monitoring is about automation and prevention

Big Data and Real-time• Big Data is a fact of life and needs to be addressed. Big Process is old wine in new bottles. Tightly couple processes and data to stay competitive.

17

Killer Apps Needed

Business need a killer app

Let companies

‣ analyze,

‣ visualize and

‣ act on

18

All this without hiring a team of Stanford Ph.Ds, OR Let developers write big-data apps without having to reinvent the wheel.

Take Away

• What problem are you attempting to solve - stop thinking about tech

• Plan collection / storage carefully

• Plan for both batch and real time

• Have a people plan

19

“The world is one big data problem. There’s a bit of arrogance in that, and a bit of truth as well.”–Andrew McAfee

Chase McMichael CEO @chasemcmichael @infinigraph