big data @ cloud scale
Post on 19-Jan-2015
306 Views
Preview:
DESCRIPTION
TRANSCRIPT
Chase McMichael CEO @chasemcmichael @infinigraph
Big Data @ Cloud ScaleTuesday, June 26, 2012
SVForum Silicon Valley Cloud Center
InfiniGraph solves a problem for marketers drowning in data by enabling them to
identify what's relevant to their customers.
We simplify trend identification empowering brands to create resonating content
marketing campaigns.
We’re drinking from the social media fire hose.
Big Data is Big Opportunity
Content insight involves a staggering amount of data:
3.2 billion pieces of content are shared each week between a billion + consumers.
How do you make sense of all this data?
Large Scale Industry Insights
Who is driving the greatest engagement in your business. What content is working and why?
How Big?
6
7
Trend Identification - Hypercuration™
Example: Real-Time trending content from 25 retail brands in one view. Know what your audience is resonating around.
Our Biggest Challenges
1. Data Store Scale (I/O & Performance)
2. Distributed Data Gathering (Drone / Broker Mesh)
3. Changing Landscape of Data (Changing API / Changing Data "Types" & Sources)
4. Categorization / Taxonomy Handling (Dealing with lots of disparate data)
Scale, Scale, Scale
@chasemcmichel @infinigraph
Big Data Problems Big Solutions
Quick Poll –
Who is doing Big Data Projects?
Open to Innovation
“It’s not information overload. It’s filter failure.” ― Clay
Shirky
@chasemcmichel @infinigraph
Big Data Requires People
• Deep expertise in statistics
• Machine learning
• Managers
• Analysts who know how to operate companies by using insights from Big Data,
13
People with:
Big Data is more than just technology
Data Scientist Must Have Four Key Skills
• They can take a data set and model it mathematically and
• Understand the math required to build those models; they
• Can actually do that, which means they have the engineering skills…
14
Someone who can find insights and tell stories from their data. Able to ask the right questions, and that is usually the hardest piece.
Batch or Real Time
15
Data can be a blessing or a curse
• Need the systems to monitor at a whole new level,
• Looking at usage patterns and
• Predicting future trouble before the trouble even happens
• Know a server pool needs tuning
• Need to provision additional services
The future of monitoring is about automation and prevention
Big Data and Real-time• Big Data is a fact of life and needs to be addressed. Big Process is old wine in new bottles. Tightly couple processes and data to stay competitive.
17
Killer Apps Needed
Business need a killer app
Let companies
‣ analyze,
‣ visualize and
‣ act on
18
All this without hiring a team of Stanford Ph.Ds, OR Let developers write big-data apps without having to reinvent the wheel.
Take Away
• What problem are you attempting to solve - stop thinking about tech
• Plan collection / storage carefully
• Plan for both batch and real time
• Have a people plan
19
“The world is one big data problem. There’s a bit of arrogance in that, and a bit of truth as well.”–Andrew McAfee
Chase McMichael CEO @chasemcmichael @infinigraph
top related