big data - challenges and risks · big data embodies new data characteristics created by today’s...
TRANSCRIPT
Beispielpräsentation Tamedia, Datum, Autor
1
Big Data - Challenges and risks
Dr. Marcel Blattner Chief Data Scientist @Tamedia:Digital
The Tamedia Digital Analytics Team
5
Thomas Gresch Julian NordtMarcel Blattner Nicolas Perony Yannick Koechlin
Digital CTO Chief Data Scientist Business Analyst Data Scientist Data Engineer
▪Three years of business experience as a management consultant
▪Technology background with a masters degree in bioinformatics from ETH Zurich
▪Holds a Phd from ETH Zurich, Chair of Systems Design
▪Strong experience with agent based systems on social graphs
▪Experienced full stackengineer with 8+ years of experience
▪Former CTO of Rayneer
▪Leading the digital transformation of Tamedia from a technical perspective
▪Head of the Tamedia Data Analytics Team and other potentially transversal platforms
▪Physicist with a strong Entrepreneurial background
▪Leading the developmentof predictive analytics
▪Physicist (Phd) with a strong background in quantitative analysis and machine learning
Big Data - what else
“Knowing the name of something does not mean to know something”
- Richard P. Feynman
Big Data - The big promise
ZH Filmfestival
The analysis of a vast amount of data
may lead to new insights.
Big Data - The big promise
ZH Filmfestival
Big Data - why is everybody talking about it?
Big Data - why is everybody talking about it?
Data accessibility
Big Data - Characteristics
ZH Filmfestival
Big data embodies new data characteristics created by today’s digital marketplace
Big data means to analyze a vast amount of data from different sources and formats in
a very short time. The aim is to generate new insights leading to
competitive advantages
Big Data - core skills
Information management
Solid information
foundation
Standardised data
management practices
Insights accessible and
available
I
Analytics Skills and Tools
Skills developed as
a core discipline
Enabled by a robust set of
tools and solutions
Develops action-oriented
insights
Act on the data
Fact-driven leadership
Analytics used as
a strategic asset
Strategy and operations
guided by insights
Data Experts Data architecture, management,
governance, policy
Tool Developers Mask complexity and
analytics to lower skills
boundaries
Visualization
Expertise Interpret data sets,
determine correlations and
present in meaningful ways
Decision Making
Executive and
Management Apply information to solve
business issues
Industry Vertical
Domain Expertise Develop hypothesis, identify
relevant business issues,
ask the right questions
Big Data - skills and job trends
Big Data - skill landscape for data analytics team
Big Data - skill shortage
Among organisations worldwide today:
1 in 10
has all the skills it needs
to be successful applying advanced technology
for business benefit
1/4
have major skill gaps
in mobile, business
analytics, and security
40%
report a skill shortage in the
ability to manage information
We face a big skill shortage.
This will continue
for the next years.
Source: IBM Tech Report 2014
Big Data - Project cycle
ZH Filmfestival
▪ Portfolio companies and TDA
generate use case ideas
▪ Ideas are noted (light weight)
▪ Use case ideas are discussed
between TDA and portfolio
companies
▪ Promising ideas are formulated
in an initial brief and decision
on use case exploration is taken
▪ Iterative and collaborative data discovery
together with portfolio companies (2 – 3
weeks)
▪ Workshop to present findings to wider portfolio
company management, present approximate
business case and get principal decision on
next steps
▪ Formalise business case and define goals, costs,
timelines and commitments and project team
▪ Formal sign off of business case
▪ Iterative design and implementation of use
case
▪ Strong collaboration with portfolio
companies
▪ Build up of prototypes and testing of use
case
▪ Delivery of business case goals
▪ Operationalization of use case
▪ Training and knowledge
transfer to portfolio companies
Acquisition Exploration Implementation Operationalization
Initial discussion
Initialbrief
Datadiscovery
IdeasOperationalize
service
Iterative development
Design service
Develop & deliver service
FormalizeBusiness
case
Iterative
Gate 1: Principal checks
Gate 2: Business case sign off
Gate 3:
Implementation sign off
Businesscase
Workshop
Foundationalagreement on
use case
Analytic canvas
Deliverables
Big Data Analytics Iterative & Exploratory Data is the structure
IT Team Delivers Data On Flexible
Platform
Business Users
Explore and Ask Any Question
Traditional Analytics Structured & Repeatable
Structure built to store data
Business Users
Determine Questions
IT Team Builds System
To Answer Known Questions
Big Data - Whats the difference to traditional analysis?
Big Data Analytics Iterative & Exploratory Data is the structure
IT Team Delivers Data On Flexible
Platform
Business Users
Explore and Ask Any Question
Traditional Analytics Structured & Repeatable
Structure built to store data
Available Information
AnalyzedInformation
Capacity constrained down sampling of available information
Big Data - Whats the difference to traditional analysis?
Big Data Analytics Iterative & Exploratory Data is the structure
Analyze ALL Available Information
Whole population analytics connects the dots
Traditional Analytics Structured & Repeatable
Structure built to store data
Available Information
AnalyzedInformation
Capacity constrained down sampling of available information
Big Data - Whats the difference to traditional analysis?
Big Data Analytics Iterative & Exploratory Data is the structure
Analyze ALL Available Information
Whole population analytics connects the dots
Traditional Analytics Structured & Repeatable
Structure built to store data
Available Information
AnalyzedInformation
Capacity constrained down sampling of available information
Carefully cleanse a small information before any analysis
AnalyzedInformation
Big Data - Whats the difference to traditional analysis?
Big Data Analytics Iterative & Exploratory Data is the structure
Analyze ALL Available Information
Whole population analytics connects the dots
Traditional Analytics Structured & Repeatable
Structure built to store data
Available Information
AnalyzedInformation
Capacity constrained down sampling of available information
Carefully cleanse a small information before any analysis
AnalyzedInformation
Analyze information as is & cleanse as needed & existing repeatable
AnalyzedInformation
Big Data - Whats the difference to traditional analysis?
Big Data Analytics Iterative & Exploratory Data is the structure
Traditional Analytics Structured & Repeatable
Structure built to store data
Big Data - Whats the difference to traditional analysis?
Big Data Analytics Iterative & Exploratory Data is the structure
Traditional Analytics Structured & Repeatable
Structure built to store data
?AnalyzedInformation
Question
DataAnswer
Hypothesis
Start with hypothesis Test against selected data
Big Data - Whats the difference to traditional analysis?
Big Data Analytics Iterative & Exploratory Data is the structure
Traditional Analytics Structured & Repeatable
Structure built to store data
?AnalyzedInformation
Question
DataAnswer
Hypothesis
Start with hypothesis Test against selected data
Data leads the way Explore all data, identify correlations
Data
Correlation
All Information
Exploration
Actionable Insight
Big Data - Whats the difference to traditional analysis?
Big Data Analytics Iterative & Exploratory Data is the structure
Traditional Analytics Structured & Repeatable
Structure built to store data
?AnalyzedInformation
Question
DataAnswer
Hypothesis
Start with hypothesis Test against selected data
Data leads the way Explore all data, identify correlations
Data
Correlation
All Information
Exploration
Actionable Insight
Analyze after landing…
Big Data - Whats the difference to traditional analysis?
Big Data Analytics Iterative & Exploratory Data is the structure
Traditional Analytics Structured & Repeatable
Structure built to store data
?AnalyzedInformation
Question
DataAnswer
Hypothesis
Start with hypothesis Test against selected data
Data leads the way Explore all data, identify correlations
Data
Correlation
All Information
Exploration
Actionable Insight
Analyze after landing… Analyze in motion…
Big Data - Whats the difference to traditional analysis?
Big Data - science?
Big data is not science (in the traditional sense)
Big Data - What is missing so far
1. A comprehensive approach to using big data.
Big Data - What is missing so far
1. A comprehensive approach to using big data.
2. Getting the right information into the hands of decision makers.
Big Data - What is missing so far
1. A comprehensive approach to using big data.
2. Getting the right information into the hands of decision makers.
3. Effective ways of turning “big data” into “big insights.”
Big Data - What is missing so far
1. A comprehensive approach to using big data.
2. Getting the right information into the hands of decision makers.
3. Effective ways of turning “big data” into “big insights.”
4. Big data skills are in short supply.
Big Data - What is missing so far
1. A comprehensive approach to using big data.
2. Getting the right information into the hands of decision makers.
3. Effective ways of turning “big data” into “big insights.”
4. Big data skills are in short supply.
5. Big data privacy issues.
Big Data - Healthcare
• Personalized medicine Genome analytics in oncology
• Precision medicine
Computer based analysis in pathology and radiology
• Personalized and cooperative treatment planning Personalized therapy planning