data driven culture & infrastructure from the ground up
TRANSCRIPT
Data-driven culture & infrastructure from the ground up
January 2017
Data-driven culture and infrastructure from the ground up
Agenda
● R&D and IT consultancy firm● Clients - Silicon Valley Startups from Round A to Round D● Hardcore Big Data Analytics team in Kazan
Data-driven culture and infrastructure from the ground up
January 2017
People driven
Data-driven culture and infrastructure from the ground up
Excel driven
January 2017
Data-driven culture and infrastructure from the ground up
BI driven
January 2017
Data-driven culture and infrastructure from the ground up
Data Driven Team
January 2017
Product Manager
Owners
Engineer UI Data Scientist
Sales/Marketing DevOps Customer
Data-driven culture and infrastructure from the ground up
Data Driven Team
January 2017
Product
Metrics
Ad-hoc reports
Reports
Data-driven culture and infrastructure from the ground up
Data Driven Team
January 2017
Product
Metrics
Ad-hoc reports
Reports
Engineer
Alerts, Errors
Metrics
Data-driven culture and infrastructure from the ground up
Data Driven Team
January 2017
Product
Metrics
Ad-hoc reports
Reports
Engineer
Alerts, Errors
Metrics
UX
Usage Patterns
A/B tests
Data-driven culture and infrastructure from the ground up
Data Driven Team
January 2017
Product
Metrics
Ad-hoc reports
Reports
Engineer
Alerts, Errors
Metrics
Data Scientist
Scripts
Models
UX
Usage Patterns
A/B tests
Data-driven culture and infrastructure from the ground up
Data Driven Team
January 2017
Product
Metrics
Ad-hoc reports
Reports
Owners
Revenue trends
Prescriptions
Insights
Engineer
Alerts, Errors
Metrics
Data Scientist
Scripts
Models
UX
Usage Patterns
A/B tests
Data-driven culture and infrastructure from the ground up
Data Driven Team
January 2017
Product
Metrics
Ad-hoc reports
Reports
Owners
Revenue trends
Prescriptions
Insights
Engineer
Alerts, Errors
Metrics
Data Scientist
Scripts
Models
Marketing
BI tools
A/B test
UX
Usage Patterns
A/B tests
Data-driven culture and infrastructure from the ground up
Data Driven Team
January 2017
Product
Metrics
Ad-hoc reports
Reports
Owners
Revenue trends
Prescriptions
Insights
Engineer
Alerts, Errors
Metrics
DevOps
Alerts, Errors
Metrics
Monitoring
Data Scientist
Scripts
Models
Marketing
BI tools
A/B test
UX
Usage Patterns
A/B tests
Data-driven culture and infrastructure from the ground up
Data Driven Team
January 2017
Product
Metrics
Ad-hoc reports
Reports
Customer
Dashboards
Personalization
Insights
Reports
Owners
Revenue trends
Prescriptions
Insights
Engineer
Alerts, Errors
Metrics
DevOps
Alerts, Errors
Metrics
Monitoring
Data Scientist
Scripts
Models
Marketing
BI tools
A/B test
UX
Usage Patterns
A/B tests
Data-driven culture and infrastructure from the ground up
Data Driven Team
January 2017
Product
Metrics
Ad-hoc reports
Reports
Customer
Dashboards
Personalization
Insights
Reports
Owners
Revenue trends
Prescriptions
Insights
Engineer
Alerts, Errors
Metrics
DevOps
Alerts, Errors
Metrics
Monitoring
Data Scientist
Scripts
Models
Marketing
BI tools
A/B test
UX
Usage Patterns
A/B tests
Data-driven culture and infrastructure from the ground up
● Product Managers - SQL
● Engineers - Production, Product, Metrics, SOFT SKILLS
● UX - Marketing, Analytics
● Sales Engineer
● Business - BI, Reports, Metrics
January 2017
New Cross Skills - Evangelism
Data-driven culture and infrastructure from the ground up
Infrastructure
Data-driven culture and infrastructure from the ground up
● You could not just BUY it
● It’s not a Data Warehouse
● It’s not easy at all
January 2017
Myths
Data-driven culture and infrastructure from the ground up
High Level
January 2017
App Servers
Production database
Logs
Storage
BI
Prescriptions
Application Data Infrastructure Visualisation
Data-driven culture and infrastructure from the ground up
Micro
January 2017
App Servers
Production database Logs agg/
Papertrail
Keen.ioClicdata
DatadogPrometheus
Application Data Infrastructure Visualisation
GA
Data-driven culture and infrastructure from the ground up
Small
January 2017
App Servers
Production database
Log agg/ Papertrail
S3
ClicdataInteranaLooker
DatadogPrometheus
Application Data Infrastructure Visualisation
Slave replica
Data-driven culture and infrastructure from the ground up
Medium
January 2017
App Servers
Prod
Log stream S3ClicdataInteranaLooker
DatadogPrometheus
Application Data Infrastructure Visualisation
Slave replica
ETL RedShift
Ad-hoc SQL
Data-driven culture and infrastructure from the ground up
Medium
January 2017
App Servers
Prod
AWS Kinesis S3ClicdataInteranaLooker
Datadog
Application Data Infrastructure Visualisation
Slave replica
AWS Lambda RedShift
Ad-hoc SQL
Data-driven culture and infrastructure from the ground up
Medium
January 2017
App Servers
Prod
Kafka S3ClicdataInteranaLooker
DatadogPrometheus
Application Data Infrastructure Visualisation
Slave replica
Spark Streaming RedShift
Ad-hoc SQL
Data-driven culture and infrastructure from the ground up
Big
January 2017
App Servers
Prod
TableauClicdataInteranaLooker
Application Data Infrastructure Visualisation
Slave replica
ETL/Spark/Streaming
RedShift
Ad-hoc SQL
Hadoop/Parquet Impala
Spark SQLData Science
www.squadex.com
125 University Avenue,
Suite 290, Palo Alto,
California, 94301
Questions, details?We would be happy to answer!