plotcon nyc: building a flexible analytics stack
TRANSCRIPT
13
Editors in an Organizations Each Month
Sales
month/month
Lost accountsYear over year
125 8 $20,187,921.56
Gross revenueNew accountsmonth/month
1.75% 0.5% 0.5%
Accounts by State
Search…Sales
Editors in an Organizations Each Month
Sales
Personal
SPACES
General
Home
Financial Reports
Sept. Board Meeting
Marketing
Retention Reports
Churn analysis
15
Business application
Website
Central warehouse
Third party applications
Analytics applications
Event logging
Transformation and aggregation
E(T?)L
Communication and information platforms
Analytics applications
E(T?)L
19
Business application
• Web application • Mobile app • Inventory management system • Point of sale system • Sales records in Excel
25
Business application
Website
Analytics applications
Analytics applications
• Google Analytics • Mixpanel • Chartbeat • Kissmetrics • Amplitude
30
Business application
Website
Central warehouse
Analytics applications
Event logging
Analytics applications
31
Website
Central warehouse
Event logging
Analytics applications
• PostgreSQL • MySQL • Oracle • MS SQL Server • Redshift • BigQuery • Hive • Spark
32
Website
Central warehouse
Event logging
Analytics applications
• PostgreSQL • MySQL • Oracle • MS SQL Server • Redshift • BigQuery • Hive • Spark
33
Website
Central warehouse
Event logging
Analytics applications
• Google Analytics • Amplitude • Heap • Segment • Snowplow • Custom built
34
Business application
Website
Central warehouse
Analytics applications
Event logging
Analytics applications
35
Business application
Website
Central warehouse
Analytics applications
Event logging
E(T?)L
Analytics applications
36
Business application
Central warehouse
E(T?)L• Fivetran • Segment • ETLeap • Stitch Data • Data Virtuality • Xplenty • Alooma • Blendo • AWS Data Pipeline • Custom built
38
Business application
Website
Central warehouse
Analytics applications
Event logging
E(T?)L
Analytics applications
39
Business application
Website
Central warehouse
Analytics applications
Event logging
E(T?)L
Analytics applications
• Salesforce • Zendesk • Stripe • Mailchimp • Twilio • JIRA • Hubspot
• Marketo • NetSuite • Square • Github • Adwords • FB Ads . . . .
40
Business application
Website
Central warehouse
Third party applications
Analytics applications
Event logging
E(T?)L
Analytics applications
E(T?)L
41
Business application
Third party applications
E(T?)L
E(T?)L• Fivetran • Segment • ETLeap • Stitch Data • Data Virtuality • Xplenty • Alooma • Blendo • AWS Data Pipeline • Custom built
43
Business application
Website
Central warehouse
Third party applications
Analytics applications
Event logging
E(T?)L
Analytics applications
E(T?)L
44
Business application
Website
Central warehouse
Third party applications
Analytics applications
Event logging
Transformation and aggregation
E(T?)L
Analytics applications
E(T?)L
45
Transformation and aggregation
• Informatica • Pentaho • Alooma • Keboola • Airflow • Custom built • Analytics applications
46
Business application
Website
Central warehouse
Third party applications
Analytics applications
Event logging
Transformation and aggregation
E(T?)L
Analytics applications
E(T?)L
50
Business application
Website
Central warehouse
Third party applications
Analytics applications
Event logging
Transformation and aggregation
E(T?)L
Analytics applications
E(T?)L
51
Analytics applications
• Salesforce • Zendesk • Mailchimp • JIRA • Hubspot • Marketo • NetSuite • Adwords • FB Ads . . . .
• Email • Slack • Internal wikis • Customer sites
52
Analytics applications
• Salesforce • Zendesk • Mailchimp • JIRA • Hubspot • Marketo • NetSuite • Adwords • FB Ads . . . .
• Email • Slack • Internal wikis • Customer sites
• Analytics applications • Custom built • Processes
54
Business application
Website
Central warehouse
Third party applications
Analytics applications
Event logging
Transformation and aggregation
E(T?)L
Communication and information platforms
Analytics applications
E(T?)L
55
Business application
WebsiteThird party applications
Analytics applications
Analytics applications
57
Business application
Website
Central warehouse
Third party applications
Analytics applications
Event logging
Transformation and aggregation
E(T?)L
Communication and information platforms
Analytics applications
E(T?)L
58
Website
Website
Third party applications
Analytics applications
ETL
Event logging
Business application
Third party applications
Communication and information platforms
Analytics applications
Transformation and aggregation
ETL
62
componentsArchitect
componentsOptimize
the systemOptimize
the systemArchitect
Mature data infrastructure
New data infrastructure
Smaller team
Larger team