pydata london 2015 - how we turned everythingme into a data driven company
TRANSCRIPT
![Page 1: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/1.jpg)
HOW WE TURNED EVERYTHINGME INTO A DATA DRIVEN COMPANY
![Page 2: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/2.jpg)
Hello, I’m Arik.
![Page 3: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/3.jpg)
![Page 4: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/4.jpg)
We’re geeks. YMMV.
![Page 5: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/5.jpg)
![Page 6: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/6.jpg)
Requirements:1. scalable2. fast3. easy to query (accessible)
![Page 7: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/7.jpg)
Amazon Redshift
“Petabyte scale; massively parallel
Fully managed; zero admin…”
![Page 8: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/8.jpg)
![Page 9: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/9.jpg)
8TB total size of data*9 Nodes ClusterLoading new data every ~5 minutes~1500 query executions per day
![Page 10: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/10.jpg)
One main fact table: fact_events
![Page 11: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/11.jpg)
Step #2Give Everyone Access
![Page 12: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/12.jpg)
BI TOOLS?
![Page 13: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/13.jpg)
psql, SQL Workbench
![Page 14: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/14.jpg)
CSV Sharing
![Page 15: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/15.jpg)
(hackathing!)
![Page 16: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/16.jpg)
![Page 17: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/17.jpg)
![Page 18: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/18.jpg)
![Page 19: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/19.jpg)
![Page 20: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/20.jpg)
![Page 21: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/21.jpg)
![Page 22: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/22.jpg)
![Page 23: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/23.jpg)
![Page 24: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/24.jpg)
![Page 25: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/25.jpg)
re:dash supports multiple data sources:PostgreSQLRedshiftMySQLBigQueryMongoDBInfluxDBGraphite
![Page 26: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/26.jpg)
Everyone at the company is using re:dash at some capacity.
8K queries created to date, with >130 dashboards.
![Page 27: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/27.jpg)
Over 25* companies using it & 24 contributors.
* Probably more. Not everyone reaches out to me.
![Page 28: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/28.jpg)
Step #3Improve
![Page 29: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/29.jpg)
Giving everyone raw access is only the first step, not the end of the road.
![Page 30: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/30.jpg)
Working with SQL all the time -> repeating yourself.
![Page 31: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/31.jpg)
Template Queries, Parameters
![Page 32: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/32.jpg)
Alerts
![Page 33: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/33.jpg)
Search & Discovery
![Page 34: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/34.jpg)
Pandas, IPython integration
![Page 35: PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company](https://reader035.vdocuments.mx/reader035/viewer/2022070521/58f011b11a28ab360c8b45c9/html5/thumbnails/35.jpg)
• Give everyone at the company access to the data
Make it easy (accessible)Avoid restrictions