final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3:...
TRANSCRIPT
![Page 1: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/1.jpg)
Final presentation
wherecamp berlin 2019
1
![Page 2: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/2.jpg)
Design Patterns of GeoData Driven Apps
Shoaib Burqhttps://geografia.com.au
http://deeplearningbox.com
WhereCamp 2019 (Berlin)
![Page 3: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/3.jpg)
Complexity in Data Driven Apps
3
greater number of data sources
more stakeholders
more applications
bigger datasets
![Page 4: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/4.jpg)
4
![Page 5: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/5.jpg)
5
![Page 6: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/6.jpg)
6
![Page 7: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/7.jpg)
7
Client Name
![Page 8: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/8.jpg)
v1 (2017) - Traditional Geo Applications
8
![Page 9: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/9.jpg)
v1: Challenges
9
No Quality Checks
Poor Schema Enforcement
Time Consuming
Not Scalable for Growth
![Page 10: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/10.jpg)
10
![Page 11: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/11.jpg)
11
v2: add workflow system
![Page 12: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/12.jpg)
12
v2: add workflow systemAdd S3 to Store
Raw Data Trigger Workflow
(AWS lambda)
![Page 13: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/13.jpg)
13
v2: add workflow systemAdd S3 to Store
Raw Data Trigger Workflow
(AWS lambda)
https://docs.prefect.io/core
![Page 14: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/14.jpg)
14
Forecasting with Machine / Deep Learning
![Page 15: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/15.jpg)
v3: Doing Data Science
15
![Page 16: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/16.jpg)
v3: Doing Data Science
16
Workstation with GPUs to train ML &
DL Models ————————— deeplearningbox.com —————————
![Page 17: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/17.jpg)
v3: Doing Data Science
17
Workstation with GPUs to build Deep Learning Models
Competing for Resources with
applicatioonManual Deployment
of ML Models
![Page 18: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/18.jpg)
v4: ML Model Tracking + Deployments
18
![Page 19: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/19.jpg)
v4: PostgreSQL read only replicas
19
![Page 20: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/20.jpg)
v4: MLFlow + RDMBS scaling
20
![Page 21: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/21.jpg)
v4: MLFlow + RDMBS scaling
21
Fix: Competing for ResourcesFix: ML
Experiment Track & Deploy
![Page 22: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/22.jpg)
Futureexpected demands from customers & team
22
![Page 23: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/23.jpg)
FutureInteractive Big Data Queries via Hadoop + Presto
23
![Page 24: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/24.jpg)
FutureReal-Time events integration (e.g. social media feed)
24
![Page 25: Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3: Doing Data Science 17 Workstation with GPUs to build Deep Learning Models Competing](https://reader035.vdocuments.mx/reader035/viewer/2022070720/5edfd6c3ad6a402d666b2372/html5/thumbnails/25.jpg)
Thank you! Questions?
Shoaib Burqhttps://geografia.com.au | http://
deeplearningbox.com
25