rob laber, lead cv engineer hristo bojinov, cto...cloud instance selection for your gpu fleet...
TRANSCRIPT
![Page 1: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/1.jpg)
Cloud Instance Selection for Your GPU FleetLessons from Developing Smart Kitchen Technology
Hristo Bojinov, CTORob Laber, Lead CV Engineer
![Page 2: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/2.jpg)
Motivation
![Page 3: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/3.jpg)
Motivation
![Page 4: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/4.jpg)
Motivation
![Page 5: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/5.jpg)
![Page 6: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/6.jpg)
The Thesis
![Page 7: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/7.jpg)
The Problem
Food ↔ People disconnect
Not-so-smart “smart kitchen”
Food info not available, not actionable
![Page 8: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/8.jpg)
The Vision
![Page 9: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/9.jpg)
What We Do
Food, personalization, technology
“The Way You Eat”
Icons made by Madebyoliver, Popcorn Arts, Freepik from www.flaticon.com are licensed by CC 3.0 BY
![Page 10: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/10.jpg)
What We Do
Food, personalization, technology
“The Way You Eat”
(⇒ Computer Vision is essential)
Icons made by Madebyoliver, Popcorn Arts, Freepik from www.flaticon.com are licensed by CC 3.0 BY
![Page 11: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/11.jpg)
Pushing the Envelope on CV
Go from here...
![Page 12: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/12.jpg)
Pushing the Envelope on CV
… to here.
![Page 13: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/13.jpg)
![Page 14: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/14.jpg)
Computer Vision at Innit
Helps us understand users
❖ Inventory, behaviors, multi-sensor fusion, market analytics❖ And, build a delightful user experience
Applications in storage and processing
❖ Recognize and act on food state❖ Visible light, depth, IR
Multi-site program (HQ, academia)
![Page 15: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/15.jpg)
CV Research
![Page 16: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/16.jpg)
Food Recognition Service (AWS)
❖ G2 & G3 instance backend (details coming up)❖ Frontend orchestrates auto and manual processing❖ Service API for 3rd party use
![Page 17: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/17.jpg)
CV Tech: Food Recognition System
Service Layer
Computer Vision
Backend
Offline ProcessingData
StoreWeb
Portal
![Page 18: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/18.jpg)
CV Tech: Food Recognition System
![Page 19: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/19.jpg)
CV Tech: Object Detection Stage
![Page 20: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/20.jpg)
CV Tech: Object Detection Stage
![Page 21: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/21.jpg)
CV Tech: Object Detection Stage
![Page 22: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/22.jpg)
CV Tech: Object Detection Stage
![Page 23: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/23.jpg)
CV Tech: Classification Stage
![Page 24: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/24.jpg)
CV Tech: Classification Stage
![Page 25: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/25.jpg)
CV Tech: Classification Stage
![Page 26: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/26.jpg)
CV Tech: Classification Stage
Controlled scene layout ⇒ precision
In-house data collection and tools
CNN based
![Page 27: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/27.jpg)
CV Tech: Product DB Image Retrieval
![Page 28: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/28.jpg)
CV Tech: Product DB Image Retrieval
❖ Exact product (or attribute) matching
❖ KAZE descriptors (GPU acceleration WIP, stay tuned)
➢ Current need to balance CPU/GPU
➢ Order-of-magnitude acceleration
❖ Hierarchical analysis in the pipeline
❖ + Precise matching using CNN
![Page 29: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/29.jpg)
Summary: The Challenge
Complex Scenes
❖ Segmentation / object detection required❖ Must handle occlusions, uncertainty❖ Highly variable lighting conditions❖ Product DB not 100% accurate❖ Non-image feeds?
![Page 30: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/30.jpg)
Cloud instances: types, plans, sizes...
![Page 31: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/31.jpg)
So what is all this?
Service Layer
Computer Vision
Backend
Offline ProcessingData
StoreWeb
Portal
![Page 32: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/32.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
Offline ProcessingData
StoreWeb
Portal
![Page 33: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/33.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
![Page 34: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/34.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Single responsibility
![Page 35: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/35.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Single responsibility
❖ Stateless GPUs
![Page 36: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/36.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Single responsibility
❖ Stateless GPUs
❖
![Page 37: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/37.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
![Page 38: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/38.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Non-vision tasks
![Page 39: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/39.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Non-vision tasks
❖ Horizontal scaling
![Page 40: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/40.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Non-vision tasks
❖ Horizontal scaling
❖ Reserved instances
![Page 41: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/41.jpg)
Decoupled Service Architecture
![Page 42: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/42.jpg)
Decoupled Service Architecture
WHY?
![Page 43: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/43.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
![Page 44: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/44.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
Service Layer
Service Layer
![Page 45: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/45.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
Service Layer
Service Layer
Cost-effective buffer
![Page 46: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/46.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
To summarize...
![Page 47: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/47.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
To summarize...
❖ Reserved instances
![Page 48: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/48.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
To summarize...
❖ Reserved instances
❖ Optimize hardware usage
![Page 49: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/49.jpg)
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
To summarize...
❖ Reserved instances
❖ Optimize hardware usage
Delegate tasks to save money!
![Page 50: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/50.jpg)
Isolated Background Jobs
Service Layer
Computer Vision
Backend
Offline ProcessingData
StoreWeb
Portal
![Page 51: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/51.jpg)
Isolated Background Jobs
Clustering
![Page 52: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/52.jpg)
Isolated Background Jobs
Big Data
![Page 53: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/53.jpg)
Isolated Background Jobs
Neural Net Training
![Page 54: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/54.jpg)
Isolated Background Jobs
Offline ProcessingData
Store
![Page 55: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/55.jpg)
Isolated Background Jobs
Offline ProcessingData
Store
❖ One-off or periodic
![Page 56: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/56.jpg)
Isolated Background Jobs
Offline ProcessingData
Store
❖ One-off or periodic
❖ Resource intensive
![Page 57: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/57.jpg)
Isolated Background Jobs
Offline ProcessingData
Store
❖ One-off or periodic
❖ Resource intensive
❖ Need it now?
![Page 58: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/58.jpg)
Spot Pricing?
![Page 59: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/59.jpg)
Spot Pricing?
❖ Auctioned resources
![Page 60: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/60.jpg)
Spot Pricing?
❖ Auctioned resources
❖ Price changes with resource availability
![Page 61: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/61.jpg)
Spot Pricing?
❖ Auctioned resources
❖ Price changes with resource availability
❖ Interrupted jobs
![Page 62: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/62.jpg)
Isolated Background Jobs
Offline ProcessingData
Store
❖ One-off or periodic
❖ Resource intensive
❖ Need it now?
Spot pricing saves money!
![Page 63: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/63.jpg)
Isolated Background Jobs
![Page 64: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/64.jpg)
Trade-Offs
![Page 65: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/65.jpg)
Trade-Offs
❖ Cost vs. Performance
![Page 66: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/66.jpg)
Trade-Offs
❖ Cost vs. Performance
❖ Local vs. Cloud
![Page 67: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/67.jpg)
Trade-Offs
❖ Cost vs. Performance
❖ Local vs. Cloud
❖ Cost vs. Complexity
![Page 68: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/68.jpg)
Trade-Offs
❖ Cost vs. Performance
❖ Local vs. Cloud
❖ Cost vs. Complexity
❖ Now vs. Later
![Page 69: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/69.jpg)
Takeaways
❖ Complex workload → use differentiated EC2 instances
❖ Profile GPU-dependent tasks, choose carefully
❖ Consider online / offline loads, spot instances
![Page 70: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/70.jpg)
Join the party! [email protected]
![Page 71: Rob Laber, Lead CV Engineer Hristo Bojinov, CTO...Cloud Instance Selection for Your GPU Fleet Lessons from Developing Smart Kitchen Technology Hristo Bojinov, CTO Rob Laber, Lead CV](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f7636545f021c034d7f0a/html5/thumbnails/71.jpg)
About Innit❖ Inform and elevate the interaction between people and food❖ 5+ years in the making, substantial funding, IP & tech❖ Partnerships (appliances, FMCG, culinary)
About the Presenters❖ R&D and Technology at Innit