cloud-scale kubernetes at ebay
TRANSCRIPT
![Page 1: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/1.jpg)
kubernetes at eBay
![Page 2: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/2.jpg)
eBay cloud services
● Cloud Infrastructure and Platform Services
● Data Services● Database Infrastructure● Analytics Infrastructure
![Page 3: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/3.jpg)
Our portfolio Manages½ Million+ Compute Cores, 150K+ compute servers200+ PB of Data InfrastructureeBay.com
Data and Infra services handlesBillions of data Queries per day20 Billion+ Image Downloads Per Day300+ TB of logs per day2 Million Monitoring Metrics Per Second
![Page 4: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/4.jpg)
![Page 5: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/5.jpg)
app platforms current lifecycle - IaaS oriented
![Page 6: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/6.jpg)
current Host Runtime - Standardized Deployment Agent
![Page 7: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/7.jpg)
Kubelet + Container Runtime
![Page 8: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/8.jpg)
current static scheduling
![Page 9: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/9.jpg)
dynamic scheduling with a central cluster manager
![Page 10: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/10.jpg)
static middleboxes bottleneck
![Page 11: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/11.jpg)
distribute network and other framework functions
![Page 12: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/12.jpg)
kubernetesOpensource
Container based runtime
Declarative, application centric abstractions
Watchable, updateable central state store
Well defined cloud provider and infrastructure plugin model
Support from various IaaS clouds (Openstack, public...)
Awesome community with great thought leadership
Waiting for the evolution of “XPaaS on Kubernetes” marketplace
![Page 13: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/13.jpg)
![Page 14: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/14.jpg)
challenges
![Page 15: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/15.jpg)
Challenges: Networking
IP Routability ✔
IP Density
Multi-tenancy
![Page 16: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/16.jpg)
overlay based
![Page 17: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/17.jpg)
L3 routed model
![Page 18: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/18.jpg)
Plans: Networking
BGP to host
Layer 7 Load Balancing
IPV6
Overlay
![Page 19: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/19.jpg)
Challenges: Security
Integrate with cloud IAM (Openstack Keystone) ✔
Container security (selinux, seccomp, Docker Content Trust)
Distributed firewalls
Native Public Key Infrastructure on Kubernetes (CFSSL/Lemur)
Policy based IAM
![Page 20: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/20.jpg)
Challenges: Storage
Integrate with cloud block storage (Openstack Cinder ) ✔
Local storage leases
![Page 21: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/21.jpg)
Challenges: Scale
Currently has 5-20K servers per AZ
Cluster Federation
![Page 22: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/22.jpg)
Plans: Scale
Scale testing lab - work with scale SIG
Ubernetes
Static provisioning using detached Kubelet
![Page 23: Cloud-Scale Kubernetes at eBay](https://reader031.vdocuments.mx/reader031/viewer/2022021922/5875863b1a28ab901c8b4d11/html5/thumbnails/23.jpg)
thanks
araveendrann
spothanis
uruddaraju
thuc.nguyen
@
ashw7n @