scaling open edx with kubernetes
TRANSCRIPT
![Page 1: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/1.jpg)
Scaling Open edX with KubernetesDevOpsDays Boston9.15.2015
![Page 2: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/2.jpg)
Who we are
Nate Aune
Morgan Robertson
![Page 3: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/3.jpg)
What we’ll cover
● Background -- Open edX
● Introducing Kubernetes
● Kubernetes concepts
● Scaling + resiliency
● Open edX on Kubernetes
![Page 4: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/4.jpg)
Open edX background
● edX: non-profit founded by MIT and Harvard
● 500+ courses, 5M students learning on edX.org
● edX released Open edX in June 2013
● Stanford, MongoDB, Salesforce, Google, Microsoft,
McKinsey, Johnson & Johnson, Smithsonian
![Page 5: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/5.jpg)
Open edX - a catalyst for innovation
212 Contributors
One of the fastest growing open source projects on Github
![Page 6: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/6.jpg)
Technical components
LMS/CMS (Django/Python)
Forum (Sinatra/Ruby)
User DB (MySQL)
Course DB (Mongo)
Tasks (Celery/RabbitMQ)
Caching (Memcache)
Proxy (Nginx)
Search (ElasticSearch)
Mapreduce (Hadoop)
![Page 7: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/7.jpg)
Hosting infrastructure
S3 for serving:
● static assets
● grade downloads
● certificate downloads
● videos (for mobile)
● Load balancer
● Application server(s)
● Database server(s)
● Search server
● Utility server (tasks)
● Caching server
● Hadoop cluster
![Page 8: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/8.jpg)
Typical scalable deployment of Open edX on AWS
![Page 9: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/9.jpg)
Introducing Kubernetes
● Scheduling + orchestration layer for containerized applications
● Abstracts your infrastructure
● Open source project by Google
● Production-ready as of July 2015
![Page 10: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/10.jpg)
Kubernetes architecture
![Page 11: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/11.jpg)
Kubernetes vs. the Docker triad
Kubernetes Swarm Compose Machine
Scheduling ✔ ✔
Service discovery ✔ ✅
Container scaling ✔ ✔
Machine provisioning ✅ ✔
Health checking ✔
Secret management ✔
Production-ready ✔
![Page 12: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/12.jpg)
Kubernetes core concepts
● Pods
● Services
● Replication controllers
![Page 13: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/13.jpg)
Pods
● Group of containers + volumes scheduled together
● Smallest deployable unit
● Containers share certain resources including network stack
![Page 14: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/14.jpg)
Services
![Page 15: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/15.jpg)
Services
● Endpoint for a set of pods
● IP address, port, and label selectors
● Use round-robin routing to direct traffic to backend pods
![Page 16: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/16.jpg)
Services + Pods
![Page 17: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/17.jpg)
Replication Controllers
● Manage pod lifecycles for a number of replicas
● Provide scaling + fault tolerance
● Use label selectors
![Page 18: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/18.jpg)
Pods + Services + Replication Controllers
![Page 19: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/19.jpg)
Scaling with Kubernetes
● Replication controllers scale pods
● Services provide a single endpoint for a group of pods
● The Kubernetes master schedules pods across nodes
![Page 20: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/20.jpg)
Resiliency with Kubernetes
● Replication controllers ensure a number of pods are running
● Services provide load balancing
● Health checks allow bad pods to be ignored/removed
![Page 21: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/21.jpg)
Open edX on Kubernetes
● Goals:
○ Multi-tenant
○ Scalable + resilient
![Page 22: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/22.jpg)
The challenge
![Page 23: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/23.jpg)
Architecture
![Page 24: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/24.jpg)
Monitoring with Sysdig
![Page 25: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/25.jpg)
Sysdig drill-down
![Page 26: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/26.jpg)
Lessons learned
● Containers should be stateless
● Put initialization tasks into separate pods that run once
● Services can be used to abstract non-containerized components
![Page 27: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/27.jpg)
Conclusion
● We’re still learning, but..
○ Kubernetes is a promising technology for providing both scalability and resiliency
![Page 28: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/28.jpg)
More info
Open edX - http://open.edx.org
Kubernetes - http://kubernetes.io
Google Container Engine - http://cloud.google.com/container-engine
![Page 29: Scaling Open edX with Kubernetes](https://reader034.vdocuments.mx/reader034/viewer/2022052116/589b41c11a28ab22038b70af/html5/thumbnails/29.jpg)
Thank you for your time!Questions?
Slides: http://bit.ly/open-edx-kubernetes
[email protected]@appsembler.com
@appsembler