scaling a conventional on- premise ci infrastructure to the cloud … · 2019-03-25 · kubernetes...
TRANSCRIPT
![Page 1: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/1.jpg)
Scaling a conventional on-premise CI infrastructure to
the cloudMikhail Advani
Sebastian CarlierMendix B.V.
![Page 2: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/2.jpg)
Who are we?● Mikhail Advani - Site Reliability Engineer at Mendix
○ Twitter - @mikhail_advani○ Website - https://mikhailadvani.github.io
● Sebastian Carlier - “Devops” Engineer at Mendix○ Twitter - @c4rlier
![Page 3: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/3.jpg)
What were we shipping?
![Page 4: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/4.jpg)
...and how?
![Page 5: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/5.jpg)
Symptoms of the problem● Tight coupling between build jobs and hardware● Periodic housekeeping on servers● Difficulty in upgrading infrastructure● Long feedback loop for builds
Source: https://xkcd.com/303/
![Page 6: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/6.jpg)
Design Goals● Fresh build agents with every run● Immutable infrastructure● Scalable infrastructure● Infrastructure usage overview
![Page 7: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/7.jpg)
Migration Strategy
![Page 8: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/8.jpg)
Infrastructure as Code● Infrastructure
![Page 9: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/9.jpg)
Infrastructure as Code● Infrastructure
○ Amazon Web Services○ Terraform○ Kubernetes via Kops○ RDS
![Page 10: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/10.jpg)
Infrastructure as Code● Infrastructure
○ Amazon Web Services○ Terraform○ Kubernetes via Kops○ RDS
● Configuration management
![Page 11: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/11.jpg)
Infrastructure as Code● Infrastructure
○ Amazon Web Services○ Terraform○ Kubernetes via Kops○ RDS
● Configuration management ○ Packer with Ansible○ Docker
![Page 12: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/12.jpg)
Code Architecture
![Page 13: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/13.jpg)
How do we secure this infrastructure?
![Page 14: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/14.jpg)
What did dynamic scaling ever do for us?
![Page 15: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/15.jpg)
Resource utilization
![Page 16: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/16.jpg)
What really worked for us?
![Page 17: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/17.jpg)
If we were to do this again, what would we change?
![Page 18: Scaling a conventional on- premise CI infrastructure to the cloud … · 2019-03-25 · Kubernetes cluster monitoring 692.59 GiB ms CPU (1m avg) BIO.420/0 CPU (1m 186.00m. 1 oas 1](https://reader033.vdocuments.mx/reader033/viewer/2022042223/5ec98732daa2c748417ef8c9/html5/thumbnails/18.jpg)
Thanks
https://xkcd.com/1629/