Tales of Training: Scaling CodeLabs with Swarm Mode and Compose
Damien DUPORTALTraining Engineer - CloudBees
@DamienDuportal - github.com/dduportal
AgendaWhat is this talk about?
● Trench Story● Drinking champagne● Training / CodeLabs● VM workload● Container workload● Orchestration
What is this talk NOT about?
● Toolings wars:○ Kubernetes vs. Swarm○ AWS vs. Azure
● Showing code (20 min only!)○ Help yourself (GitHub):
http://bit.ly/2yoYG0v
Drinking my own champagneLearn by TRYING:● Experimentation never wastes
time● Build something that works
and iterate
“CodeLab”?● Part of a training (learning experience)● Practical Exercise● Using an isolated environment ● Dedicated to a single user ● Synonym of “workshop” or
“hands on lab”
CodeLab == ProductionConstraints:● Network● Configuration diversity● User Error (Learning process)● Time boxed Sessions
CodeLab VM Content: ● Jenkins Training:
○ Store Git SCM (Gitea - Go)○ Jenkins Master + Build Agents (JVMs)○ Web Command Line (TTYDs - Python)○ WebIDE (Codiad - PHP)○ Store Artifacts (Artifactory - JVM)○ Docker Registry (Go)
Challenges● CodeLab defined by service list
○ Docker-compose● Portability
○ Docker as blueprint● Explicit definition
○ Definition as code
Step 1: VMs
Consul Servers
AWS EC2 VM
Host OS: Ubuntu
Docker & Compose
CodeLab Service Stack
Consul
AWS EC2 VM
Host OS: Ubuntu
Docker & Compose
CodeLab Service Stack
Consul
AWS EC2 VM
Host OS: Ubuntu
Docker & Compose
CodeLab Service Stack
Consul
AWS EC2 VM
Host OS: Ubuntu
Docker & Compose
CodeLab Service Stack
Consul
AWS EC2 VM
Host OS: Ubuntu
Docker & Compose
CodeLab Service Stack
Consul
AWS EC2 VM
Host OS: Ubuntu
Docker & Compose
CodeLab Service Stack
Consul
SSH Bastion
Reverse Proxies
Register into Consul
End User
Trainer
Deploys with Terraform
Load-Balancer
Reconfigure with Consul-Template
Step 1: Take AwaysIt works!Easy to begin with and to operateCost (1 attendee per VM)TTP (Time To Play)
Road to Native Orchestration● Solving Costs Issue: More attendees per VM● Solving TTP Issue:
○ Moving to container-based workload(Look at PWD speed!)
○ Un-tie Infra from Services
Step 2.1: Provide Infra
SSH Bastion“Infra Admin”
Deploys with Terraform
Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)
Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)
Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)
Swarm Manager (EC2 VM)Swarm Manager (EC2 VM)
Swarm Manager (EC2 VM)
Step 2.2: Provide Services
SSH Bastion“Infra Admin”
Deploys with Docker
Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)
Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)
Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)
Swarm Manager (EC2 VM)Swarm Manager (EC2 VM)
Swarm Manager (EC2 VM)
Deploys with Docker
Trainer
CodeLab Service Stack
Support Services
Support Services
Step 2.3: Benefits
Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)
Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)
Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)
Swarm Manager (EC2 VM)Swarm Manager (EC2 VM)
Swarm Manager (EC2 VM)
CodeLab Service Stack
Support Services
Support Services
End User
Step 2: Nice problems to have● CodeLab Scaling Pattern
○ (Deploy N times, NOT Scaling)● Monitoring / Metrics● Resource Management
CodeLab Scaling Pattern● Compose and Swarm:
○ Define a stack of services○ Scale each horizontally
● Concern here: ○ Deploy N times the same stack
Monitoring/MetricsIt is NOT a challenge
● Should have done it at Step 1● Feedback loop● A LOT of “Working Out
of the box” tools
Resource Management● Resource sharing patterns:
○ VM workloads: 1 stack for 1 VM resources○ Container workload: N stacks for
N VMs resources● JVM pattern: Capping Memory (OOM)● Need to measure and iterate
○ Automation
Step 2: Take Aways● It still works!● Easier to begin with and to operate● Cost: 4x less (4 attendees per VM)● TTP: ~10 s instead of ~10 min● Maintaining: 5 tools instead of 8