1. 1. Scaling Your Jenkins Master with Docker London, UK 23-24 June
2. 2. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW: Setup, Usage WHERE: Future 2
3. 3. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW: Setup, Usage WHERE: Future 3
4. 4. #jenkinsconf WHO am I I Worked in Queueing Networks, Simulation, Smartcards, now TV Software for Operators Develop Set-top-box OS (Canal+, Proximus, SkyIt,..) 80% of Developers 4
5. 5. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW: Setup, Usage WHERE: Future 5
6. 6. #jenkinsconf WHAT: Docker Company (Docker Inc) Open Source (Apache license) platform for building, shipping, running distributed applications Technology: manage and run lightweight Linux containers Isolation Portability, Shipping 6
7. 7. #jenkinsconf WHAT: Docker in Jenkins 7 Docker Plugin Dynamically provision slaves Run build Tear-down slave Slaves in the Cloud
8. 8. #jenkinsconf WHAT: Jenkins in Docker Official Jenkins LTS docker image From Cloudbees, sponsor of DockerCon Open Source https://github.com/jenkinsci/docker Easy to use docker pull jenkins docker run -p 8080:8080 jenkins 8
9. 9. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW: Setup, Usage WHERE: Future 9
10. 10. #jenkinsconf WHY: Situation Before 10 One Master 34 attached slaves 527 jobs 78 plugins admin users Fast-paced environment. No real control
11. 11. #jenkinsconf WHY: Issues 11 Performance Reliability Inability to upgrade Downtime => Projects spin-off masters Security
12. 12. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW: Setup, Usage WHERE: Future 12
13. 13. #jenkinsconf HOW: Setup for one master (1) Three possibilities for /var/jenkins_home inside the container: docker run -p 8080:8080 jenkins mounted from a host directory: docker run -v :/var/jenkins_home jenkins mounted from a data volume container: docker run -v /var/jenkins_home --name=data busybox true docker run --volumes-from=data jenkins 13
14. 14. #jenkinsconf HOW: Setup for one master (2) Summary for one master: pull the official Jenkins image: $ docker pull jenkins create the data volume and master containers: $ docker run -v /var/jenkins_home --name=frog-data busybox true run the master container: $ docker run -t -i -u root -p 8084:8080 -p 50004:50004 --volumes-from=frog-data --name=frog jenkins 14
15. 15. #jenkinsconf HOW: Setup n masters Launch an httpd dispatcher (e.g., Apache) 443 (https) vhosts(name) 808i Launch docker containers: the Data Volume containers, with names the Jenkins masters containers linked with --volumes- from and 808i / 5000i ports map each container port 8080 to host ports 8080+i 15
16. 16. #jenkinsconf HOW: Usage 16 Pressed Json DRAFT Apache cert.crt privat.key frog-data polka-data frog Pressed Json DRAFT pressed .json LIVE Pressed Json DRAFT Pressed Json DRAFT polka Pressed Json DRAFT Pressed Json DRAFT pressed .json LIVE Pressed Json DRAFT Pressed Json DRAFT Frog.conf ServerName frog-ci.com ProxyPass localh:8084 X-F-Port443 g6-data tools-data 443 8084 8080 8080 808i --volumes -from --volumes -from 50004 Slaves
17. 17. #jenkinsconf HOW: Admin Usage (1) Install plugins (from /tmp/plugins) $ docker run -t -i --rm --volumes-from=frog-data -v /tmp:/tmp ubuntu bash root@1a210c0a1846:/# cd /var/jenkins_home/plugins root@1a210c0a1846:/# cp /tmp/plugins/* . root@1a210c0a1846:/# exit Similar technique for: full backup debug data (SCM Sync git repos, Plots csv, etc.) 17
18. 18. #jenkinsconf HOW: Admin Usage (2) Change image stop container, destroy, create a new one $ docker stop frog $ docker rm frog $ docker run --volumes-from=frog-data --name frog $ docker logs -f frog 18
19. 19. #jenkinsconf HOW: Admin Usage (3) Three levels of backup Vmware: dump of entire docker server Docker: copy all Data Volume Jenkins: SCM Sync plugin Three levels of monitoring Vmware, Docker, Jenkins Security: Role Strategy plugin 19
20. 20. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW: Setup, Usage WHERE: Future 20
21. 21. #jenkinsconf WHERE: do we go from here? Orchestration manual setup ok for a few more => orchestration tool (e.g., Docker Compose) 21
22. 22. #jenkinsconf WHERE: do we go from here? 22 Docker vs Config/Automation tools (chef, puppet, ansible, saltstack) Before: use ansible to setup hardware/VM, install packages, deploy code, run app. After: use ansible to setup hardware/VM, install Docker, run containers. use Dockerfiles to install packages, deploy code, run app.
23. 23. #jenkinsconf WHERE: do we go from here? Monitoring (docker level) Find a way to Configure System programmatically LDAP Plugins upgrades TCP port for slaves Markup formatter SCM Sync, Roles, IRC, etc. 23
24. 24. #jenkinsconf Thank You @cmuller13 on twitter 24