running your jenkins infrastructure with clusterhq

Running your Jenkins infrastructure with ClusterHQ

@RyanWallner@ClusterHQ

https://twitter.com/RyanWallner


https://twitter.com/ClusterHQ


We will cover...

● What is Flocker?● Why use Flocker with your Jenkins Master?● 3 main ways to run Jenkins according to DockerHub● Fungible Jenkins Master● Separate volumes● Backups






Flocker is a container data volume manager that allows

stateful services to run inside containers in production.


https://clusterhq.com/flocker/introduction/

https://clusterhq.com/flocker/introduction/





Data associated with your Jenkins build pipeline

● Logs● Job output● Job configuration● Update data● Artifacts● Libraries● More!






Running your Jenkins master in a container?

Consider how this data is resilient and mobile when the container restarts, gets rescheduled or when the host running Jenkins fails.






Three main ways to run Jenkins according to DockerHub...

1. All of your data resides inside the container’s own file-system storage. Most vulnerable to loss.






2. Your data is placed into a Docker volume. This means that the data is within /var/lib/docker/volumes/ but won’t be safe if you lose your Docker host.








3. Mounting in a directory that exists on the Docker host outside of the Docker file-system. Leaves data vulnerable to node failure and container movement issues.






Fungible Jenkins Master

Start a Jenkins masterwith a Flocker volumeusing Docker Compose







Find your Jenkins admin password by running a command on your Jenkins container







You will be prompted to intall plugins. All plugins will be stored in /var/jenkins_home/plugins within your Flocker volume that’s mounted at /var/jenkins_home/







You have now added some data to your Jenkins home directory.






We can also verify that our Flocker volume is mounted so all this data is safely persisted to our EBS Flocker volume.








Now ready to handle failover. Instruct docker-compose to stop and rm -f the Jenkins build and let it start on a new host.







Add a scheduling filter telling it not to deploy to the server.







Restart your Jenkins master backup.

See it on a new node.


Note: I am using a fake IP in the ouput, yours would be different





More options...






Separate Volumes

Create separate volumes for different parts of Jenkins config






Using a Flocker volume that had plugins pre-loaded, plugin installation times decreased 4X.

Separate Volumes






Jenkins Backup Plugin

Docker Compose file with a separate backup volume that can be used to store our Jenkins home.


https://wiki.jenkins-ci.org/display/JENKINS/Backup+Plugin





Backups

After you run this Jenkins master, you can configure the backup manager with your Flocker volume.






Backups

Recovering your Jenkins master by expanding a backup in a Flocker volume and using it at your Jenkins home.






You can use the jenkins_backups folder as your $JENKINS_HOME

Backups






Bring your Jenkins server up and you can login like normal from a point in time backup from a Flocker backup volume.

Backups






Stay tuned for more on running your CI/CD infrastructure with container-based data management tools from

ClusterHQ.






running your jenkins infrastructure with clusterhq

Software