running your jenkins infrastructure with clusterhq

Running your Jenkins infrastructure with ClusterHQ

@RyanWallner@ClusterHQ

We will cover...

● What is Flocker?● Why use Flocker with your Jenkins Master?● 3 main ways to run Jenkins according to DockerHub● Fungible Jenkins Master● Separate volumes● Backups

@RyanWallner@ClusterHQ

Flocker is a container data volume manager that allows

stateful services to run inside containers in production.

@RyanWallner@ClusterHQ

Data associated with your Jenkins build pipeline

● Logs● Job output● Job configuration● Update data● Artifacts● Libraries● More!

@RyanWallner@ClusterHQ

Running your Jenkins master in a container?

Consider how this data is resilient and mobile when the container restarts, gets rescheduled or when the host running Jenkins fails.

@RyanWallner@ClusterHQ

Three main ways to run Jenkins according to DockerHub...

1. All of your data resides inside the container’s own file-system storage. Most vulnerable to loss.

@RyanWallner@ClusterHQ

2. Your data is placed into a Docker volume. This means that the data is within /var/lib/docker/volumes/ but won’t be safe if you lose your Docker host.

Three main ways to run Jenkins according to DockerHub...

@RyanWallner@ClusterHQ

Three main ways to run Jenkins according to DockerHub...

3. Mounting in a directory that exists on the Docker host outside of the Docker file-system. Leaves data vulnerable to node failure and container movement issues.

@RyanWallner@ClusterHQ

Fungible Jenkins Master

Start a Jenkins masterwith a Flocker volumeusing Docker Compose

@RyanWallner@ClusterHQ

Fungible Jenkins Master

Find your Jenkins admin password by running a command on your Jenkins container

@RyanWallner@ClusterHQ

Fungible Jenkins Master

You will be prompted to intall plugins. All plugins will be stored in /var/jenkins_home/plugins within your Flocker volume that’s mounted at /var/jenkins_home/

@RyanWallner@ClusterHQ

Fungible Jenkins Master

You have now added some data to your Jenkins home directory.

@RyanWallner@ClusterHQ

We can also verify that our Flocker volume is mounted so all this data is safely persisted to our EBS Flocker volume.

Fungible Jenkins Master

@RyanWallner@ClusterHQ

Fungible Jenkins Master

Now ready to handle failover. Instruct docker-compose to stop and rm -f the Jenkins build and let it start on a new host.

@RyanWallner@ClusterHQ

Fungible Jenkins Master

Add a scheduling filter telling it not to deploy to the server.

@RyanWallner@ClusterHQ

Fungible Jenkins Master

Restart your Jenkins master backup.

See it on a new node.

@RyanWallner@ClusterHQ

Note: I am using a fake IP in the ouput, yours would be different

More options...

@RyanWallner@ClusterHQ

Separate Volumes

Create separate volumes for different parts of Jenkins config

@RyanWallner@ClusterHQ

Using a Flocker volume that had plugins pre-loaded, plugin installation times decreased 4X.

Separate Volumes

@RyanWallner@ClusterHQ

Jenkins Backup Plugin

Docker Compose file with a separate backup volume that can be used to store our Jenkins home.

@RyanWallner@ClusterHQ

Backups

After you run this Jenkins master, you can configure the backup manager with your Flocker volume.

@RyanWallner@ClusterHQ

Backups

Recovering your Jenkins master by expanding a backup in a Flocker volume and using it at your Jenkins home.

@RyanWallner@ClusterHQ

You can use the jenkins_backups folder as your $JENKINS_HOME

Backups

@RyanWallner@ClusterHQ

Bring your Jenkins server up and you can login like normal from a point in time backup from a Flocker backup volume.

Backups

@RyanWallner@ClusterHQ

Stay tuned for more on running your CI/CD infrastructure with container-based data management tools from

ClusterHQ.

@RyanWallner@ClusterHQ