Dockerized Maven

Matthias Bertschy (http://matthiasbertschy.info)Delivering like a Pro with Docker5 October, 2016

who?

● working as a sysadmin since 2005● CISSP and RHCE● recently switched to DevOps● day-to-day work with OpenShift and Docker

about this talk

how to leverage custom Docker containers to:

● handle build dependencies on local workstations● provide clean Jenkins slaves● run tests inside orchestrated deployments● run tests inside OpenShift projects

build dependencies

pain

● every developer is different○ workstation OS

○ development tools

● every team (microservice) is different○ programming language

○ dependencies

● building quality software requires consistency● keeping the same environment throughout the pipeline is

the key

let's use Docker...

Dockerfile

FROM docker.io/centos

ENV MAVEN_VERSION 3.3.3

RUN yum update -y && \

yum install -y java-1.8.0-openjdk-headless && \

tar xzf apache-maven-$MAVEN_VERSION-bin.tar.gz -C

/usr/local/ && \

mkdir -p /usr/local/apache-maven-$MAVEN_VERSION/.m2/ && \

yum clean all

COPY contrib/settings.xml

/usr/local/apache-maven-$MAVEN_VERSION/conf/

/home/mbertschy/java

#!/bin/bash

cmd="/usr/bin/java"

cwd=`pwd`

user=`id -u`

group=`id -g`

docker run -i --privileged --user=${user}:${group}

--volume=${cwd}:${cwd} --workdir=${cwd} --entrypoint ${cmd}

--rm <mvn_container> "${@}"

/home/mbertschy/mvn

#!/bin/bash

MAVEN_VERSION=3.3.3

cmd="/usr/local/apache-maven-${MAVEN_VERSION}/bin/mvn"

cwd=`pwd`

user=`id -u`

group=`id -g`

docker run -i --privileged --user=${user}:${group}

--volume=${cwd}:${cwd} --workdir=${cwd} --entrypoint ${cmd}

--rm <mvn_container> "${@}"

usage

[mbertschy@devops tmp]$ /home/mbertschy/mvn validate

[mbertschy@devops tmp]$ /home/mbertschy/mvn package

[mbertschy@devops tmp]$ /home/mbertschy/java -jar target/*.jar

. ____ _ __ _ _

/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \

( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \

\\/ ___)| |_)| | | | | || (_| | ) ) ) )

' |____| .__|_| |_|_| |_\__, | / / / /

=========|_|==============|___/=/_/_/_/

:: Spring Boot :: (v1.2.5.RELEASE)

...

benefits

● build and test tools are centrally managed● different versions of tools do not interfere● workstation are configured in a breeze● developers can focus on producing quality code

Jenkins slaves

Jenkins: master vs slave

MASTER

● light workload

● at least one per team

(separation of duties)

● numerous / on-demand

● better virtualized

SLAVE

● heavy workload

● the need for speed

● better using bare metal

● could be shared to reduce cost

pain

● slaves should be shared between teams for cost reasons● every team (microservice) is different

○ programming language

○ dependencies

● maintaining and upgrading a slave farm is increasingly difficult without uniformity

● building quality software requires consistency● keeping the same environment throughout the pipeline is

the key

let's use Docker (again)...

... there's a plugin for that!

Docker Plugin

https://wiki.jenkins-ci.org/display/JENKINS/Docker+Plugin

● Maintainers: Kanstantsin Shautsou, Nigel Magnay● ~4000 installation per month, and growing● use a docker host to dynamically provision a slave, run a

single build, then tear-down that slave● optionally, the container can be committed, so that (for

example) manual QA could be performed by the container being imported into a local docker provider

Dockerfile

...

RUN yum install -y openssh-server rsync && \

/usr/bin/ssh-keygen -A && \

echo "UseDNS no" >>/etc/ssh/sshd_config && \

echo "GSSAPIAuthentication no" >>/etc/ssh/sshd_config && \

echo "StrictModes no" >>/etc/ssh/sshd_config && \

rm -f /run/nologin && \

useradd -u 500 -g root -p '...' jenkins && \

mkdir /home/jenkins/workspace && \

chown jenkins:root /home/jenkins/workspace

EXPOSE 22

configuration

Prerequisite: you need a docker host with docker.service listening on port 4243

● as a Jenkins administrator, go to Manage Jenkins● then Configure System● at the bottom of the page, you should find a button

proposing to Add a new cloud

usage

● in your Maven project configuration, you just have to Restrict where this project can be run

● and enter the correct label to match your new slave:

● you can then hit Build Now

usage (2)

Started by upstream project

"S11N/products-md-service-api-build" build number 118

originally caused by:

Started by an SCM change

[EnvInject] - Loading node environment variables.

Building remotely on slave01-f7d02a1538bf (mvn-build) in

workspace

/home/jenkins/workspace/S11N/production-orders-service-build

[WS-CLEANUP] Deleting project workspace...

Cloning the remote Git repository

...

benefits

● slaves are clean at every build● slaves are generic and can be reused for many build types

(mvn, npm, ...)● we can use lightweight OS for slaves (atomic CentOS)● easy maintenance● consistency

Docker Compose

what?

● tool for defining and running multi-container Docker applications

● great for development, testing, and staging environments● we use it for component testing on workstations

● define the services in docker-compose.yml● docker-compose up

docker-compose.yml

---

db:

image: postgresql-94-rhel7:latest

ports:

- "5432:5432"

environment:

POSTGRESQL_USER: myuser

POSTGRESQL_PASSWORD: ********

POSTGRESQL_DATABASE: mydb

docker-compose.yml (2)

...

product:

image: products-md-service:0.1.0-SNAPSHOT

ports:

- "18080:8080"

links:

- db

environment:

DB_URL: jdbc:postgresql://db:5432/mydb

DB_USER: myuser

DB_PASSWORD: ********

Docker

architecture

db172.17.0.2

product172.17.0.3

workstation10.10.1.32

pain

● links defined in docker-compose.yml are only resolvable from containers started by Compose

● testing and debugging should be performed as if the developer was inside Docker space

● building quality software requires consistency● keeping the same environment throughout the pipeline is

the key

let's use Docker (again)...

principle

● links defined in docker-compose.yml are only resolvable from containers started by Compose

● we already have our development environment packaged inside a container

● let's insert our developer inside the Compose world!

● we have called this pattern "the avatar"

Docker

architecture

db172.17.0.2

product172.17.0.3

workstation10.10.1.32

avatar172.17.0.4

docker-compose.yml

...

avatar:

image: <mvn_container>

ports:

- "2222:22"

links:

- db

- product

command: [/sbin/sshd, -D]

usage

[mbertschy@devops tmp]$ docker-compose up -d

Creating tmp_db_1

Creating tmp_product_1

Creating tmp_avatar_1

[...]$ rsync -a -e "ssh -p 2222" ./ jenkins@127.0.0.1/

[mbertschy@devops tmp]$ ssh -l jenkins -p 2222 127.0.0.1

jenkins@127.0.0.1's password:

[jenkins@e250549ee18d ~]$ pwd

/home/jenkins

[jenkins@e250549ee18d ~]$ ping db

PING db (172.17.0.2) 56(84) bytes of data.

64 bytes from db (172.17.0.2): icmp_seq=1 ttl=64 time=0.106 ms

benefits

● tests rely on link definitions● different versions of tools do not interfere● workstation are configured in a breeze● developers can focus on producing quality code

OpenShift

what?

Built on opensource projects:● Docker provides the abstraction for packaging apps● Kubernetes orchestrates Docker containers

OpenShift Enterprise adds:● Source code management, builds, and deployments● Image management and promotion● Application management● Team and user tracking for large developer organizations

component architecture

● database:○ 1x PostgreSQL pod

○ 1x service (SQL)

● backend:○ 1x SpringBoot pod

○ 2x service (REST, gRPC)

○ 1x route

● frontend○ 1x NGINX pod (serving node.js application)

○ 1x service (HTTP)

○ 1x route

pain

● pods and services are only accessible from containers started inside the same project

● routes can only publish HTTP (v1) based protocols● tests have to access services directly (flyway scripts for

databases, message brokers, binary protocols, ...)

● building quality software requires consistency● keeping the same environment throughout the pipeline is

the key

let's use Docker (one last time)...

principle

● pods and services are only accessible from containers started inside the same project (until 3.3...)

● we already have our development environment packaged inside a container

● let's insert our developer inside the OpenShift project!

● this is a second use of our pattern called "the avatar"

Dockerfile

...

# fix permissions to run inside openshift

RUN chmod 0770 /home/jenkins && \

chmod 2770 /home/jenkins/workspace && \

chmod 2770 /usr/local/apache-maven-$MAVEN_VERSION/.m2/

ADD oc-3.2.0.20-linux.tar.gz /usr/local/bin/

# when running from jenkins, plugin starts /sbin/sshd -D

# this default sleep is for OpenShift to keep pod running

CMD ["sleep", "infinity"]

template.yml

apiVersion: v1

kind: DeploymentConfig

metadata:

name: mvn-avatar

spec:

replicas: 1

template:

spec:

containers:

image: <mvn_container>

imagePullPolicy: Always

name: mvn-avatar

usage

[mbertschy@devops tmp]$ oc get pods

NAME READY STATUS RESTARTS AGE

mvn-avatar-1-94snc 1/1 Running 0 6d

db-products-md-srv-1-13o5f 1/1 Running 0 6d

products-md-srv-1-lw3cb 1/1 Running 0 6d

products-md-web-1-g1fu8 1/1 Running 0 6d

[mbertschy@devops tmp]$ oc rsync ./ mvn-avatar-1-94snc:/

[mbertschy@devops tmp]$ oc rsh mvn-avatar-1-94snc

sh-4.2$ ping db-products-md-srv

PING db-products-md-srv.qa-products-md.svc.cluster.local

(172.30.46.79) 56(84) bytes of data.

^C

usage from Jenkins

[products-md-ctest] $ /bin/sh -xe /tmp/hudson84804829430149.sh

++ oc get pod -l app=mvn-avatar --no-headers

++ awk '{print $1}'

+ avatarPod=mvn-avatar-1-94snc

+ oc rsh mvn-avatar-1-94snc mkdir -p S11N/products-md-deploy

+ oc rsync --progress=true S11N/products-md-deploy/

mvn-avatar-1-94snc:S11N/products-md-deploy/

+ oc rsh mvn-avatar-1-94snc bash -c 'cd

S11N/products-md-deploy ; source *-conf.sh || true ; export ;

mvn verify'

...

usage from Jenkins (2)

...

+ mkdir -p S11N/products-md-ctest/cucumberOutput

+ oc rsync

mvn-avatar-1-94snc:S11N/products-md-deploy/cucumberOutput/

S11N/products-md-ctest/cucumberOutput --no-perms=true

receiving incremental file list

20161003133154.json

benefits

● tests are run from inside OpenShift projects● native name resolution and load balancers are leveraged● developers can jump inside projects for live debugging● Jenkins integration is possible with minimal fuss

thanks