build, publish, deploy and test docker images and containers with jenkins workflow
TRANSCRIPT
![Page 1: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/1.jpg)
Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow.Nigel HarnimanSenior Solutions Architect, CloudBees Inc
![Page 2: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/2.jpg)
About me
Nigel Harniman
@harnimanBuild Engineer,
Devops and Architect
DevOps, Infra as Code,Continuous Delivery
![Page 3: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/3.jpg)
“Software is eating the world”
Marc Andreessen
![Page 4: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/4.jpg)
4
How Do You Deliver Better Software Faster?
ProdDev
![Page 5: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/5.jpg)
5
Automation is the Key
Photo courtesy of Steve Jurvetson via Flickr
![Page 6: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/6.jpg)
The Docker Advantage
![Page 7: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/7.jpg)
Docker Has PotentialAn example: Software Configuration Management Space
![Page 8: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/8.jpg)
Docker Has PotentialAn example: Software Configuration Management Space
![Page 9: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/9.jpg)
The New World Order: Containers Codify OS Config
9
ProdDev QA Staging
DEV Server/VM QA Server/VM STG Server/VMPROD Server/VM
<PROD OS config><STG OS config><QA OS config><DEV OS config>
App<code>
<APP OS config>
App<code>
<APP OS config>
App<code>
<APP OS config>
App<code>
<APP OS config>
![Page 10: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/10.jpg)
So is this how I build a Docker Image?
10
![Page 11: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/11.jpg)
Jenkins & Docker
![Page 12: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/12.jpg)
How Can You Use Jenkins & Docker Together?
+
![Page 13: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/13.jpg)
How Can You Use Jenkins & Docker Together?
1. Run Jenkins Masters & Slaves in Docker
2. Build, Test, & Deploy Docker Images from Jenkins
![Page 14: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/14.jpg)
1. Run Jenkins Masters & Slaves in DockerDocker (Cloud) – use Docker images as standardized build environments to improve isolation and elasticity
Docker Custom Build Environment – specify customized build environments as Docker containers
CloudBees Docker Shared Config – manage Docker (or Swarm) host configuration centrally in CloudBees Jenkins Operations Center
![Page 15: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/15.jpg)
2. Build, Test, & Deploy Docker Images from JenkinsBuild and Publish – build projects that have a Dockerfile and push the resultant tagged image to Docker Hub
Docker Traceability – identify which build pushed a particular container and displays the build / image details in Jenkins
Docker Hub Notification – trigger downstream jobs when a tagged container is pushed to Docker Hub
![Page 16: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/16.jpg)
Jenkins Workflow & Docker
![Page 17: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/17.jpg)
Jenkins Workflow PrimerJenkins powered CD pipelines
Jenkins Workflow
ProdDevPerf Test
BuildCommit Selenium Test Stage Deploy
Sonar Test
Pipelines Need: Branching Looping Restarts
Checkpoints Manual Input
??
![Page 18: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/18.jpg)
Key Workflow Features
18
Entire flow is one concise Groovy script using Workflow DSL• For loops, try-finally, fork-join …
Can restart Jenkins while flow is running
Allocate slave nodes and workspaces• As many as you want, when you want
Stages throttle concurrency of builds
Human input/approval integrated into flow
Standard project concepts: SCM, artifacts, plugins
![Page 19: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/19.jpg)
Jenkins Workflow + Docker
![Page 20: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/20.jpg)
20
Pipeline Stages
Build and Unit Test App
Test Docker Image
Publish Docker Image
SCM Checkoutmvn package
mvn sonar:sonarmvn verify
docker build
docker tag
docker run
notifycucumber
war
img
Sonar Analysi
s
Prepare Release
Build Docker Image
Int Test
docker push
image.inside withServer
![Page 21: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/21.jpg)
21
Build, unit test and package
Build and Unit Test App
Test Docker Image
Publish Docker Image
SCM Checkoutmvn package
mvn sonar:sonarmvn verify
docker build
docker Tag
docker run
notifycucumber
war
img
Sonar Analysi
s
Prepare Release
Build Docker Image
Int Test
docker push
image.inside withServer
![Page 22: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/22.jpg)
Build, unit test and package
stage 'Build App’
node('docker') {
docker.image(‘maven:3.3.3-jdk-8’).inside(‘-v /data:/data’ {
mkdir –p /data/mvn
writeFile file: 'settings.xml', text: ”(………)"
git 'https://github.com/cloudbees/mobile-deposit-api.git’
sh 'mvn –s settings.xml clean package’
…
Specify the Stage Name
Specify the slave labelCustom Build Env Mount volume from
slave
.m2 repo locationco and build
![Page 23: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/23.jpg)
Defining a Docker SlaveSpecify Image as
templateAssign labels
![Page 24: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/24.jpg)
24
Test the app
Build and Unit Test App
Test Docker Image
Publish Docker Image
SCM Checkoutmvn package
mvn sonar:sonarmvn verify
docker build
docker Tag
docker run
notifycucumber
war
img
Sonar Analysi
s
Prepare Release
Build Docker Image
Int Test
docker push
image.inside withServer
![Page 25: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/25.jpg)
Test the app
node('docker') {
docker.image(‘maven:3.3.3-jdk-8’).inside(‘-v /data:/data’ {
…
stage 'Sonar analysis’
sh 'mvn -s settings.xml sonar:sonar’
stage 'Integration-test’
sh 'mvn -s settings.xml verify’
step([$class: 'JUnitResultArchiver', testResults: '**/target/surefire-reports/TEST-*.xml'])
}
…
In same env as build
Sonar tests
Run API Tests
![Page 26: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/26.jpg)
26
Build, test and publish Docker image
Build and Unit Test App
Test Docker Image
Publish Docker Image
SCM Checkoutmvn package
mvn sonar:sonarmvn verify
docker build
docker Tag
docker run
notifycucumber
war
img
Sonar Analysi
s
Prepare Release
Build Docker Image
Int Test
docker push
image.inside withServer
![Page 27: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/27.jpg)
Build, test and publish Docker image
docker.withServer('tcp://192.168.99.100:2376', 'slave-docker-us-east-1-tls'){stage 'Build Docker image’
def mobileDepositApiImage
dir('.docker') {sh "mv ../target/*-SNAPSHOT.jar mobile-deposit-api.jar”
mobileDepositApiImage = docker.build "harniman/mobile-deposit-api:$
{buildVersion}”
} stage 'Test Docker image’
container=mobileDepositApiImage.run("--name mobile-deposit-api -p 8080:8080”)
sh "curl
http://<user>:<token>@<host>:8080/docker-traceability/submitContainerStatus \......// insert cucumber tests here
stage 'Publish Docker image’
withDockerRegistry(registry: [credentialsId: 'dockerhub-harniman']) {mobileDepositApiImage.push()
}
}
Bind to docker host
Change directory
Launch container
Build docker image
Bind to registryPush image
Submit traceability report
![Page 28: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/28.jpg)
28
Tagged Version
aTagged Image in Docker Huba
![Page 29: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/29.jpg)
Traceability
![Page 30: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/30.jpg)
30
Traceability
Builds on existing Jenkins artifact traceabilityAllows the tracking of the creation and use of Docker containers in Jenkins and their future use.Combine with artifact fingerprinting for a comprehensive solutionEach Build shows the image fingerprints created
Identify which build pushed a particular container and display the build / image details in Jenkins
Image fingerprints
![Page 31: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/31.jpg)
31
Traceability – registering eventsJenkins can track actions against this image such as:
• Creating a container• Container events such as start/stop
To achieve this, it is necessary to call the Traceability API – see $(JENKINS_URL)/docker-traceability/api/There are two endpoints to submit events to:/docker-traceability/submitContainerStatus
Allows to submit the current container status snapshot with a minimal set of parameters. Outputs of docker inspect $(containerId) can be directly submitted to Jenkins server using this command.
/docker-traceability/submitReport
Submits a report using the extended JSON API. This endpoint can be used by scripts to submit the full available info about the container and its environment in a single command.
![Page 32: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/32.jpg)
32
Traceability – registering events - exampleWorkflow usage example:
container = mobileDepositApiImage.run("--name mobile-deposit-api -p 8080:8080")
sh "curl http://<user>:<token>@<host>:8080/docker-traceability/submitContainerStatus \
--data-urlencode status=deployed \--data-urlencode inspectData=\"\$(docker inspect $container.id)\" \--data-urlencode environment=test \--data-urlencode hostName=mymac \--data-urlencode imageName=harniman/mobile-deposit-api"
Spin up container
Notify Jenkins
![Page 33: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/33.jpg)
33
Docker Traceability View
Docker TraceabilityContainer
![Page 34: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/34.jpg)
Container Use View
34
Deployment Events
Link to Build
![Page 35: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/35.jpg)
Dockerhub Notifications
![Page 36: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/36.jpg)
36
Docker Hub NotificationTrigger downstream jobs when a tagged container is pushed to Docker Hub
The Docker Hub Notification Trigger plugin lets you configure Jenkins to trigger builds when an image is pushed to Docker Hub. E.g. to run verification for the container.
What are the steps
Set up a WebHook Account for Notification
Set up your Docker Registry to make callbacks on Image events
Set up your builds
![Page 37: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/37.jpg)
37
Docker Hub Notification – Docker Registry WebhookIn the format:http://<user>:<token>@<jenkins_url>/dockerhub-webhook/notify
![Page 38: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/38.jpg)
38
Docker Hub Notification – Job Set up
Configure Trigger
![Page 39: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/39.jpg)
In Conclusion
![Page 40: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/40.jpg)
40
Docker and Jenkins with Workflow is the proven CD Platform
+
TESTING
STAGING
PRODUCTIONWorkflow CD Pipeline Triggers:• New application code (i.e. feature, bug, etc.)• Updated certified stack (security fix in Linux, etc.)
… will lead to a new gold image being built and available for…… TESTING
… STAGING… PRODUCTION
All taking place in a standardized/similar/consistent environment
<OS config>
Company“Gold”
Docker Img(~per app)
App<code>
(git, etc.)<OS config>
Certified DockerImages
(Ubuntu, etc.)
Jenkins Workflow
![Page 41: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/41.jpg)
CloudBees: Leading the Way for Docker and CD
Docker Workflow – Provides first-class support for Jenkins Workflow to build real world CD pipelines for containerized applications using Jenkins and Docker
Build and Publish – Builds projects that have a Dockerfile and pushes the resultant tagged image to Docker Hub
Docker Hub Notification – Triggers downstream jobs when a tagged container is pushed to Docker Hub
Docker Traceability – Identifies which build pushed a particular container that is running in production and displays that on the Jenkins builds page
Docker – Uses Docker containers as standardized build environments to improve isolation and elasticity – Dockerized Build Slaves
Docker Custom Build Environment – Specifies customized build environments as Docker containers
![Page 42: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/42.jpg)
Getting started
Docker plugin documentation:http://documentation.cloudbees.com/docs/cje-user-guide/docker-workflow.html
Workflow tutorial:https://github.com/jenkinsci/workflow-plugin/blob/master/TUTORIAL.md
Example Source Codehttps://github.com/harniman/mobile-deposit-api/blob/master/flow.groovy
![Page 43: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/43.jpg)
How Do You Manage CD at Enterprise Scale?
43
CloudBees Jenkins PlatformJenkins at Enterprise Scale for CI and CD
![Page 44: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow](https://reader036.vdocuments.mx/reader036/viewer/2022062302/5876fe2e1a28abf3398b6c2d/html5/thumbnails/44.jpg)
Thank you!Nigel Harniman@[email protected]