the road to continuous deployment (v6-16x9-jax)...continuous delivery accessible! hands-on,...

THE ROAD TO CONTINUOUS DEPLOYMENT - A CASE STUDY

MICHIEL ROOK

@michieltcs

▸ Developer, consultant, trainer, speaker

▸ @michieltcs

@michieltcs

@michieltcs

Large aging monolith

@michieltcs

Large aging monolith

Generates income

@michieltcs

Large aging monolith

Slow & complex

Generates income

@michieltcs

Large aging monolith

Slow & complex

Generates incomeTechnical debt

@michieltcs

@michieltcs

Manual releases

@michieltcs

Manual releases

Fragile tests

@michieltcs

Manual releases

Frequent issues

Fragile tests

@michieltcs

Manual releases

Frequent issues

Fragile testsLow velocity

GOALS

@michieltcs

GOALS

REDUCEISSUES

1

@michieltcs

GOALS

REDUCEISSUES

REDUCELEAD TIME

1 2

@michieltcs

GOALS

REDUCEISSUES

REDUCELEAD TIME

INCREASE PRODUCTIVITY

1 2 3

@michieltcs

GOALS

REDUCEISSUES

REDUCELEAD TIME

INCREASE PRODUCTIVITY

INCREASE MOTIVATION

1 2 3 4

@michieltcs

REFACTOR? REBUILD?

APPROACH

▸ API first

▸ Services per domain object (job, jobseeker, ...)

▸ Migrate individual pages

@michieltcs

ORIGINAL MONOLITH

DB

@michieltcs

ORIGINAL MONOLITH

PROXY

SERVICEORIGINAL MONOLITH

DB

DBDB

@michieltcs

ORIGINAL MONOLITH

PROXY

SERVICEORIGINAL MONOLITH

ORIGINAL MONOLITH

SERVICE SERVICE

SERVICE

PROXY

DB

DBDB

DB

DB DB

@michieltcs

ARCHITECTURE

FRONTENDS ARE SERVICES

@michieltcs

ARCHITECTURE

FRONTENDS ARE SERVICES

SERVICES BEHIND LOAD BALANCERS

@michieltcs

ARCHITECTURE

FRONTENDS ARE SERVICES

SERVICES BEHIND LOAD BALANCERS

ACCESS LEGACY DB'S

@michieltcs

ARCHITECTURE

FRONTENDS ARE SERVICES

SERVICES BEHIND LOAD BALANCERS

ACCESS LEGACY DB'S

SERVICES IN CONTAINERS

@michieltcs

IMPROVING

CONTINUOUS EVERYTHING

DEV BUILD / TEST

CONTINUOUS INTEGRATION

@michieltcs

DEV BUILD / TEST ACCEPTANCE PRODUCTION

CONTINUOUS DELIVERY

@michieltcs

DEV BUILD / TEST STAGING / ACCEPTANCE PRODUCTION

CONTINUOUS DEPLOYMENT

@michieltcs

EVERY COMMIT GOES TO PRODUCTION

IF IT HURTS DO IT MORE OFTEN

SMALL STEPS

@michieltcs

SMALL STEPS EARLY FEEDBACK

@michieltcs

SMALL STEPS EARLY FEEDBACK

REDUCE TIME TO RECOVER

@michieltcs

SMALL STEPS

EXPERIMENTS!

EARLY FEEDBACK

REDUCE TIME TO RECOVER

@michieltcs

@michieltcs

CHANGES IN IT PERFORMANCE OF HIGH PERFORMERS

Source: 2017 State Of DevOps report

ONLY COMMIT TO MASTER

NO BRANCHES

NO BRANCHESREALLY.

SHORT LIVED FEATURE BRANCHES?

DELAYING INTEGRATION

CONFLICTS

FEATURE TOGGLES

DECOUPLE DEPLOYMENTSFROM RELEASES

@michieltcs

FEATURE TOGGLE DEBT

CODE REVIEWS?

PAIR PROGRAMMING

CONTINUOUS INLINECODE REVIEW

BOY SCOUT RULE

DEVOPS

DEVSECOPS

DEVSECQAOPS

DEVSECQABIZOPS

DEVSECQABIZNETOPS

DEVSECQABIZSYSNETOPS

DEVTESTSECQABIZSYSNETOPS

DEV*OPS

TEAM BUILDS IT

TEAM RUNS IT

TEAM IMPROVES IT

TEAM IS RESPONSIBLE

BUILD PIPELINE

AUTOMATE REPEATABLE THINGS

CONTINUOUS TESTING

CONFIDENCE

DEFENSE IN DEPTH

UNIT TESTS

INTEGRATIONTESTS

ACCEPTANCETESTS

UI TESTS

@michieltcs

DEFENSE IN DEPTH

UNIT TESTS

INTEGRATIONTESTS

ACCEPTANCE

UI TESTS

@Test public void jobCannotBeFound() { when(jobRepository.getById(EXPECTED_JOB_ID)) .thenReturn(null); JobService jobService = new JobService(jobRepository); assertNull(jobService.getById(EXPECTED_JOB_ID)); verify(jobRepository).getById(EXPECTED_JOB_ID); }

@michieltcs

DEFENSE IN DEPTH

UNIT TESTS

INTEGRATIONTESTS

ACCEPTANCETESTS

UI TESTS

@Test public void shouldFindJob() { expectedJob = loadFixture('active_job.yml'); actualJob = repository.getById(expectedJob.getId()); assertThat(actualJob, isA(Job.class)); assertEquals(expectedJob.getId(), actualJob.getId()); }

@michieltcs

DEFENSE IN DEPTH

UNIT TESTS

INTEGRATIONTESTS

ACCEPTANCETESTS

UI TESTSScenario: Link to related job Given a job exists And there are related jobs available When that job is viewed Then a list of related jobs is shown And each related job links to the detail page of the related job

@michieltcs

DEFENSE IN DEPTH

UNIT TESTS

INTEGRATIONTESTS

ACCEPTANCETESTS

UI TESTS

@michieltcs

DEFENSE IN DEPTH

TESTER?

UNIT TESTS

INTEGRATIONTESTS

ACCEPTANCETESTS

UI

@michieltcs

UNIT TESTS

INTEGRATION TESTS

ACCEPTANCE TESTS

UI TESTS

SMOKETESTS

Cost Speed

Exploratorytesting & user

feedback

Monitoring& alerting

@michieltcs

GOOD ENOUGH

PIPELINE SPEED = KEY

PULL IMAGE

START NEW CONTAINER

WAIT FOR PORT

SMOKE TESTS / HEALTH CHECKS

ADD NEW CONTAINER TO LB

REMOVE OLD CONTAINER FROM LB

STOP OLD CONTAINER

@michieltcs

PULL IMAGE

START NEW CONTAINER

WAIT FOR PORT

SMOKE TESTS / HEALTH CHECKS

ADD NEW CONTAINER TO LB

REMOVE OLD CONTAINER FROM LB

STOP OLD CONTAINER

docker pull

@michieltcs

PULL IMAGE

START NEW CONTAINER

WAIT FOR PORT

SMOKE TESTS / HEALTH CHECKS

ADD NEW CONTAINER TO LB

REMOVE OLD CONTAINER FROM LB

STOP OLD CONTAINER

docker run

@michieltcs

PULL IMAGE

START NEW CONTAINER

WAIT FOR PORT

SMOKE TESTS / HEALTH CHECKS

ADD NEW CONTAINER TO LB

REMOVE OLD CONTAINER FROM LB

STOP OLD CONTAINER

wait_for: port=8080 delay=5 timeout=15

@michieltcs

PULL IMAGE

START NEW CONTAINER

WAIT FOR PORT

SMOKE TESTS / HEALTH CHECKS

ADD NEW CONTAINER TO LB

REMOVE OLD CONTAINER FROM LB

STOP OLD CONTAINER

uri: url: http://localhost:8080/health status_code: 200 timeout: 30

@michieltcs

PULL IMAGE

START NEW CONTAINER

WAIT FOR PORT

SMOKE TESTS / HEALTH CHECKS

ADD NEW CONTAINER TO LB

REMOVE OLD CONTAINER FROM LB

STOP OLD CONTAINER

template: src=haproxy.cfg.j2 dest=/etc/haproxy/haproxy.cfg

service: name=haproxy state=reloaded

@michieltcs

PULL IMAGE

START NEW CONTAINER

WAIT FOR PORT

SMOKE TESTS / HEALTH CHECKS

ADD NEW CONTAINER TO LB

REMOVE OLD CONTAINER FROM LB

STOP OLD CONTAINER

template: src=haproxy.cfg.j2 dest=/etc/haproxy/haproxy.cfg

service: name=haproxy state=reloaded

@michieltcs

PULL IMAGE

START NEW CONTAINER

WAIT FOR PORT

SMOKE TESTS / HEALTH CHECKS

ADD NEW CONTAINER TO LB

REMOVE OLD CONTAINER FROM LB

STOP OLD CONTAINER

docker stop

docker rm

@michieltcs

PIPELINE AS CODE

node { stage('Run tests') { sh "phpunit" sh "behat" } stage('Build docker image') { sh "docker build -t jobservice:${env.BUILD_NUMBER} ." sh "docker push jobservice:${env.BUILD_NUMBER}" } stage('Deploy staging') { sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER} -i staging deploy.yml" } stage('Deploy production') { sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER} -i prod deploy.yml" } }

@michieltcs

@michieltcs

FEEDBACK!

RESULTS

@michieltcs

Team acceptance

@michieltcs

Feature toggle cap

Team acceptance

@michieltcs

Feature toggle cap

Business alignment & agility

Team acceptance

@michieltcs

Feature toggle cap

Business alignment & agility

Focus on replacing legacy

Team acceptance

1

@michieltcs

Reduced number of

issues

1

1

@michieltcs

Reduced number of

issues

1

Build time< 10 min.(50+ per

day)

2

1

@michieltcs

Reduced number of

issues

1

Build time< 10 min.(50+ per

day)

Improved metrics & audience statistics

2 3

1

@michieltcs

Reduced number of

issues

1

Build time< 10 min.(50+ per

day)

Improved metrics & audience statistics

Increased confidence, velocity &

fun

2 3 4

LITERATURE

https://trunkbaseddevelopment.com/

Reduce cycle times. React faster to the market and test product ideas. Make things!

We make Agile, DevOps and

Continuous Delivery accessible!

Hands-on, results-oriented approach to

get you where you need to be.

Modern infrastructure and

pipelines in minutes.

THANK YOU!

@michieltcs / mrook@fourscouts.nl

www.michielrook.nl