Continuous Delivery for DatabasesPresented by: James SmithCo-Founder, DevOpsGuys

ABOUT ME

Co-founder @ DevOpsGuys

Two decades building & delivering enterprise web systems

Helped many, many companies implement Continuous Delivery practices

Found High Quality Belgian Beer in late nineties!

Who is this guy?

WHY?Why is this important?

WHY?Why is this important?

BUSINESS HAS CHANGED

SPEED MATTERS

7

DEPLOYMENT RATES

8

300 Deployments / Year

50-60 Deployments / Day

10+ Deployments / Day

Every 11.6 seconds

HIGH PERFORMERS

HIGH PERFORMING IT ORGANISATIONS DEPLOY 30

TIMES MORE FREQUENTLY, WITH 50% FEWER FAILURES WITH

8000X FASTER LEAD TIMES THAN THEIR PEERS.

9

HIGH PERFORMERS

THEY ARE ALSO TWO TIMES MORE LIKELY TO EXCEED PROFITABILITY, MARKET SHARE &

PRODUCTIVITY GOALS

THEY EXPERIENCE 50% HIGHER MARKET CAPITALIZATION GROWTH OVER 3 YEARS

10

HOW?How have things changed?

11

2010

2012

2001 (1998)

2001 (and earlier)

DEV HAS CHANGED

WE KNOW AGILE WORKS

REGULAR, RAPID, RELIABLE AND CONTROLLED DELIVERY OF WORKING SOFTWARE SYSTEMS INTO PRODUCTION

Matthew Skelton

DEVOPS CALMS MODEL

Culture Hearts & Minds, Embrace Change

Automation Of all the things; testing, deployment, infrastructure

Lean Small batch sizes, value for end-users

Measurement Of all the things; show the improvement

Sharing Open, transparent collaboration

WWW.DEVOPSGUYS.COM - @DEVOPSGUYS 15

Plan Code Build Test Release Deploy Operate

CONTINUOUS ?

16

Continuous Integration

Continuous Delivery

Continuous Deployment

DevOps

Nhan Ngo, a QA engineer at Spotify

CONTINUOUS DELIVERY

1. Build Management & CI

2. Environments & Deployment

3. Release Management & Compliance

4. Testing

5. Data Management

18

Your systems are under pinned by data and in most organisations data is the hardest and most expensive item to manage.

Data is needed by developers and testers to ensure that software products are high quality. Its needed by support teams to

troubleshoot and resolve issue effectively.

20

BUT DATABASE CHANGE IS SCARY

21Photo Credit: http://cache.lego.com/r/www/r/movie/-/media/franchises/the%20lego%20movie/explore/downloads/wallpapers/lego_wps_1600_emmet.jpg?l.r=-1410608028

NO, THIS IS SCARY

Photo Credit: His mum

SO WE MINIMISE RISK

Photo Credit: https://ideas.lego.com/projects/62456

WE DEFEND THE THREAT

Photo Credit: "Rumrunner" via Compfight cc

WE STOP CHANGE

25Photo Credit: s.kosoris via Compfight cc

CUSTOMERS GET SAD

26Photo Credit: Kalexanderson via Compfight cc

THE BUSINESS GETS ANGRY

27Photo Credit: powerpig builds via Compfight cc

AND IT GET THE BLAME!

28Photo Credit: bobsfever via Compfight cc

Dev are shipping application changes frequently

Ops are creating servers - automatically

29

WHY IS THERE SO MUCH LEGOIN YOUR SLIDE DECK?

Perfectly normal question

LEGO = DEVOPS

Rapid proto-typing & experimentation

Building blocks no right or wrong way

Promotes collaboration

Strong cultural appeal

Small batch sizes

Visibly measureable

Manufacturing

Zenon

Mindstorms

Danish phrase leg godt, which means "play well".

WHY?Why is database automation so hard?

!= CTRL C, CTRL V

Database deployment is not copying and replacing.

It is the transformation from a previous version to the next version while preserving data integrity.

Deploying database change is hard

Deploying database change frequently is even harder

WHAT?What are the common challenges?

#1: VERSION CONTROL

Photo Credit: ntr23 via Compfight cc

VERSION CONTROL

Database changes not under VCS

Worse

Communication of change

t

#2: AGAINST THE FLOW

37

Photo Credit: slang589 via Compfight cc

Photo Credit: slang589 via Compfight cc

AGAINST THE FLOW

DEV PRODUCTION

38

DEV PRODUCTION

CODE

DATA

#3: COMPLEXITY

39Photo Credit: L@go via Compfight cc

COMPLEXITY

Dependence

Centralisation

#4: DATA VOLUME

Photo Credit: http://www.calgaryherald.com/news/calgary/Gallery+LEGO+KidsFest/9848406/story.html

DATA VOLUME

Transit time

Historical data no archiving

One of the biggest impacts on Cycle time

#5: CONWAY'S LAW

Photo Credit: kirk_arts via Compfight cc

CONWAY'S LAW

"Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a

design whose structure is a copy of the organization's communication structure

- Conway, 1968

44

CONWAY'S LAW

Is (was) a database really needed? 3 Tiers anyone?

Centralised vs Decentralised?

Formalised [change] control

HOW?So what are some of the solutions?

#1: GET UNDER CONTROL

Photo Credit: RHiNO NEAL via Compfight cc

VERSION CONTROL

it should be in VCS!

it should be in VCS!

it should be in VCS!

Schema & Static/Reference Data

Reverse engineer existing schema & reference data

#2: INTEGRATE CONTINUOUSLY

49Photo Credit: ....Tim via Compfight cc

CONTINUOUS INTEGRATION

Automate your build steps (Full vs Incremental)

Build a library of manual tests

Automate your tests

50

#3: AUTOMATE TESTING

51Photo Credit: s3aphotography via Compfight cc

TEST, TEST, TEST

Select the right tests for each stage;Unit testing

Integration Testing

Deployment Validation

Behaviour Validation

Determine the right data for testingDo you need it all?

52

#4: AUTOMATION

53Photo Credit: pasukaru76 via Compfight cc

AUTOMATE ALL THE THINGS

Testing

Deployment

Back-up and more importantly restore

Archiving

Rollback

54

#5: MICROSERVICES

55

RE-ARCHITECT

Small is good

Split data along sensible partitions

Polyglot persistence

56

POLYGLOT PERSISTENCE

57

#6: CHANGE YOUR WAYS

58Photo Credit: Stéfan via Compfight cc

Technology is only half of the story

Management must think of operations as part of development

Deployment is part of development

Data retention is part of development

Fail faster, but fail safely

59

TOOLINGCan tools help?

60

SOURCE CONTROL

Source control is the foundation of Continuous Integration

Plus many, many more

CONTINUOUS INTEGRATION

Continuous Integration is the common orchestration point

62

DEPLOYMENT

Repeatable and consistent deployments every time.

63

REDGATE DLM (SQL SERVER)

64

MONITORING

Continuous delivery elevates the need for monitoring in production

THE TRUTH

Continuous Delivery relies on having 2 basic things;

1. Version Control

2. Automation

Customers see results and new features more quickly.

Shorter feedback cycles increases our ability to learn.

Improve the whole system.

Reduce firefighting.

Everyone wins!

67

EVERYBODY WINS!

68Photo Credit: Kalexanderson via Compfight cc

69Photo Credit: Walter Benson via Compfight cc