getting ci right for sql server

@_AlexYates_#SQLLeeds

Getting CI right for SQL Server

Alex Yates


DLM Consultant

workingwithdevs.com

[email protected]

Alex Yates

@_AlexYates_


@_AlexYates_ | #SQLLeeds


AgileScrumLeanDevOpsIterativeContinuous Delivery

Developers

http://commons.wikimedia.org/wiki/File:Athlete_at_starting_block.jpg


MonitoringDeployment

IntegrityPerformance

DBAs


source:%20http://commons.wikimedia.org/wiki/File:Mocking_Bird_Argument.jpg


Woah! Deployment fail?



#worksOnMyMachine

Woah! Deployment fail?



A cursor?! You just CAN’T do that?!



#worksOnMyMachine

A cursor?! You just CAN’T do that?!



Hey, you just dropped my hot-fix!



#worksOnMyMachine

Hey, you just dropped my hot-fix!



What is CI?


What is CI?

Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

www.thoughtworks.com/continuous-integration

http://www.thoughtworks.com/continuous-integration


Farm Credit Services of America (FCSA)

• 100 person IT team, 14 sub-teams

• Database version control inconsistent

• Deployment process manual

• Delivery was slow and unreliable



• Deployments easier to review• Delivery faster and more

reliable

• Standardised source control and delivery process

• Automated manual deployment tasks


The deployment pipeline


Databases are hard

• Schema changes vs existing data• Reference data vs production data• Teamwork and testing• Database drift (change outside

process, e.g. production hot-fixes)


The deployment pipeline


AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice


Version Control


Version control - LMGTFY




Unit testingDrift

General advice


There’s more than one way to skin a cat


There’s more than one way to skin automatea cat


V1 V2


V1 V2

Migrations-based solutions


V1 V2

State-based solutions


“There's nothing more reliable than keeping track of exactly the scripts you intend to run, and running them, without trying to compare state and guess.”


“There's nothing more reliable than keeping track of exactly the scripts you intend to run, and running them, without trying to compare state and guess.”

Paul Stovell, built Octopus Deploy

http://docs.octopusdeploy.com/display/OD/SQL+Server+databases

http://docs.octopusdeploy.com/display/OD/SQL+Server+databases


“As soon as you have multiple changes on a single aspect of an object, ordering and the ability to detect which change needs to be made gets very complicated.”


“As soon as you have multiple changes on a single aspect of an object, ordering and the ability to detect which change needs to be made gets very complicated.”

Gert Drapers, built DataDude

https://blogs.msdn.microsoft.com/gertd/2009/06/05/declarative-database-development/

https://blogs.msdn.microsoft.com/gertd/2009/06/05/declarative-database-development/


Migrations vs state

http://workingwithdevs.com/delivering-databases-migrations-vs-state/

http://workingwithdevs.com/delivering-databases-migrations-vs-state/


State Easier (less control) Better for

sprocs/functions Better for

large/distributed teams Better for frequent

changes Better for dependency

nightmares Drift: rolled back Better for development

Migrations More control

(harder/needs discipline) Better for data

migrations Better for small teams Better for infrequent

changes Better for simple data

stores Drift: ignored Better for automation

VS


DLMConsultants.com/model-vs-mig


Demo




Unit testingDrift

General advice


Automated builds


VCS Build Test Sync

Upgrade scriptDatabase package

Test results

The automated build process


http://www.jamesshore.com/Blog/Continuous-Integration-on-a-Dollar-a-Day.html




Build servers


Builds/Tests are run on CI agents

Builds/Tests are managed on

CI server

CI server

How do build servers work?


Demo




Unit testingDrift

General advice


Unit testing


Unit testing

www.tsqlt.org

http://www.tsqlt.org/


www.tsqlt.org

Unit testing

http://www.tsqlt.org/


Unit testing


http://workingwithdevs.com/three-odd-attitudes-towards-database-unit-testing-that-make-me-angry/

http://workingwithdevs.com/three-odd-attitudes-towards-database-unit-testing-that-make-me-angry/


Demo




Unit testingDrift

General advice


Drift


“When making changes directly on production, you are making a decision that the delay due to poor cycle time is more expensive than the risk of making a mistake.”


“When making changes directly on production, you are making a decision that the delay due to poor cycle time is more expensive than the risk of making a mistake.”

Alex Yates, built this slide


The bad stuff:

• Accidental roll-backs (state)

• Failed deployments (migrations)

• Environment inconsistency (migrations)


DriftStrategies to help

Improve cycle time Strict security policies Monitor drift (play with DDL

triggers) Redgate DLM Dashboard

(free)https://www.simple-talk.com/sql/database-administration/database-deployment-the-bits-database-version-drift/

https://www.simple-talk.com/sql/database-administration/database-deployment-the-bits-database-version-drift/


DriftStrategies to help

Improve cycle time Strict security policies Monitor drift (play with DDL

triggers) Redgate DLM Dashboard

(free)http://www.red-gate.com/products/dlm/dlm-dashboard/

http://www.red-gate.com/products/dlm/dlm-dashboard/


Demo




Unit testingDrift

General advice


• Don’t do *all the things* at once - start with VCS and drift, then CI, then RM• Don’t forget tests• Teamwork above tools/process• Keep Branching simple… especially if you aren’t using a DVCS• Decouple database deployments from other systems

• Use the right process/tool for your database and team

• Ask me for help!

General advice




Unit testingDrift

General advice


• Standardised VCS, CI and RM processes• FCSA delivering much more efficiently

• FCSA process model for sister organisations

• The team all have excellent CV’s



• Standardised VCS, CI and RM processes• FCSA delivering much more efficiently

• FCSA process model for sister organisations

• The team all have excellent CV’s


http://www.codeaperture.io/2016/09/13/how-redgate-helped-define-our-process/



Siloed sparrows suck at DLM



Heated hippos are closed minded


State Easier (less control) Better for

sprocs/functions Better for

large/distributed teams Better for frequent

changes Better for dependency

nightmares Drift: rolled back Better for development

Migrations More control

(harder/needs discipline) Better for data

migrations Better for small teams Better for infrequent

changes Better for simple data

stores Drift: ignored Better for automation

VS

Clever people consider options


Use appropriate tools


Only hire developers who write tests


Hunt his smug face - make pain on it


This stuff makes a big difference


http://www.red-gate.com/sql-source-control/

http://www.red-gate.com/sql-source-control/


DLMConsultants.com/ci


Questions?

Links: DLMConsultants.com/ciContact me: [email protected]


Image sourcesAuthor Source Information

Chiltepinster Wikimedia Commons Mocking Bird Argument.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. Source on Wikimedia Commons: “Own work”

Bit Boy Flickr The elephant in the room – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.

Nils Rinaldi Flickr Hippo fight 2/3 – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.

My own collection Taken by/property of Alex Yates Kitten, “There’s more than one way to skin a cat!”

Memegenerator.net Memegenerator.net I don’t always edit database. Content designed to be shared and delivered with credit to memegenerator.net.

Ctrl.Alt.Design ctrla.lt Social Media share icons

http://commons.wikimedia.org/wiki/File:Mocking_Bird_Argument.jpg

http://commons.wikimedia.org/wiki/File:Mocking_Bird_Argument.jpg

http://en.wikipedia.org/wiki/en:Creative_Commons

http://creativecommons.org/licenses/by-sa/3.0/deed.en

http://creativecommons.org/licenses/by-sa/3.0/deed.en

https://www.flickr.com/photos/bitboy/

https://www.flickr.com/photos/44124323641@N01/246805948

https://creativecommons.org/licenses/by/2.0/


https://www.flickr.com/photos/nilsrinaldi/

https://www.flickr.com/photos/nilsrinaldi/5158417206/in/photolist-8RQeHG-6q2btE-sa1Pq8-aoA8XG-8RQeGE-g2Jpns-kKuki6-qKVjKK-8RQeKW-oA3fvn-7SzhUB-9xGtsp-pJKvSt-kNUtzR-rmjcY7-e9rWjm-cs3KtE-4UFehR-o2N1kG-7ncr52-s6oCP-rdumpt-6CGxgo-ceMb-dz7Fq8-26AYfc-9jAfuP-owxCHJ-o8nbm1-7p7Nxk-jRQQNa-9Agi8x-4ivCVG-8zmib1-33AMcN-fiRUKK-9UZwUm-dQCLYP-bek6oK-9NqsTq-pvjgYp-qFK1Lq-7QzWQc-6bmRTg-61bqE-bvEySt-bjT3jY-5YPGPg-7ERG57-7ERrg7





http://memegenerator.net/

http://memegenerator.net/instance/59793617

http://ctrla.lt/social-media-icons-modern-metro-download/

http://ctrla.lt/social-media-icons-modern-metro-download/

getting ci right for sql server

Technology