deployment pipeline
TRANSCRIPT
![Page 1: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/1.jpg)
Continuous Delivery Workshop
Workshop materials created by Jez Humble, Martin Fowler, Tom Sulston, & Neal Ford
deployment pipelines
![Page 2: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/2.jpg)
data & infrastructure
tests, synergistic practices, incremental deployment
deployment pipelines
![Page 3: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/3.jpg)
Who are You?
QA
DBA
Developer
Manager
Operations
UX
BA
![Page 4: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/4.jpg)
Agile Transformation
QA
DBA
Developer
Manager
Operations
UX
BA
![Page 5: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/5.jpg)
Continuous Delivery
QA
DBA
Developer
Manager
Operations
UX
BA
![Page 6: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/6.jpg)
No matter how it looks at first, it's always a people problem.
![Page 7: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/7.jpg)
The Dangers of Silos
The Tragedy of the Commons
![Page 8: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/8.jpg)
Release Cadence
![Page 9: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/9.jpg)
Release Cadence
“How long would it take your organization to deploy a change that involves just one single
line of code?”
Mary Poppendieck Tom Poppendieck
“Can you do this on a repeatable, reliable basis?”
devopsy.com/blog/2013/08/16/devops-kata-single-line-of-code/
![Page 10: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/10.jpg)
Continuous Delivery Metricslead time
cycle time
the time between the initiation and completion of a production process.
the total elapsed time to move a unit of work from the beginning to the end of a physical process
![Page 11: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/11.jpg)
Continuous IntegrationIntegration early and often.
Everyone checks into trunk at least once a day.
![Page 12: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/12.jpg)
0
300
600
900
1200
pain
time
Bring the pain forward.
eager vs. late
![Page 13: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/13.jpg)
Continuous IntegrationIntegration early and often.
Everyone checks into trunk at least once a day.
![Page 14: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/14.jpg)
Continuous
IntegrationIntegration early and often.
Everyone checks into trunk at least once a day.
Deployment
Deploy as the final stage of continuous integration.
![Page 15: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/15.jpg)
Continuous
IntegrationIntegration early and often.
Everyone checks into trunk at least once a day.
Deployment
Deploy as the final stage of continuous integration.
DeliverySoftware is always in a deployable state.
![Page 16: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/16.jpg)
Modern software
is complex!https://www.thoughtworks.com/insights/blog/implications-tech-stack-complexity-executives
![Page 17: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/17.jpg)
Agile 101
Iteration 0 1 2 3 4
Analysis + Design
Development
Testing + Showcase
Integration + QA Release and operation
Customer
Centralized QA IT Operations
"Agile" team
The "last mile"
![Page 18: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/18.jpg)
Continuous Delivery
Iteration 0 1 2 3 4
Analysis + Design
Development
Testing + Showcase
Integration + QA Release and operation
Customer
Centralized QA IT Operations
"Agile" team
The "last mile"
Customer
Delivery teamConstant flow of new features into production always
production readybusiness needs > operational constraints
![Page 19: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/19.jpg)
Potential Hindrances
Lead time is too long
Last mile is too painful
Poor collaboration
![Page 20: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/20.jpg)
Identify & remove friction
![Page 21: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/21.jpg)
Continuous Integration
Fast, automated feedback on the correctness of your application every time there is a change to code
![Page 22: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/22.jpg)
Deployment Pipeline
Fast, automated feedback on the production readiness of your application every time there is a change — to code, infrastructure, or configuration
production readiness
code infrastructureconfiguration
![Page 23: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/23.jpg)
Deployment Pipelines
![Page 24: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/24.jpg)
Prerequisites
excellent automated testing at all levels
comprehensive configuration management
continuous integration
![Page 25: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/25.jpg)
commit StageCommit stage
CompileUnit test
AssembleCode analysis
source codecommit testsbuild scripts
deployable binariestest reportsmetadata
Version control
Artifact repository
Run against each check-in
Starts building a release candidate
If it fails, fix it immediately
![Page 26: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/26.jpg)
Pipeline Constructioncommit functional
testuser
acceptance teststaging …
increasing confidence in production readiness
artifactrespository
artifactrespository
artifactrespository
artifactrespository
Pipeline stages = feedback opportunities
![Page 27: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/27.jpg)
UAT StageAcceptance test stage
Configure environmentDeploy and smoke test
Acceptance testTear down
acceptance testsdeployment scriptsconfiguration data test reports
metadata
Version control
Artifact repository
binariesArtifact repository
End-to-end tests in production-like environment
Triggered when upstream stage passes
First DevOps-centric build
![Page 28: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/28.jpg)
Manual Stage
UAT, staging, integration, production, …
Push versus Pull model
Deployments self-serviced through push-button process
Later stages
Configure environmentDeploy and smoke testTear down on request
deployment scriptsconfiguration data
test reportsmetadata
Version control
Artifact repository
binariesArtifact repository
![Page 29: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/29.jpg)
Artifact repository
Source code
Commit stage
CompileCommit tests
AssembleCode analysis
reportsbinariesmetadata
Acceptance stage
Configure environmentDeploy binaries
Smoke testAcceptance tests
Capacity stage
Configure environmentDeploy binaries
Smoke testRun capacity tests
UAT
Configure environmentDeploy binaries
Smoke test
Env & app
config
reportsmetadatabinaries
Production
Configure environmentDeploy binaries
Smoke test
Env & app
config
binariesreportsmetadata
Operationsperform
push-buttonreleases
TestersSelf-servicedeploymentsDevelopers
See code metricsand test failures
Version control
![Page 30: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/30.jpg)
Machinery
www.thoughtworks.com/products/go-continuous-delivery
continuous integration ++
![Page 31: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/31.jpg)
![Page 32: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/32.jpg)
![Page 33: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/33.jpg)
![Page 34: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/34.jpg)
![Page 35: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/35.jpg)
![Page 36: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/36.jpg)
Integration Pipeline in Go CD
![Page 37: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/37.jpg)
Integration Pipeline in Go CD
![Page 38: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/38.jpg)
www.go.cd
![Page 39: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/39.jpg)
Commit stageCompileUnit testAnalysis
Build installers
Acceptance test stage
User acceptance
testing
Performance testing
Production
Increasing confidence in build's production readiness
Environments become more production-like
Faster feedback
![Page 40: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/40.jpg)
Pipeline Anti-patterns
Commit stageCompileUnit testAnalysis
Build installers
Acceptance test stage
User acceptance
testing
Performance testing
Production
Increasing confidence in build's production readiness
Environments become more production-like
Faster feedback
insufficient parallelization
ideal time: < 10 minutes
![Page 41: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/41.jpg)
Pipeline Anti-patterns
Commit stageCompileUnit testAnalysis
Build installers
Acceptance test stage
User acceptance
testing
Performance testing
Production
Increasing confidence in build's production readiness
Environments become more production-like
Faster feedback
insufficient parallelization
Mingle: 3,282 test / 53 computers = ~1 hour
![Page 42: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/42.jpg)
parallelize each stage as much as you can
make your pipeline wide, not long
reduce the number of stages as much as possible
create more stages if necessary to optimize feedback
Insufficient Parallelization Heuristic:
![Page 43: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/43.jpg)
Pipeline Anti-patterns
inflexible workflow
pipeline fans out as soon as it
makes sense to do so
![Page 44: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/44.jpg)
The Scientific Method
hypothesize
experiment
evaluate
![Page 45: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/45.jpg)
manual testing, approvals
automate almost everything
build, deploy, test, release
Principles
automatable high-value, humans only
![Page 46: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/46.jpg)
keep everything you need to build, deploy, test, & release in version control
• database creation, upgrade, downgrade, and initialization scripts
• application stack configuration scripts
• libraries • deployment scripts • tool chains
• requirements documents • test scripts • automated test cases • network configuration
scripts • technical documentation
Principles
![Page 47: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/47.jpg)
When You Hire a New Developer…
![Page 48: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/48.jpg)
Infrastructure Consistency
boxen.github.com
![Page 49: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/49.jpg)
Continuous Delivery Maturity Model
http://paulhammant.com/2013/03/13/facebook-tbd-take-2/
![Page 50: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/50.jpg)
delivery focus characteristics resultA few smart
people performing
heroics
There is an ad hoc release delivery process
• Teams rely mainly on manual testing after development is complete to find defects.
• System integration is painful and happens after development on a
module is completed.• Provisioning production-like
integrated testing environments is expensive and manual.
• Deployment process is manual.• Developers, testers, operations, and management have goals that
bring them into conflict.• Change management is ad hoc or heavyweight and often circumvented
or ignored.
Ad hoc deployments
1: initial
Continuous Delivery Maturity Model
![Page 51: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/51.jpg)
delivery focus characteristics resultTime-boxed
releases (the team sets a release date and manages to it)
There is an adaptive delivery process.• Clear product ownership and chain of
responsibility are in place.• Change management controls are implemented, including a process to detect unauthorized changes with
consequences defined.• Business participates fully and
regularly in development activities and decisions related to delivery.
• There is some automated acceptance testing.
• Production-like testing environments are available for projects early on.
• There is some scripting to reliably and repeatedly configure environments and build packages from version control.
• Teams work in iterations of one month or less and showcase integrated
Planned release: Release time box is
well defined, but duration from idea
inception to production release is greater than business
need.
2: managed
Continuous Delivery Maturity Model
![Page 52: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/52.jpg)
delivery focus characteristics resultRegular releases over a defined
period with interim milestone builds
Teams build quality into release process.
• Teams practice trunk-based development with continuous
integration of all changes.• There are enough automated tests that critical defects are detected and
prevented fast and automatically.• Provisioning of integrated testing
environments is fast and mostly automated.
• No work is considered done until it has passing automated unit and
acceptance tests associated with it.• Testers are not primarily focused on regression testing. Database changes
are versioned and scripted.
Regular release cadence: Release time box is well defined, but
duration from idea inception to
production release is greater than business
need.
3: defined
Continuous Delivery Maturity Model
![Page 53: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/53.jpg)
delivery focus characteristics result
Release on demand
Delivery teams prioritize keeping code trunk deployable over doing
new work.
• Deployment pipeline automatically rejects bad changes from version
control.• Cross-functional end-to-end product- centric teams manage products throughout life cycle.
• Comprehensive automated test suites are created through TDD/
ATDD and maintained by developers and testers working
together.• Teams monitor and manage work in process and deliver work in small
batches.
Release on demand: Software is always in a
releasable state. Release time box is
well defined and equal to, or less than, business need.
4: quantitatively managed
Continuous Delivery Maturity Model
![Page 54: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/54.jpg)
delivery focus characteristics result
Hypothesis-driven
delivery
Teams focus on optimizing cycle time to learn from customers.
• All new requirements describe how the value of the feature will
be measured.• Product teams are responsible
for implementing metrics to gather this data through
techniques such as A/B testing.• Systems are architected with
continuous deployment in mind, supporting patterns such as dark
launching to decouple deployment from release.
• Database changes are decoupled from application deployments.
Continuous deployment
capability enables business
innovation/ experimentation
5: optimizing
Continuous Delivery Maturity Model
![Page 55: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/55.jpg)
Demonstration trumps discussion.
![Page 56: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/56.jpg)
![Page 57: Deployment Pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022051008/58a1da741a28abd1778b4568/html5/thumbnails/57.jpg)
nealford.com
@neal4d