matthew skelton - how to choose tools for devops - collaboration over automation
DESCRIPTION
skeltonthatcher.com | @matthewpskelton Title: How to choose tools for DevOps - collaboration over automation Matthew Skelton has been building, deploying, and operating commercial software systems since 1998. Co-founder and Principal Consultant at Skelton Thatcher Consulting Ltd, he specialises in helping organisations to adopt and sustain good practices for building and operating software systems: Continuous Delivery, DevOps, aspects of ITIL, and software operability. Matthew founded and leads the 700-member London Continuous Delivery meetup group (http://londoncd.org.uk/), and instigated the first conference in Europe dedicated to Continuous Delivery, PIPELINE Conference (http://pipelineconf.info/). He also co-facilitates the popular Experience DevOps workshop series (http://experiencedevops.org/). With an ever-increasing array of tools and technologies claiming to 'enable DevOps', how do we know which tools to try or to choose? In-house, open source, or commercial? Ruby or shell? Dedicated or plugins? It transpires that highly collaborative practices such as DevOps and Continuous Delivery require new ways of assessing tools and technologies in order to avoid creating new silos. Matthew Skelton shares his recent experience of helping many different organisations to evaluate and select tools to facilitate DevOps; the recommendations may surprise you.TRANSCRIPT
![Page 1: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/1.jpg)
How to choose tools for DevOps and
Continuous DeliveryMatthew Skelton, Skelton Thatcher Consulting
DevOps Exchange meetup group, 24 July 2014, London, UK
#doxlon
![Page 2: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/2.jpg)
Collaboration
Learning
Singleton tools
Conway’s Law
![Page 3: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/3.jpg)
Matthew Skelton
•15 years building & operating software systems
•Cybernetics + Neuroscience• control engineering
• psychology
• ‘network’ interactions
@matthewpskelton
![Page 4: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/4.jpg)
PIPELINE Conference
London Continuous Delivery meetupgroup (#londoncd)
![Page 5: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/5.jpg)
Help orgs to adopt and sustain good engineering practices
Interim CTO/Head of X, tech strategy, architecture, workshops, delivery
![Page 6: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/6.jpg)
Recent clients
•Tourism
•Betting & gambling
•Travel booking
•Financial data & reporting
•Healthcare
![Page 7: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/7.jpg)
Common themes•Online since ~1999
•Successful in their sector
•Large, central core database(s)
•Non-aligned goals
•Need to adopt DevOps and Continuous Delivery
![Page 8: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/8.jpg)
Common needs
•Technology selection ticklists
•Team interactions
•Collaboration opportunities
•Tools as catalysts
![Page 9: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/9.jpg)
![Page 10: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/10.jpg)
Continuous Delivery
•A scientific approach to changing software systems (Dave Farley)
•Regular, rapid, repeatable, reliable changes
![Page 11: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/11.jpg)
Continuous Delivery
•Quality
•Safety
•Reliability
•Psychology
•Effectiveness
![Page 12: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/12.jpg)
![Page 13: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/13.jpg)
DevOps“Highly effective, daily collaboration between software developers and IT
operations people to produce relevant, working systems” *
*also QA/Testing, IT Service Desk, Programme Management,
Commercial, Marketing, etc.
![Page 14: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/14.jpg)
Not DevOps
“Automation”
“Build & Release”
“Infrastructure Development”
“System Administration”
![Page 15: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/15.jpg)
![Page 16: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/16.jpg)
Collaboration
![Page 17: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/17.jpg)
1. Version Control
![Page 18: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/18.jpg)
![Page 19: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/19.jpg)
![Page 20: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/20.jpg)
![Page 21: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/21.jpg)
![Page 22: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/22.jpg)
![Page 23: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/23.jpg)
![Page 24: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/24.jpg)
2. Deployment Pipeline
![Page 25: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/25.jpg)
![Page 26: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/26.jpg)
![Page 27: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/27.jpg)
![Page 28: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/28.jpg)
Challenges were:
•Limited Git skills in Service team
•Manual deployments
•‘Snowflake’ servers
•No CI
•Risks: security, automation, etc
![Page 29: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/29.jpg)
http://bit.ly/thetrainline-weekly-deploy
![Page 30: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/30.jpg)
What we did•Built a walking skeleton pipeline
•Modelled security roles and stages
• Included manual steps (at first)
•Walked people through steps
•Finally: opened firewall so everyone could see the UI
![Page 31: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/31.jpg)
Results
•Security: happy
•Service team: relieved (& happy)
•Developers: won over (& happy)
•Business: surprised (& happy)
![Page 32: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/32.jpg)
3. Log Aggregation
![Page 33: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/33.jpg)
LogStash
•ElasticSearch+ LogStash+ Kibana•ELK
•In Production, Pre-Prod, Test
•On developer machines!!!
![Page 34: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/34.jpg)
http://rashidkpc.github.io/Kibana/images/screenshots/searchss.png
![Page 35: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/35.jpg)
Collaboration & tool choice
Value collaboration as a key criterion
Orthogonal to main purpose (?)
“How does [the use of] this tool help people to collaborate?”
![Page 36: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/36.jpg)
Learning
![Page 37: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/37.jpg)
![Page 38: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/38.jpg)
![Page 39: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/39.jpg)
![Page 40: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/40.jpg)
![Page 41: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/41.jpg)
Learning & tool choice
Bring people with you
Appreciate current skills
Prefer achievable gains now
Avoid fear of too-scary tools
![Page 42: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/42.jpg)
Singleton tools(or the ‘Prize Bull’ approach)
![Page 43: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/43.jpg)
![Page 44: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/44.jpg)
Singleton tools
•Special database server
•Costly log aggregation
•Costly monitoring
•Server configuration
![Page 45: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/45.jpg)
http://www.walpapershddownload.com/highland-cattle-wallpapers/
![Page 46: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/46.jpg)
“Better features”?
Optimise globally across the teams that need to collaborate
![Page 47: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/47.jpg)
Singleton tool
Breaks feedback (learning) loop from Production
Makes CI/CD more difficult
Underestimates value of collaboration and learning
![Page 48: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/48.jpg)
Conway’s Law
![Page 49: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/49.jpg)
Mel Conway, 1968
“organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations”
http://www.melconway.com/Home/Conways_Law.html
![Page 50: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/50.jpg)
![Page 51: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/51.jpg)
DevOps Topologies
![Page 52: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/52.jpg)
DevOps Topologies
![Page 53: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/53.jpg)
DevOps Topologies
![Page 54: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/54.jpg)
DevOps Topologies
![Page 55: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/55.jpg)
Conway & Tool Choice
See the organisation as a system
Separate tools for separate teams
Shared tools for collaborative teams
http://bit.ly/DevOpsTopologies
![Page 56: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/56.jpg)
Conway’s Law•Allan Kelly - @allankellynet
•https://vimeo.com/channels/londoncd
![Page 57: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/57.jpg)
![Page 58: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/58.jpg)
How to choose tools for DevOps
Value collaboration aspects
Avoid a learning mountain: evolve tooling
Avoid Production-only tools
Consider Conway’s Law
(this list is incomplete!)
![Page 59: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/59.jpg)
Further readingHighOps Operability eBook
bit.ly/OperabilityEbook
Build Quality In
buildqualityin.com
![Page 60: Matthew Skelton - How to choose tools for DevOps - collaboration over automation](https://reader031.vdocuments.mx/reader031/viewer/2022020110/54c6bb3c4a795932518b45b6/html5/thumbnails/60.jpg)
Thank youmatthewskelton.net / @matthewpskelton
skeltonthatcher.com
HT: @Squire_Matt, @alan_parkinson