devops, microservices and platforms, oh my!Andrew Clay ShaferSenior Director of Techology
systems thinking is the new black
a stone cutter’s quest for nice things
Andrew Clay Shafer
Andrew Clay Shafer
@littleidea
Three Stone Cutters walk into a Pareto Inefficient Nash Equilibrium
start with the conclusion
devops, platforms, continuous delivery, microservices…
These things are all one…
In the beginning…
not the beginning beginning
I got a job as a developer…
I had no idea what I was doing…
I was left to my own devices…
• no experience • no mentors • no testing • no backups • no version control • no automation
¯\_(ツ)_/¯
a boy, a server and force of will
next stop… grad school
Computational Science
• model bioelectric fields• solve for measurements• visualize• rinse and repeat
learned a few things
• technical debt•Conway’s Law• subversion• little to no testing
Venture Funded Solutionsin search of problems
Linux ‘server’ in your Pocket
• custom board• 400 MHz PowerPC• 64 MB SDR SDRAM• 512 MB NAND FLASH• USB Powered• biometric auth
Big Vision
• Identity Solution• Policy Based Provisioning• Citrix Desktop• On demand ‘deb’ package• Poison Pill• Mostly Worked
Learned More Things
• Formative• Learned to Computer• Introduced to Agile™• Testing• Project Management• Lord Conway Reigns• Disassociated Identity
The next chapter
• eCommerce 2.0• J2EE• DIY Square Wheels• Outsourced Ops• Brittle• Expensive
very clever solutions
Automation…
• write a script for common tasks
• share the scripts around office
• modify the scripts for new task
• hope things works
Typical Day at Work
the mission begins
to change the relationship between people and computers
actually, needed to change the relationship between people and people
Amazon, a bookstore in Seattle, deploys code to production every 11 seconds…
–Werner Vogels, CTO Amazon
“The traditional model is that you take your software to the wall that separates development and operations, and throw it over
and then forget about it. Not at Amazon. You build it, you run it. This brings developers into contact with the day-to-day
operation of their software. It also brings them into day-to-day contact with the customer. This customer feedback loop is
essential for improving the quality of the service.”
everybody looks at Amazon and rushes to copy features
but the features are superficial
the Amazon advantage is the process and culture
Operations is the secret sauce configuration
Principles > Practices >Tools
let’s rewind that
Software in the beginning
• shipped on physical media
• hard to change after release
• runs on other peoples computers
• have to worry about bugs
• process doesn’t run very long
• no worries about uptime
Software Developer
changes ideas into code
The Process
• have a good idea!
• request a server
• get a purchase order from supervisor
• wait
• wait
• server arrives
• server gets power and network
• server gets operating system
• start to configure for deploy
Traditional IT
The System Admin
• keeps systems running
• don’t care about your application
• not paid to care
• other people waiting for their servers
• might have to worry about many other servers
• and email
• and printers
The shift to services
• the internet changes everything
• runs on other your computers
• can change your computers
• still have to worry about bugs
• process run a long time
• uptime is everything
bigger, faster, stronger
But we are an enterprise, we are different, we cannot do this, we have our ways.
service management? ITIL
seemed like a good idea
slow everything down
Is moving slow an advantage?
are misaligned incentives an advantage?
not how the web was built
competitivedisadvantage
The Stories We Tell
configuration management for the masses
The Process
• have a good idea!
• request a server
• get a server in minutes
• run configuration tools
• everything is up in minutes
Principles > Practices >Tools
Adrian Cockroft - ex-Netflix
What I learned from my time at Netflix.
Netflix Lessons
• Speed wins in the marketplace
• Remove friction from product development
• High trust, low process, no hand-offs between teams
• Freedom and responsibility culture
• Don’t do your own undifferentiated heavy lifting
• use simple patterns automated by tooling
• self service cloud makes impossible things instant
But we are an enterprise, we do not have the talent to do this.
“But Netflix has a superstar development team, we don’t”
“Netflix hired them from you, and got out of their way.”
Time
Risk
the impact of batch size
both faster and safer
–W Edwards Deming
“It is not necessary to change. Survival is not mandatory.”
Netflix built a platform to enable self service deployment
Netflix built a platform to deploy and operate microservices
Netflix built a platform to continuously deliver software
Netflix built a platform that could protect itself from failure
What Netflix did not do is build a platform for general ad-hoc automation…
A platform makes promises.
Constraints are the contract that allows the platform to keep promises.
Principles > Practices >Tools
devops refers to the practices and tools that emerge from high performing organizations
continuous delivery results as a consequence of devops practice
continuous delivery is not possible with gating and handoffs
continuous delivery is untenable, if the fixed cost of deployment is high
microservices describes the first post-cloud, post-devops, post- continuous delivery architecture
microservices leverage baseline operational capabilities and team dynamics to isolate failure and maximize velocity
continuously delivered microservices are the natural evolution for services that need to run at scale and be changed frequently
Amazon, Google, Netflix, Flickr, Etsy, WealthFront, Lyft, Uber, Shopzilla, Pied Piper, all built one off platforms to do this
continous delivery - why devops - how microservices - what
the solution: a platform
Do you want to build one?
maybe we can build one together
The Process
• have a good idea!
• push code to platform
• code is running in seconds
• self service
• self healing
• and they all lived happily ever after
no one originally set out to do devops, continuous delivery, microservices, or platforms these were natural consequences
don’t fixate on the words, fixate on the outcomes
This is not the end…
mobile, internet of things, big dataeverything is getting bigger and faster