thirty months of microservices. stairway to heaven or highway to hell? - sander hoogendoorn -...
TRANSCRIPT
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 1
@aahoogendoorn | www.ditisagile.nl
Thirty months of microservices
Sander Hoogendoornditisagile.nl
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 2
Sander Hoogendoorn
MeDad, software architect, agile coach, programmer, trainer, speaker, writerBooks, articles, conferences, courses
WorkOwner ditisagile.nlFormer CTO insurance companyFormer global agile thoughtleader Capgemini
Webwww.sanderhoogendoorn.comaahoogendoornaahoogendoorn [email protected]
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 3
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 4
As a project managerI would like to demo untested code so I embarrass myself
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 5
@aahoogendoorn | www.ditisagile.nl
Monoliths Hard to deliver, even harder to test and impossible to maintain
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 8
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 9
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 10
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 11
Dependencies will kill youA typical systems landscape
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 12
@aahoogendoorn | www.ditisagile.nl
A brief history of components and services
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 13
Client server
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 14
Component based development
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 15
Service oriented architecture
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 16
Microservices
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 17
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 18
@aahoogendoorn | www.ditisagile.nl
MicroservicesBeyond the hype?
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 19
MicroservicesBeyond the hype?
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 20
Gartner hype cycle
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 21
@aahoogendoorn | www.ditisagile.nl
MicroservicesThe clear benefits
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 22
But first … a definition
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 23
In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.
These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.
Martin Fowler
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 24
In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.
These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized managementof these services, which may be written in different programming languages and use different data storage technologies.
Martin Fowler
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 25
In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.
These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.
Martin Fowler
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 34
Opinions, opinions, opinions
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 35
Oh and by the way…Doing microservices means that you are building a distributed system
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 36
First law of distributed objectsDo not distribute your objects!
Martin Fowler
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 37
@aahoogendoorn | www.ditisagile.nl
Are microservicesa stairway to heaven?
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 38
@aahoogendoorn | www.ditisagile.nl
Or are they a highway to hell?
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 39
@aahoogendoorn | www.ditisagile.nl
From the trenchesMicroservices in real life
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 40
Where do we come from?
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 41
Where do we come from?
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 42
Where do we come from?
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 43
Outsourcing didn’t work
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 44
Where are we going to?
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 48
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 49
For the things we have to learn before we can do them, we learn by doing them
Aristotle
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 51
@aahoogendoorn | www.ditisagile.nl
So what did we learn?
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 52
Microservices require an evolutionary software architecture
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 53
@aahoogendoorn | www.ditisagile.nl
Start with some guiding principles
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 54
@aahoogendoorn | www.ditisagile.nl
Business processes first
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 57
Client thinks in business processes, so we implement business processesWe move away from the mainframe, to a new systems landscape, consisting of micro-applications and micro-componentsRequirements and documentation are modeled rather than writtenApplications implement a single elementary business processApplications and components all have their own bounded contextApplications do not have storage, and only talk to other applications and componentsComponents have their own storage (database), and only talk to other componentsCommunication between applications and components uses a simple open protocol – JSON on REST Avoid transactions as much as possible
Some guiding principles
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 60
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 61
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 69
@aahoogendoorn | www.ditisagile.nl
Designing microservicesModular design and bounded contexts
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 70
Doing big up-front design is dumb, doing no design up-front is even dumber
Dave Thomas
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 73
Single responsibility principleGroup together things that change togetherSeparate things that change for different reason
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 75
Bounded contextWhen you model larger domains, it becomes progressively harder to create this single unified model. Instead of creating a single unified model, you create several, all valid within their bounded context
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 76
The single unified domain modelOr more often the humongous data model
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 77
Bounded contexts
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 78
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 83
@aahoogendoorn | www.ditisagile.nl
RESTfulnessis not as easy as it seems
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 84
Some examples
Getlocalhost:8080/countries
Getlocalhost:8080/countries/42
localhost:8080/countries/GRC
localhost:8080/countries/isocode/GRC
localhost:8080/countries?isocode=“GRC”
Post or put? localhost:8080/countries
localhost:8080/countries/38
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 90
HTTP return codes cheat sheet
1**. Hold on 2**. Here you go
3**. Go away 4**. You fucked up
5**. I fucked up
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 91
Be conservative in what you send, be liberal in what you accept
Postel’s Law
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 93
@aahoogendoorn | www.ditisagile.nl
Testing microservicesFailing fast
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 94
Fail fast, fail often, fail forward
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 95
A (typical) service development lifecycle
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 96
What to test?
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 97
Even though you might have really brilliant testers…
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 99
User acceptance testingMinimize to the max
ZZZZZ
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 102
@aahoogendoorn | www.ditisagile.nl
Deploying microservicesContinuous integrationand build pipelines
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 104
A typical build pipelineIn different (virtual) environments
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 105
Build pipelines in Jenkins
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 106
Microservices Building multiple deployment pipelines
CodeDeveloper
TestTest
Acceptance Test
Acceptance Live
CodeDeveloper
TestTest
Acceptance Test
Acceptance Live
CodeDeveloper
TestTest
Acceptance Test
Acceptance Live
CodeDeveloper
TestTest
Acceptance Test
Acceptance Live
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 107
No one tool to rule them allWe tried Jenkins, Bamboo, Team City, GoCD and then Jenkins again…
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 108
@aahoogendoorn | www.ditisagile.nl
Some tooling we useIntelliJSonarQubeJenkinsGit & GitlabInfrastructure as code
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 109
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 110
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 111
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 112
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 113
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 117
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 118
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 119
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 120
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 122
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 123
DevOps is not easyIt feels like showing up at the wrong re-enactment
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 124
Infrastructure as code (often Groovy)And guess what … we know how to do that
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 125
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 126
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 127
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 140
@aahoogendoorn | www.ditisagile.nl
In retrospectiveSome final thoughts
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 141
Microservices are not for everyone (yet)How do we benefit from implementing microservices?
What are we going to benefit from the most?Polyglot
persistence?
Scalability?
SmallIncrements?
Enforcemodular design?
Delivering more often?
Bounded Contexts?
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 142
Will it work?
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 143
The hockey stick model
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 146
@aahoogendoorn | www.ditisagile.nl
Allow your teams to learn continuously…
@aahoogendoorn | www.ditisagile.nlDesigning, developing, testing and deploying microservices 147
@aahoogendoorn | www.ditisagile.nl
And never forgetto have fun
@aahoogendoorn | www.ditisagile.nlThirty months of microservices. Stairway to heaven or highway to hell? 161
@aahoogendoorn | www.ditisagile.nl
References and questions
www.sanderhoogendoorn.com
www.ditisagile.nl
aahoogendoorn
aahoogendoorn