an illustrated guide to microservices (boston python meetup - aug 2016)
TRANSCRIPT
An Illustrated Guide to Microservices
Rafael SchlomingCo-Founder & Chief Architect
datawire.io
This is a Python talk… really...
1. Datawire is a python shop2. We like python so much, we use it to write code in other languages (yes, we have
a compiler written in python)3. Python & Microservices complement each other really well4. We will write an actual microservice in python, just hold out for it
2
datawire.io 3
What’s a Microservice?
datawire.io
Common Microservice Definitions
It’s a service that is...
● Small● Self contained● Narrow in scope● Bounded context● Independent● Loosely coupled
Correct, but incomplete...
4
datawire.io
Common Microservice Definitions (as applied to Legos)
A piece of plastic that is…
● Small● Self contained● Narrow in scope● Bounded context● Independent● Loosely coupled
What’s missing…?
5
datawire.io
Combine to build AWESOME!
6
datawire.io
What do you get when you combine Microservices?
Topologies!
7
datawire.io 8
Death Star Topology
datawire.io
Wait, why do I want a death star?
Maybe you don’t, but your CXO does
9
Organizational Scale
AgilityVelocity
datawire.io
Topologies
10
datawire.io
Topologies have always been a thing
● Topologies aren’t new, they have always been the bread and butter of distributed systems design
● In fact we’ve all been stuck living in the same topology for a while now
11
datawire.io
Topologies are powerful
Information flows through topologies like water through a pipe or current on a wire
Probably the most important factor in the performance of a distributed system
12
datawire.io
Topologies are the purview of the Elite
Architects define the topology
Developers get to plug in business logic in a few predefined places
13
datawire.io
Microservices bring topology to Developers
Developers define the topologyBusiness logic is distributed rather than being centralized
The topology changes and grows much more quickly
14
datawire.io
Topologies can solve a lot of problems
IntegrationOperational Scale
Reliability
15
datawire.io
Operational Scale (Linear Topology)
16
Ingest Source of Truth Transform Present
Template for many data driven businesses…
datawire.io
Operational Scale (Linear Topology)
17
Ingest Source of Truth Transform Present
Template for many data driven businesses…
Network Disk Disk + CPU Network + CPU
datawire.io
Scale Components Independently
18
Ingest Source of Truth Transform Present
Template for many data driven businesses…
Network Disk Disk + CPU Network + CPU
datawire.io
Integration (Fan Out Topology)
19
datawire.io 20
Reliability
Software Bug
Centralized Business Logic can be a Single
Point of Failure
Operational Failure
datawire.io 21
Reliability
Software Bug
Decentralize Business Logic to Isolate Failures
datawire.io 22
Organizational Scale
Software Bug
Decentralized Business Logic Means● Multiple Teams● Smaller Codebases● Ramp up Developers Faster
datawire.io
Can topologies give me a death star?
They help, but not quite… you need to build topologies quickly!
23
Organizational Scale
AgilityVelocity
datawire.io
Microservices vs SOA(Service Oriented Architecture)
24
datawire.io
Common Explanations
Both are topologies of services
● Fine grained SOA● Integration pattern vs application components● Organizational factors...
We need a better definition…
● These explanations ignore the process that yields the topology
25
datawire.io
Architecture and Development are different processes
Both can yield topologies of services, but SOA won’t build your death star. For that you need agile topologies
Microservices: Service Oriented Development
Architecture:
● Lots of up front thinking● Slow iteration cycle● Months/years until you get feedback
Development:
● Experimental● Rapid iteration cycle● Minutes/hours until you get feedback
26
datawire.io
You need a new Mindset and new Tooling
So, how do I build Agile Topologies?
27
datawire.io
Mindset
28
datawire.io
What’s the new Mindset?
The role of architecture changes
● An architect’s job is to enable developers, not confine them● Architecture teams -> developer infrastructure, developer happiness, …
29
Up Front Thinking Experimentation
datawire.io
Tooling
30
datawire.io
What tools enable experimentation & rapid iteration?
You need three things when you decentralize your business logic:
1. A way to make your topology resilient to software bugs○ Where does my catchall go?○ Minimize impact of bugs reaching production○ Graceful degradation instead of catastrophic failure
2. A robust tool for deployment pipelines that enables both service and system level testing○ Where do your integration tests go?
3. Good visibility into the state of the running system○ Where does my printf go?
31
datawire.io
How do you build these tools?
It’s all about how you manage your topology
● Transition from DNS -> Discovery● Central load balancers -> Smart Endpoints
32
datawire.io
Routing table is (relatively) static
Routing policy is global
Traditional Topology Management
33
Client
DNS
Load Balancer
Serverresolve
traffic
datawire.io
Routing table is updated in realtime
Routing policy is local
Microservices Topology Management
34
Client
DiscoveryServer
heartbeatsroutes
Smart Endpoint
datawire.io
Demo
35
datawire.io
Summary
Useful definitions:
● Microservice: A node in an Agile Service Topology● Microservices: Service Oriented Development
How to build a death star (trifecta of velocity, agility, and organizational scale):
● Mindset: architecture -> experimentation● Tooling: Discovery + Smart Endpoints
With these tools you can get started in half an hour instead of spending six months rearchitecting
36
datawire.io
To learn more
Contact me:
Jobs
● https://www.datawire.io/careers/
Try
● https://github.com/datawire/mdk● https://www.datawire.io
37