Developer-first workflows (for microservices) on Kubernetes

Richard Li

datawire.io

● Building a cloud application using microservices in 2013

● Distributed systems engineers● Multiple services● Prototyping was really fast

● … then velocity slowed down by a lot when we started getting users and needing to add more features

2

Microservices at Datawire ...

datawire.io

Part 1: How do I gain the benefits of microservices?

(How do I break up my monolith?)(How do I architect my app with microservices?)(What infrastructure do I need in place before I

can benefit from microservices?)

3

datawire.io 4

Stability/Maturity

Velocity

Prototype Production Mission critical

datawire.io 5

Stability/Maturity

Velocity

Prototype Production Mission critical

datawire.io

How do I break up my monolithic architecture?How do I break up my monolithic process into

multiple independent processes?

6

datawire.io 7

Define

Code

Test

Release

Prod

datawire.io 8

Define

Code

Test

Release

Prod

Centralized process

● Specialized teams● Fixed policies (e.g.,

release criteria)

datawire.io 9

Microservices lets you run multiple processes!

datawire.io 10

Industrial Revolution.

datawire.io

Microservices is a distributed development architecture

workflow.

11

datawire.io 12

Stability/Maturity

Velocity

Prototype Production Mission critical

● How do I get to Continuous Deployment incrementally?● How do I limit the scope of PCI (audit process)?● How do I ship feature X as fast as possible?

datawire.io

Microservices is ...

● Multiple workflows○ Including your existing workflow!○ Workflows designed for different

stability/velocity tradeoffs● Simultaneous workflows

13

datawire.io

Doing things this way shifts how people operate!

● Specialists become generalists → design better holistic systems● Dev vs Ops vs QA → roadmap / API conversations● Challenging for engineers → More fun!

14

datawire.io

Part 2: How do I create multiple, simultaneous workflows?

15

datawire.io

Create self-sufficient, autonomous software teams.

16

datawire.io

Why self-sufficiency and autonomy?

● Self-sufficient○ Team does not need to rely on other teams to achieve its goals

● Autonomy○ Team is able to independently make (process) decisions on how to achieve its goals

17

datawire.io

Centralized specialist functions are a common barrier to self-sufficiency and autonomy

18

Centralized architectureCentralized infrastructure / ops*

(You might need a platform team, eventually)

datawire.io 19

datawire.io 20

Monolith

datawire.io 21

Monolith

(soon-to-be) self-sufficient, autonomous team

datawire.io

I’m stranded on API island. Do I have to build my workflow from scratch?

22

datawire.io

No! Kubernetes / Docker / Envoy give you the infra you need

23

datawire.io

Docker

● A consistent build/runtime for your service

● Run locally or in Kubernetes cluster● Massive community & ecosystem

○ 14M Docker hosts○ 3300+ contributors○ Lots of third party tools

24

datawire.io

Kubernetes

● Cloud POSIX● Declarative definition of your cloud

infrastructure

● Supported by all major vendors: Google, Microsoft, Mesos, Docker, Oracle, Red Hat, IBM all support it

● … except for Amazon.○ Yet 62% of K8S workloads on AWS

25

datawire.io

Envoy

● Modern L7 proxy, designed for cloud applications○ Resilience: global rate-limiting, circuit

breakers○ APIs for managing fleets of Envoys○ L7 observability (incl distributed tracing)

● Fast growing ecosystem & community○ Maintained by Lyft○ Google, IBM, Apple, Datawire, Verizon○ CNCF project

26

datawire.io

Cloud Native Compute Foundation

27

● Istio● SPIFFE● App dev

datawire.io

Summary

1. Microservices is a distributed workflow: multiple & simultaneous processes.2. Start building your distributed workflow by creating a self-sufficient, autonomous

team.3. Kubernetes / Docker / Envoy give you the basic operational primitives you need.

28

datawire.io

Part 3: How do I actually use these technologies to build my workflow?

29

datawire.io 30

Monolith

Prototype new feature, FAST

datawire.io 31

Stage 1: Prototyping

Existing workflow: Designed for mature app, not prototyping

Challenge: Prototyping workflow that is fast but not one-off

Strategies:● Dev infra ~= Prod infra● Self-service operational

infrastructure

datawire.io

Strategy: Dev infrastructure ~= Prod infrastructure

32

● End-to-end○ Have developers access the service the same way as end users (e.g., through an

API Gateway, ingress, etc.)

● Environmental consistency○ Insure that your dev runtime is the same as your initial production runtime○ Environmental differences between dev & prod will cause bugs. So eliminate

them upfront.

Your dev infrastructure should resemble your production infrastructure as much as possible.

datawire.io

Strategy: Self-service operational infrastructure

33

Infrastructure needs to be configured by the service team, not by operations (otherwise, there’s a bottleneck.)

Service team needs to be able to publish new services, without operations.

Service team has access to the Kubernetes manifest (maybe they use a standard template).

datawire.io

Implementing Stage 1 strategies

● Eliminate environmental differences○ Containerize your service and use the same container for dev & prod○ Template your Kubernetes manifests so you can use the same manifest for dev & prod

● End-to-end○ Use an API Gateway for dev and prod (you can even use the same one)

● Self-service operational infrastructure○ Expose the right configuration options, in a rational way, for your infrastructure to the service

team■ Kubernetes manifest■ Configuring your API Gateway

34

datawire.io 35

Monolith

Now in production, and need to ITERATE

datawire.io 36

Stage 2: Production users

Existing workflow: App-level integration testing & roll out; dedicated QA team; unit tests

Challenge: Update service quickly, while not letting bugs impact the end user

Strategies:● Software redundancy

datawire.io

Strategy: Multiple versions for software redundancy

37

End user

Primary version

Canary version

Dev version

datawire.io 38

Stage 3 Internal service consumption

Existing workflow: None

Challenge: Communicate to consumers the service contract, and support that contract in the service

Strategy:● Service level objectives● Network-level observability &

resilience

datawire.io

● Define the target level of your service as part of your API● Common SLOs relate to throughput, latency, and availability

○ This API will support up to 5,000 requests per second per consumer○ The expected latency at p50, p90, p99

● Simple implementation of SLOs can be via monitoring

Strategy: Service level objectives

39

datawire.io

Strategy: Network level observability & resilience

● Observability○ Figure out which service is causing the problem○ Need visibility at a network level○ Distributed tracing (e.g., Jaeger, Zipkin)

● Resilience○ Avoid cascade failure○ Network-level rate limiting, circuit breaking, etc.

● Service mesh○ Strategy for implementing network-level observability & resilience that’s transparent to end users○ Istio just proposed to the CNCF

40

datawire.io

Summary

1. Microservices is a distributed workflow.2. Start building your distributed workflow by creating a self-sufficient, autonomous

team.3. Kubernetes / Docker / Envoy give you the basic operational primitives you need. 4. Your workflow requirements depend on your service maturity (prototype,

production, internal dependencies).5. Strategies for building your workflow

a. Prototype: Dev ~= Prod, self-service operational infrastructureb. Production: Software redundancyc. Internal dependencies: Service level objectives, network-level observability/resilience

41

datawire.io

So how does this work in practice?

42

datawire.io

Your traditional monolith workflow ...

● Intended for iterative improvement & prevent downtime○ Code locally in dev environment○ Pull request○ Branch for release○ QA branch & integration test on staging environment○ Release on production environment

● Different environments, specialized teams

43

datawire.io

Versus microservice infrastructure & workflow ...

● Eliminate environmental differences○ Container -- same run time environment○ Kubernetes -- same deployment environment

● Self-service operational infrastructure○ Templated Kubernetes manifest○ Self-service API Gateway

● Multiple versions for software redundancy○ Facilitated by API Gateway / router

44

Will let you prototype / launch / update a microservice with a workflow that’s tailored to feature development.

datawire.io

Thank you!

● richard@datawire.io● Getting ready to launch hands-on tutorials on building your own distributed

workflow. If you’re interested in being notified, sign up here:○ https://d6e.co/montrealk8s

● If you’re interested in any of our open source tools, check them out:○ https://forge.sh for deployment○ https://www.telepresence.io for real-time live coding○ https://www.getambassador.io self-service API Gateway built on Envoy

45

END