underestimated costs of microservice architectures · • well known api contracts / specifications...

8th Light, Inc.

Colin Jones @trptcolin https://8thlight.com

Underestimated costs of microservice architectures

Microservices

🎉 🦄 🌈 Happy! 🌈🦄 🎉

🙀☠ 😨 Sad 😨 ☠🙀

Avoid microservices?

Avoid microservices?

On other hand

But to gain any benefit from microservice thinking, you have to understand what it

is, how to do it, and why you should usually do something else.

- Martin Fowler

Hype Cycle

Accusations

We underestimate costs

Benefits

independent deployability

independent scalability

independent deployability

fault tolerance

independent deployability

independent scalability

avoid dependency hell

independent deployability

independent scalability

fault tolerance

architectural boundaries

independent deployability

independent scalability

fault tolerance

avoid dependency hell

small team ownership

independent deployability

independent scalability

fault tolerance

avoid dependency hell

architectural boundaries

eliminate legacy code

independent deployability

independent scalability

fault tolerance

avoid dependency hell

architectural boundaries

small team ownership

eliminate legacy code

independent deployability

independent scalability

fault tolerance

avoid dependency hell

architectural boundaries

small team ownership

🎉 🎉

🎉

🎉microservices!

Costs & Mitigations

Well-understood costs

Latency

LatencyMitigations

• Cache responses

• Batch calls together

• Coarse-grained service API

Additional infrastructure

Latency

Additional infrastructureMitigations

• Containers

• Infrastructure automation & configuration management

• Virtual machines / cloud

• Auto-scaling (metered cost)

• Serverless

Understanding != Paying

Underestimated Costs

Data consistency

Additional infrastructure

Latency

Data consistency

Orders Service

Main App

Main DB

Data consistency

Orders Service

Main App

Orders DBMain DB

Data consistencyMitigations

• Design for eventual consistency

• Canonical source for data (aka “system of record” / “source of truth”) and derived data

• Backend sync processes

• Service teams co-own ETL for analytics/business intelligence / data warehouse

Data consistency

Failure modes

Data consistency

Additional infrastructure

Latency

Failure modes

A B

Failure modes

A B

???

Failure modes

A B

Failure modes

A B

Failure modes

A B

???

Failure modesMitigations

• Use retries (with backoff; cap the max time)

• Read the remote end to see if it succeeded

• Use fallbacks for read timeouts

• Use circuit breaker to limit cascading failures

• Use bulkheads to protect independent modules

Failure modes

Development & testing

Failure modes

Data consistency

Additional infrastructure

Latency

Development & testing

from "Testing Microservice the Sane Way" by Cindy Sridharan

Development & testing

Mitigations

• Expand testing mindset to staging/production observability efforts

• Integrate only a few services / API checking and rely more on unit tests

• Unify on lightweight runtimes and single database technology [only delays the problem; conflicts with team ownership]

• Test against external environments with services set up [risks test pollution]

• Orchestrate new isolated infrastructure for each test run

Development & testing

Observability

Failure modes

Development & testing

Data consistency

Additional infrastructure

Latency

Observability

ObservabilityMitigations

• Log aggregation with correlation IDs

• Error reporting / alerting [generally on symptoms, not causes]

• Distributed tracing tools

• Monitoring tools / dashboards

• Fancier 3rd-party observability tools

Tunnel vision

Failure modes

Development & testing

Observability

Data consistency

Additional infrastructure

Latency

Tunnel vision

Mitigations

• Measure business metrics, not team velocity

• Make sure team / service incentives are aligned with the company’s

• Rotations / team exchanges / dynamic re-teaming

• Cross-org communication

Tunnel vision

Implicit connection data

Failure modes

Development & testing

Observability

Tunnel vision

Data consistency

Additional infrastructure

Latency

Implicit connection data

Mitigations

• Well known API contracts / specifications (e.g. JSON Schema, Swagger, Protocol Buffers, Thrift)

• Centralized/standardized repository to track service metadata for service discovery

• Put all services in one codebase (monorepo) for easier searchability

• Custom tooling based on log aggregation / monitoring

Implicit connection data

Inter-team priority conflicts

Failure modes

Development & testing

Observability

Tunnel vision

Implicit connection dataData consistency

Additional infrastructure

Latency

Inter-team priority conflicts

ThemUs

Inter-team priority conflicts

Them

Consumer BUs (Consumer A) Consumer C Consumer C

Mitigations

• Make our case really well to the service team, or management

• Add staff on heavily-used microservices

• Split heavily-used services further

• Contribute to their project (aka “internal open-source”)

• Rebuild a similar service with the changes we need

Inter-team priority conflicts

Hard to change across boundaries

Failure modes

Development & testing

Observability

Tunnel vision

Inter-team priority conflicts

Implicit connection dataData consistency

Additional infrastructure

Latency

Hard to change across boundaries

Hard to change across boundaries

Mitigations

• Be deliberate about the choice to Extract Microservice

• Version your API? [controversial]

• If versioning / breaking: Have a well-defined way to communicate breaking changes / deadlines

• Sticking with the same runtime (e.g. JVM) makes Inline Microservice possible

• Cross-org communication

Hard to change across boundaries

Mitigations

• Skill and culture of backwards compatibility (SemVer, Postel’s Law)

• Don’t make breaking changes

• Well known API contracts / specifications (e.g. JSON Schema, Swagger, Protocol Buffers, Thrift)

• Consumer-driven contract tests in CI

• See also “Connection data is implicit” mitigations

Hard to change across boundaries

Failure modes

Development & testing

Observability

Tunnel vision

Inter-team priority conflicts

Implicit connection dataData consistency

Additional infrastructure

Latency

💵 💵

🎉

🎉

Hard to change across boundaries

microservices!

Alternatives

Milliservices? Centiservices?

Modules / encapsulation

Modules / encapsulation

Modules / encapsulation

Modules / encapsulation

Recommendations

List problems. Then solutions.

Then pros and cons.

Don’t believe the hype.

Be ready to pay the costs.

Make sure you’re getting the benefits.

Make the change easy. Then make the change.

Make the change easy. Then make the change.

(maybe)

Learn more

Learn more

Learn more•Ben Christensen. “Don’t Build a Distributed Monolith”: https://

www.microservices.com/talks/dont-build-a-distributed-monolith/

•Michael Feathers. “Microservices and the Failure of Encapsulation”: https://michaelfeathers.silvrback.com/microservices-and-the-failure-of-encapsulaton

•Michael Feathers. Working Effectively with Legacy Code: https://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052

•Martin Fowler. Enterprise Application Architecture: https://www.martinfowler.com/books/eaa.html

•Martin Fowler. “MicroservicePremium”: https://www.martinfowler.com/bliki/MicroservicePremium.html

•Martin Fowler. “Microservice Prerequisites”: https://www.martinfowler.com/bliki/MicroservicePrerequisites.html

•Martin Fowler. “Microservices Resource Guide”: https://www.martinfowler.com/microservices/

Learn more•Susan Fowler. Production-Ready Microservices: http://shop.oreilly.com/product/

0636920053675.do

•John Gall. The Systems Bible: https://www.amazon.com/Systems-Bible-Beginners-Guide-Large/dp/0961825170

•David Heinemeier Hansson. “The Majestic Monolith”: https://m.signalvnoise.com/the-majestic-monolith-29166d022228

•Rich Hickey. “Hammock Driven Development”: https://www.youtube.com/watch?v=f84n5oFoZBc

•Gregor Hohpe and Bobby Woolf. Enterprise Integration Patterns: http://www.enterpriseintegrationpatterns.com/

•Mike Knepper. “The Hidden Costs of Leaving a Monolith”: https://8thlight.com/blog/mike-knepper/2016/01/20/hidden-costs-of-leaving-a-monolith.html

•Dan Manges. “The Modular Monolith: Rails Architecture”: https://medium.com/@dan_manges/the-modular-monolith-rails-architecture-fb1023826fc4

•Sam Newman. Building Microservices: https://samnewman.io/books/building_microservices/

Learn more•Michael Nygard. “The Entity Microservice Antipattern”: http://

www.michaelnygard.com/blog/2017/12/the-entity-service-antipattern/

•Michael Nygard. Release It!, 2nd edition: https://pragprog.com/book/mnee2/release-it-second-edition

•Ozan Onay. “You are not Google”: https://blog.bradfieldcs.com/you-are-not-google-84912cf44afb

•Arnon Rotem-Gal-Oz. “Fallacies of Distributed Computing Explained”: http://www.rgoarchitects.com/Files/fallacies.pdf

•Cindy Sridharan. “Testing Microservices, the Sane Way”: https://medium.com/@copyconstruct/testing-microservices-the-sane-way-9bb31d158c16

•Cindy Sridharan. “Testing in Production, the safe way”: https://medium.com/@copyconstruct/testing-in-production-the-safe-way-18ca102d0ef1

•Jim Waldo, Geoff Wyant, Ann Wolrath, and Sam Kendall. “A Note on Distributed Computing”: http://web.cs.wpi.edu/~cs3013/a11/Papers/Waldo_NoteOnDistributedComputing.pdf

8th Light, Inc.

Colin Jones @trptcolin https://8thlight.com

Thank you!

•!87

Steven Mason steven@synerzip.com

469.374.0500

Your trusted partner for Agile software product development.

Synerzip

• Accelerate the delivery of your product roadmap • Address technology skill gaps • Save at least 50% with India software development • Augment your team with optional on-site professionals

Synerzip Clients

©SYNERZIP2017Corporate Overview !90

Connect with Synerzip

@Synerzip

linkedin.com/company/synerzip

facebook.com/Synerzip

Next Webinar

Presented by

Damon Poole Agile Coach and Author

Scaling Agile Organically Tuesday, September 18, 2018 at Noon CT

TEXAS | SILICON VALLEY | NYC | INDIA