microservice architecture

Microservice Architecture

Viraj Brian Wijesuriya

[email protected]

1

SCS 4120 - Software Engineering IV

BACHELOR OF SCIENCE HONOURS IN COMPUTER SCIENCEBACHELOR OF SCIENCE HONOURS IN SOFTWARE ENGINEERING

All in One Place Lecture NotesDistribution Among Friends Only

All copyrights belong to their respective owners

• Definition: – In short, the microservice architectural style is an approach to developing a

single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies [martinfowler.com]

• On the logical level, microservice architectures are defined by a– functional system decomposition into manageable and independently

deployable components

SEIV – Microservices – Introduction

2

What is Microservices Architecture?

• 2011: First discussions using this term at a software architecture workshop near Venice

• May 2012: microservices settled as the most appropriate term

• March 2012: “Java, the Unix Way” at 33rd degree by James Lewis

• September 2012: “μService Architecture“ at Baruco by Fred George

• All along, Adrian Cockroft pioneered this style at Netflix as “fine grained SOA”


3

History

James Lewis

Fred George

Adrian Crockcroft

• Area of consideration

– Web systems

– Built collaboratively by several development teams

– With traffic load that requires horizontal scaling (i.e. load balancing across multiple copies of the system)

• Observation– Such systems are often built as monoliths or layered systems


4

The Problem

• One build and deployment unit.

• One code base.

• One technology stack (Linux, JVM, Tomcat, Libraries).

• Benefits– Simple mental model for developers

• one unit of access for coding, building, and deploying

– Simple scaling model for operations • just run multiple copies behind a load balancer


5

Software Monolith

• Huge and intimidating code base for developers.

• Development tools get overburdened– refactorings take minutes

– builds take hours

– testing in continuous integration takes days

• Scaling is limited– Running a copy of the whole system is resource intense

– It doesn’t scale with the data volume out of the box

• Deployment frequency is limited– Redeploying means halting the whole system

– Redeployments will fail and increase the perceived risk of deployment


6

Problems with Software Monolith


7

Monolith Architecture

Load Balancer

Monolithic App

Account Component

CatalogComponent

RecommendationComponent

Customer ServiceComponent

Database

Monolithic applications can be successful, but increasingly people are feeling frustrations with them - especially as more applications are being deployed to the cloud .


8

Monolith vs Microservices

• A layered system decomposes a monolith into layers.

• Usually: presentation, logic, data access.

• At most one technology stack per layer– Presentation: Linux, JVM, Tomcat, Libs, EJB client,

JavaScript

– Logic: Linux, JVM, EJB container, Libs

– Data Access: Linux, JVM, EJB JPA, EJB container, Libs

• Benefits– Simple mental model, simple dependencies

– Simple deployment and scaling model


9

Layered Systems

• Still huge codebases (one per layer).

• ... with the same impact on development, building, and deployment.

• Scaling works better, but still limited.

• Staff growth is limited: roughly speaking, one team per layer works well– Developers become specialists on their layer.

– Communication between teams is biased by layer experience (or lack thereof).


10

Problems with Layered Systems


Service Oriented Architecture

• Maximum reuse [GOOD]

• Maximum canonicality [GOOD] 11




• Incremental change [BAD]

• Operationally complex [BAD] 12

Problems with Service Oriented Architecture


13

Lot of Services!!!

• Applications and teams need to grow beyond the limits imposed by monoliths and layered systems, and they do in an uncontrolled way.

• Large companies end up with landscapes of layered systems that often interoperate in undocumented ways.

• These landscapes then often break in unexpected ways.

• How can a company grow and still have a working IT architecture and vision?– Observing and documenting successful companies (e.g. Amazon, Netflix) lead

to the definition of microservice architecture principles.


14

Growing Beyond Limits


15

Need of an Architectural Pattern

Yesterday’s best practice is tomorrow’s

anti-pattern.

We inadvertently build architectures to solve outdated problems.


16

Problem with Architectural Patterns

“organizations which design systems ... are constrained to produce designs which

are copies of the communication structures of these organizations”

—Melvin Conway


17

Conway’s Law


18

Valued Business Drivers

Better

• Resilient

• Technology choice

Cheaper

• Test effort

• Cost of maintaining

Faster

• Change

• Deployment

• Execution


19

Trends in Software Development

Platform as a Service

Autonomous teams

Continuous Delivery

Agile Organization

Reactive manifesto


20

Availability

A single missing “;” brought down the Netflix website for many hours (~2008)

• The term “micro” refers to the sizing: a microservice must be manageable by a single development team (5-9 developers).

• Functional system decomposition means vertical slicing (in contrast to horizontal slicing through layers).

• Independent deployability implies no shared state and inter-process communication (often via HTTP RESTish interfaces).

• Microservice is the first architectural style developed for post-Continuous Delivery.


21

What is Microservices Architecture?


22

Other definitions

• Small and focused on doing one thing well

• Autonomous

“Loosely coupled service oriented architecture with bounded contexts”Adrian Cockcroft (Netflix)

“SOA done right”Anonymous

“… services are independently deployable and scalable, each service also provides a firm module boundary, even allowing for different services to be written in different programming languages.”Martin Fowler (Thoughtworks)

• Each microservice is functionally complete with – Resource representation– Data management

• Each microservice handles one resource (or verb), e.g. Clients, Shop Items, Carts, Checkout

• Microservices are fun-sized services, as in “still fun to develop and deploy”

• Independent Deployability is key

• It enables separation and independent evolution of– code base– technology stacks– scaling– and features, too


23

Points about Microservices Architecture

• Each service has its own software repository.

• Codebase is maintainable for developers – it fits into their brain.

• Tools work fast – building, testing, refactoring code takes seconds.

• Service startup only takes seconds.

• No accidental cross-dependencies between code bases.


24

Independent Codebase

• Each service is implemented on its own technology stacks.

• The technology stack can be selected to fit the task best.

• Teams can also experiment with new technologies within a single microservice.

• No system-wide standardized technology stack also means– No struggle to get your technology introduced to the canon– No piggy-pack dependencies to unnecessary technologies or libraries– It‘s only your own dependency hell you need to struggle with

• Selected technology stacks are often very lightweight– A microservice is often just a single process that is started via

command line, and not code and configuration that is deployed to a container.


25

Independent Technology Stack

• Each microservice can be scaled independently

• Identified bottlenecks can be addressed directly

• Data sharding can be applied to microservices as needed

• Parts of the system that do not represent bottlenecks canremain simple and un-scaled

• Microservices can be extended without affecting other services– For example, you can deploy a new version of (a part of) the UI

without re-deploying the whole system

– You can also go so far as to replace the service by a complete rewrite

– But you have to ensure that the service interface remains stable


26

Independent Scaling

• Communication between microservices is often standardized using

• HTTP(S) – battle-tested and broadly available transport protocol

• REST – uniform interfaces on data as resources with known manipulation means

• JSON – simple data representation format

REST and JSON are convenient because they simplify interface evolution


27

Inter-communication

• Bounded Context is a central pattern in Domain-Driven Design.

• It is the focus of DDD's strategic design section which is all about dealing with large models and teams.

• DDD deals with large models by dividing them into different Bounded Contexts and being explicit about their interrelationships.


28

Bounded Context

• In computing, a polyglot is a computer program or script written in a valid form of multiple programming languages, which performs the same operations or output independent of the programming language used to compile or interpret it

• Generally polyglots are written in a combination of C (which allows redefinition of tokens with a preprocessor) and a scripting language such as Lisp, Perl or sh.

• Complex applications combine different types of problems, so picking the right language for the job may be more productive than trying to fit all aspects into a single language.

• One of the interesting consequences of this is that we are gearing up for a shift to polyglot persistence- where any decent sized enterprise will have a variety of different data storage technologies for different kinds of data.

• This polyglot affect will be apparent even within a single application, a complex enterprise application uses different kinds of data, and already usually integrates information from different sources.


29

Polyglot Persistence

• Features:– Distributed architecture– separately deployed components– Service components– Bounded context– Service orchestration


30

Microservice Architecture (1)


31

Microservice Architecture (2)

Load Balancer

Account Service CatalogService

RecommendationService

Customer ServiceService

Catalog DB

API Gateway

Customer DB


32

Simple and Lightweight


33

Independent Processes


34

Language Agnostic APIs and Decoupled Services

"Language agnostic" describes a software development paradigm where aparticular language is chosen because of its appropriateness for a particular task(taking into consideration all factors, including ecosystem, developer skill-sets,performance, etc.), and not purely because of the skill-set available within adevelopment team.


35

Size matters (not)• It’s not about

lines of code used

• Small enough to

– Be handled by a single team

– Not feel “big”


36

Slicing

Change your approach to vertical partitions

• Modeled after organization’s domains • Owned by team• Top to bottom• Isolated from each other as much as possible


37

From monolithic to microservices

SEIV – Microservices – SOA vs Microservices

ACID vs BASE

38


Multiple services work together as a system

39


SOA vs Microservices• What “Traditional” SOA got Right

– Breaking monoliths into services

– Focus on integration over internal coupling

– Prefer BASE to ACID

• What “Traditional” SOA Missed

– Architecture is abstract until operationalized.

– Impact of Conway’s Law

– Didn’t support easy change (ESB pattern)

– It is often thought as decomposition within tiers, and introducing another tier – the service orchestration tier

40


SOA vs Microservices

41Microservices Focus on Change

The API is the

contract

…And the product

“This is what I

need…”

“Here is what I

have to offer…”

WSDL is the

Contract

Backend App is the Product

AP

I Man

agem

ent

Public APIs:Better Customer

Experience

SOA: Better Apps +

Integration

Microservices: Improved Developer

Experience

SEIV – Microservices – Characteristics

Characteristics

• Componentization via Services

• Organized around Business Capabilities

• Products not Projects

• Smart endpoints and dumb pipes

• Decentralized Governance

• Decentralized Data Management

• Infrastructure Automation

• Design for failure

• Evolutionary Design42


Componentization via Services• What is a component? We think about how it relates to software

instead of software itself

– Independently Replaceable, Independently Updatable

• How we can do this: Libraries or Services? We do with services

• Interaction mode: share-nothing, cross-process communication

• Independently deployable (with all the benefits), Explicit, REST-based public interface

• Sized and designed for replaceability– Upgrading technologies should not happen big-bang, all-or-nothing-style

• Downsides– Communication is more expensive than in-process– Interfaces need to be coarser-grained– Re-allocation of responsibilities between services is harder 43


Organized around Business Capabilities (1)• The microservice approach to division is different,

splitting up into services organized around business capability.

• Such services take a broad-stack implementation of software for that business area, including user-interface, persistant storage, and any external collaborations.

• Consequently the teams are cross-functional, including the full range of skills required for the development: user-experience, database, and project management.

• Line of separation is along functional boundaries, not along tiers.

44


Organized around Business Capabilities (2)

45

Will this work?


Organized around Business Capabilities (3)

46


Products not Projects• Most application development efforts that we see use a project

model: where the aim is to deliver some piece of software which is then considered to be completed.

• On completion the software is handed over to a maintenance organization and the project team that built it is disbanded.

• Microservice proponents tend to avoid this model, preferring instead the notion that a team should own a product over its full lifetime.

• A common inspiration for this is Amazon's notion of "you build, you run it" where a development team takes full responsibility for the software in production.

• This brings developers into day-to-day contact with how their software behaves in production and increases contact with their users, as they have to take on at least some of the support burden.

47

https://queue.acm.org/detail.cfm?id=1142065


Smart endpoints and dumb pipes (1)

48

What’s the problem here?


Smart endpoints and dumb pipes (2)• Applications built from microservices aim to be as decoupled and as

cohesive as possible - they own their own domain logic and act more as filters in the classical Unix sense - receiving a request, applying logic as appropriate and producing a response.

• These are choreographed using simple RESTish protocols rather than complex protocols such as WS-Choreography or BPEL or orchestration by a central tool.

• Microservice teams use the principles and protocols that the world wide web (and to a large extent, Unix) is built on.

• Often used resources can be cached with very little effort on the part of developers or operations folk.

• Uses HTTP request-response with resource API.

• Messaging over a lightweight message bus. The infrastructure chosen is typically dumb (dumb as in acts as a message router only) - simple implementations such as RabbitMQ or ZeroMQ don't do much more than provide a reliable asynchronous fabric - the smarts still live in the end points that are producing and consuming messages; in the services. 49


Decentralized Governance• Principle: focus on standardizing the relevant parts, and leverage battle-

tested standards and infrastructure

• Rather than use a set of defined standards written down somewhere on paper, prefer the idea of producing useful tools that other developers can use to solve similar problems to the ones they are facing.

• These tools are usually harvested from implementations and shared with a wider group, sometimes, but not exclusively using an internal open source model.

• What needs to be standardized– Communication protocol (HTTP)

– Message format (JSON)

• What should be standardized– Communication patterns (REST)

• What doesn‘t need to be standardized– Application technology stack

50


Decentralized Data Management• Each service can choose the persistence solution that

fits best its– Data access patterns

– Scaling and data sharding requirements

• Only few services really need enterprisey persistence

• Structured programming – shared global state is a bad thing!!!

• Microservices prefer letting each service manage its own database, either different instances of the same database technology, or entirely different database systems -an approach called Polyglot Persistence. You can use polyglot persistence in a monolith, but it appears more frequently with microservices.

51

http://martinfowler.com/bliki/PolyglotPersistence.html


Infrastructure Automation• Having to deploy significant number of services

forces operations to automate the infrastructure for– Deployment (Continuous Delivery)

– Monitoring (Automated failure detection)

– Managing (Automated failure recovery)

• Consider that:– Amazon AWS is primarily an internal service

– Netflix uses Chaos Monkey to further enforceinfrastructure resilience

52


Infrastructure as a code practices1. Use definition files

2. Self-documented systems and processes

3. Version all things

4. Continuously test systems and processes

5. Small changes rather than batches

6. Keep services available continuously

53


Design for Failure• A consequence of using services as components, is that applications

need to be designed so that they can tolerate the failure of services.

• Since services can fail at any time, it's important to be able to detect the failures quickly and, if possible, automatically restore service.

• Microservice applications put a lot of emphasis on real-time monitoring of the application, checking both architectural elements (how many requests per second is the database getting) and business relevant metrics (such as how many orders per minute are received).

• Semantic monitoring can provide an early warning system of something going wrong that triggers development teams to follow up and investigate.

• Microservice teams would expect to see sophisticated monitoring and logging setups for each individual service such as dashboards showing up/down status and a variety of operational and business relevant metrics 54


Evolutionary Design• The key property of a component is the notion of independent replacement

and upgradeability - which implies we look for points where we can imagine rewriting a component without affecting its collaborators.

• Indeed many microservice groups take this further by explicitly expecting many services to be scrapped rather than evolved in the longer term.

• The Guardian website is a good example of an application that was designed and built as a monolith, but has been evolving in a microservice direction.

• The monolith still is the core of the website, but they prefer to add new features by building microservices that use the monolith's API.

• This approach is particularly handy for features that are inherently temporary, such as specialized pages to handle a sporting event.

• Such a part of the website can quickly be put together using rapid development languages, and removed once the event is over.

• Similar approaches at financial institutions where new services are added for a market opportunity and discarded after a few months or even weeks.

55

SEIV – Microservices – Why and When

Why use Microservices Architecture• Faster and simpler deployments and rollbacks

– Independent Speed of Delivery (by different teams)

• Right framework/tool/language for each domain

• Greater Resiliency– Fault Isolation

• Better Scaling– A microservices architecture puts each element of functionality

into a separate service and scales by distributing these services across servers, replicating as needed.

• Better Availability– If architected right

56


When to use Microservices Architecture (1)

• When you need to support Desktop, web , mobile, Smart TVs, Wearable, etc... or you don't know in future which kind of devices you need to support.

• You are developing a server-side enterprise application. It must support a variety of different clients including desktop browsers, mobile browsers and native mobile applications. The application might also expose an API for 3rd parties to consume. It might also integrate with other applications via either web services or a message broker. The application handles requests (HTTP requests and messages) by executing business logic; accessing a database; exchanging messages with other systems; and returning a HTML/JSON/XML response.

• You have lot of people in your organization and organize them in way that make sense/break teams into useful group.

57


When to use Microservices Architecture (2)

58microservices approach without good cause - Microservice Envy


Microservice Premium

59

Microservice Premium

SEIV – Microservices – Challenges

Microservices Challenges

60

Can lead to chaos if not designed right …


Complexity

• Distributed Systems are inherently Complex

– N/W Latency, Fault Tolerance, Retry storms ..

• Operational Overhead

– TIP: Embrace DevOps Model

61


Service Discovery (1)

• 100s of MicroServices

– Need a Service Metadata Registry (Discovery Service)

62

Account Service CatalogService

RecommendationService

Customer ServiceService

XService Y

Service

ZService

RegistryService

(e.g. Netflix Eureka)


Service Discovery (2)

63


Chattiness (and Fan Out)

• The biggest issue in changing a monolith into microservices lies inchanging the communication pattern.

• A naive conversion from in-memory method calls to RPC leads to chattycommunications which don't perform well.

• Instead you need to replace the fine-grained communication with acoarser -grained approach.

64

2 Billion Requests per day on Edge ServiceResults in ~20 Billion Fan out requests in ~100 MicroServices

1 Request 1 Request

Monolithic App MicroServices


Microservice Silo Architecture

65


Testing• A single microservice isn‘t the whole system.

• A clear picture of upstream and downstream services is needed for integration testing

• Refer to deck: http://martinfowler.com/articles/microservice-testing/

66

http://martinfowler.com/articles/microservice-testing/


Prerequisites (1)

• Before you go into production with a microservices system, you need to ensure that you have key prerequisites in place– Rapid Provisioning

• Dev teams should be able to automatically provision new infrastructure

– Basic Monitoring• Essential to detect problems in the complex system

landscape

– Rapid Application Deployment

– Devops Culture

67microservices approach without good cause - Microservice Envy


Prerequisites (2)

• These are optional things will improve your microservicessystem

– Traceable Business Transactions

– Continuous Delivery

– Product-centered teams

– Multi-Dev Environment

– Infrastructure as code

– Have experiences and Knowledgeable about Domain-Driven Design

68


Versioning (1)• With a monolith any changes require a full build and

deployment of the entire application.

• With microservices, however, you only need to redeploy the service(s) you modified.

• This can simplify and speed up the release process.

• The downside is that you have to worry about changes to one service breaking its consumers.

• The traditional integration approach is to try to deal with this problem using versioning, but the preference in the microservice world is to only use versioning as a last resort. 69


Versioning (2)• Microservice architectures enable independent evolution of

services – but how is this done without breaking existing clients?

• There are two answers– Version service APIs on incompatible API changes

– Using JSON and REST limits versioning needs of service APIs

• Versioning is key– Service interfaces are like programmer APIs – you need to know which

version you program is in.

– As service provider, you need to keep old versions of your interface operational while delivering new versions.

70


API compatibility• There are two types of compatibility

• Forward Compatibility

– Upgrading the service in the future will not break existing clients

– Requires some agreements on future design features, and the design of new versions to respect old interfaces

– imagine processing new messages arriving to the service with new data.

– Postel’s law: ‘be conservative in what you do, be liberal in what you accept from others’

• Backward Compatibility– Newly created service is compatible with old clients

– Requires the design of new versions to respect old interfacesThe hard type of compatibility is forward compatibility!

71


Further Challenges

• Transactions – Instead of distributed transactions, compensations are used (as in

SOA)

• Authentication– Is often offloaded to reverse proxies making use of authentication

(micro)services

• Request logging– Pass along request tokens– Add them to the log– Perform log aggregation

72

SEIV – Microservices – Benefits

Benefits of Microservices

• Asynchronicity

• Integration & Disintegration

• Complected (intertwined)Deployments

• Evolutionary Architecture

• Reduction of release risk

• Backends for Frontend

73

SEIV – Microservices – Recommendations

Inverse Conway Maneuver

74

Build teams that look like the architecture you want

(and it will follow).


Efferent Coupling

• Strive for low efferent coupling for your team.

• Efferent Coupling (Ce) is also known as Outgoing Dependencies or the Number of Types inside a Package that Depend on Types of other Packages. Ce is a metric for packages. In short, this measure includes all the types within the source of the measured package referring to the types not in the measured package.

• Teams with low efferent coupling deliver relatively independently into a common integration pipeline (without fearing breaking each others builds.

75


Do choreography instead orchestration• What is service orchestration? It's the process of managing

different services and the dependencies between them such that we promote the principles of loose coupling.

• What is choreography? Choreography is the process of agreeing, between the dancers, how a dance will be performed before it is performed.

• One thing with micro-services that people get caught up on is how to do orchestration. The answer is don't. Instead of doing orchestration, do choreography.

76


Low coupling and high cohesion

• Low coupling and high cohesion.

• Coupling: Measurement of dependency between two components. Expressed as sensitivity towards propagation of changes and errors.

• Cohesion: Degree to which parts of a module belong together.

77

Coincidental cohesion (worst)

• random

Logical cohesion

• same in nature

Temporal cohesion

• executed at a particular time

Procedural cohesion

• part of fixed execution sequence

Informational cohesion

• operating on the same data

Sequential cohesion

• Output of one is used as input by another

Functional cohesion (best)

• contributing to a single task


Monolith First

• Teams using a microservices architecture have a common pattern.

• Almost all the successful microservice stories have started with a monolith that got too big and was broken up

• Almost all the cases a system that was built as a microservicesystem from scratch, it has ended up in serious trouble.

• This pattern has led many of people to argue that you shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile.

• Instead begin with a monolith, keep it modular, and split it into microservices once the monolith becomes a problem.

78


Another solution to monoliths – Cookie cutter scaling

• Microservices is an antidote to monoliths

• Cookie cutter scaling is another antidote to monoliths, with the right build infrastructure and DevOps.

79

SEIV – Microservices – Epilogue

Advantages of Microservices

• Each micro service is small and focused on a specific feature / business requirement.

• Microservice can be developed independently by small team of developers (normally 2 to 5 developers).

• Microservice is loosely coupled, means services are independent, in terms of development and deployment both.

• Microservice can be developed using different programming language (Personally I don't suggest to do it).

• Microservice allows easy and flexible way to integrate automatic deployment with Continuous Integration tools (for e.g: Jenkins, Hudson, bamboo etc..).

• The productivity of a new team member will be quick enough.

• Microservice is easy to understand, modify and maintain for a developer because separation of code,small team and focused work.

80


Disadvantages of Microservices

• Microservice architecture brings a lot of operations overhead.

• DevOps Skill required (http://en.wikipedia.org/wiki/DevOps).

• Duplication of Effort.

• Distributed System is complicated to manage .

• Default to trace problem because of distributed deployment.

• Complicated to manage whole products when number of services increases.

81

http://en.wikipedia.org/wiki/DevOps


Who uses Microservices?

• Most large scale web sites including Twitter, Netflix, Amazon and eBay have evolved from a monolithic architecture to a microservices architecture.

82

Microservices at Netflix


Building Microservices

83

Microservices Resource Guide: http://martinfowler.com/microservices/

Microservices Articles: http://martinfowler.com/tags/microservices.html

http://martinfowler.com/microservices/

http://martinfowler.com/tags/microservices.html