Microservice

Deployment

Software Engineering II

Sharif University of Technology

MohammadAmin Fazli

Deployment

Topics

Continuous Integration & Microservices

Continuous Delivery

Deployment Artifacts

Custom Images

Environments

Service Configuration

Service-to-Host Mapping

Physical to Virtual

Reading:

Building Microservices-Sam Newman-Chapter VI

2

Deployment

Continuous Integration

With CI, the core goal is to keep everyone in sync with each other, which we achieve by making sure that newly checked-in code properly integrates with existing code.

A CI server detects that

The code has been committed

Checks it out

Carries out some verification

Making sure the code compiles

Making sure tests pass

As part of this process, we often create artifact(s) that are used for further validation, such as deploying a running service to run tests against it.

3

Deployment

Continuous Integration

Benefits

Fast Feedback about code quality

Automatic binary artifacts’ creation

All the code is version controlled and all artifacts can be recreated

Traceability from deployed artifacts to the code

Traceability over what tests were run over codes and artifacts

CI checks: Jez Humble’s three questions he asks people to test if they really understand what CI is about

Do you check in to mainline once per day?

You need to make sure your code integrates.

Do you have a suite of tests to validate your changes?

Check if our code semantically integrates

When the build is broken, is it the #1 priority of the team to fix it?

Please avoid piling up non-integrated changes

4

Deployment

CI & Microservices

When thinking about microservices & CI, we need to think

about how our CI builds map to individual microservices.

1st solution: One repository+Monolithic build

5

Deployment

CI & Microservices

1st solution:

Benefits:

Fewer repositories to worry about

Simpler build

Downsides:

A one-line change to a single service cause all the other services get verified

and built and can be very time consuming

If this one-line change to a service breaks the build, no other changes can be

made to the other services until that break is fixed.

If a change cause break, which team is in charge?

6

Deployment

CI & Microservices

2nd Solution: One code repository + Individual Build

Check in/Check out process can be simpler as we have one repository to worry about

Can be easily get into the habit of checking in source code for multiple services at once which can slip into making changes that couple services together.

7

Deployment

CI & Microservices

3rd Solution: Individual Repository + Individual Build

When making a change, we run only the build and tests I need to.

We get a single artifact to deploy.

Alignment to team ownership is more clear.

If you own the service, you own the repository and the build.

Making changes across repositories can be more difficult in this

world8

Deployment

Continuous Delivery

The build pipeline concept gives us a nice way of tracking the

progress of our software as it clears each stage, helping give us

insight into the quality of our software.

Ex: Running fast and slow tests together does not get us a fast

feedback. If fast tests fail waiting for slow tests is useless. So it is

better to run them in different stages.

CD builds according to the pipeline concept.

Continuous delivery is the approach whereby we get constant

feedback on the production readiness of each and every check-in,

and furthermore treat each and every check-in as a release

candidate.

9

Deployment

Continuous Delivery

In CD we must model the process of getting our software from check-in to production and know where any given version of the software is.

Different software stages must be modeled both manual or automated

We really want a tool that embraces CD as a first-class concept.

Tools that fully support CD allow you to define and visualize these pipelines, modeling the entire path to production for your software.

By modeling the entire path to production for our software, we greatly improve visibility of the quality of our software, and can also greatly reduce the time taken between releases

10

Deployment

Platform Specific Artifacts

Most technology stacks have some sort of first-class artifact, along with tools to support creating and installing them.

Rubby gems, Java Jars, Python eggs

Depending on the technology stack, these artifacts may not be enough by themselves.

Example: sometimes they need Apache/Nginx installation and configuration

We may need some way of installing and configuring other software that we need in order to deploy and launch our artifacts.

This is where automated configuration management tools like Puppet and Chef can help.

These artifacts are specific to a certain technology stack. This may make deployment more difficult when we have a mix of technologies in play

Automation can go a long way toward hiding the differences in the deployment mechanisms of the underlying artifacts.

11

Deployment

Operating System Artifacts

One way to avoid the problems associated with technology-specific

artifacts is to create artifacts that are native to the underlying

operating system.

Rpm packages for CentOS, MSI for windows, deb for Ubuntu

Advantages of using OS-specific artifacts:

From a deployment point of view we don’t care what the underlying

technology is. We just use the tools native to the OS to install the package.

The OS tools can also help us uninstall and get information about the

packages too,

The OS tools may even provide package repositories that our CI tools can

push to

Downsides:

Difficulty in creating the packages

There is a great overhead when deploying on different OSes

12

Deployment

Custom Images

Preparing machines for a service is very time consuming even if automated configuration tools (Puppet, Chef, Ansible) are used.

Installing a JVM needs some minutes

Automatic tools can be smart and will avoid installing software that is already present. This does not mean that running the scripts on existing machines will always be fast, unfortunately, as running all the checks takes time.

Moreover, we don’t want to allow for too much configuration drift. So we should have many machine preparations and it is a real drag.

Configuration Drift is the phenomenon where running servers in an infrastructure become more and more different as time goes on, due to manual ad-hoc changes and updates, and general entropy.

13

Deployment

Custom Images

One approach to reducing this spin-up

time is to create a virtual machine image

that bakes in some of the common

dependencies we use.

When we want to deploy our software,

we spin up an instance of this custom

image, and all we have to do is install the

latest version of our service.

Because you build the image only once,

when you subsequently launch copies of

this image you don’t need to spend time

installing your dependencies, as they are

already there.

14

Deployment

Custom Images

Drawbacks:

Building images can take a long time. This means that for developers

you may want to support other ways of deploying services to ensure

they don’t have to wait half an hour just to create a binary

deployment.

Some of the resulting images can be large. This could be a real

problem if you’re creating your own images.

Moving a huge image around a network is a real problem

The tool chain required to build such an image varied from platform

to platform.

Building a VMWare image is different from building an AWS AMI, a Vagrant

image, or a Rackspace image.

Some of the tools are multi-platform like packer.io which has support for

VMWare, AWS, Racksapce Cloud, Digital Ocean, Vagrant and …

15

Deployment

Immutable Servers

By storing all our configuration in source control, we are trying to ensure that we can automatically reproduce services and hopefully entire environments at will.

But once we run our deployment process, what happens if someone comes along changes things independently of what is in source control?

Configuration Drift-the code in source control no longer reflects the configuration of the running host.

Immutable Servers Pattern: we can ensure that no changes are ever made to a running server. Instead, any change, no matter how small, has to go through a build pipeline in order to create a new machine.

We can disable SSH

16

Deployment

Environments

Different environments in CD stages:

Development Environments and Unit Testing

One for Slower Tests

One for UAT

One for Performance Testing

One for Production

17

Deployment

Environments

Our microservice should be the same throughout, but

the environment will be different.

Separate distinct collection of configuration and hosts

Differences can be more than just some configurations

Ex. our production environment for our service might consist of multiple

load-balanced hosts spread across two data centers, whereas our test

environment might just have everything running on a single host

As you move from development environment to build server

to UAT environment all the way to production, you’ll want to

ensure that your environments are more and more

production-like to catch any problems associated with these

environmental differences sooner.

18

Deployment

Service Configuration

Our services need some configuration

Ex. Username and password for database connection in different

environments

Configuration that changes from one environment to another

should be kept to an absolute minimum.

The more your configuration changes fundamental service

behavior, and the more that configuration varies from one

environment to another, the more you will find problems only

in certain environments, which is painful in the extreme.

19

Deployment

Service Configuration

Handling service configurations in CD

Building one artifact per environment, with configuration inside the artifact itself. Let’s imagine I build a Customer-Service-Test and Customer-Service-Prod

artifacts. If my Customer-Service-Test artifact passes the tests, but it’s theCustomer-Service-Prod artifact that I actually deploy, can I be sure that I haveverified the software that actually ends up in production?

There is the additional time taken to build these artifacts.

We need to know at build time what environments exist.

Problems with sensitive data: I don’t want information about production passwords checked in with my source code, but if it is needed at build time to create all those artifacts, this is often difficult to avoid

Creating one single artifact, and manage configuration separately

This could be a properties file that exists for each environment, or different parameters passed in to an install process.

Using a dedicated system for providing configuration

Useful when dealing with a large number of microservices

20

Deployment

Service-to-Host Mapping

How many services per machine?

Solutions

Multiple services per host

Application containers

Single service per host

Platform as a service

Host: A generic term which defines an operating system onto

which we can install and run services

They can be physical or virtual machines

21

Deployment

Multiple Services per Host

Benefits

It is simpler. If more services are packed on to a single host, the host

management workload doesn’t increase as the number of services

increases.

It has less cost. Even if you have access to a virtualization platform

that allows you to provision and resize virtual hosts, the

virtualization can add an overhead that reduces the underlying

resources available to your services.

This model is also familiar to those who deploy into some form of an

application container.

22

Deployment

Multiple Services per Host

Challenges

It can make monitoring more difficult.

What can we do to trace CPU for one service?

If one service is under significant load, it can end up reducing the resources available to other parts of the system.

Deployment of services can be somewhat more complex too, as ensuring one deployment doesn’t affect another leads to additional headaches.

If we use Puppet to prepare a host, but each service has different (and potentially contradictory) dependencies

This model can also inhibit autonomy of teams.

Efforts to target scaling to the service most in need of it can be complicated.

This model can limit our deployment artifact options.

23

Deployment

Application Containers

The idea is that the application container your services live in

gives you benefits in terms of improved manageability, such as

clustering support to handle grouping multiple instances

together, monitoring tools, and the like.

Like IIS for .NET applications and Servlet container for Java

applications

24

Deployment

Application Containers

Downsides

They inevitably constrain technology choice.

They also force the implementation technology

Some of them have ability to manage clusters to support shared in-

memory session state, which is in contradiction with scalability

Their monitoring abilities are not sufficient

Attempting to do proper lifecycle management of applications on top

of platforms like the JVM can be problematic, and more complex than simply restarting a JVM.

Analyzing resource use and threads is also much more complex, as

you have multiple applications sharing the same process.

They add resource overheads

25

Deployment

Single Service per Host

Benefits:

Simpler monitoring

Simpler remediation

Reduced single points of failure

Easy Scaling one service independent from others

This opens up the potential to use alternative deployment techniques

such as image-based deployments or the immutable server pattern

Downside

We have more servers to manage

It has more cost

26

Deployment

Platform as a Service

Using a platform as a service (PaaS), you are working at a higher-level abstraction than at a single host.

Taking technology specific artifacts like java WAR files and Ruby gems and automatically provision and run them.

Some of them handle scaling the system up and down

27

Deployment

Automation

One of the pushbacks against the single-service-per-host setup

is the perception that the amount of overhead to manage

these hosts will increase.

It is true if everything is done manually

2X servers, 2X work

Many things can be automated

Automation enables developers for self-service-provision

Ideally, developers should have access to exactly the same tool chain

as is used for deployment of our production services so as to ensure

that we can spot problems early on.

28

Deployment

Move from Physical to Virtual

One of the key tools available to us in managing a large

number of hosts is finding ways of chunking up existing

physical machines into smaller parts.

Traditional Virtualization

Vagrant

Linux Containers

Docker

29

Deployment

Traditional Virtualization

Virtualization allows us to slice up a physical server into separate hosts, each of which can run different things.

Type 2 virtualization: A hypervisor run on the OS whose jobs are:

It maps resources like CPU and memory from the virtual host to the physical host.

It acts as a control layer, allowing us to manipulate the virtual machinesthemselves.

Technologies like AWS, VMWare, VSPhere, Xen, KVM and …

In type 1, VMs run directly on hardware.

The problem is that the hypervisor here needs to set aside resources to do its job.

30

Deployment

Vagrant

Vagrant is a very useful deployment platform, which is normally used for dev and test rather than production.

Vagrant provides us with a virtual cloud on your laptop.

It allows us to define a set of VMs in a text file, along with how the VMs are networked together and which images the VMs should be based on.

This text file can be checked in and shared between team members.

We can spin up multiple VMs at a time, shut individual ones to test failure modes, and have the VMs mapped through to local directories so you can make changes and see them reflected immediately.

Downside:

Running lots of VMs can tax the average development machine. If we have one service to one VM, you may not be able to bring up your entire system on your local machine.

31

Deployment

Linux Containers

In Linux processes can spawn other processes

The Linux kernel’s job is to manage the tree of processes

For Linux users, there is an alternative to virtualization. Rather

than having a hypervisor to segment and control separate

virtual hosts, Linux containers instead create a separate

process space in which other processes live.

Many forms of this technology

Solaris Zones

OpenVZ

LXC is the most popular

Available on most modern Linux

distributions

32

Deployment

Linux Containers

We don’t need a hypervisor.

More available resources

Although each container can run its own operating system distribution, it has to share the same kernel

Because the kernel is where the process tree lives

Ex. Our host operating system could run Ubuntu, and our containers CentOS, as long as they could both share the same kernel.

Linux containers are much faster to provision than full-fat virtual machines.

Due to the lighter-weight nature of containers, we can have many more of them running on the same hardware than would be possible with VMs.

33

Deployment

Docker

Docker is a platform built on top of lightweight containers. It handles much of the work around handling containers for you.

In Docker, you create and deploy apps, which are synonymous with images in the VM world, albeit for a container-based platform.

Docker manages the container provisioning, handles some of the networking problems for you, and even provides its own registry concept that allows you to store and version Docker applications.

The Docker app abstraction is a useful one for us, because just as with VM images the underlying technology used to implement the service is hidden from us.

Rather than using Vagrant to host multiple independent VMs,each one containing its own service, we can host a single VM in Vagrant that runs a Docker instance.

We then use Vagrant to set up and tear down the Docker platformitself, and use Docker for fast provisioning of individual services.

34

Deployment

Docker

Docker is a simple PaaS for a single machine

Different technologies for Docker:

CoreOS: A stripped-down Linux OS that provides only the essential

services to allow Docker to run. This means it consumes fewer

resources than other operating systems, making it possible to

dedicate even more resources of the underlying machine to our

containers.

Kubernetes & CoreOS Cluster Technology: Help to manage services

across multiple Docker instances

Deis: Heroku like PaaS on top of Docker

Container as a Service (CaaS): Docker with an appropriate

scheduling layer sits between IaaS and PaaS solutions

35

Deployment

Deployment Interface

Ex: we’re developing locally and want to deploy our catalog service into our local environment.

Ex: Once I’ve checked in, our CI build service picks up the change and creates a new build artifact, giving it the build number b456.

Ex: our QA wants to pull the latest version of the catalog service into an integrated test environment to do some exploratory testing, and to help with a showcase.

36

Deployment

Environment Definition

Ex: An

environment

definition file

37

Deployment

Environment Defintion

1: We varied the size of the instances we used to be more cost

effective. You don’t need a 16-core box with 64GB of RAM for

exploratory testing!

2: Being able to specify different credentials for different

environments is key. Credentials for sensitive environments

were stored in different source code repos that only select

people would have access to.

3:We decided that by default if a service had more than one

node configured, we would automatically create a load

balancer for it.

38

Deployment

Environment Definitions

39

1: This was the name of the Puppet file to run.