using aws to build a large scale dockerized microservices

15
Using AWS to Build a Large Scale Dockerized Microservices Architecture Dr. Oliver Wahlen moovel Group GmbH Frankfurt, 30. Juni 2016

Upload: dangkhuong

Post on 13-Feb-2017

230 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Using AWS to Build a Large Scale Dockerized Microservices

Using AWS to Build a Large Scale

Dockerized Microservices ArchitectureDr. Oliver Wahlen

moovel Group GmbH

Frankfurt, 30. Juni 2016

Page 2: Using AWS to Build a Large Scale Dockerized Microservices

The moovel Group GmbH

Our vision is an ecosystem that

• simplifies mobility now

• solves the mobility challenges of the future

Page 3: Using AWS to Build a Large Scale Dockerized Microservices

Mobility as a Service

Urban areas will become more crowded:

• Today 54% of people live in cities In 2050 there will be 66%

• More people on less space

• Driving and parking will become more regulated

• High demand for new transportation concepts

Autonomous driving will become reality:

• Taxi and car sharing business will change completely

• Cost efficiency similar to public transport but faster

• Owning a car becomes less relevant

The future will be Mobility as a Service

Page 4: Using AWS to Build a Large Scale Dockerized Microservices

Entwicklung 3: Autonomes Fahren

Wir verändern die Art,

wie wir uns fortbewegen.

We are building apps, widgets and services that integrate

existing transportation providers

Benefits for the user:

• Only one registration: no need to remember passwords

or provide payment data several times

• Transparency on mobility alternatives

• Intermodal routing

Benefits for the partners:

• Get new users from other mobility providers

• Get new users from different geographic regions

moovel Technology

Page 5: Using AWS to Build a Large Scale Dockerized Microservices

Entwicklung 3: Autonomes Fahren

Wir verändern die Art,

wie wir uns fortbewegen.

• Scalability up to millions of users

• Future prove: Easy to modify components and update technologies

• Usable on global scale

• Minimal operational effort

• Teams can develop independently

Design Goals of Our System Architecture

Page 6: Using AWS to Build a Large Scale Dockerized Microservices

Entwicklung 3: Autonomes Fahren

Wir verändern die Art,

wie wir uns fortbewegen.

Avoid Vendor Lock-In vs. Reduced Operational Effort

Services currently

used by moovel

Page 7: Using AWS to Build a Large Scale Dockerized Microservices

Entwicklung 3: Autonomes FahrenWhat and Why Docker?

Docker Containers

• Provide application(s) with a virtual operating system

• Isolate applications from the host OS and other docker containers

(e.g. Files, Network)

• Are self contained run on any machine without other installs

• Are simple to build images reference a base image

Docker Registry

• Stores docker images produced by the build system

• Provides access control and maintenance of tags

• Dockerhub (Docker Inc.) vs. Amazon EC2 Container Registry

Page 8: Using AWS to Build a Large Scale Dockerized Microservices

Entwicklung 3: Autonomes FahrenWhat and Why EC2 Container Service (ECS)?

ECS is a highly scalable, fast, container management service that makes

it easy to run, stop, and manage Docker containers on a cluster of

Amazon EC2 instances.

– AWS documentation

Technology similar to Kubernetes on Google Container Engine

but tailored for AWS and fully managed

Terminology

• ECS cluster: Group of EC2 instances that allow docker execution

• ECS container instance: EC2 instance with ECS container agent

• ECS task: The running docker container on an instance

• ECS task definition: docker image, CPU, mem, port requirements, …

• ECS service: number of parallel tasks, scalability, health checks, …

Page 9: Using AWS to Build a Large Scale Dockerized Microservices

Entwicklung 3: Autonomes FahrenWhat and Why Microservices?

http://martinfowler.com/articles/microservices.html

• Services independently deployable

Continuous Deployment

• Scalable on service level

Specific to service needs:

CPU, memory

• Services have firm boundary

allows for:

different languages per service

different teams per service group

evolution of logic and technology

ECS Cluster

EC2 Instance

Docker Task

= ECS Task

Web Service

Page 10: Using AWS to Build a Large Scale Dockerized Microservices

Entwicklung 3: Autonomes FahrenConway’s Law And Microservices: A Good Fit

Organizations which design

systems [...] are constrained to

produce designs which are copies

of the communication structures

of these organizations

– Melvin Conway 1967

Product Owner

App-Developers

Backend-Devs

Search

Squad

Book

SquadUser

Squad

Pay

Squad

moovel product dev is organized in squads (feature teams):• with a long term business vision• ability to build and modularize mobile apps / componets• build their own set of microservices• choose the technology that best fits their needs

Page 11: Using AWS to Build a Large Scale Dockerized Microservices

Entwicklung 3: Autonomes FahrenContinuous Integration with Docker and AWS

Developer

code push to

SaaS Build SystemVersion Control

triggers dockerfile.tgz

S3 Bucket

Build Path

SaaS Build System Dockerhub

Deployment triggers:

develop branch: automatic

master branch: manually

Deployment Path

docker image

push to

start update

CloudformationElastic Container

Services: dev/prod

update docker

Page 12: Using AWS to Build a Large Scale Dockerized Microservices

Entwicklung 3: Autonomes Fahrenmoovel System Architecture

moovel VPC

Client App

External ELB• TLS termination

Kong API Gateway• reverse proxying

• LUA plugins

Consul• service discovery

• client load balancing

• service configuration

synchronous REST call

ECS microservice

SNS

Topic

SQS

Queue

asynchronously

publish changes

logging Kinesis ELK Stack

Page 13: Using AWS to Build a Large Scale Dockerized Microservices

Entwicklung 3: Autonomes FahrenLimitations of Existing Solution

• Difficult to realize dynamic port allocation

Currently each service has a fixed port

• Two levels of scaling introduce additional complexity:

Cluster nodes = No. of EC2 instances

ECS Tasks = No. of running Docker containers for a service

had to introduced a custom metric taskless instance count

• Service discovery with Consul:

Automates node and service registry

Provides hierarchical key value store with ACL for configuration

Consul is a very critical component

Must be operated and maintained

• Kong API gateway:

Extremely flexible and powerful nginx based reverse proxy

Requires cassandra DB

Must be operated and maintained

• Elasticsearch Logstash Kibana (ELK):

Do not use it for long term data storage!

Page 14: Using AWS to Build a Large Scale Dockerized Microservices

Entwicklung 3: Autonomes FahrenCheck-up of Design Goals

Scalability

moovel runs 80 services on 21 ECS clusters (dev+prod)

Future prove

Exchanging microservice technology is easy

We have microservices in Java, Node.js, Python, Kotlin, …

Global Scale

The world is currently operated from eu-west-1

Pub/Sub is good starting point for multi region deployment

Minimal Operational Effort

Technology is operated by 10 squads

Currently no DevOps team needed: squads build it and run

Deployments are easy, frequently done and without downtime

Teams can develop independently

Technology is operated by 10 squads

Microservices, REST and Pub/Sub are a good basis

UI decomposition in app is difficult: modularization ongoing

Page 15: Using AWS to Build a Large Scale Dockerized Microservices

Thank You