Building Bizweb Microservices with Docker

Nguyễn Minh KhôiCTO of DKT Technologydkt.com.vn

Programming Languages

Frameworks & Libraries

Web ServersDatabases

Message Queues

|S3, EC2, Route53

Cloud Services

Others

Bizweb Tech Stack

2

Bizweb Microservices Components

• Based on Spring Boot & Netflix OSS

• Service Discovery: Eureka (Server), Ribbon (Client)

• API Gateway: Zuul

• Centralized Configuration: Spring Cloud Config

• API Security: Spring Security & Spring Security OAuth

• REST API: Spring Boot

• Job Service: Kafka & Spring Boot

3

Bizweb Microservices Architecture

4

Eureka Service Discovery

• Each service has unique serviced

• Service uses Eureka Client to interact with Eureka Server:• Register: serviceId, host, port

• Renew: using heartbeats to check status

• Get Registry: return list host:port of services by serviceId

5

Zuul API Gateway

• Single entry point for clients to call API

• Edge service: not use to call between microservices

• Using Ribbon to call microservices• Client Load Balancer

• Caching

• Fault tolerance

• Zuul Filter: • Handle request in pipeline mode

• PRE, ROUTING, POST, ERROR

6

Centralized Configuration

7

Problems:- Stored on each service,

hard to control- Hard to update new

configuration- Reload config at

runtime

Centralized Configuration

8

Problems

• Take times to deploy on new servers:• Install Java

• Copy fat .jar file (~75-100MB) using FTP/SCP

• Make script to run as a Linux service

• Take times to update services: • 20 microservices + job services

• Manual update on multiple hosts

• Manual scale & choose server to deploy

• Quite hard to monitor these microservices

9

Solved with Docker & Jenkins

10

Simple Dockerfile for all services

FROM frolvlad/alpine-oraclejdk8:slim

ADD lib lib

ADD product.jar app.jar

RUN sh -c 'touch /app.jar'

ENTRYPOINT ["java","-Xmx128m","-Xms128m","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]

11

Docker Swarm Mode

• Built-In Orchestration

• Easy to startdocker swarm initdocker services create --name product product:1.2.0

• Secure by defaultdocker swarm join --token [manager_token|worker_token]

• Easy to scaledocker service scale product=10

• Rolling updatesdocker service update --update-delay 1m --update-parallelism 2 --image product:1.2.1 product

12

Swarm Manager & Worker

13

Swarm Manager & Worker

14

• Maintaining cluster state• Scheduling services• Use RAFT consensus to elect new leader

Swarm Manager & Worker

15

• Pull tasks from Managers• Receive tasks in batch (100ms or 100 tasks)• Manager node can be worker, too

Combine Netflix OSS with Docker Swarm

• Zuul for API Gateway

• Eureka for Service Discovery

• Ribbon Client for direct call between microservices

• Docker Swarm:• Manage microservices

• Deploy, scale, update microservices

16

Docker Swarm Deployment

• 3 manager nodes on 3 different physical machines

• Workers on Physical & Virtual Machines

• Using overlay network:• Communicate with Eureka Server• Direct call between microservices

• Label for services & environment:• INFRASTRUCTURE: running Zuul, Eureka, Config, OAuth Service• SERVICE: running microservices• JOB: running Job services• DEV: running all containers of dev environment

17

Docker Swarm Cluster

18

swarm manager

(primary)

config server

eureka server

zuul server

swarm manager

(secondary)

config server

eureka server

zuul server

swarm manager

(secondary)

config server

eureka server

zuul server

worker-01

microservice-02

microservice-xx

microservice-01

worker-xx

microservice-02

microservice-xx

microservice-01

worker-job

job-02

job-xx

job-01

oauth server oauth server oauth server

Bizweb Overlay Network

server-01

redis

INFRASTRUCTUREINFRASTRUCTUREINFRASTRUCTURE

SERVICE SERVICE JOBworker-dev

job-yy

microservices-xx

DEV

Docker Swarm Cluster

server-02

redis

server-03

redis

Redis Cluster

CI with Jenkins & Docker

• Environment Prerequisites:• Gitlab 8.12 (support webhooks)

• Jenkins 2

• Docker Registry 2.0

• Run on Docker

19

Data Center Zone

Office Dev Zone

VPN, IP Restriction

local dev server

20

1. Developer pushes to Gitlab repo

2. Gitlab webhooktriggers Jenkins

3. Jenkins pull code from Gitlab:- source code- Dockerfile- Jenkinsfile

4. Compile source code & build Docker image, auto tag image

5. Push image to Private Docker Registry

6. Run custom script based on branch committed

CI with Jenkins & Docker

• Using Spotify docker-maven-plugin:• Save space by caching java libraries image layer (~75MB)

• Reduce network traffic & deploy time (only 200-700KB transferred)

• Docker image auto tag: {git_commit_short_code}-{branch} -> 4b4a71ef-dev

• Custom script based on branch committed:• dev: trigger another Jenkins Server to update service

• live: manual update

21

Summary

• Docker is great way to package microservices

• Building HA system is easier with Docker Swarm Mode

• CI with Jenkins will reduce delivery time, increase productivity

22

References

• http://www.slideshare.net/juminchoi/bizweb-microservices-architecture

• https://docs.docker.com/engine/swarm/

• https://www.docker.com/use-cases/cicd

23

Contact• Nguyễn Minh Khôi – DKT Technology

• Email: khoinm@dkt.com.vn

• Facebook: https://fb.com/khoinguyen84

24

Thank you!Q&A