migration to docker with ecs
TRANSCRIPT
Migration to Docker with ECS
ECS in Data Services
Change Drivers● Per-Project ops code (CMS2, SB2, Oracle, Mumford, Catalog Services,
Banyan, CIS, CSI, CLS, External Syndication, PRSyndication, etc.)
● Local/Cloud deployments are different
● Low cost efficiency for non-prod environments
● Slow deployments
Main Goals● Improve Development Process
● Improve Deployment Process
● Reduce Costs
● Transparent Migration
Development Process Improvements● Airplane Mode
● Fast Iterations
● Pre-committed Tests
● Local ~= Cloud
Deployment Process Improvements● Single Process for Multiple Projects
● Better EC2 Resources Utilization
● Blue/Green Deployment
● Rapid Deployment
Cost Reduction Areas● Low CI/CERT efficient ratio
● CI is mostly for cloud-adopted tests
● Instance types not optimal (CPU, Memory, Storage)
Exploration● Generalize current deployment (CloudFormation + Puppet + CLI) across all
projects (Common ops Git project + Project-specific ones)
● Custom AMIs (Packer)
● Flynn
● Docker Swarm
● EC2 Container Service (ECS)
EC2 Container Service
ECS Benefits● IAM Support
● Dynamic Resources Allocation
● Pay Only for EC2 Resources
● Scaling Based on CloudWatch
● Blue/Green Deployment
● ECR (EC2 Container Registry) Integration
● Deployment Configuration Percentage
ECS Components● Cluster
● Container Instance
● Task Definition
● Scheduler
● Service
● Task
● Container
ECS Limitations● Fixed ELB to Container Relationship (One Instance - One Service)
● Limited ELB Features
● Lack of ECS Support in CloudFormation
Dynamic Services● Dynamic private IP
addresses
● Dynamic Ports
Service Discovery● Etcd
● Consul
● Zookeeper
● Airbnb Synapse
● SkyDNS
● ...
Service Registration● Registrator
● Netflix Prana
● Airbnb Nerve
● ...
Dynamic Balancing● Consul-template
● Traefik
● Tutum
● etc
Consul + Registrator + Consul-template(HAProxy)
Container Balancing Modes Port-to-Port Balancing
APP_PORT 9080
ENV SERVICE_${APP_PORT}_TAGS PTP
Name-based Balancing
APP_PORT 9080
ENV SERVICE_${APP_PORT}_TAGS ci-banyan, ci-banyan-int
Zookeeper/Badger RegistrationProblem
● Docker private socket: 172.17.0.2:9081 cat not be reachable from outside
Solution
● Consul lookup for external socket. Socket of EC2 instance: 10.0.33.156:32457
● Application config change to allow override ZK/Badger socket
ECS CloudFormation: Missing Features● Logging support not implemented
● Deployment Configuration not implemented
● Only TCP ports can be exposed
● Task Revision update not implemented
● Inactive tasks flood Web interface
ECS Toolset● Thor - Ruby-based CLI Toolkit
● CloudFormation for ECS EC2 Resources
● Ruby AWS SDK v2 for ECS Resources
Demo
Dev Side
Dev Process Changes● Use Docker
● Provide your jar and build an image
● Mount your code (for not-compiling frameworks)
● Test cloud setup locally instead of CI
● Push your image to ECR
Demo
To Do List ● Proceed with migration
● Related services in containers (EmoDB, Polloi, etc)
● CLI improvements
● Registrator for ZK and Badger
● Better scaling
Questions?
Linkshttps://github.com/bazaarvoice/data-service-deploy
https://bits.bazaarvoice.com/confluence/display/DEV/Data+Services+Docker+Deployment+Study
https://bits.bazaarvoice.com/confluence/display/DEV/Data+Services+Container+Platform+Requirements
https://www.consul.io
https://github.com/gliderlabs/registrator
https://github.com/hashicorp/consul-template