Introduction  to  Amazon  EC2  Container  Service and  setting  up  build  pipeline  with  

ECS  and  Jenkins

Swapnil DahiphaleDevOps  Engineer,  WhiteHedge  Technologies

Agenda

• Containers• Orchestration  • Why  ECS?• What  is  ECS?• Key  Components• Typical  user  workflow• Build  pipeline  with  Jenkins  and  ECS

Containers

“au tomates  the  d ep l oyment  of  an y    appl ication  as   a   l ightweight,  portable,     sel f-­‐sufficient containerthat  wi l l  run  vi rtual lyan ywhere”

What  are containers?

OS  virtualization    

Process isolation    

Automation    

ImagesServer

Guest OS

Bins/Libs Bins/Libs

App2App1

Why  use Docker?

With Docker we can solve many problems

• “it works on my machine”• reducing build & deploy time• Infrastructure   configuration –automation!• Libs  dependency hell• Cost  control  and granularity

A  container pipeline

A  container pipelineIT Operations

BaseImage

PatchesUtilities

A  container pipelineIT Operations

BaseImage

PatchesUtilities

Ruby

Redis

Logger

A  container pipelineIT Operations Developer

BaseImage

PatchesUtilities

Ruby

Redis

Logger

App

A  container pipeline

BaseImage

PatchesUtilities

IT Operations Developer

Ruby

Redis

Logger

App

Orchestration!

Orchestration  is  a  mean  to  Automate  Manual  Process  

Orchestration

Why  Orchestration?

• Microservice  Architectures• Fault  tolerance  /  High  Availability• Scalability• Resource  Utilization

Orchestration  Tools

• Kubernetes• Docker  Swarm• Amazon  EC2  Container  Service

Why  EC2  Container  Service ?

What  is  ECS?

Building  Block  Service

What  is  ECS?

Easily  Manage  Clusters  for  Any Scale

• Nothing  to run• Complete state• Control  and monitoring• Scale

ECS  List*  and  Describe*  APIactions

What  is  ECS?

Flexible  Container Placement

• Applications• Batch jobs• Multiple schedulers

What  is  ECS?

Designed  for  use  with  other  AWS services

• Virtual  Private Cloud• Elastic  Load Balancing• Elastic  Block Store• IAM• CloudTrail

What  is  ECS?

Performance  at  scale

What  is  ECS?

Secure

Your  containers,Your  instances

What  is  ECS?

Extensible

• ComprehensiveAPIs• Open  sourceagent• Customschedulers

Key  Components

Tasks

Containers

Clusters

Container  Instances

Key  Components:  Container  Instances

Amazon  EC2  instances

Docker  daemon

Amazon  ECS  agent

Key  Components:  Clusters

Regional

Resource  pool

Grouping  of  Container  Instances

Start  empty,  dynamically  scalable

Key  Components:  Tasks

Unit  of  work

Grouping  of  related  containers

Run  on  Container  Instances

Key  components:  task definitions{

"environment": [],"name": "simple-demo","image": "my-demo", "cpu": 10,"memory": 500, "portMappings": [

{"containerPort": 80,"hostPort": 80

}],"mountPoints": [

{"sourceVolume": "my-vol", "containerPath":"/var/www/my-

vol"}

],"entryPoint": [

"/usr/sbin/apache2", "-D","FOREGROUND"

],"essential": true

},

{"name": "busybox","image":"busybox", "cpu": 10,"memory": 500, "volumesFrom": [{"sourceContainer": "simple-demo"}],"entryPoint"

: [ "sh","-c"

],"command": [

"/bin/sh -c \"while true; do/bin/date > /var/www/my-vol/date; sleep 1;done\""

],"essential": false

}

{"environment": [],"name": "simple-demo","image": "amazon/amazon-ecs-sample", "cpu": 10,"memory": 500, "portMappings":[

{"containerPort": 80,"hostPort": 80

}],"mountPoints": [

{"sourceVolume": "my-vol", "containerPath":"/var/www/my-

vol"}

],"entryPoint": [

"/usr/sbin/apache2", "-D","FOREGROUND"

],"essential": true

},

Key  components:  task definitions

Essential  to  our task

10  CPU  units  (1024  is  full  CPU),    500  megabytes  of memory

Expose  port  80  in containerto  port  80  on host

Create  and  mount volumes

{

Key  components:  task definitions"name": "busybox","image": "busybox",    "cpu": 10,"memory":  500,    "volumesFrom": [{

"sourceContainer": "simple-­‐demo"}],

"entryPoint": [    "sh","-­‐c"

],"command": [

"/bin/sh   -­‐c  \"while  true; do/bin/date  >  /var/www/my-­‐vol/date;  sleep  1; done\""

],"essential": false

}

[{"image": "tutum/wordpress-stackable", "name": "wordpress","cpu": 10,"memory": 500,"essential": true, "links": ["db"

],"entryPoint": [ "/bin/sh","-c"

],"environment": […

],"portMappings": [{"containerPort": 80,"hostPort": 80

}]

},]

From  Docker Hub

Mount  volume  from  other container

Command  to exec

Typical  user workflowI  want  to  run  aservice

Typical  user workflow

Run Instances AmazonEC2

Use  custom  AMI  with    Docker  support  and    ECS  Agent. Instances    will  register  with    default cluster.

Typical  user workflow

Create  Task Definition

Declare resourcerequirements forcontainers

Shared  Data Volume

Node.jsApp Time  of day    App

Typical  user workflow

Create Service

Declare resourcerequirements forservice

Shared  Data Volume

Node.jsApp Time  of day    App

Elastic    Load    

Balancing

X 5

Typical  user workflow

Describe Service

Demo

Build  Pipeline  with  ECS  and  Jenkins

Continuous  delivery  with Jenkins

Build image

Push image

Update service

Q&A

Thank  You