amazon ec2 container service: deep dive
TRANSCRIPT
Daniele Stroppa
AWS Solutions Architect
@moviolone
Amazon EC2 Container Service:
Deep Dive
Agenda
• Cluster management
• Container scheduling
• Container deployment
• Scaling Amazon ECS
• Logging & Monitoring
• Service discovery
Cluster management
Cluster management with Amazon ECS
• Management of followers via ECS Agent
• Dispatching of sub-tasks to proper location
• Cluster state inspection
http://amzn.to/1jlHvnU
Cluster management under the hood
• Paxos-based transactional journal based data store
• Writes are committed as transaction in the journal with
order-based ID. The current value is the sum of all
transactions made as recorded by the journal.
• Reads are simply a snapshot in time of the journal. For a
write to succeed, the write proposed must be the latest
transaction since the last read.
http://bit.ly/1M9gGiv
Container scheduling
Amazon ECS Schedulers
• Task scheduler
• Run tasks once
• Batch jobs
• Service scheduler
• Load balancing
• Health management
• Scale-up and scale-down
• Update management
• AZ aware
Custom Schedulers
1. Calls the ECS List* and Describe* API operations to
determine the current state of the cluster.
2. Selects one (or more) container instances according to
the logic implemented.
3. Calls StartTask API to start a task on the selected
container instance.
http://amzn.to/1L9jmHT
Integration with Apache Mesos schedulers
• Allows you to use Apache Mesos schedulers like
Marathon and Chronos with ECS
• The ECSSchedulerDriver interprets the command given
when scheduling jobs with Mesos and starts a task with
TaskDefinition family:revision
http://bit.ly/1WXd6L3
Container deployment
CI/CD Partners
Continuous delivery with Jenkins
Build image
Push image
Update service
Continuous delivery with Jenkins
• Webhook to trigger job execution
• Docker Build and Publish plugin to build Docker image
• Amazon ECS API to update service
• Bonus – Cloudbees ECS plugin to run slave
http://amzn.to/1GbheTp
Amazon ECS CLI
• Easily create Amazon ECS clusters & supporting
resources such as EC2 instances
• Run Docker Compose configuration files on Amazon
ECS
• Available today – http://amzn.to/1jBf45a
Deploy Compose app with Amazon ECS CLI
> ecs-cli compose up
> ecs-cli compose ps
> ecs-cli compose service create
> ecs-cli compose service start
http://amzn.to/1jBf45a
Scaling Amazon ECS
Scaling with Amazon ECS CLI
> ecs-cli scale
> ecs-cli compose scale
> ecs-cli compose service scale
http://amzn.to/1jBf45a
Logging & monitoring
Logging and monitoring on Amazon ECS
• Amazon CloudWatch Logs
• Amazon CloudTrail
• Partners & third party tools
Logging with Amazon CloudWatch Logs
• Centralized logs
• Collect, set up metrics, alarms
• View as graphs
• No need to modify your containers
• One additional container:
• rsyslog
• CloudWatch Logs agent
Logging Amazon ECS API with AWS CloudTrail
{
"eventVersion": "1.03",
"userIdentity": {…},
"eventTime": "2015-10-12T13:57:33Z",
"eventSource": "ecs.amazonaws.com",
"eventName": "CreateCluster",
"awsRegion": "eu-west-1",
"sourceIPAddress": "54.240.197.227",
"userAgent": "console.amazonaws.com",
"requestParameters": {
"clusterName": "ecs-cli"
},
http://amzn.to/1LjJT8N
Logging Amazon ECS API with AWS CloudTrail
"responseElements": {
"cluster": {
"clusterArn": "arn:aws:ecs:eu-west-1:560846014933:cluster/ecs-cli",
"pendingTasksCount": 0,
"registeredContainerInstancesCount": 0,
"status": "ACTIVE",
"runningTasksCount": 0,
"clusterName": "ecs-cli",
"activeServicesCount": 0
}
},
[…]
http://amzn.to/1LjJT8N
Service discovery
Service discovery on Amazon ECS
• Amazon ECS Service Scheduler
• Amazon Route 53 private zone
• Partners & third party tools
Daniele Stroppa
AWS Solutions Architect
@moviolone
Thank you!