microservices on aws spot instances

Microservices on AWS Spot Instances

AWS Bay Area Meetup – Jan 13, 2016

Jim Bugwadia; jim@nirmata.com

High-performing IT organizations deploy 30X

more frequently with 200X

shorter lead times; they have 60X fewer failures and recover 168X faster.

2015 State of DevOps Report, Puppet Labs

Confidential 3January 14, 2016

Agenda

• AWS Spot Instances

• Microservices and Containers

• Nirmata

• Demo

Confidential 4January 14, 2016

About me

• Founder at Nirmata

• Software Developer (C++, Java, Javascript, Go)

• Large-scale distributed systems

AWS Spot Instances

Confidential 6January 14, 2016

AWS Buying Options

• On-demand Instances: hourly pricing

• Reserved Instances:• Up to 75% cheaper than on-demand pricing

• 1 – 3 year commitment

• Large upfront costs; typical breakeven at 50-80% utilization

• Spot Instances:• AWS sells excess capacity to higher bidder

• Hourly commitments at a price you name

• Can be up to 90% cheaper that on-demand pricing

• Dedicated Hosts: fully dedicated physical server

Confidential 7January 14, 2016

How Spot pricing works

• AWS determines market price based on supply and demand

• Instance is allocated to highest bidder

• Bidder pays market price

• Allocated instance is terminated (with a 2 minute warning) when market price increases above your bid price

• Diversification of instance families, instance types, availability zones increases continuity

Confidential 8January 14, 2016

c3.xlarge, 4 vCPU, 7.5 GiB; on demand price: $0.239 per Hour

Avg ~$0.03x80% of on-demand

Spot Bid Advisor: https://aws.amazon.com/ec2/spot/bid-advisor/

Spot History: https://us-west-1.console.aws.amazon.com/ec2sp/v1/spot/home?region=us-west-1

Confidential 9January 14, 2016

Spot Fleet

• Simplified bidding across instance types and regions

• Size based on number of instances, vCPU, or your own units

• Include multiple launch specifications, that vary by instance type, AMI, Availability Zone, or subnet

• Select bid strategy: Lowest price or Diversified

• API and Console!

https://console.aws.amazon.com/ec2sp/v1/spot/launch-wizard?region=us-west-1

Confidential 10January 14, 2016

AWS Spot Instances

Significant savings potential

Spot Instances are identical as On-Demand (no compromises)

But..

• Initial learning curve

• Not suitable for single point of failure applications

• Not suitable for applications coupled to infrastructure

• More complex to operate (without automation)

Microservices

Confidential 12January 14, 2016

Confidential 13January 14, 2016

An initial motivation for

Microservices was scalability

* The Art of Scalability; AKF Scale Cube

Confidential 14January 14, 2016

X-Axis Scaling

scale by replicating the entire application

Client Load Balancer Application

Application

Application

Confidential 15January 14, 2016

Y-Axis Scaling

scale by splitting the application into services

Client Load BalancerCustomers

Service

Catalog Service

Orders Service

/catalog

/customers

/orders

Confidential 16January 14, 2016

Z-Axis Scaling

Client Load BalancerCatalog

Service 3

Catalog Service 1

Catalog Service 3

/catalog [A – I]

/catalog [J – R]

/catalog [S – Z]

scale by splitting the data

Confidential 17January 14, 2016

Microservices provide Y-X axis scaling

scale by splitting the application and replicating services

Client Load BalancerCustomers

Service

Catalog Service

Orders Service

/catalog

/customers

/orders

Confidential 18January 14, 2016

Microservices provide Y-X axis scaling

scale by splitting the application and replicating services

Client Load BalancerCustomers

Service

Catalog Service

Orders Service

/catalog

/customers

/ordersBest scalability

Best availability

Best efficiency

Confidential 19January 14, 2016

1. Elastic: scales up or down independently of other services

2. Resilient: services provide fault isolation boundaries

3. Composable: uniform APIs for each service

4. Minimal: highly cohesive set of entities

5. Complete: loosely coupled with other services

19

A Microservices application is composed of several cooperating but independent services. Each Service is:

http://nirmata.com/2015/02/microservices-five-architectural-constraints/

Confidential 20January 14, 2016

Microservices Summary

Programming paradigm for the DevOps era

Builds on best practices of modular design and service-orientation

Enables agility at scale (organization, code, users)

But..

• Requires distributed programming, API foundation

• More complex to operate (without automation)

• Requires new tooling

Application Containers

Confidential 22January 14, 2016

Containers are great for packaging and managing application components!

Immutable portable application images

Standard units of operations

Separation of Dev-Ops concerns

Efficient runtime

Efficient image management

Confidential 23January 14, 2016

Container Lifecycle

For a comprehensive CLI reference: http://docs.docker.com/reference/commandline/cli/

Confidential 24January 14, 2016

Containers

Immediate benefits for any application (portability and agility)

Easy to learn and adopt

Gateway to DevOps

Runtime effeciencies

But..

• Requires separation of config from code (12 factor, etc.)

• Requires new tooling

• More complex to operate (without automation)

nirmata

Confidential 26January 14, 2016

DevOps

Public or Private Clouds

Multi-cloud Container Services

nirmata

any cloud – any device

Managed Devices

Data Centers

IT Ops

Confidential 27January 14, 2016

deploy operate optimize

Turnkey application container lifecycle management

Single control plane across public & private clouds

Highly scalable cloud-native service

Confidential 28January 14, 2016

Policy Based Orchestration

Real time Analytics

Application Blueprints

DevOps Integrations

Nirmata Control Plane

Host Agent

The Nirmata Solution

Service DB

Service Gateway

Data Plane:• Secure agents• Seamless Service Discovery• Distributed Load-balancing• Programmable routing

Host (or VM)

DNS / LB

Infrastructure

Container

Policy-based Orchestration

Microservices Infrastructure (optional)

Application Lifecycle Management

Resource Management & Scheduling

Host

Where Nirmata fits

You choose

Confidential 30January 14, 2016

Nirmata: Key Differentiators

1. Pure play application delivery & management

2. Non-intrusive and easy to use SaaS solution

3. Demand-driven Multi-Layer Scheduling

4. Split plane agent-based architecture

5. Integrated Microservices Infrastructure

6. Integrated monitoring & analytics

Demo

Confidential 32January 14, 2016

Demo Steps

1. Configure a AWS Spot Fleet Request

2. Configure a Nirmata Host Group that pools on-demand and spot instances

3. Configure scaling rules, placement rules, and constraints

4. Deploy an application across spot and on-demand instances

5. Terminate spot instances

6. Demonstrate service auto-recovery

7. Demonstrate host auto-scaling

Confidential 33January 14, 2016

Nirmata continuously monitors and optimizes application availability…

Nirmata Host Groups

Spot On-DemandSpot

1. Host Group Pooling

2. Minimum Placement Constraints

3. Spot Termination Detection

4. Host Auto-Scaling

5. Service Auto-Recovery

6. Dynamic Service Discovery & Load Balancing

Summary

Confidential 35January 14, 2016

Internet

Auto Scaling Groups

Edge Tier Mid-Tier Data Tier

Spot Fleet EBS Backed Instances

Auto-Scale Host Group

Spot Fleet Host Group

Launch ConfigHost Group

Confidential 36January 14, 2016

Key Takeways

1. AWS Spot Instances are significantly cheaper and are always available.

2. With the Microservices approach, each individual service is elastic and resilient.

3. Containers provide fast deployment times, improve utilization, and enable fully automated application operations.

4. Nirmata’s multi-cloud scheduling manages hosts and containers, provides policy-based orchestration, and maintains application availability across Spot and On-Demand instance pools!

Thank you!

https://try.nirmata.io

test drive Nirmata for free at: