autoscaling, chef and new relic
DESCRIPTION
Scaling up and down using rackspace cloud. How to use Heat Templates, and do blue green deployments with autoscaling and jenkins.TRANSCRIPT
FERNANDO
HONIG
DevOps & Automation
Autoscaling
+Chef +New Relic
ABOUT
ME
More than 10 years of experience
Worked at Intel, IBM, HP
@fernandohonig
http://github.com/fernandohonig
FERNANDO
HONIG
Senior Automation Engineer
RACKSPACE & DevOps Automation
CONFIGURATION
MANAGEMENT
APPLICATION
SCALE
APPLICATION
INSIGHT
YOU CODE
THE APP
WE’LL DO
THE REST
PROBLEMS
Why to Autoscale?
How/When to Autoscale?
AUTOSCALING SCENARIOS
SELF-HEALING
New Relic Webhooks
Application Stressing
Load Balancer and Autoscaling
HEAT ORCHESTRATION
Full Deployment Stack
Horizontal Scale Up
BLUE/GREEN DEPLOYMENTS
Jenkins + Chef + Autoscale
Autoscaling+Chef +New Relic
AUTOSCALING: What?
The act of changing the
pool of resources
able to serve requests
without manual intervention
PROBLEMS: Why?
PROBLEMS: Why?
AUTOSCALING: Strategy
SEVERITY OF SPIKE:
Is this a Reddit spike or a casual increase?
TIME TO PROVISION:
Benchmark and test the time it takes to create new infrastructure.
EARLY WARNING:
What is the earliest observed anomaly in a monitoring system that predicts
this spike?
PER NODE CAPACITY:
Load test individual nodes and your baseline platform to better understand
how much natural elasticity is in your platform.
PROBLEMS: How? When?
SCALE UP
PROBLEMS: How? When?
SCALE
DOWN
NEW RELIC
PROBLEMS: Scaling down?
SCALING DOWN,
SESSIONS, NEW
ARCHITECTURE
LOADBALANCER
WEBSITE
WWW
LOADBALANCER
WEBSITE
WWW
AUTOSCALING: Early warning!
Monitor applications performance using NewRelic
Use NewRelic APDEX score to monitor application
performance
Scale up on demand based on NewRelic stats
(Self heal)
When threshold breached fire alert to webhook
Auto register node with application in NewRelic
Scale down when compute no longer needed
AUTOSCALING:
EARLY
WARNING!
TOOLS!
CONFIG MGMT CODE MGMT MONITORING
LOADBALANCER
WEBSITE - SERVERS
VISITORS
WEBSITE STARTS HAVING MORE LOAD
STEP 1: START
WEBSITE
WWW
NEW RELIC
NEW RELIC APP MONITORING TRIGGERS
A WEBHOOK FIRING AN AUTOSCALE EVENT
STEP 2: ALERT
WWW
NEW RELIC AUTO SCALE SERVICE LOADBALANCER
WEBSITE - SERVERS
AUTOSCALING GROUP LAUNCHES MORE
COMPUTE CAPACITY
STEP 3: SCALE UP
WEBSITE+
SERVERS BUILDING
WEBSITE – SERVERS ACTIVE
WWW
NEW RELIC AUTO SCALE SERVICE LOADBALANCER
STEP 4: CONFIGURE
CHEF AUTOMATICALLY CONFIGURES
THE NEW SERVERS
WEBSITE - SERVERSWEBSITE - SERVERS
chefLOADBALANCER
WEBSITE
WWW
STEP 5: COMPLETE
THE NEW SERVERS ARE FULLY
DEPLOYED & IN PRODUCTION
NEW RELIC MONITORS THE
WEBSITE & PROVIDES
CUSTOMER WITH
PRO-ACTIVE INSIGHT
CUSTOMER NEW RELIC
WEBSITE - SERVERS
LOADBALANCER
WEBSITE
WWW
SERVERS REMOVED
STEP 6: DOWN
WHEN APPLICATION RECOVERED
OLD SERVERS ARE BEING REMOVED
WEBSITE – SERVERS ACTIVE
LOADBALANCER
WEBSITE
WWW
DEMO
AUTOSCALING: Heat Orchestration
Declare the infrastructure in one simple text file (YAML)
Create, update, connect and manage groups of cloud resources and their software components as a single unit (stack)
Automated, repeatable
HEAT
ORCHESTRATION
PROBLEM: Environments
DEV = UAT = PRD:
Do we have the same architecture across all environments?
TROUBLESHOOTING:
Is the problem related to the infrastructure or code?
QUICK ENVIRONMENT TEST:
If we need an environment just for a few hours, or for a specific purpose?
TOOLS!
CONFIG MGMT CODE MGMT MONITORING
heat_template_version: 2013-05-23
description:
# a description of the template
parameters:
# declaration of input parameters
resources:
# declaration of template resources
outputs:
# declaration of output parameters
HEAT
ORCHESTRATION
HEAT ORCHESTRATION: Code!
http://goo.gl/xApJg1
LOADBALANCER
STEP 1: DEPLOY
THE TEMPLATE CONTAINS THE
AUTOSCALING GROUP, THE LB,
THE POLICIES AND WEBHOOKS
TEMPLATE CONTAINS ALL THE
CHEF INFORMATION TO
BOOTSTRAP THE NODE
WITHIN THE RIGHT
ENVIRONMENT AND ROLE.
CUSTOMER
STACK
ELK
LOADBALANCER
TEMPLATE CONTAINS ALL THE
CHEF INFORMATION TO
BOOTSTRAP THE NODE
WITHIN THE RIGHT
ENVIRONMENT AND ROLE.
CUSTOMER
STACK
ELKSTEP 2: SCALE
THE TEMPLATE CONTAINS THE
AUTOSCALING GROUP, THE LB,
THE POLICIES, AND WEBHOOKS
AUTO SCALE SERVICE
STEP 3: CONFIGURE
THE STACK WILL BE CONFIGURED
AND ADDED TO THE LB BASED ON
CHEF AUTOMATION
chef LOADBALANCER
TEMPLATE CONTAINS ALL THE
CHEF INFORMATION TO
BOOTSTRAP THE NODE
WITHIN THE RIGHT
ENVIRONMENT AND ROLE.
STACK
ELK
DEMO
AUTOSCALING: B/G Deployment
PROBLEM: Deployments
TIME TO MARKET:
Why spend 8hs for doing a deployment? Can’t I just code my app?
AUTOMATION:
I am human, I forgot to execute this step, rollback is needed.
BRAND NEW ENVIRONMENT:
What if my old component is not compatible with the new one?
Increase efficiency in the application release cycle
Allow the customer to focus on their core business
Fully automated one click deployment
Faster delivery of code to production
Easy roll back and in timely manner
BLUE-
GREEN
DEPLOYMENTS
TOOLS!
CONFIG MGMT CODE MGMT WORKFLOW
LOADBALANCER
CUSTOMER
CUSTOMER DEVELOPS NEW VERSION
OF THEIR WEBSITE (WEBSITE 2.0)
STEP 1: DESIGN
WEBSITE 1.0
WWW
WEBSITE1.0 - SERVERS
STEP 2: DEPLOY
JENKINS
CUSTOMER DEPLOYS NEW VERSION
OF THEIR WEBSITE (WEBSITE 2.0)
NO FURTHER INTERACTION REQUIRED
FULLY AUTOMATED FROM HERE ON OUT
LOADBALANCER
CUSTOMER
WEBSITE 1.0
WWW
WEBSITE1.0 - SERVERS
STEP 3: SCALE UP
JENKINS
AUTO SCALE SPINS UP NEW SERVERS
WEBSITE 1.0 - SERVERS
AUTO SCALE SERVICE
WEBSITE 2.0 - SERVERS
LOADBALANCER
WEBSITE 1.0
WWW
STEP 4: CONFIGURE
CHEF AUTOMATICALLY CONFIGURES
THE NEW SERVERS
WEBSITE 1.0 - SERVERSWEBSITE 2.0 - SERVERS
chefLOADBALANCER
WEBSITE 1.0
WWW
STEP 5: SCALE DOWN
JENKINS
JENKINS INITITIATES THE AUTOMATIC
SPIN DOWN OF THE OLD SERVERS
WEBSITE 2.0 - SERVERS
AUTO SCALE SERVICE
WEBSITE 1.0 SERVERS
LOADBALANCER
WEBSITE 1.0
WWW
WEBSITE 2.0 - SERVERS
STEP 6: PRODUCTION
THE NEW WEBSITE IS FULLY
DEPLOYED & IN PRODUCTION
NEW RELIC MONITORS THE WEBSITE
& PROVIDES CUSTOMER WITH
PRO-ACTIVE INSIGHT
CUSTOMER NEW RELIC
LOADBALANCER
WEBSITE 2.0
WWW
DEMO
SUMMARY
Autoscaling and Strategies behind
Application Architecture and scaling
Deployment enhancement using AS
Q&A
WHAT’S
NEXT?
www.rackspace.co.uk/devops
THANK YOU
RACKSPACE® | 1 FANATICAL PLACE, CITY OF WINDCREST | SAN ANTONIO, TX 78218
US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM
© RACKSPACE LTD. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED S TATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM
RACKSPACE® | 5 MILLINGTON ROAD | HAYES, UNITED KINGDOM UB3 4AZ
UK SALES: +44 (0)20 8712 6507 | UK SUPPORT: 0800 988 0300 | WWW.RACKSPACE.CO.UK
www.rackspace.co.uk/devops