autoscaling openstack natively with heat, ceilometer and lbaas
Post on 28-Jul-2015
217 Views
Preview:
TRANSCRIPT
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
SpeakersJason GrimmConsulting Systems EngineerCisco OpenStack Private Cloud
Sharmin ChokseyTechnical LeaderCisco Cloud Services
Shixiong ShangChief Technology OfficerNephos6
AgendaIntroductions <
About The Workshop <
Environment Setup <
Background & Use Cases <
Services Enabling Autoscale <
Hands-On <
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
▪ We all work for or with Cisco, but this is not a marketing workshop▪ Highly technical and all content is 100% FOSS (Free and Open-Source Software)▪ All Materials available at https://github.com/grimmtheory/autoscale
▪ Please connect to http://cisco.webex.com/rooms/jasgrimm▪ Chat, voting, questions, recording, etc.▪ Allows us to see when everyone is ready to proceed▪ Session recording will be made available after the summit
▪ Please work with your peers at your table!▪ Bandwidth constraints and technical issues mean might not get a test environment up in time▪ Some of you may have more experience with this technology and can assist others▪ General collaboration and discussion
Workshop Tips and Logistics
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Content and ScheduleIntroductions
About The Workshop** Environment Setup **
Autoscale Background and Use CaseServices Enabling Autoscale
Hands-On WorkshopLab 1
Lab 2.1Lab 2.2Lab 3
Q & AOpen Discussion
Close
30 Minutes(or less)
45 Minutes(or more)
15 Minutes(or less, as needed)
Environment SetupVirtual Box, Ubuntu, DevStack
Install test environmentInstall content location and examplesTest environment logical architectureTest environment configurationTest environment build process
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Install Test Environment…InstructionsBefore we can begin working with LBaaS, autoscale, etc. We need an environment.
For this purpose we’ve chosen a locally installed environment due to past experiences with WIFI coverage and bandwidth at OpenStack summits.
1) Mount media
2) Copy data folder over locally
3) Install VirtualBox
4) Install Vagrant (Optional)
5) Boot Virtual Machine
DVD Local Repo USB Key Internet
HTTP HTTPMOUNTMOUNT
Step 1Mount Data
Step 2Copy Data
Step 3Install Apps
Step 4Boot VM
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Install content location and examples
Also available at:• My VM - http://192.168.1.100/autoscale• My Cloud Server - http://38.84.67.227/autoscale• VirtualBox - https://www.virtualbox.org/wiki/Downloads
Vagrantfile
Git clone example
Wget example
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Test environment logical architecture
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Test environment configurationDisabled
Trove
Sahara
Cinder
Nova Network
Tempest
Swift
Enabled
MySQL
RabbitMQ
Keystone
Horizon
Glance
Neutron
Ceilometer
Heat
OS & OpenStack Configuration$HOME = /home/vagrant/devstack
Operating System = Ubuntu 14.04.xOpenStack version = “stable/kilo”
eth0 = NAT, default GWeth1 = Management / API networketh2 = Neutron / Bridge network
Open vSwitchL3 / DHCP / Floating IP
“Flat” networking also availableNo tenant VLANs, FWaaS or VPNaaS
Vagrant / VirtualBox ConfigurationVagrant v.1.7.2
VirtualBox 4.3.28eth0 = NAT
eth1 = Host Only 192.168.33.2 /24eth2 = Host Only 0.0.0.0
VT-x off, APIC-IO off, promiscuous on eth2
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Test environment build processMethodology• Tools-based install and config of the OS,
DevStack and post-install tasks.• Automate difficult configuration of services
and pre-requisites (Neutron, Heat, Ceilometer, OVS, custom images, etc.)
Build Options• Base - OS install & DevStack configured• Staged - Base + DevStack install completed
and basic configuration (security groups, networks and subnets created, etc.)• Complete - Staged + all objects created
(LBs, VIPs, Ceilometer alarms, Heat templates, etc.)
CREATE MACHINE2 vCPU, 4 GB RAM, VT-x, APIC-IO
3 NICs - eth0 NAT, eth1 API, eth3 VMsPromiscuous mode for Neutron Bridge
DEVSTACK PRE INSTALLInstall pre-requisitesClone git repositories
Configure local.conf, bridges, routes users, groups,iptables, ovs, sysctl, permissions, etc.
INSTALL OSBase Install
Patch / UpdateInstall VBox extensions
DEVSTACK INSTALLInstall and monitorDevStack Install
DEVSTACK POST INSTALL BASIC
Add DNS to subnetsEdit security groups
Create / install SSH keys
DEVSTACK POST INSTALL ADV.
Create load balancersAdd members to pool
Create VIPs
COMPLETESnapshot the
virtual machine
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Autoscale Background and Use CasesWhat is Autoscaling…?“…a method used in cloud computing, whereby the amount of resources in a server farm scales automatically based on the load on the farm. It is closely related to, and builds upon, the idea of load balancing.”
Who uses Autoscaling…?Any workload that benefits from programmatic scaling up or down as load dictates.
Particularly true from a cost perspective when high workloads are very infrequent.
Some examples are:• Retail - Black Friday, holidays, video game / movie launch, etc.• Academic / Research - HPCC• Media - Video / Audio rendering• Analytics - Big data, data mining, data modeling• Security - Federal or private data heuristic, cyber defense, etc.
How does Autoscaling work…?
ServerLoad
StressMeter Alarm Action Server
Result+ + + +
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
HEATOpenStack Orchestration Service
Heat makes “clouds” rise
heat The heat tool is a CLI which communicates with the heat-api to execute AWS CloudFormation APIs. End developers could also use the heat REST API directly.
heat-api The heat-api component provides an OpenStack-native REST API that processes API requests by sending them to the heat-engine over RPC.
heat-api-cfn The heat-api-cfn component provides an AWS Query API that is compatible with AWS CloudFormation and processes API requests by sending them to the heat-engine over RPC.
heat-engine The heat engine’s main responsibility is to orchestrate the launching of templates and provide events back to the API consumer.
Infrastructure resource abstraction and orchestration:• Physical compute Power• Physical switches• Physical block Storage• …
OpenStack resource abstraction and orchestration:• VM Instances• Virtual Networks• Virtual Routers• …
“Typically” Virtual(See Ironic Project)
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Ceilometer Overview
1"
Polling"agents"
Meter" Transformer(s)"
New"Meter"
Publishing"Pipeline"
API" Database"
AMQP"
Receiver"
Collector"
Other"OpenStack""Service"
alarm"evaluator"
Publisher(s)"
alarm"evaluator"
Alarm"
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Ceilometer and Neutron LBaaS
2"
Polling"agents"
Meter"
Neutron"(LBaaS)"
Neutron"API"
LBaaS"Default"Meters" Type" Unit"
network.services.lb.ac0ve.connec0ons1 gauge1 connec0on1
network.services.lb.incoming.bytes1 cumula0ve1 B1
network.services.lb.total.connec0ons1 cumula0ve1 connec0on1
network.services.lb.vip1 gauge1 vip1
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Ceilometer and Neutron LBaaS
3"
Transformer(s)"
New"Meter"
Publishing"Pipeline"
Publisher(s)"
sinks:&&&&&'&name:&lb_total_conn&&&&&&&transformers:&&&&&&&&&&&'&name:&"rate_of_change"&&&&&&&&&&&&¶meters:&&&&&&&&&&&&&&&&&source:&&&&&&&&&&&&&&&&&&&&map_from:&&&&&&&&&&&&&&&&&&&&&&&&name:&"network.services.lb.total.connec;ons"&&&&&&&&&&&&&&&&&&&&&&&&unit:&"connec;on"&&&&&&&&&&&&&&&&&target:&&&&&&&&&&&&&&&&&&&&&map_to:&&&&&&&&&&&&&&&&&&&&&&&&&name:&"network.services.lb.total.connec;ons.rate"&&&&&&&&&&&&&&&&&&&&&&&&&unit:&"connec;ons/s"&&&&&&&&&&&&&&&&&&&&&type:&"gauge"&&&&&&&publishers:&&&&&&&&&&&'&no;fier://&
sources:&&&&&'&name:&lb_total_conn&&&&&&&interval:&60&&&&&&&meters:&&&&&&&&&&&'&"network.services.lb.total.connec;ons"&&&&&&&sinks:&&&&&&&&&&&'&lb_total_conn&
Configuration file: /etc/ceilometer/pipeline.yaml
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Ceilometer and Neutron LBaaS
4"
API" Database"
Receiver"
Collector"
ceilometer sample-list -m network.services.lb.total.connections.rate !+--------------------------------------+--------------------------------------------+-------+--------+---------------+---------------------+ !| Resource ID | Name | Type | Volume | Unit | Timestamp | !+--------------------------------------+--------------------------------------------+-------+--------+---------------+---------------------+ !| 9d944aec-7d66-46b5-a71b-434e4e9fa98f | network.services.lb.total.connections.rate | gauge | 0.0 | connections/s | 2015-05-13T02:45:02 | !| 9d944aec-7d66-46b5-a71b-434e4e9fa98f | network.services.lb.total.connections.rate | gauge | 0.0 | connections/s | 2015-05-13T02:44:02 | !| 9d944aec-7d66-46b5-a71b-434e4e9fa98f | network.services.lb.total.connections.rate | gauge | 0.0 | connections/s | 2015-05-13T02:43:02 | !+--------------------------------------+--------------------------------------------+-------+--------+---------------+---------------------+ !!!!ceilometer statistics -m network.services.lb.total.connections.rate !+--------+---------------------+---------------------+-----+-----+-----+-----+-------+----------+---------------------+---------------------+ !| Period | Period Start | Period End | Max | Min | Avg | Sum | Count | Duration | Duration Start | Duration End | !+--------+---------------------+---------------------+-----+-----+-----+-----+-------+----------+---------------------+---------------------+ !| 0 | 2015-05-13T02:16:02 | 2015-05-13T02:47:02 | 0.0 | 0.0 | 0.0 | 0.0 | 32 | 1860.0 | 2015-05-13T02:16:02 | 2015-05-13T02:47:02 | !+--------+---------------------+---------------------+-----+-----+-----+-----+-------+----------+---------------------+---------------------+ !
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Ceilometer and Neutron LBaaS
5"
API"
alarm"evaluator"
alarm"evaluator"
Alarm"
ceilometer alarm-threshold-create --name lb_conn_rate_hi \ !--description 'lb connection rate is too high' --severity critical --enabled True \ !--meter-name network.services.lb.total.connections.rate \ !--period 60 --evaluation-periods 3 --statistic avg --comparison-operator gt --threshold 2.0 \ !--repeat-actions True !
ceilometer alarm-list !+----------+-----------------+--------+----------+---------+------------+-----------------------------------------------------------------+ !| Alarm ID | Name | State | Severity | Enabled | Continuous | Alarm condition | !+----------+-----------------+--------+----------+---------+------------+-----------------------------------------------------------------+ !| uuid | lb_conn_rate_hi | ok | critical | True | True | network.services.lb.total.connections.rate > 2.0 during 3 x 60s | !+----------+-----------------+--------+----------+---------+------------+-----------------------------------------------------------------+ !
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Ceilometer and Heat
6"
Polling"agents"
Meter" Transformer(s)"
New"Meter"
Publishing"Pipeline"
API" Database"
AMQP"
Receiver"
Collector"
alarm"evaluator"
Publisher(s)"
alarm"evaluator"
Alarm"
Heat"
‹#›
25
Auth Model• Icehouse• Security Isolation• Keystone Domain “heat”• Stack Domain Users managed by
heat_admin_domain• Pre-configured in devstack• Requires Keystone v3
Resource Groups• Icehouse• De-coupling hierarchical nested
stacks
Stack Snapshot/Rollback
Provider Resources• Icehouse• Useful in Template composition
TripleO Heat Templates• Leverages Resource Groups,
Provider Resources and SoftwareConfig Resource
Key enhancements in Icehouse and
• Kilo
SoftwareConfig Resources• Icehouse• Useful in Template composition
‹#›26
Heat Autoscaling - Lab Exercise - 1 (Simple-Server)
Spawn a VMInjects an ssh keySecurity groupsVM port fixed ip + security groupFloating ip associated it with the fixed ipInstalls a simulated http server via the -init user-data script
heat stack-create simple-stack -f simple-server.yaml --parameters \"key_name=<key_name>\;node_name=<node_name>\;node_server_flavor=<node_server_flavor>\;node_image_name=<node_image_name>;\;floating_net_id=<floating_net_id>;\;private_net_id=<private_net_id>;\;private_subnet_id=<private_subnet_id>"
User-definedConstrained and/or default valuesType/Description
Fundamental blocks
Consumed by the Heat engine to
orchestrate
‹#›27
Heat Autoscaling - Lab Exercise - 2 Part 1 (Load balancer)
lb vip :: floating ip
curl -X GET http://128.107.2.111503 Service Not Available
‹#›28
Heat Autoscaling - Lab Ex - 2 Part 2 (Adding Members)web-server.yaml environment.yaml
heat stack-create lb-members-stack -f lb-members.yaml \-e environment.yaml --parameters \"key_name=<key_name>\;node_name=<node_name>\;node_server_flavor=<node_server_flavor>\;node_image_name=<node_image_name>;\;floating_net_id=<floating_net_id>;\;private_net_id=<private_net_id>;\;private_subnet_id=<private_subnet_id>;\;pool_id=<pool_id>\;capacity_count=<capacity_count>"
curl -X GET http://128.107.2.111Welcome to 192.168.5.116
curl -X GET http://128.107.2.111Welcome to 192.168.5.117
Nested Resource
Identically Configured
lb-members.yaml
Provider Resource
‹#›
Scales arbitrary # of resources
Resource to be scaled
29
Heat Autoscaling - Lab Ex - 3 (Autoscaling)lb-autoscale-members.yaml
Creates an Alarm
on a
Action to be taken upon Alarm
Raise
Action to be executed against
resource
Connection rate crosses 3 per second for a consecutive period of 60 seconds each, the lb_connection_rate_high will be raised and a scaleup_policy will get executed
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Summit Lab / Workshop
▪ Materials for the presentation are available for download at https://github.com/grimmtheory/autoscale
DevStack
▪ DevStack Configuration Guide – http://docs.openstack.org/developer/devstack/configuration.html
Ceilometer
Heat
▪ Heat main page - https://wiki.openstack.org/wiki/Heat
▪ Heat developer documentation -http://docs.openstack.org/developer/heat/ Heat operations documentation
Heat main github repository
Heat templates github respository
Heat user template guide
Heat developer template guide
▪ Heat Orchestration Template (HOT) guide – http://docs.openstack.org/developer/heat/template_guide/hot_guide.html
▪ Heat on DevStack guide – http://docs.openstack.org/developer/heat/getting_started/on_devstack.html
References
Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved.
References
Title LinkAutoscale workshop repository https://github.com/grimmtheory/Heat: main page https://wiki.openstack.org/wiki/HeatHeat: developer documentation http://docs.openstack.org/developer/heat/ Heat: operations documentation Heat main github repository
top related