cloud foundry and openstack - a marriage made in heaven! (cloud foundry summit 2014)
DESCRIPTION
Animesh Singh, Lead Architect and Strategist at IBM. Bring the world's best IaaS to the world's best PaaS, In this talk IBM and Rackspace are going to share their experiences of running Cloud Foundry on OpenStack. The talk will focus on how CloudFoundry and OpenStack complement each other, how they technically integrate using Cloud provider interface (CPI), how could we automate OpenStack setup for Cloud Foundry deployments, and what are some of the best practices for configuring a scalable environment.TRANSCRIPT
1
Cloud Foundry and OpenStack – Marriage Made in Heaven !
Apps
Animesh Singh, Egle Sigler, Jason Anderson@animeshsingh, @eglute, @andersonljason
CF Summit, San FranciscoJune 9-11
2
What Is OpenStack
• Open Source Project for with a very large community• Open source cloud computing platform for public and private clouds• IaaS for managing compute, storage, and networking resource• Standard and massively scalable cloud operating system
3
Who Is OpenStack
• Founded by Rackspace and NASA in 2010• Open Source global community of developers, testers, operators, and users
4
OpenStack Community
76K Commits
2000 Developers
38K Tickets
55K Mail Messages
… and counting!
http://activity.openstack.org/dash/releases/index.html?data_dir=data/whole_project
5
OpenStack Architecture
Collection of well integrated projects:• Compute (Nova)• Networking (Neutron)• Object Storage (Swift)• Block Storage (Cinder)• Identity(Keystone) • Image Service (Glance)• Dashboard (Horizon)…and others
6
OpenStack Compute (Nova)
Compute is a cloud computing fabric controller, the main part of an IaaS system.
• Component based architecture: Quickly add new behaviors• Highly available: Scale to very serious workloads• Fault-Tolerant: Isolated processes avoid cascading failures• Recoverable: Failures should be easy to diagnose, debug, and rectify• Open Standards: Be a reference implementation for a community-driven api• API Compatibility: Nova strives to provide API-compatible with popular systems like
Amazon EC2
7
OpenStack Networking (Neutron)
Network connectivity as a service, a system for managing networks and IP addresses.
Advantages of software defined networking. Create and manage:• Networks and subnets• Routers• Load balancers• Gateways• Floating IPs
8
OpenStack Block Storage (Cinder)
OpenStack Block Storage (Cinder) provides persistent block-level storage devices for use with OpenStack compute instances.
• Fully integrated with Compute and Dashboard components• Allows users to manage their own storage needs• Appropriate for performance sensitive scenarios:
• database storage • expandable file systems • providing a server with access to raw block level storage
• Snapshot management provides for volume/data backup and restoration
9
OpenStack DashboardHorizon is the canonical implementation of OpenStack’s Dashboard, which provides a web based user interface to OpenStack services including Nova, Swift, Keystone, etc.
10
Along with Rackspace, IBM is also working to accelerate OpenStack Foundation success…
Contributions across all releases
Contributions to the latest Icehouse release
http://activity.openstack.org/dash/releases/index.html
11
Why Cloud Foundry?
Meets Developer’s NeedsFocus on app development,
not provisioning VMs, databases, messaging
servers, etcAgile development model
Deploy and scale in seconds
Open Cloud PlatformThere is an increasing appetite for cloud-based mobile, social and analytics applications from line-of-business executives -
drives the need for a more open cloud development platform
Compelling Community Cloud Foundry has a compelling
community and emerging ecosystem as well as a mature set
of capabilities and robustness
Platinum Founding Sponsors1,274 711k
LINES OF CODETOTAL CONTRIBUTORS
12
IBM BlueMix (powered by Cloud Foundry)http://bluemix.net
IBM initiative to develop a Platform as a Services offering
IBM and partner cloud services
Integrated DevOps with both Browser and Eclipse-based tools
Services
Lifecycle Management
(JazzHub)
ApplicationRuntime
Runtimes &Frameworks
Middleware Application Operational Mobile ExternalData
node java ruby WorklightWebSphereLiberty
Web IDE(Eclipse Orion)
Eclipse IDEApplication
CompositionEnvironment
Create & Manage Services
Test/Run Test/Run
ExploreServices
ExploreServices
IBM Bluemix ™Check In Code Check In Code
Come, experiment with our open beta at http://bluemix.net
13
Integrate !
14
PaaS
Cloud Foundry – OpenStack integration
Cloud
Integration
IaaS
UAA
Router
DEA Pool
Service Gateway Apps
Service Connector
Health Manager
Messaging
Cloud Controller
Build Packs
Service Nodes
Cloud Foundry BOSH
Cloud Provider Interface
• Static / floating ips• Persistent disks• Custom VM
Configurations• Specialized Security
groups• Outbound Internet
connectivity
15
BOSH deployment process
Deployment Manifest• Release
name/version• # VMs, job params• Stemcells to use
Stemcell• Base OS• BOSH agent
Release• Name• Software packages• Config templates• Scripts
BOSH
DeployedCloud Foundry
Virtual Machine• Configuration• Software Packages
Virtual Machine• Configuration• Software Packages
Virtual Machine• Configuration• Software Packages
Virtual Machine• Configuration• Software packages
16
Automate (Leverage the power of community) !
17
Leverage the power of Open Source Community
• Open technologies give us a tremendous amount of power and flexibility to integrate them together seamlessly
• Lets see some of the examples of what we can do with the power of Open technologies to go from bare metal hardware to a deployed Cloud Foundry ready to push apps
Open Source for Cloud Foundry and OpenStack Automation!
18
OpenStack Installation:
• Leverage the open source Chef Cloud infrastructure Automation framework• Requires information about hardware, network environment and software repositories.• Automate OpenStack installation in approximately 45 minutes
Chef for OpenStack Install Automation
Setup Compute
Setup Network
Setup Controller
Setup Storage
19
OpenStack Discovery:
• Leverage the open source Fog gem to discover OpenStack artifacts in an automated manner • Require OpenStack credentials and discover OpenStack compute and network information.
Fog for OpenStack Discovery Automation
Discover VM Configuration Sizes
Discover NetworkSubnets
Discover Network Security Rules
Discover DHCP , DNS Gateway and floating IPs
Discover Security Credentials
20
Fog for OpenStack Setup Automation
Cloud Foundry Pre-req setup on OpenStack:
• Leverage the open source Fog gem to setup Cloud Foundry requirements in an automated manner • Setup according to best practices and guidelines – still giving users the flexibility to change if desired
Create Security Credentials
Create VM configs for Router, DEAs, Cloud Controller, Service Nodes
Create network Security Rules
Setup tenant quota
21
RUBY
BOSH and Ruby for Cloud Foundry Deployment Automation
Cloud Foundry Deployment Automation
• Automate base OS image or Stemcell modification • Automate Cloud Foundry deployment manifest file genration using Ruby ERB templates• Automate upload of Cloud Foundry core release, services and runtime frameworks, followed by Cloud
Foundry deployment
Stemcell Creation/Modification and Upload
Generate BOSH and Cloud Foundry Manifest
Upload Cloud Foundry core, Services and runtime
Deploy Cloud Foundry
Deploy Microbosh
BOSH
22
Automate end to end deployments in under 3 hours !
• Previous example signify the power of Open source community• Leveraging tools like Chef, Fog, BOSH, Ruby ERB from the community can help create powerful
automation which can be repeatedly and consistently replicated in under 3 hours !
Open way is the only way!
BOSH
RUBY
23
Scale!
24
Scaling Number of Apps on Cloud Foundry
Sample sizing for 1000 small applications
Total Resources: 160 VCPU, 500GB Memory, 1.5TB of VM disk
25
Scaling OpenStack
Optimize Internal Communication• Configure OpenStack for scaled concurrency
Optimize Performance• Configure OpenStack scheduler to evenly distribute load
Setup Highly Available Architecture for PaaS workloads
Load Balancer Nodes
DataNodes
3x
Compute Nodes
15xStorage
Nodes
2x
Controller Nodes
2x 6x
26
Scaling Cloud Foundry / BOSH
Optimize Internal Communication• Configure messaging bus for VM communication
Optimized routing and bandwidth allocation• Isolate Cloud Foundry components using multiple networks
Maintain Cloud Foundry’s Highly Available Architecture
Service Gateways
10xRouters
3x
Service Nodes
15x
DEAs20x
Cloud Controllers
2x
27
Summary
Why Cloud Foundry and OpenStack are a great fit ?
• 100% Open PaaS and IaaS solutions (no vendor lock-ins) with a growing community of contributors and sponsors on both sides
• Power of Open Source community can be leveraged to automate the deployment and lifecycle management of Cloud Foundry on OpenStack
• OpenStack meets Cloud Foundry integration requirements, and is totally configurable and adaptable to handle the scale of a PaaS solution like Cloud Foundry
Bottom Line: Its indeed a match made in Heaven !
28
What next? Join us at our Silicon Valley Meetups
http://www.meetup.com/BlueMix/
http://www.meetup.com/CloudFoundry/
29
References
Find the slides presented today here:
http://www.slideshare.net/AnimeshSingh/cloud-foundry-and-openstack-marriage-made-in-heaven-35717536For more technical details, refer to the slides and video herehttp://www.slideshare.net/AnimeshSingh/optimizing-cloud-foundry-and-openstack-for-large-scale-deploymentshttps://www.openstack.org/assets/presentation-media/A-Practical-Approach-to-HA-Final.pdfhttps://www.youtube.com/watch?v=jCwtV9n_ak4https://www.youtube.com/watch?v=FsrCGkBo4Vg