openstack - eventbooking.stfc.ac.uk · • bright integrates with ceph, ... • different base...
TRANSCRIPT
![Page 1: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/1.jpg)
OpenStack: we drink our own Champagne
OpenStack: we drink our own Champagne
Teun Docter Software developer
Teun Docter Software developer
![Page 2: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/2.jpg)
About Bright Computing
Enterprise infrastructure software company
Founded in 2009
HQ in Amsterdam & San Jose
Bare-metal & cloud provisioning, monitoring,
management of clustered systems in the data center:
HPC, Hadoop, OpenStack, storage, database and other
“clustered” systems
Customers: Boeing, ING Bank, Sanofi, NASA, DoD, DoE,
Stanford, Oracle, HP, Intel, Sinopec and >400 more
Worldwide resellers: Dell, Cisco, Cray, Huawei, Acer,
DataDirect Networks and >50 more
![Page 3: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/3.jpg)
UK resellers
![Page 4: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/4.jpg)
server
farms Hadoop
clusters
HPC
clusters
private
clouds
storage
clusters
database
clusters
Trends
- Centralization
- Consolidation
- Standardization
- Cloud
- General purpose
clusters
- Private clouds
- Across on-premise
and public cloud
![Page 5: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/5.jpg)
5
About Bright Cluster Manager
Deployment
Provisioning
Dashboards
Monitoring
Visualization
Health checking
Workload
management
Cloud extension
Environment specific
• HPC
• Hadoop
• Private cloud
![Page 6: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/6.jpg)
6
About Bright Cluster Manager
Deployment
Provisioning
Dashboards
Monitoring
Visualization
Health checking
Workload
management
Cloud extension
Environment specific
• HPC
• Hadoop
• Private cloud
![Page 7: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/7.jpg)
7
About Bright Cluster Manager
Deployment
Provisioning
Dashboards
Monitoring
Visualization
Health checking
Workload
management
Cloud extension
Environment specific
• HPC
• Hadoop
• Private cloud
![Page 8: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/8.jpg)
8
About Bright Cluster Manager
Deployment
Provisioning
Dashboards
Monitoring
Visualization
Health checking
Workload
management
Cloud extension
Environment specific
• HPC
• Hadoop
• Private cloud
![Page 9: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/9.jpg)
9
About Bright Cluster Manager
Deployment
Provisioning
Dashboards
Monitoring
Visualization
Health checking
Workload
management
Cloud extension
Environment specific
• HPC
• Hadoop
• Private cloud
![Page 10: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/10.jpg)
10
About Bright Cluster Manager
Deployment
Provisioning
Dashboards
Monitoring
Visualization
Health checking
Workload
management
Cloud extension
Environment specific
• HPC
• Hadoop
• Private cloud
![Page 11: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/11.jpg)
11
About Bright Cluster Manager
Deployment
Provisioning
Dashboards
Monitoring
Visualization
Health checking
Workload
management
Cloud extension
Environment specific
• HPC
• Hadoop
• Private cloud
![Page 12: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/12.jpg)
12
About Bright Cluster Manager
Deployment
Provisioning
Dashboards
Monitoring
Visualization
Health checking
Workload
management
Cloud extension
Environment specific
• HPC
• Hadoop
• Private cloud
node001
head node node002
node003
head node
node001 node002 node003
node004
node005
node006
node007
![Page 13: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/13.jpg)
13
About Bright Cluster Manager
Deployment
Provisioning
Dashboards
Monitoring
Visualization
Health checking
Workload
management
Cloud extension
Environment specific
• HPC
• Hadoop
• Private cloud
![Page 14: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/14.jpg)
Bright Cluster Manager
&
OpenStack
Bright Cluster Manager
&
OpenStack
Confidential. Do not distribute beyond your organization. Copyright 2013.
![Page 15: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/15.jpg)
15
Managing OpenStack Clusters
• Managing OpenStack Clusters even more difficult then other types of clusters
• Without proper infrastructure, OpenStack will not be able to run
• Setting up OpenStack manually is often extremely complex for production setups
• Bright Cluster Manager provides single-pane-of-glass to manage and monitor all aspects of OpenStack cluster
• Includes:
• Hardware (set up, configuration, monitoring)
• Operating system (provisioning, updates)
• OpenStack installation
• OpenStack configuration
• Bright Cluster Manager provides perfect environment for OpenStack to run on
![Page 16: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/16.jpg)
16
Ease of Installation
• Deploying OpenStack private clouds made simple
• Bright pre-packages OpenStack (Icehouse) on the DVD
• Bright installs Linux, OpenStack & itself
• OpenStack can be enabled during installation or afterwards
• OpenStack will run as services on top of Bright cluster
• Bright configures, manages and monitors physical cluster (hardware + OS + other middleware)
• Virtual Machines are managed through OpenStack
• Support various network setups (VLAN, VxLAN, flat, etc.)
• Bright integrates with Ceph, can be used for OpenStack
• Configuration wizard to guide the setup process
![Page 17: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/17.jpg)
17
![Page 18: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/18.jpg)
18
![Page 19: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/19.jpg)
19
![Page 20: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/20.jpg)
20
![Page 21: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/21.jpg)
21
![Page 22: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/22.jpg)
22
![Page 23: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/23.jpg)
23
![Page 24: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/24.jpg)
24
![Page 25: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/25.jpg)
25
![Page 26: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/26.jpg)
26
![Page 27: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/27.jpg)
27
![Page 28: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/28.jpg)
28
![Page 29: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/29.jpg)
29
![Page 30: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/30.jpg)
30
![Page 31: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/31.jpg)
31
![Page 32: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/32.jpg)
32
![Page 33: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/33.jpg)
33
![Page 34: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/34.jpg)
34
![Page 35: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/35.jpg)
35
![Page 36: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/36.jpg)
36
OpenStack Configuration
OpenStack configuration through roles
• OpenStack consists of several services to accomplish specific tasks
• Tasks can be assigned by assigning roles to nodes
• Example roles:
Compute (Nova) Image (Glance)
Block Storage (Cinder) Dashboard (Horizon)
Auth (Keystone) Networking (Neutron)
• Assigning/unassigning role will:
• Write out config file based on role parameters
• Start/stop/monitor relevant services
• Add/remove entries to Keystone service registry
• OpenStack configuration is dynamically updated with cluster changes (e.g. hostname, network settings)
![Page 37: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/37.jpg)
37
USE CASE: problem
Developers need clusters to test their code
Support engineers need clusters to reproduce issues
Need to quickly spin up clusters with:
• Different version of Bright, including custom builds
• Different base distributions (eg RedHat, CentOS, SUSE, etc.)
• Different types of configurations
Limited amount of hardware available
Manual setup becomes tedious and time consuming
![Page 38: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/38.jpg)
38
USE CASE: solution
We drink our own Champagne
Build our own private cloud
Runs Bright to manage the hardware and OS
Use Bright to deploy and configure OpenStack
Create set of end-user tools to spin up virtual clusters
• Command-line driven
• Make use of OpenStack API’s
• Use web based dashboards for console access
Each user has an isolated environment
Users have full control over their virtual clusters
Virtual clusters directly accessible from office network
Load is balanced automatically
Capacity can now be expanded easily
![Page 39: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/39.jpg)
39
![Page 40: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/40.jpg)
40
![Page 41: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/41.jpg)
41
![Page 42: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/42.jpg)
42
![Page 43: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/43.jpg)
43
USE CASE: plans & considerations
Automated virtual head node installation
Automated nightly installations
Head node installation using volume snapshots
Works in POC and allows big clusters to spin up in minutes
Make sure you have a fast network
DNS integration
![Page 44: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/44.jpg)
44
OpenStack in HPC
• Improved flexibility
• User can manage their code, libs, etc.
• OpenStack Ironic also allows non-virtualized instances to be created
• SR-IOV allows HPC resources (e.g. InfiniBand and GPU) to be made available directly to VMs at near-native performance (work in progress)
• Container/docker integration in OpenStack
![Page 45: OpenStack - eventbooking.stfc.ac.uk · • Bright integrates with Ceph, ... • Different base distributions (eg RedHat, CentOS, SUSE, ... PowerPoint Presentation](https://reader031.vdocuments.mx/reader031/viewer/2022022015/5b504de07f8b9a346e8e33ed/html5/thumbnails/45.jpg)
45
Conclusion
• Bright provides tried & tested method of cluster management
• Hundreds of clusters world-wide being managed using Bright Cluster Manager
• Inclusion of OpenStack management capabilities provides complete solution for setup, management & monitoring of OpenStack based private clouds.
• Single pane of glass for management of cluster & OpenStack
• Especially well suited for clusters that run various workloads (e.g. OpenStack, HPC compute, Hadoop)
• Very scalable (i.e. thousands of nodes)