jose castro leon cern – it/ois cern agile infrastructure infrastructure as a service
TRANSCRIPT
![Page 1: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/1.jpg)
Jose Castro Leon
CERN – IT/OIS
CERN Agile Infrastructure
Infrastructure as a Service
![Page 2: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/2.jpg)
Cloud Computing
2
• On-Demand Self-Service• Scalability and Efficiency• Resource Pooling• Rapid elasticity
CERN Agile Infrastructure
![Page 3: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/3.jpg)
Infrastructure as a Service
• Objectives– 90% of hardware virtualized– 300,000 VMs needed– Merge the 2 IT evaluation projects in this area
• Lxcloud & CVI
• Infrastructure as a Service (IaaS)– More efficient use of our hardware– Better tracking of usage– Enable remote management for new data centre– Consolidate support to a single service– Support potential new use cases (PaaS)
3CERN Agile Infrastructure
![Page 4: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/4.jpg)
Openstack
• Openstack provides an operating platform, or toolkit, for orchestrating clouds.
• Features– Multi Site– Scalable– Cloud interfaces
• Position ourselves as another Cloud Provider
4CERN Agile Infrastructure
![Page 5: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/5.jpg)
Openstack Community
• >150 companies supporting Openstack
• Find the complete list at:http://www.openstack.org/community/companies/
5CERN Agile Infrastructure
![Page 6: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/6.jpg)
Openstack @ CERN
Compute Scheduler
NetworkVolume
Registry Image
KEYSTONEHORIZON
NOVA
GLANCE
6CERN Agile Infrastructure
![Page 7: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/7.jpg)
Nova @ CERN
• Nova– Instance & Volume Management
• 2 APIs: OpenStack and Amazon EC2• Scheduler decides the Hypervisor
• Nova @ CERN– KVM as Hypervisor– GlusterFS for High Availability– Integration with existing network DB (LanDB)
• MAC to IP mapping• Instance to Network IP subset mapping
7CERN Agile Infrastructure
![Page 8: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/8.jpg)
Glance @ CERN
• Images Management• Configurable storage backend
– Local Storage– Openstack Swift– GlusterFS
• Multi-format image registry– RAW– VHD (Hyper-V)– VDI (VirtualBox)– QCOW2 (Qemu/KVM)– VMDK (VMWare)– OVF (VMWare, others)
8CERN Agile Infrastructure
![Page 9: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/9.jpg)
Keystone @ CERN
• Cloud Identity Service– Authentication– Authorization– Service Catalog
• Role Based Access Control• Identity Backends
– LDAP: Integrate with CERN Active Directory– SQL Database possible
9CERN Agile Infrastructure
![Page 10: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/10.jpg)
Horizon @ CERN
• Graphical UI for Openstack– Django application– Plug-in architecture– Highly integrated with Keystone– Self-service for end-users
• Integrate with CERN SSO– Shibboleth authentication
10CERN Agile Infrastructure
![Page 11: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/11.jpg)
Applications
• Different services consume IaaS resources– Server consolidation, "user VMs"– Batch services– Pilot frameworks– BOINC
• With different requirements (QoS)– Management, Storage, Availability, Flexibility– Disaster Recovery
• Scheduler is needed for automation• Monitor the resource usage
11CERN Agile Infrastructure
![Page 12: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/12.jpg)
Scheduling Architecture
12CERN Agile Infrastructure
![Page 13: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/13.jpg)
Scheduling & Accounting
• One single entry point for all services• Decouple applications from the IaaS• Distributed Computer Centre• Resource usage tracking
– Hardware usage (CPU,Memory,Disk)– Software usage (Process monitoring)
• Auditing & Billing
13CERN Agile Infrastructure
![Page 14: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/14.jpg)
IaaS Configuration
• Production nodes based on SLC6 + EPEL– Test deployment based on Fedora 16
• All hypervisors are registered in Foreman– Automatic hypervisor provisioning
• Hypervisors use Puppet to configure themselves– Puppetlabs modules for Openstack
• CERN Customization– Network DB registration
14CERN Agile Infrastructure
![Page 15: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/15.jpg)
Experiences
• Maintain the project linked with upstream• Openstack
– Participate in Openstack Design Summit
• Fedora– RPMs for F16/F17 are quickly released
• Essex EPEL6 expected end of April
– Bugzilla, Wiki, IRC: we participate & benefit– Essex Test day (March 08) good experience
• Issues– Sometimes ahead of upstream
15CERN Agile Infrastructure
![Page 16: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/16.jpg)
Summary
• Agile Infrastructure– Convergence of our infrastructures and virtual
technologies– Implement specific modules for internal use– Participate in the Community– Production ready in time for new Data Centre
16CERN Agile Infrastructure
![Page 17: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/17.jpg)
Questions
17CERN Agile Infrastructure
![Page 18: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/18.jpg)
Backup Slides
CERN Agile Infrastructure18
![Page 19: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/19.jpg)
Openstack Architecture
Compute Scheduler
NetworkVolume
Registry Image
KEYSTONE
HORIZON
NOVAGLANCE
19CERN Agile Infrastructure
![Page 20: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/20.jpg)
IaaS Configuration Use Case
• Hypervisor Configuration Workflow1. Machine is physically installed and registered
2. It’s started by the operator
3. It boots from the network and OS is installed
4. It is configured by Puppet
5. It registers itself to the Cloud Controller
6. It is ready to accept VMs
20CERN Agile Infrastructure
![Page 21: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/21.jpg)
VM Configuration
• Use the same tools as IaaS– Puppet– Foreman
• Instance could be managed or not– User decides
• Several OS supported– Image selection in Horizon– Customization of the image
21CERN Agile Infrastructure
![Page 22: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/22.jpg)
Openstack Architecture
keystone
CERN Agile Infrastructure22
![Page 23: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/23.jpg)
CERN Deployment
keystone
CERN Agile Infrastructure23
![Page 24: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/24.jpg)
Industry Context - Cloud
• Cloud computing models are now standardising– Facilities as a Service – such as Equinix, Safehost– Infrastructure as a Service - Amazon EC2, CVI or lxcloud– Platform as a Service - Microsoft Azure or CERN Web Services– Software as a Service – Salesforce, Google Mail, Service-Now, Indico
• Different customers want access to different layers– Both our users and the IT Service Managers
Applications
Platform
Infrastructure
Facilities
CERN Agile Infrastructure24
![Page 25: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/25.jpg)
Industry Context - DevOps
• DevOps community adopting tool chains with small tools interlinked for particular functions
CERN Agile Infrastructure25
![Page 26: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/26.jpg)
IaaS Approach Benefits• More efficient use of our hardware
– Use machines throughout their lifetime– Reduce impact of hardware failure– Pack server workload for day/night, batch and I/O intensive
workload• Better tracking of usage
– Consistent accounting and provisioning• Enable remote management for new data centre
– Only ‘smart hands’ locally– System administrator procedures simplified
• Consolidate support to a single service– Reach critical mass for coverage
• Easily exploit work done in industry such as load balancers– We are not special
• Support potential new use cases– PaaS, cloud APIs, bursting
CERN Agile Infrastructure26
![Page 27: Jose Castro Leon CERN – IT/OIS CERN Agile Infrastructure Infrastructure as a Service](https://reader036.vdocuments.mx/reader036/viewer/2022081515/56649f205503460f94c38bc5/html5/thumbnails/27.jpg)
Current situation
• IT has two projects in this area– Lxcloud based on OpenNebula providing EC2 services
for physics users to validate concepts and test– CERN Virtual Infrastructure on Hyper-V providing long
lived server consolidation workloads such as desktop replacements, VO Boxes and other production/test/dev workloads
• Neither solution is felt to be ideal from long term self-sustainability and scalability– O(300K) VMs needed– Currently two teams of 1.5 FTE each
CERN Agile Infrastructure27