Tim Mackey – XenServer Community Manager and Evangelist
Hypervisor Selection in CloudStack 4.3 and OpenStack HavanaUnderstanding the choices available
virtg Deep Dive Day 2014
Building a successful cloudWhat are we trying to accomplish?
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Service Offerings
• Clearly define what you want to offerᵒ What types of applicationsᵒ Who has access, and who owns themᵒ What type of access
• Define how templates need to be managedᵒ Operating system supportᵒ Patching requirements
• Define expectations around compliance and availabilityᵒ Who owns backup and monitoring
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Define Tenancy Requirements
• Department data local to departmentᵒ Where is the application data stored
• Data and service isolationᵒ VM migration and host HAᵒ Network services
• Encryption of PII/PCIᵒ Where do keys live when data location unknownᵒ Need encryption designed for the cloud
• Showback to stakeholdersᵒ More than just usage, compliance and audits
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Virtualization Infrastructure
• Hypervisor defined by service offeringsᵒ Don’t select hypervisor based on “standards”ᵒ Understand true costs of virtualizationᵒ Multiple hypervisors are “OK”ᵒ Bare metal can be a hypervisor
• To “Pool” resources or notᵒ Is there a real requirement for pooled resourcesᵒ Can the cloud management solution do better?ᵒ Real cost of shared storage
• Primary storage defined by hypervisor
• Template storage defined by solutionᵒ Typically low cost options like NFS
Choice is a good thing….
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Apache CloudStack
• Current release: 4.2.1 (4.3 imminent)
• Highly scalable
• Monolithic architecture
• Mostly written in Java
• Multi-hypervisor supportᵒ XenServer, KVM, OracleVM, vSphere,
Linux Containers, Bare metalᵒ 4.3 adds: Hyper-V
• Strong backing from Citrix, CloudOps, Shapeblue and others
Back UpDB
Management Server
MySQLDB
Replication
Management Server
Management Server
Load Balancer
InfrastructureResources
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
OpenStack
• Current release: Havana
• Scalable in 500 node blocks
• Distributed architecture
• Mostly written in Python
• Multi-hypervisor supportᵒ Group A: KVMᵒ Group B: Hyper-V, vSphere, XenServerᵒ Group C: All others deprecated in Icehouse
• Strong backing from HP, IBM, RAX, RHT, Canonical, Mirantis, Piston Cloud, SUSE, Cloudscaling
Leading hypervisor options
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
XenServer 6.2
FeatureSource code model Open Source (GPLv2)Maximum VM Density 650 (Linux)CloudStack VM Density 500CloudStack integration Direct XAPI callsOpenStack driver OpenStack nova-compute domUMaximum native cluster size 16Maximum pRAM 1 TBLargest VM 16 vCPU/128GBWindows Operating System All Windows supported by MicrosoftLinux Operating Systems RHEL, CentOS, Debian, Ubuntu, SLES, OELAdvanced features supported (CloudStack) ovs, Storage XenMotion, DMCAdvanced features supported (OpenStack) ovs, Storage XenMotion
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
vSphere 5.5
FeatureSource code model ProprietaryMaximum VM Density 512CloudStack VM Density 128CloudStack integration vCenterOpenStack driver vCenter – nova-compute node per clusterMaximum native cluster size 32Maximum pRAM 4 TBLargest VM 64 vCPU/1TBWindows Operating Systems DOS, All Windows Server/ClientLinux Operating Systems MostAdvanced features supported (CloudStack) HA, DRS, vDS, Storage vMotionAdvanced features supported (OpenStack) HA, DRS, vMotion
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
KVM
FeatureSource code model Open Source (GPLv2)Maximum VM Density 10 times the number of pCoresCloudStack VM Density 50CloudStack integration CloudStack Agent (libvirt)OpenStack driver libvirt driverMaximum native cluster size No native cluster supportMaximum pRAM 2 TBLargest VM 160 vCPU/2TBWindows Operating Systems Windows XP and higherLinux Operating Systems VariesAdvanced features supported (CloudStack) NoneAdvanced features supported (OpenStack) None
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Microsoft Hyper-V
FeatureSource code model ProprietaryMaximum VM Density 1024CloudStack VM Density 1024CloudStack integration CloudStack Agent (C# calling WMI)OpenStack driver Use Cloudbase driverMaximum native cluster Size 64Maximum pRAM 4 TBLargest VM 64 vCPU/1TBWindows Operating Systems All Windows supported by MicrosoftLinux Operating Systems RHEL, CentOS, Debian, Ubuntu, SLES, OELAdvanced features supported (CloudStack) NoneAdvanced features supported (OpenStack) None
The CloudStack 4.3 world …
Defining the network
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Flat Network – Basic Layer 3 Network
Option XenServer vSphere KVM Hyper-V
Security Groups Yes- bridge No Yes Yes
IPv6 No No Yes No
Multiple IPs per NIC Yes Yes Yes Yes
Nicira NVP Yes No Yes No
BigSwitch VNS Yes No Yes No
65.11.1.2
65.11.1.3
65.11.1.4
65.11.1.5
Public Network65.11.0.0/16
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4
DHCP, DNS
CloudStackVirtual Router
Security Group 1
Security Group 2
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
VLANs for Private Cloud
Option XenServer vSphere KVM Hyper-VMax VLANs 800 254 1024 4094
IPv6 No No Yes No
Multiple IPs per NIC
Yes Yes Yes Yes
Nicira NVP Yes No Yes No
BigSwitch VNS Yes No Yes No
MidoKura No No Yes No
VPC Yes Yes Yes Yes
NetScaler Yes Yes Yes Yes
F5 BigIP Yes Yes Yes Yes
Juniper SRX No Yes Yes Yes
Cisco VNMC No Yes No No
10.1.1.1
10.1.1.3
10.1.1.4
10.1.1.5
Public Network/Internet
Guest Virtual Network 10.0.0.0/8VLAN 100
DHCP, DNSNATLoad BalancingVPN
Public IP 65.37.14.1
Gateway10.1.1.1
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4
CloudStackVirtual Router
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Beyond the VLAN – Network Virtualization
Option XenServer vSphere KVM Hyper-V
OVS GRE tunnels Yes No No No
Nicira STT tunnel Yes Yes Yes No
MidoNet No No Yes No
VXLAN No Yes Yes No
NVGRE No No No No
Nexus 1000v No Yes No No
Juniper Contrail Yes No No No
Palo Alto Yes Yes Yes No
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Virtual Private Cloud and nTier Applications
Feature XenServer vSphere KVM Hyper-V
PVLAN Yes - ovs Yes ovs NoWeb
App
DB
Router
DC1
DC2DC3
DC4
DC5
DC6
VLAN 1
VLAN 2
VLAN 3
S2S VPN
PrivateGW
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Delivering specific network services
• KVM ᵒ IPv6ᵒ Security groupsᵒ Large quantity of VLANs
• vSphereᵒ VXLAN required vSphere Enterprise Plusᵒ Cisco Nexus 1000v and ASA 1000v require vSphere Enterprise Plus
• XenServerᵒ Security groupsᵒ Large quantity of VLANsᵒ Juniper Contrail
Instances need a home …Storage, Storage and more Storage
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Primary Storage OptionsFeature XenServer vSphere KVM Hyper-V
Local storage Yes Yes Yes YesNFS Yes Yes Yes NoSMB No No No SMB3Single path iSCSI Yes Yes Yes NoMultipath iSCSI PreSetup No No NoDirect array No VAAI No NoShared Mount No No Yes NoTemplate format VHD OVA QCOW2 VHDSolidFire Plugin Yes Yes Yes NoNetApp Plugin Yes Yes Yes NoZone wide No Yes Yes NoCeph RBD No No Yes NoClustered LVM No No Yes No
Cluster
Host
HostPrimary Storage
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Secondary Storage Options
Option XenServer vSphere KVM Hyper-V
NFS Yes Yes Yes No
Swift(1) Yes Yes Yes No
S3 compatible (2) Yes Yes Yes No
SMB No No No Yes
(1) Requires NFS staging area(2) Can be region wide, but must not have NFS secondary storage in zone
ZoneSecondary Storage
Pod
Cluster
Host
HostPrimary Storage
Core virtualization capabilitiesThe limits and features which matter
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
CloudStack FeaturesFeature XenServer vSphere KVM Hyper-V
Disk IO Statistics Yes No Yes Yes
Memory Overcommit Yes (4x) Yes No No
Dedicated resources Yes Not with HA/DRS Yes Yes
Disk IO throttling No No Yes No
Disk snapshot (running) Yes Yes No No
Disk snapshot (pluggable) Partial Partial No No
Disk snapshot (Stopped) Yes Yes Yes Yes
Memory snapshot Yes Yes Yes No
Zone wide primary storage No Yes Yes No
Resize disk Offline Online Grow Online No
High availability CloudStack Native CloudStack CloudStack
CPU sockets 6.2 and higher Yes Yes Yes
Affinity groups Yes Yes Yes Yes
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Multiple Hypervisor Support in CloudStack
• Networkingᵒ Ensure network labels matchᵒ Topology is intersect of chosen hypervisors
• Storageᵒ For system VMs to specific hypervisor typeᵒ Zone with primary storage limited
• Operationsᵒ vSphere Datacenter can not span zonesᵒ Hyper-V may not be mixed with other hypervisorsᵒ HA won’t migrate between hypervisorsᵒ Capacity planning at the cluster/pod level more difficult
The OpenStack Havana world …
Defining the networkThe goodness of Neutron (Quantum)
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Flat Network – Basic Layer 3 Network
Option XenServer vSphere KVM Hyper-V
Security Groups Yes Yes-NVP Yes No
IPv6 No No No No
Multiple IPs per NIC Yes Yes Yes Yes
Nicira NVP Yes Yes Yes No
Firewall rules Yes No Yes No
Routing Yes No Yes No
65.11.1.2
65.11.1.3
65.11.1.4
65.11.1.5
Public Network65.11.0.0/16
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4
DHCP, DNS
CloudStackVirtual Router
Security Group 1
Security Group 2
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
VLANs for Private Cloud
Option XenServer vSphere KVM Hyper-VMax VLANs 800 254 1024 4094
IPv6 No No No No
Multiple IPs Yes Yes Yes Yes 10.1.1.1
10.1.1.3
10.1.1.4
10.1.1.5
Public Network/Internet
Guest Virtual Network 10.0.0.0/8VLAN 100
DHCP, DNSNATLoad BalancingVPN
Public IP 65.37.14.1
Gateway10.1.1.1
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4
CloudStackVirtual Router
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Beyond the VLAN – Network Virtualization
Option XenServer vSphere KVM Hyper-VOVS GRE tunnels Yes No No NoNicira STT tunnel Yes Yes Yes NoVXLAN No Yes Yes NoNVGRE No No No NoNexus 1000v No Yes Yes NoNicira NVP Yes Yes Yes NoBigSwitch VNS No No Yes NoMidoKura No No Yes NoBrocade No No Yes NoPlumgrid No No Yes NoRyu No No Yes NoNEC No No Yes NoCisco VNMC No No Yes No
Instances need a home …Storage, Storage and more Storage
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Instance Storage Options – Nova and Cinder
Feature XenServer vSphere KVM Hyper-VTemplate format VHD VMDK QCOW2 VHDLocal storage Yes Yes Yes YesNFS Yes Yes Yes YesFiber HBA No No Yes NoiSCSI Yes Yes Yes YesiSCSI CHAP Yes No Yes NoCeph No No Yes NoGluster No No Yes NoZFS No No Yes No
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
OpenStack FeaturesFeature XenServer vSphere KVM Hyper-V
Pause VM Yes No Yes Yes
Inject networking Guest agent Yes Flat (Debian) Yes
Inject file Yes No Yes cloudbase-init
Serial console Yes Yes Yes No
VNC consoles Yes Yes Yes Yes
SPICE console No No Yes No
Snapshot Yes Yes Yes Yes
Set administrator password Yes No No cloudbase-init
Auto configure disk Yes No No No
Evacuate host Yes No Yes No
Volume swap No No Yes No
Volume rate limiting No No Yes No
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Multiple Hypervisor Support in OpenStack
• Capabilitiesᵒ Multiple hypervisor support varies by distroᵒ Most deployments are single hypervisorᵒ Difficult to schedule instances to compute nodesᵒ Use host aggregates
• Networkingᵒ Topology is intersect of chosen hypervisors – ML2 helps
• Operationsᵒ HA won’t migrate between hypervisorsᵒ Capacity planning at the cluster/pod level more difficultᵒ Hyper-V does not work with all Neutron plugins
Picking the “best one”When to use which hypervisor…
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
KVM
• Primary value proposition:ᵒ Low cost with available vendor supportᵒ Familiar administration modelᵒ Broad feature set with active development in both CloudStack and OpenStack
• Cloud use cases:ᵒ Linux centric workloadsᵒ Dev/test clouds ᵒ Web hostingᵒ Tenant density which dictates SDN options
• Weaknesses:ᵒ CloudStack: Requires use of an installed libvirt agentᵒ Limited native storage optionsᵒ No use of advanced native features
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Microsoft Hyper-V
• Primary value proposition:ᵒ Unlimited Windows Server VM licensesᵒ Familiar Windows management paradigm
• Cloud use cases:ᵒ Windows and Linux workloadsᵒ Dev/test clouds ᵒ .Net application web hostingᵒ Desktop as a Service clouds
• Weaknesses:ᵒ Minimal use of advanced native featuresᵒ CloudStack: First introduced with CloudStack 4.3 (not yet released)
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
vSphere
• Primary value proposition:ᵒ Broad application and operating system supportᵒ Readily available pool of vSphere administration talentᵒ Large eco-system of vendor partnersᵒ CloudStack: Many features are native implementationsᵒ Direct feature integration via vCenter
• Cloud use cases:ᵒ Private enterprise cloudsᵒ Dev/test clouds
• Weaknesses:ᵒ vSphere up-front license and ongoing support costsᵒ vCenter integration requires redundant designsᵒ CloudStack: Single data center per zone model
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
XenServer
• Primary value proposition:ᵒ Low cost with available vendor supportᵒ Broad feature set with active development in both CloudStack and OpenStackᵒ CloudStack: Large install baseᵒ Direct integration via XAPI toolstack
• Cloud use cases:ᵒ Linux centric workloadsᵒ Dev/test clouds ᵒ Web hostingᵒ Desktop as a Service cloudsᵒ Large VM density and secure tenant isolation
• Weaknesses:ᵒ Minimal use of advanced native features
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Tying it all Together
1. Define success criteria
2. Select a topology which works
3. Decide on storage options
4. Define supported configurations
5. Select preferred hypervisor(s)
6. Validate matrix
7. Build your Cloud
Work better. Live better.