Lecture - 2 and 3 Virtualization, Orchestration and

Design of Cloud Services

Salman Toor salman.toor@it.uu.se

Outline• Introduction to virtualization

• Importance

• Types

• Hardware support

• Hypervisors

• libvirt API ?

2

• Cloud virtualization

• Demo

• Virtualization beyond VMs

• Orchestration Tools

• Design of Cloud services

Virtualization• Nutshell: The abstraction of available resources

• Definition:

• Whereas, resources can be either compute, storage, network..etc

3

Virtualizationtechnologiesencompassavarietyofmechanismsandtechniquesusedtodecouplethearchitectureanduser-perceivedbehaviorofhardwareandsoftwareresourcesfromtheirphysicalimplementation.

http://www.computer.org/csdl/mags/co/2005/05/r5028.html

Virtualization Basic illustration

4http://blog.cloudpassage.com/2011/07/22/the-difference-between-virtualization-and-cloud-computing/

Virtualizationlayer

Virtualization• Large verity of platforms:

• Old concept, relaunched (article from 1974)

• Properties of virtual machines (VM) • Efficiency • Resource control • Equivalance

5

https://en.wikipedia.org/wiki/Comparison_of_platform_virtualization_software

http://cs.nyu.edu/courses/fall14/CSCI-GA.3033-010/popek-goldberg.pdf

Virtualization• Virtualization Layer

• Types of Hypervisors

• Bare-Metal • Hosted

6

HypervisororVirtualMachineMonitor(VMM)isasoftwarethatprovidesaninterfacebetweenhardwareandvirtualoperatingsystems.

Hardware

Hypervisor

OS-1 OS-2 OS-N

Bare-Metal

Hardware

OperatingSystem

Processes Hypervisor

Hosted

OS-1 OS-N

Virtualization • Virtualization could address following issues:

• Under-utilized resources

• Complicated system management

• Limited access to shared resources

• inefficient power consumption

• Tight coupling with underlying resources

• …

7

Virtualization

8

CPU

OperatingSystem

APP-1 APP-2 APP-N

Multi-tasking

CPU-1

OperatingSystem

APP-1 APP-2 APP-N

Multi-coreorHyper-threading

CPU-2

VirtualCPU-1

OperatingSystem

APP-1 APP-2 APP-N

VirtualCPU-2

OperatingSystem

APP-1 APP-2 APP-N

Virtualization

VirtualCPU-N

OperatingSystem

APP-1 APP-2 APP-N

http://www.hardwaresecrets.com/everything-you-need-to-know-about-the-intel-virtualization-technology/

Virtualization Types

• Platform virtualization

• Memory virtualization

• Desktop virtualization

• Application virtualization

• Network virtualization

• Storage virtualization

9

Virtualization Platform virtualization

• Full virtualization

• Para virtualization

• Hardware assisted virtualization

• Operating-System (OS) level virtualization

• Hybrid virtualization

10http://www.vmware.com/files/pdf/VMware_paravirtualization.pdf

Virtualization Full

• Guest operating system (VM) is unaware of host OS

• Non-critical instructions run directly on hardware

• Runtime translation of critical non-virtualizable instructions in hypervisor

• Not best in performance

11

Virtualization Para

• Thin layer interfaces between each guest OS and underlying hardware

• Need Guest kernel modification

• No need of runtime translation for critical instructions

• Superior in performance

• Requires expertise to patch the kernels

12

Virtualization Hardware assisted

• Hardware provides support to run instructions independently

• No need to patch the kernels

• Runtime translations not required

• Better performance in comparison to other variants

• Greater stability

13

Virtualization OS level

• Same OS for both Host and guest machines

• Userspace is completely isolated

• High performance

• Extremely light-weight

• Recently acquire lots of attention in Cloud world

14

Virtualization Hybrid

• Combination of • para • hardware assisted virtualization

• Address the issues related to security and system stability

• Use patched guest OS along with hardware support

15

Virtualization Memory

• Brings concepts of consolidation and cost effectiveness

• Managed with virtualizing physical memory by addition of an extra level of address translation

16

physicalmemory

CPU-0 CPU-1 CPU-1 CPU-1

VM-1 VM-2

VMmainmemory VMmainmemory

Question: Is “virtual memory” concept in a single operating system same as the concept of virtualization in terms of VMs?

Answer:NO

Virtualization Desktop and Applications

• Desktop and Applications run on servers

• Stateless thin clients connected to servers

• Efficient system management

• Requires high-end servers for system stability

17

Virtualization Network and Storage

• Similar idea of providing an abstraction layer to the physical infrastructures

• In networks abstraction will be at the level of

• Storage abstraction allows single backends to be used for different requirements

18

• Routers• Switches• loadbalances

• Gateway• Firewalls• …

• Ephemeral• Persistant• Specializestoragebackends

Virtualization Network and Storage

• Aim is to provide provides:

• Efficient infrastructure utilization • Agility • Isolation • Security • ….

19

Virtualization Hardware Acceleration

• Aims:

• enhance the performance • reduce the complexity in the hypervisors/VMM

• Intel and AMD chips support VT

• Intel models

• AMD model

20

http://ark.intel.com/Products/VirtualizationTechnology

http://support.amd.com/en-us/kb-articles/Pages/GPU120AMDRVICPUsHyperVWin8.aspx

Virtualization Concept of overcommits

• Process of allocating more than the available physical resources

• Common types:

• CPU Overcommits • Memory Overcommits • Storage Overcommits

• Strong requirement from infrastructure providers

21

Virtualization Concept of overcommits

• Pros:

• Favorable economic model • Efficient resources utilization • Support green computing

• Cons:

• Performance loss or unstable system response • Complex system understanding • VM shutdown by the hypervisor (extreme cases)

22

Virtualization CPU overcommit

• Allows more virtual CPUs than physically available

• Example: In case of 8 physical cores

• Open stack KVM allows:

23

8 * overcommit-number = total-number-of-virtual-CPUs

overcommit-number = 16.0 (max) , 1.0 (no overcommits)

http://docs.openstack.org/kilo/config-reference/content/section_compute-scheduler.html

Virtualization Memory overcommit

• Allocate more than physical memory

• Open stack KVM allows:

• 1.5 GB is required by the instance • can run on any physical host having1GB free memory

24http://docs.openstack.org/kilo/config-reference/content/section_compute-scheduler.html

overcommit-number = 1.5GB

Virtualization Hypervisors

• Contribution from industry and academia

• Xen • Project from Cambridge Computer Laboratory

• VMware • Commercial product

• KVM (Kernel-based Virtual Machine) • A product of Open Virtualization Alliance (OVA)

• Qemu • Opensource machine emulator and virtualizer

• …..

25

http://www.xenproject.org/developers/teams/hypervisor.htmlhttp://www.vmware.com/https://openvirtualizationalliance.org/what-kvmhttp://wiki.qemu.org/Main_Page

Virtualization KVM

• Hypervisor for x86 solutions with complete hardware support

• Run multiple guest OSes with private virtualized hardware: network card, memory, disk etc

• Consist of Loadable kernel modules:

• kvm.ko for core virtualization • processor specific kvm-intel.ko or kvm- amd.ko

26

Virtualization Tools KVM

• Opensource software

• Support Linux and Windows (limited)

• Peripheral support to the guest OS

• Wide variety of management tools

• Backend for compute resources in many Cloud suites

• Allow resources overcommits

27

Virtualization Tools KVM

28http://www-03.ibm.com/systems/kvm/whykvm.html

• KVM architecture for x86 system

• support nested virtual machines

Virtualization Tools Qemu

• Qemu runs in following modes:

• Emulator; hypervisor runs in the user space • Hypervisor; Hardware supported virtualization

(KQemu)

• Support for multiple OSes

• Based on Xen or KVM, Qemu supports nested virtualization

29http://wiki.qemu.org/Main_Page

Virtualization Tools Qemu

• Opensource software

• Extensive peripheral support

• Diversity of management tools

• Backend for compute resources in many Cloud suites

• Recommended for testing and development environments

30

Virtualization Tools Qemu

31

Hardware

HostOperatingSystem

QEMU QEMU

VM-1 VM-2

Hardware

HostOperatingSystem

QEMU QEMU

VM-1 VM-2

KVMmodule

QemuEmulator QemuwithKVMsupport

Virtualization Tools VMware

• Commercial product by VMware.Inc

• One of the leading server virtualization system

• frontend

• Variety of system management components

• Backend for compute resources in number of Cloud software

32

Virtualization Tools VMware

• VMware vSphere consists of:

• VMware ESXi • VMware vCenter Server • VMware VMFS • VMware Virtual SMP • VMware Clients (Desktop and Web access)

• ESXi is the virtualization platform

• vCenter Server is a service that act as a administrator to ESXi platform

33http://pubs.vmware.com/vsphere-51/topic/com.vmware.vsphere.vcenterhost.doc/GUID-302A4F73-CA2D-49DC-8727-81052727A763.html

Virtualization Tools VMware

• Overview Architecture

34http://www.vmware.com/pdf/vi_architecture_wp.pdf

• Comprehensive solution for data center virtualization

•

Role of LIBVIRT API• Virtualization API

• Provide a common and stable layer to securely communicate with guest OS

• Supports all major hypervisors

• Open source project for secure VM management tasks

• Used to build applications based on virtual environment

35http://libvirt.org/

Virtualization Interesting Articles

• A quantitative comparison between xen and kvm (2010 J. Phys.: Conf. Ser. 219

042005)

• Performance Measuring and Comparing of Virtual Machine Monitors (2008 IEEE/IFIP International Conference on Embedded and Ubiquitous Computing)

• Recommendations for Virtualization Technologies in High Performance Computing (2nd IEEE International Conference on Cloud Computing Technology and Science)

• A Comparison of Software and Hardware Techniques for x86 Virtualization (Advance level) (ASPLOS XII Proceedings of the 12th international

conference on Architectural support for programming languages and operating systems)

36

Virtualization and Clouds OpenStack

• Open source platform for build public and private Clouds

37http://www.openstack.org/

OpenStack Compute

• OpenStack Compute Project (NOVA)

38

Hardware

VMware

KVM Qemu LXC

Docker

LibvirtAPIs

OpenStackNovaAPI operatingsystem

Computehost

Cloud Virtualization Storage

• OpenStack volume (Cinder) • OpenStack object store (Swift)

39Disk-1 Disk-1 Disk-1 Disk-1

Ceph

lib-rbd

CinderGlanceNova

lib-rgw

Swift

libvirt

SNICCloudStoragebackend

Cloud Virtualization Network

• OpenStack network components (Neutron)

40

Cloud Virtualization Network

• OpenStack network components (Neutron)

41http://docs.openstack.org/networking-guide/

DEMO

42

DOES VIRTUALIZATION EFFECT THE SYSTEM PERFORMANCE?

43

Performance• Yes performance loss may occur but it is highly

dependent on

• Type of virtualization layer (Hypervisor) • Use case

• CPU bound application will perform differently than IO bound or network intensive applications

44

Performance• In comparison with the

physical node: • KVM perform 83.46% • Xen perform 97.28%

• Reason; Critical instruction test verses para-virtualization

45Performance Measuring and Comparing of Virtual Machine Monitors (2008 IEEE/IFIP International Conference on Embedded and Ubiquitous Computing)

In both cases, There is a performance different compare to physical machine.

Performance• ApplicafonLevel

• 4%performancelossevaluatedwiththe• HEPSPEC-2006(ThankstoUlfTigerstedt,CSCforhelp

withHEPSPECtests)• SystemLevel

• VMbootresponsebothatlocalvsGlusterFSbasedsetup

46Article:AscalableinfrastructureforCMSdataanalysisbasedonOpenStackCloudandGlusterfilesystem

Images and formats • Cloud images are customized disks images of OSes for

private or public clouds

• Different formats are available: • raw: An unstructured disk image format (big in size) • vhd: VMware, Xen, Microsoft, VirtualBox, and others • vdi: Supported by VirtualBox, QEMU emulator. • iso: Archive format for the data contents of an optical disc • qcow2: Supported by the QEMU emulator that can

expand dynamically and supports Copy on Write. • …

47http://docs.openstack.org/image-guide/content/image-formats.html

Contextualization

In cloud computing contextualization means providing customized computing environment

Or

Allows a virtual machine instance to learn about its cloud environment and user requirement (the ‘context’) and configure itself to run correctly

48http://www.journalofcloudcomputing.com/content/4/1/17

Contextualization• Provide scalable solution

• No need to manage fat images

• Dynamic configuration

• Typically work in two layers • Meta-data : System information handled at cloud level • User-defined-data: User specific requirements/

settings

49

Beyond Virtual Machines

50

Containers• OS level virtualization environment

• Kernelspace is shared • Userspace is separate for each linux system

(container)

• A lightweight alternative to Virtual Machines (VM)

• Shared same resources as host OS

• A simple model for packaging applications in Linux.

51https://linuxcontainers.org/

Basic Illustration of Containers

52

LinuxbasedhostOS NonLinuxhostOSUbuntu

ContainerController

ContainerEngine

Con-1Con-N

Con-2

WindowsorMAC

ContainerController

ContainerEngine

Con-1Con-N

Con-2

LinuxVM

VM and Containers

• Complete isolation • Big in size • High overhead • Flexible support of multiple

OSes • Greater stability both for

hypervisors and VMs • Better security

53

• Application level abstraction • Lightweight • Works well with Linux, limited

support for Windows • Weak security • Significant management Overhead • Not well suited for large applications • Important for micro-services design

VirtualMachine Containers

ItsisimportanttounderstandthatVMsandContainersshouldnotbeviewedascompetitors

LXC• LinuX Container (LXC) is an open source software

• Virtual environment based on separate memory, CPU, network, io etc

• Similar to the concept of chroot

• Used in most of the container based orchestration tools

• LXD is a newer version of LXC, advanced and stronger support for cloud plugins

54http://blog.scottlowe.org/2013/11/25/a-brief-introduction-to-linux-containers-with-lxc/

Container Softwares• OpenVZ

• Virtuozzo (Linux and Windows)

• Solaris Containers (Solaris)

• Spoon (Windows)

• VMware ThinApp (Windows)

55

ORCHESTRATION TOOLS

56

Orchestration • Orchestration is a process of resource contextualization based on the

automation available in the cloud systems.

• A process required for • rapid application deployment • scalability • management • high availability • Agility

• Essential for large complex applications

• A process at the level of Platform as a Service (PaaS)

Orchestration• Available tools

• CoreOS https://coreos.com/ • OpenShift https://www.openshift.com • Docker https://www.docker.com/ • Kubernetes http://kubernetes.io/ • Molns (used in Lab-2) • Heat https://wiki.openstack.org/wiki/Heat

58

HEAT • OpenStack based orchestration software to create a

human- and machine-accessible services

• Consist of template base orchestration engine

• Enables simple or composite cloud applications based on existing services

• Compatible with AWS Cloud Foundation

59https://wiki.openstack.org/wiki/Heat

HEAT Template example

60

heat_template_version:2014-10-16description:Asimpleserver.parameters:ImageID:type:stringdescription:ImageusetobootaserverNetID:type:stringdescription:NetworkIDfortheserverresources:server:type:OS::Nova::Serverproperties:image:{get_param:ImageID}flavor:m1.tinynetworks:-network:{get_param:NetID}outputs:private_ip:description:IPaddressoftheserverintheprivatenetworkvalue:{get_attr:[server,first_address]}

#heatstack-create-ftest-stack.yml-P"ImageID=cirros-0.3.4-x86_64;NetID=$NET_ID"testStack

http://docs.openstack.org/kilo/install-guide/install/yum/content/heat-verify.html

Docker• Docker package an application together with all its dependencies in

the container • Guarantees that it will always run the same regardless of the

environment • Container based orchestration tool • Docker Hub, container registory • Open source

61https://www.docker.com/whatisdocker

Docker• Docker Hub, container registry

• Image repository

• Automated Builds

• Webhooks

• Organizations

• Github and Bitbucket integrations

62https://docs.docker.com/docker-hub/

Kubernetes• Google supported orchestration software

• Open source

• Supported in public, private and multi-cloud environments

• Currently supported for Linux platform

• Support Docker containers

63

Cloud Services Design • Cloud service architecture should adhere with following features:

• Scalability

• Security

• Fault tolerant

• Vendor agnostic

• Elasticity

• Leverage different backends

64

• Loosely coupled components

• ….

Cloud Services Design • Scalability

• Horizontal scaling • Vertical scaling

• Security

• Multi level security • Authorisation • Authentication

65

Cloud Services Design • Fault-Tolerance

• Recover from failover and unexpected short interruptions

• Vendor Agnostic

• No dependency on vendor specific tools

66

Cloud Services Design• Elasticity

• Dynamically adopt according to the requirements

• Support for different backends

• Flexible enough to accommodate different users requirements

67

Cloud Services Design • Loosely coupled service model

• Components should not tightly connected with each other

• Work as micro and macro services

• ….

68