virtualization for autonomous administration of servers

1

Virtualization for autonomous administration of servers

2

PlanClient’s Specifications

Context

Objectives

Tools

The Xen Attitune ProjectRequirements

Architecture

Tests

Project ManagementProgress Report

3

Client’s Specifications

4

Context

Context Servers Cost Electrical Consumption Distributed Servers

Metric Server reservation price Per server electrical consumption

Network load and servers distances

Goal

Minimizing costs =

Bringing together low CPU consumers

Minimizing electrical consumption

= Shutting down unused

physical machines

Minimizing communication times

= Bringing together highly communicating servers

5

ContextVi

rtual

Mac

hine

1

Virtu

al M

achi

ne 2

ExpansionCriteria

ShrinkCriteria

6

ObjectivesS

erve

r 1

Ser

ver 2

Request number rising

Request number reduction

7

Objectives

Xen Paravirtualization Tools

TUNe

NFS

Illustrative Scenario

8

Hardware

Domain 0

Abstraction LayerXen Control

InterfaceModified Drivers

User Applications

Xen Control Software

ToolsXen Architecture Overview

Xen Hypervisor

User Domain 1

Abstraction Layer

Modified Drivers

User Applications

User Domain 2

Abstraction Layer

Modified Drivers

User Applications

9

Dom0-1

DomU

Dom0-2

xm m

igra

te

Filesystem

ToolsXen Migration

10

ToolsTUNe Architecture

Fractal Components

Tune Wrapper

Stubs

Legacy Software

TUNe Wrappers

11

ToolsTUNe Diagrams

S2

+wrapper : String = S2.xml+legacyFile : String = S2.tgz+host-family : String = localhost

1

1

S1

+wrapper : String = S1.xml+legacyFile : String = S1.tgz+host-family : String = localhost

event

12

The Xen Attitune Project

13

The Xen Attitune ProjectPreliminary study

Requirements

Architecture

Tests

Implementation

Tests

14

The Xen Attitune Project

RequirementsPreliminary study

Requirements

Architecture

Tests

Implementation

Tests

15

RequirementsSpecification example

R3

Requirement Formulation

Calculation results will not be affected by virtual machine migration.

RationalMigration should keep virtual machine memory state and hosted applications execution background.

Priority P0

Demand Reference

D4

Test Reference

T3

16

RequirementsSpecification document

Migration requirements P0No task processing interruptionTCP connection keepingResults consistencyCPU load independence

Migration performance study P1Interruption durationMigration durationDelay

TUNe-related requirements P2Integration in TUNeMigration Automation

17

The Xen Attitune Project

ArchitecturePreliminary study

Requirements

Architecture

Tests

Implementation

Tests

18

NFS serverDHCP server

Architecture

19

A-204-06

DHCP ServerNFS Server

Switch

A-204-02

DNS ServerNTP Server

Architecture

A-204-03 A-204-04

A-204-05

20

Architecture

dom0

Ubuntu Gutsy Gibbon / Xen Management

Xend Daemon

domU-2

Ubuntu Dapper Drake / NFS root

domU-1

GNU/Linux Debian Etch / NFS root

21

Migration

Pool Architecture

Dom0-1 Dom0-2

DomU

Event

Migrate

TUNe Administrator

Probe0-1 Probe0-2

22

Pool ArchitectureTUNe Diagrams

this.probed.migrate

Migrate

VM.start

VM-Probe.start

Start

VM.stop

VM-Probe.stop

Stop

23

The Xen Attitune Project

TestsPreliminary study

Requirements

Architecture

Tests

Implementation

Tests

24

TestsExample

T2

Title Streaming server migration between two dom0.

Pre-requisites A streaming server is running on a domU. A streaming client is connected to the server.

Test description The video is watched during the migration.

Expected results The client should not be disconnected.Requirement reference R1, R2

25

TestsCoverage matrix

RequirementsR1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12

Tests

T1 ●

T2 ● ●

T3 ● ●

T4 ● ● ● ●

T5 ● ●

T6 ● ●

T7 ● ●

T8 ● ● ●

T9 ● ● ●

T10 ● ● ●

T11 ● ●

T12 ● ●

26

TestsMigration performances

Interruption duration 1 s

Delay ~10 s

Migration duration 16 s (60% longer than RMI)

27

Migration

TestsTCP Connection Preservation

Dom0-1

DomU

Dom0-2

StreamingServer

Streaming Client

28

Test SchemeTCP Connection Preservation

Streaming Client Output

Dom0 Source Dom0 Destination

Network Traffic Monitor

29

TestsIllustrative Scenario

Dom0-1 Dom0-2

Event

TUNe Administrator

Probe0-1 Probe0-2

Migrate

CPU loading

Streaming Client

StreamingServer

DomU

Migration

30

Project Management

31

Specification document

Architecture document

Test plan

Risk Analysis

Action List

Schedule

Tutorials

Project Management

32

Project ManagementRisk Analysis

Risk Probability Criticality Product Preventive Actions Corrective Actions

Data loss (large size) 2 1 2 Copy on external hard drive Backup

Data loss (small size) 2 3 6 Copy on SVN repository Backup

Network breakdown 1 2 2Local copy of

documentation and API

Schedule adaptation

System crash (OS, VM) 1 2 2 Installation guide Reinstall

33

Project ManagementAction List

No Origin Date Description Comment Initial deadline

Revised deadline Owner Closed date

1 3/10 Book a room for the meeting

- 3/11 - Hery 3/10

2 3/10 Send an Email to supervisor

Ask for meeting confirmation

3/10 3/11 Zek 3/11

3 3/13 Bring bakeries

Donuts 3/13 - Julien C

4

5

34

Project ManagementSchedule

35

Project ManagementTutorials

Xen installation and setting up

NFS server

DNS server

DHCP server

NTP server

TUNe

36

Progress Report

37

Network Architecture

Progress ReportSoftware

ArchitectureTests TUNe Improvements

CPU Load OKMigration OKInterruption OKIntegrity OKDelay NOKStreaming NOK

Kernel 18.8Xen (dom0) Wrappers,

Deployment diagram

Reconf. Diagram,Probe TUNe modifications

Streaming scenario

kdeployJ2EE

Policies

NTP Server

DNS Server

Clock issues

Synchronisationissues

Issues resolved

Name resolution

Xen 3.1

Kernel 19-6

Kernel 18.8NFS root (domU)

DHCP Server

NFS Server

BugFilesystem

NFS Boot on DomU

Functionnal configuration

Kernel 22-14

Delay OKStreaming OK

38

Completed objectivesXen deployment

Migration performance analysis

TUNe wrapping

Tutorials production

Remaining tasksAutonomous administration

Streaming scenario with TUNe

Further improvementsJ2EE scenario

Grid 5000 deployment

Policies

Conclusion

39

Emmanuel Murzeau

Daniel HagimontLaurent BrotoMohamed TouréAlain TchanaFabien Hermenier

Special Thanks

40

Any questions ?

Test SchemeTCP Connection Preservation

Test SchemeTCP Connection Preservation

Test SchemeTCP Connection Preservation

44

TestsResults consistency Writing in a file

CPU load independence Migration with different CPU loads

Interruption duration Ping probing

Delay Comparison with local execution of LU factorizations

Migration duration Comparison with equivalent RMI calls

TCP connections keeping Migration of streaming server

45

Hardware

Domain 0

Abstraction LayerXen Control

InterfaceModified Drivers

User Applications

Xen Control Software

Preliminary StudyXen Architecture Overview

User Domain 1

Abstraction Layer

Modified Drivers

User Applications

Xen Hypervisor

User Domain 2

Abstraction Layer

Modified Drivers

User Applications

46

Hardware

Domain 0

Abstraction LayerXen Control

InterfaceModified Drivers

User Applications

Xen Control Software

Preliminary StudyXen Architecture Overview

User Domain 1

Abstraction Layer

Modified Drivers

User Applications

Xen Hypervisor

User Domain 2

Abstraction Layer

Modified Drivers

User Applications

47

Dom0-1

DomU

Dom0-2

xm m

igra

te

Preliminary StudyMigration

48

Dom0-1

DomU

Dom0-2

xm m

igra

te

Filesystem

Preliminary StudyMigration

49

Preliminary StudyTUNe Component-based Architecture

Fractal Components ADLArchitecture Description Languague

50

Preliminary StudyTUNe Deployment Diagrams

S1.configure S2.configure

S1.start

S2.start

S2

+wrapper : String = S2.xml+legacyFile : String = S2.tgz+host-family : String = localhost

1

1

S1

+wrapper : String = S1.xml+legacyFile : String = S1.tgz+host-family : String = localhost

51

Preliminary StudyTUNe Reconfiguration Diagram

this.stop

this.start

this.reconfigure

event

52

Pool ArchitectureTUNe wrappers

domU wrapper

StartStop

Migrate

domUProbe wrapper

StartStop

53

Migration

TestsResults consistency

Dom0-1

DomU

Dom0-2

Writingin a file

54

Migration

TestsCPU Load Independence

Dom0-1

DomU

Dom0-2

CPU loading

55

Progress ReportTests TUNe Improvements

CPU Load OKMigration OKInterruption OKIntegrity OKDelay NOKStreaming NOK

Network Architecture

Software ArchitectureXen 3.1

Kernel 19-6

Kernel 18.8NFS root (domU)

DHCP Server

NFS Server

BugFilesystem

NFS Boot on DomU

Functionnal configuration

Kernel 22-14

Kernel 18.8Xen (dom0) Wrappers,

Deployment diagram

Reconf. Diagram,Probe TUNe modifications

Streaming scenario

kdeployJ2EE

Policies

NTP Server

DNS Server

Clock issues

Synchronisationissues

Issues resolved

Name resolution

Delay OKStreaming OK

56

Network Architecture

Software Architecture

Tests

CPU Load OKMigration OKInterruption OKIntegrity OKDelay NOKStreaming NOK

Kernel 18.8Xen (dom0)

NTP Server

Clock issues

Synchronisationissues

Issues resolved

Functionnal configuration

Delay OKStreaming OK

Progress ReportTUNe Improvements

Wrappers,Deployment diagram

Reconf. Diagram,Probe TUNe modifications

Streaming scenario

kdeployJ2EE

Policies

DNS Server

Name resolution

Xen 3.1

Kernel 19-6

Kernel 18.8NFS root (domU)

DHCP Server

NFS Server

BugFilesystem

NFS Boot on DomU

Kernel 22-14

57

Network Architecture

Tests TUNe Improvements

Wrappers,Deployment diagram

Reconf. Diagram,Probe TUNe modifications

Streaming scenario

kdeployJ2EE

Policies

DNS Server

Name resolution

Delay OKStreaming OK

Progress ReportSoftware

Architecture

CPU Load OKMigration OKInterruption OKIntegrity OKDelay NOKStreaming NOK

Kernel 18.8Xen (dom0)

NTP Server

Clock issues

Synchronisationissues

Issues resolved

Xen 3.1

Kernel 19-6

Kernel 18.8NFS root (domU)

DHCP Server

NFS Server

BugFilesystem

NFS Boot on DomU

Functionnal configuration

Kernel 22-14

58

Migration

TestsInterruption Duration

Dom0-1

DomU

Dom0-2

Pings

Results: 1s

Monitor Computer

59

Migration

TestsDelay

DomU

Dom0-2Dom0-1

LUFactorizations

60

TestsMigration duration

DomU

Dom0-2

Dom0-1

Chrono

Ping pong Migration

Results: - Migration: 16s- RMI: 10s