Virtual Infrastructure in the Grid

Kate Keaheykeahey@mcs.anl.govArgonne National

Laboratory

01/30/06 MSI training Event

The Grid Metaphor

01/30/06 MSI training Event

The Grid Metaphor

01/30/06 MSI training Event

The Grid Metaphor

How do we store energy?

How do we charge for energy?

How do we reliably deliver energy?

What happens if a power station fails?

How do we ensure quality of service?

What elements make for a safe and efficient power Grid?

How do we make sure that supply meets demand?

01/30/06 MSI training Event

Providers and Consumers Providers

Own, operate, and contribute physical resources Require incentives to participate Low participation costs Protection from activities of the consumer Ability to control and monitor resource usage

Consumers Want on-demand access to computational resources at modest cost

The ability to configure them to meet their needs Reasonable guarantees of resource availability

Scalability Provider and consumer roles have to be decoupled

01/30/06 MSI training Event

Requirements for a Grid Execution Environment

Environment and configuration A VO should be able to provide the configuration it needs independently of the resource provider

Isolation The provider needs to be able to delegate resource usage to the VO so that the VO can’t impact the resource provider -- and thus does not need to be under its control

Resource usage and accounting The provider needs to be able to grant, enforce and account for VO resource usage in a way that is independent of how the resource is consumed

All of this must be available on-demand!

01/30/06 MSI training Event

Virtual Workspaces A virtual workspace is an abstraction of an

execution environment that can be made dynamically available to authorized clients by using well-defined protocols.

Two dimensions: Software configuration Resource quota (CPU, memory, etc.)

Examples of Workspaces: A physical machine configured to meet TeraGrid requirements

A cluster of virtual machines configured to meet OSG requirements

A cluster of physical machines running a hypervisor

01/30/06 MSI training Event

Virtual Workspace Implementations

Physical resources Allocate and configure a physical resource

Cluster on Demand (COD), Duke University Bcfg project at ANL

As a method they are inflexible and coarse-grained

Virtual resources Allocate resources for and deploy configured virtual machines

Existing efforts: In-Vigo, Virtuoso, VIOLIN, the workspace project…

Much more flexible, allowing migration and fine-grain enforcement.

01/30/06 MSI training Event

Virtual Machine Basics

Hardware

Virtual Machine Monitor (VMM) / Hypervisor

Guest OS(Linux)

Guest OS(NetBSD)

Guest OS(Windows)

A VM can serialize all of its state (including RAM) A VM image is simply a collection of files

Disk partitions, RAM, configuration file

Such image can be easily moved (migrated) between hypervisors of the same type

Such image can also be saved and used for rollbacks

VM VM VM

AppApp AppAppApp

01/30/06 MSI training Event

The Need for Speed

L X V U

SPEC INT2000 (score)

L X V U

Linux build time (s)

L X V U

OSDB-OLTP (tup/s)

L X V U

SPEC WEB99 (score)

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1.1

Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)

Paper: “Xen and the Art of Virtualization”, SOSP 2003

01/30/06 MSI training Event

Summary: What Makes VMs a Great Workspace Implementation

VM properties: Excellent isolation

Generally enhanced security, audit forensics Fine-grain enforcement potential

Details depend on implementation Customizable software configuration

Library signature, OS, maybe even 64/32-bit architectures

Serialization property VM images (include RAM), can be copied

The ability to pause and resume computations Allow migration

How do we make VMs available over the network and manage them so as to leverage this potential? Challenges: security, enforcement, protocols

01/30/06 MSI training Event

Deploying Workspaces in the Grid

WorkspaceWizard

(VW Factory)

Workspace ManagementService

(VW Repository)

Workspace Service

(VW Manager)

request a workspace

workspace meta-data

manage workspace environment

workspace metadata

terminate workspace deployment

negotiate workspace deployment

manage/monitor/renegotiate workspace deployment

manage activities within the workspace

Workspace

01/30/06 MSI training Event

Workspace Implementation Protocols: Web Service Resource Framework (WSRF)

An extension of Web Services Standard mechanisms for creation, inspection, notification, lifetime management

Globus Toolkit 4 implementation Provides secure authentication, authorization as well as tools for fast transfer, replica management, monitoring, and others.

Creating a workspace workspace meta-data (workspace image) deployment descriptor (resource allocation)

Managing a workspace renegotiate resource allocation Standard WSRF management functions

Challenges: resource assignment, negotiation, etc. To download visit http://workspace.globus.org

01/30/06 MSI training Event

Putting it All Together

R R R R RR R R R

A CB

VM1 VM2 VM3 VM4

resources

deployment capabilities

virtual machines

jobs

B D B Edeployment capabilities

Deploying a workspace requires and creates a deployment capability Required capability is described in workspace pre-requisites

Workspaces can be layered

01/30/06 MSI training Event

Applications: Edge Services (1)

Edge Service: service executing on the edge of private and public network

ESF Requirements Diverse configurations, easy to upgrade Good potential for managing resource allocation

Status: Testbed: SDSC, FNAL, UC Multiple base images have been developed One Edge Service deployed Workspace Service developed

Timeframe: ~few months

01/30/06 MSI training Event

Applications: Edge Services (2)

CDF

CMS ATLAS

Guest VO

ESF

SE CE

Site

GT4 Workspace Service & VMM

Dynamically deployed ES Wafers for each VO

Wafer images stored in SE

Compute nodes and Storage nodes

01/30/06 MSI training Event

Applications: Virtual Clusters

Extends the abstraction of a workspace to a virtual cluster

Deploys a cluster on a site SLURM, PBS/Torque implementations Configures networking, shared storage Image propagation, main deployment cost

Tech report available

01/30/06 MSI training Event

Virtual Cluster: OSG Applications

f M R I E x e c u t i o n T i m e

0

1 0 0 0

2 0 0 0

3 0 0 0

4 0 0 0

5 0 0 0

6 0 0 0

7 0 0 0

8 0 0 0

1 2 4 8

C l u s t e r S i z e ( # o f W N )

Workflow Execution Time (sec)

V - O S G C - O S G

0

1 0 0 0

2 0 0 0

3 0 0 0

4 0 0 0

5 0 0 0

6 0 0 0

7 0 0 0

1 2 4 8

C l u s t e r S i z e ( # o f W N )

Workflow Execution Time (Sec)

V - O S G

C - O S G

M o n t a g e E x e c u t i o n T i m e ( 2 0 1 j o b s )

0

2 0 0 0

4 0 0 0

6 0 0 0

8 0 0 0

1 0 0 0 0

1 2 0 0 0

1 4 0 0 0

1 6 0 0 0

1 8 0 0 0

2 4 8

C l u s t e r S i z e ( # o f W N )

Workflow Execution Time (Sec)

V - O S G C - O S G

F O A M E x e c u t i o n T i m e

0

5 0

1 0 0

1 5 0

2 0 0

2 5 0

3 0 0

3 5 0

4 0 0

4 5 0

5 0 0

4 8

C l u s t e r S i z e ( # o f W N )

Workflow Execution Time (Sec)

V - O S G C - O S G

GADU fMRI

Montage FOAM

01/30/06 MSI training Event

Conclusions

In order to grow, we need to scale In order to scale, we need to provide a

reliable tool for separating producers and consumers

Virtualization provides a useful, scalable tool to decouple providers and consumers Workspaces as physical resources Workspaces as virtual machines

Looking forward Grid economies