open science grid living on the edge: osg edge services framework kate keahey abhishek rana

Open Science Grid

Living on the Edge:OSG Edge Services

FrameworkKate Keahey

Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 2

Open Science Grid

Edge Services

• Services executing on the edge of the public and private network

Site

CDFCMS ATLAS

GuestVO SECE

Compute nodes and Storage nodes

SC05 booth presentation OSG Edge Services Framework 3

Open Science Grid

Edge Service Requirements

• Configuration Issues– The configuration requirements can vary widely per

Edge Service per community.– We need an easy way to upgrade Edge Services – Alternative Edge Services

• Resource allocation issues– Flexibility in dynamically managing resource

assignments as load demands fluctuate– Fair usage between communities– Incentives

SC05 booth presentation OSG Edge Services Framework 4

Open Science Grid

Virtual Machines• Excellent isolation, security, fine-grain forensics…• Promising enforcement properties• More control to the site & great recoverability• Flexible configuration• The ability to serialize and migrate• Modern implementation are fast

L X V USPEC INT2000 (score)

L X V ULinux build time (s)

L X V UOSDB-OLTP (tup/s)

L X V USPEC WEB99 (score)

0.00.10.20.30.40.50.60.70.80.91.01.1

Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)

SC05 booth presentation OSG Edge Services Framework 5

Open Science Grid

Virtual Workspaces

• Virtual Workspaces: environments that can be made available dynamically on the Grid – Different possible implementations

• Workspace Service– Enables secure deployment and management of a

workspace within a trusted computing base• Authentication and authorization• Deployment configuration• Resource allocation enforcement

– Workspace = deployment meta-data + image– More information and technology preview download

available at http://workspace.globus.org

SC05 booth presentation OSG Edge Services Framework 6

Open Science Grid

Edge Service Framework

Edge Service Framework

Staging Tools

WorkspaceService

ImageRepository

Image Information

ESF - Phase 1

SC05 booth presentation OSG Edge Services Framework 7

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

SC05 booth presentation OSG Edge Services Framework 8

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

SC05 booth presentation OSG Edge Services Framework 9

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

Role=VO Admin

SC05 booth presentation OSG Edge Services Framework 10

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

Role=VO Admin

SC05 booth presentation OSG Edge Services Framework 11

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

SC05 booth presentation OSG Edge Services Framework 12

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

SC05 booth presentation OSG Edge Services Framework 13

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

RemoteEntity

SC05 booth presentation OSG Edge Services Framework 14

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO User

SC05 booth presentation OSG Edge Services Framework 15

Open Science Grid

ESF - Phase 1

ESF

SE

Site

CMS

Role=VO User

CE

SC05 booth presentation OSG Edge Services Framework 16

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO User

SC05 booth presentation OSG Edge Services Framework 17

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO User

SC05 booth presentation OSG Edge Services Framework 18

Open Science Grid

ESF Project

• Phase 1 objective: create a proof-of-concept dynamic deployment of Edge Services

• Progress:• Implement and release the Workspace Service• Bubbly test: static deployment and test of a real

Edge Service on a production site• Dynamic deployment of an Edge Service and

integration into OSG infrastructure• Configuring more testbeds• Deploying and testing more Edge Services

SC05 booth presentation OSG Edge Services Framework 19

Open Science Grid

Testbed & Images• Testbed

– SDSC (Abhishek)• OSG production site, 150+ worker nodes• Intel 2.6, 4GB, RAID device, Xen 2.0.7

– FNAL (Burt)• Xeon 3.2, 4GB, RAID device, Xen 2.0.7

– UC (Rob, Borja)• TeraPort, Xen 3

– ANL, PIOCON coming online• Images

– SDSC: SL3.0.3, FC4, CentOS4.1– FNAL: SL3.0.3, SL4, LTS 3, LTS 4

• Edge Services– ATLAS: mysql-gsi db built by the DASH project

SC05 booth presentation OSG Edge Services Framework 20

Open Science Grid

Summary

• Growing ESF testbed– Xen capabilities / ESF requirements– Installation woes getting worked out

• Growing library of images with useful configurations

• Growing infrastructure

• www.opensciencegrid.org/esf