open science grid living on the edge: osg edge services framework kate keahey abhishek rana

20
Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

Upload: jack-daley

Post on 27-Mar-2015

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

Open Science Grid

Living on the Edge:OSG Edge Services

FrameworkKate Keahey

Abhishek Rana

Page 2: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 2

Open Science Grid

Edge Services

• Services executing on the edge of the public and private network

Site

CDFCMS ATLAS

GuestVO SECE

Compute nodes and Storage nodes

Page 3: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 3

Open Science Grid

Edge Service Requirements

• Configuration Issues– The configuration requirements can vary widely per

Edge Service per community.– We need an easy way to upgrade Edge Services – Alternative Edge Services

• Resource allocation issues– Flexibility in dynamically managing resource

assignments as load demands fluctuate– Fair usage between communities– Incentives

Page 4: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 4

Open Science Grid

Virtual Machines• Excellent isolation, security, fine-grain forensics…• Promising enforcement properties• More control to the site & great recoverability• Flexible configuration• The ability to serialize and migrate• Modern implementation are fast

L X V USPEC INT2000 (score)

L X V ULinux build time (s)

L X V UOSDB-OLTP (tup/s)

L X V USPEC WEB99 (score)

0.00.10.20.30.40.50.60.70.80.91.01.1

Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)

Page 5: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 5

Open Science Grid

Virtual Workspaces

• Virtual Workspaces: environments that can be made available dynamically on the Grid – Different possible implementations

• Workspace Service– Enables secure deployment and management of a

workspace within a trusted computing base• Authentication and authorization• Deployment configuration• Resource allocation enforcement

– Workspace = deployment meta-data + image– More information and technology preview download

available at http://workspace.globus.org

Page 6: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 6

Open Science Grid

Edge Service Framework

Edge Service Framework

Staging Tools

WorkspaceService

ImageRepository

Image Information

ESF - Phase 1

Page 7: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 7

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

Page 8: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 8

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

Page 9: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 9

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

Role=VO Admin

Page 10: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 10

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

Role=VO Admin

Page 11: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 11

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

Page 12: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 12

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

Page 13: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 13

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

RemoteEntity

Page 14: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 14

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO User

Page 15: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 15

Open Science Grid

ESF - Phase 1

ESF

SE

Site

CMS

Role=VO User

CE

Page 16: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 16

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO User

Page 17: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 17

Open Science Grid

ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO User

Page 18: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 18

Open Science Grid

ESF Project

• Phase 1 objective: create a proof-of-concept dynamic deployment of Edge Services

• Progress:• Implement and release the Workspace Service• Bubbly test: static deployment and test of a real

Edge Service on a production site• Dynamic deployment of an Edge Service and

integration into OSG infrastructure• Configuring more testbeds• Deploying and testing more Edge Services

Page 19: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 19

Open Science Grid

Testbed & Images• Testbed

– SDSC (Abhishek)• OSG production site, 150+ worker nodes• Intel 2.6, 4GB, RAID device, Xen 2.0.7

– FNAL (Burt)• Xeon 3.2, 4GB, RAID device, Xen 2.0.7

– UC (Rob, Borja)• TeraPort, Xen 3

– ANL, PIOCON coming online• Images

– SDSC: SL3.0.3, FC4, CentOS4.1– FNAL: SL3.0.3, SL4, LTS 3, LTS 4

• Edge Services– ATLAS: mysql-gsi db built by the DASH project

Page 20: Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana

SC05 booth presentation OSG Edge Services Framework 20

Open Science Grid

Summary

• Growing ESF testbed– Xen capabilities / ESF requirements– Installation woes getting worked out

• Growing library of images with useful configurations

• Growing infrastructure

• www.opensciencegrid.org/esf