The future of Globus(Grid meets Cloud)

Ian Foster

Computation Institute

University of Chicago & Argonne National Laboratory

2

UC/ANL/ISI perspective on Globus

Globus is a community of contributors This presentation is focused on UC/ANL/ISI

planned contributions Globus funding @ UC/ANL

NSF CDIGS: development & support DOE CEDPS: Globus.org data grid R&D caBIG/caGrid: for biomedical applications Many others…

3

UC/ANL leadership and structure

Steve Tuecke resuming day-to-day technical and project leadership

Product teams Adopted Agile scrum methods Jira.globus.org with Greenhopper

User services Consulting, operations and support Led by Paul Dave’

Grid = federation

Cloud = hosting

5

Anatomy of the Grid

Application

Fabric“Controlling things locally”: Access to, and control of resources

Connectivity“Talking to things”: communication (Internet protocols) and security

Resource“Sharing single resources”: negotiating access, controlling use

Collective“Coordinating multiple resources”: ubiquitous infrastructure services, app-specific distributed services

InternetTransport

Application

Link

Inte

rnet P

roto

col A

rchite

cture

“The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, Foster, Kesselman, Tuecke, Intl Journal of High Performance Computing Applications, 15(3), 2001.

6

Cloud Taxonomy

Software-as-a-Service (SaaS)

Platform-as-a-Service (PaaS)

Infrastructure-as-a-Service (IaaS)

7

Some strengths of Globus

Grid Security Infrastructure ecosystem GridFTP ecosystem GT4 Java Core use by caGrid Strong community

8

Some issues Facing Globus

Aging technology base Apache Axis 1.x, PureTLS

Sustainability Limited adoption of some components

MDS, RFT Confusing duplication

GRAM2 vs GRAM4 Limited collective layer, end-to-end

functionality

9

Globus ToolkitAssortment of Components for Grid Builders

Focus on Connectivity and Resource layers GRAM, GSI-OpenSSH: Run programs GridFTP: Access file systems OGSA-DAI, caGrid: Access databases GSI, Myproxy, GAARDS: Security XIO, Java Core, C Core: Communication

A few simple Collective layer components RLS: Replica tracking RFT: Reliable file transfer

10

GRAM5

Now you can have reliability and scalability Backward compatible with GRAM2

2 minor exceptions Job rendezvous for MPICH-G, stdout/err streaming

Tested w/ GRAM4 Java & C clients, Condor-G Improved functionality

Exit codes Better logging Enhanced metrics Etc.

11

Reliable File Transfer (RFT)

Replaced by Globus.org

12

MDS

Mostly used for service registries/catalogs Starting new IIS effort with this focus

Better tools (e.g. Nagios) for monitoring

13

Other Components Continuing

GridFTP RLS Myproxy GSI-OpenSSH

GAARDS Introduce OpenNebula Nimbus …

14

Metrics

New metrics collector More reliable and scalable

New metrics generators GRAM5 Myproxy Others coming

15

Globus Toolkit Roadmap

GT 5.0: 4Q2009 GridFTP, GRAM5, RSL, Myproxy, GSI-OpenSSH C libraries: GSSAPI, XIO, C Core, etc jGlobus: Security, GRAM client, GridFTP client

GT 5.2: 2Q2010 Focus: Native Packaging

GT 5.4: 4Q2010 Focus: Usability

GT 4.x maintenance and support will continue at least through end of 2010

16

CruxJava SOA for Grid Applications

Core: Apache CXF + Globus security Also: Introduce + OSGi + ServiceMix ESB WSRF subset compatibility supported Taverna workflows

Replacement for GT4 Java Core Programming model not the same

17

Many custom Collective layers

Most Grid deployments have custom, domain-specific Collective layer Built on common Resource layer components E.g. Data transfer and mirroring, workflows, …

Challenges: Expensive to develop Expensive to operating and supporting Useful for narrow community

How do we make these capabilities available to more users?

18

Globus.org ServiceCollective Layer Grid Services via Cloud SaaS

End-to-end collective layer functionality targeted toward end users Generalize lessons from custom Grids Focus on ease of use, federation

Hosted and supported by Globus team Initial focus on file transfer

Near term: Add sync, mirroring, caching Long term: Add job execution, workflows, VO

management

19

Applying SaaS Techniques

Service: Built as scale-out web application Hosted on Amazon Web Services

Client: Minimize software deployment Web 2.0

AJAX + REST Notification via email, IM, SMS, Twitter, etc. Enable mash-ups

CLI 2.0 ssh cli.globus.org …

Dynamic deployment of resource integrators Mobile clients

20

Web 2.0 GUI

21

iPhone Client

22

Upcoming Events

GlobusWorld: March 2-4 @ Argonne

Globus governance

24

“Governance” issues

Who decides what is called Globus? How do we encourage contributions? How do we encourage the development of

a healthy ecosystem of producers and consumers?

How do we achieve sustainability?

25

Governance history

First, Globus Project Argonne and USC/ISI

Then, Globus Alliance NCSA, Edinburgh, U.Chicago Commitment to cooperate

And now, dev.globus Apache-like governance structure Globus Management Committee Dev.globus incubator process