Nimrod/G and Grid Market“A Case for Economy Grid

Architecture for Service Oriented Global Grid Computing”

Rajkumar Buyya, David Abramson, Jon GiddyMonash University, Melbourne, Australiawww.buyya.com/ecogrid

www.gridcomputing.com

2100

2100 2100 2100 2100

2100 2100 2100 2100

Desktop SMPs or SuperComputersLocal

ClusterGlobal

Cluster/Grid

PERFORMANCE

Inter PlanetCluster/Grid ??

•Individual•Group•Department•Campus•State•National•Globe•Inter Planet•Universe

Administrative Barriers

EnterpriseCluster/Grid

?

Scalable HPC: Breaking Administrative Barriers

Why Grids ? Large Scale Exploration needs them—Killer

Applications. Solving grand challenge applications using

computer modeling, simulation and analysis

Life Sciences

CAD/CAM

Aerospace

Military ApplicationsDigital Biology Military ApplicationsMilitary Applications

Internet & Ecommerce

Players in Grid Computing

What users want ?Users in Grid Economy &

Strategy Grid Consumers

Execute jobs for solving varying problem size and complexity

Benefit by selecting and aggregating resources wisely Tradeoff timeframe and cost

Strategy: minimise expenses Grid Providers

Contribute “idle” resource for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements & market opportunity

Strategy: maximise returns on services

mix-and-match

Object-oriented

Internet-WWW

Problem Solving Approach

Market/Computational Economy

Grid Node N

Grid Architecture for Computational Economy

Grid User

Application

Grid Resource Broker

Grid Service Providers

Grid Explorer

Schedule Advisor

Trade Manager

Job ControlAgent

Deployment Agent

Trade Server

Resource Allocation

ResourceReservation

R1

Misc. services

Information Server(s)

R2 Rm…

Pricing Algorithms

Accounting

Grid Node1

…

Grid Middleware Services

…

…

HealthMonitor

Grid Market Services

JobExec

Info ?

Secure

Trading

QoS

Storage

Sign-on

Economy Grid = Globus + GRACE

Applications

MDS

GRAMGlobus Security Interface

Heartbeat MonitorNexus

Local Services

LSF

Condor GRD QBank

PBS

TCP

SolarisIrixLinux

UDP

High-level Services and Tools

DUROC globusrunMPI-G Nimrod/GMPI-IO CC++

GlobusView Grid Status

GASS

GRACE-TS

GARA

GridFabric

GridApps.

GridMiddleware

GridTools

GBankGMD

eCash

JVM

DUROC

Core Services

Science

Engineering Commerce Portals ActiveSheet…

A resource broker for managing and steering task farming (parametric sweep) applications on computational Grids based on deadline and computational economy.

Key Features A single window to manage & control experiment Resource Discovery Trade for Resources Resource Composition & Scheduling Steering & data management

It allows to study the behaviour of some of the output variables against a range of different input scenarios.

Nimrod/G : A Grid Resource Broker

A Glance at Nimrod-G Broker

Grid Middleware

Nimrod/G Client Nimrod/G ClientNimrod/G Client

Grid Information Server(s)

Schedule Advisor

Trading Manager

Nimrod/G Engine

GridStore

Grid Explorer

GE GISTM TS

RM & TS

Grid Dispatcher

RM: Local Resource Manager, TS: Trade Server

Globus,Legion, Condor-g,, Ninf,etc.

G

G

CL

Globus enabled node. Legion enabled node. C

L

Condor enabled node.

RM & TSRM & TS

A Nimrod/G ClientCostCost

DeadlineDeadline

Legion hosts

Globus Hosts

Bezek is in both Globus and Legion Domains

Arlington

Alexandria

Richmond

HamptonNorfolk

Virginia BeachChesapeakePortsmouth

Newport News

RoanokeApp om a toxRive r

Ja m esRive r

Shena ndo ahRive r

Rap pa hanno c kRive r

Potom acRive r

VIRGINIA77

81

64

64

66

85

Discover Discover ResourcesResources

Distribute JobsDistribute Jobs

Establish Establish RatesRates

Meet requirements ? Remaining Meet requirements ? Remaining Jobs, Deadline, & Budget ?Jobs, Deadline, & Budget ?

Evaluate & Evaluate & RescheduleReschedule

Discover Discover More More

ResourcesResources

Adaptive SchedulingAlgorithms

Execution Time (not beyond deadline)

Execution Cost (not beyond budget)

Time Minimisation Minimise Limited by budgetCost Minimisation Limited by deadline MinimiseNone Minimisation Limited by deadline Limited by budget

Adaptive Scheduling algorithms

Compose & Compose & ScheduleSchedule

Globus+Legion+Condor/G

Australia

Monash Uni.:

Linux cluster

Solaris WS

Nimrod/G

Globus +GRACE_TS

EuropeZIB/FUB: T3E/Mosix Cardiff: Sun E6500Paderborn: HPCLineLecce: Compaq SCCNR: ClusterCalabria: Cluster CERN: ClusterPozman: SGI/SP2

Globus +GRACE_TS

Asia/Japan

Tokyo I-Tech.:ETL, Tuskuba

Linux cluster

Globus/LegionGRACE_TS

North America

ANL: SGI/Sun/SP2USC-ISI: SGIUVa: Linux Cluster

Internet

Inter-Continental Grid

Experimentation on the Grid

Workload: 165 jobs, each need 5

minute of cpu time Deadline: 1 hrs. and

budget: 800,000 units Strategy: minimise cost

and meet deadline Execution Cost with cost

optimisation AU Peaktime:471205 (G$) AU Offpeak time: 427155

(G$)

Resource

Owner

Grid services

Peak time (G$)

Off peak cost

Linux cluster (60 nodes)

Monash, Australia

Globus/Condor

20 5

IBM SP2 (80 nodes)

ANL, Chicago, US

Globus/LL

5 10

Sun (8 nodes)

ANL, Chicago, US

Globus/Fork

5 10

SGI (96 nodes)

ANL, Chicago, US

Globus/Condor-G

15 15

SGI (10 nodes)

ISI, LA, US

Globus/Fork

10 20

Execution @ AU Peak Time

0

2

4

6

8

10

12

Time (minutes)

Jobs

Linux clus ter - Monash (20) Sun - ANL (5) SP2 - ANL (5) SGI - ANL (15) SGI - ISI (10)

Execution @ AU Offpeak Time

0

2

4

6

8

10

12

Time (minutes)

Jobs

Linux cluster - Monash (5) Sun - ANL (10) SP2 - ANL (10) SGI - ANL (15) SGI - ISI (20)