– n° 1

Review resources access policy, procedures,

rules and challenges:

The Italian experience and future challenges

Antonia Ghiselli

INFN-CNAF

Workshop on eInfrastructures (Internet and Grids)

The new foundation for knowledge-base Societies

Roma, Accademia Nazionale dei Lincei

9 December 2003

– n° 2

Outline Introduction:

INFN resource sharing experience in the past

INFN-Grid and the national research grid Goals and Results

Italian-Grid present status Resource access mechanism and management tools

production service :Management, operations and support organization

International Grid scenario: LCG and EGEE Challenges: Multi-grids for multi-VOs

Multi–grids :definitions and issues

Conclusions

– n° 3

INFN Computing Resource sharing in the past

80th

RJE to INFN resources by INFN users

Resource sharing within a single distributed community (agreement between sites based on common convenience )

Access policy agreement: low priority queues during the

night

Proxy logins mechanism

TORINO PADOVA

BARI

PALERMO

FIRENZE

PAVIA

GENOVA

NAPOLI

CAGLIARI

TRIESTE

ROMA

PISA

L’AQUILA

CATANIA

BOLOGNA

UDINE

TRENTO

PERUGIA

LNF

LNGS

SASSARI

LECCE

LNS

LNL

SALERNO

COSENZA

S.Piero

FERRARAPARMA

CNAF

ROMA2

MILANO

Network

useruser

user

user

user

VAX/VMS cluster

– n° 4

INFN Computing Resource sharing in the past

90th : Condor – INFN collaboration

Condor submit to INFN desktops and workstations

Users Resource sharing by INFN users

Access policy agreement: transparent access through CPU cycle stealing

~300 machines, still up.

TORINO PADOVA

BARI

PALERMO

FIRENZE

PAVIA

GENOVA

NAPOLI

CAGLIARI

TRIESTE

ROMA

PISA

L’AQUILA

CATANIA

BOLOGNA

UDINE

TRENTO

PERUGIA

LNF

LNGS

SASSARI

LECCE

LNS

LNL

SALERNO

COSENZA

S.Piero

FERRARAPARMA

CNAF

ROMA2

MILANO

Condoron WAN

useruser

user

user

user

– n° 5

INFN Computing Resource sharing in the past

1999

Globus evaluation on WAN

Preliminary grid tests to the INFN-Grid project.

TORINO PADOVA

BARI

PALERMO

FIRENZE

PAVIA

GENOVA

NAPOLI

CAGLIARI

TRIESTE

ROMA

PISA

L’AQUILA

CATANIA

BOLOGNA

UDINE

TRENTO

PERUGIA

LNF

LNGS

SASSARI

LECCE

LNS

LNL

SALERNO

COSENZA

S.Piero

FERRARAPARMA

CNAF

ROMA2

MILANO

Globustest

useruser

user

user

user

– n° 6

INFN-Grid – goals (started at 2000)1. To promote computational grid technologies research & development:

Middleware

1. Through european and international projects1. DataGrid, DataTAG, GLUE

2. Internal R&D activities

2. To implement the INFN grid infrastructure

1. National layout: 20 sites

3. To set up the national Grid Infrastructure for the national research community

1. FIRB: Grid.it

4. To participate to the implementation of the global Grid infrastructure for the LHC community

1. LCG: Tier1 and n*Tier2

5. To set up the eInfrastructure for the European Research Area

1. EU FP6: EGEE, IG-BIGEST

– n° 7

INFN-Grid – collaborations and results EU - Datagrid : middleware development

WMS = job submission to the Grid, CE and SE selection on the basis of job requirements specification, CPU load, CE-SE network

conditions….. Support for interactive jobs Job checkpointing Support for parallel jobs

Virtual Organization authentication and authorization service: VOMS (VO Membership Service, EDG/EDT)

EU – DataTAG : inter-grid Interoperability; EU-US collaboration within the GLUE framework

Grid Resources Information modeling: GLUE schema for Computing and Storage Element

Authorization/authentication service : VOMS-VOX integration (EDT-Fnal/CMS coll.) First WorldGrid demo by nov.2002 within IST2002 and SC2002 events Grid monitoring system based on GLUE schemas extension

Italian Grid.it : Grid management and support infrastructure First tools in production R&D on Resource Utilization Policies

– n° 8

TORINO PADOVA

BARI

PALERMO

FIRENZE

PAVIA

GENOVA

NAPOLI

CAGLIARI

TRIESTE

ROMA

PISA

L’AQUILA

CATANIA

BOLOGNA

UDINE

TRENTO

PERUGIA

LNF

LNGS

SASSARI

LECCE

LNS

LNL

SALERNO

COSENZA

S.Piero

FERRARAPARMA

CNAF

ROMA2

INFN CMS T2 T2/3 Atlas T2 T2/3 Alice T2 T2/3 LHCb T2 T2/3 Babar VIRGOT2 (50-80 nodes)T3 (10-15 nodes)T1 Cnaf (~200) grid.it resources INFN (15-25 nodes) INAF (5-10 nodes) INGV (NEC computers), BIO (tbd) general purpose resources (8-15 nodes)

Italian – Grid now (Site/resource map)

MILANO

National Grid(Internet)

Tot. ~ 600 nodes , next year ~ 1000

– n° 9

Resource access policies: Basic grid Authorization, authentication mechanisms

Security characteristics:

Login via X.509 certificates from PKI/Certificate Authorities (CA)

Single sign-on. The user is not required to repeat login procedures on the grid more than once.

Delegation. Once a user has successfully identified himself with the Grid, it is possible for grid

services to act on the behalf of the user as if they were the user himself.

User-based trust relationship. All trust mechanism have the user’s credential at their core.

If a user wants to access farms A and B, there should be no need for farms A and B to trust each other.

Integrated with local systems. The grid security mechanism does not supplant the local authorization

mechanism, but instead work on top of it.

New membership concept: user belongs to a Virtual Organization

– n° 10

User: CA, VO and Resource Providers Certificates are issued by a set of well-defined Certification

Authorities (CAs).

Grant authorization at the VO level. Each VO has its own VOMS server. Contains (group / role / capabilities) triples for each member of the VO.

RP’s evaluate authorization granted by VO to a user and map into local credentials to access resources

Authentication

Request

C=IT/O=INFN /L=CNAF/CN=Pinco Palla/CN=proxy

VOMSpseudo-cert

VOMSpseudo-cert

CA’s

CERN

CESNET

CNRS

GermanGrid

Grid-Ireland

INFN

NIKHEF

NorduGrid

LIP

Russian DataGrid

DATAGRID-ES

GridPP

US–DOE Root CA

US-DOE Sub CA

CrossGrid

cert-request

cert signing

cert/crl update

Service

VO-Manager(administer user membership, roles and Capabilities)

Resource provider(map into Local credential)

CAs: Policies and procedures mutual thrust

agreement

– n° 11

Resource access policies Authentication/ authorization: coded and tested procedures and tools

New issue : resource sharing according to Service Level Agreement first trials based on “grid level priority queues” ongoing research on more sophisticated mechanisms based on accounting +

resource utilization Policies management

Grid managementorganization

VO-users(RequirementsSupport)

Resource providers / AA/SLA

VO-managers(VOMS and SLA Control)

Certificate Authorities

Grid deployment planningGrid operations / support

Grid release

– n° 12

Italian Grid organization : integrates all the actors to provide flexible and efficient grid computing service

Experiments (VOs)GRID resources

Projects/owners

Coordination Committee

Management coordination

Operationscoordination

VO representatives, Grid technical coord.,Operations resp.grid experts

•Deployment Planning• resource Policy application•…….

Central management Team

Site-manResource admin

GridServicesupport

VO adminNew VO admin& support

VO Usersupport

User Application

Grid ResourceCoordination

Experimemt or research org. support

releaseConfigurationmanagement

Release distribution, documentation and porting

Grid Technical coordinationService level AgreementResource availabilityShared resources

VO admin

Support for New VO-users

– n° 13

Tools for Operations

Software repository : release maintenance and distribution

Installation and configuration: Configuration and automatic installation tools for the production infrastructure

sites

Release validation: Integration/customization of middleware release with application specific

software

GRID Site and GRID service validation Testing programs to verify and validate site and services installation

Site manager support

Grid services, VO services support and User support

Monitoring: GridICE Based on automatic resource discovery from Grid Information System Dynamic monitoring of Grid services, Grid resources and Jobs Customized view for

Grid Operation Center operators, and site managers VO-managers and Grid Users

– n° 14

0perations Portal

User documentation

site managers documentation

Software repository

Monitoring

Trouble tickets system

Knowledge base

http://grid-it.cnaf.infn.it

– n° 15

Get your personal certificate

– n° 16

How to register to a VO

– n° 17

Monitoring tool

– n° 18

VO serveratlasVO server

atlas

Grid services

INGV-Bologna

Computing Element

Storage Element

GIIS GRIS1

GRIS

Information IndexResource Broker

UserInterface

GRAM

BDII

VO serveringv

WorkerNodeWorkerNode

...WorkerNode

INFN-Padova

Computing Element

Storage Element

GIIS GRIS1

GRIS

GRAM

WorkerNodeWorkerNode

...WorkerNode

Grid Monitoring(GridICE)

RLS

– n° 19

Grid Service monitoring

– n° 20

Outline Introduction:

INFN resource sharing experience in the past

INFN-Grid and the national research grid Goals and Results

Italian-Grid present status Resource access mechanism and management tools

production service :Management, operations and support organization

International Grid scenario: LCG and EGEE Challenges: Multi-grids for multi-VOs

Multi-grids: definitions and issues

Conclusions

– n° 21

International Grids scenario

LCG : First international experience on sharing resources between national grids

Grid Resource sharing issues : how to guarantee the committed CPU power and satisfy local needs How to guarantee priorities on VO-owned resources

Different needs for different VOs (HEP experiments plans)

Management coordination

Support coordination

EGEE : project based on national grids interconnection for an increased number of VOs

Not only middleware but mainly policies, service level agreement and management coordination issues

Need to find a model …..

– n° 22

Grid access challenge: Grid and Virtual Organisations

The real problem at the basis of the grid idea is how to implement a coordinated resource sharing on a large scale for a multi-institutional and dynamic virtual organisation.

-

From computer sharing to grid sharing

From multiple users to multiple VOs (INFN experiments + others research organizations)

– n° 23

Challenges: Capability to provide multi-Grid computing service to Multi-VO

Shared Resources and Services

VO servicesand private resources

VO services

and private

resources

VO services VO services

Shared Resources and Services

VO servicesand private resources

Shared Resources and Services

General scenario

– n° 24

International VO is a multi-institutional distributed user community

Etherogeneous grid environment Dedicated VO services Dedicated resources Shared resources with different policies

EGEEItalian-Grid US-Grid

same middlewareshared resources

VO-User

VO-User

VO-User

VO-Virtual Grid on top of Multi-Grids

same core services

RB VOMSVO-monitoring

Vo-RLS

VO - Virtual GridRB

National and International Grids

Coordinated Vo-support

– n° 25

multi - grids : definitions and issues

National grid identity and authority boundaries A coordinated set of shared resources and services providing defined

SLAs.

A single management and operations organization

Specific authorization, accounting and monitoring tools

A collection of user communities (VOs)

Federation of grids, what does’t mean? Cooperating grids to provide services to the common VOs?

Which level of transparency to VO-users?

Which Interoperability Requirements: common core services? common or interoperable collective services? (level of service interoperability) Common Resource sharing policies?

What level of management/operations/support coordinations?

– n° 26

Conclusions

Production grid does not mean only efficient, stable services but also:

A topology/organizational model capable to provide the most flexible and efficient computing service to VO-users across multiple grids

Sufficient level of service quality (SLA)

Operations and support coordination

the minimum level of interoperability in order to allow VO virtual grid configuration across multiple grids