using grid computing at nikhef

21
Using Grid Computing at NIKHEF David Groep, NIKHEF 2001-10-01

Upload: shepry

Post on 22-Feb-2016

36 views

Category:

Documents


0 download

DESCRIPTION

Using Grid Computing at NIKHEF. David Groep, NIKHEF 2001-10-01. The One-Liner. R esource sharing and coordinated problem solving in dynamic multi-institutional virtual organisations. What is Grid computing?. Dependable , consistent and pervasive access - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Using Grid Computing at NIKHEF

Using Grid Computingat NIKHEF

David Groep, NIKHEF2001-10-01

Page 2: Using Grid Computing at NIKHEF

The One-Liner

• Resource sharing and coordinated problem solving in dynamic multi-institutional virtual organisations

Page 3: Using Grid Computing at NIKHEF

What is Grid computing?

• Dependable, consistent and pervasive access• Combining resources from various organizations

• `Virtual Organizations’ – user-based view on Grid

• Technical challenges:– transparent decisions for the user– uniformity in access methods– secure & crack resistant– authentication, authorization, accounting (AAA) &quota

Page 4: Using Grid Computing at NIKHEF

• Globus Project started 1997• de facto-standard• Reference implementation of Gridforum standards

• Large community effort• Basis of several projects, including EU-DataGrid

• Toolkit `bag-of-services' approach

• Successful test beds, with single sign-on, etc…

Grid Middleware

Page 5: Using Grid Computing at NIKHEF

Grid Architecture

Applications

Grid Services GRAM

Grid Security Infrastructure (GSI)

Grid FabricCondor MPI PBS Internet Linux

Application ToolkitsDUROC MPICH-G2Condor-G

GridFTPMDS

SUN

VLAM-G

Make all resources talk standard protocols

Promote interoperability of application toolkit, similar to interoperability of networks by Internet standards

ReplicaSrv

Page 6: Using Grid Computing at NIKHEF

Looking for Resources

• Per Virtual Organization (or test bed)

• Directory of Resources and their Characteristics

• Used to find `best resource out there’

DataGrid http://marianne.in2p3.fr/DutchGrid ldap://giishost.nikhef.nl:30001/o=Grid

Page 7: Using Grid Computing at NIKHEF

Submitting a Job

Page 8: Using Grid Computing at NIKHEF

Sending your Data

• Tape robots, disks, etc. share GridFTP interface• Optimize for high-speed >1Gbit/s networks

• In the future: automatic optimizations, bandwidth reservations, directory-enabled networking, …

Page 9: Using Grid Computing at NIKHEF

DataGrid Test Bed 1

• DataGrid TB1:– 14 countries– 21 major sites

– “Work Package 6”

• Submitting Jobs:– Login only once,

run everywhere– Cross administrative

boundaries in asecure and trusted way

– Mutual authorization

Page 10: Using Grid Computing at NIKHEF

DutchGrid Platform

Amsterdam

UtrechtKNMI

Delft

Leiden

Nijmegen

Enschede

• DutchGrid:– Test bed coordination– PKI security

• Participation byNIKHEF:

FOM, VU, UvA, Utrecht, Nijmegen

KNMI, SARAAMOLFDAS-II (ASCI):

TUDelft, Leiden, VU, UvA, Utrecht

Telematics Institute

Page 11: Using Grid Computing at NIKHEF

Systems around WCW

Page 12: Using Grid Computing at NIKHEF

Start using the grid

• All the necessary “client tools” are on all Linux and Solaris systems

• You just need:– Credentials/tokens for the Grid (see next slides)– Authorization to use resources

(you get all NIKHEF resources by default)– Information on which resources to use effectively

Page 13: Using Grid Computing at NIKHEF

Your Grid Credentials

• You will use resources across several domains– You may not care about security and authorization– But the remote site admin will !

• All communications are authenticated usingX.509 “Public Key” Certificates

• The technology used to securecredit card transactions on the web (https://……)

• Uniquely binds name/affiliation to a digital token

Page 14: Using Grid Computing at NIKHEF

Certification Authorities

• CA’s act as trusted third parties

• Remote sites trust the CA for a proper binding• They will not do authentication again, so

only authorization left.

• CA’s are highly valuable: crack one to impersonate others on the Grid

(and abuse resources)

• Registration Authorities do in-person ID checks

Page 15: Using Grid Computing at NIKHEF

CA’s in DataGrid

• 10 National CA’s (one per EU country)• Each one has a detailed

policy and practice statement

• NIKHEF operates the CA for DutchGridSee http://certificate.nikhef.nl/

• Get a “certificate” from the DutchGrid CAbefore you can start using the Grid

• It’s valuable, protect it with a pass phrase• One cert valid for all DataGrid sites

Page 16: Using Grid Computing at NIKHEF

The Proxy

• A `proxy certificate’ is a limited-lifetime delegationwithout a pass phrase to protect it

• Implements the single sign-on for Grid• Valid for 12 hours (by default)

• Use it to:– Run your jobs– Get access to your data

• Get it, by running grid-proxy-init

Page 17: Using Grid Computing at NIKHEF

Now see for yourself

Page 18: Using Grid Computing at NIKHEF

Getting a Certificate

• Initialize your environment for the Grid• Use the Globus local guide from

http://www.dutchgrid.nl/Org/Nikhef/• Run grid-cert-request• Send the result to [email protected]

you will be contacted by phone• Put the certificate (sent by mail) in your$HOME/.globus/usercert.pem

• Or use the Web at http://certificate.nikhef.nl/cgi-bin/reqconf

Page 19: Using Grid Computing at NIKHEF

Using the Grid

• Request authorization: [email protected]• Look what is out there using grid-info-search or

http://marianne.in2p3.fr/datagrid/giis/giis-browse.html

• Try some local hosts:– bilbo, kilogram, triangel

kilogram:davidg:1009$ globus-job-run dommel.wins.uva.nl /usr/ucb/quota -vDisk quotas for random (uid 12xxx):Filesystem usage quota limit timeleft files quota limit timeleft/home/random 13067 1500000 2000000 0 0 0kilogram:davidg:1010$

• Start running your analysis/MC/other jobs

Page 20: Using Grid Computing at NIKHEF

GridFTP

• Universal high-performance file transfer• Extends the FTP protocol with:

– Single sign-on (GSI, GSSAPI, RFC2228)– Parallel streams for speed-up– Striped access (ftp from multiple sites to be faster)

• Clients: gsincftp, globus-url-copy.

Page 21: Using Grid Computing at NIKHEF

What’s Next?

• Some of the nice user-features to come:

– Finding data files by characteristics(give me all golden decay’s)

– Moving your job to where the data is– Automatic partitioning of jobs– Support true-interactive work– Better network utilisation (faster access to data)– ………

• If you are in the DataGrid project, ask your WP leader for authorization in TB1