grid computing test beds in europe and the netherlands david groep, nikhef 2001-10-01
TRANSCRIPT
![Page 1: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/1.jpg)
Grid Computing Test bedsin Europe and the Netherlands
David Groep, NIKHEF2001-10-01
![Page 2: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/2.jpg)
Resource access: then & now
• In ye olde days (till approx 1992):– Hardly any network security– All machines in a local net were created equal– All local users happy with remote shell, rlogin, rcp
• Now:– Want to communicate globally over Gigabit WAN, but– Internet is full of firewalls and barriers,
to retain local control and keep crackers out…
• The Grid:– Bring back single sign on and trust– use the WAN as the 80’s LANs: all global users happy
![Page 3: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/3.jpg)
What is Grid computing?
• Dependable, consistent and pervasive access to (high-end) resources
• Combine resources from various organizations• User-based view: `Virtual Organizations’
• Transparent decisions for the user
• Security is of paramount importance:– Authentication, Authorization, Accounting&Quota
![Page 4: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/4.jpg)
The One-Liner
• Resource sharing and coordinated problem solving in dynamic multi-institutional virtual organisations
![Page 5: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/5.jpg)
‘Virtual Organizations’
![Page 6: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/6.jpg)
Grid Architecture
Applications
Grid Services GRAMGSI
Grid FabricCondor MPI PBS Internet Linux
Application ToolkitsDUROC MPICH-G2Condor-G
GridFTPMDS
SUN
VLAM-G
Make all resources talk standard protocols
Promote interoperability of application toolkit, similar to interoperability of networks by Internet standards
![Page 7: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/7.jpg)
• Globus Project started 1997• de facto-standard• Reference implementation of Gridforum standards
• Large community effort• Basis of several projects, including EU-DataGrid
• Toolkit `bag-of-services' approach
• Successful test beds, with single sign-on, etc…
Grid Middleware
![Page 8: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/8.jpg)
Single Sign-on
• Based on PKI (Public Key Infrastructure), using X.509 `certificates’:– Grid Credential Name (~login name for the Grid)– Private key, for signing and encryption– Certificate signed by trusted third parties (CA’s)
(see http://certificate.nikhef.nl/)
• Co-operates with local security policies
• Exchange certificates, authenticate, delegate• Limit vulnerability by using limited `proxies’
![Page 9: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/9.jpg)
Looking at Resources
• Per Virtual Organization (or test bed)
• Directory of Resources and their Characteristics
• Used to find `best resource out there’
DataGrid httap://marianne.in2p3.fr/
DutchGrid?ldap://giishost.nikhef.nl:30001/
![Page 10: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/10.jpg)
Submitting a Job
![Page 11: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/11.jpg)
Sending your Data
• Tape robots, disks, etc. share GridFTP interface• Optimize for high-speed >1Gbit/s networks
• In the future: automatic optimizations, bandwidth reservations, directory-enabled networking, …
![Page 12: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/12.jpg)
EU DataGrid Work Packages
Applications
Grid Services GRAM
Grid FabricCondor PBS Internet Linux
Application Toolkits MPICH-G2Condor-G
GridFTPMDS
SUN
WP8-10
WP1& 8-10 sw
WP2,3,5,(7)
WP4,7
![Page 13: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/13.jpg)
EU DataGrid Test bed 1
• DutchGrid embedded in
DataGrid Test bed 1
• DataGrid TB1:– 14 countries– 21 major sites
– Shared PKI– Mutual authorization
– Major applications:HEP, Earth Obs,Bio-informatics
![Page 14: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/14.jpg)
DutchGrid platform
Amsterdam
UtrechtKNMI
Delft
Leiden
Nijmegen
Enschede
• DutchGrid aims:– Share experience– Test bed coordination– PKI security
• Participation byNIKHEF:
FOM, VU, UvA, Utrecht, Nijmegen
KNMI, SARA
AMOLF
DAS (ASCI):TUDelft, Leiden, VU, UvA, Utrecht
Telematics Institute
![Page 15: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/15.jpg)
DutchGrid: build-up now
• Current startup-resources to be abused:– NIKHEF:
• 50x2 CPU’s D0 cluster• 2x10x2 (=40) CPU’s LHCb at NIKHEF(WCW) &VU• 10x2 CPU’s Alice NIKHEF(WCW)• ca. 4x2 CPU’s Alice Utrecht• ca. 10x2 CPU’s D0 Nijmegen• Lots of disk & dedicated 1.3TByte cache server
– DAS-II: 200 dual-PIII’s systems & some disk (~2TByte)• Spread over 5 locations (NIKHEF is one!)
– SARA: tape robot (>200TByte), some clusters
![Page 16: Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF 2001-10-01](https://reader035.vdocuments.mx/reader035/viewer/2022062417/551a71c2550346761a8b4995/html5/thumbnails/16.jpg)
Present and Future
• Plenty more systems (~ 5000 CPU’s in 2003/4??)• Mass storage (@SARA)• Network capacity to CERN and Fermi (SURFnet)
• Buildup of software infrastructure• Support for changing Analysis/MC software• Bringing the Grid to the (interactive) Desktop