globus toolkit massimo sgaravatto infn padova. massimo sgaravatto introduction grid services: lhc...
TRANSCRIPT
![Page 1: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/1.jpg)
Globus Toolkit
Massimo SgaravattoINFN Padova
![Page 2: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/2.jpg)
Massimo Sgaravatto
Introduction Grid Services: LHC regional centres
need distributed computing Analyze toolkits that integrate
distributed computational resources: Globus Legion Globe ….
![Page 3: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/3.jpg)
Massimo Sgaravatto
Globus Basic research on GRID problems Globus Toolkit
“Bag of services” for developing GRID tools and applications
Available for most Unix platforms Limited functionalities for Win32
![Page 4: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/4.jpg)
Massimo Sgaravatto
Globus ArchitectureApplications
Core ServicesMetacomputing
Directory Service
GRAMGlobus
Security Interface
Heartbeat Monitor
Nexus
Gloperf
Local ServicesLSF
Condor MPI
NQEEasy
TCP
SolarisIrixAIX
UDP
High-level Services and ToolsDUROC globusrunMPI Nimrod/GMPI-IO CC++
GlobusView Testbed Status
GASS
![Page 5: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/5.jpg)
Massimo Sgaravatto
Core Services Security (GSI) Information Service (MDS) Resource Management (GRAM) Remote Data Access (GASS) Communication (Nexus) Network Performance Monitor (GLOPERF) Fault detection (HBM) Executable Management (GEM) (???)
![Page 6: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/6.jpg)
Massimo Sgaravatto
Globus Security Infrastructure Main focus: authentication
single authentication not necessary to manage authentication
on each GRID node Mapping between GRID user-id and
local user-id Digital signature mechanisms
![Page 7: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/7.jpg)
Massimo Sgaravatto
Metacomputing Directory Service
Static and dynamic information about GRID components (host architecture, memory, load, network bandwidth, latency. ...)
Possibility to define new attributes Information updated by:
Globus services (i.e. GRAM Reporter) Users/Applications Other “providers” (i.e. SNMP)
Tools and API for discovering, publishing, and accessing information
Standard protocol and interface: LDAP Single MDS server (located in USA)
![Page 8: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/8.jpg)
Massimo Sgaravatto
Resource Management Management of computational resources GRAM (Globus Resource Allocation Manager)
responsible for a set of “local” resources Single interface to different resource managers
(Condor, LSF, NQE, fork, ...) Resource Specification Language (RSL) for
defining: resource requirements (machine type, memory, ...) job definition (executable, parameters, ...) I.e.: & (ResourceManagementContact = “cond.pd.infn.it:/C=US/.../CN=cond.pd.infn.it- condor”) (count=100) (executable=myapp)
![Page 9: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/9.jpg)
Massimo Sgaravatto
GRAM
CONDOR
Globus Resource Manager Architecture
Application
BrokerMDS
GRAM
LSF
GRAM
fork
Resource andJob specification
Resourcediscovery
Create jobs(Ground RSL expressions)
Site1Site2 Site3
(High level RSL expression)
RSL specialization
![Page 10: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/10.jpg)
Massimo Sgaravatto
Remote Data Access GASS (Global Access to Secondary Storage) Supported servers: GASS, HTTP, FTP Uniform naming using URL:
I.e.: x-gass://host1.pd.infn.it:1234/filename Possibility to access remote data sets (distributed
file system) Necessary to modify source code:
open() globus_gass_open() close() globus_gass_close()
Caching Possibility to move files from/to remote data
storage systems
![Page 11: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/11.jpg)
Massimo Sgaravatto
Other core services Nexus
Unicast and multicast low level communication library
Supports different programming models (message passing, RPC, etc…)
HBM (Heartbeat Monitor) Monitors the status of registered processes
GLOPERF (Globus Network Performance Measurement Tool)
Measures throughput and latency between node pairs
![Page 12: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/12.jpg)
Massimo Sgaravatto
Evaluation of the Globus Toolkit
5 sites Testbed (Bologna, CNAF, LNL, Padova, Roma1)
Use case: HLT CMS studies MC Prod. Complete HLT chain
Services Resource management
fork() Interface to different local resource managers (Condor, LSF)
Resources chosen by hand Broker to implement a Global resource manager
Data Mover to stage executable and input files to retrieve output files
![Page 13: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/13.jpg)
Massimo Sgaravatto
Use Case: CMS HLT studies
![Page 14: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/14.jpg)
Massimo Sgaravatto
Status Globus installed on 5 Linux PCs in 3
sites GSI
Seems working MDS
Lots of problems, maybe temporary? GRAM, GASS, Gloperf, HBM
Work in progress
![Page 15: Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze](https://reader036.vdocuments.mx/reader036/viewer/2022082712/56649f325503460f94c4e0f5/html5/thumbnails/15.jpg)
Massimo Sgaravatto
Other problems Documentation not well organized Collaboration with HPC Lab,
University of Lecce is very useful !