1 chep 2000, 10.02.2000roberto barbera roberto barbera (*) genius: a web portal for datagrid acat...
TRANSCRIPT
1CHEP 2000, 10.02.2000 Roberto Barbera
Roberto Barbera(*)
GENIUS: a Web Portal for DataGRID
ACAT 2002, Moscow, 24-28.06.2002
(*)work in collaboration with A. Andronico, A. Falzone and A. Rodolico
2Roberto Barbera
OutlineThe Grid Vision: the concept of Collaboratory
Grid activities in Europe: the DataGrid Project
The main “actors” of the DataGrid Project
“Easy” ed “ubiquitous” access to the Grid: the GENIUS web portal
Conclusions and perspectives
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
ACAT 2002, Moscow, 24-28.06.2002
3Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
ACAT 2002, Moscow, 24-28.06.2002
4
GRID Projects – EU IST (~37 M€)An integrated approach
ScienceIndustry / business
ApplicationsApplications
MiddlewareMiddleware& Tools& Tools
Underlying Underlying InfrastructuresInfrastructures
GR
IDS
TA
RT
CROSSGRID
DATAGRID
DATATAG
GRIDLAB
EGSO
GRIA
GRIP EUROGRID
DAMIEN
iVDGL
ACAT 2002, Moscow, 24-28.06.2002
5Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
CERN – International (Switzerland/France)
CNRS - France
ESA/ESRIN – International (Italy)
INFN - Italy
NIKHEF – The Netherlands
PPARC - UK
DataGRID main contractors
ACAT 2002, Moscow, 24-28.06.2002
6Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
Research and Academic Institutes•CESNET (Czech Republic)•Commissariat à l'énergie atomique (CEA) – France•Computer and Automation Research Institute, Hungarian Academy of Sciences (MTA SZTAKI)•Consiglio Nazionale delle Ricerche (Italy)•Helsinki Institute of Physics – Finland•Institut de Fisica d'Altes Energies (IFAE) - Spain•Istituto Trentino di Cultura (IRST) – Italy•Konrad-Zuse-Zentrum für Informationstechnik Berlin - Germany•Royal Netherlands Meteorological Institute (KNMI)•Ruprecht-Karls-Universität Heidelberg - Germany•Stichting Academisch Rekencentrum Amsterdam (SARA) – Netherlands•Swedish Research Council - Sweden
Industrial Partners•Datamat (Italy)•IBM-UK (UK)•CS-SI (France)
Other DataGRID partners
ACAT 2002, Moscow, 24-28.06.2002
7
EDG overview : work packagesEDG Project is structured in 12 Work Packages: WP1: Work Load Management System WP2: Data Management WP3: Grid Monitoring / Grid Information Systems WP4: Fabric Management WP5: Mass Storage Management WP6: Testbed and demonstrators WP7: Network Monitoring WP8: High Energy Physics Applications WP9: Earth Observation WP10: Biology WP11: Dissemination WP12: Management
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
}}MiddlewareMiddleware
}}ApplicationsApplications
Roberto BarberaACAT 2002, Moscow, 24-28.06.2002
8Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
Computational biology
ACAT 2002, Moscow, 24-28.06.2002
9Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
Medical Diagnostic Imaging
ACAT 2002, Moscow, 24-28.06.2002
Earth Observation Community GRID interactive scenario
Common access to EO missions cataloguesAcquisition plan, order, delivery
Parametric data fusion and models integration
Collaborative publishing of results
On demand high level products generation
Roberto BarberaACAT 2002, Moscow, 24-28.06.2002
11
ENVISAT (launched 01.03.2002 !)• 3500 MEuro programme cost3500 MEuro programme cost
• 10 instruments on board10 instruments on board• 200 Mbps data rate to ground200 Mbps data rate to ground• 400 Tbytes data archived/year400 Tbytes data archived/year• ~100 “standard” products~100 “standard” products• 10+ dedicated facilities in Europe10+ dedicated facilities in Europe
• ~700 approved science user projects~700 approved science user projects
• 3500 MEuro programme cost3500 MEuro programme cost
• 10 instruments on board10 instruments on board• 200 Mbps data rate to ground200 Mbps data rate to ground• 400 Tbytes data archived/year400 Tbytes data archived/year• ~100 “standard” products~100 “standard” products• 10+ dedicated facilities in Europe10+ dedicated facilities in Europe
• ~700 approved science user projects~700 approved science user projects
Earth Observation
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
Roberto Barbera
12
http://www.cern.ch
~9 km
LHC
SPS
CERN
Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
High Energy Physics
ACAT 2002, Moscow, 24-28.06.2002
13Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
High Energy Physics
ACAT 2002, Moscow, 24-28.06.2002
14
ATLAS CMS
LHCb
Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
~6-8 PetaBytes / year~O(108) events/year
~O(103) batch and interactive users
High Energy Physics
ACAT 2002, Moscow, 24-28.06.2002
15Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
1/100 of a Pb+Pb @ LHC !
Simulation and reconstruction of a “full” (central) Pb+Pb collision at LHC (about 84000 primary tracks!) takes about 24 hours of a top-PC and produces an output bigger than 2 GB.
ACAT 2002, Moscow, 24-28.06.2002
16Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
5 times the Eiffel Tower
˜1500 m
6-8 Petabytes ˜10.000.000 CD-ROM
ACAT 2002, Moscow, 24-28.06.2002
17Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
Yerevan
CERN
Saclay
Lyon
Dubna
Capetown, ZA
Birmingham
Cagliari
NIKHEF
GSI
Catania
BolognaTorino
Padova
IRB
Kolkata, India
OSU/OSCLBL/NERSC
Merida
Bari
Nantes
ALICE “grid” sites
ACAT 2002, Moscow, 24-28.06.2002
18Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
Europe: 267 institutes, 4603 usersElsewhere: 208 institutes, 1632 users
The LHC “web” in the world
ACAT 2002, Moscow, 24-28.06.2002
19Roberto BarberaACAT 2002, Moscow, 24-28.06.2002
Job Submission work-flow
UIJDL
Logging &Logging &Book-keepingBook-keeping
ResourceResourceBrokerBroker
Job SubmissionJob SubmissionServiceService
StorageStorageElementElement
ComputeComputeElementElement
Information Information ServiceService
Job Status
ReplicaReplicaCatalogueCatalogue
DataSets info
Author.&Authen.
Job S
ub
mit
Even
t
Job
Qu
ery
Job
Stat
us
Input “sandbox”
Input “sandbox” + Broker InfoGlobus RSL
Output “sandbox”
Output “sandbox”
Job Status
Pu
blis
h
grid
-pro
xy-in
it
Exp
and
ed J
DL
SE & CE info
Roberto BarberaACAT 2002, Moscow, 24-28.06.2002
21Roberto Barbera
EDG m/w has been released but…EDG software (Globus, UI, JDL, WP2, WP3, etc.) contains tens of commands/switches which also have their own logical sequences (“B” after “A”, “C” before “D” and so on).
Browsing Grid VO “directories” (users, RC’s, DB’s, etc.) requires LDAP “speaking” and tomorrow could require SQL “speaking”.
“User gridification” is a tough task for a “rookie” this does not fit with the claim that “grids” are for everybody and that grid computing will be as easy as surfing the Internet ?
Furthermore, all this holds for DataGrid. What will happen when other grids’ software (especially UI’s) will come up (PPDG, iVDGL, etc.) ? Will users have to learn tens of “grid dialects” ?
Today “grid computing” is a rather complicated experience only possible at selected machines (UI’s) this does not fit with the claim that one could do “grid computing” even from a PDA ?
Is there any way to set-up a “user-friendly” grid ?
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
ACAT 2002, Moscow, 24-28.06.2002
22Roberto Barbera
A web portal: why and how ?It can be accessed from everywhere and by “everything” (desktop, laptop, PDA, WAP phone).It can keep the same user interface to several back-ends (grid “dialects” command-line UI’s).It must be redundantly “secure” at all levels: 1) secure for web transactions, 2) secure for user credentials, 3) secure for user authentication, 4) secure at VO level.All available grid services must be incorporated in a logic way, just “one mouse click away”.Its layout must be easily understandable and user friendly.
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
ACAT 2002, Moscow, 24-28.06.2002
23Roberto Barbera
GENIUS®
(Grid Enabled web eNvironment for
site Independent User job Submission)
[https://genius.ct.infn.it]
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
OS & Net services
Basic Services
High level GRID middleware
ALICE ATLAS CMS LHCbApplications’specific layer
Other apps
GLOBUS
toolkit
DataGRID architectur
e
GENIUS web portal
ACAT 2002, Moscow, 24-28.06.2002
24Roberto Barbera
GENIUS: how it works
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
Apache
EnginFrame
GENIUShttps+java/xml+rfb
WEB Browser
EDGUI
Local
WS
the GridEDG+GSI
3-tier model
25Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: the main page
26Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: the authentication
27Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: file services
28Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: the authorization
29Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: security services
30Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: job submission
31Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: job submission
32Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: job queue
33Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: job output
34Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: personal spooler
35Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: job data
36Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: interactive analysis
37Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: the VO services
38Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: the VO services
39Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: monitoring services
40Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: monitoring services
41Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: monitoring services
42Roberto Barbera
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
GENIUS show: monitoring services
43Roberto Barbera
Present status and perspectivesCurrent implementation of GENIUS already includes: secure web transactions, user authentication and
authorization; remote interaction with the user’s file system; interfaces for job submission/control, to VO servers
(users’ and catalogues), and to monitoring systems; persistent (user’s) book-keeping and spooler system; interactive analysis ! rpm available !
Todo: multi-jobs (parallel and sequential); interface to data management and other grid services; more application-specific customizations; web-guided creation of a work flow system.
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
44Roberto Barbera
GENIUS vs. other grid portalsIt is not a toolkit. It is a complete production-ready environment which combine the concepts of “user portal” and “science portal”.No client software needs to be installed apart from the web browser. GENIUS can be accessed from everywhere.No security delegation (“à la MyProxy”) is needed. Access passwords are securely “streamed” only when needed.Interactive analysis (via VNC) and web access to personal spooling areas are possible.User file system is not limited to input and output files.EnginFrame modularity makes different customizations easy to implement. Already available for EDG m/w, GLOBUS, and LSF. Under definition for CONDOR (hungarian grids). It is compatible with the Tomcat open source java servlet container available from SUN.
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
45Roberto Barbera
ConclusionsComputational grids could represent the “natural” environment for next generation high energy physics experiments, computational bio-medecine, Earth observation and many other inter-disciplinary applications. “Grid” could be the Internet “new age” where users can seamlessly and ubiquitously access not only information but also huge computing resources and mass storage systems distributed worldwide with their own applications.However, in order to turn dreams into reality, grid access must be easy and intuitive especially for the vast majority of non-expert users and this is the mandate of the GENIUS team. Live demos of GENIUS at work could be made for interested people during breaks in the e-mail reading area.
Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy
ALICE Collaboration
ACAT 2002, Moscow, 24-28.06.2002