1 stellenbosch, africa 5 2011 - epikh workshop, 10.06.2011 the epikh project (exchange programme to...
TRANSCRIPT
1Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011www.epikh.eu
The EPIKH Project(Exchange Programme to advance e-Infrastructure Know-How)
GRID Science Gateway
Riccardo Rotondo ([email protected])
National Institute of Nuclear Physics
Africa 5 2011 – EPIKH Workshop
Stellenbosch, South Africa 10.06.2011
22Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
Outline
• Computationally intensive research• Grid access technologies• Science Gateway• The e-Collaboration environment
– The old paradigm– Science Gateway architecture– The new “brick” approach
• Use Cases:– The CHAIN Science Gateway– The SPECIAL Science Gateway– The INDICATE e-Culture Science Gateway
• Conclusions
33Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
Computationally intensive research
44Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
Several scientific domain involved
66Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
But…using Grids is not straightforward
Type = "Job";Type = "Job";
JobType = "JobType = "MPICHMPICH";";
MPIType = "MPIType = "MVAPICH2_PGI706MVAPICH2_PGI706";";
CpuNumber = 16;CpuNumber = 16;
MPIGranularity = 4;MPIGranularity = 4;
Executable = "flash2";Executable = "flash2";
StdOutput = "mpi.out";StdOutput = "mpi.out";
StdError = "mpi.err";StdError = "mpi.err";
InputSandbox = InputSandbox = {"{"watchdog.shwatchdog.sh","","mpi.pre.shmpi.pre.sh","","mpi.post.shmpi.post.sh","flash.par","summers_den","flash.par","summers_den_1e0.rates","flash2"};_1e0.rates","flash2"};
OutputSandbox = OutputSandbox = {"mpi.err","mpi.out","watchdog.out","flash_bubble.log","amr_log"};{"mpi.err","mpi.out","watchdog.out","flash_bubble.log","amr_log"};
Requirements = (other.GlueCEUniqueId == Requirements = (other.GlueCEUniqueId == "unipa-ce-01.pa.pi2s2.it:2119/jobmanager-lcglsf-hpc");"unipa-ce-01.pa.pi2s2.it:2119/jobmanager-lcglsf-hpc");
RetryCount = 0;RetryCount = 0;
$ voms-proxy-info --all subject : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy/CN=proxy issuer : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy identity : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy type : proxy strength : 1024 bits path : /tmp/x509up_u512 timeleft : 500:38:44 === VO gilda extension information === VO : gilda subject : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes issuer : /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it attribute : /gilda/Role=NULL/Capability=NULL attribute : /gilda/generic-users/Role=NULL/Capability=NULL attribute : /gilda/idl/Role=NULL/Capability=NULL timeleft : 23:59:33 uri : voms.ct.infn.it:15001
$ glite-wms-job-status https://gilda-lb-01.ct.infn.it:9000/1o4BVjqg2tJ4rN-XTAIGAg ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://gilda-lb-01.ct.infn.it:9000/1o4BVjqg2tJ4rN-XTAIGAg Current Status: Done (Success) Logged Reason(s): - Job terminated successfully Exit code: 0 Status Reason: Job terminated successfully Destination: grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda Submitted: Tue Jun 29 15:34:40 2010 CEST *************************************************************
CLI
ScriptsJDL
echo Staging Input Data \(Courtesy of European Space Agency\);#edg-rm --vo=gilda copyFile lfn:$1.N1 file://$PWD/$1.N1;lcg-cp --vo=gilda lfn:$1.N1 file://$PWD/$1.N1;echo Staging Application;gunzip beam20.tar.gz;tar xvf beam20.tar;cd beam-2.0/bin;echo Starting Application;./pds2jpg-ASAR-run.sh $1;mv $1-b*.jpg ../..cd ../..rm -fr beam-2.0;rm -fr $PWD/$1.N1;rm -fr $PWD/beam20.tar;echo Input ENVISAT Product courtesy of European Space Agencytouch ENVISAT_Product_courtesy_of_European_Space_Agencyecho No Output Packaging;echo Done!;
GSI
77Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
Grid Interface Evolution
• The way users access Grid resources has continuously evolved towards simplicity and transparency:
• Command Line– Globus and gLite CLI– Used by the enthusiastic and early adopter scientists
• GUI applications– gEclipse, Grid2Win– Good to expand the communities but difficult to maintain
• Web Interface– GENIUS, P-GRADE– Easier for new users but monolithic
• Science Gateways
88Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
The GENIUS Grid Portal
“robot” certificates on “e-tokens”
99Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
Science Gateways
A framework of tools that allows scientists to run applications with little concern for where the computation actually takes place. This is similar to cloud computing in which applications run as Web services on remote resources in a manner that is not visible to the end user. However, a science gateway is usually more than a collection of applications. Gateways often let users store, manage, catalogue, and share large data collections or rapidly evolving novel applications they cannot find anywhere else. Training and education are also a significant part of some Science Gateways
1010Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
Liferay (www.liferay.com)
• Highly-configurable, scalable, open source portal framework;
• Compatible with JSR 168/286 standards and based on modern web 2.0 technologies;
• Liferay services planned to be used:– Portal;– CMS & WCM;– Collaboration and “social” software
10
1111Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
Grid Access enable: Portlets as bricks
• Portlets can interact with the Grid e-Infrastructure
• Different approaches are available:– Execute the Command Line behind the portal;– Using API where available:
Must be in Java or other languages supported by Liferay;
– Call REST services from Javascript code in the browser;
• Additional layers between Liferay and the Grid can be necessary for some services;
• Each portlet can follow its own communication method.
1212Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
GRID Science Gateway Architecture
1. Log In
2. Verify ACL
4. Grid Credential Exchange
5. Grid Services
6. Results
7. Download data
3. Grid Services Request
1313Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
Robot Certificates in a nutshell
• Robot certificates have been introduced by several Certification Authorities to allow users, who are not familiar with personal certificates and do not belong to any Vos, to access and use Grids.
– They are extremely useful, for instance, to automatize grid service monitoring, data processing production, distributed data collection systems, etc.;
– Basically, these certificates can be used to identify a person responsible for an “unattended” service or process acting as client and/or server.
1414Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
Robot certificates & e-Token
• In order to strong reduce the risks to have the portal certificate compromised, the INFN CA has decided to issue these special certificates on board of Aladdin eToken PRO 32K smart cards.
• Each smart card can support several (up to 32) robot certificates: one for each application available on the grid portal.
• Our Science Gateways fully supports robot certificates !
1717Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
The CHAIN Science Gateway
1818Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
The CHAIN Science Gateway
1919Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
The CHAIN Science Gateway
2020Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
The INDICATE e-Culture Science Gateway (1/3)
20
http://www.indicate-project.eu
http://indicate-gw.consorzio-cometa.it
2121Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
The INDICATE e-Culture Science Gateway (2/3)
2222Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
The INDICATE e-Culture Science Gateway (3/3)
22
2323Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
Conclusions
• Conclusions– Using different frameworks it is possible to build a Science
Gateway able to interact with Grid Services;– Liferay is the most used framework and offers an easy,
customizable and user friendly interface;– Thanks to robot certificates and X.509 standard the level of
security is kept high and the resources safe; – The new “bricks” developed are being used in two projects
belonging from two distinct scientific domains.
• Future Work:– While the role of Liferay as “front-end” is clear, the interactions
of the tools behind are still under development;– Test the new Science Gateway in a production environment.
2424Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011Riccardo Rotondo
Questions?Thank you for your attention