wp3 progress report
DESCRIPTION
WP3 progress report. SEE-GRID-2 PSC05 Meeting, Thessaloniki, Greece 11-12 September 2007. Antun Balaz WP3 Leader Institute of Physics, Belgrade [email protected]. Overview. WP3 objectives and activities WP3 position in SEE-GRID-2 WP3 deliverables and milestones WP3 schedule - PowerPoint PPT PresentationTRANSCRIPT
The SEE-GRID-2 initiative is co-funded by the European Commission under the FP6 Research Infrastructures contract no. 031775
www.see-grid.eu
SEE-GRID-2
WP3 progress report
Antun BalazWP3 Leader
Institute of Physics, [email protected]
SEE-GRID-2 PSC05 Meeting, Thessaloniki, Greece11-12 September 2007
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 2
Overview
WP3 objectives and activitiesWP3 position in SEE-GRID-2WP3 deliverables and milestonesWP3 scheduleWP3 activities reportsWP3 country reportsWP3 Action points and issues
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 3
WP3 objectives & activities (1)
Develop the next-generation SEE-GRID infrastructure Next generation of EGEE middleware (gLite), the VOMS, the WMS,
information services and file catalogue services will be assessed having in mind project and WP3 objectives
SEE-GRID infrastructure deployment regarding the middleware services will follow and adapt its services according to the results of the assessment.
Support in deployment and operations of the Resource Centres Next generation monitoring services will be deployed so as to support the
over-the-board infrastructure monitoring. The current SEE-GRID helpdesk will be expanded in SEE-GRID-2, with the
main goal of full EGEE interoperability. Support the expansion and deal with the overall upgrade of the current
infrastructure by proliferation of RCs in each SEE country increasing: the total available regional resources (CPUs, storage, etc.) thus boosting the
capacity and reliability of the provision of Grid services at regional level, and the diversity and distribution of participating teams per country thus
strengthening cooperation and collaboration at national level.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 4
WP3 objectives & activities (2)
Network resource provision and assurance WP3 will also deal with network resource provision, in close cooperation
with the SEEREN2 project, thus ensuring stable connectivity for the RCs in the region.
Attention will be paid to Bandwidth-on-Demand requirements, to cater for bandwidth-intensive applications in case they need dedicated resources for particular experiments.
CA and RA guidelines and deployment Regional SEE-GRID catch-all Certification Authority (CA) will continue to
operate providing certificates for countries without a CA. Experienced CA team will provide support for per-country CA deployment
and accreditation. The cycle to establish a National Grid CA will be in compliance with the procedures and accreditation process of the EU Grid Policy Management Authority (EUGridPMA).
Operations will be strengthened so as to support per-country CA operations
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 5
WP3 objectives & activities (3)
User portal deployment and operations A user-friendly multi-Grid access portal will be deployed, enabling
universal and more flexible user access to the regional infrastructure.
The work on the SEE-GRID-2 portal should increase the user-friendliness of being able to select a grid and execute a workflow on the selected grid, so that interoperability of different grids is seamlessly and transparently solved at the application (workflow) level.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 6
WP3 objectives & activities (4)
A3.1 - Implementation of the advanced SEE-GRID infrastructure (UOB-IPB/IPP) Deals with support for configuration, deployment and operations
of the Resource Centres within the SEE-GRID pilot infrastructure, as well as transition of mature centres into EGEE.
Effort: 89 PMs Subactivities:
A3.1.1 - Expand the existing SEE-GRID topology by inclusion of new sites per SEE country
A3.1.2 - Deploy M/W components and OS in SEE Resource Centers A3.1.3 - Test the site installations in local and Grid mode A3.1.4 - Operate the SEE-GRID infrastructure A3.1.5 - Monitor the infrastructure performance and assess its usage A3.1.6 - Certify and Migrate SEE-GRID sites from Regional Pilot to
Global production-level eInfrastructures
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 7
WP3 objectives & activities (5)
A3.2 - Network Resource Provision and BoD requirements (IPP) Support liaison actions to ensure adequate network provision,
including the requirements for Bandwidth-on-demand, if and where necessary depending on the application.
Effort: 39 PMs
A3.3 - Deploy and operate Grid CAs (GRNET) Should provide CA and RA guidelines and help establish per-
country CAs to cover the authentication issues Effort: 73 PMs
A3.4 - Provide a user portal (SZTAKI) Supports the deployment of a user-friendly and multi-grid
interoperable portal for convenient Grid access and usage. Effort: 15 PMs
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 8
WP3 position in SEE-GRID-2 (1)
WP4
Users & Applications
WP3
Infrastructure & Operations
WP2
Strategies & Policies
A4.1: Select multi-disciplinary applications
A4.2: Adopt applications for the SEE user communities
A4.3: Support deployed applications
WP5
Training, Dissemination and Communication
A2.1: Study grid deployment solutions
A2.2: Deliver sustainable roadmap for SEE NGIs
A3.1: Implementation (deploy, test, operate, monitor, certify, migrate) of an advanced SEE Grid Infrastructure
A3.4: Grid access Portal
A3.2: Network resource provision
A3.3: Deployment and Operational support for accredited Grid Certification Authorities
A4.4: Assess application usage
WP1
Project Administrative and Technical Management
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 9
WP3 position in SEE-GRID-2 (2)
Results of WP2 will be used as inputs D2.1 - Regional and National Organisational and Policy Schemes D2.2 - Sustainable organizational and operational approach D2.3(a,b) - Sustainability and Impact Analysis of SEE National
Grid Initiatives
Results of WP3 used as input to WP4All partners participate in WP3Activities start in M1, end in M24WP3 planned budget is 672,512.00 €, or ~33.6% of the SEE-GRID-2 budgetWP3 is planned to take 216 PMs, or ~34.2% of all SEE-GRID-2 PMs
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 10
WP3 Deliverables & Milestones (1)
D3.1a - Infrastructure Deployment Plan, M04 (CERN) Describes the envisaged infrastructure deployment execution plan
to be followed in the region. Prepared and submitted on time
D3.2 - CA and RA guidelines for new candidates, M05 (GRNET/AUTH) This deliverable describes the guidelines and best practices of the
per-country CA and RA organization and policies. Prepared and submitted on time
D3.3 - Portal specifications and functionality, M06 (SZTAKI) This deliverable provides the characteristics and structure of the
multi-grid user-oriented portal. Prepared and submitted on time
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 11
WP3 Deliverables & Milestones (2)
D3.1b - Infrastructure Deployment Plan, M14 (CERN) Final version of D3.1. Prepared and submitted on time
We are currently in M17Future: D3.4 - Infrastructure overview and assessment, M23 (UOB-IPB) This deliverable presents an overview and assessment of the
progress in the regional infrastructure and operations in the life of the project
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 12
WP3 Deliverables & Milestones (3)
M3.1 - Infrastructure Deployment Plan Defined M04, Status: OK
M3.2 - CA and RA guidelines for new candidates defined M05, Status: OK
M3.3 - Portal operational across the pilot Grid M12, Status: OK
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 13
WP3 schedule
M01 - Start of WP3M04 - Infrastructure Deployment Plan (M3.1, D3.1a)M05 - CA and RA guidelines for new candidates (M3.2, D3.2)M06 - Portal specifications and functionality (D3.3)M12 - Portal operational across the pilot Grid (M3.3)M14 - Final Infrastructure Deployment Plan (D3.1b)M17 – This is where we are currentlyM23 - Infrastructure overview and assessment (D3.4)M24 - End of WP3 (and of the project)
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 14
WP3 activities overview
A3.1: Implementation of the advanced SEE-GRID infrastructure (UOB-IPB/IPP)A3.2: Network Resource Provision and BoD requirements (IPP)A3.3: Deploy and operate Grid CAs (GRNET)A3.4: Provide a user portal (SZTAKI)
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 15
A3.1: Overview
Infrastructure status gLite deployment status SEEGRID VO metrics and accounting
Operations SLA conformance monitoring per site Helpdesk tickets procedures and statistics analysis GOOD shifts MPI support on SEE-GRID sites
Operational & monitoring tools deployment & integration HGSM SAM (+ porting to MySQL) WiatG R-GMA Pakiti
SEE-GRID Wiki statusWP3 developmentsInfrastructure, Site and VO metrics
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 16
Infrastructure status (1)
Concerning the middleware deployments, the current SEE-GRID infrastructure supports a set of core services which provide user access to resources: Catch-all Certification Authority for the region has been officially
accredited by the EU Grid Policy Management Authority - EUGridPMA, and is currently operational thus enabling regional sites to obtain user and host certificates
Virtual Organisation Management Service (VOMS), server has been installed as an authorization system for the SEE-GRID Virtual Organisation (VO), which provides information on the user's relationship with the Virtual Organization, his/her groups, roles and capabilities
Workload management service (lcg-RB and glite-WMSLB) and Information Services (BDII) nodes (several instances) have been installed at partners’ sites and are operational
MyProxy is operational, and supports certificate renewal FTS deployed and used in production
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 17
Infrastructure status (2)
SEE-GRID total and free CPUs in the last year
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 18
Infrastructure status (3)
SEE-GRID infrastructure contains currently the following resources: 30 sites in SEE-GRID production 6 sites in certification phase (2 AL + 1 HR + 2 RO + 1 MD) CPUs: 1105 total, but unknown number available to SEEGRID VO;
increase of approx. 400 CPUs compared to PSC04 Storage: 17.6 TB (no increase)
All sites on gLite-3, with 3 sites on gLite-3.1 and the rest on gLite-3.0glite-CE final assessment by EGEE is that this service is not stable enough for production; we agreeglite-WMSLB actively usedGuides provided for deployment of gLite-3.1 WNs on SL4.5 (32-bit); in preparation guide for 64-bit WNs
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 19
Infrastructure (4): VO membership
steady growthstart of project ~90 members
end of 02/07 ~110 members
end of 08/07 ~160 members
03/07 04/07 05/07 06/07 07/07 08/07
100
110
120
130
140
150
160
170
122
127
138142
150
161
Month
VO
mem
bers
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 20
Infrastructure (5):VO members per country
AL BA BG CH GR HR HU MD ME MK RO RS TR0
5
10
15
20
25
30
3
11
23
2
4
6
29
2
4
7
27 27
16
11 11
21
5
23
0
2
4
23
13 1302/07
08/07
Country
VO
mem
bers
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 21
Operations (1)
SLA conformance monitored per site; tools used: HGSM SAM GStat WiatG Helpdesk
SLA conformance analysis: SEEGRID2-WP3-RS-018-SLA-Q5-2007-08-06.xls
Helpdesk tickets procedures GOOD shifts introduced, initial results positive Tickets handling: response times need to be improved! Problems with GOOD shifts – some partners not performing
duty!Helpdesk statistics analysis: SEEGRID2-WP3-RO-008-PSC05-Helpdesk_Statistics-2007-09-10.xls SEEGRID2-WP3-RS-020-PSC05-Tickets_not_closed-a-2007-09-11.xls
OPS role implemented in VOMS, documented, implemented & used
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 22
Operations (2)
MPI support on SEE-GRID sites Important for many applications Current support not sufficient Seems there is a problem with setup on some sites – GOOD shifts
are addressing this, but this is not sufficient
Proposal: WG to be created which should define minimal standards for MPI support, and provide template scripts and JDL files for submission of MPI jobsWG can be composed of representatives from sites supporting MPI and having experience: Bulgaria Turkey Greece Serbia
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 23
Operational & monitoring tools
Operational & monitoring tools deployment status HGSM – Turkey SAM (+ porting to MySQL) – Bosnia and Herzegovina with
CERN support BBmSAM (Bosnia and Herzegovina) WiatG R-GMA – Bulgaria Pakiti
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 24
HGSM is a central database that holds all crucial information about a grid siteIt has an interactive interface for users to see and update the information available in the databaseIt has exports to interface with other services to enable integration between HGSM and other services that make use of HGSM's informationHGSM is constantly improved with new features and new exports to enhance its functionality, role and information exchange over grid services for administrative purposes
HGSM (1): Background
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 25
Started in May-2007 with appearance of requests for new functionality and enhancements due to inefficiency of HGSM in some areas in the GIM list.Since the requests was big and required some major changes in HGSM structure, an “Internals Document” is prepared and passed to people in the discussion in the end of May.During June, the requests are anaylzed and refined down so everybody was aware of the missing parts, possible solutions and impacts of the new functionality
HGSM (2): Latest Development Period (Initiation and Planning)
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 26
In the end of June a detailed “Roadmap Document” -which covers the problems, solutions and a step-by-step action plan in every detail- is prepared and opened for discussion in the GIM list.The Document is revised and updated according to feedback taken by the list. Document yielded 4 revisions, three drafts and a final which is published on Jul 12, 2007.Active development started on Jul, 16 according to plan announced in the Document.
HGSM (3): Latest Development Period (Finalizing and Taking Action)
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 27
Following improvements are qualified for this development period: Revision of visual interface fields. Introduction universal exporting subsystem which can be used by
humans and other computers. Introduction of an importing subsystem which can import exported
data for administrators. An automatic field filling tool for administrators. Functionality improvement on various pages of visual interface. Improving the handling of supported applications in HGSM and
developing a MAUI configuration editing tool for reflecting changes to grid sites directly and automatically.
Enabling HGSM to track site information history for statistical purposes.
(and Progress So Far)
(100%)
(80%)
(0%, 10 Sep)(0%, 24 Sep)
(0%, 8 Nov)
(0%, 22 Nov)
(0%, 3 Dec)
Note: Dates indicate rough estimations of kick-off dates.
HGSM (4): The Improvements
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 28
SAM/BBmSAM
SAM Server Portal BBmSAM
Service availability and SLA calculations implemented MAINTENANCE status implemented in a better way Enhanced OVERVIEW (main) page of BBmSAM
– Showing uptime for last 24h– Filters available for: country, tier, certified status, last test state
BBmobileSAM Now also showing uptime percentage for last 24h
HGSM integration Preparation for HGSM shift to new version
Database Now running strictly off MySQL, no Oracle used Reorganization of indexes – improved performance
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 29
WiatG: Introduction
Web application for visualization of BDII information http://bdii.phy.bg.ac.yu/WiatG/pl/WiatG.pl
Used as an operational tool for site monitoring Highly responsive tool because it uses AJAX Partial refresh (client receives part by part of the page) Asynchronous (server is processing in the background, so one
may send several requests)
Current version seeks for: CE, gCE, RB, gRB, SE, LFC, FTS and GridICEDocumentation available: http://wiki.egee-see.org/index.php/WiatG
SEE-GRID-2 PSC05 meeting, Thessalonica, Greece - September 11-12, 2007
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 30
WiatG: Who is using it
Several regional projects EUMedGRID (bdii.isabella.grnet.gr) EUChinaGrid (euchina-bdii-1.cnaf.infn.it) EELA (lnx112.eela.if.ufrj.br) BalticGrid (bdii.mif.vu.lt) Int-EU-Grid (i2g-ii01.lip.pt) Health-e-Child (hec-maat-server2.cern.ch)
http://hec-maat-server1.cern.ch/WiatG/pl/WiatG.pl
ROC CERN PROD (lcg-bdii.cern.ch) PPS (pps-bdii.cern.ch) OPS (sam-bdii.cern.ch)
SEE-GRID-2 PSC05 meeting, Thessalonica, Greece - September 11-12, 2007
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 31
WiatG: Technologies
Following technologies are included in WiatG: Perl is used for LDAP connection to BDII and generation of
HTML and XML data. XML, the format for sending data from the web server to the
client. Cascading Style Sheets (CSS), a markup language used to
define the presentation style of a page. JavaScript, a scripting language. XMLHttpRequest, an object that is used to exchange data
between web client and web server. Document Object Model (DOM), which provides a logical view
of a web page as a tree structure.
SEE-GRID-2 PSC05 meeting, Thessalonica, Greece - September 11-12, 2007
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 32
WiatG: Architecture
SEE-GRID-2 PSC05 meeting, Thessalonica, Greece - September 11-12, 2007
Browser Client WiatG Server Side
BDII
LDAPUser Interface
XMLHTTPRequest ApacheHTTPServer
XMLHTTPRequestcallback()
LDAPtoXML script
Java ScriptCall
HTTP Request
Query Response
XML Data
HTML & CSSData
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 33
WiatG in action
SEE-GRID-2 PSC05 meeting, Thessalonica, Greece - September 11-12, 2007
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 34
WiatG: Further development(short term)
Addition of new services (MyProxy, localLFC, VO software tags, …)Development of the new tool “What should be at the Grid” (WsbatG) Based on the site configuration exported from HGSM/GOCDB Visually identical tool, providing the expected status of BDII in
WiatG
SEE-GRID-2 PSC05 meeting, Thessalonica, Greece - September 11-12, 2007
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 35
WiatG: Further development(long term)
SEE-GRID-2 PSC05 meeting, Thessalonica, Greece - September 11-12, 2007SEE-GRID-2 PSC05 meeting, Thessalonica, Greece - September 11-12, 2007
Alarms Dashboard
BDII web service
sBDII web service
Check correctness of
sBDII data
Check correctness of
sBDII data
Check correctness of BDII data
Check correctness of BDII data
Check equality of sBDII-BDII
information
Check equality of sBDII-BDII
information
SAM
HGSM/GOCDBweb service
Check equality of BDII-HGSM/GOCDB
information
Check equality of BDII-HGSM/GOCDB
information
Check correctness of HGSM/GOCDB
data
Check correctness of HGSM/GOCDB
data
WiatGSWiatGS
User InterfaceUser Interface
WiatGSWiatGS
User InterfaceUser InterfaceWiatGWiatG
User InterfaceUser Interface
WiatGWiatG
User InterfaceUser InterfaceWsbatGWsbatG
User InterfaceUser Interface
WsbatGWsbatG
User InterfaceUser Interface
Alarms Dashboard UI
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 36
R-GMA (1)
Accounting views for SEEGRID-only sites per site accounting:
●https://gserv1.ipp.acad.bg:8443/Accounting
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 37
R-GMA
Accounting views for SEEGRID and EGEE sites that support SEEGRID – per country/institution user accounting
https://gserv1.ipp.acad.bg:8443/Accounting-2
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 38
R-GMA (2)
Accounting views for Job success rates and other statistics –in progress, currently running on our own data only. https://gserv1.ipp.acad.bg:8443/Jmon
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 39
R-GMA (3)
Accounting views for Job success rates and other statistics –in progress, currently running on our own data only. https://gserv1.ipp.acad.bg:8443/Jmon
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 40
Pakiti: Overview
Pakiti Client Installed on all nodes Checks software versions against configured repositories Sends report once per day to pakiti server
Pakiti Server Running at the Aristotle University of Thessaloniki Main Components:
Feed– Daily reports from clients
Site Administrator’s front-end– Detailed view of the rpm package status at each node– Access is permitted only to each the administrator’s of each site via TLS Authentication
using X.509v3 Certificates
Addon Components ROC Manager’s front-end
– Aggregated view of the status of all the sites in the ROC– Developed by the AUTH GOC
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 41
Pakiti: Status of the Service
Pakiti enabled sites in SEE-GRID ROC:Bosnia Herzegovina BA-04-PMFSA
Bulgaria BG01-IPP BG02-IM BG04-ACAD BG05-SUGRID
Croatia HR-01-RBI
Greece HG-01-GRNET HG-03-AUTH
Romania RO-01-ICI
Serbia AEGIS01-PHY-SCL AEGIS02-RCUB AEGIS03-ELEF-LEDA AEGIS04-KG AEGIS05-ETFBG
Turkey TR-01-ULAKBIM TR-05-BOUN
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 42
Pakiti and reporting
The deployment of pakiti on sites is voluntarySites deploying it provide accurate information on updates status on their nodesProposal is that, in order to further improve status of SEE-GRID sites, all sites report in their 3M reports the following: Middleware version changes during the quarter Status of updates (not needed if pakiti is deployed) Major operational issues
Template will be provided for this; if pakiti is deployed) such report would just contain one line with gLite version changed and paragraph or two describing operational issues encountered (basically, structured version of section 2.3 of 3M reports, now submitted by each site)
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 43
SEE-GRID reorganized Wiki
Reorganized SEE-GRID Wiki is now the main Wiki http://wiki.egee-see.org/index.php/SEE-GRID_Wiki
Many documents still missing, main being: Participating in SEEGRID as a Site (AP1) Policy Documents (AP3) SEEGRID certification procedure (AP4) LFC (AP6) RGMA (AP7) My Proxy (AP8) BDII/RB (AP9) FTS (AP10) SAM (AP11) GridICE(AP12) How to Join the SEEGRID infrastructure as a user (AP14) Grid usage basics (AP16)
Effort needed by all partners; UKIM coordinating
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 44
WP3 developments
HGSMWiatGAccountingglite-yaim-seegridsoon-to-be deployed apt/yum repository
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 45
Infrastructure, Site and VO metrics
Infrastructure growth (CPUs, storage, memory?)Bandwidth growth?Site availabilities and downtimes (CE, SE)Accounting data (per site, per country, per application, per VO, per user community, time distribution); here SEEGRID VO and national VOs should be consideredJob success rates (per site, per country, per application, per VO, per user community, time distribution); here SEEGRID VO and national VOs should be consideredVO membership time evolution, distribution per country, per application, per user community; here SEEGRID VO and national VOs should be considered
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 46
A3.2: Status
Concerning the bandwidt-on-demand some tests were done in order to investigate the following protocols and services using our new router - CISCO 12000 XR: Resource Reservation Protocol (RSVP); Generalized Multi Protocol Label Switching (GMPLS); Differentiated Services (DiffServ); Standard activities for BoD.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 47
A3.3: Overview of the work
CA establishment in SEE Region Each country must setup each own national Certification Authorities Each CA must be accredited by the EUGridPMA see-ca-incubation mailing list ([email protected])
Support during the process of establishing a new CA and for the accreditation period
CA common procedures and best practices advices are provided Help on writing the CP/CPS documents
Process for establishing a new CA takes around one year
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 48
A3.3: Status
New accredited CAs in the Region Serbian CA (AEGIS CA)
Accreditation request on August 23, 2006 Under review by GridAUTH team and SRCE CA (on behalf of EUGridPMA) Accredited on June 1, 2007 Operational since June 10, 2007
Romanian CA (ROSA CA) Accreditation request on January 25, 2006 Under review by GridAUTH team, PK-GRID CA and CESNET CA (on behalf of
EUGridPMA) Accredited on August 1, 2007
Grid CA candidates Montenegro CA (MREN CA)
CP/CPS reviewed by GridAUTH (via see-ca-incubation mailing list) on July 10, 2007 F.Y.R.O.M. CA (MARGI CA)
Accreditation request on May 4, 2007 First CP/CPS not yet available
All candidates are encouraged to participate at the EUGridPMA meetings (Next meeting in Thessaloniki, Sept 19-21)
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 49
A3.4: Overview
Official SEE-GRID2 Portal user maintenance (52 users) Quota management, user management
Official SEE-GRID2 Portal maintenance Software (P-GRADE Portal version 2.5)
P-GRADE Portal software bug fixing
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 50
A3.4: Status and development
Portlets development has been done by turkish partner(P-GRADE Portal Development Alliance)
Already active /beta-test/ in a private portal installation.
New portlets
File Management Portletto manage the remote files through the LFC catalog and the LCG interface.
Hot topic! Intended to merge into official P-GRADE Portal v2.5.
Credential management portlet to complement the existing certificate portlet with info, change pass-phrase and destroy operations.
Intended to merge with the default certificates portlet.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 51
WP3 country reports
GreeceSwitzerland/CERNBulgariaRomaniaTurkeyHungaryAlbaniaBosnia and HerzegovinaFYR of MacedoniaSerbiaMontenegroMoldovaCroatia
Work performed since PSC04Conformance to WP3 objectivesIssues, if any
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 52
Greece
Pakiti Service enabled for SEE-GRID infrastructure https://monitor.grid.auth.gr/services/pakiti/ROC/SEE-GRID
October 2007 HG-06-EKT will support SEEGRID VO
228 CPUs, 9.3 TB Storage
Exact Resources Dedicated for SEEGRID VO to be decided.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 53
Support to WP3 operations GOOD shifts Solving operational problems
Support for SAM deployment and improvements, as well as liaison activities with SAM development teamLiaison activities with operations in other regional Grid projectsStrong involvement in operations-related developments: WiatG, WsbatG SEE-GRID apt/yum repository
Switzerland/CERN
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 54
Bulgaria (1)
Status of the infrastructure and plan for expansion 5 sites infrastructure – significantly stable with good up-time All sites upgrading now to SL4 WNs. Core services, monitoring tools:
R-GMA: graphical on-line user interface Accounting views for
– SEEGRID-only sites per site accounting– SEEGRID and EGEE sites that support SEEGRID – per country/institution user accounting– Job success rates and other statistics –in progress, currently running on our own data only.
FTS: used in production by SALUTE WMS – moved to a better hardware. BDII stability and capability improvements
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 55
Bulgaria (2)
• 5 production sites in BG
•After start of EGEE II added new cluster with 80 CPUs and low-latency Myrinet interconnect for 80 CPUs – unique resource for special MPI jobs
• BG04-ACAD (80 CPU)•BG01-IPP (12 CPU)
CPU Storage Tape
April 06 30 1 TB -
September 07
132 3.2 TB 10TB
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 56
Bulgaria (3):Petri net performance analysis
Anastas Misev from Macedonia was at IPP in a visit, sponsored by project BIS 21++, working on his Ph.D. thesis on Grid scheduling and failover. Host professor was E. AtanassovHis analysis of our RB rb001 shows that The overall success rate of the analyzed data is somewhere near 70% The percentage of the successful jobs greatly depends on the users
experience. Jobs by more experienced users have success rate above 80%.Interactive diagram helps in identifying the bottlenecks in the process model.It can show throughput time between any 2 transitionsColor coding to specify low, middle and high waiting time Conclusions: Job re-submission does not help in 99% of cases 20% of the 470 jobs submitted by one user had waiting times above 57 hours,
and all of them failed.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 57
Bulgaria (4):Job success rate – Top ten CEs
We have made a pivot analysis of the CEs and the final statuses of the jobs. The top 10 CEs are shown in the table below. Note that the percentage of successful jobs is more then 90%.
Top 10 CEs Final Status
CE name DONE ABORT CANCEL Grand Total
ce.ulakbim.gov.tr:2119/jobmanager-lcgpbs-see 754 222 976
ce002.ipp.acad.bg:2119/jobmanager-lcgpbs-see 639 66 705
ce01.ariagni.hellasgrid.gr:2119/jobmanager-pbs-see 1317 2 1319
ce01.athena.hellasgrid.gr:2119/jobmanager-pbs-see 3008 105 3113
ce01.isabella.grnet.gr:2119/jobmanager-pbs-see 768 358 1 1127
ce01.kallisto.hellasgrid.gr:2119/jobmanager-pbs-see 2188 193 2381
ce01.marie.hellasgrid.gr:2119/jobmanager-pbs-see 1296 40 1336
ce02.grid.acad.bg:2119/jobmanager-pbs-myrinet 1120 19 2 1141
ce02.grid.acad.bg:2119/jobmanager-pbs-see 2883 160 3043
ce101.grid.ucy.ac.cy:2119/jobmanager-lcgpbs-see 1008 3 1011
Grand Total 14981 1168 3 16152
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 58
Accredited by EUGridPMA on March 05, 2007
Included in the IGTF CA RPM distribution from version 1.13
Effective operations started March 21, 2007
Web-page: http://www.ca.acad.bg/
Location: IPP-BAS, Sofia, Bulgaria
Personnel:
4 CA staff members;
2 RA.
Bulgaria (5):BG.ACAD CA – Status overview
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 59
Bulgaria (6):BG.ACAD CA – Status overview
During the period March-August, 2007:
Total of 40 certificates are signed by BG.ACAD CA, including:
30 user certificates 10 host certificates
Total of 3 certificates are revoked by BG.ACAD CA.
Regular patches and updates to CA’s OS and software are applied.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 60
Bulgaria (7):BG.ACAD CA - Development
A concise end-user guide is written and published on the web-site. It covers the basics of the application process. A shell-script for easier certificate request generation is developed and published. It contains step-by-step instructions and examples.Three cron-jobs are developed on the CA’s web server. These scripts monitor the following things: Validity of the published certificates Expiration of the published certificates Expiration of the published CRL
Instant e-mail notifications to the CA’s staff members are provided.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 61
Romanian CA was accredited by EuGridPMA body and it will be operated by the Romanian Space Agency (ROSA)Site operational problems Technical: Air cooling (RO-03-UPB), Room renovation (RO-06-
UNIBUC) Non-technical(vacations and other personnel issues): all RO-01-ICI, RO-03-UPB, RO-05-INCAS, RO-06-UNIBUC: uncertified
status RO-07-NIPNE, RO-08-UVT: certified
Objectiv no. 1: Re-certify all the sites until 1st. October Migrate RO-05-INCAS to EGEE if possible
Romania
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 62
Turkey (1):Sites Operation/Ticket Handling
A new EGEE site has been added (TR-05-BOUN) by the beginning of April 2007.Dedicated resources:TR-01-ULAKBIM site (48 CPUs for seegrid, 16 CPUs for sgdemo) and TR-05-BOUN (8 CPUs for seegrid).From Classic SE to DPM migration has been completed at TR-01-ULAKBIM and TR-05-BOUN.DPM ownership patches have been done.SEE-GRID-2 accounting patches have been done.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 63
Turkey (2):Sites Operation/Ticket Handling
Although seegrid jobs has run successfully, there has been frequent SAM failures of TR-05-BOUN in the last three months due to unknown prd/sgm account problems. We will compensate this lack of availability with forthcoming good performance of the site. Within October 2007, SEE-GRID-2 TR-* sites are planned to be upgraded to Scientific Linux 4.5 together with glite 3.1 middleware.Periodical updates, security patches have been done for all SEE-GRID-2 TR-* sites.Regular user, site problems were handled through SEE-GRID and national helpdesks.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 64
Turkey (3):Core services
Smooth operation of the following core services has been enabled: Core services supporting seegrid VO: RB, BDII, WMS,
MYPROXY, P-GRADE Portal Core services supporting sgdemo VO: RB, BDII, WMS,
MYPROXY, LFC, P-GRADE Portal
RB/WMS statistics have been provided for D3.1b.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 65
Turkey (4):P-GRADE portal
File Manager Portlet for Remote Storage Elements has been reviewed and tested together with SZTAKI.Specification of the file management for remote storage elements portlet was co-authored by SZTAKI and METU, and it is developed by METU.The portlet supports: LFC interaction commands for directory management and file management through LCG file naming conventions, namely, LFN and GUID.Credential Manager Portlet for MyProxy has been reviewed and tested. The portlet is to be integrated with the P-GRADE Grid Portal.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 66
Hungary
OS upgrade to SL 4.4Glite upgrade to 3.0xRecabling of the internal network (new switch deployed)Maintenance Grid-Operator-On-Duty (GOOD) 1 week in may/june Infrastructure support, resolving Helpdesk tickets (script
installation/hardware changes/security updates)
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 67
Albania (1): Overall progress
Change of CA Follow-up of certificate problems
Received local funding for equipment Installed glite in biggest part of equipment
Creation of new sites New experimental site at INIMA New experimental site at FIE Preparations for site of FNS Preparations for site of FECO Plans for University of Elbasani and of Shkodra
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 68
Albania (2): INIMA
Old site AL-01-INIMA following upgradesNew site AL-04-INIMA with 9 nodes (4 nodes will be transferred in other universities)Problems with new status and building of INIMA…
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 69
Albania (3): FIE
Power supply problems, have to put some money to resolve the problem, to by inverters. Switched of during vacances
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 70
Albania (4): FNS
Cluster installedProblem with real IPs, in some institutions Administrative problems to get separate Internet link …
…
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 71
Bosnia and Herzegovina (1)
BA-01-ETFBL functioning correctly New 4 x WN (C2D, 1 GB RAM, 80 GB HDD) New SE – C2D, 2GB RAM, 2x320GB HDD
BA-03-ETFSA New server node - HP ML110G4, X3040, 4GB, 2x160GB New 11 WNs - HP dc5750, MT A64-35, 1GB, 160GB New Switch
BA-04-PMFSA New 4 x WN (C2D, 1 GB RAM, 160 GB HDD) New Switch
BA total now: Total CPUs: 50+ Total Storage: 1+ TB
Availability much better
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 72
Bosnia and Herzegovina (2)
SAM Server Portal BBmSAM
Service availability and SLA calculations implemented MAINTENANCE status implemented in a better way Enhanced OVERVIEW (main) page of BBmSAM
– Showing uptime for last 24h– Filters available for: country, tier, certified status, last test state
BBmobileSAM Now also showing uptime percentage for last 24h
HGSM integration Preparation for HGSM shift to new version
Database Now running strictly off MySQL, no Oracle used Reorganization of indexes – improved performance
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 73
FYR of Macedonia (1):Cluster addition equipment
MK-01-UKIM cluster 18 new nodes added using VirtualBox WN Tested with support of Antun Currently installed on the old CE node but a new CE will be
installed to support these WNs
By the end of the year 16 new CPUs will be installed (non VirtualBox) 2TB storage
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 74
FYR of Macedonia (2):Cluster addition equipment
MK-02-ETF cluster 24 new CPU added By end of september new SE and CE will be installed SE 1TB storage
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 75
FYR of Macedonia (3):New Clusters
MK-03 Still in progress Hardware purchase is done Consultations of initial installation
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 76
FYR of Macedonia (4):Other activities
Wiki pages will be provided for the installation of VirtualBox WNCA status: Software is installed. We will proceed in September with the review by EUGRID PMA. Our representative was attending the last EUGRID PMA
meeting in Turkey.
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 77
Serbia (1): Infrastructure status
6 sites across the countryCurrent number of CPUs: 195 (increase of 43 compared to PSC04)Storage: 0.4 TB (approx. the same)Expansion plans All 3rd parties already have a site We expect two new sites, one in Novi Sad (Faculty of
Agriculture, University of Novi Sad), and one in Nis (IRVAS SME)
Hardware delivery for AEGIS01-PHY-SCL is expected this week, 32 64-bit cores, and storage upgrade to more than 20 TB; another purchase is being finalized (more CPUs)
AEGIS will propose hardware purchase from Serbian National Investment Plan for the whole NGI
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 78
Serbia (2):Core services / SEEGRID Resources
LCG-RB, GLITE-WMS, BDII, MyProxy, LFC at IPBLFC at UOB for SEEGRID and SGDEMO VOVOMS for AEGIS VO, can be deployed as a backup for SEEGRID VO if necessarySupport to T-infrastructure In all core services In sites: AEGIS02-RCUB, AEGIS04-KG
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 79
CP/CPS Document finalisedObject identifier: 1.3.6.1.4.1.23658.10.1.1.0Date: 02 December 2006DNs: Issuer: C=RS, O=AEGIS, CN=AEGIS-CA Subject: C=RS, O=AEGIS, OU=XXX, CN=Subject-name Country: Must be “RS” Organization: Must be “AEGIS” OrganizationUnit: Must be the name of the subject's institute CommonName: First name and last name of the subject for user
certificates, DNS
Serbia (3): AEGIS CA
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 80
Accreditation request on August 23, 2006Under review by GridAUTH team and SRCE CA (on behalf of EUGridPMA) Accredited on June 1, 2007Operational since June 10, 2007Already issued 53 user and host certificates
Serbia (4): AEGIS CA
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 81
Serbia (5): Other activities
Leading WP3; overall activities coordination and representation at various Grid meetingsProviding core servicesWiki contributions: GLITE-3 guide SL4.5 WN gLite-3.1 guide for 32 bit and 64 bit architectures
Grid-Operator-On-Duty, doing it on shifts and coordinatingMW deployment, assessment, upgrade coordinationOperations coordinationDevelopment coordination Collaboration with EGEE Collaboration with other regional Grid projects
Development involvement Problems identification, support for debugging and patching Customizations of YAIM and providing glite-yaim-seegrid
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 82
Montenegro
Sites & upgrades 1 site (MREN-01-CIS)
WN : Upgrade from 4 to 24 CPUs and upgrade on SL 4.4 Storage : upgrade to 0.54 Tb Migration from Classic SE to DPM SE MPI support
No centralized services of SEE-GRID in UoMCA CP/CPS document was written and sent to see-ca-incubation
mailing list for approval and suggestions
Sites operation and ticket handling status/problems Few problems with DPM SE installation
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 83
Moldova (1)
MD-01-TUM site configuration, setting up and internal tests procedures
MD-01-TUM site hardware issues resolved (caused a delay in internal tests and site further operation)
Working on CA service development Learning other countries CA experience. Study of EugridPMA documents and selection of those appropriate for
conditions in Moldova
Determination of future sites hardware configuration
Organization of the tender for purchasing of the equipment according to the MoUs with 3 institutions (Institute of the Mathematics and Computer Science, State University of Medicine and Pharmaceutics, Faculty of Radio Electronics of the Technical University of Moldova)
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 84
Moldova (2)
New sites are expected to join MD-GRID infrastructure till the end of the project:
MD-02-IMI site which will be installed in the Institute of the Mathematics and Computer Science (8 Intel Xeon quad core CPUs, 1,5 TB of storage)
MD-03-SUMP site which will be installed in the State University of Medicine and Pharmaceutics (5 Intel PIV CPUs, 1 TB of storage)
MD-04-RENAM, which will be placed in the FRE TUM (Faculty of Radio Electronics of the Technical University of Moldova) NOC of the RENAM Association (8 Intel Xeon dual core CPUs, 2 TB of storage)
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 85
Croatia (1): Site status
HR-01-RBI site WNs upgraded to
Debian 4.0 gLite 3.1 TAR
sgdemo and MPI enabled node packages up-to-date
HR-02-GRF site hardware being purchased 4 nodes ~ 30 CPUs:
2 x Intel Xeon 5310 QuadCore CPU, 1.6 GHz, 8 MB L2C 8 GB ECC FBD RAM 2 x 500 GB SATA HDD 2 x Gigabit Ethernet
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 86
Croatia (2): Other activities
VOMS server regular maintenance primary for seegrid backup for sgdemo, see
National CA operated by SRCEGOOD shiftsWiki updates BDII response time standalone SAM VOMS configuration
local user support for middleware problemspreparation of review reports about seegrid VO
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 87
WP3 APs and Issues (1)
Communication / response problemsDeadlines must be reasonably set but also respectedAll sites need to resolve their operational problems, and solve all Helpdesk tickets, esp. the outstanding onesSLA conformance monitoring will continueHelpdesk improvements needed – statistics extraction needs to be perfected; currently this is difficult (end of October)Wiki reorganization and updates ASAP finishedApplication SEEGRID VO VOMS roles started to be used by WP4 application developers ASAPApplication level accounting implemented partially; should be fully implemented and used ASAP
SEE-GRID-2 PSC05 Meeting - Thessaloniki, Greece, 11-12 September 2007 88
WP3 APs and Issues (2)
Moldova to join the infrastructure by finishing certification of MD-01-TUM site ASAP3M reporting of sites to include information on updates and operational problems according to the template (to be provided)Partners should be more responsible when performing GOOD shiftsMPI WG to be established and to define standard for MPI setup of SEE-GRID sites; to finish its work until mid-OctoberInfrastructure, site, VO metrics to be precisely definedWhat happened to live UI (Boro)?SEEGRID VO commitments ASAPReview of critical SAM tests