5 th eu datagrid conference, budapest, september 2002 the european crossgrid project marcel kunze...
Post on 21-Dec-2015
216 views
TRANSCRIPT
5th EU DataGrid Conference, Budapest, September 2002
The European CrossGrid Project
Marcel Kunze
Abteilung Grid-Computing und e-Science
Forschungszentrum Karlsruhe GmbH
On behalf of the CrossGrid Collaboration
Special thanks to Marian Bubak and Jesus Marco
www.eu-crossgrid.org
Forschungszentrum Karlsruhein der Helmholtz-Gemeinschaft
5th EU DataGrid Conference, Budapest, September 2002
Outline
The CrossGrid Project
CrossGrid Aplications
Testbed Status
Deliverables and Plans
5th EU DataGrid Conference, Budapest, September 2002
EU Funded Grid Project Space (Kyriakos Baxevanidis)
GRIDLAB
GRIA
EGSO
DATATAG
CROSSGRID
DATAGRID
Applications
GRIP EUROGRID
DAMIENMiddleware
& Tools
Underlying Infrastructures
ScienceIndustry / business
- Links with European National efforts
- Links with US projects (GriPhyN, PPDG, iVDGL,…)
5th EU DataGrid Conference, Budapest, September 2002
CrossGrid Collaboration
Poland:Cyfronet & INP CracowPSNC PoznanICM & IPJ Warsaw
Portugal:LIP Lisbon
Spain:CSIC SantanderValencia & RedIrisUAB BarcelonaUSC Santiago & CESGA
Ireland:TCD Dublin
Italy:DATAMAT
Netherlands:UvA Amsterdam
Germany:FZK KarlsruheTUM MunichUSTU Stuttgart
Slovakia:II SAS Bratislava
Greece:AlgosystemsDemo AthensAuTh Thessaloniki
Cyprus:UCY Nikosia
Austria:U.Linz
21 21 institutinstitutees s
11 11 countriescountries
5th EU DataGrid Conference, Budapest, September 2002
Main Objectives
EU Vth Framework Programme (IST) Project, started March 2002
21 partners from 11 countries
New category of Grid enabled applications Computing and data intensive Distributed Interactive, near real time response (a person in a loop) Layered
New programming tools
Grid more user friendly, secure and efficient
Interoperability with other Grids
Implementation of standards
5th EU DataGrid Conference, Budapest, September 2002
Collaboration with other Grid Projects
Exchange of Information Software components
Partners DATAGRID DATATAG GRIDLAB EUROGRID and GRIP
GRIDSTART
Participation in GGF
5th EU DataGrid Conference, Budapest, September 2002
Workpackages
WP1 – CrossGrid Application Development Biomedical simulation and visualization Flooding crisis support Interactive distributed data analysis in HEP Weather forecast and air pollution modeling
WP2 - Grid Application Programming Environments Tools for parallel programming and debugging on the Grid (MPI)
WP3 – New Grid Services and Tools Portals and roaming access Resource management Monitoring Optimisation of data access
WP4 - International Testbed Organisation Integration team, certification authority, support and test procedures Based on EDG 1.2 (see talk of Marcus Hardt in WP6 parallel session)
WP5 - Project Management Coordination, architecture, dissemination
5th EU DataGrid Conference, Budapest, September 2002
Biomedical Application
CT / MRI scan
MedicalDB
Segmentation
MedicalDB
LB flowsimulation
VEWDPC
PDA
Visualization
Interaction
HDB
10 simulations/day60 GB/simulation> 20 MB/s
5th EU DataGrid Conference, Budapest, September 2002
VR-Interaction
5th EU DataGrid Conference, Budapest, September 2002
Cascade of Flood Simulations
Data sources
Meteorological simulations
Hydraulic simulations
Hydrological simulations
Users
Output visualization
5th EU DataGrid Conference, Budapest, September 2002
Example of the Flood Simulation - Flow and Water Depth
5th EU DataGrid Conference, Budapest, September 2002
Distributed Data Analysis in HEP
Objectives Distributed data access
Distributed data mining techniques with neural networks
Issues Typical interactive requests will run on o(TB) distributed data
Transfer/replication times for the whole data about one hour
Data transfers once and in advance of the interactive session
Allocation, installation and set-up of corresponding database servers before the interactive session
Integration of user-friendly interactive access (based on PROOF)
5th EU DataGrid Conference, Budapest, September 2002
Parallel ROOT Facility: PROOF
Local
Remote
Selection
Parameters
Procedure
Proc.C
Proc.C
Proc.C
Proc.C
Proc.C
PROOF
CPU
CPU
CPU
CPU
CPU
CPU
TagDB
RDB
DB1
DB4
DB5
DB6
DB3
DB2
http://root.cern.ch
5th EU DataGrid Conference, Budapest, September 2002
Distributed/parallel codes on the Grid Coupled Ocean/Atmosphere Mesoscale Prediction System
STEM-II Air Pollution Code
Integration of distributed databases
Data mining applied to downscaling weather forecast
Weather Forecast and Air Pollution Modeling
5th EU DataGrid Conference, Budapest, September 2002
Key Features of CrossGrid Applications
Data Data sources and data bases geographically distributed
To be selected on demand
Processing Large processing capacity required; both HPC & HTC
Interactive
Presentation Complex data requires versatile 3D visualisation
Support for interaction and feedback to other components
5th EU DataGrid Conference, Budapest, September 2002
Overview of the CrossGrid Architecture
Supporting Tools
1.4Meteo
Pollution
1.4Meteo
Pollution
3.1 Portal & Migrating Desktop
3.1 Portal & Migrating Desktop
ApplicationsDevelopment
Support
2.4Performance
Analysis
2.4Performance
Analysis
2.2 MPI Verification
2.2 MPI Verification
2.3 Metrics and Benchmarks
2.3 Metrics and Benchmarks
App. Spec Services
1.1 Grid Visualisation
Kernel
1.1 Grid Visualisation
Kernel
1.3 DataMining on Grid (NN)
1.3 DataMining on Grid (NN)
1.3 Interactive Distributed
Data Access
1.3 Interactive Distributed
Data Access
3.1Roaming Access
3.1Roaming Access
3.2Scheduling
Agents
3.2Scheduling
Agents
3.3Grid
Monitoring
3.3Grid
Monitoring
MPICH-GMPICH-G
Fabric
1.1, 1.2 HLA and others
1.1, 1.2 HLA and others
3.4Optimization of
Grid Data Access
3.4Optimization of
Grid Data Access
1.2Flooding
1.2Flooding
1.1BioMed
1.1BioMed
Applications
Generic Services
GRAMGRAM GSIGSIReplica CatalogReplica CatalogGIS / MDSGIS / MDSGridFTPGridFTP Globus-IOGlobus-IO
DataGridReplica
Manager
DataGridReplica
Manager
DataGrid Job Submission
Service
DataGrid Job Submission
Service
Resource Manager
(CE)
Resource Manager
(CE)
CPUCPU
ResourceManagerResourceManager
Resource Manager
(SE)
Resource Manager
(SE)Secondary
StorageSecondary
Storage
ResourceManagerResourceManager
Instruments ( Satelites,
Radars)
Instruments ( Satelites,
Radars)
3.4Optimization of
Local Data Access
3.4Optimization of
Local Data Access
Tertiary StorageTertiary Storage
Replica CatalogReplica Catalog
GlobusReplica
Manager
GlobusReplica
Manager
1.1User Interaction
Services
1.1User Interaction
Services
5th EU DataGrid Conference, Budapest, September 2002
Status after M6++
Software Requirements Specifications together with use cases
CrossGrid Architecture defined
Detailed Design documents for tools and the new Grid services (OO approach, UML)
Analysis of security issues and the first proposal of solutions
Detailed description of the test and integration procedures
Testbed first experience Sites: LIP, FZK, CSIC+USC, PSNC, AuTH+Demo Basic: EDG release 1.2 Applications:
EDG HEP simulations (Atlas,CMS) first distributed prototypes using MPI:
NN distributed training Evolutionary Algorithms
5th EU DataGrid Conference, Budapest, September 2002
CrossGrid Testbed Map
UCY NikosiaDEMO Athens
Auth Thessaloniki
CYFRONET Cracow
ICM & IPJ Warsaw
PSNC Poznan
CSIC IFIC Valencia
UAB Barcelona
CSIC-UC IFCA
Santander
CSIC RedIris Madrid
LIP Lisbon
USC Santiago
TCD Dublin
UvA Amsterdam
FZK Karlsruhe
II SAS Bratislava
Géant
5th EU DataGrid Conference, Budapest, September 2002
Sample Testbed Sites
Valencia GoG farm, Santander (GridWall), FZK
5th EU DataGrid Conference, Budapest, September 2002
Plans for the Future
Participation in production testbed with DataGrid All sites will be ready to join by end of September Common DEMO at IST 2002, Copenhagen, November 4th-6th
Collaboration with DataGrid in specific points (e.g. user support and helpdesk software)
Conference together with RI Forum and the “Across Grids” Santiago de Compostella, Spain, February 9th-14th,2003 With Proceedings (reviewed papers)
CrossGrid workshop, Linz (w/ EuroPVM/MPI 2002), September 28th-29th
5th EU DataGrid Conference, Budapest, September 2002
Linz CrossGrid Workshop Sep.28th-29th
Evaluate the current status of all tasks
Contact partners who are developing software we are going to use: discuss interfaces and functionality
Understand what we may expect as first prototypes
Coordinate the operation of testbeds
Agree about common rules for software development:Standard Operational Procedures (SOP) document written
Start to organize the first CrossGrid EU review
Meet with EU DataGrid representatives
Discuss the technology for the future (OGSA)
Details at http://www.gup.uni-linz.ac.at/crossgrid/workshop/
5th EU DataGrid Conference, Budapest, September 2002
CrossGrid Schedule and Deliverables
M6 (now) WPs 1-3 start to develop prototypes of the applications, tool environment, and the new grid services WP4 are working on launching the first testbed.
M9 (Dec.2002) WP4 internal status report WP5 CrossGrid website
M10 (Jan. 2003) WP4 First testbed prototype release
M11 (Feb. 2003) WP5 Project leaflet/brochure WP5 Dissemination and exploitation report CrossGrid conference in Santiago di Compostella, February 2002
M12 (March 2003) WP1 First software release WP2 First prototypes WP3 First prototypes WP5 Report on requirements on integration and interoperability with DataGrid WP5 Yearly report
5th EU DataGrid Conference, Budapest, September 2002
1980s: Internet 1990s: Web 2000s: Grid
Where do we need to get to ?
Applications to support an “e-society” (“Cyber-Infrastructure”)
A Grid infrastructure which hides the complexities from the users (“Invisible Computing”)
A powerful and flexible network infrastructure
Where do we need to invest ?Applications targeted at realistic problems in “e-science”
Prototypes of Grid infrastructures
Maintain and improve the GEANT network
Expression of Interest for EU FP6 program:“Enabling Grids and e-Science in Europe (EGEE)”
Grid-enabled Applications
Prototype Grid Infrastructures
Gèant: World Class Networking