the interactive european grid project paul heinzlreiter gup, university linz coregrid summer school,...
TRANSCRIPT
The Interactive European Grid Project
Paul HeinzlreiterGUP, University Linz
CoreGrid Summer School, Budapest, 7.9.2007
CoreGrid Summer School, Budapest, 7.9.2007 2
Outline
General Project Overview Research Activities Infrastructure Applications
CoreGrid Summer School, Budapest, 7.9.2007 3
i2g Partners
CoreGrid Summer School, Budapest, 7.9.2007 4
i2g History
Precursor:Crossgrid Project
First Project to deliver Interactivity for Grids
i2g improves:Key ComponentsIntegrationAdaptation to new MiddlewareNew applications
CoreGrid Summer School, Budapest, 7.9.2007 5
i2g Aims
Timeframe: May 2006 – April 2008 Interactivity User-Friendliness MPI Support Support of Demanding Applications Integration of all Components Provision of a Good User Experience Interoperability (with EGEE / gLite)
CoreGrid Summer School, Budapest, 7.9.2007 6
i2g Joint Research Activity
Interactivity & VisualizationResource BrokerInteractive ConnectionsVideo Streaming
CrossBrokerReduced Job Startup Times
Graphical User InterfaceJob ControlData HandlingJob Output
MPI SupportIntra-Site and Inter-Site
Active Security
CoreGrid Summer School, Budapest, 7.9.2007 7
(i2)glogin
Workernode
Workernode
Workernode
Workernode
Workernode
Workernode
Workernode
Workernode
Workernode
Workernode
GatekeeperGatekeeper
on the Grid
ClientClient
gloginPoint ofContact
job submit
glogin’
fork
socket
glogin’’
fork
inherited socket
return
return
Problem: GASS cache
interactivebidirectional connection
trafficforwarding
CoreGrid Summer School, Budapest, 7.9.2007 8
GVid
vtk
App
MP4 Encoder
Event Decoder
TCPTransport
SocketTransport
FileTransport
PipeTransportre
mo
teG
lut
Ap
plic
atio
nG
Vid
SD
LO
utpu
tclie
nt
MP4 Decoder
Event Encoder
TCPTransport
SocketTransport
FileTransport
PipeTransport
Grid UI
glogin gloginGSS secured $ grid-proxy-init
$ glogin -x –C altix1
CoreGrid Summer School, Budapest, 7.9.2007 9
GUI – Migrating Desktop
User-Friendly Platform Independent
Java Running Sequential & MPI
Jobs Roaming Access Server
interfaces with the UI Data Management Easily Extendable
Plugin Mechnism for new Applications
Extension for Interactive Remote Visualization
CoreGrid Summer School, Budapest, 7.9.2007 10
GUI – Migrating Desktop
CoreGrid Summer School, Budapest, 7.9.2007 11
SchedulingAgent
ResourceSearcher
ApplicationLauncher
Condor-G DAGMan
CE
WN WN
LCG
CE
WN WN
gLite
MigratingDesktop
InformationIndex
ReplicaManager
CrossBroker
Interactive Job BrokerAutomatic Job Managementfor Parallel Applications
Resource SearchingJob ConditioningLaunching, Monitoring, RetryResult Retrieval
Workflows, Interactive & Batch Jobs, MPI Support
JDL ExtensionsCompatible
Best Effort Approachfor Failures / ProblemsImproved Job Startup Time
CoreGrid Summer School, Budapest, 7.9.2007 12
MPI Support
OpenMPIJoint Effort / OSSBest of FT-MPI, LA-MPI, LAM-MPI, PACX-MPI
PACX-MPIInter-Cluster / SiteSupports Vendor-MPI
MPI-StartShell ScriptsAbstraction LayerWMS / Schedulers and MPI-ImplementationsFlexibility (relocateable Shellscripts)
PACX-MPI
LAM/MPI
LA-MPI
FT-MPI
MPIImplementations Scheduler
WMS (CrossBroker)
MPI-STARTPACX-MPI
Vendor MPI
Application
Intra Cluster Communication
Inter Cluster Communication
PACX-MPI MPI-START
CoreGrid Summer School, Budapest, 7.9.2007 13
Active Security
Security measures for interactive frameworkActive Security Tools:
Track Grid InteractionsRespond to Unwanted Security Events
Extended Security Monitoring & TrackingCoordination among Security Tools & Sites
Architecture Divided in 2 LayersMonitoring & Control Layer
Security Monitoring & Control EngineAnalysis Layer
Alert AnalysisAlert Correlation Engine
R-GMACommunication Layer
CoreGrid Summer School, Budapest, 7.9.2007 14
Integration
Most Advanced: Fusion Application – Integrates Components from JRA
MPI SupportCrossbrokerVisualizationVideostreamingGUI SupportInteractivity
CoreGrid Summer School, Budapest, 7.9.2007 15
Interactivity & Visualization
i2glogin GVid CrossBroker Migrating
Desktop Full Integration + MPI Support
Job Submission Services
CrossBroker
Logging&
Bookkeeping
Roaming Access Server
Computing Element
LRMS
Gatekeeper
JDL
Migrating Desktop
,
WorkerNode
gLogin
Simulation +visualisation
WorkerNode
vtk Ap
p
MP4 Encoder
Event Decoder
TCPTransport
SocketTransport
FileTransport
PipeTransport
remo
teGlu
tA
pp
lica
tion
GV
id
Java Video
Pla
yer
MP4 Decoder
Event Encoder
TCPTransport
SocketTransport
FileTransport
PipeTransport
Job Submission Services
CrossBroker
Logging&
Bookkeeping
Roaming Access Server
Computing Element
LRMS
Gatekeeper
JDLJDL
Migrating Desktop
,,
WorkerNode
gLogin
Simulation +visualisationSimulation +visualisation
WorkerNode
vtk Ap
p
MP4 Encoder
Event Decoder
TCPTransport
SocketTransport
FileTransport
PipeTransport
remo
teGlu
t
MP4 Encoder
Event Decoder
TCPTransport
SocketTransport
FileTransport
PipeTransport
remo
teGlu
tA
pp
lica
tion
GV
id
Java Video
Pla
yer
MP4 Decoder
Event Encoder
TCPTransport
SocketTransport
FileTransport
PipeTransport
Java Video
Pla
yer
MP4 Decoder
Event Encoder
TCPTransport
SocketTransport
FileTransport
PipeTransport
Java Video
Pla
yer
MP4 Decoder
Event Encoder
TCPTransport
SocketTransport
FileTransport
PipeTransport
Java Video
Pla
yer
MP4 Decoder
Event Encoder
TCPTransport
SocketTransport
FileTransport
PipeTransport
CoreGrid Summer School, Budapest, 7.9.2007 16
Integration: Fusion Application
Application is integrated in Migrating Desktop
Communication Channel to MPI Master over the Roaming Access Server
Communication MD – RAS can also go over TCP/IP
CoreGrid Summer School, Budapest, 7.9.2007 17
The i2g Infrastructure
12 Sites 430 CPU Cores 21 TB of Storage Multiple
Architectures i2g Middleware
Componentsavailable forApplications
CoreGrid Summer School, Budapest, 7.9.2007 18
The i2g Infrastructure (Production)
CoreGrid Summer School, Budapest, 7.9.2007 19
i2g Applications
Plasma Fusion Simulation & Visualization First Images of the Universe Brain Diagnostic Control of Pollution …
CoreGrid Summer School, Budapest, 7.9.2007 20
Interactive Plasma Fusion Simulation
Most Advanced Application
MPI-Parallel Simulation Remote Visualization Full Interaction (GUI,
Mouse) GUI Integration
Distinguished as best Application Demo at OGF in Manchester
CoreGrid Summer School, Budapest, 7.9.2007 21
Fusion Application in MD
CoreGrid Summer School, Budapest, 7.9.2007 22
First Images of the Universe
Simulation of the Measurements of the Planck Satellite to be launched in 2008.
Model ValidationRadiation Values should have Gaussian DistributionIf not, Model Error
Calculating Autocorrelations:
Sky gets divided100000 Simulations, 5 Minutes each
Output gets visualized
CoreGrid Summer School, Budapest, 7.9.2007 23
Brain Diagnostic
Developing and Testing new Diagnostic Methods for Brain Images
Requires large DatasetsDICOM ImagesCorresponding Brain MapsFinal Diagnostic Map
E.g. Perfusion Studies (Stroke Diagnosis)
Visualization and Annotation
CoreGrid Summer School, Budapest, 7.9.2007 24
Control of Pollution
Developing of a Model Suite to analyze the Evolution of Pollution Clouds.
Compute Intensive Models
Interaction enables steering of Simulation Parameters
Impact Analysis MPI-Parallel Simulation
CoreGrid Summer School, Budapest, 7.9.2007 25
i2g Offer
Support for external developers and users’ application
s
The Interactive European Grid project helps application developers in adapting their applications to work in a Grid environment as well as in parallelization of these applications. If you would like us to consider running your application in the int.eu.grid infrastructure or creating a new VO, please contact:
Isabel Campos IFCA-CSIC*, Santander, Spain
email: iscampos AT ifca.unican.es
Gridinfrastructu
re
The Project helps to integrate new clusters to the int.eu.grid infrastructure as well as to establish agreements between research communities and infrastructure providers.
If you would like to join the int.eu.grid infrastructure with your cluster, please contact:
Project support teamemail: i2g.support AT lip.pt
Education
Tutorials: for Grid beginners, Grid site administrators, application developers; some tutorials on MPI and other Grid specific.
If you would like to join the tutorial organized at our site or to organize our tutorial at your facilities, please contact:
Celso Martínez RiveroIFCA-CSIC,
Santander, Spainemail: mrivero AT
ifca.unican.es
Seminars for groups of interested people or consultancy.
If you need a consultancy or you are interested in taking part in a seminar on Grid technology, please contact:
Project Office Santander, Spain
email: inteugrid-office AT ifca.unican.es
specifying the subject, product or problem.
Disseminative
materialsGeneral information about the Project – brochures, posters, etc. General int.eu.grid presentation.
If you are interested in receiving them, please contact:
Robert PajakCYFRONET*, Kraków, Poland
email: r.pajak AT cyfronet.krakow.pl
CoreGrid Summer School, Budapest, 7.9.2007 26
New Applications?
http://dissemination.interactive-grid.eu/how_to_use_grid
Description of interactive tools Use our infrastructure for your application Online user questionaire http://dissemination.interactive-grid.eu/how_to_
use_grid/user_questionnaire/User Data formOur offer (infrastructure, tools, …)
CoreGrid Summer School, Budapest, 7.9.2007 27
i2g Information & Contact
http://www.interactive-grid.eu http://dissemination.interactive-grid.eu
Brochures, Posters, Information ... Contact Persons:
(New) Applications:Isabel Campos; [email protected](New) Infrastructure:[email protected]:Celso Martinez Rivero; [email protected]:Project Office; [email protected]