theory, grid and vo

09.04.2008

Theory, Grid and VO

Matthias Steinmetz (AIP)

09.04.2008

applicationData

analysis

Super Computer

Super computer

Telescopes

Data Archive

DataArchive

PC Cluster

PC Cluster Data analysis

User

MIDDLEWARE

Resources are "virtualized", i.e. can not be identified

individually

Characteristics of a Grid: network of IT-Ressourcen

3VODCA: Theory, Grid and VO09.04.2008

VO and Grid

What is the dividing line between VO and Grid? Not well defined Example UK:

AstroGrid covers VO and Grid aspects Example Germany

GAVO: application layerAstroGrid-D: middle ware


Theory and Grid

Benefits of the Grid Logistics (resource monitoring, scheduler/broker,

virtual organizations, …) Virtual Surveys (Millenium simulation) Enterprise computing (access to supercomputers

via grids, e.g. DEISA) Cloud computing (“task farming”) Volunteer computing (@home model) Visualization


StellarIS: resource monitoring

Grid-Ressource-Map

Basiert auf GoogleMap


Stellaris: Job monitoring

Webinterface for simple job monitoring

Time table forsubmitted

Jobs

Minutes

hours

days


Analysing Cosmological Simulations in the Virtual Observatory:Designing and Mining the Millennium Simulation Database

Gerard Lemson German Astrophysical Virtual

Observatory

ARI, Heidelberg

MPE, Garching bei München


Time evolution: merger trees


Merger trees :select prog.* from galaxies des , galaxies prog where des.galaxyId = 0 and prog.galaxyId between des.galaxyId and des.lastProgenitorId

Branching points :select descendantId from galaxies des where descendantId != -1 group by descendantId having count(*) > 1


Usage statistics

Up since August 2006 (astro-ph/...) ~210 registered users > 4.4 million queries ~ 35 billion rows (since March 2007)

# queries/day # rows/day # secs/day


Enterprise Computing:Mare Nostrum Simulations at HLRZ

WMAP3 parameters, w=0.8Testrun using the grid: 10243+10243 particles


NBODY6++ UseCase

Computer simulation of dense stellar systems

Example: globular clusters

Gravitational Star-Star interactionComplexity N2 (N: number of stars)


Enterprise ComputersEnterprise Computers

Recent Development: GPU – Graphics Cards

GeForce 8800 GTX (NVIDIA)Using CUDA LibrarySpecial Interfaces and API from GRACE project ported.

Berczik et al. 2008Spurzem et al. 2008


Cloud Computing: UseCase Dynamo

Visualization of results

Rechner 1 Rechner 2

Rechner 3 Rechner 4

2D-Darstellungder

Magnetfeldstärke auf der

Sternoberfläche

Querschnittdurchden

Stern


<jsdl:JobDefinition xmlns="http://www.gacg-grid.de/namespaces/job-mgmt/2006/08/jsdl" xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl" xmlns:jsdl-posix="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

<jsdl:JobDescription> <jsdl:JobIdentification> <jsdl:JobName> Sample Dynamo run </jsdl:JobName> <jsdl:Description>

Use Case Dynamo </jsdl:Description> <jsdl:JobProject> n/a </jsdl:JobProject> </jsdl:JobIdentification><jsdl:Resources> <jsdl:FileSystem name="HOME"> <jsdl:Description> User's home directory </jsdl:Description> </jsdl:FileSystem> </jsdl:Resources>

[…]

<?xml version="1.0" encoding="UTF-8"?>

<job> <executable>test.x</executable> <directory>/${GLOBUS_USER_HOME}/

dynamo</directory> <stdout>test.out</stdout> <maxWallTime>100</maxWallTime> <maxMemory>1</maxMemory> <fileStageIn> […]

Dynamo JDSL und RSL


Volunteer Computing: GEO600 / LIGO

Laser Interferometer Gravitational Wave Observatory


GEO600/LIGO

Network von 4 Detectors Hanford (2000m side length) USALivingston (4000m side length) USAGEO600 ( 600m side length) GermanyVirgo (3000m side length) Italy

Pathfinder for LISA, Satellite mission with 3 detectorsside length: 5*109 m!


Gravitationional waves: Data analysis via the Grid

Data analysis via small data packages, “embarrassingly parallel”.

Einstein@Home is, like SETI@Home, suitable to exploit idle cycles on work stations.

Einstein@Home is an ideal, simple Grid application, supporting many operation system.

Checkpointing and Recovery allows a very accurate control of CPU-Requirements and walltime.

Automatic software deployment job submission and job management, a good scalability of grid application can be obtained

Current workload: 30000 CPU h per week


GEO600 – Resource Integration

user friendly User-Management via VOMRS

Resource information via MDS und StellarIS

Grid Service Monitoring

automatic job submission on D-Grid resources

Job monitoring und job management using a

Laptop

data management on Astrogrid-D Storage

Cluster


Visualization of a galaxy merger

Simulation: two galaxies on collision orbit Visualization: 2D-projections of 3D snapshots

ZIB AIP

Submit Execution

ProC SubmitWorkflow

Video Workflow A

stop

exit

exit

start

GT4 submit

GT4 submit

Video Workflow B

ARI ...


Grid-Visualization

Submission Host: ZIB ProC + Master workflow

Submission of video workflows Display of videos

Execution Hosts: AIP + ZAH PiCo + Video workflow

Projection to 2D Color coding

Future: Graphics rendering at LRZ, graphics output on local host


Theory and Grid

Benefits of the Grid Logistics (resource monitoring, scheduler/broker,

virtual organizations, …) Virtual Surveys (Millenium simulation) Enterprise computing (access to supercomputers

via grids, e.g. DEISA) Cloud computing (“task farming”) Volunteer computing (@home model) Visualization

theory, grid and vo

Documents