ganga: a user-grid interface for atlas and lhcb motivation and background objectives and design...

Ganga: a User-Grid Interface for ATLAS

and LHCb

• Motivation and background• Objectives and design• Low-level tools• High-level tools and GUI• Future plans and conclusions

K. Harrison, P. Mato, A. Soroko and C. Tull

25/03/2003CHEP 2003, La Jolla, California 2

Motivation and background

ATLAS and LHCb develop applications within a common framework: Gaudi/AthenaBoth collaborations aim to exploit potential of Grid for large-scale, data-intensive distributed computing

ATLAS and LHCb develop applications within a common framework: Gaudi/AthenaBoth collaborations aim to exploit potential of Grid for large-scale, data-intensive distributed computing

Simplify management of analysis and production jobs for end-user physicists by developing tools for accessing Grid services with built-in knowledge of how Gaudi/Athena works: Gaudi/Athena and Grid Alliance (GANGA)Generic tools interfacing to the Grid can be used at other experiments.

Simplify management of analysis and production jobs for end-user physicists by developing tools for accessing Grid services with built-in knowledge of how Gaudi/Athena works: Gaudi/Athena and Grid Alliance (GANGA)Generic tools interfacing to the Grid can be used at other experiments.


Athena/GAUDI

GANGAGU

I

JobOptionsAlgorithms

Collective&

ResourceGrid

Services

HistogramsMonitoringResults

First ideas for GANGA were presented in summer 2001 (P.Mato and C.Tull) An Atlas/LHCb design team was set up, including US representatives

Motivation and background


Objectives and design

The user interacts with a single application integrating all stages of job life-time.Modular design. Each module can be accessed via Python interface or is entirely written in Python.Although Atlas and LHCb use the same framework and the same software management tool, there are significant differences in what is expected from the software, and Ganga must have the flexibility to cope.Ganga provides a set of tools to manipulate jobs and data. Tools are accessible from CLI (other scripts) or from GUI.Ganga allows access both to the local resources (LSF batch system) and to the GRID.Should follow, and contribute to, developments in LCG.

The user interacts with a single application integrating all stages of job life-time.Modular design. Each module can be accessed via Python interface or is entirely written in Python.Although Atlas and LHCb use the same framework and the same software management tool, there are significant differences in what is expected from the software, and Ganga must have the flexibility to cope.Ganga provides a set of tools to manipulate jobs and data. Tools are accessible from CLI (other scripts) or from GUI.Ganga allows access both to the local resources (LSF batch system) and to the GRID.Should follow, and contribute to, developments in LCG.


Required FunctionalityJob preparation and configuration Resource booking

Job submission User can choose between Grid and local resource management system

Job monitoring and control GUI for the resource browsing VO active services CEs SEs Query existing files in the Grid

GUI for data management tools e.g., Dataset registration to the Grid (used by Production Manager) Copy file from a Computing Element to a Storage Element Replication of files

Job preparation and configuration Resource booking

Job submission User can choose between Grid and local resource management system

Job monitoring and control GUI for the resource browsing VO active services CEs SEs Query existing files in the Grid

GUI for data management tools e.g., Dataset registration to the Grid (used by Production Manager) Copy file from a Computing Element to a Storage Element Replication of files


Athena/Gaudi ArchitectureConverter

Algorithm

Event DataService

PersistencyService

DataFiles

AlgorithmAlgorithm

Transient Event Store

Detec. DataService

PersistencyService

DataFiles

Transient Detector

Store

MessageService

JobOptionsService

Particle Prop.Service

OtherServices

HistogramService

PersistencyService

DataFiles

TransientHistogram

Store

ApplicationManager

ConverterConverter


Ganga: Python Bus Design

Server

BookkeepingDBProductio

nDB

EDG UI

PYTHON SW BUS

XML RPC server

XML RPC module

GANGA Core Module

OS Module

Athena\GAUDI

GaudiPython PythonROOT

PYTHON SW BUS

GU

I

JobConfiguration

DB

Remote user

(client)

Local JobDB

LAN/WAN

GRID

LRMS


Class Diagram of Ganga Core Module

JobHandler Requirements

JobAttributes Credentials

11… Job

JobsRegistryJobsCatalog

Application

GaudiApplicationHandler

Configuration

ConfDataBase

Parameter Executable

1

0…

1

0…

1

1

1

1…


Ganga Job and Application Handlers

ApplicationHandler

AnotherApplicationHandler

GaudiApplicationHandler

DaVinchiApplicationHandler

Address different types of applications

JobHandler

AnotherJobHandler

GridJobHandler

PBSJobHandlerLSFJobHandler

LocalJobHandler

Address different types of batch systems


Core Module Implementation

Most of base classes are developed. Serialization of objects (user jobs) is implemented with the Python pickle module.GaudiApplicationHandler can access Configuration DB for some Gaudi applications (Brunel). It is implemented with the xmlrpclib module. Ganga can create user-customized Job Options files using this DB. DaVinci and AtlFast application handlers are in the last stage of development. LSFJobHandler is implemented. It allows to submit and to get simple monitoring information for a job at LSF batch system.Much of GRID-related functionality is already implemented in GridJobHandler using EDG testbed 1.4 software. Ganga can submit, monitor, and get output from GRID jobs. JobsRegistry class provides jobs monitoring via multithreaded environment based on Python threading module

Most of base classes are developed. Serialization of objects (user jobs) is implemented with the Python pickle module.GaudiApplicationHandler can access Configuration DB for some Gaudi applications (Brunel). It is implemented with the xmlrpclib module. Ganga can create user-customized Job Options files using this DB. DaVinci and AtlFast application handlers are in the last stage of development. LSFJobHandler is implemented. It allows to submit and to get simple monitoring information for a job at LSF batch system.Much of GRID-related functionality is already implemented in GridJobHandler using EDG testbed 1.4 software. Ganga can submit, monitor, and get output from GRID jobs. JobsRegistry class provides jobs monitoring via multithreaded environment based on Python threading module


EDG Grid Interface

Job class Job class JobsRegistry classJobsRegistry classJob Handler

classJob Handler

class

Data management

service

Data management

service

Job submissionJob submission Job monitoring Job monitoring Security serviceSecurity service

dg-job-list-match

dg-job-submit

dg-job-cancel

dg-job-list-match

dg-job-submit

dg-job-cancel

grid-proxy-init

MyProxy grid-proxy-init

MyProxy

dg-job-status

dg-job-get-logging-info

GRM/PROVE

dg-job-status

dg-job-get-logging-info

GRM/PROVE

edg-replica-manager

dg-job-get-output

globus-url-copy

GDMP

edg-replica-manager

dg-job-get-output

globus-url-copy

GDMPEDG UI


Ganga: low-level tools

In Ganga, components used to define a job are mapped to Python classes.

The low-level software needs to provide representations of the most basic components, and must allow flexibility in manipulating them.

Users will have access to low-level data and methods at the command line, but in most cases shouldn’t need them

Low-level software works, but is preliminary. Some details of component representations and their

interfaces are still being discussed by development team, but most of them have already been defined.

In Ganga, components used to define a job are mapped to Python classes.

The low-level software needs to provide representations of the most basic components, and must allow flexibility in manipulating them.

Users will have access to low-level data and methods at the command line, but in most cases shouldn’t need them

Low-level software works, but is preliminary. Some details of component representations and their

interfaces are still being discussed by development team, but most of them have already been defined.


Low-level tools: Atlas example (AtlFast)

atlasSetup = GangaCommand(“source

/afs/cern.ch/user/h/harrison/public/atlasSetup.sh”)

atlfast = GangaCMTApplication(“TestRelease”,

“TestRelease-00-00-15”,”athena.exe”,

“run/AtlasfastOptions.txt”)

atlfastOutput = GangaOutputFile(“atlfast.ntup”)

workStep1 = GangaWorkStep([atlasSetup,atlfast,atlfastOutput])

workFlow = GangaWorkFlow([workStep1])

lsfJob = GangaLSFJob(“atlfastTest”,workFlow)

lsfJob.build()

lsfJob.run()

atlasSetup = GangaCommand(“source

/afs/cern.ch/user/h/harrison/public/atlasSetup.sh”)

atlfast = GangaCMTApplication(“TestRelease”,

“TestRelease-00-00-15”,”athena.exe”,

“run/AtlasfastOptions.txt”)

atlfastOutput = GangaOutputFile(“atlfast.ntup”)

workStep1 = GangaWorkStep([atlasSetup,atlfast,atlfastOutput])

workFlow = GangaWorkFlow([workStep1])

lsfJob = GangaLSFJob(“atlfastTest”,workFlow)

lsfJob.build()

lsfJob.run()

At this level, there is less built-in intelligence than in ASK, but a lot of flexibility

At this level, there is less built-in intelligence than in ASK, but a lot of flexibility


Low-level tools: LHCb example (DaVinci)

lhcbSetup = GangaCommand(“source /afs/cern.ch/lhcb/scripts/lhcbenv.sh”)

daVinciSetup = GangaCommand(“source /afs/cern.ch/lhcb/scripts/ProjectEnv.sh DaVinci v7r2”)

daVinci = GangaCMTApplication(“Phys/DaVinci”, “v7r2”,”DaVinci.exe”, “options/DaVinci.opts”)

daVinciOutput = GangaOutputFile(“dvhistos.hbook”)workStep1 = GangaWorkStep([lhcbSetup,daVinciSetup,daVinci, daVinciOutput])workFlow = GangaWorkFlow([workStep1])lsfJob = GangaLSFJob(“daVinciTest”,workFlow)lsfJob.build()lsfJob.run()

lhcbSetup = GangaCommand(“source /afs/cern.ch/lhcb/scripts/lhcbenv.sh”)

daVinciSetup = GangaCommand(“source /afs/cern.ch/lhcb/scripts/ProjectEnv.sh DaVinci v7r2”)

daVinci = GangaCMTApplication(“Phys/DaVinci”, “v7r2”,”DaVinci.exe”, “options/DaVinci.opts”)

daVinciOutput = GangaOutputFile(“dvhistos.hbook”)workStep1 = GangaWorkStep([lhcbSetup,daVinciSetup,daVinci, daVinciOutput])workFlow = GangaWorkFlow([workStep1])lsfJob = GangaLSFJob(“daVinciTest”,workFlow)lsfJob.build()lsfJob.run()

Command sequences for Atlas and LHCb are very similar

Command sequences for Atlas and LHCb are very similar


Ganga: high-level tools

In Ganga, common tasks are realized via methods of job and application handler classes (JobHandler, ApplicationHandler and their subclasses).

The low-level software needs to provide building blocks for these methods.

High-level tools are available both at the command line and through a GUI.

More advanced functionality will appear with the progress of low-level tools development.

In Ganga, common tasks are realized via methods of job and application handler classes (JobHandler, ApplicationHandler and their subclasses).

The low-level software needs to provide building blocks for these methods.

High-level tools are available both at the command line and through a GUI.

More advanced functionality will appear with the progress of low-level tools development.


Ganga: GUI Implementation

GUI is created using wxPython extension module.All job configuration data are represented as a hierarchical structure accessible via tree control. Most important job parameters are brought to the top of the tree.All job parameters defined by the user can be edited via GUI dialogs. All implemented tools are available via GUI, but some of them require more elaborated interface, e.g., Job Options browser/editor.Python interpreter is embedded into the GUI and allows user to configure interface from the command line.

GUI is created using wxPython extension module.All job configuration data are represented as a hierarchical structure accessible via tree control. Most important job parameters are brought to the top of the tree.All job parameters defined by the user can be edited via GUI dialogs. All implemented tools are available via GUI, but some of them require more elaborated interface, e.g., Job Options browser/editor.Python interpreter is embedded into the GUI and allows user to configure interface from the command line.


GUI Snapshots

Embedded Python

interpreter


GUI Snapshots

Create a new job


GUI Snapshots

Access Job (application)

options downloaded

from DB


GUI Snapshots

Define batch

system


GUI Snapshots

Define necessary resources


GUI Snapshots

Submit the job (after

configuration)


GUI Snapshots

Job has been submitted

successfully


GUI Snapshots

Job monitoring has been started

automatically


GUI Snapshots

Standard output for the “Hello

World” LSF job


Future Plans and Conclusions

One of the immediate plans is to complete implementation of Grid data management tools (interface to GDMP and replica-manager).Advanced Job Options Editor should be developed and integrated with Ganga.We plan to add new job handlers, e.g., for PBS and for local PC.While we assume that basic LHCb software is available at worker nodes, we can’t require the same for the user code. Therefore the possibility to build user software remotely with Ganga should be investigated. Web-based variant of GUI (or thin remote client) should be considered. Some questions remain related to the security issues, but we can try Globus PyCoG to overcome these problems.We plan to investigate possibilities for exploiting Grid Monitoring Architecture in Ganga.Code for the Ganga package can be obtained from Gaudi CVS repository.More information about GANGA development can be found in the web site http://ganga.web.cern.ch/ganga/

One of the immediate plans is to complete implementation of Grid data management tools (interface to GDMP and replica-manager).Advanced Job Options Editor should be developed and integrated with Ganga.We plan to add new job handlers, e.g., for PBS and for local PC.While we assume that basic LHCb software is available at worker nodes, we can’t require the same for the user code. Therefore the possibility to build user software remotely with Ganga should be investigated. Web-based variant of GUI (or thin remote client) should be considered. Some questions remain related to the security issues, but we can try Globus PyCoG to overcome these problems.We plan to investigate possibilities for exploiting Grid Monitoring Architecture in Ganga.Code for the Ganga package can be obtained from Gaudi CVS repository.More information about GANGA development can be found in the web site http://ganga.web.cern.ch/ganga/

ganga: a user-grid interface for atlas and lhcb motivation and background objectives and design...

Documents

potential of grid

usergrid interface

set of tools

lhcb motivation

background atlas

background slide

software management

management of analysis