APELAccounting Update

Dave KantCCLRC, e-Science Centre

EGEE’03, April 2005 - 2

Overview

This is a summary of the status of Accounting & Reporting following its deployment in LCG2_4_0

1. Overview2. APEL Design3. What’s New?4. LCG Accounting (OSG , NorduGrid, EGEE)

EGEE’03, April 2005 - 7

72 Sites publishing data to GOC (May 17th 2005)

Over 1.7 Million Job records

~ 50K records per week

http://goc.grid-support.ac.uk//

EGEE’03, April 2005 - 15

Whats New?

Added LHC View to the reporting interface• Requirements driven by RRB / Kors Bos• Tier-1 and Country entry points• LHC VO only• All data normalised in units of 1000 . SI2000 . Hour• Tabular Summaries per Tier1/ Country

Breakdown of Usage per Country / Tier1

Tier-1 Resource Usage per LHC VO

Graphical Plot showing Usage per VO

Select Tier1 View / Country View using navigable tree

Select time frameDrill Down to view data belonging to individual sites within a Tier1/country

EGEE’03, April 2005 - 17

Batch System Support

• APEL supports PBS and LSF• Implementations are separate and independent of

one another. Currently LCG2_4_0 has PBS support only.

• What is the current status about LSF Support?• http://goc.grid-support.ac.uk/gridsite/accounting/lsf_dev.

html• On this page you find release candidate

• edg-rgma-lsf-1.0.0-4.noarch.rpm

• LSF currently comes in three flavours (4, 5 and 6), each has a different usage record format

• New RPM edg-rgma-apel-lsf has been released to CERN for testing.

• Expect both to be part of LCG 2_5_0

EGEE’03, April 2005 - 18

CERN Data

Meeting with Harry and Thorsten (17th May)• Run APEL-LSF over full 2005 dataset• Compare APEL numbers to internal LSF reporting

EGEE’03, April 2005 - 19

LSF Deployment at CERN

1. PBS and LSF implementations use the same APEL core. This has some consequences for the normalisation model.

1. We use a global parameter to normalise data belonging to each site which is published by the CE.

2. PBS uses internal normalisation model, lsf has no such model3. Two ways to perform nomalisation in lsf:

1. normalise record by record (exact but requires converting a cpu factor into an SI00 value) or

2. globally (approximate method) as is done with PBS.

2. CERN has a heterogeneous farm. They calculate a weighted mean on a daily basis and publish this value on the CE (via ldif : GlueHostBenchMark00)

3. Accuracy of Method1. We will compare numbers with CERN very shortly.

EGEE’03, April 2005 - 20

Accounting Dissemination

1. CERN Courier 2. LCG Computing

Newsletter (slightly more technical)

3. AHM 2005 (more technical still)

EGEE’03, April 2005 - 29

APEL and gLite

Is APEL integrated in g-Lite?• Work currently in progress.• We have ported the APEL code into the gLite CVS repository but need

to understand functional differences e.g. WMS and use of Condor• 3 Components: Core + PBS plugin + LSF plugin• Sent our requirements to Erin Laure….waiting for information.

What about its deployment plan? • As soon as possible• …but would also like to add some new features

– Global Job ID to link with L&B – dn to VO mapping

EGEE’03, April 2005 - 30

LCG Accounting

Project involves combining results from all three infrastructures and presenting an aggregated view

Peer Infrastructures in LCG• Open Science Grid (Ruth Pordes, Philippe Canal, Matteo Melani)• Nordugrid (Per Oster)• EGEE• Currently, LHCView filters LHC VO data from EGEE accounting data.

EGEE’03, April 2005 - 31

Requirements

Combine results from all three infrastructures …

Ideally: Distributed queries to multiple databases• Each peer manages an accounting database• LHC VO filtering provided through a web services interface

Initial Implementation: Centralised Collection• Peers publish data into a global database• WebServices or direct MySql inserts

Common Problem: Different Grid infrastructures may use different Schemas. GGF define a schema, but quite flexible.

May need “translators” to convert from one schema to another. (already exist)