tony doyle [email protected] executive summary, gridpp oc meeting, mrc, 14 october 2002

Tony [email protected]

“Executive Summary”, GridPP OC Meeting, MRC, 14 October 2002

Tony Doyle - University of Glasgow

Excutive SummaryExcutive Summary

• Introduction

• Project Management

• Resources

• CERN

• DataGrid

• Applications

• Tier-1/A

• Tier-2

• Dissemination

• Future Funding

• the Grid is starting to work

• Major: via Project Map now under

control

• Small modifications

• Manpower OK. Hardware

• Making impact

• Engaged (posts filled + value added)

• Tier-A production mode

• Latent resources

• UK flagship project

• Preliminary planning


Update Update (to 10/10/02)(to 10/10/02)

DataGrid Q7 Update

• ATLAS taskforce– v. useful, ongoing, other

experiments

• Releases– Avoid “Big Bang” approach– Separate “Prod” & “Dev”

• 2nd EU Review– 1 month earlier

• Active in FP6 planning

• Domestic– Q7 delivered~150% contract– No major issues

Applications Q7 Update• Review of Project Map

Milestones

(DB, PC, NB – 9/10/02)– No major issues

– Feedback to experiments initiated

• GridPP-LCG-DataGrid applications overlap

• e.g. persistency framework• e.g. future milestones

(via change notice)Q7 Cumulative Effort Effort

Funded Unfunded Total Effort Effort Delivered Delivered WP Title Effort Effort Effort Expectation Shortfall % %

1 Workload Management 0.0 0.0 0.0 1.5 1.5 0.0% 55.6% 2 Data Management 0.0 11.2 11.2 4.5 -6.7 248.9% 99.3% 3 Monitoring Services 9.0 16.1 25.1 14.4 -10.7 174.3% 108.2% 4 Fabric Management 0.0 3.0 3.0 1.5 -1.5 196.7% 206.7%

5 Mass Storage Management 4.0 11.4 15.4 7.5 -7.9 205.3% 92.6% 6 Testbed 0.0 11.0 11.0 9.0 -2.0 122.2% 125.0% 7 Network 0.0 0.0 0.0 6.0 6.0 0.0% 105.8%

TOTALS 13.0 52.7 65.7 44.4 -21.3 147.9% 123.0%


GridPP OverviewGridPP Overview

EDG - UK Contributions

ArchitectureTestbed-1Network MonitoringCertificates & SecurityStorage Element R-GMALCFGMDS deploymentGridSiteSlashGridSpitfire…

Applications (start-up phase)

BaBarCDF/D0 (SAM)ATLAS/LHCbCMS(ALICE)UKQCD

£17m 3-year project funded by PPARC through the e-Science Programme

CERN - LCG (start-up phase)

funding for staff and hardware...CERN

DataGrid

Tier - 1/A

ApplicationsOperations

http://www.gridpp.ac.uk


£17m++ 3-Year Project£17m++ 3-Year Project

• Five components– Tier-1/A = Hardware + CLRC ITD Staff

– DataGrid = 15 DataGrid Posts + CLRC PPD Staff

– Applications = 13 Experiments Posts (to interface middleware)

– Operations = Travel (~100 people)+ Management + Early Investment

– CERN = 25 LCG posts + Tier-0 + LTA

6/Oct/2002

£3.79m

£5.67m

£3.67m

£2.08m£1.79m

CERN

DataGrid

Tier - 1/A

ApplicationsOperations


Financial ProfileFinancial Profile

TOTAL FY2001 FY2002 FY2003 FY2004

CERN £5,666,600 £21,625 £1,813,784 £1,923,647 £1,907,544

DataGrid £3,786,302 £591,177 £1,209,807 £1,278,227 £707,092

Tier-1/A £3,651,371 £888,000 £1,117,987 £1,391,399 £253,986

Applications £2,066,018 £62,817 £659,911 £749,032 £594,257

Operations £1,829,647 £473,060 £519,346 £539,566 £297,675

OC2 TOTAL £16,999,938 £2,036,679 £5,320,835 £5,881,871 £3,760,553

Estimated Breakdown by Financial YearProject Summary

TOTAL FY2001 FY2002 FY2003 FY2004

CERN £5,666,600 £0 £1,239,021 £2,282,111 £2,145,468

DataGrid £3,786,302 £591,177 £1,209,807 £1,278,227 £707,092

Tier-1/A £3,674,009 £888,000 £742,987 £1,766,399 £276,624

Applications £2,077,645 £91,846 £661,304 £738,557 £585,938

Operations £1,794,159 £443,060 £507,695 £523,688 £319,716

OC3 TOTAL £16,998,715 £2,014,083 £4,360,813 £6,588,981 £4,034,838

Estimated Breakdown by Financial YearProject Summary

Re-profile of CERN contribution and Tier-1/A increase in FY2002


Financial BreakdownFinancial Breakdown

CERN TOTAL FY2001 FY2002 FY2003 FY2004

Hardware £1,245,800 £0 £300,000 £425,000 £520,800

Staff £4,420,800 £21,625 £1,513,784 £1,498,647 £1,386,744

TOTAL £5,666,600 £21,625 £1,813,784 £1,923,647 £1,907,544

Tier - 1/A TOTAL FY2001 FY2002 FY2003 FY2004

Hardware £2,438,000 £888,000 £775,000 £775,000 £0

Staff £1,213,371 £0 £342,987 £616,399 £253,986

TOTAL £3,651,371 £888,000 £1,117,987 £1,391,399 £253,986

CERN TOTAL FY2001 FY2002 FY2003 FY2004

Hardware £1,245,800 £0 £0 £725,000 £520,800

Staff £4,420,800 £0 £1,239,021 £1,557,111 £1,624,668

TOTAL £5,666,600 £0 £1,239,021 £2,282,111 £2,145,468

Tier - 1/A TOTAL FY2001 FY2002 FY2003 FY2004

Hardware £2,438,000 £888,000 £400,000 £1,150,000 £0

Staff £1,236,009 £0 £342,987 £616,399 £276,624

TOTAL £3,674,009 £888,000 £742,987 £1,766,399 £276,624

Reduced Spend in FY2002(Increase in FY2003-04)


Project ManagementProject Management - 7 Elements- 7 Elements


Summary (by Project Map areas)Summary (by Project Map areas)

• Grid success is fundamental for PP

1. CERN = LCG, Grid as a Service.

2. DataGrid = Middleware built upon Globus and Condor-G. Testbed 1 deployed.

3. Applications – complex, need to interface to middleware.

LHC Analyses – ongoing feedback/development.

Other Analyses have immediate requirements. Integrated using Globus, Condor, EDG/SAM tools

4. Infrastructure = Tiered computing to the physicist desktop:

Scale in UK? 1 PByte and 2,000 distributed CPUs

GridPP in Sept 2004 5. Integration = ongoing with UK e-

science…6. Dissemination• Co-operation required with other

disciplines/industry7. Finances – under control, but need to

start looking to Year 4..• Year 1 was a good starting point.

First Grid jobs have been submitted.. • Looking forward to Year 2. Web

services ahead.. but• Experiments will define whether this

experiment is successful (or not)


LCG CreationLCG Creation

LCG Phase 1 Personnel funded from Special ContributionEffective FTEs (weighted by experience)

Only identified personnel shown

0

10

20

30

40

50

60

70

2001 2002 2003 2004 2005Years

FT

E

(wei

gh

ted

by

exp

erie

nce

)

EU

USA

CERNMat

Sweden

Israel

Hungary

Portugal

Switzerland

Spain

France

Germany

Italy

UK

Requested


DisseminationDissemination(to other e-scientists)(to other e-scientists)

e-Science ‘All Hands’ Meeting held at Sheffield, 2-4 September 2002– ~ 300 people in total– ~ 19 GridPP People attended– ~ 13 GridPP ‘Abstracts’ accepted (total ~100)– ~ 10 GridPP Posters displayed– 4 GridPP Invited talks– 3 GridPP Demonstrations

GridPP Web Page Requests


GridPP PostersGridPP Posters

ATLAS SAM

OptorSimGridPP Tier-1/A ScotGrid

BaBarLHCbCMS

Storage

10 Posters for NeSC Opening and e-Science All Hands Meeting


TestBed StatusTestBed Status10 Oct 2002 15:4510 Oct 2002 15:45


Tier-1/A: Year of GrowthTier-1/A: Year of Growth

CSF Linux Weekly CPU Use

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

100000

110000

120000

130000

140000

01

/01

/20

01

05

/02

/20

01

12

/03

/20

01

16

/04

/20

01

21

/05

/20

01

25

/06

/20

01

30

/07

/20

01

03

/09

/20

01

08

/10

/20

01

12

/11

/20

01

17

/12

/20

01

21

/01

/20

02

25

/02

/20

02

01

/04

/20

02

06

/05

/20

02

10

/06

/20

02

15

/07

/20

02

19

/08

/20

02

23

/09

/20

02

No

rma

lise

d P

45

0 C

PU

Ho

urs

P450 cpu hours

CSF Linux Accounts : Since April 2002

delphiuk4%

lhcb1%

atlas13%

antaresw5%

theory14%

cms2%

h18%

bfactory46%

sno3%

zeus4%

020406080100120140

Oct-01

Dec-01

Feb-02

Apr-02

Jun-02

Aug-02

Personal

Server

GridPP Certificates

BaBar Use


Tier-1/A developmentTier-1/A development

In the next six months the main targets will be:

• Integration within the Grid (or Grids).

• Improving monitoring and error-handling.

• Developing support structure including longer hours of cover.

• Optimising the service for LHC Data Challenges.

• Completing next year’s hardware procurement.

• Migration to UK e-Science CA (start October 30th)

• Beyond this, we recognise support of the Tier-1/A centre as our highest priority for future funding


Tier-1 and -2 Centre ResourcesTier-1 and -2 Centre Resources

Estimated resources at end of 2003 (from Institute returns)

Number of CPUs

Amount of Disk (TB)

Amount of Tape (TB)

London 1252 76 8

Northern 2015 113 33

Southern 392 29 20

ScotGrid 154 35 0

Number of CPUs

Amount of Disk (TB)

BaBar 1108 29% 76 30%

CDF 453 12% 39 15%

D0 320 8% 8 3%

ATLAS 947 25% 72 28%

CMS 304 8% 9 4%

LHCb 543 14% 40 16%

Shared Distributed Resources 2003

5/16

2/16

1/16

4/16

1/16

2/16BaBar

CDF

D0

ATLAS

CMS

LHCb

Tier-1

• Tier-1: 600 CPUs + 150 TB• Tier-2: (4000 CPUs + 200 TB)


Tier-2 SupportTier-2 Support

• We have:• Testbed rollout plan (GridPP-TB-02-UKRollout)• Testbed Support Mailing List and weekly phone conference• Experienced System Managers creating a Grid and

responding to a rapid Grid deployment cycle• A (Rolling Grant) environment where System Managers are

not the highest priority• We require:• Four additional System Manager posts in third year (within

current resources)• Higher priority for deployment and support of Tier-2 centres

(with future resources)• Fabric Management Tools (GridPP and others) leading to

reduction in System Manager workload


GridPP –GridPP – Achievements and Issues Achievements and Issues

• 1st Year Achievements• Complete Project Map

– Applications: Middleware: Hardware

• Fully integrated with EU DataGrid, LCG and SAM Projects

• Rapid middleware deployment /testing

• Integrated US-EU applications development e.g. BaBar+EDG

• Roll-out document for all sites in the UK (Core Sites, Friendly Testers, User Only).

• Testbed up and running at 16 sites in the UK

• Tier-1 Deployment• 250 GridPP Certificates issued

(personal + server)• First significant use of Grid by an

external user (LISA simulations) in May 2002

• Web page development (GridSite)

• Issues for Year 2• Status: 10 Oct 2002 15:45 GMT

– monitor and improve testbed deployment efficiency short (10 min) and long-term (monthly)…

• Importance of EU-wide development of middleware and integration with US-led approach

• Integrated Testbed for use/testing by all applications

• Common “integration” layer between middleware and application software

• Integrated US-EU applications development

• Tier-1 Grid Production Mode• Tier-2 Definitions and Deployment• Integrated Tier-1 + Tier-2 Testbed• Transfer to UK e-Science CA• Integration with other UK projects

e.g. AstroGrid, MyGrid…• Publication of YOUR work


GridPP User InterfaceGridPP User InterfacePulling it all together…Pulling it all together…

GridPP Web Links 2003?: Demonstration elements: Gridsite, GUIDO, R-GMA,…Short and Long-Term Monitoring,Local and WAN Monitoring.


SWOT AnalysisSWOT Analysis

• Strengths– Excellent people, sense of community, well resourced, well

respected, integrated into EU development, significant role in US development, good website

• Weaknesses– Slow to get going, have not convinced the UK PP community

yet, limited industry involvement

• Opportunities– Further funding, potential for wide impact (talks, demos etc),

sharing of resources by all experiments

• Threats– EDG software delays, UK PP community no longer support us


UK Grid Future PrioritiesUK Grid Future Priorities

• Continue supporting Tier-1/A staff• Continue ramp up of Tier-1/A hardware

– UK leading role in computing infrastructure for LHC

• Support for Tier-2 staff– Complementary to University funding

• Continued support for Applications development– Beyond middleware data-handling interfaces?– As part of rolling grant programme?

• Continued support for Middleware development– including Framework VI EGEE project [DataGrid’]

• Ramp up of Tier-2 hardware– available via University SRIF/JREI/future funding?

• Continued support of CERN staff• Continued ramp up of CERN hardware

ALL of

theseare

required to

address the LHC

Computing Challenge

tony doyle [email protected] executive summary, gridpp oc meeting, mrc, 14 october 2002

Documents

tony doyle university

glasgow tier

glasgow summary

cern datagrid tier

glasgow project management

glasgow update

glasgow dissemination

fy2002 slide