LHCb resources updates for 2012-

2015

Marco CattaneoCERN – LHCb

On behalf of the LHCb Computing Group

2

LHCb computing production activities in 2012

m Prompt processing (reconstruction and stripping) of new data

m “Swimming” of 2011 stripped datam MC production

o Large samples for analysis of 2011 datao Preliminary samples for 2012 data

m Reprocessing of complete 2012 dataseto Started 17th September

m Resources OK until now:o CPU usage 50% of pledge

P As expected for first halfof yeard now ramping to >100% with

reprocessingo 2012 disk pledge ~sufficient

also for reprocessed dataP Active data management

o Major shortage of tape, already using full pledge at Tier1s!P Pledge based on 2012 request, assuming shorter runP Major problem for the future (rest of 2012, but also 2013)

3

Changes since 2012 request

m New reconstructed data format:o Previously, Reco output (SDST) did not contain copy of

RAWP Both RAW and SDST files had to be staged, synchronously,

when restrippingP Very demanding on tape systems, since in general on

different tapesP Does not allow to achieve maximum throughput

o Now write FULL.DST, containing RAW+SDSTP Effectively an extra copy of RAW on Tape

m Data taking changeso Extension of pp run by 2 monthso LHCb operation at higher instantaneous luminosity

P 4x1032 cm-2s-1 (3.7 in 2011)o Higher HLT output rate

P (2011: 3kHz, 2012 forecast: 4.5kHz, 2012 actual: 5kHz)o Participation in pA (heavy ion) run in early 2013

4

Effect of changes on computing resources

m RAW data: 1.7 PBo 43% increase with respect to previous estimates (1.2

PB).P Two copies on tape

m Reconstructed data (FULL.DST): o 3.1 PB, 120 % increase with respect the previous

estimate (SDST, 1.4 PB)P One copy on tape

m Derived datasets (DST, MDST for physics analysis)o 43% increase

P disk resident, plus tape archivem Heavy Ion RAW data: 100 TB

m Higher CPU peak power for reprocessingo Originally foreseen after data-taking, now in parallel

to be ready for Winter 2013 conferencesP Clashes with prompt processing (data quality, fast

analyses)P Cannot use resources of HLT farm

5

Mitigation to fit into 2012 pledges

m CPU:o Reduce prompt processing to 50% of new data

P Only at CERN, 20% sub-contracted to Tier2sP Only for data quality etc., no physics analyses using

these datao Reprocessing extended by two months

P Only at Tier1s, 20% sub-contracted to Tier2sP Only ~60% of 2012 data available for Winter 2013

conferencesm Tape:

o Removed NOW all output of 2012 prompt reconstructionP Needed for restripping

d No new analyses until reprocessing completedo Reduce all archives (data preservation) to one copy

m Disk:o Aggressive reduction of 2012 prompt DST/MDST

copiesP In parallel with appearance of reprocessed data

o New distribution policy taking into account unused pledgeP In practice, replicas of reprocessed data will go mostly

to CERN, RAL, CNAF, GRIDKA

6

Storage: forecast for March 2013

m Disk OK but big imbalance between sites (backup slide)

m Serious shortfall in tape, no solution yeto Already cleaned up all 2012 prompt SDST to fit in

existing tapeo No room at Tier1s for second copy of new RAW, and for

ONLY copy of FULL.DST

7

2013-2014

m Activities during 2013-2014o Several restrippings, ~every 6 months

P ~ neutral in disk usage since older strippings get replacedP moderate increase in tape (archive copy)

o Full reprocessing of 2011-2012 data, during 2014P Ultimate reprocessing of this datasetP Duration stretched to fit within existing CPU resourcesP ~neutral on tape (FULL.DST replaces previous version)

o Continuing MonteCarlo production at constant rateP Higher rate than previous estimates to match increased

real data sampled But no increase in Tier2 CPU request, use HLT farm instead

P Linear increase in disk and tapem Requests for 2013-2014 (details in backup slide)

o CPU: 20% increase at Tier1 in 2013, otherwise flato Disk: 20% increase per year at Tier1, 25% per year at

CERNo Tape: 100% increase in 2013 at Tier1, 10% in 2014

P Tape increase actually needed ASAP, April 2013 is too late.

8

Resources after LS1m Date of startup for physics, and number of physics days

in 2015P LHCb nominal luminosity reached rather quickly (lumi

levelling). Assume:d Any data taking before April 2015 has insignificant impact on 2014

resourcesd 5*106 seconds in 2015, at nominal data rates (i.e. ‘normal’ year)

m Changes in data-taking rate (not precisely predictable yet)

P Average multiplicity of pp collisionsd Higher than at 8 TeV

P Effect of pileup on RAW sized With 25ns bunch spacing, lower multiple interaction rate but greater

spillover from out of time eventsP Operating luminosity at LHCb IP

d Considering up to 50% higher than in 2012, may lead to higher HLT rate

P HLT bandwidth allocation to different physics channelsd To be studied during LS1, may lead to significant changes in rate

m Assume all above lead to doubling of data rate (MB/s) from HLT

P Currently limited by DAQ to 700 MB/sP Doubles also all derived (reconstruction, stripping) data

formatsP CPU time scaled with data volume

m 2015 equivalent to 2011+2012 combined.o Resources needs are ~doubled compared to 2014 (see

backup slide)

9

BACKUP

10

Resources imbalancem Evolution of disk pledges at Tier1s

m Changes balance of resources among siteso Preferentially clean old data from sites below shareo Available space used as weight for placing replicas of

new datam E.g. 2012+2011 reprocessed data distribution will

be:

m Consequence: analysis of 2011 and 2012 reprocessed data will not use all Tier1s according to their CPU share

11

2013-2014 requests (LHCb-PUB-2012-014)

m N.B. 2012 column is what is needed to complete processing of 2012 data, after all possible mitigations

12

First estimate of resources in 2015(LHCb-PUB-2012-015)