fax fdr results
DESCRIPTION
FAX FDR results. 28 th January 2013. For reference. Indico page: https:// indico.cern.ch / conferenceDisplay.py?confId =229966 Twiki : https:// twiki.cern.ch / twiki /bin/ viewauth /Atlas/ JanuaryFDR ML based monitor at SLAC: http://atl-prod07.slac.stanford.edu:8080/ display - PowerPoint PPT PresentationTRANSCRIPT
FAX FDR RESULTS
28TH JANUARY 2013
Ilija Vukotic [email protected] 2
FOR REFERENCE• Indico page: https://indico.cern.ch/
conferenceDisplay.py?confId=229966• Twiki: https://twiki.cern.ch/twiki/bin/viewauth/Atlas/
JanuaryFDR• ML based monitor at SLAC:
http://atl-prod07.slac.stanford.edu:8080/display• CERN dashboard: http://dashb-atlas-xrootd-
transfers.cern.ch/ui/#• WAN HC based tests:http
://ivukotic.web.cern.ch/ivukotic/WAN/index.asp• FDR dedicated test submission page: http://
ivukotic.web.cern.ch/ivukotic/FDR/index.asp
Ilija Vukotic [email protected] 3
ENDPOINTS• From AGIS• Two more sites added during
the week: Lancaster and Liverpool
• RALPP and Cambridge joined but still testing against them
• PIC should join shortly• All the endpoints showed very
high availability during the week.
AGLT2
BNL-ATLAS
BU_ATLAS_TIER2
CERN-PROD
DESY-HH
INFN-FRASCATI
INFN-NAPOLI-ATLAS
INFN-ROMA1
JINR-LCG2
LRZ-LMU
MPPMU
MWT2
OU_OCHEP_SWT2
PRAGUELCG2
RAL-LCG2
RU-PROTVINO-IHEP
SWT2_CPB
UKI-LT2-QMUL
UKI-NORTHGRID-LANCS-HEP
UKI-NORTHGRID-LIV-HEP
UKI-SCOTGRID-ECDF
UKI-SCOTGRID-GLASGOW
UKI-SOUTHGRID-OX-HEP
WT2
Ilija Vukotic [email protected] 4
LINKS• WAN HC tests are continually testing full mesh of links. • That’s used for SSB cost matrix.
Ilija Vukotic [email protected] 5
LINKS
• But not all of the links worked due to authorization issue• Plot shows situation ten days ago.
Ilija Vukotic [email protected] 7
INPUT DATA
• Week started without any input data that could be used and confusion concerning dataset and file naming.
• Federica kindly provided both a list of datasets to be used (SUSY and SMWZ) and a script to replicate them automatically.
• dq2-put/dq2-get combination proved to be inadequate for task at hand.
• We decided to start with just the first dataset of SUSY sample: 19 files of 68.5 GB.
• Hiro made transfer requests to all of the US sites and than manually re-registered them.
• Simone did equivalent thing for CERN and 3 Italian sites, than continued with the DE + RU + UK cloud
• Wahid used get/put method for all of the UK sites but only the first dataset.
Ilija Vukotic [email protected] 8
INPUT DATA
• Current distribution status of the first SUSY dataset
Complete Still at CERN_PRODLancaster BNL-ATLAS Frascati
Liverpool AGLT2 IHEP
Oxford DESY-HH JINR
ECDF LRZ-LMU Prague
QMUL MPPMU
Glasgow CERN
Roma
Napoli
Ilija Vukotic [email protected] 9
WEEK PRIOR TO FDRBlue and light blue are MWT2 mostly tier3 users using tier2 data through FAX
Stream of ~100-150MB/s of testing data. Peaks every 30 min.
On 11th Jan just sent a bit more tests.
Ilija Vukotic [email protected] 10
DURING FDR WEEKUniversity of Chicago users were using FAX from both Tier3 and grid jobs at Tier2.
Will remove it from further plots.
Ilija Vukotic [email protected] 11
DURING THE WEEK
Submitting jobs through the web site.All of the details down to PandaID’s can be found there.In short: HC jobs that were normally used for local site tests where changed so they contact Oracle DB at CERN and from there get info: which FAX endpoint to use, which files to use, what to do with them copy/read, how many jobs to do etc. Jobs upload back time, MB/s, ev/s.System is very easy to use, still some space for improvement:• It lets you try to use link which (currently) does not work.• To get more than two simultaneous jobs you have to make change in HC test. Even when you request 10 simultaneous jobs you easily end up with much less, depending on how fast are your jobs and how long is the client sites ANALY_* queue.Testing pattern: • reading from same site (10 files from 10 jobs ) • from site in same cloud (10 files from 10 jobs ) • from main regional site (CERN, BNL) (10 files from 10 jobs ) • from across the pond (10 jobs 1 file)
Ilija Vukotic [email protected] 12
DURING THE WEEKStarted low: copy and read from Roma1 Bandwidth measure
Real performance measureTo Napoli penalty 30% To CERN 50%Worst case scenario
Ilija Vukotic [email protected] 16
CONCLUSIONA huge progress in FAX readiness just before and during the week. THANK YOU ALL !Sites were testing was possible showed no problems in delivering data. Performance was fluctuating a lot, depending on other transfers to/from site. Fine spatial granularity cost matrix will be essential for some applications.We gained a lot of experience and surely next FDR will be much better. Could be made as soon as:• Authentication issue properly solved• Proper implementation of both copy-to-scratch and direct-access
modes• The test data will be available from the first day