web data quality analysis 1 web data quality analysis jonathan levell (summer student) working...
DESCRIPTION
Web Data Quality Analysis 3 Check Data Quality (DQ): During MC and DST production of test data After each detector modification Produce a standalone DQ program which calls checking routines supplied by subdetector groups Provide WDQA package to display and compare sets of histograms for each subdetectorTRANSCRIPT
![Page 1: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/1.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 1
WEBDATA QUALITY
ANALYSISJonathan Levell
[email protected](Summer Student)
Working with:• Eric van Herwijnen (EvH)• Agnieszka Jacholkowska (AJ)
![Page 2: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/2.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 2
TALK OUTLINE• Introduction
• How is the Data Quality Checking done?
• What is SICBCHK?
• What is WDQA?
• Results• Typical Data• CERN/MAP
• Summary
![Page 3: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/3.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 3
• Check Data Quality (DQ):
• During MC and DST production of test data
• After each detector modification
• Produce a standalone DQ program which calls checking routines supplied by subdetector groups
• Provide WDQA package to display and compare sets of histograms for each subdetector
![Page 4: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/4.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 4
SICBMC
SICBDST
SICBCHK
WDQA
RAWH
DST1/DST2
PostScriptGIFS
NTUPLE
HTML
![Page 5: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/5.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 5
• Fortran Program
• Separate from SICBMC/SICBDST (package called: SICBCHK)
• Split into branches for each subdetector
• Branches maintained individually
• Latest version installed in September
![Page 6: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/6.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 6
Branch Directory Contact PersonFrame+WDQA SICBCHK JL*, EvH, AJMuon Detector muon Karl HarrisonCalorimeter calo Ivan KorolkoGenerator gene Paolo BartaliniRICH Detector rich Guy WilkinsonTrigger trigger Hans DijkstraTracker trkfit Matthew NeedhamVertex Locator vertex Thomas RufVertex Trigger verttrig Thomas Ruf
* = JL leaves on 15th September
![Page 7: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/7.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 7
SICBCHK.F
UGINIT.F CHKRUN.F UGLAST.F
CHKTRIG.F
SUANAL.F
For each event
Initialisation Tidy Up
SUCHECK.F Subdetector RoutinesMiscellaneous
Checks
![Page 8: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/8.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 8
• Package controlled by .lmac scripts similar to PAW’s .kumac scripts
• Small Interpreter written in C++
• Based on ROOT
• Converts ntuple files to ROOT files
• LMAC scripts can call ROOT macros
• Easy to use
• Manual currently being written
• Eventually details in Computing meeting
![Page 9: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/9.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 9
// Conversion to ROOT files.hbookin ~/work/data/<<eventtype>>.hbookfile 1 hbookin ~/work/data2/<<eventtype>>.hbookopt nowaitoutputdir velops velo.psoutputdir velo/gifssettitle Event Type: <<eventtype>> (Vertex)zone 1 2set style defaultplot 1001set ymax 250set fillcolor 5file 1 overlay 1001 // Call standard ROOT macrorootpad fit2gauss.cgifs page1waitps closeclosefile 1 close
An Example .LMAC script
![Page 10: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/10.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 10
![Page 11: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/11.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 11
![Page 12: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/12.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 12
![Page 13: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/13.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 13
![Page 14: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/14.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 14
![Page 15: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/15.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 15
![Page 16: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/16.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 16
![Page 17: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/17.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 17
![Page 18: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/18.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 18
![Page 19: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/19.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 19
![Page 20: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/20.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 20
![Page 21: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/21.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 21
![Page 22: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/22.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 22
![Page 23: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/23.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 23
![Page 24: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/24.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 24
Quantity MAP CERN Approx.Difference
Num. Good 49.2± 55.3±2.2 3scharged tracks
Num Bad 238.5± 285±14.1 3scharged tracks
Num. Neutral 12.6±0.6 14.0± 3stracks
PGEN 201±6.8 213.2±6.3 2sMultiplicity
VSCP Entries 358.0± 387.6±13.0 3s
VSCR Entries 372.4± 402.3±13.5 3s
VRPR Entries 730± 79.0±27.0 3s
Are the differences significant?
![Page 25: Web Data Quality Analysis 1 WEB DATA QUALITY ANALYSIS Jonathan Levell (Summer Student) Working with:](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a4d1b797f8b9ab0599b83a6/html5/thumbnails/25.jpg)
Web Data Quality Analysis
http://lhcb.cern.ch/wdqa/ 25
• Bigger sample required before real conclusions can be drawn.
• Differences currently at 3slevel
• Larger sample (500 events) will be requested, if possible with same random seeds.
SICBCHK/WDQA still require more work but a newversion will be released in mid-September