5. data manager 1. introduction 2. data manager duties 3. quality checking 4. problem reporting 5....

No Slide Title

5. Data Manager15. Data ManagerIntroductionData Manager DutiesQuality CheckingProblem ReportingData MonitoringHistogram PresenterTrend PresenterEvent DisplayRun & File StatusProblem DatabaseLogbook

R. JacobssonIntroductionThe LHCb Shift Crew training consists of several coursesIntroductionSafety Basic ConceptsRunning LHCbData ManagerShift Leader

All of these course are subjects to changes

This course is intended to describe the work of the Data ManagerTasksTools

Judging data quality is a difficult but VITAL task requiring experienceConsult with people for any anomaly you observe or doubt you haveMake sure you are aware of the known problems when arriving for a batch of shiftsYour job is the basis for rating the files/runs for physics analysis!

Please comment on missing or unclear topics, errors. Dont hesitate to ask questions25. Data ManagerIntroductionData Manager DutiesQuality CheckingProblem ReportingData MonitoringHistogram PresenterTrend PresenterEvent DisplayRun & File StatusProblem DatabaseLogbook

R. JacobssonData Manager DutiesAssist Shift LeaderWatch his displays, learn also how to monitor the DAQ and Slow Controls, Detector Safety System Follow experimental conditions on permanent displaysCalling expertsIn case of crisis, like Safety Alarm or DAQ troubles, the DM can replace the SL at the Run Meeting

As Data Manager you will mainly use the following toolsHistogram PresenterTrending ToolEvent DisplayRun DatabaseProblem Database (Report problems together with expert if possible)Repository for problems affecting the data qualityElog

Make sure you feedback anything that is not understandable or unclear, absolutely vital Dont ignore something because you dont understand it!Contact Online Data Quality Coordinator (Olivier Callot) or Run Coordinator daytime35. Data ManagerIntroductionData Manager DutiesQuality CheckingProblem ReportingData MonitoringHistogram PresenterTrend PresenterEvent DisplayRun & File StatusProblem DatabaseLogbook

R. JacobssonData Quality CheckingThis is of vital importance during the ramp and adjust for physics as some detectors are switched on already and during the first minutes of a Physics fill, as you can immediately detect major disasters before the automatic tasks check the data

During Physics RunsLook at histograms by regularly browse through the pages in the Shift folder pages in the Histogram Presenter to spot abnormal distributions as soon as possibleNon working part of a detector, noisy channel are easyStrange distribution, in particular in the trigger, are less easy.

Look at the recorded events on the Event Display. Spot basic problems such as large background, or parts of the detector not responding or noisy.

Look at trends of rates and detector conditions using Trending ToolShift folder with well-documented trends not yet in place

End of RunA Run might be restarted for a variety of reasons

Check writing of files and data transfer to Offline and Castor (permanent storage) using the Run Database display If there are problems, contact (after checking with your SL) the offline control room where a production shift is monitoring the data processing.45. Data ManagerIntroductionData Manager DutiesQuality CheckingProblem ReportingData MonitoringHistogram PresenterTrend PresenterEvent DisplayRun & File StatusProblem DatabaseLogbook

R. JacobssonProblem ReportingMake sure you dont ignore a problem because you dont understand itDont be afraid to ask: Gas flow seems to be stopped in the VELO?!....

Reporting stepsConsult with Shift Leader and other experts that may be aroundMake sure it is not a know problem or a consequence of some special run settingsContact the relevant expert including Run Chief to discuss the severity and how to solve itMake an entry in the logbook preferably with a screen shotRegister problem in Problem Database with link to ELOG entry Reporting from Presenter allows both making an ELOG entry with screen shot of the erroneous plot and an entry in the Problem Database with your comment automatically.If problem just occurred, restart Run (CHANGE_RUN) as soon as possible to isolate the problem as much as possible55. Data ManagerIntroductionData Manager DutiesQuality CheckingProblem ReportingData MonitoringHistogram PresenterTrend PresenterEvent DisplayRun & File StatusProblem DatabaseLogbook

R. JacobssonData MonitoringMonitoring jobs run on dedicated Monitoring Farm/Reconstruction FarmEvent Spying is on best effort, not all events pass thereMonitoring tasks can crash/hangNo entries at all in a histogram is not necessarily bad dataConsult with Shift Leader

6

5. Data ManagerIntroductionData Manager DutiesQuality CheckingProblem ReportingData MonitoringHistogram PresenterTrend PresenterEvent DisplayRun & File StatusProblem DatabaseLogbook

R. JacobssonHistogram PresenterAllows you to view histograms produced by the various tasks. There is normally a copy running permanently, on a dedicated terminal, for regular checks.

Presenter is started with the command presenter.Presenter should normally run in the ONLINE mode. It can also run in a HISTORY mode, displaying histograms from a given run, fill or time interval.Be sure you are in the ONLINE mode...Histograms in the Shift folder has descriptions and instructions

Histograms should be compared to the reference histograms bearing in mind factors such asRecent and temporary faults that have appeared since the creation of the reference histograms. Known detector problems Any run specific problems which may temporarily alter the appearance of the histograms such as high backgrounds from LHC, etc.

Learning how to use the Presenter will be one of the main goals of your first shifts.


R. JacobssonLogin to plus /group/online/presenter/run_presenter.sh

Histogram Presenter8

Load previous/Next pageStart/stop page refreshReport to LogClear histogramsPartition


R. JacobssonHistogram Presenter9


R. JacobssonHistogram PresenterReference histograms10


R. JacobssonHistogram PresenterExample of problem registered in ProblemDB as OK11


R. JacobssonHistogram PresenterExample of problem reported in ProblemDB as Severe12


R. Jacobsson(Trend Presenter)No Shift folder yet with documented trends13


R. JacobssonEvent DisplayUseful for checking the coherence of the complete detector response. Check for noise in the sub-detectors

Started on a plus node with the display on a dedicated terminal with command evdisp

The standard display mode is AUTOMATIC, i.e. a new event is displayed every 5 seconds. You can then click on the pause button to keep any specific event. You can also select a MANUAL mode where you click on a button to see the next event. There are options to select events to be displayed by type, trigger mask, etc...

Be sure that the display is running in ONLINE mode during data taking. Report all problems and suggestions to Philippe Ghez145. Data ManagerIntroductionData Manager DutiesQuality CheckingProblem ReportingData MonitoringHistogram PresenterTrend PresenterEvent DisplayRun & File StatusProblem DatabaseLogbook

R. JacobssonEvent DisplayScreen shots!155. Data ManagerIntroductionData Manager DutiesQuality CheckingProblem ReportingData MonitoringHistogram PresenterTrend PresenterEvent DisplayRun & File StatusProblem DatabaseLogbook

R. JacobssonRun and File Status (RunDB)Check that the real runs are closed and sent to Offline16


R. JacobssonProblem DBWeb link for Problem DB: http://lbproblems.cern.ch/

It consists of a list of problems potentially affecting data quality

Title is a short text, used later to list problems and should be concise and meaningful.

System is selected from a predefined list and is mainly the list of LHCb sub-systems.

Severity is selected from a predefined list, and defaults to 'unknown'.

Time Range is defined as time

Comment is text information including Author and optional Weblink. This is a write-once comment in the sense that the text can't be modified, only a new text can be added to correct it. Be careful and proofread your input before committing. Note that the database can be edited by managers in case of serious need. Please contact Olivier Callot, Jose Caicedo or Run Coordinator

History of entries displayed


R. JacobssonProblem DB18






R. JacobssonProblem DBWeb link to logbook from Problem Report21






R. JacobssonLogbook Entry24


R. JacobssonConclusionsAlright, you are now graduated Data Managers!

Thanks for volunteering! You are invaluable for Our Physics Goal!


R. Jacobsson

5. data manager 1. introduction 2. data manager duties 3. quality checking 4. problem reporting 5....

Documents

data transfer

data qualityelog

data processing

data manager15

detector conditions

known problems

spot basic problems

vital dont