gangatasks automating of job submission wit athena/athenamc in ganga hepcg workshop, siegen,...
DESCRIPTION
automating of job submission in Athena/AthenaMC I. motivation, mc simul:5 recon: evgen:50 79 tariq mahmoud hepcg workshop, siegen, need automate job (re)submissionTRANSCRIPT
![Page 1: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/1.jpg)
GangaTasksGangaTasksAutomating of job Automating of job submission wit submission wit Athena/AthenaMC in GangaAthena/AthenaMC in Ganga
hepcg workshop, siegen, 13.12.2007hepcg workshop, siegen, 13.12.2007
johannes ebke, johannes elmsheuser, johannes ebke, johannes elmsheuser, tariq tariq mahmoudmahmoud
![Page 2: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/2.jpg)
tariq mahmoud hepcg workshop, siegen, 13122007
automating of job submission in Athena/AthenaMC
outline
I. introduction, motivation
II. concept of tasksIII. some illustration IV. conclusion
![Page 3: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/3.jpg)
automating of job submission in Athena/AthenaMC
I. motivation, mc
simul:5
recon:10
1
1 3 5 97
2 4 6 108
1
evgen:50
7 9
tariq mahmoud hepcg workshop, siegen, 13122007
3 5
3
6
3 5
63
3
6
5
6
3
3
3
need automate job
(re)submission
![Page 4: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/4.jpg)
automating of job submission in Athena/AthenaMC
I. motivation, mc
tariq mahmoud hepcg workshop, siegen, 13122007
1 3 5 972 4 6 108
1
1 9
7
7
7 1 3 5 972 4 6 108
1
1 7 9
36
3 5
633
5
56
566
6
![Page 5: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/5.jpg)
1 3 5 972 4 6 108
1
1 7
36
3 5
63
1 3 5 972 4 6 108
1
1 7 9
3 5
3 5
53
1 3 5 972 4 6 108
1
1 7 9
36
3 5
63756
1 3 5 972 4 6 108
1
9
7
7
3
5
55
1 3 5 972 4 6 108
1
1 7 9
36
3 5
63
automating of job submission in Athena/AthenaMC
I. motivation, mc
1 3 5 972 4 6 108
1
1 7 9
36
3 5
63756
1 3 5 972 4 6 108
1
1 9
7
7
3
5
55
1 3 5 972 4 6 108
1
1 7 9
36
3 5
63
1 3 5 972 4 6 108
1
1 9
7
7
3
5
5
6 7
4
4
4
4need automate job
(re)submission
need job chaining
(interconnection)
tariq mahmoud hepcg workshop, siegen, 13122007
![Page 6: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/6.jpg)
automating of job submission in Athena/AthenaMC
I. motivation, analysis
complete dataset
tariq mahmoud hepcg workshop, siegen, 13122007
A,B,C: f1,f2,f3, …,fn
A,B,C f1,f2,f3A,B,C f4,f5,f6
A,B,C f7,f8,f9A,B,C f7,f8,f9
A,B,C f4,f5,f6
A,B,C f7,f8,f9A,B,C fn-2,fn-1,fn
incomplete datasetA: f1,f2,f3B: f1,f5,f6,f7,f8,f9C: f1,f2,f7,f8,f9,f10
A,C f1,f2B f5,f6
A f3
B,C f7,f8,f9C f10
![Page 7: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/7.jpg)
automating of job submission in Athena/AthenaMC II. concept of tasks
taskfew parameters: task parameters: float, name, number of events etc.
process parameters: datasets, process name, geometry etc.
input/output specifications Athena version
AbstractJobgenerated automatically: name, task, status, necessary, prerequisites, run limit, number of attempts and attempt jobs, …
AbstractJobAbstractJob
jobjob
job Ganga jobs
tariq mahmoud hepcg workshop, siegen, 13122007
![Page 8: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/8.jpg)
automating of job submission in Athena/AthenaMC II. concept of tasks: GangaTasks
MCTask
MCJobMCJobMCJob
AnaTask
AnaJob
Task
AnaJobAnaJob
repository/tools.data
TaskList(GangaObject)
GPI.tasks
tariq mahmoud hepcg workshop, siegen, 13122007
![Page 9: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/9.jpg)
automating of job submission in Athena/AthenaMC III. illustration, resubmission
tariq mahmoud hepcg workshop, siegen, 13122007
automate job (re)submission
need job chaining
(interconnection)
![Page 10: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/10.jpg)
automating of job submission in Athena/AthenaMC
III. illustration: chaining task stepsmc.total_events = 200;
mc.events_per_job={"evgen": 50, "simul": 5, "recon":10}
1 3 5 7 9
1 3 5 7 9
1 3 5 7 9
1 3 5 7 9
1 3 5 972 4 6 108
1 3 5 972 4 6 108
1 3 5 972 4 6 108
1 3 5 972 4 6 108
2 3 41 evgen:i-0
simul:i-j
recon:i-k
tariq mahmoud hepcg workshop, siegen, 13122007
![Page 11: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/11.jpg)
mc.total_events = 200 mc.events_per_job= {"evgen": 50, "simul": 5, "recon":10} mc.float=10
AbstractJob.prerequisites()
tariq mahmoud atlas sw week, cern, 24102007
automating of job submission in Athena/AthenaMC III. illustration: interconnections,
prerequisites
automate job (re)submission
job chaining (interconnection)
the user sets few parameters … and gets
RESULTS
![Page 12: GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga hepcg workshop, siegen, 13.12.2007 johannes ebke, johannes elmsheuser, tariq mahmoud](https://reader034.vdocuments.mx/reader034/viewer/2022051010/5a4d1b897f8b9ab0599be19b/html5/thumbnails/12.jpg)
automating of job submission in Athena/AthenaMC
IV. conclusion
GangaTask package: automates job management (small scale production system)works with Ganga Athena and Ganga AthenaMC
(available from Ganga443 onwards) works at cernwiki with tutorial: https://twiki.cern.ch/twiki/bin/view/Atlas/GangaTasksapplicable to any job submission-software on the grid
To do: 1. implementing site-test-code 2. making use of GangaRobot3. act on user suggestionstariq mahmoud hepcg workshop, siegen, 13122007
Configure once – run anywhere