all-in-one graphical tool for grid middleware management eddy caron, abdelkader amar, frédéric...
TRANSCRIPT
![Page 1: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/1.jpg)
All-in-one graphical tool for grid middleware management
Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro
LIP ENS Lyon, INRIA Rhône-Alpes, GRAAL project
http://graal.ens-lyon.fr
Join work with A. Cedeyn, R. Bolze, Y. Caniou, A. Chis, B. Depardon, J.-S. Gay, G. Lemahec …
![Page 2: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/2.jpg)
Summary
• Introduction• DIET short presentation• DIET DashBoard
DIET tools Workflow tools Grid tool : GRUDU
• Experiments• Conclusion and Future Work
![Page 3: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/3.jpg)
• Context : Development of a toolbox for deploying ap-plication services providers with a hierarchical archi-tecture for scalability
LA
MA
LA
LALA
Server front end
Master Agent
Local Agent
Client
MA
MA
MA
MA
Corba or JXTA
http://graal.ens-lyon.fr/DIET
• Main research issues: scheduling, heterogeneity, automatic deployment, interoperability, high performance data transfer and management, monitoring, fault tolerance, genericity of solutions for various applications, static and dynamic analysis of performance, …
• Validation: Large validation over Grid’5000.• Interoperability: DIET is compliant with the Grid-RPC
standardization for OGF • DIET used case: The Decrypthon project
- DIET was selected by IBM - • Collaborations: RNTL GASP, ACI GRID ASP, TLSE,
ACI MD GDS, ANR LEGO, ANR GWENDIA, Grid’5000
![Page 4: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/4.jpg)
DIET Dashboard
• DIET hierarchies can be deployed on grid/cluster of nodes• In Grid Environments, users need several and complex tools for the
management of resources and client/server applications.• Can be tougth to deploy a DIET hierarchy in a such a grid environment
by hand without high level tools
• The DIET DashBoard is a set of tools written in Java
• Provides to the DIET end-user, friendly-user interfaces to design, deploy and monitor the execution of client/server applications
• Also provides to the grid user tools for the allocation and monitoring resources on Grid'5000
![Page 5: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/5.jpg)
DIET Dashboard
• Extensible set of tools for the DIET community
• Based on seven tools: DIET designer DIET Mapping tool DIET Deployment tool XML GoDIETGenerator Workflow designer Workflow log service DIET resource tool aka GRUDU
DIET Tools
Workflow Tools
Grid Tools
![Page 6: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/6.jpg)
DIET Tools
Allows the user to design graphically a DIET hierarchy.
Allow the user to map DIET components on the allocated Grid'5000 resources
To help the end-user creating hierarchies from
existing frameworks based on the reserved
ressources.
It provides the basic GoDIET operations: open, launch, stop and monitoring for DIET application elements.
![Page 7: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/7.jpg)
Workflow Tools
• Applications viewed as services within DIET• Compose services to get a complete application workflow in a drag’&’drop fashion
To monitor workflows execution by displaying the DAG nodes of each workflow and their states. Three states are available: waiting, running and done. Two modes:local and remote
![Page 8: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/8.jpg)
• Context : Grid'5000 a highly reconfigurable, controlable and monitorable experimental Grid platform. Needed all-in-one GUI client-side tool for the monitoring of the Grid'5000 platform.
Grid'5000 Reservation Utility for Deployment Grid'5000 Reservation Utility for Deployment UsageUsage
• Main goals :
– Displaying the status of the platform (grid/site/job level)
– Resources allocation through the use of OAR (oar v1 & v2 are supported)
– Resources monitoring through the use of the Ganglia (site/job nodes)
– Deployment management with a GUI for KaDeploy (multiple sites at a time)
– A terminal emulator (access frontale/ site frontale/job main node connection)
– and a file transfer manager (local/remote and synchronization features)
• Web: http://graal.ens-lyon.fr/GRUDU
![Page 9: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/9.jpg)
Exp 1: Goals and Protocol of the Experiment
• Validation of the DIET architecture at large scale over different administrative domains
• Protocol DIET deployment over a maximum of processors Large number of clients Comparison of the DIET execution times with average local execution times 1 MA, 8 LA, 540 SeDs 2 requests/SeD 1120 clients on 140 machines DGEMM requests (2000x2000 matrices) Simple round-robin scheduling using
time_since_last_solve
Grid’5000
Paravent : 9 s
Lilles : 34 s Paraci : 11 s
Bordeaux : 33 s Parasol : 33 s
Sophia : 40 s Toulouse : 33 s
Lyon : 38 s Orsay : 40 s
![Page 10: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/10.jpg)
Exp 1 : Results Grid’5000
![Page 11: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/11.jpg)
Exp 2: Goals and Protocol of the Experiment• Validation of the DIET architecture at large scale over different
administrative domains in the framework of the LEGO project (ANR CICG05-11)
Grid’5000
Protocol Get the largest machine reservation
on the largest number of Grid'5000 clusters
Realize a maximum number of Ramses execution (application developed at the DAPNIA/CEA)
Stress DIET over a large number of machine and in a large period of time
But also stress Grid'5000 ... KaDeploy image with DIET and all
the mandatory tools 12 clusters on 7 sites : 979
machines for 48 hours 1 MA, 12 LA, 29 SeDs 816 machines Ramses-only
![Page 12: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/12.jpg)
Exp 2 : Results
• Comparing DAPNIA/CEA to DIET/DIET DashBoard/Grid'5000
DAPNIA/CEA Grid'5000Simulation time (h) 4464 (6 months) 28Processor number 32 1824Simulation number 50 33
Simulation Speed (simu/h) 0,0112 1,18
Ramses Example Output
![Page 13: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/13.jpg)
Conclusion and Future Work
• Conclusion : DIET is a grid middleware designed for scheduling application tasks
with a hierarchical architecture The DIET DashBoard provides to DIET users a full-featured framework
for experiments on Grid'5000 GRUDU provides to the Grid'5000 community a powerful tool
dedicated to the interaction with the grid (monitoring/reservation/deployment etc.)
The experiments show how DIET DashBoard/GRUDU can simplify the use of a grid middleware
• Future work: Support of other Batch Schedulers (like LoadLeveler or SGE) Interaction with other grids (EGEE, DAS3, etc.)
![Page 14: All-in-one graphical tool for grid middleware management Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro LIP ENS Lyon, INRIA Rhône-Alpes,](https://reader033.vdocuments.mx/reader033/viewer/2022060106/55191b5655034626428b4a69/html5/thumbnails/14.jpg)
Questions ...
Questions ?
Demoat the Exhibition Hall
16:00 - 20:00