opennebulaconf2017eu: fairshare scheduling by valentina zaccolo, indigo
TRANSCRIPT
![Page 1: OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO](https://reader031.vdocuments.mx/reader031/viewer/2022030317/5a65c66f7f8b9ab3488b505d/html5/thumbnails/1.jpg)
FairShare Scheduling for OpenNebula: architecture and performances Stefano Bagnasco, Sara Vallero and Valentina Zaccolo Istituto Nazionale di Fisica Nucleare - Torino
1 INDIGO - DataCloud receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement RIA 653549
CHEP 2016 - October 10/14 - San Francisco (CA)
Vallero Sara - Bagnasco Stefano
INDIGO - DataCloudBetter Software for Better Science
A FairShare Scheduling Service for OpenNebula
INDIGO - DataCloudBetter Software for Better Science
INDIGO – DataCloud Better Software for Better Science
![Page 2: OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO](https://reader031.vdocuments.mx/reader031/viewer/2022030317/5a65c66f7f8b9ab3488b505d/html5/thumbnails/2.jpg)
Valentina Zaccolo – OpenNebulaConf 2
Why scheduling fairly? Large public cloud
• approximately infinite resources • tenants are charged a posteriori ! applications scale in/out freely
Small scientific datacenter
• saturated regime • tenants are charged a priori ! advanced resource allocation needed
FairShare Scheduling (FaSS) service https://github.com/indigo-dc/one-fass task priorities assigned according to
• v1.0 released with ElectricIndigo in April 2017 with ONE patch • v1.2 ONE 5.4 compatibility! check installation and usage demo: https://vimeo.com/216656868, pass: fass
Default OpenNebula (ONE) scheduler is FIFO and based only on static resources partitioning among the projects " not suitable for scientific DC
initial weight
historical resource usage
![Page 3: OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO](https://reader031.vdocuments.mx/reader031/viewer/2022030317/5a65c66f7f8b9ab3488b505d/html5/thumbnails/3.jpg)
Valentina Zaccolo – OpenNebulaConf 3
High-level architecture
FIFO SCHEDULER
XML-RPC INTERFACE
fassd
RPC PM TERMINATOR
asynchronous modules
- kills running VMs - only handles queue
FaSS - stateless (no duplicated info in DB)
![Page 4: OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO](https://reader031.vdocuments.mx/reader031/viewer/2022030317/5a65c66f7f8b9ab3488b505d/html5/thumbnails/4.jpg)
Valentina Zaccolo – OpenNebulaConf 3
High-level architecture
FIFO SCHEDULER
XML-RPC INTERFACE
fassd
RPC PM TERMINATOR
asynchronous modules asks queue
queue
- reorders queue - kills pending VMs (synchronous)
- kills running VMs
influxDB - nosql - holds internal data (initial priorities, historical usage, recalculated priorities)
Algorithms - MultiFactor
FaSS - stateless (no duplicated info in DB)
- only handles queue
![Page 5: OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO](https://reader031.vdocuments.mx/reader031/viewer/2022030317/5a65c66f7f8b9ab3488b505d/html5/thumbnails/5.jpg)
Valentina Zaccolo – OpenNebulaConf 3
High-level architecture
FIFO SCHEDULER
XML-RPC INTERFACE
FaSS fassd
RPC PM TERMINATOR
asynchronous modules
queue - reorders queue - kills pending VMs (synchronous)
- kills running VMs
influxDB - nosql - holds internal data (initial priorities, historical usage, recalculated priorities)
sends queue
Algorithms - MultiFactor
- stateless (no duplicated info in DB)
- only handles queue
![Page 6: OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO](https://reader031.vdocuments.mx/reader031/viewer/2022030317/5a65c66f7f8b9ab3488b505d/html5/thumbnails/6.jpg)
4
Performances Test with three users: initial shares configured in /etc/fass/shares.conf : [huey] uid=4 gid=1 share=50
[louie] uid=6 gid=1 share=25
[dewey] uid=5 gid=1 share=25
Periodically instatiates VMs for all 3 users
Stops instantiating Louie Reaches configured shares
Instantiates all 3 users again
![Page 7: OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO](https://reader031.vdocuments.mx/reader031/viewer/2022030317/5a65c66f7f8b9ab3488b505d/html5/thumbnails/7.jpg)
4
Performances Test with three users: initial shares configured in /etc/fass/shares.conf : [huey] uid=4 gid=1 share=50
[louie] uid=6 gid=1 share=25
[dewey] uid=5 gid=1 share=25
Periodically instatiates VMs for all 3 users
• configurable TTL per user • configurable terminator action • test different algorithms • …
Stops instantiating Louie
Instantiates all 3 users again
Reaches configured shares