a proactive system to predict server utilization and dynamically allocate memory resources using

USENIX Association 26th Large Installation System Administration Conference (LISA ’12) 111

Bayllocator: A proactive system to predict server utilization anddynamically allocate memory resources using Bayesian networks and

ballooningEvangelos TasoulasUniversity of Oslo

Department of Informatics

[email protected]

Harek HaugerundOslo And Akershus University College

Department of Computer Science

[email protected]

Kyrre BegnumNorske Systemarkitekter AS

[email protected]

Abstract

With the advent of virtualization and cloud computing,virtualized systems can be found from small compa-nies to service providers and big data centers. All ofthem use this technology because of the many bene-fits it has to offer, such as a greener ICT, cost reduc-tion, improved profitability, uptime, flexibility in man-agement, maintenance, disaster recovery, provisioningand more. The main reason for all of these benefits isserver consolidation which can be even further improvedthrough dynamic resource allocation techniques. Out ofthe resources to be allocated, memory is one of the mostdifficult and requires proper planning, good predictionsand proactivity. Many attempts have been made to ap-proach this problem, but most of them are using tradi-tional statistical mathematical methods. In this paper,the application of discrete Bayesian networks is evalu-ated, to offer probabilistic predictions on system utiliza-tion with focus on memory. The tool Bayllocator is builtto provide proactive dynamic memory allocation basedon the Bayesian predictions, for a set of virtual machinesrunning in a single hypervisor. The results show thatBayesian networks are capable of providing good pre-dictions for system load with proper tuning, and increaseperformance and consolidation of a single hypervisor.The modularity of the tool gives a great freedom for ex-perimentation and even results to deal with the reactivityof the system can be provided. A survey of the currentstate-of-the-art in dynamic memory allocation for virtualmachines is included in order to provide an overview.

1 Introduction

Computers and ICT (Information and communicationstechnology) undoubtedly is emerging more and moreinto our lives. A Statistical research (2007) providedthat we would have 1 billion personal computers by2008 [18]. This was confirmed by another research in

June, 2008 [5] and both of them foresee that this numberwill be doubled by 2014-2015.

The adoption in the Enterprise increases as well asICT increases corporate productivity [1], but the com-panies obviously need to use non stop running servers.This leads to increased hardware purchase costs, as wellas peripheral costs and environmental damage comingfrom the power consumption of the servers and thecooling facilities [9]. If one considers that most of thetime (>70%) in typical deployments the servers are idle,the cost to operate underutilized servers is significant [9].Furthermore, if a server requires more computing powerat peak load and not for more than a few hours everyday, either better hardware has to be acquired, or moreservers in a load balanced topology have to be added.As an outcome, even more power consumption and idletime is added to the infrastructure. Except that, humanpresence and intervention is needed to take care of thehardware changes.

A solution to the described problem can be givenusing server consolidation and Virtualization tech-nology. Since the virtual machines have no physicalboundaries, they maintain great flexibility to manip-ulation of their consumable resources and extend theoptions to provide high Quality of Service (QoS)using scripts or other soft technologies. Probably themost famous technique to address performance issuesand underutilization, is the live migration betweendifferent hypervisors [2, 12, 4], but it needs furtherinfrastructure like the existence of a common storagedevice (SAN/NAS) and more than one physical hosts- hypervisors [4]. Small businesses (SMBs) might noteven have this additional infrastructure or spare serversto use live migration. An alternative solution could beto use the dynamic resource allocation ability of virtu-alization to dynamically change the resources allocatedto the running virtual machines on a single hypervisor

112 26th Large Installation System Administration Conference (LISA ’12) USENIX Association

at the right time in order to achieve optimal performance.

Dynamic CPU and live memory scaling is provided bymainstream virtualization technologies like KVM, XENand VMware [8, 15]. CPU is a time-shared resourceso it can be easily shared among the virtual machinesand priority can be given by the scheduler to the mostimportant virtual machines [8]. Memory, on the otherhand, is harder to share since a memory page reservedby one virtual machine, cannot be released as long asit is in use. Common tactics used for memory sharingand scaling are achieved by memory overcommitment.Memory overcommitment builds upon the assumptionthat all virtual machines residing in the same hypervisordo not need to consume 100% of their available memoryat all time. Following this assumption, more memorythan the total memory available in the hypervisor isassigned to the total number of virtual machines andsome memory addresses are used by more than oneguest, usually not at the same time.

Ballooning is a common memory overcommitmentmethod which is used in this paper. Ballooning needsa driver to be installed in the guest operating system.The driver communicates directly with the hypervisorand acts on demand. When the hypervisor needs morememory for a guest, it will choose a virtual machinewhich is the “victim” or the “donor” and the balloondriver will “inflate” in this virtual machine, trying toallocate memory pages from it and give them backto the hypervisor. Then the hypervisor will “deflate”the balloon of the guest who is the “beneficiary” andneeds to get the claimed memory, making availablethese memory addresses to it (see figure 1). Onedisadvantage of ballooning, is that even if the virtualmachine is trying to collaborate, the balloon drivermight not be able to recover the memory requestedby the host fast enough to avoid performance degra-dation [8, 13]. To avoid or reduce this undesirablebehavior, proactivity is needed. A tool which is able tolearn from repeatable events, predict virtual machinememory load beforehand and act, would help to solvethe disadvantage of real time ballooning to a large extent.

Max Memory Guest Allowed to Use: 2048MB

Available Guest Memory: 1792MB 256MB

Low Memory Pressure - Balloon Deflated

Memory Available for Applications Balloon

Max Memory Guest Allowed to Use: 2048MB

Available Guest Memory: 1024MB 1024MB

High Memory Pressure - Balloon Inflated

Memory Available for Applications Balloon

Figure 1: How ballooning works

The scope of this paper, is an attempt to create aprototype self learning, predictive system for dynamicmemory allocation in virtual machines running in asingle KVM hypervisor using Bayesian networks. ABayesian belief network is a probabilistic graphicaldirected acyclic model, indicating the conditionaldependence in a set of random variables [3]. Bayesiannetworks are popular to probabilistic artificially intel-ligent decision support systems for their good abilityto learn from new observations, perceive and planahead [7, 10, 6]. A Bayesian network first needs tobe trained by a person who is knowledgeable in theenvironment it will have to work with. Nodes of randomvariables (chosen by the trainer) have to be created andtheir directed relationship influence needs to be defined.The Bayesian network will then provide probabilitiesfor the event represented by a node to happen, giventhe probabilities of the surrounding nodes. More nodesfor more accurate results can be added to the Bayesiannetwork if there is evidence of interference from othersources, and the current ones can adapt to behavioralchanges of the system giving accurate results as moreknowledge is absorbed. This means the longer thesystem works, more confident results it will provide.

The layout of this paper is as follows: In the re-lated work section an overview of similar scoped papersand their difference from this approach are explained.The methodology section comes to give an explanationof the methods used to perform the experiments, which

2


is followed by the results, discussion, future work and aconclusion.

2 Related Work

Research in the area of virtual machine dynamic re-source allocation is highly active, as this is the key toachieve proper resource utilization, environment friend-liness, cost reduction and increased profits by physicalresource oversubscription. There are many papers foundaddressing the same topic, but a brief survey with themost closely related work with similar scope as this pa-per is gathered in this section.

2.1 Adaptive Control of Virtualized Re-sources

Adaptive Control of Virtualized Resources in UtilityComputing Environments [11]

This system is a quite simple approach based oncontrol theory, for dynamically allocating virtualizationresources in a single hypervisor to succeed in CPUQoS. The experiment comprises input data from thetest virtual machines. An actuator in the control systemcontrols the CPU scheduler of the hypervisor, to assignresources to the virtual machines so that they will notexceed 100% utilization and performance decrement.They only focus on CPU utilization.

2.2 MEmory Balancer (MEB)Dynamic Memory Balancing for Virtual Machines [19]

MEmory Balancer (MEB) is a software which dy-namically monitors the memory usage of virtualmachines, it will then calculate its memory needs, andperiodically reallocates memory to the virtual machines.

MEB has an estimator and a balancer. The estima-tor will build a Least Recently Used (LRU) memorypages histogram for each virtual machine and it will alsomonitor their swap space usage. Then the balancer runswith an interval and adjusts virtual machine memory,based on the information given by the estimator andthe available host resources. Ballooning is used for thememory management from MEB.

2.3 Memory BuddiesMemory Buddies: Exploiting Page Sharing for SmartColocation in Virtualized Data Centers [17]

Memory Buddies, is a project which will try to

maximize the efficiency of the page or memory sharingfeature available in hypervisors, between multiplehypervisors. The page sharing feature will only mergememory pages of virtual machines running in the samehypervisor, so in a data center with multiple hypervisors,memory sharing opportunities may be lost because theguest machines holding identical pages are located ondifferent hosts.

The contribution of this paper is a memory finger-printing technique to identify guests with high memorysharing potential. Then it will use live migration to movethese hosts and shutdown or start on demand hypervisorsnot in use to trim down operational costs by reducingthe energy consumption. Their evaluation shows a 17%increase of the virtual machines running in a data center.

2.4 Overdriver

Overdriver: Handling Memory Overload in an Oversub-scribed Cloud [16]

Ultimately, the target of this project is to maximizecompetitiveness and profitability of cloud providers byoversubscribing customers. Since most of the physicalresources are leased using virtual machines, oversub-scription means that the cloud provider sells moresubscriptions, or more virtual machines to their cus-tomers than what its infrastructure can actually handleif all of them would run at peak load at the same time.The coherence is that the peak load for each customeris largely transient (up to 88.1% of overloads last forless than 2 minutes) and not at the same time for all ofthem. Overdriver focuses on memory oversubscriptionas memory is not largely oversubscribed in practice likeCPU, because memory overload leads to swapping andconsequently to severely degraded performance.

Existing methods to handle memory overload usemostly migration to another physical machine that canhandle the memory requirements, but virtual machinemigration is a heavyweight process needs also highnetwork usage, best suited to handle sustained or pre-dictable overloads. They propose a new application ofnetwork memory to manage overload giving it the name“cooperative swap”. This will take swap pages and storethem to memory servers over the network. Then theypresent Overdriver, the system that it will respectivelychoose between VM migration and cooperative swap tomanage either sustained or transient overloads. Overloadwill create a probability profile for each virtual machineto decide after how much time the coming load isconsidered to be sustained load for the specific guest.When the increased memory load comes, it will first

3


use its cooperative swap feature and if the load surpassthe sustained threshold set for this guest (based on theprobability profile), a live migration will be executed.Overdriver is a reactive tool to avoid excessive memoryload.

2.5 Managing SLA ViolationsDynamic Placement of Virtual Machines for ManagingSLA Violations [2]

In this paper, the authors introduce a managementalgorithm for dynamic resource allocation in virtualizedenvironments using live migration. Their elementalgoal is to meet the Service Level Agreement (SLA) thecompany has with its customers, while reducing theoperational costs from the data centers of the company.The algorithm in this work will measure logged datafor each of the virtual machines and forecast the futuredemand. It will then remap guest virtual machines todifferent hosts. Their forecasting technique involvessome statistical analysis to determine the type of re-source usage of the guests and then classifies them tothree categories:

1. Guests without variability or periodic behavior. (Noneed to migrate)

2. Guests with slight variability and periodic behavior.(Potentially good guests for migration)

3. Guests with strong variability and periodic behav-ior. (Highest migration potential)

From those three categories the second and the thirdcategory of virtual machines are potential candidates forlive migration. The first one shows sustainable resourceusage so it does not need to change host often.

The next step is the calculation of available physi-cal resources needed to handle the predicted load andstart or shutdown non needed servers to reduce costs.This method is a proactive probabilistic method likethe one proposed in this paper but they use differentstatistical tools, while their experimental studies are onlyfocused CPU utilization which is known that it is mucheasier to handle since it is a time shared resource.

2.6 VMCTuneVMCTune: A Load Balancing Scheme for VirtualMachine Cluster Based on Dynamic Resource Alloca-tion [20]

This paper proposes VMCTune, a tool that moni-tors the resource utilization of virtual machines and

their hosts. Then it uses dynamic resource allocation forvirtual machines running on same hypervisor to achievebetter local resource utilization and if the QoS cannot bemet, it uses live migration for virtual machines amongdifferent hypervisors to achieve global load balancing.The tool focuses in Paravirtualized machines and offersa reactive solution dealing with CPU, memory andnetwork bandwidth. Paravirtualized machines are easierto handle their resources in comparison with the Fullyvirtualized ones, as they are aware that they run in avirtualized environment.

VMCTune has several tools to monitor, log thedata, schedule the resource allocation or issue a livemigration if needed and a command line tool by whichthe user can monitor the status of the virtual machinesand control the host.

Improving of scheduling algorithm of the live mi-gration is the key work mentioned as their futureresearch work in this paper, as the one used is a verysimple best-fit algorithm. For example, if a virtualmachine needs more resources and it cannot fit to thecurrent hypervisor, it will be transferred in a differentone with the available resources. There will be noattempt to squeeze resources from other hosts to achievethe best possible utilization.

3 Methodology

In the next section the design of Bayllocator will be pre-sented.

3.1 System DesignIn order to improve working efficiency and extendabil-ity, the system is designed in a modular fashion. Asillustrated in the conceptual system design in figure 2,everything runs on top of a single physical machinewhich is the hypervisor. The hypervisor is split into twological partitions. First is the space available for thevirtual machines (left part on the figure) and the latter(right part on the figure) is the space available for theadministration of the virtual machines.

Data generation is taking place on the virtual ma-chines. Data collection resides on the administrativepart of the system together with the prediction system.A script collects the data through the virtualizationlayer (the dotted separation line) and stores it in adatabase. The prediction then is made using as the datafrom the database as input and the dynamic memoryallocation mechanism will reallocate the memory usingthe virtualization layer.

4


Hypervisor

Virtual Machines

Database

BayesP(A|B)=

P(B|A)P(A)P(B)

Bayllocator

Prediction

Dynamic Memory Using BallooningDark Part of the Memory

Indicates balloon is inßated

Data CollectionDesicion Making

Figure 2: Conceptual system design

3.2 Data Generation

Controlled random data are generated and mostly usedto carry out the experiments. Data generation is a goodstarting point to explore something new. Generated datahas a unique feature. It can be controlled so that onecan make prior assumptions and expectations need to beconfirmed by the results.

For data generation, a script is created and a flow chartin figure 3 explains its working logic. The design of thisscript was made with the ambition in mind to create ran-dom data with controlled randomness by modifying thevariation. The script will generate data given 3 parame-ters:

1. Max memory to occupy (given in MB). This param-eter will set the maximum memory the script willtry to claim. The passed memory is a rough esti-mate.

2. Max threads to create. This parameter will startthe given number of random consuming memorythreads and all of them together will consume amaximum amount of memory given by the first pa-rameter.

3. Max runtime of the script. This will control the timelength the script will be generating data.

Start y threadsWith (x/y) MB max

Mem each

Start

Run timeless than z S?

Max Mem=x MB Mem Threads=y Max run time=z S

Each threadrandomly consumefrom 0 to (x/y)MB

Each threadrandomly sleeps

from 0 to 3 S

Wait for all openthreads to Þnish

their job

End

No

Yes

Figure 3: fakeload.pl - Script to generate data

After the data are collected and stored in a database,rolling averages are created to smooth the randomnesseven more.

3.3 Data Collection from Real ServersReal life data from two servers in production (nexus andstudssh) have also been collected in parallel with thegenerated data. Nexus is a webserver, while studssh isan ssh server for students, with many logged in users.A simulated prediction based on this data is run in theend, which gives an indication of how good the designedBayesian network and its predictions work on real sys-tems.

3.4 Dynamic Memory AllocationDynamic memory allocation driven by the Bayesian pre-dictions is the outcome of this paper and short term pre-diction is the main goal. A memory allocation script iscreated and it tries to reallocate the memory of the vir-tual machines 5 minutes in advance before the expectedmemory demands come.

5


3.5 Experiment DesignThe infrastructure schematic already explained briefly infigure 2. On top of this, a test environment is being builtfor the experimental part of the project. Some virtual ma-chines are created, and high memory load is generatedand collected on each one of them. Simple Bayesian net-works are tested and in the end a prototype (Bayllocator)is created to put everything together. With the final sys-tem built, a simulation is running against the real datacollected. The observed memory utilisation should fol-low the predictions closely to be considered successful,and since ballooning is implemented any slowdowns in-troduced by heavy memory utilisation should be trimmedoff.

4 Results

4.1 BayllocatorBayllocator is the tool created to accomplish the taskin this paper. Some clarifications of the principles thisprogram operates are given as bullets below and a verydetailed flow chart explains visually the entire operationsteps can be studied in figure 11. Bayllocator will:

• Make a prediction of how much memory each of thevirtual machines will need and set this amount ofmemory plus some predefined percentage (default15% more).

• Ensure that the virtual machines do not get less ormore than the minimum and maximum limits set.

• Make sure that the hypervisor will not swap anymemory. In the case of high memory pressure, it ispreferred that the virtual machines start swapping,instead of the hypervisor which is responsible tohandle them all.

• Distribute fairly (according to a percentage of theneeded memory of the total predicted for the vir-tual machines) any excessive hypervisor memory tothe virtual machines. If there is more memory thanneeded available, it can be used for disk cachingpurposes so it will boost guest performance.

• Claim memory fairly from virtual machines if theyneed more than the total amount allowed to get toavoid hypervisor swapping. In this case some of thevirtual machines might start swapping under pres-sure.

It is important to note from the list above, that eventhough the predictions are made on a per-VM basis,

Bayllocator acts with all VMs in mind, trying to dis-tribute the memory as fairly as possible based on indi-vidual predictions.

4.2 Predictions

The Bayesian prediction is being made with an R scriptwhich will be given as command line arguments thestates of the known nodes (evidence), the name of thenode which the answer is to be provided and the file withthe discrete values to learn the CPTs (Conditional Proba-bility Tables) from. The answer is a two-column commaseparated output with the first column showing the pos-sible state and the second column the probability (a per-centage) to observe this state for the given inputs. In theexample on the following code block, the Bayesian net-work is asked “what is the expected memory when it isMonday, 07:55-07:59 and the currently observed mem-ory of the guest system is between 100MB and 200MB”.The answer is 14.8% to get 100MB-200MB, 24.3% toget 200MB-300MB and so on.

$ QueryNet.R /etc/bayllocator/TestServer1.dat \

> FMU Monday h07 m55_59 mem_100_200

mem_100_200,0.148

mem_200_300,0.243

mem_300_400,0.257

mem_400_500,0.226

mem_500_600,0.091

mem_600_700,0.026

mem_700_800,0.009

4.3 Choosing the Expected Memory

The result from the Bayesian network apparently con-tains more than one probability (the previous output ofthe script has more than one row), so a single numberhas to be chosen as the expected memory. To make a fairdecision, all of the given results are used to calculate thisnumber. First the mean value of the discrete states mem-ory will be calculated and then a single number given theprobabilities of all of them as illustrated in the followingequations.

mem 100 200 = (100+200)/2 = 150mem 200 300 = (200+300)/2 = 250mem 300 400 = (300+400)/2 = 350mem 400 500 = (400+500)/2 = 450mem 500 600 = (500+600)/2 = 550mem 600 700 = (600+700)/2 = 650mem 700 800 = (700+800)/2 = 750

6


p1 = 150 ·0.148 = 22.2p2 = 250 ·0.243 = 60.75p3 = 350 ·0.257 = 89.95p4 = 450 ·0.226 = 101.7p5 = 550 ·0.091 = 50.05p6 = 650 ·0.026 = 16.9p7 = 750 ·0.009 = 6.75

7

∑n=1

pn = 348.3MB

4.4 Chosen Bayesian Network

Many Bayesian networks with simple nodes related tothe memory and date were tested, but the following oneaffected by current memory and date proved to be moresuccessful (see figure 4) and its results for the gener-ated data are presented in figure 5, while simulations onreal data with the same network are taking place in fig-ures 6,7,8. The Bayesian network nodes abbreviationsare expanded in table 1 and the quantized discrete statesused for training and querying the Bayesian network canbe seen in table 2. From these 5 nodes, the W, H, M andCMU are evident nodes (known) for the prediction to bemade, and the FMU is the query node which gives theanswers.

W H M CMU

FMU

Figure 4: Bayesian Network affected by current memoryand date.

Bayesian Network Node LabelsAbbreviation Expansion Mapping

W: Day of the WeekH: Hour of the dayM: Minutes

CMU: Current Memory in UseFMU: Future (Predicted) Memory to Use

Table 1: Abbreviation expansion mapping of Bayesiannetwork node labels.

Discrete States UsedW Bool: 0 for weekend, 1 for the rest of the daysH 24 states: 00 - 23M 12 states: 00 04 - 55 59

CMU 41 states: 100 less, 100 200, 4000 greaterFMU 41 states: Same as CMU

Table 2: Bayesian network node quantized discrete statesused for training and querying.

Server1 − Predicted versus Observed MemoryM

emor

y (K

B)

Hours

050

0000

1250

000

2000

000

Wednesday00:00:00

Thursday00:00:00

Friday00:00:00

Predicted MemoryObserved Memory

Figure 5: Predictions plotted against the observed data.

The red line in figure 5 shows the predictions whichcomes 5 minutes earlier and the black line is the actualdata observed. There is a large variation in the observeddata and this is related to the randomly generated data,but as illustrated the prediction follows quite well.

Some of the predictions often reach a certain wrongvalue in y axis, marked with a horizontal dotted line infigure 5. This value is exactly 2099200 KB and it isan indication that the system does not know how to an-swer for the given evidence. When the Bayesian net-work gets an unobserved combination of evidence, thenall of the probabilities of the query node are equal. Be-cause the discrete states defined for the memory nodesCMU and FMU are 41 (from mem 0 100, mem 100 200to mem 4000 greater every 100MB), the probability foreach state is 1/41 = 0.024390244 which gives this num-

7


nexus2Learn Dataset 1

Date

Mem

ory

(KB)

2000

0080

0000

1400

000

2000

000

201204−10

201204−13

201204−16

201204−19

201204−22

201204−25

201204−28

201205−01

201205−04

Learn Data Set Predictions| | |

Observed DataPredicted Data

Figure 6: Learning Dataset from April, 10th 2012 toApril 16th 2012. Predictions right after the learningdataset till the end of the graph.

ber as expanded in equation 1.

∑n=100,4000

n · 141

+39

∑n=1

(n ·100+50) · 141

= 2050MB (1)

2050MB ·1024 = 2099200KB

4.5 Prediction Simulation on Real Data

Three simulations ran with different training datasets innexus2. The first learning dataset for nexus2 is fromApril 10th, to April 16th (figure 6), the second from April10th, to April 22nd (figure 7), and the third one fromApril 10th, to April 25th (figure 8). All of the three sim-ulations ran have missed predictions (2099200KB val-ues as explained earlier), but it is obvious that when thesystem is trained with a larger dataset it fails much less.A comparison of the common predictions for all of thesimulations (after the vertical dotted line) shows that forthe first training set, 808/2774 predictions failed (29.1%),for the second 122/2774 (4.4%) while for the third only76/2774 which makes up the 2.7% of the total predic-tions.

If one takes a better look to the predictions of nexus2,will notice that the predictions follow steps. This isbecause the training datasets are based on discretizedand not continuous variables, so as the predictions. Ifthe memory in the system is 501MB or 600MB, bothof these observations are encoded as the memory statemem 500 600 and they represent a memory state of550MB in the reverse operation.


Date

Mem

ory

(KB)

2000

0080

0000

1400

000

2000

000

201204−10

201204−13

201204−16

201204−19

201204−22

201204−25

201204−28

201205−01

201205−04



Figure 7: Learning Dataset from April, 10th 2012 toApril 22nd 2012. Predictions right after the learningdataset till the end of the graph.


Date

Mem

ory

(KB)

2000

0080

0000

1400

000

2000

000

201204−10

201204−13

201204−16

201204−19

201204−22

201204−25

201204−28

201205−01

201205−04



Figure 8: Learning Dataset from April, 10th 2012 toApril 25th 2012. Predictions right after the learningdataset till the end of the graph.

4.6 Ballooning Evaluation

A simple evaluation was made to see the effect of dy-namic memory allocation. Two operations were runningin test server 1 and their execution time was measured.As can be seen in figure 9, the execution time of a PHPscript running on an apache 2 web server would take asmuch as 50 seconds under heavy memory load (between08:00-18:00), while it would not take more than 3 sec-onds under normal circumstances. When ballooning isinitiated, the operation looked like normal at any time.Same observations can be made for figure 10, where asecure copy operation initiated from an external machineto test server 1. The reason for this slowdown beforeballooning, is the heavy memory swapping on hard disk.Hard disk is many orders of magnitude slower than the

8


Performance on Server 1(Apache PHP Script Execution)

Weekdays

Seco

nds

to C

ompl

ete

Job

05

1015

2025

3035

4045

50

Friday00:00:00

Monday00:00:00

Thursday00:00:00

Sunday00:00:00

Wednesday00:00:00

<−−−−−−−−−Ballooning Disabled−−−−−−−−> <−Ballooning Enabled−>| | |

Figure 9: Server Performance

Performance on Server 1(SCP of a file to a different system)

Weekdays

Seco

nds

to C

ompl

ete

Job

05

1015

2025

30

Friday00:00:00

Monday00:00:00

Thursday00:00:00

Sunday00:00:00

Wednesday00:00:00

<−−−−−−−−−Ballooning Disabled−−−−−−−−> <−Ballooning Enabled−>| | |

Figure 10: Server Performance

memory. To avoid this kind of slowdowns and use themost out of the physical hypervisors, the dynamic mem-ory allocation is necessary.

5 Discussion and Conclusion

The competence of Bayllocator resides in the predictionmethod to predict system utilisation in a non traditionalway, using Bayesian networks. The focus is mainly inmemory prediction and appliance of dynamic memoryallocation based on this information to improve sharingefficiency. Due to its nature, memory is not easy toshare as the time shared resources like CPU or DiskI/O, becoming an obstacle to server consolidation, greenand low cost ICT. Highly active ongoing research triesto predict system utilisation and use this informationfor dynamic resource allocation, but most of themuse traditional statistical methods to approach theproblem. The traditional statistical methods might

involve rigorous mathematics and they usually aresingle purpose, or hard to adapt them to different needs.Bayesian networks are strongly associated with artificialintelligence. Moreover, after an extensive literaturesurvey it does not seem that effort to apply the useof Bayesian networks on server utilisation predictionexists. Bayesian networks differ significantly from thetraditional statistical methods, and their big advantage isthat they are modular directed acyclic graphs, composedof nodes representing variables related to the systemthey work with, and changing of these nodes might givetotally different results and use cases. Furthermore, eventhe output from traditional methods can be used as theinput in a Bayesian network, and more than one of themcan be combined. The modular design gives flexibilityand allows the easy combination and alteration ofdifferent parameters affecting the system.

The prototype software is made in Perl, which iscalling an R script for the Bayesian predictions. Thismodularity gives the flexibility to change the predictionmethod at any time and evaluate the guest performanceunder different circumstances. The lack of wide purposecollected data from real servers prevented from furtherexperimentation with the causal nodes affecting thefinal memory predictions. Each change to the Bayesiannetwork needed time to be tested and make assumptionson how well it works, but when one gets used to workwith Bayesian networks, there is a broad spectrum ofapplications this method can be applied, and certainlycomputing utilisation prediction and decision makingis one of them. The date related variables used in thispaper are not the best choice because the date is not thetrue reason behind high memory activity. They had tobe used though, because of the lack of representativecollected data. The true reasons are probably the numberof logged in users and the number of running processesor network connections which are factors an expertise inthe field knows, or is able to suspect. This is why thetrainer must be knowledgeable to the subject where theBayesian networks needs to work with. The date mightbe an indirect causal node since it affects the numberof connected users in a server (for example if it is aweekend less people might work on a server). Not welldirected nodes and excessive number of states decreasethe learning efficiency of the Bayesian networks, and asa result the training period needs a longer time.

6 Future Work

In this paper, an implementation of simple discreteBayesian networks and their ability to predict memoryutilisation was evaluated. Since the nodes represent dis-crete states for continuous variables, there is a loss of

9


information as illustrated by the introduced imaginarysteps of the prediction in figure 8. Hybrid Bayesian net-works which include both discrete and continuous nodesto achieve higher prediction resolution would be the firsttask for the future work list. Moreover, the focus of thispaper was in the predictions and proactivity, which arefollowed blindly by Bayllocator. Proactivity solve someproblems of reactivity which it might be slow to sud-den large memory changes, but reactivity is mandatoryin a dynamic resource allocation system because if theprediction is wrong, then the system will perform veryinadequately. Implementation of reactivity which willbe ruled by predictions is another key improvement forBayllocator. Design of a more sophisticated Bayesiannetwork with more accurately chosen nodes is also un-der work.

7 Acknowledgements

This paper is work made for a master thesis in systemadministration at the University of Oslo and Oslo andAkershus University College. Therefore, we would liketo express our gratitude to the institutes provided the testequipment, and the people involved directly or indirectly.

8 Availability

The scripts and prototype software created for thisproject are based on Perl and R statistics language. Theycan be acquired upon request by email. Further detailscan be found in the accompanying MSc thesis [14]

References[1] ABRAMOVSKY, L., AND GRIFFITH, R. Ict, corporate restructur-

ing and productivity.

[2] BOBROFF, N., KOCHUT, A., AND BEATY, K. Dynamic place-ment of virtual machines for managing sla violations. In Inte-grated Network Management, 2007. IM’07. 10th IFIP/IEEE In-ternational Symposium on (2007), IEEE, pp. 119–128.

[3] CHARNIAK, E. Bayesian networks without tears. AI magazine12, 4 (1991), 50.

[4] CLARK, C., FRASER, K., HAND, S., HANSEN, J., JUL, E.,LIMPACH, C., PRATT, I., AND WARFIELD, A. Live migration ofvirtual machines. In Proceedings of the 2nd conference on Sym-posium on Networked Systems Design & Implementation-Volume2 (2005), USENIX Association, pp. 273–286.

[5] GARTNER, I. Press release: Gartner says more than 1 billionpcs in use worldwide and headed to 2 billion units by 2014.http://www.gartner.com/it/page.jsp?id=703807, Jun 2008. [On-line; accessed 17-February-2012].

[6] KERSTEN, D., MAMASSIAN, P., AND YUILLE, A. Object per-ception as bayesian inference. Annu. Rev. Psychol. 55 (2004),271–304.

[7] KOLLER, D., AND MILCH, B. Multi-agent influence diagramsfor representing and solving games. Games and Economic Be-havior 45, 1 (2003), 181–221.

[8] KUKREJA, G., AND SINGH, S. Virtio based transcendent mem-ory. In Computer Science and Information Technology (ICCSIT),2010 3rd IEEE International Conference on (2010), vol. 1, IEEE,pp. 723–727.

[9] MEISNER, D., GOLD, B., AND WENISCH, T. Powernap: elim-inating server idle power. ACM SIGPLAN Notices 44, 3 (2009),205–216.

[10] NEAL, R. Bayesian learning for neural networks, vol. 118.Springer Verlag, 1996.

[11] PADALA, P., SHIN, K., ZHU, X., UYSAL, M., WANG, Z.,SINGHAL, S., MERCHANT, A., AND SALEM, K. Adaptive con-trol of virtualized resources in utility computing environments.ACM SIGOPS Operating Systems Review 41, 3 (2007), 289–302.

[12] SCHMID, M., MARINESCU, D., AND KROEGER, R. Aframework for autonomic performance management of virtualmachine-based services. In Proceedings of the 15th AnnualWorkshop of HP Software University Assocaiton. Hosted by AIAkhawayn University in Ifran, June (2008), pp. 22–25.

[13] SCHOPP, J., FRASER, K., AND SILBERMANN, M. Resizingmemory with balloons and hotplug. In Proceedings of the LinuxSymposium (2006), vol. 2, pp. 313–319.

[14] TASOULAS, E. Bayllocator: A proactive system to predictserver utilisation and dynamically allocate memory resources us-ing bayesian networks and ballooning. MSc thesis, University ofOslo, http://www.duo.uio.no/sok/work.html?WORKID=164621(2012).

[15] WALDSPURGER, C. Memory resource management in vmwareesx server. ACM SIGOPS Operating Systems Review 36, SI(2002), 181–194.

[16] WILLIAMS, D., JAMJOOM, H., LIU, Y., AND WEATHER-SPOON, H. Overdriver: Handling memory overload in anoversubscribed cloud. In Proceedings of the 7th ACM SIG-PLAN/SIGOPS international conference on Virtual execution en-vironments (2011), ACM, pp. 205–216.

[17] WOOD, T., TARASUK-LEVIN, G., SHENOY, P., DESNOYERS,P., CECCHET, E., AND CORNER, M. Memory buddies: ex-ploiting page sharing for smart colocation in virtualized data cen-ters. In Proceedings of the 2009 ACM SIGPLAN/SIGOPS in-ternational conference on Virtual execution environments (2009),ACM, pp. 31–40.

[18] YATES, S., DALEY, E., GRAY, B., GOWNDER, J., AND BA-TIANCILA, R. Worldwide pc adoption forecast, 2007 to 2015.Forrester Research Report (2007).

[19] ZHAO, W., WANG, Z., AND LUO, Y. Dynamic memory bal-ancing for virtual machines. ACM SIGOPS Operating SystemsReview 43, 3 (2009), 37–47.

[20] ZHOU, W., YANG, S., FANG, J., NIU, X., AND SONG, H. Vm-ctune: A load balancing scheme for virtual machine cluster usingdynamic resource allocation. In Grid and Cooperative Comput-ing (GCC), 2010 9th International Conference on (2010), Ieee,pp. 81–86.

10


Iterate throughguests - read current

memory in use

Start

Is thehypervisor

memory in 1 enoughfor the demands

of theguests?

Set total Hypervisor memory limit for allof the guests (1)

Calculate the time tomake the prediction for

each guest (default 5 mins in future)

Make the actualprediction for each

guest

Calculate percentage ofusage of the total predicted

memory correspondingto each guest (3)

Add a Þxed hypervisoroverhead and a predeÞned

memory percentage on the prediction

Fix violations byremoving memory fromeach guest according tothe calculated perc in 3

Are themin/max

memory policiesset in 2 followed

by the allguests?

Fix violations by addingor removing memory

from each guest

Set min/max memory limits the guests (2)

Set the memoryto the guests

Sleep for apredeÞned interval

Add the remaining (ifany) hypervisor memory

to the guests according tothe calculated perc in 3

Yes

No

Yes

No

Figure 11: bayllocator.pl: Main script to dynamically allocate memory using the predictions

11

a proactive system to predict server utilization and dynamically allocate memory resources using

Documents