Data Center Power Reduction by Heuristic Variation-Aware Server Placement and Chassis Consolidation

Ali Pahlavan Sharif University of Technology

Tehran, IRAN pahlavan@ce.sharif.edu

Mahmoud Momtazpour Sharif University of Technology

Tehran, IRAN momtazpour@ee.sharif.edu

Maziar Goudarzi Sharif University of Technology

Tehran, IRAN goudarzi@sharif.edu

Abstract—The growth in number of data centers and its power consumption costs in recent years, along with ever increasing process variation in nanometer technologies emphasizes the need to incorporate variation-aware power reduction strategies in early design stages. Moreover, since the power characteristics of identically manufactured servers vary in the presence of process variation, their position in the data center should be optimally determined. In this paper, we introduce two heuristic variation-aware server placement algorithm based on power characteristic of servers and heat recirculation model of data center. In the next step, we utilize an Integer Linear Programming (ILP) based variation-aware chassis consolidation technique to assign tasks to servers such that total power consumption of data center is minimized under obtained server placement. Evaluating the proposed server placement heuristics reveals that up to 14.85% power savings can be achieved under different utilization rates compared to conventional random server placement method.

Keywords-Process Variation, Task Assignment, Server Placement, Power Consumption.

I. INTRODUCTION In recent years, demand to use data centers has increased

all over the world, which consequently raises a worldwide concern on energy cost of Information Technology (IT) equipments [1]. Much research has been done in order to reduce power consumption of data centers. For example, the authors in [2] introduced a thermal-aware Computational Fluid Dynamics (CFD) based task scheduling technique to reduce power consumption. However, due to high time complexity of CFD-based methods, it may not be suitable for online scheduling. Also, a chassis consolidation technique is presented in [1] which effectively tries to power off idle chassis to save more power. Other papers such as [3], [4] use sensor-based methods that have very low timing overhead compared to the CFD technique.

As technology scales into deeper submicron regimes, the impact of process variation on the leakage power consumption of the manufactured chips becomes more substantial. According to Intel's report, leakage power consumption of a fabricated chip can deviate up to 20X from its nominal value in 180nm technology [5], and this is expected to increase in future technologies [6]. In this situation where process variation can turn a homogeneous data center into a heterogeneous one, conventional random placement of servers is ineffective and may result in a non-uniform temperature distribution in data center. This in turn leads to an unavoidable decrement in the required cooling temperature (to avoid possible hot spots) which consequently increases the power consumption of the cooling system. Moreover, utilizing conventional task scheduling methods that do not consider process variation, increases total power costs as well. Therefore, developing a

deliberate server placement method as well as variation-aware task scheduling and chassis consolidation techniques can help to reduce total power consumption.

To address these inefficiencies, this paper focuses on variation-aware server placement and task scheduling for data center power reduction. To the best of our knowledge, this is the first attempt to incorporate variability into server placement and task scheduling techniques for data center power reduction. We first propose two variation-aware server placement heuristics based on the heat recirculation model of the data center. Then, we use ILP-based variation-aware task scheduling method that incorporates chassis consolidation technique to effectively reduce the number of ON chassis to save power. We compare the results with the conventional random server placement method where the effect of variability on the power consumption of servers is ignored. We show that using the proposed variation-aware server placement heuristics along with a variation-aware task scheduling method can effectively improve the total power consumption of a data center. More specifically, we can achieve up to 14.85% power savings in different utilization rates compared to conventional random server placement method under process variation.

The rest of the paper is as follows: In Section 2, we investigate related work .Section 3 introduces data center configuration, server and cooling system power models which are used to calculate total power consumption of data center. In Section 4, we introduce the heat recirculation model, and then we propose our server placement heuristic algorithms and formulate our variation-aware task assignment method using Integer Linear Programming (ILP). Section 5 describes simulation environment and evaluation results. Finally, the conclusion and future work is followed in Section 6.

II. RELATED WORK In the past few years, there have been several attempts to

incorporate thermal-, energy- and power-aware task scheduling techniques in data centers. For example, the authors in [2] proposed a task scheduling method to minimize the total energy consumption of data center by CFD-based thermal management. The algorithm tries to minimize the total number of running servers and turns off all other idle ones. It essentially assigns the tasks to the nodes having the lowest inlet temperature which are determined by CFD simulations. In another CFD-based work [7] power analysis of a data center under temperature variation has been studied. The authors have obtained static provisioning for an arbitrary distribution of cooling resources that will lead to a reference state. They try to minimize the inlet temperature by dividing the workload on available systems.

CFD-based techniques are known to be time consuming and complex. They are not efficient in tackling online scheduling

150

The 16th CSI International Symposium on Computer Architecture and Digital Systems (CADS 2012)

978-1-4673-1482-4/12/$31.00 ©2012 IEEE

problems [8]. Therefore, fast thermal evaluation models are developed such as [3], [4]. In [3], the goal is to reduce the peak inlet temperature in order to reduce power consumption of cooling system using heat recirculation model. As a result, 20% to 30% cooling power saving is obtained at various data center utilization rates. In [4] fast prediction of temperature distribution is done using distributed sensors to reduce energy consumption in high performance data centers considering recirculation effects. Due to low time and computational overhead of such methods, they are suitable for real time and online power management. The authors in [8] demonstrated a thermal-aware resource management method considering heat recirculation model and workload characterization. Their scheduling algorithm will result in reduced power consumption without performance degradation. Reference [1] also presented a chassis consolidation technique to reduce the number of ON chassis. The optimization problem has been solved using ILP. They showed that they can achieve 13% power saving for different utilization rates in comparison with the conventional techniques with no consolidation.

Variation effects are visible in today processors and are expected to further rise with technology scaling when further approaching atomic scales. Process variation effects are already studied in high performance multiprocessor and embedded systems [9], [10], [11], but to the best of our knowledge, none of the previous works considers leakage variation in task scheduling or chassis consolidation at data center scales. We use an ILP-based chassis consolidation technique similar to [1], but our algorithms differs in that it incorporates variability analysis into task assignment and chassis consolidation processes to improve power consumption under process variation. Since the location of servers with higher leakage directly affects the temperature distribution of the data center, we also develop two heuristic variation-aware server placement algorithms that help to obtain more power improvement under different data center utilization rates.

III. DATA CENTER POWER MODEL A. Data Center Configuration

Fig. 1 shows a hierarchical data center structure including Rows, Racks, Chassis and Servers from top to bottom with cold and hot air circulation mechanism. In this structure, each row encompasses several racks. Each rack contains several chassis and each chassis consists of single- or multi-core blade servers with shared power unit and cooling fan.

For air circulation mechanism, data center has Computer Room Air Conditioning (CRAC) units to ventilate cold/hot airs to/from data center room. Data center has hot and cold air aisles. The cold air is distributed through the perforated tiles located on the elevated floor and then it is sucked by the chassis

fans from the cold aisles. The hot air exits the other side of chassis toward hot aisles and then leaves the data center room by the available intakes of CRAC on the ceiling located above the hot aisles. This data center configuration is similar to the one used in [1].

B. Power Model of Blade Servers In this work, we model total power consumption of a data

center similar to [1] considering process variation effect with two components: IT equipment’s power consumption and cooling device’s power consumption. The power consumption of IT equipment is comprised of server power consumption and chassis power consumption. The chassis power accounts for its cooling fan and AC to DC power convertor. The server power comprises two parts: uncore and core power consumption. The uncore power consumption of a server is the power consumed by its memory controller, I/O drivers and any other components except the processing cores [1]. The power consumption of the processing cores is also referred as core power. The difference between power consumption values for different utilization rates is small compared to the total power consumption of the chassis, and hence we simply assume that the active power consumption of a server is largely independent of its utilization level as also assumed in similar researches [3]. Also the tasks on the servers can be run in different voltage frequency scaling level (Lvf). The voltage frequency (V-F) level of each server is adjusted based on the type of running tasks in the process of assigning the tasks and is fixed until the end of the execution interval of that task. Therefore, the power consumption of blade servers is calculated as follow:

, , , ,1 1.s vfN L

i iBase iUncore iCore i i i i i j k i j kj kP P P P P a s b r

= == + + → = δ + +� �

(1)

where PiBase denotes ith chassis power consumption excluding the power consumption of its inner servers. PiUncore presents uncore power consumption of the ith chassis which equals uncore power consumption of all ON servers within it. So, ai and si denote the uncore power consumption of a server and the number of ON servers in the ith chassis respectively. In (1), bi,j,k denotes power consumption of jth active core in an ith chassis which is set to kth (V-F) level under process variation model. Ns is the number of servers in a chassis. We define binary variable r such that, ri,j,k = 1 if jth server in ith chassis runs at kth V-F level and ri,j,k = 0 for other V-F levels (i.e., for any i and j). The number of ON servers doesn’t exceed the number of available servers in a chassis (Ns) automatically with considering the dimension and type (binary) of variable r. Finally, the power consumption of blade servers in Nc chassis is calculated as a vector as follows:

1 1, .s vfN L

j kP a s C C b rδ

= == + + =� �� � (2)

Equation (2) is adopted from [1] which P, δ and C denote [P1,P2,…,PNc]T, [δ1,δ2,…,δNc]T and [C1,C2,…,CNc]T vectors respectively. Operator � presents element-by-element matrix and array product. Therefore, ����and ����denote ����� and �������������������� respectively. Array ��� contains core power consumption of all chassis; and C is computed as the summation of power of ON servers in all chassis. Finally, the total power consumption of IT equipments (Ps) in a data center is calculated as:

1

.cN

s ii

P P=

= �

(3)

where Pi represents power consumption of ith chassis. Figure 1. Data center configuration.

151

C. Power Consumption of Cooling Unit CRAC performance as a cold air supplier of data center

room depends on several factors such as outgoing air speed and main material used in construction of CRAC [1].

Energy consumption of CRAC is determined according to Coefficient of Performance (COP) criterion. COP shows the efficiency of CRAC and is defined with the ratio of removed amount of heat (Qs) by the cooling system to total energy consumption of CRAC (ECRAC) for cooling air process [12]. So, COP is specified as���� � �������� . In our work, we use COP model of HP Utility Data Center’s CRAC. The COP varies with supplied cold temperature of CRAC (Ts). The COP model is defined as���� � ! "#$�%& ! #$� & !'(#. So the cooling power consumption may be specified as follows [12]:

/ .CRAC sP P COP= (4)

where Ps is IT power consumption and COP denotes coefficient of performance of CRAC.

D. Total Power Cost Total power of data center contains IT power consumption

and CRAC power that are accounted in part A and B respectively. IT power dissipation contains core and uncore power consumption of servers. Therefore, data center power cost is the sum of all chassis and CRAC power consumption without considering power losses due to electrical power conversion and distribution network comprising UPS, AC-DC and DC-DC converters and also the switch gear and conductors [1]. Total power cost (PDCTC) is the summation of (3) and (4) and can be written as:

1

1(1 ) .cN

DCTC ii

P PCOP =

= + � (5)

Finally, by replacing (1) in (5), we obtain:

, , , ,1 1 1 1 1

1(1 )( ).vfc c c s LN N N N

DCTC i i i i j k i j ki i i j k

P a s b rCOP

δ= = = = =

= + + +� � ��� (6)

IV. PROPOSED SERVER PLACEMENT AND TASK ASSIGNMENT METHOD

In this section, we propose two heuristic server placement algorithms based on heat recirculation effects. Then, we present a variation-aware chassis consolidation and task assignment method using ILP technique based on the obtained server placement strategy. The proposed flow can be described as follows: First, we use data center configuration data, power characterization results of variation-affected servers, and heat recirculation model of data center as inputs for our server placement methods to obtain the location of servers in data center. Then, we feed the obtained location of servers, power characterization results of variation-affected servers with the workload specification to our variation-aware chassis consolidation algorithm to obtain the task assignment solution.

A. Heat Recirculation Model In this section, first we investigate the effect of heat

recirculation on the power consumption of chassis. Then, we introduce a thermal model of the data center based on the heat recirculation which is used in the proposed task assignment method in the next section.

Power consumption is equivalent to the amount of heat that is transferred per unit time. This fact is proved according to the law of energy conservation. Therefore, the amount of heat in a

unit of time carried by air flow is calculated a��� � )*+,$ [13], where Q is defined as the heat rate in units of Watt (W), � is defined as air density in units of Kg/m3, *�is air flow rate in term of m3/s, cp denotes the specific heat of air in units of J Kg-1 K-1 and T denotes air temperature in terms of Kelvin (K).

Any chassis consumes energy according to law of energy conservation based on the difference of outgoing temperature of hot air (outlet temperature, Tout) and cold air temperature entering the chassis (inlet temperature, Tin). So the power consumption of the ith chassis (Pi) is defined as follow:

( ).i ii i i p out inP Q P f c T Tρ= Δ � = − (7)

In order to obtain chassis power distribution, stable condition of chassis temperature and cold air temperature of CRAC is needed for reaching steady state of data center due to heat recirculation among the chassis. Therefore, similar to [1], we assume that the tasks are running for a long period of time on the data center resulting in a stationary temperature profile. This is usually the case for using High Performance Computing (HPC) tasks such as HSPICE simulation that take several hours or days for running because the time granularity will change due to different incoming workload.

Heat recirculation can be described as the amount of outlet heat rate of a chassis that affects the inlet heat rate of another chassis. The authors in [14] show that the heat recirculation can be defined as a Cross Interference Matrix (ANc×Nc) where Aij is the coefficient of thermal effect of ith chassis on the jth chassis. Hence, the inlet heat rate of ith chassis (Qi

in) is calculated as [4]:

1

.cN

i jin i ji out s

jQ P A Q Q

=

= + +� (8)

where Qiin, Qi

out, Pi and Aij are known and Qs denotes the cold air rate of CRAC. We can transform heat rate to the temperature using thermodynamic constants in a vector form because, the inlet temperature of any chassis should be less than a specific value (Tc) to avoid overheating and eventually failing servers [8]. So, the relation between temperature and power consumption can be defined similar to [4] as:

1 1. , [( ) ].Tin sT T D P D K A K K− −= + = − − (9)

where Tin denotes inlet temperature vector of Nc chassis as -$.

� $.% � / � $.

�012, P denotes power consumption vector of Nc

chassis as [P1, P2,…, PNc]T, and Ts is the column vector with same values in all entries. Matrix K is the Nc×Nc diagonal one (K = diag(K1, K2,…,KNc)) in which 3 � )*+,. Substituting P with (2) into (9), we have:

( ).in sT T D a s Cδ= + + +� (10) where Tin, should not exceed Tc for all chassis. Our goal is to minimize total power consumption of data center (6) while determining optimum Ts to reduce both CRAC and whole data center power consumption. In this paper, typical Tc is 25°C [4]. B. Server Placement Method

Since heat recirculation directly impacts total power consumption of data centers, the optimal placement of servers in data centers seems crucial. To minimize data center power consumption, we propose two server placement heuristics which are described in detail as follows. Heuristic I: Victim-MinP

Fig. 2.a shows the total accumulated thermal effect on the inlet temperature of each chassis. On the other hand, the value shown in color for ith chassis corresponds to 4 5�

���6 where Dij

152

means thermal effect of the jth chassis on the ith chassis. As illustrated in this figure, the chassis that take less effect from the others are those that are located near the data center raised floor. This is due to the fact that the cool air flows upward to enter each chassis and then evacuated by hot air intakes located on both sides of the data center room as shown in Fig. 1. Having an upward flow of the air, the inlet temperature of the chassis located at the top of each rack receives more effect from others. Fig. 2.a also suggests that chassis E of columns 1 and 5 receive the most thermal effect since they are located at the corners of the room where hot air cannot be sucked efficiently from the CRACs (see Fig. 1). We call chassis that receive the most effect from others, Victim Chassis. In our first proposed server placement heuristic, we try to allocate low-power servers to victim chassis. This balances the thermal distribution of the room and in turn, gives the opportunity to increase the cooling temperature to reduce the cooling power cost while meeting critical inlet temperature limit. Placing all low-power servers together in a limited number of victim chassis also gives the opportunity to reduce IT power consumption by using minimum number of chassis in lower utilization rates. In the case of random server placement where low-power servers are randomly distributed over higher number of chassis, variation-aware chassis consolidation fails to simultaneously utilize low-power servers and reduce the number of ON chassis effectively.

Heuristic II: Attacker-MaxP

Fig. 2.b shows the total accumulated thermal effect of each chassis on the inlet temperature of others. On the other hand, the value shown in color for jth chassis corresponds to 4 5�

��6 . As

suggested by this figure, chassis located near the raised floor have higher impact on other chassis. This is due to the upward flow of the cool air from the tiles on the raised floor. Chassis in columns 2 and 3 also have higher effect since they are surrounded by more chassis than chassis located near the corners. We call chassis that have higher effect on the inlet temperature of other chassis, Attacker Chassis. In our second proposed server placement heuristic, we try to allocate power-greedy servers to attacker chassis. In lower utilization rates, we power off the power-greedy (attacker) chassis and only use low-power chassis to minimize total power consumption of IT equipments. Since the remaining low-power chassis are located in areas that has lower effect on each other, the thermal effect is also minimized which in turn leads to an optimized power consumption of cooling system. In higher utilization rates where the use of attacker chassis is inevitable, the situation is worsened, leading to an increase in both IT and cooling system

power consumptions. Since a typical data center usually operates in 20-30% utilization rates [15], we can achieve overall improvement in data center power consumption under the proposed server placement heuristic.

C. Chassis Consolidation and Task Assignment Method We consider the steady state of data center to solve our task

assignment method. We assume that the characteristics of HPC tasks are specified as prior knowledge for setting V-F levels of servers. Desired V-F level for each server is determined from domain name servers (DNS) of the demands for instance [1]. Assume that Nl specifies the number of servers put in each V-F level as a problem input. Therefore, the total number of tasks (Nt) is calculated as 4 78 � 79

�:;86 . We modeled chassis

consolidation technique as an ILP formulation to minimize power consumption of data center. Similar to [1], we defined a binary variable Y to determine ON/OFF chassis. If Yi=1, the ith chassis is ON, else (Yi = 0) is OFF. So, the power consumption of chassis is modified as follow:

, , , ,1 1( ).s vfN L

i i i i i i j k i j kj kP Y a s b rδ

= == + +� � (11)

For ILP method, we correct the nonlinear relationship in (11) due to product of variables by adding other constraint < = � =<7� similar to [1]. This constraint states that if Yi=0, the number of ON servers in ith chassis (si) will be zero, else�> =� = 7�. Therefore, <����� � ������as well as <� � � because, � � 4 4 �����

�:;�6

�?�6 . In summary, we formulate chassis

consolidation technique for power optimization as:

OBJECTIVE FUNCTION:

, , , ,1 1 1{ ( )}.c s vfN N L

i i i i i j k i j ki j kMIN Y a s b rδ

= = =+ +� � � (12)

CONSTRAINTS:

, ,1 1

, ,1

,

1. ( )2. 1,2,...,

3. 1,2,...,

4. 1 1,2,..., , 1,2,...,

5.

s vf

vf

s i i i i i c

i i i cN L

i i j kj k

Li j kk

i j

c

c

c s

T D Y as C TY s YN i N

s r i N

r i N j N

r

δ

= =

=

+ × + + ≤≤ ≤ =

= =

≤ = =

� ��

,1 1

1

, ,

1,2,...,

6.

7. {0,1} 1,2,...,8. {0,1} 1,2,..., , 1,2,..., , 1,2,...,

c s

vf

N Nl li j

Ll tl

i

i j k

vf

c

c s vf

N l L

N N

Y i Nr i N j N k L

= =

=

��

�= =

=

∈ =∈ = = =

� ��

�����

�������

where Y denotes ON/OFF chassis as a vector [Y1,Y2,…,YNc]T.

We do not consider cooling power consumption in the objective function due to the nonlinear relationship in total power consumption (6). To solve this problem, we call ILP solver for each Ts in the range 0 to 25 with an accuracy of 0.1. Then with the comparison of feasible solutions for each Ts, the optimum total power consumption is returned as the solution similar to [1]. Note that in contrast to [1], our chassis consolidation and task assignment technique is variation-aware, meaning that we consider the effect of process variation on power consumption of servers during this process. To this end, we first measure the idle power consumption of all servers in all V-F levels to evaluate the effect of process variation on them. Then we use these profiling data to update bijk values in our formulation and determine ON servers with rijk.

Figure 2. a) Total accumulated thermal effect taken by each chassis from the others, b) Total accumulated thermal effect given by each chassis to the others.

153

V. SIMULATION ENVIRONMENT AND ANALYSIS In this paper, we use GAMS [16] to solve the ILP problem.

The parameters used to model the data center are as follows. The physical dimensions of the data center are 9.6m × 8.4m × 3.6m. There are two rows in the structure consisting of five 42U racks. Each rack is composed of five chassis with ten server slots. The type for each Blade Server is 7U Dell PowerEdge 1855 dual processor. Therefore, the data center has a total number of 1000 single core servers of the same type [3]. There are two V-F levels (LVF = 2) which among them, the first one has higher V-F level. Chassis power consumption overhead (�) and uncore power of servers (a) are considered as @ � 820W and�� � 60W respectively. Each single core server dissipates 25W and 12.5W active power at 100% (LVF1) and 60% (LVF2) voltage-frequency levels respectively [1]. The CRAC unit is responsible for providing cold air in the room with the rate of f = 8m3/s. The flow rate of ith chassis, air density and specific heat of air are considered as fi = 0.2454 m3/s, � = 1.19 Kg/m3 and cp = 1005 J Kg-1 K-1 respectively [3].

In our process variation model, we assume 40 percent of total core power is related to leakage power [17]. A Lognormal distribution is used for variation model [18], [19]. Lognormal distribution parameters are specified as μ = 3.024 and � = 0.85 which denote the mean and standard deviation respectively. These parameters are chosen in such a way that the nominal value equals the 40% of core power consumption and the leakage power deviates up to 20 times from the nominal value as observed in [5].

We assign each task only to a certain server and define different data center utilization rates based on changing number of assigned tasks. For example, if we have a total number of 1000 servers, a 40% utilized data center means that 400 servers are executing the same number of tasks [1]. In this paper, we compare total power consumption and cooling cost in the case of Random Server Placement (RSP; 1000 random placements were tested and the average was computed) and two heuristic server placement algorithms (VICTIM-MINP and ATTACKER-MAXP) both using variation-aware chassis consolidation as the next step. We consider three different workloads which are determined based on the ratio Nl to Nt. This ratio demonstrates what percentage of tasks works with LVF1 and what portion is applied for LVF2. Therefore, workload 1 denotes that all tasks (servers) work with LVF1. Similarly, workload 2 refers to the conditions in which 50% of tasks work

with LVF1 and the rest with LVF2. The last one (workload 3) is defined in such a way that all tasks work with LVF2.

In the following tables (Table I-III), total power consumption, achieved Ts, and power improvement percentage in different data center utilization rates are exhibited for the mentioned workloads. Table I shows almost 8.5% and 8.2% power savings on average for VICTIM-MINP and ATTACKER-MAXP respectively. In addition, we obtained up to 14.85% power improvement for workload 1. VICTIM-MINP has less improvement for less than 50% utilized data center but on average work better than ATTACKER-MAXP due to choosing chassis having low power consumption for higher than 60% data center utilization rates. According to table II, under conditions which 50% of tasks work with LVF1 and the rest with LVF2, results reveal 6.18% and 6.26% power savings on average and up to 9.25% and 9.57% improvements for VICTIM-MINP and ATTACKER-MAXP respectively. Table III specifies 5.82% and 5.898% power enhancement on average and up to 9.79% and 10.13% for VICTIM-MINP and ATTACKER-MAXP in workload 3 respectively. In general, we observe a descending order of power saving percentage as data center utilization rates increase. On the other hand, results show that by using the proposed server placement methods, we obtain more power savings in lower utilization rates.

For VICTIM-MINP, we try to allocate low-power servers to victim chassis. This balances the thermal distribution of the room and in turn, gives the opportunity to increase the cooling temperature to reduce the cooling power cost while meeting critical inlet temperature limit. Placing all low-power servers together in a limited number of victim chassis also gives the opportunity to reduce IT power consumption by using minimum number of chassis in lower utilization rates. For ATTACKER-MAXP in the case of low utilization rates, we can assign tasks to low power servers located in chassis having less thermal effect on the others. Again, low inlet temperature leads to choosing suitable cooling temperature. In higher utilization rates, we are forced to use power-greedy servers which are placed in locations that have higher thermal effect on others, resulting in an increase in both cooling and IT power consumption. This results in less power improvement in higher utilization rates and even a higher power consumption than RSP for the same reason. However, this is acceptable since in typical data centers, the average utilization is generally 20-30% [15].

TABLE I. TOTAL POWER CONSUMPTION AND TS WITH POWER SAVING PERCENTAGE FOR WORKLOAD 1

Data Center Utilization (%)

RSP Ts (°C)

VICTIM-MINP Ts (°C)

ATTACKER-MAXP Ts (°C)

RSP Power (KW)

VICTIM-MINP Power

(KW)

ATTACKER-MAXP

Power (KW)

Power Saving VICTIM-MINP

(%)

Power Saving ATTACKER-

MAXP (%) 10 24.101 24.4 24.8 17.256 14.77 14.693 14.41 14.8520 23.821 24.2 24.5 34.54 30.133 29.946 12.76 13.330 23.388 24.1 24.1 52.696 46.055 45.79 12.6 13.1140 22.725 23.4 23.3 71.369 62.534 62.59 12.38 12.350 21.682 21.9 22.1 91.033 81.112 80.992 10.89 11.0360 20.353 20.5 20.8 112.042 101.616 101.426 9.31 9.4770 18.969 18.7 19.2 135.2 125.425 127.064 7.23 6.0280 17.35 17.1 17.4 160.965 152.389 154.108 5.33 4.2690 15.726 15.3 15.3 190.696 188.511 190.746 1.15 -0.03

100 12.78 12.6 12.2 240.974 242.845 247.211 -0.78 -2.59Power Improvement Average (%) 8.53 8.17

154

TABLE II. TOTAL POWER CONSUMPTION AND TS WITH POWER SAVING PERCENTAGE FOR WORKLOAD 2

Data Center Utilization (%)

RSP Ts (°C)

VICTIM-MINP Ts(°C)

ATTACKER-MAXP Ts (°C)

RSP Power (KW)

VICTIM-MINP Power

(KW)

ATTACKER-MAXP Power

(KW)

Power Saving VICTIM-MINP

(%)

Power Saving ATTACKER-

MAXP (%) 10 24.101 24.7 24.8 17.256 14.103 14.052 9.25 9.5720 23.821 24.3 24.6 34.54 28.646 28.469 9.08 9.6430 23.388 24.1 24.1 52.696 43.518 43.39 9.18 9.4540 22.725 23.5 23.6 71.369 58.906 58.866 8.93 8.9950 21.682 22.3 22.4 91.033 75.699 75.604 7.83 7.9460 20.353 21 21.3 112.042 93.965 93.876 6.61 6.6970 18.969 19.5 19.5 135.2 114.199 114.84 5.29 4.7680 17.35 18.1 18.5 160.965 136.525 136.494 4.09 4.1290 15.726 16.7 16.7 190.696 164.15 165.245 1.62 0.96100 12.78 14.5 14.7 240.974 200.381 199.306 -0.02 0.52

Power Improvement Average (%) 6.19 6.27

TABLE III. TOTAL POWER CONSUMPTION AND TS WITH POWER SAVING PERCENTAGE FOR WORKLOAD 3

Data Center Utilization (%)

RSP Ts (°C)

VICTIM-MINP Ts(°C)

ATTACKER-MAXP Ts (°C)

RSP Power (KW)

VICTIM-MINP Power

(KW)

ATTACKER-MAXP Power

(KW)

Power Saving VICTIM-MINP

(%)

Power Saving ATTACKER-

MAXP (%) 10 24.101 24.7 24.8 17.256 13.531 13.48 9.79 10.1320 23.821 24.3 24.6 34.54 27.394 27.253 9.13 9.5930 23.388 24.1 24.1 52.696 41.583 41.468 8.68 8.9440 22.725 23.6 23.6 71.369 56.125 56.151 8.51 8.4750 21.682 22.2 22.3 91.033 72.202 72.131 7.09 7.1860 20.353 21.1 21.3 112.042 89.219 89.061 6.16 6.3370 18.969 19.6 19.7 135.2 108.277 108.715 4.71 4.3380 17.35 18.3 18.6 160.965 129.204 129.322 3.59 3.5190 15.726 16.5 16.8 190.696 155.099 156.234 0.89 0.16

100 12.78 14.6 14.8 240.974 189.009 187.683 -0.35 0.35Power Improvement Average (%) 5.82 5.9

VI. CONCLUSION AND FUTURE WORK In this paper, we proposed two heuristic variation-aware server

placement algorithms for data center power reduction. The proposed server placement methods try to find the best location of each server in the data center according to heat recirculation model. We also used a variation-aware chassis consolidation technique in the task assignment process to effectively shut down idle chassis in different utilization rates in order to save power. Experimental results showed that by utilizing our proposed server placement methods along with variation-aware chassis consolidation technique, up to 14.85% power improvement can be obtained by ATTACKER-MAXP in comparison with conventional random server placement.

As technology scales into deep submicron regimes, the effect of process variation on performance and power consumption of processors increases rapidly. We plan to utilize a more precise and structured process variation model based on real measurements and analyzing the effectiveness of our approach in higher leakage variations is our future work.

ACKNOWLEDGMENT This research is partially supported by grant number

17179/500/T from Research Institute for ICT of Iran. We are grateful for their support.

REFERENCES [1] E. Pakbaznia and M. Pedram, “Minimizing data center cooling and server

power costs,” ISLPED09 Proceeding of the 14th ACM/IEEE International Symposium on Low Power Electronics and Design, pp. 145-150, 2009.

[2] Q. Tang, K. S. Gupta, D. Stanzione, and P. Cayton, “Thermal-aware task scheduling to minimize energy usage of blade server based datacenters,” 2nd IEEE International Symposium on Dependable, Autonomic and Secure Computing, pp. 195-202, 2006.

[3] Q. Tang, K. Kumar, S. Gupta, and V. Georgios, “Energy-Efficient Thermal-aware task scheduling for homogeneous high-performance computing data centers: A cyber-physical approach,” IEEE Transaction on Parallel and Distributed Systems, vol. 19, no. 11, pp. 1458-1472, 2008.

[4] Q. Tang, T. Mukherjee, K. S. Sandeep, K. S. Gupta, and P. Cayton, “Sensor-based fast thermal evaluation model for energy efficient high-performance

datacenters,” Proc. Int'l Conf. on Intelligent Sensing and Info. Process, pp. 203-208, 2006.

[5] S. Borkar, et al., “Parameter variations and impact on circuits and microarchitectures,” Design Automation Conference, pp. 338-342, 2003.

[6] “International technology roadmap for semiconductors overview,” ITRS, 2008 Update, url: http://www.itrs.net/reports.html

[7] A. H. Beitelmal and C. D. Patel, “Thermo-Fluids provisioning of a high performance high density data center,” Distributed and Parallel Databases, vol. 21, no. 2-3, pp. 227-238, 2007.

[8] L. Wang, G. Laszewski, J. Dayal, and T. R. Furlani, “Thermal aware workload scheduling with backfilling for green data centers,” IEEE 28th Performance Computing and Communications Conference (IPCCC), pp. 289-296, 2009.

[9] F. Wang, C. Nicopoulos, X. Wu, Y. Xie, and N. Vijaykrishnan, “Variation-aware task allocation and scheduling for MPSoC,” IEEE/ACM International Conference on Computer-Aided Design (ICCAD’07), Nov. 2007.

[10] M. Momtazpour, M. Goudarzi, E. Sanaei, “Variation-Aware Task and Communication Scheduling in MPSoCs for Power-Yield Maximization,” IEICE Trans. Fundamentals of Electronics, Communications and Computer Sciences (Special Section on "VLSI Design and CAD Algorithms"), E93-A(12), pp. 2542-2550, Dec. 2010.

[11] M. Momtazpour, M. Goudarzi, E. Sanaei, “Static Statistical MPSoC Power Optimization by Variation-Aware Task and Communication Scheduling,” Elsevier Journal of Microprocessors and Microsystems, in press, Feb. 2012.

[12] J. Moore, J. Chase, P. Ranganathan, and R. Sharma, “Making scheduling ‘Cool’: Temperature-aware resource assignment in data centers,” Usenix Annual Technical Conference, 2005.

[13] Y. A. Cengel, “Heat transfer: a practical approach,” 2nd edition, McGraw-Hill, 2003.

[14] http://www.newservers.com/1855-specs.pdf [15] L. Barroso and U. Holzle, “The case for energy-proportional computing,”

IEEE Computer, Jan 2007. [16] R. E. Rosenthal, ‘‘GAMS-a user guide,’’ GAMS Development Corporation,

2010. [17] G. B. Sirsi, ‘‘Leakage power optimization flow,’’ International Cadence

Usergroup Conference, 2004. [18] S. Ghosh and K. Roy, “Parameter variation tolerance and error resilency:

new design paradigm for the nanoscale era,” Proceedings of the IEEE, vol. 98, no. 10, pp. 1718-1751, 2010.

[19] R. Rao, A. Srivastava, D. Blaauw, and D. Sylvester, “Statistical estimation of leakage current considering inter- and intra-die process variation,” Low Power Electronics and Design, ISLPED, pp. 84- 89, 2003.

155