rate-harmonized scheduling and its applicability to energy management

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 6, NO. 3, AUGUST 2010 265

Rate-Harmonized Scheduling and Its Applicability toEnergy Management

Anthony Rowe, Member, IEEE, Karthik Lakshmanan, Member, IEEE, Haifeng Zhu, Member, IEEE, andRagunathan (Raj) Rajkumar, Senior Member, IEEE

Abstract—This paper presents a family of Rate-HarmonizedSchedulers that can be used in reservation-based operating systemsto naturally cluster task execution and lump processor idle dura-tions. While traditional approaches to energy management havefocused on reducing dynamic switching power through DynamicVoltage and Frequency Scaling (DVFS), processor technologytrends predict a future in which static leakage power will begin todominate. To this end, most modern processors provide built-insupport for sleep modes with low leakage power. However, sub-stantial time is required to switch in/out of such sleep modes dueto mechanical oscillator stabilization delays. Significant oppor-tunities for energy saving are potentially missed due to idle gapsbetween executing tasks that are shorter than the time required toenter the sleep mode. Armed with apriori workload information,reservation-based operating systems can potentially eliminatesuch wasted idle durations using Rate-Harmonized Scheduling.An Energy-Saving Rate-Harmonized Scheduler guarantees thatevery idle duration can be used to switch into sleep mode. Thispaper also provides extensions to Rate-Harmonized Schedulingto support multicore processors. Empirical evaluation resultsare provided from an implementation in the nano-RK operatingsystem for wireless sensor networks. Energy-Saving Rate-Harmo-nized Scheduling saves 16.8% energy compared to conventionalRate-Monotonic Scheduling for the task set used in Sensor Andrewproject. At low utilization levels, Energy-Saving Rate-HarmonizedScheduling can save up to 39% energy on randomly generatedtask sets.

Index Terms—Communication systems, energy management,multitasking, processor scheduling.

I. INTRODUCTION

B ATTERY capacity continues to pose major constraints inthe design and usage of increasingly popular portable,

mobile and wearable communication devices. On the other endof the spectrum, large-scale data centers are dealing with ex-ceedingly high energy bills and cooling costs. Requirements insuch varied application domains have resulted in significant re-search interest and collaboration across communities. While dy-namic switching power minimization has been well-studied in

Manuscript received October 31, 2009; revised March 29, 2010 and May 30,2010; accepted May 30, 2010. Date of publication June 28, 2010; date of currentversion August 06, 2010. Paper no. TII-09-10-0301.

A. Rowe, K. Lakshmanan, and R. Rajkumar are with the Department ofElectrical and Computer Engineering, Carnegie Mellon University, Pitts-burgh, PA 15213 USA (e-mail: [email protected]; [email protected];[email protected]).

H. Zhu is with the Department of Electrical and Computer Engineering,Carnegie Mellon University, Pittsburgh, PA, 15213. He is currently at UnitedTechnologies Research Center, East Hartford, CT 06118-1127 USA (e-mail:[email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TII.2010.2052106

literature [5]–[8], processor manufacturing technology trendssuggest that leakage power is going to dominate in the near fu-ture. Therefore, maximizing the opportunity to put the processorin low leakage sleep states will result in significant overall en-ergy savings.

This paper proposes, analyzes, and demonstrates the benefitsof some simple and practical algorithms for managing leakagepower through sleep states. These algorithms present effectiveapproaches to energy management, when used in conjunctionwith reservation-based real-time operating systems (such asLinux/RK [3] and nano-RK [4]), where the processing, band-width and timing constraints of tasks are known a priori.

Recognizing the need for leakage power management, mostmodern microcontrollers provide built-in support for variousenergy saving modes. However, there is a long transition timeassociated with moving to lower energy states due to the over-head required for the main oscillator to startup and stabilize.On FireFly sensor nodes [9] using the Atmel ATmega1281processor, switching from an active energy state to an idleenergy state takes on the order of a few microseconds sincethe main system clock remains active. On the other hand, theround-trip transition from idle to deep-sleep takes on the orderof 10–15 ms. If the gap between two tasks is less than thisperiod, the processor is only able to switch to the idle energystate even though there is no useful work to be done. In fact, asignificant percentage of time has been observed to be spent inidle mode due to the accumulation of small gaps between tasks.

This paper introduces a family of rate-harmonized schedulers(RHS) that clusters the execution of tasks so that idle durationscan be lumped together enabling transitions to the sleep mode.One such rate-harmonized scheduling technique called Energy-Saving RHS adds a virtual sleep task in a manner that allowsevery inactive period of execution to be used as sleep time inthe system. This scheme yields many major benefits:

• A processor using energy-saving RHS can transition anyand every idle duration on the processor into the sleepmode. This includes the idle duration resulting from tasksexecuting less than their worst-case execution times. En-ergy saving is thus maximal. The only requirement is thatthe task set be feasible under energy-saving RHS, and suf-ficient conditions for feasibility are provided.

• Thanks to the maximal energy savings obtained, there is nolonger a need to manage more than two CPU energy statesfor schedulable task sets. The processor is either in thesleep state or the active state. This simplifies the schedulerdesign for real-time systems.

• The worst-case energy consumption using energy-savingrate-harmonized scheduling can be predicted, analyzed andoptimized. The benefits of analyzability are likely to man-ifest themselves over time in myriad and surprising ways.

1551-3203/$26.00 © 2010 IEEE

266 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 6, NO. 3, AUGUST 2010

Rate-harmonized schedulers have the interesting property ofclustering task execution together. Though beyond the scopeof this work, this batching property can be useful in other sce-narios. For example in a situation where multiple tasks accessa shared resource that has a significant setup cost, it would beideal avoid repeated initializations. In this paper, the shared re-source under consideration is the CPU and the penalty that isminimized is the energy lost in the setup time required for theCPU to switch from the deep sleep to the idle energy state.

A. Organization of Paper

The rest of this paper is organized as follows. Section II dis-cusses related work. Section III introduces Rate-HarmonizingSchedulers providing schedulability conditions and runtimeproperties. Section IV applies Rate-Harmonized Schedulingto the context of processor energy management. Section Vextends rate-harmonized scheduling to multicore processorsin the context of energy management. Section VI presents aperformance evaluation of our energy schemes. Section VIIprovides concluding remarks.

II. RELATED WORK

Many current sensor networking systems are designed usingnonpreemptive operating systems in order to save on memory[10], [11]. These systems are event-triggered and typically pro-vide energy savings by executing tasks as quickly as possibleand then returning to sleep. Without deadline information, it isdifficult to cluster events in order to further save energy. Dueto the increasing complexity of sensor networking tasks andthe scaling of technology, multiple preemptive operating sys-tems capable of running on microcontrollers are now publiclyavailable [4], [12], [13]. These operating systems mention useof apriori task knowledge for energy savings, but do not pro-vide schemes with additional benefits beyond standard priority-based scheduling.

For time-sensitive applications, priority-based preemptivescheduling is used to implement the rate-monotonic schedulingparadigm [14]. Given a periodic sensor task set with timingdeadlines and a priority set inversely proportional to the periodof the task, one can prove that timing guarantees are honoredas long as the total task-set utilization is below establishedbounds [14]. This work extends upon this paradigm throughthe use of phase adjustment at the cost of scheduling efficiencyto improve energy performance. [21] generalizes the originalrate-monotonic analysis to support periodic tasks with arbi-trary deadlines. This paper uses parts of this analysis to proveutilization bounds of task sets given phase adjustments due toharmonization.

Real-time scheduling experts will rightfully note that the“energy-saver” task used in energy-saving rate-harmonizedscheduling behaves like a sporadic task [22] which executes atthe highest priority from a scheduling perspective and servicesan endless queue of sleep requests. However, when used withbasic rate-harmonized scheduling, the technique exhibits an ad-ditional set of very attractive properties from an energy-savingperspective.

The techniques of dynamic voltage scaling (DVS) and dy-namic frequency scaling (DFS) have been the focus of muchresearch in recent years with the objective of reducing energyconsumption [5], [6]. This is due to the fact that the dynamicpower consumption of CMOS circuits [7], [8] is given by

, where is the power, is the average activityfactor, is the average load capacitance, is the supplyvoltage and is the operating frequency. Since the power has aquadratic dependency on the supply voltage, scaling the voltagedown is an effective way to minimize energy consumption.However, lowering the supply voltage can also adversely affectthe system performance due to increasing delay. As observed inprevious research [2], semiconductor scaling trends will resultin lower operating voltages and larger transistor density, whichresults in lower dynamic switching power and higher staticleakage power. It is therefore useful to focus on leakage powermanagement techniques. Also, many low-cost microcontrollersdo not have DVS/DFS capabilities. Our approach works evenin systems without support for DVS and DFS.

Multiple procrastination scheduling techniques [15]–[20]have been proposed that tend to cluster task execution to saveenergy. In [17], the author’s propose using procrastination oftasks to determine when a single processor system could beshutdown. [19] uses a greedy approach to procrastination byexpanding all jobs within a hyper-period in order to extend idletimes. This approach does not take into account that near-termoptimization might not allow for the processor to be completelyput to sleep in the future. [16] proposes a three-approximationsolution to minimize energy consumption on DVS singleprocessor systems while taking into account leakage current.[16], [19], and [20] all use EDF-based solutions whereas RHSaddresses fixed priority scheduling systems. [15] describes afixed-priority scheduling approach that uses online simulationto estimate how long a task can procrastinate. This incursscheduler overheads that increases runtime execution com-plexity. RHS is a simple approach that can be easily extendedwith an energy-saver task to guarantee conversion of all idletime into deep-sleep time. Without explicit synchronizationacross cores, dynamic scheduling procrastination approachesare more difficult to adapt for use in multicore systems. Weshow how using a common harmonizing period across coresthat RHS can allow an entire processor to fully sleep.

III. RATE-HARMONIZED SCHEDULING

In this section, we introduce the necessary notation, ter-minology and schedulability conditions for Rate-HarmonizedScheduling. The next section then describes the energy savingvariant of RHS.

A. Notation and Terminology

Consider a periodic task set comprising ofperiodic tasks, each with a worst-case computation time, , andperiod, . The task set is ordered such that

. The relative deadline of each task is the same as its pe-riod . Each task also has an initial arrival time of , suchthat its arrival times are at . Withoutloss of generality, assume that the phase of task , .

ROWE et al.: RATE-HARMONIZED SCHEDULING AND ITS APPLICABILITY TO ENERGY MANAGEMENT 267

This paper adopts the fixed-priority preemptive scheduling ap-proach with task priorities assigned using the rate-monotonicpolicy (i.e. inversely proportional to task periods). The utiliza-tion of task is given by . The total utilizationof the task set is the sum of the utilization of the tasks in the taskset.

The proposed approach called Rate-Harmonized Schedulingallows the execution of different real-time tasks to be clusteredtogether, thereby having the effect of lumping together idle du-rations in the processor schedule. While many variants of rate-harmonized scheduling are possible, this paper introduces onlya basic version upon which the maximal energy-saving schemecan be built.

B. The Basic Rate-Harmonized Scheduler

A general rate-harmonized scheduler (RHS) utilizes a set ofperiodic values where , to

, and the values in are harmonic.1 Theseharmonic periods are referred to as the Harmonizing Base Pe-riods, and all have the same initial phasing of .Tasks in the given task set are released accordingto their arrival patterns as in the classical periodic task set model.However, each job of a task only becomes eligible to executeat its next nearest periodic boundary of . Specifically, the

job of arrives at time but becomes eligibleto execute only at time

.In the Basic Rate-Harmonized Scheduler,

. is simply referred to as the HarmonizingPeriod. Since this is a rate-harmonized scheduler .

In the Basic RHS schedule, tasks that arrive before or afterintegral multiples of are not eligible to execute until thenext closest boundary of when they are serviced based ontheir priority. Tasks that are not eligible are delayed until thenext boundary.

is chosen so as to improve schedulability. Suppose. If , . Otherwise,

.Some properties of basic rate-harmonized scheduling are now

provided.Lemma 1: The worst-case response time of under basic

rate-harmonized scheduling is given by .Proof: The basic rate-harmonized scheduler requires that

the phasing of the harmonizing period be the sameas that of (i.e. ), and is harmonic with respectto . As a result, is eligible to execute as soon as it arrives.Being also the highest priority task, ’s worst-case responsetime is .

We now establish a critical scheduling instant [14] for anytask under basic rate-harmonic scheduling (see Fig. 1), whichresults in the longest possible response-time for .

Theorem 2: A critical instant for any task underbasic rate-harmonized scheduling occurs when is requestedsimultaneously with requests for all higher priority tasks, andhas to wait before it becomes eligible to execute (where

is an infinitesimally small positive value).

1A set of integers �� is defined as a harmonic set�� , where � is the set of positive integers

Fig. 1. This figure shows the critical instant when all tasks arrive immediatelyafter � .

Proof: Under basic rate-harmonized scheduling, all tasksbecome eligible to execute only at boundaries that are inte-gral multiples of . If arrives at simultaneously with allhigher priority tasks time-units before the next integralboundary of , task and all its higher priority tasks becomeeligible to execute only at .

If any higher priority task arrived earlier than , wouldhave been eligible to execute earlier, and the response time forcannot become worse. If any higher priority task arrived after

and at or before , all of ’s jobs arriving later will bedelayed, and the response time for can only become better (orstay the same). If arrives later than , its response time willbecome longer by letting arrive at . If arrives earlier than ,it would become eligible to execute earlier and its response timecan only become shorter (or stay the same). Hence, representsa critical instant for .

Remark: Note that in Theorem 2, relative to the classical Liuand Layland model, the additional delay of encountered bya task is concurrent with respect to the delays encountered byall its higher priority tasks.

Lemma 3: Under basic rate-harmonized scheduling, the max-imum value of is 0.5 for any task , .

Proof: The lemma follows from the choice of for thebasic rate-harmonized scheduler.

Theorem 4: A task set is feasible under basic rate-harmonizedscheduling if .

Proof: By assumption, deadline for everytask . From Theorem 2, the additional delay of encoun-tered by a task is equivalent to shortening the deadline ofin the Liu and Layland model by (with the exception offrom the proof of Lemma 1).

Hence, the ratio of the effective deadline to the period(denoted by in [21]) of , , is given by

. From Lemma 1, taskcannot constitute the bottleneck task. The minimum value

of is (using Lemma 3). The

theorem follows from [21, eq. (3.3)], which is given below forreader convenience:

ifif


A sufficient schedulability condition can also be stated usingthe fixed-point response time computation technique [23], [24].To find the worst-case response time of , let

where is an arbitrarily small unit of time. Iterate on asfollows:

until in which case, is the worst-case re-sponse time of . Check if . If ,misses its deadline and the computation can be stopped.

IV. ENERGY-SAVING WITH RATE-HARMONIZED SCHEDULING

Most modern processors have built-in support for multiplemodes of operation with each mode consuming a differentamount of power. The processor needs more or less time toswitch into and out of different power-saving modes. The lowerthe power consumption, the larger is the time required to switchinto and out of that mode. For example, a “power-aware”processor normally has:

• an active mode, wherein the processor consumes the mostamount of power but it can execute tasks waitingto be processed;

• an idle (or nap mode), where it consumes less powerthan the active mode, but no processing can take

place and a small amount of time 2 must be spent toswitch into and out of this mode;

• a deep sleep (or sleep mode), where it consumes the leastamount of power , but no processing can take placeand a sizeable amount of time must be spent toswitch into and out of the mode. This typically involvesspinning down the main oscillator which takes a significantamount of time to stabilize upon reactivation.

Normally, , and.

When the processor has no ready tasks to execute, it is oftentempting to switch the processor into the sleep mode. How-ever, the processor cannot go to sleep if a job can arrive within

units of time. This can result in significant time inter-vals when the processor is not doing any useful work but is yetnot in the deep-sleep mode.

We do not consider additional energy overheads (besidestime) for the processor to transition between states. In gen-eral, for microcontroller devices (like those used in sensornetworking applications) there is little extra penalty for theclock to begin oscillating as compared to the steady-stateoperation values. This may not be true for all classes ofgeneral-purpose processors. One could also factor in theoverhead effects by considering a new value of

, where

2ST stands for switching time.

Fig. 2. This figure shows the following task set � � �� , � � ��,� � �� with a � � � being scheduled with normal RMS on the topand with RHS on the bottom. In the bottom timeline, the tasks are harmonizedto 10 �� which is denoted by the darker vertical bars.

is the energy overhead of switching between the idle andsleep state similar to what is shown in [19].

Fortunately, the periodic nature of real-time tasks, when ap-propriately structured, provide significant insight into the futurearrival times of jobs for the processor. Real-time operating sys-tems using a reservation-based approach (e.g., Linux/RK [3]and Nano-RK [4]) can exploit this knowledge to switch the pro-cessor into low-power deep-sleep mode of operation, wheneverjobs are not expected to arrive in the near future. Thereby, en-ergy savings can be obtained without compromising the timeli-ness and QoS constraints of application tasks.

Energy Saving Rate-Harmonized Scheduling is a novel butsimple, practical and effective new technique that maximizes thepercentage of time a processor spends in the deep-sleep modewithout violating the timing constraints of the real-time task set.In fact, using this technique, every time-unit that the processoris not in active mode can be spent in deep-sleep mode. Hence,the idle-mode of processors can even be potentially eliminatedyielding hardware savings along with enhanced energy savings!

In the previous section, schedulability conditions have beenprovided for basic RHS, but properties have not been proven toshow how well the scheme works to save energy. Fig. 2 showsan example task set being scheduled with normal rate-mono-tonic scheduling (RMS) and Basic RHS with and

. It assumes that .The arrival time of each task is indicated with an arrow aboveeach timeline. The example of Fig. 2 illustrates a situation wherebasic RHS does, in fact, save energy since all of the idle pro-cessor time can indeed be converted into deep-sleep time. How-ever, this does not always occur.

The basic RHS can be extended to use a periodic EnergySaver task, , that is scheduled as the highest priority taskwith its execution time , a period

, and phasing . Whenever this task ex-ecutes, the processor can go into deep-sleep mode (forunits of time every units of time). Real-time schedulingtheorists will note that the energy-saver task will indeed be-have like a sporadic task [22] executing at the highest priority


from a scheduling perspective servicing an endless queue ofdeep-sleep requests. However, when used with basic rate-har-monized scheduling, the Energy Saver exhibits an additional setof very attractive properties. This hybrid scheme is referred toas Energy-Saving Rate-Harmonized Scheduling.

The state of the resource under consideration could be de-scribed as follows.

• The resource being scheduled is said to be busy when theresource is executing one or more of the tasks ,to . Correspondingly, a busy duration is defined to be acontiguous interval in the schedule when the resource isbusy.

• The resource being scheduled is said to be in forcedsleeping mode when the resource is executing .Correspondingly, a forced sleep duration is defined to bea contiguous interval in the schedule when the resource isin forced sleep.

• A busy-sleep duration is defined to be a contiguous intervalin the schedule when the resource is either busy or in forcedsleep. An idle duration is defined to be a contiguous in-terval when the resource is neither busy nor in forced sleep.

The following theorem allows the coupling of idle durationswith forced sleep durations.

Theorem 5: Every idle duration in an energy-saving rate-har-monized schedule will precede (and therefore be contiguous)with a forced sleep execution of .

Proof: The Energy-Saver task has higher prioritythan every task , to , and has an initial phasing of

. Hence, the resource will be in forced sleepwhen executes at intervals ,

. Correspondingly, the execution of any job ofany task , to , is only during the intervals

for .Consider any time instant when the resource becomes idle.

That is, represents the beginning of an idle duration. Due tothe execution pattern of , must lie within the interval

for some non-negative integervalue of . Now it needs to be shown that the interval

will be an idle duration which, in turn, precedes theforced sleep execution of during

.Since , any task , to , that arrives within

the interval becomes eligible to executeonly at . So, such an arrival cannot execute in theinterval of interest. If task , to , arrived at or before

, it would have become eligible to execute at orearlier. If has any execution time left at time , the energy-saving rate-harmonized scheduler must schedule at time .This contradicts the assumption that represents the start of anidle duration. The theorem follows.

Theorem 6: Every idle duration in an energy-saving rate-har-monized schedule can be used to put the resource into a deep-sleep mode without any time penalty.

Proof: From Theorem 5, all idle durations precede (and arecontiguous with) a forced sleep execution of for a durationof . This forced-sleep duration of can be extended

to include the preceding contiguous idle duration. This extendeddeep-sleep duration is longer than , which guarantees thatthere is no time penalty switching into and out of deep-sleepmode.

Given that all idle durations in the energy-saving RHSschedule can be spent in deep sleep, the deep-sleep utilizationis given by

In other words, the deep-sleep utilization is maximal given thetask set utilization of . The only condition that needs to bechecked is whether the given task set is feasible under Energy-Saving RHS.

Theorem 7: A periodic task set is feasible under Energy-Saving Rate-Harmonized Scheduling if

Proof: Under energy-saving rate-harmonized sched-uling, . Also, either or

. Under either of these conditions, andform a (high-priority) task set scheduled under rate-monotonicscheduling with harmonic periods. Hence, under RMS theory, if

, is schedulable. The highestpriority tasks and are harmonic and can be consideredto be a single task from the perspective of ’s schedulability.

Next, consider an arbitrary task , . Relative to rate-monotonic scheduling, an instance of encounters a maximumadditional delay of . Hence, the term canbe added to its computational time of , and RMS utilizationbounds can be used for testing feasibility.

A less pessimistic schedulability test utilizes the fixed-pointapproach to determine the worst-case response time of task .To find the worst-case response time of , let

where is an arbitrarily small unit of time. Iterate on asfollows:

until in which case the worst-case response timeof is . Check if . If , willmiss its deadline and the computation can be stopped.

Remark: A keen reader will note that Theorems 5–7 andthe schedulability conditions for energy-saving rate-harmo-nized scheduling do not require that bewhen . In fact, Lemma 3 andTheorem 4 are the only results that require this constraint.


Fig. 3. The task set � � �� , � � ��, � � �� with� �� being scheduled with RMS on the top, basic RHS in the middleand with Energy-Saving RHS on the bottom �� . This illustrates anexample of when the Energy-Saving RHS schedulability test is satisfied, andRHS is not able to optimally gain sleep cycles. In this example, the sufficientschedulability test was performed with � set to � . The sufficient schedu-lability conditions work if � � � always. Note, the total execution timeincluding sleep is not required to be less than � . In this case, � is preemptedafter 1 time unit and completes its execution in the next � period.

Otherwise, if other schedulability conditions are met, one canhave allowing for larger values of .

Fig. 3 illustrates the schedule of a task set where an increase inthe value of causes the basic RHS scheme to no longer beable to use all idle slots for deep sleep. However, energy-savingrate-harmonized scheduling is able to achieve 100% deep-sleeputilization of all idle slots by delaying yet-to-start tasks into thenext period. The addition of into the feasibility con-ditions does represent a scheduling penalty. However, for manyenergy-constrained systems like multihop sensor networks, thetotal utilization of the given task set is likely to be 20% or less,and any scheduling penalty only applies when task utilizationsare rather high.

A. Choosing the Period of the Energy Saver Task

The period of the Energy Saver Task plays akey role in determining the overall schedulability of the task set.A major constraint on is that it needs to be greater than

, where is the time taken by the pro-cessor to enter the deep-sleep mode. The analysis so far restricts

to equal the harmonizing period . If this condition isnot met, Theorem 5 will no longer be satisfied, and every idleduration is not guaranteed to put the resource into deep-sleepmode without timing penalties. However, for task sets that arenot schedulable with , larger values of canbe picked provided the Energy Saver Task is still assigned thehighest scheduling priority and is still harmonic to ,although optimal energy savings are no longer guaranteed.

For systems where is picked to be larger than the har-monizing period (still harmonic to ), the schedulabilitycondition given in Theorem 7 needs to be modified as follows:

For task sets with , it is not possible to provideany a priori guarantees on energy savings from the deep-sleepmode, since the highest-priority task would be released be-fore the processor can wake up from deep sleep. It should benoted here that it would not be feasible to pick any valuefor such task sets. On the other hand task sets with cumulativeutilization well below the utilization bound can be scheduledusing a value that satisfies the above schedulability condi-tions. Picking the minimum value that satisfies the aboveschedulability conditions would ensure maximum energy sav-ings. However, by picking , it is no longer guaran-teed that all the idle intervals would be converted to deep-sleepmode.

B. Energy-Saving RHS With Phase Exploitation

The analysis to date assumes very little about the phasingsof tasks except that . When the phasings ofother tasks are unknown, worst-case assumptions need to bemade. However, the admission criteria can be improved if theinitial phasings for all the tasks are known. With this knowl-edge, the maximum additional delay that a task encountersrelative to RMS scheduling can likely be reduced further below

. Let the LCM of and be . The relativephasings between multiples of and their nearest integral mul-tiples will repeat every time-units. The maximal delaybetween any arrival time of and its eligibility time is given by

This value can be used as the “blocking term” in the feasibilityconditions instead of . For example, if

, and , . The maximumdelay encountered by due to rate-harmonized scheduling isthen given by

, instead of . Hence, the


feasibility condition for task can be improved by a factor ofup to .

V. RATE-HARMONIZED SCHEDULING ON

MULTICORE PROCESSORS

Most modern multicore processors such as the Intel Core 2[25] and AMD Opteron [26] provide support for multiplelow-power modes called C-states. In these processors, the coreswithin each package can individually switch to idle states. How-ever, for these processors to switch into low-power deep sleepstates, all the cores need to simultaneously switch to idle states.This poses an interesting scheduling challenge since the idletime intervals across the processing cores are not guaranteed tooverlap. In these systems, Rate-Harmonized Scheduling helpsby providing the desirable property of clustering jobs to start atthe beginning of harmonized periods. When the harmonizingperiods are synchronized across the cores, the idle durations ineach processing core will terminate at synchronized harmonicperiods. This could significantly increase the possibility ofoverlapping idle intervals across processing cores.

Multicore scheduling algorithms for real-time systems aretraditionally classified into two categories: i) partitioned and ii)global scheduling. [27] shows that neither category strictly dom-inates the other. This paper focuses on partitioned schedulingsince global scheduling algorithms could require more intercoretask migrations, which result in poor cache performance. Parti-tioned scheduling algorithms employ bin-packing techniques toallocate tasks to processing cores. Uniprocessor admission con-trol tests can then be applied to ensure the schedulability of eachindividual core. In this work, we focus on partitioned sched-uling, where tasks are allocated to individual processing cores.

From the perspective of energy management, it is desirable tobalance the total load across the processing cores. Bin-packingheuristics such as worst-fit decreasing [28] can achieve goodload balancing properties. The Harmonizing Period is chosenglobally based on the entire task set. Time is assumed to besynchronized across the different processing cores.

Under basic rate-harmonized scheduling, the schedulabilityof each individual processing core can be determined using The-orem 4 and the accompanying response-time tests. Fig. 4 illus-trates the useful property of such basic rate-harmonized sched-uling in multicore processors where all the processor idle inter-vals will terminate at harmonized intervals, thereby increasingthe possibility of overlap among processor cores. For example,at time , both the cores are idle under Rate-HarmonizedScheduling, and the processor is able to transition to the deep-sleep mode, whereas with Rate-Monotonic Scheduling Core 1would be idle and Core 2 would be active.

Let denote the number of tasks allocated to processingcore . Let denote the tasks allocated toeach processing core ( to ), ordered in the increasingorder of periods, under worst-fit decreasing task allocation.

Energy-saving rate-harmonized scheduling can be extendedto multicore processors by creating a separate sleep task

for each individual processing core ( to forprocessing cores ). Observe that the idle

durations on each individual processing core are not necessarilyof the same length. Therefore, energy-saving rate-harmonized

Fig. 4. This figure shows the timeline of a dual-core processor running tasksets � � �� , � � ��, � � �� on core 1 and task sets � �

�� and � � �� on core 2 with a � � �. The uniform sleepingperiods across cores allows the entire processor to go to a lower power state.

scheduling as presented so far, is not optimal for multicore pro-cessors. However, energy-saving rate-harmonized scheduling

does guarantee that a sleep utilization of

is achieved.From the objective of maximizing sleep utilization, after per-

forming worst-fit decreasing allocation of tasks to processingcores, utilization bound tests from Theorem 7 can be employedto calculate the maximum value for each individualprocessing core ( to ).

is set at the global value for all the processingcores. The value for each processing core can be con-servatively determined as

If , multicore energy-saving rate-

harmonized scheduling can achieve a sleep utilization of

For systems with multicore processors, an alternative strategycould be to reduce the number of active processor cores using abin-packing heuristic like Best-Fit Decreasing and completelyshutting down the unused processor cores. An integrated ap-proach could determine the number of processor cores requiredusing best-fit decreasing, shut down the remaining processorcores and reallocate the tasks using worst-fit decreasing on theactive cores. Rate-Harmonized scheduling can then be appliedto achieve additional runtime power savings on the active pro-cessor cores. A detailed study of such an integrated approachforms an important component of our future work in this arena.


VI. PERFORMANCE EVALUATION OF

RATE-HARMONIZED SCHEDULING

In this section, the performance of rate harmonized sched-uling is compared in terms of sleep optimal efficiency andoverall impact on the power consumption of a system. The re-sults summarize the measured energy benefits from a currentlydeployed sensor networking application as part of the SensorAndrew project at Carnegie Mellon University.

Sensor Andrew is a multidisciplinary campus-wide scal-able sensor network that is designed to host a wide rangeof sensing and low-power applications. The goals of SensorAndrew are to support ubiquitous large-scale monitoring andcontrol of infrastructure in a way that is extensible, easy touse, and provides security while maintaining privacy. Targetapplications currently being developed include infrastructuremonitoring, first-responder support, quality of life for the dis-abled, water distribution systems monitoring and optimization,building power monitoring and control, social networking, andbiometric sensors for campus security. A large component ofthese applications is an underlying wireless sensor networkcomprised of the Nano-RK real-time operating system runningon the FireFly sensor networking platform.

Nano-RK is a fully preemptive reservation-based real-timeoperating system (RTOS) with multihop networking supportfor wireless sensor networks. It includes a lightweight em-bedded resource kernel (RK) with rich functionality and timingsupport capable of running on low-power microcontrollers.Nano-RK supports fixed-priority preemptive multitasking forensuring that task deadlines are met, along with support forCPU, network, as well as, sensor and actuator reservations.Tasks can specify their resource demands and the operatingsystem provides timely, guaranteed and controlled access toCPU cycles and network packets. Together, these resourcesform virtual energy reservations that allows the OS to enforcesystem and task level energy budgets.

In the current Sensor Andrew deployment, the FireFly nodesare battery operated and communicate over multiple hops toa powered gateway that has access to the Internet. The sensornetwork is primarily designed to efficiently collect sensing data,however, it also provides support for various mobile deviceinteractions. A generic communication interface is provided toallow the nodes to directly query infrastructure nodes as wellas send messages to and from the Internet via the gateway.Communication reservations in Nano-RK provide a mobilenode communication budget preventing mobile devices fromdraining more than their allotted system energy.

The current individual node functionality is supported by thefive tasks shown in Table I. The highest priority task consists ofa TDMA link layer with a period of 10 ms. This period is de-signed to support each communication slot, however the systemcan wait multiples of these periods as specified by a communi-cation schedule. The next set of tasks are responsible for man-aging network routing, sampling high frequency sensor data likethe audio sensor and recording runtime diagnostics. The diag-nostic task periodically collects information about the system’sruntime parameters and eventually writes this to an externalflash card. The diagnostic information consists of radio statis-tics as well as CPU runtime statistics. The radio statistics containnumber of transmitted packets, received packets, retries sending

TABLE ISHOWS THE WORST-CASE TASK SETS CURRENTLY RUNNING IN NANO-RK AS

PART OF THE SENSOR ANDREW PROJECT. DURING TYPICAL EXECUTION MANY

OF THESE TASKS ARE OPERATING AT MULTIPLES OF THE MINIMUM PERIODS.ALL EXECUTION TIMES ARE IN MILLISECONDS

TABLE IIMEASURED VALUES OF THE DIFFERENT SCHEMES GIVEN THE TASK SET IN

TABLE I. UNDER THE HIGHEST SYSTEM LOAD A 16.8% SAVINGS IS OBSERVED

WITH ENERGY-SAVING RHS AS COMPARED WITH RMS

a packet due to a dropped ACK and packet loss as well as av-erage signal strength values between neighbors. The CPU sta-tistics keep track of each tasks utilization as well as the timethe processor spends in idle as compared to deep sleep. Usingthese values, the calculated gain of using Energy-Saving RHS isshown in Table II. Under normal operation, many of these tasksare not operating at every period. For example, the High-Fre-quency (HF) Sensor Sampling Task would normally only exe-cute at this frequency when audio data is requested from thegateway. Given the worst-case situation when all of these tasksare executing (which does occur occasionally), Energy-EfficientRHS saves up to 16% as compared to RMS.

The experimental numbers are based on the ATmega1281processor. Table IV shows the corresponding parameters forother processors. In all of these cases, disabling the oscillatorto enter the deepest sleep mode consumes proportionally lessenergy than the processor’s idle mode. Even in cases where theidle energy of a processor is quite low, RHS can be used to fur-ther improve energy performance with little overhead.

A. RHS Performance

In this section, the various trends apparent in HarmonizedScheduling of tasks is described. A set of experiments wereperformed based on a large number of uniformly distributedrandom task sets. Each task set was given a period between 1and 200 time units and simulated over its entire hyperperiod.The Sleep Utilization Optimality metric is defined as the ratioof the total actual deep sleep time to the total non-busy time in ahyperperiod. A value of 1.0 indicates that all available idle timewas used for deep sleep.

In the first set of experiments, the parameter is ad-justed, and the impact of CPU utilization on the Sleep Utiliza-tion Optimality of the processor is studied. During these tests,the number of tasks was randomly selected to be between 5 and15. Observe that the smaller values tend to work reason-ably well with RMS because small intervals between tasks canbe converted to sleep time. If is set to 1, then all schemeswould perform identically. As shown in Fig. 5(a)–(c), as the

value increases from 10 to 50 the effectiveness of using


Fig. 5. Performance comparisons between RMS, RHS and Energy-Saving RHS. (a) Optimality of each scheme with respect to converting idle processor time intosleep time with a � value fixed at 10. Each point in the graph represents the average of 1000 simulated schedules. (b) Optimality of each scheme with respectto converting idle processor time into sleep time with a � value fixed at 20. (c) Optimality of each scheme with respect to converting idle processor time intosleep time with a � value fixed at 50. (d) The effect of increasing the number of tasks given a fixed � value of 15 and a fixed CPU utilization of 50%.V-RM is shown as a comparison to an existing static procrastination approach.

RHS becomes more prominent. We also see that as the work-load increases, the gain improves.

Fig. 5(d) shows how the number of tasks executing in thesystem also has an impact on the ability for the system to sleep.In this experiment, the value was fixed at 15 (the realvalue for the system) and the workload was fixed at 50% uti-lization. As the number of tasks increase, RMS begins to rapidlydeteriorate in performance while Energy-Saving RHS maintainsits optimal sleep utilization. As the number of tasks increases,the number of preemptions and likely phase offsets also in-creases. The difference between the schemes as the number oftasks in the system increases shows that Energy-Saving RHS ishighly scalable. We also present the V-RM scheme proposed in[15] for comparison with one of the state-of-the-art static pro-crastination schemes.

Fig. 6(a) shows the overall impact of the schemes with respectto CPU power consumption. This figure characterizes the designparameters from the FireFly v2.2 hardware. The active energyof the processor is 19.8 mW, the idle energy is 6.6 mW, and thesleep energy is 6.6 . The parameter is set to 15 ms.

Observe that with random task sets, even at a low-load the sav-ings can be quite significant, up to 39%. Table III shows an ex-ample where Energy-Saving RHS saves 33% of the total powerRMS would consume. As the workload increases, observe thatthere is not as much divergence of the lines as in the previousexamples due to the dominating active power term as the loadincreases. As the active energy in the system approaches the idleenergy, then RHS schemes will perform even better. This willbe a natural trend in microcontrollers since the silicon processtechnology is improving making the clock crystal a dominatingfactor in power consumption. Since sleep modes typically dis-able the oscillator, the energy consumption of the sleep modeshould remain significantly less than active and idle.

Fig. 6(b) shows the potential benefits of Rate-HarmonizedScheduling in a multicore context. We considered a hypothet-ical multicore processor made with Atmega1281 cores. When-ever all of the cores are in the idle state for longer than(15 ms), the entire processor can be switched into the deep sleepstate. In other cases, the individual cores will have to remain atthe idle state. We evaluated the per-core power consumption for


Fig. 6. Power performance on a single-core and multicore processor. (a) CPUpower consumption given the FireFly hardware parameters operating with thedifferent schemes. Each point in this graph is the average of 1000 randomlygenerated task sets with � between 5 and 15, period between 1 ms and 200 mswith a � fixed at 15 ms. (b) Average power consumption of each core ina multicore system with a fixed CPU utilization of 0.20. The CPU can onlytransition into deep sleep if all cores are concurrently able to sleep for �(15 ms).

TABLE IIISHOWS A TASK SET THAT WHEN � � �� AND � � ��, THE

POWER CONSUMPTION FOR A FIREFLY NODE WOULD BE 9.83 mW FOR

RMS AND 6.5 mW FOR ENERGY-SAVING RHS. IN THIS EXAMPLE,ENERGY-SAVING RHS CONSUMES 33% LESS TOTAL ENERGY

processors having 1 to 8 cores at a fixed per-core utilization of0.2. As seen in Fig. 6(b), increasing the number of cores re-duces the total sleep time under RMS. This is due to the factthat the probability of all the cores being idle simultaneouslygoes down as the number of cores increases. RHS can synchro-nize the task release intervals to harmonic boundaries, therefore,the possibility that the cores are simultaneously idle increases.

TABLE IVSHOWS THE ENERGY AND STATE TRANSITION TIMES OF VARIOUS

MICROCONTROLLERS. NOTE, MANY PROCESSORS HAVE MULTIPLE

OPERATING FREQUENCIES. THIS TABLE SHOWS AN ESTIMATE

OF A SINGLE SAMPLE OPERATING POINT

Energy-Saving RHS can schedule synchronously acrossthe processors, thereby further increasing the possibility of si-multaneous idling. Our results indicate that Energy-Saving RHSconsumes up to 40% less power than RMS on multicore proces-sors, when the per-processor utilization is around 20%. A moredetailed evaluation on other multicore platforms and workloadscenarios form a key part of our future work.

VII. CONCLUSION

Rate-Harmonized Scheduling (RHS) is a family of simpleyet powerful techniques to cluster task executions and lumpprocessor idle intervals. This serves as a novel but practicaland effective technique to maximize the percentage of time aprocessor spends in deep sleep without violating the timingconstraints of the given real-time task set. By adjusting thephasing between periodic tasks and removing the idle slotsbetween active execution that are too short for the processor tomake a round-trip transition from idle into deep-sleep, rate-har-monized schedulers increase the percentage of processor sleeptime. One particular instance of rate-harmonized schedulers,Energy-Saving RHS, has the property that every idle time-unitnot spent in active processing can be converted directly intodeep-sleep time for the processor. Not only does this improveenergy-efficiency, but it also simplifies the scheduler design.Rate-Harmonized Scheduling is then extended to also supportmulticore processors. Both a theoretical analysis and exper-imental evaluation of these schemes are given as applied tosensor networks where energy is highly important. For the taskset used in the Sensor Andrew project, Energy-Saving RHSsaves 16.8% energy compared to conventional Rate-MonotonicScheduling (RMS). For low utilization task sets, Energy-SavingRHS can save up to 39% energy compared to RMS on ran-domly generated task sets. Future work involves applyingRate-Harmonized Scheduling to other scheduling subsystemslike Disk Scheduling and Network Scheduling. A detailedanalysis of the performance benefits and practical applicabilityof Rate-Harmonized Scheduling in such subsystems needs tobe performed.

REFERENCES

[1] A. Rowe, K. Lakshmanan, H. Zhu, and R. Rajkumar, “Rate-harmo-nized scheduling for saving energy,” in Real-Time Syst. Symp. (RTSS),Dec. 2008.


[2] N. S. Kim, T. Austin, D. Baauw, T. Mudge, K. Flautner, J. S. Hu, M.J. Irwin, M. Kandemir, and V. Narayanan, “Leakage Current: Moore’sLaw Meets Static Power,” Computer, IEEE Computer Society, vol. 36,pp. 68–75, Dec. 2003.

[3] R. Rajkumar, S. Oikawa, and R. K. Portable, “A portable resourcekernel for guaranteed and enforced timing behavior,” in Proc. IEEEReal-Time Technol. Appl. Symp. (RTAS), 1999.

[4] A. Eswaran, A. Rowe, and R. Rajkumar, “Nano-RK: An energy-awareresource-centric RTOS for sensor networks,” in Proc. IEEE Real-TimeSyst. Symp., 2005.

[5] Y. Zhu and F. Mueller, “EDF scheduling of real-time tasks exploitingdynamic voltage scaling,” Real-Time Syst. J., vol. 31, pp. 33–63, Dec.2005.

[6] Y. Zhu and F. Mueller, “Exploiting synchronous and asynchronousDVS for feedback EDF scheduling on an embedded platform,” ACMTrans. Embedded Comput. Syst., vol. 7, pp. 1–26, Dec. 2007.

[7] B. M. Gordon, R. Gonzalez, and M. A. Horowitz, “Supply andthreshold voltage scaling for low-power CMOS,” IEEE J. Solid-SateCircuits, vol. 32, no. 8, 1997.

[8] R. Rajkumar and S. Saewong, “Practical voltage-scaling for fixed-pri-ority RT-systems,” in Proc. 9th IEEE Real-Time and EmbeddedTechnol. Appl. Symp. (RTAS), 2003.

[9] [Online]. Available: http://www.nanork.org/nano-RK/wiki/FireFly(viewed 5/20/2008)

[10] A. Woo, S. Hollar, D. Culler, K. Pister, J. Hill, and R. Szewczyk,“System architecture directions for network senors,” in Proc. ASPLOS,Nov. 2000.

[11] [Online]. Available: http://focus.ti.com/docs/toolsw/folders/print/z-stack.html(viewed 5/20/2008)

[12] J. Carlson, H. Dai, J. Rose, A. Sheth, B. Shucker, J. Deng, R. Han, H.Abrach, and S. Bhatti, “MANTIS: System support for multimodal net-works of in-situ sensors,” in Proc. 2nd ACM Int. Workshop on WirelessSensor Networks and Appl. (WSNA), 2003.

[13] R. Shea, E. Kohler, M. Srivastava, S. Han, and R. Rengaswamy, “SOS:A dynamic operating system for sensor nodes,” in Proc. 3rd Int. Conf.Mobile Systems, Appl. Services (Mobisys), 2005.

[14] C. L. Liu and J. W. Layland, “Scheduling algorithms for multipro-gramming in a hard real-time environment,” J. ACM, vol. 20, no. 1,pp. 46–61, 1973.

[15] J.-J. Chen and T.-W. Kuo, “Procrastination for leakage-awarerate-monotonic scheduling on a dynamic voltage scaling processor,”in Proc. ACM SIGPLAN/SIGBED LCTES, 2006, pp. 153–162.

[16] S. Irani, S. Shukla, and R. Gupta, “Algorithms for power savings,” inProc. 14th Annu. ACM-SIAM Symp. Discrete Algorithms, 2003, pp.37–46.

[17] R. Jejurikar and R. K. Gupta, “Procrastination scheduling in fixed pri-ority real-time systems,” in Proc. 2004 CM SIGPLAN/SIGBED Conf.Languages, Compilers, and Tools for Embedded Systems, 2004, pp.57–66.

[18] R. Jejurikar, C. Pereira, and R. Gupta, “Leakage aware dynamicvoltage scaling for real-time embedded systems,” in Proc. DesignAutom. Conf., 2004, pp. 275–280.

[19] L. Niu and G. Quan, “Reducing both dynamic and leakage energy con-sumption for hard real-time systems,” in Proc. 2004 Int. Conf. Com-pilers, Architecture and Synthesis for Embedded Systems, 2004, pp.140–148.

[20] J. Chen and T. Kuo, “Procrastination determination for periodic real-time tasks in leakage-aware dynamic voltage scaling systems,” in Proc.2007 IEEE/ACM Int. Conf. Comput.-Aided Design, 2007, pp. 289–294.

[21] J. Lehoczky, “Fixed priority scheduling of periodic task sets with arbi-trary deadlines,” in Proc. Real-Time Syst. Symp. (RTSS), 1990.

[22] B. Sprunt, L. Sha, and J. Lehoczky, Scheduling Sporadic and AperiodicEvents in a Hard Real-Time System, CMU/SEI-89-TR-11 CarnegieMellon University, 1989.

[23] M. Joseph and P. Pandya, “Finding response times in a real-timesystem,” Computer J., 1986.

[24] K. W. Tindell, “An extensible approach for analyzing fixed priorityhard real-time tasks,” Dept. Comput. Sci., Univ. York. York, U.K.,1992.

[25] [Online]. Available: download.intel.com/design/processor/datashts/320390.pdf

[26] [Online]. Available: support.amd.com/us/Processor_TechDocs/40036.pdf

[27] S. Lauzac, R. Melhem, and D. Mosse, “Comparison of global and par-titioning schemes for scheduling rate monotonic tasks on a multipro-cessor,” in Proc. 10th Euromicro Workshop on Real Time Syst., 1998,pp. 188–195.

[28] E. G. Coffman, Jr., M. R. Garey, and D. S. Johnson, ApproximationAlgorithms for Bin Packing: A Survey. Boston, MA: PWS Publishing,1997, pp. 46–93.

Anthony Rowe (M’07) received the B.S. degreein computer engineering from Carnegie MellonUniversity, Pittsburgh, PA, in 2003. He is currentlyworking towards the Ph.D. degree at the DepartmentElectrical and Computer Engineering, CarnegieMellon University.

His research interests include real-time systems,operating systems, wireless sensor networks, light-weight computer vision and robotics

Karthik Lakshmanan (M’09) received the B.E. de-gree in computer science from the College of Engi-neering Guindy of Anna University, Chennai, India,in 2006. He is currently working towards the Ph.D.degree at the Department of Electrical and ComputerEngineering, Carnegie Mellon University, Pittsburgh,PA.

His research interests include multicore com-puting, operating systems, real-time systems,resource management, embedded system and wire-less sensor networks.

Haifeng Zhu (M’05) received the Ph.D. degree in electrical and computer en-gineering from Carnegie Mellon University, Pittsburgh, PA, in 2005.

His research interests include real-time queueing theory, operating systems,real-time scheduling, resource management, and embedded systems.

Ragunathan (Raj) Rajkumar (SM’09) received theB.E. (Hon) degree from the University of Madras,India, in 1984, and the M.S. and Ph.D. degrees fromCarnegie Mellon University, Pittsburgh, PA, in 1986and 1989, respectively.

He is a Professor with the Departments of Elec-trical and Computer Engineering and of ComputerScience, Carnegie Mellon University. He was also theprimary founder of TimeSys Corporation, a vendor ofembedded Linux and Java products. He has chairedseveral international conferences, and has authored a

book and more than 90 publications in conferences and journals. His researchinterests include all aspects of embedded real-time systems as well as QoS sup-port in operating systems and networking.

rate-harmonized scheduling and its applicability to energy management

Documents