performance isolation in network and computing systems with

Performance Isolation in Network andComputing Systems with Multiple Inputs

Jack BrassilHP Laboratories, Princeton, NJ, USA

Email: [email protected]

Abstract— The growing use of virtualization technologies insettings such as multi-tenant compute and storage cloudschallenges us to both specify and enforce the isolation ofclients sharing network and compute resources. In thispaper we propose a novel analytical measure of performanceisolation for shared resource systems serving multiple trafficflows or computing workloads. By basing our isolationmeasure on well-known results from network calculus,we demonstrate how isolation metrics can be calculatedfor flows traversing both individual system elements andnetworks of those elements. We argue that this measure fa-cilitates the design of systems capable of ensuring that clientscan realize a specified isolation target. We present illustrativeexamples of how our quantitative isolation measure canbe used to compare the isolation properties of alternativeinstantiations of resource sharing systems ranging fromexperimental testbeds to dynamically-instantiated computeclouds. Finally, we show how next generation resourceallocators can be designed to preserve the isolation of clientsby either routing newly arriving flows, or re-arrange existingflows.

Index Terms— network calculus, deterministic queueing the-ory, security and privacy, QoS, fairness

I. I NTRODUCTION AND MOTIVATION

Consider a heavily utilized experimental networktestbed (e.g., Emulab [1], PlanetLab [2]) supporting multi-ple client experiments that must dynamically instantiate anetworked computing system based on a resource requestfrom a newly arriving client. Suppose that sufficient com-munications and computation resources are available suchthat the testbed’s resource allocation system identifiestwo feasible candidate system instantiations satisfying theclient’s resource demands. One candidate instantiationshares certain resources (e.g., link bandwidth, virtualmachines within a physical compute server) with oneset of existing test facility clients, while the secondcandidate implementation shares resources with anotherset of existing clients. Assuming that all client workloadsand traffic demands are heterogeneous and unknown,how does the resource allocator determine which of theimplementations will jointly offerall clients the maximumpossible performance isolation from each other?

Network systems researchers ability to answer thisquestion has been hampered by both the lack of a formaldefinition of performance isolation, as well as lack of con-sensus on how performance isolation should be measured.

Manuscript received August 14, 2011; revised October 5, 2011;accepted October 7, 2011.

Yet there is immediate urgency to confront this problem;elastic computing in emerging systems such as multi-tenant compute clouds require system designers to balancethe requirement to isolate clients and the desire to shareresources efficiently. By developing a method to specifyand enforce experiment isolation, clients of future networktestbeds such as GENI [3] can enjoy a greater degree ofexperiment repeatability, a highly desirable feature oftennot found in current generation testbeds.

A shared resource system should ideally provide eachclient a measure of its isolationfidelity, and meet aspecific quality-of-isolation guarantee. Further, in the ab-sence of mis-configuration or other failure, this systemisolation property should be demonstrable, if not prov-able. The demand for stronger, measurable isolation isgrowing most notably in virtualized settings (e.g., large-scale experimental testbeds, compute and storage clouds)where co-resident clients may have no pre-establishedtrust relationship, and may even have no knowledge of theexistence or identities of other clients. In such a setting,the risk of isolation failure is unknowable, and this riskis a barrier to acceptance of these new technologies.

The isolation properties of a system (or a systemelement) describe the relative ’apartness’ or ’separation’that the system offers to two (or more) concurrent sys-tem workloads.Performance isolationcharacterizes howeffectively a system maintains the performance separationof workloads while serving those workloads. An immedi-ate goal of our work is to draw attention to performanceisolation as a crucial, first-class property of resourcesharing systems, deserving of the same close attentionthe research community has paid to other resource sharingsystem properties such asfairness.

The absence of a theoretical framework to study theisolation properties of systems has left the evaluation ofisolation to be largely an empirical science. This currentstate of isolation metrology is troubling for system design;there exists no formal approach to analyzing isolation in acomplex system. As a result isolation can not be increasedsystematically, nor can the isolation of comparable sys-tems implementations be compared in a rigorous way. Weseek a theoretical framework for studying the isolationproperties of complex systems that would compliment thevaluable empirical work which is performed, to form acomplete isolation characterization.

In this paper we propose such an analytical frame-work for studying performance isolation. We do not seek

JOURNAL OF COMMUNICATIONS, VOL. 7, NO. 1, JANUARY 2012 39

© 2012 ACADEMY PUBLISHERdoi:10.4304/jcm.7.1.39-51

R1(t) R1*(t)E1

Figure 1. A network element with a single arriving and departing flow.

to consider the general problem of categorizing overallsystem isolation here – nor isolation in the presenceof system failures – but instead focus on the specificproblem of performance isolation. We build on resultspresented in earlier work [4] where we introduced ameasure of isolation for network systems with contendingflows. Here we extend that work to systems with anarbitrary number of contending workloads, and considerhow our analytical measures can be exploited by resourceallocators in experimental testbeds and clouds. We aspirenot just to categorize the performance isolation propertyof systems, but influence the system design process toincorporate design guidelines enabling a network systemdeveloper to realize a desired ’quality-of-isolation’.

This paper is organized as follows. The next sectionintroduces the reader to some of the basic conventions,approaches and results of the study of network calculus.We make no attempt to extend the theory itself, butinstead show how the theory of deterministic queuescan be used to measure performance isolation not justin networks, but more generally for complex computingand communication resource sharing systems. Section IIIintroduces our proposed measure of performance isolationfor simple network elements withN input flows. Nextwe summarize how the measure can be applied element-by-element to calculate an isolation measure for complexsystems. Sections V and VI present some challengingexamples of the application of this approach to exper-iment isolation in actual systems. Here we argue thatresource allocators can make informed decisions aboutthe relative isolation provided by alternative instantiationsof a requested experiment topology. The final sectionssummarize our contribution, discuss related work, andhighlight some opportunities for future work.

II. A NALYTICAL ISOLATION METRIC

To develop an analytical measure of performance isola-tion we begin by recalling some elementary results fromnetwork calculus; readers seeking additional backgroundare referred to the textbook by Leboudec and Thiran [5]– whose conventions we adopt here – or early work byCruz [6] [7].

Figure 1 depicts a system elementE1 with a singlearriving and departing flowR(t) andR∗(t). For the extentof this manuscript we limit our discussion to elements that

are lossless, well-behaved, and work-conserving; we willreturn to discuss these initial assumptions in Sections VIIand VIII. Such a system element could model a commu-nication or network element, where arriving and departingflows correspond to bit or packet streams (i.e., networktraffic), or a computing element where workloads describenewly arriving and completed tasks (i.e., compute jobs).For the moment, we will suppose that we are modeling anetwork element – and refer to flows and traffic – whilerecognizing that the approach also models the behavior ofshared computing elements. Unless otherwise noted, wewill restrict our attention to flows that are non-decreasingand described by a fluid model (i.e., continuous functionsin continuous time).

Suppose that flowR(t) is constrained by an arrivalcurveα(t) such that for all times ≤ t we have

R(t) − R(s) ≤ α(t − s). (1)

The arrival curve serves as an envelope for the admittedtraffic; we say thatR(t) is constrained by an arrival curveα(t) iff R ≤ R⊗α, where the operator⊗ is the min-plusconvolution given by

R ⊗ α(t) , inf0≤s≤t

{R(t − s) + α(s)}. (2)

We will frequently use the common and important set ofaffine arrival curves given by

α(t) = γr,b(t) ,

{

rt + b t ≥ 00 otherwise.

(3)

Flows that have been regulated by the well-known leakybucket traffic shaper with drain rater and bucket sizebfollow the arrival curveγr,b(t) (see Proposition 1.2.3 of[5], pg. 11).

We say that the elementE1 offers a service curveβto the arriving flow iff the departing flow satisfiesR∗ ≥R⊗β. A common service curve is the rate-latency curveβR,T given by

βR,T (t) ,

{

R (t − T ) t ≥ T0 otherwise.

(4)

Two key results we will exploit from network calculus arethe well known upper bounds on the backlog and delayof any α-constrained flow arriving to a lossless systemproviding a service curveβ. The backlog for allt is givenby

R(t) − R∗(t) ≤ sups≥0

{α(s) − β(s)}. (5)

The delay through the system at any timet is boundedby

δ(t) ≤ sups≥0

{ infτ≥0

{τ : α(s) ≤ β(s + τ)}} (6)

The following example helps make these results moreaccessible and intuitive. First, if anα-constrained ar-rival process arrives to a network element providing aminimum guaranteed service, then clearly the backlog

40 JOURNAL OF COMMUNICATIONS, VOL. 7, NO. 1, JANUARY 2012

© 2012 ACADEMY PUBLISHER

(t)!(t)

data (bits)

time (sec.)T

delay " T+ b/R

slope = R

slope = r

backlog"b+rT

b

Figure 2. The largest horizontal and vertical deviations between arrivaland service curves establish worst case delay and backlog.

and worst case delay will both be bounded. Suppose,for instance, that a flow constrained by an arrival curvefollowing γr,b(t) passes though a network element (e.g.,leaky bucket) with service curveβR,T (t), with R ≥ r.Figure 2 depicts such a result, and shows the graphicalrepresentation of the arrival and service curves.

While not shown in the figure, the arrival processis guaranteed to stay beneath the envelopeα, and thedeparture process above the envelopeR⊗β. The backlog(Eq. 5) corresponds to thevertical deviationbetween thearrival and service curves, with a maximum backlog of

b + RT bits. (7)

The delay corresponds to the horizontal deviation betweenthe arrival and service curves, which is bounded (Eq. 6)by

T +b

Rseconds. (8)

We pause to reinforce the critical idea we applythroughout this manuscript. Informally stated, Eq. 6 statestha given a system element that provides a minimumguaranteed service to a properly constrained arriving flow,we can readily write an upper bound on the delay sufferedby any packet passing through the element. In somecases such as Eq. 8, the delay bound can be both tightand simple to write analytically. We will use this delaybound as our preferred measure of system performance,and show how to derive from it an isolation measure forsystems serving two or more workloads.

III. PERFORMANCE ISOLATION WITH

MULTIPLE INPUTS

Let’s begin by considering the general case of theperformance isolation of a single workload from a set ofother concurrent workloads. Consider a network elementEN with N arriving flows ~R = {R1, R2, ..., RN}. LetIRi, ~Rx

be the measure of isolation that the element offers

R1(t)

R2(t)

R1*(t)

R2*(t)

ENRN(t) RN

*(t)

… …

Figure 3. A network element withN arriving and departing flows.

flow Ri(t) from a subset of other flows~Rx ⊂ ~R−, where~R− is the set of all arriving flows other than theith flow

(i.e., ~R = ~R− ∪ {Ri}).Let D

Ri, ~Rx(D

Ri, ~Rx=~0) be the maximum delay suf-

fered by theith flow given the presence (absence) of theflows in ~Rx. Define theperformance isolationof Ri fromthe subset of other flows~Rx as

IRi, ~Rx

, 1 −D

Ri, ~Rx− D

Ri, ~Rx=~0

DRi, ~Rx

(9)

=D

Ri, ~Rx=~0

DRi, ~Rx

. (10)

Hence, our performance isolation metric is based onthe difference in worst case delay seen by a markedflow in the presence and absence of a set of concurrent,interfering flows. In a work conserving system we have

DRi, ~Rx

≥ DRi, ~Rx=~0

≥ 0 (11)

so clearly we have

0 ≤ IRi, ~Rx

≤ 1. (12)

Alternately, we define theexposure1 of Ri to ~Rx as thecomplement of the isolation, i.e.

ERi, ~Rx

, 1 − IRi, ~Rx

. (13)

In general, completely reporting the performance isola-tion metrics of flowi traversing elementEN with N − 1concurrent flows requires specifying an(N − 1)!-tupleof isolation measures against each possible subset offlows. Fortunately, we will rarely find that level of detailnecessary; in practice we are typically interested in aflow’s isolation from eitherall other flows, or a specificconcurrent flow.

Since we will be examining the case ofN concurrentflows let’s take a moment to clarify how we treat flowsthat are concurrent with the flows on which we aremeasuring isolation. In a system with 3 or more inputswe seek to measure the isolation of theith flow against aproper subset of the remaining flows; that is, we arenot

1Usingexposurehelps us overcome the common but inconvenient useof the expressionlack of isolation.



logically turning off all other flows (i.e.,| ~Rx| < | ~R−|).Hence, since some inputs are active, they will naturallyaffect the measure of system isolation provided to theother inputs. This will be made more clear with anexample in a moment.

Discussion:Eq. 10 is inspired by the ad hoc, empiricalisolation measures where a performance measure (e.g.,throughput) of one workload is taken in the presenceand absence of one (or more) other workloads, withthese typically chosen to degrade the performance of thefirst workload to the largest degree. As we mentioned inSection I, a difficulty with this approach is the inability toknow that the chosen interfering workload is in fact theone that would degrade performance the most. Indeed,even if such a workload was correctly chosen, actuallyapplying it in a system setting might be difficult orimpractical; simpler, easily generated workloads are oftensubstituted in practice instead. Here, however, we makeno attempt to hypothesize what such a workload wouldbe, and instead use an analytically derived bound on worstcase delay forall competing workloads conforming to aspecified arrival curve.

Let’s pause to immediately address a potential issuewith our isolation measure. Many readers are aware thata common objection to the use of delay bounds derivedfrom network calculus is that – regardless of whether thebounds are tight – they arepessimistic. Here, however, ourisolation measure relies on a ratio of delay bounds, andEq. 12 establishes that the ratio is normalized. Perhapsmore importantly for the study of isolation, exploitingnetwork calculus enables our isolation measure to holdunder a wide class of inputs. In contrast, our ability togeneralize results from empirical assessments is oftenquite limited.

Representing system behavior through arrival and ser-vice curves as depicted in Fig. 2 enables us to arrive at apowerful, graphical interpretation of flow isolation. Fig.4graphically illustrates how our isolation metric is derivedas a consequence of what is effectively a change in theelement’s service curve. Under a lightened workload theshape of the element’s service curveβ deforms toβ∗. Foran affine curve, the minimum delay may reduce fromT toT ∗ and the service rate may increase fromR to R∗. Thesecond term of Eq. 14 is simply a normalized measure ofthe resulting decrease in worst-case delay, as labeled∆in the figure.

Before moving on to examples of the analysis ofsystems withN flows, let’s first focus on the simplestcase of an element with 2 inputs, i.e.,~R = {R1, R2}.Eq. 10 is simply a normalized measure of the decreasein a flow’s worst case delay bound that can be achievedby removing the second flow from the system. If thereis no decrease in delay, we say that the first workloadis (performance) isolated from the2nd workload, i.e.,IR1,R2

= 1. If we have IR2,R1= 1 and IR1,R2

= 1,we say that the two workloads are isolated from eachother. In practice, achieving such a result is impossible ina practical system sharing resources efficiently, though we

(t)

!(t)data (bits)

time (sec.)T

delay* " T*+ b/R*

slope = Rslope = r

b

T*

slope = R*

!*(t)

delay " T+ b/R

Figure 4. Under a lightened workload the element’s service curve mayeffectively shift. The reduction in worst delay∆ is the basis for ourisolation metric.

strive to design systems with the largest possible isolationmetric for each workload.

From Eq. 10 we write theperformance isolationof R1

from R2 as

IR1,R2, 1 −

DR1,R2− DR1,0

DR1,R2

(14)

=DR1,0

DR1,R2

. (15)

Here we takeDR1,R2to be the bound on the delay of

R1 in the presence ofR2, andDR1,0 to be the bound ondelay given the absence of the second input (R2 = 0).Hence, our performance isolation metric is the differencein worst case delay seen by a flow in the presence andabsence of the interfering flow.

Completely reporting the performance isolation prop-erties of elementE2 requires specifying bothIR1,R2

andIR2,R1

, where

IR2,R1=

DR2,0

DR2,R1

(16)

describes the isolation of flowR2 from flow R1.

A. Priority Queueing System

Now let’s consider an example of determining theanalytical performance isolation of flows in a system with3 inputs. Suppose we have a non-preemptive priority nodewith constant rateC serving 3 input flows – high, mediumand low priority flows labeledRH , RM andRL. Servicefor the high priority flow takes precedence; a packet inthe middle priority queue is only serviced if the higherpriority queue is empty, and a packet in the low priorityqueue is only serviced if both higher priority queues areempty. Suppose thatlMAX is the largest packet size forany flow, and that the high, medium and low priority flowsare constrained to beγrh,bh

(t), γrm,bm(t) andγrl,bl

(t).



RM(t)

RL(t)

RH*(t)

RL*(t)

C

RH(t)

RM*(t)

Figure 5. A non-preemptive priority queuing system with 3 inputs.

Following the proof of the 2 flow non-preemptivepriority node (see Proposition 1.3.4 of [5], pg. 21), theappendix sketches a proof that the high, medium and lowpriority flows are guaranteed rate-latency curves of

βC,

lMAXC

, high priority (17)

βC−rh,

lMAX+bhC−rh

, and medium priority (18)

βC−rh−rm,

bh+bmC−rh−rm

. low priority. (19)

Hence the delays seen by each flow are given by Eq. 8as

DRH ,{RM ,RL} =lMAX

C+

bh

C(20)

=lMAX + bh

C. (21)

DRM ,{RH ,RL} =lMAX + bh

C − rh

+bm

C − rh

(22)

=lMAX + bh + bm

C − rh

. (23)

DRL,{RH ,RM} =bh + bm

C − rh − rm

+bl

C − rh − rm

(24)

=bh + bm + bl

C − rh − rm

. (25)

Let’s now write two isolation measures for this system.Suppose we seek to find the isolation of the mediumpriority flow from both the high and low priority flows(i.e., I

RM , ~Rx= IRM ,{RH ,RL}). In the absence of the

higher priority and lower priority flows, the delay of themedium priority flow is given by settinglMAX → 0,bh → 0, andrh → 0 in Eq. 22, yielding

DRM , ~Rx=~0

=bm

C. (26)

The isolation of the medium priority flow from the 2 otherflows is found by inserting Eqs. 22 26 into Eq. 10 to find

IRM ,{RH ,RL} =C − rh

C

bm

lMAX + bh + bm

. (27)

Note that the isolation of the medium priority flow isheavily dependent on the rate of the high priority flow

(rh), as we expect. Though difficult to see in this ex-pression, the medium priority flow is strongly isolatedfrom the lower priority flow, depending only on that flow’smaximum packet length.

Now let’s consider the isolation of the low priority flowfrom just the medium priority flow. From our definitionin Eq. 10 and Eq. 24 we have

IRL,{RM} =C − rh − rm

C − rh

bh + bl

bh + bm + bl

. (28)

Note that the isolation of the low priority flow fromthe medium priority flow is heavily dependent onbothhigher priority flows, through the medium priority flow’sexposure to the high priority flow.

B. PGPS Queueing System

Let’s next look at the isolation performance of a net-work element that promises to offer similar isolation toeach flow. Due to its importance in assuring quality-of-service and achieving fair bandwidth sharing, consider anode with fixed service rateC employing a PacketizedGeneral Processing Scheduling (PGPS) discipline. Letthe maximum packet length of any flow beL packets.Suppose that theith arriving flow is allocated a rateci = C φi

P

j φj: i = 1, 2..., N . Then the element offers

each arriving flow a service curveβci,LC

( [5], p. 68). Ifthe ith arriving flow is constrained byγri,bi

, then we canemploy Eq. 8 to find that the maximum delay theith flowsuffers is given by

L

C+

bi

ci

. (29)

Suppose that there are a fixed numberN > 2 flowstraversing the element, and each is backlogged andgranted an identical rateci = C

N, with arriving flow rates

satisfying ri < ci. Then the isolation offered betweenflows i andj is

LC

+ biC

N−1

LC

+ biCN

≈N − 1

N. (30)

Hence, the isolation a PGPS node offers each individualflow from any other single flow approaches unity (totalisolation) as the (fixed) number of flows increases.

Since the PGPS server shares bandwidth among back-logged flows, we observe that the isolation of flowsiandj is affected by the presence of other flows. Supposethat at any instant there arek active flows, k < N ,each with an identical bandwidth share (i.e.,φi ≡ φ =Ck, k = 1, 2, ..., N ). First, let’s mark one flow for

consideration, themth flow. Suppose that the flow initiallypasses through the server withN − 1 other flows, andN−k−1 flows vanish.2 The marked flow initially receivesa rate ofC

N, but then receives a higher rateC

k. Though still

2We have not formally defined what it means for a flow to vanish.To be consistent with our definition, the setR~x remains unchanged, butN − k − 1 flows are no longer receiving service.



receiving a ’fair share’ of server resources, the resourceassigned to the flow increases, and clearly the flow isnot isolated from other flows. By our measure the systemoffers the marked flow an isolation of

LC

+ bmCk

LC

+ bmCN

≈k

N. (31)

This result, where changes to a PGPS node to improveservice clearly worsen the isolation between flows, vividlydepicts the tradeoff to be made between performanceisolation and system performance in shared resourcesystems.

IV. A PPLICATIONS TONETWORKED

COMPUTING SYSTEMS

Up to this point we have focused on defining a measureof performance isolation, and applying this measure tosimple system elements in isolation with two or moreapplied demands. But the strength of a metric based onnetwork calculus lies in its applicability to the analysis ofsystems comprising many such elements [7].

In particular, thenode concatenation theorem[5] tellsus that a series connection of two elementsEx andEy with service curvesβx and βy offers an aggregateservice curve ofβx ⊗ βy. If the aggregate service curveis known, then the isolation provided by the aggregateelement can be written directly from Eq. 10. But whatif the aggregate’s service curve is unknown? It followsthat the worst-case delay of a flow passing through asequence of elements – and hence the isolation – can beupper bounded by the sum of the delay (or isolation) ofthe individual elements. Note, however, that the sum ofthe isolation metrics of the two elements may represent alarger value than if the isolation metric could be writtendirectly from the aggregate’s service curve.

The concatenation theorem tells us that a bound onisolation of the aggregate system does not depend on theorder in which the elements are connected. Hence, theconcatenation theorem lays the groundwork for analyzingthe isolation performance of systems, and the first resultavailable to us is that the order of elements in an aggregatecan be interchanged without affecting the overall isolationproperty. Though a seemingly simple observation, thisrepresents the firstdesign rulewe can apply to the studyof system isolation.

Two other techniques we borrow from the study ofdeterministic queueing systems will also be invaluable.First, for the purposes of calculating the isolation of asystem, we can logically replace a nonlinear elementwith service curveβNL with a linear elementβL with adominatingservice curve, which we define to be a servicecurve satisfying

∫ s

0

βL ⊗ R(t) dt ≥

∫ s

0

βNL ⊗ R(t) dt, (32)

for all time s. Second, when we encounter an elementwith unknownservice curve, we can logically replace it

with an element with a known but dominating servicecurve for the purposes of calculating system isolation.

The final tools required in our system performanceisolation calculation toolkit is the identification of thespecifiedworkflow of interest, and the selection of anaggregation algorithm. The workflow is the set of comput-ing and communication elements servicing the workload.For example, consider a single client-server transaction(e.g., a file upload). The corresponding workflow mightcomprise task execution on a multi-tasking client com-puting ’element’, followed by a flow traversal of multiplenetwork elements with potentially interfering cross-traffic,followed by task execution at a web server also occupiedwith numerous other transactions. Suppose that for eachof N elements visited by the workflow we can calculatethe isolationI

Ri, ~Ryi

of the specified flowRi (arriving to

the ith element) relative to some distinct subset of otherarriving workloads~Ry. We then identify a measure of theworkflow’s end-to-end isolation by combining or aggre-gating individual measures at each element traversed. Forexample, one possible metric is the linear combination ofthe exposure of the specified workflow at each elementvisited, i.e.,

N∑

i=1

ai · (1 − IRi, ~Ryi

), (33)

where 0 ≤ ai ≤ 1, i = 1, 2..., N . That is, we takeas our measure of system isolation the weighted sum ofisolation measures of the workload passing through eachvisited element. Hence, when comparing the isolation oftwo alternative system implementations, the implemen-tation offering the lowest composite exposure would bepreferred.

Note that the decision to form a composite ’figure ofmerit’ by using Eq. 33 is flexible and can be determinedby application needs; any alternate composition algorithmmight be appropriate. For example, another natural ap-proach would be to aggregate the exposure at computeelements and communications elements separately, andcompare similar systems using both measures. Irrespec-tive of the aggregation approach selected, obtaining anend-to-end isolation measure allows us to focus on a de-sign intended to minimize exposure. Indeed, we can nowidentify anisolation budgetfor a system, and consider theselection of elements that differ in certain properties (e.g.,cost, flexibility) that meet our overall isolation budget. Insum, we are now armed with means of designing systemsto meet a desired isolation fidelity.

Let’s next consider the isolation of experiments oncomplex systems such as shared resource experimentaltestbeds. In the following examples, we highlight thepower of an isolation metric to influence overall systemdesign to maximize the isolation enjoyed by system users.

V. USING ISOLATION MEASURES INRESOURCE

ALLOCATORS

Resource allocation systems in experimental networkcomputing systems testbeds such as Emulab and Plan-



etLab receive asynchronous client requests to instantiateexperimental system topologies. To simplify allocatorimplementation and speed the identification of a fea-sible set of compute and network resources, a client’sresource specification (rspec) has historically been keptlean. Clients often lack controls (or hints) to specificallyforce the allocator to include or reject certain components,or indicate preferences for details such as VM placement.For example, Amazon’s EC2 [34] technical literatureargues that a desire to simplify system instantiation war-rants a client’s lack of VM placement control. Further,today’s allocators (e.g., Emulab’sassign) use admissioncontrol liberally; inability to map a resource request toavailable hardware in a congested sytem leads to rejectionof the client’s entire request. Designers of allocationsystems for future testbeds (e.g., GENI) envision richerand more expressive client specifications, with perhapsthe ability for clients to negotiate partial or less favorableinstantiations. For example, a client might be amenableto an incomplete system request fulfillment (e.g., duringa development or debugging phase) rather than suffer acomplete rejection.

On the other hand, Emulab and emulab-based sys-tems (e.g., DETER [8]) enable experimenters to achievea relatively high degree of inter-experiment isolationthrough a combination of admission control, permittingthe allocation of dedicated end computing systems to asingle experiment, and carefully limiting the use of sharedresources to effectively achieve an acceptable ’quality-of-isolation.’ The most common example of the latter isthe way that Emulab’s control infrastructure dynamicallycreates multiple VLANs to enable one or more ethernetswitches to segregate traffic associated with differentexperiments.

Our next goal is to explore how isolation measures canenhance the articulation of client preferences in rspecs,and enable resource allocators to evaluate candidate in-stantiations offering varying measures of performanceisolation. By carefully creating models of testbed ele-ments, we can examine the degree of isolation betweenexperiments, and in particular assess the relative degree ofisolation offered in comparable but different implementa-tions.

A. Flow Placement

Let’s begin by considering the simplest example of howisolation measures can be used for the selection of a sin-gle, preferred shared element in a testbed. Consider a newexperiment request requiring the use of a shared networkelement (e.g., a trunked VLAN link, a router egress link).Suppose that exactly two such shared network elementsare available for assignment by the resource allocator;switch 1 has an egress link capacityC1, and switch 2has an egress link capacityC2. Assume that each switchis being partially used by exactly one pre-existing flow,R1 and R2, with each flow associated with a differentexperiment.

Suppose that our model for the shared network link ineach experiment is a non-preemptive priority queue; thiswould be an acceptable model of an element such as aswitch or router port that is QoS-enabled. Let’s say thatthe pre-existing flows are assigned as the high-priorityflows in each system, and the new flow we seek to assignto either switch will be a low-priority flow. Let the flowsR1 andR2 be constrained asγr1,b1 andγr2,b2 . Supposethat the flow assigned by the new arrival isRN thatfollows γrN ,bN

. For convenience let us assume that allflows have the same maximum packet lengthl.

The flow placementproblem is as follows: to whichswitch should we assign the newly arriving flow tominimize its isolation from pre-existing flows? Figure 6depicts the placement problem. Clearly the new flow isonly exposed to the flow in the element chosen (i.e., notexposed to the other switch’s flow). It is easy to showthat for a 2-flow non-preemptive priority element the highpriority flow is isolated from the low priority flow as

IRH ,RL=

bh

l + bh

, (34)

while the low priority flow is isolated from the highpriority flow as

IRL,RH=

bl

bl + bh

C − rh

C. (35)

Eq. 35 tells us that the newly arriving flow will mini-mize its isolation from existing flows by being assignedto switch 1 if

bN

b1 + bN

C1 − r1

C1

<bN

b2 + bN

C2 − r2

C2

, (36)

and switch 2 otherwise. In the simple case where both pre-existing flows have the same burstiness and each of thetwo shared links have identical capacity, Eq. 36 confirmsour intuition that isolation of the new flow is minimizedby assigning it to the switch whose existing flow has thelowest rate. Hence, this simple example demonstrates thatfuture resource allocators can calculate isolation metricsfor both current and newly arriving experiments, andinstantiate a system – where feasible – to meet or maintaina client’s desired quality-of-isolation.

B. Flow Re-arrangement

As experiments are de-instantiated in experimentaltestbeds, resources are returned to an available resourcepool. In principle these resources are allocatable to bothnew resource requests associated with experiments toarrive in the future, as well as to existing experiments.Hence it is possible to engineer resource allocators tomodify existing instantiations of experiments to reduceisolation. Consider the following simple example; twoclients each use a virtual machine on a single physicalprocessor. If a second physical processor becomes avail-able, a VM could be migrated to that newly available,unused processor, lowering the exposure of the experi-ments’ workloads to each other.



Switch 1

R1 !r1, b1

Switch 2

RN

C2 bits/sec

C1 bits/sec

R2 !r2, b2

?

Figure 6. An envisioned resource allocation system uses isolationmeasures to assign a flow to a resource to minimize the newly arrivingclient’s exposure to existing flows.

In practice such actions are rarely taken; implementingsuch a resource allocator is difficult, and re-instantiationof experiments risks disrupting a running experiment.Hence, re-instantiations are generally limited to thosecases where an experimenter explicitlymodifiesan ex-isting experiment, an action typically done while theexperiment is halted.

A potentially less disruptive approach to maintainingnetwork isolation does not involve a complete experimentre-instantiation, but the simpler step of rearranging flowsto exchange shared resources with other already instan-tiated resources that promise less exposure to existingworkloads. In some cases such a modification mightinvolve only reconfiguring existing VLANs, an operationthat can be completed with minimal disruption.

Let’s consider a simple example. Let 2 identicalswitches each represent a resource to be shared by 6flows associated with 6 distinct experiments. Suppose thateach switch is well-modeled by a PGPS queueing system,and each flow is identically constrained and seeks anidentical bandwidth and quality-of-isolation from a sharedswitch. If 3 flows are initially assigned to each switch,then each flow on each switch is identically isolated fromthe other 2 flows. Now suppose that 2 flows on one switchvanish. While this leaves the remaining flow fully isolatedfrom the flows on the other switch, those flows sufferrelatively worse quality-of-isolation. Re-routing one flowfrom the switch with 3 flows to the switch with 1 flowclearly rebalances the system load, returning all flows toan identical isolation profile.

Though trivial, this example illustrates that flow re-arrangements can be performed with minimal disruptionin complex systems, where a resource allocator calculatesthe effect of rebalancing flows such that isolation can bepreserved even as the system’s future workloads evolve.

VI. A PPLICATIONS TOEXPERIMENTAL TESTBEDS

We next develop isolation models for very simplebut useful elements such as VLANs and traffic shapers,and show how these models can be combined to makeisolation-awareresource allocation decisions in simplesystems. Consider a resource allocator for an experi-mental testbed (e.g., Emulab) that seeks to instantiate

2 experiments using common switching and computeresources (e.g., VLANs on shared switches, VMs onshared physical servers). Now suppose the resource al-locator seeks to compare the isolation offered to eachclient by two alternative, feasible instantiations of theexperiments labeledA and B, with measures~IA and~IB. How should these be compared? In principle eachexperimenter might indicate as part of her system re-source specification (rspec) a request for a desired systemisolation profile. For instance, such a specification mightindicate a preference for maintaining stronger isolation incomputing elements rather than in networking elements.Ultimately, of course, the preferred approach to satisfyingclient isolation requirements would likely be domain-specific, and ideally negotiated with each client.

Let’s next consider how we can apply our isolationmeasure to practical network elements, and an elementarynetwork of those elements.

A. Modeling a VLAN-enabled Switch

Many experimental testbeds (e.g., Emulab, DETER[8]) enable experimenters to achieve a relatively highdegree of inter-experiment isolation by a combinationof admission control (i.e., rejecting new requests whenresources are unavailable), permitting the allocation ofdedicated end computing systems to a single experiment,and carefully limiting the use of shared resources to ef-fectively achieve an acceptable ’quality-of-isolation.’Themost common example of the latter mechanism is the waythat Emulab’s control infrastructure dynamically createsmultiple VLANs to enable one or more ethernet switchesto segregate traffic associated with different experiments.

Let’s begin by considering the simplest of Emulabshared resource systems; Figure 7 depicts two exper-iments each with two identical, dedicated computingsystems. Each experiment has a trivial network topology– a single duplex link connecting the 2 dedicated com-puters in each experiment. Each experimental topologyis instantiated as a separate VLAN on a single, sharedethernet switch, with each computing system connecteddirectly to the switch with a link of equal transmissionrate. Most designers of large scale systems simply acceptthat this configuration offerstotal performance isolationbetween experiments (i.e., no performance degradation ofone workload due to the presence of the second workloadon the switch); certainly any degradation is immaterialin most situations. Here we will show, however, that incertain cases a closer examination of VLAN performanceisolation is merited.

Since each source and destination computing system isidentical and physically separated, they do not contributeto the exposure of workloads to each other. Further, sinceeach experiment has a dedicated switch link, each flowconforms to some affine arrival curveα, for example,with a rate equal to the link transmission rateS. Theswitch, however, is a shared resource, regardless of howinsignificantly it exposes one workload to another.



As long as the switch has adequate internal bandwidthto process the arriving and departing flows, to the firstorder each VLAN is likely well modeled by an indepen-dent element characterized by a server with rateS anda very small, fixed delayT . That is, each VLAN can bemodeled by an element with service curveβS,T as definedby Eq. 4. Hence, a flow on either VLAN is unaffectedby flows on the other VLAN (i.e.,DR1,R2

= DR1,0 =DR2,0) and so the isolation given by Eq. 14 is unity (i.e.,complete isolation ofR1 and R2 from each other). Ofcourse, detailed knowledge of switch implementation andoperation could lead to a more sophisticated model of theVLAN element.

Next suppose that the client’s request for a duplex linkis in fact implemented by the interconnection of two ormore switches using a single trunk shared by both VLANsas Fig. 7 shows. This instantiation – while less preferablethan that using a single switch – is a relatively commonoutcome in systems where compute nodes are selectedfrom different racks, and a single experiment spans 2 top-of-rack switches and possible other aggregation switches.Due to the limited capacity of the trunk relative to theingress access links, a simple fixed delay would no longerbe the best model of the switch element, since it poten-tially fails to capture the interaction of network trafficbetween experiments sharing the trunked link. That is, theVLAN trunk ensures ’reachability’ but not performanceisolation. Instead, a preferred element model for a trunkedVLAN would be a fixed rate node serving two arrivingflows with a scheduling discipline and buffer capacitydetermined by the switch implementation.

One such model could be 2 queues served by a singleserver with fixed service rateS and Packetized GeneralProcessing Scheduling (PGPS) service discipline. Sup-pose that the arrival ratesR1 and R2 satisfy R1 < c1,R2 < c2, and c1 + c2 < S. Let the maximum packetlength of any flow beL packets. A PGPS server offersthe ith flow in the set ofW currently backlogged flowsa rateci = S φi

P

j φj: i = 1, 2, ..., W , and rate0 to flows

with no backlog. The element offers each arriving flow aservice curveβci,

LS

( [5], p. 68). If theith arriving flowis constrained byγri,bi

, then we can employ Eq. 8 to findthat the maximum delay theith flow suffers is

DRi,Rj=

L

S+

bi

ci

: i = 1, 2 : j = 1, 2 : i 6= j (37)

when both flows are present, and

DRi,0 =bi

S: i = 1, 2 : j = 1, 2 : i 6= j (38)

when only either flow is present. By applying our isolationdefinition from Eq. 14, a PGPS node offers flowi isolationfrom a second flow of roughlyci

S, or 1/2 for two flows

equally sharing the entire server bandwidth.Of course, it is obvious that a system instantiation inter-

connecting 2 switches with a trunked VLAN potentiallyoffers less performance isolation than one that does not;no model is needed to tell us that. But by assigning an

E2*

E2

vlan1vlan1

vlan1vlan1

vlan2

vlan2 vlan2

trunk

vlan2

switch

switch

switch

Figure 7. Two experiments each with a separate network implementedwith a VLAN on a single shared switch (upper), and two switchesinterconnected by a single link VLAN trunk (lower).

isolation measure we can nowquantifythis potential lossof isolation. As we seek to compare the performanceisolation properties of ever more complicated systems, wewill find having such a measure imperative.

Now let’s turn to the question of how this simpleelement isolation measure could be used in a resourceassignment process. First, note that certain system designrules-of-thumb emerge immediately, such as:

Given two otherwise identical implementations,a design in which the workflow traversesmoreswitches offers relatively less performance iso-lation, even in the case where each VLANelement is independently characterized by aservice curveβS,T .

This simple fact has implications in the overall designof experimental testbeds such as Emulab. If isolationwere the only factor to consider, a system designer mightchoose to arrange a particular mix of computing systemson each rack (each connected to a ’top-of-rack’ switch)to increase the likelihood that the system’s resourceallocator would contain a higher fraction of experimentsto a single rack (and hence single switch), in an effortto minimize system isolation by avoiding experimentstraversing multiple switches.

But more interesting is the joint optimization of iso-lation across multiple experiments. Suppose a resourceallocator must assign a newly arriving flow to one of 2switches, each offering a VLAN trunk to be shared withan existing flow. In one switch the existing flow is an’elephant’, and in the second switch the existing flow is a’mouse.’ How do we assign the flow associated with thenewly arriving resource request?

In essence, the penalty for exposing an existing work-load to an newly arriving workload is shared by bothexperiments. For the element modelled by a PGPS node,the isolation loss is greater for the mouse than the ele-phant. Of course, pairing the new flow with the elephantplaces the burden of greater exposure on the new flow.The preferred solution depends on the desired quality-of-



isolation of each experiment; if both experiments couldtolerate the reduction in performance isolation, then ad-mitting this resource assignment might jointly satisfy theneeds of the two experimenters.

B. Tne Node Selection Problem

Let’s next consider the application of our performanceisolation measure to the use of a PlanetLab node by mul-tiple slices (i.e., experiments). The problem of selectingapreferred set of nodes to host a slice is familiar to manyresearchers. We will next show that if an experimenteris seeking to achieve the highest possible experimentisolation, the selection of the preferred end systems needno longer be arbitrary, or be dictated by greedy overpro-visioning.

Suppose the experimenter may choose a subset ofend systems from a set ofheterogeneousend systemswith different system clock rates, and NIC transmissionspeeds, but otherwise identical network interconnection.Given knowledge of the number of active slivers on eachnode, the experimenter can select the subset of nodes thatminimizes the aggregate isolation metric for their experi-ment. Such an approach offers the potential for improvedoverall system utilization relative to existing, more ad hocnode selection approaches, such as experimenters alwayspreferring nodes with faster CPUs.

To illustrate, we sketch a rudimentary model of aPL node. PlanetLab uses a collection of technologies toensure a satisfactory degree of isolation between slivers(i.e., a node’s experiment container), including:

• PlanetLab Virtualized Network Access (VNET+)[10] to associate inbound and outbound packets withslices, and

• Linux Hierarchical Token Buffers (HTBs) to ratecontrol outbound traffic associated with each slice,and

• Linux Vservers to provide container-based sliverexecution environments.

The PlanetLab CPU scheduler grants each sliver a fairshare of the node’s available CPU service, while the HTBscheduler provides each sliver a fair bandwidth share(with minimum rate guarantee) for each sliver’s outboundpackets.

Now let’s examine a considerably simplified exam-ple motivated by the PlanetLab node selection problem.Suppose an experimenter wishes to deploy a softwarerouting function within the sliver on a prospective node.To calculate the node’s performance isolation, we proposethe following simple first-order model of a system with 2network elements. Suppose that the node hostsN activeslivers, and that that arrivals to the routing sliver areconstrained. Let’s choose a queue served at rateS bya PGPS server to model the CPU sharing by the activeslivers in the node. Hence we can derive the isolationmeasure of this first element servingN simultaneousslices from Eqs. 14, 37.

Similarly, the rate controller would be well modeledby a network element with a rate-latency curve, with

delay determined by the number of communicating slicessharing the interface, and rate determined by the slice’sbandwidth capR (i.e., the slice’s HTBceiling rate); thisservice curve is described by Eq. 4, and the correspondingdelay by Eq. 8.

Hence, the performance isolation of a candidate nodeis represented by 2 element measures. The first metricdepends on the CPU performance of the node throughS, and the second depends on the NIC speed throughR. When considering alternative, heterogeneous nodes, anexperimenter (or resource allocation system) may chooseto combine these metrics or treat them separately. Toperform a node selection based exclusively on a nodeisolation metric, this pair would be calculated for eachprospective node, and a node would be chosen whichsupports the experimenter’s desired isolation profile.

Though the node selection problem is motivated bya resource allocation problem present on systems suchas PlanetLab, we stress that considerable further work isneeded to apply the results more generally to complexexperimental testbeds. For example, our ability to modelan uncontrolled network connection between PlanetLabslivers on two separate nodes is limited. Of course, aprincipal objective of the PlanetLab system is environ-mentalrealism, not network isolation. TheVINI [13] sys-tem, however, does introduce additional network controls,including ’virtual links’ and the ability to introduce andcontrol emulated network elements. Here it seems thatit might be possible to calculate end-to-end metrics toexpress slice isolation.

In closing our discussion of applications, we note thatthis elementary treatment of complex network systemsleaves significant room for additional modeling work.Some more advanced work is available to us; the treat-ment of time-varying leaky buckets [5] would be helpfulin analyzing isolation in highly dynamic elastic systems.But perhaps the greatest challenge we face comes from thefact practical systems arelossy, while we have consideredonly lossless systems. We leave the application of knownresults for lossy systems ( [5], Chapter 9) for future work.

VII. I SOLATION AND FAIRNESS

Performance isolation is closely related to – but distinctfrom – the property offairnessbetween concurrent flowsin networks. A fairness measure expresses how wella given flow realizes a designated target allocation ofresources in the presence of other flows; equal sharingof a resource (e.g., bandwidth) is often the target. A flowis said to be treated unfairly if it receives a lesser sharethan its target.

The preferred objective of bandwidth sharing algo-rithms has long been debated [18], thoughmin-maxfairness [19] (i.e., maximizing the rate of the smallestflow under capacity constraints) has arguably been themost common objective. Proportional sharing [20], whichseeks to maximize an aggregate utility of a set of rateallocations, is a second common objective. Measuring



fairness in dynamic settings – with flows coming andgoing – continues to be an area of considerable interest.

In contrast, while performance isolation also expressesa relationship between concurrent flows, it measures howone flow is affected by both the presenceand the absenceof another; flow dynamicity is explicitly required for themeasure.

A set of flows might receive a fair bandwidth allocation,yet in general will not be isolated from each other. Tosee this, consider a system in which each ofN flowseach receive a targeted1/N

th share of a single serverwith fixed capacity. The system would be considered fair(under most measures), and this assessment would beindependent of any fixedN . But such a system doesnot isolate flows (according to our measure); the isolationof any two of theN flows changes as the total numberof flows change, while the fairness of the resources theyreceive does not. While fairness measures establish thatthe server treats each flow equally, a performance isolationmeasure captures the fact that the flows share the serverat all.

Conversely, a system can isolate flows from one anotherbut not fairly allocate resources. As a practical example, asystem that routes each of two flows along disjoint pathsisolates them from each other, but may offer no assuranceof fair bandwidth sharing if each path presents differentcross-traffic. In summary, fairness and performance isola-tion are distinct system properties, each informing us of apotentially interesting and revealing aspect of a resourcesharing system.

VIII. R ELATED WORK

The study of isolation as a property in itself has beencentered in the systems security community. Carroll [28]discusses 6 broad isolation strategies relevant to computersecurity – spatial, temporal, cryptographic, selection ofprocessing mode (e.g., online vs. offline processing), priv-ilege restriction, and those isolation capabilities achievedindirectly from system architecture (i.e., design). Roughlystated, Carroll’s definition of isolation is that one usershould be unable to impart change on a second user’sprocessing.

In earlier work Goguen and Meseguer [32] developeda formal model and analysis of information flows and theconcept ofnon-interference, with an emphasis on securitypolicies and models. Bishop [33] discussed isolation inthe context of the confinement problem [30] [29], andLampson introduced a definition oftotal isolation.

In contrast, our work has focused specifically on per-formance isolation. Work on performance isolation per-vades many communities interested in resource sharing,particularly the network systems and computing systemscommunities. Here a variety of empirical measures arerichly used as effective proxies for isolation. Quantitativemeasures of isolation appear to be most prevalent inthe electrical devices and circuits communities. Morerecently, some research groups have used ideas inspiredby network calculus to study the performance isolation

provided by algorithms in the presence of misbehavingflows [24]. Wachs, et al [23], have introduced the conceptof performance insulationin shared storage systems, andhave sought to design practical systems that maintain atarget throughput efficiency in the presence of competingworkloads.

IX. CONCLUSION

Performance isolation is a property of resource sharingsystems. Exposure – the complement of isolation – mea-sures the affect of a given workload (or set of workloads)on other workloads. In this paper we introduced a formaldefinition of performance isolation, defined a quantitativemeasure of the performance isolation between workloads,and showed how to apply network calculus as a methodfor calculating the performance isolation of multiple com-peting workloads in complex resource sharing systems.We contend that isolation merits study as a first-classproperty of resource sharing systems alongside other well-studied properties such as fairness.

We have also argued that conventional, empirical mea-sures of an individual element’s performance isolationbehavior are inadequate for the design of systems capableof meeting a desired quality-of-isolation requirement.Instead we have proposed an analytical measure of anelement’s isolation capability based on the theory of deter-ministic queueing systems. By combining isolation met-rics of elements forming a system, we have showed thatthe isolation properties of distinct system implementationscan be analyzed and compared, enabling implementors toconstruct systems that conform to their target isolationbudget. Further, the metrics can also be used to addresshow exposure can be minimized by modifications to aparticular system design.

We believe that such a rigorous approach is also neededto address other types of system isolation beyond justperformance isolation. For example, the confidentiality ofa workflow might be described by creating a measure ofthe observability of a flow at each element visited (e.g.,measured, say, in bits/time from a specified vantage point)and aggregating those metrics along the workflow’s path.Given the need to clarify the isolation capabilities of a vastnumber of new and proposed multi-client computing andcommunication resource sharing systems (e.g., computeclouds, experimental cyber testbeds), the further studyof isolation as a fundamental system property remainscritical. Simply adhering to conventional security bestpractices [27] – while appropriate in relatively staticenvironments – is difficult or impractical to apply insuch dynamic settings, and fall short of our need toground system design in an underlying theory of workflowisolation.

The emergence of the notion that isolation is a first-class system property that needs quantitative measuresmirrors early discussions of fairness. In the early 1980’san abundance of fairness metrics were introduced [14][15] [16]. To be applicable to end-to-end paths, manyof the proposed fairness metrics were based on delay,



and specifically delay variation. Dissatisfied with thesemeasures, Jain [17] proposed an alternative metric calledthe Index of Fairness, designed to meet a set of propertiesappropriate for a metric. In commenting on the immaturityof the state of fairness measures, Jain observed:

”These papers divide (flow control) algorithmsinto two categories: fair or unfair. It is notpossible to measure the fairness of a particularalgorithm.”

”Quantitative measure of fairness are not wellknown. Most studies on fairness tend to beeither qualitative of too specific to a particularapplication.”

”It is clear that fairness implies equal allocationof resources, although there is little agreementamong researchers as to what should be equal-ized. In computer networks, some want delay,others want throughput, and yet others want ...”

Many strikingly similar statements – with the termiso-lation now replacingfairness– are being echoed todayas the research community wrestles with the challengingproblem of defining and quantifying performance isola-tion.

REFERENCES

[1] Emulab, http://www.emulab.net[2] PlanetLab, http://www.planet-lab.org[3] Global Environment for Network Innovations,

http://www.geni.net[4] J. Brassil, “Analyzing Flow Isolatation in Shared Resource

Systems,” Proc. of ICCCN 2011, August, 2011.[5] J-Y. Le Boudec, P. Thiran, “Network Calculus: A

Theory of Deterministic Queuing Systems for theInternet,” Lectures Notes on Computer Science 2050,Springer, New York, 2001. Online edition (v4):http://ica1www.epfl.ch/PSfiles/netCalBookv4.pdf

[6] R. L. Cruz, “A Calculus For Network Delay, Part I:Network Elements In Isolation,”IEEE Transactions onInformation Theory, vol. 37, no. 1, pp. 114-131, Jan. 1991.

[7] R. L. Cruz, “A Calculus for Network Delay, Part II:Network Analysis,” IEEE Transactions on InformationTheory, vol. 37, no. 1, pp. 132-141, Jan. 1991.

[8] Deterlab, http://www.isi.deterlab.net[9] K. Sklower and A. Joseph, “Very Large Scale Cooperative

Experiments in Emulab-Derived Systems,”DETER Com-munity Workshop on Cyber Security Experimentation andTest 2007, Boston, August 2007.

[10] M. Huang, “VNET: PlanetLab Virtualized NetworkAccess,” PlanetLab Design Note PDN-05-029, June 2005,http://www.planet-lab.org/files/pdn/PDN-05-029/pdn-05-029.pdf.

[11] S. Bhatia, “VNET+: Associating Packets withSlices,” PlanetLab Miscellaneous Documentation,http://www.cs.princeton.edu/ sapanb/vnet/.

[12] M. Huang,“PlanetLab Bandwidth Limits,”PlanetLabMiscellaneous Documentation, http://www.planet-lab.org/doc/BandwidthLimits.

[13] A Bavier, N Feamster, M Huang, L Peterson, J Rexford,“In VINI Veritas: Realistic and Controlled Network Ex-perimentation,”Proc. of ACM SIGCOMM’06, 2006.

[14] J. Sauve, J.W. Wong, J. Field, “Fairness in Packet-Switching Networks,”Proc. of CompCon’80, WashingtonDC, pp. 466-470, 1980.

[15] M. Gerla, M. Staskauskos, “Fairness in Flow ControlledNetworks,” Proc. of ICC’81, Denver CO, 1981.

[16] M. Marsan, M. Gerla, “Fairness in Local ComputingNetworks,” Proc. of ICC’82, Philadelphia PA, 1982.

[17] Raj Jain, Dah-Ming Chiu, William Hawe, “A QuantitativeMeasure of Fairness and Discrimination for ResourceAllocation in Shared Computer Systems,”DEC TechnicalReport 301, 1984.

[18] L. Massoulie, J. Roberts, “Bandwidth Sharing: Objectivesand Algorithms,”IEEE Trans. on Networking, vol. 10, no.3, June 2002.

[19] E. Hahne, “Round-robin Scheduling for max-min Fairnessin Data Networks,” IEEE JSAC, vol. 9. pp. 1024-1039,1991.

[20] F. Kelly, A. Maulloo, D. Tan, “Rate Control for Commu-nication Networks: Shadow Prices, Proportional Fairness,and Stability,”J. Operations Research Soc., vol. 49. pp237-252, 1998.

[21] M. Karlsson, C. Karamanolis, X. Zhu, “Triage: Perfor-mance Isolation and Differentiation for Storage Systems,”Twelfth IEEE International Workshop on Quality of Service(IWQOS 2004), June 2004, pp. 67-74.

[22] S.R. Seelam and P.J. Teller , “Fairness and PerformanceIsolation: an Analysis of Disk Scheduling Algorithms,”2006 IEEE International Conference on Cluster Comput-ing, Barcelona, Sept. 2006.

[23] M. Wachs, M. Abd-El-Malek, E. Thereska, G. Ganger,“Argon: Performance Insulation for Shared StorageServers,”Proc. 5th USENIX Conf. on File and StorageTechnologies (FAST’07), Feb. 2007, San Jose, CA.

[24] Ajay Gulati, Arif Merchant, Peter J. Varman, “pClock:an arrival curve based approach for QoS guarantees inshared storage systems,”Proceedings of ACM SIGMET-RICS 2007, Vol. 35, No. 1, June 2007, pp. 13-24.

[25] R. Farrow, LAN Isolation,http://www.spirit.com/Network/net0103.html

[26] D. Pollion, M. Schiffman, “Secure Use of VLANs: An@stake Security Assessment,”@stake Research Report,August 2002.

[27] Cisco Systems, “Cisco SAFE Reference Guide,”Cisco Document OL-19523-01, August 2009.http://www.cisco.com/en/US/docs/solutions/Enterprise/Security/SAFERG/SAFErg.pdf

[28] John Millar Carroll, “Chapter 16: Isolation in ComputerSystems,” Computer Security, 3rd Ed., 1996.

[29] B. Lampson, “A Note on the Confinement Problem,” 1973.[30] B.W. Lampson, “Dynamic protection structures,”Proc.

AFIPS 1969 FJCC, Vol. 35, AFIPS Press, Niontvale, N.J.,pp. 27-38.

[31] Schroeder, NI.D., and J.H. Saitzer, “A Hardware Archi-tecture for Implementing Protection Rings,”Comm. ACM,vol. 15, no. 3, (Mar. 1972), pp. 157-170.

[32] J. Goguen, J. Meseguer, “Security Policies and SecurityModels,” Proceedings of the 1982 Symposium on Securityand Privacy. Oakland, 1982.

[33] M. Bishop, Computer Security: Art and Science, PearsonEducation Co., Boston, 2003.

[34] Amazon Elastic Compute Cloudhttp://aws.amazon.com/ec2

X. A PPENDIX

We next present a proof of 22; the proofs of 24 and 20follow similarly.

Consider some timet, and let some earlier timesbe the start of a server busy period. Service for themedium priority queue can be delayed by at most a singlepacket that arrives earlier on the low priority queue, or aconforming burst of packets arriving on the high priority



flow. Suppose that packets arrive to the low, then high,then medium priority queues in quick succession at timess, s′, ands′′, with s < s′ < s′′. In this case a single packetin the low priority queue must complete service, followedby the backlog in the high priority queue, followed finallyby the medium priority queue. During the interval(s, t]the server’s output isC(t − s). During the interval(s, t]the output of the medium priority queue is

R∗M (t) − R∗

M (s) = C(t − s) − lMAX − R∗H(t) − R∗

H(s)(39)

But at the beginning of the backlog period the totalbits on arriving and departing flows are identical, i.e.RM (s) = R∗

M (s) andRH(s) = R∗H(s), so we have

R∗M (t) − RM (s) = C(t − s) − lMAX − R∗

H(t) − RH(s)(40)

But the high priority flow is constrained toγrh,bh(t),

so we have

R∗M (t)−RM (s) ≤ [C(t − s) − lMAX − rh + bh(t − s)]

+

(41)

= (C − rh)(t − s) − lMAX − bh (42)

which is equivalent toβC−rh,

lMAX+bhC−rh

.

Using Eq. 8 with this service curve and the constrainton the arrival flow to the medium priority queue yieldsthe corresponding delay for the delay given by Eq. 22.�

Jack Brassil received the B.S. degree from the PolytechnicInstitute of New York in 1981, the M.Eng. degree from CornellUniversity in 1982, and the Ph.D. degree from the Universityof California, San Diego, in 1991, all in electrical engineering.Dr. Brassil has been with Hewlett-Packard Laboratories since1999. He currently is a Research Scientist and Program Managerin Princeton, NJ. Before joining HP he held multiple researchpositions at Bell Laboratories in Murray Hill and Holmdel,NJ. Dr. Brassil is an IEEE Fellow, a member of the IEEECommunications Society, a member of the ACM, and a memberof ACM SIGCOMM.



performance isolation in network and computing systems with

Documents