A DISTRIBUTED QOS ROUTING ARCHITECTURE FOR SCALABLE VIDEO STREAMINGOVER MULTI-DOMAIN OPENFLOW NETWORKS

Hilmi E. Egilmez∗, Seyhan Civanlar †, A. Murat Tekalp∗

∗College of Engineering, Koc University, Istanbul, Turkey†Argela Technologies, Istanbul, Turkey

ABSTRACT

This paper proposes a new Quality of Service (QoS) opti-mized routing architecture for video streaming over large-scale multi-domain OpenFlow networks managed by a dis-tributed control plane, where each controller performs opti-mal routing within its domain and shares summarized intra-domain routing data with other controllers to reduce problemdimensionality for calculating inter-domain routing. We ap-ply the proposed architecture to streaming of scalable (lay-ered) videos, where the base layer routes are dynamicallyoptimized to fulfill a required QoS level, while enhancementlayers follow traditional shortest path. We show that the pro-posed solution approaches the expensive non-scalable glob-ally optimal solution (single controller for the whole network)in terms of received video quality under various congestionscenarios.

Index Terms— Video streaming, scalable video, QoSrouting, OpenFlow network, distributed optimization

1. INTRODUCTION

Streaming media applications require stringent delay guaran-tees with little or no packet losses which cannot always bemet by the best-effort Internet. In order to provide Quality ofService (QoS), Internet Engineering Task Force (IETF) hasproposed several QoS architectures such as IntServ [1] andDiffserv [2], but none has been truly successful and widelyimplemented. This is because they are built on top of currentInternet’s best effort hop-by-hop routing architecture, missingthe broader picture of overall network resources. AlthoughMPLS [3] provides a partial solution, it lacks real-time recon-figurability and adaptivity.

OpenFlow is a programmable network architecture [4]that decouples control (routing) and forwarding (data) layersof routing. It shifts the control function of routing to a centralunit, called controller, while forwarding function remainswithin the routers; also called forwarders. OpenFlow alsoenables defining different types of flows where different set

∗ This work has been partially supported under the FP7 Project SARA-CEN. A.Murat Tekalp also acknowledges support from Turkish Academy ofSciences (TUBA).

of rules can be associated with each predefined flow. Con-troller is the brain of the network where the routing decisionsare made on a per-flow basis, and updates forwarding tables,called flow tables, associated with each flow to inform theforwarders how to direct traffic flows as depicted in Fig.1.

OpenFlow will allow network service providers to offerinnovative video services with dynamically reconfigurableQoS options and network virtualization, and has already at-tracted the attention of many commercial vendors [4]. Yet, thecurrent OpenFlow [5] only supports networks with a singlecontroller which is not scalable. FlowVisor [6] provides aninterface for virtual multiple controllers but it is for managingmultiple network slices within the same network domain. Asthe size/number of OpenFlow networks increase, the singlecontroller architecture is not scalable to manage the wholenetwork because of two main reasons: First, a single con-troller may not be able to update flow tables of all forwardersin time due to limited processing power and latency intro-duced by physically distant forwarders. Second, there wouldbe a large volume of traffic towards the controller due to mes-saging between controller and all forwarders. Therefore, it isessential to implement a distributed control plane supportingmultiple controllers. In the literature, there are distributedcontrol plane designs such as Onix [7] and HyperFlow [8],but none provide an overall network-wide QoS architecture.

In our past work, we proposed dynamic QoS routing forscalable video streaming over OpenFlow networks, wherewe have assumed that a single controller has full access toall link state information (not feasible for large networks)to determine the globally optimum routes [9]. This paperextends that work to large-scale OpenFlow networks man-aged by a distributed control plane in which each controlleris responsible for its dedicated intra-domain QoS routingand exchange messages with other controllers to help inter-domain QoS routing decisions. In the remainder of the paper,Section 2 discusses intra and inter domain QoS routing formulti-domain OpenFlow networks. Section 3 defines thedistributed dynamic QoS routing problem and introducesthe proposed solution. Section 4 discusses our simulationenvironment and presents simulation results comparing theproposed distributed approach with the non-scalable globaloptimum solution. Section 5 draws conclusions.

Fig. 1: OpenFlow architecture

(a) (b)Fig. 2: A sample multi-domain OpenFlow network: (a) com-plete network view, (b) aggregated version of the network

2. QOS ROUTING ARCHITECTURE FORMULTI-DOMAIN OPENFLOW NETWORKS

In order to ensure optimal end-to-end QoS, collecting up-to-date global network state information, such as delay, band-width, and packet loss rate for each link, is essential. Yet,over a large-scale network, this is a difficult task because ofdimensionality. The problem becomes even more difficultbecause of the distributed (hop-by-hop) architecture of thecurrent Internet. The current Internet’s state-of-the-art inter-domain routing protocols such as BGP-4 are hop-by-hop, andtherefore not suitable for optimizing end-to-end QoS. Open-Flow eases this latter point by employing a centralized con-troller. As illustrated in Fig.1, instead of sharing the stateinformation with all other routers, OpenFlow forwarders di-rectly send their local state information to the controller us-ing the OpenFlow protocol. Controller processes each for-warder’s state information and recomputes the best feasibleroutes using up-to-date global network state information.

However, the single controller solution in the currentOpenFlow specification is not scalable to large scale multi-domain networks. Therefore, there is need for a distributedcontrol plane with multiple controllers so that each controlleris responsible for a part (domain) of the network. In addi-tion, there is also need to implement a controller-to-controllerinterface that allows a logically centralized control planemanaging the overall OpenFlow network.

In the following, we propose a new simplified (aggre-gated) architecture for QoS routing over multi-domain Open-Flow networks. Fig.2(a) illustrates a sample OpenFlow net-work with multiple domains. The filled and unfilled dotsstand for forwarders (nodes) and border forwarders (bordernodes) respectively. There are two types of links which areinter-domain, and intra-domain links. In order to reduce prob-lem size, we propose to aggregate the original network by re-placing the intra-domain links by a set of completely meshedvirtual links between border forwarders that are also the end

points of inter-domain links as shown in Fig.2(b).Our proposed solution is based on the following premises:• Each domain is managed by a single controller which

is responsible for intra-domain routing and advertisingits domain’s state information to other controllers.• Inter-domain routing is calculated over an aggregated

version of the real network by a logically centralizedcontrol plane.• Before finding the inter-domain route, necessary cost

parameters of each virtual link summarizing the net-work state information has to be calculated, as dis-cussed in Section 3.• After an inter-domain route is found, each controller

optimizes its intra-domain routing by replacing the vir-tual links with actual links.• Both intra and inter domain QoS routes are found by

solving the optimization problems stated in Section 3.The key step that allows scalability is the proposed aggrega-tion of the intra-domain network information. Obviously, net-work aggregation introduces some imprecision on the globalnetwork state information, but this is tolerable and necessaryto obtain a scalable routing solution. We implicitly evaluatethe effect of topology aggregation in Section 4.

3. DISTRIBUTED OPTIMIZATION OF QOSROUTING

In this section, we pose the general QoS routing problem asa Constrained Shortest Path (CSP) problem and extend it forthe proposed QoS routing architecture discussed in Section 2.For the CSP problem, it is crucial to select a cost metric andconstraints where they both characterize the network condi-tions and support QoS requirements. Since our focus is videostreaming, we choose our QoS indicators as packet loss anddelay variation (jitter).

In our formulation, the global network, aggregated net-work and the global network without inter-domain links (i.e.,union of domains) are represented as directed simple graphsGg(Ng, Ag), Ga(Na, Aa), Gd(Nd, Ad), respectively. Ng ,Na, Nd are the set of nodes and Ag , Aa, Ad are the set ofarcs (links) in each graph. The set of virtual links is definedas Av ⊂ Aa. Note that, Ng = Nd ⊃ Na. We define the arc(i, j) as an ordered pair, which is outgoing from node i andincoming to node j and R(s, t) (subset of set of arcs) denotesthe set of routes from source node s to destination node t. Forany route r ∈ R(s, t) we define cost fC and delay variationfD measures as,

fC(r) =∑

(i,j)∈r

cij , fD(r) =∑

(i,j)∈r

dij (1)

where cij and dij are cost and delay variation coefficients forthe arc (i, j), respectively. The CSP problem can then be for-mally stated as,r∗ = arg min

r{fC(r) | r ∈ R(s, t), fD(r) ≤ Dmax} (2)

that is, finding a route r which minimizes the cost functionfC(r) subject to the delay variation fD(r) to be less than orequal to a specified value Dmax. In our case, we choose thecost metric as the weighted sum of packet loss measure anddelay variation as follows,cij = (1− β)dij + βpij for 0 ≤ β ≤ 1,∀(i, j) ∈ Ag (3)

where pij denotes the packet loss measure for the traffic onlink (i, j), β is the scale factor. The parameters pij and dij arenothing but the network state information that we discussed inSection 2. So, it is crucial that forwarders return up-to-dateestimates in order to find the precise QoS route. OpenFlowenables us to monitor the traffic statistics on a per-flow ba-sis and the controller can collect these statistics whenever itrequests [5].

The CSP problem (2) is known to be NP-complete, sothere are heuristic and approximation algorithms in the liter-ature. We propose to use the Lagrangian Relaxation BasedAggregated Cost (LARAC) algorithm which is a polynomial-time algorithm that efficiently finds a good route without de-viating from the optimal solution in O([m+ nlogn]2) [10].

In our proposed QoS routing framework, solution to intra-domain routing is straightforward. Since, each controller hasfull access to all physical links and their state information,it directly solves the CSP problem using LARAC for givensource and destination, then the QoS flows, SVC base layerpackets in our case, are forwarded accordingly. On the otherhand, inter-domain routing is not that trivial, because stateinformation is not readily available for the virtual links in theaggregated network. So, the QoS indicating network stateparameters (i.e. cij and dij) of each virtual link has to be setcleverly so that it summarizes the network state inside of eachdomain, which is not directly seen by the control plane. Then,inter-domain routing with QoS becomes feasible.

For our problem formulation, we modify the CSP problemand define the CSP problem instance as follows,P (G, (i, j)) =

arg minr{fC(r) | r ∈ R(i, j) ⊆ A, fD(r) ≤ Dmax} (4)

where G and (i, j) are the arguments of the problem instance.G represents the network and (i, j) is the ordered pair wherei and j stand for source and destination nodes. A is the set ofall arcs in G and R(i, j) is the set of all paths from node i toj. For example, P (Gg, (s, t)) is equal to the problem statedin (2). We propose two methods to select required parametersfor virtual links in the aggregated network:• Method-1: For every virtual link (i, j) ∈ Av , the con-

troller finds the best feasible path r∗ij between bordernode pair (i, j) within the domain by solving the prob-lem instance P (Gd, (i, j)). Then, the total cost and thedelay variation of r∗ij are assigned to the correspondingparameters of the virtual link between border node pair(i, j) that is cij = fC(r∗ij) and dij = fD(r∗ij).

• Method-2: For every virtual link (i, j) ∈ Av ,the con-troller finds k-disjoint best feasible paths r∗1 , r

∗2 , . . . , r

∗k

between border node pair (i, j) within the domain bysolving CSP problem k times. Then, the average costsand delay variation of paths r∗1 , r

∗2 , . . . , r

∗k are assigned

to the corresponding parameters of virtual link.After setting the cost and delay variation parameters of virtuallinks using one of the methods above, it is now possible tocalculate QoS routes. We formulate the QoS routing problemin two steps given in (5) and (6) in terms of the CSP probleminstances stated in (4),

First Step: r∗a = P (Ga, (s, t)) (5)

Second Step: r∗ =⋃L

l=1 P (Gg, r∗a(l)) (6)

where the first step formulates the inter-domain QoS routingbetween source (s) and destination (t) over the aggregatednetwork. The route r∗a denotes the best feasible inter-domainroute. The second step uses the result from the first step andformulates the end-to-end QoS routing. The route r∗ denotesthe complete QoS route where r∗a(l) is the lth arc (orderedpair) of the route, r∗a, and L is the number of arcs in r∗a.Note that, each problem instance above can be solved usingLARAC algorithm [10].

4. RESULTS

In order to simulate the proposed QoS routing optimizationframework we implemented a simulator by using the networkoptimization library LEMON [11] which has efficient opti-mization algorithms (including LARAC) for combinatorialoptimization problems with graphs and networks.

The network topology we used in our simulations has 6domains connected as shown in Fig.2. Each domain has 30nodes, which is randomly designed using GT-ITM tool [12].Hence, the overall network size is 180 nodes. The bordernodes are also selected randomly. We set all intra-domainlink capacities as 150 Mbps and inter-domain link capacitiesas 1 Gbps. The cross traffic (congestion) on each link is mod-eled as an independent Poisson random process which is agood model for bursty nature of the Internet. Also, during thesimulation runtime the statistics of each link may change de-pending on the state of the domain where it belongs. The stateof each domain is modeled as a two state Markov chain whichdecides whether the domain is in good or bad state. The linkdelays are modeled as Γ-distributed random variables withmeans 10 ms, 15 ms and 20 ms where we randomly assignthese random variables to each link. The maximum tolerabledelay variation, Dmax, is set to 250 ms.

Throughout the simulations, we used MPEG test se-quence Train and the animation video Big Buck Bunny (BBB)with resolutions 704×576 and 1280×720, respectively. Weloop both videos to obtain 900 frames lasting about 30 sec.We encode them using SVC reference software JSVM 9.19to obtain a base and an enhancement layer (see Table 1).

The simulator generates QoS routes only for the SVCbase layer packets while enhancement layer packets remain

(a)

(b)

Fig. 3: Simulation results: (a) Train, (b) Big Buck Bunny

Video Total Rate Full PSNR Base Rate Base PSNRTrain 1.3Mbps 36.89dB 0.7Mbps 33.60dBBBB 1.2Mbps 37.67dB 0.4Mbps 33.92dB

Table 1: Rate-Distortion values of the encoded sequences

on their traditional shortest path. Dynamic routing is also en-abled and rerouting occurs when at least one domain goes intoa bad state from which SVC base layer packets are passingthrough. The simulator calculates the QoS routes by follow-ing exactly the same procedure that we discussed in Section3. It first updates the virtual link parameters in the aggregatednetwork by using Method-1 and Method-2, then solves theCSP instance stated in (5) to determine inter-domain routeand finally, finds the global route from source to destinationby solving CSP instances for each domain and combining theresults as in (6).

The simulator provides us a trace driven simulation en-vironment so that we can track which specific video packetsare lost. By matching those lost packets with the NetworkAccess Layer (NAL) units of the SVC video stream, we de-tect and erase the NAL units that are lost. Then, the ma-nipulated stream is decoded and the PSNR values are mea-sured. For each QoS routing scenario, we repeat our simula-tions 50 times and the average PSNR values are calculated.The simulation results are shown in Fig.3 and we observethat the proposed distributed approaches using aggregationMethod-1(Distr(M1)) and Method-2 (Distr(M2)) closely ap-proach to the globally optimum QoS routing(Global) and sig-nificantly outperforms traditional shortest path(SP). In com-parison of network aggregation methods, Method-2 performsslightly better than Method-1 on the average. This is be-cause, Method-2 provides intra-domain summarization basedon multiple candidates of QoS routes while Method-1 is basedon single but best QoS route which may not exist after its cal-culation.

5. CONCLUSION

The proposed network aggregation method significantly re-duces the problem size down to the order of number of bor-der nodes. Comparing the link summarization methods, weobserved that Method-2 is slightly better (less than 0.5dB)and provides more stable intra-domain summarization thanMethod-1. We show that the proposed distributed optimiza-tion of QoS routing closely approaches the non-scalable glob-ally optimum solution and the discrepancy between them interms of end-user video quality of experience is less than 1dBon the average.

6. REFERENCES

[1] R. Braden, D. Clark, and S. Shenker, “Integrated ser-vices in the internet architecture: an overview,” RFC1633, Internet Engineering Task Force, June 1994.

[2] S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, andW. Weiss, “An architecture for differentiated services,”RFC 2475, Internet Engineering Task Force, Dec. 1998.

[3] E. Rosen and Y. Rekhter, “BGP/MPLS VPNs,” RFC2547, Internet Engineering Task Force, 1999.

[4] OpenFlow Consortium. [Online]. Available:http://openflowswitch.org

[5] OpenFlow Switch Specification v1.1.0. [Online]. Avail-able: http://www.openflow.org/wp/documents/

[6] R. Sherwood, G. Gibb, K. K. Yap, M. Casado, N. Mck-eown, and G. Parulkar, “Can the production network bethe testbed,” in OSDI’10, 2010.

[7] T. Koponen and et.al., “Onix: a distributed control plat-form for large-scale production networks,” in OSDI’10,2010, pp. 1–6.

[8] A. Tootoonchian and Y. Ganjali, “Hyperflow:a distributed control plane for OpenFlow,” ser.INM/WREN’10, 2010, pp. 3–3.

[9] H. E. Egilmez, B. Gorkemli, A. M. Tekalp, andS. Civanlar, “Scalable video streaming over OpenFlownetworks: an optimization framework for QoS routing,”in Proc. IEEE International Conference on Image Pro-cessing (ICIP), Sept. 2011, pp. 2241–2244.

[10] A. Juttner, B. Szviatovski, I. Mecs, and Z. Rajko, “La-grange relaxation based method for the QoS routingproblem,” in Proc. IEEE INFOCOM, vol. 2, Apr. 2001,pp. 859–868.

[11] LEMON, Library for Efficient Modeling andOptimization in Networks. [Online]. Available:http://lemon.cs.elte.hu

[12] GT-ITM, Georgia Tech InternetworkTopology Models. [Online]. Available:http://www.cc.gatech.edu/projects/gtitm/