arX

iv:1

401.

6361

v1 [

cs.M

M]

24 J

an 2

014

1

Control of Multiple Remote Serversfor Quality-Fair Delivery of Multimedia Contents

Nesrine Changuel⋆, Bessem Sayadi⋆, Michel Kieffer o,+,†

⋆ Alcatel-Lucent - Bell-Labs, 91620 NozayEmail: {nesrine.changuel, bessem.sayadi}@alcatel-lucent.com

o L2S, CNRS - SUPELEC - Univ Paris-Sud, 91192 Gif-sur-Yvette+ partly on leave at LTCI - CNRS - Telecom Paris-Tech, 75014 Paris

† Institut Universitaire de France, 75005 ParisEmail: kieffer@lss.supelec.fr

Abstract—This paper proposes a control scheme for thequality-fair delivery of several encoded video streams to mobileusers sharing a common wireless resource. Video quality fairness,as well as similar delivery delays are targeted among streams.The proposed controller is implemented within some aggregatorlocated near the bottleneck of the network. The transmission rateamong streams is adapted based on the quality of the alreadyencoded and buffered packets in the aggregator. Encodingrate targets are evaluated by the aggregator and fed backto each remote video server (fully centralized solution), ordirectly evaluated by each server in a distributed way (partiallydistributed solution). Each encoding rate target is adjusted foreach stream independently based on the corresponding bufferlevel or buffering delay in the aggregator. Communication delaysbetween the servers and the aggregator are taken into account.

The transmission and encoding rate control problems arestudied with a control-theoretic perspective. The system isdescribed with a multi-input multi-output model. Proporti onalIntegral (PI) controllers are used to adjust the video quality andcontrol the aggregator buffer levels. The system equilibriumand stability properties are studied. This provides guidelines forchoosing the parameters of the PI controllers.

Experimental results show the convergence of the proposedcontrol system and demonstrate the improvement in video qualityfairness compared to a classical transmission rate fair streamingsolution and to a utility max-min fair approach 1.

Index Terms—Command and control systems; Decentralizedcontrol; Multimedia communication; Quality of service; Stabilityanalysis.

I. I NTRODUCTION

With the development of wireless networks and widespreadof smartphones, delivery of compressed videos (video-on-demand or mobile TV broadcast services) to mobile usersis increasing rapidly. This trend is likely to continue in thecoming years [1]. To satisfy the related increasing demandfor resources, operators have to expand their network capacitywith as limited as possible infrastructure investments. Inparallel, they have to optimize the way multimedia contentsare

1Parts of this work have been presented at ACM Multimedia conference,2012. This work has been partly supported by ANR ARSSO project, contractnumber ANR-09-VERS-019-02 and by ANR project LimICoS, contractnumber ANR-12-BS03-005-01.

delivered to users while satisfyingapplication-layer quality-of-service (QoS) constraints, which are more challenging toaddress than traditionalnetwork-layer QoS constraints.

Delivered videos have a large variety of quality-rate char-acteristics, whatever the considered quality metric,e.g.,Peak Signal-to-Noise Ratio (PSNR), Structural SIMilarity(SSIM) [2], etc. These characteristics are time-varying anddepend on the content of the videos. Provisioning someconstant transmission rate to mobile users for video delivery isin general inappropriate. If videos are encoded at a constantbitrate, the quality may fluctuate with the variations of thecharacteristics of the content. If they are encoded at a variablebitrate, targeting a constant quality, buffering delays mayfluctuate significantly.

This paper proposes a controller for the quality-fair deliveryof several encoded video streams to mobile users sharinga common wireless resource. Video-on-demand or multi-cast/broadcast transmission are typical applications forthisscenario. Video encoding rate adaptation and wireless resourceallocation are performed jointly within some Media AwareNetwork Element (MANE) using feedback control loops.The aim is to provide users with encoded videos of similarquality and with controlled delivery delay, without exchanginginformation between remotely located video servers.

A. Related work

When controlling the parallel delivery of several videostreams, their rate-distortion (R-D) trade-off may be adjustedby selectively discarding frames as in [3], [4] or via anadaptation of their encoding parameters as in [5], [6]. Withscalable video encoders, such as H.264/SVC, the R-D trade-offmay be adjusted via packet filtering [7], [8]. In this case, thecontrol parameter is the number of transmitted enhancementlayers for each frame.

If several video streams are transmitted to different usersina dedicated broadcast channel with limited capacity, a blindsource rate allocation could lead to unacceptable quality forhigh-complexity video contents compared to low-complexityones. Therefore, providing fairness is an important issue thatmust be addressed.

2

Video quality fairness among encoded streams may beobtained by sharing quality information, or R-D characteristicsvia a central controller providing to each server quality orratetargets, as in [9], [10]. This technique enables the encodersto adjust their bit rate or to drop frames or quality layersdepending on the complexity of the videos and on the availabletransmission rate.

Control-theoretic approaches have been considered to ad-dress the problem of rate control in the context of videostreaming, see,e.g., [11], [12], [13], [14], [15]. In [11],a real-time rate control based on a Proportional IntegralDerivative (PID) controller is proposed for a single videostream. The main idea is to determine the encoding rate perframe based on the buffer level to maximize video qualityand minimize quality variations over frames. In [12], aflow control mechanism with active queue management anda proportional controller is considered. A flow control isused to reduce the buffer size and avoid buffer overflow andunderflow. The flow control mechanism is shown to be stablefor small buffer sizes and non-negligible round-trip times.In [13], a rate allocation algorithm, performed at the GroupofPicture (GoP) level, is performed at the sender to maximizethe visual quality according to the overall loss and the receiverbuffer occupancy. This target is achieved using a ProportionalIntegral (PI) controller in charge of determining the transmis-sion rate to drain the buffers. Later, [16] introduces a ratecontroller that uses different bit allocation strategies for Intraand Inter frames. A PID controller is adopted to minimizethe deviations between the target and the current buffer level.Buffer management is performed at the bit level and deliverydelay is not considered. Moreover, [11], [12], [13], [16]address single flow transmission.

For multi-video streaming, in [17], a distributed utilitymax-min flow control in the presence of round-trip delaysis proposed. The distributed link algorithm performs utilitymax-min bandwidth sharing while controlling the link bufferoccupancy around a target value at the cost of link underutilization using a PID controller. Stability analysis in caseof a single bottleneck and homogeneous delay is conducted.In [9], a content-aware distortion-fair video delivery scheme isproposed to deliver video streams based on the characteristicsof video frames. It provides a max-min distortion fair resourcesharing among video streams. The system uses temporalprediction structure of the video sequences with a frame dropstrategy based on the frame importance to guide resourceallocation. The proposed scheme is for video on-demandservices, where the rate and the importance of each frameare assumed calculated in advance. A proportional controlleris considered in [14] to stabilize the received video quality aswell as the bottleneck link queue for both homogeneous andheterogeneous video contents. A PI controller is consideredin [15]. Robustness and stability properties are studied.In [14] and [15], the rate control is performed in a centralizedway, exploiting the rate and distortion characteristics oftheconsidered video flows to determine the encoding rate for thenext frame of each flow. In [18], a cross-layer optimizationframework for scalable video delivery over OFDMA wirelesssystems is proposed, aiming at maximizing the sum of the

achievable rates while minimizing the distortion differenceamong multiple videos. The optimization problem is describedby a Lagrangian constrained sum-rate maximization to achievedistortion fairness among users. However the communicationdelay between the control block and the controlled servers isnot addressed.

Remotely implemented control laws are also consideredin [19] leading to the problem of stabilizing an open-loopsystem with time-varying delay. The problem of remotestabilization via communication networks is considered withan explicit use of the average network dynamics and anestimation of the average delay in the control law. The controllaw does not address video transmission issues, so no qualityconstraint on the transmitted data is considered.

The commercial products described in [20], [21] proposestatistical multiplexing systems enabling encoders to adapttheir outputs to the available channel rate. Connecting en-coders and multiplexers via a switched IP network allowscollocated and distributed encoders to be part of the multiplex-ing system. Nevertheless, in these solutions, quality fairnessconstraints between programs appear not to be consideredamong the video quality constraints.

B. Main contributions

In this paper, we propose a control system to perform jointly(i) encoding rate control of spatially spread video servers with-out information exchange between them and (ii) transmissionrate control of the encoded streams through some bottlenecklink. A MANE, located near the bottleneck link derives theaverage video quality of the data stored in dedicated buffersfed by the remote servers. This average video quality iscompared by each individual transmission rate controller tothe quality of its video flow to adjust the draining rate of thecorresponding buffer (first control input). For that purpose,programs with low quality are drained faster than programswith high quality. Dedicated encoding rate controllers observethe buffer levels to adjust the video encoding rates (secondcontrol input). The encoding rate control targets a similarbuffer level for all programs. The buffer level in bits orthe buffering delay can be adjusted via an adaptation of thevideo encoding rates,e.g., by scalability layer filtering whena scalable video coder is involved.

In a fully centralized version of the controller, the MANEis in charge of sending the encoding rate target to each videoserver. In apartly distributed version, only the individualbuffer level discrepancies are transmitted to the servers,whichare then in charge of computing their own encoding ratetarget. Communication delays between the MANE and theservers are considered in both directions. A discrete-timestate-space representation of the system is introduced. Thebuffer level (in bits) or the buffering delay has to be controlledand quality fairness between video streams has to be obtained.For that purpose, feedback loops involving PI controllers areconsidered. The quality fairness constraint among streamsleads to a coupling of the state equations related to the controlof the delivery of each stream. The system equilibrium andstability properties are studied. This provides guidelines for

3

VideoServer N

Buffer 2

Buffer N

Encoding ratecontroller 1

AverageUtility

Channel

Rc

Videostreams

Controlinputs

Controloutputs

Re

1

Re

2

Re

N

Rt

1

U1

U2

UN

B1

B2

BN

Media Aware Network Element

Encoding ratecontroller 2

Encoding ratecontroller N

VideoServer 2

VideoServer 1

Buffer 1Transmission rate

controller 1

Transmission ratecontroller 2

U

Transmission ratecontroller N

Rt

2

Rt

N

Figure 1. Structure of the proposed quality-fair video delivery system (fullycentralized version).

choosing the parameters of the PI controllers. This paperextends preliminary results obtained in [22], where control ofthe buffer level in bits is not addressed and the communicationdelay between the MANE and the servers is not explicitlyconsidered.

Section II introduces the considered system and the con-straints that have to be satisfied. The proposed solution isdescribed in Section III. The required hypotheses are listedand a discrete-time state-space representation of the systemis provided, emphasizing the coupling between equationsinduced by the fairness constraint among video streams. Theequilibrium and stability analyses are performed in Section IV.Finally, a typical application context is described in Section Vand experimental results are detailed. Robustness of theproposed control system to variations of the channel rate andof the number of video streams is shown.

II. SYSTEM DESCRIPTION

Consider a communication system in whichN encodedvideo streams provided byN remote servers arrive to somenetwork bottleneck where they have to share a communicationchannel providing a total transmission rateRc, see Figure 1.The servers deliver encoded Video Units (VUs) representinga single picture or a Group of Pictures (GoP). All VUs areassumed to be of the same durationT and the frame rateFis assumed constant over time and identical for all streams.Time is slotted and thej-th time index represents the timeinterval [jT, (j + 1)T ).

In the proposed system, a Media Aware Network Element(MANE) is located close to the bottleneck of the network, seeFigure 1. The MANE aims at providing the receivers withvideo streams of similar (objective or subjective) qualityandwith similar delivery delays. For that purpose, two feedbackloops are considered to control(i) the encoding rate and(ii)the transmission rate of each video stream, see Figure 2.

Encoded and packetized VUs are temporarily stored indedicated buffers in the MANE. We assume that autility Ui (j)measures the quality of each VUj for each streami (in termsof PSNR, SSIM, or any other video quality metric [23]). TheMANE has access toUi (j), stored,e.g., in the packet headers.The transmission rate controllers are in charge of choosingthedraining rates from each buffer so that all utilities withintheN

buffers are as close as possible. The encoding rate controllersare in charge of choosing the video encoding rates so that thebuffer levels in the MANE are adjusted around some referencelevelB0 in bits or reference delayτ0 in seconds. This controlis performed at each discrete time index.

Encoders

Buffers

Transmission ratecontroller

Encoding ratecontrollers

R jeS

i( )

Bi( )j

Rt

i( )j

UM

ReM

ReS

ReM

US

i( )j

i( )j i( )j

i( )j i( )j

Figure 2. Feedback control loops in the proposed quality-fair video deliverysystem.

The MANE evaluates the average utility and each trans-mission rate controller allocates more rate to streams withautility below average. The buffers of these streams are drainedfaster than those with utility above average. This control,donewithin the MANE, is thuscentralized. The discrepancy of eachbuffer level in bits (respectively delay in seconds) with respectto the reference levelB0 (respectively delayτ0) is processedindividually by an encoding rate controller. The encodingrate for the next VU of each video stream is then evaluatedto regulate the buffer level around the reference level. Theencoding rate may be evaluated at the MANE, in which case,the control is fullycentralized and the video encoders/serversreceive only the evaluated target bit rate. Alternatively,theencoding rates may be evaluated in adecentralized way ateach video encoder/server. For the latter case, the MANE onlyfeeds back to each server the buffer level (respectively delay)discrepancy. This solution requires all video encoders/serversto host individually an encoding rate controller, which ismainly possible for managed video servers. In this paper,the encoding rate target is evaluated within the MANE, butthe encoding parameters are evaluated in adistributed way byeach server [24].

The interaction of both control loops (transmission ratecontrol and encoding rate control) allows getting a quality-fair video delivery. Videos with a quality below average havea buffer in the MANE that is drained faster, and thus is likelyto be belowB0 or τ0. The encoding rate of such streams isthen increased, to improve their quality.

Feedback delays between the MANE and the video serversare considered. They correspond to the delays introducedwhen the servers deliver encoded packets to the MANE andwhen the MANE feeds back signalization to carry encodingrate targets to the servers. To account for the delayed utilityavailable at the MANE and the delayed encoding rate targetssent by the MANE to the servers, two state variables repre-senting the delayed encoding rates and the delayed utilities areintroduced

ReSi (j) = ReM

i (j − δ1) , (1)

andUMi (j) = US

i (j − δ2) , (2)

4

whereReMi (j) is the encoding rate evaluated at the MANE for

the j-th VU of programi. This encoding rate target reachesthe server with some delayδ1, where it is denoted byReS

i (j).On the other hand, the server sends the utility of thej-th VUof programi, denoted byUS

i (j). It arrives at the MANE withsome delayδ2 and is denotedUM

i (j), see Figure 2. The Msuperscript refers to the information available at the MANEand the S superscript refers to the information available attheserver for both rate and utility. In what follows, the stabilityof both control loops is studied.

III. STATE-SPACE REPRESENTATION

A state-space representation of the system presented inSection II is introduced to study its stability. Several additionalassumptions are needed to get a tractable representation.

A. Assumptions

1) Feedback delay: In what follows, a VU represents aGoP. All encoded VUs processed by the video servers duringthe (j − 1)-th time slot [(j − 1)T, jT ) are assumed to havereached the MANE during thej-th time slot[jT, (j + 1)T ).The encoding rate targets sent by the MANE to the serversRe

i(j) at the beginning of thej-th time slot are assumed tohave reached all video servers at the beginning of the(j + 1)-th time slot. This rate is used to encode the(j + 1)-th VUwhich will be placed in the buffer in the MANE during the(j + 2)-th time slot,etc., see Figure 3 whenδ1 = δ2 = 1.

The transmission delays between the MANE and the serversmay vary. The previous assumptions allow to cope withforward and backward delays upper bounded byT . Thisis reasonable, since the network transmission and bufferingdelays are of the order of tens of milliseconds, provided thatit is not too congested. This is less than the duration of VUswhen they represent GoPs (typically0.25 s to1 s). Followingthe bounded-delay assumption, during thej-th time slot, theMANE has only access to the utilitiesUS

i (j−2), i = 1, . . . , Nof the (j − 2)-th encoded VUs.

In the rest of the paper the superscripts S and M are omitted.ThenUi (j) is the utility of the j-th VU of the i-th streamencoded during time slotj, transmitted to the MANE duringtime slot j + 1, and fully buffered in the MANE at thebeginning of time slotj+2. Re(j) is the encoding rate targetevaluated by the MANE during thej-th time slot. Re(j)reaches the server at the beginning of time slotj + 1, seeFigure 3.

2) Source model: The following parametric rate-utilitymodel is used to describe the evolution of the utilityUi(j)as a function of the rateRe

i(j) used to encode thej-th VU ofthe i-th stream

Ui(j) = f (ai (j) , Rei(j)) , (3)

where ai (j) ∈ A ⊂ RNa is a time-varying and program-

dependent parameter vector. Note that this model is only usedto define the controller parameters so that the system is stable.Once these parameters are set, the rate-utility model is no moreneeded. For all values ofa belonging to the set of admissibleparameter valuesA, f (a, R) is assumed to be a continuous

Server MANE

R je( )R j

t( )

R je( )

U j( )

R je( +2)R j

t( +2)

U j( )

U j( -2)R je( -2)

j

j+2

j+1

j+3

R je( -1)

U( -1)j

R je( +1)

j

j+2

j+1

j+3

Figure 3. Communication delays between a video server and the MANE.

and strictly increasing function ofR, with f (a, 0) = 0. Thevariation with time ofai (j) is described by

ai(j + 1) = ai(j) + δai(j), (4)

where δai (j) represents the uncontrolled variations of thesource characteristics.

The model (3) may represent the variation with the encodingrate of the SNR, the PSNR, the SSIM, or any other strictlyincreasing quality metric.

3) Buffer model: As introduced in Section II, the MANEcontains dedicated buffers for each of theN encoded videostreams. The evolution of the level in bits of thei-th bufferbetween time slotj andj + 1 is

Bi(j + 1) = Bi(j) +(

Rei(j − 2)−Rt

i(j))

T, (5)

whereRti(j) is the transmission rate for thei-th stream and

Rei(j− 2) is the encoding rate of the(j − 2)-th VU evaluated

at the MANE at time slotj. In (5), Rei(j − 2) accounts for

the communication delay between the server and the MANE,see Figure 3.

Buffers are controlled in two different ways. A control ofthe buffer level in bitsBi(j) maintains an averaged level in bitsto prevent buffer overflow and underflow. This is appropriatefor applications with buffers of limited size. A control ofthe buffering delay helps adjusting the end-to-end deliverydelay. This type of control is better suited for delay-sensitiveapplications. Buffering delay control within the MANE alsoallows implicitly controlling the buffering delay at the clientfor live and broadcast applications. This is due to the fact thatthe end-to-end delay between the transmission of a VU by theserver and its playback by the client is constant over time forlive video2.

Let hi(j) be the number of VUs in thei-th MANE bufferat timej, the corresponding buffering delay is

τi(j) = hi(j)T. (6)

Assume that packets containing encoded VUs are segmentedto allow a fine granularity of the draining rate. Thenhi(j) be-comes quite difficult to evaluate accurately and directly withinthe MANE using only information stored in packet headers.

2Some periodic feedback may nevertheless be useful to verifythat thesystem actually behaves nominally.

5

The corresponding buffering delay is then approximativelyevaluated as

τi(j) =Bi(j)

Rei(j)

, (7)

where

Rei(j) = 1

hi(j)

∑⌊hi(j)⌋ℓ=2 Re

i(j − ℓ)

+ 1hi(j)

Rei (j − ⌈hi(j)⌉) (hi(j)− ⌊hi(j)⌋)

(8)

is the average encoding rate of the VUs stored in thei-thbuffer at timej and⌊·⌋ and⌈·⌉ denote rounding towards−∞and+∞. Since (8) still requires the availability ofhi(j), wepropose to estimate it as follows

Ri

e(j) = Re

i(j), if j 6 2,

Ri

e(j) = αRe

i(j − 2) + (1− α)Ri

e(j − 1), if j > 2,

(9)

where0 < α < 1 is some tuning parameter. Then, one getsan estimate of the buffering delay (6) using (9)

τi(j) =Bi(j)

Rei(j)

. (10)

B. Rate controllers

N coordinated transmission rate controllers andN (possiblydistributed) encoding rate controllers are considered in thevideo delivery system described in Section II. A PI controlof the transmission rate is performed. At timej, each PIcontroller takes as input the average utility evaluated by theMANE and the utility Ui (j − 2) of the VU j − 2 of thecontrolled stream. PI controllers are also used to evaluatethetarget encoding rate of each program in order to regulate thebuffer level aroundB0 or the buffering delay aroundτ0.

1) Transmission rate control: At time j, the availablechannel rateRc is shared between theN video streams. Thedelayed utility of the VU available at the MANE at timej forthe i-th stream is

Uddi (j) = Ud

i (j − 1) = Ui(j − 2) (11)

and the utility discrepancyδUddi (j) with the average utility

U(j) over theN programs at timej is

δUddi (j) =

1

N

N∑

ℓ=1

(

Uddℓ (j)− Udd

i (j))

= U(j)− Uddi (j).

(12)The PI transmission rate controller for thei-th program uses

δUddi (j) to evaluate the transmission rate allocated to each

video stream

Rti (j) = R0 +

(

K tP +K t

I

)

δUddi (j) +K t

Iφi (j) , (13)

whereK tP andK t

I are the proportional and integral correctiongains. All rates are evaluated with respect toR0 = Rc/N , theaverage encoding rate per stream that would be used when theN streams represent the same encoded video. In (13),φi (j) isthe cumulated utility discrepancy (used for the integral term)evaluated as

φi (j) = 0, if j 6 2,φi (j + 1) = φi (j) + δUdd

i (j), if j > 2.(14)

Rti (j) represents the draining rate of thei-th MANE buffer at

time j. One may easily verify that

N∑

i=1

Rti (j) = Rc. (15)

According to (13), in a first approximation, more (resp. less)transmission rate is allocated to programs with a utility below(resp. above) average.

2) Encoding rate control: The encoding rate control isperformed independently for each video stream. This allowsadistributed implementation of this part of the global controller.

For the control of the buffer level or of the buffering delay,the buffer level (5) is needed. LetRedd

i (j) be the delayedencoding rate of the VU reaching the MANE at timej for thei-th program

Reddi (j) = Red

i (j − 1) = Rei(j − 2). (16)

Using (5) and (16), one gets

Bi(j + 1) = Bi(j) +(

Reddi (j)−Rt

i(j))

T. (17)

In case of buffer level control, the encoding rate for thej-th VU of each video program is controlled to limit thedeviations ofBi(j) from the reference levelB0. At time j,the discrepancyδBi (j) betweenBi (j) andB0 is

δBi (j) = Bi (j)−B0. (18)

In case of buffering delay control, the encoding rate forthe j-th VU of each video program is controlled to limit thedeviations ofτi (j) from the reference levelτ0. At time j, thediscrepancyδτi (j) betweenτi (j) andτ0 is

δτi (j) = τi (j)− τ0 =(

Bi(j)

Reddi

(j)− τ0

)

. (19)

Buffers with positiveδBi (j) or δτi (j) contain more thanB0

bits orτ0 seconds of encoded videos. The encoding rateRei (j)

has thus to be decreased. This part of the control process isvery similar to back-pressure algorithms [25].Re

i (j) is evaluated as the output of a PI controller

Rei (j) = R0 −

KeP +Ke

I

Tδτi (j)−

KeI

TΠi (j) . (20)

where KeP and Ke

I are the proportional and integral gainsand Πi (j) is the cumulated buffer discrepancy in secondsevaluated as

Πi (j) = 0, if j 6 3,Πi (j + 1) = Πi (j) + δτi (j) , if j > 3.

(21)

For a control of the buffer level,δτi (j) is replaced byδBi (j)in (20) and (21).

Taking into account the communication delay between theMANE and the server, the encoding rate targetRe

i (j) evalu-ated at the MANE at timej reaches the video server at timej+1. Thus,Re

i (j) represents the encoding rate for thej+1-thVU. The encoding rate increases (resp. decreases) when thebuffer is below (resp. above) its reference level. The sum ofthe encoding rates is not necessarily equal toRc. This allowsto compensate for the variations of the video characteristics.

6

Considering simultaneously (5), (13), and (20), one sees thatbuffers corresponding to programs producing video with lowerutility than average are drained faster. As a consequence, theencoding rate allowed to encode the next VU of such programsis increased, potentially increasing the utility.

C. State-space representation

The state-space representation facilitates the study of thesystem equilibrium and stability properties. Two representa-tions are considered, depending on whether the buffer levelorthe buffering delay is controlled.

In the case of a control of the buffering delay, combin-ing (3), (4), (11), (13), (14), (17), (20), and (21) leads to thefollowing discrete-time nonlinear state-space representationfor the i-th video stream,i = 1, . . . , N

ai(j + 1) = ai(j) + δai(j) (22a)

adi(j + 1) = ai(j) (22b)

φi (j + 1) = φi (j) +1

N

N∑

k=1

Uddk (j)− Udd

i (j) (22c)

Πτi (j + 1) = Πτ

i (j) +

(

Bi (j)

Rei(j)

− τ0

)

(22d)

Ri

e(j + 1) = αRedd

i (j) + (1− α)Ri

e(j) (22e)

Redi (j + 1) = R0 −

KeτP +Keτ

I

T

(

Bi (j)

Rei(j)

− τ0

)

−Ke

I τ

TΠi (j)

(22f)

Reddi (j + 1) = Red

i (j) (22g)

Uddi (j + 1) = f

(

adi (j) , R

edi (j)

)

(22h)

Bi(j + 1) = Bi(j) +Reddi (j)T −R0T

−

(

(

K tP +K t

I

)

(1

N

N∑

k=1

Uddk (j)− Udd (j)) +K t

Iφi(j)

)

T

(22i)

whereadi(j) is the delayed video characteristic vector of the

(j−1)-th VU. The utilitiesUddk (j) of all video streams appear

in (22), leading to a coupling of the state-space representationsrelated to the control of the individual video streams.

When buffer level control is addressed,Bi (j) /Rei(j) is

replaced byBi (j) in (22d) and (22f), and the state (22e) doesnot appear anymore.

In the remainder of the paper, the subscriptb is for bufferlevel control and the subscriptτ is for buffering delay control.

IV. EQUILIBRIUM AND STABILITY

The steady-state behavior and the stability of the videodelivery system described by (22) for buffering delay controlas well as the simpler system for buffer level control arestudied. Due to the coupling between controllers induced bythe constraint that the discrepancy between the average utilityand the utility of each program has to be as small as possible,both characterizations have to be done on the whole system. Inthe rest of this section, we derive the equilibrium and performthe stability analysis for the control system where buffering

delays are controlled. The technique is similar when for thecontrol of the buffer level.

A. Equilibrium analysis

The system reaches an equilibrium when all terms on theleft of the state-space representation (22) do not change withtime. This leads to a system of(Na + 6)×N equations with(Na + 6)×N unknowns.

δaeqi = 0

Ueqi = 1

N

∑N

k=1 Ueqk

Beqi = τ0Ri

e,eq

Ri

e,eq= Re,eq

i

Re,eqi = R0 −

KeτIT

Πτeqi

Ueqi = f

(

aeqi , R

e,eqi

)

Re,eqi = K t

Iφeqi

, (23)

The second equation in (23) imposes thatUeq1 = · · · = Ueq

N =Ueq; all programs have thus the same utility at equilibrium.Moreover, one hasRi

e,eq= Re,eq

i andBeqi = τ0R

e,eqi , which

leads toBeqi /R

e,eqi = τ0, for i = 1, . . . , N meaning that at

equilibrium, the buffering delay is equal toτ0 for all streams.The target encoding rates at equilibriumRe,eq

i and the utilityUeq are obtained as the solution of a system ofN+1 equations

{

f(

aeq1 , R

e,eq1

)

= · · · = f(

aeqN , Re,eq

N

)

= Ueq∑N

i=1 Re,eqi = Rc , (24)

depending of the valuesaeqi ,i = 1, . . . , N of the parameter

vector of the rate-utility model. At equilibrium, they areassumed constant in time and well-estimated, see Section V.Sincef is strictly increasing withR, the rate at equilibriumas a function ofUeq is

Re,eqi = f−1

R

(

aeqi , U

eq)

, i = 1, . . . , N, (25)

with f−1R is the inverse off seen as a function ofR only. The

value ofUeq is determined from the channel rate constraint

N∑

i=1

Re,eqi =

N∑

i=1

f−1R

(

aeqi , U

eq)

= Rc. (26)

Since f (a, R) is a continuous and strictly increasing func-tion of R, f−1

R (a, U) and∑N

i=1 f−1R (ai, U) are also

continuous and strictly increasing functions ofU , with∑N

i=1 f−1R (ai, 0) = 0. Provided that

limU→∞

N∑

i=1

f−1R (ai, U) > Rc, (27)

(26) admits a unique solution.Π eqi and φeq

i ,i = 1, . . . , N ,are deduced from (23) and (25), provided thatKe

I 6= 0 andK t

I 6= 0.The equilibrium is thus unique and satisfies the control

targets considered in Section II. Similar conclusions can beobtained when the buffer level is controlled.

7

B. Linearized model

We study the local stability of the system around an equilib-rium point evaluated in Section IV-A. Linearizing (22) aroundthe equilibrium characterized in (23) one gets fori = 1, . . . , N

∆ai (j + 1) = ∆a (j) + δai (j)∆ad

i(j + 1) = ∆ai(j)

∆φi (j + 1) = ∆φi (j) +1N

∑N

k=1 ∆Uddk (j)−∆Udd

i (j)

∆Πτi (j + 1) = ∆Πτ

i (j)−1

Re,eqi

(

τ0∆Ri

e(j)−∆Bi(j)

)

∆Ri

e(j + 1) = (1− α)∆Ri

e(j) + α∆Redd

i (j)

∆Redi (j + 1) =

KeP+Ke

IT

1R

e,eqi

(

τ0∆Ri

e(j)−∆Bi(j)

)

−Ke

IT∆Πτ

i (j)

∆Reddi (j + 1)= ∆Ri

ed(j)

∆Uddi (j + 1) = ∂f

∂a

(

ad,eqi , Red,eq

i

)

∆adi (j) +

∂f∂R

(

ad,eqi , Red,eq

i

)

∆Redi (j)

∆Bi (j + 1) = ∆Bi (j) + ∆Reddi (j)T

−(

(K tP +K t

I )(

1N

∑Nk=1 ∆Udd

k (j)−∆Uddi (j)

)

+K tI∆φi (j)

)

T.

(28)Consider theN × (N ×Na) block diagonal matrix

Ξ = diag

(

∂f

∂aT

(

ad,eq1 , R

e,eq1

)

, . . . ,∂f

∂aT

(

ad,eqN , R

e,eqN

)

)

,

(29)gathering the sensitivities with respect toa of the rate-utilitycharacteristics of each stream and theN ×N diagonal matrix

Γ = diag

(

∂f

∂R

(

ad,eq1 , Re,eq

1

)

, . . . ,∂f

∂R

(

ad,eqN , Re,eq

N

)

)

(30)

gathering the sensitivity toR of the rate-utility characteristicsof each stream. Putting all coupled linearized state-spacerepresentations (28) together, one gets a linear discrete-timestate-space representation

xτ (j + 1) = Aτxτ (j) + w(j) (31)

with state vector

xτ =

(

∆a,∆ad,∆φ,∆Πτ ,∆R

e,∆Red,∆Redd,∆Udd,∆B

)T(32)

and noise input

w = ( δa 0 . . . 0 )T , (33)

representing the fluctuations of the value of the parametervector for the rate-utility model. In (32) and (33), boldfaceletters represent vectors and time indexes have been omitted.For example∆a (j) is a vectorN × Na components and∆B (j) = (∆B1 (j) , . . . ,∆BN (j))

T is a vector ofN com-ponents. From (28) and (31), one deduces

Aτ =

I 0 0 0 0 0 0 0 0

I 0 0 0 0 0 0 0 0

0 0 I 0 0 0 0 −L 0

0 0 0 I −τ0V 0 0 0 V

0 0 0 0 (1− α)I 0 αI 0 0

0 0 0 −Kτe

IT

IKτe

Tτ0V 0 0 0 −Kτe

TV

0 0 0 0 0 I 0 0 0

0 Ξ 0 0 0 Γ 0 0 0

0 0 −K tIT I 0 0 0 T I K tTL I

(34)with V = diag

(

1/Re,eq1 , . . . , 1/Re,eq

N

)

a diagonal matrix con-taining the inverse of the encoding rates at equilibrium andKτe = Kτe

P +KτeI , K t = K t

P +K tI . I and0 are identity and

null matrices of appropriate size.When studying the roots of det(zI − A) = 0, Na × N

roots at z = 1 are obtained. They correspond to the

variations of the rate-utility parameter vector (4). The matrixΞ, representing the sensitivity with respect toa of the rate-utility characteristicsf , does not appear in the expressions ofA

τ . Only the sensitivity off with respect toR, representedby Γ, impacts the stability around equilibrium. The systemstability is also influenced by the encoding rates at equilibriumvia V and determined by the PI controller gainsK t

P, K tI , K

eP,

andKeI .

In Section V, values ofK tP, K t

I , KeP, andKe

I are chosenso that the system is robust to various realizations of the rate-utility parameters. The same values of the PI gains are chosenfor all programs. A similar analysis can be done when bufferlevels are controlled.

V. EXPERIMENTAL TESTS

A. Example of application context

A typical application scenario for the proposed rate controlsystem is Mobile TV using the evolved MBMS standard [26].The MBMS architecture is composed of three main entities:BM-SC, MBMS-GW and MCE. The Multicast/Broadcast Ser-vice Center (BM-SC) is a node that serves as entry point forthe content providers delivering the video sources, used forservice announcements, session management. The MANE,considered in the paper in charge of choosing the encodingand the transmission rates, may be located at the Broad-cast/Multicast source at the entrance of the BM-SC node. TheMBMS-Gateway (GW) is an entity responsible for distributingthe traffic across the different eNBs belonging to the samebroadcast area. It ensures that the same content is sent fromall the eNBs by using IP Multicast. The Multi-cell/multicastCoordination Entity (MCE) is a logical entity, responsiblefor allocation of time and frequency resources for multi-cell MBMS transmission. As in [27], we assume that theMBMS-GW periodically notifies the MCE about the resourcerequirements of video streams so that the resources at eNBscan be re-allocated accordingly. Therefore, the BM-SC shouldensure that the encoding rate of the multiplex does not violatethe already allocated resources. This is obtained thanks totheproposed rate control scheme.

B. Simulation environment

To illustrate the properties of the proposed controllers, thissection describes a simulation of mobile TV delivery in thepreviously described context. We considerN = 6 videostreams, each of100 s long, extracted from real TV programs.Interview3 (Prog 1), Sport4 (Prog 2), Big Buck Bunny5

(Prog 3), Nature Documentary6 (Prog 4), Video Clip7 (Prog 5),and an extract ofSpiderman8 (Prog 6) in 4CIF (704 × 576)format are encoded with x.264 [28] at a frame rateF = 30 fps.GoPs of10 frames are considered, thus the GoP duration isT = 0.33 s. The videos, already encoded using MPEG-4,

3http://www.youtube.com/watch?v=l2Y5nIbvHLs4http://www.youtube.com/watch?v=G63TOHluqno5http://www.youtube.com/watch?v=YE7VzlLtp-46http://www.youtube.com/watch?v=NNGDj9IeAuI7http://www.youtube.com/watch?v=rYEDA3JcQqw8http://www.youtube.com/watch?v=SYFFVxcRDbQ

8

have been converted to YUV format usingffmpeg [29]. Theaverage rate and PSNR of the streams encoded by x.264 witha constant quantization parameterQP = 3 are provided inTable I.

Video Rate (kbit/s) PSNR (dB) ActivityProg 1 1669.9 46.06 lowProg 2 4929.1 44.23 highProg 3 3654.6 44.56 highProg 4 2215.1 44.61 lowProg 5 2811.4 46.37 mediumProg 6 3315.9 46.53 high

Table IAVERAGE RATE AND PSNROF THE SIX CONSIDERED VIDEO STREAMS

(ENCODING WITH X.264AND CONSTANTQP = 3.

The videos are then processed with the proposed controlsystem operating at the GoP level. Initially, all buffers containthree encoded GoPs corresponding to a buffering delay of1 s. The sizeBmax of the buffers is taken large enoughto support the variations of its level, occurring,e.g., duringscene changes. Here, their size in bits isBmax = 4 Mbits.The reference buffer level in bits isB0 = 400 kbits andthe reference delay isτ0 = 1.5 s. This reference delay isconsistent with a typical switching time of less than2 s, asexpected in MBMS Television services [30]. The channel rateis Rc = 4 Mbps. The encoding rates are initially consideredequal toR0 = Rc/N . These rates correspond to the outputvalue of the rate control process provided to each videoserver. The encoder is then in charge of adjusting its encodingparameters to achieve the target bit rate.

The encoding/transcoding rates are sent to the video en-coders which have to choose the encoding parameters for thenext VU. In the considered simulation, the video quality is interms of PSNR or of SSIM of the encoded VUs. This qualitymetric is transmitted to the MANE in the packet headers. Notethat the utility model (3) is only required to characterize thestability of the system and to tune the control parameters.Once the parameters have been chosen off-line, there is noneed to know precisely the model (3) within the MANE.

The proposedquality fair (QF) video delivery system iscompared to atransmission rate fair (TRF) controller whichprovides equal transmission rate to theN video streams. Inthe TRF scheme, the encoding rate is controlled to limit thebuffer level/delay discrepancy.

A comparison is also performed with autility max-min fair(UMMF) approach [17], with a proportional transmission ratecontrol limiting the buffer level discrepancy. In the UMMFapproach, the MANE tries to find the set of encoding ratesfor the next VU that maximizes the minimum utility. Thefollowing constrained optimization problem is then considered

Re (j + 1) =

arg maxRe

1,...,RN

min {f (a1(j), Re1) , . . . , f (aN (j), Re

N )}(35)

such thatN∑

i=1

Rei = Rc.

Solving (35) requires the availability at the MANE of allrate-utility characteristics (or at least all vectors of parameters

ai(j)) of the previously encoded VUs, contrary to the QFapproach, where only the actual utility of the VUs is needed.The fact thatai(j) is used in (35) for the evaluation of theencoding rate at timej + 1 accounts for the possibility forthe MANE to get only rate-utility characteristics of previouslyencoded and already received VUs. Once the value of theencoding ratesRe

1, . . . , RN are derived, a proportional (P)controller for the transmission rate is applied to evaluatethetransmission rate allocated to each video stream

Rti (j) = R0 +K t

P (Bi(j)−B0), (36)

whereK tP is the proportional correction gain.

In this section different cases are considered: Both bufferlevel and buffering delay are addressed separately includingstability analysis and results for different utility metrics. Then,the robustness of the proposed control system is analyzed byconsidering variations of the channel rate as well as of thenumber of video programs.

C. Control of the buffer level

We first focus on the system performance when the bufferlevel (in bits) is used to update the encoding rate.

1) PSNR-rate model: To tune the PI controllers of the QFsystem, the first utility function considered is the PSNR ofeach GoP. As in [31], a logarithmic PSNR-rate model is used

Ui(j) = Pi(j) = f (ai (j) , Rei(j)) (37)

= a(1)i (j) log(a

(2)i (j)Re

i(j)),

with Pi(j) the PSNR of the GoP at timej for the i-thstream. For theN = 6 considered programs, the entries ofai (j) are estimated for each GoP using four encoding trials.An example of the accuracy of the PSNR-rate model (38)

Figure 4. PSNR-rate characteristics and models for the firstGoP of theN = 6 considered programs

is shown in Figure 4, when applied on the first GoP ofthe six considered programs, with parameters estimated fromencoding trials performed at 80 kb/s, 200 kb/s, 800 kb/s, and2 Mb/s. Figure 4 illustrates the ability of (38) to predict the

9

PSNR over a wide range of rates. In addition, the correlationcoefficientr2 between experimental and predicted PSNR-ratepoints is evaluated as

r2 =σ2xy

σ2xσ

2y

(38)

with σ2x =

∑n

k=1(xk − x)2, σ2y =

∑n

k=1(yk − y)2, andσ2xy =

∑nk=1(yk − y)(xk − x), wheren is the number of rates for

each program at which the PSNR has been evaluated (xk) andpredicted (yk) using (38), and wherex and y are the averagevalues of thexk ’s and of theyk ’s. For the six programs, forn = 7, the rate values are 80 kb/s, 130 kb/s, 200 kb/s, 500 kb/s,800 kb/s, 1.4 Mb/s, and 2 Mb/s, the correlation coefficientsare r2 = [0.998, 0.996, 0.997, 0.996, 0.992, 0.985] illustratingto good fit by (38) of the PSNR-rate characteristics.

2) Controller design and stability analysis: The values ofthe parameter vectorai (1), i = 1, . . . , 6, obtained for the firstGoP of theN = 6 programs are

a1 (1) =

(

1.110.15

)

, a2 (1) =

(

1.900.17

)

, a3 (1) =

(

0.760.17

)

,

a4 (1) =

(

0.090.24

)

, a5 (1) =

(

2.500.17

)

, a6 (1) =

(

0.070.20

)

.

Oncef is specified, one may characterize the system equilib-rium. The vector of rates at equilibrium

(

Re,eq1 , . . . , Re,eq

N

)Tis

obtained by solving the system of equations in the state-spacerepresentation at equilibrium.Ξ andΓ are derived from (29),(30) and (38) as follows

Ξ =

log(a1,2 (1)Re,eq1 )

a1,1(1)a1,2(1)

0 0 . . . 0 0

0 0 log(a2,2 (1)Re,eq2 )

a2,1(1)a2,2(1)

0 . . . 0

0 0. . . 0

......

.... . .

. . . 0

0 0 0 0 . . . log(aN,2 (1)Re,eqN )

aN,1(1)aN,2(1)

(39)and

Γ = diag

(

a1,1 (1)

Re,eq1 (1)

, . . . ,aN,1 (1)

Re,eqN (1)

)

. (40)

The gains of the PI controllers have to be chosen so thatthe roots of

d(z) = det (zI−A) (41)

remains within the unit circle, for various rate-utility charac-teristics of the VUs. In (41),A may correspond toAb, thelinearized state matrix when considering buffer level control,or to A

τ with buffering delay control.To increase the robustness of the proposed approach to vari-

ations of the rate-utility characteristics,K = 10 realizationsof N = 4 random parameter vectors of the PSNR-rate modelobtained as follows

a(k)i,1 = 1

N

∑N

i=1 ai,1 (1) + η(k)i,1 , (42)

a(k)i,2 = 1

N

∑N

i=1 ai,2 (1) + η(k)i,2 ,

for k = 1, . . . ,K. In (43), η(k)i,1 and η(k)i,2 are realizations of

zero-mean Gaussian variables with varianceσ21 = 6.25×10−2

andσ22 = 2.25×10−4. The resulting PSNR-rate characteristics

obtained using (43) are represented in Figure 5. These randomrealizations describe quite well the variability of actualPSNR-rate characteristics represented in Figure 4.

0 200 400 600 800 1000 1200 1400 1600 1800 200010

15

20

25

30

35

40

45

50

Rate (kb/s)

PS

NR

(dB

)

prog 1prog 2prog 3prog 4

Figure 5. Superposition of theK = 10 realizations of theN = 4 randomPSNR-rate characteristics

−1 −0.5 0 0.5 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

real

imag

inar

y

Figure 6. Location of the roots ofdet(

zI−Ab)

for the K = 10realizations of theN = 4 random PSNR-rate characteristics

A random search of the control parameters is then per-formed. Among the values providing stability for theKrandom PSNR-rate characteristics, the one with the rootsfarthest away from the unit circle is selected to provide goodtransients.

The tuning is performed forN = 4. Good transientbehaviors have been obtained withKe,b

P = 666, Ke,bI = 33,

K tP = 66 × 103, and K t

I = 1300. The position of theroots corresponding to the video characteristic represented inFigure 5 are in Figure 6.

Figure 6 shows that all roots remain within the unit circle.This result does not prove the robustness of the proposedchoice of the control parameters, but shows that this choiceleads to a system reasonably robust to changes of the charac-teristics of the transmitted programs. Nevertheless, someofthe roots are located quite near the stability limit, which willlead to quite long transients.

10

3) Simulation results: The x.264 video encoder is used toperform on-line compression of the different programs withthe rate targets provided by the encoding rate controllers.Within the video coder, a two-pass rate control is performedto better fit the target encoding rate. The target and obtainedencoding rates may however be slightly different. The systemperformance is first measured in terms of average buffer leveldiscrepancy∆B (in bits) with respect toB0, variance of thebuffer levelσ2

B (in bits2), PSNR discrepancy∆P (in dB), andPSNR varianceσ2

P (in dB2), with

∆B = 1NM

∑Nn=1

∑Ml=1 (Bn(l)−B0) ,

σ2B = 1

N

∑NM

n=1

∑M

l=1 (Bn(l)−B0 −∆B)2,

∆PSNR = 1NM

∑N

n=1

∑M

l=1

(

Pn(l)− P (l))

,

σ2PSNR = 1

N

∑NM

n=1

∑M

l=1

(

Pn(l)− P (l)−∆PSNR

)2,(43)

whereP (l) = 1N

∑Nn=1 Pn(l) andM is the number of GoPs

in the video streams.The results withN = 6 andRc = 4 Mbit/s are summarized

in Table II in the three cases: TRF, UMMF, and QF where∆B

are in kbits andσ2B in kbit2. The PI controllers used for the

transmission rate control loop reduce the PSNR discrepancybetween the programs at a price of some increase of the bufferlevel discrepancy and variance.

Ke,bP , K

e,bI Kt

P,Kt

I |∆B| σ2

B|∆P | σ2

P

TRF 666, 0 0, 0 31.5 2.1 3.1 9.8UMMF 0, 0 3, 0 75 2 2.7 13.9

QF 666, 33 (66, 1.3)103 53.07 6.1 1.5 6.7

Table IIPERFORMANCE WHEN USINGTRF AND QF CONTROLLERS WHEN

CONTROLLING THE BUFFER LEVELS FORN = 6.

Figure 7 represents the evolution of the PSNR of theNprograms over300 GoPs using the TRF (left) and the QF(right) controllers whenN = 2, N = 4, andN = 6 witha constant channel rateRc = 4 Mbit/s. The same controllerparametersKe,b

P = 666, Ke,bI = 33, K t

P = 66 × 103, andK t

I = 1300 are used forN = 2, N = 4, andN = 6.WhenN = 2, the average PSNR of Prog2, characterized by

high activity level, is improved from36 dB to 41 dB, leadingto a significant improvement of the video quality. This is atthe price of PSNR degradation of Prog1, characterized by lowactivity level, from45 dB to 41 dB. This still corresponds toa very good quality. PSNR fairness improvements are alsoobtained whenN = 4 andN = 6.

Figure 8 shows the evolution of the buffer level of theNprograms over300 GoPs using the TRF (left) and the QF(right) controllers forN = 2, N = 4, and N = 6. Thediscrepancy between the buffer level and the reference level B0

remains limited for most of the time. When only the encodingrate is controlled, corresponding to the TRF controller, thebuffer level stabilizes aroundB0. The buffer level variationsincrease using the QF controller due to the interactions of theencoding rate and transmission rate control loops.

Figure 9 shows the evolution of the buffer level (left) and ofthe PSNR (right) of theN programs over300 GoPs when theUMMF technique is used. The choice of the proportional gain

Figure 7. Evolution of the PSNR forN = 2, N = 4, andN = 6 usingTRF (left) and QF (right) controllers when controlling the buffer levels

Time(s)0 100 200 300

0

200

400

600

800

kb

its

Time(s)

N=6

0 100 200 3000

200

400

600

800

0 100 200 3000

200

400

600

800

N=4

0 100 200 3000

200

400

600

800

kb

its

0 100 200 3000

200

400

600

800

N=2

0 100 200 3000

200

400

600

800

kb

its

B0 Prog 1 Prog 2 Prog 3 Prog 4 Prog 5 Prog 6

Figure 8. Evolution of the buffer level forN = 2, N = 4, andN = 6using TRF (left) and QF (right) controllers when controlling the buffer levels

for the transmission rate controller has no significant impacton the quality fairness, provided that the buffers remain full.A reference buffer level of400 kbits has been used to allow asatisfying behavior of the transmission rate control loop.ThePSNR remains around40 dB, but the average variance is ofthe same order of magnitude as that of the TRF solution, seeTable II. This mainly comes from the target encoding rateevaluation on (one step) outdated PSNR-rate characteristics.

4) SSIM-rate model: The quality fairness is also addressedconsidering the SSIM metric. To tune the PI controllers, an

11

Figure 9. Evolution of the buffer level and of the PSNR (right) for N = 6using the UMMF technique (left) and using the QF controllers(right) with atransmission rate control using the buffer levels

arctan SSIM-Rate utility model is considered

Ui(j) = Si(j) = f (ai (j) , Rei(j)) (44)

= a(1)i (j)atan(a(2)i (j)Re

i(j))

whereSi(j) is the SSIM of programi at time j. As before,the two entries of eachai (j) are derived from four encodingtrials performed on each of theN considered programs. Theresulting values of the parameters are

a1 (1) =

(

0.640.037

)

, a2 (1) =

(

0.610.029

)

, a3 (1) =

(

0.640.034

)

,

a4 (1) =

(

0.620.017

)

, a5 (1) =

(

0.640.22

)

, a6 (1) =

(

0.640.044

)

.

Figure 10 compares the actual SSIM-Rate characteristicsand those obtained using the model (45) for the first GoPof the six considered programs. The model (45) is ableto predict accurately the SSIM over a large range of rates.The correlation coefficient for the six programs isr2 =[0.99, 0.97, 0.99, 0.98, 0.99, 0.99] using the same rate values inthe PSNR-rate model estimation, which confirms the accuracyof the SSIM-rate model.

Using the SSIM-rate utility function (45), one is able to getthe vector

(

Re,eq1 , . . . , Re,eq

N

)Tof encoding rates as the solution

of (24). Ξ andΓ are derived from (29), (30) and (45) as follow

Ξ =

atan(a(2)1 Re,eq1 )

a(1)1 R

e,eq1

1+(a(2)1 R

e,eq1 )2

0 0 0

0. . .

. . ....

...... 0

. . . 0 0

0 . . . 0 atan(a(2)N Re,eqN )

a(1)N

Re,eqN

1+(a(2)N

Re,eqN

)2

(45)and

Γ = diag

(

a(1)1 a

(2)1

1+(a(2)1 R

e,eq1 )2

. . .a(1)N

a(2)N

1+(a(2)N

Re,eqN

)2

)

(46)

The choice of the parameters of the controllers is done as inSection V-C2. Good transient behaviors have been obtainedwith Ke,b

P = 666, Ke,bI = 33, K t

P = 66 × 104, and K tI =

Figure 10. SSIM-Rate model for the six considered programs

1.3×104. For this choice of the control gains, Figure 11 showsthe minimum, average, and maximum values of the PSNR(left) and SSIM (right) over all GoPs of theN = 6 programs.The QF controller improves quality fairness especially forthemost demanding videos, such as Prog2. The proposed QFcontroller improves also the minimum achieved quality forthese programs. In fact, even if events corresponding to theseminimum quality happens only few times, the user perceptionis sometime dominated by the worst experience, rather thanthe average. The price to be paid is, as expected, a decreaseof the quality of the less demanding programs.

D. Control of the buffering delays

This part focuses on the system performance when thebuffering delays are used to evaluate the encoding rates.

1) Controller design: The same PSNR-rate utility model asin (38) is considered in this section. Thus the matricesΞ andΓ are those in (39) and (40).

The choice of the parameters of the two PI controllersis again done as in Section V-C2. Now, the roots ofdet (zI−A

τ ) have to remain within the unit circle. Goodtransient behaviors have been obtained forN = 2, N = 4,andN = 6 with K t

P = 66× 103, K tI = 2600 Keτ

P = 66× 103,andKeτ

I = 1300.In parallel, the parameterα in (9) is tuned to provide the

best estimate of the buffering delay. Figure 12 represents themeans square error MSE(τ , τ) between the actual bufferingdelay τ and the estimated oneτ as a function ofα. Thevalueα = 0.2 provides the best estimate. The evolution withtime of the actual buffering delayτ and of its estimateτ isrepresented in Figure 13 forN = 4 usingα = 0.2 and theQF controller for the PSNR faireness. For this choice ofα,the estimate provided by (9) for the four video sequences isquite good for most of the time.

2) Results: The performance of the QF controller usingN = 2, N = 4, andN = 6 programs is evaluated in all caseswith a transmission rateRc = 4 Mps.

12

Figure 11. Minimum, average, and maximum values of the PSNR (left) and SSIM (right) over all GoPs for the TRF and the QF controllers usingN = 6programs when controlling the buffer levels

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.450

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

0.02

α

MS

E

Figure 12. MSE of the estimated buffering delay as a functionof α.

Figure 13. Evolution of the actual buffering delayτ and of its estimateτfor N = 4 usingα = 0.2.

We evaluate the PSNR discrepancy∆P (in dB) and the

PSNR varianceσ2P (in dB2) defined in (43). Additional

performance measures are the average delay discrepancy

∆τ =1

N

N∑

n=1

(

1

M

M∑

l=1

(τn(l)− τ0)

)

(47)

of the buffering delay with respect toτ0 and the variance ofthe buffering delay

σ2τ =

1

N

N∑

n=1

(

1

M

M∑

l=1

(τn(l)− τ0 −∆τ )2

)

, (48)

whereM is the number of GoPs in the video streams. Resultswith N = 6 are summarized in Table III in the two cases: QFand TRF controllers. Here again, one notices that using PI

Ke,τP ,K

e,τI K t

P, KtI ∆τ σ2

τ ∆P σ2

P

TRF 66× 103, 0 0, 0 0.25 0.12 3.8 10.5QF 66× 103, 1300 66× 103, 2600 0.6 0.35 2 10

Table IIIPERFORMANCE OFQF AND TRF CONTROLLERS WHEN CONTROLLING

THE BUFFERING DELAYS FORN = 6.

controllers for the transmission rate control loop reducesthePSNR discrepancy between the programs at the price of someincrease of the buffering delay discrepancy and variance.

Figure 14 represents the evolution of the PSNR whenconsidering the TRF controller (left) and the proposed QFcontroller (right)N = 2, N = 4, and N = 6 programs.The proposed QF controller reduces the PSNR discrepancybetween theN programs compared to the TRF controller.Compared to Figure 7, the control with the buffer level appearsto be less reactive. For example, whenN = 2, to improvethe PSNR of the second program, the PSNR of the firstprogram has to be decreased. In Figure 7, the PSNRs arealmost immediately adjusted. This is done with some delay inFigure 14. This may be due to the difficulty to accuratelyestimate the buffering delay. A better response could beobtained by increasingKe,τ

P , which relates the buffering delayand the encoding rate. This, however, would be at the price

13

of a loss in robustness of the global system to variations ofthe PSNR-rate characteristics of the programs.

0 50 100 150 200

30

40

50

GoP index

50 100 150 200

30

40

50

50 100 150 200

30

40

50

PS

NR

(dB

)

N=4

50 100 150 200

30

40

50

50 100 150 200

30

40

50

GoP index

PS

NR

(dB

)

N=6

50 100 150 200

30

40

50

N=2

PS

NR

(dB

)

PSNR 1 PSNR 2 PSNR 3 PSNR 4 PSNR 5 PSNR 6

Figure 14. Evolution of the PSNR forN = 2, N = 4, andN = 6 usingTRF (left) and QF (right) controllers when controlling the buffer delays.

Figure 15 represents the evolution of the buffering delaysof N programs of300 GoPs using the TRF (left) and the QF(right) controllers forN = 2, N = 4, and N = 6. Withthe TRF controller, the buffering delays reach rapidlyτ0 andshow a reduced variance compared to a system with a QFcontroller. The larger variations of the buffering delay for theQF controller are due to the interactions of both control loops(encoding rate and transmission rate). Again,Ke,τ

P appears tobe too low: large deviations of the buffering delay are requiredto reach PSNR fairness.

50 100 150 2000

2

4

GoP index

Del

ay (

s)

N=6

50 100 150 2000

2

4

GoP index

50 100 150 2000

2

4

50 100 150 2000

2

4

Del

ay (

s)

N=450 100 150 200

0

2

4

50 100 150 2000

2

4N=2

Del

ay (

s)

τ0 Prog 1 Prog 2 Prog 3 Prog 4 Prog 5 Prog 6

Figure 15. Evolution of the buffering delay forN = 2, N = 4, andN = 6using TRF (left) and QF (right) controllers when controlling the buffer delays.

E. Robustness of the proposed solution to variations of thenumber of users and of the channel rate

In this section, the robustness of the proposed controlsystem (buffering delay control) is evaluated with respecttovariations of the channel rate and of the number of transmittedvideo programs. Similar results are obtained when bufferlevels are controlled.

First, the numberN of transmitted video programs evolveswith time (left). Second, the rate of the channel switchesbetweenRc = 3.5 Mbits/s andRc = 5 Mbits/s (right), seeFigure 16. The PSNR is used as quality measure. When a newvideo program is transmitted, initially, it has no transmissionrate allocated by the MANE (since at timej the controllerderives the encoding rate for timej + 1). Thus, we chooseto set the encoding rate at that time asRc/N . In Figure 16(left), Prog4 is not transmitted between GoP35 and65. Thesame values for the gains of the PI controllers are used hereas in Section V-D.

Figure 16. System performance using PI controllers while multiplexing fourvideo programs using the proposed QF controller when considering variationsof the channel rate (left) and of the number of programs (right).

When the channel rate increases or when a video program isno more transmitted, the bandwidth allocation adapts rapidlyto this change by providing more rate to programs withlow video quality (here Prog2). When the channel ratedecreases or when a new video program is transmitted, thebandwidth allocation performs well, showing the robustnessof the proposed control system to variations of the channelrate and to the number of transmitted video programs.

VI. CONCLUSIONS AND PERSPECTIVES

In this paper, we propose encoding and transmission ratecontrollers for the transmission of several video streams target-ing similar video quality between streams as well as efficientcontrol of the buffering delay. The controlled system is mod-eled with a discrete-time non-linear state-space representation.PI controllers for the transmission rate and the encoding ratecontrol are considered. The delay introduced by the networkpropagation between the MANE and the encoders is taken intoaccount. This allows to test the stability of the control system

14

in presence of feedback delay. Simulation results show thatthequality fairness (measured with PSNR or SSIM) is improvedcompared to a solution providing an equal transmission rateallocation. Moreover, the jitter of the buffering delay remainsreasonable. The robustness to variations of the characteristicsof the channel and of the number of transmitted programs hasbeen shown experimentally.

Simulations are performed at the GoP granularity. Controlat the frame level should be considered, however this mayrequire a better consideration of the communication delaybetween the MANE and the encoders which may be variablewith the time. This would also require to better accountfor the delay, which significantly impedes the behavior ofthe global control system, especially when controlling thebuffering delay. Tools devoted to the control of time-delaysystems may be useful in this context, see,e.g.,[32].

REFERENCES

[1] “Cisco visual networking index: Forecast and methodology, 2008-2013,”Tech. Rep., Cisco, 2009.

[2] Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, “Imagequality assessment: from error visibility to structural similarity,” IEEETransactions on Image Processing, vol. 13, no. 4, pp. 600 – 612, April2004.

[3] Z.L. Zhang, S. Nelakuditi, R. Aggarwal, and R.P. Tsang, “Efficientselective frame discard algorithms for stored video delivery acrossresource constrained networks,”Real-Time Imaging, vol. 7, no. 3, pp.255 – 273, March 2001.

[4] T-L. Lin, Y. Zhi, S. Kanumuri, P.C. Cosman, and A.R. Reibman,“Perceptual quality based packet dropping for generalizedvideo gopstructures,” inIEEE International Conference on Acoustics, Speech andSignal Processing, April 2009, pp. 781 – 784.

[5] S. Ma, W. Gao, and Y. Lu, “Rate-distortion analysis for H.264/AVCvideo coding and its application to rate control,”IEEE Transactions onCircuits and Systems for Video Technology, vol. 15, no. 12, pp. 1533 –1544, December 2005.

[6] Z. He and D. O. Wu, “Linear rate control and optimum statistical multi-plexing for H.264 video broadcast,”IEEE Transactions on Multimedia,vol. 10, no. 7, pp. 1237 – 1249, November 2008.

[7] S. K. Srinivasan, J. Vahabzadeh-Hagh, and M. Reisslein,“The effectsof priority levels and buffering on the statistical multiplexing of single-layer H.264/AVC and SVC encoded video streams,”IEEE Transactionson Broadcasting, vol. 56, no. 3, pp. 281 – 287, September 2010.

[8] E. Maani and A.K. Katsaggelos, “Unequal error protection for robuststreaming of scalable video over packet lossy networks,”IEEE Trans-actions on Circuits and Systems for Video Technology, vol. 20, no. 3,pp. 407 – 416, March 2010.

[9] Y. Li, Z. Li, M. Chiang, and A.R. Calderbank, “Content-awaredistortion-fair video streaming in congested networks,”IEEE Trans-actions on Multimedia, vol. 11, no. 6, pp. 1182 –1193, October 2009.

[10] N. Changuel, B. Sayadi, and M. Kieffer, “Joint encoder and buffercontrol for statistical multiplexing of multimedia contents,” in IEEEGlobecom, December 2010, pp. 1 – 6.

[11] C.W Wong, O.C. Au, and H.K. Lam, “PID-based real-time rate control,”in IEEE International Conference on Multimedia and Expo, June 2004,vol. 1, pp. 221 – 224.

[12] Z. Guo and R. Rojas-Cessa, “Analysis of a flow control system fora combined input-crosspoint buffered packet switch,” inWorkshop onHigh Performance Switching and Routing, May 2005, pp. 336 – 340.

[13] Y-H. Zhang, K.H Li, C.Q. Xu, and L.M. Sun, “Joint rate allocationand buffer management for robust transmission of VBR video,” ActaAutomatica Sinica, vol. 34, no. 3, pp. 1 – 7, March 2008.

[14] Y. Huang, S. Mao, and S.F. Midkiff, “A control-theoretic approach torate control for streaming videos,”IEEE Transactions on Multimedia,vol. 11, no. 6, pp. 1072 –1081, October 2009.

[15] Y. Huang and S. Mao, “Analysis and design of a proportional-integralrate controller for streaming videos,” inIEEE Globecom, November2009, pp. 5530 – 5535.

[16] Y. Zhou, Y. Sun, Z. Feng, and S. Sun, “PID-based bit allocation strategyfor h.264/avc rate control,”IEEE Transactions on Circuits and SystemsII: Express Briefs, vol. 58, no. 3, pp. 184 – 188, March 2011.

[17] J.W Cho and S. Chong, “Utility max-min flow control usingslope-restricted utility functions,” inIEEE Globecom, December 2005, vol. 2,pp. 818 – 824.

[18] S. Cicalo and V. Tralli, “Cross-layer algorithms for distortion-fairscalable video delivery over OFDMA wireless systems,” inIEEEGlobecom Workshops, 2012, pp. 1287 – 1292.

[19] E. Witrant, C. Canudas-de Wit, D. Georges, and M. Alamir, “Remotestabilization via communication networks with a distributed control law,”IEEE Transactions on Automatic Control, vol. 52, no. 8, pp. 1480 –1485, August 2007.

[20] Motorola, “Statistical multiplexing over IP StatmuxIP solution,” Tech.Rep., Motorola, 2009, XP002565902.

[21] Harmonic, “IP-based distributed statistical multiplexing solution,” Tech.Rep., 2011.

[22] N. Changuel, B. Sayadi, and M. Kieffer, “Control of distributed serversfor quality-fair delivery of multiple video streams,” inACM internationalconference on Multimedia, October 2012, pp. 269 – 278.

[23] K. Seshadrinathan, R. Soundararajan, A.C. Bovik, and L.K. Cormack,“Study of subjective and objective quality assessment of video,” IEEETransactions on Image Processing, vol. 19, no. 6, pp. 1427 – 1441, June2010.

[24] T. Wiegand, H. Schwarz, A. Joch, F. Kossentini, and G.J.Sulli-van, “Rate-constrained coder control and comparison of video codingstandards,” IEEE Transactions on Circuits and Systems for VideoTechnology, vol. 13, no. 7, pp. 688 – 703, July 2003.

[25] K.-B. Kim and H.-J. Kim, “Back-pressure buffering scheme to improvethe cell loss property on the output buffered atm switch,” inIEEEConference on Local Computer Networks, October 1996, pp. 242 – 248.

[26] ETSI, “Multimedia Broadcast/Multicast service. (MBMS);UTRAN/GERAN requirements,” Tech. Rep., 3GPP TR 25.992-140, 2005.

[27] V. Vukadinovic and J. Huschke, “Statistical multiplexing gains ofH.264/AVC video in E-MBMS,” inInternational Symposium on WirelessPervasive Computing, May 2008, pp. 468 – 474.

[28] x264 Home Page, “Videolan organization. retrieved 2011-03-11.,” 2011.[29] ffmpeg Documentation, “http://ffmpeg.org/ffmpeg.html,” .[30] Association of Radio Industries and Businesses, “Multimedia Broad-

cast/Multicast Service (MBMS) user services stage 1 (release 8),” Tech.Rep., 3GPP TS, 2008.

[31] X. Zhu, P. Agrawal, J. Pal Singh, T. Alpcan, and B. Girod,“Rateallocation for multi-user video streaming over heterogenous accessnetworks,” in ACM Multimedia, Augsburg, September 2007, pp. 37– 46.

[32] W. Michiels and S.L. Niculescu,Stability and Stabilization of Time-Delay Systems: An Eigenvalue-Based Approach, Society for Industrial& Applied Mathematics,U.S., 2007.

Nesrine Changuel (IEEE S’09 - M’12) receivedher B.S. degree in 2006 and Eng. degree andM.S. degree in electrical engineering in 2008 fromthe Ecole Nationale Supérieure d’Electronique et deRadioélectricitée de Grenoble, France. She obtaineda PhD degree in Control and Signal Processing in2011 from the Paris-Sud University, Orsay. She iscurrently working as a research engineer in AlcatelLucent Bell Labs. Her current research interests arein the areas of video coding, Statistical multiplexingof video programs, scalable video, Rate and Distor-

tion model, resource allocation and scheduling, Markov decision processes(MDPs), and reinforcement learning. She is a Member of the IEEE.

15

Bessem Sayadiis a member of technical staff inMultimedia Technology domain at Alcatel-LucentBell Labs, France. He is member of Alcatel-LucentTechnical Academy, since 2008. He received M.Sc.(00) and Ph.D. (03) degrees in Control and Signalprocessing from Supélec, Paris-Sud University, withhighest distinction. He worked previously as apostdoctoral fellow in the National Centre for Sci-entific Research (CNRS), and as a senior researcherengineer in Orange Labs. His main research inter-ests are in the area of broadcast technology (DVB,

3GPP), video coding and transport, and resource allocationalgorithms forcommunication networks. He has authored over 55 publications in journaland conference proceedings and serves as a regular reviewerfor severaltechnical journals and conferences. He holds nine patents and has more thantwenty patent applications pending in the area of video coding and wirelesscommunications.

Michel Kieffer (IEEE M’02 - SM’07) received in1995 the Agrégation in Applied Physics at the EcoleNormale Supérieure de Cachan. He obtained a PhDdegree in Control and Signal Processing in 1999, andthe Habilitation à Diriger des Recherches degree in2005, both from the Paris-Sud University, Orsay.

Michel Kieffer is a full professor in signal pro-cessing for communications at the Paris-Sud Univer-sity and a researcher at the Laboratoire des Signauxet Systèmes, Gif-sur-Yvette. Since 2009, he is alsoinvited professor at the Laboratoire Traitement et

Communication de l’Information, Télécom ParisTech, Paris.His research interests are in signal processing for multimedia, commu-

nications, and networking, distributed source coding, network coding, jointsource-channel coding and decoding techniques, joint source-network coding.Applications are mainly in the reliable delivery of multimedia contents overwireless channels. He is also interested in guaranteed and robust parameterand state bounding for systems described by non-linear models in a bounded-error context.

Michel Kieffer is co-author of more than 120 contributions in journals,conference proceedings, or books. He is one of the co-authorof thebook Applied Interval Analysis published by Springer-Verlag in 2001 andof the book Joint source-channel decoding: A crosslayer perspective withapplications in video broadcasting published by Academic Press in 2009.He is associate editor of Signal Processing since 2008, of the Journalof Communication and Information Systems since 2011 and of the IEEETransactions on Communications since 2012. In 2011, MichelKieffer becamejunior member of the Institut Universitaire de France.