Energy-Efficient BaseBand Processing via vBBUMigration in Virtualized Cloud-Fog RAN

Rodrigo Izidoro Tinini1, Daniel Macedo Batista1, Gustavo Bittencourt Figueiredo2, Massimo Tornatore3 4,Biswanath Mukherjee4

University of Sao Paulo1, Federal University of Bahia2, Politecnico di Milano3, University of California, Davis4

rtinini, batista@ime.usp.br1, gustavo@dcc.ufba.br2, massimo.tornatore@polimi.it3, bmukherjee@ucdavis.edu4

Abstract—Cloud-Fog Radio Access Networks (CF-RAN) wereproposed as an alternative network architecture to alleviate thehigh fronthaul capacity requested in traditional Cloud RAN(CRAN) by moving some BaseBand Units (BBUs) from the cloudnodes to fog nodes closer to users. However, when BBU processingis moved into fog nodes, OPEX and CAPEX will increase, andthe cost and energy savings introduced by CRAN will alsoreduce. Moreover, mobile traffic fluctuations may lead to anunbalanced resource utilization and energy-inefficient operationin fog nodes. To address this problem, processing functions infog nodes could be activated and deactivated in function ofnetwork traffic and BBUs placed on fog nodes could be migratedto cloud nodes when network traffic is low. In this paper, wepropose an Integer Linear Programming (ILP) formulation toaddress this dynamic resource allocation problem. By meansof Network Functions Virtualization (NFV), virtualized BBUs(vBBUs) can be dynamically allocated and deallocated in fognodes. Furthermore, considering the availability of cloud nodesand the optical fronthaul, vBBUs can be migrated from fog nodesto cloud nodes in order to balance processing loads and saveenergy. Compared to a baseline incremental algorithm withoutvBBU migration, our proposal reduces blocking probability in89% and achieves power savings of 38%, while providing a verysmall rate of service interruption due to vBBUs migration.

Index Terms—5G networks, CF-RAN, VPON, NFV

I. INTRODUCTION

Cloud Radio Access Networks (CRANs) were proposedto reduce cost and energy footprint of traditional DistributedRAN (DRAN). In CRAN, BaseBand Units (BBU) are movedfrom cell sites to a BBU pool located in the cloud so that asingle infrastructure can be used to implement BBU processingof several cell sites, saving OPEX and CAPEX. At cell sites,High Power Nodes (HPN) are replaced by Remote RadioHeads (RRH) responsible for gathering user equipments (UEs)baseband signals and transmitting them for processing in thecloud. The cloud is composed of dedicated servers whereVirtual Digital Units (VDU) are executed, working as con-tainers of virtual BBUs (vBBUs), thus forming a virtualizedBBU pool [2], in which virtualized baseband processing isperformed. A virtualized BBU (vBBU) pool can offer manyadvantages for the network operation, e.g., it enables dynamicvBBU deployment to adapt to varying traffic profiles [9].

Study supported in part by CAPES - Finance Code 001, the INCTof the Future Internet for Smart Cities (CNPq 465446/2014-0, CAPES88887.136422/2017-00, and FAPESP 14/50937-1 and 15/24485-9) and CNPq311608/2017-5, 420907/2016-5, and 312324/2015-4.

Furthermore, as BBUs from different cell sites are virtualizedon the vBBU pool, vBBU intercommunication can supportmechanisms such as enhanced ICIC and Coordinated Multi-Point (CoMP), that require low delay between the vBBUs.

The centralization of BBUs enables significant cost sav-ings, but it also imposes high traffic demands on the opticalfronthaul, as baseband signals generated by RRHs must betransported by the bandwidth-intensive Common Public RadioInterface (CPRI) protocol [2]. In CPRI protocol, line ratesrange from 614.4Mbps to 24.3Gbps depending on MultipleInput-Output (MIMO) configuration of RRHs. Moreover, theCPRI-based fronthaul transport is subject to a delay limit of3ms between the RRH and its corresponding BBU [10]. So,as the fronthaul becomes more congested it will be harder tooperate under the delay threshold due to possible queuing ofRRHs transmissions and processing demands in the cloud [7],which may lead to reduced wireless coverage or even denialof services.

To alleviate the bandwidth requirements of the fronthaul,in a previous work [6], we proposed an architecture calledCloud-Fog RAN (CF-RAN) that exploits fog computing toplace vBBUs into fog nodes close to users. In CF-RAN, theoptical fronthaul is extended with optical links connectingRRHs to fog nodes. As traffic demands grow, fog nodes areactivated and vBBUs are dynamically instantiated using theemerging Network Functions Virtualization (NFV) paradigmto support BBU processing. Thus, when a RRH is active, theoperator must find a processing node to instantiate a vBBUfor baseband processing and, when a RRH is deactivated, itsinstantiated vBBU may be turned off. However, in [6] we onlyfocused on static network operation and did not investigate theeffects of migrating vBBUs to balance load in CF-RAN. Inrealistic scenarios, mobile traffic load is highly dynamic andfollows different patterns according to localization and time ofthe day. Hence, a different number of RRHs would be activatedalong the day to support such variable load. Furthermore, thisload fluctuation also leads to an unbalanced utilization of fognodes, since VDUs get less loaded when RRHs are turned off,and their corresponding vBBUs are deactivated. Hence, somevBBUs in lightly-loaded fog nodes could be migrated to thecloud in order to turn the fog nodes off, balance the load andsave power.

When a vBBU is deployed or migrated, the network oper-

ator must also allocate enough bandwidth to transmit CPRItraffic from the RRH to the vBBU. Several works assumethat the fronthaul links can be implemented over the chan-nels of a Time-and-Wavelength Division Multiplexed PassiveOptical Network (TWDM-PON) due to its high bandwidth,low transmission delays and low costs of operation [6] [8].An alternative architecture that allows the creation of VirtualPONs (VPON) on top of a TWDM-PON was suggestedin [2] [7] [8]. In such architecture, a dedicated PON iscreated to transmit the CPRI traffic from a group of RRHsto a common processing node, so that several RRHs share avirtualized BBU pool as well as the PON capacity throughTime-Division Multiplexing (TDM).

As vBBUs are dynamically deployed and deactivated incloud or fog nodes, VPONs must be dynamically created tosupport such variable bandwidth, and they must be properlyscheduled to efficiently utilize the limited wavelength capacityof the TWDM-PON. This characterizes a joint optimizationproblem in which operators must decide how to optimallyplace or migrate vBBUs and how to create VPONs to supportCPRI transmission of a newly deployed/migrated vBBU.

So, in this paper we propose an approach based on an In-teger Linear Programming (ILP) model to dynamically decidethe vBBU placement, creation of VPONs and migration ofvBBUs via reconfiguration of VPONs and reallocation of pro-cessing resources, following traffic fluctuations. Simulationsshow that the dynamic placement and migration of vBBUs leadto significant savings in power consumption and also to lowblocking probability rates. The rest of the paper is organizedas follows: Section II discusses some relevant related works;Section III introduces the proposed virtualized CF-RAN ar-chitecture; Section IV describes the problem of dynamicallyplacing vBBUs and creating VPONs; Section V presents theproposed ILP model; in Section VI results obtained fromsimulations are shown; Section VII concludes the paper.

II. RELATED WORK

Some recent works have investigated how to efficientlyplace BBUs and/or create VPONs to support transmissionsto the cloud and to fog nodes under static traffic settings.However, these works only focus on static network scenariosand algorithms to handle dynamism and migration of basebandprocessing in mobile traffic are often neglected. In [12],authors proposed the reconfiguration of VPONs as trafficchanges in order to reduce power consumption. However, thisstudy only considered the reconfiguration of the VPONs ina centralized scenario. The novelty of our work resides inminimizing active resources by promoting the reconfigurationof not only VPONs, but also processing resources (VDUs)in a distributed fog scenario where each processing resourceand even processing nodes can be dinamically turned off.In [7], authors proposed a TWDM-PON based fronthaul thatuses VPONs to reduce processing latency in CRAN. In [8],the dimensioning of wavelengths and VPONs creation tosupport transmissions both to the cloud and to fog nodeswere explored. In this work, the amount of VPONs created on

the fronthaul was decided as a function of the free basebandprocessing capacity on the cloud. Authors also found thatpower consumption from processing and transmission can bereduced when the number of created VPONs is minimized.However, the impact of traffic fluctuations on the fronthaul andallocated resources was not considered, and only static trafficprofiles were considered. In [2], the relation of bandwidthavailability and power consumption in static networks wasexplored when CPRI traffic is split between cloud and fognodes in a Hybrid CRAN (H-CRAN). Authors showed that,when more bandwidth is available on fronthaul, BBUs canbe moved to the cloud to save energy. However, the specificaccess network technology to connect fog nodes and theallocation of transmission channels for fog nodes was leftas an open problem. Overall, fog architectures and servicemigration introduce some trade offs between power, blocking,service disruption and bandwidth usage that shall be accuratelyexplored in order to maintain a balanced network operation.However, none of these works explored the characteristicsand reasons of these trade offs. In next section we presentthe problems of dynamic vBBU placement and creation ofVPONs.

III. SYSTEM ARCHITECTURE

Our proposed CF-RAN architecture is composed of both fognodes and cloud nodes (see Fig. 1 (a)). Each of these nodescan host a set of virtual containers of baseband processingcalled VDUs. A VDU is responsible for implementing thevBBUs, which are responsible for the virtualized basebandprocessing of CPRI traffic. In each VDU, the number ofvBBUs is limited by its processing capacity. If some VDUhas no longer capacity to host a vBBU, a new VDU can bedynamically turned on to deploy new vBBUs. To provide con-trol signaling between RRHs with vBBUs placed in differentVDUs, an interconnection element like a backplane ethernetswitch is used to redirect traffic between VDUs. In fact, thebackplane ethernet switch implements the X2 interface usedto control signaling in traditional LTE-networks. The transportsegment of the CF-RAN is composed of TWDM-PON linksconnecting RRHs to fog nodes and to the cloud through thefronthaul. A level 1 internal optical splitter is used to multiplexincoming traffic from RRHs to fog nodes and to a feeder fiberconnected to a level 2 optical splitter (see Fig. 1 (b)). Thelevel 2 optical splitter multiplexes several distribution fibersto the cloud nodes. Each processing node is equipped witha virtualized Optical Line Terminal (OLT) that is responsiblefor allocating wavelengths to Optical Nework Units (ONUs)connected to RRHs in order to transmit to that node. EachONU is equipped with tunable lasers that can be tuned toany available wavelength. A VPON is created for a group ofRRHs when the OLT of a processing node allocates the samewavelength to their ONUs. Note that, the proposed architectureis not the ITU-T standard architecture for TWDM-PON, butan upgrade architecture where wavelengths can be received ortransmitted in different locations and thus at different opticalsplitter.

Fig. 1. a) CF-RAN architecture and example of wavelength dimensioningthrough cloud and fog nodes; b) Internal architecture of fog/processing node

Fig. 2. a) Processing nodes unbalanced before vBBU migration, b) Processingnodes balanced after vBBU migration

Each OLT is equipped with a set of Line Cards (LCs)(Fig. 1 (b)), i.e. transceivers, that are tuned to transmit ona specific wavelength (a VPON) and that is associated witha single VDU. So, the OLT is responsible for switching theincoming traffic from different VPONs to its associated VDU.Moreover, we assume that LCs can be dynamically turned onor off as VPONs are created or deactivated. Hence, if lessVPONs are active in a node, less transceivers are used, whichresults in power savings. Note that, when using the backplaneethernet switch, a single VPON can redirect traffic to VDUsassociated to other LCs if the VDU associated to its LC hasno longer capacity, but this may increase processing delaysand the power consumed by the backplane ethernet switch.

IV. JOINT OPTIMAL BASEBAND PLACEMENT AND VPONCREATION

Placement of baseband processing functions in CF-RANinvolves two major steps: first, a processing node must bechosen to support vBBUs; second, a VPON must be createdto support the transmission of CPRI traffic to that processingnode.

We consider that vBBUs can be deployed according totraffic demands. As soon as a RRH becomes active, a vBBU

must be placed in a processing node with enough processingcapacity. As power consumption in CF-RAN mostly comesfrom active processing nodes, to save power, first, only VDUsin the cloud are activated. After vBBUs are deployed in thecloud, the operator must decide how many VPONs will becreated. In this study, we assume that the number of VPONscreated in the fronthaul is a function of the amount of basebandprocessing capacity of the cloud. For instance, if 10 RRHs witha 2X2 20MHz MIMO configuration (each of them requiring2.4Gbps) must be supported in the cloud, three VPONs with10Gbps capacity are necessary in the fronthaul to support the24Gbps generated by these 10 RRHs [10].

Only when fronthaul or cloud processing capacity is ex-hausted, the fog nodes are activated to deploy VDUs. Notethat, activating more fog nodes frees up fronthauling capacity,but it also increases OPEX, since cooling and processing func-tions (VDUs and vBBUs) need to be activated accordingly.

Similarly, after vBBUs are deployed on fog nodes, VPONsmust be created by OLTs in fog nodes to transmit CPRI trafficto fog nodes. Since the number of wavelengths is limited,the number of VPONs created by the cloud OLT must beaccurately dimensioned so that other wavelengths can be usedto create VPONs by the fog nodes OLTs if fog nodes areactivated. Instead, if all wavelengths are used to create VPONsin the cloud, it will not be possible to transmit CPRI trafficto vBBUs deployed in fog nodes. An example of an optimalVPONs dimensioning between cloud and fog nodes is depictedin Fig. 1 (a). Considering that vBBUs were placed both incloud and fog nodes, in order to support transmissions to thecloud, VPONs 1, 2, 3 and 4 were created on the fronthaul bythe cloud OLT. To support transmissions to vBBUs in fognodes, VPONs 5, 6 were created by the OLT of fog node 1and VPONs 7, 8 by the OLT of fog node 2.

However, as traffic fluctuates, the number of necessarydeployed vBBUs and VPONs changes. In the next subsectionwe propose a scheme for re-deploying vBBUs and turn offfog nodes as traffic fluctuates.

A. Dynamic Rearrangement of vBBUs and VPONs

After the initial placement of vBBUs and VPON creationare done, if traffic load fluctuates, it may be necessary to turnoff fog nodes and migrate its hosted vBBUs to the cloud. So, asvBBUs are turned off, VDUs in the cloud and in the fog nodesmay become lightly-loaded. In this case, vBBUs and VPONsplaced in fog nodes could be migrated to the cloud and fognodes deactivated to save power and bandwidth. In order toachieve this, we propose a vBBU migration mechanism relyingon NFV and Live Migration (LM) capabilities.

The vBBU migration mechanism is composed of a loadmonitor that is implemented on top of a VDU in the cloud.This load monitor measures the current load on each VDU ofthe network. It considers a load threshold value to be checkedevery time a vBBU is turned off. When a certain load thresholdis reached, it decides if it is convenient to turn off VDUs in afog node and migrate its vBBUs to the cloud. If the cloud hasenough capacity on its VDUs, vBBU migration is performed.

This process is depicted in Fig. 2. As the BBU pool in thecloud has enough capacity to support vBBUs from fog nodes 1and 2, those vBBUs are migrated to the cloud. Note that beforethe vBBU migration, two VPONs were used to transmit to fognodes. As the amount of vBBUs migrated can be supported byVPON1, VPON2 is turned off and its ONUs are reconfiguredto transmit to VPON1. However, while the migration is inprocess, the mobile service gets interrupted. So, the vBBUmigration scheme must seek to optimize the network resourcesperformance while minimizing the interruption of service.

V. ILP FORMULATION

We propose an ILP formulation to decide where to deployvBBUs and the number of VPONs to transmit CPRI load.

Input ParametersR: set of RRH traffic demands i, N : set of processing nodes

(cloud of fog nodes) n, Fin: set of binary values representingfog nodes n connected to RRH i, Vwn: set of binary valuesthat represent the availability of each VPON w to be placedon node n, W : set of available wavelengths w and VDUs, Bi:bandwidth demand of RRH i, B|W |: capacity of wavelengthw, Iw: processing capacity of VDU w, Be|N| : bandwidth ofthe backplane switch e at node n, Cn: power cost of noden, Clc: power cost of a LC, Ce: power cost of the backplaneswitch, B: a very big positive number.

Decision Variablesxiwn: = 1 if the traffic demand of RRH i is processed at

node n being transmitted at the VPON w, uiwn: = 1 if RRHi is processed at the VDU w at node n, yin: = 1 if i wasallocated to node n, x|N |: = 1 if node n is active, zwn: = 1 ifwavelength w transmits to node n, kin: = 1 if traffic from RRHi was redirected to VDU w at node n, rwn: = 1 if VDU w wasactivated to receive a redirected RRH at node n, swn: = 1 ifVDU w is active at node n, e|N |: = 1 if the backplane switche is active at node n, giwn: auxiliary variable that equals 1 iftraffic of RRH i is redirected to VDU w at node n.

Objective FunctionThe objective function (1) minimizes both the active pro-

cessing nodes and the VPONs in order to provide a power-efficient operation while optimally using the available band-width.

(1)Min.|N|∑n=1

xn.Cn +|W |∑w=1

|N|∑n=1

zwn.Clc

Constraints

(2)|W |∑w=1

|N|∑n=1

xiwn=1, ∀i∈R, (3)|W |∑w=1

|N|∑n=1

uiwn=1, ∀i∈R

(4)|R|∑i=1

uiwn≥0, ∀w,n∈W , N, (5)|N|∑n=1

yin=1, ∀i∈R

(6)|N|∑n=1

zwn≤1, ∀w∈W , (7)zwn≤Vwn, ∀w,n∈W,N

(8)yin≤Fin, ∀i,n∈R,N, (9)|R|∑i=1

|N|∑n=1

xiwn.Bi≤B|W |, ∀w∈W

(10)|R|∑i=1

|N|∑n=1

uiwn≤I|W |, ∀w∈W , (11)|R|∑i=1

kin.Bi≤Be|N| , ∀n∈N

(12)B.x|N |≥|R|∑i=1

|W |∑w=1

xiwn, ∀n∈N, (13)x|N |≤|R|∑i=1

|W |∑w=1

xiwn, ∀n∈N

(14)B.zwn≥|R|∑i=1

|N|∑n=1

xiwn, ∀w∈W , (15)zwn≤|R|∑i=1

|N|∑n=1

xiwn, ∀w∈W

(16)B.yin≥|W |∑w=1

xiwn, ∀i,n∈R,N, (17)yin≤|W |∑w=1

xiwn, ∀i,n∈R,N

(18)B.yin≥|W |∑w=1

uiwn, ∀i,n∈R,N, (19)yin≤|W |∑w=1

uiwn, ∀i,n∈R,N

(20)B.swn≥|R|∑i=1

uiwn, ∀w,n∈W,N, (21)swn≤|R|∑i=1

uiwn, ∀w,n∈W,N

(22)B.kin≥|W |∑w=1

giwn, ∀i,n∈R,N, (23)kin≤|W |∑w=1

giwn, ∀i,n∈R,N

(24)B.r|W |≥|R|∑i=1

|N|∑n=1

giwn, ∀w∈W , (25)r|W |≤|R|∑i=1

|N|∑n=1

giwn, ∀w∈W

(26)B.e|N |≥|R|∑i=1

kin, ∀n∈N, (27)e|N |≤|R|∑i=1

kin, ∀n∈N

(28)giwn≤xiwn + uiwn, ∀i,w,n∈R,W,N,

(29)giwn≥xiwn − uiwn, ∀i,w,n∈R,W , N

(30)giwn≥uiwn − xiwn, ∀i,w,n∈R,W , N

(31)giwn≤2− xiwn − uiwn, ∀i,w,n∈R,W , N

Constraints 2 to 5 ensure that each RRH i is allocatedto only one processing node, VPON and VDU. Constraint 6ensures that a VPON is allocated to at most one processingnode. Constraint 7 verifies that a RRH i is allocated only tothe cloud or to the fog node connected to it. Constraint 8 setsthe possible set of nodes that each VPON can be assignedto. Constraints 9, 10 and 11 impose the capacities of VPONs,nodes and the backplane switch capacity. Constraints 11 to14 activate processing nodes and VPON when demands areallocated to them. The rest of constraints enforce the activationof the backplane switch on each node and the activation ofadditional VDUs in case of traffic redirection.

The ILP can find the optimal solution for a single request,as well as for batch of requests. As network operation is highlydynamic, we also propose an algorithm that executes the ILP,called nfvILP, to handle dynamic traffic and to implement thevBBU migration mechanism. Algorithm 1 formally describesnfvILP. It first processes each request i incrementally andchecks if a vBBU migration is necessary (lines 2 and 10).If so, a batch containing the previously allocated requests isformed (line 3) and the ILP is executed for the batch (functionILP (.) in line 4). If not, it searches the optimal allocation ofi (line 10). When an optimal solution is found (both in caseof a single request or of a batch), the network state is updated(lines 6 and 12). If no optimal solution is found, an incomingrequest is blocked (lines 8 and 14). For the vBBU migration,the same algorithm is executed when a request departs. So,when vBBU migration is triggered, a batch of formed allocatedvBBUs is created (line(3)) and the ILP is executed to finda new optimum solution (lines 17 and 18) for the currentallocated requests in order to minimize the active resources

by performing vBBU migration. If a new optimal solution isfound, vBBUs are migrated and the network state is updated(line 20). If not, the migration is not performed (line 22).

Algorithm 1 nfvILPInput: RRH request i, Departing request j, Set of allocated Requests

BOutput: Optimum Allocation of i — Migration of vBBUs

1: for all Incoming request i do2: if Load threshold was reached? then3: B’←B + i4: Run ILP(B’)5: if B’ has a optimum global then6: nfvUpdate(Nstate)7: else8: Blocks i9: else

10: Run ILP(i)11: if Optimum global found for i then12: nfvUpdate(Nstate)13: else14: Blocks i15: for all Departing request j do16: if Load threshold was reached? then17: B’←B18: Run ILP(B’)19: if B’ has a optimum global then20: nfvUpdate(Nstate)21: else22: Do not migrate vBBUs

VI. ILLUSTRATIVE NUMERICAL EXAMPLES

To evaluate our proposal, we developed an event-drivensimulator and used the DOCPLEX Python API to implementthe ILP. Our simulations consider a CF-RAN composed of 1cloud and 4 fog nodes. Each processing node has 6 VDUs.In the cloud, each VDU can deploy 3 RRHs whereas in thefog each VDU can deploy just 1. The TWDM-PON has 6wavelengths with capacity of 10Gbps. We use the same powermodel of [2]. Finally, our scenario has 42 RRHs (maximumload of the processing nodes). We considered a typical 24hours business traffic profile slotted in periods of 1 hour asshown in Fig. 3 [4]. Regarding the vBBUs migration, weconsidered a Live Migration scheme where each vBBU takesapproximatelly a worst upper bound of 1.5ms to be migratedthrough a dedicated 10Gbps out-of-band optical channel fromthe fog node to the cloud [11].

As simulation starts, RRHs are turned off and begin tobe activated following a Poisson process with mean equal to(e/60), where e is the erlang for a given hour during thesimulation. Moreover, each RRH has a service time uniformlytaken from (0.25 hour, 1 hour), which means that each RRHcan stay turned on from 25 minutes to 1 hour. We compareour proposal to an incremental algorithm called incILP, thatincrementally runs the ILP formulation everytime each RRHbecomes active, but does not perform vBBU migration whenRRHs and vBBUs are turned off due to traffic fluctuations. Inour simulations, we studied the effects of vBBU migration fordifferent values of load threshold in VDUs used for initializing

the vBBU migration. The load threshold value corresponds tothe free processing capacity of a node, e.g., a value of 0.8means that the VDUs from a node has 80% of free capacity.

Fig. 3. Traffic load pattern

Fig. 4 (a) show the power consumption achieved by ouralgorithm. We observed that our proposal achieves higherpower savings. Compared to incILP, nfvILP achieves up to38% power savings in peak hours. It is also possible to seethat power consumption is small when vBBU migration istriggered sooner. This is so because the sooner the vBBUmigration occurs, the sooner the traffic is moved to the cloudand unbalanced resources are turned off. Regarding blockingprobability, we can observe that the vBBU migration playsan important role in reducing blocking, being able to reduceit up to 89% in comparison to incILP for the load thresholdof 0.8. We can also observe that, when more time is takento start vBBU migration, blocking tends to be more reduced.The bandwidth wastage (defined as 1− (Tcpri/Tvpons), whereTcpri is the total CPRI flow and Tvpons the amount ofavailable bandwidth) is also reduced when vBBU migrationis performed, as shown in Fig. 4 (c). This happens because,when vBBU migration is performed, nfvILP re-arranges theactive resources in an optimal scheduling that leads to theminimization of necessary VPONs to support the networkdemands. In comparison to incILP, the rate of bandwidthwastage can be reduced to at most 32% by nfvILP.

In Fig. 4 (d), the average amount of vBBUs migrationsare presented. Note that the highest amount of migrationsoccurs at the load threshold of 0.2, when fog nodes workloadare close to 100% and a higher number of vBBUs can bemigrated. This shows a clearly interplay between the powerconsumption and the triggering of vBBUs migration, as thelowest power consumption in achieved with the load thresholdof 0.2. On the other hand, although a higher number of vBBUsexpecting to be migrated leads to power-efficiency, an earlyre-arrangement of the network resources leads to an inefficientperformance and use of the network resources, leading to thehighest blocking probability experienced by nfvILP.

Fig. 5 (a) shows the average time that the baseband pro-cessing is interrupted in the network due to vBBU migration.Note that the vBBU migration only interrupts the basebandprocessing for a few seconds, which will leave to significantlygains in power consumption and blocking probability, as weobserved in Figs. 4 (a) and (b). The percentage of time that thebaseband processing is interrupted is shown in Fig. 5 (b). Note

Fig. 4. a) Power Consumption, b) Blocking Probability, c) Bandwidth Wastage, d) Average of External vBBUs Migrations

Fig. 5. a) Average down time of vBBUs during migration, b) Percentage of time that vBBUs service are interrupted due to migrations, c) Execution time ofthe algorithms

that the precentage of time in which the service is interruptedis extremely low for all values of load threshold, being under0.01%. It is important because it shows that our proposaldoes not interrupts the UEs connectivity constantly, which isimportant in order to keep a good Quality of Service (QoS).

Finally, Fig. 5 (c) shows the execution time of nfvILP. Incomparison to incILP, it shows an increased execution time,due to the fact that when vBBU migration is triggered, the sizeof the input to the algorithm is increased in function of thesize of the batch of RRHs. The execution time of nfvILP tendsto increase as the threshold value decreases, because with lowvalues of load threshold, more vBBUs in fog nodes expectsto be migrated as the migration process occurs early.

VII. CONCLUSION

In this paper we proposed an ILP formulation to dynami-cally allocate processing and network resources and performvBBU migrations in a 5G CF-RAN. This ILP formulationallows to dynamically solve the problem of placing vBBUsand creating VPONs for groups of RRHs in CF-RAN as thetraffic demands fluctuates over a day. Our proposal is able tohandle the network dynamism and achieves significant powersavings and reductions in blocking probability in comparisonto an incremental algorithm without vBBU migration. Finally,our proposed vBBU migration scheme provides very smalltimes and rates of service interruption. In our future works wewill propose a relaxed version of the ILP to solve this problemto even larger instances.

REFERENCES

[1] J. Wu, Z. Zhang, Y. Hong and Y. Wen, ”Cloud radio access network (C-RAN): a primer,” in IEEE Network, vol. 29, no. 1, pp. 35-41, Jan.-Feb.2015.

[2] X. Wang, A. Alabbasi and C. Cavdar, ”Interplay of energy and band-width consumption in CRAN with optimal function split,” 2017 IEEEInternational Conference on Communications (ICC), Paris, 2017, pp.1-6.

[3] Y. Y. Shih, W. H. Chung, A. C. Pang, T. C. Chiu and H. Y. Wei,”Enabling Low-Latency Applications in Fog-Radio Access Networks,”in IEEE Network, vol. 31, no. 1, pp. 52-58, January/February 2017.

[4] C. Peng, S. B. Lee, S. Lu, H. Luo, and H. Li. 2011. Traffic-driven powersaving in operational 3G cellular networks. In Proceedings of the 17thannual international conference on Mobile computing and networking(MobiCom ’11). ACM, New York, NY, USA, 121-132.

[5] M. Peng, S. Yan, K. Zhang and C. Wang, ”Fog-computing-based radioaccess networks: issues and challenges,” in IEEE Network, vol. 30, no.4, pp. 46-53, July-August 2016.

[6] R. I. Tinini, L. C. M. Reis, D. M. Batista, G. B. Figueiredo, M. Tornatoreand B. Mukherjee, ”Optimal placement of virtualized BBU processingin hybrid Cloud-Fog RAN over TWDM-PON,” GLOBECOM 2017 -2017 IEEE Global Communications Conference, Singapore, 2017, pp.1-6.

[7] G. B. Figueiredo, X. Wang, C. C. Meixner, M. Tornatore and B.Mukherjee, ”Load balancing and latency reduction in multi-user CoMPover TWDM-VPONs,” 2016 IEEE International Conference on Com-munications (ICC), Kuala Lumpur, 2016, pp. 1-6.

[8] R. I. Tinini, D.M. Batista and G. B. Figueiredo, ”Energy-efficientVPON formation and wavelength dimensioning in Cloud-Fog RAN overTWDM-PON,” 2018 IEEE International Symposium on Computers andCommunications, Natal-Brazil, 2018, pp. 1-6.

[9] A. Checko et al. ”Cloud RAN for mobile networksA technologyoverview.” IEEE Communications surveys and tutorials 17.1 2015: 405-426.

[10] A. de la Oliva, J. A. Hernandez, D. Larrabeiti, and A. Azcorra. (2016).An overview of the CPRI specification and its application to C-RAN-based LTE scenarios. IEEE Communications Magazine, 54(2), 152-159.

[11] S. Akoush, R. Sohan, A. Rice, A. W. Moore and A. Hopper, ”Predictingthe Performance of Virtual Machine Migration,” 2010 IEEE InternationalSymposium on Modeling, Analysis and Simulation of Computer andTelecommunication Systems, Miami Beach, FL, 2010, pp. 37-46.

[12] R. Wang, H. H. Lee, S. S. Lee and B. Mukherjee, ”Energy Saving viaDynamic Wavelength Sharing in TWDM-PON,” in IEEE Journal onSelected Areas in Communications, vol. 32, no. 8, pp. 1566-1574, Aug.2014.