optimal resource sharing in 5g-enabled vehicular …folk.uio.no/yanzhang/tvt5g2016.pdfallocate radio...

12
0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEE Transactions on Vehicular Technology 1 Optimal Resource Sharing in 5G-enabled Vehicular Networks: A Matrix Game Approach Rong Yu, Member, IEEE, Jiefei Ding, Student Member, IEEE, Xumin Huang, Student Member, IEEE, Ming-Tuo Zhou, Senior Member, IEEE, Stein Gjessing, Senior Member, IEEE, Yan Zhang, Senior Member, IEEE Abstract—Vehicular networks are expected to accommodate a large number of data-heavy mobile devices and multi-application services. Whereas, it faces a significant challenge when we need to deal with the ever-increasing demand of mobile traffic. In this paper, we present a new paradigm of 5G-enabled vehicular net- works to improve the network capacity and the system computing capability. We extend the original cloud radio access network (C-RAN) to integrate local cloud services to provide a low-cost, scalable, self-organizing and effective solution. The new C-RAN is named as Enhanced C-RAN (EC-RAN). Cloudlets in EC-RAN are geographically distributed for local services. Furthermore, Device-to-Device (D2D) and Heterogeneous Networks (HetNet) are essential technologies in 5G systems. They can greatly improve the spectrum efficiency and support the large-scale live video streaming in short-distance communications. We exploit the matrix game theoretical approach to operate the cloudlet resource management and allocation. Nash equilibrium solution can be obtained by Karush-Kuhn-Tucker nonlinear complemen- tarity approach. Illustrative results indicate that the proposed resource sharing scheme with the geo-distributed cloudlets can improve the resource utilization and reduce the system power consumption. Moreover, with the integration of the Software- define network (SDN) architecture, a vehicular network can easily reach a globally optimal solution. Index Terms—5G, vehicular network, C-RAN, software-define network, resource management, matrix game I. I NTRODUCTION Vehicular networks are facing challenges in both communi- cations capacity and computing capability when mobile traffic is explosively increased. As electric vehicle (EV) becomes popular, an explosion of mobile traffic application follows with the popularity of the Internet of Things. Vehicular networks may have several limitations: limited spectrum resource, inca- pable on-board devices, and inefficient system management. In order to improve the performance of vehicular network for flexible resource allocation, automated network organization, and advanced mobility management, we enhance the structure of vehicular network by combining with 5G technologies, such as, the cloud radio access network (C-RAN) architecture, and cloud computing [1], which presents a novel 5G-enabled Copyright (c) 2015 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to [email protected]. R. Yu, J. Ding and X. Huang are with School of Automation, Guang- dong University of Technology, China (e-mail: [email protected]; jiefeid- [email protected]; huangxu [email protected]). M. Zhou is with Shenyang Institute of Automation Guangzhou Branch, Chinese Academy of Sciences, China. Email: [email protected] S. Gjessing and Y. Zhang are with Simula Research Laboratory and University of Oslo, Norway (e-mail: steing@ifi.uio.no, [email protected]). vehicular network. This new type of networks can efficiently satisfy the demand for vehicular services with heavy data traffic or complex computing process. Facing the rapid increase of mobile services, 5G wireless network has an enticing prospect in vehicular network to enable high data-rate applications in vehicle environment [2]. 5G employs the small cells technology which enables high spatial and frequency reuse in limited coverage area [3]. C-RAN is defined as a centralized RAN, which separates baseband units (BBUs) from radio access units, and migrates BBUs to the cloud to form a BBU pool for centralized pro- cessing [4]. This approach allows the deployment of network functions in cloud data center to leverage traffic load through virtualization techniques. However, radio signals exchange is limited by the capability of fiber links between remote radio heads (RRHs) and the data center. Hence, decentralized processing and hierarchical management are significant to improve the performance of in vehicular networks. Since data transmission is easily influenced by surrounding environment [5], Device-to-device (D2D) is an efficient approach for direct and short distance transmission between vehicles. Besides, D2D communications can further enhance the communications capacity by allowing nearby devices to establish local links [6]. Study in [7] discussed the technique of D2D frequency reuse to reduce communications latency. Users can simultaneously have multiple wireless signal sources and switch between them for smooth data transmission [8]. In terms of task processing, cloud computing and software- define network (SDN) architecture are significant to improve the performance and the flexibility of vehicular networks. The main purpose of cloud computing is to provide an approach for users to run computation-intensive applications which are not easily performed on a resource-constrained mobile device [9]. Geo-distributed cloud has been regarded as an promising implementation in vehicular networks for the sake of low communications delay [10], [11]. The data center, cloudlets and RRHs form a hierarchical network which can be managed by SDN architecture for high efficiency and better quality of service (QoS) [12]. The most significant advantage of SDN is the three layers vertical integration architecture, which breaks through the control logic from underlying devices [13], [14]. In case of fast moving vehicles, cost-efficient resource allocation and joint management are important for vehicle applications to provide location based services, e.g., navigation service, and emergency alerts [15]. Automated network organization is a new feature of 5G-

Upload: others

Post on 30-Mar-2020

10 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

1

Optimal Resource Sharing in 5G-enabled VehicularNetworks: A Matrix Game Approach

Rong Yu, Member, IEEE, Jiefei Ding, Student Member, IEEE, Xumin Huang, Student Member, IEEE,Ming-Tuo Zhou, Senior Member, IEEE, Stein Gjessing, Senior Member, IEEE, Yan Zhang, Senior Member, IEEE

Abstract—Vehicular networks are expected to accommodate alarge number of data-heavy mobile devices and multi-applicationservices. Whereas, it faces a significant challenge when we needto deal with the ever-increasing demand of mobile traffic. In thispaper, we present a new paradigm of 5G-enabled vehicular net-works to improve the network capacity and the system computingcapability. We extend the original cloud radio access network(C-RAN) to integrate local cloud services to provide a low-cost,scalable, self-organizing and effective solution. The new C-RANis named as Enhanced C-RAN (EC-RAN). Cloudlets in EC-RANare geographically distributed for local services. Furthermore,Device-to-Device (D2D) and Heterogeneous Networks (HetNet)are essential technologies in 5G systems. They can greatlyimprove the spectrum efficiency and support the large-scale livevideo streaming in short-distance communications. We exploitthe matrix game theoretical approach to operate the cloudletresource management and allocation. Nash equilibrium solutioncan be obtained by Karush-Kuhn-Tucker nonlinear complemen-tarity approach. Illustrative results indicate that the proposedresource sharing scheme with the geo-distributed cloudlets canimprove the resource utilization and reduce the system powerconsumption. Moreover, with the integration of the Software-define network (SDN) architecture, a vehicular network can easilyreach a globally optimal solution.

Index Terms—5G, vehicular network, C-RAN, software-definenetwork, resource management, matrix game

I. INTRODUCTION

Vehicular networks are facing challenges in both communi-cations capacity and computing capability when mobile trafficis explosively increased. As electric vehicle (EV) becomespopular, an explosion of mobile traffic application follows withthe popularity of the Internet of Things. Vehicular networksmay have several limitations: limited spectrum resource, inca-pable on-board devices, and inefficient system management.In order to improve the performance of vehicular network forflexible resource allocation, automated network organization,and advanced mobility management, we enhance the structureof vehicular network by combining with 5G technologies,such as, the cloud radio access network (C-RAN) architecture,and cloud computing [1], which presents a novel 5G-enabled

Copyright (c) 2015 IEEE. Personal use of this material is permitted.However, permission to use this material for any other purposes must beobtained from the IEEE by sending a request to [email protected].

R. Yu, J. Ding and X. Huang are with School of Automation, Guang-dong University of Technology, China (e-mail: [email protected]; [email protected]; huangxu [email protected]).

M. Zhou is with Shenyang Institute of Automation Guangzhou Branch,Chinese Academy of Sciences, China. Email: [email protected]

S. Gjessing and Y. Zhang are with Simula Research Laboratory andUniversity of Oslo, Norway (e-mail: [email protected], [email protected]).

vehicular network. This new type of networks can efficientlysatisfy the demand for vehicular services with heavy datatraffic or complex computing process.

Facing the rapid increase of mobile services, 5G wirelessnetwork has an enticing prospect in vehicular network toenable high data-rate applications in vehicle environment [2].5G employs the small cells technology which enables highspatial and frequency reuse in limited coverage area [3].C-RAN is defined as a centralized RAN, which separatesbaseband units (BBUs) from radio access units, and migratesBBUs to the cloud to form a BBU pool for centralized pro-cessing [4]. This approach allows the deployment of networkfunctions in cloud data center to leverage traffic load throughvirtualization techniques. However, radio signals exchangeis limited by the capability of fiber links between remoteradio heads (RRHs) and the data center. Hence, decentralizedprocessing and hierarchical management are significant toimprove the performance of in vehicular networks. Since datatransmission is easily influenced by surrounding environment[5], Device-to-device (D2D) is an efficient approach for directand short distance transmission between vehicles. Besides,D2D communications can further enhance the communicationscapacity by allowing nearby devices to establish local links [6].Study in [7] discussed the technique of D2D frequency reuseto reduce communications latency. Users can simultaneouslyhave multiple wireless signal sources and switch between themfor smooth data transmission [8].

In terms of task processing, cloud computing and software-define network (SDN) architecture are significant to improvethe performance and the flexibility of vehicular networks. Themain purpose of cloud computing is to provide an approachfor users to run computation-intensive applications which arenot easily performed on a resource-constrained mobile device[9]. Geo-distributed cloud has been regarded as an promisingimplementation in vehicular networks for the sake of lowcommunications delay [10], [11]. The data center, cloudletsand RRHs form a hierarchical network which can be managedby SDN architecture for high efficiency and better quality ofservice (QoS) [12]. The most significant advantage of SDN isthe three layers vertical integration architecture, which breaksthrough the control logic from underlying devices [13], [14]. Incase of fast moving vehicles, cost-efficient resource allocationand joint management are important for vehicle applications toprovide location based services, e.g., navigation service, andemergency alerts [15].

Automated network organization is a new feature of 5G-

Page 2: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

2

enabled vehicle networks. Resource sharing provides a newaspect for allocating resource on a demand-based approach[16], [17]. It avoids some costs for resource over-utilizationor resource redundance to improve the resource utilization.However, in a geo-distributed cloudlet network, the resourceallocation of a cloudlet will interact with nearby cloudlets. Amatrix game is a confrontation of n players (n ≥ 2) whose de-cision incudes multiple factors in a normal and noncooperativecontext [18]. In [19], authors adopted a matrix game to resolvethe payoff matrix of strategic firms in electricity market.Matrix game is an approach for multiple player games andcan be a potentially efficient method for resource allocation invehicular networks.

In this paper, we propose a new paradigm for vehicular net-works in 5G communications environment, EC-RAN, whichaims to support data-heavy applications, such as augmentedreality applications. Vehicle users can access services by on-board unit (OBU) or their portable devices. With the com-bination of C-RAN and cloud computing, EC-RAN not onlyimproves communications qualities by employing the mixed-network deployment of small-cell, D2D and HeterogeneousNetworks (HetNet), but also enhances vehicle computingcapabilities by offloading mobile services to cloud. In SDNframework, management, control and execution are separatedand supposed to be managed independently in three layers.Through inter-cooperation, the three layers can perform wellas a whole system. To improve system performance, weadopt graph theory to demonstrate the features of geographicdistribution and the relationship between cloudlets. We formu-late resource allocation problem as a non-cooperation matrixgame and solve it though a non-linear concave optimizationapproach. In addition, illustrative results demonstrate thatresource sharing among cloudlets significantly improves theperformance of 5G-enabled vehicular networks and reducessystem operation cost. The major contributions of this papercan be summarized as follows.

• We define a new paradigm of 5G-enabled vehicularnetworks. A new concept of enhanced C-RAN (CloudRadio Access Network) is proposed for communication-s enhancement and computing capability improvement.Geo-distributed RRH and D2D approach can efficientlyallocate radio resource and greatly improve the commu-nications capacity of moving vehicles.

• We formulate the resource allocation of each cloudlet asa non-cooperation matrix game. In the optimal solution,resource sharing aims to improve the resource utilization.Moreover, we obtain the Nash equilibrium by employingthe KKT (Karush-Kuhn-Tucker) condition into non-linearoptimization.

• We resolve resource allocation problem in the geo-distributed cloudlets to reduce system operation cost.Through exploiting SDN framework to manage the hi-erarchical vehicular network, we can obtain the globaloptimization results with high QoS and more revenuesbased on the information of network topology.

The rest of this paper is organized as follows. In Section II,we describe the architecture of 5G-enabled vehicular network,

and discuss EC-RAN architecture which includes cloudlet,D2D communications and SDN technology. System model andproblem formulation are presented in Section III. In SectionIV, we formulate a non-cooperation matrix game and solve itby KKT conditions approach. Illustration results are presentedand discussed in Section V. Section VI concludes the paper.

II. 5G-ENABLED VEHICULAR NETWORK

5G wireless communications technologies will revolutionizecurrent vehicular network to satisfy the continuously increas-ing demand for high data-rate and mobility. The proposed5G-enabled vehicular network has three important features:i) the enhanced C-RAN (EC-RAN) and D2D technology forresilient communications, ii) the geo-distributed cloudlets forapplication processing, iii) the SDN architecture for systemmanagement and control. The integrated framework enhancesvehicular network with the advantages of robust communica-tions network, powerful processing capabilities and flexibleresource allocation. This system framework is also motivatedby the purpose of energy saving and cost reduction. In Fig.1, our proposed paradigm of 5G-enabled vehicular network ispresented.

A. 5G-enabled Communications Approaches

In Fig.1, 5G communications enable vehicles to access thebase station and communicate with cloudlets. In the traditionalapproach, vehicle communications and data processing aremainly handled by OBUs. The 802.11b/g protocal has limitedtransmission which is often affected by vehicle density andvehicle average speed [20], [21]. 5G cellular networks canoffer superior performance in terms of throughput, delay,reliability, scalability and mobility. In 5G networks, the peakdata rate for low mobility is 10 Gb/s and peak data rate forhigh mobility is 1 Gb/s. It can support transmission latencyshorter than about 1 ms for moving vehicles and also servefor high-speed trains with speed from 350 to 500 km/h [22],[23]. In vehicular network, the 5G technologies of interest areEC-RAN and D2D communications.

1) EC-RAN: BBU constitutes a pool in each cloudlet inorder to easily share signalling, data and channel state infor-mation (CSI) for users in vehicular network. This approach cangreatly improve the radio resource utilization by aggregatingand centralized dispatching the BBU bandwidth. EC-RANimplements a soft and virtualized BBU pool and distributedRRHs at remote sites. All RRHs connect to a switcher/centraldevice which can flexibly schedule radio resource in BBUpool for one RRH or a set of RRHs. The vehicle can directlycommunicate with RRHs at road side or with nearby base s-tations through wireless communications. RRHs communicatewith BBU pool through wired communications for reliabilityand low latency, such as, optical fiber links. By this approach,BBU can assign bandwidth resource for each RRH based onuser demand. Through controlling the number of RRH, we canadjust the size of small cell and significantly reduce inter-cellinterference. Moreover, resource allocation can be realized ina simplified procedure, e.g., programming the software.

Page 3: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

3

BA

C

Navigation Map

Computing

modules

BBU pool

Base station

Vehicle

Region

Service content:

· Cloud resource

allocation

· Signal processing

Vehicular cloud

Driving

assistanceEmergency

RRH

Cloudlet A

Cloudlet B

IP core network

and

data center cloud

Enhanced C-RAN

Resource

sharing

Wired communication

Wireless communication

D2D communication

Internet

Internet

Router

Small-cell

Hetnet

Fig. 1: 5G-enabled vehicular network architecture

2) D2D: D2D enables the direct transmission betweendifferent vehicle devices. It combines with D2B (device tobase station) to form HetNet. Proximity-based D2D commu-nications are emerged as a promising technology for effectivesharing of resources (spectrum and computing resources, etc.)and for users who are spatially close to each other [24].In a mixed-network deployment scenario, D2D can createopportunities for spectrum reuse and reduce communicationslatency for long-distance transmission or traffic congestion. Onthe other hand, D2D communications can potentially promoteresource sharing among vehicles through devices relayingwhich resembles ad hoc communication. In Fig.1, multipleD2D links are established in device pairs, A− B, B− C andC−D. They form a D2D cell and share the same informationbased on geographic distribution. It is similar to social networkwhich is a promising approach based on social relations [25].The other case is that D2D can relay for traffic offloading.Vehicle A is much closer to the base station and can actas a relay for its nearby vehicle C. D2B and D2D form acombined-communications network which aims to improvedata transmission performance with minimal cost. Moreover,base station communicates with data center through WAN.

D2D transmission for inter-vehicle communications will beaffected by the number of surrounding vehicles and nearbyD2D cells [26]. 5G small cells enable high spatial and frequen-cy reuse in limited coverage area. As the density of 5G smallcells grows, the transmission power of scheduled base stationis decreased and the achieved network efficiency is increased.However, the inter-cell interference will be increased. D2Dcell is a tiny cell which coexist with 5G small cells. Tominimize inter-cell interference, a 5G small cell will accepta limited number of D2D cells. We assume that the D2Dtransmission probability of vehicle j is denoted by

Pj = 1− νnj , (1)

Here, ν is the probability of D2B communication (0 < ν < 1).nj is the number of vehicles in a D2D cell.

3) Data Transmission Through 5G Core Network: Datatransmission through 5G core network is greatly affectedby the link condition. Here, we use Quality of Link (QoL)to evaluate the link condition between different cloudlets.Because it is unsuitable to exploit a link that reaches themaximum link condition. We define QoL by the probabilityof a successful packet transmission in a specific small timeinterval. Apparently, a link with faster transmission rate andlower delay will have a higher QoL, and therefore, obtaina larger chance in resource sharing. We employ the softmaxmethod by using the Boltzmann distribution theory [27]. Theprobability that a link is selected as a backhaul during resourcesharing is given by

θi,k =e

qi,kτ∑n

j=1 eqi,jτ

. (2)

Here, qi,k denotes QoL between the cloudlet i and k. τ isa temperature parameter of Boltzmann distribution. A hightemperature will cause the selection of backhaul link to be allequally probable. A Low temperature will generate a greaterdifference in the selection probability for links that differ intheir QoLs.

B. Cloud computing in 5G-enabled Vehicular Network

Cloud computing in 5G-enabled vehicular network includesdata center and cloudlets. Long distance transmission willincrease communication latency, require more bandwidth re-source and create congestion of Internet traffic flow in WideArea Network [28], [29]. Compared with the centralized cloud,the distributed cloudlets are geographically close to vehiclesand can serve as subordinate unit of data center. In Fig. 1,cloudlets are distributed in three regions to provide elasticcomputing services. Cloud computing in 5G-enabled vehicularnetwork architecture can be divided into three main scenarios.

• At the end-user side, a vehicle can access to vehicularcloud through D2B or D2D. Service can be classified

Page 4: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

4

into different purposes. Users will pay for this service bythe amount of service time and feedback the evaluationof QoS. A vehicle can also be the source of message. Forexample, a vehicle can transmit map data and share publicinformation to nearby vehicles through D2D approach.Application requirements will be directly offloaded to thelocal cloudlets through wireless communication.

• At the edge device side, a cloudlet (or called roadsidecloud, RAN cloud) is a set of resource-rich computers andBBUs which are connected to the Internet and provideservice to nearby users. Cloudlet has three importanttasks. First, it provides direct service as a service providerand caches the required data as preparation for next relat-ed service. Second, it manages the communication withinmobile users or between base station and mobile users,such as, bridging D2D connection between two nearbyEVs, allocating transmission channel for BBU pool andconducting data sharing service. Third, it serves as amiddle layer by offloading some incapable applications tothe superior data center cloud, and then returning resultsto end-user.

• At the IP core side, data center cloud is a cluster ofphysic devices and software services. It can possess bothof radio resource and computing resource and make profitby renting resource to service providers for a long-termperiod. Each service provider has limited cloud resourcewhich can based on user demand, and makes profit fromproviding service by allocating Virtual Machine to run ap-plications. Cloud storage technology is another importantapproach to increase the resource utilization efficiency ofcloud computing [30]. Moreover, a centralized BBU poolis located in data center and responsible for local BBUpool management.

In this paper, resource sharing can be conducted betweenthe nearby cloudlets. For example, when a cloudlet has manyapplication requests to process in a busy time, it may findresource assistance from the nearby cloudlets as long asbandwidth is enough to support data transmission. Resourceassistance is similar to run an application in the local cloudlet.For radio resource sharing, we considered about channel coor-dination between vehicles [31]. For a large area management,a normative control platform plays an important role in systemdesign. SDN framework brings a prospect for geo-distributedcloud computing network.

C. SDN framework in EC-RAN

The basic idea of SDN framework in EC-RAN is to separatecontrol logic (i.e., the control plane) from many underlyingcloudlets and communication equipments (i.e., the data plane),as shown in Fig.2. The decision of resource allocation is madein the management plane (a control center) which includesservice providers, network manager and baseband manager.They work independently and provide service for applicationrequests in the control center. The control plane in secondlayer is responsible for data forwarding strategies and resourcesharing strategies. The significant feature of the control planeis the virtualization of small cells in the data plane. A small

Management plane

Control plane

Data plane

· CPU resource

· Memory resource

· Bandwidth resource

CloudletCloudlet Cloudlet

The quality of

the link

The quality of

the link

Physical linkPhysical link

Small cell

Remote Radio

Head(RRH)

Base

station

Local Baseband

Unit (BBU) poolComputing

modules

Data transmission link

Control singnal link

VirtualizationVirtualization

Service Provider 1 Service Provider 2 Network manager Baseband manager

(Control center)

Fig. 2: SDN framework in EC-RAN

cell can be regarded as a node in EC-RAN network. Cloudletsare located in different small cells. The operation state ofcloudlet can be digitalized as three variables: CPU resource,memory resource and bandwidth resource. The link statebetween two connected nodes can be evaluated as the QoL.Algorithm design and control decision are made based on theinformation from the control plane. At the bottom layer, acloudlet and RRH in a base station provide a local serviceto users. Cloudlets are controlled by the manager from datacenter in the management plane and performs the controldecision. For example, BBU pool in cloudlet is responsiblefor baseband processing to reduce the interference betweendifferent RRHs or D2D and maximize energy efficiency.Baseband manager in the management plane is responsiblefor the RRH on/off management to minimize the numberof activated RRHs. The difference between BBU pool andbaseband manager is that they are working in different layers.BBU pool processes the baseband resource in the physicallayer. Baseband manager controls the bandwidth resource inthe application layer based on the virtualization of EC-RAN inthe control plane. With the management from SDN framework,5G-enabled vehicular network can make the best of the idleresource in data centers and achieve complex but efficientnetwork management functionalities.

III. PROBLEM FORMULATION

A. Network Model

We consider M geographically distributed cloudlets indifferent regions (say, one cloudlet for one region). Cloudletscan form a graph g = {V,E,w} (i = M) which is employedto describe the resource distribution and relationship of SPs.The set V represents the collection of a cloudlet Li (i =1, 2, · · · ,M ), and weight w on them denotes the link conditionbetween two regions. The network topology depends on thegeographical location of each region, e.g., the connectionbetween L1 and L2 means that these two regions are adjacent.The set E is the collection of connected edges. For example,we have eL1,L2 = 1 (eL1,L2 ∈ E). In this paper, the meaningof connected edge is more than geography connection. It alsorepresents the opportunity for cloudlet resource sharing. The

Page 5: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

5

Data

Center

Fig. 3: Cloudlet resource sharing in 5G-enabled vehicularnetwork

cloudlet resource sharing is to balance the resource utilizationin the following two scenarios.

• Resource Shortage If L1 operates with a high degreeof resource utilization, it will have few resource for newcoming applications. This situation often happens whenmany vehicle users request for service simultaneously.The traditional approach is to increase the number ofphysical devices with the condition of rasing the cost forbetter service. The disadvantage is that L1 will operateon a low degree of resource utilization in most of time.Resource sharing approach can resolve this problem in aflexible way. For example, L1 is allowed to rent resourcefrom L2, L3 and L4, as shown in Fig. 3. In other words,users are allowed to access to the nearby cloudlets andrun applications on cloudlet devices as long as the linkcondition can support data transmission. It is noted that,L1 can rent resource from more than one cloudlet.

• Ultra-dense Deployment Ultra-dense deployment is animportant feature of cloudlet. To improve energy effi-ciency, a cloudlet can turn off its power automaticallywhen it works on a low utilization and can be replacedby the nearby cloudlets. For example, at midnight mostpeople are in deep sleep. Cloud manager will evaluate theutilization of each cloudlet and turn off some cloudlets,such as L1. User in L1 will access to the nearby cloudletsfor service, such as that in L2.

Inter-cloud resource sharing is very important for cloudletsto improve profit and reduce the system power consumption.The control result includes the amount of allocating resourceand the on/off state of cloudlets. Moreover, resource sharingstrategy should satisfy the capability of each cloudlet. For acloudlet Li, the maximum capacity can be evaluated by: theCPU resource maxCi, the memory resource maxMi and thebandwidth resource maxBi.

The resource requests from user j can be denoted by Rj =(Cj ,Mj , Bj , Tj). Requests include the information of theamount of CPU resource Cj , the amount of memory resource

Mj , the required bandwidth Bj , and the maximum latencytime Tj . Every application has a specific maximum latency Tj

which should not be exceeded to provide a satisfactory QoS toa user. Therefore, link condition will be an important factorsto determine the success of resource sharing. To evaluate theresource utilization of cloudlet, we normalize the three types ofresources in a uniformed value, and make the value of resourceutilization no more than 1.

yj = cCj

maxCi+ a

Mj

maxMi+ b

Bj

maxBi, (3)

where, yj is the weighted sum of resource that a cloudlet Li

allocates to user j. c, a, b are the fixed coefficients of threekinds of resources (CPU resource, memory resource, band-width resource) and satisfy the relationship of c+ a+ b = 1.

B. Utility Function

The stimulating approach in 5G-enabled vehicular networkincludes revenue and operation utility. User will pay forservice by the amount of service time. If a cloud operatesmore applications, it can receive more revenues. However,unoccupied resource will also consume energy. Thus, thepurpose of resource management of cloud is trying to improvethe resource utilization of each cloudlet and decrease thenumber of idle or low utilization cloudlets. The utility functionof 5G-enabled vehicular network is given by

U = R

M∑i=1

yi + u

M∑i=1

log(yi + 1), (4)

where, yi is the utilization of cloudlet i, and 0 ≤ yi ≤ 1.R is the unit of revenue that user pay for service. u is thefixed coefficient of operation utility. The operation utility isa concave and increases monotonically. This indicates thatoperating in a low resource utilization is bad for utility. Fromutility function, we observe that the maximum value of utilitywill appear when resource utilization reach a good balancebetween cloudlets. The balance will be obtained throughresource sharing approach.

C. Cloudlet Resource sharing

Cloudlet resource sharing is based on the conditions ofsatisfying link quality and cloud resource redundance. Forexample, a cloudlet will prefer to cooperate with the cloudletswith more unoccupied resource than itself. We employ apenalty factor (yi)

m(m ≥ 2) which is an upward curve. Thepenalty is intended to avoid cloudlet resource over-utilization.This means that penalty will be increased as yi increasing. Asa result, Li will try to decrease utilization through resourcesharing with the nearby cloudlets. Thus, 1 − (yi)

m denotesthe willingness of cloudlet to accept new coming applications.Resource sharing aims to balance the utilization of cloudlets.Thus, the allocation function of each cloudlet is related to the

Page 6: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

6

willingness of all related cloudlets.

F (xi) = (1− (xi,i + ρi)m)+

χi,k

M∑k=1,k =i

(1− (xi,k + ρk)m),

χi,k = τkδ2

(θi,k+δ1),

yi = xi,i + ρi, yk = xi,k + ρk

τk =

{1,0,

ifif

turnturn

on,off.

(5)

where, xi,i is the amount of resource to be allocated on Li

for new coming applications. xi,k is the amount of resourcerenting from Lk. ρi is the amount of resource which has beenoccupied on a cloudlet Li. The occupied resource includes theshared resource with other cloudlets in the former optimizationresult (e.g., xk,i). Therefore, the resource utilization of Li, yi,is the sum of xi,i and ρi. τk is the decision variable (either 0 or1) of cloudlet state. θi,k is the link choice probability. δ1 andδ2 are the parameters which determine the influence from θi,k.χi,k denotes link condition. The value of xi,k should satisfythe constrain of total required resource πi. Then, we have

s.t.M∑k=1

xi,k = πi,

xi,k ≥ 0(k = 1, 2, · · · ,M).

(6)

To a single cloudlet, the solution of (5) is a non-linearoptimization problem. However, the result of all cloudletsis a matrix X = (xT

1 , xT2 , ..., x

TM )T ∈ RM , in which

x1 = (x1,1, x1,2, ..., x1,M ) is the optimal solution of a cloudletL1. The result of one cloudlet may have an effect on otherconnected cloudlets for resource sharing. For example, ifx1,2 > 0, this means that a cloudlet L1 rents resource from L2.Therefore, x1,2 is the amount of trading resource between twocloudlets. Besides, before allocating resource, we first evaluatethe occupied resource of Li by

ρi = φi +M∑

k=1,k =1

xi,k. (7)

Here, φi is the resource utilization of Li in the last time slot.We try to maximize the utility of each cloudlet based on theallocation function, since the allocation function has the sametendency as the utility of each cloudlet. This means that, if wemaximize the value of allocation function, we obtain the bestutility of cloudlet. The strategies of other cloudlets are denotedby xi = (x1, ..., xi−1, xi+1, ..., xM ). The optimal function ofa cloudlet i can be denoted by

maxxi

F (xi, xi) = (1− (xi,i + ρi)m)+

χi,k

M∑k=1,k =i

(1− (xi,k + ρk)m),

s.t.M∑k=1

xi,k = πi,

xi,k ≥ 0(k = 1, 2, · · · ,M).

(8)

IV. SOLUTION

The optimal function of all cloudlets are too complicated tobe resolved as they are tightly coupled by network topology.Equation (8) leads us to a local optimal solution of one

cloudlet, in which we find that the utilization of cloudletsare closely related with each other, and can be balanced byresource sharing. Therefore, the globally optimal solution isbased on the SDN architecture to maximize the allocationfunction of the entire cloud system. The main work of thissection is to linearize the optimal function and get the explicitform of solution.

There are two main difficulties for this problem. First, thesolution of (8) is a set of value, x1 = (x1,1, x1,2, ..., x1,M ).It includes the resource sharing decision from other cloudlets,such as (x1,2, ..., x1,M ), and the resource allocation decisionon its own devices, x1,1. The traditional concave optimizationapproach is to construct the entropy and search the optimalsolution in a feasible zone. However, this approach is time-consuming when there are many cloudlets in network. Weemploy a simplified linear approach: Karush-Kuhn-Tucker(KKT) condition, to find the optimal solution. Second, Thisproblem has special properties. All cloudlets in EC-RANtry to balance the utilization of network. Thus, the globaloptimization can be obtained when cloudlets achieve localoptimal solution. This means that the globally optimal functionequals the sum of all local optimal function. The study in [32]proposes a tree topology approach for load balance problem inhomogeneous network. However, the operation time of a treeformation will grow as the number of nodes increases, whilethe accuracy of global solution will decrease. To address theproblem, we propose a non-cooperation matrix game to obtainthe global optimization which is to motivate each cloudlet tooptimize resource management.

A. Matrix game

We consider a non-cooperative matrix game with M play-ers. For each player i, it obtains the optimal solution xi basedon other player’s strategies xi. Thus, we regard this approachas same as solving a generalized Nash equilibrium problem. Inorder to find the Nash equilibrium solution, we denote X∗ asthe generalised Nash equilibrium, X∗ = {x∗

1, x∗2, ..., x

∗n} and

x∗1 = {x∗

1,1, x∗1,2, ..., x

∗1,n}. According to the KKT conditions,

the Nash equilibrium solution is

maxX

F (X∗) =M∑i=1

((1− (x∗i,i + ρi)

m)+

M∑k=1

χi,k(1− (x∗i,k + ρk)

m))

s.t. g(xi) =M∑k=1

x∗i,k − πk = 0,

G(xi) = x∗i,k − υk ≥ 0.

(9)

In [33], the authors employ the Karush-Kuhn-Tucker (KKT)condition to transform the problem of generalised Nash equi-librium into nonlinear complementarity. To get the Nash equi-librium solution, the resource sharing function needs to satisfyone condition: to all cloudlets solution xi(i = 1, ...,M),F : RM 7→ R are concave functions.

∂F (X)∂xi,k

= −χi,k(xi,k + ρk)m−1,

∂2F (X)∂xi,k

2 = −2χi,k (if m = 2).(10)

Page 7: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

7

Therefore, F (X) is concave in each xi,k. If xi is a normalizedstationary point of the optimal problem, there exist multipliersλi, βi such that for each i = 1, 2, ...,M , we have

−∇xiF (xi, xi) +∇xig(xi)λi +∇xiG(xi)βi = 0,λi > 0⊥g(xi)λi = 0,βi ≥ 0⊥G(xi)βi = 0.

(11)

However, when the number of cloudlets are larger than two,equation (11) is very complicated to solve and get explicitexpression. We notice that Nash equilibrium solution will beobtained when we solve (8) to each cloudlet. Therefore, wefirst simplify the optimal function (8). Then, we calculate thelocal optimal solution of each cloudlet independently. Afteriteration for several rounds, the local optimal solution willgradually converge to the globally optimal solution which isthe Nash equilibrium. We refer to the following steps to obtainthe global solution.

• Step 1: Simplify the optimal function (8) of eachcloudlet. Then, calculate the related parameter ρni for eachcloudlet according to (7), where n is the index of rounds.Go to step 2.

• Step 2: Calculate the local optimal solution of eachcloudlet. Go to step 3.

• Step 3: Compare solution Xn−1 with current solutionXn. If |Xn − Xn−1| < φ, Xn is the final solution.Otherwise, back to step 1. φ is a positive value.

B. Simplification

The principle of simplification is based on the idea of ex-cluding the cloudlets which are inappropriate to rent resource.First, cloudlets without edge connection is not capable forresource sharing. Take L1 for example. L1 and L6 have noconnected edge, e(L1,L6) = 0. Thus, x1,6 = 0. Then, wenotice that a cloudlet Li only sends resource sharing request tothe connected cloudlets whose unoccupied resource are richerthan that of itself. If a cloudlet L1 rents resource from L2, theresource allocation result includes occupied resource x1,1 andrenting resource x1,2.

x1,1 =χ1,2(ρ2+π1)−ρ1

1+χ1,2

x1,2 =

{0,

1− χi,k(ρ2+π1)−ρ1

1+χ1,2,ifif

χ1,2ρ2 − ρ1 ≥ π1,χ1,2ρ2 − ρ1 < π1.

(12)

The result shows that, L1 may rent resource from L2 if thecondition of χ1,2ρ2 − ρ1 < π1 is satisfied. This condition isalso correct in multiple cloudlets resource sharing. Therefore,the resource sharing between two cloudlets is based on twoconditions: they are directly connected e(Li,Lk) = 1, and theamount of unoccupied resource is enough for resource sharingχi,kρk − ρi < πi. Otherwise, the resource sharing request willbe refused xi,j = 0.

C. Concave optimization in each cloudlet

We employ a non-linear concave optimization approachbased on KKT conditions to solve the local optimal solution

of each cloudlet. From [34], we can work out explicit solutionfor any matrix dimension M . Equation (5) can be rewritten as

L(xi, λi, βi, vi) = −F (xi) + λi(n∑

k=1

xi,k − πi)+

n∑k=1

βk(xi,k − vk2).

(13)

Here, λi, βk, vk are related parameters in Lagrange decou-pling. n is the number of cloudlets which satisfying conditionχi,kρk − ρi < πi. Then, we have a set of equations

∂L∂xi

= −∂F(xi)

∂xi +∂λi

n∑k=1

xi,k

∂xi +∂

n∑k=1

βk(xi,k−vk2)

∂xi = 0,(14)

n∑k=1

xi,k − πi = 0, (15)

xi,k ≥ 0, (16)βkxi,k = 0. (17)

The reformulation of this problem is a standard mathe-matical procedure and relatively easy to solve. The solutionof cloudlet Li will fulfill limitations from (14) to (17), nomatter how many cooperating cloudlets there are. In this paper,we only elaborate the progress of resolving the case that Li

has two cooperating cloudlets. Take n = 2 for example. Wecombine (14) and (15), then get the results

λi=−2χi,1χi,2∆+ χi,1χi,2β1 + χi,2β2 + χi,1β3

χi,1χi,2 + χi,1 + χi,2, (18)

xi,1=2χi,1χi,2∆+χi,2(β2−β1)+χi,1(β3−β1)

2(χi,1χi,2+χi,1+χi,2)−ρ1, (19)

xi,2 =2χi,2∆+ χi,2(β1 − β2) + (β3 − β2)

2(χi,1χi,2 + χi,1 + χi,2)− ρ2, (20)

xi,3 =2χi,1∆+ χi,1(β1 − β3) + (β2 − β3)

2(χi,1χi,2 + χi,1 + χi,2)− ρ3, (21)

∆ = ρ1 + ρ2 + ρ3 + πi. (22)

Here, ∆ is the sum of utilization. Then we combine (18) with(19), (20), (21) and (17), and get three equations about βi.

(2χi,1χi,2∆+χi,2(β2−β1)+χi,1(β3−β1)

2(χi,1χi,2+χi,1+χi,2)− ρ1)β1 = 0, (23)

(2χi,2∆+ χi,2(β1 − β2) + (β3 − β2)

2(χi,1χi,2 + χi,1 + χi,2)− ρ2)β2 = 0, (24)

(2χi,1∆+ χi,1(β1 − β3) + (β2 − β3)

2(χi,1χi,2 + χi,1 + χi,2)− ρ3)β3 = 0. (25)

Therefore, through calculating βi, we can get a solution of xi.According to (16), there is no less than one βi that equals tozero. Therefore, conditions can be classified into 7 cases as23 − 1 = 7. If all β′

is equal to zeros, the result of xi can bepresented as

xi,1 =2χi,1χi,2∆

2(χi,1χi,2+χi,1+χi,2)− ρ1,

xi,2 =2χi,2∆

2(χi,1χi,2+χi,1+χi,2)− ρ2,

xi,3 =2χi,1∆

2(χi,1χi,2+χi,1+χi,2)− ρ3.

(26)

If only β1 equals to zero β1 = 0, this means that onlyx1 is larger than zero. Therefore, we have the set of results

Page 8: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

8

xi = [πi, 0, 0]. If only β1 is larger than zero β1 > 0, thismeans that only x1 is equal to zero. We have

−(χi,1 + χi,2)β1 = 2ρ1(χi,1χi,2 + χi,1 + χi,2)−2χi,1χi,2∆.

(27)

And the result of xi can be presented by

xi,1 = 0,

xi,2 =χi,2∆(2χi,1+χi,2)

(χi,1χi,2+χi,1+χi,2)(χi,1+χi,2)

− χi,2

χi,1+χi,2ρ1 − ρ2,

xi,3 =χi,1∆(1+χi,1χi,2)

(χi,1χi,2+χi,1+χi,2)(χi,1+χi,2)

− χi,1

χi,1+χi,2ρ1 − ρ3.

(28)

The rest 4 cases can be resolved similarly. Thus, we narrowdown the searching area for solution xi to 8 points. Accordingto (16), we can further remove some points out of the feasiblezone of solution. Then, we go back to (8) and find the finalsolution at the point with the maximum value of F (xi, xi).The Kuhn-Tucker conditions is able to find the final solutionof more than 2 cooperated cloudlets.

Theorem 1 : When we maximize the resource sharingfunction, we will obtain the best network utility.

Proof: According to (4), the utility function is concavein ∂U

∂yi= R + u

yi+1 and ∂U2

∂2yi= − u

yi+12 < 0. According to

(10), resource sharing function is also concave. The resourcesharing result is to balance the utilization between the differentcloudlets. We assume a two-cloudlet network, whose utiliza-tion is denoted by y1 and y2 before resource sharing (y1 > y2).By maximizing (9), the optimal solution y

1 = y′

2 = y1+y2

2 isto balance the utility between them. Furthermore, the utilityis compared between before and after resource sharing.

U − U′=u(log(y1 + 1)+log(y1 + 1)−2log(y

1+1)),

= u(log( (y1+1)(y2+1)

(y′1+1)2

)),

< u(log( 2(y

′1+1)

)) < 0.

(29)

The utility before resource sharing is less than that afterresource sharing. Therefore, when we maximize the resourcesharing function, the utility of cloudlet will be maximized.

V. NUMERICAL RESULTS

A. Simulation Setup

In the simulation, the system model is supported by the trav-el data from the household travel data dictionary of southeastFlorida region [35].There are three countries and 18 regionsin the southeast Florida, Broward, Dade and Palm Beach. Weselect 9 adjacent regions with a high traffic rate. The 9 adjacentregions are shown in Fig.4. We assume that each cloudletis located in one region and responsible for local vehicularservice. The connected lines between each region are the edgesof network and indicate geographical proximity. Cloudletswith connection can form a resource sharing group. Theremote data center is responsible for managing the operationof all cloudlets. The simulation is conducted to last for 24hours.

We assume that the capability of all cloudlets are the same,and the cloud resource are shared by service providers equally.We consider two kinds of vehicular applications: service for

1 2

3 4

5 6

7 8

9

Cloudlet 1 Cloudlet 2

Cloudlet 3Cloudlet 4

Cloudlet 5 Cloudlet 6

Cloudlet 7 Cloudlet 8

Cloudlet 9

Fig. 4: An example of distributed cloudlets in the southeastFlorida

TABLE I: Resource requirement of AR applications

ApplicationsCPU Memory Bandwidth

resource resource resource

For navigation withoutD2D assist

5 5 3

For online video withoutD2D assist

4 4 4

For video and navigationwithout D2D assist

7 7 5

For navigation withoutD2D assist

5 4 2

For video and navigationwith D2D assist

7 6 3

navigation, service for online video. Since a vehicle may havemany passengers, the two kinds of vehicular applications canbe conducted at the same time. The required resources arein Table I, in which resources value are transformed into theunits of each kind of resource (e.g., one unit of CPU means4000MIPS, one unit of memory means 4000MB). We adoptthis approach to simplify the calculation as that in [36], [37].The ratios of three kinds of resource are evaluated by the cost,c = 0.5, a = 0.4, b = 0.1. Other parameters setting can befound in Table II.

B. Convergence of the proposed approach

The optimal solution of each cloudlet will be obtainedby maximizing (9). In the optimal progress, each cloudletwill interact with each other according to network topology.Following the proposed approach, the global solution willbe obtained through multiple iterations. Fig.5 shows us theprocess in obtaining the global solution. Error E means theabsolute difference between current solution and previous

solution, E =M∑k=1

|xni,k − xn−1

i,k |. After several iterations, such

as point n = 15 in Fig.5, the local solution will converge to astable state of the global optimal solution. This means that nocloudlet will adjust its resource allocation strategy to obtainan higher utility.

Page 9: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

9

TABLE II: Parameters setting

maxCi 800 a 0.5 τ 1.5

maxMi 800 b 0.4 m 2

maxBi 480 c 0.1 M 9

R 5 δ1 1 ν 0.7

u 3 δ2 0.5 φ 0.0001

n5 10 15 20 25 30 35 40 45 50

erro

r

×10 -5

0

1

2

3

4

5

6

Fig. 5: The convergence of KKT approach

C. Performance Evaluation of D2D communications

D2D approach is an attractive characteristic of 5G networks.Vehicles can employ D2D to transmit data rather than D2B.Therefore, frequency reuse will enable a cloudlet to allocateless D2B bandwidth resource by using D2D bandwidth. More-over, with less data transmission to mobile users, cloudlets cansave memory resource, such as, ROM (Read Only Memory)for task running. Fig.6 compares the total number of droppedapplications on each cloudlet. In this model, we assume thatuser will continue to send application request if they arerefused. Therefore, one application may be dropped severaltimes if a cloudlet has no enough resource to provide service.From Fig. 6, we notice that the number of dropped applicationsin D2D assisted network is less than that without D2D. Inthe case without D2D, each application will consume moreresource. A cloudlet will refuse and drop user applications ifit has not enough resource to provide service.

D. Performance Evaluation of cloudlet resource sharing

Cloudlet resource sharing is an approach to improve re-source shortage. It performs well in two situations: resourceshortage and cloudlet hibernation. In the case of resourceshortage, a cloudlet can enhance its capability by renting re-source from nearby cloudlets. Through offloading applicationon the cooperated cloudlet, the number of mobile users inservice is increasing. A cloudlet can be turned off in a stateof hibernation for energy saving. If its amount of occupiedresource is very low and it can be replaced by the nearbycloudlets. However, in the case without resource sharing, acloudlet will keep operating, even there is no user. Fig.7compares the resource utilization of each cloudlet at 8:00.

Cloudlet1 2 3 4 5 6 7 8 9

Num

ber

0

500

1000

1500

With D2D assist Without D2D assist

Fig. 6: The total number of dropped applications in 24h

Cloudlet1 2 3 4 5 6 7 8 9

Ave

rage

Res

ourc

es U

tiliz

atio

n (%

)

15

20

25

30

35

40

With resource sharing Without resource sharing

Fig. 7: The resource utilization of cloudlets at 8:00

The blue bars of cloudlets 1, 2, 4, 7, 9 are lower than yellowbars. The rest of blue bars are higher than yellow bar. Thetotal resource utilization of all cloudlets are the same. Theresults indicate that a cloudlet with a large amount of occupiedresource can reduce its resource utilization through resourcesharing, and meanwhile, improve the resource utilization ofthe cloudlet with a low occupied resource. Therefore, theresource utilization between different cloudlets will becomemore balanced. The utility of the two approaches (with andwithout resource sharing) are 5.65866 and 5.65882 respective-ly. This further illustrates that the proposed allocation strategyis optimal to network utility.

The average resource utilization of each cloudlet in a wholeday is shown in Fig.8. The utilization of blue bars are higherthan the yellow ones. It is because the cloudlet which has lessapplication requirements can improve its resource utilizationthrough resource sharing approach. The cloudlet which hasabundant resource can process more application requests byresource sharing. The number of application requirements isaffected by the number of mobile users. The result turns outthat the total utilization is significantly enhanced.

Page 10: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

10

Cloudlet1 2 3 4 5 6 7 8 9

Ave

rage

Res

ourc

es U

tiliz

atio

n (%

)

0

10

20

30

40

50

60

70

80

With resource sharing Without resource sharing

Fig. 8: The average resource utilization in a whole day

Cloudlet1 2 3 4 5 6 7 8 9

Ave

rage

Res

ourc

es U

tiliz

atio

n (%

)

0

10

20

30

40

50

60

70

80

Global optimalLocal optimal

Fig. 9: The result of average resource utilization at 9:00 inglobal optimal approach and local optimal approach

E. Performance Evaluation of the SDN Architecture

The 5G-enabled vehicular network with SDN architecturehas many benefits. One of them is that with an appropriateseparation between control and management, cloudlets arecontrolled and managed by the control center in the man-agement plane. This allows the cloud manager to control thenetwork conveniently. Without SDN architecture, a cloudletonly has the knowledge about nearby cloudlets, and themanager has to make the decision based on local information.Fig. 9 shows the result of average resource utilization at 9:00and compares the results of two approaches: with and withoutSDN architecture (global and local optimality).

The balance of resource utilization is important to show theadvantage of SDN architecture and can be evaluated accordingto the heterogeneity index. The heterogeneity index in thispaper is based on the Lorenz curve and a Gini coefficientwhich describes income inequality in microeconomics [38].Here, we employ the heterogeneity index H to evaluate the

Cloudlet1 2 3 4 5 6 7 8 9

Ope

ratin

g H

ours

0

5

10

15

20

25

Fig. 10: The total operation time of 9 cloudlets

unbalance degree of resource utilization of cloudlets.

H =

N∑i=1

N∑j=1

|yi − yj |

2N2y, (30)

y =1

N

N∑i=1

yi. (31)

Here, y is the average value of utilization of all cloudlets.N is the number of cloudlets. The heterogeneity index ofcloudlet utilization with and without SDN architecture are0.1328 and 0.1765 respectively. This shows that the globallyoptimal solution are slightly better than the local solution ofresource balance. Since the balanced result is better for utility,we come to the conclusion that with SDN architecture, the 5G-enabled vehicular network can reach the global optimality.

F. Performance Evaluation of Energy Cost

Fig. 10 shows the total operation time of 9 cloudlets inthe resource sharing approach. Let us take the cloudlet L1 forexample. It has 12 hours to turn on for work and 12 hoursto turn off for hibernation. The power-off duration is goodfor the life time of cloudlet and also reduces the operationenergy. If without resource sharing, the cloudlets need to keepon working. By combining with the result from Fig. 7, we havethe conclusion that the 5G-enabled vehicular network increasesthe utilization of resource with lower power consumption.

VI. CONCLUSION

In this paper, we propose a new paradigm of 5G-enabledvehicular network. The paradigm integrates the concept ofenhanced C-RAN, D2D communications and SDN technolo-gies. The main purpose is to provide efficient and elasticservices for mobile applications which requires large band-width resource and high computing capability. We discussthe cloudlet resource management approach which includesresource allocation and sharing. D2D approach is investigatedto increase the bandwidth resource reuse and relieve the

Page 11: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

11

communications congestion. We employ a matrix game tosolve the problem and use Karush-Kuhn-Tucker conditions towork out the explicit solutions of global optimization. Throughcloudlet resource sharing, the management approach of non-cooperation game can significantly improve the resource uti-lization and reduce the operation cost for the cloudlet devices.In future work, communications security is also importantin cloud computing [39], [40]. Especially for fast movingvehicles, the cooperation between small cells is an effectiveapproach to deal with network topology change problem, suchas, rapid authentication of mobile terminals or fast switchingbetween different wireless networks [41].

ACKNOWLEDGEMENT

The work is supported in part by programs of NSFC underGrant nos. 61422201, 61370159 and U1201253, U1301255,the Science and Technology Program of Guangdong Provinceunder Grant no. 2015B010129001, Special-Support Project ofGuangdong Province under grant no. 2014TQ01X100, HighEducation Excellent Young Teacher Program of GuangdongProvince under grant no. YQ2013057, and Science and Tech-nology Program of Guangzhou under grant no. 2014J2200097(Zhujiang New Star Program). This research is partially sup-ported by the projects 240079/F20 funded by the ResearchCouncil of Norway, and the European Commission FP7Project CROWN (grant no. PIRSES-GA-2013-627490).

REFERENCES

[1] P. Demestichas, A. Georgakopoulos, D. Karvounas, K. Tsagkaris, V. S-tavroulaki, J. Lu, C. Xiong, and J. Yao, “5g on the horizon: Keychallenges for the radio-access network,” IEEE Vehicular TechnologySociety, vol. 8, no. 3, pp. 47–53, 2013.

[2] J. G. Andrews, S. Buzzi, W. Choi, S. V. Hanly, A. Lozano, A. C. Soong,and J. C. Zhang, “What will 5g be?,” IEEE Journal on Selected Areasin Communications, vol. 32, no. 6, pp. 1065–1082, 2014.

[3] N. Bhushan, J. Li, D. Malladi, R. Gilmore, D. Brenner, A. Damnjanovic,R. T. Sukhavasi, C. Patel, and S. Geirhofer, “Network densification: Thedominant theme for wireless evolution into 5g,” IEEE CommunicationsSociety, vol. 52, no. 2, pp. 82–89, 2014.

[4] R. Wang, H. Hu, and X. Yang, “Potentials and challenges of c-ransupporting multi-rats toward 5g mobile networks,” IEEE on Access,pp. 1187 – 1195, 2014.

[5] J. Shen, H. Tan, J. Wang, J. Wang, and S. Lee, “A novel routing protocolproviding good transmission reliability in underwater sensor networks,”Journal of Internet Technology, vol. 16, no. 1, pp. 171–178, 2015.

[6] M. N. Tehrani, M. Uysal, and H. Yanikomeroglu, “Device-to-devicecommunication in 5g cellular networks: challenges, solutions, and futuredirections,” IEEE Communications Magazine, vol. 52, no. 5, pp. 86–92,2014.

[7] A.-H. Tsai, L.-C. Wang, J.-H. Huang, and T.-M. Lin, “Intelligentresource management for device-to-device (d2d) communications inheterogeneous networks,” in Proc. Wireless Personal Multimedia Com-munications (WPMC), pp. 75–79, 2012.

[8] S. Patel, M. Chauhan, and K. Kapadiya, “5g: future mobile technology-vision 2020,” International Journal of Computer Applications, vol. 54,no. 17, 2012.

[9] K. Kumar and Y.-H. Lu, “Cloud computing for mobile users: Canoffloading computation save energy?,” Computer, no. 4, pp. 51–56, 2010.

[10] J. Li, D. Li, J. Zheng, and Y. Quan, “Location-aware multi-user resourceallocation in distributed clouds,” in Advanced Computer Architecture,pp. 152–162, 2014.

[11] M. Quwaider and Y. Jararweh, “Cloudlet-based efficient data collectionin wireless body area networks,” Simulation Modelling Practice andTheory, vol. 50, no. 0, pp. 57–71, 2015.

[12] R. Kaewpuang, D. Niyato, P. Wang, and E. Hossain, “A framework forcooperative resource management in mobile cloud computing,” IEEEJournal on Selected Areas in Communications, vol. 31, no. 12, pp. 2685–2700, 2013.

[13] H. Kim and N. Feamster, “Improving network management with soft-ware defined networking,” IEEE Communications Magazine, vol. 51,pp. 114–119, February 2013.

[14] D. Kreutz, F. Ramos, P. Esteves Verissimo, C. Esteve Rothenberg,S. Azodolmolky, and S. Uhlig, “Software-defined networking: A com-prehensive survey,” Proceedings of the IEEE, vol. 103, no. 1, pp. 14–76,2015.

[15] A. Beloglazov and R. Buyya, “Optimal online deterministic algorithmsand adaptive heuristics for energy and performance efficient dynamicconsolidation of virtual machines in cloud data centers,” Concurrencyand Computation: Practice and Experience, vol. 24, no. 13, pp. 1397–1420, 2012.

[16] S. Barbarossa, S. Sardellitti, and P. Di Lorenzo, “Joint allocation ofcomputation and communication resources in multiuser mobile cloudcomputing,” in Proc. 2013 IEEE 14th Workshop on Signal ProcessingAdvances in Wireless Communications (SPAWC), pp. 26–30, 2013.

[17] D. Kusic, J. O. Kephart, J. E. Hanson, N. Kandasamy, and G. Jiang,“Power and performance management of virtualized computing envi-ronments via lookahead control,” Cluster computing, vol. 12, no. 1,pp. 1–15, 2009.

[18] S. Belhaiza, “On perfect nash equilibria of polymatrix games,” GameTheory, pp. 1–11, 2014.

[19] K. C. Sharma, R. Bhakar, and H. Tiwari, “Extreme nash equilibrium ofpolymatrix games in electricity market,” Recent Advances and Innova-tions in Engineering, pp. 1–5, 2014.

[20] Z. H. Mir and F. Filali, “Lte and ieee 802.11 p for vehicular networking:a performance evaluation,” EURASIP Journal on Wireless Communica-tions and Networking, vol. 2014, no. 1, pp. 1–15, 2014.

[21] X. Wu, S. Subramanian, R. Guha, R. G. White, J. Li, K. W. Lu,A. Bucceri, and T. Zhang, “Vehicular communications using dsrc:challenges, enhancements, and evolution,” IEEE Journal on SelectedAreas in Communications, vol. 31, no. 9, pp. 399–408, 2013.

[22] W. ChengXiang, H. Fourat, G. Xiqi, Y. XiaoHu, Y. Yang, Y. Dongfeng,A. Hadi, H. Harald, F. Sam, and H. Erol, “Cellular architecture and keytechnologies for 5g wireless communication networks,” IEEE Commu-nications Magazine, vol. 52, no. 2, pp. 122–130, 2014.

[23] M. Eiza and Q. Ni, “An evolving graph-based reliable routing schemefor vanets,” IEEE Transactions on Vehicular Technology, vol. 62, no. 4,pp. 1493–1504, 2013.

[24] L. Q. Clara, N. Huaning, P. A. Tolis, and W. Geng, “5g network capacity:key elements and technologies,” IEEE Vehicular Technology Magazine,vol. 9, no. 1, pp. 71–78, 2014.

[25] M. Tinghuai, Z. Jinjuan, T. Meili, T. Yuan, A.-D. Abdullah, A.-R.Mznah, and L. Sungyoung, “Social network and tag sources basedaugmenting collaborative recommender system,” IEICE transactions onInformation and Systems, vol. 98, no. 4, pp. 902–910, 2015.

[26] S. Caixing, L. Supeng, Z. Yan, A. Vinel, and M. Jonsson, “Performanceanalysis of connectivity probability and connectivity-aware mac proto-col design for platoon-based vanets,” IEEE Transactions on VehicularTechnology, vol. 64, no. 12, pp. 5596–5609, 2015.

[27] H. A. Tran, A. Mellouk, and S. Hoceini, “Qoe content distributionnetwork for cloud architecture,” in Proc. 2011 First InternationalSymposium on Network Cloud Computing and Applications (NCCA),pp. 14–19, 2011.

[28] D. T. Hoang, D. Niyato, and P. Wang, “Optimal admission controlpolicy for mobile cloud computing hotspot with cloudlet,” in Proc. 2012IEEE Wireless Communications and Networking Conference (WCNC),pp. 3145–3149, 2012.

[29] T. Soyata, R. Muraleedharan, C. Funai, M. Kwon, and W. Heinzelman,“Cloud-vision: Real-time face recognition using a mobile-cloudlet-cloudacceleration architecture,” in Proc. 2012 IEEE Symposium on Computersand Communications (ISCC), pp. 59–66, 2012.

[30] Y. Ren, S. jun, W. Jian, H. Jin, and S.-y. Lee, “Mutual verifiable provabledata auditing in public cloud storage,” Journal of Internet Technology,vol. 16, no. 2, pp. 317–323, 2015.

[31] Q. Wang, S. Leng, H. Fu, and Y. Zhang, “An ieee 802.11p-basedmultichannel mac scheme with channel coordination for vehicular adhoc networks,” IEEE Transactions on Intelligent Transportation Systems,vol. 13, no. 2, pp. 449–458, 2012.

[32] S. Xie and Y. Wang, “Construction of tree network with limited deliverylatency in homogeneous wireless sensor networks,” Wireless PersonalCommunications, vol. 78, no. 1, pp. 231–246, 2014.

[33] P. LI and G. LIN, “Solving a class of generalized nash equilibriumproblems,” Journal of Mathematical Research with Applications, vol. 33,no. 3, pp. 372–378, 2013.

Page 12: Optimal Resource Sharing in 5G-enabled Vehicular …folk.uio.no/yanzhang/TVT5G2016.pdfallocate radio resource and greatly improve the commu-nications capacity of moving vehicles. •

0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2536441, IEEETransactions on Vehicular Technology

12

[34] M. A. Hanson, “On sufficiency of the kuhn-tucker conditions,” Journalof Mathematical Analysis and Applications, vol. 80, no. 2, pp. 545–550,1981.

[35] Website: http://www.fsutmsonline.net/Floridatravelsurveys/FL travelsurveys.htm,2014.

[36] T. TruongHuu and C. Tham, “A game-theoretic model for dynamicpricing and competition among cloud providers,” in Processding 2013IEEE/ACM 6th International Conference on Utility and Cloud Comput-ing (UCC), pp. 235–238, 2013.

[37] H. Liang, L. X. Cai, D. Huang, X. Shen, and D. Peng, “An smdp-based service model for interdomain resource allocation in mobile cloudnetworks,” IEEE Transactions on Vehicular Technology, vol. 61, no. 5,pp. 2222–2232, 2012.

[38] H. HaiBo and W. XiaoFan, “Unified index to quantifying heterogeneityof complex networks,” Physica A: Statistical Mechanics and its Appli-cations, vol. 387, no. 14, pp. 3769–3780, 2008.

[39] Z. Xia, X. Wang, X. Sun, and Q. Wang, “A secure and dynamicmulti-keyword ranked search scheme over encrypted cloud data,” IEEETransactions on Parallel and Distributed Systems, vol. 27, no. 2,pp. 340–352, (DOI: 10.1109/TPDS.2015.2401003), 2016.

[40] F. Zhangjie, S. Xingming, L. Qi, Z. Lu, and S. Kim, Jiangang, “Achiev-ing efficient cloud search services: Multi-keyword ranked search overencrypted cloud data supporting parallel computing,” IEICE Transac-tions on Communications, vol. E98-B, no. 1, pp. 190–200, 2015.

[41] P. Guo, J. Wang, X. H. Geng, C. S. Kim, and J.-U. Kim, “A variablethreshold-value authentication architecture for wireless mesh networks,”Journal of Internet Technology, vol. 15, no. 6, pp. 929–935, 2014.

BIOGRAPHIES

Rong Yu [S05, M08] ([email protected]) receivedhis Ph.D. degree from Tsinghua University, China,in 2007. After that, he worked in the School ofElectronic and Information Engineering of SouthChina University of Technology (SCUT). In 2010,he joined the Institute of Intelligent InformationProcessing at Guangdong University of Technology(GDUT), where he is now a full professor. Hisresearch interest mainly focuses on wireless commu-nications and networking, including cognitive radio,wireless sensor networks, and home networking. He

is the co-inventor of over 10 patents and author or co-author of over 70international journal and conference papers. Dr. Yu is currently serving as thedeputy secretary general of the Internet of Things (IoT) Industry Alliance,Guangdong, China, and the deputy head of the IoT Engineering Center,Guangdong, China. He is the member of home networking standard committeein China, where he leads the standardization work of three standards.

Jiefei Ding ([email protected]) is currently pur-suing a Ph.D. degree in University of Manitoba,Winnipeg, Canada. She has received the M.S. de-gree from the Guangdong University of Technology,Guangdong Province, China. She spent five monthto study in Hong Kong University of Science andTechnology as a postgraduate visiting internshipstudent in 2015. Her research interests include cloudcomputing resource management, software-definednetwork (SDN), C-RAN in 5G networks.

Xumin Huang ([email protected]) is nowa master student of networked control systems inGuangdong University of Technology, China. Hisresearch interests mainly focus on network per-formance analysis, simulation and enhancement inwireless communications and networking.

Ming-Tuo Zhou ([email protected]) joinedShenyang Institute of Automation GuangzhouBranch, Chinese Academy of Sciences in July 2015.He was a senior research scientist at the SmartWireless Laboratory of National Institute of Infor-mation and Communications Technology (NICT)Singapore Representative Office during July 2004 toJune 2015. He was the Technical Co-Editor of IEEE802.16n and IEEE 802.16.1a, a voting member andtechnical contributor of the IEEE 802.11, 802.15,and 802.16 Working Groups. He received the Prize

of the Finnish Universities of Technology & Helsinki Consulting Group, andIEEE-SA Standards Board Acknowledges with Appreciation for contributionsto IEEE Std 802.16n, IEEE Std 802.16.1a, and IEEE Std 802.15.4m. Hiscurrent research interests include industrial wireless sensor networks andwireless smart utility networks. He is an IEEE senior member.

Stein Gjessing [M’91] ([email protected]) is anadjunct researcher at Simula Research Laboratoryand a professor of computer science in Departmentof Informatics, University of Oslo. He received hisDr. Philos. degree from the University of Oslo in1985. Gjessing acted as head of the Department ofInformatics for 4 years from 1987. From February1996 to October 2001, he was the chairman of thenational research program ”Distributed IT-System”founded by the Research Council of Norway. Gjess-ing participated in three European funded projects:

Macrame, Arches and Ascissa. His current research interests are routing,transport protocols and wireless networks, including cognitive radio and smartgrid applications.

Yan Zhang [SM’10] ([email protected]) is cur-rently Head of Department, Department of Networksat Simula Research Laboratory, Norway; and anAssociate Professor (part-time) at the Departmentof Informatics, University of Oslo, Norway. Hereceived a PhD degree in School of Electrical &Electronics Engineering, Nanyang Technological U-niversity, Singapore. He is an associate editor or onthe editorial board of a number of well-establishedscientific international journals, e.g., Wiley WirelessCommunications and Mobile Computing (WCMC).

He also serves as the guest editor for IEEE Transactions on Industrial Infor-matics, IEEE Communications Magazine, IEEE Wireless Communications,and IEEE Transactions on Dependable and Secure Computing. He serves aschair positions in a number of conferences, including IEEE PIMRC 2016,IEEE CCNC 2016, WICON 2016, IEEE SmartGridComm 2015, and IEEECloudCom 2015. He serves as TPC member for numerous internationalconference including IEEE INFOCOM, IEEE ICC, IEEE GLOBECOM, andIEEE WCNC. His current research interest include: wireless networks andreliable and secure cyber-physical systems (e.g., healthcare, transport, smartgrid). He has received 7 Best Paper Awards. He is a senior member of IEEE,IEEE ComSoc, and IEEE VT society.