clouds of virtual machines in edge networks

IEEE Communications Magazine • July 2013 630163-6804/13/$25.00 © 2013 IEEE

INTRODUCTION

The original Internet paradigm to reach a givenfinal destination focused on packet forwardingbased on IP addresses. This is no longer thecase, as in current IP networks packets are pro-cessed in intermediate nodes not only for look-ing up addresses, but also for performing anumber of additional functions, such as networkaddress translation, packet filtering, applicationacceleration over WANs, network monitoring,QoS management, and load balancing. Eachmiddle-box (closed and quite expensive) typicallysupports a limited set of special functions (layer4 or higher) and is predominantly built on dedi-cated hardware platforms.

Middle-boxes are deployed along most of thepaths from sources to destinations: that is whynetworks have lost the initial end-to-end charac-teristic of the Internet, where packets used to bejust forwarded (routed). Beyond this, middle-boxes have also represented a significant fractionof network capital and operational expenses,mostly due to network management complexi-ties.

The ossification of the Internet makes it diffi-

cult for operators to develop and deploy newnetwork functionalities, services, managementpolicies, and so on, which are essential to copewith the increasing complexity and dynamicity ofnetworks. Today, the launch of new servicesrequires lengthy and expensive processes, whichhinder the rapid take-off of new revenues in cur-rent dynamic markets. The innovation cycles ofoperators’ networks should be simplified byimproving network flexibility and adaptability tothe market dynamics.

Future networks should reduce operationalexpenditures (OPEX) and capital expenditures(CAPEX). For instance, automated managementand configuration of network equipment mayreduce the need for human intervention, thuslimiting the likelihood of wrong configurations;whereas flexible provisioning of network func-tionalities on top of an optimally shared physicalinfrastructure may reduce equipment costs andpostpone further network investments.

Improved performance of standard hardwareand emerging technologies such as softwaredefined networking (SDN) and network functionvirtualization (NFV) may help fulfill the aboverequirements.

This article argues that future network infras-tructures will be made of a huge number ofresources (compute, storage, and network I/O)being controlled dynamically, based on users’demands, quality of service (QoS) and businessobjectives, or any other changing condition.Data analytics systems and methods will allow acomprehensive autonomic loop to be exploitedthat is capable of orchestrating virtual functionsallocated to such a fabric of resources. Theedges of networks are where this innovationwave will take place, for several reasons; forexample, migration of intelligence toward theedges, pervasiveness of embedded communica-tions, computing and storage power in user’sdevices, and fewer legacies. In less than adecade, edge networks will create distributedenvironments made of clouds of virtual resources(even operated by diverse players) interconnect-ed by a simpler and less hierarchical core net-work. The core network will become stateless (asthe Internet basic protocol, where each packettravels entirely on its own without reference toany other packet), and edge networks (and data

ABSTRACT

This article addresses the potential impact ofemerging technologies and solutions, such assoftware defined networking and network func-tion virtualization, on carriers’ network evolu-tion. It is argued that standard hardwareadvances and these emerging paradigms canbring the most impactful disruption at the net-work’s edge, enabling the deployment of cloudsof nodes using standard hardware: it will be pos-sible to virtualize network and service functions,which are provided today by expensive middle-boxes, and move them to the edge, as close aspossible to users. Specifically, this article identi-fies some of key technical challenges behind thisvision, such as dynamic allocation, migration,and orchestration of ensembles of virtualmachines across wide areas of interconnectededge networks. This evolution of the networkwill profoundly affect the value chain: it will cre-ate new roles and business opportunities, reshap-ing the entire ICT world.

FUTURE CARRIER NETWORKS

Antonio Manzalini and Roberto Minerva, Telecom Italia

Franco Callegati, Walter Cerroni, and Aldo Campi, University of Bologna

Clouds of Virtual Machines in Edge Networks

MANZALINI LAYOUT_Layout 1 6/26/13 11:52 AM Page 63

IEEE Communications Magazine • July 201364

centers) will be the only stateful parts of the net-works. This transformation will enable new rolesand business opportunities, completely reshapingthe value chains in the entire telco-informationand communications technologies (ICT) world.

In essence, the key enablers of this evolutionare the advances in processing, storage, and net-working technologies that, in the short term, willallow the development of network nodes basedon standard off-the-shelf hardware, cheap butpowerful enough to run virtualized networkfunctions and services.

Edge networks will encompass a huge num-ber of inexpensive nodes and users’ devicescapable of collapsing the open systems intercon-nect (OSI layers, e.g., from L2 to L7) on stan-dard hardware solutions.

One of the main challenges behind this visionis the capability of dynamically instantiating,orchestrating, and relocating multiple virtualmachines (VMs) across the providers’ transportnetworks. Ensembles of VMs will be strictlyrelated and intertwined to implement sets of vir-tual functions and services that network opera-tors, and even users, must be able to configureand program.

The rest of this article is organized as follows.Trends and enabling technologies are described.Examples of network scenarios are discussed. Asurvey of related work is reported. We presentsome experimental results of a use case pointingout the technical challenges related to the livemigration of VMs across the WAN. We elabo-rate about the role of network operators in theenvisioned future scenarios and draw some con-clusions.

TRENDS ANDENABLING TECHNOLOGIES

Progress on the SDN paradigm has recentlysparked significant industrial interest in rethink-ing network architectures, control, and manage-ment. The SDN vision consists of decoupling thecontrol plane logic from the forwarding hard-

ware and moving the network states to a compo-nent, called a controller. This basic idea is notnovel; however, for the first time, processing,storage, and network throughput performancemay realistically support this disruption for carri-er grade services.

It should be noted that purpose-built hard-ware can still outperform general-purpose hard-ware, but the performance gap is becomingsmaller and smaller.

Virtualization of physical resources will alsohave a significant impact on network evolution.In the IT field, virtualization is already wellknown and widely deployed in data centers toexecute multiple isolated instances of a softwareentity on top of a single physical server. Virtual-ization has several benefits; for example, itincreases resource utilization and improves stateencapsulation. The extension of IT virtualizationprinciples to network equipment (e.g., routersand switches) offers several advantages in termsof optimal usage of physical resources and deep-er integration of IT and network resources [1].

Moreover, the implementation of networkprocessing functions in software, which is alreadypossible today, allows standard hardware to exe-cute them. As a consequence of this evolution,several tasks, or activities normally carried out indata centers, such as allocation, migration, andcloning of virtual resources and functions (forserver consolidation, load balancing, etc.) couldalso be performed in the network. This meansthat it should be possible to leverage theenhanced management tools used in data cen-ters today.

Network virtualization, for example, allowsoperators to collocate multiple instances of net-work functions in the same hardware, whereeach function is executed by one or multipleVMs, as illustrated in Fig. 1. As a result, net-work operators may dynamically instantiate, acti-vate, and re-allocate resources and functions, aswell as program them according to dynamicneeds, requirements, and policies.

It should be noted that NFV is complementaryto SDN and does not depend on it: the two con-cepts should not be mixed, even though they canbe combined in several ways that can potentiallycreate a great value.

To make such a combination feasible andexploitable, some technical problems remain tobe solved. The capability of moving and orches-trating sets of VMs across wide area connections(not just locally, as in data centers) is one ofthose. Available virtualization tools, currentlyused for intra-data-center applications, offeronly limited support for live migration of VMs inWANs, due to the lack of signaling and controltools spanning multiple technologies anddomains, and because of the strict constraints interms of throughput and delay. This is a point ofweakness on which future research and develop-ment activities should focus. Some experimentalresults of a use case addressing these issues andshowing its potential feasibility and limitationsare presented later in this article.

Furthermore, network operators should beable to cope with the increasing complexity ofnetwork management and control, exacerbatedby the aforementioned needs. This will require

Figure 1. Example of generalized node architecture.

L2-L7 functions processing (e.g., from virtual switch/router to apps)

VM: Virtual machine

Interfaces

Virtualization layer

Kernel OS

Standard hardware (processing, memory, packet forwarding)

...

...

VMmanager

Mini OS

VM

Mini OS

VM

Mini OS

VM


IEEE Communications Magazine • July 2013 65

the integration of autonomic and cognitive capa-bilities within virtualization solutions [2]. As anexample, one may imagine an edge networkcapable of self-learning, that is, extracting knowl-edge from the environment and using suchknowledge to improve performance. This meansembedding into its nodes and devices a numberof autonomic and cognitive functions with a setof local rules capable of changing, for instance,the “characteristics” of the local interconnec-tions of a node with the immediate neighbors.

APPLICATION SCENARIOSThis section reports some application scenariosthat could be of particular interest for networkoperators, all fitting into the reference networkscheme depicted in Fig. 2.

One example is network portability: networkservices and functions could be initially deployedand tested using a certain network and cloudenvironment (e.g. in a given domain or country);in a second phase, all services and functions (i.e.,network and server configurations, states, etc.)could be moved to another physical network andcloud environment (e.g., in another domain oranother country) even by leasing processing andnetwork physical resources from other localcloud and infrastructure providers.

Another example is network federation: differ-ent virtual networks (putting together virtual ITand communication resources) can be seamlesslyfederated, in spite of being geographicallyremote. This is the ideal scenario for an opera-tor wishing to provide de-perimeterized servicesacross different domains or countries.

Network partitioning is another possible sce-nario: a virtual network, providing certain ser-vices, can be seamlessly partitioned into smallersubnetworks to simplify administrative tasks.Maintenance can be performed on a subset ofthe infrastructure, without causing any notice-able downtime in the provided services.

From a longer-term perspective, this networktransformation will create the conditions where-by users will literally “decide and drive” futureICT networks and services. This will have a bigimpact. This floating “fog” of ICT resources atthe edge will give rise to new business modelsbased on new forms of competition and coopera-tion between existing providers and new playersentering the arena, including utilities, car manu-facturers, consumer electronics, public adminis-trations, communities, and so on.

A galaxy of new ecosystems will be created,rewarded directly by the market itself, which willbe essential encouragement for further invest-ments.

We already see that the declining costs ofcomputation, communication, and storage aremoving the means of information and entertain-ment production from a limited number of com-panies to hundreds of millions of people aroundthe planet.

Hence, ideally, at the edge it will be possibleto create, program, instantiate, or migratedynamically different types of virtual functionali-ties and services as well as alternatives of thesame. No more ossified architectures, but a sortof ephemeral (temporary) virtual network of

resources capable of self-adapting elastically andflexibly to human dynamics.

Also, we should not forget the rise of large-scale cooperative efforts under the form of opensource software and hardware development andproduction, which might soon create a furtherripple in the telco-ICT vendor markets.

These trends, in turn, will influence the net-work transformation itself by making opensource software and hardware available for carri-ers’ class pieces of equipment.

RELATED WORKS ANDOPEN CHALLENGES

SURVEY OF RELATED WORKVM live migration should be essentially trans-parent to applications: in principle, this is alreadysupported by most of the virtualization platformsfor data centers [3]. For example, most virtual-ization environments support live migration,allowing administrators to move a VM betweenphysical hosts within a LAN platform while run-ning (e.g., XenMotion and VMotion).1

However, when considering moving a VMacross WANs, low bandwidth and high latenciesover network connections may dramaticallyreduce the performance of the VM migrationand consequently the QoS/quality of experience(QoE) of applications. Some commercial solu-tions were recently announced for WAN migra-tion, but are viable only under very constrainedconditions (i.e., 622 Mb/s link bandwidth andless than 5 ms network delay).

The challenge of live migration of VM acrossWAN was analyzed in [4], where the proposedsolution (CloudNet) interconnected local net-works of multiple data centers at layer 2 so thatWAN-based cloud resources looked like localLAN resources, thus allowing LAN-based proto-cols to seamlessly operate across WAN sites.

An overlay approach to create private groupsof VMs across multiple grid computing sites wasalso investigated in [5]; but it remains to be seenhow this approach would scale for NFV in a car-rier class network.

A WAN migration system focusing on effi-ciently synchronizing disk state during migrationwas described in [6]. In this work, the Xen blockdriver is modified to support storage migration,

Figure 2. Future network scenario.

Multipleedge networks

Multipleedge networks

Data centers

Users’resourcenetworks

Users’resourcenetworks

Stateless core network

Data analytics(orchestration of virtual functions

in edge networks and data centers)

1 See the respective web-sites at http://www.xenserver5.com/xenmotion.php andhttp://www.vmware.com/products/vmotion/.



and VM disk accesses are limited when the writerequests occur faster than the network allows.

A solution for high fault tolerance using asyn-chronous VM replication was reported in [7].This method implements a quick filter of cleanpages and maps the entire physical memory ofthe guest domain for reducing the mappingoverhead.

Finally a scheme for VM migration in a fed-erated cloud environment was presented in [8].This solution is used to detect overloaded serversand automatically initiate the migration to a newlocation in the cloud, thus eliminating hot spotsand balancing the load considering CPU, memo-ry, and network as a whole.

In summary, as the prior art in the literatureand commercially available solutions are stillshowing insufficient performance results for car-rier grade networks, we argue that the vision ofan edge network made of clouds of VMs is pre-sumably a viable solution, although there arestill open challenges to be addressed by researchand development communities.

OPEN CHALLENGESOne of the main challenges is to make the migra-tion of VMs as seamless as possible, withoutdeteriorating the QoE. Such a migration can beseen from two different perspectives. The firstrelates to the migration of VMs running anapplication (e.g., video streaming, interactivemultimedia gaming). Motivations to migrate aVM running an application could be for intra- orinter-data-center load balancing (e.g., to avoidperformance degradations due to hot spots), orfollowing Users moving to other network attach-ment points (e.g., for QoE optimization). Thesecond considers the migration of VMs runninga virtual middle-box function (fully developed insoftware); this might be even more challenging,especially when the migration is executed whiletraffic is flowing. Motivations could again be net-work load balancing, traffic engineering (avoid-ing performance degradation due to host spotsand congestion), energy consumption optimiza-tion, and so on.

Typically, moving a VM between two hostsinvolves the following steps: 1. Establish connectivity (e.g., layer 2 for intra-

data-center operations) between the hosts.2. Transfer the whole disk state.3. Transfer the memory state of the VM to the

target host as the source continues runningwithout interruption.

4. Once the disk and most of the memorystates have been transferred, freeze the VMexecution for the final transition of remain-ing memory dirty pages and processor statesto the target server.Implementation and performance issues to

successfully complete these actions are wellunderstood for LANs, but not for WANs, wherebandwidth constraints and latency still adverselyaffect steps 2 and 3, and IP address consistencyis still an issue in step 4.

It is important to match the performanceparameters (e.g., total migration time and totaldowntime) of moving a VM, running applica-tions characterized by a given dirty page rate,with the network performance indicators

(throughput, latency, etc). For example, the totalmigration time is roughly given by the number ofdirty pages (depending on the VM workload)expected to be sent during the whole migrationprocess divided by the available connectionbandwidth. Another challenge is presented byapplications sensitive to the duration of therequired pause or state changes occurring duringthe live migration.

Furthermore, the migration of a batch ofVMs is also very challenging. The scenario hereconsidered envisages networks of intertwinedVMs, implementing IT and network resources.Besides this, VM live migration requireshomogenous virtualization solutions. A compre-hensive analysis of the above technical problemsis still missing and requires further investigation.

Moreover, network operators will have toface the increased complexity of managementand control, as well as the orchestration of thevirtual functions and resources previously pre-sented. This will require, among other importantfunctionalities, the exploit ation of autonomic“local vs. global” capabilities, in order to createa sort of “network operating system.”

Ultimately, an application layer with an openapplication programming interface (API) forprogramming the network at various levels willcomplete the vision.

USE CASE ANDEXPERIMENTAL RESI;TS

This section discusses, through a practical exam-ple, the degree of feasibility of one of the possi-ble scenarios previously introduced. Based onthis reference case, a testbed supporting the net-work function migration was developed. Experi-mental results show that the orchestrationfunction may be successfully implemented byextending existing protocols (Session InitiationProtocol, SIP, in this case) spanning differentlogical layers and network functions, whereas themigration of VMs still represents a performancebottleneck for carrier grade operations, whichrequire further investigation and engineeringeffort.

CASE STUDYThe basic scenario is a simple example of net-work portability and federation, as depicted inFig. 3. A user is watching a video available on avideo server, through both a physical (wireless orwired network access) and a virtual networkinfrastructure (virtual router). The former isresponsible for the pure connectivity betweenthe user and the data center where the service ishosted; the latter is responsible for the addition-al service profiling that may be required by thespecific application, that is, bandwidth reserva-tion, traffic shaping and/or isolation, and so on.

It is assumed that the network operator, at acertain point, considers it more efficient tomigrate the network service, state, and virtualinfrastructure to a different data center. Thiscould be motivated by many factors: for example,initially the user could be watching the video onthe move using mobile access, while at a latertime, he/she reaches home and connects to the

Network operators

will have to face the

increased complexity

of management and

control, as well as

the orchestration of

the virtual functions

and resources previ-

ously presented. This

will require, among

other important

functionalities, the

exploitation of auto-

nomic “local vs.

global” capabilities,

in order to create a

sort of “network

operating system.”



fixed broadband access network. In this case, theoperator decides to move the virtual resources,which were hosted in a data center serving themobile network, to another data center optimallyconnected to the fixed user access network. Thiswill allow, for example, the user to take advan-tage of a larger bandwidth now available andwatch the video stream at a higher definition.

The main actions necessary to perform thismigration are depicted in Fig. 3. Basically, whatis needed is a kind of orchestration of the migra-tion procedure, involving the whole set of VMsthat compose the virtual network infrastructure,and must be moved from one data center toanother one over the WAN, without affectingthe end user experience. In other words, weexpect the migration to be completely transpar-ent to the user, who should keep enjoying thesame video stream from the same video server atthe same network address.

Such orchestration requires a signaling plat-form able to carry cross-layer information, whichis used for coordinating all of the tasks to beperformed. An example of this orchestration wasrecently reported in [9], where a video server ismigrated while reconfiguring the underlying net-work in order to keep the reservation of thebandwidth sufficient for good customer QoE.The use case reported here exploits the samesignaling, but with the additional complexity oforchestrating the migration of a whole set ofVMs and related network state. Our use casefocuses on both network and IT virtualization.Nonetheless, SDN could be part of this scheme:for instance, in case of a more complex physicalnetwork topology, SDN can be the key compo-nent for the orchestration of network resources,which must be properly reconfigured to success-fully complete the migration.

EXPERIMENTAL SETUP

The experimental testbed for the proof of con-cept was implemented on purpose using off-the-shelf technologies. The aim was to understandthe weak points, if any, and areas where ad hocdevelopment and engineering would be requiredto meet carrier grade standards in the overalloperation.

The signaling platform for service manage-ment and orchestration is implemented usingSIP, just as an example (for more details see[10]). In summary, taking advantage of the SIPsession management capability, it is possible tocreate and maintain the network state necessaryto guarantee full consistency during the migra-tion, while the body of SIP messages is able tocarry the information used to specify what has tobe migrated, to where, and when. The signalingscheme is supported by a SIP proxy implement-ed with the OpenSIPS platform with minor add-ons to allow the correct stripping of the messagebody with the additional information. The signal-ing terminals at the user and network operatorsides consist of SIP user agents implemented asweb applications using the PHP language.

The hosting infrastructure is implemented bytwo multicore servers equipped with a LinuxCentOS distribution running VirtualBox as theVM hypervisor. The VMs used in the experi-ment include two virtual single-core Linux boxes,one acting as the video server, the other as theaccess router connecting the user to the server.In order to keep the migration latency as smallas possible, the two VMs were dimensioned withthe minimum amount of memory (512 Mbytesfor both) and disk space (7.2 Gbytes for thevideo server, 1.3 Gbyte for the access router)needed to perform their functions. Live migra-tion of the whole network infrastructure is per-

Figure 3. Schematic graphical example of the service and network migration experiment.

VMs

Host hardware 2

Host hardware 1

Videoserver

Accept incominginfrastructure

ONOFF

WAN Accessrouter

BroadbandMobile

Migrateinfrastructure

Operator

User

I am now at homeand can watch the

video in HD

Let’s see whetherthere is a betterlocation for yourvideo retrieval

network

The experimental

testbed for the proof

of concept was

implemented on pur-

pose using off-the-

shelf technologies.

The aim was to

understand the weak

points, if any, and

areas where ad hoc

development and

engineering would

be required to meet

carrier grade stan-

dards in the overall

operation.



formed through the VM teleporting functionnatively available in VirtualBox.

The two hosting servers emulating two remotedata centers are connected by an ad hoc linkbetween interfaces that are separate from thoseused for communicating with the user. Thissetup emulates the WAN interconnectionbetween data centers. In the experiment report-ed here, this link is implemented with a GigabitEthernet (1000baseT) point-to-point intercon-nection with negligible propagation delay. Thischoice again is motivated by the fact that thegoal of this preliminary experiment is to provethat the concept is feasible with reference to theoverall system architecture. A detailed investiga-tion of the role of the WAN parameters on theperformance is for further study.

In the experiment, the signaling platform exe-cutes the full live migration of network and ITresources, in the sense that both video serverand access router VMs are migrated togetherwith the corresponding virtual network. Oncethe migration is completed, the same signalingplatform triggers a live reconfiguration of thestream from low quality (LQ) to high quality

(HQ), since the user is now connected to thevideo source with broader bandwidth.

EXPERIMENTAL RESULTSFigure 4 reports the video throughput (i.e., themeasured video stream bit rate) as seen by thefinal user during the whole experiment. The bluesolid line represents the variable bit rate (VBR)LQ video stream, which fluctuates around 1 Mb/s,whereas the red dotted line represents the VBRHQ stream, with average bit rate between 2 and 3Mb/s, although some peaks reach 6 Mb/s. Theinset in Fig. 4 shows an enhanced view (zoom in)of the interval when the live migration occurs.The vertical arrows indicate the time instantswhen the live migration of the video server starts(T1) and ends (T2), when the live migration ofthe router starts (T3) and ends (T4), and whenthe stream switches from LQ to HQ (T5).

When the migration starts at T1, the userreceives the LQ stream from the video serverVM instance still running at the source host.However, as soon as the video server migrationis complete at T2, the LQ stream is interruptedbecause there is no connectivity between theaccess router still running on the source hostand the video server now running on the desti-nation host. The interruption lasts for about 5 s,which is the time needed in our testbed to com-plete the migration of the access router VM(from T3 to T4) and restore the connectivitybetween the user and the video server. Then theLQ stream resumes, as measured by the bluesolid curve rising again after T4. After a few sec-onds, at T5 the stream switches from LQ to HQ,as shown by the end of the blue solid curve as itis replaced by the red dotted one.

The system outage caused by the live migra-tion in another experiment is reported in detailin Fig. 5, which shows the packet capture of aping session where ICMP ECHO messages aresent every 100 ms. The video server is at IPaddress 172.16.3.100, while the user terminal isat IP address 192.168.107.197. This captureshows that the packets with sequence numbersfrom 216 to 245 are missing. These 30 packetsare lost during the VM migration, whichaccounts for a network outage of about 3 s. It isworth noting that, apart from this, the pingpacket flow is exactly the same before and after

Figure 4. The video data flow as seen by the user terminal.

Time (s)0

1e+06

0

Vid

eo s

trea

m b

it r

ate

(b/s

)

2e+06

3e+06

4e+06

5e+06

6e+06

7e+06

10 20 30 40 50 60 70 80 90 100 110 120

HQ streamLQ stream

Time50

T1 T2T3 T4

T5

45

1e+06

0

2e+06

3e+06

4e+06

55 60 65 70 75

Figure 5. Packet capture of a continuous ping session from the video server (172.16.3.100) to the user terminal (192.168.107.197) dur-ing live VM migration.

MANZALINI LAYOUT_Layout 1 6/26/13 11:52 AM 8


the migration, showing that IP addresses andnetwork state are kept unchanged.

Finally, Fig. 6 shows a capture of the SIP sig-naling that triggers the VM migration. The cap-ture shows the dialog between the SIP useragent at the source (IP: 10.10.10.66, UDP port:50601, controlled by either the end user or thenetwork/service manager), the SIP proxy (IP:10.10.10.66, UDP port: 5060), and the SIP useragent at the destination (IP: 10.10.10.67, UDPport: 5060).

CONCLUSIONS AND OPENCHALLENGES FOR TELCO OPERATORSIn this article, it is argued that future networksinfrastructures will be made up of a huge num-ber of virtual resources (compute, storage, andnetwork I/O) being controlled dynamically basedon users’ demands, QoS, and business objectives,as well as any other changing conditions. In par-ticular, standard hardware advances and emerg-ing paradigms, such as SDN and NFV, willenable this remarkable disruption at the edge ofcurrent networks. Virtualized network and ser-vice functions, supported today by expensivemiddle-boxes, will run at the edge of the net-work, as close as possible to the users.

The amazing increase in smart nodes anddevices at the edges will globally make availableenough processing power, data storage capacity,and communications bandwidth to provide sev-eral services with local edge resources.

Actually, we are already witnessing this evolu-tion if we consider the current shift of value fromthe network to the terminals. Services are moreand more often provided at the edge of the net-work, and this trend will continue in the future.

This transformation will turn the edge into abusiness arena composed of a multiplicity ofinteracting subdomains, operated by diverse

(both private and public) players and user com-munities.

Network operators should closely follow upon this invaluable transformation and evolvetheir business models accordingly: pursuing thetraditional approach, adopting walled gardensand being conservative, will be detrimental fortheir business in the long run, limiting theiroffering to mere pipe connectivity. On the otherhand, enabling open virtual environments at theedge will offer several business opportunities: infact, future services and data will be broadlydelivered through multiple devices, machines,and objects, mostly by using local resources.

Network operators could also play the role ofinfrastructure providers of edge networks, in con-junction with public administrations or other playerswilling to cooperate. The role is to help reduce thecomplexity and act as an “anchor” around which toorganize complex and dynamic edge systems.

In this scenario, operations normally carriedout in data centers, such as allocation, migration,and cloning of virtual resources and functionscould be advantageously applied at networkedges. This means that it should be possible tounderpin those techniques, properly enhanced,and management tools commonly used today indata centers. Nevertheless, this implies the needto overcome several technical challenges, amongthem the seamless allocation and migration ofVMs across multiple distributed servers.

In this sense, it will also be possible to over-come routing processing limitations by optimiz-ing the use of the huge amount of computingand storage power available today in large datacenters and accessible tomorrow at the edges ofnetworks. Software router architectures, forexample, may be capable of parallelizing routingfunctionality across multiple servers.

In principle, this would change the (econom-ic) equation of the network: overprovisioningconnectivity rather than just overprovisioning

Figure 6. SIP signaling sequence triggering the VM migration. The source SIP user agent is at 10.10.10.66:50601, the SIP proxy is at10.10.10.66:5060, and the destination SIP user agent is at 10.10.10.67:5060.



bandwidth (capacity). The former pays off betterthan the latter: it would be possible to create avery large number of topologies from which tochoose, even almost randomly, or program andcontrol the QoS at higher levels. As of today,overprovisioning connectivity in a network ismore expensive than overprovisioning capacity,but tomorrow this equation may change.

The feasibility of this vision — under somepractical application scenarios, taking advantageof existing technologies to implement the ses-sion-based signaling platform required to main-tain the network state while migrating the virtualresources — was demonstrated using an ad hoctestbed. The experimental results attained usingthis proof of concept proved future edge net-works made of clouds of VMs (running virtual-ized network functions and services) to bepotentially feasible, as long as the performancelimitations imposed by the current technologyare improved.

It is left for further studies how data analyticsmay enable a global autonomic loop (comple-menting local actions) for orchestrating cloudsof VMs at the edges and in the data centers.

REFERENCES[1] G. Schaffrath et al., “Network Virtualization Architec-

ture: Proposal and Initial Prototype,” Applications,Technologies, Architectures, and Protocols for Comput-er Communication, Proc. 1st ACM Wksp. VirtualizedInfrastructure Systems and Architectures, 2009.

[2] A. Manzalini et al., “Self-Optimized Cognitive Networkof Networks,” Oxford Journals’ The Computer Journal,2010, vol. 54, issue 2, pp 189–96.

[3] T. Wood et al., “CloudNet: A Platform for OptimizedWAN Migration of Virtual Machines,” Univ. of MA tech.rep., 2010.

[4] P. Ruth et al., ”Autonomic Live Adaptation of VirtualComputational Environments in a Multi-Domain Infra-structure,” Proc. 2006 IEEE Int’l. Conf. Autonomic Com-puting, Washington, DC, 2006.

[5] A. I. Sundararaj and P. A. Dinda, “Towards Virtual Net-works for Virtual Machine Grid Computing,” Proc. 3rdConf. Virtual Machine Research and Tech. Symp., 2004.

[6] R. Bradford et al., “Live Wide-Area Migration of VirtualMachines Including Local Persistent State,” Proc. 3rdInt’l. Conf. Virtual Execution Environments, San Diego,CA, 2007, pp. 169–79.

[7] S. Hacking and B. Hudzia, “Improving the Live Migra-tion Process of Large Enterprise Applications,” Proc.ACM Int’l. Wksp. Virtualization Tech. in Distrib. Com-puting, New York, NY, 2009, pp. 51–58.

[8] Y. Xu and Y. Sekiya, “Scheme of Resource OptimizationUsing VM Migration for Federated Cloud,” Proc. Asia-Pacific Advanced Network 2011, vol. 32, pp. 36–44.

[9] F. Callegati, A. Campi, and W. Cerroni,, “ApplicationScenarios for Cognitive Transport Service in Next-Gener-ation Networks,” IEEE Commun. Mag., vol. 50, no. 3,pp. 62–69.

[10] F. Callegati, A. Campi, and W. Cerroni, “Automated Trans-port Service Management in the Future Internet: Conceptsand Operations,” J. Internet Services and Applications,Springer, vol. 2. no. 2, Sept. 2011, pp. 69–79.

BIOGRAPHIESANTONIO MANZALINI ([email protected])received an M.Sc. degree in electronic engineering fromthe Politecnico of Turin. In 1990 he joined CSELT, whichthen became Telecom Italia Lab. He started his activities onresearch and development of technologies and architec-tures for future optical networks. In this R&D area, he wasactively involved in leading positions in several EURESCOMand EC funded projects (e.g. MWTN, LION, NOBEL, CAS-CADAS). He chaired two ITU-T Questions on transport net-works. He is the author of a book on networksynchronization (for SDH), and his R&D results are pub-lished in more than 80 papers. He has five patents on net-works and systems. He has been a member of technicaland program committees of several IEEE conferences. He iscurrently joining the Strategy Department of Telecom Italia,addressing R&D activities mainly concerning future telco-ICT networks and services technologies and solutions (e.g.,software defined networks, network function virtualization,autonomic/cognitive, and self-networks).

ROBERTO MINERVA ([email protected])received a Laurea in computer science cum laude from theUniversity of Bari in 1987. He works in Telecom Italia’sFuture Center, where he leads the Innovative ArchitectureGroup. He also is a contract professor at Turin’s Polytech-nic, where he teaches a course on mobile services. He hasworked in industrial research for more than 25 years deal-ing with topics such as network intelligence, SIP, servicearchitectures, and next generation networks. His currentresearch topics include edge networks, peer-to-peer sys-tems, autonomics and cognitive systems, personal data,and the Internet of Things. He has served on the TPCs ofmany international conferences, and has published morethan 40 papers in peer reviewed and international confer-ences.

FRANCO CALLEGATI [M’98, SM’11] ([email protected])is an associate professor of telecommunication networks atthe University of Bologna, Italy. His research interests are inthe field of teletraffic modeling and performance evalua-tion of telecommunication networks. He has well estab-lished research expertise in optical networking, opticalpacket and burst switching, service oriented networks,autonomic networks, and network security. He has beenactive in EU-funded research projects since FP4, where heled activities and participated in various steering commit-tees.

ALDO CAMPI ([email protected]) holds a post-doctoralposition at the Center for Industrial Research on Informa-tion and Communication Technologies of the University ofBologna. In 2007 he spent 10 months at the University ofEssex, United Kingdom, as a visiting researcher working onapplication-aware networking. His research interestsinclude optical networks, scheduling algorithms, SIP, gridnetworking, service-oriented networks, NGN architectures,and network infrastructures for cloud computing.

WALTER CERRONI [M’01] ([email protected]) is an assis-tant professor of telecommunication networks at the Uni-versity of Bologna. Previously, he was a research associateat the Italian National Inter-University Consortium forTelecommunications (CNIT). In 2008 he was a visiting assis-tant professor at the School of Information Sciences, Uni-versity of Pittsburgh, Pennsylvania. His research interestsinclude architectures and performance of dynamic opticalnetworks, next-generation cognitive and programmablenetworks, software-defined networking, and network secu-rity.

The experimental

results attained using

this proof of concept

proved future edge

networks made of

clouds of VMs to be

potentially feasible,

as long as the

performance

limitations imposed

by the current

technology are

improved.


clouds of virtual machines in edge networks

Documents