middleware-oriented deployment automation for cloud ... · open source enterprise service bus esbmt...

14
2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325, IEEE Transactions on Cloud Computing 1 Middleware-oriented Deployment Automation for Cloud Applications Johannes Wettinger, Vasilios Andrikopoulos, Frank Leymann, Steve Strauch Institute of Architecture of Application Systems (IAAS) University of Stuttgart, Stuttgart, Germany {wettinger, andrikopoulos, leymann, strauch}@iaas.uni-stuttgart.de Abstract—Fully automated provisioning and deployment of appli- cations is one of the most essential prerequisites to make use of the benefits of Cloud computing in order to reduce the costs for managing applications. A huge variety of approaches, tools, and providers are available to automate the involved processes. The DevOps community, for instance, provides tooling and reusable artifacts to implement deployment automation in an application- oriented manner. Platform-as-a-Service frameworks are available for the same purpose. In this work we systematically classify and characterize available deployment approaches independently from the underlying technology used. For motivation and evaluation purposes, we choose Web applications with different technology stacks and analyze their specific deployment requirements. Afterwards, we provision these applications using each of the identified types of deployment approaches in the Cloud to perform qualitative and quantitative measurements. Finally, we discuss the evaluation results and derive recommendations to decide which deployment approach to use based on the deployment requirements of an application. Our results show that deployment approaches can also be efficiently combined if there is no ‘best fit’ for a particular application. Index Terms—middleware-oriented deployment; application-oriented deployment; Cloud computing; DevOps; decision support 1 I NTRODUCTION Public and relatively cheap infrastructure offerings such as Amazon Web Services 1 are the foundation for the popularity and success of Cloud computing. Following the Infrastructure as a Service (IaaS) delivery model [1], resources such as virtual machines and virtual disks can be provisioned and decommissioned on demand, so the customers only pay for what they are actually using. According to the NIST definition of Cloud computing [1] services provided through the Cloud are not limited to the infrastructure level to provision infrastructure resources such as computing power and storage. There are higher-level service delivery models available, namely Platform as a Service (PaaS) and Software as a Service (SaaS). As the Cloud service market is becoming more mature, these service models gain more traction in the market with further 1. Amazon Web Services: http://aws.amazon.com offerings such as the Google App Engine 2 becoming available. In particular in the case of the PaaS model, providers essentially offer Cloud-enabled middleware solutions to their customers. Such solutions can either be provided as middleware services (e.g., database as a service) or reusable middleware components that can be used as a foundation to deploy the actual applica- tion components. Different deployment approaches are available in this environment, depending on how the provider-driven automation of the offered middleware solutions is leveraged by application developers. The implementation of holistically automated deployment processes is especially driven by the rapidly emerging DevOps paradigm [2], [3] and continuous delivery practices [4], [5]. The research presented in this article builds on our previous work [6], [7] and focuses on the charac- terization of these deployment approaches, with the intention of identifying the most efficient deployment of different types of application stacks on PaaS and IaaS solutions. The overall goal of our work is to build the foundation for a decision support system for Cloud application deployment. In addition, this work addresses some of the open issues identified in [6] and [7], in terms of providing an extended evaluation of the discussed approaches. Beside refined measurements and presentation of results, we eval- uated the reusability of deployment artifacts when following different deployment automation strategies. Thus, the original findings of [7] are refined and the resulting lessons learned are further refined. Moreover, we provide a comprehensive, systematic analysis and categorization of the state of the art as a foundation for our work. The main contributions of this work can therefore be summarized as follows: We present a systematic classification of the state of the art and the limitations of current operations automation approaches. Based on this classification, we define and charac- 2. Google App Engine: https://cloud.google.com/appengine

Upload: vuongdang

Post on 19-Jul-2018

238 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

1

Middleware-oriented Deployment Automationfor Cloud Applications

Johannes Wettinger, Vasilios Andrikopoulos, Frank Leymann, Steve StrauchInstitute of Architecture of Application Systems (IAAS)

University of Stuttgart, Stuttgart, Germany{wettinger, andrikopoulos, leymann, strauch}@iaas.uni-stuttgart.de

F

Abstract—Fully automated provisioning and deployment of appli-cations is one of the most essential prerequisites to make use ofthe benefits of Cloud computing in order to reduce the costs formanaging applications. A huge variety of approaches, tools, andproviders are available to automate the involved processes. TheDevOps community, for instance, provides tooling and reusableartifacts to implement deployment automation in an application-oriented manner. Platform-as-a-Service frameworks are availablefor the same purpose. In this work we systematically classify andcharacterize available deployment approaches independently from theunderlying technology used. For motivation and evaluation purposes,we choose Web applications with different technology stacks andanalyze their specific deployment requirements. Afterwards, weprovision these applications using each of the identified types ofdeployment approaches in the Cloud to perform qualitative andquantitative measurements. Finally, we discuss the evaluation resultsand derive recommendations to decide which deployment approachto use based on the deployment requirements of an application. Ourresults show that deployment approaches can also be efficientlycombined if there is no ‘best fit’ for a particular application.

Index Terms—middleware-oriented deployment; application-orienteddeployment; Cloud computing; DevOps; decision support

1 INTRODUCTION

Public and relatively cheap infrastructure offeringssuch as Amazon Web Services1 are the foundationfor the popularity and success of Cloud computing.Following the Infrastructure as a Service (IaaS) deliverymodel [1], resources such as virtual machines andvirtual disks can be provisioned and decommissionedon demand, so the customers only pay for what theyare actually using. According to the NIST definitionof Cloud computing [1] services provided through theCloud are not limited to the infrastructure level toprovision infrastructure resources such as computingpower and storage. There are higher-level servicedelivery models available, namely Platform as a Service(PaaS) and Software as a Service (SaaS). As the Cloudservice market is becoming more mature, these servicemodels gain more traction in the market with further

1. Amazon Web Services: http://aws.amazon.com

offerings such as the Google App Engine2 becomingavailable. In particular in the case of the PaaS model,providers essentially offer Cloud-enabled middlewaresolutions to their customers. Such solutions can eitherbe provided as middleware services (e.g., database asa service) or reusable middleware components that canbe used as a foundation to deploy the actual applica-tion components. Different deployment approaches areavailable in this environment, depending on how theprovider-driven automation of the offered middlewaresolutions is leveraged by application developers. Theimplementation of holistically automated deploymentprocesses is especially driven by the rapidly emergingDevOps paradigm [2], [3] and continuous deliverypractices [4], [5].

The research presented in this article builds onour previous work [6], [7] and focuses on the charac-terization of these deployment approaches, with theintention of identifying the most efficient deploymentof different types of application stacks on PaaS andIaaS solutions. The overall goal of our work is tobuild the foundation for a decision support systemfor Cloud application deployment. In addition, thiswork addresses some of the open issues identifiedin [6] and [7], in terms of providing an extendedevaluation of the discussed approaches. Beside refinedmeasurements and presentation of results, we eval-uated the reusability of deployment artifacts whenfollowing different deployment automation strategies.Thus, the original findings of [7] are refined and theresulting lessons learned are further refined. Moreover,we provide a comprehensive, systematic analysis andcategorization of the state of the art as a foundationfor our work. The main contributions of this work cantherefore be summarized as follows:

• We present a systematic classification of the stateof the art and the limitations of current operationsautomation approaches.

• Based on this classification, we define and charac-

2. Google App Engine: https://cloud.google.com/appengine

Page 2: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

2

terize two distinct types of deployment approaches.• We analyze the deployment requirements of three

different applications covering some of the mostpopular technologies for building Web applica-tions today.

• Based on these requirements, we implement theautomated deployment of all three applicationsusing both types of deployment approaches anddifferent Cloud infrastructures for evaluation pur-poses. This results in eight deployment scenariosfor which we measure both qualitative and quantita-tive properties and we derive a number of findings.

• We present a list of lessons learned based on thesefindings that can be used to support decisionmaking concerning which deployment approachis more appropriate for a particular application.

The remaining of this article is structured as follows:Section 2 introduces the applications to be used forevaluation purposes and analyzes their specific de-ployment requirements in order to motivate our work.Based on our investigation of the state of the art inSection 3, two major types of deployment automationapproaches are identified in Section 4. Our evalua-tion of these deployment approaches is presented inSection 5 based on the three applications and theirdeployment requirements described in Section 2. Theevaluation results are discussed in Section 6, includingthe reporting of our findings. Furthermore, an initiallist of lessons learned is derived from these findings.Finally, Section 7 concludes this article and providesan outlook in terms of future work.

2 MOTIVATION

In this section we introduce the three applicationsthat are realized based on different technologies (Sec-tion 2.1). These applications are used for the evaluationof the different deployment approaches we propose inthis paper. Moreover, we identify application-specificdeployment requirements for all three applications(Section 2.2) to be addressed in the evaluation.

2.1 ApplicationsWe chose the applications Taxi Application (Taxi App),SugarCRM, and Chat Application (Chat App) becausethey cover a set of the most important and establishedtechnologies used for Web application developmenttoday. These are in particular, Java EE and Webservices, PHP and the LAMP stack, and Node.js, respec-tively. As confirmed by various programming languageand technology stack popularity statistics such asGitHut3 and LangPop4, these applications target thecurrent TOP 5 technology stacks, and can therefore beconsidered as representative for a significantly largeset of today’s Web applications. In the context of our

3. GitHut: http://githut.info4. LangPop: http://langpop.com

Legend

Google Maps Web Services

Context as a Service

Orchestra BPEL Engine

External Comp. / Service Context Integration

Processes (CIPs) (BPEL Processes)

Taxi Company A

Customer GUI

Taxi Drivers’

GUI

Taxi Company B

Customer GUI

Taxi Drivers’

GUI

Front-End Back-End

JOnAS Application Server

Middleware Component

Application Component

Taxi Service Provider

(BPEL Process)

Clients

Communicates

JOnAS Application Server

Tenant Registry

(PostgreSQL)

Service Registry

(PostgreSQL)

Config Registry

(PostgreSQL)

Figure 1. Architecture of the Taxi Application

current research we focus on Web applications as amajor class of applications, which is highly relevantfor diverse fields such as software as a service, mobileapps, and the Internet of things. The topologies ofall three applications consist of middleware components,application components, and external services.

Taxi App – An overview of the Taxi App’s ar-chitecture developed in the scope of the EuropeanProject 4CaaSt as a demonstrator of the PaaS offeringsof the project is shown in Figure 1 [8]. A serviceprovider offers taxi management software as a serviceto different taxi companies, i.e., tenants. Taxi companycustomers, who are users of the tenant, submit theirtaxi transportation requests to the company that theyare registered with. The taxi management software(back-end) is realized as a set of business processesusing BPEL [9]. The taxi management software lever-ages context integration processes also implementedin BPEL to retrieve context information about taxi cabssuch as location and taxi driver contact details fromthe 4CaaSt platform-provided Context as a Service.Moreover, Google Maps Web Services [10] providedistance calculations between the pick-up location andthe location of the taxi cab. All BPEL processes aredeployed in the open source BPEL engine Orchestra5

version 4.9.0-M3, which itself is deployed in the JavaOpen Application Server (JOnAS)6 version 5.3.0-M4.The taxi company-specific front-ends consist of aCustomer GUI and a Taxi Drivers’ GUI, which are bothdeployed in JOnAS version 5.3.0-M4. The multi-tenant,open source Enterprise Service Bus ESBMT [11] as mes-saging middleware (Figure 1) enables loose couplingand provides a flexible integration solution by avoidinghard-coded point-to-point connections. ESBMT is basedon Apache ServiceMix7 version 4.3.0 and comes withthree registries realized as PostgreSQL8 version 9.1

5. OW2 Orchestra: http://orchestra.ow2.org6. OW2 JOnAS: http://jonas.ow2.org7. Apache ServiceMix: http://servicemix.apache.org8. PostgreSQL: http://www.postgresql.org

Page 3: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

3

Apache HTTP Server

Clients

PHP Module

SugarCRM Application

MySQL DB Server

PHP Runtime SugarCRM

DB

Legend

External Comp. / Service

Middleware Component

Application Component

Communicates

Uses

Figure 2. Architecture of SugarCRM

Node.js

Clients

Chat Application

Redis DB Server

Chat Log DB

Legend

External Comp. / Service

Middleware Component

Application Component

Communicates

Figure 3. Architecture of the Chat Application

databases [12].SugarCRM – Figure 2 provides an overview of the

architecture of SugarCRM9, an open source CustomerRelationship Management Software (CRM), whichis used for interoperability demonstration purposesby the technical committee of the Topology andOrchestration Specification for Cloud Applications(TOSCA) [13]. All relevant data such as contact detailsof the customers are stored in the SugarCRM Databasewithin a MySQL Database Server10 version 5.5.32. TheSugarCRM Web Application is implemented in PHPand is running on an Apache HTTP Server11 version2.2.22 using a PHP runtime version 5.3.10.

Chat App – The Chat App’s architecture is presentedin Figure 3. The user information and the chat logsare stored in the Chat Log database using a RedisDatabase Server12 version 2.6.14. The Chat App isbased on Node.js version 0.10.1313. All clients (Webbrowsers) communicate with the Node.js-based chatserver using the WebSocket14 protocol.

In the next section we define the general andapplication-specific deployment requirements for eachof the three applications introduced. These require-ments have to be considered when creating corre-sponding deployment plans. A deployment plan (e.g.,

9. SugarCRM: http://www.sugarcrm.com10. MySQL Server: http://www.mysql.com11. Apache HTTP Server: http://httpd.apache.org12. Redis: http://redis.io13. Node.js: http://nodejs.org14. WebSocket: https://www.websocket.org

a script) implements the logic necessary to deployapplication or middleware components.

2.2 Deployment Requirements

It can be easily observed that there are requirementsthat apply to the deployment of all three presentedapplications. Since the scope of our research is on de-ployment automation and approaches for its technicalimplementation we are focusing only on technicallyrelevant deployment requirements. We therefore donot consider additional non-functional requirementsregarding performance, costs, compliance, etc. As such,the following general deployment requirements needto be considered in this context:GR1 Middleware deployment: Configurable deployment

plans are required to deploy all middleware com-ponents involved, such as the JOnAS applicationserver for the Taxi App, or the MySQL databaseserver for SugarCRM.

GR2 Application deployment: Configurable deploymentplans are required to deploy all applicationcomponents on top of the middleware, such asthe customer user interface for the Taxi App orthe chat log database for the Chat App.

GR3 Wiring of components: The middleware as wellas the application components are not deployedin an isolated manner. Application componentsneed to be wired with each other, or withsome of the middleware components to enablecommunication between components as outlinedin Figure 1, Figure 2, and Figure 3.

Further requirements regarding the design of deploy-ment plans such as modularity, configurability, extensi-bility, and portability are discussed in [6]. In addition tothese general requirements, each presented applicationimposes additional deployment requirements. For theTaxi App [11] we identified the following requirementswhen we implemented its automated deployment:TR1 Deployment of additional tenants: After the initial

deployment has been performed, mechanismsare needed to deploy additional tenants. Thisinvolves sending SOAP messages to the Webservice-based management interface of ESBMT

as well as deploying additional WAR files (cus-tomer GUI and taxi drivers’ GUI) to the JOnASapplication server.

TR2 Registration and configuration of tenant endpoints: Inorder to register and configure tenant endpointsinside ESBMT several SOAP messages need tobe sent to its management interface both uponinitial deployment of the application and whenadditional tenants are deployed. For the Taxi Appthese messages are sent by running a numberof test cases by SoapUI’s15 command line-basedtest runner.

15. SoapUI: http://www.soapui.org

Page 4: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

4

TR3 Modification of WAR files: The WAR files thatare deployed both upon the application’s initialdeployment and when adding further tenantsneed to be modified at deployment time. Thisis because they need to connect to the tenantendpoints registered in ESBMT. Therefore, theaddress of the corresponding tenant endpointand the tenant ID have to be stored inside aproperties file located in the WAR file.

In contrast to the Taxi App (and the Chat App,later) the deployment automation of SugarCRM asdescribed in [14] was realized with distributed deploy-ment in mind, meaning that the application and thedatabase are hosted on two different virtual machines.When automating the deployment of SugarCRM, thefollowing requirements were identified:SR1 Permission settings: The permissions of particular

files and directories of the PHP application needto be set. As an example, the cache directory needsto be writable. This is a typical requirement forthe deployment of PHP applications.

SR2 Silent installation: After extracting and placingall the PHP script files of the application, a socalled silent installation needs to be triggered bycalling the install.php script using an HTTPGET request. As a result, the database structureis created and the application gets configured bycreating a default user (administrator).

SR3 Dynamic wiring at deployment time: The SugarCRMapplication needs to be wired with the databasedynamically at deployment time; the endpoint of thedatabase needs to be put to the application’s con-figuration, so the connection can be established(this requirement is a refinement of GR3).

Compared to the Taxi App and SugarCRM, theChat App’s architecture is relatively simple. Thus,automating the deployment based on deploymentplans is easier. There were two requirements identifiedduring the implementation of the deployment plans:CR1 Node package manager integration: Node.js appli-

cations such as the Chat App typically providea package.json file that holds some metadataof the application as well as its dependencieson other Node.js modules. These dependenciesare resolved at deployment time using the nodepackage manager (npm)16.

CR2 Pointer to application’s entry point: Node.js appli-cations typically consist of several scripts imple-mented in JavaScript. To start such an application,a particular script needs to be defined as entrypoint. This script is then called by the Node.jsruntime environment to initialize and start theactual application.

Different applications have therefore different re-quirements regarding their deployment. These indi-vidual requirements have to be taken into account

16. npm: http://npmjs.org

when implementing plans to automate the deploy-ment process of a particular application. In principle,there exist different approaches to realize deploymentautomation such as the one described in [6]. Becausethere is not a single approach that fits all the differentrequirements, it is necessary to be able to identifythe most suitable approach for each type of applica-tion. Thus, in the following we categorize existingapproaches to automate operations especially focusingon automated deployment. However, all relevantdeployment requirements for a particular applicationmust be satisfied when implementing its automateddeployment using any of the categorized approaches,i.e., the identified requirements are orthogonal to theapproaches outlined in the following classification.

3 STATE OF THE ART

In the following we survey various deployment ap-proaches from the literature. We consider each deploy-ment approach as a key part of a certain operationsautomation approach because operating applications asdescribed in Section 2 needs to cover additional aspectsbeside deployment. Typical examples include doing adatabase backup or scaling certain parts of the appli-cation. Thus, we consider the automated deploymentof middleware and application components as a sub-discipline of operations automation.

Figure 4 categorizes state of the art approaches toimplement operations automation. The categories arearranged in a hierarchic structure with leaves groupingthe actual automation approaches. Approaches in eachleaf group (category) in the hierarchy are furtherclassified to one of three types: (1) a provider-dependentapproach such as using a PaaS solution offered by acertain provider (e.g., Heroku17), (2) a tooling-dependentapproach such as an abstraction library to interactwith multiple providers (e.g., fog18), or a piece ofsoftware that does not have any dependencies oncertain providers (e.g., Docker [15]). Some of theprovider-dependent approaches are implicitly tooling-dependent, too. For instance, Amazon provides a setof tools19 to interact and integrate with their servicesprogrammatically. (3) Moreover, standards-based ap-proaches exist such as the Topology and OrchestrationSpecification for Cloud Applications (TOSCA) [16],which enables model-based management of Cloudapplications and infrastructure resources.

Platform-centric management approaches enable de-ployment in the PaaS model. These are usuallybased on provider-dependent platform offerings suchas Google App Engine [17] or Amazon ElasticBeanstalk [18]. The goal of the PaaS model is toprovide a platform that abstracts from the underlyinginfrastructure resources and provides “middleware

17. Heroku: http://www.heroku.com18. fog: http://github.com/fog/fog19. AWS tools: http://aws.amazon.com/tools

Page 5: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

5

Opera&ons  Automa&on  

Infrastructure  Management  

Configura&on  Management  

Pla8orm-­‐centric  Management  

Model-­‐based  Management  

Plan-­‐based  Configura&on  Management  

Container-­‐based  

Virtualiza&on  

Infrastructure-­‐centric  Modeling  

Applica&on-­‐centric  Modeling  

AWS  SDKs,  AWS  CLI,  ...    OpenStack  SDK,  Vagrant,  fog,  jclouds,  ...    CIMI,  OCCI  

Enterprise  Chef      Chef,  Puppet,  SmartFrog,  ...      TOSCA  Plan  API  

Tutum,  …      Docker,  LXC,  ...  

Heroku,  AWS  Beanstalk,  ...    Cloud  Foundry,  OpenShiK,  ...      CAMP,  (OCCI)  

AWS  Cloud-­‐FormaOon,  ...    Heat,  ...        TOSCA  

AWS  OpsWorks,  ...    Juju,  Blueprints,  ...      TOSCA  

Image-­‐based  Configura&on  Management  

Hypervisor-­‐based  

Virtualiza&on  

AWS  EC2,  FlexiScale,  ...    VMware,  OpenStack,  Xen,  ...    OVF  

Provider-­‐dependent  

 Tooling-­‐

dependent      

Standards  

Figure 4. Categorized State of the Art Approaches to Implement Operations Automation

as a service”. Thus, the application components aredirectly hosted on the platform. To host the Taxi App,SugarCRM, or Chat App using the PaaS model, several“middleware services” are required to be exposedby the platform. These middleware services can forexample provide an ESB, a BPEL engine, or a databaseserver depending on the topology of the correspondingapplication. As these middleware services may not beoffered out of the box by PaaS providers, a customPaaS environment can be built based on existinginfrastructure resources, e.g., inside the private bound-aries of a particular organization. As an example, thePaaS framework Cloud Foundry20 enables a tooling-dependent platform-centric management approach. Allplatform-centric approaches have the drawback ofrestricting control over the underlying infrastructure.For instance, the filesystem cannot by accessed inmost cases. Moreover, the programming model is oftenlimited to the use of certain frameworks and APIs. Thisis necessary in most cases to implement an efficientlayer of abstraction for the services offered by theplatform.

Higher-level model-based management approachesenable the definition of a holistic application-centricmodel of a particular Cloud service for the deployment,its structure and behavior. Such a holistic model canbe provider-dependent (e.g., AWS OpsWorks [19]),tooling-dependent (e.g., Juju21 or Blueprints [20]), orstandards-based (e.g., TOSCA [16]). In addition, thereare commercial products available that implement themodel-based approach. An example of these productsis the IBM SmartCloud Orchestrator22. The goal ofthe model-based approach is to enable top-down

20. Cloud Foundry: http://www.cloudfoundry.org21. Juju: http://juju.ubuntu.com22. IBM SmartCloud Orchestrator: http://ibm.co/CPandO

modeling by starting with a higher-level model for aCloud application. To enable the deployment of sucha model, additional artifacts such as scripts may haveto be attached to the model to perform the actualdeployment of the components that are involved.

Another model-based technique is to createinfrastructure-centric models. For instance, AWSCloudFormation23 enables the creation of provider-dependent models by defining a set of resources suchas virtual machines (VMs). Furthermore, scripts canbe embedded to install and configure middleware andapplication components on these machines. However,in contrast to the application-centric models discussedbefore, these models focus on the orchestration ofresources on the infrastructure level and do notexplicitly define relations between middleware andapplication components.

The configuration management approaches can be seenas an alternative or a complement to the model-basedapproaches. Either plan-based approaches or image-basedvirtualization techniques can be used to implementoperations automation, most probably under somekind of version control. Plans such as scripts are usedto install and configure middleware and applicationcomponents. The image-based approach with respectto the IaaS service model encapsulates the differentmiddleware and application components in VM im-ages. Today, there are many IaaS providers offeringthe deployment of VM images such as Amazon WebServices (AWS)24. In addition, there are open sourceproducts such as OpenStack [21] available to createan IaaS environment for deploying VM images. TheOpen Virtualization Format (OVF) [22] aims to be

23. AWS CloudFormation: http://aws.amazon.com/cloudformation24. Amazon Web Services: http://aws.amazon.com

Page 6: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

6

a standardized format for such images. These are allhypervisor-based virtualization approaches, meaning thegranularity is always on the level of a complete VM.Container-based virtualization approaches (e.g., Docker)enables to further virtualize a VM. Consequently,different middleware and application components canbe encapsulated in container images.

For each of the three applications introduced inSection 2, a VM image could be created to host thewhole application, or each application component canbe hosted on a separate VM, or on a separate containeron the same or different VMs. Another possibility isa compromise between both options by hosting somecomponents on separate VMs and other componentsof the same application to be grouped together tobe hosted in the same VM, e.g., depending on theresource requirements of each component. Severalapproaches are available that are focused on optimizedprovisioning of virtual machines and deploying virtualmachine images such as [23], [24], and [25].

Focusing on the IaaS model, the alternative is to usestandard images that basically provide only a plainoperating system, instead of completely pre-installedand pre-configured images. Plan-based configurationmanagement approaches proposed by the DevOpscommunity such as Chef [26], [27], Puppet [28], [29],CFEngine [30], SmartFrog [31], or Engage [32] canbe used to install and configure the actual middle-ware and application components. Scripts are usedto perform the installation and configuration [33].Conventional workflow approaches such as BPEL [9]or BPMN [34] may also be utilized to implementplans that perform these tasks. In order to managetopologies that consist of several machines and differ-ent components hosted on them, overarching plansor model-based approaches such as AWS CloudFor-mation, AWS OpsWorks, or Juju can be used. Anefficient way of combining configuration managementwith model-based management is described in [14].In addition, there are model-based approaches suchas EnStratus [35] or RightScale25 that use Chef scriptsin the background to perform the actual deployment.For the three applications considered in this work thisdeployment approach implies to have at least severalscripts to install and configure all the middleware andapplication components that are involved. In addition,there may be an overarching model that orchestratesall scripts involved.

The automation approaches shown in Figure 4 arenot meant to be mutually exclusive. For instance,configuration management is mainly focused on in-stalling and configuring middleware and applicationcomponents on an infrastructure. However, to provi-sion infrastructure resources such as virtual machines,infrastructure management approaches are typicallyrequired. Moreover, there are extensions for some

25. RightScale and Chef integration: http://goo.gl/sn1OtT

configuration management tools such as Chef knifeplugins26 to cover infrastructure management aspects,so these two management disciplines can be integratedseamlessly. To reduce dependencies on providers andtools, standards are emerging in all categories. Some ex-amples are Cloud Infrastructure Management Interface(CIMI) [36] on the level of infrastructure management,OVF for VM images (hypervisor-based virtualization),and TOSCA for model-based management.

4 PLAN-BASED DEPLOYMENT AUTOMATION

In the previous section we discussed plan-based con-figuration management as an operations automationapproach, and as an enabler for several other ap-proaches to implement the automated deployment ofmiddleware and application components. For instance,plans such as scripts are attached to topology modelsof applications in order to make them deployable.Furthermore, plans are used to install and configuresoftware components hosted on virtual machinesand containers. Platform-centric management environ-ments based on PaaS frameworks can be set up andoperated based on such plans. Thus, in the followingwe focus on characterizing, refining, and evaluating plan-based approaches to implement deployment automation asa key part of operations automation.

Taking a look at the state of the art, the DevOpscommunity focuses on providing pragmatic solutionsfor the automation of application deployment. Tech-nically, this is often (immediately or implicitly) basedon plan-based configuration management approaches,which drive the following discussions and evaluationpresented in this article. The focus is on deployingpredefined application stacks across several (virtualor physical) machines. Reusability only occurs whensimilar application stacks are being deployed by thesame teams of people. Such a deployment can becharacterized as application-oriented. The communitiesaffiliated with some of the popular DevOps tools suchas Chef [26] or Puppet [28] provide artifacts suchas Chef cookbooks27 to build deployment plans forcertain application stacks. Deployment plans based onthese artifacts can be used to automate the deploymentof the applications described in Section 2.1. Suchplans can be implemented as Chef cookbooks by (i)orchestrating existing cookbooks that are already avail-able and (ii) implementing some application-specificdeployment logic. Existing cookbooks are typicallyavailable to deploy popular middleware componentssuch as an Apache HTTP server or a PHP runtimeenvironment. Consequently, this deployment approachcan be summarized as follows:

Application-oriented Deployment: Application-specific but portable deployment plans enable thedeployment of a particular application including all

26. Chef knife plugins: https://docs.chef.io/plugin knife.html27. Chef supermarket: https://supermarket.chef.io

Page 7: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

7

application components and middleware componentsinvolved.

The portability of deployment plans enables in-frastructure abstraction [6], meaning that the plansare not bound to a specific infrastructure such as aparticular XaaS provider. As an example, most of thecookbooks provided by the Chef community realizeinfrastructure abstraction because Chef cookbooksare implemented using a domain-specific languagethat is not bound to a specific platform. Deploymentplans that follow the application-oriented deploymentapproach are typically limited in their reuse becausethey were created to automate the deployment ofa specific application. In addition, the deploymentplans involved are typically hard-wired, i.e., theyhave explicit dependencies that cannot be exchangeddynamically without changing the plans themselves.To provide an approach enabling the creation ofdeployment plans with improved reusability, in [6] weproposed the middleware-oriented deployment approach:

Middleware-oriented Deployment: Generic andreusable middleware components that are not bound toa specific application enable the deployment of Cloudapplications including (i) the middleware functionalityrequired by the application and (ii) the applicationcomponents involved.

In this case, application deployment is performedby parameterizing and executing portable deploymentplans that are attached only to the middleware compo-nents. We assume that middleware components are notbound to specific applications, so these middlewarecomponents including their deployment plans can bereused to deploy different applications of the sametype. There are no deployment plans attached toany application component when following the puremiddleware-oriented deployment approach. Conse-quently, an overarching orchestration model or systemto wire multiple deployment plans as it is typicallyrequired for non-trivial application stacks can purelyfocus on the configuration of reusable plans. This is incontrast to developing individual application-specificplans and corresponding orchestration models, whichare hard to reuse.

In case the application components are not boundto specific middleware components, this approachenables middleware abstraction. Consequently, particularmiddleware components can be arbitrarily exchanged,e.g., based on functional or non-functional require-ments. As an example, [6] shows how the JOnASapplication server in the Taxi App stack can beexchanged by a Tomcat servlet container. Because theTomcat servlet container consumes less memory andgets deployed faster, it could be typically used ina development environment instead of deploying acomplete application server such as JOnAS.

The Chef community has published cookbooks thatcan be classified under the middleware-oriented de-

ployment approach such as the application_php28

cookbook to deploy arbitrary applications or appli-cation components implemented in PHP. However,these cookbooks do not enable middleware abstrac-tion because they contain hard-wired dependenciesto other middleware components. For example, theapplication_php cookbook mentioned before hasa hard-wired dependency to the Apache HTTP serveras its underlying middleware. Consequently, thismiddleware component that provides a PHP run-time environment cannot be exchanged dynamicallywithout changing the cookbook. In addition, thesecookbooks cannot be used as deployment plans thatcan be parameterized. Usually, they provide resourcesthat can be used in other cookbooks. A developerneeds to create an additional deployment plan thatimplements the configuration and wiring of theseresources. Consequently, these cookbooks cannot beimmediately used as middleware-oriented deploymentplans as they are.

In the following we evaluate these types of ap-proaches using the applications presented in Section 2.1as the means to identify which approach fits betterwhich application.

5 EVALUATION

For evaluation purposes we implemented deploymentplans to automate the deployment of all three ap-plications described in Section 2.1 using both theapplication-oriented and the middleware-oriented ap-proach. This results in six different deployment scenar-ios (three applications multiplied by two approaches).In addition, we deploy the SugarCRM application intwo manners (centralized in one VM, and distributedacross different VMs), again using both the application-oriented and the middleware-oriented deployment ap-proach to further broaden the scope of our evaluation.This adds two additional deployment scenarios, so ourevaluation covers eight deployment scenarios in total.

Table 1 provides an overview of all deploymentplans involved in the scenarios. Technically, all plansare implemented as Chef cookbooks. However, wedo not rely on any unique feature of Chef, so thedeployment plans can be implemented using differentdeployment automation tools such as Puppet, Juju29,OpenTOSCA30, and other plan-based managementapproaches [37]. Each plan is characterized by its type:middleware means that the plan deploys one or moremiddleware components. Plans of type middleware &app generic are typically used for the middleware-oriented deployment approach to deploy middlewarecomponents as well as application components ontop of them as discussed in [6]. App specific plans aremostly used for the application-oriented deployment

28. https://supermarket.chef.io/cookbooks/application php29. Juju: http://juju.ubuntu.com30. OpenTOSCA: http://www.iaas.uni-stuttgart.de/OpenTOSCA

Page 8: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

8

Taxi App SugarCRM Legend

Middleware Plan

App Specific Plan

Explicitly depends on

Implicitly depends on

Apache_

HTTP_ServerESBMT

Chat App

Orchestra

App_ESB_

Components

PostgreSQL_

DBs

App_BPEL_

Processes

App_Tenant

PostgreSQL JOnASPHP_

Runtime_Env

MySQL

Node.js

Redis

App_Helper_

Services

SugarCRM_

App

SugarCRM_

DB

Connect_

App_to_DB

Chat_

App

Figure 5. Plan Dependencies for Application-orientedDeployment

Taxi App SugarCRM Legend

Middleware / App Generic Plan

App Specific Plan

Explicitly depends on

Implicitly depends on

Apache_

HTTP_ServerESBMT

Chat App

Orchestra

PostgreSQL JOnASPHP_

Runtime_Env

MySQL

Node.js

Redis

Connect_

App_to_DB

Figure 6. Plan Dependencies for Middleware-orientedDeployment

approach because they implement specific logic toautomate the deployment of particular applicationcomponents. Furthermore, the input parameters (Chefattributes) for each plan are identified. Figure 5 andFigure 6 show the dependencies between plans whenfollowing the application-oriented or the middleware-oriented deployment.

All deployments have been performed using two dif-ferent Cloud infrastructures: FlexiScale31 and AmazonWeb Services EC232, in order to ensure independencefrom a specific provider. Table 2 shows the evaluationsettings in detail: the Taxi App has been deployedon Ubuntu Linux 10.04 Server (64-bit) on a virtualmachine providing 2 CPU cores and 4 GB (FlexiScale)or 7.5 GB (AWS EC2) of RAM. Both SugarCRM andthe Chat App have been deployed on Ubuntu Linux12.04 Server (64-bit) based on 1 CPU core and 1GB (FlexiScale) or 1.7 GB (AWS EC2) of RAM. Forthe distributed SugarCRM deployment, two virtualmachines were used: one for the database and anotherone for the application itself. Each machine has 1 CPUcore and 1 GB (FlexiScale) or 1.7 GB (AWS EC2) ofRAM. The deployment of the database was runningin parallel to the deployment of the actual application.Then, these two machines were dynamically wired at

31. FlexiScale: http://www.flexiscale.com32. AWS EC2: http://aws.amazon.com/ec2

deployment time by exchanging the database endpointinformation using an AWS S3 bucket33. In all caseswe are using hypervisor-based VMs with a plainoperating system running the plans (Chef cookbooks)on top of it, i.e. no pre-configured VM images are used.Consequently, we are clearly following the plan-basedconfiguration management approach as discussed inSection 3.

For each deployment scenario, five measurementsare performed in total. These include three qualitativeand three quantitative measurements.

5.1 Qualitative Measurements

First, we measure the following three qualitativeproperties for each deployment plan. For all threeproperties we use an ordinal Low/Medium/High scale,from worst to best:

• Flexibility: This property expresses the degree ofcustomizability by configuring a particular de-ployment plan using input parameters at runtime.Measurable degrees:

Low No input parameters, i.e., no dynamicsat runtime.

Med Configuration options for predefinedcomponents, e.g., database credentials.

High Dynamic processing of arbitrary applica-tion components of a particular type.

• Assumption Freedom: This refers to the numberof assumptions made regarding the input of aparticular deployment plan. Obviously this canonly be measured in case the plan has any inputparameters at all. Measurable degrees:

Low Application-specific assumptions, e.g., atenant ID needs to be written into a WARfile (application component) either by thedeployment plan itself or by some kindof preprocessor.

Med Common assumptions for the correspond-ing type of application component, e.g.,a Node.js application typically owns apackage.json file that specifies its de-pendencies.

High No specific assumptions.• Reusability: The following degrees specify the

reusability of a particular deployment plan:Low No reusability, i.e., the plan was specifi-

cally created for a particular application.Med The plan can be used to deploy reusable

but fixed components such as middle-ware components that are required indifferent application stacks.

High The plan can be used to deploy arbitraryapplication components of a particulartype.

33. AWS S3: http://aws.amazon.com/s3

Page 9: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

9

Table 1Types and Input Parameters of Deployment Plans

Plan Type Input Parameters

Application-oriented deployment of Taxi App:ESBMT Middleware DB credentialsPostgreSQL Middleware DB credentialsJOnAS Middleware JOnAS configurationOrchestra Middleware noneApp_Helper_Services App Specific noneApp_ESB_Components App Specific nonePostgreSQL_DBs App Specific DB credentialsApp_BPEL_Processes App Specific noneApp_Tenant App Specific URLs of WAR files (taxi driver GUI, customer GUI), URL of SoapUI test suite

Middleware-oriented deployment of Taxi App:ESBMT Middleware & App Generic DB credentials, URLs of SoapUI test suitesPostgreSQL Middleware & App Generic DB credentials, DB specificationsJOnAS Middleware & App Generic JOnAS configuration, URLs of preprocessed WAR files (taxi driver GUI,

customer GUI)Orchestra Middleware & App Generic URLs of BPEL processes

Application-oriented deployment of SugarCRM:Apache_HTTP_Server Middleware Apache configurationPHP_Runtime_Env Middleware noneMySQL Middleware noneSugarCRM_DB App Specific DB credentialsSugarCRM_App App Specific noneConnect_App_to_DB App Specific DB credentials

Middleware-oriented deployment of SugarCRM:Apache_HTTP_Server Middleware & App Generic Apache configuration, URL of ZIP file (SugarCRM PHP scripts), permission

informationPHP_Runtime_Env Middleware & App Generic noneMySQL Middleware & App Generic DB credentialsConnect_App_to_DB App Specific DB credentials, SugarCRM admin password

Application-oriented deployment of Chat App:Node.js Middleware noneRedis Middleware noneChat_App App Specific none

Middleware-oriented deployment of Chat App:Node.js Middleware & App Generic URL of ZIP file (Chat App scripts)Redis Middleware none

Table 3 shows the measured degrees for the prop-erties flexibility, assumption freedom, and reusability foreach deployment plan.

5.2 Quantitative Measurements

In addition to the qualitative measurements, we alsomeasure the following three quantitative properties foreach deployment scenario:

• Total Complexity: This property expresses the num-ber of “atomic actions”, i.e., Chef resources34

executed during deployment.

34. Chef resources: https://docs.chef.io/resource.html

• Total Number of Plan Dependencies: This is the totalnumber of dependencies between plans for aparticular deployment scenario.

• Total Execution Time: This is the total time requiredfor the execution of all deployment plans involvedin a single scenario, measured in seconds.

Table 4 shows the measured total complexity ofeach deployment scenario, i.e., the number of Chefresources executed at deployment time for a particulardeployment scenario. Moreover, Figure 5 and Figure 6compare the total number of plan dependencies foreach deployed application. Table 5 outlines the averageexecution time in total for each deployment scenario.The average time is based on five deployment runs

Page 10: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

10

Table 2Evaluation Settings

Application Machine Type #VMs

FlexiScale:Taxi App 2 CPU cores, 4 GB memory 1SugarCRM 1 CPU core, 1 GB memory 1SugarCRM (distrib.) 1 CPU core, 1 GB memory 2Chat App 1 CPU core, 1 GB memory 1

Amazon Web Services EC2:Taxi App 2 CPU cores, 7.5 GB memory 1

(m1.large)SugarCRM 1 CPU core, 1.7 GB memory 1

(m1.small)SugarCRM (distrib.) 1 CPU core, 1.7 GB memory 2

(m1.small)Chat App 1 CPU core, 1.7 GB memory 1

(m1.small)

per scenario.

5.3 Reusability of Plans in Practice

In order to evaluate the reusability of deploymentplans in practice, we reused the plans created forSugarCRM to deploy another PHP-based application,namely WordPress35. The deployment requirementsof WordPress are similar to those of SugarCRM. Incase of middleware-oriented deployment we onlyhad to adapt the Connect_App_to_DB plan be-cause the configuration file format for storing thedatabase endpoint information is different. However,in case of application-oriented deployment two newplans had to be created based on the SugarCRM_DBand SugurCRM_App plans in addition. These areWordPress_DB and WordPress_App to deploy theWordPress application itself as well as the database. Insummary, three plans were reused, one plan was mod-ified, and no additional plan was implemented for themiddleware-oriented deployment; for the application-oriented deployment, three plans were reused, oneplan was modified, and two plans were additionallyimplemented.

6 DISCUSSION

Section 5 described the process and results of ourevaluation based on eight deployment scenarios. Thissection discusses the evaluation results and presentsfindings as well as lessons learned. In our previouswork [6] we evaluated and discussed the impact ofusing the middleware-oriented deployment approachin order to reduce the number of deployment plans forthe Taxi App. The evaluation presented in this paperbroadens the horizon by looking beyond the number

35. WordPress: http://www.wordpress.org

Table 3Qualitative Measurements of Deployment Plans

Plan Flexi- Assum. Re-bility Freedom usability

Application-oriented deployment of Taxi App:ESBMT Med High MedPostgreSQL Med High MedJOnAS Med High MedOrchestra Low – MedApp_Helper_Services Low – LowApp_ESB_Components Low – LowPostgreSQL_DBs Med High LowApp_BPEL_Processes Low – LowApp_Tenant High Low Low

Middleware-oriented deployment of Taxi App:ESBMT High High HighPostgreSQL High High HighJOnAS High Low HighOrchestra High High High

Application-oriented deployment of SugarCRM:Apache_HTTP_Server Med High MedPHP_Runtime_Env Low – MedMySQL Med – MedSugarCRM_DB Med High LowSugarCRM_App Low – LowConnect_App_to_DB Med High Low

Middleware-oriented deployment of SugarCRM:Apache_HTTP_Server High Low HighPHP_Runtime_Env Low – MedMySQL High High HighConnect_App_to_DB Med High Low

Application-oriented deployment of Chat App:Node.js Med – MedRedis Med – MedChat_App Low – Low

Middleware-oriented deployment of Chat App:Node.js High Med HighRedis Med – Med

Table 4Total Complexity of Each Scenarios

Application App.-oriented Middleware-oriented

Taxi App 139 139SugarCRM 57 57SugarCRM (distrib.) 57 57Chat App 16 16

Page 11: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

11

Table 5Average Total Execution Time λ of Deployment Plans

and Standard Deviation σ

Application App.-oriented Middleware-oriented

FlexiScale:Taxi App λ = 899 sec. λ = 937 sec.

σ = 25 sec. σ = 45 sec.SugarCRM λ = 264 sec. λ = 262 sec.

σ = 9 sec. σ = 13 sec.SugarCRM (distrib.) λ = 184 sec. λ = 180 sec.

σ = 4 sec. σ = 10 sec.Chat App λ = 219 sec. λ = 228 sec.

σ = 6 sec. σ = 11 sec.

Amazon Web Services EC2:Taxi App λ = 1316 sec. λ = 1294 sec.

σ = 53 sec. σ = 37 sec.SugarCRM λ = 372 sec. λ = 358 sec.

σ = 47 sec. σ = 36 sec.SugarCRM (distrib.) λ = 233 sec. λ = 235 sec.

σ = 9 sec. σ = 16 sec.Chat App λ = 314 sec. λ = 325 sec.

σ = 7 sec. σ = 18 sec.

of plans involved for the deployment of a singleapplication. Therefore, in the following we analyzethe measurements reported in the previous section.

6.1 Findings

One of the first findings that can be derived fromthe evaluation results shown in Table 3 is that themiddleware-oriented deployment approach generallyimproves the reusability of deployment plans. Fur-thermore, our measurements verify the observationsreported in [6] that the number of deployment plansdecreases when using the middleware-oriented de-ployment approach in general. This is due to thefact that application-specific actions are covered byparameterizing generic deployment plans attached tomiddleware components.

However, the pure middleware-oriented deploymentapproach cannot be implemented for all deploymentscenarios. Middleware-oriented deployment is typi-cally implemented based on plans of type middlewareand middleware & app generic. Because app specificplans are specifically created for particular applicationcomponents they should be avoided when follow-ing the middleware-oriented approach. In case ofimplementing a middleware-oriented deployment ofSugarCRM, for example, Table 3 shows that thereis a plan to wire the application with the database(Connect_App_to_DB). This plan is of type appspecific because it cannot be implemented in a genericmanner. As discussed in Section 2.2, the requirementsSR2 and SR3 require very application-specific actions

to be performed. These are implemented using thewiring plan.

In case generic deployment plans for deployingapplication components are attached to middlewarecomponents, typically assumptions are made before-hand regarding the application components deployedusing these plans. For instance, the Node.js plan usedin the middleware-oriented deployment expects (i) apackage.json file as described in the requirementCR1 and (ii) a pointer to the script that is the entrypoint for a particular application (CR2). Anotherexample is the JOnAS plan to deploy additional tenants(WAR files) for the Taxi App (TR1, TR3).

The flexibility of deployment plans following theapplication-oriented deployment approach is, as ex-pected, worse compared to middleware-oriented de-ployment because their implementation is tightly cou-pled to specific application components. They typicallyexpect a few rudimentary input parameters only suchas configuration options for the middleware.

When looking at Table 4 it becomes clear that thecomplexity of deploying a particular application isequal in both cases, and therefore it does not dependon the deployment approach chosen. This is becausethere is no difference on the level of “atomic actions”,i.e., on the level of Chef resources that are executed atdeployment time such as creating a particular directory,storing a configuration file, or installing a softwarepackage. However, Figure 5 and Figure 6 denote adecreased number of plans and plan dependenciesfor all deployed applications in case the middleware-oriented deployment approach is used.

Table 5 shows minimal differences betweenapplication-oriented and middleware-oriented deploy-ment for the average deployment time of each scenario.These measurements match the complexity measure-ments shown in Table 4: because the Chef resourcesexecuted at deployment time are the same, it is onlylogical that there are no significant differences interms of the plans’ total execution time. The minordifferences are due to the fact that files (middlewareand application components) are downloaded fromthe Web during deployment. Because the quality ofthe underlying HTTP connections for these downloadscould differ, there are small deviations.

Our further evaluation of reusability of plansin practice based on SugarCRM and WordPress(Section 5.3) confirms our previous conclusion:middleware-oriented deployment clearly improvesreusability of plans. Moreover, the number of plansthat need to be adapted for specific applications canbe significantly reduced.

6.2 Lessons LearnedBased on the findings presented in the previous section,we now summarize several of the lessons learned tosupport the decision which deployment approach fitswhich type of application:

Page 12: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

12

L1 – Middleware-oriented deployment plans are typicallypreferred for scenarios where conventions for applicationcomponents are already established, so there are noor minimum assumptions regarding the applicationcomponents. Examples for such conventions are thepackage.json file (requirement CR1 in Section 2.2),setting permissions for PHP applications (SR1), orsending SOAP messages using SoapUI test suites(TR2). From this perspective, middleware-orienteddeployment is preferred for the Chat App because theapplication-specific deployment requirements CR1 andCR2 can be transferred to other Node.js applications.

L2 – There are application-specific requirements thatcannot be generally transferred to other applications ofthe same type, so the implementation of middleware-oriented deployment for the Taxi App and Sugar-CRM is not as straightforward as it is for the ChatApp. For SugarCRM the Connect_App_to_DB planprevents the implementation of pure middleware-oriented deployment because application-specific de-ployment requirements (SR2, SR3) need to be fulfilled.These cannot be transferred to other PHP applications(e.g., WordPress) in general because the configura-tion of each PHP application is different. For theTaxi App, middleware-oriented deployment can berealized. However, to fulfill requirement TR3, WARfiles that are deployed using the JOnAS plan needto be preprocessed before deployment: the tenantID and the tenant endpoint are stored inside theWAR files. Furthermore, TR1 cannot be fulfilled bya single deployment plan because this is application-specific knowledge. To deploy additional tenants boththe JOnAS and the ESBMT plans have to be used incombination with certain parameters.

L3 – In case of distributed deployments, such as wedid for SugarCRM, the wiring logic can be implementedin a middleware-oriented manner only if it does not needany application-specific knowledge of how and where tostore the endpoint information. Typically, wiring plans areapplication-specific such as the Connect_App_to_DBplan in case of deploying SugarCRM. This is becausemost of the times endpoint information needs to bestored in application-specific configuration files.

L4 – Even if the middleware-oriented deployment ap-proach cannot be implemented completely, a hybrid approachcan be realized. In this case as many deployment plansas possible are implemented in a middleware-orientedmanner to improve flexibility and reusability. However,application-specific deployment actions are performedusing plans that follow the application-oriented de-ployment approach. Examples for such actions aredeploying additional tenants for the Taxi App (TR1)or wiring the SugarCRM application with the database(SR3). We followed this approach implicitly whenwe implemented middleware-oriented deployment forSugarCRM because it is impossible to implement thelogic of the application-specific Connect_App_to_DBplan in a generic manner.

L5 – The usage of middleware-oriented deployment plansinstead of application-oriented ones does not affect the totalcomplexity of a deployment scenario or the total executiontime. Consequently, performance aspects do not have tobe considered. Nonetheless, the total number of plansand plan dependencies can be reduced by followingthe middleware-oriented approach.

L6 – However, the development of plans to realizemiddleware-oriented deployment might be more complex forthe plan developers. As shown in Table 3 these planscan be parameterized using sets of input parameters.These parameters imply more dynamics in the plans’implementation increasing the complexity of the de-velopment. The gain of such an investment is a higherdegree of flexibility and reusability as shown in Table 3.

6.3 Conventions for Application Components

As mentioned in lesson L1, middleware-oriented de-ployment is typically preferred if common conventionsfor application components are established. Beside theconventions for Node.js applications (CR1) as wellas PHP applications (SR1) discussed before, there arefurther conventions that are also implemented by PaaSproviders such as Heroku or Google App Engine:

• For Java applications it is common to express alldependencies of a particular application compo-nent using a pom.xml file. These dependenciesare resolved using Maven36 at deployment time.Furthermore, a system.properties37 file canbe used to define the Java version required by aparticular application component.

• In case the Java application is built using the Playframework38, dependencies are typically definedusing a dependencies.yml file instead of apom.xml file.

• For Scala39 applications a build.propertiesfile is typically used to specify dependencies. Thisfile gets processed using sbt40 at deployment time.

• Ruby-based applications commonly make use ofBundler41 to process Gemfiles. A Gemfile isused to define both the dependencies and theRuby version to be used for a certain applicationcomponent.

• Applications based on Python usually utilizepip42 for dependency management. Therefore, arequirements.txt file must be created to pointto other Python packages.

Moreover, PaaS providers implement mechanismsto expose endpoint information such as the IP addressof the database instance using context or environment

36. Apache Maven: http://maven.apache.org37. Java system properties: http://goo.gl/ewo78l38. Play framework: http://www.playframework.com39. Scala: http://www.scala-lang.org40. sbt: http://www.scala-sbt.org41. Bundler: http://bundler.io42. pip: http://www.pip-installer.org

Page 13: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

13

variables. These variables can be read, for instance, in ascript-based configuration file (PHP, Python, Ruby, etc.)to connect application components to the database.

7 CONCLUSIONS AND FUTURE WORK

The automated provisioning and deployment of appli-cations on IaaS and PaaS solutions is one of the majorenablers in the reduction of the operational costs bymigrating to the Cloud. Tooling and approaches mostlyfrom the DevOps community have provided the meansfor such an automation through deployment plans.However, these approaches focus on the deploymentof individual, specific application stacks at a time,sacrificing reusability for efficiency and ease in thedevelopment of such deployment plans. For thisreason, in previous work [6], and approaching theproblem from a PaaS offering perspective, we pro-posed a middleware-oriented deployment approachthat promotes reusability of deployment plans acrossdifferent applications.

In this work, we expand on previous work to iden-tify and characterize two different types of deploymentapproaches (application- and middleware-oriented)based on a systematic classification of existing opera-tions automation approaches presented in the literatureand available in the market. We developed deploymentplans for three applications (Taxi App, SugarCRM, andChat App) with significantly different deploymentrequirements using the identified approaches, andwe evaluated the results across both qualitative andquantitative dimensions. Our findings show betterreusability, portability, and flexibility of middleware-oriented plans when compared to application-orientedones, without a loss in performance (i.e., deploymenttime). The results of our evaluation are independent ofa particular Cloud provider. The trade-off however forthis improvement is in the difficulty of creating suchplans. In order to demonstrate the reusability potentialof such plans in practice we reused the plans createdfor SugarCRM to deploy WordPress in an application-and middleware-oriented manner. WordPress has asimilar application stack and comparable deploymentrequirements. Finally, based on what we derivedfrom this evaluation we provide recommendationsas lessons learned with respect to deciding whichapproach to use when deploying an application.

In terms of future work, we plan to extend ourevaluation to cover even more application stacks basedon different technologies such as Ruby on Rails orDjango based on Python. Moreover, we aim to broadenthe scope of the evaluation by considering additionalaspects such as the number of code changes appliedto deployment plans over time and training costs fordevelopers and operations personnel. Based on thisadditional evaluation, further findings and lessonslearned can be derived, in addition to verifying orfalsifying the existing ones. Furthermore, as discussed

in the introduction, the overall goal of this work is toprovide a decision support system for deployment ofapplications in the Cloud. Toward this goal, a decisionsupport matrix based on the lessons learned from thiswork is currently under development. The immediategoal of this matrix is to provide the systematic meansfor decisions related to the creation of deploymentplans, as well as how to use and combine them to au-tomate the deployment of a particular application stack.Based on such a matrix, a decision support systemprototype can then be implemented. In this context,existing deployment plans such as cookbooks providedby the Chef community can be linked and proposedto the person using the decision support system. Thedecision support system will not only cover plan-based configuration management approaches, but alsofurther operations automation approaches as shownin our classification. Moreover, we plan to focus onproviding decision support on how multiple opera-tions automation approaches can be combined. Forinstance, the combination of container virtualizationand plan-based configuration management has also tobe investigated.

ACKNOWLEDGMENTS

The research leading to these results has partiallyreceived funding from the EU’s Seventh FrameworkProgramme (FP7/2007-2013) projects 4CaaSt (grantagreement no. 258862), ALLOW Ensembles (grantagreement no. 600792), and from the BMBF projectECHO (01XZ13023G).

REFERENCES[1] P. Mell and T. Grance, “The NIST Definition of Cloud Com-

puting,” 2009.[2] J. Humble and J. Molesky, “Why Enterprises Must Adopt

Devops to Enable Continuous Delivery,” Cutter IT Journal,vol. 24, 2011.

[3] M. Huttermann, DevOps for Developers. Apress, 2012.[4] J. Humble and D. Farley, Continuous Delivery: Reliable Soft-

ware Releases through Build, Test, and Deployment Automation.Addison-Wesley Professional, 2010.

[5] L. Chen, “Continuous Delivery: Huge Benefits, But ChallengesToo,” Software, IEEE, vol. PP, no. 99, 2015.

[6] J. Wettinger, V. Andrikopoulos, S. Strauch, and F. Leymann,“Enabling Dynamic Deployment of Cloud Applications Usinga Modular and Extensible PaaS Environment,” in Proceedingsof IEEE CLOUD 2013. IEEE Computer Society, pp. 478–485.

[7] ——, “Characterizing and Evaluating Different DeploymentApproaches for Cloud Applications,” in Proceedings of the IEEEIC2E 2014. IEEE Computer Society, 2014.

[8] S. Garcıa-Gomez, M. Jimenez-Ganan, Y. Taher, C. Momm,F. Junker, J. Bıro, A. Menychtas, V. Andrikopoulos, andS. Strauch, “Challenges for the Comprehensive Managementof Cloud Services in a PaaS Framework,” Scalable Computing:Practice and Experience (SCPE), vol. 13, no. 3, pp. 201–213, 2012.

[9] OASIS, “Web Services Business Process Execution Language(BPEL) Version 2.0,” 2007.

[10] Google, Inc., “Google Maps API Web Services.”[Online]. Available: http://developers.google.com/maps/documentation/webservices

[11] S. Strauch, V. Andrikopoulos, S. Gomez Saez, and F. Leymann,“ESBMT: A Multi-tenant Aware Enterprise Service Bus,” Inter-national Journal of Next-Generation Computing, vol. 4, no. 3, pp.230–249, 2013.

Page 14: Middleware-oriented Deployment Automation for Cloud ... · open source Enterprise Service Bus ESBMT [11] as mes- saging middleware (Figure 1) enables loose coupling and provides a

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for moreinformation.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2535325,IEEE Transactions on Cloud Computing

14

[12] ——, “Implementation and Evaluation of a Multi-tenant Open-Source ESB,” in Proceedings of ESOCC 2013, ser. Lecture Notesin Computer Science, vol. 8135. Springer, 2013, pp. 79–93.

[13] OASIS, “Topology and Orchestration Specification for CloudApplications (TOSCA) Version 1.0, Committee Specification01,” 2013. [Online]. Available: http://docs.oasis-open.org/tosca/TOSCA/v1.0/cs01/TOSCA-v1.0-cs01.html

[14] J. Wettinger, M. Behrendt, T. Binz, U. Breitenbucher, G. Breiter,F. Leymann, S. Moser, I. Schwertle, and T. Spatzier, “IntegratingConfiguration Management with Model-Driven Cloud Man-agement Based on TOSCA,” in Proceedings of CLOSER 2013.

[15] J. Turnbull, The Docker Book. James Turnbull, 2014.[16] T. Binz, G. Breiter, F. Leymann, and T. Spatzier, “Portable Cloud

Services Using TOSCA,” Internet Computing, IEEE, vol. 16, no. 3,pp. 80–85, 2012.

[17] D. Sanderson, Programming Google App Engine. O’Reilly Media,2009.

[18] J. Vliet, F. Paganelli, S. Wel, and D. Dowd, Elastic Beanstalk.O’Reilly Media, 2011.

[19] T. Rosner, Learning AWS OpsWorks, 2013.[20] M. Papazoglou and W. van den Heuvel, “Blueprinting the

Cloud,” Internet Computing, IEEE, vol. 15, no. 6, pp. 74–79,2011.

[21] K. Pepple, Deploying OpenStack. O’Reilly Media, 2011.[22] DMTF, “Open Virtualization Format Specification (OVF)

Version 2.0.0,” 2013. [Online]. Available: http://www.dmtf.org/standards/ovf

[23] P. Pawluk, B. Simmons, M. Smit, M. Litoiu, and S. Mankovski,“Introducing STRATOS: A Cloud Broker Service,” in Proceedingsof IEEE CLOUD 2012. IEEE Computer Society, pp. 891–898.

[24] S. Zaman and D. Grosu, “An Online Mechanism for DynamicVM Provisioning and Allocation in Clouds,” in Proceedings ofIEEE CLOUD 2012. IEEE Computer Society, pp. 253–260.

[25] M. Bjorkqvist, L. Chen, and W. Binder, “Opportunistic ServiceProvisioning in the Cloud,” in Proceedings of IEEE CLOUD 2012.IEEE Computer Society, pp. 237–244.

[26] S. Nelson-Smith, Test-Driven Infrastructure with Chef. O’ReillyMedia, Inc., 2011.

[27] M. Taylor and S. Vargo, Learning Chef: A Guide to ConfigurationManagement and Automation. O’Reilly Media, 2014.

[28] J. Loope, Managing Infrastructure with Puppet. O’Reilly Media,Inc., 2011.

[29] T. Uphill, Mastering Puppet. Packt Publishing Ltd, 2014.[30] D. Zamboni, Learning CFEngine 3: Automated System Adminis-

tration for Sites of Any Size. O’Reilly Media, Inc., 2012.[31] P. Goldsack, J. Guijarro, S. Loughran, A. Coles, A. Farrell,

A. Lain, P. Murray, and P. Toft, “The SmartFrog ConfigurationManagement Framework,” ACM SIGOPS Operating SystemsReview, vol. 43, pp. 16–25, 2009.

[32] J. Fischer, R. Majumdar, and S. Esmaeilsabzali, “Engage: ADeployment Management System,” SIGPLAN Not., vol. 47,no. 6, pp. 263–274, 2012.

[33] S. Gunther, M. Haupt, and M. Splieth, “Utilizing InternalDomain-Specific Languages for Deployment and Maintenanceof IT Infrastructures,” Very Large Business ApplicationsLab Magdeburg, Fakultat fur Informatik, Otto-von-Guericke-Universitat Magdeburg, Tech. Rep., 2010.

[34] OMG, “Business Process Model and Notation (BPMN) Version2.0,” 2011.

[35] enStratus Networks, Inc., “Cloud DevOps: Achieving AgilityThroughout the Application Lifecycle,” 2012.

[36] D. Davis and G. Pilz, “Cloud Infrastructure ManagementInterface (CIMI) Model and RESTful HTTP-based Protocol,”Technical report, Distributed Management Work Force (DMTF),Tech. Rep., 2012.

[37] U. Breitenbucher, T. Binz, O. Kopp, and F. Leymann, “Pattern-Based Runtime Management of Composite Cloud Applications,”in Proceedings of CLOSER 2013.

All links were last followed on February 15, 2016.

Johannes Wettinger is a research associateand PhD student at the Institute of Architec-ture of Application Systems (IAAS) at theUniversity of Stuttgart. His research focuseson developing and operating continuouslydelivered software applications, addressingDevOps collaboration and deployment au-tomation. Beside his participation in the EU-funded project 4CaaSt and his teaching activ-ities, Johannes is involved in further projectsrelated to Cloud computing.

Vasilios Andrikopoulos is a senior re-searcher at the Institute of Architecture ofApplication Systems (IAAS) at the Universityof Stuttgart. His research is in the areasof services science, cloud computing andinfrastructures, and software engineering withan emphasis on evolution and adaptation. Hereceived his PhD from Tilburg University, theNetherlands, where he was also a member ofthe European Research Institute in ServiceScience (ERISS). He has experience in re-

search and teaching Database Systems and Management, SoftwareModeling and Programming, Business Process Management andIntegration, and Service Engineering. He has participated in a numberof EU projects, including the NoE S-Cube and 4CaaSt.

Frank Leymann is a full professor of com-puter science and director of the Institute ofArchitecture of Application Systems (IAAS) atthe University of Stuttgart. He has publishedmany papers in journals and proceedings, co-authored four text books, and holds close to50 patents especially in the area of workflowmanagement and transaction processing. Hisresearch interests include service orientedcomputing and middleware, workflow- andbusiness process management, Cloud Com-

puting, transaction processing, integration technology, and architec-ture patterns. He is co-author of many Web Service specifications,including WSFL, WS-Addressing, WS-Metadata Exchange, WS-Business Agreement, the WS-Resource Framework set of specifi-cations, WS-HumanTask and BPEL4People; together with SatishThatte, he was the driving force behind BPEL4WS. Also, he is co-author of BPMN 2.0 and TOSCA. His background includes havingbeen IBM Distinguished Engineer and elected member of the IBMAcademy of Technology.

Steve Strauch is as a research associateand PhD student at the Institute of Archi-tecture of Application Systems (IAAS) at theUniversity of Stuttgart. His research interestsare data migration, data hosting, as wellas data security and privacy in the area ofCloud Computing, with an emphasis on theirapplication architectural aspects. He has con-tributed to the European projects COMPAS,4CaaSt, ALLOW Ensembles, and the Germangovernment funded BMBF project ECHO.