deliverable d5.4 mdi for service interoperability ... · pdf fileopentravel alliance was...

Copyright REMICS Consortium 2010-2013

REuse and Migration of legacy applications to Interoperable Cloud Services REMICS

Small or Medium-scale Focused Research Project (STREP)

Project No. 257793

Deliverable D5.4

MDI for Service Interoperability Principles and Methods

Work Package 5 Leading partner: SINTEF

Author(s): Brice MORIN (editor), Franck CHAUVEL, Nicolas Ferry

Dissemination level: Public

Delivery Date: 10-09-2013

Final Version: 1.0

Public

Copyright REMICS Consortium 2010-2013 / 32

Versioning and contribution history

Version Description Contributors

0.1 Initial draft (25 June 2013) SINTEF

0.2 Consolidated version (5 July 2013) SINTEF

0.3 Version before first review (21 August 2013) SINTEF

0.4 Update version after first review (28 August 2013) Netfective, SINTEF

0.5 Update version after second review (04 Sept 2013) IICT-BAS, SINTEF

1.0 Final version after approval by technical manager (10 Sept 2013)

SOFT, SINTEF

Public


Executive Summary The deliverable D5.4 (report) presents in details the different approaches for Model-Driven

Interoperability for cloud services, in particular:

• a mediation framework to define and assess data mediation algorithms,

• a domain-specific language to align semantically equivalent but syntactically different protocols at a behavioural level, and which compiles to WP6 models@runtime engine,

• and CloudML to enable the interoperability of a cloud service on multiple cloud providers.

This document also presents lessons learned about Model-Driven Interoperability, based on the experience gathered from the development of those three artefacts.

Public


Table of contents

EXECUTIVE SUMMARY ......................................................................................................................... 3

1 INTRODUCTION ................................................................................................................................... 5 1.1 INTEROPERABILITY IN THE REMICS METHODOLOGY AND TOOL-SUITE ...................................... 6

1.2 INTEROPERABILITY NEEDS IN DOME CASE STUDY ...................................................................... 7

2 FRAMEWORK FOR PROTOTYPING DATA MEDIATION .......................................................... 9 2.1 APPROACH OVERVIEW................................................................................................................. 9

2.2 ARCHITECTURE .......................................................................................................................... 10

2.3 USING THE MEDIATION FRAMEWORK ......................................................................................... 11

2.3.1 Pushing in the model repository ........................................................................................... 12

2.3.2 Triggering an existing mediation ......................................................................................... 13

2.3.3 Creating an ad hoc Mediation Algorithm ............................................................................. 14

2.3.4 Comparing Existing Algorithms .......................................................................................... 15

3 BEHAVIOURAL SERVICE MEDIATION ....................................................................................... 17 3.1 APPROACH OVERVIEW............................................................................................................... 17

3.2 ARCHITECTURE .......................................................................................................................... 18

3.3 USING THE DSL ......................................................................................................................... 20

4 INTEROPERABILITY ACROSS CLOUD PROVIDERS ............................................................... 25 4.1 APPROACH OVERVIEW............................................................................................................... 25

4.2 ARCHITECTURE .......................................................................................................................... 26

4.3 USING CLOUDML ...................................................................................................................... 27

5 LESSONS LEARNED .......................................................................................................................... 28

6 CONCLUSION ...................................................................................................................................... 30

Public


1 Introduction The overall goal of the REMICS project is to recover legacy systems (written in Cobol, PL/SQL etc.) into models, migrate those models to modern languages (e.g., Java) and SOA frameworks, and then to deploy the migrated systems into the cloud ecosystem. The recovery and migration process is described in WP 2, 3 and 4 deliverables. Still, the REMICS process does not terminate with the initial deployment of the migrated system to the cloud. Indeed, when a system has been migrated to the cloud, it is directly competing with a plethora of other applications, which might provide similar services. Its continuous modernization is the key to keep the system competitive over time. Failing in this task would result in a system slowly becoming obsolete, which eventually would need to be migrated again in 10-20 years.

The continuous modernization of the migrated application has impacts on the different layers of the cloud stack, in particular the SaaS layer implementing the business logic of the application. Providing interoperability mechanisms for cloud services is thus a critical concern for the modernized system to evolve in a win-win relationship with other applications deployed in the cloud, as illustrated in Figure 1-1:

• It makes it possible to seamlessly extend or replace some of the migrated services with external services provided on the cloud. These external services might be cheaper, offer better Quality of Service (QoS), etc.

• It enables external applications to seamlessly use the migrated services, for the same reasons.

Figure 1-1 Interoperability issues

In addition to modernizing the business logic of the system, one should be able to modernize the platform and the infrastructure supporting the business logic of the application. For example, it should be possible to update the version of the Java Virtual Machine (JVM), web-server and database, but also to scale resources (CPU, RAM, or storage capacity) up and down according to the needs of the application and the platform, and also according to the desired QoS.

Public


Maintenance and evolution are thus concerns that crosscut the entire application stack, where interoperability plays a major role in order to make it more seamless to replace fragments of software, platform and infrastructure.

1.1 Interoperability in the REMICS methodology and tool-suite Maintenance and evolution is a continuous process, which is initiated after the migration steps (cf. WP4), once the system has been recovered into UML models (cf. WP3). When the legacy system has been migrated to a service-oriented architecture deployed over a cloud infrastructure, it is important to ensure that the system is open and flexible enough in order to ease its evolution and maintenance and avoid its obsolescence. Maintaining the system mostly consists in replacing some services (e.g., a software service, a web-server, or a virtual machine also offered as-a-service) by other services having better properties, such a better reliability or a better response time.

However, the former service and the substitute service, while semantically equivalent, are not necessarily compatible. It requires either:

• To refactor all the other (client) parts that relied on the former fragment, so as to directly rely on the new fragment,

• Or to develop adapters that translate the API of the former fragment into the API of the new fragment, so that other (client) parts can benefit from the new fragment, with no need for any refactoring. This is the mechanism investigated in WP5.

Since the cloud ecosystem is very dynamic (services can appear, disappear or be updated), the interoperability mechanisms should be flexible enough for the cloud. WP5 will thus leverage the models@runtime technologies provided by the CloudML language and engine (jointly developed with WP4), as well as by WP6 to achieve this flexibility and aim for a seamless and continuous interoperability process. An overview of the integration of the (model-driven) interoperability concern (MDI) into the REMICS methodology and tool-suite is provided in Figure 1-2.

Figure 1-2 - Integration with the REMICS tool-suite

Public


1.2 Interoperability needs in DOME case study The DOME application, like most applications, evolves in a complex ecosystem where potentially competing services are already deployed.

There is a wide range of providers of different touristic services (such as hotels, excursions, transfers, etc.) that offer business information about their products to their clients through these interfaces. The clients willing to implement their own online integration (usually in XML) will have the possibility to obtain online business information from their providers (service availabilities and prices, hotel information) and carry out different actions (book, cancel booking, add passengers, confirm services, etc.). To perform the integration, the client must have defined his own interface to communicate with one of the providers.

The growing need of information exchange has also favoured the emergence of new data standards for the electronic exchange of business information among all sectors of the travel industry, such as Open Travel Alliance1 (OTA). Open Travel Alliance was founded in 1999 by travel companies, with a primary focus on creation of electronic message structures to facilitate communication between disparate systems in the global travel industry. Several service providers use this standard.

Due to the growing demand of online integrations in the touristic sector, a better response and adaptation time is needed. This means that the integration process between providers must be carried out in the shortest possible time and with the maximum security and reliability to increment competitiveness in touristic companies. Currently, this interoperability is manually coded by writing XSLT transformations to map (XML-based) data, and by writing the glue-code to synchronize the message exchanges from the DOME system to the new provider. This manual process is time-consuming and error-prone. REMICS, by providing guidance and automation to designers, aims at reducing the time needed to interoperate with a new provider, hence improving the reactivity of DOME.

Figure 1-3 for example illustrates the expected data mapping between the data manipulated by the DOME system, and the OTA standard, in a graphical form. The left part depicts the internal data model (as an object-oriented class diagram) used by DOME to capture travel booking information, whereas the right part shows an excerpt of the equivalent OTA standard. Arrows in between illustrate the model elements that are "semantically" equivalent between these two models. For instance, in the DOME model, the attribute "nameadl" of the class "Distri" represents actually the same concept that the attribute "count" of the class "GuestCount", on the OTA model. Building by hand such mappings is a complex, tedious and error prone tasks, bringing automated, even partially, in such data interoperability issues is very promising for DOME as it will allow interoperating with all the providers and clients that use standards.

1 http://www.opentravel.org/

Public


Figure 1-3- Expected data mappings, as provided by DOME

In addition to the mapping of data, the protocols of the services should also be aligned, so that both APIs can interoperate. Behavioural mapping includes splitting or merging messages, inferring (or asking for user inputs) missing messages, etc.

Public


2 Framework for Prototyping Data Mediation Data interoperability issues are one of the major obstacles towards large scale reuse and migration of large scale software systems (e.g., components, services). One major issue of data interoperability is the "semantic" level that may be difficult to both detect and mitigate: Communicating the price of a product represented as a single number implies for instance to agree on the related currency.

Data interoperability issues have already been extensively investigated by several communities including at least, people working on databases (entity-relationship models), documents (ontologies), and Model-driven Engineering (object-oriented models). The past decades of efforts revealed that there is indeed no silver bullet that solves data interoperability issues2, in a "press-button" fashion: While artificial intelligence may indeed help to automate most of the process, a human agent remains needed to confirm and/or sort out difficult cases. In real cases such as DOME or DISYS, effective solutions result from the selection (or the combination) of relevant existing algorithms, and in the proper pre-processing of data so as to reduce noise and maximise algorithms' effectiveness.

In this setting, the contribution of the REMICS project is a framework, which helps designers quickly prototyping ad hoc mediation algorithms, by reusing and combining well documented mediation algorithms, pre-processors, converters, etc. The key contributions of this framework are:

• A repository of existing mediation algorithms, such as synonyms sets semantic matching, similarity flooding, etc. (Such algorithms are not described in this document, but can easily be found in the related literature.)

• A repository of models on which the effectiveness of various alternative algorithms might be assessed.

• A Domain Specific Language (DSL) for rapid prototyping and execution of new mediation algorithms.

It is worth to note that although the resulting mediations are fully executable and operational, they do not intend to be put in production, especially if they are supposed to be run intensively. In such a case, one should consider building an optimized version of the resulting prototype, leveraging grid support, pipelining techniques, and other advanced mechanisms, ensuring the quality of service expected in a production setting.

2.1 Approach Overview Figure 2-1 below illustrates the main actors and the related uses cases of the REMICS mediation framework. In this case, we distinguish between two main actors: the Mediation User and the Mediation Designer. The mediation user expresses mediation needs (as did several of the REMICS partners), and may also directly try out some of the predefined mediation algorithms built-in in the framework. When such mediation algorithms need to be tailored to fit particular cases, the mediation designer comes into play and can quickly prototype and experiment with new ad hoc algorithms, by combining existing bricks provided by the framework thanks to the mediation DSL. In most complex cases, he might need to add new elementary bricks to the framework (that will be then available for other users as constructs of the DSL).

2 Schema Matching and Mapping in Data-Centric Systems and Applications -- Bellahsene, Zohra; Bonifati, Angela; Rahm, Erhard (Eds.) -- 1st Edition., 2011, XII, 314 p. http://www.springer.com/computer/database+management+%26+information+retrieval/book/978-3-642-16517-7

http://www.springer.com/computer/database+management+%26+information+retrieval/book/978-3-642-16517-7

http://www.springer.com/computer/database+management+%26+information+retrieval/book/978-3-642-16517-7

Public


Figure 2-1 Overview of the approach as a high-level UML 2 use cases diagram

The main use cases covered by the mediation framework are the following:

• Manage Models, is the responsibility of the mediation user. It helps the user in publishing the models he needs to mediate.

• Trigger Mediation gives the ability to the mediation user to select an existing mediation algorithm and to run it on a specific couple of models.

• Validate Mappings gives the ability to the mediation user to "double check" the matches identified between models elements. As there is no algorithm able to match model elements with 100 % accuracy, it is critical that the user confirms and completes the matches that are automatically computed.

• Evaluate Mediation Alternatives: let the user compare the result produced by a given mediation algorithm with an oracle-mapping (that she may have previously provided). This comparison will result in various metrics including (precision, recall, f-measure, etc.) reflecting the effectiveness of the selected algorithm. Such evaluations are also useful for the Mediation Designer who may want to evaluate the effectiveness of new algorithms that have been pushed into the framework.

2.2 Architecture The mediation framework is architected as a service-oriented framework, where each service realizes a specific use case. Figure 2-2 below depicts the overall architecture of the mediation framework, as a UML 2.x component diagram. In this figure, we used components to depict services and stereotypes to explicit the underlying technologies (e.g., REST, Scala, HTTP).

Tree services are devoted to storage, and are labelled as repositories consequently. Storing models, mappings, and evaluations do not only permit to reuse existing data schemas, but also to compare the performance of various mediation algorithms, on well accepted samples.

The mediator and the comparator support the common tasks of the Mediation User. The mediator lets the user select two models in the repository between which it wants to mediate, and

Mediation Framework

Mediation UserMediation Designer

Extend Mediation Library

Manage Models

Trigger Mediation

Ev aluate Mediation Alternativ es

Validate Mapping

Public


populate the mapping repository consequently with the newly found matches between model elements. The comparator service helps users to compare mappings resulting from different algorithms, by providing coverage statistics such as precision, recall, f-measure, or accuracy (with respect to an "ideal" mapping, so called "oracle" below).

As other RESTful services, these mediation services are natively available through HTTP requests containing JSON documents. In addition, the mediation framework is equipped with a web-based graphical user interface, which eases the use of the framework and can be used to showcase the mediation portal. This interface supports the main uses cases depicted in Figure 2-1.

Figure 2-2 The Service-oriented Architecture of the Mediation Framework

From a technical perspective, the core services are implemented in the Scala language, and provide a Scala/Java client API. The web interface is implemented in HTML 5 and Javascript, and leverages the JQuery and Twitter bootstrap libraries.

2.3 Using the mediation framework In the following, we will explain how to use the features of the mediation framework depicted on

Figure 2-2, by using the web interface or by sending REST requests to services directly. Such requests may easily be sent to any URL provided in this section by using the Chrome REST

Mediation Framework

«REST Service»Model Repository

user

«REST Service»Mapping Repository

user

«REST Service»Ev aluation Repository

user

«REST Service»Mediator Serv ice

models

library

mappings

user

«Scala Library»Mediation Library

user

user

«REST Service»Comparator Serv ice

mappings comparisons

user

Mediation Designer Mediation User

«HTML / Javascript»Web Interface

user

mediator comparator

«http»«http»«http»«http»

Public


console3 or any other REST testing tool. Please refer to D5.3 in order to run or install the mediation portal.

In addition, open-source implementation and documentation are available online at http://sintef-9012.github.com/mediation-portal/

2.3.1 Pushing in the model repository

2.3.1.1 Using the Web-based Graphical Interface

As shown Figure 2-3 below, the web interface allows users to push new models into the repository. We can see, on the left side of this screenshot, a grey panel where the user can specify a unique identifier for his model, a short description, and provide the related source file. On the right hand side, we can see the list of models currently stored in the repository.

The two other repositories, namely mappings and comparisons, are available through similar interfaces.

Figure 2-3 Accessing the repositories using the web interface

2.3.1.2 Using the REST API

The model repository is available as a REST service at the following address:

http://54.247.114.191/sensapp/mediation/repositories/models

3 https://chrome.google.com/webstore/detail/cokgbflfommojglbmbpenpphppikmonn

Public


In order to push a model in a repository, you merely need to send a JSON request specifying the name of the model, as well as its content. Models may be provided in two forms: as an XSD schema by embedding directly the XML description in the JSON document, or as a specific MoF syntax described in appendix. The snippet below illustrates a simple JSON request that pushes a XSD file into the repository.

{ "name": "my-model", "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?> ..." }

Your model will then be available at the address:

http://54.247.114.191/sensapp/mediation/repositories/models/my-model

2.3.2 Triggering an existing mediation

2.3.2.1 Using the Web-based Graphical Interface

The simplest way to mediate between two data models is to use the graphical user interface. The mediator service has its own interface, depicted below by Figure 2-4, below.

On the left side, a form lets the user specify the model between which he wants to mediate, assuming that those models are available in the model repository. The user may also select the algorithm that he wants to use to detect equivalent model elements.

On the right hand side, a table contains the match detected by the mediation framework. The user may validate or invalidate the suggestions provided by the mediator service using the buttons available in the "Valid?" column. In Figure 2-4, suggestions have been restricted to the one containing "Author". Such filtering facilities are critical to quickly search through large mappings resulting from mediation between large data models.

Figure 2-4 Using the Mediator Service to mediate between two data models

Public



Mediations are run by the "mediator" service, available at the following URL:

http://54.247.114.191/sensapp/mediator/

As for the model repository, mediations are triggered by sending a JSON message embedded in an HTTP request. Below is a sample mediation request, where we specify the URL of the two models between which we want to mediate, and the name of the algorithm that we want to use for this particular mediation.

{ "source": "my-first-model", "target": "my-second-model", "algorithm":"xsd-syntactic" }

We assume here that both models have been previously pushed in the model repository and will be available in here. The third parameter is the name of the algorithm that we do want to use. The result of this request will be a mapping, available in the mapping repository whose URL will be provided as the response of the mediation. For instance, you may receive something like:

http://54.247.114.191/sensapp/mediation/repositories/mappings/7ba5cfd1-8b43-4292-892a-b32281ef4189

where the UUID "7ba5cfd1-8b43-4292-892a-b32281ef4189" is (in this very case) the identifier of the resulting mapping, that contains the actual information computed by the mediation.

2.3.3 Creating an ad hoc Mediation Algorithm The mediation portal let you create your own mediation algorithm by combining existing ones as well as various predefined functions (including string matching, vectors distance, etc.). The framework let you create various types of algorithms that you will then be able to combine:

• Model Processor, process a single model and produces an alternate version of it. Various common processing such as pruning, relabeling or translating fall in this category. Formally, a model processor is a function π such as 𝜋:𝑀𝑜𝑑𝑒𝑙 → 𝑀𝑜𝑑𝑒𝑙

• Model Aggregator, which accepts as input a list of models and produces a single one based on specific criterions. Formally, a mapping aggregator is a function ε, such as 𝜀: 2𝑀𝑜𝑑𝑒𝑙 →𝑀𝑜𝑑𝑒𝑙

• Model Slicer, which takes a single model as input and slices it into several sub models. Formally a model slicer is a function σ, such as 𝜎:𝑀𝑜𝑑𝑒𝑙 → 2𝑀𝑜𝑑𝑒𝑙

• Mediation, which basically accepts two models (a source model to be mapped to a target model), one mapping, and refine the mapping based on a specific analysis of the two given models. Formally, a mediation is a function μ such as: 𝜇 ∶ 𝑀𝑎𝑝𝑝𝑖𝑛𝑔 × 𝑀𝑜𝑑𝑒𝑙 × 𝑀𝑜𝑑𝑒𝑙 →𝑀𝑎𝑝𝑝𝑖𝑛𝑔

• Mapping Aggregator, which accepts as input a list of mappings and produces a single one using a specific algorithm. Formally, a mapping aggregator is a function ε, such as 𝜀: 2𝑀𝑎𝑝𝑝𝑖𝑛𝑔 → 𝑀𝑎𝑝𝑝𝑖𝑛𝑔

• Mapping Slicer which accepts a single mapping as input, and break it into various sub mappings. Formally, a mapping slicer is defined as function σ, such as 𝜎:𝑀𝑎𝑝𝑝𝑖𝑛𝑔 →2𝑀𝑎𝑝𝑝𝑖𝑛𝑔

All together, the existing functions provided by the mediation framework enable to describe complex mediation algorithms. The use of the functional paradigm ensures an easy parallelisation of complex and resource consuming algorithms.

Public


However, as the Scala language used to implement our framework supports both functional and object-oriented programming, these functions are also classes, and can easily extended using inheritance. The example below illustrates how to define a simple mediation algorithm, which, given two models, make the best of the syntactic and random matching algorithms. We extend the Mediation class (so we also create a new function) and we specify its behaviour by overriding the "execute" operation.

To summarize, the creation of your own mediation algorithm requires to:

1. Create a Scala class that extends some specific classes and traits provided by the mediation framework, depending on the type of algorithm you want to create.

2. Register your algorithm in the mediation portal 3. Recompile the mediation framework

Below is an example of such a composite mediation:

package net.modelbased.mediation.samples

import net.modelbased.mediation.library.algorithm._ import net.modelbased.mediation.service.repository.mapping.data.Mapping import net.modelbased.mediation.service.repository.model.data.Model

class SampleMediation extends Mediation {

// We import the standard library of mediation algorithms import net.modelbased.mediation.library.algorithm.Commons._

/* * Here, we specify the behaviour of the mediation, by running a syntactic * match on one side, a semantic match on the other side, and finally merging the * two results by selecting the most relevant mapping entries. */ override def execute(in: Mapping, source: Model, target: Model) = { val m1 = syntacticMatch(in, source, target) val m2 = randomMatch(in, source, target) val result = aggregateByMax(Set(m1, m2)) }

}

2.3.4 Comparing Existing Algorithms Comparing the effectiveness of algorithms is critical in developing of real-life mediation algorithms. When building mediation algorithms, developers often dispose of a few test models, for which they already know what the expected mappings that should be detected automatically are. Such expected mappings, so called "oracle", can be placed in the mapping repository, and the comparator service will automatically calculate metrics including, precision, recall, or f-measure, reflecting how close a given mapping is to the oracle, and how effective is the algorithm used to generate the mapping.

2.3.4.1 Using the web-based Graphical Interface

The simplest way to leverage the comparator service is also through the web interface, as it permits to directly visualize the statistics as bar plots. Figure 2-5 below illustrates the comparison of three mappings, with a specific oracle. Oracle and subject mappings are selected on the left hand side panel, whereas the comparisons results are displayed on the right hand side.

Public


Figure 2-5 Using the comparator interface to evaluate the effectiveness of various mediation algorithms


In case one needs to evaluate the relevance of a given mediation algorithm on a given case study, he/she may want to evaluate the performance (typically metrics such as precision, recall and f-measure). This is possible using the comparator service, which given two mappings (one being considered as the oracle, or the reference) will provide you with the relevant performance hints. The comparator service is available at the following address

http://54.247.114.191/sensapp/comparator

In order to trigger the comparator service, one need to send a specific JSON message embedded in a POST HTTP request. The following code snippet illustrates such a JSON body, where we specify the name of the oracle mapping (in the mapping repository), the name of the mapping to evaluate against the oracle, and the some additional information regarding this comparison).

{ "oracle": "my-oracle", "mapping": "7ba5cfd1-8b43-4292-892a-b32281ef4189", "note": "This is a test of the comparator service" }

Following such a request, you may access the result of the comparison at the following address. The comparison contains the various number of false positive, false negative, true positive and true negative

http://54.247.114.191/sensapp/mediation/repository/comparisons/my-oracle/7ba5cfd1-8b43-4292-892a-b32281ef4189/

Statistics concerning a specific evaluation, including precision, recall, and f-measure are then also available at a different URL such as:

http://54.247.114.191/sensapp/mediation/repository/comparisons/my-oracle/7ba5cfd1-8b43-4292-892a-b32281ef4189/stats

Public


3 Behavioural Service Mediation Behavioural mediation is needed when the protocols of two collaborating services become mis-aligned, for example after a refactoring, a major revision, etc. (see Figure 3-1). Typical deviations include: splitting/merging of messages, renaming, or adding/removing missing/extra messages.

Figure 3-1 - Simple protocol misalignment

One possible solution to such a situation is of course to also refactor the APIs of all the impacted client services. The code of the impacted services (typically clients using a service) is however usually not be managed by the same service provider. More flexible solutions to behavioural service mediation are thus needed.

3.1 Approach Overview The automatic synthesis of behavioural mediator is a long-term research topic, for example addressed by FP7 CONNECT FET IP (aiming at long term impact): https://www.connect-forever.eu/

With more limited resources, REMICS aims at a more pragmatic approach consisting in providing tools and automation (where possible) to designers to help and guide them in defining and deploying behavioural mediators, as illustrated in Figure 3-2. A first set of tools is more dedicated to the user of services to monitor services and detect potential deviations. We rely on the WP6 models@runtime engine to monitor services and mediators and detect potential deviations.

https://www.connect-forever.eu/

Public


Figure 3-2 Overview of the service mediation approach

3.2 Architecture The behavioural mediation design toolkit is architected around a Domain-Specific Language (DSL) supported with a textual editor, which allows designers to express mediations, using state machines. This DSL is supported by a set of compilers targeting the following outputs:

• A lightweight simulation environment, for the rapid prototyping of mediators, based on service mock-ups

• Deployable artifacts able to actually perform a mediation according to its specification

Behavioral Mediation Framework

Serv ice User Serv ice Mediation Designer

Detect Dev iation

Analyse Dev iation

Model Mediation Deploy Mediation

Public


Figure 3-3- Overall Architecture

The overall (coarse-grained) architecture of the toolkit is depicted in Figure 3-3. The DSL is engineered using Model-Driven techniques. It is based on a metamodel defined in ECore/EMF4 (the defacto standard in MDE) and its syntax is defined with EMFText5 (initially developed in FP6 MODELPLEX). The different compilers extend this metamodel and take models (conforming to this metamodel) as input and produce JVM-compliant code, wrapped into Maven projects. Maven6 is a build automation tool typically used for Java projects and widely used to manage large industrial projects. All the code generated from this mediation DSL can be run using:

mvn clean install exec:java –Dexec.mainClass="<path-to-the-main-class>"

Maven will then download all the necessary dependencies, compile the code and run the main.

The DSL and compilers have been integrated to the open-source initiative ThingML (see www.thingml.org) aiming for the seamless integration of the Internet of Services and Internet of Things into the Cloud.

4 http://www.eclipse.org/modeling/emf/ 5 http://www.emftext.org/index.php/EMFText 6 http://maven.apache.org/

Behavioral Mediation Design

Behav ioral Mediation DSL /

Editorcompile

GUI

Simulator Compilercompile

Monitoring Compilercompile

Mediator Compilercompile

Serv ice Mediation Designer

(from Use Case Model)

http://www.thingml.org/

http://www.eclipse.org/modeling/emf/

http://www.emftext.org/index.php/EMFText

http://maven.apache.org/

Public


3.3 Using the DSL The first step consists in expressing service specifications in the behavioural mediation DSL. Only the interfaces (i.e., the messages to be exchanged) need to be specified at this point. The interfaces can then either be implemented through this DSL, or be linked to existing Java implementation (for example the code generated from the migrated UML+SoaML models in WP4).

A message simply consists of a name, together with an optional list of typed parameters. It can be compared to Java operations. However, the semantics of message passing is asynchronous. A simple example is given in Figure 3-4.

Figure 3-4 – A sub-set of the messages of the DOME system

The messages are then grouped into ports, which define the services that are provided and required by specifying an abstract protocol i.e., which messages need to be sent and received to fulfil the service. This is illustrated in Figure 3-5.

Public


Figure 3-5 A service provided by the DOME system

This abstract specification is then refined using state-machines specifying how those messages are manipulated by the service. From a conceptual point of view, these state machines are aligned with UML 2.0, complemented with a first-class action language. This action language makes it possible to model complex behaviour at a platform-independent model, with no need for the designers to "hack" some code in their models (e.g., in textual annotations that are not typed/checked). Still, this DSL enables the wrapping of existing code (e.g., Java) so that it is not required to re-model existing services. In particular, we simply delegate the execution of the service to the code that has been recovered and migrated. The wrapping of one of the DOME services is shown in Figure 3-6. All the concepts of the mediation DSL are described at www.thingml.org.

Figure 3-6 Wrapping of a DOME service in the DSL

http://www.thingml.org/

Public


Based on this specification, it is possible to generate a Java-based simulation environment for this service, as illustrated in Figure 3-7. This figure shows a service mock up (the GUI on the right hand side) connected to a service implementation running in the background (the black window). The mock up allows designer to stimulate a service provider by sending it messages, and log the responses.

Figure 3-7 - Interactive simulation of the DOME service

The DSL proposed by WP5 also seamlessly compiles to the state machine-based models@runtime engine provided by WP6, as illustrated in Figure 3-8.

Figure 3-8 A DOME service compiled to and running on the WP6 models@runtime engine

Public


Mediators are also implemented as state machines, which can basically accept the sequence of messages shown in the left-hand side of Figure 3-1, and produce the sequence shown in the right-hand side (and vice versa). In other words, mediators are responsible for translating protocols by implementing the "glue" between 2 state machines (services). The wrapping and simulation features provided by our DSL make it possible to implement mediators in an iterative way, by mocking some parts and progressively integrating with the real services.

Figure 3-9 – Two services and a mediator

In Figure 3-9, the state machine at the bottom corresponds to a recovered and migrated service from the DOME case study, which has simply been wrapped into our DSL. The top right state machine corresponds to an external service providing the same features. The top right state

Public


machine is the mediator that actually translates one protocol into another. Using our DSL, the mediator is implemented in only 45 lines of code, each line being rather simple, for a total of 600 characters. The same code written by hand in Java in WP 6, and which we can generate in a slightly more verbose but semantically equivalent form requires 150 lines of code and 5500 characters (i.e., a gain of approximately 70 % regarding lines of code, and 90 % regarding characters).

As we can see in Figure 3-10, the API of the former service and the API of the mediator are equivalent. A client can thus transparently continue to use the former service or switch to the new service via the mediator, with no need for any refactoring.

Figure 3-10 – Former API and mediator API

Public


4 Interoperability across Cloud Providers Cloud computing is a computing model enabling ubiquitous network access to a shared and

virtualized pool of computing capabilities (e.g., network, storage, processing, and memory) that can be rapidly provisioned with minimal management effort. In practice however, cloud solutions are highly heterogeneous and the provided features are often incompatible. This diversity hinders the exploitation of the full potential of cloud computing by increasing the complexity of development and administration of multi-cloud systems. This section gives an overview of CloudML, with a particular focus on its interoperability capabilities. More general features of CloudML are discussed in D4.4, D4.5 and D4.6. CloudML provides a declarative abstraction layer on top of multiple cloud providers in order to tame the interoperability issue this diversity causes.

4.1 Approach Overview The CloudML Domain-Specific Language and execution platform rely on two distinct levels in

order to maximize reusability or artefacts across providers, and abstract from this heterogeneity:

• Type level to describe cloud concerns related to the application in a cloud agnostic way. It makes it possible to describe templates of cloud applications and deployments that can be reused (instantiated) in multiple contexts;

• Instance level, which instantiates the type level in order to refine the deployment of a cloud application according to the specificities of cloud providers, etc.

From the cloud perspective, the introduction of a new layer of abstraction improves the portability and reusability of cloud related concerns amongst several clouds. Indeed, even if the system is designed for a specific platform including framework, middleware, or cloud services, these entities often rely on similar concepts which can be abstracted from the specificities of each cloud provider. Typically, the topology of the system in the cloud as well as the minimum hardware resources required to run it (e.g., CPU, RAM) can be defined in a cloud-agnostic way. Thanks to this new abstraction layer, one can map a platform specific model to one or more cloud providers. More details on this aspect of CloudML are given in REMICS deliverable D4.4 and D4.5, as well as the following publications7, 8.

As illustrated in Figure 4-1, CloudML enables application developers to define reusable and platform-independent deployment model of their applications, basically by manipulating the abstract notions of nodes and software artefacts. At this point, templates are fully independent of any cloud provider like Amazon or Rackspace. The application provider can then instantiate these templates in order to prepare the deployment on a specific infrastructure. At This time, information about cloud providers is provided in order to make the deployment possible. The deployment and sub-sequent adaptation of the cloud adaptation (for example to cope with a higher load) is then fully automated by CloudML, using some models@runtime techniques pioneered in the context of EU FP7 DiVA and ported to the Cloud in REMICS, in collaboration with MODAClouds and PaaSage FP7 IPs.

7 Towards model-driven deployment, provisioning, monitoring, and adaptation of multi-cloud systems.

N. Ferry, A. Rossini, F. Chauvel, B. Morin, and A. Solberg. In Cloud’13: IEEE 6th International Conference on Cloud Computing June 27-July 2, 2013, Santa Clara Marriott, CA, USA.

8 Managing multi-cloud systems with CloudMF. N. Ferry, F. Chauvel, A. Rossini, B. Morin, and A. Solberg. In NordiCloud’13: 2nd Nordic Symposium on Cloud Computing & Internet Technologies September 2-3 2013, Oslo, Norway

Public


Figure 4-1Overview of CloudML

4.2 Architecture The architecture of CloudML is depicted in Figure 4-2. The CloudML Model component

encapsulates the CloudML metamodel and offers a simple Java-based Create-Read-Update-Delete (CRUD) API to instantiate an manipulate CloudML models in a programmatic way. The Codecs component is responsible for the marshalling and unmarshalling of CloudML models. We currently provide two alternative formats (XMI and JSON) to represent CloudML models in a declarative way. The JSON format has been adopted in REMICS as it provides a fairly readable syntax accessible to technical users (application developers and providers). The XMI format provides interoperability with any modelling tool relying on the Eclipse Modelling Framework. In order to support interoperability across cloud providers, the Deployer component relies on the well-established jClouds API, which provides interoperability, at the programming level, with 30 cloud providers and cloud software stacks including Amazon, Azure, GoGrid, Ninefold, OpenStack, Rackspace, and vCloud. The deployer is also responsible for enacting the dynamic adaptation of the system. Finally, the UI component provides a front-end for technical users. We currently provide a textual shell and a simple graphical editor, illustrated in Figure 4-3.

Public


Figure 4-2 Architecture of CloudML

CloudML thus promotes this interoperability at the modelling level, where designers can define their cloud applications and deployments in a declarative way, rather than coding it directly in Java. The type/instance pattern of CloudML makes it possible to define reusable types, which can easily be instantiated and refined for (almost) any cloud provider.

Figure 4-3The CloudML graphical editor with a DOME model.

4.3 Using CloudML D4.4 and D.4.5 give a detailed overview on how to use CloudML in practice.

Public


5 Lessons Learned Model-Driven Service Interoperability promotes a higher level of abstraction to help service designers when they address interoperability issues. Regarding data interoperability, we learned the following lessons, based on the work carried out in the REMICS data mediation framework (Section 2):

1. Data schemas have very different semantics, syntax, and rely of very different technological spaces. A single algorithm cannot handle this heterogeneity. Instead, we have proposed in WP5 a service-oriented framework for the composition and evaluation of ad hoc mediation algorithms.

2. Data schemas represented at the modelling level (e.g., using UML or ECore class diagrams) are not significantly more abstract than data schemas expressed with XSD, database schema, hence no significant gain is to be expected from MDI when implementing data mediation algorithm. However, unifying data into a common modelling language can help rationalizing the development of mediation algorithms.

3. Ecore/EMF, the open-source de-facto standard for metamodeling, provides a feature for importing XSD schemas. This feature however does not scale to real-life XSDs used for example in the accounting domain and requires many manual fixes, hindering the potential benefits of MDI.

4. The general scalability and efficiency of Ecore/EMF is also not adapted to the manipulation of large data models. EMF models are indeed memory inefficient and the API provided by EMF is not thread-safe, making it difficult to implement parallel algorithms manipulating these models. Implementing a single threaded algorithm is also not realistic as hardware and software engineering have entered the multi-core era. More insight on EMF-limitation and a potential alternative can be found here9.

Regarding behavioural mediation, we learned the following lessons based on our tool-supported DSL for behavioural mediation (Section 3):

1. As for data mediation, the API to access the data also relies on different technological stacks. In order to cope with this heterogeneity, we provided a lightweight DSL that can seamlessly integrate with and wrap code-level artefacts.

2. The technological choices made in the migration step of REMICS (basing all the services on the JBOSS technologies and container) significantly hinder the potential for service interoperability. To some extent, JBOSS imposes a "vendor lock-in": services have to implement some interfaces so that they can run in the JBOSS container and properly communicate. An more open approach to continuous, post-migration maintenance and interoperability would have been to migrate services to lightweight REST services that do no impose strong technological constraints.

Regarding the interoperability across cloud providers (cf. Section 4), we learned the following lessons based on CloudML:

1. The initial implementation of CloudML was based on an internal DSL expressed (with) in Scala. While it proved to be a rapid way of prototyping CloudML, it encounters some acceptance issues, mostly due to the perceived complexity of Scala. The current version of CloudML is thus based on an external DSL, supported by Plain Old Java Objects (POJO).

9 An Eclipse Modelling Framework Alternative to Meet the Models@Runtime Requirements. F. Fouquet, G. Nain, B. Morin, E. Daubert, O. Barais, N. Plouzeau and J-M. Jézéquel. In MODELS’12 (Application Track): ACM/IEEE 15th International Conference on Model Driven Engineering Languages and Systems.

Public


2. The new POJO-based version of CloudML appeared to be usable (by partners not directly involved in its development) to deploy small-to-mid size application. More experience should be conducted on larger-scale systems.

3. The component-based approach adopted in CloudML, coupled with the type/instance pattern has been useful to facilitate the deployment and re-deployment of cloud application, while improving reuse and separation of concerns.

Public


6 Conclusion Interoperability is a complex challenge per se (e.g., with dedicated projects like CONNECT FET IP focusing on that sole topic). REMICS WP5 provides pragmatic tools to help and guide designers in the mediation process (rather than trying to automate everything). Regarding data mediation, WP5 provides a REST-based framework for the rapid prototyping of matching algorithm. Existing automatic matching algorithms only produces matching suggestions, which need to be further checked and completed by humans. Matching accuracy significantly decreases as noise increases in data models and large industrial mediations call for hybrid solutions combining various algorithms, as well as ad hoc pre-processing. WP5 thus provides a mean to rapidly sketch a hybrid mediation algorithm, tailored for particular cases, and to assess its effectiveness on existing data models. Regarding behavioural mediation, WP5 provides tools to generate interceptors (to monitor the interactions between components/services) and a DSL to define mediators together with a set of compilers to test mediators in a controlled environment, and dynamically deploy these mediators. This DSL compiles to the WP6 state machine-based models@runtime engine. Finally, CloudML (developed in collaboration with WP4 and other projects such as MODAClouds) enables the interoperability across multiple cloud providers. The lessons learned from these three artefacts have been discussed in Section 5.

Public


Annex 1 class MofParser extends RegexParsers { var global: Option[Node] = Some(MofAst.createGlobalScope) // Redefine the notion white space so as to eliminate comments starting with '#' protected override val whiteSpace = """(\s|#.*)+""".r def integerLiteral: Parser[Int] = """[0-9]+""".r ^^ { case v => Integer.decode(v) } def identifier: Parser[String] = """[a-zA-Z_][a-zA-Z0-9_]*""".r def dataType: Parser[DataTypeNode] = "data" ~> "type" ~> identifier ^^ { case name => new DataTypeNode(global, name.toString()) } def enumeration: Parser[EnumerationNode] = ("enumeration" ~> identifier) ~ ("{" ~> repsep(enumerationLiteral, ",") <~ "}") ^^ { case name ~ literals => new EnumerationNode(global, name.toString(), literals) } def enumerationLiteral: Parser[LiteralNode] = identifier ^^ { case i => new LiteralNode(global, i) } def `package`: Parser[PackageNode] = { ("package" ~> identifier) ~ ("{" ~> rep(definition) <~ "}") ^^ { case name ~ defs => new PackageNode(global, name, defs) } } def definition: Parser[Node] = { `package` ^^ { case p => p } | `class` ^^ { case c => c } | enumeration ^^ { case e => e } | dataType ^^ { case dt => dt } } def `class`: Parser[ClassNode] = opt("abstract") ~ ("class" ~> identifier) ~ opt(inheritance) ~ opt(("{" ~> rep(feature) <~ "}")) ^^ { case isAbstract ~ name ~ superClasses ~ features => new ClassNode(global, name, isAbstract.isDefined, features.getOrElse(Seq.empty), superClasses.getOrElse(Seq.empty)) } def inheritance: Parser[Seq[Reference]] = "extends" ~> repsep(reference, ",") ^^ { case sc => sc } def feature: Parser[FeatureNode] = identifier ~ (":" ~> reference) ~ opt(multiplicity) ~ opt(opposite) ^^ { case name ~ ref ~ mul ~ opposite => mul match { case None => new FeatureNode(global, name, ref, opposite=opposite) case Some((l, u)) =>

new FeatureNode(global, name, ref, l, u, opposite=opposite) } }

Public


def opposite: Parser[Reference] = "~" ~> reference ^^ { case ref => ref } def reference: Parser[Reference] = repsep(identifier, ".") ^^ { case all => new Reference(all: _*) } def multiplicity: Parser[(Int, Option[Int])] = "[" ~> "*" ~ "]" ^^ { case c => (0, None) } | "[" ~> integerLiteral ~ (".." ~> upperBound) <~ "]" ^^ { case l ~ u => (l, u) } def upperBound: Parser[Option[Int]] = "*" ^^^ None | integerLiteral ^^ { case v => Some(v) } }

deliverable d5.4 mdi for service interoperability ... · pdf fileopentravel alliance was...

Documents