experimentation environment development final release · 2017. 4. 25. · fispace 11.11.2015...

34
Deliverable D300.11 Experimentation environment Development Fi- nal Release WP 300 Project Acronym & Number: FIspace 604 123 Project Title: FIspace: Future Internet Business Collaboration Networks in Agri-Food, Transport and Logistics Funding Scheme: Collaborative Project - Large-scale Integrated Project (IP) Date of latest version of Annex 1: 17.03.2015 Start date of the project: 01.04.2013 Duration: 30 Status: Final Authors: Seyhun Mehmet Futacı, Aggelos Groumas, Sokratis Barmpounakis Contributors: Carmen Perea Escribano, Fabiana Fournier, Seyhun Mehmet Futacı, Eliezer Dekel, Dany Moshkovitz , Philip Reimer, Havier Romero, Vasileios Sarris Document Identifier: FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Date: 11.11.2015 Revision: 002 Project website address: http://www.FIspace.eu

Upload: others

Post on 19-Aug-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

Deliverable D30011

Experimentation environment Development Fi-nal Release

WP 300

Project Acronym amp Number FIspace ndash 604 123

Project Title FIspace Future Internet Business Collaboration Networks in Agri-Food Transport and Logistics

Funding Scheme Collaborative Project - Large-scale Integrated Project (IP)

Date of latest version of Annex 1 17032015

Start date of the project 01042013

Duration 30

Status Final

Authors Seyhun Mehmet Futacı Aggelos Groumas Sokratis Barmpounakis

Contributors Carmen Perea Escribano Fabiana Fournier Seyhun Mehmet Futacı Eliezer Dekel Dany Moshkovitz Philip Reimer Havier Romero Vasileios Sarris

Document Identifier FIspace-D30011-ExpEnvDevFinalRel-V002docx

Date 11112015

Revision 002

Project website address httpwwwFIspaceeu

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 2 of 34

The FIspace Project

Leveraging on outcomes of two complementary Phase 1 use case projects (FInest amp SmartAgriFood) aim of FIspace is to pioneer towards fundamental changes on how collaborative business networks will work in future FIspace will develop a multi-domain Business Collaboration Space (short FIspace) that employs FI technologies for enabling seamless collaboration in open cross-organizational business net-works establish eight working Experimentation Sites in Europe where Pilot Applications are tested in Early Trials for Agri-Food Transport amp Logistics and prepare for industrial uptake by engaging with play-ers amp associations from relevant industry sectors and IT industry

Project Summary

As a use case project in Phase 2 of the FI PPP FIspace aims at developing and validating novel Future-Internet-enabled solutions to address the pressing challenges arising in collaborative business networks focussing on use cases from the Agri-Food Transport and Logistics industries FIspace will focus on ex-ploiting incorporating and validating the Generic Enablers provided by the FI PPP Core Platform with the aim of realising an extensible collaboration service for business networks together with a set of innovative test applications that allow for radical improvements in how networked businesses can work in the future Those solutions will be demonstrated and tested through early trials on experimentation sites across Eu-rope The project results will be open to the FI PPP program and the general public and the pro-active engagement of larger user communities and external solution providers will foster innovation and indus-trial uptake planned for Phase 3 of the FI PPP

Project Consortium

DLO Netherlands Kuumlhne + Nagel Switzerland

ATB Bremen Germany University Duisburg Essen Germany

IBM Israel ATOS Spain

KocSistem Turkey The Open Group United Kingdom

Aston University United Kingdom CentMa Germany

ENoLL Belgium iMinds Belgium

KTBL Germany Marintek Norway

NKUA Greece University Politecnica Madrid Spain

Wageningen University Netherlands Arcelik Turkey

PlusFresc Spain EuroPoolSystem Germany

FloriCode Netherlands GS1 Germany Germany

Kverneland Netherlands Mieloo amp Alexander Netherlands

North Sea Container Line Norway OPEKEPE Greece

LimeTri Netherlands Innovators Greece

More Information

Harald Sundmaeker (coordinator)

ATB Bremen GmbH

Wiener Str 1

D-28359 Bremen Germany

e-mail Sundmaekeratb-bremende

phone +49 421 220920

wwwFIspaceeu Twitter

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 3 of 34

Dissemination Level

PU Public

PP Restricted to other programme participants (including the Commission Services)

RE Restricted to a group specified by the consortium (including the Commission Services)

CO Confidential only for members of the consortium (including the Commission Services)

Change History

Version Notes Date

001 Creation of the document 30092015

002 Refinement with partner input 11112015

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 4 of 34

Table of Contents

1 Experimentation environment components description 6

11 EE Components statusfunctionality 8

12 Class diagrams 10

13 Database diagram 17

14 Sequence diagrams 18

141 Experiment creation 18

142 Execution of experiment 19

143 Create KPIs 19

144 Backend simulator-(SDI)capabilities usage 20

2 Detailed Usage Guide 21

21 Greenhouse experiment 21

3 Detailed Greenhouse Experiment 22

31 Create a greenhouse experiment with two steps 22

32 Read an Experiment 25

321 Get all experiments 26

322 Search Experiment 27

33 Execution Management of an Experiment 28

34 Report and KPIs based for specific execution 30

4 Further experimentation 30

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario 31

6 References 33

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 5 of 34

List of Figures

Figure 1 Overall EE components overview 6

Figure 2 Experiment CRUD Search classes 10

Figure 3 ExecutionManagerService Executor Classes 11

Figure 4 ReportManager ResourceManager classes 11

Figure 5 ExecutionLogManager ScriptExecutionEngine classes 12

Figure 6 KPI manager KPI Calculator classes 13

Figure 7 KPICalculatorSerializer classes 13

Figure 8 DataProvider Classes 14

Figure 9 DataProviderSystem classes 14

Figure 10 External data access classes 15

Figure 11 Backend simulator classes 16

Figure 12 Experimentation environment Database tables 17

Figure 13 Experiment Creation- search 18

Figure 14 Execute specific experiment 19

Figure 15 Create KPIs 19

Figure 16 Backend simulator usage 20

Figure 17 experiment CRUD 22

Figure 18 Read experiment 25

Figure 19 get all experiments 26

Figure 20 Search experiment 27

Figure 21 Execution start 28

Figure 22 Log results of execution (sensor data and actions alerts from expert system) 29

Figure 23 Reports and KPIs 30

Figure 24 Backend Simulator 31

List of Tables

Table 1 Components description 8

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 6 of 34

1 Experimentation environment components description

The FIspace Experimentation Environment (EE) provides the means to different stakeholders to be able to explore interact with and understand how FIspace platform works in practice Specifically the EE pro-vides a controlled environment in which developers and business architects can test new collaborative scenarios before these processes are released for use by ldquoend usersrdquo Towards this end an overall framework comprising several core modules is used in order to be able for the users to define create and execute their experiments as well as evaluate afterwards the outcomes of the scenario execution The figure that follows (Figure 1) illustrates the EE core framework All EE core modules are implemented and available to the end users In addition almost all core modules are already deployed in one of the scenar-ios that have been used in the context of the EE

Figure 1 Overall EE components overview

A more detailed description for each one of the frameworkrsquos components follows

User Manager The User Manager handles user accounts passwords and access This includes features such as user groups and access control to data as well as users being able to assign other users permis-sions

Experiment Manager The Experiment Manager handles experiment lifecycle and experiment querying This includes creation versioning archiving and search capabilities

Experiment CRUD (Create Read Update and Delete) Provides services for experiment lifecycle using archiving instead of deletion so that traceability is never lost

Experiment Search Provides services for finding experiments according to various search criteria

Execution Manager The Execution Manager handles the concrete executions of an experiment This includes the creation of new executions (including the configuration of variables) executing (or tracking the execution of) the steps in the experiment and logging the results

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 7 of 34

Executor The Executor tracks the execution of the individual steps in an evaluation This in-cludes the automated execution of certain steps such as injecting dataevents into the test in-stance and running VUser scripts through the Script Execution Engine This component also cre-ates and updates entries in the execution log including notifications received from the actual process execution and error messages

Script Execution Engine This executes VUser scripts to automate user actions

Resource Manager The Resource Manager provides an inventory of available resources Services in-clude the ability to locate resources according to various search criteria

Reporting Reports are generated based on execution logs and KPIs

KPI Manager Manages the calculation of KPIs

Execution Log Manager The Execution Log Manager provides logging services for an execution This includes the logging of the results for each step of an execution including any received notifications during the execution of each step Also provides access to these logs

Execution Data Manager This is used to manage the access to data that is used during execution

Internal Data Provider System Used for storing and retrieving manually configured data provid-ers

External Data Access Used to retrieve data from 3rd party external systems could be used to ldquoreplayrdquo events from a real-world shipment for example

Access to these systems is configured by the tester Configuration could be UI or file driven

FIspace Test This is the FIspace public instance which is the replica of a latest stable version of the inte-grated platform hosted on the integration environment depicted on D3006 This instance is updated when the new version of the platform is released FIspace Test platform consists of different modules developed in WP200 are installed and integrated following the instructions provided by the module owners on PIE wiki[1]These instructions are refined by Integration and FIspace Test platform team for the each new release of the platform while integration is executedAfter each integration process inte-gration tests procedures are followed to provide Phase3 developers stable platform Server and services information of each module that constructs the current public version of the FIspace Test platform can be found on the bitbucket EE wikipage[2]

Backend Simulator This simulates input data from backend systems to FIspace and provides APIs to inject data to the FIspace Test systemrsquos modules The simulator reports back on events and other pro-cessed data

EE Storage This provides internal storage services for the experimentation environment

Data Provider System (app) ndash This component addresses the binding of the application to the execution data

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 8 of 34

11 EE Components statusfunctionality

Table 1 Components description

Core Component Status Functionality

ExecutionManagerService Core functionality pro-vided by NKUA

It currently starts a new execution according to an experimentstores it and provides the functionality to search for executions according to experimentidcreator Also calls Executor for the execution of StepsSteps can now be running for as time as needed

Executor Core functionality pro-vided by NKUA

Executes an experiments step and logs its result

ScriptExecutionEngine Core functionality pro-vided by NKUA for Greenhouse experiment

Executes a steps REST call or jar

BackEndSimulator Core functionality pro-vided by NKUA

First version of connecting to SDI using keycloak Abstract interface created for various backends Greenhouse backend simulator development in progress

ExecutionLogManager Core functionality pro-vided by NKUA

Logs a steps result and saves it to persistent storageAlso retrieves logs for a specific execution

Experiment CRUD Core functionality pro-vided by NKUA

Create an experiment to persistent storage retrieve it update it

Experiment Search Core functionality pro-vided by NKUA

Search by name sample using full text

ReportManager Completed for the Greenhouse Advice Re-quest scenario

Create report get report by exe-cution or experiment and search report by namedescription

KPI Manager Basic Functionality im-plemented defined the public interface of the component and storage in internal DB

Sample createread kpis Various KPIs available

User Manager Users log in through IDM for accessing UI

Users log in through IDM for ac-cessing UI DigitalSelf IDM imple-mentation is used Spring security is integrated with UI for authoriza-tion mechanisms

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 9 of 34

Core Component Status Functionality

Execution Data Manager Core functionality pro-vided by NKUA for Greenhouse experiment

In an execution data management a process can addretrieve a data provider system A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

Resource Manager Core functionality pro-vided by NKUA for Greenhouse experiment

A resource can be created and retrieved

The proposed technical specification of the EE enables the entire process from testexperiment plan-ning and configuration through execution to analysis of the test execution We introduce below terms to be used throughout this report

Step ndash A step refers to a single actiontask defined in a test scenario

Test scenario ndash This is the ordered set of steps that compose a single test

Variables ndash In the context of a test these are field names that stand for specific values during execution Variables enable flexibility in test execution as they enable running the same test with different field values

Variables binding ndash This refers to the replacement of variables values with the test data This is done by the experimenter during test execution

Experimenttest ndash This is the ordered set of steps to be carried out by an experimenter during execution Each experiment is identified by a unique ID and version An experiment may have variables to enable multiple executions of the same experiment with different data

Execution ndash This is the actual running of an experiment All variables should be bound to Data Providers before the execution of steps can begin

Vusers ndash These are virtual users that play human users in a specific experiment

Vusers scripts ndash These scripts are the ordered set of actions a Vuser performs during the execution of an experiment In other words the set of instructions carried out during execution without user interven-tion

Atomic step ndash This is the smallest (inseparable) single instruction that is carried out during the execution of a test An atomic step may contain (a) an instruction to be manually performed by a tester (b) a ref-erence to run a Vuser script or (c) an instruction to inject data provided through a variable into FIspace test

Execution log ndash This log is a file that lists actions as occurred during execution including all process and system notifications The entries in an execution log can provide insight into what happened during exe-cution of the test and provide an audit trail of information related to the execution In fact the execu-tion log is the input to the Reporting module in the EE which analyzes the log and provides performance assessment of the execution

Expected results ndashThis is the anticipated outcome of a step in a test

Actual results ndash These are the real outcome of a step as result of execution

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 10 of 34

Key Performance Indicator (KPI) ndash The KPI is a set of performance measurements related to TampL stored in the EE for the sake of performance assessment and analysis The evaluation framework specification is in the scope of Work Package 2 but the KPI(s) related to the performance assessment are stored in the EE and can be used to assess the performance of the test executed

Report ndash The report is a summary of what occurred over one or more test executions A report may in-clude performance assessment of the execution based on given KPI(s)

Injected data ndash Data fed into the test by the backend simulator module in EE is injected data Injected data is used whenever real data in real time cannot be obtained during the execution of a test In these cases the intention is to use (realtime) historical data to simulate the processes

Notifications ndash These are messages given to a user via FIspace frontend during an execution of a test Notifications are recorded in the execution log of the test

12 Class diagrams

Figure 2 Experiment CRUD Search classes

The ExperimentCRUDService class contains all the methods for the CRUD operations on Experiments

The ExperimentSearch class implements all the methods for the searching of Experiments in free text or by field

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 11 of 34

Figure 3 ExecutionManagerService Executor Classes

The Execution ManagerService class handles all the phases of an execution of an experiment It gives access to all the executions taking place or in memory

The Executor class is responsible for executing single steps of an execution and logging the result

Figure 4 ReportManager ResourceManager classes

The ReportManager class is responsible for creating reports after an execution and for giving access to stored reports Also has the responsibility of creating KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 2: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 2 of 34

The FIspace Project

Leveraging on outcomes of two complementary Phase 1 use case projects (FInest amp SmartAgriFood) aim of FIspace is to pioneer towards fundamental changes on how collaborative business networks will work in future FIspace will develop a multi-domain Business Collaboration Space (short FIspace) that employs FI technologies for enabling seamless collaboration in open cross-organizational business net-works establish eight working Experimentation Sites in Europe where Pilot Applications are tested in Early Trials for Agri-Food Transport amp Logistics and prepare for industrial uptake by engaging with play-ers amp associations from relevant industry sectors and IT industry

Project Summary

As a use case project in Phase 2 of the FI PPP FIspace aims at developing and validating novel Future-Internet-enabled solutions to address the pressing challenges arising in collaborative business networks focussing on use cases from the Agri-Food Transport and Logistics industries FIspace will focus on ex-ploiting incorporating and validating the Generic Enablers provided by the FI PPP Core Platform with the aim of realising an extensible collaboration service for business networks together with a set of innovative test applications that allow for radical improvements in how networked businesses can work in the future Those solutions will be demonstrated and tested through early trials on experimentation sites across Eu-rope The project results will be open to the FI PPP program and the general public and the pro-active engagement of larger user communities and external solution providers will foster innovation and indus-trial uptake planned for Phase 3 of the FI PPP

Project Consortium

DLO Netherlands Kuumlhne + Nagel Switzerland

ATB Bremen Germany University Duisburg Essen Germany

IBM Israel ATOS Spain

KocSistem Turkey The Open Group United Kingdom

Aston University United Kingdom CentMa Germany

ENoLL Belgium iMinds Belgium

KTBL Germany Marintek Norway

NKUA Greece University Politecnica Madrid Spain

Wageningen University Netherlands Arcelik Turkey

PlusFresc Spain EuroPoolSystem Germany

FloriCode Netherlands GS1 Germany Germany

Kverneland Netherlands Mieloo amp Alexander Netherlands

North Sea Container Line Norway OPEKEPE Greece

LimeTri Netherlands Innovators Greece

More Information

Harald Sundmaeker (coordinator)

ATB Bremen GmbH

Wiener Str 1

D-28359 Bremen Germany

e-mail Sundmaekeratb-bremende

phone +49 421 220920

wwwFIspaceeu Twitter

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 3 of 34

Dissemination Level

PU Public

PP Restricted to other programme participants (including the Commission Services)

RE Restricted to a group specified by the consortium (including the Commission Services)

CO Confidential only for members of the consortium (including the Commission Services)

Change History

Version Notes Date

001 Creation of the document 30092015

002 Refinement with partner input 11112015

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 4 of 34

Table of Contents

1 Experimentation environment components description 6

11 EE Components statusfunctionality 8

12 Class diagrams 10

13 Database diagram 17

14 Sequence diagrams 18

141 Experiment creation 18

142 Execution of experiment 19

143 Create KPIs 19

144 Backend simulator-(SDI)capabilities usage 20

2 Detailed Usage Guide 21

21 Greenhouse experiment 21

3 Detailed Greenhouse Experiment 22

31 Create a greenhouse experiment with two steps 22

32 Read an Experiment 25

321 Get all experiments 26

322 Search Experiment 27

33 Execution Management of an Experiment 28

34 Report and KPIs based for specific execution 30

4 Further experimentation 30

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario 31

6 References 33

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 5 of 34

List of Figures

Figure 1 Overall EE components overview 6

Figure 2 Experiment CRUD Search classes 10

Figure 3 ExecutionManagerService Executor Classes 11

Figure 4 ReportManager ResourceManager classes 11

Figure 5 ExecutionLogManager ScriptExecutionEngine classes 12

Figure 6 KPI manager KPI Calculator classes 13

Figure 7 KPICalculatorSerializer classes 13

Figure 8 DataProvider Classes 14

Figure 9 DataProviderSystem classes 14

Figure 10 External data access classes 15

Figure 11 Backend simulator classes 16

Figure 12 Experimentation environment Database tables 17

Figure 13 Experiment Creation- search 18

Figure 14 Execute specific experiment 19

Figure 15 Create KPIs 19

Figure 16 Backend simulator usage 20

Figure 17 experiment CRUD 22

Figure 18 Read experiment 25

Figure 19 get all experiments 26

Figure 20 Search experiment 27

Figure 21 Execution start 28

Figure 22 Log results of execution (sensor data and actions alerts from expert system) 29

Figure 23 Reports and KPIs 30

Figure 24 Backend Simulator 31

List of Tables

Table 1 Components description 8

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 6 of 34

1 Experimentation environment components description

The FIspace Experimentation Environment (EE) provides the means to different stakeholders to be able to explore interact with and understand how FIspace platform works in practice Specifically the EE pro-vides a controlled environment in which developers and business architects can test new collaborative scenarios before these processes are released for use by ldquoend usersrdquo Towards this end an overall framework comprising several core modules is used in order to be able for the users to define create and execute their experiments as well as evaluate afterwards the outcomes of the scenario execution The figure that follows (Figure 1) illustrates the EE core framework All EE core modules are implemented and available to the end users In addition almost all core modules are already deployed in one of the scenar-ios that have been used in the context of the EE

Figure 1 Overall EE components overview

A more detailed description for each one of the frameworkrsquos components follows

User Manager The User Manager handles user accounts passwords and access This includes features such as user groups and access control to data as well as users being able to assign other users permis-sions

Experiment Manager The Experiment Manager handles experiment lifecycle and experiment querying This includes creation versioning archiving and search capabilities

Experiment CRUD (Create Read Update and Delete) Provides services for experiment lifecycle using archiving instead of deletion so that traceability is never lost

Experiment Search Provides services for finding experiments according to various search criteria

Execution Manager The Execution Manager handles the concrete executions of an experiment This includes the creation of new executions (including the configuration of variables) executing (or tracking the execution of) the steps in the experiment and logging the results

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 7 of 34

Executor The Executor tracks the execution of the individual steps in an evaluation This in-cludes the automated execution of certain steps such as injecting dataevents into the test in-stance and running VUser scripts through the Script Execution Engine This component also cre-ates and updates entries in the execution log including notifications received from the actual process execution and error messages

Script Execution Engine This executes VUser scripts to automate user actions

Resource Manager The Resource Manager provides an inventory of available resources Services in-clude the ability to locate resources according to various search criteria

Reporting Reports are generated based on execution logs and KPIs

KPI Manager Manages the calculation of KPIs

Execution Log Manager The Execution Log Manager provides logging services for an execution This includes the logging of the results for each step of an execution including any received notifications during the execution of each step Also provides access to these logs

Execution Data Manager This is used to manage the access to data that is used during execution

Internal Data Provider System Used for storing and retrieving manually configured data provid-ers

External Data Access Used to retrieve data from 3rd party external systems could be used to ldquoreplayrdquo events from a real-world shipment for example

Access to these systems is configured by the tester Configuration could be UI or file driven

FIspace Test This is the FIspace public instance which is the replica of a latest stable version of the inte-grated platform hosted on the integration environment depicted on D3006 This instance is updated when the new version of the platform is released FIspace Test platform consists of different modules developed in WP200 are installed and integrated following the instructions provided by the module owners on PIE wiki[1]These instructions are refined by Integration and FIspace Test platform team for the each new release of the platform while integration is executedAfter each integration process inte-gration tests procedures are followed to provide Phase3 developers stable platform Server and services information of each module that constructs the current public version of the FIspace Test platform can be found on the bitbucket EE wikipage[2]

Backend Simulator This simulates input data from backend systems to FIspace and provides APIs to inject data to the FIspace Test systemrsquos modules The simulator reports back on events and other pro-cessed data

EE Storage This provides internal storage services for the experimentation environment

Data Provider System (app) ndash This component addresses the binding of the application to the execution data

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 8 of 34

11 EE Components statusfunctionality

Table 1 Components description

Core Component Status Functionality

ExecutionManagerService Core functionality pro-vided by NKUA

It currently starts a new execution according to an experimentstores it and provides the functionality to search for executions according to experimentidcreator Also calls Executor for the execution of StepsSteps can now be running for as time as needed

Executor Core functionality pro-vided by NKUA

Executes an experiments step and logs its result

ScriptExecutionEngine Core functionality pro-vided by NKUA for Greenhouse experiment

Executes a steps REST call or jar

BackEndSimulator Core functionality pro-vided by NKUA

First version of connecting to SDI using keycloak Abstract interface created for various backends Greenhouse backend simulator development in progress

ExecutionLogManager Core functionality pro-vided by NKUA

Logs a steps result and saves it to persistent storageAlso retrieves logs for a specific execution

Experiment CRUD Core functionality pro-vided by NKUA

Create an experiment to persistent storage retrieve it update it

Experiment Search Core functionality pro-vided by NKUA

Search by name sample using full text

ReportManager Completed for the Greenhouse Advice Re-quest scenario

Create report get report by exe-cution or experiment and search report by namedescription

KPI Manager Basic Functionality im-plemented defined the public interface of the component and storage in internal DB

Sample createread kpis Various KPIs available

User Manager Users log in through IDM for accessing UI

Users log in through IDM for ac-cessing UI DigitalSelf IDM imple-mentation is used Spring security is integrated with UI for authoriza-tion mechanisms

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 9 of 34

Core Component Status Functionality

Execution Data Manager Core functionality pro-vided by NKUA for Greenhouse experiment

In an execution data management a process can addretrieve a data provider system A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

Resource Manager Core functionality pro-vided by NKUA for Greenhouse experiment

A resource can be created and retrieved

The proposed technical specification of the EE enables the entire process from testexperiment plan-ning and configuration through execution to analysis of the test execution We introduce below terms to be used throughout this report

Step ndash A step refers to a single actiontask defined in a test scenario

Test scenario ndash This is the ordered set of steps that compose a single test

Variables ndash In the context of a test these are field names that stand for specific values during execution Variables enable flexibility in test execution as they enable running the same test with different field values

Variables binding ndash This refers to the replacement of variables values with the test data This is done by the experimenter during test execution

Experimenttest ndash This is the ordered set of steps to be carried out by an experimenter during execution Each experiment is identified by a unique ID and version An experiment may have variables to enable multiple executions of the same experiment with different data

Execution ndash This is the actual running of an experiment All variables should be bound to Data Providers before the execution of steps can begin

Vusers ndash These are virtual users that play human users in a specific experiment

Vusers scripts ndash These scripts are the ordered set of actions a Vuser performs during the execution of an experiment In other words the set of instructions carried out during execution without user interven-tion

Atomic step ndash This is the smallest (inseparable) single instruction that is carried out during the execution of a test An atomic step may contain (a) an instruction to be manually performed by a tester (b) a ref-erence to run a Vuser script or (c) an instruction to inject data provided through a variable into FIspace test

Execution log ndash This log is a file that lists actions as occurred during execution including all process and system notifications The entries in an execution log can provide insight into what happened during exe-cution of the test and provide an audit trail of information related to the execution In fact the execu-tion log is the input to the Reporting module in the EE which analyzes the log and provides performance assessment of the execution

Expected results ndashThis is the anticipated outcome of a step in a test

Actual results ndash These are the real outcome of a step as result of execution

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 10 of 34

Key Performance Indicator (KPI) ndash The KPI is a set of performance measurements related to TampL stored in the EE for the sake of performance assessment and analysis The evaluation framework specification is in the scope of Work Package 2 but the KPI(s) related to the performance assessment are stored in the EE and can be used to assess the performance of the test executed

Report ndash The report is a summary of what occurred over one or more test executions A report may in-clude performance assessment of the execution based on given KPI(s)

Injected data ndash Data fed into the test by the backend simulator module in EE is injected data Injected data is used whenever real data in real time cannot be obtained during the execution of a test In these cases the intention is to use (realtime) historical data to simulate the processes

Notifications ndash These are messages given to a user via FIspace frontend during an execution of a test Notifications are recorded in the execution log of the test

12 Class diagrams

Figure 2 Experiment CRUD Search classes

The ExperimentCRUDService class contains all the methods for the CRUD operations on Experiments

The ExperimentSearch class implements all the methods for the searching of Experiments in free text or by field

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 11 of 34

Figure 3 ExecutionManagerService Executor Classes

The Execution ManagerService class handles all the phases of an execution of an experiment It gives access to all the executions taking place or in memory

The Executor class is responsible for executing single steps of an execution and logging the result

Figure 4 ReportManager ResourceManager classes

The ReportManager class is responsible for creating reports after an execution and for giving access to stored reports Also has the responsibility of creating KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 3: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 3 of 34

Dissemination Level

PU Public

PP Restricted to other programme participants (including the Commission Services)

RE Restricted to a group specified by the consortium (including the Commission Services)

CO Confidential only for members of the consortium (including the Commission Services)

Change History

Version Notes Date

001 Creation of the document 30092015

002 Refinement with partner input 11112015

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 4 of 34

Table of Contents

1 Experimentation environment components description 6

11 EE Components statusfunctionality 8

12 Class diagrams 10

13 Database diagram 17

14 Sequence diagrams 18

141 Experiment creation 18

142 Execution of experiment 19

143 Create KPIs 19

144 Backend simulator-(SDI)capabilities usage 20

2 Detailed Usage Guide 21

21 Greenhouse experiment 21

3 Detailed Greenhouse Experiment 22

31 Create a greenhouse experiment with two steps 22

32 Read an Experiment 25

321 Get all experiments 26

322 Search Experiment 27

33 Execution Management of an Experiment 28

34 Report and KPIs based for specific execution 30

4 Further experimentation 30

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario 31

6 References 33

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 5 of 34

List of Figures

Figure 1 Overall EE components overview 6

Figure 2 Experiment CRUD Search classes 10

Figure 3 ExecutionManagerService Executor Classes 11

Figure 4 ReportManager ResourceManager classes 11

Figure 5 ExecutionLogManager ScriptExecutionEngine classes 12

Figure 6 KPI manager KPI Calculator classes 13

Figure 7 KPICalculatorSerializer classes 13

Figure 8 DataProvider Classes 14

Figure 9 DataProviderSystem classes 14

Figure 10 External data access classes 15

Figure 11 Backend simulator classes 16

Figure 12 Experimentation environment Database tables 17

Figure 13 Experiment Creation- search 18

Figure 14 Execute specific experiment 19

Figure 15 Create KPIs 19

Figure 16 Backend simulator usage 20

Figure 17 experiment CRUD 22

Figure 18 Read experiment 25

Figure 19 get all experiments 26

Figure 20 Search experiment 27

Figure 21 Execution start 28

Figure 22 Log results of execution (sensor data and actions alerts from expert system) 29

Figure 23 Reports and KPIs 30

Figure 24 Backend Simulator 31

List of Tables

Table 1 Components description 8

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 6 of 34

1 Experimentation environment components description

The FIspace Experimentation Environment (EE) provides the means to different stakeholders to be able to explore interact with and understand how FIspace platform works in practice Specifically the EE pro-vides a controlled environment in which developers and business architects can test new collaborative scenarios before these processes are released for use by ldquoend usersrdquo Towards this end an overall framework comprising several core modules is used in order to be able for the users to define create and execute their experiments as well as evaluate afterwards the outcomes of the scenario execution The figure that follows (Figure 1) illustrates the EE core framework All EE core modules are implemented and available to the end users In addition almost all core modules are already deployed in one of the scenar-ios that have been used in the context of the EE

Figure 1 Overall EE components overview

A more detailed description for each one of the frameworkrsquos components follows

User Manager The User Manager handles user accounts passwords and access This includes features such as user groups and access control to data as well as users being able to assign other users permis-sions

Experiment Manager The Experiment Manager handles experiment lifecycle and experiment querying This includes creation versioning archiving and search capabilities

Experiment CRUD (Create Read Update and Delete) Provides services for experiment lifecycle using archiving instead of deletion so that traceability is never lost

Experiment Search Provides services for finding experiments according to various search criteria

Execution Manager The Execution Manager handles the concrete executions of an experiment This includes the creation of new executions (including the configuration of variables) executing (or tracking the execution of) the steps in the experiment and logging the results

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 7 of 34

Executor The Executor tracks the execution of the individual steps in an evaluation This in-cludes the automated execution of certain steps such as injecting dataevents into the test in-stance and running VUser scripts through the Script Execution Engine This component also cre-ates and updates entries in the execution log including notifications received from the actual process execution and error messages

Script Execution Engine This executes VUser scripts to automate user actions

Resource Manager The Resource Manager provides an inventory of available resources Services in-clude the ability to locate resources according to various search criteria

Reporting Reports are generated based on execution logs and KPIs

KPI Manager Manages the calculation of KPIs

Execution Log Manager The Execution Log Manager provides logging services for an execution This includes the logging of the results for each step of an execution including any received notifications during the execution of each step Also provides access to these logs

Execution Data Manager This is used to manage the access to data that is used during execution

Internal Data Provider System Used for storing and retrieving manually configured data provid-ers

External Data Access Used to retrieve data from 3rd party external systems could be used to ldquoreplayrdquo events from a real-world shipment for example

Access to these systems is configured by the tester Configuration could be UI or file driven

FIspace Test This is the FIspace public instance which is the replica of a latest stable version of the inte-grated platform hosted on the integration environment depicted on D3006 This instance is updated when the new version of the platform is released FIspace Test platform consists of different modules developed in WP200 are installed and integrated following the instructions provided by the module owners on PIE wiki[1]These instructions are refined by Integration and FIspace Test platform team for the each new release of the platform while integration is executedAfter each integration process inte-gration tests procedures are followed to provide Phase3 developers stable platform Server and services information of each module that constructs the current public version of the FIspace Test platform can be found on the bitbucket EE wikipage[2]

Backend Simulator This simulates input data from backend systems to FIspace and provides APIs to inject data to the FIspace Test systemrsquos modules The simulator reports back on events and other pro-cessed data

EE Storage This provides internal storage services for the experimentation environment

Data Provider System (app) ndash This component addresses the binding of the application to the execution data

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 8 of 34

11 EE Components statusfunctionality

Table 1 Components description

Core Component Status Functionality

ExecutionManagerService Core functionality pro-vided by NKUA

It currently starts a new execution according to an experimentstores it and provides the functionality to search for executions according to experimentidcreator Also calls Executor for the execution of StepsSteps can now be running for as time as needed

Executor Core functionality pro-vided by NKUA

Executes an experiments step and logs its result

ScriptExecutionEngine Core functionality pro-vided by NKUA for Greenhouse experiment

Executes a steps REST call or jar

BackEndSimulator Core functionality pro-vided by NKUA

First version of connecting to SDI using keycloak Abstract interface created for various backends Greenhouse backend simulator development in progress

ExecutionLogManager Core functionality pro-vided by NKUA

Logs a steps result and saves it to persistent storageAlso retrieves logs for a specific execution

Experiment CRUD Core functionality pro-vided by NKUA

Create an experiment to persistent storage retrieve it update it

Experiment Search Core functionality pro-vided by NKUA

Search by name sample using full text

ReportManager Completed for the Greenhouse Advice Re-quest scenario

Create report get report by exe-cution or experiment and search report by namedescription

KPI Manager Basic Functionality im-plemented defined the public interface of the component and storage in internal DB

Sample createread kpis Various KPIs available

User Manager Users log in through IDM for accessing UI

Users log in through IDM for ac-cessing UI DigitalSelf IDM imple-mentation is used Spring security is integrated with UI for authoriza-tion mechanisms

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 9 of 34

Core Component Status Functionality

Execution Data Manager Core functionality pro-vided by NKUA for Greenhouse experiment

In an execution data management a process can addretrieve a data provider system A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

Resource Manager Core functionality pro-vided by NKUA for Greenhouse experiment

A resource can be created and retrieved

The proposed technical specification of the EE enables the entire process from testexperiment plan-ning and configuration through execution to analysis of the test execution We introduce below terms to be used throughout this report

Step ndash A step refers to a single actiontask defined in a test scenario

Test scenario ndash This is the ordered set of steps that compose a single test

Variables ndash In the context of a test these are field names that stand for specific values during execution Variables enable flexibility in test execution as they enable running the same test with different field values

Variables binding ndash This refers to the replacement of variables values with the test data This is done by the experimenter during test execution

Experimenttest ndash This is the ordered set of steps to be carried out by an experimenter during execution Each experiment is identified by a unique ID and version An experiment may have variables to enable multiple executions of the same experiment with different data

Execution ndash This is the actual running of an experiment All variables should be bound to Data Providers before the execution of steps can begin

Vusers ndash These are virtual users that play human users in a specific experiment

Vusers scripts ndash These scripts are the ordered set of actions a Vuser performs during the execution of an experiment In other words the set of instructions carried out during execution without user interven-tion

Atomic step ndash This is the smallest (inseparable) single instruction that is carried out during the execution of a test An atomic step may contain (a) an instruction to be manually performed by a tester (b) a ref-erence to run a Vuser script or (c) an instruction to inject data provided through a variable into FIspace test

Execution log ndash This log is a file that lists actions as occurred during execution including all process and system notifications The entries in an execution log can provide insight into what happened during exe-cution of the test and provide an audit trail of information related to the execution In fact the execu-tion log is the input to the Reporting module in the EE which analyzes the log and provides performance assessment of the execution

Expected results ndashThis is the anticipated outcome of a step in a test

Actual results ndash These are the real outcome of a step as result of execution

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 10 of 34

Key Performance Indicator (KPI) ndash The KPI is a set of performance measurements related to TampL stored in the EE for the sake of performance assessment and analysis The evaluation framework specification is in the scope of Work Package 2 but the KPI(s) related to the performance assessment are stored in the EE and can be used to assess the performance of the test executed

Report ndash The report is a summary of what occurred over one or more test executions A report may in-clude performance assessment of the execution based on given KPI(s)

Injected data ndash Data fed into the test by the backend simulator module in EE is injected data Injected data is used whenever real data in real time cannot be obtained during the execution of a test In these cases the intention is to use (realtime) historical data to simulate the processes

Notifications ndash These are messages given to a user via FIspace frontend during an execution of a test Notifications are recorded in the execution log of the test

12 Class diagrams

Figure 2 Experiment CRUD Search classes

The ExperimentCRUDService class contains all the methods for the CRUD operations on Experiments

The ExperimentSearch class implements all the methods for the searching of Experiments in free text or by field

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 11 of 34

Figure 3 ExecutionManagerService Executor Classes

The Execution ManagerService class handles all the phases of an execution of an experiment It gives access to all the executions taking place or in memory

The Executor class is responsible for executing single steps of an execution and logging the result

Figure 4 ReportManager ResourceManager classes

The ReportManager class is responsible for creating reports after an execution and for giving access to stored reports Also has the responsibility of creating KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 4: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 4 of 34

Table of Contents

1 Experimentation environment components description 6

11 EE Components statusfunctionality 8

12 Class diagrams 10

13 Database diagram 17

14 Sequence diagrams 18

141 Experiment creation 18

142 Execution of experiment 19

143 Create KPIs 19

144 Backend simulator-(SDI)capabilities usage 20

2 Detailed Usage Guide 21

21 Greenhouse experiment 21

3 Detailed Greenhouse Experiment 22

31 Create a greenhouse experiment with two steps 22

32 Read an Experiment 25

321 Get all experiments 26

322 Search Experiment 27

33 Execution Management of an Experiment 28

34 Report and KPIs based for specific execution 30

4 Further experimentation 30

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario 31

6 References 33

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 5 of 34

List of Figures

Figure 1 Overall EE components overview 6

Figure 2 Experiment CRUD Search classes 10

Figure 3 ExecutionManagerService Executor Classes 11

Figure 4 ReportManager ResourceManager classes 11

Figure 5 ExecutionLogManager ScriptExecutionEngine classes 12

Figure 6 KPI manager KPI Calculator classes 13

Figure 7 KPICalculatorSerializer classes 13

Figure 8 DataProvider Classes 14

Figure 9 DataProviderSystem classes 14

Figure 10 External data access classes 15

Figure 11 Backend simulator classes 16

Figure 12 Experimentation environment Database tables 17

Figure 13 Experiment Creation- search 18

Figure 14 Execute specific experiment 19

Figure 15 Create KPIs 19

Figure 16 Backend simulator usage 20

Figure 17 experiment CRUD 22

Figure 18 Read experiment 25

Figure 19 get all experiments 26

Figure 20 Search experiment 27

Figure 21 Execution start 28

Figure 22 Log results of execution (sensor data and actions alerts from expert system) 29

Figure 23 Reports and KPIs 30

Figure 24 Backend Simulator 31

List of Tables

Table 1 Components description 8

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 6 of 34

1 Experimentation environment components description

The FIspace Experimentation Environment (EE) provides the means to different stakeholders to be able to explore interact with and understand how FIspace platform works in practice Specifically the EE pro-vides a controlled environment in which developers and business architects can test new collaborative scenarios before these processes are released for use by ldquoend usersrdquo Towards this end an overall framework comprising several core modules is used in order to be able for the users to define create and execute their experiments as well as evaluate afterwards the outcomes of the scenario execution The figure that follows (Figure 1) illustrates the EE core framework All EE core modules are implemented and available to the end users In addition almost all core modules are already deployed in one of the scenar-ios that have been used in the context of the EE

Figure 1 Overall EE components overview

A more detailed description for each one of the frameworkrsquos components follows

User Manager The User Manager handles user accounts passwords and access This includes features such as user groups and access control to data as well as users being able to assign other users permis-sions

Experiment Manager The Experiment Manager handles experiment lifecycle and experiment querying This includes creation versioning archiving and search capabilities

Experiment CRUD (Create Read Update and Delete) Provides services for experiment lifecycle using archiving instead of deletion so that traceability is never lost

Experiment Search Provides services for finding experiments according to various search criteria

Execution Manager The Execution Manager handles the concrete executions of an experiment This includes the creation of new executions (including the configuration of variables) executing (or tracking the execution of) the steps in the experiment and logging the results

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 7 of 34

Executor The Executor tracks the execution of the individual steps in an evaluation This in-cludes the automated execution of certain steps such as injecting dataevents into the test in-stance and running VUser scripts through the Script Execution Engine This component also cre-ates and updates entries in the execution log including notifications received from the actual process execution and error messages

Script Execution Engine This executes VUser scripts to automate user actions

Resource Manager The Resource Manager provides an inventory of available resources Services in-clude the ability to locate resources according to various search criteria

Reporting Reports are generated based on execution logs and KPIs

KPI Manager Manages the calculation of KPIs

Execution Log Manager The Execution Log Manager provides logging services for an execution This includes the logging of the results for each step of an execution including any received notifications during the execution of each step Also provides access to these logs

Execution Data Manager This is used to manage the access to data that is used during execution

Internal Data Provider System Used for storing and retrieving manually configured data provid-ers

External Data Access Used to retrieve data from 3rd party external systems could be used to ldquoreplayrdquo events from a real-world shipment for example

Access to these systems is configured by the tester Configuration could be UI or file driven

FIspace Test This is the FIspace public instance which is the replica of a latest stable version of the inte-grated platform hosted on the integration environment depicted on D3006 This instance is updated when the new version of the platform is released FIspace Test platform consists of different modules developed in WP200 are installed and integrated following the instructions provided by the module owners on PIE wiki[1]These instructions are refined by Integration and FIspace Test platform team for the each new release of the platform while integration is executedAfter each integration process inte-gration tests procedures are followed to provide Phase3 developers stable platform Server and services information of each module that constructs the current public version of the FIspace Test platform can be found on the bitbucket EE wikipage[2]

Backend Simulator This simulates input data from backend systems to FIspace and provides APIs to inject data to the FIspace Test systemrsquos modules The simulator reports back on events and other pro-cessed data

EE Storage This provides internal storage services for the experimentation environment

Data Provider System (app) ndash This component addresses the binding of the application to the execution data

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 8 of 34

11 EE Components statusfunctionality

Table 1 Components description

Core Component Status Functionality

ExecutionManagerService Core functionality pro-vided by NKUA

It currently starts a new execution according to an experimentstores it and provides the functionality to search for executions according to experimentidcreator Also calls Executor for the execution of StepsSteps can now be running for as time as needed

Executor Core functionality pro-vided by NKUA

Executes an experiments step and logs its result

ScriptExecutionEngine Core functionality pro-vided by NKUA for Greenhouse experiment

Executes a steps REST call or jar

BackEndSimulator Core functionality pro-vided by NKUA

First version of connecting to SDI using keycloak Abstract interface created for various backends Greenhouse backend simulator development in progress

ExecutionLogManager Core functionality pro-vided by NKUA

Logs a steps result and saves it to persistent storageAlso retrieves logs for a specific execution

Experiment CRUD Core functionality pro-vided by NKUA

Create an experiment to persistent storage retrieve it update it

Experiment Search Core functionality pro-vided by NKUA

Search by name sample using full text

ReportManager Completed for the Greenhouse Advice Re-quest scenario

Create report get report by exe-cution or experiment and search report by namedescription

KPI Manager Basic Functionality im-plemented defined the public interface of the component and storage in internal DB

Sample createread kpis Various KPIs available

User Manager Users log in through IDM for accessing UI

Users log in through IDM for ac-cessing UI DigitalSelf IDM imple-mentation is used Spring security is integrated with UI for authoriza-tion mechanisms

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 9 of 34

Core Component Status Functionality

Execution Data Manager Core functionality pro-vided by NKUA for Greenhouse experiment

In an execution data management a process can addretrieve a data provider system A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

Resource Manager Core functionality pro-vided by NKUA for Greenhouse experiment

A resource can be created and retrieved

The proposed technical specification of the EE enables the entire process from testexperiment plan-ning and configuration through execution to analysis of the test execution We introduce below terms to be used throughout this report

Step ndash A step refers to a single actiontask defined in a test scenario

Test scenario ndash This is the ordered set of steps that compose a single test

Variables ndash In the context of a test these are field names that stand for specific values during execution Variables enable flexibility in test execution as they enable running the same test with different field values

Variables binding ndash This refers to the replacement of variables values with the test data This is done by the experimenter during test execution

Experimenttest ndash This is the ordered set of steps to be carried out by an experimenter during execution Each experiment is identified by a unique ID and version An experiment may have variables to enable multiple executions of the same experiment with different data

Execution ndash This is the actual running of an experiment All variables should be bound to Data Providers before the execution of steps can begin

Vusers ndash These are virtual users that play human users in a specific experiment

Vusers scripts ndash These scripts are the ordered set of actions a Vuser performs during the execution of an experiment In other words the set of instructions carried out during execution without user interven-tion

Atomic step ndash This is the smallest (inseparable) single instruction that is carried out during the execution of a test An atomic step may contain (a) an instruction to be manually performed by a tester (b) a ref-erence to run a Vuser script or (c) an instruction to inject data provided through a variable into FIspace test

Execution log ndash This log is a file that lists actions as occurred during execution including all process and system notifications The entries in an execution log can provide insight into what happened during exe-cution of the test and provide an audit trail of information related to the execution In fact the execu-tion log is the input to the Reporting module in the EE which analyzes the log and provides performance assessment of the execution

Expected results ndashThis is the anticipated outcome of a step in a test

Actual results ndash These are the real outcome of a step as result of execution

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 10 of 34

Key Performance Indicator (KPI) ndash The KPI is a set of performance measurements related to TampL stored in the EE for the sake of performance assessment and analysis The evaluation framework specification is in the scope of Work Package 2 but the KPI(s) related to the performance assessment are stored in the EE and can be used to assess the performance of the test executed

Report ndash The report is a summary of what occurred over one or more test executions A report may in-clude performance assessment of the execution based on given KPI(s)

Injected data ndash Data fed into the test by the backend simulator module in EE is injected data Injected data is used whenever real data in real time cannot be obtained during the execution of a test In these cases the intention is to use (realtime) historical data to simulate the processes

Notifications ndash These are messages given to a user via FIspace frontend during an execution of a test Notifications are recorded in the execution log of the test

12 Class diagrams

Figure 2 Experiment CRUD Search classes

The ExperimentCRUDService class contains all the methods for the CRUD operations on Experiments

The ExperimentSearch class implements all the methods for the searching of Experiments in free text or by field

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 11 of 34

Figure 3 ExecutionManagerService Executor Classes

The Execution ManagerService class handles all the phases of an execution of an experiment It gives access to all the executions taking place or in memory

The Executor class is responsible for executing single steps of an execution and logging the result

Figure 4 ReportManager ResourceManager classes

The ReportManager class is responsible for creating reports after an execution and for giving access to stored reports Also has the responsibility of creating KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 5: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 5 of 34

List of Figures

Figure 1 Overall EE components overview 6

Figure 2 Experiment CRUD Search classes 10

Figure 3 ExecutionManagerService Executor Classes 11

Figure 4 ReportManager ResourceManager classes 11

Figure 5 ExecutionLogManager ScriptExecutionEngine classes 12

Figure 6 KPI manager KPI Calculator classes 13

Figure 7 KPICalculatorSerializer classes 13

Figure 8 DataProvider Classes 14

Figure 9 DataProviderSystem classes 14

Figure 10 External data access classes 15

Figure 11 Backend simulator classes 16

Figure 12 Experimentation environment Database tables 17

Figure 13 Experiment Creation- search 18

Figure 14 Execute specific experiment 19

Figure 15 Create KPIs 19

Figure 16 Backend simulator usage 20

Figure 17 experiment CRUD 22

Figure 18 Read experiment 25

Figure 19 get all experiments 26

Figure 20 Search experiment 27

Figure 21 Execution start 28

Figure 22 Log results of execution (sensor data and actions alerts from expert system) 29

Figure 23 Reports and KPIs 30

Figure 24 Backend Simulator 31

List of Tables

Table 1 Components description 8

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 6 of 34

1 Experimentation environment components description

The FIspace Experimentation Environment (EE) provides the means to different stakeholders to be able to explore interact with and understand how FIspace platform works in practice Specifically the EE pro-vides a controlled environment in which developers and business architects can test new collaborative scenarios before these processes are released for use by ldquoend usersrdquo Towards this end an overall framework comprising several core modules is used in order to be able for the users to define create and execute their experiments as well as evaluate afterwards the outcomes of the scenario execution The figure that follows (Figure 1) illustrates the EE core framework All EE core modules are implemented and available to the end users In addition almost all core modules are already deployed in one of the scenar-ios that have been used in the context of the EE

Figure 1 Overall EE components overview

A more detailed description for each one of the frameworkrsquos components follows

User Manager The User Manager handles user accounts passwords and access This includes features such as user groups and access control to data as well as users being able to assign other users permis-sions

Experiment Manager The Experiment Manager handles experiment lifecycle and experiment querying This includes creation versioning archiving and search capabilities

Experiment CRUD (Create Read Update and Delete) Provides services for experiment lifecycle using archiving instead of deletion so that traceability is never lost

Experiment Search Provides services for finding experiments according to various search criteria

Execution Manager The Execution Manager handles the concrete executions of an experiment This includes the creation of new executions (including the configuration of variables) executing (or tracking the execution of) the steps in the experiment and logging the results

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 7 of 34

Executor The Executor tracks the execution of the individual steps in an evaluation This in-cludes the automated execution of certain steps such as injecting dataevents into the test in-stance and running VUser scripts through the Script Execution Engine This component also cre-ates and updates entries in the execution log including notifications received from the actual process execution and error messages

Script Execution Engine This executes VUser scripts to automate user actions

Resource Manager The Resource Manager provides an inventory of available resources Services in-clude the ability to locate resources according to various search criteria

Reporting Reports are generated based on execution logs and KPIs

KPI Manager Manages the calculation of KPIs

Execution Log Manager The Execution Log Manager provides logging services for an execution This includes the logging of the results for each step of an execution including any received notifications during the execution of each step Also provides access to these logs

Execution Data Manager This is used to manage the access to data that is used during execution

Internal Data Provider System Used for storing and retrieving manually configured data provid-ers

External Data Access Used to retrieve data from 3rd party external systems could be used to ldquoreplayrdquo events from a real-world shipment for example

Access to these systems is configured by the tester Configuration could be UI or file driven

FIspace Test This is the FIspace public instance which is the replica of a latest stable version of the inte-grated platform hosted on the integration environment depicted on D3006 This instance is updated when the new version of the platform is released FIspace Test platform consists of different modules developed in WP200 are installed and integrated following the instructions provided by the module owners on PIE wiki[1]These instructions are refined by Integration and FIspace Test platform team for the each new release of the platform while integration is executedAfter each integration process inte-gration tests procedures are followed to provide Phase3 developers stable platform Server and services information of each module that constructs the current public version of the FIspace Test platform can be found on the bitbucket EE wikipage[2]

Backend Simulator This simulates input data from backend systems to FIspace and provides APIs to inject data to the FIspace Test systemrsquos modules The simulator reports back on events and other pro-cessed data

EE Storage This provides internal storage services for the experimentation environment

Data Provider System (app) ndash This component addresses the binding of the application to the execution data

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 8 of 34

11 EE Components statusfunctionality

Table 1 Components description

Core Component Status Functionality

ExecutionManagerService Core functionality pro-vided by NKUA

It currently starts a new execution according to an experimentstores it and provides the functionality to search for executions according to experimentidcreator Also calls Executor for the execution of StepsSteps can now be running for as time as needed

Executor Core functionality pro-vided by NKUA

Executes an experiments step and logs its result

ScriptExecutionEngine Core functionality pro-vided by NKUA for Greenhouse experiment

Executes a steps REST call or jar

BackEndSimulator Core functionality pro-vided by NKUA

First version of connecting to SDI using keycloak Abstract interface created for various backends Greenhouse backend simulator development in progress

ExecutionLogManager Core functionality pro-vided by NKUA

Logs a steps result and saves it to persistent storageAlso retrieves logs for a specific execution

Experiment CRUD Core functionality pro-vided by NKUA

Create an experiment to persistent storage retrieve it update it

Experiment Search Core functionality pro-vided by NKUA

Search by name sample using full text

ReportManager Completed for the Greenhouse Advice Re-quest scenario

Create report get report by exe-cution or experiment and search report by namedescription

KPI Manager Basic Functionality im-plemented defined the public interface of the component and storage in internal DB

Sample createread kpis Various KPIs available

User Manager Users log in through IDM for accessing UI

Users log in through IDM for ac-cessing UI DigitalSelf IDM imple-mentation is used Spring security is integrated with UI for authoriza-tion mechanisms

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 9 of 34

Core Component Status Functionality

Execution Data Manager Core functionality pro-vided by NKUA for Greenhouse experiment

In an execution data management a process can addretrieve a data provider system A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

Resource Manager Core functionality pro-vided by NKUA for Greenhouse experiment

A resource can be created and retrieved

The proposed technical specification of the EE enables the entire process from testexperiment plan-ning and configuration through execution to analysis of the test execution We introduce below terms to be used throughout this report

Step ndash A step refers to a single actiontask defined in a test scenario

Test scenario ndash This is the ordered set of steps that compose a single test

Variables ndash In the context of a test these are field names that stand for specific values during execution Variables enable flexibility in test execution as they enable running the same test with different field values

Variables binding ndash This refers to the replacement of variables values with the test data This is done by the experimenter during test execution

Experimenttest ndash This is the ordered set of steps to be carried out by an experimenter during execution Each experiment is identified by a unique ID and version An experiment may have variables to enable multiple executions of the same experiment with different data

Execution ndash This is the actual running of an experiment All variables should be bound to Data Providers before the execution of steps can begin

Vusers ndash These are virtual users that play human users in a specific experiment

Vusers scripts ndash These scripts are the ordered set of actions a Vuser performs during the execution of an experiment In other words the set of instructions carried out during execution without user interven-tion

Atomic step ndash This is the smallest (inseparable) single instruction that is carried out during the execution of a test An atomic step may contain (a) an instruction to be manually performed by a tester (b) a ref-erence to run a Vuser script or (c) an instruction to inject data provided through a variable into FIspace test

Execution log ndash This log is a file that lists actions as occurred during execution including all process and system notifications The entries in an execution log can provide insight into what happened during exe-cution of the test and provide an audit trail of information related to the execution In fact the execu-tion log is the input to the Reporting module in the EE which analyzes the log and provides performance assessment of the execution

Expected results ndashThis is the anticipated outcome of a step in a test

Actual results ndash These are the real outcome of a step as result of execution

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 10 of 34

Key Performance Indicator (KPI) ndash The KPI is a set of performance measurements related to TampL stored in the EE for the sake of performance assessment and analysis The evaluation framework specification is in the scope of Work Package 2 but the KPI(s) related to the performance assessment are stored in the EE and can be used to assess the performance of the test executed

Report ndash The report is a summary of what occurred over one or more test executions A report may in-clude performance assessment of the execution based on given KPI(s)

Injected data ndash Data fed into the test by the backend simulator module in EE is injected data Injected data is used whenever real data in real time cannot be obtained during the execution of a test In these cases the intention is to use (realtime) historical data to simulate the processes

Notifications ndash These are messages given to a user via FIspace frontend during an execution of a test Notifications are recorded in the execution log of the test

12 Class diagrams

Figure 2 Experiment CRUD Search classes

The ExperimentCRUDService class contains all the methods for the CRUD operations on Experiments

The ExperimentSearch class implements all the methods for the searching of Experiments in free text or by field

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 11 of 34

Figure 3 ExecutionManagerService Executor Classes

The Execution ManagerService class handles all the phases of an execution of an experiment It gives access to all the executions taking place or in memory

The Executor class is responsible for executing single steps of an execution and logging the result

Figure 4 ReportManager ResourceManager classes

The ReportManager class is responsible for creating reports after an execution and for giving access to stored reports Also has the responsibility of creating KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 6: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 6 of 34

1 Experimentation environment components description

The FIspace Experimentation Environment (EE) provides the means to different stakeholders to be able to explore interact with and understand how FIspace platform works in practice Specifically the EE pro-vides a controlled environment in which developers and business architects can test new collaborative scenarios before these processes are released for use by ldquoend usersrdquo Towards this end an overall framework comprising several core modules is used in order to be able for the users to define create and execute their experiments as well as evaluate afterwards the outcomes of the scenario execution The figure that follows (Figure 1) illustrates the EE core framework All EE core modules are implemented and available to the end users In addition almost all core modules are already deployed in one of the scenar-ios that have been used in the context of the EE

Figure 1 Overall EE components overview

A more detailed description for each one of the frameworkrsquos components follows

User Manager The User Manager handles user accounts passwords and access This includes features such as user groups and access control to data as well as users being able to assign other users permis-sions

Experiment Manager The Experiment Manager handles experiment lifecycle and experiment querying This includes creation versioning archiving and search capabilities

Experiment CRUD (Create Read Update and Delete) Provides services for experiment lifecycle using archiving instead of deletion so that traceability is never lost

Experiment Search Provides services for finding experiments according to various search criteria

Execution Manager The Execution Manager handles the concrete executions of an experiment This includes the creation of new executions (including the configuration of variables) executing (or tracking the execution of) the steps in the experiment and logging the results

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 7 of 34

Executor The Executor tracks the execution of the individual steps in an evaluation This in-cludes the automated execution of certain steps such as injecting dataevents into the test in-stance and running VUser scripts through the Script Execution Engine This component also cre-ates and updates entries in the execution log including notifications received from the actual process execution and error messages

Script Execution Engine This executes VUser scripts to automate user actions

Resource Manager The Resource Manager provides an inventory of available resources Services in-clude the ability to locate resources according to various search criteria

Reporting Reports are generated based on execution logs and KPIs

KPI Manager Manages the calculation of KPIs

Execution Log Manager The Execution Log Manager provides logging services for an execution This includes the logging of the results for each step of an execution including any received notifications during the execution of each step Also provides access to these logs

Execution Data Manager This is used to manage the access to data that is used during execution

Internal Data Provider System Used for storing and retrieving manually configured data provid-ers

External Data Access Used to retrieve data from 3rd party external systems could be used to ldquoreplayrdquo events from a real-world shipment for example

Access to these systems is configured by the tester Configuration could be UI or file driven

FIspace Test This is the FIspace public instance which is the replica of a latest stable version of the inte-grated platform hosted on the integration environment depicted on D3006 This instance is updated when the new version of the platform is released FIspace Test platform consists of different modules developed in WP200 are installed and integrated following the instructions provided by the module owners on PIE wiki[1]These instructions are refined by Integration and FIspace Test platform team for the each new release of the platform while integration is executedAfter each integration process inte-gration tests procedures are followed to provide Phase3 developers stable platform Server and services information of each module that constructs the current public version of the FIspace Test platform can be found on the bitbucket EE wikipage[2]

Backend Simulator This simulates input data from backend systems to FIspace and provides APIs to inject data to the FIspace Test systemrsquos modules The simulator reports back on events and other pro-cessed data

EE Storage This provides internal storage services for the experimentation environment

Data Provider System (app) ndash This component addresses the binding of the application to the execution data

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 8 of 34

11 EE Components statusfunctionality

Table 1 Components description

Core Component Status Functionality

ExecutionManagerService Core functionality pro-vided by NKUA

It currently starts a new execution according to an experimentstores it and provides the functionality to search for executions according to experimentidcreator Also calls Executor for the execution of StepsSteps can now be running for as time as needed

Executor Core functionality pro-vided by NKUA

Executes an experiments step and logs its result

ScriptExecutionEngine Core functionality pro-vided by NKUA for Greenhouse experiment

Executes a steps REST call or jar

BackEndSimulator Core functionality pro-vided by NKUA

First version of connecting to SDI using keycloak Abstract interface created for various backends Greenhouse backend simulator development in progress

ExecutionLogManager Core functionality pro-vided by NKUA

Logs a steps result and saves it to persistent storageAlso retrieves logs for a specific execution

Experiment CRUD Core functionality pro-vided by NKUA

Create an experiment to persistent storage retrieve it update it

Experiment Search Core functionality pro-vided by NKUA

Search by name sample using full text

ReportManager Completed for the Greenhouse Advice Re-quest scenario

Create report get report by exe-cution or experiment and search report by namedescription

KPI Manager Basic Functionality im-plemented defined the public interface of the component and storage in internal DB

Sample createread kpis Various KPIs available

User Manager Users log in through IDM for accessing UI

Users log in through IDM for ac-cessing UI DigitalSelf IDM imple-mentation is used Spring security is integrated with UI for authoriza-tion mechanisms

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 9 of 34

Core Component Status Functionality

Execution Data Manager Core functionality pro-vided by NKUA for Greenhouse experiment

In an execution data management a process can addretrieve a data provider system A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

Resource Manager Core functionality pro-vided by NKUA for Greenhouse experiment

A resource can be created and retrieved

The proposed technical specification of the EE enables the entire process from testexperiment plan-ning and configuration through execution to analysis of the test execution We introduce below terms to be used throughout this report

Step ndash A step refers to a single actiontask defined in a test scenario

Test scenario ndash This is the ordered set of steps that compose a single test

Variables ndash In the context of a test these are field names that stand for specific values during execution Variables enable flexibility in test execution as they enable running the same test with different field values

Variables binding ndash This refers to the replacement of variables values with the test data This is done by the experimenter during test execution

Experimenttest ndash This is the ordered set of steps to be carried out by an experimenter during execution Each experiment is identified by a unique ID and version An experiment may have variables to enable multiple executions of the same experiment with different data

Execution ndash This is the actual running of an experiment All variables should be bound to Data Providers before the execution of steps can begin

Vusers ndash These are virtual users that play human users in a specific experiment

Vusers scripts ndash These scripts are the ordered set of actions a Vuser performs during the execution of an experiment In other words the set of instructions carried out during execution without user interven-tion

Atomic step ndash This is the smallest (inseparable) single instruction that is carried out during the execution of a test An atomic step may contain (a) an instruction to be manually performed by a tester (b) a ref-erence to run a Vuser script or (c) an instruction to inject data provided through a variable into FIspace test

Execution log ndash This log is a file that lists actions as occurred during execution including all process and system notifications The entries in an execution log can provide insight into what happened during exe-cution of the test and provide an audit trail of information related to the execution In fact the execu-tion log is the input to the Reporting module in the EE which analyzes the log and provides performance assessment of the execution

Expected results ndashThis is the anticipated outcome of a step in a test

Actual results ndash These are the real outcome of a step as result of execution

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 10 of 34

Key Performance Indicator (KPI) ndash The KPI is a set of performance measurements related to TampL stored in the EE for the sake of performance assessment and analysis The evaluation framework specification is in the scope of Work Package 2 but the KPI(s) related to the performance assessment are stored in the EE and can be used to assess the performance of the test executed

Report ndash The report is a summary of what occurred over one or more test executions A report may in-clude performance assessment of the execution based on given KPI(s)

Injected data ndash Data fed into the test by the backend simulator module in EE is injected data Injected data is used whenever real data in real time cannot be obtained during the execution of a test In these cases the intention is to use (realtime) historical data to simulate the processes

Notifications ndash These are messages given to a user via FIspace frontend during an execution of a test Notifications are recorded in the execution log of the test

12 Class diagrams

Figure 2 Experiment CRUD Search classes

The ExperimentCRUDService class contains all the methods for the CRUD operations on Experiments

The ExperimentSearch class implements all the methods for the searching of Experiments in free text or by field

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 11 of 34

Figure 3 ExecutionManagerService Executor Classes

The Execution ManagerService class handles all the phases of an execution of an experiment It gives access to all the executions taking place or in memory

The Executor class is responsible for executing single steps of an execution and logging the result

Figure 4 ReportManager ResourceManager classes

The ReportManager class is responsible for creating reports after an execution and for giving access to stored reports Also has the responsibility of creating KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 7: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 7 of 34

Executor The Executor tracks the execution of the individual steps in an evaluation This in-cludes the automated execution of certain steps such as injecting dataevents into the test in-stance and running VUser scripts through the Script Execution Engine This component also cre-ates and updates entries in the execution log including notifications received from the actual process execution and error messages

Script Execution Engine This executes VUser scripts to automate user actions

Resource Manager The Resource Manager provides an inventory of available resources Services in-clude the ability to locate resources according to various search criteria

Reporting Reports are generated based on execution logs and KPIs

KPI Manager Manages the calculation of KPIs

Execution Log Manager The Execution Log Manager provides logging services for an execution This includes the logging of the results for each step of an execution including any received notifications during the execution of each step Also provides access to these logs

Execution Data Manager This is used to manage the access to data that is used during execution

Internal Data Provider System Used for storing and retrieving manually configured data provid-ers

External Data Access Used to retrieve data from 3rd party external systems could be used to ldquoreplayrdquo events from a real-world shipment for example

Access to these systems is configured by the tester Configuration could be UI or file driven

FIspace Test This is the FIspace public instance which is the replica of a latest stable version of the inte-grated platform hosted on the integration environment depicted on D3006 This instance is updated when the new version of the platform is released FIspace Test platform consists of different modules developed in WP200 are installed and integrated following the instructions provided by the module owners on PIE wiki[1]These instructions are refined by Integration and FIspace Test platform team for the each new release of the platform while integration is executedAfter each integration process inte-gration tests procedures are followed to provide Phase3 developers stable platform Server and services information of each module that constructs the current public version of the FIspace Test platform can be found on the bitbucket EE wikipage[2]

Backend Simulator This simulates input data from backend systems to FIspace and provides APIs to inject data to the FIspace Test systemrsquos modules The simulator reports back on events and other pro-cessed data

EE Storage This provides internal storage services for the experimentation environment

Data Provider System (app) ndash This component addresses the binding of the application to the execution data

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 8 of 34

11 EE Components statusfunctionality

Table 1 Components description

Core Component Status Functionality

ExecutionManagerService Core functionality pro-vided by NKUA

It currently starts a new execution according to an experimentstores it and provides the functionality to search for executions according to experimentidcreator Also calls Executor for the execution of StepsSteps can now be running for as time as needed

Executor Core functionality pro-vided by NKUA

Executes an experiments step and logs its result

ScriptExecutionEngine Core functionality pro-vided by NKUA for Greenhouse experiment

Executes a steps REST call or jar

BackEndSimulator Core functionality pro-vided by NKUA

First version of connecting to SDI using keycloak Abstract interface created for various backends Greenhouse backend simulator development in progress

ExecutionLogManager Core functionality pro-vided by NKUA

Logs a steps result and saves it to persistent storageAlso retrieves logs for a specific execution

Experiment CRUD Core functionality pro-vided by NKUA

Create an experiment to persistent storage retrieve it update it

Experiment Search Core functionality pro-vided by NKUA

Search by name sample using full text

ReportManager Completed for the Greenhouse Advice Re-quest scenario

Create report get report by exe-cution or experiment and search report by namedescription

KPI Manager Basic Functionality im-plemented defined the public interface of the component and storage in internal DB

Sample createread kpis Various KPIs available

User Manager Users log in through IDM for accessing UI

Users log in through IDM for ac-cessing UI DigitalSelf IDM imple-mentation is used Spring security is integrated with UI for authoriza-tion mechanisms

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 9 of 34

Core Component Status Functionality

Execution Data Manager Core functionality pro-vided by NKUA for Greenhouse experiment

In an execution data management a process can addretrieve a data provider system A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

Resource Manager Core functionality pro-vided by NKUA for Greenhouse experiment

A resource can be created and retrieved

The proposed technical specification of the EE enables the entire process from testexperiment plan-ning and configuration through execution to analysis of the test execution We introduce below terms to be used throughout this report

Step ndash A step refers to a single actiontask defined in a test scenario

Test scenario ndash This is the ordered set of steps that compose a single test

Variables ndash In the context of a test these are field names that stand for specific values during execution Variables enable flexibility in test execution as they enable running the same test with different field values

Variables binding ndash This refers to the replacement of variables values with the test data This is done by the experimenter during test execution

Experimenttest ndash This is the ordered set of steps to be carried out by an experimenter during execution Each experiment is identified by a unique ID and version An experiment may have variables to enable multiple executions of the same experiment with different data

Execution ndash This is the actual running of an experiment All variables should be bound to Data Providers before the execution of steps can begin

Vusers ndash These are virtual users that play human users in a specific experiment

Vusers scripts ndash These scripts are the ordered set of actions a Vuser performs during the execution of an experiment In other words the set of instructions carried out during execution without user interven-tion

Atomic step ndash This is the smallest (inseparable) single instruction that is carried out during the execution of a test An atomic step may contain (a) an instruction to be manually performed by a tester (b) a ref-erence to run a Vuser script or (c) an instruction to inject data provided through a variable into FIspace test

Execution log ndash This log is a file that lists actions as occurred during execution including all process and system notifications The entries in an execution log can provide insight into what happened during exe-cution of the test and provide an audit trail of information related to the execution In fact the execu-tion log is the input to the Reporting module in the EE which analyzes the log and provides performance assessment of the execution

Expected results ndashThis is the anticipated outcome of a step in a test

Actual results ndash These are the real outcome of a step as result of execution

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 10 of 34

Key Performance Indicator (KPI) ndash The KPI is a set of performance measurements related to TampL stored in the EE for the sake of performance assessment and analysis The evaluation framework specification is in the scope of Work Package 2 but the KPI(s) related to the performance assessment are stored in the EE and can be used to assess the performance of the test executed

Report ndash The report is a summary of what occurred over one or more test executions A report may in-clude performance assessment of the execution based on given KPI(s)

Injected data ndash Data fed into the test by the backend simulator module in EE is injected data Injected data is used whenever real data in real time cannot be obtained during the execution of a test In these cases the intention is to use (realtime) historical data to simulate the processes

Notifications ndash These are messages given to a user via FIspace frontend during an execution of a test Notifications are recorded in the execution log of the test

12 Class diagrams

Figure 2 Experiment CRUD Search classes

The ExperimentCRUDService class contains all the methods for the CRUD operations on Experiments

The ExperimentSearch class implements all the methods for the searching of Experiments in free text or by field

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 11 of 34

Figure 3 ExecutionManagerService Executor Classes

The Execution ManagerService class handles all the phases of an execution of an experiment It gives access to all the executions taking place or in memory

The Executor class is responsible for executing single steps of an execution and logging the result

Figure 4 ReportManager ResourceManager classes

The ReportManager class is responsible for creating reports after an execution and for giving access to stored reports Also has the responsibility of creating KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 8: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 8 of 34

11 EE Components statusfunctionality

Table 1 Components description

Core Component Status Functionality

ExecutionManagerService Core functionality pro-vided by NKUA

It currently starts a new execution according to an experimentstores it and provides the functionality to search for executions according to experimentidcreator Also calls Executor for the execution of StepsSteps can now be running for as time as needed

Executor Core functionality pro-vided by NKUA

Executes an experiments step and logs its result

ScriptExecutionEngine Core functionality pro-vided by NKUA for Greenhouse experiment

Executes a steps REST call or jar

BackEndSimulator Core functionality pro-vided by NKUA

First version of connecting to SDI using keycloak Abstract interface created for various backends Greenhouse backend simulator development in progress

ExecutionLogManager Core functionality pro-vided by NKUA

Logs a steps result and saves it to persistent storageAlso retrieves logs for a specific execution

Experiment CRUD Core functionality pro-vided by NKUA

Create an experiment to persistent storage retrieve it update it

Experiment Search Core functionality pro-vided by NKUA

Search by name sample using full text

ReportManager Completed for the Greenhouse Advice Re-quest scenario

Create report get report by exe-cution or experiment and search report by namedescription

KPI Manager Basic Functionality im-plemented defined the public interface of the component and storage in internal DB

Sample createread kpis Various KPIs available

User Manager Users log in through IDM for accessing UI

Users log in through IDM for ac-cessing UI DigitalSelf IDM imple-mentation is used Spring security is integrated with UI for authoriza-tion mechanisms

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 9 of 34

Core Component Status Functionality

Execution Data Manager Core functionality pro-vided by NKUA for Greenhouse experiment

In an execution data management a process can addretrieve a data provider system A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

Resource Manager Core functionality pro-vided by NKUA for Greenhouse experiment

A resource can be created and retrieved

The proposed technical specification of the EE enables the entire process from testexperiment plan-ning and configuration through execution to analysis of the test execution We introduce below terms to be used throughout this report

Step ndash A step refers to a single actiontask defined in a test scenario

Test scenario ndash This is the ordered set of steps that compose a single test

Variables ndash In the context of a test these are field names that stand for specific values during execution Variables enable flexibility in test execution as they enable running the same test with different field values

Variables binding ndash This refers to the replacement of variables values with the test data This is done by the experimenter during test execution

Experimenttest ndash This is the ordered set of steps to be carried out by an experimenter during execution Each experiment is identified by a unique ID and version An experiment may have variables to enable multiple executions of the same experiment with different data

Execution ndash This is the actual running of an experiment All variables should be bound to Data Providers before the execution of steps can begin

Vusers ndash These are virtual users that play human users in a specific experiment

Vusers scripts ndash These scripts are the ordered set of actions a Vuser performs during the execution of an experiment In other words the set of instructions carried out during execution without user interven-tion

Atomic step ndash This is the smallest (inseparable) single instruction that is carried out during the execution of a test An atomic step may contain (a) an instruction to be manually performed by a tester (b) a ref-erence to run a Vuser script or (c) an instruction to inject data provided through a variable into FIspace test

Execution log ndash This log is a file that lists actions as occurred during execution including all process and system notifications The entries in an execution log can provide insight into what happened during exe-cution of the test and provide an audit trail of information related to the execution In fact the execu-tion log is the input to the Reporting module in the EE which analyzes the log and provides performance assessment of the execution

Expected results ndashThis is the anticipated outcome of a step in a test

Actual results ndash These are the real outcome of a step as result of execution

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 10 of 34

Key Performance Indicator (KPI) ndash The KPI is a set of performance measurements related to TampL stored in the EE for the sake of performance assessment and analysis The evaluation framework specification is in the scope of Work Package 2 but the KPI(s) related to the performance assessment are stored in the EE and can be used to assess the performance of the test executed

Report ndash The report is a summary of what occurred over one or more test executions A report may in-clude performance assessment of the execution based on given KPI(s)

Injected data ndash Data fed into the test by the backend simulator module in EE is injected data Injected data is used whenever real data in real time cannot be obtained during the execution of a test In these cases the intention is to use (realtime) historical data to simulate the processes

Notifications ndash These are messages given to a user via FIspace frontend during an execution of a test Notifications are recorded in the execution log of the test

12 Class diagrams

Figure 2 Experiment CRUD Search classes

The ExperimentCRUDService class contains all the methods for the CRUD operations on Experiments

The ExperimentSearch class implements all the methods for the searching of Experiments in free text or by field

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 11 of 34

Figure 3 ExecutionManagerService Executor Classes

The Execution ManagerService class handles all the phases of an execution of an experiment It gives access to all the executions taking place or in memory

The Executor class is responsible for executing single steps of an execution and logging the result

Figure 4 ReportManager ResourceManager classes

The ReportManager class is responsible for creating reports after an execution and for giving access to stored reports Also has the responsibility of creating KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 9: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 9 of 34

Core Component Status Functionality

Execution Data Manager Core functionality pro-vided by NKUA for Greenhouse experiment

In an execution data management a process can addretrieve a data provider system A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

Resource Manager Core functionality pro-vided by NKUA for Greenhouse experiment

A resource can be created and retrieved

The proposed technical specification of the EE enables the entire process from testexperiment plan-ning and configuration through execution to analysis of the test execution We introduce below terms to be used throughout this report

Step ndash A step refers to a single actiontask defined in a test scenario

Test scenario ndash This is the ordered set of steps that compose a single test

Variables ndash In the context of a test these are field names that stand for specific values during execution Variables enable flexibility in test execution as they enable running the same test with different field values

Variables binding ndash This refers to the replacement of variables values with the test data This is done by the experimenter during test execution

Experimenttest ndash This is the ordered set of steps to be carried out by an experimenter during execution Each experiment is identified by a unique ID and version An experiment may have variables to enable multiple executions of the same experiment with different data

Execution ndash This is the actual running of an experiment All variables should be bound to Data Providers before the execution of steps can begin

Vusers ndash These are virtual users that play human users in a specific experiment

Vusers scripts ndash These scripts are the ordered set of actions a Vuser performs during the execution of an experiment In other words the set of instructions carried out during execution without user interven-tion

Atomic step ndash This is the smallest (inseparable) single instruction that is carried out during the execution of a test An atomic step may contain (a) an instruction to be manually performed by a tester (b) a ref-erence to run a Vuser script or (c) an instruction to inject data provided through a variable into FIspace test

Execution log ndash This log is a file that lists actions as occurred during execution including all process and system notifications The entries in an execution log can provide insight into what happened during exe-cution of the test and provide an audit trail of information related to the execution In fact the execu-tion log is the input to the Reporting module in the EE which analyzes the log and provides performance assessment of the execution

Expected results ndashThis is the anticipated outcome of a step in a test

Actual results ndash These are the real outcome of a step as result of execution

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 10 of 34

Key Performance Indicator (KPI) ndash The KPI is a set of performance measurements related to TampL stored in the EE for the sake of performance assessment and analysis The evaluation framework specification is in the scope of Work Package 2 but the KPI(s) related to the performance assessment are stored in the EE and can be used to assess the performance of the test executed

Report ndash The report is a summary of what occurred over one or more test executions A report may in-clude performance assessment of the execution based on given KPI(s)

Injected data ndash Data fed into the test by the backend simulator module in EE is injected data Injected data is used whenever real data in real time cannot be obtained during the execution of a test In these cases the intention is to use (realtime) historical data to simulate the processes

Notifications ndash These are messages given to a user via FIspace frontend during an execution of a test Notifications are recorded in the execution log of the test

12 Class diagrams

Figure 2 Experiment CRUD Search classes

The ExperimentCRUDService class contains all the methods for the CRUD operations on Experiments

The ExperimentSearch class implements all the methods for the searching of Experiments in free text or by field

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 11 of 34

Figure 3 ExecutionManagerService Executor Classes

The Execution ManagerService class handles all the phases of an execution of an experiment It gives access to all the executions taking place or in memory

The Executor class is responsible for executing single steps of an execution and logging the result

Figure 4 ReportManager ResourceManager classes

The ReportManager class is responsible for creating reports after an execution and for giving access to stored reports Also has the responsibility of creating KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 10: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 10 of 34

Key Performance Indicator (KPI) ndash The KPI is a set of performance measurements related to TampL stored in the EE for the sake of performance assessment and analysis The evaluation framework specification is in the scope of Work Package 2 but the KPI(s) related to the performance assessment are stored in the EE and can be used to assess the performance of the test executed

Report ndash The report is a summary of what occurred over one or more test executions A report may in-clude performance assessment of the execution based on given KPI(s)

Injected data ndash Data fed into the test by the backend simulator module in EE is injected data Injected data is used whenever real data in real time cannot be obtained during the execution of a test In these cases the intention is to use (realtime) historical data to simulate the processes

Notifications ndash These are messages given to a user via FIspace frontend during an execution of a test Notifications are recorded in the execution log of the test

12 Class diagrams

Figure 2 Experiment CRUD Search classes

The ExperimentCRUDService class contains all the methods for the CRUD operations on Experiments

The ExperimentSearch class implements all the methods for the searching of Experiments in free text or by field

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 11 of 34

Figure 3 ExecutionManagerService Executor Classes

The Execution ManagerService class handles all the phases of an execution of an experiment It gives access to all the executions taking place or in memory

The Executor class is responsible for executing single steps of an execution and logging the result

Figure 4 ReportManager ResourceManager classes

The ReportManager class is responsible for creating reports after an execution and for giving access to stored reports Also has the responsibility of creating KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 11: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 11 of 34

Figure 3 ExecutionManagerService Executor Classes

The Execution ManagerService class handles all the phases of an execution of an experiment It gives access to all the executions taking place or in memory

The Executor class is responsible for executing single steps of an execution and logging the result

Figure 4 ReportManager ResourceManager classes

The ReportManager class is responsible for creating reports after an execution and for giving access to stored reports Also has the responsibility of creating KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 12: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 12 of 34

The Resource Manager is implementing methods for crud operations on various types of resources (files etc)

Figure 5 ExecutionLogManager ScriptExecutionEngine classes

The ScriptExecutionEngine is responsible for executing a scriptrestjar of an executionrsquos step

The ExecutionLogManager is implementing methods for the correct creation and storing of logs of execu-tions

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 13: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 13 of 34

Figure 6 KPI manager KPI Calculator classes

Figure 7 KPICalculatorSerializer classes

The KPIManager class gives access to creating and utilizing KPIs

The KPICalculatorFactory interface is creating a KPICalculator implemented class for calculating specific KPIs

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 14: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 14 of 34

A dynamic way of creating KPIs has been made with KPICalculatorSerializer helping class which serializ-es KPICalculator classes to stream and the opposite so as to be able to have an abstract way for gener-ating new types

Figure 8 DataProvider Classes

Data provider classes storeexpose data to several stakeholders through REST API

Figure 9 DataProviderSystem classes

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 15: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 15 of 34

Data provider classes storeexpose data to several stakeholders through REST API

Figure 10 External data access classes

A data provider system can include several data providers that storeexpose data to several stakeholders through REST API

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 16: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 16 of 34

Figure 11 Backend simulator classes

BackendSimulator interface is enabling access to FIspace with capabilities offered from SDI so that it can mock applications usage

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 17: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 17 of 34

13 Database diagram

Figure 12 Experimentation environment Database tables

The database tables are designed to store any information needed and utilized by the Experimentation environment Relation between data has been kept where needed and is indicated by lines

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 18: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 18 of 34

14 Sequence diagrams

141 Experiment creation

Figure 13 Experiment Creation- search

A user can create an Experiment using the EE UI or the EE core REST API directly by sending the Exper-iment to be created details to the ExperimentCRUD component

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 19: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 19 of 34

142 Execution of experiment

Figure 14 Execute specific experiment

A new execution for given experiment id can be created using the Execution Manager Service Then it retrieves all the Experimentrsquos information calls the Executor for starting to execute steps one by one and log the results

143 Create KPIs

Figure 15 Create KPIs

A report can be created for an execution based on the logs A KPI is chosen from the KPI manager to be instantiated and calculated

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 20: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 20 of 34

144 Backend simulator-(SDI)capabilities usage

Figure 16 Backend simulator usage

The Experimentation environment core offers connection with FIspace Test for implementing and execut-ing experiments that include functionality offered by the platform

More specifically this is done by the use of the BackendSimulator interface It can be implemented as specific scenario BackendSimulator For example GreenhouseBackendSimulator This enables using also Keycloak authentication to login to the SDI and have access to the API Then the Backendsimulator can register an experimenting capability for an application and initialize it

This will enable an experiment to call the Backendsimulator so as to use a specific experimenting capabil-ity for an application

More insights in relation to the latest Backend Simulator features will be provided in the last section of the Experimentation Environment section of this document which implements a scenario related to the Prod-uct Information Application (PIA)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 21: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 21 of 34

2 Detailed Usage Guide

The experimentation environment is in general responsible for the creationmodification of FIspace experiments and their execution These experiments are a set of several steps relative to a specific scenario (eg greenhouse logistics washing machine FIspace platform) Each step has specific description parameters and a scriptjarexecutableREST that is responsible for executing the specific scenariorsquos functionality for this step

The experimentation environment consists of a frontend and a backend implementation

21 Greenhouse experiment

Below a description for the creation execution of the greenhouse advice scenario follows

Some of the information given must be exact for the correct definition of the experiment Other information is optional or not used so a test value can be given

1) The experimenter defines and creates the Greenhouse experiment which has two steps ndash a) Get the sensor values of a specific farm from the backend simulator ndash b) Feed these values to the expert system and get the advice

2) The new experiment appears in the experiments list ndash The details of the experiments can be seen ndash The experiment can be deleted

3) The experiment search functionality shows available experiments based on the query in a full text context

4) The experimenter can see all the details of all experiments

5) The experimenter executes the experiment

6) A new execution id is produced

7) The raw logs for the execution are shown

8) A report based on these logs can be created with the following KPIs

ndash a)Response time for the whole execution ndash b)Step execution time ndash c)Validity of response time ndash d)Success rate of steps

9) The experimenter can see the reports that are produced

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 22: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 22 of 34

3 Detailed Greenhouse Experiment

31 Create a greenhouse experiment with two steps

Figure 17 experiment CRUD

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 23: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 23 of 34

Create Experiment

Form input Value Type

Experiment ID give a unique id eg 13ldfa9

Experiment Name eg Greenhouse experiment demo

Creator eg Bill

Guid ID same as experiment id

version eg 13

ExperimentType Greenhouse Must be exactly this If it is another type Other for example

Step 0

Form input Value Type

Actor eg Bill

Description REST Must be exactly this It is a REST call

Data Description eg test

Expected Result eg test

Id a unique id eg dakd032

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri httpserver_urlportBackendSimula-torgetSensorDatafarmidnumberofdata

Must be exactly this It is the link where to fetch data from specific farm

Description GET Must be exactly this It is a GET call

V User Script test

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 24: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 24 of 34

Step 1

Form input Value Type

Actor eg Bill

Description JAR Must be exactly this

Data Description eg test

Expected Result eg test

Id a unique id eg dakd0332

Link

Form input Value Type

Additional data eg test

Name eg mylink

Uri test

Description test

V User Script homeubuntuv1jar Must be exactly this It is a specific jar executable It can be chosen from the dropdown menu of scripts after creating an experiment executable

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 25: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 25 of 34

32 Read an Experiment

Figure 18 Read experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 26: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 26 of 34

321 Get all experiments

Figure 19 get all experiments

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 27: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 27 of 34

322 Search Experiment

Figure 20 Search experiment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 28: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 28 of 34

33 Execution Management of an Experiment

We refresh the experiments list and pick the created experiment Then we start the execution We wait for a while until a new execution id emerges Afterwards we can push the get logs button and see what happened in the ex-periment We can see the sensor values and the response from the expert system for these values

Figure 21 Execution start

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 29: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 29 of 34

Figure 22 Log results of execution (sensor data and actions alerts from expert system)

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 30: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 30 of 34

34 Report and KPIs based for specific execution

From the KPIs list we pick a KPI and press Create Report

Afterwards we can push get Report and see the KPI value We can do this procedure for all the KPIs and Get Report returns all of them

Figure 23 Reports and KPIs

4 Further experimentation

An experiment definition can also be created for any kind of experiment however with some constraints For ex-ample a non-greenhouse experiment can be created with several steps executables per step and various defini-tions Therefore someone can see the log results of the steps and some KPIs such as response time which are not domain specific Design for experiments for other trials is an ongoing process

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 31: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 31 of 34

5 Backend Simulator Final Release extensions ndash Interconnection with Product Information and BOXMANS appsrsquo scenario

In order to support the latest scenario executions new developments and extensions have been taking place mainly related to the Backend Simulator module of the Experimentation Environment (EE)

The latest scenario taking place in the EE core involves several new FIspace apps ie the Product Information app (PIA) Tailored Information for Consumers (TIC) Box Management (BOXMAN) app and Risk Management app (RISKMAN) Before proceeding to describing the Backend Simulatorrsquos developments it would be useful to provide a brief description of the scenario

In this scenario product information (ie product quality-related information) shall be exchanged between FIspace userscompanies (ieapps of these userscompanies) We focus on 2 apps in the first part and more (at least 2 more) in further parts

Communication between Initial Apps Connection between multiple PIA (Product Information App) in-

stances as well as forwarding product information from PIA towards TIC (Tailored Information for Con-

sumers app) Instances both of the same FIspace usercompany

Involvement of Open Call Apps Forwarding of packaging information from PIA towards Boxman (Box

Management app) instances both of the same FIspace usercompany as well as providing of product in-

formation on demand from PIA towards Riskman (Risk Management app) instances both of the same

FIspace usercompany

As extended information has already been provided in previous deliverable regarding the Product Information Exchange scenario -if required by the reader- additional information can be found in the respective FIspace wiki page [3]

Regarding the Backend Simulator module at first a Backend Simulator has to be defined for the specific scenario A Backend Simulator is a component that imitates a FIspace backend service In other words it provides a capability through the EE SDI so that it can be used in experiments defined through EE core So for each scenario a Backend simulator implementation is needed

BackendSimulator

-InitializeConfiguration()

-utilizeCapability()

Figure 24 Backend Simulator

The initialization method will do the following

Get a token from Keycloak to be authenticated in EE SDI using an application id secret

Search for the capability type that is wanted to be used for sending and receiving messages in this mock service

Register a business process template

Register the capabilities that will be offered by the Backend simulator

There can be capabilities offering information or receiving information according to messages defined in the capability types

Register a business process for these capabilities

An example Backend Simulator implementation imitating a sensor values provider capability is available Here

The utilizeCapability method is exposed in the REST interfaces implemented by the Backend Simulator The REST interfaces will use the capability type messages so as to be able to exchange messages with the EE SDI

The Backend Simulator can have simplified REST APIs that will do queries and communicate with SDI for fetching information after interacting with the capability Or it can be used as any service that FIspace offers

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 32: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 32 of 34

Following the backend simulator guide [4] that can be found in the wiki in the Experimentation Environment wiki we are simulating the customer PIA backend and the BOXMAN backend Please note that there is no need to cre-ate separate backend simulators in order to simulate different backends as a single backend simulator can imple-ment all the required capabilities for a scenario

Initialisation First we create the initialisation resource class named ldquoCapsInitResourcerdquo that is used to

register everything that is needed for the execution of the scenario on the FIspace side This includes the

registration of a Business Process Template Business Roles a Business process and the needed Capabili-

ties

Capabilities Implementation The needed capabilities are implemented in classes ldquoCapabilityResourcerdquo

and ldquoCapsUsageResourcerdquo

The web application is packaged as a war file and then deployed in a Jetty web server that has an external IP ad-dress This is needed so that FIspace can reach back the simulator

The backend simulator code is published as a sub-module of the Experimentation Environment any interested readers in developing a simulator are advised to review it so they can see a concrete example [5]

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 33: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment

FIspace 11112015

FIspace-D30011-ExpEnvDevFinalRel-V002docx Page 33 of 34

6 References

[1] httpsbitbucketorgfispacecorewikiPreliminary20Integration20Environment

[2] httpsbitbucketorgfispacecorewikiExperimentation20Environment

[3] httpsbitbucketorgfispacecorewikiproductInformationExchangeScenario

[4] httpsbitbucketorgfispaceexperimentation-environmentwikiBackendSimHowTo

[5] httpsbitbucketorgfispaceexperimentation-

environmentsrcfefeca852b9ffe6d21b1ac57f328b3e1777 349b2PiaBackendSimulatorat=default

Page 34: Experimentation environment Development Final Release · 2017. 4. 25. · FIspace 11.11.2015 FIspace-D300.11-ExpEnvDevFinalRel-V002.docx Page 6 of 34 1 Experimentation environment