Forecasting QoS Attributes Using LSTM NetworksGary White, Andrei Palade, Siobhan Clarke

School of Computer Science and Statistics,Trinity College Dublin, Dublin, Ireland

{whiteg5, paladea, siobhan.clarke}@scss.tcd.ie

Abstract—Many modern software systems and applications arebuilt using heterogeneous services provided by a range of devices,from high-power devices located in the Cloud to potentiallyresource-constrained and/or mobile services from IoT devicesat the edge of the network. The large growth in the numberof these services has led to some functionally similar services.When selecting services, a critical criterion is Quality of Service(QoS), which includes factors such as response time, location andcost. As the value of dynamic QoS attributes vary with time, thereis a need to accurately forecast future QoS values to identify ifa service may be about to fail. In this paper, we propose usingan LSTM-based neural network to forecast future QoS values.We evaluate the use of an LSTM network against the existingstate of the art in experiments using an established web servicedataset and a new dataset collected by deploying services on lowpower IoT devices, which we publicly release. This mixture ofdatasets covers the heterogeneity that would be expected in atypical IoT environment.

I. INTRODUCTION

The development of Internet of Things (IoT) technologyhas made it possible to connect various smart devices to-gether through a number of communication protocols. Thesedevices have led to a wide variety of services from multiplesources such as monitoring sensors, surveillance cameras andactuators. The management of the services provided by thesedevices is recognised as one of the most important areas offuture technology and has gained wide attention from researchinstitutes and industry in a number of different domainsincluding transportation, healthcare, industrial automation andemergency response [1], [2].

Devices that provide software services in these domains canbe resource-constrained and/or mobile, which can lead to theQoS degrading or the service not responding [3]. Changesin time-varying QoS attributes, such as response time andthroughput, can indicate that a service is about to fail ordegrade in quality. There is a need to accurately forecast theseQoS attributes not only for the currently executing servicesbut also the values reported by users for candidate services[4]. These services can be utilised if degradation is forecastin the currently executing services [5]. Such functionality canbe contained in a middleware to choose the optimal serviceat design time and conduct a dynamic service adaptation ifthe QoS begins to degrade [6]. Applications can have a rangeof QoS requirements based on the domain and sensitivity ofthe information. With services available from a variety ofdevices, many providers offer services that provide equivalentfunctionalities. Such redundant services can be used during aservice adaptation that is responding to forecast QoS changes

(e.g. unacceptable response time). To achieve this, knowledgeabout QoS values of the services is required to make timelyand accurate adaptation decisions such as: when to triggeran adaptation action; which working services to replace andwhat candidate service to choose. For applications in criticaldomains such as healthcare and emergency response, it isespecially important for a system to react to degrading servicequality and handle runtime service adaptation by selectingsuitable replacement services.

Detecting when a service is about to fail or degrade is adifficult task as there are a number of QoS factors, which canindicate a change in the service, such as, response time, loca-tion and cost [7]. QoS forecasting approaches have focused ontraditional time-series analysis to make predictions of futureQoS values such as average, linear and ARIMA models [8],[9], [10], [11]. Traditional models, such as ARIMA requiremanual tuning to set seasonality and other parameters. Whilethere are time-series models that can incorporate exogenousvariables [12], they suffer from the curse of dimensionalityand require frequent retraining. To more effectively dealwith exogenous variables, Opitz introduced a combination ofunivariate modeling and a machine-learned model to handleresiduals [13]. The resulting two-stage model, however, ishard to tune, requires manual feature extraction and frequentretraining, which is prohibitive for large time-series datasets.

A recurrent neural network (RNN) is the most recognizedneural network for time-series forecasting. Unlike standardfeed-forward neural networks, RNNs can utilize past data andforecast prospective data through its feedback path. Recentresults have shown promise for the use of RNNs especiallyLong short-term memory (LSTM) and Gated recurrent units(GRU) architectures [14], [15], [16], [17], which have beenshown to produce improved forecasting results even in highvariance situations. They do this by end-to-end modelling,incorporating exogenous variables and automatic feature ex-traction abilities [18]. An LSTM network can model complexnonlinear feature interactions, which is critical to modellingcomplex extreme events [19]. Similar to the LSTM unit, theGRU has developed from the basic architecture of RNNs bymodifying the functional form to make each recurrent unitadaptively capture dependencies on different time scales [17].

In this paper, we develop an LSTM network to forecast QoSvalues from services on a number of heterogeneous devicesand test against the current state of the art methods anda GRU network. The remainder of the paper is organisedas follows: Section II describes the need for reliable IoT

applications and how accurate QoS forecasting can be usefulboth for monitoring currently executing services and selectingthe correct candidate service. Section III describes currentforecasting approaches and the GRU and LSTM networkimplementations used in the experiments. Section IV describesthe experiments and Section V presents the results. SectionVI outlines the related work and Section VII concludes andpresents future work.

II. RELIABLE IOT APPLICATIONS

Figure 1 shows a small scale IoT scenario, with servicesprovided from different service types including web services(WS), residing on resource rich devices, wireless sensornetworks (WSN), which may be resource-constrained andcontrolled by a software defined network, and autonomousservice providers (ASP), who are independent mobile usersin the environment with intermittent availability. The servicesprovided by devices in the environment are registered in thegateways and can be used in a number of applications in avariety of different domains.

One way to categorise these applications is by their QoS re-quirements, which are based on the sensitivity and criticality ofthe application. IoT applications can typically be categorisedas best effort (no QoS), differentiated services (soft QoS) andguaranteed services (hard QoS) [7]. In the hard QoS case,there are strict hard real-time QoS guarantees. This would befor safety critical applications such as monitoring patients ina hospital or collision avoidance in a self-driving car system.Soft QoS doesn’t require hard real-time guarantees but needsto be able to reconfigure and replace services that fail. Thiscould be a routing application, which uses air quality, floodingand pedestrian traffic predictions [20] to provide the best routethrough the city. If one of the services is about to fail, themiddleware should re-compose the application using suitablereplacement services. The final case is best effort, where thereare no guarantees when a service fails. The ability to accuratelyforecast QoS values is needed to provide hard or soft QoS, sois a crucial problem for future reliable applications.

A. IoT Middleware

A middleware can be used to provide additional relia-bility, when working in a service-oriented environment bymonitoring currently executing services and providing rec-ommendations for candidate services based on personalisedQoS predictions. Figure 2 shows a middleware architecture,including the QoS monitor used to collect QoS informationfrom a number of different user requests. We implementedthis middleware in the gateways to register, compose, monitorand execute the services in the environment. The services havea range of different QoS properties including response-time,location and energy consumption. The communication linksfor invoking these services are diverse, which will affect thepersonal QoS experience of the users, so they are monitoredusing the monitoring engine. As these devices may be re-source constrained and mobile, it adds additional complexity

User Request

WS3: Flood PredictionService

WS3: Air PolutionService

Service Providers

Gateways

User Request

ASP3: Best RouteService

WSN1: Water LevelService

WS2: Flood PredictionService

ASP4: Air PollutionService

WS1: Best Route Service

GW2:192.168.2.2

GW1:192.168.2.1

GW3:192.168.2.3

GW5:192.168.2.5

GW4:192.168.2.4GW6:192.168.2.6

ASP2: Possible RoutesService

Guidepost

ASP1: Noise Service

Fig. 1: Small Scale Demonstration

User Request Request Handler Service Registration Engine

Service DiscoveryEngine Web Service

Provider

QoS MonitorMonitoring

EnginePrediction

Engine

Service Composition& Execution Engine

Service Registry

AutonomousService Provider

AsynchronousRequest

Publish/SubscriptMQTT

Service Interface

WSN

Fig. 2: Middleware Architecture

compared to traditional web services that are static and morereliable.

The main components of the middleware are the RequestHandler (RH), the Service Registration Engine (SRE), theService Discovery Engine (SDE), the QoS Monitor and theService Composition & Execution Engine (SCEE). The RHestablishes a request/response communication channel withthe user and forwards the request to the other middlewarecomponents. The SRE registers the available services in theenvironment. The SDE uses a backward-planning algorithmto identify concrete services, which can be used to satisfythe request and sends this list of services to the SCEE.

The QoS monitor is used to monitor these services and canidentify possible candidate services to switch to if one ofthe services begins to degrade, using a collaborative filteringalgorithm in the prediction engine [21]. The prediction enginealso contains an algorithm to forecast future QoS values ofcurrently executing services, to identify that a service may beabout to fail.

The prediction of future degradation of services and possibleSLA violations allows for proactive measures to take place toensure reliable IoT applications. When the prediction enginefirst identifies that a service may commit an SLA violation itsends an alert to the service discovery engine for a proactiveservice discovery, to identify if any new services have enteredthe environment to replace the service that may commit anSLA violation. If there is not enough time for a full servicerediscovery then a message is sent to the composition engineto compose a new path using services already registered. Thisallows for the replacement of services in a composition beforethe user identifies that there is a problem with the composition.The algorithm to make these predictions is the focus of thispaper, in which we propose an LSTM based approach that wecompare to a persistence, ARIMA and GRU based approach.

III. FORECASTING QOS ATTRIBUTES

This section introduces the models that have traditionallybeen used for QoS factor forecasting and also the GRU andLSTM models. These are the approaches on which we conductexperiments on to access the accuracy of LSTM.

A. Persistence Models

The persistence (naıve) method is the simplest forecastingmethod in time-series research and is used as a baseline model.The model uses the last observed time-series value yi as itsforecast yi+1 of the next value in the predicted time series.

yi+1 = yi (1)

B. ARIMA

AutoRegressive Integrated Moving Average (ARIMA) mod-els were originally proposed by Box and Jenkins [22] to modeltime series data and forecast their future values. ARIMAmodels for forecasting consist of three components: Auto-Regressive (AR), Integrated (I) and Moving Average (MA).These parts can be used together or independently to performtime-series forecasting. For each component, there is a number(p for AR, d for I and q for MA) that must be determinedfor an ARIMA model to make forecasts. Suitable values aredetermined by analysing the past training data to determineif integration (I) is needed to make the time series stationary(mean and variance do not change over time). Once the timeseries has been made stationary the AR and MA lags canbe calculated by constructing an Autocorrelation and PartialAutocorrelation plot and choosing a suitable number of lags.In our experiments we automate this process by performing agrid search over component values from 0-3, as these valuescontained the best performing models based on a sample of

the dataset. If no integration is required the time series yt issaid to be modelled by an ARMA model of orders p and q,denoted by ARMA(p,q), if it satisfies:

φp(B)yt = θq(B)εt (2)

where, εt is a sequence of independent normal errors withzero mean and variance σ2. The backshift operator B isdefined as Bxt = xt−1. The autoregressive polynomial isφp(B) = (1 − φ1B − φ1B

2 − · · · − φpBp) with order

p and θq(B) = (1 + θ1B + θ1B2 + · · · + θqB

q) is themoving average polynomial with order q. The autoregressiveand moving average coefficients are φ = (φ1, φ2, . . . , φp)

T

and θ = (θ1, θ2, . . . , θp)T respectively. The model (2) can be

simplified and written:

yi+1 =

p∑i=0

θiyt−1 +

p∑i=0

θiεt−1 + εt (3)

where, yt+1 is the next step in the time series, p are thepast stationary observations, εt is the current error and εt−i

for i = 1, . . . , q are the past errors. If d differences have beendone to transform the original non-stationary time series ztinto the stationary one yt, then zt is said to be generated by anAutoregressive Integrated Moving Average (ARIMA) model oforders p, d, and q and denoted by ARIMA(p,d,q). The mainassumptions of ARIMA models are serial dependency, normal-ity, stationary and invertibility. Investigating whether the giventime series data satisfies these assumptions is crucial task,as unsatisfied assumptions lead to incorrect ARIMA modelsthat in turn will provide incorrect forecasts. We validate theseassumptions using the training data when creating the model.

C. GRU

Gated recurrent units have developed from recurrent neuralnetworks (RNNs), which have been previously applied tovarious sequence learning tasks such as natural languageprocessing, gas market models and speech signal modelling[23], [24], [25]. Despite their early success, the difficultyof training RNNs led to proposals to improve their basicarchitecture [26]. GRU and LSTM networks are the mostsuccessful variants of the traditional RNN architecture. GRUshave developed from the basic architecture of RNNs bymodifying the functional form to make each recurrent unitadaptively capture dependencies on different time scales [17].The GRU has gated units that modulate the flow of informationinside the unit. The activation hjt of the GRU at time t is alinear interpolation between the previous activation hjt−1 andthe candidate activation hjt :

hjt = (1− zjt )hjt−1 + zjhjt : (4)

where an update gate zjt decides how much the unit updatesits activation, or content. The update gate is computed by

zjt = σ(Wzxt + Uzht−1)j (5)

h ~h In

Out

z

r

(a) Gated Recurrent Unit

c

~c

i

In

Outo

f

(b) Long Short-Term Memory

Fig. 3: Illustration of (a) GRU (b) LSTM (a) r and z are thereset and update gates, and h and h are the activation and thecandidate activation. (b) i, f and o are the input, forget andoutput gates, respectively. c and c denote the memory cell andthe new memory cell content.

The GRU, does not have any mechanism to control thedegree to which its state is exposed, but exposes the wholestate each time. The candidate activation hjt is computedsimilarly to that of the traditional recurrent unit [27],

hjt = tanh(Wxt + U(rt � ht− 1))j (6)

where rt is a set of reset gates and � is an element-wise multiplication. When off (rjt close to 0), the reset gateeffectively makes the unit act as if it is reading the first symbolof an input sequence, allowing it to forget the previouslycomputed state. The reset gate rjt is computed similarly tothe update gate:

rjt = σ(Wrxt + Urht−1)j (7)

This process is illustrated graphically in Figure 3a.

D. LSTM

LSTMs have also developed from recurrent neural networks(RNNs), but improved on the initial architecture by storing andretrieving information over a long period of time with explicitgating mechanisms and a built-in error carousel [28], [25].This addressed the difficulties of training RNNs, in which thebackpropagation dynamics caused the gradients in an RNN toeither vanish or explode. Unlike the traditional RNN, whichsimply computes a weighted sum of the input and applies anonlinear function, each jth LSTM unit maintains a memorycjt at time t. The output hjt , which is the activation of theLSTM unit is then:

hjt = ojt tanh(cjt ) (8)

where ojt is an output gate that modulates the amount ofmemory content exposure. The output gate is computed by:

ojt = σ(Woxt + Uoht−1 + Voct)j (9)

where σ is a logistic sigmoid function and Vo is a diagonalmatrix. This procedure of taking a linear sum between the

existing state and the newly computed state is similar to theGRU unit, however LSTMs can control the degree to which itsstate is exposed. The memory cell cjt is updated by partiallyforgetting the existing memory and adding a new memorycontent cjt :

cjt = f jt cjt−1 + ijtc

jt−1 (10)

The introduction of gates allow the LSTM unit to decidewhether to keep the existing memory or to overwrite it. Thisimproves on the traditional recurrent unit, which overwrites itscontent at each time-step. Intuitively, if the LSTM unit detectsan important feature from an input sequence at early stage, iteasily carries this information (the existence of the feature)over a long distance, hence, capturing potential long-distancedependencies. This process is illustrated graphically in Figure3b.

IV. EXPERIMENTAL SETUP

A. Dataset Description

The goal of this study is to compare the QoS forecastingaccuracy of different models in the context of dynamic QoSfactors from heterogeneous devices. These forecasts can beused for QoS-aware service selection with a focus on servicesthat provide the best QoS, while avoiding SLA violations. Weuse two datasets to evaluate the proposed LSTM model, oneestablished dataset of web services [29] and a newly createddataset from services deployed on low power devices. The webservices dataset consists of monitored QoS data collected byinvoking 10 real services every hour for four months [29]. Thedataset contains (i) whether the service was available or not,(ii) the SOAP output message, (iii) the observed response time,(iv), the throughput, as the size of the SOAP message dividedby the response time, (iv) whether or not the service generatedan exception. In this paper, we limit our investigation to theresponse time, measured in milliseconds.

We also created our own dataset on an alternative timeinterval with services deployed on raspberry pis. The device,which acts as the user is four hops away from the servicesand is invoked every 30 seconds for 1 month. We invokethe services every every 30 seconds instead of every hourto access the impact that monitoring frequency can have onprediction accuracy. The response time of each of the servicesis measured and the dataset is publicly available on our website1. For our experimentation we chose not to prune the timeseries for outliers, i.e., values above Q3+1.5*IQR, where Q3is the third quartile and IQR is the interquartile range. Thisis to evaluate the robustness of the forecasting approaches tooutliers. The experiments are conducted on a subset of thedata, which uses 5 minute intervals to allow the experimentsto be completed in a timely manner, while still showing theimpact of increasing the monitoring frequency.

Table I shows the hardware configurations of the devicesthat are used for data collection. The elements in the Device

1https://www.scss.tcd.ie/∼whiteg5/data/sensor data.zip

TABLE I: Hardware configurations of the devices used tocollect the data.

Device Configuration

Linux Workstation

Model: Dell Optiplex 755OS: Ubuntu Linux 14.04 LTSCPU: 2 GHzRAM: 4 GB

Raspberry Pi

Model: Raspberry Pi 3 BOS: Raspbian Jessie Release 8.0CPU: 1.2 GHzRAM: 1 GB

Arduino Model: Arduino WeMos D1R2Microcontroller: ESP8266EX

column represent the type of the devices. These names will beused in the description of the rest of the experimental setupwithout referring to the model. The elements in the Configura-tion column represent a detailed hardware description of eachdevice. The setup included a Raspberry Pi 3 Model B [30],two Arduino WeMos D1R2 [31] devices, a Linux Workstationand a Cisco router model Linksys EA6400 [32]. The LinuxWorkstation is used to simulate a number of virtual users usingApache JMeter version 3.3 [33]. For the services, we allocated10 virtual users to invoke the service. This invocation wasinitiated according to a Poisson Timer with λ = 100 ms anda delay offset of 300 ms.

B. Sensor Services Description

This section presents the hardware setup that was used tocollect the data from services using real sensors. In addition tothe devices presented in Table I, we used the following sensorsto collect real physical data: a HC-SR501 Infrared MotionSensor, a MQ-2 Smoke/LPG/CO Gas Sensor, a BMP180Barometer Pressure/Temperature/Altitude Sensor, a DHT11FR4 Temperature/Humidity Sensor Module and a Keyes KY-018 Photo resistor Module. These sensors are connected tothe Arduino devices using the I/O pins, and their data offeredas RESTful services. Figure 4 shows the layout of the devicesused to collect the data. The details about the connection of thesensors to the Arduino boards are omitted to avoid overloadingthe figure. Instructions of how these sensors can be connectedto an Arduino board can be provided through a number ofdocumented resources (e.g., Arduino [34]).

All devices used in the experiments connected througha router. Two different types of connections are used: theRaspberry Pi and the Arduinos are connected wirelessly to therouter, while the Linux Workstation uses a wired connection.We used the ESP8266WiFi library [35] to establish a wirelessconnection between the Arduinos and the router, and thestandard WiFi driver for the Raspberry Pi device.

The head node (Raspberry Pi) acts as the user. The data fromeach sensor connected to each Arduino device is made avail-able as a service. Arduino 192.168.1.200 enables the followingservices: motion service, tempDHT service, humDHT service,gas service. Arduino 192.168.1.201 enables the followingservices: realAltitude service, tempBMP service, pressureAt-SeaLevel service, altitude service, photosensor service, pres-sure service. The services on each Arduino were implemented

192.168.1.100 192.168.1.201192.168.1.200

Head Node

Linux Workstation

192.168.1.101

S1S2

S4S3 S1

S2

S4S3

Wired Connection

Wireless Connection

Legend

S1S2

S4S3 Services Deployed

User

Fig. 4: Experimental Setup

using the ESP8266WebServer library [35]. For each service weallocate one virtual user using JMeter to increase the workloadof the services.

C. Metrics

To evaluate the forecasting accuracy of the proposed algo-rithms, we use standard error metrics: the Root Mean SquareError (RMSE), which is defined as:

RMSE =

√1

N

∑i

(yi − y∗i )2 (11)

where yi is the QoS value of the service observed by theuser at time i and y∗i denotes the forecast QoS value of theservice. RMSE gives large weight to extreme errors due tothe squaring term.

D. Performance Comparison

In our experiments we compare the performance of thepersistence, ARIMA, GRU and LSTM models. The datasetis divided into a training and test set, where 70% of thedata is used for training and the remaining data is used totest the model. The forecasts are made one time-step aheadand are evaluated against the known value using the metricsdefined in the previous section. We train the GRU and LSTMmodel using a batch size of 1, 3000 epochs and 4 neurons. Wecompile the network using the Adam optimisation algorithmand mean absolute percentage error as the loss function. Theseparameters were selected after some experimentation with thetraining dataset. As neural networks are sensitive to unscaleddata we normalize the dataset using a MinMaxScaler to givea feature range from -1 to 1. Each of the models repeat theexperiment 10 times with the average error metrics reportedin the Tables II-II and the standard deviation in Figures 5-6.

TABLE II: Web Services

Service Persistance ARIMA GRU LSTMAmazon 10975.65 10977.66 11159.63 10614.35Bliquidity 7132.73 7133.44 7246.60 7131.87CurrencyConverter 47234.75 47245.70 43416.07 45669.66FastWeather 3755.30 3756.76 4022.64 3931.14GetJoke 3193.56 3194.36 3278.34 3143.64Google 5226.96 5228.01 5254.03 5225.74HyperlinkExtractor 4677.66 4678.55 4974.97 4825.17QuoteOfTheDay 3188.34 3189.00 3238.42 3216.08StockQuotes 10512.41 10514.29 10527.75 10494.04XMLDaiyFacts 10207.65 10209.83 10207.18 9995.93

For the persistence and ARIMA model we get the same modelfor each of the runs so the standard deviation is 0.

V. RESULTS

A. Web Services

Table II shows the RMSE for the forecasting models on theestablished web services dataset. Unlike previous evaluations,we have included all outliers in the dataset to evaluate howrobust the models are in response to outliers. As the servicesfor this dataset are invoked every hour there can be a largechange in the response time, which makes it difficult toforecast values. The LSTM model performs the best for 60%of the services, persistence for 30% and GRU for 10%.

The results are also plotted in Figure 5, which shows theaverage RMSE and the standard deviation over the 10 runs.The GRU and LSTM models perform well and reduce theprediction error when compared to current state of the artARIMA models. One of the surprises of this experiment is thatthe naıve persistence model performs better than state of theart models for three of the services. This could be because theservices are monitored every hour, which is a low frequencythat can lead to a lot of variance in the response time that canbe difficult to forecast.

B. IoT Services

Table III shows the results for the IoT dataset. The datasetis based on response times of services deployed on low powerdevices, which are invoked every 5 minutes over a period of 1month. As the dataset is collected with greater frequency, weget much less deviation. The LSTM model improves resultsfor 80% of the services with the GRU and persistence modelimproving 10% of the services each.

The results are also plotted in Figure 6, which shows theaverage RMSE and the standard deviation over the 10 runsfor the IoT Services. Compared to Figure 5 we can see theeffect the increase in monitoring frequency has on the results.If the services are monitored more frequently it leads to lesssudden jumps in QoS values, which can be forecast accuratelyusing the LSTM model. If the monitoring frequency is too lowit can lead to sudden jumps that can be hard to forecast. Inthose cases simple persistence models can be affective.

VI. RELATED WORK

A number of techniques have previously been used forQoS attribute forecasting. A recent study conducted a detailed

Amazon Bliquidity CurrencyConverter FastWeather GetJoke

0

10000

20000

30000

40000

50000

RM

SE

Persistence

ARIMA

GRU

LSTM

(a)

Google HyperlinkExtractor QuoteOfTheDay StockQuotes XMLDailyFacts

0

2000

4000

6000

8000

10000

RM

SE

Persistence

ARIMA

GRU

LSTM

(b)

Fig. 5: Web Services

TABLE III: IoT Services

Service Persistance ARIMA GRU LSTMAltitude 3.150 3.150 3.134 3.047Gas 5.890 5.891 5.342 5.064HumDHT 6.103 6.104 6.107 5.517Motion 7.810 7.811 7.389 6.892Photosensor 7.488 7.489 7.488 6.758Pressure 4.683 4.684 4.683 4.514PressureAtSeaLevel 7.020 7.030 9.168 8.644RealAltitude 4.301 4.302 4.295 4.192TempBMP 8.109 8.109 6.551 7.178TempDHT 7.412 7.413 7.412 7.351

empirical analysis of time series forecasting for the dynamicquality of web services [11], however they do not take intoaccount using LSTM networks as we propose. Time seriesforecasting techniques can be categorised into two groups:classical methods based on statistical/mathematical conceptsand modern heuristic methods based on artificial intelligencealgorithms [36]. The former includes exponential smoothingmodels, regression models, ARIMA models, threshold modelsand GARCH models. The latter includes artificial neural net-

Altitude Gas HumDHT Motion Photosensor

0

1

2

3

4

5

6

7

8

RM

SE

Persistence

ARIMA

GRU

LSTM

(a)

Pressure PressureAtSeaLevel RealAltitude TempBMP TempDHT

0

2

4

6

8

10

12

RM

SE

Persistence

ARIMA

GRU

LSTM

(b)

Fig. 6: IoT Services

works, which we extend to include LSTMs and evolutionarycomputing-based approaches.

Classical QoS forecasting methods have included tradi-tional AR-based methods/variations, such as SETAR, ARIMAand GARCH models. Two time-series methods ARIMA andGARCH have been combined to produce more accurate QoSforecasting results than traditional ARIMA but requiring ex-tra processing time [10]. Two common time series methodsARIMA and SETARMA were also considered, but the methodused depends on the linearity of the predicted QoS time seriestraining data. The incoming time series is tested to detectwhether it is linear, in which case the ARIMA method is usedotherwise the SETARMA method is adopted. The traditionalARMA model has also been used without the integrated (I)component used in ARIMA [37], [38].

The approaches based on traditional AI algorithms havefocused on ANNs but have not compared their approachagainst any established method [39], [40], [41]. RNNs havedeveloped in recent years as the most used neural network fortime-series forecasting as it can utilise past data through its

feedback path. Recent experiments using the LSTM architec-ture have shown promising results in other domains such asevent forecasting [15], acoustic modelling [42] and sequencetagging [43]. Initial attempts to use deep learning for long-term service composition have used generated synthetic data,which limits the validity of the results [16]. In our experimentswe consider real QoS factors gathered from real-life services.

VII. CONCLUSIONS

The results have provided some interesting insights forQoS attribute forecasting. They show that LSTM networkscan improve the prediction accuracy compared to traditionalforecasting approaches such as ARIMA. We have also shownthe importance of frequency of service monitoring. When theservices are measured every hour there is a large variationin the response time, which can make it difficult to forecastaccurately. This leads to the baseline forecasting model out-performing current state of the art models for three of theservices in the web service dataset in Table II.

When the monitoring frequency is increased to 5 minutesthe state of the art models perform better and are able toincrease the prediction accuracy in almost all the services.This tradeoff can be mapped to the categorisation of QoSrequirements by criticality in Section II. For hard and softIoT applications we can use an LSTM model and increasethe frequency of the monitoring engine. For best effort wecan maintain a lower monitoring frequency and use a simplepersistence model. In future work we plan to optimise ourcurrent network architecture to improve forecasting accuracyby experimenting with different layers and levels of dropout.

We also plan to increase the IoT dataset that we are using byinvoking services from multiple locations around the world ona variety of IoT devices. We plan to record this for an extendedperiod of time (four months). This dataset will allow for acomprehensive evaluation of our optimised LSTM network.This data will be made publicly available on our website1,which will allow for further research on QoS forecasting on acomprehensive dataset for direct comparison between currentstate of the art forecasting methods.

The goal of this research is to use forecasted values todetect future SLA violations in currently executing services toallow for proactive service discovery and recomposition beforethe violation happens. This approach can also be incorporatedinto service recommendation using collaborative filtering byforecasting future QoS values for similar users based on his-torical invocations. There is no single most suitable time-seriesmethod for dynamic QoS for every situation and requirement.In this paper we consider the requirements of forecasting QoSfactors for dynamic service re-composition, which requiresaccurate forecasting of atomic services. This will allow formore accurate service recommendations, which take into ac-count temporal information. The combination of collaborativefiltering and an accurate QoS forecasting technique wouldallow for soft QoS guarantees in IoT applications, which isa crucial requirement for the adoption of these applicationsby users.

ACKNOWLEDGMENT

This work was funded by Science Foundation Ireland (SFI)under grant 13/IA/1885.

REFERENCES

[1] I. Lee and K. Lee, “The internet of things (iot): Applications, invest-ments, and challenges for enterprises,” Business Horizons, vol. 58, no. 4,pp. 431 – 440, 2015.

[2] A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, andM. Ayyash, “Internet of things: A survey on enabling technologies,protocols, and applications,” IEEE Comm. Surveys Tutorials, vol. 17,no. 4, pp. 2347–2376, 2015.

[3] D. Miorandi, S. Sicari, F. D. Pellegrini, and I. Chlamtac, “Internetof things: Vision, applications and research challenges,” Ad HocNetworks, vol. 10, no. 7, pp. 1497 – 1516, 2014. [Online]. Available:http://www.sciencedirect.com/science/article/pii/S1570870512000674

[4] G. White, A. Palade, C. Cabrera, and S. Clarke, “Quantitative evaluationof qos prediction in iot,” in 2017 47th Annual IEEE/IFIP InternationalConference on Dependable Systems and Networks Workshops (DSN-W),June 2017, pp. 61–66.

[5] G. White, A. Palade, and S. Clarke, “Qos prediction for reliable servicecomposition in iot,” in Service-Oriented Computing: 15th InternationalConference, ICSOC 2017, Workshop (ISyCC), Malaga, Spain, November13-16, 2017, Proceedings. Springer International Publishing, 2017.

[6] L. Zeng, B. Benatallah, A. H. H. Ngu, M. Dumas, J. Kalagnanam, andH. Chang, “Qos-aware middleware for web services composition,” IEEETransactions on Software Engineering, vol. 30, no. 5, pp. 311–327, May2004.

[7] G. White, V. Nallur, and S. Clarke, “Quality of service approaches iniot: A systematic mapping,” Journal of Systems and Software, vol. 132,pp. 186 – 203, 2017. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S016412121730105X

[8] Z. Ye, S. Mistry, A. Bouguettaya, and H. Dong, “Long-term qos-awarecloud service composition using multivariate time series analysis,” IEEETrans. Serv. Comp., vol. 9, no. 3, pp. 382–393, 2016.

[9] A. Amin, L. Grunske, and A. Colman, “An automated approach toforecasting qos attributes based on linear and non-linear time seriesmodeling,” in 2012 Proceedings of the 27th IEEE/ACM ASE, 2012, pp.130–139.

[10] A. Amin and A.Colman, “An approach to forecasting qos attributes ofweb services based on arima and garch models,” in IEEE 19th ICWS,2012, pp. 74–81.

[11] Y. Syu, J.-Y. Kuo, and Y.-Y. Fanjiang, “Time series forecasting fordynamic quality of web services: An empirical study,” Journal ofSystems and Software, 2017.

[12] T. H. Lai, “Time series analysis univariate and multivariate methods:William w.s. wei, (addison-wesley, reading, ma, 1990),” InternationalJournal of Forecasting, vol. 7, no. 3, pp. 389–390, 1991. [Online].Available: http://EconPapers.repec.org/RePEc:eee:intfor:v:7:y:1991:i:3:p:389-390

[13] T. Opitz, “Modeling asymptotically independent spatial extremes basedon Laplace random fields,” ArXiv e-prints, Jul. 2015.

[14] X. SHI, Z. Chen, H. Wang, D.-Y. Yeung, W.-k. Wong, and W.-c.WOO, “Convolutional lstm network: A machine learning approach forprecipitation nowcasting,” in Advances in Neural Information ProcessingSystems 28, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, andR. Garnett, Eds. Curran Associates, Inc., 2015, pp. 802–810.

[15] N. Laptev, J. Yosinski, L. E. Li, and S. Smyl, “Time-series extreme eventforecasting with neural networks at uber,” in International Conferenceon Machine Learning, Conference Proceedings.

[16] H. Labbaci and B. M. Y. Aklouf, “A deep learning approach for quality-aware long-term service composition.”

[17] K. Cho, B. van Merrienboer, D. Bahdanau, and Y. Bengio,“On the properties of neural machine translation: Encoder-decoderapproaches,” CoRR, vol. abs/1409.1259, 2014. [Online]. Available:http://arxiv.org/abs/1409.1259

[18] M. Assaad, R. Bone, and H. Cardot, “A new boosting algorithmfor improved time-series forecasting with recurrent neural networks,”Information Fusion, vol. 9, no. 1, pp. 41–55, 2008.

[19] O. Ogunmolu, X. Gu, S. Jiang, and N. Gans, “Nonlinear systemsidentification using deep dynamic neural networks,” arXiv preprintarXiv:1610.01439, 2016.

[20] X. Wang, J. Liono, W. Mcintosh, and F. D. Salim, “Predicting the cityfoot traffic with pedestrian sensor data,” 2017.

[21] G. White, A. Palade, C. Cabrera, and S. Clarke, “IoTPredict: collabora-tive QoS prediction in IoT,” in 2018 IEEE International Conference onPervasive Computing and Communications (PerCom) (PerCom 2018),Athens, Greece, Mar. 2018.

[22] L.-M. Liu, G. B. Hudak, G. E. Box, M. E. Muller, and G. C. Tiao,Forecasting and time series analysis using the SCA statistical system.Scientific Computing Associates DeKalb, IL, 1992, vol. 1, no. 2.

[23] J. Schmidhuber, “Deep learning in neural networks: An overview,”Neural Networks, vol. 61, pp. 85–117, 2015, published online 2014;based on TR arXiv:1404.7828 [cs.NE].

[24] P. J. Werbos, “Generalization of backpropagation with application to arecurrent gas market model,” Neural networks, vol. 1, no. 4, pp. 339–356, 1988.

[25] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical Evaluationof Gated Recurrent Neural Networks on Sequence Modeling,” ArXive-prints, Dec. 2014.

[26] R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty of trainingrecurrent neural networks,” in International Conference on MachineLearning, 2013, pp. 1310–1318.

[27] D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translationby jointly learning to align and translate,” CoRR, vol. abs/1409.0473,2014. [Online]. Available: http://arxiv.org/abs/1409.0473

[28] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” NeuralComput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997. [Online]. Available:http://dx.doi.org/10.1162/neco.1997.9.8.1735

[29] B. Cavallo, M. Di Penta, and G. Canfora, “An empirical comparisonof methods to support qos-aware service selection,” in Proceedingsof the 2Nd International Workshop on Principles of EngineeringService-Oriented Systems, ser. PESOS ’10. New York, NY, USA:ACM, 2010, pp. 64–70. [Online]. Available: http://doi.acm.org/10.1145/1808885.1808899

[30] “Raspberry pi 3 model b,” https://www.raspberrypi.org/products/raspberry-pi-3-model-b/, accessed: March 31, 2018.

[31] “Arduino wemos d1,” https://wiki.wemos.cc/products:d1:d1, accessed:March 31, 2018.

[32] “Cisco linksys ea6400,” https://www.linksys.com/us/p/P-EA6400/, ac-cessed: March 31, 2018.

[33] “Apache jmeter,” http://jmeter.apache.org, accessed: March 31, 2018.[34] “Arduino homepage,” https://www.arduino.cc/, accessed: March 31,

2018.[35] “Arduino libraries,” https://github.com/esp8266/Arduino/tree/master/

libraries, accessed: March 31, 2018.[36] N. Wagner, Z. Michalewicz, M. Khouja, and R. R. McGregor, “Time

series forecasting for dynamic environments: the dyfor genetic programmodel,” IEEE transactions on evolutionary computation, vol. 11, no. 4,pp. 433–452, 2007.

[37] M. Godse, U. Bellur, and R. Sonar, “Automating qos based serviceselection,” in 2010 IEEE International Conference on Web Services,July 2010, pp. 534–541.

[38] Y. Xia, J. Ding, X. Luo, and Q. Zhu, “Dependability prediction ofws-bpel service compositions using petri net and time series models,”in 2013 IEEE Seventh International Symposium on Service-OrientedSystem Engineering, March 2013, pp. 192–202.

[39] T. Senivongse and N. Wongsawangpanich, “Composing services ofdifferent granularity and varying qos using genetic algorithm,” in Pro-ceedings of the World Congress on Engineering and Computer Science,vol. 1, 2011.

[40] Y. Syu, Y.-Y. Fanjiang, J.-Y. Kuo, and S.-P. Ma, “Applying genetic pro-gramming for time-aware dynamic qos prediction,” in Mobile Services(MS), 2015 IEEE International Conference on. IEEE, 2015, pp. 217–224.

[41] M. H. Zadeh and M. A. Seyyedi, “Qos monitoring for web servicesby time series forecasting,” in Computer Science and InformationTechnology (ICCSIT), 2010 3rd IEEE International Conference on,vol. 5. IEEE, 2010, pp. 659–663.

[42] H. Sak, A. Senior, and F. Beaufays, “Long short-term memory recurrentneural network architectures for large scale acoustic modeling,” inFifteenth Annual Conference of the International Speech CommunicationAssociation, 2014.

[43] Z. Huang, W. Xu, and K. Yu, “Bidirectional lstm-crf models forsequence tagging,” arXiv preprint arXiv:1508.01991, 2015.