Tracking and Predicting Link Quality in Wireless Community Networks (WCN)

3rd Int. Workshop on Community Networks and Bottom-up-Broadband,

CNBuB 2014

October 8th, 2014. Larnaca, Cyprus

P. Millán1, C. Molina1, E. Molina2, Davide Vega2, R. Meseguer2, B. Braem3, C. Blondia3

 1Universitat Rovira i Virgili, Tarragona, Spain

2Universitat Politècnica de Catalunya, Barcelona, Spain3University of Antwerp - iMinds, Antwerpen, België

• Motivation

• [Link Quality] Prediction in [Wireless] Networks

• Experimental Methodology & Results

• Conclusions & Future Work

OLSROutlineOutline

2

Motivation3

MotivationMotivationCommunity networks create measurable social impact:

provide the right and opportunity of communication

4

• These large, decentralized, dynamic and heterogeneous structures raise challenges– What is the effect of the unreliability and

asymmetrical characteristics of wireless communications on routing protocols and network performance?

– Link quality tracking is a key method to applywhen routing packets through an unreliable network.

– Routing algorithms should avoid weak links whenever possible and as soon as possible.

MotivationMotivation

5

• Link quality estimation/prediction approach increases the improvements in routing performance achieved through link quality tracking.– RT metrics do not provide enough information to detect

degradation/activation of a link at the right moment.– Prediction techniques are needed

to foresee link quality changes in advance and take the appropriate measures.

MotivationMotivation

6

– Main contributions:• Use of time series analysis

to estimate link quality in the routing layer for real-world wireless mesh community networks.

• A detailed evaluation of the results obtained from several learning algorithms, showing the potential of time series to estimate link quality.

• Clear evidence that link quality values computed through time series algorithms can make accurate predictions in those WCN.

In this work we presenta link quality analysis and prediction

of Funkfeuer wireless mesh community network

7

Prediction in WCN8

• Energy Efficient Routing:– Lifetime Prediction Routing (LPR),

Minimum Drain Rate (MDR), E-DSR routing protocol.

• Routing Traffic Reduction:– OLSRp, Kinetic Multipoint Relaying (KMPR).

• Network Reliability:– Mobile Gambler’s Ruin (MGR).

• Link Quality prediction.

Goals of Network PredictionGoals of Network Prediction

9

• Link quality tracking:– To select higher quality links

that maximize delivery rate and minimize traffic congestion.

• Link quality prediction:– To determine beforehand

which links are more likely to change their behavior.

• Result:– The routing layer can make

better decisions at the appropriate moment.

Link Quality Prediction in WCNLink Quality Prediction in WCN

10

• Measure the quality of the links between nodes based on physical or logical metrics.

• Physical metrics focus on the received signal quality:– LQI (Link Quality Indication), SNR (Signal-to-Noise Ratio),

RSSI (Received Signal Strength Indication).

• Logical metrics focus on % of lost packets:– RNP (Required Number of Packets),

ETX (Expected Transmission Count), PSR (Packet Success Rate)

• To select the more suitable neighbor nodes when making routing decisions.

Link Quality Estimators (LQE) metricsLink Quality Estimators (LQE) metrics

11

• Routing protocol for wireless sensor networks that uses a learning-enabled method for link quality assessment.– Also uses time series analysis to improve the routing protocol.

MetricMapMetricMap

12

MetricMap:• Evaluates a small wireless sensor

network.• Gives only a flavor of the

potential of time series analysis to predict link quality.

• Applies a time series analysis to predict current link quality values.

• Uses a cross-validation method, which uses a subset of the sample data to validate LQE.

Our work:• We evaluate a large wireless

mesh community network.• We perform a detailed and deep

analysis of this potential.

• We use a time series to predict future link quality values.

• We use new data to validate the link quality estimation (LQE).

• Funkfeuer WCN (Austria):– 2.000+ links, OLSR-NG routing protocol.

• Open data set (Confine Project):– OLSR info, 404 nodes, 7 days, degree: 3.5, diameter: 16.– 1.032 links with variations in LQ (if all nodes: higher prediction accuracy).

• Link Quality:– ETX = 1 / (LQ × NLQ), LQ = %HELLO received.

• Time Series Analysis & Forecasting:– Training and test sets validation approach.– Weka: machine learning/data mining approach to model time series,

encodes time dependency via additional input fields (“lagged” variables).

• Metrics and Plots: • Mean Absolute Error (MAE). MAE = sum(abs(predicted - actual)) / N• Boxplots: classic representations of a statistical distribution of values.

Experimental MethodologyExperimental Methodology

13

• A sample of variation of LQ values of a link over a day

Variation of LQ valuesVariation of LQ values

14

Results

15

Comparison of learning algorithmsComparison of learning algorithms

Time series analysis and prediction

can be used to predict the next link quality value?

4 classification algorithms:• Support Vector Machines (SVM)• k-Nearest Neighbors (KNN)• Regression Trees (RT)• Gaussian Processes for Regression (GPR)

16

Data sets:

• Training: 1728 instances (6 days)

• Test: 288 instances (1 day)

Lag window: last 12 instances

BEST WORSE

Very high success rate:

>95%

Learning algorithms: error variabilityLearning algorithms: error variability

17

The four algorithms achieved a similar performance

for most of the links(median, 1st & 3rd quartile)

Some outliers have high errors

… that increase the average values

T-test result: RT is a good candidate

to predict LQ.

Impact of lag window sizeImpact of lag window size

What is the impact of lag window in the prediction of next LQ value?

18Same experimental setup

BEST

WORSE

These results are similar or even better than results obtained by other algorithms:

RT is the best candidate

T-test result: our results

do not provide clear evidence of

the best window size.

>97%

Prediction of some steps aheadPrediction of some steps ahead

Time series analysis and prediction can be used to predict the value of LQ some time steps ahead into the future?

19

Same experimental setup

The values of third quartile and outliers grow with steps ahead values.

These differences in the variability of errors lead to the differences in the average MAE.

Good results for all values of steps ahead

Average MAE grows slower than linear

>97%

Degradation of RT model over timeDegradation of RT model over time

What is the accuracy of the prediction models

over time?

20

Average MAE of the overall network and its approximation

to a linear function

Linear function:

slope = 0.0212

b = 0.0132

A linear function can be used to model the degradation of the RT over time

½ day 6 days

Variability of errors increases linearly with

the number of instances of the test data set

It is important to train the model again

after a period of time

Linear function: we could easily determine a trade-off between error & frequency of model updates.

97%84%

Evolution of prediction error over timeEvolution of prediction error over time

21

The larger the size of the training data set, the smaller the error

RT model was trained at

time 0

288 values predicted (1728 instances for training)

288 values predicted (288 instances for training)

Impact of the size of the training data set in the prediction error

Further analysis would be necessary to determine an ideal size for the training data.

22

ConclusionsFuture Work

• Time series analysis is a promising approach to accurately predict LQs in WCN

Routing protocol performance can be improved by providing information to make, at the right time, appropriate decisions to maximize delivery rate and minimize traffic congestion

• All algorithms achieved percentages of success between 95% and 98% when predicting the next value of LQ, being the Regression Tree the best one.

Prediction accuracy could have been even better including all the WCN links (not only those with variations).

• Prediction of values that are more than one step ahead also achieves high success ratios, between 97% and 98%.

• The size of the training data set is a key factor to achieve high accuracy of predictions.

– The bigger the data set size, the smaller the degradation of the error over time.

OLSRLink-Quality Prediction: ConclusionsLink-Quality Prediction: Conclusions

23

OLSRFuture WorkFuture Work

1) Identify which links contribute the most to the error of the link quality prediction

2) Understand what factors make difficult to predict the behavior of these links

3) Extend the analysis presented in this research work to other community networks, such as Guifi.net, to see if the observed behavior can be generalized.

24

Questions?

Thanks for Your Attention

3rd Int. Workshop on Community Networks and Bottom-up-Broadband,

(CNBuB 2014) October 8th, 2014. Larnaca, Cyprus

Questions?

3rd Int. Workshop on Community Networks and Bottom-up-Broadband,

(CNBuB 2014) October 8th, 2014. Larnaca, Cyprus