mobile communications systems gsm/umts/lte

Click here to load reader

Post on 30-Jan-2022

0 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

Relatório FinalG S M / U M T S / L T E
EEvvaalluuaattiioonn ooff QQuuaalliittyy ooff SSeerrvviiccee
Evaluation of the quality of GSM, UMTS and LTE voice services, data
services and radio network coverage in the Municipalities of Odemira and Vila Flor.
December 2014
Evaluation of Voice Services, Data Services and Network Coverage | Municipalities of Odemira and Vila Flor | September 2014 3
PRIOR NOTE
ANACOM HOLDS NO RESPONSIBILITY FOR ANY ANALYSIS, STUDY OR COMMENTARY
BASED ON THE RESULTS OF THIS REPORT.
Evaluation of Voice Services, Data Services and Network Coverage | Municipalities of Odemira and Vila Flor | September 2014 5
Contents
I.II Key Findings and Conclusions ............................................................................... Error! Bookmark not defined.
Evaluation of Voice Services, Data Services and Network Coverage | Municipalities of Odemira and Vila Flor | September 2014 7
I EXECUTIVE SUMMARY
I.I GENERAL FRAMEWORK
In September 2014, Autoridade Nacional de Comunicações (ANACOM) undertook an evaluation of the
quality of the voice and data services and GSM, UMTS and LTE radio network coverage provided by
the following operators in the main urban areas and along the main roads of the Municipalities of
Odemira and Vila Flor: MEO - Serviços de Comunicações e Multimédia, S.A. (MEO), NOS
Comunicações, S.A. (NOS) and VODAFONE PORTUGAL - Comunicações Pessoais, S.A. (Vodafone).
This study is based on field tests conducted with moving vehicles and outdoor antennas, using an
automatic measurement system, reflecting the various aspects that affect quality of services (end-to-end
measurements). Measurements were carried out simultaneously in the same locations and with the
same parameterization for the three operators.
Studies which seek to gauge the user´s perception of quality, such as the present study, are known as
Quality of Experience Studies (QoE), setting out to determine the user experience in terms, for example,
of service accessibility, retention and integrity. In contrast, analyses of Network Performance or Quality
of Service seek to assess the capacity of a network (or a part of a network) to provide a service with a
determined level of quality. This covers functions, mechanisms and procedures implemented by the
cellular network and by terminal equipment, which ensure the provision of the quality of service
negotiated between the terminal equipment and the core network infrastructure.
However, this does not mean that the results of a Quality of Experience Study represents quality as
perceived by an average user or by the majority of users, because such studies are always subject to
the conditions under which they were performed; so, for example, in the case of this study, it is
recognized that, in the most part, data services are not usually accessed outdoors using a laptop
computer connected to the network via a data card and an external antenna (as in the tests conducted).
Therefore, although the present study is a Quality of Experience Study, its analysis may not reflect, in
particular with regard to data services and respective coverage, the actual perception of the majority of
users.
Quality of Experience is usually expressed in terms of human feelings, such as "Excellent", "Good",
"Acceptable", "Poor" and "Bad", while Network Performance is a purely technical concept measured,
8 QoS-Mobile Networks and Services | GSM-UMTS-LTE
expressed and understood in the context of a network or network components, with little meaning as far
as the user is concerned.
To evaluate the voice service, capacity for call establishment and termination was measured, together
with the perceptibility of the communication.
In assessing the data service, an analysis was made of file transfer capacity (FTP upload / download),
Internet navigation (HTTP web browsing) and also transmission latency (ping).
To evaluate radio network coverage, measurements were made of the radio signal levels received by
the mobile terminal equipment (downlink) for each of the technologies used by the operators.
The main quality indicators analyzed for the services covered by the study were as follows:
1. Radio Network coverage - GSM, UMTS and LTE network availability;
2. Accessibility of the Voice Service - probability of success when establishing calls;
3. Voice Call Establishment Time - Length of time taken by the network to establish a
communication after correct submission of the request (destination telephone number);
4. Voice Call Termination Rate - Probability, after a call is successfully established, that it will
remain active for a period of time and terminate normally, i.e. as intended by the user;
5. Voice Call Audio Quality - Perceptibility of the conversation during a call;
6. Termination Rate of Sessions Using the Data Service - Probability that a session using the
data service - file transfer or Internet browsing - is established and is maintained successfully,
i.e., it remains active during the entire period required for the file transfer or to allow the transfer of
the entire web page;
7. Data Transfer Speed - Quantifies the average speed of data transfer during a file transfer
session;
8. Web Page Transfer Time - Quantifies the average time taken to download a reference web
page;
9. Data Transmission Latency - quantifies the time needed for an information packet to travel from
the user´s device to the Content Server or visa versa.
Evaluation of Voice Services, Data Services and Network Coverage | Municipalities of Odemira and Vila Flor | September 2014 9
Fieldwork was conducted, on working days and during normal working hours from 15 to 25 September
2014. 3,169 test calls were performed, 12,808 sessions using data services and 1,736,852
measurements taken of the radio network signal, which corresponds to about 54 hours of
measurements over approximately 2,242 kilometres.
Given the penetration rate of these services, the diversity of terminal equipment used and the very
subjectivity inherent to each user, it is impossible to accurately reproduce the conditions of each
consumer's interaction with the networks. A study of this type cannot be completed without adopting
certain compromises and estimates, within limits which ensure that the results obtained can be viewed
as a reliable indicator of the overall performance of mobile communication systems. The transposition or
extrapolation of the results to specific situations must be done with prudence, at the risk of drawing
conclusions that distort the reality of the situation.
The adopted technical and methodological options directly influence the results and should be taken into
account in their analysis. The following options are particularly relevant:
The testing was conducted exclusively using a technical solution (hardware and software) and
was processed completely automatically, allowing the homogeneous setting of measurement
conditions for the three operators and eliminating any subjectivity inherent to a human user;
The following terminal equipment was used: Nokia C7 and Huawei E392;
The tests were conducted with moving vehicles and outdoor antennas;
The duration of the conversation phase of voice calls was 120 seconds;
Voice and data tests were performed with automatic selection of GSM, UMTS and LTE
infrastructure;
The Servers, used in the data service tests, were hosted in Portugal, in a data centre at FCT -
Fundação para a Ciência e a Tecnologia, I.P. (Foundation for Science and Technology);
In data tests, stable content was used, i.e. web pages and files that had no quantitative or
qualitative changes over time;
File Transfer Tests (FTP upload / download) were performed according to the concept of fixed
data transfer time, i.e., data transfer is performed over a predefined period of time instead of the
full transfer of files of a fixed size;
10 QoS-Mobile Networks and Services | GSM-UMTS-LTE
Radio network coverage indicators, in particular, UMTS and LTE network coverage, did not take
network load into account (number of simultaneous users and type of services used);
The GSM, UMTS and LTE network coverage indicators presented in this study result from the
analysis of the radio channel provided by operators prior to the study´s performance. Due to an
omission by Vodafone, some of the LTE channels of this operator were not analyzed. According
to the operator, these channels were used on a local basis - Zambujeira do Mar, Guarda,
Bragança, Évora and Coimbra -, and this omission had a direct impact on the LTE network
coverage indicator reported for Vodafone in the urban centres of mainland Portugal and on the
statistical significance of this indicator;
In some parts of the Portuguese territory, especially near the Spanish border, performance of the
LTE networks of Portuguese operators was affected by Spanish DTT during the period of testing.
With the shutdown of Spanish transmitters, which occurred in early 2015, the performance of
Portuguese LTE networks in these locations is expected to improve;
The generic test scenario used by ANACOM in this study corresponds to a use of
communications services which is the most favourable given the strength of signal reception on
the mobile terminal device, due to the non-consideration of signal loss because of indoor
environments and due to use of mobile terminal equipment with outdoor antennas. This situation,
is not very significant as far as the Voice Service is concerned, but has significant impact on
Radio Network Coverage and on Data Services, resulting in higher levels of performances than
could be obtained indoors;
The servers used were dedicated exclusively to the performance of these tests, so that, in the context
of the study conducted, there are no limitations on processing power and information storage,
whereby availability and access bandwidth can have a positive influence on the results of the data
transmission speed (as opposed to the results that may be obtained by a general user accessing
conventional servers). Furthermore, the servers used, which function along the same lines as servers
available on the Internet, are identified by the respective IP addresses and not through their FQDN
(Fully Qualified Domain Name), ensuring that the measurements were not negatively affected by DNS
(Domain Name System) resolution problems or caching strategies;
The results of the study only reflect the behaviour of networks in the locations and at the times of
actual measurement;
Evaluation of Voice Services, Data Services and Network Coverage | Municipalities of Odemira and Vila Flor | September 2014 11
these improvements can cause momentary degradations of the service in the geographical areas
were work is performed.
Evaluation of Voice Services, Data Services and Network Coverage | Municipalities of Odemira and Vila Flor | September 2014 13
I.II KEY FINDINGS AND CONCLUSIONS
In the municipalities of Odemira and Vila Flor, extensive geographic areas persist with poor or non-
existent radio network coverage, affecting the performance of electronic communications services
supported by the mobile communications systems of the operators MEO, NOS and Vodafone.
The levels of coverage and performance of the services reported in this study are much lower than in
study conducted in the main urban centres and along the main roads of mainland Portugal (conducted
from 30 September to 3 December 2014), with very significant differences observed in performance
between the different operators, technologies and services analyzed.
GSM, UMTS and LTE Radio Network Coverage
The municipalities of Odemira and Vila Flor include extensive geographic areas with poor or non-
existent network coverage.
In the municipality of Odemira (Figure 1), the GSM coverage provided by MEO and NOS is acceptable
overall, while Vodafone provides poor coverage. All operators provide bad UMTS network coverage.
The LTE coverage of MEO and Vodafone is very weak, especially Vodafone´s, while the coverage
provided by NOS is virtually non-existent.
In the municipality of Vila Flor (Figure 2), the GSM coverage provided by MEO and NOS is generally
good and the coverage provided by Vodafone is acceptable. In terms of UMTS, Vodafone provides
good network coverage while MEO and NOS provide poor levels of coverage. LTE coverage is very
weak in this municipality, in particular as provided by Vodafone.
Comparing the results reported in these municipalities with the results obtained in the study conducted
in the main urban centres and along the main roads of mainland Portugal (Table 1 and Table 2), it is
noted that levels of good or acceptable coverage in these municipalities are much lower, especially
when it comes to UMTS and LTE technologies.
14 QoS-Mobile Networks and Services | GSM-UMTS-LTE
Figure 1 – Indicator: Radio Network Coverage, in the Municipality of Odemira.
Table 1 – Radio Network Coverage: Difference between the results observed in the study performed in Odemira (Sept. 2014) and in the study conducted in the major urban centres and along the main roads of mainland Portugal (30 Sept. - 3 Dec. 2014).
GSM UMTS LTE
R ad
io N
et w
or k
C ov
er ag
-------------------- -------------------- -------------------- -------------------- -------------------- -------------------- -------------------- -------------------- --------------------
Evaluation of Voice Services, Data Services and Network Coverage | Municipalities of Odemira and Vila Flor | September 2014 15
Figure 2 – Indicator: Radio Network Coverage, in the Municipality of Vila Flor.
Table 2 - Network Coverage: Difference between the results observed in the study performed in the municipality of Vila Flor (Sept. 2014) and in the study conducted in the major urban centres and along the main roads of mainland Portugal (30 Sept. - 3 Dec. 2014).
GSM UMTS LTE
R ad
io N
et w
or k
C ov
er ag
-------------------- -------------------- -------------------- -------------------- -------------------- -------------------- -------------------- -------------------- --------------------
Voice Service
The performance of the voice service, although generally lower than as reported in the study conducted
in the major urban centres and along the main roads of mainland Portugal (Table 3 and Table 4), is still
good according to most of the indicators analyzed. In Odemira, MEO and NOS provide the best levels of
performance, and in Vila Flor, MEO stands out positively (
Figure 3, Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8).
Figure 3 – Indicators: Accessibility of Service and Call Termination Rate in the Municipality of Odemira.
97,5% 98,2%97,5% 98,2% 94,0% 96,2%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Voice Service Mobile g Fixed
MEO NOS Vodafone
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Voice Service FixedgMobile
MEO NOS Vodafone
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Voice Service Mobile g Fixed
MEO NOS Vodafone
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Voice Service FixedgMobile
MEO NOS Vodafone
Evaluation of Voice Services, Data Services and Network Coverage | Municipalities of Odemira and Vila Flor | September 2014 17
Figure 4 – Indicators: Accessibility of Service and Call Termination Rate, in the Municipality of Vila Flor.
Table 3 - Voice Service: Difference between the results observed in the study performed in the Municipality of Odemira (Sept. 2014) and in the study conducted in the major urban centres and along the main roads of mainland Portugal (30 Sept. - 3 Dec. 2014).
MEO NOS Vodafone
V o
ic e
S er
vi ce
Accessibility of Service -1.9% -0.7% -1.1% -2.1% -5.0% -3.2%
Call Termination Rate -0.7% -1.2% -0.8% 0.1% -3.0% -3.7%
Average Call Establishment Time [s] 0.6 0.6 0.4 0.4 0.2 0.3
Average Call Audio Quality [MOS] -0.04 -0.12 -0.01 0.00 -0.15 -0.21
Table 4 - Voice Service: Difference between the results observed in the study performed in the Municipality of Vila Flor (Sept. 2014) and in the study conducted in the major urban centres and along the main roads of mainland Portugal (30 Sept. - 3 Dec. 2014).
MEO NOS Vodafone
V o
ic e
S er
vi ce
Accessibility of Service -3.8% -0.6% -4.2% -5.4% -3.4% -2.2%
Call Termination Rate -1.4% -1.4% -2.4% -2.7% -3.9% -4.6%
Average Call Establishment Time [s] 0.1 -0.1 0.6 0.7 0.0 0.0
Average Call Audio Quality [MOS] -0.04 0.04 -0.13 -0.20 -0.07 -0.14
95,6% 97,5% 94,4% 96,6%95,5% 95,3%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Voice Service MobilegFixed
MEO NOS Vodafone
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Voice Service FixedgMobile
MEO NOS Vodafone
18 QoS-Mobile Networks and Services | GSM-UMTS-LTE
Figure 5 – Indicator: Call Establishment Time, in the Municipality of Odemira.
Figure 6 – Indicator: Call Establishment Time, in the Municipality of Vila Flor.
4, 6
17 ,0
3, 3
4, 7
15 ,7
Evaluation of Voice Services, Data Services and Network Coverage | Municipalities of Odemira and Vila Flor | September 2014 19
Figure 7 – Indicator: Call Audio Quality, in the Municipality of Odemira.
Figure 8 – Indicator: Call Audio Quality, in the Municipality of Vila Flor.
File Transfer Service (FTP)
MEO NOS Vodafone
MEO NOS Vodafone
MEO NOS Vodafone
MEO NOS Vodafone
20 QoS-Mobile Networks and Services | GSM-UMTS-LTE
In Odemira (Figure 9 and Figure 11), MEO was the best performing operator with acceptable levels
reported in terms of File Transfer Session Termination Rate and Average Data Transfer Speed.
Vodafone reports acceptable levels for downloads and weak levels for uploads in terms of Average Data
Transfer Speed. NOS provides Average Data Transfer Speed that is bad for downloads and uploads. As
regards the File Transfer Session Termination Rate indicator, the performance of both Vodafone and
NOS is poor.
In the Municipality of Vila Flor (Figure 10 and Figure 12), the performance of all operators is poor as
regards File Transfer Session Termination Rate – in particular Vodafone at levels below 65%. With
respect to the Average Data Transfer Speed indicator, for downloads, MEO provides acceptable results,
while Vodafone and NOS are weak. For uploads all operators recorded poor performance.
The results recorded in the municipalities of Odemira and Vila Flor are significantly lower than those
recorded in the major urban centres and along the main roads of mainland Portugal (Table 5 and Table
6). On the negative side, Average Data Transfer Speed for uploads provided by NOS in Odemira stands
out, as it is less than 8% of the speed registered in the major urban centres and along the main roads of
mainland Portugal.
Figure 9 – Indicator: Data Session Termination Rate, in the Municipality of Odemira.
Figure 10 – Indicator: Data Session Termination Rate, in the Municipality of Vila Flor.
Table 5 - File Transfer Service (FTP): Difference between the results observed in the study performed in Odemira (Sept. 2014) and in the study conducted in the major urban centres and along the main roads of mainland Portugal (30 Sept. to 3 Dec. 2014).
MEO NOS Vodafone
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MEO NOS Vodafone
MEO NOS Vodafone
Evaluation of Voice Services, Data and Radio Coverage | Municipalities of Odemira and Vila Flor | September 2014 21
File Transfer (FTP)
Session Termination Rate -5.9% -6.4% -9.4% -9.4% -7.5% -7.2%
-------------------- -------------------- -------------------- -------------------- -------------------- --------------------
Table 6 - File Transfer Service (FTP): Difference between the results observed in the study performed in the municipality of Vila Flor (Sept. 2014) and in the study conducted in the major urban centres and along the main roads of mainland Portugal (30 Sept. to 3 Dec. 2014).
MEO NOS Vodafone
File Transfer (FTP)
Session Termination Rate -21.9% -22.0% -16.1% -16.7% -31.0% -30.5%
-------------------- -------------------- -------------------- -------------------- -------------------- --------------------
Figure 11 – Indicator: Data Transfer Speed, in the Municipality of Odemira.
Figure 12 – Indicator: Data Transfer Speed, in the Municipality of Vila Flor.
13 .6
MEO NOS Vodafone
MEO NOS Vodafone
MEO NOS Vodafone
MEO NOS Vodafone
Internet navigation (HTTP web browsing)
In the Municipality of Odemira (Figure 13), the performance of the mobile communication system
provided by MEO was poor in terms of its capacity for the establishment and maintenance of Internet
browsing sessions. The performance provided for the same indicator by Vodafone and NOS was bad.
Average Web Page Transfer Time is poor for MEO and Vodafone, and bad for NOS.
In the municipality of Vila Flor (Figure 14), the performance of mobile operators is even lower than in
Odemira. All operators have bad levels in terms of Internet Browsing Session Termination Rate, in
particular Vodafone,…