hakometar and hakometar plus - itu · 2018-10-16 · provides additional information that can be...
TRANSCRIPT
HAKOMetar
and
HAKOMetar Plus
2HAKOM © 2018 3.10.2018.
Introduction
Number of users who daily access and use wide range of the information society services increases rapidly.
Basic requirements for a usable access to Internet – broadband access speed and possibility to connect to any point of choice and use the services with acceptable quality.
As the availability of broadband access increases, so does the number of user complaints increase
– achievable broadband speeds, blocking of some applications or slowing downsome traffic in the network.
All those problems related to the way the traffic is treated in the network are referred to as network neutrality issues.
3HAKOM © 2018 3.10.2018.
Legal Basis
To be ahead of the network neutrality issues and to decrease the number ofcomplaints on Internet speed, HAKOM introduced measures for increasedtransparency towards end users.– measures have been embedded in the Ordinance on Manner and Conditions for the
Provision of Electronic Communications Networks and Services.
Operators must advertise minimum broadband speed in the same way andmanner as they advertise maximum speed.
Operators of fixed communications networks must specify and guaranteeminimum broadband speed for the contracted communications services.– minimum speed for packages with broadband access speed up to 10Mb/s - at least
50% of the maximum (advertised) broadband speed during the peak traffic (19-23h)*.
– minimum speed for the packages with broadband access speed above 10 Mb/s - atleast 70% of the maximum (advertised) broadband speed during the peak traffic*.
To check the speed of the broadband access service a new tool namedHAKOMetar has been developed.– a certified tool for broadband speed tests prepared for the HAKOM.
* from 1.1.2017 for all packages - at least 70% of the maximum (advertised) broadband speed during thepeak traffic
4HAKOM © 2018 3.10.2018.
Legal Basis
Motivation for making HAKOMetar - to gather much more data about theend user connection in order not just to be able to provide informationabout the possible quality degradation, but to be able to understand thereasons for it, if the measurements prove its existence.
There are a number of tools on the Internet today that can be used forupload and download speed measurements. However, HAKOMetarprovides additional information that can be used for a deeper analysis ofthe measurement results.
Correct measurement: – Three measurements within five days (3/5) – whole cycle
– Between 19:00h and 23:00h
Results of the tests represent adequate proof in the procedure for theresolution of complaints made by end users.– If the results showed that the operator did not ensure to the end user minimum
broadband access speed - user can change to a package which is moreappropriate for the delivered broadband speed or to terminate the contractwithout compensation, at the user’s discretion.
5HAKOM © 2018 3.10.2018.
Broadband Measurement Tool - HAKOMetar
In November 2012 HAKOM, in cooperation with CARNet, launched certified software tool for measuring the speed of the broadband Internet access.
Client application measurement
– The application is available on HAKOM’s website and operators website
– Multiplatform app – Java based (run on most OS’s)
– Detailed measurements
– Two ways of running application: a) WEB start java and b) standalone application
– Unique ID for each started measuring cycle,
For the correct use, the following conditions must be met:
– end user must connect his/her client device directly with the wired connection tothe CPE set up as a terminating device by the ISP
– only this client device (and the CPE) should be present in the local area network ofthe end user
– end user must close all applications on his/her device apart from HAKOMetar
– client device should have enough CPU, disc space and memory resources available
6HAKOM © 2018 3.10.2018.
Broadband Measurement Tool - HAKOMetar
Functional levels:
Collecting data about the user (identification), about the operator (IPaddress -> name of the operator) and about users computer and localnetwork with/from which the measurements are performed
Measuring the availability of data content at the reference servers;measuring the availability of data content on the Internet
Beside speed measuring of other Internet access quality parameters:delay, jitter, packet loss
Data Recording
Sending complaint by e-mail to the operator (optional)
7HAKOM © 2018 3.10.2018.
Measurement topology with all the steps that will ensure correct measurement
Operator networkCARNet network
Customer’s
local
networkCustomer
terminal
Connection to
operator’s network
1.
Internet
6.
2.
3.
5.4.
Croatian Internet
Exchange point
(CIX)
1. Download application from a server and activation2. & 3. Collecting data about local area network and user computer4. Measurement toward reference servers5. Measurement toward Internet6. Record test results in the database on a server
8HAKOM © 2018 3.10.2018.
1. Collecting data about the user
Data collected from subscriber:
Name and surname Address, postal code and city Phone number (fixed) Contact phone number User name to access the network (if there is one) Name of the operator with which the user/subscriber has contract The contracted access speed e-mail address
9HAKOM © 2018 3.10.2018.
2. Collecting data about userslocal network
Data collected about local network:
Computer network interface (IP and MAC address) – wireinterface is mandatory
Network interface load
Active connection
ARP table - reveals computers in the local network
Routing table
DNS server used by consumer computer
CPU/Memory usage
Operating system
• ……
10HAKOM © 2018 3.10.2018.
3. Measuring the availability of data content at the reference servers
Checking reference server load
Measuring the download and upload transmission speeds
Availability of devices (ping) and traceroute to the server (delay,path)
Useful for:
assessing speed connection to / from CARNet
shared services between CARNet and operators
This measurement can be used only as a information!!!!!
11HAKOM © 2018 3.10.2018.
4. Measuring the availability of data content
on the Internet
Parallel HTTP and FTP test
establishment of a large number of simultaneously HTTP and FTP connections to the data content on the Internet -> stabilization
checking CPU load and memory usage
Measurement objective: to obtain maximum speed between theuser and the public internet
Possible limitations:
user's PC is too weak to carry out measurements (indicated by measuring the performance of users PC)
Operator blocks some protocols (operator has obligation to publish transparent any possible restrictions to the end user)
Operator blocks some addresses (solution: alternative address is taken)
Results can be used as a proof in case of customer complaintswhich can be submitted to operator through HAKOMetar
12HAKOM © 2018 3.10.2018.
5. Data Recording
The data is recorded locally on the user's computer
8193_2012-11-13_not_valid.txt
8193_2012-11-13_valid.txt
8193_2012-11-13_11-29-34_key.txt
Data is written to the server (HAKOM)
Checking the validity of the information reported
13HAKOM © 2018 3.10.2018.13
Implementation of HAKOMetar and enabling users to measure with it - 5 November 2012
Available on: http://www.hakom.hr/default.aspx?id=1144
In almost 6 years of HAKOMetar availability large number of measurements was performed
Most common reasons for unvalid measurements:
• test was made outside the term of 19-23 hours (until Januar 2015)
• during the HTTP/FTP download test, traffic on the network card was significantly increased compared to the traffic caused by the test
• CPU overload during HTTP/FTP download test
In practice…
14HAKOM © 2018 3.10.2018.
Analysis of the Results
Total number of started cycles - over 75,000 in the observed period• compared with the total number of measurements (approx. 110 000)- many
users started new measurement cycles without previously completing thealready opened measurements).
Over 2000 completed cycles - several hundreds of cycles (approx. 700)has grounds for appeal based on the measured speeds that were well-below the contracted (minimal) speed
Individual measurements with reduction of some precondition rules (i.enumber and period of measurement cycles) has shown that the majorityof the users who have performed the measurements using HAKOMetar areachieving minimum speeds stipulated by the Ordinance.
15HAKOM © 2018 3.10.2018.
Conclusion
Transparency of contracts:satisfied customers andprotection for customers and operators by providing legalcertainty in the interpretation of the rights and obligations of theparties
HAKOMetar:
facilitated the resolving of complaints on the quality of services
stimulated competition among service providers – in the endthat leads to improvement in quality of service and lessregulatory intervention
HAKOMetar Plus
17HAKOM © 2018 3.10.2018.
Introduction
HAKOMetar Plus – mobile (crowd sourcing) application for iOS and Androidsmartphones (vendor: alladin-IT)
Measurement tool for users which measures the download and upload datarate as well as the Ping and signal strength• Information for users about current service quality of their internet connection
(mobile and WLAN network)• Awareness and knowledge on actual data rates and service quality; Comparable
test results between users• Easy to use, easy to understand• Low data consumption• Flexible (different platforms, mobile/fixed)
Transparent measurement tool• Open Data principle (Tests results are available as open data)• Open Source principle (Android and iOS App)
Presentation of results• Map with all test results with filter options (eg. technology: mobile 2G/3G/4G,
WLAN; download/upload/Ping/Signal strength)• Statistics on all tests with filter options
18HAKOM © 2018 3.10.2018.
Motivation
REGULATION (EU) 2015/2120 OF THE EUROPEAN PARLIAMENT AND OF THECOUNCIL of 25 November 2015
• The provisions on safeguarding of open internet access should be complemented byeffective end-user provisions which address issues particularly linked to internet accessservices and enable end-users to make informed choices.
• …national regulatory authorities should assess the impact on the availability and generalquality of internet access services by analysing, inter alia, quality of service parameters(such as latency, jitter, packet loss), the levels and effects of congestion in the network,actual versus advertised speeds, the performance of internet access services ascompared with services other than internet access services, and quality as perceived byend-users.
BEREC Guidelines on the Implementation by National Regulators ofEuropean Net Neutrality Rules (BoR(16)127)
• It would help make the rights enshrined in the Regulation more effective if NRAs were toestablish or certify one or more monitoring mechanisms that allow end-users todetermine whether there is non-conformity of performance and to obtain relatedmeasurement results for use in proving non-conformity of performance of their IAS. Theuse of any certified mechanism should not be subject to additional costs to the end-userand should be accessible also to disabled end-users
Three different servers:
Measurement server (physical server)Control server (2 x virtual servers)
Measurement server is located in CIX:2x 1Gb interfacededicated VLAN partIPv4/IPv6 connectivity
HAKOMetar Plus architecture
HAKOM CIX
Hakometar Plus Control Server
- Web Server
- Database Server
- Map Server
- Statistics server
Hakometar Plus Measurement Server
- Speedtest Server
- QoS Server
- Searching through results
- Statistics
- Map presentation
20HAKOM © 2018 3.10.2018.
HAKOMetar Plus
Test Procedure
Initialization
Downlink pre-test
Latency test
Downlink test
Uplink pre-test
Uplink test
Finalization
Initialization – establishing a connection to the Control server to send the testrequest (basic information about the device, the current time and timezone, thelanguage and the client uuid )
Downlink pre-test – ensures that the Internet connection is in an “active” state andgives rough estimate of the bandwidth
Ping test – sending brief text messages and measuring time interval in millisecondsbetween sending the request and receiving the response from the server
Downlink test – client simultaneously requests and the server continuously sendsdata streams consisting of fixed-size chunks
Uplink pre-test - Similar to download pre-test but in opposite direction
Uplink test - Similar to download test but in opposite direction
Finalization – test results are submitted to the Control Server
Source: “Specure NetTest”
21HAKOM © 2018 3.10.2018.
HAKOMetar Plus (2)
QoS measurements
Website Rendering – load and render HTML Reference website (Kepler)
DNS – detect DNS-Hijacking (DNS Spoofing), DNS correctness
TCP Ports – detecting blocked TCP ports (npr. 25, 110, 20, 80,..)
UDP Ports – detecting blocked UDP ports (npr. 53, 5060, 27015…)
Voice over IP – short simulated VoIP call in order to measure VoIP (RTP)capability
Non Transparent connection– detection of Non Transparent Proxys
Unmodified content– detection of content modification (MD5 hash).
Traceroute - determining network path to destination
22HAKOM © 2018 3.10.2018.
HAKOMetar Plus
23HAKOM © 2018 3.10.2018.
HAKOMetar Plus
24HAKOM © 2018 3.10.2018.
HAKOMetar Plus – graphical and statistical depiction of results
25HAKOM © 2018 3.10.2018.
Measurement Data
Basic data• Downlink and Uplink data rate, Latency• Date and timeNetwork information• Type and technology• Amount of transferred data (traffic)• Provider information• NAT statusLocation information• GPS coordinates (if available)• IP locationDevice information• Manufacturer , Model, OS versionAdvanced• Signal strength (only for Android)• CPU & RAM usage
…
26HAKOM © 2018 3.10.2018.
Measurement Data – published with explicit approval of the user
ping (latency), speed and quantity of transmitted data in both directions(downlink/uplink);
Quality and transparency parameters (e.g. signal strength, connectivityon different ports, modifications of content, transfer time for a referenceweb page), and reliability of the measurement;
Test parameters (test method, test progress, test server, testconfiguration);
Name and country of service provider;
Personal data:• Public measurement ID (generated upon each measurement);• Geographic location of the client and direction of movement;• Type of device/model, OS;• Public measurement ID (generated upon each measurement);
27HAKOM © 2018 3.10.2018.
Measurement Data – which shall not be published
Randomly generated client ID (client UUID) and the number of performedtests;
information on the client (language, local IP, version, test time and timeband);
Public IP address of the user;
WLAN network ID (WiFi SSID), numerical WLAN ID (WiFi BSSID) andencryption type
28HAKOM © 2018 3.10.2018.
In April 2017. HAKOMetar Plus was launched – until now downloaded over 14000 times (iOS and Android marketplace).
provides information about current service quality of internet connection(mobile and WLAN network) and helps in building awareness andknowledge on actual data rates and service quality.
Results of the tests are informative - they don’t represent adequateproof in the procedure for the resolution of complaints made by endusers.
Statistics: over 40 000 measurements in mobile network and 110 000measurements in WLAN network
results which are summarised into aggregated values for differentcategories and geographically represented on maps can be used forcomparison of IAS offers in the market.
HAKOM uses these results for cross-checking that the publishedinformation (coverage maps and available speeds in mobile networks)are consistent with monitoring results.
HAKOMetar Plus
