Integrated Information System for Inter-Domain QoS Monitoring, Modelling and Verification

D. Hetzer, I. Miloucheva**, U. Hofmann**, J. Quittek***, F. Saluta****T-Systems Nova, Deutsche Telekom, Berlin, Germany,

**SalzburgResearch, Salzburg, Austria, *** NEC Europe, Heidelberg, Germany,

****TILAB Turin, Italy

Abstract

The paper is focused on the INTERMON toolkit – a visual data mining system for integrated end-to-end and inter-domain QoS (Quality of Service) monitoring, analysis and modelling of application traffic - developed in the framework of INTERMON European IST project [1].The INTERMON toolkit is aimed at automated measurement and modelling of end-to-end QoS, inter-domain performance and border router traffic in an inter-domain environment for different time scales as well as visual data mining based on relating of statistics and models describing end-to-end QoS, inter-domain performance and border router traffic [20]. Tools for distributed measurement, modelling and visual data mining using relational data base are integrated in the INTERMON architecture. Concepts like ”spatial composition” of inter-domain QoS and “policy-based” performance measurement and traffic collection at border routers are considered. The paper compares the INTERMON inter-domain toolkit with current existing systems in the area of QoS analysis and verification. The key points of INTERMON architecture discussion address:

- Integration of tools for automated Internet structure analysis, monitoring, modelling and visual data mining using common data base for the purpose of QoS monitoring and verification in an inter-domain environment

- Data base design and data mining to support requirements for spatial composition of inter-domain QoS and automated producing of monitoring, modelling and analysis reports considering different aggregation intervals

- Open architecture concept with flexible import/export interfaces for measurement and modelling data (QoS, traffic).

Scenario of INTERMON toolkit usage for integrated QoS measurement, modelling and verification in an inter-domain environment based on long and short term measurements and modelling is explained.

1. Introduction

QoS measurement architectures with different objectives and technologies are developed in research activities [2], [3], [4], [5], [6], [7], [8], [9].

[12] compares different QoS measurement infrastructures with special focus on monitoring techniques (passive and active techniques). Example for the active QoS monitoring in an interconnected DiffServ inter-domain environment is the Qbone Internet2 architecture [8]. An automated event detection for active measurement systems is discussed in [13]. As a regular service to ISP’s based on active measurements and traffic generation, the RIPE NCC Test Traffic Measurement Service is proposed [5] .

Research efforts on advanced QoS monitoring architecture today are addressing the problems of information and data mining techniques to relate large amounts of measurement data for modelling, simulation and visualization

purposes. Integration of monitoring and modelling data for the purpose of macroscopic QoS data mining is discussed by CAIDA Macroscopic Internet data collection and analysis project [12]. The CAIDA macroscopic data mining uses combination of active and passive monitoring tools, such as skitter tool, based on active probing monitors on different locations to discover and measure global Internet topology [6], [14], as well as passive monitoring tools for relating the workload, [15], [16].

Considering the state-of-the-art of QoS analysis, modelling and verification in an inter-domain Internet environment, there is a challenge for QoS data mining systems which allow:

- Relating of different kinds of measurement and modelling information for inter-domain QoS provision (such as path resources, traffic and performance, application QoS and traffic)

- Automated inferring of monitoring and modelling information for different kind of QoS parameters and traffic flows based on measurement scenarios

- Complex specification of measurement scenarios in order to obtain QoS prediction models and automated producing of QoS measurement and modelling reports for different time scales

- Mapping of application traffic to QoS parameter results by using of application class models (VoIP, streaming audio and video) in order to study inter-domain admission and resource control

- Obtaining of spatial composition of inter-domain performance metrics and traffic volume

- Data mining techniques and data reduction concepts that support visual mining of large amounts of inter-domain QoS and traffic measurement data

- Complex performance analysis and problem detection based on automated inferring of passive and active measurement data obtained by different tools using time dependencies.

To meet these requirements, the INTERMON toolkit is proposed. INTERMON is an inter-domain QoS information and visual data mining system focused on “policy based” collection of different kind of QoS measurement and modeling information for the purpose of inter-domain QoS analysis, verification and traffic flow optimization [1]. Considering inter-domain topology and geographical structures, INTERMON integrates in a relational data base following kinds of measurement and modeling information:

- Application end-to-end QoS measurement data and statistical models- Inter-domain performance measurements and statistical models- Border router traffic flow monitoring data and statistical models.

INTERMON “policy based monitoring” of inter-domain QoS and Service Level Agreement (SLA) is aimed at integrated monitoring, modelling and verification in different time scales (short and long term).

For this purpose, the INTERMON data mining includes novel concepts, such as:

- Integration of tools for inter-domain Internet structure analysis, QoS monitoring, modelling and visual data mining using common data base

- Data base design for spatial composition of inter-domain QoS and automated obtaining of measurement, analysis and modelling reports considering different time scales

- Open architecture concept with flexible import/export interfaces of measurement and modeling data. The “policy based” monitoring in INTERMON is enhancing the concept of the distributed measurement architecture (DMA) developed within the AQUILA IST project [9]. AQUILA DMA is aimed at verification and evaluation of the operation of Internet QoS enabled architectures (DiffServ, MPLS and IntServ) [10], [11]. Different measurement techniques based on common data base are integrated in AQUILA DMA in order to study and verify QoS of applications and to relate them to network performance, admission control and traffic parameters:

- active network probing for study of network performance parameters- passive monitoring of application and traffic flows to verify QoS of applications.

This paper is structured in the following sections. Section 2 describes the functional components of INTERMON toolkit. Section 3 discusses the concept of INTERMON data base for obtaining of spatial composition of inter-domain performance parameters to end-to-end QoS and automated QoS analysis in different time scales. Section 4 is addressing the open architecture design with flexible import/export measurement and modelling interfaces. Section 5 is focussed on scenarios for QoS/SLA analysis and verification in an inter-domain environment.

2. INTERMON integrated toolkit for inter-domain QoS analysis

The functional components of the INTERMON toolkit are integrated based on common data base relating topological, measurement and modelling information for different kind of parameters (end-to-end QoS, inter-domain performance metrics, traffic) and Graphical User Interface (GUI). The following figure shows the functional components of the INTERMON toolkit and their interfaces to the data base and management operations (i.e. toolkit interaction control).

Figure 1: Intermon toolkit – integration of functional components

The following functional components are defined in the INTERMON architecture in order to provide measurement and modelling information for inter-domain QoS analysis of applications and border router traffic engineering:

- Inter-domain structure discovery is aimed to obtain border router level and Autonomous System (AS) level inter-domain connectivity and topology considering characteristics of ASs and border routers of inter-domain connections. The inter-domain topology is derived from a given source (AS or user end system) to specific destination(s) (one or many ASs or user end systems) by geographical and topological mapping.

- QoS monitoring . End-to-end QoS monitoring is intended to obtain QoS parameter values between end-systems based on specified end-to-end measurement scenario using active or/end passive QoS measurement techniques. Inter-domain performance monitoring is aimed at measurement of inter-domain performance metrics, such as delay and loss, based on inter-domain measurement scenario. Inter-domain performance metrics are defined between two border routers of the inter-domain path.

- QoS modelling components. INTERMON modelling functions are used to obtain statistical modelling data per measured end-to-end QoS and/or inter-domain performance parameters. Long and short term modelling data requirements are derived dependent on the measurement scenario specification (measurement duration and aggregation intervals, measurement sampling frequency). Statistical modelling data, such as accumulative distribution, autocorrelation function, prediction models, e.g. Auto Regressive Integrated Moving Average (ARIMA), are obtained per end-to-end QoS and/or inter-domain performance parameters for a given time interval. The dependencies between the metrics are quantified by cross correlation analysis. Statistical models are stored in the INTERMON data base in order to be used for simulation of end-to-end QoS of applications in an inter-domain environment. Measurement based simulation concept of INTERMON means integration of prediction models derived from INTERMON measurement scenarios into inter-domain simulation environment including specific

inter-domain and QoS simulation facilities, such as inter-domain routing protocol (BGP-4) and QoS technologies for inter-domain traffic engineering (DiffServ, MPLS).

- Border router traffic monitoring and modelling. To infer inter-domain QoS and traffic analysis, INTERMON includes border router traffic monitoring (by ISP provider) derived from the requirements for IP Flow Import/Export (IPFIX) architecture [17]. Based on the border router traffic measurements obtained for different scenarios and aggregation intervals, border router traffic flow models and traffic matrix are obtained. The border router traffic flows are measured considering different granularities (total traffic volume, traffic per source and destination end systems, traffic per source, destination and transit autonomous systems, traffic per application class and protocol). Border router traffic measurements and models are used to provide visual data mining for ISP inter-domain traffic engineering.

- Visual data mining . The objective of visual data mining concerning inter-domain QoS/SLA and traffic engineering is to “translate” the huge amount of monitored and modeled data into useful views supporting operational decisions of ISP operators and end users. Such views are aimed at topological and geographical presentation of spatial composition of inter-domain QoS (i.e. mapping of inter-domain performance parameters to end-to-end QoS), inferring of different kinds of end-to-end application QoS and inter-domain performance parameters, as well as border router traffic flow and matrix visualization for inter-domain traffic engineering (considering different flow granularities such as protocol and application classes).

The INTERMON users may request the execution of the mentioned functions for inter-domain structure analysis, monitoring, modelling and visual data mining through the GUI menus and parameters. The user interface includes management mechanisms for “policy based” data collection, interaction control, and specification of configuration and interaction parameters.

3. Inter-domain QoS data base for measurement and modelling information

The INTERMON data base defines relationships between different kinds of entities which are stored and processed with the INTERMON integrated tools for:

- Inter-domain structure discovery

- QoS monitoring (end-to-end QoS and inter-domain performance, border router traffic)

- Modelling of inter-domain and end-to-end QoS as well as border router traffic.

The “IM-user” entity is intended to identify the user of the INTERMON system. Based on the “IM-user” the rules for sharing of “export/import” inter-domain QoS and traffic information are defined. Each user of the INTERMON toolkit (ISP operator or application user) builds its “individual” INTERMON data base. Entities describing inter-domain topology, end-to-end QoS, inter-domain performance and border router traffic data are included in the INTERMON data base.

“Policy” controlled data collection defines rules for sharing of inter-domain QoS base data repositories (monitoring, modelling and topology data) between different INTERMON users. Based on this, a “global inter-domain QoS view” could be obtained combining the “individual inter-domain QoS views”. The concept of policy controlled data collection allows inter-domain performance measurements and models for QoS / SLA analysis and verification as viewed from one ISP operator or application end-user to be reused by other ISP operators or application end-users.

The design of the INTERMON data base is optimised to provide information for:

- inter-domain QoS and SLA verification, especially to obtain spatial composition of inter-domain performance to the end-to-end QoS in an inter-domain environment

- inter-domain traffic engineering, focussing on border router traffic flow measurements and modelling considering different flow granularities.

The spatial composition of inter-domain performance (QoS) is base for QoS/SLA verification of applications in an inter-domain environment and focus of this paper. It is defined by the mapping of the inter-domain performance measurement and modelling data to the end-to-end QoS measurement and modelling data considering topological information. Obtaining of spatial composition is supported by definition of entities and relationships describing the inter-domain topology of the end-to-end connections, i.e. inter-domain connection entities are linked to inter-domain paths describing the specific end-to-end connection.

The integrated measurement and modelling concept is based on relating of the modelling entities (derived per end-to-end QoS parameter, inter-domain performance metric and border router traffic flow) to the corresponding measurement statistics and result entities (describing aggregated statistics results) of the specific measurement scenario for a given time aggregate.

Entity groups are differentiated in order to explain the complex data base design:

- Topological structure entities describing inter-domain usage and topology- Border router traffic measurement and modelling entities focussed on border router traffic measurement

scenarios and flow models- Inter-domain performance measurement and modelling entities addressing measurement scenarios and

models of inter-domain connection- End-to-end QoS measurement and modelling entities defining end-to-end measurement scenarios and

models belonging to specific end-to-end connection and corresponding inter-domain path. Each end-to-end connection could have different inter-domain paths.

- Traffic description entities used to specify the generated traffic for the inter-domain and end-to-end measurement scenarios.

Using the “entity group” approach, the INTERMON data base is divided into data base “parts”. The following figure illustrates the logical design of the INTERMON data base including entities, groups of entities and their relationships.

Figure 2: INTERMON logical data base model - entity groups and relationships

The INTERMON data base stores the results and the measurement statistics for the measurement scenarios in a time dependent manner dependent on the specified “measurement aggregation interval”. The measurement scenario is the main entity, based on which measured parameter, statistics and corresponding statistical models are related. Dependent on the kind of measurement (end-to-end QoS, inter-domain performance parameter, border router traffic), there are different types of measurement scenarios. The results of measurement scenarios are inferred based on time parameters such as measurement aggregation intervals. On this way, spatial

composition of inter-domain performance metrics or border router traffic can be related to the end-to-end QoS of applications.In the next figure, the end-to-end measurement scenario data base part (entities and relationships) are illustrated:

Figure 3: End-to-end QoS parameter measurement and modelling entities with relationships

Each measurement scenario is described with measurement duration, measurement aggregation interval for the results and statistics type, as well as the measurement frequency. Considering these parameters of the measurement scenario specification, data mining functions for automated measurement and modelling reports for different aggregation intervals are defined to interpret the measurement and modelling data (per hour , day, week, month, year).

The modelling entities, such as accumulative distribution, autocorrelation function, and Auto Regressive Integrated Moving Average (ARIMA) prediction models, are linked to the measurement results and statistics for a given measurement aggregation interval. INTERMON data mining functions are specified in order to obtain automated generation of modelling reports for a different kind of aggregation intervals (e.g. short term : inter-domain routing, long-term : network planning ).

The relationships between measurement results, statistics and models belonging to different measurement scenarios are derived from their “time states”, which are calculated from parameters describing measurement scenarios and their results such as measurement aggregation interval, duration, and measurement frequency.

4. INTERMON open and flexible policy based data collection

The INTERMON data base includes measurement results and statistics obtained from the policy controlled data collection. The INTERMON monitoring tools use remote meters and adapters for execution of measurement/monitoring scenarios specified with the monitoring tools. The remote meters are configured by the monitoring tools to filter their results and to parameterize adapters for the specified measurement scenarios in order to interact with the INTERMON data base.The policy based data collection in INTERMON is open for interoperation with other QoS architectures and flexible to integrate different kind of measurements and statistics into the relational data base. INTERMON open architecture design is intended to support import/ export measurement and modelling interfaces between different INTERMON users and towards other QoS monitoring and modelling systems.

The policy based collection of measured data in INTERMON uses meters belonging to the INTERMON QoS monitoring components, which obtain measurement results and statistics at different collection points: border routers, end-systems and measurement points selected for inter-domain performance metric collection. While the structure discovery and modelling toolkits are designed to operate directly with the INTERMON data base, the monitoring toolkits could use so called adapters for flexible filtering of the measured data and integrating it to the INTERMON data base.Adapters allows different interfaces to the INTERMON data base entities considering the requirements for inter-domain data mining. Adapters could be also used for integration of different kind of external measurement and modelling data into the INTERMON data base. Examples for adapters used for integration of external measurement and modelling data into the INTERMON data base are:

- Traffic trace filter transforming ISP border router traffic traces into INTERMON data base entities (i.e. border router traffic flow measurement scenarios and results)

- Translator of external QoS measurement data repositories to INTERMON data base entities (i.e. measurement scenarios and their corresponding results). For instance, such adapter could be written to transform AQUILA data base measurement data to INTERMON measurement data presentation.

Adapters as well as remote meter are configured by the corresponding tools integrated in the INTERMON GUI considering the measurement scenarios.The adapter concept allows to reuse the great amount of QoS monitoring data obtained by other QoS monitoring architectures developed in European projects [10], [18], [19].The following figure presents the open and flexible approach of INTERMON data collection architecture:

Figure 4: Open and flexible architecture design with adapters for filtering of source measurement data and export/import interfaces

5. INTERMON toolkit scenario for QoS/SLA verification of applications in an inter-domain environment

There are different benefits in using the INTERMON toolkit for QoS monitoring, modelling and forecasting as well as SLA planning and verification in an inter-domain environment:

- The QoS/SLA verification could be studied automatically for different aggregation intervals (i.e. INTERMON automated measurement reports per hour, day, week, month, year) to provide insight in the impact of different factors on the QoS perceived by an application in different time scales.

- Based on spatial composition of inter-domain performance and traffic to end-to-end QoS, the inter-domain connection and the border router(s) of the inter-domain path, could be identified, which are responsible for the degradation of end-to-end QoS of applications.

- Integrated measurement and modelling provides the possibility to predict the end-to-end and inter-domain QoS, to specify and plan the application QoS and SLA for different inter-domain routes. Especially in case of QoS degradation, there is a need to find an alternative inter-domain path for the provision of end-to-end services. For this purpose, the monitoring of application QoS for different aggregation intervals is complemented with measurement based short and long term QoS modelling for the specific inter-domain route.

- The usage of INTERMON emulated traffic generation allows to obtain QoS measurements for different load conditions (i.e. multiplexing of application traffic flows) and to plan the SLA in the inter-domain environment dependent on the border router traffic loads.

- Predictive long and short term models of inter-domain parameters derived from the monitoring data in the INTERMON data base allow to take decisions on application traffic mapping to inter-domain routing paths, and border router resources.

A simple scenario is used to illustrate the INTERMON QoS/SLA analysis and verification concept for the Voice over IP traffic flows considering different inter-domain paths for the end-to-end connection in an inter-domain environment. A multi homed transit domain (domain C) for VoIP traffic based on multiple border egress nodes leading to the destination AS of the VoIP traffic is assumed. The Voice over IP QoS/SLA inter-domain scenario is given in the following figure:

Figure 5: Scenario for inter-domain QoS and SLA verification of VoIP application

The following diagram shows the usage of INTERMON functional components for the inter-domain SLA analysis and verification:

Figure 6 Usage of INTERMON components for inter-domain QoS and SLA verification for VoIP application traffic

The scenario designed to support “optimal” inter-domain QoS / SLA analysis and verification could have more variants if inter-domain QoS interfaces (DiffServ, MPLS) are considered which allow different traffic mapping strategies at the egress border router.

6. Conclusions

The integrated inter-domain QoS monitoring, modelling and simulation concept developed in the framework of the INTERMON project, was presented. The focus was the novel concept of inter-domain data mining in INTERMON and the data base design for integrated measurement and modelling tools. The use of the INTERMON data base for the spatial composition of inter-domain performance to the end-to-end QoS was discussed and scenario for inter-domain QoS/SLA verification based on INTERMON toolkit was proposed. Another main usage of INTERMON toolkit is the inter-domain traffic engineering [20] based on border router measurement and modelling using IPFIX [17]. The border router traffic measurement and modelling as well as visual data mining for inter-domain ISP traffic engineering in the framework of INTERMON project is topic of further work.

7. References

[1] Advanced architecture for INTER-domain quality of service MONitoring, modelling and visualisation, INTERMON-IST-2001-3412 http://www.ist-intermon.org

[2] Co-operative Association for Internet Data Analysis (CAIDA) http://www.caida.org

[3] Surveyor: An Infrastructure for Internet Performance Measurements http://bsdi4.advanced.org/~kalidindi/papers/INET/inet99.html

[4] B. Cheswick. Internet mapping project. http://www.cs.belllabs.com/who/ches/map/

[5] RIPE Test Traffic project http://www.ripe.net/ripencc/mem-services/ttm

[6] The Skitter project. http://www.caida.org/Tools/Skitter.

[7] T. Hansen, J. Otero, T. McGregor, H.-W. Braun. Active Measurement Data Analysis Techniques http://moat.nlanr.net/Papers/AMP_case_studies/

[8] Internet 2 Qbone http://www.internet2.edu/qos

[9] F Strohmeier, H. Dörken, B. Hechenleitner. AQUILA Distributed QoS Measurement. In Proc. of COMOCON8 Conference, Crete, Greece, pages 177-185, June 2001.

[10] AQUILA project page http://www.ist-aquila.org

[11] I. Miloucheva, U. Hofmann, F. Strohmeier, T. Pfeiffenberger, Integrated toolkit for performance analysis in intra- and inter-domain environment, The 10th International Conference on Telecommunication Systems, Modeling and Analysis Monterey, CA, USA, October 3-6, 2002

[12] M. Murray, KC Claffy, Measuring the Immeasurable: Global Internet Measurement Infrastructure, Proceedings PAM 2001, p. 159-167, Amsterdam, April 2001

[13] A.J. McGregor, H.-W. Braun, Automated Event Detection for Active Measurement Systems, Proceedings PAM 2001, Amsterdam, April 2001

[14] CAIDA skitter monitor locations. http://www.caida.org/tools/measurement/skitter/monitors.xml.

[15] http://www.caida.org/tools/measurement/coralreef/)

[16] http://www.caida.org/tools/measurement/cflowd/).

[17] Requirements for IP Flow Export, J. Quittek, T. Zseby, B. Claise, G. Carle, S. Zander, K.C. Norseth, August 2002 (work in progress), Internet-Draft http://www.ietf.org/internet-drafts/draft-quittek-ipfx-req-05.txt

[18] The CADENUS Project (1999) Creation and Deployment of End-User Services in Premium IP Networks, http://www.cadenus.org/ (March 10, 2002)

[19] Traffic Engineering for Quality of Service in the Internet, at Large Scale http://www.ist-tequila.org

[20] A. Nassri (ed.) Requirements and Analysis Report, INTERMON Deliverable 1, July 2002, http://www.ist-intermon.org/