contributions to automated testing of network service interactions

Automated Testing of Network Service Interactions

Contributions to Automated Testing of Network Service

Interactions

Rudy Deca1 ,Omar Mahrez1, Omar Cherkaoui1, Yvon Savaria2 and

Doug Slone3

Notere 2005, Ottawa/Gatineau, Canada

1University of Québec at Montréal, 2Ecole Polytechnique de Montréal, 3Cisco Systems, Inc.


R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone

2

Contents

The problem context The Meta-CLI Model The ScriptMaker tool Conclusion

3

The Problem context

Connectionless statelessnetworks

Connection-oriented

circuit-switching stateful network services

Hybrid, contradictory

network service characteristics

Network convergence

NewSolutionsrequired

More services more features

more parameters

More combinationsmore interactions

more configurationsmore testsrequired

More environment

states

Increase in the number of states

Causes: Effects:



4

Contents

The problem context The Meta-CLI Model

General presentation Structure• Operations

Abstracting the environment The ScriptMaker tool Conclusion



5

The Meta-CLI Model solution

Abstracts: the network service and device configuration

states the environment states the dynamic changes of the connectionless,

stateless IP protocols Benefits:

Automated adaptation to environment changes Efficient generation and modification of

configurations, test scripts and tests for service interactions.

6

Modelling a service configuration as a conceptual tree structure

Arcs: Hierarchical

relationships Inclusion relationships

Nodes: Configuration command modes Configuration commands Configuration parameters

Meta-CLI modelling

Created

Modified

device_1

service_1

service_1

Command mode

M

Command P

Command Q

Parameter R

Parameter S

Command mode

NConfiguration

commands and parameters



7

Contents


General presentation • Structure Operations

Configuration Validation

Abstracting the environment The ScriptMaker tool Conclusion



8

Meta-CLI Model Operations Operation types:

Configuration Validation

Distribution: Single device Multiple devices of a network

Operation attributes: Atomicity Order (causal, sequential) Granularity (varying from parameters to

services)


9

Configuring services using the Meta-CLI Model The main idea of the Meta-CLI configuration: compose thedevice and network service configuration models.

Deviceconfiguration

record

Networkservice

Conversion

Co

nve

rsio

n

Meta-CLI Model

Capsule

Construct

Co

mp

ositio

n

Configuration Construct + RetroversionDevice

configurationrecord +



10

Generic service configuration operations

Instantiate service configurations on devices. Instantiate service configuration parameter values. Instantiate service configuration constraints. Adapt service configuration to device and interface

environment. Adapt service configuration to device and interface

role. Determine environment support for various feature

combinations Determine compatibility of services and features. Determine environment support for service and

feature combinations.

11

Examples of validation dependencies

Validation dependencies between: configuration commands: T depends on Q configuration parameters: S and U depend on R

Dep.U-R

device_1

Command P

Value1

Parameter R

Value3

Parameter U

Value4

Command Q

Value2

device_2

Command P

Value5

Parameter S

Value7

Parameter V

Value8

Command T

Value6

Dependency S-R

Dependency T-Q



12

Network service and feature interactions

Mutual exclusion, due to: Competition for resources Accessing the same parameters/variables Incompatible goals

Influence, due to : Scale factor Performance factor

Service combination constraints: Common supporting environment Existence of mandatory parameters



13

Contents


• General presentation Abstracting the environment

The ScriptMaker tool Conclusion



14

Environment heterogeneity solution

The configuration environment: network-level component

network topology, technology, Protocol, Role.

equipment-level component. hardware components

device type, interface type, medium type, etc.

software components operating system, software version, etc.

15

Adapt CLI commands to different environments

Command P is instantiated on device_1with one parameter (R) and on device_2 with two parameters (R,S).

Command P

Parameter R

Parameter S

service_1

env1, env2

Command P

env1, env2

Paramet. R

env1, env2

Paramet. S

env2

device_1 + service_1device_2 + service_1

device_1

env1

Command P

Parameter R

device_2

env2

16

Adapt CLI commands to different device and interface roles

Command P is instantiated: on device_1 with parameter R and on device_2 with parameter S, based on the different roles assumed

by the two devices.

Command P

Parameter S

service_1

role1, role2

Command P

role1, role2

Paramet. R

role1

Paramet. S

role2

device_1 + service_1

device_1

role1

Command P

Parameter R

device_2

role2

device_2 + service_1

17

Addition of a network service to a device configuration

The generic network service model is accessed from the repository.

A service instance model is generated and adapted to the given environment

and then composed with the device configuration model.

Modification of a network service in a device configuration

The network service is already configured on an equipment when an environment change occurs.

The network service model instance in the old environment is extracted from the device configuration model.

The generic network service model is accessed from the repository.

A new instance model is generated for the new environment and composed with the device configuration model.



19

Determining environment support for various feature combinations

The information is obtained by querying the tree model.

Service features

service_1 P P P P

Feature R NP P NP P

Feature S NP NP P P

Environment instances

env1 S S NS NS

env2 S S S S

service_1

env1, env2

Command P

env1, env2

Feature R

env1, env2

Feature S

env2 P = is present

NP = is not present

S = supports

NS = does not support



20

Contents

The problem context The Meta-CLI Model The ScriptMaker tool

Functionality• Architecture• Generation scenario• Conversion scenario

Conclusion



21

Functionality of the ScriptMaker tool

Aimed to regression testing. Provides an effective, automated and easy way to

generate/modify testing scripts with test cases featuring various combinations of network services and features.

Solves the problem of the existence of a large number of : combinations among existing network services; features and parameters of the services; environment states in which the service parameters are

configured differently. Uses a modular approach for components:

Service configurations Traffic Routing, topology and connection among router interfaces Configurations validation



22

Contents


• Functionality Architecture• Generation scenario• Conversion scenario

Conclusion



23

Architecture of the ScriptMaker tool



24

Contents


• Functionality• Architecture Generation scenario• Conversion scenario

Conclusion


25

The script generation scenario

Specific CLI servicecode

Genericservice

capsule stubs

Envi-Ron-ment

Selected Specific TC, TS,

Templates

Selected Generic TC, TS,

Templates

Signaling

OK

Environment (descriptor)

Feature name(s)

Script

Environment (descr.)

Feature name(s)

OK

Script

Script Manager Block

Test SelectionBlock

Test Filling Block Test Binding Block

Meta-CLI Model Block



26

The test selection block

Selected TC, TS,

Template names

Featurename(s)

Env.desc.

Envi-ron-

ment

Feature name(s) Constr

Constraints

Selected TC, TS, Templates

TC, TS, Template

names

Feature name(s)

Test Selector

Test Organizer

Spec Archive

Test Bench

Environment Constraint Database



27

The test filling block

Selected Spec CLI auxiliary, validation commands and TCL Traffic, Topology

commands

Test Filler

Converter

InstantiatorCode Generator

Selected Gen CLI auxiliary, validation commands and TCL

Traffic, Topology commands

Selected generic stubs Kx, Kv, Trf, Top

Selected Gen stubs Kx, Kv, Trf, Top

Selected Gen CLI auxiliary, validation commands and TCL

Traffic, Topology commands

Environment

Selected Spec CLI auxiliary, validation commands and TCL Traffic, Topology commands

Ks = Service capsuleKv = Validation capsuleKx = Auxiliary capsule



28

The Meta-CLI Model block

Meta-CLI Model manager

Service Instantiator

Service Retriever

Selected genericservice capsule

Selected generic service stubs Ks

Selected generic service capsule Environment descriptor

Selected specific

CLI service

Capsules (models)

Selected generic service capsule

Selected generic

service stubs



29

Contents


• Functionality• Architecture• Generation scenario Conversion scenario

Conclusion


30

Envi-Ron-ment

The script conversion scenario

selected servicesauxiliariesvalidationstopologies

traffics

Generic TC, TS, Template

servicesauxiliariesvalidationstopologies

traffics

service

Filled Selected Specific TC, TS,

Template

Selected Generic TC, TS,

Template

Signaling

Old script

Environment

Feature name(s)

Script

Environment

Feature name(s)

Script

New script

Script Manager Block

Test Selection

Block

Test Filling Block

Test Binding Block

Meta-CLI Model Block

ParsingBlock

Specific CLI servicecode



31

Conclusion Problem:

New Internet services, hybrid, with numerous: parameters features interactions heterogeneous environments.

FSMs not always useful. Solutions:

The Meta-CLI Model: Generic service configuration Environment-aware

The ScriptMaker tool: Automated generation/modification of testing scripts; Modular design; Uses the Meta-CLI for abstracting and reusing service

configurations.



32

Questions

contributions to automated testing of network service interactions

Documents

metacli configuration

device configuration

environment support

interface environment

sloneconfiguring services

qconfiguration parameters

omar mahrez1

omar cherkaoui1