ttcn-3 based implementation of etsi tispan ims benchmark george din, diana vega, razvan petre,...

TTCN-3 based Implementation of ETSI TISPAN IMS Benchmark

George Din, Diana Vega, Razvan Petre, Andreas Hoffman

Fraunhofer Institute for Open Communication Systems

TTCN-3 User Conference, 30.05.2007 2

Contents

Motivation

What do we test?

How do we test?

Implementation details

Benchmark Results

Conclusions


Motivation

Goal - performance benchmark for IMS components Performance and scalability testing of all IMS and related components with

simulated real-world traffic Measurement and analysis of important QoS parameters

Why Creation of objective means to compare overall IMS of different systems by

performance (and price) Check ability of hardware/software to run the IMS

How Define standard scenarios and traffic models for the work load Define the metrics to be measured Standardize the test procedure, the test parameters and the Benchmark test

report

Where Standardization of IMS benchmarking at ETSI TISPAN WG6 Version 1.0 of IMS benchmarking standard has been released.


System under Test

ChargingFunctions

IP Transfer (Access and Core)

MRFP

BGCF

MRFC

SLF

MGCF

Oth

er IP

Ne

two

rks

Resource and Admission Control Subsystem

IWF

I-BGF

IBCF

AS

PS

TN

/ ISD

N

SGF

T-MGF

NetworkAttachementSubsystem

Gm

Gq’

Mp Mn

MxMg

Mj

Gq’

Dh

Dx

Rf / Ro

Rf/Ro

Mw

Mw

Mx

Mx

Sh

Cx

Ut

ISC/Ma Iw

Ic

Ie

Ib

MrMk

Mle2 I/S-CSCF

UE

UPSF

IMSCore

FokusOpen IMS

UE

P-CSCF


Test Report

Benchmark Information Model

Use case 1 (e.g.: registration)

Scenario 1.1 (e.g. user not registered)

Message flow

Metrics & graphs to collect

SuT parameters

Design objectives

Scenario 1.2 (e.g. already registered)

Benchmark Test

Traffic Set

Preamble RegistrationPhase

Su

TC

hara

cte

risti

cs

Traffic Profile

Initial load

Step duration

Step Increase Amount

Number Of Steps

Other parameters

Use case 2 (e.g.: session setup)

Scenario 2.1 (e.g. successful call)

Message flow

Metrics & graphs to collect

SuT parameters

Design objectives

Scenario 2.2 (e.g. rejected call)

Backg

rou

nd

Load

X% of scenario 1.1

y% of scenario 2.1

z% of scenario 2.2

Disclaimer

Benchmark Test Run

SuT configuration and parameters

Metrics and graphics with Design Objective Capacity (overall and for each scenario)

Test System configuration and parameters

Observations and interpretations

Exceptions

Metrics and graphics with SuT resource consumption


Scenario: Successful Call

INVITE

100 TryingINVITE

180 Ringing

200 OK

180 Ringing

200 OK

ACKACK

BYEBYE

200 OK200 OK

REGISTER

401

REGISTER

200 OK

Registration Section Tear Up Tear Down

Client 1 IMS CN

REGISTER

401

REGISTER

200 OK

Client 2 IMS CN

Client1

Client2CN

Talking Time

Ringing Time


Scenario: Successful Call

INVITE

100 TryingINVITE

180 Ringing

200 OK

180 Ringing

200 OK

ACKACK

BYEBYE

200 OK200 OK

Section Tear Up Tear DownClient1 Client2CN

TRT-SES1

TRT-REL1

TRT-SES2

Talking Time

Ringing Time


Estimated Maximum Load

NoLoad

InitialLoad

InitialCallLoad

StopConditionMet

Time ToStopCondition

PlusLoad

PlusLoad

PlusLoad

LoadRamp

LoadSteady

PlusLoad

StabilityOr SettlingTime

Linear, Immediate?

Linear, Immediate?

Stability?Stability?

RateIncrease

Fluctuations,Steady state?

Fluctuations,Steady state?

Benchmark Procedure


TTCN-3 ATS Architecture

Scenario-Independent

Scenario-dependant

UserState

Machine

UserState

Machine

Imports from

Benchmark

Testcase

Benchmark

Testcase

Module for Instant Messaging

Scenarios

DataTemplates

Module for Registration

Scenarios

DataTemplates

Module forCalling

Scenarios

DataTemplates

CommonTypes

CommonBehavior

CommonTemplates

BenchmarkConstants

BenchmarkParameters


TestAdapter

usrN+1

ComponentHandler 2

usrN+2 usr2N

Users Database

usr1 usr2 usrN

ComponentHandler 1

Users Database

Test Configuration

SuT

ComponentSender

User 1 stack User 2 stack User N stack

Load Generator


User Concept

1. A user is a state machine running a scenario

2. A user may be „callee“ or/and „caller“

3. A user may create > 1 calls

4. A user may be reused to create other calls

5. A user may call randomly any other user


Simultaneous Scenarios

calling

calling mes

sag

ing

mes

sag

ing

calling

calling calli

ng

calli

ng

Same user involved in more than another

instance of same scenario

Same user involved in different scenario


Population Clusters

all other scenarios like messaging, calling,

reregistration

b c

registration(not registerred yet)

a

deregistration(registerred and not involved in any other

scenario)


TTCN-3 Message Types

// general type for a SIP Request // can be extended by any// other request type (i.e. INVITE_Request)type record Request { integer transactionId, RequestLine requestLine, MessageHeader msgHeader, charstring messageBody optional}

type set MessageHeader { Authorization authorization optional, CallId callId optional, Contact contact optional, CSeq cSeq optional, Expires expires optional, From fromField optional, RecordRoute recordRoute optional, Route route optional, ServiceRoute serviceroute optional, To toField optional, Via via optional, MaxForwards maxForwards optional, ContentLength contentLength optional, WwwAuthenticate wwwAuthenticate optional, Event event optional}

// [20.7 RFC2617 3.2.2]type record Authorization { FieldName fieldName(AUTHORIZATION_E), //Credentials body charstring body optional}


TTCN-3 State Handling

alt { // P-CSCF --------REQUEST--------> User [] p2SUT.receive (TRequest) -> value request {

// P-CSCF --------INVITE--------> Userif (match(request, Request_INVITE_r)) {}// P-CSCF --------MESSAGE-------> User else if (match(request, Request_MESSAGE_r)) {}// P-CSCF --------ACK--------> Userelse if (match(request, Request_ACK_r)) {}……………….

} // P-CSCF --------RESPONSE--------> User [] p2SUT.receive (TResponse) -> value response {

// P-CSCF -------- 200 OK --------> Userif (match(response, Response_200_r)) { if (response.msgHeader.cSeq.seqMethod == "INVITE") {} else if (response.msgHeader.cSeq.seqMethod == "BYE") {} ………………. else if (response.msgHeader.cSeq.seqMethod == "REGISTER") {}}// P-CSCF -------- 401 OK --------> Userelse if (match(response, Response_401_r)) {}……………….

}}


Messages Creation

// P-CSCF --------RESPONSE--------> User[] p2SUT.receive (TResponse) -> value response { // P-CSCF -------- 200 OK --------> User if (match(response, Response_200_r)) {

if (response.msgHeader.cSeq.seqMethod == "INVITE") {

ackReq := ACK_Request_s;ackReq.transactionId := getNewTransactionId(); ackReq.requestLine.method := ACK_E;

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ackReq.msgHeader.callId := response.msgHeader.callId;ackReq.msgHeader.cSeq := response.msgHeader.cSeq;ackReq.msgHeader.cSeq.seqMethod := "ACK";

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ackReq.msgHeader.fromField := response.msgHeader.fromField;ackReq.msgHeader.toField := response.msgHeader.toField;ackReq.msgHeader.via := response.msgHeader.via;ackReq.msgHeader.route := getServiceRoute(response);

userChannel := getChannel(response);p2SUT.send (ackReq) to userChannel;delTransactionId(response.transactionId);


Processing of Messages

messageprocessor

RESP

REQ

identify

updatestate

validatetransaction

state

process response

SUT

test component

Platform adaptor


Adaptation Layer

Aim: Maximize TTCN-3 system performance !

Handles multiple parallel connections with SUT

Uses Pools of Threads and Listeners

Smaller number of threads and context switches mutex-es and semaphores contended message queues

Test Management (TM)

ComponentHandling

(CH)

CodingDecoding

(CD)

TTCN-3Executable

(TE)

System Adapter (SA) Platform Adapter (PA)

TTCN-3Executable

(TE)

System under Test

Test System

System Adapter (SA)


Adapter

Transactions

Stack

ListeningPoint

SipProvider

SipListener

SIPMessages

SIPEvents

Stack

ListeningPoint

SipProvider

SipListener

SIPMessages

SIPEvents

SUT

PTC PTC


Test Adapter Design

Test System (TS)

6: handlemessage

TE

SA

3: read request

Message Handlers Thread Pool

9: write ready

Initiation Dispatcher (Reactor)

2: read ready

1: request message

System under Test(SUT)

4: messageready

5: notify TE(put message in the appropriate port queue)

7: response message

10: write message

8: send responsemessage

• I/O operations are

made completely

separately by the

processing of the message.

• Concurrent I/O are possible on a SMP system.

Advantages:


Overall Traffic


CPU Monitoring on SuT


Distribution of SAPS


Average Delay for Invite Requests


Conclusions

TTCN-3 specification has a big impact on the performance of the hole test system Using a component to simulate more than one user Using a component sender Using a tree like mechanism for matching the

messages

Efficient Adaptation Layer Design has an important role in the overall performance Smart codec/decoder system Non-blocking I/O for protocol stack Optimized runtime for specific needs of the

benchmark


Thank you !

Time for questions …

ttcn-3 based implementation of etsi tispan ims benchmark george din, diana vega, razvan petre,...

Documents

fromfield optional

tofield optional

callid callid optional

route route optional

contact contact optional

cseq cseq optional

callsa user

scenarioa user