ixchariot training

IxChariot

Course # 985-0300

Copyright 2009

Introduction to IxChariot

©2007 Ixia. All rights reserved.

Console

Endpoint 1

Endpoint 2

What is IxChariot?

IxChariot is a software-based network performance testing and

measurement tool

Tests data, video, voice, broadband and wireless transport networks

Emulates application traffic and measures end-to-end performance and

quality

Used to…

Emulate applications to ensure the integrity, performance and reliability of a

network before application deployment

Benchmark network performance and troubleshoot connectivity issues

Test functionality of network equipment under varying traffic profiles


Application Testers (e.g. AppTest)

Application-level Network Testers (IxChariot)

Packet Generators (e.g., IxExplorer)

Application

Presentation

Session

Transport

Network

Data Link

Physical

What Layers are Tested by IxChariot?

{ {


Qualifying hardware and software

Profiling a new application to predict its impact

Benchmarking to establish Service-Level

Agreements

Verifying the network after changes are made

Voice over IP assessment testing

Multicast interoperability testing

Triple-Play testing with VoIP, Video, and Data

traffic

Wireless network testing

How is IxChariot used?


How is IxChariot Used in Wireless?

Tests wireless roaming performance

IxChariot is part of the official test plans for:

– Wi-Fi 802.11a/WPA™ Test Plan

– Wi-Fi 802.11b/WPA Test Plan

– Wi-Fi 802.11g/WPA Test Plan

– Wi-Fi PDA Test Plan

– Wi-Fi WMM™ Test Plan

– Wi-Fi WPA2 Test Plan

Using IxChariot


Test Instructions

Application Scripts

TCP, UDP, RTP

IPX, SPX

IPv4, IPv6

VoIP, Multicast

TCP, UDP, RTP

IPX, SPX

IPv4, IPv6

VoIP, Multicast

Test Results

IxChariot’s Basic Design

Results Reporting

• .html

• .csv

• .txt


Performance Endpoints

Installed as a service or daemon

Execute scripts

Measure performance

Return observed data

30+ Operating systems supported

Including Windows (including Win CE), Solaris, Linux, AIX, MVS,

OS/2, NetWare, Tru64 UNIX, FreeBSD, SCO UnixWare, IRIX


Lab 1 Application Rollout

10

Lab Objectives

Students who complete this lab will be able to:

1. Create a Basic IxChariot test.

2. Understand what StackManger is and what it is used for.

3. Replicate a pair and run a multi-pair IxChariot test.

4. Edit an IxChariot Script.

5. Begin to Interpret IxChariot test results.


From Address

To Address

Protocol

Script

Modify Script

Service Quality

Defining the Endpoint Pair


Selecting Script for Endpoint Pair


Types of Application Scripts

IPX, RTP, or UDP

protocols only

No acknowledgments or

separate connections

made

Bader Benchmark scripts

Internet scripts

Business scripts


Application Script

Run on Endpoint Pairs to emulate application traffic


Choosing Run Options

Ending a Test Run

Run Until. . .

Reporting Timings

Real-Time or Batch Reporting

Polling Endpoints

Whether and how often


Real-Time versus Batch Reporting

Real-Time Reporting

Lets you see results as they happen but

Test data flows may skew the test results

Batch Reporting

Minimized Console traffic doesn’t skew the results but

You must poll to monitor a currently-running test


Lab 2 Voice over IP (VoIP)

17


Lab 2 Voice over IP (VoIP)

Lab Objectives


1. Create a basic IxChariot VoIP test

2. Learn what MOS means and how IxChariot calculates MOS

3. Generate a full duplex VoIP call

4. Analyze call quality using IxChariot VoIP metrics

VoIP Testing with IxChariot


How does IxChariot test VoIP?

IxChariot tests the network’s ability to handle VoIP traffic

Real Time Protocol

Voice Quality

Measure E-model and R value

VoIP Standards


VoIP Overview

Real Time Protocol

Voice Quality

E-model and R value

VoIP Standards


Voice Quality

Voice Quality is a complex issue: It depends upon the Codec used

It depends upon any QoS settings

It depends upon consistency of reception

It depends upon lost data

It depends upon throughput

It has several possible scoring methods

Scoring methods: MOS (ITU P.800)

Mean Opinion Score

PSQM (ITU P.861) /PSQM+

Perceptual Speech Quality Measure

MNB (ITU P.861)

Measuring Normalized Blocks

PESQ (ITU P.862)

Perceptual Evaluation of Speech Quality

PAMS (British Telecom)

Perceptual Analysis Measurement System

The E-model (ITU G.107)


Why is Quality so Important?

Voice traffic is small compared to data traffic.

A voice conversation is a two-way exchange, but

voice traffic is two one-way unicasts.

The traffic is time sensitive – delays can make the

conversation walkie-talkie like

Lost data does not get retransmitted because it is

streaming media

Consecutive lost data can create large gaps in the

conversation


Voice Quality Measurements

PSQM (Perceptual Speech Quality Measurement)

PSQM+

PESQ (Perceptual Evaluation of Speech Quality)

MNB (Measuring Normalized Blocks)

PAMS (Perceptual Analysis Measurement System)

British Telecom variation, requires licensing fee

MOS (Mean Opinion Score)

Subjective – based upon representative samples


E-model and R value

E model – ITU G.107

R value maps to a MOS score


IxChariot’s Role in VoIP Testing


Using the VoIP Features of IxChariot


Special VoIP Pair Settings

Special Features for VoIP

Five different codec types, emulating Different compression algorithms

Data rates

Datagram sizes

Flexibility in changing datagram sizes

Silence suppression & voice activity

rate

Configurable jitter buffer

Quality of service

Performance and quality metrics

Scoring methods: Jitter

lost data

consecutive lost datagrams

one-way delay

Mean Opinion Score


Lab 3 Video and Multicast

29


Lab 3 Video and Multicast

Lab Objectives


1. Use IxChariot to emulate real video traffic

2. Understand IxChariot’s video metrics

3. Analyze and trouble-shoot video quality issues

Video and Multicast testing with IxChariot


IP Video testing with IxChariot

Tests the network’s ability to deliver IP Video Quality of

Experience (QOE)

IP Video Metric Ideal

Established Video Metrics

Media Delivery Index (Delay Factor / Media Loss Rate)


Evaluate Network Performance with Datagrams

Datagram behavior can indicate the following

disorders:

Retransmissions (Duplicates Sent)

Network Loops (Duplicate packets received, packets arriving out of

order)

Lost packets vs. packets received late in either direction

Multiple paths (packets arriving out of order)

Frame size sensitivity


RTP: Real Time Transport Protocol

Used by many leading voice and video applications

Provides end-to-end transport functions suitable for

applications transmitting real-time data

Independent of the transport and network layers

Documented in RFC 1889


B

A

C

B C

D1 D2=D1 D3¹D2

E1 sends

E2 receives

Network

E1 E2

A

Jitter


Jitter


Test multicast interoperability among different devices

Datagrams

Sending/Receiving IP Multicast Traffic

Setting Up Multicast Groups

Evaluating Network Performance with Datagrams

Tuning Datagram Performance

Multicast testing with IxChariot


Datagrams

Sending/Receiving IP Multicast Traffic

Setting Up Multicast Groups

Evaluating Network Performance with Datagrams

Tuning Datagram Performance

Multicast and Datagram Statistics


10.20.134.7

10.20.134.3

10.20.134.4

10.20.134.5

10.20.134.6

10.20.134.8

10.20.134.9

10.20.134.10

10.20.134.11

10.20.134.12

10.1.14.254

Traditional Traffic Flows


10.1.14.254 225.1.1.1

Applications that Use IP Multicast


Multicast Routing

Protocol

Eth

ern

et

Eth

ern

et

Eth

ern

et

Router

Router

Router

Router

Router

Router

Tower

box

Eth

ern

et

Receiving IP Multicast Traffic


Multicast

Sender

Multicast

Receivers

X Single

Data Send

data propagated

to receivers

Multicast

Routers

Eth

ern

et

Eth

ern

et

Eth

ern

et

Eth

ern

et

Router

Router

Router

Router

Router

Router

Tower

box

replicated

data send

data pruned where no receivers

Sending IP Multicast Traffic


Performance Considerations

Is my network ready? Raw capacity

Upgrades

Configuration

How much IP Multicast traffic? Receiver’s quality

• Send data rate

• Buffering

• Lost data rate

• Other network traffic

How many simultaneous senders?

What happens during congestion? TCP applications vs. UDP applications

Does QoS or bandwidth management help?


Lab 4 Triple Play

44


Lab Objectives


1. Comprehensively assess the performance of the Triple Play

network

2. Interpret Voice, Video, and Date metrics

3. Generate HTML reports

Lab 4 Triple Play


Designing a Good Test

The Ideal Test:

Runs long enough for small relative precision

Avoids timing records shorter than 10ms.

Doesn’t generate too many timing records

But, generates enough records to show fluctuations


Console

E1 E2 Firewall

Admissible port =

explicit port number in script

Testing through a Firewall


Console

E1 E2

Firewall

• For setup: 10115 – default but now

configurable

• For routing timing records --

User configurable

(Change User Settings notebook)

Test Data through a Firewall


Test Results

Interpreting Test Results

Timing Records

Measured Time

Confidence intervals

Relative precision

Three primary statistics

Throughput (really “Goodput” – RFC 3511)

Average throughput from Layer 7

Transaction rate

Number of script transactions executed each second

Response time

Time (in seconds) required for one transaction


IxChariot Main Results Window

All results from tests are viewed in both graphical and

tabular format


Granular data on each timing record available

Timing Records


Feeling Good about Test Results

95% Confidence Interval

Well-known statistical measurement for the

reliability of the calculated average

95% likelihood that the actual average lies

between the lower and upper bound indicated

by the 95% Confidence Interval

Relative Precision

"Good" Relative Precision value is 10.00 or less

Can compare relative precision values regardless of

type of script run


Compare Tests


Working With Graphs

Creating Bar Graphs

Creating Pie Charts

Creating Histograms

Recognizing a Problem


Graph Types


Modifying the Axis

The graph axis scale is created automatically,

sometimes overemphasizing measurements

Adjusting the scale can clarify measurements


Using Pie Charts

Pie Charts highlight Throughput and Transaction rate

differences

ixchariot training

Documents