efficient simulation of network performance by importance ...presentation of thesis, 1998-06-09 1...

Presen

Efficient Simulation of Network Performance by

Importance Sampling

Poul E. Heegaard

Norwegian University of Science and Technology Trondheim, Norway

Outline

• MotivationSYSTEM SIMULATION MODEL

9

2

45

6

781

• Main contributions

• Rare event simulation

• Importance sampling

• Adaptive parameter biasing

• Network simulations

• Closing remarks

tation of thesis, 1998-06-09 1

NTNU - Dept. of Telematics

Presen

Motivation

Task: evaluation of system which are characterized by

- being large and complex

- being distributed with tight logical couplings

- have strict quality of service

SYSTEM SIMULATION MODEL

9

2

45

6

78

1

Evaluation means:

- analytic (efficient, but needs oversimplification)

- simulation (flexible, but inefficient)

- measurements (efficient, but expensive and inflexible)



Presen

Motivation

=> Simulation is flexible means for system evaluation,

=> Speed-up is required,

=> Importance sampling is efficient,

=> Optimal/good parameters are essential.

1e-09

2e-09

3e-09

4e-09

5e-09

6e-09

Optimal change of measure

prop

erty

of

inte

rest

Simulation results

Exact, = 4.90e-09

Error bars

Insensitive to changes in f*(s)

change of measure

This thesis:

Can importance sampling be applied in network simulations?



Presen

Why is network simulation with rare events a problem?

3

5

4

6

2

110

9

23

4

5

6

78

1

0.1

1

10

100

1000

1e-07 1e-06 1e-05 0.0001 0.001 0.01 0.1 1

Spe

ed-

up g

ain

[lo

g]

Exact blocking [log]

Domain of interest (<1e-06)!

users

link failures

Restrictive quality of service requirements

- model size and complexity

- direct simulation of properties

is very inefficientdependent on rare events



Presen

Main contributions

- Adaptive biasing for importance sampling in perfor-mance simulations of networks

- Flexible modelling framework

- Network simulation for feasibility demonstrations

1e-11

1e-10

1e-09

1e-08

1e-07

0 1 2 3 4 5 6 7 8 9 10generators

simulation results

exact results

bloc

king

pro

babi

litie

s

with error bars

- Heuristics importance sampling experiments

- Combination of speed-up techniques

- Comparisons of rare event techniques

- Application of importance sampling to MPEG and ATM



Presen

Speed-up techniques

SYSTEM SIMULATION MODEL

9

2

45

6

781

PARALLEL AND DISTRIBUTED SIMULATION

Parallel and distributed

1

1 N

N

2 N

1

Parallel independent replicated simulation

SPEED-UPS?

less than N

close to N

HYBRID TECHNIQUES COMBINES ANALYTIC SOLUTIONS WITH SIMULAT- decomposition in time or space

- conditional sampling

T1 T2T3 T4 T5

X1

X0 | X1=x

x-1xx+1

RARE EVENT PROVOKING TECHNIQUEVARIANCE MINIMIZATION EXPLOITS

-0.5

0

0.5

1

1.5

2

2.5

0 2 4 6 8 10 12 14 16

X a

nd Y

YX

KNOWN OR INTRODUCED CORRELATION

- control variable

RESTART with 4 levels

Original distribution

1

10-2

10-4

10-6

10-8

5 10 15 20

Impo

rtan

ce s

amp

ling

CHANGE THE SAMPLING DISTRIBUTIO

- antithetic variates - common random

simulation (PADS)

number



Presen

Rare event simulation techniques

Objective:

Change the underlying sampling distribution to provoke rare events of interest to occur more often.

Known approaches:

- RESTART

- Importance sampling



Presen

5

1

5

2

5

3

5

Importance sampling

2 4 6 8 10 12 14 x

f(X)=original model

y

f*(X)=stressed model Rare event:Pf g X( ) 1=( ) 1«

Importance sampling:Pf g X( ) 1=( ) P

fg X( ) 1=( )«

Observation: g X( ) I x y( )= , e.g.:

-> overflow of MPEG cells in a multiplexer

-> blocked calls

Relation: Ef g X( )( ) Ef g X( ) X( )( )= where

X( )f X( )f X( )------------= the likelihood ratio between f X( ) and f X( )

Estimator: X

1n--- Xi( ) g Xi( )

i 1=

n

=

where Xi are samples from f x( )



Presen

Importance sampling heuristics (1)

- Use likelihood ratio as indication of goodness of simu-lation results

- no analytic solution available

- no direct simulation results

- use knowledge of E L( ) 1=

- Two observations from experiments:

(i) L 1 and rel.error L( ) 1« => ̂IS is good if its relative error rel.error ̂IS( ) 1«

(ii) L 1« or rel.error L( ) 1 :

=> ̂IS is poor even if the rel.error ̂IS( ) 1« .



Presen

Importance sampling heuristics (2)

- The sampling distribution is heavy tailed under too strong biasing (infinite variance)

R

running mean of IS

true value E[]

^



Presen

Modelling framework

3

5

4

6

2

110

9

23

4

5

6

78

1

user

link failures

Model feasibilities- different resource requirements- different quality of service requirements- pre-emptive priorities- alternative routing on overload and failures

Objectives: assessment of- blocking probability- rerouting probability- disconnection probability- consequence of pre-emption- consequence of failures

priority=1

priority=2

primary route user type B

secondary route user type Aprimary route

type A

usertype B

user type A



Presen

Mapping to state space model (1)

3

5

4

6

2

110

9

23

4

5

6

78

1

2

1

2

61

5

10

3

Resource Generator

1

2

pool 103

2



Presen

Mapping to state space model (2)

blocking pool 10 =>

generator 2

blocking pool 3

blocking pool 2

gene

rato

r 1

Resource Generator

1

2

pool 103

2

ARGET SUBSPACE 10

0,0 0,1 0,2 0,3 0,4

1,0 1,1 1,2 1,3 1,4

2,0 2,1 2,2 2,3

3,0 3,1 3,2

current state



Presen

#entities gen. 1,#entities gen. 2

arrival of entity from gen. 2

departure of entity from gen. 2

TARGET SUBSPACE(resource constraint,

boundary, barrier

STATE

EVENTS

State space model



Presen

Importance sampling in network simulations

Challenges

STATE SPACE MODEL

3

10

2

BARRIER

Gen

1

Gen 1

Gen 2

3

210

Gen 2

• Multidimensional state models

• Balanced dimensioning, i.e. no bottlenecks

Previous solutions to change of measure

• Identify a bottleneck

3

10

2

Gen

1

Identify the bottleneckand change the measureaccording to this • Drift towards the bottle-

neck barrier

• Fixed change of measure

• If no single bottleneck => inefficient solution!



Presen

Adaptive change of measure

3

10

2

Gen

1

3

10

2

Gen

1

3 2 10

importance

3

10

2

Gen

1

3 2 10

importance

3 2 10

importance

Choose a barrier directionat random, and change the measure toward this.

Approached barrier 3:Choose the barrier directionagain according to the newrelative importance estimates

Approached barrier 10:Choose the barrier directionagain according to the newrelative importance estimates

A new, adaptive approach

• Towards the most important barrier at current state.

• State dependent change.

• A good estimate on the “current importance of all barrier” is required.

Algorithm

At each state:

(i) Estimate the currentimportance of all barriers

(ii) Choose a direction

(iii) Induce drift in the chosen direction.

Step (i)-(iii) are repeated for every state.



Presen

Estimation of target importance

- Requirements:

1. Sufficiently accurate

2. Robust

3. Efficient

- Simplification:

- only the relative importance is of interest

- use the greatest importance contribution, Hj1

˜( )

=> must identify the sub-path from current state ˜

toa state in the target sub-space

˜j .



Presen

Efficient search for the sub-path

- find the sub-path with the largest contribution to Hj ˜

( )

- exploit the Markov properties

0,0 1,0 2,0

0,1

0,2 1,2 2.2 3,2

3,12,1

4,2

5,0

5,1

0,0 1,0 2,0 2,1 3,1 3,2 4,2 5,2

5,2

x = 0 1 2 4 5 7 8 9

x ck1j– xx ck2j–

k1˜

x ck1j– 1˜

k1+( )

k2˜

x ck2j– 1˜

k2+( )

k1˜

x ck1j– ( )

k2˜

x ck2j– ( )

number of resources allocated



Presen

Network example 1

- No priority nor alternative routing

5

3

4

6

2

1

10

3

9

412

2

7

1

711

8 5

86

5

1e-11

1e-10

1e-09

1e-08

1e-07

0 1 2 3 4 5 6 7 8 9 10

generators

simulation results with error bars

exact results

blo

cki

ng

pro

bab

ilit

ies

- Compared with exact results

- Simulation more efficient than numerical calculations



Presen

Network example 2

- Improving the quality of service by rerouting

3

5

4

6

2

110

9

2

3

4

5

6

78

1

blo

ck

ing

pro

ba

bil

itie

s

Upper bounds of blocking

1e-10

1e-09

1e-08

1e-07

1e-06

1e-05

1e-04

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Simulated blocking probability of generator 5

(a) With primary route only

generators

- Simulation results close to rough blocking estimates

- Mean likelihood ratio close to 1 with low relative error

- Significant speedup over direct simulation observed



Presen

Network example 3

- Disturbing low priority traffic

1e-07

1e-06

1e-05

1e-04

1e-03

bloc

king

pro

babi

lity CASE 3.1:

low priority

CASE 3.2:mixed with

CASE 3.3:mixed with

high priority traffic high priority trafficand exposed to

gen.23 gen.31 gen.23 gen.23gen.31 gen.31

traffic only

link failures

- Best results for generator 23 and 31 in accordance to the biasing setup of importance sampling

- No speed-up compared to direct simulation (loss prob-ability in order of 10 4– .

- The mean likelihood ratio is 0.742 for case 3.3 => overbiased?



Presen

Closing remarks

- importance sampling in performance simulation of tel-ecom networks with:

- balanced utilisation of resources

- users with different quality of service requirements

- preemptive priority and rerouting

- link and node failures

- new, adaptive importance sampling biasing proposed

- flexible modelling framework applied

- feasibility demonstrated

- heuristics for importance sampling experiments

- combination of speed-up techniques

- importance sampling in other applications

- further development of fundament required before inclusion of rare event techniques in simulation tool



efficient simulation of network performance by importance ...presentation of thesis, 1998-06-09 1...

Documents