ie 303 discrete-event simulationtests for uniformity kolmogorov, smirnov tests chi-square tests test...

L E C T U R E 6 : R A N D O M N U M B E R G E N E R A T I O N

IE 303 Discrete-Event Simulation

Review of the Last Lecture

Continuous Distributions Uniform distributions

Exponential distributions and “memoryless property”

Gamma Distributions

Erlang Distributions

Weibull Distribution

Normal Distribution

Triangular Distribution

Lognormal Distribution

Truncated Normal Distribution

Convolution

Maximum Likelihood Estimation

Outline of Lecture 6

Random Number Generation in Simulation

Definition of Pseudo Random Numbers

Linear Congruent Method

Random Number Streams

Test for Random Numbers

Tests for Uniformity

Kolmogorov, Smirnov Tests

Chi-square Tests

Test for Statistical Independence from History

Outline

In this lecture we will learn how to generate random numbers and statistical tests for measuring ‘randomness’.

Why do we learn this?? Suppose you start your IE career at a moderately large manufacturing company and

you are faced with a design of parallel assembly lines problem. However your company cannot afford buying ARENA or Flexim (commercial license

costs 32500€ per PC, 32000$ for 2 PC). Your boss suggest you two options: a) You might be allowed for 2 or 3 weeks to build your own program (in Java or

C++) which solves the problem using discrete event simulation. b) He is ok with buying a cheaper simulation software package which has never

been used by someone else, i.e. unknown reliability of results! What to do??

Empirical Distribution

An empirical distribution, either discrete or continuous, is a distribution whose parameters are the observed values in a sample of data.

This is in contrast to parametric distribution families such as the Exponential(λ), Normal(μ,σ) or Poisson(λ), which are characterized by specifying a small number of parameters such as the mean, the variance, the rate of activity, etc.

An empirical distribution may be used when it is impossible or unnecessary to establish that a random variable has any particular parametric distribution.

One advantage of an empirical distribution is that nothing is assumed beyond the observed values in the sample; however, this is also a disadvantage because the sample might not cover the entire range of possible values.

03-Nov-16

5


Example: Customers at a local

restaurant arrive at lunchtime in groups of from one to eight persons. The number of persons per party in the last 300 groups has been observed; the results, the relative frequencies are summarized in the tables below. An histogram and the empirical cdf are depicted as well.

03-Nov-16

6


03-Nov-16

7

Random Number Generation

Random numbers are a necessary basic ingredient in the simulation of all discrete systems.

Most computer languages have a subroutine or function that will generate a random number.

Similarly, simulation languages generate random numbers that are used to generate event times and other random variables.

Any random number generation should satisfy two properties of random numbers: Uniformity

independence


Random numbers should come from Uniform(0,1) and each random number should be independent from its history?

But why????


Lets revisit our coin tossing game between Harry and Tom. Charlie, the referee of the game, successively tosses a fair coin. Heads: Tom -> Harry $1

Tails: Tom <- Harry $1

Coin tosses should be fair (the same probability of having heads or tails) – uniformity.

New games should be independent of the previous results – statistical independence.

-25

-20

-15

-10

-5

0

5

10

15

20

25

30

1 8

15

22

29

36

43

50

57

64

71

78

85

92

99

Ha

rr

y's

Win

nin

gs

Pseudo Random Numbers

Random numbers generated by computers are called pseudo random numbers.

“Pseudo” means false, so false random numbers are being generated!

In this instance, “pseudo” is used to imply that the very act of generating random numbers by a known method removes the potential for true randomness.

If the method is known, the set of random numbers can be replicated. But then these numbers are not truly random.

Whereas in computer simulation of discrete-event systems replicability is actually a desired feature.


In the generation of pseudo-random numbers certain problems or errors can occur.

These errors, or departures from ideal randomness, are all related to the two properties: Uniformity and independence. Some examples of which are: The generated numbers might not be uniformly distributed. The generated numbers might be discrete-valued instead of continuous-valued. The mean and/or the variance of the generated numbers might be too high or too

low. There might be dependence. The following are examples: – autocorrelation

between numbers; – numbers successively higher or lower than adjacent numbers; – several numbers above the mean followed by several numbers below the mean.

There are statistical tests to detect such departures from uniformity and independence. Once a generator (a method or a routine) is found to be defective, it will be dropped in favor of an acceptable generator.


Desired Qualities of a Pseudo-Random Number Generator The routine should be fast. A simulation experiment could require

many millions of random numbers. The generator needs to be computationally efficient.

The routine should be portable to different programming languages. It is desirable that the simulation program will produce the same results wherever it is executed.

The routine should have a sufficiently long cycle.

The random numbers should be replicable.

The generated random numbers should closely approximate the ideal statistical properties of uniformity and independence.


The linear congruent method produces integers, X1, X2, ... between 0 and m – 1 by following a recursive relationship:

Random number is

The initial value X0 is called the seed, a is called the multiplier, c is called the increment, and m is called the modulus.


The selection of the values for a, c, m and X0 drastically affects the statistical properties and the cycle length.

Variations of this method are quite common in the computer generation of random numbers.

Example

Use X0 = 27, a = 17, c = 43, and m = 100. Here, the integer values generated will all be between 0 and 99 because of the value of the modulus. Also, notice that random integers are being generated, are discrete rather than continuous. These random integers should appear to be uniformly distributed on the integers 0 to 99.


Generated random numbers…

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 5 10 15 20

There is sth wrong…. a) Theoretically the numbers generated

assume values only from the set {0, 1/m, 2/m, ... , (m-1)/m} because each Xi is an integer in the set {0, 1, 2, ... , m-1}. Hence obtained random numbers follow a discrete distribution.

b) These numbers has a period of 5!

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 5 10 15 20

So what???


Discreteness can be overcome if the modulus m is a very large integer.

Values such as m=231–1 and m=248 are common use in generators appearing in many simulation languages.

It is desirable the values assumed by Ri , i = 1, 2, ..., leave no large gaps on [0,1].

Also notice that in practical applications, the generator should have the largest possible period. Maximal period should be achieved by the proper choice of a, c, m, and X0.


Three choices:

For m a power of 2, say m = 2b, and c ≠ 0, the longest possible period is P = m = 2b, which is achieved whenever c is relatively prime to m (that is the common factor of c and m is 1) and a = 1 + 4k, where k is an integer.

For m a power of 2, say m = 2b, and c = 0, the longest possible period is P = m /4 = 2b–2, which is achieved if the seed X0 is odd and if the multiplier, a, is given by a = 3 + 8k or a = 5 + 8k, for some k = 0, 1, ...

For m a prime number and c = 0, the longest possible period is P=m– 1, which is achieved whenever the multiplier, a, has the property that the smallest integer k such that ak – 1 is divisible by m is k=m–1.


Example Find the period of the generator for a = 13,

c=0, m=26 = 64, and X0 = 1, 2, 3, and 4. The solution is given in the table on the right.

When the seed is 1 or 3, the sequence has period 16. However, a period of length 8 is achieved when the seed is 2 and a period of length 4 occurs when the seed is 4.

In this example the maximal period is given by the second rule given above, m = 24, and c = 0, the longest possible period is P = 64 /4 = 16. As mentioned before, the maximal period is achieved when the seed is odd and a has the form given by 5 + 8k. When the seed is 1, the density is 4/64=0.0625.

Note that modulus operation is very efficient when the modulo is a power of 2 which is another consideration when selecting a, c, and m.


Example

Different values have been extensively tested. The values for a, c, and m have been selected to ensure that the characteristics desired in a generator are most likely to be achieved. By changing X0, the user can control the repeatibility of the stream.

Let a = 75 = 16807, m = 231 –1 = 2,147,483,647 (a prime number), and c = 0. These choices satisfy the conditions that insure a period of P = m – 1, (well over 2 billion). Further, specify the seed X0 = 123,457. The first few numbers generated are as follows:


The seed for a linear congruential random number generator is the integer value X0 that initializes the random-number sequence. Since the sequence of integers X0, X1, X2, ... , XP ,produced by a generator repeats, any value in the sequence could be used to “seed” the generator.

For a linear congruential generator, a random number stream is nothing more than a convenient way to refer to starting seed taken from the sequence X0, X1, X2, ... , XP.

Tests for Random Numbers

Generated numbers should be tested if they satisfy the two property of random numbers:

Uniformity

Independence

To this end, we use hypothesis tests. Uniformity is tested with

Kolmogorov Smirnov

Chi-square test

Whereas independence is tested with

Autocorrelation test


Hypothesis statements in these tests are

The level of significance denoted by α satisfies:

α= P{reject H0 | H0 is true}

which means that the critical value for the test statistic is selected such that the probability of rejecting the null hypothesis, H0, equals α.

Uniformity Independence


For a reliable decision on the acceptance of a random number generator, statistical tests must be done on a large number of samples, say M.

If the number of rejected samples significantly exceeds the expected number of rejected tests, aM, then we shall discard the given random number generator as the random numbers generated by it do not possess the property which we are testing for.

Kolmogorov Smirnov Test

END OF LECTURE 6

Next Lecture: Statistical Tests (Chp. 7), @LMF 208 (Computer Lab)

Kolmogorov Smirnov Test

Chi-Square Test

ie 303 discrete-event simulationtests for uniformity kolmogorov, smirnov tests chi-square tests test...

Documents