mas1302: computational probability and statistics: week1ndw/week1.pdf · mas1302:computational...

Introduction - Randomness

MAS1302:

Computational Probability and Statistics:Week1

Dr. David Walshaw

February 1, 2007

Dr. David Walshaw

MAS1302:Computational Probability and Statistics:Week1


Course Arrangements

Teaching slots:

Tuesday 11: Bedson LT1.Wednesday 11: Bedson TC 1.46.

Thursday 3: Bedson TC 1.46.

Dr. David Walshaw



Course Arrangements

Teaching slots:



Computer practical groups (weeks 3, 5, 7, 9)

Group 1: Monday 3pm: King George VI Lawn/Naid PCGroup 2: Monday 4pm: King George VI Lawn/Naid PC

Dr. David Walshaw



Course Arrangements

Teaching slots:



Computer practical groups (weeks 3, 5, 7, 9)

Group 1: Monday 3pm: King George VI Lawn/Naid PCGroup 2: Monday 4pm: King George VI Lawn/Naid PC

Assessment:

20% ICA; 20% Computer Projects; 60% Exam

Dr. David Walshaw



Course Structure

1 Introduction

Dr. David Walshaw



Course Structure

1 Introduction

2 Simulating Discrete Random Behaviour

Dr. David Walshaw



Course Structure

1 Introduction


3 Simulating Continuous Random Behaviour

Dr. David Walshaw



Course Structure

1 Introduction



4 Simulating Decision Problems - Gambling

Dr. David Walshaw



Course Structure

1 Introduction



4 Simulating Decision Problems - Gambling

5 Fitting Simple Statistical Models

Dr. David Walshaw



1. Introduction - Randomness

1.1 The Quantification of Uncertainty

Dr. David Walshaw





The world around us does not behave in a deterministic way.Instead we are continually faced with chance occurrences.

Dr. David Walshaw






Uncertainty is inherent in nature, from the behaviour offundamental physical particles, to that of genes andchromosomes, through to human behaviour.

Dr. David Walshaw






Uncertainty is inherent in nature, from the behaviour offundamental physical particles, to that of genes andchromosomes, through to human behaviour.

The methodology for exploring uncertainty involves the use ofrandom numbers.

Dr. David Walshaw



Example 1.1.1 How good are you at being ‘random’?

(Slide 1 of 1)

Four students were each asked to draw up a sequence of tenrandom integers between 0 and 9; their responses are shown below:

Student ‘Random’ sequence1 7 5 2 1 9 3 4 6 0 82 1 3 4 5 1 6 7 0 4 23 3 4 1 1 6 9 0 2 1 74 8 2 2 2 7 8 3 3 3 4

Dr. David Walshaw



Example 1.1.1 How good are you at being ‘random’?

(Slide 1 of 1)

Four students were each asked to draw up a sequence of tenrandom integers between 0 and 9; their responses are shown below:

Student ‘Random’ sequence1 7 5 2 1 9 3 4 6 0 82 1 3 4 5 1 6 7 0 4 23 3 4 1 1 6 9 0 2 1 74 8 2 2 2 7 8 3 3 3 4

How good do you think each of these students was at ‘beingrandom’? What does this even mean??

To answer these questions, we need to think more carefully aboutwhat we mean by randomness.

Dr. David Walshaw



Definition 1.1.1 Random integers (Slide 1 of 1)

A set of non–negative integers is a sequence of random integers if

every position in the sequence is equally likely to be occupiedby any one of the integers being used, and

positions are filled independently.

Dr. David Walshaw



Definition 1.1.1 Random integers (Slide 1 of 1)

A set of non–negative integers is a sequence of random integers if

every position in the sequence is equally likely to be occupiedby any one of the integers being used, and

positions are filled independently.

If all the digits 0, 1, 2, . . . , 9 are used, then every position in thesequence must have equal probability 1/10 of containing a 0 say,and the same probability of containing a 1, or a 2, etc. Allpositions must be filled independently, so a 0 is just as likely to befollowed by another 0 as it is by a 1, 2, etc.

Dr. David Walshaw



1.2 Obtaining random numbers

Suppose we need to obtain a list of random digits 0, 1, 2, . . . , 9.How might we go about this? There are several options:

Dr. David Walshaw




Suppose we need to obtain a list of random digits 0, 1, 2, . . . , 9.How might we go about this? There are several options:

Fair ten–sided dieIf the sides are labelled from 0 to 9 then tosses of this die willyield the required digits.

Dr. David Walshaw




Dr. David Walshaw




Tosses of a fair coinToss a fair coin four times. The following equally likelyoutcomes could correspond to the integers shown:

H H H H 0 H T H T 5H H H T 1 H T T H 6H H T H 2 H T T T 7H H T T 3 T H H H 8H T H H 4 T H H T 9

Dr. David Walshaw




Tosses of a fair coinToss a fair coin four times. The following equally likelyoutcomes could correspond to the integers shown:

H H H H 0 H T H T 5H H H T 1 H T T H 6H H T H 2 H T T T 7H H T T 3 T H H H 8H T H H 4 T H H T 9

The coin has no ‘memory’, and so each block of 4 tosses isindependent of any other. If any outcome other than those listedoccurs, we can ignore and toss a new set of 4. This method israther inefficient as a lot of the time a combination of outcomes isrejected.

Dr. David Walshaw




Dr. David Walshaw




Other physical devicesIt is possible to construct more complicated mechanicaldevices than a coin or a die, such as ‘wheels of fortune’, orphysical devices based on gamma rays emitted fromradioactive elements.

Dr. David Walshaw




Other physical devicesIt is possible to construct more complicated mechanicaldevices than a coin or a die, such as ‘wheels of fortune’, orphysical devices based on gamma rays emitted fromradioactive elements.

Tables of random numbersIn 1955 the RAND corporation published a table of 1,000,000random digits. Statistical tables also give tables of randomdigits. These could be consulted (see handout).

Dr. David Walshaw




Dr. David Walshaw




The decimal expansion of π

A sequence of random digits can be obtained by reading thetable by row, by column, or by any other rule (i.e. take every10th digit); see handout.

Dr. David Walshaw




The decimal expansion of π

A sequence of random digits can be obtained by reading thetable by row, by column, or by any other rule (i.e. take every10th digit); see handout.

One problem with all of these techniques is the fact that thequestion “is this sequence truly random?” always remains; forexample, the die could become worn with time, resulting in bias.

Dr. David Walshaw




Dr. David Walshaw




Another problem is that modern computer simulations oftenrequire many large sequences of random numbers, and themethods above are not very practical in this respect!

Dr. David Walshaw




Another problem is that modern computer simulations oftenrequire many large sequences of random numbers, and themethods above are not very practical in this respect!

Finally, computer simulations should be reproducible. So, ideally,we would like to be able to generate the identical sequence ofrandom numbers on multiple occasions.

Dr. David Walshaw



1.3 Pseudo - random numbers

It is fairly easy to see that a sequence of numbers which isgenuinely random cannot be reproduced on demand!

Dr. David Walshaw





However, it is possible to invent algorithms which generatesequences of pseudo-random numbers. These are not genuinelyrandom, but behave like completely random numbers.

Dr. David Walshaw





However, it is possible to invent algorithms which generatesequences of pseudo-random numbers. These are not genuinelyrandom, but behave like completely random numbers.

The advent of digital computers gave rise to a number oftechniques that use a recursive relation, that is the number ri in asequence is a function of the preceding numbers ri−1, ri−2, . . .Since they are generated using an algorithm, they are completelyreproducible.

Dr. David Walshaw



Example 1.3.1 (Slide 1 of 1)

Consider the following ‘random’ numbers:

r0 = 13, r1 = 21, r2 = 57, r3 = 69, r4 = 73, r5 = 41.

Can you see the pattern? What is r6?

Dr. David Walshaw




Consider the following ‘random’ numbers:

r0 = 13, r1 = 21, r2 = 57, r3 = 69, r4 = 73, r5 = 41.

Can you see the pattern? What is r6?

In fact the pattern is very difficult to see. These numbers havebeen developed using a recursive relation, and it turns out that r6is equal to 97. We shall see why in section 1.4.

Dr. David Walshaw



1.4 Congruential generators

Consider the set N0 of non–negative integers. That is,

N0 = 0, 1, 2, . . ..

Let ‘mod’ represent the modulo operation, so that forx ,m ∈ N

0, x 6= 0, (x) mod m means that x is divided by m andthe remainder is taken as the result.

Dr. David Walshaw




1 What is 13 mod 4?

Dr. David Walshaw




1 What is 13 mod 4? Answer = 1.

Dr. David Walshaw





2 What is 19 mod 5?

Dr. David Walshaw






Dr. David Walshaw



Congruential generators

Now consider the relation

ri = (ari−1 + b) mod m, i = 1, 2, . . . ,m (∗)

where r0 is the initial number, known as the seed, and a, b,m ∈ N0

are the multiplier, additive constant and modulo respectively.

Dr. David Walshaw



Congruential generators

Now consider the relation

ri = (ari−1 + b) mod m, i = 1, 2, . . . ,m (∗)

where r0 is the initial number, known as the seed, and a, b,m ∈ N0

are the multiplier, additive constant and modulo respectively.

The modulo operation means that at most m different numberscan be generated before the sequence must repeat – namely theintegers 0, 1, 2, . . . ,m − 1. The actual number of generatednumbers is h ≤ m, called the period of the generator.

Dr. David Walshaw




Example 1.4.2Selecting a = 17, b = 0, m = 100, r0 = 13 in relation (*)generates the following sequence:

i 0 1 2 3 4 5 6 7 8 9

ri 13 21 57 69 73 41 97 49 33 61

i 10 11 12 13 14 15 16 17 18 19

ri 37 29 93 81 77 9 53 1 17 89

Dr. David Walshaw




Example 1.4.2Selecting a = 17, b = 0, m = 100, r0 = 13 in relation (*)generates the following sequence:

i 0 1 2 3 4 5 6 7 8 9

ri 13 21 57 69 73 41 97 49 33 61

i 10 11 12 13 14 15 16 17 18 19

ri 37 29 93 81 77 9 53 1 17 89

The next value, r20, is found to be 13 so that this sequence thenrepeats. Thus, this sequence has period 20. This was the sequencethat was used in Example 1.3.1.

Dr. David Walshaw



Computer generation of pseudo–random numbers - remarks

Computer generation of pseudo–random numbers - remarks

1 As computers essentially use numbers to base 2, generatorsgenerally use m = 2k , where k is a very large number (k ∈ N).

2 Ideally, we want the period of the sequence to be as large aspossible.

Dr. David Walshaw



Theorem 1.4.1 - Maximum Periodicity

Theorem 1.4.1

For the relation

ri = (ari−1 + b) mod m, i = 1, 2, . . . ,m (∗),

the maximum period, m, is achieved for b > 0 if, and only if:

Dr. David Walshaw



Theorem 1.4.1 - Maximum Periodicity

Theorem 1.4.1

For the relation

ri = (ari−1 + b) mod m, i = 1, 2, . . . ,m (∗),

the maximum period, m, is achieved for b > 0 if, and only if:

(i) b and m have no common factors other than 1;

(ii) (a − 1) is a multiple of every prime number that divides m;

(iii) (a − 1) is a multiple of 4 if m is a multiple of 4.

ProofBeyond the scope of this course.

Dr. David Walshaw



Theorem 1.4.1 - Remarks

Remarks

1 If m = 2k , then if a = 4c + 1 for some positive integer c , (ii)and (iii) will hold.

2 Similarly, for (i) to be true then b must be a positive oddinteger if m = 2k .

3 As a real example, the Numerical Algorithms Group (NAG)Fortran library uses k = 59, b = 0 and a = 1313 in itsrandom number generator.

Dr. David Walshaw




Check to see if the maximum period can be achieved if thecongruential method with the following parameters is used togenerate a sequence of pseudo–random numbers:

1. a=16, b=5, m=20

Dr. David Walshaw





1. a=16, b=5, m=20

All three conditions must be satisfied for the maximum period tobe obtained, so we check each in turn.

Condition (i):

Dr. David Walshaw





1. a=16, b=5, m=20


Condition (i): False. b = 5 and m = 20 have a common factor, 5.

Dr. David Walshaw





1. a=16, b=5, m=20


Condition (i): False. b = 5 and m = 20 have a common factor, 5.

Hence the maximum period is not achieved.

Dr. David Walshaw



Example 1.4.3 (Slide 2 of 4

2. a=16, b=3, m=20

Condition (i):

Dr. David Walshaw




2. a=16, b=3, m=20

Condition (i): True. b and m have no common factors other than1.

Condition (ii):

Dr. David Walshaw




2. a=16, b=3, m=20


Condition (ii): False. (a − 1) = 15, and this is not divisible by 2,But 20 is divisible by 2.

Dr. David Walshaw




2. a=16, b=3, m=20


Condition (ii): False. (a − 1) = 15, and this is not divisible by 2,But 20 is divisible by 2.


Dr. David Walshaw




3. a=11, b=3, m=20

Condition (i):

Dr. David Walshaw




3. a=11, b=3, m=20


Condition (ii):

Dr. David Walshaw




3. a=11, b=3, m=20


Condition (ii): True. (a − 1) = 10, which is divisible by both 2 and5, which are the only primes which divide 20.

Condition (iii):

Dr. David Walshaw




3. a=11, b=3, m=20



Condition (iii): False. m = 20 is a multiple of 4, but (a − 1) = 10isn’t.

Dr. David Walshaw




3. a=11, b=3, m=20



Condition (iii): False. m = 20 is a multiple of 4, but (a − 1) = 10isn’t.


Dr. David Walshaw




4. a=21, b=3, m=20

Condition (i):

Dr. David Walshaw




4. a=21, b=3, m=20


Condition (ii):

Dr. David Walshaw




4. a=21, b=3, m=20



Condition (iii):

Dr. David Walshaw




4. a=21, b=3, m=20



Condition (iii): True. m = 20 is a multiple of 4, and (a − 1) = 20is too.

Dr. David Walshaw




4. a=21, b=3, m=20



Condition (iii): True. m = 20 is a multiple of 4, and (a − 1) = 20is too.

Hence the maximum period of 20 is achieved.

Dr. David Walshaw



1.5 Using random numbers to estimate probabilities

Example 1.5.1 The ‘birthday problem’

Consider the following problem:

In a class of n students, what is the probability that at least onepair of students shares a birthday?

Dr. David Walshaw



Mathematical solution to Example 1.5.1 (Slide 1 of 1)

See MAS1301 Semester 1 notes, Chapter 2.

Dr. David Walshaw



Simulation solution to Example 1.5.1 (Slide 1 of 1)

In computer practicals we will find a good approximation to thesolution by simulating many classes of students.

This will show that simulation can give us good approximateanswers. This is important for more complicated problems, wherethe mathematical solution is not obtainable!

Dr. David Walshaw



Example 1.5.2 The game show problem

In this classic brain–teaser, a prize is hidden behind one of threedoors and a contestant is asked to pick a door. There are goatsbehind the other two doors.

Dr. David Walshaw




In this classic brain–teaser, a prize is hidden behind one of threedoors and a contestant is asked to pick a door. There are goatsbehind the other two doors.

The host, knowing where the prize is, opens one of the other doorsto reveal a goat (the host will always open a door with a goatbehind it at this point).

Dr. David Walshaw




The host then offers the contestant the choice of staying with thedoor chosen originally or changing to the other unopened door.

Dr. David Walshaw




The host then offers the contestant the choice of staying with thedoor chosen originally or changing to the other unopened door.

The question is whether it is better for the contestant to stay or tochange at this point.

Dr. David Walshaw




The answer to this question is not intuitive. Basically, the theorysays that if the contestant changes their mind, the odds of themwinning the prize double!

Dr. David Walshaw



Remarks about the game show problem (Slide 1 of 1)

This problem, also known as the Monty Hall problem, caused hugecontroversy in the United States in the 1990s, with manymathematics professors claiming that the correct solution waswrong! For a discussion, visit

http://www.willamette.edu/cla/math/articles/marilyn.htm

or to play the game visit

http://math.ucsd.edu/∼crypto/Monty/monty.htm

Dr. David Walshaw



Remarks about the game show problem (Slide 1 of 1)

This problem, also known as the Monty Hall problem, caused hugecontroversy in the United States in the 1990s, with manymathematics professors claiming that the correct solution waswrong! For a discussion, visit

http://www.willamette.edu/cla/math/articles/marilyn.htm

or to play the game visit

http://math.ucsd.edu/∼crypto/Monty/monty.htm

As we shall see in practical sessions in this course, one very easyway of checking the truth is to simulate repeated instances of thegame show scenario unfolding!

Dr. David Walshaw


mas1302: computational probability and statistics: week1ndw/week1.pdf · mas1302:computational...

Documents