A Guide to Simulation

Springer Science+Business Media, LLC

Paul BratleyBennett L. FoxLinus E. Schrage

A Guide to SimulationSecond Edition

With 32 Illustrations

Springer

Paul Bratley Departement d'informatique et de

recherche operationnelle Universite de Montreal Montreal, P.Q. H3C 317 Canada

Bennett L. Fox Department of Mathematics

University of Colorado

Denver, CO 80217-3364

U.S.A.

Linus E. Schrage Graduate School of Business University of Chicago Chicago, IL 60637 U.S.A.

Mathematics Subject Classifications: 90CXX, 65ClO, 68U20

Library of Congress Cataloging in Publication Data Bratley, Paul.

A guide to simulation. Bibliography: p. Includes indexes. 1. Digital computer simulation. I. Fox.

Bennett L., 1938- II. Schrage, Linus E. III. Title. QA76.9.C65B73 1987 001.4'34 86-26287 ISBN 978-1-4612-6457-6 ISBN 978-1-4419-8724-2 (eBook) DOI 10.1007/978-1-4419-8724-2

© 1987 by Springer Science+Business Media New York Originally published by Springer-Verlag New York Inc. in 1987 Softcover reprint of the hardcover 2nd edition 1987 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden.

Typeset by Composition House Ltd., Salisbury, England.

98765 4 3 2

SPIN 10534734

ToRonald and Margaret BratleyDaniel and Kate FoxAlma and William Schrage

Preface to the Second Edition

Changes and additions are sprinkled throughout. Among the significant newfeatures are :

• Markov-chain simulation (Sections 1.3, 2.6, 3.6, 4.3, 5.4.5, and 5.5);• gradient estimation (Sections 1.6, 2.5, and 4.9);• better handling of asynchronous observations (Sections 3.3 and 3.6);• radically updated treatment of indirect estimation (Section 3.3);• new section on standardized time series (Section 3.8);• better way to generate random integers (Section 6.7.1) and fractions

(Appendix L, prog ram UNIFL);• thirty-seven new problems plus improvements of old problems.

Helpful comments by Peter Glynn, Barry Nelson, Lee Schruben, and PierreTrudeau stimulated several changes. Our new random integer routineextends ideas of Aarni Perko. Our new random fraction routine implementsPierre L'Ecuyer's recommended composite generator and provides seeds toproduce disjoint streams. We thank Springer-Verlag and its late editor,Walter Kaufmann-Bilhler, for inviting us to update the book for its secondedition. Working with them has been a pleasure. Denise St-Michel againcontributed invaluable text-editing assistance.

Preface to the First Edition

Simulat ion means driving a model of a system with suitable inputs andobserving the corresponding outputs. It is widely applied in engineering,in business, and in the physical and social sciences. Simulation method­ology draws on computer science, statistics, and operations research andis now sufficiently developed and coherent to be called a discipline in itsown right. A course in simulation is an essential part of any operations re­search or computer science program. A large fraction of applied work inthese fields involves simulation ; the techniques of simulation, as tools, areas fundamental as those of linear programming or compiler construction,for example. Simulation sometimes appears deceptively easy, but perusalof this book will reveal unexpected depths. Many simulation studies arestatistically defective and many simulation programs are inefficient, Wehope that our book will help to remedy this situation. It is intended to teachhow to simulate effectively .

A simulation project has three crucial components, each of which mustalways be tackled :

(1) data gathering, model building, and validation;(2) statistical design and estimation;(3) programming and implementation.

Generation of random numbers (Chapters 5 and 6) pervades simulation,but unlike the three components above, random number generators neednot be constructed from scratch for each project. Usually random numberpackages are available. That is one reason why the chapters on randomnumbers, which contain mainly reference material, follow the chapters deal­ing with experimental design and output analysis.

x Preface to the First Edition

A simulation technique is not really appreciated until a feeling for howit might be implemented in practice is acquired . We therefore give workedexamples to illustrate the theory. Mastering modeling comes only by doingit ; but to give inexperienced readers the proper perspective the book beginswith a detailed discussion of modeling principles. The specific models givenoccasionally in the book are there only to illustrate technical points. Theyare simplistic and are not to be taken seriously as reasonable representationsof real-world problems. On the other hand, we point out a common error insimulation models : excessive detail.

A simulation course should attempt to cover only the substantial amountof material that deals directly with simulation. Time does not allow it toserve also as a vehicle for teaching probability, statistics, computer pro­gramming, or data structures. For understanding most of this book onecourse -in each of these topics should suffice. This knowledge should becomplemented by a certain mathematical maturity, such as is normallyacquired in a rigorous undergraduate course in advanced calculus or linearalgebra. If the instructor briefly reviews a few points in class, good studentscan get by with fewer formal prerequisites. The rare allusions to moreadvanced material can be skimmed without loss of continuity. Chapter 7,on simulation programming, requires almost no mathematical background.Only Section 1.7, a few problems in Section 1.9, and Chapter 7 requirecomputer science background beyond elementary programming to be fullyappreciated.

The ordering of the material in the book reflects what we consider to bethe most natural logical flow. The progression is not always from easy tohard: Chapters 2 and 3 are probably the most difficult in the book and Sec­tion 1.9 has some of the most challenging problems. Other orderings arecertainly feasible, as the Precedence Diagram given after the Contentsindicates. It shows only the most prominent relationships among thechapters which are all to some extent interrelated. Course structure is flexible.

For example, an instructor can structure a computer science orientedcourse based on this book at the last-year undergraduate level by focusingmainly on Chapters 1, 5, 6, and 7. Sections 2.1.1, 3.1, 3.2, and 4.1-4.8 canalso be covered without difficulty in an undergraduate course that other­wise deals with computer science aspects of simulation. To concentrateon simulation programming languages cover Sections 1.1-1.5 and 1.8, andChapter 7. This should be complemented by a detailed study, with pro­grammed examples, of at least one of the special-purpose languages wediscuss. Chapter 7 is not intended as a substitute for the reference manualsof the languages compared there . Rather, we aim to teach critical apprecia­tion of general principles. To fully appreciate our critique, the studentshould at least have access to supplementary documentation; ideally, heshould have the facilities to try several languages .

Although such a course may perhaps be thought sufficient for the com­puter science student or the programmer, it is not adequate for a designer

Preface to the First Edition Xl

with overall responsibility for a simulation project. In this wider context webelieve that the student should acquire proper understanding of the theorybefore he rushes to get simulation programs running on a computer. Theordering of the chapters follows this principle. A course based on this bookemphasizing statistical aspects of simulation can normally be expected to be atthe first-year graduate level. If the students already have some computerscience background, as we would expect, much of Chapters 1, 7, and 8 canbe left as reading assignments.

Most of the book can be covered in a one-semester course. The instructorcan flesh out a two-semester course by presenting case studies and havingthe students formulate and validate models, write simulation programs,and make statistical analyses. If there is time left over, a few papers fromthe recent research literature can be discussed, or a particular simulationlanguage can be studied in detail. The entire book is also suitable for in­dependent study and as a reference for practicing professionals with diverseinterests and backgrounds.

The problems are often an integral part of a development. Some testcomprehension. Others contain results of independent interest, occasion­ally used later. They vary from almost trivial to quite hard. They are placedwhere we think it will do the reader the most good . All should be read beforeproceeding: try most.

Simulation is unique in the opportunities it offers to reduce the variancesof performance estimators by astute experimental design. The book exploresthese opportunities thoroughly. We highlight the roles of mono tonicity andsynchronization in inducing correlation of appropriate sign for variancereduction. One consequence is that we favor inversion as a way of generatingnonuniform random numbers. This lets us choose nonuniform randomnumber generators to discuss quite selectively among the mushroomingnumber of papers on the subject. Many of these papers are cited but nottreated in detail.

Occasionally we take viewpoints not universally shared and emphasizecertain aspects that others slight. The central role that we give to synchron­ization and our consequent concentration on inversion is one example ofthis. Probably controversial in some quarters is our position that finite­horizon models are usually appropriate, especially in a simulation context.For such models, multiple replications form the basis for fairly straight­forward estimation and statistical inference. Infinite-horizon models areartifices that may occasionally be convenient fictions. Sometimes useful,they are too often used. For them, no method of constructing confidenceintervals for unknown parameters that is in all respects both completelyrigorous and practical has been devised. This probably explains why so manyapproaches have been discussed in the simulation literature. We cover themain ones: batch means and regenerative, spectral , and autoregressivemethods. When pungency, not blandness, is called for our tone is sometimesfrankly tendentious.

Xli Preface to the First Edition

The book treats all aspects of discrete-event digital simulation in detail.We discuss so-called continuous and hybrid simulation briefly in Sections1.4.2. and 1.4.3. The Contents, a summary of notation, and a precedencediagram follow this preface. References cited are listed in a separate sectionnear the end of the book, ordered by author and year of publication. To avoidcluttering the text, longer programmed examples are banished to an appendix.

We acknowledge the help received from many colleagues and friends.Among those whose suggestions, comments, and criticism of earlier ver­sions of this book were particularly valuable, we thank Robert Cooper,Eric Denardo, Luc Devroye , U. Dieter, Peter Glynn, Donald Iglehart,Donald Knuth, Patrice Marcotte, Barry Nelson , Bruce Schmeiser, andLudo Van der Heyden . Edward Russell supplied the example program ofFigure X.7.5. Kevin Cunn ingham wrote the programs to draw many of thefigures. We are particularly grateful to those who do not share our opinionsbut freely gave their help anyway.

Denise St-Michel nurtured our manuscript through endless revisionson our local text-editing system. For her care and cheer, we are grateful.We also thank Yves Courcelles, whose software made life much easier. NoahWebster settled many an argument.

Our work was generou sly supported by the Natural Sciences and Engin­eering Research Council of Canada.

Contents

Notation

Precedence Diagram

CHAPTER 1

Introduction1.1. Systems, Models, and Simulation1.2. Verification, Approximation, and Validation

1.2.1. Verifying a Program1.2.2. Approximation and Validation

1.3. States, Events, and Clocks1.4. Simulation-Types and Examples

1.4.1. Synchronous and Asynchronous Discrete-Event Simulation1.4.2. Continuous Simulation1.4.3. Hybrid Simulation

1.5. Introduction to Random Numbers1.6. Perspective on Experimental Design and Estimation1.7. Clock Mechanisms1.8. Hints for Simulation Programming1.9. Miscellaneous Problems

CHAPTER 2Variance Reduction2.1. Common Random Numbers

2.1.1. Informal Approach2.1.2. Formal Development2.1.3. Auxiliary Results

2.2. Antithetic Variates2.3. Control Variates

xixxxi

II899

II141718242526303234

44484851565659

xiv

2.4. Stratification2.5. Importance Sampl ing2.6. Conditional Monte Carlo2.7. Jackknifing

CHAPTER 3Output Analysis3.1. Introduction

3.1.1. Finite-Horizon Versus Steady-State Performance3.1.2. Fixed Sample Size Versus Sequential Sampling

3.2. Analysis of Finite-Horizon Performance3.2.1. Absolute Performance Estimation3.2.2. Relative Performance Estimation

3.3. Analysis of Steady-State Performance3.3.1. Batch Means3.3.2. Regenerative Methods3.3.3. Spectral Analysis Methods3.3.4. Autoregressive Methods3.3.5. Recommendations

3.4. Analysis of Transaction-Based Performance3.5. Indirect Estimation via r = A.S3.6. Problems3.7. Renewal Theory Primer3.8. Standardized Time Series

3.8.1. Steady State3.8.2. Transients

CHAPTER 4Rational Choice of Input Distributions4.1. Addition and the Normal Distribution4.2. Multiplication and the Lognormal4.3. Memorylessness and the Exponential4.4. Superposition, the Poisson, and the Exponential4.5. Minimization and the Weibull Distribution4.6. A Mixed Empirical and Exponential Distribution4.7. Extreme Values and Spacings4.8. When Not to Use a Theoretical Distribution4.9. Nonstationary Poisson Processes

CHAPTER 5Nonuniform Random Numbers5.1. Introduction5.2. General Methods

5.2.1. Inversion5.2.2. Tabular Approximation of the Inverse Transform5.2.3. Empirical cdf's5.2.4. A Mixed Empirical and Exponential Distribution5.2.5. Rejection

Contents

63657074

77777881828284859095

100103106107108109113120121123

124126128129130130131132133135

145145146147149150150151

Contents

5.2.6. Generalized Rejection5.2.7. Composition5.2.8. The Alias Method for Discrete Distributions5.2.9. Functional Approximations of the Inverse Transform5.2.10. Other Ingenious Method s

5.3. Continuous Distributions5.3.1. The Non standard Normal Distr ibution5.3.2. The Multivariate (Dependent) Normal Distribution5.3.3. Symmetric Stable Variates5.3.4. The Cauchy Distribution5.3.5. The Lognormal Distribution5.3.6. The Exponential Distribution5.3.7. The Hyperexponential Distribution5.3.8. The Laplace and Exponen tial Power Distr ibutions5.3.9. Erlang and Gamma Distributions5.3.10. The Beta Distribution5.3.11. The Chi-square Distribution5.3.12. The F-Distribution5.3.13. The r-Distribution5.3.14. The Weibull Distribution5.3.15. The Gumbel Distribution5.3.16. The Logistic Distr ibution5.3.17. The Generalized Lambd a Distribution5.3.18. Nonhomogeneous Poisson Processes

5.4. Discrete Distributions5.4.1. The Binomial Distribution5.4.2. The Poisson Distribution5.4.3. Compound Poisson Distr ibut ions5.4.4. The Hypergeometric Distribution5.4.5. The Geometric Distribution5.4.6. The Negative Binomial and Pascal Distributions5.4.7. Multid imensional Poisson Distributions

5.5. Problems5.6. Timings

CHAPTER 6

Uniform Random Numbers6.1. Random Introductory Remarks6.2. What Constitutes Randomness6.3. Classes of Generators

6.3.1. Random Devices6.3.2. Tables6.3.3. The Midsquare Method6.3.4. Fibonacci and Additive Congruential Generators6.3.5. Linear Congruential Generators6.3.6. Linear Recursion mod 2 Generators6.3.7. Combinations of Generators

6.4. Choos ing a Good Generator Based on Theoret ical Considerations6.4.1. Serial Correlation of Linear Congruential Generators

xv

152156158159161164164164165167168168169170171172174175175176177178178178181181182183183184185186186189

192192192194195195195196196198203205205

XVI Contents

6.4.2. Cycle Length of Linear Congruential Generators 2066.4.3. Cycle Length for Tausworthe Generators 2086.4.4. The Spectral Test 208

6.5. Implementation of Uniform Random Number Generators 2116.5.1. Multiplicative Generator With Modulus 2" 2116.5.2. Multiplicat ive Generators With Prime Modulus 2126.5.3. Implementing the Tausworthe Generator 215

6.6. Empirical Testing of Uniform Random Number Generators 2166.6.1. Chi-square Tests 2176.6.2. Kolmogorov-Smirnov Tests 2186.6.3. Tests Specificallyfor Uniform Random Numbe r Sequences 219

6.7. Proper Use of a Uniform Random Number Generator 2206.7.1. Generating Random Integers Uniform Over an Arbitrary Interval 2206.7.2. Nonrandomness in the Low-order Bits of Multiplicative Generators 2216.7.3. Linear Congruential Generators and the Box-Muller Method 223

6.8. Exploiting Special Features of Uniform Generators 2246.8.1. Generating Antithetic Variates With a Multiplicative Congruential

Generator 2246.8.2. Generating a Random Number Stream in Reverse Order 2256.8.3. Generating Disjoint Sequences 226

CHAPTER 7Simulation Programming 2287.1. Simulation With General-Purpose Languages 229

7.1.1. The Simplest Possible Clock Mechanism 2307.1.2. Generating Random Variates 2327.1.3. Data Structures in Fortran 2337.1.4. A Complete Simulation in Fortran 2347.1.5. Fortran Simulation Packages 2367.1.6. Our Standard Example-the Naive Approach 2377.1.7. Simulation Using Pascal 241

7.2. Simscript 2437.2.1. Data Structures in Simscript 2447.2.2. Simscript and Simulation 2457.2.3. A Complete Simulation in Simscript 2477.2.4. The Standard Example in Simscript 2497.2.5. Processes and Resources 2507.2.6. Summing-up 250

7.3. GPSS 2517.3.1. The Basic Concepts 2517.3.2. Resources in GPSS 2547.3.3. Generating Random Variates 2557.3.4. A Complete Simulation in GPSS 2577.3.5. The Standard Example in GPSS 2607.3.6. Summing-up 261

7.4. Simula 2637.4.1. The Class Concept in Simula 2637.4.2. Simulation Using System Classes and Procedures 2667.4.3. A Complete Simulation in Simula 269

Contents

7.4.4 . Demos7.4.5 . The Standard Example in Simula With Demos7.4.6 . Summing-up

7.5. General Considerations in Simulation Programming7.5.1. Language Design7.5.2. System Considerations in Simulation7.5.3. Statistical Considerations

CHAPTER 8Programming to Reduce the Variance8.1. Choosing an Input Distribution

8.1.1. Using an Exact Method8.1.2. Inverting a Tabulated Distribution8.1.3. Using a Mixed Empirical and Exponential Distribution8.1.4. Testing Robustness

8.2. Common Random Numbers8.3. Antithetic Variates8.4. Control Variates

8.4.1. The Simple Approach8.4.2. Regression With Splitting8.4.3. Regression With Jackknifing

8.5. Strat ified Sampling8.6. Importance Sampling8.7. Conditional Monte Carlo8.8. Summary

APPENDIX A

The Shapiro-Wilk Test for Normality

APPENDIX L

Routines for Random Number Generation

APPENDIX X

Examples of Simulation Programming

References

Author Index

Subject Index

xvii

271274275276276278279

281282282282284285287289291291293294295296299300

303

306

348

375

389

393

Notation

The following notations and abbreviations are used without further expla­nation throughout the book.

Section 1.2.3 : Chapter I, Section 2, Subsection 3.Problem 1.2.3 : Problem number 3 in Chapter I, Section 2(likewise for example, figure, lemma, proposition, table, and theorem).

log

log ,

rxlLxJg(n) = O(f(n))

g(n) = o(f(n))

oAlBic queue

FIFO

LIFO

=

natural logarithm

base 2 logarithm

ceiling of x = smallest integer ~ xfloor of x = largest integer ::;; xmeans that there is a finite constant M such that Ig(n) I ::;;Mlf(n)1 for all sufficiently large nmeans that g(n)lf(n) -+ 0 (as n -+ 0 or as n -+ 00 , dependingon the context)

the empty set

a queue with arrival distribution A, service time distributionB, and c servers (replacing A or B by M indicates that thecorresponding distribution is exponential)

the first-in, first-out queue discipline (sometimes calledFCFS: first-come, first-served)

the last-in, first-out queue discipline (sometimes calledLCFS).

convergence in distribution of random variables (and weakconvergence of probability measures)

xx

iid

cdfpdf

pmf

independent and identically distributed

cumulative distribution functionprobability density function

probability mass function

Notation

With a few exceptions we use capital letters to denote random variables,and corresponding lower-case letters to denote their values. Unless thecontext makes it clear otherwise, U denotes a random variable uniformlydistributed on (0,1).

Precedence Diagram

1.2 = Chapter 1, Section 23 ~ 4 = 3 is a prerequisite for 45 H 6 = 5 and 6 can be read profitably in parallel

rt>.\:::Y

..IIIIIII

~~

rz-.\::;J

r;7\\:Y

AIIJIIII

0088~I

II

II

II

cj