TEXTS AND READINGS IN PHYSICAL SCIENCES - 2

Numerical Methods

for Scientists and Engineers

Third Edition

Texts and Readings in Physical Sciences

Managing Editors H. S. Mani, Chennai Mathematical Institute, Chennai. hsmani@gmail.com

Ram Ramaswamy, Vice Chancellor, University of Hyderabad, Hyderabad. r.ramaswamy@gmail.com

Editors

Kedar Damle (fIFR, Mumbai) kedar@tifr.res.in Debashis Ghoshal GNU, New Delhi) dghoshal@mail.jnu.ac.in Rajaram Nityananda (NCRA, Pune) rajaram@ncra.tifr.res.in Gautam Menon (IMSc, Chennai) menon@imsc.res.in Tarun Souradeep (IUCAA, Pune) tarun@iucaa.ernet.in

Volumes published so far

1. Field Theories in Condensed Matter Physics, Sumathi Roo (Ed.) 2. Numerical Methods for Scientists and Engineers (3/E), H. M. Antia 3. Lectures on Quantum Mechanics (2/E), Ashok Oas 4. Lectures on Electromagnetism, Ashok Oas 5. Current Perspectives in High Energy Physics, Oebashis Ghoshal (Ed.) 6. Linear Algebra and Group Theory for Physicists (2/E), K. N. Srinivasa Roo 7. Nonlinear Dynamics: Near and Far from Equilibrium,] K. Bhattacharjee

and S. Bhattacharyya 8. Spacetime, Geometry and Gravitation, Pankcg Sharan 9. Lectures on Advanced Mathematical Methods for Physicists, Sunil Mukhi

and N. Mukunda 10. Computational Statistical Physics, Sitangshu Bikas Santra and

Purusattam Roy (Eds.) 11. The Physics of Disordered Systems, Gautam 1. Menon and

Purusattam Roy (Eds.)

Numerical Methods for Scientists and Engineers

Third Edition

H. M. Antia

~HINDUSTAN U LQJ U BOOK AGENCY

Published by

Hindustan Book Agency (India) P 19 Green Park Extension New Delhi 110016 India

email: info@hindbook.com www.hindbook.com

Copyright © 1991, First Edition, Tata Mcgraw-Hill Publishing Company Limited. Copyright © 2002, Second Edition, Hindustan Book Agency (India) Copyright © 2012, Hindustan Book Agency (India)

No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording or by any information storage and retrieval system, without written permission from the copyright owner, who has also the sole right to grant licences for translation into other languages and publication thereof.

All export rights for this edition vest exclusively with Hindustan Book Agency (India). Unauthorized export is a violation of Copyright Law and is subject to legal action.

ISBN 978-93-80250-40-3 ISBN 978-93-86279-52-1 (eBook) DOI 10.1007/978-93-86279-52-1

Texts and Readings in the Physical Sciences

The Texts and Readings in the Physical Sciences (TRiPS) series of books aims to provide a forum for physical scientists to describe their fields of research interest. Each book is intended to cover a subject in a detailed manner and from a personal viewpoint, so to give students in the field a unique exposure to the subject that is at once accessible, pedagogic, and contemporary. The monographs and texts that have appeared so far, as well as the volumes that we plan to bring out in the coming years, cover a range of areas at the core as well as at the frontiers of physics.

In addition to texts on a specific topic, the TRiPS series includes lecture notes from a thematic School or Workshop and topical volumes of contributed articles focussing on a currently active area of research. Through these various forms of exposition, we hope that the series will be valuable both for the beginning graduate student as well as the experienced researcher.

H.S. Mani Chennai

R. Ramaswamy Hyderabad

Contents

Preface

Preface to the second edition

Preface to the First Edition

Notation

List of Computer Programs

1 Introduction 1.1 Errors in Numerical Computation 1.2 Truncation Error 1.3 Programming

Bibliography Exercises

2 Roundoff Error 2.1 Number Representation 2.2 Roundoff Error ..... 2.3 2.4

Error Analysis . . . . . Condition and Stability Bibliography Exercises ..... .

3 Linear Algebraic Equations 3.1 Introduction ..... 3.2 3.3 3.4

3.5 3.6 3.7

Gaussian Elimination . . . . . . . Direct Triangular Decomposition Error Analysis . . . . . . . . . Matrix Inversion ...... . Singular Value Decomposition Iterative Methods Bibliography Exercises ....

xiii

xv

xix

xxiii

xxv

1 2 4 8

11 11

13 13 22 32 42 50 51

59 60 63 71 77 87 88 98

102 103

viii

4 Interpolation 4.1 Polynomial Interpolation . . . . . . . . . . 4.2 Divided Difference Interpolation Formula. 4.3 Hermite Interpolation. . . 4.4 Cubic Spline Interpolation . . . 4.5 B-splines . .. . . . . . . . . . . 4.6 Rational Function Interpolation 4.7 Interpolation in Two or More Dimensions 4.8 Spline Interpolation in Two or More Dimensions

Bibliography Exercises

5 Differentiation 5.1 Differentiation of Interpolating Polynomials 5.2 Method of Undetermined Coefficients 5.3 Extrapolation Method

Bibliography Exercises

6 Integration

7

6.1 Newton-Cotes Quadrature Formulae 6.2 Extrapolation Methods . 6.3 Gaussian Quadrature 6.4 Roundoff Error . . 6.5 Weight Function 6.6 Improper Integrals 6.7 Automatic Integration 6.8 Summation. . .... 6.9 Multiple Integrals . .. 6.10 Rules Exact for Monomials. 6.11 Monte Carlo Method .. . 6.12 Equidistributed Sequences

Bibliography Exercises .. . . . .. .

Nonlinear Algebraic Equations 7.1 Real Roots ... . . . . 7.2 Fixed-Point Iteration .. 7.3 Method of False Position 7.4 Secant Method ... . . 7.5 Newton-Raphson Method 7.6 Brent's Method 7.7 Complex Roots . ... . . 7.8 Muller 's Method ... . . 7.9 Quadrature Based Method

Contents

111 112 117 128 130 135 141 147 150 152 153

159 159 164 168 172 172

175 177 187 194 198 201 209 220 229 236 242 247 254 259 260

271 273 276 278 281 285 289 294 297 301

Contents

7.10 7.11 7.12 7.13 7.14 7.15 7.16 7.17

Real Roots of Polynomials Laguerre's Method .... Roundoff Error . . . . . . Criterion for Acceptance of a Root Ill-conditioning . . . . . . . . . System of Nonlinear Equations Newton's Method . Broyden's Method Bibliography Exercises

8 Optimisation 8.1 Golden Section Search 8.2 Brent's Method .... 8.3 Methods Using Derivative 8.4 Minimisation in Several Dimensions. 8.5 Quasi-Newton Methods. 8.6 Direction Set Methods 8.7 Linear Programming 8.8 Simulated Annealing

Bibliography Exercises ..

9 Statistical Inferences 9.1 Elementary statistics ............ . 9.2 Monte Carlo Methods ........... . 9.3 Experimental Errors and their Propagation

Bibliography Exercises ..... .

10 Functional Approximations 10.1 Choice of Norm and Model. 10.2 10.3 10.4 10.5 10.6 10.7 10.8 10.9 10.10 10.11 10.12 10.13

Linear Least Squares . . . . Nonlinear Least Squares .. Least Squares Approximation in Two Dimensions Discrete Fourier Transform. . . . Fast Fourier Transform . . . . . . FFT in Two or More Dimensions Inversion of Laplace Transform Pade Approximations . . . Chebyshev Expansions . . . . . Minimax Approximations ... Discrete Minimax Approximations L1-approximations Bibliography . . . . . . . . . . . .

ix

304 308 313 315 319 323 328 333 335 336

345 347 353 356 359 363 371 379 390 395 396

401 401 416 418 422 423

425 426 432 451 457 459 464 473 475 479 485 494 503 508 510

x

Exercises ....... .

11 Algebraic Eigenvalue Problem 11.1 Introduction .. . 11.2 Power Method ..... .

11.3 Inverse Iteration. . . . . 11.4 Eigenvalues of a Real Symmetric Matrix

11.5 The QL Algorithm ........... . 11.6 Reduction of a Matri~ to Hessenberg Form. 11.7 Lanczos Method. . . . . . . . . . . . . . . . 11.8 Q R Algorithm for a Real Hessenberg Matrix . 11.9 Roundoff Errors.

Bibliography

Exercises ....

12 Ordinary Differential Equations 12.1 Initial Value Problem ... 12.2 Stability of Numerical Integration Methods 12.3 Predictor-Corrector Methods

12.4 Runge-Kutta Methods .. . 12.5 Extrapolation Methods .. .

12.6 Stiff Differential Equations . 12.7 Boundary Value Problem. 12.8 Finite Difference Methods 12.9 Eigenvalue Problem .. . 12.10 Expansion Methods .. . 12.11 Some Special Techniques

Bibliography

Exercises

13 Integral Equations 13.1 Introduction ............... .

13.2 Fredholm Equations of the Second Kind

13.3 Expansion Methods .......... .

13.4 Eigenvalue Problem .......... .

13.5 Fredholm Equations of the First Kind 13.6 Inverse Problems ........... .

13.7 Volterra Equations of the Second Kind

13.8 Volterra Equations of the First Kind

Bibliography Exercises .............. .

Contents

511

523 524

529 534

541 549

553 557

558 564 566 567

573 574

578 587

600 607 610 618 626 633 640 642 647

648

657 658

661 669

672

676

680 685

692 695

696

Contents

14 Partial Differential Equations 14.1 Introduction ..... . 14.2 Diffusion Equation in Two Dimensions ..... 14.3 General Parabolic Equation in Two Dimensions 14.4 Parabolic Equations in Several Space Variables 14.5 Wave Equation in Two Dimensions 14.6 General Hyperbolic Equations ... . 14.7 Elliptic Equations .......... . 14.8 Successive Over-Relaxation Method. 14.9 Alternating Direction Method 14.10 Fourier Transform Method 14.11 Finite Element Methods

Bibliography Exercises

Appendix A Answers and Hints A.l Introduction ........ . A.2 Roundoff Error . . . . . . . A.3 Linear Algebraic Equations A.4 Interpolation A.5 Differentiation ....... . A.6 Integration....... .. . A.7 Nonlinear Algebraic Equations. A.8 Optimisation ....... . A.9 Statistical Inferences .... . A.lO Functional Approximations A.11 Algebraic Eigenvalue Problem A.12 Ordinary Differential Equations A.13 Integral Equations ..... . A.14 Partial Differential Equations

Bibliography . . . . . . . . .

Index

Xl

701 702 707 717 724 728 734 741 748 753 756 757 763 765

773 773 774 779 783 787 789 795 801 803 804 811 814 818 819 822

823

xii Contents

Appendices Band C are not included in the printed form but can be found only in the online material at http://www.tifr.res.in/ antia/nmse3.html along with the machine readable versions of the Fortran and C programs. For convenience the pages are numbered continuously following the Index. The material in the Appendices is also included in the printed Index with respective page numbers.

Appendix B Fortran Programs B.1 Introduction ...... . B.2 Roundoff Error . . . .. B.3 Linear Algebraic Equations B.4 Interpolation B.5 Differentiation ....... . B.6 Integration ......... . B.7 Nonlinear Algebraic Equations. B.8 Optimisation ....... . B.9 Statistical Inferences .... . B.10 Functional Approximations B.11 Algebraic Eigenvalue Problem B.12 Ordinary Differential Equations B.13 Integral Equations ..... . B .14 Partial Differential Equations

Bibliography ........ .

Appendix C C Programs C.1 Introduction ........ . C.2 Roundoff Error .. . .. . C.3 Linear Algebraic Equations C.4 Interpolation C.5 Differentiation ....... . C.6 Integration.......... C.7 Nonlinear Algebraic Equations. C.8 Optimisation ....... . C.9 Statistical Inferences .... . C.10 Functional Approximations C.11 Algebraic Eigenvalue Problem C.12 Ordinary Differential Equations C.13 Integral Equations ..... . C.14 Partial Differential Equations

Bibliography . . . . . . . . .

861 861 866 867 874 884 885 909 923 929 933 966 973 988 996

1004

1007 1007 1011 1012 1019 1029 1030 1053 1064 1071 1074 1107 1113 1128 1136 1144

Preface

During the last twenty years since the publication of the first edition of the book, the speed as well as memory of computers have increased by a few orders of magnitude. However, the precision with which the floating point operations are being handled on these computers has not improved. As a result, the relevance of roundoff errors in numerical computations has increased substantially. In this book, an attempt is made to demonstrate that with proper care the errors do not grow very fast with the size of computational problem. Thus the book is probably more relevant today than it was twenty years back. The main aim of this book has been to not only provide suitable algorithms for numerical computations, but also to explain their limitations.

Based on experience in teaching, some new topics have been added to the second edition. A notable omission in the second edition of the book was statistical analysis of data. This has been addressed by the addition of a new Chapter (Chapter 9) on Statistical Inference. This Chapter introduces basic statistical distributions, which are useful in the next Chapter on approxima­tions. In view of this addition, the initial part of Chapter 10 on approximation has also been edited and two new subsections on least squares fit when there are errors in both x and y, and on Maximum Likelihood methods have been added. In keeping with these additions some computer programs have also been added and a few have been modified.

Further, keeping in view convenience of readers the computer programs and related material in Appendices Band C are now made available online at http://www.tifr.res.in/ antia/nmse.html Although, the Appendices do not appear in the printed version, the corre­sponding page numbers are still included in the printed index and the table of contents. These page numbers refer to description of programs which is in­cluded in the Appendices. The programs themselves are obviously in separate files and there is no page number associated with them. The online material also includes some examples of usage and instructions for using these routines.

Although, these computer programs have been tested to some extent, it is not possible for an individual to test all these thoroughly and there are likely to be some bugs in these routines. A few bugs in the earlier versions have been corrected. Hence, before using these subprograms, the reader is urged to verify their correctness. Even if there are no "bugs" in the program, it may not neces-

XIV Preface

sarily give the correct result in all cases, since the algorithm used in writing the program itself may not be capable of solving all possible problems. It should be noted that these routines are not a substitute for high quality numerical soft­ware that is available. Apart from the mathematical subroutine libraries, there are also a number of software packages which allow the mathematical problems to be specified in a convenient form for numerical solution. Nevertheless, it is not possible for any program or software to give correct results for all possi­ble problems. Thus it is advisable to use only those software where the user is aware of which technique is actually implemented, so that their limitations may be known. The routines in the online material are also provided in the same spirit. They may not necessarily give the correct result in all cases, but since the algorithm used is known to the readers, they can modify these to suit their requirements.

Since the publication of the second edition, a few of the readers have pointed out misprints in the book and bugs in the computer programs. Besides a few misprints were detected during revision. These have been corrected, hope­fully, without addition of new bugs. I am thankful to Nirvikar Prasad, Lavish Pabbi and Nilesh Sutar for pointing out misprints. It is possible that I have forgotten some others who too pointed out the errors and I hope that they will accept my apology. Careful readers may notice that in some of the examples the numbers have changed as compared to previous edition. In almost all cases, these are not due to misprints or bugs, but because of using a different compiler and computer and the accompanying differences in roundoff errors. In some of the cases, the old tables have been retained even though the results using cur­rent machines were somewhat different. Apart from these a major change in the manuscript was from plain TEX to g\TEX format, which involved extensive editing of equations and tables. Hopefully, no errors have been introduced dur­ing this transformation! I am thankful to Ram Ramaswamy for encouragement and help in bringing out this third edition of the book.

August 2012 H. M. Antia

Preface to the Second Edition

During the last ten years since the publication of the first edition of the book, the speed as well as memory of computers have increased by almost two orders of magnitude. However, the precision with which the floating point operations are being handled on these computers has not improved. In fact, the 32/64-bit IEEE representation is now more or less standard on all computers, while some of the older machines used longer bit sequences to represent the same numbers. Although, many current compilers support quadruple precision arithmetic with 128 bit representation for floating point numbers, these are not implemented in hardware and consequently the execution is very slow. This imbalance is mainly due to the fact that nowadays the computers are being largely used for non-numerical work. As a result, the computer manufacturers do not have sufficient incentive to provide more accurate arithmetic operations. On the other hand, because of availability of faster computers and larger memory, the scientists and engineers are now handling larger computational problems, using arithmetic with comparatively low precision. Clearly, more effort is now required to control the errors in computation. In this book, an attempt is made to demonstrate that with proper care the errors do not grow very fast with the size of computational problem. Thus the book is probably more relevant today than it was ten years back. The main aim of this book has been to not only provide suitable algorithms for numerical computations, but also to explain their limitations.

In keeping with the increase in computing power and memory some topics requiring these increased power have been added in the second edition. This includes interpolation and approximations in multiple dimensions, simulated annealing and the inverse problems. A notable omission in the first edition was the treatment of B-splines, which provide a convenient set of basis flUlctions for approximations. This has been added in this edition. Similarly, with increas­ing popularity of C language for computer programming, this edition includes computer programs in both Fortran and C. During the last ten years a new ver­sion of Fortran, namely, Fortran 90 has emerged, but the subroutines described in Appendix B, still use Fortran 77 as the corresponding compilers are more readily available. A number of new programs have been added bringing the total number to over two hundred. Most of the new additions involve B-splines or approximations in multiple dimensions. Besides a number of functions to

xvi Preface to the Second Edition

calculate some of the common mathematical functions have been added. It is obviously more useful to have programs in a machine readable form, rather than the printed form as was the case in the first edition. Thus, in this edition the computer programs in Fortran and C which are described in Appendices B and C, respectively, have been shifted to a CD. To reduce the size of the book, these Appendices themselves are also available only on the CD. Although, the Appendices do not appear in the printed version, the corresponding page num­bers are still included in the printed Index and the table of contents. These page numbers refer to description of programs which is included in the Appendices. The programs themselves are obviously in separate files and there is no page number associated with them. The CD also includes some examples of usage and instructions for using these routines.

Although, these computer programs have been tested to some extent, it is not possible for an individual to test all these thoroughly and there are likely to be some bugs in these routines. A few bugs in the earlier versions have been corrected. Hence, before using these subprograms, the reader is urged to verify their correctness. Even if there are no "bugs" in the program, it may not neces­sarily give the correct result in all cases, since the algorithm used in writing the program itself may not be capable of solving all possible problems. It should be noted that these routines are not a substitute for high quality numerical software that is available. Apart from the mathematical subroutine libraries, there are also a number of software packages which allow the mathematical problems to be specified in a convenient form for numerical solution. My expe­rience with these softwares has not been particularly good as quite often they produce incorrect results without any warning. Thus it is advisable to use only those softwares where the user is aware of which technique is actually imple­mented, so that their limitations may be known. The routines in the attached CD are also provided in the same spirit. They may not necessarily give the correct result in all cases, but since the algorithm used is known to the readers, they can modify these to suit their requirements.

During the last ten years, significant advances have taken place in tech­niques for parallel processing and for solution of partial differential equations, but these are not reflected in this edition as they were thought to be beyond the scope of a general book. The algorithms presented in this book are gener­ally meant for use on computers with sequential processing, though in many cases it may be possible to implement an equivalent algorithm on machines with parallel processing.

Since the publication of the first edition, a few of the readers have pointed out misprints in the book and bugs in the computer programs. Besides a number of misprints were detected during revision. Most of the misprints were discov­ered in answers to exercises. These have been corrected, hopefully, without addition of new bugs. I am thankful to S. Basu, R. S. Bhalerao, A. K. Mohap­atra and B. Pal for pointing out misprints and bugs. It is possible that I have forgotten some others who too pointed out the errors and I hope that they will accept my apology. Careful readers may notice that in some of the examples

xvii

the numbers have changed as compared to previous edition. In almost all cases, these are not due to misprints or bugs, but because of using a different compiler and computer and the accompanying differences in roundoff errors. In some of the approximation problems the difference arises merely because a different set of random numbers was used. In some of the cases, the old tables have been retained even though the results using current machines were somewhat differ­ent. I am thankful to S. M. Chitre, M. S. Raghunathan and R. Ramaswamy for encouragement and help in bringing out this second edition of the book.

November 2001 H. M. Antia

Preface to the First Edition

This book has evolved out of lecture courses delivered to Research Scholars at the Tata Institute of Fundamental Research during the period 1978- 88. The book attempts to provide a practical guide for numerical solution of mathe­matical problems on digital computers. It is intended for both students and research workers in science and engineering. Readers are expected to have a reasonable background in mathematics as well as computer programming.

The enormous variety of problems that can be encountered in practice makes numerical computation an art to some extent and as such the best way of learning it is to practice it. Trying to learn numerical methods by just attending lectures or by just reading a book is like trying to learn how to play football by the same method. All students are therefore urged to tryout the methods themselves on any problem that they may fancy, not necessarily the ones given in this book. In fact, the reader will probably not appreciate the contents, unless a serious attempt is made to use numerical methods to solve problems on computers.

There is a wide difference between numerical methods for computers and those for desk calculators. The enormous increase in speed, enables us to solve much larger problems on computers and frequently such problems are more sensitive to various errors during calculations. Apart from the tremendous in­crease in speed, the main qualitative difference is that in computation with desk calculators the intermediate results are always available whether we want them or not. Hence, in most cases, it is rather straightforward to detect the numerical errors creeping in the computation and the corrective action can be taken easily. Further, any unexpected result can also be tackled appropriately. But in computation with modern computers, the intermediate results are usu­ally not available and consequently all possibilities have to be considered while writing the programs. Error propagation in the computation should also be es­timated beforehand. Otherwise, the error accumulation could render a plausible appearing set of results completely meaningless and no indication of this diffi­culty may be present in the final results. This makes the task of computation conceptually more difficult, even though the process itself is far more efficient. In particular, the error estimation is far more important with digital computers than it was with desk calculators. As a result, a detailed discussion of errors and pitfalls in numerical calculations has been included in this book, even though

xx Preface to the First Edition

rigorous proofs may not always be given. An attempt is also made to illustrate the effect of errors in numerical computations by giving appropriate examples, rather than giving detailed theory of error propagation.

Keeping in view the intended readership, detailed derivations and proofs of numerical methods and their properties have been avoided. As far as possible, a sufficient discussion of basic assumptions behind each numerical method and its limitations are provided. In most cases, the numerical methods are presented in a nonalgorithmic form. It was felt that the algorithmic form may give a very rigid picture of these methods, while in actual practice, considerable flexibility is required to tailor these methods for effective practical implementation. Con­sidering the wide variety of possible problems, no algorithm can successfully solve the entire class of problems that may arise in practice. For example, no single algorithm can integrate all possible functions that we can think of, even if it is restricted to the class of continuous and bounded functions over finite intervals. In fact, given a fixed algorithm, we can construct examples where the algorithm will fail miserably. On the other hand, given a Riemann integrable function, we can evaluate the integral using some algorithm based on the trape­zoidal rule, provided "sufficient" computer time is available. Nevertheless, over one hundred Fortran subprograms implementing some of the numerical meth­ods are provided in the Appendix B. These subprograms may serve to fill the gaps left in description of the algorithm and can also be useful for solving prob­lems. While an attempt has been made to test these subprograms, it is not possible to guarantee their correctness. Hence, before using these subprograms, the reader is urged to verify their correctness. Even if there are no "bugs" in the program, it may not necessarily give the correct result in all cases, since the algorithm used in writing the program itself may not be capable of solving all possible problems.

In many cases, more than one method for solving the same class of prob­lems is described. Although, there is some discussion about the merits of each of these methods, it is not generally possible to give algorithmic description for the choice of the best method for a given problem. Numerous examples are included in the text to illustrate the working of numerical methods, as well as to bring out the pitfalls in these methods. In addition, a number of exercises are given at the end of each chapter to help the students in improving their understanding of the material covered in the chapter. Short answers or hints to most of the exercises are given in Appendix A. In case of numerical problems, the answer should generally be correct to all figures specified there. Once again, it is almost impossible to guarantee the correctness of all answers, and read­ers should be prepared for some misprints and blunders. The exercises are of varying difficulty and type: Some of the exercises are simple proofs or deriva­tions of material covered in the text, while others provide results which are extension of subject matter considered in the text. Apart from these, there are computational problems, some of which are expected to expose the limitations of numerical methods.

XXI

This book is intended to cover all elementary topics in numerical compu­tations. However, it is difficult to fulfill this intention in actual practice and readers may find some of their favourite topics missing. In fact, the topics of each of the chapters (and some of the sections) have themselves been the sub­jects of a full-fledged book. Consequently, in each chapter, the emphasis is on simple problems where the intricacies of numerical methods can be easily un­derstood and demonstrated. For more advanced topics, it may be necessary to consult specialised books, which have been listed in the Bibliography at the end of each chapter. This Bibliography is by no means exhaustive, but is only intended to guide the readers in extending their knowledge. The choice of top­ics included in this book is obviously biased by my experience. A glance at the table of contents will reveal the subject matter covered in the text.

The first two chapters give an introduction to errors in numerical com­putations. Different types of errors, as well as techniques for estimating them are described in these chapters. Subsequent chapters discuss numerical meth­ods for various mathematical problems. Attempt has been made to make each of these chapters self-contained as far as possible, so that readers with some background in numerical methods can directly go through the concerned chap­ter, preferably after reading the first two chapters. However, certain amount of back referencing is unavoidable. In particular, Chapters 9-13, which deal with more advanced topics require some material from the previous chapters.

It may not be possible to cover the entire material included in this book in any course, but by leaving out appropriate portions it should be possible to use the book effectively. The choice of topics which can be excluded or included will naturally depend on the experience and taste of the teacher. The book is intended to cover most of the important topics in numerical methods, so that it may be useful also as a reference book for research workers and engineers.

The material in this book has been drawn from a variety of sources, for which I am indebted to many numerical analysts. In particular, the in­fluence of the books by A. Ralston and P. Rabinowitz, and J. H. Wilkinson, should be clear to anyone who is familiar with them. It was after some per­suasion from B. M. Udgaonkar that I undertook the task of writing this book. I am thankful to him for constant encouragement during the preparation of this book, and seeing the book through to its final stage. I am thankful to B. Banerjee, S. M. Chitre, M. H. Gokhale, P. S. Joag, D. Narasimha, T. Padmanabhan and R. K. Singh for going through and commenting on parts of the manuscript. The preparation of a book of this size has undoubtedly placed a strain on the forbearance of both my colleagues in the Astrophysics Group and my mother, and I am thankful to them for their patient coop­eration. I am also thankful to the Tata Institute of Fundamental Research for allowing me a generous use of the facilities for preparing the book. Fi­nally, I wish to thank D. E. Knuth for his excellent computer typesetting lan­guage TEX., without which I would not have thought of writing this book.

H. M. Antia

Notation

The table below is a short list of symbols and notation used in this book. More detailed explanation, if necessary, may be found at the first use of the symbol.

Page First Meaning Example Used

Reference to Exercises at the end of Chapter {9} 19 {1O.2} 46

Reference to Bibliography at the end of Chapter Hamming (1973) 1

Ceiling operation Ixl 36 Closed interval [a, b] 112 Complex conjugate z* 62 Derivatives f'(x), f"(x) 57

fCkl(x) 114 Factorial n! 11 Floor operation lxJ 39 Identity matrix I 45 Kronecker's delta Dij 81 Open interval (a , b) 31 Order of magnitude O(h2) 5 Vectors - boldface letters x 44

List of Computer Programs

CASSUM ROUND

GAUELM MATINV CROUT CROUTH CHOLSK

GAUBND

SVD SVDEVL

DIVDIF

DIVDIFO NEARST SPLINE SPLEVL SMOOTH BSPLIN BSPINT BSPEVL

RATNAL POLY2 LINRN LOCATE BSPINT2

Chapter 2 Roundoff Error

Cascade sum of a finite series Round a floating-point number to a specified number of digits

Chapter 3 Linear Algebraic Equations

Solve a system of linear equations using Gaussian elimination Calculate inverse of a square matrix using Gaussian elimination Solve a system of linear equations using Crout's algorithm Iterative refinement of solution of a system of linear equations Solve a system of linear equations with symmetric positive defi­nite matrix using Cholesky's decomposition Solve a system of linear equations with a band matrix using Gaussian elimination with partial pivoting Singular value decomposition of a matrix Solve a system of linear equations using singular value decompo­sition

Chapter 4 Interpolation

Calculate interpolation and its derivatives using divided differ­ence formula Divided difference interpolation formula (no derivatives version) Find nearest point in an ordered table using bisection Calculate coefficients of interpolating cubic spline Evaluate the cubic spline and its derivatives at a specified point Draw a smooth curve through a set of points using cubic spline Calculate B-spline basis functions on a set of knots Calculate coefficients of B-spline interpolation Evaluate function value and its derivatives using B-spline expan­sion Calculate rational function interpolation Calculate polynomial interpolation in two dimensions Calculate linear interpolation in n dimensions Find the bracketing subinterval in an ordered table Calculate coefficients of B-spline interpolation in two dimensions

xxvi

BSPEV2

BSPINTN BSPEVN BSPEVNl

BSPEVN2

DRVT

SIMSON SPLINT BSPQD ROMBRG EPSILN GAUSS GAUCBY

GAUCBl

GAUCB2

GAUSQ2

GAUSQ

GAULAG

LAGURE HERMIT GAULG2

GAULOG

List of Computer Programs

Evaluate function value and derivatives using B-spline expansion in two dimensions Calculate coefficients of B-spline interpolation in n dimensions Evaluate function value using B-spline expansion in n dimensions Evaluate function value and first derivative using B-spline ex­pansion in n dimensions Evaluate function value and first and second derivatives using B-spline expansion in n dimensions

Chapter 5 Differentiation

Differentiation using h ---+ ° extrapolation

Chapter 6 Integration

Integration using Simpson's 1/3 rule Integrate a tabulated function using cubic spline Integrate a B-spline expansion Romberg integration Integration using E-algorithm Integration using Gauss-Legendre formula Integration using Gauss-Chebyshev formula with weight func-tion,

w(x) = 1/ J(x - A)(B - x)

Integration using Gauss-Chebyshev formula with weight func-tion,

w(x) = J(x - A)/(B - x)

Integration using Gauss-Chebyshev formula with weight func-tion,

w(x) = J(x - A)(B - x)

Integration over (0, A] with square root singularity using a com­bination of Gaussian formulae Integration over (0, A] using Gaussian formula with weight func-tion,

w(x) = I/VX Integration over [A, 00) using a combination of Gaussian formu­lae Integration over [A, 00) using Gauss-Laguerre formula Integration over (-00,00) using Gauss-Hermite formula Integration over (0, A] with logarithmic singularity using a com­bination of Gaussian formulae Integration over (0, A] using Gaussian formula with weight func­tion,

w(x) = In(A/x)

GAUSRC

GAULEG

GAUJAC

LAGURW

GAUHER

GAUSWT

FILON ADPINT KRONRD GAUS16

CAUCHY EULER BSPQD2 BSPQDN MULINT

NGAUSS SPHND

STRINT

STROUD MCARLO RANI

RANF

EQUIDS

BISECT SECANT SECANC SECANI

NEWRAP BRENT SEARCH

XXVll

Calculate weights and abscissas of Gaussian formula using recur­rence relation of orthogonal polynomials Calculate weights and abscissas of Gauss-Legendre quadrature formulae Calculate weights and abscissas of Gauss-Jacobi quadrature for­mulae Calculate weights and abscissas of Gauss-Laguerre quadrature formulae Calculate weights and abscissas of Gauss-Hermite quadrature formulae Calculate weights and abscissas of Gaussian formula using mo­ments of weight function Integration of an oscillatory function using Filon's formula Adaptive integration over a finite interval Integration using Gauss-Kronrod formula for use with AD PINT Integration using 16 point Gauss-Legendre formula for use with ADPINT Calculate Cauchy principal value of an integral Summation of alternating series using Euler transformation Integrate a B-spline expansion in two dimensions Integrate a B-spline expansion in n dimensions Multiple integration using product Gauss rule with varying num­ber of points Multiple integration using a specified product Gauss rule To convert from hyper-spherical coordinates to Cartesian coor­dinates Multiple integration using monomial rules with varying number of points Multiple integration using a specified monomial rule Multiple integration using Monte Carlo method Generate a sequence of random numbers with uniform distribu­tion Generate a sequence of random numbers with uniform distribu­tion Multiple integration using equidistributed sequences

Chapter 7 Nonlinear Algebraic Equations

Solve a nonlinear equation using bisection Solve a nonlinear equation using secant iteration Complex roots of a nonlinear equation using secant iteration Solve a nonlinear equation using secant iteration (with reverse communication) Solve a nonlinear equation using Newton-Raphson method Solve a nonlinear equation using Brent's method Locate complex zeros by looking for sign changes

XXVlll

ZROOT ZROOT2

MULLER MULER2

DELVES

CONTUR NEWRAC

POLYR LAGITR

POLYC LAGITC

DAVIDN NEWTON BROYDN

BRACKM GOLDEN BRENTM DAVIDM BFGS

LINMIN FLNM

NMINF LINMNF FLN

SIMP LX SIMPX

SHSORT GAMMAP BETAP BETSER

List of Computer Programs

Complex roots of a nonlinear equation with deflation Complex roots of a nonlinear equation with deflation, function value in scaled form, f(x) x 2i (x)

Complex root using Muller's method Complex root using Muller's method with function value in a scaled form, f(x) x 2i (x)

Complex zeros of an analytic function using quadrature based method Contour integration over a circular contour for DELVES Complex root of a nonlinear equation using Newton-Raphson method All roots of a polynomial with real coefficients One root of a polynomial with real coefficients using Laguerre's method All roots of a polynomial with complex coefficients One root of a polynomial with complex coefficients using La­guerre's method Solve a system of nonlinear equations using Davidenko's method Solve a system of nonlinear equations using Newton's method Solve a system of nonlinear equations using Broyden's method

Chapter 8 Optimisation

Bracketing a minimum in one dimension Minimisation in one dimension using golden section search Minimisation in one dimension using Brent's method Minimisation in one dimension using cubic Hermite interpolation Minimisation in n dimensions using quasi-Newton method with BFGS formula Line search for quasi-Newton method Calculate the function value for line search for quasi-Newton method Minimisation in n dimensions using direction set method Line search for direction set method Calculate the function value for line search for direction set method Solving a linear programming problem using simplex method Simplex method for a linear programming problem in the stan­dard form

Chapter 9 Statistical Inferences

Sorting an array in ascending order using shell sort algorithm Calculate incomplete Gamma function Calculate incomplete Beta function Calculate incomplete Beta function using a power series approx­imation

BETCONI

BETCON

BETAI

FBETA RANGAU

IRANBIN

IRANPOI

PCOR

POLFIT POLEVL

POLFITI

POLORT

POLFIT2

POLEV2

POLFITN

POLEVN

POLEVNl

POLEVN2

LLSQ

BSPFIT BSPFIT2

BSPFITW2

BSPFITN

XXIX

Calculate incomplete Beta function using a continued fraction approximation Calculate incomplete Beta function using an alternative contin­ued fraction approximation Calculate incomplete Beta function by directly evaluating the integral Calculate the integrand for BETAI Generate a sequence of random numbers with Gaussian distri­bution Generate a sequence of random numbers with binomial distribu­tion Generate a sequence of random numbers with Poisson distribu­tion Calculate the probability that two uncorrelated sequences will give a correlation coefficient exceeding a given value

Chapter 10 Functional Approximations

Least squares polynomial fit using orthogonal polynomials Evaluate the fitted polynomial and its derivatives at a specified point Least squares polynomial fit using orthogonal polynomials, sim­plified version for multiple data sets Evaluate the orthogonal polynomial basis functions at a given point Least squares polynomial fit using orthogonal polynomials in two dimensions Evaluate the fitted polynomial and its derivatives at a specified point in two dimensions Least squares polynomial fit using orthogonal polynomials in n dimensions Evaluate the fitted polynomial at a specified point in n dimen­sions Evaluate the fitted polynomial and first derivative at a specified point in n dimensions Evaluate the fitted polynomial and first and second derivatives at a specified point in n dimensions Linear least squares fit in n dimensions to a user defined set of basis functions Least squares fit to B-spline basis functions in one dimension Least squares fit to B-spline basis in two dimensions with equal weights Least squares fit to B-spline basis in two dimensions with arbi­trary weights Least squares fit to B-spline basis in n dimensions with equal weights

xxx

BSPFITWN

LINFITXY

NLLSQ DFT

FFT FFTR FFTN LAPINV POLD RMK RMKI

RMKD RMKDl

PADE CREBCF CREBEX CREBAP REMES

FM

GAMMA GAMMAL

ERF ERFC BJO BJI BJN BYO BJYO BYI BJYl BYN SPRBJN BIO

BIl BIN

List of Computer Programs

Least squares fit to B-spline basis in n dimensions with arbitrary weights Least squares straight line fit when there are errors in both x and y values Calculate the X2 function for a nonlinear least squares fit Discrete Fourier transform of complex data with arbitrary num­ber of points Fast Fourier transform of complex data Fast Fourier transform of real data Fast Fourier transform of complex data in n dimensions Inverse Laplace transform Evaluate a polynomial and its derivative at any point Evaluate a rational function at any point Evaluate a rational function at any point (constant term in de­nominator 1) Evaluate a rational function and its derivative at any point Evaluate a rational function and its derivative at any point (con­stant term in denominator 1) Calculate coefficients of Pade approximations Convert from power series to Chebyshev expansion and vice versa Calculate the coefficients of Chebyshev expansion Rational function approximation using Chebyshev polynomials Minimax approximation to mathematical functions using Remes algorithm Calculate error in rational function approximation for use with REMES Calculate Gamma function at real x, r(x) Calculate natural logarithm of Gamma function at real x, In lr(x)1 Calculate Error function at real x Calculate complementary Error function at real x Calculate Bessel function of first kind of order zero, Jo(x) Calculate Bessel function of first kind of order one, Jdx) Calculate Bessel function of first kind of integral order, In(x) Calculate Bessel function of second kind of order zero, Yo(x) Calculate Bessel function of first and second kind of order zero Calculate Bessel function of second kind of order one, Y1 (x) Calculate Bessel function of first and second kind of order one Calculate Bessel function of second kind of integral order, Yn(x) Calculate spherical Bessel function of integral order, jn (x) Calculate modified Bessel function of first kind of order zero, 10 (x) Calculate modified Bessel function of first kind of order one, II (x) Calculate modified Bessel function of first kind of integral order, 1n(x)

BKO

BKI

BKN

DAWSON FERMM05 FERM05 FERM15 FERM25 PLEG PLM YLM MINMAX POLYLI LINLI

SIMPL1

INVIT TRED2

TRBAK

TQL2

TRIDIA

STURM

TINVIT

HEREVP BALANC BALBAK

EL!vlHES

HQR

RKM

XXXI

Calculate modified Bessel function of second kind of order zero, Ko(x) Calculate modified Bessel function of second kind of order one, KI(X) Calculate modified Bessel function of second kind of integral or­der, Kn(x) Calculate the value of Dawson's integral Calculate the Fermi integrals for k = -1/2 Calculate the Fermi integrals for k = 1/2 Calculate the Fermi integrals for k = 3/2 Calculate the Fermi integrals for k = 5/2 Calculate the Legendre polynomial, Pc (x) Calculate the associated Legendre function, pr(x) Calculate the spherical harmonic, yr ((), ¢) Rational function minimax approximation to discrete data Polynomial L1-approximation to discrete data Linear L1-approximation to discrete data for arbitrary basis functions Modified simplex method for LP problems in Ll-approximation

Chapter 11 Algebraic Eigenvalue Problem

Eigenvalue and eigenvector using inverse iteration Reduction of a real symmetric matrix to symmetric tridiagonal form using Householder transformations Back-transform eigenvectors of tridiagonal matrix to original ma­trix Eigenvalue problem for a symmetric tridiagonal matrix using QL­algorithm Specified eigenvalues and eigenvectors of a symmetric tridiagonal matrix using Sturm sequence and inverse iteration Locate eigenvalues of a symmetric tridiagonal matrix using Sturm sequence Eigenvalue and eigenvector of a symmetric tridiagonal matrix using inverse iteration Eigenvalue problem for a complex Hermitian matrix Balancing a general real matrix Back-transform eigenvectors of balanced matrix to original ma­trix Reduce a real matrix to Hessenberg form using Gaussian elimi­nation Eigenvalues of a Hessenberg matrix using QR-algorithm

Chapter 12 Ordinary Differential Equations

Initial value problem using fourth-order Runge-Kutta method with adaptive step size

XXXll

RK4 RK2 MSTEP

ADAMS STRT4 GEAR EXTP FDM

GEVP

GAUBLK

SETMAT

BSPODE

FRED FREDCO FUNK

RLS

FORW VOLT VOLT2

CRANK

LINES ADM

LAX SOR

AD!

List of Computer Programs

One step of integration using fourth-order Runge-Kutta method One step of integration using second-order Runge-Kutta method Initial value problem using predictor-corrector method with adaptive step size One step of integration using fourth-order Adams method Starting values for multistep method using Runge-Kutta method One step of integration using fourth-order stiffly stable method Initial value problem using extrapolation method Two-point boundary value problems using Finite Difference Method Eigenvalue problem in differential equations using finite differ­ences Solve a system of linear equations involving finite difference ma­trix Generate finite difference matrix for a system of differential equa­tions Two-point boundary value problem using expansion method with B-spline basis functions

Chapter 13 Integral Equations

Solve a Fredholm equation using quadrature method Solve a Fredholm equation using collocation method Integrand = K(x, t)¢j(t), for evaluating integrals in collocation method Solve a linear inverse problem using regularised least squares technique Solve the forward problem Solve a linear Volterra equation using trapezoidal rule Solve a nonlinear Volterra equation of the second kind using Simpson's rule

Chapter 14 Partial Differential Equations

Linear second-order parabolic equation using Crank-Nicolson method Nonlinear parabolic equations using the method of lines Parabolic equation in two space variables using alternating di­rection method Nonlinear hyperbolic equations using the Lax-Wendroff method Linear second-order elliptic equations using the successive over­relaxation (SOR) method Linear second-order elliptic equations using the alternating di­rection implicit iterative (AD!) method