a brief history of numerical libraries
TRANSCRIPT
![Page 1: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/1.jpg)
A Brief History of Numerical Libraries
Sven HammarlingNAG Ltd, Oxford
&University of Manchester
![Page 2: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/2.jpg)
First – Something about Jack
![Page 3: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/3.jpg)
Jack’s thesis(August 1980)
30 years ago!
![Page 4: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/4.jpg)
TOMSAlgorithm589
![Page 5: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/5.jpg)
![Page 6: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/6.jpg)
![Page 7: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/7.jpg)
![Page 8: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/8.jpg)
![Page 9: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/9.jpg)
![Page 10: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/10.jpg)
![Page 11: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/11.jpg)
Small Selection of Jack’s Projects
• Netlib and other software repositories • NA Digest and na-net• PVM and MPI• TOP 500 and computer benchmarking• NetSolve and other distributed computing
projects• Numerical linear algebra
![Page 12: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/12.jpg)
![Page 13: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/13.jpg)
Onto the Rest of the Talk!
![Page 14: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/14.jpg)
Rough Outline• History and influences• Fortran• Floating Point Arithmetic• Libraries and packages• Proceedings and Books• Summary
![Page 15: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/15.jpg)
Ada Lovelace(Countess Lovelace)
Born Augusta AdaByron
1815 – 1852
The language Adawas named after her
![Page 16: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/16.jpg)
“Is thy face like thy mother’s, my fair child!Ada! sole daughter of my house and of my heart?When last I saw thy young blue eyes they smiled,And then we parted,-not as now we part,but with a hope”
Childe Harold’s Pilgramage,Lord Byron
![Page 17: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/17.jpg)
![Page 18: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/18.jpg)
Program for the Bernoulli Numbers
![Page 19: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/19.jpg)
Manchester Baby, 21 June 1948 (Replica)
19
![Page 20: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/20.jpg)
Kilburn/TootillProgram to compute the highest proper factor
took 52 minutes1.5 million instructions3.5 million store accesses
182
182
![Page 21: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/21.jpg)
First published numerical library, 1951
First use of the word subroutine?
![Page 22: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/22.jpg)
![Page 23: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/23.jpg)
Quality Numerical Software• Should be:
– Numerically stable, with measures of quality of solution
– Reliable and robust– Accompanied by test software– Useful and user friendly with example programs– Fully documented– Portable– Efficient
![Page 24: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/24.jpg)
“I have little doubt that about 80 per cent. of all the results printed from the computer are in error to a much greater extent than the user would believe, ...''
Leslie Fox, IMA Bulletin, 1971
![Page 25: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/25.jpg)
“Giving business people spreadsheets is like giving children circular saws.
The average spreadsheet programmer does little or no advance planning, has no idea whether his or her algorithms are correct … builds in few or no cross-checks, and does little or no testing.
It is a cast iron certainty that the vast majority of spreadsheets contain errors”
Jack Schofield, The Guardian, 2003
![Page 26: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/26.jpg)
N. J. Higham. Accuracy and Stability of Numerical Algorithms. SIAM, Philadelphia, PA, USA,second edition, 2002.
![Page 27: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/27.jpg)
“Since the use of the punched-card equipment required the use of an operator, it encouraged user participation generally, and this was a distinctive feature of Pilot ACE operation…Speaking for myself I gained a great deal of experience from user participation, and it was this that led to my own conversion to backward error analysis.”Wilkinson (1980) in A History of Computing in the Twentieth Century, Academic Press
![Page 28: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/28.jpg)
John Backus 1924 - 2007
![Page 29: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/29.jpg)
![Page 30: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/30.jpg)
Fortran
• Fortran lives on – now Fortran 2003• Fortran 2008 under discussion
• What happened to poor Ada?
![Page 31: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/31.jpg)
Portability• For portability, needed a model of floating point
arithmetic• IFIP/WG 2.5. B Ford et al. See:
http://www.nsc.liu.se/~boein/ifip/projects/p1.txt• W S Brown A simple but realistic model of
floating-point computation ACM Trans. Math. Software,7, 445–480, 1981
• NAG: Chapter X02• LAPACK: xLAMCH
![Page 32: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/32.jpg)
IEEE Arithmetic Formats
38 8
308 16
4932 20
Format Pr ecision Exponent Approx Range Approx precision ( )Single 24 bits 8 bits 10 10Double 53 bits 11 bits 10 10Extended 64 15 10 10
u± −
± −
± −≥ ≥
W. Kahan's self-portrait
![Page 33: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/33.jpg)
T. J. Dekker, W. Hoffmann; Algol 60 procedures in numerical algebra part 2; MC Tracts 23, Mathematisch Centrum, Amsterdam (1968)
![Page 34: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/34.jpg)
B. S. Garbow, J.M. Boyle, J. J. Dongarra, and C. B.Moler. Matrix Eigensystem Routines – EISPACKGuide Extension. Springer-Verlag, Berlin, Germany, 1977.
B. T. Smith, J.M. Boyle, B. S. Garbow, Y. Ikebe, V. C. Klema, and C. B.Moler. Matrix EigensystemRoutines – EISPACK Guide. Springer-Verlag, Berlin, Germany, 1974.
1976
![Page 35: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/35.jpg)
NAG and IMSL
• The NAG Library and IMSL were both released in 1971
• NAG was a UK University project, started in 1970, but became a not for profit company in 1976
• IMSL was commercial from the outset and is now owned by Rouge Wave
• Both Libraries have comprehensive numerical and statistical routines
![Page 36: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/36.jpg)
Others
• Other long standing libraries are:– BCSLIB from Boeing, has its origins in the
mid-1960s– HSL from Harwell (now developed and
supported by RAL) began in 1963– SLATEC, an open source collection, has its
origins in 1974. Released in 1981?
![Page 37: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/37.jpg)
NAG 40th
• NAG is celebrating 40 years on May 13th
![Page 38: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/38.jpg)
Basic Linear Algebra Subprograms
• Level 1 BLAS, 1979• Level 2 BLAS, 1988
– Vector machines• Level 3 BLAS, 1990
– Hierarchical memory, shared memory parallel• Dates are for TOMS publication
![Page 39: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/39.jpg)
![Page 40: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/40.jpg)
![Page 41: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/41.jpg)
![Page 42: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/42.jpg)
42
![Page 43: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/43.jpg)
Discussion of linear equation solvers on the Pilot ACE
“An interesting feature of the codes is that they made a very intensive use of subroutines; the addition of two vectors, multiplication of a vector by a scalar, inner products, etc., were all coded this way.”
Wilkinson, 1980The History of Computing in the 20th Century.
![Page 44: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/44.jpg)
Efficient Use of Data
“Since all machines have stores of finite size often divided up into high speed and auxiliary sections, storage considerations often have a vitally important part to play.”
Wilkinson, MTAC, 1955
![Page 45: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/45.jpg)
![Page 46: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/46.jpg)
![Page 47: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/47.jpg)
![Page 48: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/48.jpg)
Emphasis on error and condition estimates, as well as efficiency
![Page 49: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/49.jpg)
![Page 50: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/50.jpg)
![Page 51: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/51.jpg)
![Page 52: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/52.jpg)
Software Code Sizes
• NAG Fortran Library, Mark 21– Source: 28.3 Mb– Stringent test programs: 42.1 Mb– Example programs: 4.4 Mb– XML documentation: 136 Mb
• LAPACK 3.0– Source: 12.1 Mb– Testing: 10.9 Mb– Timing: 6.5 Mb– Users’ Guide: 407 pages
![Page 53: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/53.jpg)
Measures of Solution Quality
( )
DGESVX is an 'expert' driver for solving , , , , , ,Estimate of 1/Estimated forward error for
Componentwise relative backward error for j
AX B
AX
X
κ
=DGESVX( RCOND FERR BERR WORK INFO)
RCOND :
FERR(j):
BERR(j):
(smallest relative change in any element of and that makes an exact solution)
Reciprocal of pivot growth factor (1 )0 if is singular or nearly singular
j
j
A BX
gA>
WORK(1):
INFO :
![Page 54: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/54.jpg)
Proceedings of IMA Conference, Loughborough, 1973
Published 1974
![Page 55: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/55.jpg)
Proceedings of IMA Conference, Sussex 1977
Published 1978
![Page 56: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/56.jpg)
Proceedings of IFIP Conference, Boulder 1981
Published 1982
Included “Floating-point Parameters, Models and Standards by Jim Cody
![Page 57: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/57.jpg)
Jim Cody in 1969
Sadly passed away on June 24th, 2009
![Page 58: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/58.jpg)
State of Libraries and Packages in 1984
![Page 59: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/59.jpg)
Articles on:
• Observations on Mathematical Software Effort (Cody)
• LINPACK (Dongarra, Stewart)• FUNPACK (Cody)• EISPACK (Dongarra, Moler)• MINPACK (Moré, Sorensen, Garbow, Hillstrom)• Software for ODEs (Shampine, Watts)
59
![Page 60: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/60.jpg)
... (cont’d)• Sources of Information on Quadrature
Software (Kahaner)• A Survey of Sparse Matrix Software (Duff)
– included discussion of various sparse matrix packages, including SPARSPACK, ITPACK, FFTPACK and the Harwell Library
• Mathematical Software for Elliptic Boundary Value Problems (Boisvert, Sweet)– included discussion of ELLPACK and FISHPACK
60
![Page 61: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/61.jpg)
... (cont’d)• The IMSL Library (Aird)• The SLATEC Common Mathematical Library
(Buzbee)• The Boeing Mathematical Software Library
(Erisman, Neves, Philips)• The PORT Mathematical Subroutine Library
(Fox)• The Evolving NAG Library Service (Ford, Pool)
61
![Page 62: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/62.jpg)
Today
• BCSLIB, IMSL, HSL and NAG are all alive• A number of the PACKs are still available;
LAPACK is probably the most active (but needs more funding!)
• SLATEC?
• PETSc• Trilinos (maintained repository)• ...
![Page 63: A Brief History of Numerical Libraries](https://reader035.vdocuments.mx/reader035/viewer/2022062312/62a7e99431a0f9448a676523/html5/thumbnails/63.jpg)
Big Challenge for the Future
• Multicore and hybrid chips– PLASMA, MAGMA, ...
• Dongarra: “There is no Moore’s law for software”
• Congratulations to Jack and all of ICL, past and present