multiprocessor computer architectures : algorithmic design ... · multiprocessor computer...

341

Upload: danghanh

Post on 05-Apr-2018

275 views

Category:

Documents


5 download

TRANSCRIPT

Page 1: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Loughborough UniversityInstitutional Repository

Multiprocessor computerarchitectures : algorithmicdesign and applications

This item was submitted to Loughborough University's Institutional Repositoryby the/an author.

Additional Information:

• A Doctoral Thesis. Submitted in partial fulfilment of the requirementsfor the award of Doctor of Philosophy of Loughborough University.

Metadata Record: https://dspace.lboro.ac.uk/2134/10872

Publisher: c© A.S. Roomi

Please cite the published version.

Page 2: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

This item was submitted to Loughborough University as a PhD thesis by the author and is made available in the Institutional Repository

(https://dspace.lboro.ac.uk/) under the following Creative Commons Licence conditions.

For the full text of this licence, please go to: http://creativecommons.org/licenses/by-nc-nd/2.5/

Page 3: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Dx crcn 10

LOUGHBOROUGH UNIVERSITY OF TECHNOLOGY

LIBRARY

AUTHOR/FILING TiTlE i . _____________ ~ ~ .9_"::"_ '--I ___ A~ _____ ----- ----- --;

---- --- ----- ---------------- ----- --- ----- - - ----- ---. ACCESSION/COPY NO.

o '+-0:> 0 I S-S"3 0 ----------------- .. -- ---- -- --- --------- - - -- - - --VOL. NO. CLASS MARK

Ul 1994

30 JUN

040015530 3

111111 11111111111111111111111111

Page 4: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS
Page 5: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

MULTIPROCESSOR COMPUTER ARCHITECTURES:

ALGORITHMIC DESIGN AND APPLICATIONS

BY

AKEEL S. ROOM!, B.Sc.,M.Sc.

A Doctoral Thesis

Submitted in Partial Fulfilment of the Requirements

For the Award of Doctor of Philosophy

of Loughborough University of Technology

August, 1989.

SUPERVISOR: PROFESSOR D. J. EVANS; PH. D-:','6?'s6:; ," •• ,.;.:r-

Department of Computer Studies c;.;

.~

© by A.S. Roomi, 1989.

Page 6: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

! I ;~;;;~ UnNers-; . of TQChnology Library

Dale J'.... '! 0

Page 7: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

CERTIFICATE OF ORIGINALITY

This is· to certify that I am responsible for the work submitted I

in this thesis, that the original work is my own except as specified

in acknowledgements or in footnotes,. and that neither the thesis nor

the original work contained therein has been submitted to this or

any other institution for a higher degree.

A.S. RooMI.

Page 8: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

To

My Paren ts ,

My Wife, Berjua~,

and my Children,

Amar, Maytham and Heatr.or,

with love,

AkeeL

Page 9: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

'.

ACKNOWLEDGEMENTS

It is the special privilege of authorship, that at the end of

one's exertions, it is possible to pause, look back and formally

thank the many people without whose active participation, the

scheduled completion of a doctoral thesis would have been impossible.

Professor D.J. Evans, Head of the Parallel Processing Centre,

Loughborough University of Technology, in addition to being an effective

and extremely conscientious supervisor and Director, is also endowed

with a gently persuasive manner. This combination was ideal in guiding

me through the perilous area of parallel computers. I am most grateful

to him for his painstaking comments and useful suggestions throughout

the last three years.

My profound gratitude to the Ministry of Higher Education, Iraqi

Government, for the award of a three year scholarship to enable me to

undertake this research.

I am grateful to the staff and research students of the Department

of Computer Studies for their kind cooperation during my research. I

. wish to thank in particular, Dr. W.S. Yousif, for his useful suggestions

during the early stage of this work.

Finally, I would like to express my appreciation·to my wife,

Berjual, who provided an environment and the encouragement essential

to the completion of the work.

Page 10: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

ABSTRACT

The contents of this thesis are concerned with the implementation

of parallel algorithms for solving partial differential equations

(POEs) by the Alternative Group EXplicit (AGE) method and an

investigation into the numerical inversion of the Laplace transform

on the Balance 8000 MIMO system.

Parallel computer architectures are introduced with different

types of existing parallel computers including the Oata":Flow computer'

and VLSI technology which are described from both the hardware and

implementation points of view. The main characteristics of the Sequent

parallel computer system at Loughborough University is presented, and

performance indicators, i.e., the speed-up and efficiency factors are

defined for the measurement of parallelism in the system. Basic ideas

of programming such computers are also outlined.

Basic mathematical definitions and a general description and

classification of POE's and its related discretised matrix are

introduced in Chapter 3.

In Chapter 4, the parallel version of the AGE method is developed.

for one and two dimensional elliptic POE's. The AGE method is

suitable for parallel computers as it possesses separate and independent

tasks. Therefore three synchronous and asynchronous strategies have

been used in the implementation of the method. The timing results

and the efficiency of these implementations were compared. A

. computational complexity analysis of the parallel AGE method is also

included.

Page 11: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

The eigenvalues and the corresponding eigenvectors of the Sturm

Liouville problem are found by using the AGE method with different

boundary conditions.

In Chapter 5, the three parallel AGE strategies are also

implemented on time dependent POE's. The parallel AGE method was

applied on the second order parabolic equation with Oirichlet

boundary conditions and the diffusion-convection equation with a

comparison presented. Then the parallel AGE method on a two

dimensional parabolic equation and a hyperbolic equation is discussed.

A parabolic POE with derivative boundary condition is also solved

using the AGE method. A new AGE formula based on a O'Yakonov splitting

of the matrix is used to solve one and two dimensional parabolic POE's.

By comparing and analysing the results, yields an algorithm with

reduced computational complexity and greater accuracy for multi­

dimensional problems.

Finally, in Chapter 6, the problem of numerically inverting the

Laplace transform has been investigated and numerical results were

obtained to compare th", different methods. An idea to improve the

accuracy by. imposing a Romberg integration is suggested. Attempts to

a~celerate the convergence of the slowly converging series is also

investigated. A parallel algorithmic form of an accurate method for

the numerical inversion of the Laplace transform is implemented.

The thesis concludes by summarizing the main results obtained

and suggestions for further work are included.

Page 12: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

CONTENTS

PAGE

CHAPTER 1: PARALLEL COMPUTER ARCHITECTURES. AN INTRODUCTION

1.1 Introduction 1

1.2 Via Parallelism 4

1.3 Architectural Classification Schemes 8

1.3.1 Flynn 's Parallel Computen; Classification 8

1.3.2 Feng's Parallel Computers Classification 10

1.3.3 Shore's Parallel Computers Classification 13

1.3.4 Handler's Parallel Computers 16 Classification

1.3.5 Other Parallel Computers Classification 17

1.4 Pipeline Computers

1.5 SIMD

1.6 MIMD

1.7 Data-Flow Computers

1.8 VLSI Systems and Transputers

1.8.1 Transputer System

1.9 The Balance 8000 System

CHAPTER 2: PARALLEL PROGRAMMING AND LANGUAGES

2.1 Introduction

2.2 Parallel Programming

2.2.1 Implicit Parallelism

2.2.2 Explicit Parallelism

2.3 Programming the Balance System

2.3.1 Multitasking Terms and Concepts

19

21

23

32

35

38

41

47

49

52

54

60

61

Page 13: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

2.3.2 Data Partitioning with DYNIX

2.3.3 Function Partitioning with DYNIX

2.4 Parallel Algorithms

2.4.1 The Structure of Algorithms for Multiprocessor Systems

CHAPTER 3: BASIC MATHEMATICS, GENERAL BACKGROUND

3,1 Introduction

3.2 Classification of Partial_Differential Equations

3;3 Types of Boundary Conditions

3.4 - Basic Matrix Algebra

3.4.1 Vectors and Matrix Norms

3.4.2 Eigenvalues and Eigenvectors

PAGE

66

70

73

78

80

81

84

87

91

93

3.5 NUMERICAL SOLUTION OF PDE'S BY FINITE DIFFERENCE METHOD 95

3.5.1 Finite Difference Approximation

3.5.2 Derivation of Finite Difference Approximations

-3.5.3 Consistency, Efficiency, Accuracy and Stability

3.6 Methods of Solution

3.6.1 The Direct Methods

3.6.2 The Iterative Methods

3.6.3 The Block Iterative Methods

3.6.4 Alternating Direction Implicit (ADI) Methods

3.6.5 Alternating Group Explicit (AGE) Method

95

103 -

107

109

109

III

- 116

116

123

Page 14: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

PAGE

CHAPTER 4: STEADY STATE PROBLEMS, PARALLEL EXPLORATIONS

4.1 Introduction 131

4.2 Parallel AGE Exploration 132

4.3 Experimental Results for the One Dimensional Problem 137

4.4 Experimental Results for the Two Dimensional Problem. 150

4.5 The AGE Method for Solving Boundary Value Problems with Neumann Boundary Conditions 172

4.5.1 Formulation of the Method

4.5.2 Numerical Results

4.6 The AGE Method for Solving Sturm Liouville Problem

4.6.1 Method of Solution

4.6.2 Numerical Result

4.7 Conclusions

CHAPTER 5: . TIME DEPENDENT PROBLEMS, PARALLEL EXPLORATIONS

5.1. Introduction

5.2 . Experimental Results for the Diffusion­Convection Equation

172

177

181

182

185

187

189

190

5.3 Experimental Results for the Two-Dimensional Parabolic Problem 202

5.4 Experimental Results for the Second Order Wave Equation

5.5 The Numerical Solution of One-Dimensional parabolic Equations by the AGE Method with

220

D' Yakonov Splitting 224

5.5.1 The AGE Method

5.5.2 Numerical Results

225

229

Page 15: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

5.6 The Numerical solution of Two-Dimensional parabolic Equation by the AGE Method with

PAGE

D' Yakonov Splitting 234

5.6.1 Numerical Results 240

5.7 A New ?trategy for the Numerical Solution of the Schrodinger Equation 244

5.7.1 OUtline of the Method

5.7.2 Numerical Results

5.8 Conclusions

245

248

251

CHAPTER 6: NUMERICAL INVERSION OF THE LAPLACE TRANSFORMATIONS, SOME INVESTIGATIONS AND PARALLEL EXPLORATIONS

6.1 Introduction

6.2 The Numerical Inversion of the Laplace Transform

6.3 Numerical Experiments

6.3.1 The Implementation of the Fast Fourier

253

255

258

Transform Technique 268

6.4 Parallel Implementation of the Numerical Inversion of the Laplace Transform 271

6.5 Conclusions 275

CHAPTER 7: CONCLUSIONS AND FINAL REMARKS 276

REFERENCES 281

APPENDIX A: A LIST OF SOME SELECTED PROGRAMS 290

Page 16: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS
Page 17: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

CHAPTER 1

PARALLEL COMPUTER ARCHITECTURES,

AN INTRODUCTION

Anyone who says he knows how computers

should be buil~ should have his head examined.

Computer Architecture

J.E. Thornton.

Page 18: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

1

1 .. 1 INTRODUCTION

High-performance, flexible and reliable computers are increasingly

in demand from many scientific and engineering applications, which may

be 4equired to be solved in real time. Since conventional computers

have a limited speed and reliability, the satisfaction of these

requirements can only be achieved by a high-performance computer

system. The achievement of high performance not only depends on using

faster and more reliable hardwar~ devices, but also on different

computer architectures and processing techniques. Therefore, parallel

computer systems need to be developed further.

In earlier times, relays (in the 1940s) and vacuum tubes (in the

1950s) were used as switching devices and they were interconnected

with wires and solder joints. Central Processing Unit (CPU) structure

was bit-serial and arithmetic was done on a bit-by-bit fixed point

basis. By the early 1960's, transisters (invented in 1948) were used

i~ computer circuits .. Passive components such as resistors and

capacitors were also included in these circuits. All of these devices

were mounted on some kind of circuit boards, the most complex of which

consisted of a number of layers of conductors. and insulating material.

These provided interconnections between the elementary devices as well

as their mechanical support. Many improvements to ··computer architectures

were subsequently carried out. For example, Sperry Rand built a

computer system with an independent 1/0 processor which operated in

parallel with one or two processing units. Core memory was still

used in many computer systems. Then, solid-state memories replaced

the core memories.

By the late 1960's, Integrated Circuits (ICs) were in use,

Page 19: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

,,'

2

followed by Large Scale Integrated (LSI) techniques, providing on one

silicon chip several transistors, the required resistors and capacitors

as well as interconnection paths.

Following the rapid advance in LSI technology, the Very Large Scale

Integration (VLSI) circuits have been developed with which enormously

complex digital electronic systems can be fabricated on a single chip

of silicon. Devices which once required many complex components can

now be built with just a few VLSI chips, reducing the difficulties in

reliability, performance and heat dissip'ation that arise from standard

small-scale and medium-scale integrate.

Until a few years ago, the current state of electronic technolo~y

was such that all factors affecting computational speed were almost

minimized and any further computational speed increase could only be

achieved through both increased switching speed and increased circuit

density. Due to the basic physical laws, the intended breakthrough

seemed unlikely to be achieved mainly because we are fast approaching

the limits of optical resolution. Hence, even if switching times are

almost instantaneous, distances between any two points may not be

small enough to minimize the ,propagation delays and thus improve

'computational speed. Therefore, the achievement of even faster

computers is conditioned by the use of new approaches that do not depend'

on breakthroughs in device technology, but rather on imaginative

applications of the skills involved in computer architecture.

Obviously, one approach to increasing speed is through Parallelism.

The ideal objective is to create a computer system containing p

processors, connected in some cooperating fashion, so that it is p

times faster than a computer with a single processor. These parallel

Page 20: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

computer systems or multiprocessors as they are commonly known, not

only increase the potential processing speed, but they also increase

the overall throughput, flexibility, reliability and provide fault

tolerance in case of processor failures'"

3

Page 21: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

4

1.2 VIA PARALLELISM

Parallelism, the notion of the parallel way of thinking was

conceived long before the emergence of truly parallel computers. It

is thought that the earliest reference to parallelism is in Le Manebrea's

,publication', entiJ:led "Sketch of the Analytical Engine" invented by

C. Babbage. There, reporting on the utility of the conceived machine,

he wrote:

"Likewise when a long series of identical computations is to

be performed, the machine can be brought into play sO,as to

give several results at the same time, which will greatly

abridge the whole amount of the processes".

Babbage's notion was neither implemented in the final design of

his calculating engine or elsewhere, due to the lack of technological

development accordingly, though, the notion of the parallel way of

thinking had been conceived.

The division of ~omputer systems into generations is determined

by the device technology, system architecture, processing mode and

languages used. 'We are currently in the fourth generation, while the

'. fifth generation is on the horizon.

The first generation (1938-1953). The first electronic digital

computer, ENIAC (Electronic Numerical Integrator And Computer), in 1946

marked the beginning of the first generation of computers. Using

vacuum tubes, magnetic drums as central memories and electronic valves

as their switching components, with gate delay times of approximately 1 ~s.

* Following Babbage' s lecture in Turin, describing his "difference engine"

a young Italian engineer wrote a detailed account of the machine in

French (published in October 1842). Ada, Lady Lovelace translated the paper into English.

Page 22: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

5

The second generation (1952-1963). Transistors were invented in

1948, while the first transistorized digital computer (TRADIC) , was

built by Bell Laboratories in 1954. The propagation delay times of.

using the germanium transistor is approximately 0.3 ~s. Assembly

languages were used until the development of high-level languages,

FORmula TRANslation (FORTRAN), in 1956 and ALGOrithmic Language (ALGOL),

·in 1960.

The third generation (1962-1975). This generation was marked by

the use of Small-Scale Integrated (551) and Medium-Scale Integrated

·(MSI) circuits as the basic building blocks. High-level languages

were greatly enhanced with intelligent compilers during this period.

Multiprogramming was well developed to allow the simultaneous

execution of many program segments interleaved with I/O operations.

Virtual memory was developed by using hierarchically structured memory

systems. The propagation delay was about 10 ns, and later, around the

1970's, it became slightly less than 1 ns.

The fourth generation (l972-present). This generation is

characterised by enhanced levels of circuit integration through the

use of LSI circuits for both logic and memory sections. High~level

languages were extended to handle both scalar and vector data; Most

operating systems were time-sharing, using virtual memories.

Vectorizing compilers appeared in the second generation of vector

machines like the Cray-l (1976) and the Cyber-205 (1982). High-speed

main frames and supercomputers appeared as multiprocessor systems, like

the Univac 1100/80 (1976), Fujitsu M382 (1981), the IBM 3081 (1980),

and the Cray X-MP (1983). A high degree of pipelining and multi­

processing is greatly emphasized in commercial supercomputers.· A

Page 23: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Massively Parallel Processor (MPP) was custom-designed in·1982.

All these various multiple processor architectures can be

categorized in four distinct organizations: Associative, Parallel,

Pipelined and Multiprocessors.

An attempt by Hockney and Jesshope [Hockney, 1981] to summarize

the principal ways to introduce the notion of parallel processing at

the hardware level of the various computer architectures, res~lts in:

1. the application of pipe lining-assembly lines-techniques in

order to improve the performance of the arithmetic or control

units. A process is decomposed into a certain number of

elementary subprocesses each of which being capable of

execution on dedicated autonomous units;

2. the arrangement of several independent units, operating in

parallel, to perform some basic prinCipal functions such

as logic, addition or multiplications;

3. the arrangement of an array of processing elements (PE's)

executing concurrently the same instruction on a set of

different data, where the data is stored in the PE's

private memories;

4. the arrangement of several independen t processors, working

6

in a cooperative manner towards the solution of a single task

by communicating via a shared or common memory, each of them

being a complete computer, obeying its own stored instructions.

To illustrate alternative hardware and software approaches, in

the following sections, we shall select principal significant

architectures, which differ sufficiently from each other. Specifically

Page 24: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

for the Multiprocessor class, the Balance 8000, parallel processing

system, at Loughborough University of Technology, is described in

more detail, due to the fact that it was extensively used during the

carrying out of the present research.

7

Page 25: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

8

1.3 ARCHITECTURAL CLASSIFICATION SCHEMES

To date many classification schemes have been proposed. In this

section we shall briefly present the theoretical concepts of the

architectures taxonomy given by Flynn (1966), Feng (1972), Shore (1973)

and Handler (1977).

1.3~1 Flynn's Parallel Computers Classification

In 1966, M.J. Flynn [Flynn, 1966] classified computer organizations

into four categories according to the multiplicity of instructions and

data streams. For convenience he adopted two definitions: the

instruction stream, as a sequence of instructions which are to be

executed by the system, and the data stream, as a sequence of data

called for, by the instruction stream.

Flynn's four machine organizations as shown in Figure 1.1 are:

1. Single Instruction Single Data (SISD);

2. Single Instruction Multiple Data (SIMD);

3. Multiple Instruction Single Data (MISD);

4. Multiple Instruction Multiple Data (MIMD).

SISD:

SIMD:

This is the classical von Neumann model. A single stream of

instructions operates on a single data stream. It may have more.

than one functional unit operating under the supervision of one

control unit. Examples are IBM 3600/91, CDC-NASF, Fujitsu

FACOM-230/75.

This is the class to which array processors and pipeline

. processors belong. All the processors elucidate the same

instructions and perform them on different data. Because of

Page 26: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

9

8I8D 8IMD

C C

T [ J. 1

Pl

P2

P n

P

I 1 -1 N

8 I J r 8

1 8

2 8

n

MISD MIMD

I C C

1

c:5 C2

C n

T

Q Cl C n

Pi P2 P

n Pi P2 P

n

N N

I S

1 I 5J EJ S

n

FIGURE 1.1: Flynn's Parallel Computer Classification

rc: control unit; P: processor; N: data organisation network; S: store).

Page 27: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

10

their simple form machines of this kind can have a large

number of processors. Examples are ICL/DAP. Illiac-IV. STARAN.

MISD: (Chains of processors), there are n processor units, each

receiving distinct instructions operating over the same data

stream. No real embodiment of this class exists.

MIMD: This is the multiple processor version of "SIMD. All processors

elucidate different instructions and operate on different data.

Most multiprocessor systems and multiple computer systems can

be classified in this category. Examples are C.mmp. Balance

8000. 21000. Cray-2.

1.3.2 Feng's parallel Computers Classification

In his classification Tse-Yun Feng [Feng. 1972] has proposed the

use of degrees of parallelism in various computer architectures. The

Tfl2ximum paralleUsm degree P, is defined as the maximum number of binary

digits (bits) that can be processed within a unit time by a computer

"" system. " If p. are the number of bits that can be processed within the 1.

ith

processor cycle and T is the processor cycle indexed by i=1.2 ••..• T.

then the average parallelism degree."p is defined by. a

T

L Pi i=l

= --"'--=--­T (1.3.1)

typically. p.~p. Accordingly the utilization rate ~ of a computer 1.

system within T cycles is,

T

Pa L Pi

i=l IJ. = (1.3.2) p T.p

Page 28: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

11

Figure 1.2 emphasizes the classification of computers by their

maximum parallelism degrees, where the horizontal axis shows the

word length n, while the vertical axis corresponds to a bit-slice*

length m.

If c be a given computer, then the maximum parallelism degree

p·(c) is represented by the product of the word length n and the bit-

slice length m; that is,

p(c) n.m . (L 3.3)

Obviously., p(c) is equal to the area of the rectangle defined by the

integers nand m.

There are four types of processing methods that can be seen from

Figure 1. 2:

1. Word-Serial and Bit-Serial (WSBS);

2. Word-Parallel and Bit-Serial (WPBS);

3. Word-Serial ·and Bit-Parallel (WSBP);

4. Word-Parallel and Bit-Parallel (WPBP).

WSBS: which is the conventional serial computer (von-Neumann);

One bit (n=m=l) is processed at a time.

WPBS: (n=l,m>l) I since an m bit-slice is processed at a time, so it

is termed bit-slice processing. Examples are STARAN, MPP.

WSBP: (n>l, m=l) , found in most existing computers. Since one word of

n bits is processed at a time, hence, it has been called word-

* The bit-slice is a string of bits, one from each of the words at the

same vertical bit position. As an example, the TI-ASC has a word

length of sixty-four and four arithmetic pipelines, and each pipe has

eight pipeline stages, so there are thirty-two bits per each bit-slice

in the four pipes.

Page 29: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

16384 - - .. MPP

.5

:S IJ' <: Q)

..:I Q) U • .-1 .-< ., 1> • .-1 <Il

: (1,16384)

288 - _ . ...1

256

64

16

1

• Staran (1,256)

__ t __ _

1

C.DDDp ------ - ---~(16,16)

, -

1 PDP-ll------i'-----

1 (16,1) I' ,

16

Word Length (n)

PEPE --"(32,32) , ,

, 1 -

i _____ _

I

, , I

12

Illiac IV - • (64,64)

, ,IBM 370/168 1 Cray-l

-----------<t : (32,1) 1 (64,1)

32 64

FIGURE 1.2: Feng's Parallel Computer Classification System

Page 30: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

WPBP:

slice processing. Examples are IBM 370/168UP, CDC 6600.

(n>l, m>l) , known as fully parallel processing, in which an

array of n·m bits is processed at one time. Examples are

TI-ASC, C.mmp, Balance 8000, 21000.

1.3.3 Shore's Parallel Computers Classification

13

In 1973 Shore [Shore, 1973] presented a classification of parallel

computer systems based on their constituent hardware components. There

are six different types of machines according to his proposal, and all

existing computers could belong to one of them. Figure 1. 3 shows the

six different types and what they are.

1. Machine 1, consists of an Instruction Memory (IM) , a single Control

Unit (CU) , a Processing Unit (PU) , and a Oata Memory (OM). Examples

are the Cray-l, COC 7600, etc.

2. Machine 11, is obtained from Machine I by simply changing the way

the data is read from OM. Machine 11 reads a bit from every word

in the memory, instead of reading all bits of a single word.

Examples are ICL OAP, STARAN, etc.

3. Machine Ill, this machine is derived from the combination of

Machines I and I1a There are two PUs, one horizontal and one

vertical. An example is Sander's Associates OMEN 60 .

. 4. Machine IV consists of a single CU and as many as possible,

independent PE's, each of which has a PU and OM. Communication

between these components is restricted to take place only through

the CU. Example is PEPE.

5. Machine V is derived from Machine IV by adding interconnections

between processors. An example is ILLIAC IV computer.

Page 31: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

14

IM CU IM

Horizontal CU PU

Word-slice Vertical Byte slice. OM PU OM

1. Machine I 2 . Machine 11

CU IM

L G Horizontal PU PU PU

CU PU 1 2 n

Vertical PU OM

OM .OM OM 1 2 n

3. Machine III 4 • Machine IV

FIGURE 1.3: Shore's Parallel Computer Classification

Page 32: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

PU 1

OM 1

PU 2

DM 2

CU

5. Machine V

CU

PU + OM

6. Machine VI

PU n

OM n

FIGURE 1.3: Shore's Parallel Computer Classification (continued)

15

Page 33: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

6. Machine VI, the difference between this machine and the previous

machines,is that the PU's and the DM are no longer individual

hardware components, but instead they are constructed on the same

re board. Examples are the associative memories and associative

processors.

1.3.4 Handler Parallel Computers Classification

Wolfgang Handler [Handler, '1977J suggested his classification

outline for identifying the parallelism degree and pipelining degree

built ,into the ,hardware structures of a computer system. There are

three subsystem levels of parallel-pipeline proces'sing according to

hi~ classification:

1. Processor Control Unit (PCU);

2. Arithmetic Logic Unit (ALU);

3. Bit-Level Circuit (BLe).

PCU and ALU are well defined. Each PCU corresponds to one

processor. The ALU is equivalent to the PE in SIMD array processors.

The BLe corresponds to the combinational logic circuitry needed to

perform I-bit operations in the ALU. Let C be defined as a computer

'system, then C can be characterized by a triple containing six

independent entities, as defined below:

16

T(C) ; <KxKl,DxDl,wxWl> (1.3.4)

where,

D ; the number of ALU's (or PE's) under the control of one PCU;

K the number of processors (PCU's) within the computer;

W the word length of an ALU (or PE);

Page 34: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Dl = the number of ALU's that can be pipelined;

Kl = the number of PCU's that can be pipelined;

Wl = the number of pipeline stages in all ALU's (or PE's).

17

AS an example, the Texas Instrument's Advanced Scientific Computer

(TI-ASC) has one controller controlling four arithmetic pipelines, each

has 64-bit word lengths and .8-stages. Thus, we have

T(ASC) = <lxl,4 xl,64 x8> = <1,4,64x8> . (1.3:5)

·1.3.5 Other Parallel Computers Classification

There are some other classification approaches, less significant

than the former four, based mainly upon the notion of parallelism.

Hobbs, et al [Hobbs, 1970] in 1970, distinguished the parallel

architectures into Multiprocessors, Associative processors, Network or

Array processors and Functional ma~hines.

Murtha.and Beadles [Murtha, 1964] based their taxonomy view upon

the parallelism properties, attempting to underline the differences

between multiprocessors and a highly parallel organization. According

to them the parallel organization could be classified into the general­

purpose network computers, the special-purpose network computers with

global parallelism and finally the non-global semi-independent network

computers with local parallelism.

Page 35: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

18

1.4 PIPELINE COMPUTERS

In this section, the structures of pipeline computers and vector

processing principles are studied. Pipelining offers an economical way

to realize temporal parallelism in digital computers. To achieve

pipe lining , one must subdivide the input task (process) into a sequence

of subtasks, each of which can be executed on a dedicated facility,

called a stage or station. Stations are connected via buffers or latches.

The pipeline computers have distinct pipeline processing capabilities,

'e.g. there can be pipe lining between the processor and the I/O unit.

Within the processor, there can be pipelining between instruction.

A pipeline processor consists of a sequence of.processing circuits,

called stages, through which a data stream passes. Each stage does

some partial processing, on the data and a final result is obtained

after the data has passed through all these stages of the pipeline.

Figure 1.4 exemplifies a typical pipeline computer. This diagram

shows both scalar arithmetic pipelines and vector arithmetic pipelines.

The instruction PU is itself pipelined with three stages.

As an example, consider the process of performing the instruction

(F), decode the instruction (D), fetch the operand (0), and finally its

execution (E).

In a non-pipelined computer, the above steps E~St be completed

be~ore the next instruction can be issued as shown in Figure 1.5,

----011 Instruction processing I ·

FIGURE 1.5: Non-Pipelined Processor.

Page 36: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

1/

,

Scalar processor

Scalar data r1 SPl l K

If - - - cal- -1 SP2 I - - - - -- .. - -.- -- - - -I

I ar .... 1 I'egi- I

1 ~ter I

I I

Main I ~ spn-1 - - -Memory Instruc Instru: Scalar - - ------

tion -tion K Scalar pipeline

fetch decode fetch

0 IS +- (F) (D) (0)

Vector processor

K vector --- - -

J fetch -I VPl

- - -, 1 I J vp2 1 kiect- 1

Instruction processing I ~ ~ --- - - -- - - - - -- - - ~ or

K Regi-, I

sten I

data 1 VPm Vector

Vector pipeline

FIGURE 1.4: Functional structure of a Modern Pipeline Computer with Scalar and vector Capabilities.

(IS: Instruction stream; 0: Operand fetch;

K: ControZ signaZ)

19

I

J

J I I ,

Page 37: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

While in the pipel·ined computer the four stages F,D,O, and E are

executed in an overlapped manner (Figure 1.6). After constant time

intervals, the output of one stage is switched to the next station.

A new instruction is fetched (F) in every time cyCle, and stage (E)

produces an output every time cycle, because the time to perform an

instruction consists of multiple pipeline cycles ..

Pipe lining can be presented at more than one level in the design

of computers. Ramamoorthy and Li [Ramamoorthy, 1977] introduced many·

theoretical considerations of pipelining and presented a survey of

comparisons between various pipeline machine designs.

Stations

SI S2 S3 S4

FIGURE 1.6: A Pipelined Processor.

20

Page 38: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

21

1.5 SIMD

The SIMD (array processor) is a synchronous parallel computer with

"multiple arithmetic logic units (ALU) , called processing elements (PEs).

These identical PEs, arranged in an array form and controlled by a single

control unit, which decodes and broadcasts the instructions to all

processors within the array. Each PE has its own private memory which

provides it with its own data stream. Hence the PEs are synchronized

to perform the same function at the same time. ·Two essential reasons

for building array processors are firstly, economic for it is cheaper

to build P processors with only a single control unit rather than P

similar computers. The second reason concerns interprocess~r

communication, the communication bandwidth can be more fully utilized.

An array processor computer is depicted in Figure 1.7.

The advantages of SIMD computers seem to be greatest when applied

to the solution of problems in matrix algebra or finite difference

methods for the solution of the partial differential equation. It is

known that most algorithms in this area require the same type of

operations to be repeated on large numbers of data items. Hence, the

problem can be distributed to the processors that can run simultaneously.

Obviously, a complete interconnection network, where each processor

"is connected to all other processors, is expensiye and unmanageable by

both the designer and the user of the system. Therefore some other

interconnection patterns are proposed to be of primary importance in

determining the power of parallel computers.

The interconnection pattern used in ILLIAC IV is that the

processors are arranged in a two-dimensional array where each processor

Page 39: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

22

is connected to its nearest four neighbours. Also processors can be

arranged as a regular mesh in a perfect shuffle pattern or in various

special-purpose configurations for merging, sorting or other specific

applications.

I/O

I control I I2roc~S:S:QX:

CU (Scalar processing)

I Control I memoQ!

Data bus I I contra 1 I - - - --- - - - ---I

1

1 PEl I PE2 PEn I

I I

Iprocesso~ ~rocessorl Iprocesso3 I

~ I--- - -- +- I I

I memory I I memory I I memory I I I

(Array I processing) I

I 1

I

'"

Inter-PE connection network (data routing)

FIGURE 1.7: Array Processors Computer.

Page 40: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

23

1.6 MULTIPROCESSING SYSTEMS (MIMD)

In this section our emphasis is on m~ltiprocessing computers. The

American National Standards Institute (ANSI) , defined the multiprocessor

as: "A computer employing two or more processing units under integrated

control", which is hardly complete since the two most significant

images for this type of computer, i.e. the sharing and interaction

concepts were not included.

In 1977 Enslow [Enslow, 1977] offered a complete definition of.a

multiprocessor system depending on· its characteristics. As well as the

ANSI definition, he included the following conditions:

1. All the processors with approximately comparable capabilities, must share

access to a cocmon memory, I/O channels, control units and devices.

2. The entire complex is under the control of a single operating system

providing the interaction between processors and their.program at

the task, instruction and data levels. The basic diagram of a MIMD

machine of P processors is illustrated in Figure 1.8. Although

each processor includes its own CU, a high level CU may be used to

control the transfer of data and to assign tasks and sequences of

operations between the different processors.

The .universal class of MIMD comp)lters may be classi·fied, depending

on the amount of interactions, into two main classes:

1. A tightly-coupled system;.

2. A loosely-coupled system.

The tight~y-coup~ed processors, as illustrated in Figure 1.9a,

where the processors operate under the strict control of the bus

Page 41: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

24

CU connection 1- - - - - - - - - - - I

, I I

,- --) CUl ~ -,~ CUP E-

I I

1 ,..

I '" , , I .~ I

I I I

1 , I --I

1 I I I I , , ."

=1 Channel H Input .~

~ - _.

Channel H OUtput I-> Interconnection network

(switch)

k- - - , Secondary I Channel 1"---1 memory

i j

I

! -.-I , I - - - I , I

.c ,- -----) Control flow

I l Data flow Main memory

I

FIGURE 1.8: A -Typical MIMD System.

Page 42: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

25

assignment scheme which is implemented in hardware at the bus/processor

interface.

The loosely-coupled processor, as shown in Figure 1.9b, where the

communication and the interaction between processors took place on the

basic of information exchange.

The tightly-coupled multiprocessor has a noticeable performance

advantage over the.loosely-coupled multiprocessor.

Examples of some multiprocessor systems which we shall briefly

discuss to exemplify the machin~s characteristics are;

1. The S.l system;

2. The Neptune system.

The S.l multiprocessor system which can be described as a high­

speed general-purpose multiprocessor developed at LLN laboratory. The

S.l is implemented with the S.l uniprocessor called Mark IIAs, as

illustrated in Figure 1.10. This structure consists of 16 independent

Mark 'IIA uniprocessors which share 16 memory bank's via a crossbar

switch. Each processor has a private cache which is transparent to the

user. Each-uniprocessor, crossbar switch and memory bank is connected

to a diagnostic processor which can probe, report and change the internal

state of all modules that it monitors. For the S.l system, there'ex~s

a single user operating system multi-user operating system and advanced

operating system. It also supports multitasking by the division of

problems into cooperating tasks.

The Neptune system, is another system built at Loughborough in 1981

under professor Evans, and used extensively for the development of

parallel MIMD algorithms (see Barlow et al [Barlow, 1981J).

Page 43: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

26

Shared memory

-

J Processor Processor Processor

1 2 3

a. Tightly-coupled

. Memory Memory 0 1 2 3 .

I i I

: ;

;

Processor Processor Processor 1 2 3

b. Loosely-coupled

FIGURE 1.9: Multiprocessor System.

Page 44: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

27

1-1 Memory 0 - - ------ Memory 15

4

Controller ( Hg~agnostic I ~~agnost~c Controller 15 rocessor rocessor --.J

I

r I I I TI I

- -Crossbar

- _ switch Diagnostic processor

T i Uniprocessor 0 Uniprocessor 15

Data InstructioIl cache cache

M

F

P

I

A

Diagnostic I/O

~ --- -! I/O

store store processor

0 7

Mass storage

Real time units

I/O +---< I/O !-'> I/O process

-or 0

I Peripheral equipment

procelE -- -or

-,

1

Data Instruction cache cache

.

M

1-14 F

~--- - P

I

A

Diagnostic I/O I/O

store 1'---- .... store processor

0 7

Mass storage

Real time units

I/O +-l I/O ~ I/O process

-<)r -- -0

Peripheral equipment

rrocess -<)r 7

FIGURE 1.10: The Structure of the S.l Mark IIA Multiprocessor

Page 45: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

The Neptune system constitutes four Texas Instruments 990/10

mini-computers configured, as illustrated in Figure 1.11. The

instruction sets include both 16-bit and 8-bit byte addressing

capability. The system comprises of four processors each with its

private memory of 128Kb and one processor (po) has a separate 10Mb

disc drive. The access to the local memory is made by a local bus

(or TILINE), which is attached to each processor, ~ince the TILINE

coupler is designed so that the shared memory follows continuously

28

from the local memory of each processor, thus, each processor can

access 192kb of memory. Each processor runs under the powerful DXIO

uniprocessor operating system which is a general-purpose, multitasking

system. The DXIO operating system has been adapted to enable the

processors to run independently and yet to cooperate in running programs

with data in the shared memory.

Although the processors are identical in many hardware features,

they are different in their speed. The relative speed of processors

PO,Pl,P2 and P3 are 1.0, 1.037, 1.006 and 0.978 respectively. This

Will, however, reduce the efficiency of the system and decrease the

performance measurements of an algorithm with synchronization.

Various interconnection networks, which is the main factor for

multiprocessor hardware system organization, with different character­

istics such as bandwidth, delay and cost, ranging from the shared

common bus to the crossbar switch, have been suggested. Enslow in 1977

[Enslow, 1977] identified three intrinsic different organizations,

namely:

1. The time-shared common bus;

Page 46: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

29

I I I

Shared I-- memory

I I P

2 I I M2

I I

..

~ (

I P

l I \. Disc

I

I I I Bc )\ I I ~ Disc

~ W I P: processor

I M: memory

( Disc )

FIGURE 1.11: The Neptune Configuration

Page 47: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

30

2. Crossbar switch networks;

3. Multiport memory.

The time-shared common bus which represents the simplest inter-

connection system for either single or multiple processors. It consists

of a common communication path connecting all the functional units,

which are a number of processors, memories, and I/O devices. The system

capacity is limited by the bus bandwidth and system performance may be

demoted by adding new functional units.

To overcome the insufficienty of the time-shared bus organization,

the crossbar switch is used. The crossbar switch provides a separate

.path for every processor, memory module, and I/O unit. So that if the

multiprocessor system contains P processors and M memories, the cross-

bar needs (pxM) switches. In fact, it i·s difficult to have a large

system based on the crossbar switch concept because the complexity

2 grows at the rate of O{n ) for n devices. The important characteristics

of these systems are the extreme simplicity of the switch-to-functional

unit interfaces and the ability to support concurrent transfers for all

memory modules.

In the multiport memories systems, the functions of control,

switching between processors, and priority arbitration are centralized

at the memory interface. Hence every processor has a private bus to

every passiv~ unit, i.e. memory and I/O units.·

The prinCipal distinguishing features of such. a system are

expensive memory control, expansion from uniprocessor to multiprocessor

system using the same hardware, a large number of cables and connectors

and system limitation by memory port deSign.

Page 48: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Besides the three presented interconnection networks, there are

many others which can be valuable for multiprocessor organization such

as the·Omega network [Lawrie, 1975], the Augmented Data Manipulator

[Siegel, 1979] and the Delta network [Patel, 1981].

In Section 1.9 we will also present the Balance 8000 system as a

multiprocessor system.

31

Page 49: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

32

1.7 DATA-FLOW COMPUTERS

A new approach to parallel processing i.e. Data Flow is briefly

outlined in this section, whereas in the previous sections, the computer

architectures reviewed are known as control flow (CF) (von Neumann)

machines. In the CF computer, the program is stored in the memory as a

serial procession of instructions. Hence, this is one of the essent~al

difficulties in the utilization of the parallelism of algorithms in the

. ·CF model of computation.

An alternative architectural model for computer systems is

composed to assert the parallelism of algorithms, this model is the

Data-Flow (DF) (also known as a Data-Driven) model of computation. In

the DF system, the course of computation is controlled by the flow of

data in the program. In other words, an operation is executed as and

when its operands are available. This means that the sequence of

operations in the DF system respond to the preference constraint

imposed by the algorithm used rather than by the location of the

instructions in the'memory. On that account, the DF computer can

perform Simultaneously in parallel as many instructions as it is given

and distributes the result to all subtask instructions which make use

of this partial result as an operand.

The DF computers can be grouped, depending on the problem. tackled,

into two main classes:

1. The static structure;

2. The dynamic structure.

In the static structure, the loops and subroutine calls are

unfolded at compile time so that each instruction is performed only

Page 50: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

once. Figure 1.12 illustrates the static structure DF machine, which

consists of the following components: a store which holds the

instruction cells (packets) having space for the operation, operands

with their pointers to the successors, and a set of operating units to

execute the operations. The maximum· throughput is determined by the

speed and the number of operating units, the memory bandwidth and by

the interconnection system. The most significant factors that reduce

the throughput are the degree of concurrency available in the program;

the memory access and interconnection network conflicts and finally

the broadcasting of the results.

33

In the dynamic structure, the operands are labelled so that a

single copy of the same instruction can be used several times for

different instances of the loop or subroutine. Figure 1.13 shows the

dynamic structure DF· machine. The main components of the Manchester

DF computer [Gurd, 1985] are the token queue that stores computed

results, the token matching unit that combines the corresponding tokens

into instruction arguments, the instruction store that contains the

ready-to-execute instructions, the operating units and the I/O switch

for communication with the host. The degradation factors-are similar

to those of the static case except the additional overhead in token

label matching.

Because of the above recount discredit factors, DF systems are

only captivating for cases in which the con currency exhibited is of

several hundred instructions long. The significant advantages of·the

DF computers are the exploitation of the concurrency at a low level of

the performance hierarchy, since it allows the maximum utilization of all

the available concurrency.

Page 51: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

34

Operating units

~

I Inst . cell I ~ ... ... g ~ ...,

Q) I Q) c: c:

c: c: 0 I 0 .... . ... ..., I

..., 01 III

:!Ol ... ..., ... . ... ..., I cell I .Q

Ul Inst. ... .... ..: Q

FIGURE 1.12: The Static Data-Flow Computer.

hOr Token queue

1 To

I/O switch Matching unit OVerflow unit

r Fr om host

Instruction store

Processing units

FIGURE 1.13: The Dynamic Data-Flow Computer.

Page 52: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

35

1.8 VLSI SYSTEMS AND TRANSPUTERS

Attributable to current maturity in hardware technology, Large

Scale Integrated (LSI) circuitry has become so close-knit that a single

silicon LSI chip may contain tens of thousands of transistors. Whereas

the rapid advance in LSI technology leads to the Very Large Scale

Integrated (VLSI) circuits where the number of transistors that the LSI

circuits contain will be increased by another factor of 10 to 100. This

advent in the LSI and VLSI chips has given a large boost to the research

and development of array processor and multiprocessor architectures.

The key factors of VLSI technology are: its capacity to implement

enormous numbers of devices on a chip, its low cost and the high degree

of integration. While the main VLSI problem is to overcome the design

complexity. The size of wires and transistors approach the limits of

photolithographic constancy, for it becomes literally impossible to

achieve further diminutiveness and actual circuit area becomes a key

issue. In addition, the chip area is also limited in order to maintain

high chip yield and the number of pins is limited by the finite size

of the chip perimeter. These restrictions form the basis of the VLSI

paradigm.

The separation between the processor from its memory and the

-limited opportunities for synchronous processing are the principal

problem in the conventional (von Neumann) computers. The VLSI-designs

put forth more flexibility than conventional (von Neumann) systems to

overcome these difficulties, since memory and processing architectures

can be implemented with the same technology and in close vicinity.

Many authors investigate the requirements of parallel architectures

Page 53: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

for VLSI, among those are Kung [Kung, 1982], Dew [Dew, 1982] and

Seitz [Seitz, 1982].

Dew classified VLSI architectures as:

1. Simple and regular design, where it accommodates a few modules

which are replicated many times while the grain of the modules

depends on the application.

36

2. The design must have a very high degree of parallelism through both

pipelining and multiprocessing.

3~ Communication and switching, since the major differences.between

VLSI design and the earlier digital technologies is that the

communication paths will dominate both the area and the time delay.

This is because the speed of the devices increases as the feature

size decreases while the propagation time along a wire does not.

One of the novel ideas to emerge from VLSI research into parallel

architectures, is that of systoZic array processors. The concept of

systolic architectures, pioneered by Kung [Kung, 1982], is basically

a gen·eral methodology of directly mapping .algorithms onto an array of

PEs. The significant differences between a systolic array processor

and multiprocessor lattice are that in a systolic array, the

communication is only to neighbouring processing cells (i.e. no global

bus), the communication· with the outside world occurs only at the

boundary cells; and the processing cells are Ithardwired" and not

programmed from the host. The fundamental principle of asystolic

architecture, the systolic array in particular, is illustrated in

Figure 1.14. By replacing a single PE with an array· of PEs, a higher

computation throughput can be achieved without increasing the memory

bandWidth.

Page 54: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

37

Memory

PE

a. The Conven~ional Organization.

Memory

L-----lPE PE PE PE PE PE

b. A Systolic Array Processor

FIGURE 1.14: Systolic Design Principle.

One problem correlated with the systolic array systems, is that

the data and control movements are manipulated by global timing­

reference beats, so in order to synchronize the cells,:. extra delays are

often used to ensure correct-timing~ To overcome this conflict Kung

[Kung, 1985J suggested to take advantage of the data and control flow

locality, inherently possessed by most algorithms. This enables a data­

driven, self-timed approach to array processing. Ideally, such an

approach constitutes the requirement of correct timing by correct

sequencing a

Page 55: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

38

1.8.1 Transputer System

Another important development, which is set to make a major impact

in the field of multiprocessor systems is the INMOS Transputer [INMOS,

1984). What distinguishes the INMOS transputer from its competitors is

that it has been designed to exploit VLSI. The transputer is a single

chip microprocessor containing a memory, processor and communication

links for connection to other transputers, which affords direct hardware

support for the parallel language OCCAM. The structure of a transputer

is given in Figure 1.15. In the transputer INMOS have taken as many

components of a traditional von-Neumann computer as possible and

implemented them on a single lcm2

chip, while at the same time

presenting a high level of support for a synchronous view of· computation.

The communication between VLSI devices has a very much lower bandwidth

than communication between subsystems on-chip. Transputer systems

communicate asynchronously and therefore they can be synthesized with

each processor. In transputerports, all components perform

concurrently; each of the four links and the floating-point coprocessor

can all execute useful work while the processor is performing other

instructions.

Since the transputer is designed for the OCCAM language, so

concurrency may be described between transputers in the system or

within a single transputer, which means the transputer can support

internal concurrency. The processor contains a scheduler which

enables any number of processes to run on a single transputer sharing

processor time, while each link presents two unidirectional channels

for point to point communication. Processes are held on two process

Page 56: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

---l

Reset Ana1yze Error Boot from rom c1x vcc

·'gnd

not mem not mem not mem rd not mem rf

mem wai mem con f

--n--j

-l

-

:=

) System 7 services

On-chip ram /

I

~

Application and I

Specification I Interface

K

FIGURE 1.15: Transputer Architecture.

/ /

/ .I

I /

/ /

J I

Memory

I

Processor

Link interface

Link interface

Link interface

Link interface

Event

.

39

~i no

~, ut 0

; n 1

ut 1 .01

~i n 2

ut 2 ~o

~i n 3

ut 3

~~. vent ego vent ck. em ego em ran.

a m r m g

?

Page 57: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

queues. The active queue, which holds the process being executed and

any other active processes waiting to be executed while the inactive

queue holds those processes waiting on an input, output or timer

interrupt. Since there is a·little status to be saved, so the

process swap times are very small depending on the instruction being

executed. To carry out adding and deleting processes from the active

process queue, there are two further micro-instructions. These are

start process and end process. Input message and output message

instructions are for communication between processes on the transputer

or between processes on different transputers.

40

Page 58: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

41

1.9 THE BALANCE 8000 SYSTEM

The Balance 8000 is an expandable, high-performance, MIMD parallel

computer that employs from 2 to 12 32-bit CPUs in a tightly-coupled manner,

using a new processor pool architecture. By sharing its processing

load among up to 12 architecturally identical microprocessors and

employing a single copy of a Unix-based operating system, the Balance

8000 system eliminates the barriers associated with multiprocessor

systems. It delivers up to 5 million instruction/s(MIPs), and its

power grows almost linearly when more processors are added. To make

the most. efficient use of its multiprocessing power, the system

dynamically balances its load, that is it automatically and continuously

assigns jobs to run on any processor that is currently idle or busy

with an assigned lower-priority job.

At the same time, the system is easily extendible. We can add

CPU's, memory, and I/O subsystems within a node, or more nodes within

a distributed network, or more distributed and local-area networks _

all with no changes in software. From the hardware point of view,

the system consists of a pool of 2 to 12 processors, a high-bandwidth

bus, up to 28 Mbytes of primary storage, a diagnostic processor, up to

4 high-performance I/O channels, and up to 4 IEEE-796 (Multibus) bus

couplers. It is managed by a version of the UNIX 4.2 BSD operating"

system, enhanced to provide compatibility with UNIX system V and to

exploit the Balance parallel architecture. Figure 1.16 evinces the

main functional blocks of the Balance 8000 system.

Each processor in the pool is a subsystem containing three VLSI

parts, a 32-bit CPU, a hardware floating-point accelerator, and a paged

virtual memory management unit. Each two subsystems are on one circuit

Page 59: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

f-- Custom devices

Multibus f-- interface

board

Disk

Multibus adapter board

O---t-'-O Terminal

mux

I \

2-12 32-bits

CPUs

Memory 2-28

Megabytes

SB 8000 bus I ,

I I I I

,- - - -'-'----I

; Special I I purpose I , accelerators I

I '- - - - - - ---'

, , I , , , , ,

r --~ --- --I

I Special , , I purpose I controllers I , 1_- _______ I

FIGURE 1.16: Balance 8000 Block Diagram

SCED board

Ethernet

I

/

\

I

Ul :1

.Q

H Ul U Ul

}--

\ System console

.J I

r::::: =>

Disk

..... /

db

Page 60: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

43

board. To reduce all the processor wait-periods and minimizing the

bus traffic in the system, each processor contains a cach~ memory.

The two-way set associative cache consists of 8 Kbytes of very high-

speed memory accessed instructions and data, which means that,

requests for the same data are satisfied from the cache, rather than

from the primary storage.

Designing a cach~ for a processor pool architecture is difficult

for several reasons. / Since data in each cache represents a copy of

some data in the primary memory, it is important that all copies and

/

the original remain the same, even when a cache is updated. To ensure

that, the system employs a write-through mechanism, in which each write

cycle goes through to the bus and memory, in addition to updating the

appropriate cach~. Also, if two processors have both recently read

/

the same data into their respective caches, and one of them updates

/ its cache, the second processor can not use its new state data,

because of the caches bus-watching logic. As illustrated in Figure

1.17, this logic continuously monitors all write cycles on the bus

/

and compares addresses with those in its own cache to see if any writes

affect its own contents. / When such an address appears, the cache

invalidates the entry in question.

The last component of the processor subsystem is the "System Link

and Interrupt ControZZer" (SLIC) chip. The SLIC appears with every

processor in the system, as well as on every memory controller, I/O

channel, and bus controller board. Communication between the SLICs is

accomplished with a simple command-response packet carried over a

dedicated bus. The controller serves several functions. Firstly it is

the key element of the system's global interrupt system. Secondly, its

Page 61: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

CPU 1

/

Cache 1 / Cache cont­roller

. ~'."

CPU 2

r-­r--

Block marked invalid

I " ,.,.; .-..... ---.--.----- --::'--~

FIGURE 1.17: Bus-Watching Logic Monitor

Main memory

Block updated

" " ~

, I

.1 : )1 :

r;'

I'

Page 62: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Processor interface

Interrupt controller

I I

SLIC 1

~ L-l~~-~f,~,~·~~~§·~~_~·~~ ___ I~/_O ____ J~

~;- .

Semaphore cache

Receiver Transmi t ter~ contention

SLIC.1 bus

resolution

Start bit

SLIC packet

Message D t priority a a

Processor interface

Interrupt controller

I I

SLIC N

Error control

Semaphore. cache-

Receiver ~ransmitter ~ contention

resolution

J

-I/O --

I-

____ ~~SL-I-~~C-lO-C-k--------_________________ . ____ ~--~--------------------------

FIGURE 1.18: SLIC Chips

Page 63: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

46

purpose is to manage a cach~ of single-bit unit-secaphores. Finally,

the controller serves as a convenient communication path among modulesa

For example, system diagnostic and debugging routines take modules on­

and off-line using the SLIC bus, which carries error management

information.

The DYNIX operating system is an enhanced version of UNIX 4.2 BSD

that can also emulate UNIX system V at the system-call and command

levels. To support the Balance multiprocessing architecture, the DYNIX

operating system kernel has been made completely &~areable so that

multiple CPUs can execute identical system calls and other kernel code

simultaneously. The DYNIX kernel also adjusts the memory allocation

for each process to moderate the process's paging rate and to turn

virtual memory performance for the entire system.

Applications programming on the Balance 8000 system is supported

by compilers for the main programming languages. We can use a single

language or a combination of languages to suit. our application. Later

in Chapter 2, we will explain in detail information on the language

tools available for use with the DYNIX operating system.

Page 64: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS
Page 65: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

· CHAPTER 2

PARALLEL PROGRAMMING AND LANGUAGES

One must have. a good memory to be ab~e

to keep the promises one makes.

F.W. Nietzsche.

Page 66: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

2.1 INTRODUCTION

The recent advances in hardware technology and computer

architecture, lead to faster and more powerful parallel computer

systems, from which we benefit with a considerable throughput and

speed when they are applied to solve large problems. Problems for

parallel computer systems require some extra programming facilities

which come under the heading of parallel programming, to distinguish

it 'from the conventional programming of single-processor computers.

AS explained by the various architectures of existing parallel

computers, parallelism can be achieved in a variety of ways.

Attempting to summarize all these possible known ways of achieving

parallelism and categorize them into several distinct levels, w~

obtain:

a. Job level { b. Program level { c. Instruction level

d. Arithmetic and bit level

{

between jobs

between phases of. a job

between parts of a program

within Do-loop

between phases of instruction execution

between elements of a vector operation

within arithmetic logic circuits

The design of algorithms for parallel computers is greatly

influenced by the computer architectures and the high-level

47

Page 67: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

languages which have been used.

This chapter will elucidate parallel programming and parallel

algorithms.

48

Page 68: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

2.2 PARALLEL PROGRAMMING

The two new concepts behind the recent ideas of parallel

programming theory are parallelism and asynchronism of programs.

Gill [Gill, 1958] defined parallel programming as the control of

two or more operations which are performed virtually simultaneously,

and each of which entails following a stream of instructions.

49

There seem to have been two trends in the development of high­

level languages. Those that owe their existence to an application or

class of applications, such as FORTRAN, COBOL and C, and those that

have been developed to further the art of computer science, such as

. ALGOL , LISP, PASCAL and PROLOG. The development of the former to

some extent has been stifled by the establishment of standards.

Conversely, to some extent, the lack of standards and the desire to

invent have led to the proliferation of versions of the latter. An

attempt to produce a definitive language; incorporating the 'best'

features of the known art and to bind these into an all-embracing

standard was made by the U.S. Department of Defense, and the

resulting language ADA [Tedd et aI, 1984], has been adopted.

Although con currency is addressed in ADA, there is now far more

practical experience of concurrency and new languages have been

developed, such as OCCAM, which treat concurrency in a simpler,

more consistent, and more formal manner.

The numerous and vastly different applications and underlying

models of parallelism will require radically different language

structures. Hockney and Jesshope [Hockney, 1988] suggested three

major divisions in language development:

Page 69: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

1. Imperative languages.

2. Declarative languages.

3. Objective languages.

Imperative language is one in which the program instructs the

computer to perform sequences of operations, or if the system allows

it disjoint sequences of instructions operating concurrently_

Imperative languages have really evolved from early machine code,

by successive abstraction away from the hardware and its limited

control structures. This has had beneficial effects, namely the

improvement of programmer productivity and the .pcrtability obtained

by defining a machine independent programming environment. An

imperative language, however, even at its highest level of

abstraction, will still reflect the algorithmic steps in reaching a

solution. In addition to the retention of this notion of sequences,

these languages also retain a strong flavour of the linear address

space; still found in most machines. Harland [Harland, 1985]

introduced the concurrency, where many disjoint sequences of

instructions may proceed in parallel. By abstracting concurrency,·

the non-deterministic sharing of the CPU's cycles could be obtained.

The declarative style of programming has had the most profound.

effect on computer architecture research during this period. This

style of programming does not map well onto the classical von-Neumann

architecture, with its heavy use of dynamic data structures. It is

also based on a more mathematical foundation, with the aim of moving

away from descriptions of algorithms towards a rigorous specification

of the problem. These languages are based either on the calculus of

50

Page 70: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

functions, lambda calculus, or on a subset of predicate logic. Since

these declarative languages are based on mathematics, so it is

·possible to formally verify the software systems created with them.

A further advantage of the declarative approach is that such

languages can supply implicit parallelism, as well as implicit

sequencing [Shapiro 1984] •

51

Objective language is based on two main techniques, encapsulation

and inheritance, which is a more pragmatic foundation than the rigour

of logic or functional languages. Hence it can provide a potential

solution to the software problem. It also provides a model of

computation which can be implemented on a distributed system.

Encapsulation is the most straight-forward and is often used as a

good programming technique [Booch, 1986]. Encapsulation hides data

and gives access to methods or procedures to access that data which

is only provided through the shared procedures.

By encapsulating a programmer's efforts in the creation of such

constrained objects, a mechanism must be provided in order to enhance

that object, which is the second technique of objective languages.

Inheritance allows the programmer to create classes of objects, where

those classes of objects may involve common access mechanisms or

common data formats. The mechanism for implementing inheritance is

to replace the procedure call to an object by a mechanism involving

message passing between objects.

There are at least three emerging parallel software design

approaches based upon the concealment of the parallelism by the

hardware structure. In other terms, for some architectures, the

parallelism is hidden by the hardware itself whilst for others it is

Page 71: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

revealed to the user so that appropriate decisions are made as and

when needed. The first of these approaches, is the automatic

translation of sequential programs or the implicit parallelism.

The second approach is explicit parallelism, in which the programmer

manages the concurrency of the applications by coding directly in a

concurrent language. The third approach, advocated by Backus and

Dennis [Backus, 1978] is based on the functional language model and

is implemented on most DF computers. Relyirig on the programmer's

ability, the former method could rapidly become unworkable as it is

impossible to keep "juggling" with a large nuinber of tasks. The

functional approach, which is the most natural form of handling

parallelism can achieve the highest degree of concurrency, since the

instructions are scheduled for execution directly by the availability

of their operands. However, the high cost of implementing this

unstructured low-level con currency makes this method of less

importance, at least for the pr~sent moment.

The explicit and implicit parallelism detection approaches will

be discussed in paragraphs (2.2.1) and (2.2.2).

2.2.1 Implicit parallelism

52

Many of the existing sequential software exhibits naturally some

form of synchroneity which needs only to be identified and then

exploited in the. design of parallel algorithms. The implicit approach

is one of the approaches to parallelism that relies on the implicit

detection of parallel processable tasks within a sequential algorithm.

This approach of parallelism is associated with sophisticated

compiling and supervisory programs and their related overheads. Its

Page 72: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

effectiveness lies in that it is independent of the programmer, and

existing programs need not be modified to take advantage of inherent

parallelism. However, in implicit parallelism, it will be necessary

to analyse the program to_see how it can be divided into tasks, and

then the compiler could detect parallel relationships between tasks

as well as carrying out the normal compiling work for a program to be

run on a serial computer.

53

Different methods have been developed which possess the feasibility

of automatically recognising parallelism in computer programs.

Bernstein [Bernstein,· 1966] proposed a method based on set theory.

His presented theory is based on four different ways of utilizing a

memory location by a sequence of instructions or tasks. These

conditions are:

1. The location is only fetched during the execution of a task.

2. The location is only stored during the execution of a task.

3. The first operation involving this location is a fetch.

One of the succeeding operations stores in this location.

4. The first operation involving this location is store. One

of the succeeding operations fetches from this location.

Following this work, Evans and Williams [Evans, 1978] have

described a method of locating parallelism within ALGOL-type

programming language , where some constructs such as loops, if

statements, and assignment statements are studied.

One of the most studied detection schemes that has been given

much consideration is the implicit detection of the inherent

parallelism within the computation of arithmetic expressions.

Page 73: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Because of the sequential nature of most of the uniprocessor systems,

the run-time of any arithmetic expression computation is always

proportional to the number of operations. This run-time can be

further reduced on a parallel system by concurrently processing many

parts of the expression. In fact, the commutativity and associativity

were extensively used in order to reduce the height of the computational

tree representation. For example,· consider the expression,

(a*b*c*d*e*f*g*h)

which can be rearranged in a form suitable for parallel processing,

«(a*b)*(c*d))*«e*f)*(g*h)))

As it can be seen in Figure 2.1 and 2.2 which depicts the tree

representation of the above expression for a sequential and a parallel

processor respectively, the run-time was reduced by four time units.

Many algorithms have previously been proposed for recognizing

parallelism at the expression level, some of which are those

suggested by Squire [Squire, 1963], Hellerman [Hellerman, 1966],

Stone [Stone, 1967], Baer and Bovert [Baer, 1968), Kuck [Kuck, 1977],

and Wang and Liu [Wang, 1980].

2.2.2 Explicit Parallelism

In explicit parallelism, the programmer has to specify explicitly

those tasks that can be executed synchronously by means of special

parallel constructs added to a high-level programming. language.

Although these programming constructs can be time consuming and

difficult to implement they can offer significant algorithm design

flexibility.

Page 74: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Level 7

Level 6

Level 5

Level 4 f

Level 3 e

Level 2

Level 1 c

Level 0 a b

FIGURE 2.1: Binary Tree Representation of the Expression (a*b*c*d*e*f*g*h) for a Serial Computer.

a b c d e f

9

FIGURE 2.2: Binary Tree Feoresentation of the Expression (a*b*c*d*e*f*g*h) for a Parallel Computer.

55

9 h

Page 75: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

56

Significant research has been done on this approach with a

particular interest on those parallel task issues such as task

declaration, activation, termination, synchronisation, and

communication. In other terms, a synchronous program consists of

sequential processes that are carried out simultaneously. These

processes cooperate on common tasks by exchanging data through shared

variables.

Dijkstra [Dijkstra, 1965J proposed the utilization of semaphores

and introduced two new primitives (P and V) that greatly simplified

the processes of synchronisation and communication. A software

·implementation of these two primitives in terms of an indivisible

instruction, the test-and-set instruction, was installed in many

systems.

Dennis and van Horn [Dennis, 1966J proposed a very straightforward

mutual exclusion lock out mechanism. Critical regions are enclosed

within a LOCK W-UNLOCK W pair, where W is an arbitrary one-bit

variable.

The parallelism in the explicit approach can also be indicated

by using language constructs that exploits the parallelism in

algorithms, Anderson [Anderson, 1965J introduced five parallel

constructs, the FORK, JOIN, TERMINATE, OBTAIN and RELEASE statements

which are presented below in an ·ALGOL-68 format;

label:

label: TERMINATE L ,L2

, .•. ,L ; 1 n

Page 76: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

The FORK statement initiates a separate control for each of the

n segments with the labels L .• Only local labels must be used and ~

their scope is defined as the block scope in which this statement

is declared. The next sequence of paths may only be initiated when

all the forked paths of the previous level have completed their

performances.

The JOIN statement which is associated with the FORK statement

and must occur in the same level in the program is used to terminate

the parallel processes that have been forked. This action is

57

implemented by including a code that causes test bits to be available,

thus allowing the forked paths to be ~ynchronised after they are

completed (Figure 2.3).

The TERMINATE statement is used to explicitly terminate program

paths which have been dynamically activated by the fork statement,

thus avoiding the creation of a backlog of meaningless incomplete

activations. Really the JOIN and TERMINATE statements are control

counters, decreasing by one after the execution of one statement,

comparing each time to zero; if different than zero, the path is

terminated and the processor is free to execute the next path in the

queue, otherwise, the processor goes to the next program segment.

The OBTAIN statement is used to provide exclusive access to the

listed variables by a single process. It is used to avoid mutual'

interference by locking-out other parallel program paths from the use

of these variables. If this statement occurs in a block then the

variables should be the same variables occurring in higher level

blocks.

) )

Page 77: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

58

FIGURE 2.3: The Fork/Join Technique.

Page 78: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

The RELEASE statement which is implemented with the OBTAIN

statement. It releases those variables ·that have been locked out by

an OBTAIN statement.

More specifically the OBTAIN/RELEASE concept is an approach

implemented to assist in solving the synchronization problem.

59

The above presented statements are directly implemented as library

functions and supplied with enough information to control parallel and

multiprogramming activities. In general, the FORK statement would be

substituted at the compilation time by a special code that when

executed would create as many parallel paths as the number of labels

following the FORK statement. Each of these paths is assigned to the

available processors and usually the first path is carried out by the

same processor that carries out the FORK statement itself. If the

number of created processes ~s greater than the number of available

processors, the excess paths are kept in a queue until a processor

becomes free.

The labels used in a parallel program are cross-referenced at

the compilation time by. arranging them on a forward reference list

which is loaded by all the labels contained in the labels list of an

instruction.

To recapi·tulate, in the explicit approaCh the process of

parallelization is under the entire responsibility of the programmer,

·a fact which jeopardizes program determinancy.

Page 79: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

60

2.3 PROGRAMMING THE BALANCE SYSTEM

The hardware configuration and the operating system of the Balance

MIMD system was described in Chapter 1. This section describes the

programming languages supported by Sequent for use with the DYNIX

operating system. These languages include C, FORTRAN77, pascal; and

assembly languages.

The Balance system supports the two basic kinds of parallel

programming: multiprogramming.and multitasking.

Multiprogramming is an operating system feature that allows a

computer to execute multiple unrelated programs concurrently. The

multi-user, multiprogramming UNIX environment adapts quite naturally

to the Balance multiprocessing architecture and automatically

schedules processes for optimal throughput. In other versions of the

UNIX operating system, executable processes wait in a run queue; when

the CPU suspends or terminates execution of one process, it switches

to the process at the head of the run queue. DYNIX balances the

system load among the available processors, keeping all processors

busy as long as there is enough work available, thus using the full

computing capability of each processor.

Multitasking is a programming technique that allows a single

application to consist of multiple processes executing concurrently.

The DYNIX operating system automatically does multitasking for some

applications.

The Balance language software includes multitasking extensions

to C, pascal, and FORTRAN. The DYNIX Parallel programming Library

(PPL) includes routines to create, synchronize, and terminate parallel

processes from C, Pascal and FORTRAN programs. The DYNIX gprof

Page 80: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

61

utility creates a programs execution profile, a listing that shows us

which subprograms (subroutines or functions) account for most of a

program's execution time.

2.3.1 Multitasking Terms and Concepts

In the DYNIX operating system, a new process is created by using

a system call called a fork. The new (or child) process is a

duplicate copy of the old (or parent) process, with the same data,

register contents, and program counter. If the parent has files open

or has access to shared memory, the child has access to the same files

and shared memory.

A UNIX forking operation is relatively expensive (about 55

millisecond). Therefore, a parallel application typically forks as

many processes as it is likely to need at the beginning of the program,

and does not terminate any process until the program is complete,

since the process can wait in a busy loop during certain code sequences.

Typically, multitasking programs include both shared and private

data. Shared data is accessible by both parent and child processes,

while private data is. accessible by only one process. There are

several advantages to sharing data. Firstly it uses less memory than

having multiple copies. Secondly it avoids the overhead of making

copies of the data for each process. Finally it provides a simple

and efficient mechanism for communication between processes. If the

program includes any shared data, the process's virtual memory space

also contains a shared data area and a shared heap (Figure 2.4).

Tasks can be scheduled among processes using three types of

algorithms,

Page 81: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Shared Memory

Shared

Data

Process 1

Private Data

, ! Private i

I Heap ;

! __ I - .. - r ,- -- - - --

I

I Shared 1

i Data i

- - - -- -,- - .J -- -,

i 1 Shared I i i , Stack I _____ ..J

,------;

I Shared 1<----+----'1 Heap

, I I

i_ - l--.J l'

Private Stack

FIGURE 2.4: Process Virtual Memory Contents.

62

Page 82: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

1. prescheduling;

2. static scheduling;

3. dynamic scheduling.

In prescheduling, we need to determine the task division, before

compiling the program. Prescheduled programs cannot automatically

balance the computing load according to the data or the number of

CPUs in the system. Therefore, this method is appropriate only for

applications where each process is performing a different task.

63

In static scheduling, the tasks are scheduled by the processes at

run time, but they are divided in some predetermined way. For example,

if a program includes lOO-iterations, using static scheduling, each

process might execute 10 iterations of the loop, if the program uses

10 processes.

In dynamic scheduling, each process schedules its own tasks at

run time by checking a task queue or a "do-me-next" array index.

For example, a dynamically scheduled program might perform a matrix

multiply, with each process computing three matrix elements and then

returning for more until all the work is done.

Dynamic scheduling produces· dynamic load balanCing, i.e. all

processes keep working as long as there is work to be done, while

static scheduling produces static load balancing, i.e. the division

of tasks is statically determined, several processes may stand· idle

while one processor completes its share of the job.

In any place where two or more parallel processes can read and

write the same data structure constitutes a dependency, because the

results of the program depend on when a given process references that

Page 83: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

64

data structure. To ensure correct results, code sections containing

dependencies cannot be executed simultaneously by multiple processes,

where the code section, that contains dependencies are critical regions.

There are two basic types of dependencies:

1. access dependencies;

2. order dependencies.

Access dependencies can yield incorrect results if two or more

processes try to access a shared data structure at the same time,

whilst the order dependencies can yield incorrect results if two or

more processes try to access a shared data structure at the same time

or in the wrong order.

The best way of handling dependencies in a program is rewrite

the code to eliminate them. The dependencies are sometimes inherent

in the application. In these instances, we must set up the processes

so that they communicate with each other to execute the dependent code

sections. This communication can ,be set up by using mechanisms

called semaphors and locks)· dlhere the .semaphore is a shared data

structure used to synchronize the actions of multiple cooperating

processes. Meanwhile the lock ensures that only one process at a time

can access a shared data structure. The lock has two values: lock

and unlock. Before attempting to access a shared data structure, a

process waits until the lock associated with the data structure is

unlocked, indicating that no other process is accessing the data

structure. The process then locks the lock, accesses the data

structure and unlocks the lock. While a process is waiting for a

lock to become unlocked, it spins in a tight loop, producing no work -

Page 84: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

hence the name "spinlock ll• This spinning is also referred to as a

busy wait.

Figure 2.5 illustrates how a lock is used to prevent multiple

processes from executing a dependent section simultaneously.

acquire lock execute

release lock

Pi -------

P2

p3

dependent section

I

I , wai t for I execute ___________ 1 _______________ _ ,

dependent: section I

lock acquire lock

release lock

I wait for lock ,execute

----------_______________ 1 __________ _

65

)

,

release lock

)

acquire dependent lock section

time )

FIGURE 2.5: Role of Lock in Protecting Dependent Sections.

An Event is something that must happen before a .task or process

can proceed. Events have two values: posted and cleared. One or

more processes wait for an event until another process posts the

event and hence the waiting processes proceed. It is required to be

cleared by the waiting process; by a master process or by another

process.

I/O in parallel programs is complicated. These complications

can usually be reduced by performing I/O only during sequential

phases of the program or by designating one process as a server to

perform all I/O.

Page 85: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

2.3.2 Data Partitioning with DYNIX

This subsection explains how we structure FORTRAN programs for

data partitioning. It also explains how we shall use the DYNIX PPL

to execute loops in parallel. The data partitioning method is

sometimes called microtasking. Microtasking programs create multiple

independent processes to execute loop iterations in parallel. It has

the following characteristics:

1. the parallel processes share some data and cr"eate their

own private copies of other data;

2. the division of the computing load adjusts automatically

to the number of available processes;

3. the program controls data flow and synchronization by using

the tools specially designed for data partitioning.

The microtasking program works as follows:

a. Each loop to be executed in parallel is contained in a

subprogram •

b. For each loop, the program calls a special function which

forks a set of child processes and assigns an identical

copy of the subprogram to each process for parallel

execution. The special function creates a copy of any

private data for each process.

c. Each copy of the subprogram executes some of the loop

iterations.

d. " If the loop being executed in parallel is not completely

independent, the subprogram may contain calls to functions

that synchronize the parallel processes at critical points

by using locks, barriers, and other semaphores.

66

Page 86: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

e. When all the loop iterations have been executed, control

returns from the subprogram. At this point, the program

either terminates the parallel processes or leaves them to

spin in a busy-wait state until they are needed again.

67

There are three sets 'of routines in the DYNIX PPL, a microtasking

library, a set of routines for general use with data partitioning

programs, and a set of routines for memory allocation in data

partitioning programs.

The microtasking library routines allow us to fork a set of

child processes, assign the processes to execute loop iterations in

parallel, and synchronize the processes as necessary to provide proper

data flow between loop iterations. Table 2.1 lists the microtasking

routines in the PPL.

The data partitioning routines include a routine to determine

the number of available CPUs and several process synchronization

routines that are more flexible than those available in the micro­

tasking library. Table 2.2 illustrates the general-purpose data

partitioning routines in the PPL.

The memory allocation routines allow a data partitioning program

to allocate and de-allocate shared memory and to change the amount of

shared and private memory assigned to a process. Table 2.3 shows the

memory allocation routines in the PPL. More detail on PPL can be

found in Section 3p in Volume 1 of the DYNIX Programmer's Manual.

Before we convert a loop into a subprogram for data partitioning,

we must analyze all the variables in the loop and determine two things:

Page 87: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

68

ROUTINES DESCRIPTIONS

m fork Execute a subprogram in parallel

Return process identification number

Return number of child processes

Terminate child processes

m lock Lock a lock

m multi End single_process code section

m next Increment global counter

Suspend child process execution

Resume child process execution

Set number of child processes

Begin single-process code section

Check in at barrier

m unlock Unlock a lock

TABLE 2.1: Parallel Programming Library Microtasking Routines

Page 88: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

69

ROUTINES DESCRIPTIONS

: Return number of CPUs on_line

s init barrier Initialize a barrier

s init lock Initialize a lock

s lock or s clock Lock a lock

S LOCK C macro for s lock

s unlock Unlock a lock

s UNLOCK C macro for s unlock

5 wait barrier Wait at a barrier

TABLE 2.2: Parallel Programming Library Data-Partitioning Routines

ROUTINES DESCRIPTIONS

brk or sbrk Change private data segment size

shbrk or shsbrk Change shared data" segment size

shfree Do_allocate shared data memory

shmalloc Allocate shared data memory

TABLE 2.3: Parallel Programming Library Memory-Allocation Routines

Page 89: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

a. Which data can be shared between parallel processes and

which must be local to each parallel process.

b. Which variables cause dependencies· or criticaL regions,

code sections which can yield incorrect results when

executed in parallel.

To complete the development of our data partitioning program, we

need to do the following things:

1. Invoke the appropriate compiler with the proper options to

link the program with the PPL.

2. Execute the program and check the results.

3 .. If necessary, we use the DYNIX parallel symbolic debugger,

Pdbx to debug the program.

Thus, if we need to compile and link a FORTRAN program, we enter

the following command:

fortran-e-F!SHCOM!program.name -lpps,

70

this command compiles a FORTRAN source file and links the·object.code

with the PPL, producing an executable file named (a.out). It also

places all COMMON blocks declared with the (-F) option into shared

memory. The (-e) option makes the FORTRAN code compatible with the C

subroutines in the PPL. To execute the program, simply enter the name

of the executable file as· a DYNIX command. The default file name is

a. out.

2.3.3 Function Partitioning with DYNIX

This subsection describes the facilities provided by the Balance

system to support function partitioning applications. Function

Page 90: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

71

partitioning involves creating multiple processes and having them

perform different operations on the same data set. The processes may

be created within a single program or they may be independent programs

created at the operating system level. The DYNIX PPL contains several

routines that can be used for function partitioning applications.

The fork( ) system call creates a duplicate copy of the current

process. The parent process sets up a shared memory region and one or

more locks, then forks one or more child processes to Shar~ the work.

The children inherit the parent's complete memory image', including

access to shared memory and·locks. Child processes are identical to the

parent and they can be designed to choose their own tasks based on the

order of their creation. The new process is created from a file that

. contains either executable object code or a shell script. The PPL

routines s init barrier and s wait barrier initialize a barrier and - -cause processes to spin until all related processes arrive at the

synchronization point. Processes can send and receive signals among

themselves, and'to handle special events such as terminal interrupts.

If a child process determines that the parent will not need any help

for a significant amount of time, the child can relinquish its processor

for use by other applications. The parent can send the child a wakeup

signal when required. Since a Balance system can have multiple

processes running simultaneously, some programs that use UNIX signals

may behave differently on the Balance system than on a uniprocessor.

For example, if each of P child processes sends a signal to their

parent, the parent will not necessarily receive P signals. This type

of race condition is also possible on uniprocessors, but may not

manifest itself until the program is parted to a multiprocessor.

Page 91: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

72

The simplest and most efficient mechanism for interprocess

communication is a semaphore in the Balance shared memory. The

interprocess communication subsystem provides the ability to transfer

data directly between processes using system calls. Finally the

interprocess communication facilities are extremely useful for certain

types of applications.

Page 92: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

73

2.4 PARALLEL ALGORITHMS

Long before parallel computers had been constructed many

researchers have studied parallel algorithms (since the 1960's).

However, designing parallel algorithms became more important and

interesting as the development of parallel computer architecture

advanced. Therefore, a variety of algorithms have been designed from

different view points and for the various parallel architectures which

were described in Chapter 1.

The structure of parallel algorithms has to be such that they are

fully independent computations. Although this case cannot always be

encountered, some algorithms may already contain independent

computations that are not needed to be reorganized. Such algorithms

are said to have inherent parallelism.

Stone [Stone, 1973] highlights some of the problem areas in

parallel computation, these include the necessity to rearrange the

data in memory for efficient parallel computation: the recognition that

efficient sequential algorithms are not necessarily efficient on

parallel computers and conversely, that sometimes inefficient sequential

algorithms can lead to very efficient parallel algorithms and lastly

the possibility' of applying transformations to sequential algorithms

to yield new algorithms suitable for parallel execution.

Kung [Kung, 1980] identified three orthogonal dimensions of the

space of parallel algorithms: concurrency control, module granularity

and a communication geometry. Concurrency control is needed in

parallel algorithms to ensure the correctness of the concurrent

execution, because more than one task module can be executed at a

time. The module granuU2rity.of a parallel algorithm reflects whether

Page 93: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

74

or not the a·lgorithm tends to be communication intensive. This must

be taken into consideration for efficiency reasons. If the task

modules of a parallel algorithm are connected to represent intermodule

communication, then a geometric layout of the resulting network is

referred to as the communication geometry of the algorithm.

As was exemplified by the various architectures of existing

parallel computers, parallelism can be achieved in a variety of ways,

the ·same holds with parallel algorithms, since a close correspcndence

must exist between architectures and algorithms. Consequently, since

there exists such a variety, the question to answer now is, "How to

choose amongst the different alternatives·in order to solve a specific

problem, or what type of problem is better adapted to a given

architecture?"

Since, in most cases, performance is the reason why parallelism

is being investigated, it must be considered as a very critical issue.

The study of how to design algorithms, for different parallel

architectures, might reveal that an algorithm requires a peculiar

feature of that architecture to run efficiently ..

As mentioned previously, in SIMD computers, the number of

processors tends to be large compared with that of MIMD computers.

In general, we can say that the algorithm designed for SIMD comput·ers·

requires a high degree of parallelism because this type have up to·

order pm,· i.e. o (pm) processors, while a MIMD computer has up to O(p)

processors, where p is the number of subtasks of the problem and m is

an integer index representing problem complexity. This does not mean

that an algorithm designed for a MIMD computer cannot be run on a

SIMD computer.

Page 94: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

75

Since the processors of a MIMD computer are asynchronous, they

need not necessarily be involved on the same problem. On the other

hand, the processors of a SIMD computer are synchronous and hence they

cannot be used to run independent computations that are not identical

and they must remain idle when not required.

The performance of an algorithm is defined by the absolute

arithmetic answer to some relevantly set quantities, such as, the

Computation time, Speed-up, and Efficiency of the algorithm. In real

. machines, actual computation times are often proportional to the total·

. number of arithmetic operations in the programs, Whilst, in cases of

programs with little arithmetic, are proportional to the number of

memory accesses, or the n~mber of I/O transmissions. We will use T p

to denote the computation time on a computer with P processors, and

Tl to denote the computation time of an uniprocessor computer.

The term Speed-up (S ) of a P processor computer over a sequential p

computer is defined as,

S Tl

~l = , P T p.

and the Efficiency (E p

) is defined as,

S E = J>.

~ 1 P P

,

and to compare two parallel algorithms for the same problem the

following measure of effectiveness (F ) is introduced, p

where,

F P

c p

S =...l:'.

c p

PT P

~ 1 ,

(2.4.1)

(2.4.2)

(2.4.3)

(2.4.4)

Page 95: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

measures the cost of the algorithm.

The following simple example is given for illustration. Suppose

that it is required to form the inner or scalar product,

16 A 2

i=l a.b. ~ ~

To carry this out with a single processor sequentially requires 16

multiplications and 15 additions. If we take the number of °additions

(assume 1 multi. ~ 1 addition)", as a measure of the time needed, then

76

normalising by assuming the time for a single addition to be the unit,

owe have Tl =31. If two processors were available we would form the

two products·,

simultaneously, requiring 15 time units, and then form,

d

at the next stage, requiring a further time unit. Thus A would be

obtained in T2

=15+1=l6 time units.

If now we had three processors A could be formed in the following

three stages:

Cl = albl+a2b2+a3b3+a4b4+a5b5

C2 = a6b6+a7b7+aebe+agbg+alOblO

C3 allbll+a12b12+a13b13+a14b14+a15b15

d l = C

l+C

2 d2

= c3+a16b16 ,

el

dl

+d2

= A •

Page 96: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

This requires T3

=9+3+l=13 time units.

If four processors were available then A could be formed in the

following stages:

This requires T4

=7+1+1=9 time units.

A table of the performance measures can now be constructed.

Table 2.4 shows that with increasing p, S increases steadily, while . p

E decreases. p FpTl' however, has a maximum when p=8 which indicates

that p=8 is the optimal choice of number of processors for this

calculation.

P T C S E P P P P

·1 31 31 1 1

2 16 32 1.93 0.96

3 13 39 2.38 0.79

4 10 40 3.10 0.77

8 6 48 5.16 0.64

16 5 80 6.2 0.38

16 TABLE 2.4: The Performance Measure of I aib

i.

i=l

F T =S E pIp P

1

1.85

1.88

2.38

3.30

2.4

77

Page 97: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

78

We can find in the literature that many algorithms are suitable for

SIMD computers. Examples of such algorithms have been introduced by

Miranker [Miranker, 1971], Stone [Stone, 1971,1973b] and Wyllie

[Wyllie, 1979]. On the other hand, although implementing algorithms

on MIMD computers are more difficult than their implementation on SIMD

computers, many parallel algorithms have been developed to run on MIMD

computers. 'For example, Muraoka [Muraoka, 1971] showed how

parallelism 'is exploited in applying algebraic expressions on a MIMD

computer. Baudet et al [Baudet, 1980] has developed a variety of

parallel iterative algorithms suitable for MIMD computers.

2.4.1 The Structure of Algorithms for Multiprocessor Systems

A parallel algorithm for a multiprocessor is a set of n concurrent

processes which may operate simultaneously and cooperatively to solve

a given problem. Synchronization and the exchange of data is needed

between processes to ensure that the parallel algorithm works correctly

and effectively to ,solve a given problem. Therefore, at some stage in

the performance of a process there may be some pcints where the

processes communicate with other processes. These points are called

the "interaction points". The interaction pcints divide a process

,into stages. Hence, at the end of each stage, ,a process may

communicate with some other processes before the next stage of the

computation is initiated.

Parallel algorithms for multiprocessors may be classified into

asynchronous and synchronous parallel algorithms. Due to the inter­

actions between the processes, some processes may be blocked at

certain times. The parallel algorithm in which some processes have to

Page 98: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

wait on other processes is called a synchronized algorithm. The

weakness of a synchronized algorithm is that all the processes that

have to synchronize at a given point wait for the .slowest amongst

them. To overcome this problem, an asynchronous algorithm is

suggested. For a parallel asynchronous algorithm, processes are

generally not required to wait for each other and communication is

achieved by using global variables stored in shared memory. Small

delays may occur because of concurrent accesses to the shared memory.

/

79

Page 99: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS
Page 100: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

CHAPTER 3

BASIC MATHEMATICS, GENERAL BACKGROUND

Most nwnerical analysts have

no interest in arithmetic.

B. Parlett.

Page 101: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

3.1 INTRODUCTION

In the present Chapter we shall scrutinize the necessary

preliminary mathematical background of differential equations, and

review all the related notations, conditions, and concepts essential

for their proper use. The mathematical formulation of any important

- scientific and engineering problem.· involving rates of change with

respect to two or more independent variables, leads to Partial

Differential Equations (POE's) or a set of such equations. Another

class of differential equations which govern physical systems is

Ordinary Differential Equations (ODE's) in which only one independent

variable is present in the differential equations.

80

With the use of automatic digital computers becoming widespread,

the numerical methods are found to be an attractive alternative, since

the analytical solution for the majority of these equations is

.extremely difficult or too inconvenient to be obtained. A number of

approaches have been developed over the years for the treatment of

POE's, the most important and wideiy used of these are the methods of

finite elements and finite differences. A compleoentary description of

the finite difference methods will now be given.

Page 102: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

81

3.2 CLASSIFICATION OF PARTIAL DIFFERENTIAL EQUATIONS

The most general mathematical form of the second-order PDE's of

two independent variables, x and y, (these may both be space

coordinates, or one may be a space coordinate and the other the time

variable), with a dependent variable U, which can be expressed as,

a2 . 2 + B ____ U + ~ +

axay a/ ~+

ax Eau

ay +Fu+G= O. (3.2.1)

Equation (3.2.1) can be further classified according to the nature

of its coefficients:

1. Linear, if the coefficien~s A,B,C, ... ,G, are constants or

functions of one or both independent variables x and y.

2. Non-linear, if any of the coefficients A,B,C, ..• ,G, are

functions of the dependent variables, U, or its derivatives.

3. Semi-Linear, if the coefficients A,B, ... ,G, are functions of

the independent variables x and y only.

4. Quasi-linear, if the coefficients A,B, and C, are functions

of x,y,U, ;~, and ;~ but not of second-order derivatives.

5. Homogeneous, if G=O, otherwise it is called inhomogeneous.

6. Se If-Adjoint , if it can be replaced. by

a au + --(C(y)--)·+ FU + G ay ay

o

Additionally, equation (3.2.1) can be classified into three

particular types according to whether the discrLffiinant (~4AC) is

greater than, equal to, or less than zero. The general second-

order quasi-linear PDE has the form,

2 ~+

axay

2 c~ al

+ G o , (3.2.2)

Page 103: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

82

au au where A,S,e, and G may be functions of x,y,U,-- and -- but not the ax ay

second-order derivatives. In this section, let us consider the

following denotation for the first- and second-order derivatives,

2 a

2u a

2u au au a u = M; = N; -2 = P; = Q, and -- = S. ax ay axay 2 ax ay

Let R be a curve on the x-y plane on which the values of its derivatives

above satisfy equation (3.2.2). Therefore, the differentials of M and

N in directions tangential to R satisfy the equations,

aM dx aM Pdx + Qdy dM = + -- dy , ax ay (3.2.3)

and

dN aN dx aN Qdx + Sdy = + -- dy = , ax ay (3.2.4)

where

AP + BQ + CS + G = 0 , (3.2.5)

and ~ is the slope of the tangent to R at point p(x,y).

Elimination of P and S from (3.2.5) using (3.2.3) and (3.2.4)

results. in,

~(dM-QdY) + BQ + d~(dN-Qdx),+ G = 0 ,

i. e. I

B (dy) + C} _ {A dM ~ + ~ + Gdy} = 0 dx dxdx dx dx . (3.2.6)

Now, by choosing the curve R so that the slope of the tangent,

at every point on it, is a root of the equation,

B (~) + C = 0 J (3.2.7)

hence, the Q term is eliminated.

Therefore, equation (3.2.6) leads to,

Page 104: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

83

A dM ~ + CdN + Gdy dx dx dx dx ° . (3.2.8)

Consequently, it is apparent that every point p(x,y) of the

solution domain there are two directions, given by the roots of equation

(3.2.7), along which there is a relationship, given by equation (3.2.8),

between the total differentials dM and dN with respect to x and y.

The directions given by the roots of equation (3.2.7) are called

characteristic directions and the PDE is considered to be hyperbolic,

parabolic, or elliptic according to whether these roots are real and

distinct, equal, or complex, respectively, i.e. according to whether

2 > B-4AC = 0. <

The above classification scheme is rather interesting, since the

coefficients A·,B and C are functions of the independent variables x and

y and/or the dependent variable U, thus this classification depends in

general on the region in which the PDE is defined.

For example, the differential equation,

° , 2 2

is hyperbolic in the region when x-y >0, parabolic along. the

2 2 2 2 boundary x -y =0, and elliptic in the region where x -y <0.

Typical, parabolic and hyperbolic PDE's result from diffusion,

equalization or oscillatory processes and the usual independent

variables are time and space. On the other hand the elliptic PDE is

generally associated with steady-state or equilibrium problems.

Page 105: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

3.3 TYPES OF BOUNDARY CONDITIONS

The solution of a PDE has to satisfy, in particular, some

boundary conditions arising from the formulation of the problem

itself. Usually, the elliptic PDE's are classified as boundary value

problems, since as in Figure 3.1, the boundary conditions. are given.

round the (closed) region.

y

U=f on aR br-----------------,

O~--------------~----__. a

x

FIGURE 3.1: Boundary Value Problem.

The parabolic and hyperbolic types of equations are either

initial value problems or initial boundary value problems, where the

84

initial or/and boundary conditions are supplied on the side of the open

region, and the solution proceeds towards the open side (see Figure

3.2a,b and 3.3a,b).

In accordance with the specific type of the occurred boundary

conditions defined on the boundary oR of the region R, many different

basic categories of problem can be distinguished. These are:

1. DirichZet probZem, where the solution u is specified at each

point on oR.

Page 106: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

. U(O,t) given

for t>o

t Open

1 aU =--k2 at U(a,t) given

for t>O

° I---------..Jal,-~ .... x

U(x,O) given

on [O,a)

(a)

t

85

L-~ ___________ -+ X

U'(x,O) given on the

entire initial" line

(b)

FIGURE 3.2: Initial Boundary Value Problem for a Parabolic Equation

U(O,t) given

for t>o

t

°

Open

U(a,t) given for t>O

t

U(O,t) given

for t>o

~--------~-~. x aU .

U and at gl.ven a

on

t=ox [0, aJ

(a)

2 1 a U

c 2 at2

L----------~x aU

U and at given on

t=Ox[O,w)

(b)

FIGURE 3.3: Initial Boundary Value Problem for Hyperbolic Equation

Page 107: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

2. Neumann problem, where values of the normal derivatives ~~ are

aU given on aR, where ~ denotes the directional derivative of U

along the outward normal to aR.

3. Mixed problem, where the solution u is specified on part of aR

and ~~ is specified on the remainder of aR.

4. Periodic problem, the solution u has to satisfy the periodicity

conditions, for example,

I aul ul x = u x+£' an x

aul = an x+£

where £ is the period, x and x+£ are on aR.

5. Robin's problem, where a combination of U and its derivatives

are given along the boundary, i.e., aU + eau given on aR. an

86

Page 108: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

87

3.4 BASIC MATRIX ALGEBRA

Numerical approaches such as the finite difference and finite

... """~ element methods to solve ODE's and. PDE'sTyields a system of linear,

simultaneous equations which can be represented as a matrix system.

Methods of solving such systems depend on some matrix properties, for

example, irreduoibility, diagonal dominanoe, and positive definiteness

of the coefficient matrix of the system.

A matrix is defined as a two-dimensional array with each element

denoted as a, "where i and j represent the row and the column 1.,J

respectively of the array in which the element appears. A matrix A,

say, is of size (nxm) if it has n 'rows and m columns, and can be

denoted'by,

A = [a, ,1 = l. , J

rall la21 I I

I I I

~~l

a - ______ a 12 Im

I a -_

n2 -a run

when n~l, then we'have a row veotor, and for m=l, then it is a

(3.4.1)

oolumn veotor. The vectors are usually denoted by a small underlined

letter with a singl'e subscript, such as b, represents the i th element l.

of the vector b. The vector £, for example, whose elements are b ,b , . 1 2

... ,b is of order n and is. denoted by, n

b2 ,

b , (3.4.2) , I ,

ti n

Page 109: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

A square matrix is a matrix of order n, where n=m. In this

thesis, all matrices used are square matrices, unless otherWise

stated. The set of elements a, .' where i=1,2, ... ,n of the matrix J.,J.

A (3.4.1) is the diagonaZ of A. The transpose of the matrix A is

denoted by AT and is obtained by interchanging the rows and columns

of A. The determinant of A is denoted by det(A) or jAI. An inverse,

-1 A , of a given matrix A, if it exists, is a square matr·ix such" that,

-1 -1 A A AA = I ,

where I is the identity (unit) matrix whose order is the same as that

of A and is defined as follows,

a .. = 1, for all i=1,2, ... ,n, J.,J.

a .. ; 0, for all i,j;1,2, ... ,n and i,lj. J.,J

If A possesses an inverse then it is non-sin~AZar othe~ise it

is singuZar. On the other hand, A is singular if IAI;o, and non-

Singular if IAI,Io.

If the entries of a matrix A are complex numbers, the conjugate

of A is the matrix A whose entries are the conjugates of the

corresponding entries of A, i. e., if A; [a. .1 then A; (a. .1. The 1,J 1,J

Hermitian transpose (conjugate transpose) of A, d~oted by AH, is the

transpose of A, i.e.,

H -T T A ; (A) ; A

The sum of the diagonal elements of a matrix A is called the trace of n

A, denoted by tr(A) , i.e. , tr (A) ; I a. .. i=1 1.,1.

88

Page 110: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

A permutation matrix P=[P .. J, is a matrix which has elements of l.,)

zeros and ones only with exactly one non-zero element in each row and

column. For any permutation matrix P we have,

T = P P I ,

hence -1 = P

Definition 3.1:

1.

2.

3.

4.

5.

The matrix A=[a .. 1 is said to be: l.,)

T Symmetric, if A=A ;

. -1 T Orthogonal, if A =A

.. l.. f H Herm~t1an, A =A;

Null, if a .. =0, (i=j=1,2, ... ,n); l.,)

Sparse, if a relatively large number of its elements a. . are l. , )

zero;

6. Dense (full), if a relatively large number of its elements a. . l.,)

are non-zero;

7. Diagonal, if a. .=0 for i;ij; l. , )

8. Lower triangular, if a =0 for i,j

9. Upper triangular, if a. .=0 for l., )

Definition 3.2:

The matrix A=[a .. 1 is called: l.,)

i<j;

Dj.

1. Banded, if a .. =0 for ji-jj>r, where (2r+l) is the bandwidth of A; l.,)

2. Tridiagonal, if r=l;

3. Quindiagonal, if r=2;

4. Block diagonal, if each Di

, where i=1,2, ... ,r (Figure 3.4d) is a

square matrix.

89

Page 111: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

rx I x x

x x x X \ 0 \ \ \ C

\ \ \ \ \ \ \ \

\ "- \ \

A = \ , A = \ , , r=l

\ \ \ , , , \ \ \

\ \

" \

0 \ 0 \ ,

\ \ \ , X X X X

X X x

a. Diagonal matrix b. Tridiagonal matrix

-x x x Dl

x x x x 0 D2 0 \

x X X X x \ \ \ \ \

\ \ \ \ \ \ A = \ \ \ \ , r=2, A = \ \ \ \ \ \ \ , \ \ \ \ , , , , \ ,

\ x x x x x \

0 \ x x x x \

0 ,

j \

X x D n

c. QUindiagonal matrix d. Block diagonal matrix

FIGURE 3.4: Types of Banded Matrices (where x denotes a non-zero element)

Defini tion 3.3:

An (nxn) matrix A is diagonally dominant if,

la, ,I ~ ~,~

n

I j=l j;o!i

and for at least one i,

1 a , ,I, for all 1 ~ i~n , ~,J

90

Page 112: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

la .. 1 > ~,~

la .. 1 ~,J

Definition 3.4:

If a matrix A is Hermitian, and

for all ':0.+0, ·then A is positive defipite. A is non-negative definite

Definition 3.5:

. (1) (2) (3) A sequence of matr1ces A lA lA , ... , of the same dimension

is said to converge to a matrix A, if and only if,

Urn 11 A-A (k) i I = ° , k-

where 11 11 is a suitable norm.

Let A be a square matrix, ·then A converges to zero if the sequence

. (1) (2) (3) of matr1ces A lA lA , ... , converges to the null matrix, 'and is

divergent otherwise.

Definition 3.6:

A matrix A=[a .. 1 of order n has property A if there exists two ~, )

disjoint subsets 51 and 52 of W={1,2, ... ,n} such that if iFj and if

either ai,jFo or aj,iFO, then i E 51 and j E 52 or else i E 52 and

j E 51'

Page 113: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Definition 3.7:

A matrix A of order n is consistently ordered if for some t

there exist disjoint subsets Sl'S2, ... ,St of W={1,2, ... ,n} such that

t

L Sk=W' and such that if i and j are associated, then j E Sk+l if k=l j>i and j E Sk_l if j<i where Sk is the subset containing i.

3.4.1 Vectors and Matrix Norms

92

The measure of the size or magnitude ofoa vector matrix, is called

its norm and is denoted by I I 11. If u is a vector then its norm.is

denoted by I I~I I, and it is a non-negative number satisfying the

following three axioms:

1. II~II =0 for ~=O, and II~II >0 if ~;!o;

2. Ila~1 i=lal.II~11 for any complex scalar a;

3 .. 11~+~II~II~11+i I~II for the vectors u and v.

Similarly,. as above we can define the matrix norm. The norm of

a matrix A of order n, is denoted as I IAI I, satisfying the following

axioms:

1. I IAI I~O, with equality only when A is the null matrix;

2. 11 aA 11 = I a I . 11 A 11, for any complex scalar a;

3. IIA+BII~IIAII+IIBII, for any matrices A and B (the triangle

inequality);

4. 11 AB I I ~ 11 A 11 . 11 B I I, for any matrices A and B (the Schwarz

inequa li ty) .

There are common types of norms. Some of them are:

i. The Euclidean norm, denoted by IIAII ' is given by, E

Page 114: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

\' 2 ! =(Lla .. 1) .

. . 1.,J ~,J

ii. The spectral or L2

.norm, denoted by IIAlls' is given by,

iii. The Ll norm, denoted by IIAI11

, is given by,

n

2 la. .1 . i=l 1., J

max j

iv. The L= norm, denoted by ! IAI loo' is given by,

L ~

3.4.2 Eig·envalues and Eigenvectors

93

Suppose that A is (nxn) matrix and u be a non-zero column vector

of order n. If there exists a scalar A such that,

Au = AU (3.4.3)

then Ais called an eigenvalue of A and u its corresponding eigenvector

of A.

Equation (3.4.3) can he written as,

(A-AI)U = 0 . (3.4.4) .

The non-trivial solution, ~O to this matrix equation exists if and

. only if the matrix of the system is singular, i.e.,

det(A-AI) o . (3.4.5)

Equation (3.4.5) is called the characteristic equation of A and

the left-hand side is called the characteristic polynomial of A, which

Page 115: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

94

can be written as,

n-l n n a +a l+ ... +a 11 +(-1) 1 o 1 n- o . (3.4.6)

n Since the coefficient of 1 is not zero. equation (3.4.6) has

always n roots (complex or real) which are the n eigenvalues of the

matrix ~. namely. 11 .1 2 •...• 1n

(not necessarily all distinct). each

of them possessing a corresponding eigenvector. Wilkinson

[Wilkinson. 1965J de·scribed many methods in detail for obtaining the

eigenvalues, along with the corresponding eigenvectorsw

Page 116: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

95

3.5 NUMERICAL SOLUTION OF FOE'S BY FINITE DIFFERENCE METHOD

Numerical methods for the solution of FOE's have been studied

extensively for many years and a number of approaches have been

developed. The most widely used are the finite element and. finite

difference methods. Finite difference methods are found to be

discrete techniques where the domain of interest is represented by

a set of points (nodes): The information between these points is

commonly obtained using Taylor series expan~ions.

3.5.1 Finite Difference Approximation

It is necessary for understanding the finite difference technique

to consider the nomenclature and fundamental concepts encountered in

this form of approximation theory. The basic concept is to subdivide

the domain of the solution of the given FDE by a net with a finite

number of mesh points.' The derivative at each point is then replaced

by a finite difference approximation. In this sub-section, we shall

develop certain finite difference representations for one- and two-

independent-variable system using Taylor series expansions.

Let us first, consider u(x), in which u is a continuous function

of the single independent variable x. By discretizing the x domain

.... into a set of points (nodes) such that,

u(X,) _ u(ih) _ u, , i~O,1,2,3, .. , ~ ~

and replacing the location x, by ih, the nodal coordinates are ~

(3.5.1) .

specified as the product of the integer i and grid spacing h, where

h is assumed constant and to be a small quantity less than unity (see

Figure 3.5).

Page 117: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

u

OL---~h~--2~h---3~h~--4·h---45~h~~~---n+h--~X

FIGURE 3.5: Finite Difference Discretization of u=u(x) using

constant Mesh Spacing h.

y

} k

x •

h i, j+l

(a)

0-i-l,j i,j

i, j-l

(b)

FIGURE 3.6: Two-Dimensional Finite Difference Grid.

96

o i+l,j

Page 118: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

97

in the two-dimensional case the function u(x,y) may be

specified at any nodal location as,

u(x, ,y;) ~ J

_ u(ih,jk) - i=O,l,2, ... U. . I

~,J

j;O,1,2, .. , (3.S.2)

The spacing in the x-direction is h and in the y-direction, k. The

integers i and j refer to the location of u along the x and y co-

ordinates, respectively (see Figure 3.6).

The Taylor series expansion for u(x) can be written at the

point x, as, ~

u (x , ) +hu' I, h2

Uxxli

h3

u(x+h) ; +- + - u I + · .. ~ x ~ 21 3! xxx.

~

(3.S.3a)

h 2 h 3 , u(x-h) ; u (x,) -hu I + -u I uxxxl, + · ..

~ xi 2! xx. 31 ~ ~

(3.S.3b)

Rearranging these equations, we may write,

u(x,+h)-u(x,) h h 2

uxl, ~ ~

; -u I - -u I + h 2! xx i 3! xxx.

~ ~ (3.S.4a)

u(x,)-u(x,-h) h h2

uxl. ~ ~

; + - u I -u I + · .. h 2! xx. 3! xxx. ~ ~ ~

(3.S.4b)

Therefore, two possible approximations to the first derivative of U

. at x, are given by (3.S.Sa,b). ~

u (x, +h) -u' (x, )

Uxl, " ~ ~

h -~

or

U (x,) -u (x, -h)

Uxli

" ~ ~

h -

u -u i+l i

h (3.S.Sa)

U,-U, 1 ~ ~-

h (3.S.Sb)

Page 119: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Because the series has been arbitrarily truncated, there is clearly

an error, E., say, associated with this approximation. This error ~

can be characterized by the first and largest term of the truncated

series, which yields,

O(h)

We say that this error is of order h, O(h). The O(h). error is

in absolute value smaller than ch, (c is constant) for sufficiently

small h.

Adding (3.5.4a) and (3.5.4b) and solving for u I ' results into x i

u -u i+l i-I

98

u I = x i 2h (3.5.6)

2 with the truncation error of order O(h ).

While subtracting (3.5.4b) from (3.5.4a) and solving for

we obtain,

u I = xxi

U. l-2u.+u. 1 ~+ 1. 1.-

h2

2 with the truncation error of order O(h ).

We can proceed further to derive many finite difference

u I ' xx . ~

(3.5.7)

approximations for u(x,y). New concepts involve the use of u. . instead ~,J

of u. and the fact that a partial derivative with respect to x implies ~

that y is held constant. Using (3.5.5a) in conjunction with Figure

3.6b as an illustration, we can write directly,

au .. ~ ax

u. I .-u .. 1.+,] 1.,)

h + O(h) (3.5.8a)

Page 120: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

au, , ~ ay

u -u i,j+l i,j

k

99

+ 0 (k) • (3.5.8b)

So, in the approximation to u! ,we hold the subscript j x , , ~,J

constant while in u! we hold i constant. The two-dimensional y , ,

. 1. , J equivalent of (3.5.8a,b),

u -2u +u '1 ' '1

uxxl. ~

1.+ 1. 1.-

h2

is obtained in the same way as,

u I -yy , , ~, J

u -2u +u i+l,j i,j i-l,j

h2

2 + 0 (h ) , (3.5.9a)

(3.5.9b)

The important extension is how to derive mixed derivatives, if we

need the finite difference representation for u , xy

u I -xy , , ~,J

[au I I ax ay, '

~,J

A finite differenc~ representation of (3.5.10) is now readily

obtained using (3.5.6) as follows,

...£..[l'o! I = J:..[aul ax ay " 2h ay , 1 ' 1.,J 1.+ ,]

au --ayl'l,l

1.- , J

(3.5.10)

1 .2h

u, 1 ' 1-u , 1 ' 1 [ 1.+ ,]+ 1.+ ,J-2k

k2

+-u I + ... 3! YYY, 1 '

1. + I J

=

u, 1 ' l-u, 1 ' 1 1.- ,]+ 1.- ,J+

2k k 2

- - u I + 3! YYY, 1 '

1.- ,J

1 u, 1 ' l-u, 1 ' 1-u , 1 ' l+u , 1 ' [ 1.+ ,J+ 1.- ,]+ 1.+ ,]- 1.- ,]

2h 2k

2 2 O(h ) + 0 (k ) ,

••• 1

Page 121: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

and when hand k are equal, this equation becomes

In Table 3.1, a list of the most frequently used finite

difference approximation for u(x,y) are illustrated.

Derivative

u I . xx .. 1.,J

Finite Difference Approximation

u -u i+l,j i,j

h

u -u i,j i-l,j

h

u'l,-U'l' 1.+,] 1- ,J 2h

-u +4u -3u i+2,j i+l,j i,j

2h

Ui+l,j+l-Ui_l,j+l+Ui+l,j_l-Ui_l,j_l

4h

u, 1 ,-2u, ,+u, 1 ' 1.+,] 1.,] 1.- ,J

h2

-u +16u -3OU +16u -u i+2,j i+l,j i,j i-l,j i-2,j

12h2

Ui+l,j+l-Ui+l,j-l-Ui-l,j+l+Ui-l,j+l

4h2

100

(3.5.11)

Order of Error

O(h)

O(h)

TABLE 3.1: Finite difference approximation in two independent

variables with k=h

Page 122: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

101

The two-dimensional finite difference approximations can be

extended in a straightforward manner to three space dimensions, or

two space dimensions and time. The approximation of derivatives in

the third space dimension is analogous to that presented earlier.

To conclude this sub-section, we illustrate now the finite

difference approximation is used to represent PDE's. Let us consider

the heat flow parabolic equation,

u xx (3.5.12)

and the Laplace or Poisson elliptic equation,

u + u xx yy

o (3.5.13a)

or u + U = f (x ,y)

xx yy (3.5.13b)

Temporarily, we shall ignore the initial and boundary conditions

and concentrate on the derivation of some finite difference forms for

the POE which wili hold within the domain of interest.

A finite difference representation of (3.5.12) is readily

. obtained using information from Table 3.1. A term-by-term

substitution yields,

u. , l-u, , 1.,]+ 1.,)

k

u, 1 ' - 2u , , +u, 1 ' 2 1.+,] 1.,] 1.- ,]

2 + 0 (k ,h ) .

h

Neglecting the truncation error, the finite difference equation at

the level (j+l) becomes,

AU + (1-2A)U, , + AU, 1 ' i+l,j 1,J 1.- ,J

(3.5.14)

where A The error associated with (3.5.14) is of order

Page 123: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

102

2 O(k,h ). The discrete diagram for (3.5.14) appears in Figure 3.7a.

The Laplace equation (3.5.13a) can be approximated in exactly

the same manner using Table 3.1,

u -2u +u i+l,j i,j i-l,j +

h2

u -2u +u i,j+l i,j i,j-l

k2

2 2 + O(h ,k ) ; o ,

which can be rearranged to yield,

1 -2[u. 1 .+u. 1 .1 h ~+,J 1-,]

1 + "2[U' '+l+u .. 11 k 1,J 1.,J-

(3.5.15)

The discrete diagram for equation (3.5.15) is given in Figure 3.7b.

When k;h, equation (3.5.15) reduces to,

u. 1 . +u. 1 . +u. . l+u . . 1 1.+,) 1.-,J 1,J+ 1,)-

which is also second-order accurate.

i, j+l

o i-l,j i,j i+l,j i-l,j

(a)

(3.5.16)

i, j+l

i,j i+l,j

i,j-1

(b)

FIGURE 3.7: Computational Molecules for Finite Difference Approximations

Page 124: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

103

3.5.2 Derivation of Finite Difference Approximations

Previously, we have shown that by using truncated Taylor series

expansions, we can replace a PDE with a finite difference approximation.

The truncated terms lead to the truncation error, which provides a

measure of the accuracy of the finite difference approximation. For

any PDE there are many possible finite difference representations, and

the schemes with smaller truncation error are to be preferred. This

is not always true, since other features in addition to the truncation

error make a particular finite difference approximation a feasible

candidate for a computation.

Let us consider equation (3;5.12) as a model for developing the

finite difference approximations. There are two types of approximation,

explicit and implicit. Table 3.2 shows some of the well-known finite

difference methods. The truncation error and the stability require­

ment of each method are also included.

Page 125: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

FINITE DIFFERENCE FORM

Cwssical Explicit

l-

ui , j+l = (1-2A)ui,j+A(ui+l,j+ui_l,j)

. Dufort-Frankel Explicit

2.

(1+2A)ui ,j+l = 2A(ui+l,j+Ui_l,j)+(1-2A)Ui,j_l

Richardson Explicit 3.

= U. j_l+2X(u. 1 j+ui _l .)-4Xui . U i,j+l 1., 1.+ , ,J ,J

Special Explicit 4.

1 u .. 1 = U. + 6(u. 1 .-2ui .+U. 1 j) ~, J+ l.,j 1.+ ,J ,J 1.-,

TABLE 3.2: Well-known Finite Difference Methods

STABILITY CONDITION

A ~ 1

Uncondit-ionally

Stable

Unstable

Stable

TRUNCATION ERROR

O(k,h2

)

O(k2

,h2

)

O(k2,h

2)

O(k2

,h4

)

COMPUTATIONAL MOLECULE

~ cS

continued ••• .... o A

Page 126: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

,

FINITE DIFFERENCE FORM

Baciauards (FuUyJImpZicit 5.

(1+2>.)u ->. (u +u ) = u, i,j+l i-l,j+l i+l,j+l 1,j

Crank-Nicolson Implicit 6.

(2+2>')Ui,j+l->'(Ui_l,j+l+ui+l,j+l) = (2-2>, )u i , j +>. (u i _l , j +ui+l, j)

Weighted ImpZicit

7. (1+2>'9)Ui,j+l->'9(ui_l,j+l+ui+l,j+l) = A(1-9) (ui_l,j+ui+l,j)+

o~e~l (1-2>. (1-9) )ui,j

Douglas Implicit 8.

5 >. 1 >. 1 (~>')Ui,j+l-(2 - 12) (Ui-l,j+l+ui+l,j+l) = (- + -) (u +

2 12 i-l,j 5

u i +l ,j)-(6->,)u i ,j

TABLE 3.2: Well-known Finite Difference Methods

STABILITY TRUNCATION CONDITION ERROR

Uncondit-ionally

O(k,h2) Stable

Uncondit-ionally

O(k2

,h2

) Stable

O(k2 ,h2) 1 ; >. ~ (2-49) ,for 9 =

O(k,h2) O~9~1

for e " ;

Uncondit-ionally

O(k2

,h4

) ,Stable

COMPl1I'ATIONAL MOLECULE

..-/

V

.J

~

0 -\.. -0

.f

continued ••• ,... o '"

Page 127: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

FINITE DIFFERENCE FORM STABILITY TRUNCATION COMPUTATIONAL CONDITION ERROR MOLECULE

r'\

Variation of Douglas Implicit Uncondit-

9. 151 1 5 ionally 0(k2

,h4

) (S-A)Ui+l,j+l+(4+2A)Ui,j+l+(S-A)Ui_l,j+l = ~i+l,j·~i,j+ Stable

1 1 5 1 6"i-l, j- 24"i+l, j-l- ~i. ;-1- 24"i-1. ;-1

0 SauZ'ev's Alternating (b) I Uncondit- 0--0 10. (a) Ui ,j+l-Ui,j = A (u. 1 j -U. j -U. . 1 +U. 1 . 1) ionally 0(k,h2

) ~+ ,. ~, 1., J+ 1.-, J+

~ Stable (b)

U i ,j+2-U i,j+l = A tu -u -u +U ) i+l,j+2 i,j+2 i,j+l i-l,j+l

TABLE 3.2: Well-known Finite Difference Methods 2 au a U to-=--

at ax2

Page 128: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

107

3.5.3 Consistency, Efficiency, Accuracy and Stability

When a PDE (say, equation (3.5.12» is approximated by a finite

difference analogue, one naturally expects that the difference scheme

indeed represents the differential system in some sense. By this, we

mean that a difference system is consistent with a differential system

when the former becomes identical with the latter in the limit as h,k+O.

Clearly, consistency is a fundamental requirement.

Efficiency refers to the amount of computational work done·by the

computer in solving a problem over unit time-length.

Accuracy of a numerical solution depends on two major classes of

errors, i.e., round-off and truncation errors. Round-off errors

characterise the differences between the solution furnished by the

computer and the exact solution of the difference equations.

Truncation errors are caused by the approximations involved in

representing differential equations. Truncation errors depend upon

the spatial grid size h and step size k. Intuitively, one would

assume that the accuracy of any finite difference solution would be

decreased by increasing the grid sizes, Since, evidently, if the grid

spacing is reduced to zero, the discretized equivalent becomes

identical to the continuous field. Magnitude of truncation errors can

be estimated using Taylor, series expansion.

In recommending a numerical method, we need to strike a balance

between efficiency and accuracy. This is because a method might have

incurred more work to attain accuracy. On the other hand, one may

settle for a less accurate method in favour of its simplicity and

computing cost effectiveness.

Page 129: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

The final concept to be studied is stability. If u(x,t) is the

exact solution and u .. is the solution of the finite difference l.,)

equations, the error of the approximation at the point (i,j) is

((u .. )-u(ih,jk». One is interested to know the behaviour of l.,)

lu .. -u(ih,jk) I, as j->«> for fixed h,k. That is, whether the solution l.,)

is bounded (stable) as the index j-><o. Also of interest is the

behaviour of lu .. -u(ih,jk) I.- as h,k->O. That is, whether the l.,]

difference scheme is convepgent.

It is clear in both cases that as the number of cycles of

calculations become large there is a possibility for unlimited

amplification of errors, and hence, the total accumulated error will

quickly swamp the solution rendering it worthless. It can therefore

be said that a numerical method is stable if a small error at any

stage produces a smaller cumulative error.

Lax's equivalence theorem gives a relation between consistency,

stability and convergence of the approximations of linear initial

value problems by finite difference equations.

Theorem 3.1

Given a properly posed initial boundary value problem and a

finite difference approximation to it that satisfies the consistency

condition, then s~ability is the necessary and sufficient condition

for convergence.

Various techniques are available for a quantitative treatment of

the stability of finite difference schemes. Among those commonly used

are the linearised Fourier analysis method (van Neumann criterion),

the matrix method, the maximum principle and the energy method.

108

Page 130: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

3.6 METHODS OF SOLUTION

As mentioned in the previous section, the application of the

finite difference methods for solving PDE's (say, equation (3.5.12»

yields a system of linear simultaneous equations which can be

represented in matrix notation as,

109

Au = b (3.6.1)

where A is.a coefficient matrix of order (nxn) , the order of the

matrix A equals the number of interior mesh points, b is a column

vector containing known source and boundary terms, and u is the

unknown column vector.

·This section deals with well-known methods of solving the system

(3.6.1). Usually the methods used lie in two classes, the class of

direct methods (or elimination methods) and the class of iterative

methods (or indirect methods) which mainly depend upon the structure

of the coefficient matrix A. So, if,A is a large sparse matrix,

iterative methods are usually used, since these will not change the

structure of the original matrix and therefore preserve sparsity.

Another advantage of. iterative methods, not possessed by direct

methods, is their frequent extension to the solution of sets of non­

linear equations.

3.6.1 Direct Methods

The direct methods are based ultimately on the process of the

elimination of variables. The most widely used methods are:

1. Gaussian ELimination (GE) method, which involves a finite number

of transformations of a given system of equations into an upper

Page 131: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

110

triangular system which is much more easily solved. Precisely the

number of the transformation is one less than the size of the given

system. If any of the diagonal elements of the matrix A in the

system (3.6.1) becomes zero during the elimination process, we re­

order the equations. The reorders are referred to as pivoting

which is normally employed to preserve stability against rounding

error.

There are two basic well known pivoting schemes. Partial pivotin·g

and complete pivoting. In the partial pivoting strategy we choose

an element of largest magnitude in the column of each reduced matrix

as the pivot, elements of rows which have previously been pivoted

being excluded from consideration. Whilst in the complete pivot,

the pivot at each stage of the reduction is chosen as the element

of largest magnitude in the submatrix of rows which have not been

pivotal up to now, regardless of the position of the element in the

matrix. This may require both row and column interchange. The

complete pivoting is time consuming in execution and is not

frequently used.

2. Gauss-Jordan (GJ) method, which is an alte·rnative to GE leads to a

diagonal matrix rather than triangular at the end of the process.

In this method, the elements above the diagonal are made zero at

the same time that zeros are created below the diagonal, and hence

the solution can be obtained by dividing the components of the

right-hand side vector, £, by the corresponding diagonal element,

i.e., there is no need for the back substitution stage as in GE.

Page 132: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

111

3. Triangular, ·or LV Decomposition, is a modification of the GE. The

matrix A is transformed into the product of two matrices Land U,

where L is a lower triangular and U is an upper triangular matrix

with one's on its diagonal entries. Then equation (3.6.1) can be

written as,

LUu ; b . (3.6.2)

The solution of (3.6.1) by this algorithm follows from (3.6.2) by

introducing an auxiliary vector, X (say), such that the system

(3.6.2) will be split into two triangular systems,

(3.6.3a)

Uu X (3.6.3b)

The two vectors X and ~ can be obtained from (3.6.3a,b) by forward

and backward substitution processes respectively.

The amount of work in the LU and GE are the same, while the GJ

method requires almost 50% more operations than the GE method.

3.6.2 The Iterative Methods

It is well known that iterative methods can make use of the great

speeds of modern-day computers which are used in large scale computations

for solving the matrix equations which arise from finite difference

approximations to PDE's.

In the iterative method, a sequence of approximate solution vectors

(u(k)} are required to solve the non-singular system (3.6.1) such that

(k) -1 U -+ A b as k-to;!

Page 133: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

112

Without loss of generality, let us consider that the (nxn) non-

singular coefficient matrix A of the system (3.6.1) can be expressed as,

A ; Q - S , (3.6.4)

where Q and S are also (nxn) matrices, and Q is non-singular. This

expression then represents a splitting of the matrix A. Equation

(3.6.1) then becomes,

Su + b (3.6.5)

Different splittings of the matrices Q and S will clearly give

different iterative methods. Some of these methods are:

1. The Jacobi Method. In this method we assume that Q;D and S;E+F,

where D is the main diagonal element of A. E and F are strictly

lower and upper (nxn) triangular matrices respectively. Hence

equation (3.6.5) can be written as,

Du ; (E+F)u + b (3.6.6)

-1 By the assumption that A is non-singular, thus D exists and we

can replace the system (3.6.6) by the equivalent system,

-1 -1 u ; D (E+F)~ + D b

The Jacobi iterative method is defined by,

(k+l) u

(k) ~ +~

(3.6.7)

(3.6.8)

where B is the Jacobi iterative matrix associated with the matrix A

Page 134: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

113

and is given by,

-1 B = D (E+F) ,

-1 (k) and ~=D £. In this method the components of the vector u

b d wh 'l 'th of u(k+l) must e save 1 e comput1ng e components

2. The Gauss-Seidel (CS) Method, which is based on the immediate use

, (k+l) (k) of the 1mproved values u

i instead of u

i . By setting Q=D-E

and S=F, the matrices D,E and F as defined before, equation (3.6.5)

becomes,

(D-E) ~ = Fu+b

Then, the GS iterative method is defined by,

(k+l) (k+l) (k) b Du ; Eu +Fu + (3.6.9)

-1 By multiplying both sides by D ,equation (3.6.9) can be written

as,

(k+l) u

(k+l) (k) L~ +R~ +~

-1 -1 -1 where L=D E, R=D F, and ~=D £

(3.6.10)

Since L is strictly lower triangular matrix then det(I-L)=L Hence

-1 (I-L) is a non-singular matrix and (I-L) exists, and the GS

iterative method will take the form,

(k+l) L (k) ~ = ~ +!,

where L is the CS itel'ative matl'ix, and it is given by,

and

L -1

(I-L) R

-1 t = (I-L) ~.

(3.6.11)

The computational advantage of this method is that it does not

Page 135: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

(k+l) require the simultaneous storage of the two approximations lli

and u(k) in the course of the computations as does the Jacobi 1.

iterative method.

3. The Simultaneous Overrelaxation (JOR) Method, is a modification of

114

the Jacobi method. ~(k+l) If we assume that u is the vector obtained

from the Jacobi method, then from (3.6.8),

~(k+l) (k) u = B~ +SL (3.6.12)

(k+l) and by choosing a real parameter w, the actual vector u of

this iteration method is determined from,

(k+l) u (3.6.13)

~(k+l) Elimination of u between equations (3.6.12) and (3.6.13) leads

to

(k+l) (k) U =Bu +wg

or- - (3.6.14)

where B is the iteration matrix of the JOR method and is given w

by -1

B = [wD (E+F)+(l-w)Ij . w

The real parameter w is called the relaxation faetor. If w=l, we

have the Jacobi method. If w>l «1) then we are in a sense

carrying out the operation of "overrelaxation" (underrelaxation)

a t each of the nodal points. Both, the Jacobi and JOR methods are

clearly independent of the order in which the mesh points are

scanned.

Page 136: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

4. The Successive OverreZaxation (SOH) Method, which is the same as

(k+l) the JOR method except that one uses the values of ui

whenever

115

possible. So, from equation (3.6.10) the 50R method is defined as,

(k+l) (k+l) (k) (k) u = W(L!:!. +R!:!. +.<I) + (l-w)!:!.

Equation (3.6.15) can be written in the form,

(k+l) (I-WL)!:!.

(k) [wR+ (l-w)I l!:!. +w.<I

(3.6.15)

(3.6.16)

But (I-wL) is non-singular for any choice of w, since det(I-wL)=l.

So we can solve (3.6.16) for u(k+l) obtaining

(k+l) (k) -1 u = L u +(I-wL) wg

w- -

where L is the SOH iteration matrix and is given by, w

L w

-1 = (I-wL) (wR+(l-w)I).

(3.6.17)

(3.6.18)

For w=l, we have the G5 method. And w>l «1) corresponds to the

cases of overrelaxation (underrelaxation) at each of the nodal points.

The G5 and 50R methods are both dependent upon the order in which

the points are scanned.

s. The Syrrunetric SOH (SSOH) Method, whereas this method involves two

half iterations using the SOR method. The first half iteration is

the ordinary 50R method while the second half iteration is the SOR

method using the mesh points in reverse order. Hence, we can define

the 550R iterative method by,

(k+!) (k) -1 u = L u + (I-wL) w.9.

w- (3.6.19a)

and

Page 137: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

116

(k+l) (k+~) -1 u = K u +(I-wR) wg (3.6.19b) w- -

where u(k+~) is an intermediate approximation to the solution,

L , is given in (3.6.18) and K is given by, w w

-1 K = (I-wR) [wL+(l-w)I]

w (3.6.20)

(k+~ ) By eliminating u between equations (3.6.19a) and (3.6.19b)

we have,

(k+l) (k) u = H u +d

or- --w

where H is the 880R iteration matrix and is given by, w

H = K L w w w

and

-1 -1 (I-wR) (I-wL) [wL+(l-w)I] [wR+(l-w)I]

d --w

-1 -1 w(2-w) (I-wR) (I-wL) .2.

3.6.3 Block Iterative Methods

(3.6.21)

(3.6.22a)

(3.6.22b)

In our previous discussion of the iterative methods for solving

equation (3.6.1), we dealt with point iterative methods, that is, at

each step the approximate solution is modified at a single point of the

domain. An extension of these methods are the block (group) iterative

methods in which several unknowns are connected together in the

iteration formula in such a way that a linear system must be solved

before anyone of them can be determined. Equation (3.6.1) can be

written as,

(3.6.23)

In the system (3.6.23), assume that the equations and the unknowns u. ~

are partitioned into t groups such that ui

' i=1,2, ... ,n1

, constitute

Page 138: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

the first group; i=n1+l,n

1+2, ... ,n

2, constitute the second group, and

th in general, u" n l<i<n constitute the 5 group and nn=n.

1. s- ... S A.

117

Young [Young 1971] defines an ordered grouping n of W;{1.2.3 •...• n}

as a subdivision of W into disjoint subsets Gl

.G2 •...• GR. such that

Gl U G2 •... U GR. ; W. Two ordered groupingsn and n' defined by Gl

.G2

... ,G! and G1 "G2 , , ... ,Gt

" respectively are identical if £=1' and if

Gl=Gl ,· G2=G 2 , •...• GR.=GR.'

Evidently. this partitioning n imposes a partitioning of the matrix

A into blocks (group~ofthe form.

A - --2.2

A = I

I (3.6.24)

A A -----A R..l R..2 R..R.

where the diagonal blocks A, ., 1~i~2 are square, non-singular matrices. 1..1.

From this partitioning of the matrix A, we define the matrices,

D = ,

o

and

0 , ,

F =

, , , ,

o

, ,

Al •2 - - - - -, , , , , , , , , , , , , ,

0 , , , ,

0, "- ,

A2 •1 , , o

• E , , , , , ,

, , , , , , , A ~----A'

R..l R..R.-l

A 1.R.

, , AR._l.R.

0

, , '0

(3.6.25)

Page 139: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

118

where D is a block diagonal matrix and the matrices E and Fare

strictly lower and upper block triangular matrices respectively, and

A-D-E-F. (3.6.26)

Now, for the column vector ~ in equation (3.6.23) we define

column vectors U1 'U2""'U where U is formed from _u by deleting all - - -~ -s

elements of ~ except those corresponding to group s. Similarly,

define column vectors ~1'~2'.'./~2 for the given vector £, we also

define the submatrices A, of for i,j=l,2, ... ,R., such that each A .. 1.,) l.,J

are formed from the matrix A by deleting all rows except those

corresponding to G, and all columns except those corresponding to G,. 1 J

The system (3.6.23) evidently can be written now in the equivalent

form,

~

I j=l

A, ,U, 1,J-:J

= c, -1

i=l, .... ,R.. (3.6.27)

If we suppose that all the submatrices A, , are non-singular. Then, 1,1

the bZock Jaaobi iterative method is defined by

,or equivalently,

where,

B, ' 1,J

and V -i

=

~

I j=l j;li

~ I A, ,u~k)

j=l 1, J-:J

j;li

(k) B, ,u,

1,J-:J

-1

{ -A, , A,

l.,j 1,1 =

l 0

-1 A, , C, , 1,1 ""'2

if i;lj

if i=j

(3.6.28)

(3.6.29)

(3.6.30)

(3.6.31)

Page 140: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

We may write (3.6.28) in the matrix form,

wher~,

and

(k+l) u

(U) + v

E(U) and F(u) are again strictly lower and upper triangular matrices.

In a similar manner to the pOint case, we can derive the block

GS, JOR, SOR and SSOR methods.

The determination of a suitable value for the relaxation factor

w of the SOR method is of paramount importance, and in particular the

optimum value of w, denoted by ~' which minimizes the spectral radius

of the SOR iteration matrix and thereby maximize the rate of

convergence of the method.

Young [Young, 1954] proved that when a matrix possesses property

A, then it can be transformed into what he termed a consistently

ordered matrix. Under this condition the eigenvalues A of the SOR

iteration matrix L associated with A are related to the eigenvalues ~ w

of the corresponding Jacobi iteration matrix B by the equation

119

2 (A+w-l)

A 2 2

= w ~ (3.6.32)

from this equation it can be seen that,

! A-W~A +w-l = 0 • (3.6.33)

If we assume that A is symmetric, then, the eigenvalues of B are real

Page 141: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

120

and occur in pairs ±~. Let 11 denote the largest eigenvalue of B, it

can be shown that wb defined by,

or equivalently,

2

~ l+/l-"ii~

is the value of W which minimizes p(Lw

)' i.e. WFWb

, then

p (L ) > p (L ) W wb

Using this optimum value of w, it can also be shown that,

r-:2 l-/l-ii~

~ l+1I-U~

Moreover, for any w in the range O<w<2, we have,

p (L ) W

=

_ 2..2 ! r[w~+(W ~ ;4(W-l» J2

lw-I l

(3.6.34a)

(3.6.34b)

(3.6.35)

(3.6.36)

(3.6.37)

The estimation of wb depends on whether P(B) or p(L) can be estimated.

Several methods have been suggested by carre [Carn" 1961J, Varga

[Varga, 1962J and Hageman and Kellogg [Hageman, 1968J.

Page 142: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

3.6.4 Alternating Direction Implicit (ADI) Methods

To increase convergence, another form of splitting was suggested,

which consists of an initial first half iteration in the row direction

followed by a second half iteration in the column direction. Such

methods are aptly designated AZtePnating Direction ImpZicit methods or

ADI methods for short.

121

The first ADI methods were developed by Peaceman and Rachford (PR)

[Peaceman, 1955] for solving the matrix equation,

Au = b (3.6.38)

where the (nXn) matrix A is non-singular and can be represented as the

sum of the three (nxn) matrices,

A = H + V + r . (3.6.39)

We can make the following assertions about the matrix A:

1. A is a real, symmetric and positive definite, irreducible matrix

with non-positive off-diagonal entries. Derived from the

discretisation of a two dimensional elliptic operation.

2. H and V are real, symmetric, diagonally dominant matrices with

positive diagonal entries and non-positive off-diagonal entries,

derived from row and column discretisation of .elliptic operation.

3. L is a non-negative diagonal matrix.

In a typical situation H and V would be tridiagonal or could be

made so by a permutation of the rows and corresponding columns.

By using (3.6.39) we can write the matrix equation (3.6.38) as a

pair of matrix equations,

Page 143: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

122

(H+!l: +rI) ~ ~ b- (V+!l: -rI) u

(3.6.40)

(V+!l:+rI)u ~-(H+!l:-rI)~

for any positive scalar r. If we let,

then the Peaceman-Rachford Alternating Direction Implicit method is

defined by,

(3.6.41)

where the r's are positive acceleration parameters chosen to make the

process converge rapidly.

Since the matrices (Hl+rk+1I) and (Vl+rk+lI) are, after suitable

permutations, tridiagonal non-singular matrices, the above implicit

process can be directly carried out by the simple algorithm based on

the GE. Indeed, the ADI method is derived from the observation that

for the first equation (3.6.40) we solve first along horizontal mesh

lines, and then for the second equation of (3.6.40) we solve along

vertical mesh lines. The vector u(k+!) is treated as an auxiliary

vector which is discarded as soon as it has been used in the'

, (k+l) calculatlon of u .

The two equations of (3.6.41) are now combined to give,

u(k+l) ~ T u(k)+g (b), k~O , r - r k+l k+l

(3.6.42a)

where, (3.6.42b)

and

Page 144: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

123

T r

(3.6.42c)

The convergence of the ADI scheme can be easily shown for the

stationary case with constant parameters r.=r. If we let, 1

then by similarity, T and T have the same eigenvalues. Hence from r r

(3.6.42c) we obtain,

P (Tr ) = P (Tr ) ~ 11 Tr 11

~ 11 (Hl-rI) (Hl+rI)-lll 11 (Vl-rI) (vl+rI)-lll ,

where P(Tr

) is the spectral radius of Tr

. Since HI and VI are

symmetric and positive definite, then in the L2 norm, we find,

Ir-~.

Ir+~: < I ,

where ~i' l~i~n are the eigenvalues (positive) of HI' A similar

argument applies to the norm involving VI' Hence, p(T )<1 for all r>O, r.

and therefore the PR iteration (3.6.41) converges.

3.6.5 Alternating Group Explicit (AGE) Method

As we have already seen in the subsection (3.6.4), the ADI method

was developed to obtain the solution implicitly in the horizontal and

vertical directions.

Evans [Evans, 1985], however, derive another method where the

analysis of which is analogous to the ADI scheme. This Explicit

Iterative method employs the fractional splitting strategy on tri-

diagonal systems of difference schemes and which has proved to be stable.

Page 145: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

124

Its rate of convergence is governed by the acceleration parameter r.

Let us recall equation (3.6.1) where the matrix A and the two

vectors u and b are given by,

d c l a d c , ,

0 , , , , , , , , , , , , A = , , ,

, , (3.6.43) , , , , , , , , 0

, , , , ,

l a d c

a d

u - (u1

,u2 ,···,un

)T

and b (b ,b

2, .. .,b )T =

1 n

We now consider a class of methods for solving the system (3.6.1),

which is based on the new splitting of the matrix A into the sum of

the matrices,

where Gl and G2 satisfy the conditions Gl+rI and G2+rI are non­

singular for any r>O. And Gl

,G2

are given by,

hd c

a hd

=

'­I I hd

la

o

c

·4 __ _ -,-

L ___ _

I

I

'0 i

+ I hd , I la ,

-~ c

hd (nxn)

(3.6.44)

(3.6.45a)

Page 146: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

and hd ---n I

I --I t-I

1 hd C I

I I _ _ ,_a_ _hd_ l _ -I r -

1 , 0 G2

= , (3.6.4Sb) , , ,

" r- -,- -1 -,hd C

,In.n, 0

I , hd 1 -"- la_

, I

"

if n is even (odd number of intervals) , and,

hd I I --, -'- r- -'- - -

hd C 1 , ,

1 1 a hd , , 0

- - - 1-:: -1- -1

Gl

= " (3.6.46a) I

~ -, -, ,

1 0

, hd C

a hd (nxn)

and

ihd C I l a hd 1 1

-1- -,- - -+ - - - -1

I hd Cl 1 1 0

a hd _1-__ L_ - ~!--, G

2 = ,

'- ,I (3.6.46b) , "

- - -" I-1 hd C

! 1

0 I

I 1 1 , ,a

- h,d, :- - J - , - ,I , , ' -, I I

, hd , (nXn)

if n is odd (even number of interval s) , with hd=d/2.

Page 147: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

It is assumed that the following conditions are satisfied:

i. Gl+rI and G2+rI are non-singular for any r>o.

ii. for any vectors ~l and ~2 and for any r>O, the systems,

and

are more easily solved in explicit form since they consist

of only (2 x 2) subsystems.

We shall be concerned here with the situation where Gl

and G2

are

either small (2 x 2) block systems or can be made so by a suitable

permutation of their rows and corresponding columns. This procedure

is convenient in the sense that the work required is much less than

would be required to solve the original system (3.6.1) directly.

By using (3.6.44) we can write the matrix equation (3.6.1) in the

form,

126

(3.6.47)

(k+l) (k+l) Following a strategy similar to the ADI method, u and u

can be determined implicitly by,

or .explicitly by,

(G +rI)u(k+1) 1 -.

(k+l) (G +rI)u . 2 -

u (k+t)

(k+t) b-(G -rI)u - 1 -

If we combine the above two equations into the form,

(3.6.48a)

(3.6.48b)

Page 148: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

where,

and b --x

(k+l) (k) u = T u +b

r- --x

The matrix T is called the AGE iteration matrix. r

127

(3.6.49)

(3.6.50)

We now seek to analyse the convergence properties of the AGE

method. Let us assume that U is the exact solution, then,

and (3.6.51)

.. (k) (k) LeL~ =~ -Q be the error vector associated with the vector iterate

(k) u Therefore from (3.6.48b) and (3.6.51) we have,

(Gl +rI)~ (k+!) (k)

-(G -rIle 2 -

similarly,

and hence (k+l)

e = T r

= -(G _rI)e(k+!) 1 -

(k) e

where T is given in (3.6.50)., r

To indicate the convergence properties of T , we have the r

following theorem.

Theorem 3.2

If Gl

and G2

are real positive definite matrices and if r>O then

p(T )<1, where p(T ) is the spectral radius of T . r r r

Proof: If we define the matrix T as, r

Page 149: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

~ -1 Tr = (G

2+rI)T

r(G

2+rI)

-1 -1 = (Gl-rI)(Gl+rI) (G

2-rI)(G

2+rI) ,

~

then it is evident that T is similar to T , and hence from the r r

properties of the matrix norms we have,

However, since Gl and G2

are symmetric and since (Gl-rI) commutes

-1 with (Gl+rI) we have,

11 (Gl-rI) (Gl

+rI) -111

=

-1 p«Gl-rI) (Gl+rI) )

I (),-r) I

~x ().+r)

where)' ranges over all eigenvalues of Gl

. But since Gl

is positive

definite, its eigenvalues are positive. Therefore,

The same argument applied to the corresponding matrix product with G2

shows that I I (G2-rI) (G2+rI)-11 1<1, and we therefore conclude that,

P (T ) = p 6' ) ~ liT 11 < 1 , r r r

hence the convergence follows.

It is possible to determine the optimum parameter r such that the

bound for p(T ) is minimized. r

Let us assume that Gl

and G2

are real positive definite matrices

and that the eigenvalues ). of Gl

and ~ of G2

lie in the ranges,

128

o < a ~ ).,~ ~ b • (3.6.52)

Page 150: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Evidently, if r>O we have,

= ( max a~A~b

I A-r! I A+r ) ( max a~J.l~b !

J.l-r! ) J.l+r

[ max a~y~b

129

= q,(a,b;r) (3.6.53)

Since (y-r)/(y+r) is an increasing function of y we have,

max a~y~b

When r=~, then,

la-rl !b-r\ = max( la+rl' b+r )

Ib-r! = Ib+r

Ib.- fa

/b+ fa

Moreover, if O<r<~, we have,

Ib-rl Ib+rl -

and if ~<r, then,

la-r\ la+r

2/b (rab-r)

(b+r) (/b+fa)

2/b (r..:/ab)

(r+a) (1b+1a)

Therefore ~(a,b;r) is minimized when r=/ab and

> 0 ,

> 0 .

Thus, r=1ab is optimum in the sense that the bound ~(a,b:r) for

p(T ) is minimized. r

For an efficient implementation of the AGE algorithm, it is

essential to vary the acceleration parameters rk

from iteration to

iteration. This will result in a substantial improvement in the rate

of convergence of the AGE method, equation (3.6.48a) can be written as,

Page 151: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

130

.(3.6.54)

; b-(G +r I)u(k+1) - 1 k+l -

, k~O •

The best values of rk

+l

can be ascertained provided Gl

and G2 are

commutative (see Evans, 1985b).

Page 152: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS
Page 153: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

CHAPTER 4

STEADY STATE PROBLEMS,

PARALLEL EXPLORATIONS

It isn't reaLLy

Anywhere!

It's somewhere else

Instead! .

A.A. Milne.

Page 154: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

131

4.1 INTRODUCTION

Previously, in section (3.6) we conclude that the point (Explicit)

methods have natural extensions to block iterative processes in which

groups of components of x(k) are modified simultaneously. A faster

rate of convergence may be obtained, if a group of points are

evaluated at once in one iteration step rather than solving the

individual points.

In this chapter, a method involving a: (2x2) block is implemented

in parallel. Three parallel strategies of the AGE method are

presented which is an ideal parallel algorithm since it can be

subdivided into a number of small independent tasks. Also, these

tasks can be done at the same time without interfering with each

other, for solving one, two or more dimensional boundary value

problems which were developed and implemented on the Balance 8000

system. These include synchronous and asynchronous versions of the

algorithms. The results from these implementations were compared as

well as the presentation of the timing results and the performance

analYSis of the best strategy.

A one dimensional boundary value problem with boundary condition

involving a derivate is also solved using the AGE iterative method

with the D'Yakonov splitting formula. The same three parallel

strategies were also implemented on this kind of problem.

The Sturm-Liouville problem,

d dU dx(P(X)dx) - q(x)U + Ap(X)U ; 0 ,

where p,q and p are real functions of x in the interval a~x~b, subject

to the boundary conditions at the points a and b, is also solved using

the AGE algorithm, where the parameter A and the vector solution U are

determined.

Page 155: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

4.2 PARALLEL AGE EXPLOITATION

The basic concepts of the AGE scheme for solving a system of

equations was presented in subsection (3.6.5). In this section we

proceed with the parallel implementation of this method.

Three strategies are investigated and implemented to solve one

and two dimensional boundary value problems. The strategies are

mainly concerned with the way the problem to be solved is decomposed

into many tasks that can be run in parallel. In the first two

strategies the problem is solved by decomposing its interval into

subsets and assig~~~ach subset to different processors which can run

them in parallel, whilst in the third strategy the problem is solved

by decomposing its domain into partitions and each partition is

assigned to different processors.

These three strategies are programmed on the Balance MIMD system

using both the synchronous and asynchronous approach. The results

from the implementations of these approaches, such as the timing

needed to solve the" problem, number of iterations required and the

'sp~d-up' ratios are obtained and compared.

Using these strategies, shared memory is used to hold the input,

the results from the first sweep and the final output component values.

These values can then be accessed by different processes. Before the

process iterates on its task, it needs to read all its components

first from private memory, then it releases all the values of the

components for the next iteration. In the different parallel versions,

different mesh sizes are evaluated. The results shown in this chapter

are an average of many runs.

To demonstrate these strategies, let us consider the differential

132

Page 156: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

133

equation, d

2U

- --- + q(x)U = f(x) , dx

2 (4.2.1)

subject to the two-point boundary conditions,

U(a) = Cl, U(b) = 8 , (4.2.2)

where Cl and 8 are given real constants, and f(x) and q(x) are given

real continuous functions in a~x~b, with q(x)~O.

For simplicity, we place a uniform mesh of size hi where

h = (b-a) (n+l)

on the interval a~x~b, and we denote the mesh points of the discrete

problem by,

x, = a+ih, ~

as illustrated in Figure 4.1 below.

x =a o

FIGURE 4.1

x n-l

x n x =b

n+l

Using Table 3.1, the finite difference replacement of equation

(4.2.1) is given by,

2 -u. +(2+h q. )u.-u.

~-l ~]. ~+l

which can be written in matrix notation as,

Au = £ '

where A as in (3.6.43), with d=2+h 2q. and a=c=-l. ~

If we split A into component matrices Gl

and G2

, where Gl

and G2

as defined in (3.6.45a,b), we can find u(k+!) and u(k+l)

(4.2.3)

(4.2.4)

Page 157: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

134

explicitly by,

Figures 4.2 and 4.3, show the computational molecule for the

(k+!) and (k+l) sweep (first and second sweep) .

. (k+!) (k+!)

0----; (k) (k)

i-l i i+l i+2 i-l i i+l i+2

FIGURE 4.2: Explicit Computational Molecule for the (k+!)th Sweep.

(k) (k)

"""-0-0 (k+! ) (k+!)

i-l i i+l i+2 i-l i i+l i+2

FIGURE 4.3: Explicit Computational Molecule for the (k+l)th Sweep.

Page 158: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

In'the first strategy, the mesh of points (Figure 4.1) is

decomposed into subsets of points, each of which are assigned to a

processor. Then each processor computes its own subset in two sweeps.

In the first sweep, the evaluation of two successive pOints (in their

natural order) at a time starting from the first two pOints and

terminates after evaluating the· last two points .. While the second

sweep is started after the first sweep has been completed. In the

second sweep we evaluate the first point then each two successive

points at a time and the last point is evaluated on its own.

As an example, given the interval shown in Figure 4.1. If we

start evaluating the mesh points by taking a pair of points at a time,

then the order of the first sweep is,.

(Xl ,x2) ,(x3 ,X4 ),···, (xn

_l

,xn

)

After the completion of the first sweep, the ordering of the mesh

pOints in the second sweep will be,

(Xl)' (x2 ,x3), (x4 ,XS)"'" (x

n_

2,x

n_

l), (x

n) •

So, a single iteration is terminated.after evaluating all the

points in the given interval in both first and second sweeps.· A test

of convergence is carried out by one processor. If all the components

of the mesh are obtained with the required accuracy then the procedure

terminates otherwise further iterations are needed until all the

components have converged.

As in the first strategy, in the second strategy the problem mesh

of points (Figure 4.1) is decomposed into subsets each of which are

aSSigned to a processor. Each processor computes its own subset in two

sweeps. In the first sweep, we evaluate each two successive points in _ .. ,

Page 159: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

136

an odd-even (red-black) manner. That is, we first evaluate all the

odd points then followed by all the even points. In the second sweep,

the evaluation is carried out as in the first sweep aspect, i.e., odd

points are evaluated first and then followed by the even ones.

As an example, suppose that n;lO (in Figure 4.1). Then in the

first sweep we start evaluating the odd mesh points in the following

order xl ,x3

,xs ,x7

and Xg then followed by the even points in the order

of x2

,x4

,x6

,xS

and xlO · Whilst in the second sweep, we first evaluate

the odd points and in the order x1

,x3

,xS

,x7

and Xg then followed by

the even points in the order of x2

,x4

,x6

,xa and xlO ·

Then, a test of convergence is carried out by one processor and if

all the components of the mesh points are obtained with the required

accuracy then the procedure terminates otherwise further iterations are

needed until convergence is achieved.

The third strategy is completely different from the other two

strategies, in the manner of partitioning the domain (domain

decomposition) on the processors. The problem domain is partitioned

into subtasks, and each of these tasks are assigned to a processor.

If P is the number of processors and n is the number of points in the

interval (Figure 4.1), then each partition contains points.

Each processor then computes its own n points in two sweeps . p

asynchronously without waiting for the other processors to complete

their computations. At the end of each iteration, each processor

checks to ensure convergence. If convergence is obtained, the processor

sets its flag and tests the remaining flags to ensure that the other

groups have also set their flags, otherwise further iterations are

required.

Page 160: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

137

4.3 EXPERIMENTAL RESULT FOR THE ONE DIMENSIONAL PROBLEM

Consider the linear problem,

+ U = x , (4.3.1)

subject to the boundary conditions,

U(O) = 1, U(n/2) n 2 - 1 • (4.3.2)

The exact solution for this problem is given by,

U (x) = cosx-sinx + x .. (4.3.3)

By following the finite difference discretisation procedure

given in Table 3.1, equation (4.3.1) can be approximated to obtain the

linear difference equation,

2 -u + ( 2-h ) u -u

i-I i i+l

where x,=ih, for i=1,2, ... ,n. ~

2 = -h x

i

The boundary conditions are replaced by the values,

where h =

U o = I,

(~ -0)

(n+l)

u n+l

(4.3.4)

(4.3.5)

The linear system (4.3.4) can be represented in matrix notation as,

(4.3.6)

(4.3.7)

Page 161: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

u = (u ,u2

, ... ,u )T - 1 n

and b (bl ,b2 ,··· ,bn

)T =

2 2 where d=2-h , c=a=-l and b,=-h x" for i=1,2, •.• ,n.

~ ~

138

If we take n even and split A into Gl

and G2

equation (4.3.6) can

be written as

(G +G )u = b , 1 2 - -

where Gl and G2

are as given in equation (3.6.45a,b), with

(4.3.8)

h2

hd=l-2"

(k+l) d (k+l) Hence by' applying the AGE method, u an u can be determined

successively, by,

(k+l) u

(4.3.9)

where r is the iteration parameter. It is obvious that the (2x2) sub-

matrices of (Gl+rI), (G2+rI), (Gl-rI) and (G

2-rI) can be determined and

-1 -1 (Gl+rI) ,(G

2+rI) are easily invertible as shown below.

-1

w

1- - --

l

t­IW

I -1 w r - - - -

-,-o

-,-

'0 -1- _ __ _

:J :-1 I

(4.3.lOa)

Page 162: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

and

, I

I 1

0

, I

-1 -1 ,

I t + --, ,-1 -,- w,

1-wJ

-v -1 I I

--! - ~,- _ L l

I I Iv -11 I -1 v I

,-- --I

I - I ,0 -,- - -1,"- I , '

f', ... " I

-I-I "" "' ... '1

-1--'-'1-

v - -I ,v

'-1 ,-

o

-' -

I , v -1

'-1 , , v

'l , I

vi -L ... - ... - - L

-1 I - -1--1 o ' I ~-I

" " " .. "" I I "', "-

, "' I ' ~

0, '-

~,~ , -1 1--IV -, ' 1-1 v I ,- - -:--l • • v . .-

139

(4.3.10b)

(4.3.1ce)

(4.3.10d)

" " v where w=hd+r and v=hd-r, and the (2x2) submatrices of Gi

, G2

, G1

and

v G

2 have the forms,

" r1

-J v

r1

-1] G = I and G

vJ

hence, Iw II " -1 1 2 G * ~ wJ

' where det=(w -1) • det

Page 163: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

(k+l) . (k) Therefore, the vector u can be determ~ned from U in "two

steps, we first determine u(k+!) explicitly as follows:

ul

(k+! )

f u

2 1

1

1

w --j-

Iw , -4- -----

b2-vu

2+u

3

b3

+u2

-vu3

140

= -- * ___ 1_1_ det * b -vu +u (4.3.11a) 4 I 4 5

- - - I u n-ll

LU J L n

I I .

1 b: +u -vu

J 1 n-l n-2 n-

w Lb -vu n n

1

Now by using the values of u(k+!) obtained from above, we can determine

(k+l) u explicitly.

(k+l) Idet/w: u

l -,---I

-, 1 u 2 w 11 I

1 1 l- e w 1 - 1- - "*. * = *- ,

I det i' .... ' ""-l 1- -, - - ~1 _1-

I 1 w 1 , I

t --; 0 , u 11 W , n-l

f- -t "'1 u I det/w n ,

. The convergence test used was the average test,

(k+l) (k) (k) abs(u. -u. )/(l+abs(u. )) < E ,

1. 1. 1.

-4 where E=O.l*lO .

Ibl-vu l +u2

l (k+!)

b2

+ul

-vu2

b3 ~vU3 +u4 (43.11b)

b I +u -vu n-2 n-3 n-2

b -vu +u n-l' n-l n

b +u -vu n n-l n

This problem was implemented in parallel on the Balance 8000 system

using the three strategies described previously. In all these parallel

implementations a different number of points within the given interval

was taken. The optimal iteration parameter r is also obtained from the

numerical experiments by choosing the one that gives the smallest

Page 164: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

141

number of iterations.

The results from the first parallel synchronous strategy are

illustrated in Table 4.1. From this table, it is clear that the optimal

timing results are obtained when the number of subsets are equal to the

number of available processors.

Table 4.2 shows the results obtained from implementing the second

parallel synchronous strategy. By comparing the results of Tables 4.1

and 4.2, we notice that the times from the first strategy are less than

that of the second strategy, i.e., evaluating the points using the

first strategy takes less time to converge than that of the second

strategy. This is mainly due to the distribution of the components

within each strategy. In the second strategy, we first notice from

its implementation that the number of computational operations are

higher than that of the first strategy. Secondly, there is a

possibility in the second strategy that during the evaluation of its

components the old values may be used. which means extra iterations will

be needed. Whilst in the first strategy, the most recent values of the

components will be used in the evaluation process and a greater rate

of convergence is achieved.

Page 165: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

142

No.of No.of Elapsed time No.of Speed-up

points in seconds r iterations processors

12 1 2.293 0.495 17 1

2 1.325 17 1. 730

3 1.177 I 17 1.948

I 4 1.079

, .17 2.125

5 0.971 17 2.361

6 1.040 I 17 2.204 i

7 1.134 17 2.022

8 1.207 17 1.899

9 1.273 17 1.801

48 1 7.423 0.513 25 1

2 3.791 25 1.958

3 2.595 25 2.595

4 2.000 25 3.711

5 1. 721 25 4.313

6 1.431 25 5.187

7 1.435 25 5.172

8 1.152 25 6.443

9 1.182 25 6.280

TABLE 4.1: The Results from the First Synchronous Strategy

Page 166: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

1 , I

: I

- - --No.of No.of Elapsed time No.of points in seconds

r iterations processors

72 1 13.661 0.515 32

2 6.89 32

3 4.643 32

4 3.533 32

5 3.128 32

! 6 2.419 32

7 2.403 32

.' 8 2.052 32

9 1.855 32

120 1 17.339 0.560 39

2 8.928 39

3 6.112 39

4 4.544 39

5 3.750 39

6 3.336 39

7 2.947 39

8 2.567 39

9 2.202 39 .

TABLE 4.1: The Results from the First Synchronous Strategy (continued) •

143

Speed-up

1

1.982

2.942

3.866

4.367

5.647

5.684

6.657

7.364

1

1.942

2.836

3.815

4.623

5.197

5.883 -

6.754

7.874

Page 167: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

144

No.of No.of Elapsed time No.of Speed-up points in seconds

r iterations processors

12 1 2.64 0.495 19 1

2 1.559 21 1.693

3 1.280 21 2.062

4 1.221 21 I 2.162 !

I 5 1.102 21 I 2.395 i I 6 1.178 21 2.241

i 7 1.258 21 2.098

8 1. 296 21 2.037 · "

9 1.306 21 2.021

48 1 I 7.614 0.513 25 1

2 I 4.013 28 1.897

3 I 2.993 28 2.543 j

4 2.547 28 2.989

5 2.001 28 3.805

6 1.854 28 4.106

7 1. 772 28 4.296

8 1.431 28 5.320

9 1.322 28 5.759

TABLE 4.2: The Results from the Second Synchronous Strategy

Page 168: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

i

--

No.of No.of Elapsed time No.of Speed-up r

points processors in seconds iterations

72 1 13.992 0.515 33 1

2 7.133 35 1.961

3 5.042 35 2.775

4 4.001 35 3.497

5 3.828 35 3.655

6 2.772 35 5.047

7 2.512 36 5.570

8 2.152 35 6.501

9 1.927 35 7.261

120 1 17 .443 0.560 44 1

2 9.128 44 1.910

3 7.113 44 2.452

4 5.544 44 3.146

5 4.750 44 3.672

6 4.336 44 4.022

7 3.947 44 4.419

8 2.567 44 6.795

9 2.403 44 7.258

TABLE 4.2: The Results from the Second Synchronous Strategy (continued)

145

Page 169: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

146

The third strategy was implemented using the asynchronous

approach. The results from this implementation are shown in Table 4.3.

By comparing the results obtained from the first and third strategies,

we notice that the timing of asynchronous implementation takes less

time to converge than that of the synchronous approach and this is

due to the synchronization overheads needed after each iteration in

the synchronous implementation. Also, from both Tables 4.1 and 4.3,

it is clear that the better efficiency can be obtained· by using the

third strategy. This is because the speed-up ratios of the asynchronous

implementation is higher than that of the .synchronous one.

To conclude from the results of the 3 strategies, we can say that

in the implementation of the one dimensional boundary value problem

using the parallel AGE method the best results are obtained when the

problem domain is decomposed into a number of subsets, each of which

is assigned to a processor where the number of processors is equal to

the number of subsets.

There are extra overheads incurred by the system, which degrades

the parallel algorithm performance in both the synchronous and

asynchronous implementations. These overheads are the generation of

the parallel paths and the synchronization at the end of each iteration

cycle.

The timing results obtained from these strategies are diagrammatically

illustrated in Figure 4.4.

The computational complexity for the sequential algorithm for each

point in each iteration is equal to (8 additions + B multiplications).

Now for n mesh of points, each processor will evaluate f.':'.l points with PI

total computational complexity equal to T:[(8 additions + 8 multiplication)

n * -] operations per iteration. p

Page 170: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

147

No.of No.of Elapsed time No.of Speed-up points in seconds

r iterations processors

12 1 2.451 10. 495 14 1

2 1.244 14 1.970

3 1.068 14 2.294

4 0.821 14 2.985

5 0.699 14 3.506

6 0.599 14 4.091

7 0.506 14 4.843

8 0.473 14 5.181

9 0.426 14 5.753

48 1 6.188 0.513 21 1

2 3.109 21 1.990 I

3 2.088 21 2.963

4 1.574 21 3.931

5 1.245 21 4.970

6 1.052 21 5.882

7 0.989 21 6.256

8 0.936 21 6.611

9 0.893 21 6.929

TABLE 4.3: The Results from the Third Asynchronous Strategy

Page 171: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

No.of No.of Elapsed time No.of Speed-up r points processors in seconds iterations

72 1 13.915 p.515 28

2 6.996 29

3 4.701 29

4 3.519 29

5 2.807 29

6 2.342 29

-; 2.119 29

8 1.989 29

9 1. 785 29

120 1 19.663 0.563 37

2 9.904 37

3 6.574 38

4 4.990 38

5 3.948 38

6· 3.058 38

7 2.819 37

8 2.711 37

9 2.493 37

TABLE 4.3: The Results from the Third Asynchronous Strategy (continued) .

1

1.988

2.960

3.954 .

4.957

5.941

6.566

6.995

7.795

1

1.985

2.991

3.940

4.980

6.430

6.975

7.253

7.887

148

Page 172: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

o 2

No. of procs.

149

• First Strategy ... Second Strategy .. Third Strategy n=4l!

FIGURE4.4: The Timing Results for the One Dimensional Boundary Value Problem.

Page 173: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

150

4.4 EXPERIMENTAL RESULTS FOR THE TWO DIMENSIONAL PROBLEM

The concept of the AGE method is now extended to the case.of the

two dimensional problem. Consider the Dirichlet problem on the region

R,

° , (x, y) ER, (4.4.1)

subject to the boundary conditions,

U(O,y) U(l,y) ~ ° , o~y~l (4.4.2a)

U(x,O) ~ U(x,l) ~ sinrrx, o~x~l (4.4.2b)

The exact solution of this problem is given by,

U(x,y) ~ sech~*cosh(rr(y-!»*sinrrx (4.4.3)

By following the finite difference discretisation procedure given

in Table 3.1, equation (4.4.1) can be approximated to,

-u. 1 .-u. 1 . +4u. .-u. . l-u. . 1 ~ 0, l~i,j!,n, (4.4.4) 1-,] 1+,J 1.,J 1.,J- 1.,J+

and

is

x. =ih, Yj~jh, ~

If

even)

we assume

internal y

(0,1)

°

(0,0)

for O~i,j:;nTl.

that R is a regular region and we order the 2

(n n

mesh points row-wise, as shown in Figure 4.5.

sinTTX I

-- - -2

n _. -

0

- -n+l n+2 2n-l 2n

1 2 P In-l n

sin1TX (1,0)

FIGURE 4.5

Page 174: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

, ..

~pplying (4.4.4) at each mesh point yields the system Au=b.

f4 -1

-1

" 4 -1 , ,

, , " , , " , " " -1 4

"

1-1

I " I "

" I

" I -1 f

, , , '. ,

" o /- - --

-1 4 I

!-l_ I~ - - - -I....... ......... .......

............... - ........... -1

I , ................. _ ......... ............... - ...... -I

-1 - - - - - -1-, ,

_ ::::--.._ -1

o

L

'4 -1 I

- I '

sin1fx 1 n-

sin1TX n

o , ,

I sin~rrx 2 n -n

sinrrx 2 1 n -n+ , , I

l

- -1

-1

-1 ~I

*

III 1

u n I

151

(4.4.5)

Page 175: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

If we split A into the sum of its constituent symmetric and

positive definite matrices Gl

,G2

,G3

and G4

we have,

where Gl and G2

are the differences in the x plane and G3

and G4

are the differences in the y plane.

Then,

, : ""'- '

, 1

-,-t- -'- ~ , "",' "- I

, ' "-I I - r- --: ,- ...... - '-I

'-, '- I "-

- 1

,,- ,"I , " ...... ~ '- '- I

" I , " _, - ~ - _I __

:~:~ , I 2 2

(n Xn )

1 , :: ~~ag (A) and,

152

(4.4.6)

(4.4.7a)

(4.4.7b)

Page 176: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

and,

n

2 ____ ..11-1 n __ _

-1 I

1 1

--I---~ , I, , t'" '. I

" 1 I " , L

-1'---

'I

1

o 1 - __ I

1

1

1

-1 1

1 , -1 1 I

-1- -l,-, " ,

153

2 2 - - - __ n -1 n

o

+------

I ',',',0' , - - - - - - - -1- -' ~

Tl I I I

(4.4.8a)

1 2

111 - - t- -

2 I 1

o

3--------n-

, 1

-1 1

1 1 --'----h 1

,0 1 , 3 1 -1 1 , ,_I -, - - 7,- r - - -t I

I " ,

- -1-

o

" " , , " 1'1 ___ 1._ 'I , -1 ' , '-I 1

- :- -~-.: - -:-, , 1

-'- -..0-'=-1 1

------1

o

I

1-1 1- -

r

-1 ' I

I' t-I, ' " , I, " "I 1 ' , , 'I

o

-1 o

I-~ r. 1 ,I

I ,-I

o

1 2 2 (n xn )

2 n

I

I

'I f - -, - ,- - ,- (4.4.8b)

1- r - t- - 1- - j

1 -11 1 1 0 1

,- 1.::.1_ ].1,"" __ -1-, " 1

1- r - -1-',011 -1--;-o 1 1 1

- L- _:-1 ~ ;-J '1 '2 2 , (n xn )

,- 1-

Page 177: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

(;4

154

Also, by reordering the points column-wise i.e. along the y-

direction, then we find that G3

and G4

have the same structure as G1

and G2

respectively, i.e.,

1

1 11 -1

n+l- - - ~ - -

o

2 - - - -

-I

I

- n

- - - 1

n+2 ,

-1 I

11 -1----1-

, , I

I' "" 1

I I n

1 '<'" ,"," I -----~---

2n I

I

1 I

o

(n-l)~1

n2 L

1 n+l 2n+l- - - -

1 ~~---T-~--,1 -11 1

1 .1 1 0 n+l

1

11 1

1_1

- - - 2 - - - -n -I 1

- - L_I I

2n+l ~~ !:_ L_ I 1

G2

; 1 1 2

- - I--

1 I ~, 1

I' " '~ I I - ;- - 1- -, - - - _1 __

o I ,1 -1, 1

~ '_1 1 i 1 _-1--'. _ _ ~ _L: ____ _

_ 1. , '1 -, __ 1..:1_ " , " ,

2 2n---(n-l)n n

I

o

o

, -"''-I - _ __ , 11 I

;­I n

- -1-I 11

T-----

n

I I

2

o

-11

I 1 '-1 1_ _lL

I," I " ' - ~'t

1 1 1 c 1-1

c -+ -I

- -j --1

(4.4.Bc)

(4.4.Bd)

Page 178: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

t

Figures 4.6 and 4.7 illustrate the way the points are

,.., "" represented in Gl ,G2 ,G3

and G4

in the x and y-directions respectively.

I I

: : I I , I

I , I I I

I I , I , , , ,

I I , , I 1 I , ,

I , , I

I I I I , I

, I

, I ,

I , ,

J I'-'

y

1--- -- - --

I : I I , I , I I ,

- - - - - I ,

- - - - - I ,

. -- .

--I , 1 ,

I I I , I , I

I I , , r--'

t

x

FIGURE 4.6

y

I I I

,_ - - .1 , , - - - - 1

I

------------

- _____ L

r-4'----- - -------

t

FIGURE 4 _ 7

155

t

Page 179: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

156

The Douglas-Rachford formula for the AGE fractional scheme then

takes the form,

( ) (k+l/4)

G +rI u 1 -

(k) (rI-G -2G -2G -2G)u +2b

1 2 3 4- -

(G +rI)u(k+!) 2 -

(k) (k+l/4) ~ G~ +~

(G +rI)u(k+3/4) 3 -

(k) (k+!) ~ G3~ +r~ .

(k) (k+3/4) G4~ +~

(k+l/4) (k+!) (k+3/4) Then we can obtain the values of U I U , u and

(k+l) , ~ respectlvely from,

and

(k+l/4) -1 (k) u ~ (Gl+rI) [(rI-Gl-2G2-2G3-2G4)~ +2b] ,

u(k+!) (G2+rI)-1[G~(k)+r~(k+l/4)]

u(k+3/4) ~ (G3+rI)-1[G3~(k)+~(k+!)]

(k+l) u

(4.4.9a)

(4.4.9b)

(4.4.9c)

(4.4.9d)

(4.4.lOa)

(4.4.l0b)

(4.4.lOc)

(4.4.lOd)

Since the matrices (G,+rI), i~1,2,3,4 are all (2x2) block sub-1.

matrices, then they are all easily invertable as shown below. To find

the inverses of the above systems for Gl

, we let w~l+r, so,

.- II :1 " _1 *Iw ~ Gl

~ , then Gl

~

det lL wJ '-

where 2

det~ (w -1) , then,

Page 180: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

(G1+rI)-1 ~ 1

det *

and,

1 * ~ det

157

~ :! : 0 - - - .1- _I ___ I

I''', I o I'~~'I ,

- - -1- ~

L

L

o I , W I I I

11

o

- - '1,

I '1 ..... - - - - - - - - -/ wl

- - - -t I,' " I "," I " , , ,

, " , ___ 1 ___ -,-Iw l'

, - -, - ,- - - , 1, wl I 0 ~ r - ...., ~,\, , "

w I

, I 1 I ,0 ,1 __ w+,,, _I_

t'" I ':""" I

,- - -.!... ~~~ -I IW o I ,

I - -. I ,

-, 1

-I

'1

I

1

c

(4.4.11a)

l

'1 -I - - - L , "

w, -, I , _1.. __ '_,_

o

'det/wl L - "

~, 'I , , 4-,1- -

,det/w I _.1 __

-----1 I .- :- -J _- _-I

Iw , ,1 I

11 1

2'" L , , I' , ~

o 1 r .l­

f , \.1 -+ - !.. ~ -I­

l, o -~ ,

IW

I '1 w I

r ~--l

:det/~ (4.4.11b)

Page 181: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

-1 -1 The inverses (Gl+rI) and (G

2+rI) are numbered in the order

1,2,3, ... ,nxn, in the x-direction. Also (G3+rI)-1 is similar to

-1 (4.4.lla) and (G4+rI) is similar to (4.4.llb) but they have a

different ordering and are applied in the y-direction.

We now consider the iterative formulae, equation (4.4.10) at each

of the four intermediate levels:

(i) For the solution at the first intermediate level (the (k+l/4)th

iterate), equation (4.4.l0a) is rearranged as,

u (k+l/4) -1 (k)

(Gl-i-rI) [(rI-(Gl+2G2+2G3+2G4))~ +2£1

(k) Since the coefficients of u , are all matrices of the same size, we

can add up their elements paying attention to their direction and

r -1

7 -2

-2 7

I , , " "

c = I

-2

-1

'-2 I; I; 1 \

, 1 , " \ ,

" ,

, -1

0

" , 7

-1

" -2' , \ \ \

7 , -'.21 ",-1-:2-, _-" ' ."i

-=- - -=--2 ~ ~I '-2 17 -1 , , , ' . , I

, \

I I \

\ ,

-21

'-1 7 I

-2

-2

7 ,

, ,

o

-1

, "

" ., ,

" -1

-2

" , 7

7

-2

-1

158

-1 7

(4.4.12)

Page 182: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

159

then, (k+l/4) -1 (k) u = (G

l +rI) [(rI-C) ~ +~)

Now let t=rI-7, where 7 is the value along the diagonal of C.

Then the new coefficient matrix of ~ (k) , will have the same structure

as C, but will have the value t along the diagonal. Hence,

(k+l/4) u

1 det

f 11

! I C 1 Wl - --' - --I I,', . ~ 'I

I-~ , ,

- -I -I ~,

0 I IW 1 1

o I

l ~ Jl_ w-l L _____ J

" I' ", 1 ,,' ,I

* - _ _ _ - J ~,'-t 1 w 11

1

T --

o 1

11 1

I r-

:0

1

10

L I ':1

~ (k) (k) (k) I . 1 ,

tUl +u2 +2un+l+2sLnnxl

(k) (k) (k) (k) . ul +tu2 +~u3 +2un +2+2sLnnx

2 I

(k) (k) I (k) (k) . 2u 2+tu l+u +2u2 1+2sLnnx 1 n- n- n n- n-

(k) (k) (k) . u l+tu +2u

2 +2sLnnX

n- n Inn I

(k) I(k) (k) .

1

wJ

12u 2 +tu 2 +u 2 +2swnx 2 n -2n+l ,n -n+l n -n+2 n -n+l

2 (k) + (k) +t (k) +2 (k) u 2 u 2 u 2 u 2 +2sinnx 2

n -2n+2 n -n+l n -n+2 n -n+3 n -n+2 I I I

(k) (k) (k) (k) . 2u 2 +2u 2 +tu 2 +u 2+2sLnnx 2

n -n-2 n -2 n -1 n n -1

(k) (k) (k) . 2u 2 +u 2 +tu 2 +2sLnnX 2

n -n-l n -1 n n

*

(4.1O.13a)

Page 183: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

160

By carrying out these matrix vector operations, we obtain the

(k+l/4) values of u at the first intermediate level.

(ii) For the solution at the second intermediate level (the (k+!)th

iterate, equation (4.4.l0b),

(k+!) _ (G + I)-l[G (k) (k+l/4) 1 ~ - 2 r ~ +~ ,

can be written in a matrix form as,

(k+! ) ·u

1 det *

det/w I

I

L

I W

11

,0 I

"I

le 1 _wL

, :::: 1-I" , I ", '

_I

0

-, -IW

11 r

_1-

I

-I

I

11

I ,- - -I I 0

- --T

I ..

o

-r

-, .l-

10

wl 1 ' 1 , "-

'\1 1 ,

I- ~ 'W

11

- _I I

-

--l 1,

Wl

'-, ,det/w ,

*

Page 184: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

U +ru1 r(k) (k+l/4)

~k) (k) (k+l/4) u2

+u3

+ru2

(k) (k) (k+l/4) u2

+u3

+ru3 , ,

, (k) 1 (k+l/4) u +ru

n n ,

I (k) 1 (k+l/4) u 2 +ru 2

n -n+l n -n+l I

161

(k) (k) (k+l/4) u 2 +u 2 +ru2 (4.4 .13b) I n -n+2 n -n+3 n -n+2

(k) (k) (k+l/4) u 2 +u 2 +ru 2

n -n+2 n -n+3 n -n+3 1

I u 2 +ru 2 Ln n

(k) : (k+l/4) j The multiplication of the above matrix vector will give the values of

(k+! ) u .

(iii) Now for the third intermediate level (the (k+3/4)th iterate).

If we reorder the mesh points column-wise parallel to the y-axis,

equation (4.4.10c) is transformed to,

Then we have,

(k+3/4) u ~

-1 (k) (k+!) ~ (Gl +rI) [Gl~ +~ 1

w l' 1

1 '0 ~_~I __ L __ I

," 1 I,,, " I , " _..1_'-' __ I o 1 ,w 1

o

(k+3/4) u _1_* ___ 1-

, 1 1 ,1 w

-,-" -l-~ , "I

1 , , , ' 1 -1------

Iw l'

det

1 o

*

Page 185: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

r.: (k) (k) (k+!) ,ul +un+1 +ru

1 (k) (k) (k+!)

u +u +ru 1 n+l n+l

I

(k) : (k) (k+!)

l

u 2 +u 2 +ru 2 n -2n+l n -n+l n -2n+l

(k) (k) (k+i) u 2 +u 2 +ru 2

n -2n+l n -n+l n -n+l I , I

(k) (k) (k+i) u +u +ru

n 2n n (k) (k) (k+!)

u +u +ru n 2n 2n

I (k) I (k) (k+!)

u 2 +u 2 +ru 2 I n -n n n -n

It (k) (k) (k+i) u 2 +u 2 +ru 2

n -n n n

where c stands for column-wise ordering.

Hence the values of u(k+3/4) will be obtained from the above

matrix vector multiplication

(iv) At the fourth and final level (the (k+l)th iterate). In a

similar manner, equation (4.4.10d) is transformed to,

Then, we have,

(k+l) u -c ) -l[ (k) (k+3/4)] ; (G 2+rI G~ +~

162

(4.4.13c)

Page 186: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

(hI) u -c

1 det

fCIet/w I .I--I I' -1-

, W , 1 0 I

'1 w' 1 1 --t- - , - - -I

h,' I 1

"" 'I I '-.' , - - - ,:J 1'- --' , ,w

,0 , -' -

~l _ wJ __ , I , ;det/w

- - - I" __ - ~ - ---!. _1-

* - - -

~ ,~ , f ,"

- Ideth.' ,- - .-!

I

1 '

w

o 1- - -I...!

l ruik) +ruik+3/4) l

(k) (k) (k+3/4) u +u +ru

n+l 2n+l n+l (k) (k) (k+3/4)

u +u +ru n+l 2n+l 2n+l ,

I (k)' (k+3/4)

u 2 +ru 2 n -n+1 n -n+1

I , (k) I (k+3/4)

u +ru n n (k) (k) (k+3/4)

u2n

+u3n

+ru2n

(k) (k) (k+3/4) JI u +u +ru I 2n :3n 3n

l (k) 1 (k+3/4) u 2 +ru 2

n n

o

0

-

- I -11

- -,

10

, 'I

163

l

T

I

T -I - -w l ,-,

,det/w

(4.4.l3d)

then the values of u(k+l) will be obtained from the above matrix vector

multiplication.

*

Page 187: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Hence, the AGE scheme corresponds to sweeping through the mesh

parallel to the coordinate x .and y axes involving at each stage the

solution of (2 X 2) block systems. The AGE iterative procedure is

continued until convergence is reached.

Again, in all these parallel implementations the optimal

iteration parameter r was obtained from the experiments by choosing

the one that gives the best execution time.

Tables 4.4, 4.5 and 4.6 represent the results obtained from the

implementation of these strategies. As in the one dimensional

problem, the asynchronous strategy performs better than the other two

strategies, giving much smaller elapsed timings and show a near linear

speed-up. This is due to the absence of any synchronization points,

which implies less overheads and no contention for shared data.

Also, from these three tables it is clear that better efficiency

can be obtained by using the asynchronous approach rather than the

synchronous approach. This is because the speed-up ratios of the

.asynchronous implementation is higher than that of the synchronous.

Figure 4.8 shows the run time results obtained from using the

three parallel AGE strategies when the matrix size of the problem is

equal to 12x12.

In .conclusion, we can say from Figure 4.8 that the best results

are obtained when the third strategy of the parallel AGE iterative

method is used to solve the two dimensional boundary value problem.

164

Page 188: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

165

. Matrix size No.of Elapsed time r No.of Speed-up processors in seconds iterations

12x12 1 32.992 1'-.600 11 1

2 16.849 11 1.959

3 11. 535 11 2.860

4 8.888 11 3.711

5 7.650 11 4.312

6 6.362 11 5.185

7 6.380 11 i 5.171 I I

8 5.122 11 I 6.441 I I

9 I 5.256 11 I 6.277 , ,

24x24 1. I 151.796 1.443 17 1

2 76.555 17 1.982

3 51.596 17 2.942

4 39.263 17 3.866

5 34.731 17 4.370

6 26.879 17 5.647

7 23.845 17 6.365

8 21. 211 17 7.156

9 19.877 17 7.636

TABLE 4.4: The Results from the First Synchronous Strategy

Page 189: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Matrix size No.of Elapsed tilDe r No.of processors in seconds iterations

36x36 1 288.991 1.037 23

2 148.803 23 I

3 101.867 I 23

! 4 75.747 I 23

5 62.502 I 23 I

6 55.607 I 23

7 49.118 23

8 42.792 23

9 36.713 23

48x48 1 500.095 0.900 27

2 252.324 27

3 172.438 27

4 132.491 27

5 103.213 27

6 93.055 27

7 82.777 27

8 73.408 27

9 64.208 27

TABLE 4.4: The Results from the First Synchronous Strategy (continued)

166

Speed-up

1

1.942

2.836

3.815

4.623

5.197

5.883

6.753

7.871

1

1.981

2.900

3.774

4.845 I·

5.374

6.041

6.812

7.788

Page 190: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

167

Matrix size No.of Elapsed time r No.of Speed-up processors in seconds iterations ,

12x12 1 38.339 1.600 13 1

2 19.912 14 1.925 , 3 13.475 14

I 2.845 I

!

4 10.549 I I

14 3.634 I

5 9.266 14 4.137

I 6 7.533 14 5.089 I 7 7.217 14 5.312

8 6.181 14 6.202

9 6.072 14 6.314

24 x 24 1 175.358 1.447 21 1

2 89.104 23 1.968

3 60.367 23 2.904

4 46.396 23 3.779

5 40.191 23 4.363

6 31.936 23 5.490 I·

7 27.635 23 6.345

8 24.744 23 7.086

9 25.001 23 7.623

TABLE 4.5: The Results from the Second Synchronous Strategy

Page 191: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Matrix size No.of Elapsed time r No.of processors in seconds iterations

36 x36 1 335.788 1.033 27

2 174.223 29 I i 3 119.875 29

i 4 I 88.776 29

5 73;784 I 29

6 63.418 30

., 52.647 30

8 50.158 30

9 48.182 30

48x48 1 280.361 0.893 33

2 293.139 33

3 201.763 33

4 156.271 33

5 125.217 33

6 109.466 35

7 96.504 35

8 83.092 35

9 76.452 35

TABLE 4.5: The Results from the Second Synchronous Strategy (continued)

168

Speed-up

1

1.927

2.801

3.782

4.550

5.294

6.378

6.694

6.969

1

1.979

2.876

3.713

4.634

5.301

6.013

6.984

7.591

Page 192: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

169

Matrix size No.of Elapsed tilDe r No.of Speed-up processors in seconds iterations

12x12 1 30.699 1.600 9 1

2 15.602 9 1.967

3 10.631 9 2.887

4 7.922 9 3.875

5 7.131 9 4.305

6 6.021 9 5.098

7 4.723 9 6.235

8 4.103 9 7.482

9 3.929 9 7.813

24x24 1 132.702 1.441 17 1

2 66.795 17 1.989

3 44.069 17 3.011

4 34.081 17 3.893

5 29.014 17 4.573

6 23.568 17 5.630

7 20.259 17 6.550

.8 18.256 17 7.268

9 17.235 17 7.699

TABLE 4.6: The Results from the Third Asynchronous Strategy

Page 193: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Matrix size No.of Elapsed time r No.of Speed-up processors in seconds iterations

36x36 1 231.932 1.030 20

2 116.612 20

3 78.355 20

4 58.662 . I 20

5 46.786 I 20

6 39.045 , 20

7 33.662 20

8 31.968 20

9 29.037 , 20

, 48x48 1 493.327 0.903 i 25

2 245.332 25

3 168.241 25

4 12".493 25

5 98.244 25

6 88.243 25

7 78.763 25

8 69.447 25

9 62.027 25

TABLE 4.6: The Results from the Third Asynchronous Strategy (continued)

1

1.988.

2.960

3.953

4.957

5.940

6.890

7.255

7.987

1

2.010

2.932

3.869

5.021

5.590

6.263

7.103

7.953

170

Page 194: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

j

8

6

4

2

o 2 4

No. of ProC9.

6

-Go First Strategy .. Second Strategy ... Third Strategy

1'1 = 2.'t

8

171

10

FIGURE4.8: The Timing Results for the Two-Dimensional Boundary Value Problem.

Page 195: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

172

4.5 THE AGE METHOD FOR SOLVING BOUNDARY VALUE PROBLEMS WITH NEUMANN

BOUNDARY CONDITIONS

In this section, the AGE iterative method as discussed previously

in Section (3.5), is used to solve a boundary value problem with

boundary conditions involving derivative or Neumann boundary conditions.

Here another formulation of the AGE method using the D'Yakonov

[D'Yakonov, 1963] splitting formula is used.

4.5.1 Formulation of the Method

Consider the differential equation,

d2

U - --- + q(x)U ; f(x) , ~2

over the line segment a~x~b, subject to the boundary conditions,

i.

ii.

iii.

dui ~I a

U(a)

001 ~I a

= a , U(b) ; a ;

a, dui ; S ~Ib

; a, ~Ib ; S·

(4.5.1)

(4.5.2)

Here, a and a are given real constants, and q(x) and f(x) are given

real continuous functions on a~x~b, with q(x)~O.

For the differential equation (4.5.1), the strategy of the finite

difference method is to replace the above equation by a difference

equation. Hence, we cover the interval a~x~b, by a uniform mesh of

size, h ; (b-a)

(n+l)

and denote the mesh points of the discrete problem by,

(4.5.3)

Page 196: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

173

Therefore, the finite difference replacement of equation (4.5.1) can

be obtained (using Table 3.1) and is given by,

+ q,u, 1. 1.

with a truncation error of order O(h2

) .

In matrix notation (4.5'.4) can be written as,

A~ = £ '

(4.5.4) ,

(4.5.5)

where A and b have the following forms, according to the boundary

conditions, these are:

i. Idl c -1

fl -I

fbl l 1 I

a2

d2

c2 u

2 Ib~ 1

.... 0 i , .... I

" " ....

I ....

-, , " , -, , " , I ........ .'" .,

I , , "

.... I

>j Ib' 0

....

l '-il-l d n-l cn_11 I n-l

a dn J ~n J n Ln

(4.5.Ga)

2 where d, = 2+h q, , a, c, = -1, i=1,2, ... ,n, with a

1 c = 0

1. 1. 1. 1. n 2

and bl

= a+h f (xl) , 2

i=2,3, ... ,n-l, b, = h f (x, ) , 1. 1.

2 b = h f(x )+2hS.

n n

Page 197: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

ii.

-.. -.. -.. -.. -..

o

L

where c, = l.

and, b, = l.

iii.

-..

" -.. -.. -..

a n-1

o

d n-l

a -1 n

-1, i=I,2, ... ,n, with al

2 2 h f(x,), b h f(x )+6,

l. n n

= c = 0 n

b n-1

Ibn L J

174

(4.5.Gb)

i=2,3, ... ,n-I.

l = I (4.5.Gc)

Let us proceed with equation (4.5.Gb), and follow the approach of

Evans [Evans, 1985], by splitting A into,

(4.5.7)

where,

Page 198: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

and

1

o I hd

2 I

1 I - - - - 1,-,- - "I - - -

I" , ,-" , I

I ,'" 'I , , ____ -' ,'I

- - - Thd

- -I n-l I o

L 'a , n I

hd I 1 - - _1_

1 hd I 2

L _ _ _

1

'a , 3

- --I

, - 1- - ,.

o

l

c n-2

hdn

_l

where n is even representing an odd number of

1

-1--

1

1 ,hd, I , I!.J ,

intervals,

175

(4.S.8a)

(4.5.8b)

and hd.=d./2. ~ ~

By using equation (4.S.7), the matrix equation (4.5.5) can be written

in the form,

(4.5.9)

Another formulation similar to (3.6.48b), with the same accuracy

can be derived from the AGE method using the D'Yakonov [D'Yakonov, 19631

splitting formula which can be written explicitly, as,

(k+!) -1 (k) u (Gl+rI) [(Gl-rI)(G2-rI)~ +e.1 (4.5.lOa)

u(k+l) = (G2+rI)-1[~(k+!)1 , (4.5.10b)

where u(k+!) is an intermediate value and r is the iteration

parameter.

Page 199: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

and

Hence, from equation (4.5.10) u (k+t) (k+1)

and u are given by,

1 (k+t)

i 1

1 det

l (k+1)

1 det

*

~ I L

-c 1

W2 _.1

o

I, \

I \ \ 1\ \ \ I I , \ \ I , ' ,

, \ I

;- -.:..'.! I W I n-1

, -a , n

o I - --\

i i \

f *

(k) 2 I (k) (k)

l

a v u +v U +c v u +b n-1 n-1 n-2 n-1 5 n-1 n-1 n n-1

J a a u(k)+a v u(k)+v2u(k)+b L n-1 n n-2 , n n-1 n-1 n n n

ret I I

w~ L ___ I I W -c I

I 1.2 2 I

1_ - :.-~3_ w3 I'

\' \ * I ,\ \

I I \ \ ,

,\ \ , , , 1- -:-I 0

1- -:-L

--~ I

I _ 1_-

IW I n-2

I-a I n-1

I - '- -/ -c I

n-2 1 I W ,

_n-_11 _ I -i I de1:.1 IW J I n

* I I I I

176

(4.5.11a)

(4.5.11b)

Page 200: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

177

where, v.;hd.-r, w,=hd.+r, i=1,2, ... ,n and det=w.w, l-c,a. 1. ~ ~ 1. 1. 1 1.+ 1..1+

By carrying out the vector multiplications in equation (4.5.11a,b)

(k+t) we obtain the values of u

4.5.2 Numerical Results

d (k+l)

an u .

A number of experiments were conducted to demonstrate the

application of the AGE algorithm with D'Yakonov splitting on boundary

value problems. The iteration parameter r was chosen so as to provide

the most rapid convergence and the convergence criterion was taken as

-6 £=10 .

Consider the following problem taken from Fox [Fox, 1957],

d2

U

dx2

- U = 0 ,

subject to the boundary conditions,

0<':>0:.<1

: I = 1 , U(l) = 0 •

Ix=o

The analytical solution is given by,

where,

U

1 A = -.=....~

2 (l+e )

x -x Ae + Be

andB 2

-e

(4.5.12)

(4.5.13)

(4.5.14)

The following tables show the results obtained by implementing

this method for a different number of points.

Page 201: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

178

n=10, r=0.560

Numerical Sol. Exact Sol. Absolute Error Relative Error Number of iterations

-0.67375 -0.67371 0.00004 0.000059 23

-0.59144 -0.59140 0.00004 0.000067

-0.51401 -0.51397 0.00004 0.000077

-0.44083 -0.44080 0.00003 0.000068 .

-0.37130 -0.37127 0.00003 0.000080

-0.30485 -0.30482 0.00003 0.000098

-0.24090 -0.24088 0.00002 0.000083

-0.17898 -0.17894 0.00004 0.000223

-0.11851 -0.11848 0.00003 0.000253

-0.05901 -0.05900 0.00001 0.000169

TABLE 4.7

Page 202: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

179

n=16, r=0.590

i Number of Numerical Sol. Exact Sol. Absolute Error Relative Error iterations

-0.70408 -0.70405 0.00003 0.000042 28

-0.64899 -0.64895 0.00004 0.000046

-0.59616 -0.59610 0.00006 0.000100

-0.54533 -0.54530 0.00003 0.000055

-0.49645 -0.49640 0.00005 0.000100

-0.44925 -0.44921 0.00004 0.000089

-0.40363 -0.40358 0.00005 0.000123

-0.35938 -0.35934 0.00004 0.000111

-0.316309 -0.31635 0.00004 0.000129

-0.27448 -0.27445 0.00003 0.000109

-0.23353 -0.23350 0.00003 0.000128

-0.19340 -0.19336 0.00004 0.000206

-0.15392 -0.15389 0.00003 0.000194

-0.11499 -0.11496 0.00003 0.000260

-0.07647 -0.07642· 0.00005 0.000654

-0.03816 -0.03814 0.00002 0.000524

TABLE 4.8

Again, the same three parallel strategies discussed earlier, were

implemented. The timing results from this implem~~tation gives no

significant difference from the AGE implementation with Peaceman-

Rachford formulation (Section 4.3).

Page 203: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

7

6

5

4

2

1

2 4

No. of Procs. 6

-Go First Strategy ... Second Strategy .. Third Strategy

h " It.

8

180

10

FIGURE4.9: The Timing Results for the one-Dimensional Boundary Value Problem with D'Yakonov.

Page 204: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

181

4.6 THE AGE METHOD FOR SOLVING STURM LIOUVILLE PROBLEM

Consider the differential equation,

d dU dx(P(X)dx) - q(x)U + ApU : 0 , (4.6.1)

where p,q and p are real functions of x in the interval a~x~b, subject

to the boundary conditions,

U(a) : Cl, U(b): S , (4.6.2)

where Cl and S are given real constants.

The problem is to determine the unknown function U and the unknown

parameter A using the AGE iterative method.

x =a o FIGURE 4.10

x n-l

x n

x :b n+l

Now when the interval a~x~b is divided up into n equal subintervals

h (Figure 4.10) and the derivatives approximated by the central

difference expressions (Table 3.1), we obtain,

Pi --2(u. 1-2u.+u. 1) h ~- ~ ~+

i=1,2, ... ,n and h = (b-a) (n+l)

- q.u. : -Ap.u. 1. 1. 1. 1.

In matrix notation (4.6.3) can be written in the form,

Au : AU

where,

(4.6.3)

(4.6.4)

Page 205: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

182

Idl cl -I la2

d2

c2 0

"-, "- "-

"- "- "-

I "- "- "-A = "- "- (4.6.5) "- , , "- "- I "- ,

"-, , , , , 0 "- , ,

c I a d n-l n-l n-ll

a d J n n

2 -(2p,+q,h ) d, ~ ~ l<i<n = , ~

h2

p,-p~ h/2 ~ ~ 2<i<n (4.6.6) a, = ,

~ h

2

p, +p~ h/2 ~ ~ l<i<n-l c, =

~ '2 h'

and >: /;;\ ' l<i<n ~ , - :A !~' 11

"-~

The tridiagonal matrix A is real and diagonally dominant (if,

d,>-a,+c,) with positive diagonal entries which are ideal conditions to ~ ~ ~

consider iterative methods of solution [Varga, 1962).

4.6.1 Method of solution

By splitting the matrix A into the sum of two submatrices,

where, I

C ! - --I

1

c -I n-ll

hd i nJ

(4.6;7)

(4.6.8a)

Page 206: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

and ~d~ ~ __

I hd2

I I a 3 [- -- - -

I I

-t-

-I _

hd 1 3 I __ _ 1_

" ' I, , "

I ' , , r , , .

o , 1

. I I

','~' -I-- - 'hd - c

-l

1 I n-2 n-2 I

e" I

- - - J - - - ~ a~_~ - ~d,,:.l ~ - -J . '1 I hd I' : n

if n is even, and hd.~d,/2, i~1,2, ... ,n. ~ ~

183

(4.6.8b)

By using (4.6.7) the matrix equation (4.6.4) can now be written

in the fona,

and by following a strategy similar to the ADI method, [Peaceman,

19551 , (k+!) d (k+l) b d t ' d' I' 'tl b u an u can e e erml.ne llDp l.C 1. Y y,

and

(Gl +rI) ~ (k+!)

(k+l) (G2+rI)~

- (k) (k) A~ - (G

2 -rI) ~

~ ~u(k+!)_(G -rI)u(k+!) - 1-

or explicitly by, (k+!)

u -1 (k) (k)

(Gl+rI) ['X~ -(G2-rI)~ 1

and u(k+l) ~ (G2+rI)-l[A~(k+!)-(G2-rI)~(k+!)1

where r is the iteration parameter, given by,

(4.6.9)

(4.6.10)

(4.6.11)

where ~ and v are the minimum and maximum eigenvalues of the submatrices

of Gl

and G2

From equation (4.6.11) u(k+!) and u(k+l) are given by,

Page 207: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

and

and

fl Tk+!)

u2

1 *

I det 1

: I [n-l l un J

I 1 * =

I det

un -

J Lu n

~: , 2

1-

l 0

-c I 1,

, , "

, , I' " ,

, 1

" , , I . ," , • I

~--~~-'Wn~l

I-a , n

fdet/ wl

,

1- - - ~ - - - - -1 Iw -c 1

: 1 2 21

o l -----1

1*

184

(k) (k) ul -vlul

- (k) (k) (k) A u 2 -v 2u 2 -c2u 3 ~ (k) (k) (k) AU

3 -a

3u

2 -v

3u

3 1

, (k) (k) (k) A U -a u -vu

n-l n-l n-2 n-ll ~ (k) (k) J A U -v u

n n n

(4.6_12a)

W I n-l ---1---,

~ det/w I L

, ,

I n~

- (k+!) 1 (k+!) (k+!) 1 All -v U -c u n-l n-l n n-l n

- (k+!) (k+!) (k+!) 1 All -a u -v u n n n-l n n I

(4_6_12b)

Hence, the numerical procedure would be as follows:

Given an initial guess eigenvalue A (0) and an initial eigenvector u(O)

we determine a new solution ~(l) , by using the AGE algorithm. To

determine a new value of A (1) , we use "RayZeigh's quotient" [Fox, 1957),

Page 208: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

185

(l)A (0) u • u

A (1) (1) (0)

(4.6.13) u • u

Then we continue until the procedure gives no further change in the

value of A. This procedure is only suitable for deriving the smallest

eigenvalue A (n) otherwise diagonal dominance of A will be lost and the

AGE algorithm will diverge.

4.6.2 Numerical Result

In order to explore this method, we present here some numerical

experiments. Consider the problem of Mathien's equation,

d2

U --- + (A-2qcos2x)U = 0 , dx

2 (4.6.14)

\'lhere,

q is constant and A the eigenvalue, subject to the boundary

conditions, U(O) = 0, U(21T) = 0 . (4.6.15)

The following tables show the results obtained by solving this

problem by the AGE method.

n=9, -5

£=0,1*10 ,r=1.500, q=l

A=0.890461, Number of Iterations = 17

The Solution Vector A*U. A*U. ~ ~

0.133369 0.11877 0.11876

. 0.257145 0.22900 0.22898

0.359889 0.32047 0.32047

0.428808 0.38183 0.38184

0.453195 0.40354 0.40355

0.428808 0.38182 0.38184

0.359889 0.32048 0.32047

0.257145 0.22900 0.22898

0.133369 0.11877 0.11876

TABLE 4.9

Page 209: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

186

-5 n=13, £=0.1*10 , r=1.500, q=l

A=0.894524, Number of iterations = 22

The Solution Vector A*U, A*U, 1. 1.

0.081095 0.07269 0.07254 0.159266 0.14243 0.1424,

0.231265 0.20710 0.20687 0.293353 0.26238 0.26241

0.341572 0.30557 0.30554 0.372259 0.33288 0.33299'

0.383806 0.34232 0.34243

0.372260 0.33292 0.33300

0.341571 0.30548 0.30554 0.293356 0.26259 0.26241 0.231261 0.20682 0.20687 0.159269 0.14272 0.14247

0.081091 0.07246 0.07254

TABLE 4.10

Page 210: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

4.7 CONCLUSIONS

This chapter included the study of synchronous and asynchronous

AGE iterative methods, where the mesh of points of a problem to be

solved was partitioned into n processes. The implemented parallel AGE

methods have been used to solve one and two dimensional boundary value

problems. Three strategies of the parallel AGE methods have been used

to solve these problems.

In both the one and two dimensional problem, the third strategy

showed a better improvement over the other two. Since, in the other

two strategies we need to synchronize the processes at the end of each

iteration step, which in turn, degrades. the performance of the

algorithms.

Comparison, between the first and second strategies shows that

better results obtained when using the first strategy. This is due to

the total number of computational operations in the second strategy

being higher than that of the first strategy and also because of the

re-orderings needed in the second strategy.

From the experimental results it can be seen that the shared data

overhead in the third strategy implementation is less than that of the

first and second strategy implementation.

187

To solve a boundary value problem involving derivatives at its

boundaries, by using AGE algorithm, we noticed from the experimental

results, that the absolute errors and the number of iterations is less

than many other iterative methods, whilst, the implementation of the

three parallel strategies gives almost nearly the same speed-up obtained.

Finally, it can be seen that the parallel AGE method is suited for

Page 211: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

the parallel implementation on a MIMD computer which results in the -

almost linear speed-ups obtained from their implementation.

To find the eigenvalues and the corresponding eigenvectors of the

Sturm-Liouville problem,· by the AGE method, we noticed experimentally,

that the results obtained by applying this method is better in

comparison with many other methods.

188

Page 212: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS
Page 213: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

CHAPTER 5

TIME DEPENDENT PROBLEMS,

PARALLEL EXPLORATIONS

We look before and after

and pine for what. is not •.

Shelley.

Page 214: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

189

5.1 INTRODUCTION

The parallel AGE iterative method was developed and implemented to

solve one and two dimensional boundary value problems in Chapter 4.

In this Chapter, we will implement the same three parallel strategies

presented earlier in Section 4.2 to solve one and two dimensional

parabolic equations as well as the one dimensional wave equation.

The parallel AGE method was developed and implemented on the

Balance 8000 system using its available 5 processors. These include

synchronous and asynchronous versions of the algorithm. The results

from these implementations were compared as well as the performance

analysis of the best method presented.

The AGE iterative method with the D'Yakonov splitting formula to

solve the one and two dimensional diffusion equation is also

represented in this chapter. This new strategy yields an algorithm

with reduced computational complexity and greater accuracy for multi­

dimensional problems. The same three parallel strategies were also

implemented on this kind of problem formulation.

A new method for solving the (nxn) complex tridiagonal matrices

derived from the finite difference/element discretization of the

Schrodinger equation is also presented and confirmed by numerical

experiments.

Page 215: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

190

5.2 EXPERIMENTAL RESULTS FOR THE DIFFUSION-CONVECTION EQUATION

This experiment involved the parallel implementation of the AGE

method for solving the diffusion-convection equat~on. Consider the

following problem,

aU a2

u au O<x<1 . F70 E ---p , at

ax2 ax (5.2.1)

with the initial condition,

U(x,O) = ° , O~x~l , (5.2.2)

and the Dirichlet boundary conditions,

U(O,t) ° U(l,t) = 1 t~o . (5.2.3)

In this example the coefficient E and p assumed the same values of 1.

The exact solution is given by,

U (x, t) (ePx/e:.. l )

(eP/':.l) + 2 L

n=l

n L(x-l) ~(_-=l~)~n~n ____ ~ e2E

2 2 (mr) + (piE)

2 2 , ( ) ,.[ (nn) E+P /4EJt

S1n nTTX • e (5.2.4)

A uniformly-spaced network whose mesh points are x,=ih, t,=jk, 1 J

for i=O,l,2, .•. ,n+l and j=0,1,2". "m+l, is used ",ith h=(n:l) ,

T k k=(m+l) and A=h2 ' the mesh ratio.

divided as illustrated.

The real line O~x~l is thus

x n-l x x =1

n n+l

Page 216: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

At the point p(x, ,t, ,) the derivatives in equation (5.2.1) are ~ J+,

approximated by the finite difference discretizations given in Table

3.1. Thus we have,

-;(E+K)U, 1 ' l+(l+E)U, , I-HE-K)U, 1 ' 1 = ;(E+K)U, ,+ 1.- ,J+ 1.,)+ 1.+ ,J+ 1.-1,J

191

(l-E)u, ,+; (E-K)u, 1 ' , 1.,J 1.+ ,]

(5.2.5)

2 2 which is the Crank-Nicolson formula with O(h,k ) accuracy, where

E=EA and K=;pAh. When all the points l~i~n within a line are

considered, equation (5.2.5) generates a tridiagonal system of linear

equations of the form,

d c 1 I~l l [hI l I a d c u

2 b2

/ a d c 0 I " , ,

"- I , , , I I

"-"- "-,

I = (5.2.Ga) "- "-, I , "- "- I I

"- "- J I I I I , , "- "- I

0 , , ,

/bLl/ "- "- d' u'

L a

n-l

a d (nxn) Ln J (nxl) '~n J (nxl)

or A~=!?, (5.2.Gb)

where a=-;(E+K), d=(l+E) and c=-;(E-K).

Also,

;(E+K)UO ,+;(E+K)UO

' l+(l-E)Ul

,+;(E-K)U2

' ,J ,J+ ,) ,J

b, = ;(E+K)u, 1 ,+(l-E)u, ,+;(E-K)u, 1 ' , i=2,3, ... ,n-l .... 1.- ,J 1.,] 1.+ ,J

b = ;(E+K)u ,+(l-E)u ,+!(E-K)u ,+;(E-K)u , n n-l,J n,) n+1,J n+1,J+1

Let us now assume that we have an even number of intervals

(corresponding to an odd number of internal points, i.e. n odd) on

the real line O~x~l. We can then perform the following splitting of

the coefficient matrix A,

Page 217: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

192

A (5.2.7) where,

[h~ :_ _ _ i _ _ ~ __ ~ r ,hd cl'

1 la hd 1 :0 1- -:- - -:;,:,~ :- --I

1 ' , , I 1

I ' , 1

_____ 1_ '~'---J ,I 1 I hd c

1

I 0 I I

_ I : . ,a hd

(5.2;8a)

and [hd c

a hd c ~J ,

(5.2.8b)

where hd;d/2.

Hence, the AGE iterative method with.Peaceman-Rachford formula

. (k+!) (k+l) .. can be applied to determ1ne ~ and ~ exp11c1tly by,

and

(k+!) u

(k+l) -1 (k+i) U ; (G2+rI) [(rI-Gl)~ +£1 (5.2.9)

where the matrices (Gl+rI), (G2+rI), (rI-G

l), (rI-G

2) are represented

by

(5.2.10a)

Page 218: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

193

~ cl ,

l I 0 I

wl I -"Go' - -1-- r-

"" '- I I (G

2+rI) = I "" , (5.2.lOb) , ,

I I " ... ...... 1

r~ -, - - ~- -r-

I 'w c I I

I la I t- -1'-- --f- _w_ ~j , W ,

Ivl I I --1 --t-_-.1 - -1-1 Iv cl

~ I I 0 'a vi ,- -1"- I I ", (rI-G

l) I "" "- (5.2.lOc)

I t '- .... ' ~~ I --.1 I- - -'-'1---

I 0 'v

J L I I 'a , ,

r c il 0 v ----t- - f--" ,

I,"" I (rI-G

2) I "",~

(5.2.l0d) = ," 'I I 1- __ I __ "::......'L, -I I Iv c , .

I c I I I I 1- la

_ v ~j -I -,-, v

L I ,

where w=hd+r and v=r-hd.

It is clear that (Gl+rI) and (G2+rI) are block diagonal matrices.

All the diagonal elements except the first (or the last at (G2

+rI)) are

(2x2) submatrices. Therefore, (Gl+rI) and (G2+rI) can be eaSily

inverted by merely inverting their (2X2) block diagonal entries.

(k+t) Then, from equation (5.2.9), u d

(k+l) an u are given by,

Page 219: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

= _1_ * det

I -a - --I-

I

o L

_....J _

10 I

I , -a I

- - (k) (k) I IVU l -cu2 +bl (k) (k)

-aul

+vu2

+b2

. (k) (k) b vu3 ,cu4 + 3

(k) ! (k) b -ell +

vu n-l n-2 n-2

. -au (k) +vu (k) +b 1

n-2 n-l n-ll (k)

["'n +bn J

194

(S.2.11a) and

l (k+l)

1 det *

2 where det=w -ac.

-, 1- (k+t> b l I vUl + 1

I (k+,) (k+,) 'b vu 2 -eu3 T 2

f -c ;

-a w I - --+--

o

o

, ' - -,---I, " ' I "" " " , I

I

_1-I

" ' " I , ' --r--

-c I I

I

IW

I-a w 1 1 -, ---~:t~~

(k+,) (k+,) b ' -au

2 tvu

3 + 3

I I I

I (k+,) (k+,) b vu -ell +

tl n(~+!> n (k+,) bn-1J -au 1 +VU + n- n n

(S.2.11b)

The corresponding explicit expressions for the AGE equations are

obtained by carrying out the multiplications in (S.2.11a,b). Thus

we have,

(i) At level (k+t>

and

where,

(k+!) u i +l

Page 220: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

195

{ 0 for i=n-l A -aw, B = vw, C -cv, E. wb.-cb. 1 , D =

c2

otherwise, ~ ~ ~+

2 { 0 for i=n-l

~ ~ ~ ~

and A a , B -av, C = vw, E. = wbi+l-ab i , D = ~

otherwise. -cw

with the following computational molecules (Figure 5.1) .

1 k+!

k

1 2

1 k+! k+!

k k

i-I i i+l i+2 i-I i i+l i+2

FIGURE 5.1: The AGE Method At Level (k+!)

(ii) At level (k+l)

(k+l) u

i u. 1 . u. 1 . 2 . e 1.- 1. 1.+ 1+ l. = (P (k+!)+QU(k+!)+R (k+!)+su(k+!)~T )/d tj

i=1,3, .. . ,n-2, (k+l)

lli+l = (-P (k+!) ~QU (k+!) +R~ (k+!) ~S (k+!) T"') Id t

u. 1 +. u. 1 + u. 2 +. e 1- 1. 1+ 1+ 1.

Page 221: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

196

and (k+l) ( (k+!) (k+!) +b )/w u -au 1 +vu n n- n n

where,

{ 0 for i;l 2 p Q ; vw, R ; -cv, 5 c , T. wb.-cb. 1 ,

-aw otherwise ~ ~ ~+

and

{ o for i;l

~ .....,

'V ~ "-' p ;

2 , Q ; -av, R ; vw, 5 ; -cw, T. = -ab. +wb. l'

~ ~ ~+

a otherwise

with its computational molecules given by Figure 5.2.

1 k 1 k

k+!

i-I i i+l i+2 i-I i HI i+2

k+l

k+t

FIGURE 5.2: The AGE Method At Level (k+l)

The computational complexity per iteration per point is 42

multiplications + 28 additions, while it usually requires only two

iteratioIl3 for convergence for small time step k.

Page 222: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

In order to compare the three parallel strategies we list the

timing results with the speed-up ratios of the three strategies in

Tables 5.1, 5.2 and 5.3, where a different number of mesh sizes are

chosen. From these tables, we notice that the asynchronous strategy

(Table 5.3) achieves better speed-up ratios than the other strategies.

These speed-ups are mostly linear, i.e., of order p, where p is the

number of processors that are in use. However, if we consider the

timing results in Tables 5.1 and 5.2, we see that the algorithm of

the first strategy requires less time.

197

The difference in the running times for the three strategies is

due to the fact that in the first two strategies we need to synchronise

the processors, while in the third strategy, each processor works on

its subtasks without waiting for the other processors. Also, the

difference in the running-times between the first two strategies is

due to the reordering needed in the second strategy.

The timing results of the three versions are represented in

Figure 5.3.

Page 223: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

-7 k=O.OOl, t=O.l, A=O.l, £=5.0xl0

No. of points No. of processors

9 1

2

3

4

5

-7 k=0.OO5, t=0.5, A=0.7, £=5.0xl0

No. of points No. of processors

11 1

2

3

4

5

-7 k=0.OO5, t=1.0, A=l, £=5.0xlO

No. of points No. of processors

13 1

2

3

4

5

r

0.5

r

0.5

r

0.5

Time (second) Speed-up

10.330 1

5.695 1.813

4.096 2.521

I 3.296 3.134

: 2.556 4.041

Time (second) Speed-up

12.512 1

6.890 1.815

5.995 2.087

4.339 2.883

3.229 3.874

Time (second) Speed-up

14.791 1

8.968 1.649

7.348 2.012

6.352 2.328

5.181 2.854

TABLE 5.1: The Results From Implementing The First Strategy.

198

Page 224: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

-7 k=O .001, t=O.l, \=0.1, E=5.0xl0

No. of points No. of processors

9 1

2

3

4

5

-7 k=0.OO5, t=0.5, \=0.8, E=5.0xl0

No. of points No. of processors

11 1

2

3

4

5

-7 k=0.OO5, t=1.0, \~1, E=5.0xlO

No. of points No. of processors

13 1

2

3

4

5

Time Speed-up r

(second)

0.5 11.117 1

6.146 1.808

4.454 2.495

3.771 2.948

2.993 3.714

Time Speed-up r

(second)

0.5. 13 .364 1

7.321 1.825

6.562 2.036

4.511 2.962

4.068 3.285

Time r

(second) Speed-up

0.5 15.821 1

9.479 1.669

7.993 1.979

6.897 2.293

5.433 2.912

TABLE 5.2: The Results From Implementing The Second strategy.

199

Page 225: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

-7 k=O.OOl, t=O.l, A=O.l, c=5.0xl0

No. of points No. of processors

9 1

2

3

4

5

-7 k=0.005, t=0.5, A=0.8, c=5.0xl0

No. of points No. of processors

11 1

2

3

4

5

-7 k=0.005,t=1.0, A=l, c=5.0xl0

No. of points No. of processors

13 1

2

3

4

5

Time Speed-up r

(second)

0.5 9.603 1

5.247 1.830

3.779 2.541

2.833 3.389

2.297 4.180

Time Speed-up r

(second)

0.5 11.480 1

6.152 1.866

4.977 2.306

3.788 3.030

2.993 3.835

Time Speed-up r

(second)

0.5 13.544 1

7.971 1.699

6.804 1.990

5.707 2.373

4.669 2.900

TABLE 5.3: The Results From Implementing The Third Strategy.

200

Page 226: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

o 2 3

No. of proc8. 4

- First Strategy ... Second Strategy ... Third Strategy

h:r13

FIGURE 5.3:The Timing Results for the Diffusion Convection Equation.

201

6

Page 227: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

202

5.3 EXPERIMENTAL RESULTS FOR THE TWO-DIMENSIONAL PARABOLIC ·PROBLEM

Now consider the two-dimensional heat equation,

aU at

a2u + --2 + H(x,y,t) , O~K,y~l , t~T , (5.3.1)

ay -t

with H(x,y,t)=sinx siny e -4,

where the theoretical solution is given by,

U(x,y,t) -t 2 2

sinx siny e + x + y , O~x,y~l, t~O . (5.3.2)

The initial and boundary conditions are defined so as to agree with

the exact solution. At the point p(x. ,y.,t ) in the solution domain, 1 J r

the value of U (x, y, t) is defined by u.. where x. =ih, y. =jh for ~/J,r 1. J

O~i,j~n+l 1

and h (n+1) The increment in the time t,k is chosen such

that t =rk for r=0,l,2, .... r

The mesh ratio is defined

A weighted finite difference approximation to (5.3.1) at the

point (i,j,r+t) (Table 3.1) leads to the five point formula,

-ABu. l· l+(1+4~e)u.. l-lBU. l· l-lBu,. 1 1- ABu .. 1 1 = 1. - I J I r+ 1. , J , r+ .1. + , J I r+ 1. , J - I r+ 1. , J + ,r+

A(1-6)u. 1 . +(1-4A(1-6) )u .. +A(1-6)u. 1 . +A(1-6)u .. 1 + 1.- ,J,r 1.,J,r 1.+ ,],r 1.,)- ,r

A(1-6)u .. 1 +kH.. l' for i,j=1,2; ... ,no 1.,J+,r l.,J,r+-z

(5.3.3)

We notice that when 6 takes the values 0, t and 1, we obtain

the classical explicit, the crank-Nicolson and fully implicit schemes

2 2 2 2 whose truncation errors are O(h ,k), O(h ,k ) and O(h ,k) respectively.

Let us proceed with Crank-Nicolson scheme since it is more

accurate and unconditionally stable. Hence, the weighted finite

difference equation (~.5.3) can be expressed in the more compact

matrix form, as,

Au (k+l) BU(k) + b +!l. (5.3.4)

Page 228: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

where,

d

a , ,

a

A

1-I I

C

d , , , , ,

C , , , a

, , d

a ---~ , ,

,

203

..., I

, , o , ,

, C , , dl Cl

(5.3.5a)

1- ..... -- 0 ___ ....... C

I --... ............... -......... I o - - ......... -a I ....... - ..... --~-I c -I

I r a Id C

, I la d , I " o , "

C.

,,", " " , , , ,

, , a I

I l where d=1+2A,a=c=-A/2,

and the matrix B is of the form,

le s

s

f e

f ~

B

o

l where e=1-2A, f=S=A/2.

I IS , I " I \ ,

\

\ \ ,

I __ s-I-

I" , 0 s r, ....... " .... I ...." .... O ," . " . I '.' - - - ~-

I e f. I

I

\ If \ " \

" \ \

\ f'

I

"

s

e

"

" ' " .... ', , " a d' c

o

s ,

a d l 2 2 .-l (n xn )

I I I

-, s 1 (5.3.5b)

--I

I , , " ,

I e si

, , , ". f'

f ~ (n2xn2)

Page 229: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

204

The vector b consists of the boundary values where,

A A A A -[u +u ]+ -[u +u J,-u -t- -:-U , ••• , 2 O,l,r 1,O,r 2 O,l,r+l l,O,r+l 2 2,O,r 2 2,O,r+l

A A A -u + -u ,-[u +u J 2 n-l,O,r 2 n-l,O,r+l 2 n,O,r n+l,l,r

A + -[u +u J

2 n,O,r+l n+l,l,r+l

b. J

lu +~2 . l'o, ... ,o,lu . +A 20,j,r O,J,r+ 2 n+l,J,r ~n+l,j,r+l'

for j=2,3, ... ,n-l,

b ; J" [u +u J + J" [u +u J, lu + n 2 O,n,r l,n+l,r 2 O,n,r+l l,n+l,r+l 2 2,n+l,r

-2AU

2 1 1 ' ••• , lu + lu , ,n+ ,r+ . 2 n-l,n+l,r 2 n-l,n+l,r+l

J,,[u +u J+ J,,[u +u J , 2 n,n+l,r n+l,n,r 2 n,n+l,rTl n+l,n,r+l

and the vector ~ contains the source term of (5.3.3) given by,

g. ; k(Hl . ~,H2' ~, ... ,H . », for j;1,2, ... ,n. J ,J,r+"l ,J,r+"1 n,J,r+"l

We observe from (5.3.5) that A has the form,

~:\i __ 'I~ :~:~ 0 '

A

~ :~ ~-~--I ~ _ L _ 1 - - 1- - --

L I ~i ~ 22 (n Xn )

Now, if we split A into the sum of its constituent symmetric and

Page 230: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

positive definite matrices G1

, G2

, G3

and G4

, we have,

where,

Therefore,

where,

o -, , I

l

o

-;~J 122

(n Xn )

l ! - --I I ---I

diag (G2

) = %ctiag(A) ,

205

(5.3.6)

Page 231: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Fd 1 ,

--1--+ 1 hd cl

1 1 - ,- - --I

1 l o

1

I;:-::~- -11--- - -- - - --I

, ...................

+ ___ ~>':-~ _, __ , __ , __ J

o

L

Cl

1 ,- - - - - --

o

L

I hd 1 1 1

1- - - - - - - -1- - --, 1

1 Ihd cl' 0 1

1 'a hd 1 , ,- -1- - ---1,-, -1-

I" ' , " I '"" J + -'-f--,

1 1 Ihd

o la ,

c

,-

l I

206

(5.3.7b)

Page 232: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

207

also, by reordering the points column-wise, i.e. along the y-direction

we found that G3 and G4

have the same structure as GI

and G2

respectively

i. e. ,

I n+l 2 n+2

lfhd 1 1

1- --1- - _L - -I-- - - -.: n+l I hd cl,

I I, 0 I. 1 1

---" ~ .!:d-.! __ , ____ I t I'" t I , I'" 'I

, - - -1'-':: ~- - --' o 1 Ihd c 1

n 2n - - - - - - -(n-l) n

o

2 n

l I 1

1

h~ , -;: "' _ _ ~ _ _ _ _ _ _ _ _ _ _

I, ~" I ' " I

C

, , ,,' " ,I

, , 1 ---1--'-'--

Ihd I' 1_ -.1 ______ 1-- __ _

I hd c' J~

1_ -.l_

C hd'

__ 1,= _...L --­, , ",,~, 1

- -~ ~ ~:- -- -I C ' I hd cJ I ,

1 ' I a hd (2 2) 1 n Xn

(S.3.7c)

Page 233: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

and, 1 n+l

Cl

~

G =G = 4 2

wi th hd=d/4.

2n+L __ . ___ 2. ___ n_

o

. I ,­, .L

o

2 - -n

l

208

(s.3.7d)

The Douglas-Rachford formula for the AGE fractional scheme then

has the form,

(G +rI)u(k+l/4) = 1 -

(k) (k) (rI-G -2G

2-2G

3-2G)u +2Bu +2b+2a

1 4- - -.<.

(G +rI)u(k+t) = (k) (k+l/4), 2 _ G~ +ru

(k) (k+t) = G3~ +~

(k) (k+3/4) G4~ +~

(5.3.8)

In a similar manner to equation (4.4.9) (Section 4.4), we now examine

the above iterative formula at each of the four intermediate levels:

(i) At the first intermediate level (the (k+l/4)th step)

Using the expression (s.3.8a), we obtain,

(k+l/4) u -1 (k) (k)

= (Gl+rI) [(rI+Gl)-2A)~ +2B~ +2(£+9.)] (5.3.9)

Page 234: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

where,

-+

o I

o

I a _w-.! I ;' ....

10 I

-, -I " ....... "'" I , ........ '" :I

1_ 1_ ..L ,--'_ , I IOw

209

l I

(5.3.10)

cl I '-

,a ~ (2 2) n xn

with w=r+hd.

By letting c=«rI+Gl

)-2A), we have,

c

f -2d -2c

1-2a w-2d -c

I -a w-2d -2c

a-=-=-----

l o

le

\

-2a w-2d -c I

\

\

\

=a w-2g I __ Cl _

1:::-::::..----- __ IC­_-_-_-a __ ~~-I- __

la .w-2d -2c

I ' ,-2a w-2d -c

o

I \ -a w-2d

\

, (

al

"-'-

, "-

,

c

-2c

"-, "-

" , , -2a w-2d -c

Page 235: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Therefore,

1 I I

I I knl I I

1 I

, U

Lnj

r F-2d

1 = --

det

-2c

w-2d

I

L -c 1-2a

I -a w-2d -2e " " " "

" " " "

1 " " " " " " "

1 I

1 !-a--------2a

,.

0

'w ,

o

-,-

-c , ,

l .:....- --

I ~ ---I

-r - -- I

01 _ J'-:~ :!il_

I T

r<· I

" ' I '," I - - ~J-

1 ___ L

--1

" " " "

re I

\

\ \ I

\ I

" " -e I I

\ I \ I

w-2d I \ ,

-a _w-=-2~1 _ 5 L __ _ . , 1\" '\ :c - .­

\\ ......:

--. a! \ \. .

I I w 0, :

: -a .:J

l 1

0 I

I - ______ 1

* - - e

11- - - -- -- - - - - - - I -lw-=2d - -2e

I o

a I' I

\ 1-2a w-2d -c I \ I \

\ , \ ,

I I a,

-a w-2d -2c, ,

-2a w-2d -c

-a W-2dl

-.J

210

*

(k)

~lil U

12

I, 1

U1n +

I

1

Page 236: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

2*

~ : s

's I \

I' " " \

\ \

1

,", " , '" , " \

f e s \ I

I o

f e I S _______ ,-- _.J - - - - - - -f _ ", Is_

- I "",- - 1 -- __ - ,,~ I -- -... __ - -_fl _ ' " I - - s 1 - - - I f -' le s - - - - -

I \. I , \

If o \

\ \

\ I

£1

,-l 1£1 +.'Il

b2

+g2 1- -

1 :

1 I

2 * 1

I 1 1 1 I

b +g ---n-l ---n-ll

I b +g C' ---n ~

)

, e s , ' , , ' " , " , ,

" " f

, ,

e

~ f

211

By carrying out these matrix vector multiplications, we obtain the .

f (k+l/4)

values 0 u at the first intermediate level.

(ii) At the second intermediate level (the (k+t)th step)

From equation (5.3.8b), we have,

(5.3.11)

Hence, equation (5.3.11) yields,

Page 237: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

- ~(k+,)

Ulll u

12 I I

I :

= _1_* u ln det

rw -c 1

I 0 I

.::a_ ~ 1-, -.:.. __ L ___ I

~ __ C,~ __ ~ ___ I:

1 w -c o I' , 1

I -a w i --,- _ ..J _ --I , I

,det/wl ------- - - ,,,- ~

k'" 1 '," ....

o

" ,'! --------'w -c

r - - ---

I. ,J Ln l

r, f c: I la w' 1

0 I

I~- ~~'~:- -:-, - I ~'I - -'- ~

I ,W cl o 1 I I w l

o

l I

o I

__ -L. _ la - 1- __ ,

" 'w I -....! -'- - - - - - -- ------I I" 1 ", I ,,, - - - - - - ,_,..L. - r -,w c

- -I ,- - I

1 '10 1 I 1 1 I a wj-- _...1_ ,- ',,- I I

"" , t ,_-_"" I I

I r ~"~~ -c 1-

10 ' I I

:a w ~ ~J

o

L _ ~_

'0 I_a w I !- 1'-'- -I -- I

" " ' , ........... ,,! I " , -t_ _ _ I _ ....!.. :...J _ ---l _ _

o 1 IW -c 1

- ,

, -1-

I 1 I

U'2J _n

+ r*

I

I-a -l -

U In ,

I I

1 _ , 1

I )

*

The multiplication of the above matrix-vector operands will give the

values of u(k+').

212

Page 238: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

213

(iii) At the third intermediate level (the (k+3/4)th step)

By reordering the mesh points column-wise parallel to the y-axis

we found that

and (5.3.12)

where c stands for a column-wise ordering of the mesh points. Therefore,

equation (5.3.8c) can be transformed to give,

(k+3/4) -1 (k) (k+!) ~(c) = (Gl+rI) [Gl~(C) +~(c) J,

and hence, equation (5.3.13) yields,

_1_* det

Idet/w 1 1 ---Iw

r-_ I -a -I -

1- 1

I , 0 L

1

I I "1 - -I -

-cl 0

w' - 'I, ,- 1-·

I' " """1 , " I j--"'r:--'

w -c I I ,

o

I-a w I - ',_, -I - - - - - -

I'"'' I , " ' , ,,~

- - - - - - - -'ljet/;'- --1--;

_ __ l __ .J _ I

'-

0

I I

IW

1

I-a -,

.!..

-c I I

w'

1

, j, ' I ," I" ,,'!

-I-~ -, W

0

(5.3.13)

l

o

--cl L '-a ~

Page 239: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

214

o

:a w I - - - - - -1- - - - - --_. - . 1 ' ,

""" I +r*

- '- ' , ,," I

~ '---iWi - - 1- - - _.

1_'1- __ '_, I - r . -. Iw C :0

o I J

th f 1 of U (k+ 3/4 ) d f th ere ore, the va ues will be obtaine ram e

multiplication of the above matrix vector operands.

(iv) At the fourth (and final) intermediate level (the (k+l)th step)

Similarly, by reordering the mesh points_column-wise parallel

to the y-axis we have,

Then, the last equation of (5.3.8) is transformed to give,

which yields,

(k+l) u -(c)

(5.3.14)

(5.3.15)

Page 240: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

u nl 1 I , I

(k+l) Iw -c I·

I 0 ~a _ :-' L I

I' ~ \. i , , 1 " ,

--~'-r- --t--I ·w-c

o I

- - -1-,

I-a _..L_

I , 1 1- - - '-

~-* det I

I I L

o

- - --; , , , 1 __ l_,_

I

c 1

,0

1-I

J

, . ! -a

o

c

* - -,

'0 w' -~'-i---!--

" \. ' I : ,,\. 'I

....!... ~~ __ J._ 'w -c I I

:-a w 1 -, -, ,det/~

[U l (k+3/4) 1

I 111 I

IU~ll I ! I I '

J

215

(k+l) The values of u will be obtained from multiplying the above matrix

vector operands.

Page 241: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

216

Thus, the AGE scheme corresponds to sweeping through the mesh

parallel to the coordinate x and y axes. The iterative procedure is

continued until the requirement I (k+l) (k) I· .. u. . -u. . ~e: 15 satl.sfl.ed, where 1.,J 1.,J

£ is the converge criterion.

Now, we concentrate our attention on the implementation of this

problem on the parallel Balance 8000 system. We abbreviate our

discussion in this section only on the first and the third strategies,

in order to compare the synchronous and asynchronous strategies.

Tables 5.4 and 5.5 represent the results obtained from these

implementations respectively. From these tables, we notice that the

asynchronous version achieves better speed-up ratios, where these

speed-ups are mostly linear.

The difference in the running times for the two strategies is

due to the fact that the synchronous version generates a large amount

of communication overheads in the system. Also, the synchronisation

cost will be large if the number of tasks is greater than the number

of available processors. This is because, the tasks that complete

their job remain idle for a long time whilst waiting for the other

tasks to complete their job. The same situation occurs as many times

as the algorithm iterates to achieve convergence. Clearly, this

generates a significant amount of overheads that degrade the

performance of the algorithm.

On the other hand, in the asynchronous strategy, the number of

subtasks generated is equal to the number of cooperating processors

where each task is iterated to evaluate its points without any

synchronisation. Therefore, the asynchronous strategy which has no

synchronisation delays is better than the synchronous version.

Page 242: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

In Figure 5.4, the timing results of the two strategies are

illustrated.

-5 k=0.OOO2, t=0.OO18, A=0.06, £=10

Matrix size No. of processors

7x7 1

2

3

4

5

-5 k=0.OO04, t=0.OO36, A=0.04, £=10

Matrix size No. of Processors

Time Speed-up r

(second)

1.5 23.118 1

13.061 1.170

10.141 2.279

8.036 2.876

7.447 3.104

Time Speed-up r

(second) ----

9x9 1

2

3

4

5

-5 k=o.ool, t=O.l, A=0.OO5, £=10

Matrix size No. of processors

13x13 1

2

3

4

5

1.5

r

1.5

26.684 1

14.845 1.797

9.622 2.773

8.426 3.166

I 7.420 3.596

I Time Speed-up

(second) I I

I ! 29.261 1

16.634 I 1.759 I I ,

I 14.598 I 2.004

I ! 10.81'1 I 2.705 I I

I 8.822 3.316

TABLE 5.4: The Results From Implementing The First Strategy

--

I , ,

217

I

Page 243: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

-5 k=0.OOO2, t=0.OO18, A=0.06, £=10

Matrix size No. of processors

7 x 7 1

2

3

4

5

-5 k=0.OOO4, t=0.OO36, A=0.04, £=10

Matrix size No. of processors

9x9 1

2

3

4

5

-5 k=O.OOl, t=O.l, A=0.005, £=10

Matrix size No. of processors

13x13 1

2

3

4

5

r

1.5

r

1.5

r

1.5

Time Speed-up

(second)

22.908 1

12.876 1.779

9.818 2.333

7.742 2.958

6.085 3.764

Time Speed-up

(second)

24.843 1

13 .377 1.857

9.420 2.637

8.095 3.068

6.309 3.937

Time Speed-up'

(second)

27.905 1

14.134 1.974

13.004 2.145

9.556 2.920

8.099 3.445

TABLE 5.5: The Results From Implementing The Third Strategy

218

Page 244: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

219

4

3

-G- First Strategy ... Third Strategy n .c:: 't

o~~~~==~~~~~==~~~==7 o 2 3 4 5 6

No, of proe.,

FIGURE 5,4: The Timing Results for the Two-Dimensional Heat Equation,

Page 245: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

220

5.4 EXPERIMENTAL RESULTS FOR THE SECOND ORDER WAVE EQUATION

Consider the one-dimensional wave equation,

a2u 2 ' O~x~l, O~t<T ,

ax (5.4.1)

subject to the initial conditions,

U(x,O) = ~in(nx) , (5.4.2a)

aU at (x ,0) = 0 , (5.4.2b)

and the boundary conditions,

U(O,t) U(l,t) = 0 • (5.4.2c)

The exact solution is given by,

1 . U(x,t) = as~nnx COSnX . (5.4.3)

From the expectation of achieving stability advantages, a general

implicit finite difference discretisation to (5.4.1) at the· j+l,j

and j-l time levels (Table 3.1) is,

2 2 2 -aA u, 1 ' 1+(l+2aA )u, , l-aA u, 1 ' 1

1.- ,J+ 1.,)+ 1.+ ,)+ 2

- (1-2a)A u, ,+ ~-l,J

2 2 2 2 2(1-(1-2a)A )u, ,+(1-2alA u, 1 ,+aA u, 1 ' 1-(1+2aA )u

1.,J 1.+ ,) 1.- ,)- i,j-l

2 , 1 2 +OA u. 1 ' 1 ' 1.= , I a 0 • , n , 1+ ,J- (5.4.4)

with a=l for unconditional stability, A=i and truncation error of

order 0(h2

,k2

) with its computational molecule given by Figure 5.5.

Equation (5.4.4) gives a tridiagonal system of equations at

the (j+l)th time level, which can be displayed in matrix form as,

Page 246: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

221

i-l,j+l

i-l,j i,j i+l,j

i-l,j-l i,j-l. i+l,j-l

FIGURE 5.5

Id c l r' I Fl l la

I d c 0 u

2 !b2 I I .... .... " "

.... "-

I: I " .... .... ....

I : " " .... " " .... , I I (S.4.Sa) "

, "

.... , I

I ! " " " .... " I I " .... , 0

, , " " I a d cl lUn- 1 /bn_ll I

L a dJ U J 'b J n l!n

or Au = b , (S.4.5b)

where, 2 2 a = c = -aA and d 1+aA.

The vector ~ is defined by,

22 2 2 b

l = [2(l-(l-2a)A )u

l .+(l-2a)A u

2 .1+[-(1+2aA )u

1 '_1+aA u

2 '_11

IJ ,] ,J,J

2 2 +aA [uo . l+u

O '_ll+(l-2a)A Uo .

,J+,J ,J

Page 247: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

and

b i

222

2 2 2 2 (1-2a)A u. 1 .+2(1-a)1 )u .. +(1-2a)A u. 1 .+aA u. 1 .

~- ,J 1.,) 1.+,J 1- ,)-1

2 2 (1+2aA )u .. l+aA u. 1 . l' i=2,3, ... ,n-l,

1,)- 1+ ,J-

b n

2 2 2 2 = [(1-2a)A u 1 .+2(1-(1-2a)A)u .1+[aA u 1 . 1-(1+2aA)u . 11

n- ,J n t ) n- ,)- n,J-

2 2 +al [u 1 . l+u 1 . 11+(1-2a)A u 1 . n+ ,J+ n+ ,J- n+ ,)

The u values on the first time level are given by the initial

condition (S.4.2a). Values on the second time level are obtained

by applying the forward finite difference approximation to equation

(S.4.2b) at t=O,

or,

u -u au(x 0)" i,l i,O at i' k

= 0

u. 1 = u. ,which is a first order approximation. 1., 1.,0

, ~olution on the third and subsequent time levels are

generated iteratively by applying the AGE algorithm along the lines

(k+!) and (k+l).

When we implemented the AGE algorithm on equation (S.4.5b), we

arrived at the same form of equations along the lines (k+!) and (k+l)

as those that we have derived in Section (5.2). These explicit

equations are given by (5.2.11a,b).

By implementing this algorithm in parallel using the first and

third strategies, we obtain similar results to those obtained

previously in Section 5.2. These can be predicted because the

computation load on each of the processors is roughly the same.

Figure 5.6 illustrates the results obtained from implementing these

two strategies.

Page 248: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

5

4

3

1

o 1 2

No. of Proes.

3 4

.... First Strategy

... Third Strategy h:13

5

FIGURE 5.6: The Timing Results for the Second Order Wave Equation.

223

6

Page 249: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

224

5.5 THE NUMERICAL SOLUTION OF ONE-DIMENSIONAL PARABOLIC EQUATIONS BY

THE AGE METHOD WITH THE D'YAKONOV SPLITTING

In this section, the AGE iterative method for solving the one-

dimensional parabolic problem is introduced using a D'Yakonov splitting

strategy. It is known that the diffusion equation when solved by

the AGE method yields an algorithm with reduced computational

complexity and greater accuracy for multi-dimensional problems.

Consider the heat conduction problem,

au at

with the boundary conditions,

U(O.t) = gl

U(~.t)

and the initial condition,

U(x.O) = f(x)

I t~O , (5.5.1)

(5.5.2)

(5.5.3)

Let us assume that the rectangular solution domain R is covered

by a rectangular grid with grid spacing h.k in the x and t directions

respectively. The grid point (x.t) are given by.

x. = ~

ih fori=1,2, ... ,n

and t. = jk J

for j=O.1.2 ••..

A crank-Nicolson implicit finite difference approximation to

(5.5.1) at the jth and (j+l)th time level with principal truncation

2 2 error O(h .k ). (Table 3.1). can be written as.

-Au. 1 . l+(2+2A)u. . l-Au. 1 . 1 = All. 1 .+(2-2A)u .. +Au. 1 .• 1.- ,J+ 1.,)+ 1.+ ,J+ 1.- ,J 1.,) 1.+,]

h i=1,2, ... ,n; j=O,l,2, ... ; and A = k2

For the totality of points i=l.2.3 •...• n. equation (5.5.4) can

be written in matrix form as,

(5.5.4)

Page 250: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

i. e. I

where,

and

c l d c ,

L

Au ; b

I U n-l

u J n

;

fb l I 1 I b 2 I

I

c ; a ; -A, d;2A+2 and the column vector b is,

b, ~

AU ,+(2-2A)ul

,+AU2

,+Agl O,J ,),J

AU, 1 ,+(2-2A)U, ,+AU, 1 ' 1- ,J 1,J 1.+,J

b ; AU 1 ,+(2-2A)U ,+AU 1 ,+Ag2 n n- I] n, J n+,)

5.5.1 Formulation of the AGE Method

By splitting A matrix into components Gl

and Gi such that,

A ; Gl

+ G2

'

where Gl

and G2

are as defined in (5.2.8a) and (5.2.8b),

respectively with hd;(2+2A)/2.

225

(5.5.5)

(5.5.6)

(5.5.7)

By using equation (5.5.7) the matrix equation (5.5.6) can be

written in the form,

(5.5.8)

Another formulation similar to the Peaceman-Rachford splitting and

which has the same accuracy can be derived by using the D'Yakonov

splitting formula. This can be written explicitly as,

(k+l)* -1 (k) U ; (Gl+rI) [(Gl-rI) (G2-rI)~ +!?] (5.5.ga)

(k+l) -1 (k+l) * U ; (G

2 +rI) [~ ] (5.5.9b)

Page 251: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

226

(k+l)* where u is an intermediate value and r is L~e iteration

parameter.

Since (Gl+rI) and (G2+rI) are easily invertible, then from

(k+l) * d (k+l) equa tion (5. 5 . 9) u an u are given by,

I~l l (k+l) *

fde~/~ ~ I I I

1- ____ 1 __ _

u2

u' n-l

and

1

I

1

I

1 det

1 det

*

IW -c I

!:"~ _w -L _ I

r -c :

-a W I

1- - -...,---' "-

1

, , "-"- , ,

"- ,

.--

*1 ...... "- ',I

"- ' ,',1

1-.I- - - -- 1;

1 I

0 I

1- I-a - 1- - - - I'

L 2

where v=hd-r, w=r+hd and det=w -ac.

1

10 I 1 : - --I

0

'w

, -a ,

I-

1 r

-C

1

j

I

I - -I

1

-c

WI -!---,

pet/w J

f2Ul-V~U2 +bl 2 1k)

\vaul+v u2-vcu3+c u

4+b

2 , 2 2

* la u l -vau2+v u 3-vcu4 +b3

; 2 I I-vau 2+v u l-vCU +b 1 i 2 n- n- 2 n n- I 1 a u -vcu +v u +b J L n-2 n-l n n

(5.5.lOa)

fl l (k+l) *

IU 2

I 1 I I 1

* 1 (5.5.l0b)

I u'

1 n-l r 1 ' n J

By carrying out the multiplication 'in equation (5.S.l0a,b)

we have,

(i) At level (k+l) *,

(k+l)* u

l

(k + 1) * u,

1. = (A1U, 1+B1U,+Clu. 1+D1U. 2+E.)/det

1.- 1. 1.+ 1.+ 1.

Page 252: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

where,

(k+l)* u i +l

Al = 2 -(vwa+a c)

2 Bl = (v w+vac)

2 Cl = -(vwc+v c)

A2

B2

C2

(va 2 2

= +wa )

2 = -(v a+vwa)

2 = (vac+v w) ,

\ 0 i=n Dl = loa: i=:-l

(wc +vc ) otherwise i 2 -(ac +vwc) otherwise

\

E, = (wb,-cb, 1) ~ ~ 1.+

El.' = (wb,-ab, 1) , l. l. +

with the following computational molecules (Figure, 5.7).

1 (k+l)*

(k)

1 2

1 (k+l)*

(k)

i-l i i+l i+2 i-l i i+l i+2

FIGURE 5.7: The AGE Method At Level (k+l)*.

227

Page 253: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

(ii) At level (k+l) ,

(k+l) u.

1 (wu.-cu. l)/det

1 1+

(k+l) = (-au.+wu. l)/det Ui +1 ~ 1+

(k+l) u = u /w ,

n n

228

i:;l,3, ... ,n-l, 2.

with computational molecules given by (Figure 5.8),

(k-t 1)

8--8 (k+l)*

i i+l i i+l

1 (k+l)

l/w (k+l) *

n

FIGURE 5.8: The AGE Method At Level (k+l).

The computational complexity can be easily derived, which is 6

multiplications and 6 additions per point per AGE iteration plus

. th . . h 2 2 11 some pre-computat1on of e quant1t1es sue as c , w , etc. Norma y,

because the previous solution is a good approximate solution then only

two iterations are usually required.

Page 254: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

5.5.2 Numerical Results

A number of numerical experiments were conducted on a model

problem to demonstrate the application of the AGE algorithm with the

D'Yakonov splitting strategy on parabolic problems.

We considered the following problem,

aU at

subject to the initial condition,

U(x,O) = sinx, O~X~TI

and the boundary conditions,

U(O,t) = 0

U(n,t) 0, t~O

The exact solution is given by,

-t U(x,t) = e sinx

Tables 5.6, 5.7 and 5.8 present the numerical solution, absolute and

relative errors and the exact solution of this problem at appropriate

grid points for values of nt=5,11 and 21. The results confirm the

accuracy given by the D'Yakonov splitting is approximately equivalent

to the Peaceman-Rachford and better than Douglas-Rachford splitting

as expected.

By implementing this algorithm in parallel using the three

strategies discussed earlier, we found that the speed-ups are

slightly better than those found in Section 5.2. Figure 5.9 shows

the running times achieved by applying these three strategies.

229

Page 255: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

k-O 005 h-O In r-O 5 - . - . - . -6 E-l0 -

~ O.in

Numerical 0.302948 solution

Absolute -5 D 5.0xl0 Error

Relative 1.6xl0 -4

Error

Numerical 0.302963 Solution

DR Absolute Error

6.5xlO -5

Relative 2.1xl0 -4 Error

Numerical 0.302948 Solution

PR Absolute Error

5.0xl0 -5

Relative Error

1. 6xlO -4

Exact Solution 0.302898

TABLE 5.6

0.2n

0.576241

9.4xl0 -5

1.6x1O -4

0.576269

1.2xl0 -4

2.1xl0 -4

0.576241

9.5xl0 -4

1.6xl0 -4

0.576146

0.3n 0.4n O.sn 0.6n

0.793127 0.932377 0.980359 0.9328"0

1. 3xl0 -4

1.5xl0 -4

1.6xl0 -4

1.5x1O -4

. 1.6xl0 -4 1.6xlO

-4 1.6xl0

-4 1.6x1O

-4

0.793166 0.932423 0.980407 0.932423

1.6xl0 -4

1.9xl0 -4

2.0xl0 -4

1.9xlO -4

2.1xl0 -4 2.1x1O

-4 2.1xl0

-4 2.1xl0 -4

0.793127 0.932377 0.98036 0.932377

1.3xl0 -4

1.5xl0 -4 1.6xl0

-4 1.5xl0

-4

1. 6xl0 -4

1.6xl0 -4

1.6xl0 -4

1.6xl0 -4

0.792997 0.932224 0.980199 0.932249

LEGEND D: D'Yakonov splitting PR: Peaceman-Rachford splitting DR: Douglas-Rachford splitting

0.7n 0.8n

O. "193127 0.576241

1.3xl0 -4

9.4xl0 -5

1.6xlO -4

1.6xl0 -4

0.793166 0.576269

1.6xl0 -4

1.2xl0 -4

2.1xl0 -4 2.1xl0 -4

0.793128 0.576241

1.3xl0 -4

9.4xl0 -5

" -4 1. 6xl0

" -4 1.6xl0

0.792997 0.576146

0.9n

0.302948

5.0xl0 -5

-4 1.6xl0

0.302963

6.4xl0 -4

2.1xl0 -4

0.302948

s.oxl0 -5

1.6xl0 -4

0.302890

No.of Iters.

2

4

2

" w o

Page 256: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Time level nt=13, -6

k=O.OOs, h=O.ln, r=O.s, E=lO

~ O.ln

Numerical 0.291164 solution

D Absolute Error

1.4xlO -4

Relative Error

4.2x l0-4

Numerical 0.291207 solution

Absolute -4 DR Error 1.8xlO

Relative -4 Error

6.3xlO

Numerical Solution 0.291165

PR Absolute Error

1. 4xlO- 5

Relative 4.9 xlO- 5 Error

Exact Solution 0.291021

0.2n 0.3n

0.553838 0.762278

2.7 xlO-4 3.7X10-4

4.2 x l0-4 4.2xlO-4

0.553909 0.762390

3.5xlO -4 4.8 xlO-4

6.3 X10-4 6.3xlO-4

0.553828 0.762279

-5 2.7 xlO . 3.7 X10-5

4.9 XlO-s 4.9 XlO-5

0.553555 0.761903

No.of 0.4n O.sn 0.6n 0.7n 0.8n 0.9n Iters.

0.896112 0.942228 0.896112 0.762278 0.553828 0.291164

4.4xlO-4 4.6xlO-4 4.4xlO-4 3.7xlO -4 2.7 X10-4 1.4xlO -4 2

4.2xlO-4 4.2 Xl0-4 4.2 Xl0-4 4.2 X10-4 4.2 X10-4 4.2 X10-4

0.896244 0.942366 0.896244 0.762391 0.553909 0.211207

5.7 xlO -4

6.0xlO -4 s.7XlO-4

4.8xlO -4

3.SxlO -4 1.8xlO -4 4

6.3xlO-4 6.3 XlO-4 6.3 xlO-4 6.3 xlO-4 6.3 XlO-4 6.3 xlO-4

0.896113 0.94228 0.896113 0.762279 0.553828 0.291165

4.4 X10-5 4.6xlO-5 4 .4 xlO-5 3.7xlO-5 2.7 XlO-s 1.4x l0 -5 2

4.9 X10-s 4.9XlO-s 4.9 XlO- 5 4.9 XlO-s 4.9 xlO-s 4.9 XlO-s

0.895671 0.941765 0.895671 0.761904 0.553444 0.291022

TABLE 5.7

Page 257: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Time level nt=21, -6

k=0.OO5, h=O.ln, r=0.5, £=10

~ O.ln 0.2n

Numerical 0.279839 0.532286

Solution

Absolute -4 -4 D 2.2xlO 4.3xlO Error

Relative -4 -4 Error 8.2xlO 8.2xlO

Numerical Solution 0.279908 0.532417

Absolute -4 -4 DR 2.9xlO 5.6xlO Error

Relative -3 -3 Error 1.OxlO 1.OxlO

Numerical solution 0.279840 0.532287

Absolute -4 4.3xlO -4 PR 2.2x1.0 Error

Relative -4 -4 Error

8.2xlO 8.2x10

Exact Solution 0.285446 0.542951

0.3n 0.4n 0.5n

--

0.732629 0.861257 0.905576

6.0xlO -4

7.0xlO -4

7.4xlO -4

8.2xlO -4

8.2xlO -4

8.2xlO -4

0.732809 0.861468 0.905801

7.8xlO -4

9.1xlO -4

9.6xlO -4

1.0x10 -3

l.OxlO -3

1.0x10 -3

0.732630 0.861258 0.905580

6.0xlO -4

7.0xlO -4

7.4xlO -4

8.2xlO -4

8.2x10 -4

8.2xlO -4

0.747307 0.878513 0.923723

TABLE 5.8

0.6n 0 .. 7n

0.861257 0.732629

7.0xlO -4

6.0xlO -4

8.2xlO -4

8.2xlO -4

0.861468 0.732809

9.1xlO -4

7.8xlO -4

1.OxlO -3

l.OxlO -3

0.861258 0.732630

7.0xlO -4 6.0xlQ

-4

-4 -4 8.2xlO 8.2xlO

0.878513 0.747308

0.8n

0.532286

4.3xlO -4

8.2xlO -4

0.532417

5.6xlO -4

l.OxIO -3

0.532287

4.3xlO -4

8.2x10 -4

0.542951

0.9n

0.279839

2.3xlO -4

8.2xlO -4

0.279908

2.9xlO -4

l.OxlO -3

0.279840

'-4 2.2xlO

8.2xlO -4

0.285446

No.of Iters.

2

4

2

'" w

'"

Page 258: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

4

3

1

o 1 2

No. of procB

3 4

+ First Strategy ... Second Strategy ... Third Strategy n .. \3

5 6

FIGURE 5.9: The Timing Results for the One-Dimensional Heat Equation with D'Yakonov.

233

Page 259: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

234

5.6 THE NUMERICAL SOLUTION OF TWO-DIMENSIONAL PARABOLIC EQUATIONS BY

THE AGE METHOD WITH D'YAKONOV SPLITTING

The AGE method with D'Yakonov splitting for solving a one-

dimensional parabolic differential equation described in Section 5.5,

can be readily extended to problems involving higher space dimensions.

Consider the two-dimensional heat equation,

au at , O~x,y~N and t~o ,

with the initial conditions,

U(x,y,O) = F(x,y,t) ,

and the boundary conditions are,

U(x,O,t)

and

(5.6.1)

(5.6.2)

(5.6.3)

At the point p (x. ,y . ,t ) in the solution domain R, the value of 1 J r '

U(x,y,t) is denoted by u, , ,where x,=ih, y.=jh, for O~i,j~n+l ~,J,r ~ J

and h N (n+l)

The increment in time t,k is chosen such that t =rk, r

for r=O,1,2, •... A weighted finite difference approximation to

(5.6.1) (see Table 3.1) at the point (i,j,r+i) is given by,

-leu. l' 1+(1+4le)u.. l-leu. l' l-leu .. 1 l-lBu .. 1 '1 ~- ,],r+ 1,J,r+ 1+ ,J,r+ 1,)- ,r+ 1,)+ ,r+

_ l(l-B)u. . +(1-4l(1-B»u .. +l(l-S)u. . +A(l-B)u .. J.-l,),r . 1,],r 1+1,J,r l.,)-l,r

k +A(l.,.B)u .. 1 ,for i,j=1,2, ... ,n, r=O,1,2, .•. , A = -2 and O~e~l.

1,J+ ,r h

This approximation can be displayed in a more compact matrix form as,

(k+l) (k) Au ;; B~ + E ' (5.6.4)

where the vector b consists of the boundary values with,

Page 260: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

235

bl

A(l-S)[u +u ]+A8[u +u ],,,(1-8)u + O,l,r 1,O,r O,l,r+l l,O,r+l 2,O,r

A8u2 1.··· .,,(1-8)u 1 +,,8u '1.,,(1-8) [u + ,O,r+ n- ,O/r n-l,O,rT . n,O,r

u ]+,,8[u +u ] • n+l,O,r n/O,r+l n+l,l,r+l

bJ. ; ,,(1-8)Uo . +,,8Uo ' 1.0 •...• 0."(1-8)u l' +A8u l' 1

,J,r ,J,r+ n+ IJ,r n+ ,],r+

for j=2,3, ... ,n-l,

and

b ; 1.(1-8) [u +u ]+1.8 [u +u ].A(1-8)u n O,n,r l,n+l,r O,n,r+l l,n+l,r+l 2,n+l,r

+A8u •...• A(1-8)u +,,8u • 2,n+l,r+l n-l,n+l,r n-l,n+l,r+l

1.(1-8) [u +u ] +1.8 [u +u ] n,n+l,r n+l,n,r n,n+l,r+l n+l,n,r+l

The coefficient matrices A and B in equation (S.6.4) take the same

block tridiagonal form as in (S.3.Sa) and (S.3.Sb) respectively.

If we choose 8;! and split A into the sum of constituent

symmetric and positive definite matrices Gl

.G2

.G3

and G4

• such that.

(S.6.S)

where Gl .G2 .G3

and G4

are the same as in (S.3.7a). (S.3.7b). (S.3.7c)

and (S.3.7d) respectively.

Analogous to Section S.S. the D'Yakonov splitting for equation

(S.6.4) then takes the form.

(G +rI)u(k+l/4) ; 1 -.

(k) [(G -rI) (G

2-rI) (G

3-rI)(G -rI)]u +b

1 4 - -

(G +rI)u (k+!) ; 2 -

(G +rI)u(k+3/4) 3 -

(G4

+rI).!:: (k+l)

(k+l/4) u

(k+! ) ; U

(k+3/4) u

(a) 1 (b)

(S.6.6)

(c) j (d)

Page 261: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

236

Let us consider the above iterative formulae at each of the

four intermediate levels;

(i) At the first intermediate level (the (k+l/4)th step)

If we reorder the mesh points column-wise parallel to the y-axis,

~ '" we find that G3~=Gl~(c) and G4u=G2 ~(c)' where the suffix c stands

for column-wise ordering. Therefore, equation (5.6.6a) can be written

as,

r l (k+l/4) IUll1

= 1

det *

fd~t/:~- - ~

IW -c I 0 I-a w

~---I' - 1,,- I -I

"~,', I I 1 " --r - -1- --I

, 0 , W -c

l o

I _____ ...L :-a ~I,' j---- -- ------

o

L

I'" I , " I ,,,I , -1--1--'--1-'--­

ldet/W 1

~ --I­

I Iw

'-a -,

1-

T -1- -

-c I ,0 W, ~-I

I, .;~, I

I " ....... ',

o '_ 1

,w

I

'-a

-c

WI

*

Page 262: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

r 2

w

wa

L

o

wc 2

w wc

o

2 c

" ' , 2'

, ,

a wa

o

c

Iwa I 2 a wa

o

wc 2

w

2 c

wc

l

- - --I I

o I , , , ,

o

, I 2,

w __ L I,.... I

I" " ""'-," I

I _, +-Iw

2

I wa

I 2 la I

o

,

wc 2

w

"

wc 2

o

2 c

wa w wc '- " '-. '

"-"­,

, " "­

" " "- 2' ,

"-a wa

o ,

*

"-2

w

I U nl

I I

237

+

(5.6.7a)

B . t th 1 eh th 1 of u(k+l/4) y carry1ng ou e necessary a 9 ra, e va ue

can be derived.

(ii) At the second intermediate level (the (k+!)th step)

From equation (5.6.6b) we have,

Page 263: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

(kd) U (G )

-1 (k+1/4) = 2+rI ~

which yields,

I \ '

I"

k+t) -c f 10

-a W j

1 J det "I

- T ..... ==---,- - - - T-I" : ," ! ,,-,-

- - 1- ~ 'TW -c!

o '-

1- _______ -

o

, I

238

o

, ---------

-c -,

1

1

IUln

-,-_oil : o 1 : ,

-a 1-

w -~,- "

... ,," I t" " '

1

, , 1

I i" '. - - - - - 1- -/ c "w" -c,

'-a w' /

:- - t:t~J " , J

U 2 n

fl."nd u(k+t) by " th" t" 1 "1" " Hence, we can carry1ng out 1S ma r1X-vector mu t1P 1cat1on.

(iii) At the third intermediate level (the (k+3/4)th step)

By reordering the mesh points column-wise parallel to the ""'axis,

equation (5.6.6c) can be written as,

(k+3/4) u -(c)

and hence, we have,

(G + I) -1 (k+t) 1 r ~(c)

Page 264: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Iul jk+3/4) ~e~/:: ___ : ___ : __ _ u ,I Iw -e-, I 0 ~l'l I

I-a w I I - f- - -, - - r

I, I I I~~, I

t ",,' , " 'I +---'--',---" I I 0 ,w -e I

: I _1_* U nl ( det

-- -'-j I-a w I

- -.J - - T ,,­, , I ,,'

, 'I , I

___ L

I

, " '''-

o

- - - - - -, -- --det/w , -,

,

w -e: : 0 I I

o 1- _ =-a _ w--; ,,- : _ _ __ I I - ~\~- --- I

::a -; t~2J 0,

L and by carrying out the necessary algebra, we find the values of

(k+3/4) u

(iv) At the fourth intermediate level (the-(k+l)th step)

In a similar manner to the third step, equation (S.6.6d) can be

expressed as,

which yields,

(k+l) u -(c)

( )-1 (k+3/4)

= G2+rI ~(e)

239

Page 265: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

240

(k+l)

:0 , I -,

o r l (k+3/4) u

ll

u2l

IU3l

_1_* -unl det 1

\-

I

I : j LU

2 n L

-C, I -a w

_1_- ' 1

---,--

- - --

0

, -c

I

w l

o

By carrying out the necessary algebra, we can find the values of

(k+l) u .

It can be readily shown that the method formulated above is

second order accurate AGE algorithm in k. Hence, the D'Yakonov split

gives a second order AGE method. The computational work involved at

each stage is the solution of 2x2 block systems, or in explicit form

as simple recurrence relations. The iterative procedure is continued

until convergence- to a specific level of accuracy £. is achieved.

5.6.1 Numerical Results

A number of numerical experiments were conducted on the model

problem, i.e. the diffusion equation to demonstrate the application

of the AGE algorithm with the D' Yakonov splitting strategy on a two

dimensional parabolic problem.

Page 266: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

241

Consider the following problem,

" ,2 "2 oU ~ + ~ +

2 2 H(x,y,t) (5.6.8)

at ax ay H(x,y,t) ; sinxsiny exp(-t)-4, defined in the region O~x,y~l, t<O.

We choose the exact solution to be,

-t 2 2 U(x,y,t) = sinx siny e + x +y , O~x,y~l, t~o.

The initial and boundary conditions are defined so as to agree with

the exact solution.

The numerical results for different values of x and y were

compared with the corresponding results obtained in different formulae.

It is generally observed from Tables 5.9, 5.10 and 5.11 that the AGE

algorithm using D'Yakonov splitting produces more accurate results

with less number of iterations.

Page 267: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

-4 x=O.l, k=O.OOl, h=O.l, t=l.O, A=O.l, r=1.5, £=10

~ 0.1 0.2 0.3 0.4

AGE-D l.oxlO -5

l.oxlO -5

2.1xlO -5

3.2xlO -5 --

AGE-CN 1.75xlO -5

3.4xlO -5

4.98xlO -5

6.29xlO -5

AGE-IMP 2.0xlO -5

3.9xlO -5

5.7xl0 -S

7.3xlO -5

--- -------- --------Exact

0.029018 Solution 0.067946 0.126695 0.205177

0.5 0.6

3.7xlO -5 4.0xlO

-5 .-

7.24xlO -5

7.69xlO -5

8.4xlO -S

9.0xlO -S

0.303308 0.421006

TABLE 5.9: The Absolute Errors of the Numerical Solution of Problem (S.6.8)

x=0.5, k=O.OOl, h=O.l, t=l.O, A=O.l, r=l.S, £=10 -4

I~ 0.1 0.2 0.3 0.4 O.S 0.6

AGE-D 1.7xlO -5 2.3xlO-S 3.3xlO

-S 3.6xlO

-6 S .0xlO

-S S.lxlO

-S

AGE-CN -5 -4 -4 -4 -4 -4 ".24xlO 1.42xlO 2.07xlO 2.62xlO 3.04xlO 3.25xlO

AGE-IMP 8.41xlO -S l.66xlO -4 2.41xlO -4

3.07xlO -4 3.58xlO -4 3.87xlO

-4

--Exact Solution 0.303308 0.376183 0.468197 0.S78931 0.707976 0.854943

TABLE S.lO: The Absolute Errors of the Numerical Solution of problem (5.6.8)

.. ~.-.. -

0.7 0.8 0.9 No.of Iters. -

4.1xlO -5

4.0xlO -5

3.1xlO -5 3 ----

7.42xlO -5

6.21xlO -5

3.78xlO -5

3 -- ------- -------- --

8.9xlO -5

7.6xlO -S

4.8xlO -S

4

0.SS8194 0.714801 0.890760

0.7 0.8 0.9 No.of iters.

4.9xlO -S

5.0xlO -S 4.3xlO-5

3 ---

3.16xlO -4

2.68xlO -4

1.6SxlO -4

3 - ,-- -- -

3.38xlO -4 3.34xlO -4 2.1SxlO -4

4 --- -_._--

0.019463 1.201191 0.399808

Page 268: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

-4 x=0.9, k=O.OOl, h=O.l, t=l.O, A=O.l, r=1.5, £=10

~ 0.1 0.2 0.3 0.4

-5 -5 -5 -4 AGE-D 3.1xlO 4.9 XlO 8.0xlO 1.oxlO

3.78xlO -5

7.45xIO -5 -4 -4

AGE-eN 1.09xlO 1.4xlO

-5 -5 -4 -4 AGE-IMP 4.84xlO 9.59xlO 1. 4xlO 1.81xlO

Exact 0.890760 0.990813 1.109460 1.246013

Solution

0.5 0.6

1.2xlO -4

1.3XIO -4

1.65xlO -4

1.81xlO -4

2.15xlO -4

2.4xlO -4

1.399809 1.5-/0209

TABLE 5.11: The Absolute Errors of the Numerical Solution of Problem (5.6.8)

0.7 0.8 0.9 No.of iters.

1.5xlO -4 1.4xlO

-4 1.oxlO

-5 3

1.83xlO -4

1. 63xlO -4

1.09xlO -4

3 ----,---_ .. - - - - .--~---- -,,-- - .

2.48xlO -4

2.29xlO -4

1.63xlO -4

4 -.----- _._--- -

1.756611 1. 958450 2.175209

Page 269: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

244

5.7 A NEW STRATEGY FOR THE NUMERICAL SOLUTION OF THE SCHRODINGER

EQUATION

The numerical solution of the one space dimensional Schrodinger's

equation which is a well known equation in Quantum Mechanics is

obtained by a new direct method which depends on separating the real

and imaginary parts of the discretized complex tridiagonal matrices

which are obtained from a Crank-Nicolson formulation of the

differential equation into a new decoupled form.

Given the equation,

(5.7.1)

with the initial condition,

U(x.O) - H(x) • t-O • (5.7.2)

and the boundary conditions,

U(O.t) - " (5.7.3)

A uniformly spaced network whose mesh points are x -rh. t.-jk. r J

R. for r-O.l.2 ••••• n+l and j-O.l.2 •••.• m+l is used with h-(n+l)

T k=-- and the

(m+l) mesh ratio A- k2 .

h 'A weighted approximation to the differential equation (5.7.1) at

the point (x .t. ,) is given by. (Table 3.1). r J+,

-A9u 1 . 1+(2A9-i)u . l-ASu 1 . 1 - A(1-9)u 1 .-(2A(1-9)+i)u . r- ,J+ r,J+ r+ ,J+ r- ,J r , )

+A(l-S)u 1 .• r-l.2 •...• n. j-O.l.2 •... and O~S~l. (5.7.4) . r+ I)

This approximation can be written in a more compact matrix form as,

Page 270: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

245

e -i -A6 l ~'l I b l I I u 2 I l -A6 2A6- i -A6 i b 2 " " " 0 ' I " " " I I

" " , I I

I I I

I " "-,

I : " "- "- , " " "- I "- "-

"- "- "- I I

" , " , ,

Ib;_~ L

0 -A6 2A6-i -A6

G:-J -A6 2A6-i ~n

(s.7.sa)

i. e. ,

Au = b (s.7.sb)

where, b

l = A(1-6) (uo ,+u

2 ,)+A6Uo ' 1-(2A(1-6)+i)u

l j

IJ I) ,J+ ,

b r

A(1-6)(u 1 ,+u l,)-(2A(1-e)+i)u ,. r=2.3 •...• n-l r- I) r+,J r,)

and b = A(1-6) (u 1 ,+u 1 ,)-(2A(1-e)+i)u ,+A6u 1 ' 1 .

n n- IJ n+,J n,) n+ ,J+

Here b is a column vector of order n consisting of the boundary

values as well as known u values at the time level j while u are the

values at the time level (j+l) which we seek. If we let 6=!. we

recall that (5.7.4) corresponds. to the well known Crank-Nicolson

2 2 method with accuracy of order O(h .k ).

5.7.1 Outline of the Method,

The complex tridiagonal system (s.7.5b) can be rewritten as.

(C+iD)~ = b (5.7.6)

where, d

a

C = (s.7.7a)

Page 271: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

246

I l I

c 0 I "-

" , D ; "

l "- , ,

0 , "-

"-" c

(5.7.7b)

a=AI d;->./2 and c;l.

Now let, u ; (v+iw) , (5.7.7c)

b ; (f+ig) (5.7.7d)

The above system (5.7.7) can further be rewritten in a real variable

form as,

F -01 rvl

ifl

cJ ;

19 J (5.7.8)

~ lwJ

which is obtained by separating out the real and imaginary parts of

the complex coefficient matrix, i.e.,

(C+iD) (v+iw) (f+ig) , (5.7.9)

to give, CV-Dw; f (5.7.lOa)

Dv+CW ; 9 (5.I.lOb)

The system (5.7.l0a,b) ·can further be reduced into the uncoupled

form by premultiplication of the matrix,

~" Dl I ~

to give,

2 2 01 fvl rCf+Dg l rc +D (5.7.11)

~ 2 j lwJ ~Df+C9J C +D

Page 272: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

247

which represents two linear systems, with the same coefficient matrix

for the unknown components v and w, which need to be solved only once,

but with different right hand sides.

Since C was originally of tridiagonal form, then the matrix c2

will be quindiagonal and of the form,

-2 2 2

l d +a 2ad a

d2+2a

2 2 2ad 2ad a I

I

2 2 2 2 a 2ad d +2a 2ad a , C " "

... " "- "- "-"- "- "-

"

I "- "- "-

" "- "- ... ... "- "-"- "- ... ... ... "- ... ... ... ...

c2 J ... ... ... ... "- ... "-

"- "- ... "-

... .(5.7.12) " ..... 2 2'..... ..... ... 2 "- 2.

a 2ad d +2a 2ad a ,

0 2 2ad d

2+2a

2 2adl

L a

d2

+a 2 J 2

a 2ad

whilst D 2

will remain in diagonal form.

Thus, the solution of the complex system has been reduced to

solving two identical quindiagonal systems in real variables of the

form,

2 2 2ad

2

l rl l ~ I' +a +1 a IWl

2 2 2 12ad . d +2a +l 2ad a

IV2 I w2

la2 2 2 2 0 I I I I 2ad d +2a +1 2ad a

I ... ... ... I I

"- "-...

"-"- ... ... "- "- ... ... ...

I "- ... ... "- ... ... ... ... I I "- "-...

"-... "- , .... "- "- I ... "- ... ... ... ... ... ... ... ... "-... 2 2 ... 2 ... ... 2

0 a 2ad d +2a +1 2ad a

2 2 2 2ad I a 2ad d +2a +1 v

n-l \Wn_ll

I 2 2 2 \ L'n J L

a 2ad d +a +1 ~ L'n J

Real Sol. Imag.Sol.

Page 273: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

248

[fl l gl l f2 g2

[ I I

I I ~ I (5.7.13)

I I I I I

f g~-l n-l

~n I [n ·1 J

RHSl RHs2

which can be solved directly by efficient elimination procedures

involving no inte,rchanges.

Since the coefficient matrix is' unchanged, then the Gaussian

elimination algorithm need only be applied once to the matrix whilst

the operations on the two right hand sides can be carried out

simultaneously.

By the use of the quindiagonal solver [Conte, 1965] the

computational complexity for the algorithm is now of order O(n)

multiplications, while the Gaussian elimination is of order o(n3

)

multiplications.

5.7.2 Numerical Results.

Consider the problem of solving the Schrodinger equation,

au a2u -i at ---2' O~x~n/2 , (5.7.14)

ax with the initial condition,

u = sinx + icosx ,

and the boundary conditions,

U(O,t) ~

U(n/2,t)

-t ie

-t ~ e

(5.7.15a)

(5.7.15b)

Page 274: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

249

The following tables illustrate the results obtained by solving

the problem (5.7.14) by this method.

Numerical solution Exact solution

Real Imag. Real Imag.

0.139508 0.970233 0.139497 0.970222

0.276164 0.940504 0.276154 0.940494

0.407198 0.891629 0.407189 0.891620

0.529943 0.824604

I 0.529935 0.824596

0.641904 0.740795 0.641894 0.740785

0.740794 0.641903 0.740785 0.641894

0.824604 0.529943 0.824596 0.529935

I 0.891631 0.407200 I 0.891620 0.407189 I

I 0.940504 0.276164 I 0.940494 0.276154

0.970231 0.139506 I

0.970222 I

0.139497 , , : I

TABLE 5.12: Solution At Time Level nt=5, t=0.OO2

Page 275: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

250

Numerical Solution Exact Solution

Real Imag. Real Imag.

0.131382 0.913729 0.131373 0.913720

0.260082 0.885734 0.260072 0.885724

0.383487 0.839707 0.383476 0.839696

0.499082 0.776583 0.499074 0.776575

0.604522 0.697645 0.604513 0.697645

0.697655 0.604523 0.697645 0.604513

0.776583 0.499082 0.776575 0.499074

0.839705 0.383485 0.839696 0.383476

0.885734 0.260082 0.885724 0.260072

0.913731 0.131383 I 0.913720 0.131373

TABLE 5.13: Solution At Time Level nt;17, t;0.08

Numerical Solution Exact Solution

Real Imag. Real Imag.

0.128728 0.895636 0.128772 0.895628

0.254931 0.868194 0.254922 0.868185

0.375894 0.823080 0.375883 0.823069

0.489203 0.761209 0.489192 0.761198

0.592553 0.683841 0.592543 0.683831

0.683838 0.592551 0.683830 0.592543

0.761207 0.489201 0.761198 0.489192

0.823079 0.375893 0.823069 0.375883

0.868204 0.254931 0.868185 0.254922

0.895636 0.128780 0.895628 0.128772

TABLE 5.14: Solution At Time Level nt;21, t;O.l

Page 276: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

251

5.8 CONCLUSIONS

The parallel implementations of the AGE iterative method to solve

one and two dimensional parabolic problems as well as the second order

hyperbolic problem are investigated in this Chapter. Using the three

parallel strategies described in Chapter 4, we notice that the

asynchronous version achieves a better speed-up ratio, where these

speed-ups are mostly linear.

This is because of the overheads generated in the other two

strategies and also because of the idle time that is spent waiting

for the other processors to complete their computations. There is

also a time that is needed to order the pOints in the second strategy

which also degrades the efficiency of the algorithm.

From the experiments we notice from the results obtained from

the first two strategies that there is no sufficient parallelism

available to enable the processors to work efficiently. This is

because of the granularity of the 2x2 blocks is too small to keep

the processors always busy.

The numerical experiments carried out on the solution of the

model problem of one and two dimensional parabolic differential

equations using the AGE method with D'Yakonov .splitting yields an

algorithm with reduced computational complexity and greater accuracy

for multi-dimensional problems.

Solving the (nxn) complex tridiagonal matrices by a new direct

b method'presented in section 5.7. This direct method enables us to

reduce the. complex system into two identical real quindiagonal

systems. Since the coefficient matrix is unchanged, then the Gaussian

Page 277: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

elimination algorithm need only be applied once to the matrix whilst

the operations on the two right hand sides can be carried out

simultaneously.

252

Page 278: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS
Page 279: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

CHAPTER 6

NUMERICAL INVERSION OF THE LAPLACE

TRANSFORMATIONS) SOME INVESTIGATIONS

AND PARALLEL EXPLORATIONS

You ean observe a ~ot

just by watohing.

Y. Berra.

Page 280: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

253

6.1 INTRODUCTION

The Laplace transform is an important technique which is used to

find the solution of differential equations with assigned boundary and

initial conditions in applied sciences and engineering. Already,

extensive tables of the Laplace transforms are available. However, it

is interesting to investigate and obtain numerically the inverse

Laplace transforms.

Another technique, that of integral transforms, which had its

origin in Heaviside's work, has been developed during the last few

years and has certain advantages over the classical methods.

The integral transform F(s) of a given function f(t) in the range

a~t~b is defined as follows,

b

F(s) ; J k(s,t)f(t)dt , a

(6.1.1)

where k(s,t) is a known function of sand t, known as the kernel of

the transform, provided that the integral exists.

"In the application of integral transforms to the solution of

differential equations, use has so far been made of five different

kernels. These five transforms are:

(a) The Laplace transform,

F (s) ; ( -st

f(t)e dt,

Cb) The Fourier sine and cosine transform,

F(s) f(t)sin(st) dt cos(st) ,

(c) The Complex Fourier transform,

F (s) ;

(6.1.2)

(6.1.3)

(6.1.4)

Page 281: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

254

(d) The Hankel transform,

r~

F(s) ; J f (t) t o

J (st)dt n

(6.1.5)

where J (t) is the Bessel function of the first kind of order n. n

(e) The Mellin transform,

F (s) ; r~

J f (t) o

The effect of applying an integral transform to a partial

(6.1.6)

differential equation is to exclude temporarily a chosen independent

variable and to leave a partial differential equation in one less

variable for solution. The solution of this equation will be a function

of s and the remaining variables. When this solution has been

obtained, it has to be inverted to recover the Zost variable. Thus,

if t is the variable eliminated and F(s) is one of the transforms given

above, we first obtain auxiliary equations giving F in terms of s and

the remaining independent variables. These are solved for F and then

inverted to obtain f(t).

In this chapter, we shall briefly describe the Laplace transform

and its numerical inversion. A comparison between different methods

suggested for the numerical inversion of the Laplace transform is

given in order to choose an appropriate method. Then an attempt is

made to obtain more accurate results by applying extrapolation

techniques in a Romberg quadrature strategy is considered. Then we

implement the numerical inversion of the Laplace transform suggested

by Crump [Crump, 1976] on the Balance 8000 MIMD parallel system

incorporating some parallelism.

Page 282: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

255

6.2 THE NUMERICAL INVERSION OF THE LAPLACE TRANSFORM

The Laplace transform of a real function f:R + R, with f(t);O

for t<o and its inversion formula are defined as,

r~

F(s) ; L[f(t)] J

-st f(t)e dt, (6.2.1)

o

f(t) -1

L [F(s)] ~+~ st J F(s)e dt (6.2.2)

v-im

with s;v+iw;v,w E Rand i;l-l. v E R is arbitrary, but greater than

the real parts of all the singularities of F(s). The integrals in

(6.2.1) and (6.2.2) exist for Re(s»a E R if,

(a) f is locally integrable,

(b) there exists a to~O and k,a E R, such that If(t) I~keat

for all t>-to' (i.e. of exponential order),

(c) for all t E (O,~) there is a neighbourhood in which f

is of bounded variation.

In the following we always assume that f fulfills the above

conditions and in addition that there are no singularities of F(s) to

the right of the origin" (for the case of singularities to the right,

a suitable translation of the imaginary axis can be performed).

Although there exists extensive tables of transforms and their

inverses, it is highly desirable to have methods for approximate

numerical inversion for computer implementation. A large number of

different methods have been devised for the numerical evaluation of

the Laplace inversion integral. In this section we briefly survey some

of these methods, where many of them are either orthogonal series

Page 283: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

expansions or weighted sums of values of the transform at a set of

pOints.

Schmittroth [Schmittroth, 1960] has described a method in which

the inverse transform is obtained from the complex inversion integral

by use of numerical quadrature. This method gives good results, but

if the inverse transform is required for a large number of values of

the independent variable, the quadrature procedure must be repeated

for each value of the independent variable.

In cases where the inverse is required for many values of the

independent variable, it is convenient to obtain L~e inverse as a

series expansion in terms of a set of linearly independent functions.

Norden [Norden, 1955] has described two such methods in which the

expansion coefficients are calculated by solving a system of

simultaneous equations. The problem of solving simultaneous linear

equations can be reduced to one of solving a triangular system.

Salzer [Salzer, 1958] described a method, using orthogonal

polynomials (function) which was later refined by Shirtlifte and

Stephenson [Shirtlifte, 1961]. They attempt an approximate

evaluation of the inversion integral using Gaussian quadrature in the

complex plane. In this method it is necessary to find all roots, real

and complex, of a polynomial of high degree.

Lanczos and Papoulis [Lanczos, 1956] describe methods in which

the inverse transform is obtained as a series expansion in terms of

trigonometric functions, Legendre polynomials or Laguerre polynomials.

Papoulis obtained the inverse transform as a series expansion in

terms of Laguerre functions. The expansion coefficients are obtained

256

Page 284: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

257

from the coefficients of the Taylor series expansion of the Laplace

transform by solving a triangular system of linear equations. The

disadvantage of this method, is the necessity of obtaining the Taylor

series expansion of the Laplace transform. Lanczos found the inverse

transform as a series expansion in terms of Laguerre functions, by

applying a con formal mapping to the Laplace transform and then developing

the resulting function in a Taylor series expansion.

Weeks [Weeks, 1966] further refined the ideas of Lanczos and

Papoulis. He obtained the coefficients in the expansion of the inverse

transform directly by trigonometric interpolation applied to the Laplace

transform. The resulting algorithm is quite suitable for automatic

computation.

Longman [Longman, 1975] described a method in which the inverse

function F(s) is replaced by Pade approximants ~ (s) in the inversion n,m

integral. Unfortunately, the construction of the rational function

~ (s) requires a knowledge of the Taylor expansion of F(s) about the n,m

origin, and this makes it impossible to implement a procedure which

uses only values of F(s) .

Finally, Dubner and Abate [Dubner, 1968] showed that the inverse

integral may be approximated by a certain Fourier series. This method

will be described in detail in the next section.

Page 285: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

258

6.3 NUMERICAL EXPERIMENTS

As stated in the previous section, a number of numerical inversion

methods have been developed during the last few years. In this

section, we shall confine ourselves to the methods using Fourier

series approximations.

We shall describe the derivation of the Dubner and Abate method

(OUbner, 1968]. Then, the natural continuation of this method as

suggested by Durbin [Durbin, 1974] and Crump [Crump, 1976] is

described. A comparison between these methods is given by a number of

experimental results. Finally, we attempt to increase the accuracy

by using the Romberg integration method.

Let f(t) be a real function of t satisfying (6.2.3) conditions.

Then, by expanding equation (6.2.1) we have,

F(S) = (00 -vt

J e f(t)coswtdt - i

o

-vt e f(t)sinwtdt,

= Re{F(v+iw)} + iIm{F(v+iw)}.

Also, by expanding equation (6.2.2), we have,

f(t) =

or

f (t)

1 211i

vt e 211

+ i

+W

f evt(coswt+isinwt)(Re{F(s)}+iIm{F(s)}idw

_W

r+W l J-!Im{F(S)}Coswt+Re{F(S)}Sinwt)dWJ

(6.3.1a)

(6.3.1b)

(6.3.2a)

(6.3.2b)

The imaginary part in (6.3.2b) cancels out because of the parity

of Re{F(s)} and Im{F(s)}. Again by the same argument, we can have,

Page 286: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

259

f (t) (6.3.2c)

For t<O, f(t)=O, which means that,

rm J (Re{F(s)}coswt+lm{F(s)}sinwt)dw O. (6.3.2d)

o

Consequently, we obtain 3 formulas for the Laplace inverse of

f(t) corresponding to F(s). These 3 are:

(a) f (t)

(b) f (t)

(c) f (t) =

2 vt

- e 11

vt e

11

Re{F(s)}coswtdw ,

r'"

J Im{F(s)}sinwtdw,

o

r'" Jo(Re{F(S)}Coswt-Im{F(S)}sinwt)dW

(6.3.3a)

(6.3.3b)

(6.3.3c)

Fourier series were first used by Dubner-Abate, where they

suggested the following. Let h(t) be a real function of t with h(t)=O

for t<o. Consider sections of h(t) in intervals like (nT, (n+l)T) ,

construct an infinite set of 2T-periodic functions g (t) (Figure 6.1), n

where,

r h (nT-t) -T~t~O (a)

1 g (t) = i h (nT+t) O~t~T (b)

t (6.3.4) n

h ((n+2)T-t) T~t~2T , n:;:O, 2,4, ... , (c) l J

r h ( (n+l) T+t) -T~t~O

(a) 1

gn (t) i h ((n+l)T-t) O,t,T (b) ( (6.3.5) I

l h( (n-l)T+t) T~t~2T, n;;l, 3 15, ••• , (c) J

Page 287: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

o

-T o

-T o

T

T

T

-vt h(t)=e f(t)

2T

2T

2T

FIGURE 6.1

3T

3T

3T

Then by developing each g (t) into cosine Fourier series we have, n

where,

g (t) n

A n,k

L k=l

A cos knt/T , n,k

(n+l)T 2 r .

h(t)cosknt/T dt. T JnT

Since it is always possible to write,

260

(6.3.6)

(6.3.7)

Page 288: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

261

h (t) (6.3.8a)

or

we have,

L n=O

f (t)

A n,k

vt e h (t) , (6.3.8b)

-vt e f(t)cos knt!T dt (6.3.9a)

i Re{F(v+ikn!T)} (6.3.9b)

L n=O

vt e g (t)

n

2evt

= -T-[ lRe{F (v) }+ L Re{F(v+ikn!T}cosknt!T) (6.3.9c) k=l

By using the expressions (6.3.4b), (6.3.4c), (6.3.Sb), (6.3.Sc),

(6.3.8a) and (6.3.8b) we have,

L n=O

vt e g (t)

n

m

f (t) + \' -vkT 2vt L e (f(2kT+t)+e f(2kT-t»

k=l

f(t)+error l(V,t,T)

In conclusion, for any O~t~2T, we can write,

f(t)+errorl(v,t,T) 2 vt _e_[ !Re{F (v) }+ .T

l Re{F(v+ikn!T)}cosknt!T), k=l

(6.3.10)

Equation (6.3.10) is the Dubner-Abate formula, where the error is a

function of v,t/T. The factor L -2vkT 2vt

e f(2kT-t)e is the most k=l

disturbing one since it increases exponentially with t. Numerically

the Dubner-Abate method is only valid for t~T!2.

Similar numerical inversion techniques can be developed that

utilize Im{F(s)} rather than Re{F(s)}. Durbin [Durbin, 1973)

suggested the following,

Page 289: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

262

Consider h(t) in the interval (nT, (n+l)T) with an· infinite set of

odd 2T-periodic functions k (t). (Figure 6.2). n

h (t) =e-vtf (t)

T 2T 3T

-T ° T 2T 3T

-T ° T 2T 3T

FIGURE 6.2

By def ini t-ion we hav.e,

{ h (t) nT~t~(n+l)T

k ( t) n -h(2nT-t) (n-l)T:st:;:nT , n=O,1,2, ...

(a) } (b)

Similarly, on the intervals (-T,T), (O,T) and (T,2T), we have,

(6.3.11)

Page 290: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

-h (nT-t) -T::t::O (a) 1 k (t) = h (nT+t) O~t~T (b) I n

-h ( (n+2) T-t) T::t~2T n=O, 2,4, ..... (c) J

r h ( (n+l) T+t) -T::t~O (a)

1 k (t) =

1 -h ((n+l)T-t) O~t:;T (b)

n

h( (n-l)T+t) T~t~2T n=1,3,5, ...... (c) l J

Hence, the Fourier representation for each odd function k (t) is, n

k (t) n

I B sin krrt/T , n,k

where, we find that,

B n,k

(n+1)T r -vt

= J e f (t) sink rrt/T dt .

nT

263

(6.3.12)

(6.3.13)

(6.3.14)

(6.3.15)

Then, by summing (6.3.15) over n and comparing it with equation

(6.3 .1a), we have,

B n,k

-vt e f(t)sinkrrt/T dt

Again, summing (6.3.14) over n and multiplying both sides by

vt e , we can have an expression similar to equation (6.3.9c),

t 2 vt . L e

V k (t) = - --e--[Im{F(v+ikrr/T)}sinknt/T]

n T

00

n=O

(6.3.16a)

(6.3.16b)

(6.3.17)

Likewise, on the interval (O,2T), using the equations (6.3.8a),

(6.3.8b), (6.3.12c), (6.3.13b) and (6.3.13c), we find,

k (t) n

f (t) + L k=l

-2vkT 2vt e [f(2kT+t)-e f(2kT-t)]. (6.3.18)

Page 291: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

264

Hence, another representation for f(t) is,

f(t)+error2(v,t,T) 2evt

iW

] - -T- ~~O Im{F (V+ik1T/T) }sink1Tt/T] . (6.3.19)

The advantages of OUrbin method are twofold. First, the error bound

on the inverse f(t) becomes independent of t, instead of being

exponential in t. Second, the trigonometric series obtained for f(t)

in terms of F(s) is valid on the whole period 2T of the series.

Kenny S. Crump [Crump, 1976] utilized a combination of both real

and the imaginary parts of the Fourier series. His proposal is as

follows:

For n=O,l,2, ... I define 9 (t), -oo<t<co, by, n

-vt g (t) = e f(t) , 2nT~t~2(n+l)T ,

n

where g (t) is a periodic function with 2T period. The Fourier series n

representation of g (t) is then given by, n

g (t) = n

lA + L {A kcos(k1Tt/T)+B k Sin (k1Tt /T)}, n,D k=l n t n,

where the Fourier coefficients A and B are given by, n,k n,k

and

A n,k

B n,k

(2(n+l)T_vt = (liT) J e f(t) cos (k1Tt/T) dt

2nT (2 (n+l)T_vt

= (l/T)j e f(t)sin(k1Tt/T)dt 2nT

By summing equation (6.3.20) with respect to n, we have,

L n=O

g (t) n

(liT) [!F(v)+ L Re{F(v+ik1T/T)}cos(k1Tt/T) k=l

-Im{F(v+ik1T/T)}sin(k1Tt/T) ] ,

(6.3.20)

(6.3.21)

Page 292: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

265

which can be written in the form of the inverse Laplace transform as,

f(t)+error3(v,t,T) vt

= (e /T) [tF(v)+ I Re{F(v+ikrr/T)}cos(krrt/T) k=l

-Im{F(v+ikrr/T)}sin(krrt/T)}] (6.3.22)

where, 00 '"

error3(v,t,T) = evt I g (t)

n = e

vt I exp{-v(2nT+t)}f(2nT+t) n=l n=l

or

error3(v,t,T) I -2nvT e f( 2nT+t) (6.3.23) n=l

However, by comparing errorx, error2, and error3 we notice that

error3 = (errorl + error2)/2

To increase the rate of convergence of equation (6.3.22) and

thereby reduce the truncation error Crump suggested the use of either

the Euler transformation (first used by Simon, Stroot and Weiss

[Simon, 1972] or the epsilon algorithm [Wynn, 1962]. He also showed

the superiority of using the epsilon algorithm in speeding the rate of

convergence.

In 1984, Honig and Hirdes [Honig, 1984] attempted to accelerate

the convergence of equation (6.3.19) by using three acceleration methods.

The epsilon algorithm, the minimum-maximum method and a method based

on the curve fitting. Also, they tested other acceleration methods

such as the Euler transformation and Aitken's extrapolation procedure

but these methods turned out to be less efficient.

To speed up the computation and to increase accuracy Dubner-Abate

and Durbin introduced the use of the Fast Fourier Transform (FFT)

Page 293: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

266

techniques [Cooley and Tukey, 1965], (this technique is described in

the next section).

To fulfil our comparison between these methods, we list the results

obtained for solving the two problems,

(1) F (s) 2 = sits +1) f (t) t/2 sin t

2 = l/(s +s+l) f (t) = ~ exp(-t/2)sin(tI3!2).

13 ( 2) F (s)

The results of these problems are illustrated in Table 6.1 and 6.2.

The trapezoidal rule lacks the degree of accuracy, which is

generally required from a quadrature formula. However, Romberg

integration is a method that has wide applications because it uses

the trapezoidal rule to give preliminary approximations, and then

applies the Richardson extrapolation process to obtain improvements

to these approximations.

To improve the accuracy of the final results for a reduced number

of terms n, we applied the Romberg integration to the numerical

inversion of the.Laplace transformation methods that were suggested by

Dubner-Abate, Durbin and Crump.

Although the Romberg strategy appears to give more accurate

results for small time intervals, they are too small to be of

practical use in applying it to the full range of the transform.

Table 6.3 shows the numerical inversion of the Laplace transform with

Romberg integration by using the Durbin method.

Page 294: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

267

F(S)~s/(S2t-1), f(t)~t/2 sint.

t Exact Solution Dubner and Abate Durbin Crump

1 0.4207355 0.420732 0.4207785 0.4207348

2 0.9092975 0.909297 0.9092443 0.9092966

3 0.2116800 0.211683 0.2116644 0.2116795 i

4 -1.5136050 -1.513614 -1. 512939 -1.5136050

5 -2.3973107 -2.39'7300 -2.3972"16 -2.3973100

6 -0.8382466 -0.838262 -0.8382199 -0.8382477

7 2.2994530 2.299529 2.299371 2.2994370

8 3.9574329 3.957423 3.957483 3.9574310

9 1.8545331 1.854535 1. 854252 1. 854533

10 -2.7201055 -2.720956 -2.719565 -2.7201030

TABLE 6.1

2 2 . t::' F(s)~l/(s +s+l) , f(t)~ - exp(-t/2)sin(dt/2).

13

t Exact Solution Dubner and Abate Durbin Crump

1 0.5335073 0.5335338 0.5335343 0.5335066

2 0.4192797 0.4192651 0.4192649 0.4192776

3 0.1332426 0.1332654 0.1332644 0.1332419

4 -0.0495299 . -0.0494962 -0.0~94953 -0.0495370

5 -0.0879424 -0.0879581 -0.0879563 -0.0879449 .

6 -0.0508923 -0.0508995 -0.0508996 -0.0508929

7 -0~0076437 -0.0076448 -0.0076477 -0.0076439

8 0.0127151 0.0127121 0.0127137 0.0127149

9 0.0128046 0.0128020 0.0128079 0.0128046

10 0.0053854 0.0053849 0.0053881 0.0053859

TABLE 6.2

Page 295: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

F(s)=(2f-;-1/(s+l»; f(t)=2-exp(-t).

Increment in t=O.OOOOl

0.4686862

0.4687000 0.4687046

0.4686812 0.4686749 0.4686729

0.4686969 0.4687022 0.4687039 0.4687044

0.4686954 0.4686948· 0.4686943 0.4686942 0.4686942

TABLE 6.3

6.3.1 The Implementation of the Fast Fourier Transform (FFT)

Technique

Although the earliest discoveries of the FFT technique was in

1942 (Danielson and Lanczos), this technique became generally known

only in the mid-1966's from the work of Cooley and Tllkey [Cooley,

1965]. This technique reduced the computational time of the Fourier

transform from N2

to less than Nlog2

N, where N is usually taken to be

a power of· 2.

An attempt to introduce the use of the FFT for numerically

inverting the Laplace transform was first suggested by Wing [Wing,

1968]. Dubner-Abate also applied the implementation of the FFT

technique to their method. Their implementation is as follows:

The finite Fourier transform pair X(j)" A(k) is defined by,

N-l

268

x (j) \' 2Tfi L A(k)exp(~ jk) , (6.3.24)

k=O

for j=0,1,2, ... ,N-l. Also equation (6.3.10) may be written as,

Page 296: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

f (t) -

vt e

T { I

n=-CIO

269

Re{F(v+ i1Tn)} exp(i1Tn t)}. T T

(6.3.25)

Now, if we require the value of f(t) at the equidistant points

t=j~t, where j=O,1,2, •.. ,~, i.e., ~t is the desired sampling

interval and the maxilllum t-values is t max 1

:: ~L)t. Then, we have

either T=2t or T=tN~t. From these definitions we can write max

equation (6.3.25) as,

f (j~t) N-l

( 21Ti = I A(k) exp -- jk)

b (j) N ,

k=O where,

~

A(k) 1 I Re{F(v+

21Ti (k+nN»} = N~t N

n=-CIO

and,

b(j) = 2exp(vj~t)

Hence, the right hand side of equation (6.3.26a) is of the same

form as equation (6.3.24), thereby permitting the use of the FFT

(6.3.26a)

(6.3.26b)

(6.3.26c)

technique. In a silllilar manner, Durbin, applied the FFT technique to

his method.

In 1987, Hsu and Dranoff [Hsu, 1987] ilIlplemented the FFT

technique to the method that is described by Crump (equation 6.3.22).

By applying the trapezoidal rule approximation to equation (6.3.2b)

with W=k1T/t and ~W=1T/t, we obtain,

vt e

f(t) ~ 2T { I k=-co

F(v+ik1T/T)exp(ik1Tt/T)}.

Assuming t=j~T, then equation (6.3.27) can be written as,

f (j~T)

where,

A(k) =

= exp(vjllT) { 2T

N-l I {A(k)exp(21Tikj/N)}}

k=O

F{v+i1T(k+nN)/T}, n=-IXI

(6.3.27)

(6.3.28)

(6.3.29)

Page 297: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

270

with N~T;2T and j;0,1,2, ... ,N-l. Now equation (6.3.28) and (6.3.29)

can be computed by the FFT technique as follows. First, A(k) can be

obtained from F(s) by the use of equation (6.3.29). Then, the complex

conjugate of A(k) will be the input to the FFT subroutine and the

output will be X(j) for j;0,1,2, ... ,N-l. The inverse function is then

given by,

f (j~T) = exp(vj~T)

2T x (j) j==O,1,2, .. : ,N-l, (6.3.30)

where X(j) is the complex conjugate of X(j). Since the imaginary part

of X(j) is zero, X(j);X(j).

There are some disadvantages of the FFT technique, when applied

to the numerical inversion of the Laplace transforms. These

disadvantages are:

(i) When only a small number of transform values are required

it is not economic to use FFT. Since the Fourier transform

needs only O(N) operations for each transformed value

required.

(ii) N must be a highly composite number (i.e. N;2m). Otherwise

it is necessary to add a string of zero values to the data

to make the length a power of 2.

(iii) If a re-ordering has to be carried out, no real savings in

computing time is achieved for N<SO.

Thus, on the remarks above, we can say that when N is too small

or only a small number of transformed values are required, the FFT

algorithm is not economic.

Page 298: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

271

.6.4 PARALLEL IMPLEMENTATION OF THE NUMERICAL INVERSION OF THE LAPLACE

TRANSFORM

We give in this section a parallel treatment of Crump's

expression that numerically inverts the Laplace transform,

~

vt \' f(t) - (e /T) [!F(v) + L {Re{F(v+iklT/T) }cos(klTt/T) k=l

-Im{F(v+iklT/T)}sin(klTt/T)}]

Numerical experiments with this parallel algorithm have been

carried out on the Balance-8000 multiprocessor MIMD system, where we

used all the available 5 processors.

The speed-up values represent the ratio of the time taken by the

parallel version over the sequential version, which contained none of

the special programming constructs required for parallel processing

over the parallel version.

Tables 6.4 and 6.5 lists the elapsed execution times (in seconds)

minus the cost of forking child processes and child page table build

up, since child processes do not automatically inherit the parent's

page table when they are created.

The speed-up values are plotted in Figures 6. 3 and 6.4 .

Page 299: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

F(s)=exp(-4rs)

No. of Points

10

3 -; f(t)= 2exp(-4/t) (TIt)

Elapsed time No. of Processors in seconds

1 55.008

2 34.942

3 27.828

4 22.422

5 17.522

TABLE 6.4

-; -1 F(s)=s exp(-s )

-; ; f(t)=(TIt) cos(2t)

Elapsed time No. of Points No. of Processors in seconds

10 1 31.994

2 16.628

3 10.784

4 8.828

5 7.989

TABLE 6.5

272

Speed-up

1

1.574

1.976

2.453

3.139

Speed-up

1

1.924

2.966

3.624

4.004

Page 300: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

273

5

4

3

o~~~~~~~~==~~~==~~=7 o 2 3 4 5 6

No. 01 proes.

FIGURE 6.3: The Timing Results for the First Problem.

Page 301: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

274

1 2 3 4 5 6 No. of proc8.

FIGURE 6.4: The Timing Results for the Second Problem.

Page 302: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

.6.5 CONCLUSION

In this chapter, we have presented different methods for

numerically inverting the Laplace transform. Each of these methods

has its own shortcoming. Although we recommend Crump's method, there

is no general technique to invert the Laplace transform function

numerically.

The proper formula for the numerical inversion of Laplace

transform based on the Fourier series method should contain both the

sine and cosine function terms. Eliminating "either the sine or cosine

function and replacing the integration by a discrete formula will

distort the original function into an even or odd function with a

concomitant decrease in the working interval of the inverted function

from (O,2T) to (O,T).

275

An attempt to increase the accuracy by applying the Romberg

integration resulted in giving accurate results for small time intervals,

which is too small to be of practical use in applying it to the full

range of the transform.

Although the direct FFT technique was found to be easy in

application to numerically inverting the Laplace transform, it has

been shown that it has many disadvantages as outlined earlier.

The improvement in performance attainable by the implementation of

the Laplace inversion transform is considerable: Which means that the

numerical inversion of the Laplace transform algorithm is a viable

parallel algorithm.

Page 303: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS
Page 304: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

CHAPTER 7

CONCLUSIONS AND FINAL REMARKS

I am no prophet -

and here's no great matter.

T.S. Eliot.

Page 305: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

276

In the past computers have developed more processing power,

firstly, by an increase in the density of integration and, secondly,

by the operational acceleration of their basic switching elements.

Both L~ese methods, however, lead to higher power dissipation per unit

area to which there are fundamental thermodynamic limits.

For further improvement of the performance attainable by present

day computers technologically, it is necessary that radical new

developments must take place.

ParaZZeZ processing, therefore, is widely viewed as the only

natural and feasible way forward to achieve a significant increase in

processing power. Parallel computers, as has been discussed in the

introductory chapters of this thesis, are classified in various

different types, each of which has an effective range of problems to

which it is most suitable for solving.

To date, in concern with the imminent next generation of

computer systems, the epitome of contemporary computing - the

supercomputer - consists of banks of high-speed processors, operating

in parallel on arrays of numbers or closely coupled in a pipeline,

with further processors to pass the information to and from secondary

storage. The total assembly achieves a very high throughput of data,

and makes use of densely packed VLSI chips.

In general, programming parallel systems is more difficult than

programming uniprocessor systems, and this had led to the parallelism

being concealed on most existing MIMD systems. This kind of system

consists of the primary testbed for all numerical algorithms designed

and analysed herein. The actual prototype on which the extensive bulk

Page 306: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

277

of our experiments and measurements were carried out was the Balance

8000 MIMD system at Loughborough University of Technology.

The problem which arises here lies in making the p processors

cooperate, so that one problem can be appropriately partitioned amongst

them to be solved with greater speed than could be solved on a uni-

processor.

The analysis of an algorithm is important for different reasons.

The most straightforward reason is to discover its vital statistics

in order to evaluate its suitability for various applications or

compare it with other algorithms. Generally, the vital statistics of

interest are the time and space, most often time. Actually, in.all

the algorithms presented in this thesis, we were interested in

determining the times for an implementation of a particular algorithm

on the Balance 8000 parallel system. From that analysis, we were able

to measure the relative speed-up ratio of each algorithm· which was the

main criteria to exploit the efficiency by using parallelism.

In this theSis, we have· demonstrated a characterization of the

architecture of current parallel computers such as SIMD and MIMD

computers and how the research in this area is developing as technology

advances where the whole computer can be assembled on a silicon VLSI

chip, which may contain millions of transistors.

In the second chapter, we have showed how to program parallel

systems. However, as we have demonstrated different applications

require different language structures. One of the major divisions in

language development is that between the imperative and declarative

style of programming. The other major development is the object-oriented

Page 307: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

approach to programming. We have also showed that some specific

languages have been used on a parallel computer by adding some

parallel constructs.

In Chapters 4 and 5, the parallel AGE iterative method, to solve

one and two dimensional elliptic and parabolic equations as well as

278

the wave equation, was developed and implemented on the Balance 8000

parallel system. The implementation of the parallel AGE iterative

method was programmed using different versions and strategies involving

synchroneity and asynchroneity together with natural or red-black

ordering. It is clear that the implementation of the different

strategies presented different 'timings and losses when they were run

on the Balance 8000 parallel system.

It can be seen from the experimental results that the best

results were obtained when the problem is solved using the asynchronous

strategy of the parallel AGE iterative method. This is due to the

synchronization overheads occurring at the end of each iteration.

Also, we noticed that the results obtained by applying the first

strategy is better than that of the second'strategy. This is due to

the total number of computer operations in the second strategy being

higher than that of the first strategy. As the old values' are used

while evaluating the next point using the second strategy.

The Sturm-Liouville problem is also solved using the AGE

iterative method, in Chapter 4. The eigenvalues and the corresponding

'eigenvectors were determined.

The AGE iterative method with D'Yakonov splitting strategy to

solve the diffusion equation is presented in Ch'apter 5. The new

Page 308: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

strategy yields an algorithm with reduced computational complexity and

greater accuracy for multi-dimensional problems. The same three

parallel strategies were implemented on this kind of formulation.

279

A new strategy for solving the (nXn) complex tridiagonal matrices

derived from the finite difference/element discretisation of the

Schrodinger equation is presented in Chapter 5. In this strategy, the

solution of the complex system has been reduced to solve two identical

quindiagonal systems, in real variables. These can then be solved

directly by efficient elimination procedures involving no interchanges.

By the use of the quindiagonal solver, the computational complexity for

the algorithm is now of order O(n) multiplications, whilst the Gaussian

elimination procedure for the original matrix is considerably more.

The fundamental importance of the Laplace transform resides in its

ability to lower the transcendence level of an equation. Ordinary

differential equations involving f(t) are reduced to algebraic

equations for F(s) .

The evaluation of inverse Laplace transforms of functions is a

problem of fundamental importance in pure and applied mathematics.

Thus, it is often desirable to have a convenient means of computing

·f(t) numerically for various values of t from the given values of F(s) ,

where th~ function F(s) might be either a complicated expression

whose poles and residues are too difficult to obtain, or F(s) might be

itself known only numerically as a computed function of s without any

knowledge as to its explicit analytic form.

A large number of different methods for numerically inverting the

Laplace transform has been developed during the last few years. In

Page 309: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Chapter 6, we have briefly mentioned some of these methods, whilst we

restrict ourselves to the methods using Fourier series approximations.

We carried out a number of experimental tests to demonstrate the

simplicity and the accuracy of these methods. To speed up these

methods, we implemented the Crump's method on the Balance 8000 MIMD

parallel system, where the resulting times are represented

diagramatically.

Further work is required to include more than one iteration

parameter r in the AGE iterative method for which an improvement in

the rate of convergence is predicted.

280

The study in Chapter 5 of the new strategy for the numerical

solution of the Schrodinger equation can be easily extended to two

dimensional equations as well as to the nonlinear Schrodinger equation.

Finally, more extensive work is required on the numerica~

inversion of the Laplace transform. Techniques for its efficient

application to determine the numerical solution of certain classes of

linear parabolic and hyperbolic equations especially those involving

independent variables, time t and two spatial variables x and y. The

basic idea follows the usual Laplace transform approach in that after

applying the transform to the differential equation the number of

independent variables is reduced by one and the resulting subsidiary

equation involves a complex parameter s instead of the variable t.

The numerical application of a suitable inversion procedure

means that the subsidiary equation has to be solved for several

different values of the parameter s, in order to obtain the solution

of a parabolic or hyperbolic problem at some specified time t.

Page 310: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS
Page 311: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

REFERENCES

Page 312: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

281

Anderson, J .P. (1965): "Program Structu:r>es for PaI'aZZeZ PI'ocessing",

Comro. of ACM, Vol.8, No.12, pp.786~788.

Baer, J.L. (1982): "Techniques to ExpZoit PaI'aUeUsm", in Parallel

Processing Systems, An Advance Course, ed. Evans, D.J.,

Cambridge Univ. Press, pp.75-99.

Barlow, R.H., Evans, D.J., Newman I.A. and Woodward, M.C. (1981):

"A Guide to Using the Neptune PaI'aUeZ PI'ocessing System",

Internal Report, Computer Studies Dept. L.U.T.

Baudet, G.M., Brent, R.P. and Kung, H.T .. (1980): "ParaUeZ Execution

of a Sequence of Tasks on an Asynchronous MuUipI'ocessor",

The Australian Computer J., Vol.12, No.3, pp.105-112.

Bernstein, A.J. (1966): "AnaZysis of PI'ogI'ams fop ParaUeZ Processing",

IEEE Trans. on Electronic Computers, Vol: EC-15, No.5,

pp.757-763.

Booch, G. (1986): "Object OPiented Development", IEEE Trans. Software

Eng., Vol. SE-12, pp.211-221 .

.,. Carre, B.A. (1961): "The Determination of the Optimum AcceZeI'ation

FactoI' fop Successive OveI'I'eZaxation" , The Comput.J., Vol.4,

pp.73-78.

Page 313: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Conte, 5.0. [1965]: "Elementary Nwneric:al Analysis", McGraw-Hi11 Inc.,

New York.

Cooley, J.W. and Tukey, J.W. [1965]: "An Algorithm for the Mac:hine

Calc:ulation of Complex Fourier Series", Maths.Comp.J.,

vol.19, pp. 297-301.

Crump, K.S. [1976]: "Numeric:al Inversion of Laplac:e Transforms Using a

Fourier Series Approximation", J. ACM, Vol. 23, No. 1,

pp. 89-96.

282

Davies, B. and Martin, B. [1979]: "Numeric:al Inversion of the Laplac:e

Transform: A Survey and Comparison of Methods", J. of Comput.

Phys., Vol.33, pp.1-32.

Dennis, J.B. and Van Horn, E.C. [1966]: "Programming Serrrmtic:s for

Multiprogrammed Computations", Comm. of the ACM, Vol. 9,

pp.143-155.

Dijkstra, E. W. [1965]: "Guarded Commands Nondeterminanc:y and Formal

Derivation of Programs", Comm.Assoc.Comput., Vol.18, pp.457.

Douglas, J. and Rachford, H.H. [1956]: "On the Numeric:al Solution of

Heat Conduc:tion Problems in Two and Three Spac:e Variables",

Trans.Amer.Math.Soc., Vol.82, pp.421-439.

Page 314: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Dubner, H. and Abate, J. [1968]: "Nwnerical Inversion of LapZace

Transforms by Relating Them to the Finite Fourier Cosine

Transform", . J .ACM, vol.15 , No.1, pp .115":123.

Durbin, F. [1973]: "Nwnerical Inversion of Laplace Transforms: An

Effective Improvement of Dubner and Abate's Method",

Comput.J., Vo1.17, No.4, pp.371-376.

D' Yakonov, Y. [1963]: "On the Application Distintegrating Difference

Operators, ·Z,Vycil.Mat.i.Mat.Fiz., vol.3, pp.385-388.

Enslow, P.H. [1977]: "Multiprocessor Organization - A Survey",

Comput. Surveys, Vol. 9, No. 1, pp.l03-129.

Evans, D.J. and Williams, S.A. [1978]: "Analysis a"d Detection of

Parallel Processable Code". The Comput.J. Vo1.23, No. 1,

pp.66-72.

Evans, D.J. [1985]: "The AGE Matrix Iterative Method". 3rd Franco

South Asian Math.Conf., Kuala Lumpur, Malaysia.

Feng, T.Y. [1974]: "Data Manipulation Functions iT; Parallel Processors

and Their Implementations", IEEE Trans. on Comp., Vol. C-23,

No.3, pp.309-318.

F1ynn. M. J. [1966]: "Very High Speed Computing Sysoems", Proc. IEEE,

Vo1.54, pp.1901-1909.

283

Page 315: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Fox, L. [1957]: "The Nwnerical Solution of Two-Point Boundary

Problems in Ordinary Differential Equations",

The Clarendon Press, Oxford.

Gill, S. [1958]: "Parallel Prograrmling", Comp.J. Vol. 1, pp.2-10.

Gurd, J.R., Kirkham, C.C. and Watson, I. [1985]: "The Manchester

Prototype Data-Flow Computer", Comm. ACM, No.l, pp.34-52.

Hageman, L.A. and Kellogg, R.B. [1968]: "Estimating Optimwn Over­

relaxation Parameters", Maths. of Comp., Vol.22, pp. 60-68.

Handler, W., [19-i7]: "The Impact Of Classification Schemes on Computer

Architecture", Int.Conf. on Parallel Proc., pp.7-15.

Harland, D.M. [1985]: "ToWards a Language for Concurrent Processes",

Software Practice and Experience, Vol.15, pp.839-888.

Hellerman, H. [1966]: "Parallel Processing of Algebraic Expressions",

IEEE Trans. on Electronic Computers, Vol. EC-15, pp.82-91.

Hobbs, L.C. and Thesis, D.J. [1970]: "SuY'Vey of Parallel Processor

Approaches and Techniques", in parallei System: Technology

and Applications, ed. by Hobbs, Spartan Books, New York,

pp.3-20.

284

Page 316: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Honig, G. and Hirdes, U. [1984]: "A Method for the Nwnerical

Inversion of Laplace TransfoI'Tlls", J .Comput. and Appl.Math.,

Vol. 10, pp.113-l32.

Hockney, R.W. and Jesshope, C.R. [1988]: "Parallel Computers:

Architecture, Programming and Algorithms", Adam Hilger,

Bristol.

285

Hsu, J.T. and Dranoff, J.S. [1987]: "Nwnerwal Inversion of Certain

Laplace TransfoT'Tlls by the Direct Application of FFT Algorithm",

Comp.Chem.Engng., Vol. 11, No.2, pp. 101-110.

INMOS [1984]: "Occam Programming Manual", Englewood Cliffs, N.J.:

Prentice Hall.

Kuck, D.J. [1977]: "A Su:r>vey of Parallel Machine Organization and

Programming", Comp. Surveys, Vol. 9, No. 1, pp.29-59.

Kung, H.T. [1980J: "The Structure of Parallel Algorithms", Advances

in Computers, Vol. 19, pp.65-ll2.

Kung, H.T. [1982J: "Notes on VLSI Computation", in Parallel Processing

Systems, ed. Evans, D.J., Cambridge Univ. Press, pp.339-356.

Lanczos, c. [1956J: "Applied Analysis", Prentice-Hall, Englewood Cliffs,

New Jersey.

Page 317: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Lawrie, D.H., Layman, T., Baer, D. and Randal, J.M. [1975]: "Glipnir -

A Programming Language for ILLIAC IV", Comm. ACM, Vol.1B,

pp.157-164.

2B6

Miranker, W.L. [1971]: "A Survey of Parallelism in Numerical Analysis",

SIAM Review, Vol. IB, pp.524-547.

Muraoka, Y. [1971]: "Parallelism Exposure and Exploitation in Programs",

Ph.D. Thesis, Univ. of Illinois at Urbana-Champaign.

Murtha, J. and Beadles, R. [1964]: "Survey of the Highly Parallel

Information Processing Systems", Prepared by Westinghouse

Electroc Corp., Aerospace Division, ONR Rep. No. 4755.

Patel, J.H. [19Bl]: "Performance of Processor-Memory Interconnections

for Multiprocessors", IEEE Trans. Comp., Vol. c-31, No. 10,

pp. 771-7BO.

Peaceman, D.W. and Rachford, H.H. [1955]: "The Numerical Solution of

Parabolic and Elliptic Differential Equations", J.Soc.lndust.

Appl.Math., Vol. 3, pp.2B-41.

Ramamoorthy, C.V. and Li, H.F. [1977]: "Pipeline kPchitecture",

Comp. Survey, Vol. 9, No. 1, pp.61-102.

Salzer, W.E. [195B]: "Tables for the Numerical Calculation of Inverse

Laplace Transforms", J.Math. and Phys., Vol.37, pp.B9-109.

Page 318: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Saul'yev, V.K. [1964]: "Integrotion of Equations of Parabolic Type by

the Method of Nets", G.T. Tee, Transl., Pergamon, New York.

Schmittroth, L.A. [1960]: "NumeI'ical InveI'sion of Laplace Tr>ansfoI'lTls",

Comm. ACM, Vol. 3, pp.171-173.

Schapery, R.A. [1962]: IIPr>OC. 4th V.S. Nat. CongI'., Appl.Mech.",

ASME 2, pp.1075-1085.

Seigel, H. T. [1979]: InteI'connection Networks foI' MIMD Machines 11,

IEEE Comp.J., pp.57~65.

Seitz, C.L. [1982]: IIEnsemble AI'chitecture for VLSI - A Survey and

Taxonomy 11, in Proc. of MIT Conf. on Advanced Research in

VLSI, ed. by Penfield, P. and Artech House Jr., pp.130-135.

Shapiro, E. Y. [1984]: IIA Subset of ConcurI'ent Prolog and its

InterpretoI'II, ICOT Inst. for New Generation Technology,

Technical Rep. TRoo3.

287

Shirtlifte, C.J. and Stephenson, D.G. [1961]: IIA ComputeI' Oriented

Adoption of SalzeI' IS Methods fOI' InveI'ting Laplace Tr>ansfoI'lTls ll ,

Ibid., Vol. 40, pp.135-141.

Shore, J.E. [1973]: "Second Thoughts on PaI'allel Processing ll , Comput.

Elect.Engng., pp.95-109.

Page 319: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Simon, R.M., Stroot, M.T. and Weiss, G.H. [1972J: "NwneT'ical

InveT'sion of Laplace Transforms with Application to

Percentage Labeled Experiments", Comput. Biomed. Rev.,

Vol.6, pp. 596-607.

Stone, H.S. [1967J: "One-Pass Compilation of Arithmetic Expressions

288

for a ParaLLel Processor", Comm. ACM, Vol. 10, No.4, pp.220-223.

Stone, H.S. [1971J: "ParaLLel Processing with the Perfect Shuffle",

IEEE Trans.Comp., vol. C-20, pp.153-161.

stone, H.S. [1973J: "Problems of ParaLLel Computation", in Complexity

of Sequential and Parallel Algorithms, ed. Traub, J.F.,

Academic press, New York.

Stone, H.S. [1973J: "An Efficient ParaLLel Algorithm for the Solution

of a Tr>idiagonal Linear System of Equations", J. of ACM,

Vol.20, No.l, pp.27-38.

Squire, J.S. [1963]: "A Translation Algorithm for Multiprocessor

Computers", Proe. 15th ACM Natl.Conf.

Tedd, M., Crespi-Reghizzi, S. and Natali, A. [1984J: "Ada for

Mul tipT'ocessors", Ada Companion Series, Cambridge Universi ty·

Press.

Page 320: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

Varga, R.S. [1962J: "Matrix Iterative Analysis", Prentiee-Ball,

Eng1ewood Cliffs.

Wilkinson, J.B. [1965J: "The Algebraic Eigenvalue Problem", Oxford

University Press, Oxford.

Wing, o. [1967J: "An Efficient Method of Nwnerical Inversion of

Laplace Transforms", Arehs.E1eetron.Comp., vo1.2, pp.153.

Wyllie, J.C. [1979J: "The Complexity of Parallel Computation",

Ph.D. Thesis, Comp.Sei. Dept., Corne11 Univ., U.S.A.

Wynn, P. [1962J: "Upon a Second Confluent Form of the £-Algorithm",

Proe. Glasgow Math.Assoe., Vol. 5, pp.160.

Young, D. [1954J: "Iterative Methods for Solving Partial Difference

Equations of Elliptic Type", Trans .Amer .Math .Soe., Vol. 76,

pp.92-111.

Zakian, V. [1969J: E1eetron.Lett. 5, pp.120-121.

289

Page 321: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS
Page 322: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

APPENDIX A

A LIST OF SOME SELECTED PROGRAMS

Page 323: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

1 c 2 c 3 c 4 c 5 c 6 c 7 c 8 c 9 c 10 c 11 c 12 c 13 c 14 c 15 c 16 c 17 c 18 c 19 20 c 21 22 c 23 24 25 26 27 28 29 30 31 32 33 c 34 35 36 37 38 39 40

* *

Program 4.1 One-Dimensional Boundary Value Problem Solved Using the Parallel AGE iterative Method with the Thrid Strategy. d**2U/dx**2+U = x Boundary Conditions U(O) = 1 ,U(1tI2)=1tI2 -1 . Exact Solution U(x)= cosx - sinx + x . n : The order of the system m: The iteration counter a : The subdiagonal of A c : The superdiagonal of A d : The diagonal of A hd : Half the diagonal of A b: The right hand side vector r : The iteration parameter

Set up common block to hold shared variables COMMON /SHCOM/a,b,c,hd,m,n,r,v,w,eps,det,u/ Declare subroutine age as argument to a function EXTERNAL age Declare system calls integer m_fork,m_kill-procs,m_set-procs integer time1,time2,bite_size ParameterOmax=50) dimension aOmax),bUmax),cOmax),dOmax),hdOmax),detOmax)

,u(0:jmax),u1 (0:jmax),u2(0:jmax),u3(0:jmax),wOmax) ,vOmax)

data eps,pi/0.000001,3.1415926/ r=1.44 h=(pi/2.0)/real(n+ 1) hh=h*h Input matrix size. do 10 i=1,n a(i)= -1.0 c(i)=a(i) d(i)=(2.0-hh) hd(i)=d(i)/2.0 b(i)= - hh*i*h u(i)=0.1

290

Page 324: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

41 10 42 43 44 45 46 47 48 c 49 50 51 52 53 54 55 c 56 c 57 58 c 59 60 c 61 62 c 63 64 65 100 66 67 101 68 69 70 71 c 72 73 74 75 76 c 77 78 79 c

continue a(1 )=0.0 c(n)=O.O u(0)=1.0 u(n+ 1 )=piJ2.0 - 1.0 b(1 )=1.0 - hh*h b(n)=(pi/2.)-hh*h*n Divide work up into rn/bite_size] chunks x=real(n)/real(bite_size) if(real(int(x)).eq.x)then no--procs=x else no--procs=int(x)+ 1 endif Set the number of processors to be used and therefore the number of chunks to be evaluated at the same time i=m_set--procs(no--procs) Start timing call 310ck_time(time1) Fork offspring then assist them to evaluate subroutine i=m_fork(age) Finish time call _clock_time(time2) write(*,1 00)(time2-time1 )/1 00.0 format(5x,'The time = ',f8.5) write(*,1 01 )(u(i),i=1,n) format(5x,f8.5) stop end subroutine age Common block contains shared variable list

. COMMON /SHCOM/a,b,c,hd,m,n,r,v,w,eps,det,u/ parameterOmax=50) dimension aUmax),bUmax),cUmax),hdOmax),u(O:jmax),

* u1 (O:jmax),u2(O:jmax),detUmax) System call INTEGER m_next integer path,start,finish,bite_size Get process path numbers

291

Page 325: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

80 81 c 82 c 83 102 84 85 c 86 c 87 88 89 90 91 92 93

·94 11 95 96 30 97 98 99 12 100 c 101 c 102 c 103 104 105 106 107 108 109 110 111 13 112 c 113 c 114 c 115 c 116 c 117 118 c 119

path=m_nextO start=(path-1 )*bite_size-1 Whilst there is more work loop if(start.le. n)then finish=start+bite_size-1 If this is the last item of work,then set finish pointing to the end of the last item if(path.eq.no-procs) finish=n do 103 k=start, finish n1 =n-1 n2=n-2 do 11 i=1,n w(i)=hd(i)H v(i)=hd(i)-r det(i)=(w(i)·w(i+ 1 )-a(i+ 1 )*c(i)) m=O· iflag=O m=m+1 do 12 i=1,n u1 (i)=u(i) ••••• The first Sweep •••••

==============

u(k+ 1/2)=(G1 HI)<-1 >·[b-(G2-rl)·u(k)] do 13 i=1,n1,2 i1=i+1 i2=i+2 i3=i-1

v1 =b(i)-a(i)·u1 (i3)-v(i)·u1 (i) v2=b(i1 )-v(i1 )*u1 (i1 )-c(i 1 )·u1 (i2) u2(i)=(w(i1 )·v1-c(i)·v2)/det(i) u2(i 1 )=( -a(i 1 )·v 1 +w(i)·v2)/det(i) continue

..... The second sweep •••••

================

u(k+ 1 )=(G2HI)<-1 > ·[b-(G1-rl)·u(k+ 1/2)]

u(1 )=(b(1 )-v(1 )·u2(1 )-c(1 )·u2(2))/w(1)

do 15 i=2,n,2

292

Page 326: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

120 . 121

122 123 124 125 126 127 15 128 c 129 130 c 131 132 17 133 134 103 135 136 137 138 139 c 140 141

i1=i+1 i2=i+2 i3=i-1 v1 =b(i)-a(i)*u2(i3)-v(i)*u2(i) v2=b(i)-v(i)*u2(i1 )-c(i1 )*u2(i2) u(i)=(w(i 1 )*v1-c(i)*v2)/det(i) u(i1 )=(-a(i1 )*v1 +w(i)*v2)/det(i) continue

u(n)=(b(n)-a(n)*u2(n 1 )-v(n)*u2(n))/w(n)

do 17 i=1 ,n if(abs(u(i)-u1 (i)).gt.eps) iflag=1 if(iflag .eq.1) goto 30 continue path=m_nextO start=bite_size*(path-1 ) goto 102 endif All done so return to main program return end

293

Page 327: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

•. . '. "

1 2 3 4 5 6 7 8 9 10. 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

c c c c c c c c

c

10 c

11

Program 4.2 This program is to calculate the eigenvalues and the corresponding eigenvectors,by the use of the AGE iterative method.

d**U/dx**2+(lemda-2*q*cos2x)U=0. Boundary Conditins, U(O)=O , U(7tl2.)=1t parameter(jmax=50) data eps,pi/0.000001 ,3.14159261 reallemda,lemda2,le read(*:)n

dimension a(jmax),b(jmax),c(jmax),d(jmax),u(jmax) * ,da(jmax),ud(jmax),le(jmax),ude(jmax)

choose the accelaration parameter r=1.50 h=pi/real(n+ 1) hh=h*h q=1.0 do 10 i=1,n xi=i*h r d(i)=(2.0/hh+2.0*q*COS(2·ce*h» hd(i)=d(i)/2.0

.do 11 i=1,n _.a(i)=-1 .O/hh ~.c(i)=a(i) ' .. u(i)=0.1 ·u(O)=O.O 'c:u(n+ 1 )=0.0 '. a(1 )=0.0 7c(n)":0.O -;'~sum:l =0.0

.:_. _.7., _'_

"." ,::"'.: ~ .... : .

34 -.. sum2=0.0 35 7od(1 )=hd(1 )*u(tJ+C(1)*d(2)" ~, •. , ...-.

36 do 13 i=2,n-1 37 13 ud(i)=a(i)*u(i-1 )+hd(i)*u(i)+c(i)*u(i+ 1) 38 ud(n)=a(n)*u(n-1 )+hd(n)*u(n) 39 do 14 i=1 ,n

294

Page 328: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

40 sum1 =sum1 +ud(i)*u(i) 41 14 sum2=sum2+u(i)*u(i) 42 lemda2=sum1/sum2 43 1=0 44 15 lemda=lemda2 45 30 1=1+1 46 16 do 17 i=1 ,n 47 b(i)=lemda*u(i) 48 c call the AGE subroutine. 49 c 50 call age(a,b,c,hd,m,n,eps,u,r) 51 sum1=0.0 52 sum2=0.0 53 ud(1 )=hd(1 )*u(1 )+c(1 )*u(2) 54 do 17 i=2,n-1 55 17 ud(i)=a(i)*u(i-1 )+hd(i)*u(i)+c(i)*u(i+ 1) 56 . ud(n)=a(n)*u(n-1 )+hd(n)*u(n) 57 d018i=1,n 58 . sum1=sum1+u(i)*ud(i) 59 18 . sum2=sum2+u(i)*u(i) 60 c Calculate a new value for lemda 61 'c 62 63 64 65 66 67 68 19 69 70 71 20 .72 21

--- <--73 c 74 75 76 22 77 78

lemda2=sum1/sum2 if(abs((lemda2-lemda)/lemda).gt.eps) goto 30

• i -·.lemda=lemda2; '." .~ ... write(*,19) n,eps,m,r write(*,20)(u(i),i=1,n) write(* ,21) lemda,1

, "format(/5x,'n,,;~';i3,5x/epsc', f9.6, . c* /5x,' No. of iterations=',i4,5x,'r = ',f9.6

* /5x,'The solution vector is '/5x,22(' " ')) . - format(5x,f1 0.6)~-.. format(5x,'~em~a= ',f9.6,4x;T= ',i3) ; !- '" , .. ude(1 )=hd(1 )*u(1 )+c(1 )*u(2) - .

do 22 i=2,n-1 ude(i)=a(i)*u(i-1 )+hd(i)*u(i)+c(itu(i+ 1) ude(n)=a(n)*u(n-1 )+hd(n)*u(n) do 23 i=1,n

295

Page 329: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

t!.1· .

79 23 le(i)=lemda·u(i) 80 write(·,24) 81 write(· ,25) 82 24 format(1I2x,'A·u(i)',8x,' lemda·u(i)' ) 83 25 format(2x,·"nllllnnnllllllnll""" I,ax,'""""""""""""" ') 84 do 26 i=1 ,n 85 26 write(·,27) ude(i),le(i) 86 27 format(2x,f9.6, 11 x,f9.6) 87 stop 88 end 89 C

90 C

91 subroutine age(a,b,c,hd,m,n,eps,u,r) 92 parameterGmax,;,,50) 93 dimension aGmax),bGmax),cGmax),hdGmax),uGmax), ' 94 • u1 Gmax),u2Gmax),detGmax),wGamx),vUmax) 95 real length 96 n1=n-1 97 n2=n-2 98 do 28 i=1,n 99 w(i)=hd(i)+r 100 v(i)=hd(i)-r 101 28 det(i)=(w(i)·w(i+ 1 )-a(i+ 1 )·c(i» 102 m=O 103 29 m=m+1 104 do;31 i=1,n 105 31 u1 (i)=u(i) 106 c ••••• The first sweep •••••

107 c ============== 108 do 32 i=1 ,n1 ,2 109 i1=i+1 110 111 112 113 114 115 116 117 118

32 c c

119 c

i2=i+2 i3=i-1

vi =b(i)-a(i)·u1 (i3)-v(i)·u1 (i) v2=b(i1 )-v(i1 )·u1 (i1 )-c(i1 )·u1 (i2) u2(i)=(w(i1 )·v1-c(i)*v2)/det(i) u2(i1 )=(-a(i1 )*v1 +w(i)·v2)/det(i) continue

..... The second sweep •••••

================

296

1

Page 330: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

297

120 c 121 u(1 )=(b(1 )-v(1 )·u2(1 )-c(1 )·u2(2))/w(1) 122 do 33 i=2,n2,2

123 i1=i+1 124 i2=i+2 125 i3=i-1

126 v1 =b(i)-a(i)·u2(i3)-v(i)·u2(i)

127 v2=b(i1 )-v(i)·u2(i1 )-c(i1 )·u2(i2)

128 u(i)=(w(i1 )*v1-c(i)*v2)/det(i)

129 u(i1 )=(-a(i1 )·v1 +w(i)·v2)/det(i)

130 33 continue 131 u( n )=(b(n )-a( n )*u2( n 1 )-v( n)· u2( n) )/w( n)

132 do 34 i=1,n 133 33 if(abs(u(i)-u1 (i)).gt.eps) goto 29

134 sumsq=O.O. 135 do 34 1=1,n

136 34 sumsq=sumsq+u(i)**2

137 length=sqrt(sunsq) .

138 do 351=1,N

139 35 u (i) =u (i)/length 140 return 141 end

- - ~ ...... -.. ... :.... .............. .

'': '."

Page 331: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

1 2 c 3 c 4 c 5 c 6 c 7 c 8 c 9 c 10 c 11 c 12 c 13 c 14 c 15 c 16 c 17 c 18 c 19 c 20 c 21 c 22 c 23 c 24 c 25 c 26 c 27 c 28 c 29 c 30 c 31 32 33 34 35 36 37 38 39 40

*

Program 5.1 One-dimensional diffusion-convection equation solved using the parallel AGE algorithm with the second strategy. dU/dt=edU**2U/dx**2-kdU/dx , 0<x<1 ,O<t<oo. Boundary conditions, U(O,t)=O, U(1,t)=1 Initial condition U(x,O)=O 0<x<1.

n: The order of the system a: The subdiagonal of A c: The superdiagonal of A d: The diagonal of A hd: Half the diagonal of A r: The acceleration parameter tl: Maximum value in the t-direction k: Increments in the time sx: Maximum value in the x-direction h: Increments in the x-axis nt: Number of time level parameterOmax=50) dimension u(0:jmax),u1 (0:jmax),u2(0:jmax),u3(0:jmax)

,u40max)

integer*4 nprocs,status,time1, time2 nprocs: The number of processors for each run status: This variable is used to store the return value of

the libr~ry routine that actually sets the number of processors;- : ,",:

write(*,,) 'Number-of procs',n ' read(*, *) nprocs;"n'.'-:: data·eps,kmax;pilO.00005',300,3.14159261 tl 0 1 :.-- ... = ; "- " .. , .. ,---. sx';-l.O r=0.5 k=0.001 nt=21 i1=1 i2=1

.: =.

298

Page 332: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

299 ,

41 h=sxl(n+ 1) 42 hh=h·h 43 nt1 =nt-1 44 rl=klhh 45 c Input matrix size 46 alf=0.5·rl·h 47 a=-0.5·(rl-alf) 48 c=-0.5·(rl+alf) 49 d=(1.0HI) 50 hd=d/2. 51 v=r-hd 52 w=r+hd 53 det=w·w-a·c 54 c 55 do 10 i=O,n 56 u(i)=O.O 57 10 . u3(i)=u(i) 58 . a1=-c·w 59 b1=rl·w 60 c1 =-a·rl 61 d1=a·a 62 a2=c·c 63 b2=-c·rl 64 d2=-a·w 65 s1=d1 66 s2=d2 67 do 11 jt=1,nt1 68 t1 =jt·k 69 jt2=jt+ 1 70 n(n+1)=1.0 71 u3(n+ 1 )=u(n+ 1) 72 ea=0.5·(rl+alf) 73 f=1.0-rl 74 _ g=0.5·(rl-alfj ..

75 do 12 k=1,kmax ... :.' '.: ."; <'.~':'~ ~.

76 ic=1 77 c ••••• Fi rst sweep .....

78 c =========== 79 c u(k+ 1/2)=(G1 HI)<-1 >·[(rl-G2)·u(k)+b]

Page 333: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

80 c 81 82 83 c 84· c 85 c 86 c 87 88 c 89 90 c

b11 =ea·u3(O)-c·u3(O)+f·u3(1 )+g·u3(2) u1 (1 )=(v·u(1 )-a·u(2)+b11 )/w Set the number of processors. Makes the beginning of the timed section of code, which includes everything except VO.

status=m_setJ>rocs(n0J>rocs)

call_clock_time(time1)

91 c$doacross share(u1,u,ea,u3,f,g,w,a,c,a1,b1,c1,d1,det,a2,b2, 92 c$& d2),local(b11,b22,r1) 93 do 13 i=3,n-2,2

.94 b11 =ea·u3(i-1 )+f·u3(i)+g·u3(i+ 1) 95 b22=ea·u3(i)+f·u3(i+ 1 )+g·u3(i+2) 96 r1 =w·b11-a·b22 97 13 u1 (i)=(a1·u(i-1 )+b1·u(i)+c1·u(i+ 1 )+d1·u(i+2)+r1 )/det 98 c$doacross share(u1,u,ea,u3,f,g,w,a,c,a1,b1,c1,d1,det,a2,b2,d2) 99 c$& ,local(b11,b22,r2) 100 do 14 i=2,n-3,2 101 b11 =ea·u3(i-1 )+fu3(i)+g·u3(i+ 1) 1 Q2 b22=ea·u3(i)+f·u3(i+ 1 )+g·u3(i+2) 103 r2=w·b22-c·b11 104 14 u1 (i)=(a2·u(i-1 )+b2·u(i)+b1·u(i+ 1 )+d2·u(i+2)+r2)/det 1 C5 b11 =ea·u3(n-2)+f·u3(n-1 )+g·u3(n) 106 b22=ea·u3(n-1 )+f·u3(n)+g·u3(n+ 1 )-a 107 r1=w·b11-a·b22 1Q8 109 11':) 11, c 112 c 113 c 11-4 c 115 11:5· 117 11,3 11'3

r2=w·b22-c·b11 u1 (n-1 )=(a1·u(n-2)+b1·u(n-1 )+c1·u(n)+r1 )/det u1 (n)=(a2·u(n-2)+b2·u(n-1 )+b1·u(n)+r2)/det ••••• Second sweep •••••

============

u(k+ 1 )=(G2+rI)<-1 >·[(rl-G1 )*u(k+ 1/2)+b]

b11 =ea·u3(O)+f·u3(1 )+g·u3(2) b22=ea·u3(1 )+f·u3(2)+g·u3(3) r11 =w·b11-a·b22 r22=w'b22-c·b11 u2(1 )=(b1·u1 (1 )+c1·u1 (2)+s1·u1 (3)+r11 )/det

300

Page 334: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

120 121 122 123 124 125

u2(2)=(b2*u1 (1 )+b1*u1 (2)+s2*u1 (3)+r22)/det c$doacross share(u2.ea.u3.f.g.w.a.c.a1.b1.c1.s1.u1.det.a2.b2. c$& s2).local(b11.b22.r11)

126 15 127

do 15 i=3.n-2.2 b11 =ea*u3(i-1 )+f*u3(i)+g*u3(i+ 1) b22=ea*u3(i)+f*u3(i+ 1 )+g*u3(i+2) u2(i)=(a1*u1 (i-1 )+b1*u1 (i)+c1*u1 (i+ 1 )+s1*u1 (i+2)

* +r11 )/det 128 129 130 131

c$doacross share(u2.ea.u3.f.g.w.a.c.a1.u1.b1.c1.s1.det.a2.b2. c$& s2).local(b11.b22.r22)

132 133 134 16 135 136 137 138 c 139 c 140 c 141 c 142 143 144 18 145 17 146 147 19 148 149 150 20 151 c 152 c 153 c 154 155 156 157 158 159

do 16 i=4.n-1.2 b11 =ea*u3(i-1 )+f*u3(i)+g*u3(i+ 1) b22=ea*u3(i)+f*u3(i+ 1)+g*u3(i+2) r22=w*b22-c*b11 u2(i)=(a2*u1 (i-1 )+b2*u1 (i)+b1*u1 (i+ 1 )+s2*u1 (i+2)

* +r22)/det bn=ea*u3(n-1 )+f*u3(n)+g*u3(n+ 1 )-a u2(n)=(-c*u1 (n-1 )+v*u1 (n)+bn)/w Generate solutions on each time level. Set ic=1 for successful convergence and 0 otherwise. begin iterative process.

do 17i=1.n if(abs(u2(i)-u(i))-eps)17.17.18 ic=O continue do 19i=1.n u(i)=u2(i) if(ic.ne.1) goto 12 do 20 i=1.n u3(i)=u2(i) Terminates the timing. Terminates the child processors creatrd by the c$doacross. call 310ck_time(time2) call m_kill..J)rocs print* ,'The time = ·.(time2-time1 )/1 00.0 do 21 jw=i1.nt1.i1 ifUt.eq.jw)then write(* .23)jt2.t1

301

Page 335: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

160 161 23 162 163 22 164 165 166 167 168 21 169 170 12 171 172 24 173 * 174 11 175 176

write(*,22)(u(i),i=1,n) format(/,'The Age iterative solutions at time level

* nt=',i31 'time t c' ,f1 0.6) format{2x,1 Of1 0.6)

goto 21 els goto 21 endif continue goto 11 continue write(* ,24 )kmax,jt2 format(,Method fails to converge in ',i4, 'iteration',!

'at time level nt =' ,i3) continu stop end

302

Page 336: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

1 c 2 c 3 c 4 c 5 c 6 c 7 c 8 c 9 c 10 c 11 c 12 c 13 c 14 c 15 c 16 c 17 c 18 c 19 c 20 c 21 22 23 24 25 c 26 c 27 c 28 c 29 30 c 31 32 33 34 35 36 37 38 39 40

*

Program 5.2 One-dimensional heat equation solved using the parallel AGE algorithm with the first strategy. AGE with D'Yakonov splitting. dU/dt=d**2U/dx**2 , 0<x<1, ,O<t<nt. Boundary conditions, U(o,t)=O ,U(7t,t)=O ,O<t<nt. Initial condition, U(x,O)=sinx O<x<7t. n: The order of the syste a: The subdiagonal of A c: The superdiagonal of A d: The diagonal of A hd: Half the diagonal of A r: The acceleration parameter tl: Maximum value in the t-direction k: Increments in the time

sx: Maximum value in the x-direction h: Increments in the x-axis nt: Number of time level parameter Gmax= 100) dimension u(0:jmax),u1 (0:jmax),u2(0:jmax),u3(0:jmax),

u4(0:jmax),erGmax),reGmax) integer*4 nprocs,status,time1,time2 nprocs: The number of processors for each run status: This variable is used to store the return value of the

library routine that actually sets the number of processors.

data pi,eps,kmaxl3.1415926,0.000001 ,3001

write(*, *)'Number of procs,n' read(* :)nnprocs,n tl=0.1 k=0.005 sx=pi r=0.5 nt=21 i1=4 i2=1 h=pi/(n+ 1)

303

Page 337: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

41 42 43 44 c 45 c 46 47 48 49

.50 51 52 53 54 55 56 57 10 58 c 59 c 60 61 62 63 64 65 66 67

.68 c 69 70 71 72 73 74 75 14 76 77 c 78 c 79

hh=h·h nt1 =nt-1 rl=klhh Input data

a=-rl c=a hd=1.0+r1 w=hd+r v=r-hd aa=a·a cc=c·c w=v·v det=w·w-a·c do10i=1,n u(i)=sin(i·h) u4(i)=u(i)

a1 =-(v·w·c+a·cc) b1 =(w·w+v·a·c) c1 =-(v·w·a+w·a) d1 =(aa'w+v'aa) a2=(v·cc+w·cc) b2=-(w·c+v·w·c) c2=(v·a·c+w·w) d2=-(aa·c+v·w·a)

do 12 jt=1,nt1 t1 =jt*k jt2=jt+ 1 do 13 k=1 ,kmax ic=1 do 14 i=1 ,n u3(i)=exp(-t1 )*sin(i·h) ••••• First Sweep .....

========== u(k+ 1/2)=(G1 +ri)<-1 >·[(ri-G2)·u(k)+bl b11 =(2.0-2.0·rl)·u4(1 )+rI·u4(2)

304

Page 338: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

80 81 c 82c 83 c 84 c 85 86 c 87

c

u1 (1 )=(w·u(1 )-v·a·u(2)+b11 )/w Set the number of processors. Makes the beginning of the timed section of code, which includes everything except 1/0.

status=m_setJlrocs(noJlrocs)

call_clock_time(time1 ) 88 89 90 91 92

c$doacross share(u4,rl,u1,u,a1,b1,c1,d1,a2,b2,c2,d2,det,w,c,a), c$& local(b11,b22,r1,r2)

93 94 95 96 97 15 98 99 100

101 102 103 104 105 106 c

do 15 i=2,n-3,2 . b11 =rl·(u4(i-1 )+u4(i+ 1 ))+(2.0-2.0·rl)·u4(i)

b22=rl·(u4(i)+u4(i+2))+(2.0-2.0·rl)·u4(i+ 1) . r1 =(w·b11-a·b22) r2=-(c·b11-w·b22) u1 (i)=(a1·u(i-1 )+b1·u(i)+c1·u(i+ 1 )+d1·u(i+2)+r1 )/det u1 (i+ 1 )=(a2·u(i-1 )+b2·u(i)+c2·u(i+ 1 )+d2·u(i+2)+r2)/det b11 =rl·(u4(n-2)+u4(n))+(2.0-2.0·rl)*u4(n-1) b22=rl·(u4(n-1 )+(2.0-2.0·rl)·u4(n) r1 =(w·b11-a·b22)

r2=(c·b11-w·b22) u1 (n-1 )=(a1·u(n-2)+b1·u(n-1 )+c1·u(n)+r1 )/det u1 (n)=(a2·u(n-2)+b2·u(n-1 )+c2·u(n)+r2)/det ••••• Second Sweep •••••

=============

u(k+ 1 )=(G2+ri)<-1 >··u(k+ 1/2) 107 c 108 109 110

u2(1 )=(w·u1 (1 )-a·u1 (2))/det u2(2)=(-c·u1 (1 )+w·u1 (2))/det

c$doacross share(u2,u1,a,c,w,det),local(i) do 16 i=3,n-1,2 111 u2(i)=(w·u1 (i)-a·u1 (i+ 1 ))/det

16 u2(i+ 1 )=(-c·u1 (i)+w·u1 (i+ 1 ))/det u2(n)=(u1 (n))/w

112 113

.114 115 c 116 c 117 c 118 c 119 c

Generate solutions on each time level. Set ic=1 for successful convergence and 0 otherwise. Begin iterative process.

305

Page 339: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

306

120 d017 i=1,n 121 if(abs(u2(i)-u(i))-eps)17, 17,18 122 18 ic=O 123 17 continue 124 do 19 i=1,n 125 19 u(i)=u2(i) 126 do 20 i=1,n 127 20 er(i)=abs(u(i)-u3(i)) 128 do 21 i=1,n 129 21 re(i)=abs( (u3(i)-u(i) )/u3(i)) 130 if(ic.ne.1) goto 13 131 do 22 i=1,n 132 22 u4(i)=u2(i) 133 c Terminates the timing .. 134 c Terminates the child processors created by 135 c the c$doacross. 136 call _clock_time(time2) 137 call m_kill-procs 138 print· ,'The time =',(time2-time1 )/1 00.0 139 do 23 jw=i1 ,nt1 ,i1 140 ifUt.eq.jw) then 141 write(· ,24) jt2,t1 142 write(· ,25) I 143 write(· ,26) 144 write(· ,27) 145 write(· ,28)( u (i), i= 1 ,n, ic2) 146 write(· ,29) 147 write(· ,27) 148 write(· ,28)( u3(i) ,i= 1 ,n, ic2) 149 write(· ,30) 150 write(· ,27) 151 write(·,28)(er(i),i=1,n,ic2) 152 write(· ,31 ) 153 write(· ,27) 154 write(·,28)(re(i),i=1,n,ic2) 155 24 format(/,'AGE iterative solution at time level nt = ',i31 156 • 'Time t = ',f10.6) 157 25 format(, Method converges with I = ',i4, ' iterations ',I) 158 26 format(, The numerical solutions is ') 159 27 format(, nlllltltlllllllllnllllllftllllllltllll"""""l1nn",,"""n ')

Page 340: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

160 28 161 29 162 30 163 31

164 165 166 167 168 23 169 170 13

format(1 Of1 0.6) formate' The exact solution is formate' The absolute error is format(f The relative error is

goto 23 else goto 23 endif continue goto 12 continue write(" ,32) kmalS,jt2

') ') ')

171 172 173

32 format(, Method fails to converge in ',i4, ' At time level nt = ',i4) "

174 175 176

177

c 12continu

stop end

307

iterations' ,I

Page 341: Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS