Guest Editor: Philip J. Hatcher University of New Hampshire Dept. of Computer Science College of Englneerlng and Physlcal Sciences KIngsbury Hall 33 College Rd Durham, NH 03824-3591 Internet: pjhQcs.unh.edu

Fall 1994

The Impact of H igh Performance Fortran Programming tools for commercial parallel sys- tems tend to be both vendor- and hardware- specific. This lack of portable tools has severely limited the growth of parallel computing. Poten- tial customers are reluctant to compound the already high r isks of using leading-edge hardware by using software that cannot be easily trans- ferred to future systems as the hardware tech- nologv inevitably and rapidly evolves.

Also frightening to potential customers is the perception that programming a parallel computer is considerably more difficult, and therefore more expensive, than programming a conventional single-processor system. This perception is caused largely by the inherent difficulty of explic- itly multithreaded programming models, and the failure of automatic parallelization efforts.

Multithreaded execution requires the pro- grammer to coordinate multiple, concurrently executing, independent instruction streams. The difficulty of reasoning about concurrent inde- pendent threads leads to subtle bugs involving timing issues, such as race conditions and dead- lock. The introduction of time as a significant correctness factor, as opposed to simply an efl- ciency issue, makes the conceptual jump from serial programming to parallel programming a large one indeed for most programmers.

The goal of compiling serial programs for effective parallel execution remains elusive. Many problems require a new algorithm to be solved efficiently in parallel, and it is unreasonable to expect a compiler to perform algorithm devel- opment. But even when a serial program can be the basis for an effective parallel implementation, current compilers are often unable to derive that implementation. This is particularly true for sys- tems with distributed memory, for which com- pilers not only have to locate parallelism but also have to determine effective data layout strategies.

Recently there has been growing acceptance of at least a short-term solution to these difficul- ties: tools based on the data-parallel program- ming model. The data-parallel model is concep- tually based on single-threaded execution, hut it provides parallelism by supplying constructs that apply operations to data aggregates. For exam

ple, a data-parallel language allows a program- mer to specify vector addition as a single opera- tion that takes two vectors operands and produces a vector result. Conceptually the addition is con- currently executed elementwise across the vec- tors, with the constraint that all input operands are fetched before any output operand is stored.

The synchronous nature of data-parallel pro- gramming, which was motivated by SIMD (sin- gle instruction, multiple data) hardware, elimi- nates the possibility of bugs arising from time issues. In addition, since the data-parallel model is single-threaded, associated tools such as debug- gers can have a similar feel to those provided for serial platforms. This makes the conceptual jump from serial programming to data-parallel pro- gramming a small one for most programmers.

Also supporting the acceptance of the data- parallel model is the realization that many prob- lems that need high-performance implementa- tions are amenable to data-parallel solutions. These are often those problems whose serial solutions can be the basis of an effective parallel program anyway, but recasting the serial solution in a data-parallel notation may circumvent the limitations of current automatic parallelizers.

In fact, the compiler technology exists today to implement data-parallel programs across the variety of both serial and parallel architectures. (Data parallelism at the source code level can he very useful on high-performance serial architec- tures that are increasingly built upon internal par- allelism, such as multiple or pipelined functional units.) Consequently, there are high hopes that portable parallel programming tools hased on the data-parallel model will soon be widely available for commercial systems.

Acting on these trends, the High Performance Fortran Forum was organized in 1992 to develop a quasistandard for extending Fortran 90 to better support computationally intensive applications across a wide variety of high-performance archi- techireS. With participants from industry, acade- mia, and government laboratories, the forum con- cluded its initial work with the release of a specification for High Perfomlance Foman in Jan- uary 1993. HPF builds on the data-parallel con-

13

r------ Other information The High Performance Fortran Ian- guage specification is available elec- tronically via anonymous ftp from the public/hpff directory at titan.cs.rice.edu, or from Theresa Chapman, CITI/CRPC, Box 1892, Rice University, Houston, TX 77251. The specification was also reprinted in Scientific Computing, Vol. 2, No. 1. An excellent introduction to the language by David Loveman appeared in the February 1993 issue of IEEE Parallel & Distributed Tech- noloa. Finally, The High Pe~fomzance Fo7t7-an Handbook by Charles Koel- be1 et al. (MIT Press, 1994) thor- oughly covers the language.

I-m-~-p-

structs in Fortran 90, most notably the fea- tures that allow operations on entire arrays, by providing additional data-parallel con- structs (such as the FORALL statement) and by providing directives to let programmers specify how data should be distributed among the processors of a parallel machine.

The broad participation in the develop- ment of the HPF specification engendered expectations that HPF would soon be commercially available on a wide variety of hardware platforms. This still seems to be the case - as of April 1994 there were six announced commercial HPF products and 11 announced commercial HPF efforts, according to HPF Forum statis- tics. Some ofthese compilers are expected to become available during 1994.

What will be the short-term impact of these compilers? Will they generate effi- cient enough code that programmers will give up vendor- and hardware-specific tools? Will they effectively support a broad-enough range of applications? Will programmers adopt a set of Fortran exten- sions that are not formally standardized? This special issue assesses several HPF compiler projects through a series of short reports written by the vendors themselves. These reports were solicited by a general request posted to the HPF Forum elec- tronic mailing list: hpff@cs.rice.edu; for more information about the list, send a message to hpff-request@cs.rice.edu.

This issue also explores the long-term effects of HPF. Despite the broad partic- ipation in the HPF process, there remains considerable controversy regarding the language specification.

14

DID THE HPF SPECIFICATION GO FAR ENOUGH? Barbara Chapman, Piyush Mehrotra, and Hans Zima have been outspoken critics of HPF. Their article, “Extending HPF for Advanced Data Parallel Applications,” explores shortcomings they perceive in HPF’s ability to support applications oper- ating on “irregular” data structures. Such applications include adaptive multigrid codes, sweeps over unstructured meshes, and particle-in-cell codes.

1s HPF’s FOCUS ON DATA PARALLELISM TOO RESTRICTIVE? Many problems exhibit elements of task or control parallelism. A key example is a multi- disciplinary simulation that incorporates multiple models, such as the coupling of an atmosphere model with an ocean circula- tion model. Each submodel may have a nat- ural data-parallel implementation, but it is unwieldy or inefficient to shoehorn the two tnodels into a single data-parallel program. Ian Foster’s article, “Task Parallelism and High-Performance Languages,” provides the motivation for integrated support of both data and task parallelism, and dis- cusses the potential of using the Fortran 1M extensions in the context of HPF.

The article by Thomas Gross, David O’Hallaron, andjaspal Subhlok, “Task Par- allelism in a High Performance Fortran Framework,” describes how the authors extended HPF with additional directives that describe task parallelism, and reports their experience in using the system to solve a series of problems in a pipelined manner.

How CAN WE WRITE EFFICIENT HPF PROGRAMS? HPF’s emphasis on data-parallel program- ming greatly enhances the programmer’s ability to write a corvect program, but it does little to help write an eficient program. The programmer must still grapple with issues such as load balance, communication and synchronization costs, and scalability. In fact, the HPF strategy of building on stan- dard Fortran 90 arrays, and in particular of not adding a special construct to access “nonlocal” data, severely limits the trans- parency of HPF programs. For example, a programmer cannot tell by inspecting an

individual statement whether that statement will cause interprocessor data movement on a distributed-memory machine (and there- fore whether it will be relatively expensive). Instead, the programmer must scan the pro- gram to determine information about the data distribution of the operands, and per- haps analyze the reaching definitions of the index expressions of the operands.

Therefore, HPF by itself is most likely inadequate for guiding programmers in the writing of efficient programs. HPF compilers must be augmented with asso- ciated tools for performance prediction, evaluation, and tuning. The article by V&ram Adve and his colleagues, “Require- ments for Data-Parallel Programming Environments,” provides their vision of the necessary HPF tool set.

How IS HPF RELATED TO AUTOMATIC PARALLELIZATION? Will HPF preclude the need for automatic parallelization? Will the development of effective automatic parallelization tech- nology eliminate the need for HPF? Or do these two approaches complement each other? The article by William Blume and his colleagues, “Automatic Detection of Parallelism: A Grand Challenge for High- Performance Computing,” touches on these issues and provides a status report on automatic pardllelization.

High Performance Fortran is an important first step toward portable, standard pro- gramming tools for high-performance computers. The HPF Forum recognizes the need to take further steps once more experience - and a consensus for those next steps-has been obtained. The forum has reconvened in 1994 to provide correc- tions and interpretations to the initial HPF specification, to encourage implementa- tions, and to consider support for addi- tional features. In particular the forum plans to study features for supporting irreg- ular data, parallel I/O, and task parallelism.

Philip J. Hatcher is an associate professor of computer science at the University of New Hampshire, and coauthor (with Michael J. Quinn) of Data-Parallel Prop-l-amnmiizy 1712 !WMD Computw~ (MIT Press, 1691). He“received a PhD in comwter science from the Illinois Insti- tute of Tech’no1og.y in 1085.

IEEE Parallel & Distributed Technology