Denotational Semantics of aPara-Functional Programming Language

Paul Hudak

July 1986

Yale UniversityDepartment of Computer Science

Box 2158 Yale StationNew Haven, CT 06520Arpanet: hudak@yale

Abstract

A para-functional programming language is a functional language that has been ex-tended with special annotations that provide an extra degree of control over parallelevaluation. Of most interest are annotations that allow one to express the dynamicmapping of a program onto a known multiprocessor topology. Since it is quite de-sirable to provide a precise semantics for any programming language, in this papera denotational semantics is given for a simple para-functional programming languagewith mapping annotations. A precise meaning is given not only to the normal func-tional behavior of the program (i.e., the answer), but also to the operational notionof where (i.e., on what processor) expressions are evaluated. The latter semantics isaccomplished through an abstract entity called an execution tree.

This research was supported in part by the National Science Foundation under GrantsDCR-8403304 and DCR-8451415, and the Department of Energy under Grant

DE-FG02-86ER25012.

0

Denotational Semantics of aPara-Functional Programming Language

Paul Hudak

Yale UniversityDepartment of Computer Science

1 Introduction

An often-heralded advantage of functional languages is that parallelism in a functional pro-gram is implicit; it is manifested solely through data dependencies and the semantics ofprimitive operators. This is in contrast to more conventional languages, where explicit con-structs are typically used to invoke, synchronize, and in general coordinate the concurrentactivities. Many of the earlier functional languages were in fact developed simultaneouslywith work on highly-parallel dataflow and reduction machines [1, 5, 6, 17], and such researchcontinues today (see [11, 12, 15] and the survey in [22]). In all of these efforts the parallelismin a program is detected by the system and allocated to processors automatically. Yet it isoften the case that a programmer knows a particularly good (and perhaps provably optimal)decomposition and distribution strategy for a program executing on a particular machine,but one can never expect a compiler to determine such optimal mappings in all cases. Al-though complete user unawareness is a noble goal, we cannot ignore the needs of those whowish to express the decomposition and mapping themselves.

To meet these needs a class of para-functional programming languages was introduced in[9, 10, 13] that provides this extra degree of expressiveness through annotations to the sourceprogram. The term “para-functional” is intended to convey not only parallel computation,but also the fact that the annotations provide an operational semantics that is truly “extra,”or “beyond,” the functional semantics of the language.

In this paper the formal semantics of a para-functional programming language is inves-tigated in depth. Providing a formal semantics for any programming language is highlydesirable – I believe it is essential – since it removes any ambiguities about the behavior ofprograms written in the language. Although several formal models are available for express-ing a program’s behavior, a denotational semantics was chosen here because of its abstractmathematical flavor and independence from a particular operational, or machine model.

Aside from being precise about the meaning of a para-functional program, the goal ofthis work is also to show that the semantics is relatively straightforward. This is importantif one believes (as I do) that a clean denotational semantics implies a clean programming

1

language. It is particularly important in the context of annotations, since such a linguisticdevice is often regarded as rather ad hoc. I hope to show that in fact annotations can begiven a very precise denotational semantics that is both simple and intuitive.

2 Preliminaries

To make the discussion as general as possible, a very simple para-functional programminglanguage is first defined having a small set of primitive operators that are left mostlyunspecified.1 In this way we may concentrate on the semantical issues rather than be boggeddown with syntax.

The para-functional language, which I will call PFL, is block-structured and lexicallyscoped. It’s abstract syntax is:

c ∈ Con, constants, including primitive functions.x, f ∈ V ar, variables.

b ∈ Blk, blocks, defined by:b ::= e whererec f1 = e1, . . . , fn = en

e ∈ Exp, expressions, defined by:e ::= c | x | b | λx.e | e1 e2 | e1 on e2

p ∈ Prog = Exp programs (i.e., just expressions).

PFL can be viewed as an unrestricted lambda calculus with constants, whose syntax allowsgiving names to expressions by writing them in a (mutually-recursive) whererec clause. Notethat PFL is a higher-order language – lambda expressions may be used in arbitrary contexts,as can the “named” functions fi. Function application is assumed to be left-associative (sothat programs look much like “curried” programs written in many conventional functionallanguages), but otherwise parentheses are used to disambiguate expressions. Expressions ofthe form “e1 on e2” will be explained in the next section.

The intent of PFL is to model normal-order reduction in the lambda calculus, and thiswill be formalized in the next section using denotational semantics. Arguments to functionsare evaluated “lazily,” as are the (mutually-recursive) equations in a whererec clause (thus

1The description of a much more complete and syntactically “sugared” para-functional language calledParAlfl is contained in [9, 10, 13].

2

their order is irrelevant). An example of a simple program in PFL is:

(map fac lst)whererec lst = (pair 2 (pair 3 (pair 4 nil))),

fac = λn. if (= n 0) 1 (∗ n (f (− n 1))),map = λf. λl. if (= l nil)

nil(pair (f (head l)) (map f (tail l)))

which returns the list (2, 6, 24). In this example it is assumed that if , =, ∗, −, pair, head,tail, nil and the integers are elements of Con. Note that instead of Lisp’s traditional cons,car, and cdr to construct and select from lists, PFL uses pair, head, and tail, respectively,to emphasize that they are lazy list operators.

3 Mapped Expressions: An Intuitive Description

The fundamental idea behind para-functional programming is quite simple. Consider a PFLexpression such as (+ e1 e2). The strict semantics of + allows the subexpressions e1 ande2 to be executed in parallel – this is an example of what is meant by saying that “theparallelism in a functional program is implicit.” But suppose now that we wish to expressprecisely where (i.e., on which processor) the subexpressions are to be evaluated; we maydo so quite simply by annotating the subexpressions with approriate mapping information.An expression annotated in this way is called a mapped expression, which in PFL has thefollowing form:

exp on proc

and intuitively declares that exp is to be computed on the processor identified by proc.The expression exp is the body of the mapped expression, and represents the value to whichthe overall expression will evaluate (and thus can be any valid PFL expression, includinganother mapped expression). The expression proc must evaluate to a processor id. Withoutloss of generality the processor ids, or pids, are assumed to be integers, and there is somepre-defined mapping from those integers to the physical processors they denote.

Returning now to the example, we may annotate the expression (+ e1 e2) as follows:

(+ (e1 on 0) (e2 on 1))

where 0 and 1 are processor ids. Of course, this static mapping is not very interesting. Itwould be nice, for example, if we were able to refer to a processor relative to the currentlyexecuting one. PFL allows this through the use of the reserved identifier self , which whenevaluated returns the pid of the currently executing processor. Using self we can now bemore creative. For example, suppose we have a linear array of processors that are numbered

3

consecutively; we may then rewrite the above expression as:

(+ (e1 on (left self)) (e2 on (right self)))whererec left = λpid. (− pid 1),

right = λpid. (+ pid 1)

which denotes the computation of the two subexpressions in parallel on the two neighboringprocessors (for simplicity we ignore boundary conditions), with the sum being computed onself .

But self does even more. Indeed, the most important aspect of the interaction betweenself and mapped expressions is that self is dynamically bound in function calls. Thus in:

( (+ ((f x) on pid1) ((f y) on pid2))whererec f = λa. (∗ a a) on self ) on pid3

x2 is computed on processor pid1, y2 on processor pid2, and the sum on processor pid3 (theannotation “... on self” in the definition of f is used only for emphasis, and is otherwisesuperfluous).

To see that it is desirable to dynamically bind self , consider that one may wish successiveinvocations of a recursive call to be executed on different processors – this cannot be expressedwith lexically bound annotations. For example, consider the following variation of the list-of-factorials program given earlier, again using a linear array of processors:

((map fac lst) on 0)whererec lst = (pair 2 (pair 3 (pair 4 nil))),

fac = λn. if (= n 0) 1 (∗ n (fac (− n 1))),map = λf. λl. if (= l nil)

nil(pair (f (head l)) ((map f (tail l)) on (right self))),

right = λpid. (+ pid 1)

Note that the recursive call to map is mapped onto the processor to the right of the currentone, and thus the elements 2, 6, and 24 in the result list are computed on processors 0, 1,and 2, respectively.

Para-functional programming languages have been shown to be adequate in express-ing a wide range of deterministic parallel algorithms in a concise and perspicuous manner[9, 10, 13]. Examples include several “divide-and-conquer” algorithms; simple yet inter-esting programs such as several variations of a parallel fibonacci generator; numerical al-gorithms such as matrix multiplication, matrix-vector product, solving linear systems ofblock-matrices, and Jacobi’s method for solving pde’s; and systems programs such as dis-tributed databases and resource management. Topologies considered include linear arrays,rings, trees, two-dimensional grids, tori, and hypercubes.2 It is beyond the scope of this

2Mapping annotations are not particularly useful in a shared memory machine (other than to denote taskgranularity), unless the machine is heterogeneous, in which case it might be desirable to map a particularclass of computations to a machine tailored for that class.

4

paper to discuss the para-functional programming style any further, although its advantagescan be summarized as follows:

• It is very flexible. The annotations are easily adapted to any functional language, andany network topology may be captured by the notation.

• The annotations are natural and concise. There are no special control constructs, nomessage-passing constructs, and in general no forms of “excess baggage” to express therather simple notion of “where to compute things.”

• With some minor constraints, if a para-functional program is stripped of its annota-tions, it is still a perfectly valid functional program. This means that it can be writtenand debugged on a uniprocessor that ignores the annotations, and then executed on aparallel system for increased performance. The ability to debug a program indepen-dently of the parallel machinery is invaluable.

4 Denotational Semantics for PFL

Thus far only an informal description of the operational semantics of PFL has been given, andin so doing many non-trivial details have been glossed over. For example: (1) In a mappedexpression, where is the pid expression evaluated? (2) Where is an unmapped expressionevaluated that appears at the top level of a program? (3) Just what are the precise bindingrules for the identifier self? In particular, self in a function body is evaluated when thefunction is applied, not when it is created, but what exactly does that mean, especially inthe context of curried functions? One approach to providing these details is to anticipateas many of the above questions as possible, and answer each of them in operational terms.This informal approach is unfortunately error-prone and susceptible to ambiguity. A betterapproach is to give a formal semantics that captures the desired operational properties, andthat is what is done here.

For the remainder of this paper the reader is assumed to be familiar with the standardnotational conventions of denotational semantics, such as outlined in [7] and summarizedas follows: Double brackets are used to surround syntactic objects, as in E [[exp]]. Squarebrackets are used for environment update, as in env[e/x]. Angle brackets are used for tupling,as in < e1, e2, e3 >. The notation λx y. exp is shorthand for λx. (λy. exp), and fix x. exp isshorthand for fix(λx. exp), where fix is the standard fixpoint operator. Domain equationsare assumed to define chain-complete partial orders; the bottom element in a domain D isdenoted ⊥D, and “d ∈ D = Exp” defines the domain (or set) D with “canonical” elementd. D1 + D2 shall generally refer to the coalesced sum of the domains D1 and D2 (thus thesum of two flat domains is also flat). For simplicity all domain/sub-domain coercions areomitted, since in all cases the context makes the meaning clear.

5

4.1 Standard Semantics

The following equations define the functional domains of PFL:

Bas = Int + Bool + Pid + · · · domain of basic values.D = Bas + (D → Pid → D) + {error} domain of denotable values.

Env = V ar → (D + {unbound}) domain of environments.Pid = Int domain of processor ids.

Int and Bool are the standard flat domains of integers and boolean values, respectively; Basmay contain other such primitive domains as well.

The standard semantics of PFL is captured by the functions:

Ep : Prog → DE : Exp → Env → Pid → DK : Con → D

Note that E takes not only an expression exp ∈ Exp and an environment env ∈ Env, butalso an argument cpid ∈ Pid that represents the “currently executing processor.” This isneeded in order to capture the dynamic binding of self , which cannot be done with thelexical environment env alone.

Ep and E are defined by:

Ep[[p]] = E [[p]] null env root pid

E [[c]] env cpid = K[[c]]E [[x]] env cpid = env[[x]]

E [[self ]] env cpid = cpidE [[λx.e]] env cpid = λd pid. E [[e]] env[d/x] pidE [[e1 e2]] env cpid = (E [[e1]] env cpid) (E [[e2]] env cpid) cpid

E [[e whererec f1 = e1,. . . ,fn = en]] env cpid = E [[e]] env′ cpid where

env′ = fix env′. env[ (E [[e1]] env′ cpid)/f1,. . . ,(E [[en]] env′ cpid)/fn ]

E [[e on pid]] env cpid = let newpid = E [[pid]] env cpidin if newpid = ⊥D then ⊥D

else E [[e]] env newpid

where null env = λx.unbound, and root pid ∈ Pid is the id of some “root processor” onwhich all computations begin (and presumably end).

Note the interaction between the equations for lambda abstraction and function applica-tion. In particular, the pid is bound dynamically in function calls; that is why the functional

6

subdomain of D is (D → Pid → D) rather than just (D → D) as in a more conventionalsemantics. In other words, a function is “invoked” on the processor on which it is applied,not the one on which it is created.

The function K gives meaning to constants (including primitive functions). As with Bas,for the most part the definition of K is left unspecified, but here are a few canonical partsof its definition:

K[[true]] = trueK[[if ]] = λpred p1 con p2 alt p3. if pred = true then con

else if pred = false then altelse error

K[[+]] = λe1 p1 e2 p2. if (e1 ∈ Int) and (e2 ∈ Int)then e1 + e2 else error

K[[pair]] = λ h p1 t p2 f p3. f h tK[[head]] = λ l p1. l (λ h t. h)K[[tail]] = λ l p1. l (λ h t. t)

The “pi” arguments are the dynamically bound pids, which are ignored in this context. Notethat meaning is given to the lazy list operators through the use of higher-order functions.

4.2 Consistency and Termination Properties of Mapped Expres-

sions

The denotational semantics given thus far is a relatively standard one for functional lan-guages; the only thing new is the treatment of the mapping annotations. Indeed, my purposein providing the semantics is to make the meaning of the annotations precise, and to showthat their effect on the standard semantics is minimal. In this section two theorems arestated that capture this innocuous behavior; the proofs are relegated to appendices.

Consistency. The first theorem essentially states if self is restricted to pid expressions,then adding annotations can only weaken (in the domain ordering) the answer returned bya program.

Theorem 1 (Consistency): Let p be any PFL program in which the identifier self appearsonly in pid expressions, and let p′ be the same as p but with all annotations removed (i.e.,each occurrence of a mapped expression of form “exp on pid” is replaced with the body exp).Then (Ep p) � (Ep p′).

Proof: See Appendix 1.

The reason for the syntactic constraint on self is that once all mapping annotationsare removed from a program, all remaining occurrences of self will have the same value –namely, root pid. Thus removing the annotations might change the value of the program. For

7

the simplest example of this, suppose root pid = 0 and consider the programs “self on 1”(whose value is 1) and “self” (whose value is 0).

The inequality � cannot be strengthened without placing further constraints on theprogram, for two reasons. First of all, it is possible that adding annotations may cause aprogram to diverge. Perhaps the simplest example of this is the program p = [[1 on (f 0)whererec f = λx. (f x)]]. Such termination properties of pid expressions are addressed later.

The second reason is that D is a non-flat domain, and thus conventional non-termination(i.e., ⊥) is only one form of a weaker result. For example, consider the programs p =[[pair 1 (1 on ⊥)]] and p′ = [[pair 1 1]], where pair is the lazy list constructor defined in thelast section. The program p evaluates to the pair 〈1,⊥〉, and p′ evaluates to the pair 〈1, 1〉.Thus (Ep p) � (Ep p′), even though p “terminates.”

Of course, adding annotations cannot cause a non-terminating program to terminate:

Corollary 1a: Given p and p′ as described in Theorem 1, (Ep p′ = ⊥) ⇒ (Ep p = ⊥).

And furthermore, if p evaluates to a proper (i.e., non-⊥) element in D’s flat subdomainBas, then an equivalence property can be stated:

Corollary 1b: Given p and p′ as described in Theorem 1, (Ep p = ⊥) ∧ (Ep p ∈ Bas) ⇒(Ep p = Ep p′).

Both of these corollaries follow directly from Theorem 1.

Termination. As mentioned earlier, annotations may cause a program to diverge if oneof the pid expressions diverges. In other words, if the system diverges when determining onwhich processor to execute the body of a mapped expression, then it will never get around tocomputing the value of that expression. The simple example p = [[1 on (f 0) whererec f =λx. (f x)]] was given earlier, but a more subtle example is p = [[(f 0) whererec f =λx. 1 on (f x)]], in which the non-termination is due to the recursive call to f in the pidexpression itself.

I wish to make the following intuitive claim: If [[exp]] is an expression that is evaluatedin environment env during program execution, then changing [[exp]] to [[exp on pid]] cannotchange the meaning of the program as long as [[pid]] (evaluated in environment env) termi-nates. Unfortunately, the informal nature of this claim introduces ambiguities, since it is notclear what “is evaluated . . . during program execution” means. Indeed, one could imaginethat an “eager” implementation on a parallel machine might evaluate many things that a“lazy” implementation on a sequential machine would not.

To make this notion precise, let us define “is evaluated” to refer to what must be evaluatedto fully define the answer. Formally, let G be the functional describing E ; i.e., E = fix G(the precise definition of G is easily derived from the equations defining E given in Section

8

4.1). Next assume that all expressions in a program are unique,3 and define G′ by:

G′ E exp env cpid = if (exp = exp′) ∧ (env = env′) ∧ (cpid = cpid′)then ⊥else G E exp env cpid

and let E ′ = fix G′. Thus E ′ is just like E except at point 〈exp′, env′, cpid′〉, where it returnsthe value ⊥. Note that the full environment in which an expression is evaluated is a pair,the lexical environment env and the current processor id pid.

Definition: A program p is said to depend on the value of exp′ in the environment pair〈env′, cpid′〉 iff (E ′ p null env root pid) = (E p null env root pid).

In other words, if we can change the behavior of a program by causing exp′ in environ-ment 〈env′, cpid′〉 to diverge, then it must be the case that p depends on that evaluation.4

This definition is subtle when a program evaluates to a functional value; for example, theprogram p = [[λx.x]] depends on [[x]] in all environment pairs 〈null env[d/x], root pid〉,where d is any proper element of D. This is because p evaluates to the function λd pid.E [[x]] null env[d/x] root pid.

Definition: Let exp = [[e on pid]] be any mapped expression in program p, and let p′ be thesame as p except that exp is replaced by [[e]] (i.e., the annotation is removed). Then exp issaid to be safe in program p iff for every environment pair 〈env, cpid〉 in which p′ dependson [[e]], E [[pid]] env cpid = ⊥.

We can now strengthen the result of Theorem 1:

Theorem 2 (Equivalence): Let p and p′ be two PFL programs as described in Theorem1. With the additional constraint that all mapped expressions in p are safe, (Ep p) = (Ep p′).

Proof: See Appendix 2.

Summary. To summarize, Theorems 1 and 2 essentially state the conditions under whichit is safe to add mapping annotations to a PFL program. Although Theorem 2 subsumesTheorem 1, it is stated separately so that the effects of a completely syntactic constraint,namely limiting self so as to only appear in pid expressions, can be isolated from the effectsof an untestable constraint, namely the termination property of a pid expression.

Although neither constraint is severe,5 there are practical reasons for occasionally wantingto violate the first one, i.e., for wanting to use the value of self in other than a pid expression.The most typical situation where this arises is in a non-isotropic topology where certainprocessors form a “boundary” for the network (for example, the leaf processors in a tree, or

3This assumption could be avoided if we attached unique labels to expressions, as in [?], but assuminguniqueness is a more convenient solution for oue purposes here.

4This can be thought of as a generalized notion of “strictness” [14].5In practice pid expressions are almost always very simple expressions, such as self + 1 or 2 ∗ self .

9

the edge processors in a mesh). There exist distributed algorithms whose behavior at suchboundaries is different from their behavior at internal nodes. To express this, one needs toknow when execution is occurring at the boundary of the network, which can be convenientlydetermined by analyzing the value of self . An example of this is given in [13].

One final note: I have made no mention of the possibility of a pid expression returningan element not in Pid, such as error. Such possibilities were ignored since the theorems holddespite them; i.e., the only important issue was whether or not the pid expression terminated.A more complete semantics might specify on which processor a mapped expression wasevaluated whose pid expression evaluated to an element not in Pid (for example, it couldinherit the enclosing pid). I have omitted such detail primarily for simplicity.

5 Execution Tree Semantics

In the last section the “standard” denotational semantics of PFL was given, i.e., the valuereturned by a PFL program. I now wish to provide a denotational semantics for a non-standard aspect of a PFL program, namely, the notion of “where” (i.e., on which processor)each subexpression will be evaluated. This is done by associating with each expression expan execution tree that reflects the evaluation history of exp. Intuitively, the root of exp’sexecution tree t is labelled with the processor on which exp will be evaluated, and eachimmediate subtree of t is the execution tree of each immediate subexpression of exp.

However, since a PFL expression may evaluate to a function under the standard se-mantics, and that function may be passed arbitrarily to/from other functions, it is alsonecessary to associate with each expression a special object (also a function) that when in-voked appropriately returns an execution tree corresponding to the body of the function;i.e., corresponding to the tree that would result when the function is applied. This functionis called an expression’s abstracted behavior. The combination of an expression’s executiontree and its abstracted behavior is called its overall behavior. Behaviors are defined formallyby:

Beh = Etree × AbsBeh domain of behaviors.Etree = Pid + (Pid × Etree) + (Pid × Etree × Etree) domain of execution trees.

AbsBeh = Beh → D → Pid → Beh domain of abstracted behaviors.

Note that an execution tree may have infinite depth, corresponding to a non-terminatingevaluation, but is always finitely branching, reflecting the fact that an expression only has afinite number of (indeed, at most two) subexpressions. The notations pid : (), pid : (t1), andpid : (t1, t2) denote instances of leaf, unary, and binary execution trees, respectively, whoseroots are labelled pid and whose children are the sub-trees ti.

For a behavior b (∈ Beh), the notations bt (∈ Etree) and bf (∈ AbsBeh) denote the twosubcomponents of b (traditionally the notations b↓1 and b↓2 are used). Intuitively, bf b′ d p

10

returns the behavior that would result from applying, on processor p, the expression expto an argument whose value is d and whose behavior is b′. This will become much clearershortly.

5.1 Formal Execution Tree Semantics for PFL

First the domain of behavior environments is defined by:

Benv = V ar → (Beh + {unbound})

and a special error element err ∈ AbsBeh is defined as:

err = λb d p. < p : (), err >

err is the abstracted behavior to be associated, for example, with an integer constant, sinceit is an error to apply such a constant. Note that the abstracted behavior that results fromapplying such a constant is still err, as it should be, since it is also an error to apply such aconstant to more than one argument; that is why err is defined recursively.

Three semantic functions, Bp, B, and KB, are now introduced that capture the behaviorsof programs, expressions, and constants, respectively:

Bp : Prog → BehB : Exp → Benv → Env → Pid → BehKB : Con → AbsBeh

Bp is defined in terms of B:

Bp[[prog]] = B[[prog]] null benv null env root pid

where null env and root pid are as described earlier, and null benv is the empty behaviorenvironment λx.unbound.

Intuitively, B[[exp]] be e cp denotes the behavior of the PFL expression exp given thebehavior environment be, the standard environment e, and the currently executing processor

11

cp. B is defined by:

B[[c]] be e cp = < cp : (),KB[[c]] >B[[x]] be e cp = let b = be[[x]]

in < cp : (bt), bf >B[[self ]] be e cp = < cp : (), err >

B[[λx.exp]] be e cp = < cp : (), λb d p. B[[exp]] be[b/x] e[d/x] p >B[[e1 e2]] be e cp = let b1 = B[[e1]] be e cp,

b2 = B[[e2]] be e cp,b3 = b1f b2 (E [[e2]] e cp) cp

in < cp : (b1t, b3t), b3f >B[[e whererec f1 = e1,

. . . ,fn = en]] be e cp = B[[e]] be′ e′ cp where

be′ = fix be′. be[ (B[[e1]] be′ e′ cp)/f1,. . .(B[[en]] be′ e′ cp)/fn ]

e′ = fix e′. e[ (E [[e1]] e′ cp)/f1,. . .(E [[en]] e′ cp)/fn ]

B[[exp on pid]] be e cp = let bpid = B[[pid]] be e cp,dpid = E [[pid]] e cp,bexp = B[[exp]] be e dpid

in < cp : (bpidt, bexpt), bexpf >

The first equation simply states that constants are evaluated on the currently executingprocessor cp, and their abstracted behaviors are determined by KB (which will be definedshortly). The second equation reflects the fact that the value of an identifier is “moved” to thecurrent processor cp, but that it is evaluated on the processor on which it was defined, whichis information contained in the behavior environment be. The third equation essentiallydeclares that self is computed on processor self (since the value of self is cp!), and that itis an error to apply self as a function.

The next two equations treat lambda abstraction and function application in turn. Notethat the execution tree for an application contains one subtree for the evaluation of thefunction, and one for the evaluation of the body once the function is called; the executiontree for the argument (b2) will get “spliced” into the body’s execution tree (b3) whenever thecorresponding formal parameter is referenced. Also note how the dynamic binding of selfis accomplished by passing cp to the abstracted behavior of e1 at the time e1 is applied.

The next equation gives the execution tree for a whererec clause; here the two envi-ronments be and e are updated with the execution trees and functional values, respectively,corresponding to the fi. The last equation defines the behavior of a mapped expression:Clearly exp in the expression “exp on pid” should be evaluated on processor pid; however,that requires first computing the value of pid, which is done on processor cp.

12

The remaining, and in many ways crucial, portion of the execution tree semantics is thedefinition of KB, which defines the abstracted behavior of constants. The definition of KB isgiven piecemeal below:

KB[[true]] = err

which reflects the fact that simple constants such as boolean values cannot be applied.

KB[[if ]] = λ bp dp pp. < pp : (bpt),if dp = true then λ bc dc pc. < pc : (bct),

λ ba da pa. < pa : (), bcf >>else if dp = false then λ bc dc pc. < pc : (),

λ ba da pa. < pa : (bat), baf >>else err >

To understand this definition for if , one must realize that in PFL’s “curried” syntax a con-ditional takes the form if pred con alt, and thus each partial application may be annotated,as in:

(((((if pred) on pp) con) on pc) alt) on pa

That is why KB[[if ]] is an abstract function “three levels deep,” each level taking the map-ping parameters for each partial application. This behavior can be described as follows:The predicate pred is evaluated on processor pp; if it is true, then the consequent con isevaluated on processor pc. Although the resulting function will then ignore the behavior ofthe alternate expression alt, it is nevertheless applied on processor pa, which is reflected bythe leaf node pa : (). The result of applying if to three arguments in this case is essentiallythe execution trees for the predicate and consequent together with the abstracted behaviorof the consequent, as expected. A description similar to the above applies to the case wherethe predicate is false. If the predicate is other than true or false, an error situation isdenoted, as in the standard semantics.

Next let’s consider a strict primitive operator:

KB[[+]] = λ b1 d1 p1. < p1 : (b1t),λ b2 d2 p2. < p2 : (b2t), err >>

After the description of if this behavior should be clear; the arguments are evaluated inturn, resulting in a behavior whose execution tree contains the trees for both arguments,showing as well which processor the applications occurred on. The err portion in the resultsimply indicates that the result of a numeric addition cannot be a function.

Appendix 3 contains definitions of KB for the lazy list operators pair, head, and tail.

13

6 An Example

In this section a detailed example of a small PFL expression and its execution tree will begiven. The expression exp is similar to one given earlier in Section 3:

( (+ ((f x) on 1) ((f y) on 2))whererec f = λa. (∗ a a) ) on 3

Let’s assume that evaluation of exp takes place on processor cp, in environments be ∈ Benvand e ∈ Env that have bindings for both x and y. In other words, we want to computeB[[exp]] be e cp. For exposition it is convenient to work “bottom-up,” first computing thebehavior for f , which we shall call f :

f = B[[λx. (∗ x x)]] be e 3= < 3 : (), λ b d p. B[[∗ x x]] be[b/x] e[d/x] p >= < 3 : (), λ b d p. let b1 = B[[∗ x]] be[b/x] e[d/x] p

b3 = b1f b d pin < p : (b1t, b3t), b3f >

but b1 = let b1′ = B[[∗]] be[b/x] e[d/x] pb3′ = b1′f b d p

in < p : (b1′t, b3′t), b3

′f >

so f = < 3 : (), λ b d p. < p : (p : (p : (), p : (bt)), p : (bt)), err >>

f , of course, will ultimately be applied to the behaviors for x and y, which we will call x andy, respectively. To simplify things that follow, let:

T1 = (ff x d 1)t = 1 : (1 : (1 : (), 1 : (xt)), 1 : (xt))

and T2 = (ff y d 2)t = 2 : (2 : (2 : (), 2 : (yt)), 2 : (yt))

Next let’s compute B[[(f x) on 1]] be[f /f ] e[.../f ] 3 where ... refers to the standard denotation(∈ D) of f . Calling this behavior B1, we have:

B1 = let bexp = B[[f x]] be[f /f ] e[.../f ] 1= < 1 : (1 : (3 : ()), T1), err >

in < 3 : (3 : (), bexpt), bexpf >= < 3 : (3 : (), 1 : (1 : (3 : ()), T1)), err >

A similar behavior results from B[[(f y) on 2]] be[f/f ] e[.../f ] 3, which we will call B2:

B2 = < 3 : (3 : (), 2 : (2 : (3 : ()), T2)), err >

14

Letting exp′ = (+ ((f x) on 1) ((f y) on 2)), and continuing bottom-up, we have:

B[[exp′]] be[f/f ] e[.../f ] 3

= let b1 = B[[+ ((f x) on 1)]] be[f /f ] e[.../f ] 3

b2 = B[[(f y) on 2]] be[f /f ] e[.../f ] 3b3 = b1f b2 (E [[(f y) on 2]] e[.../f ] 3) 3

in < 3 : (b1t, b3t), b3f >= < 3 : (3 : (3 : (), 3 : (B1t)), 3 : (B2t)), err >

We are now prepared to compute the behavior of the original expression exp:

B[[exp]] be e cp = let bpid = B[[3]] be e cp =< cp : (), err >dpid = E [[3]] be e cp = 3bexp = B[[exp′ whererec f = λa. (∗ a a)]] be e 3

in < cp : (bpidt, bexpt), bexpf >

but bexp = B[[exp′]] be[f /f ] e[.../f ] 3= < 3 : (3 : (3 : (), 3 : (B1t)), 3 : (B2t)), err >

so B[[exp]] be e cp = < cp : (cp : (), 3 : (3 : (3 : (), 3 : (B1t)), 3 : (B2t))), err >

The resulting execution tree is shown in Figure 1, where the parse tree for the originalexpression has been super-imposed to show the correspondance between the two.

7 Commentary

I have argued that the execution tree semantics makes precise a particular operational aspectof a PFL program. I feel this is important, since it removes any ambiguity about suchbehavior, and makes it clear to an implementor of a para-functional programming languagejust what the behavior should be. But there is another useful feature: the semantics allowsone to build an interpreter for a PFL program that returns the program’s execution treealong with its standard value. Such an interpreter can be viewed as a simulator of a parallelmachine, and can be very useful in helping one debug a PFL program, as well as verifying thatan implementation is faithful to the semantics. Such an interpreter in fact has been writtenin Alfl[8], a lazy functional language similar to SASL and lazy ML. Using a language suchas Alfl allows one to write the semantic equations almost verbatim from their denotationalform.6 An interesting future research project is to integrate the interpreter into a real-timegraphical environment that allows one to see the execution tree unfold dynamically.

Although in some sense the execution tree semantics seems complex, in fact it is hard toimagine a simpler semantics that formally captures the higher-order nature of the language

6However, efficiency concerns may drive one to an alternative implementation. Using such a language isat least an excellent semantic prototyping system.

15

Figure 1: Sample Execution Tree

16

and the fine granularity at which expressions may be annotated.7 It should be noted thatif primitive syntax were added to PFL to accommodate things like the conditional, arith-metic operators, and list constructors (for example, if...then...else, a + b, and a b), then thesemantics would be somewhat simplified since there would be no partial applications thatcould be annotated. On the other hand, the semantics of such constructs would have to beexpressed in B rather than in KB (thus cluttering up the top-level equations) and the resultwould be less general than that presented here.

It should also be pointed out that the standard environment e is needed by B in orderto make the execution tree semantics exact (for example, to determine the value of a pidexpression, and to determine which arm of a conditional gets evaluated). On the other hand,an abstract execution tree semantics would probably not require the standard environment,especially if the abstraction is intended to be compile-time computable. Such an abstractinterpretation could be very useful, since it might allow one to predict at compile-time theprocessor utilization patterns of a program. The methodology for doing this is beyond thescope of this paper, and is a topic of current research.

Finally, I wish to point out that the information provided by self is philosophicallyno different from providing the depth of the current execution stack, or the value of thecurrent program counter, or the register containing the value of a certain variable, or anyother arbitrary implementation-dependent parameter. From a semantic viewpoint, just asthe meaning of an expression is normally given as a function of a “current continuation”and “current environment,” the operational meaning that I am trying to convey is givenas a function of a “currently executing processor.” And just as in Scheme [4] one is givenaccess to the current continuation via calls to the primitive function call−with− current−continuation, I am providing access to the currently executing processor via the dynamicvariable self .

8 Related Work

The first use of annotations that I know of to control the behavior of recursion equationsappeared in a paper by Schwarz [19], which was motivated by earlier work on transforma-tional programming by Burstall and Darlington [2]. More recently several researchers haveused annotations in a parallel programming language. They include Shapiro’s “systolic pro-gramming” in Concurrent Prolog [20] (whose mapping semantics was derived from earlierwork on “turtle programs” in Logo [18]), Keller and Lindstrom’s work on distributing datastructures in a functional database environment [16], Burton’s annotations to the lambdacalculus to provide control over “lazy,” “eager,” and “parallel” execution [3], and Sridharan’s“semi-applicative” programming style to control evaluation order [21]. The work presentedhere, however, is the first that I know of giving a formal semantics to any kind of annotations.

7For example, an application [[e1 . . . en]] has 2n − 1 distinct objects that can be annotated – the nindividual subexpressions plus the n − 1 partial applications!

17

9 Acknowledgements

I would like to thank Lauren Smith at Yale (now at Los Alamos National Laboratory),Jonathan Young and Ben Goldberg at Yale, Bob Keller at the University of Utah (now atQuintus Computer), and Will Clinger at Tektronix, whose comments helped improve thisresearch. Also thanks to the four anonymous referees, in particular referee “B,” who forcedme to be precise about what it means to evaluate something in a denotational semantics.

Appendix 1: Proof of Theorem 1

Before proving Theorem 1, we first prove a lemma:

Lemma: Let [[exp]] be any PFL expression with no mapped expressions and no occurrencesof self . Then for any cpid1, cpid2 ∈ D, E [[exp]] env cpid1 = E [[exp]] env cpid2.

Proof: By a simple structural induction on [[exp]], where we note that cpid affects the valueof E [[exp]] env cpid in only two places:

1. As the value for self ; but this can’t happen since self does not occur in [[exp]].

2. As the value of a pid expression; but this can’t happen either since there are no mappedexpressions in [[exp]]. �

Proof of Theorem 1: We will prove a slightly more general result, namely: E [[exp]] �E [[exp′]], where [[exp]] and [[exp′]] are related in the same way that p and p′ are. We proceedby structural induction on [[exp]]. The atomic possibilities for [[exp]] are [[c]] and [[x]] (since byassumption self cannot appear in other than pid expressions). In these cases [[exp]] = [[exp′]],and the theorem holds trivially; thus the basis case is proved. Now consider the compositepossibilities for [[exp]] in turn:

1. [[exp]] = [[e1 e2]], [[exp′]] = [[e′1 e′2]]. By the induction hypothesis E [[ei]] � E [[e′i]], i = 1, 2.Then, because the functions are monotonic, ((E [[e1]] env cpid) (E [[e2]] env cpid)) �((E [[e′1]] env cpid)(E [[e′2]]env cpid)) for all env, cpid, and the theorem holds.

2. [[exp]] = [[λx.e]], [[exp′]] = [[λx.e′]]. By the induction hypothesis E [[e]] � E [[e′]]. Fromthis and again from monotonicity we conclude that (λd pid. E [[e]] env[d/x] pid) �(λd pid. E [[e′]] env[d/x] pid), and the theorem holds.

3. [[exp]] = [[e on pid]], [[exp′]] = [[e′]]. Starting with the induction hypothesis, we have:

E [[e]] � E [[e′]](λenv cpid. E [[e]] env cpid) � (λenv cpid. E [[e′]] env cpid) Eta conversion(λenv cpid. E [[e]] env newpid) � (λenv cpid. E [[e′]] env newpid)

where newpid = E [[pid]] env cpid Substitution(λenv cpid. E [[e]] env newpid) � (λenv cpid. E [[e′]] env cpid) By the LemmaE [[exp]] � E [[exp′]] By definition of E

18

4. [[exp]] = [[e whererec f1 = e1, ..., fn = en]], [[exp′]] = [[e′ whererec f1 = e′1, ..., fn =e′n]]. By the induction hypothesis E [[e]] � E [[e′]], and E [[ei]] � E [[e′i]], i = 1, ..., n. Nowlet:

newenv = fix newenv. env[ (E [[ei]] newenv cpid)/fi ]newenv′ = fix newenv′. env[ (E [[e′i]] newenv′ cpid)/fi ]

for some env and cpid. A simple fixpoint induction is sufficient to show that newenv �newenv′. From this we conclude that (E [[e]] newenv) � (E [[e′]] newenv′), and thetheorem holds. �

Appendix 2: Proof of Theorem 2

Proof of Theorem 2: As in the proof of Theorem 1, we prove a slightly more generalresult, namely, that if p depends on some expression exp in the environment pair 〈env, cpid〉,then E [[exp]] env cpid = E [[exp′]] env cpid, where exp and exp′ are related in the same waythat p and p′ are. Also as with Theorem 1, we proceed by structural induction on [[exp]].The atomic possibilities for [[exp]] are [[c]] and [[x]] (since by assumption self cannot appearin other than pid expressions). In such cases [[exp]] = [[exp′]], and the theorem holds trivially;thus the basis case is proved. Now consider the composite possibilities for [[exp]] in turn:

1. [[exp]] = [[e1 e2]], [[exp′]] = [[e′1 e′2]]. By the induction hypothesis E [[ei]] env cpid =E [[e′i]] env cpid, i = 1, 2 (part of this assumption is that exp depends on e2; if not,then the following argument still holds, since e2 cannot affect the value of exp). Thisimplies that ((E [[e1]] env cpid) (E [[e2]] env cpid)) = ((E [[e′1]] env cpid)(E [[e′2]]env cpid)),and the theorem holds.

2. [[exp]] = [[λx.e]], [[exp′]] = [[λx.e′]]. By definition E [[exp]] env cpid = λd pid. E [[e]] env[d/x] pid.Clearly exp depends on e in all environment pairs 〈env[d/x], pid〉 in which d is a properelement, and thus by the induction hypothesis E [[e]] env[d/x] pid = E [[e′]] env[d/x] pid,and the theorem holds.

3. [[exp]] = [[e on pid]], [[exp′]] = [[e′]]. By assumption E [[pid]] env cpid = ⊥, and we have:

E [[exp]] env cpid = E [[e]] env newpidwhere newpid = E [[pid]] env cpid By definition

E [[e′]] env cpid = E [[e′]] env newpid By the lemma

Clearly exp depends on e in environment 〈env, newpid〉, and thus by the inductionhypothesis E [[e]] env newpid = E [[e′]] env newpid, and the theorem holds.

4. [[exp]] = [[e whererec f1 = e1, ..., fn = en]], [[exp′]] = [[e′ whererec f1 = e′1, ..., fn =e′n]]. Let:

newenv = fix newenv. env[ (E [[ei]] newenv cpid)/fi ]newenv′ = fix newenv′. env[ (E [[e′i]] newenv′ cpid)/fi ]

19

It is clear that exp depends on e in environment 〈newenv, cpid〉, so by the inductionhypothesis E [[e]] newenv cpid = E [[e′]] newenv cpid. It remains to be shown thatE [[e′]] newenv cpid = E [[e′]] newenv′ cpid, which can be done using fixpoint induction onnewenv and newenv′, as follows: The equality holds trivially for newenv0 = newenv′

0 =⊥Env. Now suppose it holds for newenvj and newenv′

j ; then we wish to prove thevalidity of the equality:

E [[e′]] newenvj+1 cpid = E [[e′]] newenv′j+1 cpid

where newenvj+1 = env[ (E [[ei]] newenvj cpid)/fi ]newenv′

j+1 = env[ (E [[e′i]] newenv′j cpid)/fi ]

There are two possibilities for each ei:

(a) exp depends on ei in environment 〈newenvj, cpid〉, so by the structural inductionhypothesis E [[ei]] newenvj cpid = E [[e′i]] newenvj cpid. But then by the fixpointinduction hypothesis E [[e′i]] newenvj cpid = E [[e′i]] newenv′

j cpid. Thus with respectto ei the equality holds.

(b) exp does not depend on ei, and thus fi’s value in newenvj and newenv′j is irrele-

vant, and the equality still holds.

The equality thus holds for newenvj+1 and newenv′j+1, and by induction it holds at

the limits, newenv and newenv′. Thus the theorem holds. �

20

Appendix 3: Abstracted Behaviors for Lazy List Operators

KB[[pair]] = λ b1 d1 p1. < p1 : (),λ b2 d2 p2. < p2 : (),

λ b3 d3 p3.d3 < p1 : (b1t), b1f > < p2 : (b2t), b2f > >>

KB[[head]] = λ b d p. let b′ = bf err (λ h t. h) pin < p : (bt, b

′t), b

′f >

KB[[tail]] = λ b d p. let b′ = bf err (λ h t. t) pin < p : (bt, b

′t), b

′f >

In the equation for pair, p1 : () and p2 : () denote where the partial applications occur, butbecause of lazy evaluation they are leaf nodes, indicating that the elements aren’t actuallyevaluated. To understand the remainder of the semantics, recall the standard definition ofpair, head, and tail given in Section 4.1:

K[[pair]] = λ h t f. f h tK[[head]] = λ l. l (λ h t. h)K[[tail]] = λ l. l (λ h t. t)

Note in KB[[head]] how the function (λ h t. h) is passed to the abstracted behavior of the listwhere it is used to select the correct behavior, just as in the standard semantics it selectsthe correct element of the list; tail works similarly. Further note in KB[[pair]] that p1 : (b1t)indicates that the first element, once eventually selected, is computed on the processor onwhich it was originally mapped; similarly for the second element. Finally, note in KB[[head]]and KB[[tail]] that b′f ensures us that functions can be placed in lists and selected for lateruse.

21

References

[1] Arvind and V. Kathail. A multiple processor data flow machine that supports gen-eralized procedures. In Proc. 8th Annual Sym. Comp. Arch., pages 291–302, ACM –SIGARCH 9(3), May 1981.

[2] R.M. Burstall and J. Darlington. A transformation system for developing recursiveprograms. JACM, 24(1):44–67, 1977.

[3] F.W. Burton. Annotations to control parallelism and reduction order in the distributedevaluation of functional programs. ACM Trans. on Prog. Lang. and Sys., 6(2), April1984.

[4] Clinger, W. et al. The Revised Revised Report on Scheme, or An UnCommon Lisp. AIMemo 848, Massachusetts Institute of Technology, August 1985.

[5] A.L. Davis. The architecture and system method of DDM-1: a recursively-structureddata driven machine. In Proc. Fifth Annual Symposium on Computer Architecture,1978.

[6] J.B. Dennis and D.P. Misunas. A preliminary architecture for a basic data-flow proces-sor. In Proc. of the 2nd Annual Symposium on Computer Architecture, pages 126–132,ACM, IEEE, 1974.

[7] J.C. Gordon. The Denotational Description of Programming Languages. Springer-Verlag, New York, 1979.

[8] P. Hudak. ALFL Reference Manual and Programmer’s Guide. Research ReportYALEU/DCS/RR-322, Second Edition, Yale University, October 1984.

[9] P. Hudak. Exploring para-functional programming. Research Report YALEU/DCS/RR-467, Yale University, Department of Computer Science, April 1986.

[10] P. Hudak. Para-functional programming. Computer, 19(8):60–71, August 1986.

[11] P. Hudak and B. Goldberg. Distributed execution of functional programs using serialcombinators. In Proceedings of 1985 Int’l Conf. on Parallel Proc., pages 831–839, August1985. Also appeared in IEEE Trans. on Computers, Vol C-34, No. 10, October 1985,pp. 881-891.

[12] P. Hudak and B. Goldberg. Serial combinators: “optimal” grains of parallelism,pages 382–388. Springer-Verlag LNCS 201, September 1985.

[13] P. Hudak and L. Smith. Para-functional programming: a paradigm for programmingmultiprocessor systems. In 12th ACM Sym. on Prin. of Prog. Lang., pages 243–254,January 1986.

22

[14] P. Hudak and J. Young. Higher-order strictness analysis for untyped lambda calculus.In 12th ACM Sym. on Prin. of Prog. Lang., pages 97–109, January 1986.

[15] R.M. Keller and F.C.H. Lin. Simulated performance of a reduction-based multiproces-sor. IEEE Computer, 17(7):70–82, July 1984.

[16] R.M. Keller and G. Lindstrom. Approaching distributed database implementationsthrough functional programming concepts. In Int’l Conf. on Distributed Systems, May1985.

[17] R.M. Keller, G. Lindstrom, and S. Patil. A loosely-coupled applicative multi-processingsystem. In AFIPS, pages 613–622, AFIPS, June 1979.

[18] S. Pappert. Mindstorms: Children, Computers and Powerful Ideas. Basic Books, 1980.

[19] J. Schwarz. Using annotations to make recursion equations behave. IEEE Trans. onSoftware Engineering, SE-8(1):21–33, January 1982.

[20] E. Shapiro. Systolic programming: a paradigm of parallel processing. Dept. of AppliedMathematics CS84-21, The Weizmann Institute of Science, August 1984.

[21] N.S. Sridharan. Semi-applicative programming: an example. Technical Report, BBNLaboratories, November 1985.

[22] P.C. Treleaven, D.R. Brownbridge, and R.P. Hopkins. Data driven and demand drivencomputer architecture. ARM 15, The University of Newcastle Upon Tyne ComputingLab, July 1980.

23