Analysis of CilkAnalysis of Cilk

A Formal Model for Cilk

A thread: maximal sequence of instructions in a procedure instance (at runtime!) not containing spawn, sync, return

For a given computation, define a dag Threads are vertices Continuation edges within procedures Spawn edges Initial & final threads (in main)

Work & Critical Path

Threads are sequential: work = running time

Define TP = running time on P processors

Then T1 = work in the computation

And T∞ = critical-path length, longest path in the dag

Lower Bounds on TP

TP ≥ T1 / P (no miracles in the model, but they do happen occasionally)

TP ≥ T∞ (dependencies limit parallelism)

Speedup is T1 / TP

(Asymptotic) linear speedup means Θ(P)

Parallelism is T1 / T∞ (average work available at every step along critical path)

Greedy Schedulers

Execute at most P threads in every step Choice of threads is arbitrary (greedy) At most T1 / P complete steps (everybody

busy) At most T∞ incomplete steps

• All threads with in-degree = 0 are executed• Therefore, critical path length reduced by 1

Theorem: TP ≤ T1 / P + T∞

Linear speedup when P = O(T1 / T∞)

Cilk Guarantees:

TP ≤ T1 / P + O(T∞) expected running time

Randomized greedy scheduler The developers claim: TP ≈ T1 / P + T∞

in practice This implies near-perfect speedup

when P << T1 / T∞

But Which Greedy Schedule to Choose?

Busy leaves property: some processor executes every leaf in the dag

Busy leaves controls space consumption;can show SP = O(P S1)

Without busy leaves, worst case is SP = Θ(T1), not hard to reach

A processor that spawns a procedure executes it immediately; but another processor may steal the caller & execute it

Work Stealing Idle processors search for work on other

processors at random When a busy victim is found, the theif steals

the top activation frame on victim’s stack (In practice, stack is maintained as a

dequeue) Why work stealing?

• Almost no overhead when everybody busy• Most of the overhead incurred by theives

• Work-first principle: little impact on T1 / P term

Some Overhead Remains To achieve portability, spawn stack is

maintained as an explicit data structure Theives steal from this dequeue Could implement stealing directly from

stack, more complex and nonportable Main consequence:

• Spawns are more expensive than function calls

• Must do significant work at the bottom of recursions to hide this overhead

• Same is true for normal function calls, but to a lesser extent

More Cilk FeaturesMore Cilk Features

Inlets

x = spawn fib(n-1)is equivalent to:cilk int fib(int n) { int x = 0; inlet void summer(int result) { x += result; } if (n<2) return n; summer( spawn fib(n-1) ); summer( spawn fib(n-2) ); sync; return x;

Inlet Semantics Inlet: an inner function that is called

when a child completes its execution An inlet is a thread in a procedure (so

spawn, sync are not allowed) All the threads of a procedure

instance are executed atomically with respect to one another (i.e., not concurrently)

Easy to reason about correctness x += spawn fib(n-1)is an implicit inlet

Aborting Work

An abort statement in an inlet aborts already-spawned children of a procedure

Useful for aborting speculative searches Semantics:

• Children may not abort instantly• Aborted children do not return values, so don’t

use these values, as in x = spawn fib(n-1)• Does not prevent future spawns; be careful

with sequences of spawns

The SYNCHED Built-In Variable

True only if no children are currently executing

False if some children may be executing now

Useful for avoiding space and work overheads that reduce the critical path when there is no need to

Cilk’s Memory Model

Memory operations of two threads are guaranteed to be ordered only if there is a dependence path between them (ancestor-descendant relationship)

Unordered threads may see inconsistent views of memory

Locks

Mutual-exclusion variables Memory operations that a thread

performs before releasing a lock are seen by other threads after they acquire the lock

Using locks invalidates all the performance guarantees that Cilk provides

In short, Cilk supports locks but don’t use them unless you must

Useful but Obsolete

Cilk as a library• Can call Cilk procedures from C, C++,

Fortran• Necessary for building general-purpose C

libraries Cilk on clusters with distributed memory

• Programmer sees the same shared-memory model

• Used an interesting memory-consistency protocol to support shared-memory view

• Was performance ever good enough?

Some Open ProblemsSome Open Problems

Perhaps good enough for a thesis

Open Issues in Cilk

Theoretical question about the distributed-memory version: is performance monotone in the size of local caches?

Cilk as a library: resurrect Distributed-memory version: resurrect,

is it fast enough? Can you make it faster?

Parallel Merge Sortin Cilk

Parallel Merge Sortin Cilk

Parallel Merge Sort

merge_sort(A, n) if (n=1) return spawn merge_sort(A, n/2) spawn merge_sort(A+n/2, n-n/2) sync merge(A, n/2, n-n/2)

Can’t Merge In Place!

merge_sort(A, T, n, AorT) if (n=1) { T0=A0 ; return }

spawn merge_sort(A, T, n/2, !TorA) spawn merge_sort(A+n/2, n-n/2, !

TorA) sync if (TorA=A) merge(A, T, n/2, n-n/2) if (TorA=T) merge(T, A, n/2, n-n/2)

Analysis

Merging uses two pointers, move the smaller into sorted array

T1(n) = 2T1(n/2) + Θ(n)

T1(n) = Θ(n log n) We fill the output element by element T∞(n) = T∞(n/2) + Θ(n)

T∞(n) = Θ(n) Not very parallel . . .

Parallel Merging

p_merge(A,n,B,m,C) // C is the output swap A,B if A is smaller if (m+n = 1) { C0=A0 ; return }

if (n = 1 /* implies m = 1 */) { merge ; return } locate An/2 between Bj and Bj+1 (binary search)

spawn p_merge(A,n/2,B,j,C) spawn p_merge(A+n/2, n-n/2, B+ j, n-j,

C+n/2+j) sync

Analysis of Parallel Merging

When we merge n elements, both recursive calls merge at most 3n/4 elements

T∞(n) ≤ T∞(3n/4) + Θ(log n) T∞(n) = Θ(log2 n) Critical path is short! But the analysis of work is more complex (extra

work due to binary searches) T1(n) = T1(αn) + T1((1-α)n) + Θ(log n), ¼ ≤ α ≤ ¾ T1(n) = Θ(n) using substitution (nontrivial) Critical path for parallel merge sort T∞(n) = T∞(n/2) + Θ(log2 n) = Θ(log3 n)

Analysis of ParallelMerge Sort

T∞(n) = T∞(n/2) + Θ(log2 n) = Θ(log3 n)

T1(n) = Θ(n) Impact of extra work in practice? Can find the median of 2 sorted arrays

of total size n in Θ(log n) time, leads to parallel merging and merge-sorting with shorter critical paths

Analysis of ParallelMerge Sort

T∞(n) = T∞(n/2) + Θ(log2 n) = Θ(log3 n)

T1(n) = Θ(n) Impact of extra work in practice? Can find the median of 2 sorted arrays

of total size n in Θ(log n) time, leads to parallel merging and merge-sorting with shorter critical paths

Parallelizing an algorithm can be nontrivial!

Cache-EfficientSorting

Cache-EfficientSorting

Caches

Store recently-used data Not really LRU

• Usually 1, 2, or 4-way set associative• But up to 128-way set associative

Data transferred in blocks called cache lines

Write through or write back

Temporal locality: use same data again soon

Spatial locality: use nearby data soon

Cache Misses inMerge Sort

Assume cache-line size = 1, LRU, write back

Assume cache holds M words When n <= M/2, exactly n reads, write

backs When n > M, at least n cache misses

(cover all cases in the proof!) Therefore, number of cache misses is

Θ(n log n/M) = Θ(n log n – log M) We can do much better

The Key Idea

Merge M/2 sorted runs into one, not 2 into 1 Keep one element from each run in a heap,

together with a run label Extract the min, move to sorted run, insert

another element from same run into heap Reading from sorted runs & writing to sorted

ouput removes elements of the heap, but this cost is O(n) cache misses

Θ(n logM n) = Θ(n log n / log M)

This is Poly-Merge Sort

Optimal in terms of cache misses Can adapt to long cache lines, sorting on disks,

etc Originally invented from sorting on tapes on a

machine with several tape drives Often, Θ(n log n / log M) is really Θ(n) in practice Example:

• 32 KB cache, 4+4 bytes elements• 4192-way merges• Can sort 64 MB of data in 1 merge, 256 GB in 2 merges • But more merges with long cache lines

From Quick Sort To Sample Sort

Same number of cache misses with normal quick sort

Key idea• Choose a large random sample, Θ(M) elements• Sort the samples• Classify all the elements using binary searches• Determine size of intervals• Partition• Recursively sort the intervals

Cache miss # probably similar to merge sort

Distributed-MemorySorting

Distributed-MemorySorting

Issues in Sample Sort

Main idea: Partition input into P intervals, classify elements, send elements in ith interval to processor i, sort locally

Most of the communication in one global all-to-all phase

Load balancing: Intervals must besimilar in size

How do we sort the sample?

Balancing the Load

Select a random sample of sP elements OK even if every processor selects s The probability of an interval larger

than cn/P grows linearly with n, shrinks exponentially with s; a large s virtually ensures uniform intervals

Example:• n = 109, s = 256•Pr[max interval > 2n/P] < 10-8

Sorting the Sample

Can’t do it recursively! Sending the sample to one processor

• Θ(sP+n/P) communication in that processor

• Θ(sP log sP+n/P log( n/P )) work• Not scalable, OK for small P

Using a different algorithm that works well for small n/P, e.g., radix sort

Distributed-MemoryRadix Sort

Sort blocks of r bits from least significant to most significant; use a stable sort

Counting sort of one block• Sequentially, count occurrences using a 2r

array, compute prefix sums, use as pointers In parallel, every processor counts

occurrences, then pipelined parallel prefix sums, send elements to destination processor

Θ((b/r) (2r + n/P) work & comm / proc

Odds and EndsOdds and Ends

CPU Utilization Issues

Avoid conditionals• In sorting algorithms, the compiler &

processors cannot predict outcome, so the pipeline stalls

• Example: partitioning in qsort without conditionals

Avoid stalling the pipeline• Even without conditionals, rapid use of

computed values stalls the pipeline• Same example

Dirty Tricks

Exploit uniform input distributions approximately (quick sort, radix sort)• Fix mistakes by bubbling

To avoid conditionals when fixing mistakes• Split the array into small blocks• Use and’s to check for mistakes in a block• Fix a block only if it contains mistakes• Some conditionals, but not many• Compute probability of mistakes to optimize

The ExerciseThe Exercise Get sort.cilk, more instuctions inside Convert sequential merge sort into

parallel merge sort• Make it as fast as possible as long is it is a

parallel merge sort (e.g., make the bottom of the recursion fast)

Convert the fast sort into the parallel fastest sort you can

Submit files in home directory, one page with output on 1, 2 procs + possible explanation (on one side of the page)