Parallel Scheduling of Complex DAGs under Uncertainty

Grzegorz Malewicz

Terminology

Graph – G(V,E) DAG – Directed Acyclic Graph Chain – A path Antichain – subset of V, A, such that there is

no path between two nodes of A.

Terminology - cont

Width of a DAG – the largest cardinality of an antichain in the DAG

A set of chains is said to cover a DAG if every node in the DAG is a node in at least one of the chains.

Dilworth’s theorem states that the width of a graph equals to the minimum number of covering chains.

Source – A node without arcs from it. Sink – A node without arcs pointing to it.

The model

Set of workers – {1,2,…,n} denoted by [n]. Set of tasks – {1,2,…,t} denoted by [t]. The tasks has mutual dependencies which is

represented by a DAG where every node is a task and every directed edge u→v represents task v depends upon completion of task u.

Task t is eligible if all the tasks it’s depends on was successfully executed.

We’ll denote by E(X) the set of all eligible tasks when all the tasks in set X had executed.

The model - cont

A set of tasks satisfies precedence constraints if, for every task in the set, all parents of the task are also in the set.

The execution is consist of synchronous steps. In each of those steps, a central daemon assign a

task from the group E(X) {∅} to every worker. A worker i, completes task j successfully in a certain

step with probability of Pi,j.

The execution completes when every task was executed successfully by at least one worker.

Regimen definition

Formally, the set theory representation of a regimen is:

Σ : 2|V| → ([n] → (V{ }))∅ For any subset Z of tasks that satisfies

precedence constraints, the value Σ(Z) is a function from [n] to the set E(Z) { }∅

ROPAS

Recomputation and Overbooking allowed Probabilistic dAg Scheduling Problem

Given such DAG, G, where for every task

j there exist at least one i such that Pi,j > 0, find a regimen ∑* which minimize the expected completion time.

Lemma 2.1

Denote by BX, The minimum expected completion across all regimen which start with tasks X already executed.

Lemma 2.1: For any set X of tasks that satisfies precedenceconstraints, BX is finite.

Proof: We define a regimen that has finite expectation; thus minimum expectation must be finite, too.

We take a topological sort t1, ..., tm of the subdag of G induced by tasks other than X.

The probability that every worker fails to execute task tj is:

The expected value to execution:

,1

1 1j

n

i ti

P

,1

1

1 1j

n

i ti

P

Optimized algorithm for restricted ROPAS

• We show an dynamic programming algorithms which solves the ROPAS problem in polynomial time under the following restrictions:

1. The number of workers is at most constant.

2. The DAG width is at most constant. Later, we show that if we drop any of those

restrictions, the problem will become NP complete.

Theorem 3.1

Definitions: X - Set of tasks that satisfies precedence constraints and

does not contain all sinks of G. D0, D1, ..., Dk - All the distinct subsets of E(X) where D0= .∅ ai - The probability that Di is exactly the set of tasks

executed by workers in the assignment Σ(X). Xi = X Di. TXi - The expected time to completion for regimen Σ starting

with tasks Xi already executed.

Then the following relation holds:

011

11

i

k

X i Xik

T aTa a

Theorem 3.1 - Proof

T - The time to completion for the regimen that starts with tasks X already executed.

T = TL+TR, where TL is the time until at least one task of E(X) has been executed, and TR is the rest of the time. Exp[T] = Exp[TL] + Exp[TR]

TL~G(P), where P = 1-a0 = a1+a2+…+ak. Exp[TL] = 1/P = 1/(a1+a2+…+ak)

Calculating Exp[TR] – The remaining time to execution

0

[ ] Pr[remaining time is n]Rn

Exp T n

i i1Pr[ ] 0

i i1 0Pr[ ] 0

1

Pr[remaining time is n] Pr[remaining time is n | D ] Pr[D ]

[ ] Pr[D ] Pr[remaining time is n | D ]

Pr[ ]

i

i

i

i kD

Ri k nD

k

i Xi

Exp T

D T

Let Di be the event: “Di is exactly the set of tasks first executed by the workers”.

Theorem 3.1 - conclusions

1 2

1 1 11 2 1 2

11 2 1 2

11 2

[ ] [ ] [ ]

1[ ]

1[ ] Pr[ ]

1 1[ ] [ ]

11

i

i i

i

i

L R

Lk

k k ki X

R i X i Xi i ik k

k

L R i Xik k

k

i Xik

Exp T Exp T Exp T

Exp Ta a a

aTExp T D T aT

a a a a a a

Exp T Exp T aTa a a a a a

aTa a a

Admissible evolution of execution graph For every DAG G we can construct a graph

G’(V’,E’) called “admissible evolution of execution” which has single source and single sink.

Every node in G’ represents a set of nodes (or tasks) from G.

This special graph is also a DAG. Every execution flow of every ROPAS

regimen on G can be represented by a path from the source to the sink of this graph.

Admissible evolution of execution graph – Construction The construction of this graph is inductive:

We begin with a set V’={ }.∅ For any node XV’ that doesn’t contain all sinks

of G calculate the set of eligible tasks E(X) in G. Take all non-empty subsets D E(X) and add to

V’ the node XD if it is not there already. Add an arc (X, XD) to V’.

Since G is finite, the inductive process clearly defines a unique directed graph G’.

Admissible evolution of execution graph – Properties G’ is a DAG – it cannot has any cycle because

every arc is in the from (X, XD) and X has strictly more elements than XD.

Nodes of G’ are exactly the sets of tasks that satisfy precedence constraints => By induction – if X satisfy precedence constraints so

does XD. <= Pick any subset Y of tasks that satisfies precedence

constraints. Let t1, ..., t|Y | be a topological sort of the sub-DAG of G induced by Y. Clearly, tj belongs to the set of tasks eligible when tasks t1, ..., tj−1 have been executed, for any j. So if {t1, ..., tj−1} is a node of G’, so is {t1, ..., tj}. Since

∅ is a node of G’, Y must also be.

Admissible evolution of execution graph Properties – cont’d G’ has a single source - We add a node Y to

G’ only if there is an arc leading to Y from some other node. So Y cannot be a source. However, ∅ is a source because no arc leads to a set with the same number or fewer elements.

G’ has a single sink – For any XV’, E(X)∅ except X=V.

The algorithm OPT

When processing each XV’ define two values: A number SX - The expected time to completion for regimen ΣOPT

starting with tasks X already executed. An assignment ΣOPT(X).

Pick a topological sort Y1, ..., Ym of G’ and process it in the reverse order.

Start by setting Sym to 0 and ΣOPT(Ym) to ∅. For any 1h<m consider all possible (|E(Yh)|+1)n assignment to n

workers: For any assignment, calculate the probability a i that Di is

exactly the set of tasks executed by workers in the assignment.

Find the assignment which minimizes the expression:

Set SYh to the value of this expression and ΣOPT(Yh) to this assignment.

11 2

11

i

k

i Xik

a Sa a a

Theorem 3.3

The regimen ΣOPT‘s expected time to completion when starting with tasks X already executed is the minimal expected time to completion that any regimen can achieve (denoted by BX).

Theorem 3.3 - Proof

The following invariant holds during the reverse order processing of V’: For all i such that h ≤ i ≤ m, SYi = BYi and SYi is equal to the

expected time to completion of ΣOPT that starts with tasks Yi already executed.

The invariant is clearly true for i=m because G’ has exactly one sink.

By induction, assume the invariant holds for 2≤i≤m (i.e. SYi = BYi). According to theorem 3.1:

And it’s exactly equals to SYi

11 2

1min 1

i i

k

Y i Xik

B aTa a a

Theorem 3.4

The OPT algorithm works at polynomial time when the DAG’s width and the workers number are at most constants.

Proof: Let the width of the graph be w and number of workers n. By Dilworth’s theorem, the DAG can be covered by w chains,

each chain can have a length of at most t so there are at most (t+1)w distinct subsets of tasks that satisfy precedence constraints.

In each step, the algorithm consider assignment of at most t+1 tasks to each worker. So there are at most (t+1)n calculations in each step.

We only need to evaluate probabilities ai for at most 2n of the sets Di, because the workers can get assigned to at most n different tasks,

The computations step of OPT is at most C1(t+1)w (t+1)n 2n+C2, thus polynomial.

Part II

NP hardness

X3C - Exact 3-Set cover

Instance: F = {S1, …, Sm} of subsets of a finite set U, and |Si| = 3, |U| = 3n for some n.

Objective: Find n sets in F that are disjoint and have U as their union.

The problem is a sub-problem of the exact set cover and thus NP hard.

MP – Multiplicative partition

Instance: Number n, integer sizes s1, ..., sn where si≥ 2.

Question: Is there a set P such that:

The MP problem is NP hard. We show a polynomial time reduction from

X3C.

i ii P i P

s s

MP is NP hard

Given an instance of X3C, we associate consecutive prime numbers with each element of the universe (U).

Each 3-subset is converted into a size that is the product of the associated primes.

We add two more sizes: C which is

p1a1-2 * p2

a2-2 * …pkak-2

where ai is the number of occurrences of pi and pi occur more than once. D which is pi1 * pi2 * pi3 * … * pin where pij appear once.

MP ≥ X3C - example

S4S1S2 S3

S5

=2

=3

=5

=7

=11

=13

MP ≥ X3C - example

S1=2*3*13=78 S2=2*5*11=110 S3=5*7*11=385 S4=3*5*11=165 S5=2*7*11=154 C=21*30*51*70*112=1210 D=13

2*3*13 * 5*7*11 * 21*30*51*70*112 =

2*5*11 * 3*5*11 * 2*7*11 * 13

Partition with Minimum Sum of Products (PMSP) Instance: Number n, rational numbers: 0 <

r1, ..., rn < 1. Objective: Find a set P that minimizes: Claim: PMSP is NP hard: Proof: Pick any instance of MP with integer

s1, …, sn.

Look at the instance of PMSP where ri=1/si2.

i ii P i P

r r

PMSP is NP hard

The function ex+ed-x in the domain {0≤x≤d} is minimized exactly when x=d/2 and it’s value is 2ed/2.

s1, …, sn can be partitioned to two sets of equal multiplication iff:

Thus if we would know the answer to PMSP in polynomial time – we’ll know the answer to MP in polynomial time.

ln

ln ln/2

1

since x= :

2 2i i

i P i P

x

nr d rd

i i ii P i P i

e

r r e e e r

1

min 2n

i i iP S

i P i P i

r r r

Theorem 4.3

The ROPAS Problem restricted to the DAG with two independent tasks (where the number of workers may grow) is NP-hard.

Proof: Polynomial time reduction from PMSP: Consider a DAG with only two independent tasks: right task

and left task. Each worker executes a task successfully with probability

of pi. Assume we split the tasks to N workers where l is the

probability that at least one of the workers assigned to the left task will succeed and r is same probability regarding the right task.

Theorem 4.3 – cont.

By theorem 3.1, the expected time to completion:

the regimen initially assigns every worker to a task, and when one task remains to be executed, the regimen assigns all workers to the task. So T{left}=T{right}=T and we get:

Since L and R partition {1, ..., n}, then 1-(1-l)(1-r) is just the probability that at least one of the n workers succeeds and is independent of how workers are assigned to left and right task.

Thus the expectation is minimized if, and only if, lr is maximized across partitions L and R of {1, ..., n}.

{ } { }1 (1 ) (1 )

1 (1 )(1 )left rightl r T r l T

Tl r

1

1 (1 )(1 )

TlrT T

l r

Theorem 4.3 – cont.

Since L and R is a partition – The value in the 3rd term is independent of R and L.

So lr is minimized iff

is minimized.

1 (1 ), r 1 (1 ) 1 (1 )

1 (1 ) (1 ) (1 ) (1 )

i i ii L i R i L

i i i ii L i L i L i L

l p p p

LR p p p p

(1 ) (1 )i ii L i L

p p

FIRMSSU Problem

Fixed Ratio Many Subsets with Small Union

Instance: Number k, nonempty subsets S1, ..., S3k of the set {1, ..., 3k} whose union is {1, ..., 3k}.

Question: Are there 2k of these subsets whose union has cardinality at most 2k?

ROPAS ≥FIRMSSU

Tasks A1, B1 can only be executed by the first worker.

Tasks A2, B2,C2 can only be executed by the second worker.

Round 2/3n2 (24)

Round 2/3n2+2/3n (28)

Round n2+2/3n (40)

Round n2+n (42)

Suppose the instance of FIRMSSU is negative… Pick any regimen. For the reason that the instance is negative,

the union of any 2k subsets has cardinality at least2k + 1.

But the cardinality of any group is n, so at least2/3n2+n sources of A1 must be executed in order to have2/3n sinks of B2 eligible.

Therefore, at most 2/3n−1 sinks of B2 can get executed by the end of round 2/3n2 + 2/3n.

By the end of this round, no sinks of C2 can be executed,because each depends on execution of 2/3n2+2/3n other tasks.

Hence, by the end of that round, there are at least1/3n+1+1/3n2 unexecuted tasks that can only be executedby the second worker.

As a result the regimen takes at least n2 + n + 1 rounds to complete computation.

ROPAS is inaproximable

When there are no restriction on both the DAG width and the number of workers - ROPAS is inaproximable with a factor better than 5/4, unless P=NP.

Similar proof like the former. Assume there are 2/3n2 workers of type 1

and 2/3n2 workers of type 2. The next DAG can be scheduled in 4 rounds

iff an instance of FIRMSSU is positive.

Round 1

Round 3

Round 4