a common mistake size/communication trade-off specific tradeoffs
DESCRIPTION
Designing Efficient Map-Reduce Algorithms. A Common Mistake Size/Communication Trade-Off Specific Tradeoffs. Jeffrey D. Ullman Stanford University. Research Is Joint Work of. Foto Afrati (NTUA) Anish Das Sarma (Google) Semih Salihoglu (Stanford) U. Motivating Example. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/1.jpg)
Designing Efficient Map-Reduce Algorithms
A Common MistakeSize/Communication Trade-OffSpecific Tradeoffs
Jeffrey D. UllmanStanford University
![Page 2: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/2.jpg)
2
Research Is Joint Work of Foto Afrati (NTUA) Anish Das Sarma (Google) Semih Salihoglu (Stanford) U.
![Page 3: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/3.jpg)
Motivating Example
The Drug Interaction ProblemA Failed AttemptLowering the Communication
![Page 4: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/4.jpg)
4
The Drug-Interaction Problem Data consists of records for 3000 drugs.
List of patients taking, dates, diagnoses. About 1M of data per drug.
Problem is to find drug interactions. Example: two drugs that when taken together
increase the risk of heart attack. Must examine each pair of drugs and compare
their data.
![Page 5: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/5.jpg)
5
Initial Map-Reduce Algorithm The first attempt used the following plan:
Key = set of two drugs {i, j}. Value = the record for one of these drugs.
Given drug i and its record Ri, the mapper generates all key-value pairs ({i, j}, Ri), where j is any other drug besides i.
Each reducer receives its key and a list of the two records for that pair: ({i, j}, [Ri, Rj]).
![Page 6: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/6.jpg)
6
Example: Three Drugs
Mapperfor drug
2
Mapperfor drug
1
Mapperfor drug
3
Drug 1 data{1, 2} Reducer
for {1,2}
Reducerfor
{2,3}
Reducerfor
{1,3}
Drug 1 data{1, 3}
Drug 2 data{1, 2}
Drug 2 data{2, 3}
Drug 3 data{1, 3}
Drug 3 data{2, 3}
![Page 7: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/7.jpg)
7
Example: Three Drugs
Mapperfor drug
2
Mapperfor drug
1
Mapperfor drug
3
Drug 1 data{1, 2} Reducer
for {1,2}
Reducerfor
{2,3}
Reducerfor
{1,3}
Drug 1 data{1, 3}
Drug 2 data{1, 2}
Drug 2 data{2, 3}
Drug 3 data{1, 3}
Drug 3 data{2, 3}
![Page 8: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/8.jpg)
8
Example: Three DrugsDrug 1 data{1, 2}
Reducerfor
{1,2}
Reducerfor
{2,3}
Reducerfor
{1,3}Drug 1 data
Drug 2 data
Drug 2 data{2, 3}
Drug 3 data{1, 3}
Drug 3 data
![Page 9: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/9.jpg)
9
What Went Wrong? 3000 drugs times 2999 key-value pairs per drug times 1,000,000 bytes per key-value pair = 9 terabytes communicated over a 1Gb
Ethernet = 90,000 seconds of network use.
![Page 10: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/10.jpg)
10
The Improved Algorithm They grouped the drugs into 30 groups of 100
drugs each. Say G1 = drugs 1-100, G2 = drugs 101-200,…, G30 =
drugs 2901-3000. Let g(i) = the number of the group into which drug i
goes.
![Page 11: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/11.jpg)
11
The Map Function A key is a set of two group numbers. The mapper for drug i produces 29 key-value
pairs. Each key is the set containing g(i) and one of the
other group numbers. The value is a pair consisting of the drug number i
and the megabyte-long record for drug i.
![Page 12: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/12.jpg)
12
The Reduce Function The reducer for pair of groups {m, n} gets that
key and a list of 200 drug records – the drugs belonging to groups m and n.
Its job is to compare each record from group m with each record from group n. Special case: also compare records in group n with
each other, if m = n+1 or if n = 30 and m = 1. Notice each pair of records is compared at
exactly one reducer, so the total computation is not increased.
![Page 13: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/13.jpg)
13
The New Communication Cost The big difference is in the communication
requirement. Now, each of 3000 drugs’ 1MB records is
replicated 29 times. Communication cost = 87GB, vs. 9TB.
![Page 14: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/14.jpg)
Theory of Map-Reduce AlgorithmsReducer SizeReplication RateMapping SchemasLower Bounds
![Page 15: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/15.jpg)
15
A Model for Map-Reduce Algorithms1. A set of inputs.
Example: the drug records.2. A set of outputs.
Example: One output for each pair of drugs.3. A many-many relationship between each
output and the inputs needed to compute it. Example: The output for the pair of drugs {i, j} is
related to inputs i and j.
![Page 16: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/16.jpg)
16
Example: Drug Inputs/Outputs
Drug 1
Drug 2
Drug 3
Drug 4
Output 1-2
Output 1-3
Output 2-4
Output 1-4
Output 2-3
Output 3-4
![Page 17: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/17.jpg)
17
Example: Matrix Multiplication
=i
jj
i
![Page 18: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/18.jpg)
18
Reducer Size Reducer size, denoted q, is the maximum
number of inputs that a given reducer can have. I.e., the length of the value list.
Limit might be based on how many inputs can be handled in main memory.
Or: make q low to force lots of parallelism.
![Page 19: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/19.jpg)
19
Replication Rate The average number of key-value pairs created
by each mapper is the replication rate. Denoted r.
Represents the communication cost per input.
![Page 20: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/20.jpg)
20
Example: Drug Interaction Suppose we use g groups and d drugs. A reducer needs two groups, so q = 2d/g. Each of the d inputs is sent to g-1 reducers, or
approximately r = g. Replace g by r in q = 2d/g to get r = 2d/q.
Tradeoff!The bigger the reducers,the less communication.
![Page 21: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/21.jpg)
21
Upper and Lower Bounds on r What we did gives an upper bound on r as a
function of q. A solid investigation of map-reduce algorithms
for a problem includes lower bounds. Proofs that you cannot have lower r for a given q.
![Page 22: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/22.jpg)
22
Proofs Need Mapping Schemas A mapping schema for a problem and a reducer
size q is an assignment of inputs to sets of reducers, with two conditions:1. No reducer is assigned more than q inputs.2. For every output, there is some reducer that
receives all of the inputs associated with that output.
Say the reducer covers the output.
![Page 23: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/23.jpg)
23
Mapping Schemas – (2) Every map-reduce algorithm has a mapping
schema. The requirement that there be a mapping
schema is what distinguishes map-reduce algorithms from general parallel algorithms.
![Page 24: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/24.jpg)
24
Example: Drug Interactions d drugs, reducer size q. Each drug has to meet each of the d-1 other
drugs at some reducer. If a drug is sent to a reducer, then at most q-1
other drugs are there. Thus, each drug is sent to at least (d-1)/(q-1)
reducers, and r > (d-1)/(q-1). Half the r from the algorithm we described. Better algorithm gives r = d/q + 1, so lower
bound is actually tight.
![Page 25: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/25.jpg)
25
The Better Algorithm The problem with the algorithm dividing inputs
into g groups is that members of a group appear together at many reducers. Thus, each reducer can only productively compare
about half the elements it gets. Better: use smaller groups, with each reducer
getting many little groups. Eliminates almost all the redundancy.
![Page 26: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/26.jpg)
26
Optimal Algorithm for All-Pairs Assume d inputs. Let p be a prime, where p2 divides d. Divide inputs into p2 groups of d/p2 inputs each. Name the groups (i, j), where 0 < i, j < p. Use p(p+1) reducers, organized into p+1 teams
of p reducers each. For 0 < k < p, group (i, j) is sent to the reducer
i+kj (mod p) in group k. In the last team (p), group (i, j) is sent to
reducer j.
![Page 27: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/27.jpg)
27
Example: Teams for p = 5
i = 0
1
2
1
3
432
4
j = 0
Team 0
![Page 28: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/28.jpg)
28
Example: Teams for p = 5
i = 0
1
2
1
3
432
4
j = 0
Team 1
![Page 29: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/29.jpg)
29
Example: Teams for p = 5
i = 0
1
2
1
3
432
4
j = 0
Team 2
![Page 30: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/30.jpg)
30
Example: Teams for p = 5
i = 0
1
2
1
3
432
4
j = 0
Team 3
![Page 31: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/31.jpg)
31
Example: Teams for p = 5
i = 0
1
2
1
3
432
4
j = 0
Team 4
![Page 32: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/32.jpg)
32
Example: Teams for p = 5
i = 0
1
2
1
3
432
4
j = 0
Team 5
![Page 33: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/33.jpg)
33
Why It Works Let two inputs be in groups (i, j) and (i’, j’). If the same group, these inputs obviously share
a reducer. If j = j’, then they share a reducer in team p. If j j’, then they share a reducer in team k
provided i + kj = i’ + kj’ (all arithmetic modulo p). Equivalently, (i-i’) = k(j-j’). But since j j’, (j-j’) has an inverse modulo p. Thus, team k = (i-i’)(j-j’)-1 has a reducer for which
i + kj = i’ + kj’.
![Page 34: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/34.jpg)
34
Why It Is Optimal The replication rate r is p+1, since every input is
sent to one reducer in each team. The reducer size q = p(d/p2) = d/p, since each
reducer gets p groups of size d/p2. Thus, r = d/q + 1. (d/q + 1) - (d-1)/(q-1) < 1 provided q < d.
But if q > d, we can do everything in one reducer, and r = 1.
The upper bound r < d/q + 1 and the lower bound r > (d-1)/(q-1) differ by less than 1, and are integers, so they are equal.
![Page 35: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/35.jpg)
The Hamming-Distance = 1 Problem
The Exact Lower BoundMatching Algorithms
![Page 36: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/36.jpg)
36
Definition of HD1 Problem Given a set of bit strings of length b, find all
those that differ in exactly one bit. Example: For b=2, the inputs are 00, 01, 10, 11,
and the outputs are (00,01), (00,10), (01,11), (10,11).
Theorem: r > b/log2q. (Part of) the proof later.
![Page 37: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/37.jpg)
37
Algorithm With q=2 We can use one reducer for every output. Each input is sent to b reducers (so r = b). Each reducer outputs its pair if both its inputs
are present, otherwise, nothing. Subtle point: if neither input for a reducer is
present, then the reducer doesn’t really exist.
![Page 38: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/38.jpg)
38
Algorithm with q = 2b
Alternatively, we can send all inputs to one reducer.
No replication (i.e., r = 1). The lone reducer looks at all pairs of inputs that
it receives.
![Page 39: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/39.jpg)
39
Splitting Algorithm Assume b is even. Two reducers for each string of length b/2.
Call them the left and right reducers for that string. String w = xy, where |x| = |y| = b/2, goes to the
left reducer for x and the right reducer for y. If w and z differ in exactly one bit, then they will
both be sent to the same left reducer (if they disagree in the right half) or to the same right reducer (if they disagree in the left half).
Thus, r = 2; q = 2b/2.
![Page 40: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/40.jpg)
40
Proof That r > b/log2q Lemma: A reducer of size q cannot cover more
than (q/2)log2q outputs. Induction on b; proof omitted.
(b/2)2b outputs must be covered. There are at least p = (b/2)2b/((q/2)log2q) =
(b/q)2b/log2q reducers. Sum of inputs over all reducers > pq = b2b/log2q. Replication rate r = pq/2b = b/log2q.
Omits possibility that smaller reducers help.
![Page 41: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/41.jpg)
Algorithms Matching Lower Bound
q = reducersize
b
21
21 2b/2 2b
All inputsto onereducer
One reducerfor each output Splitting
Generalized Splitting
41
r = replicationrate
![Page 42: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/42.jpg)
Matrix Multiplication
One-Job MethodTwo-Job MethodComparison
![Page 43: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/43.jpg)
43
Matrix Multiplication Assume n n matrices AB = C. Aij is the element in row i and column j of matrix
A. Similarly for B and C.
Cik = j Aij Bjk. Output Cik depends on the ith row of A, that is, Aij
for all j, and the kth column of B, that is, Bjk for all j.
![Page 44: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/44.jpg)
44
Computing One Output Value
=Row i
Column k
A B C
![Page 45: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/45.jpg)
45
Reducers Cover Rectangles Important fact: If a reducer covers outputs Cik
and Cfg, then it also covers Cig and Cfk. Why? This reducer has all of rows i and f of A as
inputs and also has all of columns k and g of B as inputs.
Thus, it has all the inputs it needs to cover Cig and Cfk.
Generalizing: Each reducer covers all the outputs in the “rectangle” defined by a set of rows and a set of columns of matrix C.
![Page 46: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/46.jpg)
46
The Responsibility of One Reducer
![Page 47: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/47.jpg)
47
Upper Bound on Output Size If a reducer gets q inputs, it gets q/n rows or
columns. Maximize the number of outputs covered by
making the input “square.” I.e., #rows = #columns.
q/2n rows and q/2n columns yield q2/4n2 outputs covered.
![Page 48: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/48.jpg)
48
Lower Bound on Replication Rate Total outputs = n2. One reducer can cover at most q2/4n2 outputs. Therefore, 4n4/q2 reducers. 4n4/q total inputs to all the reducers, divided by
2n2 total inputs = 2n2/q replication rate. Example: If q = 2n2, one reducer suffices and the
replication rate is r = 1. Example: If q = 2n (minimum possible), then r
= n.
![Page 49: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/49.jpg)
49
Matching Algorithm Divide rows of the first matrix into g groups of
n/g rows each. Also divide the columns of the second matrix
into g groups of n/g columns each. g2 reducers, each with q = 2n2/g inputs
consisting of a group of rows and a group of columns.
r = g = 2n2/q.
![Page 50: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/50.jpg)
50
Picture of One Reducer
=n/g
n/g
![Page 51: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/51.jpg)
51
Two-Job Map-Reduce Algorithm
A better way: use two map-reduce jobs. Job 1: Divide both input matrices into
rectangles. Reducer takes two rectangles and produces partial
sums of certain outputs. Job 2: Sum the partial sums.
![Page 52: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/52.jpg)
52
Picture of First Job
I
J
J
K
I
K
A CB
For i in I and k in K, contributionis j in J Aij × Bjk
![Page 53: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/53.jpg)
53
First Job – Details Divide the rows of the first matrix A into g
groups of n/g rows each. Divide the columns of A into 2g groups of n/2g. Divide the rows of the second matrix B into 2g
groups of n/2g rows each. Divide the columns of B into g groups of n/g. Important point: the groups of columns for A
and rows for B must have indices that match.
![Page 54: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/54.jpg)
54
Reducers for First Job Reducers correspond to an n/g by n/2g
rectangle in A (with row indices I, column indices J) and an n/2g by n/g rectangle in B (with row indices J and column indices K). Call this reducer (I,J,K). Important point: there is one set of indices J that
plays two roles. Needed so only rectangles that need to be multiplied are
given a reducer.
![Page 55: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/55.jpg)
55
The Reducer (I,J,K)
I
J
J
K
I
K
A CB
n/g n/gn/g
n/gn/2g
n/2g
2g reducers contribute tothis area, one for each J.
![Page 56: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/56.jpg)
56
Job 1: Details Convention: i, j, k are individual rows and/or
column numbers, which are members of groups I, J, and K, respectively.
Mappers Job 1: Aij -> key = (I,J,K) for any group K; value = (A,i,j,Aij). Bjk -> key = (I,J,K) for any group I; value = (B,j,k,Bjk).
Reducers Job 1: For key (I,J,K) produce xiJk = j in J Aij Bjk for all i in I and k in K.
![Page 57: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/57.jpg)
57
Job 2: Details
Mappers Job 2: xiJk -> key = (i,k), value = xiJk.
Reducers Job 2: For key (i,k), produce output Cik = J xiJk.
![Page 58: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/58.jpg)
58
Comparison: Computation Cost The two methods (one or two map-reduce jobs)
essentially do the same computation. Every Aij is multiplied once with every Bjk. All terms in the sum for Cik are added together
somewhere, only once. 2 jobs requires some extra overhead of task
management.
![Page 59: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/59.jpg)
59
Comparison: Communication Cost
One-job method: r = 2n2/q; there are 2n2 inputs, so total communication = 4n4/q.
Two-job method with parameter g:Job 2: Communication = (2g)(n2/g2)(g2) = 2n2g.
Number of output squares
Area of each square
Number of reducerscontributing toeach output
![Page 60: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/60.jpg)
60
Communication Cost – Continued
Job 1 communication: 2n2 input elements. Each generates g key-value pairs. So another 2n2g. Total communication = 4n2g.
Reducer size q = (2)(n2/2g2) = n2/g2. So g = n/q. Total communication = 4n3/q.
Compares favorably with 4n4/q for the one-job approach.
![Page 61: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/61.jpg)
61
Summary Represent problems by mapping schemas Get upper bounds on number of covered
outputs as a function of reducer size. Turn these into lower bounds on replication
rate as a function of reducer size. For HD = 1 and all-pairs problems: exact match
between upper and lower bounds. 1-job matrix multiplication analyzed exactly. But 2-job MM yields better total
communication.
![Page 62: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/62.jpg)
62
Research Questions Get matching upper and lower bounds for the
Hamming-distance problem for distances greater than 1. Ugly fact: For HD=1, you cannot have a large
reducer with all pairs at distance 1; for HD=2, it is possible.
Consider all inputs of weight 1 and length b.
![Page 63: A Common Mistake Size/Communication Trade-Off Specific Tradeoffs](https://reader036.vdocuments.mx/reader036/viewer/2022062310/568165a8550346895dd88e3f/html5/thumbnails/63.jpg)
63
Research Questions – (2)1. Give an algorithm that takes an input-output
mapping and a reducer size q, and gives a mapping schema with the smallest replication rate.
2. Is the problem even tractable?3. A recent extension by Afrati, Dolev, Korach,
Sharma, and U. lets inputs have weights, and the reducer size limits the sum of the weights of the inputs received.
What can be extended to this model?