parallel algorithms on networks of processors
DESCRIPTION
Parallel Algorithms on Networks of Processors. Roy (Hutch) Pargas, PhD Computer Science (UNC Chapel Hill) School of Computing, Clemson University [email protected]. Outline. What are parallel algorithms? Why use them? Challenges for parallel algorithm designers Choosing a network - PowerPoint PPT PresentationTRANSCRIPT
Parallel Algorithms on Networks of Processors
Roy (Hutch) Pargas, PhD Computer Science (UNC Chapel Hill)School of Computing, Clemson University
2
OutlineWhat are parallel algorithms? Why use them?
Challenges for parallel algorithm designers
Choosing a network
Partitioning the data
Designing the algorithm
Example
Recurrences (binary tree)
Analysis (Speedup and Efficiency)
Summary and Conclusions
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
3
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
4
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
5
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
6
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
7
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
8
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
9
New ProcessorsFaster and Cheaper
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
January2011
10
Partition the Data
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
11
Organize the Processors
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
12
Build a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
13
Build a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
14
Choosing a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
15
Choosing a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
16
Choosing a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
17
Choosing a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
18
Choosing a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
19
Are There Really Any Multiprocessing
Systems in Use Today?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
20
Are There Really Any Multiprocessing
Systems in Use Today?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
HamburgJune 2011Top 500
21
SupercomputersNEC/HP Tsubame
(Japan)
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
1.192 petaflops ≈ 1.28 quadrillion floating point
operations per sec
73,278 Xeon cores
Infiniband grid network
22
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
SupercomputersDawning Nebulae
(China)1.27 petaflops ≈ 1.36
quadrillion floating point operations per sec
9280 Intel 6-core Xeon processors = 55,680 cores
Infiniband grid network
23
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
SupercomputersCray Jaguar (USA)
1.75 petaflops ≈ 1.876 quadrillion floating point
operations per sec
224,256 AMD cores
3D torus network
24
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
SupercomputersNUDT Tianhe-1A
(China)2.566 petaflops ≈ 2.75
quadrillion floating point operations per sec
14336 CPUs
Undisclosed proprietary network
25
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
SupercomputersFujitsu “K” (Japan)
K = “kei” = Japanese for 10 quadrillion
8.162 petaflops ≈ 9 quadrillion floating point operations per
sec
68,544 8-core SPARC64 processors = 548,352
cores
3-dimensional torus network called Tofu
26
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
TOP500
Top 500 Computers in the World
27
Where Does that Leave Us?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
28
Where Does that Leave Us?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
In a wonderful playground of mathematical algorithmic design where imagination and creativity are key!
29
Where Does that Leave Us?
In a wonderful playground of mathematical algorithmic design where imagination and creativity are key!
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
30
Where Does that Leave Us?
In a wonderful playground of mathematical algorithmic design where imagination and creativity are key!
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
31
Where Does that Leave Us?
In a wonderful playground of mathematical algorithmic design where imagination and creativity are key!
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
32
Challenges for Parallel Algorithm
DesignersChoosing a network
Partitioning the problem
Designing the parallel algorithm
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
33
So Let’s Try It:
Choosing a network
Partitioning the problem
Designing the parallel algorithm
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
34
So Let’s Try It:Elliptic Partial Diff
EqnsChoosing a network
Partitioning the problem
Designing the parallel algorithm
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
35
Elliptic PDEs
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Problems involving second-order elliptic partial differential equations are equilibrium problems. Given a region R bounded by a curve C and that the unknown function z satisfies Laplace’s or Poisson’s equation in R, the objective is to approximate the value of z at any point in R. The method of finite differences is an often used numerical method for solving this problem. The basic strategy is to approximate the differential equation by a difference equation and to solve the difference equation.
36
Designing the Algorithm
Why Solve Linear Recurrences?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
37
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
38
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Solving Block Tridiagonal Systems which leads to
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
39
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Solving Block Tridiagonal Systems which leads to
Solving Tridiagonal Systems which leads to
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
40
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Solving Block Tridiagonal Systems which leads to
Solving Tridiagonal Systems which leads to
Solving Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
41
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Solving Block Tridiagonal Systems which leads to
Solving Tridiagonal Systems which leads to
Solving Linear Recurrences many many many times
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
42
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Solving Block Tridiagonal Systems which leads to
Solving Tridiagonal Systems which leads to
Solving Linear Recurrences many many many times
Why Use a Binary Tree? Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
43
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Key Idea: Successfully solving the original pde problem depends upon solving recurrences quickly and efficiently.
44
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Consider the following set of n equations:x0 = a0
x1 = a1 + b1 x0
x2 = a2 + b2 x1
...
xn-1 = an-1 + bn-1 xn-2
45
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Consider the following set of n equations:x0 = a0
x1 = a1 + b1 x0
x2 = a2 + b2 x1
...
xn-1 = an-1 + bn-1 xn-2
Can we solve for xi in parallel?
46
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
For uniformity:x0 = a0 + b0 x-1 b0=0, x-1=dummy variablex1 = a1 + b1 x0
x2 = a2 + b2 x1
...
xn-1 = an-1 + bn-1 xn-2
47
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
For uniformity:x0 = a0 + b0 x-1
x1 = a1 + b1 x0
x2 = a2 + b2 x1
...
xn-1 = an-1 + bn-1 xn-2
Observe, ifxi = a + b xj
xj = a’ + b’ xk
Thenxi = (a + ba’) +bb’ xk
= a” + b” xk
48
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Notation changex0 = a0 + b0 x-1 C0,-1 = (a0,b0)x1 = a1 + b1 x0 C1,0 = (a1,b1)x2 = a2 + b2 x1 C2,1 = (a2,b2)...
xn-1 = an-1 + bn-1 xn-2 Cn-1,n-2 = (an-1,bn-1)
Observe, ifxi = a + b xj
xj = a’ + b’ xk
Thenxi = (a + ba’) +bb’ xk
= a” + b” xk
49
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Notation changex0 = a0 + b0 x-1 C0,-1 = (a0,b0)x1 = a1 + b1 x0 C1,0 = (a1,b1)x2 = a2 + b2 x1 C2,1 = (a2,b2)...
xn-1 = an-1 + bn-1 xn-2 Cn-1,n-2 = (an-1,bn-1)
Observe, ifxi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
50
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
51
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
One last observation:
52
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
One last observation:If any variable is expressed in terms of
the dummy variable x-1 (e.g., x0 = a0 + b0 x-1) then that variable is solved.
53
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
One last observation:If any variable is expressed in terms of
the dummy variable x-1 (e.g., x0 = a0 + b0 x-1) then that variable is solved. So Ci,-1 = (a,b)
54
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
One last observation:If any variable is expressed in terms of
the dummy variable x-1 (e.g., x0 = a0 + b0 x-1) then that variable is solved. So Ci,-1 = (a,b) means that b=0
55
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
One last observation:If any variable is expressed in terms of
the dummy variable x-1 (e.g., x0 = a0 + b0 x-1) then that variable is solved. So Ci,-1 = (a,b) means that b=0 and that xi = a + b x-1 = a + 0 x-1 = a
56
C0,-
1
C1,
0
C2,
1
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6 Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear Recurrences
57
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
58
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
59
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
60
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
61
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
62
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
63
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
C7,-
1C7,-1
64
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
C7,-
1C7,-1
Solved variable
65
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
C7,-
1C7,-1
Solved variables
66
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
C7,-
1C7,-1
How do we solve for the other variables?
67
C6,5
C7,-1
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
C7,-
1
In the downsweep!
68
C6,5
C7,-1
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
(x7,x-
1)
69
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
(x7,x-
1)
70
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
x3
(x7,x-
1)
71
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
72
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C3,1 C1,-1
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5C7,5 C5,3
x1
73
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x7,x5) (x5,x3) (x3,x1) (x1,x-1)
74
(x7,x5)
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
75
(x7,x5)
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
(x7,x6) (x6,x5) (x5,x4)(x0,x-1)(x1,x0)(x4,x3) (x3,x2) (x2,x1)
76
(x7,x5)
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
(x7,x6) (x6,x5) (x5,x4)(x0,x-1)(x1,x0)(x4,x3) (x3,x2) (x2,x1)
(x7,x6) (x0,x-
1)(x6,x5) (x5,x4) (x4,x3) (x3,x2) (x2,x1) (x1,x0)
77
(x7,x5)
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
(x7,x6) (x6,x5) (x5,x4)(x0,x-1)(x1,x0)(x4,x3) (x3,x2) (x2,x1)
(x7,x6) (x0,x-
1)(x6,x5) (x5,x4) (x4,x3) (x3,x2) (x2,x1) (x1,x0)
Leaves contain solutions!
78
(x7,x5)
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
(x7,x6) (x6,x5) (x5,x4)(x0,x-1)(x1,x0)(x4,x3) (x3,x2) (x2,x1)
(x7,x6) (x0,x-
1)(x6,x5) (x5,x4) (x4,x3) (x3,x2) (x2,x1) (x1,x0)
But we can do better!
79
(x7,x5)
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
(x7,x6) (x6,x5) (x5,x4)(x0,x-1)(x1,x0)(x4,x3) (x3,x2) (x2,x1)
(x7,x6) (x0,x-
1)(x6,x5) (x5,x4) (x4,x3) (x3,x2) (x2,x1) (x1,x0)
Pipelining the solution
80
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
81
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
82
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
83
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
84
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
85
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
86
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
87
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
88
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
89
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
90
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
91
AnalysisT1 = Time on one processor
Tn = Time on n processors
S = Speedup = T1/Tn (ideal: S = n)
E = Efficiency = S/n (ideal: E = 1)
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
92
Speedup and Efficiency
Single PassAssume 1 floating point operation requires 1 time unit
T1 = (n−1) (1 mult + 1 add) = 2n−2 time units
n leaves 2n processors T2n = (log2n)( 2 mults + 1 add) // upsweep
+ (log2n)(1 mult + 1 add) // downsweep= 5 log2n time units
S = Speedup = T1/T2n = (2n−2)/(5 log2n)
E = Efficiency = S/2n = (2n−2)/[ (5 log2n) (2n) ]
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
93
Speedup
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0 200 400 600 800 1000 12000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
Series1
Speedup and Efficiency
Single Pass
94
Speedup Efficiency
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0 200 400 600 800 1000 12000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
Series1
0 50 100 150 200 250 300 3500
0.01
0.02
0.03
0.04
0.05
0.06
Series1
Speedup and Efficiency
Single Pass
95
With Pipelining. Assume kn equations for large k
T1 = (kn−1) (1 mult + 1 add) = 2kn−2 time units
n leaves 2n processors T2n = (log2n)( 2 mults + 1 add) // pipefill up
+ (k – 2 log 2n) (5) // pipeline on k−2log 2n waves
+ (log2n)( 1 mult + 1 add) // pipedrain
down= 5(k – log2n) time units
S = Speedup = T1/T2n = (2kn−2)/[5(k – log2n )]
E = Efficiency = S/2n = (2kn−2)/[5(k – log2n ) (2n)]
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Speedup and Efficiency
With Pipelining
96
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Speedup and Efficiency
With PipeliningSpeedup
0 200 400 600 800 1000 12000
50
100
150
200
250
300
350
400
450
Series1
97
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Speedup and Efficiency
With PipeliningSpeedup Efficiency
0 200 400 600 800 1000 12000
50
100
150
200
250
300
350
400
450
Series1
0 200 400 600 800 1000 12000.1999
0.2000
0.2001
0.2002
0.2003
Series1
98
Technique Can Work For
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Second-order linear recurrencesx0 = a0
x1 = a1 + b1 x0
x2 = a2 + b2 x1 + c2 x0
...
xn-1 = an-1 + bn-1 xn-2 + cn-1 xn-3
Higher order linear recurrences
99
Technique Can Work For
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Quotients of linear recurrencesx0 = a0
xi = (ai + bi xi-1)/(ci + di xi-1) i=1,2,…, n-1
Other recurrences
100
Summary and Conclusions
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
101
Summary and Conclusions
Chip technology is going to get even better/faster/cheaper for the foreseeable future.
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
102
Summary and Conclusions
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!)
103
Summary and Conclusions
Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!)
More massively parallel processing systems are going to be built and will become even better/faster/cheaper.
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
104
Summary and Conclusions
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!)
More massively parallel processing systems are going to be built and will become even better/faster/cheaper.
The challenging world of parallel algorithmic design beckons and awaits creative minds.
105
Summary and Conclusions
Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!)
More massively parallel processing systems are going to be built and will become even better/faster/cheaper.
The challenging world of parallel algorithmic design beckons and awaits creative minds. Yes, this means you!
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
106
Where in the World is Clemson
University?
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
We are here!
Roy Pargas, Clemson University [email protected]
July 30, 2011
107
Thank you for your kind attention!
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Questions?
108
Extra Slides
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
109
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Links
1. Fujitsu K Computer (K = “kei” = Japanese word for 10 quadrillion)
http://www.fujitsu.com/global/about/tech/k/http://en.wikipedia.org/wiki/K_computer
2. NUDT “Tianhe-1A” Computerhttp://blog.zorinaq.com/?e=36http://en.wikipedia.org/wiki/Tianhe-I
3. Cray Jaguarhttp://en.wikipedia.org/wiki/Jaguar_(computer)http://www.nccs.gov/jaguar/
4. Dawning Nebulaehttp://en.wikipedia.org/wiki/Dawning_Information_Industryhttp://www.theregister.co.uk/2010/05/31/top_500_supers_jun2010/
5. NEC/HP Tsubame 2.0 http://en.wikipedia.org/wiki/TOP500
110
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
111
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
112
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
113
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
114
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
1-Degree
0 1
115
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
1-Degree
0 1
0 1
116
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
1-Degree
0 1
0 1
0 1
117
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
1-Degree
0 1
0 1
0 1
118
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
1-Degree
0 1
00
01
10
11
2-Degree
119
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
3-Degree
0-Degree
1-Degree
0 1
00
01
10
11
2-Degree
120
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
3-Degree
4-Degree
0-Degree
1-Degree
0 1
00
01
10
11
2-Degree
121
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
3-Degree
4-Degree
5-Degree
122
All-to-All Communication
(Hypercube)
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Problem
Motivation