1 synthesis of distributed arraysamir kamil synthesis of distributed arrays in titanium amir kamil...
Post on 19-Dec-2015
229 views
TRANSCRIPT
![Page 1: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/1.jpg)
1Synthesis of Distributed Arrays Amir Kamil
Synthesis of Distributed Arrays in Titanium
Amir Kamil
U.C. BerkeleyMay 9, 2006
![Page 2: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/2.jpg)
2 Amir KamilSynthesis of Distributed Arrays
Background• Titanium is a single program, multiple data
(SPMD) dialect of Java• All threads execute the same program text
• Designed for distributed machines• Global address space – all threads can access
all memory• But much slower to access remote memory than local
memory
![Page 3: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/3.jpg)
3 Amir KamilSynthesis of Distributed Arrays
Grids – The Abstract View• Grids used extensively in scientific codes• Ideally, programmer specifies:
• Size of grid• Operations on each cell
grid[2d] g = new grid[[0,0] : [100,100]];setup(g);for (int i = 0; i < iterations; i++) { foreach (p in g.domain()) { g[p] = (g[p+[0,-1]] + g[p+[0,1]] + g[p+[1,0]] + g[p+[-1,0]]) / 4; }}
![Page 4: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/4.jpg)
4 Amir KamilSynthesis of Distributed Arrays
Grids – The Reality
• Grids must be distributed across processors• Global accesses are slow, local accesses are fast• Load balancing is difficult
• Some problems require multiple levels of refinement
• Access patterns must be tailored for problem and machine
![Page 5: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/5.jpg)
5 Amir KamilSynthesis of Distributed Arrays
Grid Distribution
• Regular partitioning• Blocked, Cyclic distributions
• Can also partition irregularly• Ghost cells at boundaries used to cache data
![Page 6: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/6.jpg)
6 Amir KamilSynthesis of Distributed Arrays
Multi-level Grids
Level 1
Level 2
• Parts of the grid may require higher resolution
• Each level distributed separately
• Lower levels are refinements of upper levels
• Some notion of consistency between levels
![Page 7: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/7.jpg)
7 Amir KamilSynthesis of Distributed Arrays
Access Patterns
• Different problems require different access patterns• Data dependency• Cache effects
• Examples:• Blocked accesses (linear algebra)• Red/black (multigrid)
![Page 8: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/8.jpg)
8 Amir KamilSynthesis of Distributed Arrays
Problem #1 – Game of Life
• 2D grid• Blocked in both dimensions• Ghost cells of width 1 at boundaries
array data { dimension[2]; distribution[BLOCKED(length[1] / 3), BLOCKED(3 * length[2] / Ti.numProcs())]; boundary[GHOST(1), GHOST(1)];}
![Page 9: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/9.jpg)
9 Amir KamilSynthesis of Distributed Arrays
Problem #2 – Knapsack
• 2D grid• Blocked in one dimension• Blocked access pattern in other
dimension• Ghost cells of width 1 at boundaries
array data { dimension[2]; distribution[BLOCKED(length[1] / Ti.numProcs()), NONE]; access[NONE, BLOCKED(2)]; boundary[GHOST(1), NONE];}
![Page 10: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/10.jpg)
10 Amir KamilSynthesis of Distributed Arrays
Grid Usage
• Generated grids mostly used as if they’re normal, global grids• Array access ([], []=) to any cell supported
• Ghost cells automatically updated by calling synchronize() method
• Methods provided to restrict access to local elements, specified pattern• e.g. myDomain(), myBlocks()
![Page 11: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/11.jpg)
11 Amir KamilSynthesis of Distributed Arrays
Future Work
• Optimize certain access patterns in compiler• e.g. can remove owner computation when iterating over
local domainforeach (p in grid.myDomain()) grid[p] = …
• Add basic support for multiple levels of refinement
![Page 12: 1 Synthesis of Distributed ArraysAmir Kamil Synthesis of Distributed Arrays in Titanium Amir Kamil U.C. Berkeley May 9, 2006](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d395503460f94a13c3f/html5/thumbnails/12.jpg)
12 Amir KamilSynthesis of Distributed Arrays
Future Future Work
• Add more distribution types• e.g. Blocked-Cyclic• Irregular partitioning
• Add load balancing• Support irregular grids
• e.g. AMR
• Add other boundary conditions• e.g. shared cells
• Improve compiler support by adding optimizations, analysis