![Page 2: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/2.jpg)
Cluster Computing
Overview
• Introduction • Goal/Idea • Phases • Assignments • Tools • Timeline/Exam • General info
![Page 3: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/3.jpg)
Cluster Computing
Introduction • Supercomputers are expensive • Workstations are cheap • Supercomputers from workstations are
cheap (but they may be hard to program)
• Several approaches to this problem exist
![Page 4: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/4.jpg)
Cluster Computing
Goal/Idea • The goal is that after this class you will be
able to solve supercomputing class problems using a 'cheap' cluster solution
• The idea is that the only real way to learn this is by solving a set of problems - using a cluster
![Page 5: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/5.jpg)
Cluster Computing
Approach
• Model classic supercomputer architectures to clusters
• Port the basic software from the SC architectures
• Emulate the hardware that might be missing
![Page 6: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/6.jpg)
Cluster Computing
Phases • The class will cover five distinct approaches
to cluster computing: • Physical Shared Memory (not a cluster
actually) • Abstract Machines • Emulated Massively Parallel Processors • Emulating Remote Memory Machines • Distributed Shared Memory
![Page 7: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/7.jpg)
Cluster Computing
Evolution
![Page 8: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/8.jpg)
Cluster Computing
Tools • All the tools will be available in time (I hope) • We will (primarely) use Python for programming
• In fact the choice of language is free but one language will be recommended for each step
• Other tools that will be covered are: • Parallel Virtual Machines • Message Passing Interface • Remote Method Invocation • Tspaces, PastSet and TMem
![Page 9: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/9.jpg)
Cluster Computing
Tools • Most tools are available in identical or
similar form for use with other languages • All the tools should be considered
experimental and problems should be reported to me ASAP
![Page 10: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/10.jpg)
Cluster Computing
Litterature
• Notes • Papers • I can suggest books as they are needed
![Page 11: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/11.jpg)
Cluster Computing
Assignments • There will be five (5) programming
assignments, one for each phase we cover • All assignments will be fun!
![Page 12: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/12.jpg)
Cluster Computing
Assignments • This is a class on cluster computing, thus the
assignments will be accompanied by a sequential version of the problem, that you can work from
• Assignments must be documented and a report (there will be a page limit) submitted with the code
• In the end you only need to hand in 3 assignments
![Page 13: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/13.jpg)
Cluster Computing
Assignments
![Page 14: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/14.jpg)
Cluster Computing
General Idea • We cannot hope to implement one SC
application within one class – let alone five! • Most SC applications are based on a
computational kernel which is fairly small and which takes up as much as 99.999...% of the total runtime
• You will port sequential verisons of such kernels to run on clusters
• To make it all fun we simulates cartoon traps
![Page 15: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/15.jpg)
Cluster Computing
Road Map
• Fractals are examples of applications of the type we call embarrassingly parallel
• A typical example of an compute intensive application with many independent sub-results
• Very simple to write • Achieves very good speedup
![Page 16: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/16.jpg)
Cluster Computing
Road Map
![Page 17: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/17.jpg)
Cluster Computing
Race Trap
• Traveling Salesman Problem is a classic supercomputing problem
• The chosen algorithm is a typical Producer-Consumer approach
• Is representative for global optimization problems
• May achieve good speedup
![Page 18: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/18.jpg)
Cluster Computing
Race Trap
![Page 19: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/19.jpg)
Cluster Computing
Wind Trap
• Virtual wind-tunnel • An actual scientific application • 2D – but a 3D model exists that use the
same access pattern • Represents the pipelined application type • Can achieve really good speedup
![Page 20: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/20.jpg)
Cluster Computing
Wind Trap
![Page 21: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/21.jpg)
Cluster Computing
Frosty Trap
• Successive Over Relaxation • A very common computational kernel in
many scientific applications • A typical example of grid-communication
applications • Can achieve very good speedup
![Page 22: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/22.jpg)
Cluster Computing
Frosty Trap
![Page 23: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/23.jpg)
Cluster Computing
Clone Machine
• Ray tracing is a real computational problem
• An example of an application that can achieve perfect speedup with small problems and good speedup on large (real) problems
![Page 24: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/24.jpg)
Cluster Computing
Clone Machine
![Page 25: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/25.jpg)
Cluster Computing
eScience Track • eScience is a new field at KU
– Masters of eScience (cand scient escience) will start September 1st
• Starting next year this class will be given in the eScience context – Thus there are new assignments on their way
• You can choose to do these assignments instead of the original ones – The older ones are well tested the new ones are
not!!!
![Page 26: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/26.jpg)
Cluster Computing
Tumor treatment
• Monte Carlo simulations are examples of applications of the type we call embarrassingly parallel
• A typical example of an compute intensive application with many independent sub-results
• Very simple to write • Achieves very good speedup
![Page 27: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/27.jpg)
Cluster Computing
Tumor treatment
![Page 28: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/28.jpg)
Cluster Computing
Tumor treatment
![Page 29: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/29.jpg)
Cluster Computing
Protein Folding
• Protein is an actual supercomputing problem
• The chosen simplifications and algorithm is a typical Producer-Consumer approach
• Is representative for global optimization problems
• May achieve good speedup
![Page 30: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/30.jpg)
Cluster Computing
Protein folding
![Page 31: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/31.jpg)
Cluster Computing
Wind Tunnel
• Virtual wind-tunnel • An actual scientific application • 2D – but a 3D model exists that use the
same access pattern • Represents the pipelined application type • Can achieve really good speedup
![Page 32: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/32.jpg)
Cluster Computing
Wind Tunnel
![Page 33: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/33.jpg)
Cluster Computing
Heat Equation
• Successive Over Relaxation • A very common computational kernel in
many scientific applications • A typical example of grid-communication
applications • Can achieve very good speedup
![Page 34: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/34.jpg)
Cluster Computing
Heat Equation
![Page 35: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/35.jpg)
Cluster Computing
NBody Dynamics
• An actual application • Widespread use with different forces
• Can be pipelinede • And achieve really good speedup
![Page 36: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/36.jpg)
Cluster Computing
NBody Simulation
![Page 37: Introduction to Cluster Computing - Aalborg Universitetpeople.cs.aau.dk/~bt/PhDSuperComputing2011... · · 2010-02-02Cluster Computing Introduction ... • We will (primarely) use](https://reader031.vdocuments.mx/reader031/viewer/2022022009/5ae9c4307f8b9a6d4f912c50/html5/thumbnails/37.jpg)
Cluster Computing
Exam
• 3 of your assignments written into one delivery