hot topics in parallel computing -...

Hot Topics in Parallel ComputingProf. Richard (Rich) Vuduc

CSE 8803-HPC

[01] Tuesday, August 23, 2010

Tuesday, August 24, 2010

Introductions

Who?

Program & year?E.g., CS PhD, AE MS, …

One interest?E.g., dissertation topic, “compilers,” bowling, …

2


Origins & philosophy:

This course gets its name from an annual workshop called, “Hot Topics in Parallelism (HotPar).”

The spirit of HotPar is to discuss and debate the path forward for parallel computing, which many believe is at a kind of turning point.

Of course, this depends strongly on YOU!


Position:Parallel computing is a 50 year old field. Yet, now is the time to study it.

Question:Really? Why or why not?




Applications!– Data & compute intensive

Hardware!– Latency and power

Q: Software in-between?


“Extreme scale” systems:Climate modeling

This laptop ~ 1010 ops / s

Today: “Petascale” systems~ 1015 ops/s

To predict climate change 10-50 years into the future, need ~ 1018 ops/s system –“Exascale”

Need 108 = 100 million laptops

6No one yet knows how to build or the “best” way to program such a system!


Meeting Diarist Application“Parlab All”

3

Opportunities and Challenges of Parallelizing Speech Recognition.Jike Chong, UC Berkeley; Gerald Friedland, Adam Janin, Nelson Morgan, and Chris Oei, ICSI @ BerkeleyIn HotPar, June 2010. – http://www.usenix.org/events/hotpar10/tech/


http://www.usenix.org/events/hotpar10/tech/

http://www.usenix.org/events/hotpar10/tech/

2-core CPU


2-core CPU+ 32-core GPU (↑perf+power)


2-core CPU+ 32-core GPU (↑perf+power)+ 16-core GPU (↓perf+power)


26-core smartphone. Chinese entrepreneur Steve Chao to showcase the Ouidoo at the Shanghai World Expo. [engadget.com]

12


Phones for HPC?

13




(from H&P 4th, App. K)

“The turning away from the conventional organization came in the middle 1960s … Electronic circuits are ultimately limited in their speed of operation by the speed of light …”– W. Jack Bouknight, Illiac IV System (1972)

“… sequential computers are approaching a fundamental physical limit on their potential computational power. Such a limit is the speed of light …”– Angela L. DeCegama, The Technology of Parallel Processing, Volume I (1989)

“… today’s multiprocessors … are nearing an impasse as technologies approach the speed of light …”– David Mitchell, The Transputer: The Time is Now (1989)


We’ve been here before:The Dead Supercomputer Society

ACRIAlliantAmerican SupercomputerAmetekApplied DynamicsAstronauticsBBNCDCConvexCray ComputerCray ResearchCuller-HarrisCuller ScientificCydrome

Dana/Ardent/Stellar/StardentDenelcorElxsiETA SystemsEvans and Sutherland Computer DivisionFloating Point SystemsGalaxy YH-1Goodyear Aerospace MPPGould NPLGuiltech

Intel Scientific ComputersInternational Parallel MachinesKendall Square ResearchKey Computer LaboratoriesMasParMeikoMultiflowMyriasNumerixnCubePrisma

Thinking MachinesSaxpyScientific Computer Systems (SCS)Silicon Graphics Inc (now Rackable)Soviet SupercomputersSun (bought by Oracle)SupertekSupercomputer Systems (SSI)SuprenumVitesse Electronics

15Industry is placing a huge bet on parallelism. Are we ready? Will we succeed?


http://www.paralogos.com/DeadSuper/ACRI/index.html

http://www.paralogos.com/DeadSuper/ACRI/index.html

http://www.paralogos.com/DeadSuper/Alliant/index.html

http://www.paralogos.com/DeadSuper/Alliant/index.html

http://www.paralogos.com/DeadSuper/Misc/AmSup.html




http://www.paralogos.com/DeadSuper/Misc/Ametek.html

http://www.paralogos.com/DeadSuper/Misc/Ametek.html

http://www.paralogos.com/DeadSuper/Misc/ADI.html

http://www.paralogos.com/DeadSuper/Misc/ADI.html

http://www.paralogos.com/DeadSuper/Misc/BBN.html

http://www.paralogos.com/DeadSuper/Misc/BBN.html

http://www.paralogos.com/DeadSuper/Convex/index.html

http://www.paralogos.com/DeadSuper/Convex/index.html

http://www.paralogos.com/DeadSuper/Cray/index.html

http://www.paralogos.com/DeadSuper/Cray/index.html

http://www.paralogos.com/DeadSuper/Culler/index.html

http://www.paralogos.com/DeadSuper/Culler/index.html

http://en.wikipedia.org/wiki/Cydrome

http://en.wikipedia.org/wiki/Cydrome

http://en.wikipedia.org/wiki/Ardent_Computer




http://en.wikipedia.org/wiki/Elxsi

http://en.wikipedia.org/wiki/Elxsi

http://www.paralogos.com/DeadSuper/ETA/index.html

http://www.paralogos.com/DeadSuper/ETA/index.html

http://www.paralogos.com/DeadSuper/ESCD/index.html






http://en.wikipedia.org/wiki/Floating_Point_Systems




http://www.paralogos.com/DeadSuper/Misc/Gould.html

http://www.paralogos.com/DeadSuper/Misc/Gould.html

http://en.wikipedia.org/wiki/Kendall_Square_Research




http://www.paralogos.com/DeadSuper/Misc/Key.html




http://en.wikipedia.org/wiki/MasPar

http://en.wikipedia.org/wiki/MasPar

http://www.meiko.com/

http://www.meiko.com/

http://en.wikipedia.org/wiki/Multiflow

http://en.wikipedia.org/wiki/Multiflow

http://en.wikipedia.org/wiki/NCUBE

http://en.wikipedia.org/wiki/NCUBE

http://en.wikipedia.org/wiki/Thinking_Machines

http://en.wikipedia.org/wiki/Thinking_Machines

http://www.paralogos.com/DeadSuper/Soviet/




http://en.wikipedia.org/wiki/Supertek

http://en.wikipedia.org/wiki/Supertek

Course Logistics

Rich Vuduc (instructor), richie@gatech or AIM: VuducOfficeHoursJonathan Brownsworth (TA), j.brownsworth@gatech

Sign up for course website: gtcse8803hpc.ning.com

Read 2–4 papers per week.

Submit two paragraph written summaries as “blogs” on ning site.

We all take turns leading discussions each week and “blogging” a summary of it (again, on ning site)

Do a research-oriented final project; think “publication”

16


Grading

Class participation: 25%

Write-ups: 25% – May miss 3 days without penalty

Final project: 50%

17


Read an submit write-up before class

Write-up format: Concise (2 paragraph) summary & critique

What is the main claimed contribution?What do you think is the real contribution?

Strengths or weaknesses in the methodology?

What new questions are raised by this paper?

Practice for “real” life research and reviewing

18

Readings and your write-ups


Thursday 8/26: No class. Instead, go to theNVIDIA CUDA Center of Excellence Kick-off in Klaus 1116.

Assignment: Go any time between 9am to 3:30pm, and write-up a one to two-paragraph of what you learn

Post on your ning.com blog before next class (8/31)

19


Week Tuesday

Aug 24: Intro Aug 26: NVIDIA CCOE Kick-off

Aug 31: Rich Sep 2: Rich / Jonathan

Sep 7: NO CLASS Sep 9: NO CLASS

Sep 14: Regular class Sep 16: Regular class


hot topics in parallel computing -...

Documents