Multi-core Programming

Introduction Topics

Topics

• General Ideas• Moore’s Law• Amdahl's Law• Processes and Threads• Concurrency vs. Parallelism

General Ideas

“Andy giveth, and Bill taketh away.”

No mater how fast hardware gets, software quickly expands of overwhelm the new

hardware performance.

“Technology happens, it's not good, it's not bad. Is steel good or bad?”

“Unfortunately, many programs are so big that there is no one individual who really knows all the pieces, and so the amount of code sharing you get isn't as great. Also, the opportunity to go back and really rewrite something isn't quite as great, because there's always a new set of features that you're adding on to the same program.”

General Ideas

The major processor manufacturers and architectures, from Intel and AMD to Sparc and PowerPC, have run out of room with most of their traditional approaches to boosting CPU performance.

Silicon manufacturers are turning to hyperthreading and multicore architectures.

General Ideas

Ramifications of this change will hit hard in software development.– Typical software solutions can no longer suffice,

relying on hardware for speed increases.– Software will need to adapt to the new hardware

platforms.– Traditional programming models break down in n-

core processor environments.

General IdeasIntel CPU Introductions over time:

• Clock Speed increases until ~2003 then levels off.

• Number of transistors in a processor form continues to increase.

Quad Core I7 2011 1,160,000,0008 Core Itanium 2012 3,100,000,00062 Core Xeon 2012 5,000,000,000GK110 2012 7,100,000,000

General Ideas

Over the past 30 years, CPU designers have achieved performance gains through:– Clock Speed : Doing the

same work faster.– Execution Optimization :

Doing more work per clock cycle.

– Cache : Stay away from RAM as much as possible.

1 : CPU 0 , 2 : Cache L2 CPU 0, 3 : CPU 1, 4 : Cache L2 CPU 15 : System Request Interface, Crossbar Switch, Memory Controller,

Hypertransport

General Ideas

The previously mentioned “speedup” methods are concurrency agnostic methods – i.e. they will work with any sequential code base.

CPU Performance increase hit a wall in 2003 from a traditional approach.

General Ideas

It is becoming increasingly difficult to exploit faster CPU speeds.

Key physical limitations are standing in the way:1. Heat2. Power Consumption3. Current Leakage

Moore’s Law

From: Electronics Magazine – April 1965“The complexity for minimum component costs has

increased at a rate of roughly a factor of two per year ... Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years. That means by 1975, the number of components per integrated circuit for minimum cost will be 65,000. I believe that such a large circuit can be built on a single wafer.”

Moore’s Law Worded differently:

The number of transistors on a chip will double about every two years.

Basically, Moore’s Law became a self fulfilling prophesy.

Moore’s Law

Heat, power consumption, and current leakage are limiting the sheer “transistor” size for creation of a single CPU.

Consumer demands are for smaller, more portable, yet feature rich digital devices.

Moore’s Law

Moore’s predictions still hold – transistor counts continue to rise…

… however, performance gains are going to be accomplished in fundamentally different ways.

Most current applications will no longer benefit from the free “clock cycle” ride without significant redesign.

Moore’s Law

Near future performance gains will be achieved through:– Hyperthreading: Running two or more threads in parallel

inside a single CPU.– Multicore: Running two or more CPUs on a single silicon

form factor (CPU chip).– Cache: Bringing instructions and data across the memory

access bus in blocks and storing “on-die” for fast access.

1 : CPU 0 , 2 : Cache L2 CPU 0, 3 : CPU 1, 4 : Cache L2 CPU 15 : System Request Interface, Crossbar Switch, Memory Controller,

Hypertransport

Amdahl’s LawAmdahl's law states that the overall

speedup of applying the improvement will be:

P - proportion of a computation where the improvement has a speedup of S.

Example:

– If an improvement can speed up 30% of the computation, P will be 0.3

– If the improvement makes the portion affected twice as fast, S will be 2.

– Adding a second processor to a multithreaded application sets S to 2.

The speedup of a program using multiple processors in parallel computing is limited by the sequential fraction of the program.

Amdahl’s Law

To get rich applications running on shrinking form factors, Moore’s Law and Amdahl’s Law become increasingly important in the design of software based systems.

Moore’s law dictates continued growth of CPU systems – driven toward multicore because of current physical limitations.

Amdahl’s law dictates the extent to which multiple cores will increase software system performance.

Processes and Threads

• Modern operating systems load programs as processes

• Resource holder• Execution

• A process starts executing at its entry point as a thread

• Threads can create other threads within the process– Each thread gets its own stack

• All threads within a process share code & data segments

Code segment

Data segment

thread

main()

…thread thread

Stack Stack

Stack

Concurrency vs. Parallelism– Concurrency: two or more threads are in progress

at the same time:

– Parallelism: two or more threads are executing at the same time

– Multiple cores needed

Thread 1

Thread 2

Thread 1

Thread 2

Trends

• Multicore programming is here to stay• Multicore programming is more than multithreaded

programming– A poorly written multithreaded program can cause

performance decrease in a multicore environment– Software must be carefully designed to execute properly in a

multicore environment• Technology “pundits” are claiming multicore programming

is the “new OO”.• Multicore programming is being touted as “a more

important leap in software development than that of OO”.