massively parallel architectures

1

Massively Parallel Architectures

Colin Egan, Jason McGuiness and Michael Hicks(Compiler Technology and

Computer Architecture Research Group)

2

Presentation Structure

Introduction

Conclusions

Questions? (ask as we go along and I’ll also leave time for questions at then end of this presentation)

3

The memory wall

High performance computing can be achieved through instruction level parallelism (ILP) by:

Pipelining,Multiple instruction issue (MII).

But both of these techniques can cause a bottle-neck in the memory subsystem, which may impact on overall system performance.

The memory wall.

4

Overcoming the memory wall

We can improve data throughput and data storage between the CPU and the memory subsystem by introducing extra levels into the memory hierarchy.

L1, L2, L3 Caches.

5


But …This increases the penalty associated with a miss in the memory subsystem.Which impacts on the average cache access time.Where:

average cache access time = hit-time + miss-rate * miss-penalty

for two levels of caches:average access time = hit-timeL1 + miss-rateL1 * hit-timeL2 + miss-rateL2 * miss-penaltyL2because themiss-penaltyL1 = hit-timeL2 + miss-rateL2 * miss-penaltyL2

for three levels of caches:average access time = hit-timeL1 + miss-rateL1 * hit-timeL2 + miss-rateL2 * hit-timeL3 +

miss-rateL3 * miss-penaltyL3

6


The average cache access time is therefore dependent on three criteria: hit-time, miss-rate and miss-penalty.

Increasing the number of levels in the memory hierarchy does not improve main memory access time.

Recovery from a cache miss causes a main memory bottle-neck, since main memory cannot keep up with the required fetch rate.

7


The overall effect:can limit the amount of ILP,can impact on processor performance,increases the architectural design complexity,increases hardware cost.

8


So perhaps we should improve the performance of the memory subsystem.

Or we could place processors in memory.Processing In Memory (PIMs).

9

Processing in memory

The idea of PIM is to overcome the bottleneck between the processor and main memory by combining a processor and memory on a single chip.

The benefits of PIM are:reduced latency,a simplification of the memory hierarchy,high bandwidth,multi-processor scaling capabilities

Cellular architectures.

10


Reduced latency:In PIM architectures accesses from the processor to memory:

do not have to go through multiple memory levels (caches) to go off chip,do not have to travel along a printed circuit line,do not have to be reconverted back down to logic levels,do not have to be re-synchronised with a local clock.

Currently there is about an order of magnitude reduction in latencythereby overcoming the memory wall.

11


Beneficial side-effects of reducing latency:total silicon area is reduced,power consumption is reduced.

12


But …processor speed is reduced,the amount of available memory is reduced.

13

Cellular architectures

PIM is easily scaled,multiple PIM chips connected together forming a network of PIM cells (Cellular architectures) to overcome these two problems.

14


Cellular architectures are threaded:each thread unit is independent of all other thread units,each thread unit serves as a single in-order issue processor,each thread unit shares computationally expensive hardware such as floating-point units,there can be a large number (1,000s) of thread units – therefore they are massively parallel.

15


Cellular architectures have irregular memory access:

some memory is very close to the thread units and is extremely fast,some is off-chip and slow.

Cellular architectures, therefore, use caches and have a memory hierarchy.

16


In Cellular architectures multiple thread units perform memory accesses independently.

This means that the memory subsystem of Cellular architectures requires some form of memory access model that permits memory accesses to be effectively served.

17


Examples:Bluegene Project

Cyclops (IBM)DIMES

Gilgamesh (NASA)Shamrock (Notre Dame)

18

Cyclops

Developed by IBM at the Tom Watson Research Center.Also called Bluegene/C in comparison with the later version of Bluegene/L.

19

Logical View of the Cyclops64 Chip Architecture

Processor

ChipBoard

Crossbar Network

TU TU TU…

SP SP SP

FPU SR

TU TU TU…

SP SP SP

FPU SR

TU TU TU…

SP SP SP

FPU SR

…

MEM

OR

YB

AN

K

MEM

OR

YB

AN

K

MEM

OR

YB

AN

K

MEM

OR

YB

AN

K

MEM

OR

YB

AN

K

MEM

OR

YB

AN

K

MEM

OR

YB

AN

K

MEM

OR

YB

AN

K

…

6 * 4 GB/sec

4 GB/sec

50 MB/sec

1 Gbits/sec

Off-

Chi

p M

emor

y

OtherChips via 3D mesh

Off-

Chi

p M

emor

yO

ff-C

hip

Mem

ory

Off-

Chi

p M

emor

y

IDEHDD

4 GB/sec

6 * 4 GB/sec

SP SP SP SP SP SP SP SP

MSOffice2

Processors : 80 / chipThread Units (TU) : 2 / processorFloating-Point Unit (FPU) : 1 / processorShared Registers (SR) : 0 / processorScratch-Pad Memory (SP) : n KB / TU

On-chip Memory (SRAM) : 160 x 28KB = 4480 KBOff-chip Memory (DDR2) : 4 x 256MB = 1GBHard Drive (IDE) : 1 x 120GB = 120 GBSP is located in the memory banks and is accessed directly by the TU and via the crossbar network by the other TUs.

MSOffice2 NOTE ABOUT SCRATCH-PAD MEMORIES:

There is one memory bank(MB) per TU on the chip. One part of the MB is reserved for the SP of the corresponding TU. The size of theSP is customizable.

The TU can access its SP directly without having to use the crossbar network with a latency of 3/2 for ld/st. The other TUs can access the SP of another TU by accessing the corresponding MB through the network with a latency of 22/11 for ld/st. Even TUs from the same processor must use the network to access the SPs of each other.

The memory bank can support only one access per cycle. Therefore if a TU accesses its SP while another TU accesses the same SP through the network, some arbitration will occur and one of the two accesses will be delayed. , 03/10/2003

20

DIMES

Delaware Interactive Multiprocessor Emulation System (DIMES)

under development by the Computer Architecture and Parallel Systems Laboratory (CAPSL) at the University of Delaware, Newark, DE. USA under the leadership of Prof. Guang Gao.

21

DIMES

DIMESis the first hardware implementation of a Cellular architecture,is a simplified ‘cut-down’ version of Cyclops,is hardware validation tool for Cellular architectures,emulates Cellular architectures, in particular Cyclops, cycle by cycle, is implemented on at least one FPGA,has been evaluated by Jason McGuiness.

22

DIMES

The DIMES implementation that Jason evaluated:supports a P-thread programming model,is a dual processor

where each processor has four thread units, has 4K of scratch-pad (local) memory per thread unit,has two banks of 64K global shared memory,has different memory models:

scratch pad memory obeys the program consistency model for all of the eight thread units,global memory obeys the sequential consistency model for all of the eight thread units,

is called DIMES/P2.

23

DIMES

DIMES/P2.

64K GlobalMemory

Thread Unit 0

4K Scratch pad

Thread Unit 1

4K Scratch pad

Thread Unit 2

4K Scratch pad

Thread Unit 3

4K Scratch pad

Processor 0

64K GlobalMemory

Thread Unit 0

4K Scratch pad

Thread Unit 1

4K Scratch pad

Thread Unit 2

4K Scratch pad

Thread Unit 3

4K Scratch pad

Network

Processor 1

24

Gilgamesh

This material is based on:Sterling TL and Zima HP, Gilgamesh: A Multithreaded Processor-In-Memory Architecture for Petaflops Computing, IEEE, 2002.

25

Gilgamesh

PIM-based massively parallel architectureExtend existing PIM capabilities by:

Incorporating advanced mechanisms for virtualizing tasks and data.Providing adaptive resource management for load balancing and latency tolerance.

26

Gilgamesh System Architecture

Collection of MIND chips interconnected by a system network

MIND: Memory, Intelligence, Network DeviceA MIND chip contains:

Multiple banks of DRAM storageProcessing logicExternal I/O interfaceInter-chip communication channels

27

Gilgamesh System Architecture

Fig 1 S&Z

28

MIND Chip

MIND chip is multithreaded.MIND chips communication:

Via parcels,Parcel can be used to perform conventional operation as well as to invoke methods on a remote chip.

29

MIND Chip Architecture

Fig 2 S&Z

30

MIND Chip Architecture

Nodes provide storage, information processing and execution control.Local wide registers serve as thread state, instruction cache, vector memory row buffer.Wide ALU performs multiple operations on separate field simultaneously.System memory bus interface connects MIND to workstation and server motherboard.Data streaming interface deals with rapid high data bandwidth movement.On-chip communication interconnects allow sharing of pipelined FPUs among nodes.

31

MIND Node Structure

Fig 3 S&Z

32

MIND Node Structure

Integrating memory block with processing logic and control mechanism.Wide ALU is 256 bits, supporting multiple operations.ALU doesn’t provide explicit floating point operations.Permutation network rearrange bytes in a complete row.

33

MIND PIM Architecture

High degree of memory bandwidth is achieved byAccessing an entire memory row at a time: data parallelism.Partitioning the total memory into multiple independently accessible blocks: multiple memory banks.

Therefore:Memory bandwidth is square order over off-chip architecture.Low latency.

34

Parcels

A variable length of communication packet that contains sufficient information to perform a remote procedure invocation.It is targeted to a virtual address which may identify individual variables, blocks of data, structures, objects, or threads, I/O streams.Parcel fields mainly contain: destination address, parcel action specifier, parcel argument, continuation, and housekeeping.

35

Multithreaded Execution

Each thread resides in a MIND node wide register.The multithread controller determines:

When the thread is ready to perform next operation.Which resource will be required and allocated.

Threads can be created, terminated or destroyed, suspended, and stored.

36

Shamrock Notre Dame

The material is based on:Kogge PM, Brockman JB, Sterling T, and GaoG, Processing in memory: chips in petaflops. Presented in the Workshop of ISCA 97’.

37

Shamrock Notre Dame

A PIM architecture developed in the University of Notre Dame

38

Shamrock Architecture Overview

Fig 2 Kogge

39

Node Structure

Fig 3 Kogge

40

Node Structure

A node consists of logic for a CPU, four arrays of memory and data routing

Two separately addressable memories on each face of the CPU.

Each memory array is made up of multiple sub-units:

allows all bits read from a memory in one cycle to be available at the same time to the CPU logic.

41

Shamrock Chip

Nodes are arranged in rows, staggering so that the two memory arrays on one side of one node impinge directly on the memory arrays of two different nodes in the next row:

Each CPU has a true shared memory interface with four other CPUs.

42

Shamrock Chip Architecture

A PIM architecture developed in the University of Notre Dame

43

Shamrock Chip

Off-chip connections can:Connect to adjoining chips in the same tile topology.Allow a processor on the outside to view the chip as memory in the conventional sense.

44

Massively Parallel Architectures

Questions?

Colin Egan, Jason McGuiness and Michael Hicks(Compiler Technology and

Computer Architecture Research Group)

massively parallel architectures

Documents