module r r rrr r rrrrr rr r r r r access regulation to hot-modules in wormhole nocs isask’har...

Module

Module

Module

Module

Module

Module

Module

Module

Module Module Module

Module

Module

Module

R

R

R R R

R

RR R R R

R R

R

Module

R

R

R

Access Regulation toHot-Modules in Wormhole NoCs

Isask’har (Zigi) Walter

Supervised by:Israel Cidon, Ran Ginosar and Avinoam Kolodny

Or: Hot-Modules, Cool NoCs

Technion – Israel Institute of Technology

May 2007Hot-Modules in Wormhole

NoCs 2

Hot-Modules

NoC is designed and dimensioned to meet QoS requirements- Buffer sizing, routing, router arbitration, link capacities, …

NoC designers cannot tune everything- Modules typically have limited capacity

High-demanded, bandwidth limited modules create edge bottlenecks- In SoC, often known in advance

Off-chip DRAM, on-chip special purpose processor

System performance is strongly affected- Even if the NoC has infinite bandwidth


NoCs 3

Hot Module (HM) in NoC Wormhole, BE NoC

At high Hot Module utilization, multiple worms “get stuck” in the network

Two problems arise:- System Performance- Source Fairness

IP(HM) In

terf

ace


NoCs 4

IP3Interface

IP2

Inte

rfa

ce

IP1(HM) In

terf

ace

HM is not a local problem. Traffic not destined at the HM suffers too!

Hot Module Affects the SystemProblem

#1


NoCs 5

Multiple locally fair decisions

Global fairness

HM

Inte

rfac

e

The limited, expensive HM resource isn’t fairly shared

Source Fairness ProblemProblem

#2


NoCs 6

Our Approach

Problem is not caused by the NoC- But rather by a congested end-point

Solution should address the root cause- Not the symptoms

Utilize existing NoC infrastructure

Solve both problems- Simple and efficient


NoCs 7

Hot Module Congestion

During congested periods, sources should not inject packets towards the HM- Will experience increased delay anyway- Better wait at the source, not in the network

Keep routers unmodified!


NoCs 8

IP1

Control

IP4

NoC

Interface

Interface

IP3

IP2(HM)

HM Allocation Control Basics

Inte

rfac

eA

llocati

on

Con

trolle

r

Interface


NoCs 9

IP1

IP4

NoC

Interface

Interface

IP3

IP2(HM)

Inte

rfac

eControl


Allocati

on

Con

trolle

r

Interface


NoCs 10

IP1

Control

NoCIP2

(HM)

Allocati

on

Con

trolle

r

Interface

IP3

IP4

Interface


Inte

rfac

e

Interface


NoCs 11

HM Control Packets

The HM Controller receives all requests and can employ any scheduling policy

Control packets are sent using a high service level- Bypassing (blocked) data packets!

Dest.

Req. C

redit

Source

Dest.

Credit

Source

Credit request packet Credit reply packet


NoCs 12

Input ports Output ports

BufSize

SL 0

SL 1

CR

OS

S-B

AR

Scheduler CREDITControlCREDIT

SL 2

SL 3

SL 0

SL 1

SL 2

SL 3

Multiple Priority Router

Control packets


NoCs 13

Enhanced Request packet The request may include additional data as

needed- payload’s priority, deadline, expiration time, etc.

Dest.

Deadline

Expiration

Priority

Req. C

redit

Source

……

Optional fields

Credit request packet


NoCs 14

SRC

Size

Priority

deadline

Expiration……

The HM Allocation Controller is customized according to system’s requirements

HM Allocation Controller

PendingRequests

Table

LocalArbiter

CreditRequests

CreditReplies

Requests Decoder

Reply Encoder

Optional

HM Access Controller


NoCs 15

Short packets are not negotiated Source’s quota is slowly self-refreshing The mechanism is turned-off when the

network is not congested Crediting modules ahead of time hides

request-grant latency- For light-load periods

Further Enhancements


NoCs 16

Not Classic Flow-Control

Flow-control protects destination’s buffer- A pair-wise protocol

HM access regulation protects the system- Many-to-one protocol


NoCs 17

Results – Synthetic scenario Hotspot traffic

- All-to-one traffic with all-to-all background traffic

High network capacity Limited hot module bandwidth HM controller arbitration: Round-robin

Module

Module

HM

Module

Module

Module

Module

Module

ModuleModule Module Module

ModuleModule Module Module

R

R

R

R

R R

R

RR R R

RR R R

R


NoCs 18

System Performance

Without regulation

WithRegulation

X30

X10

Average Packet Latency


NoCs 19

Hot vs. non-Hot Module Traffic

HM Trafficwithout regulation

Background TrafficWithout regulation

HM Trafficwith regulation

Background TrafficWith regulation

Using regulation, non-HM traffic latency is drastically reduced

X40

Average Packet Latency


NoCs 20

Source Fairness

Source#16no regulation

Source#5no regulation

Source#5with regulation

Source#16with regulation

2

6

1

5

3

7

4

8

109 11 12

1413 15 16

R

R

R

R

R R

R

RR R R

RR R R

R


NoCs 21

Fairness in Saturated Network

Hot-Module Utilization: 99.99% Regulated Hot-Module Utilization: 98.32%

Simulation results for a 4-by-4 system,Data packet length: 200 flitsControl packet length: 2 flits

No allocation controlWith allocation control


NoCs 22

MPEG-4 Decoder

Real SoC Over provisioned NoC Two hot-modules

VU AU MED CPU

RAST

SDRAM SRAM1 SRAM2 IDCT

ADSP UP SAMP

BAB RISC

25% of all traffic

22% of all traffic

SDRAM SRAM2


NoCs 23

Results – MPEG-4 Decoder

@80% load: X2 reduction @80% load: X8 reduction

All traffic HM/non-HM traffic breakdown

X2

X8


NoCs 24

The HMs are better utilized

Without regulation, the hot-modules are only 60% utilized- Traffic to one HM blocks the traffic to the other!

No allocation controlWith allocation control

1HM1 2HM1 3HM1 4HM1 9HM1 10HM1 11HM1 8HM2 10HM2 11HM2 12HM2 Total

Flows destined at HM1

Significant differences in BW!

Flows destined at

HM2


NoCs 25

Hot-Module Placement


NoCs 26

Summary Hot-modules are common in real SoCs

Hot-modules ruin system performance and are not fairly shared- Even in NoCs with infinite capacity- The network intensifies the problem- But can also provide tools for resolving it

Simple mechanism achieves dramatic improvement- Completely eliminating the HM effects

Hot-Modules, Cool NoCs!


NoCs 27

Thank you!

Questions?

[email protected]

Hot-Modules, Cool NoCs!

M odule

M odule M odule

M odule M odule

M odule M odule

M odule

M odule

M odule

M odule

M odule

QNoCResearch

GroupGroup

ResearchQNoC

module r r rrr r rrrrr rr r r r r access regulation to hot-modules in wormhole nocs isask’har...

Documents

nonhm tr

creditenhanced request

bandwidth limited modules

expensive hm resource

congestedcrediting modules

credit request packetcredit

noc designers

router arbitration