yeditepe universityyeditepe universitygkucuk/fall2012/cse427/lecture4_pipelining.pdfyeditepe...
TRANSCRIPT
CSE 427: CSE 427: Computer ArchitectureComputer Architecture
PipeliningPipeliningPipeliningPipeliningInstructor: Gürhan KüçükInstructor: Gürhan Küçük
Yeditepe UniversityYeditepe University
Lecture notes based on notes by Mark D HillLecture notes based on notes by Mark D. Hill and John P. ShenUpdated by Mikko Lipasti
PipeliningPipelining
ForecastBi Pi– Big Picture
– Datapath– Control– Data Hazards
StallsForwarding
– Control Hazards– Exceptions
© Hill, Shen, Lipasti2
p
MotivationMotivationInstructions Cycles TiInstructions Cycles
Program InstructionTimeCycle
( d i N)
X X
(CPI) ( l ti )
Single cycle implementation(code size N) (CPI) (cycle time)
g y p– CPI = 1
Cycle = imem + RFrd + ALU + dmem +– Cycle = imem + RFrd + ALU + dmem + RFwr + muxes + controlE 500+250+500+500+250+0+0 2000– E.g. 500+250+500+500+250+0+0 = 2000ps
– Time/program = N x 2ns
© Hill, Shen, Lipasti3
Idea!Idea!We use 500 MHz clock with 2ns cycle timetimeBut, we can reduce cycle time and divide the task into several stages so we canthe task into several stages so we can speed things up
i 0 5 l i 2 GH l k– i.e. 0.5ns cycle time => 2 GHz clock
© Hill, Shen, Lipasti4
MulticycleMulticycle
Multicycle implementation:Cycle:Instr:
1 2 3 4 5 6 7 8 9 10
11
12
13Instr:
i F D X M Wi+1 F D Xi+1 F D Xi+2 F D X Mii+3 Fi+4
© Hill, Shen, Lipasti5
But…But…
Multicycle implementation– CPI = 3, 4, 5– Cycle = max(memory, RF, ALU, mux, control)– =max(500,250,500) = 500ps– Time/prog = N x 4 x 500 = N x 2000ps = N x 2ns
Would like:– CPI = 1 + overhead from hazards (later)– Cycle = 500ps + overhead– In practice, ~3x improvement
© Hill, Shen, Lipasti6
Big PictureBig Picture
Instruction latency = 5 cyclesInstruction throughput = 1/5 instr/cycleCPI = 5 cycles per instructionCPI 5 cycles per instructionInstead
Pi li i i i lik l h– Pipelining: process instructions like a lunch buffet
– ALL microprocessors use itE.g. Pentium-IV, Athlon, Power4
© Hill, Shen, Lipasti7
Big PictureBig Picture
Instruction Latency = 5 cycles (same)Instruction throughput = 1 instr/cycleCPI = 1 cycle per instructionCPI 1 cycle per instructionCPI = cycle between instruction completion = 1completion = 1
© Hill, Shen, Lipasti8
Ideal PipeliningIdeal PipeliningComb LogicComb. Logicn Gate DelayL BW = ~(1/n)
GateDelayL Gate
DelayLn--2n--2 BW = ~(2/n)
GateDelayL Gate
DelayL GateDelayLn--3
n--3n--3 BW = ~(3/n)
Bandwidth increases linearly with pipeline depth
Delay y y3 3 3 ( )
Bandwidth increases linearly with pipeline depthLatency increases by latch delays
© Hill, Shen, Lipasti9
Ideal PipeliningIdeal PipeliningCycle:Instr:
1 2 3 4 5 6 7 8 9 10
11
12
13Instr: 0 1 2 3
i F D X M Wii+1 F D X M Wi+2 F D X M Wi+3 F D X M Wi+4 F D X M W
© Hill, Shen, Lipasti10
Pipelining IdealismsPipelining Idealisms
Uniform subcomputationsi li i i h l d l– Can pipeline into stages with equal delay
Identical computations– Can fill pipeline with identical work
Independent computations– No relationships between work units
Are these practical?– No, but can get close enough to get significant
speedup
© Hill, Shen, Lipasti11
ComplicationsComplications
Datapathi ( ) i i i fli h– Five (or more) instructions in flight
Control– Must correspond to multiple instructions
Instructions may have – data and control flow dependences (coming up!)– i.e. units of work are not independent
One may have to stall and wait for another
© Hill, Shen, Lipasti12
An ExampleAn ExampleConsider an unpipelined machine. Assume that it has 20 ns clock and that it useshas 20 ns clock and that it uses
cycles for execution frequencyALU 2 30%Branch 5 20%Memory 7 50%
Suppose that due to clock skew and setupSuppose that due to clock skew and setup, pipelining the machine adds 2 ns overhead to the clock. Ignoring any latency impact, how much
d i h i i i illspeedup in the instruction execution rate will we gain from a pipeline?
© Hill, Shen, Lipasti13
D t th (Fi 6 11)D t th (Fi 6 11)Datapath (Fig. 6.11)Datapath (Fig. 6.11)
© Hill, Shen, Lipasti14
MIPS MIPS Datapath Datapath With t Pi li iWith t Pi li i
What Is What Is PipeliningPipelining Without PipeliningWithout PipeliningPipeliningPipelining
© Hill, Shen, Lipasti15
Pipelined DatapathPipelined DatapathStart with single-cycle datapath Pipelined executionPipelined execution– Assume each instruction has its own datapath
But each instruction uses a different part in every– But each instruction uses a different part in every cycle
– Multiplex all on to one datapathp p– Latches separate cycles (like multicycle)
Ignore hazards for now M�u�x
0
1g– Data– Control Instruction�
memory
Address
4
0
Add Add�result
Shift�left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
Add
PC
M�u�
1Registers
Read�data 1
Read�data 2
Read�register 1
Read�register 2
Write�register
Write�
Read�data
ALU�result
M�u�x
ALUZero
ID/EX
Data�
Address
© Hill, Shen, Lipasti16
32
0Write�data
u�x
16Sign�
extend
Write�data 1
x memory
Pipelined DatapathPipelined DatapathM�u�x
0
1
4 Add Add�result
IF/ID EX/MEM MEM/WB
Add
ID/EX
Address
result
Shift�left 2
truct
ion
PC Read�data 1
Read�register 1
Read�Instruction�
memory 0
Inst
0Write�
M�u�x
1Registers Read�
data 2
Read�register 2
Write�register
Write�data
Read�data
1
ALU�result
M�u�x
ALUZero
Data�memory
Address
32
data
16Sign�
extend
© Hill, Shen, Lipasti17
The Basic Pipeline For MIPSThe Basic Pipeline For MIPSpp
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 6 Cycle 7Cycle 5
Reg ALU DMemIfetch Reg
Cycle Cycle Cycle 3 Cycle 4 Cycle 6 Cycle 7Cycle 5
I
Reg ALU DMemIfetch Reg
nst
Reg ALU DMemIfetch Reg
r.
Or
Reg ALU DMemIfetch Reg
rderr
© Hill, Shen, Lipasti18
MIPS FunctionsMIPS FunctionsWhat Is What Is PipeliningPipelining S u ct o sS u ct o sPipeliningPipelining
M�u�x
0
1
Passed To Next StageIR Mem[PC]NPC PC + 4
4 Add Add�result
Shift�left 2
IF/ID EX/MEM MEM/WB
Add
ID/EX
C C
Instruction�memory
Address
0
Inst
ruct
ion
PC
0Write�
M�u�x
1Registers
Read�data 1
Read�data 2
Read�register 1
Read�register 2
Write�register
Write�data
Read�data
1
ALU�result
M�u�x
ALUZero
Data�memory
Address
32
data
16Sign�
extend
Instruction Fetch (IF):Send out the PC and fetch the instruction from memory into the instruction register (IR); increment the PC by 4 to address the next sequential instruction.IR holds the instruction that will be used in the next stage.NPC h ld th l f th t PC
© Hill, Shen, Lipasti19
NPC holds the value of the next PC.
MIPS FunctionsMIPS FunctionsWhat Is What Is PipeliningPipelining
M�u�x
0
1
S u ct o sS u ct o sPipeliningPipelining
4 Add Add�result
Shift�left 2
IF/ID EX/MEM MEM/WB
Add
ID/EX Passed To Next StageA Regs[IR6..10]
Instruction�memory
Address
0
Inst
ruct
ion
PC
0Write�
M�u�x
1Registers
Read�data 1
Read�data 2
Read�register 1
Read�register 2
Write�register
Write�data
Read�data
1
ALU�result
M�u�x
ALUZero
Data�memory
Address
B Regs[IR11..15]
Imm ((IR16) ##IR16..3132
data
16Sign�
extend
(( 16) 16..31
Instruction Decode/Register Fetch Cycle (ID):Decode the instruction and access the register file to read the registers.The outputs of the general purpose registers are read into two temporary registers (A & B) for use in later clock cycles.We extend the sign of the lower 16 bits of the Instruction Register.
© Hill, Shen, Lipasti20
MIPS FunctionsMIPS FunctionsWhat Is What Is PipeliningPipelining S u ct o sS u ct o sPipeliningPipelining
Passed To Next StageIf Memory Reference:
M�u�x
0
1 If Memory Reference:ALUOutput A + Imm
If Reg-Reg ALU Inst.:ALUO t t A f B
4 Add Add�result
Shift�left 2
IF/ID EX/MEM MEM/WB
Add
ID/EX
ALUOutput A func. B
If Reg-Imm ALU Inst.:ALUOutput A func. Imm
Instruction�memory
Address
0
Inst
ruct
ion
PC
0Write�
M�u�x
1Registers
Read�data 1
Read�data 2
Read�register 1
Read�register 2
Write�register
Write�data
Read�data
1
ALU�result
M�u�x
ALUZero
Data�memory
Address
p
If Branch:ALUOutput NPC + Imm<<2C d (A 0)
32
data
16Sign�
extend
Cond (A op 0)
Execute Address Calculation (EX):We perform an operation (for an ALU) or an address calculation (if it’s a load p p ( ) (or a Branch).If an ALU, actually do the operation. If an address calculation, figure out how to obtain the address and stash away the location of that address for
© Hill, Shen, Lipasti21
ythe next cycle.
MIPS FunctionsMIPS FunctionsWhat Is What Is PipeliningPipelining S u ct o sS u ct o sPipeliningPipelining
M�u�x
0
1
Passed To Next StageIf LOAD Inst:LMD Mem[ALUOutput]
4 Add Add�result
Shift�left 2
IF/ID EX/MEM MEM/WB
Add
ID/EX
e [ UOutput]
If STORE Inst:Mem[ALUOutput] B
Instruction�memory
Address
0
Inst
ruct
ion
PC
0Write�
M�u�x
1Registers
Read�data 1
Read�data 2
Read�register 1
Read�register 2
Write�register
Write�data
Read�data
1
ALU�result
M�u�x
ALUZero
Data�memory
Address
[ p ]
If Branch Inst:if (Cond)
32
data
16Sign�
extend
PC ALUOutputelse
PC NPCMEMORY ACCESS (MEM):If this is an ALU, do nothing., gIf a load or store, then access memory.If this is a branch, then update PC
© Hill, Shen, Lipasti22
MIPS FunctionsMIPS FunctionsWhat Is What Is PipeliningPipelining S u ct o sS u ct o sPipeliningPipelining
M�u�x
0
1
Passed To Next StageIf Reg-Reg ALU:Regs[IR16..20] ALUOutput
4 Add Add�result
Shift�left 2
IF/ID EX/MEM MEM/WB
Add
ID/EX
If Reg-Imm ALU:Regs[IR11..15] ALUOutput
Instruction�memory
Address
0
Inst
ruct
ion
PC
0Write�
M�u�x
1Registers
Read�data 1
Read�data 2
Read�register 1
Read�register 2
Write�register
Write�data
Read�data
1
ALU�result
M�u�x
ALUZero
Data�memory
Address
If LOAD Inst:Regs[IR11..15] LMD
32
data
16Sign�
extend
WRITE BACK (WB):Update the registers from either the ALU or from the data loaded.p g
© Hill, Shen, Lipasti23
Things to NoticeThings to Notice
For some instructions no processing is d i t i tdone in certain stages:– Register-to-register instructions: No process in
MEM stage– STORE instructions: No process in WB stage p g
© Hill, Shen, Lipasti24
Things to Notice Things to Notice -- IIII
Instructions may read their operands only h th i ID twhen they are in ID stage
Reason: Reduces read port requirements p qon the Register File– Complexity reductionComplexity reduction– Power/Energy reduction
© Hill, Shen, Lipasti25
Things to Notice Things to Notice -- IIIIII
Instructions may write their results only h th i WB twhen they are in WB stage
Reason: Reduces write port requirements p qon the Register File– Complexity reductionComplexity reduction– Power/Energy reduction
© Hill, Shen, Lipasti26
Pipeline HurdlesPipeline HurdlesPipeline HurdlesPipeline HurdlesDefinition : Conditions that lead to incorrect behavior if not fixed
Structural hazard – two different instructions use same h/w in same cycle
Data hazard – two different instructions use same storage – must appear as if the instructions execute in correct orderpp
Control hazard– one instruction affects which instruction is next
ResolutionPipeline interlock logic detects hazards and fixes them simple solution: stall -simple solution: stall increases CPI, decreases performance better solution: partial stall -some instruction stall, others proceed better to stall early than late
© Hill, Shen, Lipasti27
some instruction stall, others proceed better to stall early than late
Structural HazardsStructural HazardsWhen two ormore differentinstructions want
Time (clock cycles)instructions wantto use samehardwareresource in same
lI Load Reg A
LU DMemIfetch Reg
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 6 Cycle 7Cycle 5
cycle
e.g., MEM usesthe same memory
nst
Instr 1 Reg ALU DMemIfetch Reg
t e sa e e o yport as IF asshown in thisslide.
r.
OInstr 2 Reg A
LU DMemIfetch Reg
Orde
Instr 3 Reg ALU DMemIfetch Reg
U Rer Instr 4 Reg A
LU DMemIfetch Reg
© Hill, Shen, Lipasti28
Structural HazardsStructural HazardsThis is another
Time (clock cycles)Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 6 Cycle 7Cycle 5
way of looking at the effect of a stall.I
nLoad Reg A
LU DMemIfetch Reg
Cycle Cycle Cycle 3 Cycle 4 Cycle 6 Cycle 7Cycle 5
nst Instr 1 Reg A
LU DMemIfetch Reg
r.
OInstr 2 Reg A
LU DMemIfetch Reg
rde
Stall Bubble Bubble Bubble BubbleBubble
r Instr 3 Reg ALU DMemIfetch Reg
© Hill, Shen, Lipasti29
Structural HazardsStructural Hazards
This is another way to represent the stall we saw on h l fthe last few pages.
© Hill, Shen, Lipasti30
Structural HazardsStructural HazardsStructural HazardsStructural HazardsDealing with Structural HazardsDealing with Structural Hazards
Stall – low cost, simple – Increases CPIIncreases CPI – use for rare case since stalling has performance effect
Pipeline hardware resource p– useful for multi-cycle resources – good performance – sometimes complex e.g., RAM
Replicate resource – good performance
increases cost (+ maybe interconnect delay)– increases cost (+ maybe interconnect delay) – useful for cheap or divisible resources
© Hill, Shen, Lipasti31
Structural HazardsStructural HazardsSt uctu a a a dsSt uctu a a a dsStructural hazards are reduced with these rules:
Each instruction uses a resource at most onceAlways use the resource in the same pipeline stageUse the resource for one cycle only
Many RISC ISA’a designed with this in mindSometimes very complex to do this. For example, memory of necessity is used in
the IF and MEM stages.
Some common Structural Hazards:Memory - we’ve already mentioned this one.Floating point Since many floating point instructions require many cyclesFloating point - Since many floating point instructions require many cycles,it’s easy for them to interfere with each other.Starting up more of one type of instruction than there are resources. Forinstance, the PA-8600 can support two ALU + two load/store instructions perinstance, the PA 8600 can support two ALU two load/store instructions percycle - that’s how much hardware it has available.
© Hill, Shen, Lipasti32
Structural HazardsStructural HazardsThis is the example on Page 144.
We want to compare the performance of two machines. Which machine is faster?Machine A: Dual ported memory - so there are no memory stallsMachine B: Single ported memory, but its pipelined implementation has a 1.05 times faster clock rate
Assume:Ideal CPI = 1 for bothLoads are 40% of instructions executed
© Hill, Shen, Lipasti33
Data HazardsData HazardsData HazardsData Hazards
These occur when at any time, there are instructions active thatneed to access the same data (memory or register) locations.
Where there’s real trouble is when we have:
instruction Ainstruction Binstruction B
and B manipulates (reads or writes) data before A does. Thisi l t th d f th i t ti i th hit tviolates the order of the instructions, since the architecture
implies that A completes entirely before B is executed.
© Hill, Shen, Lipasti34
Data HazardsData HazardsRead After Write (RAW)
InstrJ tries to read operand before InstrI writes itExecution Order is:InstrIInstrJInstrJ
I: ADD R1,R2,R3J: SUB R4,R1,R3
Caused by a “Dependence” (in compiler nomenclature). This hazard results from an actual need for communication.
R1I J
R1
© Hill, Shen, Lipasti35
Data HazardsData HazardsWrite After Read (WAR)
InstrJ tries to write operand before InstrI reads i– Gets wrong operand
Execution Order is:InstrIInstrJInstrJ
I: SUB R4,R1,R3 J: ADD R1,R2,R3
– Called an “anti-dependence” by compiler writers
K: MUL R6,R1,R7
– Called an anti-dependence by compiler writers.This results from reuse of the name “R1”.
Can’t happen in MIPS 5 stage pipeline because:– All instructions take 5 stages, and– Reads are always in stage 2, and – Writes are always in stage 5
I JR1
© Hill, Shen, Lipasti36
Data HazardsData HazardsWrite After Write (WAW)
InstrJ tries to write operand before InstrI writes it– Leaves wrong result ( InstrI not InstrJ )
Execution Order is:InstrIInstrJInstrJ
I: SUB R1,R4,R3 J: ADD R1,R2,R3
Called an “output dependence” by compiler writersThis also results from the reuse of name “R1”
K: MUL R6,R1,R7
This also results from the reuse of name R1 .
Can’t happen in MIPS 5 stage pipeline because: – All instructions take 5 stages, andAll instructions take 5 stages, and – Writes are always in stage 5
Will see WAR and WAW in later more complicated pipes
I JR1
© Hill, Shen, Lipasti37
I J
Data Flow GraphsData Flow Graphs I1
R15
I1 MUL R15, R10, R4I2 ADDU R24, R6, R15
I2I2 ADDU R24, R6, R15I3 LOAD R25, R24, #0I4 MUL R13, R24, R14I5 ADDI R14 R6 R13
I3
R24
R24
I5 ADDI R14, R6, R13I6 LOAD R15, R14, #0 R15
I4
R13 R14
I5
R14
© Hill, Shen, Lipasti38I6
R14
Data HazardsData HazardsSimple Solution to RAW
• Hardware detects RAW and stalls • Assumes register written then read each cycle
+ low cost to implement, simple low cost to implement, simple -- reduces IPC
• Try to minimize stalls
Minimizing RAW stalls
• Bypass/forward/short-circuit (We will use the word “forward”)• Use data before it is in the register
+ reduces/avoids stalls -- complex
• Crucial for common RAW hazards
© Hill, Shen, Lipasti39
Data HazardsData HazardsTime (clock cycles)
In
ADD R1,R2,R3 Reg ALU DMemIfetch Reg
IF ID/RF EX MEM WB
nst SUB R4,R1,R3 Reg A
LU DMemIfetch Reg
r.
OAND R6,R1,R7 Reg A
LU DMemIfetch Reg
rde
OR R8,R1,R9
O 10 1 11
Reg ALU DMemIfetch Reg
R LU DM mIf t h Reg
The use of the result of the ADD instruction in the next three instructions causes a hazard, since the register is not written until after those instructions read it.
r XOR R10,R1,R11 Reg AL DMemIfetch Reg
© Hill, Shen, Lipasti40
hazard, since the register is not written until after those instructions read it.
Figure 3.9
Data HazardsData HazardsForwarding is the concept of making data available to the input of the ALU for subsequent instructions even though theData HazardsData Hazards
Forwarding To AvoidData Hazard
subsequent instructions, even though the generating instruction hasn’t gotten to WB in order to write the memory or registers.
Data HazardTime (clock cycles)
IADD R1 R2 R3 R LU DMIf h Rn
st
ADD R1,R2,R3
SUB R4 R1 R3 Reg ALU DMemIfetch Reg
Reg AL DMemIfetch Reg
r.
O
SUB R4,R1,R3
AND R6,R1,R7 Reg ALU DMemIfetch Reg
g A
Orde
AND R6,R1,R7
OR R8,R1,R9 Reg ALU DMemIfetch Reg
er
, ,
XOR R10,R1,R11 Reg ALU DMemIfetch Reg
© Hill, Shen, Lipasti41Figure 3.10
Data HazardsData Hazards The data isn’t loaded until after the MEM stagethe MEM stage.
Time (clock cycles)
In
LOAD R1,R2,#0 Reg ALU DMemIfetch Reg
nstr
SUB R4,R1,R6 Reg ALU DMemIfetch Reg
r.
O AND R6,R1,R7 Reg ALU DMemIfetch Reg
rde OR R8 R1 R9 Reg A
LU DMemIfetch Reg
Th i t h h d ith f di
r OR R8,R1,R9
© Hill, Shen, Lipasti42
There are some instances where hazards occur, even with forwarding.Figure 3.12
Data HazardsData Hazards The stall is necessary as shown herehere.
Time (clock cycles)
Inst
LOAD R1, R2,#0 Reg ALU DMemIfetch Reg
r.
OSUB R4,R1,R6 RegIfetch A
LU DMem RegBubble
rde AND R6,R1,R7 Ifetch A
LU DMem RegBubble Reg
OR R8,R1,R9
rIfetch
ALU DMemBubble Reg
There are some instances where hazards occur even with forwarding
© Hill, Shen, Lipasti43
There are some instances where hazards occur, even with forwarding.Figure 3.13
Data HazardsData HazardsThis is another representation Data HazardsData Hazards of the stall.
LOAD R1, R2, #0 IF ID EX MEM WBLOAD R1, R2, #0 IF ID EX MEM WB
SUB R4, R1, R5 IF ID EX MEM WB
AND R6, R1, R7 IF ID EX MEM WB, ,
OR R8, R1, R9 IF ID EX MEM WB
LOAD R1, R2, #0 IF ID EX MEM WB
SUB R4, R1, R5 IF ID stall EX MEM WB
AND R6, R1, R7 IF stall ID EX MEM WB
OR R8, R1, R9 stall IF ID EX MEM WB
© Hill, Shen, Lipasti44
Data HazardsData Hazards Pipeline Scheduling
Instruction scheduled by compiler - move instruction in order to reduce stall.
ADDI Rb, R0, b -- code sequence for a = b+c before schedulingADDI Rb, R0, b code sequence for a b c before scheduling ADDI Rc, R0, c ADD Ra, Rb, Rc -- stall STORE Ra, R0, aADDI R R0 d f d f b f h d liADDI Re, R0, e -- code sequence for d = e-f before scheduling ADDI Rf, R0, f SUB Rd, Re, Rf -- stall STORE Rd, R0, dS O d, 0, d
Arrangement of code after scheduling.
ADDI Rb R0 bADDI Rb, R0, b ADDI Rc, R0, c ADDI Re, R0, e ADD Ra, Rb, Rc, ,ADDI Rf, R0, f STORE Ra, R0, aSUB Rd, Re, RfSTORE Rd R0 d
© Hill, Shen, Lipasti45
STORE Rd, R0, d
Data HazardsData Hazards Pipeline Scheduling
scheduled unscheduled
gcc 31%42%
54%
tex
spice
25%
14%65%
42%
0% 20% 40% 60% 80%
tex 25%
% loads stalling pipeline
© Hill, Shen, Lipasti46
CControl Hazardsontrol HazardsCControl Hazardsontrol Hazards
Control hazards can cause a greater performanceloss for pipelines than do data hazardsloss for pipelines than do data hazards
A l h d h d fi d hA control hazard occurs when we need to find thedestination of a branch, and can’t fetch any newinstructions until we know that destinationinstructions until we know that destination
© Hill, Shen, Lipasti47
Example (quicksort/MIPS)Example (quicksort/MIPS)# for (; (j < high) && (array[j] < array[low]) ; ++j );# R10 = j# R9 = high# R9 = high# R6 = array# R8 = low
BGE R10, R9, doneMUL R15, R10, #4ADDU R24, R6, R15LOAD R25, R24, #0MUL R13, R8, #4MUL R13, R8, #4ADDU R14, R6, R13LOAD R15, R14, #0BGE R25, R15, done
cont:cont:ADDU R10, R10, #1. . .
done:
© Hill, Shen, Lipasti48
ADDU R11, R11, #-1
Control Hazard on BranchesControl Hazard on BranchesTh St St llTh St St ll
Control HazardsThree Stage StallThree Stage Stall
10: BEQ R1,R3,#36
U
Reg ALU DMemIfetch Reg
14: AND R2,R3,R5
U
Reg ALU DMemIfetch Reg
18: OR R6,R1,R7 Reg ALU DMemIfetch Reg
U
22: ADD R8,R1,R9
36 XOR R10 R1 R11
Reg ALU DMemIfetch Reg
U36: XOR R10,R1,R11 Reg AL DMemIfetch Reg
© Hill, Shen, Lipasti49
Branch Stall ImpactBranch Stall ImpactControl Hazards Branch Stall ImpactBranch Stall ImpactIf CPI = 1, 30% branch, Stall 3 cycles => new CPI = 1.9!
(Whoa! How did we get that 1.9???)Two part solution to this dramatic increase:
– Determine branch taken or not sooner, AND– Compute taken branch address earlier
MIPS branch tests if register = 0 or ^ 0
MIPS Solution:MIPS Solution:– Move Zero test to ID/RF stage– Adder to calculate new PC in ID/RF stage
must be fastmust be fast can't afford to subtract compares with 0 are simple Greater-than, Less-than test sign-bit, but not-equal must OR all bits , g , qmore general compares need ALU
– 1 clock cycle penalty for branch versus 3
© Hill, Shen, Lipasti50
In the next chapter, we’ll look at ways to avoid the branch all together.
Five Branch Hazard Five Branch Hazard Control HazardsAlternativesAlternatives
#1: Stall until branch direction is clear
#2: Predict Branch Not Taken– Execute successor instructions in sequenceExecute successor instructions in sequence– “Squash” instructions in pipeline if branch actually taken– Advantage of late pipeline state update
47% MIPS branches not taken on average– 47% MIPS branches not taken on average– PC+4 already calculated, so use it to get next instruction
#3 P di t B h T k#3: Predict Branch Taken– 53% MIPS branches taken on average– But haven’t calculated branch target address in MIPS
MIPS still incurs 1 cycle branch penaltyOther machines: branch target known before outcome
© Hill, Shen, Lipasti51
Control DependenceControl DependenceOne instruction affects which executes next
STORE R4 R5 #0– STORE R4, R5, #0– BNE R2, R3, loop– SUB R6 R7 R8– SUB R6, R7, R8
Cycle: 1 2 3 4 5 6 7 8 9 1 1 1 1Cycle:Instr:
1 2 3 4 5 6 7 8 9 10
11
12
13
STORE F D X M WSTORE F D X M WBNE F D X M WSUB F D X M W
© Hill, Shen, Lipasti52
SUB F D X M W
Control Dependence Control Dependence -- StallStallDetect dependence and stall
STORE R4 R5 #0– STORE R4, R5, #0– BNE R2, R3, loop– SUB R6 R7 R8– SUB R6, R7, R8
Cycle: 1 2 3 4 5 6 7 8 9 1 1 1 1Cycle:Instr:
1 2 3 4 5 6 7 8 9 10
11
12
13
STORE F D X M WSTORE F D X M WBNE F D X M WSUB F D X M W
© Hill, Shen, Lipasti53
SUB F D X M W2-cycle bubble
Predict not takenPredict not takenDetect dependence and stall
STORE R4 R5 #0– STORE R4, R5, #0– BNE R2, R3, loop– SUB R6 R7 R8
If loop is not taken
– SUB R6, R7, R8
Cycle: 1 2 3 4 5 6 7 8 9 1 1 1 1Cycle:Instr:
1 2 3 4 5 6 7 8 9 10
11
12
13
STORE F D X M WSTORE F D X M WBNE F D X M WSUB F D X M W
© Hill, Shen, Lipasti54
SUB F D X M W
Control Dependence Control Dependence -- StallStallDetect dependence and stall
STORE R4 R5 #0– STORE R4, R5, #0– BNE R2, R3, loop– SUB R6 R7 R8
If loop is taken
– SUB R6, R7, R8
Cycle: 1 2 3 4 5 6 7 8 9 1 1 1 1Cycle:Instr:
1 2 3 4 5 6 7 8 9 10
11
12
13
STORE F D X M WSTORE F D X M WBNE F D X M WMUL F D X M W
© Hill, Shen, Lipasti55
MUL - - F D X M WSUB is flushed from the pipeline
Control Dependence Control Dependence -- StallStallDetect dependence and stall
STORE R4 R5 #0– STORE R4, R5, #0– BNE R2, R3, loop– SUB R6 R7 R8
If loop is taken
– SUB R6, R7, R8
Cycle: 1 2 3 4 5 6 7 8 9 1 1 1 1Cycle:Instr:
1 2 3 4 5 6 7 8 9 10
11
12
13
STORE F D X M WSTORE F D X M WBNE F D X M WMUL F D X M W
© Hill, Shen, Lipasti56
MUL F D X M W2-cycle bubble
Control Hazards Five Branch Hazard
#4: Execute Both Paths
Alternatives
#5: Delayed Branch– Define branch to take place AFTER a following instruction
branch instructionsequential successor1
ti lsequential successor2........sequential successorn
branch target if takenBranch delay of length n
branch target if taken
– 1 slot delay allows proper decision and branch target address in 5 stage pipelinepipeline
– MIPS uses this
© Hill, Shen, Lipasti57
Delayed BranchDelayed BranchControl Hazards
Delayed BranchDelayed Branch
Where to get instructions to fill branch delay slot?– Before branch instruction– From the target address: only valuable when branch taken– From fall through: only valuable when branch not taken– Cancelling branches allow more slots to be filled
Compiler effectiveness for single branch delay slot:– Fills about 60% of branch delay slots– About 80% of instructions executed in branch delay slots useful in
computation– About 50% (60% x 80%) of slots usefully filled
Delayed Branch downside: 7-8 stage pipelines, multiple instructions issued per y g p p , p pclock (superscalar)
© Hill, Shen, Lipasti58
Evaluating Branch Evaluating Branch Control HazardsAlternativesAlternatives
Pipeline speedup = Pipeline depth1 +Branch frequency ×Branch penalty
Scheduling Branch CPI speedup v Speedup vScheduling Branch CPI speedup v. Speedup v.scheme penalty unpipelined stall
Stall pipeline 3 1.42 3.5 1.0Predict taken 1 1.14 4.4 1.26Predict not taken 1 1.09 4.5 1.29Delayed branch 0.5 1.07 4.6 1.31y
Conditional & Unconditional = 14%, 65% change PC
© Hill, Shen, Lipasti59
Pipelining Introduction Pipelining Introduction Control HazardsSummarySummary
Just overlap tasks, and easy if tasks are independentSpeed Up Š Pipeline Depth; if ideal CPI is 1, then:p p p p ; ,
Speedup =Pipeline Depth
XClock Cycle Unpipelined
Hazards limit performance on computers:
Speedup =1 + Pipeline stall CPI
XClock Cycle Pipelined
– Structural: need more HW resources– Data (RAW,WAR,WAW): need forwarding, compiler scheduling
C t l d l d b h di ti– Control: delayed branch, prediction
© Hill, Shen, Lipasti60
Compiler “Static” Compiler “Static” Prediction ofPrediction of
Control HazardsPrediction ofPrediction ofTaken/Untaken BranchesTaken/Untaken BranchesThe compiler can program what it thinks
the branch direction will be. Here are the results when it does so.
70% 14%
the results when it does so.
redi
ctio
n
40%
50%
60%
Rat
e
8%
10%
12%
cy o
f Mis
pr
20%
30%
40%
pred
ictio
n
4%
6%
8%
Freq
uenc
0%
10%
s uc o cc d a v
Mis
0%
2%
c o c a
alvi
nn
com
pres
s
dodu
espr
esso gc
hydr
o2d
mdl
jsp2 or
a
swm
256
tom
catv
alvi
nn
com
pres
s
dodu
c
espr
esso gc
c
hydr
o2d
mdl
jsp2 or
a
swm
256
tom
catv
© Hill, Shen, Lipasti61
Always taken Taken backwardsNot Taken Forwards
Compiler “Static” Prediction Compiler “Static” Prediction Control HazardsofofTaken/Untaken BranchesTaken/Untaken Branches
Improves strategy for placing instructions in delay slot
Two strategies– Backward branch predict taken, forward branch not takenac wa d b a c p ed ct ta e , o wa d b a c ot ta e– Profile-based prediction: record branch behavior, predict branch based on
prior run
© Hill, Shen, Lipasti62
Evaluating Static Branch Evaluating Static Branch Control HazardsPrediction StrategiesPrediction Strategies
Misprediction ignores frequency of branch an
ch
10000
100000
q y“Instructions between mispredicted branches” is a better metric ed
icte
d br
a
1000
s pe
r mis
pr10
100
nstr
uctio
ns
1
nn ss duc
sso
gcc
2d p2 ora
56 atv
I
alvi
n
com
pre
dod
espr
es
g
hydr
o2
mdl
jsp o
swm
25
tom
ca
P fil b d Di i b d
© Hill, Shen, Lipasti63
Profile-based Direction-based
Resolution of Pipeline HazardsResolution of Pipeline Hazards
Pipeline hazardsi l i l i f d d– Potential violations of program dependences
– Must ensure program dependences are not violatedd l iHazard resolution
– Static: compiler/programmer guarantees correctness– Dynamic: hardware performs checks at runtime
Pipeline interlock– Hardware mechanism for dynamic hazard resolution– Must detect and enforce dependences at runtime
© Hill, Shen, Lipasti64
Pipelined DatapathPipelined Datapath -- SummarySummary
Instruction flow– add and load– Write of registersg– Pass register specifiers
Any info needed by a later stage getsAny info needed by a later stage gets passed down the pipeline– e.g. store value through EX
© Hill, Shen, Lipasti65
Pipelined Pipelined ControlControl
Control– Set by 5 different instructions– Divide and conquer: carry IR down the pipeq y p p
MIPS ISA requires the appearance of sequential executionsequential execution– True of most general purpose ISAs
© Hill, Shen, Lipasti66
Pipelined ControlPipelined Control
IF and IDN– None
EX– ALUop, ALUsrc, RegDst
MEM– Branch, MemRead, MemWrite
WBWB– MemtoReg, RegWrite
© Hill, Shen, Lipasti67
PCSrc
0
1
M�u�x
IF/ID ID/EX EX/MEM MEM/WB
Add�
Add
PC Address ctio
nBranch
4
Read�Read�register 1
RegWrite
MemWrite
Add Add�result
Shift�left 2
Instruction�memory
Address
Inst
ruc
MemtoRegALUSrc
0
0Registers
Write�register
Write�data
Read�data 1
Read�data 2
Read�register 2
M�u�x
1
Read�data M�
u�x
1Address
Data�memory
ZeroALU�
resultALU
Zero
Instruction�
16 32Instruction�[15– 0]
0
Sign�extend
1Write�data
ALU�control MemRead
6
Instruction�[20– 16]
ALUOpInstruction�[15– 11]
0
1
M�u�x
© Hill, Shen, Lipasti68
RegDst
Figure 6.29Figure 6.29
C t l M
WB
WBInstructionControl
EX
M
M
WB
WB
IF/ID ID/EX EX/MEM MEM/WB
© Hill, Shen, Lipasti69
Figure 6.30Figure 6.30PCSrc
Control M
WB
WB
ID/EX
EX/MEM
MEM/WB
M�u�x
0
1
Add
Branch
4 Add Add�result
rite
EX M WBIF/ID
PC
Instruction�memory
Inst
ruct
ion
Mem
toR
eg
Branch
ALUSrc
0Registers
Read�data 1
Read�
Read�register 1
Read�register 2
ALU�Zero
R d�
Shift�left 2
Reg
Wr
ALU
Mem
Writ
e
Address
memory
16 32Instruction�[15 0]
0
0g
Write�register
Write�data
Read�data 2
M�u�x
1
ALU�result
Write�data
Read�data
M�u�x
1
6
AddressData�
memory
Instruction�[20– 16]
ALUOp
[15– 0]
M�u�x
0
1
Sign�extend
ALU�control
MemRead
Instruction�[15– 11]
© Hill, Shen, Lipasti70
RegDst1
Pipelined ControlPipelined Control
Controlled by different instructionsDecode instructions and pass the signals down the pipep pControl sequencing is embedded in the pipelinepipeline
© Hill, Shen, Lipasti71
PipeliningPipelining
Not too complex yet– Data hazards– Control hazards– Exceptions
© Hill, Shen, Lipasti72
Data HazardsData HazardsMust first detect hazards
ID/EX.WriteRegister = IF/ID.ReadRegister1
ID/EX WriteRegister = IF/ID ReadRegister2ID/EX.WriteRegister = IF/ID.ReadRegister2
EX/MEM.WriteRegister = IF/ID.ReadRegister1
EX/MEM.WriteRegister = IF/ID.ReadRegister2
MEM/WB WriteRegister = IF/ID ReadRegister1MEM/WB.WriteRegister = IF/ID.ReadRegister1
MEM/WB.WriteRegister = IF/ID.ReadRegister2
© Hill, Shen, Lipasti73
Data HazardsData Hazards
Not all hazards because– WriteRegister not used (e.g. STORE)– ReadRegister not used (e.g. ADDI, JMP)g ( g , )– Do something only if necessary
© Hill, Shen, Lipasti74
dest src1 src2opcode … I1
Data HazardsData Hazardsd 1 2d
C C
I2
Hazard Detection Unitdest src1 src2opcode … I2
– Several 5-bit (or 6-bit) comparatorsResponse? Stall pipelinep p p– Instructions in IF and ID stay
IF/ID i li l t h t d t d– IF/ID pipeline latch not updated– Send ‘nop’ down pipeline (called a bubble)– PCWrite, IF/IDWrite, and nop mux
© Hill, Shen, Lipasti75
Data HazardsData Hazards
Detecting dependencies over memory l tilocations– In a typical datapath, actual writes to the main
memory are serialized via write buffersData fromCache
Address Data to be written
Writes to the Main Memory
© Hill, Shen, Lipasti76
Write Buffers (FIFO)
Data HazardsData Hazards
Detecting dependencies over memory l tilocations– When the effective address of a LOAD is
computed, it is compared with the effective addresses of all the writes in the Write BuffersData from
Cache
Address Data to be written The Effective Address of the LOAD
C
C
C
C
© Hill, Shen, Lipasti77
Write Buffers (FIFO) Writes to the Main Memory Wider Comparators(typically 32 bit)
Data HazardsData Hazards
A better response – forwarding– Also called bypassing
Comparators ensure register is read after it p gis writtenInstead of stalling until write occursInstead of stalling until write occurs– Use mux to select forwarded value rather than
i t lregister value– Control mux with hazard detection logic
© Hill, Shen, Lipasti78
Write before Read RFWrite before Read RF
Register file designh l k– 2-phase clocks common
– Write RF on first phase– Read RF on second phase
Hence, same cycle:– Write R1– Read R1
No bypass needed– If read before write or DFF-based, need bypass
© Hill, Shen, Lipasti79
Control Flow HazardsControl Flow Hazards
Control flow instructions– branches, jumps, jals, returns– Can’t fetch until branch outcome known– Too late for next IF
© Hill, Shen, Lipasti80
Control Flow HazardsControl Flow Hazards
What to do?– Always stall– Easy to implementy p– Performs poorly
1/6th instructions are branches each branch– 1/6th instructions are branches, each branch takes 3 cyclesCPI 1 3 1/6 1 5 (l b d)– CPI = 1 + 3 x 1/6 = 1.5 (lower bound)
© Hill, Shen, Lipasti81
Control Flow HazardsControl Flow Hazards
Predict branch not takenS d ti l i t ti d i liSend sequential instructions down pipelineKill instructions later if incorrectMust stop memory accesses and RF writes– Including loads (why?)g ( y )
Late flush of instructions on misprediction– Complex– Complex– Global signal (wire delay)
© Hill, Shen, Lipasti82
Control Flow HazardsControl Flow Hazards
Even better but more complex– Predict taken– Predict both (eager execution)( g )– Predict one or the other dynamically
Adapt to program branch patternsAdapt to program branch patternsLots of chip real estate these days
– Pentium III IV Alpha 21264Pentium III, IV, Alpha 21264
Current research topic
© Hill, Shen, Lipasti83
Control Flow HazardsControl Flow Hazards
Another option: delayed branches (later in branch handling techniques!)branch handling techniques!)– Always execute following instruction
“d l l ”– “delay slot”– Put useful instruction there, otherwise ‘nop’
Losing popularity– Just a stopgap (one cycle, one instruction)pg p ( y , )– Superscalar processors (later)
Delay slot just gets in the way (special case)
© Hill, Shen, Lipasti84
y j g y ( p )
Exceptions and PipeliningExceptions and Pipelining
ADD R1, R2, R3 overflowsA surprise branch– Earlier instructions flow to completionEarlier instructions flow to completion– Kill later instructions
S PC i EPC t PC t EX h dl t– Save PC in EPC, set PC to EX handler, etc.
© Hill, Shen, Lipasti85
ExceptionsExceptions
Even worse: in one cycleI/O i– I/O interrupt
– User trap to OS (EX)– Illegal instruction (ID)– Arithmetic overflow– Hardware error– Etc.Etc.
Interrupt priorities must be supported
© Hill, Shen, Lipasti86
ReviewReview
Big PictureD t thDatapathControl– Data hazards
StallsForwarding or bypassing
– Control flow hazardsBranch prediction
Exceptions
© Hill, Shen, Lipasti87
p