pipelining - ii
DESCRIPTION
Pipelining - II. Adapted from CS 152C (UC Berkeley) lectures notes of Spring 2002. 30. 40. 40. 40. 40. 20. A. B. C. D. Revisiting Pipelining Lessons. 6 PM. 7. 8. 9. Time. Pipelining doesn’t help latency of single task, it helps throughput of entire workload - PowerPoint PPT PresentationTRANSCRIPT
Pipelining - II
Adapted from CS 152C (UC Berkeley) lectures notes of Spring 2002
Revisiting Pipelining Lessons• Pipelining doesn’t help
latency of single task, it helps throughput of entire workload
• Pipeline rate limited by slowest pipeline stage
• Multiple tasks operating simultaneously using different resources
• Potential speedup = Number pipe stages
• Unbalanced lengths of pipe stages reduces speedup
• Time to “fill” pipeline and time to “drain” it reduces speedup
• Stall for Dependences
A
B
C
D
6 PM 7 8 9
Task
Order
Time
30 40 40 40 40 20
• Structural Hazards– Hardware design
• Control Hazard– Decision based on results
• Data Hazard– Data Dependency
Revisiting Pipelining Hazards
Control Signals for existing Datapath
The Right to Left Control can lead to hazards
MUX
PC
ADD
Registers
ADD
ADD
SignExtend
4
InstructionMemory
Address
Instruction
Read Reg1
Read Reg2
Write Reg
Write Data
Read Data1
ReadData2 M
UX
DataMemory
Address
Write Data
Read Data
MUX
IF: Instruction Fetch ID: Instruction Decode/ register file read
EX: Execute/address calculation MEM: Memory Access WB: Write back
16 32
Shift left 2
Zero
Place registers between each step
MUX
PC
ADD
Registers
ADD
ADD
SignExtend
4
InstructionMemory
Address
Instruction
Read Reg1
Read Reg2
Write Reg
Write Data
Read Data1
ReadData2 M
UX
DataMemory
Address
Write Data
Read Data
MUX
16 32
Shift left 2
Zero
IF/ID ID/EX EX/MEM MEM/WB
Example10 lw r1, r2(35)14 addI r2, r2, 320 sub r3, r4, r524 beq r6, r7, 10030 ori r8, r9, 1734 add r10, r11, r12
100 and r13, r14, 15
Start: Fetch 10
Exe
c
Reg
. Fi
le
Mem
Acc
ess
Dat
aM
em
A
B
SReg
File
IR
Inst
. Mem
D
Dec
ode
MemCtrl
WB Ctrl
M
rs rt im
10 lw r1, r2(35)14 addI r2, r2, 320 sub r3, r4, r524 beq r6, r7, 10030 ori r8, r9, 1734 add r10, r11, r12
100 and r13, r14, 15
IF
PC
Nex
t PC
10
=
n n n n
Fetch 14, Decode 10
Exe
c
Reg
. Fi
le
Mem
Acc
ess
Dat
aM
em
A
B
SReg
File
IR
Inst
. Mem
D
Dec
ode
MemCtrl
WB Ctrl
M
2 rt im
10 lw r1, r2(35)
14 addI r2, r2, 320 sub r3, r4, r524 beq r6, r7, 10030 ori r8, r9, 1734 add r10, r11, r12
100 and r13, r14, 15
lw r1
, r2(
35)
ID
IF
PC
Nex
t PC
14
=
n n n
Fetch 20, Decode 14, Exec 10
Exe
c
Reg
. Fi
le
Mem
Acc
ess
Dat
aM
em
r2
B
SReg
File
IR
Inst
. Mem
D
Dec
ode
MemCtrl
WB Ctrl
M
2 rt 35
10 lw r1, r2(35)
14 addI r2, r2, 3
20 sub r3, r4, r524 beq r6, r7, 10030 ori r8, r9, 1734 add r10, r11, r12
100 and r13, r14, 15
lw r1
addI
r2, r
2, 3
EX
PC
Nex
t PC
20
=
n n
Fetch 24, Decode 20, Exec 14, Mem 10
Exe
c
Reg
. Fi
le
Mem
Acc
ess
Dat
aM
em
r2
B
r2+3
5
Reg
File
IR
Inst
. Mem
D
Dec
ode
MemCtrl
WB Ctrl
M
4 5 3
10 lw r1, r2(35)
14 addI r2, r2, 3
20 sub r3, r4, r5
24 beq r6, r7, 10030 ori r8, r9, 1734 add r10, r11, r12
100 and r13, r14, 15
lw r1
sub
r3, r
4, r5
addI
r2, r
2, 3
ID
IF
EX
M
PC
Nex
t PC
24
=
n
Fetch 30, Dcd 24, Ex 20, Mem 14, WB 10
Exe
c
Reg
. Fi
le
Mem
Acc
ess
Dat
aM
em
r4
r5
r2+3
Reg
File
IR
Inst
. Mem
D
Dec
ode
MemCtrl
WB Ctrl
M[r2
+35]
6 7
10 lw r1, r2(35)
14 addI r2, r2, 3
20 sub r3, r4, r5
24 beq r6, r7, 100
30 ori r8, r9, 1734 add r10, r11, r12
100 and r13, r14, 15
lw r1
beq
r6, r
7 10
0
addI
r2
sub
r3
ID
IF
EX
M WB
PC
Nex
t PC
30
=
Fetch 100, Dcd 30, Ex 24, Mem 20, WB 14
Exe
c
Reg
. Fi
le
Mem
Acc
ess
Dat
aM
em
r6
r7
r2+3
Reg
File
IR
Inst
. Mem
D
Dec
ode
MemCtrl
WB Ctrl
r1=M
[r2+3
5]
9 xx
10 lw r1, r2(35)
14 addI r2, r2, 3
20 sub r3, r4, r5
24 beq r6, r7, 100
30 ori r8, r9, 17
34 add r10, r11, r12
100 and r13, r14, 15
beq
addI
r2
sub
r3r4
-r5
100
ori r
8, r9
17
ID
IF
EX
M WB
PC
Nex
t PC
100
=
Pipelining Load Instruction
• The five independent functional units in the pipeline datapath are:– Instruction Memory for the Ifetch stage– Register File’s Read ports (bus A and busB) for the
Reg/Dec stage– ALU for the Exec stage– Data Memory for the Mem stage– Register File’s Write port (bus W) for the Wr stage
Clock
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7
Ifetch Reg/Dec Exec Mem Wr1st lw
Ifetch Reg/Dec Exec Mem Wr2nd lw
Ifetch Reg/Dec Exec Mem Wr3rd lw
Pipelining the R Instruction
• Ifetch: Instruction Fetch– Fetch the instruction from the Instruction Memory
• Reg/Dec: Registers Fetch and Instruction Decode• Exec:
– ALU operates on the two register operands– Update PC
• Wr: Write the ALU output back to the register file
Cycle 1 Cycle 2 Cycle 3 Cycle 4
Ifetch Reg/Dec Exec WrR-type
Pipelining Both L and R type
• We have pipeline conflict or structural hazard:– Two instructions try to write to the register file at
the same time!– Only one write port
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9
Ifetch Reg/Dec Exec WrR-type
Ifetch Reg/Dec Exec WrR-type
Ifetch Reg/Dec Exec Mem WrLoad
Ifetch Reg/Dec Exec WrR-type
Ifetch Reg/Dec Exec WrR-type
Ops! We have a problem!
Important Observations• Each functional unit can only be used once per
instruction• Each functional unit must be used at the same
stage for all instructions:– Load uses Register File’s Write Port during its 5th
stage
– R-type uses Register File’s Write Port during its 4th stage
Ifetch Reg/Dec Exec Mem WrLoad1 2 3 4 5
Ifetch Reg/Dec Exec WrR-type1 2 3 4
Solution• Delay R-type’s register write by one cycle:
– Now R-type instructions also use Reg File’s write port at Stage 5– Mem stage is a NOOP stage: nothing is being done.
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9
Ifetch Reg/Dec Mem Wr
Ifetch Reg/Dec Mem WrR-type
Ifetch Reg/Dec Exec Mem WrLoad
Ifetch Reg/Dec Mem WrR-type
Ifetch Reg/Dec Mem WrR-type
Ifetch Reg/Dec Exec WrR-type Mem
Exec
Exec
Exec
Exec
1 2 3 4 5
Datapath (Without Pipeline)IR <- Mem[PC]; PC <– PC+4;
A <- R[rs]; B<– R[rt]
S <– A + B;
R[rd] <– S;
S <– A + SX;
M <– Mem[S]
R[rd] <– M;
S <– A or ZX;
R[rt] <– S;
S <– A + SX;
Mem[S] <- B
If CondPC < PC+SX;
Exe
c
Reg
. Fi
le
Mem
Acc
ess
Dat
aM
em
A
B
SReg
File
Equ
al
PC
Nex
t PC
IR
Inst
. Mem
D
M
Datapath (With Pipeline)IR <- Mem[PC]; PC <– PC+4;
A <- R[rs]; B<– R[rt]
S <– A + B;
R[rd] <– M;
S <– A + SX;
M <– Mem[S]
R[rd] <– M;
S <– A or ZX;
R[rt] <– M;
S <– A + SX;
Mem[S] <- B
if Cond PC < PC+SX;
M <– S
Exe
c
Reg
. Fi
le
Mem
Acc
ess
Dat
aM
em
A
B
SReg
File
Equ
al
PC
Nex
t PC
IR
Inst
. Mem
D
M
M <– S
Mem
Structural Hazard and Solution
Instr.
Order
Time (clock cycles)
Load
Instr 1
Instr 2
Instr 3
Instr 4A
LUMem Reg Mem Reg
AL
UMem Reg Mem Reg
AL
UMem Reg Mem RegA
LUReg Mem Reg
AL
UMem Reg Mem Reg
Control Hazard - #1 Stall
• Stall: wait until decision is clear• Impact: 2 lost cycles (i.e. 3 clock cycles per
branch instruction) => slow
Instr.
Order
Time (clock cycles)
Add
Beq
Load
AL
UMem Reg Mem Reg
AL
UMem Reg Mem RegA
LUReg Mem RegMem
Lostpotential
Control Hazard – #2 Predict
• Predict: guess one direction then back up if wrong• Impact: 0 lost cycles per branch instruction if right,
1 if wrong (right 50% of time)• More dynamic scheme: history of 1 branch
Instr.
Order
Time (clock cycles)
AddBeqLoad
AL
UMem Reg Mem Reg
AL
UMem Reg Mem Reg
Mem
AL
UReg Mem Reg
Control Hazard - #3 Delayed Branch
• Delayed Branch: Redefine branch behavior (takes place after next instruction)
• Impact: 0 clock cycles per branch instruction if can find instruction to put in “slot” ( 50% of time)
Instr.
Order
Time (clock cycles)
Add
Beq
Misc
AL
UMem Reg Mem Reg
AL
UMem Reg Mem Reg
Mem
AL
UReg Mem Reg
Load Mem
AL
UReg Mem Reg
Data Hazards (RAW)
• Dependencies backwards in time are hazards
Instr.
Order
Time (clock cycles)
add r1,r2,r3
sub r4,r1,r3
and r6,r1,r7
or r8,r1,r9
xor r10,r1,r11
IF
ID/RF EX MEM WBAL
UIm Reg Dm Reg
AL
UIm Reg Dm Reg
AL
UIm Reg Dm Reg
Im
AL
UReg Dm Reg
Data Hazards [contd…]• “Forward” result from one stage to another
Instr.
Order
Time (clock cycles)
add r1,r2,r3
sub r4,r1,r3
and r6,r1,r7
or r8,r1,r9
xor r10,r1,r11
IF
ID/RF
EX MEM WBAL
UIm Reg Dm Reg
AL
UIm Reg Dm RegA
LUIm Reg Dm Reg
Im
AL
UReg Dm Reg
AL
UIm Reg Dm Reg
Data Hazards [contd…]
Reg
• Dependencies backwards in time are hazards
• Can’t solve with forwarding: • Must delay/stall instruction dependent on loads
Time (clock cycles)
lw r1,0(r2)
sub r4,r1,r3
IF
ID/RF EX MEM WBAL
UIm Reg Dm
AL
UIm Reg Dm RegStall
Hazard DetectionI-Fetch DCD MemOpFetch OpFetch Exec Store
IFetch DCD ° ° °StructuralHazard
I-Fetch DCD OpFetch Jump
IFetch DCD ° ° °
Control Hazard
IF DCD EX Mem WB
IF DCD OF Ex Mem
RAW (read after write) Data Hazard
WAW Data Hazard (write after write)
IF DCD OF Ex RS WAR Data Hazard (write after read)
IF DCD EX Mem WB
IF DCD EX Mem WB
CPSC614Lec 2.28
• Read After Write (RAW) InstrJ tries to read operand before InstrI writes it
• Caused by a “Data Dependence” (in compiler nomenclature). This hazard results from an actual need for communication.
Three Generic Data Hazards
I: add r1,r2,r3J: sub r4,r1,r3
CPSC614Lec 2.29
• Write After Read (WAR) InstrJ writes operand before InstrI reads it
• Called an “anti-dependence” by compiler writers.This results from reuse of the name “r1”.
• Can’t happen in MIPS 5 stage pipeline because:– All instructions take 5 stages, and– Reads are always in stage 2, and – Writes are always in stage 5
I: sub r4,r1,r3 J: add r1,r2,r3K: mul r6,r1,r7
Three Generic Data Hazards
CPSC614Lec 2.30
Three Generic Data Hazards• Write After Write (WAW)
InstrJ writes operand before InstrI writes it.
• Called an “output dependence” by compiler writersThis also results from the reuse of name “r1”.
• Can’t happen in MIPS 5 stage pipeline because: – All instructions take 5 stages, and – Writes are always in stage 5
• Will see WAR and WAW in later more complicated pipes
I: sub r1,r4,r3 J: add r1,r2,r3K: mul r6,r1,r7
Hazard Detection• Suppose instruction i is about to be issued and a
predecessor instruction j is in the instruction pipeline.
• A RAW hazard exists on register if Rregs( i ) Wregs( j )• A WAW hazard exists on register if Wregs( i ) Wregs( j )• A WAR hazard exists on register if Wregs( i ) Rregs( j )
Window on execution:Only pending instructions cancause hazardsInst J
Inst INew Inst
InstructionMovement:
Computing CPI
2211
typetypetypetypestall
stallbase
freqSTALLfreqSTALLCPICPICPICPI
• Start with Base CPI• Add stalls
•Suppose: –CPIbase=1–Freqbranch=20%, freqload=30%–Suppose branches always cause 1 cycle stall–Loads cause a 2 cycle stall
•Then: CPI = 1 + (10.20)+(2 0.30)= 1.8
Summary
• Control Signals need to be propagated• Insert Registers between every stage to
“remember” and “propagate” values• Solutions to Control Hazard are Stall,
Predict and Delayed Branch• Solutions to Data Hazard is “Forwarding”• Effective CPI = CPIideal + CPIstall