anne bracy cs 3410• add pipeline registers (flip-flops)for isolation • each stage begins by...

AnneBracyCS3410

ComputerScienceCornellUniversity

The slides are the product of many rounds of teaching CS 3410 by Professors Weatherspoon, Bala, Bracy, McKee, and Sirer.

insn0.fetch, dec, exec

Single-cycle

insn1.fetch, dec, exec

Pipelinedinsn0.decinsn0.fetch

insn1.decinsn1.fetchinsn0.exec

insn1.exec

5-stagePipeline• Implementation• WorkingExample

Hazards• Structural• DataHazards• ControlHazards

Write-BackMemory

InstructionFetch Execute

InstructionDecode

extend

registerfile

control

memory

din dout

addrPC

memory

IF/ID ID/EX EX/MEM MEM/WB

computejump/branch

targets

1 2 3 4 5 6 7 8 9Cycle

Latency:Throughput:

IF ID EX MEM WB

Latency: 5cyclesThroughput: 1insn/cycle CPI=1

• Breakdatapath intomultiplecycles (here5)• Parallelexecutionincreasesthroughput• Balancedpipelineveryimportant

• Sloweststagedeterminesclockrate• Imbalancekillsperformance

• Addpipelineregisters(flip-flops) forisolation• Eachstagebeginsbyreadingvaluesfromlatch• Eachstageendsbywritingvaluestolatch

• Resolvehazards

Stage PerformFunctionality Latchvaluesofinterest

Fetch UsePCtoindexProgramMemory,incrementPC

Instructionbits(tobedecoded)PC+4(tocomputebranchtargets)

Decode Decodeinstruction,generatecontrolsignals,readregisterfile

Controlinformation,Rdindex,immediates,offsets,register values(Ra,Rb),PC+4(tocomputebranchtargets)

ExecutePerformALUoperationComputetargets(PC+4+offset,etc.)incasethisisabranch,decideifbranchtaken

Controlinformation,Rdindex, etc.ResultofALUoperation,valueincasethisisastoreinstruction

Memory Performload/storeifneeded,addressisALUresult

Controlinformation,Rdindex,etc.Resultofload,passresultfrom execute

Writeback Selectvalue,writetoregisterfile

instructionmemory

addr mc

00=readword

Restofp

ipeline

pc-sel

pc-regpc-rel

pc-abs•PC+4•pc-reg (PCregisters:JR)•pc-rel (PC-relative: BEQ,BNE)•pc-abs(PCabsolute:JandJAL)

Restofp

ipeline

Stage1:InstructionFetch

registerfile

extend imm

decode

result

Stage2:InstructionDe

pc-rel

pc-abs

EX/MEM

Restofp

ipeline

branch?im

mpc-sel

pc-reg

target

MEM/WB

Restofp

ipeline

Stage3:Execute

EX/MEM

memory

din doutaddr

mctarget

branch?pc-sel

pc-rel

pc-abs

pc-reg

Stage4:M

MEM/WB

result

ID/EX EX/MEM MEM/WB

din dout

addrinst

target

instmem

Consideranon-pipelinedprocessorwithclockperiodC (e.g.,50ns).IfyoudividetheprocessorintoN stages(e.g.,5),yournewclockperiodwillbe:

A. CB. NC. lessthanC/ND. C/NE. greaterthanC/N

• Instructionssamelength• 32bits,easytofetchandthendecode

• 3typesofinstructionformats• Easytoroutebitsbetweenstages• Canreadaregistersourcebeforeevenknowing

whattheinstructionis• Memoryaccessthroughlwandswonly

• AccessmemoryafterALU

add r3 ß r1, r2 nand r6 ß r4, r5 lw r4 ß 20(r2)add r5 ß r2, r5sw r7 à 12(r3)

Assume8-registermachine

extend

Time:018

Registerfile

Datamem

Bits11-15Bits16-20

add312

912187

Bits26-31

Fetch:add312

add312

extend

Time:119

Registerfile

Datamem

Bits11-15Bits16-20

0nand645

912187

Bits26-31

Fetch:nand645

nand 645 add312

extend

Time:220

Registerfile

Datamem

Bits11-15Bits16-20

0lw420(2)

912187

Bits26-31

Fetch:lw420(2)

lw420(2) nand 645 add312

extend

Time:321

Registerfile

Datamem

Bits11-15Bits16-20

0add525

912187

Bits26-31

Fetch:add525

add525 lw420(2) nand 645 add312

extend

Time:4

18=0100107=000111-------------------3=111101

Registerfile

Datamem

Bits11-15Bits16-20

162012

0sw712(3)

945187

Bits26-31

Fetch:sw712(3)

sw712(3) add525 lw420(2) nand 645add312

extend

Time:523

Registerfile

Datamem

Bits11-15Bits16-20

0945187

Bits26-31

Nomoreinstructions

nop sw712(3) add525 lw420(2) nand 645

extend

Time:624

Registerfile

Datamem

Bits11-15Bits16-20

0945997

Bits26-31

Nomoreinstructions

nop nop sw712(3) add525 lw420(2)

extend

Time:725

Registerfile

Datamem

Bits11-15Bits16-20

9459916

Bits26-31

Nomoreinstructions

nop nop nop sw712(3) add525

SlidesthankstoSallyMcKee

extend

Time:826

Registerfile

Datamem

Bits11-15Bits16-20

9459916

Bits21-23

Nomoreinstructions

nop nop nop nop sw712(3)

extend

Time:927

Pipeliningisgreatbecause:

A. Youcanfetchanddecodethesameinstructionatthesametime.

B. Youcanfetchtwoinstructionsatthesametime.C. Youcanfetchoneinstructionwhiledecoding

another.D. Instructionsonlyneedtovisitthepipeline

stagesthattheyrequire.E. CandD

Correctnessproblemsassociatedw/processordesign

1. StructuralhazardsSameresourceneededfordifferentpurposesatthesametime(Possible:ALU, RegisterFile,Memory)

2. DatahazardsInstructionoutputneededbeforeit’savailable

3. ControlhazardsNextinstructionPCunknownattimeofFetch

addr3,r2,r1nopnop

addr6,r5,r4

datamem

instmem

IF ID Ex M WIF ID Ex M W

IF ID Ex M W

add r3, r2,r1nopaddr6,r5,r4

Problem: NeedtoreadfromandwritetoRegisterFileatthesametimeSolution: negateRFclock:writefirsthalf,readsecondhalf

IF ID Ex M W

Dependence:relationshipbetweentwoinsns• Data:twoinsnsusesamestoragelocation• Control: 1insnaffectswhetheranotherexecutesatall• Notabadthing,programswouldbeboring otherwise• Enforcedbymakingolderinsngobeforeyoungerone

– Happensnaturallyinsingle-/multi-cycledesigns– Butnotinapipeline

Hazard:dependence&possibilityofwronginsnorder• Effectsofwronginsnordercannotbeexternallyvisible• Hazardsareabadthing:mostsolutionseithercomplicatethehardwareorreduceperformance

DataHazards• registerfile(RF)readsoccurinstage2(ID)• RFwritesoccurinstage5(WB)• RFwrittenin½half,readinsecond½halfofcycle

x10: add r3 ß r1, r2x14: sub r5 ß r3, r4

1.Isthereadependence?2.Isthereahazard?

A) YesB) NoC) Cannottellwiththe

informationgiven.

Whichofthefollowingstatementsistrue?

A.Whetherthereisadatadependencebetweentwoinstructionsdependsonthemachinetheprogramisrunningon.B.Whetherthereisadatahazardbetweentwoinstructionsdependsonthemachinetheprogramisrunningon.C.BothA&BD.NeitherAnorB

IF ID MEM

IF ID MEM WB

Clockcycle1 2 3 4 5 6 7 8 9

sub r5,r3,r4

lwr6,4(r3)

or r5,r3,r5

sw r6,12(r3)

addr3,r1,r2

IF ID MEM

IF ID MEM WB

Clockcycle1 2 3 4 5 6 7 8 9

sub r5,r3,r4

lwr6,4(r3)

or r5,r3,r5

sw r6,12(r3)

addr3,r1,r2

backwardsarrowsrequiretimetravel

IF ID MEM

IF ID MEM WB

Clockcycle1 2 3 4 5 6 7 8 9

sub r5,r3,r4

lwr6,4(r3)

or r5,r3,r5

sw r6,12(r3)

addr3,r1,r2

IF ID MEM

IF ID MEM WB

Clockcycle1 2 3 4 5 6 7 8 9

sub r5,r3,r4

lwr6,4(r3)

or r5,r3,r5

sw r6,12(r3)

addr3,r1,r2

ID/EX EX/MEM MEM/WB

din dout

addrinst

instmem

Detecting Data Hazards

IF/ID.Ra ≠0?

Ra==? Ra==

add r3, r1, r2subr5,r3,r4

Stall=(IF/ID.Ra !=0&& (IF/ID.Ra ==ID/EX.Rd||IF/ID.Ra ==EX/M.Rd))

1. DoNothing• ChangetheISAtomatchimplementation• “Heycompiler:don’tcreatecodew/datahazards!”

(Wecandobetterthanthis)

2. Stall• Pausecurrentandsubsequentinstructionstillsafe

3. Forward/bypass• Forwarddatavaluetowhereitisneeded

(Onlyworksifvalueactuallyexistsalready)

HowtostallaninstructioninIDstage• preventIF/IDpipelineregisterupdate

– stallstheIDstageinstruction

• convertIDstageinsn intonop forlaterstages– innocuous“bubble”passesthroughpipeline

• preventPCupdate– stallsthenext(IFstage)instruction

ID/EX EX/MEM MEM/WB

din dout

instmem

addr3,r1,r2subr5,r3,r5orr6,r3,r4addr6,r3,r8

Ifhazard:

WE=0MemWr=0RegWr=0

detecthazard

datamem

Dinstmem

Rd RdRd

/stall

addr3,r1,r2

(MemWr=0RegWr=0)

NOP=If(IF/ID.rA ≠0&&(IF/ID.rA==ID/Ex.RdIF/ID.rA==Ex/M.Rd))

subr5,r3,r5

orr6,r3,r4 (WE=0)

STALLCONDITIONMET

datamem

Dinstmem

Rd RdRd

/stall

(MemWr=0RegWr=0)

addr3,r1,r2subr5,r3,r5

(MemWr=0RegWr=0)

orr6,r3,r4 (WE=0)

STALLCONDITIONMET

datamem

Dinstmem

Rd RdRd

/stall

(MemWr=0RegWr=0)

orr6,r3,r4 (WE=1)NOSTALLCONDITIONMET:suballowedtoleavedecodestage

Clockcycle1 2 3 4 5 6 7 8

addr3,r1,r2

subr5,r3,r5

or r6,r3,r4

addr6,r3,r8

addr3,r1,r2

subr5,r3,r5

or r6,r3,r4

addr6,r3,r8

IF ID Ex M W

IF ID Ex M

ID* ID*

IF* IF*

IF ID Ex

2StallCycles

1. DoNothing• ChangetheISAtomatchimplementation• “Compiler:don’tcreatecodewithdatahazards!”

(Nicetry,wecandobetterthanthis)

2. Stall• Pausecurrentandsubsequentinstructionstillsafe

3. Forward/bypass• Forwarddatavaluetowhereitisneeded

(Onlyworksifvalueactuallyexistsalready)

datamem

Dinstmem

forwardunit

detecthazard

IF/ID ID/Ex Ex/Mem Mem/WB

datamem

Dinstmem

forwardunit

detecthazard

Twotypesofforwarding/bypass• ForwardingfromEx/Mem registerstoExstage(M®Ex)• ForwardingfromMem/WBregistertoExstage(W® Ex)

addr3,r1,r2

subr5,r3,r1

datamem

instmem

IF ID Ex M W

Problem:EXneedsALUresultthatisinMEMstageSolution:addabypassfromEX/MEM.DtostartofEX

Ex/Mem

datamem

instmem

DetectionLogicinExStage:forward=(Ex/M.WE&&EX/M.Rd !=0&&

ID/Ex.Ra ==Ex/M.Rd)||(sameforRb)

Ex/Mem

addr3,r1,r2

subr5,r3,r1

orr6,r3,r4

datamem

instmem

IF ID Ex M WIF ID

IF WEx M WID Ex M

Problem:EXneedsvaluebeingwrittenbyWBSolution:AddbypassfromWBfinalvaluetostartofEX

Mem/WB

add r3, r1,r2subr5,r3,r1orr6,r3,r4

datamem

instmem

DetectionLogic:forward=(M/WB.WE&&M/WB.Rd !=0&&

ID/Ex.Ra ==M/WB.Rd &&not(ID/Ex.WE &&Ex/M.Rd !=0&&

ID/Ex.Ra ==Ex/M.Rd)||(sameforRb)

Mem/WB

add r3, r1,r2subr5,r3,r1orr6,r3,r4

addr3,r1,r2

sub r5,r3,r4

lwr6,4(r3)

or r5,r3,r5

sw r6,12(r3)

addr3,r1,r2

sub r5,r3,r4

lwr6,4(r3)

or r5,r3,r5

sw r6,12(r3)

IF ID Ex M W

Ex M W

ID Ex M

IF ID Ex

IF ID Ex M W

Datadependencyafteraloadinstruction:• ValuenotavailableuntilaftertheMstageàNextinstructioncannotproceedifdependent

THEKILLERHAZARD57

datamem

instmem

lwr4,20(r8)orr6,r3,r4

lwr4,20(r8)

or r6,r3,r4

datamem

instmem

lwr4,20(r8)

or r6,r3,r4

datamem

instmem

IF ID Ex

datamem

instmem

NOPorr6,r4,r1 lwr4,20(r8)

lwr4,20(r8)

or r6,r3,r4

IF ID Ex M W

IF ID Ex M WID*Stall

datamem

instmem

NOPorr6,r4,r1 lwr4,20(r8)

lwr4,20(r8)

or r6,r3,r4

IF ID Ex M W

IF ID Ex M WID*Stall

datamemim

Dinstmem

forwardunit

detecthazard

Stall=If(ID/Ex.MemRead &&IF/ID.Ra ==ID/Ex.Rd

Mostfrequent3410non-solutiontoload-usehazardsWhyisthis“solution”sosososososoawful? 63

datamemim

Dinstmem

forwardunit

detecthazard

ForwardingvaluesdirectlyfromMemorytotheExecutestagewithoutstoringtheminaregisterfirst:

A. Doesnotremovetheneedtostall.B. AddsonetoomanypossibleinputstotheALU.C. Willcausethepipelineregistertohavethe

wrongvalue.D. Halvesthefrequencyoftheprocessor.E. BothA&D

TwoMIPSSolutions:• MIPS2000/3000:delayslot

– ISAsaysresultsofloadsarenotavailableuntilonecyclelater

–Assemblerinsertsnop,orreorderstofilldelayslot

• MIPS4000onwards:stall– Butreally,programmer/compilerreorderstoavoidstallingintheloaddelayslot

for (i = 0; i < max; i++) {n += 2;

}i = 7;n--;

x10 addi r1, r0, 0 # i=0x14 Loop: addi r2, r2, 2 # n x+= 2x18 addi r1, r1, 1 # i++x1C blt r1, r3, Loop # i<max?x20 addi r1, r0, 7 # i = 7x24 subi r2, r2, 1 # n++

i à r1Assume:n à r2max à r3

ControlHazards• instructionsarefetchedinstage1(IF)• branchandjumpdecisionsoccurinstage3(EX)à nextPCnotknownuntil2cycles after branch/jump

x1C blt r1, r3, Loop x20 addi r1, r0, 7

x24 subi r2, r2, 1

Branchnot taken?NoProblem!

Branchtaken?Justfetched2addi’sà Zap&Flush

1C blt r1,r3,L20 addi r1,r0,724 subi r2,r2,1

datamem

instmem D

NOPIF ID Ex M W

IF ID NOP NOPNOPIF NOP NOP NOP

branchcalc

decidebranch

IF ID Ex M W

IfbranchTaken®Zap

• preventPCupdate• clearIF/IDlatch• branchcontinues

NewPC=1C

14 L:addi r2,r2,2

1C blt r1,r3,L20 addi r1,r0,724 subi r2,r2,1

datamem

instmem D

NOPIF ID Ex M W

IF ID NOP NOPNOPIF NOP NOP NOP

branchcalc

decidebranch

IF ID Ex M W

IfbranchTaken®Zap

• preventPCupdate• clearIF/IDlatch• branchcontinues

NewPC=1C

14 L:addi r2,r2,2

Foreverytakenbranch?OUCH!!!

1. DelaySlot• YouMUSTdothis• MIPSISA:1insn afterctrlinsn always executed

• Whetherbranchtakenornot

2. ResolveBranchatDecode• SomegroupsdothisforProject2,yourchoice• Movebranchcalc fromEXtoID• Alternative:justzap2nd instructionwhenbranchtaken

3. BranchPrediction• Notin3410,buteveryprocessorworthanythingdoesthis

(nooffense!)

datamem

instmem D

branchcalc

decidebranchNewPC=1C

1C blt r1, r3, Loop F D X

20 addi r1, r0, 7 F D

24 subi r2, r2, 1 F

for (i = 0; i < max; i++) {n += 2;

}i = 7;n--;x10 addi r1, r0, 0 # i=0x14 Loop: addi r2, r2, 2 # n x+= 2x18 addi r1, r1, 1 # i++x1C blt r1, r3, Loop # i<max?x20 nopx24 addi r1, r0, 7 # i = 7x28 subi r2, r2, 1 # n++

i à r1Assume:n à r2max à r3

datamem

instmem D

branchcalc

decidebranchNewPC=1C

20 nop F D

24 addi r1, r0, 7 F

datamem

instmem D

NewPC=1C

branchcalc

decidebranch

20 nop F D

14 Loop:addi r2,r2,2 F

x10 addi r1, r0, 0 # i=0x14 Loop: addi r2, r2, 2 # n x+= 2x18 addi r1, r1, 1 # i++x1C blt r1, r3, Loop # i<max?x20 nop

x10 addi r1, r0, 0 # i=0x14 Loop: addi r1, r1, 1 # i++x18 blt r1, r3, Loop # i<max?x1C addi r2, r2, 2 # n x+= 2

Compiler transforms code

datamem

instmem D

NewPC=1C

branchcalc

decidebranch

20 addi r2,r2,2 F D

14 Loop:addi r1,r1,1 F

MostprocessorsupportSpeculativeExecution• Guess directionofthebranch

– Allowinstructionstomovethroughpipeline– Zapthemlaterifguessturnsouttobewrong

• Amustforlongpipelines

Datahazardsoccurwhenaoperand(register)dependsontheresultofapreviousinstructionthatmaynotbecomputedyet.Pipelinedprocessorsneedtodetectdatahazards.

Stalling,preventingadependentinstructionfromadvancing,isonewaytoresolvedatahazards.StallingintroducesNOPs(“bubbles”)intoapipeline.IntroduceNOPsby(1)preventingthePCfromupdating,(2)preventingwritestoIF/IDregistersfromchanging,and(3)preventingwritestomemoryandregisterfile.Nops significantlydecreaseperformance.

Forwardingbypassessomepipelinedstagesforwardingaresulttoadependentinstructionoperand(register).Betterperformancethanstalling.

ControlhazardsoccurbecausethePCfollowingacontrolinstructionisnotknownuntilcontrolinstructionisexecuted.Ifbranchistakenà needtozapinstructions.1cycleperformancepenalty.

DelaySlotscanpotentiallyincreaseperformanceduetocontrolhazards.Theinstructioninthedelayslotwillalwaysbeexecuted.Requiressoftware(compiler)tomakeuseofdelayslot.Putnop indelayslotifnotabletoputusefulinstructionindelayslot.

WecanreducecostofacontrolhazardbymovingbranchdecisionandcalculationfromExstagetoIDstage.Withadelayslot,thisremovestheneedtoflushinstructionsontakenbranches.

anne bracy cs 3410• add pipeline registers (flip-flops)for isolation • each stage begins by...

Documents

latch versus register

latch up prevention

the q-q plot of p-values for predicting outcomes ... -...

outdoors latch air

biodiversity assessment method operational manual – stage...

latch assessment tool

latch prestashop español

door latch

latch unix español

latches. outline pulse-triggered latch s-r latch gated...

latch openldap

understanding grs enq and latch usage and contention ·...

latch powershell español

rts latch top pull magnet latch bottom bracket latch...

1 sequential circuit latch & flip-flop. 2 contents...

latch & register inference

shira latch:: portfolio

m23 chain latch

orgnew-...gunakan latch 1 bila data yang dipertahankan harus...

latch security scenarios