ECE 486/586

Computer Architecture

Lecture # 18

Spring 2019

Portland State University

Lecture Topics

• Speculative Execution

– Tomasulo’s Algorithm to Handle Speculation and Precise Exceptions

Reference:

• Chapter 3: Section 3.6

Speculative Execution

• Key Idea: Speculate on the outcome of branches and execute the program as if our guesses were correct• Instructions on the speculative path continue to be fetched, issued and

executed, as if the branch prediction was always correct

• Need to take corrective action if the speculation was incorrect

Dynamic Scheduling and Speculative Execution

• In the original Tomasulo Algorithm, an instruction writes its result to the destination register, as soon as it completes execution• Allows later instructions to consume the result as soon as it is ready

• Problem # 1: What happens if an instruction followed a mis-predicted branch and was able to write its result to its destination register before the branch outcome could be verified?• We must “undo” any writes done by this instruction and other subsequent

instructions

• Problem # 2: What happens if an instruction followed a mis-predicted branch and raised an exception?• To ensure precise exceptions, the exception raised by a mis-speculated

instruction should not be handled

Hardware-based Speculation

• Solution: Separate bypassing of results from the actual completion of an instruction• A speculative instruction allowed to “bypass” its results to other instructions

but cannot write results to destination registers/memory

• Instruction completion happens only after the instruction is no longer speculative. At this time, results can be written

• Extend Tomasulo’s algorithm to incorporate a “commit” step

• Instructions execute out of order but commit in order

Re-order Buffer (ROB)

• Holds the results of an instruction between the time the instruction completes execution and the time the instruction commits

• Implemented as a circular buffer (with “head” and “tail” pointers)

• Instruction enter and exit the ROB in program order

• When an instruction reaches the head of the ROB• Check if this instruction is a mispredicted branch or raised an exception

– If yes, flush the ROB and re-direct fetch

– If no, the instruction writes its results to the register file/memory and the “head” pointer starts pointing to the next instruction

Tomasulo’s Algorithm with ROB

Steps in Instruction Execution

• Issue

• Get instruction from instruction queue

• Check for free ROB entry and reservation status (RS)

• If available, allocate the ROB entry and RS– Record the ROB entry number in the RS. Will be used to tag results, when

placed on CD

• If ROB entry or RS not available, stall the instruction

• Execute

• Write Result

• Commit

Steps in Instruction Execution

• Issue

• Execute• If one or more source operands not available, monitor CDB (wait for the ROB

tag of the producer instruction to show up on the CDB)

• When both operands are available, execute instruction

• Write Result

• Commit

Steps in Instruction Execution

• Issue

• Execute

• Write Result• Write the result to CDB with the tag of ROB entry

• Any instruction waiting in the reservation stations pick up the result from CDB (after matching the ROB entry number)

• Result also written to the ROB entry

• Mark the reservation station as available

• Commit

Steps in Instruction Execution

• Issue

• Execute

• Write Result

• Commit (aka “retire” or “graduate”)• Get the instruction from head of ROB

• If the instruction has not completed execution, do nothing

• Else– If the instruction is a mispredicted branch (or had an exception)

» Flush ROB

» Restart execution at correct target

– Else

» Write the result to destination register/memory

» Remove instruction from ROB (advance head pointer)

Example