Pipelining Analogy

•
Pipelined laundry: overlapping execution
–
Parallelism improves performance


Four loads:

Speedup
= 8/3.5 = 2.3

Non-stop:

Speedup
= 2n/0.5n + 1.5 ≈
4
= number of
stages
FIGURE 4.24 The laundry analogy for pipelining. Ann, Brian, Cathy,
and Don each have dirty clothes to be washed, dried, folded, and
put away. The washer, dryer, “folder,” and “storer” each take 30
minutes for their task.
 LEGv8 Pipeline
Five stages, one step per stage
1. IF: Instruction fetch from memory
2. ID: Instruction decode & register read
3. EX: Execute operation or calculate address
4. MEM: Access memory operand
5. WB: Write result back to register
 Example: Pipeline Performance
•
Assume time for stages is
–
100ps for register read or write
–
200ps for other stages
•
Compare pipelined datapath with single-
cycle datapath

Instr Instr fetch Register ALU op Memory Register Total

read access write time
LDUR 200ps 100 ps 200ps 200ps 100 ps 800ps
STUR 200ps 100 ps 200ps 200ps 700ps
R-format 200ps 100 ps 200ps 100 ps 600ps
CBZ 200ps 100 ps 200ps 500ps

Figure 4.25
 Pipeline Performance
Single-cycle (Tc=
800ps)

Pipelined (Tc=
200ps)

Figure 4.26
 Pipeline Speedup
• If all stages are balanced
– i.e., all take the same time

• If not balanced, speedup is less

• Speedup due to increased throughput
– Latency (time for each instruction) does not decrease
 Pipelining and ISA Design
• LEGv8 ISA designed for pipelining
– All instructions are 32-bits
•
Easier to fetch and decode in one cycle
•
c.f. x86: 1- to 17-byte instructions
– Few and regular instruction formats
•
Can decode and read registers in one step
– Load/store addressing
•
Can calculate address in 3rd stage, access memory in 4th
stage
– Alignment of memory operands
•
Memory access takes only one cycle
 Hazards
Situations that prevent starting the next
instruction in the next cycle
• Structure hazards
– A required resource is busy
• Data hazard
– Need to wait for previous instruction to complete its
data read/write
• Control hazard
– Deciding on control action depends on previous
instruction
 Structure Hazards
• Conflict for use of a resource
• In LEGv8 pipeline with a single memory
– Load/store requires data access
– Instruction fetch would have to stall for that cycle
•
Would cause a pipeline “bubble”
• Hence, pipelined datapaths require
separate instruction/data memories
– Or separate instruction/data caches
 Data Hazards
•
An instruction depends on completion of
data access by a previous instruction

– ADDX19, X0,X1
SUBX2, X19,X3
 Data Hazards
•
Use result when it is computed
–
Don’t wait for it to be stored in a register
–
Requires extra connections in the datapath

FIGURE 4.28 Graphical representation of forwarding.

 Load-Use Data Hazard
•
Can’t always avoid stalls by forwarding
–
If value not computed when needed
–
Can’t forward backward in time!

FIGURE 4.29 We need a stall even with forwarding when an R-format

instruction following a load tries to use the data
 Code Scheduling to Avoid Stalls
•
Reorder code to avoid use of load result in
the next instruction
•
C code for A = B + E; C = B + F;

LDUR X1, [X0,#0] LDUR X1, [X0,#0]

LDUR X2, [X0,#8] LDUR X2, [X0,#8]
stall ADD X3, X1, X2 LDUR X4, [X0,#16]
STUR X3, [X0,#24] ADD X3, X1, X2
LDUR X4, [X0,#16] STUR X3, [X0,#24]
stall ADD X5, X1, X4 ADD X5, X1, X4
STUR X5, [X0,#32] STUR X5, [X0,#32]
13 11
cycles cycles
 Control Hazards
•
Branch determines flow of control
–
Fetching next instruction depends on branch
outcome
–
Pipeline can’t always fetch correct instruction
•
Still working on ID stage of branch

•
In LEGv8 pipeline
–
Need to compare registers and compute
target early in the pipeline
–
Add hardware to do it in ID stage
 Stall on Branch
•
Wait until branch outcome determined before
fetching next instruction

FIGURE 4.30 Pipeline showing stalling on every conditional branch as solution to control
hazards. This example assumes the conditional branch is taken, and the instruction at the
destination of the branch is the ORR instruction. There is a one-stage pipeline stall, or bubble,
after the branch. In reality, the process of creating a stall is slightly more complicated. The effect
on performance, however, is the same as would occur if a bubble were inserted
 Branch Prediction
• Longer pipelines can’t readily
determine branch outcome early
– Stall penalty becomes unacceptable
• Predict outcome of branch
– Only stall if prediction is wrong
• In LEGv8 pipeline
– Can predict branches not taken
– Fetch instruction after branch, with no delay
 More-Realistic Branch Prediction
• Static branch prediction
– Based on typical branch behavior
– Example: loop and if-statement branches
•
Predict backward branches taken
•
Predict forward branches not taken
• Dynamic branch prediction
– Hardware measures actual branch behavior
•
e.g., record recent history of each branch
– Assume future behavior will continue the trend
•
When wrong, stall while re-fetching, and update history
 Pipeline Summary
The BIG Picture
•
Pipelining improves performance by
increasing instruction throughput
–
Executes multiple instructions in parallel
–
Each instruction has the same latency
•
Subject to hazards
–
Structure, data, control
•
Instruction set design affects complexity
of pipeline implementation
 LEGv8 Pipelined Datapath

MEM

Right-to-left WB
flow leads to
hazards

FIGURE 4.32 The single-cycle datapath from Section 4.4 (similar to Figure 4.17)
 Pipeline registers
•
Need registers between stages
–
To hold information produced in previous cycle

FIGURE 4.34 The pipelined version of the datapath in Figure 4.32.

 LOAD

FIGURE 4.35
IF and ID:
First and second
pipe stages of an
instruction, with
the active
portions of the
datapath in Figure
4.34 highlighted
FIGURE 4.36 EX: The third pipe stage of a load instruction, highlighting the
portions of the datapath in Figure 4.34 used in this pipe stage. The register is
added to the sign-extended immediate, and the sum is placed in the EX/MEM
pipeline register.
FIGURE 4.37

MEM and WB: The

fourth and fifth pipe
stages of a load
instruction,
highlighting the
portions of the
datapath in Figure
4.34 used in this
pipe stage
Corrected Datapath for Load
 Store

FIGURE 4.38 EX: The third pipe stage of a store instruction. Unlike the third stage
of the load instruction in Figure 4.36, the second register value is loaded into the
EX/MEM pipeline register to be used in the next stage. Although it wouldn’t hurt to
always write this second register into the EX/MEM pipeline register, we write the
second register only on a store instruction to make the pipeline easier to
understand.
FIGURE 4.39
MEM and WB:
The fourth and fifth pipe
stages of a store
instruction.

In the fourth stage, the

data are written into
data Memory for the
store. Note that the
data come from the
EX/MEM pipeline
register and that
nothing is changed in
the MEM/WB pipeline
register. Once the data
are written in memory,
there is nothing left for
the store instruction to
do, so nothing happens
in stage 5.
FIGURE 4.40 The corrected pipelined datapath to handle the load instruction
properly. The write register number now comes from the MEM/WB pipeline
register along with the data. The register number is passed from the ID pipe
stage until it reaches the MEM/WB pipeline register, adding five more bits to
the last three pipeline registers. This new path is shown in color.
FIGURE 4.41 The portion of the datapath in Figure 4.40 that is used in all five
stages of a load instruction.
Graphically Representing Pipelines
multiple-clock-cycle pipeline
diagrams single-clock-cycle
pipeline diagrams

Consider the following five-instruction

sequence: LDUR X10, [X1,#40]

SUB X11, X2, X3
ADD X12, X3, X4
LDUR X13,
[X1,#48] ADD
X14, X5, X6

Figure 4.42 shows the multiple-clock-cycle

pipeline diagram for these instructions.
 Multi-Cycle Pipeline Diagram
•
Form showing resource usage

FIGURE 4.42 Multiple-clock-cycle pipeline diagram of five instructions

 Multi-Cycle Pipeline Diagram
•
Traditional form

FIGURE 4.43 Traditional multiple-clock-cycle pipeline diagram of five instructions in

Figure 4.42.
 Single-Cycle Pipeline Diagram
•
State of pipeline in a given cycle

FIGURE 4.44 The single-clock-cycle diagram corresponding to clock cycle 5 of the

pipeline in Figures 4.42 and 4.43.
 Pipelined Control (Simplified)

FIGURE 4.45 The pipelined datapath of

Figure 4.40 with the control signals
identified
 Pipelined Control
•
Control signals derived from instruction
– As in single-cycle implementation

FIGURE 4.49 The eight control lines for the final three stages.
 Pipelined Control

FIGURE 4.50 The

pipelined datapath
of Figure 4.45, with
the control signals
connected to the
control portions of
the pipeline
registers.