Multiprocessor Interconnection Networks

CS 740 November 19, 2003

Topics Network design issues Network Topology Slides from TCM & CS

Networks
How do we move data between processors? Design Options: Topology Routing Physical implementation Switching (circuit or packet) Flow control (how) (when) (what) (which)

CS 740 F03

Evaluation Criteria:
Latency Bisection Bandwidth Contention and hot-spot behavior Partitionability Cost and scalability Fault tolerance
P P

Buses
P

Bus

Simple and cost-effective for small-scale multiprocessors Not scalable (limited bandwidth; electrical complications)

CS 740 F03

CS 740 F03

Crossbars
P

Rings
Crossbar
P

Each port has link to every other port + Low latency and high throughput - Cost grows as O(N^2) so not very scalable.

Cheap: Cost is O(N). Point-to-point wires and pipelining can be used to make them very fast. + High overall bandwidth - High latency O(N) Examples: KSR machine, Hector
P P

- Difficult to arbitrate and to get all data lines into and out of a centralized crossbar. Used in small-scale MPs (e.g., C.mmp) and as building block for other networks (e.g., Omega).

Ring

CS 740 F03

CS 740 F03

Trees
Cheap: Cost is O(N). Latency is O(logN). Easy to layout as planar graphs (e.g., H-Trees). For random permutations, root can become bottleneck. To avoid root being bottleneck, notion of Fat-Trees (used in CM5)
H-Tree

Hypercubes

0-D

1-D

2-D

3-D

4-D

Fat Tree

Also called binary n-cubes. # of nodes = N = 2^n. Latency is O(logN); Out degree of PE is O(logN) Minimizes hops; good bisection BW; but tough to layout in 3-space Popular in early message-passing computers (e.g., intel iPSC, NCUBE) Used as direct network ==> emphasizes locality

CS 740 F03

CS 740 F03

Multistage Logarithmic Networks

Key Idea: have multiple layers of switches between destinations. Cost is O(NlogN); latency is O(logN); throughput is O(N). Generally indirect networks. Many variations exist (Omega, Butterfly, Benes, ...). Used in many machines: BBN Butterfly, IBM RP3, ...

Omega Network
Omega Net w or k 000 001 010 011 100 101 110 111 000 001 010 011 100 101 110 111

All stages are same, so can use recirculating network. Single path from source to destination. Can add extra stages and pathways to minimize collisions and increase fault tolerance. Can support combining. Used in IBM RP3.

CS 740 F03

10

CS 740 F03

Butterfly Network
But t er f l y Net w or k 000 001 010 011 100 101 110 111 spl i t on MSB spl i t on LSB 000 001 010 011 100 101 110 111

k-ary n-cubes
4-ary 3-cube

Equivalent to Omega network. Easy to see routing of messages. Also very similar to hypercubes (direct vs. indirect though). Clearly see that bisection of network is (N / 2) channels. Can use higher-degree switches to reduce depth.

Generalization of hypercubes (k-nodes in a string) Total # of nodes = N = k^n. k > 2 reduces # of channels at bisection, thus allowing for wider channels but more hops.
12 CS 740 F03

11

CS 740 F03

Relationship of Butterflies to Hypercubes

Real World 2D mesh

1824 node Paragon: 16 x 114 array Wiring is isomorphic Except that Butterfly always takes log n steps
13 CS 740 F03 14 CS 740 F03

Advantages of Low-Dimensional Nets

What can be built in VLSI is often wire-limited LDNs are easier to layout:
more uniform wiring density (easier to embed in 2-D or 3-D space) mostly local connections (e.g., grids)

Embeddings in two dimensions

Compared with HDNs (e.g., hypercubes), LDNs have:

shorter wires (reduces hop latency) fewer wires (increases bandwidth given constant bisection width) increased channel width is the major reason why LDNs win!

6x3x2

LDNs have better hot-spot throughput

more pins per node than HDNs

Embed multiple logical dimension in one physical dimension using long wires
CS 740 F03 16 CS 740 F03

15

Properties of Some Topologies

Topology 1D Array 1D Ring 2D Mesh 2D Torus Degree Diameter Ave Dist Bisection 2 2 4 4 N-1 N/2 N/3 N/4 1 2 N1/2 2N1/2 nk/4 n/2

Switching Alternatives
Circuit Switching Packet Switching
Store-and-forward Cut-through Virtual cut-through Store & Forward Routing wormhole S o u rc e
3 2 1 0 3 2 1 3 2 3 0 1 0 2 1 0 3 2 1 0 3 2 1 3 2 3 0 1 0 2 1 0 3 2 1 0 Tim e 3 2 1 0

2 (N1/2 - 1) 2/3 N1/2 N1/2 nk/2 1/2 N1/2 nk/4 n

C u t -Through R o u ting Dest 3 2 1 0 3 2 1 3 2 3 0 1 2 3 0 1 2 3 0 1 0 2 1 0 3 2 1 0 Dest

k-ary n-cube2n

Hypercube n =log N

All have some bad permutations

many popular permutations are very bad for meshes (transpose) ramdomness in wiring or routing makes it hard to find a bad one!
17 CS 740 F03 18

CS 740 F03

Routing
Recall: routing algorithm determines
which of the possible paths are used as routes how the route is determined R: N x N -> C, which at each switch maps the destination node nd to the next channel on the route

Routing Mechanism
need to select output port for each input packet
in a few cycles

Issues:
Routing mechanism arithmetic source-based port select table driven general computation Properties of the routes Deadlock free
19 CS 740 F03

Reduce relative address of each dimension in order

Dimension-order routing in k-ary d-cubes e-cube routing in n-cube

20

CS 740 F03

Routing Mechanism
need to select output port for each input packet quickly! Simple arithmetic in regular topologies
ex: x, y routing west (-x) x < 0 east (+x) x > 0 south (-y) north (+y) processor in a grid

Routing Mechanism (cont)

P3 P2 P1 P0

Source-based
message header carries series of port selects used and stripped en route CRC? Packet Format? CS-2, Myrinet, MIT Artic

x = 0, y < 0 x = 0, y > 0 x = 0, y = 0

Table-driven
message header carried index for next port at next switch o = R[i] table also gives index for following hop o, I = R[i ] ATM, HPPI

Reduce relative addr of each dim in order

Dimension-order routing in k-ary d-cubes e-cube routing in n-cube
21 CS 740 F03

22

CS 740 F03

Properties of Routing Algorithms

Deterministic
route determined by (source, dest), not intermediate state (i.e. traffic)

Deadlock Freedom
How can it arise?
necessary conditions: shared resource incrementally allocated non-preemptible think of a channel as a shared resource that is acquired incrementally source buffer then dest. buffer channels along a route constrain how channel resources are allocated ex: dimension order

Adaptive
route influenced by traffic along the way

Minimal
only selects shortest paths

Deadlock free
no traffic pattern can lead to a situation where no packets mover forward

How do you avoid it?

How do you prove that a routing algorithm is deadlock free

24 CS 740 F03

23

CS 740 F03

Proof Technique
Resources are logically associated with channels Messages introduce dependences between resources as they move forward Need to articulate possible dependences between channels Show that there are no cycles in Channel Dependence Graph
find a numbering of channel resources such that every legal route follows a monotonic sequence

Example: k-ary 2D array

Theorem: x,y routing is deadlock free Numbering
+x channel (i,y) -> (i+1,y) gets i similarly for -x with 0 as most positive edge +y channel (x,j) -> (x,j+1) gets N+j similary for -y channels

=> no traffic pattern can lead to deadlock Network need not be acyclic, only channel dependence graph

Any routing sequence: x direction,1 turn, y 2 3 00 01 02 03 direction is increasing 2 1 0

18 17 10 17 20 16 30 19 31 32 33 18 21 22 23 11 12 13

25

CS 740 F03

26

CS 740 F03

Channel Dependence Graph

1 00 18 10 17 20 16 30 19 31 32 33
16 19 1 2

Examples
Why is the obvious routing on X deadlock free?
butterfly? tree? fat tree?

2 01 02 1 11 12

3 03 0 13
17 18 18 17

1 2 18 17 1 2 17 18 1 2 16 19

2 1 18 17 2 1 17 18 2 1 16 19 2 1

3 0 18 17 3 0 17 18 3 0 16 19 3 0

2 17

18 21 22 23

Any assumptions about routing mechanism? amount of buffering? What about wormhole routing on a ring?
2 1

0 3 4 5 6 CS 740 F03 7

27

CS 740 F03

28

Deadlock free wormhole networks?

Basic dimension-order routing doesnt work for k-ary d-cubes
only for k-ary d-arrays (bi-directional)

Breaking deadlock w/virtual chans

Idea: add channels!

provide multiple virtual channels to break dependence Input Output cycle Ports Ports good for BW too!
Cross-Bar

Packet switches from lo to hi channel

Dont need to add links, or xbar, only buffer resources

This adds nodes the the CDG, remove edges?

29 CS 740 F03 30 CS 740 F03

Routing Design Summary

Routing Algorithms restrict the set of routes within the topology
simple mechanism selects turn at each hop arithmetic, selection, lookup

Deadlock-free if channel dependence graph is acyclic

limit turns to eliminate dependences add separate channel resources to break dependences combination of topology, algorithm, and switch design

Deterministic vs adaptive routing

31

CS 740 F03