You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/streaming.rst
+50-70Lines changed: 50 additions & 70 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,127 +9,107 @@ Streaming
9
9
10
10
When dealing with problems that involve a lot of data such as images or large matrices, it is often the case that the data for the problem does not fit on the combined local memory of the Epiphany processor. In order to work with the data we must then use the larger (but much slower) external memory, which slows the programs down tremendously.
11
11
12
-
For these situations we provide a *streaming* mechanism. When writing your program to use streams, it will work on smaller chunks of the problem at any given time -- such that the data currently being treated is always local to the core. The EBSP library prepares the next chunk to work on while the previous chunk is being processed such that there is minimal downtime because the Epiphany cores are waiting for the slow external memory.
12
+
For these situations we provide a *streaming* mechanism. When writing your program to use streams, it will work on smaller tokens of the problem at any given time -- such that the data currently being treated is always local to the core. The EBSP library prepares the next token to work on while the previous token is being processed such that there is minimal downtime because the Epiphany cores are waiting for the slow external memory.
13
13
14
14
Making and using down streams
15
15
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
16
16
17
-
There are two types of streams, *up* and *down* streams. A *down* stream contains data to be processed by an Epiphany core, while an *up* stream contains results from computations performed by the Epiphany core. Every stream (both up and down) has a *target processor*, *total size* and a *chunk size*. The target processor is simply the processor id of the core that should receive the content of the stream. The total size is the total number of bytes of the entire set of data. This set of data then gets partitioned into chunks consisting of the number of bytes set by the chunk size. This size need not be constant (i.e. it may vary over a single stream), but for our discussion here we will assume that it is constant.
17
+
A stream contains data to be processed by an Epiphany core, and can also be used to obtain results from computations performed by the Epiphany core. Every stream has a *total size* and a *token size*. The total size is the total number of bytes of the entire set of data. This set of data then gets partitioned into tokens consisting of the number of bytes set by the token size. This size need not be constant (i.e. it may vary over a single stream), but for our discussion here we will assume that it is constant.
18
18
19
-
A stream is created before the call to ``ebsp_spmd`` on the host processor. The host prepares the data to be processed by the Epiphany cores, and the EBSP library then performs the necessary work needed for each core to receives its chunk. Note that this data is copied efficiently to the external memory upon creation of the stream, so that the user data should be stored in the ordinary RAM, e.g. allocated by a call to ``malloc``. A stream is created as follows::
19
+
A stream is created before the call to ``ebsp_spmd`` on the host processor. The host prepares the data to be processed by the Epiphany cores, and the EBSP library then performs the necessary work needed for each core to receives its token. Note that this data is copied efficiently to the external memory upon creation of the stream, so that the user data should be stored in the ordinary RAM, e.g. allocated by a call to ``malloc``. A stream is created as follows::
This will create ``bsp_nprocs()`` identical streams containing user data, one for each core. These streams are chopped up in ``256/32 = 8`` chunks. If you want to use these streams in the kernel you need to *open* them and *move chunks* from a stream to the local memory. Every stream you create on the host gets is identified by the order in which they are created. For example, the stream we created above will obtain the id ``0`` on every core. A second stream (regardless of whether it is up or down) will be identified with ``1``, etc. *These identifiers are shared between up and down streams, but not between cores*. Opening a stream is done by using this identifier::
ebsp_open_down_stream(&(void*)address, // a pointer to the address store
36
-
0); // the stream identifier
28
+
This will create a stream containing user data. This stream is chopped up in ``256/32 = 8`` tokens. If you want to use this streams in the kernel of a core you need to *open* it and *move tokens* from a stream to the local memory. Every stream you create on the host gets is identified by the order in which they are created, starting from index ``0``. For example, the stream we created above will obtain the id ``0``. A second stream (regardless of whether it is up or down) will be identified with ``1``, etc. *These identifiers are shared between cores*. Opening a stream is done by using this identifier, for example, to open a stream with identifier ``3``::
37
29
38
-
After this call, address will contain the location in the local memory of the first chunk, but the data is not necessarily there yet (it might still be copying). To ensure that the data has been received we *move* a chunk::
After this call, the stream will start copying data to the core, but the data is not necessarily there yet (it might still be copying). A stream can only be opened by *a single core at a time*. To access this data we *move* a token::
42
36
43
-
The first two arguments are identical to those of ``ebsp_open_down_stream``. The ``double_buffer`` argument gives you the option to start writing the next chunk to local memory (using the DMA engine), while you process the current chunk that just moved down. This can be done simultaneously to your computations, but will take up twice as much memory. It depends on the specific situation whether double_buffered mode should be turned on or off. Subsequent blocks are obtained using repeated calls to ``ebsp_move_chunk_down``.
37
+
// Get some data
38
+
void* buffer = NULL;
39
+
bsp_stream_move_down(&mystream, &buffer, 0);
40
+
// The data is now in buffer
44
41
45
-
If you want to use a chunk multiple times at different stages of your algorithm, you need to be able to instruct EBSP to change which chunk you want to obtain. Internally the EBSP system has a *cursor* for each stream which points to the next chunk that should be obtained. You can modify this cursor using the following two functions::
42
+
The first argument is the stream object that was filled using ``bsp_stream_open``. The second argument is a pointer to a pointer that will be set to the data location. The final ``double_buffer`` argument, gives you the option to start writing the next token to local memory (using the DMA engine), while you process the current token that you just moved down. This can be done simultaneously to your computations, but will take up twice as much memory. It depends on the specific situation whether double buffered mode should be turned on or off. Subsequent blocks are obtained using repeated calls to ``bsp_stream_move_down``.
46
43
47
-
// reset the cursor of the first stream to its first chunk
48
-
ebsp_reset_down_cursor(0);
44
+
If you want to use a token multiple times at different stages of your algorithm, you need to be able to instruct EBSP to change which token you want to obtain. Internally the EBSP system has a *cursor* for each stream which points to the next token that should be obtained. You can modify this cursor using the following two functions::
49
45
50
-
// move the cursor of the first stream forward by 5 chunks
51
-
ebsp_move_down_cursor(0, 5);
46
+
// move the cursor of the stream forward by 5 tokens
47
+
bsp_stream_seek(&mystream, 5);
52
48
53
-
// move the cursor of the first stream back by 3 chunks
54
-
ebsp_move_down_cursor(0, -3);
49
+
// move the cursor of the stream back by 3 tokens
50
+
bsp_stream_seek(&mystream, -3);
55
51
56
-
Note that this gives you random access inside your streams. Therefore our streaming approach should actually be called *pseudo-streaming*, because formally streaming algorithms only process chunks in a stream a constant number of times. However on the Epiphany we can provide random-access in our streams, leading to different semantics such as moving the cursor.
52
+
When you exceed the bounds of the stream, it will be set to the final or first token respectively. Note that this gives you random access inside your streams. Therefore our streaming approach should actually be called *pseudo-streaming*, because formally streaming algorithms only process tokens in a stream a constant number of times. However on the Epiphany we can provide random-access in our streams, opening the door to different semantics such as moving the cursor.
57
53
58
54
Moving results back up
59
55
^^^^^^^^^^^^^^^^^^^^^^
60
56
61
-
Up streams work very similar to down streams, however no data has to be supplied by the host since it is generated by the Epiphany. We construct an up stream in the following way::
A stream can also be used to move results back up, for example::
70
58
71
-
The array ``upstream_data`` holds pointers to the generated data by each processor. In the kernel you can *open* these streams similarly to down streams::
59
+
int* buffer1 = ebsp_malloc(100 * sizeof(int));
60
+
int* buffer2 = ebsp_malloc(100 * sizeof(int));
61
+
int* curbuffer = buffer1;
62
+
int* otherbuffer = buffer2;
72
63
73
-
// in the kernel
74
-
float* up_address = NULL;
75
-
ebsp_open_up_stream(&(void*)up_address, // a pointer to the address store
76
-
1); // the stream identifier
64
+
ebsp_stream s;
65
+
bsp_stream_open(&s, 0); // open stream 0
66
+
while (...) {
67
+
// Fill curbuffer
68
+
for (int i = 0; i < 100; i++)
69
+
curbuffer[i] = 5;
77
70
78
-
Note that this stream has the identifier ``1`` on each core. The up_address now points to a portion of *local memory* that you can fill with data from the kernel. To move a chunk of results up we use::
If we use a double buffer, then after this call ``up_address`` will point to a new portion of memory, such that you can continue your operations while the previous chunk is being copied up. Again, this uses more local memory, but does allow you to continue processing the next chunk.
79
+
Here, we have two buffers containing data. While filling one of the buffers with data, we move the other buffer up. We do this using the ``bsp_stream_move_up`` function which has as arguments respectively: the stream handle, the data to send up, the size of the data to send up, and a flag that indicates whether we want to *wait for completion*. In this case, we do not wait, but use two buffers to perform computations and to send data up to the host simulatenously.
84
80
85
81
Closing streams
86
82
^^^^^^^^^^^^^^^
87
83
88
84
The EBSP stream system allocates buffers for you on the cores. When you are done with a stream you should tell the EBSP system by calling::
89
85
90
-
ebsp_close_down_stream(0);
91
-
ebsp_close_up_stream(0);
86
+
bsp_stream_close(&my_stream);
92
87
93
-
which will free the buffers for other use.
88
+
which will free the buffers for other use, and allow other cores to use the streams.
0 commit comments