<li><a href="#ParallelScalablePipelineCreateAScalablePipelineModuleTask">Create a Scalable Pipeline Module Task</a></li>

<p>Unlike <a href="classtf_1_1Pipeline.html" class="m-doc">tf::<wbr />Pipeline</a> (see <a href="ParallelPipeline.html" class="m-doc">Parallel Pipeline</a>) that requires instantiation of all pipes at the construction time, Taskflow provides a scalable alternative called <a href="classtf_1_1ScalablePipeline.html" class="m-doc">tf::<wbr />ScalablePipeline</a> to allow variable assignments of pipes using range iterators. A scalable pipeline is thus more flexible for applications to create a pipeline scheduling framework whose pipeline structure depends on runtime variables.</p><section id="ParallelScalablePipelineIncludeHeaderFile"><h2><a href="#ParallelScalablePipelineIncludeHeaderFile">Include the Header</a></h2><p>You need to include the header file, <code>taskflow/algorithm/pipeline.hpp</code>, for creating a pipeline scheduling framework.</p><pre class="m-code"><span class="cp">#include</span> <span class="cpf">&lt;taskflow/algorithm/pipeline.hpp&gt;</span><span class="cp"></span></pre></section><section id="ParallelScalablePipelineCreateAScalablePipelineModuleTask"><h2><a href="#ParallelScalablePipelineCreateAScalablePipelineModuleTask">Create a Scalable Pipeline Module Task</a></h2><p>Similar to <a href="classtf_1_1Pipeline.html" class="m-doc">tf::<wbr />Pipeline</a>, <a href="classtf_1_1ScalablePipeline.html" class="m-doc">tf::<wbr />ScalablePipeline</a> is a composable graph object that can be used to embed a <em>pipeline scheduling framework</em> in a taskflow. The key difference between <a href="classtf_1_1Pipeline.html" class="m-doc">tf::<wbr />Pipeline</a> and <a href="classtf_1_1ScalablePipeline.html" class="m-doc">tf::<wbr />ScalablePipeline</a> is that a scalable pipeline can accept <em>variable</em> assignments of pipes rather than requiring all pipes to present at construction/programming time. Users define a linear range of pipes, each of the same callable type, and apply that range to construct a scalable pipeline. Between successive runs, users can reset the pipeline to a different range of pipes. The following code creates a scalable pipeline using four parallel lines to schedule tokens through three serial pipes defined in a custom vector, then resetting that pipeline to a new range of five serial pipes:</p><pre class="m-code"><span class="n">tf</span><span class="o">::</span><span class="n">Taskflow</span> <span class="n">taskflow</span><span class="p">(</span><span class="s">&quot;pipeline&quot;</span><span class="p">);</span>

<span class="n">executor</span><span class="p">.</span><span class="n">run</span><span class="p">(</span><span class="n">taskflow</span><span class="p">).</span><span class="n">wait</span><span class="p">();</span></pre><p>The program defines a uniform pipe type of <code>tf::Pipe&lt;std::function&lt;void(tf::Pipeflow&amp;)&gt;&gt;</code> and keep all pipes in a vector that is amenable to change. Then, it constructs a scalable pipeline using two range iterators, <code>[first, last)</code>, pointing to the beginning and the end of the pipe vector, resulting in a pipeline of three serial stages:</p><div class="m-graph"><svg style="width: 22.250rem; height: 18.438rem;" viewBox="0.00 0.00 356.00 295.08">

<span class="go">o -&gt; o -&gt; o -&gt; o -&gt; o</span></pre><p>Each pipe has the same type of <code>tf::Pipe&lt;std::function&lt;void(tf::Pipeflow&amp;)&gt;&gt;</code> and is kept in a vector that is amenable to change. We construct the scalable pipeline using two range iterators pointing to the beginning and the end of the vector. At each pipe stage, the program propagates the result to the next pipe by adding one to the result stored in a custom data storage, <code>buffer</code>. The pipeline scheduler will generate five scheduling tokens and then stop.</p>

<p>Taskflow 3.4.0 is the newest developing line to new features and improvements we continue to support. It is also where this documentation is generated. Many things are considered <em>experimental</em> and may change or break from time to time. While it may be difficult to be keep all things consistent when introducing new features, we continue to try our best to ensure backward compatibility.</p><section id="release-3-4-0_download"><h2><a href="#release-3-4-0_download">Download</a></h2><p>To download the newest version of Taskflow, please clone the master branch from <a href="https://github.com/taskflow/taskflow">Taskflow&#x27;s GitHub</a>.</p></section><section id="release-3-4-0_system_requirements"><h2><a href="#release-3-4-0_system_requirements">System Requirements</a></h2><p>To use Taskflow v3.4.0, you need a compiler that supports C++17:</p><ul><li>GNU C++ Compiler at least v8.4 with -std=c++17</li><li>Clang C++ Compiler at least v6.0 with -std=c++17</li><li>Microsoft Visual Studio at least v19.27 with /std:c++17</li><li>AppleClang Xode Version at least v12.0 with -std=c++17</li><li>Nvidia CUDA Toolkit and Compiler (nvcc) at least v11.1 with -std=c++17</li><li>Intel C++ Compiler at least v19.0.1 with -std=c++17</li><li>Intel DPC++ Clang Compiler at least v13.0.0 with -std=c++17 and SYCL20</li></ul><p>Taskflow works on Linux, Windows, and Mac OS X.</p></section><section id="release-3-4-0_summary"><h2><a href="#release-3-4-0_summary">Release Summary</a></h2></section><section id="release-3-4-0_new_features"><h2><a href="#release-3-4-0_new_features">New Features</a></h2><section id="release-3-4-0_taskflow_core"><h3><a href="#release-3-4-0_taskflow_core">Taskflow Core</a></h3><ul><li>Improved the pipeline performance using vertical stack optimization</li><li>Added <a href="classtf_1_1ScalablePipeline.html" class="m-doc">tf::<wbr />ScalablePipeline</a> to allow programming variable lengths of pipes</li><li>Added an example of attaching data to a task (examples/attach_data.cpp)</li><li>Added an example of text processing pipeline (examples/parallel_text_pipeline.cpp)</li><li>Added an example of graph processing pipeline (examples/parallel_graph_pipeline.cpp)</li><li>Added an example of taskflow processing pipeline (examples/parallel_taskflow_pipeline.cpp)</li></ul></section><section id="release-3-4-0_cudaflow"><h3><a href="#release-3-4-0_cudaflow">cudaFlow</a></h3></section><section id="release-3-4-0_syclflow"><h3><a href="#release-3-4-0_syclflow">syclFlow</a></h3></section><section id="release-3-4-0_utilities"><h3><a href="#release-3-4-0_utilities">Utilities</a></h3></section><section id="release-3-4-0_profiler"><h3><a href="#release-3-4-0_profiler">Taskflow Profiler (TFProf)</a></h3></section></section><section id="release-3-4-0_bug_fixes"><h2><a href="#release-3-4-0_bug_fixes">Bug Fixes</a></h2><ul><li>Fixed compilation error by non-portable include of <code>immintrin.h</code> (<a href="https://github.com/taskflow/taskflow/issues/371">#371</a>)</li><li>Fixed compilation error by old version of doctest (<a href="https://github.com/taskflow/taskflow/issues/372">#372</a>)</li></ul><p>If you encounter any potential bugs, please submit an issue at <a href="https://github.com/taskflow/taskflow/issues">issue tracker</a>.</p></section><section id="release-3-4-0_breaking_changes"><h2><a href="#release-3-4-0_breaking_changes">Breaking Changes</a></h2></section><section id="release-3-4-0_deprecated_items"><h2><a href="#release-3-4-0_deprecated_items">Deprecated and Removed Items</a></h2></section><section id="release-3-4-0_documentation"><h2><a href="#release-3-4-0_documentation">Documentation</a></h2><ul><li>Revised <a href="ParallelPipeline.html" class="m-doc">Parallel Pipeline</a><ul><li>Added <a href="ParallelPipeline.html#ParallelPipelineLearnMore" class="m-doc">Learn More about Taskflow Pipeline</a></li></ul></li><li>Revised <a href="Examples.html" class="m-doc">Learning from Examples</a><ul><li>Added <a href="TextProcessingPipeline.html" class="m-doc">Text Processing Pipeline</a></li><li>Added <a href="GraphProcessingPipeline.html" class="m-doc">Graph Processing Pipeline</a></li><li>Added <a href="TaskflowProcessingPipeline.html" class="m-doc">Taskflow Processing Pipeline</a></li></ul></li><li>Added <a href="ParallelScalablePipeline.html" class="m-doc">Parallel Scalable Pipeline</a></li></ul></section><section id="release-3-4-0_miscellaneous_items"><h2><a href="#release-3-4-0_miscellaneous_items">Miscellaneous Items</a></h2><p>We have published Taskflow in the following venues:</p><ol><li>Dian-Lun Lin and Tsung-Wei Huang, &quot;<a href="https://ieeexplore.ieee.org/document/9664223">Accelerating Large Sparse Neural Network Inference using GPU Task Graph Parallelism</a>,&quot; <em>IEEE Transactions on Parallel and Distributed Systems (TPDS)</em>, accepted, 2022</li></ol><p>Please do not hesitate to contact <a href="https://tsung-wei-huang.github.io/">Dr. Tsung-Wei Huang</a> if you intend to collaborate with us on using Taskflow in your scientific computing projects.</p></section>