Mark Raasveldt
Parallel Quacking
Parallel Quacking
▸ When building DuckDB we have mostly
focused on building a functional system
▸ Avoid premature optimization
▸ Avoid adding optimizations that prevent
adding features
Parallel Quacking
▸ Suddenly people are benchmarking our system
▸ Including benchmarks in research papers
▸ Yikes!
▸ We haven’t exactly spend a lot of time
optimizing…
Parallel Quacking
▸ We are now pretty happy with functionality
▸ Window functions, subqueries, collations,
(recursive) CTEs, Parquet/Pandas/CSV
readers, …
▸ Maybe we should start optimizing!
Parallel Quacking
▸ DuckDB is currently single-threaded
▸ Parallelism is an obvious performance boost
▸ More importantly: parallelism requires a
structural change to the code
▸ Optimizations need to account for parallelism
▸ Optimizing a single-threaded HT is pointless if
we have to throw it away once we add
parallelism!
Parallel Quacking
▸ Parallelism is actually our oldest open issue!
▸ Created one month after the initial commit
▸ So it’s about time :)
DBMS Parallelism
▸ Short intro to DBMS parallelism
▸ DBMS have two types of parallelism
▸ Inter-query and intra-query parallelism
▸ Inter-query: multiple different queries
can be executed in parallel
▸ Intra-query: a single query can be
parallelized
DBMS Parallelism
▸ Most systems have inter-query
▸ We already had this
▸ Most useful for OLTP systems
▸ Many concurrent clients requests, etc
DBMS Parallelism
▸ Intra-query is not part of most OLTP
systems
▸ e.g. MySQL/PostgreSQL/SQLite
▸ Not useful for small queries
▸ Only useful for complex queries
▸ Aka OLAP systems
DBMS Parallelism
▸ Exchange operator: original way of
doing parallelism
▸ Parallelism is encapsulated in the
exchange operator
▸ All other ops are unaware of parallelism
▸ Easy to bolt onto existing systems
[1993] Encapsulation of Parallelism and
Architecture-Independence in Extensible
Database Query Execution
Goetz Graefe et al.
DBMS Parallelism
DBMS Parallelism
▸ MonetDB uses system similar to exchange
operator
▸ Individual ops are parallelism-unaware
▸ Data is partitioned by mitosis (mergetable?)
▸ Ops execute sequentially on partitions
▸ Result is combined by mat.pack
DBMS Parallelism
▸ Exchange operator works to parallelize queries
▸ It is nice to bolt on to an existing system
▸ Don’t need to change any operators!
▸ But has partitioning/merging overhead…
▸ Works well for certain queries1, not for many
others
▸ 1ungrouped aggregates or aggregates with low
amount of groups
Morsel-Driven Parallelism
▸ Alternative: Morsel-driven parallelism
▸ Parallelism-aware operators
▸ Query is divided into pipelines
▸ Those pipelines are executed in parallel
[2014] Morsel-Driven Parallelism: A
NUMA-Aware Query Evaluation
Framework for the Many-Core Age
Viktor Leis et al.
Morsel-Driven Parallelism
SELECT …
FROM S
JOIN R USING (A) 3: Probe HTs and output result
JOIN T USING (B); (depends on 1 and 2)
1: HT Build “T”
2: HT Build “S”
Morsel-Driven Parallelism
SELECT …
FROM S
JOIN R USING (A)
JOIN T USING (B);
HT Build “T”
HT Build “S”
▸ HT builds of S and T can be trivially parallelized
▸ No shared data
▸ Limited parallelizability: depends on Q complexity…
Morsel-Driven Parallelism
▸ Need to parallelize inside a pipeline
▸ How to do that?
▸ Contention happens at endpoints
▸ Scan of T
▸ HT build at join HT Build “T”
▸ Use parallelism-aware operators at endpoints
▸ The rest of the operators (HT probe, projection,
filter, etc…) don't need to be aware
Morsel-Driven Parallelism
TPC-H SF100, 32 cores
[2014] Morsel-Driven Parallelism: A
NUMA-Aware Query Evaluation
Framework for the Many-Core Age
Viktor Leis et al.
Morsel-Driven Parallelism
TPC-H SF100, 32 cores
[2014] Morsel-Driven Parallelism: A
NUMA-Aware Query Evaluation
32 cores, 64 hardware threads Framework for the Many-Core Age
Viktor Leis et al.
Morsel-Driven Parallelism
TPC-H SF100, 32 cores
[2014] Morsel-Driven Parallelism: A
NUMA-Aware Query Evaluation
32 cores, 64 hardware threads Framework for the Many-Core Age
Viktor Leis et al.
Morsel-Driven Parallelism
TPC-H SF100, 32 cores
[2014] Morsel-Driven Parallelism: A
NUMA-Aware Query Evaluation
32 cores, 64 hardware threads Framework for the Many-Core Age
Viktor Leis et al.
Morsel-Driven Vegetable Soup
▸ Morsel-driven parallelism seems like the way to go
▸ How can we add it to our vegetable soup?
Parallelism in DuckDB
▸ DuckDB uses a pull-based volcano execution model
▸ "Vector Volcano”
▸ Every operator implements a GetChunk operator
▸ Recursively calls GetChunk on children
▸ Until we reach a data source (e.g. table scan)
Parallelism in DuckDB
▸ BuildHashTable: pull everything from RHS (build-side)
▸ ProbeHashTable: pull single chunk from LHS (probe
side)
Parallelism in DuckDB
▸ Have to split up building from probing
▸ Create individual pipelines
▸ Design interface that allows for parallel-aware execution
Parallelism in DuckDB
▸ Contention is in the source and sink of a pipeline
▸ Most difficult contention is in the sink
▸ Splitting up a scan is relatively simple
Parallelism in DuckDB
▸ Sink Interface
▸ Sink has two states
▸ Global state: single state per sink
▸ Local state: single state per thread
▸ Actual content depends on the operator
Parallelism in DuckDB
▸ Sink Interface
▸ Sink takes as input the two states + a DataChunk
▸ Called repeatedly until the source data is exhausted
Parallelism in DuckDB
▸ Sink Interface
▸ Combine is called after source of a single thread is
exhausted
▸ Combine is the final chance to merge any changes
in the local sink state to the global state
Parallelism in DuckDB
▸ Sink Interface
▸ Finalize is called after all tasks related to the sink
are completed
Parallelism in DuckDB
▸ Example: Ungrouped Aggregate
▸ Global state holds the aggregate result, and a lock
Parallelism in DuckDB
▸ Example: Ungrouped Aggregate
▸ Local state holds a thread-local aggregate, and
some intermediates
Parallelism in DuckDB
▸ Example: Ungrouped Aggregate
▸ Sink: Aggregate into thread-local aggregation
Parallelism in DuckDB
▸ Example: Ungrouped Aggregate
▸ Combine: Merge local state into global state
Parallelism in DuckDB
▸ Example: Ungrouped Aggregate
▸ Finalize: Nothing, we are done
▸ (both Combine and Finalize are optional)
Parallelism in DuckDB
▸ Splitting up scans
▸ Splitting up scans is generally not very difficult
▸ But we have multiple types of scans
▸ Base table, parquet, CSV, aggregate HT, etc…
▸ How to split up depends on scan type
Parallelism in DuckDB
▸ Interface for parallel scans:
▸ One task is created for every invoked callback
▸ Implementation is optional
▸ No implementation -> scan will not be parallelized
Parallelism in DuckDB
▸ Currently only implemented for base table
▸ One task for every 100 vectors (102,400 tuples)
▸ Parquet/Pandas is not very complicated
▸ CSV can also benefit…
▸ Future work!
Parallelism in DuckDB
▸ Creating the pipelines
▸ Created by a single traversal of the query tree
▸ Encounter a pipeline breaker: create a new
pipeline
Parallelism in DuckDB
Encounter hash join: create build pipeline in RHS*
SELECT … and create a dependency in main pipeline
FROM S
JOIN R USING (A)
JOIN T USING (B);
* This image is taken from HyPer which builds on the LHS - we build on the RHS.
Is there a standard? Should we switch this? Is it even important?
Parallelism in DuckDB
SELECT …
FROM S
JOIN R USING (A)
Another hash join: create another
JOIN T USING (B); build pipeline and dependency
Parallelism in DuckDB
TPC-H Q1
P1 (depends on P2)
Scans the aggregate HT!
P2
This 0 is a bug in our profiler with parallel execution atm, TODO
Parallelism in DuckDB
▸ Notes on parallelism
▸ The final pipeline (i.e. the one that outputs
results) is not parallelized
▸ Doesn’t matter for TPC-H (there is always a Top-N
or ORDER BY…)
▸ But can definitely matter for other queries!
▸ We can push a “materialize” operator that
materializes in parallel
▸ Future work!
Parallelism in DuckDB
▸ Notes on load balancing
▸ Pipelines are split into tasks
▸ Tasks are scheduled in a concurrent queue
▸ Worker threads work on these tasks in scheduled
order
▸ Except the calling thread: this thread works on its
own query
▸ Short queries will not have to wait for long queries
▸ Every query has at least one thread working on it
Parallelism in DuckDB
▸ NUMA Awareness
▸ TODO :)
Preliminary Results
▸ Results
▸ Before we implemented splitting of scans we were
curious
▸ How much does TPC-H benefit from inter-pipeline
parallelization?
Preliminary Results
▸ Small speedup in some queries
▸ Most queries are dominated by a single pipeline!
* Actually 3 threads, due to an off-by-one :)
Preliminary Results
▸ Preliminary results (including splitting of pipelines)
▸ Notes
▸ We did not implement a good aggregate HT yet!
▸ Currently global HT that is locked on every sink
▸ Join HT/scan also have a (low) amount of contention
▸ Did not have much time to look at it yet
▸ This was all finished last Thursday :)
Preliminary Results
▸ Preliminary results
Preliminary Results
▸ Preliminary results
Preliminary Results
▸ Q1
Parallel Sequential
Preliminary Results
▸ Q18 Sequential
Preliminary Results
▸ Q18 Parallel
Future Work
▸ Future Work
▸ Rework aggregate hash table
▸ More profiling of contention (specifically in scans)
▸ Parallel window functions, ORDER BY, Top N…
▸ Parallel Parquet/CSV/Pandas scans
▸ Expand profiler to better display parallelism/
pipelines
