As promised in the last call, here's an explanation of the thought process behind this proposal. This explanation has some overlap with the Alternatives section of #1073.

Explicit synchronization.

The basis of this proposal is that it is important that the synchronization be visible to the developers. The multi-queue feature's goal is to expose coarse-grained parallelism of the hardware to more advanced WebGPU users. These users will want to control that if the hardware supports multiple queues, then for example the memory-bound workload on one queue runs in parallel of the fixed-function-bound workload on the other queue because it maximizes utilization.

The DAG of execution of work on multiple queues needs to be in direct control of the developer and easily visible in the code. This is why the approach starts with the assumption that the transfers of resources have to be explicit and will produce validation errors if done wrong (instead of UB or additional implicit synchronization). Developers need the guarantee that the DAG of execution that's used is exactly the one that they specified.

Explicit synchronization via "fences".

One possibility, like @RafaelCintron commented above, or @kvark suggested in #1066 is that the operation of doing a transfer on a queue is in itself an "explicit synchronization" in that it shown in the code. The receiving queue's following operations (or following operations that use the transferred resource) would make sure it waits until the transferring queue has completed commands before the transfer.

This is a semantic that is sound, but there's several drawbacks.

The first one is that it adds constraints on the order in which the DAG is defined. All proposals need to require that the DAG be specified in a topological order, but having the resource transfers themselves define the synchronization reduces flexibility for the application on the order in which it defines the DAG. Basically if a developer wants to produce the following DAG:

queueA: -A1------A2--->
             \
              \
queueB: -B1------B2--->

Then they needs to do the following calls:

queueB.submit(B1);
queueA.submit(A1, transferToB);

queueA.submit(A2);
queueB.submit(B2)

But it would be much more convenient for them to do the following because it means the code for submitting to A and B can be more independent:

queueA.submit(A1, transferToB);
queueA.signal(42);
queueA.submit(A2);

queueB.submit(B1);
queueB.wait(queueA, 42);
queueB.submit(B2);

For composability of code it would be even better thing would be to be able to wait on a queue before signaling on another one but that brings additional issues so it isn't suggested in the proposal above.

Transfers being synchronization also makes the synchronization less visible. It hides one half of the synchronization, the one for the receiving queue, inside the transfer command. This makes the synchronization invisible in the code that uses the receiving queue. It also buries the synchronization inside dictionaries in a list that's passed as one of the argument function call. The list of transfers could be built inside the deep rendering logic and bubbled up, then in the code that does the submit you'd see queue.submit(commands, transfers), good luck figuring out what synchronization is implied!

Finally some synchronization is useful even without transfers. Like discussed above, the application may want to synchronize queues just to make sure that the workloads being executed in parallel have different bottlenecks. This could be done with fake transferred, but really the synchronization and the transfer parts should be more orthogonal.

So overall it's not enough to have transfers given in a call to submit do the synchronization. Synchronization should be done with GPUFence-like signal and wait primitives that build the DAG very explicitly, and transfers should build on that for validation.

Merging GPUFence in GPUQueue.

The validation for transfers need to ensure that there's a path in the execution DAG from the transfer to the use of the transferred resources. I thought on and off for the past year on how to do this perfectly when using GPUFence. It's hard. The complexity comes from the fact that for each fence signal value we need to know what it implies for other fences (so that we can correctly handle chains of DAG edges). It might look simple if you decide to decay each (fence, value) pair to the underlying implementation's (queue, value) pair, but then how do you handle correctly situations where a queue signals multiple fences without operations in the middle: is that the same (queue, value) pair or different ones? Each of these paths lead to a lot of spec and implementation complexity.

In #1073 I gave up and proposed to make the transfers require synchronization through the same GPUFence and gave up on chains of DAG edges. It works, is "easily" implementable but is kind of surprising. The other option that came up in a discussion with @kvark was to have a single fence per queue because then there isn't the complexity of propagating DAG edges from/to the same queue via multiple different GPUFence objects. Once you decide to have a single GPUFence per GPUQueue it makes sense to merge them.

Merging the GPUQueue and GPUFence has other benefits in addition to allowing this multi-queue proposal: it makes the API simpler (less concepts, less indirection), it helps composability because there is a single execution serial for the GPUQueue (instead of each middleware creating its own GPUFence). It also encodes at a type-level the previous constraints that a GPUFence had to always be signaled by the same queue and with increasing values.

I don't see a drawback today in merging GPUFence in GPUQueue and my intuition is that we aren't going to have issues with native API changes adding semantic to their native fence object that we'll have trouble replicating.

The first one is that it adds constraints on the order in which the DAG is defined.

For the example you gave, since the API is required to put the wait for A1 right before the submission of B2, you can submit the work as you have in the second section (labeled as the much more convenient one) even with implicit waits. How do explicit waits make things more flexible? It's already the case you can't wait for particular number before you signal that number so I must be missing something.

Transfers being synchronization also makes the synchronization less visible ... good luck figuring out what synchronization is implied!

WebGPU already hides synchronization for buffer mapping and all memory barriers. What is it about multi-queue synchronization that you feel needs the synchronization to be explicit?

Finally some synchronization is useful even without transfers. Like discussed above, the application may want to synchronize queues just to make sure that the workloads being executed in parallel have different bottlenecks.

Can you give an example of a scenario or workload that would benefit from explicit transfers where implicit transfers would otherwise lead to sub-optimal execution or incorrect results?

For the example you gave, since the API is required to put the wait for A1 right before the submission of B2, you can submit the work as you have in the second section (labeled as the much more convenient one) even with implicit waits. How do explicit waits make things more flexible? It's already the case you can't wait for particular number before you signal that number so I must be missing something.

The point that wasn't very clear is that if the transfers are the synchronization, then the following code will schedule B1 after A1, so there's more constraints on your code's structure if you want to control the exact shape of the DAG. Or the WebGPU implementation knows that the resource is only used in B2, but then it decides on the DAG in a manner that's not very predictable and explicit.

queueA.submit(A1, transferToB);
queueA.submit(A2);

queueB.submit(B1);
queueB.submit(B2)

WebGPU already hides synchronization for buffer mapping and all memory barriers. What is it about multi-queue synchronization that you feel needs the synchronization to be explicit?

Can you give an example of a scenario or workload that would benefit from explicit transfers where implicit transfers would otherwise lead to sub-optimal execution or incorrect results?

Actually synchronization for buffer mapping is explicit, this is why we have mapAsync. Multiqueue needs explicit synchronization because the developers need to control which parts are run in parallel to maximize utilization of the various parts of the GPU. See for example the "Async Post Processing" of Doom 2016's Advances in Realtime Rendering deck.

Multiqueue needs explicit synchronization because the developers need to control which parts are run in parallel to maximize utilization of the various parts of the GPU.

I agree that native APIs let you do powerful things but that's because the developer knows more about their data dependencies than what they can express to the API. However, WebGPU is (by design) more limiting because any graph you build must be a subset of what the validation layer is willing to let you create. In a sense, the validation layer's graph is the upper limit of the graph the web developer can build. For portability, that graph should be speced.

The WebGPU API knows what resources are owned each queue and which ones are used in each command list. Does there exist a set of WebGPU instructions such that the developer can build a better graph than the API but still pass the API's validation and be free of data races? If so, that would be a good example to talk about.

Actually synchronization for buffer mapping is explicit, this is why we have mapAsync

Pedantic: Having buffer mapping use promises is one form of being explicit, yes. My point was that we didn't spec buffer mapping by having the web developer signal and wait on fence values. One can say that passing a list of resources to submit with the destination queue clearly spelled out is also a form of being explicit.

The WebGPU API knows what resources are owned each queue and which ones are used in each command list. Does there exist a set of WebGPU instructions such that the developer can build a better graph than the API but still pass the API's validation and be free of data races? If so, that would be a good example to talk about.

Yes, by making workloads that maximize different resources of the GPU at the same time. See the presentation linked that says they specifically overlap shadow map generation with post-processing because stress the fixed function while the other stresses ALUs. Explicit control helps achieve this level of detail, otherwise developers who are interested in it will have to fake signal-wait synchronization with dummy resource transfers.

Pedantic: Having buffer mapping use promises is one form of being explicit, yes. My point was that we didn't spec buffer mapping by having the web developer signal and wait on fence values. One can say that passing a list of resources to submit with the destination queue clearly spelled out is also a form of being explicit.

I think the analogy is interesting but not too relevant as buffer mapping in CPU - GPU synchronization with the interesting part being how it is exposed in javascript while multi queue is GPU - GPU synchronization that's pipelined on an asynchronous coprocessor.

For composability of code it would be even better thing would be to be able to wait on a queue before signaling on another one but that brings additional issues so it isn't suggested in the proposal above.

As an aside, it occurs to me that if we made flushing buffered-up queue work (submits, writes, waits) explicit, with a flush() call (as was briefly discussed in the past), then we could allow out of order waits and validate in flush().

I see that @RafaelCintron raised exactly the concerns I had after reading @Kangz long comment 🥇

I don't think explicit sync gives you any more composability or flexibility. You still have to submit things in order on different queues, and in addition you have to care about fence values. What explicit improves, however, is that now you can see the DAG by reading the GPUQueue calls only. You can reason about dependencies, and consequently it becomes easier both to read and maintain multi-queue code. So it's a solid win for the API quality.

I brought back the fence code this PR was originally removing, as requested.

I believe this PR is sufficiently simple for both users and implementors to start prototyping. I don't think we should rush into trying to smooth things into being more automated/implicit until we get more feedback. After all, the current group has little-to-no experience with multiple GPU queues, so we shouldn't try too hard to make it "ideal" for users until we know more.
If there are no blockers by the next call, I suggest we proceed.

@kvark , I am definitely in favor of web developers explicitly transferring resources between queues via the transfers dictionary passed to submit.

However, I question the need to also require web developers to put waits and signals. Since the WebGPU implementation will be fully validating waits and signals and prevents developers from waiting on a number that hasn't been signaled, I think it can do a good enough job for developers. The transfers dictionary already provides a level of explicitness which I feel is adequate.

If we need more explicit control, we can add it as a a future change when we consider advanced features like (more) explicit memory barriers and bindless.

@RafaelCintron thank you for feedback. I think having transfers with implicit wait takes an interesting position in the solution space that may see like a good compromise, but there are caveats. Here is the "explicitness ladder" in this space:

1. Fully implicit transfers and waits. Suffers from the need to do a dummy submission on Vulkan, which reportedly is slow at least on mobile GPUs. Also, the DAG is not visible at all, making it hard to reason about the behavior of a program.
2. Implicit waits with transfers that are hints to the implementation. If the actual usage diverges from the hint, e.g. resource was hinted to transfer to queue A but ends up being used on queue B, then the implementation would have to create a dummy submission to A with the only purpose or receiving that resource and sending it to B. That would be a very unfortunate code path to have, I'm sure no users would want to hit it. Also, DAG is only partially visible: we see what resources are getting transferred, but we don't see where the semaphore/fence waits are going to happen upon submissions (i.e. where the queues are synced up).
3. Implicit waits with explicit transfers that are required. If the actual usage diverges, an error is generated, and the submission is dropped. This may or may not break the whole application. Again, the DAG is only partially visible.
4. Fully explicit waits and transfers. If the actual usage diverges, an error is generated, and the submission is dropped. This may or may not break the whole application. The DAG is fully visible at the GPUQueue level.

So when we consider the transfers without the waits, we are talking about (2) or (3), both of which have several issues to them. Were you suggesting the transfers to be hints (as in (2)) or strong requirements (as in (3))?

So when we consider the transfers without the waits, we are talking about (2) or (3), both of which have several issues to them.

I am referring to 3: explicit transfers with errors (and therefore dropped submission) when you use a resource that doesn't belong to a queue. The developer knows the 'sending' queue they specify is signaled and the 'receiving' queue waits.

@RafaelCintron

The developer knows the 'sending' queue they specify is signaled and the 'receiving' queue waits.

To clarify, the developer doesn't know when that wait is going to happen. Imagine that we are the developers, and we just made a submission to queue 'A' with resource transfer into queue 'B'. Then suppose we are making a submission to queue 'B'. Question: will there be a queue wait for 'A'?

In order to answer that question, we'll need to know if that resource is used in the submission we are making. If it's not used, then the wait will happen some time later (some of the following submissions). But figuring out if it's used is very difficult:

GPUTexture <- GPUTextureView <- GPUBindGroup <- GPURenderBundleEncoder <- GPURenderPassEncoder <- GPUCommandBuffer

So in order to trace a usage of a texture, we as developers (or reviewers) would have to chase it through up to 5 levels of indirection (!). In other words, we can't expect anyone to reason about this with high level of confidence. Explicit wait allows the users to state their assumption and avoid guessing.

To clarify, the developer doesn't know when that wait is going to happen. Imagine that we are the developers, and we just made a submission to queue 'A' with resource transfer into queue 'B'. Then suppose we are making a submission to queue 'B'. Question: will there be a queue wait for 'A'?

You correct. I was assuming the reason for the transfer is the developer actually wants to use the resource on queue B.

So in order to trace a usage of a texture, we as developers (or reviewers) would have to chase it through up to 5 levels of indirection (!). In other words, we can't expect anyone to reason about this with high level of confidence. Explicit wait allows the users to state their assumption and avoid guessing.

I must be missing something but I don't understand why developers need to do all of this 'chasing' and 'guessing'. If you transfer the resource to another queue and you use it in the queue, the API will do the right thing to prevent data races. The same is true for memory barriers. Yes, you can chase and guess when the API puts the memory barriers by doing a similar analysis you did for signals and waits. Or you can trust that the API is doing it right.

The only persuasive argument I've seen for requiring waits and signals is when the developer can do a better job than WebGPU at putting them in the right place because (perhaps) they know more about the hardware than WebGPU does. But having developers optimize for one piece of hardware risks the "optimization" doesn't work as well on all hardware and we want good portability. This is why I suggest we consider adding explicit waits/signals as a followon feature when we come to those use cases in the wild.

I think users can implement graph-checking code themselves if they have a need for it. I don't think we absolutely need to offer that in the core API here.

@jdashg the concerns I expressed yesterday wasn't about validation. The problem is that multi-queue is an expert level feature and that expert developers want to use it to control what operations are done in parallel on the multiple queues. For example in the Doom 2016 presentation that use it to run shadow map generation and post-processing in parallel for "bottleneck maximization" (concept name courtesy of @kainino0x).

If we have an implicit synchronization on the receiving end, then it is hard for the developers to know that shadow map, and only shadow maps will run in parallel to the post-processing. If some other part of their render graph runs in parallel of the post-processing, it could kill the "bottleneck maximization" and lose the performance benefits.

For example if the browser synchronizes the first time a resource is used on the receiving queue, the developers could make a change that happens to use a transferred resource in the previous submit, causing synchronization to move earlier and breaking the bottleneck maximization silently. Another example is if the browser synchronizes the next submit on the receiving queue (which is technically valid, nothing in the spec would say what exact execution graph you'd get), then likewise it silently produces an execution graph that breaks their assumption (and the perf gains). That's why I strongly believe we need explicit synchronization on the receiving queue, otherwise the feature will have much reduced benefits because it will silently lose the perf gains super easily.

I think we had good discussions and a general direction for prototyping, but would like to reiterate that I don't think multi-queue should be a required feature or even specced as optional for v1 because:

1. it's clear now that what we have can be extended for multi-queue
2. multi-queue is incredibly complicated to get right and implement when other extensions would give much more bang for buck (FP16, subgroups, even bindless)
3. there are still a ton of things to figure out (queue discovery, concurrent access, ...) and it will delay a release of WebGPU v1 for marginal gains

Closing stale PR; we will revisit this as a proposal when we get to looking back at the multi-queue label.