Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

kainino0x
Copy link
Contributor

@kainino0x kainino0x commented Nov 13, 2021

Original PR: #1666
Discussion: #2124

The change proposed here destroys "stale" textures (those which no longer represent the video's current frame) just before rAF callbacks (and before rVFC callbacks, which occur before rAF callbacks, if applicable, because once rVFC fires there's no need for the old texture). It exposes a readonly attribute "expired" which the application can check to know if it should re-import.

Notably, expiry occurs even if the texture was imported just prior to rVFC/rAF. This means it's possible to import on an event (say, "ontimeupdate") and have the texture become invalid before getting a chance to use it. I think this is OK because (1) applications should be importing later, inside their rendering task (rAF or rVFC callback), and (2) the strict timing requirements should mean developers find any such errors immediately.
Alternatively there could be a provision that keeps such textures alive a bit longer, but this could result in applications accidentally introducing an extra (video) frame of latency unnecessarily.


Preview | Diff


Preview | Diff

Copy link
Contributor

@Kangz Kangz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but don't we need something on the GPUExternalTexture to know if it is still alive?

@kainino0x
Copy link
Contributor Author

You're right, I was cranking out spec PRs and totally forgot about that! Will mark as draft until that's added.

@kainino0x kainino0x marked this pull request as draft November 16, 2021 02:44
@toji
Copy link
Member

toji commented Nov 17, 2021

LGTM as well! (Minor added bonus to this approach I haven't seen anybody comment on: It could reduce BindGroup churn.)

As for notifying developers that the texture is still alive, seems like a promise would be appropriate?

function updateVideoTexture() {
  videoTexture = gpuDevice.importExternalTexture({ source: video });
  videoTexture.expired.then(updateVideoTexture);
}

@kainino0x
Copy link
Contributor Author

kainino0x commented Nov 23, 2021

As for notifying developers that the texture is still alive, seems like a promise would be appropriate?

This would be very similar to requestVideoFrameCallback (rVFC) but phrased as a promise instead. However since it's tied to the end of the frame lifetime instead of the beginning, it could lead to developers to re-importing the video when the previous frame expires rather than when they need it, which could result in <1 animation frame of latency when the video advances between the beginning and end of the animation frame, because video frame advancement is totally desynchronized with the animation loop AFAIU. E.g. with both video and animation updating at the same rate, say 60Hz:

externalTexture.expired.then where expiry happens AFTER rAF callbacks:

  • Video element advances to frame 2
  • rAF callback, app renders frame 1 <- outdated render
  • Expire imported frame 1
  • Page renders
  • expired.then callback imports frame 2 <- doesn't run until now because "update the rendering" was running
  • Video element advances to frame 3
  • rAF callback renders frame 2 <- outdated render

rVFC:

  • Video element advances to frame 2
  • rVFC callback imports frame 2
  • rAF callback renders frame 2 <- up-to-date render
  • Expire imported frame 1
  • Page renders
  • Video element advances to frame 3
  • rVFC callback imports frame 3
  • rAF callback renders frame 3 <- up-to-date render

That said, working through this exercise made me realize that a synchronous isExpired that changes after rAF callbacks is even worse:

externalTexture.isExpired where it changes AFTER rAF callbacks:

  • Video element advances to frame 2
  • rAF callback, app checks if frame 1 is still valid; it is, so it renders frame 1 <- outdated render
  • Expire imported frame 1
  • Page renders
  • Video element advances to frame 3
  • rAF callback, app checks if frame 1 is still valid; it isn't, so imports and renders frame 3 <- up-to-date render
  • Expire imported frame 2
  • Page renders
  • Video element advances to frame 4
  • rAF callback, app checks if frame 3 is still valid; it is, so it renders frame 3 <- outdated render

In conclusion, I need to think about this some more.

@kainino0x
Copy link
Contributor Author

kainino0x commented Nov 23, 2021

TIL requestVideoFrameCallback callbacks (as drafted) are called in "update the rendering" immediately before rAF callbacks.

Note: The effective rate at which callbacks are run is the lesser rate between the video’s rate and the browser’s rate. When the video rate is lower than the browser rate, the callbacks' rate is limited by the frequency at which new frames are presented. When the video rate is greater than the browser rate, the callbacks' rate is limited by the frequency of the update the rendering steps. This means, a 25fps video playing in a browser that paints at 60Hz would fire callbacks at 25Hz; a 120fps video in that same 60Hz browser would fire callbacks at 60Hz.

https://wicg.github.io/video-rvfc/#video-rvfc-procedures

@kainino0x
Copy link
Contributor Author

kainino0x commented Nov 23, 2021

Imitating the lifetimes in rVFC would fix it:

externalTexture.isExpired where it changes BEFORE rAF (before or after rVFC?):

  • Video element advances to frame 2
  • Expire imported frame 1
  • rAF callback, app checks if frame 1 is still valid; it isn't, so imports and renders frame 2 <- up-to-date render
  • Page renders
  • Video element advances to frame 3
  • Expire imported frame 2
  • rAF callback, app checks if frame 2 is still valid; it isn't, so imports and renders frame 3 <- up-to-date render
  • Page renders

Though we might still want to guarantee that an imported frame lasts at least until the next end-of-rAF in case it was imported in some other async code. (This doesn't affect this case study.)

@kainino0x kainino0x force-pushed the longer-video-lifetime branch from fce719a to 9a4074b Compare November 24, 2021 03:00
@github-actions
Copy link
Contributor

Previews, as seen when this build job started (9a4074b):
WebGPU | IDL
WGSL
Explainer

@kainino0x kainino0x force-pushed the longer-video-lifetime branch from 9a4074b to 52baf82 Compare November 24, 2021 03:15
@kainino0x kainino0x requested a review from Kangz November 24, 2021 03:17
@kainino0x kainino0x marked this pull request as ready for review November 24, 2021 03:18
@kainino0x
Copy link
Contributor Author

Updated description with latest changes.

@github-actions
Copy link
Contributor

Previews, as seen when this build job started (52baf82):
WebGPU | IDL
WGSL
Explainer

@Kangz
Copy link
Contributor

Kangz commented Nov 24, 2021

Just to make sure I completely understand #2302 (comment), see is what would happen? (it sounds good)

  • Video element advances to frame 2
  • Update the rendering
    • Expire imported frame 1
    • rAF callback, app checks if frame 1 is still valid; it isn't, so imports and renders frame 2 <- up-to-date render
    • Page renders
  • Video element advances to frame 3
  • Update the rendering
    • Expire imported frame 2
    • rAF callback, app checks if frame 2 is still valid; it isn't, so imports and renders frame 3 <- up-to-date render
    • Page renders

@kvark kvark requested a review from kdashg November 24, 2021 15:27
@kainino0x
Copy link
Contributor Author

Yup that looks correct.

Technically the video element can advance at any time (I think). I tried to diagram this out yesterday but the resulting diagram was kind of incomprehensible. The rules as written in the spec make more sense. (Just before rVFC/rAF, textures expire if they're now outdated.)

@kainino0x kainino0x linked an issue Dec 2, 2021 that may be closed by this pull request
@kainino0x kainino0x added this to the V1.0 milestone Dec 4, 2021
@kainino0x
Copy link
Contributor Author

Editors to decide exactly what input we need from the group, or specific individuals.

@kainino0x
Copy link
Contributor Author

kainino0x commented Jan 29, 2022

I'm putting this on the tacit resolution queue for next meeting (although I'll be OOO). Here's a summary for the meeting:


The current spec says imported video textures are closed very quickly (in a microtask after the current task). This leads to developers needing to reimport frequently, which makes them feel they should copy out the data instead of reimporting. Additionally, browsers may need a cache for reimport performance.

This PR extends the lifetime until the frame is not current on the source video anymore. For predictability, expiry always happens at a very specific point - within a frame after the video advances, the lifetime ends immediately before requestVideoFrameCallback callbacks, which are just before requestAnimationFrame callbacks.

It also adds a boolean attribute GPUExternalTexture.expired which tells the app to reimport to get a new frame. Apps use this if WebGPU rendering isn't driven by requestVideoFrameCallback already. (Could be approximated by observing requestVideoFrameCallback, but that's a bit indirect, slightly imprecise, and not in all browsers. This is direct and easy to use.)

@kainino0x kainino0x added tacit resolution queue Editors have agreed and intend to land if no feedback is given and removed for webgpu editors meeting labels Jan 29, 2022
@kainino0x kainino0x self-assigned this Feb 16, 2022
@kainino0x kainino0x force-pushed the longer-video-lifetime branch from 9c78cf1 to a2d998f Compare February 18, 2022 00:09
@kainino0x kainino0x enabled auto-merge (squash) February 18, 2022 00:10
@kainino0x kainino0x removed the tacit resolution queue Editors have agreed and intend to land if no feedback is given label Feb 18, 2022
@kainino0x kainino0x merged commit 3b3e617 into gpuweb:main Feb 18, 2022
@kainino0x kainino0x deleted the longer-video-lifetime branch February 18, 2022 00:15
github-actions bot added a commit that referenced this pull request Feb 18, 2022
)

SHA: 3b3e617
Reason: push, by @kainino0x

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@github-actions
Copy link
Contributor

Previews, as seen when this build job started (a2d998f):
WebGPU | IDL
WGSL
Explainer

github-actions bot added a commit that referenced this pull request Feb 18, 2022
)

SHA: 3b3e617
Reason: push, by @kainino0x

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
github-actions bot added a commit that referenced this pull request Feb 18, 2022
)

SHA: 3b3e617
Reason: push, by @kainino0x

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@kdashg
Copy link
Contributor

kdashg commented Feb 23, 2022

WebGPU meeting minutes 2022-02-16
  • Extend lifetime of GPUExternalTexture imported from video element #2302
    • KN: Importing external textures from video elements
    • Conservatively spec'ed initially. Textures would expire quickly.
    • Devs need to re-import the same frame multiple times. Looks expensive, even though we'd cache inside the browser.
    • Extends the lifetime in a well-defined way - until frame's no longer current.
    • Expires texture at very specific time in frame's lifetime.
    • Think this is a fairly safe change, and better constraint than what we had previously. Should work better for programming against.
    • Does add a boolean attribute to GPUExternalTexture saying when it's expired. Transitions at a well-defined point.
    • KG: this is at some microtask boundary?
    • KN: yes. Right before requestAnimationFrame, or if you have requestVideoFrameCallback - it's right before that.
    • MM: if I import a video frame, and next line does the same thing - guaranteed to get the same frame?
    • KN: I don't think so. Not 100% sure. Videos behave asynchronously, change behind the scenes.
    • MM: what happens in WebGL if you texImage2D twice in a row?
    • KG / KN: not guaranteed to get the same frame. Same guarantees here.
    • KG: if you had 1000 fps video for example you could theoretically get two different frames.
    • BJ: "expires" flag and requestVideoFrame loop aren't necessarily ticking for each frame in this case.
    • KN: if you imported twice you could get 2 different objects that expire at 2 different times.
    • MM: memory use would be proportional to frame rate of video if you imported a bunch of times.
    • KG: potentially.
    • BJ: could you observe in a lightweight way that you did get a different object back? Guaranteed to get same JS object back if frame didn't change?
    • KN: pretty sure you get a new JS object.
    • BJ: imagining blog post where someone tries to determine frame rate of video by repeatedly re-fetching frames.
    • Discussion about frame rates, time stamps of frames.
    • KG: texture uploads from video are naive in WebGL - tried to provide additional information like width, height, timestamp - didn't end up finishing those proposals.
    • KN: and WebCodecs came along while implementing this, and VideoFrame handles this case.
    • KN: concerns with landing this?
    • MM: no - just think some more design work may be needed to ensure the tutorial Brandon hypothesized doesn't get written.
    • MM: one more question - there's requestVideoFrame - will these handles expire at the next 60 Hz rAF or the next frame of the video they came from?
    • KN: the latter. Doesn't expire until after the video frame isn't current any more. At next boundary when video advances to next frame - the GPUExternalTexture will expire. Last and current external textures' lifetimes may overlap slightly.
    • KG: think we're good to land this.
    • MM: this is good, you can write a naive requestVideoFrame loop and it'll work.
    • KN: yes. Can drive rendering both by video frame rate and animation frame rate. Works well in both cases.

Daasin added a commit to FOSS-Archives/WebGPU that referenced this pull request Mar 21, 2022
* Always allow reassociation (gpuweb#2403)

Fixed: gpuweb#2402

* Update question.md

* Fix GPUBuffer.destroy when "mapping pending" (gpuweb#2411)

Fixes gpuweb#2410

* CopyExternalImageToTexture: Support copy from webgpu context canvas/offscreenCanvas (gpuweb#2375)

Address gpuweb#2350.

* Fix typo in requestDevice and clarify error cases (gpuweb#2415)

* Disallow empty buffer/texture usages (gpuweb#2422)

* Specify dispatchIndirect behavior when exceeding limit (gpuweb#2417)

The rest of the behavior of this command isn't specified yet, but this
gets this into the spec so we can close the issue and edit later.

Fixes gpuweb#323

* Require buffer bindings to have non-zero size (gpuweb#2419)

* Require depth clear value in 0.0-1.0 (gpuweb#2421)

* Require depth clear value in 0.0-1.0

* clarify handling of union

* Fix error conventions for optional features (gpuweb#2420)

* Fix error conventions for optional features

* relax claim

* Add increment and decrement statements (gpuweb#2342)

* Add increment and decrement statements

Phrased in terms of += and -= complex assignments

Fixes: gpuweb#2320

* Remove the note about ++ and -- being reserved

* Specify order of evaluation for expressions (gpuweb#2413)

Fixes gpuweb#2261

* Expressions (and function parameters) are evaluated in left-to-right
  order
* Remove todos that are covered elsewhere in the spec
  * variable lifetime
  * statement and intra-statement order

* Remove [[block]] attribute from sample (gpuweb#2439)

* Enable dynamic indexing on matrix and array values (gpuweb#2427)

Fixes: gpuweb#1782

* Behaviour of empty statement is {Next} (gpuweb#2432)

Fixes: gpuweb#2431

* wgsl: Fix TODOs in Execution section (gpuweb#2433)

- in Technical overiew, say that evaluation of module-scope constants
  is the first thing to be executed
- Remove "Before an entry point begins" because that's now fully covered
  by the Technical overview
- Add introductory prose in the "Execution" top level section
- remove parens from "Program order (within an invocation)" section.

* wgsl: Fix "Entry Point" section TODOs (gpuweb#2443)

- Link 'stage' attribute text to definitions later on
- Move definition of "entry point" to the top of the "Entry Points"
  section, away from the "Entry point declaration" section.
- Rework and simplify the first part of "Entry point declaration".
  Link to other parts of the spec, e.g. to user-defined function.

* wgsl: Allow 0X for hex prefix (gpuweb#2446)

Fixes: gpuweb#1453

* Specify compilation message order/locations are impl-defined (gpuweb#2451)

Issue gpuweb#2435

* Disallow pipe for hex literals and allow capital (gpuweb#2449)

* Remove [SameObject] from GPUUncapturedErrorEvent.error (gpuweb#2423)

Implements the same behavior by prose rather than by WebIDL attribute.
The WebIDL attribute isn't currently valid on union types, and we have
to define this in prose anyway since [SameObject] is pure documentation
(has no behavioral impact on its own).

Fixes gpuweb#1225

* Make GPUDevice.lost return the same Promise object (gpuweb#2457)

Fixes gpuweb#2147

* Require alignment limits to be powers of 2 (gpuweb#2456)

Fixes gpuweb#2099

* Define GPUTextureViewDimension values (gpuweb#2455)

including the order of faces in cube maps.

Fixes gpuweb#1946

* Restore the box around algorithm divs (gpuweb#2453)

When the spec template changed, algorithms stopped having an outline
around them, which makes the spec hard to read.

* Add source image orientation to copyExternalImageToTexture (gpuweb#2376)

* Add 'originBottomLeft' attribute in GPUImageCopyTextureTagged

Resolved gpuweb#2324

* Simplify the description and move originBottomLeft to GPUImageCopyExternalImage

* Update spec/index.bs

Address Kai's description.

Co-authored-by: Kai Ninomiya <[email protected]>

* Fix typo

* Apply suggestions from code review

* Update spec/index.bs

Co-authored-by: Kai Ninomiya <[email protected]>

* Clarify that attachments may not alias (gpuweb#2454)

Fixes gpuweb#1358

* Fix examples classes, globals, and previews (gpuweb#2412)

* Rework encoder state and mixins (gpuweb#2452)

* GPUDebugCommandsMixin

* Move state and command list to a GPUCommandsMixin

* Propagate commands in endPass

* fix which [[commands]] is appended

* nits

* "Validate"->"Prepare" the encoder state

* Fully describe validation of render attachments (gpuweb#2458)

* Fully describe validation of render attachments

Fixes gpuweb#2303

* typos

* more typo

* Texture format caps for MSAA and resolve (gpuweb#2463)

* Texture forma caps for MSAA and resolve

* Fix missing columns, add notes

* Add multisample flags even where rendering isn't supported

* [editorial] wgsl: left shifts are logical (gpuweb#2472)

* Remove 'read','read_write','write' as keywords, image formats as keywords (gpuweb#2474)

* Texture format names are not keywords

Issue: gpuweb#2428

* read, write, read_write are not keywords

Fixes: gpuweb#2428

* Only define image format names usable for storage textures (gpuweb#2475)

* Only define image format names usable for storage textures

Fixes: gpuweb#2473

* Sort texel format names by channel width first

Make it consistent with the other tables in the WGSL spec,
and with the Plain Color Formats table in the WebGPU spec.

* [editorial] Rename "built-in variable" -> "built-in value" (gpuweb#2476)

* Rename "built-in variable" -> "built-in value"

Fixes: gpuweb#2445

* Rewrite builtin-in inputs and outputs section

It needed an overhaul because with pipeline I/O via entry point
parameters and return types.  Previously it was phrased in terms
of *variables*, and some things just didn't make sense.

Added rules expressing the need to match builtin stage and direction
with entry point stage and parameter vs. return type.
This also prevents mixing builtins from different stages or conflicting
directions within a structure.

* Move Limits section to under "WGSL Program" (gpuweb#2480)

I think it makes more sense there.

* Fix declaration-and-scope section for out-of-order decls (gpuweb#2479)

* Fix declaration-and-scope section for out-of-order decls

Also reorganize to bring "resolves to" closer to the definition of "in scope".

Fixes: gpuweb#2477

* Apply review feedback

* Behaviors: Ban obviously infinite loops (gpuweb#2430)

Fixes: gpuweb#2414

* Clarify fract (gpuweb#2485)

* Officially add Brandon as a spec editor (gpuweb#2418)

Brandon has de facto equal standing as an editor and I think it's time
to recognize it.

* Require 1D texture mipLevelCount to be 1. (gpuweb#2491)

Fixes gpuweb#2490

* Add simple examples for create, init, and error handling functions

* Address feedback from Kai

* Tweak to device loss comments

* wsgl: Add bit-finding functions. (gpuweb#2467)

* wsgl: Add bit-finding functions.

- countLeadingZeros, countTrailingZeros
   - Same as MSL clz, ctz
- firstBitHigh, firstBitLow
   - Same as HLSL firstbithi, firstbitlow
   - Same as GLSL findMSB, findLSB
   - Same as SPIR-V's GLSL.std.450 FindSMsb FindUMsb, FindILsb

Fixes: gpuweb#2130

* Apply review feedback

- Better description for countLeadingZeros countTrailingZeros
- For i32, we can say -1 instead of |T|(-1)

* Apply review feedback: drop "positions"

* wgsl: Add extractBits, insertBits (gpuweb#2466)

* wgsl: Add extractBits, insertBits

Fixed: gpuweb#2129 gpuweb#288

* Formatting: break lines between parameters

* insertBits operates on both signed and unsigned integral types

* Add mixed vector-scalar float % operator (gpuweb#2495)

Fixes: gpuweb#2450

* wgsl: Remove notes about non-ref dynamic indexing (gpuweb#2483)

We re-enabled dynamically indexing into non-ref arrays and matrices in
gpuweb#2427, as discussed in gpuweb#1782.

* Disallow aliasing writable resources (gpuweb#2441)

* Describe resource aliasing rules

Fixes gpuweb#1842

* Update spec/index.bs

Co-authored-by: Kai Ninomiya <[email protected]>

* Update spec/index.bs

Co-authored-by: Kai Ninomiya <[email protected]>

* Editorial changes

* Editorial: split out aliasing analysis

* Consider only used bind groups and consider visibility flags

* Consider aliasing between read-only and writable bindings

* Tentatively add note about implementations

* editorial nits

* Fix algorithm

* Remove loop over shader stages

* Rephrase as "TODO figure out what happens"

* clarify

* Add back loop over shader stages

* map -> list

Co-authored-by: Myles C. Maxfield <[email protected]>
Co-authored-by: Myles C. Maxfield <[email protected]>

* Fix map/list confusion from gpuweb#2441 (gpuweb#2504)

I forgot to save the file before committing my last fix to gpuweb#2441.

* Fix a typo in packing built-in functions list (gpuweb#2513)

* Remove tiny duplicates (gpuweb#2514)

This removes a few tiny duplicates found in the spec.

* integer division corresponds to OpSRem (gpuweb#2518)

* wgsl: OpMod -> OpRem for integers

* OpURem -> OpUMod again

Co-authored-by: munrocket <[email protected]>

* Remove stride attribute (gpuweb#2503)

* Remove stride attribute

Fixes: gpuweb#2493

Rework the examples for satisfying uniform buffer layout, using align
and stride.

* Remove attribute list from array declaration grammar rule

Fixes: gpuweb#1534 since this is the last attribute that may be applied to a type declaration.

* Switch to `@` for Attributes (gpuweb#2517)

* Switch to `@` for Attributes

* Convert new examples

* Struct decl does not have to end in a semicolon (gpuweb#2499)

Fixes: gpuweb#2492

* wgsl: float to integer conversion saturates (gpuweb#2434)

* wgsl: float to integer conversion saturates

Fixes a TODO

* Saturation follows rounding toward zero.

Simplify the wording of the rule.

Explain what goes on at the extreme value (as discussed in the issue
and agreed at the group), how you don't actually get the max value
in the target type because of imprecision.

* Store type for buffer does not have to be structure (gpuweb#2401)

* Store type for buffer does not have to be structure

* Modify an example showing a runtime-sized array as the store type
  for a storage buffer.

Fixes: gpuweb#2188

* Update the API-side rules about minBindingSize

The store type of the corresponding variable is not always
going to be a structure type. Qualify the rule accordingly.

* Rework minimum binding size in both WebGPU and WGSL spec

Define 'minimum binding size' in WGSL spec, and link to it from WebGPU.
Repeat the rule in both places (to be helpful).

The minimum binding size for a var with store type |T| is
max(AlignOf(T),SizeOf(T)), and explain why the AlignOf part is needed:
it's because sometimes we have to wrap |T| in a struct.

This also replaces the old rule in WGSL which confusingly dependend
on the storage class.  The storage class aspect is already embedded
in the alignment and size constraints for the variable.

* Simplify minimum binding size to Sizeof(store-type)

Underlying APIs don't need the extra padding at the end of any
structure which might wrap the store type for the buffer variable.

* Update API-side to SizeOf(store-type)

* Apply review feedback

- Link to SizeOf in the WGSL spec
- More carefully describe the motivation for the min-binding-size
  constraint.

* Simplify, and avoid using the word "mapping"

"Mapping" is how the buffer's storage is paged into host-accessible
address space. That's a different concept entirely, and would only
confuse things.

* Remove duplicated words (gpuweb#2529)

* Remove duplicated words `be`

Remove two duplicated `be` from the WebGPU spec.

* remove another duplicated `the`

* editorial: streamline array and structure layout descriptions (gpuweb#2521)

* Simplify array layout section

- move definition of element stride to start of memory layout section
- remove redundant explanation of array size and alignment
- remaining material in that example is just examples
- Add more detail to examples, including computing N_runtime for
  runtime-sized array

* Streamline structure layout section

- Make 'alignment' and 'size' defined terms
- Don't repeat the rule for overall struct alignment and size.
- Rename "Structure Layout Rules" to "Structure Member Layout" because
  that's all that remains.
  - Streamline the text in this section.

Fixes: gpuweb#2497

* Apply review feedback:

- state constraints at definition of the align and size attributes
- rename 'size' definition to 'byte-size'
- use the term "memory location" when defining alignment.
- rename the incorrectly-named "lastOffset" to "justPastLastMember"
- in the description of internal layout, state the general rule that the
  original buffer byte offset k must divide the alignment of the type.

* Change notation: say i'th member instead of M<sub>i</sub>

* Remove stray sentence fragment

* Change GPUObjectBase.label from nullable to union-with-undefined (gpuweb#2496)

* Separate loadOp and clear values

* Add note explaining how dispatch args and workgroup sizes interact (gpuweb#2519)

* Add a note explaining how the dispatch arguments and workgroup sizes interact

* Address feedback

* Address feedback from Kai

* Refine the supported swapchain formats (gpuweb#2522)

This removes the "-srgb" formats, and adds "rgba16float".

Fixes: gpuweb#1231

* wgsl: Fix example's builtin name (gpuweb#2530)

* wgsl: detailed semantics of integer division and remainder (gpuweb#1830)

* wgsl: detailed semantics of integer division and remander

Adds definition for `truncate`

For divide by zero and signed integer division overlow, say they
produce a "dynamic error".

Fixes: gpuweb#1774

* Assume polyfill for the overflow case

This pins down the result for both division and remainder.

* Specify definite results for int division, % by zero

These are no longer "dynamic errors".

For integer division:   e1 / 0 = e1

For integers,           e1 % 0 = 0

The % case is somewhat arbitrary, but it makes this true:
    e1 = (e1 / 0) + (e1 % 0)

Another way of formulating the signed integer cases is to forcibly
use a divisor of 1 in the edge cases:
     where MINIT = most negative value in |T|
     where Divisor = select(e2, 1, (e2==0) | ((e1 == MININT) & (e2 == -1)))
then
     "e1 / e2" = truncate(e1 / Divisor)
     "e1 % e2" = e1 - truncate(e1/Divisor) * Divisor

The unsigned integer case is similar but doesn't have (MININT,-1) case.

* Add override declarations (gpuweb#2404)

* Add override declarations

* Refactor `var` and `let` section
  * have a general value declaration subsection
  * subsections on values for let and override
  * move override requirements from module constants to override
    declarations
* introduce override keyword
* remove override attribute and add id attribute
  * literal parameter is now required
* Update many references in the spec to be clearer about a let
  declaration vs an override declaration
* update examples
* Make handling of `offset` parameter for texture builtins consistent
  * always either const_expression or module scope let

* Changes for review

* combine grammar rules
* refactor validation rules for overrides
* fix typos

* add todo for creation-time constant

* fix example

* combine grammar rules

* Rename storage class into address space (gpuweb#2524)

* Rename storage class into address space

* Davids review findings, plus renaming of |SC|

* Add an explainer for Adapter Identifiers to facilitate further design discussion.

* Explain smoothStep better (gpuweb#2534)

- Use more suggestive formal parameter names
- Give the formula at the function definition, not just at the
  error bounds explanation.

* Fix clamp arity in error bounds section (gpuweb#2533)

Also at the definitions, use more suggestive formal parameter names (e,low,high)
instead of the less readable (e1,e2,e3)

* Use consistent capitalisation in section titles (gpuweb#2544)

* remove some unnecessary `dfn` tags

* Remove unnecessary todos (gpuweb#2543)

* Defer API linkage issues to the API spec
* remove issues and todos that are covered in the API
* remove todo about array syntax

Co-authored-by: David Neto <[email protected]>

* Add GPUTextureDescriptor viewFormats list (gpuweb#2540)

* Add GPUTextureDescriptor viewFormats list

Initially allows only srgb formats; further rules for format
compatibility can follow.

Issue: gpuweb#168
CC: gpuweb#2322

* note on canvas config

* Enforce presence of an initializer for module-scope let (gpuweb#2538)

* Enforce presence of an initializer for module-scope let

* Since pipeline-overridable constants were split from let declarations
  the grammar for let declarations can enforce the presence of an
  initializer
* Remove global_const_intiailizer since it was only used in for a single
  grammar production (module-scope let) and only had a single grammar
  itself
* Update extract_grammar to initialize type declarations with zero-value
  expressions

* fix typos

* Make depth/stencil LoadOp and StoreOp optional again

This change was landed as part of gpuweb#2387 but was then accidentally
reverted when gpuweb#2386 landed out of order.

* Add the uniformity analysis to the WGSL spec (gpuweb#1571)

* Add the uniformity analysis to the WGSL spec

Make the information computed more explicit per Corentin's suggestion

Add uniformity of builtins, limit the singleton rule to {Next}, do some minor cleanup

Make the typography more uniform and hopefully less confusing

Add rule for switch, and simplify rule for if

Clarify the role of CF_start, and remove two instances of the word 'simply'

Remove TODO and allow accesses to read-only global variables to be uniform

Mark functions that use implicit derivatives as ReturnValueCannotBeUniform

* s/#builtin-variables/#builtin-values/ after rebasing

* Add (trivial) rules for let/var, as suggested by @alan-baker and @dneto

* Add rules for non-shortcircuiting operators, as suggested by @alan-baker and @dneto

* Use the rowspan attribute to simplify the tables in the uniformity section

* Fix syntax of statement sequencing/blocks in the uniformity rules, following an earlier fix to the behavior analysis.

* s/adressing/addressing/, as suggested by @alan-baker in an earlier review.

* Clarify 'local variable'

* Deal with non-reconvergence at the end of functions

* s/global/module-scope/, s/built-in variable/built-in value/, and mention let-declarations

* Address the last issues found by @dneto0

* CannotBeUniform -> MayBeNonUniform

* Apply Dzmitry's suggestions

* s/MustBeUniform/RequiredToBeUniform/g

Co-authored-by: Robin Morisset <[email protected]>

* Vectors consist of components (gpuweb#2552)

* Vectors consist of components

* Update index.bs

* Make depth/stencil LoadOp and StoreOp optional again (pt.2)

This change was landed as part of gpuweb#2387 but was then accidentally
reverted when gpuweb#2386 landed out of order.

* WGSL: Replace [SHORTNAME] with WGSL (gpuweb#2564)

Fixes gpuweb#1589

* Fix step() logic (gpuweb#2566)

* Relax vertex stride requirements (gpuweb#2554)

* s/endPass/end/ for pass encoders (gpuweb#2560)

Fixes gpuweb#2555

* Fix canvas resizing example

* Rework "validating texture copy range" for 1D/3D textures. (gpuweb#2548)

That algorithm special cased 1D and 2D textures, making empty copies
valid for 2D and not for 1D. 3D textures where just not discussed.

Fix this by just checking that the copy fits in the subresource size,
and also turn "validating texture copy range" into an algorithm with
arguments.

Co-authored-by: Dzmitry Malyshau <[email protected]>

* Add optional trailing comma for the attribute syntax. (gpuweb#2563)

All places that use a variable number of comma-separated things now
support trailing commas. However things with a fixed number of
comma-separated arguments don't. They are:

 - array_type_decl
 - texture_sampler_types
 - type_decl
 - variable_qualifier

Fixes gpuweb#1243

* wgsl: reserve `demote` and `demote_to_helper` (gpuweb#2579)

* if,switch param does not require parentheses (gpuweb#2585)

Fixes: gpuweb#2575

* Add while loop (gpuweb#2590)

Update behaviour analysis and uniformity analysis.

Fixes: gpuweb#2578

* Allow sparse color attachments and ignored FS outputs (gpuweb#2562)

* Allow sparse color attachments and ignored FS outputs

Fixes gpuweb#1250
Fixes gpuweb#2060

* Update pipeline matching rules

Co-authored-by: Dzmitry Malyshau <[email protected]>

* Render -- as is (gpuweb#2576)

* Render -- as is

* Use backticks

* Use backticks for plus plus too

* Better separate Security and Privacy sections (gpuweb#2592)

* Better separate Security and Privacy sections

They were largely already separate but the header levels were a
bit confusing so this CL normalizes them and renames the sections
to "Security Considerations" and "Privacy Considerations" as
requested by the W3C security review guidelines.

Also expands the privacy section with a brief header, another
mention of driver bugs as a potentially identifying factor, and
a note indicating that discussions about adapter identifiers are
ongoing.

* Simplify adapter info privacy considerations note.

* Remove SPIR-V mappings (gpuweb#2594)

* Remove most references to SPIR-V opcodes and types in the
  specification
* References remain transitively in the Memory Model section as it is
  necessary for the specification
* Removed goal section as they only described SPIR-V

* Complete the Errors & Debugging section

* Addressed feedback

* Update spec/index.bs

Co-authored-by: Kai Ninomiya <[email protected]>

* Update spec/index.bs

Co-authored-by: Kai Ninomiya <[email protected]>

* Make firing the unhandlederror event optional in the algorithm

* Refactored algorithms for more sensible names.

* Fix typo in "validating texture copy range" argument (gpuweb#2596)

* Fix a typo in a note "applicaitions". (gpuweb#2602)

* Disallow renderable 3D textures (gpuweb#2603)

* `createSampler` creates a `GPUSampler` (gpuweb#2604)

Correct the link that errantly pointed to `GPUBindGroupLayout`.

* Extend lifetime of GPUExternalTexture imported from video element (gpuweb#2302)

* Extend lifetime of GPUExternalTexture imported from video element

Original PR: gpuweb#1666
Discussion: gpuweb#2124

* adjust lifetime, add flag

* editorial

* wgsl: reserve 'std', 'wgsl' (gpuweb#2606)

Fixes: gpuweb#2591

* wgsl: Fix typos (gpuweb#2610)

`signficant` -> `significant`
`consectuive` -> `consecutive`

* wgsl: Rename `firstBitHigh` and `firstBitLow`

The current names are confusing, as `High` or `Low` may refer to scanning from the MSB or LSB, or that it is scanning for the bits `1` or `0`.

By renaming to `firstLeadingBit` and `firstTrailingBit` the ambiguity is reduced, and we have a consistent terminology with `countLeadingZeros` / `countTrailingZeros`.

* Update override examples (gpuweb#2614)

Fixes gpuweb#2613

* Update WGSL syntax for overridable constants

* Fix a typo in FP32 internal layout (gpuweb#2615)

* Fix a typo in FP32 internal layout

In the internal layout of float32, Bits 0 through 6 of byte k+2 contain
bits 16 through 22 of the fraction, which has a total of 23 bits.

* remove duplicated "bit"

* WGSL style guide: in progress extensions developed outside main spec (gpuweb#2616)

Fixes: gpuweb#2608

* Clarify when a built-in function name can be redefined (gpuweb#2621)

* wgsl: Cleanup internal layout of matrix type (gpuweb#2624)

* Cleanup internal layout of matrix type

Use alignOf() to cleanup the description of internal layout of matrix type.

* Use "i x AlignOf()" instead of "AlignOf() x i"

* [editorial] Fix uniformity table widths (gpuweb#2623)

* Reduce table width by allowing more line breaks
* Make op consistently italicized

* Add break-if as optional at end of continuing (gpuweb#2618)

* Add break-if as optional at end of continuing

A break-if can only appear as the last statement in a continuing
clause.

Simplifies the rule about where a bare 'break' can occur: It
must not be placed such that it would exit a continuing clause.

Fixes: gpuweb#1867

Also refactor the grammar to make:
  continuing_compound_statement
  case_compound_statement
These are called out as special forms of compound statement, so
that the scope rule of declarations within a compound statement
also clearly apply to them.

* Add statement behaviour of break-if

The expresison always includes {Next}.
When the expression is true, the {Break} behaviour is invoked.
Otherwise, the {Next} behaviour is invoked.

So it's   B - {Next} + {Next, Break}
or   B + {Break}

* Add uniformity analysis for break-if

* Apply review feedback

- Tighten the wording about where control is transferred for break and
  break-if
- Allow "break" to be used in a while loop.

* Avoid double-avoid

* wgsl: Reserve words from common programming languages (gpuweb#2617)

* wgsl: Reserve words from common programming languages

Reserves words from C++, Rust, ECMAScript, and Smalltalk

Add a script to automatically generate the contents of the _reserved
grammar rule.

Update extract-grammar.py to strip HTML comments before processing.

* List WGSL as a reason for a keyword reservation

* Reserve 'null'

* Reserve keywrods from GLSL 4,6 and HLSL

* Use ECMAScript 2022 instead of ECMAScript 5.1

* Reserve HLSL keywords

* Add acosh, asinh, atanh builtin functions (gpuweb#2581)

* Add acosh, asinh, atanh builtin functions

Use the polyfills from Vulkan.

Fixes: gpuweb#1622

* Result is 0 in regions that make no mathematical sense: acosh, atanh

* wgsl: Cleanup the internal memory layout of vector using SizeOf (gpuweb#2626)

* Cleanup the internal memory layout of vector using SizeOf

Descript the internal memory layout of vector types vecN<T> with
SizeOf(T) rather than literal number.

* Fix typo

* Fix typos of accuracy of exp and exp2 (gpuweb#2634)

Fix the accuracy requirement of exp and exp2 to 3 + 2 * abs(x) ULP.

* Reland: Only allow depth/stencil load/store ops when they have an effect

* Validate query index overwrite in timestampWrites of render pass (gpuweb#2627)

Vulkan requires the query set must be reset between uses and the reset
command must be called outside render pass, which makes it impossable to
overwrite a query index in same query set in a render pass, but we can
do that in different query set or different render pass.

* Add definitions for uniformity terms (gpuweb#2638)

* add definitions (and link back to them) for:
  * uniform control flow
  * uniform value
  * uniform variable
* define the scope of uniform control flow for different shader stages

* Allow statically unreachable code (gpuweb#2622)

* Allow statically unreachable code

Fix gpuweb#2378

* modify behavior analysis to allow statically unreachable code
  * unreachable code does not contribute to behaviors
* modify uniformity analysis to not analyze unreachable code
  * unreachable statements are not added to the uniformity graph

* Improve examples

* Clarify when sequential statement behaviour leads to a different
  behaviour from that of the individual statement
* improve example comment formatting to reduce possible horizontal
  scrolling

* Name in enable directive can be a keyword or reserved word (gpuweb#2650)

Fixes: gpuweb#2649

Also simplify description of where an enable directive can appear.
They must appear before any declaration.

* GPUDevice.createBuffer() requires a valid GPUBufferDescriptor (gpuweb#2643)

* Typo in definition of finish()

* Allow unmasked adapter info fields to be requested individually.

* Update design/AdapterIdentifiers.md

Co-authored-by: Kai Ninomiya <[email protected]>

* Explicitly note that declined consent rejects the promise

* Uses commas to separate struct members intead of semicolons (gpuweb#2656)

Fixes gpuweb#2587

* Change the separator for struct members from semicolons to commas
  * Comma is optional after the last member
* Changes the grammar to require one or more struct members
  * Already required by the prose of the spec

* [editorial] Compress expression tables (gpuweb#2658)

* [editorial] Compress expression tables

* Combine arithmetic and comparison expression table entries for
  integral and floating-point entries
  * since SPIR-V mappings were removed there is no strong need to have
    separate entries

* improved wording

* make online should die on everything (gpuweb#2644)

* Commiting GPUCommandBuffers to the wrong GPUDevice is an error (gpuweb#2666)

Eventually we'll want to add more logic to make sure that command buffers are only valid on
the queue they're created from. Right now, though, every device just has exactly one queue,
so matching devices is the same as matching queues.

* Mipmap filtering might be extended separately from min/mag filtering in the future

* Add a way of setting the initial label for the default queue

* add rAF/rVFC examples

* [Process] Add RequirementsForAdditionalFunctionality.md

* Addressing Kai and Dzmitry's comments

* GPUSamplerDescriptor.maxAnisotropy gets clamped to a platform-specific maximum (gpuweb#2670)

Co-authored
@kainino0x kainino0x added the needs-cts-issue This change requires tests (or would need tests if accepted), but may not have a CTS issue filed yet label Jun 29, 2022
juj added a commit to juj/wasm_webgpu that referenced this pull request Aug 17, 2022
@lokokung
Copy link
Contributor

lokokung commented Dec 6, 2022

Removing 'needs-cts-issue' for now because this is obsolete and will be updated in a future pull request to fix #3324.

@lokokung lokokung removed the needs-cts-issue This change requires tests (or would need tests if accepted), but may not have a CTS issue filed yet label Dec 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Determine GPUExternalTexture lifetime
5 participants