Introduce selection outline rendering pipeline #17583
Conversation
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17583/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/17583/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/17583/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
Devhost visualization test reporter: |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
|
Devhost visualization test reporter: |
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
Devhost visualization test reporter: |
|
Visualization tests for WebGPU |
|
Hey @noname0310. Thanks for the PR! We are going to review it asap, but because the team is mostly oof, it can take a little time. In the meantime, can you provide a PG so that we can test it?
I don't know how we should handle the fact that depth rendering stores an additional identifier with the depth; it's something we need to think about... I also wonder if it should be in the main package or in the addon/ repository... cc @sebavan (be patient, as he is currently on vacation). Points to keep in mind:
You can have a look at how SSAO2 has been implemented, for an example on how to support both the regular code path and the frame graph path. |
|
WebGL2 visualization test reporter: |
|
https://playground.babylonjs.com/?snapshot=refs/pull/17583/merge#LA850M#2 As you may have already seen on the forum, I'm sharing the PG. Currently, rendering the outline of a mesh with thin instances is supported, but rendering each individual thin instance separately is questionable. This is because rendering only M selected thin instances out of N thin instances belonging to a single mesh requires discarding fragments in the shader or setting the mesh scale to 0 in the vertex transform, which looks quite unnatural. I plan to understand the implementation using a pre-pass renderer and refactor it to the same level as the existing rendering pipeline. Relatedly, I estimate this might take up to a week. :) |
|
Devhost visualization test reporter: |
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
Devhost visualization test reporter: |
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
Raised the occlusionThreshold constant from 0.0000001 to 1.0000001 in both GLSL and WGSL selection outline fragment shaders.
|
Devhost visualization test reporter: |
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
The implementation is complete, but I want to add an optimization that activates post-process and mask rendering only when one or more selections exist, Is there a good example I can reference to implement this? |
|
Devhost visualization test reporter: |
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
WebGL2 visualization test reporter: |
|
Devhost visualization test reporter: |
|
Visualization tests for WebGPU |
The addSelection and setSelection methods now return early if the meshes array is empty. The setSelection method also supports an optional disablePipeline parameter to dispose the pipeline when the selection is empty.
|
Devhost visualization test reporter: |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
|
Ready for review :) The My proposed considerations are as follows:
Here is a test PG: https://playground.babylonjs.com/?snapshot=refs/pull/17583/merge#LA850M#3 |
|
Looks really good in the playground !!! I will have a detailed look later today, thanks for the patience. |
|
Let me move to draft during our discussion time to prevent accidental merge. The main point for me would be to see if we could use the geometry buffer to render the depth information as well as the selection info. Currently we use this selection like shader in both outline and gpu picking and I guess other effects would benefit from this feature. The feature could then be a simple postprocess or maybe an outline layer if we wanted the flood buffer approach for anti-aliased and sizable outlines. What do you think ? If it sounds a bit much I could give it a stab on Friday ? |
For the GPUPicker, I don’t think rendering a selection ID mask as a postprocess or layer is a good fit. The key reason is that this task doesn’t run every frame. it only renders when the user requests a picking operation. That said, I agree that separating the pipeline for rendering the geometry buffer would be a good idea for reusability. In particular, making it a postprocess doesn’t make sense, since rendering the mask is not a pixel-wise operation. In my current implementation, I already separated the selection ID rendering into its own My view is that the right direction is to slightly extend and refactor Also, in the selection outline pipeline, I already considered future support for the jump flood(JFA) approach by adding a parameter to choose the method later on. If making |
This PR rewrites the following Node Material implementation.
https://forum.babylonjs.com/t/selection-outliner-blender-style-outlines/61598
Below is how it works in my application. The sphere in the background is highlighted, and you can see the outline becoming dimmer in areas where occlusion occurs.
The current implementation is incomplete. It is not yet ready to be merged.
The remaining tasks are as follows:
Another consideration is that currently, the Mask render pass is implemented using ShaderMaterial and RenderTargetTexture, rather than being created as a separate renderer class (e.g. DepthRenderer).
While separating it into a distinct Renderer class would improve reusability, I'm concerned that having all binding code (isReady and bindToSubmesh) explicitly exposed might negatively impact maintainability.
Please let me know if there are any design considerations.
I'd especially appreciate it if it could be integrated with Frame Graph.
However, since I haven't used Frame Graph myself, supporting this would likely need to be requested from @Popov72 .