Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

almarklein
Copy link
Collaborator

@almarklein almarklein commented Jul 8, 2025

Context

Previously, Pygfx automatically distinguished between opaque and transparent fragments on a per-fragment level. With the new blending in Pygfx (current main), this distinction will have to be made per object. This means that somewhere a decision will have to be made whether an object is opaque or transparent.

The PyGfx material.alpha_mode is the easy way to control this behavior. The two values that make the most sense for Fastplotlib are:

  • 'auto': applies blending, and writes depth when material.opacity is 1.
  • 'blend': applies blending but does not write depth: relies on the correct render order.

This PR uses 'blend' for all WorldObjects. We can use material.render_queue to control the order of axes, grids, legends etc, with respect to other objects. This means that they do not write to the depth buffer.

Using 'auto' would also work, but self-intersecting objects like noisy lines or scattered points would be prone to show artifacts in semi-transparent regions (e.g. the aa edge).

If we add support for 3D objects (e.g. meshes) these should probably use `alpha_mode='auto' or 'opaque'.

Places where users can introduce transparency

(Let me know if I missed one!)

  • the graphic's alpha argument.
  • alpha in colormaps.
  • alpha in a uniform color
  • alpha in per-point colors.
  • alpha in image pixels.

Ideas (at the start of this PR)

Some ideas, some compatible, some mutually exclusive:

  • The alpha value passed to the graphic could be used to set world_object.material.opacity.
  • Also allow RGB colors, and maybe even promote that a bit over RGBA.
  • Automatically make a graphic transparent when alpha < 1. Easy to do either in fpl or pygfx. Covers quite some use-cases.
    • Somewhat intuitive: people who wonder why their transparent image is opaque may set alpha and see that it works then. Then they can set it to 0.999 to fix it.
  • For other cases (e.g. images or per-point colors), require setting alpha_mode.
    • A bit harder on users, but easy enough to document?
  • Or check the alpha values in images and colorlists, and automatically set the alpha_mode when values < 1 are present.
    • Makes it easier for users, but adds complexity and possible source of bugs.
  • Default to "dither", which always works. Then users can set alpha_mode="blend" for smoother results of transparent objects.

cc @Korijn because this is is a topic that touches both fpl and pygfx

@kushalkolar
Copy link
Member

kushalkolar commented Jul 8, 2025

Thanks! Will check this in detail later today, but I wanted to mention this: #754

We want to make alpha a separate graphic feature and thus an entirely independent Graphic property (in general we're trying to make the constructor as symmetric as possible with the settable properties, just like pygfx) . This would probably be a good time to do that?

@almarklein
Copy link
Collaborator Author

We want to make alpha a separate graphic feature and thus an entirely independent Graphic property

Good to know this was already on the agenda! Yeah I think it makes sense both from an API perspective, but also technically; changing a color is fine, but changing an alpha value might affect how you want to blend things.

@Korijn
Copy link

Korijn commented Jul 11, 2025

Then they can set it to 0.999 to fix it.

This seems a bit awkward. I foresee users would complain about having to do this.

  • Or check the alpha values in images and colorlists, and automatically set the alpha_mode when values < 1 are present.

From your list of options, this seems most straight forward.

  • Default to "dither", which always works. Then users can set alpha_mode="blend" for smoother results of transparent objects.

After thinking about this for a while, I guess what I would expect from a 2D plotting library as a user:

  • All graphics have alpha_mode="blend"
  • Graphics are drawn in the order that I add them to the figure, transparent or not, so the depth buffer is kind of ignored I guess

I'm not sure if fastplotlib also does 3D plotting, but in that case I would expect it to behave as pygfx does naturally.

By the way, I appreciate you calling for my thoughts, so I'm sharing them here, but I really have not been keeping up with fastplotlib's development for a while, so take it with a grain of salt.

@kushalkolar
Copy link
Member

  • Or check the alpha values in images and colorlists, and automatically set the alpha_mode when values < 1 are present.

From your list of options, this seems most straight forward.

I'm leaning towards this as well.

After thinking about this for a while, I guess what I would expect from a 2D plotting library as a user:

  • Graphics are drawn in the order that I add them to the figure, transparent or not, so the depth buffer is kind of ignored I guess

When using an orthographic projection graphics are stacked in z.

I'm not sure if fastplotlib also does 3D plotting, but in that case I would expect it to behave as pygfx does naturally.

Everything is 3D just like pygfx, perspective camera for everything. We just don't have a high level API for meshes yet, I've rarely used meshes myself so I hope someone else would be able to contributre a high level API for that. Or maybe meshes are so specific that the pygfx level of abstraction is the best for it, I don't know yet.

@almarklein
Copy link
Collaborator Author

When using an orthographic projection graphics are stacked in z.

Do you mean that the objects are offset in their ob.local.z?

Is it possible to clearly identify whether the current graphic is in a 2D or 3D scene? If that is the case we could indeed default to blend.

Or maybe meshes are so specific

Haha!. Some Pygfx users consider meshes the "normal object" and think all the non-meshes are special 😉

@Korijn
Copy link

Korijn commented Jul 15, 2025

When using an orthographic projection graphics are stacked in z.

And what determines the stacking order? Insertion order?

@kushalkolar
Copy link
Member

kushalkolar commented Jul 15, 2025

When using an orthographic projection graphics are stacked in z.

Do you mean that the objects are offset in their ob.local.z?

Yup!

When using an orthographic projection graphics are stacked in z.

And what determines the stacking order? Insertion order?

Yup. Though the z can be set when added, or later at any time.

Is it possible to clearly identify whether the current graphic is in a 2D or 3D scene? If that is the case we could indeed default to blend.

If camera.fov == 0

You can switch between them and it's something I do all the time when looking at low dimensional projections with lines and scatters.

@almarklein almarklein mentioned this pull request Sep 2, 2025
4 tasks
@almarklein
Copy link
Collaborator Author

Tests pass again! (or am I missing something?)

@almarklein
Copy link
Collaborator Author

oh lol, all tests are skipped 😆

@almarklein almarklein marked this pull request as ready for review September 2, 2025 10:11
@almarklein almarklein requested a review from clewis7 as a code owner September 2, 2025 10:11
Copy link

github-actions bot commented Sep 2, 2025

📚 Docs preview built and uploaded! https://www.fastplotlib.org/ver/alpha

@Korijn
Copy link

Korijn commented Sep 2, 2025

How are you feeling about this refactor @almarklein, now that you can experience the revamped transparency system as a user?

@almarklein
Copy link
Collaborator Author

How are you feeling about this refactor @almarklein, now that you can experience the revamped transparency system as a user?

I'm feeling pretty good about it in general.

I felt rather disappointed that we cannot abstract the whole transparency thing away from the user. But after having done quite a bit of research on the topic I'm confident that even with the more advanced methods (which would consume more memory and/or be slower) it'd still not be fool proof. That's what it is.

So the only way to go is to shove the problem on the user 😄 , but as gentle as possible, and providing multiple different options to deal with it. I think we've now now done that pretty well, providing pretty powerful options, but also easy-to-use shorthands with alpha_mode.

As for this PR in Fastplotlib, I think it helps a lot if we can determine/assume the scene to be 2D, so we can just use 'blend'. If no object writes depth, Pygfx will not create the depth buffer, so it'd be a bit lighter too.

@kushalkolar
Copy link
Member

if we can determine/assume the scene to be 2D, so we can just use 'blend'. If no object writes depth, Pygfx will not create the depth buffer, so it'd be a bit lighter too

3D is used often, easiest way to check is if camera.fov > 0

This will also need some refactoring of the graphic features, will add more thoughts soon

@Korijn
Copy link

Korijn commented Sep 4, 2025

You may want to consider introducing an Enum or a dict with a list of custom render queues, just so they're easier to organize and manage.

@almarklein
Copy link
Collaborator Author

Ok, the approach I implemented now:

  • Background and overlays use alpha_mode=='blend'; we don't really need the depth buffer there.
  • Same for annotations like the selector tools, but these are rendered in the same scene so I used render_queue=3500 so they always render later, and they don't use the depth buffer (no write and no test).
  • For these underlays and overlays I used 'ob.render_order` to define the order of rendering within that group, instead of using z.
  • Axes are solid, except for the text labels, for which we want aa.
  • Lines and Points have alpha_mode='auto' by default, but user can change with an arg. The aa is turned on if the alpha mode is 'blend'.

The screenshot tests fail. obviously. What's the best way to update these? Same as in Pygfx?

This will also need some refactoring of the graphic features, will add more thoughts soon.

What kind of changes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants