Change pcolormesh snapping (fixes alpha colorbar/grid issues) [AGG] #16090

greglucas · 2020-01-04T14:30:27Z

PR Summary

This fixes longstanding issues with dark or white lines showing up in colorbars and grids when alpha is present.
(#1188 and many other references to similar issues, it is all over stackoverflow as well)

It also appears to fix some grid positioning issues as well: #7341

Consequences

There are more than 40 failing images due to this change (I think all due to pixel offsets in colorbar/pcolormesh image comparisons from what I can tell). These images would all have to be updated. I also suspect that lots of downstream libraries depending on matplotlib would be impacted by this in their image comparisons as well.

I am not an expert on the Agg/rendering machinery, so I welcome feedback/discussion on this.

Timing appears inconsequential (actually faster) during my basic tests of increasing the mesh size, but I haven't done anything rigorous.

Proposed Changes

There are two issues here.

Push the edgecolor logic up into the Python callers (for some reason if antialiasing was set and edgecolors was empty the C++ would override edgecolors. That magic validations should happen upstream if this is desired.
Call the mesh generation code with check_snap set to true.

Comparison results

Before:

After:

Demo code:

import numpy as np
import matplotlib
import matplotlib.pyplot as plt
print(matplotlib.__version__, matplotlib.get_backend())

import time

N = 10

x = np.arange(N)
y = np.arange(N)
X, Y = np.meshgrid(x, y)
z = (X[:-1, :-1] + 1)*(Y[:-1, :-1] + 1)

# Set up the colormaps
cmap = plt.get_cmap('viridis')
cmap_data = cmap(np.arange(cmap.N))
# Set a linear ramp in alpha
cmap_data[:, -1] = np.linspace(0.2, 1, cmap.N)
# Create new colormap
cmap_alpha = matplotlib.colors.ListedColormap(cmap_data)
norm = matplotlib.colors.Normalize(vmin=np.min(z), vmax=np.max(z))

ec = 'none'
cmap1 = cmap

# Row 1: alpha_cmap: aa False
# Row 2: normal_cmap: aa False
# Row 3: alpha_cmap: aa True
# Row 4: normal_cmap: aa True
# col 1: rectangle ec: none
# col 2: rectangle ec: 'face'
# col 3: diagonal ec: none
# col 4: diagonal ec: 'face'

t0 = time.time()
fig, axs = plt.subplots(4, 4, figsize=(8, 8), sharex=True, sharey=True, constrained_layout=True)


def plot_func(axs, i, j):
    if i in (0, 2):
        cm = cmap_alpha
    else:
        cm = cmap
    if i in (0, 1):
        aa = False
    else:
        aa = True

    if j in (0, 2):
        ec = 'none'
    else:
        ec = 'face'

    if j in (0, 1):
        mesh = axs[i, j].pcolormesh(X, Y, z, norm=norm, cmap=cm,
                             antialiased=aa, edgecolors=ec)
    else:
        mesh = axs[i, j].pcolormesh(X+0.2*Y, Y+0.2*X, z, norm=norm, cmap=cm,
                             antialiased=aa, edgecolors=ec)
    if i == 0 and j == 0:
        fig.colorbar(mesh, ax=axs[0:2, :], orientation='vertical')
    if i == 1 and j == 0:
        fig.colorbar(mesh, ax=axs[2:4, :], orientation='vertical')
    axs[i, j].set_xticks([])
    axs[i, j].set_yticks([])

# Bottom row labels
axs[3, 0].set_xlabel('ec: none')
axs[3, 1].set_xlabel('ec: face')
axs[3, 2].set_xlabel('ec: none')
axs[3, 3].set_xlabel('ec: face')

# Left column labels
axs[0, 0].set_ylabel('aa: False')
axs[1, 0].set_ylabel('aa: False')
axs[2, 0].set_ylabel('aa: True')
axs[3, 0].set_ylabel('aa: True')

for i in range(4):
    for j in range(4):
        plot_func(axs, i, j)

# plt.show()

fig.savefig('pcolormesh_snap.png')

t1 = time.time()
print("Total execution time (s): ", t1-t0)

anntzer · 2020-01-04T14:50:21Z

{aa:True ec:none} looks worse after? (basically https://github.com/jklymak/contourfIssues?) The tradeoff may be worth it, I'm just wondering whether that's the reason the code got in to start with?

greglucas · 2020-01-04T14:57:26Z

I tried going back in the blame and couldn't find any explanations for these values/decisions. Probably in some really old initial commits. This way would at least let the user decide what they want to do now.

Also, if you increase the mesh size you can see that this addition at least maintains some semblance of alpha, whereas all of the previous plots look like they lose all transparency.

Also of note, this is only for flat shading. My guess is that this could be extended to other pathways as well (gouraud, markers, etc...). But, I wanted to at least get the discussion started and see if this is worthwhile or not on a very common use-case (colorbars calling pcolormesh).

anntzer · 2020-01-04T15:05:33Z

The changes look fine to me, but perhaps we need to check whether some image comparison failures can be fixed by actually setting edgecolor to their previous effective value.

jklymak · 2020-01-04T15:37:33Z

When I looked at this before the issue was that this was inconsistent between output types as well. So good to check png and PDF

greglucas · 2020-01-06T17:00:28Z

@jklymak I'm not sure what you mean by inconsistencies? I am only changing the AGG code, so I would assume this is only modifying pdf when the quadmesh calls are rasterized (maybe all the time because there is no specific quadmesh function in the backend_pdf file?)

matplotlib/lib/matplotlib/backends/backend_pdf.py

Lines 88 to 93 in 45cfd87

    
           # TODOs: 
        
           # 
        
           # * encoding of fonts, including mathtext fonts and unicode support 
        
           # * TTF support has lots of small TODOs, e.g., how do you know if a font 
        
           #   is serif/sans-serif, or symbolic/non-symbolic? 
        
           # * draw_quad_mesh

Of course, per your nice write-up of contourf issues, there are various antialiasing issues with the viewers themselves that come into play if that's what you're referring to?

As a note, a lot of the image tests are actually both png and pdf in the test suite, so those are covered here. A lot of those are different because the colorbars have changed now.

I'm also going to ping @mdboom and @efiring as they have commented a lot on this in the past and probably have an inkling if I'm missing something here or should implement this in a different way.

As another discussion, the _draw_path_collection_generic call in the most general sense actually sets snap and curve to true.

matplotlib/src/_backend_agg.h

Lines 1053 to 1069 in a6f5a28

    
           _draw_path_collection_generic(gc, 
        
                                         master_transform, 
        
                                         gc.cliprect, 
        
                                         gc.clippath.path, 
        
                                         gc.clippath.trans, 
        
                                         path, 
        
                                         transforms, 
        
                                         offsets, 
        
                                         offset_trans, 
        
                                         facecolors, 
        
                                         edgecolors, 
        
                                         linewidths, 
        
                                         linestyles, 
        
                                         antialiaseds, 
        
                                         offset_position, 
        
                                         true, 
        
                                         true);

QuLogic · 2020-01-07T05:14:53Z

For interest's sake, I looked at the effect of this on Cartopy. It changes results for 7 tests, 6 of them that are specifically for pcolormesh, plus the colorbar in test_contour_label. I can't say exactly whether one or the other is better, except the colorbar one which looks better aligned with the ticks.

nunocalaim · 2020-01-09T13:27:14Z

Let me add that I am facing similar issues that I would like to see "fixed".

I wonder if there is a temporary fix to this? Thanks

greglucas · 2020-01-09T15:02:27Z

@nunocalaim, you can try and use a different backend (Cairo for example). I've had success using mplcairo and added an alpha-blended colormap example over there.

efiring · 2020-04-04T07:14:56Z

This looks to me like an improvement. In the demo, the only comparison that is slightly worse is aa:True, ec:None, skewed, no alpha. The aa:True, ec:None, with alpha cases are remarkably improved. The desired target should be the ec:None cases because for this sort of plot ec:face doesn't make sense--it is just an attempt to work around the rendering problem.

tacaswell · 2020-04-04T19:12:46Z

@jklymak Can you be responsible for 👍 / 👎 on this change?

efiring · 2020-04-04T19:46:15Z

Does the snapping make sense with aa:True?
Since this PR is changing 2 things, and since it improves some configurations but seems to make one worse, perhaps the 2 changes shouldn't always go together.

jklymak · 2020-04-04T20:55:54Z

Is the snapping the problem with the skewed aa:True, ec:none case?

This change is small and seems reasonable to me. CI has lost its artifacts, so not sure how many images this hits...

efiring · 2020-04-05T01:02:46Z

I just did a test so I could see the result with and without the snap.
Result: setting snap to true cleans up the colorbar with alpha, and the non-skewed aa:true, ec:none case, without visibly hurting or helping any of the other cases.

Based on the given test case, I see the changes in this PR as a substantial net gain. The penalty in the one case--skewed, aa:true, ec:none--can still be hacked away at the user level by setting edgecolor='face'. This is actually an API cleanup, because it means the user's setting of edgecolor is being respected.

greglucas · 2020-04-05T01:12:27Z

I just pushed up a new commit to update the snap within the collection itself. Not sure if that is actually needed or not, but, now we'll have some artifacts up in CI again. On my local system it was ~40 images. All basically just improvements in pixel positioning.

@anntzer, I'm wondering if this is worth updating all of these images if you're going to be working on a new image difference routine this summer in GSOC? This many images would be ideal for something like that...

anntzer · 2020-04-05T10:27:19Z

If you're willing to wait a couple of months for the new image comparison system (#16447) to come in, I guess there's a reasonable chance that it will and that would certainly be nicer than adding another 40 new baseline images. (Given the current release cycle timescale I guess that would just mean missing 3.3 and going in in 3.4?) So I would prefer waiting, but if others feel strongly about it I'm not going to block the PR over that (note that I haven't reviewed carefully the latest iterations, but looks like there's plenty of other qualified reviewers already on it :-)).

greglucas · 2020-04-06T03:12:53Z

I just realized that the pcolormesh_alpha test doesn't even fail with this update, when I would have thought it would!

matplotlib/lib/matplotlib/tests/test_axes.py

Line 1220 in d1f07d3

def test_pcolormesh_alpha():

Because you need to actually request antialiased=True in the pcolormesh call to force the new version. Right now, the pdf version looks good, but the png is atrocious.

Current png

With this update and with a small update to the gouraud shading too, I can get it to look OK:

Possible AGG update

But, even then there are still some aliased artifacts between the solid regions (lower left panel) that don't look that great. I decided to check mplcairo on this too and it is much nicer (also less than half the saved png size).

mplcairo

Summary

I'm not in any absolute rush to get this in. I just found this to be a nice update to the defaults so put it out there for feedback and conversation. But, it is really quite confusing (for me) to figure out all of the scanlines and various combinations of snapped, curved, antialiased, rgba/rgba_pre, and how all of those should be put together into the AGG mpl code, so I have no idea how to get it to the "best" possible AGG rendering.

To me, this (middle AGG png) still seems like just a half-solution, because it would update a lot of images, but maybe not even to a "correct" state.

greglucas · 2020-06-26T13:59:26Z

I decided to go through and look at all of the tests that were failing and updated all 40+ of the images. Every one of them had a colorbar or pcolormesh in it and all of the changes were essentially pixel snapping and as far as I can tell they all looked like improvements more closely aligning with where the ticks are placed (some previously had a pixel or more offset).

Looks like it is passing all of the tests now.

jklymak

I think we should merge this. Hopefully the test image PR mitigates the problem with changing a bunch of images. But I don't think that should hold this up.

anntzer · 2020-06-26T17:32:11Z

Can we just gate the behavior on a (possibly private) rcparam that is disabled by the test suite (except perhaps for a single test that checks the new behavior)? We can always get rid of it once the baseline images project is done, or even just decide to drop it later and put in the new images if we really want to.

greglucas · 2020-06-27T03:19:44Z

I'm going to be gone for a week or two here without any access to this, but I can look into making some kind of switch in the test suite when I get some time after that. (I didn't add up how large each of the new images are in the repo either to know how much this will balloon everything)

The other problem we'll run into though is that matplotlib will be just fine with the rcparam switch, but we'll still likely impact other packages that depend on matplotlib and do image comparisons including colorbars or quadmeshes.

Things to do still:

This should probably have a What's New entry or some other way of notifying people this is updated (Bug fix?)
Put an example in demonstrating how to use these parameters (adapting the demo above).

anntzer · 2020-06-27T11:20:32Z

We could even expose the rc publically with a note that this is solely to stabilize test images and may be removed in the future (there's precedence for that: text.kerning_factor basically serves a similar purpose).

greglucas · 2020-07-05T20:29:58Z

Thanks for the suggestion about the kerning rcparam. I followed that idea and it seems to do the trick here with everything still passing and no images needing to be updated right away.

I added a quick what's new entry clearly demonstrating the improvement:
https://40534-1385122-gh.circle-artifacts.com/0/doc/build/html/users/next_whats_new/pcolormesh_alpha_improvement.html

I think this is ready for a proper review and to see if anyone has any other suggestions for a different way to deal with the images issue, or anything else requested for this PR.

efiring

I think this is a net improvement both in performance and in code consistency and clarity. There are still cases with room for improvement, primarily 2 of the cases with slanted edges and ec=none. (I think ec should be none, so I don't care about the ec='face' cases.) The problem with those remaining cases is probably deeper in the guts of agg, so it shouldn't block this PR.

anntzer · 2020-07-06T06:17:54Z

doc/users/next_whats_new/pcolormesh_alpha_improvement.rst

+
+Due to how the snapping keyword argument was getting passed to the AGG backend,
+previous versions of Matplotlib would appear to show lines between the grid
+edges of a mesh with transparency. This version now applies sub-pixel snapping


I thought this was snapping to full pixels? (Maybe I'm just confused.)

Good question, I'm confused now too!... My recollection from reading the AGG documentation a while ago was that it has sub-pixel resolution (I think I recall there was an extra buffer added somewhere for this in the scanlines or similar?). I may have conflated the sub-pixel resolution with the ability to snap at a sub-pixel level.

Here is a write-up on the text pixel positioning (sub-pixel level) http://agg.sourceforge.net/antigrain.com/research/font_rasterization/
I'm not entirely clear if that is applicable to these other geometries though.

Finding documentation on all of this is quite challenging, so if you are pretty sure it is actually to pixels then I can just drop the 'sub'.

The quick and dirty solution would be to just write "snapping" without saying snapping to what ;) TBH 1) I'm not super certain here either, I think it's pixel snapping but would need to dig more into it (which I'm not going to do right now) and 2) I don't think that many users will care about the technical details anyways ;)

Good suggestion :) I just removed 'sub-pixel' altogether.

anntzer · 2020-07-06T16:26:42Z

src/_backend_agg.h

                                  linewidths,
                                  linestyles,
                                  antialiaseds,
                                  OFFSET_POSITION_FIGURE,
-                                  false,
+                                  true, // snap


This actually only enables checking whether the snap flag is set, rather than enabling snapping (I think); perhaps the comment could be slightly clearer? e.g. just replace snap by check_snap.

Yes, that is correct. Updated with the suggestion.

…oices from Python, do the logic up there.

…update many baseline images.

… prevent baseline image updates.

jklymak · 2020-07-15T23:06:59Z

Lets get this in master and see what it breaks. Thanks @greglucas !

greglucas · 2020-07-15T23:10:43Z

Downstream libraries I'm guessing! I only hope it is for the better.

jklymak · 2020-07-15T23:13:13Z

Are there some we should ping?

greglucas · 2020-07-15T23:15:21Z

xarray immediately comes to mind. @QuLogic and I can take care of Cartopy. Honestly, anyone who has image comparisons that include colorbars will be impacted by this, and I would guess that is a significant number.

jklymak · 2020-07-15T23:29:20Z

ping @jhamman @dcherian at xarray

@ngoldbaum at yt

If you have image comparisons (particularly w/ colorbars) they may start failing on matplotlib master. Either update your images ;-) or you can set plt.rcParams['pcolormesh.snap'] = False

Also any big complaints about this, please let us know!

ngoldbaum · 2020-07-16T00:28:27Z

@munkm re yt image comparison tests ^

dstansby · 2020-07-16T11:45:55Z

Just hit this in sunpy image tests, but all looks fine apart from minor colorbar changes 👍 (https://56597-2165383-gh.circle-artifacts.com/0/.tmp/py37-figure-devdeps/figure_test_images/fig_comparison.html and scroll down to a figure with colorbars to see the change if anyone's interested)

jklymak · 2020-07-16T14:31:21Z

@dstansby thanks! Sorry, I forgot about sunpy in the above. We need a list of packages we know use image tests.

tacaswell assigned jklymak Apr 4, 2020

tacaswell added this to the v3.3.0 milestone Apr 4, 2020

greglucas force-pushed the pcolormesh_snapping branch from 922636a to 94c5d4f Compare April 5, 2020 01:06

QuLogic modified the milestones: v3.3.0, v3.4.0 May 5, 2020

greglucas mentioned this pull request May 5, 2020

Heatmap: nan cause "color leak" in other cells #17323

Open

greglucas force-pushed the pcolormesh_snapping branch 2 times, most recently from 1c9311f to 4d7b0b6 Compare June 26, 2020 04:21

jklymak approved these changes Jun 26, 2020

View reviewed changes

dstansby added the backend: agg label Jun 29, 2020

greglucas force-pushed the pcolormesh_snapping branch 2 times, most recently from 060ff1f to 40194c8 Compare July 4, 2020 20:20

efiring approved these changes Jul 5, 2020

View reviewed changes

anntzer reviewed Jul 6, 2020

View reviewed changes

greglucas force-pushed the pcolormesh_snapping branch 2 times, most recently from 1c44544 to f29553f Compare July 6, 2020 15:02

anntzer reviewed Jul 6, 2020

View reviewed changes

greglucas added 5 commits July 6, 2020 11:16

AGG: Forcing quadmesh to call snap. Also, don't override edgecolor ch…

6a8645f

…oices from Python, do the logic up there.

Adding snap capabilities to gc within quadmesh.

3f963ff

Adding a temporary pcolormesh snapping rcParam to remove the need to …

b38c7de

…update many baseline images.

Updating tests with a new rcparam override for pcolormesh snapping to…

36d2f15

… prevent baseline image updates.

Adding a whats new entry for pcolormesh snapping.

f31c51a

greglucas force-pushed the pcolormesh_snapping branch from f29553f to f31c51a Compare July 6, 2020 17:17

jklymak merged commit d85adce into matplotlib:master Jul 15, 2020

greglucas deleted the pcolormesh_snapping branch July 15, 2020 23:09

dstansby mentioned this pull request Jul 16, 2020

Update figure test hashes sunpy/sunpy#4367

Merged

greglucas mentioned this pull request Jul 16, 2020

ColorbarBase draws edges in slightly wrong positions. #4584

Closed

greglucas mentioned this pull request Jul 28, 2022

[Bug]: Hist2d is less sensitive to narrow bins in new version, resulting in different plots #23505

Closed

QuLogic mentioned this pull request Apr 26, 2023

Small glitches on heatmaps #9831

Closed

greglucas mentioned this pull request Dec 1, 2023

pcolormesh with colormaps having transparency produces a moire pattern for many projections SciTools/cartopy#2295

Closed

Uh oh!

Change pcolormesh snapping (fixes alpha colorbar/grid issues) [AGG] #16090

Change pcolormesh snapping (fixes alpha colorbar/grid issues) [AGG] #16090

Uh oh!

Conversation

greglucas commented Jan 4, 2020

PR Summary

Consequences

Proposed Changes

Comparison results

Before:

After:

Demo code:

Uh oh!

anntzer commented Jan 4, 2020

Uh oh!

greglucas commented Jan 4, 2020

Uh oh!

anntzer commented Jan 4, 2020

Uh oh!

jklymak commented Jan 4, 2020

Uh oh!

greglucas commented Jan 6, 2020

Uh oh!

QuLogic commented Jan 7, 2020

Uh oh!

nunocalaim commented Jan 9, 2020

Uh oh!

greglucas commented Jan 9, 2020

Uh oh!

efiring commented Apr 4, 2020

Uh oh!

tacaswell commented Apr 4, 2020

Uh oh!

efiring commented Apr 4, 2020

Uh oh!

jklymak commented Apr 4, 2020

Uh oh!

efiring commented Apr 5, 2020

Uh oh!

greglucas commented Apr 5, 2020

Uh oh!

anntzer commented Apr 5, 2020

Uh oh!

greglucas commented Apr 6, 2020

Current png

Possible AGG update

mplcairo

Summary

Uh oh!

greglucas commented Jun 26, 2020

Uh oh!

jklymak left a comment

Choose a reason for hiding this comment

Uh oh!

anntzer commented Jun 26, 2020

Uh oh!

greglucas commented Jun 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anntzer commented Jun 27, 2020

Uh oh!

greglucas commented Jul 5, 2020

Uh oh!

efiring left a comment

Choose a reason for hiding this comment

Uh oh!

anntzer Jul 6, 2020

Choose a reason for hiding this comment

Uh oh!

greglucas Jul 6, 2020

Choose a reason for hiding this comment

Uh oh!

anntzer Jul 6, 2020

Choose a reason for hiding this comment

Uh oh!

greglucas Jul 6, 2020

Choose a reason for hiding this comment

Uh oh!

anntzer Jul 6, 2020

greglucas commented Jun 27, 2020 •

edited

Loading

ngoldbaum commented Jul 16, 2020 •

edited

Loading