Fix unit handling in errorbar for astropy. #19872

anntzer · 2021-04-05T18:49:57Z

Unfortunately, this bug only triggers with astropy's unit
implementation, but not with either of matplotlib's "test" units
(testing.jpl_units and test_units.Quantity). So I don't have a
self-contained way to add a test...

Closes #19526 (comment).

PR Summary

PR Checklist

Has pytest style unit tests (and pytest passes).
Is Flake 8 compliant (run flake8 on changed files to check).
New features are documented, with examples if plot related.
Documentation is sphinx and numpydoc compliant (the docs should build without error).
Conforms to Matplotlib style conventions (install flake8-docstrings and run flake8 --docstring-convention=all).
New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).

Unfortunately, this bug only triggers with astropy's unit implementation, but not with either of matplotlib's "test" units (testing.jpl_units and test_units.Quantity). So I don't have a self-contained way to add a test...

jklymak · 2021-04-05T19:43:54Z

This seems fine. It would be nice if astropy could kick us back a mock-units interface that shadows theirs and added some tests. Its pretty hard for us to not break things if we can't directly test them.

jklymak · 2021-04-05T19:44:28Z

lib/matplotlib/axes/_axes.py

@@ -3427,13 +3427,17 @@ def extract_err(name, err, data, lolims, uplims):
                the note in the main docstring about this parameter's name.
            """
            try:
-                low, high = np.broadcast_to(err, (2, len(data)))
+                np.broadcast_to(err, (2, len(data)))


Is it possible to check for this directly, rather than via an error? Maybe this is just as efficient?

I can't find anything under https://numpy.org/doc/stable/reference/routines.array-manipulation.html#changing-number-of-dimensions :(

... I guess I just mean int(len(err)/2)==len(err)/2, or whatever is more elegant than that?

Performance should not matter. This should be proportional to len(data) in computation/mempry and is only called ~~once~~ twice for the whole errorbar function.

timhoffm · 2021-04-05T20:43:49Z

lib/matplotlib/axes/_axes.py

@@ -3427,13 +3427,17 @@ def extract_err(name, err, data, lolims, uplims):
                the note in the main docstring about this parameter's name.
            """
            try:
-                low, high = np.broadcast_to(err, (2, len(data)))
+                np.broadcast_to(err, (2, len(data)))


Performance should not matter. This should be proportional to len(data) in computation/mempry and is only called ~~once~~ twice for the whole errorbar function.

pllim · 2021-04-06T06:15:53Z

Thank you for the quick fix!

mock-units interface that shadows theirs

Not sure how easy that would be. Perhaps @mhvk can advise.

pllim · 2021-04-06T13:38:46Z

To close the loop, I can confirm that this patch works for astropy. Thanks again!

jklymak · 2021-04-06T14:10:25Z

Great! Thanks for testing our Master branch!

mhvk · 2021-04-07T15:36:50Z

lib/matplotlib/axes/_axes.py

@@ -3427,13 +3427,17 @@ def extract_err(name, err, data, lolims, uplims):
                the note in the main docstring about this parameter's name.
            """
            try:
-                low, high = np.broadcast_to(err, (2, len(data)))


Sorry I looked at this so late. A simpler solution here is to add subok=True...

Just today, I decided to look at how matplotlib does this, because I need a similar interface (accept a scalar and 1 d array or a 2 d array - just like err here. I guess that's a rare case that that happens, but should I open a PR to simplify this back to np.boradcast_to with subok?

mhvk · 2021-04-07T15:39:47Z

On providing a mock unit/quantity: that would be good, but also somewhat tricky, as our Quantity class overrides numpy ufuncs and functions (like np.broadcast_to here), hence the implementation of the mock class might not be much smaller than the real one... On the other hand, if it would OK for some tests to depend on astropy (perhaps only in a weekly/monthly job or so), that should not be too tricky.

tacaswell · 2021-04-07T19:17:32Z

We already have some "importorskip" tests for pandas so, depending on how long installing the astorpy wheels adds to the CI, I am nominally 👍 on adding some that depend on astropy. I suspect we can add a module that looks like

import pytest
pytest.importorskip('astropy')
from astropy.tests import the, tests, that, hit, unit, plotting

which should both skip the whole module if astropy is not installed and let us run the astropy tests without having to re-implement them!

timhoffm · 2021-04-07T19:32:32Z

Not sure if that's possible, but could the tests be pytest.marked in astropy so that we can collect them by mark and don't need to hard-code them on our side?

mhvk · 2021-04-07T19:38:40Z

Our tests are super-localized (to astropy.visualization) so this could work well (either the marking or the importing). If someone could point me to where the equivalent pandas tests live, I can try to have a go at it.

pllim · 2021-04-07T19:39:29Z

Or you can wait for us to open issues when things fail... 😆

jklymak · 2021-04-07T21:14:45Z

Is this really an approach we want to take? I would rather we actually define what rules we expect unit implementations to follow and then test against those rules, rather than bless a particular implementation of those rules. But absent anyone actually defining those rules, I guess this is ok as a stopgap.

The problem with unit support is that it is hard. The fact that astropy had to dig into the ufuncs hopefully makes that clear. Do we trust that another downstream package has made the same low-level choices? What range of choices are we going to accept?

timhoffm · 2021-04-07T21:31:33Z

Or you can wait for us to open issues when things fail... 😆

is the minimal stopgap solution. Since astropy thankfully tests against master, we at least realize if something gets broken before any release. That way we keep astropy working without explicit unit rules and without formally blessing astropy as reference implementation. (Yes, this is stealing away from a proper solution, but may be good enough for the time being).

mhvk · 2021-04-07T23:01:39Z

OK, definitely fine with astropy just being a slightly delayed test case! Would be the same with some weekly or monthly test as well....

Fix unit handling in errorbar for astropy.

4e16a84

Unfortunately, this bug only triggers with astropy's unit implementation, but not with either of matplotlib's "test" units (testing.jpl_units and test_units.Quantity). So I don't have a self-contained way to add a test...

anntzer added Release critical For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions. topic: units and array ducktypes labels Apr 5, 2021

anntzer added this to the v3.5.0 milestone Apr 5, 2021

jklymak approved these changes Apr 5, 2021

View reviewed changes

timhoffm approved these changes Apr 5, 2021

View reviewed changes

timhoffm merged commit abe4117 into matplotlib:master Apr 5, 2021

anntzer deleted the eu branch April 5, 2021 21:28

mhvk reviewed Apr 7, 2021

View reviewed changes

Uh oh!

Fix unit handling in errorbar for astropy. #19872

Fix unit handling in errorbar for astropy. #19872

Uh oh!

Conversation

anntzer commented Apr 5, 2021

PR Summary

PR Checklist

Uh oh!

jklymak commented Apr 5, 2021

Uh oh!

jklymak Apr 5, 2021

Choose a reason for hiding this comment

Uh oh!

anntzer Apr 5, 2021

Choose a reason for hiding this comment

Uh oh!

jklymak Apr 5, 2021

Choose a reason for hiding this comment

Uh oh!

timhoffm Apr 5, 2021

Choose a reason for hiding this comment

Uh oh!

timhoffm Apr 5, 2021

Choose a reason for hiding this comment

Uh oh!

pllim commented Apr 6, 2021

Uh oh!

pllim commented Apr 6, 2021

Uh oh!

jklymak commented Apr 6, 2021

Uh oh!

mhvk Apr 7, 2021

Choose a reason for hiding this comment

Uh oh!

hamogu Apr 8, 2021

Choose a reason for hiding this comment

Uh oh!

mhvk commented Apr 7, 2021

Uh oh!

tacaswell commented Apr 7, 2021

Uh oh!

timhoffm commented Apr 7, 2021

Uh oh!

mhvk commented Apr 7, 2021

Uh oh!

pllim commented Apr 7, 2021

Uh oh!

jklymak commented Apr 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

timhoffm commented Apr 7, 2021

Uh oh!

mhvk commented Apr 7, 2021

Uh oh!

Uh oh!

jklymak commented Apr 7, 2021 •

edited

Loading