Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Bug]: Matplotlib 3.7 has yet again changed the class returned by plt.gca() #25228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
KelSolaar opened this issue Feb 16, 2023 · 11 comments
Closed

Comments

@KelSolaar
Copy link
Contributor

Bug summary

Hello,

It is the second time in 2.5 years that a Matplotlib release breaks all our doctests because plt.gca() returns a different type. It also happened around July 2020: colour-science/colour@d5f6800

There is no notice in the release notes and it is kind of wrecking havoc on our end. I will be using longer ellipsis from now on but I was wondering if you guys had considered using SemVER?

Code for reproduction

N/A

Actual outcome

Colour's doctests are sad! :(

Expected outcome

Happy doctests!

Additional information

No response

Operating system

No response

Matplotlib Version

3.7

Matplotlib Backend

No response

Python version

No response

Jupyter version

No response

Installation

None

@rcomer
Copy link
Member

rcomer commented Feb 16, 2023

I think the minimal example here is

import matplotlib.pyplot as plt

fig = plt.figure()
ax1 = fig.add_subplot(1, 2, 1)
ax2 = fig.add_subplot(1, 2, 2, projection="3d")

print(repr(ax1))
print(repr(ax2))

v3.6.3

<AxesSubplot: >
<Axes3DSubplot: >

v3.7.0

<Axes: >
<Axes3D: >

Though I note that you can still do

isinstance(ax, matplotlib.axes.SubplotBase)

to check whether your axes was added on a grid. So this possibly only affects users who specifically check the repr, e.g. in doctests.

@anntzer
Copy link
Contributor

anntzer commented Feb 16, 2023

I do not believe that reprs should be considered as covered by any API stability unless otherwise documented (Otherwise, even adding a __repr__ method to a class that previously just inherited the default repr of all objects would be an API break, and how would you even warn about that?). This is consistent e.g. with the intent of the CPython developers themselves (https://discuss.python.org/t/documenting-python-versioning-and-stability-expectations/11090#unstable-api-10).

@jklymak
Copy link
Member

jklymak commented Feb 16, 2023

Yes, I haven't looked at your tests, but I agree with the above that testing our repr seems very fragile.

We also strongly encourage developers to run CI on our release candidates when they are announced to sort out problems before they hit an actual release. We build wheels for these. We also make a nightly wheel build available https://matplotlib.org/devdocs/users/installing/index.html#installing-a-nightly-build that you could incorporate into your CI (probably as a different run than against released versions) - we have such a run and it is extremely useful to catch numpy changes before they hit a release, and we have been alerted to issues with downstream libraries because they run our nightly's. I think astropy or sunpy do this with our nightlies, and it has caught some potential snafus.

@ksunden
Copy link
Member

ksunden commented Feb 16, 2023

As others have pointed out, we do not consider repr changes to be breaking api changes.

In fact, adding a __repr__ (@anntzer's hypothetical) is precisely what caused the July 2020 version of this.

The fact of the matter is Python is a dynamically typed/duck-typed language, and what matters as a library is that users of the public api are as minimally affected by API changes as possible.

In fact, AxesSubplot was an undocumented subclass of Axes.
The public API of gca was and is that it returns a subclass of Axes.
Technically, there is a type narrowing for 3.7 from "SubplotBase or Axes" to "Axes", but Axes was always a documented possible return from gca.
We have Merged those two classes so there is no reason to keep SubplotBase (it remains an alias so that users who directly invoke it still can, but it is no longer a separate class)

In my opinion at least, return types are very much allowed to change, as long as behavior remains consistent. (Especially if the new return type was always a documented possible return type) (See also the ArrayList class that was introduced in order to warn that that return would become immutable... trying to mutate the result is at least one additional cause of failures you linked, though that very much did have warnings, but doctests don't surface such deprecation warnings... even Numpy was caught by that one)

I was wondering if you guys had considered using SemVER?

Strict SemVer is not actually used by any general purpose, popular scientific Python library, to my knowledge. Numpy, Scipy, Pandas, etc (including Python the language itself) all have deprecation policies that include a best effort to retain backwards compatibility, where possible for minor releases. Some use date-based versioning (which I have a personal affinity towards, but I think mpl has good reason to use the scheme we currently use), most use something more "Major changes"."Feature releases, breaking changes allowed"."bugfix patch".

If we were to adopt a strict semver, I don't think it would actually be any different as to when we do releases or what goes in them... we would just bump the first number of the release instead of the second. (If anything, I think we would be more willing to make breaking changes)

Instead, we use the first number to indicate large changes that will break the majority of users (e.g. default style changes, as was the case for 2 -> 3). If you truly want the SemVer like experience (and I don't think you do, long term), you can treat the "Minor" releases as the version that can contain some backwards incompatibilities.

@dopplershift
Copy link
Contributor

I don't have much to add, other than while I appreciate your appeal to SemVer, this is precisely one of those cases that argues against SemVer being practical. Most devs would not have considered these (changing a repr or changing a return to an API-compatible type) to be breaking changes. Therefore, no one would have bumped the version for that change.

This leads many to the end conclusion that the only way to be completely SemVer "pure" is to consider every change to be a breaking change.

xkcd workflow comic

@KelSolaar
Copy link
Contributor Author

Hello,

Given the issue title and description I wrote, I would hope that it is clear that I understood that the class type/name itself changed. I haven't compared the code between 3.7 and < 3.7 so I don't know what happened under the hood and there are no mention about it in the release notes.

Now please consider it from my vantage point: the Class name/type has (possibly) changed, it means that not only my doctests are breaking but also that my static typing annotations are now possibly wrong.

Another relevant one pertaining to the discussion is that the way the Artists, e.g. Lines are accessed for an Axes has changed without any notes and is breaking code:

image

Cheers,

Thomas

@ksunden
Copy link
Member

ksunden commented Feb 16, 2023

In point of fact, the lines change did have warnings issued since matplotlib 3.5. See also numpy/numpy#23209 where they had a similar problem (with adding rather than removing).

Your tests from before our release have 1426 warnings, some of those are likely our deprecation warnings for this and possibly other pending deprecations.

As for typing annotations, until #24976 is complete (and even then will probably be documented as provisional and thus liable to change without warning for a release cycle or two) it is not recommended to type hint matplotlib code. While we are working towards the goal of being typed, it is simply not ready yet, and if you are overspecifing, you can expect that changes will be required once we do have type hints.

The returned type is the documented type still, so if you are going to type hint, it must include Axes (the "new" return type) as a possible return, which has always been true.
This is, in fact, exactly what you type hint them as (though I would perhaps stray away from using the pyplot namespace for your type hints... its not 'incorrect' nor likely to change, but I would tend to type hint as axes.Axes as that is the more natural place for the type hint to reference).

The diff you show for the lines change is both over-complicated (the 3.7 code will work for all versions of matplotlib, remove as a function of Artist was added 16 years ago), but also is doing semantically different things (pop(0) will remove one line, the for loop will remove all lines).

The following should suffice if equivalent code is desired:

axes.lines[0].remove()

We thank you for the report, but have discussed this and believe the changes are in line with our balancing of api stability/documented behavior vs making the code on our end better. As such I'm going to close this issue as we do not have any action to take on our end.

@ksunden ksunden closed this as not planned Won't fix, can't repro, duplicate, stale Feb 16, 2023
@tacaswell
Copy link
Member

For completeness, directly mutating the child lists was deprecated in 3.5 ( https://matplotlib.org/stable/api/prev_api_changes/api_changes_3.5.0.html#axes-children-are-no-longer-separated-by-type) and warned in both the 3.5 and 3.6 series.
Making them read-only is noted at https://matplotlib.org/stable/api/prev_api_changes/api_changes_3.7.0.html#modification-of-axes-children-sublists

Looking at #23573 we did not consider this an API change because you can still import SubplotBase, isinstance(ax, SubplotBase) will still pass, and we expect the API to be identical. Previously the classes were dynamically generated 😱 so were not actually importable.

If you find any differences in behavior (other than repr) please report those as bugs!

@KelSolaar
Copy link
Contributor Author

KelSolaar commented Feb 16, 2023

Thanks you all for the details.

@rcomer
Copy link
Member

rcomer commented Feb 17, 2023

@KelSolaar, going back to your doctests, I looked at some of your rendered docs and I have the impression that what you really want is the example, and the check on the repr is perhaps incidental. Is that right? If so, I believe you can add ; to the end of the final line, e.g.

>>> plot_planckian_locus(planckian_locus_colours="RGB");

which should suppress that text output and you never need to worry about whether the reprs change again.

@KelSolaar
Copy link
Contributor Author

Hi @rcomer,

The text output serves two purposes for us: Understanding the return type when you read the docstrings, and also that it matches expectations. People will often read the usage example first in the docs and then take a look at the definition signature if they need more info.

Cheers,

Thomas

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants