Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Drop support for python 2 #9404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tacaswell opened this issue Oct 14, 2017 · 18 comments
Closed

Drop support for python 2 #9404

tacaswell opened this issue Oct 14, 2017 · 18 comments
Labels
API: changes Release critical For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.
Milestone

Comments

@tacaswell
Copy link
Member

In line with our commitment to drop python2 by 2020, make Matplotlib 3.0 python 3 only.

Mailing list discussions:

https://mail.python.org/pipermail/matplotlib-devel/2016-May/000374.html
https://mail.python.org/pipermail/matplotlib-devel/2017-October/000892.html

count down to py2 EOL: https://pythonclock.org/

@tacaswell tacaswell added API: changes Release critical For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions. labels Oct 14, 2017
@tacaswell tacaswell added this to the 3.0 milestone Oct 14, 2017
@QuLogic
Copy link
Member

QuLogic commented Oct 14, 2017

One thing to note from that discussion is that 3.0 is Python 3 only, but is not intended to break any other API.

@anntzer
Copy link
Contributor

anntzer commented Oct 14, 2017

I would suggest deprecating passing a lot of long argument lists positionally -- to be decided on a case-by-case basis. For example, for hist, I'd say any argument starting from range (or at least starting from density) should become kwonly.
(We can implement a decorator that tries to bind the function arguments using the kwonly signature, with a fallback with deprecation warning to a non-kwonly signature.)

@tacaswell
Copy link
Member Author

I owe the list a bunch of writing on this, but I would prefer that any big API changes be done by extending in a different direction and module rather than trying to mutate the exsting APIs into the new ones.

This is rather uncharacteristic of me, but ...

@Zac-HD
Copy link
Contributor

Zac-HD commented Jan 9, 2018

We should have an issue label for things that are waiting to drop Python 2, and apply it to this issue, #9912, and probably a few others!

@anntzer
Copy link
Contributor

anntzer commented Feb 1, 2018

wxpython 4 has finally been released as the first official wx to support Py3; we should drop the workarounds/conditional paths that were needed to support the wx betas as well.

@mscuthbert
Copy link
Contributor

From my experience bringing a smaller but still substantial project (music21) from Py2/3 to Py3 only, from PyPI stats, about 90% of our users on the Py3-only release are on Python 3.6, so if your user base is similar you're not going to limit the audience too much by making Py3.6, as opposed to 3.4 or 3.5, the minimum supported version. And you'll get f-strings and a fully functional Pathlib system, which were some of the main nice things of Py3 that I miss by still supporting 3.4

@ImportanceOfBeingErnest
Copy link
Member

Matplotlib is used a lot in environments which are deeply entangled to hardware. Updating such a system to a new python version is a major step (you do not simply get the new version from conda or so). Especially in science labs you may simply not have the manpower to do such a step and you cannot affort a PhD student spending half a year on that. What will hence happen is that those systems are stuck with the last matplotlib version supporting py2 or py3.4.

This is fine, but please make sure that they still have easy access to that version even after 2020.

@matthew-brett
Copy link
Contributor

@mscuthbert - as a matter of interest - what proportion of your missing 10% of Python 3 users were on Python 3.4? I have the impression that isn't very widely used.

Matplotlib has to be a little bit more careful than other libraries, because it's near the bottom of quite a few software stacks, so when matplotlib drops support for - say - 3.5 - everyone above us in the stack has to upgrade Python.

@ImportanceOfBeingErnest
Copy link
Member

ImportanceOfBeingErnest commented Apr 6, 2018

Discussion on such topic would require to be based on some more substantial grounds. Therefore find here a visualization of download numbers from pypi.

image

Method: The data is obtained from the the-psf:pypi.downloads dataset queried via the GoogleBigQuery API. A summary of this approach can be found in this blog post, from which the respective SQL queries are adapted for the case of interest here.

Analysis: We observe that ~60% of matplotlib downloads during the month between 5th of march and the 5th of April are made up by the version for python 2.7. Some 35% of downloads are for the various python 3 versions.
Those numbers correlate well with the download numbers for other packages from the scientific python universe, like numpy and pandas.
The percentage of downloads of python 2 compared to overall python downloads is slowly dropping within the last 2 years from over 80% to 70%, while the percentage of matplotlib downloads for python 2 compared to all matplotlib downloads is fluctuating between 50% and 70% with no clear tendency to be seen.

Discussion: Unfortunately those numbers do not provide an estimate of the actual usage of matplotlib or python. What can be concluded is that dropping python 2 support deprives a majority of downloaded python versions of the option to use the newest matplotlib.
Should there be any correlation between download number and usage base, the step to drop python 2 support would cut the use base by half.

@efiring
Copy link
Member

efiring commented Apr 6, 2018

"What can be concluded is that dropping python 2 support deprives a majority of downloaded python versions of the option to use matplotlib."

This conclusion is not correct. (1) They still have the option to use mpl v2.2. (2) In many cases they also have the option to switch to Python 3, and will do so over the next couple years as support for Python 2 declines. (Your last conclusion is also incorrect.)

I understand the difficulties with switching from 2 to 3, and it is unfortunate that the history of Python development led us to this mess; but we're stuck with it.

I'm tempted to close this issue, since we have made our decision and we are proceeding; but perhaps @tacaswell has reasons for wanting to leave it open.

@tacaswell
Copy link
Member Author

@ImportanceOfBeingErnest Thank you for digging up and plotting that data!

Master branch is currently py3.5+ and the next feature release will be Matplotlib 3.0 this summer cut off the master branch. We are not going to go back on that.

We will be back porting critical bug fixes to 2.2.x and doing bug fixes for the 2.2 series until Jan 1 2020 (see https://matplotlib.org/devel/coding_guide.html#branches-and-backports). If there is someone who wants to take point on more extensive backports (non-critical bug fixes) and manage the 2.2.x release I am open to that, but so far no one has volunteered.

There are no plans to remove old versions from pypi, you will be able to install Matplotlib 2.2.x on python 2.7, just as if you are using python 2.6 now and pip install matplotlib you will get 1.4, however to get new features you have to use newer versions of python.

Interestingly these numbers do not line up with https://www.jetbrains.com/research/python-developers-survey-2017/#python-3-adoption which reports a 80/20 py3/py2 split for self-reported data scientists (which being self reported may be wishful reporting) nor with https://anaconda.org/conda-forge/matplotlib/files?sort=ndownloads&version=2.2.2&sort_order=desc which is ~69% (37885/17160) py3 (which may represent 'power users' who are aware of conda-forge).

I suspect that the pip installs are going to undergo a phase transition in the near future as ubuntu 18.04 will have only python 3.6 installed and fedora is the process of orphaning all of their ptyhon2 packages.

At some level this is a question of engineering resources on one hand we have (possibly a minority) of users saying they can not afford the engineering cost to move to python3 there for upstream projects must continue to pay (forever?) the engineering cost of supporting python2 for all new features.

@ImportanceOfBeingErnest
Copy link
Member

@efiring The sentence reads

What can be concluded is that dropping python 2 support deprives a majority of downloaded python versions of the option to use the newest matplotlib.

(I might have edited that a couple some minutes after posting, so if you are responding to some email that could explain the difference?)

So (1) does not apply. (2) is nonsense, a python 2 version cannot decide to be a python 3 version.
(Last) You may replace "correlation" by "proportionality" if that makes you happy.


@tacaswell I did not know about the jetbrains developper survey you mention. I did know about a previous one from 2016 which reported 50:50 between py2 and py3. So whoever calls themselves a developper and is in the mood to fill out surveys will likely not suffer from the transition.

I'm sure the truth lies somewhere in between. The survey has a limited representativity. And the pypi raw data only shows actual downloads, and not the usage. In that sense both cannot be used to draw accurate conclusions but are still very interesting to look at and judge on what might happen in the next months and years. So I think it's useful to have those numbers available. And I think one can even digg a bit deeper, resolving matplotlib versions time-dependently (unfortunately, my BigQuery quota is eaten up for this month, so I cannot do that now) to see if there is/was a python 2.6 vs 2.7 effect seen for matplotlib.

It is good to be very transparent on all of those steps. I think this increases acceptance for it.
Possibly a clear anouncement on the homepage (the entry page matplotlib.org) would also help. Written in an assuring way like:

Matplotlib will transition its development completely to python 3. This means starting with version 3.0, to be released in summer 2018, it will require python 3.5 or higher to be run (see full requirements here, >link<). The latest version to support python 2.7 is matplotlib 2.2.x. If you are using python 2.7 and cannot upgrade, don't worry, matplotlib 2.2.x is and will still be available through the usual channels (>link to how to install<, possibly updated with explicit section about 2.7). Critical bugfixes are still being backported until 2020 when python itself drops support for version 2. Also the matplotlib 2.2.x documentation will stay available with no plan to ever remove it.

@timhoffm
Copy link
Member

timhoffm commented Apr 6, 2018

I don't want to join the discussion. Only a quick remark which may help understanding the numbers:

From my experience, pip users are mostly linux/venv. They tend to use the preinstalled python version and just update the packages. In particular the LTS versions then lead to a lag in the python version. So there is reasonable hope that python 3 usage will increase with the upcoming linux distros.

In contrast, anaconda users tend to install the latest python version in their environments.

@matthew-brett
Copy link
Contributor

IPython did this split about a year ago. They use environment markers to express the fact that the latest releases require Python 3, and therefore, Python 2 installing via pip finds and installs the older LTS release series (as the higher version number versions get rejected, because of the Python 3 requirement).

@dopplershift
Copy link
Contributor

Another point in the discussion is that we're not doing this in a vacuum. Pandas, xarray, numpy, and Python itself (among others) are all completely moving on from 2.7 in 2020.

@QuLogic
Copy link
Member

QuLogic commented Apr 7, 2018

@matthew-brett yes, we have implemented the same thing as IPython (based on their pycon talk), so 2.2 should continue to be the default for Python2 even when we release 3.0.

@timhoffm
Copy link
Member

Is there still something to be done here or can this be closed?

@mscuthbert
Copy link
Contributor

It could be closed since this discussion has run its course, but in other projects I've been on that have dropped support for Python 2, there tends to be a followup issue that lists a bunch of maintenance issues that allow for removing legacy code, such as, removing all calls/dependencies to six, removing u'' strings, updating super() to new syntax (seems to be in progress); that could happen here or it could be a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API: changes Release critical For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.
Projects
None yet
Development

No branches or pull requests

10 participants