-
-
Notifications
You must be signed in to change notification settings - Fork 10.9k
CI: Upload nighlighties to new location #23792
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Should post to the mailing list, and also make a case for the change. My first guestion is "Why?". |
This is being worked on during the scientific Python summit - the main motivation is to move to a better name and more visibly-community-owned location, i.e. "scipy" -> "scientific python". See also networkx/networkx#6700. This will only affect CI and @bsipocz et. al. will handle all the downstream updates. |
A bit of context - the Scientific Python Developer Summit is happening this week. A part of the effort is to ensure that Scientific Python core packages follow the upcoming SPEC documents and support their implementation across the ecosystem. One of the specific changes is a move of the nightly wheels from Once all of the packages that currently use One more step will need to happen here though. The upload requires an upload token stored as a repository secret (as you have right now). However, you will need to generate a new token for the new anaconda organisation. Let us know who is managing this at the moment so we can give you a proper access to the new repo so you can generate a new token, unless @jarrodmillman already added you. The same change is happening in the other packages as well. |
@martinfleis I suspect that this is more related to promoting scientific python and bringing everything under that umbrella. Why not, for instance, make it numfocus site instead :) What about the staging repositories? It feels like meddling to start with the implementation rather than the argument. |
If I understand the intent correctly, the idea is to move the archive to a new group with a better name, without any change in policy, making this is a cosmetic change. Is that correct? Here is the page for the owners of the old location. Among them I recognize @bsipocz and @tylerjereddy (maybe more)? who are listed as participants in the summit. If indeed the intent is cosmetic, they can invite all the other owners to the new org. |
We have encouraged all downstream package authors, of both public and private packages, to use these nightlies (see, e.g., https://numpy.org/devdocs/dev/depending_on_numpy.html#testing-against-the-numpy-main-branch-or-pre-releases) for years now. I don't see how you're going to be able to open PRs to the repo of every consumer. This feels a bit disruptive for little gain. |
right, we won't be able to find the private packages |
You could use both indexes for a few years, until we see the download statistics drop off |
We could upload both for a bit, the thing I would worry is about someone using an outdated channel for months. Maybe we can prevent that e.g. by uploading an "invalid" wheel that prints out a message? If this helps with getting a lot more things into one location (and we want that), say astropy, Scikit-HEP, ... Than maybe the disruption is fine? I doubt that many use the channel overall and most of them opt-in to be alpha testers. And since many would be added soonish it would be the last good time to do cosmetic changes, if it is just cosmetic (if those packages prefer the new name). |
That was suggested over in SciPy at scipy/scipy#18504 (comment) |
What SciPy is an is not is a real source of confusion for most of our newcomers. For folks who have been long enough the community to know the backstories, it's clear, automatic and does not matter. But for everyone else, it's strange. Moving to a new org which is not tied to a specific project is sensible and is a clear messaging (helpful for many things.) NumFOCUS' current purpose is not really to provide infrastructure nor development tools. That's one of the role of Scientific Python though which makes it, IMHO, the perfect place to have that. As for downstream users, pushing a broken release with instructions should do it. It's a one line change, pretty much insignificant for folks really using nightlies. If they do use nightlies, it's probably because something in NumPy is breaking them from time to time. This would just be yet another similar "breakage" with a clear resolution path. Also stats on how much these are downloaded don't really show a lot of usage https://anaconda.org/scipy-wheels-nightly/numpy/files |
We discussed this briefly today and I don't think anyone had serious concerns, so putting this i. @jarrodmillman and @bsipocz are planning to make sure downstream is being moved timely, if there is some fallout, it is easy to revert. Thanks @jarrodmillman! |
FYI, the first batch of wheels are now (almost) uploaded: https://anaconda.org/scientific-python-nightly-wheels/numpy so downstream can change their sources (assume they don't rely on other packages like SciPy that didn't upload yet). |
People had concerns and expressed them here. Maybe they were not serious, but neither mitigation policy was implemented:
and the download numbers would indicate that the communication to the downstream users was not complete, even though the NumPy documentation now points to the new site. |
@bsipocz what is the state of the tool that shall auto-open PRs in downstream projects pointing to the new location? |
Transition is indeed not complete. Right now its probably mainly pandas that needs to merge (to unblock sklearn also). SciPy needs to upload to the new url, but it sounds like that will happen today. I do want the broken upload also, but it only makes sense after scipy, pandas, and sklearn are done. If that didn't happen in a week, I guess we may have to do the dual upload. |
No description provided.