MAINT,BUG,DOC: Fix errors in _add_newdocs #13876

eric-wieser · 2019-06-30T19:28:56Z

Intended to be backported to 1.16.x and 1.17.x - at the very least, the first two commits, but doing the whole patch-set is probably fine too.

Brought on by noticing some mistakes in gh-13847 that we'd ideally detect at build time.

…hon 3 Avoid using `_add_newdocs` if these functions do not exist. Leaving the version-checking here so that we can backport to 1.16

This must have been lost in the multiarray / umath merge. Found by noticing that `add_newdocs` was being called on an object that does not exist.

`ndarray._as_parameter_` is not a real thing

…ay.py`

This caught us trying to document members that don't exist.

eric-wieser · 2019-06-30T19:54:06Z

numpy/core/multiarray.py

@@ -15,7 +16,7 @@
 from numpy.core._multiarray_umath import *
 from numpy.core._multiarray_umath import (
    _fastCopyAndTranspose, _flagdict, _insert, _reconstruct, _vec_string,
-    _ARRAY_API, _monotonicity
+    _ARRAY_API, _monotonicity, _get_ndarray_c_version


Fixes pymc-devs/pymc#3340, sympy/sympy#16556, and probably some others.

The name begins with a '_', shouldn't we be discouraging its use rather than adding more private methods to the public API?

Perhaps, but removing it from the API broke downstream packages, and I don't think it was a deliberate choice.

Also, this member is even documented, so I think is intended to be "public but only for library use" or something.

charris · 2019-07-01T16:50:22Z

Hmm, I see that you dealt with potential back compatibility problems, so that's good. The new functions and add_newdoc should have official NumPy docstrings. I suppose that could be done later but it would probably be best to do it now when you have been looking at the functions.

eric-wieser · 2019-07-02T05:24:30Z

I think adding full docstrings for the internal functions is probably not a good use of time or vertical space.

Note that add_newdocs does not appear in our web docs, and I'm inclined to say it shouldn't be public either. Either way, that means that it doesn't really matter if the docstring is in the standard format.

charris · 2019-07-02T15:36:51Z

If a function has arguments, they need to be documented. And tell me about vertical space again :)

numpy/core/function_base.py

mattip · 2019-07-02T19:04:57Z

numpy/core/_add_newdocs.py

 from numpy.core import numerictypes as _numerictypes
 from numpy.core import dtype
-from numpy.core.function_base import add_newdoc
+from numpy.core.function_base import add_newdoc as _add_newdoc


Rather than changing the name on import, could you change function_base.add_newdoc to function_base._add_newdoc

np.add_newdoc is public API, I think your suggestion will break that.

I see. What a mess: there is numpy.core.function_base and numpy.lib.function_base, each adds different things to __all__.

Since the implementation is specific to the _multiarray_umath module, why should add_newdoc be part of the public API? Its only use is at startup to modify the uninitialized tp_doc slot in _multiarray_umath methods.

... but not really relevant to this PR

I don't think np.add_newdoc should be public, but as a bugfix this PR tries not to break the status quo and leaves it where it is.

Either way, evidently changing add_newdocs is a can of worms, so let's not mix it with the bug fixes in this patch.

eric-wieser · 2019-07-03T06:36:38Z

I rolled back the last commit in this patch set, because I think most of the comments were directed at it, and it's also the least useful commit to backport

eric-wieser · 2019-07-03T06:39:46Z

If a function has arguments, they need to be documented.

I'm going to claim immunity from this until the follow-up PR, since now this one doesn't add any new arguments or non-trivial functions :)

mattip · 2019-07-03T15:36:49Z

reopening to trigger shippable rebuild

tylerjereddy · 2019-07-06T22:12:27Z

Strange, I manually restarted the Shippable build to see if that helps

mattip · 2019-07-06T23:08:37Z

Shippable seems to be working now, thanks @eric-wieser

eric-wieser · 2019-07-07T23:19:32Z

I'll spin off some backport PRs tonight, and resubmit the removed commit from this PR

eric-wieser · 2019-07-08T05:42:42Z

@mattip, your squash merge ruined my attempt to share the commits across maintenance branches. I've created a commit to "undo" the squash merge at gh-13936

This reverts commit 7ac7fa9.

…-python MAINT: Revert #13876, "MAINT,BUG,DOC: Fix errors in _add_newdocs"

PerryGrossman · 2019-09-01T17:13:19Z

Hi all,
Thanks for your work on this!
I am trying to resolve the numpy/theano issue that comes up when I use pymc3 (on Ubuntu 18.04 and a fresh Anaconda install). I tried updating numpy and theano; but I can only get to Theano 1.0.3 from conda. I pip installed Theano; but I can't figure out how to confirm that I have Theano 1.0.4.
I can't "conda install numpy 1.17" (package not found). "pip install numpy 1.17" indicates that the requirement is already satisfied.

I am trying work through this:
https://towardsdatascience.com/bayesian-modeling-airlines-customer-service-twitter-response-time-74af893f02c0
It fails at:
with pm.Model()
...

with a '_get_ndarray_c_version' error.

Suggestions?

mattip · 2019-09-01T17:30:13Z

@PerryGrossman why did you add that comment to this issue? Please open a new issue if you think numpy is at fault here, and follow the instructions vis-a-vis reporting python and numpy configuration. A stacktrace would also be helpful.

eric-wieser added 00 - Bug component: documentation 03 - Maintenance labels Jun 30, 2019

eric-wieser force-pushed the forbid-_add_newdocs-python branch 2 times, most recently from 5380585 to c6e69fc Compare June 30, 2019 19:34

eric-wieser added 5 commits June 30, 2019 12:41

BUG: Remove items from multiarray.__all__ which do not exist on pyt…

d474517

…hon 3 Avoid using `_add_newdocs` if these functions do not exist. Leaving the version-checking here so that we can backport to 1.16

BUG: Add missing np.core.multiarray._get_ndarray_c_version function

ae65a53

This must have been lost in the multiarray / umath merge. Found by noticing that `add_newdocs` was being called on an object that does not exist.

DOC: Remove documentation for property that does not exist

c927298

`ndarray._as_parameter_` is not a real thing

DOC: Remove docstrings which are duplicated from `numpy/core/multiarr…

598fcb4

…ay.py`

BUG: Don't silence errors in add_newdoc caused by bad arguments

b592e8f

This caught us trying to document members that don't exist.

eric-wieser force-pushed the forbid-_add_newdocs-python branch from c6e69fc to 8b8952c Compare June 30, 2019 19:42

eric-wieser mentioned this pull request Jun 30, 2019

Theano breaks on new numpy pymc-devs/pymc#3340

Closed

eric-wieser commented Jun 30, 2019

View reviewed changes

eric-wieser mentioned this pull request Jun 30, 2019

DOC: Adds documentation of functions exposed in numpy namespace #13847

Merged

8 tasks

eric-wieser added this to the 1.17.0 release milestone Jun 30, 2019

eric-wieser added the 09 - Backport-Candidate PRs tagged should be backported label Jun 30, 2019

eric-wieser modified the milestones: 1.17.0 release, 1.18.0 release Jun 30, 2019

mattip reviewed Jul 2, 2019

View reviewed changes

numpy/core/function_base.py Outdated Show resolved Hide resolved

mattip reviewed Jul 2, 2019

View reviewed changes

eric-wieser force-pushed the forbid-_add_newdocs-python branch from 8b8952c to b592e8f Compare July 3, 2019 06:36

eric-wieser changed the title ~~MAINT/BUG/DOC: Fix errors in _add_newdocs and prevent them being reintroduced~~ MAINT/BUG/DOC: Fix errors in _add_newdocs Jul 3, 2019

mattip closed this Jul 3, 2019

mattip reopened this Jul 3, 2019

mattip merged commit 7ac7fa9 into numpy:master Jul 6, 2019

This was referenced Jul 7, 2019

MAINT,BUG,DOC: Fix errors in _add_newdocs #13932

Merged

MAINT/BUG/DOC: Fix errors in _add_newdocs #13933

Merged

This was referenced Jul 8, 2019

MAINT: Warn if _add_newdocs.py is used to add docstrings to pure-python objects #13935

Merged

Un-squash 13876 #13936

Closed

charris changed the title ~~MAINT/BUG/DOC: Fix errors in _add_newdocs~~ MAINT,BUG,DOC: Fix errors in _add_newdocs Jul 8, 2019

charris added component: numpy._core 04 - Documentation and removed component: documentation 09 - Backport-Candidate PRs tagged should be backported labels Jul 8, 2019

eric-wieser added a commit that referenced this pull request Jul 9, 2019

Revert "MAINT/BUG/DOC: Fix errors in _add_newdocs (#13876)"

42a086d

This reverts commit 7ac7fa9.

eric-wieser mentioned this pull request Jul 9, 2019

MAINT: Revert #13876, "MAINT,BUG,DOC: Fix errors in _add_newdocs" #13943

Merged

mattip added a commit that referenced this pull request Jul 9, 2019

Merge pull request #13943 from numpy/revert-13876-forbid-_add_newdocs…

3085cb7

…-python MAINT: Revert #13876, "MAINT,BUG,DOC: Fix errors in _add_newdocs"

eric-wieser mentioned this pull request Jul 9, 2019

MAINT,BUG,DOC: Fix errors in _add_newdocs #13944

Merged

eric-wieser mentioned this pull request Feb 5, 2020

API: _get_ndarray_c_version imported but not added to __all__ #15518

Closed

Uh oh!

MAINT,BUG,DOC: Fix errors in _add_newdocs #13876

MAINT,BUG,DOC: Fix errors in _add_newdocs #13876

Uh oh!

Conversation

eric-wieser commented Jun 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

charris commented Jul 1, 2019

Uh oh!

eric-wieser commented Jul 2, 2019

Uh oh!

charris commented Jul 2, 2019

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eric-wieser Jul 2, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eric-wieser commented Jul 3, 2019

Uh oh!

eric-wieser commented Jul 3, 2019

Uh oh!

mattip commented Jul 3, 2019

Uh oh!

tylerjereddy commented Jul 6, 2019

Uh oh!

mattip commented Jul 6, 2019

Uh oh!

eric-wieser commented Jul 7, 2019

Uh oh!

eric-wieser commented Jul 8, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PerryGrossman commented Sep 1, 2019

Uh oh!

mattip commented Sep 1, 2019

Uh oh!

Uh oh!

eric-wieser commented Jun 30, 2019 •

edited

Loading

eric-wieser Jul 2, 2019 •

edited

Loading

eric-wieser commented Jul 8, 2019 •

edited

Loading