compare_chararrays now has a docstring, whether or not it shows up in the documentation I don't know.

@charris Yes, to be clear most of these have docstrings -- they just don't appear in the online documentation generated by Sphinx.

Related: #10106

Hi,
I would like to work on this ticket.

@jicksy let us know if you need help getting started: what you tried and where you get stuck.

To make this ticket actionable it needs some decisions. Here is the easiest part, the functions that definitely need to be documented (i.e. review/improve docstring, and add to reference guide):

    'deprecate': 'numpy.lib.utils.deprecate',
    'deprecate_with_doc': 'numpy.lib.utils.<lambda>',
    'get_include': 'numpy.lib.utils.get_include',
    'show_config': 'numpy.__config__.show',
    'who': 'numpy.lib.utils.who',

@jicksy I would suggest to start with those.

Hi, I'd like to work on this!

Some more proposed decisions:

Deprecate in numpy namespace because is available in np.testing or np.lib:

   'Tester': 'numpy.testing._private.nosetester.NoseTester',

Document and add to reference guide:

        'compare_chararrays': 'numpy.core._multiarray_umath.compare_chararrays',

Deprecate completely (trivial alias with one keyword):

        'ndfromtxt': 'numpy.lib.npyio.ndfromtxt',
        'mafromtxt': 'numpy.lib.npyio.mafromtxt',

Deprecate because it was never meant to be public:

        'int_asbuffer': 'numpy.core._multiarray_umath.int_asbuffer',
        'safe_eval': 'numpy.lib.utils.safe_eval'

@smellslikekeenspirit please go for it! I suggest to comment here what part you are working on, because there seem to be multiple people interested.

Hello! @rgommers I am starting with the first few functions that you mentioned. It is my very first time contributing to a complex project or even to documentation at this scale, so I am having a hard time locating the right document.
#10106 mentions that the new documentation should be added to numpy/add_newdocs.py but I couldn't find this directory.
What I found was numpy/core/_add_newdocs.py Am I on the right path?

This is different than #10106. Many of these already have docstrings, but they do not appear in the rendered documents. For instance, `compare_chararrays' has a docstring, but searching for it in the docs comes up empty. One choice would be to add it somewhere, perhaps in Miscellaneous routines, another would be to deprecate it. If you wish to pursue this, you should start with one of the items on the list and issue a PR with that fix. Then add a checklist to the PR with the other items you wish to handle so we can monitor progress. You may choose to leave the harder ones for another PR.

I'd think compare_chararrays goes right under chararray in http://www.numpy.org/devdocs/reference/routines.char.html#convenience-class. To get it there, add it to the listing in /doc/source/reference/routines.char.rst.

Ok, understood. I'll keep both of you posted here

@rgommers I did as you said. I have a couple more questions:

1. As I understand it, listing compare_chararrays should automatically reference it to the Sphinx document. Is that what Sphinx does?
   For background: I couldn't check the results locally after I made the addition on my repo, so I used the inspect-element feature on the devdocs to understand what was happening. It seems an HTML row is added for every addition to the lists. That is not to be done manually, am I right?

2. Should I issue a PR from my feature branch to the numpy/master?

As I understand it, listing compare_chararrays should automatically reference it to the Sphinx document. Is that what Sphinx does?

Yes, Sphinx translates rst documentation file into html (or pdf) content, and the Sphinx autosummary extension basically takes care of seeing that function name in the routine listing, extracting the docstring of that named function, then rendering it as html including the appropriate links.

Should I issue a PR from my feature branch to the numpy/master?

Indeed

I couldn't check the results locally

This first PR will be quite straightforward so it's probably first time right. However if you're going to try more involved changes, you really should try to get make html to work; waiting for the CI system each time to check something isn't workable.

@rgommers I am starting on removing the deprecated parts as suggested by @seberg as I see Chelwin has started on adding listing . As you wrote in one of the comments above, these four functions , ndfromtxt, mafromtxt, int_asbuffer, safe_eval, need to be deprecated. Could you give me some pointers on what content exactly needs to be removed, apart from their listing in doc/source/user/basics.io.genfromtxt.rst ? Also let me know if I am getting this wrong. Thanks a lot!

xref #9312 which began documenting show_config

@kritisingh1 your doc changes so far look fine. Deprecating functionality is a bit more involved than editing docs, you need to:

• issue a DeprecationWarning when the function is called
• ensure that that warning is silenced when running the tests for that function
• edit the function docstring to note that it is deprecated (can possibly be done with np.deprecate or np.deprecate_with_doc, or manually as in the PR linked below).

This is a good example of deprecating a function: https://github.com/numpy/numpy/pull/12244/files

@rgommers what are we planning to replace the functionalities which ndfromtxt and mafromtxt provide with? Or we are just deprecating the functionalities completely? Since you mentioned above that they are trivial alias with one keyword, there should be an alternative way to achieve the tasks performed by those functions.

@kritisingh1 you can look at the implementation of those functions for the answer. They are both calling genfromtxt with the usemask keyword. ndfromtxt is even setting usemask to False which is already the genfromtxt default. So it's identical to genfromtxt. mafromtxt is genfromtxt(..., usemask=False).

These functions look like internal helper functions, and in my opinion, do not need to be documented/ exposed to public. This issue is a couple PRs far from being closed so a general agreement would be a great help.
'_add_newdoc_ufunc': 'numpy.core._multiarray_umath._add_newdoc_ufunc', 'add_docstring': 'numpy.core._multiarray_umath.add_docstring', 'add_newdoc': 'numpy.core.function_base.add_newdoc', 'add_newdoc_ufunc': 'numpy.core._multiarray_umath._add_newdoc_ufunc'

agreed, no need to document them

@kritisingh1 could you comment on whether we can close #13214, and add a checklist here of what if anything remains to be done?

agreed, no need to document them

add_newdoc is used extensively twice by scipy: https://github.com/scipy/scipy/search?q=add_newdoc&unscoped_q=add_newdoc

I think perhaps we should consider it public API.

Yes we can close #13214
As for the checklist, the following functions need to be documented /deprecated : -

• Tester
• FastCopyAndTranspose
• get_array_wrap
• int_asbuffer
• maximum_sctype
• recfromtxt
• recfromcsv
• safe_eval
• set_string_function
  And in a separate PR, we can discuss about what to do with show_config since that function triggered a lot of discussion on how do we want to go about it. xref DOC: show_config documentation #9312

agreed, no need to document them

add_newdoc is used extensively twice by scipy: https://github.com/scipy/scipy/search?q=add_newdoc&unscoped_q=add_newdoc

I think perhaps we should consider it public API.

Good catch, thanks. I just remembered SciPy had its own, but we use the NumPy one as well. So yes, public it is ....

It would be nice to improve the add_newdoc docstring a little if we're adding it to the refguide.

It would be nice to improve...

Done in gh-13946

Would it be possible to remove Tester, it imports unittest and tempfile which contribute to 20% of the import time.

Are we sure that it's actually 20%? Remember the transitive import times might just move under a different module.

It seems to me that we could hide those imports within Tester itself, without answering the question of when to remove it

Yeah. I've been benchmarking import times. Nothing else imports unit test

I can try to see if we can lazy import those two, but looking at the comment, it seems innocent enough to remove all together

I can try to see if we can lazy import those two, but looking at the comment, it seems innocent enough to remove all together

Removing Tester does nothing I think unless you also remove importing numpy.testing. which implies removing numpy.test, numpy.linalg.test, etc. Which doesn't sound too appealing.

@hmaarrfk there are existing issues on import time and lazy importing, please continue there rather than mix in the discussion here.

Back in #12385 (comment), @rgommers (responding to a suggestion from @eric-wieser) said that add_newdoc would remain public. Does that decision still stand? SciPy CI jobs running Python 3.12 and NumPy main dev branch are failing because SciPy uses from numpy.lib import add_newdoc.

@rgommers, sorry for the spam pings if that's a known problem with a fix in progress. I haven't followed all private/public decisions for 2.0 very closely.

Thanks for the pings @WarrenWeckesser. It's being fixed up right now, and looks like it'll be staying in the numpy.lib namespace. See gh-24507 for the full contents of the numpy.lib redesign.

I think we can close this issue. Most objects have been cleaned up now in the main namespace. Here is the status:

• staying and have been documented: get_include and show_config
• still available but to be cleaned up: compare_chararrays, get_array_wrap (these are tracked as part of ENH: Overhaul of NumPy main namespace [NEP 52] #24306)
• removed from the main namespace: everything else

A few functions (e.g., byte_bounds) will be kept but in a namespace in/under numpy.lib; the attribute error message from trying to access it in the main namespace will tell the user where it moved to.

Thanks everyone!