Hi @rgommers!

I pushed another batch of changes addressing review comments. Currently, in this PR:

• Removed from ... import * in the main namespace
• Made submodules being imported lazily
• Removed a set of names from the top namespace listed in the “remove” section in ENH: Overhaul of NumPy main namespace [NEP 52] #24306 (comment).

[EDIT 9.08.2023] Changelog is reported in PR's description/top message

I've got one question: These are used quite a bit in the codebase, what should I use in place of NZERO and PZERO? Just 0.0 and -0.0?

I'd probably change the usages in the tests to import these constants from numpy.core.umath. They probably don't need explicit docs anymore now that they're not public, so you could remove the entries for them from add_newdoc.py.

Great to see import * starting to go away! 🚀

Hi @rgommers @ngoldbaum,

This PR is ready for a review - some CI staged failed due to the fact that Pandas uses np.NINF, which was removed here. As before, I will propagate these changes in separate PRs in other related libraries.

This work's primary purpose is to remove * imports and clean a few names that are surely being removed from the top namespace. Next stage of cleanup will be in a separate PR.

Tests are failing because pandas fails to import in the doctests build. Will be fixed once https://github.com/pandas-dev/pandas/pull/54468/files is merged and a new pandas wheel gets uploaded.

Tests are failing because pandas fails to import in the doctests build. Will be fixed once https://github.com/pandas-dev/pandas/pull/54468/files is merged and a new pandas wheel gets uploaded.

Looks like we're in luck; pandas uploads nightlies frequently and it's already fixed, the job passed.

I'm not sure about the use of globals(), it isn't great indeed. But it's a little tricky to figure out what to replace it with. We can punt on that for now I'd say.

The one thing I did notice is that the list of public symbols still isn't 100% correct for submodule names. From https://numpy.org/neps/nep-0052-python-api-cleanup.html#cleaning-up-the-submodule-structure I think __dir__ and __all__ should contain all the namespaces that are listed as "regular", "special-purpose" and "legacy", and the ones listed for "removal" can be left out (or included now and removed later). And all the other ones can be lazy-loaded.

The one thing I did notice is that the list of public symbols still isn't 100% correct for submodule names. From https://numpy.org/neps/nep-0052-python-api-cleanup.html#cleaning-up-the-submodule-structure I think __dir__ and __all__ should contain all the namespaces that are listed as "regular", "special-purpose" and "legacy", and the ones listed for "removal" can be left out (or included now and removed later). And all the other ones can be lazy-loaded.

@rgommers In this PR I wanted to mostly focus on removing from ... import *. The PR with Part 3 that I'm working on I cover exact refactor of main namespace (following NEP), but I can include all target submodules here already. Should I move it here?

This looks good to me now:

>>> l = np.__dir__()
>>> l2 = [str(s) for s in np.load('../bldnumpy/main.npy')]  # __dir__ from current `main`
>>> set(l2) - set(l)
{'PZERO', 'add_newdoc', 'MAY_SHARE_EXACT', 'ALLOW_THREADS', 'UFUNC_BUFSIZE_DEFAULT', 'CLIP', 'RAISE', 'NAN', 'UFUNC_PYVALS_NAME', 'NINF', 'FPE_OVERFLOW', 'version', 'FPE_INVALID', 'FPE_UNDERFLOW', '_UFUNC_API', 'MAXDIMS', 'WRAP', 'MAY_SHARE_BOUNDS', 'add_docstring', 'NZERO', 'add_newdoc_ufunc', 'FPE_DIVIDEBYZERO', 'PINF', '_add_newdoc_ufunc', 'FLOATING_POINT_SUPPORT', 'BUFSIZE'}
>>> set(l) - set(l2)
{'f2py', 'array_api', 'typing', 'distutils', '__numpy_submodules__'}

>>> {s for s in set(np.__dir__()) - set(np.__all__)}
{'__getattr__', '__package__', '_NoValue', '_pyinstaller_hooks_dir', '__builtins__', '__spec__', '_typing', '__loader__', '__git_version__', '__config__', '__all__', '_globals', '__path__', '__doc__', '__NUMPY_SETUP__', '__dir__', '_distributor_init', '__file__', '_pytesttester', '_version', '__future_scalars__', '__cached__', '_utils', '_type_info', '_int_extended_msg', '__name__', '__former_attrs__', '_CopyMode', '_msg', '__numpy_submodules__', '_mat', '_specific_msg', '_version_meson'}

>>> {s for s in set(np.__dir__()) - set(np.__all__) if not s.startswith('_')}
set()
>>> {s for s in set(np.__all__) - set(np.__dir__()) if not s.startswith('_')}
set()

@rgommers One last thing - I double checked other libraries that are impacted and I noticed that SciPy uses from numpy.lib import add_newdoc, and I removed add_newdoc also from lib here, but shouldn't - let me revert that change.

Thanks Mateusz! I know this is fiddly, but in the long run it'll be worth it to have PRs that are independent and scoped to a well-defined purpose.

I'll look at merging this once CI is happy.

Thanks Mateusz! I know this is fiddly, but in the long run it'll be worth it to have PRs that are independent and scoped to a well-defined purpose.

I'll look at merging this once CI is happy.

@rgommers Looks that CI completed. I did one more round checking usages of removed items in other libraries and found one in SciPy that I missed: scipy/scipy#19045

In it went - thanks again Mateusz!