Thanks to visit codestin.com
Credit goes to github.com

Skip to content

gh-69605: Check for already imported modules in PyREPL module completion #139461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ambv merged 24 commits into from
Jan 5, 2026
+107 −3
Merged

gh-69605: Check for already imported modules in PyREPL module completion #139461

Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
6ed2776
PyREPL module completion: check for already imported modules
loic-simon Sep 30, 2025
48fd43f
Add blurb
loic-simon Sep 30, 2025
7ac428e
Better convey intent
loic-simon Oct 1, 2025
6515e2f
[TEMP] debug tests on windows using modern technology (print statements)
loic-simon Oct 1, 2025
7dbb906
[TEMP] More debugging, where is my module??
loic-simon Oct 2, 2025
ac3065a
[TEMP] More debugging, where is my module?? (bis)
loic-simon Oct 2, 2025
75a33da
[TEMP] Day 57, deep into debugging, I still don't know where is my mo…
loic-simon Oct 2, 2025
ce124b1
[TEMP] Moar logs
loic-simon Oct 2, 2025
ee7047f
Merge branch 'main' into pyrepl-module-completion-check-for-already-i…
loic-simon Oct 3, 2025
3f362cd
[TEMP] Is it a FileFinder cache issue??
loic-simon Oct 3, 2025
ed8ce73
[TEMP] Looks like a cache issue indeed
loic-simon Oct 3, 2025
19c49bb
Tests: clean FileFinder cache
loic-simon Oct 3, 2025
16e44af
Remove all debugging junk
loic-simon Oct 3, 2025
14f6175
Small if refactor
loic-simon Oct 5, 2025
bdd7bdf
Merge branch 'pyrepl-module-completion-check-for-already-imported-mod…
loic-simon Oct 5, 2025
78e4737
Full test coverage for new code
loic-simon Oct 11, 2025
e3f1ddb
Merge branch 'python:main' into pyrepl-module-completion-check-for-al…
loic-simon Oct 11, 2025
2644400
Merge branch 'main' into pyrepl-module-completion-check-for-already-i…
tomasr8 Dec 28, 2025
5fa70cf
Merge branch 'main' into pyrepl-module-completion-check-for-already-i…
loic-simon Jan 1, 2026
d332e14
Check __spec__.has_location + refactor
loic-simon Jan 1, 2026
1a5327c
Rename private helper
loic-simon Jan 1, 2026
d48f243
Simplify implementation
loic-simon Jan 2, 2026
e235e20
Fix find_spec call
loic-simon Jan 2, 2026
f6757fe
Remove unused import
loic-simon Jan 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions Lib/_pyrepl/_module_completer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,23 @@ def _find_modules(self, path: str, prefix: str) -> list[str]:
if path is None:
return []

modules: Iterable[pkgutil.ModuleInfo] = self.global_cache
modules: Iterable[pkgutil.ModuleInfo]
imported_module = sys.modules.get(path.split('.')[0])
if imported_module:
# Module already imported: only look in its location,
# even if a module with the same name would be higher in path
imported_path = (imported_module.__spec__
and imported_module.__spec__.origin)
if not imported_path:
# Module imported but no spec/origin: propose no suggestions
return []
if os.path.basename(imported_path) == "__init__.py": # package
imported_path = os.path.dirname(imported_path)
import_location = os.path.dirname(imported_path)
Comment thread
loic-simon marked this conversation as resolved.
Outdated
modules = list(pkgutil.iter_modules([import_location]))
Copy link
Copy Markdown
Member

@tomasr8 tomasr8 Dec 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this is correct, it's going to find submodules but we actually want this to be the top-level modules

Copy link
Copy Markdown
Contributor Author

@loic-simon loic-simon Jan 1, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it's OK, because os.path.dirname make us search in the folder containing the module origin, not on the origin itself (that's why we need to do it twice for a package):

>>> import os, pkgutil, typing, concurrent
>>>
>>> typing.__spec__.origin  # single-file module
'<venv>/lib/python3.14/typing.py'
>>> concurrent.__spec__.origin  # package
'<venv>/lib/python3.14/concurrent/__init__.py'
>>>
>>> loc = os.path.dirname(typing.__spec__.origin)  # or do it twice for concurrent
>>> [mod for mod in pkgutil.iter_modules([loc]) if mod.name in ("typing", "concurrent")]
[ModuleInfo(module_finder=FileFinder('<venv>/lib/python3.14'), name='concurrent', ispkg=True), 
ModuleInfo(module_finder=FileFinder('<venv>/lib/python3.14'), name='typing', ispkg=False)]

While refactoring this into a separate function I ended up rewriting the whole thing, it should be more explicit now (and it checks module.__package__ instead of the os.path.basename(imported_path) == "__init__.py" hack)

else:
modules = self.global_cache

is_stdlib_import: bool | None = None
for segment in path.split('.'):
modules = [mod_info for mod_info in modules
Expand Down Expand Up @@ -196,7 +212,6 @@ def global_cache(self) -> list[pkgutil.ModuleInfo]:
"""Global module cache"""
if not self._global_cache or self._curr_sys_path != sys.path:
self._curr_sys_path = sys.path[:]
# print('getting packages')
self._global_cache = list(pkgutil.iter_modules())
return self._global_cache

Expand Down
92 changes: 90 additions & 2 deletions Lib/test/test_pyrepl/test_pyrepl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import itertools
import os
import pathlib
import pkgutil
import re
import rlcompleter
import select
Expand Down Expand Up @@ -1090,17 +1091,104 @@ def test_hardcoded_stdlib_submodules(self):
self.assertEqual(output, expected)

def test_hardcoded_stdlib_submodules_not_proposed_if_local_import(self):
with tempfile.TemporaryDirectory() as _dir:
with (tempfile.TemporaryDirectory() as _dir,
patch.object(sys, "modules", {})): # hide imported module
dir = pathlib.Path(_dir)
(dir / "collections").mkdir()
(dir / "collections" / "__init__.py").touch()
(dir / "collections" / "foo.py").touch()
with patch.object(sys, "path", [dir, *sys.path]):
with patch.object(sys, "path", [_dir, *sys.path]):
events = code_to_events("import collections.\t\n")
reader = self.prepare_reader(events, namespace={})
output = reader.readline()
self.assertEqual(output, "import collections.foo")

def test_already_imported_stdlib_module_no_other_suggestions(self):
with (tempfile.TemporaryDirectory() as _dir,
patch.object(sys, "path", [_dir, *sys.path])):
dir = pathlib.Path(_dir)
(dir / "collections").mkdir()
(dir / "collections" / "__init__.py").touch()
(dir / "collections" / "foo.py").touch()

# collections found in dir, but was already imported
# from stdlib at startup -> suggest stdlib submodules only
events = code_to_events("import collections.\t\n")
reader = self.prepare_reader(events, namespace={})
output = reader.readline()
self.assertEqual(output, "import collections.abc")

def test_already_imported_custom_module_no_other_suggestions(self):
with (tempfile.TemporaryDirectory() as _dir1,
tempfile.TemporaryDirectory() as _dir2,
patch.object(sys, "path", [_dir2, _dir1, *sys.path])):
dir1 = pathlib.Path(_dir1)
(dir1 / "mymodule").mkdir()
(dir1 / "mymodule" / "__init__.py").touch()
(dir1 / "mymodule" / "foo.py").touch()
importlib.import_module("mymodule")

dir2 = pathlib.Path(_dir2)
(dir2 / "mymodule").mkdir()
(dir2 / "mymodule" / "__init__.py").touch()
(dir2 / "mymodule" / "bar.py").touch()
# Purge FileFinder cache after adding files
pkgutil.get_importer(_dir2).invalidate_caches()
# mymodule found in dir2 before dir1, but it was already imported
# from dir1 -> suggest dir1 submodules only
events = code_to_events("import mymodule.\t\n")
reader = self.prepare_reader(events, namespace={})
output = reader.readline()
self.assertEqual(output, "import mymodule.foo")

del sys.modules["mymodule"]
# mymodule not imported anymore -> suggest dir2 submodules
events = code_to_events("import mymodule.\t\n")
reader = self.prepare_reader(events, namespace={})
output = reader.readline()
self.assertEqual(output, "import mymodule.bar")

def test_already_imported_custom_file_no_suggestions(self):
# Same as before, but mymodule from dir1 has no submodules
# -> propose nothing
with (tempfile.TemporaryDirectory() as _dir1,
tempfile.TemporaryDirectory() as _dir2,
patch.object(sys, "path", [_dir2, _dir1, *sys.path])):
dir1 = pathlib.Path(_dir1)
(dir1 / "mymodule").mkdir()
(dir1 / "mymodule.py").touch()
importlib.import_module("mymodule")

dir2 = pathlib.Path(_dir2)
(dir2 / "mymodule").mkdir()
(dir2 / "mymodule" / "__init__.py").touch()
(dir2 / "mymodule" / "bar.py").touch()
events = code_to_events("import mymodule.\t\n")
reader = self.prepare_reader(events, namespace={})
output = reader.readline()
self.assertEqual(output, "import mymodule.")
del sys.modules["mymodule"]

def test_already_imported_module_without_origin_or_spec(self):
with (tempfile.TemporaryDirectory() as _dir1,
patch.object(sys, "path", [_dir1, *sys.path])):
dir1 = pathlib.Path(_dir1)
for mod in ("no_origin", "no_spec"):
(dir1 / mod).mkdir()
(dir1 / mod / "__init__.py").touch()
(dir1 / mod / "foo.py").touch()
module = importlib.import_module(mod)
assert module.__spec__
if mod == "no_origin":
module.__spec__.origin = None
else:
module.__spec__ = None
events = code_to_events(f"import {mod}.\t\n")
reader = self.prepare_reader(events, namespace={})
output = reader.readline()
self.assertEqual(output, f"import {mod}.")
del sys.modules[mod]

def test_get_path_and_prefix(self):
cases = (
('', ('', '')),
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Core_and_Builtins/2025-09-30-21-59-56.gh-issue-69605.qcmGF3.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix edge-cases around already imported modules in the :term:`REPL`
auto-completion of imports.
Loading