Thanks to visit codestin.com
Credit goes to github.com

Skip to content

bpo-42133: update parts of the stdlib to fall back to __spec__.loader when __loader__ is missing #22929

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 7, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions Doc/whatsnew/3.10.rst
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,12 @@ by :func:`curses.color_content`, :func:`curses.init_color`,
support is provided by the underlying ncurses library.
(Contributed by Jeffrey Kintscher and Hans Petter Jansson in :issue:`36982`.)

doctest
-------

When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
(Contributed by Brett Cannon in :issue:`42133`.)

encodings
---------
:func:`encodings.normalize_encoding` now ignores non-ASCII characters.
Expand All @@ -198,6 +204,18 @@ Added the *root_dir* and *dir_fd* parameters in :func:`~glob.glob` and
:func:`~glob.iglob` which allow to specify the root directory for searching.
(Contributed by Serhiy Storchaka in :issue:`38144`.)

inspect
-------

When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
(Contributed by Brett Cannon in :issue:`42133`.)

linecache
---------

When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
(Contributed by Brett Cannon in :issue:`42133`.)

os
--

Expand All @@ -217,6 +235,12 @@ The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default
instead of :mod:`pickle` protocol ``3`` when creating shelves.
(Contributed by Zackery Spytz in :issue:`34204`.)

site
----

When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
(Contributed by Brett Cannon in :issue:`42133`.)

sys
---

Expand Down
18 changes: 11 additions & 7 deletions Lib/doctest.py
Original file line number Diff line number Diff line change
Expand Up @@ -222,13 +222,17 @@ def _load_testfile(filename, package, module_relative, encoding):
if module_relative:
package = _normalize_module(package, 3)
filename = _module_relative_path(package, filename)
if getattr(package, '__loader__', None) is not None:
if hasattr(package.__loader__, 'get_data'):
file_contents = package.__loader__.get_data(filename)
file_contents = file_contents.decode(encoding)
# get_data() opens files as 'rb', so one must do the equivalent
# conversion as universal newlines would do.
return _newline_convert(file_contents), filename
if (loader := getattr(package, '__loader__', None)) is None:
try:
loader = package.__spec__.loader
except AttributeError:
pass
if hasattr(loader, 'get_data'):
file_contents = loader.get_data(filename)
file_contents = file_contents.decode(encoding)
# get_data() opens files as 'rb', so one must do the equivalent
# conversion as universal newlines would do.
return _newline_convert(file_contents), filename
with open(filename, encoding=encoding) as f:
return f.read(), filename

Expand Down
7 changes: 5 additions & 2 deletions Lib/inspect.py
Original file line number Diff line number Diff line change
Expand Up @@ -707,10 +707,13 @@ def getsourcefile(object):
if os.path.exists(filename):
return filename
# only return a non-existent filename if the module has a PEP 302 loader
if getattr(getmodule(object, filename), '__loader__', None) is not None:
module = getmodule(object, filename)
if getattr(module, '__loader__', None) is not None:
return filename
elif getattr(getattr(module, "__spec__", None), "loader", None) is not None:
return filename
# or it is in the linecache
if filename in linecache.cache:
elif filename in linecache.cache:
return filename

def getabsfile(object, _filename=None):
Expand Down
11 changes: 8 additions & 3 deletions Lib/linecache.py
Original file line number Diff line number Diff line change
Expand Up @@ -165,9 +165,14 @@ def lazycache(filename, module_globals):
if not filename or (filename.startswith('<') and filename.endswith('>')):
return False
# Try for a __loader__, if available
if module_globals and '__loader__' in module_globals:
name = module_globals.get('__name__')
loader = module_globals['__loader__']
if module_globals and '__name__' in module_globals:
name = module_globals['__name__']
if (loader := module_globals.get('__loader__')) is None:
if spec := module_globals.get('__spec__'):
try:
loader = spec.loader
except AttributeError:
pass
get_source = getattr(loader, 'get_source', None)

if name and get_source:
Expand Down
11 changes: 9 additions & 2 deletions Lib/site.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,15 @@ def makepath(*paths):
def abs_paths():
"""Set all module __file__ and __cached__ attributes to an absolute path"""
for m in set(sys.modules.values()):
if (getattr(getattr(m, '__loader__', None), '__module__', None) not in
('_frozen_importlib', '_frozen_importlib_external')):
loader_module = None
try:
loader_module = m.__loader__.__module__
except AttributeError:
try:
loader_module = m.__spec__.loader.__module__
except AttributeError:
pass
if loader_module not in {'_frozen_importlib', '_frozen_importlib_external'}:
continue # don't mess with a PEP 302-supplied __file__
try:
m.__file__ = os.path.abspath(m.__file__)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Update various modules in the stdlib to fall back on `__spec__.loader` when
`__loader__` isn't defined on a module.