Add `Final` for stdlib constants #12318

max-muoto · 2024-07-11T04:35:24Z

Use the following script: https://gist.github.com/max-muoto/3bdfd495d4ee48dba99c16513e855983 to add Final to all module-level standard library constants. (Used libcst as opposed to the native ast module, since it was the easiest way to preserve comments/formatting details across all modules)

Technically this script makes MY_CONSTANT: Literal[1] into MY_CONSTANT: Final[Literal[1]], but I ran the Ruff rule redundant-final-literal to fix cases of this.

Otherwise:

MY_CONSTANT = 1
MY_CONSTANT_2: int = 1

# Becomes:
MY_CONSTANT: Final = 1
MY_CONSTANT_2: Final[int] = 1

With TypeVar, ParamSpec, TypeVarTuple, and NewType excluded.

Additionally, ths this excludes the wintypes.pyi module (see commit here), adding final leads Pyright to raise a Variable not allowed in type expression warning. I'll dig a bit deeper post this going out, and possibly put up an issue on Pyright's end, since this initially seems like a bug to me.

The other commits on this PR are mostly fixing things like _TypeVar that my script didn't account, and removing an extra import of Final in typing.pyi and typing_extensions.pyi.

max-muoto · 2024-07-11T05:00:31Z

I think this uncovered a PyType bug, it choked on this (I think believing it was associated with the RegexFlag enum). Reverted the change for this one line: 76720ec

Edit: Same issue here.

JelleZijlstra · 2024-07-11T05:09:02Z

We probably can't merge this with all the mysterious new error: _T? not callable. I'm not sure what's causing that, maybe the new imports messed up mypy's handling of the builtin import cycle.

max-muoto · 2024-07-11T05:10:29Z

We probably can't merge this with all the mysterious new error: _T? not callable. I'm not sure what's causing that, maybe the new imports messed up mypy's handling of the builtin import cycle.

That's from an older run, it's not occurring anymore if you look at the latest mypy primer. (I accidentally had some Final = _TypeVar cases that I had to fix manually, my script only accounted for typing.TypeVar and TypeVar)

max-muoto · 2024-07-11T05:29:39Z

Okay, consolidated the commits fixing PyType related stuff: abb0a27

I there's two issues here (the enum one mentioned above) and another with callables, I'll report both, and once they're fixed this commit can be unwinded.

…sion)

max-muoto

For whatever reason one of the Literal[X] -> CONSTANT: Final = X fixed this issue: c2be29c

github-actions · 2024-07-11T06:42:16Z

Diff from mypy_primer, showing the effect of this PR on open source code:

sphinx (https://github.com/sphinx-doc/sphinx)
+ sphinx/ext/autodoc/importer.py: note: In function "import_object":
+ sphinx/ext/autodoc/importer.py:183:25: error: Cannot assign to final name "TYPE_CHECKING"  [misc]
+ sphinx/ext/autodoc/importer.py:189:25: error: Cannot assign to final name "TYPE_CHECKING"  [misc]

urllib3 (https://github.com/urllib3/urllib3)
+ src/urllib3/util/ssl_.py:136: error: Cannot assign to final name "OP_NO_COMPRESSION"  [misc]
+ src/urllib3/util/ssl_.py:136: note: Error code "misc" not covered by "type: ignore" comment
+ src/urllib3/util/ssl_.py:137: error: Cannot assign to final name "OP_NO_TICKET"  [misc]
+ src/urllib3/util/ssl_.py:137: note: Error code "misc" not covered by "type: ignore" comment
+ src/urllib3/util/ssl_.py:140: error: Cannot assign to final name "PROTOCOL_TLS"  [misc]
+ src/urllib3/util/ssl_.py:140: note: Error code "misc" not covered by "type: ignore" comment
+ src/urllib3/util/ssl_.py:141: error: Cannot assign to final name "PROTOCOL_TLS_CLIENT"  [misc]
+ src/urllib3/util/ssl_.py:141: note: Error code "misc" not covered by "type: ignore" comment

mkdocs (https://github.com/mkdocs/mkdocs)
+ mkdocs/tests/__init__.py:4: error: Cannot assign to final name "_MAX_LENGTH"  [misc]

setuptools (https://github.com/pypa/setuptools)
+ setuptools/tests/contexts.py:74: error: Cannot assign to final name "ENABLE_USER_SITE"  [misc]

AlexWaygood · 2024-07-11T09:04:43Z

It might be good to split this into two PRs: a first PR that makes changes like this, which are arguably no-brainers; we can merge this first PR quickly and easily:

- FOO: Literal[42]
+ FOO: Final = 42

And a second PR that makes changes like this, which will probably require more careful review from us:

- BAR = 42
+ BAR: Final = 42
- BAZ: int
+ BAZ: Final[int]

max-muoto · 2024-07-11T15:26:51Z

It might be good to split this into two PRs: a first PR that makes changes like this, which are arguably no-brainers; we can merge this first PR quickly and easily:
- FOO: Literal[42]
+ FOO: Final = 42
And a second PR that makes changes like this, which will probably require more careful review from us:
- BAR = 42
+ BAR: Final = 42
- BAZ: int
+ BAZ: Final[int]

Sounds good - let me split out that first change.

max-muoto and others added 5 commits July 10, 2024 23:35

Add final for stdlib constants

c7a3f13

[pre-commit.ci] auto fixes from pre-commit.com hooks

e5ca0bc

Exclude wintypes

d188634

Remove extra Final import

b37c63e

remove script

9be4c88

This comment has been minimized.

Sign in to view

max-muoto and others added 4 commits July 10, 2024 23:49

Edge case fixes

5f351b7

Remove final import

e50791a

Another edge case

5c071a0

[pre-commit.ci] auto fixes from pre-commit.com hooks

08b98f3

max-muoto marked this pull request as ready for review July 11, 2024 04:56

PyType bug?

76720ec

max-muoto mentioned this pull request Jul 11, 2024

Consistency with Final usage in Standard Library Stubs #12311

Open

This comment has been minimized.

Sign in to view

Resolve PyType issues

abb0a27

max-muoto force-pushed the final-for-constants branch from 6e1f5a1 to abb0a27 Compare July 11, 2024 05:28

This comment has been minimized.

Sign in to view

max-muoto and others added 8 commits July 11, 2024 01:10

Fix Literal[x] -> Final = x, and nested cases

c9cf0fe

[pre-commit.ci] auto fixes from pre-commit.com hooks

307415d

Bring back fixes

efbcdc3

Resolve PyType issues

9805df5

fix

2cda2bf

remove erroneous imports

be9d441

Ruff didn't catch negative? (probably due to it being on an older ver…

492f8b5

…sion)

[pre-commit.ci] auto fixes from pre-commit.com hooks

f5172d0

max-muoto and others added 7 commits July 11, 2024 01:16

Another negative

77c56ef

[pre-commit.ci] auto fixes from pre-commit.com hooks

759b654

Exclude wintypes

41ea404

_Param

2ca6bc2

invalid syntax

daf66af

Unused

c2be29c

pytpe issue

80c5e6f

max-muoto commented Jul 11, 2024

View reviewed changes

max-muoto mentioned this pull request Jul 13, 2024

Use Final for Constant Literals in the stdlib #12332

Merged

max-muoto closed this Sep 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `Final` for stdlib constants #12318

Add `Final` for stdlib constants #12318

max-muoto commented Jul 11, 2024 •

edited

Loading

This comment has been minimized.

max-muoto commented Jul 11, 2024 •

edited

Loading

JelleZijlstra commented Jul 11, 2024

This comment has been minimized.

max-muoto commented Jul 11, 2024 •

edited

Loading

This comment has been minimized.

max-muoto commented Jul 11, 2024 •

edited

Loading

This comment has been minimized.

max-muoto left a comment •

edited

Loading

github-actions bot commented Jul 11, 2024

AlexWaygood commented Jul 11, 2024

max-muoto commented Jul 11, 2024

Add Final for stdlib constants #12318

Add Final for stdlib constants #12318

Conversation

max-muoto commented Jul 11, 2024 • edited Loading

This comment has been minimized.

max-muoto commented Jul 11, 2024 • edited Loading

JelleZijlstra commented Jul 11, 2024

This comment has been minimized.

max-muoto commented Jul 11, 2024 • edited Loading

This comment has been minimized.

max-muoto commented Jul 11, 2024 • edited Loading

This comment has been minimized.

max-muoto left a comment • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Jul 11, 2024

AlexWaygood commented Jul 11, 2024

max-muoto commented Jul 11, 2024

Add `Final` for stdlib constants #12318

Add `Final` for stdlib constants #12318

max-muoto commented Jul 11, 2024 •

edited

Loading

max-muoto commented Jul 11, 2024 •

edited

Loading

max-muoto commented Jul 11, 2024 •

edited

Loading

max-muoto commented Jul 11, 2024 •

edited

Loading

max-muoto left a comment •

edited

Loading