experiment: handle `hash = None` in somewhat more intuitive fashion #19168

sterliakov · 2025-05-29T16:37:44Z

This is the first step towards checking hashability of dict keys (and Hashable compatibility in general). This PR implements the following rules:

__hash__ = None is only allowed on classes who inherited __hash__ directly from object (no redefinitions in MRO)
def __hash__ is allowed on subclasses whose parents declare __hash__ = None - it is a relatively common solution to derive a hashable subclass, and LSP violation only means assert SomeCls.__hash__ is None will not pass for subclasses. This sounds safe enough.
Classes defining __eq__ without __hash__ get their __hash__ assigned to None to match runtime semantics. Protocols are exempt from this as defining __eq__ on protocol just requires classes that support __eq__, not necessarily unhashable ones.

Refs #19043.

…k in subclasses, only allowed to override `object.__hash__`

…-dunder-hash-consistently

github-actions · 2025-06-03T20:46:35Z

Diff from mypy_primer, showing the effect of this PR on open source code:

build (https://github.com/pypa/build): 2.29x faster (28.7s -> 12.5s in single noisy sample)

typeshed-stats (https://github.com/AlexWaygood/typeshed-stats): 1.07x slower (97.1s -> 103.8s in single noisy sample)

prefect (https://github.com/PrefectHQ/prefect): 1.09x slower (87.9s -> 95.9s in single noisy sample)
+ src/prefect/_internal/concurrency/calls.py:273: error: Unused "type: ignore" comment  [unused-ignore]

mkdocs (https://github.com/mkdocs/mkdocs)
+ mkdocs/utils/templates.py:47: error: Subclass of "ExtraScriptValue" and "str" cannot exist: would have incompatible method signatures  [unreachable]

mongo-python-driver (https://github.com/mongodb/mongo-python-driver)
+ bson/regex.py:112: error: Unused "type: ignore" comment  [unused-ignore]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/arrays/base.py:2207: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/generic.py:1868: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/indexes/base.py:5209: error: Unused "type: ignore" comment  [unused-ignore]

asynq (https://github.com/quora/asynq)
+ asynq/tests/test_tools.py:183: error: Unused "type: ignore" comment  [unused-ignore]

schema_salad (https://github.com/common-workflow-language/schema_salad)
+ schema_salad/jsonld_context.py: note: In function "pred":
+ schema_salad/jsonld_context.py:53:31: error: Subclass of "str" and "MutableMapping[Any, Any]" cannot exist: would have incompatible method signatures  [unreachable]
+ schema_salad/jsonld_context.py:54:21: error: Statement is unreachable  [unreachable]
+ schema_salad/schema.py: note: In function "get_anon_name":
+ schema_salad/schema.py:422:27: error: Subclass of "str" and "Mapping[Any, Any]" cannot exist: would have incompatible method signatures  [unreachable]
+ schema_salad/schema.py:423:17: error: Statement is unreachable  [unreachable]

jax (https://github.com/google/jax)
+ jax/_src/core.py:523: error: Unused "type: ignore" comment  [unused-ignore]
+ jax/_src/core.py:811: error: Unused "type: ignore" comment  [unused-ignore]
+ jax/_src/earray.py:33: error: Unused "type: ignore" comment  [unused-ignore]
+ jax/_src/prng.py:300: error: Unused "type: ignore" comment  [unused-ignore]
+ jax/experimental/sparse/_base.py:34: error: Unused "type: ignore" comment  [unused-ignore]

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/python_api.py:88: error: Unused "type: ignore" comment  [unused-ignore]
+ src/_pytest/_code/source.py:50: error: Unused "type: ignore" comment  [unused-ignore]
+ src/_pytest/_code/code.py:77: error: Unused "type: ignore" comment  [unused-ignore]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/datastructures/structures.py:648: error: Unused "type: ignore" comment  [unused-ignore]
+ src/werkzeug/datastructures/headers.py:108: error: Unused "type: ignore" comment  [unused-ignore]
+ src/werkzeug/routing/rules.py:912: error: Unused "type: ignore" comment  [unused-ignore]

static-frame (https://github.com/static-frame/static-frame)
+ static_frame/core/series.py:3170: error: Incompatible return value type (got "IndexBase", expected "TLabel | ndarray[Any, Any]")  [return-value]
+ static_frame/core/frame.py:6688: error: Argument 1 to "index_from_optional_constructors" has incompatible type "TypeBlocks"; expected "TIndexInitializer"  [arg-type]
+ static_frame/core/frame.py:6688: note: Following member(s) of "TypeBlocks" have conflicts:
+ static_frame/core/frame.py:6688: note:     Expected:
+ static_frame/core/frame.py:6688: note:         def __iter__(self) -> Iterator[TLabel]
+ static_frame/core/frame.py:6688: note:     Got:
+ static_frame/core/frame.py:6688: note:         def __iter__(self) -> Iterator[TypeBlocks]
+ static_frame/core/frame.py:6688: note:     Expected:
+ static_frame/core/frame.py:6688: note:         def __iter__(self) -> Iterator[Sequence[TLabel]]
+ static_frame/core/frame.py:6688: note:     Got:
+ static_frame/core/frame.py:6688: note:         def __iter__(self) -> Iterator[TypeBlocks]
+ static_frame/core/frame.py:6688: note:     Expected:
+ static_frame/core/frame.py:6688: note:         def __iter__(self) -> Iterator[TLabel]
+ static_frame/core/frame.py:6688: note:     Got:
+ static_frame/core/frame.py:6688: note:         def __iter__(self) -> Iterator[TypeBlocks]
+ static_frame/core/frame.py:6688: note:     Expected:
+ static_frame/core/frame.py:6688: note:         def __iter__(self) -> Iterator[Sequence[TLabel]]
+ static_frame/core/frame.py:6688: note:     Got:
+ static_frame/core/frame.py:6688: note:         def __iter__(self) -> Iterator[TypeBlocks]
+ static_frame/core/frame.py:6914: error: Argument 1 to "index_from_optional_constructors" has incompatible type "TypeBlocks"; expected "TIndexInitializer"  [arg-type]
+ static_frame/core/frame.py:6914: note: Following member(s) of "TypeBlocks" have conflicts:
+ static_frame/core/frame.py:6914: note:     Expected:
+ static_frame/core/frame.py:6914: note:         def __iter__(self) -> Iterator[TLabel]
+ static_frame/core/frame.py:6914: note:     Got:
+ static_frame/core/frame.py:6914: note:         def __iter__(self) -> Iterator[TypeBlocks]
+ static_frame/core/frame.py:6914: note:     Expected:
+ static_frame/core/frame.py:6914: note:         def __iter__(self) -> Iterator[Sequence[TLabel]]
+ static_frame/core/frame.py:6914: note:     Got:
+ static_frame/core/frame.py:6914: note:         def __iter__(self) -> Iterator[TypeBlocks]
+ static_frame/core/frame.py:6914: note:     Expected:
+ static_frame/core/frame.py:6914: note:         def __iter__(self) -> Iterator[TLabel]
+ static_frame/core/frame.py:6914: note:     Got:
+ static_frame/core/frame.py:6914: note:         def __iter__(self) -> Iterator[TypeBlocks]
+ static_frame/core/frame.py:6914: note:     Expected:
+ static_frame/core/frame.py:6914: note:         def __iter__(self) -> Iterator[Sequence[TLabel]]
+ static_frame/core/frame.py:6914: note:     Got:
+ static_frame/core/frame.py:6914: note:         def __iter__(self) -> Iterator[TypeBlocks]

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ pandas-stubs/core/generic.pyi:67: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas-stubs/core/frame.pyi:312: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas-stubs/core/indexes/base.pyi:72: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas-stubs/core/groupby/generic.pyi:167: error: Return type "Iterator[tuple[ByT, Series[S1]]]" of "__iter__" incompatible with return type "Iterator[tuple[Hashable, Series[S1]]]" in supertype "BaseGroupBy"  [override]
+ pandas-stubs/core/groupby/generic.pyi:378: error: Return type "Iterator[tuple[ByT, DataFrame]]" of "__iter__" incompatible with return type "Iterator[tuple[Hashable, DataFrame]]" in supertype "BaseGroupBy"  [override]

beartype (https://github.com/beartype/beartype): 1.54x faster (17.3s -> 11.3s in single noisy sample)
+ beartype/_check/metadata/metadecor.py:249: error: Unused "type: ignore" comment  [unused-ignore]
+ beartype/_check/metadata/metacheck.py:97: error: Unused "type: ignore" comment  [unused-ignore]

xarray (https://github.com/pydata/xarray)
+ xarray/core/_typed_ops.py:93: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/_typed_ops.py:362: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/_typed_ops.py:736: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/_typed_ops.py:1164: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/_typed_ops.py:1376: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/_typed_ops.py:1502: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/dataset.py:1430: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/dataset.py: note: In class "Dataset":
+ xarray/core/dataset.py:1430: error: Cannot override instance variable (previously declared on base class "DatasetOpsMixin") with class variable  [misc]
+ xarray/core/dataset.py:1430: note: Error code "misc" not covered by "type: ignore" comment
+ xarray/core/dataset.py: note: At top level:
+ xarray/core/dataarray.py:1312: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/dataarray.py: note: In member "rename" of class "DataArray":
+ xarray/core/dataarray.py:2549: error: Argument "name" to "_replace" of "DataArray" has incompatible type "Hashable | Mapping[Any, Hashable]"; expected "Hashable | Default | None"  [arg-type]
+ xarray/core/dataarray.py: note: At top level:
+ xarray/core/datatree.py:301: error: Unused "type: ignore" comment  [unused-ignore]

sterliakov added 2 commits May 29, 2025 18:13

Make behaviour of __hash__ = None more intuitive: can be rolled bac…

25d10b1

…k in subclasses, only allowed to override `object.__hash__`

Missing get_proper_type

a17a313

This comment has been minimized.

Sign in to view

sterliakov added 3 commits June 1, 2025 16:31

Allow any such overrides

31a5c2a

Fix stub for mypyc test

22a2729

Adjust remaining test, replace ellipsis with pass for consistency

eccc702

This comment has been minimized.

Sign in to view

Do not add implicit __hash__ = None to protocols

fad22f2

This comment has been minimized.

Sign in to view

sterliakov added 3 commits June 3, 2025 20:53

Merge remote-tracking branch 'upstream/master' into feature/st-handle…

320effb

…-dunder-hash-consistently

Support in assignments too

bd953b4

Handle allow_incompatible_override consistenly instead

18815c7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

experiment: handle `hash = None` in somewhat more intuitive fashion #19168

experiment: handle `hash = None` in somewhat more intuitive fashion #19168

Uh oh!

sterliakov commented May 29, 2025 •

edited

Loading

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Jun 3, 2025

Uh oh!

Uh oh!

Uh oh!

experiment: handle __hash__ = None in somewhat more intuitive fashion #19168

Are you sure you want to change the base?

experiment: handle __hash__ = None in somewhat more intuitive fashion #19168

Uh oh!

Conversation

sterliakov commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Jun 3, 2025

Uh oh!

Uh oh!

experiment: handle `hash = None` in somewhat more intuitive fashion #19168

experiment: handle `hash = None` in somewhat more intuitive fashion #19168

sterliakov commented May 29, 2025 •

edited

Loading