Thanks to visit codestin.com
Credit goes to github.com

Skip to content

bpo-41428: Implementation for PEP 604 #21515

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 62 commits into from
Sep 9, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
4c4c701
Add new type object for union types
MaggieMoss Jul 10, 2020
07b5c29
Add test.
MaggieMoss Jul 10, 2020
32c207b
Add basic tp_* fields to union object
MaggieMoss Jul 15, 2020
50b33c2
Squash this commit.
MaggieMoss Jul 15, 2020
8901145
Squash this commit.
MaggieMoss Jul 15, 2020
6450321
Add union_traverse method.
MaggieMoss Jul 15, 2020
8a09a92
Handle Py_None in union.
MaggieMoss Jul 16, 2020
e3c80e9
Clean up syntax with Py_None
MaggieMoss Jul 16, 2020
165c323
Update getattro method.
MaggieMoss Jul 16, 2020
7f0018e
Reduce code duplication when adding union types together.
MaggieMoss Jul 17, 2020
cd584d5
Add __or__ method to _SpecialGenericAlias
MaggieMoss Jul 17, 2020
ba031c1
Add support for NewType with unions.
MaggieMoss Jul 17, 2020
e62de73
Remove old code from abstract.c
MaggieMoss Jul 17, 2020
f6ab05b
Fix warning for incompatible pointer type.
MaggieMoss Jul 17, 2020
9c20fbc
Add support for union use in subclasses.
MaggieMoss Jul 19, 2020
9f48989
Fix code style inconsistencies.
MaggieMoss Jul 22, 2020
a0cd651
Change is_not_union -> is_union
MaggieMoss Jul 22, 2020
a5c6850
Implement dedup and flatten.
MaggieMoss Jul 24, 2020
5255f4f
Add basic tp_repr to unionobject.
MaggieMoss Jul 24, 2020
1732fad
Update is_unionable.
MaggieMoss Jul 24, 2020
31653dc
Clean up whitespace.
MaggieMoss Jul 25, 2020
2ffb050
Update test_types.
MaggieMoss Jul 25, 2020
3fed944
Fix style and grammar issues.
MaggieMoss Jul 27, 2020
a1c703b
Implement NE for union richcompare.
MaggieMoss Jul 27, 2020
c4acc73
Remove is_generic_alias helper method.
MaggieMoss Jul 27, 2020
6a8dfe9
Fix negative ref count.
MaggieMoss Jul 27, 2020
af59751
Small code style fixes.
MaggieMoss Jul 27, 2020
cae1a38
Add unionobject to pythoncore.vcxproj.filters
MaggieMoss Jul 27, 2020
0cb47c1
Remove semicolon after PyObject_HEAD
MaggieMoss Jul 27, 2020
265be78
Use _PyTuple_Resize instead of variable length array.
MaggieMoss Jul 28, 2020
6fddcd6
Add error handling.
MaggieMoss Jul 28, 2020
60f023a
Add additional tests for union_richcompare.
MaggieMoss Jul 28, 2020
99ba1fd
📜🤖 Added by blurb_it.
blurb-it[bot] Jul 28, 2020
ad945fd
Update error check for PySet_New call.
MaggieMoss Jul 30, 2020
0586c41
Allow Any and other _SpecialForm
MaggieMoss Jul 30, 2020
e44a657
Update repr to handle NoneType properly.
MaggieMoss Jul 30, 2020
ba3110c
Add Union to types.py
MaggieMoss Jul 30, 2020
7bec987
Merge branch 'PEP605-implementation' of github.com:MaggieMoss/cpython…
MaggieMoss Jul 30, 2020
e052c53
Remove commented out code.
MaggieMoss Jul 31, 2020
15e0d29
Update test with nested unions.
MaggieMoss Jul 31, 2020
2e31288
Add additional tests for optionals.
MaggieMoss Jul 31, 2020
2a12a43
Add support for int | int == int
MaggieMoss Jul 31, 2020
7116cd9
Add more error handling to unionobject
MaggieMoss Jul 31, 2020
2c0befb
Handle NoneType vs. None
MaggieMoss Aug 3, 2020
8d59e44
Don't allow generics in isinstance calls.
MaggieMoss Aug 4, 2020
9f014e2
Add errors for union subclass.
MaggieMoss Aug 5, 2020
30bb872
Implement code review suggestions.
MaggieMoss Aug 12, 2020
6fe4956
Increase code sharing between is_typing_name and is_new_type
MaggieMoss Aug 12, 2020
bae3d2a
Add tests, fix reference leaks.
MaggieMoss Aug 12, 2020
17a7277
Update tp_flags for unionobject.
MaggieMoss Aug 12, 2020
f47bce0
Return int.
MaggieMoss Aug 13, 2020
0735352
Fix NoneType comparison side effect.
MaggieMoss Aug 13, 2020
b2910f0
Merge branch 'PEP605-implementation' of github.com:MaggieMoss/cpython…
MaggieMoss Aug 14, 2020
e9e5a5e
Remove GC flag from unionobject.
MaggieMoss Aug 14, 2020
385a338
Implement code review suggestions
MaggieMoss Aug 18, 2020
dddc598
Refactor and remove PyUnion_New
MaggieMoss Aug 18, 2020
8c06c1d
General cleanup, fix reference leaks, add new test for possible crashes
pablogsal Sep 8, 2020
37699f1
Add newline
pablogsal Sep 8, 2020
47a4cb3
Make unionobject APIs private
pablogsal Sep 9, 2020
7f89abd
Add minor test for GenericAlias
pablogsal Sep 9, 2020
d0be56f
Remove ellipsis from repr.
MaggieMoss Sep 9, 2020
59f06f7
Remove commented out test code.
MaggieMoss Sep 9, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions Include/internal/pycore_unionobject.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#ifndef Py_INTERNAL_UNIONOBJECT_H
#define Py_INTERNAL_UNIONOBJECT_H
#ifdef __cplusplus
extern "C" {
#endif

#ifndef Py_BUILD_CORE
# error "this header requires Py_BUILD_CORE define"
#endif

PyAPI_FUNC(PyObject *) _Py_Union(PyObject *args);
PyAPI_DATA(PyTypeObject) _Py_UnionType;

#ifdef __cplusplus
}
#endif
#endif /* !Py_INTERNAL_UNIONOBJECT_H */
32 changes: 32 additions & 0 deletions Lib/test/test_isinstance.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

import unittest
import sys
import typing



Expand Down Expand Up @@ -208,6 +209,25 @@ def test_isinstance_abstract(self):
self.assertEqual(False, isinstance(AbstractChild(), Super))
self.assertEqual(False, isinstance(AbstractChild(), Child))

def test_isinstance_with_or_union(self):
self.assertTrue(isinstance(Super(), Super | int))
self.assertFalse(isinstance(None, str | int))
self.assertTrue(isinstance(3, str | int))
self.assertTrue(isinstance("", str | int))
self.assertTrue(isinstance([], typing.List | typing.Tuple))
self.assertTrue(isinstance(2, typing.List | int))
self.assertFalse(isinstance(2, typing.List | typing.Tuple))
self.assertTrue(isinstance(None, int | None))
self.assertFalse(isinstance(3.14, int | str))
with self.assertRaises(TypeError):
isinstance(2, list[int])
with self.assertRaises(TypeError):
isinstance(2, list[int] | int)
with self.assertRaises(TypeError):
isinstance(2, int | str | list[int] | float)



def test_subclass_normal(self):
# normal classes
self.assertEqual(True, issubclass(Super, Super))
Expand All @@ -217,6 +237,8 @@ def test_subclass_normal(self):
self.assertEqual(True, issubclass(Child, Child))
self.assertEqual(True, issubclass(Child, Super))
self.assertEqual(False, issubclass(Child, AbstractSuper))
self.assertTrue(issubclass(typing.List, typing.List|typing.Tuple))
self.assertFalse(issubclass(int, typing.List|typing.Tuple))

def test_subclass_abstract(self):
# abstract classes
Expand Down Expand Up @@ -251,6 +273,16 @@ def test_isinstance_recursion_limit(self):
# blown
self.assertRaises(RecursionError, blowstack, isinstance, '', str)

def test_subclass_with_union(self):
self.assertTrue(issubclass(int, int | float | int))
self.assertTrue(issubclass(str, str | Child | str))
self.assertFalse(issubclass(dict, float|str))
self.assertFalse(issubclass(object, float|str))
with self.assertRaises(TypeError):
issubclass(2, Child | Super)
with self.assertRaises(TypeError):
issubclass(int, list[int] | Child)

def test_issubclass_refcount_handling(self):
# bpo-39382: abstract_issubclass() didn't hold item reference while
# peeking in the bases tuple, in the single inheritance case.
Expand Down
114 changes: 114 additions & 0 deletions Lib/test/test_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,20 @@

from test.support import run_with_locale
import collections.abc
from collections import namedtuple
import inspect
import pickle
import locale
import sys
import types
import unittest.mock
import weakref
import typing

class Example:
pass

class Forward: ...

class TypesTests(unittest.TestCase):

Expand Down Expand Up @@ -598,6 +605,113 @@ def test_method_descriptor_types(self):
self.assertIsInstance(int.from_bytes, types.BuiltinMethodType)
self.assertIsInstance(int.__new__, types.BuiltinMethodType)

def test_or_types_operator(self):
self.assertEqual(int | str, typing.Union[int, str])
self.assertNotEqual(int | list, typing.Union[int, str])
self.assertEqual(str | int, typing.Union[int, str])
self.assertEqual(int | None, typing.Union[int, None])
self.assertEqual(None | int, typing.Union[int, None])
self.assertEqual(int | str | list, typing.Union[int, str, list])
self.assertEqual(int | (str | list), typing.Union[int, str, list])
self.assertEqual(str | (int | list), typing.Union[int, str, list])
self.assertEqual(typing.List | typing.Tuple, typing.Union[typing.List, typing.Tuple])
self.assertEqual(typing.List[int] | typing.Tuple[int], typing.Union[typing.List[int], typing.Tuple[int]])
self.assertEqual(typing.List[int] | None, typing.Union[typing.List[int], None])
self.assertEqual(None | typing.List[int], typing.Union[None, typing.List[int]])
self.assertEqual(str | float | int | complex | int, (int | str) | (float | complex))
self.assertEqual(typing.Union[str, int, typing.List[int]], str | int | typing.List[int])
self.assertEqual(int | int, int)
self.assertEqual(
BaseException |
bool |
bytes |
complex |
float |
int |
list |
map |
set,
typing.Union[
BaseException,
bool,
bytes,
complex,
float,
int,
list,
map,
set,
])
with self.assertRaises(TypeError):
int | 3
with self.assertRaises(TypeError):
3 | int
with self.assertRaises(TypeError):
Example() | int
with self.assertRaises(TypeError):
(int | str) < typing.Union[str, int]
with self.assertRaises(TypeError):
(int | str) < (int | bool)
with self.assertRaises(TypeError):
(int | str) <= (int | str)
with self.assertRaises(TypeError):
# Check that we don't crash if typing.Union does not have a tuple in __args__
x = typing.Union[str, int]
x.__args__ = [str, int]
(int | str ) == x

def test_or_type_operator_with_TypeVar(self):
TV = typing.TypeVar('T')
assert TV | str == typing.Union[TV, str]
assert str | TV == typing.Union[str, TV]

def test_or_type_operator_with_forward(self):
T = typing.TypeVar('T')
ForwardAfter = T | 'Forward'
ForwardBefore = 'Forward' | T
def forward_after(x: ForwardAfter[int]) -> None: ...
def forward_before(x: ForwardBefore[int]) -> None: ...
assert typing.get_args(typing.get_type_hints(forward_after)['x']) == (int, Forward)
assert typing.get_args(typing.get_type_hints(forward_before)['x']) == (int, Forward)

def test_or_type_operator_with_Protocol(self):
class Proto(typing.Protocol):
def meth(self) -> int:
...
assert Proto | str == typing.Union[Proto, str]

def test_or_type_operator_with_Alias(self):
assert list | str == typing.Union[list, str]
assert typing.List | str == typing.Union[typing.List, str]

def test_or_type_operator_with_NamedTuple(self):
NT=namedtuple('A', ['B', 'C', 'D'])
assert NT | str == typing.Union[NT,str]

def test_or_type_operator_with_TypedDict(self):
class Point2D(typing.TypedDict):
x: int
y: int
label: str
assert Point2D | str == typing.Union[Point2D, str]

def test_or_type_operator_with_NewType(self):
UserId = typing.NewType('UserId', int)
assert UserId | str == typing.Union[UserId, str]

def test_or_type_operator_with_IO(self):
assert typing.IO | str == typing.Union[typing.IO, str]

def test_or_type_operator_with_SpecialForm(self):
assert typing.Any | str == typing.Union[typing.Any, str]
assert typing.NoReturn | str == typing.Union[typing.NoReturn, str]
assert typing.Optional[int] | str == typing.Union[typing.Optional[int], str]
assert typing.Optional[int] | str == typing.Union[int, str, None]
assert typing.Union[int, bool] | str == typing.Union[int, bool, str]

def test_or_type_repr(self):
assert repr(int | None) == "int | None"
assert repr(int | typing.GenericAlias(list, int)) == "int | list[int]"

class MappingProxyTests(unittest.TestCase):
mappingproxy = types.MappingProxyType
Expand Down
6 changes: 0 additions & 6 deletions Lib/test/test_typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -244,8 +244,6 @@ def test_subclass_error(self):
issubclass(int, Union)
with self.assertRaises(TypeError):
issubclass(Union, int)
with self.assertRaises(TypeError):
issubclass(int, Union[int, str])
with self.assertRaises(TypeError):
issubclass(Union[int, str], int)

Expand Down Expand Up @@ -347,10 +345,6 @@ def test_empty(self):
with self.assertRaises(TypeError):
Union[()]

def test_union_instance_type_error(self):
with self.assertRaises(TypeError):
isinstance(42, Union[int, str])

def test_no_eval_union(self):
u = Union[int, str]
def f(x: u): ...
Expand Down
1 change: 1 addition & 0 deletions Lib/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,7 @@ def wrapped(*args, **kwargs):


GenericAlias = type(list[int])
Union = type(int | str)


__all__ = [n for n in globals() if n[:1] != '_']
31 changes: 28 additions & 3 deletions Lib/typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,6 @@
# namespace, but excluded from __all__ because they might stomp on
# legitimate imports of those modules.


def _type_check(arg, msg, is_argument=True):
"""Check that the argument is a type, and return it (internal helper).

Expand Down Expand Up @@ -145,7 +144,7 @@ def _type_check(arg, msg, is_argument=True):
return arg
if isinstance(arg, _SpecialForm) or arg in (Generic, Protocol):
raise TypeError(f"Plain {arg} is not valid as type argument")
if isinstance(arg, (type, TypeVar, ForwardRef)):
if isinstance(arg, (type, TypeVar, ForwardRef, types.Union)):
return arg
if not callable(arg):
raise TypeError(f"{msg} Got {arg!r:.100}.")
Expand Down Expand Up @@ -205,7 +204,7 @@ def _remove_dups_flatten(parameters):
# Flatten out Union[Union[...], ...].
params = []
for p in parameters:
if isinstance(p, _UnionGenericAlias):
if isinstance(p, (_UnionGenericAlias, types.Union)):
params.extend(p.__args__)
elif isinstance(p, tuple) and len(p) > 0 and p[0] is Union:
params.extend(p[1:])
Expand Down Expand Up @@ -578,6 +577,12 @@ def __init__(self, name, *constraints, bound=None,
if def_mod != 'typing':
self.__module__ = def_mod

def __or__(self, right):
return Union[self, right]

def __ror__(self, right):
return Union[self, right]

def __repr__(self):
if self.__covariant__:
prefix = '+'
Expand Down Expand Up @@ -685,6 +690,12 @@ def __eq__(self, other):
def __hash__(self):
return hash((self.__origin__, self.__args__))

def __or__(self, right):
return Union[self, right]

def __ror__(self, right):
return Union[self, right]

@_tp_cache
def __getitem__(self, params):
if self.__origin__ in (Generic, Protocol):
Expand Down Expand Up @@ -784,6 +795,11 @@ def __subclasscheck__(self, cls):
def __reduce__(self):
return self._name

def __or__(self, right):
return Union[self, right]

def __ror__(self, right):
return Union[self, right]

class _CallableGenericAlias(_GenericAlias, _root=True):
def __repr__(self):
Expand Down Expand Up @@ -870,6 +886,15 @@ def __repr__(self):
return f'typing.Optional[{_type_repr(args[0])}]'
return super().__repr__()

def __instancecheck__(self, obj):
return self.__subclasscheck__(type(obj))

def __subclasscheck__(self, cls):
for arg in self.__args__:
if issubclass(cls, arg):
return True



class Generic:
"""Abstract base class for generic types.
Expand Down
2 changes: 2 additions & 0 deletions Makefile.pre.in
Original file line number Diff line number Diff line change
Expand Up @@ -429,6 +429,7 @@ OBJECT_OBJS= \
Objects/typeobject.o \
Objects/unicodeobject.o \
Objects/unicodectype.o \
Objects/unionobject.o \
Objects/weakrefobject.o

##########################################################################
Expand Down Expand Up @@ -1122,6 +1123,7 @@ PYTHON_HEADERS= \
$(srcdir)/Include/internal/pycore_sysmodule.h \
$(srcdir)/Include/internal/pycore_traceback.h \
$(srcdir)/Include/internal/pycore_tuple.h \
$(srcdir)/Include/internal/pycore_unionobject.h \
$(srcdir)/Include/internal/pycore_warnings.h \
$(DTRACE_HEADERS)

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Implement PEP 604. This supports (int | str) etc. in place of Union[str, int].
15 changes: 9 additions & 6 deletions Objects/abstract.c
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
/* Abstract Object Interface (many thanks to Jim Fulton) */

#include "Python.h"
#include "pycore_unionobject.h" // _Py_UnionType && _Py_Union()
#include "pycore_abstract.h" // _PyIndex_Check()
#include "pycore_ceval.h" // _Py_EnterRecursiveCall()
#include "pycore_pyerrors.h" // _PyErr_Occurred()
Expand Down Expand Up @@ -918,7 +919,6 @@ binary_op(PyObject *v, PyObject *w, const int op_slot, const char *op_name)
Py_TYPE(w)->tp_name);
return NULL;
}

return binop_type_error(v, w, op_name);
}
return result;
Expand Down Expand Up @@ -2491,7 +2491,6 @@ object_isinstance(PyObject *inst, PyObject *cls)
PyObject *icls;
int retval;
_Py_IDENTIFIER(__class__);

if (PyType_Check(cls)) {
retval = PyObject_TypeCheck(inst, (PyTypeObject *)cls);
if (retval == 0) {
Expand All @@ -2511,7 +2510,7 @@ object_isinstance(PyObject *inst, PyObject *cls)
}
else {
if (!check_class(cls,
"isinstance() arg 2 must be a type or tuple of types"))
"isinstance() arg 2 must be a type, a tuple of types or a union"))
return -1;
retval = _PyObject_LookupAttrId(inst, &PyId___class__, &icls);
if (icls != NULL) {
Expand Down Expand Up @@ -2604,10 +2603,14 @@ recursive_issubclass(PyObject *derived, PyObject *cls)
if (!check_class(derived,
"issubclass() arg 1 must be a class"))
return -1;
if (!check_class(cls,
"issubclass() arg 2 must be a class"
" or tuple of classes"))

PyTypeObject *type = Py_TYPE(cls);
int is_union = (PyType_Check(type) && type == &_Py_UnionType);
if (!is_union && !check_class(cls,
"issubclass() arg 2 must be a class,"
" a tuple of classes, or a union.")) {
return -1;
}

return abstract_issubclass(derived, cls);
}
Expand Down
Loading