Thanks to visit codestin.com
Credit goes to github.com

Skip to content

bpo-34213: frozen dataclass with "object" attr bug #8452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ericvsmith merged 5 commits into from
Aug 12, 2018
Merged

bpo-34213: frozen dataclass with "object" attr bug #8452

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
fea7cf1
bpo-34213: frozen dataclass with "object" attr bug
VadimPushtaev Jul 24, 2018
3171e9c
bpo-34213: news for frozen dataclasses bug
VadimPushtaev Jul 24, 2018
4d7ee27
bpo-34213: frozen dataclass with "object" attr bug
VadimPushtaev Jul 26, 2018
4b0ea3e
bpo-34213: __builtins__ bug
VadimPushtaev Jul 26, 2018
d08afd9
bpo-34213: "__builtins" typo fix
VadimPushtaev Jul 26, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
bpo-34213: frozen dataclass with "object" attr bug
  • Loading branch information
@VadimPushtaev
VadimPushtaev committed Jul 26, 2018
commit 4d7ee274a3901c0a4b57b07ae9b3063854a6cd70
33 changes: 11 additions & 22 deletions Lib/dataclasses.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,10 +186,6 @@ def __repr__(self):
# @dataclass.
_PARAMS = '__dataclass_params__'

# The name of an attribute on the class that stores a reference to
# the object class in case it is shadowed by field name.
_OBJECT = '__dataclass_object__'

# The name of the function, that if it exists, is called at the end of
# __init__.
_POST_INIT_NAME = '__post_init__'
Expand Down Expand Up @@ -361,19 +357,19 @@ def _create_fn(name, args, body, *, globals=None, locals=None,
return locals[name]


def _field_assign(frozen, name, value, self_name, object_expression):
def _field_assign(frozen, name, value, self_name):
# If we're a frozen class, then assign to our fields in __init__
# via object.__setattr__. Otherwise, just use a simple
# assignment.
#
# self_name is what "self" is called in this function: don't
# hard-code "self", since that might be a field name.
if frozen:
return f'{object_expression}.__setattr__({self_name},{name!r},{value})'
return f'__builtins__["object"].__setattr__({self_name},{name!r},{value})'
Copy link
Member

@ericvsmith ericvsmith Jul 26, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per your other comment: I did not know this was how you had to access __builtins__ inside exec. Although we could get around this by passing adding {"__builtins__": __builtins__} to the globals passed in to exec. I'll have to think about that: there might be other reasons to set globals["__builtins__"].

Copy link
Contributor Author

@VadimPushtaev VadimPushtaev Jul 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's remarkable, isn't that.

Copy link
Contributor Author

@VadimPushtaev VadimPushtaev Jul 26, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it is dict only if you provide the globals argument:

>>> exec('print(type(__builtins__))')
<class 'module'>
>>> exec('print(type(__builtins__))', None, None)
<class 'module'>
>>> exec('print(type(__builtins__))', {}, None)
<class 'dict'>

That means that the right thing to do is to include __builtins__ in the globals dict or always use {} instead of None.

return f'{self_name}.{name}={value}'


def _field_init(f, frozen, globals, self_name, object_expression):
def _field_init(f, frozen, globals, self_name):
# Return the text of the line in the body of __init__ that will
# initialize this field.

Expand Down Expand Up @@ -424,7 +420,7 @@ def _field_init(f, frozen, globals, self_name, object_expression):
return None

# Now, actually generate the field assignment.
return _field_assign(frozen, f.name, value, self_name, object_expression)
return _field_assign(frozen, f.name, value, self_name)


def _init_param(f):
Expand All @@ -446,7 +442,7 @@ def _init_param(f):
return f'{f.name}:_type_{f.name}{default}'


def _init_fn(fields, frozen, has_post_init, self_name, object_expression):
def _init_fn(fields, frozen, has_post_init, self_name):
# fields contains both real fields and InitVar pseudo-fields.

# Make sure we don't have fields without defaults following fields
Expand All @@ -469,7 +465,7 @@ def _init_fn(fields, frozen, has_post_init, self_name, object_expression):

body_lines = []
for f in fields:
line = _field_init(f, frozen, globals, self_name, object_expression)
line = _field_init(f, frozen, globals, self_name)
# line is None means that this field doesn't require
# initialization (it's a pseudo-field). Just skip it.
if line:
Expand Down Expand Up @@ -845,9 +841,6 @@ def _process_class(cls, init, repr, eq, order, unsafe_hash, frozen):
# also marks this class as being a dataclass.
setattr(cls, _FIELDS, fields)

# Copy reference to "object" in case it is shadowed by a field name.
setattr(cls, _OBJECT, object)

# Was this class defined with an explicit __hash__? Note that if
# __eq__ is defined in this class, then python will automatically
# set __hash__ to None. This is a heuristic, as it's possible
Expand All @@ -869,19 +862,15 @@ def _process_class(cls, init, repr, eq, order, unsafe_hash, frozen):
# Include InitVars and regular fields (so, not ClassVars).
flds = [f for f in fields.values()
if f._field_type in (_FIELD, _FIELD_INITVAR)]
# The name to use for the "self" param in __init__.
# Use "self" if possible.
self_name = '__dataclass_self__' if 'self' in fields else 'self'
# The way to refer to "object" in __init__.
# Use "object" if possible.
object_expression = (f'{self_name}.{_OBJECT}'
if 'object' in fields else 'object')
_set_new_attribute(cls, '__init__',
_init_fn(flds,
frozen,
has_post_init,
self_name,
object_expression,
# The name to use for the "self"
# param in __init__. Use "self"
# if possible.
'__dataclass_self__' if 'self' in fields
else 'self',
))

# Get the fields as a list, and include only real fields. This is
Expand Down
37 changes: 35 additions & 2 deletions Lib/test/test_dataclasses.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,16 +196,49 @@ def test_field_named_object(self):
@dataclass
class C:
object: str
c=C('foo')
c = C('foo')
self.assertEqual(c.object, 'foo')

def test_field_named_object_frozen(self):
@dataclass(frozen=True)
class C:
object: str
c=C('foo')
c = C('foo')
self.assertEqual(c.object, 'foo')

Copy link
Member

@ericvsmith ericvsmith Jul 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to these tests, please add a test for field names that are all identifiers in __builtins__ which don't start with double underscores, except for None, True, and False. You can use make_dataclass to dynamically generate a class with those members. I think this will work to generate the field names: [n for n in __builtins__.__dict__.keys() if not n.startswith('__') and not n in ('None', 'False', 'True')]. You'll want to do this for both a normal dataclass and a frozen one. Also, you'll want to test order=True.

Copy link
Member

@ericvsmith ericvsmith Jul 26, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like:

names = [n for n in __builtins__.__dict__.keys() if not n.startswith('__') and not n in ('None', 'False', 'True')]
C = make_dataclass('C', names)
C(*([0] * len(names)))

Which works in 3.7.0, and

names = [n for n in __builtins__.__dict__.keys() if not n.startswith('__') and not n in ('None', 'False', 'True')]
C = make_dataclass('C', names, frozen=True)
C(*([0] * len(names)))

Which fails in 3.7.0.

[Edit: fix frozen example]

def test_field_named_like_builtin(self):
# Attribute names can shadow built-in names
# since code generation is used.
# Ensure that this is not happening.
exclusions = {'None', 'True', 'False'}
builtins = sorted(
b for b in __builtins__.__dict__.keys()
if not b.startswith('__') and b not in exclusions
)
C = make_dataclass('C', [(name, str) for name in builtins])

c = C(*[name for name in builtins])

for name in builtins:
self.assertEqual(getattr(c, name), name)

def test_field_named_like_builtin_frozen(self):
# Attribute names can shadow built-in names
# since code generation is used.
# Ensure that this is not happening
# for frozen data classes.
exclusions = {'None', 'True', 'False'}
builtins = sorted(
b for b in __builtins__.__dict__.keys()
if not b.startswith('__') and b not in exclusions
)
C = make_dataclass('C', [(name, str) for name in builtins], frozen=True)

c = C(*[name for name in builtins])

for name in builtins:
self.assertEqual(getattr(c, name), name)

def test_0_field_compare(self):
# Ensure that order=False is the default.
@dataclass
Expand Down
3 changes: 1 addition & 2 deletions Misc/NEWS.d/next/Library/2018-07-25-00-40-14.bpo-34213.O15MgP.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
Frozen dataclasses didn't work properly with an attribute called "object".
Now they do.
Allow frozen dataclasses to have a field named "object". Previously this conflicted with an internal use of "object".