Thanks to visit codestin.com
Credit goes to github.com

Skip to content

bpo-34213: frozen dataclass with "object" attr bug #8452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ericvsmith merged 5 commits into from
Aug 12, 2018
Merged

bpo-34213: frozen dataclass with "object" attr bug #8452

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
fea7cf1
bpo-34213: frozen dataclass with "object" attr bug
VadimPushtaev Jul 24, 2018
3171e9c
bpo-34213: news for frozen dataclasses bug
VadimPushtaev Jul 24, 2018
4d7ee27
bpo-34213: frozen dataclass with "object" attr bug
VadimPushtaev Jul 26, 2018
4b0ea3e
bpo-34213: __builtins__ bug
VadimPushtaev Jul 26, 2018
d08afd9
bpo-34213: "__builtins" typo fix
VadimPushtaev Jul 26, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
bpo-34213: frozen dataclass with "object" attr bug
  • Loading branch information
@VadimPushtaev
VadimPushtaev committed Jul 24, 2018
commit fea7cf1466d7fef90b8c17011d296b95112e034f
33 changes: 22 additions & 11 deletions Lib/dataclasses.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,10 @@ def __repr__(self):
# @dataclass.
_PARAMS = '__dataclass_params__'

# The name of an attribute on the class that stores a reference to
# the object class in case it is shadowed by field name.
_OBJECT = '__dataclass_object__'

# The name of the function, that if it exists, is called at the end of
# __init__.
_POST_INIT_NAME = '__post_init__'
Expand Down Expand Up @@ -357,19 +361,19 @@ def _create_fn(name, args, body, *, globals=None, locals=None,
return locals[name]


def _field_assign(frozen, name, value, self_name):
def _field_assign(frozen, name, value, self_name, object_expression):
# If we're a frozen class, then assign to our fields in __init__
# via object.__setattr__. Otherwise, just use a simple
# assignment.
#
# self_name is what "self" is called in this function: don't
# hard-code "self", since that might be a field name.
if frozen:
return f'object.__setattr__({self_name},{name!r},{value})'
return f'{object_expression}.__setattr__({self_name},{name!r},{value})'
Copy link
Member

@ericvsmith ericvsmith Jul 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the only change you need to make in this file is to use __builtins__.object instead of object here. Since identifiers that start with double underscores are reserved for Python, we don't need to support a field named __builtins__.

Copy link
Contributor Author

@VadimPushtaev VadimPushtaev Jul 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's indeed a right thing do, I completely forgot about __builtins__.

However, surprisingly (at least for me) you should do __builtins__['object']__, not __builtins__.object since __builtins__ is a dictionary inside exec, not a module: “If the globals dictionary does not contain a value for the key __builtins__, a reference to the dictionary of the built-in module builtins is inserted under that key.” — https://docs.python.org/3/library/functions.html#exec

return f'{self_name}.{name}={value}'


def _field_init(f, frozen, globals, self_name):
def _field_init(f, frozen, globals, self_name, object_expression):
# Return the text of the line in the body of __init__ that will
# initialize this field.

Expand Down Expand Up @@ -420,7 +424,7 @@ def _field_init(f, frozen, globals, self_name):
return None

# Now, actually generate the field assignment.
return _field_assign(frozen, f.name, value, self_name)
return _field_assign(frozen, f.name, value, self_name, object_expression)


def _init_param(f):
Expand All @@ -442,7 +446,7 @@ def _init_param(f):
return f'{f.name}:_type_{f.name}{default}'


def _init_fn(fields, frozen, has_post_init, self_name):
def _init_fn(fields, frozen, has_post_init, self_name, object_expression):
# fields contains both real fields and InitVar pseudo-fields.

# Make sure we don't have fields without defaults following fields
Expand All @@ -465,7 +469,7 @@ def _init_fn(fields, frozen, has_post_init, self_name):

body_lines = []
for f in fields:
line = _field_init(f, frozen, globals, self_name)
line = _field_init(f, frozen, globals, self_name, object_expression)
# line is None means that this field doesn't require
# initialization (it's a pseudo-field). Just skip it.
if line:
Expand Down Expand Up @@ -841,6 +845,9 @@ def _process_class(cls, init, repr, eq, order, unsafe_hash, frozen):
# also marks this class as being a dataclass.
setattr(cls, _FIELDS, fields)

# Copy reference to "object" in case it is shadowed by a field name.
setattr(cls, _OBJECT, object)

# Was this class defined with an explicit __hash__? Note that if
# __eq__ is defined in this class, then python will automatically
# set __hash__ to None. This is a heuristic, as it's possible
Expand All @@ -862,15 +869,19 @@ def _process_class(cls, init, repr, eq, order, unsafe_hash, frozen):
# Include InitVars and regular fields (so, not ClassVars).
flds = [f for f in fields.values()
if f._field_type in (_FIELD, _FIELD_INITVAR)]
# The name to use for the "self" param in __init__.
# Use "self" if possible.
self_name = '__dataclass_self__' if 'self' in fields else 'self'
# The way to refer to "object" in __init__.
# Use "object" if possible.
object_expression = (f'{self_name}.{_OBJECT}'
if 'object' in fields else 'object')
_set_new_attribute(cls, '__init__',
_init_fn(flds,
frozen,
has_post_init,
# The name to use for the "self"
# param in __init__. Use "self"
# if possible.
'__dataclass_self__' if 'self' in fields
else 'self',
self_name,
object_expression,
))

# Get the fields as a list, and include only real fields. This is
Expand Down
14 changes: 14 additions & 0 deletions Lib/test/test_dataclasses.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,20 @@ class C:
first = next(iter(sig.parameters))
self.assertEqual('self', first)

def test_field_named_object(self):
@dataclass
class C:
object: str
c=C('foo')
Copy link
Member

@ericvsmith ericvsmith Jul 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add spaces around =.

self.assertEqual(c.object, 'foo')

def test_field_named_object_frozen(self):
@dataclass(frozen=True)
class C:
object: str
c=C('foo')
Copy link
Member

@ericvsmith ericvsmith Jul 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add spaces around =.

Copy link
Contributor Author

@VadimPushtaev VadimPushtaev Jul 26, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only reason I omitted spaces is they are already omitted in test_field_named_self which I used as a guideline. Should I fix that c=C as well? In this PR or in a separate one?

Copy link
Member

@ericvsmith ericvsmith Jul 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd just fix this particular case here, and another PR for all other cases of the same problem.

self.assertEqual(c.object, 'foo')

Copy link
Member

@ericvsmith ericvsmith Jul 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to these tests, please add a test for field names that are all identifiers in __builtins__ which don't start with double underscores, except for None, True, and False. You can use make_dataclass to dynamically generate a class with those members. I think this will work to generate the field names: [n for n in __builtins__.__dict__.keys() if not n.startswith('__') and not n in ('None', 'False', 'True')]. You'll want to do this for both a normal dataclass and a frozen one. Also, you'll want to test order=True.

Copy link
Member

@ericvsmith ericvsmith Jul 26, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like:

names = [n for n in __builtins__.__dict__.keys() if not n.startswith('__') and not n in ('None', 'False', 'True')]
C = make_dataclass('C', names)
C(*([0] * len(names)))

Which works in 3.7.0, and

names = [n for n in __builtins__.__dict__.keys() if not n.startswith('__') and not n in ('None', 'False', 'True')]
C = make_dataclass('C', names, frozen=True)
C(*([0] * len(names)))

Which fails in 3.7.0.

[Edit: fix frozen example]

def test_0_field_compare(self):
# Ensure that order=False is the default.
@dataclass
Expand Down