Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ENH: Add support for np.dtype(ctypes.Union) #12273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
eric-wieser opened this issue Oct 26, 2018 · 11 comments
Closed

ENH: Add support for np.dtype(ctypes.Union) #12273

eric-wieser opened this issue Oct 26, 2018 · 11 comments

Comments

@eric-wieser
Copy link
Member

This ought to be as simple as constructing a dtype with all the fields overlapping.

This will go in _dtype_ctypes.py, and there's a test that needs activating in test_dtype.py.

@euler16
Copy link

euler16 commented Oct 27, 2018

Can I work on this @eric-wieser ? Though I would require some guidance on how to work on this since I am begining with Open Source.

@eric-wieser
Copy link
Member Author

You'll want to change these lines:

elif issubclass(t, _ctypes.Union):
# TODO
raise NotImplementedError(
"conversion from ctypes.Union types like {} to dtype"
.format(t.__name__))

Such that this test passes:

@pytest.mark.xfail(
reason="Unions are not implemented",
raises=NotImplementedError)
def test_union(self):
class Union(ctypes.Union):
_fields_ = [
('a', ctypes.c_uint8),
('b', ctypes.c_uint16),
]
expected = np.dtype(dict(
names=['a', 'b'],
formats=[np.uint8, np.uint16],
offsets=[0, 0],
itemsize=2
))
self.check(Union, expected)

@bmakos
Copy link
Contributor

bmakos commented Nov 1, 2018

@euler16 - are you still working on this? If not I'd like to do it (I am also new to open source contributions).

@euler16
Copy link

euler16 commented Nov 2, 2018

yes I am, I will send a PR soon

@eric-wieser
Copy link
Member Author

@bmakos: If you want to work on something very similar, #10532 is probably only a little more complex

@bmakos
Copy link
Contributor

bmakos commented Nov 2, 2018

@bmakos: If you want to work on something very similar, #10532 is probably only a little more complex

Ok, I will take a look, see what I can do. Thank you!

@eric-wieser
Copy link
Member Author

@euler16: Are you close? It would be great if this could make it into 1.16 alongside @bmakos' work

@eric-wieser
Copy link
Member Author

Milestoned as 1.16 since the release notes added in #12392 claim this is already done!

@bmakos
Copy link
Contributor

bmakos commented Nov 16, 2018

So this indeed is simple, I wrote the code, but then decided to see how it behaves with a Union containing a packed structure.

>>>class Foo(ctypes.Structure):
    _fields_ = [('one', ctypes.c_uint8.__ctype_be__),('two', ctypes.c_uint32.__ctype_le__)]
    _pack_ = 1
    
>>>class Union(ctypes.Union):
    _fields_ = [
        ('a', ctypes.c_uint8),
        ('b', ctypes.c_uint16),
        ('c', ctypes.c_uint32),
        ('d', Foo),
    ] 
    
>>>ctypes.sizeof(Foo)
5
>>>ctypes.sizeof(Union)
8

In my code I constructed the dtype from Union with itemsize=ctypes.sizeof(Union))), which in this case gives size 8, while the correct size in my view would be 5.

As we are doing this to support ctypes the sensible choice is to still go with itemsize=ctypes.sizeof(Union))), but the ctypes Union constructor may need a fix (I will see how can I open a bug, never did this).

@eric-wieser
Copy link
Member Author

which in this case gives size 8, while the correct size in my view would be 5.

8 is correct - the struct has 3 bytes of padding inserted to meet the alignment requirements of uint32

@bmakos
Copy link
Contributor

bmakos commented Nov 17, 2018

I see now that if I also make the Union packed to 1 the size is 5. I assumed the rule to calculate the size was to take sizeof of the largest member as it is (in this case a packed Structure). Thank you for the clarification.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants