Refactor Managed-Native conversion - ICustomMarshaler #407

vmuriart · 2017-02-26T02:00:49Z

What does this implement/fix? Explain your changes.

Refactors conversions from Managed to Native type using ICustomMarshaler's to handle the various conversions scenarios and manage releasing unmanaged memory.

Does this close any currently open issues?

#408

Any other comments?

This doesn't need to implement Native to Managed conversions as it's not needed for the purposes of the pr.

vmuriart · 2017-02-26T02:03:09Z

@dmitriyse in case you wanted to review.

codecov · 2017-02-26T17:49:32Z

Codecov Report

Merging #407 into master will decrease coverage by -0.24%.
The diff coverage is 51.42%.

@@            Coverage Diff             @@
##           master     #407      +/-   ##
==========================================
- Coverage   63.62%   63.38%   -0.24%     
==========================================
  Files          61       61              
  Lines        5264     5277      +13     
  Branches      861      863       +2     
==========================================
- Hits         3349     3345       -4     
- Misses       1698     1713      +15     
- Partials      217      219       +2

Flag	Coverage Δ
#Embedded_Tests	`33.05% <51.42%> (-0.17%)`	❌
#Python_Tests	`59.95% <51.42%> (-0.18%)`	❌
#Setup_Linux	`74.5% <51.42%> (ø)`	✅
#Setup_Windows	`70.5% <51.42%> (ø)`	✅

Impacted Files	Coverage Δ
src/runtime/debughelper.cs	`0% <0%> (ø)`	✅
src/runtime/converter.cs	`77.89% <100%> (ø)`	✅
src/runtime/runtime.cs	`84.37% <100%> (+3.63%)`	✅
src/runtime/CustomMarshaler.cs	`49.05% <49.05%> (ø)`
src/runtime/interop36.cs

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7dba617...78939c5. Read the comment docs.

vmuriart · 2017-02-26T17:52:56Z

Expanded it to include #408 as well since its related.

den-run-ai · 2017-02-27T03:58:04Z

src/runtime/CustomMarshaler.cs

+                Marshal.Copy(bStr, 0, mem, bStr.Length);
+            }
+            catch (Exception)
+            {


no exception throwing here?

what you mean? like re-throwing the Exception? That part was meant to de-allocate the memory in case something goes wrong, otherwise keep it.

isn't failing on Marshal.Copy need to be thrown as Exception? If not, how is this handled later?

I'll throw the exception again.

Added the throw in the latest commit

den-run-ai · 2017-02-27T04:00:20Z

src/runtime/CustomMarshaler.cs

+            return mem;
+        }
+
+        public static ICustomMarshaler GetInstance(string cookie)


What is the purpose of string cookie?

Its part of the implementation of the interface. On the link below scroll down to Implementing the GetInstance Method. For our use case we don't need the cookie though but left it since its part of the implementation.

https://msdn.microsoft.com/en-us/library/system.runtime.interopservices.icustommarshaler(v=vs.110).aspx#Remarks

Our old monosupport actually was an ICustomMarshaler too but it depended on mono libraries to work. It also had the same signature, except it was called s

oh, i see on Mono: // The Utf32Marshaler was written Jonathan Pryor and has been placed in the PUBLIC DOMAIN.

@vmuriart so is custom marshalling dependent on COM interop now?

This static method is called by the common language runtime's COM interop layer to instantiate an instance of the custom marshaler.

No. I think that line is in reference that the COM Interop layer can call the instance. Its probably related to the whole cookie thing in which the instance returned depends on which cookie is passed.

Why is previous Mono implemented Marshalled string not a null-terminating like you added here?

because it didn't do the marshal at all, it passed it mono to do it. Mono itself then did it here

Useful resources https://msdn.microsoft.com/en-us/library/system.runtime.interopservices.icustommarshaler(v=vs.110).aspx https://limbioliong.wordpress.com/2013/11/03/understanding-custom-marshaling-part-1/ https://github.com/mono/mono/blob/master/mcs/class/Mono.Posix/Mono.Unix/UnixMarshal.cs http://stackoverflow.com/a/33514037/5208670

Refactor PyString_FromStringAndSize Link explains why `MarshalAs(UnmanagedType.LPWStr)` or `CharSet.Unicode` don't work http://stackoverflow.com/a/25128147/5208670

Its redundant with PyUnicode_AsUnicode now that the signature is fixed between UCS2/UCS4. Apply char conversion that work on both UCS2/UCS4

This won't change during runtime.

den-run-ai · 2017-02-28T03:22:34Z

I'm fine with this pull request but it is a little above my knowledge.

den-run-ai · 2017-03-18T06:46:09Z

@vmuriart somehow I completely missed the whole point of this PR. Do you know why I did not see your PR description while reviewing on github?

Add ICustomMarshaler Utf8Marshaler
Refactor PyString_FromStringAndSize
Link explains why MarshalAs(UnmanagedType.LPWStr) or CharSet.Unicode don't work

http://stackoverflow.com/a/25128147/5208670

vmuriart · 2017-03-18T16:58:51Z

i dont understand you question

den-run-ai · 2017-03-18T17:36:04Z

I missed the headers of commits in this PR, e.g.: vmuriart@e487076 I'm not sure if there is any better way to expose these headers during PR?

…

On Sat, Mar 18, 2017, 11:58 AM Victor Uriarte ***@***.***> wrote: i dont understand you question — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#407 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgZ5W9FxuZc5GcKb49fmp_C1kYQdXZFks5rnA1MgaJpZM4MMNTo> .

vmuriart · 2017-03-19T00:46:18Z

Did you try the commit tab?

den-run-ai · 2017-03-19T02:26:25Z

Ok, now I see. So by default the descriptions are collapsed. Thank you for the suggestion.

vmuriart requested review from filmor, tonyroberts and den-run-ai February 26, 2017 02:01

den-run-ai reviewed Feb 27, 2017

View reviewed changes

vmuriart added 8 commits February 27, 2017 18:57

Add ICustomMarshaler StrArrayMarshaler

d01f25f

Refactor Marshals

74440a6

Add ICustomMarshaler Utf8Marshaler

e487076

Refactor PyString_FromStringAndSize Link explains why `MarshalAs(UnmanagedType.LPWStr)` or `CharSet.Unicode` don't work http://stackoverflow.com/a/25128147/5208670

Match PyUnicode_AsUnicode signature in UCS2/UCS4

ccd4521

Refactor GetManagedString

07f87de

Remove internal PyUnicode_AS_UNICODE

edafdf2

Its redundant with PyUnicode_AsUnicode now that the signature is fixed between UCS2/UCS4. Apply char conversion that work on both UCS2/UCS4

Refactor Encoding check

78939c5

This won't change during runtime.

vmuriart force-pushed the custom_marshal branch from 84209c6 to 78939c5 Compare February 28, 2017 02:05

vmuriart merged commit 01a35cb into pythonnet:master Feb 28, 2017

vmuriart deleted the custom_marshal branch February 28, 2017 03:48

vmuriart mentioned this pull request Feb 28, 2017

PyUnicode_AsUnicode function signature mismatch on UCS2/UCS4 #408

Closed

Refactor Managed-Native conversion - ICustomMarshaler #407

Refactor Managed-Native conversion - ICustomMarshaler #407

Uh oh!

Conversation

vmuriart commented Feb 26, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this implement/fix? Explain your changes.

Does this close any currently open issues?

Any other comments?

Uh oh!

vmuriart commented Feb 26, 2017

Uh oh!

codecov bot commented Feb 26, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

vmuriart commented Feb 26, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

den-run-ai Feb 28, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

den-run-ai commented Feb 28, 2017

Uh oh!

den-run-ai commented Mar 18, 2017

Uh oh!

vmuriart commented Mar 18, 2017

Uh oh!

den-run-ai commented Mar 18, 2017 via email

Uh oh!

vmuriart commented Mar 19, 2017

Uh oh!

den-run-ai commented Mar 19, 2017

Uh oh!

Uh oh!

vmuriart commented Feb 26, 2017 •

edited

Loading

codecov bot commented Feb 26, 2017 •

edited

Loading

den-run-ai Feb 28, 2017 •

edited

Loading