Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Wrap returned objects in interface if method return type is interface #1240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 1, 2020

Conversation

danabr
Copy link
Contributor

@danabr danabr commented Sep 25, 2020

What does this implement/fix? Explain your changes.

This allows callers to call all methods of an interface, regardless of
whether the method was implemented implicitly or explicitly. Before this
change, you had to make an explicit cast to the interface to be able to
call the explicitly implemented method. Consider the following code:

namespace Python.Test {
    public interface ITestInterface
    {
        void Foo();
        void Bar();
    }

    public class TestImpl : ITestInterface
    {
        public void Foo() { };
        public void ITestInterface.Bar() { };
        public void Baz() { };

        public static ITestInterface GetInterface()
        {
            return new TestImpl();
        }
    }
}

And the following Python code, demonstrating the behavior before this
change:

from Python.Test import TestImpl, ITestInterface

test = TestImpl.GetInterface()
test.Foo() # works
test.Bar() # AttributeError: 'TestImpl' object has no attribute 'Bar'
test.Baz() # works! - baz

After this change, the behavior is as follows:

test = TestImpl.GetInterface()
test.Foo() # works
test.Bar() # works
test.Baz() # AttributeError: 'ITestInterface' object has no attribute 'Baz'

This is a breaking change due to that Baz is no longer visible in
Python.

Does this close any currently open issues?

No.

Any other comments?

See #1233 for an alternative approach, that exposes methods of explictly implemented interfaces without need to cast the object first.

Checklist

Check all those that are applicable and complete.

  • Make sure to include one or more tests for your change
  • If an enhancement PR, please create docs and at best an example
  • Add yourself to AUTHORS
  • Updated the CHANGELOG

@codecov-commenter
Copy link

codecov-commenter commented Sep 25, 2020

Codecov Report

Merging #1240 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1240   +/-   ##
=======================================
  Coverage   86.25%   86.25%           
=======================================
  Files           1        1           
  Lines         291      291           
=======================================
  Hits          251      251           
  Misses         40       40           
Flag Coverage Δ
#setup_linux 64.94% <ø> (ø)
#setup_windows 72.50% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d44f1da...c46ab75. Read the comment docs.

@danabr danabr mentioned this pull request Sep 25, 2020
4 tasks
Copy link
Member

@lostmsu lostmsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am actually unfamiliar with that side of Python.NET, but if we make this change, can users then somehow cast the interface back to the class? If that is not so yet, such capability must be added (perhaps as a Python property?)

@filmor , do you know if you can unwrap InterfaceObject from Python?

@danabr
Copy link
Contributor Author

danabr commented Sep 28, 2020

I also found that I have failed to handle array subscription. If a method returns a ISomething[] , then the individual elements will not be wrapped in the interface when you access them (e.g. arr[0]).

@danabr danabr force-pushed the auto-cast-ret-val-to-interface branch from 10158f4 to 010c290 Compare September 28, 2020 11:45
@danabr
Copy link
Contributor Author

danabr commented Sep 29, 2020

I've addressed the issue with returning arrays of interface objects.

What remains is the ability to "downcast"/unwrap objects, as @lostmsu mentioned. As far as I have been able to see, there is no way of doing that today. One could add a property as @lostmsu suggests - the question then becomes how to name that so that the risk of colliding with one of the interface members is minimized. Perhaps there is a subset of names that would be valid in Python, but not in .NET so one can avoid collisions completely?

@danabr
Copy link
Contributor Author

danabr commented Sep 29, 2020

Another approach would be to provide a service class that can be used for downcasting/unwrapping. A simple implementation of that today is:

namespace Python.Test {
    public class Cast
    {
        public static object Downcast(object input) // note the return type!
        {
            return input;
        }
    }
}

Which you then can use like this:

from Python.Test import Cast
ob = Cast.Downcast(iface_obj)

The reason I bring this to attention is that it highlights the incompleteness of my approach in this PR. I have tried to make sure that whenever a .NET method returns an interface, the interface is all that Python sees (proper implementation hiding). However, I have done nothing about the corresponding case for classes. If a method is declared to return a value of type ParentClass and returns a ChildClass, Python-land is free to access any member of ChildClass. Having different behavior for classes and interfaces does not seem like a good idea.

Either we should apply implementation hiding everywhere (like done for interfaces in this PR), or nowhere (which is the case today). The former is more in line with what you would get in C#. The latter is perhaps what users of Python.NET wants (?), since that is the behavior that is currently there.

I'd be happy to hear your thoughts on what is the right way forward.

@lostmsu
Copy link
Member

lostmsu commented Sep 29, 2020

@danabr completeness is not necessary. If there's a bug around C# new methods (not) hiding base class methods with the same name, let us track it separately.

The problem with no hiding is that it makes calling certain overloads potentially impossible due to name conflicts.

@filmor do you know if we have anything like Downcast methods already available?

If not, I suggest InterfaceObject to expose __implementation__ property pointing to the unwrapped instance (with converters applied) and __raw_implementation__ without any conversions.

@danabr danabr force-pushed the auto-cast-ret-val-to-interface branch from dbc89a4 to d5308fc Compare September 30, 2020 15:40
@danabr
Copy link
Contributor Author

danabr commented Sep 30, 2020

I've added the requested properties __implementation__ and __raw_implementation__. Alas, I failed to come up with a good way to test the latter (namely that no conversions are applied). I was hoping I could make some clever use of custom encoders to test it.

What remains now is properly documenting that the properties are available. Should that be done somewhere in this repo, or on the wiki, or on pythonnet.github.io?

I also need to make a note of the changes in the CHANGELOG, and make it clear that this is a breaking change.

@lostmsu
Copy link
Member

lostmsu commented Sep 30, 2020

@danabr to test __raw_implementation__ you can do IComparable with Int32, and ensure, that type of the raw object is actually System.Int32. Currently Python.NET conversions always turn primitive .NET int types to Python's int.

pythonnet.github.io seems to have only basic example, so you might want to create a new wiki page for the interface objects.

Yeah, a note in CHANGELOG would be helpful.

@danabr danabr force-pushed the auto-cast-ret-val-to-interface branch from d5308fc to a7d2829 Compare October 1, 2020 04:40
@danabr danabr marked this pull request as ready for review October 1, 2020 04:41
@danabr
Copy link
Contributor Author

danabr commented Oct 1, 2020

to test raw_implementation you can do IComparable with Int32, and ensure, that type of the raw object is actually System.Int32.

Great idea! I used that to add a test. I've also updated the CHANGELOG. I'll have a look at updating the Wiki if this gets merged.

Thanks for all the help and guidance, @lostmsu!

@lostmsu
Copy link
Member

lostmsu commented Oct 1, 2020

@danabr seems like a few tests are failing because the InterfaceObject has no tp_iter.

@danabr
Copy link
Contributor Author

danabr commented Oct 1, 2020

Argh, I wonder how come I did not notice before. I do run the tests locally. But now I can reproduce it. I'll look into it.

@danabr danabr force-pushed the auto-cast-ret-val-to-interface branch from a7d2829 to a604b3d Compare October 1, 2020 14:47
@danabr
Copy link
Contributor Author

danabr commented Oct 1, 2020

It was an unexpected interaction with the work in one of my other PRs. I must have forgotten to run the tests after rebasing on master. Now all tests pass locally. Hopefully they will pass in CI as well.

Comment on lines 86 to 87
Runtime.PyObject_SetAttrString(objPtr, "__implementation__", Converter.ToPython(impl));
Runtime.PyObject_SetAttrString(objPtr, "__raw_implementation__", CLRObject.GetInstHandle(impl));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two must be computed only on demand.

Copy link
Contributor Author

@danabr danabr Oct 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I fixed it by implementing tp_getattro. Note that the properties do not show up when you do dir on an interface object. If that is desired I should probably go the long route via tp_getset.

danabr added 4 commits October 1, 2020 21:41
This allows callers to call all methods of an interface, regardless of
whether the method was implemented implicitly or explicitly. Before this
change, you had to make an explicit cast to the interface to be able to
call the explicitly implemented method. Consider the following code:

```C#
namespace Python.Test {
    public interface ITestInterface
    {
        void Foo();
        void Bar();
    }

    public class TestImpl : ITestInterface
    {
        public void Foo() { };
        public void ITestInterface.Bar() { };
        public void Baz() { };

        public static ITestInterface GetInterface()
        {
            return new TestImpl();
        }
    }
}
```

And the following Python code, demonstrating the behavior before this
change:

```python
from Python.Test import TestImpl, ITestInterface

test = TestImpl.GetInterface()
test.Foo() # works
test.Bar() # AttributeError: 'TestImpl' object has no attribute 'Bar'
test.Baz() # works! - baz
```

After this change, the behavior is as follows:
```
test = TestImpl.GetInterface()
test.Foo() # works
test.Bar() # works
test.Baz() # AttributeError: 'ITestInterface' object has no attribute 'Baz'
```

This is a breaking change due to that `Baz` is no longer visible in
Python.
Even when a method is declared to return an array of interfaces, the CLR
may use an array of the concrete type. Keep track of the intended type
in `ArrayObject` so that elements of the array can be properly wrapped in
`InterfaceObject` when accessed.
@danabr danabr force-pushed the auto-cast-ret-val-to-interface branch from 33ed60f to c46ab75 Compare October 1, 2020 19:42
@lostmsu lostmsu merged commit 50d947f into pythonnet:master Oct 1, 2020
@C-SELLERS C-SELLERS mentioned this pull request Feb 15, 2021
4 tasks
Martin-Molinero pushed a commit to QuantConnect/pythonnet that referenced this pull request Apr 26, 2022
Reflect PR#8 MISSING CONVERTER.CS L516-528 Changes

Reflect PR #14

Reflect PR #15

Reflect PR #19

Reflect PR #25

Reflect PR #34

Reflect PR #35

Implement List Conversion, Reflect PR #37 Tests

Reflect PR #38 Partial: Assembly Manager Improvements

Reflect PR #38

Reflect PR #42 KeyValuePairEnumerableObject

Reflect PR #10 Runtime DecimalType

Add TimeDelta and DateTime tests

Fix DecimalConversion test for float conversion

Converter mod tweaks

Adjust a few broken PyTests

Use _pydecimal to not interfere with Lean/decimal.py

Add MethodBinder tests

MethodBinder implicit resolution

Fix bad cherry pick

Refactoring precedence resolution

Deal with operator binding

Fix `TestNoOverloadException` unit test

Fix for DomainReload tests

Add InEquality Operator Test

Dont PyObjects precedence in Operator methods

Revert "Merge pull request pythonnet#1240 from danabr/auto-cast-ret-val-to-interface"

This reverts commit 50d947f, reversing
changes made to d44f1da.

Fix Primitive Conversion to Int

Post rebase fix

Add PrimitiveIntConversion test

Add test for interface derived classes

Add to Authors.md

Load in current directory into Python Path

Include Python Lib in package

Update as QuantConnect.PythonNet; include console exe in package

Drop MaybeType from ClassManager for performance

Package nPython from same configuration

Address KWargs and Params; also cleanup

Add unit tests

Add pytest params unit test to testrunner

Remove testing case from TestRuntime.cs

Fix HandleParamsArray

Test case

Version bump

Update QC Tests

Refactor Params Fix

Fix assembly info

Handle breaking PyTests

Cleanup

Optimize Params Handling

First reflection improvements

Add TypeAccessor improvements and a bunch more tests

More improvements

Bump version to 2.0.2

Revert ClassManager changes

Remove readonly

Replace FastMember with Fasterflect

Add global MemberGetter/MemberSetter cache

Minor changes

Make Fasterflect work with all regression tests

Fix performance regressions

Revert accidental pythonnet/runtime/.gitkeep removal

Handle sending a python list to an enumerable expecting method

- Converter with handle sending a python List to a method expecting a
  csharp enumerable. Adding unit test

Bump version to 2.0.3

Update to net5.0

- Updating all projects to target net.50
- Remove domain test since it's not supported in net5.0

Bump pythonNet version 2.0.4

Add reproducing test

Apply fix

Catch implicit conversion throw

Cleanup solution

Cleanup V2

Assert Error message

Small performance improvement

Drop print statement from unit test

Bump version to 2.0.5

Bump references to new version

Fix for methods with different numerical precision overloads

  - Fix for methods with different numerical precision overloads. Method
    precedence will give higher priority to higher resolution numerical
    arguments. Adding unit test

Version bump to 2.0.6

KeyValuePair conversion and performance

- Improve DateTime conversion performance
- Add support for KeyValuePair conversions
- Minor improvements for convertions and method binder

TypeManager and decimal improvements

Reduce unrequired casting

Version bump to 2.0.7

Apply fixes

Project fix for linux systems

Add unit test

Converter cleanup

More adjustments and fixes

Add additional Py test & cleanup

Use the generic match when others fail

Add test for non-generic choice

Address review

Cleanup

Version bump 2.0.8

Add performance test, also add caching

Make Cache static to apply to all binders

Make adjustments from testing

Add test where overload exists with an already typed generic parameter

use `ContainsGenericParameters` to check for unassigned generics

Implement fix

Add accompanying test

Add additional tests

Fix minor issue with py Date -> DateTime

Refactor solution, use margs directly convert in ResolveGenericMethod

Version Bump 2.0.9

Add missing exception clearing. Adding unit test

Version bump 2.0.10

Handle readonly conversion to list. Adding unit tests

Bump version to 2.0.11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants