Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add support for running pytest from dotnet core 2.0 (dotnet <path>/netcoreapp2.0/nPython.dll) #590

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dmitriyse opened this issue Dec 12, 2017 · 16 comments
Assignees

Comments

@dmitriyse
Copy link
Contributor

Environment

  • Pythonnet version: Any
  • Python version: Any
  • Operating System: Any

Details

  • Describe what you were trying to get done.

    pytest suite now works unly under Windows/Mono but not under .Net Core 2.0

  • What commands did you run to trigger this issue? If you can provide a
    Minimal, Complete, and Verifiable example
    this will help us understand the issue.

import sys
sys.path.append('/users/isys/icronyn/pyenvs/pythonnet/lib/python2.7/site-packages/pytest-2.9.1-py2.7.egg')
sys.path.append('/users/isys/icronyn/pyenvs/pythonnet/lib/python2.7/site-packages/py-1.4.31-py2.7.egg')
import pytest
pytest.main()
============================================================================== test session starts ==============================================================================
platform linux2 -- Python 2.7.5, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /users/isys/icronyn/git/pythonnet, inifile: 
collected 376 items 

src/tests/test_array.py .....................................F..
src/tests/test_callback.py ..
src/tests/test_class.py .......F..F..FF...
src/tests/test_clrmethod.py FFFFF
src/tests/test_compat.py ......FFFF....F.
src/tests/test_constructors.py ....
src/tests/test_conversion.py ....................
src/tests/test_delegate.py ...
Unhandled Exception: System.MissingMethodException: Constructor on type 'System.Reflection.Emit.TypeBuilder' not found.
   at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, StackCrawlMark& stackMark)
   at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes)
   at System.Activator.CreateInstance(Type type, Object[] args)
   at Python.Runtime.DelegateManager.GetDelegate(Type dtype, IntPtr callable) in /users/isys/icronyn/git/pythonnet/src/runtime/delegatemanager.cs:line 159
   at Python.Runtime.DelegateObject.tp_new(IntPtr tp, IntPtr args, IntPtr kw) in /users/isys/icronyn/git/pythonnet/src/runtime/delegateobject.cs:line 61
   at Python.Runtime.MetaType.tp_call(IntPtr tp, IntPtr args, IntPtr kw) in /users/isys/icronyn/git/pythonnet/src/runtime/metatype.cs:line 148
   at Python.Runtime.Runtime.Py_Main(Int32 argc, String[] argv)
   at Python.Runtime.PythonConsole.Main(String[] args) in /users/isys/icronyn/git/pythonnet/src/console/pythonconsole.cs:line 37
Aborted
@dmitriyse
Copy link
Contributor Author

Bad news:

I tried to add this support and found too many big and small problems.
So this feature requires up to a full working week.

I will publish my current work state in a new branch little bit later.

@Cronan
Copy link
Contributor

Cronan commented Dec 13, 2017

Thanks for this @dmitriyse - sorry for raising so many issues.
Let me know if there's anything I can do to help, I'm happy to run tests on your branch when you're ready

@dmitriyse
Copy link
Contributor Author

@Cronan thank you for a lot of testing efforts. You issues and reports are very useful. Probably I will reopen some of them later for separate problem solving.
At first look this was a one issue, but after deep dive I found at least two separate big problems.

@Cronan
Copy link
Contributor

Cronan commented Jan 8, 2018

Hi @dmitriyse , where are you with this? Is there anything I can do?

@dmitriyse
Copy link
Contributor Author

Hi @Cronan, unfortunately I am busy now. I forgot to publish my current progress on this issue. I will create branch today.

@dmitriyse
Copy link
Contributor Author

The pytests core 2.0 fixes branch has been created. https://github.com/pythonnet/pythonnet/tree/pytests-core

@Cronan
Copy link
Contributor

Cronan commented Jan 10, 2018

I'll take a look today, thanks @dmitriyse

@Cronan
Copy link
Contributor

Cronan commented Jan 19, 2018

@dmitriyse definitely looking better, gets a lot further. It fails a few tests, mainly due to missing classes in .Net core like System.Drawing, Hashtable, a few others, but crashes hard when it gets to here:

_____________________________ test_sys_argv_state ______________________________

filepath = <function make_filepath at 0x2226410>

    def test_sys_argv_state(filepath):
        """Test sys.argv state doesn't change after clr import.
        To better control the arguments being passed, test on a fresh python
        instance with specific arguments"""
    
        script = filepath("argv-fixture.py")
>       out = check_output([sys.executable, script, "foo", "bar"])

src/tests/test_sysargv.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/tests/_compat.py:71: in check_output
    output = subprocess.check_output(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

popenargs = (['/users/isys/icronyn/pyenvs/tmp-2eb586eb98020886/bin/python', '/users/isys/icronyn/git/pythonnet/src/tests/fixtures/netstandard2.0/argv-fixture.py', 'foo', 'bar'],)
kwargs = {}, process = <subprocess.Popen object at 0x1c04a50>, output = ''
unused_err = None, retcode = 1
cmd = ['/users/isys/icronyn/pyenvs/tmp-2eb586eb98020886/bin/python', '/users/isys/icronyn/git/pythonnet/src/tests/fixtures/netstandard2.0/argv-fixture.py', 'foo', 'bar']

    def check_output(*popenargs, **kwargs):
        r"""Run command with arguments and return its output as a byte string.
    
        If the exit code was non-zero it raises a CalledProcessError.  The
        CalledProcessError object will have the return code in the returncode
        attribute and output in the output attribute.
    
        The arguments are the same as for the Popen constructor.  Example:
    
        >>> check_output(["ls", "-l", "/dev/null"])
        'crw-rw-rw- 1 root root 1, 3 Oct 18  2007 /dev/null\n'
    
        The stdout argument is not allowed as it is used internally.
        To capture standard error in the result, use stderr=STDOUT.
    
        >>> check_output(["/bin/sh", "-c",
        ...               "ls -l non_existent_file ; exit 0"],
        ...              stderr=STDOUT)
        'ls: non_existent_file: No such file or directory\n'
        """
        if 'stdout' in kwargs:
            raise ValueError('stdout argument not allowed, it will be overridden.')
        process = Popen(stdout=PIPE, *popenargs, **kwargs)
        output, unused_err = process.communicate()
        retcode = process.poll()
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
>           raise CalledProcessError(retcode, cmd, output=output)
E           CalledProcessError: Command '['/users/isys/icronyn/pyenvs/tmp-2eb586eb98020886/bin/python', '/users/isys/icronyn/git/pythonnet/src/tests/fixtures/netstandard2.0/argv-fixture.py', 'foo', 'bar']' returned non-zero exit status 1

/usr/lib64/python2.7/subprocess.py:575: CalledProcessError
----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/users/isys/icronyn/git/pythonnet/src/tests/fixtures/netstandard2.0/argv-fixture.py", line 10, in <module>
    import clr
ImportError: No module named clr

@Cronan
Copy link
Contributor

Cronan commented Jan 19, 2018

@dmitriyse Below please find the tests that failed "normally" - some could be fixed by choosing classes that are part of .Net Standard 2.0 ...

=================================== FAILURES ===================================
____________________ test_boxed_value_type_mutation_result _____________________

    def test_boxed_value_type_mutation_result():
        """Test behavior of boxed value types."""
    
        # This test actually exists mostly as documentation of an important
        # concern when dealing with value types. Python does not have any
        # value type semantics that can be mapped to the CLR, so it is easy
        # to accidentally write code like the following which is not really
        # mutating value types in-place but changing boxed copies.
    
>       from System.Drawing import Point
E       ImportError: No module named Drawing

src/tests/test_array.py:1146: ImportError
_____________________________ test_basic_subclass ______________________________

    def test_basic_subclass():
        """Test basic subclass of a managed class."""
>       from System.Collections import Hashtable
E       ImportError: cannot import name Hashtable

src/tests/test_class.py:64: ImportError
___________________________ test_struct_construction ___________________________

    def test_struct_construction():
        """Test construction of structs."""
>       from System.Drawing import Point
E       ImportError: No module named Drawing

src/tests/test_class.py:121: ImportError
____________________________ test_override_get_item ____________________________

    def test_override_get_item():
        """Test managed subclass overriding __getitem__."""
>       from System.Collections import Hashtable
E       ImportError: cannot import name Hashtable

src/tests/test_class.py:172: ImportError
____________________________ test_override_set_item ____________________________

    def test_override_set_item():
        """Test managed subclass overriding __setitem__."""
>       from System.Collections import Hashtable
E       ImportError: cannot import name Hashtable

src/tests/test_class.py:193: ImportError
______________________ test_set_and_get_property_from_py _______________________

    def test_set_and_get_property_from_py():
        """Test setting and getting clr-accessible properties from python."""
>       t = ExampleClrClass()
E       TypeError: no constructor matches given arguments

src/tests/test_clrmethod.py:30: TypeError
______________________ test_set_and_get_property_from_clr ______________________

    def test_set_and_get_property_from_clr():
        """Test setting and getting clr-accessible properties from the clr."""
>       t = ExampleClrClass()
E       TypeError: no constructor matches given arguments

src/tests/test_clrmethod.py:39: TypeError
__________________ test_set_and_get_property_from_clr_and_py ___________________

    def test_set_and_get_property_from_clr_and_py():
        """Test setting and getting clr-accessible properties alternatingly from the clr and from python."""
>       t = ExampleClrClass()
E       TypeError: no constructor matches given arguments

src/tests/test_clrmethod.py:49: TypeError
________________________ test_method_invocation_from_py ________________________

    def test_method_invocation_from_py():
        """Test calling a clr-accessible method from python."""
>       t = ExampleClrClass()
E       TypeError: no constructor matches given arguments

src/tests/test_clrmethod.py:67: TypeError
_______________________ test_method_invocation_from_clr ________________________

    def test_method_invocation_from_clr():
        """Test calling a clr-accessible method from the clr."""
>       t = ExampleClrClass()
E       TypeError: no constructor matches given arguments

src/tests/test_clrmethod.py:72: TypeError
_________________________ test_dotted_name_import_from _________________________

    def test_dotted_name_import_from():
        """Test dotted-name 'import from'."""
>       from CLR.System import Xml
E       ImportError: cannot import name Xml

src/tests/test_compat.py:115: ImportError
___________________ test_dotted_name_import_from_with_alias ____________________

    def test_dotted_name_import_from_with_alias():
        """Test dotted-name 'import from' with aliasing."""
>       from CLR.System import Xml as myXml
E       ImportError: cannot import name Xml

src/tests/test_compat.py:134: ImportError
_________________________ test_from_module_import_star _________________________

    def test_from_module_import_star():
        """Test from module import * behavior."""
        count = len(locals().keys())
>       m = __import__('CLR.System.Management', globals(), locals(), ['*'])
E       ImportError: No module named Management

src/tests/test_compat.py:154: ImportError
_________________________ test_explicit_assembly_load __________________________

    def test_explicit_assembly_load():
        """Test explicit assembly loading using standard CLR tools."""
        from CLR.System.Reflection import Assembly
        from CLR import System
        import sys
    
        assembly = Assembly.LoadWithPartialName('System.Data')
        assert assembly is not None
    
>       import CLR.System.Data
E       ImportError: No module named Data

src/tests/test_compat.py:176: ImportError
_____________________________ test_module_get_attr _____________________________

    def test_module_get_attr():
        """Test module getattr behavior."""
        import CLR.System as System
    
        int_type = System.Int32
        assert is_clr_class(int_type)
    
>       module = System.Xml
E       AttributeError: Xml

src/tests/test_compat.py:228: AttributeError
___________________________ test_interface_property ____________________________

    def test_interface_property():
        """Test properties of interfaces. Added after a bug report
           that an IsAbstract check was inappropriate and prevented
           use of properties when only the interface is known."""
>       from System.Collections import Hashtable, ICollection
E       ImportError: cannot import name Hashtable

src/tests/test_property.py:144: ImportError

@dmitriyse
Copy link
Contributor Author

dmitriyse commented Jan 19, 2018

The main problem here is the implicit assembly loading algorithm. It should be hard redesigned to allow to map class to an assembly under .Net Core 2.0. I did not try to figureout some solution. Maybe I will have some time to this at the Febrary.

@dmitriyse
Copy link
Contributor Author

I already tried to link required nuget packages but this is not enough.

@Cronan
Copy link
Contributor

Cronan commented Jan 23, 2018

To be honest, if I’m writing C# libs to be run in “full flavour” .Net 4, and in Core, I either need to stick to Standard 2.0 libraries, or I’m going to be using additional explicit references for Core. Maybe that’s the same for pythonnet? As far as the tests go, I was going to suggest changing them to use .Net Standard 2.0 types only - what do you think?

@Cronan
Copy link
Contributor

Cronan commented Jan 23, 2018

.Net Core 2.0 apps in VS2017 solve the problem by adding a heap of built-in references when you create a new project ...

classlibrary2 - microsoft visual studio_2018-01-23_09-04-05

classlibrary2 - microsoft visual studio_2018-01-23_09-05-15

@dmitriyse
Copy link
Contributor Author

Hi! I need to think about this solution.

@dmitriyse
Copy link
Contributor Author

Please publish your changes in some temporary fork.

@dmitriyse
Copy link
Contributor Author

.Net Core 2.1 preview should be released soon. So I prefere to wait it before resolve this issue, some differencies between Classic .Net Framework 4.x and .Net Core 2.0 will be resolved.

@filmor filmor closed this as completed Jun 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants