Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Finish comparison operator impl and add tests
  • Loading branch information
christabella committed Jan 12, 2021
commit 31f3ed1309ddb7358f0a7ec315bdd006d048aaad
18 changes: 18 additions & 0 deletions src/embed_tests/TestOperator.cs
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,12 @@ public void SymmetricalOperatorOverloads()

c = a > b
assert c == (a.Num > b.Num)

c = a == b
assert c == (a.Num == b.Num)

c = a != b
assert c == (a.Num != b.Num)
");
}

Expand Down Expand Up @@ -339,6 +345,12 @@ public void ForwardOperatorOverloads()

c = a > b
assert c == (a.Num > b)

c = a == b
assert c == (a.Num == b)

c = a != b
assert c == (a.Num != b)
");
}

Expand Down Expand Up @@ -392,6 +404,12 @@ public void ReverseOperatorOverloads()

c = a > b
assert c == (a > b.Num)

c = a == b
assert c == (a == b.Num)

c = a != b
assert c == (a != b.Num)
");

}
Expand Down
46 changes: 46 additions & 0 deletions src/runtime/classbase.cs
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ internal class ClassBase : ManagedType
[NonSerialized]
internal List<string> dotNetMembers;
internal Indexer indexer;
internal Hashtable richcompare;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not have it as Dictionary<int, MethodObject>, mapping directly from {Py_EQ, ...} to corresponding operator implementation?

internal MaybeType type;

internal ClassBase(Type tp)
Expand All @@ -35,6 +36,15 @@ internal virtual bool CanSubclass()
return !type.Value.IsEnum;
}

public readonly static Dictionary<int, string> PyToCilOpMap = new Dictionary<int, string>
{
[Runtime.Py_EQ] = "op_Equality",
[Runtime.Py_NE] = "op_Inequality",
[Runtime.Py_GT] = "op_GreaterThan",
[Runtime.Py_GE] = "op_GreaterThanOrEqual",
[Runtime.Py_LT] = "op_LessThan",
[Runtime.Py_LE] = "op_LessThanOrEqual",
};

/// <summary>
/// Default implementation of [] semantics for reflected types.
Expand Down Expand Up @@ -72,6 +82,42 @@ public static IntPtr tp_richcompare(IntPtr ob, IntPtr other, int op)
{
CLRObject co1;
CLRObject co2;
IntPtr tp = Runtime.PyObject_TYPE(ob);
var cls = (ClassBase)GetManagedObject(tp);
// C# operator methods take precedence over IComparable.
// We first check if there's a comparison operator by looking up the richcompare table,
// otherwise fallback to checking if an IComparable interface is handled.
if (PyToCilOpMap.ContainsKey(op)) {
string CilOp = PyToCilOpMap[op];
if (cls.richcompare.Contains(CilOp)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, you'd do cls.richcompare.TryGetValue(op, out var methodObject) instead of having separate ContainsKey check.

var methodObject = (MethodObject)cls.richcompare[CilOp];
IntPtr args = other;
var free = false;
if (!Runtime.PyTuple_Check(other))
Copy link
Member

@lostmsu lostmsu Jan 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the code inside this if should ever be skipped. How would a > (1,2) work if a was a .NET object with > operator? Because (1,2) is a tuple, they'd be passed to Invoke as two args instead of 1.

Consequently, free will alwasy be true and not needed.

Probably a good idea to add a test. I believe you'd need an operator >(SomeClass a, PyObject b) defined and to check b indeed receives the tuple (1,2).

Copy link
Contributor Author

@christabella christabella Jan 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed if b were a tuple this if would be skipped. Once removing the if, the operator method does receive b as PyObject, but I get an AccessViolationException : Attempted to read or write protected memory. so I think b is wrongly parsed? I suppose a tuple when converted into a PyObject should probably not look like this:

image

Copy link
Contributor Author

@christabella christabella Jan 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that error message might have been a flaky bug, because now the variable states are the same but the error is:

Message: 
    Python.Runtime.PythonException : TypeError : '>=' not supported between instances of 'int' and 'tuple'

Copy link
Member

@lostmsu lostmsu Jan 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because to work with PyObject instances in C# callbacks you must hold GIL. E.g. the method body must be inside using (Py.GIL()) { ... block. It applies to debugging too.

Copy link
Contributor Author

@christabella christabella Jan 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I didn't know about that. I put the operator >(Obj a, PyObject b) body in a Py.GIL block and can now see that b has a value of {(1, 2)}
image

I added 2/6 of the tuple comparison tests but it's done in a bit of a strange way. In the test I'm assuming that the PyObject is a tuple; is that okay?


(Below is resolved)

However, the same error message persists, and when I try to debug

result = binding.info.Invoke(binding.inst, BindingFlags.Default, null, binding.args, null);

I get this error image

Maybe I'm missing something simple again, should the GIL block should also be wrapping that statement? But that would make the change bigger than I thought. Are tuples commonly used enough for comparison operators, or can I put the tuple test in a different PR?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need to add a test with a tuple for each comparison operator. One is enough.

Copy link
Member

@lostmsu lostmsu Jan 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The failure on the screenshot is only happening because you have debugger attached. It tries to call ToString on some PyObject to show in Watches window, but it does not work without Py.GIL

{
// Wrap the `other` argument of a binary comparison operator in a PyTuple.
args = Runtime.PyTuple_New(1);
Runtime.XIncref(other);
Runtime.PyTuple_SetItem(args, 0, other);
free = true;
}

IntPtr value;
try
{
value = methodObject.Invoke(ob, args, IntPtr.Zero);
}
finally
{
if (free)
{
Runtime.XDecref(args); // Free args pytuple
}
}
return value;
}
}

switch (op)
{
case Runtime.Py_EQ:
Expand Down
4 changes: 4 additions & 0 deletions src/runtime/classmanager.cs
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,7 @@ private static void InitClassBase(Type type, ClassBase impl)
ClassInfo info = GetClassInfo(type);

impl.indexer = info.indexer;
impl.richcompare = new Hashtable();

// Now we allocate the Python type object to reflect the given
// managed type, filling the Python type slots with thunks that
Expand All @@ -284,6 +285,9 @@ private static void InitClassBase(Type type, ClassBase impl)
Runtime.PyDict_SetItemString(dict, name, item.pyHandle);
// Decref the item now that it's been used.
item.DecrRefCount();
if (ClassBase.PyToCilOpMap.ContainsValue(name)) {
impl.richcompare.Add(name, iter.Value);
}
}

// If class has constructors, generate an __doc__ attribute.
Expand Down
10 changes: 9 additions & 1 deletion src/runtime/operatormethod.cs
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ static OperatorMethod()
["op_UnaryNegation"] = new SlotDefinition("__neg__", TypeOffset.nb_negative),
["op_UnaryPlus"] = new SlotDefinition("__pos__", TypeOffset.nb_positive),
["op_OneComplement"] = new SlotDefinition("__invert__", TypeOffset.nb_invert),
["op_Equality"] = new SlotDefinition("__eq__", TypeOffset.tp_richcompare),
["op_Inequality"] = new SlotDefinition("__ne__", TypeOffset.tp_richcompare),
["op_GreaterThan"] = new SlotDefinition("__gt__", TypeOffset.tp_richcompare),
["op_GreaterThanOrEqual"] = new SlotDefinition("__ge__", TypeOffset.tp_richcompare),
["op_LessThan"] = new SlotDefinition("__lt__", TypeOffset.tp_richcompare),
Expand Down Expand Up @@ -79,6 +81,12 @@ public static bool IsOperatorMethod(MethodBase method)
}
return OpMethodMap.ContainsKey(method.Name);
}

public static bool IsComparisonOp(MethodInfo method)
{
return OpMethodMap[method.Name].TypeOffset == TypeOffset.tp_richcompare;
}

/// <summary>
/// For the operator methods of a CLR type, set the special slots of the
/// corresponding Python type's operator methods.
Expand All @@ -91,7 +99,7 @@ public static void FixupSlots(IntPtr pyType, Type clrType)
Debug.Assert(_opType != null);
foreach (var method in clrType.GetMethods(flags))
{
if (!IsOperatorMethod(method))
if (!IsOperatorMethod(method) || IsComparisonOp(method)) // We don't want to override ClassBase.tp_richcompare.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Direct comment to the reason why we don't want to override, otherwise it does not explain anything and simply says what code in this method does (which is generally not very useful).

Something like "comparison operators are handled by ClassBase.tp_richcompare"

{
continue;
}
Expand Down