ZIO Mock handle polymorphic methods #3136
Conversation
|
Rebased, fix for dotty and added more test coverage. Interesting findings: needed to hint the compiler with value type in fully polymorphic case, but only in the I'm not sure if we can do much about it, but I guess at least we could absorb the defect at higher level and turn it into a nice error message telling the developer that he should add type hints for the compiler. TODO:
|
|
Almost ready. Finishing touches on |
|
It seems like the generated output of macro is correct, however for some reason the compiler fails to recognize the mocked methods do implement the service interface. See #zio-macros on discord for details. |
|
@adamgfraser @jdegoes all set and ready! Just one problem with dotty: Also dotty#4163 states it can be used manually, but for that we'd have to fork all Mocks for default services into |
Yes, we could, the effort is underway, you can contribute to https://github.com/7mind/dotty-typetag-research - but there are also upstream blockers currently, missing APIs in tasty-reflect: scala/scala3#8514
The ticket states it's fixed on master, why not try to switch to current latest nightly build? |
|
Aparently in latest Also tried |
|
@ioleo |
|
You could also try replacing ClassTag with a custom summoner with a fallback for Nothing: final class ClassTagWrapper[A](cls: ClassTag[A])
object ClassTagWrapper extends LowPrio {
implicit def fromClassTag[T: ClassTag] = new ClassTagWrapper(scala.reflect.classTag[T])
}
trait LowPrio {
implicit val clsNothing: ClassTagWrapper[Nothing] = ...
} |
|
@neko-kai I just noticed there are newer nightlies published under |
|
Regarding dotty blockers for izumi-reflect, you may upvote or write a comment on these issues to bring attention for them to the dotty team: scala/scala3#8514 regarding |
|
|
Also regarding the tasty-reflect issues you mentioned: please open PRs to add the stuff you want to the tasty-reflect API, there's only one person working on tasty-reflect who cannot do all the work himself. It just requires figuring out what method in the compiler does what you want, then adding a forwarder to that method in https://github.com/lampepfl/dotty/blob/master/library/src/scala/tasty/Reflection.scala |
|
@smarter We are using ClassTag to overcome type erasure of generic parameters. Our effect type
Now, if an effect can't fail (it either succeeds or computes forever) we represent that as type |
|
@neko-kai Regarding the "issue fixed in master" upon rereading the thread I see Martin was talking about the original issue: def foo[T <: String: ClassTag](f: T => Int) = 1
def bar(f: String => Int) = foo(f)
^
No ClassTag available for Nothingwhich is diffrent case from ours. |
And since you can't construct a value of type |
|
@smarter Yes, but the reason we're asking for |
|
I see you're using private[zio] def taggedIsSubtype[A, B](left: TagType, right: TagType): Boolean =
right.isAssignableFrom(left)Assignability of runtime classes is a separate concept from subtyping, so even if there was a |
| } | ||
|
|
||
| abstract class LowPrio { | ||
| implicit val classTagNothing: ClassTagBox[Nothing] = new ClassTagBox(ClassTag(classOf[Nothing])) |
There was a problem hiding this comment.
Maybe next we'll disallow classOf[Nothing] to prevent this kind of misuse.
There was a problem hiding this comment.
To be 100% clear, you should not use classOf[Nothing]:
- It's not going to work properly (
taggedIsSubtype(taggedTagType(classTagNothing), anyOtherTag)will always return false, when it should always return true). - It leads to the same kind of soundness hole we had previously, so we'll have to disallow it and your code will stop compiling.
There was a problem hiding this comment.
@smarter This wouldn't make much sense, in Scala 2 classOf cannot fail, even for class-less types like abstract type members. There is always an available erasure for any concrete type.
ClassTag could fail before, so that's not totally new, but classOf on concrete types cannot fail on current Scala
There was a problem hiding this comment.
I'm not sure why you insist this is misuse. Nothing is just a special case we're not particuralily interested in, it just has to be supported becouse it is used all over ZIO as a marker that an effect can't fail. I'll try to give it some more thought tomorrow, or maybe you could explain in more details why this is wrong? 🙏
There was a problem hiding this comment.
This wouldn't make much sense, in Scala 2 classOf cannot fail, even for class-less types like abstract type members.
That doesn't seem to be true, I tried 2.13.1:
scala> def foo[T] = classOf[T]
^
error: class type required but T found
scala> class Foo { type A; classOf[A] }
^
error: class type required but Foo.this.A foundI'll try to give it some more thought tomorrow, or maybe you could explain in more details why this is wrong
classOf[Nothing] returns scala.runtime.Nothing$ a dummy class that is never instantiated and is not a subclass or superclass of anything else. You told me you needed this ClassTag for Method, I looked at Method and saw:
taggedIsSubtype(self.inputTag, that.inputTag) &&
taggedIsSubtype(self.errorTag, that.errorTag) &&
taggedIsSubtype(self.outputTag, that.outputTag)and the definition of taggedIsSubtype is:
private[zio] def taggedIsSubtype[A, B](left: TagType, right: TagType): Boolean =
right.isAssignableFrom(left)This will never return true when left is classOf[Nothing], even though Nothing is a subtype of every other type so this is wrong (it is wrong in many other ways too, there are many situations where subtyping and subclassing are different).
There was a problem hiding this comment.
That doesn't seem to be true, I tried 2.13.1:
It succeeds for all stable paths. (except type projections)
classOf[Nothing] returns scala.runtime.Nothing$ a dummy class that is never instantiated
It's used in Scala 2 in erasure of methods returning Nothing. Also .asInstanceOf[Nothing] will try to cast to that class - it's not total compiler fiction, Nothing does refer to a material class currently.
def erasure: Nothing = ???
public scala.runtime.Nothing$ erasure()
Whenever working with java reflection it still makes sense to e.g. check method return type against classOf[Nothing], as long as the latter exists in bytecode, so I disagree that it would make any sense to remove it.
For ClassTag, Array can use an opaque type over ClassTag that disallows bottoms without changing ClassTag or breaking binary compatibility, I'll leave it for you to judge whether that's worth the indirection, but I do think that a soundness problem with Array should lead to changes in Array, not to changes in ClassTag, a thing that's only tangentially related to Array and makes sense for as long as scala.runtime.Nothing$ actually exists in bytecode.
This will never return true when left is classOf[Nothing], even though Nothing is a subtype of every other type so this is wrong (it is wrong in many other ways too, there are many situations where subtyping and subclassing are different).
This is true, but there are also other cases where isAssignableFrom will fail when in Scala it would succeed either at runtime or at compile-time (intvs.Intvs.Integer). (@ioleo Clearly you'll need to add special cases in taggedIsSubtype for this to work, yep)
There was a problem hiding this comment.
It succeeds for all stable paths. (except type projections)
I don't understand what you mean by that.
Whenever working with java reflection it still makes sense to e.g. check method return type against classOf[Nothing], as long as the latter exists in bytecode, so I disagree that it would make any sense to remove it.
In general, you can't rely on classOf to give you a type that matches exactly the result type of a method, for example a method that returns (Int, Int) in source code will return scala.Tuple2$mcII$sp in bytecode, but classOf[(Int, Int)] returns scala.Tuple2.
I do think that a soundness problem with Array should lead to changes in Array, not to changes in ClassTag, a thing that's only tangentially related to Array
At this point, I think that making array is the only justified usage of ClassTag, it's possible to use it to avoid unchecked warning, but this is actually unsound and will have to be removed: scala/scala3#7554. Otherwise it seems to be mostly used incorrectly (like in this PR, where it is mistaken as a way to do subtype checks at runtime).
This is true, but there are also other cases where isAssignableFrom will fail when in Scala it would succeed either at runtime or at compile-time (intvs.Intvs.Integer
Exactly, you can't do subtype checks at runtime using ClassTag at all, so it doesn't matter if ClassTag[Nothing] exists or not, this is fundamentally the wrong approach.
Clearly you'll need to add special cases in taggedIsSubtype for this to work
No amount of special cases that you add will ever make this correct in general.
There was a problem hiding this comment.
I don't understand what you mean by that.
I was wrong, classOf only works for class types and intersections.
At this point, I think that making array is the only justified usage of ClassTag, it's possible to use it to avoid unchecked warning
Fine, but the ecosystem uses ClassTag for a lot besides that, e.g. printing the class name is pureconfig - ClassTag[Nothing] is completely correct for printing.
Exactly, you can't do subtype checks at runtime using ClassTag at all, so it doesn't matter if ClassTag[Nothing] exists or not, this is fundamentally the wrong approach.
No amount of special cases that you add will ever make this correct in general.
It can make it correct for a narrow amount of well known cases - such as unparameterized concrete class types, which is all that's necessary for now until izumi-reflect for dotty is ready and will be able handle a larger amount of known cases.
There was a problem hiding this comment.
Fine, but the ecosystem uses ClassTag for a lot besides that
I think the ecosystem will have to switch to other typeclasses depending on their usecase. Either making their own (like Typeable in shapeless: https://github.com/milessabin/shapeless/blob/shapeless-3/core/src/main/scala/shapeless/typeable.scala, or reusing the stuff in dotty like scala.quoted.Type (currently only usable from inside an inline def, but maybe that limitation can be lifted for globally accessible types).
|
Updated code providing manually |
|
So indeed, as @smarter pointed:
The failing suite deals with Tuples and therefore To progress from here I'm marking this test as ignored on dotty (and added appropriate comment). Since there are not many dotty users yet (and dotty is itself not stable) I think it's OK for now to merge it "as is", knowing that for some cases it's broken on dotty. Meanwhile I'll try to get involved with dotty and submit PRs for missing tasty API parts that are blocking izumi-reflect support. |
This is my inital take on polymorphic methods. I have not had enough time to thoroughly explore if there are no undesired side effects, so I'm putting this out and will come back to this tomorrow.
What is still missing is updateing the
@mockablemacro to generate proper tags for polymorphic methods, but the initial tests (seeMockSpec) look promising.Fixes #2978.