Better typing for overloaded higher-order methods #6871
Conversation
This comment has been minimized.
This comment has been minimized.
| @@ -2973,7 +3091,7 @@ trait Types | |||
| override def mapOver(map: TypeMap): Type = { | |||
| val pre1 = if (pre.isInstanceOf[ClassInfoType]) pre else map(pre) | |||
| if (pre1 eq pre) this | |||
| else OverloadedType(pre1, alternatives) | |||
| else OverloadedType(pre1, alternatives) // TODO: shouldn't we also map over the alternatives? | |||
There was a problem hiding this comment.
That's tricky, because we can't just go an clone alternatives here, as these symbols are serving double duty of carrying an info and linking to an actual member.
For the particular case of AsSeenFrom, we take care to later call memberType on the eventually selected alternative.
8114db5 to
f9a82f9
Compare
|
@SethTisue could you run a community build on this one? |
|
specs2 fails with: |
|
@niktrop heads up: we're generalizing type inference in 2.13 to allow overloading of methods like |
|
@retronym Thanks! I'll have a look. |
5a37ea2 to
22a1ec3
Compare
|
Another shot at community building: https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-community-build/1249/ |
|
Next failure is in gigahorse: @SethTisue could you double check to see if there's another new failure? scodec-bits had a test fail, while it was passing in the last community build that I compared to -- for now assuming that's not related to this PR |
|
I've seen the scodec-bits test failure in other runs, no worries there. this PR seems to have made shapeless green when it was red before, was that on purpose? /cc @milessabin (shapeless being green adds two failures, case-app and play-json, but I checked and the errors are similar to those from older runs before shapeless regressed, so no worries there) (also note that as of run 1250 we have moved to a newer SHA, fd820e4. so if you end up wanting to do another community build run, I'd suggest rebasing this PR onto fd820e4 first.) |
|
The shapeless progression could be if it used method values as arguments
for map, as that’s the kind of thing this is trying to fix (before we only
handled syntactically obvious function literals as arguments to overloaded
methods)
…On Wed, Jul 18, 2018 at 23:08 Seth Tisue ***@***.***> wrote:
I've seen the scodec-bits test failure in other runs, no worries there.
this PR seems to have been made shapeless green when it was red before,
was that on purpose?
(shapeless being green adds two failures, case-app and play-json, but I
checked and the errors are similar to those from older runs before
shapeless regressed, so no worries there)
(also note that as of run 1250 we have moved to a newer SHA, fd820e4. so
if you end up wanting to do another community build run, I'd suggest
rebasing this PR onto fd820e4 first.)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#6871 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAFjy3mep6CZ-Ms2kiZKvNAaeuTNg4znks5uH6PigaJpZM4U9BYG>
.
|
|
The gigahorse fail boils down to: 🤔 we should probably support that 😄 |
|
Note to self (so I remember after vacation :-)): I have some further non-essential WIP to also make TypeVars prototypes at overload_proto_wip. |
|
@adriaanm It'd be nice if we could establish a list of all the test cases related to the overloading resolution changes in 2.13 so that we can try to align dotty with it. |
|
Anything that I’ve worked on should have “overload” in the test file name.
(I’m on vacation until Aug 2)
…On Fri, Jul 20, 2018 at 20:24 Guillaume Martres ***@***.***> wrote:
@adriaanm <https://github.com/adriaanm> It'd be nice if we could
establish a list of all the test cases related to the overloading
resolution changes in 2.13 so that we can try to align dotty with it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#6871 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAFjy4c9Oyo6PoMZYdPt3uh0pUhuZTPAks5uIiBIgaJpZM4U9BYG>
.
|
|
shapeless was failing before because of a change in the constructor signature of |
|
re-review by @retronym? |
the results here were the same as run 1254, another 2.13.x run from around the same time, so I'd say we have a clean bill of health. |
|
I've run the performance tests for the It also sees the ~2% slowdown. Not sure yet how much of that is inherent to the change vs something we can ratchet down (e.g. by caching whether a given class symbol is a functional interface or not) |
|
@retronym are you comfortable with merging this for M5 and working on performance for RC1? |
| @@ -629,7 +631,7 @@ trait Implicits { | |||
| private def matchesPtView(tp: Type, ptarg: Type, ptres: Type, undet: List[Symbol]): Boolean = tp match { | |||
| case MethodType(p :: _, restpe) if p.isImplicit => matchesPtView(restpe, ptarg, ptres, undet) | |||
| case MethodType(p :: Nil, restpe) => matchesArgRes(p.tpe, restpe, ptarg, ptres, undet) | |||
| case ExistentialType(_, qtpe) => matchesPtView(normalize(qtpe), ptarg, ptres, undet) | |||
| case ExistentialType(_, qtpe) => matchesPtView(methodToExpressionTp(qtpe), ptarg, ptres, undet) | |||
| * - there is at least one FunctionN type expected by one of the overloads: | ||
| * in this case, the expected type is a FunctionN[Ti, ?], where Ti are the argument types (they must all be =:=), | ||
| * and the expected result type is elided using a wildcard. | ||
| * This does not exclude any overloads that expect a SAM, because they conform to a function type through SAM conversion |
There was a problem hiding this comment.
hmmm.. i though sam conversion works on literals, i.e., a function literal can get a sam type. if the expected type is a Function, isn't the literal typed with a Function type? how can that get converted to a sam type later?
anyway, it works as intended, so ignore my rambling.
scala> trait S[-T, +U] { def apply(x: T): U }
defined trait S
scala> object C { def m(f: String S Int) = 0; def m(f: String => String) = 1 }
defined object C
scala> C.m(x => 1)
res5: Int = 0
There was a problem hiding this comment.
It is first assigned the Function type, but the expected type is still the SAM. That gap is closed during adapt, case (14).
This will serve as a hook into type checking to represent expectation of types that cannot (cheaply) be encoded as existing types.
Normally, overload resolution types the arguments to the alternatives
without an expected type. However, typing function literals and
eta-expansion are driven by the expected type:
- function literals usually don't have parameter types, which are
derived from the expected type;
- eta-expansion right now only happens when a function/sam type is
expected.
(Dotty side-steps these issues by eta-expanding regardless of
expected type.)
Now that the collections are full of overloaded HO methods, we should
try harder to type check them nicely.
To avoid breaking existing code, we only provide an expected type (for
each argument position) when:
- there is at least one FunctionN type expected by one of the
overloads: in this case, the expected type is a FunctionN[Ti, ?],
where Ti are the argument types (they must all be =:=), and the
expected result type is elided using a wildcard. This does not
exclude any overloads that expect a SAM, because they conform to a
function type through SAM conversion
- OR: all overloads expect a SAM type of the same class, but with
potentially varying result types (argument types must be =:=)
We allow polymorphic cases, as long as the types parameters are
instantiated by the AntiPolyType prefix.
In all other cases, the old behavior is maintained: Wildcard is
expected.
(Slightly) more formally:
Consider an overloaded method `m_i`, with `N` overloads `i = 1..N`,
and an expected argument type at index `j`, `a_ij`:
```
def m_1(... a_1j, ...)
..
def m_N(... a_Nj, ...)
```
Any polymorphic method `m_i` will be reduced to the monomorphic case
by pushing down the method's `PolyType` to its arguments `a_ij`.
The expected type for the argument at index `j` will be more
precise than the usual `WildcardType` (`?`), if all types `a_1j..a_Nj`
are function-ish types that denote the same parameter types `p1..pM`.
A "function-ish" type is a `FunctionN[p1,...,pM]` (or
`PartialFunction`), or the equivalent SAM type. (We first unwrap any
PolyTypes.)
The non-wildcard expected type will be
- `PartialFunction[p1, ?]`, if an `a_ij` expects a partial function;
- else, if there is a subclass of `FunctionM` among the `a_ij`,
it is `FunctionM[p1, pM, ?]`;
- else, if all `a_ij` are of the same SAM type (allowing
for varying result types), that SAM type (with `?` result type).
In each case, any type parameter not already resolved by overloading,
or the outer context, it approximated by `?`.
PS: type equivalence is decided as `tp1 <:< tp2 && tp2 <:< tp1`,
and not `tp1 =:= tp2` (the latter is actually stricter).
This comment has been minimized.
This comment has been minimized.
53a1121 to
cd70540
Compare
This comment has been minimized.
This comment has been minimized.
Propagate more type information to improve eta-expansion and function parameter type inference for polymorphic, higher-order overloaded methods.
|
I think this is good to go, assuming we are ok with working on regaining the 2% performance drop by RC1. |
|
Let me take a look at some profiles to see how likely it is we can reclaim the performance. |
|
Some initial ideas: Implement I see some stack traces like: Debug to that point and then figure out what's going on. Is that a cyclic error that's getting caught higher up? Let's make val deferredMembers = (
tp.membersBasedOnFlags(excludedFlags = BridgeAndPrivateFlags, requiredFlags = METHOD).toList.filter(
mem => mem.isDeferred && !isUniversalMember(mem)
) // TODO: test
)
// if there is only one, it's monomorphic and has a single argument list
if (deferredMembers.lengthCompare(1) == 0 &&
deferredMembers.head.typeParams.isEmpty &&
deferredMembers.head.info.paramSectionCount == 1)
deferredMembers.head |
TODO:
For better typing for an overloaded higher-order method such as
map(prevalent in the new collections/ spark apis), regardless of whether the actual argument is a function literal, a method reference that
will be eta-expanded (based on the expected type), or a method value (
m _).Before, we triggered this based on the syntactic shape of the arguments, but that results (in our case)
in difference between
foo,foo _andx => foo(x)as arguments to overloaded method.The first commit (ProtoType generalizes [Bounded]WildcardType) sneakily sets the scene, pretty
specifically for the second one, but I do expect it to be useful for other cases.
Normally, overload resolution types the arguments to the alternatives without an expected type.
However, typing function literals and eta-expansion are driven by the expected type:
function literals usually don't have parameter types, which are derived from the expected type;
eta-expansion right now only happens when a function/sam type is expected.
(Dotty side-steps these issues by eta-expanding regardless of expected type.)
Now that the collections are full of overloaded HO methods, we should try harder to type check them
nicely.
To avoid breaking existing code, we only provide an expected type (for each argument position) when:
there is at least one FunctionN type expected by one of the overloads: in this case, the expected
type is a FunctionN[Ti, ?], where Ti are the argument types (they must all be =:=), and the expected
result type is elided using a wildcard. This does not exclude any overloads that expect a SAM,
because they conform to a function type through SAM conversion
OR: all overloads expect a SAM type of the same class, but with potentially varying result types
(argument types must be =:=)
We allow polymorphic cases, as long as the types parameters are instantiated by the AntiPolyType prefix.
In all other cases, the old behavior is maintained: Wildcard is expected.
(Slightly) more formally:
Consider an overloaded method
m_i, withNoverloadsi = 1..N, and an expected argument type atindex
j,a_ij:Any polymorphic method
m_iwill be reduced to the monomorphic case by pushing down the method'sPolyTypeto its argumentsa_ij.The expected type for the argument at index
jwill be more precise than the usualWildcardType(
?), if all typesa_1j..a_Njare function-ish types that denote the same parameter typesp1..pM.A "function-ish" type is a
FunctionN[p1,...,pM](orPartialFunction), or the equivalent SAM type.(We first unwrap any PolyTypes.)
The non-wildcard expected type will be
PartialFunction[p1, ?], if ana_ijexpects a partial function;FunctionMamong thea_ij, it isFunctionM[p1, pM, ?];a_ijare of the same SAM type (allowing for varying result types),that SAM type (with
?result type).In each case, any type parameter not already resolved by overloading, or the outer context, it
approximated by
?.PS: type equivalence is decided as
tp1 <:< tp2 && tp2 <:< tp1, and nottp1 =:= tp2(the latter isactually stricter).