[prototype] Introduce KeyGrouping to make it possible to enable OrderedSerialization without code changes
#1857
Conversation
|
Timur Abishev seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
There was a problem hiding this comment.
My knee jerk is reaction is that we should be very suspect of an unusual macro here. We have thought about having OrderedSerialization required but I want to explore this carefully. Another approach is a wrapper that requires OrderedSerialization.
If you have users with conflicting Orderings, can we examine that a bit more? How commonly? What are the use cases? Why does it conflict with OrderedSerialization which is a superset?
| val desc = ordSerDesc(implicitInScope.toString()) | ||
| .getOrElse(implicitInScope.toString()) | ||
| val path = | ||
| if (pos.source.path.contains("/workspace/")) |
| // implicit def unsafeConvert[T](ord: Ordering[T]): KeyGrouping[T] = KeyGrouping(ord) | ||
|
|
||
| implicit def convert[T]: KeyGrouping[T] = macro KeyGroupingMacro[T] | ||
| } |
There was a problem hiding this comment.
Can you comment what this macro is and why it is needed?
| @@ -1 +1 @@ | |||
| version in ThisBuild := "0.17.2" | |||
| version in ThisBuild := "0.17.2" No newline at end of file | |||
There was a problem hiding this comment.
This version is out of date.
| } | ||
| import scala.math.Ordering | ||
|
|
||
| case class KeyGrouping[T](ord: Ordering[T]) |
There was a problem hiding this comment.
If we introduce a new type here I’d like to be a sealed trait and either be an Ordering or an OrderedSerialization.
|
Thinking back at this... We could have an alternative approach: |
We're considering to enable
OrderedSerializationfor most users at Twitter. Currently we have a blocker for that - users needs to change a source code to enable it (and not only pass a parameter, which is much more beneficial for us because we have auto-tuning infrastructure to make incremental land of such things possible).I've tried to prototype how we can workaround this - basic idea is to introduce
KeyGroupingclass which holds implicitly providedOrderingand generatedOrderedSerialization. With this approach we can choose which one of them to use in runtime.Another pros of this approach:
OrderedSerializationif they want to use it - it would be provided by defaultOrderedSerializationon top level you can accidentally break some other ordering use cases.Another thing which is nice about this approach is the fact it's source backward compatible.
This is just prototype (it's not even holds both
OrderedSerializationandOrdering, I used it to gatherOrdering's stats across our repo) but it illustrates the idea.@johnynek what do you think?