Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Fix #4997: Add linkTimeIf for link-time conditional branching. #5110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

sjrd
Copy link
Member

@sjrd sjrd commented Jan 10, 2025

Thanks to our optimizer's ability to inline, constant-fold, and then eliminate dead code, we have been able to write link-time conditional branches for a long time. Typical examples include polyfills, as illustrated in the documentation of LinkingInfo:

if (esVersion >= ESVersion.ES2018 || featureTest())
  useES2018Feature()
else
  usePolyfill()

which gets folded away to nothing but

useES2018Feature()

when linking for ES2018+.

However, this only works because both branches can link during the initial reachability analysis. We cannot use the same technique when one of the branches would refuse to link in the first place. The canonical example is the usage of the JS ** operator, which does not link under ES2016. The following snippet produces good code when linking for ES2016+, but does not link at all for ES2015:

def pow(x: Double, y: Double): Double = {
  if (esVersion >= ESVersion.ES2016) {
    (x.asInstanceOf[js.Dynamic] ** y.asInstanceOf[js.Dynamic]).asInstanceOf[Double]
  } {
    Math.pow(x, y)
  }
}

This commit introduces LinkingInfo.linkTimeIf, a conditional branch that is guaranteed by spec to be resolved at link-time. Using a linkTimeIf instead of the if in def pow, we can successfully link the fallback branch on ES2015, because the then branch is not even followed by the reachability analysis.

In order to provide that guarantee, the corresponding LinkTimeIf IR node has strong requirements on its condition. It must be a "link-time expression", which is guaranteed to be resolved at link-time. A link-time expression tree must be of the form:

  • A Literal (of type int, boolean or string, although strings are not actually usable here).
  • A LinkTimeProperty.
  • One of the boolean operators.
  • One of the int comparison operators.
  • A nested LinkTimeIf (used to encode short-circuiting boolean && and ||).

The ClassDefChecker validates the above property, and ensures that link-time expression trees are well-typed. Normally that is the job of the IR checker. Here we can do in ClassDefChecker because we only have the 3 primitive types to deal with; and we must do it then, because the reachability analysis itself is only sound if all link-time expression trees are well-typed.

The reachability analysis algorithm itself is not affected by LinkTimeIf. Instead, we resolve link-time branches when building the Infos of methods. We follow only the branch that is taken. This means that Infos builders now require the coreSpec, but that is the only additional piece of complexity in that area.

LinkTimeIfs nodes are later removed from the trees during desugaring.


At the language and compiler level, we introduce LinkingInfo.linkTimeIf as a primitive for LinkTimeIf. We need a dedicated method to compile link-time expression trees, which does incur some duplication, unfortunately. Other than that, linkTimeIf is straightforward, by itself.

The problem is that the whole point of linkTimeIf is that we can refer to link-time properties, and not just literals. However, our link-time properties are all hidden behind regular method calls, such as LinkInfo.esVersion. For optimizer-based branching with ifs, that is fine, as the method is always inlined, and the optimizer can then see the constant. However, for linkTimeIf, that does not work, as it does not follow the requirements of a link-time expression tree.

If we were on Scala 3 only, we could declare esVersion and its friends as an inline def, as follows:

inline def esVersion: Int =
  linkTimePropertyInt("core/esVersion")

The inline keyword is guaranteed by the language to be resolved at compile-time. Since the linkTimePropertyInt method is itself a primitive replaced by a LinkTimeProperty, by the time we reach our backend, we would see the latter, and all would be well. The same cannot be said for the @inline optimizer hint, which is all we have.

We therefore add another language-level feature: @linkTimeProperty. This annotation can (currently) only be used in our own library. By contract, it must only be used on a method whose body is the corresponding linkTimePropertyX primitive. With it, we can define esVersion as:

@inline @linkTimeProperty("core/esVersion")
def esVersion: Int =
  linkTimePropertyInt("core/esVersion")

That annotation makes the body public, in a way. That means the compiler back-end can now replace call sites to esVersion by the LinkTimeProperty.

Semantically, @linkTimeProperty does nothing more than guaranteed inlining (with strong restrictions on the shape of body).

@sjrd sjrd force-pushed the link-time-if-with-desugaring-phase branch 2 times, most recently from 23ace4a to 7e80c96 Compare January 10, 2025 23:31
@sjrd sjrd force-pushed the link-time-if-with-desugaring-phase branch 3 times, most recently from df03f46 to b35acbe Compare February 6, 2025 15:49
@sjrd sjrd changed the title LinkTimeIf - on top of a separate desugaring phase Fix #4997: Add linkTimeIf for link-time conditional branching. Feb 6, 2025
@sjrd sjrd marked this pull request as ready for review February 6, 2025 15:51
@sjrd sjrd requested a review from gzm0 February 6, 2025 15:51
@sjrd sjrd force-pushed the link-time-if-with-desugaring-phase branch from b35acbe to 322b521 Compare March 16, 2025 16:43
Copy link
Contributor

@tanishiking tanishiking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for re-visiting this! I left a few comments but overall looks good from my side.

@sjrd sjrd force-pushed the link-time-if-with-desugaring-phase branch from 322b521 to a77c06f Compare March 18, 2025 09:08
@sjrd sjrd force-pushed the link-time-if-with-desugaring-phase branch from a77c06f to ba96702 Compare April 7, 2025 16:05
@sjrd sjrd force-pushed the link-time-if-with-desugaring-phase branch from ba96702 to a34ef73 Compare April 22, 2025 08:12
@gzm0
Copy link
Contributor

gzm0 commented Apr 26, 2025

We therefore another language-level feature: @linkTimeProperty.

Typo: add "add"?

Copy link
Contributor

@gzm0 gzm0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! This is much less messy then I expected :P

Main higher level concern is about the potentially unnecessary spread of link time tree evaluation v.s. validation. Otherwise mostly minor things.

* Returns `None` if any subtree that needed evaluation was an invalid
* `LinkTimeProperty` (i.e., one that does not pass the [[validate]] test).
*/
def tryEvalLinkTimeBooleanExpr(tree: Tree): Option[Boolean] =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is consistent with the current design of LinkTimeProperties, but IMO this really does not belong here.

In general, in the Scala.js linker, we separate data from it's related functionality. To me, it would feel much more natural to stop at the linkTimeProperties map.

I feel the structure here deserves some love:

  • With the introduction of desugar, this should probably be in frontend?
  • I'm unsure whether LinkTimeValue is worth the abstraction overhead: it does strip literals of their position, but it also seems to introduce quite some boilerplate (we could have linkTimeProperties: Map[String, Literal]).

I'm not entirely sure how to best proceed here. WDYT?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One motivation of the current design (in standard and with LinkTimeValue) is to support user configurable link-time properties in the future. I believe the data should stay here, under that assumption.

The logic that manipulates that data could probably move to frontend. But it's also required by the Analyzer, which is in analyzer, not frontend. So it would have to be private[linker] anyway. I'm not sure we would gain much by moving it.

For now I left this as is.

@gzm0
Copy link
Contributor

gzm0 commented Apr 26, 2025

Whoops, I forgot to review the last commit. Will do that now.

Copy link
Contributor

@gzm0 gzm0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of last commit.

@sjrd sjrd force-pushed the link-time-if-with-desugaring-phase branch from a34ef73 to 742b2fe Compare April 26, 2025 12:00
Copy link
Member Author

@sjrd sjrd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I addressed all the comments so far.

@@ -48,7 +48,7 @@ final class Analyzer(config: CommonPhaseConfig, initial: Boolean,
checkIRFor: Option[CheckingPhase], failOnError: Boolean, irLoader: IRLoader) {

private val infoLoader: InfoLoader =
new InfoLoader(irLoader, checkIRFor)
new InfoLoader(irLoader, checkIRFor, config.coreSpec)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LinkTimeProperties is private[linker]. We designed it like that when we introduced the LinkTimeProperty node. Its "API" was a bit messy, and we were not ready to commit to expose it.

But the methods to generate Infos are public (though allowed to change per our versioning policy). That means the type LinkTimeProperties cannot appear in the signatures of Infos.InfoGenerator. So we have to pass the entire CoreSpec instead, and hence we need to thread the full CoreSpec all the way from the Analyzer.

The same explanation applies to all your similar comments.

@@ -138,6 +139,20 @@ private[linker] object Desugarer {
case prop: LinkTimeProperty =>
coreSpec.linkTimeProperties.transformLinkTimeProperty(prop)

case LinkTimeIf(cond, thenp, elsep) =>
val cond1 = transform(cond)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. That's not needed.

* Returns `None` if any subtree that needed evaluation was an invalid
* `LinkTimeProperty` (i.e., one that does not pass the [[validate]] test).
*/
def tryEvalLinkTimeBooleanExpr(tree: Tree): Option[Boolean] =
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One motivation of the current design (in standard and with LinkTimeValue) is to support user configurable link-time properties in the future. I believe the data should stay here, under that assumption.

The logic that manipulates that data could probably move to frontend. But it's also required by the Analyzer, which is in analyzer, not frontend. So it would have to be private[linker] anyway. I'm not sure we would gain much by moving it.

For now I left this as is.

@sjrd sjrd requested a review from gzm0 April 26, 2025 12:04
@sjrd sjrd force-pushed the link-time-if-with-desugaring-phase branch from 742b2fe to 8e95468 Compare April 26, 2025 13:48
@gzm0
Copy link
Contributor

gzm0 commented May 4, 2025

My thoughts on the location/API of LinkTimeProperties (sorry for the disconnected block, I'm on Mobile):

  • let's design for what we have right now, not what we might have. In other words, let's ignore a potential ability to introduce user configurable link time properties for now (unless that would become impossible)
  • we seem to have a double use of the linker.standard package (semi stable api, common stuff for the linker). I think this hurts us now, where we need to pass more things (core spec) than strictly needed.
  • the analyzer and checker packages conceptually belong to frontend. IMO the absence of package hierarchy is not an issue, as long as we are lenient with visibility so checker / analyzer can access frontend stuff.

Proposal (in a separate PR)

  • Move LinkTimeProperties to frontend, make it public.
  • Move the LinkTimeProperties value out of CoreSpec (separate PR): the backend has no knowledge of them anymore, the fact that they are derived from the core spec is secondary
  • Move transformeLinkTime property to Desugarer (I'm assuming it's the only call site, otherwise I'm unsure what to do).
  • Move validate to Analyzer (assuming it's the only call site).
  • For both of the above, expose linkTimeProperties in LinkTimeProperties, maybe even make AnyVal or even just the Map.

I feel this will lead to a cleaner separation of logic and data. LMk what you think.

@sjrd
Copy link
Member Author

sjrd commented May 4, 2025

I pushed a commit that implements your proposal on top of what's currently in this PR. This is mostly for discussion. I assume that by "in another PR" you mean a PR that would happen before this one. So if you agree with the changes here I can extract the parts that would fit in main.

Overall this looks fine to me. But there still was no really good location for the logic that evaluates a link-time condition. I put it in the companion object of LinkTimeProperties and kept it private[linker].

@gzm0
Copy link
Contributor

gzm0 commented May 6, 2025

I assume that by "in another PR" you mean a PR that would happen before this one.

Yes.

So if you agree with the changes here I can extract the parts that would fit in main.

I think the changes look good and are an improvement.

But there still was no really good location for the logic that evaluates a link-time condition.

Maybe just a standalone object LinkTimeEvaluator?

I put it in the companion object of LinkTimeProperties and kept it private[linker].

I think that can also work.

@sjrd
Copy link
Member Author

sjrd commented May 7, 2025

Reorganization commit in #5163.

sjrd and others added 3 commits May 7, 2025 22:09
As well as the IR version to 1.20-SNAPSHOT.
Thanks to our optimizer's ability to inline, constant-fold, and
then eliminate dead code, we have been able to write link-time
conditional branches for a long time. Typical examples include
polyfills, as illustrated in the documentation of `LinkingInfo`:

    if (esVersion >= ESVersion.ES2018 || featureTest())
      useES2018Feature()
    else
      usePolyfill()

which gets folded away to nothing but

    useES2018Feature()

when linking for ES2018+.

However, this only works because both branches can *link* during
the initial reachability analysis. We cannot use the same technique
when one of the branches would refuse to link in the first place.
The canonical example is the usage of the JS `**` operator, which
does not link below ES2016. The following snippet produces good
code when linking for ES2016+, but does not link at all for ES2015:

    def pow(x: Double, y: Double): Double = {
      if (esVersion >= ESVersion.ES2016) {
        (x.asInstanceOf[js.Dynamic] ** y.asInstanceOf[js.Dynamic])
          .asInstanceOf[Double]
      } {
        Math.pow(x, y)
      }
    }

---

This commit introduces `LinkingInfo.linkTimeIf`, a conditional
branch that is guaranteed by spec to be resolved at link-time.
Using a `linkTimeIf` instead of the `if` in `def pow`, we can
successfully link the fallback branch on ES2015, because the then
branch is not even followed by the reachability analysis.

In order to provide that guarantee, the corresponding `LinkTimeIf`
IR node has strong requirements on its condition. It must be a
"link-time expression", which is guaranteed to be resolved at
link-time. A link-time expression tree must be of the form:

* A `Literal` (of type `int`, `boolean` or `string`, although
  `string`s are not actually usable here).
* A `LinkTimeProperty`.
* One of the boolean operators.
* One of the int comparison operators.
* A nested `LinkTimeIf` (used to encode short-circuiting boolean
  `&&` and `||`).

The `ClassDefChecker` validates the above property, and ensures
that link-time expression trees are *well-typed*. Normally that is
the job of the IR checker. Here we *can* do in `ClassDefChecker`
because we only have the 3 primitive types to deal with; and we
*must* do it then, because the reachability analysis itself is only
sound if all link-time expression trees are well-typed.

The reachability analysis algorithm itself is not affected by
`LinkTimeIf`. Instead, we resolve link-time branches when building
the `Infos` of methods. We follow only the branch that is taken.
This means that `Infos` builders now require access to the
`linkTimeProperties` derived from the `coreSpec`, but that is the
only additional piece of complexity in that area.

`LinkTimeIf`s nodes are later removed from the trees during
desugaring.

---

At the language and compiler level, we introduce
`LinkingInfo.linkTimeIf` as a primitive for `LinkTimeIf`. We need
a dedicated method to compile link-time expression trees, which
does incur some duplication, unfortunately. Other than that,
`linkTimeIf` is straightforward, by itself.

The problem is that the whole point of `linkTimeIf` is that we can
refer to *link-time properties*, and not just literals. However,
our link-time properties are all hidden behind regular method calls,
such as `LinkInfo.esVersion`. For optimizer-based branching with
`if`s, that is fine, as the method is always inlined, and the
optimizer can then see the constant. However, for `linkTimeIf`,
that does not work, as it does not follow the requirements of a
link-time expression tree.

If we were on Scala 3 only, we could declare `esVersion` and its
friends as an `inline def`, as follows:

    inline def esVersion: Int =
      linkTimePropertyInt("core/esVersion")

The `inline` keyword is guaranteed by the language to be resolved
at *compile*-time. Since the `linkTimePropertyInt` method is itself
a primitive replaced by a `LinkTimeProperty`, by the time we reach
our backend, we would see the latter, and all would be well.
The same cannot be said for the `@inline` optimizer hint, which is
all we have.

We therefore add another language-level feature:
`@linkTimeProperty`. This annotation can (currently) only be used
in our own library. By contract, it must only be used on a method
whose body is the corresponding `linkTimePropertyX` primitive.
With it, we can define `esVersion` as:

    @inline @linkTimeProperty("core/esVersion")
    def esVersion: Int =
      linkTimePropertyInt("core/esVersion")

That annotation makes the body public, in a way. That means the
compiler back-end can now replace *call sites* to `esVersion` by
the `LinkTimeProperty`.

Semantically, `@linkTimeProperty` does nothing more than guaranteed
inlining (with strong restrictions on the shape of body).

Co-authored-by: Rikito Taniguchi <[email protected]>
We use a `linkTimeIf` to select a `bigint`-based implementation of
`parseFloatDecimalCorrection` when they are supported. We need a
`linkTimeIf` in this case because it uses the JS `**` operator,
which does not link below ES 2016.

The `bigint`-based implementation avoids bringing in the entire
`BigInteger` implementation, which is a major code size win if that
was the only reason `BigInteger` was needed.
@sjrd sjrd force-pushed the link-time-if-with-desugaring-phase branch from a096130 to f0e7a33 Compare May 7, 2025 20:09
@sjrd
Copy link
Member Author

sjrd commented May 7, 2025

Rebased. I also introduced LinkTimeEvaluator, as you suggested. That seems nicer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants