Introduce`LinktimeProperty` and get rid of `JSLinkingInfo` #5025

tanishiking · 2024-08-30T12:34:16Z

Previously, accessing link-time information via scala.scalajs.runtime.linkingInfo required interacting with a JavaScript object. That introduced JS interop in the Wasm backend causing a slowdown, and could hinder the optimization pipeline (In fact, we folded the access to JSLinkingInfo into Literals in OptimizerCore).

This PR introduces

LinkTimeProperty that is a new IR node that is guaranteed to be transformed into a Literal at the optimizer or backend stage.
Add scala.scalajs.LinkingInfo.linkTimeProperty(Int|Boolean|String) APIs that emits LinkTimeProperty IRs.
Deprecate scala.scalajs.runtime.linkingInfo in favor of scala.scalajs.LinkingInfo and scala.scalajs.js.special.fileLevelThis.

For instance, linkTimePropertyInt("core/esVersion") emits the IR LinkTimeProperty("core/esVersion")(IntType), which will later be transformed into an IntLiteral by the optimizer or linker backend. (I didn't make it generic because generics makes it compilation a bit complicated for keeping it's type after erasure / the only types we want here are Literal types required by LinkingInfo).

For backward compatibility, we introduced a deserialization hacks:

Transforms JSSelect(JSLinkingInfo(), StringLiteral(...)) into the corresponding LinkTimeProperty.
An isolated JSLinkingInfo will be deserialized into a JSObjectConstr() containing the corresponding LinkTimeProperty values.

linker/shared/src/main/scala/org/scalajs/linker/standard/LinkTimeProperties.scala

ir/shared/src/main/scala/org/scalajs/ir/Trees.scala

project/JavalibIRCleaner.scala

library/src/main/scala/scala/scalajs/runtime/LinkingInfo.scala

library/src/main/scala/scala/scalajs/runtime/package.scala

library/src/main/scala/scala/scalajs/js/special/package.scala

ir/shared/src/main/scala/org/scalajs/ir/Trees.scala

compiler/src/main/scala/org/scalajs/nscplugin/GenJSCode.scala

linker/shared/src/main/scala/org/scalajs/linker/frontend/optimizer/OptimizerCore.scala

linker/shared/src/main/scala/org/scalajs/linker/analyzer/Infos.scala

tanishiking · 2024-09-02T13:55:09Z

Not yet written tests
I kinda implemented the deserialization hack, but how can I test the deserialization hack?

sjrd · 2024-09-02T14:04:26Z

I kinda implemented the deserialization hack, but how can I test the deserialization hack?

There are two things we do to test deserialization hacks.

First, as part of linker2_12/test, we run the org.scalajs.linker.BackwardsCompatTest suite. It loads the published libraries of all the previous 1.x versions and makes sure it can link trivial programs together with them.

The second and most important thing is we do two-commit PRs for these changes. In the first commit, we do all the changes except changes to the compiler and libraries. This way, we force the deserialization hacks to be necessary during that commit for all the tests to pass. The second commit then changes the compiler and libraries. You can see an example of this system in the current PR for Class_x operations: #4998.

ir/shared/src/main/scala/org/scalajs/ir/Serializers.scala

ir/shared/src/main/scala/org/scalajs/ir/Trees.scala

linker/shared/src/main/scala/org/scalajs/linker/standard/LinkTimeProperties.scala

compiler/src/main/scala/org/scalajs/nscplugin/GenJSCode.scala

ir/shared/src/main/scala/org/scalajs/ir/Trees.scala

sjrd · 2024-09-09T07:07:14Z

Is there any reason that the third commit (validation) is not squashed into the first commit (introduction of the IR node)?

tanishiking · 2024-09-09T07:52:31Z

I made it a separate commit since it includes some messy changes (passing LinkTimeProperties around) and thought it might be easier to review this way. However, I can squash it (for 2-stage testing?)

sjrd · 2024-09-09T17:06:01Z

Yes, it should at least be tested before the commit that changes the compiler, since we need the deserialization hack to produce valid code.

I think we should squash it even if it is messy. I don't think we have precedent for delayed validation for that reason.

An alternative is to first introduce non-functional refactorings (in this case passing new things around), then have the functional commit that uses those refactorings.

tanishiking · 2024-09-10T02:06:03Z

Squashed 👍

linker/shared/src/main/scala/org/scalajs/linker/backend/emitter/FunctionEmitter.scala

sjrd

Looks good overall. Here is a detailed review to iron out the details.

We also need new tests for user-defined globals with the name "this". Both through a dynamic-style access and an @JSGlobal("this") access. We also need to test direct js.typeOf(...) versions, since the latter produces special IR when the argument is a JSGlobalRef (it produces a JSTypeOfGlobalRef).

ir/shared/src/main/scala/org/scalajs/ir/Serializers.scala

linker/shared/src/main/scala/org/scalajs/linker/standard/LinkTimeProperties.scala

linker/shared/src/test/scala/org/scalajs/linker/AnalyzerTest.scala

project/JavalibIRCleaner.scala

test-suite/js/src/test/scala/org/scalajs/testsuite/library/LinkTimePropertyTest.scala

sjrd · 2024-09-20T16:38:05Z

test-suite/js/src/test/scala/org/scalajs/testsuite/library/LinkingInfoTest.scala

@@ -51,13 +51,4 @@ class LinkingInfoTest {
    assertEquals(11, ESVersion.ES2020)
    assertEquals(12, ESVersion.ES2021)
  }
-
-  @Test def isolatedJSLinkingInfo(): Unit = {


We should keep this test in the second commit, since the method still exists (as deprecated).

gzm0

Drive-by comment 🚗

linker/shared/src/main/scala/org/scalajs/linker/standard/LinkTimeProperties.scala

sjrd

You'll need to include a commit that changes the version numbers as first commit. For example
24d417a

sjrd · 2024-09-30T11:53:15Z

compiler/src/test/scala/org/scalajs/nscplugin/test/JSGlobalScopeTest.scala

@@ -342,7 +342,7 @@ class JSGlobalScopeTest extends DirectTest with TestHelpers {
        "extends", "false", "finally", "for", "function", "if", "implements",
        "import", "in", "instanceof", "interface", "let", "new", "null",
        "package", "private", "protected", "public", "return", "static",
-        "super", "switch", /*"this",*/ "throw", "true", "try", "typeof", "var",
+        "super", "switch", "throw", "true", "try", "typeof", "var",


This change should be integrated in the first commit.

Or rather: the first commit should not touch this. It's the second commit that allows this in global selections in the compiler; not the first one.

Right, the first commit shouldn't touch the test 😵‍💫 thanks!

Ah, no this change (remove "this" from the rejectAllReservedIdentifiers) have to be in the first commit.

Indeed we allow this in global selections in the compiler on second commit, but we change the requirements of the IR in the first commit (isValidJSGlobalRefName which is referred from the requirements of JSGlobalRef)

Ah but then I think we should add an explicit rejection of "this" in the compiler in the first commit (and then remove it in the second commit).

This way we know that the first commit does not change anything user-visible. Only the second commit introduces user-level changes.

force pushed the commits to mix this change into the first commit 🙇
d5c2f7f

Oops, I commented at a same timing

Ah but then I think we should add an explicit rejection of "this" in the compiler in the first commit (and then remove it in the second commit).
This way we know that the first commit does not change anything user-visible. Only the second commit introduces user-level changes.

👍

So we're going to have the following changes in the first commit

diff --git a/compiler/src/main/scala/org/scalajs/nscplugin/GenJSCode.scala b/compiler/src/main/scala/org/scalajs/nscplugin/GenJSCode.scala index ef5d20f12..c3dff7493 100644 --- a/compiler/src/main/scala/org/scalajs/nscplugin/GenJSCode.scala +++ b/compiler/src/main/scala/org/scalajs/nscplugin/GenJSCode.scala @@ -6758,7 +6758,7 @@ abstract class GenJSCode[G <: Global with Singleton](val global: G) implicit pos: Position): js.JSGlobalRef = { propName match { case js.StringLiteral(value) => - if (js.JSGlobalRef.isValidJSGlobalRefName(value)) { + if (js.JSGlobalRef.isValidJSGlobalRefName(value) && value != js.JSGlobalRef.FileLevelThis) { if (value == "await") { global.runReporting.warning(pos, s"$actionFull of the global scope with the name '$value' is deprecated.\n" +

and remove the change + "this" from JSGlobalScopeTest in the second commit

done, sorry for making a mess

ir/shared/src/main/scala/org/scalajs/ir/Printers.scala

ir/shared/src/main/scala/org/scalajs/ir/Serializers.scala

ir/shared/src/main/scala/org/scalajs/ir/Trees.scala

linker/shared/src/main/scala/org/scalajs/linker/analyzer/Analyzer.scala

compiler/src/test/scala/org/scalajs/nscplugin/test/OptimizationTest.scala

library/src/main/scala/scala/scalajs/runtime/package.scala

library/src/main/scala/scala/scalajs/runtime/LinkingInfo.scala

library/src/main/scala/scala/scalajs/runtime/package.scala

test-suite/js/src/test/scala/org/scalajs/testsuite/library/LinkTimePropertyTest.scala

tanishiking · 2024-10-01T06:52:54Z

Thank you for the thorough review! I think addressed all the feedbacks, and now waiting for the CI

sjrd

Cool, looks great!

I noticed a few tiny details, but otherwise LGTM.

@gzm0 Any last-minute concerns? I believe the public API on this is very limited at this point.

linker/shared/src/main/scala/org/scalajs/linker/standard/CoreSpec.scala

linker/shared/src/main/scala/org/scalajs/linker/standard/LinkTimeProperties.scala

ir/shared/src/test/scala/org/scalajs/ir/PrintersTest.scala

project/BinaryIncompatibilities.scala

test-suite/js/src/test/scala/org/scalajs/testsuite/jsinterop/MiscInteropTest.scala

gzm0 · 2024-10-01T12:12:30Z

Any last-minute concerns? I believe the public API on this is very limited at this point.

No, API is limited so looks good.

Is it deliberate that we do the compiler changes later?

tanishiking · 2024-10-01T12:16:09Z

Is it deliberate that we do the compiler changes later?

Yes, it is for testing deserialization hack 👍

#5025 (comment)

sjrd · 2024-10-01T12:16:38Z

Is it deliberate that we do the compiler changes later?

That's what we always do for these pairs of commits with deserialization hacks, isn't it? Or are you talking about something else?

ref: scala-js#5000 Previously, accessing link-time information via `scala.scalajs.runtime.linkingInfo` required interacting with a JavaScript object. That introduced JS interop in the Wasm backend causing a slowdown, and could hinder the optimization pipeline (In fact, we folded the access to `JSLinkingInfo` into `Literal`s in `OptimizerCore`). `LinkTimeProperty` is a new IR node that is guaranteed to be transformed into a `Literal` at the optimizer or backend stage. We plan to introduce a new primitive, such as `linkTimePropertyXXX`, which will be transformed into a `LinkTimeProperty` in a later commit. Additionally, we will update `scala.scalajs.LinkingInfo` to use `linkTimePropertyXXX` instead of `runtime.linkingInfo`, allowing us to eliminate the JS object when accessing link-time information. This commit also deprecates the `JSLinkingInfo` IR node. For backward compatibility, we introduced a deserialization hack that transforms `JSSelect(JSLinkingInfo(), StringLiteral(...))` into the corresponding `LinkTimeProperty`. An isolated `JSLinkingInfo` will be deserialized into a `JSObjectConstr()` containing the corresponding `LinkTimeProperty` values. Also, this commit introduces validation for `LinkTimeProperty` during reachability analysis. The Analyzer now verifies that the `LinkTimeProperty` in the IR has a valid name and type pair, ensuring it can be resolved using the provided link-time information. If an invalid `LinkTimeProperty` is detected, an error will be recorded in the `Analysis`

TLDR: - Add `scala.scalajs.LinkingInfo.linkTimeProperty` APIs that emits `LinkTimeProperty` IRs. - Deprecate `scala.scalajs.runtime.linkingInfo` in favor of `scala.scalajs.LinkingInfo` and `scala.scalajs.js.special.fileLevelThis`. - Keeping `scala.scalajs.runtime.linkingInfo` because Scala3 compiler looks like depends on the symbol of `runtime.linkingInfo` https://github.com/scala/scala3/blob/67cd3ebe3f70bb9045cc966af5b96a623a7da973/compiler/src/dotty/tools/backend/sjs/JSDefinitions.scala#L182-L183 This commit introduces new APIs under `scala.scalajs.LinkingInfo`: `linkTimeProperty(Int|String|Boolean)`. These APIs allow us to define link-time properties in a more optimized and JS-independent way, replacing the previous `scala.scalajs.runtime.linkingInfo`. For instance, `linkTimePropertyInt("core/esVersion")` emits the IR `LinkTimeProperty("core/esVersion")(IntType)`, which will later be transformed into an `IntLiteral` by the optimizer or linker backend. `scala.scalajs.runtime.linkingInfo` is now deprecated in favor of `scala.scalajs.LinkingInfo`. For cases where `scala.scalajs.runtime.linkingInfo.fileLevelThis` was used, users should migrate to `scala.scalajs.js.special.fileLevelThis`.

tanishiking commented Aug 30, 2024

View reviewed changes

linker/shared/src/main/scala/org/scalajs/linker/standard/LinkTimeProperties.scala Outdated Show resolved Hide resolved

tanishiking force-pushed the linktime-property branch from 1b2c4ab to e518c18 Compare August 30, 2024 12:43

tanishiking commented Aug 30, 2024

View reviewed changes

ir/shared/src/main/scala/org/scalajs/ir/Trees.scala Outdated Show resolved Hide resolved

project/JavalibIRCleaner.scala Outdated Show resolved Hide resolved

tanishiking mentioned this pull request Aug 30, 2024

#4997 Allow for link-time dispatching #5000

Closed

sjrd reviewed Aug 30, 2024

View reviewed changes

tanishiking force-pushed the linktime-property branch 2 times, most recently from 38c150d to 03e6b54 Compare September 2, 2024 12:55

tanishiking commented Sep 2, 2024

View reviewed changes

linker/shared/src/main/scala/org/scalajs/linker/analyzer/Infos.scala Outdated Show resolved Hide resolved

sjrd reviewed Sep 2, 2024

View reviewed changes

tanishiking force-pushed the linktime-property branch 7 times, most recently from e5c7f0a to 292e9ca Compare September 9, 2024 04:52

tanishiking changed the title ~~WIP: IntroduceLinktimeProperty and get rid of JSLinkingInfo~~ IntroduceLinktimeProperty and get rid of JSLinkingInfo Sep 9, 2024

tanishiking marked this pull request as ready for review September 9, 2024 06:57

tanishiking commented Sep 9, 2024

View reviewed changes

ir/shared/src/main/scala/org/scalajs/ir/Trees.scala Outdated Show resolved Hide resolved

tanishiking force-pushed the linktime-property branch from 292e9ca to aab4f4a Compare September 10, 2024 01:52

tanishiking commented Sep 18, 2024

View reviewed changes

linker/shared/src/main/scala/org/scalajs/linker/backend/emitter/FunctionEmitter.scala Show resolved Hide resolved

tanishiking force-pushed the linktime-property branch 3 times, most recently from e8c58cd to 4f21e7f Compare September 20, 2024 02:31

sjrd requested changes Sep 20, 2024

View reviewed changes

gzm0 reviewed Sep 24, 2024

View reviewed changes

linker/shared/src/main/scala/org/scalajs/linker/standard/LinkTimeProperties.scala Outdated Show resolved Hide resolved

tanishiking force-pushed the linktime-property branch 9 times, most recently from afa8ce1 to 3c0c519 Compare September 30, 2024 10:17

sjrd requested changes Sep 30, 2024

View reviewed changes

tanishiking force-pushed the linktime-property branch from 3c0c519 to 30105d0 Compare September 30, 2024 14:08

Bump the version to 1.18.0-SNAPSHOT for the upcoming changes.

9f0d899

tanishiking force-pushed the linktime-property branch from 30105d0 to f3b38df Compare October 1, 2024 06:40

sjrd requested changes Oct 1, 2024

View reviewed changes

tanishiking force-pushed the linktime-property branch from a73512f to e26f056 Compare October 1, 2024 08:49

sjrd approved these changes Oct 1, 2024

View reviewed changes

tanishiking force-pushed the linktime-property branch from 87b9b43 to d5c2f7f Compare October 1, 2024 12:06

tanishiking force-pushed the linktime-property branch from b8d607e to 7bf410d Compare October 1, 2024 12:34

sjrd approved these changes Oct 1, 2024

View reviewed changes

sjrd merged commit 441bde2 into scala-js:main Oct 2, 2024
3 checks passed

tanishiking deleted the linktime-property branch October 2, 2024 08:26

tanishiking mentioned this pull request Oct 10, 2024

Make Scala.js Wasm backend suitable for standalone Wasm VMs (a.k.a. support "server-side Wasm") #4991

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce`LinktimeProperty` and get rid of `JSLinkingInfo` #5025

Introduce`LinktimeProperty` and get rid of `JSLinkingInfo` #5025

tanishiking commented Aug 30, 2024 •

edited

Loading

tanishiking commented Sep 2, 2024

sjrd commented Sep 2, 2024

sjrd commented Sep 9, 2024

tanishiking commented Sep 9, 2024

sjrd commented Sep 9, 2024

tanishiking commented Sep 10, 2024 •

edited

Loading

sjrd left a comment

sjrd Sep 20, 2024

gzm0 left a comment

sjrd left a comment

sjrd Sep 30, 2024

sjrd Sep 30, 2024

tanishiking Oct 1, 2024

tanishiking Oct 1, 2024

sjrd Oct 1, 2024

tanishiking Oct 1, 2024

tanishiking Oct 1, 2024 •

edited

Loading

tanishiking Oct 1, 2024 •

edited

Loading

tanishiking Oct 1, 2024

tanishiking commented Oct 1, 2024 •

edited

Loading

sjrd left a comment

gzm0 commented Oct 1, 2024

tanishiking commented Oct 1, 2024

sjrd commented Oct 1, 2024

IntroduceLinktimeProperty and get rid of JSLinkingInfo #5025

IntroduceLinktimeProperty and get rid of JSLinkingInfo #5025

Conversation

tanishiking commented Aug 30, 2024 • edited Loading

tanishiking commented Sep 2, 2024

sjrd commented Sep 2, 2024

sjrd commented Sep 9, 2024

tanishiking commented Sep 9, 2024

sjrd commented Sep 9, 2024

tanishiking commented Sep 10, 2024 • edited Loading

sjrd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gzm0 left a comment

Choose a reason for hiding this comment

sjrd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tanishiking Oct 1, 2024 • edited Loading

Choose a reason for hiding this comment

tanishiking Oct 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tanishiking commented Oct 1, 2024 • edited Loading

sjrd left a comment

Choose a reason for hiding this comment

gzm0 commented Oct 1, 2024

tanishiking commented Oct 1, 2024

sjrd commented Oct 1, 2024

Introduce`LinktimeProperty` and get rid of `JSLinkingInfo` #5025

Introduce`LinktimeProperty` and get rid of `JSLinkingInfo` #5025

tanishiking commented Aug 30, 2024 •

edited

Loading

tanishiking commented Sep 10, 2024 •

edited

Loading

tanishiking Oct 1, 2024 •

edited

Loading

tanishiking Oct 1, 2024 •

edited

Loading

tanishiking commented Oct 1, 2024 •

edited

Loading