Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Deadlock in concurrent initialization of ir.Names and ir.Types. #5135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
sjrd opened this issue Mar 3, 2025 · 5 comments
Closed

Deadlock in concurrent initialization of ir.Names and ir.Types. #5135

sjrd opened this issue Mar 3, 2025 · 5 comments
Assignees
@sjrd
Labels
bug Confirmed bug. Needs to be fixed.
Milestone
v1.19.0

Comments

@sjrd
Copy link
Member

sjrd commented Mar 3, 2025

Our mutual initialization of ir.Names and ir.Types can cause a deadlock if both start concurrently from two different threads. Here is the relevant portion of a thread dump:

"pool-152-thread-1" #357 prio=5 os_prio=0 cpu=28.62ms elapsed=11.37s tid=0x00007fa9dd0fa590 nid=0x45b88 in Object.wait()  [0x00007fa846ffb000]
   java.lang.Thread.State: RUNNABLE
        at org.scalajs.ir.Types$.<init>(Types.scala:443)
        - waiting on the Class initialization monitor for org.scalajs.ir.Names$
        at org.scalajs.ir.Types$.<clinit>(Types.scala)
        at org.scalajs.ir.TestIRBuilder$.<init>(TestIRBuilder.scala:73)
        at org.scalajs.ir.TestIRBuilder$.<clinit>(TestIRBuilder.scala)
        at org.scalajs.ir.HashersTest.<init>(HashersTest.scala:48)
        ...

"pool-152-thread-4" #361 prio=5 os_prio=0 cpu=45.37ms elapsed=11.37s tid=0x00007fa9dd340a00 nid=0x45b8d in Object.wait()  [0x00007fa8487fb000]
   java.lang.Thread.State: RUNNABLE
        at org.scalajs.ir.Names$MethodName$.constructor(Names.scala:508)
        - waiting on the Class initialization monitor for org.scalajs.ir.Types$
        at org.scalajs.ir.Names$.<init>(Names.scala:656)
        at org.scalajs.ir.Names$.<clinit>(Names.scala)
        at org.scalajs.ir.Names$ClassName$.apply(Names.scala:550)
        at org.scalajs.ir.Names$ClassName$.apply(Names.scala:553)
        at org.scalajs.ir.Serializers$.<init>(Serializers.scala:54)
        at org.scalajs.ir.Serializers$.<clinit>(Serializers.scala)
        at org.scalajs.ir.Serializers$Hacks.<init>(Serializers.scala:2587)
        at org.scalajs.ir.SerializersTest.testHacksUseBelow(SerializersTest.scala:22)
        ...

I had occasionally noticed ir2_12/test to be stuck, but I had always attributed that to an sbt glitch. Fortunately (in a sense), adding new tests for Names in #5003 turned the likelihood of this happening from "very rare" to "very often". That let me diagnose the issue properly.

The problem is that we have an inter-dependency between the constructor of object Names and that of object Types.

For "Names depends on Types": in the constructor of object Names we have

scala-js/ir/shared/src/main/scala/org/scalajs/ir/Names.scala

Lines 653 to 666 in cf05126

final val NoArgConstructorName: MethodName =
MethodName.constructor(Nil)
/** This is used to construct a java.lang.Class. */
final val ObjectArgConstructorName: MethodName =
MethodName.constructor(List(ClassRef(ObjectClass)))
/** Name of the static initializer method. */
final val StaticInitializerName: MethodName =
MethodName(SimpleMethodName.StaticInitializer, Nil, VoidRef)
/** Name of the class initializer method. */
final val ClassInitializerName: MethodName =
MethodName(SimpleMethodName.ClassInitializer, Nil, VoidRef)

which requires the Types.XRefs of primitive types. Those are vals in object Types, so we need to initialize object Types (this wouldn't happen if they were objects, but they are vals because they are instances of a case class).

For "Types depends on Names": in the constructor of object Types we have

scala-js/ir/shared/src/main/scala/org/scalajs/ir/Types.scala

Lines 383 to 397 in cf05126

val BoxedClassToPrimType: Map[ClassName, PrimType] = Map(
BoxedUnitClass -> UndefType,
BoxedBooleanClass -> BooleanType,
BoxedCharacterClass -> CharType,
BoxedByteClass -> ByteType,
BoxedShortClass -> ShortType,
BoxedIntegerClass -> IntType,
BoxedLongClass -> LongType,
BoxedFloatClass -> FloatType,
BoxedDoubleClass -> DoubleType,
BoxedStringClass -> StringType
)
val PrimTypeToBoxedClass: Map[PrimType, ClassName] =
BoxedClassToPrimType.map(_.swap)

which similarly accesses vals defined in object Names.

This is not an issue in sequential initialization, because the interdependencies only require vals defined before the cycle starts anew. So whether we start from Names or from Types, we get all the things initialized in the right order. But in a concurrent scenario, the global initialization locks on JVM class initializers (rightly) create a deadlock.

We need to break this cycle somehow. My best idea so far is to move all those vals to a third object WellKnownNames, that neither Types nor Names is allowed to depend on.
@sjrd sjrd added the bug Confirmed bug. Needs to be fixed. label Mar 3, 2025
@sjrd sjrd added this to the v1.19.0 milestone Mar 3, 2025
@sjrd sjrd self-assigned this Mar 3, 2025
@gzm0
Copy link
Contributor

gzm0 commented Mar 9, 2025

I've had a look at this, and it seems like the abstraction culprit is MethodName: Method names depend on types, types do not depend on method names (and lest we introduce structural subtyping, this is not going to change). However, types depend on other names (notably class names).

Because we group MethodName / ClassName (and their well-known names) into Names, a circular dependency emerges. Have you considered moving the well-known names into the companion of their relevant type?

You will also note (unless I made a mistake thinking about this), that doing so will make all the "it is OK to use well known names" comments go away in #5136.

@sjrd
Copy link
Member Author

sjrd commented Mar 9, 2025

I considered that, but it would be inconvenient. We often want to import "all" well-known names with a wildcard import. Or even just all well-known ClassNames. If the well-known names are in the companion objects, and we wildcard-import from them, we'll get other unwanted things, like the apply methods. We could of course use

import ClassName.{apply => _, _}

everywhere. I don't think that's great, though.

@gzm0
Copy link
Contributor

gzm0 commented Mar 9, 2025

Ah, yes. That's a bit annoying, indeed. What about ClassNames and MethodNames? (IMO the well-known can be implied).

I just feel that putting everything in WellKnownNames only masks the problem.

I'm also wondering if the choice of putting all names inside Names is really helpful: All types / companions already have Name as a suffix of their name :-/

@sjrd
Copy link
Member Author

sjrd commented Mar 9, 2025

I just feel that putting everything in WellKnownNames only masks the problem.

Hum, on the contrary. If there is still a problem somewhere, the fact that they're all in a single WellKnownNames object ensures that we will deterministically get a crash, even in fully sequential runs.

If we have ClassNames and MethodNames, we could end up with a similar situation if they inadvertently depend on each other.

@gzm0
Copy link
Contributor

gzm0 commented Mar 9, 2025

Ouch... I have not thought about it this way. But yes indeed, being able to detect the problem in the future clearly trumps "nice" organization.

I have to admit, I feel this is quite sad in a language focusing so much on static analysis like Scala :-/

I'll review #5136 in detail then.

@gzm0 gzm0 closed this as completed in 82910a0 Mar 10, 2025
gzm0 added a commit that referenced this issue Mar 10, 2025
@gzm0
Merge pull request #5136 from sjrd/fix-ir-names-types-deadlock
2ed0b39 
Fix #5135: Move well-known names to a new object WellKnownNames.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sjrd sjrd

Labels
bug Confirmed bug. Needs to be fixed.
Projects
None yet
Milestone
v1.19.0
Development

No branches or pull requests
2 participants
@sjrd @gzm0