Broad spectrum performance optimizations for `zio-test` #9404

kyri-petrou · 2024-12-13T02:45:10Z

This PR optimizes zio-test across most of its components with the main focus on:

Reducing the number of fibers created when invoking tests by using the scheduled executor directly for logging messages instead of forking fibers
Optimizes TestClock's performance. Codebases that rely heavily on it should see a significant decrease in test execution time with these changes
Reduce allocations and unnecessary flatmaps / effect evaluations wherever possible

…zio-performance

# Conflicts: # core/shared/src/main/scala/zio/ZIO.scala # project/MimaSettings.scala # test/shared/src/main/scala/zio/test/TestClock.scala # test/shared/src/main/scala/zio/test/TestReporters.scala # test/shared/src/main/scala/zio/test/TestServices.scala # test/shared/src/main/scala/zio/test/package.scala

# Conflicts: # test-sbt/jvm/src/main/scala/zio/test/sbt/ZTestRunnerJVM.scala # test-sbt/shared/src/main/scala/zio/test/sbt/ZTestEventHandlerSbt.scala # test/shared/src/main/scala/zio/test/ZIOSpecAbstract.scala

# Conflicts: # test/shared/src/main/scala/zio/test/TestRandom.scala # test/shared/src/main/scala/zio/test/ZIOSpecAbstract.scala # test/shared/src/main/scala/zio/test/package.scala

# Conflicts: # test-junit-engine/src/main/scala/zio/test/junit/ZIOTestClassRunner.scala # test/shared/src/main/scala/zio/test/Spec.scala # test/shared/src/main/scala/zio/test/TestSuccess.scala # test/shared/src/main/scala/zio/test/package.scala

# Conflicts: # test/shared/src/main/scala/zio/test/TestAspect.scala

hearnadam · 2025-03-22T22:41:15Z

test/shared/src/main/scala/zio/test/package.scala

+  private def warningEmptyGen[R, E, A](
+    f: (() => Boolean) => ZIO[R, E, A]
+  )(implicit trace: Trace): ZIO[R, E, A] =
+    ZIO.suspendSucceedUnsafe { implicit unsafe =>


I noticed there isn't a ZIO.runtimeWith[A] method, though if you used that here loggers could correctly be overridden.

So the thing here is that we really don't want them to be overridden, because the majority of users disable loggers in tests. But this warning must be printed out to the user.

This however begs the question: Why use a logger and not print directly with println? Frankly I can't think of a good enough answer to that, so perhaps we should change it?

Ah okay, that seems fine to me.

hearnadam · 2025-11-28T03:41:48Z

test/shared/src/main/scala/zio/test/Annotations.scala

  val tag: Tag[Annotations] = Tag[Annotations]

  final case class Test(ref: Ref.Atomic[TestAnnotationMap]) extends Annotations {
+    private implicit val unsafe0: Unsafe = Unsafe


Should we use a def to avoid the field?

I ended up inlining it where it was needed, it was only used in 3 places anyways.

hearnadam · 2025-11-28T03:43:36Z

test/shared/src/main/scala/zio/test/Annotations.scala

-        get(TestAnnotation.fibers).flatMap {
-          case Left(_) => ZIO.succeed(SortedSet.empty[Fiber.Runtime[Any, Any]])
+        get(TestAnnotation.fibers).map {
+          case Left(_) => SortedSet.empty[Fiber.Runtime[Any, Any]]


Suggested change

case Left(_) => SortedSet.empty[Fiber.Runtime[Any, Any]]

case _: Left[?] => SortedSet.empty[Fiber.Runtime[Any, Any]]

Moved this case at the bottom with a wildcard case _: instead

test/shared/src/main/scala/zio/test/Annotations.scala

hearnadam · 2025-11-28T03:54:09Z

core/jvm-native/src/main/scala/zio/ClockPlatformSpecific.scala

  }

-  private[this] def makeService(): ScheduledExecutorService = {
+  private[this] def makeService(): ScheduledThreadPoolExecutor = {


Is the type annotation name change necessary?

So the reason is because above we have this:

private[this] val service = makeService() // Type of `service` also changed from ScheduledExecutorService -> ScheduledThreadPoolExecutor

and then we have usages of service.schedule. I wanted to avoid the interface dispatch call so I changed the type to the concrete class instead. Since this is a private val it won't cause any issues

test/shared/src/main/scala/zio/test/TestClock.scala

hearnadam · 2025-11-28T04:15:40Z

test/shared/src/main/scala/zio/test/TestClock.scala

+          val f = ZIO
+            .logWarning(suspendedWarning)
+            .zipRight(suspendedWarningState.set(SuspendedWarningData.done))
+          val cancel = Clock.globalScheduler.schedule(() => Runtime.default.unsafe.run(f), 5.seconds)


Should we add support for running synchronously until we reach an async boundary? I suppose it should be the same scheduler, but it seems slightly incorrect to use the default runtime here.

f in this case is a synchronous effect (logging a message and setting the state of the Ref.

This brings me back to my previous comment whether we do need to use the logger or whether we can simply use println

I think we could use println, which would be much more lightweight in this context.

test/shared/src/main/scala/zio/test/TestClock.scala

hearnadam · 2025-12-01T23:28:56Z

test/shared/src/main/scala/zio/test/package.scala

-                            .onExecutor(Runtime.defaultExecutor)
-                 _ <- (child.interrupt *> fiber.interrupt).forkDaemon.onExecutor(Runtime.defaultExecutor)
-               } yield ()
+        _ <- ZIO.whenZIODiscard(child.hasChildrenAlive) {


Can you explain to me when this can happen? Won't the exitValue be non-null (since we already awaited promise completion, and thus never have children alive?

Won't the exitValue be non-null

Not necessarily. The promise will be completed when the effect has evaluated and produced a result, not when the Fiber completes. It is possible for the effect to be evaluated but for the fiber not to have completed yet in the case that there are children that need to be interrupted. See here:

zio/core/shared/src/main/scala/zio/internal/FiberRuntime.scala

Lines 444 to 464 in c519b61

val interruption = interruptAllChildren()

if (interruption eq null) {

if (inbox.isEmpty) {

finalExit = exit

if (supervisor ne Supervisor.none) supervisor.onEnd(finalExit, self)(Unsafe)

// No more messages to process, so we will allow the fiber to end life:

self.setExitValue(exit)

} else {

// There are messages, possibly added by the final op executed by

// the fiber. To be safe, we should execute those now before we

// allow the fiber to end life:

tell(FiberMessage.Resume(exit))

}

effect = null

} else {

effect = interruption.flatMap(_ => exit)(id.location)

}

Ah, that makes sense.

hearnadam · 2025-12-01T23:30:06Z

test/shared/src/main/scala/zio/test/package.scala

+  private def warningEmptyGen[R, E, A](
+    f: (() => Boolean) => ZIO[R, E, A]
+  )(implicit trace: Trace): ZIO[R, E, A] =
+    ZIO.suspendSucceedUnsafe { implicit unsafe =>


Ah okay, that seems fine to me.

test/shared/src/main/scala/zio/test/TestAspect.scala

test/shared/src/main/scala/zio/test/TestClock.scala

# Conflicts: # test/shared/src/main/scala/zio/test/ZIOSpecAbstract.scala

hearnadam · 2025-12-24T17:13:38Z

test/shared/src/main/scala/zio/test/TestAspect.scala

-            else ZIO.fail(TestFailure.die(new RuntimeException("did not fail as expected"))),
-          _ => ZIO.fail(TestFailure.die(new RuntimeException("did not fail as expected")))
+            if (assertion(failure)) TestSuccess.succeedEmptyExit
+            else Exit.fail(TestFailure.die(new RuntimeException("did not fail as expected"))),


pre-existing: I feel like these should render the failure, because IIRC in this case or the die it does not.

I tried to figure out a way to render the failure in a meaningful way and couldn't, so instead I decided to re-raise the failure and document this behaviour. I think that should be better UX-wise than what we currently have.

hearnadam · 2025-12-24T17:19:49Z

test/shared/src/main/scala/zio/test/TestClock.scala

-              case _                                           => ZIO.fail(())
-            }
+    private def freeze: IO[Unit, collection.Map[FiberId, Fiber.Status.Suspended]] = {
+      implicit val trace: Trace = Trace.empty


Generally, should we be using val or def for these locally scoped implicits?

Yeah good question. Ideally we'd want to inline them, but since that's not possible in a way that can be cross-compiled I think the next-best thing is to use a def and rely on the JVM to inline the method body.

Although I think we should probably do this in a separate PR as there will be a lot of places that we'd need to do that

hearnadam · 2025-12-24T17:24:16Z

test/shared/src/main/scala/zio/test/TestClock.scala

+          val f = ZIO
+            .logWarning(suspendedWarning)
+            .zipRight(suspendedWarningState.set(SuspendedWarningData.done))
+          val cancel = Clock.globalScheduler.schedule(() => Runtime.default.unsafe.run(f), 5.seconds)


I think we could use println, which would be much more lightweight in this context.

kyri-petrou · 2025-12-27T07:13:58Z

@hearnadam I added non-effectful logging of warning messages (see 0f32e0f).

I kept more-or-less the original format of the log message (without adding timestamps etc) just because I didn't want to change the behaviour too much.

kyri-petrou added 6 commits December 13, 2024 07:17

Broad scope performance optimizations for zio-test

9f2a087

Add exclusions to mima

68d5ee5

Fix errors

0a835b5

Poll multiple times for suspended fibers

2d24282

Optimize TestClock#awaitSuspended

10e7515

Increase sleep time

93ff2ab

kyri-petrou mentioned this pull request Dec 14, 2024

Ensure that TestClock methods are thread-safe #9408

Merged

kyri-petrou and others added 16 commits December 15, 2024 07:34

Merge remote-tracking branch 'upstream/series/2.x' into improve-test-…

b699b01

…zio-performance

Change TestClock sleep time and cleanups

d17617a

fmt

cdc3bef

Fix merge errors

aa20d7d

Increase sleep timer

cd1fbd8

Recurse outside of par methods

0b9391c

Fix mima and JS

39a4414

Reduce parallelism of ZStreamSpec

52cfd13

Fix for Scala 2.12

a36530e

Merge branch 'series/2.x' into improve-test-zio-performance

8166ade

Reduce sleep timer

bb7444f

Increase sleep timer again

f96048c

Add yielding

09bb888

Merge branch 'series/2.x' into improve-test-zio-performance

a098a6f

Increase sleep timer

35337ec

kyri-petrou mentioned this pull request Feb 11, 2025

zio-test: TestClock can hang when parallel tests adjust time #9512

Open

kyri-petrou added 5 commits February 11, 2025 04:30

Increase sleep timer

6d4e33f

Merge branch 'series/2.x' into improve-test-zio-performance

85439f4

Increase sleep timer

16fabec

Deprecate TestAspect.fibers and minor optimizations

79d7556

Merge branch 'series/2.x' into improve-test-zio-performance

5c2d192

kyri-petrou mentioned this pull request Apr 14, 2025

Inefficient output of results in zio-test #9783

Closed

kyri-petrou added 13 commits September 1, 2025 04:42

Fix merge conflicts

17423f3

Cleanups

c854768

Fix for Scala 3

4633320

Improve TestClock#run

930a663

Remove debug

a32d61b

Increase sleep time and move permit

915fd37

Merge branch 'series/2.x' into improve-test-zio-performance

5e6ee44

# Conflicts: # test-sbt/jvm/src/main/scala/zio/test/sbt/ZTestRunnerJVM.scala # test-sbt/shared/src/main/scala/zio/test/sbt/ZTestEventHandlerSbt.scala # test/shared/src/main/scala/zio/test/ZIOSpecAbstract.scala

Fix merge conflict

959c0d8

Merge branch 'series/2.x' into improve-test-zio-performance

c091cdd

# Conflicts: # test/shared/src/main/scala/zio/test/TestRandom.scala # test/shared/src/main/scala/zio/test/ZIOSpecAbstract.scala # test/shared/src/main/scala/zio/test/package.scala

Fix merge conflicts

c603c60

Fix console rendering for JS/Native

a3252f6

Fix merge conflicts

4d770b4

kyri-petrou marked this pull request as ready for review November 26, 2025 07:37

kyri-petrou added 3 commits November 26, 2025 10:34

Increase TestClock sleep timer and cleanups

1cbbd76

Merge branch 'series/2.x' into improve-test-zio-performance

c519b61

# Conflicts: # test/shared/src/main/scala/zio/test/TestAspect.scala

Make aspects into vals

9784273

hearnadam reviewed Nov 29, 2025

View reviewed changes

kyri-petrou added 3 commits November 29, 2025 06:14

Merge branch 'series/2.x' into improve-test-zio-performance

9d48d4d

PR comments

6846419

Merge branch 'series/2.x' into improve-test-zio-performance

c54247e

hearnadam reviewed Dec 6, 2025

View reviewed changes

kyri-petrou added 2 commits December 9, 2025 04:06

PR comments

abfbcf8

Merge branch 'series/2.x' into improve-test-zio-performance

1191b2f

# Conflicts: # test/shared/src/main/scala/zio/test/ZIOSpecAbstract.scala

kyri-petrou requested a review from hearnadam December 24, 2025 07:56

hearnadam previously approved these changes Dec 24, 2025

View reviewed changes

Add non-effectful test warning logging

0f32e0f

kyri-petrou dismissed hearnadam’s stale review via 0f32e0f December 27, 2025 07:12

hearnadam approved these changes Dec 27, 2025

View reviewed changes

	case Left(_) => SortedSet.empty[Fiber.Runtime[Any, Any]]
	case _: Left[?] => SortedSet.empty[Fiber.Runtime[Any, Any]]

	val interruption = interruptAllChildren()

	if (interruption eq null) {
	if (inbox.isEmpty) {
	finalExit = exit

	if (supervisor ne Supervisor.none) supervisor.onEnd(finalExit, self)(Unsafe)

	// No more messages to process, so we will allow the fiber to end life:
	self.setExitValue(exit)
	} else {
	// There are messages, possibly added by the final op executed by
	// the fiber. To be safe, we should execute those now before we
	// allow the fiber to end life:
	tell(FiberMessage.Resume(exit))
	}

	effect = null
	} else {
	effect = interruption.flatMap(_ => exit)(id.location)
	}

Broad spectrum performance optimizations for zio-test #9404

Are you sure you want to change the base?

Broad spectrum performance optimizations for zio-test #9404

Conversation

kyri-petrou commented Dec 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kyri-petrou Nov 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kyri-petrou commented Dec 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Broad spectrum performance optimizations for `zio-test` #9404

Broad spectrum performance optimizations for `zio-test` #9404

kyri-petrou commented Dec 13, 2024 •

edited

Loading

kyri-petrou Nov 29, 2025 •

edited

Loading