All committers have signed the CLA.

I think this use case is already addressed by the timed test aspect.

Thanks, timed does the job, I changed the implementation so that duration is extracted from the timing annotation.

I still have a problem though, the duration of a failing test is reported as 0. (I rebased on master so my branch already includes this fix: #4988). The timing annotation seems to disappear before the event is reported to sbt. Could it be caused by suite transformations, e.g. the one here:

@MichalPawlicki No that line of code only runs if the suite can't be created at all (e.g. the suite of tests was supposed to be generated based on contents from a file and couldn't be loaded). The time of failing tests is being reported correctly in tests with SBT so I would look in the implementation of the JUnit runner itself.

So basically, this seems to work. When I break a random test, annotate it with timed, and run it with the sbt testOnly task,
sbt generates a report in target/test-reports which includes the failure with a duration:

<testcase classname="zio.test.TestAspectSpec" name="around evaluates tests inside context of Managed" time="0.208">
  <failure message="No Exception or message provided"/>
</testcase>

I'm just having problems with making it work in ZTestFrameworkSpec. I'll look further into this.

I spent some time debugging it, but I can't get the assertion in ZTestFrameworkSpec to work. I guess there's some subtle difference between running testOnly (or test) from sbt and what happens in ZTestFrameworkSpec. The only thing that I found is that sbt test creates a TaskDef with a SuiteSelector (like here), but changing this in ZTestFrameworkSpec still doesn't help. At this point TestAnnotationMap no longer contains the timing annotation for a failed test .

(Note: this is not an issue with JUnit runner, but with JUnit test reports generated by default by sbt).

Any idea what else might be wrong? Would it be an option to merge the change in ZTestEvent without adding a test in ZTestFrameworkSpec, so that users could use it?

@MichalPawlicki That's fine.

@adamgfraser this is ready for review. 🙂 In the end, I added a test, but just for a passing spec.