#1790 - A suite annotated with `.only("some-label")`, will ignore all non matching labels #1838

dkarlinsky · 2019-09-26T13:32:23Z

New method Spec.only(), that when applied on a suite will ignore all tests that don't have the given substring in their label.
Requires evidence that L to be String and the Spec is ZSpec

Example:

import zio.test.TestOnly._
suite("suite")(
    test("test1") {
      assert(1, equalTo(1))
    },
    test("test2") {
      assert(1, equalTo(1))
    }
  ).only("test2")

~~This requires an outer suite to be marked with @@ thisOnlySupport.~~
~~Or TeatAspects.runOnlySupport needs to be added to default aspects.~~

~~To make this work I had to add a notion of tags to TestCase~~

~~@adamgfraser WDYT?~~

jdegoes · 2019-09-26T18:18:00Z

I understand this is a useful scenario to support, I think we should reflect on how we can do this (and if we can, in the right way now), since in this design we cannot enforce the "outer suite" requirement.

adamgfraser · 2019-09-26T18:28:03Z

One idea I had is once we get effectual suites in perhaps we can have a function like:

def testOnly[R, E, L, T](f: TestAspectPoly => Spec[R, E, L, T]): Spec[R, E, L, T] = {
  val specs = for {
    ref <- Ref.make[Vector[Spec[R, E, L, T]]
    only = new TestAspectPoly { ... }  // TestAspect that adds any tests it is called on to the ref
    _ <- f(only)
    specs <- ref.get
  } yield specs
  Spec.suite("Only", specs, None)

Then you could use it like:

testOnly { only =>
  suite("MySuite")(
    test1,
    test2 @@ only,
    test3
  )
}

jdegoes · 2019-09-28T11:38:18Z

@adamgfraser Yes, that's a much better API! It should work for now, and we can think about whether to do this with annotations later.

adamgfraser · 2019-10-04T03:11:18Z

@dkarlinsky We just merged #1664 which makes suites effectual so I think now we really have the tools to implement your idea. Do you want to rework this PR along the lines I suggested? Would be great to get some feedback on working with the new effectual suites functionality and I think this would be a nice feature to have.

dkarlinsky · 2019-10-04T05:34:53Z

Yes, I would like to work on this. I'm currently on vacation in Australia, though :) I do have my laptop with me, so I'll try to get some work in, but not sure how much.

…

On Fri, Oct 4, 2019, 1:11 PM Adam Fraser ***@***.***> wrote: @dkarlinsky <https://github.com/dkarlinsky> We just merged #1664 <#1664> which makes suites effectual so I think now we really have the tools to implement your idea. Do you want to rework this PR along the lines I suggested? Would be great to get some feedback on working with the new effectual suites functionality and I think this would be a nice feature to have. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1838?email_source=notifications&email_token=AAMBET2ZXEVVDOD3R3LH7ODQM2X57A5CNFSM4I22B4NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAKHAYY#issuecomment-538210403>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMBET75QFS5WPJ6HXRH5TDQM2X57ANCNFSM4I22B4NA> .

adamgfraser · 2019-10-04T10:03:53Z

@dkarlinsky Sounds good. No rush. Enjoy your vacation!

dkarlinsky · 2019-10-04T14:25:01Z

Thanks!

…

On Fri, Oct 4, 2019, 8:04 PM Adam Fraser ***@***.***> wrote: @dkarlinsky <https://github.com/dkarlinsky> Sounds good. No rush. Enjoy your vacation! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1838?email_source=notifications&email_token=AAMBET6GML75GYKCC5PORKLQM4IJNA5CNFSM4I22B4NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEALFTEQ#issuecomment-538335634>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMBETYJTRNLK64HUCQ6CT3QM4IJNANCNFSM4I22B4NA> .

jdegoes · 2019-10-25T22:19:59Z

I will close this, @dkarlinsky, feel free to re-open when you have time to add this great feature!

dkarlinsky · 2019-10-26T09:18:52Z

@jdegoes Back from vacation (landed yesterday) and now ready to give this another shot :)
Looks like I don't have permissions to reopen the PR, though...

jdegoes · 2019-10-26T11:37:36Z

Done!

dkarlinsky · 2019-10-27T09:20:54Z

Thank you!

…

On Sat, Oct 26, 2019 at 2:37 PM John A. De Goes ***@***.***> wrote: Done! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1838?email_source=notifications&email_token=AAMBET75SGMKR5DND3AMQZLQQQTYPA5CNFSM4I22B4NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECKGBGY#issuecomment-546594971>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMBETYZPIPGGDFVOOMFUV3QQQTYPANCNFSM4I22B4NA> .

adamgfraser

Nice progress on this. I left some comments below. I think we can both improve the ergonomics of this and simplify the implementation a lot by using filterLabels and either calling toString on the labels or requiring evidence that they are already strings so we can do a substring search on them. This will also make the API consistent with what we are doing in the SBT test runner for "only".

test-tests/shared/src/main/scala/zio/test/TestUtils.scala

test-tests/shared/src/test/scala/zio/test/TestOnlySpec.scala

test/shared/src/main/scala/zio/test/TestOnly.scala

adamgfraser

Just a few minor comments. Excited to get this in!

test/shared/src/main/scala/zio/test/Spec.scala

test-tests/shared/src/test/scala/zio/test/SpecSpec.scala

test/shared/src/main/scala/zio/test/Spec.scala

adamgfraser · 2019-11-09T14:56:30Z

@dkarlinsky We merged #2192 which created the same test file and is creating a merge conflict. Should be easy to drop your tests in.

Now `.only(labelFilter)` can be added to a suite. If no matching labels are found, the test fails with exception.

…ethod

* `.only()` will only work on ZSpec with L <: String. * Using `filterLabel()`. * Moved `only()` method into Spec (no longer an extension method)

adamgfraser

Looks great!

…all non matching labels (zio#1838) * Rewrote the whole thing. Now `.only(labelFilter)` can be added to a suite. If no matching labels are found, the test fails with exception. * Fixing method overloading issue in dotty, by renaming the offending method * added scala docs * Simplified implementation. * `.only()` will only work on ZSpec with L <: String. * Using `filterLabel()`. * Moved `only()` method into Spec (no longer an extension method) * Moved test to the newly created `SpecSpec` * adapting to ZSpec changes * scalafmt * minor fixes, based on PR comments * some more minor fixes, based on PR comments

jdegoes closed this Oct 25, 2019

jdegoes reopened this Oct 26, 2019

dkarlinsky force-pushed the 1790-only branch from 5de96ab to 7a20aef Compare November 3, 2019 11:09

dkarlinsky changed the title ~~#1790 - A test annotated with @@ thisOnly, will be the only one that runs~~ #1790 - A suite annotated with .only('some-label), will ignore all non matching labels Nov 3, 2019

dkarlinsky changed the title ~~#1790 - A suite annotated with .only('some-label), will ignore all non matching labels~~ #1790 - A suite annotated with .only("some-label"), will ignore all non matching labels Nov 3, 2019

adamgfraser suggested changes Nov 4, 2019

View reviewed changes

dkarlinsky force-pushed the 1790-only branch 2 times, most recently from a126bd8 to fb0c0cc Compare November 8, 2019 18:53

adamgfraser suggested changes Nov 9, 2019

View reviewed changes

adamgfraser reviewed Nov 9, 2019

View reviewed changes

test/shared/src/main/scala/zio/test/Spec.scala Outdated Show resolved Hide resolved

adamgfraser reviewed Nov 9, 2019

View reviewed changes

test-tests/shared/src/test/scala/zio/test/SpecSpec.scala Outdated Show resolved Hide resolved

adamgfraser reviewed Nov 9, 2019

View reviewed changes

test/shared/src/main/scala/zio/test/Spec.scala Outdated Show resolved Hide resolved

dkarlinsky added 6 commits November 9, 2019 20:26

Rewrote the whole thing.

e47aa6d

Now `.only(labelFilter)` can be added to a suite. If no matching labels are found, the test fails with exception.

Fixing method overloading issue in dotty, by renaming the offending m…

de79882

…ethod

added scala docs

7b85513

Simplified implementation.

5400536

* `.only()` will only work on ZSpec with L <: String. * Using `filterLabel()`. * Moved `only()` method into Spec (no longer an extension method)

Moved test to the newly created SpecSpec

cf031b7

adapting to ZSpec changes

7545dad

dkarlinsky added 3 commits November 9, 2019 20:27

scalafmt

7b0bdc0

minor fixes, based on PR comments

e7425e5

some more minor fixes, based on PR comments

17f7ab5

dkarlinsky force-pushed the 1790-only branch from 69054a6 to 17f7ab5 Compare November 9, 2019 18:32

adamgfraser approved these changes Nov 12, 2019

View reviewed changes

adamgfraser merged commit a2ac67f into zio:master Nov 12, 2019

dkarlinsky deleted the 1790-only branch November 12, 2019 18:52

adamgfraser mentioned this pull request Nov 18, 2019

A new TestAspect "only" to run single test in a Suite #1790

Closed

reibitto mentioned this pull request Mar 27, 2020

zio-test: @@ only TestAspect #3226

Open

Uh oh!

#1790 - A suite annotated with .only("some-label"), will ignore all non matching labels #1838

#1790 - A suite annotated with .only("some-label"), will ignore all non matching labels #1838

Uh oh!

Conversation

dkarlinsky commented Sep 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jdegoes commented Sep 26, 2019

Uh oh!

adamgfraser commented Sep 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jdegoes commented Sep 28, 2019

Uh oh!

adamgfraser commented Oct 4, 2019

Uh oh!

dkarlinsky commented Oct 4, 2019 via email

Uh oh!

adamgfraser commented Oct 4, 2019

Uh oh!

dkarlinsky commented Oct 4, 2019 via email

Uh oh!

jdegoes commented Oct 25, 2019

Uh oh!

dkarlinsky commented Oct 26, 2019

Uh oh!

jdegoes commented Oct 26, 2019

Uh oh!

dkarlinsky commented Oct 27, 2019 via email

Uh oh!

adamgfraser left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adamgfraser left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adamgfraser commented Nov 9, 2019

Uh oh!

adamgfraser left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

#1790 - A suite annotated with `.only("some-label")`, will ignore all non matching labels #1838

#1790 - A suite annotated with `.only("some-label")`, will ignore all non matching labels #1838

dkarlinsky commented Sep 26, 2019 •

edited

Loading

adamgfraser commented Sep 26, 2019 •

edited

Loading