Usage examples

import zio.{UIO, ZIO}
import zio.test.mock.{Method, Mock, Mockable}

// module
trait Cache {
  val cache: Cache.Service[Any]
}

object Cache {
  // service
  trait Service[R] {
    def get(id: Int): ZIO[R, Nothing, String]
    def set(id: Int, value: String): ZIO[R, Nothing, Unit]
    val clear: ZIO[R, Nothing, Unit]
  }

  // capability tags
  object Service {
    case object get   extends Method[Int, String]
    case object set   extends Method[(Int, String), Unit]
    case object clear extends Method[Nothing, Unit]
  }

  // mock implementation
  implicit val mockable: Mockable[Cache] = (mock: Mock) =>
    new Cache {
      val cache = new Service[Any] {
        def get(id: Int): UIO[String]              = mock(Service.get)(id)
        def set(id: Int, value: String): UIO[Unit] = mock(Service.set)(id, value)
        val clear: UIO[Unit]                       = mock(Service.clear)
      }
    }

  // helper object for easy access to service capabilities
  object > extends Service[Cache] {
    def get(id: Int)                = ZIO.accessM(_.cache.get(id))
    def set(id: Int, value: String) = ZIO.accessM(_.cache.set(id, value))
    val clear                       = ZIO.accessM(_.cache.clear)
  }
}

The capability tags, mock implementation and the helper object is just some boilerplate that can be autogenerated by a macro. The macros are already available in zio-macros project.

With macros the example above can be shortened to:

import zio.ZIO
import zio.macros.access.Accessable
import zio.macros.mock.Mockable

@Accessable
@Mockable
trait Cache {
  val cache: Cache.Service[Any]
}

object Cache {
  trait Service[R] {
    def get(id: Int): ZIO[R, Nothing, String]
    def set(id: Int, value: String): ZIO[R, Nothing, Unit]
    val clear: ZIO[R, Nothing, Unit]
  }
}

Note: currently the macros project depends on a locally published version of zio, as it needs the Mock, Mockable and Method types from this PR, so you can't use it yet. As soon as this PR is merged and a new zio with its contents published I will update the macros project and it will become "usable".

Having all that machinery in place, this is how it would be used in tests:

import zio.{UIO, ZIO, ZManaged}
import zio.test.{assertM, suite, testM, DefaultRunnableSpec}
import zio.test.mock.MockSpec
import zio.test.Assertion.{anything, equalTo}

object CacheSpec extends DefaultRunnableSpec(

  suite("Cache")(
      testM("mock get 4 times") {
        import Cache.mockable
        val managedEnv = (
           MockSpec.expectM(Cache.Service.get)(equalTo(1))(_ => UIO.succeed("foo")) *>
           MockSpec.expectM_(Cache.Service.get)(equalTo(2))(UIO.succeed("bar")) *>
           MockSpec.expect(Cache.Service.get)(equalTo(3))(_ => "baz") *>
           MockSpec.expect_(Cache.Service.get)(equalTo(4))("acme")
        )
        val app =
          for {
            str1 <- Cache.>.get(1)
            str2 <- Cache.>.get(2)
            str3 <- Cache.>.get(3)
            str4 <- Cache.>.get(4)
          } yield s"$str1 $str2 $str3 $str4"

        managedEnv.use { env =>
          val result = app.provide(env).flatten
          assertM(result, equalTo("foo bar baz acme"))
        }
      }
    , testM("mock get > set > get") {
        import Cache.mockable
        val managedEnv = (
          MockSpec.expect_(Cache.Service.get)(equalTo(1))("foo") *>
          MockSpec.expectIn(Cache.Service.set)(equalTo(2 -> "bar"))  *>
          MockSpec.expect_(Cache.Service.get)(equalTo(3))("baz")
        )
        val app =
          for {
            str1 <- Cache.>.get(1)
            _    <- Cache.>.set(2, "bar")
            str2 <- Cache.>.get(3)
          } yield s"$str1 $str2"

        managedEnv.use { env =>
          val result = app.provide(env).flatten
          assertM(result, equalTo("foo baz"))
        }
      }
    , testM("mock clear > get") {
        import Cache.mockable
        val managedEnv = (
          MockSpec.expectOut_(Cache.Service.clear)(()) *>
          MockSpec.expect_(Cache.Service.get)(equalTo(1))("")
        )
        val app = Cache.>.clear *> Cache.>.get(1)

        managedEnv.use { env =>
          val result = app.provide(env).flatten
          assertM(output, equalTo(""))
        }
      }
  )
)

I think it is just we have a namespace conflict because all the existing mock objects use Mock as the name for their implementations. So like MockRandom.Mock is the actual implementation of the MockRandom service. We can probably come up with a different naming convention.

Anyway, will review in more detail later but very excited about this!

Added combinators and updated example in PR description.

Damn this is nice!

Looking good! I like how it reuses assertions 👍

Work in progress. Do not merge yet - I plan to add a few things today and tomorrow.

Ready for another round of review. Still on TODO list:

• fix CI bugs
• add examples
• add docs
• add capability tags and mockable implementation for core modules (Clock, Console, etc)
• rename/move mock modules to fake/stub following the semantics defined in Martin Fowlers post

Okay, will take a look tonight.

We should figure out what we want to do on the tests. (...) Regardless we should align one way or the other because it doesn't make much sense to have some tests written in ZIO Test and others written in the bootstrap testing framework.

As stated above, I think it's OK to test zio.test using itself as the test framework. Similar concept is largely estabilished in the compilers space, where once a compiler is able to compile itself it is considered a huge milestone and a testimony of its maturity.

I'd like to hear from others @iravid @ghostdogpr @jdegoes @neko-kai WDYT?

@sideeffffect That's really nice! Any idea why is seems to show the same failure twice?

@ioleo @adamgfraser

Personally, I have no real opinion on whether ZIO Test should be tested with ZIO Test itself. I think on the one hand, it makes it slightly more difficult for maintainers, because if one things break, many things break, and it can be hard to figure out what the root cause is; but on the other hand, it reduces code duplication and is a way of "dogfooding". Indeed, it is a sign of compiler maturity when a language can bootstrap itself, and this process generally increases contributions because the same people using the "language" can now contribute in the same "language".

My opinion is mainly that we should NOT mix two styles, that is, go all-in on one way or the other.

🤷‍♂️ sorry, @adamgfraser no idea, I've just enabled this feature recently and I am no expert in CircleCI

EDIT: only thing I know is that it's based on reports from test-reports from sbt: https://www.scala-sbt.org/1.x/docs/Testing.html#Test+Reports

I have the same issue with bitbucket pipelines at work (results reported twice) so I guess the issue is in test-reports?

@ioleo One more substantive comment. Would it make sense to add sequence/traverse methods to MockSpec? Right now it is easy to expect A, B, and thenC`. But if I want to expect a method to get called a hundred times there doesn't seem to be an easy way to do that other than implementing those combinators yourself. We could also definitely add in a subsequent PR.

@jdegoes Those are good points. I had liked the idea of having an "independent check" on ZIO Test but you are right that it would make it easier for contributors and would let us replace more ad hoc implementations of various testing functionality with the ones in ZIO Test. I also looked at some other testing frameworks and Scalacheck, Specs2, and Hedgehog all test against themselves (though Hedgehog has a funny note about this very issue).

Most of all I agree that we should only have one way of writing tests for ZIO Test. So if we are going to switch we should make a relatively concerted effort to move everything over. Maybe this could be a good issue for the next hackathon?

@ghostdogpr @adamgfraser @jdegoes Added documentation entries:

• howto/module_pattern.md
• howto/mocking_services.md

I'm not sure if I need to do something to have their links displayed in the left menu of "How to" section?
Please review.

Regarding the discussion:

• add sequence/traverse yes I intend to add more combinators, but this has already grown huge and I've already had to rebase it on several occasions (a lot of post-hackaton PRs getting merged 🎉), so lets leave this for future PRs (also, when it's "out" and we actually get to use it it will become more apparent which combinators we're missing)
• testing ZIO Test with itself I completely agree with @jdegoes and adding my 2 cents to this - during this PR I've had some issues (failing suites), however the "basic framework" output was not helpful, I had to add additional printlines and modify the test code a little to actually get an idea what went wrong - I know that ZIO Test's output is far more helpful (:heart: @adamgfraser ) and had we used it I'd struggle less to get this resolved
• ^^^ but I believe this is an issue for another PR (if we agree to move to ZIO test testing itself)

@ghostdogpr regarding the release notes, here is a quick overview what has changed:

• moved package zio.test.mock to zio.test.environment
• renamed MockConsole, MockClock, MockSystem and MockRandom to Test* equivalents (within the moved package)
• added mocking framework under zio.test.mock package
• added rendering of zio.test.mock.MockException defects in zio.test.DefaultTestReporter

If you need more detailed info I'll be on gitter tomorrow (aka in ~8h from now).

@ioleo you need to edit https://github.com/zio/zio/blob/master/website/sidebars.json for the sidebar.

I created a draft for v1.0.0-RC14 release notes with your information so that whoever does the release will have it.

You can run mdoc and docs/docusaurusCreateSite in sbt to check locally that the docs are okay.

@ioleo This looks great! Awesome job adding a ton of documentation. I think there is one comment from my previous review that is still outstanding and I left some minor comments on the documentation you added. Completely agree with you about adding the sequence and traverse methods in a subsequent PR. I'm a little concerned that the "how to" organizational structure is not going to scale as we get more documentation, but I think that and migrating the ZIO Test internal test suite to ZIO test are issues for another day. From my end I'm good to merge once the latest round of comments are resolved.

Rebased & resolved conflicts. Will get to comments now.

mdoc had failed on my examples as if it used old ZIO version (without the classes/traits from this PR), so I added the :fail tag to ignore the error

I think this can be merged now and we can deal with why was mdoc failing in a seperate PR.
@adamgfraser @ghostdogpr