The Effect Pattern and Effect Systems in Scala

Introduction

We love functional programming for good reasons. Pure functions are predictable. The substitution model makes reasoning easy. Referential transparency means we can replace expressions with their values without changing program behavior. But here’s the tension that every functional programmer faces: the real world is messy. We need randomness, database queries, HTTP requests, and file I/O. How do we reconcile functional purity with the side effects that make our programs useful?

Let’s explore the different approaches to managing side effects in Scala, from the established monadic effect systems like Cats Effect and ZIO to the novel direct-style effect handlers enabled by Scala 3’s context functions.

Setup

The code in this article uses several libraries to illustrate different approaches. To follow along, ensure you have Scala 3 set up in your development environment. Moreover, let’s add the following dependencies to our build.sbt file:

1
libraryDependencies ++= Seq(
2
  "org.typelevel" %% "cats-effect" % "3.6.3",
3
  "dev.zio"       %% "zio"         % "2.1.24",
4
  "io.getkyo"     %% "kyo-core"    % "1.0-RC1"
5
)

The Problem: Side Effects Break Purity

Let’s take an example. Imagine a drunk gambler at a casino who wants to flip a coin. Because they’re drunk, the flip might go wrong. They might drop the coin, flip it into their drink, or miss the catch entirely. In code, this might look like:

1
import scala.util.Random
2

3
def drunkFlip(): String = {
4
  val caught = Random.nextBoolean()
5
  val heads =
6
    if (caught) Random.nextBoolean()
7
    else throw new Exception("We dropped the coin!")
8
  if (heads) "Heads" else "Tails"
9
}

This code has problems that go beyond the gambler’s alcohol consumption. It uses randomness, so calling it twice might yield different results. It throws exceptions, which can break our program in unexpected ways. We can’t test it without actual randomness. We can’t compose it safely with other operations because it might explode at any moment.

To understand why this is so problematic, we need to talk about referential transparency. An expression is referentially transparent if we can replace it with its value without changing the program’s behavior. In other words, if val x = expr and we can freely substitute x with expr (and vice versa) everywhere in our code without altering what the program does, then expr is referentially transparent. This property is the foundation of the substitution model, which lets us reason about programs by replacing expressions with their values, step by step, just like simplifying an algebraic equation.

Let’s try to apply the substitution model to our drunk flip. Consider the following code:

1
val result1 = drunkFlip()  // Might throw, might return "Heads" or "Tails"
2
val result2 = result1      // Same value as result1

At first glance, result1 and result2 always hold the same value. Now, if drunkFlip() were referentially transparent, we should be able to substitute result1 with its definition drunkFlip() in the second line without changing the program’s behavior. Let’s try:

1
val result1 = drunkFlip()  // Say it returns "Heads"
2
val result2 = drunkFlip()  // A fresh call — might return "Tails" or throw!

The program changed. Before the substitution, result2 was guaranteed to equal result1. After the substitution, result2 might be different because each call to drunkFlip() can produce a different result or even throw an exception. We replaced equals with equals and got a different program. The substitution model breaks down, and drunkFlip() is not referentially transparent.

Why does this matter so much? Because referential transparency enables two powerful reasoning techniques. The first is equational reasoning: the ability to understand code by substituting equals for equals, just like we do in mathematics. If val x = 2 + 3, we know x is 5, and we can replace x with 5 anywhere. With pure functions, we can do the same. With drunkFlip(), we can’t. Each “equal” expression might evaluate to something different, so our algebraic reasoning breaks down.

The second technique is local reasoning: the ability to understand a piece of code by looking only at that piece, without needing to know what the rest of the program is doing. When functions are pure, we can read a function’s signature and body and understand exactly what it does. With side effects, that’s no longer possible. Our drunkFlip() function depends on hidden global state (the random number generator) and produces hidden control flow (the exception). To understand what happens when we call it, we need to consider the entire program’s state. Side effects destroy locality.

Side effects have invaded our pure functional world. We can’t apply the substitution model. We lose equational and local reasoning. We say that the function is impure, and impurity spreads like a virus through our codebase.

Is it all doom and gloom? Not at all. Functional programming has a solution for managing side effects while preserving purity.

The solution is the Effect Pattern, a technique for tracking and controlling side effects. The pattern has two core principles. First, type tracking means effects are encoded in types, so we can see what a function does just by looking at its signature. Second, deferred execution means building the effect description separately from running it. But how do we implement this pattern? The Scala ecosystem offers several approaches, each with different trade-offs. Some use the traditional monadic style with flatMap and map. Others leverage Scala 3’s context functions to make effects look almost like imperative code. We’ll explore both paths and understand when each makes sense.

The Effect Pattern Foundation

The Effect Pattern rests on two fundamental principles that transform how we handle side effects. First, type tracking means we encode effects in types. Instead of def getUser(): User, we write def getUser(name: String): IO[User] or def getUser(name: String): ZIO[Database, Exception, User]. The type signature tells us this function performs effects. We can see what kind of effects it uses without reading the implementation. If we use the IO type, we’re saying “this function might do side effects when executed”. If we use ZIO[Database, Exception, User], we’re saying “this function needs the capability to access to a Database, might fail with an Exception, and produces a User on success”. This explicitness is powerful. It makes reasoning about code easier. (Don’t worry, you don’t need to know what Cats Effect IO or ZIO ZIO are to understand the concepts we’ll explore).

The type system becomes documentation.

Second, deferred execution means we separate description from execution. Think of it like a recipe versus cooking. A recipe describes steps: mix flour and eggs, bake at 350 degrees, and frost when cool. But the recipe doesn’t bake anything. It’s just information on paper. Cooking is when you actually perform those steps and produce food. Similarly, an effect is a description: read this file, parse this JSON, query this database. But nothing happens when you create the effect. Execution happens later, when you explicitly run it at the boundaries of your system.

So, if we call the getUser function we defined earlier, we get back an effect description. Nothing happens yet:

1
val user: IO[User] = getUser("rcardin")  // Just a description, no execution

Why go through this effort? The benefits are concrete and immediate. Consider referential transparency first. As we said, without the Effect Pattern, our drunk flip breaks substitution. With the Effect Pattern, we get referential transparency back:

1
val result1 = IO(drunkFlip())  // A description, not an execution
2
val result2 = result1
3
// result1 and result2 are identical descriptions
4
// The substitution model works again!

Testability improves dramatically. We can swap in mock implementations instead of running real effects. Need to test code that queries a database? Provide a test handler that returns predictable values. No actual database required. No HTTP calls to external services. No waiting for file I/O. Tests run fast and deterministically.

Composition becomes natural and safe. We chain effects together using combinators. Build complex workflows from simple pieces. Retry on failure, run operations in parallel, and add timeouts. All without changing the core logic:

1
// Effect composition with retry policy
2
val resilientUser: IO[User] =
3
  getUser("rcardin")
4
    .timeout(5.seconds)
5
    .retry(Schedule.exponential(1.second).jittered && Schedule.recurs(3))
6
    .handleErrorWith { error =>
7
      IO.println(s"Failed to fetch user: $error") *>
8
        IO.raiseError(error)
9
    }

It’s not important which specific library we use. The above code wrapped the getUser effectful operation with retries, timeouts, and error handling, and we still have an IO[User] back.

Now that we understand what the Effect Pattern gives us, how do we actually implement it? The implementation of the Effect Pattern is called an Effect System. As we said, we need two key features: type tracking and deferred execution. Then, we need structures to manage and execute effects.

The Scala ecosystem has explored several approaches, each with different trade-offs. Let’s start with the most established path: monadic effect systems.

Monadic Effect Systems

The most established approach to effect systems belongs to functional programming and it uses monads (if you need a refresher on monads, check out the article An Introduction to Monads in Scala). If we’ve worked with Scala’s Option, Try, or Future, we’ve already used monads. The pattern is consistent across all of them: wrap computations in a type, chain them with flatMap and map, and run them when ready. Monadic effect systems apply this familiar pattern to side effects. Let’s explore three major implementations that each solve the Effect Pattern in slightly different ways.

1
import cats.effect.IO
2
import cats.effect.std.Random
3

4
def drunkFlip: IO[String] =
5
  for {
6
    random <- Random.scalaUtilRandom[IO]
7
    caught <- random.nextBoolean
8
    heads  <- if (caught) random.nextBoolean
9
              else IO.raiseError(new Exception("We dropped the coin!"))
10
  } yield if (heads) "Heads" else "Tails"

1
object Main extends IOApp.Simple {
2
  def run: IO[Unit] = drunkFlip.flatMap(result => IO.println(result))
3
}
4

5
// Or
6

7
val result: String = drunkFlip.unsafeRunSync()
8
val resultF: Future[String] = drunkFlip.unsafeToFuture()

1
def drunkFlipF[F[_]: Monad](using R: Raise[F, String], A: Random[F]): F[String] =
2
  for {
3
    caught <- A.nextBoolean
4
    heads <-
5
      if (caught) A.nextBoolean
6
      else R.raise("We dropped the coin")
7
  } yield if (heads) "Heads" else "Tails"

1
import zio._
2

3
def drunkFlip: ZIO[Random, String, String] =
4
  for {
5
    caught <- Random.nextBoolean
6
    heads <-
7
      if (caught) Random.nextBoolean
8
      else ZIO.fail("We dropped the coin")
9
  } yield if (heads) "Heads" else "Tails"

1
val drunkFlipOnConsole: ZIO[Random & Console, String, Unit] =
2
  drunkFlip.flatMap { result =>
3
    Console.printLine(result)
4
  }

1
object Main extends ZIOAppDefault {
2
  override def run =
3
    drunkFlipOnConsole.provide(Random.live, Console.live)
4
}

1
import kyo.*
2

3
def drunkFlip: String < (Abort[String] & Sync) =
4
  for {
5
    caught <- Random.nextBoolean
6
    heads  <- if (caught) Random.nextBoolean else Abort.fail("We dropped the coin")
7
  } yield if (heads) "Heads" else "Tails"

1
val partialResult: Result[String, String] < Sync = Abort.run { drunkFlip }

1
opaque type <[+A, -S]

Direct-Style Effect Handlers

What if we could write code that looks like this:

1
def drunkFlip(using Random, Raise[String]): String = {
2
    val caught = Random.nextBoolean
3
    val heads  =
4
      if (caught) Random.nextBoolean
5
      else Raise.raise("We dropped the coin")
6
    if (heads) "Heads" else "Tails"
7
  }

We abandoned for a moment the functional programming world. No IO. No for-comprehension. No flatMap. Just normal-looking sequential code that reads like the imperative version we started with. This is what direct-style effect handlers enable using Scala 3’s context functions. So, without further ado, let’s see how this works.

1
def greet(name: String)(using logger: Logger): Unit =
2
  logger.log(s"Hello, $name")

1
def greet(name: String): (logger: Logger) ?=> Unit =
2
  logger.log(s"Hello, $name")

1
def greet(name: String): Logger ?=> Unit =
2
  summon[Logger].log(s"Hello, $name")

1
given Logger = new Logger
2
greet("Alice")  // Now prints "Hello, Alice"
3

4
// Or explicitly:
5
greet("Bob")(using new Logger)  // Prints "Hello, Bob"

1
def innerOperation(using logger: Logger): Unit =
2
  logger.log("Inner operation")
3

4
def outerOperation(using logger: Logger): Unit = {
5
  logger.log("Starting outer")
6
  innerOperation  // Compiler passes Logger automatically!
7
  logger.log("Ending outer")
8
}

1
trait Random {
2
  def nextBoolean: Boolean
3
}
4

5
trait Raise[E] {
6
  def raise(error: E): Nothing
7
}

1
def drunkFlip(using Random, Raise[String]): String = {
2
    val caught = Random.nextBoolean
3
    val heads  =
4
      if (caught) Random.nextBoolean
5
      else Raise.raise("We dropped the coin")
6
    if (heads) "Heads" else "Tails"
7
  }

1
object Random {
2
  def run[A](program: Random ?=> A): A = {
3
    val randomCapability = new Random {
4
      def nextBoolean: Boolean = scala.util.Random.nextBoolean()
5
    }
6
    program(using randomCapability)
7
  }
8
}
9

10
object Raise {
11
  def either[E, A](program: Raise[E] ?=> A): Either[E, A] = {
12
    import scala.util.boundary, boundary.break
13
    boundary {
14
      val raiseEffect = new Raise[E] {
15
        def raise(error: E): Nothing = break(Left(error))
16
      }
17
      Right(program(using raiseEffect))
18
    }
19
  }
20
}

1
val result: Either[String, String] = Random.run {
2
  Raise.either {
3
    drunkFlip
4
  }
5
}

1
val partialResult: Raise[String] ?=> String = Random.run {
2
  drunkFlip
3
}

1
def test(fixed: Boolean)(program: Random ?=> Boolean) = {
2
  program(using new Random() {
3
    override def nextBoolean: Boolean = fixed
4
  })
5
}

1
def flipTwice(using Random): Int = {
2
  val first = Random.nextBoolean
3
  val second = Random.nextBoolean
4
  (if (first) 1 else 0) + (if (second) 1 else 0)
5
}
6

7
def flipMany(using Random): Seq[Boolean] = {
8
  Seq.fill(10)(Random.nextBoolean)
9
}
10

11
def analyze(using Random): String = {
12
  val count = flipTwice
13
  val flips = flipMany
14
  s"Got $count true values in first two flips, and ${flips.count(identity)} in next ten"
15
}

1
// Monadic: program is a reusable data structure (pseudocode)
2
val program: IO[Int] = IO(Random.nextInt())
3
program.retryN(3)              // re-invokes the computation three times
4
program.race(program)          // two independent executions
5

6
// Direct-style: program is a function, not data
7
val program: Random ?=> Int = Random.nextInt
8
// Can't retry or race without wrapping in explicit lambdas

1
// Direct-style: retries require an explicit loop
2
def retry[A](n: Int)(program: => A): A =
3
  try program
4
  catch
5
    case e: Exception if n > 0 => retry(n - 1)(program)
6

7
// Usage
8
retry(3) {
9
  Random.run { Random.nextInt }
10
}

1
import scala.concurrent.Future
2
import scala.concurrent.ExecutionContext.Implicits.global
3

4
val f: Future[Int] = Future(expensiveComputation()) // Already running!
5
val g: Future[Int] = f // g shares f's result, not the computation

1
val program: Random ?=> Int = Random.nextInt // Nothing happens yet
2
// Only executes when we provide the handler:
3
Random.run { program } // Now it runs

Direct-Style versus Monadic: Trade-offs and Honest Assessment

Let’s compare approaches directly. Here’s the Cats Effect version:

1
def drunkFlip: IO[String] =
2
  for {
3
    random <- Random.scalaUtilRandom[IO]
4
    caught <- random.nextBoolean
5
    heads  <- if (caught) random.nextBoolean
6
              else IO.raiseError(new Exception("We dropped the coin!"))
7
  } yield if (heads) "Heads" else "Tails"

And here’s the direct-style version:

1
def drunkFlip(using Random, Raise[String]): String = {
2
    val caught = Random.nextBoolean
3
    val heads  =
4
      if (caught) Random.nextBoolean
5
      else Raise.raise("We dropped the coin")
6
    if (heads) "Heads" else "Tails"
7
  }

The difference is evident. The direct-style version looks almost identical to the original imperative code we started with. No flatMap or for-comprehension wrapping our logic. No explicit IO wrapping every operation. Yet it’s still type-safe because the effects are in the signature. And it’s still deferred because of the machinery we built. We’ve achieved the Effect Pattern off the functional programming world.

As we said earlier, the goal was to make effects look like imperative code. But let’s be honest. Direct style isn’t perfect. It makes trade-offs that might matter for some applications. The question is whether those trade-offs are worth it for your specific situation.

Let’s examine what we lose and what we gain.

What we lose is strict referential transparency at the value level. Consider this code:

1
def drunkFlip(using Random, Raise[String]): String = {
2
  val genRand = Random.nextBoolean
3
  val caught = genRand
4
  val heads  =
5
      if (caught) genRand
6
      else Raise.raise("We dropped the coin")
7
  if (heads) "Heads" else "Tails"
8
}

When the handler for Random is provided, the genRand value is evaluated eagerly. Each time we reference genRand, we get the same value. This breaks referential transparency at the value level. We can’t replace genRand with Random.nextBoolean without changing behavior, because genRand is only evaluated once.

To regain something similar to referential transparency, we need to define genRand as a function:

1
def drunkFlip(using Random, Raise[String]): String = {
2
  def genRand = Random.nextBoolean
3
  val caught = genRand
4
  val heads  =
5
    if (caught) genRand
6
    else Raise.raise("We dropped the coin")
7
  if (heads) "Heads" else "Tails"
8
}

As you can see, genRand is now a def rather than a val, so each time we call it, we get a fresh random value. It’s not referential transparency, since we’re not representing programs through values, but we’re just calling functions. Moreover, we have to remember to use def for computations that need re-evaluation.

This has consequences beyond a single function. As we discussed earlier, referential transparency enables equational reasoning and local reasoning. Without it, we lose the ability to refactor code by freely substituting equals for equals. In monadic style, if we see val x = someEffect and val y = someEffect, we know they are two independent descriptions of the same computation, and we can extract them into a shared value without changing behavior. In direct style, the same refactoring might silently change our program because val binds eagerly. Extracting a common subexpression into a val or inlining a val back into its usage sites are everyday refactoring moves, and in direct style they can introduce subtle bugs. We need to be careful about whether we’re dealing with a value or a computation, and that distinction is no longer visible in the code itself.

This connects to a deeper point. As we saw in the “What Kind of Deferral Is This?” section, monadic IO gives us programs as data: we can retry, race, and compose program descriptions freely. In direct-style, the computation is a function, not data. In terms of continuations, monadic IO gives us multi-shot continuations (the runtime can run the same program description multiple times), while direct-style gives us one-shot continuations (each computation runs exactly once). As we’ll see shortly, this tells us which effects we can express and which we can’t.

However, do we really need referential transparency? Referential transparency is a core concept in programming models where we can express programs as values. In this setup, we need to be able to substitute values freely without changing program behavior. This is crucial for reasoning about code, composing it, and ensuring correctness. In particular, it’s crucial for managing the effects’ control flow safely.

If we return to a setup where we use an imperative style, we lose the ability to treat our programs as values and thus the full power to manipulate their control flow. In fact, the direct-style approach we presented can’t express all possible effects (e.g., the Choice effect). This is because we can’t express all types of continuations with context functions. (Yes, it’s all a matter of continuations). Context functions can express a subset of continuations called one-shot continuations.

To make this concrete, consider a realistic example: we’re building a configuration validator that needs to find all valid combinations of database and cache settings. With Kyo’s Choice effect, we can express this naturally in monadic style:

1
def compatible(db: String, cache: String): Boolean =
2
  !(db == "sqlite" && cache == "redis-cluster")
3

4
def validConfigs: (String, String) < Choice =
5
  for {
6
    db    <- Choice.eval("postgres", "mysql", "sqlite")
7
    cache <- Choice.eval("redis", "redis-cluster", "memcached")
8
    _     <- Choice.dropIf(!compatible(db, cache))
9
  } yield (db, cache)
10
// Produces ALL valid combinations at once
11
val allValid: Seq[(String, String)] < Any = Choice.run(validConfigs)

Each call to Choice.eval invokes the continuation, everything after it in the for-comprehension, once per element in the sequence. For three databases and three caches, the continuation after the first Choice.eval runs three times, each spawning three more runs. The filter then prunes incompatible pairs. This requires multi-shot continuations: the ability to run the same continuation multiple times with different inputs. In monadic style, flatMap receives the continuation as a function A => F[B], and the Choice monad can call that function as many times as it needs.

Direct-style context functions can’t express this. When a handler provides an effect instance and calls program(using effectInstance), the program body runs exactly once. There’s no mechanism for the handler to “replay” or “restart” the computation from a certain point with a different value. This is a fundamental limitation of one-shot continuations: each continuation is invoked at most once. So while we could write a Choice trait with a eval method, there’s no way to implement a handler that explores all branches.

In many practical applications, the effects that we can express with direct-style handlers are sufficient. Most applications don’t need the full power of continuations or advanced control flow. They just need to manage side effects such as randomness, I/O, and error handling in a safe, composable way. In these common cases, direct-style handlers provide a simpler, more approachable way to implement the Effect Pattern.

The other limitation is that context functions are Scala 3 only. Teams still on Scala 2 can’t use this approach. There’s no way to backport context functions to Scala 2. Also, understanding using parameters and how context functions work requires learning Scala 3 features. That said, understanding context functions is arguably easier than understanding monads and flatMap. The barrier to entry is lower, but it’s still a barrier for teams without Scala 3 experience.

What we gain is readability. Code looks imperative and familiar. A developer who has never seen functional programming can read direct-style effects and understand what’s happening. They might not immediately understand how using parameters work, but the sequential flow is obvious. There’s no mental overhead of tracking monadic contexts or figuring out what a for-comprehension desugars to.

We also gain freedom from accidental concurrency. This was probably the biggest practical problem with Future-based code: two Future values created independently could run in parallel without us intending it. In direct-style, sequential code runs sequentially. If we want parallelism, we have to use constructs like structured concurrency. What we write is what we get, no hidden concurrency surprises.

We also gain approachability. The cognitive load is dramatically lower compared to monadic approaches. No need to understand flatMap chains. No confusion about when to use map versus flatMap. No nested for-comprehensions to mentally parse. Just write sequential code that looks like what you mean, and let the compiler thread capabilities for you. This significantly lowers the barrier to teams adopting functional effect management.

To summarize the trade-offs across all three approaches, here’s a quick comparison:

As we can see, direct-style sits in the middle. It avoids Future’s pitfalls while being more approachable than full monadic IO, at the cost of some guarantees that monadic systems give us.

Direct style isn’t the best for every problem. Some use cases benefit from explicit monadic structure, especially when you need the additional operations that monads provide or when you’re already deeply invested in a monadic ecosystem. But it’s a valid alternative in many cases. The choice depends on team experience, project constraints, and personal preference. Both approaches implement the Effect Pattern correctly.

Conclusion

We’ve explored two approaches to the Effect Pattern in Scala. Functional programming monadic systems like Cats Effect, ZIO, and Kyo use explicit effect types and flatMap composition. They’re powerful, battle-tested, and widely used in production systems worldwide. The learning curve is steep, but the payoff is robust effect management with rich combinators for composition, error handling, and concurrency. Direct-style systems use context parameters and functions to achieve the same goals with different syntax. They’re more readable and approachable, but they trade some referential transparency at the value level, and the set of effects they can express is limited. Both can implement type tracking and deferred execution. The difference is in syntax and cognitive load. In other words, both let us manage side effects safely. They just express that safety differently.

When should we use monadic approaches? When we’re working on Scala 2 projects that can’t upgrade to Scala 3. When our team is already comfortable with functional patterns and monadic reasoning. When we need maximum type safety and are willing to pay the learning curve cost. When we need the rich combinators and ecosystem that mature effect libraries provide. These are all good reasons to choose the monadic path.

When should we use direct style? On Scala 3 projects where context functions are available. When our team is new to functional programming, the monadic learning curve would slow down development. When we prioritize code readability and want effects to look like imperative code. When we’re building a new system, we can choose our abstractions freely. When we believe that 80% of our use cases don’t need the full power of monadic effects.

The Effect Pattern is the real insight here. Whether we express it with monads or context parameters and functions, we’re separating description from execution. That separation is what makes side effects manageable, testable, and composable. The implementation details are secondary to understanding that core principle. Once we understand the Effect Pattern, we can choose the implementation that fits our context. Both paths lead to safer, more maintainable code. The journey matters less than the destination.

We’ve built our own minimal effect system to understand the principles. But we don’t have to roll our own in production. The Scala ecosystem has direct-style libraries that provide implementations. Two notable ones are: Ox and YAES.