Add alternative combinator to ZSTM #3414

mijicd · 2020-04-19T22:35:27Z

Problem

The current orElse implementation does not retry left once provided alternative retries but stays "stuck" retrying only the alternative.

Since our STM implementation is inspired by the one in Haskell, I made a small program to double-check the expected behavior.

The example is taken from a fantastic STM workshop by Michael Snoyman, which you can check out here.

#!/usr/bin/env stack
-- stack --resolver lts-13.21 script
import Control.Applicative (Alternative (..))
import Control.Concurrent (threadDelay)
import Control.Concurrent.Async
import Control.Concurrent.STM
import Control.Exception
import Control.Monad (forever)

data Boom = Boom deriving Show

instance Exception Boom

-- | Add 5 to the variable every 100ms
payAlice :: TVar Int -> IO ()
payAlice aliceVar = forever $ do
  atomically $ do
    alice <- readTVar aliceVar
    writeTVar aliceVar $! alice + 5
  threadDelay 100000 -- 100ms, threadDelay takes nanoseconds

-- | Transfer 40 from Alice to Bob.
transfer
  :: TVar Int -- ^ Alice
  -> TVar Int -- ^ Bob
  -> IO ()
transfer aliceVar bobVar = atomically $ do
  let amount = 40
  alice <- readTVar aliceVar

  -- each of the following variants fail the entire transaction
  -- throwSTM Boom <|> check (alice >= amount)
  -- check (alice >= amount) <|> throwSTM Boom

  -- retries both (2nd retry is obviously redundant but it was failing test for ZSTM)
  check (alice >= amount) <|> retry

  writeTVar aliceVar $ alice - amount
  bob <- readTVar bobVar
  writeTVar bobVar $ bob + amount

main :: IO ()
main = do
  aliceVar <- newTVarIO 0
  bobVar <- newTVarIO 0
  race_ (payAlice aliceVar) (transfer aliceVar bobVar)
  alice <- atomically $ readTVar aliceVar
  bob <- atomically $ readTVar bobVar
  putStrLn $ "Alice has: " ++ show alice
  putStrLn $ "Bob has: " ++ show bob

Experimenting with alternative combinations, I've noticed that our implementation "swallows" failures.

jdegoes · 2020-04-19T22:39:23Z

core/shared/src/main/scala/zio/stm/ZSTM.scala

   * Named alias for `<>`.
   */
-  def orElse[R1 <: R, E1, A1 >: A](that: => ZSTM[R1, E1, A1]): ZSTM[R1, E1, A1] =
+  def orElse[R1 <: R, E1 >: E, A1 >: A](that: => ZSTM[R1, E1, A1]): ZSTM[R1, E1, A1] =


We have a problem with this: ZIO#orElse will try the right if the left fails. That's the "error swallowing" behavior you mention.

Maybe what this is saying is we need two combinators: one for doing "retry" fallback, and one for doing "error" fallback. What we used to have was one combinator for doing both. Or maybe doing two things at once is sufficiently common to warrant conflating them?

This is something I briefly discussed with Adam. The current behavior of orElse (alternative) is wrong according to the cited paper and Haskell's implementation. The only thing I wouldn't do is leave it incorrect. Other than that, we are going to be asymmetric with something.

mijicd · 2020-04-20T06:40:00Z

@jdegoes @adamgfraser As discussed over discord, I've reverted the old orElse behavior, and introduced another combinator called alternative (with symbolic alias <|>) that will have Haskell's semantic.

core/shared/src/main/scala/zio/stm/ZSTM.scala

Co-Authored-By: John A. De Goes <[email protected]>

This reverts commit 19f1162.

core/shared/src/main/scala/zio/stm/ZSTM.scala

jdegoes · 2020-04-20T16:39:01Z

docs/datatypes/stm.md

 ```

-This will cause the transfer to fail immediately if the sender does not have money because of the semantics of `orElse`.
+This will cause the transfer to fail immediately if the sender does not have money because of the semantics of `orElse`.


This one still mentions orElse instead of orRetry

Co-Authored-By: John A. De Goes <[email protected]>

Add test for Schedule#onDecision and documentation for scheduling (zio#3346) * Add test for Schedule#onDecision and documentation for scheduling * Add documentation to datatypes, remove console functions from ScheduleSpec * fix mdoc compile issue specialize ChunkBuilder (zio#3400) Implement ZRefM (zio#3391) * implement ZRefM * fix Scala 2.13 compatibility issue * address review comments * implement SubscriptionRef * make constructor private * remove managed variant * remove unused import Implement ZIO#mapEffect (zio#3404) * implement mapEffect * implement mapPartial Improve TArray's performance (zio#3407) * Benchmark TArray * Implement fold and transform via unsafe * Improve readability * Implement find via unsafe * Implement findLast via unsafe * Implement foldM via unsafe * Benchmark indexWhere * Implement transformM via unsafe * Implement indexWhere via unsafe * Implement lastIndexOf via unsafe * Remove duplicated code * Implement foldM via ZIO.foldLeft * Remove 0-sized array from benchmark * Implement reduceOption via unsafe * Implement findM via foldM * Help inference via Option.empty * Implement findM via iterate * Implement findLastM via iterate * Remove useless val * Format the sources Update sbt-header to 5.6.0 (zio#3411) Add AssertionM for effectfull assertions (zio#3403) Add result field to AssertionValue to avoid test reevaluation. Protect AssertionValue.assertion to avoid test reevaluation. Use lazy val tryValue = Try(value) to avoid exception reevaluation: lazy val stores only successful case. implement launch (zio#3415) Add alternative combinator to ZSTM (zio#3414) * Drop duplicated test * Remove unnecessary .unit calls * Align the naming * Write failing tests * Keep semantically correct orElse tests * Fix test expectations * Fix orElse behavior on failures * Fix orElse behavior on retries * Reduce flakiness * Revert orElse changes * Introduce alternative combinator * Update docs * Reduce flakiness * Make right retry explicit * Race updater and transaction * Move left-right recovery test to JVM tests * Drop E from orElseSucceed Co-Authored-By: John A. De Goes <[email protected]> * Rename alternative to orTry Co-Authored-By: John A. De Goes <[email protected]> * Revert "Move left-right recovery test to JVM tests" This reverts commit 19f1162. * Tag retry test as JVM-only * Fix compilation issues * Fix docs * Improve scaladoc Co-Authored-By: John A. De Goes <[email protected]> * Fix doc mistake Co-authored-by: John A. De Goes <[email protected]> Update sbt-scalafix to 0.9.15 (zio#3423) Optimize TQueue (zio#3421) Update sbt-ci-release to 1.5.3 (zio#3425) ZIO Test: Support Laws for Capabilities Parameterized on Two Types (zio#3420) * implement ZLaws2 * use AnyF Back up Chunk[Boolean] by Chunk.BitChunks (zio#3419) fix publishing (zio#3433) WIP: Scala.js upgrade to v1.0.0 (zio#2392) (zio#2479) * Update to Scala.js 1.0.0-RC2 * fix for passing testJS * formatting fix * Merge remote-tracking branch 'upstream/master' into feature/scalajs-1.0.0-RC2 * Merge branch 'master' of https://github.com/zio/zio into feature/scalajs-1.0.0-RC2 * sonatype resolver added for izumi-reflect; magnolia updated to v0.12.8 Fix Scala.js build (post zio#2479) (zio#3436) * Fix Scala.js build (post zio#2479) * Move back to shared/Executor source and add jvm/js/native ExecutorPlatformSpecific trait * Fix imports causing build failure on 2.13/dotty Update Magnolia (zio#3432) * Update Magnolia There is a bug in Magnolia versions < 0.15 which causes derivation for case classes with default values to fail. The update should fix it. * Add scala-reflect % Provided dependency, now required for magnolia & exclude scala-compiler Co-authored-by: Kai <[email protected]> Co-authored-by: Kai <[email protected]> Update nyaya-gen to 0.9.1 (zio#3438) Remove sonatype public resolver (zio#3434) Add specialized accessor operations to Chunk (zio#3431) * Add specialized accessor operations to Chunk * Generate only non-empty chunks using chunkWithIndex generator Add ZIO STM blogpost (zio#3443) generalize (zio#3439) Update scala-java-time to 2.0.0 (zio#3446) suspend effects (zio#3444) Fix Publishing on Master (zio#3437) * fix publishing * fix bug add Scalac logo to the Sponsors section in Readme (zio#3379) Implement ZIO#service (zio#3422) * implement getService * add variants * rename Add Diagnostic Code to Has (zio#3377) * add diagnostic code * Fix environment composition * Fix 2.11/dotty Co-authored-by: ioleo <[email protected]> Use short forms where it makes sense (zio#3452) * Use short forms where it makes sense * One more `none` usage. Update to izumi-reflect 1.0.0-M1, use izumi-reflect on dotty (zio#3445) * Update to izumi-reflect-1.0.0-M1, depend on izumi-reflect in dotty * Use one VersionSpecific file for dotty & scala-2 * Adding type signatures to Mock modules to workaround scala/scala3#8764 error: ``` [error] -- Error: PureModule.scala:77:86 [error] 77 | val static = ZIO.accessM[PureModule](_.get.static) [error] | ^ [error] |could not find implicit value for izumi.reflect.Tag[zio.test.mock.module.PureModule.Service]. Did you forget to put on a Tag, TagK or TagKK context bound on one of the parameters in zio.test.mock.module.PureModule.Service? e.g. def x[T: Tag, F[_]: TagK] = .... [error] |I found: [error] | [error] | { [error] | izumi.reflect.Tag.apply[zio.test.mock.module.PureModule.Service](classOf[Any] [error] | , [error] | { [error] | <empty> :izumi.reflect.macrortti.LightTypeTag [error] | } [error] | ):izumi.reflect.Tag[zio.test.mock.module.PureModule.Service] [error] | } [error] | [error] |But method tagFromTagMacro in object Tag does not match type izumi.reflect.Tag[zio.test.mock.module.PureModule.Service]. [error] | [error] |The following import might make progress towards fixing the problem: [error] | [error] | import izumi.reflect.Tag.tagFromTagMacro ``` * add type signature to SpecSpec to workaround error: ``` [error] -- Error: SpecSpec.scala:17:51 [error] 17 | val layer = ZLayer.succeed(new Module.Service {}) [error] | ^ [error] |could not find implicit value for izumi.reflect.Tag[zio.test.SpecSpec.Module.Service]. Did you forget to put on a Tag, TagK or TagKK context bound on one of the parameters in zio.test.SpecSpec.Module.Service? e.g. def x[T: Tag, F[_]: TagK] = .... [error] |I found: [error] | [error] | { [error] | izumi.reflect.Tag.apply[zio.test.SpecSpec.Module.Service](classOf[Any], [error] | { [error] | <empty> :izumi.reflect.macrortti.LightTypeTag [error] | } [error] | ):izumi.reflect.Tag[zio.test.SpecSpec.Module.Service] [error] | } [error] | [error] |But method tagFromTagMacro in object Tag does not match type izumi.reflect.Tag[zio.test.SpecSpec.Module.Service]. [error] | [error] |The following import might make progress towards fixing the problem: [error] | [error] | import izumi.reflect.Tag.tagFromTagMacro ``` * Lift `exceptDotty` from HasSpec & ComposedMockSpec (TaggedSpec still won't pass because deriving is not implemented) * add type signatures to MockSpecUtils, update comments * Workaround type inference-caused test failures in PolyMockSpec - Any was inferred for `Tag` now, probably maybe because Tag parameter is AnyKind, add workaround * Fix `@mockable` macro, restrict valid `Tagged` definitions to just izumi.reflect.Tag * sbt fix fix type signature (zio#3455) Update util-core to 20.4.1 (zio#3457) generalize type signature (zio#3460) doc(test_effects) : add aspect ignore (zio#3462) upgrade monix version (zio#3467) Implement ZManaged#flattenM (zio#3464) * implement flattenM * format * fix Scaladoc reorder type parameters (zio#3463) Implement ZManaged#release (zio#3466) * implement ZManaged#release * rename Use ChunkBuilder more prominently in Chunk (zio#3453) * make Chunk use ChunkBuilder * fix lint failure * consistent parenthesis use * use parens for Chunkbuilder#make * followup * retrigger ci Update reactor-core to 3.3.5.RELEASE (zio#3472) Update magnolia to 0.16.0 (zio#3473) Update nyaya-gen to 0.9.2 (zio#3474) added blog to list (zio#3471) * updated blog for RC18 + http4s .21.x * added new blog to resource page Co-authored-by: Pierre Ricadat <[email protected]> Add zio-easymock to libraries with zio support list (zio#3481) Add ZIO#filterNot and ZSTM#filterNot (zio#3477) * Add ZIO#filterNot and ZSTM#filterNot * Fix typos implement assertElements and assertElements

mijicd added 8 commits April 19, 2020 18:06

Drop duplicated test

e3dbb01

Remove unnecessary .unit calls

cc5e70e

Align the naming

f184f24

Write failing tests

25b68e0

Keep semantically correct orElse tests

6bfd660

Fix test expectations

1ba247d

Fix orElse behavior on failures

77e9ff1

Fix orElse behavior on retries

72de252

mijicd requested review from adamgfraser and jdegoes April 19, 2020 22:35

jdegoes reviewed Apr 19, 2020

View reviewed changes

mijicd added 3 commits April 20, 2020 01:02

Reduce flakiness

3484aa3

Revert orElse changes

3095bd7

Introduce alternative combinator

45e233a

Update docs

5705056

mijicd changed the title ~~Fix STM.orElse behavior~~ Add alternative combinator to ZSTM Apr 20, 2020

mijicd added 4 commits April 20, 2020 09:28

Reduce flakiness

ca26840

Make right retry explicit

39c15d5

Race updater and transaction

4442d27

Move left-right recovery test to JVM tests

19f1162

jdegoes reviewed Apr 20, 2020

View reviewed changes

core/shared/src/main/scala/zio/stm/ZSTM.scala Outdated Show resolved Hide resolved

jdegoes reviewed Apr 20, 2020

View reviewed changes

core/shared/src/main/scala/zio/stm/ZSTM.scala Outdated Show resolved Hide resolved

mijicd and others added 6 commits April 20, 2020 17:20

Drop E from orElseSucceed

5adb4a1

Co-Authored-By: John A. De Goes <[email protected]>

Rename alternative to orTry

f158c85

Co-Authored-By: John A. De Goes <[email protected]>

Revert "Move left-right recovery test to JVM tests"

df3e381

This reverts commit 19f1162.

Tag retry test as JVM-only

a62d4ae

Fix compilation issues

9a54295

Fix docs

a1bd96c

mijicd requested a review from jdegoes April 20, 2020 16:21

jdegoes reviewed Apr 20, 2020

View reviewed changes

core/shared/src/main/scala/zio/stm/ZSTM.scala Outdated Show resolved Hide resolved

jdegoes reviewed Apr 20, 2020

View reviewed changes

mijicd and others added 2 commits April 20, 2020 19:15

Improve scaladoc

b300dd9

Co-Authored-By: John A. De Goes <[email protected]>

Fix doc mistake

a68c7eb

jdegoes approved these changes Apr 20, 2020

View reviewed changes

mijicd merged commit 58b7771 into zio:master Apr 20, 2020

mijicd deleted the fix-or-else branch April 20, 2020 20:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add alternative combinator to ZSTM #3414

Add alternative combinator to ZSTM #3414

Uh oh!

mijicd commented Apr 19, 2020 •

edited

Loading

Uh oh!

jdegoes Apr 19, 2020

Uh oh!

mijicd Apr 19, 2020

Uh oh!

mijicd commented Apr 20, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jdegoes Apr 20, 2020

Uh oh!

mijicd Apr 20, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Add alternative combinator to ZSTM #3414

Add alternative combinator to ZSTM #3414

Uh oh!

Conversation

mijicd commented Apr 19, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Uh oh!

jdegoes Apr 19, 2020

Choose a reason for hiding this comment

Uh oh!

mijicd Apr 19, 2020

Choose a reason for hiding this comment

Uh oh!

mijicd commented Apr 20, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jdegoes Apr 20, 2020

Choose a reason for hiding this comment

Uh oh!

mijicd Apr 20, 2020

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mijicd commented Apr 19, 2020 •

edited

Loading