Generalize Combinators for Effectually Transforming Collections #4037
Conversation
| /** | ||
| * Applies the functionw `f` to each element of the `Iterable[A]` in parallel, | ||
| * and returns the results in a new `List[B]`. | ||
| * Applies the functionw `f` to each element of the `Collection[A]` in parallel, |
There was a problem hiding this comment.
Not from your PR but I just noticed a small typo here.
There was a problem hiding this comment.
Thanks! Fixed now!
| }, | ||
| testM("empty input") { | ||
| assertM(ZIO.forkAll(List.empty).flatMap(_.join))(equalTo(List.empty)) | ||
| assertM(ZIO.forkAll(List.empty[ZIO[Any, Nothing, Unit]]).flatMap(_.join))(equalTo(List.empty)) |
There was a problem hiding this comment.
It works fine on Scala 2.12. We don't have to pay it if scala/bug#12104 is resolved.
| tArray <- makeStair(N).commit | ||
| findFiber <- tArray.findLast(_ % largePrime == 0).commit.fork | ||
| _ <- STM.foreach(0 until N)(i => tArray.update(i, _ => 1)).commit | ||
| _ <- STM.foreach(List.range(0, N))(i => tArray.update(i, _ => 1)).commit |
There was a problem hiding this comment.
The old 0 until N no longer works?
There was a problem hiding this comment.
Ranges work if it is an inclusive range but not an exclusive range. The same issue applies to Future.traverse. This seems like a bug. Opened an issue for it at scala/bug#12105.
| } | ||
| dest.map(_.result()) | ||
| } | ||
| ZIO.foreach(self)(f) |
| def poll: UIO[Option[Exit[E, Collection[A]]]] = | ||
| UIO | ||
| .foreach(fibers)(_.poll) | ||
| .foreach[Fiber[E, A], Option[Exit[E, A]], Iterable](fibers)(_.poll) |
There was a problem hiding this comment.
Yikes! These are some pretty monstrous type annotations. Could partial application trick help here?
There was a problem hiding this comment.
This can be rewritten as UIO.foreach(fibers: Iterable[Fiber[E, A]])(_.poll).
| ZManaged(ZIO.foreach(as)(f(_).zio).map { result => | ||
| val (fins, _) = result.unzip | ||
| (e => ZIO.foreach(fins.reverse)(_.apply(e)), ()) | ||
| (e => ZIO.foreach(fins.toList.reverse)(_.apply(e)), ()) |
There was a problem hiding this comment.
Seems like we can no longer work with Iterable. Could a low priority implicit build from compat work to restore that?
There was a problem hiding this comment.
I think this actually isn't an issue with our implementation at all. reverse isn't defined on Iterable so since we no longer automatically return a List we don't have a Seq that can be reverse.
| supervisedFibers.flatMap { fibers => | ||
| ZIO | ||
| .foreach(fibers)(_.status.filterOrFail { | ||
| .foreach[Any, Unit, Fiber.Runtime[Any, Any], Fiber.Status, Iterable](fibers)(_.status.filterOrFail { |
There was a problem hiding this comment.
😱 If there's a way to fix this, we really should explore that.
There was a problem hiding this comment.
This one is particular bad because Set is invariant. All we really need here is a way to widen Set to Iterable, which unfortunately the compiler won't do for us automatically here. On Scala 2.12 we can just do foreach(fibers.toIterable)(???) but that doesn't work on Scala 2.13 because now toIterable returns this.type so it doesn't actually widen. Let me see if we can use a type ascription on fibers.
There was a problem hiding this comment.
This one is fixed now with the new foreach variant for Set.
Currently combinators for effectually transforming collections generally take an
Iterable[A]and return aList[B], resulting in loss of information and requiring separate versions of combinators to preserve the specific collection type. This results in code duplication (e.g.Chunk#mapMversusZIO.foreach) and lack of support for some collection types (e.g. there is not currently a combinator to effectually transform aVectorand preserve its specific collection type).This PR uses the
BuildFrommachinery in the Scala standard library to support transformation methods that return the same collection typeZIO,ZManaged, andZSTM. Variants offoreachthat are specialized toChunkare removed sinceChunkis a subtype ofIterableandChunk#mapMand variants are reimplemented in terms ofZIO.foreach.Resolves #2781. Resolves #3369. Resolves #3942. Resolves #4032.