Support leftovers in Sinks #3722
Conversation
|
This is coming along really nicely. Thank you for picking this up @simpadjo. With your help we will restore happiness to ZSink land ;-) |
|
Deleted
|
|
TODO:
|
| { | ||
| case Left(e) => ZIO.fail(e) | ||
| case Right(z) => restart.as(Chunk.single(z)) | ||
| case (Left(e), _) => ZIO.fail(e) |
There was a problem hiding this comment.
This is a weak spot. We have no contract that says that a transducer cannot be called after failing (and we shouldn't have such a contract). That means we're dropping leftovers inputs after failing.
How about placing these leftovers in a Ref that will be read on every push?
There was a problem hiding this comment.
I don't disagree with you, although I can't come up with a testcase so far.
Seems that once you called stream.transduce(t) you can't see beyond the first error.
| case (None, Some(z1)) => ZIO.succeedNow(RightDone(z1)) | ||
| case (None, None) => ZIO.succeedNow(BothRunning) | ||
| case (Some((z, l)), Some((z1, l1))) => { | ||
| val minLeftover = if (l.length > l1.length) l1 else l |
There was a problem hiding this comment.
What's your reasoning behind using the minimum? I can see arguments for both choices, just wondering what you're thinking.
There was a problem hiding this comment.
Consider (s1.zipPar(s2)).zip(s3). We have two lineages of consumers: s1.zip(s3) and s2.zip(s3).
Returning the maximum means that one of the lineages consumes some elements twice.
Which is the most surprising behavior IMO.
We have consumers that ignore some of the input but no consumer processes an element twice (w/o explicitly being asked for).
Btw, what are the arguments for the opposite?
|
Thank you @simpadjo for your work on this 🎉 |
#3711
Problems:
Transducer >>> Sink : Sinkcomposition is impossible nowIZSink#contramapgoneAchievements:
ZSink.head.flatmap(_ => Zsink.count)works!