add ZTransducer#utf16decode #3934
Conversation
| def last[O]: ZTransducer[Any, Nothing, O, Option[O]] = | ||
| foldLeft[O, Option[O]](Option.empty[O])((_, a) => Some(a)) | ||
|
|
||
| def magicSequence[R, E, I, O](n: Int)(f: Chunk[I] => ZTransducer[R, E, I, O]): ZTransducer[R, E, I, O] = |
There was a problem hiding this comment.
I think this can be more general in that we could have a combinator that "decides" on a transducer after the first element has been emitted and is reset on a push(None).
Should we add something like this? Signature would be:
ZTransducer[R, E, I, O] {
def replace[R1 <: R, E1 >: E, I1 <: I, O1](f: O => ZTransducer[R1, E1, I1, O1]): ZTransducer[R1, E1, I1, O1]
}
There was a problem hiding this comment.
This is kind of similar to ZStream#peel but in a transducer form. The infix form would be self >>> ZStream.magicSequence no?
Either way would be great if we can just find a better name before merging. Something around branching, headers?
There was a problem hiding this comment.
True, so we probably don't need an additional combinator.
What about branchAfter for the name? As in "after reading n values, branch"
| ZRef.makeManaged[Option[Byte]](None).map { stateRef => | ||
| { | ||
| case None => | ||
| // should we raise an error here if we have leftovers? |
There was a problem hiding this comment.
What should we do here? If we do not fail the transducer we will lose data as there is no way to decode the orphaned byte
There was a problem hiding this comment.
With the UTF-8 decoding, if the stream ends (which is what None signifies) in the middle of 2/3/4 byte sequence, we just defer to the underlying charset decoder (which if I recall correctly just drops the malformed bytes). I'm fine with adopting the same approach here.
In the future, the None case will really signal the end of stream and we will stop allowing transducers to be restarted during stream processing. This will be done by separating transducers for which restarts do make sense (all the folds basically) to a separate abstraction (ZAggregator or something like that).
There was a problem hiding this comment.
Ah, I see. Thanks for the explanation 👍
|
branchAfter sounds great to me
…On 9 Jul 2020, 14:06 +0300, Maxim Schuwalow ***@***.***>, wrote:
@mschuwalow commented on this pull request.
In streams/shared/src/main/scala/zio/stream/ZTransducer.scala:
> @@ -560,6 +561,52 @@ object ZTransducer {
def last[O]: ZTransducer[Any, Nothing, O, Option[O]] =
foldLeft[O, Option[O]](Option.empty[O])((_, a) => Some(a))
+ def magicSequence[R, E, I, O](n: Int)(f: Chunk[I] => ZTransducer[R, E, I, O]): ZTransducer[R, E, I, O] =
True, so we probably don't need an additional combinator.
What about branchAfter for the name? As in "after reading n values, branch"
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Thank you @mschuwalow! |
part of #3865