ZStream.mapZIOPar[Unordered] optimization attempt + ZIO.forkIn optimization #9801
Conversation
…t due to env propagation issues
|
@kyri-petrou @ghostdogpr , I'd appreciate your input on this |
|
FYI, I already provided some optimisation for |
…, but it introduces more concurrent pressure on the scope resulting with inconsistent benchmarks results
… atomics, but it introduces more concurrent pressure on the scope resulting with inconsistent benchmarks results" This reverts commit 1f8aa79.
…e one managing the finalizer from within the child fiber. introduce additional benchmarks showing how can the scope's concurrency limits can be bypassed
|
@kyri-petrou , @guizmaii @ghostdogpr , benchmark results ( |
… old impl as a private[zio] (which hopefully pass the mima tests), modify the benchmark names accordingly
|
benchmarks after renaming the |
…ffect that effectively executes the finalizer when explicitly invoked
|
I'm closing this in order to reopen it with a more appropriate branch name and description. please see #9813 |
this toy branch started as an attempt to optimize ZStream.mapZIOPar and ZStream.mapZIOParUnordered (and the counterpart ZPipeline operations) but ended up with an alternative optimized implementation for ZIO.forkIn.
the idea behind the optimization is to avoid the intermediate additional scope object required by the current implementation and leverage some low level atomic ops to track the fiber and finalizer states.
I've currently modified the two stream operators to leverage this implementation and also added two variations to the ForkAllBenchmark, please see the relevant benchmarks results below:
ZStream relevant benchmarks, baseline:
ZStream relevan bencharks in this branch:
introducing this work as a draft PR in order to discuss the optimization approach.