Implement Balanced Concatenation for Chunk #3754
Conversation
|
@adamgfraser Maybe I am completely wrong, but you may not need depth to do that, the size difference between left and right is enough to choose where to land the new piece in the tree. For reference: https://github.com/zio/zio/pull/3358/files#diff-290e96e6c64027a32c36d751a35d65deR671-R735 |
|
I went back to the code I wrote at the time, with the perf test, this was doing a decent job and perfect balancing on the repeated concatenation. final def concat[A](l: NonEmpty[A], r: NonEmpty[A]): NonEmptyChunk[A] =
l match {
case Concat(ll, lr) if ll.size >= lr.size + r.size => Concat(ll, concat(lr, r))
case _ => Concat(l, r)
}This one is the same version without the recursion, and manage the case were the RHS is larger (code duplication). def concat[A](l: NonEmpty[A], r: NonEmpty[A]): NonEmptyChunk[A] =
if (l.length == r.length) Concat(l, r)
else if (l.length > r.length) {
val lefts: Array[NonEmptyChunk[A]] = Array.ofDim[NonEmptyChunk[A]](16)
var n: Int = 0
var point: NonEmptyChunk[A] = l
var continue = true
while (continue && n < 16) {
point match {
case _ if point.length <= r.length =>
continue = false
case Concat(ll, lr) if ll.length >= lr.length + r.length =>
lefts(n) = ll
n += 1
point = lr
case _ =>
continue = false
}
}
var res = Concat(point, r)
var i = n - 1
while (i >= 0) {
res = Chunk.Concat(lefts(i), res)
i -= 1
}
res
} else {
val rights = Array.ofDim[NonEmptyChunk[A]](16)
var n: Int = 0
var point: NonEmptyChunk[A] = r
var continue = true
while (continue && n < 16) {
point match {
case _ if point.length <= l.length =>
continue = false
case Concat(rl, rr) if rr.length >= rl.length + l.length =>
rights(n) = rr
n += 1
point = rl
case _ =>
continue = false
}
}
var res = Concat(l, point)
var i = n - 1
while (i >= 0) {
res = Chunk.Concat(res, rights(i))
i -= 1
}
res
}The issue was, due to the limited structure of the With your work on repeated appends and pretends (thanks 🙏 ), the performance differences may not be visible on regular use-cases. |
|
Looks great @adamgfraser. Can you add a repeated concatenation benchmark to see the perf. hit? |
|
Yes. It depends a lot on usage pattern. For balanced concatenation it doesn't make much of a difference. For degenerate cases where you repeatedly concatenate a single element it is quite a hit. Maybe when we concatenate we could check the length of the chunks and treat it as an append or prepend if one of them has a length of one to avoid the very worst case? |
Resolves #3238. Resolves #3753.
Implements balanced concatenation for Chunk based on the algorithm used in Conc-Trees. This also addresses the stack overflow issue.