So, here's the deal: the existing pool implementation is terribly slow. For some merge cases, you can see more than 70% of the runtime dominated by pool allocations!

Why is it so slow? Because it does a lot of very complex calculations & pointer wrangling just for the sake of keeping a "free list" and trying to squeeze some bytes out of each allocation. IMO this basically defeats the point of a memory pool/slab allocator.

I've re-implemented the pool to look more like a slab allocator. There's always an existing slab; when the existing slab has not enough space to fulfill the requested allocation, we allocate a new slab.

This implementation is hundreds of times faster than the old pool for most cases. For instance: a particular merge of the cdnjs repository that I discussed with @ethomson used to take 17s on my machine. 10 of those seconds were spent in the pool (lol wtf?).

With the new implementation, the merge finishes in 5.2s.

Questions

• How does the new aggressive slab algorithm impact memory usage? This is hard to guess. For the specific case I'm testing, it doesn't, because all the allocated strings are of similar size. For some pathological cases with very large strings, I could see this wasting a little bit of memory. Frankly, given the terrible performance of the old pool, I'm confident the tradeoff will always be positive.
• What happens with the APIs that allowed freeing pointers from the pool? They are gone -- it's not possible to free specific pointers from a traditional slab allocator because there's no free-list. Fortunately, only one place of the existing code was calling the pool free API: the iterators code. Removing the call will increase memory usage on the intermediate steps of the iterator, until the pool is fully freed. I think this will be acceptable too.
• What can possibly go wrong? I am not sure. It's a pretty large change in a critical code path of the library, but the resulting code is so significantly simpler that I'm pretty confident I haven't introduced any regressions. I'd appreciate @carlosmn and @ethomson's careful eyes on this.