iter-merge

A high-performance iterator merging library for Rust that efficiently combines multiple iterators into a single iterator, yielding smallest item first.

Note: only compares the first items across provided iterators. The output would be sorted only if the iterators themselves are sorted.

Features

• Lazy iterator consumption: each .next() advances only one iterator
• Support for custom comparison functions
• Multiple storage backends: Vec and Array (for no-std and no-alloc compatibility)
• Additional Peekable methods

Usage

use iter_merge::merge;

let iter1 = vec![1, 3, 5, 7];
let iter2 = vec![2, 4, 6, 8];

let result = merge([iter1, iter2]).into_vec();

assert_eq!(result, vec![1, 2, 3, 4, 5, 6, 7, 8]);

Custom Comparison

use iter_merge::{VecStorage, comparators::ByOrd};

let mut merged = VecStorage::from_iter([
    vec![9, 6, 3],
    vec![8, 5, 2],
    vec![7, 4, 1],
])
.into_builder()
.max_by(ByOrd) // {min|max}{_by|_by_func|_by_key}
.build();

let result = merged.into_vec();
assert_eq!(result, vec![9, 8, 7, 6, 5, 4, 3, 2, 1]);

Array storage (no-alloc compatible)

use core::pin::pin;

use iter_merge::{ArrayStorage, MergeIter};

// First create a fixed-capacity array storage
let storage = ArrayStorage::from_arr([
    [1, 3, 5],
    [2, 4, 6]
]);

// In order to construct a MergeIter you need to pin that storage.
// You won't be able to modify it once you've pinned it.
let storage = pin!(storage);
let mut merge = storage.build();
assert!(merge.eq([1, 2, 3, 4, 5, 6]));

Performance

It's 1.45-1.65x faster than itertools::kmerge in my benchmarks and scales as O(item_count + log₂(iterator_count)) (just as well as itertools::kmerge)

Features

The crate supports following feature flags:

• alloc: Enables heap-allocated storage with VecStorage and methods like MergeIter::into_vec

Testing

Fuzzing: Use cargo-fuzz

cargo +nightly fuzz run fuzz_correctness
cargo +nightly fuzz run fuzz_usage

Miri:

cargo +nightly miri test

Benchmarks:

RUSTFLAGS='-C target-cpu=native --cfg benchmarking' cargo bench --bench benchmarks

(Benchmarking dependencies are behind benchmarking cfg option)

MSRV

The minimum supported Rust version for this crate is 1.68.0. This may change in a major release, but it is unlikely.

Contributing

Contributions are welcome. For major changes, please open an issue first to discuss what you would like to change.

Related Projects

• itertools - General iterator utilities (includes kmerge)

License