Sharing benchmark observations on decimal floating-point compression: comparing integer mapping with Gorilla/Chimp on time-series datasets #117970

Hello ClickHouse community,

ClickHouse has ongoing work around floating-point codecs (including Gorilla, DoubleDelta, T64, and ALP in Parquet/MergeTree). I wanted to share some recent benchmark observations and implementation findings on lossless floating-point compression from fastalp (also on crates.io).

Benchmark results on real-world time-series data

I evaluated fastalp against classic floating-point compression algorithms (Gorilla, Chimp, Chimp128, Patas) on public time-series datasets including city_temperature.csv, Stocks-Germany-sample.txt, and SSD_HDD_benchmarks.csv. Across 25 iterations per dataset on Apple Silicon (M2 Max), the unified latency and ratio results were:

Overall throughput across standard IEEE 754 float benchmarks:

Implementation observations

1. Decimal integer mapping: In real-world telemetry, sensors, and financial quotes, floating-point numbers often represent physical measurements with decimal precision (e.g. 21.45, 0.003). Mapping values to integers via adaptive decimal scaling (factor 10^e) shifts the compression domain from IEEE 754 mantissa XORing to integer Frame-of-Reference (FOR) bitpacking.

2. Exact decimal division: Precomputed floating-point multiplication (value * 10^e) occasionally introduces 1-ULP precision noise due to IEEE 754 rounding (for example, 0.35 * 100.0 evaluates to 34.99999999999999, casting to 34 instead of 35). Adding an exact division path during sampling and reconstruction eliminates these pseudo-exceptions and preserves bit-packing efficiency.

3. First-order difference (Delta-ALP): For monotonic timestamps, cumulative metrics, and smooth waveforms, cascading first-order delta encoding onto the scaled integers significantly narrows the residual range, reducing bitwidth without altering the random access properties of columnar blocks.

4. SIMD vectorized decode: Pure register SIMD unpacking enables decoding latencies of 0.423 µs per 1000 values (~2.36 billion values/sec), achieving 55 to 77 GB/s linear memory bandwidth.

I hope these observations and benchmark metrics provide useful data points for the ClickHouse community as columnar float codecs continue to evolve.