Tags: databendlabs/databend
Tags
fix(cluster): close exchange buffer on send error (#19857) * fix(cluster): close exchange buffer on send error * fix(query): address exchange close clippy issues * fix: special handle local channel --------- Co-authored-by: Winter Zhang <[email protected]>
feat(storage): use from_slice instead of from_reader for NDJSON parsi… …ng (#19843) perf(storage): use from_slice instead of from_reader for NDJSON parsing Replace serde_json::from_reader with serde_json::from_slice in the NDJSON row decoder. from_reader wraps the byte slice in an IoRead adapter that reads byte-by-byte, disabling serde_json's internal SIMD-friendly slice fast path. Benchmark (20k rows, ~890 MiB real OTEL trace NDJSON, avg ~45KB/row): - from_reader: 1.947s (457 MiB/s) - from_slice: 0.768s (1160 MiB/s) -- 2.54x faster No behavior change: the input is already a &[u8] slice.
feat(fuse): support per-column STATS_TRUNCATE_LEN for string statisti… …cs (#19815) * feat(fuse): support per-column STATS_TRUNCATE_LEN for string statistics Add STATS_TRUNCATE_LEN column option to CREATE TABLE DDL, allowing users to specify a custom string truncation length (1-4096) for column-level min/max statistics instead of the default 16 bytes. Changes: - Add STATS_TRUNCATE_LEN keyword to parser and column definition AST - Store field_stats_truncate_len in TableMeta (proto field 39) - Plumb truncate length through WriteSettings, BlockBuilder, and ColumnStatisticsState to gen_columns_statistics - Add trim_string_min/max_with_len helpers for custom-length trimming - Show STATS_TRUNCATE_LEN in SHOW CREATE TABLE output - Add proto-conv v175 test and sqllogictest coverage * z * z * z * z
refactor(spill): extract MemoryPool from SpillsBufferPool and make po… …ol size per-session configurable (#19805) * refactor(spill): extract MemoryPool from SpillsBufferPool and make pool size per-session configurable - Extract MemoryPool struct from SpillsBufferPool with chunk-based allocation and blocking wait - Add session-level setting 'spill_writer_memory_pool_size_mb' (default 20MB) to control per-writer pool size - Remove global 'buffer_pool_memory' from SpillConfig, making memory management more flexible - MemoryPool uses lazy allocation up to max_chunks, then blocks waiting for buffer release * z * z
refactor(spill): extract MemoryPool from SpillsBufferPool and make po… …ol size per-session configurable (#19805) * refactor(spill): extract MemoryPool from SpillsBufferPool and make pool size per-session configurable - Extract MemoryPool struct from SpillsBufferPool with chunk-based allocation and blocking wait - Add session-level setting 'spill_writer_memory_pool_size_mb' (default 20MB) to control per-writer pool size - Remove global 'buffer_pool_memory' from SpillConfig, making memory management more flexible - MemoryPool uses lazy allocation up to max_chunks, then blocks waiting for buffer release * z * z
PreviousNext