Accurate Simulation of Distributed Training Jobs with Network Contention Modeling — accepted to IEEE MASCOTS 2026, Genova, Italy (Oct 2026). A GPU-cluster simulator that models distributed-training jobs under dynamic network contention. (Co-first author.) [code]
Found and closed a silent no-op in --kv-cache-dtype — quantized KV dtypes were accepted and logged as a memory saving while the pool stayed in the model dtype, merged.
`fp8`, `fp8_e5m2`, `int8_per_token_head`, `int4_per_token_head` and `nvfp4*` all fell straight through `check_and_update_config`. vLLM core printed its "reduces the GPU memory footprint" line and the server came up healthy, while the paged KV cache stayed allocated in the model dtype. Only `turboquant_*` was rejected, and only incidentally, by an unrelated guard. So the flag users reach for to halve KV memory did nothing and logged the opposite.
Fixed by rejecting the unsupported names next to that guard, with a `NotImplementedError` pointing at `--kv-cache-dtype auto` or TurboQuant; `auto`, `float16` and `bfloat16` untouched. Verified on an M1 Max (vLLM 0.29.0, MLX 0.32.1): `auto` still allocates 46,401 blocks × 196,608 B, and `fp8` — which previously allocated that same pool while advertising a saving — now fails at startup with actionable text. Four regression cases fail on main and pass here.
Found and fixed silent KV-cache under-reporting for MLA and YOCO layouts — up to ~half the allocated GPU pool was unreachable, merged into v0.3.0.dev.
`get_kv_cache_spec` advertised a KV layout vllm-metal never allocated. For MLA it emitted a plain `FullAttentionSpec` with a hardcoded 2× K/V page size, even though MLA caches a single latent tensor per layer — so vLLM planned against exactly half the pool. For YOCO it emitted a spec for every layer, even though only the leading layers own a cache. The remaining Metal buffers sat allocated and unreachable — the exact mirror of the over-subscription guard I added in #527.
Fixed by describing the layout that was actually allocated (`MLAAttentionSpec` for MLA; specs only for owning layers for YOCO), leaving the physically-correct byte budget untouched. Validated on real weights : capacity round-tripped 0.500 → 1.000 (MLA) and 0.4286 → 1.000 (YOCO), with max concurrency at 4,096 tokens roughly doubling — 75× → 150× and 123× → 287× — outputs bit-identical before and after.
Found and fixed a silent out-of-bounds GPU write in vLLM's Apple Silicon backend — merged into v0.3.0.dev.
`--num-gpu-blocks-override` was honored by the vLLM scheduler but never reached vllm-metal's paged KV allocator, so an override above the profiled capacity let the engine address blocks that were never allocated on the Metal side. The result was a silent out-of-bounds GPU write — the server started and reported healthy while the scheduler oversubscribed the pool. Added fail-fast validation that rejects any engine KV config larger than the allocated pool. My first contribution to the project.
Found and fixed an AR-Diffusion KV pool leak that permanently bricks the DreamZero OpenPI server after a few sessions — fix under review.
Every new `session_id` took AR-Diffusion KV pool blocks that were never released, so four runs of the shipped OpenPI example were enough to exhaust the pool — after which every request failed to allocate, permanently. The process stayed up and `/health` kept returning 200, so a liveness probe never noticed.
A session's blocks are freed only by eviction, and the sole eviction trigger was the `MAX_DREAMZERO_SESSIONS = 64` count cap — chosen independently of pool capacity. The pool floor is one session's window (24 blocks at ~721 MB each on this config), so hitting 64 sessions would need roughly 277 GB of KV pool: the cap is unreachable, and raising `gpu_memory_fraction` only delays the failure.
Fixed by evicting LRU sessions on pool capacity as well as count, so the bound the cap was meant to provide actually holds. Validated on `GEAR-Dreams/DreamZero-DROID`, 1×GB10 (DGX Spark) : 8 consecutive runs of the shipped example clean where run 4 previously died — pool exhaustions 14 → 0, free blocks returning to full every session, action outputs identical to a pre-fix run. Added a regression test that drives session-id churn with the shipped cap unchanged.
Found and reported a race-condition crash in LMCache's local disk backend.
Under heavy cache eviction, read_file removed a missing key from the index on FileNotFoundError, but the caller then accessed that same key without checking — a KeyError that took down the whole engine.
Diagnosed the root cause, reported it with the failing code path, and verified the maintainer's fix on A100×4 over a multi-hour repro.