Motivation

The oneshot MoE sorting kernel runs on a single CU and wins for small token counts, but the multi-phase (MP) kernel distributes work across all CUs and is faster past a hardware-dependent crossover. The current dispatch heuristic (moe_sorting_is_oneshot) only checks LDS capacity — whether oneshot can run — not whether it should. On gfx950 with 128 experts, oneshot is selected up to ~61 tokens even though MP is faster past ~20 tokens.
A fixed threshold doesn't generalize: the crossover depends on CU count, memory subsystem, and expert count, which vary across CDNA generations (gfx90a/gfx942/gfx950) and models (8 to 256 experts).

Technical Details

Refactors moe_sorting_opus() into moe_sorting_opus_oneshot() + an autotuned dispatcher in csrc/include/moe_sorting_opus.h.
On first encounter of each (tokens, num_experts) pair with dispatch_policy=0:

1. LDS capacity check — if oneshot doesn't fit, go straight to MP (unchanged behavior)
2. Time both oneshot and MP with hipEvent on the current stream
3. Cache the winner in a static unordered_map
4. Re-run the winner to produce correct output
   Subsequent calls with the same pair hit the cache (~ns lookup, zero GPU overhead). Autotuning cost is ~80 unique pairs × 3 kernel launches = ~240 launches, absorbed by vLLM's existing warmup phase.
   dispatch_policy=1 (force oneshot) and dispatch_policy=2 (force MP) bypass the autotune entirely and behave exactly as before.

Safety

• Strict superset of the old behavior: above the LDS capacity limit the codepath is identical. Below it, the autotune either agrees with the old heuristic (same perf) or switches to MP (only if MP measured faster).
• No impact on non-MoE models: the function is never called for dense architectures.
• No impact on few-expert MoE models (e.g. Mixtral, 8 experts): tokens × 8 rarely enters the autotunable zone; the LDS capacity check handles dispatch.

Test Plan

• Existing op_tests/test_moe_sorting.py for correctness at all rerouted shapes
• 5-repeat A/B serving benchmarks (ISL=1000, OSL=100, random dataset) on two MoE models with different expert counts and TP configurations

Test Result

Measured on gfx950 (MI355X), vLLM 0.22.0: