AITER Development Roadmap (2026 Q3)

Modeled on the SGLang AMD roadmap (sgl-project/sglang#23494). Last updated 2026-05-30.

Contributions and feedback are welcome.

Legend: ✓ done · ▶ in progress · ○ planned. Each item links a tracked PR/issue.

Focus

• Model enablement velocity: Day-N kernels for the frontier MoE models driving inference demand — MiniMax-M2.5, GLM-5.x, GPT-OSS, DeepSeek-V4.
• MXFP4 across the stack: MoE, GEMM, attention, and Sage attention on MXFP4 with assured accuracy.
• MLA completeness: Close the remaining MLA feature gaps (FP8 KV cache, small head counts, speculative decode).
• Build & architecture: CK-Free build, faster Python-binding compile, config loading without rebuild — reduce time-to-kernel for agentic workflows.
• Training kernels (new): First grouped-GEMM APIs for MoE training, beyond inference.

Feature and Performance Improvements (Q3 planned)

• MLA capability completion
  PoC: @larryli2-amd @minmengdie
  Goal: Fill the high-priority MLA gaps for production MoE serving.

  • ○ FP8 KV cache for MLA [Feature]: Add FP8 KV Cache support to Aiter MLA #899 (must-have)
  • ○ MLA decode nhead=8 [Feature]: decode MLA kernel support nhead=8 #726 (high priority)
  • ○ MLA decode ASM qseqlen>4 for spec-decode [Feature]: Add MLA decode ASM kernels for qseqlen > 4 to enable larger speculative decoding draft tokens #2760 (@larryli2-amd)
  • ○ head_dim=256 for fmha_v3_fwd [Feature]: Add head_dim=256 support to fmha_v3_fwd kernels #2013

• Frontier MoE model enablement
  PoC: @peymanr @akii96 @andyluo7
  Goal: Day-N kernels for the models with active customer demand.

  • ▶ MiniMax-M2.5 FP8 on MI355X [RFC] MiniMax-M2.5 FP8 — Marathon Optimized (MI355X) #3191 [RFC] MiniMax-M2.5 FP8 — Marathon Optimized (MI355X) #3192 [RFC] MiniMax-M2.5 FP8 — Marathon Optimized (MI355X) #3193 [RFC] MiniMax-M2.5 FP8 — Marathon Optimized (MI355X) #3195 (@peymanr)
  • ▶ GLM-5.1 FP8 blockscale GEMM/FMoE tunings Add GLM-5.1 FP8 blockscale GEMM/FMoE tunings for gfx942 (MI300X/MI325) #3422 (@akii96)
  • ○ gqa_ratio=8 MLA kernels for GLM-5 HiSparse [MLA] Missing gqa_ratio=8 kernels in mla_decode_stage1_asm_fwd (blocks GLM-5 HiSparse on MI355X TP=8) #2821 (@andyluo7)
  • ○ gpt-oss MoE with bias [Feature]: AITER fused MoE to support gpt-oss MoE with bias #898 (high priority)

• MXFP4 full stack
  PoC: @haoyangli0109 @aoli26 @ksikiric
  Goal: MXFP4 MoE / GEMM / attention with minimal accuracy loss.

  • ▶ MXFP4 MoE at TP=4/8 via CKTile a4w4 [Bugfix] Enable MXFP4 MoE at TP=4/8 via CKTile a4w4 kernels and quant fixes #3412 (@haoyangli0109)
  • ○ FlyDSL MXScale FP8 / A8W4 GEMM [gfx1250][FlyDSL] Add PTPC FP8 GEMM #3106 (@aoli26)
  • ○ Sage MXFP4 attention return LSE [Triton] Sage MXFP4 return LSE #3349 (@ksikiric)

• MoE kernel optimization
  PoC: @valarLip @nsusanto @zx3xyy
  Goal: Next-stage MoE perf (no-combine, preshuffle, EP reduce).

  • ○ no-combine flag 2-stage MoE [FlyDSL MoE] Add no combine feature #3408 [feat] add no_combine flag in 2-stage MoE backend #3361 (@zx3xyy)
  • ○ Gluon MoE a8w4 preshuffle weights [Gluon] Moe a8w4 preshuffle weights #3405 (@nsusanto)
  • ○ FlyDSL MoE EP reduce path flydsl moe: EP reduce path with masked gather + tuned-key fix #3377 (@yadaish)

• Training kernels (new direction)
  PoC: @jayfurmanek
  Goal: Enable MoE training, not just inference.

  • ○ FP8 / MXFP8 grouped-GEMM API for MoE training [Feature] [Training] FP8/MXFP8 grouped-GEMM API for MoE training #3038 (@jayfurmanek)

• Build & architecture RFCs
  PoC: @carlushuang @sunway513 @tjtanaa
  Goal: Reduce time-to-kernel and build complexity.

  • ○ CK-Free build (build AITER without Composable Kernel) [RFC] CK-Free Build: Enable AITER builds without Composable Kernel #2073 (@sunway513)
  • ○ Migrate pybind11 → apache-tvm-ffi [Feature]: Migrate Python bindings from pybind11 to apache-tvm-ffi #2054 (@carlushuang)
  • ○ Improve Python-binding compile time [RFC][Agentic/Low-Latency] improve compile time of python binding #2076 (@carlushuang)
  • ○ Replace CK MoE sorting with Opus [Feature]: replace CK moe sorting with opus #2093 (@carlushuang)
  • ○ Load custom tuned configs without AITER_REBUILD=1 [Feature]: Allow loading custom tuned kernel configuration file without AITER_REBUILD=1 with Prebuilt AITER #2797 (@tjtanaa)

• gfx1250 / MI450 enablement (new)
  PoC: @sunway513 @farlukas @aoli26
  Goal: Bring AITER kernel coverage to gfx1250 across FlyDSL, Gluon, Triton, and CK paths; clear the known JIT/asm bring-up bugs.

  • ▶ FlyDSL gfx1250 MoE 2-stage kernel modules [FlyDSL] Add gfx1250 MoE 2-stage kernel modules #3241 (@XingerZhu)
  • ▶ FlyDSL gfx1250 mha integration [gfx1250][FLYDSL] mha with qkdim 192 for DSv3 & KIMI #3216 (@Zzz9990)
  • ▶ FlyDSL MXScale FP8 / A8W4 GEMM [gfx1250][FlyDSL] Add PTPC FP8 GEMM #3106 (@aoli26)
  • ▶ Gluon MXFP4 preshuffled GEMM [Gluon][gfx1250] Gemm MXFP4 preshuffled #3359 (@Boss2002n)
  • ▶ Gluon fused_mxfp4_quant [Gluon] fused_mxfp4_quant for gfx1250 #3093 (@amd-jrosas)
  • ▶ Gluon MoE A4W4 kernel GFX1250 Gluon MoE A4W4 Kernel #2337 [TRITON] [GLUON] GFX1250 Gluon MoE A4W4 Kernel #2513 (@farlukas)
  • ▶ Triton GEMM A16W16 [Triton] [gfx1250] GEMM A16W16 Kernel #2971 (@azaidy)
  • ○ C++ JIT build support gfx1250 C++ JIT build support #2301 (@sunway513) — unblock JIT build failures gfx1250: C++ JIT build fails — DPP broadcast + vmem-to-lds-load-insts not supported #2299
  • ○ Fix asm_gemm_a16w16 crash gfx1250: asm_gemm_a16w16 crashes with 'get_heuristic_kernel not find kernel for bf16gemm' #2519 · gemm_afp4wfp4 inf/garbage on gfx1250 gemm_afp4wfp4 Triton kernel produces inf/garbage on gfx1250 #2543

• Communication / distributed
  PoC: @hubertlu-tw @xytpai
  Goal: Native collective ops and EP scaling.

  • ▶ Native int all-gather + gfx942 opt Add native integer all-gather dtype support and optimize gfx942 custom all-gather #3430 (@hubertlu-tw)
  • ○ Reduce-scatter dim!=0 fix Fix reduce scatter when dim != 0 #3395 (@yitingw1)

• CI quality gates
  PoC: @gyohuangxin
  Goal: Catch regressions before release.

  • ○ Perf-regression benchmark tests in releases [Feature]: Add benchmark tests to detect performance regressions in Aiter releases #2102 (@gyohuangxin)

TODO: WIP.