Thanks to visit codestin.com
Credit goes to github.com

Skip to content

mxfp4fp8 fmha gfx950#3495

Open
jcaraban wants to merge 1 commit into
i8fp8_fmha_gfx950from
mxfp4fp8_fmha_gfx950
Open

mxfp4fp8 fmha gfx950#3495
jcaraban wants to merge 1 commit into
i8fp8_fmha_gfx950from
mxfp4fp8_fmha_gfx950

Conversation

@jcaraban
Copy link
Copy Markdown
Contributor

@jcaraban jcaraban commented Jun 2, 2026 •
edited
Loading

Motivation

Follows #3186, now with mxfp4 Q·K

bench_sage --kernel=all  (b=1 hq=5 sq=75600 sk=75600 d=128 input=transformer):
kernel             time(ms)     TFLOPS          MAE         MaxE       Cosine
------------------------------------------------------------------------------
sage_fp8             9.7205    1505.20    5.493e-04    6.250e-02     0.998512
sage_mxfp4           7.9692    1835.98    2.522e-03    4.375e-01     0.968982
aiter_fp8            7.2131    2028.43    8.275e-04    2.305e-01     0.996718
aiter_i8fp8          7.8374    1866.87    7.299e-04    9.375e-02     0.997669
aiter_mxfp4          6.6262    2208.10    2.528e-03    4.141e-01     0.968871   <---
aiter_bf16          12.2339    1195.97    0.000e+00    0.000e+00     1.000000

Technical Details

Test Plan

Test Result

Submission Checklist

@jcaraban jcaraban requested review from a team, JohnNikolay84 and minmengdie June 2, 2026 19:28
@jcaraban jcaraban changed the base branch from main to i8fp8_fmha_gfx950 June 2, 2026 19:29
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 3495 --add-label <label>

@jcaraban jcaraban requested a review from valarLip June 2, 2026 19:29
@jcaraban jcaraban mentioned this pull request Jun 3, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JohnNikolay84 JohnNikolay84 Awaiting requested review from JohnNikolay84

@minmengdie minmengdie Awaiting requested review from minmengdie

@valarLip valarLip Awaiting requested review from valarLip

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jcaraban