Checklist

Attention Kernel

• MHA (fwd (prefill, decode), bwd)
• MLA (fwd decode)
• GQA/MQA (fwd (prefill, decode), bwd)
• Sparse Attention (fwd (prefill, decode), bwd)
• Linear Attention (fwd (prefill, decode), bwd)
• Mamba2 (fwd (prefill, decode), bwd)
• Bitnet (fwd (prefill, decode), bwd)
• NSA (fwd (prefill, decode), bwd)
• GTA (fwd (prefill, decode), bwd)
• GLA (fwd (prefill, decode), bwd)
• SeerAttention

Features

• Profiling
• Check results
• Auto-tune