> ## Documentation Index
> Fetch the complete documentation index at: https://bench.flashinfer.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Model Coverage

# Model Kernel Coverage

This document tracks which kernels are supported in FlashInfer-Bench for each model.

* ✅ Definition JSON exists and workload has been collected
* 🟡 Definition JSON exists but workload has not yet been collected
* ❌ Definition is referenced in `models.ts` but the file does not exist (missing)
* — Module exists in the architecture but no definition is mapped (unmapped)

## Summary

| Model                     | Architecture         |     Coverage    |
| ------------------------- | -------------------- | :-------------: |
| DeepSeek V3/R1            | MLA + Dense/MoE      |    🟡 Partial   |
| DeepSeek V3.2             | DSA + Dense/MoE      | ✅ Fully covered |
| Llama 3.1 8B              | GQA + Dense          | ✅ Fully covered |
| Llama 3.1/3.3 70B         | GQA + Dense          |    🟡 Partial   |
| Llama 3.2 3B              | GQA + Dense          |    🟡 Partial   |
| Mistral 7B v0.3           | GQA + Dense          |    🟡 Partial   |
| Mistral Nemo 12B          | GQA + Dense          |    🟡 Partial   |
| Mixtral 8x7B              | GQA + MoE            |    🟡 Partial   |
| Mixtral 8x22B             | GQA + MoE            |    🟡 Partial   |
| Qwen2.5 7B                | GQA + Dense          |    🟡 Partial   |
| Qwen2.5 72B               | GQA + Dense          |    🟡 Partial   |
| Qwen3 8B                  | GQA + Dense          |    🟡 Partial   |
| Qwen3 30B A3B             | GQA + MoE            |    🟡 Partial   |
| Qwen3 32B                 | GQA + Dense          |    🟡 Partial   |
| Qwen3 235B A22B           | GQA + MoE            |    🟡 Partial   |
| Qwen3 Next 80B A3B        | GDN + GQA + MoE      |    🟡 Partial   |
| Kimi K2                   | MLA + MoE            |    🟡 Partial   |
| Phi-4 14B                 | GQA + Dense          |    🟡 Partial   |
| Llama 3.1 405B            | GQA + Dense          |    🟡 Partial   |
| Llama 4 Scout 17B-16E     | GQA + MoE            |    🟡 Partial   |
| Llama 4 Maverick 17B-128E | GQA + MoE            |    🟡 Partial   |
| Mistral Small 3.1 24B     | GQA + Dense          |    🟡 Partial   |
| GLM-4.6                   | GQA + Dense          |  ❌ Not covered  |
| MiniMax-Text-01           | Lightning Attn + MoE |  ❌ Not covered  |
| MiniMax M2                | GQA + MoE            |    🟡 Partial   |
| Gemma 3 27B               | GQA + Dense          |    🟡 Partial   |
| Qwen3 14B                 | GQA + Dense          |    🟡 Partial   |
| NemotronH 47B             | GQA + Mamba2 Hybrid  |  ❌ Not covered  |

***

## DeepSeek V3 / R1

**Architecture**: 61 decoder layers, MLA attention, hybrid Dense+MoE FFN

| Definition                                                     | Op Type     | Status |
| -------------------------------------------------------------- | ----------- | :----: |
| `rmsnorm_h7168`                                                | rmsnorm     |    ✅   |
| `fused_add_rmsnorm_h7168`                                      | rmsnorm     |    ✅   |
| `rmsnorm_h1536`                                                | rmsnorm     |    ✅   |
| `rmsnorm_h512`                                                 | rmsnorm     |    ✅   |
| `gemm_n256_k7168`                                              | gemm        |    ✅   |
| `mla_ragged_prefill_causal_h16_qk192_vo128`                    | mla\_ragged |    ✅   |
| `mla_paged_prefill_causal_h16_ckv512_kpe64_ps1`                | mla\_paged  |    ✅   |
| `mla_paged_prefill_causal_h16_ckv512_kpe64_ps64`               | mla\_paged  |    ✅   |
| `mla_paged_decode_h16_ckv512_kpe64_ps1`                        | mla\_paged  |    ✅   |
| `mla_paged_decode_h16_ckv512_kpe64_ps64`                       | mla\_paged  |    ✅   |
| `moe_fp8_block_scale_ds_routing_topk8_ng8_kg4_e32_h7168_i2048` | moe         |    ✅   |
| `top_k_sampling_from_probs_v129280`                            | sampling    |    ✅   |
| `top_k_top_p_sampling_from_probs_v129280`                      | sampling    |    ✅   |
| `top_p_sampling_from_probs_v129280`                            | sampling    |    ✅   |

**Coverage**: 13 / 14 definitions present. Missing: MLA ragged prefill definition.

***

## DeepSeek V3.2

**Architecture**: 61 decoder layers, DSA (DeepSeek Sparse Attention) replacing dense MLA, hybrid Dense+MoE FFN. Standard serving configuration: **TP=8**.

DSA introduces a learned TopK indexer that selects a sparse subset of KV pages before running attention, reducing computation for long contexts while preserving accuracy.

| Definition                                                     | Op Type    | Status |
| -------------------------------------------------------------- | ---------- | :----: |
| `rmsnorm_h7168`                                                | rmsnorm    |    ✅   |
| `fused_add_rmsnorm_h7168`                                      | rmsnorm    |    ✅   |
| `rmsnorm_h1536`                                                | rmsnorm    |    ✅   |
| `rmsnorm_h512`                                                 | rmsnorm    |    ✅   |
| `dsa_topk_indexer_fp8_h64_d128_topk2048_ps64`                  | dsa\_paged |    ✅   |
| `dsa_sparse_attention_h16_ckv512_kpe64_topk2048_ps1`           | dsa\_paged |    ✅   |
| `dsa_sparse_attention_h16_ckv512_kpe64_topk2048_ps64`          | dsa\_paged |    ✅   |
| `moe_fp8_block_scale_ds_routing_topk8_ng8_kg4_e32_h7168_i2048` | moe        |    ✅   |

**Coverage**: 8 / 8 definitions present. Fully covered.

***

## Llama 3.1 8B

**Architecture**: 32 decoder layers, GQA attention, dense MLP

| Definition                                   | Op Type     | Status |
| -------------------------------------------- | ----------- | :----: |
| `rmsnorm_h4096`                              | rmsnorm     |    ✅   |
| `fused_add_rmsnorm_h4096`                    | rmsnorm     |    ✅   |
| `gemm_n6144_k4096`                           | gemm        |    ✅   |
| `gemm_n4096_k4096`                           | gemm        |    ✅   |
| `gemm_n28672_k4096`                          | gemm        |    ✅   |
| `gemm_n4096_k14336`                          | gemm        |    ✅   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps1`  | gqa\_paged  |    ✅   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps64` | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps1`          | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps64`         | gqa\_paged  |    ✅   |
| `gqa_ragged_prefill_causal_h32_kv8_d128`     | gqa\_ragged |    ✅   |
| `top_k_sampling_from_probs_v128256`          | sampling    |    ✅   |
| `top_k_top_p_sampling_from_probs_v128256`    | sampling    |    ✅   |
| `top_p_sampling_from_probs_v128256`          | sampling    |    ✅   |

**Coverage**: 14 / 14 definitions present. Fully covered.

***

## Qwen3 30B A3B

**Architecture**: 32 decoder layers, GQA attention, MoE FFN (30 MoE + 2 dense layers)

| Definition                                                | Op Type                  | Status |
| --------------------------------------------------------- | ------------------------ | :----: |
| `rmsnorm_h128`                                            | rmsnorm                  |    ✅   |
| `rmsnorm_h2048`                                           | rmsnorm                  |    ✅   |
| `fused_add_rmsnorm_h2048`                                 | rmsnorm                  |    ✅   |
| `gemm_n128_k2048`                                         | gemm                     |    ✅   |
| `gemm_n2048_k4096`                                        | gemm                     |    ✅   |
| `gemm_n5120_k2048`                                        | gemm                     |    ✅   |
| `gqa_paged_prefill_causal_h32_kv4_d128_ps1`               | gqa\_paged               |    ✅   |
| `gqa_paged_prefill_causal_h32_kv4_d128_ps64`              | gqa\_paged               |    ✅   |
| `gqa_paged_decode_h32_kv4_d128_ps1`                       | gqa\_paged               |    ✅   |
| `gqa_paged_decode_h32_kv4_d128_ps64`                      | gqa\_paged               |    ✅   |
| `gqa_ragged_prefill_causal_h32_kv4_d128`                  | gqa\_ragged              |    ✅   |
| `top_k_sampling_from_probs_v151936`                       | sampling                 |    ✅   |
| `top_k_top_p_sampling_from_probs_v151936`                 | sampling                 |    ✅   |
| `top_p_sampling_from_probs_v151936`                       | sampling                 |    ✅   |
| MoE gate / topk / experts                                 | moe                      |    —   |
| `moe_fp8_block_scale_renorm_topk8_e128_h2048_i768`        | moe EP=1                 |   🟡   |
| `trtllm_fp4_block_scale_moe_topk8_e128_h2048_i768`        | moe (TRT-LLM FP4)        |   🟡   |
| `trtllm_fp4_block_scale_routed_moe_topk8_e128_h2048_i768` | moe (TRT-LLM FP4 routed) |   🟡   |
| `trtllm_fp8_per_tensor_scale_moe_topk8_e128_h2048_i768`   | moe (TRT-LLM FP8)        |   🟡   |

**Coverage**: 14 / 14 referenced definitions present. MoE kernels added (not yet mapped in `models.ts`).

***

## Qwen3 Next 80B A3B

**Architecture**: 48 layers total — 36 GDN (linear attention) + 12 GQA (standard attention), all layers use MoE FFN. Standard serving configuration: **TP=2 or TP=4**.

| Definition                                                 | Op Type                        | Status |
| ---------------------------------------------------------- | ------------------------------ | :----: |
| `rmsnorm_h2048`                                            | rmsnorm                        |    ✅   |
| `fused_add_rmsnorm_h2048`                                  | rmsnorm                        |    ✅   |
| `gdn_prefill_qk16_v32_d128_k_last`                         | gdn TP=1                       |   🟡   |
| `gdn_prefill_qk8_v16_d128_k_last`                          | gdn TP=2                       |    ✅   |
| `gdn_prefill_qk4_v8_d128_k_last`                           | gdn TP=4                       |    ✅   |
| `gdn_decode_qk16_v32_d128_k_last`                          | gdn TP=1                       |   🟡   |
| `gdn_decode_qk8_v16_d128_k_last`                           | gdn TP=2                       |    ✅   |
| `gdn_decode_qk4_v8_d128_k_last`                            | gdn TP=4                       |    ✅   |
| `gdn_mtp_qk16_v32_d128_k_last`                             | gdn TP=1                       |   🟡   |
| `gdn_mtp_qk8_v16_d128_k_last`                              | gdn TP=2                       |    ✅   |
| `gdn_mtp_qk4_v8_d128_k_last`                               | gdn TP=4                       |    ✅   |
| `gqa_paged_prefill_causal_h8_kv1_d256_ps1`                 | gqa\_paged TP=2                |    ❌   |
| `gqa_paged_decode_h8_kv1_d256_ps1`                         | gqa\_paged TP=2                |    ❌   |
| `gqa_ragged_prefill_causal_h8_kv1_d256`                    | gqa\_ragged TP=2               |    ✅   |
| MoE gate / topk / experts (GDN layers)                     | moe                            |    —   |
| MoE gate / topk / experts (GQA layers)                     | moe                            |    —   |
| `moe_fp8_block_scale_renorm_topk10_e128_h2048_i512`        | moe EP=1                       |   🟡   |
| `trtllm_fp4_block_scale_moe_topk10_e128_h2048_i512`        | moe (TRT-LLM FP4, EP=4)        |   🟡   |
| `trtllm_fp4_block_scale_routed_moe_topk10_e128_h2048_i512` | moe (TRT-LLM FP4 routed, EP=4) |   🟡   |

**Coverage**: 10 / 14 referenced definitions present. MoE definition added (shared across GDN and GQA layers).

Missing GDN definitions: TP=1 prefill and decode (qk16\_v32). Missing GQA: h=8, kv=1, d=256 (TP=2 of original h=16, kv=2, d=256).

***

## Llama 3.1 / 3.3 70B

**Architecture**: 80 decoder layers, GQA attention, dense MLP. Standard serving configuration: **TP=4** (from sgl-cookbook).

Llama 3.1 70B and 3.3 70B share identical architecture dimensions; only training data and context window differ.

| Definition                                   | Op Type          | Status |
| -------------------------------------------- | ---------------- | :----: |
| `rmsnorm_h8192`                              | rmsnorm          |    ❌   |
| `fused_add_rmsnorm_h8192`                    | rmsnorm          |    ❌   |
| `gqa_paged_prefill_causal_h16_kv2_d128_ps1`  | gqa\_paged TP=4  |    ✅   |
| `gqa_paged_prefill_causal_h16_kv2_d128_ps64` | gqa\_paged TP=4  |    ✅   |
| `gqa_paged_decode_h16_kv2_d128_ps1`          | gqa\_paged TP=4  |    ✅   |
| `gqa_paged_decode_h16_kv2_d128_ps64`         | gqa\_paged TP=4  |    ✅   |
| `gqa_ragged_prefill_causal_h16_kv2_d128`     | gqa\_ragged TP=4 |    ✅   |
| `gemm_n10240_k8192`                          | gemm             |   🟡   |
| `gemm_n8192_k8192`                           | gemm             |   🟡   |
| `gemm_n57344_k8192`                          | gemm             |   🟡   |
| `gemm_n8192_k28672`                          | gemm             |   🟡   |
| `top_k_sampling_from_probs_v128256`          | sampling         |    ✅   |
| `top_k_top_p_sampling_from_probs_v128256`    | sampling         |    ✅   |
| `top_p_sampling_from_probs_v128256`          | sampling         |    ✅   |

**Coverage**: 14 / 14 definitions present. GQA kernels shared with Qwen3-32B (same h=16, kv=2, d=128 at TP=4).

***

## Llama 3.2 3B

**Architecture**: 28 decoder layers, GQA attention, dense MLP.

| Definition                                   | Op Type     | Status |
| -------------------------------------------- | ----------- | :----: |
| `rmsnorm_h3072`                              | rmsnorm     |   🟡   |
| `fused_add_rmsnorm_h3072`                    | rmsnorm     |   🟡   |
| `gqa_paged_prefill_causal_h24_kv8_d128_ps1`  | gqa\_paged  |    ✅   |
| `gqa_paged_prefill_causal_h24_kv8_d128_ps64` | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h24_kv8_d128_ps1`          | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h24_kv8_d128_ps64`         | gqa\_paged  |    ✅   |
| `gqa_ragged_prefill_causal_h24_kv8_d128`     | gqa\_ragged |    ✅   |
| `gemm_n5120_k3072`                           | gemm        |   🟡   |
| `gemm_n3072_k3072`                           | gemm        |   🟡   |
| `gemm_n16384_k3072`                          | gemm        |   🟡   |
| `gemm_n3072_k8192`                           | gemm        |   🟡   |
| `top_k_sampling_from_probs_v128256`          | sampling    |    ✅   |
| `top_k_top_p_sampling_from_probs_v128256`    | sampling    |    ✅   |
| `top_p_sampling_from_probs_v128256`          | sampling    |    ✅   |

**Coverage**: 6 / 14 definitions present. GQA ragged prefill kernel added. Missing: rmsnorm h3072, paged GQA variants, and GEMM kernels for hidden=3072.

***

## Mistral 7B v0.3

**Architecture**: 32 decoder layers, GQA attention, dense MLP. Shares identical hidden, attention, and MLP dimensions with Llama 3.1 8B (hidden=4096, 32q/8kv heads, intermediate=14336).

| Definition                                   | Op Type     | Status |
| -------------------------------------------- | ----------- | :----: |
| `rmsnorm_h4096`                              | rmsnorm     |    ✅   |
| `fused_add_rmsnorm_h4096`                    | rmsnorm     |    ✅   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps1`  | gqa\_paged  |    ✅   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps64` | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps1`          | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps64`         | gqa\_paged  |    ✅   |
| `gqa_ragged_prefill_causal_h32_kv8_d128`     | gqa\_ragged |    ✅   |
| `gemm_n6144_k4096`                           | gemm        |    ✅   |
| `gemm_n4096_k4096`                           | gemm        |    ✅   |
| `gemm_n28672_k4096`                          | gemm        |    ✅   |
| `gemm_n4096_k14336`                          | gemm        |    ✅   |
| `top_k_sampling_from_probs_v32000`           | sampling    |    ❌   |
| `top_k_top_p_sampling_from_probs_v32000`     | sampling    |    ❌   |
| `top_p_sampling_from_probs_v32000`           | sampling    |    ❌   |

**Coverage**: 11 / 14 definitions present. Missing: sampling definitions for vocab\_size=32000.

***

## Mistral Nemo 12B

**Architecture**: 40 decoder layers, GQA attention (explicit head\_dim=128), dense MLP. Standard serving configuration: **TP=1** (from sgl-cookbook).

| Definition                                   | Op Type     | Status |
| -------------------------------------------- | ----------- | :----: |
| `rmsnorm_h5120`                              | rmsnorm     |   🟡   |
| `fused_add_rmsnorm_h5120`                    | rmsnorm     |   🟡   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps1`  | gqa\_paged  |    ✅   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps64` | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps1`          | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps64`         | gqa\_paged  |    ✅   |
| `gqa_ragged_prefill_causal_h32_kv8_d128`     | gqa\_ragged |    ✅   |
| `gemm_n6144_k5120`                           | gemm        |    ❌   |
| `gemm_n5120_k4096`                           | gemm        |    ❌   |
| `gemm_n28872_k5120`                          | gemm        |    ❌   |
| `gemm_n5120_k14436`                          | gemm        |    ❌   |
| `top_k_sampling_from_probs_v131072`          | sampling    |    ❌   |
| `top_k_top_p_sampling_from_probs_v131072`    | sampling    |    ❌   |
| `top_p_sampling_from_probs_v131072`          | sampling    |    ❌   |

**Coverage**: 7 / 14 definitions present. GQA defs are shared with Llama 3.1 8B; rmsnorm h5120 is now shared with Qwen3 14B. Missing: all GEMM (different hidden=5120 input dim), sampling v131072.

***

## Mixtral 8x7B

**Architecture**: 32 decoder layers, GQA attention, sparse MoE FFN (8 experts, top-2 routing). Shares attention and normalization dimensions with Llama 3.1 8B / Mistral 7B.

| Definition                                   | Op Type     | Status |
| -------------------------------------------- | ----------- | :----: |
| `rmsnorm_h4096`                              | rmsnorm     |    ✅   |
| `fused_add_rmsnorm_h4096`                    | rmsnorm     |    ✅   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps1`  | gqa\_paged  |    ✅   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps64` | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps1`          | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps64`         | gqa\_paged  |    ✅   |
| `gqa_ragged_prefill_causal_h32_kv8_d128`     | gqa\_ragged |    ✅   |
| `gemm_n6144_k4096`                           | gemm        |    ✅   |
| `gemm_n4096_k4096`                           | gemm        |    ✅   |
| MoE experts (top-2, 8 experts, inter=14336)  | moe         |    —   |
| `top_k_sampling_from_probs_v32000`           | sampling    |    ❌   |
| `top_k_top_p_sampling_from_probs_v32000`     | sampling    |    ❌   |
| `top_p_sampling_from_probs_v32000`           | sampling    |    ❌   |

**Coverage**: 9 / 12 referenced definitions present. MoE uses standard top-2 routing (not DeepSeek FP8 block-scale), so the existing MoE definition does not apply (unmapped). Missing: sampling v32000.

***

## Mixtral 8x22B

**Architecture**: 56 decoder layers, GQA attention, sparse MoE FFN (8 experts, top-2 routing). All dimensions are new (hidden=6144, 48q/8kv heads). Standard serving configuration: **TP=2** (from sgl-cookbook), giving 24q/4kv heads per GPU.

| Definition                                   | Op Type     | Status |
| -------------------------------------------- | ----------- | :----: |
| `rmsnorm_h6144`                              | rmsnorm     |    ❌   |
| `fused_add_rmsnorm_h6144`                    | rmsnorm     |    ❌   |
| `gqa_paged_prefill_causal_h24_kv4_d128_ps64` | gqa\_paged  |    ✅   |
| `gqa_paged_prefill_causal_h48_kv8_d128_ps1`  | gqa\_paged  |    ❌   |
| `gqa_paged_prefill_causal_h48_kv8_d128_ps64` | gqa\_paged  |    ❌   |
| `gqa_paged_decode_h48_kv8_d128_ps1`          | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h48_kv8_d128_ps64`         | gqa\_paged  |    ❌   |
| `gqa_ragged_prefill_causal_h48_kv8_d128`     | gqa\_ragged |    ❌   |
| `gemm_n8192_k6144`                           | gemm        |    ❌   |
| `gemm_n6144_k6144`                           | gemm        |    ❌   |
| MoE experts (top-2, 8 experts, inter=16384)  | moe         |    —   |
| `top_k_sampling_from_probs_v32768`           | sampling    |    ❌   |
| `top_k_top_p_sampling_from_probs_v32768`     | sampling    |    ❌   |
| `top_p_sampling_from_probs_v32768`           | sampling    |    ❌   |

**Coverage**: 3 / 13 referenced definitions present. TP=2 prefill + decode definitions added. Missing: rmsnorm, remaining GQA variants, GEMM, and sampling definitions.

### Mixtral 8x22B at TP=2

At tensor parallelism TP=2, attention head counts are halved (48→24 q-heads, 8→4 kv-heads).

| Definition                                   | Op Type    | Status |
| -------------------------------------------- | ---------- | :----: |
| `gqa_paged_prefill_causal_h24_kv4_d128_ps64` | gqa\_paged |    ✅   |
| `gqa_paged_prefill_causal_h24_kv4_d128_ps1`  | gqa\_paged |    ✅   |
| `gqa_paged_decode_h24_kv4_d128_ps64`         | gqa\_paged |    ✅   |

**Coverage**: 3 / 3 TP=2 attention definitions present (prefill ps1 + ps64, decode ps64).

***

## Qwen2.5 7B

**Architecture**: 28 decoder layers, GQA attention, dense MLP.

| Definition                                   | Op Type     | Status |
| -------------------------------------------- | ----------- | :----: |
| `rmsnorm_h3584`                              | rmsnorm     |    ❌   |
| `fused_add_rmsnorm_h3584`                    | rmsnorm     |    ❌   |
| `gqa_paged_prefill_causal_h28_kv4_d128_ps1`  | gqa\_paged  |    ❌   |
| `gqa_paged_prefill_causal_h28_kv4_d128_ps64` | gqa\_paged  |    ❌   |
| `gqa_paged_decode_h28_kv4_d128_ps1`          | gqa\_paged  |    ❌   |
| `gqa_paged_decode_h28_kv4_d128_ps64`         | gqa\_paged  |    ❌   |
| `gqa_ragged_prefill_causal_h28_kv4_d128`     | gqa\_ragged |    ❌   |
| `gemm_n4608_k3584`                           | gemm        |   🟡   |
| `gemm_n3584_k3584`                           | gemm        |   🟡   |
| `gemm_n37888_k3584`                          | gemm        |   🟡   |
| `gemm_n3584_k18944`                          | gemm        |   🟡   |
| `top_k_sampling_from_probs_v152064`          | sampling    |   🟡   |
| `top_k_top_p_sampling_from_probs_v152064`    | sampling    |   🟡   |
| `top_p_sampling_from_probs_v152064`          | sampling    |   🟡   |

**Coverage**: 9 / 14 definitions present. Missing: all rmsnorm, GQA, and GEMM definitions for hidden=3584.

***

## Qwen2.5 72B

**Architecture**: 80 decoder layers, GQA attention, dense MLP. Standard serving configuration: **TP=8** (from sgl-cookbook).

| Definition                                  | Op Type          | Status |
| ------------------------------------------- | ---------------- | :----: |
| `rmsnorm_h8192`                             | rmsnorm          |    ❌   |
| `fused_add_rmsnorm_h8192`                   | rmsnorm          |    ❌   |
| `gqa_paged_prefill_causal_h8_kv1_d128_ps1`  | gqa\_paged TP=8  |    ❌   |
| `gqa_paged_prefill_causal_h8_kv1_d128_ps64` | gqa\_paged TP=8  |    ❌   |
| `gqa_paged_decode_h8_kv1_d128_ps1`          | gqa\_paged TP=8  |    ❌   |
| `gqa_paged_decode_h8_kv1_d128_ps64`         | gqa\_paged TP=8  |    ❌   |
| `gqa_ragged_prefill_causal_h8_kv1_d128`     | gqa\_ragged TP=8 |    ❌   |
| `gemm_n10240_k8192`                         | gemm             |    ❌   |
| `gemm_n8192_k8192`                          | gemm             |    ❌   |
| `gemm_n59392_k8192`                         | gemm             |    ❌   |
| `gemm_n8192_k29696`                         | gemm             |    ❌   |
| `top_k_sampling_from_probs_v152064`         | sampling         |   🟡   |
| `top_k_top_p_sampling_from_probs_v152064`   | sampling         |   🟡   |
| `top_p_sampling_from_probs_v152064`         | sampling         |   🟡   |

**Coverage**: 3 / 14 definitions present. Missing: rmsnorm h8192, all GQA definitions (h8\_kv1\_d128 at TP=8), all GEMM definitions for hidden=8192.

***

## Qwen3 8B

**Architecture**: 36 decoder layers, GQA attention, dense MLP. Shares hidden size and attention dimensions with Llama 3.1 8B (hidden=4096, 32q/8kv heads, head\_dim=128), but uses a larger MLP intermediate size (22016 vs 14336).

| Definition                                   | Op Type     | Status |
| -------------------------------------------- | ----------- | :----: |
| `rmsnorm_h4096`                              | rmsnorm     |    ✅   |
| `fused_add_rmsnorm_h4096`                    | rmsnorm     |    ✅   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps1`  | gqa\_paged  |    ✅   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps64` | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps1`          | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps64`         | gqa\_paged  |    ✅   |
| `gqa_ragged_prefill_causal_h32_kv8_d128`     | gqa\_ragged |    ✅   |
| `gemm_n6144_k4096`                           | gemm        |    ✅   |
| `gemm_n4096_k4096`                           | gemm        |    ✅   |
| `gemm_n44032_k4096`                          | gemm        |    ❌   |
| `gemm_n4096_k22016`                          | gemm        |    ❌   |
| `top_k_sampling_from_probs_v151936`          | sampling    |    ✅   |
| `top_k_top_p_sampling_from_probs_v151936`    | sampling    |    ✅   |
| `top_p_sampling_from_probs_v151936`          | sampling    |    ✅   |

**Coverage**: 12 / 14 definitions present. Missing: gate\_up GEMM (`gemm_n44032_k4096`, intermediate=22016 × 2) and down GEMM (`gemm_n4096_k22016`). All normalization, attention, and non-MLP GEMM kernels are shared with Llama 3.1 8B.

***

## Qwen3 32B

**Architecture**: 64 decoder layers, GQA attention, dense MLP. hidden=5120, 64 query heads, 8 KV heads, head\_dim=128, intermediate=25600. Standard serving configuration: **TP=4**.

| Definition                                   | Op Type          | Status |
| -------------------------------------------- | ---------------- | :----: |
| `rmsnorm_h5120`                              | rmsnorm          |    ✅   |
| `fused_add_rmsnorm_h5120`                    | rmsnorm          |    ✅   |
| `gqa_paged_prefill_causal_h16_kv2_d128_ps1`  | gqa\_paged TP=4  |    ✅   |
| `gqa_paged_prefill_causal_h16_kv2_d128_ps64` | gqa\_paged TP=4  |    ✅   |
| `gqa_paged_decode_h16_kv2_d128_ps1`          | gqa\_paged TP=4  |    ✅   |
| `gqa_paged_decode_h16_kv2_d128_ps64`         | gqa\_paged TP=4  |    ✅   |
| `gqa_ragged_prefill_causal_h16_kv2_d128`     | gqa\_ragged TP=4 |    ✅   |
| `gemm_n10240_k5120`                          | gemm (QKV)       |    ❌   |
| `gemm_n5120_k8192`                           | gemm (o\_proj)   |    ❌   |
| `gemm_n51200_k5120`                          | gemm (gate\_up)  |    ❌   |
| `gemm_n5120_k25600`                          | gemm (down)      |    ❌   |
| `top_k_sampling_from_probs_v151936`          | sampling         |    ✅   |
| `top_k_top_p_sampling_from_probs_v151936`    | sampling         |    ✅   |
| `top_p_sampling_from_probs_v151936`          | sampling         |    ✅   |

**Coverage**: 10 / 14 definitions present. RMSNorm shared with Qwen3 14B (same hidden=5120). GQA kernels shared with Llama 3.1/3.3 70B (same h=16, kv=2, d=128 at TP=4). Missing: all GEMM definitions.

***

## Qwen3 235B A22B

**Architecture**: 94 decoder layers, GQA attention, sparse MoE FFN (128 experts, top-8 routing). Uses head\_dim=128 (hidden=8192, 64 query heads). Standard serving configuration: **TP=8, EP=2** (FP8 variant from sgl-cookbook). With 4 KV heads, effective per-device TP for attention is TP=4 (kv=1 per device).

| Definition                                                     | Op Type                        | Status |
| -------------------------------------------------------------- | ------------------------------ | :----: |
| `rmsnorm_h4096`                                                | rmsnorm                        |    ✅   |
| `fused_add_rmsnorm_h4096`                                      | rmsnorm                        |    ✅   |
| `gqa_paged_prefill_causal_h16_kv1_d128_ps1`                    | gqa\_paged TP=4                |    ❌   |
| `gqa_paged_prefill_causal_h16_kv1_d128_ps64`                   | gqa\_paged TP=4                |    ✅   |
| `gqa_paged_decode_h16_kv1_d128_ps1`                            | gqa\_paged TP=4                |    ❌   |
| `gqa_paged_decode_h16_kv1_d128_ps64`                           | gqa\_paged TP=4                |    ❌   |
| `gqa_ragged_prefill_causal_h16_kv1_d128`                       | gqa\_ragged TP=4               |    ❌   |
| `gemm_n4608_k4096`                                             | gemm                           |    ❌   |
| `gemm_n4096_k4096`                                             | gemm                           |    ✅   |
| `moe_fp8_block_scale_renorm_topk8_e128_h4096_i1536`            | moe EP=1                       |   🟡   |
| `moe_fp8_block_scale_ds_routing_topk8_ng?_kg?_e64_h4096_i1536` | moe EP=2                       |    ❌   |
| `trtllm_fp4_block_scale_moe_topk8_e64_h4096_i1536`             | moe (TRT-LLM FP4, EP=2)        |   🟡   |
| `trtllm_fp4_block_scale_routed_moe_topk8_e64_h4096_i1536`      | moe (TRT-LLM FP4 routed, EP=2) |   🟡   |
| `top_k_sampling_from_probs_v151936`                            | sampling                       |    ✅   |
| `top_k_top_p_sampling_from_probs_v151936`                      | sampling                       |    ✅   |
| `top_p_sampling_from_probs_v151936`                            | sampling                       |    ✅   |

**Coverage**: 8 / 14 referenced definitions present. MoE EP=1 + TRT-LLM FP4 EP=2 definitions added. Missing: all GQA defs (head\_dim=64), QKV GEMM. The o\_proj GEMM and rmsnorm are shared with other h=4096 models.

***

## Kimi K2

**Architecture**: 61 decoder layers, MLA attention (same structure as DeepSeek V3), sparse MoE FFN (384 total experts, top-8 routing). Standard serving configuration: **TP=8, EP=4** (from sgl-cookbook).

Kimi K2 uses DeepSeek V3-style MLA with the same kv\_lora\_rank=512 and qk\_rope\_head\_dim=64, but has 64 attention heads (vs 128 in DeepSeek V3). With TP=8 this gives h=8, requiring separate MLA definitions from DeepSeek V3's h=16.

| Definition                                                      | Op Type         | Status |
| --------------------------------------------------------------- | --------------- | :----: |
| `rmsnorm_h7168`                                                 | rmsnorm         |    ✅   |
| `fused_add_rmsnorm_h7168`                                       | rmsnorm         |    ✅   |
| `rmsnorm_h1536`                                                 | rmsnorm         |    ✅   |
| `rmsnorm_h512`                                                  | rmsnorm         |    ✅   |
| `mla_paged_prefill_causal_h8_ckv512_kpe64_ps1`                  | mla\_paged TP=8 |   🟡   |
| `mla_paged_prefill_causal_h8_ckv512_kpe64_ps64`                 | mla\_paged TP=8 |    ❌   |
| `mla_paged_decode_h8_ckv512_kpe64_ps1`                          | mla\_paged TP=8 |   🟡   |
| `mla_paged_decode_h8_ckv512_kpe64_ps64`                         | mla\_paged TP=8 |    ❌   |
| `mla_ragged_prefill_causal_h8_qk192_vo128`                      | mla\_ragged     |   🟡   |
| `moe_fp8_block_scale_ds_routing_topk8_ng1_kg1_e384_h7168_i2048` | moe EP=1        |   🟡   |
| `moe_fp8_block_scale_ds_routing_topk8_ng?_kg?_e96_h7168_i2048`  | moe EP=4        |    ❌   |
| `moe_fp8_block_scale_ds_routing_topk8_ng1_kg1_e48_h7168_i2048`  | moe EP=8        |   🟡   |
| `top_k_sampling_from_probs_v160000`                             | sampling        |    ❌   |
| `top_k_top_p_sampling_from_probs_v160000`                       | sampling        |    ❌   |
| `top_p_sampling_from_probs_v160000`                             | sampling        |    ❌   |

**Coverage**: 6 / 15 definitions present. RMSNorm definitions are shared with DeepSeek V3 (same hidden=7168 and sub-module dims). MoE EP=1 and EP=8 definitions added. All MLA defs require new h=8 variants; MoE EP=4 variant (e=96) and sampling (v=160000) still missing.

***

## Phi-4 14B

**Architecture**: 40 decoder layers, GQA attention (unusual 10 KV heads), dense MLP. All dimensions are new for this project.

| Definition                                    | Op Type     | Status |
| --------------------------------------------- | ----------- | :----: |
| `rmsnorm_h5120`                               | rmsnorm     |   🟡   |
| `fused_add_rmsnorm_h5120`                     | rmsnorm     |   🟡   |
| `gqa_paged_prefill_causal_h40_kv10_d128_ps1`  | gqa\_paged  |    ✅   |
| `gqa_paged_prefill_causal_h40_kv10_d128_ps64` | gqa\_paged  |    ❌   |
| `gqa_paged_decode_h40_kv10_d128_ps1`          | gqa\_paged  |    ❌   |
| `gqa_paged_decode_h40_kv10_d128_ps64`         | gqa\_paged  |    ❌   |
| `gqa_ragged_prefill_causal_h40_kv10_d128`     | gqa\_ragged |    ❌   |
| `gemm_n7680_k5120`                            | gemm        |    ❌   |
| `gemm_n5120_k5120`                            | gemm        |   🟡   |
| `gemm_n35840_k5120`                           | gemm        |    ❌   |
| `gemm_n5120_k17920`                           | gemm        |    ❌   |
| `top_k_sampling_from_probs_v100352`           | sampling    |    ❌   |
| `top_k_top_p_sampling_from_probs_v100352`     | sampling    |    ❌   |
| `top_p_sampling_from_probs_v100352`           | sampling    |    ❌   |

**Coverage**: 4 / 14 definitions present. rmsnorm h5120 is now shared with Qwen3 14B; `gemm_n5120_k5120` (o\_proj shape) is shared since 40q\*128=5120=hidden; `gqa_paged_prefill_causal_h40_kv10_d128_ps1` has workloads collected (20/20 PASSED). Missing: remaining GQA defs (unusual 10 KV-head config), most GEMM, sampling v100352.

***

## Llama 3.1 405B

**Architecture**: 126 decoder layers, GQA attention, dense MLP. Standard serving configuration: **TP=4** (from sgl-cookbook). Uses the same Llama architecture as Llama 3.1 8B / 3.3 70B but at significantly larger scale (hidden=16384).

| Definition                                   | Op Type          | Status |
| -------------------------------------------- | ---------------- | :----: |
| `rmsnorm_h16384`                             | rmsnorm          |    ❌   |
| `fused_add_rmsnorm_h16384`                   | rmsnorm          |    ❌   |
| `gqa_paged_prefill_causal_h32_kv2_d128_ps1`  | gqa\_paged TP=4  |    ❌   |
| `gqa_paged_prefill_causal_h32_kv2_d128_ps64` | gqa\_paged TP=4  |    ❌   |
| `gqa_paged_decode_h32_kv2_d128_ps1`          | gqa\_paged TP=4  |    ❌   |
| `gqa_paged_decode_h32_kv2_d128_ps64`         | gqa\_paged TP=4  |    ❌   |
| `gqa_ragged_prefill_causal_h32_kv2_d128`     | gqa\_ragged TP=4 |    ❌   |
| `gemm_n18432_k16384`                         | gemm             |    ❌   |
| `gemm_n16384_k16384`                         | gemm             |    ❌   |
| `gemm_n106496_k16384`                        | gemm             |    ❌   |
| `gemm_n16384_k53248`                         | gemm             |    ❌   |
| `top_k_sampling_from_probs_v128256`          | sampling         |    ✅   |
| `top_k_top_p_sampling_from_probs_v128256`    | sampling         |    ✅   |
| `top_p_sampling_from_probs_v128256`          | sampling         |    ✅   |

**Coverage**: 3 / 14 definitions present. Sampling definitions are shared with Llama 3.1 8B (same vocab). Missing: rmsnorm h16384 and all GQA/GEMM definitions for this scale (TP=4 gives h=128/4=32 q-heads, kv=8/4=2 — the h32\_kv2 configuration does not exist in current definitions).

***

## Llama 4 Scout 17B-16E

**Architecture**: 48 decoder layers, interleaved GQA attention (NoPE global + RoPE local in 1:3 ratio), sparse MoE FFN (16 total experts, top-1 routing). Standard serving configuration: **TP=8** (from sgl-cookbook). Multimodal (vision+text).

> **Note**: Exact config.json values (hidden\_size, intermediate\_size) are pending verification from HuggingFace. Parameters below are estimates from the public model spec (17B activated parameters, 16 experts).

| Definition                                                | Op Type                                  | Status |
| --------------------------------------------------------- | ---------------------------------------- | :----: |
| `rmsnorm_h5120`                                           | rmsnorm                                  |   🟡   |
| `fused_add_rmsnorm_h5120`                                 | rmsnorm                                  |   🟡   |
| `gqa_paged_prefill_causal_h5_kv1_d128_ps1`                | gqa\_paged TP=8                          |    ✅   |
| `gqa_paged_prefill_causal_h5_kv1_d128_ps64`               | gqa\_paged TP=8                          |   🟡   |
| `gqa_paged_decode_h5_kv1_d128_ps1`                        | gqa\_paged TP=8                          |    ✅   |
| `gqa_paged_decode_h5_kv1_d128_ps64`                       | gqa\_paged TP=8                          |   🟡   |
| `gqa_ragged_prefill_causal_h5_kv1_d128`                   | gqa\_ragged TP=8                         |   🟡   |
| MoE experts (top-1, 16 experts, standard routing)         | moe                                      |    —   |
| `trtllm_fp4_block_scale_moe_topk1_e16_h5120_i8192`        | moe (TRT-LLM FP4, Llama4 routing)        |   🟡   |
| `trtllm_fp4_block_scale_routed_moe_topk1_e16_h5120_i8192` | moe (TRT-LLM FP4 routed, Llama4 routing) |   🟡   |
| `trtllm_fp8_per_tensor_scale_moe_topk1_e16_h5120_i8192`   | moe (TRT-LLM FP8)                        |   🟡   |
| `top_k_sampling_from_probs_v202048`                       | sampling                                 |   🟡   |
| `top_k_top_p_sampling_from_probs_v202048`                 | sampling                                 |   🟡   |
| `top_p_sampling_from_probs_v202048`                       | sampling                                 |   🟡   |

**Coverage**: 8 / 13 definitions present. rmsnorm h5120 shared with Qwen3 14B. TRT-LLM FP4 + FP8 MoE kernels added (top-1, 16 experts, Llama4 routing). Missing: ps64 GQA variants (no definition files), sampling v202048.

***

## Llama 4 Maverick 17B-128E

**Architecture**: Same base architecture as Llama 4 Scout but with 128 total experts (vs 16). Standard serving configuration: **TP=8** (from sgl-cookbook). hidden\_size=5120, 40 q-heads, 8 kv-heads, head\_dim=128, intermediate\_size=8192.

| Definition                                                 | Op Type                                  | Status |
| ---------------------------------------------------------- | ---------------------------------------- | :----: |
| `rmsnorm_h5120`                                            | rmsnorm                                  |   🟡   |
| `fused_add_rmsnorm_h5120`                                  | rmsnorm                                  |   🟡   |
| `gqa_paged_prefill_causal_h5_kv1_d128_ps1`                 | gqa\_paged TP=8                          |    ✅   |
| `gqa_paged_prefill_causal_h5_kv1_d128_ps64`                | gqa\_paged TP=8                          |   🟡   |
| `gqa_paged_decode_h5_kv1_d128_ps1`                         | gqa\_paged TP=8                          |    ✅   |
| `gqa_paged_decode_h5_kv1_d128_ps64`                        | gqa\_paged TP=8                          |   🟡   |
| `gqa_ragged_prefill_causal_h5_kv1_d128`                    | gqa\_ragged TP=8                         |   🟡   |
| MoE experts (top-1, 128 experts, standard routing)         | moe                                      |    —   |
| `trtllm_fp4_block_scale_moe_topk1_e128_h5120_i8192`        | moe (TRT-LLM FP4, Llama4 routing)        |   🟡   |
| `trtllm_fp4_block_scale_routed_moe_topk1_e128_h5120_i8192` | moe (TRT-LLM FP4 routed, Llama4 routing) |   🟡   |
| `trtllm_fp8_per_tensor_scale_moe_topk1_e128_h5120_i8192`   | moe (TRT-LLM FP8)                        |   🟡   |
| `top_k_sampling_from_probs_v202048`                        | sampling                                 |   🟡   |
| `top_k_top_p_sampling_from_probs_v202048`                  | sampling                                 |   🟡   |
| `top_p_sampling_from_probs_v202048`                        | sampling                                 |   🟡   |

**Coverage**: 10 / 13 definitions present. rmsnorm h5120 shared with Qwen3 14B. TRT-LLM FP4 + FP8 MoE kernels added (top-1, 128 experts, Llama4 routing). Same base dimensions as Llama 4 Scout; MoE expert count differs. ps1 GQA workloads collected; ps64 definitions present but workloads pending.

***

## Mistral Small 3.1 24B

**Architecture**: 40 decoder layers, GQA attention (explicit head\_dim=128), dense MLP. Standard serving configuration: **TP=2** (from sgl-cookbook). Shares the same attention configuration as Mistral Nemo 12B (hidden=5120 with explicit head\_dim=128 giving 32 effective query heads).

| Definition                                   | Op Type     | Status |
| -------------------------------------------- | ----------- | :----: |
| `rmsnorm_h5120`                              | rmsnorm     |   🟡   |
| `fused_add_rmsnorm_h5120`                    | rmsnorm     |   🟡   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps1`  | gqa\_paged  |    ✅   |
| `gqa_paged_prefill_causal_h32_kv8_d128_ps64` | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps1`          | gqa\_paged  |    ✅   |
| `gqa_paged_decode_h32_kv8_d128_ps64`         | gqa\_paged  |    ✅   |
| `gqa_ragged_prefill_causal_h32_kv8_d128`     | gqa\_ragged |    ✅   |
| `gemm_n6144_k5120`                           | gemm        |    ❌   |
| `gemm_n5120_k4096`                           | gemm        |    ❌   |
| `gemm_n28672_k5120`                          | gemm        |    ❌   |
| `gemm_n5120_k14336`                          | gemm        |    ❌   |
| `top_k_sampling_from_probs_v131072`          | sampling    |    ❌   |
| `top_k_top_p_sampling_from_probs_v131072`    | sampling    |    ❌   |
| `top_p_sampling_from_probs_v131072`          | sampling    |    ❌   |

**Coverage**: 7 / 14 definitions present. GQA kernels are shared with Mistral Nemo 12B and Llama 3.1 8B; rmsnorm h5120 is now shared with Qwen3 14B. Missing: GEMM defs with k=5120 input dim (Mistral-specific intermediate sizes), sampling v131072.

***

## GLM-4.6

**Architecture**: Dense transformer with Dual Chunk Attention (DCA) — a variant of full attention with rotary embeddings. Served on Together AI and Fireworks; sgl-cookbook shows **TP=8, EP=8** (high-throughput configuration), suggesting a very large MoE variant.

> **Note**: Exact architecture parameters for GLM-4.6 require verification from the HuggingFace config.json (`zai-org/GLM-4.6`). The params below are based on the SGLang `glm4.py` defaults and may not reflect the actual model dimensions.

| Definition                  | Op Type   |       Status       |
| --------------------------- | --------- | :----------------: |
| `rmsnorm_h4096`             | rmsnorm   | ❌ (if hidden=4096) |
| `fused_add_rmsnorm_h4096`   | rmsnorm   |          ❌         |
| GQA or custom DCA attention | attention |          —         |
| MoE FFN (if applicable)     | moe       |          —         |
| Sampling (vocab TBD)        | sampling  |          —         |

**Coverage**: 0 / ? definitions present. Architecture requires research. DCA attention may use standard GQA kernels at the computation level (FlashInfer's paged/ragged wrappers) or require custom handling. Run `/track-models --model-name glm46 --hf-repo-id zai-org/GLM-4.6` to fetch the exact config and update this section.

***

## MiniMax-Text-01

**Architecture**: Hybrid linear + softmax attention with MoE FFN. Uses a 7:1 ratio of Lightning Attention (linear) to standard Softmax Attention layers per 8-layer block, plus sparse MoE (32 experts, top-2 routing). Total parameters: \~456B with \~45.9B activated. 80 decoder layers, 64 attention heads, head\_dim=128, hidden\_size=6144.

**Lightning Attention** is a novel linear attention variant that does not use the standard softmax attention mechanism. It is **not currently supported by FlashInfer** and requires a new op type.

| Definition                                     | Op Type         |           Status          |
| ---------------------------------------------- | --------------- | :-----------------------: |
| `rmsnorm_h6144`                                | rmsnorm         |             ❌             |
| `fused_add_rmsnorm_h6144`                      | rmsnorm         |             ❌             |
| Lightning Attention layers (7/8 of all layers) | lightning\_attn | ❌ (op type not supported) |
| Softmax Attention layers (1/8 of all layers)   | gqa\_paged      |             ❌             |
| MoE experts (top-2, 32 experts)                | moe             |             —             |
| Sampling (vocab TBD)                           | sampling        |             —             |

**Coverage**: 0 / ? definitions present. The primary blocker is Lightning Attention — a linear attention variant not yet in FlashInfer. The softmax attention layers (GQA-style) also require new definitions for this model's specific dimensions. To add support, a new `lightning_attn` op type would first need to be defined.

***

## MiniMax M2

**Architecture**: 62 decoder layers, GQA attention (6:1 ratio, 48 q-heads / 8 kv-heads, head\_dim=128, hidden\_size=3072), MoE FFN with sigmoid routing (256 experts, top-8, FP8 block-scale quantization). Total parameters: \~230B with \~10B activated.

Note: MiniMax M2 is a separate model from MiniMax-Text-01 (which uses Lightning Attention). M2 uses standard GQA attention.

| Definition                                     | Op Type                | Status |
| ---------------------------------------------- | ---------------------- | :----: |
| `rmsnorm_h3072`                                | rmsnorm                |   🟡   |
| `fused_add_rmsnorm_h3072`                      | rmsnorm                |   🟡   |
| `rope_with_cos_sin_cache_neox_style_d128_rd64` | rope                   |   🟡   |
| `gqa_paged_prefill_causal_h6_kv1_d128_ps1`     | gqa\_paged             |   🟡   |
| `gqa_paged_prefill_causal_h6_kv1_d128_ps64`    | gqa\_paged             |   🟡   |
| `gqa_paged_decode_h6_kv1_d128_ps1`             | gqa\_paged             |   🟡   |
| `gqa_paged_decode_h6_kv1_d128_ps64`            | gqa\_paged             |   🟡   |
| `gqa_ragged_prefill_causal_h6_kv1_d128`        | gqa\_ragged            |   🟡   |
| `gemm_n8192_k3072`                             | gemm (fused qkv\_proj) |   🟡   |
| `gemm_n3072_k6144`                             | gemm (o\_proj)         |   🟡   |
| `gemm_n256_k3072`                              | gemm (MoE gate)        |   🟡   |
| MoE gate / topk / experts                      | moe                    |    —   |
| `top_k_sampling_from_probs_v200064`            | sampling               |   🟡   |
| `top_k_top_p_sampling_from_probs_v200064`      | sampling               |   🟡   |
| `top_p_sampling_from_probs_v200064`            | sampling               |   🟡   |

**Coverage**: 14 / 15 definitions present. Workloads not yet collected.

***

## Gemma 3 27B

**Architecture**: 62 decoder layers, GQA attention (2:1 ratio, 32 q-heads / 16 kv-heads, explicit head\_dim=128 decoupled from hidden\_size=5376), dense MLP with GeGLU activation.

Note: `hidden_size=5376` is non-standard; head\_dim is explicitly 128 (not 5376/32=168). This is a multimodal model (vision+text) but the language backbone uses standard transformer attention.

| Definition                                    | Op Type             | Status |
| --------------------------------------------- | ------------------- | :----: |
| `rmsnorm_h5376`                               | rmsnorm             |   🟡   |
| `fused_add_rmsnorm_h5376`                     | rmsnorm             |   🟡   |
| `gqa_paged_prefill_causal_h32_kv16_d128_ps1`  | gqa\_paged          |   🟡   |
| `gqa_paged_prefill_causal_h32_kv16_d128_ps64` | gqa\_paged          |   🟡   |
| `gqa_paged_decode_h32_kv16_d128_ps1`          | gqa\_paged          |   🟡   |
| `gqa_paged_decode_h32_kv16_d128_ps64`         | gqa\_paged          |   🟡   |
| `gqa_ragged_prefill_causal_h32_kv16_d128`     | gqa\_ragged         |    ✅   |
| `gemm_n4096_k5376`                            | gemm (q\_proj)      |   🟡   |
| `gemm_n2048_k5376`                            | gemm (k/v proj)     |   🟡   |
| `gemm_n5376_k4096`                            | gemm (o\_proj)      |   🟡   |
| `gemm_n21504_k5376`                           | gemm (gate/up proj) |   🟡   |
| `gemm_n5376_k21504`                           | gemm (down proj)    |   🟡   |
| `top_k_sampling_from_probs_v262208`           | sampling            |   🟡   |
| `top_k_top_p_sampling_from_probs_v262208`     | sampling            |   🟡   |
| `top_p_sampling_from_probs_v262208`           | sampling            |   🟡   |

**Coverage**: 15 / 15 definitions present. All dimensions are unique to this model: hidden=5376, intermediate=21504, vocab=262208. GQA ratio is 2:1 (vs 4:1 for Llama/Qwen), so kv\_heads=16 (not 8). Workloads not yet collected.

***

## Qwen3 14B

**Architecture**: 40 decoder layers, GQA attention (5:1 ratio, 40 q-heads / 8 kv-heads, head\_dim=128), dense MLP. Standard serving configuration: **TP=2** (from sgl-cookbook), giving 20 q-heads and 4 kv-heads per device.

| Definition                                   | Op Type                   | Status |
| -------------------------------------------- | ------------------------- | :----: |
| `rmsnorm_h5120`                              | rmsnorm                   |   🟡   |
| `fused_add_rmsnorm_h5120`                    | rmsnorm                   |   🟡   |
| `gqa_paged_prefill_causal_h20_kv4_d128_ps1`  | gqa\_paged TP=2           |   🟡   |
| `gqa_paged_prefill_causal_h20_kv4_d128_ps64` | gqa\_paged TP=2           |   🟡   |
| `gqa_paged_decode_h20_kv4_d128_ps1`          | gqa\_paged TP=2           |   🟡   |
| `gqa_paged_decode_h20_kv4_d128_ps64`         | gqa\_paged TP=2           |   🟡   |
| `gqa_ragged_prefill_causal_h20_kv4_d128`     | gqa\_ragged TP=2          |    ✅   |
| `gemm_n7168_k5120`                           | gemm (qkv\_proj combined) |   🟡   |
| `gemm_n5120_k5120`                           | gemm (o\_proj)            |   🟡   |
| `gemm_n34816_k5120`                          | gemm (gate\_up combined)  |   🟡   |
| `gemm_n5120_k17408`                          | gemm (down proj)          |   🟡   |
| `top_k_sampling_from_probs_v151936`          | sampling                  |    ✅   |
| `top_k_top_p_sampling_from_probs_v151936`    | sampling                  |    ✅   |
| `top_p_sampling_from_probs_v151936`          | sampling                  |    ✅   |

**Coverage**: 14 / 14 definitions present. The rmsnorm\_h5120 definition is also shared with Mistral Nemo 12B, Mistral Small 3.1 24B, Phi-4 14B, and Llama 4 Scout/Maverick. Non-sampling workloads not yet collected.

***

## NemotronH 47B

**Architecture**: 52 decoder layers total — hybrid of standard GQA (Transformer) and Mamba2 SSM layers. Uses 20 GQA attention layers and 32 Mamba2 layers in an interleaved pattern. Standard serving configuration: **TP=8** (from sgl-cookbook).

**Mamba2 SSM** (Structured State Space Model) is a linear recurrent architecture that does not use softmax attention. It maintains a fixed-size state matrix updated at each step, analogous to a hidden state in RNNs. Mamba2 is **not currently supported as an op type in FlashInfer-Bench** and requires defining a new `mamba_ssu` (Selective State-space Unit) operation type before this model can be tracked.

| Definition                             | Op Type    |           Status          |
| -------------------------------------- | ---------- | :-----------------------: |
| `rmsnorm_h{hidden}`                    | rmsnorm    |        ❌ (dims TBD)       |
| GQA attention layers (20 layers, TP=8) | gqa\_paged |             ❌             |
| Mamba2 SSM layers (32 layers)          | mamba\_ssu | ❌ (op type not supported) |
| MLP / MoE FFN                          | gemm / moe |             ❌             |
| Sampling                               | sampling   |             ❌             |

**Coverage**: 0 / ? definitions present. The primary blocker is the Mamba2 SSM op type — a selective state-space operation not yet defined in FlashInfer-Bench. This is analogous to MiniMax-Text-01's Lightning Attention blocker. To add support, a new `mamba_ssu` op type schema would first need to be defined. Once that exists, the GQA attention layers could reuse existing definitions if dimensions match.

***