Root Mean Square Normalization with hidden_size=128. Captured from Qwen3-30B-A3B. Epsilon is fixed at 1e-6.
status:verified
model:qwen3-30b-a3b
Axes
batch_size
varhidden_size
128Signature
Inputs
| Name | Type | Shape |
|---|---|---|
hidden_states | bfloat16 | [batch_size, hidden_size] |
weight | bfloat16 | [hidden_size] |
Outputs
| Name | Type | Shape |
|---|---|---|
output | bfloat16 | [batch_size, hidden_size] |
Reference Implementation
Loading editor...
Loading solutions…