StableLM-2: partial RoPE and a first LayerNorm-with-bias path in a dense ROCm engine

1. Contribution

Claimed. A from-scratch port of StableLM-2-1.6B (Stability AI) onto a dense ROCm inference engine that does not depend on vendor BLAS/DNN libraries reproduces a Hugging Face transformers f16 prefill-logits reference at cosine 0.999997, argmax 12/12 on the tested prompts. The port required two engine firsts: partial RoPE (only a rotary span of 16 of the 64 head dimensions is rotated, NeoX split-half over that span; the remaining dimensions pass through unrotated) and the engine's first non-RMS normalization path — LayerNorm-with-bias (a new fp16 layer-norm kernel), plus QKV bias. A full-RoPE negative control (rotating the entire head) collapses to cosine -0.62, argmax 3/12, establishing that the partial-RoPE rotary span is load-bearing.

NOT claimed. This is a correctness-validation (numerical-parity) result, not a novel algorithm and not a quality or performance result. It does not claim generation quality, throughput, or any benchmark-scale evaluation. It does not claim that q8 (measured cosine 0.998910, argmax 11/12) matches f16. It does not claim coverage of larger StableLM variants. The parity is a small-sample smoke test against one reference, at f16 (the engine's highest precision), not a statistically powered benchmark.

2. Methods

3. Results

4. Ablations / Negative controls

5. Threats to validity / Limitations

• Sample size. Parity is measured over a small set of prompts totalling 12 token positions — a correctness smoke test, not a benchmark-scale evaluation. The exact prompt strings and count are not recorded in the source notes.
• Single run / no variance. No repeated-run statistics or variance bounds are reported for any number; all figures are single-run.
• Parity != quality. Logit cosine and argmax match the reference's numerics; they say nothing about generation quality, coherence, or downstream task behavior.
• Precision ceiling. f16 is the engine's highest precision (no f32 forward), so the residual gap from 1.0 is irreducible here; q8 (0.998910, 11/12) is honestly below f16 and is not claimed to match it.
• Quantization scope. q8 is reported but not validated to f16-equivalence; quantized generation/throughput is out of scope.
• Baseline weakness. The transformers reference version is not recorded in source notes; reproduction requires pinning and validating the reference independently before trusting the gate.
• Scope. Result applies only to StableLM-2-1.6B f16 prefill-logits parity on the target RDNA2 GPU; it does not transfer to other families, larger StableLM variants, generation, or throughput.
• Evidence level. Marked experimental; this reflects a numerical-parity validation, not a controlled benchmark — readers should not read "experimental" as benchmark-scale.

6. Reproducibility / Provenance

• Reference. transformers (version not recorded in source notes) with identical token ids; validate the reference first — confirm it loads real weights and produces coherent logits before comparing.
• Negative control. Re-run with full-RoPE (rotate the whole head) to reproduce the -0.62 / 3/12 collapse, confirming the partial-RoPE span is load-bearing.
• Library independence. Linked libraries are the HIP runtime plus system only (no vendor BLAS/DNN libraries).
• Scope note. The shared Qwen2-style attention handler was narrowed to exclude the StableLM/LayerNorm path; no-regress was re-checked on Qwen2.5 / OLMo-2 / Phi-3 (12/12 each).
• Build / run. Compiled with the standard HIP toolchain targeting the RDNA2 architecture and linked against the HIP runtime.