Bloom in a sovereign engine: ALiBi as the third positional paradigm
Ran Tao (Octoryn Research)
摘要
Sovereign (no vendor BLAS) port of Bloom-560m to an in-house ROCm dense inference engine on an RDNA2-class GPU reproduces a Hugging Face transformers f16 prefill-logit reference at min cosine 1.0 and argmax 6/6 over 6 token positions. It adds ALiBi (per-head linear pre-softmax bias) and a pre-block embedding layernorm, completing coverage of all three positional paradigms: RoPE, learned-absolute, and ALiBi. A negative control surfaced a latent NaN bug in the generic rotary path; guarding it off restored parity. Correctness smoke test only: no quality, throughput, or memory measured; f16 only.
本 Research Object 以其原始语言(英文)发表。
Contribution
Claimed. A sovereign (no vendor BLAS) port of Bloom-560m to an in-house ROCm dense inference engine reproduces a Hugging Face transformers f16 prefill-logit reference at minimum cosine similarity 1.0, argmax 6/6 on a single RDNA2-class GPU. The engineering substance is threefold: (1) ALiBi (Attention with Linear Biases) implemented as a per-head geometric-series slope applied as a linear pre-softmax bias, following the published HF formulation, with no rotary embedding and no learned positional embeddings; (2) a pre-block embedding layernorm that normalizes embeddings before the first transformer block (a first for this engine); and (3) de-interleaving of the fused QKV tensor. With this port the engine spans all three major positional paradigms it targets — RoPE variants, learned-absolute, and ALiBi. A load-bearing bug was identified and fixed: the engine's generic rotary path mis-gated for this architecture, where a degenerate rotary configuration produced NaN; conditioning the rotary path off for ALiBi models restored parity.
NOT claimed. This is a correctness-validation (logit-parity) result, not a novel algorithm — ALiBi is an existing published scheme and this is a faithful reimplementation. The note does not claim anything about generation quality, throughput, latency, or memory footprint; none were measured. It does not claim long-context ALiBi extrapolation behaviour, larger Bloom variants, quantized accuracy, or any other model family. Parity of logits is not a proxy for downstream text quality. The altitude here is honest: an engineering correctness check at smoke-test scale, not a benchmark-scale evaluation.
Methods
| Field | Value |
|---|---|
| Model | Bloom-560m |
| Hardware | RDNA2-class GPU (16 GB) |
| Stack | In-house ROCm dense inference engine; exact toolchain version not recorded in source notes |
| Oracle / baseline | Hugging Face transformers (recent version), f16 |
| Procedure | Direct logit-parity: same weights and identical input ids fed to engine and oracle; compare full logit vectors |
| Precision | f16 (quantized precision not validated in this note) |
| Metric + gate | Per-position cosine similarity of logit vectors + argmax match; reported as min cosine and argmax hit-rate |
| Sovereignty check | Dynamic-linkage audit: only the HIP runtime and system libraries linked — no rocBLAS/hipBLAS/MIOpen/MKL |
| Stimulus / inputs | Prompt(s) over 6 token positions (argmax 6/6); exact prompt text not recorded in source notes |
Results
| Measurement | Value |
|---|---|
f16 min cosine vs transformers | 1.0 |
| f16 argmax match | 6/6 |
| Sovereignty (linkage audit) | No vendor BLAS linked |
Ablations / Negative controls
| Control | Observation |
|---|---|
| Generic rotary path left enabled for an ALiBi architecture | The rotary computation, under a degenerate configuration for this model, produced NaN — parity broke. Conditioning the rotary path off and routing through the ALiBi bias restored correctness. This is a discovered bug, not a designed sweep, but it confirms the ALiBi path is load-bearing. |
Threats to validity / Limitations
- Sample size. Parity covers a single model at 6 token positions (argmax 6/6). This is a correctness smoke test, not a benchmark-scale evaluation. No held-out prompt set, no statistical sampling.
- Single run, no variance. Numbers are reported from individual runs; no repetition or variance is recorded.
- Parity != quality. Logit cosine/argmax agreement with the oracle does not establish generation quality, coherence, or downstream task performance — none were measured.
- No performance data. Throughput, latency, and memory footprint are entirely absent; no claim should be read into them.
- Precision ceiling. Only f16 was validated. Quantized accuracy is untested. Cosine 1.0 is at f16 resolution, not f32.
- Scope bounds. Only Bloom-560m on an RDNA2-class GPU. Larger Bloom variants, other ALiBi models, long-context extrapolation, and other linear-bias schemes are out of scope.
- Oracle/version provenance. Exact
transformersversion, toolchain version, and prompt text are not recorded in the source notes — a reproducibility gap to close. - Evidence level. Marked
experimental; given the smoke-test sample size this should be read as a correctness validation, not a broad empirical study.
Reproducibility / Provenance
| Item | Value |
|---|---|
| Build approach | Compiled directly against the HIP runtime for the target RDNA2 architecture; no vendor BLAS linked |
| Oracle validation | Confirm the Hugging Face reference actually loads real Bloom weights before trusting parity |
| Toolchain version | Not recorded in source notes |
| Family lineage | An ALiBi / linear-bias positional model is a first for this engine, extending its positional-paradigm coverage |
声明边界
作者对范围的明确界定——本工作证明了什么、未证明什么——沿用自 Octoryn Research 的发表模型。
证明
- A sovereign ALiBi implementation for Bloom-560m reproduces a Hugging Face f16 prefill-logit reference at minimum cosine 1.0 and argmax 6/6 over 6 token positions.
- The in-house inference engine now covers all three major positional paradigms: RoPE, learned-absolute, and ALiBi.
- A latent NaN-producing bug in the generic rotary path was identified and fixed, confirming the ALiBi path is load-bearing.
未证明
- Larger Bloom variants or long-context ALiBi extrapolation behaviour.
- Generation quality, throughput, latency, or memory footprint — none were measured.
- Other linear-bias positional schemes beyond ALiBi.
- Quantized accuracy or f32-level precision; only f16 was validated.
- Benchmark-scale correctness: 6 token positions is a smoke test, not a held-out evaluation.
适用于
- f16 prefill-logit parity for Bloom-560m on an RDNA2-class GPU versus a Hugging Face transformers reference fed identical input ids.
不适用于
- Quantized, generation, throughput, latency, or memory claims, or any other model family.
- Long-context extrapolation or larger Bloom variants.
- Any claim requiring more than a single-model, 6-position smoke test.
作者
- Ran Tao — 调查研究, 写作
引用本文
引用格式
Tao, R., Octoryn Research. (2026). Bloom in a sovereign engine: ALiBi as the third positional paradigm (TR-2026-0010). Octopus Research Institute.
BibTeX
@techreport{oritr20260010,
title = {Bloom in a sovereign engine: ALiBi as the third positional paradigm},
author = {Tao, Ran and {Octoryn Research}},
institution = {Octopus Research Institute},
year = {2026},
note = {Permanent ID TR-2026-0010. Not peer reviewed.}
}披露
- 资助
- 硬件与基础设施由 Octoryn / Octopus Core Pty Ltd 提供。
- 利益冲突
- Octoryn 提供商业推理与治理工具;相关发现独立报告。