Granite: four scalar multipliers, and what a negative control cannot see

1. Contribution

Claimed. A sovereign (no vendor dense-matmul library) port of IBM Granite-3.1-2B to a ROCm dense inference engine reproduces Hugging Face f16 prefill logits at cosine 0.999997, argmax matching on every measured token position on an AMD RDNA2-class consumer GPU. The architectural feature isolated and validated is a set of four scalar multipliers woven through an otherwise standard Llama-style forward (dense, GQA, RMSNorm, RoPE, tied embeddings): one applied after the token embedding, one on each residual add, one replacing the conventional attention scaling, and one applied to the final logits, all reusing a single in-place f16 scaling path. Negative controls demonstrate that the embedding, residual, and attention multipliers are load-bearing.

NOT claimed. This is a correctness-validation result, not a novel algorithm and not a generation-quality, throughput, or benchmark-scale evaluation. Parity of logits is not parity of generated text quality. The note makes no probability-calibration claim; indeed it documents that the logits multiplier is invisible to the parity metrics used (see Ablations). Results are bounded to Granite-3.1-2B at f16 on the tested RDNA2 GPU class and do not extend to larger Granite variants, the MoE Granite line, other quantizations, or other families. The altitude is honest: an engineering port verified against an oracle on a small correctness smoke test.

2. Methods

3. Results

The forward pass carries four scalar multipliers applied, respectively, after the token embedding, on each residual add, in place of the conventional attention scaling, and on the final logits before the LM head. The specific constants are part of the proprietary port and are abstracted out here; their roles, not their values, are what the ablations characterize.

4. Ablations / Negative controls

The passing logits-multiplier control is the methodological core of this note. A global scalar applied to the final logits scales every logit equally; it changes neither the direction of the logit vector (cosine) nor the position of the maximum (argmax). The multiplier is real and affects absolute logit magnitudes and calibrated probabilities, but the parity metrics literally cannot see it. A passing negative control does not always prove a component irrelevant — it can mean the metric is invariant to it.

5. Threats to validity / Limitations

• Sample size. A small fixed set of token positions on an unrecorded number of prompts is a correctness smoke test, not a benchmark-scale evaluation. The exact prompt(s) and prompt count are not recorded in the source notes.
• Parity != quality. Logit cosine/argmax parity says nothing about generation quality, coherence over long contexts, or downstream task accuracy.
• Metric blind spot, named explicitly. The chosen metrics (cosine, argmax) are invariant to a global logit scalar, so the passing logits-multiplier control cannot verify that multiplier. This is a property of the evaluation, not evidence the multiplier is unnecessary.
• Precision ceiling. f16 is the engine's highest-precision path; there is no f32 reference forward inside the engine, so residual sub-f16-ULP disagreements are irreducible here. cosine 0.999997 (not 1.0) is consistent with f16 rounding.
• Single-run / no variance. Parity is reported as point values with no repetition or variance characterization.
• Scope / quantization bounds. Validated only for Granite-3.1-2B at f16 on the tested RDNA2 GPU class. No claim for larger Granite, MoE Granite, other quantizations, or other hardware.
• Gate threshold not recorded. The exact numeric pass gate for cosine is not recorded in the source notes; "PASS/FAIL" outcomes are carried from the source's reported verdicts.
• evidenceLevel. Retained as "experimental"; the limitations above (small sample, single-run) are the honest qualifiers and are not silently upgraded.

6. Reproducibility / Provenance

Procedure to reproduce in spirit: load Granite-3.1-2B at f16 in both the engine and transformers with identical token ids, dump per-position logits, compute cosine + argmax over the measured positions, then re-run with each of the four multipliers disabled in turn to reproduce the three load-bearing failures and the honest pass of the logits-multiplier control.