Falcon in a sovereign engine: the first parallel attention+MLP block

1. Contribution

Claimed. This is an engineering/correctness-validation result, not a novel algorithm. We port Falcon-7B-Instruct into an independent dense inference engine on a consumer GPU, deliberately built without any vendor BLAS/DNN libraries, and show that its f16 prefill logits match a public reference implementation. The port is the engine's first parallel-block architecture: a single input layernorm feeds BOTH the attention and MLP sublayers, which are then summed into one residual update, in contrast to the strictly sequential block ordering (norm, then attention with residual, then a second norm, then MLP with residual) of the prior sequential-block families. It also adds multi-query attention (a single shared key/value head) and an exact-erf GELU activation. Measured parity is cosine 1.0 with full argmax agreement (22/22) over two prompts, and a force-sequential negative control collapses to cosine 0.977 with zero argmax matches (0/12), demonstrating that the parallel structure is load-bearing for correctness.

NOT claimed. This note does NOT claim anything about generation quality, coherence at length, long-context behavior, throughput, or latency. It does NOT claim correctness under quantization — the port is f16-only. It does NOT claim coverage of other parallel-block variants beyond Falcon-7B-Instruct. The parity evaluation is a correctness smoke test (two prompts), not a benchmark-scale evaluation. "Coherent output" is an informal observation, not a measured quality metric.

2. Methods

3. Results

4. Ablations / Negative controls

5. Threats to Validity / Limitations

• Sample size. Parity is two prompts spanning 22 token positions — a correctness smoke test, not a benchmark-scale evaluation. The negative control reports 12 positions.
• Single-run / no variance. Results are reported as single point values; no repeated runs, variance, or confidence intervals are recorded.
• Parity is not quality. Logit cosine and argmax match against the oracle bound numerical fidelity of the forward pass; they say nothing about generation quality, coherence at length, instruction-following, or long-context behavior. "Coherent output" is an informal observation.
• Precision ceiling. The engine has no f32 forward path; f16 is its highest precision. Sub-f16-ULP ties (as documented for sibling families) could in principle perturb argmax at low margins; none were noted here, but the ceiling is real.
• Quantization out of scope. The port is f16-only; no quantized correctness is established.
• Oracle version not pinned in this note. The exact reference-implementation version used for THIS run is not recorded; sibling notes pin a recent version. A skeptical reader should validate the oracle version before trusting the parity number, and confirm the oracle itself loads real weights.
• Scope. Claims apply to Falcon-7B-Instruct on the tested consumer GPU only; no other parallel-block variant is tested.
• Evidence level. Retained as "experimental." This is a correctness-validation result with a small stimulus set, not an algorithmic contribution.

6. Reproducibility / Provenance (high-level)

Architecture summary (high-level)

• Parallel block: a single input layernorm output is reused for both the attention and MLP sublayers, whose outputs are summed into one residual update; the intermediate residual of the sequential design is skipped.
• Multi-query attention: a single shared key/value head, with the fused query/key/value projection laid out so no de-interleaving step is required.
• Activation: an exact-erf GELU rather than the tanh approximation.