Gemma4-26B-A4B on Apple Silicon: a drop-in isomorphism falsified (NO-GO), with a conditional new-port ceiling

1. Contribution

Claimed. We falsify the hypothesis that Gemma4-26B-A4B is a drop-in isomorphism of an existing, mature Qwen3-30B-A3B MoE inference substrate (the "drop into the existing MoE path and inherit its single-stream throughput" framing). The hypothesis is rejected against the ground-truth bf16 tensor layout on four load-bearing axes: expert weight layout, attention geometry, activation function, and a novel scalar stack. We then state a conditional outcome — a new, purpose-built Gemma4 port is structurally adjacent to the 30B-class regime and, if built, would carry an estimated single-stream low-bit throughput ceiling expressed as a range on Apple Silicon — together with a high-level effort judgment (non-trivial; not a free 30B-class win).

NOT claimed. No model was loaded, nothing was run, nothing was quantized (consistent with a crash-safe policy against resident out-of-memory on a memory-constrained Apple Silicon machine). The throughput range is an estimate/prediction, not a measurement. We do not claim a working port, a parity result, or any on-device throughput. We make no claim about any other Gemma variant.

2. Methods

This is a read-only static analysis. Architecture and shape facts were read directly from the published bf16 weights (config, safetensors index, and tensor headers) and from internal engine source; no execution occurred.

3. Results

The deep-dive verdict is NO-GO for the drop-in framing: Gemma4-26B-A4B is not structurally isomorphic to Qwen3-30B-A3B. What matches and what breaks:

Matches (router arithmetic). The MoE routing configuration — top-k selection over a large expert pool with softmax-top-k routing — aligns. The existing grouped-expert routing path is parameterized by gate dimensions, group size, and top-k, so the difference in routed-expert intermediate width is a pure dimension change, with no code change required.

Break 1 — expert weight layout. Gemma4 stores experts as stacked fused multi-dimensional bf16 tensors per layer (gate and up projections fused into one tensor, plus a separate down-projection tensor, indexed by expert), whereas the existing resave/resident paths assume many per-expert two-dimensional tensors per layer. A new un-stacking/slicing and fused gate/up split loader is required; the break is at load/quantize time, not at the GEMM.

Break 2 — shared dense MLP plus routed MoE hybrid. Every layer carries BOTH an always-on dense SwiGLU MLP block AND the routed MoE experts, at different intermediate widths. The existing MoE path is pure-routed (residual plus the weighted sum of selected experts) with no parallel always-on dense branch; new forward wiring is required.

Break 3 — activation. The existing glue and fused decode kernels are SwiGLU throughout; Gemma4 uses GeGLU (tanh-approximate GELU gating). A GeGLU variant of the relevant kernels plus CPU-side glue is needed, forking the single-encoder hot path.

Break 4 — heterogeneous attention plus novel scalars. Gemma4 uses per-layer-type dual head dimensions (a smaller head dim on sliding-window layers, a larger one on full-attention layers), differing KV-head counts per type, key/value sharing on full layers (no separate value projection), dual RoPE base frequencies per type, partial rotary application on full layers, and an alternating sliding-to-full layer schedule with a bounded sliding window. It also introduces novel scalar parameters (a per-layer scalar and router-side scale vectors) with no analogue in the Qwen3 path. The existing decode path bakes in a single uniform attention geometry; modeling two alternating geometries is the dominant cost.

Prediction (estimate, not measured). After a correct port, single-stream low-bit decode on Apple Silicon is predicted in the high-teens to mid-twenties tokens per second, with a first correct-but-unfused cut landing lower and a per-layer-type fused variant landing higher. The existing 30B-class anchor is stated as not reachable without solving the extra per-layer dense-MLP GEMM, the small-matrix inefficiency of the narrow routed experts at batch size one, and the dual-geometry attention inside one fused encoder.

External anchors (proxies, directional only). Public third-party numbers for the dense Gemma-3-27B model on high-end Apple Silicon span roughly the high-twenties to low-thirties tokens per second depending on the runtime; projected onto the relevant target hardware this points to a realistically lower band for a dense 27B. Low-trust vendor-blog Gemma4 claims likely understate the A4B MoE. The internal anchor is the engine's own Qwen3-30B-A3B low-bit single-stream throughput.

4. Ablations / negative controls

This is itself a negative result (a falsified isomorphism); there are no executed ablations because no model was run. The falsification is by ground-truth tensor shapes: full-attention layers show query projections consistent with the larger head dim and key projections with a reduced KV-head count and no separate value projection, while sliding-window layers show projections consistent with the smaller head dim and a larger KV-head count — proven by shapes, not by inference. RoPE on full layers resolves to a partial rotary applied to the low portion of the head dimension with no positional encoding on the remainder.

5. Threats to validity / limitations

• No execution. Nothing was loaded, run, quantized, or measured. The throughput band is an engineering estimate, not data; treat it as a fence, not a target.
• Open residual/scalar order. The exact placement and order of the novel per-layer and router-side scalars, and whether the dense-MLP and routed-expert outputs are summed pre- or post-norm (the feedforward block carries multiple norms), are not yet pinned against the HF Gemma4 reference. These are unresolved parity risks at write time.
• Per-attention-scale assumption. A query pre-attention scalar is not present in the config and is assumed to default to the inverse square root of the head dimension, but this must be verified per geometry since the head dim differs between sliding and full layers.
• External benchmarks are proxies. Gemma-3-27B is dense, not MoE; the comparison hardware differs from the target; vendor-blog numbers are low-trust. They bound loosely, not precisely.
• Effort estimate is a judgment, not a costed plan.
• Stale handler. The existing Gemma handling code is metadata-only and its inline comment contradicts the real config (it assumes a non-MoE 1:1 alternating layout, whereas the real config is MoE with a different sliding-to-full ratio); it is a scaffold to rewrite, not a working base.

6. Reproducibility

• Sources (read-only). The published Google bf16 release of the model: its config, safetensors index, and safetensors tensor headers.
• Engine. An internal Apple Silicon text-inference engine that provides a low-bit resident MoE forward and store path, a low-bit resave path for compressed tensors, a metadata-only Gemma handling scaffold, a SwiGLU activation kernel, and fused single-encoder decode kernels. The analysis compares the published Gemma4 tensor layout against these existing components at a structural level only.
• To reproduce the analysis. Read the cited config, index, and tensor headers and compare the expert and attention tensor shapes against the Qwen3-30B per-expert two-dimensional assumptions. No build or run is needed (and is discouraged on a memory-constrained Apple Silicon machine).