研究可问责、可及且具备主权的人工智能。

Five structurally distinct LLM families were each hand-ported into one custom Apple-GPU decode engine and profiled until each broke under single-stream 4-bit decode on a single M1 Ultra; only one satisfied all five rules. The result is a spec of five rules for fast single-stream decode on unified-memory Apple Silicon: low active params per token, vector-load-aligned quantization, experts big enough to fill the GPU at batch one, uniform attention, and a fusion-friendly non-hybrid layout. Meta-finding: architecture identity does not predict runtime cost; the rules are necessary, not sufficient.

发布于 2026-06-15 · 版本 3

A read-only static deep-dive falsifying that Gemma4-26B-A4B is structurally isomorphic to a mature Qwen3-30B-A3B MoE single-stream inference path on Apple Silicon, able to drop in and inherit its throughput. Ground-truth bf16 shapes break it on four axes: a stacked fused-expert layout, a dense-MLP plus routed-MoE hybrid, GeGLU not SwiGLU gating, and a heterogeneous sliding/full attention geometry with novel scalars. Verdict: NO-GO for a config-swap; only the MoE router carries over. A purpose-built port has a conditional throughput ceiling as a range. Nothing was run; figures are estimates.

发布于 2026-06-15 · 版本 3

Falsify-first root cause for a fatal trap that killed a single-resident 4-bit ~30B sparse-MoE decode daemon under concurrent decode on one Apple Silicon machine. An orchestration-free reproducer isolates the trigger: concurrent chat and interleaved chat-plus-embed crash, while embeddings-only and sequential traffic survive, refuting an "embeddings poison decode" guess. The mechanism is a batch-composition-change wipe of resident KV pools tripping a re-seed precondition no steady-state decode meets. Serializing GPU decode prevents it but gives no speedup; the per-stream fix is unimplemented.

发布于 2026-06-15 · 版本 3