状态: 活跃研究
主权与本地 AI
本地与端侧推理、模型优化、边缘部署、隐私感知推理,以及多供应商可移植性。
范围
我们评估有用推理在本地或边缘可运行的程度、模型如何针对受限硬件优化,以及工作负载如何在供应商间保持可移植——降低对任何单一外部服务的依赖。
本方向的研究项目
- 状态: 实验性Apple Silicon 推理运行时
面向 Apple Silicon 及其他加速器上本地推理的内核设计、内存搬运与带宽上限研究——整合自早期 Octoryn Research 的工作。
- 状态: 活跃研究本地与主权推理
端侧与边缘推理、面向受限硬件的模型优化,以及降低单一供应商依赖的多供应商可移植性。
开放问题
- RQ-001假设Apple GPU 能否支持持久化内核的全局同步?
发表
- Apple-Silicon-friendly LLM architecture: substrate laws reverse-engineered from a model bake-off
- Gemma4-26B-A4B on Apple Silicon: a drop-in isomorphism falsified (NO-GO), with a conditional new-port ceiling
- A falsify-first root cause for a concurrent 4-bit decode crash: batch-composition KV-pool wipe trips a re-seed precondition
- DiffusionGemma-26B compatibility assessment: NEEDS_ARCH_SUPPORT on Apple Silicon (pre-port, external llama.cpp baseline)
- Diffusion-LLM Decode Quality: An Honest Regression, Negative Evidence, and a Canvas-Geometry Reattribution
- Federation continuity primitives: an event-sourced session log with deterministic replay, cross-machine replication, and self-healing recovery
- Sovereign gpt-oss-20B (MXFP4) on Apple Silicon: a fine-grained-MoE small-matrix occupancy ceiling
- Per-stream KV-slot isolation for parallel low-bit decode on Apple Silicon: crash fix with byte-identical solo parity and no-regression
- Batched resident MoE experts on Apple Silicon: a flat parallel expert pass, ~1.6x decode, parity bit-identical (single-run)
- Batched-concurrent 4-bit serving on Apple Silicon: weight-stream amortization with an honest sublinear ceiling
- Amortizing the per-token host-sync setup tax in fused all-layer q4 decode on Apple M1 Ultra (1.57x), then occupancy-bound
- A sovereign ARM-NEON CPU inference backend (Raspberry Pi 4B): Q8 and multicore are complementary, not additive
- Independent GGUF Forward Pass and Bit-Identical Cross-Backend Consistency for a 26B Text-Diffusion Model (forward only, not full decode)
- Multi-card inference topology: request-boundary aggregation on a latency-bound LAN
- Single-stream int4 decode of a 30B MoE on Apple Silicon: steady-state throughput with byte-identical, vendor-BLAS-free parity
- Cross-chip determinism: 96.875% byte-identical decode across two chips, live
- Resident packed-int4 decode: ~5x over host-streaming on a 24GB-class GPU
- FP8 grouped-expert prefill with on-chip int4-to-e4m3 dequant
- A DiT self-attention flash kernel on RDNA2: 9.4x op, 4.67x end-to-end
- An honest red result: EOS fixed, but forward degradation persisted
- ASR phase 1: a sovereign Whisper-tiny path landed
- A sovereign dense engine reproduces HF logits across 16 transformer families
- Cross-box tensor-parallel decode is latency-bound on commodity 2.5GbE
- Porting Mamba (selective-scan SSM) into a sovereign sequence-mixing engine
- RWKV in a self-built engine: a second non-attention paradigm (WKV/RNN)
- Bloom in a sovereign engine: ALiBi as the third positional paradigm
- Falcon in a sovereign engine: the first parallel attention+MLP block
- Fully-sovereign text-to-video on Apple Silicon (Wan2.1), parity to diffusers
- OLMoE: the first sparse-MoE in a dense sovereign engine (and an eps trap)
- Granite: four scalar multipliers, and what a negative control cannot see
- OLMo-2: reordered (post) normalization and a loader-clobber root-cause
- StableLM-2: partial RoPE and a first LayerNorm-with-bias path in a dense ROCm engine
- Phi-3-mini: fused-tensor direct loading in a sovereign inference engine
- The direct HF-logits parity harness (and the compressed-tensors version trap)
- Continuous batching with paged KV: a dependency-free CUDA throughput path
- A multi-week debugging retrospective on an Apple Silicon 4-bit inference engine: command-buffer folding, an encoder leak, and cold-bank reservation
- True standalone text-to-video on gfx1030, sovereign (no vendor BLAS)
- True CUDA text-to-video with live UMT5 encode, dependency-minimal (no cuBLAS)
- 1024px image generation end-to-end on a consumer RDNA2 GPU, and a NULL-deref segfault root-caused
- Wan 3D-VAE decode parity on CPU (Swift): pinning the layout contract
- Loading image/video weights directly from HF safetensors (portability fix)
- A sovereign DP router with mid-stream byte-identical failover
- 论本地推理的实际边界:一则研究札记