Loading image/video weights directly from HF safetensors (portability fix)

Contribution

Claimed. This is an engineering / portability fix that removes a hard blocker: the CUDA image and video generation pipelines previously depended on prebaked, box-specific weight layouts and were therefore unrunnable on a fresh consumer-class (sm_89) CUDA box. After this change, (a) the video DiT and VAE load directly from public Hugging Face safetensors because the keys are byte-identical to the prebaked layout, gated behind a single load-mode flag, and (b) the image pipeline rebuilds its full weight root from public HF SDXL weights via a portable preparation step on any box. Both paths were validated on a consumer-class sm_89 GPU against the prior pipeline outputs: video frames cosine 1.0, image 1024px cosine 0.99999995, with a self-contained runtime dependency graph (no vendor BLAS library linked).

NOT claimed. This is a weight-loading / portability correctness result, not a novel algorithm and not a performance or quality result. It does not establish any throughput or token-rate number, does not measure or assert generation quality beyond the numeric parity, and does not establish portability to non-CUDA backends. The parity figures are a correctness smoke test (one image resolution and a frames-level video check), not a benchmark-scale evaluation. Altitude: low — this is plumbing that makes already-validated pipelines reproducible from public weights.

Methods

Results

Ablations / Negative controls

This is the single load-bearing observation the notes record: the image VAE must use the fp32 shard, not fp16.

Threats to validity / Limitations

• Sample size is a smoke test, not a benchmark. The image result is a single resolution (1024px) and the video result is a frames-level cosine; the source records no multi-prompt, multi-seed, or multi-resolution sweep. Treat both as correctness spot-checks.
• Single run, no variance. Each cosine is a point measurement; no repeated runs, confidence interval, or seed variance is recorded.
• Parity is not quality. Cosine 1.0 / 0.99999995 against the prior pipeline shows the portable load reproduces the prior numeric output; it says nothing about perceptual generation quality, which was not assessed here.
• Cosine 0.99999995 is not bit-exact. The image path is near-identical but not byte-identical (unlike the byte-identical video keys); the residual is unexplained in the source.
• Scope bound to CUDA sm_89. Verified only on a consumer-class sm_89 target; non-CUDA backends are out of scope and unproven.
• fp32 image-VAE constraint. The fp16 image-VAE shard is known to degrade decode; the portability claim holds only with the fp32 shard.
• Stack versions absent. Exact CUDA / diffusers / HF-weight-revision pins are not recorded in the source notes, weakening exact reproducibility.
• Evidence level. Given the single-resolution / single-run nature, readers should read the result as "the portable path was re-run and matched the prior pipeline," not as an independent multi-environment reproduction.

Reproducibility / Provenance

• Video: enable the HF-direct load mode to load the DiT and VAE directly from public HF safetensors (keys byte-identical to the prebaked layout).
• Image: run the portable preparation step to rebuild the full weight root from public HF SDXL; the VAE step must use the fp32 shard.
• Validate first: confirm against the prior prebaked-layout pipeline output (the oracle here) before trusting a fresh-box run.
• Build: compiled for the sm_89 target with standard optimization flags.
• Self-containment: the runtime dependency graph links only the core CUDA runtime, with no vendor BLAS / DNN library.
• Exact run commands and version pins: not recorded in source notes.