Cross-Rendering Semantic Binding in an Engineered Small-Cell Schema (Mechanism Findings)

Context

This phase of the DAWM program asks a single falsifiable question: can two independent renderings of the same situation — a text rendering and a visual rendering — be encoded into the same inspectable semantic graph state (entities + typed relations + endpoint bindings)? This is the small-cell, engineered-schema form of semantic binding. If relation endpoints cannot be grounded across renderings, then rendering-to-state decoupling has not been validated even in a hand-built schema.

This report consolidates five mechanism findings: a small-cell close-out (a decisive negative result), a post-hoc diagnostic, and a relation-binding closure summary.

Finding 1 — The decoded-structure floor is real and schema-caused

Across all checkpoints in both small-cell runs, decoded structural equality (a structural-isomorphism test) and layout invariance sat at the floor — not small-and-noisy, but floored. This persisted even when the dense convergence loss was low (encoders agreeing at the logit level) and when reasoner trajectory similarity reached a moderate level (the two encoders producing nearly identical answer distributions).

Finding statement: the encoded-state schema permits dense-space convergence without argmax-space convergence. Convergence loss aligns the logit distributions over free entity-ID and relation slots, but nothing forces the two encoders to assign the same identity to the same underlying entity. If the text encoder maps an entity to one slot and the visual encoder maps it to a different slot, the decoded sets are disjoint and isomorphism is zero even under perfect dense alignment. The criterion "discrete decoded structures match" is therefore not enforceable by loss surgery alone under a free-slot schema.

Patch evidence: a cross-rendering reasoner patch lifted visual reasoning accuracy above the relaxed acceptance bar and produced a mid-run trajectory-similarity peak, but a late-training regression showed the patch delays rather than cures the drift. A multi-layout patch produced no detectable layout-invariance effect (within noise of zero).

Finding 2 — Entity-identity prior collapse is a real but secondary failure mode

An initially-diagnosed "argmax floor from free entity-ID slots" was corrected: canonical ground-truth supervision was already shared across encoders. The true mode is entity-identity prior collapse under multi-objective training — the per-slot entity-identity loss rose above the uniform-baseline level, with encoders predicting the marginal identity distribution rather than per-situation identities. Some high-frequency identities retained per-identity accuracy in the mid-to-high range, so signal existed but was washed out by reconstruction / convergence / reasoning gradient interaction.

Finding statement: entity identity is learnable in the small-vocabulary regime but collapses toward the marginal under a larger vocabulary and heavier reconstruction weight. Entity-identity collapse is a wall, not the wall.

Finding 3 — The dominant wall is ungrounded edge-endpoint binding (the headline)

A post-hoc oracle decomposition isolated the true bottleneck. In the small-vocabulary regime (where entity identities are learnable), holding components fixed by oracle reveals the structural break: with everything predicted, isomorphism stays at the floor and edge agreement is low; supplying oracle entities while leaving edges predicted barely moves isomorphism; supplying oracle edges while leaving entities predicted lifts isomorphism and edge agreement substantially; and the all-oracle control recovers perfect agreement.

Component accuracy showed the break clearly: relation labels were strong and existence-gates were strong, but edge endpoints were weak (source endpoint accuracy near chance-plus, destination endpoint accuracy markedly lower, on ground-truth edges). Slot-space exact-edge match rate was near zero.

Finding statement: even with perfect (oracle) entity identities, predicted edge sets almost never match. The failure is not entity-identity prior collapse alone — it is ungrounded edge-endpoint binding under a free edge-slot graph schema. A free-edge-slot list asks one path to solve three problems at once (entity identity, relation label, and assignment of each relation instance to the same ordered endpoint-slot pair across renderings); the third is the brittle one. Text and visual agree on relation labels far more often than on endpoint structure, and threshold sweeps do not rescue isomorphism.

Finding 4 — Pair-indexed edges remove the permutation problem

The diagnostic motivated a schema reframe: replace the free edge-slot list with a pair-indexed edge field — an adjacency representation indexed over (entity-slot, entity-slot, relation-vocabulary) — so endpoints become the coordinate system of the edge representation rather than ungrounded categorical outputs.

A pair-grid smoke test confirmed the direction: text-vs-visual edge agreement lifted substantially over the old free-slot baseline; the structural-isomorphism test rose off the floor to a nonzero level; and relation-label accuracy on ground-truth edges stayed high.

Finding statement: pair-indexed (coordinate-addressed) edges remove the free-slot permutation problem and recover cross-rendering edge agreement and nonzero isomorphism in the regime where entity identities are already learnable. The wall moves rather than vanishes: the residual problem becomes the sparse-adjacency / edge-existence field — over-prediction of edges (high recall, low precision) rather than endpoint confusion.

Finding 5 — Factorized decode composition closes the relation-binding wall

The closure mechanism is factorized decode composition: the single overloaded relation-logit path is split into three distinct evidence channels, composed at decode time:

• a directed / ordered carrier gives pair geometry and direction (a direction-blind carrier was insufficient; an ordered carrier restored directional evidence, and an auxiliary direction head learned endpoint-role reversal);
• a local-patch evidence channel gives relation-label evidence;
• a pair-used (candidate-score) field gives edge existence / coordinate support.

At evaluation the signed direction route is applied with a fixed route weight, gated by the edge-existence evidence at a larger fixed gate weight. (Exact composition weights are not reported here.) Naive scalar bridge/gate losses either suppressed true edges or re-admitted endpoint-role hallucinations; keeping the roles separate and composing them at decode time is what worked.

Finding statement: a relation edge is best decoded as a composition of relation-label evidence, pair-geometry-and-direction evidence, and coordinate edge-existence evidence. This reproduces across three seeds and larger validation.

Three-seed evidence (held-out validation): across all three seeds, the true-edge-predicted-none rate and the no-edge false-positive rate stayed within the working closure bands, and evidence top-k edge agreement was consistently nonzero and comparable across seeds. Endpoint-role false positives were controlled after normalization.

What is closed

• Visual relation failure is decomposed mechanistically rather than treated as unlearnable.
• Pair-coordinate localization is no longer the dominant unexplained wall under the closure decode.
• Endpoint-role hallucination is controlled without broad true-edge collapse.
• The mechanism reproduces across three seeds and larger validation.
• The next phase can build on factorized semantic-state formation rather than continuing broad bridge/gate sweeps.

What is NOT claimed

This is an engineered-schema mechanism milestone — it validates a path for rendering-derived state formation, not the whole world-model architecture. Explicitly NOT claimed: full paradigm validation; a learned (versus hand-built) semantic schema; semantic hierarchy/scoping; native multimodal generality beyond text/visual; activation-graph memory; that this exact decode is optimal; or that relation binding remains solved at much larger vocabularies or open-world rendering distributions.