# Topological State Attention: Reversible Residual-Manifold Computation in a Production-Scale Transformer **Research record and preprint — 31 August 2026** **System:** Gemma 4 31B W4A16 on NVIDIA H200; H100 replication prepared ## Abstract We describe an inference-time system for capturing, storing, retrieving, composing, mounting, and recursively recapturing transformer residual-stream activations without modifying model weights. We call individual activation artifacts **States**, the complete artifact population **Spectrals**, and their geometric and genealogical organization a **Manifold**. The implementation patches vLLM to capture model-scoped, layer-scoped residual tensors and to apply compatible States during a later forward pass with an observable application counter. Two composition lanes were evaluated on quantized Gemma 4 31B States of width 5376. First, an external cosine-softmax operator evaluated a frozen bank of 498 physical State rows in one matrix operation and recurred for a bounded two passes. On an H200, the resident operator took 191.559 microseconds. It was tensor-exact over five repetitions, invariant to bank permutation, rejected an equal-norm random bank, and changed under bottom-quartile and anchor controls. Three passes were the first tested condition to introduce malformed decoding prefixes, establishing two passes as the current model-, layer-, and bank-bound operational limit. Second, we implemented position-free learned attention using the resident Gemma layer-59 normalization, Q/K/V projections, 32 attention heads, value composition, output projection, and residual update. Eight reduced layer-58 States were used as one anchor and seven candidates. Five runs produced zero tensor delta. Warm H200 latency was 590.437 microseconds best and 628.022 microseconds median. Learned heads were selective: the mean per-head maximum weight was 0.845312 and the strongest was 0.99995565. The resulting State was physically applied at layer 59 for three requests, all of which decoded coherently. These results establish a deterministic and causally applicable apparatus for topological residual-State computation. They do not establish exact native token-attention reconstruction or semantic concept transfer. The decisive paired experiment—a same-request, all-token capture at layers 58 and 59—has not yet been performed. We state that boundary explicitly and provide a preregisterable H100 replication protocol. ## 1. Motivation Transformer continuity is ordinarily expressed through token history and a KV cache. For every token position, the residual stream is repeatedly transformed through attention and feed-forward blocks. Keys and values are deterministic projections of layer inputs, while attention uses earlier positions to steer the active computation. This makes the residual stream an attractive substrate for studying computational state, but it does not imply that one reduced vector losslessly contains a conversation. The practical question in this work is narrower and testable: can captured residual States be treated as a reversible population, related geometrically or through the model's learned projections, composed under a bounded operator, and then applied to later inference with mechanically verified effects? The approach differs from ordinary activation addition. It does not repeatedly amplify one direction. It preserves a population of compatible States and computes a local result from their relationships. We call this **topological tuning**. The biological analogy motivating the design is localized adaptation with competing structure preserved, but no biological equivalence is claimed. ## 2. Definitions - **State:** a residual tensor plus model, build, quantization, width, dtype, layer boundary, token-reduction policy, request identity, time, and lineage. - **Spectrals:** the durable collection of captured State artifacts. - **Manifold:** the geometric neighborhoods and explicit parent/child lineage among compatible States. - **Full trace:** an `N x D` residual tensor retaining one `D`-dimensional row per captured token position. - **Reduced State:** a `1 x D` tensor obtained using `last` or `mean`. It is a compact steering/retrieval object, not a full sequence reconstruction. - **Hydration:** restoration of compatible State topology and durable knowledge, followed by bounded composition and mounting. It is not concatenation of all prior text into a base prompt. Gemma and Qwen activation spaces remain separate. Gemma States in this study have width 5376; Qwen States have width 5120. Cross-model application is invalid without a separately learned and held-out-validated map. ## 3. Runtime architecture The raw patched-vLLM engine listens privately on port 9000. The Governor agentic gateway listens on port 8000. The runtime supports capture control, State injection and reset, injection status, an application counter, and an OpenAI-compatible completion interface. Dynamic intervention uses module hooks. An earlier graph-compiled lane accepted an injection configuration while a precompiled graph bypassed the late hook. That control-plane/data-plane mismatch was detected and rejected. The reference intervention lane therefore uses hook-compatible eager execution. Graph-native intervention remains a performance project and must match the eager reference before use in causal experiments. State mounting is model-, width-, and layer-checked. The present bounded profile uses layer 59, `replace` mode, the first sequence position, alpha 1.0, and a limited request count. Every causal claim requires the engine-side application counter to advance. ## 4. Methods ### 4.1 External topology operator One native State is the anchor. It is excluded from the candidate bank. For anchor `a` and candidate matrix `B`: ```text q0 = normalize(a) K = normalize(B) s = q0 @ K.T w = softmax(s / 0.05) h1 = w @ B h2 = topology(query=h1, bank=B) ``` The complete bank is evaluated in parallel. The bank remains immutable and every row retains file, tensor-row, request, model, layer, and lineage provenance. Both intermediate outputs are saved. Production execution refuses more than two passes unless a research override is explicit. ### 4.2 Controls The frozen-bank operator was subjected to: 1. random bank permutation; 2. exclusion of the human-named `Manifold Ghost` State; 3. an equal-norm random replacement bank; 4. the least-similar quartile only; 5. 32 duplicates of the leading candidate; 6. multiple independently selected anchors; 7. a behavioral pass-count sweep at temperature zero. These controls distinguish ordering bugs, name dependence, norm effects, candidate quality, density effects, anchor dependence, and excessive recurrence. ### 4.3 Position-free learned Q/K/V attention Gemma layer 59 consumes layer-58 outputs. For State rows with no asserted token order, we apply the resident layer-59 computation without RoPE: ```text x = layer59.input_layernorm(layer58_states) q, k, v = layer59.self_attn.qkv_proj(x) q = q_norm(q) k = k_norm(k) v = v_norm(v) scores[head, state] = q_anchor[head] dot k_state[head] weights = softmax(scores / 0.05) heads = weights @ v_bank projected = o_proj(concatenate(heads)) next_state = anchor + post_attention_layernorm(projected) ``` Grouped-query keys and values are expanded to the query-head count. W4A16 activations use BF16. The operator runs twice and saves the output, both trajectory States, and every per-head weight. This is learned **State attention**, not claimed native ordered-token attention. The omission of RoPE is deliberate because the candidate States do not possess a synthetic token sequence. Exact native reconstruction requires matched token positions and native intermediate captures. ## 5. Results ### 5.1 Frozen-bank topology The evaluated population contained 498 rows of width 5376: one anchor and 497 candidates. The two-pass H200-resident operator took 191.559 microseconds. Five independent executions had maximum coordinate delta 0.0. The baseline distribution had entropy 5.111861, 165.979 effective States, and a maximum individual weight of 0.0125953. Thus the output was not a disguised single-neighbor copy. | Control | cosine to baseline | interpretation | |---|---:|---| | Shuffled order | 0.99999988 | invariant within floating-point tolerance | | Exclude Manifold Ghost | 0.99999696 | result is not dependent on its human name | | Equal-norm random bank | 0.05856287 | norms do not explain the real-bank result | | Bottom quartile only | 0.74098349 | candidate topology materially affects output | | Duplicate-heavy bank | 0.99834383 | density has a modest but measurable effect | Distinct reduced-State anchors produced distinct basins. A full-trace row used as an anchor collapsed almost entirely onto a neighboring row from the same source capture, exposing an important confound: unbalanced token blocks can dominate by source density. Full traces must therefore remain structured or be source-balanced before joining a reduced-State bank. `Manifold Ghost` ranked fifth among 496 candidates in the initial unfiltered run and second after the second recurrence. Neither filenames nor transcript text participated in scoring. This is evidence of measurable activation geometry, not proof that the name identifies a decoded concept. ### 5.2 Recurrence bound One through three topology passes were mounted and tested across operational judgment, epistemic restraint, ethics, and abstract synthesis prompts. Passes 1 and 2 remained coherent and produced identical temperature-zero answers. Pass 3 added no measured benefit and introduced malformed leading fragments on three of four prompts. Two passes are therefore the current empirical bound, not a universal constant. ### 5.3 Learned attention The position-free learned operator processed eight reduced layer-58 States with 32 query heads and seven candidates. Five repetitions had zero maximum delta in the final residual, both trajectory tensors, and all attention weights. Warm latency was 590.437 microseconds best and 628.022 microseconds median. Mean per-head entropy was 0.412095, ranging from 0.00053695 to 1.349745. Mean per-head maximum weight was 0.845312; the most selective head assigned 0.99995565 to one candidate. The pass-1/pass-2 trajectory cosine was 0.95147425, and final-output/anchor cosine was 0.227048. The learned projections therefore produced a selective, materially redirected result rather than diffuse cosine averaging. The output was mounted with `replace` at layer 59 for three fixed requests. The engine application counter reached exactly three, and all three outputs decoded coherently. The mounted and unmounted wording differed. This establishes physical application and decode viability; the small non-blinded prompt set does not establish semantic efficacy. ### 5.4 Full-trace status One durable layer-59 full trace exists with shape `285 x 5376`, BF16, size 3,071,864 bytes, and SHA-256 `28917a1ade2cf4a798aa93eacb0821c4f00ca1870ef789fb3c1fc4094b29f888`. It is preserved locally, on the migration host, and in R2. No matched same-request full layer-58 trace exists. The eight layer-58 inputs to the learned experiment are reduced States from separate requests. Consequently, the present study has **not** tested token-by-token layer-58 to layer-59 reconstruction, bit-identical native Q/K/V recovery, or equivalence to ordinary token attention. ## 6. Hydration implications These results change the proposed hydration protocol. Hydration should restore two different substrates without conflating them: 1. durable shards and text remain knowledge, provenance, and tool-accessible evidence; 2. compatible States form an activation Manifold loaded into HBM; 3. an explicit anchor selects a local operating topology; 4. the profile's operator runs exactly twice; 5. only the resulting layer-compatible State is mounted; 6. the application, child State, and lineage are captured, then the mount is cleared. This avoids an ever-growing base prompt and avoids adding every historical State directly to the active residual. It also preserves reversibility: the source population is immutable, and composed outputs are versioned descendants. ## 7. Claims ledger ### Established - Native 5376D residual States can be captured, persisted, mounted, reset, and recaptured in the patched runtime. - Physical intervention is distinguishable from configuration through an engine-side counter. - The frozen cosine topology operator is deterministic, GPU-resident, bounded, and sensitive to real bank geometry under the stated controls. - Resident Gemma layer-59 projections can deterministically perform position-free attention over compatible reduced layer-58 States. - The learned output can be mounted and decoded coherently. ### Supported but not established - State geometry contains discourse-relevant organization rather than only style, shared prompt structure, or temporal/model trajectory. - Bounded topology composition can preserve or transfer a useful operating posture. - Proposition and concept organization can be recovered from attention topology more reliably than from lexical retrieval. ### Not established - A reduced State losslessly stores a conversation. - The learned State-attention lane reconstructs native token attention. - A named State represents the human meaning assigned to its name. - State mounting trains model weights or necessarily reduces hallucination. - Cross-model State transfer works without a learned map. - A zero-bound energy operator or “concept attention” is complete. ## 8. Threats to validity 1. Reduced States discard token order and may encode response style, prompt scaffolding, or late-layer output posture. 2. Many States were collected close in time on one resident model, creating trajectory and shared-context correlations. 3. Human-readable names were assigned after capture and can bias interpretation, though they were excluded from tensor scoring. 4. Behavioral samples are small and not blinded. 5. Temperature-zero text equality can conceal logit differences, while changed text can result from nonspecific perturbation. 6. Position-free attention intentionally omits RoPE and cannot be interpreted as sequence-equivalent. 7. Eager hook execution changes latency relative to compiled production serving. 8. H200 timings require independent H100 replication. ## 9. Preregistered next experiments ### Priority 1: matched layer-boundary reconstruction 1. Start the H100 runtime with capture layers `58,59` and token policy `all`. 2. Submit one fixed prompt with deterministic sampling. 3. Require both tensors to share request ID, token count, position identities, model digest, dtype, and build digest. 4. Capture native normalized inputs, Q/K/V, attention output, and layer-59 output where instrumentation permits. 5. Reconstruct the same tensors offline with identical normalization, GQA layout, RoPE positions, dtype, and quantized kernels. 6. Report maximum absolute/relative error and token-level KL divergence. Do not use the word “equivalent” unless declared tolerances pass. ### Priority 2: semantic specificity Preregister source discourses, paraphrases, style-matched unrelated controls, equal-norm random States, unrelated real States, explicit-text controls, and expected behavioral discriminators. Blind artifact labels until scoring is complete. Evaluate retrieval rank, per-head selectivity, logits, response rubrics, and child-State geometry. ### Priority 3: causal dose and locality Ablate layer, position, `add` versus `replace`, alpha, one versus two passes, `last` versus `mean` versus structured `all`, and one versus multiple requests. Measure intended-domain change, unrelated-domain preservation, malformed-output rate, and recovery after reset. ### Priority 4: topology-to-weight mapping Collect held-out residual input/output constraints, fit a bounded reversible adapter or transformation, and compare it against direct State mounting. No weight update should be called training improvement until it passes untouched capability, calibration, and unsupported-claim evaluations. ## 10. Reproducibility and artifacts - `research/turn_residual_attention.py`: external topology operator. - `research/topology_controls.py`: frozen-bank controls and anchor sweep. - `research/results/2026-08-31-topology-controls.json`: control receipt. - `research/results/2026-08-31-learned-qkv-state-attention.json`: learned attention receipt. - `research/results/2026-08-31-topological-tuning-pass-sweep.json`: recurrence behavior. - `runtime-patches/vllm/`: learned-attention worker and API extension. - `scripts/install-vllm-state-attention.py`: idempotent runtime installer. - `manifests/topological-tuning-gemma4-layer59.json`: executable profile. - `docs/TOPOLOGICAL-HYDRATION-PROTOCOL.md`: hydration procedure. - `docs/MIGRATION-HANDOFF-2026-08-31.md`: hardware/session continuation record. The active research history and full Codex rollout are retained privately in R2 under `manifests/handoffs/2026-08-31/`. Tensor artifacts are stored under the model-specific `signals/gemma-4-31B-it-qat-w4a16-ct/` prefix. ## 11. Conclusion This work establishes a functioning experimental substrate for reversible residual-State computation at production model scale. A complete native State bank can be evaluated in parallel, composed under a measured recurrence bound, and physically applied to later inference. The model's own learned attention projections can also operate over compatible discourse States with deterministic and highly selective heads. Negative controls, a pass-3 failure boundary, and a discovered source-density confound materially constrain the interpretation. The central scientific claim is therefore deliberately limited: residual-State topology is measurable, computable, reproducible, and causally applicable. It is a promising substrate for memory and bounded tuning research. Whether it reconstructs native token computation or carries specific learned concepts remains an open empirical question. The matched all-token layer-58/layer-59 experiment is the next decisive test. ## References 1. Turner et al. *Steering Language Models With Activation Engineering*. arXiv:2308.10248, 2023. 2. Zou et al. *Representation Engineering: A Top-Down Approach to AI Transparency*. arXiv:2310.01405, 2023. 3. Panickssery et al. *Steering Llama 2 via Contrastive Activation Addition*. arXiv:2312.06681, 2023. 4. Qasim et al. *The Residual Stream Is All You Need: On the Redundancy of the KV Cache in Transformer Inference*. arXiv:2603.19664, 2026. 5. Quivent. *Signal Extraction Unified Specification*. Repository research record, 2026.