RUNTIME_GUARDRAIL_ARCHITECTURE.md

systems/RUNTIME_GUARDRAIL_ARCHITECTURE.md

<!-- The runtime GUARDRAIL spine: the deterministic scaffolding that makes a dumb local LLM reliable at runtime for "Humanity" on consumer hardware. Operationalizes the /dev/continuo lesson (a lesser model wrapped in code that encodes its failure/reward patterns hits near-perfect reliability) for a shipping game. Extends docs/RUNTIME_GENERATIVE_LAYER.md (the split + the alive-not-slop framing) and docs/CANON_INTEGRITY_ARCHITECTURE.md (the canon graph + recall = the truth store). Sits in the T99 operational tier alongside PIPELINE_ADAPTER/ROADMAP; invents zero world canon; the cardinal do-not-invent rule binds. Absolute numbers (latency ms, VRAM, quant) stay illustrative shape pending a Phase-5M-class runtime perf audit. -->

Humanity — Runtime Guardrail Architecture

The safety spine that lets a dumb local LLM ship without slop or lore-breaks

Companion to docs/RUNTIME_GENERATIVE_LAYER.md (which fixes *what* the model may generate) and to docs/CANON_INTEGRITY_ARCHITECTURE.md (which builds the truth store). This doc specifies the *guardrail* — the five deterministic mechanisms that make the strategic-only local LLM reliable at runtime on a player's own PC / Xbox / PlayStation, fully offline, inside a frame budget. It is the runtime analog of the harness/ F1–F10 gates: the code that catches the model's mistake classes before a player ever sees them.

---

0. The bet, stated precisely (the continuo lesson, operationalized)

/dev/continuo (C:\dev\continuo\CLAUDE.md) proved a specific, transferable thing: a *dumb* 14B model reached near-perfect reliability on untrusted extraction not by being smart but by being wrapped in deterministic code that encoded every failure/reward pattern and gave it perfect-precision recall over a bounded, structured truth store. Its constitution names the exact moves we inherit:

• §4.2 write-time-structured, read-time-cheap — "expensive AI work once, at ingestion; queries touch small pre-compiled ID-addressed context." Our runtime version: the expensive canon recall happens *at asset-bake time*, per entity, deterministically; the per-frame path touches only a small frozen struct.
• §4.4 hard token budgets everywhere — over budget ⇒ compaction, never expansion. Ours: hard token caps per generation class, enforced by the decoder, not hoped for.
• §4.6 forced provenance — "every generated claim cites segment IDs or is dropped/flagged." Ours: every named entity in a generation must resolve to a canon-graph node in the entity's whitelist, or the line is dropped to authored fallback. Cite-or-drop, at runtime.
• §4.11 self-contained for the customer — "no external AI tool is ever a dependency; free tier works fully offline forever." Ours verbatim: model, graph, packs, validator, fallback bank all ship in the build; zero network calls.
• M4 detect, surface, one-tap fix — never silently fail or guess. Ours: every guardrail failure is *logged loudly* and *falls back silently to the player* — never a silent wrong answer.
• M7 make illegal states unrepresentable. Ours: grammar-constrained decoding makes malformed/out-of-vocab output *structurally impossible*, and the generation API has *no write path into the sim* — the two highest-value guardrails are the ones that delete a failure class rather than catch it.

The continuo scorecard was a number: fraction of extractions that passed all deterministic checks. Ours is the same number — first-try guardrail pass rate — and §4 below makes it the ship gate.

The load-bearing inversion: the model owns *zero* facts. The harness/canon_graph.json (3,118 nodes / 11,027 edges, live on disk) owns *all* of them. The model performs surface texture over a truth the deterministic layer already proved. It cannot lie because it is never asked to know.

---

1. Scope lock — what the local LLM is allowed to touch (Josh's hard constraint, made structural)

Josh's constraint: mostly deterministic code; the local LLM is confined to STRATEGIC generation only — goals, dispositions, plans, dialogue, adaptation, faction-intent — NEVER per-frame mechanics. This is not a policy the model is asked to respect; it is a wall the architecture enforces. The model's output is consumed only through a typed API surface that *has no per-frame or state-mutating verbs*.

1.1 — The strategic/tactical boundary (the wall)

The rule that draws the line: the model proposes strategy in a vocabulary the deterministic layer defines; the deterministic layer disposes. A boss's *plan* ("bait the roll-spammer, then punish") is strategic and may be model-chosen — but every *step* of executing it is a deterministic behavior-tree action from a fixed vocabulary, on a Sigma-bounded timer (COMBAT_ENCOUNTER_SYSTEM.md §8). The model picks *which* authored tactic; the sim *runs* it. A hallucinated tactic that isn't in the vocabulary is unrepresentable (§3 constrained decode) and, if it somehow appeared, unexecutable (no matching action id).

1.2 — Strategic generation is still guarded

Confining the model to strategy does not make strategy safe — a bad *goal* ("provoke the protected Sámi community," HL_0003) is as canon-breaking as a bad *fact*. So the guardrail loop (§2–§5) wraps strategic generation exactly as it wraps dialogue: a faction-intent choice is validated against the entity's hard-line rule vector before it reaches the sim; a plan graph is checked so every node is a real action id and no node violates a protected-community flag. Strategy is generated inside the same four-stage sandwich as speech.

---

2. The guardrail loop — one model call, four deterministic stages

Every generation — strategic or cosmetic — is a deterministic sandwich around a single bounded model call. The model is the meat; it never touches the outside world directly. This is the runtime shape of the build-time cascade recall → generate → lint → baseline loop.

  IN: (entity_id, event, live_sim_state_slice)
       |
  [1] CONTEXT PACK   — deterministic. Assemble a frozen, canon-true prompt from the
       |               entity's precomputed pack + the live state slice.  (§3)
       v
  [2] CONSTRAINED    — local LLM. Grammar/enum/canon-vocab-constrained decode, hard token
      DECODE           cap, prefix-cached persona preamble.  Malformed = impossible.  (§4)
       |
       v
  [3] VALIDATE       — deterministic. Battery of cheap checks vs the canon graph + rule
       |               vector (the runtime cousin of lint_canon_graph.py's G-gates).  (§5)
       |                 PASS  -> emit
       |                 FAIL  -> [4]
       v
  [4] FALLBACK       — deterministic. Emit the authored line/plan for (entity, state_bucket).
                       Log the miss to the reward corpus. Never block the frame.  (§6)

Four properties hold by construction: the prompt is always canon-true (stage 1 is code, not model); the output is always well-formed (stage 2 makes malformation unrepresentable); the output is always canon-valid or discarded (stage 3 is cite-or-drop); and the player always gets *something shippable* (stage 4 is the authored floor). The model is pure upside — never a dependency, exactly as continuo's free tier works fully offline with the model as enhancement, not requirement.

---

3. Deliverable (1) — the CONTEXT PACK: recall.py's "all-canon-for-a-node" query, frozen per entity

This is recall.py's recall node <id> union query (CANON_INTEGRITY_ARCHITECTURE.md §D, query shape 2 — "the load-bearing cascade query") run once at asset-bake time per entity, not per frame. It is continuo's §4.2 move exactly: pay the expensive, must-be-perfect recall once, deterministically, and hand the dumb model a bounded, pre-verified truth slice it cannot escape.

3.1 — What a pack is

A context pack is a small frozen struct, produced by a new offline exporter recall packs that walks canon_graph.json outward from the entity node and emits, per entity:

✅ Launch posture RULED (D-COMBAT-TD R7, Josh 2026-07-19) — the entity-class scalpel at the high-care chapters (Ch 29 / 42 / 57): bosses + generic ambient fauna ship runtime-generation ON under the standard V1–V5 guardrail, but each such boss's fact_whitelist excludes the Dreaming / songline / joik nodes (HL_0007). Community and cultural NPCs, and folklore-bearing fauna, are AUTHORED-ONLY; any entity whose care_band is high_care/elevated_care but is left untagged defaults to authored-only (fail-safe). Every affirmatively-ON tag takes a two-reviewer audit pre-launch, and any later community-NPC flip-on is gated on a human source-fidelity review, never the §7 metric alone. Cite basis: HL_0002 (retained) + HL_0003 / HL_0005 / HL_0007. This ruling supersedes the §9 open-item recommendation below.

3.2 — Why per-entity-frozen is the whole efficiency story

Because the pack is precomputed and immutable at entity-load, the per-frame cost is zero graph traversal. Runtime prompt assembly is: load cached pack (already resident) → splice in the live sim-state slice (a handful of scalars: current phase, player's last action class, integrity band, recent-event flags) → hand to the decoder. The frozen persona preamble is KV-prefix-cached on the inference server (continuo §4.2 read-time-cheap), so only the short variable tail is ever decoded. On consumer hardware this is the difference between shippable and not: no per-frame graph walk, no per-frame retrieval, no growth without bound (continuo §4.5 — the pack is current-truth state, not an append-only context that balloons).

3.3 — The live-state slice is the *only* dynamic input, and it is trusted

Everything canon in the prompt is frozen and pre-verified. The only per-call variability is the live-state slice, and it comes from the deterministic sim, not the model — the boss's own next-move intent, the player's measured player_habit_vector (COMBAT_ENCOUNTER_SYSTEM.md §4.9), the worldstate delta. So when a boss "reads you," the tactical fact is *true by construction* (the sim measured it); the model only chooses how to voice it. This is what makes the adaptivity real and safe at once: the model never invents the tactic, it phrases a tactic the sim handed it.

3.4 — Player text is a retrieval key, never a prompt (injection defense)

For talk systems, raw player text is never concatenated into the prompt as instruction. It is (a) intent-classified by a small local classifier / keyword map to a topic id, which selects a whitelist slice, and (b) quarantined behind a delimiter marked untrusted, with the system preamble stating so. The player cannot steer the model outside its pack because the pack, not the player, defines what is knowable. (Continuo M1 — eliminate the injection surface at the source.)

---

4. Deliverable (2) — CONSTRAINED DECODING: structurally & canonically invalid output made impossible

Continuo M7 — *make illegal states unrepresentable*. The highest-value guardrail is the one that deletes a failure class instead of catching it. Constrained decoding removes malformed and out-of-vocabulary output *before it can be sampled*, so stage 3 never has to catch F8 (malformed structure) at all.

4.1 — Three layers of constraint, tightest-first

1. Structural grammar (GBNF / JSON-schema / regex-constrained sampler — llama.cpp grammars, Outlines, or the engine-embedded equivalent). The output *shape* is guaranteed: a bark is a JSON object {emotion: <enum>, line: <string ≤N tokens>}; a strategic choice is {intent: <enum>, target: <enum>}; a plan is an array of nodes each {action: <action_id ∈ action_vocabulary>, arg: <enum>}. The decoder cannot emit malformed JSON, an out-of-enum emotion, an unknown action_id, or an over-length line. This single mechanism deletes: structural corruption, invalid strategic enums, and hallucinated action ids (a plan step outside action_vocabulary is off the grammar and un-sampleable).

2. Canon-vocabulary constraint (the novel layer — see §4.2). For fields that name entities, the grammar's terminal for "proper noun slot" is restricted at decode time to the union of the entity's fact_whitelist surface forms plus a common-noun escape. The model literally *cannot spell* a person/place/item outside its whitelist, because those tokens are masked out of the proper-noun position. This preempts F1 (hallucinated entity) and F2 (retired alias) at the sampler, not the validator.

3. Hard token cap per class. Barks 12–20 tokens; ambient ≤40; probe responses ≤80; a strategic plan ≤K nodes. Continuo §4.4 — over budget ⇒ truncate to fallback, never expand. Short outputs are faster, cheaper, and have less surface to go wrong.

4.2 — The improvement over the state of the art (do not copy — better it)

Standard grammar-constrained decoding (Outlines, llama.cpp grammars, XGrammar) constrains output to a *static* schema — a fixed JSON shape or a fixed enum known at compile time. The novelty here is a per-entity dynamic canon-vocabulary mask driven by the canon graph: the set of legal proper-noun tokens is not a compile-time constant, it is the *runtime* fact_whitelist of *this* entity, compiled to a token-id allow-set when the pack loads and installed as a logit mask on the proper-noun grammar terminal. Two entities standing next to each other run the same model and the same base grammar but decode against different legal-name universes — the Ch 10 villager physically cannot utter "Maatherion"; the Ch 69 sage can. This is retrieval-augmented *generation-time* constraint (the graph shapes the sampler, not just the prompt), and it collapses the two most dangerous lore-break classes (invented names, retired aliases) from "validated after the fact" to "impossible to sample." It is cheap: a precomputed token-id set per pack and a logit mask applied only at proper-noun positions (a tiny fraction of decode steps), so it costs near nothing per token on consumer hardware. Stage 3's name-resolution check (§5) then becomes a belt-and-suspenders assertion that should never fire — and if it does, it means a pack-build bug, which is a build-time failure, not a runtime one.

The design rule: prefer the constraint that makes the mistake unrepresentable over the check that catches it. Everything a grammar or a mask can prevent, it prevents; the validator (§5) only catches what cannot be structurally forbidden (semantics: intent, register, slop).

---

5. Deliverable (3) — POST-GENERATION VALIDATION: the deterministic gate (where continuo lives)

The runtime cousin of lint_canon_graph.py's G-gate battery: cheap deterministic checks that catch the *specific* ways this dumb model fails, ordered fast-and-common first, any hard-fail → fallback. Each check maps to a build-time gate so build and runtime enforce the same canon from the same source of truth (continuo: one Constitution, cited).

V1–V3 are correctness gates (hard-fail → immediate fallback, no retry — correctness is not negotiable and a retry burns the latency budget). V4–V5 are quality gates (soft-fail → one cheap retry if the latency budget allows, else fallback). Every failure — hard or soft — writes {entity, state, prompt_hash, output, failed_check, latency} to the rotating runtime_generation_log. That log is the reward corpus (§7). The validator is *stateless and cheap*: regexes, a set-membership test against a precomputed token/name set, a length check, an edit-distance against a K-line ring buffer, and one small intent classifier — all comfortably inside the frame-off-thread budget.

---

6. Deliverable (3, cont.) — AUTHORED FALLBACK: the shippable floor

Continuo M4 — never silently fail or guess; surface the miss, ship the safe answer. Every generative slot has a deterministic authored fallback keyed by (entity_id, state_bucket) — the same state_bucket_map in the pack (§3.1). The bank per bucket is 3–6 hand-written lines (RNG-picked, Sigma-bounded so it doesn't repeat) authored alongside the few-shot exemplars.

Fallback triggers on any of: a V1–V3 hard fail, a V4–V5 soft-fail after the one-retry budget, a latency-budget miss (§8), or a decoder/grammar error. For strategic output, the fallback is the deterministic default behavior the sim would run absent the model — the authored behavior-tree default, the scripted disposition. So a boss whose strategic generation misses simply plays its hand-authored fight; the player sees a solid, canon-clean encounter, never a hole, hang, or slop line.

This is what makes the whole system safe to ship at AAA quality: the local LLM is upside, never a dependency. Worst reachable output is a hand-written line or the authored default behavior — identical to shipping the game with no LLM at all. The model can only *improve* on the floor; it can never drop below it. (Continuo §4.11: the free tier — here, the fallback-only mode — works fully, offline, forever.)

---

7. Deliverable (4) — FAILURE/REWARD DISCIPLINE: the F1–F10 gates as runtime code

Continuo's real lesson was not "add a model" — it was enumerate the specific ways the dumb model fails, encode each as a deterministic catcher, and turn every caught failure into a reward signal that hardens the system. This is the runtime analog of the harness's F1–F10 gate discipline. Each row is a real, cheap, deterministic mechanism, most-prevented-by-construction-first:

The reward loop closes exactly like continuo's and like this repo's harness. Every caught failure writes to the local runtime_generation_log, which (a) grows the authored-fallback bank, (b) grows the denylist/slop sets, and (c) becomes eval fixtures. A nightly/CI offline job replays the logged prompts against candidate model+prompt versions and scores them with the *same §5 checks* — yielding a truth-recall-style metric: first-try guardrail pass rate, directly analogous to continuo's near-perfect scorecard. You ship a model+prompt version only when its first-try pass rate clears the bar. The fallback floor guarantees quality never drops below "shippable" regardless of the model version — so the model can be upgraded fearlessly across the game's life. This is the whole continuo bet applied to runtime: the model gets smarter over time, but the system is already correct on day one because the deterministic scaffolding — not the model — guarantees canon.

---

8. Deliverable (5) — OFFLINE & DETERMINISM-WHERE-NEEDED GUARANTEES

The layer must be invisible, fully offline, deterministic where reproducibility matters, and never the reason a frame drops. On consumer hardware (a player's own GPU shared with the game), efficiency is the gating constraint, so every choice below is an efficiency choice as much as a safety one.

8.1 — Fully offline, self-contained (continuo §4.11)

Model weights, canon_graph.json, the per-entity packs, the validator, the denylists, the alias index, and the fallback bank ship in the build. Zero network calls; no cloud inference; no telemetry required to function. The runtime_generation_log is local-only; §7's tuning corpus uploads only on explicit opt-in. The game is fully playable — at the authored floor — with the LLM feature disabled entirely.

8.2 — The efficiency budget (illustrative shape; locked at a runtime perf audit)

• Latency budgets, off the render thread. Barks/reactions ≤120 ms P99 soft, 250 ms hard → fallback. Strategic re-plans run on discrete events (phase change, aggro shift), not per frame, with a wider window. Ambient ≤300 ms, pre-generated during idle. Probe responses ≤700 ms with a "thinking" beat allowed. Combat inference never runs on the render thread; a miss drops to authored, never stalls the sim.
• Model + VRAM. A 3–8B quantized model (Q4_K_M / Q5), ~2–5 GB VRAM, co-resident with the game which owns most of the budget. One shared instance — persona differences come entirely from the context pack, never from per-character fine-tunes (that is *why* the pack must carry the persona).
• Batching + scheduling. One inference server batches concurrent requests (a crowd; a boss + adds). Priority queue: combat barks > player-facing probe > ambient. Ambient is speculative/pre-warmed during low load and cached per (entity, state_bucket), so a crowd never causes a burst. Prefix-cache the frozen persona preamble (§3.2) so only the variable tail decodes.
• Console parity. The 3–8B/Q4 target and the single-shared-instance rule are chosen so the same budget holds on Xbox/PlayStation shared memory, not just a high-end PC. Where a platform cannot afford runtime inference at all, the fallback-only mode (§6) *is* the ship configuration for that platform — no code path changes, the floor is already the whole game.

8.3 — Determinism where needed (mirrors the harness's declared-baseline discipline)

• The sim is fully deterministic (§1 tactical tier). Generated text is cosmetic and seedable: seed the sampler from (world_seed, entity_id, event_counter) so a replay/speedrun/bug-repro reproduces the exact same lines. Strategic *choices* that feed the sim are drawn from the same seeded stream, so a replay reproduces the same strategy.
• A "deterministic text" / accessibility mode forces authored-fallback-only — reproducible, esports-legit, and identical to the no-LLM configuration. Available in every difficulty tier including The Forgotten One.
• No feedback into sim state. Generated text is a pure output leaf (F10); strategic proposals are consumed as *bounded choices over an authored vocabulary*, so the sim's state space is unchanged whether the model is on or off. This is the harness discipline ported: reproducible by default, any variance is declared (seeded) and logged, and the trustworthy signal (here, a stable replay) never degrades silently.

---

9. Buildability — how this lands on what already exists

Nothing here is greenfield; it extends artifacts on disk today.

• Graph already built. harness/canon_graph.json (3,118 nodes / 11,027 edges) + harness/build_canon_graph.py + harness/lint_canon_graph.py exist. The runtime layer adds one offline exporter, recall packs, on top of the planned harness/recall.py — it reuses the exact recall node <id> union query and the recall aliases reverse index to emit frozen per-entity packs. (recall.py itself is the planned Stage-4 component in CANON_INTEGRITY_ARCHITECTURE.md §D/§E; this spec is a consumer of it, and gives it a second caller alongside the cascade.)
• Hard lines already machine-addressable. T0_Hard_Lines carries applies_to_pipeline, applies_to_chapter_range, applies_to_high_care_chapter, severity, cross_reference_array. The rule-vector compiler reads these directly; add pipeline enum value Runtime so runtime-relevant lines are selectable.
• Personas already have substrate. T0_Voice_Registry.voice_substrate + T0_Character_Index + T0_Creature_Roster.folklore_behavior_substrate are the persona-card source; few-shots + fallback lines are authored as cascade VO work.
• Validation reuses gate logic. §5 V1–V3 are the runtime cousins of lint_canon_graph.py G1/G2/G4/G9; they share the denylist / alias / hard-line sources so build-time and runtime enforce identical canon.
• Determinism discipline reused. The seedable sampler + authored-fallback-only mode mirror the harness's declared-baseline discipline: reproducible by default, degradation declared and logged, the trustworthy signal never silently degrades.

Open items for Josh (flagged, not decided — the do-not-invent rule binds)

• [DECISION NEEDED - JOSH] Launch posture: which entity classes ship with runtime generation ON vs authored-only. Proposed: bosses + named NPCs + fauna ON; culturally high-care Ch 29/42/57 NPCs authored-only at launch given HL_0003/HL_0005 stakes — the model assists content *authoring* for protected communities, not runtime speech. This is a §17.1 posture call, not a technical one.
• [DECISION NEEDED - JOSH] Target local model + quant (3B vs 8B) and the per-class latency/VRAM numbers — a playtest/eval call once the §7 first-try-pass-rate harness exists and a runtime perf audit runs (the runtime analog of Phase 5M).
• [DECISION NEEDED - JOSH] Whether strategic generation (goals/dispositions/plans/faction-intent) ships in v1 at all, or v1 ships cosmetic-only (barks/flavor) with strategy behind the authored behavior trees until the §7 pass-rate clears a higher bar for the higher-stakes strategic output.
• Do-not-invent reminder: any generated name/fact the design *wants* to become canon routes through Josh as [DECISION NEEDED - JOSH]; the runtime never self-canonizes. Bosses/schools/rosters remain Claude-proposes / Josh-approves per the Batch-1 contract.

---

10. One-paragraph synthesis

The pervasive local-LLM layer ships safely because a *dumb* model, confined by an architectural wall to strategic-and-cosmetic output only (§1), performs surface texture over a truth store it cannot touch — the exact /dev/continuo bet (a lesser model wrapped in deterministic code that encodes its failure/reward patterns reaches near-perfect reliability), operationalized on our canon graph. The safety spine is five deterministic mechanisms: a context pack that freezes recall.py's all-canon-for-a-node query per entity so the prompt is always canon-true and the per-frame cost is zero graph work (§3); constrained decoding whose novel per-entity canon-vocabulary mask makes structurally *and* canonically invalid output impossible to sample, not merely caught (§4); a post-generation validator that is the runtime cousin of the harness G-gates, cite-or-drop against the same alias/hard-line sources (§5); an authored fallback floor that makes the model pure upside and never a dependency (§6); and a failure/reward discipline — the F1–F10 gates as runtime code, with a nightly first-try-pass-rate metric analogous to continuo's scorecard — that hardens the system from every caught miss while the deterministic scaffolding guarantees canon on day one (§7). All of it runs fully offline on consumer hardware, deterministic where reproducibility matters (§8), and lands on artifacts already on disk (§9). The life is real; the lies are structurally impossible.

Generated by harness/site/structure_site.py — the URL path is the repo path. review root