constrained-decoding technique sourced
Grammar-constrained decoding for valid game actions
Decoding that masks every token which cannot extend a valid string of a schema, regular expression, or grammar, so a model's output always parses as a legal game action or data record.
mechanism
The output format (a JSON Schema, regular expression, or context-free grammar) is compiled into an automaton. At each decoding step the engine computes which vocabulary tokens keep the partial output a valid prefix and sets the logits of all others to minus infinity before sampling. Outlines precomputes an index from finite-state-machine states to allowed tokens; XGrammar handles full CFGs by prechecking context-independent tokens and checking context-dependent ones with a persistent stack, overlapping grammar work with GPU execution; hosted APIs convert schemas to CFGs and cache them; llama.cpp accepts GBNF grammars for local models. For games, the schema is built from current state: an enum of legal verbs, target ids that exist, and numeric fields with bounds, so every output maps to an action the engine can execute. A simpler variant scores a fixed list of candidate actions and returns the chosen index.
limitations
- Validity is not correctness: a well-formed action can still be the wrong action or reference stale state, so engines must re-check preconditions.
- Format constraints can lower reasoning quality; a common workaround is free-form reasoning followed by a constrained final answer, which costs tokens.
- Schema coverage varies: hosted APIs support a JSON Schema subset, and llama.cpp silently skips unsupported features such as nested $ref and uniqueItems and slows down with many optional items.
- First use of a new schema adds compile latency (OpenAI reports under 10 seconds, up to a minute), so per-turn dynamic schemas need caching.
- Output can still fail validation if generation hits a token limit.
relations
5 stored edgesDilemma cards
Card prompts, two to four options, and per-meter effect values can be generated in the game's card schema with effect sizes bounded by the grammar, leaving designers to curate rather than write.
Generative content on demand
Generated items, recipes, or characters can be written straight into typed game state (enum types, bounded stats, existing ids) instead of shown as text or parsed with retries, because invalid records cannot be emitted.
Natural-language interaction
Free player text can be mapped onto the closed set of verbs and target ids legal in the current state, so conversation reliably triggers real game actions rather than only replies.
Player legislation
Laws proposed in plain language can be compiled into the world's structured condition-and-effect rule language by grammar-constrained generation, so players need not learn the rule editor.
Utility action selection
An LLM can be dropped into the action-selection slot by constraining it to choose among the same enumerated actions the utility scorer would rank, keeping the engine's action set and preconditions intact.
backlinks
computed from other recordsAgent ambassadors between player worlds, Bottleneck explainer for production networks, Why-query over the simulation log, Content-elastic event systems, Drama-aware storyteller, Event-sourced simulation runtime, Generated inspection shifts, Generative decoration catalogs, Intent-level orders, Mechanic modding by description, Local-first AI simulation games, Natural-language legislation, Negotiating faction AI, Placement that tells stories, Salience-gated language agents, Specification programming games
claims
6 with sourcesOpenAI's Structured Outputs (August 2024) reached 100% on a complex JSON-schema-following eval with gpt-4o-2024-08-06, versus under 40% for gpt-4-0613 without constraints.
XGrammar reports up to 100x faster grammar-constrained generation than prior engines and near-zero end-to-end overhead when integrated with an inference server.
Outlines showed that regex and CFG guidance can be implemented as finite-state-machine transitions with a precomputed vocabulary index and little per-token overhead.
Strict format restrictions such as JSON or XML significantly reduced LLM reasoning accuracy compared with free-form answers, with stricter formats causing larger drops.
Concordia's language-model interface includes a sample_choice call that scores a list of candidate responses and returns an index, raising an error after repeated invalid attempts, which is how its agents pick from closed option sets.
No source found documents constrained decoding in a specific shipped game; adoption in games is inferred from engine support (llama.cpp GBNF, hosted APIs) rather than observed.