episode 6 of 12 8:07 12 scenes book chapter 7
What AI shipped
Which AI techniques have shipped in simulation games, and how far. Neural rendering is the one that is normal; talking characters, opponents, and tools players never see have shipped in fewer games, and the few that change a game's state can break its replays.
play · 8:07in the book
the chapter this episode followschapter 7 4,422 words about 19 min 33 records
What shipped
In this chapter: How the dataset counts maturity · The one norm: frames · Assets, places, and motion · Characters that talk · Societies in the lab · Opponents and tools · What writes into the simulation · Prototypes beside the engine · Reproducibility · How it was received · Exercise: classify an AI feature.
games in this episode
12 on screen, in order of appearanceAI Dungeon
A text adventure in which a large language model continues the story after each player input, so actions, dialogue, and world facts the player types are improvised into narration rather than resolved by authored rules.
Microsoft Flight Simulator (2020)
A flight sim that streams a photorealistic model of the whole Earth built from map and photogrammetry data and flies aircraft through live weather with a surface-level aerodynamic model.
Where Winds Meet
An open-world martial-arts RPG whose NPCs answer free-text and voice chat through a live language model, included as an early large-audience deployment of runtime LLM characters.
on screen 3:05
Whispers from the Star
A voice-driven sci-fi narrative game in which the player helps stranded astronaut Stella survive on the planet Gaia through open-ended spoken or typed conversation answered by an AI-generated, voiced performance.
on screen 3:05
Gran Turismo 7
A console racing simulator that combines a detailed vehicle dynamics model and car collection with GT Sophy, a deep reinforcement learning opponent added as a permanent rival in 2023.
on screen 4:22
Gran Turismo
A PlayStation racing game that paired a tuned handling model and 140 production cars with license tests, prize credits, and parts tuning in a Simulation mode career.
on screen 4:40
Townscaper
A town-building toy in which clicking colored blocks onto an irregular grid lets a wave function collapse solver assemble houses, arches, stairways, bridges, and gardens, with no goals or resources.
on screen 5:09
Caves of Qud
A science-fantasy roguelike RPG with simulated factions, liquids, body parts, and procedurally generated histories, built partly with wave function collapse.
on screen 5:34
Infinite Craft
A browser crafting game in which combining any two elements yields a new element whose name and emoji a language model generates on first request and a shared database then fixes for all players.
techniques in this episode
15 on screen, in order of appearanceLLM game master and narrative director
A language model that adjudicates player and NPC intentions, decides outcomes, generates events and quests, and narrates them, either free-form or checked against a typed world state and canon.
Neural rendering in the graphics pipeline
Small neural networks inside the renderer upscale frames, generate extra frames, denoise ray tracing, decode compressed textures and layered materials, and since DLSS 5 re-light the final frame, all without touching simulation state.
ML reconstruction of the planet from aerial imagery
Machine learning classifies buildings, vegetation, and terrain in satellite and aerial imagery and procedural rules rebuild them as 3D scenery, which Microsoft Flight Simulator streams for the whole Earth.
Text- and image-to-3D asset generation
Open 3D generative models (TRELLIS, Hunyuan3D 2.x, Roblox Cube) produce textured meshes from a text prompt or image in seconds to a minute, and inZOI exposes image-to-3D object creation to players.
On-device small language models for characters
Pruned, distilled, and 4-bit quantized language, speech, and embedding models that run on the player's GPU beside rendering to drive character decisions, dialogue, and voice without per-token server cost.
Realtime speech-to-speech and expressive TTS for characters
Streaming speech recognition, speech-native language models, and context-aware text-to-speech that let characters hold spoken, interruptible conversations with roughly 200-250 ms response onset and generated emotional delivery.
Deep reinforcement learning policies for NPC opponents and navigation
Neural control policies trained with deep reinforcement learning inside the game's own simulation drive opponents and NPCs, as with Sony AI's Gran Turismo Sophy, a selectable rival in Gran Turismo 7 since 2023.
Monte Carlo tree search for game AI
A best-first search that grows a game tree from repeated simulated playouts and picks branches with a bandit rule such as UCT, letting AI players plan from a forward model without a hand-tuned evaluation function.
Agent-based automated playtesting and balancing
Studios train imitation, reinforcement learning, and search agents to play builds, estimating level difficulty, pass and churn rates, progression pacing, map coverage, and exploits before players reach them.
Wave Function Collapse and constraint-based tile generation
An example-driven constraint solver that fills a grid by repeatedly collapsing the lowest-entropy cell to one tile and propagating adjacency constraints, shipped in games including Townscaper, Bad North, and Caves of Qud.
Generative agents (memory stream, reflection, planning)
LLM-driven characters that store observations in a natural-language memory stream, retrieve them by recency, importance, and relevance, synthesize reflections, and plan their day top-down.
Learned kinematic motion synthesis
Neural networks generate or compress kinematic character animation, as in Ubisoft's learned motion matching, EA's HyperMotion, and inZOI's motion extraction from uploaded video.
LLM-adjudicated state transitions (LLM as game mechanic)
A language model computes the outcome of an interaction (two elements combined, an action on an object) from a text description of the inputs, with results cached so every player sees the same rule.
Diffusion and autoregressive neural game engines
Action-conditioned generative models trained on recordings of one existing game predict each next frame from recent frames and player input, so the model itself plays the role of the game engine.
on screen 6:12
Genie foundation world models
Google DeepMind's Genie models generate playable worlds from a text or image prompt and render them in real time from player input, with Genie 3 reaching 720p at 24 fps and about a minute of visual memory.
on screen 6:12
shorts from this episode
1 · under a minute each
play · 0:40short 6 0:40 from chapter 7 · episode 6
The one AI technique that's normal
Of the AI techniques in our dataset, exactly one has become normal in games, and it never touches the simulation.
transcript
every number checked against the dataset when the episode was renderedread the transcript, scene by scene · 12 scenes
Cold open
You're a vampire, and you can't go in until someone invites you. So you knock, and you talk, out loud, into a real microphone. Nobody wrote the resident's answer. A language model wrote it, from a personality the developers gave her and the words you just said [1].
When the model is the game
The catalog's earliest game of this kind is AI Dungeon, from 2019. You type what you do, and a language model writes what happens next: GPT-2 at launch, GPT-3 for the premium model a year later [2]. There's no inventory behind the text, and no world state at all. A fact lasts as long as it stays in the model's context window. Pick up a key, play for an hour, and the story forgets you have it. The fix the game gives you is a memory box where you write the key down yourself.
The dataset counts both games as boundary cases. Nothing in either is simulated outside the model's text. They show what a game looks like when the model is the whole game. Most games have a simulation to protect, and AI has been allowed into them in a few specific places.
How shipped is shipped
The dataset holds 45 AI and math techniques, each a dated result that changes at least one game mechanic, and it judges each one by how far it has got in games, which is a harder test than how far it has got in AI. Constrained decoding, which forces a model's answer into a fixed format, is everywhere in software. No source in the dataset documents a shipped game using it, so here it counts as research. By that test, 1 technique has become normal in games, 12 have shipped in at least one game, and the rest are prototypes and papers.
The one that's normal
The one that's normal is neural rendering. Upscaling rebuilds a sharp image from a smaller one, and frame generation predicts whole frames between the ones the game actually drew. By January 2026, more than 250 titles supported multi-frame generation [3], and Microsoft Flight Simulator 2020 supports DLSS frame generation [4].
A generated frame sits between two frames the game computed, and the simulation never learns it exists. The rules, the saves, and the multiplayer code don't change, and the dataset records it as keeping the game reproducible. When it went wrong, with the reveal of DLSS 5, the complaints were about faces and art direction [5].
Places, objects, and motion
Other techniques change what a simulation is drawn with, mostly before you press play. Microsoft Flight Simulator rebuilt the planet from map imagery: machine learning picked out buildings, trees, and terrain, and procedural rules turned them into about one and a half billion buildings [6]. That happened once, offline, so every player sees the same town. In inZOI, players turn a photo into a 3D object [7]. A generated object has to be saved to be seen again, because asking twice gives you a different chair.
This is also where most of the store's AI sits. By July 2025, about 7 percent of Steam's titles disclosed generative AI, and visual assets came up in about 60 percent of those disclosures [8].
Characters that talk
Characters that talk did ship, in a handful of games. In inZOI, a small language model running on your own graphics card lets typed prompts steer what a Zoi thinks and does [7]. It picks from inZOI's own actions, with the game's animations and rules, and replaces the formula that used to choose the next one. Where Winds Meet answers free-text and voice chat with a live model [9], while its quests and combat stay written by hand. Whispers from the Star is one stranded astronaut, voiced live, whose survival depends on what you tell her [10]. And in June 2026, KRAFTON ran a two-week beta of PUBG Ally, a teammate that takes spoken orders through a model on your own machine [11].
Voice got cheaper and the models got smaller. Small models got small enough that a two-billion-parameter one fits in about one and a half gigabytes of video memory, beside the game's own graphics [12] [13]. And voice got cheaper fast. One leading realtime speech service cut its audio prices from 100 and 200 dollars per million tokens in October 2024 to 32 and 64 dollars ten months later [14] [15]. The cost shows up elsewhere. A local model can be seeded and run the same way twice. A voice model or a full language-model character can't: the same words can get a different answer, and a replay has to store what was said.
Opponents
The oldest shipped technique in the set plays opponents. Monte Carlo tree search, from 2006, plans by playing many quick games forward from the current position; Total War: Rome II used it for campaign decisions [16] [17]. Gran Turismo Sophy learned to drive with deep reinforcement learning on many copies of the Gran Turismo simulator, with a reward that encodes racing's sportsmanship rules, and in 2022 it beat four of the world's best Gran Turismo drivers head to head [18]. It reached players as a limited event in February 2023 [19] and became a permanent rival that November [20]. It drives the same tire and suspension model you do. Before it shipped, the strong driver had to be made fair, tunable to your level, and small enough to run every frame, and it reached more cars one content update at a time.
Tools that never reach you
Two more techniques shipped where players mostly can't see them. Agent playtesting runs a build with trained bots before release: a network that learned from logged Candy Crush moves predicted how hard new levels would be for people [21], and EA used search agents to balance careers in The Sims Mobile [22]. Wave-function collapse, a constraint solver that fills a grid tile by tile, builds levels in Bad North and Caves of Qud, and in Townscaper it reruns every time you click [23]. Both give the same answer every time from the same seed.
What writes into the simulation
Only a few shipped techniques change the game's state, the part your save file and a replay depend on. A model's answer can be cached as a rule, which Infinite Craft does, and the next episode starts there. Language-model characters choose actions in inZOI and a few AI-native games. And trained or search-based opponents drive cars and armies. The 13 shipped techniques reach mechanics that make up 23 percent of what catalog games use.
Nothing learned has replaced a game's physics or its world in a catalog game. The models that would replace the engine itself are prototypes. GameNGen ran DOOM at 20 frames a second on one TPU, and its authors say it can't make new games [24]. When Google opened Genie 3 to subscribers, each world allowed sixty seconds of play, a limit DeepMind put down to compute, and a reviewer watched characters walk through walls [25]. Microsoft's Quake II demo ran at about ten frames a second, with health counters that couldn't be trusted [26].
These models keep the game's state as pixels. Health, position, the layout of the map exist only in the frames they remember, so whatever scrolls out of memory can come back different. A simulation game needs the cow in the field on day one to be the same cow on day three hundred.
Replays
Of the 34 techniques that can run while you play, 10 make a game impossible to replay exactly, and 16 can be made reproducible. The fix is the one Infinite Craft uses: ask the model once, store the answer, and treat the stored answer as the rule. A replay reads the record and never asks again.
How it was received
Developers' use of generative AI held steady while their opinion of it fell. In GDC's surveys, 36 percent reported using it in both 2025 and 2026, and the share who think it harms the industry went from 30 to 52 percent [27] [28]. inZOI drew negative reviews after announcing generative features [29], and ARC Raiders re-recorded some synthetic voice lines with actors [30]. AI Dungeon had met the other side of the problem earlier: its 2021 content filter flagged innocent stories, and human moderators read the flagged private ones, which brought a wave of angry reviews [2].
Close
Next episode counts every technique in the set, shipped or not, against every mechanic, and starts with the one shipped game that lets a language model write its rules.