How this was made

Cold open

This is an AI agent adding three mobile games to the dataset behind this series. The records are fine. The checks fail anyway, on a sentence in the paper about vehicle control and building placement, two mechanics none of these games combines. Adding data broke a claim the data hadn't stopped supporting.

What went wrong

The sentence said that building placement and hands-on vehicle control, the flying and driving of flight and racing sims, are both common and almost never meet. To be in that table, a mechanic had to be used by at least eight percent of the catalog. Embodied skill control was used by exactly 14 games, right on the bar. Three new games without it pushed it under eight percent. The pair dropped out of the table, and the sentence about it failed.

The fix changed the rule to a fixed count of games, 14, so adding games can't push a mechanic out anymore. And an assertion about a row that has left a table now fails with a message that says so, instead of the misleading "unknown reference" it gave that day. Before the change went in, the same checks ran against copies of the catalog with the three games added and up to twenty duplicate games on top. Under the old rule, the three games alone broke a sentence. Under the new one, nothing broke.

That agent was one of four sent through the project's task list before any person was invited to contribute, each working only from the written instructions and reporting what slowed it down. Their reports rewrote four of the instructions and several of the checks.

Everything is a build

The whole dataset is a folder of plain-text records. Everything else is built from it on every change: the paper, the website, the book, and the script for this episode. Nothing computed is stored. A mechanic's prevalence, a game's cluster, a bet's priority: each is worked out again on every build, because a stored copy would drift.

Sentences are the hard part, because they interpret numbers. So every number in this series is a token the build fills in, and every sentence that interprets one carries a check. When episode 1 said the five most common parts don't simulate anything, the build checked all five ranks. If the data stops agreeing, the build fails until somebody rewrites the line.

The rules came from failures

Most of the rules exist because something went wrong once. The first catalog was built by agents working in parallel, each taking a slice of games. An outside reviewer found that they'd recorded each game's distinctive parts and skipped the obvious ones: pause, building placement, tech trees. The finding that opened this series, that the most common parts don't simulate anything, wasn't visible in that first data. The gaps were filled, and the analysis was redone.

The same review caught the paper calling some techniques impossible to reproduce when their own records said they could be. Now a sentence about a technique is checked against that technique's record.

A later review removed one game at a time and re-ran everything. Several claims about which games belong together broke, because the samples used to test the clusters were reshuffled whenever the catalog changed. Each game's place in each of the 1000 samples is now fixed by a hash of its name, and a claim that two games belong together has to hold in at least 0.9 of them.

Even the number of clusters follows a written rule: the smallest number whose clusters stay stable across the samples. When the rule was first applied it gave 24, and that was fixed so new games wouldn't reshuffle everything. On today's data the rule would pick 19. The gap sits inside the stated tolerance, so the build doesn't object, and episode 3 only claims the groupings that survive resampling either way.

Runs

Every change to the dataset belongs to a run, and every run leaves a record: who operated it, whether a person or an agent did the work, which model, what it was for, and what it actually did, including what didn't work. Runs are never rewritten. A record names the run that created it and every run that reviewed it, so the history of any fact can be traced back. A single pull request carries one run, changes a limited number of records, never deletes one, and is signed off by the person responsible for it.

The build is deterministic. The same records produce the same files, byte for byte, and the checks prove it on every change by building twice under different settings and comparing. That's how an edition of the book or a version of this script can name the exact version of the data it came from.

Evidence, and who checks it

Every record has a status, and the status is a claim about evidence. Asserted means no source yet. Sourced means it cites one. Verified means someone other than its creator checked it. When sources disagree, the record keeps both. Utopia, the Intellivision city builder, is dated 1982 by Wikipedia and 1981 by Guinness World Records, and its record says so and marks itself contested [1] [2].

Nothing is verified yet. A record can only be verified by a different person from the one who created it, and if both are using agents, a different model. Every run so far has been an agent working under the dataset's maintainer: 42 runs, one operator. Until other people join, nothing can move up. The same goes for scores. Every rubric score in this series is one model's reading of written anchors, and a second rater has to pass a calibration set of 20 mechanics before their scores count.

The video, rendered 2026-10-26, words this scene differently: the dataset has changed since.

What the checks can't see

The checks have limits. The project fetches each source and confirms a quote is really on the page; the first time it ran on agent-written records, it found 26 of 32 quotes, warned on 5, and failed 1, which turned out to be on the page after all. It can't confirm the quote supports the claim. The run log records which model an agent used, but the model is self-declared. The assertions keep sentences in line with the data and say nothing about whether the data matches the world. And the catalog leans toward games documented in English, which no check can see past.

Close

The dataset is open, and anyone can extend it under the same rules: catalogue a game, add quotes, verify a record, rate scores, resolve a prediction. The last three need someone other than the person who built it, which is why the guide for invited contributors starts there. If you disagree with a record, there's a form for that too, and the disagreement goes into the record, the way Utopia's release year did. When those contributions arrive, some sentences in this series will fail the build, the ones saying nothing is verified, every prediction is open, and every run had one operator. The next version will have different numbers in those places, and runs from more than one person behind them. The last chapter of the book goes through each of these rules in more detail, with the records behind them.