Diffusion Models Are Real-Time Game Engines

https://arxiv.org/abs/2408.14837

GameNGen: DOOM at 20 fps on one TPU-v5; PSNR 29.43; raters 58%/60% on 1.6 s/3.2 s clips; 64-frame context (~3 s); trained on 70M frames from a PPO agent (50M env steps); 128 TPU-v5e for training; limitations: memory, heuristic errors, no game creation.

Diffusion and autoregressive neural game engines