# Open-ended co-generation of environments and agents *Technique* · `open-ended-environment-generation` · status: **sourced** Systems that keep generating new environments or tasks at the edge of an agent's ability and train agents on them, from POET's co-evolved terrains to OMNI-EPIC's foundation-model-written task code and SIMA 2 practicing in Genie 3 worlds. | field | value | |---|---| | aliases | POET, Enhanced POET, OMNI-EPIC, SIMA 2 self-improvement, automatic curriculum generation | | category | Reinforcement learning | | year | 2024 | | maturity | Research | | trl | 4 | | runtime | Offline authoring, Server real-time | | determinism impact | Mitigable | | horizon | 3-5 years | | risks | Controllability, Evaluation difficulty, Compute cost | | builds on | [Genie foundation world models](../technique/genie-world-models.md), [Generalist instructable game agents (SIMA)](../technique/generalist-game-agents.md) | | affects count *(computed)* | 5 | ## Summary Systems that keep generating new environments or tasks at the edge of an agent's ability and train agents on them, from POET's co-evolved terrains to OMNI-EPIC's foundation-model-written task code and SIMA 2 practicing in Genie 3 worlds. ## Mechanism POET keeps a population of environment-agent pairs. It mutates environment parameters (for a 2D walker: roughness, gap width, stump height), admits a child environment only if current agents score within a band that is neither too easy nor too hard, optimizes each agent in its paired environment, and periodically tests whether an agent from another pair does better there, transferring it if so. Enhanced POET adds a novelty measure based on how agents rank on an environment and a more flexible environment encoding. OMNI-EPIC replaces parameter mutation with a foundation model that writes environment and reward code for a physics simulator, conditioned on an archive of solved and failed tasks; a model of interestingness rejects near-duplicates, an RL agent trains on each task, and a success check decides whether the task enters the archive. SIMA 2 has Gemini propose tasks and estimate rewards in commercial games and in Genie 3 generated worlds, then trains later agents on its own experience. The shared loop is: generate a task, check it is learnable, train, archive, repeat. ## Limitations - Cost is dominated by training or evaluating an agent on every candidate task, so the loop runs offline and is too slow to respond to a live player directly. - Interestingness and success are judged by foundation models or generated code that can be wrong, and generated reward code can be satisfied by unintended behavior. - SIMA 2 still struggles with very long-horizon tasks, short memory, precise low-level control, and robust visual understanding, which limits the tasks it can practice. - Worlds generated by video world models are approximate and short-horizon rather than rules-exact simulations, so success checks inside them are approximate. ## Relations **affects** - [Scenario objectives](../mechanic/scenario-objectives.md) (Enables new form): Objectives stop being a fixed authored list: a model writes each task and its success check as code, admitted only if it is neither already solved nor out of reach, so a sandbox can keep issuing new challenges. - [Procedural terrain](../mechanic/procedural-terrain.md) (Deepens): Terrain parameters are mutated against measured agent or player performance, as POET did with walker terrain roughness, gaps, and stumps, so maps grow harder along the axes a player has already mastered. - [Adaptive threat scaling](../mechanic/adaptive-threat-scaling.md) (Deepens): Challenge scales with measured learning progress per task family rather than with wealth or time survived, using a band that admits only tasks the current agent or player can partly solve. - [Creature learning](../mechanic/creature-learning.md) (Deepens): A creature or companion can improve through self-proposed practice tasks with model-estimated rewards, the loop SIMA 2 used, rather than only through explicit player reward and punishment. - [Villager requests](../mechanic/villager-requests.md) (Deepens): Requests can be generated beyond fixed templates, with completion conditions written as generated code over simulation state and an archive of past requests used to avoid repeats. **builds on** - [Genie foundation world models](../technique/genie-world-models.md) - [Generalist instructable game agents (SIMA)](../technique/generalist-game-agents.md) ## Backlinks (computed) **applied in:** [Simulation games as agent benchmarks](../opportunity/sims-as-agent-benchmarks.md) ## Claims - POET (2019) co-generated environment challenges and agents, producing behaviors that solved challenges direct optimization and a direct-path curriculum could not, and transfer between environments was essential. *(confidence: high)* [[277](../source/src-wang-2019-poet.md)] - Enhanced POET (2020) added a domain-general measure of meaningful novelty, an efficient transfer heuristic, a more flexible environment encoding, and a measure of ongoing open-ended innovation. *(confidence: high)* [[278](../source/src-wang-2020-enhanced-poet.md)] - OMNI-EPIC (2024) used foundation models to write environment and reward code for tasks judged both learnable and interesting, kept proposing new tasks, and adjusted difficulty to the agent's learning progress. *(confidence: medium)* [[279](../source/src-faldor-2024-omni-epic.md)] - SIMA 2 (November 2025) learned new games through self-directed play from Gemini-proposed tasks and estimated rewards without new human data, improved on previously failed tasks across generations including in Genie 3 worlds, and remained a limited research preview. *(confidence: medium)* [[128](../source/src-deepmind-2025-sima-2-blog.md)] ## Sources - [Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions](../source/src-wang-2019-poet.md) - [Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions](../source/src-wang-2020-enhanced-poet.md) - [OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code](../source/src-faldor-2024-omni-epic.md) - [SIMA 2: An Agent that Plays, Reasons, and Learns With You in Virtual 3D Worlds](../source/src-deepmind-2025-sima-2-blog.md) --- Provenance: created in run `2026-10-24-techniques-rl-design`.