Introducing Structured Outputs in the API

https://openai.com/index/introducing-structured-outputs-in-the-api/

JSON Schema converted to a CFG; invalid tokens masked each step; gpt-4o-2024-08-06 scored 100% on complex schema following vs under 40% for gpt-4-0613. Limits: JSON Schema subset; first request with a new schema adds latency (under 10 s, up to a minute); max_tokens can still break output; values can be wrong.

Grammar-constrained decoding for valid game actions