Efficient Guided Generation for Large Language Models

https://arxiv.org/abs/2307.09702

Outlines: recasts generation as transitions of a finite-state machine with a precomputed index from states to allowed vocabulary tokens, constraining output to regular expressions and context-free grammars with little overhead.

Grammar-constrained decoding for valid game actions