LAB 06 / CONSTRAINED DECODING

A prompt makes the right token likely.
A schema makes the wrong one impossible.

This continues How an LLM Generates One Token. Every token still gets a logit. Now code outside the model removes the ones the schema doesn't allow before one is picked.

CONSTRAINED DECODING · ONE TOKEN AT A TIME

Why a schema beats asking for JSON

Watch one score get written, then replay three real failures token by token.

Asking for JSON: 4 / 102 malformed · Schema: 0 / 102

01 / THE ANSWER

Same scoring calls, 102 each. Asking for JSON in the prompt: 4 malformed. Sending a schema: 0.

Probabilities are illustrative. The 4 and 0 in 102 are measured on a scoring pipeline I built and tested (Sonnet 5). Tokens are simplified: real tokens can span several characters.

Loading interactive diagram…

Use ← and → to step, space to play or pause.

Why the failures can't happen.

A trailing comma, a second object and a bare quote all broke prompted JSON in testing. Under a schema, each one meets a step where the token that caused it can't be picked.

Illustrative probabilities, measured counts.

The bars are illustrative and the tokens simplified. The 4 and 0 malformed in 102 calls are measured on one pipeline and one model. This is the general mechanism, not any provider's implementation.