WHAT IT SHOWS
Why the failures can't happen.
A trailing comma, a second object and a bare quote all broke prompted JSON in testing. Under a schema, each one meets a step where the token that caused it can't be picked.
LAB 06 / CONSTRAINED DECODING
This continues How an LLM Generates One Token. Every token still gets a logit. Now code outside the model removes the ones the schema doesn't allow before one is picked.
CONSTRAINED DECODING · ONE TOKEN AT A TIME
Watch one score get written, then replay three real failures token by token.
01 / THE ANSWER
Same scoring calls, 102 each. Asking for JSON in the prompt: 4 malformed. Sending a schema: 0.
Probabilities are illustrative. The 4 and 0 in 102 are measured on a scoring pipeline I built and tested (Sonnet 5). Tokens are simplified: real tokens can span several characters.
Use ← and → to step, space to play or pause.
WHAT IT SHOWS
A trailing comma, a second object and a bare quote all broke prompted JSON in testing. Under a schema, each one meets a step where the token that caused it can't be picked.
THE BOUNDARY
The bars are illustrative and the tokens simplified. The 4 and 0 malformed in 102 calls are measured on one pipeline and one model. This is the general mechanism, not any provider's implementation.