Outlines and Guidance: structured LLM output tutorial

Recorded at PyCon DE & PyData on April 23, 2025 and uploaded on October 5, 2025. The library behavior described reflects that session. Emek Gözlüklü explains how language models select their next token and where developers can intervene. The talk starts with input and output embeddings, then follows raw logit scores through softmax and sampling. Examples use the LLAMA-3B-instruct model identified in the session description.

Greedy selection, temperature, top-K and top-P each change how tokens are chosen. Beam search keeps multiple candidate sequences; Gözlüklü describes its higher computational cost and tendency toward less diverse output. He also introduces confidence margins and entropy as uncertainty measures, alongside approaches that use raw logits.

The practical focus is structured generation for a local LLM. Token masking excludes continuations that violate a rule, such as a JSON format or a regular expression. Outlines examples cover classification and parsing into Pydantic models. Gözlüklü says fixed schema text can reduce the tokens the model needs to generate. He describes context-free grammar support through Lark as experimental.

Guidance receives a shorter walkthrough covering selection, chat context and token healing. The latter backs up a token at the prompt boundary to reduce completion bias. The speaker says this feature requires local models because it needs logit access. In the Q&A, he describes using loss over expected continuations to evaluate prompts for a specific use case.