Guidance
A Python library that constrains LLM output with grammars and JSON schemas, supporting local Transformers and llama.cpp backends plus OpenAI.
Guidance is an MIT-licensed, open source Python library for developers who need language model output to follow a defined format. It works with local LLM backends including Transformers and llama.cpp, as well as OpenAI's cloud service. It's for application code.
Its main distinction is control over generation itself. Developers can restrict responses to a fixed set of choices, match a regular expression, or require structured JSON through a JSON schema or Pydantic model. With a backend that fully supports Guidance, context-free grammars can enforce more complex output syntax, such as nested markup. These constraints govern format, not the factual accuracy of the answer.
Guidance combines generation with ordinary Python logic, including loops, conditions and tool use. Reusable functions let developers compose larger workflows and grammars, while named captures make generated text available to the surrounding application. A Jupyter notebook widget displays output and distinguishes model-generated text from text supplied by the grammar.
When a grammar determines part of the output in advance, Guidance can insert that text without asking the model to generate it. This reduces model computation and GPU use, particularly for structured output with predictable punctuation or tags.
Local backends run the model on your hardware; the OpenAI backend uses a cloud service. Grammar validation and tests with the Mock model work offline without model API calls.
