NeMo Guardrails

NeMo Guardrails is a Python toolkit for developers building AI assistants that need rules for what they say and which actions they take. It adds checks around conversations, document-based answers and tool use.

• Conversation control: Define topic boundaries and use Colang to specify dialogue paths for tasks such as authentication or customer support.
• Content filtering: Reject or edit user messages, retrieved document passages and generated answers, including masking sensitive information before it reaches the model or user.
• Safety and accuracy checks: Apply built-in moderation, fact-checking, hallucination detection, and checks for jailbreaks and prompt injection, with NVIDIA safety models and external checking APIs as additional options.
• Tool and chain checks: Check the inputs and outputs of assistant actions, and optionally connect guardrails to LangChain chains or call chains within a guarded conversation.
• Evaluation: Test topic restrictions, factual checks, moderation and hallucination checks with the included evaluation tool.

The guardrails layer runs in your Python application and works with models including LLaMa-2, Falcon, Vicuna and OpenAI GPT models. Using an external model or checking API sends the relevant requests to that service. The library also sends anonymous usage telemetry to NVIDIA about deployment patterns and safety feature use; per-request tracing is separate and can go to your own observability backend.