LLM Guardrails, Security and Sandboxes

Opik is an open-source LLM observability and evaluation platform for developers building AI agents and RAG applications. Its Apache 2.0 licensed platform can be self-hosted on your own hardware or servers; Comet also offers a hosted service. Self-hosting lets teams keep their observability deployment in their own environment.

Agno is a Python framework and runtime for developers building customer-facing or internal AI agents. You can run its agent platform locally with Docker, on your own servers or in your cloud. The open-source framework uses the Apache 2.0 license, and the platform keeps sessions, memory, knowledge and traces in your database.

Cua gives AI agents access to computers they can inspect and operate, with tools for desktop automation, local virtual machines, and hosted fleets. It's for developers building agents that work across native apps and browsers, or evaluating how well those agents complete computer tasks. You bring the agent and model.

LiteLLM gives platform teams one place to manage access to LLMs across providers. Its self-hosted AI gateway puts cloud services and internal or locally hosted models behind an OpenAI-compatible API, so applications can change models without changing their integration. Developers can also use its Python SDK directly.

Promptfoo is an open source CLI and library for testing prompts, AI agents, and RAG applications. It runs evaluations locally and helps developers compare model responses while security teams look for weaknesses in the applications built around them. The project is MIT licensed.

APIPark is a self-hosted AI gateway and developer portal for teams that need to manage access to models and business APIs in one place. It connects Ollama alongside cloud providers such as OpenAI, Claude, Gemini and DeepSeek. The gateway runs on your infrastructure; requests to cloud providers still go to those services.

LLM Guard is a Python security toolkit for developers building applications around large language models. It checks prompts and generated responses for risks such as prompt injection, sensitive data exposure and harmful language. The project is archived and no longer maintained, including its associated models on Hugging Face.

PyRIT is an MIT-licensed, open source Python framework for security professionals and engineers assessing generative AI systems. It combines automated attack testing with human-led investigations through CoPyRIT, a web interface served locally. The framework runs locally, but prompts go to the target services you choose; cloud targets and cloud-based scorers process requests outside your machine.

Rebuff is a prompt injection detector for developers building LLM applications that accept untrusted input. It combines checks for suspicious prompts with a record of past attacks and tests for leaked prompt content. The project is archived and no longer maintained.

Vigil is a self-hosted security scanner for developers and researchers who want to check LLM inputs and responses for prompt injection, jailbreak attempts, and other suspicious content. It combines several detection methods and includes attack signatures and datasets, so teams can assess known threats without building every detector themselves. It is experimental alpha software for research and is open source under Apache 2.0.

GoModel is a self-hosted AI gateway for developers and platform teams that want one API for local models and cloud providers. It accepts OpenAI- and Anthropic-compatible requests, so applications can keep their existing SDKs while the gateway handles provider selection and usage controls.

BricksLLM is a self-hosted AI gateway for teams that need to control access and spending across LLM applications. It sits between applications and model providers, applying cost and rate limits to individual API keys. The gateway is open source under the MIT license and runs locally or on your server through Docker, with PostgreSQL and Redis.

Llama Guard is Meta's collection of downloadable AI content moderation models for developers building LLM applications. It checks user inputs and model responses for content that violates safety policies, including text and images. Developers can use the models in their own deployments or access moderation through Meta's hosted Llama API.

Plano, formerly Arch Gateway, is a self-hosted AI gateway for developers building applications with multiple agents or model providers. It puts routing, guardrails and request tracing in a separate service, so each agent doesn't need its own implementation of that infrastructure. It's open source under Apache 2.0.

Portkey Gateway is a self-hosted AI gateway for developers whose apps need to use local models and cloud providers through one OpenAI-compatible API. It routes requests to Ollama, OpenAI, Anthropic, Google Gemini and other backends, with controls for handling failures and checking model inputs and outputs.

Kong Gateway puts API, LLM and MCP traffic behind a shared gateway on your own infrastructure. It's for platform teams that need consistent access controls and traffic policies across services and AI applications. The open-source gateway uses the Apache 2.0 license and runs natively on Kubernetes through Kong's official Ingress Controller.

Giskard is an open-source Python library for testing AI agents, paired with a commercial security platform and assessment service. It's for developers checking agent behavior and security teams assessing deployment risks. The library runs in your own environment under Apache 2.0; Giskard Hub is available hosted or on-premise.

Higress is a self-hosted AI gateway for developers and teams managing model APIs and the tools their AI agents call. It puts LLM traffic and MCP servers behind a shared entry point, with authentication, traffic controls and monitoring. The open-source edition uses the Apache 2.0 license and runs locally in Docker without registration. Alibaba Cloud also offers a fully managed gateway.

Docker MCP Gateway gives developers one local connection point for the tools their AI applications use. It runs MCP servers in isolated Docker containers and shares their configuration across clients such as Claude Code, Cursor and Zed. This reduces repeated setup when several coding assistants need access to the same databases, APIs or development tools.

DeepTeam is a Python framework that runs locally to test chatbots, AI agents, and retrieval-augmented generation (RAG) pipelines for security and safety failures. Built on DeepEval, it's open source under Apache 2.0 and aimed at developers and security teams assessing AI applications before deployment or during ongoing development.

ECC is a free, MIT-licensed open source toolkit that adds repeatable engineering workflows to coding assistants. It runs alongside your chosen agent and works with self-hosted models or cloud providers through that agent's supported endpoints. It's for developers who want planning, testing and review to follow a consistent process across coding sessions.

RamaLama runs and serves AI models on your own hardware using OCI containers. It's aimed at developers who want local chat or a self-hosted inference API with a container workflow they can also use in production. The project uses the MIT license.

Guardrails AI is an open-source Python framework for developers who need to check what goes into an LLM and what comes back. It runs within your application or as a self-hosted service. The framework uses the Apache 2.0 license and helps address risks such as policy violations, hallucinations, and data leakage before outputs reach users.

Garak is an open-source LLM vulnerability scanner for developers and security teams assessing models or dialogue systems. It tests local models as well as cloud services, so you can assess a model running on your own hardware or an application exposed through an API. The Python tool uses the Apache 2.0 license.

Presidio is a self-hosted Python framework for developers and organizations that need to detect and remove personally identifiable information (PII) from text, images and structured data. It can identify names, credit card numbers, social security numbers and other sensitive details, then redact, mask or anonymize them. It's open source under the MIT license.

NeMo Guardrails is an open-source Python toolkit for developers who need control over how an AI assistant responds and uses tools. It runs within your application or as a self-hosted server, including in Docker. The library uses the Apache 2.0 license.