Open-Source LLM Observability and Tracing

Bifrost is a self-hosted AI gateway for developers and teams whose applications use multiple model providers. It puts Ollama, custom model deployments, and cloud services behind one OpenAI-compatible API, so applications can switch models without maintaining a separate integration for each provider.

agentacct is a local dashboard for developers who want to review what their coding agents did across projects and compare usage with recorded results. It brings task activity, file changes and test outcomes together with token counts and estimated costs. Claude Code and Codex have the fullest task records; it also supports OpenCode, Kimi Code and Hermes, with coverage varying by capability.

Sim is an AI agent workspace for teams that need to build automations and control how people use them across an organization. You can self-host it on your own server or cloud with Docker or Kubernetes. Its open-source code uses the Apache 2.0 license.

Mastra is a TypeScript framework for developers building AI agents and applications on their own servers or inside existing web apps. Its server runs locally or as a standalone deployment; Mastra Cloud provides a hosted alternative. Model routing connects to providers such as OpenAI, Anthropic, and Gemini, so running the framework locally doesn't keep those model requests on your machine.

Dify is a source-available platform for teams building AI agents and apps on a visual canvas. Its Community Edition runs on your own server with Docker. Dify also offers a hosted cloud service, while Enterprise deployments can run in a VPC or on a self-hosted server. The Community Edition uses a custom Apache 2.0 derivative license.

Opik is an open-source LLM observability and evaluation platform for developers building AI agents and RAG applications. Its Apache 2.0 licensed platform can be self-hosted on your own hardware or servers; Comet also offers a hosted service. Self-hosting lets teams keep their observability deployment in their own environment.

Agno is a Python framework and runtime for developers building customer-facing or internal AI agents. You can run its agent platform locally with Docker, on your own servers or in your cloud. The open-source framework uses the Apache 2.0 license, and the platform keeps sessions, memory, knowledge and traces in your database.

Langfuse is an open source observability and evaluation platform for teams building LLM applications and AI agents. It shows the steps behind a response so developers can investigate failures, slow requests, and cost. Teams can run its MIT-licensed core on their own servers with Docker Compose or Kubernetes, or use Langfuse Cloud as a hosted service.

LiteLLM gives platform teams one place to manage access to LLMs across providers. Its self-hosted AI gateway puts cloud services and internal or locally hosted models behind an OpenAI-compatible API, so applications can change models without changing their integration. Developers can also use its Python SDK directly.

Pezzo is an open-source platform for developers and teams managing prompts and monitoring LLM applications. You can run the full stack locally with Docker Compose, keeping the prompt management and monitoring platform on infrastructure you control. Its source code uses the Apache 2.0 license.

APIPark is a self-hosted AI gateway and developer portal for teams that need to manage access to models and business APIs in one place. It connects Ollama alongside cloud providers such as OpenAI, Claude, Gemini and DeepSeek. The gateway runs on your infrastructure; requests to cloud providers still go to those services.

GoModel is a self-hosted AI gateway for developers and platform teams that want one API for local models and cloud providers. It accepts OpenAI- and Anthropic-compatible requests, so applications can keep their existing SDKs while the gateway handles provider selection and usage controls.

Agent Router is an open source AI gateway for teams whose agents use both model APIs and MCP tools. It runs on a laptop, a dedicated gateway, or Kubernetes, and gives applications one OpenAI-compatible entry point for cloud providers and self-hosted inference. The project uses the Apache 2.0 license.

BricksLLM is a self-hosted AI gateway for teams that need to control access and spending across LLM applications. It sits between applications and model providers, applying cost and rate limits to individual API keys. The gateway is open source under the MIT license and runs locally or on your server through Docker, with PostgreSQL and Redis.

Langtrace is an open-source observability tool for developers who need to debug LLM applications and track their performance. You can run it locally or on your own servers with Docker and Docker Compose. Its traces follow OpenTelemetry standards.

Lunary is a self-hosted LLM observability and prompt management platform for teams building chatbots and AI agents. It brings production traces, user conversations and prompt versions into one place so developers can investigate errors and teams can assess response quality. You can host it in your own infrastructure or use its cloud service.

TensorZero is a self-hosted platform for developers building LLM applications. The project is archived and no longer maintained. It combines a model gateway with tools for inspecting responses, evaluating workflows, and improving prompts using production data and human feedback.

Plano, formerly Arch Gateway, is a self-hosted AI gateway for developers building applications with multiple agents or model providers. It puts routing, guardrails and request tracing in a separate service, so each agent doesn't need its own implementation of that infrastructure. It's open source under Apache 2.0.

Prompt flow is an MIT-licensed, open-source toolkit for developers who build LLM applications and need to test their behavior before deployment. Its development tools run locally, while an optional cloud version in Azure AI supports team collaboration. Feature development has ended.

Laminar is an open-source platform for developers who need to see why an AI agent failed and check whether a fix worked. You can self-host it with Docker or on Kubernetes, including AWS and GCP, or use its managed cloud service. It uses the Apache 2.0 license.

Kong Gateway puts API, LLM and MCP traffic behind a shared gateway on your own infrastructure. It's for platform teams that need consistent access controls and traffic policies across services and AI applications. The open-source gateway uses the Apache 2.0 license and runs natively on Kubernetes through Kong's official Ingress Controller.

vLLM Production Stack is an open source inference stack for teams serving LLMs on their own Kubernetes GPU clusters. It brings request routing and monitoring around vLLM, so applications can move from one serving instance to a distributed deployment without changing their code. It requires a GPU-enabled Kubernetes environment.

OpenLIT is a self-hosted platform for developers who need to understand how their LLM applications and AI agents behave. It connects model calls with tool activity, retrieval and agent steps, so teams can investigate errors and compare cost, latency and output quality across a workflow.

Higress is a self-hosted AI gateway for developers and teams managing model APIs and the tools their AI agents call. It puts LLM traffic and MCP servers behind a shared entry point, with authentication, traffic controls and monitoring. The open-source edition uses the Apache 2.0 license and runs locally in Docker without registration. Alibaba Cloud also offers a fully managed gateway.

OpenLLMetry adds LLM tracing to the OpenTelemetry monitoring stack a team already uses. It's for developers who need to follow model calls alongside database activity and API requests in their AI applications. The extensions run within your application and send standard OpenTelemetry data to your chosen monitoring destination.

Claude Code Router is an open-source local model gateway for developers who use coding agents and want to manage their model providers in one place. It runs on macOS, Windows and Linux, with Docker and a CLI with a browser interface also available. The project uses the MIT license.

Rasa is an AI agent platform for product teams building customer-facing text and voice assistants. Teams can deploy agents on their own infrastructure and choose their models and data arrangements. Its CALM engine combines language model understanding with business flows whose code enforces rules, so an assistant can handle conversational wording while following defined processes.

TruLens is an open-source Python tool for developers who need to find why an AI agent gives a wrong answer or spends too much on a task. It pairs step-level traces with evaluation scores, so you can connect failures to retrieval, reasoning or tool calls. It uses the MIT license and can write results to a database you run.

Latitude is a self-hosted AI agent monitoring platform for teams that need to find production failures and check that fixes work. It connects recurring problems to the sessions that caused them, then can dispatch Claude Code or Cursor with enough context to make a fix and open a pull request.

Evidently is an Apache 2.0 Python library for evaluating, testing and monitoring ML and LLM systems. It works with tabular and text data, including predictive models and RAG applications. You can run one-off evaluations or self-host its open-source monitoring UI. Evidently Cloud is a separate hosted service.

Helicone combines an AI gateway with LLM observability for engineers building agents, chatbots, and document processing apps. You can self-host the open source observability platform under Apache 2.0 or use the hosted service. It also integrates with Ollama for apps that run models locally. Its hosted gateway routes requests to external AI providers, so those requests leave your machine.

DeepEval is a Python framework for testing AI agents, RAG pipelines, and chatbots in your own environment. It's for developers and ML teams who need to compare models or prompts and catch quality regressions before deployment. The open-source framework uses the Apache 2.0 license and fits into Pytest, Python scripts, notebooks, and CI/CD.

BAML is a programming language for developers building AI agents, with typed model calls and local tracing built into the language. It runs standalone on macOS, Linux and Windows, or alongside an existing application. The language is open source under Apache 2.0, and works offline.

Arize Phoenix is a self-hosted platform for developers who need to understand why an AI agent failed and test changes before shipping them. It runs on a laptop, in Docker, or on Kubernetes. Self-hosting keeps traces on your infrastructure; Phoenix Cloud provides a hosted alternative. Phoenix uses the Elastic License 2.0 (ELv2), a source-available license.

MLflow brings agent tracing, LLM evaluation, and model experiment tracking into a platform you can run locally or on your own servers. It's for developers and teams who need to understand failures, compare changes, and monitor AI applications in production. It's open source under Apache 2.0.