Tools tagged with "LLM tracing"

Bifrost is a self-hosted AI gateway for developers and teams whose applications use multiple model providers. It puts Ollama, custom model deployments, and cloud services behind one OpenAI-compatible API, so applications can switch models without maintaining a separate integration for each provider.

agentacct is a local dashboard for developers who want to review what their coding agents did across projects and compare usage with recorded results. It brings task activity, file changes and test outcomes together with token counts and estimated costs. Claude Code and Codex have the fullest task records; it also supports OpenCode, Kimi Code and Hermes, with coverage varying by capability.

Pipecat is a Python framework for developers building conversational AI agents that handle speech, video, text, and images. You can run it on your own machine or servers, wherever Python runs. It's open source under the BSD 2-Clause license.

Sim is an AI agent workspace for teams that need to build automations and control how people use them across an organization. You can self-host it on your own server or cloud with Docker or Kubernetes. Its open-source code uses the Apache 2.0 license.

Mastra is a TypeScript framework for developers building AI agents and applications on their own servers or inside existing web apps. Its server runs locally or as a standalone deployment; Mastra Cloud provides a hosted alternative. Model routing connects to providers such as OpenAI, Anthropic, and Gemini, so running the framework locally doesn't keep those model requests on your machine.

Langflow is a visual builder for developers creating AI agents and retrieval-augmented generation (RAG) applications. You can run it locally or on your own server, with Docker support and desktop apps for Windows and macOS. The open-source software uses the MIT license. A hosted cloud offering provides a separate deployment option.

LightRAG combines knowledge graphs with vector search to answer questions across a document collection. It's a self-hosted Python framework for developers building document assistants, particularly where answers depend on relationships between facts in different files, such as legal or financial material.

Opik is an open-source LLM observability and evaluation platform for developers building AI agents and RAG applications. Its Apache 2.0 licensed platform can be self-hosted on your own hardware or servers; Comet also offers a hosted service. Self-hosting lets teams keep their observability deployment in their own environment.

Agno is a Python framework and runtime for developers building customer-facing or internal AI agents. You can run its agent platform locally with Docker, on your own servers or in your cloud. The open-source framework uses the Apache 2.0 license, and the platform keeps sessions, memory, knowledge and traces in your database.

Langfuse is an open source observability and evaluation platform for teams building LLM applications and AI agents. It shows the steps behind a response so developers can investigate failures, slow requests, and cost. Teams can run its MIT-licensed core on their own servers with Docker Compose or Kubernetes, or use Langfuse Cloud as a hosted service.

Pydantic AI is a Python SDK for developers building AI agents into their own applications. Its main draw is Pydantic validation across agent tools and results, so an agent can return structured data that application code can check and use. The SDK is MIT licensed.

GoModel is a self-hosted AI gateway for developers and platform teams that want one API for local models and cloud providers. It accepts OpenAI- and Anthropic-compatible requests, so applications can keep their existing SDKs while the gateway handles provider selection and usage controls.

Agent Router is an open source AI gateway for teams whose agents use both model APIs and MCP tools. It runs on a laptop, a dedicated gateway, or Kubernetes, and gives applications one OpenAI-compatible entry point for cloud providers and self-hosted inference. The project uses the Apache 2.0 license.

LLMKube is a free, open-source Kubernetes operator for teams and homelab owners running local LLM inference across their own hardware. It manages Linux GPU servers and Apple Silicon Macs together, so a mixed fleet can serve models through the same platform. It uses the Apache 2.0 license.

Vanna is a self-hosted Python framework for building AI agents that answer database questions in plain language. It's for teams adding chat to analytics products or internal data tools, especially when each user needs different access to the data. The project is archived and no longer maintained. Its open-source code uses the MIT license.

Flowise is a visual builder for AI agents and chatbots that can run locally or on your own server, including through Docker. It's for developers and teams building LLM applications with connected workflow blocks. The project is archived and no longer maintained.

Discourse AI is the official AI plugin bundled with Discourse. Forum administrators can enable its features independently, including an AI bot, semantic search, topic and chat summaries, spam detection and writing assistance. It runs inside a Discourse community, which you can host on your own server.

Langtrace is an open-source observability tool for developers who need to debug LLM applications and track their performance. You can run it locally or on your own servers with Docker and Docker Compose. Its traces follow OpenTelemetry standards.

Lunary is a self-hosted LLM observability and prompt management platform for teams building chatbots and AI agents. It brings production traces, user conversations and prompt versions into one place so developers can investigate errors and teams can assess response quality. You can host it in your own infrastructure or use its cloud service.

TensorZero is a self-hosted platform for developers building LLM applications. The project is archived and no longer maintained. It combines a model gateway with tools for inspecting responses, evaluating workflows, and improving prompts using production data and human feedback.

Plano, formerly Arch Gateway, is a self-hosted AI gateway for developers building applications with multiple agents or model providers. It puts routing, guardrails and request tracing in a separate service, so each agent doesn't need its own implementation of that infrastructure. It's open source under Apache 2.0.

Prompt flow is an MIT-licensed, open-source toolkit for developers who build LLM applications and need to test their behavior before deployment. Its development tools run locally, while an optional cloud version in Azure AI supports team collaboration. Feature development has ended.

Text Generation Inference (TGI) is a self-hosted LLM server for developers and teams serving models through an API on their own hardware. The repository is archived; its README describes maintenance mode and recommends other inference engines for new deployments. Its focus is handling concurrent generation requests and making efficient use of GPU memory.

Laminar is an open-source platform for developers who need to see why an AI agent failed and check whether a fix worked. You can self-host it with Docker or on Kubernetes, including AWS and GCP, or use its managed cloud service. It uses the Apache 2.0 license.

OpenLIT is a self-hosted platform for developers who need to understand how their LLM applications and AI agents behave. It connects model calls with tool activity, retrieval and agent steps, so teams can investigate errors and compare cost, latency and output quality across a workflow.

OpenLLMetry adds LLM tracing to the OpenTelemetry monitoring stack a team already uses. It's for developers who need to follow model calls alongside database activity and API requests in their AI applications. The extensions run within your application and send standard OpenTelemetry data to your chosen monitoring destination.

Claude Code Router is an open-source local model gateway for developers who use coding agents and want to manage their model providers in one place. It runs on macOS, Windows and Linux, with Docker and a CLI with a browser interface also available. The project uses the MIT license.

Rasa is an AI agent platform for product teams building customer-facing text and voice assistants. Teams can deploy agents on their own infrastructure and choose their models and data arrangements. Its CALM engine combines language model understanding with business flows whose code enforces rules, so an assistant can handle conversational wording while following defined processes.

TruLens is an open-source Python tool for developers who need to find why an AI agent gives a wrong answer or spends too much on a task. It pairs step-level traces with evaluation scores, so you can connect failures to retrieval, reasoning or tool calls. It uses the MIT license and can write results to a database you run.

Latitude is a self-hosted AI agent monitoring platform for teams that need to find production failures and check that fixes work. It connects recurring problems to the sessions that caused them, then can dispatch Claude Code or Cursor with enough context to make a fix and open a pull request.

Wren AI is a self-hosted data agent for teams and agent builders who need answers based on agreed business definitions. It turns plain-language questions into SQL and interactive dashboards, using the same definitions when someone asks directly or through Claude, ChatGPT or Gemini over MCP.

Helicone combines an AI gateway with LLM observability for engineers building agents, chatbots, and document processing apps. You can self-host the open source observability platform under Apache 2.0 or use the hosted service. It also integrates with Ollama for apps that run models locally. Its hosted gateway routes requests to external AI providers, so those requests leave your machine.

Stagehand is an open-source browser automation SDK for developers building AI agents that interact with websites and extract structured data. It can run with Chrome on your own machine or use Browserbase's cloud browsers. Local runs require Chrome. The project uses the MIT license and supports TypeScript, Python, and Go.

Text Embeddings Inference is a self-hosted server for developers who need text embeddings for search and retrieval applications. It serves models through a REST API on your own hardware and can run offline once model weights are downloaded. The Rust project is open source under Apache 2.0.

DeepEval is a Python framework for testing AI agents, RAG pipelines, and chatbots in your own environment. It's for developers and ML teams who need to compare models or prompts and catch quality regressions before deployment. The open-source framework uses the Apache 2.0 license and fits into Pytest, Python scripts, notebooks, and CI/CD.

FastGPT is a self-hosted AI agent builder for teams that want assistants to answer questions using company documents and carry out business workflows. Its visual editor connects model calls, knowledge retrieval and tools into applications for customer support, internal knowledge search and document review. You can run the platform on your own server through Docker or use the vendor's hosted service.