Tools tagged with "LLM tracing"

Coze Loop helps developers test and monitor AI agents, with tools for investigating failed runs and comparing prompt changes. Its self-hosted edition brings prompt development, evaluation, and execution traces into one browser interface.

MCP Context Forge is a gateway and registry for teams connecting AI clients to multiple tools, agents and services. It gives them one place to discover available services, control access and inspect calls.

Strands Harness SDK is an open source toolkit for developers building AI agents who need control over their behavior. It handles the agent loop while letting you choose its components.

Arize Phoenix helps developers investigate failures in AI agents and LLM applications and compare proposed fixes. It connects recorded application activity with evaluations and repeatable experiments.

Apache Burr is a self-hosted Python framework for developers building AI agents, chatbots, and other applications with branching decisions. It models application logic as a state machine, so developers can see which action ran, what changed, and why execution took a particular path. Its library and monitoring UI run on your own infrastructure, and its open-source code uses the Apache 2.0 license.

AgentGUI gives researchers and developers a visual workspace for supervising long-running AI agents. Its pixel-art office represents tasks as desks, so you can follow several jobs and intervene while they run.

Ragpi is a self-hosted AI assistant API for teams that want to answer technical questions using their own project material. It lets developers add documentation-based support to a product, website or community.

Claude DevTools is a local debugger for developers who want to understand what happened during a Claude Code session. It reads saved logs from terminal and IDE sessions without modifying Claude Code.

Torrix is a self-hosted LLM monitoring tool for teams investigating AI spending and failed requests. It keeps request records in your infrastructure and brings usage analytics and individual traces into one dashboard.

Atlas is a desktop workspace for developers who use coding agents across projects and want to keep the context behind their code changes. It brings agent conversations, code review and project history into one window.

Discourse AI adds community assistants, search and moderation to a Discourse forum you can host on your own server. It's for forum operators who want AI features inside their community site. Forum hosting and AI processing are separate choices: the plugin supports a choice of providers, and its AI API services have a self-hosting option.

DeepEval is an LLM evaluation framework for developers testing AI agents, RAG pipelines and chatbots. It helps compare prompts and models and catch quality regressions within Python tests or CI/CD.

OpenLIT helps developers investigate failures and assess the quality of existing AI applications and coding agents. It connects model requests with the tools, retrieval and agent steps involved in producing an answer.

Langfuse helps teams debug LLM applications and AI agents, then test changes against production examples. It brings traces, evaluations and prompt revisions into one place.

Genkit is Google's framework for developers building AI agents and chatbots into web and mobile apps. It brings model connections, application workflows and testing tools into one framework.

Vibe-Trading is an installed Python research app with terminal and web interfaces for traders and analysts testing strategies or reviewing trades. The workspace runs on your machine, but its AI requests go to OpenRouter by default. Research also fetches external market data and web content.

Vanna is a Python framework for adding database chat to analytics applications and internal tools. The project is archived and no longer maintained.

Text Generation Inference (TGI) serves LLMs on your own hardware for developers building chat apps and text generation services. The project is archived and no longer receives updates.

Flowise is a visual builder for developers and teams creating AI agents and chatbots from connected workflow blocks. The project is archived and no longer maintained.

TensorZero is a self-hosted platform for developers who need to compare and improve models in LLM applications. The project is archived and no longer maintained.

Evalite is a TypeScript evaluation tool for developers testing LLM-powered apps. It pairs code-based tests with a local interface for reviewing how an app responds.

Chainlit's Python framework lets developers self-host conversational AI applications and provide a browser chat interface around their own application logic. Its OpenAI assistant example uses the OpenAI Python SDK’s standard connection, sending model requests to OpenAI’s cloud service. Self-hosting the chat server and supplying model inference are separate choices.