Open-Source LLM Observability and Tracing

Aiursoft Ollama Gateway gives teams controlled access to shared local LLM servers. It connects AI applications and coding assistants to Ollama or OpenAI-compatible backends through a central gateway.

Laminar helps developers inspect AI agent runs, find recurring failures and test fixes. It connects production traces with debugging and evaluation tools.

MASFactory is a Python framework for developers building workflows in which several AI agents collaborate. It combines natural-language design, code authoring and visual editing so teams can define the work and inspect how agents carry it out.

OpenRath is a Python runtime for developers building multi-agent workflows that need an inspectable record of their work. It keeps context and evidence together so engineering and research teams can trace how a result was produced.

LeanCTX helps developers control the information their AI coding tools receive. It runs locally between those tools and project files or command output, while your existing application keeps control of the model and workflow.

Agno is a framework and runtime for developers building customer-facing or internal AI agents. It combines agent development, service hosting and a web interface for managing runs on infrastructure you control.

MCPJam helps MCP server developers check how their tools and apps behave across AI clients before deployment. It combines hands-on debugging with repeatable tests for differences between ChatGPT, Claude, Cursor, and Copilot.

Manifest LLM Gateway connects AI agents and apps to local models, cloud providers and subscriptions through a shared endpoint. It's for builders and teams who want to choose where requests go, handle provider failures and track inference usage.

MindWalk helps developers review coding-agent sessions by showing where an agent explored and changed a repository. It turns Claude Code, Codex and pi logs into a visual replay for comparing the agent's work with the task you assigned.

RocketRide is an AI pipeline builder and runtime for developers creating document processing, search, and agent workflows. Its visual editor sits in VS Code, alongside the application code that uses those workflows.

NVIDIA NeMo Agent Toolkit is an open-source Python library for developers building self-hosted servers that expose AI tools and workflows through MCP or A2A. It also measures and evaluates agent workflows across frameworks. Its introductory NVIDIA NIM workflow uses hosted inference and requires an NVIDIA account and API key.

Workstream is a local developer dashboard for people who track code reviews and project work across several services. It runs on your machine and opens in a browser, with local SQLite storage. The project is open source under Apache License 2.0.

Prompty is a prompt format and developer toolkit for building AI applications. It keeps prompt text, model settings, inputs and tool definitions in a portable Markdown file that developers can share across editors and language runtimes.

Best Claude HUD keeps session and repository information visible for developers working in Claude Code's terminal interface. It helps you check the active model, available context and working tree without leaving the conversation.

Latitude is an AI agent monitoring platform for teams investigating production failures. It links recurring problems to the conversations and tool calls behind them, then gives coding agents the context to work on a fix.

Claude Code Agent Monitor gives developers one place to follow coding agents and inspect their work across Claude Code, Cursor and Codex. It keeps conversations and activity available for review after a session ends.

Lunary is an LLM observability and prompt management platform for teams building AI agents and chatbots. It helps developers investigate production errors and review how people interact with their applications.

Tracely helps developers trace AI agents and turn production failures into tests that catch the same problem before another release. It uses recorded runs as test cases, so teams can build regression coverage from failures their agents have actually encountered.

Agents Observe is a self-hosted dashboard for developers who want to inspect what Claude Code and Codex do during coding sessions. It helps you follow parallel agents and trace the commands or edits behind an unexpected result.

ToolHive runs and manages Model Context Protocol (MCP) servers for developers and platform teams that want control over the tools their AI clients can access. It supports individual desktop use and shared deployments on your own infrastructure.

Mastra is a TypeScript framework for developers building self-hosted AI agents and applications with local models through LM Studio. It supports gpt-oss, Qwen3 and DeepSeek through an OpenAI-compatible model server. The framework can also call OpenAI, Anthropic and Gemini, which send model requests to those remote providers.

Prompt flow is a Python toolkit for developers who want to run and evaluate LLM application workflows in their own development environment. Its chatbot examples call OpenAI or Azure OpenAI remotely using API credentials. Workflow execution and evaluation can run locally, while those model requests leave the machine.

Dingo combines rule checks, LLM judges and agent-based evaluation to assess AI training data, fine-tuning datasets and production systems. It's an open-source Python tool under Apache 2.0 for data engineers, ML practitioners and researchers who need to find data problems and compare model or application quality. Evaluation can run locally or across a Spark cluster.

VoltAgent is a TypeScript AI agent framework for developers building assistants and automated workflows with local models through Ollama. Its VoltOps console can be self-hosted or used as a cloud service. Model execution depends on the provider: Ollama supports local inference, while OpenAI, Anthropic and Google connections send requests to those remote services.

Dify is a visual builder for teams creating AI agents and applications that use their own data and tools. Its canvas makes application logic visible so teams can inspect and test how a task proceeds.

agentacct is a local dashboard for developers who want to see what coding agents did across projects and compare their work with resource use. It brings recorded activity and results into task receipts rather than relying on an agent's claim that a job is finished.

ClawLess is a browser app and SDK for developers who want AI agents to write and execute code in an isolated workspace. Execution runs in your browser without a backend server, but model requests go to Anthropic, OpenAI or Google through API keys, so AI tasks need online services.

AgentRQ lets people assign work to AI agents, follow their progress and answer requests for decisions from a browser, phone or desktop app. Agents run on computers you own; the task dashboard can run on your server or the hosted service.

Ringer coordinates parallel AI coding agents on your machine. It's for developers who want to divide implementation work among workers and judge the results with executable checks.

NVIDIA Object Oriented Agents (NOOA) is a Python framework for developers building AI agents in their own projects. It supports local models through Ollama and vLLM via LiteLLM, with Qwen3 examples for both backends. It also supports Anthropic and OpenAI models, which send requests to those remote providers.

Semantica is a Python toolkit for teams building AI systems that need shared context, traceable decisions and policy checks. It connects facts and decisions to their evidence in queryable knowledge graphs.

CliRelay is an AI gateway for teams that want to manage coding assistants through a shared service on their own server. It builds on CLIProxyAPI and brings subscription accounts and provider API keys under central control.

Opik is an LLM observability and evaluation platform for teams building AI agents and retrieval-based applications. It helps developers inspect failures and compare behavior across development, testing and production.

Agentgateway is an open-source proxy for platform teams connecting applications and agents to models, tools and other agents. It brings routing and access controls together across AI protocols and ordinary service traffic.