Tools tagged with "LLM tracing"

Ralph TUI coordinates AI coding agents on your machine or servers as they work through a task backlog. It's for developers who want autonomous execution with progress and agent output visible in a terminal.

Langflow is a visual builder for developers creating AI agents and applications that answer questions using connected data. It lets you assemble workflows visually and customize their components in Python.

MindWalk helps developers review coding-agent sessions by showing where an agent explored and changed a repository. It turns Claude Code, Codex and pi logs into a visual replay for comparing the agent's work with the task you assigned.

RocketRide is an AI pipeline builder and runtime for developers creating document processing, search, and agent workflows. Its visual editor sits in VS Code, alongside the application code that uses those workflows.

NVIDIA NeMo Agent Toolkit is an open-source Python library for developers building self-hosted servers that expose AI tools and workflows through MCP or A2A. It also measures and evaluates agent workflows across frameworks. Its introductory NVIDIA NIM workflow uses hosted inference and requires an NVIDIA account and API key.

AI Documentation Generator analyzes repositories and writes documentation for developers who need to understand an unfamiliar codebase or prepare it for new contributors. It's open source under the MIT license and runs as a Python application, in Docker, or through a Claude Code plugin.

Oh My Subagents coordinates Codex and Claude agent teams for developers and researchers handling coding, investigations or independent code review. It saves delegated work and progress so teams can resume interrupted tasks.

Prompty is a prompt format and developer toolkit for building AI applications. It keeps prompt text, model settings, inputs and tool definitions in a portable Markdown file that developers can share across editors and language runtimes.

Latitude is an AI agent monitoring platform for teams investigating production failures. It links recurring problems to the conversations and tool calls behind them, then gives coding agents the context to work on a fix.

Albert coordinates Claude Code agents for developers who want to hand off a coding goal that spans several tasks. It plans the work and continues through successive rounds until it reaches the goal or a requested review checkpoint.

Claude Code Agent Monitor gives developers one place to follow coding agents and inspect their work across Claude Code, Cursor and Codex. It keeps conversations and activity available for review after a session ends.

Claw Orchestrator gives developers one interface for multiple coding agents and checks their results independently. You define acceptance criteria, and the runtime executes the checks and keeps the evidence on disk.

Lunary is an LLM observability and prompt management platform for teams building AI agents and chatbots. It helps developers investigate production errors and review how people interact with their applications.

Tracely helps developers trace AI agents and turn production failures into tests that catch the same problem before another release. It uses recorded runs as test cases, so teams can build regression coverage from failures their agents have actually encountered.

Agentlas OS is an open-source AI agent framework for projects that need specialist teams and shared project context. It runs on macOS, Windows and Linux, with local document indexing, semantic search and memory retrieval. You supply the model. Using a cloud model sends requests to that provider and may incur charges.

ProofShot is a local browser verification CLI for developers reviewing UI work produced by AI coding agents. It saves browser interactions and errors as review artifacts so you can inspect what the agent tested.

OpenClaw.NET is a self-hosted AI agent runtime for a personal assistant or a shared team gateway. It's an independent .NET implementation inspired by OpenClaw, with compatibility for supported parts of that ecosystem.

KADATH is a self-hosted AI agent runtime for developers and researchers comparing agents against a measurable goal. It runs successive generations of agents and selects stronger approaches, but model calls require OpenAI's cloud API, an API key and internet access.

Agents Observe is a self-hosted dashboard for developers who want to inspect what Claude Code and Codex do during coding sessions. It helps you follow parallel agents and trace the commands or edits behind an unexpected result.

ToolHive runs and manages Model Context Protocol (MCP) servers for developers and platform teams that want control over the tools their AI clients can access. It supports individual desktop use and shared deployments on your own infrastructure.

Mastra is a TypeScript framework for developers building self-hosted AI agents and applications with local models through LM Studio. It supports gpt-oss, Qwen3 and DeepSeek through an OpenAI-compatible model server. The framework can also call OpenAI, Anthropic and Gemini, which send model requests to those remote providers.

SoloMD is a Markdown editor for writers, researchers and developers who want plain text notes with AI assistance. It combines a Typora-style writing interface with tools for searching a note collection and letting agents work on it.

Prompt flow is a Python toolkit for developers who want to run and evaluate LLM application workflows in their own development environment. Its chatbot examples call OpenAI or Azure OpenAI remotely using API credentials. Workflow execution and evaluation can run locally, while those model requests leave the machine.

Auto Browser lets AI agents operate a live browser that a human can take over when a login or fragile page needs attention. It's built for internal dashboards, account workflows and operator-assisted browser QA.

VoltAgent is a TypeScript AI agent framework for developers building assistants and automated workflows with local models through Ollama. Its VoltOps console can be self-hosted or used as a cloud service. Model execution depends on the provider: Ollama supports local inference, while OpenAI, Anthropic and Google connections send requests to those remote services.

WhipCode is an open-source coding agent for developers tackling jobs that exceed a single model context window. It lets agents divide the work while you follow their progress and send further instructions.

Maat Agent is a coding assistant for developers who want testable evidence before accepting AI edits. It runs in a terminal or an Electron desktop app on macOS, Windows and Linux, and checks the actual project files against acceptance criteria set before work begins. Failed checks send their output back to the model for another attempt; if the work still fails, the agent reports failure.

Devo is a coding agent for developers who want to choose their model provider and run the agent on their own machine. Its desktop app and terminal interface handle coding and other engineering tasks that require multiple steps.

Open Science Desktop brings research agents, notebooks and project files into one workbench for researchers who want to inspect the work behind an AI-generated result. Agents can carry a research topic through literature review, experiments, analysis and manuscript writing.

agentacct is a local dashboard for developers who want to see what coding agents did across projects and compare their work with resource use. It brings recorded activity and results into task receipts rather than relying on an agent's claim that a job is finished.

ClawLess is a browser app and SDK for developers who want AI agents to write and execute code in an isolated workspace. Execution runs in your browser without a backend server, but model requests go to Anthropic, OpenAI or Google through API keys, so AI tasks need online services.

OpenCodeReview is a locally installed AI code review CLI for developers who want findings tied to specific code lines. It sends code to a configured LLM endpoint through OpenAI- or Anthropic-compatible APIs. The local CLI handles the review process; model inference depends on that endpoint.

EvalScope is a Python evaluation framework for teams comparing AI models or assessing a model service before deployment. It connects scores to the prompts, predictions and reviews behind them so you can examine individual results.

Weft is a language and framework for developers who want to run AI workflow orchestration on their own machine. Its runtime requires Docker, and its graph editor works in VS Code and compatible forks. Self-hosting the workflow doesn't make its model calls local: the included WhatsApp support bot uses OpenRouter for inference.