Tools tagged with "LLM tracing"

AgentRQ lets people assign work to AI agents, follow their progress and answer requests for decisions from a browser, phone or desktop app. Agents run on computers you own; the task dashboard can run on your server or the hosted service.

Ringer coordinates parallel AI coding agents on your machine. It's for developers who want to divide implementation work among workers and judge the results with executable checks.

NVIDIA Object Oriented Agents (NOOA) is a Python framework for developers building AI agents in their own projects. It supports local models through Ollama and vLLM via LiteLLM, with Qwen3 examples for both backends. It also supports Anthropic and OpenAI models, which send requests to those remote providers.

XAgent is an experimental self-hosted AI agent for multi-step coding, research and data-analysis tasks. It plans subtasks, revises plans as work progresses and dispatches actions to agents. A command-line interface and browser UI let you guide a run or provide missing information.

RankLLM is a Python toolkit for search researchers and developers who want to compare how AI models reorder retrieved documents by relevance. Its main focus is listwise reranking, where a model considers several candidates together. It supports local inference and optional cloud APIs, and it's open source under the Apache 2.0 license.

Agenta is an AI agent workspace for teams that want to delegate work and manage recurring tasks together. Agents plan their own steps, use connected tools and carry feedback into later work.

Pydantic AI is a Python SDK for developers building agents into their applications. It uses Pydantic validation to give application code structured results and check the arguments agents pass to tools.

LLMKube manages self-hosted LLM inference for teams and homelab owners who need several machines to serve models together. It brings Linux GPU servers and Apple Silicon Macs into the same Kubernetes-managed fleet.

Semantica is a Python toolkit for teams building AI systems that need shared context, traceable decisions and policy checks. It connects facts and decisions to their evidence in queryable knowledge graphs.

Opik is an LLM observability and evaluation platform for teams building AI agents and retrieval-based applications. It helps developers inspect failures and compare behavior across development, testing and production.

Langroid is a Python framework for developers building applications where AI agents divide up a task. Each agent can maintain its own conversation and use documents or tools, then delegate work to other agents.

Agentgateway is an open-source proxy for platform teams connecting applications and agents to models, tools and other agents. It brings routing and access controls together across AI protocols and ordinary service traffic.

Rasa Open Source is a Python framework for developers building text and voice assistants on their own servers. It provides natural-language understanding, dialogue management and integrations with messaging channels.

BrowserSkill connects AI agents to your signed-in Chrome or Microsoft Edge profile for web tasks and website debugging. It's for people who want agents to work with existing accounts and developers investigating browser bugs.

PenguinHarness builds runnable AI agent applications from plain-language requirements on your computer or server. It's for developers who want agents to handle app creation, evaluation, tuning and deployment.

Agent Console helps developers and teams monitor Claude Code and Codex sessions through a local dashboard. It reads the agents' transcripts on your machines so you can compare usage and spot sessions that need attention.

BeeAI Framework is for developers building AI agents and systems where several agents work together. It lets developers enforce rules on agent behavior while retaining the model's ability to reason.

MCP Toolbox for Databases connects coding assistants and AI agents to databases. It's for developers who want assistants to inspect data during development or use defined database actions in applications.

Hermes HUD is a browser dashboard for people who already run the Hermes AI agent and want to inspect its activity and persistent memory. It requires a running agent and reads its local data.

MartinLoop runs around coding agents to keep software jobs within agreed boundaries and check their results. It's for developers and engineering teams who want to delegate longer tasks while retaining limits, recovery information and a record of what happened.

Helicone helps developers inspect and debug LLM applications, including agents, chatbots, and document processing pipelines. It combines request monitoring with a gateway for routing calls across AI providers.

Pi Extensible Workflows is an extension for developers who want repeatable processes involving several AI agents. It coordinates resumable workflows inside the Pi host environment and lets you inspect work while it runs.

Agent Router is an open source AI gateway for teams connecting agents to model providers and MCP tools. It gives applications a shared entry point while platform teams manage access and routing.

Bifrost is an AI gateway for developers and teams building applications that use local LLMs or multiple cloud providers. It gives those applications a shared API for model access and a central place to manage usage.

OrcaReplay is a local AI agent debugger for developers investigating failed coding runs or testing how another model handles the same task. It captures what happened beyond the chat transcript, including failed commands and changed files.

VibeHQ Hub coordinates coding agents for developers who want several agents to work on the same project without conflicting interfaces or unfinished handoffs. It adds shared coordination to Claude Code, Codex and Gemini CLI while preserving each agent's native capabilities.

Mirascope is an open-source development library for building AI applications with local models or cloud providers. Its documented local backends include Ollama, vLLM and LM Studio, plus direct inference through MLX on Apple Silicon Macs. Developers can use a shared model interface while keeping inference on their own hardware or sending requests to OpenAI, Anthropic or Google.

nono is a security runtime for people running terminal AI agents on their own machines or servers. It uses operating system boundaries to restrict an agent's access to files, network services and credentials.

AgentField is a self-hosted AI agent platform for developers building services that need to coordinate tasks under shared access rules. It combines execution, identity and audit records in infrastructure you control.

Aiden is a local AI agent for people who want help carrying out coding, research, and file management tasks. It plans multi-step work and keeps records you can inspect to check what happened and whether the result meets the goal.

WebOperator is an AI browser agent for automating tasks in the Chrome tabs you already use. It retains your existing logins and sessions, and can use a local model or a remote provider.

BabyAGI is a self-hosted Python framework for experienced developers exploring AI agent development. Its local application manages reusable code and provides a browser dashboard. Functions can depend on external services and API keys, including an OpenAI API key.

TruLens helps developers assess AI agents and retrieval-augmented generation (RAG) apps, then trace poor results to the steps that caused them. It connects evaluation scores with execution traces so teams can compare changes in quality, latency and cost.

Claude Code Router is a local gateway for developers using coding agents such as Claude Code, Codex and OpenCode. It keeps provider selection and request routing in one app, while the selected cloud provider processes model requests remotely. Those requests leave your machine.