LLMOps: Gateways, Observability and Guardrails

Giskard Hub is an AI agent testing platform available hosted or on-premise for organizations that need testing within their own environment. Developers can also run its open-source Python library locally, but the library's LLM judges and attack generators require a model provider and API key, with integrations for OpenAI and Anthropic. Local execution doesn't make those evaluations offline.

Reverse Skill gives AI coding agents workflows for reverse engineering, authorized penetration testing and CTF challenges. It's for security researchers who want their agent to select an analysis method and work with the tools available on their machine.

KADATH is a self-hosted AI agent runtime for developers and researchers comparing agents against a measurable goal. It runs successive generations of agents and selects stronger approaches, but model calls require OpenAI's cloud API, an API key and internet access.

ai-jail is a sandbox for developers who want to limit the damage an AI coding agent can cause through mistaken commands. It targets accidental changes outside a project, and doesn't provide a security boundary against malware or hostile code.

Agents Observe is a self-hosted dashboard for developers who want to inspect what Claude Code and Codex do during coding sessions. It helps you follow parallel agents and trace the commands or edits behind an unexpected result.

ToolHive runs and manages Model Context Protocol (MCP) servers for developers and platform teams that want control over the tools their AI clients can access. It supports individual desktop use and shared deployments on your own infrastructure.

Mastra is a TypeScript framework for developers building self-hosted AI agents and applications with local models through LM Studio. It supports gpt-oss, Qwen3 and DeepSeek through an OpenAI-compatible model server. The framework can also call OpenAI, Anthropic and Gemini, which send model requests to those remote providers.

DarkMoon runs autonomous penetration tests for security teams, DevSecOps engineers and red teamers. It discovers a target's technology stack, assigns specialist AI agents and tests vulnerabilities across connected systems.

Twinny is an AI coding assistant for developers who want completion and help with their repository inside VS Code, using models on their own hardware or a chosen server.

Prompt flow is a Python toolkit for developers who want to run and evaluate LLM application workflows in their own development environment. Its chatbot examples call OpenAI or Azure OpenAI remotely using API credentials. Workflow execution and evaluation can run locally, while those model requests leave the machine.

Docker MCP Gateway connects AI applications to tools and data through a shared gateway. It's for developers who want coding assistants to use the same server configuration while local tool servers run in isolated Docker containers.

Dingo combines rule checks, LLM judges and agent-based evaluation to assess AI training data, fine-tuning datasets and production systems. It's an open-source Python tool under Apache 2.0 for data engineers, ML practitioners and researchers who need to find data problems and compare model or application quality. Evaluation can run locally or across a Spark cluster.

VoltAgent is a TypeScript AI agent framework for developers building assistants and automated workflows with local models through Ollama. Its VoltOps console can be self-hosted or used as a cloud service. Model execution depends on the provider: Ollama supports local inference, while OpenAI, Anthropic and Google connections send requests to those remote services.

FinGPT is a financial LLM framework for researchers and developers building tools that interpret market news and company information. It includes trained models, financial datasets, and tools for adapting models to specific finance tasks.

Dory is a native macOS app for developers who want containers, Linux machines and coding agent environments on their own Mac. It provides a local runtime managed through both a graphical app and a CLI.

WindsurfAPI, also called DevinAPI, connects coding assistants and other AI clients to models through your own Devin account. The gateway runs on your hardware, but it sends prompts and images to Devin's cloud for inference, so it requires internet access.

Dify is a visual builder for teams creating AI agents and applications that use their own data and tools. Its canvas makes application logic visible so teams can inspect and test how a task proceeds.

Adrian is a runtime security tool for teams whose AI agents can use tools or act on external systems. It checks planned actions against the agent's intended task and can intervene before an unsafe action runs.

Molmo is Ai2's family of vision-language models for working with images and text on your own hardware. It's aimed at developers and researchers who want to train models or evaluate them on visual tasks.

agentacct is a local dashboard for developers who want to see what coding agents did across projects and compare their work with resource use. It brings recorded activity and results into task receipts rather than relying on an agent's claim that a job is finished.

ClawLess is a browser app and SDK for developers who want AI agents to write and execute code in an isolated workspace. Execution runs in your browser without a backend server, but model requests go to Anthropic, OpenAI or Google through API keys, so AI tasks need online services.

EvalScope is a Python evaluation framework for teams comparing AI models or assessing a model service before deployment. It connects scores to the prompts, predictions and reviews behind them so you can examine individual results.

E2B provides isolated execution environments for developers building coding assistants, computer-use agents and background jobs. You supply the model and agent logic; it supplies the machine where the agent works.

Portkey Gateway routes application requests across model providers through an OpenAI-compatible API. It's for developers who need shared routing and failure controls when working with multiple AI backends.

LightEval is a Python toolkit from Hugging Face for developers and researchers comparing LLMs or investigating where their responses fail. It evaluates models already loaded in memory as well as models accessed through a service.

Gauntlet Loop is an AI agent skill for people who want generated websites, writing or software judged against a specific example. It writes a prompt that asks an agent to build, compare and revise work against a reference you choose.

AgentRQ lets people assign work to AI agents, follow their progress and answer requests for decisions from a browser, phone or desktop app. Agents run on computers you own; the task dashboard can run on your server or the hosted service.

Ringer coordinates parallel AI coding agents on your machine. It's for developers who want to divide implementation work among workers and judge the results with executable checks.

Lethe is a local document de-identification app for people who work with confidential contracts, client records or deal documents and want to use AI without sharing the names involved. It replaces names with consistent placeholders, then restores them in AI-generated replies, including summaries and translations.

NVIDIA Object Oriented Agents (NOOA) is a Python framework for developers building AI agents in their own projects. It supports local models through Ollama and vLLM via LiteLLM, with Qwen3 examples for both backends. It also supports Anthropic and OpenAI models, which send requests to those remote providers.

Stakpak is an open source AI agent for developers and DevOps teams managing production apps on their own machines or servers. It works interactively in a terminal or runs continuously to investigate problems, take corrective action and ask for human help.

RankLLM is a Python toolkit for search researchers and developers who want to compare how AI models reorder retrieved documents by relevance. Its main focus is listwise reranking, where a model considers several candidates together. It supports local inference and optional cloud APIs, and it's open source under the Apache 2.0 license.

MCP Gateway & Registry gives platform teams a shared catalog and access gateway for the tools their AI agents and coding assistants use. It addresses scattered server connections, credentials and access policies across an organization.

Pydantic AI is a Python SDK for developers building agents into their applications. It uses Pydantic validation to give application code structured results and check the arguments agents pass to tools.