LLMOps: Gateways, Observability and Guardrails

Agent Safehouse is a local macOS sandbox for developers who want coding agents to edit a project while limiting their access to unrelated files. The operating system enforces the restrictions, so they don't depend on the agent following written instructions.

OSWorld is a benchmark for researchers and developers testing AI agents that use computers. It measures task completion in real desktop environments, including workflows that cross application boundaries.

Shuru gives AI agents a disposable Linux virtual machine on your Mac. It's for developers who want coding assistants to execute code, install packages and use tools in an environment isolated from the host.

ChainForge helps developers test how reliably LLMs respond before choosing a prompt or model for an application. Its visual flows turn prompt variations into repeatable comparisons rather than isolated chat experiments.

DSPy is a Python framework for developers building language model applications with local backends such as Ollama on a laptop or SGLang on a GPU server. It connects application code to those model servers, which handle inference on your hardware. It also supports cloud providers, so where model requests go depends on the backend you choose.

Oumi helps ML researchers and teams build specialized AI models for their own tasks. It brings data preparation, training, evaluation, and deployment into one platform, with reusable recipes and components you can customize.

Waggle is a local MCP server for developers coordinating AI agents across coding tools. It lets agents pass references to plans, reports and other artifacts while keeping a record of who created them and what recipients read.

Claw Admin is a web dashboard for people managing OpenClaw Gateway and Hermes Agent. It puts agent conversations, behavior controls and activity monitoring in one browser interface.

Hypha is a TypeScript AI agent framework for developers building applications that need controlled tool use and recoverable work. It separates an agent's planning and tool choices from the execution layer that decides which actions may proceed.

Kong Gateway gives platform teams a shared place to manage API, LLM and MCP requests. It helps teams apply access and traffic policies across services and AI applications on infrastructure they control.

Marbor is a self-hosted AI inference gateway for operators serving models across their own GPU fleet. It places requests using live hardware and model state to reduce cold loads and avoid deployments that can't serve the requested model.

ModelScope is an AI model hub with a Python library for developers and researchers who want to run models locally or adapt them to their own data. It covers language, vision, speech, multimodal tasks and scientific computing.

BAML is a programming language for developers building AI agents, with a documented Ollama client for models running locally and tracing that runs on your machine. The OllamaClient integration is documented for the canary toolchain. The standard OpenAI connection sends model requests to OpenAI; installing BAML locally does not make that connection offline.

UltraRAG is a self-hosted framework for researchers and developers building AI systems that answer questions using their own documents. Its Python application and Docker deployment provide a browser workspace for knowledge bases, workflow design and evaluation. Generation can use local models through vLLM or Hugging Face. Embeddings can use local Sentence Transformers or a configured OpenAI service; the OpenAI cloud connection sends embedding inputs to that service.

h5i gives AI agents tools to investigate vulnerabilities in web applications you own or are authorized to test. Teams can review the evidence behind findings and turn confirmed attack flows into security regression checks.

Kibitz helps developers follow several Claude Code or Codex sessions and direct work from one place. It turns agent activity into readable commentary, reducing the need to switch between terminal tabs to see what each session is doing.

Sim is an AI agent workspace for teams building automations and managing their use across an organization. It brings workflow creation, shared data and run monitoring into one place. You can self-host the workspace, but chat uses a Sim-managed service and requires sign-in to obtain a Chat API key.

Reef is self-hosted infrastructure for developers who want AI agents to learn from their interactions. It connects live requests and feedback to changes in model weights or the prompts, rules, and skills that guide an agent.

New API is a self-hosted AI gateway for developers and teams sharing access to multiple model providers. It lets apps and agents keep one connection while administrators change upstream services.

WebVoyager is a Python AI agent that controls a browser on your machine or a Linux server to carry out tasks on live websites. It's a research project for developers studying browser automation. Model decisions depend on OpenAI's cloud API, which receives page observations and requires an API key and internet access; the local browser doesn't provide local model inference.

Director's Console is a self-hosted workspace for filmmakers and AI artists who want to develop shot prompts, generate media through ComfyUI, and organize the results by project.

Agent Flow helps developers inspect how Claude Code and Codex carry out tasks. It turns local agent activity into a visual record for tracing failures, repeated work and slow tool calls.

SkillSpector scans AI agent skills before installation or publication. It helps people reviewing Claude Code, Codex CLI, Gemini CLI and MCP skills spot risks in their instructions, permissions and code.

unleashd, listed as Buddies in its repository, is a local workspace for developers coordinating multiple coding agents. It keeps each agent's identity and memory across conversations, so a team can continue work with earlier context.

LLM Router is a self-hosted AI gateway for teams connecting applications to local LLM servers and cloud providers. It puts a shared interface in front of your models so clients can use the same gateway across backends.

ASSERT is a local-first Python framework for developers testing AI agents and LLM applications against their own requirements. It turns written expectations into evaluations for quality, safety, and regression testing.

rizzo-pii replaces personal information in Italian legal text with placeholders before you share it with an AI service. It's built for law firms, accountants and notaries who need to work with sensitive documents.

Archestra Community Edition is a self-hosted AI platform for teams managing access to company tools and models. It gives coding assistants, internal agents, and staff chat a shared place to enforce policies and track activity.

NVIDIA Dynamo is a self-hosted inference framework for teams serving generative AI models across multiple GPUs or server nodes. It coordinates inference engines so teams can manage capacity, response latency and cached context across a cluster.

RamaLama runs AI models for local chat or a self-hosted REST API. It's for developers who want a container workflow for trying models on their own hardware and serving them in production.

Ollama Grid Search helps developers compare model responses before choosing a model and prompt for a task. It runs experiments across combinations of models, prompts and inference settings, then presents the results for visual review.

Clawk gives coding agents a disposable Linux VM for installing packages, running builds and starting services locally. It's for developers who want an agent to control its working environment while keeping unshared host files and the host keychain outside the guest.

GoModel is a self-hosted AI gateway for developers and platform teams sharing access to local models and cloud providers. It centralizes provider choice and usage policies behind APIs compatible with OpenAI and Anthropic SDKs.

Sandbox Agent lets developers control coding agents from their own applications through a shared API. It's for teams building agent-powered software that needs remote sessions and isolated code execution without a separate integration for each agent.