Self-Hosted LLM Gateways and Routers

FreeLLMAPI gives developers one API for experimenting with cloud models and local backends. The router runs locally, but cloud inference still sends requests to the providers you enable.

Free Claude Code lets developers use coding agents with local models or a choice of cloud providers. It connects existing clients to a shared model catalog rather than requiring a separate coding workflow.

OmniRoute is an AI gateway for developers who want their coding assistants and other AI clients to share one set of provider connections. It routes requests across models and accounts when a service fails, reaches a rate limit or runs out of quota.

LangWatch helps AI teams test agent behavior before release and investigate production failures. Teams can reproduce a failed conversation and check how an agent handles it again.

CC Switch is a desktop manager for developers who use multiple AI coding assistants or API providers. It brings their settings into one interface and preserves existing plugins, hooks and other custom settings when switching providers.

AIClient2API is a self-hosted AI API proxy for developers connecting chat apps and coding assistants to multiple model providers. It exposes services such as Antigravity, Codex, Grok, and Kiro through interfaces those apps already support.

Higress is an AI gateway for teams building AI apps and agents that need shared controls across model requests and tool calls. It brings model APIs, MCP services and Kubernetes inference workloads behind one entry point.

Future AGI helps teams test AI agents, investigate production failures and compare prompt changes. It brings evaluation and monitoring into the same platform so teams can connect poor responses to the steps that caused them.

OpenZiti LLM Gateway is a self-hosted API proxy for developers and teams connecting AI applications to inference servers across private networks and cloud services. Its optional zrok integration uses OpenZiti for identity-based access and end-to-end encryption, so clients and backends can connect across network boundaries without opening inbound ports.

LiteLLM is an AI gateway for platform teams managing model access across applications and providers. It puts cloud and self-hosted models behind a shared interface so developers can switch providers without rewriting their integration.

AIBrix provides infrastructure components for teams serving large language models on Kubernetes. It's aimed at organizations that need to manage inference capacity, request routing and models across a cluster.

Paritok is a self-hosted gateway for developers whose coding agents send large file reads, tool output and conversation histories to an LLM. It reduces that input before the request reaches the upstream model.

opencodex is a local AI gateway for developers who want to choose which models power their coding assistants. It connects Codex, Claude Code, Claude Desktop and Grok Build to alternative providers while keeping each tool's native interface.

Kong Gateway gives platform teams a shared place to manage API, LLM and MCP requests. It helps teams apply access and traffic policies across services and AI applications on infrastructure they control.

Marbor is a self-hosted AI inference gateway for operators serving models across their own GPU fleet. It places requests using live hardware and model state to reduce cold loads and avoid deployments that can't serve the requested model.

New API is a self-hosted AI gateway for developers and teams sharing access to multiple model providers. It lets apps and agents keep one connection while administrators change upstream services.

LLM Router is a self-hosted AI gateway for teams connecting applications to local LLM servers and cloud providers. It puts a shared interface in front of your models so clients can use the same gateway across backends.

Archestra Community Edition is a self-hosted AI platform for teams managing access to company tools and models. It gives coding assistants, internal agents, and staff chat a shared place to enforce policies and track activity.

NVIDIA Dynamo is a self-hosted inference framework for teams serving generative AI models across multiple GPUs or server nodes. It coordinates inference engines so teams can manage capacity, response latency and cached context across a cluster.

GoModel is a self-hosted AI gateway for developers and platform teams sharing access to local models and cloud providers. It centralizes provider choice and usage policies behind APIs compatible with OpenAI and Anthropic SDKs.

DEEIX Chat is a self-hosted AI workspace for individuals and teams that want one place to use multiple model providers. It combines everyday chat with controls for organizations managing shared model access and usage.

ServiceStack AI Server is a self-hosted gateway for developers and teams whose applications use several AI providers. It runs in a Docker container and gives apps one API integration for local models, cloud LLMs and media processing. Teams can change providers without changing their client integrations.

MLflow helps developers and teams inspect AI applications, compare changes and track model experiments. It brings agent debugging and evaluation into a platform you can run locally or on your own servers.

Plano is a self-hosted AI gateway for developers building applications with multiple agents. It handles shared routing, orchestration and monitoring outside the agents, so developers don't need to build those functions into each one.

llm-fw inspects model requests for developers and teams using AI tools, SDKs and agents. It blocks suspected prompt injection and removes outgoing secrets before forwarding approved traffic to the model endpoint.

CascadeFlow is a library for developers who need to control AI agent spending and behavior during a run. It works inside the application, assessing model calls, tool calls and sub-agent handoffs before deciding how execution should continue.

Aiursoft Ollama Gateway gives teams controlled access to shared local LLM servers. It connects AI applications and coding assistants to Ollama or OpenAI-compatible backends through a central gateway.

PasteGuard is a privacy proxy for teams that use AI with customer records, client files or project secrets. It runs locally or on your own server and checks requests before they reach a model provider.

llama-swap is a local AI proxy for people who want several models available to their apps without keeping them all loaded. It starts the server a request needs and replaces the active server when another model is requested.

OGX, formerly Llama Stack, is an AI application server for developers building chat apps, document search and AI agents. It brings model access and the services those applications need into a server you can host yourself.

Manifest LLM Gateway connects AI agents and apps to local models, cloud providers and subscriptions through a shared endpoint. It's for builders and teams who want to choose where requests go, handle provider failures and track inference usage.

LocalAI is a self-hosted AI runtime for developers and teams that want to run models on their own hardware. It brings language, speech, vision and media generation into one server that existing AI applications can connect to.

Twinny is an AI coding assistant for developers who want completion and help with their repository inside VS Code, using models on their own hardware or a chosen server.

WindsurfAPI, also called DevinAPI, connects coding assistants and other AI clients to models through your own Devin account. The gateway runs on your hardware, but it sends prompts and images to Devin's cloud for inference, so it requires internet access.