Self-Hosted LLM Gateways and Routers
Proxies like LiteLLM that put many models behind one API and handle routing, caching, spending limits and keys.
54.7KUpdated 1 day agoMIT
macOS · Windows · Linux · Android · Docker · Web#Context compression#llama.cpp backend#LM Studio integration
FreeLLMAPI gives developers one API for experimenting with cloud models and local backends. The router runs locally, but cloud inference still sends requests to the providers you enable.
71.6KUpdated 19 hours ago
macOS · Windows · Linux · Web · VS Code · JetBrains#Agent Client Protocol#Code execution#llama.cpp backend
Free Claude Code lets developers use coding agents with local models or a choice of cloud providers. It connects existing clients to a shared model catalog rather than requiring a separate coding workflow.
88.7KUpdated 5 hours agoMIT
macOS · Windows · Linux · Android · Docker · Web#Context compression#Guardrails#llama.cpp backend
OmniRoute is an AI gateway for developers who want their coding assistants and other AI clients to share one set of provider connections. It routes requests across models and accounts when a service fails, reaches a rate limit or runs out of quota.
155.5KUpdated 13 hours agoMIT
macOS · Windows · Linux#Agent Skills#MCP#OpenAI-compatible API
CC Switch is a desktop manager for developers who use multiple AI coding assistants or API providers. It brings their settings into one interface and preserves existing plugins, hooks and other custom settings when switching providers.
338Updated 3 weeks agoApache-2.0
macOS · Windows · Linux · Docker · Web#llama.cpp backend#LM Studio integration#MLX
Olla puts a single API endpoint in front of existing inference servers. It routes requests across mixed backends and switches servers when one fails before a response starts.
3.9KUpdated 2 months agoMIT
#Guardrails#Human approval#LLM tracing
CascadeFlow is a library for developers who need to control AI agent spending and behavior during a run. It works inside the application, assessing model calls, tool calls and sub-agent handoffs before deciding how execution should continue.
13.3KUpdated 2 weeks agoApache-2.0
Docker · Web#Code execution#MCP#OpenAI-compatible API
InsForge is a backend platform for developers building apps with AI coding agents. It gives agents access to backend services and the context they need to create, check and debug an app.
365Updated 1 day agoApache-2.0
Docker · Web#MCP#Multi-user access#Multimodal input
ORBIT is a self-hosted AI gateway for teams building assistants over internal data, especially in regulated environments. It brings model access, data retrieval, and agent tools together on infrastructure the team controls.
52.2KUpdated 16 hours agoAGPL-3.0
Linux · Docker · Web#Multi-user access#Multimodal input#OpenAI-compatible API
New API is a self-hosted AI gateway for developers and teams sharing access to multiple model providers. It lets apps and agents keep one connection while administrators change upstream services.
2.3KUpdated 18 hours agoApache-2.0
Docker · Web#AI red teaming#Guardrails#LLM tracing
Future AGI helps teams test AI agents, investigate production failures and compare prompt changes. It brings evaluation and monitoring into the same platform so teams can connect poor responses to the steps that caused them.
Headroom
7 videos80.1KUpdated 5 hours agoApache-2.0
macOS · Windows · Linux#Context compression#MCP#Multi-agent workflows
Headroom reduces the context that LLM apps and coding agents send to a model. It's for developers whose prompts fill up with repetitive tool results, logs, file contents and retrieved documents. Compression runs locally, but model requests still go to the chosen LLM provider.
1.3KUpdated 2 days agoMIT
macOS · Windows · Linux · Docker · Web#Guardrails#llama.cpp backend#LLM tracing
GoModel is a self-hosted AI gateway for developers and platform teams sharing access to local models and cloud providers. It centralizes provider choice and usage policies behind APIs compatible with OpenAI and Anthropic SDKs.
1.1KUpdated 3 hours agoApache-2.0
macOS · Linux · Docker · Web#Agent Skills#Hybrid search#MCP
MCP Gateway & Registry gives platform teams a shared catalog and access gateway for the tools their AI agents and coding assistants use. It addresses scattered server connections, credentials and access policies across an organization.
9KUpdated 20 hours agoGPL-3.0
Docker · Web#Agent Skills#Multimodal input#OpenAI-compatible API
AIClient2API is a self-hosted AI API proxy for developers connecting chat apps and coding assistants to multiple model providers. It exposes services such as Antigravity, Codex, Grok, and Kiro through interfaces those apps already support.
122Updated 2 years ago
Docker · Web#Distributed execution#Ollama integration
ServiceStack AI Server is a self-hosted gateway for developers and teams whose applications use several AI providers. It runs in a Docker container and gives apps one API integration for local models, cloud LLMs and media processing. Teams can change providers without changing their client integrations.
3.1KUpdated 1 day agoMIT
macOS · Windows · Linux · Docker · Web#Multimodal input#OpenAI-compatible API#Streaming inference
WindsurfAPI, also called DevinAPI, connects coding assistants and other AI clients to models through your own Devin account. The gateway runs on your hardware, but it sends prompts and images to Devin's cloud for inference, so it requires internet access.
6.3KUpdated 4 hours agoApache-2.0
Docker · Web#Guardrails#LLM tracing#MCP
Agentgateway is an open-source proxy for platform teams connecting applications and agents to models, tools and other agents. It brings routing and access controls together across AI protocols and ordinary service traffic.
5.2KUpdated 5 hours agoApache-2.0
#Batch processing#Distributed execution#LoRA
AIBrix provides infrastructure components for teams serving large language models on Kubernetes. It's aimed at organizations that need to manage inference capacity, request routing and models across a cluster.
63.5KUpdated 45 minutes ago
#Guardrails#MCP#Multi-user access
LiteLLM is an AI gateway for platform teams managing model access across applications and providers. It puts cloud and self-hosted models behind a shared interface so developers can switch providers without rewriting their integration.
7.2KUpdated 24 hours agoMIT
macOS · Windows · Linux · Docker · Web#OpenAI-compatible API#Streaming inference
GPT-Load is a self-hosted AI gateway for developers connecting applications to multiple AI providers and accounts. It centralizes credentials and routing behind one endpoint, so each application doesn't need its own failure handling.
1.1KUpdated 6 days agoMIT
Docker · Web#Multi-user access#OpenAI-compatible API#Role-based access
CliRelay is an AI gateway for teams that want to manage coding assistants through a shared service on their own server. It builds on CLIProxyAPI and brings subscription accounts and provider API keys under central control.
1.3KUpdated 1 week agoApache-2.0
Web#LoRA#Multimodal input#Ollama integration
KubeAI is a Kubernetes operator for teams serving AI models on their own infrastructure or cloud clusters. It manages model servers and adjusts capacity to demand, including starting models from zero running replicas.
10Updated 10 hours ago
Linux · Docker · Web#Multi-user access#Multimodal input#Ollama integration
Aiursoft Ollama Gateway gives teams controlled access to shared local LLM servers. It connects AI applications and coding assistants to Ollama or OpenAI-compatible backends through a central gateway.
1.4KUpdated 1 day agoApache-2.0
Web · VS Code#Context compression#GGUF#Hugging Face integration
Paritok is a self-hosted gateway for developers whose coding agents send large file reads, tool output and conversation histories to an LLM. It reduces that input before the request reaches the upstream model.
267Updated 10 hours agoApache-2.0
macOS · Linux#GGUF#Git integration#Guardrails
LLMKube manages self-hosted LLM inference for teams and homelab owners who need several machines to serve models together. It brings Linux GPU servers and Apple Silicon Macs into the same Kubernetes-managed fleet.
38.2KUpdated 3 days agoMIT
macOS · Windows · Linux · Docker · Web#Browser automation#LLM tracing#MCP
Claude Code Router is a local gateway for developers using coding agents such as Claude Code, Codex and OpenCode. It keeps provider selection and request routing in one app, while the selected cloud provider processes model requests remotely. Those requests leave your machine.
7.6KUpdated 2 days agoMIT
Docker · Web#llama.cpp backend#LM Studio integration#Ollama integration
Manifest LLM Gateway connects AI agents and apps to local models, cloud providers and subscriptions through a shared endpoint. It's for builders and teams who want to choose where requests go, handle provider failures and track inference usage.
50.4KUpdated 4 hours agoMIT
macOS · Linux · Docker · Web#Code execution#Human approval#llama.cpp backend
LocalAI is a self-hosted AI runtime for developers and teams that want to run models on their own hardware. It brings language, speech, vision and media generation into one server that existing AI applications can connect to.
2.7KUpdated 2 weeks agoApache-2.0
Web#OpenAI-compatible API#Prompt caching
vLLM Production Stack is an open source LLM serving stack for teams running models on Kubernetes GPU clusters. It lets applications move from a single vLLM instance to a distributed deployment without changing application code.
13.4KUpdated 6 months agoMIT
Docker · Web#Guardrails#Ollama integration#OpenAI-compatible API
Portkey Gateway routes application requests across model providers through an OpenAI-compatible API. It's for developers who need shared routing and failure controls when working with multiple AI backends.
108Updated 1 month agoApache-2.0
macOS · Windows · Linux#Ollama integration#OpenAI-compatible API#Streaming inference
OpenZiti LLM Gateway is a self-hosted API proxy for developers and teams connecting AI applications to inference servers across private networks and cloud services. Its optional zrok integration uses OpenZiti for identity-based access and end-to-end encryption, so clients and backends can connect across network boundaries without opening inbound ports.
3.7KUpdated 14 hours agoMIT
Docker · Web · VS Code#Git integration#Hybrid search#llama.cpp backend
Twinny is an AI coding assistant for developers who want completion and help with their repository inside VS Code, using models on their own hardware or a chosen server.
212Updated 1 day agoApache-2.0
Docker#Guardrails#llama.cpp backend#LLM tracing
Inference Gateway is a self-hosted LLM proxy for developers building apps or AI agents that use multiple model providers. It gives those clients one OpenAI-compatible API so they can switch providers without rewriting their application integration.
4Updated 11 hours agoApache-2.0
macOS · Windows · Linux · Docker · Web#llama.cpp backend#LLM tracing#MLX
Marbor is a self-hosted AI inference gateway for operators serving models across their own GPU fleet. It places requests using live hardware and model state to reduce cold loads and avoid deployments that can't serve the requested model.