Self-Hosted LLM Gateways and Routers

Proxies like LiteLLM that put many models behind one API and handle routing, caching, spending limits and keys.

64 tools
Favicon of FreeLLMAPI

FreeLLMAPI

10 videos
A self-hosted LLM router that combines cloud providers and local OpenAI-compatible backends, with automatic failover and encrypted provider keys.

54.7KUpdated 1 day agoMIT

macOS · Windows · Linux · Android · Docker · Web#Context compression#llama.cpp backend#LM Studio integration

A local coding agent proxy for macOS, Windows and Linux that connects Claude Code and Codex to local models or cloud providers.

71.6KUpdated 19 hours ago

macOS · Windows · Linux · Web · VS Code · JetBrains#Agent Client Protocol#Code execution#llama.cpp backend

Favicon of OmniRoute

OmniRoute

15 videos
A self-hosted AI gateway connects coding tools to local runtimes and cloud providers through one OpenAI-compatible endpoint, with automatic fallback.

88.7KUpdated 5 hours agoMIT

macOS · Windows · Linux · Android · Docker · Web#Context compression#Guardrails#llama.cpp backend

Favicon of CC Switch

CC Switch

4 videos
A desktop AI coding tool manager switches API providers, routes requests locally and shares MCP servers and skills on macOS, Windows and Linux.

155.5KUpdated 13 hours agoMIT

macOS · Windows · Linux#Agent Skills#MCP#OpenAI-compatible API

A self-hosted LLM proxy that routes requests across Ollama, vLLM and other backends, with automatic failover and a built-in monitoring dashboard.

338Updated 3 weeks agoApache-2.0

macOS · Windows · Linux · Docker · Web#llama.cpp backend#LM Studio integration#MLX

An open-source AI agent runtime library that controls model and tool calls inside your app, with Ollama, vLLM and cloud provider support.

3.9KUpdated 2 months agoMIT

#Guardrails#Human approval#LLM tracing

Favicon of InsForge

InsForge

17 videos
A self-hosted backend platform lets AI coding agents manage Postgres, authentication, files and functions through MCP, with a hosted option available.

13.3KUpdated 2 weeks agoApache-2.0

Docker · Web#Code execution#MCP#OpenAI-compatible API

A self-hosted AI gateway connects local or hosted models to private files, databases, and agent tools through an OpenAI-compatible API.

365Updated 1 day agoApache-2.0

Docker · Web#MCP#Multi-user access#Multimodal input

Favicon of New API

New API

1 video
A self-hosted AI gateway connects apps to OpenAI, Anthropic and Gemini APIs with provider routing, access controls and usage tracking.

52.2KUpdated 16 hours agoAGPL-3.0

Linux · Docker · Web#Multi-user access#Multimodal input#OpenAI-compatible API

A self-hosted AI agent evaluation platform with text and voice simulations, OpenTelemetry tracing, and support for Ollama and vLLM.

2.3KUpdated 18 hours agoApache-2.0

Docker · Web#AI red teaming#Guardrails#LLM tracing

Headroom

7 videos
A local context compression tool that reduces LLM inputs, caches originals for retrieval, and connects to coding agents and OpenAI-compatible clients.

80.1KUpdated 5 hours agoApache-2.0

macOS · Windows · Linux#Context compression#MCP#Multi-agent workflows

A self-hosted AI gateway connects local and cloud models through OpenAI- and Anthropic-compatible APIs, with caching, failover and usage tracking.

1.3KUpdated 2 days agoMIT

macOS · Windows · Linux · Docker · Web#Guardrails#llama.cpp backend#LLM tracing

A self-hosted MCP gateway and AI asset registry centralizes discovery, OAuth access and audit logs for MCP servers, agents and skills.

1.1KUpdated 3 hours agoApache-2.0

macOS · Linux · Docker · Web#Agent Skills#Hybrid search#MCP

A self-hosted AI API proxy connects apps to client-based providers and converts requests between OpenAI, Claude, and Gemini protocols.

9KUpdated 20 hours agoGPL-3.0

Docker · Web#Agent Skills#Multimodal input#OpenAI-compatible API

A self-hosted AI API gateway that routes requests to Ollama, cloud LLMs and media services, with centralized access controls and usage monitoring.

122Updated 2 years ago

Docker · Web#Distributed execution#Ollama integration

A self-hosted API gateway connects Devin cloud models to OpenAI, Anthropic and Gemini clients, with account failover and a web dashboard.

3.1KUpdated 1 day agoMIT

macOS · Windows · Linux · Docker · Web#Multimodal input#OpenAI-compatible API#Streaming inference

Route cloud and local LLM traffic, MCP tool calls and agent communication through a self-hosted gateway with access policies, tracing and spend controls.

6.3KUpdated 4 hours agoApache-2.0

Docker · Web#Guardrails#LLM tracing#MCP

Open-source LLM serving infrastructure for Kubernetes that routes requests, scales capacity and supports multi-node inference with vLLM integrations.

5.2KUpdated 5 hours agoApache-2.0

#Batch processing#Distributed execution#LoRA

Favicon of LiteLLM

LiteLLM

16 videos
A self-hosted AI gateway and Python SDK connects cloud and local models through an OpenAI-compatible API, with spend controls and request routing.

63.5KUpdated 45 minutes ago

#Guardrails#MCP#Multi-user access

A self-hosted AI gateway pools API keys and subscription accounts, with automatic failover and support for OpenAI, Anthropic and Gemini protocols.

7.2KUpdated 24 hours agoMIT

macOS · Windows · Linux · Docker · Web#OpenAI-compatible API#Streaming inference

A self-hosted AI gateway connects Claude Code, Codex and Gemini CLI to shared provider accounts, with usage tracking and access controls.

1.1KUpdated 6 days agoMIT

Docker · Web#Multi-user access#OpenAI-compatible API#Role-based access

An open-source Kubernetes operator serves AI models through vLLM and Ollama, with an OpenAI-compatible API and scaling from zero.

1.3KUpdated 1 week agoApache-2.0

Web#LoRA#Multimodal input#Ollama integration

A self-hosted AI gateway that connects Ollama and OpenAI-compatible backends to coding assistants, with API keys, auditing and model aliases.

10Updated 10 hours ago

Linux · Docker · Web#Multi-user access#Multimodal input#Ollama integration

Favicon of Paritok

Paritok

2 videos
A self-hosted coding-agent gateway that reduces prompts sent to LLMs using Ollama or vLLM and lets agents retrieve exact original text.

1.4KUpdated 1 day agoApache-2.0

Web · VS Code#Context compression#GGUF#Hugging Face integration

Favicon of LLMKube

LLMKube

2 videos
A self-hosted LLM inference operator for Kubernetes that supports NVIDIA, AMD and Apple Silicon GPUs and serves an OpenAI-compatible API.

267Updated 10 hours agoApache-2.0

macOS · Linux#GGUF#Git integration#Guardrails

A local gateway that routes coding agents to cloud model APIs on macOS, Windows and Linux. MIT licensed; cloud providers process requests remotely.

38.2KUpdated 3 days agoMIT

macOS · Windows · Linux · Docker · Web#Browser automation#LLM tracing#MCP

A self-hosted LLM gateway routes local and cloud models through one OpenAI-compatible endpoint, with automatic request repair and provider fallbacks.

7.6KUpdated 2 days agoMIT

Docker · Web#llama.cpp backend#LM Studio integration#Ollama integration

Favicon of LocalAI

LocalAI

2 videos
A self-hosted AI runtime that serves local models through an OpenAI-compatible API, with CPU-only support, voice tools and built-in agents.

50.4KUpdated 4 hours agoMIT

macOS · Linux · Docker · Web#Code execution#Human approval#llama.cpp backend

A self-hosted LLM serving stack built on vLLM, with an OpenAI-compatible API, request routing and GPU cluster monitoring under Apache 2.0.

2.7KUpdated 2 weeks agoApache-2.0

Web#OpenAI-compatible API#Prompt caching

An open-source AI gateway connects Ollama and cloud providers through an OpenAI-compatible API, with retries, fallbacks and guardrails under MIT.

13.4KUpdated 6 months agoMIT

Docker · Web#Guardrails#Ollama integration#OpenAI-compatible API

A self-hosted LLM gateway for Linux, macOS and Windows that connects Ollama, vLLM and cloud APIs through an OpenAI-compatible interface. Apache 2.0 licensed.

108Updated 1 month agoApache-2.0

macOS · Windows · Linux#Ollama integration#OpenAI-compatible API#Streaming inference

Favicon of Twinny

Twinny

1 video
An open-source AI coding assistant for VS Code with local or hosted models, inline edits, code review and a self-hosted team gateway.

3.7KUpdated 14 hours agoMIT

Docker · Web · VS Code#Git integration#Hybrid search#llama.cpp backend

A self-hosted LLM proxy connects Ollama, llama.cpp and cloud providers through an OpenAI-compatible API, with MCP tools and agent delegation.

212Updated 1 day agoApache-2.0

Docker#Guardrails#llama.cpp backend#LLM tracing

A self-hosted AI inference gateway routes Ollama, vLLM, TGI, llama.cpp and MLX requests by GPU capacity and model state, with optional cloud overflow.

4Updated 11 hours agoApache-2.0

macOS · Windows · Linux · Docker · Web#llama.cpp backend#LLM tracing#MLX

Favicon of LogotyperLogotyper
Got an idea for an AI tool? I'm Eugene, the designer and developer behind Localhosted. I'll help you build it, from the first sketch to launch.
Start your project
Favicon of Logotyper
Favicon of LogotyperLogotyper
Got an idea for an AI tool? I'm Eugene, the designer and developer behind Localhosted. I'll help you build it, from the first sketch to launch.
Start your project
Favicon of Logotyper

More in LLMOps and Security