Bifrost
A self-hosted AI gateway with an OpenAI-compatible API for Ollama and cloud providers, Apache 2.0 licensing, model fallbacks, and usage monitoring.
Bifrost is a self-hosted AI gateway for developers and teams whose applications use multiple model providers. It puts Ollama, custom model deployments, and cloud services behind one OpenAI-compatible API, so applications can switch models without maintaining a separate integration for each provider.
The gateway runs on infrastructure you control, either as an HTTP service with a web dashboard or embedded in a Go application. Model requests go to the selected backend: local models can use Ollama, while requests to OpenAI, Anthropic, AWS Bedrock, Google Vertex, and other cloud providers leave your infrastructure. Bifrost is open source under Apache 2.0, and commercial private deployments are also available.
Automatic fallbacks route requests to another model or provider when one fails. Load balancing distributes traffic across providers and API keys. Semantic caching reuses responses to similar requests to reduce repeated model calls, and the shared interface handles text, images, audio, and streaming.
Teams can manage spending and access through budgets and virtual keys. The MCP gateway centralizes connections to external tools, including databases and web search. A built-in dashboard and OpenTelemetry support help teams inspect usage and request behavior.
Bifrost works with existing OpenAI, Anthropic, Google GenAI, LiteLLM, and LangChain integrations. Its Go implementation emphasizes low request overhead, and custom plugins let teams add monitoring or application-specific logic.
