Bifrost

Bifrost is an AI gateway for developers and teams building applications that use local LLMs or multiple cloud providers. It gives those applications a shared API for model access and a central place to manage usage.

• Model access: Connect Ollama, custom model deployments, OpenAI, Anthropic, AWS Bedrock and Google Vertex through an OpenAI-compatible interface that supports text, images, audio and streaming.
• Request routing: Automatic fallbacks switch providers or models when requests fail, while load balancing distributes traffic across providers and API keys.
• Budgets and access: Set spending limits for teams and projects, give virtual keys separate budgets and permissions, and track usage and audit logs.
• Agent tools: The MCP gateway brings tool connections together with centralized authentication and policies for access to files, web search and databases.
• Monitoring and caching: Inspect requests in the built-in dashboard, connect OpenTelemetry, and reuse responses to semantically similar requests to reduce repeated model calls.

The gateway runs as a self-hosted HTTP service or inside a Go application. It works with existing OpenAI, Anthropic, Google GenAI, LiteLLM and LangChain integrations. The free, open-source software uses the Apache 2.0 license; commercial private deployments are also available.

Local model requests can go to Ollama on your infrastructure. Choosing a cloud provider sends requests to that provider. Private in-VPC deployments avoid routing data through Bifrost or Maxim servers.