Agent Router (formerly Envoy AI Gateway)

Agent Router is an open source AI gateway for teams whose agents use both model APIs and MCP tools. It runs on a laptop, a dedicated gateway, or Kubernetes, and gives applications one OpenAI-compatible entry point for cloud providers and self-hosted inference. The project uses the Apache 2.0 license.

Applications can reach Anthropic, AWS Bedrock, Vertex AI, Azure OpenAI, and self-hosted vLLM through the same API. Model aliases let teams change the underlying model without exposing provider-specific names to each application. Fallback routing handles provider failures, while token limits apply by team, application, or model.

Provider credentials stay with the router rather than in agent code. Agents authenticate to that entry point, and platform teams manage API keys and cloud credentials centrally. The MCP gateway combines tools from multiple servers into a catalog and filters what each caller can access, keeping the tool selection relevant to each agent.

Agent Router manages routing policy; Envoy handles requests. For teams serving their own models, Kubernetes InferencePool support enables endpoint selection that accounts for inference workloads. Observability follows OpenTelemetry GenAI conventions and records model choice, token usage, time to first token, and fallback events without agent-side instrumentation.

Self-hosting puts the gateway under your control; requests still go to whichever model provider or MCP server you choose. Tetrate also offers a hosted router service that runs outside your own infrastructure.