Agentgateway
Route cloud and local LLM traffic, MCP tool calls and agent communication through a self-hosted gateway with access policies, tracing and spend controls.
Agentgateway is an open-source proxy for platform teams connecting applications and agents to models, tools and other agents. It brings routing and access controls together across AI protocols and ordinary service traffic.
- LLM routing: Reach OpenAI, Anthropic, Gemini or self-hosted models through an OpenAI-compatible API, with load balancing and failover.
- Tools and agents: Federate MCP servers, control which tools each caller can invoke, and route agent communication through A2A.
- Inference routing: Distribute requests across Kubernetes inference workers using GPU utilization, cached context, model adapters and queue depth.
- Access and content policies: Apply authentication, role-based permissions, rate limits and redaction to model requests and MCP traffic.
- Usage and spend: Track requests, token consumption and costs through OpenTelemetry, Prometheus and a browser dashboard, with token budgets and spending limits.
The Apache 2.0 gateway is self-hosted as a standalone application, in Docker or on Kubernetes. It supports Ollama and includes a browser playground for testing model routes. Cloud model requests go to the selected providers; content checks using OpenAI moderation or AWS Bedrock Guardrails also use those external services.
Logotyper
