Portkey Gateway
An open-source AI gateway under the MIT license, with an OpenAI-compatible API for Ollama and cloud providers. Runs locally with Node.js or Docker.
Portkey Gateway is a self-hosted AI gateway for developers whose apps need to use local models and cloud providers through one OpenAI-compatible API. It routes requests to Ollama, OpenAI, Anthropic, Google Gemini and other backends, with controls for handling failures and checking model inputs and outputs.
The gateway runs locally with Node.js or Docker, and its code is open source under the MIT license. A local console shows request logs. Where model processing happens depends on the backend you choose: Ollama can run models on your hardware, while requests to cloud providers go to those services. Portkey also offers a hosted gateway and enterprise private deployments.
Routing goes beyond selecting a model. Automatic retries and fallbacks can recover from failed requests, while load balancing distributes traffic across providers or API keys. Conditional routing and request timeouts give developers more control over how their apps handle different requests. Response caching can reduce repeated model calls.
Guardrails check both prompts and replies against defined rules. Developers can use built-in checks or bring their own. The same API covers text, vision, speech and image generation, and supports streaming responses and OpenAI realtime APIs.
