BricksLLM

BricksLLM is a self-hosted AI gateway for teams that need to control access and spending across LLM applications. It sits between applications and model providers, applying cost and rate limits to individual API keys. The gateway is open source under the MIT license and runs locally or on your server through Docker, with PostgreSQL and Redis.

Its controls suit teams sharing model access across users, organizations, applications or development environments. You can restrict which models and endpoints a key can use, then track requests and costs by user or organization. That makes it useful for internal AI services, customer-facing applications and shared access for students.

BricksLLM supports OpenAI, Azure OpenAI, Anthropic and Deepinfra, plus vLLM for local LLM deployments. It also connects to custom models and deployments, and supports all OpenAI endpoints. Hosting the gateway yourself doesn't make cloud inference local: requests sent to an external provider still go to that provider.

Caching, request retries and failover help applications handle repeated requests and provider failures. Privacy controls include blocking or masking personally identifiable information and controlling what gets logged. The documented PII detection feature uses AWS credentials, so that feature involves an external service. Datadog integration supports monitoring.

A managed cloud offering is also available, with a dashboard for administering the gateway.