any-llm
A Python LLM library that connects to Ollama, OpenAI-compatible local servers and cloud providers through one API. Open source under Apache 2.0.
any-llm is a Python library for developers who want the same application to work with local LLM servers and cloud providers. It connects to Ollama and custom OpenAI-compatible endpoints, alongside OpenAI, Anthropic, Mistral and Azure / Microsoft Foundry. A shared interface reduces the provider-specific code needed to try another model or change where inference runs.
The library runs in your Python application; the selected backend handles inference. Local endpoints let you use models on your own hardware or server, while cloud providers receive requests through their APIs and require provider credentials. any-llm is open source under the Apache 2.0 license.
Its capabilities include chat completions, streaming responses, tool calling, text embeddings, reasoning and content moderation. It also exposes the OpenResponses API for agentic applications, with availability depending on the provider. Consistent exception handling gives applications a common way to handle errors across backends.
any-llm uses official provider SDKs and doesn't depend on a particular agent framework. Full Python type hints support editor assistance. Direct functions suit scripts and notebooks, while the AnyLLM class supports reusable provider clients and access to provider metadata. Developers moving from LiteLLM can retain their existing API keys and environment variables.
