any-llm

any-llm is a Python library for developers building applications that need to work with different LLM providers. A shared interface reduces the provider-specific code needed to switch between local servers and cloud APIs.

• Local and cloud backends include Ollama, custom OpenAI-compatible endpoints, OpenAI, Anthropic, Mistral and Azure / Microsoft Foundry.
• Responses API access supports applications using the OpenAI-style Responses API when the selected provider implements it.
• Reusable provider clients support applications that make repeated requests, while direct functions suit scripts and notebooks.
• Framework independence lets developers use the library without adopting a particular agent framework; it uses official provider SDKs and includes Python type hints.

The library runs inside your Python application, and the selected backend performs inference. An Ollama or compatible local server can run models on your hardware; cloud providers receive requests through their APIs and require provider credentials. Feature availability depends on the backend.

any-llm requires Python 3.11 or newer and is open source under the Apache 2.0 license. Developers migrating from LiteLLM can keep their existing API keys and environment variables.