LLM Client Libraries and SDKs

Transformers is a Python library for developers and researchers running pretrained AI models or training models on their own hardware. It covers text, images, audio, video and multimodal tasks through shared model definitions.

agent-desktop lets developers give AI agents control of macOS apps such as Finder, Safari and Xcode. It identifies controls through the app's accessibility tree, so agents can act on buttons and text fields by their UI structure.

Cua supplies desktops and automation tools for developers building AI agents that operate native apps and browsers. You bring your own agent and model; Cua provides the computer environment and ways to interact with it.

OpenAI Agents SDK is a Python framework for developers building tool-using agents with local models or cloud APIs. It supports gpt-oss through Ollama, with LM Studio also documented as a local model option. OpenAI models use the remote Responses API by default and require an API key, even when the agent runtime and workspace run on your machine.

OpenLLMetry helps developers trace LLM applications alongside their existing database and API activity. It extends OpenTelemetry, so teams can send AI application traces to the monitoring system they already use.

Agently is a Python AI agent framework for developers building assistants and service backends with local Ollama models or hosted model APIs. It runs within your application, without requiring a separate orchestration service. Model requests go to the endpoint you choose, so using a remote provider sends the request data outside your machine.

LiteLLM is an AI gateway for platform teams managing model access across applications and providers. It puts cloud and self-hosted models behind a shared interface so developers can switch providers without rewriting their integration.

HippoRAG is a Python memory and retrieval framework for developers building LLM applications that need to connect facts across documents. It uses a knowledge graph to find related information for questions that require several pieces of evidence.

wllama lets developers add local LLM inference to browser apps without a separate inference server. It brings llama.cpp to JavaScript and TypeScript projects through WebAssembly, so models run on the user's device.

NeMo Guardrails is a Python toolkit for developers building AI assistants that need rules for what they say and which actions they take. It adds checks around conversations, document-based answers and tool use.

LangGraph is an AI agent framework for developers building applications on their own machines, including workflows that use local models through LangChain's ChatOllama integration. Ollama can run models such as gpt-oss locally. LangGraph also works with remote model providers, so where model requests go depends on the connection your application uses.

Fast Agent is a terminal coding assistant and Python toolkit for developers building AI agents and automated workflows. You can choose local LLMs or hosted providers and use the same toolkit for interactive work, scripting, and evaluation.

Stagehand is an open-source browser agent SDK for developers automating websites and collecting structured data in local Chrome. Model inference is separate from browser execution: direct provider calls send model context remotely, while a custom callback can connect a self-hosted model. Local runs require Chrome.

DataDreamer is a Python library for researchers and developers who need to create training datasets and adapt LLMs to specific tasks. It connects data preparation and model training in reproducible workflows.

LLM Compressor prepares models for deployment with vLLM by reducing their storage and memory requirements through quantization. It's for developers working with local checkpoints or models from Hugging Face repositories.

Liger Kernel is a Python library for developers training or fine-tuning LLMs on their own hardware. Fused kernels reduce memory use during model operations and post-training.

DSPy is a Python framework for developers building language model applications with local backends such as Ollama on a laptop or SGLang on a GPU server. It connects application code to those model servers, which handle inference on your hardware. It also supports cloud providers, so where model requests go depends on the backend you choose.

Ollama JavaScript is a library for developers adding model chat, generation or retrieval features to JavaScript applications. It connects those applications to an Ollama server, including one running locally.

RubyLLM is an AI framework for Ruby and Rails developers building chat and agent features into their applications. Its shared API lets you switch between local and hosted models without writing a separate integration for each provider.

BAML is a programming language for developers building AI agents, with a documented Ollama client for models running locally and tracing that runs on your machine. The OllamaClient integration is documented for the canary toolchain. The standard OpenAI connection sends model requests to OpenAI; installing BAML locally does not make that connection offline.

Pathway's Live Data Framework is a Python framework for developers building RAG applications and analytics over changing data. It runs locally or on your own servers, including through Docker. Its incremental engine updates results as data arrives, so applications can work with changing documents and events.

SimpleMem is a memory system for developers building AI agents that need to recall past conversations and stored media without passing the full history to a model each time.

FastAPI MCP is a Python library for developers who want AI applications to use an existing FastAPI service through the Model Context Protocol (MCP). It creates tools from API endpoints, so developers don't have to build a separate MCP interface for each one.

WebLLM is an engine for developers building browser-based AI assistants and Chrome extensions. It runs language models on the user's device, so applications don't need a separate inference server to process prompts.

PaddleNLP is a Python toolkit for developers training and deploying language models on their own infrastructure. Built on PaddlePaddle, it covers pretraining, fine-tuning, model compression and inference across several accelerator families.

CocoIndex is an open-source AI indexing framework for developers processing codebases and documents on their own hardware. Its code indexing example uses sentence-transformers for embeddings and writes to pgvector or LanceDB, giving it a concrete role in preparing searchable context for agents and LLM apps.

RAG-Anything is a Python framework for developers building question-answering systems over documents that mix prose with visual or structured content. Built on LightRAG, it suits research papers, technical manuals and financial reports where answers depend on figures, tables or formulas.

Transformers.js is a JavaScript library for developers adding on-device AI to web apps. It runs models in the browser, so processing inputs doesn't require a separate inference server.

Sandbox Agent lets developers control coding agents from their own applications through a shared API. It's for teams building agent-powered software that needs remote sessions and isolated code execution without a separate integration for each agent.

LangChainGo is a Go implementation of LangChain for developers adding language models to their own applications. It provides reusable components for connecting models to documents, tools, and conversation history.

any-llm is a Python library for developers building applications that need to work with different LLM providers. A shared interface reduces the provider-specific code needed to switch between local servers and cloud APIs.

Guidance is a Python library for developers who need language model responses in a defined format. It controls output during generation for applications that expect structured data or specific syntax.

smol developer is a Python coding agent with a self-hosted API for generating codebases from product specifications. It's for developers prototyping applications or adding code generation to their own software. Its default generation workflow uses GPT-4 through the OpenAI Python SDK, whose standard API endpoint sends model requests to OpenAI’s cloud. Self-hosting controls the agent server, while this standard model connection remains remote.

Vercel AI SDK is a free TypeScript library for developers building apps and agents that use local models through Ollama or LM Studio. It runs within your application; those backends run the models. Its default model route uses Vercel AI Gateway, which sends requests to a cloud service, while the documented local integrations connect to your own model server.