LLM Client Libraries and SDKs
Libraries for adding LLM features to an app: clients like the Vercel AI SDK, structured output with DSPy or Outlines, and UI kits like Gradio.
A CLI and Python library for local LLMs and remote APIs, with SQLite prompt history, embeddings, and support for OpenAI-compatible endpoints.
12.6KUpdated 1 week agoApache-2.0
#LM Studio integration#Multimodal input#OpenAI-compatible API
LLM is an Apache-2.0 command-line tool and Python library for sending prompts to local models and remote APIs. Local model support comes through plugins; cloud providers require their own API access. It can also connect to an arbitrary OpenAI-compatible Chat Completions endpoint, including LM Studio.
LLM Libraries and SDKsLocal Model RunnersTerminal Chat Clients+0
An open-source tensor format and library for storing model weights safely, with Rust and Python support, partial loading, and an Apache 2.0 license.
3.9KUpdated 1 week agoApache-2.0
Safetensors is a file format and library for developers who store, share, or load AI model weights on their own hardware or servers. It avoids the arbitrary code execution risk of PyTorch's pickle-based files while supporting fast access to tensor data. The project is open source under Apache 2.0, with a Rust implementation and Python support.
LLM Libraries and SDKs+0
An MIT-licensed Python framework for AI agents and LLM apps, with model, tool and data integrations. LangChain.js serves JavaScript and TypeScript.
147.3KUpdated 1 day agoMIT
#Human approval#RAG#Streaming inference
LangChain is an MIT-licensed open-source framework for developers building AI agents and applications powered by LLMs. It provides a shared interface for models, tools and data connections, so developers can change providers or test workflows without rebuilding the whole application.
Agent FrameworksLLM Libraries and SDKsRAG Frameworks and Pipelines+0
An open-source Java AI framework connects Spring applications to Ollama or cloud models, with document retrieval, tool calling and MCP support.
9.5KUpdated 21 hours agoApache-2.0
#MCP#Ollama integration#RAG
Spring AI is an open-source Java framework for developers adding AI to Spring applications. It connects application data and APIs to models through a common interface, with Ollama support for local LLM use and integrations with cloud providers such as OpenAI, Anthropic and Amazon Bedrock. The framework runs within your application; your choice of model provider determines whether model requests stay local or go to a cloud service.
LLM Libraries and SDKsRAG Frameworks and Pipelines+0
JavaScript client for Ollama in Node.js and browsers. Connect to local models or Ollama's cloud with an open-source, MIT-licensed library.
4.4KUpdated 2 days agoMIT
Web#LoRA#Multimodal input#Ollama integration
Ollama JavaScript connects Node.js and browser applications to models running through Ollama. It's for developers building chat interfaces, AI agents or other apps that need a local LLM backend. The library is open source under the MIT license, with TypeScript types and an API that follows Ollama's REST interface.
LLM Libraries and SDKs+0
Python library for running GGUF models locally through llama.cpp, with a self-hosted OpenAI-compatible server and CPU or GPU support.
10.6KUpdated 1 week agoMIT
macOS · Windows · Linux#GGUF#Hugging Face integration#llama.cpp backend
llama-cpp-python brings llama.cpp model inference into Python applications and exposes it through a self-hosted OpenAI-compatible server. It's for developers building local AI applications or connecting existing API clients to models on their own hardware. The package is open source under the MIT license.
Inference Libraries and BindingsInference Servers and ClustersLLM Libraries and SDKs+0
An open-source JavaScript library that runs AI models in your browser using ONNX Runtime, with CPU execution and optional WebGPU acceleration.
16.3KUpdated 1 week agoApache-2.0
Web#Hugging Face integration#Image-to-image#Multilingual
Transformers.js is a JavaScript library for developers building web apps that run AI models on the user's device. Inference happens in the browser, so an app doesn't need a separate model server to process its inputs. The library is open source under Apache 2.0.
Inference Libraries and BindingsLLM Libraries and SDKsOn-Device and In-Browser AI+0
Open-source Python framework for checking LLM inputs and outputs, generating structured data, and running guards in your app or a self-hosted service.
7.5KUpdated 1 month agoApache-2.0
#Guardrails#Structured output
Guardrails AI is an open-source Python framework for developers who need to check what goes into an LLM and what comes back. It runs within your application or as a self-hosted service. The framework uses the Apache 2.0 license and helps address risks such as policy violations, hallucinations, and data leakage before outputs reach users.
Guardrails, Security and SandboxingLLM Libraries and SDKs+0
An open-source AI agent SDK for C#, Python and Java. Build workflows with local models through Ollama, LMStudio or ONNX, or connect to cloud providers.
28.6KUpdated 1 day agoMIT
macOS · Windows · Linux#LM Studio integration#MCP#Multi-agent workflows
Semantic Kernel is an MIT-licensed SDK for developers adding AI agents to their applications. It supports local models through Ollama, LMStudio and ONNX, alongside cloud services such as OpenAI and Azure OpenAI. You choose the model backend. The SDK runs on Windows, macOS and Linux and supports C#, Python and Java.
Agent FrameworksLLM Libraries and SDKs+0
Open-source browser agent SDK for local Chrome or Browserbase cloud browsers, with natural-language actions and structured extraction. MIT licensed.
25.5KUpdated 2 days agoMIT
#LLM tracing#Structured output
Stagehand is an open-source browser automation SDK for developers building AI agents that interact with websites and extract structured data. It can run with Chrome on your own machine or use Browserbase's cloud browsers. Local runs require Chrome. The project uses the MIT license and supports TypeScript, Python, and Go.
Computer and Browser AgentsLLM Libraries and SDKsWeb Scraping for AI+0
A Python library that constrains LLM output with grammars and JSON schemas, supporting local Transformers and llama.cpp backends plus OpenAI.
21.8KUpdated 4 months agoMIT
#llama.cpp backend#Structured output#Tool calling
Guidance is an MIT-licensed, open source Python library for developers who need language model output to follow a defined format. It works with local LLM backends including Transformers and llama.cpp, as well as OpenAI's cloud service. It's for application code.
LLM Libraries and SDKs+0
Open-source AI framework for semantic search, RAG and agents. Runs locally or in Docker, with Hugging Face, llama.cpp and cloud models via LiteLLM.
13KUpdated 22 hours agoApache-2.0
Docker#Agent Skills#Hugging Face integration#Knowledge graphs
txtai is a Python framework for developers building search applications, chat with their data, and AI agents on their own hardware or servers. Its embeddings database combines sparse and dense vector search with graphs and relational data, so the same system can find related content and supply context to language models. It's open source under Apache 2.0.
Agent FrameworksLLM Libraries and SDKsMCP Servers and Agent SkillsRAG Frameworks and PipelinesVector Databases+0
Open-source Python framework for building modular LLM systems, with typed outputs, tool-using agents, and automatic prompt optimization. MIT licensed.
38.4KUpdated 4 days agoMIT
#Code execution#MCP#Multimodal input
DSPy is a Python framework for developers building AI applications whose tasks need clear inputs, predictable output types, and measurable results. You define what a language model should produce, then compose those tasks into a larger program. It's open source under the MIT license.
Agent FrameworksLLM Fine-TuningLLM Libraries and SDKsPrompt Management and PlaygroundsRAG Frameworks and Pipelines+0
An open-source Python MCP framework for building servers, connecting to local or remote tools, and adding interactive interfaces to conversations. Apache 2.0.
27.9KUpdated 1 day agoApache-2.0
#MCP#Structured output#Tool calling
FastMCP is an open-source Python framework for developers connecting AI agents to their own tools and data through the Model Context Protocol (MCP). It supports locally running servers and connections to remote servers, with server development and client access in the same framework. It uses the Apache 2.0 license.
LLM Libraries and SDKsMCP Servers and Agent Skills+0
A Python library for asking questions about SQL, CSV and parquet data, with chart generation and a separate hosted business intelligence app.
23.8KUpdated 11 months ago
Docker#Code execution#RAG
PandasAI is a Python library for people who want to ask questions about their data in plain language. It works with SQL databases, CSV and parquet data, and can answer questions across multiple pandas DataFrames. Developers can use it in their own applications; analysts can use conversational queries to reduce the code they write for individual questions.
Chat With Spreadsheets and DatabasesLLM Libraries and SDKs+0
LLM training framework with ready-made research scripts, NVIDIA GPU parallelism, and Hugging Face checkpoint conversion through Megatron Bridge.
18KUpdated 1 day ago
Docker#Distributed execution#Hugging Face integration#Quantization
Megatron-LM is a Python framework for research teams training large language models on NVIDIA GPU infrastructure. It pairs ready-made training scripts with Megatron Core, a library developers can use to build their own training systems. Its focus is distributed training, with benchmarks on H100 clusters spanning thousands of GPUs.
LLM Libraries and SDKsQuantization and Model ConversionTraining Frameworks+0
An MIT-licensed Python framework for document-based AI apps. It works with Ollama, while its separate document services run locally or in the cloud.
52.4KUpdated 2 days agoMIT
#Ollama integration#RAG#Reranking
LlamaIndex is an MIT-licensed Python framework for developers building AI agents and apps that answer questions using their own data. It connects documents and other sources to language models, then helps an app find the relevant material when a user asks something. Its open source framework can work with models served through Ollama.
Agent FrameworksDocument Parsing for LLMsLLM Libraries and SDKsRAG Frameworks and Pipelines+0
An open-source Python library for evaluating LLM apps and RAG systems, with custom metrics, test data generation, and LangChain and LlamaIndex integrations.
15.9KUpdated 7 months agoApache-2.0
Ragas is an open-source Python library for developers who need repeatable evaluations of LLM applications and retrieval-augmented generation (RAG) systems. It combines model-based scoring with traditional metrics so teams can compare application changes using test results rather than manual judgments alone. Its license is Apache 2.0.
Datasets and Synthetic DataLLM Evaluation and TestingLLM Libraries and SDKs+0
An open-source Python framework for AI chat apps you can self-host, with customizable interfaces, OAuth and integrations including LangGraph and HuggingFace.
12.5KUpdated 1 month agoApache-2.0
Web#LLM tracing#Multi-user access#Streaming inference
Chainlit is an open-source Python framework for developers building conversational AI apps with their own application logic. You can run the app on your own server and give users a browser chat interface. It's a framework for creating an app, rather than a ready-made chatbot with a fixed model.
AI Bots for Chat AppsLLM Libraries and SDKsSelf-Hosted Chat Web UIs+0
Open-source Java library for LLM apps on the JVM, with Ollama support, tool calling, RAG, and integrations for Spring Boot and Quarkus.
13.2KUpdated 1 day agoApache-2.0
#MCP#Ollama integration#RAG
LangChain4j is an Apache 2.0 open-source Java library for developers building chatbots, assistants and AI agents in JVM applications. It connects application code to local LLM backends such as Ollama as well as cloud providers such as OpenAI and Google Vertex AI. Where model requests go depends on the backend you choose.
Agent FrameworksLLM Libraries and SDKsRAG Frameworks and Pipelines+0
An open-source Ruby AI framework for Ruby and Rails apps, with Ollama, hosted providers and OpenAI-compatible endpoints under an MIT license.
4.4KUpdated 1 day agoMIT
#Human approval#Multi-agent workflows#Multimodal input
RubyLLM is an MIT-licensed AI framework for developers building Ruby and Rails applications with local or hosted models. Its shared API lets an application switch between Ollama, cloud providers such as Anthropic and OpenAI, and OpenAI-compatible endpoints without rewriting its model integration. The framework runs in your application; model processing happens at the local or hosted backend you choose.
Agent FrameworksLLM Libraries and SDKsRAG Frameworks and Pipelines+0
.NET library for local LLM apps using Ollama, with streaming chat, embeddings and model management. Open source under MIT; also supports Ollama cloud.
1.4KUpdated 2 months agoMIT
#Multimodal input#Ollama integration#Streaming inference
OllamaSharp is a C# library for developers building .NET applications around Ollama. It connects to Ollama on your own machine or a remote server and covers the full Ollama API, including model management alongside chat and embeddings. It's open source under the MIT license.
LLM Libraries and SDKs+0
An MIT-licensed LLM data extraction library that validates structured outputs with Pydantic and works with Ollama, llama-cpp-python, vLLM and cloud APIs.
14KUpdated 3 weeks agoMIT
#llama.cpp backend#Ollama integration#Streaming inference
Instructor is an open-source library for developers who need structured data from local LLMs or cloud models. It turns natural-language input into typed objects that applications can use, with validation and retries built into the extraction process. Its focus is data extraction.
LLM Libraries and SDKs+0
An open-source Python library that constrains LLM outputs to schemas and grammars, with support for local backends including Ollama and llama.cpp.
15.9KUpdated 1 month agoApache-2.0
#llama.cpp backend#MLX#Ollama integration
Outlines is an open-source Python library for developers who need LLM responses to match a defined structure. It constrains output during generation, reducing the need to repair malformed JSON or retry responses that don't fit an application's requirements. The library uses the Apache 2.0 license.
LLM Libraries and SDKs+0
Local LLM library runs GGUF models through llama.cpp in Node.js, Bun and Electron. MIT licensed, with GPU support and JSON schema enforcement.
2.2KUpdated 3 days agoMIT
macOS · Windows · Linux#Batch processing#GGUF#Guardrails
node-llama-cpp is an open source library for developers adding local LLM inference to JavaScript and TypeScript applications. It connects Node.js, Bun and Electron to llama.cpp, running GGUF models on your own machine. Its MIT license allows use in commercial projects.
Inference Libraries and BindingsLLM Libraries and SDKsTerminal Chat Clients+0
An open-source programming language for AI agents that runs on macOS, Linux and Windows, with typed model calls, local tracing and built-in evaluations.
9.4KUpdated 1 day agoApache-2.0
macOS · Windows · Linux · iOS · Android · Web#Batch processing#LLM tracing#Structured output
BAML is a programming language for developers building AI agents, with typed model calls and local tracing built into the language. It runs standalone on macOS, Linux and Windows, or alongside an existing application. The language is open source under Apache 2.0, and works offline.
Agent FrameworksLLM Evaluation and TestingLLM Libraries and SDKsLLM Observability and Tracing+0
Open-source Python AI agent framework under MIT. Coordinate agents and tools, use local or hosted sandboxes, and connect to OpenAI or other model providers.
29.8KUpdated 24 hours agoMIT
macOS · Windows · Linux#Code execution#Guardrails#Human approval
OpenAI Agents SDK is an open-source Python framework for developers building AI apps that need to use tools, delegate tasks, or work across multiple steps. Its runtime manages agent turns and conversation state while letting developers express workflows in ordinary Python. It uses the MIT license.
Agent FrameworksLLM Libraries and SDKsVoice Agent Frameworks+0
Open-source Python library for compressing local or Hugging Face LLM checkpoints, with Apache 2.0 licensing and vLLM-compatible output.
3.8KUpdated 23 hours agoApache-2.0
#Hugging Face integration#Quantization
LLM Compressor is an open-source Python library for developers preparing models to run on their own hardware with vLLM. It reduces model size and memory requirements through quantization, and accepts local checkpoints or models from Hugging Face repositories. It's licensed under Apache 2.0.
LLM Libraries and SDKsQuantization and Model Conversion+0
A local LLM engine that runs models in the browser with WebGPU acceleration, OpenAI API compatibility, and an Apache 2.0 license.
19.2KUpdated 2 weeks agoApache-2.0
Web · Browser Extension#OpenAI-compatible API#Streaming inference#Structured output
WebLLM runs language models directly in a user's browser, using WebGPU for GPU acceleration. It's an open-source engine for developers building web-based AI assistants and Chrome extensions that process prompts on the user's device rather than an inference server. The project uses the Apache 2.0 license.
Inference Libraries and BindingsLLM Libraries and SDKsOn-Device and In-Browser AI+0
An open-source Python AI framework for self-hosted agents and document search, with local model support and an Apache 2.0 license.
26.6KUpdated 1 day agoApache-2.0
Docker#Guardrails#Hugging Face integration#Hybrid search
Haystack is a Python framework for developers building self-hosted AI agents, document search, and apps that answer questions using their own data. Its modular pipelines let teams control which information reaches a model and inspect how retrieval, memory, tools, and generation contribute to an answer. It's open source under Apache 2.0.
Agent FrameworksLLM Libraries and SDKsRAG Frameworks and Pipelines+0
A self-hosted LLM guardrails toolkit for Python apps. Control conversations, check model responses, and connect to LLaMa-2, Falcon or OpenAI models.
7.2KUpdated 23 hours ago
Docker#Guardrails#LLM tracing#Tool calling
NeMo Guardrails is an open-source Python toolkit for developers who need control over how an AI assistant responds and uses tools. It runs within your application or as a self-hosted server, including in Docker. The library uses the Apache 2.0 license.
Guardrails, Security and SandboxingLLM Evaluation and TestingLLM Libraries and SDKs+0
Python client for Ollama with local and cloud model access, streaming chat, embeddings, and async support. Open source under the MIT license.
10.6KUpdated 2 days agoMIT
#Batch processing#Ollama integration#Streaming inference
Ollama Python connects Python applications to models running through Ollama on your own machine or to Ollama's cloud service. It's for developers adding local LLM features to scripts, chat applications, or other Python projects. The library is open source under the MIT license and requires a running Ollama service for local use.
LLM Libraries and SDKs+0
A local LLM merging toolkit that combines model weights without extra training. Runs on CPU or GPU and supports Llama, Mistral and PyTorch checkpoints.
7.4KUpdated 3 weeks agoLGPL-3.0
#Hugging Face integration#LoRA
mergekit combines existing language models into a single model on your own hardware, without additional training or access to the original training data. It's for developers and researchers who want to combine fine-tuned capabilities or adjust the balance between model behaviors. The Python toolkit is open source under LGPL-3.0.
Fine-Tuning and TrainingLLM Libraries and SDKs+0
A free TypeScript library for AI apps and agents, with streaming, provider switching, and chat UI hooks for React, Next.js, Svelte, and Vue.
27.1KUpdated 2 hours ago
#Agent Skills#Streaming inference#Structured output
Vercel AI SDK is a TypeScript library for developers building chatbots, generative interfaces, and AI agents. It gives an application one way to work with models from providers such as OpenAI, Anthropic, and Google, so teams can change providers without rebuilding the parts of their app that handle responses. It suits developers who want control over the application they build while choosing how it connects to model services.
Agent FrameworksLLM Libraries and SDKs+0
An open source Python library for AI web apps that run on your computer, with options to share a public link or host on Hugging Face Spaces.
43.6KUpdated 1 day agoApache-2.0
Web#MCP
Gradio turns a Python function, model, or API into a web interface. It runs on your computer, making it useful for developers and researchers who want people to try an AI project through a browser. It's open source under the Apache 2.0 license.
LLM Libraries and SDKsVisual AI App Builders+0
