Tools tagged with "OpenAI-compatible API"

Slotstream runs Qwen3.8-Flash-Next on Apple Silicon Macs that don't have enough RAM to hold the whole model. It's aimed at people with 16 to 64 GB of memory who want local chat, image questions or a model backend for coding agents. Most model weights stay on the SSD, while frequently used expert networks stay in memory. The full model remains available.

PenEcho is an open-source workspace that puts handwritten notes, diagrams and AI-generated widgets on an infinite canvas. It runs on your computer as a macOS or Windows app, or through Node.js with a browser interface. It's for people who want to work through technical ideas visually and give AI feedback by drawing directly on the result.

Reef is self-hosted infrastructure for developers who want AI agents to improve through feedback on actual interactions. It connects inference and learning with versioned deployment, so an agent can update its model weights or its prompts, rules, and skills while continuing to serve requests. It's open source under Apache 2.0.

Pooled runs a single open model across browser tabs on laptops, desktops and phones, combining their memory when the model won't fit on one device. It's for people who want local AI chat or a coding assistant using hardware they already have. It's open source under the MIT license and requires no account or per-device installation.

Meetily is a local AI meeting assistant for people who want meeting notes while keeping recordings on their own device. It captures calls from Zoom, Google Meet, Microsoft Teams and other meeting software without placing a bot in the meeting. You can watch the transcript appear during the call, then generate a summary.

Bifrost is a self-hosted AI gateway for developers and teams whose applications use multiple model providers. It puts Ollama, custom model deployments, and cloud services behind one OpenAI-compatible API, so applications can switch models without maintaining a separate integration for each provider.

SurfSense is an open-source NotebookLM alternative that turns documents into cited answers and editable deliverables on your computer. It's a desktop app for Windows, macOS and Linux, aimed at people working with confidential files, research or study material. You don't need an account.

Graphiti is a self-hosted Python framework for developers building AI agents that need to remember changing facts. It builds knowledge graphs from conversations, structured records and unstructured text, so an agent can query current information or recover what was true earlier. It's open source under Apache 2.0.

exo is a local LLM runner that combines your devices into a cluster, letting you use models too large for one machine's memory. It's for people who want to run large models on their own hardware and developers connecting existing AI clients to local inference. It runs on macOS and Linux under the Apache 2.0 license.

Jan gives people a ChatGPT-style chat interface for AI models running on their own computer. It's free and available for Windows, macOS and Linux. Local chats work offline, with the model and conversation kept on your machine. You can also use cloud models when you want access to a hosted provider.

vLLM is an open source engine for serving large language models on hardware you control. It suits developers and teams that need to handle many requests through an API while making efficient use of memory and compute. It's licensed under Apache 2.0 and can run with GPUs or on a CPU.

LibreChat is a self-hosted ChatGPT alternative for people and teams that want local models and cloud AI services in one chat interface. It runs on a server you control and is open source under the MIT license. You can switch models without moving to a separate chat app.

Dify is a source-available platform for teams building AI agents and apps on a visual canvas. Its Community Edition runs on your own server with Docker. Dify also offers a hosted cloud service, while Enterprise deployments can run in a VPC or on a self-hosted server. The Community Edition uses a custom Apache 2.0 derivative license.

Crush is a terminal AI coding assistant for developers who want to work with their code and tools through a model they choose. The app runs on your machine, while model requests go to your selected local backend or cloud provider. It supports macOS, Linux, Windows (PowerShell and WSL), Android, FreeBSD, OpenBSD and NetBSD.

Open Notebook is a self-hosted alternative to Google's NotebookLM for researchers, students, and professionals who want AI help with their own research materials. It runs locally or on your server through Docker and is open source under the MIT license. You choose which content the AI can access and which provider processes it.

LM Studio is a desktop application for downloading and running language models on macOS, Windows and Linux. You can search for models, manage downloads and chat with them through the app. Downloaded models can run offline, including document chat that uses files on your computer.

llama.cpp runs language models on your own hardware and can serve them from a machine you control. It’s an MIT-licensed, open source inference engine for people building local AI apps, running a private model server, or using a model directly from the command line. It supports vision-language models too.

LLaMA-Factory is an open-source framework for developers and researchers who want to adapt language and multimodal models on their own hardware. It brings training and inference into one toolkit, with support for LLaMA, Qwen3, Qwen3-VL, DeepSeek, Gemma, Mistral and LLaVA. Its license is Apache 2.0.

Cline is an AI coding agent for developers who want help understanding a codebase and changing it. Its shared agent core is open source under Apache 2.0. You can use it in VS Code, JetBrains IDEs, or a terminal. A native desktop app runs on macOS and Windows. No editor is required for desktop sessions. The JetBrains plugin itself is not open source.

LiteLLM gives platform teams one place to manage access to LLMs across providers. Its self-hosted AI gateway puts cloud services and internal or locally hosted models behind an OpenAI-compatible API, so applications can change models without changing their integration. Developers can also use its Python SDK directly.

Open WebUI gives individuals and teams a self-hosted place to chat with local LLMs and cloud models. It runs on your own computer or server, including through Docker, and can work entirely offline with local models.

Ollama runs language models on your own computer or server. It provides a command-line runner and a local API for people building AI applications or connecting existing tools to models they host themselves. The software is distributed under the MIT license.

SGLang is a self-hosted inference framework for teams that need to serve language and multimodal models on their own hardware. It runs on a single GPU or across distributed clusters and exposes an OpenAI-compatible API. The project is open source under the Apache 2.0 license.

LocalAI runs language models, speech, vision and image generation on hardware you control. It's for developers and teams that want a self-hosted AI server for their apps without sending model requests to a cloud service. Its OpenAI-compatible API works with existing clients, and it also accepts Anthropic, Ollama and ElevenLabs API calls.

Zed is a native code editor for developers who want AI coding assistance and collaboration in the same desktop app. It runs on macOS, Linux, and Windows, with a minimal interface and a focus on responsive editing. Its AI agents can work in parallel on tasks that involve editing files, exploring code, and running tools.

Unsloth brings model training and everyday AI use into a desktop app for people who want to run models on their own hardware. Its no-code interface covers chat, fine-tuning and media generation on macOS, Windows and Linux. The Unsloth software is open source under Apache 2.0.

RecurseChat is a paid Mac app for people who want to chat with AI and ask questions about their files on their own computer. It runs local LLMs without an internet connection or a server, keeping local conversations on the device. The same app also connects to Claude and ChatGPT, so you can choose local processing or a cloud provider.

APIPark is a self-hosted AI gateway and developer portal for teams that need to manage access to models and business APIs in one place. It connects Ollama alongside cloud providers such as OpenAI, Claude, Gemini and DeepSeek. The gateway runs on your infrastructure; requests to cloud providers still go to those services.

PyRIT is an MIT-licensed, open source Python framework for security professionals and engineers assessing generative AI systems. It combines automated attack testing with human-led investigations through CoPyRIT, a web interface served locally. The framework runs locally, but prompts go to the target services you choose; cloud targets and cloud-based scorers process requests outside your machine.

GoModel is a self-hosted AI gateway for developers and platform teams that want one API for local models and cloud providers. It accepts OpenAI- and Anthropic-compatible requests, so applications can keep their existing SDKs while the gateway handles provider selection and usage controls.

Agent Router is an open source AI gateway for teams whose agents use both model APIs and MCP tools. It runs on a laptop, a dedicated gateway, or Kubernetes, and gives applications one OpenAI-compatible entry point for cloud providers and self-hosted inference. The project uses the Apache 2.0 license.

BricksLLM is a self-hosted AI gateway for teams that need to control access and spending across LLM applications. It sits between applications and model providers, applying cost and rate limits to individual API keys. The gateway is open source under the MIT license and runs locally or on your server through Docker, with PostgreSQL and Redis.

RouteLLM is a self-hosted Python framework for developers who want to split requests between a stronger LLM and a cheaper model. It judges which prompts need the stronger model, so an application doesn't have to send every request to its most expensive provider. You control the cost-quality tradeoff through a routing threshold.

One API is a self-hosted LLM API gateway for developers and teams that want to share model access across apps or users. It puts cloud providers and Ollama behind an OpenAI-compatible API, so clients can use one endpoint across different backends. It's open source under MIT and runs on your own server as a single executable or in Docker.

Extended OpenAI Conversation is a custom component for Home Assistant users who want an AI assistant that can act on their home. It builds on OpenAI Conversation and adds device control, automation creation, and access to historical device states. It runs inside Home Assistant, while a separate model backend handles the conversation.

LLMKube is a free, open-source Kubernetes operator for teams and homelab owners running local LLM inference across their own hardware. It manages Linux GPU servers and Apple Silicon Macs together, so a mixed fleet can serve models through the same platform. It uses the Apache 2.0 license.