Tools tagged with "OpenAI-compatible API"

ChatOllama is an AI agent app whose main interface is a command-line client running on your machine. It's for people who want an agent to work with local files while choosing between models served by Ollama and hosted services. The repository license modifies Apache 2.0: commercial hosting and multi-tenant services require authorization, and frontend branding must remain intact.

AstrBot brings AI assistants into messaging apps such as Telegram, Discord, Slack, QQ and WeCom. It's an open source platform under AGPL-3.0 for people building personal companions, customer support bots or team automation. You can run it on your own computer or server, including through Docker, or use its desktop app for browser-style chat.

TabbyAPI is a self-hosted LLM API server built around ExLlamaV3, for people who want local model inference behind an OpenAI-compatible API. It's the official server for that backend. The project targets personal use and small groups, and its maintainers explicitly advise against using it for production workloads.

Data-Juicer is a Python framework for preparing AI datasets on your own machine or a distributed Ray cluster. It's for researchers and teams curating model training data, agent interaction records or documents for retrieval. The project is open source under Apache 2.0.

Mattermost Agents brings AI assistants into a self-hosted Mattermost workspace, with a choice of local models or cloud providers. It's for teams that want help with long discussions and meeting recordings while controlling where their collaboration data goes. The plugin is open source under Apache 2.0.

llm-d is an open-source stack for teams serving large language models on their own Kubernetes clusters. It coordinates model servers such as vLLM and SGLang across multiple machines, with routing and resource management for production traffic. It uses the Apache 2.0 license.

LLM is an Apache-2.0 command-line tool and Python library for sending prompts to local models and remote APIs. Local model support comes through plugins; cloud providers require their own API access. It can also connect to an arbitrary OpenAI-compatible Chat Completions endpoint, including LM Studio.

text-generation-webui, also called TextGen, runs language models on your own hardware through a desktop app or a self-hosted browser interface. It's for people who want private chat and writing tools, and developers who need a local model API. It works offline without telemetry; web search and page fetching use the internet.

Kokoro-FastAPI runs the Kokoro-82M speech model on your own machine or server and exposes an OpenAI-compatible speech API. It's for developers adding local text-to-speech to assistants, reading apps or audiobook workflows. Speech generation runs locally, and the API doesn't require an OpenAI account.

ThunderAI brings AI writing and email processing into Thunderbird for people who want help with their inbox while choosing where their messages go. It can use local models through Ollama or an OpenAI-compatible server such as LM Studio. Cloud connections send the selected content to ChatGPT, the OpenAI API, Google Gemini or Claude instead.

ms-swift is a Python framework for developers and researchers who want to train and deploy language or multimodal models on their own hardware. It brings fine-tuning, evaluation and model serving into one project, with support for Qwen3, DeepSeek-R1, Llama4 and Mistral, plus multimodal models such as Qwen3-VL and InternVL3.5. It's open source under Apache 2.0.

Continue is an open-source coding agent available as a CLI, VS Code extension and JetBrains plugin. Continue was acquired by Cursor, and the project is no longer actively maintained. Its repository is read-only, and its software uses the Apache 2.0 license.

Alpaca is an open source AI chat client for people who want to run models on their own device. It uses Ollama to download and manage local models, then lets you chat without an internet connection. Conversations stay on your device in SQLite files, and local models have no direct internet access.

llama-cpp-python brings llama.cpp model inference into Python applications and exposes it through a self-hosted OpenAI-compatible server. It's for developers building local AI applications or connecting existing API clients to models on their own hardware. The package is open source under the MIT license.

KoboldCpp pairs local model inference with a browser interface built for chat, creative writing and roleplay. A fork of llama.cpp, it bundles KoboldAI Lite with tools for keeping character details and story context alongside your conversations. It's open source under AGPL-3.0.

Skyvern uses vision models to carry out tasks in a browser, such as filling forms, collecting data, and working through logins. It's for teams automating websites whose layouts change, as well as developers who want AI actions alongside Playwright. Instead of depending on fixed page selectors, it identifies visible elements and decides how to interact with them.

aichat brings Ollama and cloud AI services into the same terminal interface for developers and people who work at the command line. It runs locally on macOS, Linux and Windows, with Android support through Termux. Model processing happens through the backend you choose: Ollama supports local models, while providers such as OpenAI, Claude and Gemini process requests in the cloud.

Aider is an Apache 2.0 open-source coding assistant for developers who work in a terminal and keep their projects in Git. It can help start a project or make changes to an existing codebase. You can use local LLMs or connect to cloud models from providers including Anthropic, DeepSeek and OpenAI.

Feeds Fun is a news reader for people whose RSS subscriptions produce more articles than they want to read. It assigns tags automatically, then uses rules you define to score articles by topic. You can self-host it with Docker or use the hosted service at feeds.fun.

OpenRLHF is a self-hosted Python framework for researchers and teams training language models with human feedback or custom rewards. It runs on your own NVIDIA GPU hardware, with Docker support and distributed training across servers. It's open source under Apache 2.0.

MLC LLM is an open-source compiler and deployment engine for developers who want to run language models on their own hardware or inside apps. Its main distinction is the range of devices it targets: the same underlying engine, MLCEngine, serves desktop, browser and mobile deployments. The project uses the Apache 2.0 license.

mistral.rs is an open source inference engine for running models on your own computer or self-hosted server. It's for developers building AI applications and people who want local chat, multimodal models and agent tools in the same runtime. The Rust project uses the MIT license.

Obsidian Copilot brings AI agents into your vault to find notes by meaning, edit drafts and organize research. It's for writers, researchers and other Obsidian users who want AI to work directly with their existing notes. Agents create Markdown files, wikilinks and canvases that remain in your vault.

Elia is an open source terminal chat client for people who prefer a keyboard-focused interface to a browser chat app. It connects to local LLMs through Ollama or LocalAI and also works with hosted models such as ChatGPT and Claude. The interface runs on your machine; where prompts go depends on the model you choose.

PrivateGPT is a self-hosted API layer for developers building AI applications around local models. It adds document retrieval, database access and agent tools to an existing model server. Local workflows can work offline and keep data within your environment; web search and connections to online providers need internet access.

Lemonade is an open source local AI server for people who want to use models on their own hardware or connect them to apps and agents. It handles chat, coding, image generation, speech, transcription, and embeddings. A built-in interface lets you use those capabilities directly, while its server makes them available to other software.

Local Deep Research is a self-hosted AI research assistant for people who need cited answers drawn from academic papers, the web and their own documents. It can produce a quick summary or pursue a complex question through repeated searches, then assemble a structured report. It's open source under MIT.

LMDeploy is an open-source toolkit for developers serving language and vision-language models on their own hardware. It combines model compression with inference and self-hosted APIs, so teams can use it for batch processing or as the model backend for an application. It uses the Apache 2.0 license.

TensorRT-LLM is a library for developers running LLMs on their own NVIDIA GPUs or self-hosted servers. It focuses on inference performance, with support for a single GPU, multiple GPUs, or deployments spread across several machines. Its PyTorch architecture lets teams adapt models and extend the runtime in Python.

Speaches is a self-hosted speech server for developers who want transcription, translation and speech generation on their own hardware. Its OpenAI-compatible API lets applications use local speech models through tools and SDKs built for OpenAI's API. The project is open source under the MIT license.

Xinference serves language, speech and multimodal models through a shared API on your own computer or servers. It's an open source platform under Apache 2.0 for developers and researchers who want to build applications around models they host. You can also deploy it on cloud infrastructure.

llama-swap is a self-hosted proxy for people running several AI models on their own hardware. It starts the model server a request needs and swaps out another when necessary, so you don't have to keep every model loaded or manage separate API connections in your apps.

ChatterUI is an Android chat app for people who want to run a local LLM on their phone or use the same interface with a remote model. It supports assistant conversations and character chats, with controls for how chats are structured and how models generate replies. It's open source under AGPL-3.0.

GPT4All is a local AI chatbot for people who want to run language models on their own desktop or laptop and keep conversations on their machine. Its LocalDocs feature lets you ask questions about your own documents without sending them to a cloud service. It suits developers, teams and individuals who want control over their models and data.

subgen generates subtitles on your own hardware for personal media libraries, including films and shows that don't have usable subtitles available. It's an open source, MIT-licensed Python service that runs in Docker or as a standalone application. Speech recognition runs locally using Whisper models through faster-whisper and stable-ts, with support for CPU processing and NVIDIA GPUs through CUDA.

Marker is a local document converter for developers and teams turning PDFs, scans and Office files into structured text. It preserves tables, equations and page structure for document processing and AI workflows. Its pipeline reads embedded PDF text and uses Surya OCR where text is missing or damaged, rather than reading every page through a vision model.