Tools tagged with "llama.cpp backend"

Graphiti is a self-hosted Python framework for developers building AI agents that need to remember changing facts. It builds knowledge graphs from conversations, structured records and unstructured text, so an agent can query current information or recover what was true earlier. It's open source under Apache 2.0.

Crush is a terminal AI coding assistant for developers who want to work with their code and tools through a model they choose. The app runs on your machine, while model requests go to your selected local backend or cloud provider. It supports macOS, Linux, Windows (PowerShell and WSL), Android, FreeBSD, OpenBSD and NetBSD.

LM Studio is a desktop application for downloading and running language models on macOS, Windows and Linux. You can search for models, manage downloads and chat with them through the app. Downloaded models can run offline, including document chat that uses files on your computer.

MinerU parses documents locally into structured text for AI agents, RAG systems and knowledge bases. It's for people working with scanned PDFs, academic papers and Office files whose tables, formulas or page layouts need more care than plain text extraction.

Ollama runs language models on your own computer or server. It provides a command-line runner and a local API for people building AI applications or connecting existing tools to models they host themselves. The software is distributed under the MIT license.

Qwen3 is a family of language models from Alibaba Cloud’s Qwen team for people who want to run models locally or on their own servers. It spans smaller and larger dense models as well as mixture-of-experts models. The weights are publicly available.

LocalAI runs language models, speech, vision and image generation on hardware you control. It's for developers and teams that want a self-hosted AI server for their apps without sending model requests to a cloud service. Its OpenAI-compatible API works with existing clients, and it also accepts Anthropic, Ollama and ElevenLabs API calls.

Zed is a native code editor for developers who want AI coding assistance and collaboration in the same desktop app. It runs on macOS, Linux, and Windows, with a minimal interface and a focus on responsive editing. Its AI agents can work in parallel on tasks that involve editing files, exploring code, and running tools.

QMD is a local search engine for people with Markdown notes, meeting transcripts, or documentation they want to search themselves or make available to an AI agent. It accepts exact keywords and natural-language queries, with indexing and model inference running on your own machine. It's open source under MIT.

GoModel is a self-hosted AI gateway for developers and platform teams that want one API for local models and cloud providers. It accepts OpenAI- and Anthropic-compatible requests, so applications can keep their existing SDKs while the gateway handles provider selection and usage controls.

LLMKube is a free, open-source Kubernetes operator for teams and homelab owners running local LLM inference across their own hardware. It manages Linux GPU servers and Apple Silicon Macs together, so a mixed fleet can serve models through the same platform. It uses the Apache 2.0 license.

h2oGPT is a self-hosted ChatGPT alternative for people who want to chat with local models and ask questions about their own documents. The project is archived and no longer maintained. It's open source under Apache 2.0, with support for Linux, macOS, Windows and Docker.

LMQL is a programming language for developers who need model calls and ordinary Python logic in the same program. It lets you define rules for generated text, including types, length limits, allowed answers and stopping phrases. Those rules apply during generation, so you can constrain intermediate responses as well as the final output.

Twinny is an AI coding assistant for VS Code that lets developers choose where their models run: on their own computer, a private server or a hosted API. It's for individuals and teams who want code suggestions and repository chat with control over where their code goes. The extension and team gateway are open source under the MIT license.

Amica is a locally runnable interface for talking with customizable 3D AI characters. It's for people who want an animated, voiced character as the face of their AI assistant, with a choice of local LLM backends or cloud services. The project builds on Pixiv's ChatVRM.

Mikupad is a browser-based LLM frontend packaged as a single HTML file, for writers and people who want control over generated text. It connects to a model backend of your choice and supports both direct text continuation and chat with instruct models. It's open source under AGPL-3.0.

MindMac is a native macOS AI chat client for people who want to use local LLMs and cloud services in the same app. Its inline mode lets you ask questions or generate text inside Notes, Mail and browsers without switching to a chat window. It runs on Intel and Apple Silicon Macs with macOS 13 or newer.

Backyard AI's desktop app uses llama.cpp, a runtime for running language models locally. It's for people looking for a desktop AI app on Windows or macOS, with separate Mac downloads for Intel and Apple Silicon hardware.

Serge is a self-hosted chat interface for people who want to run language models on their own hardware and talk to them in a browser. The project is archived and no longer maintained. It uses llama.cpp to run models locally, with Alpaca as a named chat model and LLaMA also referenced in its memory requirements.

LlamaGPT is a self-hosted ChatGPT alternative for people who want general chat or coding help on their own computer or home server. It runs models locally and keeps conversation data on your device. After the initial model download, it works offline.

Layla is an offline AI assistant for Android and iOS that runs language models on your phone. It's for people who want private everyday chat or an AI companion with custom characters, memory and roleplay. Local conversations stay encrypted on your device; optional cloud mode sends requests to your chosen hosted provider.

Lollms WebUI is a local, single-user AI interface for people who want text chat and media generation in one place. It runs on Windows, macOS and Linux, with Docker support, and lets writers, developers and other users choose models and task-specific personalities. It's free and open source under Apache 2.0. The project receives minimal maintenance.

Nexa SDK is an on-device AI inference framework for developers building applications that process text, images or audio on users' hardware. It runs models locally across CPUs, GPUs and NPUs, with a shared interface for different backends. Its scope includes language and vision models, speech recognition, speech synthesis and image generation.

DB-GPT is a self-hosted AI data assistant for teams analyzing business data and developers building data applications. It turns plain-language requests into SQL queries and Python analysis, then produces charts, dashboards, or HTML reports. You can run it on macOS or Linux, with Docker deployment also supported.

Nemotron is NVIDIA's family of AI models for developers building agents that reason, write code and call tools. You can run models locally for private, offline work or deploy them on your own servers. NVIDIA publishes model weights, training data and recipes so teams can inspect and adapt the models for their applications.

lm-evaluation-harness lets researchers and model developers compare language models using shared academic benchmarks and public prompts. It runs evaluations against local models and benchmarks, or sends requests to a hosted model API. EleutherAI's Python framework is open source under the MIT license and powers Hugging Face's Open LLM Leaderboard.

LLM Farm runs large language models offline on iOS and macOS. It's for people who want on-device AI chat or need to compare how different models perform on Apple hardware before choosing one for a project. The app is open source under the MIT license, with ggml and llama.cpp handling local inference.

Falcon-H1 is a family of language models from the Technology Innovation Institute for developers building local AI applications or serving models on their own hardware. Its hybrid design combines attention with Mamba2 state space models to reduce memory use and speed up inference. Model sizes cover smaller devices and larger server deployments.

MiniCPM-V is a family of local vision-language models for developers building apps that interpret images and video on their own hardware. It supports iOS, Android and HarmonyOS, as well as Mac deployment and server inference. The current repository states that MiniCPM-o/V code and model weights use Apache 2.0.

Yi is a family of open-weight language models from 01.AI for developers, researchers, and businesses that want to run English and Chinese models on their own hardware. It includes models for conversation and base models for fine-tuning, with license terms that depend on the release. Yi-1.5 code and weights use Apache 2.0; the original Yi releases have separate community terms.

Intel IPEX-LLM is a library for developers running or fine-tuning models on Intel hardware. The project is archived and no longer maintained. Intel reports known security issues and no longer accepts patches or provides updates. The code is open source under Apache 2.0.

Orpheus TTS is an open-source text-to-speech system for developers building voice applications or adapting speech models to their own recordings. It runs locally and uses a Llama backbone to generate speech with control over emotion and intonation. The code uses the Apache 2.0 license.

gptme is a self-hosted AI agent that works directly in your terminal, with access to your files and installed tools. It's for developers who want a coding assistant in their own environment, and people who want an agent for data analysis or other knowledge work. The software is free under the MIT license and doesn't require a gptme account.

HuggingChat UI is the open-source chat application behind Hugging Face's hosted HuggingChat. You can run it on your own computer or server and connect it to a local LLM backend or a cloud provider. It's for people and teams who want a browser-based ChatGPT alternative with control over the chat service and where its data lives. The code uses the Apache 2.0 license.

SenseVoice is a local speech recognition model that adds language, emotion and sound-event tags to transcriptions. It's for developers building voice applications or analyzing recordings on their own hardware, particularly those working with Mandarin and Cantonese. The project is open source under the MIT license.

Harbor is a CLI and companion app for people experimenting with AI on their own hardware. It manages a local LLM development environment, connecting model backends to chat interfaces and supporting services so you don't have to configure each connection yourself. It's open source under Apache 2.0.