Tools tagged with "GGUF"

Llama Desktop runs language models on your computer for private chat and use in other apps. It's for people who want a local ChatGPT alternative and developers connecting coding agents, editors or scripts to their own models.

QMD searches Markdown notes, meeting transcripts, documentation and knowledge bases on your own machine. It's for people who want to find material by its wording or meaning, and make that search available to an AI agent.

Candle is a Rust machine learning framework for developers embedding local AI in applications or deploying models on their own servers. It supports inference and training, including small CPU deployments for serverless applications.

Personal LLM runs AI chat on your phone for people who want to ask questions without sending them to a cloud model. Chat, photo analysis and document questions work offline after you download a model and any required image support files.

BitNet is Microsoft's local LLM inference framework for developers and researchers working with compatible 1-bit models. Its bitnet.cpp runtime focuses on faster inference and lower energy use on your own hardware.

GPT4All is a local AI chatbot for people who want language models and document conversations on their own computer. Model inference and LocalDocs processing run on the device without a cloud connection.

PrivateLM is an AI chat app for people who want to run models on their phone or tablet and choose when to use a cloud provider. It supports local inference on Android and iOS; the browser version requires cloud APIs.

MiniCPM is a family of compact language models for developers building local assistants and coding agents. It targets on-device workloads and hardware with limited computing resources.

RecurseChat is a Mac app for people who want private AI conversations and answers drawn from their own documents. Local models run without an internet connection or a separate server.

SenseVoice is a local speech recognition model for developers building voice apps or analyzing recordings. Its SenseVoiceSmall model adds information about the language, emotion and surrounding sounds to speech transcriptions.

Thuki is a floating AI assistant for Mac users who want help with the text or screen they're already viewing. It appears above other apps, including fullscreen windows, and automatically brings selected text into the conversation.

Noema is a local AI assistant for people who want to chat with models and get answers from their files on Apple devices. It brings models, documents and tools into a native workspace on iPhone, iPad, Mac and Vision Pro.

llama.rn is a library for developers adding local LLM features to React Native apps on iOS and Android. It brings llama.cpp inference onto the device, so an app can process model requests on the user's hardware.

CloverPal is an Android app for private AI conversations and character roleplay on your phone. It runs models on-device, so you can chat offline once you've downloaded or imported a model.

LocaLM runs AI models on your own hardware for chat and optional tasks such as coding, document questions and media generation. It's for people who want a single local tool and can choose which extra capabilities to add.

Pocket AI Lab runs AI models on an iPhone and helps you judge which ones the phone can handle. It's for people who want private chat on their device and want to check memory and storage before downloading model weights.

Open-LLM-VTuber is an AI companion for people who want to talk with a customizable animated character. Local language and speech models let it work fully offline, with conversations staying on your device.

MiniCPM-V is a family of local vision-language models for developers building applications that interpret images and video. It can run on phones and other hardware you control.

llama-swap is a local AI proxy for people who want several models available to their apps without keeping them all loaded. It starts the server a request needs and replaces the active server when another model is requested.

Docker Model Runner lets developers run AI models on their own computers or servers and connect them to applications. It combines local inference with model distribution through container registries.

Nexa SDK is an on-device AI framework for developers building apps that run models on users' hardware. It covers language, vision and audio tasks through a shared interface for local inference.

ACE-Step Studio is a desktop AI music generator for musicians and anyone who wants to create songs on their own hardware. It turns a short idea, or a description with lyrics, into a full vocal track or instrumental.

llama.cpp is a local LLM inference engine for developers building AI applications or running models on hardware they control. It supports language and vision-language models, with command-line tools and a server API for using the engine directly or connecting other software.

Soup is a local LLM fine-tuning tool for developers and teams adapting models to their own datasets. It brings training, evaluation and export into one workflow that runs on your own hardware without requiring a cloud service.

llama-cpp-python is for developers adding local LLM inference to Python applications or connecting existing API clients to models on their own hardware. It brings llama.cpp into a Python library and a self-hosted server.

Qube is a desktop AI assistant for people who want to chat with models and work with private documents on their own hardware. It combines a document library with persistent memory and research tools that provide citations you can inspect.

Harbor helps people experiment with local LLMs by connecting model backends, chat interfaces and supporting services. Its CLI and companion app manage those connections in a shared development environment.

OpenVINO Model Server is a self-hosted inference server for developers and teams serving AI models on their own machines or servers. It uses OpenVINO to run workloads on x86 CPUs, including Xeon, with acceleration for Intel integrated and discrete GPUs and NPUs. It's open source under Apache 2.0.

Maid is an Android AI chat app for people who want to chat with models on their phone and access remote models from the same interface. Local conversations work without an internet connection.

FunASR is a speech-to-text toolkit for developers building transcription services and voice applications. It supports offline recordings and live speech on local hardware or self-hosted servers, with capabilities that depend on the model and runtime.

Atomic Agent is a local AI agent for people who want an assistant to work with their browser, files and code on their own hardware. Its model and task loop can run offline, while sessions and memory stay on disk. It's free and open source under the MIT license.

Layla is an AI assistant for people who want private chat, a personal companion or character roleplay on their phone. It runs local models on Android and iOS, so you can keep using it offline after downloading a model.

lm-evaluation-harness is a Python framework for researchers and developers comparing language models on shared benchmarks. Public prompts help make results comparable across model checkpoints and research papers.

PowerInfer is a local LLM inference engine for developers and researchers running large models on a PC with a consumer GPU. Its performance gains depend on sparse models, which use only part of their network for each input.