Tools tagged with "GGUF"

llama.vscode is a coding assistant for developers who want AI suggestions, chat, and an agent inside VS Code. It lets you use models on your own hardware or connect to external providers.

vLLM serves language models on hardware you control. It's for developers and teams building model APIs that need to handle concurrent requests while using memory and compute efficiently.

text-generation-webui, also called TextGen, runs language models on your own hardware for chat, writing and application integrations. You can use it as a desktop app or a self-hosted browser interface.

Local Agent Studio is a Windows desktop app for building AI agents and connecting them in visual workflows. It suits first-time agent builders and developers prototyping private automation. Local and cloud agents can work together in the same workflow, with progress visible for each step.

Nemotron is NVIDIA's family of AI models for developers building agents that reason, write code and use tools. Teams can evaluate and adapt the models using published weights, training data and training recipes.

Garak is an LLM vulnerability scanner for developers and security teams assessing models or dialogue systems. It attempts to provoke unwanted responses so teams can identify weaknesses in the systems they test.

VoxInput is an open-source dictation tool for Linux and macOS that types spoken words into the active application. It's for people who want speech input across their desktop, including applications without their own dictation feature. On Linux, it also works in the system console.

Lucebox is a local AI workstation for developers and teams who want to run models on their own hardware. Its inference engine is also available separately for supported AMD and NVIDIA machines.

Vyact is a desktop workspace for people who want a local LLM to help with files, writing and code. It brings conversations and searchable documents together, with optional connections to email services and Chrome.

Tellama is a local LLM chat app for Android users who want conversations to stay on their phone. Chat with an installed compatible model works offline, and you don't need an account.

KoboldCpp runs language models on your own hardware for chat, creative writing and roleplay. It's a fork of llama.cpp with a bundled KoboldAI Lite browser interface.

llmserve is a terminal app for people who keep local LLM models across several folders or inference tools. It brings model discovery and server management into one interface so you can choose an existing model and an available backend.

Lollms WebUI is a local AI interface for writing, coding assistance and media generation through a choice of models. The project receives minimal maintenance, and its interface has no built-in user authentication.

InvokeAI is a free, open-source image generation application for artists who want to create and edit artwork on their own hardware. Its locally hosted browser interface combines direct canvas work with repeatable generation workflows.

FastLLM is an open-source inference engine for people running large language models on their own machines or self-hosted servers. Its main draw is hardware flexibility: large mixture-of-experts (MoE) models can spread their weights across GPU memory, system RAM and disk when they don't fit entirely on a graphics card.

wllama lets developers add local LLM inference to browser apps without a separate inference server. It brings llama.cpp to JavaScript and TypeScript projects through WebAssembly, so models run on the user's device.

Transcribe Offline turns recordings and live speech into text for people who need to review conversations without uploading audio to a cloud service. It pairs transcript editing with audio playback in a desktop app.

Paritok is a self-hosted gateway for developers whose coding agents send large file reads, tool output and conversation histories to an LLM. It reduces that input before the request reaches the upstream model.

Sidekick is a native macOS app for people who want AI answers grounded in their documents and research materials. Its built-in llama.cpp engine runs GGUF models such as Qwen3.5 without a separate model runner.

PocketPal AI runs language models on phones and tablets for people who want an assistant they can use without sending conversations to a server. Core chat works offline after a model download and doesn't require an account.

OpenCommit is a command-line tool for developers who want AI to write commit messages from their Git changes. It can create the commit after you review the proposed message and also fits into an IDE's existing Git workflow.

SpeakoFlow lets you dictate into other apps and ask an AI assistant for help without leaving your work. It's for people who want voice typing, spoken conversations and meeting notes with processing on their own computer.

audio.cpp runs audio models locally for developers building speech tools and people generating or processing audio on their own hardware. It brings multiple audio tasks into one C++ runtime built on ggml.

Priv AI is an on-device assistant for people who want to chat with AI and use it to examine personal health and financial information. It works offline after a model download and doesn't require an app account.

Lemonade is a local AI server for people who want to chat with models on their own hardware or use them in apps and agents. It includes a built-in interface for direct use and APIs for connecting other software.

Zeraix is a local AI desktop for people who want models to work with their files and carry out multi-step tasks. It combines chat, project tools, and agents in one workspace.

Voice-Clone-Studio brings voice cloning and speech production into one browser interface for people making podcasts, audiobooks and dialogue recordings. It combines several engines with tools for preparing samples and training custom voices.

AI File Sorter helps people clear crowded Downloads folders, photo collections and stored archives. It proposes folders and filenames for you to review before it moves or renames files.

ChatterUI is an Android chat app for people who want to use a local LLM on their phone or connect to a remote model. It supports assistant and character conversations, with control over reply generation and chat formatting.

OuteTTS is a local text-to-speech library for developers adding generated speech to their applications. It uses reference audio to reproduce a speaker's voice, including their style, emotion and accent.

TensorSharp runs local AI models for .NET developers and people who want chat, media generation and agent tools on their own hardware. You can embed it in a C# application or use its CLI, browser interface or companion app, TensorAgent.

EXAONE is a family of language models from LG AI Research for developers building multilingual AI applications on their own hardware. It combines everyday language tasks with reasoning for harder problems and supports agents that call external tools.

Soul-of-Waifu is an AI companion app for creating characters and roleplaying with them. A character carries the same personality and memories between conversations, RPG campaigns and a desktop companion.

RamaLama runs AI models for local chat or a self-hosted REST API. It's for developers who want a container workflow for trying models on their own hardware and serving them in production.