Tools tagged with "GGUF"

biniou brings AI chat and media creation into one browser interface on your own computer or server. It's for people who want to work with several kinds of generative AI without switching between separate apps.

Offeline is a local AI chat app for people who want to talk to AI on their phone or tablet without sending conversations to a hosted chatbot. It processes chats on the device.

Off Grid AI is a local AI app for people who want chat, document analysis and media tools on their own phone or computer. Downloaded models work offline, and the core app doesn't require an account.

GPTQModel is a Python toolkit for developers compressing LLMs and running them on their own hardware or servers. It combines model preparation, quality evaluation and inference so you can assess compression choices within one workflow.

Adrian is a runtime security tool for teams whose AI agents can use tools or act on external systems. It checks planned actions against the agent's intended task and can intervene before an unsafe action runs.

llamafile packages a language model and its runtime in a single executable for people who want to run models locally or share a ready-to-run model with others. Recipients don't need to install a separate runtime.

llmware is a Python framework for developers building applications that answer questions using their own documents or carry out multi-step AI agent workflows. It focuses on small, specialized models that can run on laptops, edge devices, and self-hosted servers.

WhisperJAV generates subtitles for Japanese adult videos on your own computer. It targets difficult audio with quiet dialogue, loud vocalisations and long stretches that can cause speech recognition models to invent or repeat words.

Netron visualizes neural network and machine learning models for developers and researchers who want to examine their structure.

Airux Pocket AI runs AI conversations on Android phones for people who want offline chat and photo analysis without cloud inference. You don't need an account, and downloaded models run directly on your phone.

Unsloth is a desktop app for people who want to chat with models, train them on their own data, and create media on their own hardware. Its no-code interface brings these tasks into one workspace.

Home-LLM connects Home Assistant to language models for voice and chat control of smart devices. It's for Home Assistant users who want those requests handled on their own hardware.

Lexicon is a writing assistant and rich-text editor for people who want to proofread and revise drafts on their own computer. Grammar checks work without an AI model, while optional local AI tools help rewrite longer passages and whole documents.

NeuTTS is a collection of on-device text-to-speech models for developers building voice assistants and speech-enabled apps. It generates audio on your own hardware rather than sending speech generation to a cloud API.

Atomic Chat is a local LLM app for people who want to chat with models on their own device or use them with coding assistants and AI agents. It began as a fork of Jan by Menlo Research.

LLMKube manages self-hosted LLM inference for teams and homelab owners who need several machines to serve models together. It brings Linux GPU servers and Apple Silicon Macs into the same Kubernetes-managed fleet.

Falcon-H1 is a family of language models from the Technology Innovation Institute for developers building AI applications on their own hardware. Smaller models suit edge devices, while larger variants target server deployments.

OfflineLLM is an on-device AI chat app for Android users who want to run models locally without sending conversations to a cloud service. It works fully offline and has no network permissions.

Ollama runs open models for people who want local AI applications, model chat or a backend for coding assistants. It provides a command-line interface and an API, with a choice between inference on your hardware and its cloud service.

Qwen3 is a language model family from Alibaba Cloud's Qwen team for people building chat applications or running models on their own hardware. Its public weights include dense and mixture-of-experts models in smaller and larger sizes.

mistral.rs is an inference engine for developers who want to run AI models on their own computer or server. It supports browser chat and inference embedded directly in Python or Rust applications.

DwarfStar is a local LLM inference engine for people who want to run large DeepSeek, GLM and Qwen models on their own computers or servers. It focuses on selected models and hardware setups, and requires the project's own GGUF files rather than accepting arbitrary GGUF downloads.

Modly is a desktop app for artists, game developers and people preparing models for 3D printing. It turns a photo or text prompt into a textured 3D mesh using AI models on your own GPU.

SmolChat is an Android app for chatting with language models on your phone. It gives people who want on-device AI a chat interface for their own GGUF model files, including small language models.

Handy is a desktop speech-to-text app for people who want to dictate into the apps they already use. It transcribes speech on your computer and inserts the text into the active text field.

node-llama-cpp is a library for developers adding local AI to JavaScript and TypeScript applications. It uses llama.cpp to run GGUF models on your hardware and works in Node.js, Bun and Electron.

Moltis is a personal AI agent server for people who want an assistant that remembers earlier conversations and carries out tasks on their own hardware. You can reach the same agent through a browser or messaging apps.

Aphelion is a desktop app for people who want character conversations, creative writing and coding help from models running on their own computer. It works offline after the first model download and doesn't require an account.

Claude Code Local lets developers use Claude Code with a model running on their own computer. It suits people who want to keep coding offline or continue working when they hit cloud usage limits.

MLX Tune is a Python library for adapting AI models on an Apple Silicon Mac. Formerly unsloth-mlx, it is an independent project and is not affiliated with Unsloth. Its Unsloth-compatible API lets you prototype training locally with minimal changes to existing scripts, then move to NVIDIA hardware with Unsloth. Training runs through Apple's MLX framework and doesn't require a cloud GPU.

Jina Embeddings is a family of models for developers building semantic search, retrieval-augmented generation (RAG) and AI agent applications. The models convert content into vectors that a search system can compare for relevance.

VoxCPM is a local text-to-speech system for developers and audio creators who want to generate speech with voices they can design or clone. It supports multilingual narration and control over how a voice delivers the text.

FlashRank is a Python library that runs reranking models locally on CPU, without Torch or Transformers. It's for developers building search systems or retrieval-augmented generation (RAG) apps who want to reorder retrieved passages by relevance before passing them to an LLM.

Amuse is a Windows app for people who want to generate media and text with models running on their own hardware. It combines generation with editing tools for both generated output and existing local video.