Tools to Run AI Models Locally

SwarmLLM is a ChatGPT alternative for people who want to run AI models on their own hardware. It can use a single computer or combine several machines to handle a model that won't fit on one.

RapidOCR is an OCR toolkit for developers who need to extract text from images within their own applications. It uses models derived from PaddleOCR and can run recognition offline on local hardware.

Candle is a Rust machine learning framework for developers embedding local AI in applications or deploying models on their own servers. It supports inference and training, including small CPU deployments for serverless applications.

Personal LLM runs AI chat on your phone for people who want to ask questions without sending them to a cloud model. Chat, photo analysis and document questions work offline after you download a model and any required image support files.

Parler-TTS is a text-to-speech library for developers adding speech to applications and researchers adapting speech models. It generates audio with a voice style you describe in text.

WhisperIME converts speech to text on Android for people who want offline dictation in their keyboard or a standalone transcription app. It uses Whisper for recognition on the device.

WhisperLive turns speech into text as audio arrives, using OpenAI's Whisper models. It's for developers and teams who want to run transcription on their own machine or server and connect it to live audio or recorded files.

BitNet is Microsoft's local LLM inference framework for developers and researchers working with compatible 1-bit models. Its bitnet.cpp runtime focuses on faster inference and lower energy use on your own hardware.

Piper generates spoken audio from text on local hardware. It's for developers who need a text-to-speech engine for their applications or self-hosted services.

Magnitude is a local LLM engine for people who want to run AI agents on their own hardware. It adapts model execution to your machine and connects to coding assistants you already use.

GPT4All is a local AI chatbot for people who want language models and document conversations on their own computer. Model inference and LocalDocs processing run on the device without a cloud connection.

spaCy is a Python library for developers and data scientists who need to extract structured information from text in their own applications. It runs on macOS, Windows and Linux, with model packages that become part of your application. It's free and open source under the MIT license.

PrivateLM is an AI chat app for people who want to run models on their phone or tablet and choose when to use a cloud provider. It supports local inference on Android and iOS; the browser version requires cloud APIs.

Whats Reader lets you revisit exported WhatsApp conversations and their attachments without sending the files to a service. It works with individual and group chats, so you can review old conversations or find details buried in messages and voice notes.

Talk DAT! is a free, open-source dictation app for people who want to turn speech into formatted writing across their desktop apps. It runs on Windows 10 and 11 and Apple silicon Macs with macOS 14 or later. Downloaded models let it work offline without an account, keeping speech and text on your computer.

RTranslator helps people hold spoken conversations in different languages using their Android phones. It runs translation locally and works offline.

Chatterbox is a text-to-speech model family for developers and creators who want to generate voices on their own hardware. It suits narration, voice agents and interactive media, with models for English and multilingual speech.

RecurseChat is a Mac app for people who want private AI conversations and answers drawn from their own documents. Local models run without an internet connection or a separate server.

FastVLM is a family of local vision-language models for developers and researchers building applications that answer questions about images. It focuses on reducing the delay before an answer starts, especially with high-resolution images.

Dictate Keyboard is a voice keyboard for people who prefer speaking to typing in Android apps. It builds on FlorisBoard and lets you choose where speech recognition happens.

ChatLLM Web is a browser app for people who want AI help with text and code while keeping their working data on their device. It combines chat, model selection, and an agent that can work with files you choose.

Thuki is a floating AI assistant for Mac users who want help with the text or screen they're already viewing. It appears above other apps, including fullscreen windows, and automatically brings selected text into the conversation.

Noema is a local AI assistant for people who want to chat with models and get answers from their files on Apple devices. It brings models, documents and tools into a native workspace on iPhone, iPad, Mac and Vision Pro.

DeepFace is a Python library for developers adding face recognition and facial analysis to applications. It brings several recognition models behind a shared interface and handles the image preparation needed to compare faces.

Deltafin runs full Kimi K3 on a single device for researchers and developers exploring large models on their own hardware. It preserves the original expert weights and complete expert bank, but responses are slow and long conversations take substantially longer than short completions.

llama.rn is a library for developers adding local LLM features to React Native apps on iOS and Android. It brings llama.cpp inference onto the device, so an app can process model requests on the user's hardware.

Pooled runs a single open model across laptops, desktops and phones in browser tabs. It lets people combine their devices' memory to use a model that wouldn't fit on one device, for shared chat or coding tasks.

Offline Translator is an Android app for translating material when you don't have an internet connection or want translation to stay on your device. It handles everyday text as well as images and PDF/ODT documents.

CloverPal is an Android app for private AI conversations and character roleplay on your phone. It runs models on-device, so you can chat offline once you've downloaded or imported a model.

LocaLM runs AI models on your own hardware for chat and optional tasks such as coding, document questions and media generation. It's for people who want a single local tool and can choose which extra capabilities to add.

Private LLM runs AI chat on iPhone, iPad and Mac for people who want to use language models without sending conversations to a cloud service. It works offline after the first model download and doesn't require an account.

Pocket AI Lab runs AI models on an iPhone and helps you judge which ones the phone can handle. It's for people who want private chat on their device and want to check memory and storage before downloading model weights.

TorchAO is a PyTorch library for developers who want to train and run models on their own hardware with less memory. It supports language models and image or video generation, and works with torch.compile and FSDP2 across most Hugging Face PyTorch models.

Cactus is an on-device AI engine for developers adding automation to mobile apps, wearables and other small devices. Its Needle model turns requests into tool calls locally, including when the device has no internet connection.