Local AI Apps and Tools for Linux

llama-cpp-python brings llama.cpp model inference into Python applications and exposes it through a self-hosted OpenAI-compatible server. It's for developers building local AI applications or connecting existing API clients to models on their own hardware. The package is open source under the MIT license.

RamaLama runs and serves AI models on your own hardware using OCI containers. It's aimed at developers who want local chat or a self-hosted inference API with a container workflow they can also use in production. The project uses the MIT license.

KoboldCpp pairs local model inference with a browser interface built for chat, creative writing and roleplay. A fork of llama.cpp, it bundles KoboldAI Lite with tools for keeping character details and story context alongside your conversations. It's open source under AGPL-3.0.

aichat brings Ollama and cloud AI services into the same terminal interface for developers and people who work at the command line. It runs locally on macOS, Linux and Windows, with Android support through Termux. Model processing happens through the backend you choose: Ollama supports local models, while providers such as OpenAI, Claude and Gemini process requests in the cloud.

Aider is an Apache 2.0 open-source coding assistant for developers who work in a terminal and keep their projects in Git. It can help start a project or make changes to an existing codebase. You can use local LLMs or connect to cloud models from providers including Anthropic, DeepSeek and OpenAI.

FramePack is an open source desktop app for making videos from a still image and a written motion prompt. It runs on Windows and Linux, with generation handled by your own NVIDIA GPU. It suits people who want to make AI video locally and see the clip develop as it renders.

Semantic Kernel is an MIT-licensed SDK for developers adding AI agents to their applications. It supports local models through Ollama, LMStudio and ONNX, alongside cloud services such as OpenAI and Azure OpenAI. You choose the model backend. The SDK runs on Windows, macOS and Linux and supports C#, Python and Java.

Typesense combines typo-tolerant site search with vector and semantic search in a self-hosted engine. It's for developers building searchable apps, product catalogs or AI search over their own data. The C++ engine uses an in-memory architecture for low-latency results as users type.

python-audio-separator separates recordings into vocals, instrumentals and individual instruments on your own hardware. It's an open-source Python package under the MIT license, aimed at karaoke creators and developers who want audio separation in scripts or their own applications.

Buzz transcribes and translates speech on your own computer using OpenAI's Whisper. It's for people who need transcripts or subtitles from recordings, plus live captions from a microphone. Local transcription works offline; the optional OpenAI Whisper API sends audio to a cloud service.

Draw Things is an AI image generation app for iPhone, iPad and Mac that keeps generation on your device and works offline. It's for people who want to create and edit images without sending that work to a cloud service, including artists developing character concepts or trying out apparel designs.

SillyTavern is a locally installed LLM frontend for AI hobbyists who want detailed control over character chats and prompts. It builds on TavernAI as an independently developed fork and brings text models, image generation and voice into one interface. It's open source under AGPL-3.0.

LivePortrait animates a still portrait using facial expressions and head movements from a driving video. It's for creators who want to animate faces or edit motion in portrait videos on their own hardware. It works with realistic photos as well as portraits in oil paintings, sculptures and 3D renders.

MLC LLM is an open-source compiler and deployment engine for developers who want to run language models on their own hardware or inside apps. Its main distinction is the range of devices it targets: the same underlying engine, MLCEngine, serves desktop, browser and mobile deployments. The project uses the Apache 2.0 license.

Chatbox is an AI chat client for people who want local models and cloud providers in the same app. It connects to Ollama for local LLM use and supports GPT, Claude, Gemini, Grok and DeepSeek with your own API keys. Chatbox also offers its own hosted model service.

mistral.rs is an open source inference engine for running models on your own computer or self-hosted server. It's for developers building AI applications and people who want local chat, multimodal models and agent tools in the same runtime. The Rust project uses the MIT license.

DocsGPT is an MIT-licensed, open-source platform for teams that want AI search, assistants and agents over their own documents. It can run on your servers with local models, including fully air-gapped deployments where documents and questions stay inside your network. Answers include the source title and page number so readers can check the evidence.

pgvectorscale adds an index for large embedding datasets to PostgreSQL databases that use pgvector. It's for application developers and database administrators who want to keep AI similarity search in their existing database, with more control over search speed and storage use.

TRELLIS is a local AI model for generating 3D assets from images or text prompts, aimed at 3D artists and researchers exploring asset creation. It can produce meshes, radiance fields and 3D Gaussians from the same underlying representation, so you can choose an output suited to your rendering or editing work.

Text Embeddings Inference is a self-hosted server for developers who need text embeddings for search and retrieval applications. It serves models through a REST API on your own hardware and can run offline once model weights are downloaded. The Rust project is open source under Apache 2.0.

Moondream is a vision model for developers building software that needs to understand images. It can answer questions about a picture, write captions, locate objects, identify points and segment regions. The open-weight models can run on your own hardware, including in an air-gapped environment. The repository code is licensed under Apache 2.0; check each model checkpoint’s own terms for use.

Roboflow Inference is a self-hosted computer vision server for teams building camera and image analysis systems. It runs on your own computer, server, or edge device and combines model predictions with workflows for tracking, counting, measuring, and responding to events. Roboflow also offers hosted servers and a Serverless Cloud API, where processing runs on its infrastructure.

ebook2audiobook turns non-DRM ebooks into narrated audio with chapters and metadata, for readers who want audio editions of their own books. It runs locally on Windows, macOS and Linux, with Docker support and a browser interface built with Gradio. It's open source under Apache 2.0.

Lemonade is an open source local AI server for people who want to use models on their own hardware or connect them to apps and agents. It handles chat, coding, image generation, speech, transcription, and embeddings. A built-in interface lets you use those capabilities directly, while its server makes them available to other software.

Windmill turns ordinary scripts into APIs, background jobs and internal apps that teams can share through a browser. It's for developers building internal software, data pipelines or AI agents who want to keep their code and infrastructure under their own control. You can self-host it with Docker or Kubernetes, or use Windmill Cloud, where Windmill hosts the platform.

Letta is an open-source platform for AI agents that retain memory across conversations and use experience to improve over time. Formerly called MemGPT, it's for people building or using assistants that need continuity beyond a single chat. Agents can run locally or on a self-hosted server.

Local Deep Research is a self-hosted AI research assistant for people who need cited answers drawn from academic papers, the web and their own documents. It can produce a quick summary or pursue a complex question through repeated searches, then assemble a structured report. It's open source under MIT.

YuE is an open-source music generation project for musicians, songwriters and developers who want to turn lyrics and a style prompt into songs with vocals and accompaniment. Its YuE2 models create an editable melody and chord score before generating the recording, so you can review the composition and change musical details before hearing the result.

NVIDIA Container Toolkit lets Docker containers use NVIDIA GPUs on a Linux machine or server. It's for developers and server operators who need GPU acceleration for containerized workloads, including self-hosted AI software. The project is open source under the Apache 2.0 license.

Audiblez turns EPUB e-books into M4B audiobooks using Kokoro-82M text-to-speech on your own computer. It's for readers who want spoken versions of their books and control over the narrator, reading speed and sections included. The app is open source under the MIT license.

Playwright MCP lets AI agents control a browser by reading structured page information rather than interpreting screenshots. It's for developers building browser automation, exploratory tests or agent workflows that need to keep a browser session open across repeated actions. The server runs locally on macOS, Windows and Linux, or as a self-hosted service.

NVIDIA DCGM monitors and manages NVIDIA data-center GPUs on your own Linux servers. It's for infrastructure teams running GPU clusters, including those hosting AI workloads, who need to track hardware health, investigate slow jobs and control power use. It supports x86_64 and aarch64 (SBSA) systems.

OneTrainer is an open-source application for training diffusion models on your own machine, with dataset preparation and model previews in the same interface. It's for people adapting image or video models with their own training data. It runs on Windows, macOS and Linux under the AGPL-3.0 license.

bitsandbytes is an open-source Python library for developers who need to fit large language model inference or fine-tuning into less memory on their own hardware. It works with PyTorch and carries the MIT license. Its focus is the memory cost of model weights and training, rather than a chat interface.

Xinference serves language, speech and multimodal models through a shared API on your own computer or servers. It's an open source platform under Apache 2.0 for developers and researchers who want to build applications around models they host. You can also deploy it on cloud infrastructure.

llama-swap is a self-hosted proxy for people running several AI models on their own hardware. It starts the model server a request needs and swaps out another when necessary, so you don't have to keep every model loaded or manage separate API connections in your apps.