DwarfStar

DwarfStar is a local LLM inference engine for people who want to run large DeepSeek, GLM and Qwen models on their own computers or servers. It focuses on selected models and hardware setups, and requires the project's own GGUF files rather than accepting arbitrary GGUF downloads.

• Chat and coding: Use a terminal chat interface or the built-in coding agent, which runs inference directly and can save and resume text sessions.
• Self-hosted serving: Connect Pi, OpenCode, Codex CLI or Claude Code through its HTTP server, with multiple concurrent sessions on CUDA systems.
• Models larger than RAM: SSD streaming lets supported models exceed available memory, while multi-GPU and multi-machine execution can combine hardware resources for larger workloads.
• Image input: GLM 5.3 Flash and Qwen3.8 Flash Next can process images with matching vision encoders.

DwarfStar is open source under the MIT license. It runs locally through Metal on Apple Silicon Macs, CUDA on NVIDIA hardware including DGX Spark and Ada Lovelace cards, and ROCm on Strix Halo systems such as the Framework Desktop.

Memory needs are substantial: the stated Mac starting points are 64 GB for Qwen Flash Next and 96 GB or more for DeepSeek and GLM, with SSD streaming available for smaller machines. Qwen's disk-resident n-gram data requires a fast local SSD. Model support can change, and the project may remove models when replacements arrive. The native coding agent can't save sessions containing images.