OfflineLLM

An offline AI chat app for Android that runs GGUF models through llama.cpp, with optional Vulkan GPU acceleration and no network permissions.

OfflineLLM is an on-device AI chat app for Android users who want to run models locally without sending conversations to a cloud service. It works fully offline and has no network permissions.

  • Your own models: Use GGUF models through llama.cpp, including Gemma 3, Qwen3.5 and native chat support for Gemma 4 E2B and E4B.
  • CPU or GPU processing: Run inference on the CPU or use optional Vulkan GPU acceleration, with automatic CPU fallback if GPU loading fails.
  • Chat and translation: Read responses as they generate, view Markdown formatting, translate text and hear replies through Android's text-to-speech system.
  • Conversation management: Search and rename chats, export or restore them as JSON, and check a context indicator that shows how much conversation capacity you're using.
  • Privacy controls: Protect access with an optional biometric lock, keep settings encrypted and overwrite files before deletion; the app doesn't log prompts or responses.

The app requires Android 13 or later on a 64-bit ARM device, and you supply the model files. Available RAM determines which models fit: recommendations range from small Gemma models for phones with 2 to 4 GB of RAM to Gemma 4 E4B for devices with 8 GB. GPU performance depends on the phone, and some Mali devices can generate replies faster on the CPU.

Similar to OfflineLLM