LiteRT-LM
An open-source framework for running LLMs locally on mobile, desktop and browser apps, with LiteRT acceleration and Apache 2.0 licensing.
LiteRT-LM is Google's open-source framework for developers building apps that run language models on the user's device. It uses LiteRT for local inference across mobile devices, desktops and browsers, with GPU and NPU acceleration. The project uses the Apache 2.0 license.
Supported models include Gemma, Llama, Phi-4 and Qwen. Beyond text generation, the framework handles vision and audio inputs, plus function calling for AI agents that use tools. EmbeddingGemma 2 provides on-device embeddings for text, images, video and audio in a shared vector space, which supports semantic search across those media types.
Platform coverage includes Android and iOS, along with Linux, macOS, Windows and Raspberry Pi through the Python APIs. Browser apps use JavaScript or TypeScript with WebGPU acceleration. Developers can also work through Kotlin and C++ APIs, while the community-maintained flutter_gemma package supports Flutter apps. Native Swift APIs target iOS and are in early preview.
The framework works with .litertlm model files and provides a builder for converted LiteRT models. Its command-line tools support model imports and can expose an OpenAI-compatible embeddings endpoint. Developers can inspect a model's capabilities and runtime requirements before loading it. LiteRT-LM also powers on-device generative AI features in Chrome, Chromebook Plus and Pixel Watch.
