LiteRT Torch
A Python library that converts PyTorch models for on-device AI with LiteRT. Runs on Linux and uses the Apache 2.0 license.
LiteRT Torch is a Python library for converting PyTorch models into .tflite files for the LiteRT runtime. Developers can prepare models on Linux and deploy them into Android, iOS and IoT applications that run inference on the device. Code is licensed under Apache 2.0.
The PyTorch converter builds on torch.export() and supports a broad set of Core ATen operators. It is a beta release. General conversion has broad CPU coverage and initial GPU and NPU support; the actual supported operations depend on the target. Exported models run through the separate LiteRT runtime.
The Generative API helps author and quantize mobile-oriented transformer models for LiteRT-LM. This API is alpha, currently supports CPU and GPU, and has planned NPU support. It should not be treated as a stable, universal converter for every transformer model.
Install litert-torch on Linux with Python 3.10 or later; the README recommends Python 3.11 and documents PyTorch and TensorFlow dependencies. Packaging a converted model and tokenizer into a .litertlm container uses the litert-lm-builder tool provided through the nightly package.
