llama.rn

A React Native library runs GGUF models locally on iOS and Android through llama.cpp, with GPU acceleration and compatible image and audio models.

llama.rn is a library for developers adding local LLM features to React Native apps on iOS and Android. It brings llama.cpp inference onto the device, so an app can process model requests on the user's hardware.

Text generation and search: Run GGUF models for text completion or document reranking, including jina-reranker-v2-base-multilingual-GGUF and bge-reranker-v2-m3-GGUF.
Image and audio understanding: Compatible multimodal models can interpret images and audio with matching mmproj projector files, using local files or base64 inputs.
Structured responses and tool calling: JSON schema and GBNF constraints control output structure, while Jinja templates support model calls to app functions.
Concurrent requests: The library can handle multiple generation requests and queue work when its available processing slots are occupied.

The project is open source under the MIT license and requires React Native's New Architecture. GPU acceleration uses Metal on iOS and OpenCL on compatible Android devices, with Qualcomm Adreno 700-series and newer hardware supported and tested. In the iOS runtime on Apple Silicon, Metal is the supported inference path because CPU-only inference has a known crash.

Multimodal processing needs more memory than text generation. Image and audio inputs must come from local files or base64 data rather than HTTP URLs; supported audio formats include WAV and MP3.