Cactus

Cactus is an on-device AI engine for developers building automation into mobile apps, wearables and embedded devices. Its Needle model handles tool calling locally, so a device can turn a request into an action without an internet connection. The focus is small devices, including smart home hardware, robots and microcontrollers.

Needle uses compact, 2-bit models to fit automation into hardware with limited memory. The broader engine also handles chat, image input and speech transcription. It supports streaming responses, embeddings and retrieval over local data, with a vector index for search.

Model families tested with the engine include Gemma, Qwen and Liquid, plus Whisper and Parakeet for speech. Cactus runs on iOS, Android and Mac, with bindings for Swift, Kotlin, Flutter and React Native. Python and Rust bindings support other application integrations.

Local inference runs on the user's hardware. An optional hybrid mode routes difficult requests to a cloud service based on the local model's confidence, so requests sent through that path leave the device. Developers can choose local execution for offline automation or use cloud handoff for queries the local model struggles with.

The C++ engine includes its own quantization and ARM-optimized kernels, aimed at reducing the memory and processing demands of inference. Its API covers tool calling alongside speech and vision, allowing an app to use those capabilities through the same engine.