Needle 3: local tool calling demo and setup overview

Needle 3 is Cactus Compute's small model for on-device tool calling, structured extraction and text embeddings. The presenter describes it as open source under Apache 2.0 and cites binary sizes of 8 to 29 MB. General chat capability is a tradeoff: the focus is turning requests into actions and structured outputs.

The sandbox demonstration uses a simulated home with predefined controls for lights, a thermostat and a door lock. The presenter asks it to turn on the bathroom light and lock the front door, then shows the corresponding state changes. These examples illustrate tool selection within a defined environment rather than unrestricted device control.

The architecture discussion covers Simple Attention Networks, CQ2 quantization and selectable subnetworks with two to 20 layers. The presenter cites 29 to 121 million parameters and published Raspberry Pi 5 speeds of 400 to 4,000 tokens per second for decoding and 1,000 to 10,000 for prefill. These are reported figures, not measurements from the sandbox demonstration. Comparisons with larger models concern specific tasks and tuning conditions.

Suggested uses include offline smart home controls, robots and phone assistants. The closing setup overview introduces the cactus-needle package, importing needle, defining a weather tool and invoking an agent run. It is a brief introduction rather than a complete deployment walkthrough.