Cua Driver overview: background agents and CUABench

The Cua team presents Cua Driver as an open source tool for background computer use on macOS, Windows and Linux. An AI agent observes a window through an accessibility tree and a screenshot, then attempts an accessibility action before falling back to a background pixel click. The speakers say this lets agents work without taking over the user's screen, but note that background behavior differs across operating systems. Running the driver on a laptop does not establish that model inference runs locally.

CUABench evaluates tasks through a setup function, a reference trajectory and an evaluator that checks the environment. The team reports a 4K benchmark pass rate increase from 62% to 80%, with 34% fewer tokens, after replacing the agent's built-in computer tool with Cua Driver. They attribute the improvement mainly to observing one window rather than the whole desktop. Their circuit-design evaluation was harder: the best tested agent fully passed six of 25 tasks, all edits to existing schematics. Tasks starting with a blank schematic had a 0% success rate.

Cua Fleet addresses GPU idle time during sandbox startup and resets with a pool that scales to demand. The talk lists Windows, Linux and Android sandboxes, with macOS planned. The Android discussion describes background tool use through the activity framework and containerized workloads, with limits on direct GUI control.