UI-TARS Desktop and Agent TARS: modes and setup

This English summary uses the full Japanese captions for the UI-TARS explainer. UI-TARS Desktop is introduced as ByteDance's open source GUI agent for Mac and Windows. The speaker explains how it sends screenshots to a vision-language model, identifies screen coordinates, and executes mouse or keyboard actions. It repeats that loop after each action, allowing it to work with interfaces that lack an automation API.

The overview separates two choices: what the agent controls and where it operates. Computer Operator covers desktop applications and files; Browser Operator focuses on web tasks. Local mode controls the user's PC, while remote mode uses a cloud environment. The speaker says the remote trial ended in August 2025 ; this is the presenter’s account of remote availability.

Running the desktop app locally does not by itself keep screenshots private. That depends on the model configuration. The video discusses UI-TARS 1.0 and 1.5 models available through Hugging Face for self-hosted inference, plus a cloud model option. Setup covers the provider URL, API key and model name.

The speaker describes mouse movement as a way to pause automation and regain control. HTML reports contain action logs and screenshots for review. Repeated form entry and research across websites are suggested uses; screenshot processing introduces delays that limit tasks needing immediate reactions.

Agent TARS is also introduced as a general AI agent with a CLI and Web UI. The video describes launching it through npx with Node.js and connecting external tools through MCP.