Cua and CUA-S1: Open-Source Infrastructure for AI Computer Use

Cua is an open-source ecosystem designed to provide AI agents with the necessary infrastructure to interact with native desktop applications and browsers across macOS, Windows, and Linux. The framework introduces "Computer-Use 2.0," a paradigm where agents seamlessly transition between executing code, calling APIs, and interacting with graphical user interfaces (GUIs) within a single task.

CUA-S1: Specialized "System 1" Decision Models

CUA-S1 is a family of small, specialized models designed for fast, bounded decisions—referred to as "System 1" decisions—rather than general-purpose planning or token-by-token text generation. These models are intended to complement a general-purpose agent's reasoning capabilities by handling specific, repetitive interface interactions.

Focus on Decision Scoring

The initial research profile for CUA-S1 focuses on forms. Instead of generating a response, these models score decisions based on structured interface elements and document values. This approach allows application code to order the actions, while the optional Cua Driver handles the actual execution with explicit action boundaries.

Availability and Licensing

CUA-S1 is currently a source-only research release. The Python model code, synthetic data generation, and evaluation tools are MIT-licensed and available on GitHub, while the model weights and datasets (such as CUA-S1-FORMS) are hosted on Hugging Face.

Cua Driver: Cross-Platform Desktop Automation

Cua Driver provides the tools required for agents to inspect and operate native desktop applications and browsers. It supports macOS, Windows, and Linux, and offers background delivery, allowing agents to interact with apps without moving the user's pointer or taking focus, provided the platform and application support it.

Agents can connect to the Cua Driver via a CLI, Model Context Protocol (MCP), or typed SDKs. It is designed to integrate with existing agent frameworks such as Claude Code, Codex, Cursor, and OpenClaw.

Infrastructure for Agent Deployment

Cua provides two primary methods for provisioning the environments where agents operate:

Cua Fleets

Cua Fleets allow users to provision isolated cloud desktops. Using the Sandbox SDK, developers can claim a desktop from a pool, run commands, capture screenshots, and interact with applications in a sandboxed environment.

Lume

Lume is a tool for creating and managing local macOS and Linux VMs on Apple Silicon using Apple's Virtualization.Framework. This allows developers to run agents in isolated local environments, such as creating a macOS Tahoe VM from an Apple restore image and connecting via SSH.

Cua Bench: Evaluation and Training

Cua Bench is a framework for building computer-use tasks, evaluating agent performance, and exporting trajectories for training. It allows for the creation of simulated tasks that do not require a VM, Docker, or model API keys, enabling rapid testing and verification of agent rewards.

Community Insights and Technical Discussion

Community members have noted the architectural shift toward decision scoring over text generation as a particularly interesting approach to computer use. Discussions have also emerged regarding the potential for a hierarchy of specialist models:

Would the idea be to have many specialist models (forms, wikipedia, final cut, etc) and have a parent model choose which is best... with the idea an llm would give a larger goal and trigger this cascade of specialists to quickly do the task?

This suggests a potential evolution where a high-level planner (System 2) delegates specific GUI interactions to a suite of specialized CUA-S1 models (System 1).