e2b-dev/surf

What it solves

Surf provides a way for AI agents to interact with a full virtual desktop environment. It solves the problem of limiting AI to simple API calls or text-based interactions by allowing an agent to use a computer like a human would—clicking, typing, and navigating through a Linux-based desktop in a secure, isolated sandbox.

How it works

Surf integrates three main components: a Next.js frontend, E2B's desktop sandbox, and OpenAI's Computer Use API. When a user provides a natural language instruction, the backend processes it via OpenAI to generate specific computer actions (such as clicks or keystrokes). These actions are executed within the E2B virtual desktop and the resulting visual updates are streamed back to the user's browser in real-time using Server-Sent Events (SSE).

Who it’s for

Developers and AI researchers who want to build or test agents that can operate a computer interface, automate complex multi-step workflows across different applications, and provide a visual feedback loop for AI-driven computer interaction.

Highlights

• Virtual Desktop Sandbox: Runs a secure, isolated Linux-based desktop in the cloud.
• Real-Time Streaming: Visual updates and AI reasoning are streamed to the UI as they happen.
• Conversational Interface: A chat-based system for sending instructions and managing the sandbox instance.
• OpenAI Integration: Leverages OpenAI's specialized computer use capabilities to translate natural language into desktop actions.