drawgent – AI coding agent on a live Excalidraw canvas

TL;DR

drawgent connects your own Claude Code, Codex, or opencode instance to an Excalidraw whiteboard, allowing you to type AGENT: notes or use laser zones on the canvas and have the LLM edit the diagram live. This creates a bidirectional workflow where visual brainstorming directly drives code‑aware diagram updates.

What drawgent does

Live canvas editing – The agent watches a screenshot + scene snapshot, issues MCP tool calls (add, update, delete elements), and marks the note DONE: when finished.
Multiple back‑ends – Supports Claude Code, Codex, and opencode via their respective ACP adapters. No agent binaries are bundled; you supply your own CLI.
Two interaction modes –
1. Chat panel on the right side of the web UI for typed requests.
2. On‑canvas triggers (AGENT: text notes or laser zones) that automatically invoke the agent after a short pause.
Version‑controlled diagrams – With --diagram the canvas is persisted as a clean .excalidraw file that can be committed to Git and will sync back to the live view on changes.

Quick start checklist

Install and log in to one of the supported agents (claude, codex, opencode).
Run drawgent setup <agent> – the command validates the CLI, login status, ACP bridge, canvas tools, and Chrome renderer, writing ~/.config/drawgent/config.toml.

In your repository directory, start a session:

drawgent up               # launches a new agent session and opens the canvas
# or attach to an existing session
drawgent up --attach

Optionally persist the diagram:

drawgent up --diagram docs/architecture.excalidraw

Architecture at a glance

drawgent architecture

Browser UI – React‑based editor with chat panel and laser‑zone handling.
drawgent server (Rust) – Manages ACP sessions, MCP tool server, Chrome renderer, and WebSocket sync.
Agent process – Forked or live‑attached Claude/Codex/opencode session that receives MCP calls.
Chrome headless – Renders the canvas for vision (get_screenshot) and element geometry.

Core commands

Command	Purpose
`drawgent setup <agent>`	Validates environment, installs ACP adapters, configures Chrome, writes `config.toml`.
`drawgent up`	Starts a new agent session, launches the UI, and serves the canvas on `127.0.0.1:7300`.
`drawgent up --attach [id]`	Connects the UI to an already‑running agent session (Claude fork, opencode live TUI, or Codex queue).
`drawgent up --diagram <path>`	Persists the live canvas to a `.excalidraw` file that is git‑tracked.
`drawgent up --room <url>`	Joins an existing Excalidraw.com room as a collaborative bot (`🤖 Agent`).
`drawgent mcp`	Stand‑alone MCP server exposing canvas tools for custom scripts.
`drawgent serve`	Low‑level server for containerised deployments; you specify agents via `--agent` flags.

Using the canvas effectively

AGENT: notes – Place the prefix next to a shape or draw an arrow to it. After you stop typing, drawgent waits ~2.5 s, resolves the target, and replies with a green DONE: note.
Laser zones – Activate Excalidraw’s laser (K), circle a region, and a red outline appears. The chat panel opens with a chip listing the covered elements (e.g., 🔴 Laser zone · API, Redis). The next message is sent together with the zone data; the outline disappears once the agent finishes.
Permission model – By default drawing tools are auto‑approved (canvas permission). Other actions (e.g., file writes) require explicit approval in the chat panel.

Persistence and collaboration

Git‑tracked diagrams – The .excalidraw file contains only live elements, is pretty‑printed, and updates only when the canvas changes, yielding clean diffs.
Room mode – When you supply an Excalidraw.com room URL, drawgent joins as a collaborator. Traffic is end‑to‑end encrypted with the room key, and the local UI mirrors the remote room.
Docker option – docker compose up --build runs a canvas server (drawgent + Chromium) without any agents, useful for shared whiteboards or room bridges.

Build and distribution

From source – npm ci && npm run build builds the web assets, then cargo install --path . produces a binary.
Static Linux – make musl creates a fully static x86_64 binary; set MUSL_TARGET for ARM.
macOS universal – make darwin cross‑builds a universal binary (Intel + Apple Silicon) from Linux using Zig; the arm64 slice is ad‑hoc signed.
Dependencies – Chrome/Chromium is required for rendering; the installer can download a headless shell or use the system package manager.

Community insights from Hacker News

seemaze points out that Excalidraw already provides a first‑party MCP endpoint and server (https://mcp.excalidraw.com). drawgent’s custom MCP layer adds live‑attach capabilities for Claude and Codex that the official endpoint does not yet expose.

armanj shares a personal workflow using Mermaid inside Obsidian, noting that while Mermaid is “agent‑friendly,” drawgent offers a richer visual feedback loop by editing the canvas directly.

4ndrewl emphasizes that the real value of diagramming lies in the thinking it provokes, a principle that drawgent supports by letting the LLM iterate on sketches instantly.

raesene9 mentions the Excalidraw Obsidian plugin as an alternative; however, drawgent’s ability to persist diagrams in the repo and to attach to existing agent sessions provides tighter integration with codebases.

bad_haircut72 recommends the whiteboard‑MCP service (whiteboard-mcp.com) as another option for architecture diagram generation, highlighting the growing ecosystem of MCP‑compatible tools.

brumar released a similar project, whiteboard‑agents, which can be examined for implementation ideas and contrasts with drawgent’s focus on Excalidraw and multi‑agent support.

ramoz argues that HTML‑based whiteboards could give agents richer semantics than JSON‑heavy formats like Excalidraw, but drawgent already demonstrates that LLMs can manipulate Excalidraw’s JSON model effectively.

Limitations to be aware of

Chrome dependency – The current renderer relies on a Chrome/Chromium binary; a native renderer is planned but not yet available.
Claude attach model – Because Claude Code lacks a public injection API, drawgent forks a new Claude session for each canvas attachment.
Codex live attach – Implemented but not fully tested with a logged‑in Codex account at the time of writing.
Single scene per workspace – Only one live canvas and one .excalidraw file are supported; images or other assets are not synced.

Getting involved

Run the test suite – cargo test covers fractional indices and room encryption; the Node scripts (smoke.mjs, e2e-browser.mjs, etc.) validate end‑to‑end canvas interactions.
Contribute – The repository is primarily Rust (≈ 77 % of the code) with a small JavaScript/HTML front‑end. Issues around native rendering, multi‑scene support, or broader MCP compatibility are good entry points.
Share workflows – Community members have already posted alternative whiteboard tools, Mermaid‑based plugins, and custom MCP servers; sharing concrete use‑cases helps the project mature.