abi/screenshot-to-code

screenshot‑to‑code

What it does – Takes a UI screenshot, a Figma mock‑up, or even a short screen‑recording and, with the help of large language models, produces clean front‑end code that you can run immediately. The generated code can include the original assets (logos, images) extracted from the screenshot.

Supported front‑ends – HTML + Tailwind, HTML + CSS, React + Tailwind, Vue + Tailwind, Bootstrap, Ionic + Tailwind.

AI back‑ends – By default it tries the newest Google Gemini (3 Flash, 3.1 Pro), OpenAI GPT‑5.5/5.4 Mini, Anthropic Claude Opus 4.6‑4.8, and a Replicate image model (z‑image‑turbo). You can mix‑and‑match providers; the app will pick the strongest model available for each step.

How it works

1. Asset extraction – Gemini analyses the screenshot, pulls out real logos and images, and can remove backgrounds via Replicate.
2. Code generation – The chosen LLM receives a description of the layout and the target stack and returns the corresponding markup, CSS/Tailwind classes, and component code.
3. Optional preview – A headless Chromium instance (via Playwright) renders the generated page so the system can compare the output with the original screenshot and flag obvious mismatches.
4. Video mode – When you upload a short screen‑recording, the backend extracts key frames and builds a prototype that mimics the interaction flow.

Running it

• Hosted demo – Visit https://screenshottocode.com for an instant try‑out.
• Self‑hosted – Clone the repo, provide at least one API key (OpenAI, Anthropic, or Gemini), and start the FastAPI backend (poetry install && poetry run uvicorn main:app …). Then launch the Vite‑React frontend (pnpm install && pnpm dev).
• Docker – docker‑compose up --build after placing an .env with your keys.

Key configuration

Variable	Needed for	What it enables
OPENAI_API_KEY	Any	GPT‑5.5 / 5.4‑Mini code generation
ANTHROPIC_API_KEY	Any	Claude code generation
GEMINI_API_KEY (recommended)	Any	Gemini code generation, asset extraction, video mode
REPLICATE_API_KEY (recommended)	Optional	Image editing, background removal, image generation

FAQs & troubleshooting – The README links to a troubleshooting guide and a common‑error fix in the issue tracker. You can also set an OpenAI proxy via OPENAI_BASE_URL if you’re behind a firewall.

Examples – The repo shows side‑by‑side comparisons for NYTimes, Instagram, and Hacker News pages, demonstrating that the tool can recreate complex news‑article layouts, social‑media feeds, and discussion‑board interfaces.

Who might use it – Front‑end developers who want a quick scaffold from a design, product teams prototyping UI ideas, or anyone curious about AI‑assisted code generation.