AvaLovelace1/BrickGPT

[ICCV 2025 Best Paper] Official repository for BrickGPT, the first approach for generating physically stable toy brick models from text prompts.

What it solves

BrickGPT addresses the challenge of generating toy brick models from text prompts that are not only visually representative of the prompt but also physically stable and actually buildable by humans.

How it works

The system is based on a fine-tuned version of the Llama-3.2-1B-Instruct model, trained on a custom dataset called StableText2Brick. It converts text descriptions into a brick-by-brick text format (specifying height, width, and coordinates). To ensure the resulting structures are viable, it can utilize physics-based stability analysis (via Gurobi) or a simpler connectivity-based method to reject and regenerate unstable or colliding bricks.

Who it’s for

This tool is designed for researchers in AI-driven 3D generation, designers of physical toy models, and hobbyists interested in automated brick-based construction.

Highlights

  • Physically Stable: Specifically designed to create structures that can be assembled in the real world.
  • Text-to-Brick Generation: Generates detailed brick layouts (LDraw format) directly from natural language prompts.
  • Stability Analysis: Integrates with Gurobi for high-accuracy physics-based stability verification.
  • Comprehensive Pipeline: Includes capabilities for texturing, coloring, and converting 3D meshes into brick structures.