ldraw-nova: Open-Source Agentic LEGO Model Generator

Overview

ldraw-nova is an open-source agentic tooling system that allows AI agents to design buildable, physical LEGO models. Instead of attempting to place bricks manually through a chat interface, the system provides agents with a specialized toolset and documentation, enabling them to generate Python scripts that output LDraw source code—a low-level assembly language for LEGO CAD models.

This approach solves a fundamental limitation of LLMs: their struggle with precise geometry and spatial mathematics. By shifting the agent's task from calculating coordinates to writing code that calculates coordinates, ldraw-nova enables the creation of complex 3D models that can be rendered in 3D viewers, exported as glTF files for Blender, or viewed in VR via Meta Quest 3.

The "Code-as-Geometry" Architecture

The core innovation of ldraw-nova is the realization that LLMs are significantly more proficient at generating Python code than they are at performing raw geometric calculations. The system employs a multi-stage pipeline to move from a natural language prompt to a physical model:

Planning: The agent analyzes the prompt and references instructions.md and other design guides to plan required parts, submodels, and overall aesthetics.
Plan Generation: The agent creates a plan.json file that describes the model's geometry and structure.
Script Generation: The agent writes a generator.py script based on the plan. This script acts as a compiler, translating the high-level plan into the specific LDraw assembly language.
Execution: The generator.py script is executed to produce the final .mpd or .ldr LDraw source files.
Iterative Refinement: The agent renders images of the current model, inspects them, and adjusts the Python code to fix gaps or positioning errors.

Technical Implementation and Tooling

ldraw-nova is deployed as a dockerized web application requiring two repositories: ldraw-nova (the core logic) and ldraw-nova-docker (the configuration and web interface).

Key Technical Components

Semantic Search: The system uses jev-rerank, a semantic search tool backed by TypeSafe's Jev System One AI model, to help agents find suitable LEGO parts and example models. If a TypeSafe API key is unavailable, the system falls back to Full Text Search (FTS).
Headless Rendering: Agents use headless rendering to visually inspect their progress and iteratively refine the model.
Collision and Gap Detection: Specialized tools allow agents to detect when parts are overlapping or not properly connected.
LDraw Integration: The system outputs standard LDraw files, making the models compatible with industry-standard tools like LDView, LeoCAD, and Studio.

Agent Knowledge Base

The system provides agents with a comprehensive library of guides to ensure structural integrity and visual appeal, including:

Vehicle and Spaceship Workflows: Specific strategies for building complex transport models.
Structural and Mechanism Workflows: Instructions for using LEGO Technic parts to create functional machines.
Visual Design Guides: Documentation on improving shape, color, and detail.
Validation Tools: Guides on snapping and geometry checks to ensure the model is physically buildable.

Current Limitations and Development Roadmap

While the system demonstrates the capability of high-end models (specifically GPT-6 Astra and Claude Opus 5.5) to "vibe-code" complex geometry, several challenges remain:

Model Cost and Speed: Generation is currently slow and expensive, requiring high-end LLMs to achieve correct results for large models.
VR Performance: The Meta Quest 3 integration currently suffers from performance and model-handling issues.
Domain Expansion: The author aims to improve the generation of minifigures (humans/animals) and more complex Technic engines.
Low-End Agent Support: There is an ongoing effort to adapt tooling and documentation so that smaller models (e.g., Claude Haiku) can produce viable results.

Community Insights and Perspectives

Discussion around ldraw-nova highlights a broader trend of using LLMs for physical design. Some users have reported similar success using Claude with FreeCAD and MCP to design 3D-printed cases and milled aluminum parts, noting that the primary struggle for AI remains the "assembly logic"—understanding how parts physically fit together in a constrained space.

Other community members have noted the potential for this technology to evolve into robotics, though they emphasize that a physics engine would be required for truly functional robotic designs. There is also a philosophical divide among users, with some viewing the automation of LEGO design as a loss of the "pure-imagination" aspect of the hobby.