The State of AI-Driven 3D Modeling: Analyzing the OpenSCAD LLM Benchmark

The intersection of Large Language Models (LLMs) and Computer-Aided Design (CAD) is moving from simple geometry to complex architectural reconstruction. A recent benchmark focusing on OpenSCAD—a code-based 3D modeling tool—has put several top-tier AI agents to the test by challenging them to recreate the Pantheon. The results highlight a significant leap in spatial reasoning, but also reveal a deep divide between benchmark success and the actual user experience of the tools.

The Pantheon Benchmark: Antigravity 2.0 Takes the Lead

The core of the benchmark involved using LLMs to generate OpenSCAD code capable of producing a 3D model of the Pantheon. OpenSCAD is uniquely suited for this task because it is not a visual drag-and-drop interface but a functional programming language for 3D objects.

According to the results, Antigravity 2.0 emerged as the top performer. The most striking detail was its ability to implement the Pantheon's signature interior ceiling pattern—the repeated square coffers visible through the oculus. This level of detail suggests a sophisticated ability to either synthesize visual references with architectural knowledge or leverage internal training data to fill in gaps that weren't explicitly visible in the provided reference images.

The 'Vibe Coding' vs. Engineering Divide

While the benchmark results are impressive, the community response reveals a critical tension between "vibe coding" (generating something that looks correct) and actual engineering precision. Several developers and engineers have raised concerns about the validity of the benchmark:

  • Training Data Leakage: Some critics argue that because the Pantheon is one of the most documented buildings in history, the model may not be "reasoning" about the architecture but simply recalling existing OpenSCAD models or detailed descriptions from its training set.
  • One-Shot vs. Iteration: A common critique is that a single-attempt benchmark doesn't reflect real-world CAD work. As one user noted, the real challenge isn't one-shotting a famous building, but the iterative process of debugging a part to fit a specific physical tolerance.
  • Manufacturability: There is a call for benchmarks to separate "syntax validity" (does the code run?) from "manufacturable output" (can it actually be 3D printed without failing?).

Real-World Applications: From Grommets to Enclosures

Despite the skepticism surrounding benchmarks, individual user reports suggest that LLMs are becoming genuinely useful for functional 3D printing. Users have reported success in several niche areas:

  • Rapid Prototyping: One user successfully used Claude to create a parametrized OpenSCAD model of a bike cable routing grommet based on photos and digital caliper measurements, achieving a near-perfect fit on the first try.
  • Complex Enclosures: Another user described using an LLM to build a printable enclosure for a power supply unit (PSU), utilizing technical spec sheets and diagrams to determine screw holes and mounting points.
  • Educational Onboarding: For those with no prior CAD experience, LLMs are acting as a bridge, teaching users that 3D modeling can be approached like software engineering.

The Tooling Gap: Models vs. Products

A recurring theme in the feedback is the frustration with the delivery of these models. While the underlying Gemini or Claude models are powerful, the software wrappers (the IDEs and CLIs) are often seen as unstable.

Users of Antigravity 2.0, in particular, complained about broken TUIs (Text User Interfaces), aggressive usage limits, and deployment bugs. This highlights a fundamental truth in the current AI era: the model is often far ahead of the product. As one developer put it, "the models are the product," implying that the user experience of the IDE is an afterthought compared to the raw intelligence of the LLM.

Conclusion: The Path to Professional CAD

The shift toward code-based CAD like OpenSCAD and CadQuery makes AI integration seamless. However, for AI to move from "impressive demos" to "professional engineering tools," it will need to move beyond recreating known landmarks and toward solving functional, constrained problems. The next frontier will likely be the integration of unit tests for 3D geometry—asserting that a profile doesn't exceed a 45-degree slope or that internal volumes are precisely calculated—bringing the rigor of software engineering to the world of physical manufacturing.

Sources