GPT-5.6 Luna vs. GPT-6 Astra for Code Review

Executive Summary

GPT-5.6 Luna is a cost-effective alternative for general code correctness reviews, finding 75% as many verified bugs as GPT-6 Astra while costing only 3.6% as much. However, Luna exhibits a significant performance gap in security-sensitive areas, specifically authentication and permission logic, making it unsuitable for high-risk code review without human or high-capability model oversight.

Performance and Cost Comparison

In a benchmark of 50 public pull requests from the AI-Code-Review-Evals organization (including code from Cal.com, Sentry, Discourse, Keycloak, and Grafana), GPT-5.6 Luna and GPT-6 Astra were tested on their ability to identify real bugs while ignoring style and documentation issues.

Key Metrics

Metric	GPT-5.6 Luna	GPT-6 Astra
Verified Bugs Found	69	92
Precision (Verified/Total Findings)	74%	96%
Total Cost (50 PRs)	$0.20	$5.66
Cost per Verified Bug	$0.0030	$0.061
Mean Time per Review	23s	36s

Luna's cost advantage is substantial: a single review costs approximately $0.0041 compared to Astra's $0.113. Per verified bug, Astra is 20x more expensive than Luna.

Analysis of Model Weaknesses

Security and Authorization Gaps

Luna's performance degrades sharply when reviewing identity and access management (IAM) code. In the Keycloak codebase—which focuses on authentication and permission logic—Luna found only 6 verified bugs compared to Astra's 14.

On a bug-class basis, Luna identified only 9 of 24 verified security bugs, whereas Astra identified 19. The benchmark highlights that cheap models struggle with "global" logic—where a bug is not visible on a single line but emerges from how a change affects the overall permission model.

Precision and Noise

Luna's precision is significantly lower than Astra's. With a 74% precision rate, approximately one in four of Luna's findings is a false positive. This creates a "noise" problem for developers, who may begin to ignore AI suggestions if the false positive rate is too high.

Complementary Strengths

Despite Astra's overall superiority, Luna found 25 verified bugs that Astra missed. Of the 143 total verified bugs in the pool, 44 were found by both models. Combining both models on every pull request would have identified 117 bugs (82% of the total) for a total cost of $5.86.

Community Insights and Counterpoints

Industry practitioners provided several critical perspectives on the application of these models in real-world workflows:

The Cost of False Positives: Multiple developers argued that the $0.10 price difference per PR is negligible compared to the engineering cost of triaging false positives. One user noted that "dealing with false positives is expensive."
Integration Strategy: Some suggested that AI should not be piped directly into CI/CD pipelines. Instead, AI should be used by the PR author or a human reviewer to filter noise before it reaches the author.
Agentic Workflows: Experienced users recommended a "coordinator" approach. For example, using a high-capability model (like Fable) to coordinate multiple specialized sub-agents (like Luna) focused on specific areas such as complexity or performance, then synthesizing the results.
The "Reasonableness" Trap: One developer warned against trusting low-capability models because they are "far better at sounding reasonable than being reasonable," which can lead to human complacency and the accumulation of "small" errors that eventually create systemic failures.

Methodology

The study used 50 public pull requests where defects were intentionally introduced. Findings were verified by a dual-judge system: an issue was only counted as "verified" if both GPT-6 Astra and GPT-5.6 Sol agreed it was a real bug. The models were provided with the diff and no additional repository history or call graphs. Prices were based on Luna's $0.20/$1.20 (input/output) and Astra's $10/$50 per million tokens.