GPT-5.5 and GLM-5.2: Analyzing the Correlation Between Model Size and Hallucinations

Large Model Scale vs. Truthfulness

Increasing parameter count and training data scaling are yielding diminishing returns in model intelligence and may actively increase hallucination rates. While the largest proprietary models continue to lead in raw benchmark scores, open-weight models like GLM-5.2 (753B parameters, 40B active) are now performing within a narrow margin of the largest closed-weight models, suggesting a plateau in actual intelligence.

According to the AA-Omniscience benchmark, larger models are significantly more prone to hallucinating answers when they do not know the correct response, rather than abstaining. The hallucination rates for several leading models are as follows:

• DeepSeek V4 Pro (1.6T params): 94% hallucination rate
• GPT-5.5: 86% hallucination rate
• Fable 5: 48% hallucination rate
• Opus 4.8: 36% hallucination rate
• GLM-5.2: 28% hallucination rate

The "I Don't Know" Problem in Massive Models

Massive models often fail to recognize technical impossibilities or logical fallacies, leading them to generate confidently incorrect responses. This is attributed to training on vast volumes of factual, non-theoretical data where questions almost always have an answer, teaching the model to always provide one.

In a comparative test involving a complex Python architectural flaw (multiplexed I/O in a single-threaded task without yielding), GLM-5.2 identified the technical impossibility in 12 seconds using approximately 800 reasoning tokens. In contrast, DeepSeek V4 Pro spent over three minutes in a reasoning loop and utilized nearly ten times the reasoning tokens, only to produce a confidently incorrect solution.

Technical Counterpoints and Interpretations

Community analysis suggests that hallucination rates on the AA-Omniscience benchmark should be interpreted with caution, as they are conditional on the model not knowing the answer.

Accuracy vs. Abstention

Some analysts argue that a high hallucination rate does not necessarily mean a model is less useful. For instance, while GLM-5.2 has a lower hallucination rate (28%), it only answered 25% of questions correctly on the same benchmark. GPT-5.5 (xhigh) answered 57% correctly, suggesting that while it hallucinates more when it is wrong, it is correct more often overall.

Training Policy vs. Model Size

There is significant debate over whether model size is the primary driver of hallucinations. Several points were raised:

• Training Bias: Models are trained on curated datasets (like books) where questions typically have answers. They lack the "fear" or uncertainty calibration found in humans, who are trained to admit ignorance.
• Optimization Trade-offs: Hallucination rates may be a result of the training policy and RLVR (Reinforcement Learning from Verifiable Rewards) targets rather than parameter count. Some suggest that promoting "I don't know" as a valid answer during training could mitigate this issue.
• Model-Specific Issues: Some users reported that GLM-5.2, despite its lower hallucination rate, can be more prone to "straying" from requirements or making unfounded conclusions in specific coding tasks.

The Modern LLM Trilemma

The current state of AI development faces a trilemma between raw capability, uncertainty calibration (hallucination rate), and computational efficiency. The industry is shifting away from blind scaling of parameter counts and reasoning budgets, as evidenced by the recent US government restriction of Claude Fable 5 due to national security risks associated with a single jailbreak.

Moving forward, the selection and training of AI models must balance the ability to provide a correct answer with the ability to recognize when a solution is impossible or unknown.