KPMG Pulls AI Usage Report Due to LLM Hallucinations
KPMG has retracted its professional services report, "Redefining excellence in the age of agentic AI," after discovering that the document contained AI-generated hallucinations. This incident highlights the critical failure of quality assurance processes in high-value professional services reports produced with the help of generative AI.
The Retraction of the Agentic AI Report
KPMG pulled the report titled "Redefining excellence in the age of agentic AI" following the discovery of hallucinations—instances where the AI model generates plausible-sounding but factually incorrect information. This retraction serves as a warning to firms using Large Language Models (LLMs) to produce market research and strategic advisory reports.
Industry-Wide Patterns of AI Hallucinations in Consulting
This failure is not an isolated incident within the professional services sector. Community discussion suggests that other major consulting firms have faced similar issues. For example, users pointed to a previous incident involving Deloitte in Australia, where AI-generated inaccuracies occurred in government-related work.
Technical Mitigation Strategies for AI-Generated Content
To prevent hallucinations in technical or financial reports, the implementation of automated validation layers is essential. A common technical approach is to use a "sub-agent" architecture where a separate AI agent is tasked specifically with validating all references, figures, and citations against source material.
As noted by technical contributors in the community:
The crazy thing is the level of effort to say, "have a sub agent validate all references and figures" is so low. It would have prevented 99% of the face palms.
This cross-checking mechanism reduces the risk of the model using outdated figures or fabricating citations, which is a critical requirement for reports sold at high price points.
Analysis of the AI Hype Cycle
The retraction of the KPMG report underscores a systemic issue where the generative AI hype cycle is accelerating. Some observers argue that the tech itself is writing its own hype, creating a feedback loop where the AI-generated content is used to promote the same technology's capabilities, often at the cost of factual accuracy.