The Case Against Using AI for Substantive Writing

Using AI to generate substantive text—such as blog posts, research reports, and thoughtful emails—is generally counterproductive because the act of writing is inextricably linked to the act of thinking. When authors delegate the drafting process to a Large Language Model (LLM), they bypass the critical cognitive effort required to identify gaps in their logic, refine their arguments, and ensure precision in their claims.

The Writing Process as a Thinking Process

Writing is not merely the act of recording pre-formed thoughts; it is a primary mechanism for developing those thoughts. Forcing ideas into structured sentences and paragraphs surfaces contradictions and research inadequacies that remain hidden during the outlining or brainstorming phases.

As noted by Paul Graham, putting ideas into words serves as a "severe test," often revealing that a writer does not understand a topic as well as they initially believed. Similarly, Clara Collier emphasizes that the transition from an outline to finished text is where a writer frequently realizes that certain points should not be juxtaposed or that a thesis is flawed. By using an LLM to bridge this gap, the author avoids the necessary cognitive struggle, resulting in a weaker final argument and a shallower understanding of the subject.

Subtle Inaccuracies and the "Convincing" Nature of AI Prose

AI-generated text is frequently characterized by a density of vague, uninformative, or subtly incorrect phrases that are difficult for the average reader to detect. Because LLMs are designed to be statistically probable and convincing, they often produce prose that sounds authoritative while lacking informational value.

Case Study: AI Chip Smuggling

In an analysis of a paragraph generated by Claude regarding AI chip smuggling, several systemic issues were identified:

Vagueness: Phrases like "export controls are only as strong as enforcement" provide little to no actionable information.
Subtle Errors: Describing AI chips as "compact" ignores the fact that smuggled goods are often entire servers, and the claim that global supply chains make smuggling "feasible" is imprecise.
Misleading Statistics: Providing a wide range of estimates (tens of thousands to hundreds of thousands) without specifying the year or the definition of "high-end chip" can mislead the reader, especially when the lower estimates are likely incorrect.
Empty Rhetoric: Closing sentences that describe a policy as being "hollowed out" often serve as "applause lights"—text that sounds important but conveys no new data.

Because AI writes in a way that is "maximally convincing," these errors are harder to spot than typical human errors. This creates a high cognitive load for the editor, who must meticulously vet every word to ensure accuracy.

The Implicit Contract Between Writer and Reader

There is an implicit contract in substantive communication: the reader offers their attention in exchange for the writer's genuine thought and effort. When a text is AI-generated and presented as human-written, this contract is violated.

Using AI to write without disclosure is viewed as misleading and rude, akin to sending a sloppily written draft while pretending it was carefully crafted. When readers discover a text is AI-generated, trust in both the content and the author is diminished because the reader no longer knows if the author has actually put the necessary thought into the claims being made.

Recommended AI Use Cases for Writing

While delegating the generation of substantive text is discouraged, AI can be a powerful tool when used to enhance human thinking rather than replace it.

Productive Applications

Brainstorming and Analysis: Using AI to search for information, transcribe audio, or analyze data to inform the writing process.
Copy Editing: Using AI for line editing or tightening prose, provided every edit is deliberately accepted or rejected by a human.
Critique and Workshopping: Asking an AI to critique a draft or provide multiple ways of phrasing a difficult sentence to help the author find the right idiom.
Logistics and Coordination: Drafting short, formulaic emails for corporate coordination where the primary goal is efficiency rather than substantive thought.

Non-Productive Applications

Drafting from Outlines: Turning bullet points into prose, which bypasses the thinking process.
Undisclosed AI Generation: Presenting AI-generated text as one's own, which erodes professional trust.

Community Perspectives and Counterpoints

Discussion among technical professionals suggests a divide between those who view writing as a cognitive exercise and those who view it as a production task.

"I've spent a considerable amount of time... reviewing and improving a work document that was the output of an LLM. Given the number of people involved and the final level of effort, I'm firmly convinced that writing it manually would have been faster and resulted in a higher quality product."

Conversely, some argue that for non-native English speakers or those in high-volume corporate environments, AI is an essential tool for clarity and professionalism:

"I often use AI to help me write emails and nearly every time it provides more concise well structured versions of what I have to say... If these tools are available to you and you don't use them, then I think that would be disrespectful [to the reader's time]."

Others suggest the same logic applies to software engineering, noting that "writing code is thinking" and that AI-generated code often suffers from the same subtle, hard-to-notice errors that plague AI-generated prose.