Microsoft Exec Calls AI Scraping ‘Largest Theft of Labor in Human History’ – New Unredacted Filings Reveal

Takeaway

Newly unsealed documents from the New York Times lawsuit reveal that a senior Microsoft executive publicly called the companies’ AI‑training data scraping "the largest theft of labor in human history," and the filings detail massive, paywall‑bypassing scraping that directly reduced the Times’ click‑through rates by up to 93%.

Legal Context – Fair Use vs. Alleged Theft

The lawsuit, filed in 2023, alleges that OpenAI and Microsoft violated copyright law by training large language models (LLMs) on the Times’ articles without permission. Judges have so far leaned toward the AI companies, treating training as a potential fair use exception. Fair use requires that the use does not substitute for the original work or harm its market. The unredacted filings contain internal admissions that directly contradict that defense:

Microsoft’s internal Copilot data showed a 93% drop in click‑throughs to the Times when users received answers from the AI instead of visiting the site.
Executives described the situation as a “doom loop” that hurts both the model’s performance and the web ecosystem.
Both Microsoft and OpenAI leadership called the AI products “existential threats” to publishers, acknowledging they are substitutive rather than merely transformative.

These statements suggest the companies were aware that their models could replace the original content, a key factor against a fair‑use claim.

Scale of the Scraping Operation

The filings quantify the magnitude of the data collection:

Over 91,692 copies of works from the New York Times, Daily News, and Center for Investigative Reporting were found in OpenAI’s mid‑training datasets.
A Common Crawl‑derived dataset contained more than 2 million documents from nytimes.com alone.
Project Mango, a joint Microsoft‑OpenAI initiative, assembled at least 160,903 unique works from the news publishers.
Internal memo author Brent Hecht (Director of Applied Science, Microsoft) called the practice “an astonishing theft of unprecedented proportions” and labeled it “the largest theft of labor in human history.”

Methods Used to Bypass Paywalls

The documents describe deliberate technical work to evade paywalls:

OpenAI researchers discussed a “hack to get around NYT paywall,” to which CEO Greg Brockman replied, “ah nice.”
Scraping was performed via the Bing index and Common Crawl, with subsequent stripping of copyright notices to prevent model outputs from reproducing attribution text.
Training datasets such as WebText and WebText2 were heavily weighted toward scraped news content.

Executive Testimony on Licensing and Responsibility

Both Microsoft and OpenAI executives have publicly acknowledged the need for licensing:

“Anything that is paywalled should be licensed by anyone who wants to use it…for grounding or training,” – Satya Nadella, CEO of Microsoft (deposition).

Nadella also said he would have demanded that OpenAI re‑train its models if he had known they scraped paywalled material.

OpenAI’s head of ChatGPT, Nick Turley, wrote internally that publishers face an “existential threat” and that the models are “largely substitutive.”

Economic Impact on Publishers

The filings provide concrete evidence of revenue loss:

Copilot’s answer engine reduced the Times’ click‑through rate by up to 93% compared to traditional Bing search.
Microsoft’s internal document called the situation “highly unusual” because an end‑product is threatening the economic foundations of its content suppliers.

These figures illustrate a direct market‑substitution effect, undermining the fair‑use argument that the AI does not replace the original work.

Reactions from Legal Counsel and the Industry

Steven Lieberman, counsel for the New York Daily News, stated:

“The evidence revealed here for the first time shows that OpenAI and Microsoft knew that what they were doing was wrong.”

The filing also notes that the U.S. government, via a brief filed earlier this month, sided with OpenAI on the fair‑use defense, highlighting the stark contrast between governmental support and the internal admissions now on record.

Community Perspectives from Hacker News

The Hacker News discussion reflects a range of viewpoints:

Moral outrage: Users liken the scraping to cultural robbery and compare it to historical looting (e.g., @jacquesm, @pianoben).
Technical nuance: Some argue that reading a text is not theft, emphasizing the difference between human learning and an infinitely copyable AI model (e.g., @47282847, @heaney-555).
Economic concerns: Comments note the broader impact on labor and the potential for AI to displace creators (e.g., @tom2026hn, @leonidasrup).
Policy suggestions: Proposals include taxing AI companies, embedding royalties in model outputs, or making models public‑service assets (e.g., @AdamN, @bentt).

These reactions underscore the societal debate over how to balance AI innovation with creators’ rights.

Implications for Future AI Development

The unredacted filings provide the first public glimpse into internal acknowledgments that AI training practices may be illegal under current copyright law. If courts accept these admissions, the fallout could include:

Mandatory licensing for any paywalled content used in model training.
Potential damages based on the demonstrated loss of traffic and revenue to publishers.
Increased regulatory scrutiny of AI data‑collection practices, possibly leading to new legislation.
Industry shifts toward more transparent data pipelines, open‑source datasets, or public‑service model hosting.

Conclusion

The unsealed documents expose a systematic, large‑scale effort by Microsoft and OpenAI to scrape paywalled news content, strip copyright notices, and train LLMs that directly compete with the original publishers. Executives themselves have labeled the practice a “theft of labor,” and internal metrics show severe traffic loss for the Times. These revelations challenge the prevailing fair‑use defense and could reshape the legal and economic landscape for generative AI.