Anthropic Project Panama and the Destruction of Physical Books for AI Training

AI Companies are Digitizing and Destroying Physical Books for Training Data

AI companies are acquiring millions of secondhand physical books to obtain high-quality training data from the pre-2022 era, which is less likely to be contaminated by AI-generated content. In a process termed "destructive scanning," these books are stripped of their bindings and cut to size for high-speed scanning, after which the physical originals are discarded.

This practice is driven by three primary factors:

Competitive Advantage: By destroying the physical copies, companies prevent competitors from acquiring the same rare materials for their own training sets.
Legal Risk Mitigation: Digital copies stored on private servers are easier to manage than physical archives in the context of copyright disputes.
Cost Efficiency: Destructive scanning is significantly cheaper and faster than non-destructive methods, which require more human labor to correct page distortion.

Anthropic's "Project Panama"

Anthropic's "Project Panama" has emerged as a primary example of this practice. According to details from a $1.5 billion copyright settlement (Anthropic v. Bartz), the company spent tens of millions of dollars to purchase millions of print books. These books were then processed by service providers who removed the bindings and scanned the pages into PDFs with machine-readable text before discarding the paper originals.

While the court ruled that the use of these books to train the Claude LLM was "exceedingly transformative" and constituted fair use under Section 107 of the Copyright Act, the process has drawn intense criticism for its impact on the physical preservation of knowledge.

The Call for Global Scanning Initiatives

Anna's Archive, a large-scale shadow library, is calling for a global volunteer effort to scan and upload rare books, journals, and archives before they are acquired and destroyed by AI firms. The organization argues that this process creates a "monopoly on knowledge," where the only surviving copies of certain texts exist as private data on corporate servers.

Volunteers are encouraged to scan materials from libraries and archives worldwide to ensure that human-authored knowledge remains in the public domain, particularly as AI-generated content begins to dominate the internet, making it harder to distinguish original human thought from synthetic outputs.

Technical and Ethical Debates

The practice of destructive scanning has sparked a polarized debate among technical experts and archivists:

Arguments Against Destruction

Loss of Cultural Heritage: Critics argue that rare books are more than their text; they contain typography, material history, and design that are lost during scanning.
Irreversibility: Once a rare copy is destroyed, it cannot be recovered, effectively making the price of access to that information infinite if the corporate entity decides to restrict it.
Comparison to Historical Loss: Some observers liken the trend to the burning of the Library of Alexandria, where centuries of knowledge were permanently erased.

Arguments in Favor or Neutral

Redundancy of Mass-Market Books: Many argue that most books purchased are mass-market duplicates. If thousands of copies of a book exist, destroying one copy for a digital version does not harm humanity.
Digital Preservation: Some contend that a digital copy is more durable and accessible than a rotting physical book in a used bookstore.
Institutional Safeguards: Others point out that the Library of Congress and the British Library already preserve copies of almost every book published in their respective territories, rendering the "destruction" of a few secondhand copies insignificant.

Proposed Solutions

Legislative Mandates: Suggestions include requiring AI companies to release high-resolution scans of any books no longer under copyright to the public.
Governmental Archiving: Proposals have been made for the government to require companies to provide original scans and OCR results to national archives for safekeeping until copyright expires.
Non-Destructive Requirements: Some suggest a tiered approach where truly rare books are scanned non-destructively, while mass-produced commodities are handled via standard pipelines.