Meta's Revised Commercial Terms Still Allow Broad Scraping, Legal Experts Warn Before October 30 Deadline

Meta removed the word "train" from its upcoming terms of service for business accounts, but legal experts say the revised language still grants the company broad permission to scrape websites associated with those accounts — and still leaves robots.txt compliance as a voluntary promise rather than a contractual right.

The revision, published Friday afternoon, followed a Nieman Lab story on Thursday that reported the new commercial terms would permit broad scraping of websites "associated with" business accounts and possibly allow Meta to use that content to train its AI models. The new terms are slated to take effect on October 30, and business accounts have no option to opt out before then.

What Meta changed in the revised terms

Meta's revision removed the word "train" from the list of allowed use cases and narrowed those uses to "advertising and business products" instead of all Meta products. The company also added language about platform security and integrity.

The revised section now reads, in part:

You authorize us (or any entity acting on our behalf) to use automated software (such as web scrapers, bots, robots, spiders, crawlers, and user agents) and create test credentials to access, index, cache, analyze, collect, retrieve, or crawl the domains, web pages, or URLs (and any content available through them) that are associated with your products or services, or that are made available through your or users' use of Meta Company Products and services, and to periodically pull updates from such domains, web pages, or URLs in order to provide, develop, improve, research, and maintain advertising and business products (including business, commercial and related products to help you create and deliver ads and other commercial content, provide users information about your products and services, and connect you with users who may be interested in your products and services) and maintain the integrity and security of Meta Company Products.

The original version, as archived, read:

You authorize us (or any entity acting on our behalf) to use automated software (such as web scrapers, bots, robots, spiders, crawlers, and user agents) and create test credentials to access, index, cache, analyze, collect, retrieve, or crawl the domains, web pages, or URLs (and any content available through them) that are associated with your products or services, or that are made available through your or users' use of Meta Company Products and services, and to periodically pull updates from such domains, web pages, or URLs. Meta may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products, services, and features provided under these Commercial Terms and any other applicable supplemental terms.

Why legal experts say the revision does not address the core concern

Three legal experts consulted by Nieman Lab were skeptical of the revised terms and Meta's statements about them. Their objections center on three points: the word "develop" still covers AI training, robots.txt is not legally enforceable, and the phrase "associated with your products or services" still permits scraping entire sites.

"Develop" is broad enough to include "train"

Matt Topic, a media and intellectual property attorney at Loevy & Loevy, said the change in wording does not change the substance of the permission:

I think publishers should remain concerned. The revised version of the new terms still allows Meta to use that scraped content to develop any "business product." That doesn't seem to be a defined term, and presumably Meta is developing AI business products. I think "develop" is broad enough to include "train."

Meta currently lists examples of its business products as "ads, Meta Business Tools and Meta Business Suite."

robots.txt is a preference, not a legal right

Benjamin Sobel, a law professor at the University of Wisconsin, explained that robots.txt signals a website's preferences, but compliance with those preferences is entirely voluntary. He compared it to a homeowner putting up a sign reading, "Don't look at my house" — the sign expresses a preference but does not create an enforceable legal right.

You can still express the preference, and unless it's a legally enforceable preference, it can still be ignored all the same.

Meta has not included any specific language about respecting robots.txt preferences in its revised terms of service.

"Associated with your products or services" still covers entire sites

James Grimmelmann, a professor of digital and information law at Cornell Law School, said the added language about permitted uses does not restrict the scope of what can be crawled:

[The update] doesn't address my concerns. While it adds more language about the permitted uses, [the] "develop…advertising and business products" [language] is still broad enough to drive a truck through. This does nothing to restrict the "associated with your products or services" [language] that would let Meta scrape entire sites.

What Meta says about robots.txt and the revision

On Saturday, a Meta spokesperson said the company revised the terms because "it wasn't clear in the initial consolidation of terms that robots.txt preferences are respected. News sites, like all other content owners, still have the ability to disable the crawler via robots.txt flags. That has been the case, and it has not changed."

In a separate statement, the spokesperson said: "We've heard you and want to clarify: the crawling terms we consolidated only applied to our ads and business products, and we've clarified that in the terms themselves. This is not about scraping news sites, and these updates don't give us new rights that supersede those preferences."

The spokesperson also said Meta is incorporating its "Ad Creative Generative AI Terms" into its general commercial terms to make them easier to understand.

How Meta's scraping rights expanded

Meta's current terms already permit scraping of websites associated with business accounts, but only in a very specific scenario. That language appears in Meta's terms of service for when accounts use Meta's generative AI features to run advertisements, and the scraped content can only be used to "optimize your ads."

Adding that language to more general terms of service — and allowing scraped content to be used for all "advertising and business products" — is a substantial expansion of Meta's scraping permissions. Under the revised terms, Meta can deploy automated software such as web scrapers, bots, spiders, crawlers, and user agents; create "test credentials" to access, index, cache, analyze, collect, retrieve, or crawl content; and periodically pull updates from those domains, web pages, and URLs.

Who is affected and what the October 30 deadline means

Virtually every news organization has at least one Meta business account, which allows companies to manage Facebook Pages, Threads accounts, Instagram profiles, and WhatsApp channels from one place. There is no toggle or form for business accounts to reject the new terms before the October 30 effective date. The only way to avoid granting the permission is to remove the commercial presence across Meta's platforms entirely.

Publisher and trade body reactions

At least three journalism organizations publicly raised concerns about the forthcoming terms as a result of Nieman Lab's reporting.

The European Publishers Council announced, prior to Friday's update, that it would formally petition the European Union's telecommunications regulatory authorities and its executive body, the European Commission, to "scrutinize these terms urgently, including their implications for copyright, contractual fairness and abuse of market power."

The United Kingdom News Media Association, the trade organization representing British media, issued a statement on Friday, October 16, calling for government officials to intervene. Chief executive Theo Bamber said:

This shameless landgrab attempt from Meta to scrape anything and everything from news publishers and business websites to train their AI products is the worst kind of arrogant tech platform bullying. We urgently need Ministers and regulators to stand up for British creators and the business community by calling out this outrageous behaviour and preventing Meta from setting this dreadful precedent.

The News/Media Alliance, the largest news publishing trade association in the United States, denounced the terms. President and CEO Danielle Coffey called the update "outrageous and probably illegal" and said:

The company's decision to withhold access to its audience unless businesses allow Meta to scrape content from their websites is an appalling, bad-faith policy, and a clear abuse of the company's market power. It cannot be allowed to be implemented on October 30.

Technical mitigations discussed by practitioners

Commenters on Hacker News argued that site operators should not treat robots.txt as a security control. One commenter wrote: "Never rely on robots.txt. Configure it, but do not rely on it. A bot will do what a bot can do." The same commenter recommended closing the connection at the server level, ideally silently closing the socket and suppressing the first TCP RST from the web daemon and the second from the operating system.

Another commenter reported that user-agent blocking is no longer effective against Meta's crawler: "Blocking by UA is pointless now, meta-externalagent rotates and a chunk comes off non-Meta ASNs. We matched on ASN plus TLS JA4 and that actually stuck."

A separate commenter questioned whether the terms change anything in practice, noting that Meta already scrapes their website on a fairly regular basis even with robots.txt in place, and asking whether the point of the change is to justify more intense scraping.

Key facts at a glance

Meta removed the word "train" from its upcoming business-account terms of service and limited permitted scraping uses to "advertising and business products."
The revised terms still allow broad scraping of domains, web pages, and URLs "associated with" a business's products or services, and still contain no binding language about respecting robots.txt.
Meta says robots.txt preferences are respected and that the update does not grant rights that supersede those preferences.
Legal experts Matt Topic, James Grimmelmann, and Benjamin Sobel all said the revision leaves their concerns unaddressed.
The new terms take effect on October 30, with no opt-out for business accounts before that date.
The European Publishers Council, the UK News Media Association, and the US News/Media Alliance have all publicly objected.